ML19073A249
| ML19073A249 | |
| Person / Time | |
|---|---|
| Issue date: | 03/31/2019 |
| From: | Reed Anzalone, Attard A, Ellen Brown, Timothy Drzewiecki, Jim Gilmer, Syed Haider, Joshua Kaizer, Mathew Panicker Office of Nuclear Reactor Regulation |
| To: | |
| Meyd, Donald | |
| References | |
| NUREG/KM-0013 | |
| Download: ML19073A249 (121) | |
Text
CREDIBILITY ASSESSMENT FRAMEWORK FOR CRITICAL BOILING TRANSITION MODELS A Generic Safety Case to Determine the Credibility of Critical Heat Flux and Critical Power Models Draft Report for Comment
COMMENTS ON DRAFT REPORT Any interested party may submit comments on this report for consideration by the NRC staff.
Comments may be accompanied by additional relevant information or supporting data. Please specify the report number NUREG/KM-0013 in your comments, and send them by the end of the comment period specified in the Federal Register notice announcing the availability of this report.
Addresses: You may submit comments by any one of the following methods. Please include Docket ID NRC-2019-0043 in the subject line of your comments. Comments submitted in writing or in electronic form will be posted on the NRC website and on the Federal rulemaking website http://www.regulations.gov.
Federal Rulemaking Website: Go to http://www.regulations.gov and search for documents filed under Docket ID NRC-2019-0043.
Mail comments to: Office of Administration, Mail Stop: TWFN-7-A60M, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, ATTN: Program Management, Announcements and Editing Staff.
For any questions about the material in this report, please contact: Joshua Kaizer, Reactor Engineer, 301-415-1532 or by e-mail at Joshua.Kaizer@nrc.gov.
Please be aware that any comments that you submit to the NRC will be considered a public record and entered into the Agencywide Documents Access and Management System (ADAMS). Do not provide information you would not want to be publicly available.
3
CREDIBILITY ASSESSMENT FRAMEWORK FOR CRITICAL BOILING TRANSITION MODELS A Generic Safety Case to Determine the Credibility of Critical Heat Flux and Critical Power Models Draft Report for Comment
COMMENTS ON DRAFT REPORT Any interested party may submit comments on this report for consideration by the NRC staff.
Comments may be accompanied by additional relevant information or supporting data. Please specify the report number NUREG/KM-0013 in your comments, and send them by the end of the comment period specified in the Federal Register notice announcing the availability of this report.
Addresses: You may submit comments by any one of the following methods. Please include Docket ID NRC-2019-0043 in the subject line of your comments. Comments submitted in writing or in electronic form will be posted on the NRC website and on the Federal rulemaking website http://www.regulations.gov.
Federal Rulemaking Website: Go to http://www.regulations.gov and search for documents filed under Docket ID NRC-2019-0043.
Mail comments to: Office of Administration, Mail Stop: TWFN-7-A60M, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, ATTN: Program Management, Announcements and Editing Staff.
For any questions about the material in this report, please contact: Joshua Kaizer, Reactor Engineer, 301-415-1532 or by e-mail at Joshua.Kaizer@nrc.gov.
Please be aware that any comments that you submit to the NRC will be considered a public record and entered into the Agencywide Documents Access and Management System (ADAMS). Do not provide information you would not want to be publicly available.
3
NUREG/KM-0013 Credibility Assessment Framework for Critical Boiling Transition Models A generic safety case to determine the credibility of critical heat flux and critical power models Manuscript Completed:
Date Published:
Prepared by:
J.S. Kaizer R. Anzalone E. Brown M. Panicker S. Haider J. Gilmer T. Drzewiecki A. Attard (retired, unable to comment on final version)
U.S. Nuclear Regulatory Commission Office of Nuclear Reactor Regulation
1 ABSTRACT 2 Critical boiling transition (CBT) occurs when a flow regime that has a higher heat transfer rate 3 transitions to a flow regime that has a significantly lower heat transfer rate. Models that predict a 4 CBT are a necessary part of reactor safety analysis because they are used to determine plant 5 safety limits. Therefore, the review of CBT models has been a focus of the U.S. Nuclear 6 Regulatory Commission (NRC) since its inception in 1975.
7 This work presents a generic safety case in the form of a credibility assessment framework that 8 combines aspects of goal structuring notation and maturity assessment. This framework is 9 focused on the credibility assessment of CBT models with specific application to reactor safety 10 analysis. The NRC has performed many such assessments and has generated this framework 11 based on the experience of current and former NRC staff, as well as previous staff reviews as 12 summarized in staff evaluations. This document includes a survey of the important technical and 13 regulatory literature; a detailed technical discussion of CBT models and their application; and a 14 suggested framework for CBT models. This NUREG/KM summarizes the knowledge the NRC 15 staff has developed over the course of 40 years of CBT model and analysis reviews.
iii
1 v
1 TABLE OF CONTENTS 2 ABSTRACT ...............................................................................................................................iii 3 TABLE OF CONTENTS ............................................................................................................ v 4 LIST OF FIGURES ...................................................................................................................vii 5 LIST OF TABLES ......................................................................................................................ix 6 ACKNOWLEDGMENTS ............................................................................................................xi 7 ABBREVIATIONS AND ACRONYMS .....................................................................................xiii 8 1 INTRODUCTION................................................................................................................... 1 9 Why Use the Term Critical Boiling Transition? .................................................................2 10 What Is Credibility? .............................................................................................................2 11 What Is a Credibility Assessment Framework? ..................................................................3 12 Credibility Assessment Framework for Critical Boiling Transition Models ..........................6 13 2 BACKGROUND ON CRITICAL BOILING TRANSITION ...................................................... 7 14 Literature Survey ...............................................................................................................7 15 2.1.1 Technical References ...........................................................................................7 16 2.1.2 Regulatory References .......................................................................................12 17 Critical Boiling Transition Phenomena .............................................................................15 18 2.2.1 Departure from Nucleate Boiling .........................................................................15 19 2.2.2 Dryout .................................................................................................................16 20 2.2.3 Other Flow Regimes and Transitions..................................................................16 21 Determining When Critical Boiling Transition Occurs ......................................................16 22 2.3.1 Critical Heat Flux Models ....................................................................................17 23 2.3.2 Critical Power Models .........................................................................................17 24 2.3.3 Semi-empirical Modeling.....................................................................................17 25 2.3.4 Conservative vs. Non-Conservative Predictions .................................................18 26 Applying a Critical Boiling Transition Model .....................................................................18 27 2.4.1 Applying a Critical Boiling Transition Model in a Pressurized-Water 28 Reactor ..............................................................................................................18 29 2.4.2 Applying a Critical Boiling Transition Model in a Boiling-Water Reactor .............20 30 2.4.3 Applying a Steady-State Model to Transient Conditions .....................................21 31 Addressing Uncertainties and Errors ...............................................................................21 32 3 CREDIBILITY ASSESSMENT FRAMEWORK ....................................................................23 33 G1Experimental Data ...................................................................................................24 34 3.1.1 G1.1Credible Test Facility ...............................................................................24 35 3.1.2 G1.2Accurate Measurements .........................................................................27 36 3.1.3 G1.3Reproduction of Local Conditions............................................................38 37 G2Model Generation ....................................................................................................46 38 3.2.1 G2.1The Mathematical Form ..........................................................................47 39 3.2.2 G2.2Method for Determining Coefficients .......................................................53 40 G3Validation through Error Quantification ...................................................................56 41 3.3.1 G3.1Calculating Validation Error .....................................................................57 42 3.3.2 G3.2Data Distribution in the Application Domain ............................................59 43 3.3.3 G3.3Inconsistency in the Validation Error .......................................................68 44 3.3.4 G3.4Calculating Model Uncertainty .................................................................73 45 3.3.5 G3.5Model Implementation .............................................................................77 46 4
SUMMARY
AND CONCLUSION .........................................................................................83 47 5 REFERENCES ....................................................................................................................85 48 APPENDIX A LISTING OF ALL GOALS........................................................................... A-1 vi v
1 2
vii
1 LIST OF FIGURES 2 Figure 1 Goals ............................................................................................................................. 4 3 Figure 2 Framework .................................................................................................................... 5 4 Figure 3 Decomposition of G Main Goal ............................................................................... 23 5 Figure 4 Decomposition of G1Experimental Data.................................................................. 24 6 Figure 5 Decomposition of G1.1Credible Test Facility ........................................................... 25 7 Figure 6 Decomposition of G1.2Accurate Measurements ..................................................... 28 8 Figure 7 Decomposition of G1.3Reproduction of Local Conditions ....................................... 38 9 Figure 8 Decomposition of G2Model Generation................................................................... 47 10 Figure 9 Decomposition of G2.1The Mathematical Form ...................................................... 47 11 Figure 10 Decomposition of G2.2Method for Determining Coefficients ................................... 53 12 Figure 11 Decomposition of G3Validation through Error Quantification .................................. 57 13 Figure 12 Regions in the Application Domain ............................................................................. 60 14 Figure 13 Decomposition of G3.2Data Distribution in the Application Domain ........................ 62 15 Figure 14 Decomposition of G3.3Inconsistencies in the Validation Error ................................ 69 16 Figure 15 Decomposition of G3.4Quantification of the Models Error ...................................... 75 17 Figure 16 Decomposition of G3.5Model Implementation ......................................................... 78 18 viii vii
1 ix
1 LIST OF TABLES 2 Table 1 Key Textbooks for the Review of CBT Models ............................................................... 7 3 Table 2 Key Papers for the Review of CBT Models .................................................................... 8 4 Table 3 Industry Reports Associated with CBT Models for PWRs .............................................. 9 5 Table 4 Industry Reports Associated with CBT Models for BWRs ............................................ 11 6 Table 5 Regulatory References Associated with CBT Models .................................................. 13 7 Table 6 Evidence for G1.1.1Test Facility Description ............................................................ 25 8 Table 7 Evidence for G1.1.2Test Facility Comparison ........................................................... 27 9 Table 8 Experimental Parameters Measured or Controlled ...................................................... 28 10 Table 9 Evidence for G1.2.1Test Facility QA Program .......................................................... 30 11 Table 10 Evidence for G1.2.2Statistical Design of Experiment ............................................... 31 12 Table 11 Evidence for G1.2.3Data Fidelity .............................................................................. 34 13 Table 12 Evidence for G1.2.4Instrumentation Uncertainty Impact .......................................... 35 14 Table 13 Evidence for G1.2.5Repeated Test Points................................................................ 36 15 Table 14 Evidence for G1.2.6Quantified Heat Losses ............................................................. 37 16 Table 15 Evidence for G1.3.1Equivalent Geometric Dimensions ............................................ 39 17 Table 16 Evidence for G1.3.2Prototypical Grid Spacers ......................................................... 41 18 Table 17 Evidence for G1.3.3Axial Power Shapes .................................................................. 43 19 Table 18 Evidence for G1.3.4Radial Power Peaking (PWR) ................................................... 44 20 Table 19 Evidence for G1.3.4Radial Power Peaking (BWR) ................................................... 45 21 Table 20 Evidence for G1.3.5Differences in the Test Assembly.............................................. 46 22 Table 21 Evidence for G2.1.1Necessary Parameters ............................................................. 51 23 Table 22 Evidence for G2.1.2Reasoning for the Mathematical Form ...................................... 52 24 Table 23 Evidence for G2.2.1Identification of Training Data ................................................... 54 25 Table 24 Evidence for G2.2.2Calculation of the Models Coefficients ..................................... 55 26 Table 25 Evidence for G2.2.3Calculation of Model-Specific Factors and Constants............... 56 27 Table 26 Evidence for G3.1Calculating Validation Error.......................................................... 59 28 Table 27 Evidence for G3.2.1Identification of Validation Data ................................................. 63 29 Table 28 Evidence for G3.2.2Defining the Application Domain ............................................... 64 30 Table 29 Evidence for G3.2.3Understanding the Expected Domain ....................................... 65 31 Table 30 Evidence for G3.2.4Validation Error Data Density in the Expected Domain ............. 66 32 Table 31 Evidence for G3.2.5Sparse Regions......................................................................... 67 33 Table 32 Evidence for G3.2.6Restricted to the Application Domain ........................................ 68 34 Table 33 Evidence for G3.3.1Identifying Non-poolable Data Sets........................................... 71 35 Table 34 Evidence for G3.3.2Identifying Non-conservative Subregions.................................. 72 36 Table 35 Evidence for G3.3.3Appropriate Trends ................................................................... 73 37 Table 36 Evidence for G3.4.1Error Database .......................................................................... 75 38 Table 37 Evidence for G3.4.2Validation Error Statistics .......................................................... 76 39 Table 38 Evidence for G3.4.3Model Uncertainty Bias ............................................................. 77 40 Table 39 Evidence for G3.5.1Same Computer Code .............................................................. 78 41 Table 40 Evidence for G3.5.2Same Evaluation Methodology ................................................. 79 42 Table 41 Evidence for G3.5.3Transient Prediction .................................................................. 80 43 x
ix
1 x
1 ACKNOWLEDGMENTS 2 Frameworks, such as presented in this paper, are the result of tremendous effort by numerous 3 individuals. While these individuals and their technical contributions are too numerous to list, the 4 authors offer special thanks to Robert Weisman and Julie Ezell for their legal review and advice 5 which resulted in significant improvement to the document.
xi xii
1 x
1 ABBREVIATIONS AND ACRONYMS 2 1-D one-dimensional 3 2-D two-dimensional 4 3-D three-dimensional 5 AOO anticipated operational occurrence 6 ASME American Society of Mechanical Engineers 7 BWR boiling-water reactor 8 CBT critical boiling transition 9 CFR Code of Federal Regulations 10 CHF critical heat flux 11 CP critical power 12 DNB departure from nucleate boiling 13 DNBR departure from nucleate boiling ratio 14 G Goal 15 GSN goal structuring notation 16 LOCA loss-of-coolant accident 17 M&S modeling and simulation 18 MDNBR minimum departure from nucleate boiling ratio 19 NRC U.S. Nuclear Regulatory Commission 20 PCT peak cladding temperature 21 PWR pressurized-water reactor 22 R- or K-factor relative power factor 23 SAFDL specified acceptable fuel design limit 24 SLMCPR safety limit minimum critical power ratio 25 SRP Standard Review Plan 26 SSC systems, structures, and components 27 V&V verification and validation xiii xiv
1 1 INTRODUCTION 2 Critical boiling transition 1 (CBT) is defined as a transition from a boiling flow regime that has a 3 higher heat transfer rate to a flow regime that has a significantly lower heat transfer rate. For 4 scenarios in which the heat transfer is controlled by the heat flux (such as in nuclear fuel 5 assembly), the reduction in heat transfer rate caused by the CBT results in an increase in the 6 surface temperature in order to maintain the heat flux. If the reduction in the heat transfer rate and 7 resulting increase in surface temperature is large enough, the surface may weaken or melt. In a 8 nuclear power plant, this cladding softening or melting is considered fuel damage.
9 To ensure that the fuel is not damaged during normal operation or anticipated operational 10 occurrences (AOOs), computer simulations of the fuel are performed to predict the 11 thermal-hydraulic conditions that would occur in the fuel assemblies during various scenarios. The 12 resulting thermal-hydraulic conditions are then input to a CBT model. 2 That CBT model predicts 13 the power which is required for a CBT to occur at the given thermal-hydraulic conditions. Hence 14 the margin to CBT can be obtained by comparing the current power at the specific location in the 15 fuel assembly to the power at which CBT occurs at the same thermal-hydraulic conditions. The 16 U.S. Nuclear Regulatory Commission (NRC) has historically accepted that one way to 17 demonstrate the avoidance of fuel damage during all normal operation and AOOs is to 18 demonstrate that there is margin to a CBT.
19 Because of the importance of CBT models, a major focus in reactor safety analysis is to 20 determine whether the proposed models can correctly predict CBT. The NRC has reviewed many 21 CBT models over the years and has documented why each model was found acceptable (i.e.,
22 able to correctly predict CBT) in the corresponding safety evaluation. The authors of this 23 document have used those safety evaluations along with their own expertise to produce a 24 framework for assessing the credibility of CBT models.
25 This document includes two main sections. The first section contains a brief background of 26 literature relevant to the assessment of CBT models followed by a discussion of the CBT 27 phenomena and how such phenomena are commonly modeled. The second section describes 28 the development of the credibility assessment framework for CBT models and provides detailed 29 aspects of that framework as well as the evidence 3 commonly used to demonstrate that the 30 criteria in the framework have been satisfied. In total, this document is meant to act as a textbook 31 for those interested in the assessment of CBT models.
1 Many terms have been used to describe these models, including critical heat flux, critical power, critical quality versus boiling length, departure from nucleate boiling, dryout, burnout, and flow boiling crisis.
2 Historically, the models are commonly referred to as correlations because they correlate the CBT phenomenon to other variables in the flow field. However, the term correlation has a very specific meaning in statistics; therefore, this document will refer to them as models.
3 Evidence as used throughout this document is not intended to mean the rules and legal principles that govern the proof of facts in a legal proceeding. Rather, as used in this document, evidence is the available body of facts or information indicating whether a belief or proposition is true or valid.
1
1 Why Use the Term Critical Boiling Transition?
2 Hewitt and Hall-Taylor (1970) discussed a wide range of terms used to describe the phenomenon 3 associated with dryout and critical heat flux (CHF). They noted that the large diversity of terms 4 tends to be confusing and this diversity reflects a continuing search for a term which is both 5 descriptive and scientifically accurate. They analyzed the most common terms used (burnout, 6 departure from nucleate boiling (DNB), dryout, and CHF); recognized that each term had its own 7 inadequacies and merits; and chose burnout as the least unsatisfactory term. Unfortunately, the 8 current literature on the subject does not reflect their choice, which seems to have settled mostly 9 on the term critical heat flux, although dryout and DNB are still commonly used.
10 Although CHF is technically independent of any specific phenomena, it is very closely tied to the 11 phenomena of DNB, which occurs when nucleate boiling becomes inadequate to transfer the heat 12 at the fuel surface to the coolant. At that point, the boiling regime begins to depart from nucleate 13 boiling and begins transition boiling, which is the boiling regime between nucleate boiling and 14 film boiling. The close association between CHF and the phenomenon of DNB is likely due to the 15 fact that CHF is the quantity used to determine whether DNB will occur in a pressurized-water 16 reactor (PWR). However, CHF is typically not the quantity used to determine whether dryout (i.e.,
17 the drying out of the thin annular film in contact with the fuel cladding) has occurred in a 18 boiling-water reactor (BWR). Additionally, the heat flux that causes a phenomenon to occur (i.e.,
19 the CHF) is different from the phenomenon itself. In technical discussions, the authors found it 20 necessary to separate the phenomenon from any quantity associated with it.
21 Even considering all of these arguments, the authors of this document, like Hewitt and Hall-Taylor, 22 were hesitant to introduce new terminology and initially decided to use the common term critical 23 heat flux. However, as the discussion became more detailed and finer distinctions were 24 necessary, the authors reluctantly decided that a different term was necessary and could not be 25 avoided. Therefore, the authors chose to use the term Critical boiling transition, because it better 26 describes the pertinent phenomena and allows for the necessary distinctions. Because CBT is a 27 new term, we repeat its definition here: CBT 4 is defined as a transition from a boiling flow regime 28 that has a higher heat transfer rate to a flow regime that has a significantly lower heat transfer 29 rate.
30 What Is Credibility?
31 The term credibility has seen wide application in the modeling and simulation (M&S) community, 32 specifically in the areas focusing on Verification and Validation (V&V). However, the term is often 33 left undefined. The American Society of Mechanical Engineerings (ASME) V&V 10 Guide for 34 Verification and Validation in Computational Solid Mechanics (2006) did not formally define the 35 term, but did equate it to trustworthiness. Initially, NASA (2008) discussed the term, but 36 purposefully chose not to define it and instead relied on the usual sense of the English language.
37 Later, they defined the term as the quality to elicit belief or trust in modeling and simulation 38 results (NASA 2008B). Oberkampf and Roy (2010) do provide a definition for credibility of 39 computational results - results of an analysis that are worthy of belief or confidence, but this 40 definition is not much more detailed than ASMEs connection between credibility and 41 trustworthiness. While credibility is intimately linked with trust, the component which is missing 42 from these definitions is how much trust is needed in the specific use of the model. Therefore, the 43 authors of this work have chosen to use a definition based on the work of Kaizer et al., (2015) 4 While CBTs can exist on other surfaces, this work is concerned only with fuel rods used in light water nuclear power plants.
2
1 which captures the underlying link to trustworthiness, but maintains awareness of the necessity 2 to make a decision.
3 Credibility is defined as the determination that an object (in this particular instance, a model) can 4 be trusted for its intended purpose. As defined, this is a binary determination. Thus, an object is 5 either deemed credible (i.e., can be trusted for its intended purpose) or not credible (i.e., cannot 6 be trusted for its intended purpose). There are two interesting consequences from this definition of 7 credibility. First, there is no middle ground, all objects must either be credible or not credible.
8 Second, there is no degree of credibility. That is, by definition one object cannot be more 9 credible than another. The authors fully acknowledge that some objects may certainly be more 10 trusted than other objects. For example, one individual may have more experience and therefore 11 be more trusted than another individual, or one simulation may be very well vetted and therefore 12 be more trusted than another simulation. However, the credibility of those objects is defined to be 13 binary (i.e., credible, not credible) because decisions themselves are binary (i.e., yes or no).
14 What Is a Credibility Assessment Framework?
15 A credibility assessment framework provides a means to assess whether an object can be trusted 16 for its intended purpose. Such a framework can be thought of as one form of a safety case. A 17 safety case is defined as a structured argument, supported by a body of evidence that provides a 18 compelling, comprehensible, and valid case that a system is safe for a given application in a given 19 operating environment. 5 Although various ways exist to provide a safety case (e.g., every safety 20 evaluation produced by the NRC can be thought of as the documentation of a safety case or 21 collection of safety cases), this document makes use of concepts formalized in goal structuring 22 notation (GSN). GSN (GSN Working Group, 2011) is a graphical argumentation notation that 23 can be used to document explicitly the individual elements of any argument (claims, evidence, 24 and contextual information) and, perhaps more significantly, the relationships that exist between 25 these elements (i.e., how claims are supported by other claims, and ultimately by evidence, and 26 the context that is defined for the argument). See Denney et al. (2011) for an example of GSN.
27 The framework presented here combines the logic structure of GSN with the evaluation aspects 28 of maturity assessment. Maturity assessment (Kaizer et al., 2015) is focused on measuring how 29 mature an object is in specific attributes compared to its possible minimum and maximum 30 amount of maturity in those attributes. Maturity assessment frameworks, such as the Predictive 31 Capability Maturity Model (Oberkampf et al., 2007) and NASA-STD-7009 (NASA 2008B), focus 32 on the evidence that is available and is a means to rank that evidence in a manner useful to a 33 decision maker. For a more detailed description of a maturity assessment and its history, see 34 Oberkampf and Roy (2010).
35 The credibility assessment framework used in this document is unique in that it combines these 36 two concepts by using the logical structure of goals from GSN and the evaluation of the possible 37 evidence from maturity assessment. The framework is generated from a single main goal. That 38 main goal is then logically decomposed into subgoals. By logical decomposition, we mean the 39 act of generating a set of sub-goals which are logically equivalent to the original goal (i.e.,
5 This document uses the definition provided by the United Kingdoms Ministry of Defense (2007). Other U.S. government agencies which have made use of this concept include NASA (2015) and the FDA (2014). The authors use of the UK Ministry of Defense definition in this document does not imply USNRC approval of regulatory principles or approaches employed in the UK, nor should the use of the definition be understood to be an NRC endorsement of such principles or approaches as acceptable for use in the US.
3
1 necessary and sufficient for the original goal to be met). This decomposition is expressed using 2 GSN notation. Each subgoal can either be further logically decomposed into other subgoals, or 3 if no further decomposition is deemed useful, the subgoal may be considered a base goal and 4 evidence must be provided to demonstrate that the base goal is true. The evidence which is 5 commonly provided is given in a maturity table, where it is ranked from least to most mature. A 6 simple example to illustrate the logic is given below.
7 The main goal (G) is written as a conclusion, such as G - It is safe to drive over the bridge.
8 Notice that this goal is somewhat ambiguous. What is meant by safe? While there is common 9 agreement that it should be safe to drive over a bridge, there is disagreement as to what safe 10 means in this instance. Such ambiguity is often encountered, but frameworks such as the one 11 provided in this document can be used to define what these ambiguous terms (such as safe) 12 mean in practice.
13 The main goal, G, is then logically decomposed into a set of sub-goals, where each sub-goal must 14 be necessary (i.e., if the sub-goal is false, the main goal must also be false) and the set of sub-15 goals must be sufficient (i.e., if the set of sub-goals is true, the main goal must also be true) to 16 demonstrate that the main goal is true. This simple example has two subgoals: (1) The bridge 17 can withstand the weight of my car. and (2) There will not be a natural disaster while I am driving 18 over the bridge. These goals are given in Figure 1 below.
19 20 Figure 1 Goals 21 Each subgoal (e.g., G1 and G2) must either be further decomposed into additional sub-goals, or 22 evidence provided to determine if those sub-goals could be considered true. For this example, no 23 further decomposition was considered. Potential levels of evidence that could be provided to 24 demonstrate that each subgoal is true (i.e., has been met) are given in Figure 2 below.
4
1 2 Figure 2 Framework 3 The evidence provided is the justification for concluding that the specific base goal is true (i.e., has 4 been met). This evidence is ranked from least to most mature, or providing the least certain 5 justification that the base goal is met to the most certain justification. With higher levels of 6 evidence (e.g., level 3 as opposed to level 1), we can be more certain that the associated base 7 goal is true. Thus, an individual driving over a bridge on his or her daily commute would likely 8 require a very low level of evidence to determine the bridge is credible (i.e., safe to drive across).
9 In all likelihood, the individual may not even consciously think about the credibility of the bridge, or 10 if he or she did, the individual would likely rely on low levels of evidence. However, if the bridge 11 were used to transport heavy haul freight (i.e., oversized loads), a much higher level of evidence 12 would likely be required before the bridge was deemed credible.
13 The specific pieces of evidence which are considered by this framework are given in Figure 2. If 14 any other evidence (i.e., levels of G1 or G2) are used to demonstrate that the associated goal is 15 true, that evidence should be placed in its appropriate rank in the table. Thus, one could argue 16 that seeing another car drive over the bridge right before they do is evidence that the bridge can 17 withstand the weight of their own car. If this evidence is going to be used, it should be ranked 18 according to the other evidence already in the table (likely falling between levels 2 and 3 and 19 requiring a re-numbering of the table).
20 Notice that the ambiguity of the word safe in the main goal G has now been removed. That is, by 21 saying It is safe to drive over the bridge, we have not only defined safe as meaning G1 and G2 22 are true, but we would also state what evidence was given (e.g., Level 3 for G1 and Level 2 for 23 G2). Thus, the ambiguous word safe is explicitly defined using the framework.
24 Additionally, anything not specified in the framework was not considered in determining credibility.
25 Because the framework explicitly establishes the assumptions underlying an assessment, it can 26 be helpful when identifying any areas that may need further consideration (that is, additional sub-5
1 goals or evidence levels). For example, an individual could argue that our sample framework lacks 2 a sub-goal that accounts for the driving ability of other drivers on the bridge. Another may argue 3 that our first sub-goal should not only consider the weight of our car, but all other vehicles on the 4 bridge at the same time. One of the largest advantages to these frameworks is that others can 5 quickly and easily determine what was and what was not considered. Further, the framework 6 could be updated quickly and easily to account for any omissions.
7 Credibility Assessment Framework for Critical Boiling Transition Models 8 The credibility assessment framework presented in this work is focused on critical boiling 9 transition models. While this framework was generated based on the NRC staffs experience 10 reviewing these models, the framework itself is more broadly applicable to any use of any CBT 11 model. This includes the entire spectrum of possible uses from something as simple as a 12 homework problem to something as significant as reactor safety analysis, and all uses in between.
13 It is important to remember that the appropriate evidence level will change based on the models 14 intended use. Thus, the level of evidence appropriate for reactor safety analysis will likely be 15 much higher than that which is appropriate for a homework problem.
16 As this framework is applicable to any use of a CBT model (including, but not limited to, reactor 17 safety analysis), the authors have chosen to use a broader terminology when describing the 18 details of the framework as it can be applied to determining credibility. The process of determining 19 credibility involves two distinct roles: the analyst and the assessor. 6 It is the role of the analyst to 20 generate the model, gather the evidence, and present the argument that the model can be 21 trusted. It is the role of the assessor to determine if the evidence presented is sufficient to justify 22 that the model can be trusted for its intended purpose. In regulatory environments, these roles are 23 usually filled by separate individuals from different organizations, the analyst being the applicant 24 and the assessor being the regulatory agency staff member (e.g., at the NRC, this role is typically 25 called a reviewer). However, in other environments both roles could be performed by individuals 26 from the same organization (i.e., internal peer review), and in some cases could be performed by 27 the same individual (e.g., a homework problem).
6 The assessor is not a reference to a specific role as defined by other national or international organizations. Instead, the word was chosen solely based on the fact that the person who applies the credibility assessment framework is making an assessment, and is therefore an assessor.
6
1 2 BACKGROUND ON CRITICAL BOILING TRANSITION 2 Literature Survey 3 This section provides a literature survey of the references considered important for the NRC 4 review of CBT models. Many references associated with CBT phenomena exist; however, the 5 following are of special interest because they are commonly cited in discussions of the models 6 used in nuclear power reactors. For convenience, the references have been separated into 7 technical references (i.e., textbooks, articles, and industry reports) and regulatory references.
8 2.1.1 Technical References 9 Tables 1, 2, 3, and 4 list the key technical references for CBT models.
10 Table 1 Key Textbooks for the Review of CBT Models Author Title Date Hewitt and Annular Two-Phase Flow 1970 Hall-Taylor Tong Boiling Crisis and Critical Heat Flux 1972 Todreas Nuclear Systems I: Thermal Hydraulic Fundamentals 1990 and Kazimi Lahey and The Thermal Hydraulics of a Boiling Water Nuclear 1993 Moody Reactor Tong and Boiling Heat Transfer and Two-Phase Flow 1997 Tang 11 7
1 Table 2 Key Papers for the Review of CBT Models Author Title Date 1756 Leidenfrost On the Fixation of Water in Diverse Fire (1966)
Tong et al. Influence of Axially Nonuniform Heat Flux on DNB 1965 1965-Macbeth An Appraisal of Forced Convection Burnout Data 1966 A Correlation of Burnout Data for Uniformly Heated Barnett Annuli and Its Uses for Predicting Burnout in 1966 Uniformly Heated Rod Bundles Healzer et Design Basis for Critical Heat Flux Condition in 1966 al. Boiling Water Reactors Prediction of Departure from Nucleate Boiling for an Tong 1967 Axially Non-Uniform Heat Flux Distribution Studies on Burnout: Part 3A New Correlation for Biasi et al. Round Ducts and Uniform Heating and Its 1967 Comparison with World Data Gellerstedt Correlation of Critical Heat Flux in a Bundle Cooled 1969 et al. by Pressurized Water A Correlation of Rod Bundle Critical Heat Flux for Hughes 1970 Water in the Pressure Range 150 to 725 psia Statistical Concepts and Techniques for Developing, Piepel and Evaluating, and Validating CHF Models and 1993 Cuta Corresponding Fuel Design Limits Groeneveld The 2006 CHF Look-Up Table 2007 Uniform versus Nonuniform Axial Power Distribution Yang et al. 2014 in Rod Bundle CHF Experiments Identification of Nonconservative Subregions in Kaizer Empirical Models Demonstrated Using Critical Heat 2015 Flux Models CHF Data Used to Generate 2006 Groeneveld CHF Groeneveld 2016 Lookup Tables 2
8
1 Table 3 Industry Reports Associated with CBT Models for PWRs CBT Model Title Date Correlation of Critical Heat Flux in a Bundle Cooled B&W-2 1970 by Pressurized Water C-E [Combustion Engineering] Critical Heat Flux:
Critical Heat Flux Correlation for C-E Fuel CE-1 1976 Assemblies with Standard Spacer Grids, Part 1 Uniform Axial Power Distribution Exxon Nuclear DNB Correlation for PWR Fuel XNB DNB 1983 Designs New Westinghouse Correlation WRB-1 for WRB-1 Predicting Critical Heat Flux in Rod Bundles with 1984 Mixing Vane Grids WRB-2 VANTAGE 5H Fuel Assembly 1985 C-E Critical Heat Flux: Critical Heat Flux Correlation CE-1 for C-E Fuel Assemblies with Standard Spacer 1984 (modified)
Grids, Part 2Non-Uniform Axial Power Distribution Departure from Nucleate Boiling Correlation for High ANFP DNB 1990 Thermal Performance Fuel BWU The BWU Critical Heat Flux Correlations 1996 Modified WRB-2 Correlation, WRB-2M, for WRB-2M Predicting Critical Heat Flux in 17x17 Rod Bundles 1999 with Modified LPD Mixing Vane Grids The BWU Critical Heat FIux Correlations BWU Applications to the Mark-B11 and Mark-BW17 2000 Addendum 1 MSM Designs Application of BWU-Z CHF Correlation to the BWU Mark-BW 17 Fuel Design with Mid-Span Mixing 2002 Addendum 2 Grids Addendum 1 to WCAP-1 4565-P-A Qualification of ABB-NV and ABB Critical Heat Flux Correlations with VIPRE-01 2004 ABB-TV Code Departure from Nucleate Boiling Correlation for High HTP 2005 Thermal Performance Fuel BHTP BHTP DNB Correlation Applied with LYNXT 2005 BWU The BWU-B11R CHF Correlation for the Mark-B11 2005 Addendum 3 Spacer Grid Westinghouse Correlations WSSV and WSSV-T for WSSV and Predicting Critical Heat Flux in Rod Bundles with 2007 WSSV-T Side Supported Mixing Vanes ACH-2 The ACH-2 CHF Correlation for the U.S. EPR 2007 9
CBT Model Title Date Addendum 2 to WCAP-14565-P-A Extended ABB-NV Application of ABB-NV Correlation and Modified (extended) 2008 ABB-NV Correlation WLOP for PWR Low Pressure and WLOP Applications Westinghouse Next Generation Correlation WNG-1 (WNG-1) for Predicting Critical Heat Flux in Rod 2010 Bundles with Split Vane Mixing Grids WRB-1 and Thermal Design Methodology 2013 WRB-2 KCE-1 Critical Heat Flux Correlation for PLUS7 KCE-1 2012 Thermal Design The ORFEO-GAIA and ORFEO-NMGRID Critical ORFEO 2016 Heat Flux Correlations 1
10
1 Table 4 Industry Reports Associated with CBT Models for BWRs CBT Model Title Date Loss-of-Coolant Accident and Emergency Core GE transient Cooling Models for General Electric Boiling 1971 CHF Water Reactors General Electric Thermal Analysis Basis Data, GEXL 1977 Correlation and Design Application ANFB ANFB Critical Power Correlation 1990 R-Factor Calculation Method for GE11, GE12, R-Factors 1999 and GE13 Fuel 10x10 SVEA Fuel Critical Power Experiments D2 1999 and CPR Correlations: SVEA-96+
10x10 SVEA Fuel Critical Power Experiments D1 2000 and CPR Correlations: SVEA-96 GEXL96 GEXL96 Correlation for ATRIUM-9B Fuel 2001 GEXL10 GEXL10 Correlation for GE12 Fuel 2001 GEXL80 GEXL80 Correlation for SVEA96+ Fuel 2004 10x10 SVEA Fuel Critical Power Experiments D4 2005 and CPR Correlation: SVEA-96 Optima2 GEXL97 Correlation Applicable to ATRIUM-10 GEXL97 2008 Fuel D4 (Modified SVEA-96 Optima2 CPR Correlation (D4):
2009 R-Factor) Modified R-factors for Part-Length Rods D4 (High and SVEA-96 Optima2 CPR Correlation (D4): High 2009 Low Flow) and Low Flow Applications GEXL17 GEXL17 Correlation for GNF2 Fuel 2009 SPCB SPCB Critical Power Correlation 2009 GEXL14 GEXL14 Correlation for GE 14 Fuel 2011 ACE/ATRIUM-10 ACE/ATRIUM-10 Critical Power Correlation 2014 ACE/ATRIUM-10 ACE/ATRIUM 10XM Critical Power Correlation 2014 XM 10x10 SVEA Fuel Critical Power Experiments D5 and New CPR Correlation: D5 for SVEA-96 2013 Optima3 ACE/ATRIUM-11 ACE/ATRIUM-11 Critical Power Correlation 2015 2
11
1 2.1.2 Regulatory References 2 The regulatory references are separated into the following types:
3
- Regulations. The Code of Federal Regulations (CFR) sets forth regulations that licensees 4 must satisfy.
5
- Guidance. Following NRC guidance is one way to satisfy the corresponding regulations.
6 Such guidance can be found in NRC Regulatory Guides and NRC publications in specified 7 NUREGS. In addition, the application regulations require an applicant to identify and 8 describe all differences in design features, analytical techniques, and procedural 9 measures proposed for a facility compared to those in NUREG-0800, Standard Review 10 Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition 11 (SRP). Previous safety evaluations can also inform the staffs review of an application.
12
- Generic Communications. The NRC may choose to send out a general communication on 13 an issue for numerous reasons. Generic communications include administrative letters, 14 bulletins, circulars, generic letters, information assessment team advisories, information 15 notices, regulatory issue summaries, security advisories, and documents for comment.
16 Table 5 lists the regulatory references associated with CBT models in reactor safety analyses.
12
1 Table 5 Regulatory References Associated with CBT Models Type Title Date 10 CFR Part 50, Domestic Licensing of Production and Utilization Facilities, Appendix A, Regulations General Design Criteria for Nuclear Power N/A Plants, General Design Criterion 10, Reactor Design Regulations 10 CFR 50.36, Technical specifications N/A 10 CFR 50.34, Contents of Applications; Regulations N/A Technical Information 10 CFR Part 50, Appendix B, Quality Assurance Regulations Criteria for Nuclear Power Plants and Fuel N/A Reprocessing Plants Guidance SRP Section 4.2, Fuel System Design 2007 Guidance SRP Section 4.4, Thermal and Hydraulic Design 2007 Information Notice 2014-01, Fuel Safety Limit Generic Calculation Inputs Were Inconsistent with 2014 Communication NRC-Approved Correlation Limit Values NQA Quality Assurance Requirements for Standard 2015 Nuclear Facility Applications 2 10 CFR Part 50, Appendix A, General Design Criterion 10 3 General Design Criterion (GDC) 10 in 10 CFR Part 50, Appendix A, is the principal regulation 4 associated with a CBT. This criterion introduces the concept of specified acceptable fuel design 5 limits (SAFDLs). In essence, SAFDLs are those limits placed on certain variables to ensure that 6 the fuel does not fail. One such SAFDL is associated with CBT. Because the decrease in heat 7 transfer following a CBT could result in fuel failure, a SAFDL is used to demonstrate that a CBT 8 does not occur during normal operation and AOOs. Therefore, fuel failure is precluded during 9 normal operation and AOOs.7 10 SRP Section 4.4 includes the following two SAFDLs for use in accounting for the uncertainties 11 involved in developing and using a CBT model (e.g., uncertainties in the values of process 12 parameters, core design parameters, calculation methods, instrumentation) and ensuring that 13 fuel failure is precluded:
7 Experiencing such a transition may not immediately result in fuel failure. The decrease in heat transfer and subsequent increase in fuel temperature may not be enough to cause the cladding to weaken or melt.
Therefore, the point of CBT is considered to be a conservative limit compared to the actual point of fuel damage.
13
1 (1) There should be a 95-percent probability at the 95-percent confidence level that the hot 2 fuel rod in the core does not experience a CBT during normal operation or AOOs.
3 4 (2) At least 99.9 percent of the fuel rods in the core will not experience a CBT during normal 5 operation or AOOs.
6 Typically, SAFDL No. 1 is associated with PWRs, and SAFDL No. 2 is associated with BWRs.
7 8 Before May 21, 1971, when the GDC took effect, the Atomic Energy Commission (AEC), the 9 predecessor to the NRC, approved construction permits for nuclear power plants based on plant-10 specific Principal Design Criteria (PDC) that applicants proposed in their construction permit 11 applications as required by the then-extant provisions of 10 CFR 50.34(a). The AEC published 12 proposed General Design Criteria in the Federal Register (32 FR 10213) on July 11, 1967, 13 sometimes referred to as the AEC Draft GDC, which were generally consistent with the PDC 14 previously proposed in applications for construction permits. AEC Draft GDC 6 is the relevant draft 15 GDC and is substantially similar to the current GDC 10. AEC Draft GDC 6 also calls for the 16 reactor core to be designed with appropriate margin to specified limits that preclude fuel damage.
17 10 CFR 50.36 18 The second regulation associated with a CBT is 10 CFR 50.36, part of which focuses on defining 19 technical specification safety limits. There are multiple limits that are associated with CBT models 20 used during plant operation. These limits can be operating limits, alarms, analysis limits, and 21 safety limits. Generally, only the safety limit and associated limiting conditions for operation 22 (LCOs) and surveillance requirements (SRs) are included in the plants technical specifications.
23 The safety limit associated with CBT is typically focused on an accurate quantification of the 24 uncertainty of the CBT model and may also include the quantification of additional uncertainties as 25 well.
26 10 CFR 50.34 27 The third regulation associated with a CBT is in 10 CFR 50.34, which focuses on defining the 28 information that a licensee must present to ensure safe operation. Specifically, 29 10 CFR 50.34(a)(4) requires that the Preliminary Safety Analysis Report (PSAR) include 30 determination of the margins of safety during normal operation and AOOs. One of these is the 31 margin to CBT, which verifies that fuel failure is precluded during normal operation and AOOs 32 through analysis.
33 10 CFR Part 50, Appendix B 34 The fourth regulation associated with a CBT appears in 10 CFR Part 50, Appendix B. It requires 35 licensees to include certain structures, systems, and components (SSCs) in a quality assurance 36 program that satisfies specific criteria. Appendix B, Criterion III, requires that specified design 37 control measure be applied to the design of safety-related SSCs and these measures apply to 38 safety analyses for these SSCs. The CBT model is a key component of the safety analysis subject 39 to 10 CFR Part 50, Appendix B.
40 Other Regulations 41 Both 10 CFR 50.46 and 10 CFR Part 50, Appendix K, Section I.C.4 focus on the modeling of a 42 nuclear power plant during accident scenarios. While many of these scenarios involve the use of 43 CBT models, a different model may be used than is used to analyze SSC performance during 14
1 AOOs. For example, the CBT models used during LOCAs are typically low pressure, conservative 2 models, which are not necessarily fuel design specific. While these models are reviewed by the 3 NRC as a part of any accident evaluation model, they typically are not a major focus during those 4 reviews.
5 Critical Boiling Transition Phenomena 6 A CBT occurs when a flow regime that has a higher heat transfer rate transitions to a flow regime 7 that has a significantly lower heat transfer rate. In nuclear fuel rods, the heat flux from the fuel 8 pellet to the fuel cladding is mostly independent of the heat transferred from the cladding surface 9 to the coolant. As a result, the cladding temperatures will increase until the new heat transfer 10 mechanisms can remove all of the heat from the pellet; the primary mechanism for post-CBT heat 11 transfer will dictate the magnitude of the cladding surface temperature increase. Typically, the 12 post-CBT heat transfer mechanism transfers heat at a much lower rate (i.e., it is less efficient) 13 than the pre-CBT mechanism, and therefore causes a dramatic increase in clad temperature. The 14 temperature increase resulting from CBT could cause the fuel rod cladding surface to weaken or 15 melt and result in fuel failure, therefore it is considered a critical transition. Hence, the heat flux at 16 which this transition occurs is known as the critical heat flux, the assembly power at which this 17 transition occurs is known as the critical power, and the quality at which this transition occurs is 18 known as the critical quality.
19 The difference in the rate of heat transfer associated with the flow regimes before and after the 20 transition is a convenient way to understand the phenomena of CBT. The sections below discuss 21 the two most common critical boiling transitions, DNB and dryout.
22 2.2.1 Departure from Nucleate Boiling 23 Departure from nucleate boiling results from a change in the flow regime from nucleate boiling to 24 film boiling and is chiefly a concern in PWRs. During nucleate boiling, the bulk coolant, which is 25 mostly liquid with some vapor, is in intimate contact with the cladding. Vapor is generated as 26 bubbles on the cladding surface at nucleation sites. These bubbles grow on the surface, detach, 27 and flow into the bulk coolant stream. As each bubble leaves the surface, cooler liquid fills the 28 space near the surface that was formerly occupied by the bubble, and the boiling process is 29 repeated. The growth, transport, and collapse of the bubbles increases turbulence close to the 30 wall and causes increased mixing in the thermal boundary layer. Ultimately, this boiling results in 31 extremely high heat transfer rates; therefore, the cladding surface is able to support high heat 32 fluxes at relatively low surface temperatures.
33 Departure from nucleate boiling occurs when bulk liquid is prevented from coming into contact 34 with the surface. The ultimate cause of the phenomenon is not fully understood but is believed to 35 be bubble crowding that prevents liquid from contacting the surface. Once liquid coolant can no 36 longer contact the surface, heat transfer to the liquid through convection is no longer possible, and 37 the only mechanisms that transfer heat to the bulk liquid coolant are conduction through the vapor 38 and radiation from the surface. At normal cladding temperatures, both of these types of heat 39 transfer mechanisms are relatively inefficient, and the surfaces temperature must dramatically 40 increase to remove the heat generated in the pellet. This temperature increase is large enough to 41 cause the surface to become unwettable, thus creating a dry patch. This dry patch may spread 42 axially along the rod and blanket a large majority of the rod in vapor. Thus, the flow regime 43 transitions to film boiling. This rapid increase in surface temperature may also result in fuel failure 44 in a very short period of time.
15
1 2.2.2 Dryout 2 Dryout results from a change in the flow regime from annular flow around the fuel rods to 3 dispersed flow and is mostly a concern in BWRs. In annular flow, a thin liquid film surrounds the 4 cladding, and the bulk flow is mostly vapor with some liquid droplets. Convection transfers heat 5 from the cladding to the annular film, causing some of the liquid in the annular film to evaporate 6 from the film surface and thus adding more vapor to the bulk flow. It is currently believed that 7 evaporation is the only boiling that occurs in the annular film boiling regime; no vapor formation 8 occurs at nucleation sites, and no bubbles are generated.
9 As the coolant flows up the channel, it carries the liquid film up along the cladding surface. This 10 results in entrainment of liquid droplets from the annular film into the bulk coolant, thus reducing 11 the amount of liquid in the film. However, some of the droplets in the bulk coolant are also 12 deposited back onto the film. This deposition will increase the amount of liquid in the film and is a 13 chief concern in the design of grid spacers for BWR assemblies. In summary, as the liquid film 14 flows up the cladding, evaporation and entrainment remove liquid from the film while deposition 15 adds liquid to the film.
16 Dryout occurs when the annular film disappears completely. Upon reaching dryout, the bulk fluid 17 transitions from annular flow to dispersed flow. In dispersed flow, there is no continuous liquid film 18 on the cladding, and the bulk flow consists of a mixture of vapor and dispersed liquid droplets.
19 Convection occurs between the vapor and the fuel rod. The droplets also act as a heat sink, as 20 they are in the heated vapor and may absorb heat from the vapor as well as impact the heated 21 rod (assuming the rod is still wettable). Generally, radiation is not a significant mode of heat 22 transfer until the surface temperature become much higher (at which point, the rod is typically 23 unwettable). Although the heat transfer is less in dispersed flow than in annular flow, it is still 24 substantial. As a result, the increase in cladding temperature is typically not as dramatic as that 25 resulting from DNB. However, sustained time in dryout will eventually result in fuel failure.
26 2.2.3 Other Flow Regimes and Transitions 27 It is important to recognize that the flow regimes inside a reactor core are not precisely defined.
28 Further, potential transitions occur between flow regimes that are not considered critical or do 29 not result in a crisis because the transition would not significantly reduce heat transfer. For 30 example, different portions of a PWR fuel assembly may be in subcooled nucleate boiling (i.e.,
31 boiling which occurs when the bulk of the liquid is sub-cooled and not at saturation), nucleate 32 boiling, and annular flow. Although a shift from nucleate boiling to another regime is technically a 33 departure, it is only considered DNB if the new regime has a significantly lower heat transfer rate.
34 It is also important to note that the same CBT model will generally be applied in every flow regime 35 in a given reactor type and is not associated with only a single flow regime. Thus, a specific DNB 36 model used in a PWR will not only be used to predict whether the flow regime has transitioned 37 from nucleate to film boiling, but will also be used to predict a transition from subcooled nucleate 38 to film boiling or a transition from annular flow to film boiling, as all of those flow regimes can exist 39 in a PWR assembly.
40 Determining When Critical Boiling Transition Occurs 41 Given certain key parameter values (e.g., flow rate, power, pressure, temperature), a CBT model 42 predicts either the CHF or the critical power (CP) of the assembly that would cause a CBT. This 43 predicted value is then compared to the current heat flux or assembly power to determine the 16
1 margin to CBT. Typically, CHF models are used in PWRs, whereas CP models are used in 2 BWRs.
3 2.3.1 Critical Heat Flux Models 4 Critical heat flux is the cladding surface heat flux that causes a CBT for a given set of local 5 conditions. It is chiefly associated with PWRs and the phenomenon of DNB; however, as stated 6 earlier, CHF models can also predict other CBTs (e.g., the transition from annular flow to film 7 boiling). CHF models are developed through experiments where, under a given set of inlet flow 8 conditions, power is increased until CHF is observed. A computer code is used to calculate the 9 local flow conditions from the boundary conditions of the experiment, and CHF is correlated to 10 those local flow conditions. Thus, when a computer code is used to simulate an AOO, the CHF 11 model can use the local conditions calculated at any location in the core to predict the critical heat 12 flux at that location. The predicted CHF is then compared with the local heat flux to determine the 13 margin to CBT at that location.
14 2.3.2 Critical Power Models 15 Critical power is the assembly power that causes a CBT. It is chiefly associated with BWRs and 16 the phenomenon of dryout; however, as stated earlier, CP models can also predict other critical 17 flow transitions. Further, the term CP model is something of a misnomer because these models 18 do not generally correlate CP to local conditions (as a CHF model does); instead, they correlate 19 the critical quality (i.e., the quality that causes a CBT) to the boiling length (i.e., distance from the 20 point of initiation of bulk boiling to the location of a CBT). 8 Thus, when a computer code is used to 21 simulate an AOO, the inlet conditions (e.g., power, inlet flow) along with certain local conditions 22 are used to calculate the quality at various axial elevations in the fuel. The quality at each axial 23 elevation is compared to the critical quality at that elevation by assuming that the boiling length is 24 the elevation of the location under consideration. Generally, the critical quality is much greater 25 than the predicted quality; therefore, the assembly power is increased until, at some axial 26 elevation, the critical quality is equal to the predicted local quality. The lowest assembly power at 27 which at least one location equals a quality greater than or equal to the critical quality is known as 28 the CP.
29 2.3.3 Semi-empirical Modeling 30 Since 1970, tremendous strides have been made in the generation of CBT models; however, 31 these models are still predominantly semi-empirical (i.e., the models are based more on 32 experimental data than on first-principle physics). Known physical behavior is often used to inform 33 the models mathematical form, but empirical coefficients are still needed to ensure accurate 34 model predictions. In effect, this means that, although the model may be informed by physics, 35 they are not treated as theoretical models, but are treated as empirical or data-driven models in 36 that they must be validated with experimental data and should not be used outside of the range of 37 their validation database.
8 This is not exclusively true because other models are more mechanistic than critical quality/boiling length correlations. Regardless, even these mechanistic boiling transition models do not generally correlate CP directly to fluid conditions.
17
1 2.3.4 Conservative vs. Non-Conservative Predictions 2 For CBT models, conservative means that the model will predict a CBT before the actual 3 occurrence of the phenomenon (e.g., at lower powers, at lower flow rates, at lower qualities).
4 Conversely, non-conservative means that the model will predict a CBT after the actual 5 occurrence of the phenomenon (e.g., at higher powers, at higher flow rates, at higher qualities).
6 Applying a Critical Boiling Transition Model 7 Unlike many closure models 9 that are developed directly from experimental data, CBT models 8 may 10 call for input that typically cannot be measured directly. In such instances, a 9 thermal-hydraulic computer code is used. This code will calculate the values of key variables 10 needed by the CBT model (e.g., local quality, local mass flux) using some set of field equations 11 and any necessary closure models. Thus, the development and, more importantly, the validation 12 of a CBT model may highly depend on the thermal-hydraulic computer code used and the code 13 options selected. For this reason, using such a CBT model in a different code or in the same code 14 with substantially different code options (e.g., different two-phase closure models) would call for a 15 complete re-validation using those new code or code options.
16 The application of CBT models can be somewhat confusing. Many closure models, such as 17 Dittus-Boelter (1930), operate as a simple function. The function takes in certain inputs and 18 returns an output. Thus, the validation of such a function would ensure that, given the correct 19 inputs, the function returns the correct output. Although CBT models follow a similar process, the 20 models themselves cannot typically be used or validated in such a simple manner. For example, 21 consider an experiment of a test assembly whose power is increased until a CBT occurs. For this 22 experiment, the inlet flow rate and temperature, axial and radial power shapes, pressure, and 23 assembly power have been measured. However, most CBT models do not use this measured 24 data, but require a different set of data to make a prediction. Hence, the measured data from the 25 experiment is input into a computer code and that code generates the required input data such 26 that the CBT model can make a prediction. The following sections describe how this simple 27 situation would be evaluated using methods commonly applied in a PWR and a BWR.
28 2.4.1 Applying a Critical Boiling Transition Model in a Pressurized-Water Reactor 29 In a PWR assembly, a prediction of the CHF would be calculated for each subchannel at each 30 axial elevation. Consider a 5x5 assembly that contains 25 rods, 16 internal subchannels (i.e.,
31 between the rods), and 20 external subchannels (i.e., between the rods and the channel wall).
32 Assume that the assembly has a height of 3.65 meters (12 feet) and that the computer code uses 33 an axial nodalization of 7.62 centimeters (3 inches). In total, each subchannel would have 48 axial 34 elevations; given the 36 subchannels (internal plus external), that results in a total of 1,728 nodes, 35 each of which would have its own CHF prediction from the CBT model. Which of the 1,728 36 predictions should be compared with the single measured CHF from the experiment?
9 Closure models are those additional models needed in order to close the problem. They provide the additional relations needed for the number of equations to equal to the number of unknowns so the problem can be solved. They supplement the conservation equations.
10 Typically, CHF models used in PWRs are subject to this restriction. The subchannel code provides detailed information about the local flow conditions that the CHF model uses to make a prediction. Dryout models are generally less affected because they do not need detailed information about the local flow conditions.
18
1 At first glance, comparing the predicted CHF at the location where the CHF was indicated in the 2 test would seem to be the best approach. Suppose that a thermocouple on one of the inside rods 3 (i.e., a rod internal to the 5x5 array and not on the boundary) was the first thermocouple to 4 indicate that a CHF occurred. Further, suppose that this thermocouple is located at an elevation of 5 2.74 meters (9 feet). This rod would be a member of four subchannels; thus, there may be no way 6 to determine in which of the four surrounding subchannels CHF actually occurred. This does 7 seem to make the problem more tractable because, instead of considering 1,728 nodes, that 8 number is reduced to 4 nodes. However, in such experiments, multiple rods will experience the 9 temperature rise associated with a CBT.11 While one rod at one axial elevation will achieve such a 10 temperature rise first, the temperature rise used to indicate CHF is somewhat arbitrary in that a 11 slightly different criterion may result in a different CBT value. For example, changing the CBT 12 criterion from a rise of 16.67 degrees Celsius (C) (30 degrees Fahrenheit (F)) to a rise of 11.1 13 degrees C (20 degrees F) may result in the selection of a different rod in CBT. Suppose there 14 were five thermocouples indicating that CHF was very likely occurring at those locations. That 15 would mean that of the 1728 nodes in the bundle, 20 would need to be considered for the CBT 16 point.
17 The measured heat flux at the time of CHF could be compared to each of the predicted CHF 18 values in the 20 nodes; however, it is not clear how a single predicted CHF value could be 19 objectively chosen. While a ratio of measured CHF to predicted CHF could be found at each 20 point (the measured value from the experiment and the predicted value from the CBT model) 21 which of these 20 values should be taken as the value from this test? The maximum value, the 22 minimum value, the mean of all 20 values? The usual practice is described below.
23 It is important to remember that the overall goal of a CBT model is to determine whether a CBT 24 will occur. Thus, the validation process should focus on ensuring that the model appropriately 25 predicts a CBT, and not necessarily that the model predicts CBT at the correct location. Thus, 26 when using the model to make predictions measured CHF data will not be available for the 27 reactor assembly under normal operation and AOO conditions. Considering the 5x5 assembly, 28 only 1,728 predictions of CHF for each time step of the scenario will be available. Therefore, those 29 predicted values of CHF are typically compared to the local values of heat flux to determine which 30 of the nodes is closest to CHF using the departure from nucleate boiling ratio (DNBR). The DNBR 31 is defined as the ratio of the predicted CHF of a node to the current heat flux of a node.
32 Equation 1 gives the DNBR.
= (1) 33 Notice that as long as the node is far from the conditions that cause CHF, the value of DNBR will 34 be greater than 1. As the node approaches those conditions, the DNBR value approaches 1, and 35 when heat flux in the node is equal to the CHF, the DNBR is equal to 1. Given that these 36 simulations are used to demonstrate that CHF does not occur, the DNBR in all of the nodes 37 should always be greater than 1. Further, the node with the smallest DNBR, commonly called 38 minimum departure from nucleate boiling ratio (MDNBR), is the node closest to the conditions that 39 cause CHF.
40 These concepts of DNBR and MDNBR are used to select a predicted CHF value to compare 41 with the measured CHF value from the experiment. From the 1,728 nodes, the node that 11 The temperature rise selected is usually on the order of 11.1 to 27.78 C (20 to 50 degrees F) in under 1 second.
19
1 contains the MDNBR could be used as the predicted node, and the CHF prediction at this node 2 could be the predicted CHF. This may or may not be one of the 20 nodes discussed earlier, but 3 using the CHF from this node as the predicted CHF results in a much more representative error 4 of how the CBT model will be applied in practice. While the analyst may know which sub-channel 5 and what elevation CHF occurred at in the experiment, this information is not known in the real-6 world scenario. Thus, this information should not be used in determining the models error.
7 Instead, the predicted CHF value should be determined using the same method that will be used 8 when the model is applied in the real-world scenario.
9 Note that the MDNBR location (and hence the predicted CHF value) may change during model 10 development. Thus, as the model changes during its development, different nodal locations in 11 different subchannels would likely be determined to be more the limiting node.
12 2.4.2 Applying a Critical Boiling Transition Model in a Boiling-Water Reactor 13 In a BWR assembly, the calculation of the predicted CP would consider each fuel rod in the 14 assembly individually. Consider a 5x5 assembly that contains 25 rods that has a height of 3.65 15 meters (12 feet) an axial nodalization of 7.62 centimeters (3 inches). Most BWR methods do not 16 model all of the rods and subchannels; instead, they model only a single rod surrounded by a 17 single subchannel of fluid. Modeling all of the subchannels is considered unnecessary because 18 the fuel assembly is contained within a channel; therefore, the water cannot flow between 19 assemblies. To account for the varying thermal-hydraulic conditions at the different locations in 20 the assembly, two different factors are used to convert the results of the single rod analysis and 21 make it applicable to the entire assembly.
22 The first factor is a relative power factor, commonly called the R- or K-factor. The R- or K-factor 23 accounts for the power in a specific rod compared to the powers in the surrounding rods. In the 24 above example, a different R- or K-factor would be calculated for each of the 25 rods depending 25 on each rods individual power, which can change over the cycle. The second factor is a 26 thermal-mixing factor, commonly called an additive constant. The thermal-mixing factor accounts 27 for the thermal performance at that specific xy location in the assembly. In the above example, a 28 different thermal-mixing factor would be calculated for each of the 25 rods depending on the xy 29 location of each rod in the assembly; that factor would not change for that assembly design.
30 Ideally, the local conditions calculated in the assembly could be directly correlated to the CP.
31 However, this is not the case. A change in power has a dramatic impact on the entire flow field 32 along the length of the assembly, and integral, not local, effects are commonly considered the 33 cause of the CP and its associated phenomenon of dryout. 12 To determine the CP, the mass flow 34 rate, axial and radial power shape, and pressure are fixed. The quality at a given elevation can 35 then be compared to the predicted critical quality from the CBT model given the boiling length 36 (i.e., the length from the start of boiling to the elevation of interest). The power input to the model 37 is increased or decreased until the calculated quality at that location is equal to the critical quality.
38 The corresponding power is the CP.
12 This consideration of integral effects, as well as the concept of flow memory (Tong 1965), seems to be somewhat of a misnomer. Although what occurs upstream shapes the flow field, CBT occurs at a single location based on the conditions of the local fluid and the heat from the wall. If those local fluid conditions could be modeled perfectly, a consideration of integral effects would not be necessary. However, because of modeling limitations, many of the important parameters of that local fluid cannot be directly modeled; therefore, concepts such as flow memory are useful as modeling simplifications.
20
1 Because there are 25 rods, there could be 25 different CPs for each axial elevation. However, 2 because many CBT models correlate the critical quality to the boiling length, it is not necessary to 3 perform calculations below the boiling length. Additionally, it is not necessary to determine the 4 power that would cause a CBT at a certain axial elevation. For example, suppose a CBT occurred 5 on rod 17 at an axial elevation of 10 feet. If an analyst wanted to determine what power would 6 cause a CBT at 8 feet, the power would need to be increased. However, increasing the power to 7 cause a CBT at 8 feet would not make much sense because the goal is to avoid a CBT entirely, 8 and at the current power level, a CBT has occurred. Thus, it is not the power that causes a CBT at 9 every elevation that is important; instead, it is the lowest power that causes a CBT at any 10 elevation at or below the top of the active fuel that is most important.
11 2.4.3 Applying a Steady-State Model to Transient Conditions 12 Generally, CBT models are generated with steady-state data (i.e., the test facility reaches a 13 steady state and slowly increases the power until CBT occurs). Information from those data points 14 is then used to generate CBT models. However, when the data are applied in a reactor safety 15 analysis, the CBT model is applied to the transient (i.e., time-varying) conditions occurring during 16 a scenario. Historically, this application of a correlation developed on steady-state data to 17 transient conditions has been considered conservative, and often a few transient tests are 18 performed to demonstrate that the prediction of a CBT model is conservative when it is applied in 19 a transient fashion.
20 Addressing Uncertainties and Errors 21 Many uncertainties and errors are associated with a CBT model. First and foremost, some of 22 these uncertainties have specific meanings and should be defined. In this work, a distinction is 23 made between an error and an uncertainty. The term error focuses on the difference between 24 specific predicted values and their corresponding specific actual value. For example, the error in 25 a single measurement (absolute error or relative error) is a comparison of the true value to the 26 measured value. The term uncertainty focuses on quantifying the variability of a set of values for 27 future predictions. For example, while a prediction is generally a single value, it may be better to 28 think of that prediction as a range of values where that range is defined by the uncertainty in the 29 prediction. The various forms of uncertainties discussed throughout this document are defined as 30 follows:
31
- Instrumentation uncertainty is associated with a specific instrument used in the 32 experiment. This uncertainty is a result of the underlying precision of the instrument, and is 33 typically provided by the manufacturer of the device in question. Examples include the 34 +/-0.50 degrees C (+/-0.90 degrees F) of a K-type thermocouple or the 1 percent uncertainty 35 of a pressure transducer. Generally, instrumentation uncertainty (future behavior) is 36 approximated through the instrumentation error (past behavior).
21
1
- Measurement uncertainty is the total uncertainty associated with recording the 2 measurement from a piece of instrumentation. Although this is often considered to be 3 simply the instrumentation uncertainty, that may be an oversimplification. Uncertainty is 4 often associated with recording the value from the instrument. Data-logging systems 5 typically read in voltages, but not all measurements are provided as a voltage, and these 6 values would need to be converted. Additionally, some uncertainty occurs in the voltage 7 reading of the data-logging system itself. For example, pressure transducers often provide 8 an output between 4 to 20 milliamperes. This output must be converted through a resistor 9 before it can be measured as a voltage. The uncertainty of the resistance in that resistor 10 should be accounted for in the measurement uncertainty because it may not have been 11 accounted for in the instrumentation uncertainty.
12
- Experimental uncertainty is the total uncertainty associated with recording the value of 13 quantity of interest from an experiment. In many instances, an instrument that measures 14 the quantity of interest may not be available, or even if one is available, that measurement 15 may depend on multiple instruments. For example, the uncertainty associated with the 16 CHF measurement would at least need to consider uncertainties associated with the 17 measured power, the manufacturing tolerances of the heater rods (which influence the 18 axial heat and heat flux shape), and the thermocouples used to determine when a CHF 19 event occurs.
20
22
- Model application error is similar to model error, but it accounts for the fact that the CBT 23 model is not used as a standalone equation, but used in a larger calculational framework.
24
- Validation error is a sample from the population of the model application error. If we 25 consider the model application error as a set which contains the entire population of all 26 possible uses of the model, then the validation error is the sample from that population for 27 which a CHF or CP value was measured in a particular experiment.
28
- Model uncertainty is associated with the application of the CBT model in a future analysis.
29 This may also be referred to as the predictive capability of the model. This uncertainty 30 quantifies the difference (or ratio) between the power at which a model predicts CBT will 31 occur and the power at which CBT would actually occur. Note that it is not only the 32 uncertainty of how the model predicted the experimental data (i.e., the validation error) but 33 also includes how the model would have predicted other experimental data (i.e., other 34 samples from the model application error) and how that experimental data relates to the 35 real world system of interest of the fuel assembly in a nuclear power plant.
36
- Plant parameter uncertainties are associated with specific plant parameters, such as flow, 37 power, and pressures. Although these uncertainties do not generally affect the CBT model 38 directly, they are used along with the CBT model to generate the safety limit.
22
1 3 CREDIBILITY ASSESSMENT FRAMEWORK 2 This section discusses the development of a credibility assessment framework for CBT models.
3 As described above, this framework is a generic safety case expressed using concepts from GSN 4 and maturity assessment. This framework was developed based on the experience of members of 5 the NRC technical staff, documented safety evaluations from previous NRC reviews, and various 6 documents found in the open literature. While it was the goal of the authors to have this 7 framework be applicable to all uses of a CBT (i.e., from a homework problem to reactor safety 8 analysis), much of the evidence is based on the evidence that has been historically used for CBT 9 models applied in reactor safety analysis.
10 The purpose of the framework is summarized as the main goal, G - The CBT model can be 11 trusted. Everything which follows is focused on demonstrating that this main goal is true and 12 defines exactly what is meant by the statement The CBT model can be trusted. The main goal is 13 decomposed into the three subgoals in Figure 3 below.
14 15 Figure 3 Decomposition of G Main Goal 16 As discussed above, the goals (G, G1, G2, G3) are intentionally ambiguous. While there may be 17 no consensus on what is meant by the words trusted, appropriate, logical, and sufficient, 18 most will agree that for a CBT model to be trusted, its experimental data must be appropriate, the 19 model must be logical, and the validation must be sufficient. The further development of the 20 framework through continued decomposition of each goal into sub-goals and specification of the 21 possible levels of evidence, acts to more clearly define these ambiguous terms.
22 The bulk of this section will focus on the decomposition of all sub-goals into base goals. 13 For 23 each base goal, we provide a discussion of the levels of evidence used for demonstrating that the 24 base goals are true and a discussion of the evidence levels that have been historically used for 25 CBT models in reactor safety analysis.
13 A goal that is not decomposed further but is supported by evidence.
23
1 G1Experimental Data 2 Experimental data are the cornerstone of a CBT model. The data are used to generate the 3 coefficients of the model and validate the model. Additionally, previous experimental data often 4 used influence the form of the model. Therefore, it is essential that experimental data are 5 appropriate. The three subgoals in Figure 4 are used to demonstrate that the experimental data 6 are appropriate.
7 8
9 Figure 4 Decomposition of G1Experimental Data 10 11 3.1.1 G1.1Credible Test Facility 12 Test facilities that are used to measure CBT primarily focus on measuring key flow parameters 13 that occur during the critical transition. Experimental data has been collected at multiple research 14 facilities and universities over many years (Groeneveld 2007). However, because the time, effort, 15 and resources needed to set up a reliable facility are quite significant, most CBT data used in the 16 nuclear industry have historically come from one of the following facilities:
17
- Columbia Universitys Heat Transfer Research Facility (closed in 2003) 18
- General Electric Companys ATLAS test loop facility in San Jose, CA (closed) 19
- Stern Laboratories in Hamilton, Ontario (still in use) 20
- AREVAs KATHY loop in Karlstein, Germany (still in use) 21
- Westinghouse Electric Corporations FRIGG and ODEN loops in Vsters, Sweden, for 22 BWRs and PWRs, respectively (still in use) 24
1 The two subgoals in Figure 5 are used to demonstrate the credibility of the test facility.
2 3 Figure 5 Decomposition of G1.1Credible Test Facility 4 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 5 discuss the evidence that could be used to demonstrate that these two base goals (G1.1.1 and 6 G1.1.2) have been satisfied. Additionally, a discussion is provided on the evidence that has been 7 historically used for CBT models applied in reactor safety analysis.
8 G1.1.1Test Facility Description 9 The test facility contains the test loop, the control equipment, interconnected piping, and 10 instrumentation needed to perform the experiment. Test loops usually consist of a test section 11 (which contains the simulated test assembly), pressurizer, heat exchangers, pumps, pressure 12 transducers (both absolute and differential), flow meters, and thermocouples. The test assembly 13 contains the simulated fuel rods, which not only supply the power to the test section but also 14 contain the thermocouples that indicate when a CBT occurs.
15 The description of the test facility must enable the assessor to understand how the facility 16 operates and how the data were obtained. For assessors familiar with CBT testing and for 17 established test facilities, a reference that describes the facility is typically sufficient 18 documentation. In the past, having the assessor visit the test facility and witness testing first hand 19 has greatly increased the assessors understanding, reducing the total time needed for the 20 assessment, particularly for new assessors and/or new test facilitates. Table 6 gives the evidence 21 commonly provided to demonstrate that this goal has been satisfied.
22 Table 6 Evidence for G1.1.1Test Facility Description G1.1.1 The test facility is well understood.
Level Evidence A reference that describes the test facility in appropriate detail has 1 been provided. At a minimum, the reference includes loop, test section, and heater rod descriptions.
2 The assessors have visited the test facility. Additionally, a reference that describes the test facility in appropriate detail has been provided.
25
At a minimum, the reference includes loop, test section, and heater rod descriptions.
1 2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 1 has been most commonly accepted by the NRC staff, but Level 2 has resulted in 4 increased review efficiency. Because the goal of the reference describing the test facility is to 5 allow the assessor to fully understand the function of the test facility including operation, control, 6 and measurement capabilities, it has often been found to be convenient to have the assessor visit 7 the test facility and witness testing. This is especially true for new assessors unfamiliar with a test 8 facility, but also true for experienced assessors who have not reviewed data from a particular test 9 facility for some period of time. Visiting a test facility and observing testing has been a much more 10 efficient way for the assessor to gain an understanding of the test facility than by reading 11 documentation alone. A significant portion of an assessors time is spent gaining an 12 understanding of the test facility. The assessor must understand the facility to such an extent that 13 he or she is able to fully understand a complete test run including how the various pieces of 14 equipment interact. Thus, actually visiting the test facility greatly increases the rate of 15 understanding, typically leading to a reduction in the time needed to perform the assessment and 16 fewer questions.
17 G1.1.2Test Facility Comparison 18 The test facility description is used as an indicator to determine if the facility is capable of 19 generating accurate data. However, another key piece of evidence is the validation of the test 20 facility itself. One type of validation frequently used is a comparison of the measured CBT data to 21 the results from another credible facility. The justification for the test facilities should be based on 22 factors other than the test facility itself (e.g., comparison to a benchmark, reproduction of data 23 from another facility, or reproduction of known phenomena).
24 Most facilities in use today have been compared to their older counterparts (for example, many 25 facilities have performed tests to compare to data collected at Columbia University). However, 26 because of the proprietary nature of the test sections, it may be difficult to obtain comparisons to 27 actual CBT data. Therefore, though a new facility would be under the greatest scrutiny in this 28 framework, it may have difficulty meeting this criterion. When comparisons to actual CBT 29 measurements are not possible, the assessor should compare the test facility under evaluation to 30 measured quantities from other experiments (e.g., in the open literature) with similar phenomena.
31 Table 7 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
26
1 Table 7 Evidence for G1.1.2Test Facility Comparison G1.1.2 The test facility has been verified by comparison to an outside source.
Level Evidence The test facility has been verified by comparison of data obtained at the 1 facility to some benchmarks or some known phenomenological behavior.
The test facility has been verified by comparison of data obtained from 2 tests at the facility to data other than CBT data obtained from a credible facility.
The test facility has been verified by comparison of CBT data obtained 3
at the facility to CBT data obtained from a credible facility.
The test facility has been verified by comparison of CBT data obtained 4 at the facility to CBT data obtained over the same application domain as that of the proposed model at a credible facility.
2 3 Historical Evidence Levels for Reactor Safety Analysis 4 Evidence at Level 2 and Level 3 have been most commonly accepted by the NRC staff. This is 5 largely due to the fact that most test facilitates in operation today are 2nd generation facilitates, and 6 part of their initial testing program was to establish consistency with the data taken from 1st 7 generation facilities. When comparisons to actual CBT measurements are not possible, it is 8 possible for the assessor to consider other measured quantities besides CBT from other 9 experiments with similar phenomena.
10 3.1.2 G1.2Accurate Measurements 11 In order for the test data to be relied upon, the test facility needs to provide accurate 12 measurements of all important experimental parameters, including the measurement of CHF or 13 CP. It is important to note that neither CHF nor CP is a directly measured parameter (like flow rate 14 or pressure); instead, the CHF or CP value is inferred from the assembly power, axial and radial 15 power peaking, and a thermocouple indication that signifies CBT has occurred in the test facility 16 and where in the test section CBT has occurred.
17 Typically, five experimental parameters are directly measured or controlled. 14 The type of control 18 used for the experimental parameters depends on the type of data being taken. Usually, the 19 desired values are programmed into a computer, and the computer will maneuver the control 20 equipment to the desired state point. Table 8 presents the methods used to measure and control 21 each experimental parameter.
14 Although the axial heat flux shape is very important for obtaining the local power and may be changed through the exchange of test rods, it is not a measured value during the experiment and, therefore, will be treated in Section 3.1.3 on local conditions.
27
1 Table 8 Experimental Parameters Measured or Controlled Parameter Method of Measurement Typical Method of Control Absolute and differential pressure Pressure A pressurizer on the test loop cells on the test section Power (including Rectifiers that supply power to the Reading from rectifiers radial power peaking) simulated fuel rods Inlet Flow Rate Flow meter at the inlet Valve at the inlet or pump speed Heat exchanger or mixer at the Inlet Temperature Thermocouple at the inlet inlet N/A (the change in rod Rod Temperature Thermocouples inside the temperature is not controlled, but is Change simulated fuel rods a response quantity) 2 The six subgoals in Figure 6 are used to demonstrate the accuracy of the measurements.
3 4 Figure 6 Decomposition of G1.2Accurate Measurements 5 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 6 discuss the evidence that could be used to demonstrate that these six base goals have been 7 satisfied. Additionally, a discussion is provided on the evidence that has been historically used for 8 CBT models applied in reactor safety analysis.
28
1 G1.2.1 Test Facility Quality Assurance (QA) Program 2 A determination regarding the credibility of a test facility is often assessed by reviewing the quality 3 assurance program applicable to the test facility. Typically, an assessment of a facilitys QA 4 program involves determining its compliance with a standard (e.g., ASMEs NQA-1, Quality 5 Assurance Requirements for Nuclear Facility Applications). While different QA standards will have 6 different elements, the following represent some of the issues that should be addressed:
7
- Calibrated instrumentation - Routine calibration of the instrumentation is necessary to 8 ensure that an instrument is resulting in a precise measurement and to quantify any 9 instrumentation error (i.e., accuracy and precision). Generally, the instrumentations 10 calibration is checked on a routine basis, with the calibration interval set to account for 11 instrument drift over time and drift due to operation. This check should be performed often 12 enough to avoid having to recalibrate the instrumentation after its use. If an instrument 13 does need to be recalibrated after a test, it likely means that the last set of data points 14 taken with that instrument were taken when the instrument was out of calibration. At a 15 minimum, a calibration check should be performed at both the beginning and end of a test 16 campaign. The general assumption is that, if an instrument is within its calibration 17 specification at the beginning and end of a campaign, there is very little chance that it was 18 out of its specification at any time during the campaign. Note that, contrary to the 19 discussion above, the heater rod thermocouples used to detect a CBT are often not 20 calibrated because the absolute value of the temperature is not used. Instead, as 21 previously discussed, a change in temperature over a period of time is used as the 22 criterion for determining that a CBT has occurred. However, the thermocouples used to 23 determine fluid and wall temperatures elsewhere in the test loop should be calibrated.
24 NQA-1, Requirement 12 Control of Measuring and Test Equipment provides more details 25 on instrument calibration.
26 27
- Appropriate equipment - The experimental parameters measured in CBT experiments 28 are provided in Table 8 above. Therefore, instrumentation should be employed to measure 29 these parameters. However, as instrumentation may fail or provide anomalous 30 measurements, a common practice is to employ redundant and diverse instrumentation.
31 Redundant instrumentation is necessary to ensure that (1) instrumentation remains in 32 calibration, and (2) an instrument which suddenly becomes uncalibrated does not greatly 33 impact the resulting experimental data. Further, diverse instrumentation (i.e., use of a 34 different process to perform the measurement) helps achieve a higher degree of 35 confidence that the final measurement is accurate because it reduces the potential for 36 common cause failures that could result in inaccurate measurements.
37 38
- Trained personnel - There are many appropriate ways in which the data could be 39 obtained. It is important that the personnel performing the tests have been trained on the 40 test procedure and test equipment, and are able to follow the test procedure in order to 41 ensure consistent experimental results.
42 43
- Condition of test equipment and the item to be tested - The test equipment, including 44 the instrumentation, the test section, and all connected piping, should be demonstrated to 45 be in working order. Generally, the bulk of these activities is performed during the 46 shakedown testing, which ensures the test facility is behaving as expected.
47 29
1
- Suitable environmental conditions - As CBT tests are often performed in state of the art 2 experimental test facilities, the conditions for both the equipment and the personnel are 3 generally suitable environments.
4 5
- Provisions for data acquisition - As the data will be used to validate the CBT model, the 6 acquisition of the data are of paramount importance. While there are multiple data 7 acquisition systems that could be used, it is important for specific procedures to be 8 developed and used in order to determine how the data are reduced to the final set of 9 measured values.
10 11 Table 9 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
12 13 Table 9 Evidence for G1.2.1Test Facility QA Program G1.2.1 The test facility has an appropriate quality assurance program.
Level Evidence A QA program exists that reflects the basic tenets of quality assurance 1 as referenced by a widely accepted international quality organization (e.g., NQA-1).
A QA program exists that reflects the basic tenets of quality assurance as referenced by a widely accepted international quality organization 2 (e.g., NQA-1). Documentation is provided that outlines how the design, construction, and test activities were conducted consistent with the QA Program. It is clear that the base expectations of QA were applied.
A QA program exists that reflects the basic tenets of quality assurance as referenced by a widely accepted international quality organization (e.g., NQA-1). Documentation is provided that outlines how the design, 3 construction, and test activities were conducted consistent with the QA Program. It is clear that the base expectations of QA were applied.
Audit reports properly identify, track, and indicate correction of conditions adverse to quality and are available for inspection.
14 Historical Evidence Levels for Reactor Safety Analysis 15 Level 3 has been most commonly accepted by the NRC staff. While the CBT assessor does not 16 typically examine the QA program in the same detail as a QA inspector, previous NRC reviews 17 have shown that understanding the QA program helped the assessor gain an improved 18 understanding of how the data were taken, controlled, reduced, and then used to generate the 19 model. It is important to note that most assessors have typically limited their review to confirming 20 that some type of QA program was in place, rather than providing an extensive review of that 21 program itself.
30
1 G1.2.2Statistical Design of Experiment 2 The goal of the statistical design of the experiment (Box, Hunter, and Hunter, 1978) is to ensure 3 that the testing methods do not introduce any biases into the figure of merit (i.e., the CHF or CP 4 value). Most of the statistical methods used to quantify the uncertainty treat all errors as random.
5 This is equivalent to assuming that each measurement is taken at a randomly determined 6 experimental state point 15 that is completely independent of any measurements taken before or 7 after. However, that is generally not possible for CBT experiments. First, large changes in 8 pressure (and sometimes flow rate) can put tremendous stresses on the test section. Second, 9 changes in flow cause the test section to reach a new thermal equilibrium, which may take a long 10 time. As such, it is often not feasible to dramatically change the flow rate or pressure between test 11 points. Because of these issues, the order in which the test points are taken is typically not 12 random. Table 10 gives the evidence commonly provided to demonstrate that this goal has been 13 satisfied.
14 Table 10 Evidence for G1.2.2Statistical Design of Experiment The experiment has been appropriately statistically designed (i.e., the G1.2.2 value of a system parameter from any test was completely independent from its value in the test before and after the test).
Level Evidence One or more system parameters were randomized, but no 1
consideration was given to other system parameters.
One or more system parameters were randomized, and some 2
consideration was given to all other system parameters.
One or more system parameters were randomized, and those 3 parameters that were not randomized between tests were randomized in larger test blocks.
4 All system parameters were completely randomized.
15 Historical Evidence Levels for Reactor Safety Analysis 16 Level 3 has been most commonly accepted by the NRC staff. In general, the design of the 17 experiment attempts to randomize the system parameters as much as possible between each 18 test. Since testing is often split into groups (e.g., a set of tests at a single pressure and/or flow 19 rate), parameters are often randomized between test groups. For example, if the pressure were 20 held constant during a group of tests, then the pressures from group to group should be 21 randomized. As much as possible, flow rates are also randomized for a fixed pressure. Because 22 randomization (i.e., independence) is a key assumption in all of the statistics performed on the 23 data and because it is generally not possible to guarantee randomization through the design of 24 the experiment, repeated test points have become a vital part of demonstrating that there are no 25 biases in the test facility.
15 By state point, we mean the value of each variable that completely determines the state of the system.
31
1 G1.2.3Data Fidelity 2 The method used to obtain CBT data should result in an accurate measurement of CBT. There 3 are typically two different types of tests used in CBT experiments: (1) those used to obtain 4 steady-state data and (2) those used to obtain transient data. It is vital that assessors understand 5 exactly what is occurring in each of these tests. Therefore a careful evaluation of the test 6 constraints, input assumptions, and expected result ranges should be employed.
7 Measuring a Steady-State Data Point 8 For steady-state data, the objective is to determine the state point at which CBT occurs. A state 9 point is a coordinate in an n-dimensional space defined by all of the parameters which make up 10 the system. In general, there are two main types of state point: experimental state points and 11 model state points. For an experimental state point, the parameters of interest are those 12 parameters that influence the overall experiment (e.g., system pressure, total power (including 13 radial and axial peaking), inlet flow rate, and inlet temperature). For a model state point, the 14 parameters of interest are those parameters needed by the model to make a prediction of CHF or 15 CP. Depending on the model itself, these generally include global as well as local parameters as 16 well as parameters that are not measured in the experiment (e.g., local mass flux, local quality).
17 For PWRs, the values of parameters that are not measured in the experiment are obtained using 18 a subchannel code that predicts the local flow behavior in the subchannels using the experimental 19 parameters as boundary conditions. In a sense, the subchannel code used can be thought of as 20 the means by which the experimental state point is transformed into a model state point.
21 It is important to note that a reactor almost never operates at a steady state, especially during an 22 AOO. Because the models are based on steady-state data, the model effectively treats each AOO 23 as if it were made up of a multitude of steady-state state points and determines the heat flux or 24 assembly power that causes a CBT at those individual state points. Multiple previous applications 25 of steady-state models have been demonstrated to be conservative (i.e., a model developed with 26 steady-state data will generally underpredict the heat flux or assembly power that causes a CBT),
27 and it is common for analysts to provide data demonstrating that this conservative assumption 28 remains true for each individual CBT model.
29 The following standard procedure is used to measure a steady-state data point:
30 (1) An experimental state point is chosen. As previously discussed, a single value of 31 pressure, power, power shape, inlet flow rate, and inlet temperature is generally 32 chosen. Usually, the initial power is chosen to be somewhat lower than that expected 33 to cause a CBT.
34 (2) The experimental facility is driven to the state point. Generally, a computer operates 35 the control system to allow for finer control.
36 (3) Once the initial state point is reached, power is slowly increased while maintaining 37 steady conditions on the other experimental parameters. Some variation in the values 38 of the experimental parameters will exist, but this variation should be kept small and 39 should be accounted for in test procedures. Although steady-state CBT data could be 40 obtained by varying any one of the experimental parameters in an appropriate 41 direction while keeping the others constant (e.g., decreasing the flow rate), such data 42 are usually obtained by slowly increasing the power.
32
1 (4) As the power is slowly increased, the rod internal thermocouples are monitored. A 2 CBT is assumed to have occurred if the temperature indicated by one of the 3 thermocouples increases by a specified amount over a specified small period of time 4 or if some maximum temperature is reached.
5 (5) Once a CBT occurs, power is reduced and the values of the parameters that make up 6 the experimental state point are written to a file. These data, along with the known 7 axial and radial power shape, can then be used to calculate either the CHF or the CP.
8 Measuring a Transient Data Point 9 The objective for transient data are to determine the lowest power level at which a specific 10 transient will cause a CBT. In this case, a specific transient is defined through specified 11 time-dependent functions for each experimental parameter. Typically, a computer controls the 12 experimental parameters to ensure that the test achieves the desired behavior of the 13 time-dependent functions. However, not all experimental parameters will vary during the transient 14 (e.g., pressure is almost never varied because of the strain this would place on the test loop). In 15 this sense, steady state can be considered a special type of transient where all time-dependent 16 functions are held constant.
17 It is also important to note that each AOO is not directly mapped to a specific transient test.
18 Although some AOOs can be mapped into a transient test (e.g., loss of flow), this is not possible 19 with all AOOs. AOOs involving rapid changes in pressure are especially challenging because any 20 rapid change in pressure in the test loop could put the loop at risk. Therefore, additional analysis 21 is usually performed to determine how the transient testing bounds the AOOs.
22 One of the similarities between transient and steady-state testing is the objective of the test. In 23 each case, the objective is to determine the minimum power at which a CBT will occur for some 24 set of initial and boundary conditions. It is important that the focus is on obtaining the minimum 25 power at which a CBT occurs under some set of conditions. Simply finding any power which 26 causes a CBT is not useful as one can always be caused by any sufficiently high power. For 27 example, every conceivable transient will result in a CBT at Grahams number 16 of watts, or even 28 10100 watts (much smaller than Grahams number of watts). This does not mean that CBT will 29 occur only at a power of Grahams number because it will obviously occur at much lower powers.
30 Therefore, the objective is to determine the minimum power at which a CBT occurs for those initial 31 and boundary conditions. Thus, if those conditions (either steady state or transient) occur in a 32 reactor and if that minimum power is not reached, a CBT would not occur.
33 The following standard procedure is commonly used to measure a transient data point:
34 (1) A specific transient is chosen. As previously discussed, time-dependent functions of 35 pressure, power, inlet flow rate, and inlet temperature are generally chosen.
36 (2) The experimental facility is driven to the state point. Generally, a computer operates 37 the control system to allow for finer control.
16 Grahams number is one of the largest numbers known in mathematics. It is many orders of magnitude larger than the total number of particles in the observable universe.
33
1 (3) Once the initial condition state point is reached, the transient is started. The values of 2 experimental parameters are defined as time-dependent functions that are controlled 3 to within their desired magnitude by the control system.
4 (4) The rod internal thermocouples are monitored during the transient. A CBT is assumed 5 to have happened if the temperature indicated by one of the thermocouples increases 6 by a specified amount over a specified small period of time.
7 (5) The magnitude of the initial power can be either increased or decreased, and the 8 same transient can be run again to determine the minimum power at which a CBT 9 occurs. Frequently, the same transient is performed multiple times to determine the 10 minimum power.
11 (6) Once the minimum power at which a CBT occurs is known, power is reduced, and the 12 values of parameters that make up the experimental state point are written to a file.
13 These data, along with the known axial and radial power shape, can then be used to 14 calculate either the CHF or the CP for that transient.
15 Table 11 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
16 Table 11 Evidence for G1.2.3Data Fidelity The method used to obtain critical boiling transition data results in an G1.2.3 accurate measurement.
Level Evidence A reference has been provided that describes the method used to 1
obtain results from both steady-state and transient tests.
A reference has been provided that describes the method used to obtain both steady-state and transient tests. The assessors have 2
examined the reference and believe that it will result in accurate measurements of the CBT for both steady-state and transient tests.
A reference has been provided that describes the method used to obtain both steady-state and transient tests. The assessors have 3 examined the reference and believe that it will result in accurate measurements of the CBT for both steady-state and transient tests.
Additionally, the assessors have observed the method in practice.
17 Historical Evidence Levels for Reactor Safety Analysis 18 Levels 2 and 3 have been most commonly accepted by the NRC staff. An accurate measurement 19 of CBT has three main focuses: (1) Ensuring the state point (i.e., pressure, mass flux, inlet 20 subcooling, power) has been measured and maintained during the entire test run within some 21 small uncertainty; (2) Ensuring that any CBT that would occur is captured in the data; (3) Ensuring 22 that the power at which CBT was recorded was the lowest power that would cause a CBT at that 23 state point. A large part of the review process is spent in gaining an understanding of how the 24 data are taken, reduced, and then used to generate the model. To that end, observing the 25 experiment has been one of the most efficient ways to gain this information.
34
1 G1.2.4Instrumentation Uncertainty Impact 2 Accurate measurements are vital to the success of any experimental program. Therefore, the flow 3 rates, temperatures, pressures, and powers must be measured accurately and precisely, and their 4 associated instrumentation uncertainty must be kept low. Typically, the models uncertainty does 5 not directly account for instrumentation uncertainties; instead, such uncertainties are treated as 6 part of the randomness of the data. If those uncertainties are reasonably low over the range for 7 which the measurements are taken, this assumption is generally valid. Table 12 gives the 8 evidence commonly provided to demonstrate that this goal has been satisfied.
9 Table 12 Evidence for G1.2.4Instrumentation Uncertainty Impact The instrumentation uncertainties have been demonstrated to have a G1.2.4 minimal impact on the measured CHF or CP.
Level Evidence 1 The instrumentation uncertainties have been quantified.
The instrumentation uncertainties have been quantified and an analysis is used to demonstrate that the uncertainties result in a minimal impact on the measured CHF or CP.
2 OR The instrumentation uncertainties have not been quantified, but repeated test points allow those uncertainties to be captured directly in the CHF or CP value.
The instrumentation uncertainties have been quantified and an analysis is used to demonstrate that the uncertainties result in a 3
minimal impact on the measured CHF or CP. This has further been demonstrated by experiments (e.g., repeated test points).
10 Historical Evidence Levels for Reactor Safety Analysis 11 Level 3 has been the most commonly accepted by the NRC. While a quantitative analysis of the 12 instrumentation uncertainties on the measured CHF or CP values is possible, it is often more 13 complicated than simply taking additional data points to measure the uncertainty directly. While 14 such an analysis does assume that the instrumentations uncertainty remains constant over the 15 course of the test, this can usually be confirmed by performing an additional test at the same state 16 point to generate a repeat test point.
17 G1.2.5Repeated Test Points 18 The instrumentation uncertainty may be obtained from the instrumentation manufacturer or during 19 calibration. However, the uncertainty on the measured CHF or CP at the location of interest 20 (i.e., the experimental uncertainty) cannot be obtained so easily. This uncertainty is a combination 21 of the instrument uncertainty; uncertainties of other input parameters (e.g., axial power shape, 22 selection of the subchannel of interest); and the method used to combine all of the parameters to 23 generate a measured CHF or CP at the location of interest.
24 Because the CHF or CP at the location of interest cannot be directly measured, the experimental 25 uncertainty should be determined by obtaining a measurement of CHF or CP at the 35
1 experimental state point multiple times over the entire test cycle and analyzing the variability in the 2 results. Some variation in the input parameters will occur because obtaining the exact same 3 experimental state point (i.e., pressure, flow rate, and inlet subcooling) is not possible, but this 4 variability should be small compared to the uncertainty in the measured CHF or CP value. A 5 number of repeated test points should be taken at multiple experimental state points and at 6 various times during the test campaign to ensure that the behavior of the test facility has not 7 changed and to provide a quantitative estimate of the uncertainty in the measured CHF or CP.
8 The variability in the resulting CHF or CP values should be much lower than the quantified 9 uncertainty of the model. If it is not, this is evidence that there is an error in determining the 10 models uncertainty. Table 13 gives the evidence commonly provided to demonstrate that this 11 goal has been satisfied.
12 Table 13 Evidence for G1.2.5Repeated Test Points The uncertainty in the CHF or CP is quantified through repeated tests G1.2.5 at the same state points.
Level Evidence 1 No repeat test points have been taken.
One repeat test point was taken over the test campaign. The variability 2
in the resulting CHF or CP value was reasonably low.
Multiple repeat test points were taken over the test campaign at various 3 input parameters. The variability in the resulting CHF or CP values was reasonably low.
13 Historical Evidence Levels for Reactor Safety Analysis 14 Level 2 and Level 3 have been most commonly accepted by the NRC staff. Aside from satisfying 15 this goal (G.1.2.5), multiple repeat test points (Level 3) can also be used as evidence that the 16 behavior of the test assembly remains consistent over the time frame of the test. The repeated 17 test points may become much more important if other aspects of the behavior of the test assembly 18 are called into question. For example, if there is a geometry change during testing, then the 19 impact of that change could be determined to be minimal if there are an adequate number of 20 repeated test points. The variability from repeat test points is typically small compared to the 21 uncertainty of the CBT model. Additionally, due to the limitations on the statistical design of the 22 experiment, multiple repeat test points are one way to provide evidence that the errors are indeed 23 random and that each experimental state point can be considered independent of every other 24 state point.
25 G1.2.6Quantified Heat Losses 26 Along with accurate flow, pressure, temperature, and power measurements, the test section heat 27 losses should also be quantified. Because the CHF or CP is obtained from the power 28 measurement, ignoring the heat losses would result in a measured CHF or CP higher than the 29 actual CHF or CP value by the amount of heat loss. This would result in a non-conservative 30 measurement.
31 Typically, test section heat losses are kept very low through active means. In many cases, the test 32 section may sit in a heated water bath to ensure minimum heat loss through the walls. Generally, 36
1 while the absolute value of the test section heat losses to the surroundings increases as the test 2 assembly power increases, the percentage of the heat losses relative to the test assembly power 3 actually decreases (i.e., the fraction of heat dissipated in the fluid in the test section is lower for 4 higher powered tests). Therefore, the bounding heat losses are generally quantified through a test 5 conducted at a low assembly power. The assessor needs to establish whether the measured CHF 6 or CP data were corrected for the heat losses before the development of the CHF or CP model. If 7 not, the assessor should consider the inherent non-conservatism. Table 14 gives the evidence 8 commonly provided to demonstrate that this goal has been satisfied.
9 Table 14 Evidence for G1.2.6Quantified Heat Losses The heat losses from the test section are quantified, appropriately low, G1.2.6 and duly accounted for in the measured data.
Level Evidence Heat losses have been quantified and are minimal, but they have not 1
been removed from the power used to calculate the CHF or CP.
Heat losses have been quantified and have been removed from the 2
power used to calculate the CHF or CP.
10 Historical Evidence Levels for Reactor Safety Analysis 11 Level 1 has been most commonly accepted by the NRC staff. Generally, the percentage of heat 12 loss is calculated for each test. The percentage of heat loss is usually estimated to be greater 13 than that of the actual heat loss measured, but it should still be very low compared to the overall 14 power. Overestimating the heat loss is conservative for the reason given above. While it is 15 generally desirable to minimize heat losses from the test section, it is not strictly necessary as 16 long as the heat losses are measured and accounted for the in power measurement.
37
1 3.1.3 G1.3Reproduction of Local Conditions 2 The local conditions in the reactor fuel assembly should be reproduced in the test assembly to 3 ensure that experimental data taken in the laboratory apply to the reactor fuel assembly placed in 4 the reactor. The five subgoals in Figure 7 are used to demonstrate the reproduction of local 5 conditions.
6 7 Figure 7 Decomposition of G1.3Reproduction of Local Conditions 8 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 9 discuss the evidence that could be used to demonstrate that these five base goals have been 10 satisfied. Additionally, a discussion is provided on the evidence that has been historically used for 11 CBT models applied in reactor safety analysis.
12 G1.3.1Equivalent Geometric Dimensions 13 The test assembly provides the structure in which the flow field will be established. The flow field 14 details, many of which will not be measured or directly reproduced in the computer simulation, will 15 directly affect the CBT. Therefore, the flow field in the test assembly should be as similar as 16 possible to the flow field in the reactor fuel assembly.
17 To ensure a similar flow field, the test assembly is manufactured as a prototypical fuel assembly.
18 This includes the fuel rod pitch and diameter, guide tube rod location and diameter, part-length 19 rod height and axial and radial locations, flow areas, number of grid spacers, distances between 20 grid spacers, grid spacer heights relative to the bottom of the fuel assembly, and total assembly 38
1 height. Each of these dimensions should be within the design tolerances of the reactor 2 assemblies.
3 For BWRs, the test assembly is typically full size (e.g., 8x8, 9x9, 10x10) or symmetric (5x5).
4 However, for PWRs, a full-size assembly (e.g., 15x15, 17x17) would require a substantial amount 5 of power. Therefore, smaller 5x5 or 6x6 test assemblies are used. In the early days of CBT 6 testing, 4x4 or smaller assemblies were used, but the unheated channel wall surrounding the test 7 assembly had too large an effect on the interior subchannels. Therefore, 4x4 (and smaller) 8 assemblies are considered too small to provide an adequate representation. 17 9 Note that heater rods are potentially subject to large electromagnetic forces caused by the current 10 flowing through them. These forces must be countered or the rods will bend and the subchannel 11 flow area will change during testing. In indirectly heated rods, the direction of the current in 12 adjacent rods can be reversed to counter the electromagnetic forces. However, this is not possible 13 in directly heated rods because the electric potential must be the same in all rods at each grid 14 spacer. Therefore, in order to maintain the sub-channel size in directly heated rod bundles, simple 15 support grids are commonly used. These grids provide structural support and are designed to 16 have minimal impact on the flow field. Often, the grids are only needed in sections of the 17 assembly where there are large spans between mixing vane grids. Table 15 gives the evidence 18 commonly provided to demonstrate that this goal has been satisfied.
19 Table 15 Evidence for G1.3.1Equivalent Geometric Dimensions The test assembly used in the experiment should have geometric G1.3.1 dimensions equivalent to those of the fuel assembly used in the reactor for all major components.
Level Evidence Many of the components in the test assembly have geometric dimensions equivalent to those of fuel assemblies used in reactors and are within the design tolerance of the fuel assemblies that will be used 1
in the reactor. Any components that do not have equivalent geometric dimensions have dimensions that would result in a conservatively lower prediction of the power or heat flux that causes a CBT.
The vast majority of the components in the test assembly have equivalent geometric dimensions that are within the design tolerance of 2 the fuel assemblies that will be used in the reactor. The few components that do not have equivalent geometric dimensions would have a minimal impact on CBT measurements.
All components in the test assembly have equivalent geometric 3 dimensions that are within the design tolerance of the fuel assemblies that will be used in the reactor.
17 This is not referred to as the cold-wall effect, even though it is due to the impact of the outer cold wall. The term cold-wall effect is reserved for the effect of control rod guide tubes and instrument tubes on CHF performance.
39
1 Historical Evidence Levels for Reactor Safety Analysis 2 Level 2 has been most commonly accepted by the NRC staff. For some older CBT models, the 3 heated length was varied to cover a wider range of fuel. While this could be understand to be level 4 1, it would strongly depend on the importance of the heated length in the CBT model.
5 While there may be instances in which a CBT model may only achieve Level 1, the demonstration 6 that level 1 is acceptable is challenging as it is difficult to prove that the CBT model would produce 7 conservative predictions under all conditions.
8 G1.3.2Prototypical Grid Spacers 9 One of the most important parts of the prototypical assembly is the grid spacer. The spacers 10 ensure that the rods maintain the same pitch as the assembly used in the reactor. The spacers 11 are also the major source of turbulence which acts to increase the heat transfer from the fuel rods.
12 Grid spacers are specifically designed to increase the power or heat flux at which a CBT occurs.
13 In BWRs, the grid spacer is typically designed to increase deposition by directing more of the 14 water droplets entrained in the vapor flow back onto the liquid film. Great care is taken to ensure 15 that the liquid film is not separated (stripped) from the fuel rod in the vicinity of the grid spacer. In 16 PWRs, the grid spacer is typically designed to strip the bubble layer from near the fuel rod surface 17 to reduce bubble crowding and to enhance turbulence and mixing in the subchannel.
18 Arguably, the design of the grid spacer will have a larger impact on the CBT than any other input 19 parameter. The grid spacers increase the margin to CBT through their increase in turbulence or 20 increase in deposition on the fuel rod. However, the current generation of the computer 21 simulations that make use of CBT models do not directly simulate the impact of the spacers; 22 therefore, the CHF or CP model must capture the spacers impact. The number of mixing vanes, 23 the shape of the vanes, the location of the vanes in the subchannel, the surface area of the vanes, 24 the angle of the vanes, and the direction of swirl caused by the vanes can all affect the thermal 25 mixing in the fuel assembly subchannel. Therefore, it is vital that the grid spacer used in the test 26 assembly is prototypical when compared to the grid spacer used in the reactor core.
27 Unfortunately, it is not always possible to use prototypical grid spacers. Therefore, if such grid 28 spacers cannot be used, the grid spacers used in the test section should result in conservative 29 behavior compared to the grid spacers in the reactor core. However, it is very difficult to prove that 30 the one grid spacer will result in conservative behavior under all conditions when compared with 31 another grid spacer. Therefore, demonstrating conservative behavior can be a challenge.
32 Additionally, fuel assemblies may be comprised of different grid spacer types at different axial 33 elevations. Therefore, the same grid types should appear in the test assembly and at the same 34 elevations as in reactor fuel. Table 16 gives the evidence commonly provided to demonstrate that 35 this goal has been satisfied.
40
1 Table 16 Evidence for G1.3.2Prototypical Grid Spacers The grid spacers used in the test assembly should be prototypical of G1.3.2 the grid spacers used in the reactor assembly.
Level Evidence The grid spacers used in the test assembly will result in a conservative 1 under-prediction of the true thermal mixing caused by the grid spacers in the reactor assembly.
The grid spacers used are very similar to those that will be used in the 2
reactor assembly but with some slight differences.
The grid spacers used are identical to those that will be used in the 3 reactor assembly except for the number of rods (e.g., a 6x6 cutout of a 17x17 assembly).
The grid spacers used are identical to those that will be used in the 4 reactor assembly (either identical in size or a symmetric cut of the grid spacer).
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 3 has been most commonly accepted by the NRC staff for PWRs, and Level 4 has been 4 most commonly accepted for BWRs. PWRs typically operate at a higher linear power density, 5 have more rods per assembly, and have fewer assemblies per core. Therefore, it is impractical to 6 test an entire PWR assembly in a test facility because the power needed would be too high.
7 Additionally, PWR methods use a true subchannel analysis and, therefore, model the grid 8 spacers impact on the local fluid quantities. On the other hand, BWR methods use a simplified 9 subchannel analysis that considers only assembly-averaged flow parameters and, therefore, calls 10 for experimental details on every fuel rod in the assembly. For this reason, BWR tests use 11 full-sized assemblies or representative symmetric sub-assemblies.
12 Levels 1 and 2 are not common in reactor safety analyses, as even small changes in the grid 13 spacer can have major impacts to the flow field.
14 G1.3.3Axial Power Shapes 15 It is important to reproduce the local powers created by the reactor assembly in the test assembly.
16 This is generally done by testing combinations of axial and radial power shapes. Although the fuel 17 rods in the reactor can take on an almost infinite number of axial power shapes, generally only 18 three shapes (cosine, up-skew, and down-skew) are used in testing for BWR models, and three 19 shapes (uniform, cosine, and up-skew) are used in testing for PWR models. Additionally, because 20 of the current experimental designs, the only way to change the axial power shape even in 21 modern CBT testing is to replace the test rods, which is a major undertaking. Every test rod, 22 regardless of whether it is directly or indirectly heated, is constructed to produce a specific axial 23 power shape.
24 In a directly heated rod, the rod is connected to a power source at the top and bottom, and 25 electricity flowing through the rod itself generates the heat for the test. The axial power shape is 26 manufactured into the rod by adjusting the rods wall thicknessthis impacts the rods electrical 27 resistance and hence the power produced at different elevations. The outside rod diameter is held 41
1 constant, and the inside diameter is changed to make the rods cross-sectional area thicker or 2 thinner. If the rod walls cross-sectional area is increased by making the wall thicker, the resistivity 3 of that section will decrease, and the power produced per unit length will decrease. Conversely, if 4 the rod walls cross-sectional area is decreased by making the wall thinner, the resistivity of that 5 section will increase, and the power will increase. Because the highest rod power occurs at the 6 thinnest areas, which are not easy to manufacture, the uncertainty on this peak power 7 (i.e., thickness of the rod) was historically one of the largest uncertainties in the experiment.
8 In an indirectly heated rod, a heating coil is placed inside the rod and the power shape is 9 controlled by modifying the dimensions of the coil. This coil is then slid into a clad, which acts as 10 the surface of the test rod. Because PWR testing calls for high heat fluxes, PWR testing generally 11 uses directly heated rods. BWR testing may use either directly or indirectly heated rods.
12 Although any number of axial power shapes could be prescribed in the manufacturing of the rods, 13 typically the rods will have one of four shapes: (1) uniform, (2) cosine, (3) up-skew, or 14 (4) down-skew. Aside from the uniform power shape, each power shape represents a different 15 situation or a different time in the core life. Historically, the uniform power shape was the first 16 power shape used in testing because of the ease of manufacturing (i.e., tubes of a constant wall 17 thickness). However, such a shape always results in a CBT at the very top of the assembly. This 18 situation is considered unphysical (i.e., it does not occur in actual reactors), and questions have 19 recently been raised (Yang et al., 2014) on the usefulness of such uniform test data.
20 Consequently, the uniform power shape has been used less frequently in modern CBT testing.
21 Because early CBT data were based on testing that assumed a uniform power shape, a method 22 was needed to convert the models predictions so the models could be used for the nonuniform 23 power shapes that occur in reactors. One method used was the Tong factor (Tong et al., 1965).
24 Initially, the Tong factor was not a part of the CHF model. Instead, it was used to correct the 25 prediction of the CHF model. The factor attempts to adjust the predicted CHF based on the given 26 axial power shape, some information on local conditions, and the elevation under consideration.
27 However, as CHF models have developed, this shape dependence has become more integrated 28 into the model itself.
29 Ultimately, it is important to ensure that the axial power shapes tested bound all possible power 30 shapes for which the CBT model will be used. One way to demonstrate this is by training a model 31 (i.e., statistically determining its coefficients using regression) with one axial power shape and 32 validating it with another. Table 17 gives the evidence commonly provided to demonstrate that this 33 goal has been satisfied.
42
1 Table 17 Evidence for G1.3.3Axial Power Shapes The axial power shapes in the test assembly should reflect the G1.3.3 expected or limiting axial power shapes in the reactor assembly.
Level Evidence Only one axial power shape was used in the test assembly. However, 1 a justification for why the single axial power shape was sufficient is provided.
The commonly tested axial power shapes were used in the test 2 assembly. Further, an explanation of why those shapes were appropriate was provided.
A number of axial power shapes were used in the test assembly.
Further, it was demonstrated that the CBT model was able to make 3
accurate predictions of axial power shapes whose data were not used as training data for the model.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 2 and Level 3 have been most commonly accepted by the NRC staff. Generally, cosine, 4 up-skew, and down-skew power shapes are used for BWR fuel testing, and cosine and up-skew 5 (and maybe uniform) power shapes are used for PWR fuel testing. Level 1 has been used in the 6 past to confirm a models behavior on similar fuel or to make a small modification to an existing 7 model but not to qualify a new model. Level 3 is sometimes used as it is often easier to 8 demonstrate through test data that the CBT model is insensitive to axial power shape than to 9 provide other justification.
10 G1.3.4Radial Power Peaking 11 It is important to reproduce the local powers experienced by the reactor assembly in the test 12 assembly. Generally, this has been done by testing a combination of axial and radial power 13 shapes. Varying the radial power shape (i.e., radial power peaking) is generally much easier than 14 varying the axial power shape because it can be done by simply supplying more power to select 15 rods in the test assembly and does not necessitate replacing the rods in the assembly.
16 The importance of the radial power peaking is different for BWR and PWR testing. In PWR 17 testing, the radial power peaking tends to be used to ensure that the CBT occurs away from the 18 outside wall and near the central locations of the test assembly. Because the assembly is only a 19 portion (e.g., 5x5, 6x6) of the entire assembly (e.g., 14x14, 17x17), there is a desire to ensure that 20 the CBT occurs closer to the center of the test assembly and away from any edge effects of the 21 wall, as such a boundary does not exist in an open lattice core. The model predicting a CBT is 22 applied over every subchannel in a fuel assembly, and the resulting predicted CHF is compared to 23 the heat flux from the fuel rods. Although the radial power peaking will affect the heat flux from the 24 fuel rods and consequently the local fluid conditions, the computer code directly simulates all of 25 those impacts.
26 However, the radial peaking in BWR testing serves a different purpose as a result of how BWR 27 CP correlations are applied. In the current generation of CP correlations, assembly-average 28 thermal-hydraulic conditions and pin powers are used as inputs to the correlation. The margin to 29 dryout in the assembly is then calculated based on the limiting R- or K-factor. R- or K-factors are 43
1 calculated for each rod based on the pin power distribution of the surrounding rods and the rod 2 additive constant, which is a correlated parameter developed for each rod. Radial power peaking 3 in BWR testing is therefore used to drive different rods into dryout so an additive constant can be 4 determined for each individual rod (or its symmetric partners). This constant accounts for the local 5 thermal-hydraulic conditions in the fluid surrounding the rod in a way that is similar to the 6 subchannel code used in PWR CHF analysis. The testing should be performed over the full range 7 of R- and K-factors expected in the reactor so that the local thermal-hydraulic effects are properly 8 captured in the additive constant.
9 Because of this difference between PWR and BWR CBT modeling, the criteria for BWRs and 10 PWRs are different. Table 18 gives the evidence commonly provided to demonstrate that this 11 criterion (PWR only) has been satisfied.
12 Table 18 Evidence for G1.3.4Radial Power Peaking (PWR)
The radial power peaking in the test assembly should reflect the G1.3.4 expected or limiting radial powers in the reactor assembly.
Level Evidence Radial power distributions are consistent with those peaking factors 1
expected in reactor fuel.
Radial power distributions are higher than those peaking factors 2
expected in reactor fuel.
Radial power distributions in the test rods result in a hot subchannel 3 (i.e., a subchannel surrounded by peaked rods that have higher peaking factors than those normally expected in reactor fuel).
13 Historical Evidence Levels for Reactor Safety Analysis 14 Level 3 has been most commonly accepted by the NRC staff for PWR fuel. Generally, the hot 15 subchannels are designed toward the interior of the test assembly to ensure the CBT does not 16 occur on an exterior rod, which may be influenced by the channel wall.
17 18 Table 19 gives the evidence commonly provided to demonstrate that this criterion (BWR only) has 19 been satisfied.
44
1 Table 19 Evidence for G1.3.4Radial Power Peaking (BWR)
The radial power peaking in the test assembly should reflect the G1.3.4 expected or limiting radial powers in the reactor assembly.
Level Evidence 1 A wide range of radial power peaking was tested.
The testing procedure ensured that each rod experienced dryout in multiple tests over multiple different radial power distributions, thus 2 ensuring the thermal-hydraulic behavior captured in the R- or K-factor and any rod additive constant would be based on the appropriate rod behavior.
The testing procedure ensured that each rod experienced dryout in multiple tests over multiple different radial power distributions, thus ensuring the thermal-hydraulic behavior captured in the R- or K-factor 3 and any rod additive constant would be based on the appropriate rod behavior. Additionally, the radial power peaking tested bound the possible radial powers that could be observed during normal conditions and any transients.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 2 has been most commonly accepted by the NRC staff for BWR fuel. Generally, the tests 4 are focused on peaking each rod in the assembly to ensure a sufficient database for calculating 5 the additive constant. Often, every single rod in the assembly does not need to be peaked 6 because there is some flow symmetry; therefore, only some locations need to be investigated, 7 assuming the assembly behaves symmetrically. If the assembly does not behave symmetrically, 8 more rods in the assembly would need to be peaked to obtain measurements of their 9 performance.
10 G1.3.5Differences in the Test Assembly 11 The test assembly used in the experiment and the actual fuel assembly used in the reactor should 12 have few differences, if any. Because much of the important flow behavior of the assembly is not 13 modeled in the computer simulation but captured through the empirical CBT model, the test 14 assembly used to generate that model must be very similar to the actual fuel assembly. However, 15 the two assemblies will likely always have small differences that must be understood and 16 demonstrated to have little-to-no impact. Table 20 gives the evidence commonly provided to 17 demonstrate that this goal has been satisfied.
45
1 Table 20 Evidence for G1.3.5Differences in the Test Assembly Any differences between the test assembly and the reactor assembly should have a minimal impact on the flow field. This includes G1.3.5 components that are not in the reactor assembly but are needed for testing purposes.
Level Evidence The main flow features of the test assembly are the same as those of 1 the fuel assembly, with analysis demonstrating that all differences are small.
The main flow features of the test assembly are the same as those of 2 the fuel assembly, with experiment demonstrating that all differences are small.
The test assembly is identical to a symmetric portion (e.g., 5x5) of the 3
actual fuel assembly.
4 The test assembly is identical to the actual fuel assembly.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 3 has been most commonly accepted by the NRC staff for PWR fuel because of the 4 reduced fuel assembly size (a 17x17 reactor assembly is a 5x5 or 6x6 test assembly) and the use 5 of support spacers. Level 3 or level 4 is most common for BWRs because the entire assembly (or 6 a very large portion of it) can often be used in the test. Levels 1 and 2 are uncommon, as it is very 7 difficult to justify the use of a CBT model on fuel which is very different from that which was tested.
8 There are known issues that create deviations between the test assembly and the fuel assembly 9 used in the reactor. For example, in BWR testing, the part-length rods can sometimes prove 10 problematic; therefore, the test assembly may be very similar but not exactly identical to the actual 11 fuel assembly. Because these experiments are very costly and very difficult, differences between 12 the test and fuel assembly are not uncommon. In some past cases, data were discarded because 13 of such differences, and additional testing had to be conducted. In other cases, the differences 14 were small enough that the data were acceptable for use and additional testing was unnecessary.
15 Much is left to the experience and engineering judgment of the assessor and the analyst.
16 G2Model Generation 17 The statement The model has been generated in a logical fashion is intentionally broad because 18 the decision to rely on the model rests mostly on the validation data rather than its method of 19 generation. Additionally, a model could be generated in many ways, and any or every one of 20 those ways could be acceptable. Arguably, it would be possible to guess both the model form and 21 coefficients. If such a model were appropriately validated, showed reasonable physical behavior 22 over the range of its intended use, and had quantified uncertainty, there would be no reason to 23 disallow the use of that model, even though it was based on a guess.
24 Although any number of methods could be used to generate a CBT model, understanding what 25 method was used and the reasoning behind that method is helpful to the assessor. Therefore, the 26 criteria in this section are less focused on ensuring that a specific method was followed and more 27 focused on ensuring that whatever method was followed is explained and is logical.
46
1 The field of machine learning has addressed the general process used to generate a model (and 2 many of the concerns in that process). Therefore, many of the concepts and terms used in that 3 field will be used here. The two subgoals in Figure 8 are used to demonstrate that the model was 4 generated in a logical fashion.
5 6 Figure 8 Decomposition of G2Model Generation 7 3.2.1 G2.1The Mathematical Form 8 The mathematical form of the model must be appropriate in that all relevant parameters appear as 9 variables in the model and the model form itself is reasonable. Typically, the mathematical form of 10 the model is chosen based on an organizations past experience. The two subgoals in Figure 9 11 are used to demonstrate that the mathematical form of the model is appropriate.
12 13 Figure 9 Decomposition of G2.1The Mathematical Form 14 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 15 discuss the evidence that could be used to demonstrate that these two base goals have been 16 satisfied. Additionally, a discussion is provided on the evidence which has been historically used 17 for CBT models applied in reactor safety analysis.
47
1 G2.1.1Necessary Parameters 2 CHF models are typically represented as a function of several (5 to 10) parameters, where each 3 variable is generally based on a local parameter in the subchannel. The following are the most 4 common parameters:
5
- pressure 6
- local mass flux 7
- local quality 8
- inlet enthalpy 9
- heated hydraulic diameter (to account for any cold-wall effect) 10
- grid spacing 11
- other flow or geometry parameters 12 CP models are also represented by functions of several variables but typically not by local 13 parameters of the subchannel; instead, they are generally based on fuel assembly inlet 14 parameters, including the following:
15
- pressure 16
- inlet mass flux 17
- inlet subcooling 18
- R- or K-factor (related to local peaking) 19
- additive constant (related to the flow/enthalpy redistribution of a specific spacer design) 20
- other flow or geometry parameters 21 Pressure 22 Pressure can have a first-order impact on the fluid properties, the flow regime, and thus the 23 predicted CBT. Most AOOs occur at pressures close to the system pressure. The major exception 24 to this is the main steamline break in a PWR, which typically has the lowest pressure of any 25 AOO 18. Because the pressure encountered during a main steamline break is usually much lower 26 than the normal operating pressure, a specific low-pressure CHF model is often used.
27 Mass Flux 28 For PWRs, a local mass flux is used in the calculation of the CHF. This local mass flux is obtained 29 from a subchannel code because PWRs have an open lattice core, and significant mixing 30 between fuel assemblies can occur. For example, it is a common practice in PWR safety analyses 31 to conservatively model the flow entering the hot assembly by reducing it by a small percentage.
32 Because it is an open lattice core, the flow redistributes rather quickly, and this impact is almost 33 negligible after only a few grid spacers. However, at higher axial elevations, the hotter 34 subchannels will generate increased vapor, thus increasing the pressure drop and driving fluid to 35 other subchannels (and potentially into adjacent assemblies). Because the local mass flux 36 calculated by the subchannel code can have a first-order effect on the prediction of the CBT, the 37 code (and all of the selected modeling options) is considered part of the CHF model. Any change 18 While a main steam line break is formally classified as an accident and not an AOO, many plants analyze them to the stricter standard of an AOO. Limited fuel failure is permitted in a postulated accidents where no fuel failure is permitted in an AOO.
48
1 to the code or selection of any different modeling options would warrant revalidation of the CHF 2 model with the new code or modeling options.
3 For BWRs, the local mass flux is typically not necessary because the fuel assembly is bounded by 4 its channel, and mixing between assemblies does not occur. Therefore, CBT can be correlated to 5 the inlet mass flux. Although a mass exchange occurs between the vapor flow, the liquid droplets, 6 and the fluid film, this exchange is modeled through the calculation of the quality, and the CP 7 model itself captures the entire process.
8 Local Quality 9 For PWRs, the local quality has a first-order impact on the CBT. One thing which seems to have a 10 large impact on the local quality is the power shape. Tongs factor (or similar shape factors) 11 accounts for different axial power shapes by reducing (or increasing) the heat flux that is needed 12 to predict a CHF. Tongs factor is supposed to account for the history of the flow that would be 13 affected by axial power shape. One theory is that the Tong factor accounts for the radial distance 14 between the heated wall, the void location in the flow, and the void concentration. Although the 15 quality calculated is the total quality of the subchannel, it is quality near the wall that would likely 16 have the largest impact on CHF. Thus, a shape factor like Tongs is used to account for this 17 quality distribution in a specific cell of the subchannel. Voids closer to the wall may result in a 18 lower CHF than would voids in the center of the channel because voids at the wall could influence 19 bubble crowding and hence influence the CHF.
20 For BWRs, the local quality is more of a predictive parameter than a correlating parameter. Many 21 CP models correlate the current boiling length to a critical quality. In such a model, ensuring that 22 CP has not occurred is synonymous with ensuring that the current quality is lower than the critical 23 quality.
24 Inlet Enthalpy 25 The inlet enthalpy is used to determine how close the inlet flow conditions are to boiling (e.g., inlet 26 subcooling). If the inlet subcooling is high, boiling will generally occur at higher axial elevations in 27 the fuel assembly, and a higher power will be needed to cause a CBT. Although inlet subcooling 28 can be low, some amount of inlet subcooling is typically necessary or else the start of boiling can 29 occur outside of the fuel assembly and it is not possible to define a boiling length. Models that 30 correlate boiling length to a critical quality inherently assume that the entire boiling length will be in 31 the fuel assembly, which would therefore typically imply that the flow enters the assembly with 32 some subcooling. Even if this assumption is not used, it is usually very difficult to test conditions 33 with zero or negative inlet subcooling (i.e., flow is already boiling).
34 Inlet subcooling is not as relevant for CHF models as they focus more on local conditions. More 35 generally, PWRs operate with inlet conditions that are much farther from saturation (i.e., more 36 subcooled) than BWRs. However, experimental validation should be used to confirm that the flow 37 at the inlet is subcooled if necessary.
38 Heated Hydraulic Diameter 39 Typically, for CHF models, the subchannel heated hydraulic diameter (or a ratio of the heated 40 hydraulic diameter to the true hydraulic diameter) is used instead of the actual hydraulic diameter 41 because of the difference between the behavior of a subchannel surrounded by four rods and the 42 behavior of a subchannel that contains an unheated guide tube. The guide tube is considered a 49
1 cold wall; therefore, its impact is known as the cold-wall effect. Although a guide tube may 2 change the hydraulic diameter of a subchannel, some guide tubes are of similar size to a fuel rod 3 and, therefore, would have minimal impact on the hydraulic diameter of the channel. However, 4 because the guide tube is unheated, it would have a large impact on the heated hydraulic 5 diameter.
6 Although it is important to explicitly account for the cold-wall effect in PWRs, it is not directly 7 addressed in BWRs. Generally, the K- or R-factor and the additive constants would account for 8 any impact from the water rods or channel box in a BWR.
9 Grid Spacing 10 If the grid spacing (i.e., the distance between two grids) does not vary for a fuel design, obtaining 11 test data at multiple grid spacings is not necessary. However, if the grid spacing can change 12 (e.g., intermediate flow mixers are positioned between some spacer grids), the effect of the 13 distance between all possible combinations of the grids should be accounted for the CBT model.
14 Typically, CBT occurs just upstream of (i.e., below) a grid spacer. For PWRs, the turbulence is 15 maximized just downstream of (i.e., above) a grid and decreases as the fluid travels further from 16 the grid, reaching a minimum just upstream of the next grid. Therefore, longer spans between 17 grids result in more reduction in turbulence and less mixing, thus increasing the potential for a 18 CBT. For BWRs, the grids direct the droplets entrained in the vapor core to the liquid film on the 19 fuel rod, thus increasing the liquid film thickness. However, as the flow moves downstream from 20 the grid, the additional deposition caused by the grid decreases and the liquid film evaporates and 21 is entrained by the vapor flow. If the deposition rate falls off too quickly or if evaporation or 22 entrainment is too great, the film may dry out before it reaches the next grid where deposition will 23 increase once again.
24 Additionally, the grids themselves act as fins. Thus, while a CBT would be expected to occur just 25 upstream of a grid, it would be highly unlikely to occur inside a grid because some amount of heat 26 transfer occurs from the rod to the grid and the grid to the coolant. Additionally, the grids 27 themselves are often covered in water, either from the continuous flow field in a PWR or from 28 droplets in a BWR.
29 R- or K-Factor and Additive Constants (BWR only) 30 The R- or K-factors and additive constants account for the impacts of various phenomena on CP 31 predictions for each rod position. The additive constants are terms that account for the increase or 32 decrease in mixing at some xy location in the grid assembly. These terms are obtained from 33 experimental testing and generally stay fixed for a particular rod xy location. The R- or K-factors 34 include the impact of the various power levels of the surrounding rods on the rod in question.
35 These factors and constants have been colloquially termed the poor mans subchannel code.
36 Instead of simulating a large number of subchannels in the hot assembly, a BWR analysis will 37 simulate only a single rod surrounded by a single subchannel at assembly-averaged conditions.
38 The R- or K-factors are then used, along with the additive constants, to determine the behavior of 39 the rods at each xy location in the assembly.
40 Other Parameters 41 CBT models may use other parameters. Historically, the heated length has been used, but recent 42 work suggests that this is not the best length parameter to correlate against because the boiling 50
1 length (i.e., distance from the start of boiling to the current location under consideration) has a 2 larger impact on the CBT (Wieckhorst et al., 2013; Wieckhorst et al., 2015).
3 Table 21 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
4 Table 21 Evidence for G2.1.1Necessary Parameters The mathematical form of the model contains all the necessary G2.1.1 parameters.
Level Evidence 1 The model contains all the parameters measured in the experiment.
The model parameters include those which have been commonly used 2 in previous models and are considered to be the parameters that have the most significant impacts on a CBT.
It is demonstrated from first principles that the model contains all the 3
necessary parameters.
5 Historical Evidence Levels for Reactor Safety Analysis 6 Level 2 has been most commonly accepted by the NRC staff. Typically, the CBT model includes a 7 few parameters in addition to those measured in the experiment. Level 3 is considered an ideal 8 situation, and the authors are not aware of a complete first-principle understanding of phenomena 9 associated with a CBT. This is especially true for DNB, for which the phenomenon involved is 10 much more complex than dryout because it involves multiple length scales and a strong 11 dependence on turbulence. It is possible that a claim of thorough understanding of the first 12 principles of CBT could be demonstrated by developing a correlation using very little training data 13 and validating it against a wide variety of conditions.
14 G2.1.2Reasoning for the Mathematical Form 15 Currently, there is no known best mathematical form for CBT models, which are expressed as 16 multivariate functions because a complete first-principle understanding of the underlying 17 phenomena does not exist. Additionally, because of nonlinear behavior, it may be difficult to 18 separate the impact of the chosen mathematical form and the impact of the chosen values for 19 coefficients. Thus, even identical mathematical forms can behave much differently with different 20 choices of coefficients. Although there is no single correct way to generate the mathematical 21 form, the method behind generation of the form should be described to ensure that it is 22 reasonable to the assessor. Additionally, because the validation process will quantify the models 23 uncertainty, this criterion focuses on understanding how the mathematical form was generated 24 rather than on ensuring that it was generated in a particular manner. Table 22 gives the evidence 25 commonly provided to demonstrate that this goal has been satisfied.
51
1 Table 22 Evidence for G2.1.2Reasoning for the Mathematical Form The reasoning for choosing the mathematical form of the model should G2.1.2 be discussed and should be logical.
Level Evidence 1 The basis of the models mathematical form is described.
The basis of the models mathematical form is described. The 2 description includes the development of the form and justification of the essential elements of the form.
A very thorough description of the origins of the mathematical form of the model is provided. This description includes the history of the form, 3
the justifications for using the form, and the process for generating the form.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 2 has been most commonly accepted by the NRC staff. In many cases, the development of 4 the mathematical model has occurred over the course of many years and has been influenced by 5 numerous factors. Although it is helpful for the assessor to understand this history, and it has 6 previously increased the review efficiency, it is not strictly necessary. Thus, Level 3 and Level 1 7 are not uncommon.
8 In general, as long as the model has been validated with data that covers its expected range of 9 use, contains all the necessary parameters, and has a logical form, then the specific form of the 10 model would have a minor impact on model predictions. A model with a logical form will generate 11 relevant predictions over the entire application domain. Trends between data points should be 12 reasonable in that the model should not be discontinuous and the trends should be well-behaved 13 mathematically. Because there are a large variety of mathematical forms that could be chosen, 14 the specific form should not result in unreasonable predictions (e.g., very high, very low, negative, 15 complex numbers) inside the expected domain.
52
1 3.2.2 G2.2Method for Determining Coefficients 2 The process for determining the values of the models coefficients should be appropriate. Again, 3 the meaning of appropriate in terms of a models coefficients is vague. Although only a single set 4 of the coefficients would result in the lowest error, as judged by some norm (e.g., the Euclidian 5 norm), minimizing this error is often not the most important criterion when determining the 6 coefficient values. Instead, great care is usually taken to ensure that the model reflects actual 7 physical behavior rather than simply minimizing the error. Thus, many of the coefficients for a 8 model are chosen to ensure that the model has certain desired trends. The three subgoals in 9 Figure 10 are used to demonstrate that the method for determining the coefficients is appropriate.
10 11 12 13 Figure 10 Decomposition of G2.2Method for Determining Coefficients 14 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 15 discuss the evidence which could be used to demonstrate that these three base goals have been 16 satisfied. Additionally, a discussion is provided on the evidence that has been historically used for 17 CBT models applied in reactor safety analysis.
18 G2.2.1Identification of Training Data 19 The training data are the experimental data used to generate the coefficients of the model. They 20 are distinguished from the validation data, which are the experimental data that are used in the 21 validation process. Ideally, different data should be used for each role. Typically, some large 22 percentage (usually between 70 and 100 percent) of the experimental data will be used as training 23 data. Table 23 gives the evidence commonly provided to demonstrate that this goal has been 24 satisfied.
53
1 Table 23 Evidence for G2.2.1Identification of Training Data The training data (i.e., the data used to generate the coefficients of the G2.2.1 model) should be identified.
Level Evidence 1 100% of the experimental data are used as training data.
2 Between 90-100% of the experimental data are used as training data.
3 Between 80-90% of the experimental data are used as training data.
4 Between 70-80% of the experimental data are used as training data.
5 Between 60-70% of the experimental data are used as training data.
6 Between 50-60% of the experimental data are used as training data.
7 Between 40-50% of the experimental data are used as training data.
8 Between 30-40% of the experimental data are used as training data.
9 Between 20-30% of the experimental data are used as training data.
10 Between 10-20% of the experimental data are used as training data.
11 Between 0-10% of the experimental data are used as training data.
12 None of the experimental data are used as training data.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Levels 1-3 have been most commonly accepted by the NRC staff. As there is no minimum or 4 maximum portion of the data that should be used to train the model, this criterion focuses more on 5 identifying what data are used to train the model rather than on ensuring that a certain amount is 6 (or is not) training data.
7 In general, all experimental data should be either training or validation data. Thus, if 70 percent of 8 the data are training data, the remaining 30 percent could be used as validation data. Section 3.3 9 discusses the criteria on the amount of validation data. However, one way to demonstrate the 10 power of a specific model is to have a very small percentage of training data and a very large 11 percentage of validation data.
12 G2.2.2Calculation of the Models Coefficients 13 Again, there is typically no single best way to calculate the models coefficients. For PWRs, 14 because of the simplicity of the CHF model, the focus is typically on reducing overall error.
15 However, the models for BWRs are generally more complex; therefore, the focus is typically on 16 ensuring that the model has the desired behavior as a function of certain parameters. Whichever 17 method is used, the assessors should understand that method. Table 24 gives the evidence 18 commonly provided to demonstrate that this goal has been satisfied.
54
1 Table 24 Evidence for G2.2.2Calculation of the Models Coefficients The method for calculating the models coefficients should be G2.2.2 described.
Level Evidence A brief description of the method for calculating the models coefficients 1
is provided.
A detailed description of the method for calculating the models 2
coefficients is provided.
A very thorough description of the method for calculating the models coefficients is provided. This includes the walkthrough for gathering the 3
experimental data, the data reduction process, and the methods used to generate the coefficients.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 2 has been most commonly accepted by the NRC staff. The method for calculating the 4 models coefficients tends to be very detailed. The models are treated as strictly data driven 5 models (i.e., empirical or semi-empirical) in that there is no assumption that the model form 6 contains any ability to predict the physics besides that which it demonstrates through its validation.
7 While it is possible that the model form may be based on equations from first-principle physics, it 8 is not assumed that the model contains any inherent ability to predict the underlying physical 9 mechanisms of the CBT. Therefore, there is no best practice in terms of the manner in which the 10 models coefficients are calculated.
11 Because the models uncertainty will be quantified with validation data, choosing the models 12 coefficients is mostly focused on reducing the models uncertainty. In the extreme case, the 13 models coefficients could be guessed and, as long as the models uncertainty is quantified, the 14 model would still be acceptable for use (all non-linear regressions require a guess of the model 15 coefficients as a starting point). Further, it is common for the model coefficients to be chosen to 16 ensure some known behavior over specific ranges of the model, and not simply to ensure the 17 smallest validation error.
18 G2.2.3Calculation of Model-Specific Factors and Constants (BWR Only) 19 The R- or K-factor and additive constants are part of the coefficients of the model itself. However, 20 they are often treated separately from the calculation of other coefficients in the model. They are a 21 very important part of BWR simulations because they allow local fuel rod behavior to be modeled 22 without using detailed local conditions, so their generation should be well understood. Table 25 23 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
55
1 Table 25 Evidence for G2.2.3Calculation of Model-Specific Factors and 2 Constants The method for calculating the R- or K-factor and the additive constants (for both full-length and part-length rods) should be described. Further, a G2.2.3 description of how such values are calculated if dryout is not measured on the rod under consideration should be provided (BWRs only).
Level Evidence 1 A brief description of the method for calculating these values is provided.
A detailed description of the method for calculating these values is 2
provided.
A very thorough description of the method for calculating these values is provided. This includes a walkthrough for gathering the experimental 3
data, the data reduction process, and the methods used to generate these values.
3 Historical Evidence Levels for Reactor Safety Analysis 4 Level 2 has been most commonly accepted by the NRC staff. The method for calculating the R- or 5 K-factor and additive constants tends to be very detailed. It is important for the assessor to 6 understand the process so that he or she can confirm that the behavior modeled in the R- or K-7 factors and additive constants would result in a reasonable prediction of CBT in a BWR.
8 G3Validation through Error Quantification 9 Validation is the accumulation of evidence used to assess the claim that a model can predict a 10 physical quantity (Oberkampf and Roy, 2010). Thus, validation is a never-ending process 11 because more evidence can always be obtained to bolster this claim. However, at some point, 12 when the accumulation of evidence is considered sufficient to make the judgment that the model 13 can be trusted for its given purpose, the model is said to be validated. This is not to say that 14 further validation would not be useful but rather that it is believed that the validation currently 15 provided demonstrates that the model can be trusted for its specific use. The authors believe that 16 Anderson and Bates were very wise to begin the first chapter of their book on validation 17 (Anderson and Bates, 2001) with a quote from the National Research Council: Absolute validity 18 of a model is never determined (National Research Council, 1990).
19 Because of the desire to ensure that the models prediction is conservative, any bias or 20 uncertainty, or both, in the models prediction of CHF or CP should be adequately quantified such 21 that safety analyses can account for it. This process is uncertainty quantification. The first step in 22 this process is to use the experimental data (i.e., the validation data) along with the models 23 prediction of that experimental data to calculate the validation error. If the validation error is 24 appropriately distributed through the models application domain and if any inconsistencies in the 25 validation error are accounted for, statistics from the validation error can be used to determine the 26 models uncertainty. The five subgoals in Figure 11 are used to demonstrate that the model has 27 sufficient validation through the quantification of its error.
56
1 2 Figure 11 Decomposition of G3Validation through Error Quantification 3
4 3.3.1 G3.1Calculating Validation Error 5 Typically, model error is thought of as the difference between the actual value that occurs in 6 nature and the predicted value of the model. If the model is simple enough or if the experiment is 7 complex enough, the measured value from the experiment can be used as the actual value. 19 8 Ideally, the error could be calculated from the measured value of the instrumentation and the 9 models prediction under the same conditions. However, this is often oversimplification. Instead, 10 the error of interest should not be the model error but the model application error (i.e., what is the 11 error of using the model in the same manner as it will be applied in the safety analysis).
12 To clarify, one way to calculate the model error is to measure the heat flux or power at the location 13 of a CBT and consider this the measured value and then use the CBT model along with the flow 14 conditions at the time of the CBT to obtain a predicted value at that same location. However, a 15 CBT model is generally not applied in this manner. First, it is typical for multiple rods to experience 16 a CBT at the same time. Second, it is typical for the same rod to experience a CBT at different 17 elevations at the same time. Third, the definition for a rod experiencing a CBT is somewhat 18 variable. Generally, the criteria for determining the occurrence of a CBT is some specified change 19 in temperature over a short time span. During testing, a number of thermocouples may register a 20 change just under this amount; therefore, the rods are not considered to have experienced a CBT.
21 However, under this definition it is possible that a CBT may still have occurred. These challenges 22 could make determination of a single measured value from an experiment very difficult.
23 Additionally, the objective is not to ensure that the CBT model can be trusted for predicting the 24 behavior of an experiment for which the heat flux or power that causes a CBT are known; instead, 25 the objective is to determine whether the model can be trusted when applied in a reactor safety 26 analysis where the heat flux or power will be unknown. The interest is not in the model error but in 19 This statement ignores any differences between the measured value of a quantity and the actual value of that quantity; the discussion on instrumentation uncertainties addresses these differences.
57
1 the model application error. For this reason, the measured and the predicted values should be 2 related to how a reactor safety analysis applies the model.
3 For example, the focus of PWR safety analysis is to determine which of the subchannels have the 4 MDNBR value because this subchannel would be the closest one to experiencing a CBT. Thus, 5 when a transient is simulated, the MDNBR is obtained, and if that value is greater than some 6 safety limit, CBT is precluded. This method of analysis differs from the experiment in two main 7 ways. First, the experiment determines which rod experienced a CBT, but the simulation 8 determines which subchannel has the MDNBR. Second, because of how a CBT is defined in the 9 experiment, it is common for more than one rod and even more than one location on the same rod 10 to register as having experienced a CBT, but the simulation produces only one MDNBR value.
11 Because the model is applied using the MDNBR, the measured and predicted values should be 12 related to the DNBR value.
13 The term validation error was chosen to represent the error of interest for two main reasons. The 14 first reason is to distinguish it from the model error, which is commonly thought of as a difference 15 between the models prediction and a measurement. Determining the measured and predicted 16 values is not as straightforward as many may consider. The second reason is that this error could 17 have been called a model application error, but that term was not chosen for a different reason.
18 The model application error is defined as the total population of error of the possible uses of the 19 model inside the expected domain. If an experimental measurement of CBT could be obtained at 20 every point in the expected domain (i.e., an infinite number of points), than that infinite set would 21 be the actual model application error. The validation error is a sample from the model application 22 error population. The validation error is based on the validation data, which only exist at a finite 23 number of points in the expected domain. This distinction is important because one of the key 24 assumptions is that the validation error is a representative sample of the model application error.
25 Generally, the validation error for a model is either represented as an absolute error 26 (i.e., measured - predicted), or as a relative error (e.g., (measured - predicted)/measured). CBT 27 models in particular use a form of the relative errormeasured/predicted is commonly used for 28 PWR validation and predicted/measured is commonly used for BWR validation. Thus, for PWRs, 29 values that are below 1 are non-conservative (i.e., a CBT occurred at heat fluxes below the 30 models prediction), and values that are above 1 are conservative (i.e., a CBT occurred at heat 31 fluxes above the models prediction). Conversely, for BWRs, values that are below 1 are 32 conservative (i.e., a CBT occurred at powers above the models prediction), and values that are 33 above 1 are non-conservative (i.e., a CBT occurred at powers below the models prediction).
34 Table 26 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
58
1 Table 26 Evidence for G3.1Calculating Validation Error G3.1 The correct validation error has been calculated.
Level Evidence 1 The validation error is a sample for the population of the model error.
The validation error is a sample for the population of the model 2
application error.
The model is applied such that the populations of the model error and 3 model application error are identical. The validation error is a sample from this population.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 2 has been most commonly accepted by the NRC staff. While CBT models are often 4 considered as stand-alone models, they are used as part of larger thermal-hydraulic 5 methodologies. Thus, the error in a CBT model is typically quantified as if it is being used inside 6 the larger methodology (level 2) rather than used as a standalone model (level 1). Level 3 would 7 be ideal as it would mean that model can be treated as a standalone equation.
8 3.3.2 G3.2Data Distribution in the Application Domain 9 The validation error data points should be appropriately distributed throughout the application 10 domain. Consider each of the N input variables used by the model as a dimension (e.g., pressure, 11 mass flux, inlet subcooling). The set of all inputs could be used to generate an N-dimensional 12 application space, and the application domain is the domain in this space over which the model 13 could be applied to predict CHF or CP. Typically, the application domain is defined as an 14 n-orthotope which is a two-dimensional (2-D) rectangle, a three-dimensional (3-D) box, or a 15 hyper-rectangle in dimensions greater than 3-D. This shape, the generalization of a rectangle to 16 higher dimensions, is a simplification of the true shape of the application domain and is used 17 because it can be easily defined by N inequalities (corresponding to the number of dimensions in 18 the application space). Using this shape allows a computer program to easily determine whether 19 the current location in the application space is inside or outside of the application domain. For 20 example, the boundaries on the pressure are typically given as follows:
21 (2) 22 To ensure the model should be used to make a prediction, the computer code will check to ensure 23 that the current pressure is between the minimum and maximum pressure of the application 24 domain.
25 Defining the application domain as a set of independent inequalities is computationally 26 convenient, but the model may not be valid over that entire domain. Consider the following 27 simplified 2-D domain. Six types of subregions can be defined within the 2-D application space, 28 depending on their proximity to validation error data points and their position relative to the 29 application domain. These six types of subregions, shown in Figure 12, would also exist in 30 application spaces of higher dimensions.
59
1 2 Figure 12 Regions in the Application Domain 3 Region 1Well Covered 4 The first type of region is any region in the application domain that both contains data and is 5 surrounded by data. In this region, the data are not sparse, and the region would be considered 6 well covered. Although it is tempting to believe that the entire application domain is well 7 covered, this is only the ideal and is generally not true in practice.
8 Region 2Localized Hole 9 The second type of region is any region in the application domain that contains little to no data but 10 is surrounded by data and thus forms a hole. As the number of dimensions of the application 11 domain increases (i.e., Figure 1 shows a 2-D application domain, but it is common to have 12 domains of six or more dimensions), it is not always clear whether the use of the model in such a 13 region should be considered interpolation or extrapolation. In either circumstance, as long as the 14 region itself is not too big, the use of the model in such regions is generally accepted as justified.
15 Note that there will always be a hole between data points because the space is continuous, and 16 the data exist only at discrete points. However, the assessor must exercise judgment about how 17 far apart data are to constitute a localized hole.
18 Region 3Edge 19 The third type of region is any region in the application domain that contains little to no data and is 20 only partially surrounded by data and, therefore, is at an edge. Although uses of the model near 21 the bulk of the data would seem reasonable, at some point the region of interest becomes 22 sufficiently distant from the validation error data that the model cannot be considered validated 23 and should not be used in the absence of other justification.
60
1 Region 4Isolated Known Unknown 2 The fourth type of region is any region in the application domain that contains no data and is 3 somewhat far from any region that does contain data; however, it is a region over which the model 4 can be justified. For example, one common, conservative modeling assumption is to construct 5 CBT models such that the predicted CHF or CP will be 0 at a mass flux of 0. In reality, as the 6 mass flux goes to 0, the predicted CHF will go to a pool-boiling CHF value, which is much higher 7 than 0. Thus, while the region may not have data, the use of the model in the region would be 8 known to be very conservative.
9 Region 5Isolated Region 10 The fifth type of region is any region in the application domain that contains no data and is far from 11 any region that does contain data. In other words, it is an isolated region. Moreover, it is an 12 isolated region in which the models behavior is unknown. The application domain likely only 13 includes such a region because the choice to represent the domain was a rectangle. The use of 14 the model in such regions of the application domain should be precluded, but that would only be 15 accomplished by defining a more complicated shape for the application domain. In 2-D, this could 16 be easily done. However, many real models are in six or even more dimensions, and the 17 representation of complex shapes in multiple dimensions is very difficult. Although the application 18 domain will likely always be defined as a hyper-rectangle, the domain where the analyst expects 20 19 to use the model is actually closer to a hyper-jelly bean (as described by one engineer).
20 This is a concern with any higher dimensional model and is one reason why the application 21 domain needs to contain validation error data that span the expected range of use. Although 22 isolated unknown regions are always possible, the best way to ensure that the model will never be 23 used in such a region is to ensure that all conceivable regions where the model will actually be 24 used have data.
25 It is important to realize that the models prediction in an isolated unknown region is suspect. On 26 the one hand, the model may have been developed such that it happens to provide reasonable 27 estimates of the CBT in that region. On the other hand, it may predict something completely 28 unphysical in that region such as a negative heat flux or negative power. Model developers tend 29 to understand where these regions exist, and apply the models only in the regions that contain 30 data. However, a new user who is unfamiliar with the models development process could easily 31 pick up the model, start using it, and wonder why it is making very strange predictions in certain 32 regions.
33 Region 6Outside Region 34 The sixth type of region is any region that is outside the application domain. The computer code 35 using the model will flag the use of the model as improper only in these outside regions.
36 Considering these regions, the six subgoals in Figure 13 are used to demonstrate that the 37 validation error data were appropriately distributed throughout the application domain.
38 20 Hence, this document separates the two domains. The Application Domain is the domain over which the model is applied, and is an n-dimensional rectangle. However, the Expected Domain is where the analysts would expect the model to be used, and is subset of the application domain, but generally a much more complex shape that cannot easily be well defined.
61
1 2
3 Figure 13 Decomposition of G3.2Data Distribution in the Application Domain 4 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 5 discuss the evidence that could be used to demonstrate that these six base goals have been 6 satisfied. Additionally, a discussion is provided on the evidence that has been historically used for 7 CBT models applied in reactor safety analysis.
8 G3.2.1Identification of Validation Data 9 The validation data are the experimentally measured values that are used to quantify the models 10 error. Ideally, these data should be independent from the training data. The model will be used to 11 make predictions about the CBT throughout the application domain. The focus of validation is to 12 quantify the error of those predictions. Although it may seem that use of the training data would be 13 appropriate, the model has already been tuned to that data. Thus, quantifying the error of the 14 training data would provide an estimate of how well the model can predict data that were used to 15 generate the model. This is different from how well the model can predict data that were not 16 used to generate the model. Because substantially more data points appear in the application 17 domain (an infinite number) than were used to generate the model and because these points are 18 the ones of most interest in future uses of the model, the focus should be on generating an 19 estimate of the error over those points which were not used to generate the model. Thus, 20 experimental data that have not been used to train the model should be held in reserve and used 21 only to validate the model because the models behavior using these data are indicative of the 22 type of predictions that will be made in its future uses.
23 However, in many instances, the validation data and the training data are one and the same.
24 There are methods in machine learning that can be applied to determine whether the selection of 25 the training data affects the resulting uncertainty, such as random subsamples and k-folds. In 26 each of these methods, the data are randomly separated into subsets of training and validation 27 data. The training data are used to develop the coefficients of the model, and the validation data 28 are used to determine the overall uncertainty of the model. Then, the process is repeated with a 29 different randomly-selected data set assigned to training and the remaining data assigned to 30 validation. Processes like these can provide reasonable estimates of the impact of using the same 62
1 training data as validation data. Even for well-formed models, using the same dataset for training 2 and validation can increase uncertainty by 2 to 3 percent. This increase is small but far from 3 negligible, and it may be higher or lower depending on the circumstances. Table 27 gives the 4 evidence commonly provided to demonstrate that this goal has been satisfied.
5 Table 27 Evidence for G3.2.1Identification of Validation Data The validation data (i.e., the data used to quantify the models error)
G3.2.1 should be identified.
Level Evidence Validation data have been identified, and they are the same as the 1
training data.
Validation data have been identified, and they are the same as the 2 training data. To quantify this impact, a method such as k-folds or random subsamples has been used.
3 The validation data are independent from the training data.
6 Historical Evidence Levels for Reactor Safety Analysis 7 Level 3 (specifically, a 70/30 or 80/20 split between the training and validation data) has been 8 most commonly accepted by the NRC staff. In a sense, this is similar to performing a single 9 k-folds calculation or a single random calculation of subsamples. Level 2 has been used in the 10 past, but the models error in predicting data that were not used to generate the model will almost 11 always be greater than its error in predicting data that were used to generate the model.
12 Therefore, using the same data for training data as validation data often involves additional work, 13 such as k-folds or random subsamples. If only Level 1 is achieved, a bias may need to be added 14 to account for the fact that the resulting error is likely lower than actually expected.
15 G3.2.2Defining the Application Domain 16 The application domain should be defined such that the computer code applying the model is able 17 to determine whether the model should be used for a given set of input parameters. Generally, 18 this is done using inequalities such as those given in the following expressions:
(3)
(4) 19 Defining the application domain in such a manner results in a hyper-rectangle, which contains 20 many regions in which no data exist. Although a more accurate method of defining the application 21 domain could be used to only specify the region which contains data, such alternative methods do 22 not currently exist. Table 28 gives the evidence commonly provided to demonstrate that this goal 23 has been satisfied.
63
1 Table 28 Evidence for G3.2.2Defining the Application Domain G3.2.2 The application domain of the model should be mathematically defined.
Level Evidence The application domain has been mathematically defined as a 1
hyper-rectangle.
The application domain has been mathematically defined as a shape 2
other than a hyper-rectangle to better capture its true shape.
The application domain has been mathematically defined in terms of a 3
maximum allowable distance from validation error data.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 1 has been most commonly accepted by the NRC staff. Because application domains 4 defined as hyper-rectangles often contain many regions which are technically part of the 5 application domain, but contain no data and are far from where the plant operates, there is 6 generally a desire for the analyst to not only specify the application domain, but also to 7 understand the expected domain.
8 G3.2.3Understanding the Expected Domain 9 The application domain is defined as the domain in the N-dimensional input space over which the 10 model could be applied. However, that domain is different from the domain over which the model 11 is expected to be applied. The expected domain is defined as the domain in the N-dimensional 12 input space over which the model will likely be applied because it corresponds to state points that 13 occur during normal operation or AOOs. Unlike the application domain, which is mathematically 14 defined so that a computer can determine whether the model is being used outside of that 15 domain, the expected domain is generally not formally defined due to its complex shape. For 16 example, if the application domain is represented as a box in a series of 2-D plots of one input 17 parameter versus another input parameter, the expected domain would be represented by some 18 region in each box.
19 Ideally, as knowledge progresses, the application domain would become closer to the expected 20 domain, and both domains would contain only regions with data. Table 29 gives the evidence 21 commonly provided to demonstrate that this goal has been satisfied.
64
1 Table 29 Evidence for G3.2.3Understanding the Expected Domain G3.2.3 The expected domain of the model should be understood.
Level Evidence 1 Each parameter in the model is considered separately.
2-D plots (parameter versus parameter) that contain the locations of the validation error data and the expected range of those parameters 2 during normal operation and AOOs are provided. The expected ranges N(N1) are well covered by validation error data (N parameters = plots).
2 Another method that considers more than two parameters at a time 3
(e.g., 3-D plots) is used.
4 A method that considers all N parameters at the same time is used.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 2 has been most commonly accepted by the NRC staff. Although used in the past, Level 1 4 completely ignores any correlations between the input parameters themselves. For example, if 5 there is only low-pressure data at low mass flows and high-pressure data at high mass flows, the 6 models prediction in the region that has both low-pressure and high mass flow would not have 7 any associated validation error data. However, to determine whether this situation exists, the data 8 should be plotted with more than one input parameter at once (i.e., at least a 2-D plot).
9 Just as Level 1 reduces the problem to a single dimension in the N-dimensional input space, 10 Level 2 reduces the problem to two input dimensions. Both dimensional reductions cause the loss 11 of information, but the information loss caused by reductions from N dimensions to 2-D is believed 12 to be less significant. Ideally, all N dimensions could be considered at the same time, but the 13 authors are not currently aware of a method for doing so.
14 G3.2.4Validation Error Data Density in the Expected Domain 15 The expected domain should have adequate data density to ensure adequate coverage for future 16 uses of the model. Typically, the regions with the most data will be those regions in which the 17 plant will be close to normal operating conditions. However, these regions are not necessarily the 18 same as the regions in which the plant would be closest to experiencing a CBT. Thus, although 19 certain regions are expected to have a very high density of validation error data, the entire 20 expected domain should be well covered. Note that the entire application domain will likely not be 21 covered, due to the practice of representing the application domain as a hyper-rectangle. While it 22 is not necessary for the entire application domain to be well covered with validation data, it is 23 necessary for the expected domain to be well covered. Table 30 gives the evidence commonly 24 provided to demonstrate that this goal has been satisfied.
65
1 Table 30 Evidence for G3.2.4Validation Error Data Density in the Expected Domain There should be adequate validation error data density throughout the G3.2.4 expected and application domains.
Level Evidence Each input parameter is considered independently from all others.
Few regions have sparse data, and the models use in those regions 1 can be justified.
Thus, the problem is treated as N number of 1-D spaces.
A set of two input parameters are considered in combination. The data density is sufficient, only a few regions of sparse data exist, and the models use in those regions can be justified. All possible combinations 2 of sets of two input parameters are considered.
Thus, the problem is treated as 2 number of 2-D spaces.
A set of three input parameters are considered in combination. The data density is sufficient, only a few regions of sparse data exist, and the models use in those regions can be justified. All possible 3 combinations of sets of input parameters are considered.
Thus, the problem is treated as 3 number of 3-D spaces.
All input parameters are considered in combination. The data density is sufficient, only a few regions of sparse data exist, and the models use 4 in those regions can be justified.
Thus, the problem is treated as a single N-D space.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 2 has been most commonly accepted by the NRC staff. Again, Level 1 is considered 4 insufficient because it ignores any correlations between the input parameters themselves. Level 3 5 would require the use of 3D plots, and such plots are difficult to represent in a 2D document (i.e.,
6 on a sheet of paper). For this reason, assessors have previously found it important to obtain the 7 data used to correlate and validate the modelthis data can be used to generate 3-D plots, which 8 can be examined in detail on a computer. In addition, the number of dimensions observed at the 9 same time can be increased to four dimensions by using a color gradient on the points. Higher 10 dimensional plots are possible, but understanding such plots as the number of dimensions grows 11 becomes difficult.
12 As of yet, there are no precise limits on data density. Even the concept of data density is difficult 13 to define precisely, as the volume over which the data density would be determined contains 14 dimensions that cannot be easily combined in a meaningful way. Therefore, the density in each 15 region is generally judged to be sufficient if it is similar to previous densities from past approved 16 models. It is expected that there will be a large cluster of points around the normal operating 17 conditions and fewer points at the extremes of the expected domain.
66
1 Finally, levels 1-3 are graphical methods that rely on qualitative judgment. Level 4 considers some 2 method that is quantitative, but the authors are not aware of any such method that currently 3 exists.
4 G3.2.5Sparse Regions 5 As discussed above, there may be sparse regions in the application domain for a variety of 6 reasons. Usually, sparse regions appear in the application domain because of the method chosen 7 to describe the domain (e.g., as a hyper-rectangle). However, these regions in the application 8 domain may not be a part of the expected domain. Such regions in the application domain but not 9 in the expected domain should be identified, but further justification is not necessary, as the model 10 is not expected to be used in those regions. However, any sparse region that lies within the 11 expected domain would need further justification as the model would be expected to be used in 12 that region. Table 31 gives the evidence commonly provided to demonstrate that this goal has 13 been satisfied.
14 Table 31 Evidence for G3.2.5Sparse Regions Sparse regions (i.e., regions of low data density) in the expected and G3.2.5 application domains should be identified and justified.
Level Evidence 1 There are many sparse regions in the expected domain.
There may be some sparse regions in the application domain. There 2
may be some sparse regions in the expected domain.
There may be some sparse regions in the application domain. There 3 may be some sparse regions in the expected domain, but the use of the CBT model in these regions is justified.
There may be some sparse regions in the application domain. There 4
are no sparse regions in the expected domain.
There are no sparse regions in either the application or the expected 5
domain.
15 Historical Evidence Levels for Reactor Safety Analysis 16 Level 3 has been most commonly accepted by the NRC staff. There may be sparse regions that 17 are at the edges of the models intended use (e.g., low mass flux or high mass flux), though 18 additional justification is usually provided for these regions. As discussed above, there are 19 numerous ways to justify the use of a model in a sparse region. The most common are: (1) 20 demonstrating that the model is conservative in the region, (2) demonstrating that it is not possible 21 for the fuel assembly to operate in the region, and (3) demonstrating that the region is not, in fact, 22 a sparse region. However, there are often instances in which the model does need to be used in a 23 region that is sparse (or at least has a very low data density). In these instances, a bias applied to 24 the model in the region in question may address the sparseness of the data without unnecessarily 25 negatively impacting the models predictions in other parts of the application domain. In the higher 26 dimensional spaces that are typical of most real application domains, the issue of sparse regions 27 becomes more difficult to understand and define.
67
1 G3.2.6Restricted to the Application Domain 2 Restricting the CBT model to its application domain is important. There are a variety of ways in 3 which this restriction can be placed and upheld on the computer code using the CBT model. Table 4 32 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
5 Table 32 Evidence for G3.2.6Restricted to the Application Domain G3.2.6 The model should be restricted to its application domain.
Level Evidence The computer code does not check whether the model is being used outside of its application domain. Instead, the code analyst ensures 1
that the model was used only inside of its application domain when reviewing the code output.
If the computer code attempts to use the model outside of its 2 application domain, the codes output marks it as a warning; however, the simulation continues to run.
If the computer code attempts to use the model outside of its 3 application domain, the codes output marks it as an error; however, the simulation continues to run.
If the computer code attempts to use the model outside of its 4 application domain, the codes output marks it as an error, and the simulation immediately quits running.
6 Historical Evidence Levels for Reactor Safety Analysis 7 Levels 3 and 4 have been most commonly accepted by the NRC staff. Level 1 would present 8 human-factors issues and should not be used if more than a few simulations are needed in the 9 particular application. Level 2 could also be present human-factors issues because users may not 10 recognize the severity of the application domain violation. In general, appropriate evidence for 11 G3.2.6 depends on the QA program the simulation is performed under and whether the restriction 12 to the application domain is the responsibility of that QA program or of the computer code itself.
13 3.3.3 G3.3Inconsistency in the Validation Error 14 Statistics from the validation error will be used as estimates of parameters from the population of 15 the model application error in order to quantify the uncertainty of the CBT model. This assumes 16 that the model application error can be described as a single population with the same distribution 17 and parameters (e.g., mean, variance) over the entire application domain and that the validation 18 error is a representative sample of this distribution.
19 As discussed by Box, Hunter, and Hunter (1978) one of the key assumptions in the data are the 20 assumption of independence. If the model application error is dependent on its location in the 21 application domain, it would be a collection of many populations, not a single population. Piepel 22 and Cuta (1993) argue that the validation error would not likely be from a single population; 23 instead, it would contain subregions in the application domain where the validation error would be 24 from different populations. Although the authors agree that this is likely the case, the assumption 25 of a single underlying population and independence should be reasonable as long as the 68
1 validation error is consistent and no obvious non-conservatisms exist. 21 The three subgoals in 2 Figure 14 are used to demonstrate that any inconsistencies in the validation error have been 3 appropriately addressed.
4 5 Figure 14 Decomposition of G3.3Inconsistencies in the Validation Error 6 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 7 discuss the evidence which could be used to demonstrate that the three base goals have been 8 satisfied. Additionally, a discussion is provided on the evidence which has been historically used 9 for CBT models applied in reactor safety analysis.
10 G3.3.1Identifying Non-poolable Data Sets 11 The validation error is typically made up of multiple sets of data. The validation error data may be 12 taken at low pressures, high flows, different axial power shapes, slightly different geometries, and 13 so on. Analysts generally assume that all of this data are poolable, i.e., that all of the data can be 14 treated as if they came from a single underlying population. If this is true, then the validation error, 15 which is based on the validation data, may be a representative sample from this larger population, 16 and therefore a good estimate of the behavior of the total population of model application error.
17 However, there are a number of reasons why the validation error may not be a representative 18 sample of the overall population of the model application error. First and foremost, the validation 19 error itself may represent several different populations. That is, pooling all of the validation errors 20 from each data set into a single validation error may be incorrect. For example, the CBT model 21 may make much better predictions at low pressures than at high mass fluxes. Pooling the data 22 would obscure this difference. The assumption of poolability should be tested by identifying key 23 data subsets in the validation error data set and by determining whether those data sets are 24 indeed from the same population.
21 In this sense, non-conservative means that the prediction of the CBT model over predicts the CHF or CP value by an amount greater than that accounted for by any uncertainty applied to the model. For CHF correlations, this would typically be the 95/95 value.
69
1 Although statistical tests can be performed to determine whether two subsets are from the same 2 population (i.e., have the same distribution shape, the same mean, the same variance), caution 3 should be used. Incorrectly determining that the data sets are from different populations when, in 4 fact, they are from the same population is common. This is known as a Type 1 error or a false 5 positive. The probability of Type 1 errors increases with each additional test performed. For 6 example, using the common significance value of 5 percent, the probability of obtaining a false 7 positive after one test is only 5 percent. However, if 14 tests are performed with the same 8 significance value, the probability of obtaining at least one false positive is over 50 percent. Thus, 9 even if the data are from the same population, performing 14 tests will more than likely result in 10 the conclusion that the data are from different populations. Therefore, these tests should be 11 applied only when necessary.
12 For PWRs, data should be separated (at a minimum) by axial power shape and by subchannel 13 type (rod and guide tube) as these are the main data sets that have been shown to be non-14 poolable. If any of the sets are not poolable, the models uncertainty should be derived from the 15 limiting data set. For BWRs, data should be separated (at a minimum) by axial power shape.
16 Thus, it should be determined whether all power shapes are poolable data sets. If they are not 17 poolable, the models uncertainty should be derived from the limiting data set.
18 The following statistical tests are commonly used during this process:
19
- Analysis of variance, commonly known as ANOVA, for equality of means 20
- T-test, for equality of means 21
- F-test, for equality of variances 22
- Chi-square test, for equality of variances 23
- DAgostinos test, for normality 24
- Wilks-Shapiro test, for normality 25
- Anderson-Darling test, for normality 26 27 If the validation error is made from (i.e., contains) data sets that are not poolable, the most limiting 28 or most conservative data set should be chosen if using a single value to quantify the models 29 uncertainty. Table 33 gives the evidence commonly provided to demonstrate that this goal has 30 been satisfied.
70
1 Table 33 Evidence for G3.3.1Identifying Non-poolable Data Sets The validation error should be investigated to ensure that it does not G3.3.1 contain any subgroups that are obviously not from the same population (i.e., non-poolable).
Level Evidence 1 No subgroups were analyzed for poolability.
All relevant subgroups were investigated, and there was statistical evidence that the groups were from different populations. Therefore, 2
the statistics from the limiting subgroup data set were used to determine the models uncertainty.
All relevant subgroups were investigated, and there was no statistical evidence that the groups were from different populations. The statistics 3
from the combined data sets were used to determine the models uncertainty.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 2 has been most commonly accepted by the NRC staff. If Level 1 were presented, it would 4 generally call for additional work and justification to be acceptable in reactor safety analysis, as it 5 is very common for models to have different predictive behavior over their application domain.
6 Level 3 is also common, but not as often achieved, as there is usually a subgroup which is slightly 7 more limiting than the others.
8 G3.3.2Identifying Non-conservative Subregions 9 Another key assumption is the assumption of statistical independence of the data (i.e.,
10 independent and identically distributed or iid) in the expected domain. As Piepel and Cuta (1993) 11 point out, the models uncertainty will likely vary over the expected domain. Therefore, an effort is 12 made to determine whether any obvious non-conservative subregions can be identified in the 13 validation error. The absence of such a subregion does not prove that statistical independence 14 exists; however, the authors are not aware of any other means to make such a determination.
15 Historically, non-conservative subregions have been identified by reviewing plots of the validation 16 error versus the various input parameters (e.g., pressure, mass flux, quality). The lack of a visual 17 trend in these plots was the justification that the models uncertainty did not vary over the 18 application domain. However, Kaizer (2015) points out that this visual one-dimensional (1-D) 19 plotting method ignores dependences among the various parameters and that non-conservative 20 subregions in the expected domain can be missed. Therefore, he proposed another method that 21 can be used to analyze data in up to three dimensions at a time. Although this proposed method 22 has a visual component to identify suspected non-conservative regions, it uses a statistical test to 23 determine whether the subregion is, in fact, non-conservative.
24 Because this method is limited to three dimensions, only the most important input parameters are 25 typically investigated together. For PWRs, those three parameters are typically the mass flux, 26 pressure, and local quality. For BWRs, those three parameters are typically the mass flux, 27 pressure, and inlet temperature (or subcooling). Other combinations should also be investigated 28 as necessary.
71
1 Proving that non-conservative subregions do not exist is not the objective. Such proof would call 2 for taking a very large number of data points. Given the limited data available in the validation 3 error data set, the only statement that can be confirmed is that no obvious non-conservative 4 subregion has been identified. However, if a non-conservative subregion is found, the model 5 uncertainty in that region would need to be increased to reflect the models predictive capability in 6 that region.
7 Table 34 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
8 Table 34 Evidence for G3.3.2Identifying Non-conservative Subregions The expected domain should be investigated to determine if it contains G3.3.2 any non-conservative subregions that would impact the predictive capability of the model.
Level Evidence Plots of each model input parameter versus the validation error (i.e., predicted versus measured or measured versus predicted) are 1
provided. This visual method (e.g., the 1-D method) demonstrates that there are no trends in the validation error with any input parameter.
Plots of each model input parameter versus the validation error (i.e., predicted over measured or measured over predicted) are provided. This visual method (e.g., the 1-D method) demonstrates that 2 there are no trends in the validation error with any input parameter.
Additionally, a method similar to the one proposed by Kaizer (2015) is used to demonstrate that there are no obvious non-conservative subregions in the application domain.
A method further refined from the one proposed by Kaizer (2015) is used. Such a method is able to consider all N-dimensions at the same 3
time and does not call for the user to visually identify any suspected non-conservative subregions.
9 Historical Evidence Levels for Reactor Safety Analysis 10 Level 1 has historically been most commonly accepted by the NRC staff. However, recent reviews 11 have used Level 2. The method discussed in Level 2 has revealed multiple non-conservative 12 subregions that required additional analysis or testing. Level 3 is ideal as it would be a completely 13 objective; however, the authors are not currently aware of any such method.
14 G3.3.3Appropriate Trends 15 Certain trends common in CHF and CP models could be expected in future models. Generally, 16 these trends can be seen by analyzing the plots of CHF or CP versus each of the various model 17 parameters. This includes both an examination of all the data at once and an examination of only 18 selected portions of the data (e.g., CHF at nominal pressures with decreasing mass flux).
19 Depending on the situation, the measured and predicted CHF and CP may need to be analyzed 20 separately. Table 35 gives the evidence commonly provided to demonstrate that this goal has 21 been satisfied.
72
1 Table 35 Evidence for G3.3.3Appropriate Trends The models predictions trend as expected in each of the various G3.3.3 model parameters.
Level Evidence Plots of the validation error (i.e., predicted over measured or measured 1
over predicted) versus each model input parameter are provided.
Plots of the measured or predicted CBT values versus each model 2
input parameter are provided. All trends are as expected.
Plots of the measured and predicted CBT values versus each model 3
input parameter are provided. All trends are as expected.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 1 has been most commonly accepted by the NRC staff. The trends should not only be 4 smooth and continuous, but also conform to known behavior of the associated phenomena. It is 5 often helpful to compare the trends of the current model with trends from previously approved 6 models. Generally, further details of this criterion are investigated only if inconsistent behavior in 7 the expected domain is suspected.
8 3.3.4 G3.4Calculating Model Uncertainty 9 In CHF models used in PWRs and CP models used in BWRs, the model uncertainty is obtained 10 from the validation error. However, the means of calculating the model uncertainty and its 11 application to reactor safety analysis vary greatly.
12 Departure from Nucleate Boiling Ratio Limit Used in Pressurized-Water Reactors 13 For CHF models used in PWRs, the models uncertainty is applied in the DNBR limit. That limit is 14 used to ensure that there will be at least a 95-percent probability at the 95-percent confidence 15 level that the hot fuel rod in the core does not experience a DNB or CBT condition during normal 16 operation or AOOs. This DNBR limit is solely dependent on the CHF models performance and is 17 independent of any conditions at the plant (e.g., the loading pattern).
18 The DNBR limit is a statistical limit derived from the validation error. The validation error (usually 19 represented as a ratio of the measured CHF to the model predicted CHF) is assumed to be a 20 representative sample from the population of the model application error. Therefore, the 21 95th percentile of the population of the model application error is estimated using the 95/95 value 22 from the validation error. In other words, the 95th percentile of the validation error is estimated 23 using a process that will overestimate the percentile 95 percent of the time (e.g., Owens method 24 (Owen, 1963) and Wilks method (Wilks 1941; Wilks 1943)). This 95/95 value is then used as the 25 DNBR limit and bounds the uncertainty of the CHF model.
26 For example, if the measured versus predicted values are normally distributed, the DNBR limit 27 could be determined to be the 95/95 value calculated, as prescribed by Owen, with the k-value 28 obtained from Owens tables. Equation 5 is used to calculate the 95/95 value:
29 95/95 = (5) 73
1 Where 95/95 is the 95/95 value, µ is the mean of the measured to predicted values, k is a factor 2 from Owens tables, and is the standard deviation of the measured to predicted values.
3 Generally, the DNBR limit is simply the reciprocal of the 95/95 value; however, a conservative 4 bias is usually added. Generally, this bias () may simply come from rounding up the DNBR limit 5 to a number with 3 significant figures (i.e., 1.133 becomes 1.14), but additional biases may also 6 be added to account for other non-conservatism in the model (for more details see Information 7 Notice 2014-01 (U.S. Nuclear Regulatory Commission, 2014)). Equation 6 gives the resulting 8 DNBR limit:
1 9 = + (6) 95/95 10 Safety Limit Minimum Critical Power Ratio Used in Boiling-Water Reactors 11 For CP models used in BWRs, the safety limit minimum critical power ratio (SLMCPR) reflects the 12 models uncertainty. That limit is used to ensure at least 99.9 percent of the fuel rods in the core 13 will not experience a CBT during normal operation or AOOs. Unlike the DNBR limit, the SLMCPR 14 does not depend solely on the CP models performance, but instead also depends on some 15 conditions at the plant (e.g., the core design).
16 A separate methodology is used to determine the SLMCPR, and the uncertainty in the CP model 17 is an input to that methodology. Usually, this uncertainty is represented by the standard deviation 18 of the models prediction of the experimental data 22 (i.e., the standard deviation of the validation 19 error). This standard deviation is used to capture the models uncertainty. If the mean of the 20 validation error is greater than one or if the sample is not normal, than the models uncertainty is 21 increased by artificially increasing the standard deviation before it is used in the SLMCPR 22 methodology. Mean values of less than one are generally not credited in determining the 23 SLMCPR.
24 Conservative Calculation of the Model Statistics 25 The models uncertainty is quantified using statistics from the validation error. Those statistics are 26 estimates of the parameters from the population of the model application error. Thus, the statistics 27 of the validation error should be calculated in such a manner that they bound the true model 28 application error. The three subgoals in Figure 15 are used to demonstrate that the validation 29 error has been appropriately quantified.
22 It should be noted that in PWRs, the validation error is given as the ratio of the measured value to the predicted value, but in BWRs the validation error is usually given as the ratio of the predicted value to the measured value.
74
1 2 Figure 15 Decomposition of G3.4Quantification of the Models Error 3 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 4 discuss the evidence which could be used to demonstrate that the three base goals have been 5 satisfied. Additionally, a discussion is provided on the evidence which has been historically used 6 for CBT models applied in reactor safety analysis.
7 G3.4.1Error Database 8 It may not be appropriate to use the entire validation error database to calculate the models 9 statistics, especially if the expected domain has non-poolable data sets or non-conservative 10 subregions. Therefore, the assessor should confirm that the statistics used to generate the 11 validation error are from an appropriate sample of data. Table 36 gives the evidence commonly 12 provided to demonstrate that this goal has been satisfied.
13 Table 36 Evidence for G3.4.1Error Database The validation error statistics should be calculated from an appropriate G3.4.1 database.
Level Evidence The models uncertainty was calculated using the entire database of 1
validation error.
The models uncertainty was calculated using a subset of the validation 2
error, which resulted in a more conservative calculation.
The models uncertainty was calculated from the limiting subset of the 3
validation error, which resulted in a more conservative calculation.
14 Historical Evidence Levels for Reactor Safety Analysis 15 Level 1 has been most commonly accepted by the NRC staff, but it generally assumes the data 16 are poolable and does not contain any non-conservative subregions. Level 2 is often provided if it 17 appears that a subset may be more limiting, but there is no definitive proof. Generally, if definitive 18 proof exists that a specific subset if most limiting, then the uncertainty is often calculated from only 19 the data in that subset (Level 3).
75
1 G3.4.2Validation Error Statistics 2 The method used to calculate the validation error statistics should be appropriate. This generally 3 means ensuring that the assumptions of any method used are fulfilled (e.g., if Owens method is 4 used to calculate the 95/95 value, the distribution of the validation error should be normally 5 distributed). Statistical methods may call for the data (i.e., the validation error) to (1) have the 6 same mean and variance (i.e., homoscedasticity), (2) be from the same distribution, (3) be from a 7 normal distribution, and (4) be independent and identically distributed (i.e., iid data). If populations 8 within the data do not have the same mean or variance, a conservative mean or variance can be 9 chosen to bound the model uncertainty. If the data are not normally distributed, a nonparametric 10 method (such as the Wilks method) can be used to calculate the model uncertainty. However, if 11 the data are not independent and identically distributed, the models predictive capability would 12 vary depending on the location in the application domain, and the models uncertainty would have 13 to account for this variability. Table 37 gives the evidence commonly provided to demonstrate that 14 this goal has been satisfied.
15 Table 37 Evidence for G3.4.2Validation Error Statistics The validation error statistics should be calculated using an appropriate G3.4.2 method.
Level Evidence The data used to calculate the models uncertainty appear to be independent and identically distributed. The method used to calculate 1 the statistics is a parametric method. Although the necessary preconditions of such a method were not satisfied, assumptions could be made to ensure that the resulting uncertainty was conservative.
The data used to calculate the models uncertainty appear to be independent and identically distributed, and one of the following applies:
- The method used to calculate the statistics is a parametric 2 method. The assumptions of such a method were demonstrated to be true (i.e., there is no reason to believe they are false) through statistical testing.
- The method used to calculate the statistics is a nonparametric method.
16 Historical Evidence Levels for Reactor Safety Analysis 17 Level 2 has been most commonly accepted by the NRC staff. However, Level 1 has been used 18 when the resulting statistics could be justified to be conservative.
19 G3.4.3Model Uncertainty Bias 20 After the models uncertainty is calculated, it is commonly biased in a conservative direction. For 21 example, a vendor may want to use a three-digit number as the DNBR limit. Thus, if the DNBR 22 limit were calculated as 1.2301, it would be rounded up to 1.24 (which is equivalent to the 23 addition of a bias of 0.0099). However, sometimes a bias is added to account for an uncertainty 24 that the model does not address. Table 38 gives the evidence commonly provided to demonstrate 25 that this goal has been satisfied.
76
1 Table 38 Evidence for G3.4.3Model Uncertainty Bias G3.4.3 The models uncertainty should be appropriately biased.
Level Evidence 1 The model needed a large bias (> 1%).
2 The model needed a small bias (< 1%).
3 The model needed no bias, or the only biasing was due to rounding.
2 Historical Evidence Levels for Reactor Safety Analysis 3 Level 3 has most commonly been accepted by the NRC staff, but Level 2 has also been 4 commonly accepted. A larger bias (i.e., greater than 1 percent) indicates that some uncertainty in 5 the model was not accounted for, which is generally avoided. In addition, because such biases 6 are generally applied based on engineering judgment, not experimental data, the bias itself is 7 subjective. Although situations arise that warrant the use of large biases, it is far from desirable 8 because there is often little justification for choosing the specific bias instead of a larger or smaller 9 value.
10 3.3.5 G3.5Model Implementation 11 Once the models uncertainty has been quantified by experimental data, the model can be applied 12 in a reactor safety analysis. However, the implementation of the model in the analysis should be 13 consistent with its use during validation.
14 For some CBT models, this may mean that the same computer code is used in both the validation 15 and the application of the models. Although certain inputs to the CBT model (e.g., pressures, flow 16 rates, power) would be expected to change depending on the situation in which the model was 17 used, those inputs may depend less on the situation and more on which closure models were 18 selected in the computer code that exercises the CBT model. In those situations, it may be 19 possible to change the inputs to the CBT model without changing the inputs to the computer code 20 itself (e.g., plant conditions) but merely by changing the closure models chosen. Therefore, it is 21 important to ensure that if the inputs to the CBT model depend on closure models in the computer 22 code that implements the CBT model, the same closure models are used in both the validation 23 and the application of the CBT model.
24 The reason for this is that the CBT model was validated with those closure models being applied, 25 and the uncertainty was quantified using only that set of closure models. The CBT model could be 26 used with another set of closure models, but the uncertainty would need to be quantified again 27 (i.e., determine the new validation error with the new closure models). Re-validation of the model 28 and re-quantification of the uncertainty is not necessarily a major exercise. The experimental data 29 already exist, and a new data set of validation errors can be obtained using the changed model, 30 code, or options. For the framework discussed here, only Criteria G3.2.1 and G3.2.2 would likely 31 need to be confirmed because the evidence supplied to justify all other criteria would likely remain 32 the same; however, this should be borne out by the analysis.
77
1 If the models prediction in the changed code is similar to its prediction in the previous code, the 2 evidence used to justify these two criteria may even remain the same. The three subgoals in 3 Figure 16 are used to demonstrate that the model has been correctly implemented.
4 5 Figure 16 Decomposition of G3.5Model Implementation 6 No further decompositions of the subgoals were deemed useful. Therefore, the sections below 7 discuss the evidence which could be used to demonstrate that these three base goals have been 8 satisfied. Additionally, a discussion is provided on the evidence which has been historically used 9 for CBT models applied in reactor safety analysis.
10 G3.5.1Same Computer Code 11 The computer code and the options used to specify the closure models and any other functionality 12 of that computer code should be the same. This is a much larger concern in PWRs because a 13 subchannel simulation contains many more uses of the field equations and closure models. The 14 direct modeling of the thermal-hydraulic response of the assembly should be consistent from the 15 validation to the application of a CBT model. Table 39 gives the evidence commonly provided to 16 demonstrate that this goal has been satisfied.
17 Table 39 Evidence for G3.5.1Same Computer Code The model has been implemented in the same computer code that was G3.5.1 used to generate the validation error.
Level Evidence The model has been implemented in a computer code very similar to 1
the one that was used to generate the validation error.
The same computer code with the same closure models and code 2 options that was used to generate the validation error will be used to perform any reactor safety analysis.
78
1 Historical Evidence Levels for Reactor Safety Analysis 2 Level 1 has been most commonly accepted by the NRC staff for BWRs and Level 2 for PWRs.
3 Many CBT models used for BWRs calculate the critical power of an assembly, and therefore the 4 computer code used generally does not calculate complex local thermal-hydraulic phenomena, 5 but rather more general parameters like assembly quality. As these formulations rely on few 6 closure models, it is possible to use the same CBT model in multiple BWR analysis codes.
7 However, because PWR analysis is performed at the sub-channel level, there are multiple closure 8 models used. Those closure models calculate the local parameters that are used by the CBT 9 model. Because the CBT model predictions could be changed by changing the closure models, 10 changing a computer code generally involves re-analyzing the validation data with the new code 11 for the PWRs.
12 G3.5.2Same Evaluation Methodology 13 It is important not only to use the same computer code to implement the CBT model but also to 14 implement the model in the same manner. Although the comparison of the measured values to 15 the predicted values is the basis for the validation, as discussed above, this comparison is 16 generally not as simple as comparing the CHF or CP at the location in the test assembly that 17 experienced a CBT to the predicated value at that location; therefore, a distinction was drawn 18 between model error and model application error. Another way of referring to the same evaluation 19 methodology is to ensure that the manner in which the model will be used (i.e., model application 20 error), is consistent with how the validation error was determined.
21 Section 3.3, which defines validation error, discusses the reasoning for this. Here, the authors will 22 only reiterate that the goal of using the CBT model is to ensure that a CBT does not occur, not to 23 ensure that, if a CBT does occur, the model predicts the exact location where it occurs. If the 24 model is able to identify the location, that can be evidence that the model is well correlated with 25 the physics of the assembly, but it is not a requirement and, moreover, may not be useful when 26 determining whether the model is appropriate.
27 Table 40 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
28 Table 40 Evidence for G3.5.2Same Evaluation Methodology The models prediction of the CBT is being applied using the same G3.5.2 evaluation methodology used to predict the validation data set for determining the validation error.
Level Evidence The model is implemented using a very similar evaluation 1
methodology.
2 The model is implemented using the same evaluation methodology.
29 Historical Evidence Levels for Reactor Safety Analysis 30 Level 2 has been most commonly accepted by the NRC staff. Level 1 could be used if an analysis 31 demonstrated that the changes would not affect the models uncertainty.
79
1 G3.5.3Transient Prediction 2 Like many other thermal-hydraulic models, many CBT models are developed using data taken 3 under steady-state conditions but applied in a transient 23 simulation. Although this is a common 4 practice, it should be justified, especially for models that contain integrals over space or time. This 5 is generally more of a focus in BWRs than it is in PWRs and is ultimately demonstrated through 6 transient tests. Those tests generally use time-varying inputs for power, flow, subcooling, or a 7 combination of these parameters. The goal is to demonstrate that there are no transients at which 8 a CBT occurred but was not predicted and, secondarily, to demonstrate that there were no tests in 9 which CBT was predicted (i.e., should have occurred) but did not occur.
10 Again, the goal of these tests is to demonstrate how well the model predicts whether CBT will 11 occur; therefore, these transient tests should be conducted close to conditions that cause a CBT.
12 Tests that are run too far from those conditions in either direction (i.e., either a test that was very 13 far from a CBT actually occurring such as a very low-power test or a test in which a CBT must 14 occur such as a very high-power test) would not be useful.
15 Table 41 gives the evidence commonly provided to demonstrate that this goal has been satisfied.
16 Table 41 Evidence for G3.5.3Transient Prediction The model results in an accurate or conservative prediction when it is G3.5.3 used to predict transient behavior.
Level Evidence 1 No experimental justification is provided.
2 Some experimental justification is provided.
3 Statistically significant experimental justification is provided.
17 Historical Evidence Levels for Reactor Safety Analysis 18 Level 1 or Level 2 have been commonly accepted by the NRC staff for PWRs, whereas Level 2 or 19 Level 3 have been commonly accepted for BWRs. The reason for the additional testing is likely 20 due to the manner in which CBT is modeled differently in BWRs and PWRs. In PWRs, the CBT is 21 based on local sub-channel parameters. Historically, it has been shown that CBT models which 22 are generated with steady state data will accurately or conservatively predict CBT during a 23 transient. This same assumption about CBT models being made with steady state data being 24 conservative for transient data are also made for models used in BWRs. However, while it is 25 possible to confirm this assumption through testing on a BWR test assembly, such testing would 26 be very difficult on a PWR test assembly.
27 For Level 3, by statistically significant, the authors mean that there were enough conservative 28 predictions from transients (i.e., those in which the CBT model was correct) to account for any 29 situations in which the CBT model may have been non-conservative. For example, if the CBT 30 model was non-conservative in a single test but conservative in only eight tests, its predictive 23 In this context, transient means time varying.
80
1 capability would be in question, as eight tests is generally considered to be too small a number to 2 determine any statistical significance.
81
1 4
SUMMARY
AND CONCLUSION 2 This work presents a generic safety case that can be used to determine the credibility of CBT 3 models. This safety case was generated through the experience of many experts at the NRC, 4 previously written safety evaluations, and documents in the open literature. This document 5 captures the knowledge and experience of multiple NRC staff members over many years. The 6 document presents a background on CBT including a literature survey, a description of the 7 underlying phenomena, and how those phenomena are commonly modeled. The document also 8 presents a credibility assessment framework, which combines the structure from GSN with the 9 capability of maturity assessment. The elements of the framework provided in this document have 10 been applied in multiple reviews at the NRC and have decreased total review time, increased 11 review consistency, and increased review efficiency.
83
1 5 REFERENCES 2 [1] Hewitt, G.F., and N.S. Hall-Taylor, Annular Two-Phase Flow, Pergamon Press, Oxford, 3 United Kingdom, 1970.
4 [2] Tong, L.S., and Y.S. Tang, Boiling Heat Transfer and Two-Phase Flow, Second Edition, 5 Taylor & Francis Group, Washington, DC, 1997.
6 [3] ASME. Guide for verification and validation in computational solid mechanics. ASME V&V 7 10-2006, New York, NY, American Society of Mechanical Engineers, 2006.
8 [4] NASA. Towards a Credibility Assessment of Models and Simulations, 49th 9 AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference.
10 AIAA-2008-2156, 2008.
11 [5] NASAB. Standard for Models and Simulations. NASA-STD-7009, Washington, DC, 12 National Aeronautics and Space Administration, 2008.
13 [6] Oberkampf, W.L., and C.J. Roy, Verification and Validation in Scientific Computing, 14 Cambridge University Press, Cambridge, United Kingdom, 2010.
15 [7] Kaizer, J.S., A.K. Heller, and W.L. Oberkampf, Scientific Computer Simulation Review, 16 Reliability Engineering and System Safety, 138:210-218, 2015.
17 [8] Ministry of Defense, Defense Standard 00-56, Safety Management Requirements for 18 Defense Systems, Part 1: Requirements, Issue 4, 2007.
19 [9] Denney, E.W., and Pai, G.J., Safety Case Patterns: Theory and Applications, NASA/TM-20 2015-218492, Feburary 2015.
21 [10] Food and Drug Administration, Infusion Pumps Total Product Life Cycle - Guidance for 22 Industry and FDA Staff, December 2, 2014.
23 [11] Goal Structure Notation (GSN) Working Group, GSN Community Standard Version 1, 24 Origin Consulting Limited, York, United Kingdom, 2011.
25 [12] Denney, E., G. Pai, and I. Habli, Towards Measurement of Confidence in Safety 26 Cases, Proceedings of the 5th International Symposium on Empirical Software 27 Engineering and Measurement, pp. 380-383, September 2011.
28 [13] Oberkampf, W.L., M. Pilch, and T.G. Trucano, Predictive Capability Maturity Model for 29 Computational Modeling and Simulation. SAND2007-5948, Albuquerque, NM, Sandia 30 National Laboratories, 2007.
31 [14] Tong, L.S., Boiling Crisis and Critical Heat Flux, U.S. Atomic Energy Commission, 1972.
32 [15] Todreas, N.E., and M.S. Kazimi, Nuclear Systems I: Thermal Hydraulic Fundamentals, 33 Taylor & Francis Group, Washington, DC, 1990.
34 [16] Lahey, R.T., and F.J. Moody, The Thermal Hydraulics of a Boiling Water Nuclear 35 ReactorSecond Edition, American Nuclear Society, La Grange Park, IL, 1993.
36 [17] Leidenfrost, J.G., On the Fixation of Water in Diverse Fire, A Tract about some qualtiies 37 of common water, 1756, translated, Wares, C., International Journal of Heat Mass 38 Transfer, 9:1153, 1966.
39 [18] Tong, L.S., H.B. Currin, P.S. Larsen, and O.G. Smith, Influence of Axially Nonuniform 40 Heat Flux on DNB, AIChE Chemical Engineering Symposium Series, 62(64):35-40, 41 1965.
85
1 [19] Macbeth, R.V., An Appraisal of Forced Convection Burnout Data, Proceedings of the 2 Institute of Mechanical Engineers, 1965-1966.
3 [20] Barnett, P.G., A Correlation of Burnout Data for Uniformly Heated Annuli and Its Use for 4 Predicting Burnout in Uniformly Heated Rod Bundles, Atomic Energy Establishment 5 Winfrith (AEEW-R) 463, 1966.
6 [21] Healzer, J.M., J.E. Hench, E. Janssen, and S. Levy, Design Basis for Critical Heat Flux 7 Condition in Boiling Water Reactors, APED-5, September 1966.
8 [22] Tong, L.S., Prediction of Departure from Nucleate Boiling for an Axially Non-Uniform Heat 9 Flux Distribution, Journal of Nuclear Energy, 21:241-248, 1967.
10 [23] Biasi, L., G.S. Clerici, S. Garribba, R. Sala, and A. Tozzi, Studies on Burnout: Part 3A 11 New Correlation for Round Ducts and Uniform Heating and Its Comparison with World 12 Data, Energia Nucleare 14:530-536, 1967.
13 [24] Gellerstedt, J.S., R.A. Lee, W.J. Oberjohn, R.H. Wilson, and L.J. Stanek, Correlation of 14 Critical Heat Flux in a Bundle Cooled by Pressurized Water, Two-Phase Flow and Heat 15 Transfer in Rod Bundles, American Society of Mechanical Engineers, New York, NY, 16 1969.
17 [25] Hughes, E.D., A Correlation of Rod Bundle Critical Heat Flux for Water in the Pressure 18 Range 150 to 725 psia, IN-1412, Idaho Nuclear Corporation, Idaho Falls, ID, 1970.
19 [26] Piepel, G.F., and J. M. Cuta, Statistical Concepts and Techniques for Developing, 20 Evaluating, and Validating CHF Models and Corresponding Fuel Design Limits, SKI 21 Technical Report, 93:46, 1993.
22 [27] Groeneveld, D.C., J. Shan, A.Z. Vasic, L.K.H. Leung, A. Durmayaz, J. Yang, S.C. Cheng, 23 and A. Tanase, The 2006 CHF Look-Up Table, Nuclear Engineering and Design, 24 237(15):1909-1922, 2007.
25 [28] Yang, B., J. Shan, J. Gou, H. Zhang, A. Liu, and H. Mao, Uniform versus Nonuniform 26 Axial Power Distribution in Rod Bundle CHF Experiments, Science and Technology of 27 Nuclear Installations, Volume 2014, 2014.
28 [29] Kaizer, J.S., Identification of Nonconservative Subregions in Empirical Models 29 Demonstrated Using Critical Heat Flux Models, Nuclear Technology, 190:65-71, 2015.
30 [30] Groeneveld, D.C., CHF Data Used to Generate 2006 Groeneveld CHF Lookup Tables, 31 NUREG/KM-0011, U.S. Nuclear Regulatory Commission, Washington, DC, 2017.
32 [31] Babcock & Wilcox, Correlation of Critical Heat Flux in a Bundle Cooled by Pressurized 33 Water, BAW-10000, Babcock & Wilcox, Lynchburg, VA, March 1970, Agencywide 34 Documents Access and Management System (ADAMS) Accession No. ML082490748 35 (Proprietary Information, Nonpublicly Available).
36 [32] Combustion Engineering (C-E), C-E Critical Heat Flux: Critical Heat Flux Correlation for 37 C-E Fuel Assemblies with Standard Spacer Grids, Part 1Uniform Axial Power 38 Distribution, CENPD-162-P-A, C-E, Stamford, CT, September 1976, ADAMS Accession 39 No. ML083010357 (Proprietary Information, Nonpublicly Available).
40 [33] Exxon Nuclear Company, Exxon Nuclear DNB Correlation for PWR fuel Designs, 41 XN-NF-621(P)(A), Revision 1, Richland, WA, September 1983, ADAMS Accession 42 No. ML16265A315 (Proprietary Information, Nonpublicly Available).
43 [34] Motley, F.E., K.W. Hill, F.F. Cadek, and J. Shefcheck, New Westinghouse Correlation 44 WRB-1 for Predicting Critical Heat Flux in Rod Bundles with Mixing Vane Grids, 86
1 WCAP-8762-P-A, Westinghouse Electric Company, Pittsburgh, PA, July 1984, ADAMS 2 Accession No. ML080630433 (Proprietary Information, Nonpublicly Available).
3 [35] Westinghouse Electric Company, VANTAGE 5H Fuel Assembly, WCAP-10444-P-A, 4 Pittsburgh, PA, September 1985, ADAMS Accession No. ML080650257 (Proprietary 5 Information, Nonpublicly Available).
6 [36] C-E, C-E Critical Heat Flux: Critical Heat Flux Correlation for C-E Fuel Assemblies with 7 Standard Spacer Grids, Part 2Non-Uniform Axial Power Distribution, CENPD-207-P-A, 8 Stamford, CT, December 1984, ADAMS Accession No. ML16260A362 (Proprietary 9 Information, Nonpublicly Available).
10 [37] AREVA, Departure from Nucleate Boiling Correlation for High Thermal Performance 11 Fuel, ANF-1224(P)(A), and Supplement 1 to ANF-1224(P)(A), Lynchburg, VA, April 1990.
12 [38] Farnswoth, D.A., and G.A. Meyer, The BWU Critical Heat Flux Correlations, 13 BAW-10199P-A, Revision 0, Framatome Technologies, Lynchburg, VA, February 1996.
14 [39] Smith III, L.D., M.W Lloyd, Y.X. Sung, and W.J. Leech, Modified WRB-2 Correlation, 15 WRB-2M, for Predicting Critical Heat Flux in 17x17 Rod Bundles with Modified LPD Mixing 16 Vane Grids, WCAP-15025-P-A, Westinghouse Electric Company, Pittsburgh, PA, 17 April 1999, ADAMS Accession No. ML081610106 (Proprietary Information, Nonpublicly 18 Available).
19 [40] Farnsworth, D.A., and G.A. Meyer, The BWU Critical Heat FIux Correlations Applications 20 to the Mark-B11 and Mark-BW17 MSM Designs, BAW-10199P-A, Addendum 1, 21 Framatome Cogema Fuels, Lynchburg, VA, December 2000, ADAMS Accession 22 No. ML003777245 (Proprietary Version, Nonpublicly Available).
23 [41] Farnsworth, D.A., and G.A. Meyer, Application of BWU-Z CHF Correlation to the 24 Mark-BW 17 Fuel Design with Mid-Span Mixing Grids, BAW-10199P-A, Addendum 2, 25 Framatome ANP, Lynchburg, VA, September 2002, ADAMS Accession 26 Nos. ML022560552 (Proprietary Version, Nonpublicly Available) and ML022560550 27 (Nonproprietary Version, Publicly Available).
28 [42] Sung, Y.X., P.F. Joffre, and P.A. Hilton, Addendum 1 to WCAP-1 4565-P-A Qualification 29 of ABB Critical Heat Flux Correlations with VIPRE-01 Code, WCAP-14565-P-A 30 Addendum 1-A, Westinghouse Electric Company, Pittsburgh, PA, August 2004, ADAMS 31 Accession Nos. ML042610371 (Proprietary Version, Nonpublicly Available) and 32 ML042610368 (Nonproprietary Version, Publicly Available).
33 [43] AREVA, Departure from Nucleate Boiling Correlation for High Thermal Performance 34 Fuel, EMF-92-153(P)(A), Revision 1, January 2005, ADAMS Accession 35 Nos. ML051020019 (Proprietary Version, Nonpublicly Available) and ML051020017 36 (Nonproprietary Version, Publicly Available).
37 [44] Farnsworth, D., and K.R. Greene, BHTP DNB Correlation Applied with LYNXT, 38 BAW-10241(P)(A), Revision 1, AREVA, Lynchburg, VA, July 2005, ADAMS Accession 39 Nos. ML052500092 (Proprietary Version, Nonpublicly Available) and ML052500075 40 (Nonproprietary Version, Publicly Available).
41 [45] Farnsworth, D., The BWU-B11R CHF Correlation for the Mark-B11 Spacer Grid, 42 BAW-10199P-A, Addendum 3, Framatome ANP, Lynchburg, VA, November 2005, 43 ADAMS Accession No. ML070170690 (Proprietary Version, Nonpublicly Available) 44 ML042990354 (Nonproprietary Version, Publicly Available).
87
1 [46] Joffre, P.F., Y.R. Chang, R. Kapoor, Y.X. Sung, L.D. Smith III, and P.A. Hilton, 2 Westinghouse Correlations WSSV and WSSV-T for Predicting Critical Heat Flux in Rod 3 Bundles with Side Supported Mixing Vanes, WCAP-16523-P-A, Westinghouse Electric 4 Company, Pittsburgh, PA, August 2007, ADAMS Accession Nos. ML072570633 5 (Proprietary Version, Nonpublicly Available) and ML072570327 (Nonproprietary Version, 6 Publicly Available).
7 [47] Farnsworth, D.A., and R.L. Harne, The ACH-2 CHF Correlation for the U.S. EPR, 8 ANP-10269P-A, AREVA, Lynchburg, VA, December 2007, ADAMS Accession 9 Nos. ML080790191 (Proprietary Version, Nonpublicly Available) and ML080790193 10 (Nonproprietary Version, Publicly Available).
11 [48] Joffre, P.F., R. Kapoor, Y.X. Sung, and P.A. Hilton, Addendum 2 to WCAP-14565-P-A 12 Extended Application of ABB-NV Correlation and Modified ABB-NV Correlation WLOP for 13 PWR Low Pressure Applications, WCAP-14565-P-A, Addendum 2-P-A, Westinghouse 14 Electric Company, Pittsburgh, PA, April 2008, ADAMS Accession Nos. ML081280713 15 (Proprietary Version, Nonpublicly Available) and ML081280712 (Nonproprietary Version, 16 Publicly Available).
17 [49] Joffre, P.F., Y.X. Sung, R. Mathur, L.D. Smith III, and P.A. Hilton, Westinghouse Next 18 Generation Correlation (WNG-1) for Predicting Critical Heat Flux in Rod Bundles with Split 19 Vane Mixing Grids, WCAP-16766-P-A, Westinghouse Electric Company, Pittsburgh, PA, 20 February 2010, ADAMS Accession Nos. ML100850532 (Proprietary Version, Nonpublicly 21 Available) and ML100850528 (Nonproprietary Version, Publicly Available).
22 [50] Mitsubishi Heavy Industries, Ltd., Thermal Design Methodology, MUAP-07009-P-A, 23 Tokyo, Japan, August 2013, ADAMS Accession Nos. ML13284A072 (Proprietary Version, 24 Nonpublicly Available) and ML13284A069 (Nonproprietary Version, Publicly Available).
25 [51] Korea Hydro & Nuclear Power Company, KCE-1 Critical Heat Flux Correlation for PLUS7 26 Thermal Design, APR1400-F-C-TR-12002-P, Gyeongju, South Korea, November 2012, 27 ADAMS Accession Nos. ML13018A158 (Proprietary Version, Nonpublicly Available) and 28 ML13018A147 (Nonproprietary Version, Publicly Available).
29 [52] AREVA, The ORFEO-GAIA and ORFEO-NMGRID Critical Heat Flux Correlations, 30 ANP-10341(P), Lynchburg, VA, August 2016, ADAMS Accession Nos. ML16238A076 31 (Proprietary Version, Nonpublicly Available) and ML16238A078 (Nonproprietary Version, 32 Publicly Available).
33 [53] Slifer, B.C., and J.E. Hench, Loss-of-Coolant Accident and Emergency Core Cooling 34 Models for General Electric Boiling Water Reactors, NEDO-10329, Equation C-32, 35 General Electric Company, San Jose, CA, April 1971.
36 [54] General Electric Company, General Electric BWR Thermal Analysis Basis (GETAB):
37 Data, Correlation and Design Application, NEDO-10958-PA, San Jose, CA, 38 January 1977, ADAMS Accession Nos. ML092820214 (Proprietary Version, Nonpublicly 39 Available) and ML102290144 (Nonproprietary Version, Publicly Available).
40 [55] AREVA, ANFB Critical Power Correlation, ANF-1125(P)(A), and Supplements 1 and 2, 41 Richalnd, WA, April 1990, ADAMS Accession No. ML081820434 (Proprietary Information, 42 Nonpublicly Available).
43 [56] General Electric Hitachi Nuclear Energy, R-Factor Calculation Method for GE11, GE12, 44 and GE13 Fuel, NEDC-32505P-A, Revision 1, Wilmington, NC, July 1999, ADAMS 45 Accession No. ML060520637 (Proprietary Information, Nonpublicly Available).
88
1 [57] Harris, W.R., and Y.Y. Yung, 10x10 SVEA Fuel Critical Power Experiments and CPR 2 Correlations: SVEA-96+, CENPD-389-P-A, ABB Combustion Engineering Nuclear Power, 3 Inc., Windsor, CT, September 1999, ADAMS Accession Nos. ML993470286 (Proprietary 4 Version, Nonpublicly Available) and ML993420024 (Nonproprietary Version, Publicly 5 Available).
6 [58] Harris, W.R., and Y.Y. Yung, 10x10 SVEA Fuel Critical Power Experiments and CPR 7 Correlations: SVEA-96, CENPD-392-P-A, Revision 00, CE Nuclear Power, LLC, 8 Monroeville, PA, September 2000, ADAMS Accession Nos. ML003767392 (Proprietary 9 Version, Nonpublicly Available) and ML003767366 (Nonproprietary Version, Publicly 10 Available).
11 [59] Harrington, R., and J.G.M. Anderson, GEXL96 Correlation for ATRIUM-9B Fuel, 12 NEDC-32981P, Global Nuclear Fuel, Wilmington, NC, September 2001, ADAMS 13 Accession Nos. ML003755947 (Proprietary Version, Nonpublicly Available),
14 ML012490537 (Safety Evaluation, Proprietary Version, Nonpublicly Available), and 15 ML012670193 (Nonproprietary Version, Publicly Available).
16 [60] Harrington, R., and J.G.M. Anderson, GEXL10 Correlation for GE12 Fuel, 17 NEDC-32464P, Revision 2, Global Nuclear Fuel, Wilmington, NC, September 2001, 18 ADAMS Accession No. ML012760512 (Proprietary Information, Nonpublicly Available).
19 [61] Harrington, R., GEXL80 Correlation for SVEA96+ Fuel, NEDC-33107P-A, Revision 1, 20 Global Nuclear Fuel, Wilmington, NC, October 2004, ADAMS Accession 21 Nos. ML043210062 (Proprietary Version, Nonpublicly Available) and ML043210058 22 (Nonproprietary Version, Publicly Available).
23 [62] Harris, W., M. Majed, G. Norback, and Y.Y. Yung, 10x10 SVEA Fuel Critical Power 24 Experiments and CPR Correlation: SVEA-96 Optima2, WCAP-16081-P-A, Westinghouse 25 Electric Company, Pittsburgh, PA, March 2005, ADAMS Accession No. ML051260213 26 (Proprietary Information, Nonpublicly Available) ML003676083 (Nonproprietary Version, 27 Publicly Available).
28 [63] Global Nuclear Fuel, GEXL97 Correlation Applicable to ATRIUM-10 Fuel, 29 NEDC-33383P, Revision 1, Wilmington, NC, June 2008, ADAMS Accession 30 No. ML082070090 (Proprietary Information, Nonpublicly Available) and ML082070088 31 (Nonproprietary Version, Publicly Available).
32 [64] Norback, G., and W. Harris, SVEA-96 Optima2 CPR Correlation (D4): Modified R-factors 33 for Part-Length Rods, WCAP-16081-P-A, Addendum 2-A, Westinghouse Electric 34 Company, Pittsburgh, PA, February 2009, ADAMS Accession Nos. ML072200243 35 (Proprietary Version, Nonpublicly Available) and ML072200242 (Nonproprietary Version, 36 Publicly Available).
37 [65] Norback, G., and W. Harris, SVEA-96 Optima2 CPR Correlation (D4): High and Low Flow 38 Applications, WCAP-16081-P-A, Addendum 1-A, Westinghouse Electric Company, 39 Pittsburgh, PA, March 2009, ADAMS Accession Nos. ML091060144 (Proprietary Version, 40 Nonpublicly Available) and ML091060143 (Nonproprietary Version, Publicly Available).
41 [66] Global Nuclear Fuel, GEXL17 Correlation for GNF2 Fuel, NEDC-33292P, Revision 3, 42 Wilmington, NC, June 2009, ADAMS Accession No. ML091830641 (Proprietary 43 Information, Nonpublicly Available) and ML091830624 (Nonproprietary Version, Publicly 44 Available).
45 [67] AREVA, SPCB Critical Power Correlation, EMF-2209(NP)(A), Revision 3, Richland, WA, 46 September 2009, ADAMS Accession Nos. ML093650230 (Proprietary Version, 89
1 Nonpublicly Available) and ML093650235 (Nonproprietary Version, Publicly Available) and 2 ML111290532 (Nonproprietary Version, Publicly Available).
3 [68] Global Nuclear Fuel, GEXL14 Correlation for GE 14 Fuel, NEDC-32851P-A, Revision 5, 4 Wilmington, NC, April 2011, ADAMS Accession No. ML111290535 (Proprietary 5 Information, Nonpublicly Available).
6 [69] AREVA, ACE/ATRIUM-10 Critical Power Correlation, ANP-10249(P)(A), Revision 2, 7 Richland, WA, March 2014, ADAMS Accession Nos. ML14175A226 (Part 1 of 3, 8 Proprietary Version, Nonpublicly Available); ML14175A227 (Part 2 of 3, Proprietary 9 Version, Nonpublicly Available); ML14175A228 (Part 3 of 3, Proprietary Version, 10 Nonpublicly Available), and ML14175A229 (Nonproprietary Version, Publicly Available).
11 [70] AREVA, ACE/ATRIUM 10XM Critical Power Correlation, ANP-10298(P)(A), Revision 1, 12 Richland, WA, March 2014, ADAMS Accession Nos. ML14183A739 (Part 1 of 3, 13 Proprietary Version, Nonpublicly Available); ML14183A743 (Part 2 of 3, Proprietary 14 Version, Nonpublicly Available); ML14183A748 (Part 3 of 3, Proprietary Version, 15 Nonpublicly Available); and ML14183A734 (Nonproprietary Version, Publicly Available).
16 [71] Bergmann, U., M. Hemlin, K. Bergman, and J-M. Le Corre, 10x10 SVEA Fuel Critical 17 Power Experiments and New CPR Correlation: D5 for SVEA-96 Optima3, 18 WCAP-17794-P, November 2013, ADAMS Accession Nos. ML13333A276 (Proprietary 19 Version, Nonpublicly Available) and ML13333A275 (Nonproprietary Version, Publicly 20 Available).
21 [72] AREVA, ACE/ATRIUM-11 Critical Power Correlation, ANP-10335PP, Richland, WA, 22 February 2015, ADAMS Accession Nos. ML15062A552 (Part 1 of 2, Proprietary Version, 23 Nonpublicly Available); ML15062A555 (Part 2 of 2, Proprietary Version, Nonpublicly 24 Available); and ML15062A554 (Nonproprietary Version, Publicly Available).
25 [73] U.S. Code of Federal Regulations, General Design Criteria for Nuclear Power Plants, 26 Appendix A to Domestic Licensing of Production and Utilization Facilities, Part 50, 27 Chapter I, Title 10, Energy.
28 [74] U.S. Code of Federal Regulations, ECCS Evaluation Models, Appendix K to Domestic 29 Licensing of Production and Utilization Facilities, Part 50, Chapter I, Title 10, Energy.
30 [75] U.S. Code of Federal Regulations, Contents of Applications; Technical Information, 31 Section 34 of Domestic Licensing of Production and Utilization Facilities, Part 50, 32 Chapter I, Title 10, Energy.
33 [76] U.S. Code of Federal Regulations, Quality Assurance Criteria for Nuclear Power Plants 34 and Fuel Reporcessing Plants, Appendix B to Domestic Licensing of Production and 35 Utilization Facilities, Part 50, Chapter I, Title 10, Energy.
36 [77] U.S. Nuclear Regulatory Commission, Fuel System Design, Section 4.2 of 37 NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for 38 Nuclear Power Plants: LWR Edition, Revision 3, March 2007, ADAMS Accession 39 No. ML070740002.
40 [78] U.S. Nuclear Regulatory Commission, Thermal and Hydraulic Design, Section 4.4 of 41 NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for 42 Nuclear Power Plants: LWR Edition, Revision 2, March 2007, ADAMS Accession 43 No. ML070550060.
90
1 [79] U.S. Nuclear Regulatory Commission, Fuel Safety Limit Calculation Inputs Were 2 Inconsistent with NRC-Approved Correlation Limit Values, Information Notice 2014-01, 3 February 21, 2014, ADAMS Accession No. ML13325A966.
4 [80] ANSI/ASME NQA-1, Quality Assurance Program Requirements for Nuclear Power 5 Plants, American National Standards Institute/American Society of Mechanical Engineers, 6 New York, NY, 2015 7 [81] Dittus, F.W., and L.M.K. Boelter, Heat Transfer in Automobile Radiators of the Tublar 8 Type, University of California Publications in Engineering, 2, pp 443-461, 1930.
9 [82] Box, G.E.P., W.G. Hunter, and J.S. Hunter, Statistics for Experimenters, John Wiley &
10 Sons, Ltd., New York, NY, 1978.
11 [83] Wieckhorst, O., S. Opel, R. Harne, and F. Filhol, Challenges in CHF Correlation 12 Development, LWR Fuel Performance Meeting (TopFuel 2013), September 15-19, 2013.
13 [84] Wieckhorst, O., H. Gabriel, R. Harne, M. Anghelescu, and O. Martinie, ORFEOA CHF 14 Correlation for GAIA, AREVAs Advanced PWR Fuel Assembly Design, 16th International 15 Meeting on Nuclear Reactor Thermal Hydraulics (NURETH-16), Chicago, IL, 16 August 30-September 4, 2015.
17 [85] Anderson, M.G., and P.D. Bates (Eds.), Model Validation: Perspectives in Hydrological 18 Science. John Wiley & Sons, Ltd., Chichester, United Kingdom, 2001.
19 [86] National Research Council, Ground Water Models; Scientific and Regulatory Applications, 20 The National Academies Press, Washington, DC, p. 303, 1990.
21 [87] Owen, D.B., Factors from One-Sided Tolerance Limits and the Variables Sampling 22 Plans, SCR-607, Sandia Corporation, Albuquerque, NM, 1963.
23 [88] Wilks, S.S., Determination of Sample Sizes for Setting Tolerance Limits, The Annals of 24 Mathematical Statistics, 12(1), 91-96, 1941.
25 [89] Wald, A., An Extension of Wilks Method for Setting Tolerance Limits, The Annals of 26 Mathematical Statistics, 14(1), 43-55, 1943.
91
92 1 APPENDIX A LISTING OF ALL GOALS GOAL The critical boiling transition model can be trusted.
The experimental data supporting the critical boiling transition G1 model are appropriate.
The experimental data have been collected at a credible test G1.1 facility.
G1.1.1 The test facility is well understood.
The test facility has been verified by comparison to an outside G1.1.2 source.
G1.2 The experimental data have been accurately measured.
G1.2.1 The test facility has an appropriate quality assurance program.
The experiment has been appropriately statistically designed (i.e., the value of a system parameter from any test was G1.2.2 completely independent from its value in the test before and after the test).
The method used to obtain critical boiling transition data G1.2.3 results in an accurate measurement.
The instrumentation uncertainties have been demonstrated to G1.2.4 have a minimal impact on the measured critical heat flux or critical power.
The uncertainty in the critical heat flux or critical power is G1.2.5 quantified through repeated tests at the same state points.
The heat losses from the test section are quantified, G1.2.6 appropriately low, and duly accounted for in the measured data.
The test assembly reproduced the local conditions in the G1.3 reactor fuel assembly.
The test assembly used in the experiment should have G1.3.1 geometric dimensions equivalent to those of the fuel assembly used in the reactor for all major components.
The grid spacers used in the test assembly should be G1.3.2 prototypical of the grid spacers used in the reactor assembly.
The axial power shapes in the test assembly should reflect the G1.3.3 expected or limiting axial power shapes in the reactor assembly.
The radial power peaking in the test assembly should reflect G1.3.4 the expected or limiting radial powers in the reactor assembly.
Any differences between the test assembly and the reactor assembly should have a minimal impact on the flow field. This G1.3.5 includes components that are not in the reactor assembly but that are needed for testing purposes.
G2 The model was generated in a logical fashion.
A-1
G2.1 The mathematical form of the model is appropriate.
The mathematical form of the model contains all the G2.1.1 necessary parameters.
The reasoning for choosing the mathematical form of the G2.1.2 model should be discussed and should be logical.
The process for determining the models coefficients was G2.2 appropriate.
The training data (i.e., the data used to generate the G2.2.1 coefficients of the model) should be identified.
The method for calculating the models coefficients should be G2.2.2 described.
The method for calculating the R- or K-factor and the additive constants (for both full-length and part-length rods) should be G2.2.3 described. Further, a description of how such values are calculated if dryout is not measured on the rod under consideration should be provided (boiling-water reactors only).
The model has sufficient validation as demonstrated through G3 appropriate quantification of its error.
G3.1 The correct validation error has been calculated.
The validation error is appropriately distributed throughout the G3.2 application domain.
The validation data (i.e., the data used to quantify the models G3.2.1 error) should be identified.
The application domain of the model should be G3.2.2 mathematically defined.
G3.2.3 The expected domain of the model should be understood.
There should be adequate validation error data density G3.2.4 throughout the expected and application domains.
Sparse regions (i.e., regions of low data density) in the G3.2.5 expected and application domains should be identified and justified.
G3.2.6 The model should be restricted to its application domain.
Any inconsistencies in the validation error have been G3.3 accounted for appropriately.
The validation error should be investigated to ensure that it G3.3.1 does not contain any subgroups that are obviously not from the same population (i.e., non-poolable).
The expected domain should be investigated to determine if it G3.3.2 contains any non-conservative subregions that would impact the predictive capability of the model.
The models predictions trend as expected in each of the G3.3.3 various model parameters.
A-2
The models uncertainty has been appropriately calculated G3.4 from the validation error.
The validation error statistics should be calculated from an G3.4.1 appropriate database.
The validation error statistics should be calculated using an G3.4.2 appropriate method.
G3.4.3 The models uncertainty should be appropriately biased.
G3.5 The model has been correctly implemented.
The model has been implemented in the same computer code G3.5.1 that was used to generate the validation error.
The models prediction of the CBT is being applied using the G3.5.2 same evaluation methodology used to predict the validation data set for determining the validation error.
The model results in an accurate or conservative prediction G3.5.3 when it is used to predict transient behavior.
1 A-3
NUREG/KM-0013 Credibility Assessment Framework for Critical Boiling Transition Models A Generic Safety Case to Determine the Credibility of Critical Heat Flux and Critical March 2019 Power Models Draft Report for Comment J.S. Kaizer, R. Anzalone, E. Brown, M. Panicker, S. Haider, J. Gilmer, T. Drzewiecki, Technical and A. Attard Division of Safety Systems Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Washington, DC 20555-0001 Same as above J.S. Kaizer Critical boiling transition (CBT) occurs when a flow regime that has a higher heat transfer rate transitions to a flow regime that has a significantly lower heat transfer rate. Models that predict a CBT are a necessary part of reactor safety analysis because they are used to determine plant safety limits. Therefore, the review of CBT models has been a focus of the U.S. Nuclear Regulatory Commission (NRC) since its inception in 1975.
This work presents a generic safety case in the form of a credibility assessment framework that combines aspects of goal structuring notation and maturity assessment. This framework is focused on the credibility assessment of CBT models with specific application to reactor safety analysis. The NRC has performed many such assessments and has generated this framework based on the experience of current and former NRC staff, as well as previous staff reviews as summarized in staff evaluations. This document includes a survey of the important technical and regulatory literature; a detailed technical discussion of CBT models and their application; and a suggested framework for CBT models. This NUREG/KM summarizes the knowledge the NRC staff has developed over the course of 40 years of CBT model and analysis reviews.
Critical heat flux, critical power, departure from nucleate boiling, critical quality, boiling crisis, burnout, dryout, critical boiling transition
United States Nuclear Regulatory Commission NUREG/KM-0013 Draft March 2019