ML18022A060

Report - ACRS Assessment of the Quality of Selected NRC Research Projects - Fy 2017

ML18022A060
Person / Time
Issue date:	12/31/2017
From:	Dennis Bley Advisory Committee on Reactor Safeguards
To:	Weber M Office of Nuclear Regulatory Research
Nourbakhsh H
Shared Package
ML18022A054	List: ML18022A060 ML18022A068
References
Download: ML18022A060 (22)

Text

Assessment of the Quality of Selected NRC Research Projects by the Advisory Committee on Reactor Safeguards - FY 2017 December 2017 U.S. Nuclear Regulatory Commission Advisory Committee on Reactor Safeguards Washington, DC 20555-0001

ii ABOUT THE ACRS The Advisory Committee on Reactor Safeguards (ACRS) was established as a statutory Committee of the Atomic Energy Commission by a 1957 amendment to the Atomic Energy Act of 1954. The functions of the Committee are described in Sections 29 and 182b of the Act. The Energy Reorganization Act of 1974 transferred the Atomic Energy Commission licensing functions to the U.S.

Nuclear Regulatory Commission (NRC), and the Committee has continued serving in the same advisory role to the NRC.

The ACRS provides independent reviews of, and advice on, the safety of proposed or existing NRC-licensed reactor facilities and the adequacy of proposed safety standards. The ACRS reviews power reactor and fuel cycle facility license applications for which the NRC is responsible, as well as the safety-significant NRC regulations and guidance related to these facilities. The ACRS also provides advice on radiation protection, radioactive waste management, and earth sciences in the agency's licensing reviews for fuel fabrication and enrichment facilities, and waste disposal facilities. On its own initiative, the ACRS may review certain generic matters or safety-significant nuclear facility items. The Committee also advises the Commission on safety-significant policy issues and performs other duties as the Commission may request. Upon request from the U.S. Department of Energy (DOE), the ACRS provides advice on U.S. Navy reactor designs and hazards associated with DOE's nuclear activities and facilities. In addition, upon request, the ACRS provides technical advice to the Defense Nuclear Facilities Safety Board.

ACRS operations are governed by the Federal Advisory Committee Act, which is implemented through NRC regulations at Title 10, Part 7, of the Code of Federal Regulations. ACRS operational practices encourage the public, industry, State and local governments, and other stakeholders to express their views on regulatory matters.

iii MEMBERS OF THE ADVISORY COMMITTEE ON REACTOR SAFEGUARDS Dr. Ronald G. Ballinger, Professor of Nuclear Science and Engineering and Materials Science and Engineering, Massachusetts Institute of Technology, Cambridge, Massachusetts Dr. Dennis C. Bley (Chairman), President, Buttonwood Consulting, Inc., Albuquerque, New Mexico Mr. Charles H. Brown, Jr., Senior Advisor for Electrical Systems, Syntek Technologies, Inc.,

Arlington, Virginia Dr. Margaret Sze-Tai Y. Chu, Former Director of the Department of Energys Office of Civilian Radioactive Waste Management, Washington, District of Columbia Dr. Michael L. Corradini (Vice-Chairman), Professor Emeritus of Engineering Physics, University of Wisconsin, Madison, Wisconsin Dr. Vesna B. Dimitrijevic, Retired Technical Consultant, AREVA, Marlborough, Massachusetts Dr. Walter L. Kirchner, Retired Technical Staff Member, Argonne National Laboratory, Argonne, Illinois, and Los Alamos National Laboratory, Los Alamos, New Mexico Dr. Jose March-Leuba, Principal of MRU and Associate Professor of Nuclear Engineering, University of Tennessee, Knoxville, Tennessee Dr. Dana A. Powers, Retired Senior Scientist, Sandia National Laboratories, Albuquerque, New Mexico Mr. Harold B. Ray, Retired Executive Vice President, Southern California Edison Company, Rosemead, California Dr. Joy L. Rempe, Principal, Rempe and Associates, LLC, Idaho Falls, Idaho Dr. Peter C. Riccardella (Member-at-Large), Senior Associate, Structural Integrity Associates, Inc., Centennial, Colorado Mr. Gordon R. Skillman, Principal, Skillman Technical Resources, Inc., Hershey, Pennsylvania Mr. John W. Stetkar, Principal, Stetkar & Associates, Hot Springs, Arkansas Mr. Matthew Sunseri, Retired President and Chief Executive Officer of Wolf Creek Nuclear Operating Corporation, Burlington, Kansas

iv ABSTRACT In this report, the ACRS presents the results of its assessment of the quality of selected research projects sponsored by the NRC Office of Nuclear Regulatory Research. An analytic/deliberative methodology was adopted by the Committee to guide its review of research projects. The methods of multi-attribute utility theory were used to structure the objectives of the review and develop numerical scales for rating each project with respect to each objective.

The results of the evaluations of the quality of the selected research projects are summarized as follows:

NUREG-2208, Validation of Computational Fluid Dynamics Methods Using Prototypic Light Water Spent Fuel Assembly Thermal-Hydraulic Data

- This project was found to be satisfactory, a professional work that satisfies research objectives.

NUREG/CR-7222, Tsunami Hazard Assessment Based on Wave Generation, Propagation, and Inundation Modeling for the U.S. East Coast

- This project was found to be satisfactory, a professional work that satisfies research objectives.

v CONTENTS Page ABSTRACT..................................................................................................................... iv FIGURES........................................................................................................................ vi TABLES.......................................................................................................................... vi ABBREVIATIONS.......................................................................................................... vii

1. INTRODUCTION....................................................................................................... 1

2. METHODOLOGY FOR EVALUATING THE QUALITY OF RESEARCH PROJECTS................................................................................................................ 3

3. RESULTS OF QUALITY ASSESSMENT................................................................... 5 3.1 Validation of Computational Fluid Dynamics Methods Using Prototypic Light Water Spent Fuel Assembly Thermal-Hydraulic Data....................................... 5 3.2 Tsunami Hazard Assessment Based on Wave Generation, Propagation, and Inundation Modeling for the U.S. East Coast. 9

4. REFERENCES........................................................................................................ 14

vi FIGURES Page

1. The Value Tree used for Evaluating the Quality of Research Projects................................... 3 TABLES

1. Constructed Scales for the Performance Measures............................................................... 4

2. Summary Results of ACRS Assessment of the Quality of the Project NUREG-2208, Validation of Computational Fluid Dynamics Methods Using Prototypic Light Water Spent Fuel Assembly Thermal-Hydraulic Data................................................................................ 6

3. Summary Results of ACRS Assessment of the Quality of the Project NUREG/CR-7222, Tsunami Hazard Assessment Based on Wave Generation, Propagation, and Inundation Modeling for the U.S. East Coast......................................................................................... 10

vii ABBREVIATIONS ACRS Advisory Committee on Reactor Safeguards BWR boiling-water reactor CFD computational fluid dynamics COMCOT Cornell Multi-grid Coupled Tsunami FY fiscal year GCI grid conversion index MOST Method of Splitting Tsunamis NOAA National Oceanic and Atmospheric Administration NPP nuclear power plant NRC Nuclear Regulatory Commission PCT peak cladding temperature PWR pressurized-water reactor RES Office of Nuclear Regulatory Research U.S.

United States

1 1 INTRODUCTION The Nuclear Regulatory Commission (NRC) maintains a safety research program to ensure that the agency's regulations have sound technical bases. The research effort is needed to support regulatory activities and agency initiatives while maintaining an infrastructure of expertise, facilities, analytical tools, and data to support regulatory decisions.

The Office of Nuclear Regulatory Research (RES) is required to have an independent evaluation of the effectiveness (quality) and utility of its research programs. This evaluation is required by the NRC Strategic Plan that was developed as mandated by the Government Performance and Results Act. Since fiscal year (FY) 2004, the Advisory Committee on Reactor Safeguards (ACRS) has been assisting RES by performing independent assessments of the quality of selected research projects [1-13]. The Committee established the following process for conducting the review of the quality of research projects:

RES submits to the ACRS a list of candidate research projects for review because they have reached sufficient maturity that meaningful technical review can be conducted.

The ACRS selects a maximum of four projects for detailed review during the fiscal year.

A panel of three to four ACRS members is established to assess the quality of each research project.

The panel follows the guidance developed by the ACRS Full Committee in conducting the technical review. This guidance is discussed further below.

Each panel assesses the quality of the assigned research project and presents an oral and a written report to the ACRS Full Committee for review. This review is to ensure uniformity in the evaluations by the various panels.

The ACRS submits an annual summary report to the RES Director.

Based on later discussions with RES, the ACRS made the following enhancements to its quality assessment process:

After familiarizing itself with the research project selected for quality assessment, each panel holds an informal meeting with the RES project manager and representatives of the user office to obtain an overview of the project and the user office's insights on the expectations for the project with regard to their needs.

In addition, if needed, an additional informal meeting is held with the project manager to obtain further clarification of information prior to completing the quality assessment.

The purposes of these enhancements were to ensure greater involvement of the RES project managers and their program office counterparts during the review process and to identify objectives, user office needs, and perspectives on the research projects.

2 An analytic/deliberative decision-making framework was adopted for evaluating the quality of NRC research projects. The definition of quality research adopted by the ACRS includes two major characteristics:

Results meet the objectives The results and methods are adequately documented Within the first characteristic, the ACRS considered the following general attributes in evaluating the NRC research projects:

Soundness of technical approach and results Has execution of the work used available expertise in appropriate disciplines?

Justification of major assumptions Have assumptions key to the technical approach and the results been tested or otherwise justified?

Treatment of uncertainties/sensitivities Have significant uncertainties been characterized?

Have important sensitivities been identified?

Within the general category of documentation, the projects were evaluated in terms of the following measures:

Clarity of presentation Identification of major assumptions In this report, the ACRS presents the results of its assessment of the quality of the research projects associated with:

NUREG-2208: Validation of Computational Fluid Dynamics Methods Using Prototypic Light Water Spent Fuel Assembly Thermal-Hydraulic Data NUREG/CR-7222: Tsunami Hazard Assessment Based on Wave Generation, Propagation, and Inundation Modeling for the U.S. East Coast These projects were selected from a list of candidate projects suggested by RES.

The methodology for developing the quantitative metrics (numerical grades) for evaluating the quality of NRC research projects is presented in Section 2 of this report. The results of the assessment and ratings for the selected projects are discussed in Section 3.

3 2 METHODOLOGY FOR EVALUATING THE QUALITY OF RESEARCH PROJECTS To guide its review of research projects, the ACRS has adopted an analytic/deliberative methodology [14-15]. The analytical part utilizes methods of multi-attribute utility theory [16-17]

to structure the objectives of the review and develop numerical scales for rating the project with respect to each objective. The objectives were developed in a hierarchical manner (in the form of a "value tree"), and weights reflecting their relative importance were developed. The value tree and the relative weights developed by the Full Committee are shown in Figure 1.

Figure 1. The Value Tree used for Evaluating the Quality of Research Projects The quality of projects is evaluated in terms of the degree to which the results meet the objectives of the research and of the adequacy of the documentation of the research. It is the consensus of the ACRS that meeting the objectives of the research should have a weight of 0.75 in the overall evaluation of the research project. Adequacy of the documentation was assigned a weight of 0.25. Within these two broad categories, research projects were evaluated in terms of subsidiary "performance measures":

Justification of major assumptions (weight: 0.12)

Soundness of the technical approach and reliability of results (weight: 0.52)

Treatment of uncertainties and characterization of sensitivities (weight: 0.11)

Research Quality Success Documentation Results Meet the Objectives Clarity of Presentation Identification of Major Assumptions Soundness of Technical Approach/Results Uncertainties/

Sensitivities Addressed Justification of Major Assumptions 0.16 0.09 0.12 0.52 0.11 0.25 0.75

4 Documentation of the research was evaluated in terms of the following performance measures:

Clarity of presentation (weight: 0.16)

Identification of major assumptions (weight: 0.09)

To evaluate how well the research project was performed with respect to each performance measure, constructed scales were developed as shown in Table 1. The starting point is a rating of 5, Satisfactory (professional work that satisfies the research objectives). Often in evaluations of this nature, a grade that is less than excellent is interpreted as pejorative. In this ACRS evaluation, a grade of 5 should be interpreted literally as satisfactory. Although innovation and excellent work are to be encouraged, the ACRS realizes that time and cost place constraints on innovation. Furthermore, research projects are constrained by the work scope that has been agreed upon. The score was, then, increased or decreased according to the attributes shown in the table. The overall score of the project was produced by multiplying each score by the corresponding weight of the performance measure and adding all the weighted scores.

As discussed in Section 1, a panel of three to four ACRS members was formed to review each selected research project. Each member of the review panel independently evaluated the project in terms of the performance measures shown in the value tree. The panel deliberated the assigned scores and developed a consensus score, which was not necessarily the arithmetic average of individual scores. The panel's consensus score was discussed by the Full Committee and adjusted in response to ACRS members' comments. The final consensus scores were multiplied by the appropriate weights, the weighted scores of all the categories were summed, and an overall score for the project was produced. A set of comments justifying the ratings was also produced.

Table 1. Constructed Scales for the Performance Measures SCORE RANKING INTERPRETATION 10 Outstanding Creative and uniformly excellent 8

Excellent Important elements of innovation or insight 5

Satisfactory Professional work that satisfies research objectives 3

Marginal Some deficiencies identified; marginally satisfies research objectives 0

Unacceptable Results do not satisfy the objectives or are not reliable

5

3. RESULTS OF QUALITY ASSESSMENT 3.1 Validation of Computational Fluid Dynamics Methods Using Prototypic Light Water Spent Fuel Assembly Thermal-Hydraulic Data Introduction Applicants submit spent nuclear fuel dry storage cask designs to NRC for certification under Title 10 of the Code of Federal Regulations (10 CFR) Part 72, Licensing Requirements for Independent Storage of Spent Nuclear Fuel, High-Level Radioactive Waste, and Reactor-Related Greater Than Class C Waste. The NRC staff performs its technical review of these designs in accordance with 10 CFR Part 72 and NUREG-1536, Standard Review Plan for Spent Fuel Dry Storage Systems at a General License Facility, Revision 1, July 2010. To ensure that the cask and the fuel material temperatures of the dry cask storage system remain within allowable limits or criteria for normal, off-normal, and accident conditions, the NRC staff performs a thermal review as part of its technical review.

Recent applications increasingly have performed thermal-hydraulic analyses using computational fluid dynamics (CFD) codes (e.g., ANSYS FLUENT [18]) to demonstrate the adequacy of the thermal design. RES has recently completed validation studies of the FLUENT CFD code to assist the staff in making regulatory decisions to ensure adequate protection for storage and transportation casks. The results of these validation studies are documented in NUREG-2208

[19]. The scope of this quality review is limited to this report.

The NUREG-2208 validation studies were based on data obtained from a previous NRC-sponsored experimental program conducted at Sandia National Laboratories to study thermal-hydraulic conditions and zirconium fire propagation during a complete loss-of-coolant event in spent fuel pools. In the first-phase separate effects test program (preignition) heat transfer and fluid flow phenomena (i.e., natural circulation induced mass flow) were investigated. These test results are documented in NUREG/CR-7143 [20], NUREG/CR-7215 [21], and NUREG/CR-7216

[22]. The fuel assembly experimental data from these studies provided reliable information for various fuel assembly heat loads that could be used to validate the analytical methods. Combined with current methods to determine modeling and application uncertainty, the measured data offered additional confirmation on the adequacy of the applied analytical methods.

General Observations The research summarized in NUREG-2208 relates to the ANSYS FLUENT, Version 14.5, validation studies performed for the uniform axial, electrically-heated test assemblies to simulate:

1) a single full-length commercial 17x17 pressurized-water reactor (PWR) fuel bundle, 2) a 17x17 PWR 1x4 configuration where the center fuel assembly was electrically heated and the four surrounding assemblies were unheated. and 3) a single test assembly to simulate a full-length commercial 9x9 boiling-water reactor (BWR) fuel bundle.

For each of the configurations, detailed and porous media models were developed to validate code results based on the experimental data. Steady-state and transient simulations were performed at the representative decay heat power values tested. Peak cladding temperature (PCT), exterior wall temperature, and mass flow rate from modeling predictions were compared

6 to the experimental data. Results showed good agreement between predictions and experimental data for both the CFD and porous media models, the latter with the provision when using proper input. Parametric studies were conducted to assess model sensitivity and a grid conversion index (GCI) method published by the American Society of Mechanical Engineers (ASME),

Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer, was used to calculate the discretization uncertainty of the model. The staff further recommended that the lessons and guidelines from this work be added to NUREG-2152, Computational Fluid Dynamics Best Practice Guidelines for Dry Cask Applications.

The consensus scores for this project are shown in Table 2. The score for the overall assessment of this work was evaluated to be 5.0 (satisfactory, a professional work that satisfies research objectives).

Table 2. Summary Results of ACRS Assessment of the Quality of the Project, "Validation of Computational Fluid Dynamics Methods Using Prototypic Light Water Spent Fuel Assembly Thermal-Hydraulic Data Performance Measures Consensus Scores Weights Weighted Scores Clarity of presentation 6.5 0.16 1.04 Identification of major assumptions 5.5 0.09 0.50 Justification of major assumptions 4.5 0.12 0.54 Soundness of technical approach/results 4.5 0.52 2.34 Treatment of uncertainties/sensitivities 5.0 0.11 0.55 Overall Score 5.0 Comments and conclusions within the evaluation categories are provided below.

Clarity of Presentation (Consensus Score is 6.5)

The report documents the research project in sufficient detail to allow the reader to follow the work without having to reference the original sources for experimental details and data. The CFD

7 modeling is also explained in sufficient detail for the reader to understand the approaches taken in using the FLUENT code for both the detailed and porous models. In particular, the porous media treatment of equivalent flow resistance, and radial and axial effective conductivities were well presented. The presentation in Section 3 of predicted versus test results for PCT and mass flow rates was reasonably comprehensive for purposes of comparison and examining trends in model predictions (over or under). The code verification process of numerical error estimation using the GCI method is well documented in Section 4. Overall, the research program was well documented, and clarity of presentation was more than satisfactory.

Identification of Major Assumptions (Consensus Score is 5.5)

The major assumptions in this work are related to the simplification of the geometry for the porous model values of effective thermal conductivity, both radial and axial, and for the wall boundary conditions. Multiple detailed CFD runs at different power levels were performed to generate a database of temperature ranges that was used to estimate effective thermal conductivity values for the porous model. These were also compared to results reported by TRW Environment Safety Systems using finite element methods [23]. Another important parameter for the porous media model is the fluid flow frictional coefficient for each assembly configuration. The values used in this project were obtained from detailed FLUENT calculations of each assembly, as a function of power. The values were adjusted in the porous media model to match the detailed PCT results for the PWR and BWR assemblies. This was also a satisfactory approach.

Justification of Major Assumptions (Consensus Score is 4.5)

While adequately discussed, the justification of the methodology for adjusting the friction factor in the porous media model to match values from the detailed PCT calculations raises questions about other uncertainties in the experimental data and the modeling approach. This parameter was flagged as a matter of concern (underestimated) in the user-need request for this research

[24], and bears further justification and sensitivity analysis. More effort here would have strengthened the justification for the analytical approach to parameter selection.

Soundness of Technical Approach/Results (Consensus Score is 4.5)

The CFD analyses used data from four of the five pre-ignition test series: two PWR single assembly tests of slightly different pool cell sizes (outer dimension or effective hydraulic diameter),

the PWR 1x4 configuration, and the full-length BWR single assembly. Data from a 1x4 shortened BWR configuration were not used because it was conducted with forced circulation, which is not prototypical of natural-circulation, dry cask applications. The geometrical complexity of the fuel assemblies was captured in the detailed CFD model and then approximated in the porous model.

As discussed above, effective radial and axial thermal conductivities were generated for the single bundles and the peripheral unheated assemblies. The porous media frictional coefficients were derived from equations 2.6 and 2.7, then corrected empirically to produce the same PCTs under steady-state power as obtained from the detailed model. Given that the authors demonstrate that all of the tests will be in laminar flow conditions, the rationale for different friction coefficients for different power levels should have been discussed.

Several trends in the calculational results for both the detailed and porous media models are noted. In general, for the PWR tests, the mass flow rates for steady-state and transient tests are

8 under-predicted, which leads to over-predicted PCT values as a function of axial height, as well as over-predicted external wall temperatures. Because this is a natural circulation problem, these two deviations from the observed experimental data are linked, but there was no discussion of this modeling offset. Treatment of thermal mass uncertainty (Section 3.1.2.1) was also questionable. The density and specific heat capacity of MgO (simulated fuel) were changed by almost a factor of two to obtain better predictions of PCT as a function of time in transient tests.

Density, mass, and specific heat capacity should be some of the best-known variables in modeling the fuel assembly, least subject to modeling uncertainty. For example, channel boundary heat losses, particularly in 1x4 configurations, are likely a larger uncertainty during heat-up and transients than simulated fuel density.

For the BWR steady-state and transient tests, experimental temperature values did not reach steady-state. For the transient tests, PCT is only compared between detailed and porous models (found to be consistent) and not experimental data; however, at about 12 to 13 hours1.50463e-4 days <br />0.00361 hours <br />2.149471e-5 weeks <br />4.9465e-6 months <br />, predictions of PCT versus both time and axial height compare favorably with experimental data if somewhat over-predicted. Further elaboration here would have been valuable in interpreting results.

Treatment of Uncertainties/Sensitivities (Consensus Score is 5.0)

The sensitivity study using the GCI method to calculate the discretization uncertainty of the model was well elaborated upon in Section 4. This seems a good approach and should be considered in other numerical modeling applications, as appropriate.

Lacking was a more in-depth examination of the impact of key modeling parameters on predicted results; in particular, the effect of frictional coefficients and effective thermal conductivities, which are inherent to this modeling approach, have more associated uncertainty than physical parameters such as fuel density. While the work overall achieved the stated objectives, given the user need request and the extrapolation required when applied to actual spent fuel casks, more treatment of uncertainty/sensitivity of modeling variables may have proven insightful. This is of importance as acceptance limits are more closely approached in licensing reviews of spent nuclear fuel dry cask storage.

9 3.2 Tsunami Hazard Assessment Based on Wave Generation, Propagation, and Inundation Modeling for the U.S. East Coast Subsequent to the 2004 and 2005 series of tsunamis in southeastern Asia, NRC conducted an in-depth review of past tsunami evaluations and guidelines for the Atlantic and Gulf coast nuclear power plants (NPPs). Although the NRC staff concluded that these coastal NPPs are adequately protected, the 2004 Indian Ocean tsunami raised the level of concern for an extreme tsunami-initiated event, which could potentially exceed the dimensions of all of the recorded events taken into consideration in the design basis for those NPPs. NRCs previous tsunami design guidelines for these coastal facilities considered historical tsunami records but did not explicitly characterize design-basis tsunamigenic sources including earthquakes, submarine landslides, and other potential sources for the Atlantic and Gulf coasts. Consequently, NRC sponsored a series of research projects at the National Oceanic and Atmospheric Administration (NOAA) and the United States Geologic Survey to further the staffs understanding of tsunamis and their potential sources so that quantitative tsunami wave criteria can be available to assess the tsunami hazards for Atlantic and Gulf coast NPP sites. As a part of this effort, a tsunami hazard assessment based on wave generation, propagation, and inundation modeling for the United States (U.S.) East Coast was performed. The results of this study are documented in NUREG/CR-7222 [25]. The scope of this quality review is limited to this report.

NUREG/CR-7222 presents results from a tsunami hazard assessment for the Atlantic coast of the U.S. The study makes use of the Pacific Marine Environmental Laboratory pre-computed database of over a thousand synthetic tsunami sources to identify potentially hazardous tsunami events for the eastern U.S. coastline, particularly the area of Virginia Beach, Virginia. The historical 1755 Lisbon tsunami event is used to validate the simulations by comparing the computed results with the evidence of tsunami impact along the Caribbean arc.

In this investigation, a segment of the Caribbean seismic arc located north of Puerto Rico, between the U.S. Virgin Islands and Hispaniola and known as the Puerto Rico Trench, is identified as the most hazardous tsunami source for the U.S. eastern coastline. For potential seismic events of magnitudes between w 8.6 and w 8.9, the modeled run-up heights are between 3.5 and 5 m in Virginia Beach. In addition to the seismically generated tsunami hazard, the impact of potential tsunamis generated by the possible future collapse of the flank of the Cumbre Vieja volcano in La Palma (Canary Islands), and by the Currituck landslide on the Atlantic continental shelf of the U.S. are also investigated. For the landslide events, the Eulerian-Lagrangian hydrocode, iSALE is used to compute the generated landslide and the solution is coupled to three different tsunami simulation models. Special attention is paid to wave dispersion effects by comparing results from these three different simulations using the shallow water wave equations, the weakly non-linear Boussinesq equations, and the strongly nonlinear Boussinesq equations.

The Method of Splitting Tsunamis (MOST) code is used to compute the non-dispersive shallow water wave solution with numerical dispersion adjusted to match that prescribed by linear theory in deep water.

The results of this study show that dispersive effects tend to be weak when the tsunami propagates over shallow areas of the continental shelf, with good agreement between simulations computed with MOST and with the dispersive Boussinesq-type models.

10 The consensus scores for this project are shown in Table 2. The score for the overall assessment of this work was evaluated to be 4.5 (a professional work that satisfies research objectives).

Table 3. Summary Results of ACRS Assessment of the Quality of the Project, Tsunami Hazard Assessment Based on Wave Generation, Propagation, and Inundation Modeling for the U.S. East Coast Performance Measures Consensus Scores Weights Weighted Scores Clarity of presentation 4.7 0.16 0.75 Identification of major assumptions 5.0 0.09 0.45 Justification of major assumptions 3.8 0.12 0.46 Soundness of technical approach/results 5.0 0.52 2.60 Treatment of uncertainties/sensitivities 2.0 0.11 0.22 Overall Score 4.5 Comments and conclusions within the evaluation categories are provided below.

Clarity of Presentation (Consensus Score is 4.7)

In general, the report is well written and easy to understand. The figures provide useful graphic information to support the text although some are difficult to understand at first. For example, Figure 2.7 illustrates a normalized inshore response to the C51 scenario along the 12-meter isobath with a nominal spacing of 5 km. The graphic intends to facilitate the association of impact features with the geography; however, it is qualitative in nature and without quantitative references is difficult to understand the point being conveyed.

It seems evident that different authors wrote Sections 2 and 3 (seismic), compared to Sections 4 and 5 (landslide). The report tends to repeat some information about the NOAA pre-existing forecast models (short-term inundation forecast tool) and the importance of non-linear modeling over the continental shelf. However, that may be intentional for emphasis in each of the analysis sections.

11 Finally, the Forward and Abstract of this NUREG/CR state this report documents results of a comprehensive study. Our review indicates that the Executive Summary provides a more accurate description of the scope of this research and the content of this report.

Identification of Major Assumptions (Consensus Score is 5.0)

The report does not describe details of the models in the MOST computer code that is used for the shallow-water wave propagation analyses. However, the intent of this report is to demonstrate applications of those models, rather than describe the models themselves.

The report clearly identifies the assumptions that have the most important influence on the shallow-water wave propagation analyses. Those assumptions include:

Characteristics of the seismic and landslide sources (location, magnitude, displacement)

Boundary conditions for the shallow-water models at the interface with the open-ocean models Grid size used for most of the analyses Justification of Major Assumptions (Consensus Score is 3.8)

The report does not always clearly describe the near-shore or onshore location for which the analysis results are presented. For example, the results summarized in Section 2 seem to be indexed to a 12-meter near-shore depth. It is not clear whether the Virginia Beach analyses summarized in Section 3.3 are indexed to a 5-meter near-shore depth or extend to onshore inundation.

Different grid resolutions are used for the analyses in various sections of the report. It is noted that the grid resolution can result in substantial differences in the estimated wave heights, depending on the near-shore bathymetry. The open-ocean propagation models use a 4 arc-minute grid resolution (approximately 7.2 km). Section 2.3 discusses the effects from using three smaller grid resolutions for the shallow-water models. "Grid C" has a resolution of 30 arc-seconds (approximately 750 meters). The resolutions for "Grid A" and "Grid B" are not identified.

"Grid C" was used for the analyses that are summarized in Section 2. That decision is justified by computational resources and a desire to use a consistent grid for the entire U.S. coastline.

However, the report often emphasizes the realism of the detailed results. It is not apparent why the applied "Grid C" resolution provides appropriate analyses for specific coastal locations.

Most of the detailed analyses in Section 3 identify the grid size that was used for a particular analysis. For example, the Virginia Beach analyses in Section 3.3 use progressively smaller grids of 24 arc-seconds (approximately 600 meters), 8 arc-seconds (approximately 200 meters), and 3 arc-seconds (approximately 75-90 meters) as the waves approach the shoreline.

12 The La Palma landslide analyses in Section 4 use a 1 arc-minute (approximately 1500 meters) grid resolution for the open-ocean models, due to computational resource considerations. It is not apparent whether a 1 arc-minute grid resolution is also used for the shallow-water models. If so, that rather coarse resolution may influence the noted dissipation of amplitude as the waves traverse the continental shelf.

The Currituck landslide analyses in Section 5 apparently use a 9 arc-second grid resolution.

The discussion in each section does not justify why each given set of grid resolutions provides the most realistic results.

Soundness of Technical Approach / Results (Consensus Score is 5.0)

As noted previously, this report is not intended to benchmark or fully validate the shallow-water wave propagation models in the MOST computer code. The report contains limited comparisons between the model results and historical records of inundations in Portugal, Morocco, and the Caribbean from the tsunami generated by the 1755 Lisbon earthquake.

The report also contains qualitative comparisons between the MOST model results and those from other models. Section 2.5 notes that MOST provides results that are similar to the Cornell Multi-grid Coupled Tsunami (COMCOT) model. Section 4.6 notes that the iSALE-MOST models for landslide wave generation and propagation do not match experimental wave runup data as well as the Los Alamos National Laboratory SAGE model.

Treatment of Uncertainties / Sensitivities (Consensus Score is 2.0)

The report states:

"The primary goal of this research is to assist the Nuclear Regulatory Commission (NRC) in assessing the potential for tsunami impact along the U.S. Atlantic coast by identifying potential tsunamigenic sources and providing tools for deterministic assessment of tsunami propagation and coastal inundation." [emphasis added]

The analyses that are summarized in this report are analogous to the source-to-site ground motion response models and the site-specific bedrock-to-basemat soil response models in a seismic hazards analysis. It is a clear expectation of contemporary probabilistic seismic hazard analyses that those models should account for aleatory uncertainties that derive from variability in the evaluated parameters and epistemic uncertainties about the physical processes or assumptions that are used in the models.

This report mentions the notion of uncertainty only twice. In Section 2.3, it is noted that "Grid C" is used for those analyses "to avoid the uncertainties associated with the placement of sub-regional 'tiles' for the entire coast". In Section 4.6, it is noted that "using even the non-linear version of MOST for the propagation phase may be a possible source of uncertainty".

Sources of uncertainty that may affect the predicted wave heights and time histories at a given site include:

13 The model grid resolution. This is illustrated in Figure 2.3 and in general discussions throughout the report. No systematic analyses are provided to show how the results vary with different grid sizes for specific sources and sites.

Differences between the MOST model and other wave propagation models. The report briefly mentions comparisons to the COMCOT model for seismically-caused tsunamis and the SAGE model for landslide wave generation and propagation. However, no quantitative information is provided to compare the results for specific sources and sites.

Perhaps the largest source of uncertainty that is not addressed in the report is the effects from characterizing the tsunami source. The analyses imply that the characteristics of potential seismic and landslide sources must be known precisely. That precision is reinforced throughout the report. However, its effects are most apparent in the analyses that are summarized in Figures 3.11 through 3.13 for the 1755 Lisbon earthquake and Figures 3.17 through 3.25 for the Caribbean seismic sources. One might argue that those comparative analyses illustrate the effects from variations in the location, magnitude, and extent of an earthquake. However, the report provides no insight about how the uncertainties in seismic source characterization can be treated in the wave propagation models.

Section 4 contains another example of the effects from the need to precisely characterize the tsunami source. The analyses of wave propagation from the La Palma landslide conclude that the wave amplitude decreases substantially as the waves traverse the U.S. continental shelf. This behavior contrasts to the increase in wave amplitude for tsunamis that are caused by a seismic displacement. Thus, it seems apparent that the assumed time evolution of the seismic or landslide forcing function and the corresponding deep-ocean wavelength have a very important effect on the boundary conditions at the continental shelf edge and the shallow-water propagation models. As noted in Section 4.4 and Section 4.6, this may also be a source of uncertainty in the MOST models or assumptions for shallow-water propagation.

We see in the report that there are large differences in predicted wave size and inundation results between the smaller and larger source magnitudes (i.e., from negligible to disastrous). In our opinion, what is needed are estimates of frequency of occurrence versus source magnitude, along the lines of the central and eastern U.S. seismic source evaluation (NUREG-2115 [26]).

14

4. REFERENCES

1.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects, November 18, 2004 (ML043240107).

2.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2005, November 4, 2005 (ML053110211).

3.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2006, October 17, 2006 (ML062900517).

4.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2007, October 19, 2007 (ML072890363).

5.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2008, October 22, 2008 (ML082890373).

6.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2009, September 16, 2009 (ML091940495).

7.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2010, November 15, 2010 (ML1031202321).

8.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2011, September 19, 2011 (ML11262A210).

9.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2012, October 22, 2012 (ML12293A451).

10.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2013, November 21, 2013 (ML13323B189).

11.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2014, November 25, 2014 (ML14322A944).

12.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2015, November 17, 2015 (ML15287A332).

13.

Advisory Committee on Reactor Safeguards, ACRS Assessment of the Quality of Selected NRC Research Projects - FY 2016, January 11, 2017 (ML17011A227).

14.

National Research Council, Understanding Risk: Informing Decisions in a Democratic Society. National Academy Press, Washington, DC, 1996.

15.

Apostolakis, G.E., and Pickett, S.E., "Deliberation: Integrating Analytical Results into Environmental Decisions Involving Multiple Stakeholders," Risk Analysis, 18:621-634, 1998.

15

16.

Clemen, R., Making Hard Decisions, 2nd Edition, Duxbury Press, Belmont, CA, 1995.

17.

Keeney, R.L., and H. Raiffa, Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Wiley, New York, 1976.

18.

ANSYS, Inc., ANSYS FLUENT User Guide Version 14.5, Canonsburg, PA, 2012.

19.

U.S. Nuclear Regulatory Commission, NUREG-2208, Validation of Computational Fluid Dynamics Methods Using Prototypic Light Water Spent Fuel Assembly Thermal-Hydraulic Data, March 2017 (ML17062A567).

20.

U.S. Nuclear Regulatory Commission, NUREG/CR-7143, Characterization of Thermal-Hydraulic and Ignition Phenomena in Prototypic, Full-Length Boiling Water Reactor Spent Fuel Pool Assemblies After a Postulates Complete Loss-of Coolant Accident, March 2013 (ML13072A056).

21.

U.S. Nuclear Regulatory Commission, NUREG/CR-7215, Spent Fuel Pool Project Phase I: Pre-Ignition and Ignition Testing of a Single Commercial 17x17 Pressurized Water Reactor Spent Fuel Assembly under Complete Loss of Coolant Accident Conditions, April 2016 (ML16112A022).

22.

U.S. Nuclear Regulatory Commission, NUREG/CR-7216, Spent Fuel Pool Project Phase II: Pre-Ignition and Ignition Testing of a 1x4 Commercial 17x17 Pressurized Water Reactor Spent Fuel Assemblies under Complete Loss of Coolant Accident Conditions, April 2016 (ML16112A084).

23.

TRW Environmental Safety Systems, Inc., Spent Nuclear Fuel Effective Thermal Conductivity Report, prepared for U.S. Department of Energy, July 11, 1996.

24.

U.S. Nuclear Regulatory Commission, User Need Request for Research to Perform Computational Fluid Dynamics Validation Using Boiling-Water Reactor And Pressurized-Water Reactor Fuel Assembly Thermal-Hydraulic Data, January 23, 2014 (ML13261A314).

25.

U.S.

Nuclear Regulatory Commission, NUREG/CR-7222, Tsunami Hazard Assessment Based on Wave Generation, Propagation, and Inundation Modeling for the U.S. East Coast, July 2016 (ML16193A021).

26.

U.S. Nuclear Regulatory Commission, NUREG-2115, Central and Eastern United States Seismic Source Characterization for Nuclear Facilities, Volumes 1-6, January 2012 (ML12048A804, ML12048A833, ML12048A851, ML12048A858, ML12048A859, ML12048A860).