ML21306A292

Rev.0 Public Comment Resolution Table

ML21306A292
Person / Time
Issue date:	01/20/2022
From:	Patrick Raynaud NRC/RES/DE/CIB
To:
Song K
Shared Package
ML21259A190	List: ML21306A292 ML21334A158 ML21334A260
References
DG-1382 RG-1.245, Rev 0
Download: ML21306A292 (20)

Text

Response to Public Comments on Draft Regulatory Guide (DG)-1382 Preparing Probabilistic Fracture Mechanics Submittals Proposed New of Regulatory Guide (RG) 1.245 On September 23rd, 2021, the NRC published a notice in the Federal Register (86 FR 52927) that Draft Regulatory Guide, DG-1382, (Proposed new Regulatory Guide (RG) 1.245), was available for public comment. The Public Comment period ended on October 25th, 2021. The NRC received comments from the organizations and people listed below. The NRC has combined the comments and the staff responses in the following table.

1. Matthew Walter, BWRVIP Senior Technical Leader 2. Cédric Sallaberry, Senior Research Mathematician Electric Power Research Institute (EPRI) Engineering Mechanics Corporation of Columbus 1300 West W.T. Harris Boulevard 3518 Riverside Drive, Suite 202 Charlotte, NC 28262-8550: Columbus, OH 43221-1735 ADAMS Accession No. ML21300A073 ADAMS Accession No. ML21306A074

3. J. Brian Hall, Fellow Engineer Westinghouse Electric Company 1332 Beulah Road Pittsburgh, PA 15235-5082 ADAMS Accession No. ML21306A076 January 2022

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

1. Walter General We believe that the overall plan for a graded approach, as No change to the documents as a result of this Comment identified in Regulatory Position C.1, as well as the set of comment.

items to be included in the PFM analysis and submittal as identified in Regulatory Position C.2 define a reasonable framework that meets the objective of having standardized content and a graded approach for PFM submittals to NRC.

We also agree that key content from the EPRI white paper has been incorporated in DG-1382.

Furthermore, we agree that presubmittal discussions will be key to ensure NRC and industry alignment regarding the choice of categorization within several of the tables shown in DG-1382.

2. Walter A, page 1 Delete extra comma after and holders of The NRC agrees with this comment. The extra comma was deleted.

3. Walter B, page 6 The second of the two referenced sentences seem somewhat The NRC agrees with this comment. The two redundant with the first and generally provides little sentences were combined for simplicity and clarity.

additional clarity. Consider deleting or revising the second sentence.

4. Walter B, page 6 The October 23, 2018 public meeting followed publication The NRC agrees with this comment. The mention of of NRCs 2018 technical letter report, which is dated the September, 2018 technical letter report was moved September 13, 2018. To clarify the chronology in this before the discussion of the October, 2018 public background section, it is suggested to discuss first NRCs meeting.

technical letter report, followed by the October 23, 2018 public meeting and BWRVIP Letter 2019-016, and finally this Draft RG-1382 and Draft NUREG/CR-7278.

5. Walter B, page 6 Add reference for approved methodology The NRC agrees with this comment. A reference has been added.

2

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

6. Walter C, page 8 We agree that identification of deviations from this RG No change to the documents as a result of this would help streamline NRCs review, as well as comment.

correspondence between NRC and industry as part of the review process.

7. Walter C, page 9 Sections 2.2, 2.4 and 2.5 should be included somewhere in The NRC agrees with this comment. Sections 2.2 and this Figure. Sections 2.2. and 2.4 seem to fit under Step 1 2.4 were added as references to Determine the and Section 2.5 seems to fit under Step 2. suitability of the PFM code for the application.

Further, Section 2.5 is addressed in Comment 8.

8. Walter C, page 9 Selection of appropriate models (Sec. 2.5) should be The NRC agrees with this comment. In response, included in the Plan step. Selection of appropriate models (Sec. 2.5) added as a bullet to Step 1 Plan in Figure C-1.

9. Walter C, page 9 Since NRC is encouraging applicants to have pre-submittal The NRC agrees with this comment. Addressed in discussions/meetings with NRC (Sec. 2.2), this should be Comment 7.

part of the Plan step.

10. Walter C, page 9 Suggest referencing Sec. 2.2 and Sec. 2.4 for 3rd bullet in The NRC agrees with this comment. Addressed in Plan step. Comment 7.

11. Walter C, page 9 Use lower case for from The NRC agrees with this comment. The word from is revised as suggested.

12. Walter C, page 11 Please specify that direct access entails access to the The NRC agrees with this comment. The words software executable not the source code. executable program have been added to clarify.

3

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

13. Walter C, page 11 We concur with this discussion of acceptance criteria, but The NRC agrees in part and disagrees in part with this we suggest that the discussion of acceptance criteria be comment. Although the NRC agrees that acceptance extended to cover some specifics. We believe inclusion of criteria are an important aspect of PFM regulatory some specific information would be productive and valuable applications, the RG purposely does not address for applicants who apply PFM. We suggest that the Reg acceptance criteria because it is too broad of a topic Guide list acceptance criteria that were previously approved and would therefore detract from the specific purpose by NRC and remain acceptable to NRC, including the of the RG, describing acceptable best-practices for acceptable range of use for each set of approved acceptance generating PFM submittals to the NRC. In fact, PFM criteria. The discussion should include the rationale that could be applied to a large number of specific areas, NRC applied to find that each set of acceptance criteria was thus it is not possible to comprehensively address and acceptable. This information would be valuable to applicants provide acceptance criteria. Acceptance criteria and who are considering whether derivation of new acceptance QoI are problem specific, and the RG is by design a criteria is needed. Ideally, NRC would publish acceptance generic document. Some acceptance criteria may be criteria for the most common types of expected PFM found in relevant guidance for each application. The applications. For example, for xLPR applications related to NRC may decide to address acceptance criteria leak-before-break, the acceptance criteria presented in separately in the future, on an application-specific ML21217A088 could be referenced. basis. For example, for leak-before-break, NRC could PFM is often used to investigate both structural integrity and develop a RG with acceptance criteria leak tightness of pressure boundary components. Structural integrity is assessed in terms of the calculated frequency of unstable rupture, while leak tightness is considered through the calculated frequency of through-wall crack penetration and leakage. Risk-informed decision making principles in accordance with RG 1.200 and RG 1.174 do not appear suited for development of acceptance criteria for leak tightness and the calculated frequency of small pressure boundary leaks that do not have direct safety consequences.

Could the Reg Guide discuss NRCs position on acceptance criteria for the calculated frequency of small pressure boundary leaks that do not have direct safety consequences?

4

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

14. Walter C, page What are the limits of the approval for these three codes? The NRC agrees in part and disagrees in part with this 12,13 Can references be added documenting NRCs approval of comment. There are no specific documents approving FAVOR and xLPR, along with the use case(s) and ranges of the NRC codes, but their validation regions are inputs for which these codes are considered approved? described in each codes manual. For clarity, Considering that NRC has approved these codes, what references to the manuals for xLPR and FAVOR were additional actions would be required of a user to accept them added and the SE approving SRRA is now referenced to be exercised under their Appendix B QA program to as well.

perform analyses in support of a licensing submittal? Does the NRC approval process sufficiently assess the As long as applicants follow their Appendix B QA effectiveness of the developmental and maintenance SQA program, they meet the requirements of Appendix B.

program such that the user may then cite NRC approval as Applicants need to follow their implementation of the basis for code acceptance under their Appendix B Appendix B programs.

program?

The only NRC-approved codes mentioned are xLPR, The NRC did not write an SE approving the VIPER or FAVOR and SRRA. A question was asked during the VIPER-NOZ codes generally. The NRC allowed their August 10, 2021 ACRS meeting (ML21223A043): What use for a specific application. For Clarity, the constitutes an NRC-approved code? The response by the following statement was added to the RG: there may NRC was that the above-mentioned codes were included, be instances where a code was approved just for a but also codes where a safety evaluation report would have specific application, and these are considered been written for a code for a specific application. There have approved for the same exact type of application.

been other PFM codes that have been used in reports where This sentiment is true for xLPR and FAVOR as well.

a NRC safety evaluation was written. These include: Thus, other clarifying edits were made to Section 2.4

• The VIPER code was used for RPV shell weld evaluations of the RG.

in BWRVIP-05 which has an NRC Safety Evaluation

• The VIPER-NOZ code was used for RPV nozzle evaluations in BWRVIP-108-A and BWRVIP-241 A which both have an NRC Safety Evaluations 5

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

15. Walter C, page 13 The intent of these tables when the comment column is The NRC agrees with this comment. The multiple subdivided into multiple entries for a given Category is entries were removed and bullet points were added to potentially ambiguous. To ensure they are viewed as help reduce the ambiguity. All tables were adjusted to complimentary to each other rather than alternative this format.

selections, clarify the intent that the applicability of all listed comments should be considered when planning a PFM application.

16. Walter C, page 13 Can references be added documenting the validated range of Yes. A reference has been added.

use for the NRC-approved codes?

17. Walter C, page 14 QV-1B is by definition outside a validated range; therefore, The NRC agrees with this comment. The language for how can this be within the same validated range. The M-1 and M-2 has been revised per the definitions of Submittal Guideline implies the model is within a validated QV-1A and QV-1B.

range.

6

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

18. Walter C, page 14 Quantitative goodness-of-fit-analyses were not performed The NRC agrees with this comment. The suggested for all xLPR modules and should not be required for models language has been incorporated into the RG.

in category M-2. For xLPR, module validation was performed to assess the predictive capability of modules relative to a combination of plant operating experience, laboratory data, alternative software/models, and/or engineering judgment. Although quantitative metrics were encouraged for xLPR module validation, visual assessment (e.g., plotting module predictions along with data being applied for validation) was also considered an appropriate technique. Thus, we believe similar flexibility should be included in the submittal guidelines for models in category M-2.

Suggest updating guideline to:

document a comparison of model predictions for the entire new range to applicable supporting data, predictions made using alternative models, and/or using engineering judgment, optionally supported using quantitative methods such as goodness-of-fit analyses

19. Walter C, page 14 What are the criteria of a well-established model? The NRC agrees that the criteria for a well-established model should be clearer. To clarify, a guidline was added that states, Provide justification for model as being well-established by supporting references and engineering judgement.

20. Walter C, page 15 Should this be sensitivity analyses or both? No, this sentence is correct as written. The first sentence refers to sensitivity analyses; the second is correctly stated as sensitivity studies.

7

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

21. Walter C, page 15 Although the term important variable is defined in the The NRC disagres with this comment.

glossary as a variable whose uncertainty contributes substantially to the uncertainty in the response, additional The NRC recognizes that there is subjectivity in what guidance on identification of important variables is constitutes an important variable, and believes that needed. Such additional guidance may consist of either more specific criteria cannot be defined that would be objective criteria defining important variables or input applicable in all cases. Engineering judgement should regarding the NRC expectation for the number/fraction of be used by applicants when deciding which variables included variables that are important. For example, if all are important. No change was made to the RG.

inputs are considered to be important, one could also make the argument that none of the inputs are important.

22. Walter C, page 16 Remove first line in the table caption. Also, update List of The NRC agrees with this comment. Revisions were Tables in page 30. made as suggested.

23. Walter C, page 16 In addition to listing distribution type and parameters, if The NRC agrees with this comment. Sampling applicable, sampling frequency (e.g., component-to- frequency was added to table C-5 in the RG component, within-component, flaw-to-flaw) should also be andSections 3.2.2 and 3.3.2 of NUREG/CR-7278.

listed.

8

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

24. Walter C, page 16 Depending on the application, some input values may be The NRC agrees with this comment. NRC defined conservative for some QoIs while being nonconservative for QoI to be singular, so assessment of the other QoIs. For example, a larger bending moment tends to conservatism in input values should be conducted for be conservative in that it results in a smaller critical each QoI of interest.

circumferential crack size, but it also has nonconservative effects in that it results in larger leak rates that are more The proposed edits have been incorporated.

easily detectable for a given through-wall circumferential crack size. We suggest that the RG acknowledge this possibility.

It is suggested to update the text to:

along with any known conservatisms or non-conservatisms in that numerical value and the rationale for such conservatisms or non-conservatisms.

and including any known conservatisms or non-conservatisms in the specified input distributions and the rationale for the conservatism or non-conservatism.

25. Walter C. page 16 These two guidelines seem quite parallel and should be The NRC agrees with this comment. The suggested consolidated into one guideline. Suggested wording: If revision has been incorporated.

applicable, list uncertainty classification (aleatory or epistemic) and provide the corresponding rationale.

9

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

26. Walter C, page 18 It is possible that in exercising an NRC-approved code The NRC agrees with this comment. The suggested outside of the validated range, discretization may also be revision has been incorporated.

impacted. Suggest adding the following sentence to this bullet:

This verification should also be documented in cases where the use of a QV-1 code exercised outside of the validated range (i.e., QV-1B) may directly impact discretization convergence.

27. Walter C, page 18 These descriptions were initially difficult to follow. A The NRC agrees that the descriptions could be hard to decision tree/flowchart may be a more effective method for follow. Figure C-2 was added for clarity to represent defining each of these categories (e.g., first split being the decision tree.

acceptance criteria met with at least one order of magnitude of margin vs. less than one order of magnitude of margin).

28. Walter C, page 19 Correct imbalanced square brackets. The NRC agrees with this comment. The suggested revision has been incorporated.

29. Walter C, page 20 Revise sentence to the applicant should determine the The NRC agrees with this comment. The suggested sensitivity analysis category for each PFM analysis and revision has been incorporated.

document

30. Walter C, page 21 As QV-1 is a header, suggest only listing QV-1A. The NRC agrees with this comment. The suggested revision has been incorporated.

10

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

31. Walter C, page To ensure that the same inputs are applied, it is suggested The NRC agrees with the comment that the definition 21,24 to include a reference to the previously approved inputs for of same inputs should be clarified. Added language QV-1A codes. explaining that that Qol refers to the characteristic (not Please define what is intended by same inputs. As used, the actual numerical value) and the refers to the the context leaves it subject to interpretation. On one parameter chosen (not the actual parameter value).

extreme, numerically identical inputs would make it impractical to evaluate a new problem. Alternatively, the intent may be that new inputs must not have been introduced to the input set of the NRC approved code version.

Additional guidance to clarify the intent regarding the definition of same is needed.

32. Walter C, page 21 Where would QV-1B fall? QV-1C should be identified as The NRC agrees with the comment regarding QV-1C, part of SA-3 and SA-4. QV-2 and QV-3 should be identified QV-2, and QV-3 and thetable was updated as part of SA-5 and SA-6. accordingly. QV-1A and QV-1B fall under SA-1 and SA-2.

33. Walter C, page 21 What additional documentation should be included? Based on the comment, the NRC beleives that the guidance is not precise enough and should be further clarified. In response, the reference to additional information was removed, and the guidance for category SA-5 was updated to say, See the submittal guidelines for SA-3.

34. Walter C, page 22 'Or' for guideline directly above or the entire row above? It Based on the comment, the NRC believes that futher seems like submittal guidelines for O-1 should be included clarification is needed. In response, brackets were for O-3 regardless of the selection. added to clarify the reference to or.

35. Walter C, page 24 This guideline is already included in Table C-2 and does not The NRC agrees with this comment. The subject seem relevant for this table. guidline has been deleted.

11

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

36. Walter C, page 14 When the idea for what became xLPR was being refined, a The NRC agrees with the comment that modular modular approach that placed the relevant design approaches are beneficial. No action needed.

phenomenological models into generally self-contained modules was selected to facilitate code modification should analytical needs or knowledge in these individual areas evolve. However, Category M-1 highlights an unexpected benefit realized from this early modularity decision. Each xLPR module was independently verified and validated establishing confidence in the underlying models and investing them with a degree of portability beyond xLPR.

Already several other PFM applications supported by EPRI have relied on the V&V pedigree of xLPR modules and repurposed them to address problems outside that for which xLPR is applicable. While clearly not relevant for every software development project, in a regulated environment where confidence in analytical models must be rigorously established, such a modular design approach can also have significant secondary benefits.

37. Walter Draft There is no mention in this document about presubmittal The NRC disagrees with this comment. Such NUREG/CR meetings with the NRC. Presubmittal meetings are guidance is not appropriate in the NUREG.

-7278, encouraged in Draft Guide 1382, Section 2.2. Guidance General should be given in NUREG/CR-7278 as to the timeliness of With that said,language has been added in Section 2.2 Comment these meetings. For example: Should they be done early in of the RG to indicate that it may be desirable to have the project lifecycle or not until the submittal is ready to be the presubmittal meetings early in the lifecycle of a sent to the NRC? project, but presumbittal meeting timing is ultimately left up to the applicant.

38. Walter Draft This sentence on lines 47-48 appears to address a slightly The NRC agrees with this comment. A separate bullet NUREG/CR different topic than the rest of this bullet. Should it be a has been included.

-7278, page separate bullet?

xiii 12

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

39. Walter Draft Insert missing carriage return between defined acronyms for The NRC agrees that a return is missing. Revised as NUREG/CR BIC and CDF. suggested.

-7278, page xv

40. Walter Draft Of the listed examples, cold head cracking appears to be a The NRC agrees with this comment that cold head NUREG/CR vaguely described subset of primary water stress corrosion cracking is a subset of PWSCC. As a result, cold head

-7278, page cracking that is more fully described as the first example. cracking was deleted from the example list.

3 Additional information should be included if cold head cracking is intended to describe something other than PWSCC. Otherwise consider deleting this example from the list.

41. Walter Draft Consider also referencing the BWRVIP as well since several The NRC agrees that the BWRVIP should be added as NUREG/CR of their reports are also listed. a reference. Revised as suggested.

-7278, page 3

42. Walter Draft The latest revision of this report is MRP-335 R3-A (EPRI The NRC agrees that the latest revision of the report NUREG/CR 3002009241). Suggest updating this reference to reflect the should be referenced. The reference has been updated

-7278, page latest revision of this report. to latest version of the report.

3,5

43. Walter Draft Delete extra period at the end of the sentence. The NRC agrees that the additional period should be NUREG/CR deleted. Revised as suggested.

-7278, page 7

13

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

44. Walter Draft This statement appears to be a bit broader than either the The NRC agrees with the comment. Language has NUREG/CR Purpose of the draft RG presented on page 1 or what this been added to clarify that this is acceptable to use as

-7278, page document actually contains. Preparing a thoughtful, a piece of evidence for design-basis changes.

7 competently documented PFM analysis is only one element in risk-informed decisionmaking. It is not by itself guidance on the overall RI decisionmaking process. Consider revising this statement.

45. Walter Draft Table 2-1 was very useful in facilitating our review of DG- No action taken in response to this comment.

NUREG/CR 1382 and NUREG/CR-7278.

-7278, page 7

46. Walter Draft To be consistent with Table C-1 of Draft Guide 1382, add The NRC disagrees with this comment. Table C-1 is NUREG/CR NUREG/CR-7278 Section 2.2.1 to RG Section 2.3, Section specific to mapping content to section 3 of the

-7278, page 2.2.2 to RG Section 2.4 and Section 2.2.3 to RG Section 2.5. NUREG. Draft Guide 1382 included only the content 8 related to section 2. This mapping is deliberately as-is and therefore no changes have been made.

47. Walter Draft For RG Section 2.1, EPRI White Paper Table 1 Item 9 and The NRC agrees in part and disagrees in part with this NUREG/CR Table 2 Items 7 and 8 are relevant. For Section 2.9, EPRI comment. For the lines corresponding to RG sections

-7278, page White Paper Table 1 Item 5 is relevant. For Section 2.10, 2.1 and 2.9, NRC Agrees with the comment, and items 8 EPRI White Paper Table 1 Item 7 is relevant. were added in Table 2.1 of NUREG/CR-7278 as suggested in the comment. For the line corresponding to RG section 2.10 (output uncertainty characterization), NRC disagrees that EPRI white paper Table 1 Item 7 is relevant

48. Walter Draft Delete extra periods at the end of the sentence. The NRC agrees that the extra period should be NUREG/CR deleted. Revised as suggested.

-7278, page 9

14

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

49. Walter Draft "Uncertain assumptions" sounds somewhat redundant. The NRC agrees with the comment. Updated NUREG/CR Consider whether this is the best wording to use here. uncertain assumptions to various assumptions.

-7278, page 13

50. Walter Draft Leak detection does not impact the probability of leakage. The NRC agrees with this comment. - Leak detection NUREG/CR Suggest changing the last sentence of this paragraph to: has been deleted.

-7278, page For example, no mitigation and 10-year inspection intervals 14 both impact the assessments.

51. Walter Draft 10 CFR 50 Appendix B, which requires NQA-1, represents The NRC agrees in part and disagrees in part with the NUREG/CR the highest level of QA for software development. The comment. In general, applicants are required to follow

-7278, page xLPR and FAVOR codes have not been developed to this their appendix B program. This RG does not exempt 15 standard. As noted in the Draft NUREG text, the SQA applicants from the requirements of 10 CFR 50 process should follow a graded approach. Thus, for some Appendix B.

uses of PFM models, it may not be necessary to meet the most stringent requirements for all aspects of the software Whether NQA-1 applies depends on the applicants development. Appendix B program. The guidance in section 2.4 of An explanation should be provided indicating that it is not the RG does not require NQA-1 compliance, and the expectation that all SQA requirements per the currently instead describes a graded approach for SQA and NRC-approved versions of NQA-1 apply in all cases to PFM V&V activities.

software that falls under the purview of this NUREG and Reg Guide. The NRC agrees that FAVOR and xLPR were not strictly developed to NQA-1 standards.

No change made in response to this comment.

52. Walter Draft If there is no experimental data for validation of the current The NRC agrees with this comment. The language has NUREG/CR code, then there presumably wasn't any such data for that been revised to, with comparable software that has

-7278, page same problem aspect when other software reached the been verified (and ideally validated) previously.

15 validation step.

15

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

53. Walter Draft The words used in analysis appear to add very little to this The NRC agrees with this comment. -The words NUREG/CR sentence. Consider deleting them. used in analysis have been deleted.

-7278, page 15

54. Walter Draft This bullet appears focused on the base code while the next The NRC agrees with this comment. The sentence has NUREG/CR bullet addresses problem-specific changes to that code. been deleted.

-7278, page Therefore, it appears inconsistent to include the following 16 sentence here:

Have the physics models been changed for the specific application?

55. Walter Draft This sentence mixes a general description "purely The NRC agrees with this comment. The language NUREG/CR deterministic analysis" with a specific one - "PFM analysis". has been revised as suggested.

-7278, page Not all probabilistic analyses are fracture mechanics so it 18 would be more consistent to replace PFM with "probabilistic."

56. Walter Draft Consider deleting "These" from the sentence. It seems to The NRC agrees with this comment. The language NUREG/CR imply a specific set or class of sensitivity studies that has been revised as suggested.

-7278, page appears both unnecessary and potentially confusing.

18

57. Walter Draft KIC should be defined as the plane strain fracture toughness, The NRC agrees with this comment. The language NUREG/CR consistent with ASME Code Section XI and other industry has been revised as suggested.

-7278, page documents.

19

58. Walter Draft The broken cross-reference (Reference 0) should be re- The NRC agrees with this comment. The broken NUREG/CR linked with the intended reference. cross-reference has been updated.

-7278, page 35 16

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

59. Walter Draft Although such different conclusions can be drawn when The NRC agrees in part and disagrees in part with this NUREG/CR separating or when not separating epistemic and aleatory comment. The mean results obtained likely would not

-7278, page uncertainties, it is important to also point out that the mean be identical due to sampling, but the results should be 38 results obtained using either approach should be the same similar. Thus, a sentence has been added for clarity, (e.g., see the results and conclusions drawn from Case which states, Regardless of the approach for 1.1.23 documented in ML21217A088). preserving the separation of uncertainties or not, the mean results using either approach should be similar.

60. Walter Draft Please provide examples of goodness-of-fit hypothesis test, The NRC agress that it would be helpful to add a few NUREG/CR such as Student t-test examples of goodness-of-fit hypothesis tests. A few

-7278, page examples have been added.

43

61. Walter Draft Row numbers 7 through 10 are not left justified. The NRC agrees with the comment. However, the NUREG/CR line numbers will not appear in the final document, so

-7278, page no correction is needed.

71

62. Walter Draft Suggest rewording this summary paragraph to indicate that The NRC agrees with this comment. In response to NUREG/CR the subsections of Section 2.2 only cover a handful of topics this comment additional wording was added stating,

-7278, page from DG-1382. The topics included are quantities of interest Section Error! Reference source not found. is 99 and acceptance criteria, software quality assurance and intended for applicants of all experience levels. Each verification and validation, as well as models. Also indicate subsection introduces an element of content that that the remaining topics (regulatory context, information would be expected in a PFM submittal and only made available to NRC staff, PFM software, supporting covers a handful of topics from [RG-1.245]. Topics documents, inputs, uncertainty propagation, convergence, include: QoI and acceptance criteria, software quality sensitivity analyses, output uncertainty characterization, and assurance, verification and validation, and models.

sensitivity studies) are discussed directly in DG-1382. Remaining topics are discussed directly in [RG-1.245]. It identifies representative circumstances for a submittal and describes a graded approach for the specific information to provide to the NRC.

17

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

63. Walter Draft Limiting this definition to only experimental data seems The NRC agrees with this comment. The word NUREG/CR overly restrictive. Presumably experimental data will be experimental has been removed.

-7278, page better controlled and characterized but there are times where 101 calibrating a model to well-characterized field data is appropriate as well.

64. Walter Draft Suggest replacing the L SEP symbols shown with regular The NRC agrees with this comment. The language NUREG/CR spaces between the words systematic method and for has been revised as suggested.

-7278, page assessing.

106

65. Draft In page 22 first paragraph, it is said that LHS is considered a In this context, targeted simply implies a method Sallaberry NUREG/CR targeted sampling methods. My understanding of a targeted selectively chosen to decrease the number of

-7278, page method is that the sampling density is changed to cover a realizations for convergence, per the previous 22 specific region of the input space to better cover the sentence - no change made.

statistics of one output of interest. LHS stratification is output independent and thus does not seem to match the definition of a targeted method (but maybe my definition is not correct).

66. Draft Furthermore, quasi-Monte Carlo Techniques (such as Halton The methods from this comment are considered to be Sallaberry NUREG/CR Sequences and Lp-Tau) work similarly to LHS by trying to too specific for this purpose. The references should be

-7278 minimize the distance between two points and better cover able to point the user to the appropriate method and a the sample space without prior knowledge of the model and literature review is outside the context of this report.

outputs of interest. Are these methods covered or are they considered to be too much specific to be included in the document?

18

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

67. Draft In section 4.2.2 (p. 44-48) rank correlation and functional These methods are not intended to be prescriptive, but Sallaberry NUREG/CR relationship are presented as methods to insure good merely examples for this report. No, copula has not

-7278, page representation of physical relations between inputs. Have been considered as a potential method to relax some 44-48 copula been considered as potential method that relaxed of the correlation constraints.

some of the correlation constraints ?

68. Hall C, page 9 Figure C-1: step 4: Determine a set of sensitivity studies The NRC agrees with this comment. - Sensitivity could be performed in the analysis plan. studies could be part of the analysis plan, but this is not always the case. Language was added in parentheses in step 4.

69. Hall C, page 9 Figure C-1: step 5: Iterate on the analysis process to refine The NRC agrees in part and disagees in part with this model results; the analysis plan should have an established comment. The analysis plan should have an acceptance criteria. Suggest deleting this bullet. established acceptance criteria, but this bullet 2.3, 4th bullet: What does rare probability mean? e.g.: emphasizes that it is likely to be an iterative process.

Value or distribution In response to this comment, rare probability was revised to a probability in the extreme tails of the distribution

70. Hall C, page 14 Table C-3: M-4: Does 'well-established' mean published in a No, well-established does not necessarily mean peer-reviewed journal (see M-5)? Well established could be published in a peer-reviewed journal (although internal to a company and not necessarily to the entire publication in a peer reviewed journal would qualify industry. as well-established). The NRC agrees with the 2.8, 4th line would not change significantly, if comment that well-established could be internal to a company and not the entire industry. No change was made in RG-1.245 as a result of this comment: the text in table C-3 of the RG for category M-4 provides the necessary flexility as-is.

The suggested addition of the word significantly to Section 2.8, Line 4 has been incorporated in the RG.

19

Commenter Section of Specific Comments NRC Resolution DG-1382 or NUREG/CR

-7278

71. Hall Glossary, Important variable is not used in the text. The NRC agrees with this comment that important page 26-27 Random variable is only used in other definitions. variable is not used in the text. Updated the glossary to instead define important input variable. .

20