View source for PWROG-19027



* Rev 2: [[URL::https://www.nrc.gov/docs/ML2021/ML20213C660.pdf]]
* Rev 0: [[URL::https://www.nrc.gov/docs/ML1927/ML19273A417.pdf]]

Portions endorsed by [[Regulatory Guide::Regulatory Guide 1.200]] revision 3.

=text=
{{#Wiki_filter:

==1 INTRODUCTION==
The Pressurized Water Reactor (PWR) Owners Group (PWROG) Project Authorization (PA)
RMSC-1647 (Reference 1) supported the interaction between the PWROG Risk Management
Committee (RMSC), the United States Nuclear Regulatory Commission (USNRC), the Boiling
Water Reactor Owners Group (BWROG) and the Nuclear Energy Institute (NEI) for the
development of technical requirements that can be used evaluate the technical adequacy of a
Newly Developed Method (NDM) to be adopted in a plant Probabilistic Risk Assessment (PRA).
The objective of this program is to provide clear criteria for the review of NDMs used in support
of risk informed applications. This PWROG project develops the definitions, processes and
technical requirements necessary to implement NDMs. Concurrent and parallel modification by
the NEI of the PRA peer review process documented in NEI 17-07 (Reference 2) accommodates
the review of new methods. This report includes the resolution of comments from the USNRC
that were discussed at a public meeting on August 21st, 2019 (Reference 3).
Ultimately, the intent is that the requirements for the newly developed method be added in the
future edition of the Standard for Level 1/Large Early Release Frequency Probabilistic Risk
Assessment for Nuclear Power Plant Applications that is being developed by the American
Nuclear Society (ANS) and American Society of Mechanical Engineers (ASME) Joint Committee
on Nuclear Risk Management (JCNRM).
Revision 2 of this report considers feedback from the JCNRM following the December 2019 ballot
for the next edition of the Standard and interaction between the PWROG and commenters of the
PRA Standard.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 2-1
PWROG-19027-NP July 2020
 Revision 2
2 DEFINITION OF NEWLY DEVELOPED METHOD
A number of operating definitions are needed to support the development of the NDM
requirements and review process. The following definitions are therefore proposed to be included
in the update of Section 1-2.2 of the ASME/ANS PRA Standard. For each definition, multiple
sources were used, including the current versions of the PRA Standard (Reference 4), the
combined NEI peer review guidance (Reference 2), the USNRC Regulatory Guide (RG) 1.200
(Reference 5), and the ANS glossary (Reference 6). The resulting proposed definitions take into
considerations all these sources and modify the definitions as judged to be needed for the
purposes of the PRA Standard.
PRA Upgrade: A change in the PRA that results in the applicability of one or more
Supporting Requirements (SRs) that were not previously included within the PRA (e.g.,
performing qualitative screening in Part 4 when this HRL was previously not applicable or
the addition of a new hazard model), an implementation of a PRA method in a different
context, or the incorporation of a PRA method not previously used.
PRA Method: An analytical approach used to satisfy a supporting requirement or
collection thereof in the PRA. An analytical approach is generally a compilation of the
analyses, tools, assumptions, and data used to develop a model.
Model: A qualitative and/or quantitative representation that is constructed to portray the
inherent characteristics and properties of what is being represented (e.g., a system,
component or human performance, theory or phenomenon). A model may be in the form,
for example, of a structure, schematic or equation. Method(s) are used to construct the
model under consideration.
PRA: A quantitative assessment of the risk, including technical elements for modeled
hazards, associated with plant operation and maintenance that is measured in terms of
frequency of occurrence of risk metrics, such as core damage or a radioactive material
release and its effects on the health of the public [also referred to as a probabilistic safety
assessment (PSA)]
PRA Maintenance: A change in the PRA that does not meet the definition of PRA
upgrade.
State-of–Practice: Those practices that are widely implemented throughout the
commercial nuclear power industry, have been shown to be technically acceptable in
documented analyses or engineering assessments, and have been shown to be
acceptable in the context of the intended application.
Consensus Method/Model: A method/model that the USNRC has used or accepted for
the specific risk-informed application for which it is proposed.
The above definitions allow for the definition of an NDM as follows:
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 2-2
PWROG-19027-NP July 2020
 Revision 2
Newly Developed Method: A PRA method that has either been developed separately
from a state-of-practice method or is one that involves a fundamental change to a stateof-practice method. A newly developed method is not a state-of practice or a consensus
method.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 3-1
PWROG-19027-NP July 2020
 Revision 2
3 USE OF NEWLY DEVELOPED METHODS IN PLANT PRA
A new method can be used to expand or refine an existing PRA but is always envisioned to be an
upgrade, which implies a focused scope peer review of an existing PRA. If a new method is used
in a PRA, it is necessary to confirm whether the new method that is applied meets the definition
of newly developed method.
With the definitions discussed in the previous section, the process described in the following
Figure 3-1 can be therefore considered.
Figure 3-1: Peer Review Determination Process
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 3-2
PWROG-19027-NP July 2020
 Revision 2
Step 1a: Does the change apply a method not previously used?
Based on the definition of a PRA method that is provided above, in this Step, the analyst should
review the change to determine if a new analytical approach was used to meet any SRs.
Step 1b: Does the new method is a consensus method
If the method was already successfully reviewed through the NDM review process or is a
consensus method, then there is no need for assessment of its technical adequacy and only a
peer review of the application of the method is needed.
Step 1c: Does the change represent a “Newly Developed Method”?
In this step, the analyst should evaluate whether the change would represent a Newly Developed
Method as defined above. If the new method simply represents implementation of a state-ofpractice method that was not previously implemented in the plant PRA being changed, this would
be categorized as a PRA Upgrade.
Step 2: Are any new SRs applicable (including new Hazards)?
This step defines a PRA upgrade as those PRA changes that support any SRs that were
previously not reviewed or were previously not applicable (N/A) are now applicable and should
be reviewed. It is noted that this includes new hazards or new parts of the ASME/ANS PRA
Standards.
If the original PRA Peer review was performed against Capability Category (CC) CC-I of the
standard, any PRA changes that were made that support SRs moving from CC-I to CC-II are
considered an upgrade.
The definitions of “state-of-practice”, “consensus method”, “PRA method”, and “newly developed
method” are provided in Section 2 and are used to decide whether a PRA method needs to
undergo a dedicated peer review against the requirements presented in Section 4.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 4-1
PWROG-19027-NP July 2020
 Revision 2
4 REQUIREMENTS FOR NEWLY DEVELOPED PRA METHODS
This section summarizes the recommendations to JCNRM pertaining to the definition of a new
set of requirements for newly developed PRA methods and specific expectations for the peer
reviews of such methods. The generic expectation is that newly developed methods satisfy the
set of requirements discussed in this section. This section is to be considered in conjunction with
the proposed definitions of state of practice and of newly developed method; based on these
definitions (see Section 2) it is clear that there is no expectation to retroactively apply these
requirements to every method used in every PRA, however, it is expected that methods used in
a PRA will generally meet these requirements in principal.
The requirements have been discussed in dedicated workshops with PWROG, BWROG, NEI,
USNRC and JCNRM representatives. Three (3) pilots peer reviews have been performed
following an initial drafting of the requirements and of the peer review expectations (see
References 7, 8 and 9). The requirements documented in this report incorporate feedback and
lessons learned from such pilots as well from a review performed by JCNRM members not
involved in the development of the requirements.
It is recommended that requirements for assessing the technical adequacy of a newly developed
method used in PRA be documented in a new section (e.g., Section 1-7) of Part 1 of the PRA
Standard. The following wording is recommended to be added in the new section.
Section 1-7
Newly Developed Methods
1-7.1 INTRODUCTION
This Section states requirements for newly developed
methods explicitly developed for use in PRA to support
risk-informed decisions for nuclear power plants. The high
level and supporting requirements for the Newly
Developed Methods are contained in Tables 1.7.2-1
through 1.7.2-7.
1-7.2 OBJECTIVE
The objective of the newly developed method
requirements is to ensure that a newly developed
method is technically adequate and are as follows:
a) The newly developed method has clearly
defined scope and limitations
b) The newly developed method is based on
sound engineering and relevant science
c) The newly developed method has proper
treatment of assumptions and uncertainties
d) The newly developed method is based on
appropriate and well understood data
e) The newly developed method produces results
that are consistent with expectations
f) The newly developed method is clearly
documented in such a way that knowledgeable
personnel can understand them without
ambiguity and that there is enough
documentation so that it can be peer reviewed.
The objectives above are intended to be applicable
to a large spectrum of methods, although it is
understood that not all the supporting requirements
could be applicable to all methods. In some cases,
depending on the method scope and purpose, some
of the SRs may not be applicable. In addition, the
SRs are designed to be able to address a stand-alone
method (i.e., independent from its implementation
on a specific plant PRA). It is recognized that, in
some circumstances, a method can be so plant or site
specific (especially in the external hazard domain)
that a full review of the method can only be
performed within its implementation. In such cases,
it is envisioned that some of the NM SRs could be
overlapping with Part specific SRs (e.g., SRs in Part
8). In such cases, the technical SRs in the appropriate
part may take priority to some NM SRs.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 4-1
PWROG-19027-NP July 2020
 Revision 2
Designator Requirement
HLR-NM-A The purpose and scope of the newly developed method shall be clearly stated.
HLR-NM-B The newly developed method shall be based on sound engineering and science relevant to
its purpose and scope.
HLR-NM-C The data (note that data can be numeric or non-numeric in nature) shall be relevant to the
newly developed method, technically sound, and properly analyzed and applied.
HLR-NM-D Uncertainties in the newly developed method shall be characterized. Sources of model
uncertainties and related assumptions shall be identified
HLR-NM-E
The results of the newly developed method shall be reproducible, reasonable and consistent
with the assumptions and data, and given the purpose and scope of the newly developed
method.
HLR-NM-F The documentation of the newly developed method shall provide traceability of the work
and facilitate incorporation of the newly developed method in a PRA model.
Table 1-7.2-2 Supporting Requirements for HLR-NM-A
The purpose and scope of the newly developed method shall be clearly stated (HLR-NM-A).
Index No.
NM-A Requirement
NM-A1
ENSURE that the stated purpose of the newly developed method (i.e., what is being
achieved by the newly developed method) is consistent with the scope (established
boundary) of the newly developed method.
NM-A2 ENSURE that the applicability and limitations of the newly developed method are
consistent with the purpose and scope in NM-A1.
NM-A3
Based on the limitations and applicability of the newly developed method, IDENTIFY
which areas of the PRA for which the newly developed method is intended to be used, and
those for which it is specifically not intended for (e.g., hazards, technical elements, plant
features, SRs impacted by the newly developed method).
Table 1-7.2-3 Supporting Requirements for HLR-NM-B
The newly developed method shall be based on sound engineering and science relevant to its purpose and scope
(HLR-NM-B).
Index No.
NM-B Requirement
NM-B1
ESTABLISH the technical bases for the newly developed method by using approaches
founded on established mathematical, engineering and/or scientific principles (e.g.,
established through operating experience, tests, benchmarking, or acceptance by the
scientific community).
NM-B2
If empirical models are used, ENSURE that they are supported by sufficient data which is
relevant to the newly developed method and, to the extent possible, that the experimental
data are shown to be repeatable.
NM-B3 IDENTIFY assumptions used to develop the technical bases of the newly developed
method.
NM-B4 JUSTIFY the rationale for the assumptions identified in NM-B3 (e.g., backed by appropriate
operational experience).
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 4-2
PWROG-19027-NP July 2020
 Revision 2
Table 1-7.2-4 Supporting Requirements for HLR-NM-C
The data (note that data can be numeric or non-numeric in nature) shall be relevant to the newly developed
method, technically sound, and properly analyzed and applied (HLR-NM-C).
Index No.
NM-C Requirement
NM-C1
IDENTIFY the data needed to support the development of the newly developed method
(e.g., relevant plant-specific data, industry-wide current operating experience and data, or
experimental or test data).
NM-C2 COLLECT relevant data consistent with current technical state-of-practice.
NM-C3 DEMONSTRATE that the data used, including experimental data or test data, is relevant to
and supports the technical basis of the newly developed method.
NM-C4 SPECIFY the basis for exclusion of data identified in NM-C1.
NM-C5 ANALYZE data (e.g., modifications to the data, use of data in a different context or beyond
the original ranges, statistical analysis) using technically sound basis or criteria.
NM-C6 ENSURE that data is applied consistent with the purpose and scope of the newly developed
method.
Table 1-7.2-5 Supporting Requirements for HLR-NM-D
Uncertainties in the newly developed method shall be characterized. Sources of model uncertainties and related
assumptions shall be identified (HLR-NM-D).
Index No.
NM-D Requirement
NM-D1
Note (1)
CHARACTERIZE the parameter uncertainties associated with the newly developed
method consistent with the intended scope and purpose of the method; this
characterization could include, for example, specifying the uncertainty range,
qualitatively discussing the uncertainty range, or identifying the parameter estimate
as conservative or bounding.
NM-D2 IDENTIFY the sources of model uncertainty associated with assumptions identified in
NM-B3.
NM-D3
CHARACTERIZE the model uncertainties (identified in NM-D2) associated with the
newly developed method; this characterization may be in the form of sensitivity
studies.
Notes:
(1) Depending on the purpose and scope of the method, uncertainty distributions may need to be
explicitly calculated to allow for application of a method for risk significant items to meet Capability
Category II of related technical SRs in other parts of the Standard.
Table 1-7.2-6 Supporting Requirements for HLR-NM-E
The results of the newly developed method shall be reproducible, reasonable and consistent with the
assumptions and data, and given the purpose and scope of the newly developed method (HLR-NM-E).
Index No.
NM-E Requirement
NM-E1
REVIEW the results from the newly developed method to determine that they are
reproducible, reasonable and consistent with assumptions and data addressed in the SRs
under HLR-NM-B and HLR-NM-C.
NM-E2 COMPARE the results of the newly developed method with existing methods and, when
possible, IDENTIFY causes for substantial differences.
NM-E3 ENSURE uncertainties do not preclude meaningful use of the newly developed method
results.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 4-3
PWROG-19027-NP July 2020
 Revision 2
Table 1-7.2-7 Supporting Requirements for HLR-NM-F
The documentation of the newly developed method shall provide traceability of the work and facilitate
incorporation of the newly developed method in a PRA model (HLR-NM-F).
Index No.
NM-F Requirement
NM-F1
DOCUMENT the newly developed method specifying what is used as input, the technical
basis and the implementation expectations and limitations by addressing the following, as
well as other details needed to fully document how the set of the NM SRs are satisfied:
(a) the purpose and scope of the newly developed method
(b) the intended use of the newly developed method
(c) the limitations of the newly developed method
(d) the detailed technical basis for the newly developed method
(e) the data source, collection process and data manipulation performed in support of
the newly developed method
(f) the assumptions and uncertainties associated with the newly developed method
(g) the interpretation of the results of the newly developed method in the framework of
the intended use and application
NM-F2
DOCUMENT the intended process by which the newly developed method can be applied
to a PRA model consistently with the intended use of the newly developed method and
taking into account the purpose, scope and limitations.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 5-1
PWROG-19027-NP July 2020
 Revision 2
5 PEER REVIEW FOR NEWLY DEVELOPED METHODS
This section discusses potential clarifications and modifications of the peer review program to
accommodate a peer review of method for which, using the criteria and definition defined above,
it is concluded that a peer review is needed. NEI 17-07 (Reference 2) addresses the specifics of
the peer review; this section is not presenting updates specific to NEI 17-07 but rather generic
clarifications on the process.
5.1 CHANGES TO THE PEER REVIEW SECTION IN THE PRA STANDARD
The following specific changes in Part 1 of the PRA Standard are recommended. Sections not
mentioned here below are considered fully applicable (i.e., no recommended changes).
It is recommended that Section 1-6.1.1 (Documentation and Self-Assessment) be removed
completely. The documentation of a self-assessment is part of the process and is captured in
NEI-17-07. It does not need to be included in the Standard.
It is recommended that Section 1-6.1.4 (Method) be rewritten as follows:
1-6.1.4 Peer Review Process
The review shall be performed using a written process that assesses the requirements of the
Technical Requirements section of each respective Part of this Standard and addresses the
requirements of the Peer Review Section of each respective Part of this Standard.
The peer review process shall consist of the following elements:
(a) selection of the peer review team
(b) training in the peer review process
(c) an approach to be used by the peer review team for assessing if the PRA meets the
supporting requirements of the Technical Requirements section of each respective Part
of this Standard
(d) management and resolution of potential differing professional opinions
(e) documentation of the results of the review
When included in the scope of a peer review, a newly developed method shall be reviewed
following the dedicated requirements discussed in Section 1-7.
It is recommended that Section 1-6.2.2 (Individual Team Members) be rewritten as follows:
1-6.2.2 Individual Team Members
The peer review team members individually shall be:
(1) knowledgeable of the requirements in this Standard for their area of review
(2) experienced in performing the activities related to the PRA Technical Elements for
which the reviewer is assigned
(3) independent from the team that developed the PRA model or the method being peer reviewed.
(4) Subject matter experts should be included to judge the technical adequacy of non-PRA
engineering evaluations and to confirm that the applicable envelope defining the limits of the
method are identified.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 5-2
PWROG-19027-NP July 2020
 Revision 2
It is recommended that Section 1-6.2.3 (Review Team Members for PRA Upgrades) be removed
completely.
It is recommended that Section 1-6.3 be rewritten as follows:
1-6.3 REVIEW OF PRA TECHNICAL ELEMENTS
The peer review team shall use the requirements of this section, as complemented by hazard
specific requirements presented in the Peer Review Section of each respective Part of this
Standard for the PRA. These hazard-group–specific requirements are stated in the
corresponding peer review section of each Part (e.g., 2-3.3, 3-3.3). The peer review team shall
review the technical requirements of the hazard group to determine if the method and the
implementation of the method for each PRA Technical Element meet the requirements of this
Standard. Additional material for those Elements may be reviewed depending on the results
obtained. The judgment of the reviewer shall be used to determine the specific scope and depth
of the review in each PRA Technical Element.
The results of the appropriate hazard group PRA, including models and assumptions, and the
results of each PRA Technical Element shall be reviewed to determine their reasonableness
given the design and operation of the plant (e.g., investigation of cutset or sequence
combinations for reasonableness).
The PRA configuration control program is reviewed against the requirements presented in
Section 1-5; any newly developed method included in the scope of the peer review is reviewed
against the requirements of Section 1-7. It is noted that a newly developed method can be peer
reviewed within the scope of a plant PRA (i.e., concurrently with its implementation in a plant
PRA) or via a dedicated stand-alone peer reviewed. In this second case the implementation of
the method is peer reviewed in a separate peer review.
Even if exceptions to the requirements of Section 1-6.2.4(c) occur, concerning the composition
of the peer review team or the duration of the review, all SRs relevant to the scope of the peer
review of the PRA are to be reviewed.
The extent of a focused-scope peer review includes all Supporting Requirements (e.g., not just
those for which Findings were cited), within the High Level Requirement(s) containing
Supporting Requirements with Findings. New Findings may be issued even for Supporting
Requirements that did not have previous Findings since a focused-scope peer review
encompasses all the Supporting Requirements within an affected High Level Requirement.
It is recommended that Section 1-6.6.1 be rewritten as follows:
1-6.6.1 Peer Review Team Documentation
The peer review team’s documentation shall demonstrate that the review process appropriately
implemented the review requirements.
Specifically, the peer review documentation shall include the following:
(a) identification of the version of the PRA reviewed
(b) a statement of the scope of the peer review
(c) the names of the peer review team members
(d) a brief resume on each team member describing the individual’s employer, education, PRA
training, and PRA and PRA Technical Element experience and expertise
(e) the elements of the PRA reviewed by each team member
(f) a discussion of the extent to which each PRA Technical Element was reviewed, including
justification for any supporting requirements within the peer review scope that were not
reviewed
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 5-3
PWROG-19027-NP July 2020
 Revision 2
(g) results of the review identifying any differences between the requirements in the Technical
Requirements section of each respective Part of this Standard and Section 1-5 and the method
implemented, defined to a sufficient level of detail that will allow the resolution of the
differences
(h) identification and significance of exceptions and gaps relative to the Standard’s requirements,
in sufficient detail to allow the resolution of the gaps that the peer reviewers have determined
to be material to the PRA
(i) an assessment of PRA assumptions that the peer reviewers have determined to be material to
the PRA
(j) differences or dissenting views among peer reviewers
(k) recommended alternatives for resolution of any differences
(l) an assessment of the CC of the SRs (i.e., identification of what CC is met for the SRs)
It is recommended that Section 1-6.6.2 be rewritten as follows:
1-6.6.2 Resolution of Peer Review Team Comments
Resolution of Peer Review Team comments (Facts and Observations (F&Os) shall be
documented. The resolution of the F&Os findings shall describe how the deficiency was
addressed such that the associated SR can now be demonstrated to be met.
Emphasis is given on whether an F&O is resolved via a PRA maintenance or a PRA upgrade.
A focused scope peer review is performed to address any PRA upgrade.
It is recommended that Section 1-6.6.3 be removed completely as it is now addressed by
Section 1-7.
There is no need to make any change in the hazard specific peer review sections (e.g.,
Section 2-3), as there is nothing hazard-specific that needs to be addressed in the review of a
method.

== 5.2 NEWLY DEVELOPED METHOD PEER REVIEW REPORT ==
This section discusses the structure and content of a NDM peer review report.
If the NDM is peer reviewed in conjunction with a plant PRA and is documented as part of the
plant PRA peer review, dedicated sections in the reports need to be created for the NDM review
portion of the review, including the same content discussed below.

=== 5.2.1 NDM Peer Review Report Structure and Content ===
The following is the recommended content of the report for a NDM peer review.

Introduction
Purpose
A short description of the purpose of the report.
Scope
The method that is being peer reviewed needs to be uniquely identified.

Also, it should be clarified which technical supporting requirements should be
peer reviewed when the NDM is applied to a plant PRA
Peer Review Process
Overview of Review Process
A short summary of the process used needs to be presented, mainly with
reference to the NEI peer review guidance and to the PRA Standard (or to the
document that contains the review requirements).
Peer Review Schedule and Reviewer Assignments
A short summary of the steps followed during the review process.
A very short biography of all the peer review team members that confirms the
reviewer’s qualifications and pre-requisites should be provided.
Reviewed Information
All the documentation reviewed in support of the NDM review should be listed.
Summary of Review Results and Observations
Method Characterization
The reviewer’s understanding of how the method that was reviewed qualifies
as a newly developed method, based on the NDM definition.
If a NDM is an evolution of existing methods, it would be helpful to provide
indications of whether the original methods were already peer reviewed (or
being part of a plant PRA that was peer reviewed)
Technical Adequacy
A summary of each SRs in the NDM section of the Standard is met, not met,
not reviewed or not applicable. A short rationale needs to be provided for each
SR that is judged not applicable.
For each applicable SRs, facts and observations linked to the SR should be
provided (this is a summary, although all detailed F&O wording needs also to
be provided).
Assessment of the Requirements
Documentation of the reviewer assessment for each SR that is judged
applicable. The assessment should explicitly indicate the basis for the SR
assessment as met or not met.

Facts and Observations
F&Os written against the method.
Conclusions
A summary of the overall report. A positive statement on the technical adequacy of
the method needs to be repeated for clarity in the conclusions.
Reviewer Resumes and Qualification Review
Documentation of the reviewer pre-requisites along with each review team
members resume.
The judgment of the review lead about the review team meeting the reviewer
qualification requirements.
The method owner acceptance of the review team and concurrence on the
qualification.

An appendix has to be provided that includes a short summary of the overall
review with no proprietary information. The minimum information needed in this
section is:
:a. Unique identification of the method reviewed
:b. Process followed for the review (i.e., reference to NEI guidance)
:c. Review team composition (full resume not required)
:d. Method characterization and technical adequacy summary
:e. SR assessments and associated basis
:f. F&Os wording and/or basis for closure of F&Os.
:g. List of SRs to be peer reviewed in a focused scope review applying the method
Alternatively, if the full NDM peer review report is determined to be non-proprietary, the full report can be made publicly available; in this case the non-proprietary summary would not be needed.

=== 5.2.2 Public Availability of NDM Peer Review Reports ===
While there is no expectation that the full NDM peer review report is made publicly available, the non-proprietary Appendix of the peer review report is expected to be made available to the public so that both the NRC and any utility applying in the future for the method can reference the fact that the method has been peer reviewed.

There is no mandatory vehicle to be used to ensure public availability of the non-proprietary
appendix of the peer review report but the following mechanisms can be considered as examples:
:a. The non-proprietary appendix of the peer review report can be made available on the web site of the method developer and owner organization (e.g., EPRI, PWROG, etc.)
:b. The appendix can be added in an Appendix to the method primary report (e.g., EPRI report)
:c. The appendix can be made available to the NRC to be loaded on ADAMS (no formal request of review or endorsement would be needed). Note that “publicly available” is not necessarily to be intended as “for free”.

== 5.3 PRA METHODS PEER REVIEW OUTCOME ==
The following Figure 5-1 summarizes the possible outcomes from an NDM peer review and the
repercussion on the use of the method in a plant PRA model. Beyond the proposed peer review
process, submittal of the method to the NRC for a direct assessment remains a valid option.
If the suggested peer review process is used to assess the technical adequacy of the method,
then two main possible outcomes from a newly developed method peer review are:
a. If all the NM SRs are MET and there are no open findings, the method is considered
technically adequate and can be implemented in a plant PRA. Following implementation,
the plant will undergo a focused scope peer review where the review team will assess if
the method is adequately applied to the plant PRA and is performed against the technical
supporting requirements in the relevant Part(s) of the PRA Standard.
b. If there are NM SRs that are NOT MET or there are open finding against some NM SRs,
the conclusion will be that the newly developed method is NOT technically adequate. In
this situation, it would not be recommended for a plant to apply the method in the plant
PRA. It is possible to envision that some of the open findings may be easy to disposition
(e.g., a minor documentation finding or related to a portion of a method that is not used by
the plant). It becomes the plant responsibility to address those findings in the application
of the method in the plant PRA.
The F&O closure process (see Appendix E of Reference 2) can be used to close findings
associated with an NDM. It is to be noted that the F&O closure process requires the
developer of the method (and the independent assessment team) to evaluate whether a
finding is closed through a maintenance or an upgrade activity (which would result in the
need for a follow up NDM peer review to be performed). Note that the operating definitions
of PRA upgrade and PRA maintenance may not be directly applicable to a PRA method.
However, the concept remains valid when considering that a PRA method maintenance
activity would be:
 a correction of an error that does not change the intent or the conclusions for
the method;
 the processing of more input data with the same process that does not change
in the intent of the conclusion of the method;
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 5-7
PWROG-19027-NP July 2020
 Revision 2
 the expansion of documentation for data and assumptions already used (but not
appropriately documented in origin);
 performance of more sensitivities to discuss uncertainties and or to confirm the
applicability of the method within the original intended range of application;
 clarification of the documentation in support to implementation of the method.
A PRA method upgrade activity would be an action that changes the intent of the method,
such as:
 extension/change of the scope/applicability of the method;
 a fundamentally different way to process input/output data (beyond usage of a
different tool to perform the same process function)
Following an F&O closure review and/or a focused scope peer review for the NDM, an
update of the summary information needs to be made again available to the public to
document the new technical adequacy assessment.
Figure 5-1: Newly Developed Method Peer Review Flow Chart
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 6-1
PWROG-19027-NP July 2020
 Revision 2

==6 CONCLUSIONS==
The PWROG worked with a number of stakeholders to define a set of technical requirements to
be associated to a method used in PRA for a plant. A number of definitions were also proposed
that would support a process for which newly developed methods could be explicitly peer
reviewed within the PRA peer review program to ascertain the technical adequacy of the method.
Specific changes to the upcoming edition of the ANS/ASME PRA Standard are proposed in this
report to support the NDM definition, technical requirements and per review process. This
document would enable the upcoming NEI peer review guidance to support and manage the
review of NDMs concurrently or before their implementation in a plant PRA.
An efficient process for the review of NDMs would support a more streamlined acceptance by the
USNRC of PRA changes in support to risk-informed application and license amendments.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
 WESTINGHOUSE NON-PROPRIETARY CLASS 3 7-1
PWROG-19027-NP July 2020
 Revision 2

==7 REFERENCES==
1. PA-RMSC-1647, R1 Model Maintenance vs Upgrade Workshop.
2. NEI 17-07, Performance of PRA Peer Reviews Using the ASME/ANS PRA Standard.
Revision 2, August 2019. (ADAM ML19228A242).
3. Public Meeting with the Nuclear Energy Institute to Discuss the Use of New Probabilistic
Risk Assessment Methods Following the Issuance of an Amendment to Utilize a RiskInformed Process. August 21st, 2019. (ADAM ML19207A443).
4. ASME/ANS RA-Sb-2013, Addenda to ASME/ANS RA-S-2008, Standard for Level 1/Large
Early Release Frequency Probabilistic Risk Assessment for Nuclear Power Plant
Applications.
5. USNRC Regulatory Guide 1.200, Revision 2, An Approach for Determining the Technical
Adequacy of Probabilistic Risk Assessment Results for Risk-Informed Activities. March
2009. (ADAM ML090410014).
6. American Nuclear Society, Standard Committee. Glossary of Definitions and Terminology
(as of April 2019). Available on the ANS web site: http://www.ans.org/standards/toolkit/
7. PWROG-19019-NP, Revision 0-A, Newly Developed Method Peer Review Pilot – EDG
Component Reliability Data Development – PA-RMSC-1647. June 2019.
8. PWROG-19020-NP, Revision 1, Newly Developed Method Peer Review Pilot – General
Screening Criteria for Loss of Room Cooling in PRA Modeling – PA-RMSC-1647. April 2020.
9. Draft Jensen Hughes report BD000000.000-RPT-01 – Newly Developed Method Peer
Review Pilot – fraction of Complete Fire Damage to a NUREG-2178 Vol 1 Group 4 Electrical
Cabinet. Draft 0a as of June 2019.
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
PWROG-19027-NP Revision 2 Proprietary Class 3
**This page was added to the quality record by the PRIME system upon its validation and shall not be considered in the page numbering of this document.**
Approval Information
Author Approval Maioli Andrea Jul-10-2020 18:01:02
Reviewer Approval Labarge Nathan R Jul-13-2020 13:01:32
Manager Approval Sadlon Dan Jul-13-2020 14:35:15
Manager Approval Mirizio Damian S Jul-27-2020 12:57:50
Files approved on Jul-27-2020
*** This record was final approved on 7/27/2020 12:57:50 PM. (This statement was added by the PRIME system upon its validation)
}}