ML083430462

NEI 05-04, Rev 2, Process for Performing Internal Events PRA Peer Reviews Using the Asme/Ans PRA Standard.

ML083430462
Person / Time
Site:	Nuclear Energy Institute
Issue date:	11/30/2008
From:	AREVA NP, GE-Hitachi Nuclear Energy Americas, Westinghouse
To:	Office of Nuclear Reactor Regulation, Office of Nuclear Regulatory Research, Nuclear Energy Institute
References
NEI 05-04, Rev 2
Download: ML083430462 (58)

Text

NEI 05-04, Rev. 2 PRA Peer Review Process NEI 05-04, Rev. 2 Process for Performing Internal Events PRA Peer Reviews Using the ASME/ANS PRA Standard Prepared for Nuclear Energy Institute (NEI)

Risk Informed Applications Task Force (RATF) and NEI PRA Peer Review Task Force Prepared by BWROG/General Electric Hitachi PWROG/Westinghouse Electric Co./AREVA NP Inc.

November 2008

NEI 05-04, Rev. 2 PRA Peer Review Process NOTICE Neither NEI, nor any of its employees, members, supporting organizations, contractors, or consultants make any warranty, expressed or implied, or assume any legal responsibility for the accuracy or completeness of, or assume any liability for damages resulting from any use of, any information, apparatus, methods, or process as disclosed in this report or that such may not infringe privately owned rights.

November 2008 ii

NEI 05-04, Rev. 2 PRA Peer Review Process ACKNOWLEDGEMENTS This report was prepared using information in a draft guidance document obtained through the Nuclear Energy Institute (NEI) Risk Applications Task Force (RATF). In particular, the CE Owners Group (CEOG) (now part of the Westinghouse Owners Group (WOG)) developed an initial follow-on peer review process, documented in Westinghouse WCAP-16091, upon which the NEI draft document is based and has made this information available to the other Owners Groups, through the auspices of NEIs RATF.

WCAP-16091 was prepared by David Finnicum of Westinghouse. The transformation of the original work into a draft industry document was accomplished by Barry Sloane (then with Westinghouse), Greg Krueger (ERIN Engineering, formerly of Exelon), David Miskiewicz (Progress Energy - Florida), David Finnicum (Westinghouse), and Stanley Levinson (AREVA NP). Additional review and input was provided by the RATF and other interested utility and industry personnel.

The draft NEI document, which envisioned a broader scope peer review with several options, was modified and simplified by Barry Sloane as WCAP-16181-NP, Revision 0, a draft document that was never officially released. Sloanes version was intended to be used for follow-on peer review of individual PRA technical elements, and included less detailed supporting material and process guidance. This simplification matched the guidance provided by NEIs RATF and was the impetus to create the original NEI document. This document was reviewed by the B&W Owners Groups (B&WOGs) Risk-Informed Applications Committee, NEIs RATF, and the original team of Sloane (Dominion Energy), Krueger, Miskiewicz, Finnicum, and Levinson. The comments provided by Earl Page are specifically acknowledged.

With revisions to Regulatory Guide 1.200 and the desire to support risk-informed application, there was a need to revise NEI 05-04. Revision 1 was produced incorporating some of the early lessons learned from the industry. NEI formed the PRA Peer Review Task Force, generally composed of Peer Review Team leads, to continue the identification of lessons learned, and to integrate them into Revision 2 of the NEI 05-04. The contributions to Revision 2 by Barry Sloane, David Finnicum, Greg Krueger, Dennis Henneke (General Electric), Stanley Levinson, Tom Morgan (Maracor), Ed Krantz (Curtiss Wright), are acknowledged and appreciated. In particular, the insights offered by Vince Andersen (ERIN Engineering) following the Vermont Yankee Peer Review are acknowledged.

Coordination of the PRA Peer Review Task Force and revision to NEI 05-04, as well as NEI 07-12 (PRA Peer Review process for Fire PRAs) has been ably performed by Victoria Anderson, NEI. Her involvement has helped keep this effort on track and to provide the industry with a timely revision of NEI 05-04.

November 2008 iii

NEI 05-04, Rev. 2 PRA Peer Review Process EXECUTIVE

SUMMARY

This document provides guidance material for conducting and documenting a peer review for Probabilistic Risk Assessments (PRAs) using the ASME/ANS PRA Standard RA-S-2008a (Revision 1, Addendum A). The original intend of NEI 05-04 was to provide a methodology for PRA Peer Reviews as a follow-on to the NEI 00-02 methodology. With the release of ASME and ANS Standards (to form the basis of a Peer Review) and with many operating plants and plants-to-be-built performing PRAs, the emphasis of this document has changed from follow-on peer reviews to simply peer reviews performed against an industry consensus standard..

Peer review has proven to be a valuable process for establishing technical adequacy of nuclear power plant probabilistic risk assessments (PRAs). All US plants have performed a peer review of their base PRA internal events, at-power model. PRA consensus standards continue to develop; the last being the Combined Standard (Revision 1) produced through a cooperative effort of ASME and ANS, which includes internal events, fires, and external events. Shortly after release, an Addendum was developed. The NRC is planning to endorse Revision 1 (and possible the Addendum as well).

With the NRC endorsement of ASME RA-Sc-2007 with RG 1.200, Rev. 1, plants submitting risk-informed applications have need to comply with RG 1.200, Rev. 1 since January 2008.

With the expected endorsement via RG 1.200, Rev. 2, there continues to be a need in the industry to perform follow-on PRA peer reviews, either full-scope or focused reviews. The need for these follow-on reviews is generally driven by significant changes or upgrades to a portion (e.g., a single PRA technical element) or all of the previously peer-reviewed PRA. This document provides a methodology for performance of follow-on peer reviews. In addition, with PRA being performed for new plants (and performing peer reviews), this document has been revised to clearly state the peer review methodology can be equally used for a plant that has not yet performed a PRA peer review.

The revision to NEI 05-04 was deemed necessary in light of the large number of focused and full (Internal Event) PRA peer reviews that are being conducted so that plant PRAs will conform with RG 1.200, Rev. 2. It is expected that as of January 1, 2010, all risk-informed applications will be expected to make reference to the ASME/ANS PRA Standard and the state of the PRAs technical adequacy to support the application.

November 2008 iv

NEI 05-04, Rev. 2 PRA Peer Review Process TABLE OF CONTENTS SECTION TITLE PAGE

1.0 INTRODUCTION

.........................................................................................................1 1.1 Purpose...............................................................................................................1 1.2 Background ........................................................................................................1 1.3 Scope ..................................................................................................................2 2.0 GENERAL OVERVIEW OF PEER REVIEW PROCESS .......................................... 4 3.0 ASSIGMENT OF CAPABILITY CATEGORIES ........................................................5 3.1 Assigning Capability Categories for Peer Reviews against the ASME PRA Standard ........................................................................................5 3.2 Comparison against Grading Process for NEI 00-02 ........................................6 4.0 PRA PEER REVIEW: ASME PRA STANDARD SCOPE ............................................9 4.1 Scope ..................................................................................................................9 4.2 Host Utility Requirements .................................................................................9 4.3 Self-Assessment ...............................................................................................10 4.4 Peer Review Team ...........................................................................................11 4.5 Peer Review Schedule......................................................................................12 4.6 Peer Review Process ........................................................................................12 4.7 PRA Peer Review Report.................................................................................16

5.0 REFERENCES

............................................................................................................17 TABLES Table 1 - Interpretation of Supporting Requirement .................................................................6 APPENDICES Appendix A - Sample Fact and Observation Form .............................................................. A-1 Appendix B - Sample Summary Tables ...............................................................................B-1 Appendix C - PRA Peer Review Orientation/Training Slides .............................................C-1 Appendix D - Example PRA Peer Review Lessons Learned Form ..................................... D-1 November 2008 v

NEI 05-04, Rev. 2 PRA Peer Review Process ACRONYMS AOT Allowed Outage Time AS Accident Sequence Analysis (PRA Technical Element)

ASME American Society of Mechanical Engineers B&WOG Babcock & Wilcox Owners Group BWR Boiling Water Reactor BWROG BWR Owners Group CE Combustion Engineering CEOG CE Owners Group CNRM Committee on Nuclear Risk Management DA Data Analysis (PRA Technical Element)

DE Dependency Analysis (PRA Technical Element)

EPRI Electric Power Research Institute F&O Fact & Observation (form)

GL Generic Letter HFE Human Failure Event HLR High Level Requirement HR Human Reliability (PRA Technical Element)

HRA Human Reliability Analysis IE Initiating Event, Initiating Event Analysis (PRA Technical Element)

IF Internal Flood, Internal Flood Analysis (PRA Technical Element)

ISI In-service Inspection IST In-service Testing LE Level 2 (LERF) Analysis PRA Technical Element)

LERF Large Early Release Frequency MOV Motor Operated Valve MU Maintenance and Update N/A Not Applicable NEI Nuclear Energy Institute NRC Nuclear Regulatory Commission PRA Probabilistic Risk Assessment PSA Probabilistic Safety Analysis QA Quality Assurance QU Quantification and Results Interpretation (PRA Technical Element)

R&R Risk & Reliability (Workstation)

SC Success Criteria SR Supporting Requirement SSC System, Structure, and Component SY System Analysis (PRA Technical Element)

TH Thermal Hydraulic Analysis TS Technical Specification WOG Westinghouse Owners Group November 2008 vi

NEI 05-04, Rev. 2 PRA Peer Review Process

1.0 INTRODUCTION

1.1 Purpose In the late 1990s/early 2000s, the nuclear utilities undertook a voluntary program of performing peer reviews of their plant-specific Probabilistic Risk Assessments (PRAs)a using the process defined in NEI 00-02 (Reference 1). The purpose of this guidance document is to provide a process for performing full scope and focused peer reviews of a PRA against the current ASME/ANS PRA Standard (RA-S-2008a), Revision 1, Addendum A (Reference 1) for utilities that have upgraded or significantly revised their PRAs.

This document provides guidance material for conducting and documenting a peer review for PRAs using the ASME/ANS PRA Standard. The original intent of NEI 05-04 was to provide a methodology for PRA Peer Reviews as a follow-on to the NEI 00-02 methodology. With the release of ASME and ANS Standards (to form the basis of a Peer Review) and with many operating plants and plants-to-be-built performing PRAs, the emphasis of this document has changed from follow-on peer reviews to simply peer reviews performed against an industry consensus standard. In addition, with PRA being performed for new plants (and performing peer reviews), this document has been revised to clearly state the peer review methodology can be equally used for a plant that has not yet performed a PRA peer review.

With revisions to Regulatory Guide 1.200 and the desire to support risk-informed application, there was a need to revise NEI 05-04. Revision 1 was produced incorporating some of the early lessons learned from the industry. The current revisions to NEI 05-04 were deemed necessary in light of the large number of focused and full (Internal Event) PRA peer reviews that are being conducted so that plant PRAs will conform to RG 1.200, Rev. 2. It is expected that as of January 1, 2010, all risk-informed applications will be expected to make reference to the ASME/ANS PRA Standard and the state of the PRAs technical adequacy to support the application. NEI formed the PRA Peer Review Task Force, generally composed of Peer Review Team leads, to continue the identification of lessons learned, and to integrate them into Revision 2 of the NEI 05-04.

In general, a follow-on peer review implies that an NEI 00-02 review has already been conducted, and at least the level A and B Fact & Observations (F&Os) from that review have been addressed. A follow-on peer review would be needed (actually required by the ASME PRA Standard, Reference 2) as a result of a PRA upgrade, performed either in response to the original peer review or as a result of the normal evolution of the PRA model. A change that constitutes a PRA upgrade is defined in Section 2 of the ASME PRA Standard (see Section 1.2 of this document). Appendix A of Reference 2 provides examples to help determine the difference between PRA update and PRA maintenance. In some cases, a follow-on peer review may be requested for the entire PRA model because of changes made to the methodology throughout the PRA model. Thus, a follow-on peer reviews scope can be as narrow as a single PRA technical element, or as expansive as a peer review of the entire PRA.

a Some referenced documents may use the term Probabilistic Safety Assessment (PSA) instead of PRA. These terms are considered equivalent terms in the context of this document.

November 2008 1

NEI 05-04, Rev. 2 PRA Peer Review Process The process described in this document will support a full-scope PRA peer review or a focused PRA Peer Review (covering one or more PRA Technical Elements). These PRA Peer Reviews can either be follow-on peers, as described above, or the first PRA Peer Review received (e.g.,

for a new plant that has yet been constructed or operated).

1.2 Background In 1997, the Boiling Water Reactor Owners Group (BWROG) developed a process for performing a peer review of a plants Level 1 at-power PRA models that would assess the capability of the PRA for various risk-informed applications and also assess whether a process was in place to provide a means for the long-term maintenance of that level of capability. The key features of the BWROG process were a highly structured schedule for a focused review of the PRA and a set of 11 checklists to be used to review ten technical elements of a PRA plus the program in place for maintenance of the PRA models, and a four-level grading scheme for the eleven technical areas.

The Combustion Engineering Owners Group (CEOG) adopted the BWROG peer review process with some slight modifications. In parallel, the Nuclear Energy Institute (NEI), working with the Westinghouse Owners Group (WOG) and the Babcock & Wilcox Owners Group (B&WOG),

adopted the BWROG peer review process and revised the checklists to incorporate Pressurized Water Reactor (PWR) specific items as needed. NEI issued NEI-00-02 as the industry standard for performing PRA peer reviews. The industry peer review process presented in NEI 00-02 was intended to cover a single peer review of a utilitys PRA with on-going maintenance of the capability of the PRA covered by reviewing the utilitys PRA maintenance and update process to ensure that it was sufficient to maintain the PRA at the appropriate capability level.

In April 2002, the American Society of Mechanical Engineers (ASME) issued ASME RA-S-2002, the PRA Standard; this has been updated several times. The most current release was updated with Addendum C in August 2007 (Reference 2). Section 5.4 of the standard requires a peer review for PRA upgrades. (Note: The ASME PRA Standard defines PRA upgrade as the incorporation into a PRA model of a new methodology or significant changes in scope or capability. This could include items such as new human error analysis methodology, new data update methods, new approaches to quantification or truncation, or new treatment of common cause failure.)

The overall scope and set of detailed requirements in the ASME PRA Standard are somewhat different than that of NEI 00-02. Thus, peer reviews conducted in accordance with NEI 00-02 do not cover the full scope of the ASME PRA standard. In Appendix B of Regulatory Guide 1.200 (RG 1.200) (Reference 3), the Nuclear Regulatory Commission (NRC) recognized the validity of the peer reviews conducted in accordance with NEI 00-02 as partially covering the scope of the ASME PRA Standard and they endorsed the concept of performing a self-assessment to show compliance with ASME PRA Standard requirements, including those not covered by the NEI 00-02 peer reviews. Appendix B of RG 1.200 explicitly identifies which ASME PRA Standard requirements are either not covered by the NEI peer review checklists or are only partially covered and thus specifies the scope of an incremental self-assessment (i.e., gap analysis) to November 2008 2

NEI 05-04, Rev. 2 PRA Peer Review Process bring the NEI review to adequate equivalence with the ASME PRA Standard, given that an NEI peer review has been previously performed.

The process defined in this document is derived from prior Westinghouse and industry processes (References 5, 6, and 7).

1.3 Scope The scope of this document is to provide a process for performing a full-scope or focused PRA peer review for nuclear plants. These peer reviews may be performed for plants that have already had an NEI 00-02 peer review, an NEI 05-04 peer review, or no previous peer review.

This document addresses full-scope or focused peer reviews to address changes made to a technical element of the PRA that result in significant changes to the methodology or which have otherwise been determined to result in an upgrade of the technical element. The scope of this PRA peer review is the entire scope of the ASME/ANS PRA Standard (Internal Events)b that is applicable to the particular technical element being subject to the peer review. The process described in this document is applicable to a PRA peer review for whatever reason the utility has for performing the review, e.g., PRA upgrades per the definition in the ASME PRA Standard, methodology changes, etc.

b ASME and ANS have released Rev. 1 of the ASME/ANS PRA Standard (Combined Standard) that includes in its scope for a Level 1 PRA the following hazard groups: internal events, external events, internal floods, and internal fires. This guidance document (NEI 05-04) will continue to refer to the ASME/ANS PRA Standard, and in doing so, mean just the requirements related to internal events (Part 2 of the Combined Standard or Section 2 of Addendum A). A separate guidance document is being prepared for performing Fire PRA Peer Reviews; this is expected to be published as NEI 07-12.

November 2008 3

NEI 05-04, Rev. 2 PRA Peer Review Process 2.0 GENERAL OVERVIEW OF PEER REVIEW PROCESS The purpose of the PRA peer review process is to provide a method for establishing the technical capability and adequacy of a PRA relative to expectations of knowledgeable practitioners, using a set of guidance that establishes a set of minimum requirements. Full-scope and focused peer reviews that cover the scope of the ASME/ANS PRA Standard will use the supporting requirements (SRs) in Part 2 of the ASME/ANS PRA Standard (or Section 2 of Addendum of Revision 1 of the ASME/ANS PRA Standard. In the case of a follow-on PRA peer Review, these may be supplemented, as appropriate, by the results of the original NEI 00-02 peer review.

The PRA peer review process is a tiered review process in which the reviewer begins with a relatively high level examination of the PRA technical element(s) against the requirements, and progresses successively to additional levels of detail as necessary to ensure the robustness of the model until all of the requirements are adequately reviewed.

Implementing the review involves a combination of a broad scope examination of the PRA element(s) within the scope of the review and a deeper examination of portions of the PRA element(s) based on what is found during the review. The SRs provide a structure, which in combination with the peer reviewers PRA experience provides the basis for examining the various PRA technical elements. The supporting requirements help to ensure completeness in the review.

If a reviewer discovers a question or discrepancy, it is expected that a more thorough, detailed search will be conducted.

In general, it is essential to focus the review on the relevant application-specific results of the PRA to ensure that the review directly addresses intended plant applications of the PRA. For example, if the results of a PRA indicated relatively low importance of a diesel generator(s), then a risk-informed submittal to increase the allowed outage time (AOT) of the diesel generator(s) would be supported by the PRA, assuming the peer review showed an adequate and technically sound PRA model.

November 2008 4

NEI 05-04, Rev. 2 PRA Peer Review Process 3.0 ASSIGNMENT OF CAPABILITY CATEGORIES One of the outcomes of the peer review process is the assignment of Capability Categories, which are used to indicate the relative capability level of each technical element based on the SRs as defined in the ASME/ANS PRA Standard. For full-scope or focused PRA peer reviews against the ASME/ANS PRA Standard scope, the utility PRA will be assigned a Capability Category for each SR reviewed. This section discusses Capability Categories: what they mean, how to assign them, and some historical reference to the grades used NEI 00-02 process.

In general, it is essential to focus the review on the specific conclusions of the PRA to ensure that the review directly addresses intended plant applications of the PRA. It is important to note that neither the high level requirements (HLRs), PRA Technical Elements, nor the entire PRA are assigned an overall Capability Category. However, each SR is assigned a Capability Category, as applicable to the specific PRA Peer Review.

The major benefit of the review process, however, is not the assigned SR Capability Categories, but rather the recommendations for improvements and the acknowledgments of the strengths of the PRA. Additional beneficial outcomes of the review process are the exchange of information regarding PRA techniques, experiences, and applications among the host utility and utility review personnel, and an anticipated evolving level of consistency from review to review.

3.1 Assigning Capability Categories for Peer Reviews Against the ASME PRA Standard Part 2 of the ASME/ANS PRA Standard (Section 2 of Addendum A) presents the risk assessment technical SRs. These requirements are specified in terms of Capability Category requirements with increasing scope and level of detail, increasing plant-specificity, and increasing realism as SRs satisfy Capability Category I through Capability Category III. See Table 1-1.3-1 of the ASME/ANS PRA Standard (Reference 2) (Table 1.1.4-2 in Addendum A).

For a peer review against the ASME/ANS PRA Standard, the applicable portions of a host utilitys PRA will be reviewed against Part 1 (Section 1-5, PRA Configuration Control) and Part 2 (containing the SRs) of the ASME/ANS PRA Standard, following the guidance of Section 1-6 of the ASME/ANS PRA Standard. (For Addendum A, PRA Configuration Control is in Section 1.5, SRs are in Section 2). For each SR reviewed, the host utilitys PRA will be assigned a Capability Category for that SR.

For each Capability Category, the SRs define the minimum requirements necessary to meet that Capability Category. Some of the SR action statements apply to only one Capability Category, while others extend across two or three Capability Categories. When an action statement spans multiple categories, it applies equally to each Capability Category. When necessary, the differentiation between Capability Categories is made in other associated SRs. The interpretation of a SR whose action statement spans multiple categories is stated in Table 1. It is intended that, by meeting all the SRs under a given HLR, a PRA will satisfy the intent of that HLR.

November 2008 5

NEI 05-04, Rev. 2 PRA Peer Review Process Table 1 -- Interpretation of Supporting Requirements Action Statement Peer Review Interpretation of the Supporting Spans Finding Requirement All Three Capability Meets SR Capable of supporting applications in Categories (I/II/III) all Capability Categories Does not meet SR Does not meet minimum standard Meets Individual Capable of supporting applications Single Capability SR requiring that Capability Category or Category lower (I or II or III) Does not meet Does not meet minimum standard any SR Meets SR for Capable of supporting applications CC I/II requiring Capability Category I or II Lower Two Capability Meets SR for Capable of supporting applications in Categories (I/II) CC III all Capability Categories Does not meet SR Does not meet minimum standard Meets SR for Capable of supporting applications in CC II/III all Capability Categories Upper Two Capability Meets SR for CC I Capable of supporting applications Categories (II/III) requiring Capability Category I Does not meet SR Does not meet minimum standard If there are instances where it appears that this approach leads the reviewer(s) to question the adequacy of the requirement for the higher Capability Categories, the reviewer(s) will document the interpretation of the SR that has been applied, and the host utility or any member of the Peer Review Team may submit an Inquiry to the ASME Committee on Nuclear Risk Management (CNRM) requesting a clarification.

When the peer review consists of a team of reviewers (i.e., more than one reviewer; see Section 4.4), the determination of the Capability Category for each SR will be based on the consensus of the review team. No Capability Categories will be assigned to the HLRs, but a qualitative assessment of the applicable HLRs in the context of the PRA technical element summary will be made based on the associated SR Capability Categories.

The applicable portions of the PRA and associated documentation will also be reviewed for conformance to the requirements of Sections 4.2 and 4.3 of the ASME PRA Standard as part of the overall review.

3.2 Comparison Against Grading Process for NEI 00-02 For the sake of comparison between the Capability Categories of the ASME PRA Standard and the grades assigned during the NEI 00-02 Peer Reviews, a brief discussion of what the NEI 00-02 November 2008 6

NEI 05-04, Rev. 2 PRA Peer Review Process grades mean is warranted. This will facilitate any conversion, when appropriate and applicable from the original peer review to the follow-on peer review.

Under the NEI 00-02 grading process, the grade is meant to convey the ability of the PRA sub-element to support particular types of applications. The implementation of the PRA peer review process uses checklists that include the criteria to be used to grade each of the elements of the PRA.

These checklists are contained in Appendix B of NEI 00-02.

The distinctions in grade level are assigned based on the ability of the reviewed item to support applications of varying complexity. These distinctions between the checklist item grades were more explicitly defined in subtier criteria that were developed subsequent to the original checklists and used in some of the later industry peer reviews. However, it is important to note that all the PRA applications will likely be a blend of probabilistic and deterministic assessments. Therefore, the grades also implicitly define the required level of deterministic assessments that are needed in conjunction with the PRA.

Grade 1 This grade corresponds to the attributes needed for identification of plant vulnerabilities, i.e.,

responding to NRC Generic Letter (GL) 88-20. Most PRAs are expected to be capable of meeting these requirements.

There may be substantial conservatisms included in the modeling, analysis and data to achieve a Grade 1. These conservatisms may still allow the identification of outliers and vulnerabilities, and prioritization of certain issues, but they limit the ability to use a PRA (with a substantial number of sub-elements at Grade 1) for most other risk-informed applications.

Grade 2 Grade 2 corresponds to the attributes needed for risk ranking of systems, structures, and components (SSCs). A PRA with elements certified at this grade would provide assurance that, on a relative basis, the PRA methods and models yield meaningful rankings for the assessment of SSCs, when combined with deterministic insights (i.e., a blended approach). Grade 2 is thus acceptable for Grade 1 applications and for applications that involve the risk ranking.

Grade 3 This review grade extends the requirements to ensure that risk significance determinations made by the PRA are adequate to support regulatory applications, when combined with deterministic insights. Therefore, a PRA with elements certified at Grade 3 can support physical plant changes when it is used in conjunction with other deterministic approaches that ensure that defense-in-depth is preserved.

Grade 3 is acceptable for Grades 1 and 2 applications, and also for assessing safety significance of equipment and operator actions. This assessment can be used in licensing submittals to the NRC to November 2008 7

NEI 05-04, Rev. 2 PRA Peer Review Process support positions regarding absolute levels of safety significance if supported by deterministic evaluations.

Grade 4 This review grade requires a comprehensive, intensively reviewed study that has the scope, level of detail, and documentation to ensure the highest capability of PRA analyses and the robustness of the results. Routine reliance on the PRA as the basis for certain changes is expected as a result of this grade. Few PRAs have many elements (or subelements) eligible for this grade.

Grade 4 is acceptable for Grades 1, 2, and 3 applications, and also usable as a primary basis for developing licensing positions that may change hardware, procedures, requirements, or methods (inside or outside the licensing basis).

In general, the following approximate correspondence exists between the two grading systems:

NEI 00-02 ASME PRA Standard Grade 1 No equivalent grade Grade 2 Capability Category I Grade 3 Capability Category II Grade 4 Capability Category III November 2008 8

NEI 05-04, Rev. 2 PRA Peer Review Process 4.0 PRA PEER REVIEW: ASME PRA STANDARD SCOPE This section describes the process that will be used to perform a full-scope or focused peer review within the scope of the ASME/ANS PRA Standard (Reference 2).

4.1 Scope A full-scope or focused peer review will cover the set of HLRs and SRs for the applicable PRA technical elements in Part 2 of the ASME/ANS PRA Standard (Section 2 of Addendum A).

Further, the scope may be limited within a PRA technical element to only the SRs that are germane to a specific PRA upgrade (e.g., re-evaluation of pre-initiator human error probabilities). The focused peer review may be limited to a single PRA technical element, or may include multiple (or all) technical elements. This process should also be applicable for the utility conducting a peer review simply to validate their self-assessment (as per NEI 00-02),

since the self-assessment is against the requirements in the ASME/ANS PRA Standard. The process is equally valid for a utility having a peer review for a PRA developed to support a new plant (e.g., a paper design that is not yet build or operating).

It is expected that, in addition to the original NEI 00-02 peer review, the host utility will have performed a self-assessment of their PRA against that portion of the ASME PRA Standard not covered by the NEI peer review scope as defined in Table B-4 of RG 1.200 (i.e., gap analysis).

A self-assessment can also be performed against a previous version of the ASME/ANS PRA Standard, or against the current version of the PRA Standard to ensure that the PRA generally comports with the Capability Category II requirements. The results of this self-assessment will be used to help focus the peer review of the PRA for compliance with those ASME SRs not covered by the NEI peer review scope. The host utility should not request a PRA peer review until this self-assessment is completed.

4.2 Host Utility Requirements It is expected that, prior to requesting a full-scope or focused peer review, the host utility will address the technical issues identified during the original NEI 00-02 peer review or any subsequent PRA Peer Reviews that have occurred that apply to the technical elements to be covered by the new peer review. This includes updating and reviewing the associated documentation. It is also recommended that, when the host utility is satisfied that the applicable PRA peer review issues have been resolved, they perform a self-assessment of the compliance of their PRA, as related to the peer review issues, with the applicable requirements in the ASME PRA Standard.

It is expected that a gap analysis will be performed prior to the scheduling of a peer review, and the results of the gap analysis will be available to the peer review team. When a host utility requests a PRA peer review, the documentation accompanying the request should include verification that their PRA meets comports with RG 1.200. The decision on whether an November 2008 9

NEI 05-04, Rev. 2 PRA Peer Review Process appropriate internal gap analysis has been completed by the host utility will be made by a representative of the respective Owners Group, such as the PRA peer review coordinator or the proposed PRA peer review team leader.

The host utility should provide the peer review team with a package of relevant information in advance of the full scope or focused peer review, to allow adequate review by the team. This package should contain at least the following items:

a. A detailed description of the scope of the intended full-scope or focused peer review.

This should be sent early enough to permit feedback from the peer reviewers to resolve any issues prior to performing the review (as agreed to between the host utility and the Team Lead). (Scope may have been discussed during the planning stages, but the actually review personnel should be very clear on the scope details.)

b. A copy of the host utility self-assessment of their PRAs compliance with the ASME/ANS PRA Standard. This should include the basis for their assessment of compliance for each ASME/ANS SR with references to those portions of their PRA documentation that demonstrate the appropriate degree of compliance.

c. A copy of the NEI 00-02 peer review or those portions associated with the scope of the full-scope or focused peer review, to the extent that this information is still pertinent. If a peer review was performed against the ASME PRA Standard after the NEI 00-02 review, then the results (and resolution) of the subsequent review need to be provided to the Peer Review Team. If the scope of the subsequent review was less than the NEI 00-02 review, then those portions of the NEI 00-02 review still applicable need to be provided as well.

d. A summary of the changes made to the applicable portions of the PRA since the original NEI 00-02 peer review. This should include explicit identification of what was done to resolve each relevant F&O with a significance level of A or B. For subsequent reviews, this would include F&Os characterized as findings.

e. Copies of any PRA documents that were revised as a result of the changes to the PRA. If the changes affect a large number of the PRA documents, examples can be provided. If only example documents are provided, a list of all revised documents should also be provided. These documents should then be available for the review team when the full-scope or focused peer review is conducted.

f. A copy of the latest PRA Quantification Report, if this is based on results obtained using the upgraded technical elements being reviewed. The report should include a summary of CDF and LERF results, and discussion of the results and insights.

In general, the material supplied to the peer review team is the host utilitys decision. However, the more information that can be provided in advance, the more the on-site visit will be facilitated. Providing documentation and/or the PRA computer model prior to the visit may November 2008 10

NEI 05-04, Rev. 2 PRA Peer Review Process permit the reviewer(s) to become more familiar with the PRA model and conduct a more effective on-site review.

It is recommended that the review be conducted at the location that provides the best access to relevant documentation, as delays due to document retrieval difficulties are not acceptable during on-site reviews. In addition, the host utilitys PRA staff should be available to the PRA peer review team while they are on site.

4.3 Self-Assessment The detailed self-assessment of compliance with the ASME/ANS PRA Standard should identify, for each SR to be reviewed, the Capability Category that the PRA supports. For each SR to be reviewed, the documentation should include a statement of the Capability Category that is met, the basis for the assessment, and references to the specific PRA documents, and appropriate sections, which support the assessment. It is expected that for those SRs (i.e., applicable NEI 00-02 subelements) that received a Grade 3 or 4 in an NEI 00-02 review and for which no self-assessment is required by Appendix B of Regulatory Guide 1.200, it is reasonable to assign a Capability Category II unless the SR compliance has been altered by a PRA update and/or a specific self-assessment supports a different Capability Category. However, the mapping of NEI 00-02 grades to Capability Categories is not also clean, especially with Grade 2/Capability Category I; peer reviewers should, at their discretion, confirm any Capability Category assignment based solely on the NEI 00-02 review.

The ASME and ANS has granted the Electric Power Research Institute (EPRI) a number of licenses for the use of an electronic version of the ASME/ANS PRA Standard. EPRI has expanded the ePSA module of the Risk and Reliability (R&R) Workstation to incorporate an ACCESS' database that includes the ASME/ANS PRA Standard SRs, as well as provisions for documenting a self-assessment of a PRA against the ASME/ANS PRA Standards and for documenting a peer review of the self-assessment (Reference 4). This database also includes provisions for documenting the results of the NEI 00-02 peer review and the actions taken to correct any identified deficiencies. The ePSA module is available to all EPRI members and it, or an equivalent process, may be used to document the self-assessment.

4.4 Peer Review Team Section 1-6 of the ASME/ANS PRA Standard (Section 1.6 of Addendum A) provides guidance for PRA peer reviews. Section 1-6.2 of the ASME/ANS PRA Standard (Section 1.6.2 of Addendum A) provides specific peer review team requirements that must be met. Specifically, Section 1-6.2.3 (Section 1.6.2.4) allows a single expert to perform the peer review of a single technical PRA element, given that the expert has appropriate knowledge and experience. It is assumed with regard to the independence requirement of Section 1-6.2.1 (1.6.2.2) that reasonable and practicable interpretation will be made allowing, as needed, use of non-involved utility personnel from other sites for multi-site utilities, use of current contractors (on-site or otherwise)

November 2008 11

NEI 05-04, Rev. 2 PRA Peer Review Process involved in other work, etc. A requirement of absolute independence coupled with the need for adequate technical expertise can be difficult to achieve in some situations.

When multiple PRA technical elements are included in the full-scope or focused peer review, a Lead Reviewer may be assigned for each of the PRA technical elements (e.g., System Analysis) to be reviewed, from among the members of the review team, based on member qualifications.

The responsibilities of the Lead Reviewer are to coordinate the general review for the technical element, conduct the final consensus session, and to prepare the summary for the technical element at the end of the review. In addition to Lead Reviewers, there will also be a Technical Lead, responsible for the overall technical scope and content of the review, and a Team Lead (or facilitator), responsible for ensuring the review is conducted on schedule and provide an interface with the host utility. Depending on the size of the review team, these two functions may be performed by the same individual.

The number of members of the peer review team and their specific expertise and required level of qualification is a function of the number of PRA technical elements that are being reviewed.

Such decisions should be recommended to the host utility by the designated Technical Lead.

However, it is strongly suggested that all reviewers have a minimum of three years of experience. This level of experience is necessary because of the time pressure for completing the reviews. Even with appropriately qualified individuals, experience suggests that a PRA peer review covering all of the SRs requires a minimum of six reviewers for one week. It is further recommended that each member of the team have at least five years of nuclear power plant industry experience.

4.5 Peer Review Schedule Adequate time should be allocated for the peer review process. The amount of time required, and the associated logistics, will depend on the scope of the review, the number of reviewers examining each technical element, and the availability of supporting documentation. Prior to the peer review, the lead reviewer for the technical element being reviewed should review all host utility-supplied information to confirm the ability of the review team to complete the peer review in the scheduled time. Should the schedule be determined to be inadequate, either the schedule should be modified or additional information requested of the host utility to facilitate the review in the available time.

4.6 Peer Review Process The review team will focus on reviewing, for the technical elements to be reviewed, the host utilitys self-assessment of the applicable elements against the corresponding scope in RG 1.200, Appendix B, and the degree to which the PRA meets the applicable requirements in the ASME/ANS PRA Standard SRs.

Depending on the size of the peer review team and the scope of the peer review (e.g, full-scope, focused), the team may be sub-divided into sub-teams to review the various aspects of the PRA November 2008 12

NEI 05-04, Rev. 2 PRA Peer Review Process within the scope of the review. The composition of the sub-teams may vary from day-to-day to meet the review needs for each day. Such an approach was used for the original NEI 00-02 peer review, and example schedules are available from those reviews. As the peer review process is very intense and focused because of the amount of material to cover in a limited period of time, schedules and element assignments should be considered flexible, though the Team Lead needs to ensure that all the material is adequately reviewed.

Prior to the start of the review, the review team members will perform a refresher review of the applicable portions of the ASME PRA Standard, with emphasis on Section 6.0, and establish a common perspective regarding the general grading philosophy consistent with the ASME PRA Standard. The applicable HLRs in Section 4.5 will also be briefly reviewed to ensure the team is familiar with the high level scope of the review. A set of orientation/training slides that can be used is included in Appendix C. While the ASME PRA Standard training that is being developed may contain useful background information, the materials from that training should not be used as interpretations of the Standard. As noted in Section 3.1, Inquiries on the interpretation of specific SRs may have been forwarded to the ASME CNRM. The set of Inquiries that have been resolved by CNRM should be obtained from the ASME CNRM Secretary and reviewed prior to conducting a Peer Review.

At the beginning of the review for each technical element, the reviewer(s) should review the HLRs for the element and preview the individual SRs. In Appendix A of RG 1.200, Rev. 1, the NRC has provided a Regulatory Position relative to some of the specific SRs in the ASME PRA Standard. The peer reviewer(s) should consider these NRC clarifications and qualifications, where applicable, during the review, and note the extent to which the PRA element(s) being reviewed address these positions. The reviewer(s) should provide an assessment relative to the NRCs clarifications and qualifications, particularly those in Table A-1 (Appendix A) of RG 1.200.

The starting point for the review of each SR is typically the host utilitys self-assessment when available. This will provide the utilitys assessment of the Capability Category that they think their PRA meets for that SR and the basis for this assessment. The self-assessment should also provide pointers to the associated PRA documentation. The reviewers look at the basis statement and review the associated documentation to a sufficient level of detail to make their own assessment.

The reviewers are not limited to the referenced documents. The reviewers may request that they be allowed to review any pertinent documentation they believe is needed to make their assessment.

Assessment of the SRs can be recorded in tables such as Tables B-10 through B-18 in Appendix B of this document.b As the SRs are purposefully open to some interpretation, there may need to be some discussion to determine the appropriate assignment of a Capability Category, or even determine if a SR is considered to be met. The reviewers must consider the whole of the PRA and not be overly focused on a specific discrepancy. To declare that an an SR is not met, a preponderance of b

The SR tables in Appendix B do not necessarily reflect the latest version of the ASME/ANS PRA Standard. Users should confirm that the structure of Tables B-10 through B-18 conforms to the version being applied, and make changes (e.g., indicating appropriate SR numbering and Capability Categories for the SRs) as needed. These tables current match Addendum A of Revision 1 of the ASME/ANS PRA Standard.

November 2008 13

NEI 05-04, Rev. 2 PRA Peer Review Process evidence must be observed. In cases where an SR description includes an example, the reviewers should be cautioned that conformance with the example is not necessary to meet that SR.

Determination of the status of an SR should be guided by the following approach from RG 1.200

[3]:

... [If] there are a few examples in which a specific requirement has not been met, it is not necessarily indicative that this requirement has not been met. If, the requirement has been met for the majority of the systems or parameter estimates, and the few examples can be put down to mistakes or oversights, the requirement would be considered to be met. If, however, there is a systematic failure to address the requirement (e.g., component boundaries have not been defined anywhere), then the requirement has not been complied with.

During the review of an SR (whether covered by the NEI 00-02 checklist or not), if the reviewers identify any issues/problems that impact the capability of the PRA, they will document these problems using an F&O form equivalent to that presented in Appendix A of this report. The F&Os specify the PRA element and SR of concern, and describe the PRA level of compliance with the criteria. The issue documented may be a weakness (finding), a strength (best practice),

or a simple observation (suggestion). It should be noted that even in cases where an SR has been assessed to meet CC II or III, the review team may document an F&O finding. Such findings are typically for non-systematic discrepancies that the PRA peer review team judges require correction. The F&O includes an assessment of the importance of the observation on the level of capability of the SR, and, for weaknesses, a proposed resolution for the weakness. The importance of each observation is classified as a:

Finding - an observation (an issue or discrepancy) that is necessary to address to ensure:

the technical adequacy of the PRA (relative to a Capability Category),

the capability/robustness of the PRA update process, or the process for evaluating the necessary capability of the PRA technical elements (to support applications)

Suggestion - an observation considered desirable to maintain maximum flexibility for PRA applications and consistency with industry practices. Failing to resolve a suggestion should have no significant impact on the PRA results or the integrity of the PRA. Some examples of a suggestion include:

editorial and minor technical items recommendations for consistency with industry practices (e.g., replacing a given consensus model with a more widely used model) recommendations to enhance the PRAs technical capability as time and resource permit observations regarding PRA technical adequacy that may affect one or more risk-informed applications This approach of classifying F&Os replaces the A/B/C/D approached used in the original NEI 00-02 Peer Reviews, and the modification (with combined A/B) recommended in the original version of this document. The finding/suggestion approach should be simpler and less time November 2008 14

NEI 05-04, Rev. 2 PRA Peer Review Process consuming (for the reviewers) to implement, as making the distinction between a finding and a suggestion should be more evident (with less controversy). This approach will also prevent any findings from being relegated to a C category, which may have occurred with some previous Peer Review F&Os. The disposition of F&Os will be the same as previous Peer Reviews, with the host utility responsible for reconciling the findings e.g., placing them in their corrective action program (or the equivalent). In general, a finding would correspond to an A/B F&O, while a suggestion would correspond to C and D F&O, for utilities that may have established a procedure to deal with PRA F&Os.

Originally, the S classification was used indicate a PRA strength. This classification should be reserved for items that would represent best industry practice, to the extent that utilities (with findings) would want to emulate. Accordingly, and to avoid confusion with suggestion, this classification will be designated best practice, and identified with a BP.

Each technical element has a HLR and a number of associated SRs with respect to documentation.

In general, the documentation HLRs require that the documentation be sufficient to facilitate peer reviews by describing the processes used, providing the assumptions used and their bases, and providing the associated SRs specific details for each technical element. Assessing the Capability Category for the documentation SRs does not require a separate review for each SR. At the start of the review for a given technical element, the review team may review the documentation HLR and SRs for that element to identify any unique documentation aspects for that technical element. At the completion of the review of the technical element, the reviewers for that element may assess the PRA compliance with the documentation SRs based on availability, scope and completeness of the documentation that they used to review the technical SRs for the technical element.

At the end of the review for each technical element being reviewed, the team members will conduct consensus discussions to assign Capability Categories to the SRs. The consensus session for a particular technical element will be led by the Lead Reviewer.

In documenting the F&Os, it is important to note that the reviewers need not match F&Os to SRs one-to-one. F&Os on common SRs that cross several PRA Technical Elements should be combined into a single F&O (i.e., uncertainty, documentation for peer review and applications). It should also be noted that for different technical issues affecting a single SR, it may be appropriate to write separate F&Os.

As stated in Section 1-6.1 of the ASME/ANS PRA Standard (Section 1.6.1 in Addendum A), The peer review need not assess all aspects of the PRA against all requirements in the Technical Requirements Section ; however, enough aspects of the PRA shall be reviewed for the reviewers to achieve consensus on the adequacy of methodologies and their implementation for each PRA element. The set of key review areas identified in Sections 1-6.3 and 1-6.6 of the ASME/ANS PRA Standard (Sections 1.6.3 and 1.6.6 for Addendum A) for the technical element(s) being peer reviewed must be addressed.

In performing the review of a given technical element, the Lead Reviewer may elect to skip the review of selected SRs if the other reviewers determine that they can achieve consensus on the adequacy of the PRA with respect to the HLR associated with the SRs that are not reviewed.

November 2008 15

NEI 05-04, Rev. 2 PRA Peer Review Process Before electing to skip any SRs, the Lead Reviewer should consult the appropriate portion of section 6.3 to ensure that the review will be consistent with the appropriate requirements in Section 6.3. The review sub-team must document their basis for skipping the given SR.

The reviewers should specifically address assumptions and sources of uncertainty in the elements being reviewed. Such assumptions and uncertainties, their potential impact on the baseline PRA results, and the manner in which the host utilitys quantification process addresses them, should be reviewed. The host utilitys characterization of uncertainty should be qualitative. Their opinions and suggestions regarding these assumptions and uncertainty sources, as well as where the issue arises in the model, should be documented. This treatment of assumptions and sources of uncertainty for the base PRA is consistent with the NRC FRN clarification of RG 1.200, Rev. 1 (Reference 8).

Section 1-5 of the ASME/ANS PRA Standard (Section 1.5 in Addendum A) provides the requirements for a PRA configuration control program, and should be used by all PRA peer review teams. The full-scope or focused Peer Review Team should provide a summary assessment of how well the PRA maintenance program satisfies ASME/ANS PRA Standard Section 1-5 (Section 1.5) requirements relative to the technical element(s) being reviewed. The requirements defined by the Maintenance and Update (MU) checklist in NEI 00-02 may be used as guidance for this summary assessment for the specific technical element(s). The Maintenance and Update (MU) checklist from the NEI 00-02 process can be used as a guide to indicate specific items that should be considered to satisfy the requirements of Section 1-5 (Section 1.5).

As noted in Section 4.3, EPRIs ePSA tool can be used to review the results of the original NEI 00-02 peer review, status of F&Os, and results of the host utilitys self-assessment. The ePSA tool can also be used by the full-scope or focused peer review team, at the direction and discretion of the host utility, to record their findings, e.g., new F&Os as a result of the follow-on review. The tables in Appendix B can also be used to record peer review results. Regardless of the tool used, all Capability Category assignments, comments, observations, and recommendations should be made available in an electronic form to the Technical Lead (to prepare the final report) and the host utility (for review). It is further suggested that a sequential F&O log be maintained throughout the review, with the identification format of TE-SR-## being used throughout, where TE identifies the technical element, SR identifies the supporting requirement, and ## is the sequential number for the F&O for that SR. Appendix E contains a sample F&O log that can be used during reviews.

In the peer review process, the assignment of the Capability Categories for the individual SRs are established by a consensus process that requires that all reviewers agree with the final assigned Capability Categories. If a condition arises where a minority of reviewers (one or more) cannot come to consensus, then, at the request of any peer reviewer, differences or dissenting views among peer reviewers should be documented. The documentation for any dissenting opinions should be included as a note to the SR, and be included in an appendix with any recommended alternatives for resolution. The dissenting opinion is provided for information to the host utility, and should not be characterized as an F&O finding. This process should only be used in the most exceptional situations, as, from the perspective of the host utility, this is a highly undesirable situation.

Therefore, the review team should strive to achieve a consensus position on all review elements.

November 2008 16

NEI 05-04, Rev. 2 PRA Peer Review Process It is recommended that (except for a one-day visit) there is a daily debrief with the host utility.

There purpose of a debrief would be to (a) inform the host utility of any expected concerns with the PRA, (b) clearly delineate any owed information from the host utility, (c) identify any new requested information, (d) as appropriate, seek clarification or confirmation on prepared F&Os, and (e) exchange any other relevant information. The timing and duration of such meetings should be mutually agreed to by the peer review team lead and the host utility.

In the course of performing the PRA peer review, insights will be developed related to the process (as described in this guidance document) or PRA practices (e.g., identification of a best practice).

Such insights (i.e., lessons learned) should be documented and transmitted to NEI for subsequent updates. Appendix D provides an example Lessons Learned form that can (optionally) be used.

4.7 PRA Peer Review Report The output of the peer review is a written report documenting both the details and the summary findings of the review. The report should address the following:

• Clear definition of the scope of the peer review

• Summary of the results of the review for each technical element within the scope of the review, organized at the HLR level. The result summaries should focus on the general results of the reviews of the SRs.

• Summary of any A or B level F&Os from the original NEI 00-02 peer review that the PRA Peer Review Team do not believe have been resolved, or F&Os from a subsequent Peer Review (after the NEI 00-02 review). F&Os from the original Peer Review need not be considered if their SRs are within the scope of a subsequent Peer Review.

• The rationale for not accepting the resolution of the F&Os from the original NEI 00-02 peer review or subsequent Peer Review, as well as any F&Os generated as a result of the full-scope or focused peer review. F&Os from the original Peer Review need not be considered if their SRs are within the scope of a subsequent Peer Review.

• Summary of any new Finding F&Os generated during the full-scope or focused peer review.

• Summary of identification of assumptions and sources of uncertainty, their impacts, and the reviewers opinion regarding their treatment.

• Identification of the assessed Capability Category for each SR within the scope of the review.

The principal results, conclusions, and recommendations of the Peer Review Team should be communicated to the host utility at the completion of the onsite review, and included in the report.

The resumes of the peer review team members should also be included.

The host utility should only expect one round of comments (i.e., there will not be multiple draft reports provided for utility review), and should not expect that the review team would hold teleconferences or other meetings with the utility in order to review comment resolutions.

Additionally, as time does not allow for the PRA peer review team to provide the host utility with November 2008 17

NEI 05-04, Rev. 2 PRA Peer Review Process early results and then to meet to discuss interpretations, etc. during the on-site review, consensus/debate meetings with the host utility during the on-site review should be avoided outside the context of any daily debriefs.

The utility is welcome and encouraged to comment on the draft PRA peer review report. Such comments can address factual technical issues, as well as interpretations of the ASME/ANS PRA Standard. The team lead is responsible for resolving these comments with the team and issuing a final report. Note, however, that interpretation of the ASME/ANS PRA Standard SRs needs to be directed to ASME via the Inquiry process - this can be done by either the team lead or the host utility, however since the Peer Review Team is a transitory group, it is recommended that the host utility seek an interpretation. The utility should not expect that the review team would rescind an F&O or revise an SR CC assessment based on the host utility stating they will address the issue.

The review is to determine the state of the PRA at the time of the review; the team does not have the time either on-site or during the report development stage to reconsider issues based on revised work transmitted by the utility.

The peer review report should be made part of the host utilitys PRA documentation file for future internal and external reference. The sponsoring Owners Group should maintain a design record copy, but it should not be accessible to others than the host utility. Team members should retain documentation of their participation in the PRA peer review, but should not redistribute any notes or utility documentation.

November 2008 18

NEI 05-04, Rev. 2 PRA Peer Review Process

5.0 REFERENCES

1. Probabilistic Risk Assessment (PRA) Peer Review Process Guidance, NEI 00-02, Revision A3, Nuclear Energy Institute, October 2000.

2. Standard for Level 1/Large Early Release Frequency Probabilistic Risk Assessment for Nuclear Power Plant Applications, Addendum a, ASME/ANS RA-S-2007, American Society of Mechanical Engineers, December 2007.

3. Regulatory Guide 1.200, Revision 1, An Approach For Determining the Technical Adequacy of Probabilistic Risk Assessment Results For Risk-Informed Activities, U.S.

Nuclear Regulatory Commission, Office of Nuclear Regulatory Research, January 2007.

4. ePSA PRA Documentation Module, Beta Version 2, The Electric Power Research Institute, May 2003.

5. Process for Performing PRA Peer Review Follow-on Reviews and for Performing PRA Peer Reviews Using the ASME PRA Standard, WCAP-16091, Westinghouse Electric Company, LLC, June 2003.

6. Process for Performing Follow-on PRA Peer Reviews Using the ASME PRA Standard, Rev. 0 (Draft, Not Issued), NEI-03-xx, September 2003.

7. Process for Performing Follow-on PRA Peer Reviews of Individual PRA Technical Elements Using the ASME PRA Standard, WCAP-16181-NP, Rev. 0 (Draft, Not Issued),

Westinghouse Electric Company, LLC, November 2003.

8. Notice of Clarification to Rev. 1 of Regulatory Guide 1.200, An Approach for Determining the Technical Adequacy of PRA Results for Risk-Informed Activities, FRN July 27, 2007, Accession number: ML071170054.

November 2008 19

NEI 05-04, Rev. 2 PRA Peer Review Process APPENDIX A SAMPLE FACT AND OBSERVATION FORM November 2008 A-1

NEI 05-04, Rev. 2 PRA Peer Review Process FACT/OBSERVATION REGARDING PRA TECHNICAL ELEMENTS OBSERVATION (ID: )c / Technical Element _ / Supporting Requirement ____

LEVEL OF SIGNIFICANCE:

BASIS FOR SIGNIFICANCE POSSIBLE RESOLUTION LEVELS OF SIGNIFICANCE FOR FACTS AND OBSERVATIONS Finding An observation (an issue or discrepancy) that is necessary to address to ensure the technical adequacy of the PRA, the capability of the PRA, or the robustness of the PRA update process.

Suggestion An observation considered desirable to maintain maximum flexibility in PRA applications and consistency with Industry practices, or simply to enhance the PRAs technical capability as time and resources permit, at the discretion of the host utility. Also includes editorial or minor technical item left to the discretion of the host utility.

BP Represents best industry practice, to the extent that other PRA owners would want to emulate.

c A suggested format for F&O ID number is ee-sr-##, where ee is the 2 letter code for the Technical Element (e.g.,

HR for Human Reliability Analysis), sr is the identifier for the specific supporting requirement (e.g., A3), and ## is a sequential number for F&Os for the given SR. For example, HR-A3-02 would be the second F&O referring to supporting requirement HR-A3.

November 2008 A-2

NEI 05-04, Rev. 2 PRA Peer Review Process APPENDIX B SAMPLE

SUMMARY

TABLES Note: The supporting requirement level tables in this Appendix do not necessarily reflect the latest version of the ASME PRA Standard. Users should confirm that the structure of Tables B-10 through B-18 conforms to the version being applied, and make changes (e.g., indicating appropriate SR numbering and Capability Categories for the SRs) as needed.

November 2008 B-1

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-1 PRA Technical Element Summary: Initiating Event Analysis High Level Summary of High Level Initiating Event Analysis Summary Requirement Number Requirement (by High Level Requirements)

HLR-IE-A Reasonably complete identification of Initiating Events HLR-IE-B Appropriate grouping of Initiating Events HLR-IE-C Estimation of frequency of Initiating Events HLR-IE-D Documentation Table B-2 PRA Technical Element Summary: Accident Sequence Analysis High Level Summary of High Level Accident Sequence Analysis Summary Requirement Number Requirement (by High Level Requirements)

HLR-AS-A Accident sequence analyses define plant-specific sequences for each initiating event HLR-AS-B Accident sequence analyses address sequence level dependencies HLR-AS-C Documentation November 2008 B-2

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-3 PRA Technical Element Summary: Success Criteria High Level Summary of High Level Success Criteria Summary Requirement Number Requirement (by High Level Requirements)

HLR-SC-A Overall success criteria are defined and are based on the as-built, as-operated plant HLR-SC-B Thermal/hydraulic and other supporting analyses capable of providing the success criteria and event timing used in the analyses HLR-SC-C Documentation Table B-4 PRA Technical Element Summary: Systems Analysis High Level Summary of High Level System Analysis Summary Requirement Number Requirement (by High Level Requirements)

HLR-SY-A Reasonably complete treatment of causes of system, failure/unavailability HLR-SY-B Reasonably complete treatment of common cause failures HLR-SY-C Documentation November 2008 B-3

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-5 PRA Technical Element Summary: Human Reliability Analysis High Level Summary of High Level Human Reliability Analysis Summary Requirement Number Requirement (by High Level Requirements)

HLR-HR-A Systematic process used to identify routine actions that may impact equipment availability HLR-HR-B Screening of events based on plant-specific operational practices HLR-HR-C Impact of failure of activities characterized as Human Failure Events (HFEs)

HLR-HR-D Assessment of probabilities use systematic process HLR-HR-E Set of operator responses established using systematic review of relevant procedures HLR-HR-F Failure to perform required actions represented by HFEs HLR-HR-G Assessment of probabilities uses well-defined and self-consistent process HLR-HR-H Recovery actions modeled only if plausible and feasible HLR-HR-I Documentation November 2008 B-4

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-6 PRA Technical Element Summary: Data Analysis High Level Summary of High Level Data Analysis Summary Requirement Number Requirement (by High Level Requirements)

HLR-DA-A Parameters clearly defined HLR-DA-B Components grouped into homogenous groups based on common design, environmental and service conditions HLR-DA-C Generic parameter estimates and plant-specific data collection consistent with parameter definitions HLR-DA-D Parameter estimates based on relevant generic and plant-specific data HLR-DA-E Documentation November 2008 B-5

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-7 PRA Technical Element Summary: Internal Flooding High Level Summary of High Level Internal Flooding Summary Requirement Number Requirement (by High Level Requirements)

HLR-IFPP-A Flood areas identified HLR-IFPP-B Flood areas identified documentation HLR-IFSO-A Flood sources and mechanisms identified HLR-IFSO-B Flood sources and mechanisms identified HLR-IFSN-A Flooding scenarios developed for each flood source HLR-IFSN-B Flooding scenario documentation HLR-IFEV-A Flood-induced initiating events identified and frequency quantified HLR-IFEV-B Flood-induced initiating events documentation HLR-IFQU-A Flood-induced accident sequences quantified HLR-IFQU-B Accident sequence quantification documentation November 2008 B-6

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-8 PRA Technical Element Summary: Accident Sequence Quantification and Results Interpretation High Level Summary of High Level Quantification/Results Interpretation Summary Requirement Number Requirement (by High Level Requirements)

HLR-QU-A Level 1 quantification quantifies core damage frequency HLR-QU-B Quantification uses appropriate models and codes and accounts for limitations HLR-QU-C Quantification determines that identified dependencies are appropriately addressed HLR-QU-D Quantification results reviewed and important contributors identified HLR-QU-E Uncertainties in PRA results are characterized HLR-QU-F Documentation November 2008 B-7

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-9 PRA Technical Element Summary: LERF Analysis High Level Summary of High Level LERF Analysis Summary Requirement Number Requirement (by High Level Requirements)

HLR-LE-A Core damage sequences appropriately grouped into plant damage states HLR-LE-B LERF evaluations include credible severe accident phenomena HLR-LE-C LERF evaluations include analysis of containment system performance HLR-LE-D LERF evaluations include analysis of containment structural performance HLR-LE-E Frequency of containment failure modes leading to LERF quantified and aggregated HLR-LE-F Quantification of LERF addresses important risk factors and sources of uncertainty HLR-LE-G Documentation November 2008 B-8

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-10 Assessment of Supporting Requirement Capability Categories For Initiating Events HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-IE-A IE-A1 IE-A2 IE-A3 IE-A4 IE-A3a IE-A5 IE-A4 IE-A6 IE-4a IE-A7 IE-A5 IE-A8 IE-A6 N/A IE-A9 IE-A7 N/A IE-A10 HLR-IE-B IE-B1 IE-B2 IE-B3 IE-B4 IE-B5 November 2008 B-9

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-10 Assessment of Supporting Requirement Capability Categories For Initiating Events HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-IE-C IE-C1 IE-C2 IE-C1a IE-C3 IE-C1b IE-C4 IE-C2 IE-C5 IE-C3 IE-C6 IE-C4 IE-C7 IE-C5 N/A N/A IE-C8 IE-C6 IE-C9 IE-C7 IE-C10 IE-C8 IE-C11 IE-C9 IE-C12 IE-C10 IE-C13 IE-C11 IE-C14 IE-C12 IE-C15 IE-C13 HLR-IE-D IE-D1 IE-D2 IE-D3 November 2008 B-10

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-11 Assessment of Supporting Requirement Capability Categories For Accident Sequence Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-AS-A AS-A1 AS-A2 AS-A3 AS-A4 AS-A5 AS-A6 AS-A7 AS-A8 AS-A9 AS-A10 AS-A11 HLR-AS-B AS-B1 AS-B2 AS-B3 AS-B4 AS-B5 AS-B6 AS-B5a AS-B7 AS-B6 HLR-AS-C AS-C1 AS-C2 AS-C3 November 2008 B-11

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-12 Assessment of Supporting Requirement Capability Categories For Success Criteria HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-SC-A SC-A1 SC-A2 SC-A3 SC-A4 SC-A4 SC-A4a SC-A5 SC-A6 HLR-SC-B SC-B1 SC-B2 N/A SC-B3 SC-B4 SC-B5 HLR-SC-C SC-C1 SC-C2 SC-C3 November 2008 B-12

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-13 Assessment of Supporting Requirement Capability Categories For Systems Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR Reviewed F&Os I II III HLR-SY-A SY-A1 SY-A2 SY-A3 SY-A4 SY-A5 SY-A6 SY-A7 SY-A8 SY-A9 SY-A10 SY-A10 SY-A11 SY-A11 SY-A12 SY-A12 SY-A12a SY-A13 SY-A12b SY-A14 SY-A13 SY-A15 SY-A14 SY-A16 SY-A15 SY-A17 SY-A16 SY-A18 SY-A17 SY-A19 SY-A18 SY-A20 SY-A18a SY-A21 SY-A19 SY-A22 SY-A20 SY-A23 SY-A21 SY-A24 SY-A22 HLR-SY-B SY-B1 November 2008 B-13

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-13 Assessment of Supporting Requirement Capability Categories For Systems Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR Reviewed F&Os I II III SY-B2 N/A N/A SY-B3 SY-B4 SY-B5 SY-B6 SY-B7 SY-B8 SY-B9 SY-B10 SY-B10 SY-B11 SY-B11 SY-B12 SY-B12 SY-B13 SY-B13 SY-B14 SY-B14 SY-B15 SY-B15 SY-B16 HLR-SY-C SY-C1 SY-C2 SY-C3 November 2008 B-14

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-14 Assessment of Supporting Requirement Capability Categories For Human Reliability Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-HR-A HR-A1 HR-A2 HR-A3 HLR-HR-B HR-B1 HR-B2 HLR-HR-C HR-C1 HR-C2 HR-C3 HLR-HR-D HR-D1 HR-D2 HR-D3 N/A HR-D4 HR-D5 HR-D6 HR-D7 HLR-HR-E HR-E1 HR-E2 HR-E3 HR-E4 N/A HLR-HR-F HR-F1 HR-F2 November 2008 B-15

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-14 Assessment of Supporting Requirement Capability Categories For Human Reliability Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-HR-G HR-G1 HR-G2 HR-G3 HR-G4 HR-G5 HR-G6 HR-G7 HR-G8 HR-G9 HLR-HR-H HR-H1 HR-H2 HR-H3 HLR-HR-I HR-I1 HR-I2 HR-I3 November 2008 B-16

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-15 Assessment of Supporting Requirement Capability Categories For Data Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-DA-A DA-A1 DA-A2 DA-A1a DA-A3 DA-A2 DA-A4 DA-A3 HLR-DA-B DA-B1 DA-B2 HLR-DA-C DA-C1 DA-C2 DA-C3 DA-C4 DA-C5 DA-C6 DA-C7 DA-C8 DA-C9 DA-C10 DA-C11 DA-C12 DA-C11a DA-C13 DA-C12 DA-C14 DA-C13 DA-C15 DA-C14 DA-C16 DA-C15 November 2008 B-17

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-15 Assessment of Supporting Requirement Capability Categories For Data Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-DA-D DA-D1 DA-D2 DA-D3 DA-D4 N/A DA-D5 DA-D6 DA-D7 DA-D6a DA-D8 DA-D7 HLR-DA-E DA-E1 DA-E2 DA-E3 November 2008 B-18

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-16 Assessment of Supporting Requirement Capability Categories For Internal Flooding Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-IFPP-A IFPP-A1 IF-A1 (formerly IFPP-A2 IF-A1a HLP-IF-A)

IFPP-A3 IF-A1b IFPP-A4 IF-A3 IFPP-A5 IF-A4 HLR-IFPP-B IFPP-B1 (formerly IFPP-B2 incorporated in HLP-IF-F) IFPP-B3 HLR-IFSO-A IFSO-A1 IF-B1 (formerly IFSO-A2 IF-B1a HLP-IF-B)

IFSO-A3 IF-B1b IFSO-A4 IF-B2 IFSO-A5 IF-B3 IFSO-A6 IF-B3a HLR-IFSO-B IFSO-B1 (formerly IFSO-B2 incorporated in HLP-IF-F) IFSO-B3 November 2008 B-19

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-16 Assessment of Supporting Requirement Capability Categories For Internal Flooding Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-IFSN-A IFSN-A1 IF-C1 (formerly IFSN-A2 IF-C2 HLP-IF-C)

IFSN-A3 IF-C2a IFSN-A4 IF-C2b IFSN-A5 IF-C2c IFSN-A6 IF-C3 IFSN-A7 IF-C3a IFSN-A8 IF-C3b IFSN-A9 IF-C3c IFSN-A10 IF-C4 IFSN-A11 IF-C4a IFSN-A12 IF-C5 IFSN-A13 IF-C5a IFSN-A14 IF-C6 IFSN-A15 IF-C7 IFSN-A16 IF-C8 IFSN-A17 IF-C9 HLR-IFSN-B IFSN-B1 (formerly IFSN-B2 incorporated in HLP-IF-F) IFSN-B3 HLR-IFEV-A IFEV-A1 IF-D1 (formerly IFEV-A2 IF-D3 HLP-IF-D)

IFEV-A3 IF-D3a IFEV-A4 IF-D4 IFEV-A5 IF-D5 IFEV-A6 IF-D5a November 2008 B-20

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-16 Assessment of Supporting Requirement Capability Categories For Internal Flooding Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os IFEV-A7 IF-D6 IFEV-A8 IF-D7 HLR-IFEV-B IFEV-B1 (formerly IFEV-B2 incorporated in HLP-IF-F) IFEV-B3 HLR-IF-E IFQU-A1 IF-E1 IFQU-A2 IF-E3 IFQU-A3 IF-E3a IFQU-A4 IF-E4 IFQU-A5 IF-E5 IFQU-A6 IF-E5a IFQU-A7 IF-E6 IFQU-A8 IF-E6a IFQU-A9 IF-E6b IFQU-A10 IF-E7 IFQU-A11 IF-E8 HLR-IFQU-B IFQU-B1 (formerly IFQU-B2 incorporated IFQU-B3 in HLP-IF-F)

November 2008 B-21

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-17 Assessment of Supporting Requirement Capability Categories For Accident Sequence quantification and Results Interpretation HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-QU-A QU-A1 QU-A2 QU-A2a QU-A3 QU-A2b QU-A4 QU-A3 QU-A5 QU-A4 HLR-QU-B QU-B1 QU-B2 QU-B3 QU-B4 QU-B5 QU-B6 QU-B7 QU-B7a QU-8b QU-B7b QU-B9 QU-B8 QU-B10 QU-B9 HLR-QU-C QU-C1 QU-C2 QU-C3 HLR-QU-D QU-D1 QU-D1a QU-D2 QU-D1b QU-D3 QU-D1c QU-D4 QU-D3 N/A QU-D5 QU-D4 QU-D6 QU-D5a QU-D7 QU-D5b November 2008 B-22

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-17 Assessment of Supporting Requirement Capability Categories For Accident Sequence quantification and Results Interpretation HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-QU-E QU-E1 QU-E2 QU-E3 QU-E4 HLR-QU-F QU-F1 QU-F2 QU-F3 QU-F4 QU-F5 QU-F6 November 2008 B-23

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-18 Assessment of Supporting Requirement Capability Categories For LERF Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-LE-A LE-A1 LE-A2 LE-A3 LE-A4 LE-A5 HLR-LE-B LE-B1 LE-B2 LE-B3 HLR-LE-C LE-C1 LE-C2 LE-C2a LE-C3 LE-C2b N/A LE-C4 LE-C3 LE-C5 LE-C4 LE-C6 LE-C5 LE-C7 LE-C6 LE-C8 LE-C7 LE-C9 LE-C8a LE-C10 LE-C8b LE-C11 LE-C9a LE-C12 LE-C9b LE-C13 LE-C10 November 2008 B-24

NEI 05-04, Rev. 2 PRA Peer Review Process Table B-18 Assessment of Supporting Requirement Capability Categories For LERF Analysis HLR SR Former Capability Category Not Associated Summary of Assessment SR I II III Reviewed F&Os HLR-LE-D LE-D1a LE-D1a LE-D1b LE-D1b LE-D2 LE-D2 LE-D3 LE-D3 LE-D4 LE-D4 LE-D5 LE-D5 LE-D6 LE-D6 HLR-LE-E LE-E1 LE-E2 LE-E3 LE-E4 HLR-LE-F LE-F1 LE-F1a LE-F2 LE-F1b LE-F3 HLR-LE-G LE-G1 LE-G2 LE-G3 LE-G4 LE-G5 LE-G6 November 2008 B-25

NEI 05-04, Rev. 2 PRA Peer Review Process APPENDIX C PRA PEER REVIEW ORIENTATION/TRAINING SLIDES November 2008 C-1

NEI 05-04, Rev. 2 PRA Peer Review Process November 2008 C-2

NEI 05-04, Rev. 2 PRA Peer Review Process APPENDIX D EXAMPLE PRA PEER REVIEW LESSONS LEARNED FORM November 2008 D-1

NEI 05-04, Rev. 2 PRA Peer Review Process PRA Peer Review Team LESSONS LEARNED INPUT FORM Process Lessons Learned:

PRA Lessons Learned:

Review Team Member (optional):

Process Lessons Learned Process lessons learned are any noted Peer Review process deficiencies or enhancement ideas that may be used in the improvement of future Peer Reviews PRA Lessons Learned PRA lessons learned are any noted good PRA practices or PRA deficiencies of note that should be considered in future Peer Reviews or by utilities in the enhancement to their PRAs.

November 2008 D-2

NEI 05-04, Rev. 2 PRA Peer Review Process APPENDIX E EXAMPLE F&O Sequential Number Selection Log November 2008 E-1

NEI 05-04, Rev. 2 PRA Peer Review Process F&O Sequential Number Selection Log F&O Number Reviewer November 2008 E-2