ML18025C024

NEI 12-13 Staff Comments

ML18025C024
Person / Time
Issue date:	03/07/2018
From:	Office of Nuclear Reactor Regulation
To:
M. Drouin 415-2091
Shared Package
ML18025C022	List: ML18025C024 ML18025C025
References
NEI 12-13
Download: ML18025C024 (7)

Text

The table below provides the staffs comments on NEI 12-13. A discussion of the staffs concern (issue) and the staff proposed resolution is provided in the table. In the proposed staff resolution, the staff clarification or qualification is indicated in either bolded text (i.e., bold) or strikeout text (i.e., strikeout); that is, the necessary additions or deletions to the guidance for the staff to have no objection are provided.

The NRC staffs acceptance of the process described in NEI 12-13, as modified in the table below, does not mean that the NRC agrees with any conclusions of peer review team that execute NEI 12-13, as modified. The NRC staff will review new methods as needed to support risk informed activities, including licensing actions. NRC staff understand that NEI 12-13 will not be revised or updated; however, the NRC expects the issues described in the enclosed table to be considered by a peer review team for a peer review of a seismic, high winds, and external flooding PRAs.

ID Index Issue Proposed Staff Resolution Section 2.1 It is recognized that because of the unique aspect of a To follow the third paragraph of Section 2.1:

seismic PRA, a form of sequencing the peer review may be needed. However, the way the guidance is written, it can be The peer reviews may be performed in various phases of the development of interpreted (e.g., one week onsite) as not supporting an the PRA. It is recognized that the unique and discrete aspects of seismic in-process approach. The guidance need to distinguish PRA (i.e., hazard analysis, fragility analysis, and event and fault tree between an in-process and all at once approach. modeling) lends itself to some form of sequenced peer reviews that may Regardless, each approach has to meet (1) the requirements occur during the development of the PRA (i.e., an in-process PRA peer of an independent peer review as stated in the PRA standard review). However, regardless of whether the peer review being performed is as endorsed in RG 1.200, and (2) the process described in an in-process peer review or a final peer review after the PRA is completed, NEI 12-13. either approach needs to meet:

1. the requirements for an independent peer review as stated in the ASME/ANS PRA standard and as endorsed in RG 1.200, and 1

2. the process described in NEI 12-13.

Peer review findings from an in-process review may be formalized as part of that in-process peer review or deferred as a draft finding to the final peer review following the completion of the PRA. An in-process peer review is not considered to be final until the final peer review is performed following the completion of the PRA. In addition to creation of any new findings, the final peer review would assess any draft findings from in-process PRA peer reviews, which may require a re-review of the related PRA aspects.

Licensees that use an in-process peer review must assure that the independence of the members of the peer review team is maintained given that those members will also participate in the final peer review.

Enclosure

ID Index Issue Proposed Staff Resolution Page 2, Recently approved processes need to be explicitly separated as expansive as a peer review of the entire External Hazards PRA. The F&O Section 1.1, from the Follow-on Peer Review to avoid confusion. independent assessment process is not a substitute for the Follow-on Peer 2

2nd Review.

paragraph Page 1 An incorrect version of the ASME/ANS PRA Standard is cited The Peer Review Team is guided by the high level requirements (HLRs) and 3 as guidance for peer review team. supporting requirements (SRs) in the applicable Parts of the ASME/ANS PRA 1.1, 2nd Standard [5]( ASME RA-Sb-2003 [6]( ASME/ANS RA-Sa-2009).

paragraph Page 2, Internal events F&Os that were not appropriately addressed

• F&Os that were not addressed closed by a focused scope peer review or Section 1.1, prior to the External Hazards PRA Peer Review may have a independent assessment prior to the External 3rd and 4th significant detrimental effect on the external hazard PRA.

paragraph The review of Internal Events PRA model issues pertinent to the External Hazards PRA undergoing the peer review is required to be addressed in the 4 self-assessment, as discussedion in Section 1.4 below. The External Hazards PRA Peer Review is required to review all of the findings level F&Os from the internal events PRA peer review and determine whether the resolution was appropriate and in accordance with the endorsed or accepted ASME/ANS PRA standard.

Page 2 The staff has a number of clarifications and qualifications to An External Hazards PRA Peer Review requires the completion of an Internal 5 NEI 00-02 and NEI 05-04. Events PRA Peer Review (using NEI 00-02 and/or NEI 05-04 (and considering 1.1, 3rd the staffs position in Regulatory Guide 1.200)) and addressing the F&Os.

paragraph Page 5, A high-quality self-assessment is an important part of The self-assessment is key to ensuring that the overall Peer Review process is Section 1.4, ensuring a successful External Hazard PRA peer review. completed within the scheduled time and that all of the required review is External However, it is unclear whether the self-assessment is completed. The self-assessment is required to be performed prior to the peer Hazards required in whole or in part (throughout the document, review and must include a self-assessment of:

PRA Peer including page A-12 which indicates it is optional but Review recommended).

• The referenced Internal Events PRA against the SRs listed in Table 6 D-1 Preparatory Review and

• The seismic, high winds, or external flooding PRA against the Self- respective SRs listed in Part 5, Part 7, or Part 8.

Assessment ID Index Issue Proposed Staff Resolution Section 1.5, The peer review should also determine the appropriateness additional time for walkdowns and review of the identification, analysis and Last of the identification of unit-specific analysis needs in addition documentation 7 paragraph to the actual analysis.

Section 1.6, Standards are moving away from three capability categories. Three Different Capability Category levels 8 1st paragraph Section 1.6, The threshold for a Not Met should be similar to that for a may result in a Not Met assignment when none a preponderance of the 9 1st Met. Therefore, it is not necessary that none of the requirements for an SR capability requirement are not met.

paragraph requirements are met.

Page 15, Any resolved Inquiries that are used in the interpretation of and discussed in a pre-visit telecom, as necessary. Any resolved Inquiries that Step 4, 2nd SR(s) for the peer review need to be documented explicitly. are used as part of the peer review will be documented in the peer review 10 paragraph report along with the specific SRs that were interpreted using each Inquiry.

Page 18, New information should not be provided subsequent to the New information provided this new information.

Step 11 peer review teams departure from the peer-review location.

The peer-review is intended to capture the snapshot of the model. New information subsequent to the departure of the 11 peer review team is outside of the scope of the peer review and should be part of the resolution of the F&O/open item.

Providing information after the peer review team has left the site is also inconsistent with the performance of actual peer reviews.

Page 14 The staff has a clarification to Section 4-2.2 of ASME/ANS Selection of a Peer Review Team can alsoThe host utility can request particular Ra-Sa-2009. expertise beyond the general expertise identified in the respective "Peer-Review 12 Section 2.1, Team Composition and Personnel Qualifications" section of each Part of the 4th ASME/ ANS PRA Standard (and considering the staffs position in Regulatory paragraph Guide 1.200), if more specialized skills are needed.

Page 20 The External Hazards PRA Peer Review Team should meet In addition to the requirements in Section 1-6, each Part of the PRA Standard 2.2, the requirements in Sections 1-6.2 and the peer review includes requirements.. the review team should be is assembled to meet those 13 footnote 8 section in each applicable external hazard Part of the requirements.

ASME/ANS PRA Standard.

ID Index Issue Proposed Staff Resolution Page 20, There have been some recent questions and concerns With the exception of individuals who have worked on or directly supervised the Section 2.2, regarding the independence of peer review team or subject PRA, there are no automatic exclusion criteria; however, the host utility 2nd independent assessment team members. may question the independence of any proposed Peer Review Team member.

paragraph The term worked on is intended to include any utility staff or contractors 14 that had any association with the portion of the External Hazard PRA that they are reviewing. Similarly, an external hazard PRA team member who had an association with the basis internal events PRA model would not meet the independence requirement for reviewing the closure of the associated internal events findings.

Page 21 The External Hazards PRA Peer Review Team should meet The desired needed attributes of the Peer Review Team, as a whole, are as 2.2, the requirements in Sections 1-6.2 and the peer review follows:

section in each applicable external hazard Part of the 15 Paragraph ASME/ANS PRA Standard.

before numbered bullets Page 21 There is some confusion in the use of the terms expert and The intent is to ensure that there is more than one peer reviewer with experience 2.2, reviewer with experience. expert in each key External Hazards PRA process, but not to require two experts 16 in each skill set.

3rd paragraph Page 22 The External Hazards PRA Peer Review Team should meet

• Experience Expectations Needs for Peer Review Team Lead:..

2.2, the requirements in Sections 1-6.2 and the peer review

• Experience Expectations Needs for Individual Peer Review Team 17 section in each applicable external hazard Part of the Members:..

bullet titles ASME/ANS PRA Standard.

• Additional Experience Expectations Needs for the Team as a Whole..

Page 22 The External Hazards PRA Peer Review Team should meet Specialized expertise in seismic, high winds, external flood or other External 2.2, the requirements in Sections 1-6.2 and the peer review Hazards PRAs should be strongly considered is needed if these hazards are being section in each applicable external hazard Part of the reviewed.

18 7th ASME/ANS PRA Standard.

paragraph, last sub-bullet Page 23 The language about the review team is confusing. The peer The process requires the reviewers to follow a very tight schedule and is most 2.2, review team should have fully qualified members. likely to be successful if the team needs to consist of fully qualified members.

19 1st paragraph ID Index Issue Proposed Staff Resolution Page 22 Education beyond the Bachelor's degree does not Additionally, a directly applicable advanced degree in 20 2.2, necessarily equate to practical experience. Engineering/Science/Mathematics can may be counted towards years of experience.

footnote 9 Page 25, The requirement to review the changes to the internal events Add a paragraph to discuss the requirements associated with reviewing the internal Section 3.2, model against appropriate Part 2 SRs is included, but the events PRA F&Os and their disposition.

21 2nd requirement to review all findings level internal events PRA paragraph F&Os and their dispositions is not included.

Page 26, Any resolved Inquiries that are used in the interpretation of assignment of a Capability Category for the SR. All such instances will be Section 3.2, SR(s) for the peer review need to be documented explicitly. documented in the peer review report along with the specific SRs that were 22 1st interpreted using each Inquiry.

paragraph Page 26, Based on lessons learned in the implementation of NEI 07- Unreviewed Analysis Method - an observation regarding the use of methods that Section 3.2, 12, the definition of a UAM should be revised to include all are new or beyond the expected expertise of the review team or, and for which the last new methods or changes to existing methods which have not review would exceed the time and capability of the External Hazards PRA Peer paragraph been vetted by a broad technical community, even if they Review team. When aAn F&O is written with this classification, would need the were reviewed by the peer reviewer. Such methods should method would need to be reviewed by a separate body of experts.

be flagged as UAMs and documented in the peer-review report. A definition of what constitutes a new method is necessary which is consistent with established staff position. New Method - an observation regarding the use of methods that are new. An 23 F&O written with this classification will be reviewed during the peer review, the peer review report will identify it explicitly as a new method along with the aspect(s) that makes it novel, and the reason(s) why the method was found to be acceptable or unacceptable (in whole or part) to the peer review team. [F&Os with this classification cannot be closed out via the F&O closure process described in NEI 05-04/07-12/12-13 Appendix X process (ML17086A431) or a follow-on peer review unless the method no longer meets the current definition of new method.]

Page 27, The External Events PRA expert panel has not been set-up An External Events PRA expert panel may be formed by the industry to evaluate Section 3.2, in the 5 years since this trial use version of the guidance was Unreviewed Analysis Method F&Os to assist utilities in dispositioning these items.

24 footnote issued. As yet, the industry has not agreed to form such an expert panel. [The NRC will request UAMs as per the above definition if needed to support risk informed activities, including licensing actions.]

ID Index Issue Proposed Staff Resolution Page 28, It is not clear whether the terms key assumptions and key assumptions and uncertainty sources Section 3.2, uncertainty sources are used in a manner consistent with 25 1st RG 1.200.

paragraph Page 27 Combining F&Os should be the exception rather than the When writing the F&Os, it is important to note that the reviewers should need not 3.2, rule. This could be misinterpreted by the current language as match F&Os to SRs one-to-one; however, there may be situations where the 26 being the reverse. need to match is not always warranted. For example, F&Os on common SRs 3rd that cross several PRA technical elements should be combined into a single F&O paragraph (i.e., uncertainty,.

Page 27 Some related requirements from the Part 2 of the ASME/ANS During the review of a given technical element, the Lead Reviewer may elect to 3.2, PRA standard, are incorporated by reference. Section 1-6.3 skip selected SRs. document their basis for not reviewing the given SR.

of the ASME internal events PRA standard states:

5th paragraph The review team shall use the requirements of the Peer Review Section of each respective Part of this Standard for While Section 1-6.1 of the ASME/ANS PRA standard states that not all the PRA Elements being reviewed to determine if the aspects of the PRA need to be assessed, this statement is intended to limit methodology and the implementation of the methodology for how much of the model needs to be considered when determining whether 27 an SR or HLR is met. The SRs form the basis for determining whether the each PRA Element meet the requirement of this Standard.

Further it states: The HLRs and the composite of the SRs of related HLR is met, and every SR in the HLR needs to be assessed by the the Technical Requirements Section of each respective Part review team.

of this Standard shall be used by the peer review team to assess the completeness of a PRA Element. Contrary to this, NEI 07-12 would allow the peer review team to elect to skip selected SRs.

Page 28 Although the context implies as much, it is only the model The host utility should provide at least a qualitative characterization of model 3.2, uncertainty characterization that should be qualitative. uncertainty.

28 Parameter uncertainty should be quantitative.

1st paragraph Page 29 One major benefit of the peer review process is the SR The major benefits of the review process however, is not are the SR assignments, 3.3, assignments, since these assignments improve the efficiency but rather as well as the recommendations for improvements and the 29 of NRC's review of a risk-informed submittal. acknowledgements of the unique strengths of the PRA.

2nd paragraph ID Index Issue Proposed Staff Resolution Page 30 Since Capability Category II is the general goal for SRs in an When a reviewer assesses an SR as a Capability Category I or not met, there 3.3, External Events PRA, those SRs which receive a Capability must be an F&O (finding) written. The host utility may request that the Peer Category I rating are important to the review of any Review Team review against Capability Category I or Capability Category II; this 2nd application of a FPRA. Thus, a finding should be written for choice may be made on a per-technical element basis. If the host utility chooses to paragraph any SR receiving a Capability Category I, even if the licensee be reviewed against Capability Category I for a given SR, an F&O need not be 30 after table has stated that is all they are trying to achieve. Any SR written for those SRs if assessed as Capability Category I the utility does not which is not met should be accompanied by one or more necessarily need to address any findings based on an assessment of F&Os. Capability Category I. Further, it is important to note that the team may write an F&O regardless of the Capability Category assessment for a given SR. It is expected required that a "Finding" F&O is written for an SR assessed as Not Met..

Page 32 Confusing language A certain level of subjectivity is expected ..this does not necessarily mean that 3.4, the SR is considered not met. There should be a preponderance of evidence to 31 conclude that an SR is not met. Any noncompliance should be documented with last bullet an F&O. However, there should be a preponderance of evidence to conclude that an SR is not met.

Page 34, The peer review report needs to explicitly identify any new (fourth bullet) Any new methods including aspect(s) of the new method that Section 4.1, methods that were reviewed. make it novel, the reason(s) why the method was reviewed and found to be 2nd acceptable or unacceptable to the peer review team along with the paragraph identification of any associated findings.

32

[This does not imply NRC acceptance of the peer review teams conclusions.

The NRC staff will review new methods as needed to support risk informed activities, including licensing actions. ]