ML18025C024
ML18025C024 | |
Person / Time | |
---|---|
Issue date: | 03/07/2018 |
From: | Office of Nuclear Reactor Regulation |
To: | |
M. Drouin 415-2091 | |
Shared Package | |
ML18025C022 | List: |
References | |
NEI 12-13 | |
Download: ML18025C024 (7) | |
Text
Enclosure The table below provides the staff's comments on NEI 12-13. A discussion of the staff's concern (issue) and the staff proposed resolution is provided in the table. In the proposed staff resolution, the staff clarification or qualification is indicated in either bolded text (i.e., bold) or strikeout text (i.e., strikeout); that is, the necessary additions or deletions to the guidance for the staff to have no objection are provided. The NRC staff's acceptance of the process described in NEI 12-13, as modified in the table below, does not mean that the NRC agrees with any conclusions of peer review team that execute NEI 12-13, as modified. The NRC staff will review new methods as needed to support risk informed activities, including licensing actions. NRC staff understand that NEI 12-13 will not be revis ed or updated; however, the NRC expects the issues described in the enclosed table to be considered by a peer review team for a peer review of a seismic, high winds, and external flooding PRAs.
ID Index Issue Proposed Staff Resolution 1 Section 2.1 It is recognized that because of the unique aspect of a seismic PRA, a form of sequencing the peer review may be needed. However, the way the guidance is written, it can be interpreted (e.g., "one week onsite") as not supporting an "in-process" approach. The guidance need to distinguish between an "in-process" and "all at once" approach. Regardless, each approach has to meet (1) the requirements of an independent peer review as stated in the PRA standard as endorsed in RG 1.200, and (2) the process described in NEI 12-13. To follow the third paragraph of Section 2.1: The peer reviews may be performed in various phases of the development of the PRA. It is recognized that the unique and discrete aspects of seismic PRA (i.e., hazard analysis, fragility analysis, and event and fault tree modeling) lends itself to some form of sequenced peer reviews that may occur during the development of the PRA (i.e., an in-process PRA peer review). However, regardless of whether the peer review being performed is an in-process peer review or a final peer review after the PRA is completed, either approach needs to meet: 1. the requirements for an independent peer review as stated in the ASME/ANS PRA standard and as endorsed in RG 1.200, and 2. the process described in NEI 12-13. Peer review findings from an in-process review may be formalized as part of that in-process peer review or deferred as a draft finding to the final peer review following the completion of the PRA. An in-process peer review is not considered to be final until the final peer review is performed following the completion of the PRA. In addition to creation of any new findings, the final peer review would assess any draft findings from in-process PRA peer reviews, which may require a re-review of the related PRA aspects. Licensees that use an in-process peer review must assure that the independence of the members of the peer review team is maintained given that those members will also participate in the final peer review.
ID Index Issue Proposed Staff Resolution 2 Page 2, Section 1.1, 2 nd paragraph Recently approved processes need to be explicitly separated from the Follow-on Peer Review to avoid confusion. -as expansive as a peer review of the entire External Hazards PRA. The F&O independent assessment process is not a substitute for the Follow-on Peer Review. 3 Page 1 1.1, 2 nd paragraph An incorrect version of the ASME/ANS PRA Standard is cited as guidance for peer review team. The Peer Review Team is guided by the high level requirements (HLRs) and supporting requirements (SRs) in the applicable Parts of the ASME/ANS PRA Standard [5]( ASME RA-Sb-2003
[6]( ASME/ANS RA-Sa-2009).
4 Page 2, Section 1.1, 3 rd and 4 th paragraph Internal events F&Os that were not appropriately "addressed" prior to the External Hazards PRA Peer Review may have a significant detrimental effect on the external hazard PRA.
- -F&Os that were not addressed closed by a focused scope peer review or independent assessment prior to the External-The review of Internal Events PRA model issues pertinent to the External Hazards PRA undergoing the peer review is required to be addressed in the self-assessment , as discuss edion in Section 1.4 below. The External Hazards PRA Peer Review is required to review all of the findings level F&Os from the internal events PRA peer review and determine whether the resolution was appropriate and in accordance with the endorsed or accepted ASME/ANS PRA standard.
5 Page 2 1.1, 3 rd paragraph The staff has a number of clarifications and qualifications to NEI 00-02 and NEI 05-04. An External Hazards PRA Peer Review requires the completion of an Internal Events PRA Peer Review (using NEI 00-02 and/or NEI 05-04 (and considering the staff's position in Regulatory Guide 1.200)) and addressing the F&Os.
6 Page 5, Section 1.4, External Hazards PRA Peer Review Preparatory Review and Self-Assessment A high-quality self-assessment is an important part of ensuring a successful External Hazard PRA peer review. However, it is unclear whether the self-assessment is required in whole or in part (throughout the document, including page A-12 which indicates it is "optional but recommended"). The self-assessment is key to ensuring that the overall Peer Review process is completed within the scheduled time and that all of the required review is completed. The self-assessment is required to be performed prior to the peer review and must include a self-assessment of:
- The referenced Internal Events PRA against the SRs listed in Table D-1
- The seismic, high winds, or external flooding PRA against the respective SRs listed in Part 5, Part 7, or Part 8.
ID Index Issue Proposed Staff Resolution 7 Section 1.5, Last paragraph The peer review should also determine the appropriateness of the identification of unit-specific analysis needs in addition to the actual analysis. -additional time for walkdowns and review of the identification, analysis and documentation-8 Section 1.6, 1 st paragraph Standards are moving away from three capability categories. Three Different Capability Category levels-9 Section 1.6, 1 st paragraph The threshold for a "Not Met" should be similar to that for a "Met". Therefore, it is not necessary that none of the requirements are met. -may result in a "Not Met" assignment when none a preponderance of the requirements for an SR capability requirement are not met. 10 Page 15, Step 4, 2 nd paragraph Any resolved Inquiries that are used in the interpretation of SR(s) for the peer review need to be documented explicitly. -and discussed in a pre-visit telecom, as necessary. Any resolved Inquiries that are used as part of the peer review will be documented in the peer review report along with the specific SRs that were interpreted using each Inquiry. 11 Page 18, Step 11 New information should not be provided subsequent to the peer review team's departure from the peer-review location. The peer-review is intended to capture the "snapshot" of the model. New information subsequent to the departure of the peer review team is outside of the scope of the peer review and should be part of the resolution of the F&O/open item. Providing information after the peer review team has left the site is also inconsistent with the performance of actual peer reviews. New information provided- -this new information. 12 Page 14 Section 2.1, 4 th paragraph The staff has a clarification to Section 4-2.2 of ASME/ANS Ra-Sa-2009.
Selection of a Peer Review Team can also-The host utility can request particular expertise beyond the general expertise identified in the respective "Peer-Review Team Composition and Personnel Qualifications" section of each Part of the ASME/ ANS PRA Standard (and considering the staff's position in Regulatory Guide 1.200), if more specialized skills are needed. 13 Page 20 2.2, footnote 8 The External Hazards PRA Peer Review Team should meet the requirements in Sections 1-6.2 and the peer review section in each applicable external hazard Part of the ASME/ANS PRA Standard. In addition to the requirements in Section 1-6, each Part of the PRA Standard includes requirements-.. the review team should be is assembled to meet those requirements.
ID Index Issue Proposed Staff Resolution 14 Page 20, Section 2.2, 2 nd paragraph There have been some recent questions and concerns regarding the independence of peer review team or independent assessment team members. -With the exception of individuals who have worked on or directly supervised the subject PRA, there are no automatic exclusion criteria; however, the host utility may question the independence of any proposed Peer Review Team member.
The term "worked on" is intended to include any utility staff or contractors that had any association with the portion of the External Hazard PRA that they are reviewing. Similarly, an external hazard PRA team member who had an association with the basis internal events PRA model would not meet the independence requirement for reviewing the closure of the associated internal events findings.
15 Page 21 2.2, Paragraph before numbered bullets The External Hazards PRA Peer Review Team should meet the requirements in Sections 1-6.2 and the peer review section in each applicable external hazard Part of the ASME/ANS PRA Standard. The desired needed attributes of the Peer Review Team, as a whole, are as follows: 16 Page 21 2.2, 3 rd paragraph There is some confusion in the use of the terms 'expert' and 'reviewer with experience.' The intent is to ensure that there is more than one peer reviewer with experience expert in each key External Hazards PRA process, but not to require two experts in each skill set. 17 Page 22 2.2, bullet titles The External Hazards PRA Peer Review Team should meet the requirements in Sections 1-6.2 and the peer review section in each applicable external hazard Part of the ASME/ANS PRA Standard.
- Experience Expectations Needs for Peer Review Team Lead:-..
- Experience Expectations Needs for Individual Peer Review Team Members:-..
- Additional Experience Expectations Needs for the Team as a Whole-.. 18 Page 22 2.2, 7 th paragraph, last sub-bullet The External Hazards PRA Peer Review Team should meet the requirements in Sections 1-6.2 and the peer review section in each applicable external hazard Part of the ASME/ANS PRA Standard. Specialized expertise in seismic, high winds, external flood or other External Hazards PRAs should be strongly considered is needed if these hazards are being reviewed. 19 Page 23 2.2, 1 st paragraph The language about the review team is confusing. The peer review team should have fully qualified members. The process requires the reviewers to follow a very tight schedule and is most likely to be successful if the team needs to consist of fully qualified members.
ID Index Issue Proposed Staff Resolution 20 Page 22 2.2, footnote 9 Education beyond the Bachelor's degree does not necessarily equate to practical experience. Additionally, a directly applicable advanced degree in Engineering/Science/Mathematics can may be counted towards years of experience. 21 Page 25, Section 3.2, 2 nd paragraph The requirement to review the changes to the internal events model against appropriate Part 2 SRs is included, but the requirement to review all findings level internal events PRA F&Os and their dispositions is not included. Add a paragraph to discuss the requirements associated with reviewing the internal events PRA F&Os and their disposition. 22 Page 26, Section 3.2, 1 st paragraph Any resolved Inquiries that are used in the interpretation of SR(s) for the peer review need to be documented explicitly. -assignment of a Capability Category for the SR. All such instances will be documented in the peer review report along with the specific SRs that were interpreted using each Inquiry.
23 Page 26, Section 3.2, last paragraph Based on lessons learned in the implementation of NEI 07-12, the definition of a UAM should be revised to include all new methods or changes to existing methods which have not been vetted by a broad technical community, even if they were reviewed by the peer reviewer. Such methods should be flagged as UAMs and documented in the peer-review report. A definition of what constitutes a new method is necessary which is consistent with established staff position. Unreviewed Analysis Method - an observation regarding the use of methods that are new or beyond the expected expertise of the review team or, and for which the review would exceed the time and capability of the External Hazards PRA Peer Review team. When aAn F&O is written with this classification, would need the method would need to be reviewed by a separate body of experts.
New Method - an observation regarding the use of methods that are new. An F&O written with this classification will be reviewed during the peer review, the peer review report will identify it explicitly as a new method along with the aspect(s) that makes it novel, and the reason(s) why the method was found to be acceptable or unacceptable (in whole or part) to the peer review team. [F&Os with this classification cannot be closed out via the F&O closure process described in NEI 05-04/07-12/12-13 Appendix X process (ML17086A431) or a follow-on peer review unless the method no longer meets the current definition of "new method".]
24 Page 27, Section 3.2, footnote The External Events PRA expert panel has not been set-up in the 5 years since this trial use version of the guidance was issued. An External Events PRA expert panel may be formed by the industry to evaluate Unreviewed Analysis Method F&Os to assist utilities in dispositioning these items. As yet, the industry has not agreed to form such an expert panel. [The NRC will request UAMs as per the above definition if needed to support risk informed activities, including licensing actions.]
ID Index Issue Proposed Staff Resolution 25 Page 28, Section 3.2, 1 st paragraph It is not clear whether the terms "key assumptions and uncertainty sources" are used in a manner consistent with RG 1.200. -key assumptions and uncertainty sources- 26 Page 27 3.2, 3 rd paragraph Combining F&Os should be the exception rather than the rule. This could be misinterpreted by the current language as being the reverse.
When writing the F&Os, it is important to note that the reviewers should need not match F&Os to SRs one-to-one; however, there may be situations where the need to match is not always warranted. For example, F&Os on common SRs that cross several PRA technical elements should be combined into a single F&O (i.e., uncertainty,-. 27 Page 27 3.2, 5 th paragraph Some related requirements from the Part 2 of the ASME/ANS PRA standard, are incorporated by reference. Section 1-6.3 of the ASME internal events PRA standard states: "The review team shall use the requirements of the Peer Review Section of each respective Part of this Standard for the PRA Elements being reviewed to determine if the methodology and the implementation of the methodology for each PRA Element meet the requirement of this Standard." Further it states: "The HLRs and the composite of the SRs of the Technical Requirements Section of each respective Part of this Standard shall be used by the peer review team to assess the completeness of a PRA Element." Contrary to this, NEI 07-12 would allow the peer review team to "- elect
to skip selected SRs." During the review of a given technical element, the Lead Reviewer may elect to skip selected SRs. - document their basis for not reviewing the given SR.
While Section 1-6.1 of the ASME/ANS PRA standard states that not all aspects of the PRA need to be assessed, this statement is intended to limit how much of the model needs to be considered when determining whether an SR or HLR is met. The SRs form the basis for determining whether the related HLR is met, and every SR in the HLR needs to be assessed by the review team.
28 Page 28 3.2, 1 st paragraph Although the context implies as much, it is only the model uncertainty characterization that should be qualitative. Parameter uncertainty should be quantitative. The host utility should provide at least a qualitative characterization of model uncertainty. 29 Page 29 3.3, 2 nd paragraph One major benefit of the peer review process is the SR assignments, since these assignments improve the efficiency of NRC's review of a risk-informed submittal. The major benefits of the review process however, is not are the SR assignments, but rather as well as the recommendations for improvements and the acknowledgements of the unique strengths of the PRA.
ID Index Issue Proposed Staff Resolution 30 Page 30 3.3, 2 nd paragraph after table Since Capability Category II is the general goal for SRs in an External Events PRA, those SRs which receive a Capability Category I rating are important to the review of any application of a FPRA. Thus, a finding should be written for any SR receiving a Capability Category I, even if the licensee has stated that is all they are trying to achieve. Any SR which is "not met" should be accompanied by one or more F&Os. When a reviewer assesses an SR as a Capability Category I or not met, there must be an F&O (finding) written. The host utility may request that the Peer Review Team review against Capability Category I or Capability Category II; this choice may be made on a per-technical element basis. If the host utility chooses to be reviewed against Capability Category I for a given SR, an F&O need not be written for those SRs if assessed as Capability Category I the utility does not necessarily need to address any findings based on an assessment of Capability Category I. Further, it is important to note that the team may write an F&O regardless of the Capability Category assessment for a given SR. It is expected required that a "Finding" F&O is written for an SR assessed as Not Met--.. 31 Page 32 3.4, last bullet Confusing language A certain level of subjectivity is expected -..this does not necessarily mean that the SR is considered not met. There should be a preponderance of evidence to conclude that an SR is not met. Any noncompliance should be documented with an F&O. However, there should be a preponderance of evidence to conclude that an SR is not met. 32 Page 34, Section 4.1, 2nd paragraph The peer review report needs to explicitly identify any new methods that were reviewed. (fourth bullet) Any new methods including aspect(s) of the new method that make it novel, the reason(s) why the method was reviewed and found to be acceptable or unacceptable to the peer review team along with the identification of any associated findings.
[This does not imply NRC acceptance of the peer review team's conclusions. The NRC staff will review new methods as needed to support risk informed activities, including licensing actions. ]