ML16211A132
| ML16211A132 | |
| Person / Time | |
|---|---|
| Site: | Vogtle  |
| Issue date: | 08/11/2016 |
| From: | Mary Drouin, Alan Kuritzky NRC/RES/DRA/PRAB |
| To: | |
| Alan Kuritzky 301-415-1552 | |
| References | |
| Download: ML16211A132 (53) | |
Text
Level 3 PRA Project QA Plan, Rev. 0 Level 3 PRA Project QA Plan Rev.
Date Description 0
08/11/2016 Original issuance*
- Earlier versions of the Level 3 PRA project QA plan appear as Section 18 of the Level 3 PRA project technical analysis approach plan (ADAMS Accession No. ML13296A064)
Level 3 PRA Project QA Plan, Rev. 0
Level 3 PRA Project QA Plan, Rev. 0 i
Contents List of Acronyms........................................................................................................................ iv 1
Established Methods, Tools and Data.................................................................................. 1 2
Qualified Personnel.............................................................................................................. 1 3
PRA Model Configuration Control......................................................................................... 2 4
Technical Reviews............................................................................................................... 3 4.1 Technical Advisory Group............................................................................................ 3 4.2 Internal Self-Assessment............................................................................................. 5 4.3 External Peer Reviews.............................................................................................. 16 4.3.1 PRA Standard Peer Review........................................................................ 16 4.3.2 Independent Expert Peer Review................................................................ 18 4.4 Advisory Committee on Reactor Safeguards............................................................. 19 4.5 Public Review and Comment..................................................................................... 19 5
Documentation Control....................................................................................................... 19 5.1 Storage and Access of Project Information................................................................ 20 5.2 Upload of Information onto the SharePoint Site......................................................... 21 5.3 Documentation Control of Licensee Information........................................................ 21 5.4 Documentation Backup............................................................................................. 22 5.5 Use of External Storage Media.................................................................................. 22 5.6 Working Document Folders....................................................................................... 22 5.7 Use of Templates and Forms for Documentation....................................................... 23 5.8 Site Visits.................................................................................................................. 26 5.9 Documentation Control for NRC Contractors............................................................. 28 5.10 Non-Disclosure Agreement to Allow Access to Proprietary Information..................... 29 5.11 Project Documentation Markings............................................................................... 29 5.12 Guidance for Addressing Potential Technical Issues................................................. 30 5.13 Future Plant Modifications......................................................................................... 35 5.14 Organization of the Various Types of Information on the SharePoint Site.................. 35 6
Technical Reports.............................................................................................................. 37 6.1 Staff Technical Reports............................................................................................. 37 6.2 Contractor Technical Reports.................................................................................... 42 7
QA Program Implementation Audits................................................................................... 43
Level 3 PRA Project QA Plan, Rev. 0 ii LIST OF TABLES Table 1 TAG Review Documentation Template........................................................................ 4 Table 2 Self-Assessment and Peer Review Criteria Where Standards Do Not Exist................ 7 Table 3 Self-Assessment Documentation Template................................................................. 8 Table 4 Overall Results of Self-Assessment Process............................................................... 9 Table 5 Example: Mapping of the HLRs of SFP PRA and At-Power Level 1 PRA..................10 Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA.............................11 Table 7 External Peer Review Documentation Template........................................................17 Table 8 Documentation of Meetings and Discussions.............................................................23 Table 9 Documentation for Level 1 Internal Events Initiating Event Analysis...........................24 Table 10 Issue Documentation Template................................................................................34 Table 11 Summary of Level 3 PRA Project Information on SharePoint Site............................36 Table 12 Sign-Off Sheet for Staff Technical Reports...............................................................38 Table 13 Document Checklist.................................................................................................39 Table 14 QA Review and Acceptance Form for Staff (Supporting) Technical Reports.............42 Table 15 QA Review and Acceptance Form for Contractor Technical Reports........................43 LIST OF FIGURES Figure 1 Process Used for Self-Assessment............................................................................. 5
Level 3 PRA Project QA Plan, Rev. 0 iv List of Acronyms ACRS Advisory Committee on Reactor Safeguards CDF Core Damage Frequency COR Contracting Office Representative DCS Dry Cask Storage DORL Division of Operating Reactor Licensing DRA Division of Risk Analysis HLR High Level Requirement IEPR Independent Expert Peer Review INL Idaho National Laboratory LERF Large Early Release Frequency LRF Large Release Frequency NRC U.S. Nuclear Regulatory Commission NRR Office of Nuclear Reactor Regulation PRA Probabilistic Risk Assessment PRAB Probabilistic Risk Assessment Branch PWR Pressurized Water Reactor PWR OG PWR Owners Group QA Quality Assurance QC Quality Control RES Office of Nuclear Regulatory Research RG Regulatory Guide SFP Spent Fuel Pool SNC Southern Nuclear Operating Company SPAR Standardized Plant Analysis Risk SR Supporting Requirement SRA Senior Reactor Analyst SSC Structure, System, and Component TAAP Technical Analysis Approach Plan TAG Technical Advisory Group
Level 3 PRA Project QA Plan, Rev. 0
Level 3 PRA Project QA Plan, Rev. 0 1
QUALITY ASSURANCE PLAN Quality Assurance (QA) is a key factor in any analysis to ensure and demonstrate the technical acceptability of the analysis and probabilistic risk assessment (PRA) model fidelity.
The objective of QA is to ensure that both the technical approach (methods, tools, data) is appropriate, and that implementation of the technical approach is appropriately performed.
To achieve this objective, QA involves seven major elements which are discussed in the following sections:
Section 1 - Use of established methods, tools and data Section 2 - Qualified personnel Section 3 - PRA model configuration control Section 4 - Technical review of the methods, tools, data, and developed models Section 5 - Documentation control Section 6 - Technical reports Section 7 - QA program implementation audits 1
ESTABLISHED METHODS, TOOLS AND DATA The PRA model will generally be based on state-of-practice methods, tools (e.g., computer codes) and data, that is, those that have been established and accepted (including verification and validation where applicable) in the risk community (i.e., U.S. Nuclear Regulatory Commission (NRC) and industry). Examples of sources include:
Consensus standards Internal and external guidance documents Accepted generic structures, systems and components (SSCs) performance data (where plant specific data is not available)
Validated codes For each technical task1, the method, tools and data being used will be documented along with the basis for their acceptability (e.g., NRC endorsement). This documentation is identified in each technical task in Technical Analysis Approach Plan (TAAP) report and described in Section 5.
2 QUALIFIED PERSONNEL Qualified individuals are needed to perform the work. Their qualifications depend on whether the analyst is (1) a performer or (2) a reviewer.
A performer is an individual who develops some aspect of the PRA model. Their role, either as a team leader, a task leader, or an analyst will need to have some level of expertise.
Certainly, an analyst can develop the qualifications with on the job training; however, the task and team leaders need to be more experienced personnel who bring actual experience in the area they are leading. If an analyst has little to no experience, their work will be closely supervised and monitored by their task leader. PRA consensus standards and Regulatory 1 Technical tasks are the technical steps that will be performed to accomplish the technical element.
Level 3 PRA Project QA Plan, Rev. 0 2
Guide (RG) 1.2002 do not prescribe qualifications for the team performing the actual work.
Moreover, one of the major objectives of the Level 3 PRA project is to train inexperienced staff in how to construct a PRA model.
A reviewer is an individual who has some role in reviewing the actual work and making judgments with regard to its technical acceptability. In this regard, these individuals must have a certain level of expertise and on the job training is not acceptable. Both RG 1.200 and the PRA standards provides peer review personnel qualifications. These requirements should be met unless otherwise justified.
3 PRA MODEL CONFIGURATION CONTROL Ensuring that the analysts are using the same information and same models and that the reviews are being performed on the most recent model and documentation is important in ensuring the fidelity of the PRA model. Developing a PRA model involves numerous tasks being performed by many different analysts. It is, therefore, essential that the information collected and the models developed for this project be controlled so that all of the analysts use the same information and models. The control of the developed models is discussed in this section. The control of information is discussed in Section 5.
The Idaho National Laboratory (INL) will host and maintain the SAPHIRE-based models developed as part of the Level 3 PRA project. INL will provide the necessary technical management and oversight to ensure efforts by INL or NRC staff (including work performed by other NRC contractors and provided to INL by the NRC) to create, revise or otherwise modify the Level 3 PRA project models are coordinated and the models are properly integrated. These model enhancements may include the creation, addition, revision or other modification of a low-power/shutdown model, all-hazards model (e.g., fire, external flooding, seismic, etc.), Level 2 PRA model, multi-unit model, spent fuel pool model, or other extended model applicable to the construct of the overall Level 3 PRA project model.
To the extent practicable, the methodology, quality, and philosophy used to develop the current set of Standardized Plant Analysis Risk (SPAR) models for the 100 operating commercial nuclear power plants will be used to develop the external event model, low-power/shutdown model, extended Level 1 PRA model, and Level 2 PRA model for the Level 3 PRA project. This includes model construct, event nomenclature, assumptions, preferred technical positions, and other key aspects of the existing models to allow NRC staff the ease of use of the models.
INL will identify a single point of contact to act as the Level 3 PRA project model coordinator (Coordinator). The Coordinator will maintain a log and track all permanent revisions to the model including the reason for the revision, assumptions, deviations from preferred technical positions, and any other information deemed important to understanding the model or the revision to the model. The Coordinator will ensure that the appropriate model revision is being used and that the effort results in a properly integrated model. The Coordinator will also coordinate INL model integration activities. Version control software, suitable to this task and with sufficient documentation capabilities, may be used by INL, subject to approval by the NRC staff.
2 Regulatory Guide1.200, An Approach For Determining The Technical Adequacy Of Probabilistic Risk Assessment Results For Risk-Informed Activities, Revision 1, U.S. Nuclear Regulatory Commission, Washington, DC, January 2007.
Level 3 PRA Project QA Plan, Rev. 0 3
When multiple revisions to the enhanced Vogtle model are planned by INL or NRC staff, INL will coordinate the activities of the different modelers. This is to ensure that the model developers use the appropriate model version(s) and that the final product does not include models that were constructed based on an obsolete model version.
INL will also perform quality control (QC) and QA reviews of the new or revised models. This is to ensure that the model represents the as-built, as-operated plant to the extent practicable. Similar QA criteria and processes used for the existing SPAR models will be used to review the Level 3 PRA project models. This includes (as appropriate and as practical) satisfying the criteria and processes in the Standardized Plant Analysis Risk (SPAR) Model QA Plan,3 the latest approved INL QC/QA processes, applicable sections of Volume 3 of the RASP Handbook,4 RG 1.200, and other applicable guidance.5 4
TECHNICAL REVIEWS In ensuring technical acceptability, different types of review will be performed. These involve five types which are discussed in the following sections:
Section 4.1 - review by a Technical Advisory Group Section 4.2 - internal self-assessment Section 4.3 - external peer reviews Section 4.4 - review by the Advisory Committee on Reactor Safeguards Section 4.5 - public review and comment Each of these reviews has different objectives and scope which are described below.
4.1 Technical Advisory Group The objective of the Technical Advisory Group (TAG), as specified in the TAG charter,6 is to:
(1) review progress in the development of the Level 3 PRA, and (2) provide insight, advice, and guidance on (a) the technical bases, tools, methods, models, and data for the project, (b) the interpretation of the results of the various PRA models and the overall PRA model, and (c) the response to comments received from the external peer reviews of the study. In this role, the TAG will serve as an ongoing review team that will provide review and feedback as the project progresses. Also, as part of its initial review responsibility, the TAG will review the TAAP to provide feedback on the approach being used to perform the work.
3 Standardized Plant Analysis Risk (SPAR) Model QA Plan, Revision 0, U.S. Nuclear Regulatory Commission, Washington, DC, September 2006 (not publicly available).
4 Risk Assessment of Operational Events Handbook, Volume 3 - SPAR Model Reviews, Revision 1, U.S.
Nuclear Regulatory Commission, Washington, DC, September 2007 (not publicly available).
5 For example: American Nuclear Society, American National Standard External-Events PRA Methodology, ANSI/ANS-58.21-2003, December 2003.
6 Charter for the Technical Advisory Group on the Full-Scope Site Level 3 Probabilistic Risk Assessment Project, ADAMS Accession Number ML120410123 (not publicly available).
Level 3 PRA Project QA Plan, Rev. 0 4
As stated earlier, the approach used for the Level 3 PRA project will be based on plant information and established methods, tools and data. Where the plant information or the methods, tools or data do not exist to develop certain aspects of the PRA model, other sources such as expert opinion will be used. The TAG will play a key role in addressing the acceptability of such proposed approaches. Furthermore, it is expected that the TAG will play a fundamental role in resolving technical or programmatic issues that may arise.
The TAG will consist of senior technical staff in the area of PRA, and in supporting technical areas (e.g., seismic hazard and plant response), as well as an experienced PRA representative from the Electric Power Research Institute and from industry7. The Office of Nuclear Regulatory Research (RES)/Division of Risk Analysis (DRA) staff will chair and coordinate the TAG, which will meet periodically. The TAG Chairman will be responsible for leading and moderating the TAG meetings, and will serve as the TAG spokesperson, as necessary, in briefings to NRC and project management. The TAG Coordinator, in consultation with the Level 3 PRA Project Program Manager and the TAG Chairman, will develop and disseminate the agenda for each TAG meeting. The TAG Coordinator will also be responsible for organizing and recording the minutes of the TAG meetings and maintaining an electronic repository to provide reports, publications, and other technical information as background for all TAG meetings.
Table 1 provides a template for the TAG review documentation. This template (or a similar documentation format) is to be used to document the results of the TAG reviews performed for the Level 3 PRA project.
Table 1 TAG Review Documentation Template SR Finding Recommended Resolution Implemented Resolution Reviewer:
Responsible Analyst:
Risk Source:
Hazard: [e.g., internal events]
Level: [1, 2 or 3]
Technical Element:
Date:
7 This individual was initially a staff member of NextEra Energy Resources and then became an employee of Westinghouse.
Reactor, Spent Fuel Pool, Dry Cask Storage, Integrated Risk Describe the finding, what is the issue, why it is a concern; explanation needs to clearly explain the concern and the basis for the concern.
Describe the recommendation to resolve the concern; the explanation needs to be sufficiently detailed so that the analyst understands what needs to be revised in the PRA to resolve the concern.
Analyst describes the response to the finding and recommendation, describing how it was resolved; the explanation should not be just an accept, but an explanation of exactly how it was resolved (e.g.,
how the PRA model was revised).
List the applicable supporting requirement (SR) using the standard index number; if an SR is not applicable, then use the technical element 2 to 4 digit abbreviation (xxxx) and the finding numbered sequentially (yy) with an T (i.e., xxxx-yy-T). If criteria were developed and used, then reference the criterion number (see Table 2).
Level 3 PRA Project QA Plan, Rev. 0 5
4.2 Internal Self-Assessment The objective of the internal self-assessment is to further ensure the technical acceptability of the work as the PRA model is being developed. The PRA model will be developed based on established and accepted methods, tools, and data as documented in, for example, consensus standards and guidance documents. For each technical element, a review of the work is performed using the process described below.
The full-scope site Level 3 PRA model consists of models developed by the Southern Nuclear Operating Company (SNC) for Vogtle Units 1 and 2, and those developed internally by the NRC. Parts of the Vogtle PRA model have received an industry peer review, using the ASME/ANS Level 1 PRA Standard.8 The self-assessment process will take advantage of the industry peer review. Figure 1 provides the process for self-assessment. This process involves 5 steps as discussed below.
Figure 1 Process Used for Self-Assessment Generally the self-assessment is performed by the technical element leader, responsible analyst, or may be performed by an internal NRC team. If the work is performed by a contractor, the self-assessment is performed by an NRC team (with contractor support). The purpose of using an NRC team instead of the contractor to perform the self-assessment is for the NRC to have ownership of the work; that is, to understand the details of constructing the model.
In Step 1, the self-assessment reviewer determines whether an independent industry peer review was performed. This decision will determine the scope of the self-assessment; that is, the analyst is determining whether the self-assessment can take advantage of the independent peer review performed on the Vogtle PRA. If an independent peer review was not performed, then the reviewer needs to perform a complete self-assessment (Step 4). If 8 ASME/ANS RA-Sa-2009, Standard for Level 1/Large Early Release Frequency Probabilistic Risk Assessment for Nuclear Power Plant Applications, Addendum A to RA-S-2008, ASME, New York, NY, American Nuclear Society, La Grange Park, Illinois, February 2009.
PRA model (technical task) acceptable Self-assessment complete No Yes Yes Step 1:
Step 4:
Step 2:
Step 3:
Step 5:
Revise PRA model (Technical task) accordingly No Perform a self-assessment Document the resolution of the findings and the self assessment Was an independent industry peer review performed?
Were the peer review findings adequately addressed or not significant?
Level 3 PRA Project QA Plan, Rev. 0 6
an independent peer review was performed, then the significance of the peer review results and resolutions needs to be assessed (Step 2).
In Step 2, the reviewer determines if the findings from the peer review were addressed and if they appropriately addressed the issue. If the peer review findings were adequately addressed or were not adequately addressed but determined not to be significant to the PRA, then the reviewer goes to Step 4 to perform the self-assessment. If the peer review findings are determined to not be adequately addressed and are significant to the PRA, then the reviewer needs to revise the PRA model to correct the issue (Step 3).
Significance can be determined both qualitatively and quantitatively, as follows:
Qualitative -
The finding can result in changing the basic structure of the PRA model (e.g., success criteria such that the accident sequence progression is changed, different initiating events and/or frequencies, different human events and/or frequencies, different equipment failure probabilities).
Quantitative -
Significant accident sequences are impacted. A significant sequence is one of the set of sequences, defined at the functional or systemic level that, when ranked, compose 95% of the core damage frequency (CDF) or the large early release frequency/large release frequency (LERF/LRF), or that individually contribute more than ~1% to the CDF or LERF/LRF.
Significant basic event/contributors are impacted. Significant basic events (i.e.,
equipment unavailabilities and human failure events) are those that have a Fussell-Vesely9 importance greater than 0.005 or a risk-achievement worth greater than 2.
In Step 3, the reviewer revises the PRA model to resolve the inadequacy. After the PRA is revised, the reviewer goes to Step 4 to perform the self-assessment.
In Step 4, the self-assessment is performed using the guidance in RG 1.200. As such, the self-assessment:
Uses a set of desired PRA characteristics and attributes as the basis for review Uses a minimum list of review topics to ensure coverage, consistency, and uniformity Reviews PRA methods Reviews application of methods 9 Risk Reduction Worth: The decrease in risk if a plant feature (e.g., system or component) were assumed to be optimized or were assumed to be made perfectly reliable. Depending on how the decrease in risk is measured, the risk reduction worth can either be defined as a ratio or an interval. Risk Achievement Worth: The increase in risk if a plant feature (e.g., system or component) was assumed to be failed or was assumed to be always unavailable. Depending on how the increase in risk is measured, the risk achievement worth can either be defined as a ratio or an interval. Sometimes risk achievement worth is referred to as risk increase. Fussell-Vesely: For a specified basic event, Fussell-Vesely importance is the relative contribution of a basic event to the calculated risk. This relative or fractional contribution is obtained by determining the reduction of setting the probability of the basic event to zero. Birnbaum Importance: An indication of the sensitivity of the accident sequence frequency to a particular basic event.
Level 3 PRA Project QA Plan, Rev. 0 7
Reviews assumptions and assesses their validity and appropriateness Determines if the PRA represents the as-built and as-operated plant Reviews results of each PRA technical element for reasonableness Reviews PRA maintenance and update process Reviews PRA modification attributable to use of different model, techniques, or tools Reviews against modifications to the standard, if there is a standard In evaluating the above, if a standard exists, then the requirements in the standard are used as the basis for the self-assessment in determining whether, for example, the desired attributes and characteristics provided in RG 1.200, Section 1 are met. If a PRA standard does not exist for a particular hazard or technical element, then criteria are developed to perform the self-assessment. These criteria are detailed enough to judge the technical acceptability of the work. They should be of consistent detail as in the standard for hazards or technical elements addressed by a standard. These criteria are documented using Table 2 (or a similar documentation format). Once the self-assessment (Step 4) is complete, the reviewer should go to Step 5 to document the results.
Table 2 Self-Assessment and Peer Review Criteria Where Standards Do Not Exist Criteria Criteria Source of Risk:
Hazard:
PRA Level:
Technical Element:
In Step 5, the reviewer documents the self-assessment using Tables 3 and 4 (or a similar documentation format). Table 3 can be generated using the ePSA Risk and Reliability software. This program populates some of the fields in the table automatically based on the ASME/ANS Level 1 PRA standard. For those parts of the PRA not covered by this standard, the ePSA software cannot be used, and the analyst will have to create the table using the template and the criteria developed and documented in Table 2. The purpose of Table 4 is to provide a high level summary of the conclusions of the self-assessment.
After Step 5, the initial self-assessment is complete.
The elements of the Level 1 PRA that require complete or focused review can be assessed using the guidance in RG 1.200 supported by the requirements provided in the ASME/ANS PRA Standard. For those aspects of the PRA models that do not have a final consensus standard, but do have a standard that is being developed, they will be reviewed using the high level requirements stipulated in the latest draft of the specific standards. This process In numbering the criteria, use the technical element 2-4 digit abbreviation (xxxx) and the criteria numbered sequentially (yy) with a C (i.e., xxxx-yy-C).
Level 3 PRA Project QA Plan, Rev. 0 8
will be used for the self-assessment review of the Level 2, Level 3, and low power and shutdown PRA. The PRA models for which a standard does not exist or is not being developed (i.e., dry cask storage [DCS], spent fuel pool [SFP]), elements of these models that have similar bases as compared to those of the Level 1 PRA (e.g., initiating event analysis, data analysis, human reliability analysis, accident sequence analysis, consequence analysis, source term determination, quantification/uncertainty analysis, etc.) can be reviewed using the requirements for the similar technical areas in the Level 1 through Level 3 PRA standards bearing in mind the differences in the requirements related to reactor versus those for the DCS/SFP.
Table 3 Self-Assessment Documentation Template Section Finding ID Cat II Requirement Self-Assessment Finding Comment Resolution Describe the assessment, describe what was done, whether a concern was found or not. Describe the concern, why it is a concern; explanation needs to clearly explain the concern and the basis for the concern. If no concern is found, describe the basis for why it is believed the requirement (or criterion) was met. Describe the proposed fix to resolve the concern; the explanation needs to be sufficiently detailed so that it is understood what needs to be revised in the PRA to resolve the concern.
The supporting requirement for Capability Category II from the ASME/ANS Level 1 PRA standard is generated automatically if using the ePSA software; however, if this standard does not apply, the template is used and the defined criteria are entered manually from Table 2.
As a result of the self-assessment, if a concern is found, i.e., a finding, then a Y is marked; if there is no finding, then an N is marked.
Any additional explanations that are relevant to the self-assessment are discussed.
Analyst describes the response to the finding and the proposed fix, describing how it was resolved; the explanation should not be just an accept, but an explanation of exactly how it was resolved (e.g., how the PRA model was revised).
Level 3 PRA Project QA Plan, Rev. 0 9
Table 4 Overall Results of Self-Assessment Process Criteria Conclusion Reviewer:
Responsible Analyst:
Risk Source:
Hazard:
Level:
Technical Element:
Date:
Vogtle Industry Peer Review 1
Was an independent peer review performed on the Vogtle PRA?
2 Was the scope of the peer review adequate?
3 Did the peer review meet the staff position defined in Regulatory Guide 1.200 for an acceptable peer review?
4 Were the peer review findings adequately addressed in the PRA?
General Conclusions 5
Is the identified list of information needed to accomplish the task reasonably complete?
6 Does the plant information appropriately represent the as-built and as-operated plant?
7 Was the plant information used in an acceptable manner?
8 Are the assumptions for each task identified?
9 Are the assumptions for each task adequately justified (appropriate)?
10 Do the results (both interim and final) appear reasonable given the design, operation and historical performance of the plant?
Specific Conclusions For example, the initiating event analysis for a SFP PRA uses similar techniques and processes as those used for a Level 1 reactor PRA. The high level requirements for the reactor PRA model can be used for the SFP PRA model (the specifics of SFP are presented in parenthesis) as indicated below:
HLR-IE-A -
The initiating event analysis shall provide a reasonably complete identification of initiating events.
HLR-IE-B -
The initiating event analysis shall group the initiating events so that events in the same group have similar mitigation requirements to facilitate an efficient but realistic estimation of CDF (or fuel damage frequency)
Describe the conclusion and the basis for the conclusion; may refer to self-assessment table..
Describe unique or specific conclusions, if any, and the basis for the conclusion.
Level 3 PRA Project QA Plan, Rev. 0 10 HLR-IE-C -
The initiating event analysis shall estimate the annual frequency of each initiating event or initiating event group Individual supporting requirements can be tailored for use in SFP PRA self-assessment Table 5 provides an example self-assessment process for the SFP PRA. In the absence of any standard, the technical elements of the SFP PRA defined in the TAAP are compared to the similar elements of the Level 1 reactor at-power internal events PRA discussed in the ASME/ANS Standard. Tables 5 and 6 identify both the high level requirements and the supporting requirements that are common and applicable for the self-assessment review of the SFP PRA.
Table 5 Example: Mapping of the HLRs of SFP PRA and At-Power Level 1 PRA Task At-Power Level 1 PRA Technical Elements (HLR)
IE Analysis IE Analysis Identification Grouping Analysis Identification10 Grouping Analysis Operating Cycle Discretization 11 2
Accident Sequence Analysis Accident Sequence Analysis CDF Accident Scenario Description Treatment of Dependencies Fuel Uncovery Accident Scenario Description Treatment of Dependencies 3
Systems Analysis Systems Analysis Treatment of Causes for System failure Treatment of CCF Treatment of Dependencies Treatment of Causes for System Failure Treatment of CCF Treatment of Dependencies 4
Success Criteria Structural Analysis Defining Overall SSC and Human Action Success Criteria Using Thermal/Hydraulic, Structural and other supporting Engineering Bases to Drive SC Defining Overall SSC and Human Action Success Criteria Using Thermal/Hydraulic, Structural and other supporting Engineering Bases to Drive SC Identification of FP failure modes and locations SFP Structural Integrity Analysis SSCs Structural Integrity Analysis 10 Includes hazard and low-likelihood event screening.
11 Discretizing the reactor operating cycle into a finite set of operating cycle phases (OCPs) can be considered to be akin to the plant operating states considered in a low power and shutdown PRA, with respect to the amount of decay heat that needs to be considered. This process determines the time available to respond to an accident, before fuel damage occurs.
Level 3 PRA Project QA Plan, Rev. 0 11 Table 5 Example: Mapping of the HLRs of SFP PRA and At-Power Level 1 PRA Task At-Power Level 1 PRA Technical Elements (HLR)
Data Analysis Data Analysis 6
Human Reliability Analysis Human Reliability Analysis Identifying routines of activities Screening of activities Defining HFEs Assessing HFE Probability Identifying Operator Accident Response Defining Response HFEs Assessing Response HFE Probability Modeling Recovery Actions Identifying routines of activities Screening of activities Defining HFEs Assessing HFE Probability Identifying Operator Accident Response Defining Response HFEs Assessing Response HFE Probability Modeling Recovery Actions 7
Quantification Quantification Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA Technical Element HLR Supporting Requirement Applies (Y/N)
Comment 1
IE-A IE-A1 Y
Except instead of core damage (CD) it considers fuel damage (FD)
IE-A2 Y
Except the IE categories reduce to fuel uncovery and loss of power IE-A3 Y
IE-A4 Y
IE-A5 Y
IE-A6 Y
IE-A7 Y
IE-B IE-B1 Y
IE-B2 Y
IE-B3 Y
Note: The timing and the effect on the operability and performance of operators and relevant mitigating systems is one criterion to consider. The operating cycle discretization influences this timing factor.
IE-B4 Y
IE-B5 N
IE-C IE-C1 Y
IE-C2 Y
IE-C3 Y
IE-C4 Y
IE-C5 Y
IE-C6 Y
Screening the low-frequency events IE-C7 Y
IE-C8 Y
IE-C9 Y
Level 3 PRA Project QA Plan, Rev. 0 12 Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA Technical Element HLR Supporting Requirement Applies (Y/N)
Comment IE-C10 Y
IE-C11 Y
IE-C12 Y
IE-C13 Y
IE-C14 N
2 AS-A AS-A1 Y
AS-A2 Y
Except that instead of preventing core damage, fuel damage should be considered AS-A3 Y
AS-A4 Y
AS-A5 Y
AS-A6 Y
AS-A7 Y
AS-A8 Y
Except that instead of the core damage end state, the fuel damage end state should be considered AS-A9 Y
AS-A10 Y
AS-A11 Y
AS-B AS-B1 Y
AS-B2 Y
Except for examples AS-B3 Y
AS-B4 Y
AS-B5 Y
AS-B6 Y
AS-B7 Y
Except examples (b) and (c) 3 SC-A SC-A1 N
Applies to fuel damage SC-A2 Y
Modifies the parameters and SCs to be used in determining the fuel damage SC-A3 Y
SC-A4 Y
If applicable SC-A5 Y
SC-A6 Y
SC-B SC-B1 Y
SC-B2 Y
SC-B3 Y
SC-B4 Y
Except for fuel damage SC-B5 Y
4 SY-A SY-A1 Y
SY-A2 Y
SY-A3 Y
SY-A4 Y
SY-A5 Y
Except for fuel damage SY-A6 Y
Level 3 PRA Project QA Plan, Rev. 0 13 Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA Technical Element HLR Supporting Requirement Applies (Y/N)
Comment SY-A7 Y
SY-A8 Y
SY-A9 Y
SY-A10 Y
SY-A11 Y
SY-A12 Y
SY-A13 Y
SY-A14 Y
SY-A15 Y
SY-A16 Y
SY-A17 Y
SY-A18 Y
SY-A19 Y
SY-A20 Y
SY-A21 Y
SY-A22 Y
SY-A23 Y
SY-A24 Y
SY-B SY-B1 Y
SY-B2 Y
SY-B3 Y
SY-B4 Y
SY-B5 Y
SY-B6 Y
SY-B7 Y
SY-B8 Y
SY-B9 Y
SY-B10 Y
SY-B11 Y
SY-B12 Y
SY-B13 Y
SY-B14 Y
SY-B15 Y
5 HR-A HR-A1 Y
HR-A2 Y
HR-A3 Y
HR-B HR-B1 Y
HR-B2 Y
HR-C HR-C1 Y
HR-C2 Y
HR-C3 Y
HR-D HR-D1 Y
HR-D2 Y
Level 3 PRA Project QA Plan, Rev. 0 14 Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA Technical Element HLR Supporting Requirement Applies (Y/N)
Comment HR-D3 Y
HR-D4 Y
HR-D5 Y
HR-D6 Y
HR-D7 Y
HR-E HR-E1 Y
HR-E2 Y
Except for preventing or mitigating fuel damage HR-E3 Y
HR-E4 Y
HR-F HR-F1 Y
HR-F2 Y
HR-G HR-G1 Y
HR-G2 Y
HR-G3 Y
HR-G4 Y
HR-G5 Y
HR-G6 Y
HR-G7 Y
HR-G8 Y
HR-H HR-H1 Y
HR-H2 Y
HR-H3 Y
6 DA-A DA-A1 Y
DA-A2 Y
DA-A3 Y
DA-A4 Y
DA-B DA-B1 Y
DA-B2 Y
DA-C DA-C1 Y
DA-C2 Y
DA-C3 Y
DA-C4 Y
DA-C5 Y
DA-C6 Y
DA-C7 Y
DA-C8 Y
DA-C9 Y
DA-C10 Y
DA-C11 Y
DA-C12 Y
DA-C13 Y
DA-C14 Y
DA-C15 Y
Level 3 PRA Project QA Plan, Rev. 0 15 Table 6 Applicability of SRs of the At-Power Level 1 PRA to the SFP PRA Technical Element HLR Supporting Requirement Applies (Y/N)
Comment DA-C16 Y
DA-D DA-D1 Y
DA-D2 Y
DA-D3 Y
DA-D4 Y
DA-D5 Y
DA-D6 Y
DA-D7 Y
DA-D8 Y
7 QU-A QU-A1 Y
QU-A2 Y
Except for fuel damage frequency QU-A3 Y
Except for fuel damage frequency QU-A4 Y
Except for fuel damage frequency QU-A5 Y
QU-B QU-B1 Y
QU-B2 Y
QU-B3 Y
The example applies to fuel damage frequency QU-B4 Y
QU-B5 Y
QU-B6 Y
Except for fuel damage frequency QU-B7 Y
QU-B8 Y
QU-B9 Y
QU-B10 Y
QU-C QU-C1 Y
QU-C2 Y
QU-C3 Y
QU-D QU-D1 Y
QU-D2 Y
QU-D3 Y
QU-D4 Y
QU-D5 Y
QU-D6 Y
Except for fuel damage frequency QU-D7 Y
QU-E QU-E1 Y
QU-E2 Y
QU-E3 Y
Except for fuel damage frequency QU-E4 Y
Level 3 PRA Project QA Plan, Rev. 0 16 4.3 External Peer Reviews The objective of the external peer reviews is to provide independent reviews of the technical acceptability of the developed PRA model and its results. There are two types of peer review planned which are discussed in the following sections:
Section 4.3.1 - PRA Standard Peer Review Section 4.3.2 - Independent Expert Peer Review The first peer review is similar to the peer reviews performed by industry and follows the peer review process as required by the ASME/ANS PRA standard and employs the NEI peer review guidance. The purpose of the ASME/ANS peer review is to assess the PRA to the extent necessary to determine if the methodology and its implementation meet the requirements of this standard. And... to determine strengths and weaknesses in the PRA.
The peer reviewers are industry individuals whose qualifications as acceptable peer reviewers are provided in the ASME/ANS PRA standard as endorsed in RG 1.200. 12 A major qualification includes independence from the team who developed the PRA model under review.
The second peer review is also an independent review performed by a team of experts.
Many of these reviewers are likely to come from academia and national laboratories.
4.3.1 PRA Standard Peer Review ASME/ANS have developed PRA standards which provide the necessary technical requirements for what constitute a technically acceptable PRA based on state-of-the practice methods. One objective of the Level 3 PRA project is to develop a PRA based on current state-of-the-practice methods.
To the extent practical, the PRA standard peer reviews will be conducted for all major parts of the Level 3 PRA project at various points throughout the performance of the study. This approach will allow peer review findings to be addressed in a timely manner. It will, as opposed to performing one large, comprehensive external peer review at the end of the project, minimize the extent of potential re-work.
Where PRA standards (either final or draft for trial use) are available, they will provide the basis for the peer review. If a standard is in draft for trial use stage, the peer review part of the standard will be reviewed and additional guidance will be developed, if needed, to make it acceptable to the staff. If a PRA standard does not exist (e.g., spent fuel pool), review criteria will be developed to support the peer review of the PRA scope item.
The reviews will be performed consistent with the process described in RG 1.200 and supplemented with other related guidance. The peer review teams will be comprised of individuals who are independent from the project. It is envisioned that the standard peer reviews will be performed by industry (e.g., the Pressurized Water Reactor (PWR) Owners Group (OG) and consultants), supplemented by NRC staff (e.g., Regional senior reactor analysts (SRAs)). In determining whether the technical requirements in the standard have been met, the level of detail of the PRA model review goes beyond the technical bases, tools, methods, models, assumptions and data for the project, as well as interpretation of the study 12Regulatory Guide 1.200, Rev. 2, An Approach for Determining the Technical Adequacy of Probabilistic Risk Assessment Results for Risk-Informed Activities, March 2009.
Level 3 PRA Project QA Plan, Rev. 0 17 results. It also involves reviewing how the various models (e.g., accident sequence development, systems analyses) were constructed. In this regard, actual Vogtle-specific information is needed. The peer reviewers are required to sign a non-disclosure agreement since this information is proprietary.
The scope of the peer review will be documented prior to each peer review and provided to the peer review team. Table 7 provides a suggested format for documenting the peer review findings (it is the same as the TAG review documentation template previously provided in Table 1).
It is expected that the peer review team will generate a peer review report. This report will describe the process, team members (and their qualifications), and basis for review findings.
It is further expected that the Level 3 PRA project task leader will review the peer review findings and document how each finding will be resolved. The results of the standard peer reviews will be provided to the Level 3 Program Manager and to the Document Controller.
Table 7 External Peer Review Documentation Template SR Finding Recommended Resolution Implemented Resolution Reviewer:
Responsible Analyst Risk Source:
Level : [1,2,3]
Hazard: [e.g., internal events]
Technical Element:
Date:
o High Significance -- the issue needs resolution to ensure the technical adequacy of the PRA, the capability of the PRA, or the robustness of the PRA update process.
o Medium Significance -- The issue needs resolution to maintain maximum flexibility in PRA applications and consistency with Industry practices (as endorsed by the NRC) or simply to enhance the PRAs technical capability as time and resources permit. It is unlikely that the technical adequacy of the PRA is impacted.
o Low Significance -- The issue that does not impact the technical adequacy of the PRA.
Describe the finding, what is the issue, why it is a concern; explanation needs to clearly explain the concern and the basis for the concern.
Describe the recommendation to resolve the concern; the explanation needs to be sufficiently detailed so that the analyst understands what needs to be revised in the PRA to resolve the concern.
Analyst describes the response to the finding and recommendation, describing how it was resolved; the explanation should not be just an accept, but an explanation of exactly how it was resolved (e.g., how the PRA model was revised).
The level of significance of the concern should be listed including the basis for level of significance assessed; see below for explanation of significance.
List the applicable supporting requirement (SR) using the standard index number; if an SR is not applicable, then use the technical element 2-4 digit abbreviation (xxxx) and the finding numbered sequentially (yy) with an P (i.e., xxxx-yy-P). If criteria were developed and used, then reference the criterion number (see Table 2).
Reactor, Spent Fuel Pool, Dry Cask Storage, Integrated Risk
Level 3 PRA Project QA Plan, Rev. 0 18 4.3.2 Independent Expert Peer Review The purpose of the Independent Expert Peer Review (IEPR) is not to determine if the ASME/ANS PRA standard requirements were met, but to perform a high level peer review comprised of known national or international PRA experts primarily from academia and national laboratories. The main objective is to determine strengths and weaknesses in the PRA. In this regard, the IEPR involves reviewing the technical bases, tools, methods, models, assumptions and data for the project, as well as interpretation of the study results. It does not involve reviewing how the various models (e.g., accident sequence development, systems analyses) were constructed This IEPR is intended to be performed at the end of the project.
The scope of the peer review will be documented prior to each peer review and provided to the IEPR team. The documentation of the IEPR will include the following:
Identification of the reviewer, the part of the Level 3 PRA project reviewed (which radiological sources, PRA Level(s), operating state(s), and hazard(s)).
Description of the findings, what is the issue, why it is a concern (i.e., the basis for the concern).
Identification of the level of significance of the issue and the basis for the significance.
The significance will be identified as:
o High Significance -- the issue needs resolution to ensure the technical adequacy of the PRA, the capability of the PRA, or the robustness of the PRA update process.
o Medium Significance -- The issue needs resolution to maintain maximum flexibility in PRA applications and consistency with Industry practices (as endorsed by the NRC) or simply to enhance the PRAs technical capability as time and resources permit. It is unlikely that the technical adequacy of the PRA is impacted.
o Low Significance -- The issue that does not impact the technical adequacy of the PRA.
Description of the proposed recommendation to resolve the concern; the explanation needs to be sufficiently detailed so that the analyst understands what needs to be revised in the PRA to resolve the concern.
It is expected that the IEPR team will generate a peer review report. This report will describe the process, team members (and their qualifications), and basis for review findings. It is further expected that the Level 3 PRA project task leader will review the IEPR findings and document how each finding will be resolved; the explanation should not be just an accept, but an explanation of exactly how it was resolved (e.g., how the PRA model was revised).
The results of the IEPR will be provided to the Level 3 Program Manager and to the Document Controller.
Level 3 PRA Project QA Plan, Rev. 0 19 4.4 Advisory Committee on Reactor Safeguards The objective of the Advisory Committee on Reactor Safeguards (ACRS) review for the Level 3 PRA project is to: (1) monitor progress in the development of the Level 3 PRA and (2) provide insight, advice, and guidance on the technical bases, tools, methods, models, assumptions and data for the project, as well as on interpretation of the study results.
The ACRS Reliability and PRA Subcommittee will be briefed approximately twice a year to obtain their feedback on the technical approaches and assumptions employed in the Level 3 PRA project.
4.5 Public Review and Comment As part of the documentation, a final summary of the results of the Level 3 PRA project will be published. This report will provide the various results of the study, and will also summarize the various tools, methods, models, assumptions and data used. This summary report (or reports) will be published for public review and comment.
A public meeting will be held to brief the public on the report(s) and answer questions. A second meeting will be held to provide responses to the public comments.
Each team leader is responsible for addressing the public comments associated with their part of the study.
5 DOCUMENTATION CONTROL Documentation control is a key factor in any analysis to ensure and demonstrate the technical acceptability of the analysis. For each technical task, the method, tools, data and other information being used will be documented along with the basis for their acceptability (e.g., NRC endorsement). The documentation for each technical task is identified in the TAAP, and the document control process for this project is described in this section.
As mentioned above, the information to be documented includes the following:
Methods Tools Data Other information - this includes the various information (other than methods, tools and data) used to develop the PRA model; for example:
plant design information reflecting the normal and emergency configurations of the plant plant operational information with regard to plant procedures and practices plant history (plant, system, and component performance) plant test and maintenance procedures and practices engineering aspects of the plant design Analytical work Results Given the large amount of information of various types required to construct and report the results of the Level 3 PRA project, an appropriate medium is needed to store and access this
Level 3 PRA Project QA Plan, Rev. 0 20 information. This medium has to have the ability for the project analysts to store, retrieve, edit, and control the information. SharePoint has been selected to be the medium, and the primary repository for Level 3 PRA project information will be referred to as the Level 3 PRA SharePoint site.
The Level 3 PRA project Documentation Coordinator will primarily be responsible for document control. The Documentation Coordinator will be in charge of the various tasks needed to ensure the SharePoint site runs smoothly and remains organized, and will be responsible for receiving information from the licensee, processing it, and ensuring that the information gets to contractors and the SharePoint site in a reasonable timeframe, as well as ensuring that vital information is routinely backed up.
Documentation control for this project involves the following major elements, each of which is described in a separate section below:
Section 5.1 - Storage and access of project information Section 5.2 - Upload of information onto the SharePoint site Section 5.3 - Documentation control of licensee information Section 5.4 - Documentation backup Section 5.5 - Use of external storage media Section 5.6 - Working document folders Section 5.7 - Use of templates and forms for documentation Section 5.8 - Site Visits Section 5.9 - Documentation control for NRC Contractors Section 5.10 - Non-disclosure agreement to allow access to proprietary information Section 5.11 - Project documentation markings Section 5.12 - Guidance for addressing potential technical issues Section 5.13 - Future plant modifications Section 5.14 - Organization of the various types of information on the SharePoint site 5.1 Storage and Access of Project Information As mentioned above, SharePoint has been selected as the medium to store and access the Level 3 PRA project information. SharePoint has the necessary flexibility to organize and store the information in a manner consistent with the needs of the project. It also allows for dynamic changes to the organization and site as new needs arise over the course of the project.
Moreover, controls can be used to limit access to the information; for example, who is allowed to access the information and who is allowed to edit documents. These controls will help ensure that files are not accidentally deleted or edited without the authors approval.
SharePoint also has an established backup procedure that ensures data integrity. Therefore, SharePoint provides a mechanism to ensure that information will not be lost or corrupted.
The information stored on the SharePoint site is only accessible by the project team members who have access to the NRCs local area network.
Level 3 PRA Project QA Plan, Rev. 0 21 5.2 Upload of Information onto the SharePoint Site As the work progresses, the project team members will occasionally need to place files onto the Level 3 PRA Project SharePoint site. These files will include information that only the individual analyst will need to access, or that needs to be shared with other members of the task team or with the entire project team. Moreover, there may need to be restrictions, for example, on who has permission to edit these files.
Although most team members may not edit or modify most of the files stored on the SharePoint site, any project team member has permission to upload files into the temporary storage location titled, Inbox. Once a file is uploaded into the Inbox, the Documentation Coordinator will move the file from the Inbox to its proper read-only location. In order to upload files, there is a link on the right hand side of the front page that is titled, Inbox:
Upload documents to the L3PRA website. This page can also be found by clicking:
http://portal.nrc.gov/edo/res/dra/L3PRA/Inbox_Library/Forms/AllItems.aspx Once on the Inbox page, the upload button is clicked and the analyst chooses the files to be placed on the site. In uploading each file, a brief description of the file and the last edited date is included in the Notes section. The restrictions on who has access, edit capability, etc., can be found in Table 12 for the different types of information.
5.3 Documentation Control of Licensee Information The information received from the licensee will also be stored on the SharePoint site. The information on the SharePoint site will be read-only, with the exception of the personal working files (discussed in Section 5.6). This administrative control will prevent inadvertent changes to information obtained from the licensee. All information received from the licensee will also be maintained on read only CD-ROMs or DVDs so that, in the event of an inadvertent change on SharePoint, the original data can be restored. Moreover, there is information received from the licensee which is proprietary and not available to the public, and therefore needs to be protected. When information is received from the licensee in support of this project, a proprietary determination is conducted for each submittal.13 Once this proprietary determination is conducted and approved by the Office of the General Counsel, the information is placed on the SharePoint site for all NRC Level 3 PRA Project Team members. The specific SharePoint folder that contains this information is clearly marked as Proprietary. If this information is needed by a contractor to perform their work, the information is then copied onto an encrypted external media device (usually a CD-ROM, marked as Proprietary, if applicable) and sent to the contractor along with a notice, if applicable, that the CD-ROM contains proprietary information and should be handled appropriately.
In addition, the licensee may occasionally send updated information, or may resend the same information. These occurrences may cause confusion as to which version of the information is the most current. It is, therefore, essential that the information be administratively controlled such that different information is not being used in developing the model by different analysts. The use of SharePoint for file hosting will greatly simplify this process. The 13 RES Office Instruction ADM-003, Revision 1, Procedures for Handling Request to Withhold Proprietary Information, May 11, 2012, ADAMS Accession Number ML12132A139 (not publicly available).
Level 3 PRA Project QA Plan, Rev. 0 22 Documentation Coordinator will ensure that the data on the SharePoint site is the most current, up-to-date information that the NRC has received from the licensee, and will notify the entire project team when new information from the licensee is added to the SharePoint site. This notification will identify what information is being added and whether it updates any information currently existing on the site.
5.4 Documentation Backup Using SharePoint to store and access all the information connected with the Level 3 PRA Project will ensure a high level of data integrity. The files on SharePoint are backed-up several times a week and copies are maintained both onsite and offsite. If SharePoint is corrupted, this process ensures that there will be minimal loss of information, and progress of the project can continue given an extreme event. In addition to this automatic NRC backup of the information, once a week the Documentation Coordinator will copy all of the information on the Level 3 PRA SharePoint site onto an external media device. This backup of the files will be stored onsite for rapid recovery of files. Information that is not able to be placed on the SharePoint site will also be backed up and maintained.
5.5 Use of External Storage Media There may be types of information that are not permitted to be uploaded onto the NRCs SharePoint Site. This type of information generally involves large files and executable files (e.g., Access Database files and files that end in.exe). Therefore, an external media storage device that has been approved for use on NRC equipment will be available, on request, for project team members to back up these files. This external media device will be stored and maintained by the Document Coordinator.
In addition, some Level 3 PRA team members may develop work products that will not be able to be backed up onto the working documents section of SharePoint, described in Section 5.6. An example of this type of work is the MELCOR calculations being completed on high performance computers. The personnel working on these types of files will be given a separate external media device that will allow them to regularly back up their work.
5.6 Working Document Folders For this project, there is a tremendous amount of information that is part of the technical work performed (e.g., code calculations) that is essential to retain. This information is critical in being able to understand how the PRA model was ultimately constructed. To ensure that this information is not lost, each analyst will store their work on the SharePoint site. The site will have a section with a separate folder assigned to each major technical area of the study.
These working document folders will be viewable by all members of the project team; however, write access will only be available to the cognizant task leader. At their request, task leaders can request the Documentation Coordinator to provide write access for their folder to other ream members (e.g., if multiple team members are collaborating on the development of a document or file).
Each analyst of the Level 3 PRA project will store their working files and other important information relevant to the project in the associated working document folder on the SharePoint site instead of their personal computer or some other location. Given the back-up features in place for the Level 3 PRA project information on the SharePoint site, this will
Level 3 PRA Project QA Plan, Rev. 0 23 ensure that all the necessary information being used in the project is properly saved and stored.
5.7 Use of Templates and Forms for Documentation As the work is being performed and decisions are being made in constructing the PRA, it is important to document this information. To ensure the needed amount of information is documented and that it is documented consistently among the analysts, documentation templates/forms have been created. These templates and forms (or similar documentation formats), which will be stored on the SharePoint site, address the following information:
Results and resolution of reviews (i.e., TAG, self-assessment, and external peer reviews) - see Tables 1, 3, 4, and 7 Criteria used for self-assessment (where no standard exists) - see Table 2 Results of meetings: TAG, internal discussions, SNC, briefings, ACRS - see Table 9 Working files - see Tables 9 and 10 Technical issues and their resolution - see Table 1 During meetings, discussions, and briefings, there can be significant decisions made with regard to the PRA. It is essential to document this information. Table 9 provides a template for documenting meetings and discussions. In many instances, there may be issues that are identified and need to be addressed. These issues will be documented via the process described in Section 5.10.
In performing the work to develop the PRA model, various information, assumptions, etc., are used at different stages (e.g., for the different technical elements). It is essential to document this information. Table 10 provides a template for documenting this information, using initiating event analysis as an example.
Table 8 Documentation of Meetings and Discussions DATE:
TOPIC:
SUMMARY
OF MEETING/DISCUSSION:
A high level summary of the major points.
Level 3 PRA Project QA Plan, Rev. 0 24 Table 8 Documentation of Meetings and Discussions CONCLUSIONS Num Decision Basis for Decision ACTION ITEMS Num Item Assignee Due Status Table 9 Documentation for Level 1 Internal Events Initiating Event Analysis Sources of Information (Inputs)
Source Description Data Item Value Distribution Description Assumptions Describe the source of information (inputs) used in the technical elements, the actual input may be attached; inputs from other tasks should also be included.
List each event that has a parameter value, provide its value and uncertainty interval and describe the basis for both; this may be an attachment to the table.
List and describe each decision made during the meeting/discussions and the bases for the decision; include in the discussion on the decision where and how the PRA model is impacted; can be a high level discussion (e.g., revised Level 1 internal events success criteria).
Describe each action item identified during the meeting/discussion, who is assigned the action item, the due date of the action item, and the status of the action item, including the date for the reported status. When completed, not complete with the completion date.
Describe each assumption, give the basis fro the assumption, and describe how the PRA model would be impacted (e.g., new initiating event, revised success criteria)
Level 3 PRA Project QA Plan, Rev. 0 25 Table 9 Documentation for Level 1 Internal Events Initiating Event Analysis Note:
An assumption is a decision or judgment that is made in the development of the PRA model. An assumption is either related to a source of model uncertainty or is related to scope or level of detail.
An assumption related to a model uncertainty is made with the knowledge that a different reasonable alternative assumption exists. A reasonable alternative assumption is one that has broad acceptance within the technical community and for which the technical basis for consideration is at least as sound as that of the assumption being made. An assumption related to scope or level of detail is one that is made for modeling convenience.
Sources of Model Uncertainty Source Characterization Note:
A source of model uncertainty is one that is related to an issue for which there is no consensus approach or model (e.g., choice of data source, success criteria, reactor coolant pump seal loss-of-coolant accident model, human reliability model) and where the choice of approach or model is known to have an impact on the PRA results in terms of introducing new accident sequences, changing the relative importance of sequences, or significantly affecting the overall CDF, LERF, or LRF estimates that might have an impact on the use of the PRA in decision-making.
Documentation Criteria Criteria Documentation Description DOCUMENT the processes used to select, group, and screen the initiating events and to model and quantify the initiating event frequencies, including the inputs, methods, and results. This documentation includes the functional categories considered and the specific initiating events included in each the systematic search for plant-unique and plant-specific support system initiators the systematic search for RCS pressure boundary failures and interfacing system LOCAs the approach for assessing completeness and consistency of initiating events with plant-specific experience, industry experience, other comparable PRAs and FSAR initiating events the basis for screening out initiating events the basis for grouping and subsuming initiating events The final grouping of initiators for which accident sequence development will be performed the dismissal of any observed initiating events, including any credit for recovery the derivation of the initiating event frequencies and the recoveries used List each source of model uncertainty, describe the source, and describe how the PRA model would be impacted (e.g.,
new initiating event, different grouping).
Provide a brief discussion of how the criteria were met; can reference another document that provide the necessary information.
Level 3 PRA Project QA Plan, Rev. 0 26 Table 9 Documentation for Level 1 Internal Events Initiating Event Analysis the approach to quantification of each initiating event frequency the frequencies quantified for initiating event group the justification for exclusion of any data Other Documentation Criteria 5.8 Site Visits During the course of developing the Level 3 PRA model, it will be necessary for cognizant staff members to visit either SNC headquarters, the Vogtle plant site, or the surrounding Vogtle plant site area. The purpose of these visits is to (1) gather additional information not obtainable via documentation, and/or (2) confirm understanding of information provided.
A site visit generally involves:
Discussions with various on-site personnel (e.g., engineering, operations, maintenance) and off-site personnel (e.g., local law enforcement regarding evacuation)
Walk-down of the site and/or the surrounding evacuation area To ensure that the purpose of the visit is achieved, the team leads participating in the site visit will prepare a site visit plan prior to the visit. This plan will be forwarded to SNC (or other appropriate organization) so that the licensee (or other organization) is prepared for the visit.
The site plan will include the following:
Dates of visit Names of NRC staff and contractors attending, including their role and responsibility in the Level 3 PRA project SNC or other organization personnel to be interviewed The places at the site (or surrounding area) to be visited List of questions and issues to be discussed List any unique documentation requirements.
Level 3 PRA Project QA Plan, Rev. 0 27 It is equally important to document the results of the site visit. This documentation will include the following:
Dates of the visit Names of NRC staff and contractors on the site visit Names of SNC and other organization personnel (including their position) interviewed Specific questions and issues discussed along with a summary of the discussion Site areas visited with specific observations Summary of discussions; should identify the specific topic and details of the discussion General observations and conclusions made as a result of the visit If the intent of the visit is to access the actual Vogtle site, it is preferred that the NRC staff have unescorted access so as not to be a burden to SNC. To obtain unescorted access, the following must be performed:
Each NRC staff member on the site visit must have completed NRC site access training (i.e., H-100 (NRC Site Access Training) or H-101 (NRC online Site Access Refresher Training), as appropriate) within the last 12 months Region II must be notified. This notification will be performed by the NRC Level 3 PRA project manager, and will include the following information for each traveler:
Name (as it appears on NRC badge)
NRC badge number Clearance level (L, L(h), Q, or NC)
Site access training Completion date of training Type of training (H-100, H-101, or non-NRC training at a specified power plant)
Nuclear power plant/site to be visited Date(s) of visit The Region will notify the security department at the Vogtle site, by letter, of the upcoming visit. The letter will inform plant security that the NRC staff have the necessary access training and to provide them with a badge allowing unescorted access.
It is expected that the NRC contractors will be escorted (by NRC staff). However, Region II should still be notified of their participation in the visit, so that they are included in the letter
Level 3 PRA Project QA Plan, Rev. 0 28 that the Region sends to plant security. This will facilitate the badging process. It is also expected that all contractors will complete the NRC site access training so that they do not have to undergo such training at the site. The information to be provided to the Region for each contractor includes:
Name (as it appears on drivers license)
Company Site access training Completion date of training Type of training (H-100, H-101, or non-NRC training at a specified power plant)
Nuclear power plant/site to be visited Date(s) of visit 5.9 Documentation Control for NRC Contractors This project will involve a substantial amount of work developed by NRC contractors. For example, the SPAR models and SAPHIRE program were developed and are hosted by INL for the NRC under previous contracts. Under the Level 3 PRA contract, INL will also host the models for this project. It is expected that the NRC Contractors working on this project will have their own internal information and document control system. It is the Contracting Officers Representatives (CORs) responsibility to ensure that the contractor has an adequate plan to store and backup their work. The COR should document this finding using the review template.
When a document comes to the NRC from a contractor, it will be sent to the COR and technical lead. The technical lead will decide whether the information should be stored only on the SharePoint site, or also in ADAMS. In making this determination, the technical lead will need to consider the following factors:
Status of the information (e.g., draft, mark-up, final product)
Whether the document is a deliverable specified in the contract Likelihood that the information will ultimately be contained, in whole or in part, in another stored document As general guidance, final products and other contract deliverables should be stored in ADAMS (as well as on the SharePoint site). Most other information will just be stored on the SharePoint site. Information will be stored on the SharePoint site using the procedure outlined in Section 5.2. The technical analyst will make the determination whether the information should be stored in their working document folder in SharePoint or in some other SharePoint location (if the latter, this should be coordinated with the Documentation Coordinator). Generally, contractor information that is final and is being used as reference material should be stored in, for example, a SharePoint location for the associated technical element. Contractor information that is not final should be stored in the technical analysts
Level 3 PRA Project QA Plan, Rev. 0 29 associated working document folder. Additional information on the review and acceptance of contractor technical reports is provided in Section 6.2.
5.10 Non-Disclosure Agreement to Allow Access to Proprietary Information To support the Level 3 PRA project, the NRC has collected a substantial amount of proprietary information about the Vogtle plant and its PRA. To ensure that the staff does not violate the conditions under which the licensee has provided this information, each project team member receives the following electronic message which they must acknowledge before being granted access to the proprietary information area of the Level 3 PRA SharePoint site:
The proprietary information submitted by SNC for Vogtle Units 1 and 2 was provided to the NRC on a voluntary basis and can only be used to support the Level 3 PRA project. In no circumstances can this information be used to support a regulatory decision (including, but not limited to, inspection activities and license reviews). Furthermore, this information shall not be redistributed beyond the Level 3 PRA project team. Please acknowledge your understanding of this information by clicking the vote button above.
5.11 Project Documentation Markings All documents generated as part of this project (either by staff or contractors) that contain licensee-provided proprietary information should have each page marked with a header and footer that states "Official Use Only - Proprietary Information."
In addition, all documents (by either staff or contractors) that contain licensee-provided proprietary information and that are placed in ADAMS, should include the following disclaimer on the cover page:
This document contains proprietary information voluntarily supplied by Southern Nuclear Operating Company to support the Level 3 PRA Project. Per NRR Office Instruction LIC-204, Revision 3 (January 2007), and RES Office Instruction ADM-003, Revision 1 (May 2012), this information should not be used to support an NRC review and approval of a licensee application or a document, or for any other NRC decision.
It should be further noted that the proprietary information submitted by SNC for Vogtle Units 1 and 2 was provided to the NRC on a voluntary basis and can only be used to support the Level 3 PRA project. In no circumstances will this information be used to support a regulatory decision (including, but not limited to, inspection activities and license reviews). Aside from submitting documents into ADAMS with the disclaimer above, documents containing SNC proprietary information should not be distributed beyond the Level 3 PRA project team.
Level 3 PRA Project QA Plan, Rev. 0 30 5.12 Guidance for Addressing Potential Technical Issues In developing the Level 3 PRA model, technical issues will arise that may impact the PRA results or insights. These issues can include:
potential issues that may call into question the technical rigor or adequacy of the SNC Vogtle PRA14 (e.g., potential model errors or deficiencies that may require changes to the model) or related quality control activities (e.g., self-assessment or peer review) issues that require a decision by the Level 3 PRA Project Management Team or discussion with the Level 3 PRA TAG or other experts (e.g., selection of significant assumptions or a choice between different analytical methods, models, or approaches); further technical analysis beyond that described in the TAAP; and/or coordination across technical areas.
An important consideration is that these issues are likely to involve proprietary PRA and plant information submitted by SNC that must be protected from public disclosure or misuse. SNC has voluntarily submitted substantial amounts of proprietary PRA and plant information to the NRC in support of the Level 3 PRA project and, for the reasons detailed below, this information is not to be used to support regulatory decisionmaking:
Under the requirements specified in 10 CFR 2.390, Public inspections, exemptions, requests for withholding, proprietary information submitted will be withheld from public disclosure if it is of a type normally held in confidence by SNC. All proprietary information submitted by SNC is reviewed and controlled as described in Section 5.3.
Non-proprietary versions of these documents, which would normally be submitted to support a license amendment or regulatory use, will not be developed to support this research project.
Information submitted by SNC for this project does not support any regulatory decision and is not required to be done under oath and affirmation or docketed, as would normally be done for a licensing submittal (e.g., see 10 CFR 50.30).
This information is not being submitted either to support a licensing application or by the Commission's regulations, orders, or license conditions, and consequently the requirements of 10 CFR 50.9, Completeness and accuracy of information, do not apply.
Consequently, it is important to have a process for addressing potential issues that also ensures that appropriate separation between the Level 3 PRA project and regulatory decisionmaking is maintained. For this project, a process has been developed for resolving technical issues, communicating technical concerns to SNC staff, and turning issue follow-up over to the appropriate regulatory process when appropriate.
For the purposes of this process, the following terms are used:
14 The Level 3 PRA Project is associated with only Units 1 and 2 at the Vogtle site. Units 3 and 4 (currently under construction) are outside of the scope of this project.
Level 3 PRA Project QA Plan, Rev. 0 31 Level 3 PRA Project Management Team - In this context, refers to the Level 3 PRA project Program Manager, Principle Technical Advisor, and RES/DRA/PRAB Branch Chief.
Cognizant staff - project team members that include, at a minimum, the technical lead, but may also include other technical analysts on the project team that are involved with identification or resolution of the issue.
The following process is used to ensure that issues identified in the performance of the Level 3 PRA project that have the potential to impact regulatory decisionmaking are handled in an appropriate manner.
- 1.
When a Level 3 PRA project staff member or contractor identifies an issue (or potential issue), the cognizant staff will assess what impact the issue could have on the PRA (i.e., the significance of the issue) and whether the issue could call into question the technical rigor or adequacy of the SNC Vogtle PRA or related quality control activities. The cognizant staff will then summarize the issue and its potential impact on the PRA in a document (see Table 11). This documentation shall be forwarded to the Level 3 PRA Project Management Team as soon as practical after the issue is identified, at which point the issue will be added to the Level 3 PRA project issue tracking spreadsheet.
General guidance for determining whether an issue should be documented and tracked includes:
Issue may call into question the technical rigor or adequacy of the SNC Vogtle PRA Issue involves a choice between different analytical methods, models, or approaches Issue requires additional work beyond that described in the TAAP Issue requires coordination across technical areas (e.g., an unresolved technical issue that has the potential to materially impact modeling decisions made in two or more technical areas)
Issue warrants communication to the entire Level 3 Project team for awareness Any other issue a project team member determines should be included or would be of interest to the Level 3 PRA Project Management Team
- 2.
For those issues that potentially question the technical adequacy of the SNC Vogtle PRA, the Program Manager will coordinate a meeting or discussion with SNC to address the identified issue. The meeting or discussion will include SNC staff (as identified by SNC), the Level 3 PRA Project Management Team, and the cognizant staff. The results of the meeting or discussion will be documented in accordance with project procedures (see Table 9 for documenting discussions/meetings). To facilitate the discussion, the Program Manager may forward the summary description of the issue (in its entirety or in part) to SNC prior to the meeting or discussion. In accordance with project communication protocols, the meeting/discussion will be coordinated with the NRR/DORL Project Manager and the SNC Licensing Director.
Level 3 PRA Project QA Plan, Rev. 0 32
- a.
Following the discussion with SNC (and after reviewing any additional information identified by SNC), the Level 3 PRA management team and cognizant project staff may determine that the issue is adequately resolved because, for example:
- i.
SNC provided additional information or clarification to resolve the issue.
ii.
SNC and the NRC used different methods or approaches, both of which are acceptable.
iii.
The issue was determined to not have a significant impact on the PRA results or insights.
If the issue has not been resolved, it will be evaluated by the Level 3 PRA Project Management Team to determine if a technical inadequacy (i.e., error) of the SNC Vogtle PRA actually exists. It will be assumed that any technical inadequacy issue that is not resolved by the cognizant staff will be considered to be an error in the SNC Vogtle PRA unless the Level 3 PRA Project Management Team determines that the issue has been resolved.
- b.
Once an error of the SNC Vogtle PRA has been identified, appropriate SNC staff will be contacted (in accordance with project communication protocols and in coordination with the Office of Nuclear Reactor Regulation (NRR)/
Division of Reactor Licensing (DORL) Project Manager) and informed of the details of the error, including the potential for the error to impact PRA results and/or insights or call into question the adequacy of quality control activities.
SNC will be requested to conduct a review of the error and inform the NRR/DORL Project Manager of the result of this review. This review will include consideration of any licensing and/or regulatory applications of the PRA. The Level 3 PRA cognizant staff will prepare a written summary of the notification of SNC staff of the error which the RES/DRA/PRAB Branch Chief will forward to the NRR/DORL Vogtle Project Manager and appropriate NRR/DORL Branch Chief (either by formal memo or email notification).
Once the error has been communicated to SNC and the NRR/DORL Project Manager, the Level 3 PRA project team is not responsible for any further follow-up on the potential regulatory implications of the error.
- c.
Once the error has been turned over to NRR/DORL, it is recognized that the Level 3 PRA project team may proceed with an appropriate technical resolution consistent with the overall project objectives. The error will continue to be documented and tracked using Table 11 as the error is resolved within the context of the Level 3 PRA project.
- 3.
For those issues that require a decision, further technical analysis, and/or discussion beyond the cognizant staff, the cognizant staff member who has the lead for the issue will set up a project team meeting to discuss the issue. This meeting should include all cognizant staff and the Level 3 PRA Project Management Team. For those issues requiring further technical analysis, the cognizant staff, in consultation with the Level 3 PRA Project Management Team, will determine what technical analysis will be performed to resolve the issue. In determining what analysis to perform, consideration will be given to the potential impact that the issue may have on the PRA
Level 3 PRA Project QA Plan, Rev. 0 33 results or insights, the resources required for the additional analysis, and the availability of requisite staff.
The results of the meeting or discussion will be documented in accordance with project procedures (see Table 9 for documenting discussions/meetings). If the cognizant staff and Level 3 PRA Project Management Team cannot resolve the issue during the meeting, then one or more of the following actions will be taken:
- a.
The cognizant staff member who has the lead for the issue will organize a meeting with other knowledgeable staff or contractors.
- b.
The Level 3 PRA Program Manager will communicate to the TAG coordinator that the project team wishes to discuss the issue with the TAG.
- c.
The Level 3 PRA Program Manager will coordinate a meeting or discussion with SNC to get more information related to the issue, as needed. The meeting or discussion will include SNC staff (as identified by SNC), the Level 3 PRA Project Management Team, and the cognizant staff. To facilitate the discussion, the Program Manager may forward the summary description of the issue (in its entirety or in part) to SNC prior to the meeting or discussion. In accordance with project communication protocols, the meeting/discussion will be coordinated with the NRR/DORL Project Manager and the SNC Licensing Director.
For all of the above actions, the results of any meetings or discussions will be documented in accordance with project procedures (see Table 9 for documenting discussions/meetings), and the issue tracking spreadsheet will be updated accordingly. Also, as part of the resolution of the issue, it is possible that a potential error or deficiency may be identified in the SNC Vogtle PRA or related quality control activities. If so, this issue will be addressed as discussed in Step 2, above.
- 4.
The different types of issues discussed above are to be tracked using Table 11 (or similar format). This process involves the following:
Once the cognizant staff has entered the issue into Table 11, the table is forwarded to the Documentation Coordinator.
When there is any new information related to the issue, that information is forwarded to the Documentation Coordinator.
The Documentation Coordinator will update and maintain the master list of all the issues, which will reside on the Level 3 PRA project SharePoint site.
Level 3 PRA Project QA Plan, Rev. 0 34 Table 10 Issue Documentation Template Technical Analyst ___________________
Status High Significance -- An issue needing resolution to ensure the technical adequacy of the PRA, the capability of the PRA, or the robustness of the PRA update process.
Medium Significance -- An issue whose resolution is needed to maintain maximum flexibility in PRA applications and consistency with Industry practices (as endorsed by the NRC) or simply to enhance the PRAs technical capability as time and resources permit. It is unlikely that the technical adequacy of the PRA is impacted.
Low Significance -- An issue that does not impact the technical adequacy of the PRA.
Implemented Resolution Recommended Resolution Level of Significance Issue Num Describe the concern, what is the issue, why it is a concern; explanation needs to clearly explain the concern and the basis for the concern. If the issue is associated with a specific SR, then note and.
Describe the sources of information (inputs) used in the technical element, the actual input may be attached; inputs from other tasks should also be included.
Describe the recommendation to resolve the concern; the explanation needs to be sufficiently detailed so that the analyst understands what needs to be revised in the PRA to resolve the concern, and the basis for the recommended.A high A A high level summary of the major points discussed Describe the response to the issue and recommendation, describing how it was resolved; the explanation should not be just an accept, but an explanation of exactly how it was resolved (e.g., how the PRA model was revised), and the basis for the Describe each assumption, give the basis for the assumption, and describe how the PRA model would be impacted (e.g., new initiating event, different grouping).
The level of significance of the concern should be listed including the basis for level of significance assessed; see below for explanation Provide a brief discussion of how the criteria were met; can reference another document that provides the evidence.
Use the technical element 2-4 digit abbreviation (xxxx) and the issue numbered sequentially (yy) followed by an I (i.e.,
xxxx-yy-I). Note, for issues that may cut across multiple technical elements, use the 4 digit abbreviation CCCC.
Describe the status, what has been accomplished, what still needs to be done, and the date of the status (i.e., the status will continually be updated, each time it is updated, the date needs to be changed accordingly).
Once the issue has been resolved (i.e., the PRA model has incorporated the resolution),
the status should state complete and the date completed. Part of the status is whether the issue has been reviewed by the TAG (the review by the TAG is documented elsewhere).List and Describe each action item identified during the meeting/discussion, who is assigned the action item, the due date of the action item, and the status of the action item, including the date for the reported status. When completed, note complete with the completion date
Level 3 PRA Project QA Plan, Rev. 0 35 5.13 Future Plant Modifications One objective of the Level 3 PRA project model is to ensure it reflects the as-built, as-operated plant. However, the Level 3 PRA project will take several years to complete, and the plant design and operation are likely to change over time. Therefore, the potential exists that the Level 3 PRA project model may not reflect the as-built, as-operated plant at the time of project completion. Consequently, criteria are needed to determine which future modifications under consideration by SNC are incorporated into the model.
The following criteria are used to determine which, if any, future Vogtle plant modifications will be included in the NRC Level 3 PRA model:
The potential modification is risk significant, There is a regulatory commitment that the proposed plant change will be completed by the time the Level 3 PRA model is completed, If procedures and training are required, they meet the guidelines of RIS 2008-15 and they are implemented in a timeframe that does not impede the overall Level 3 PRA
- schedule, The effect of the modification has already been evaluated by the NRC (e.g., safety evaluation report issued) and accepted, and There is sufficient information for the Level 3 PRA project to understand the proposed change.
All of the above criteria must be met for the plant modification to be included in the Level 3 PRA model. If one of the criteria is not met, then the plant modification will not be included.
However, sensitivity studies may be performed to determine its impact on the PRA. The basis for including and not including a plant modification will be documented using Table 10.
5.14 Organization of the Various Types of Information on the SharePoint Site The Level 3 PRA project has different types of information that need to be stored and accessed. The various types of information are summarized in Table 12. Also provided in Table 12 are the access control settings for the different types of information.
Level 3 PRA Project QA Plan, Rev. 0 36 Table 11 Summary of Level 3 PRA Project Information on SharePoint Site Brief Description of Folder Contents Access Control*
General L3PRA Project Documents General documents relating to the work performed in support of this project (e.g., briefings, TAAP documents)
Read/Write Access:
Documentation Coordinator Read-Only Access:
All Level 3 PRA Project Team Members No Access:
All other NRC staff Reference Documents (Including Vogtle Site Information)
Plant specific information previously available at the NRC (e.g., FSAR, IPE, IPEEE)
General non-plant specific information (e.g., dry cask storage information)
Proprietary plant specific information sent by Vogtle in support of this project (e.g.,
PRA models and documentation, plant procedures, system diagrams)
Read/Write Access:
Documentation Coordinator Read-Only Access:
All Level 3 PRA Project Team Members*
No Access:
All other NRC staff Task Group Technical Documents Personal working files Read/Write Access:
Documentation Coordinator Each team member will have read/write access to their own working files.**
Read-Only Access:
All Level 3 PRA Project Team Members No Access:
All other NRC staff Technical Advisory Group Documents TAG information (e.g.,
meeting minutes)
Read/Write Access:
Documentation Coordinator TAG Coordinator Read-Only Access:
All Level 3 PRA Project Team Members No Access:
All other NRC staff Inbox: Upload Documents to the L3PRA Website Miscellaneous documents uploaded to the site that have not yet been filed Read/Write Access:
Documentation Coordinator Read-Only Access:
All Level 3 PRA Project Team Members No Access:
All other NRC staff
- To access the proprietary information area of this folder, project team members need to acknowledge the non-disclosure statement (as discussed in Section 5.9).
- Write access may be shared with other project team members as the request of the owning individual.
Level 3 PRA Project QA Plan, Rev. 0 37 6
TECHNICAL REPORTS Before technical reports are made available outside the Level 3 PRA project, it needs to be determined that each is ready for release. The technical report may either be one generated by a Level 3 PRA project member or a contractor supporting the Level 3 PRA project.
6.1 Staff Technical Reports There are two general types of staff technical reports developed as part of the Level 3 PRA project. The first type are reports that document major project milestones (e.g., the reactor, at-power, Level 1 PRA for internal events). The determination of whether this type of staff technical report is ready for release is accomplished in a four-step process and documented on a sign-off sheet, as described below. A second type of staff-generated technical report are those that support the major project reports (e.g., a report documenting a set of MELCOR calculations performed to resolve a specific issue or set of issues). The determination of whether this latter type of report is ready for release is addressed at the end of this section.
The sign-off sheet is the cover page of each technical report and involves the following:
- 1)
The task analyst (originator) performs a final check that all the necessary steps have been performed. These steps include (1) completion of all the necessary documentation and (2) completion of the self-assessment including its documentation.
Once the task analyst believes the work is ready for release, the analyst signs on the release form. By signing the form, the individual is confirming that he/she is the individual taking responsibility for the documented work.
- 2)
A separate individual performs a technical review. This individual is usually someone associated with the project who has technical knowledge in the subject area. Once the findings from the reviewer are adequately addressed and resolved, the reviewer then signs off on the release form. By signing the form, the reviewer is confirming that he/she has performed a technical review and is approving the technical content, except where noted with comments.
- 3)
A review is performed by a member of the Level 3 PRA Project Management Team.
By signing the form, this individual is approving the document revision and confirming that it is ready for external review.
- 4)
Once the above reviews are performed, the document is ready to be released to the TAG for review. This signature, generally provided by the Level 3 PRA Program Manager, confirms that the document revision has been provided to the TAG Chairman.
Each time a new version of the document is produced, a new sign-off sheet should be completed. In addition, a revision log should be included in the report and updated with each revision. The revision log includes the revision number, the date the Level 3 PRA Project Management Team member signed off on the revision, and a description of the major changes to the report and the reasons why. For the initial version of the report, Initial issuance is entered under the description heading.
Table 13 provides the template to be used for the sign-off sheet.
Level 3 PRA Project QA Plan, Rev. 0 38 Table 12 Sign-Off Sheet for Staff Technical Reports Title of Document Revision Number: _______________________
ADAMS ML Number: _______________________
Document Approvals ORIGINATOR: ___________________________________________________________
Name (printed) Signature Date TECHNICAL REVIEWER: ___________________________________________________________
Name (printed) Signature Date L3PRA MANAGEMENT REVIEW: ____________________________________________________________
Name (printed) Signature Date TAG REVIEW INITIATED: _____________________________________________________________
Name (printed) Signature Date Finally, in order to track the status of the technical reports (and their revision), a document checklist template has been developed (see Table 14). The document checklist will be completed collaboratively by the document originator and the Level 3 PRA Program Manager. The document checklists will not be included in the reports, but will be stored separately in a binder that will be maintained by the Level 3 PRA Program Manager.
A major inclusion as part of the documentation is to note the other participants in the work.
This documentation should note the specific areas they contributed so that there is an historical account for all participants in the future.
Level 3 PRA Project QA Plan, Rev. 0 39 Table 13 Document Checklist Revision 0 (only)
- Applicable code and model version (e.g., SAPHIRE 8.0.9.525, Model R01).......... ____________
SAPHIRE Version ________________ Model Version ________
- Changes in PRA Model of Record verified (i.e., configuration control of model)..... ____________
- Site visit trip reports completed................................................................................ ____________
- Document Consistent with format guidance (RES OI ADM-017 Preparation of NUREG Series Reports)................................................................................... ____________
Proofread (e.g., good, plain English; spell check; grammar check).................. ____________
Originator signature on Reviewer Sheet............................................................ ____________
Revision Number included................................................................................. ____________
Revision No. ____
Revision log sheet included after sign-off sheet................................................ ____________
Acknowledgement page included...................................................................... ____________
- All issues on the issue tracking list addressed in the document.............................. ____________
- Self-assessment performed and documented......................................................... ____________
- Model, document, and self-assessment documentation reflect resolution of self-assessment comments...................................................................................... ____________
- Technical review performed and technical reviewer signature on Reviewer Sheet ____________
- L3 PRA project management review performed and signature on Reviewer Sheet____________
- Document entered into ADAMS and accession number included on document..... ____________
SNC Review Revision No. ____ Full Review ____ Focused Review ____
- Document sent to SNC for Fact Check and Proprietary Review............................. ____________
- SNC comments added to issue tracking list............................................................ ____________
- Model, document, and issue tracking list reflect resolution of SNC comments, and new revision number (if changed)............................................................................ ____________
Revision No. ____
TAG Review Revision No. ____ Full Review ____ Focused Review ____
- Document sent to TAG for review............................................................................ ____________
- Consensus TAG review comments added to issue tracking list.............................. ____________
- Model, document, and issue tracking list reflect resolution of consensus TAG comments, and new revision number (if changed).................................................. ____________
Revision No. ____
PWROG Peer Review Revision No. ____ Full Review ____ Focused Review ____
- Material sent to PPR team 4 weeks in advance of PPR.......................................... ____________
- Logistics for PPR meeting performed (e.g., rooms, equipment, documentation).... ____________
Level 3 PRA Project QA Plan, Rev. 0 40 Table 13 Document Checklist
- Draft PPR report sent to NRC for review and comment.......................................... ____________
- NRC comments on PPR report sent to PPR team.................................................. ____________
- PPR comments added to issue tracking list............................................................ ____________
- Model, document, and issue tracking list reflect resolution of PPR comments, and new revision number (if changed)............................................................................ ____________
Revision No. ____
ACRS Review Revision No. ____ Full Review ____ Focused Review ____
- Document sent to ACRS.......................................................................................... ____________
- Presentation prepared and approved for ACRS briefing......................................... ____________
- ACRS briefed on: ______________________........................................................ ____________
- L3 PRA management identified ACRS comments to be added to issue tracking list...................................................................................................................... ____________
- Model, document, and issue tracking list reflect resolution of ACRS comments, and new revision number (if changed)............................................................................ ____________
Revision No. ____
Final L3 PRA Project Management Approval
- L3 PRA project management approval of document as Revision No. ____........... ____________
Revisions
- Originator modifies model and document as appropriate based on new information......................................................................................................... ____________
Applicable code and model version (e.g., SAPHIRE 8.0.9.525, Model R01)... ____________
SAPHIRE Version ________________ Model Version ________
Verify changes in PRA Model of Record (i.e., configuration control of model). ____________
- Originator signature on Reviewer Sheet.................................................................. ____________
Revision Number included................................................................................ ____________
Revision No. ____
Revision log sheet updated............................................................................... ____________
- Self-assessment performed and documented for modified portion of model.......... ____________
- Model, document, and self-assessment documentation reflect resolution of self-assessment comments..................................................................................... ____________
- Technical review performed for modified portion of model and technical reviewer signature on Reviewer Sheet................................................................................... ____________
- Document entered into ADAMS and accession number included on document (new accession number for each revision).............................................................. ____________
- L3 PRA project management review performed and signature on Reviewer Sheet................................................................................................................. ____________
Additional reviews required (L3 PRA management enters full, focused, or none, as appropriate):
SNC for Fact Check and Proprietary Review ____________
TAG review ____________
Level 3 PRA Project QA Plan, Rev. 0 41 Table 13 Document Checklist PWROG review ____________
ACRS review ____________
L3 PRA project management approval on revised document issued as Revision No. ____.................................................................................................... ____________
Comments: _____________________________________________________________________
As discussed at the beginning of this section, a second type of staff-generated reports are those that support the major project reports (e.g., a report documenting a set of MELCOR calculations performed to resolve a specific issue or set of issues). The determination of whether this type of staff technical report is ready for release is accomplished in a two-step process and documented on a sign-off sheet, as described below.
The sign-off sheet is the cover page of each technical report and involves the following:
- 1)
The task analyst (originator) performs a final check that the report is complete and appropriately formatted. Once the task analyst believes the work is ready for release, the analyst signs on the release form. By signing the form, the individual is confirming that he/she is the individual taking responsibility for the documented work.
- 2)
A separate individual performs a technical review. This individual is usually someone associated with the project who has technical knowledge in the subject area. Once the findings from the reviewer are adequately addressed and resolved, the reviewer then signs off on the release form. By signing the form, the reviewer is confirming that he/she has performed a technical review and is approving the technical content, except where noted with comments.
Each time a new version of the document is produced, a new sign-off sheet should be completed. In addition, a revision log should be included in the report and updated with each revision. The revision log includes the revision number, the date the technical reviewer signed off on the revision, and a description of the major changes to the report and the reasons why. For the initial version of the report, Initial issuance is entered under the description heading.
Table 15 provides the template to be used for the sign-off sheet.
Level 3 PRA Project QA Plan, Rev. 0 42 Table 14 QA Review and Acceptance Form for Staff (Supporting) Technical Reports Title of Document Revision Number: _______________________
ADAMS ML Number: _______________________
Document Approvals:
ORIGINATOR: ____ ______________________________________________
Name (printed) Signature Date TECHNICAL REVIEWER:
Name (printed) Signature Date 6.2 Contractor Technical Reports The determination of whether a contractor technical report is ready for release is accomplished through an acceptance review by the Level 3 PRA project task leader. When NRC staff sign off on a project document that includes contractor work in either the main report or an attachment, or references contractor work, the staff are not necessarily guaranteeing the technical adequacy of the contractor work, but are confirming that the work is appropriate for the project objectives, that the context of the work is consistent with other parts of the overall analysis, and that they can talk to the work at a high level to a third party audience.
Once the task leader believes the work is ready for release, the leader signs on the QA review and acceptance form (see Table 16). By signing the form, the individual is confirming that he/she is the individual taking responsibility for the documented work.
Level 3 PRA Project QA Plan, Rev. 0 43 Table 15 QA Review and Acceptance Form for Contractor Technical Reports Title of Document Revision Number: _______________________
ADAMS ML Number: _______________________
Document Approval L3PRA TECHNICAL LEAD:
Name (printed) Signature Date 7
QA PROGRAM IMPLEMENTATION AUDITS Periodic audits of the implementation of the Level 3 PRA project QA plan will be performed and will cover a representative sampling of project activities in order to verify compliance with QA plan requirements. The Level 3 PRA Project Management team will determine the scheduling of these audits and how they are to be carried out.