ML20155K827
| ML20155K827 | |
| Person / Time | |
|---|---|
| Issue date: | 06/10/2020 |
| From: | Matthew Humberstone NRC/NRR/DRA/APOB |
| To: | Linthicum R PWROG Risk Management Committee |
| References | |
| Download: ML20155K827 (7) | |
Text
UNITED STATES NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20555-0001 June 10, 2020 Mr. Roy Linthicum PWROG Risk Management Committee Chairman 630-926-3034 Roy.linthicum@exeloncorp.com
SUBJECT:
REGULATORY AUDIT
SUMMARY
RE: REGULATORY AUDIT IN SUPPORT OF REVIEWING INDUSTRY COLLECTION AND ANALYSES OF OPERATIONAL EXPERIENCE DATA FOR FLEX EQUIPMENT TO SUPPORT CREDITING FLEX EQUIPMENT IN REGULATORY APPLICATIONS.
Dear Mr. Linthicum:
In support of the ongoing effort to incorporate FLEX strategies into probabilistic risk assessment (PRA) models, the Pressurized Water Reactor Owners Group (PWROG) led an industry effort to collect and analyze FLEX equipment data to provide generic reliability estimates for standard FLEX equipment. As part of this effort, the PWROG developed draft report, PWROG-18043-P rev.0, FLEX Equipment Data Collection and Analysis. Since this approach is different from the conventional approach used to collect and analyze non-FLEX equipment operating experience (OpE) data via combined efforts of the Institute for Nuclear Power Operations (INPO) and the Idaho National Engineering Lab (INL), both the PWROG and the U.S. Nuclear Regulatory Commission (NRC) agreed that an NRC audit of the draft process would be beneficial.
On March 24th and 25th, 2020, staff from the Office of Nuclear Reactor Regulation, Region I, and the Office of Nuclear Regulatory Research conducted a remote audit of the PWROGs effort.
The audit included participation by INL staff and contractors, who are responsible for collecting and analyzing the non-FLEX data for use in developing component reliability parameters used in the NRC Standardized Plant Analysis Risk models. The staff audit evaluated the licensees report in light of long-standing NRC/INL data practices established for estimating reliability parameters for use in PRA, such as NUREG/CR-6823, Handbook of Parameter Estimation for Probabilistic Risk Assessment (2003) and NUREG/CR-6928, Industry-Average Performance for Components and Initiating Events at U.S. Commercial Nuclear Power Plants (2007). The NRC audit team and the PWROG extended the audit to provide additional time to review the FLEX equipment database and to further explore the unique statistical techniques used to analyze some of the data. After several additional discussions, the NRC closed out the audit on May 4, 2020.
The PWROGs effort and the NRCs audit, address one of the ongoing technical challenges to crediting FLEX and should provide a technical basis for incorporation of reasonable FLEX failure probabilities in both NRC and industry PRA models. The audit was closed out with a discussion on the audit teams observations, which is provided herein. During the close-out session, the PWROG discussed their plans to review the NRCs comments and to consider updating their data collection process and draft report.
As part of the review, INL staff and contractors developed technical report INL/EXT-20-58327, Evaluation of Weakly Informed Priors for FLEX data, (Agencywide Documents Access and Management System (ADAMS) Accession No. ML20155k835) which analyzed the weakly informed prior (WIP) statistical method that the PWROG used for FLEX equipment that did not have adequate run or start data. The observations listed below include insights from the INL analysts and their technical report.
The NRC audit team makes the following observations in the following areas:
PROCESS
- 1. The NRC audit team had several questions regarding the overall process, for example, the need for, but lack of, a failure definition. An explicit definition helps ensure a consistent approach and would limit the subjectivity and personal biases of individual reviewers. For example, some diesel generator start tests considered successes based on the availability and use of battery charging cables. It is uncertain that the analyzed data had verified that this contingency had been proceduralized to require a jumper source to remain at the location of the FLEX equipment in order to recover from a failed start attempt which would allow crediting the test as a successful start.
- 2. As part of the American Society of Mechanical Engineers/American Nuclear Society PRA standard requirement to use current, up-to-date data, a formal update process should be established and included in the overall data handling process. This becomes more important as licensees start relying more on their PRA models in risk-informed regulatory processes (i.e., 10 CFR 50.69, Technical Specification Task Force-505).
Along with a formalized update process, a formal data collection process would ensure a consistent approach to gathering FLEX OpE data.
- 3. Another observation discussed with the PWROG was the choice of the component boundary definitions when compared to the NRCs current definitions. The PWROG stated that they intended to use similar boundary definitions as the NRCs analyses and agreed to look at identified differences to see if errors were made. Because of inherent differences between installed and FLEX equipment boundary definitions, attention will be required for licensees and staff to appropriately model component and system failure probabilities in both the NRC and the licensees PRA models.
DATA COLLECTION
- 1. The NRC audit team had several questions and discussions regarding the overall data pedigree. There were discussions regarding the basis of the preventive maintenance (PM) frequencies and the possibility that these frequencies may not appropriately represent authentic equipment starts, which are necessary in order to generate reliability parameters. Specifically, the NRC audit team asked if the PMs provided by the licensees represented actual equipment starts and were not merely checklist-type rundowns of equipment availability which omitted running the equipment. The PWROG confirmed that, to the best of their understanding, the PMs did represent actual equipment starts but agreed that confirming this would be beneficial. There was also discussion about overlapping PMs listed for the same equipment and the potential for
double counting of equipment starts. For example, if one start and run fulfills the requirements for both a monthly and annual test, only one start and run should be recorded. These concerns are partially a reflection of the fact that the PWROG process is in its infancy compared to the established INPO process used by licensees since 1998, with the introduction of the Equipment Performance and Information Exchange database. As a result, individual licensee responses may not be consistent due to differing interpretations of the data request.
- 2. Another data pedigree observation was that some of the data was collected before licensees declarations of order compliance. There was some concern that failures occurring during these periods may not have been consistently captured in a licensees corrective action program. This was an area of ongoing concern for the NRC audit team and was discussed with the PWROG. The audit team believes through discussions with PWROG that the intent was to utilize test data after a licensee had made their declaration of order compliance and submittal of the final integrated plan. The audit team believed that approach would ensure the integrity of the data used.
- 3. Another topic that the NRC audit team focused on was whether the equipment run data represented loaded conditions. There were several conversations on the importance of limiting test results to fully loaded runs and questions regarding the likely inclusion of partial or non-loaded runs which would have the effect of artificially reducing failure rates in runtime calculations. Specifically, unloaded runs may not identify some failures which could be observed with the machines loaded (i.e. engine control modules, cooling systems, etc.).
DATA ANALYSIS
- 1. The NRC audit team reviewed the three data analysis methods that the PWROG used to develop the FLEX equipment failure probabilities. Out of the three methods, the audit teams review had concerns regarding the use of the WIP method. The WIP method was used for equipment categories that contain less than 50 demands or less than 100 hours0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br /> of run time (i.e., in cases where the test results were sparse). The audit team inquired if there were other ways to better categorize the equipment where the challenges associated with limited data could be eliminated. The PWROG indicated that these categories were established early in their study before they knew that these categories would possess such limits. The PWROG displayed an openness to investigating a revision to this approach, should they decide to update their analysis.
- 2. The NRC audit team received significant input from the INL staff and contractors, specifically with respect to the technical report INL/EXT-20-58327, Evaluation of Weakly Informed Priors for FLEX data. (ADAMS Accession No. ML20155k835) The concerns expressed by the INL statisticians centered around the WIP methods heavy reliance on engineering judgement. For example, there were discussions on: the proper range factor to use for new portable equipment, which attempts to measure the components failure rate uncertainty; the proper additional scaling factors Em values to use; and what other scaling factor should be chosen? As such, the NRC audit team voiced their
overall concern with the use of a method that relies heavily on subjective engineering judgement.
- 3. The NRC audit team questioned the use of the permanently installed equipment as the foundation for the inference function used to create the failure rates and commented if other equipment (e.g., other FLEX equipment) would provide a more accurate foundation to reflect FLEX performance as standby equipment.
- 4. Confirmatory calculations performed by INL revealed potential errors in the PWROGs WIP results. Specifically, it appears the PWROG used the beta equation when calculating alpha for the four fail-to-start WIP distributions.
- 5. The NRC audit team also noted that the current PWROG analysis lacks the common-cause failure (CCF) analysis as well as the unavailability analysis, which would be important elements to be included in PRA models reflecting FLEX equipment. The PWROG acknowledged the limitations and stated that such analyses might be added in the future when updating the FLEX data analysis.
CONCLUSION The NRC audit team recognizes the substantial amount of work that the PWROG completed in collecting and analyzing the FLEX data. During the audit and at its conclusion, the PWROG discussed their plans to use the NRC audit teams observations described in this audit summary to inform their final report and their data collection processes moving forward. If these observations and concerns are addressed in the PWROGs updated approach, this will provide a robust basis for FLEX equipment failure probabilities. The staff believes that the feedback provided to the owners group should help them as they proceed towards an industry-wide, NRC-approved data collection and analysis approach for FLEX.
contains a list of documents reviewed by the team during the audit.
If you have any questions, please contact me at 301-415-1464 or via e-mail at Matthew.Humberstone@nrc.gov.
Sincerely,
/RA/
Matthew Humberstone, PhD Reliability and Risk Analyst PRA Oversight Branch Division of Risk Assessment Office of Nuclear Reactor Regulation
Enclosure:
- 1. List of NRC Participants
- 2. List of Documents Reviewed
ML20155K827 OFFICE NRR\\DRA\\APOB NRR\\DRA\\APOB NAME MHumberstone AZoulis DATE 05 /18/20 05/21/20
LIST OF PARTICIPANTS U.S. Nuclear Regulatory Commission (NRC) Audit Team NAME ORGANIZATION Matthew Humberstone NRC/NRR Frank Arner NRC/RI Michael Montecalvo NRC/NRR John Lane NRC/RES Zhegang Ma INL/PRA Shawn W. St Germain INL/PRA John Schroeder INL/PRA(retired)
Andrea Mack INL/Statistics Cynthia Gentillon INL/Statistics(retired)
Cory Atwood INL/Statistics(retired)
Tom Wierman INL/PRA(retired)
LIST OF DOCUMENTS REVIEWED DURING THE AUDIT The following documents were reviewed by the NRC audit team during the audit. The documents generally included the implementation of the industrys FLEX OpE data analysis program; data analysis methods; calculations and bases for the methods; and FLEX OpE data.
PWROG-18043-P revision 0, FLEX Equipment Data Collection and Analysis PWROG-14003-NP, Implementation of FLEX Equipment in Plant-Specific PRA Models PWROG FLEX database Memo, Matthew M. Degonish from Westinghouse to Matthew Humberstone, NRC, forwarded to INL 4/15/2020 (with a table describing the NRC 2015 Parameter Update estimates used in the study and a print-out of Failure Rate Calculations.xlsx)