IR 05000395/2024040
| ML24121A017 | |
| Person / Time | |
|---|---|
| Site: | Summer  |
| Issue date: | 05/01/2024 |
| From: | Renee Taylor Division Reactor Projects II |
| To: | Carr E Dominion Energy Co |
| References | |
| EA-23-093 IR 2024040 | |
| Download: ML24121A017 (24) | |
Text
SUBJECT:
VIRGIL C. SUMMER NUCLEAR STATION - 95001 SUPPLEMENTAL INSPECTION REPORT 05000395/2024040 AND FOLLOW-UP ASSESSMENT LETTER
Dear Eric S. Carr:
On March 21, 2024, the U.S. Nuclear Regulatory Commission (NRC) completed a supplemental inspection using Inspection Procedure (IP) 95001, "Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs, and discussed the results of this inspection and the implementation of your corrective actions with Mr. Robert Justice, Site Vice President, and other members of your staff.
The NRC performed this inspection to review your stations actions in response to a White finding in the Mitigating Systems cornerstone which was documented and finalized in NRC Inspection Report 05000395/2023091. On January 25, 2024, you verbally informed the NRC that your station was ready for the supplemental inspection and followed up with written notification or readiness by letter dated March 7, 2024 (ADAMS ML24067A023).
The NRC determined that your staffs evaluation identified the cause of the White finding.
Specifically, the root causes of the failure to identify and correct the failure mechanism that affected the fuel oil system piping of the 'A' Emergency Diesel Generator (EDG) were: (1) that SAP-0999, "Corrective Action Program," Revision 13 and subsequent revisions, failed to drive effective evaluation and resolution of EDG Fuel Oil (FO) piping cracks; and (2) The FO piping design was less than adequate for maintenance adjustments and introduced challenges when realigning the piping to correct leaking connections.
The inspectors determined that the root cause evaluation was documented at a sufficient level of detail, included relevant operating experience, and identified the root causes, extent of conditions, and extent of causes of the performance issue. Based on the results of the inspection, the inspectors concluded the objectives of the IP were met.
The NRC determined that completed or planned corrective actions were sufficient to address and preclude repetition of the performance issue that led to the White finding. Therefore, in accordance with the guidance in Inspection Manual Chapter (IMC) 0305, "Operating Reactor May 1, 2024 Assessment Program," the performance issue will be closed and no longer considered as an Action Matrix input as of March 21, 2024, the date of the 95001 inspection exit and regulatory performance meetings. The NRC determined the performance at V.C. Summer Unit 1 to be in the Licensee Response Column of the Reactor Oversight Process Action Matrix as of March 21, 2024.
No findings or violations of more than minor significance were identified during this inspection.
This letter, its enclosure, and your response (if any) will be made available for public inspection and copying at http://www.nrc.gov/reading-rm/adams.html and at the NRC Public Document Room in accordance with Title 10 of the Code of Federal Regulations 2.390, Public Inspections, Exemptions, Requests for Withholding.
Sincerely, Ryan C. Taylor, Chief Reactor Projects Branch 3 Division of Reactor Projects Docket No. 05000395 License No. NPF-12
Enclosure:
As stated
Inspection Report
Docket Number:
05000395
License Number:
Report Number:
Enterprise Identifier:
I-2024-040-0004
Licensee:
Dominion Energy
Facility:
Virgil C. Summer Nuclear Station
Location:
Jenkinsville, SC
Inspection Dates:
March 18, 2024, to March 21, 2024
Inspectors:
S. Battenfield, Operations Engineer
J. Parent, Resident Inspector
A. Ponko, Senior Construction Inspector
Approved By:
Ryan C. Taylor, Chief
Reactor Projects Branch 3
Division of Reactor Projects
SUMMARY
The U.S. Nuclear Regulatory Commission (NRC) continued monitoring the licensees performance by conducting a 95001 supplemental inspection at Virgil C. Summer Nuclear Station, in accordance with the Reactor Oversight Process. The Reactor Oversight Process is the NRCs program for overseeing the safe operation of commercial nuclear power reactors.
Refer to https://www.nrc.gov/reactors/operating/oversight.html for more information.
List of Findings and Violations
No findings or violations of more than minor significance were identified.
Additional Tracking Items
Type Issue Number Title Report Section Status NOV 05000395/2023002-01 Failure to Identify and Correct a Condition Adverse to Quality Associated with the EDG Fuel Oil System EA-23-093 95001 Closed
INSPECTION SCOPES
Inspections were conducted using the appropriate portions of the inspection procedures (IPs) in effect at the beginning of the inspection unless otherwise noted. Currently approved IPs with their attached revision histories are located on the public website at http://www.nrc.gov/reading-rm/doc-collections/insp-manual/inspection-procedure/index.html. Samples were declared complete when the IP requirements most appropriate to the inspection activity were met consistent with Inspection Manual Chapter (IMC) 2515, Light-Water Reactor Inspection Program - Operations Phase. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel to assess licensee performance and compliance with Commission rules and regulations, license conditions, site procedures, and standards.
OTHER ACTIVITIES
- TEMPORARY INSTRUCTIONS, INFREQUENT AND ABNORMAL 95001 - Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response)
Inputs The inspectors reviewed and selectively challenged aspects of the licensees problem identification, causal analysis, and corrective actions in response to a White finding and associated Notice of Violation (NOV) of 10 CFR Part 50, Appendix B, Criterion XVI, for the licensee's failure to identify and correct the failure mechanism that affected the fuel oil (FO)system piping of the 'A' Emergency Diesel Generator (EDG). Notification of the NRC's updated assessments were documented in inspection reports 05000395/2023002 (ADAMS Accession Number: ML23223A006), 05000395/2023090 (ADAMS Accession Number: ML23268A467),05000395/2023091 (ADAMS Accession Number: ML23342A000).
Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs (1 Sample)
- (1) From March 18 - 21, 2024, the inspectors conducted an onsite review to verify all aspects of IP 95001 were met.
INSPECTION RESULTS
Assessment 95001 95001 assessment of failure to identify and correct a condition adverse to quality (CAQ) affecting the fuel oil system piping of the 'A' EDG Objective: Ensure that the root and contributing causes of significant individual and collective white performance issues are understood.
Under this objective, the inspectors reviewed the root cause evaluations (RCEs) the licensee conducted for the licensees failure to identify and correct the failure mechanism that affected the FO system piping of the A EDG. Their review consisted of an evaluation of the following: the licensee's identification of the issue(s), when and how long the issue(s) existed, prior opportunities for identification, documentation of significant plant-specific consequences and compliance concerns, use of systematic methodology to identify causes with a sufficient level of supporting detail, consideration of prior occurrences, identification of extent-of-condition and extent-of-cause, and identification of any potential programmatic weaknesses in performance.
NRC Assessment: The team concluded that this objective was met. The licensees RCE determined that the organization failed to correct the common failure mechanism of the FO system leaks and failed to identify the risks associated with the cracked piping leaks. Specifically, the leaks were considered a low-level deficiency and characterized as broke/fix issues. Work repairs were prioritized based on leakage magnitude and perceived risk. Additionally, the station relied heavily on the vendors position that the FO piping design was adequate. SAP-0999, "Corrective Action Program," Revision 13, and subsequent revisions failed to drive effective evaluation and resolution of EDG FO piping cracks (Root Cause (RC)1). The organization displayed inaccurate risk perception by not regarding a total pipe failure, at an EDG threaded connection, as a credible risk (Contributing Cause (CC)1). There was less than adequate management oversight to ensure appropriate technical conscience was applied to EDG Fuel System leaks (CC2). The FO piping design was determined to be inadequate for mechanical adjustments and introduced challenges when performing maintenance to correct leaking connections (RC2).
a. Identification. On November 2, 2022, the A EDG failed to complete the 24-hour run as required by Technical Specification (TS) 4.8.1.1.2.g.7 due to a fuel oil leak on a threaded connection. The NRC determined that the licensee failed to identify and correct the failure mechanism affecting the FO system piping, leaving the station vulnerable to EDG FO system failure. The NRC determined that this was a violation of 10 CFR 50 Appendix B, Criterion XVI, Corrective Action. Per RCE CA12034701, the licensee stated that this failure of the EDG FO line was self-revealing.
b. Exposure Time. The RCE determined that due to the degradation mechanism related to this event which only affected the EDG during its operating / running time, and since the EDG has limited surveillance run time each year, the exposure time was significant at 162 days. The inspectors determined that the licensee appropriately assessed the exposure time.
c. Identification Opportunities. In general, the licensee appropriately identified prior occurrences and identification opportunities. RCE CA12034701 documented that missed opportunities existed to identify and correct the failure mechanism that affected the FO system piping of the A EDG prior to its failure on November 2, 2022. The RCE documented that the previous corrective action program (CAP) procedure (SAP-0999) did not require a formal operating experience (OE) review to be performed for Quick Cause Evaluations (QCEs). The 2014 QCE (condition report (CR)-14-04592) was not required to include a formal OE review but did reference industry OE reports. This was a missed opportunity to apply referenced OE related to threaded connection fatigue failure caused by long-term vibration (OE249110) and fuel oil piping configuration that does not allow for minor variances from maintenance (OE309021). The RCE documented that a search of the licensee's CRs was performed based on the Equipment ID in PAMS (the CAP software) going back to January 1, 2003. This search produced a list of several CRs that represent missed identification opportunities. List as follows:
- CR-03-03500 - A EDG right bank ruptured threads - Same location as 2022.
- CR-14-04564 - A EDG male end threaded connection left bank - hairline crack between threads.
- CR-14-05797 - B EDG right bank fuel oil supply header inlet pipe connection (hairline) similar to CR-14-04564 / CR-03-03500.
- CR-15-01668 - A EDG minor leak - right bank header - could not identify location - replaced piping - no damage found.
- CR-16-04952 - B EDG FO seepage right bank header - disassemble union, cleaned and reassembled - no damage.
- CR-18-00899 - A EDG left bank supply piping - cracked bushing - identified on CR-18-05963 and CR-19-01935.
- CR-20-00712 - A EDG post-maintenance leakage (cracked pipe nipple) after repair of leakage identified in CR-18-00899 also CR-20-00735.
- CR-20-03909 - A EDG FO leak left bank above the duplex filter - removed piping, cleaned, reassembled.
d. Risk and Compliance. The RCE documented that an assessment of safety consequences was performed in accordance with PI-AA-300-3001, Attachment 3, Safety Consequences Evaluation. NRC/Dominion investigated the risk consequence using detailed probabilistic risk assessment (PRA) tools and concluded that the significance of the event was White (low-to-moderate) through the Reactor Oversight Process significance determination process. The assessment also determined that RC1 and RC2 had an impact on the Mitigating Systems Regulatory Cornerstone due to the impact on nuclear safety defense in depth. The assessment determined that there was no reasonable assurance that A EDG remained capable of providing emergency AC power to all engineered safety feature loads during a loss of offsite power following the through wall crack of the right bank fuel header pipe nipple, which resulted in this being considered a mitigating system performance index (MSPI) failure to run. Based on their review, the inspectors concluded the RCE demonstrated an understanding of the significant plant consequences and compliance concerns associated with the event and the White performance issue. The NRCs risk evaluation of the White performance issue was documented in inspection report 05000395/2023091 (ML23342A000).
e. Methodology. The RCE employed a systematic evidence-based methodology to determine the root cause and contributing causes of the White finding and the A EDG inoperability including, Event and Causal Factors Charting, Hazard Barrier Target Analysis, and Organizational/Programmatic Effectiveness Checklist.
f. Level of Detail. The inspectors determined the RCE was performed commensurate with the safety significance and complexity of the performance issue and was sufficient detail to identify the root and contributing causes, extent of conditions, and extent of causes. The RCE team utilized a formal cause analysis process to identify the problems and determine corrective actions.
g. Operating Experience. The inspectors determined that the licensee appropriately considered prior occurrences and operating experience during the root cause evaluation. The RCE determined that there were several CRs and industry OE that were missed identification opportunities and procedure SAP-0999 failed to drive effective evaluation and resolution of EDG FO piping cracks.
h. Extent of Condition and
Cause.
The licensee used the same object - same application, same object - other application, similar object - same application, and similar object - other application methodologies to evaluate the extent of condition and the extent of cause. The inspectors reviewed the safety culture traits in NUREG-2165, Safety Culture Common Language, referenced in IMC 0310-06, to determine if these were appropriately considered during the licensees evaluations of the root causes, extent of conditions, and extent of causes.
i. Common
Cause.
No common causes were identified by either the licensee or the inspectors.
Objective: Ensure that the extent-of-condition and extent-of-cause of individual and collective white performance issues are identified.
Under this objective, the inspectors independently assessed the Organizational/Programmatic Effectiveness Checklist and the Hazard Barrier Target Analysis, documented in the RCE, to assess the licensee's extent-of-condition and extent-of-cause.
NRC Assessment: The team concluded that this objective was met. Pertaining to the extent of condition, the RCE determined that the original equipment manufacturer (OEM) design and configuration of both of the licensee's EDGs are identical, that the consequence of a cracked pipe would be similar to the event described in CR1211780, and that the consequences would be the same for both EDGs. The RCE also determined that the extent of condition was limited to other sections of schedule 40 carbon steel piping, within MSPI and PRA risk significant systems, 1 diameter or smaller, and in high vibration environments. Pertaining to the extent of cause, the RCE determined that the review of conditions was bounded to non-condition adverse to quality (NCAQ) CRs written within three years prior to the implementation of licensee procedure PI-AA-200, "Corrective Action." CAQ CRs were excluded because those conditions would have been required to be closed to a CR action, work order (WO), or other program and would have been screened for repeat conditions per SAP-0999. The date was set for three years prior to PI-AA-200 implementation because a three-year search was specified by PI-AA-200 for CAQ repeat condition searches. CRs written after February 2022 were screened per PI-AA-200 requirements per the post-implementation monitoring performed by the CAP. The RCE determined that there was minimal risk and consequence because the RCE's extent of cause evaluation included other procedures that could impact condition screening and evaluation, and the three-year review provides a reasonable time period to detect equipment vulnerabilities per PI-AA-200. The inspectors determined that the licensee had sufficiently addressed the Design Control attribute of the Mitigating Systems cornerstone during the extent of condition review by evaluating where schedule 40 3/4" piping was installed in the FO system of the EDGs and modifying the system to ensure the availability, reliability, and capability of the EDGs to respond to initiating events to prevent undesirable consequences.
Extent of Condition and
Cause.
The licensee used the same object - same application (Tier 1), same object - other application (Tier 2), similar object - same application (Tier 3), and similar object - other application (Tier 4) methodologies to evaluate the extent of condition and the extent of cause. For the extent of condition, Tier 1, the RCE determined that the B EDG was also potentially impacted by this condition since its right bank FO header supply schedule 40 3/4 inch-diameter by 15-inch-long pipe nipple was identical to the A EDG right bank FO header supply schedule 40 3/4 inch-diameter by 15-inch-long pipe nipple. For the extent of condition, Tier 2, the RCE determined that the object of the condition, being a 15-inch-long 3/4-inch schedule 40 pipe nipple is a component unique to the EDG skid. It was specifically fabricated and supplied by the OEM (Fairbanks Morse) and is used nowhere else on the EDGs or other safety-related plant components in other MSPI or PRA Risk Significant Systems. For the extent of condition, Tier 3, the RCE determined that schedule 40 3/4-inch-diameter piping was used in other applications of EDG piping and that the schedule 40 3/4-inch-diamter by 2-inch-long pipe nipple was also susceptible to crack formation at the root of the thread which could propagate during engine operation based on a review of internal plant operating experience. The RCE also determined that the potential existed for a similar condition to occur on schedule 40 threaded piping that may be installed in other EDG subsystems such as lubricating oil, jacket water, and intercooler systems. For the extent of condition, Tier 4, the RCE determined that schedule 40 3/4-inch-diameter piping is used in other applications of plant piping. For the extent of cause, Tier 1, the RCE determined that SAP-0999 impacted the screening of EDG FO crack CRs and required higher level evaluations. For the extent of cause, Tier 2, the RCE determined that SAP-0999 impacted the screening of equipment CRs and required higher level evaluations. For the extent of cause, Tier 3, the RCE determined that no other procedures had the potential to impact CR screening, including repeat conditions, and evaluation of EDG FO cracks since PI-AA-200 has superseded SAP-0999 and did not contain the same vulnerabilities as SAP-0999. PI-AA-200 directed all plant equipment conditions to be screened as CAQ and Attachment 6 directed the Condition Reports Team (CRT) to consider Level of Effort Evaluations (LEE) or other evaluations for repetitive CAQ conditions. For the extent of cause, Tier 4, the RCE determined that no other procedures had the potential to impact CR screening, including Repeat Conditions, and evaluation of other equipment conditions since PI-AA-200 provided guidance on screening CRs and assigning evaluations, and it did not contain the same vulnerabilities as SAP-0999. Using IP 95001, Supplemental Inspection Response to Action Matrix Column 2 (Regulatory Response) Inputs, the inspectors determined that the licensee appropriately identified the extent of condition and extent of cause in the RCE.
Objective: Ensure that completed corrective actions to address and preclude repetition of white performance issues are timely and effective.
Under this objective, the inspectors assessed the appropriateness and timeliness of the licensee's corrective actions.
NRC Assessment: The team concluded that this objective was met. The inspectors determined that these corrective actions were both timely and adequate to prevent recurrence.
a. Completed Corrective Actions to Prevent Recurrence.
1. Corrective Action to Preclude Repetition (CAPR)-1: Implementation of PI-AA-200,
"Corrective Action," and revision to Attachment 7, Risk and Uncertainty Model, to include Through-wall leakage in Safety-Related piping as an example of a Medium Risk event. CAPR-1 was implemented to address RC1 (SAP-0999 failed to drive effective evaluation and resolution of EDG FO piping cracks). The inspectors determined that this corrective action was both appropriate and timely in addressing the licensees failure to identify and correct a CAQ for the EDG fuel oil system that left the system vulnerable to premature piping cracks because PI-AA-200 had already been implemented and replaced SAP-0999. Unlike SAP-0999, PI-AA-200 requires all plant equipment related conditions to be screened as CAQs and requires repetitive CAQs to be considered for evaluations. Implementation of PI-AA-200 addressed the gaps in the CAP process regarding screening of plant equipment related conditions and recommended evaluations for potential repeat events as stated in the RCE.
2. CAPR-2 & CAPR-3: Develop Emergency Diesel Generator redesign per VC-23-
00022, "Emergency Diesel Generator Fuel Piping Upgrade," that improves design margin and maintainability (CAPR-2). Implement the Emergency Diesel Generator redesign per VC-23-00022, Emergency Diesel Fuel Piping Upgrade (CAPR-3). CAPR-2 & CAPR-3 addressed RC-2 (FO piping design is less than adequate for maintenance adjustments and introduces challenges when realigning the piping to correct leaking connections). The inspectors determined that these corrective actions were both appropriate and timely in addressing the licensees failure to identify and correct a CAQ for the EDG fuel oil system that left the system vulnerable to premature piping cracks in that the design change was performed in accordance with 10 CFR Part 50, Appendix B, Criterion III, Design Control. Specifically, the design change was subject to design control measures consistent with the requirements of 10 CFR Part 50, Appendix B, and was approved by the designated responsible organization.
The design basis document for the diesel generator engine support and control systems (DG) states that the fuel oil system shall be designed and fabricated in accordance with the American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel Code (BPVC),Section III, Class 3. An exception from these requirements, however, is provided for the on-engine piping. This piping, between the fuel pump, duplex filters, and fuel headers was designed, fabricated, and installed by the original equipment manufacturer and supplied with the diesel generator. It was designated as safety-related and was originally fabricated with 3/4-inch-diameter and 1-inch-diameter nominal pipe size (NPS) schedule 40 carbon steel pipe in accordance with American Society for Testing and Materials (ASTM) A-53 using threaded connections.
As part of the design change, the on-skid ASTM A-53 schedule 40 carbon steel piping, between the fuel pump, duplex filters, and fuel headers was replaced with ASME SA-106 Grade B Schedule 80 piping. The threaded piping joints were replaced with socket welded connections at all locations except at the existing threaded fittings integral to the fuel pump, duplex filter, and fuel headers and at the check valve adjacent to the fuel pump. To provide fatigue resistance comparable to butt welded connections, the socket welds were made using a 2-to-1 weld leg configuration (weld leg along the pipe side of the weld equal to twice the Code-required weld leg dimension) as recommended in Technical Report TR-113890 published by the Electric Power Research Institute (EPRI). Additionally, the rigid pipe fittings in the vicinity of the fuel headers were replaced with sections of flexible hose to mitigate the transmission of vibrations through the fuel oil piping from engine operation. Lap joint flanges were also installed between the flexible hose and piping to allow for assembly/disassembly and ease of fit-up. Engine mounted piping supports were redesigned and reconfigured to provide support for the modified piping. The inspectors verified that the material used for the on-skid fuel oil piping systems modifications was rated for the design pressure and temperature provided in the equipment suppliers design specification.
The revised on-skid fuel oil piping was, in general, designed and fabricated in accordance with USA Standard Code for Pressure Piping, Power Piping, USAS B31.1.0-1967, published by ASME. All welding, however, was specified to be performed using ASME BPVC,Section III, Class 3 guidance, inspection, and testing. USAS B31.1.0-1967, Section 114.2 precludes the use of threaded joints where vibration is expected to occur. As described above, the licensee replaced the majority of threaded joints with welded socket connections. However, threaded joints were not eliminated entirely. The licensee determined that the use of threaded joints at the existing threaded fittings integral to the fuel pump, duplex filter, and fuel headers and at the check valve adjacent to the fuel pump was justified since the joints are not subject to constant vibration and the measured vibrations at these locations were low.
The inspectors reviewed the pipe stress analysis of the modified fuel oil supply lines. The inspectors verified that the acceptance criteria used was consistent with the design specification for the EDG service system piping (VC-SPEC-000-DSP-544F)and that all calculated stresses were below the allowable limits.
The licensee performed post modification vibration testing of the redesigned on-skid fuel oil system piping. The licensee used the methodology described in ASME Standard, Operation and Maintenance of Nuclear Power Plants, 2020 edition (OM-2020), Division 2, Part 3 to evaluate the modified on-skid piping. The inspectors reviewed the Engineering Technical Evaluations (ETEs) completed by the licensee and verified that the measured deflections met the acceptance criteria.
The inspectors reviewed calculations for the design of the revised piping supports and work orders associated with the on-skid fuel oil piping system modifications. The inspectors verified that the calculated stresses and interaction ratios of the redesigned supports did not exceed the allowable limits. The inspectors also verified that welding and bolting of pipe joints, flanges, and supports, as applicable, was completed satisfactorily and that no leaks were observed during leak testing of the completed piping modifications.
b. Other Completed Corrective Actions.
1. The following corrective actions (CAs) were implemented by the licensee to address
RC-1 (SAP-0999 failed to drive effective evaluation and resolution of EDG FO piping cracks):
a.
CA-1 (CA12194078): Engineering Personnel to discuss with Site Engineering the causes, lessons learned and discuss documentation of OE review performed as part of Level of Effort evaluations with Engineering Systems.
b.
CA-2 (CA12194087): CAP Personnel to discuss the causes, lessons and documentation of OE review performed as part of Level of Effort evaluations with CAP department.
c.
CA-3: Review PI-AA-200, Attachment 6, Item 1 (if an event is determined to be a potential repeat, Condition Report Review Team (CRT) should consider recommending a Level of Effort Evaluation or a higher level of review for the subsequent evaluation):
i.
CAP Personnel to discuss with Engineering Leadership and Duty Supervisors to establish criteria for consistent application.
ii.
CAP Personnel to discuss with CAP Facilitators personnel to ensure it is applied consistently.
iii.
CAP Personnel to discuss with CRT members how this requirement will be applied.
Additionally, as part of PI-AA-200 implementation, a change management plan was completed to identify actions needed for procedure and software implementation. That action plan included:
- Site-wide communications through the D-15 newsletter on implementation timeline and process differences.
- Discussions at the CRT meeting to discuss CR Screening and changes to Unit Evaluator responsibilities as that title was changed to Department Corrective Action Coordinator (DCAC) under PI-AA-200.
- A presentation at the Accountable Leaders Meeting (ALM) covering high level differences between the programs.
- Multiple training avenues were made available. A computer-based training (CBT) module was created and released for "General User Training" and in-person sessions were held for "Super Users" who would have additional responsibilities which included operations personnel, station leadership, and DCACs.
- Post-implementation support was made available for several weeks after PI-AA-200 and PAMS (the CAP software) implementation which included back-shift support for process and technical issues.
- As part of a post-implementation review, CRs rescreened from NCAQ to CAQ were tracked at the Prescreen, CRT, and Corrective Action Assignment Review Team (CAART) levels to ensure CRs were being screened per the new requirements. That tracking was ended in November of 2022, successfully meeting the closure goal with less than 5% of CRs rescreened from NCAQ to CAQ during that period.
The inspectors determined that these corrective actions were both appropriate and timely in addressing the licensees failure to identify and correct a CAQ for the EDG fuel oil system that left the system vulnerable to premature piping cracks because all of the above corrective actions have been implemented to assist plant personnel in screening, tracking, and evaluation of CAQs.
2. The following CAs were implemented by the licensee to address RC-2 (FO piping
design is less than adequate for maintenance adjustments and introduces challenges when realigning the piping to correct leaking connections):
a.
CA-4 (CA12194098): Perform 10 CFR 21 Evaluation for Reportability using LI-AA-301, 10 CFR 21 Discovery and Evaluation Checklist.
b.
CA11670087 (24 Hour Run A DG Fuel Oil Leak with increasing leak rate): The System Engineer should determine the most efficient and cost-effective way to eliminate the vulnerability through options such as a design change or provide a method of pipe assembly to avoid the introduction of additional stress due to misalignment conditions. Present the resulting recommendation to the Plant Health Committee to establish long term corrective actions to minimize recurrence.
The inspectors determined that these corrective actions were both appropriate and timely in addressing the licensees failure to identify and correct a CAQ for the EDG fuel oil system that left the system vulnerable to premature piping cracks in that all of the above corrective actions have been implemented to assess the effectiveness of the EDGs original FO piping and to resolve the design vulnerability.
3. The following CAs were implemented by the licensee to address Contributing Cause
(CC)-1, "The organization displayed inaccurate risk perception by not regarding a total pipe failure, on an Emergency Diesel Generator threaded connection, as a credible risk":
a.
Corrective Action for Contributing Cause (CACC)-1 (CA12194130): "Implement the Culture of Prevention Model as part of the Self-Awareness & Self-Correction Dashboard action items. Emphasize Consequence Bias aspect of Culture of Prevention."
b.
CACC-2 (CA12194132): "Revise ER-AA-PRS-1001, "Plant Health Committee," to ensure sustainability for the periodic review of long-standing, repetitive or chronic equipment issues through an annual requirement to review long-standing/chronic or repeat issues in the Plant Health Steering Committee (PHSC) and provide criteria to support the review in supporting attachments as needed."
c.
CACC-3 (CA12194140): "Revise OAP-100.6, Control Room Conduct and Control of Shift Activities, Enclosure B (Shift Manager 1300 Meeting Agenda) to include any Prevention, Detection, Correction (PDC) concerns that need to be evaluated or elevated."
d.
CACC-4 (CA12194141): "Revise the CRT Agenda to include line item: 'Are there any CRs that need risk review per PSEG-44, Degraded Equipment Risk Assessment review or equivalent? If so, add PI1 PO&C Code to CR Trends Tab.'"
e.
CACC-5 (CA12194142): "Create PDC Model Posters and post around the site."
f.
CACC-6 (CA12194143): "Conduct Leadership Training on the Prevention, Detection, and Correction Model."
g.
CACC-7 (CA12194144): "Submit WOBS (licensee observation database) attribute updates as needed to include Culture of Prevention Model, per PI-AA-5002. Generate follow-on assignments as needed for incorporation into WOBS."
h.
CACC-8: "Perform a collegial review of a sample of CAQ CRs written since PDC Model implementation to determine if the PDC mindset is being appropriately applied during screening. Consider if issues are being evaluated at the appropriate level, are equipment issues being assigned PSEG-44 or equivalent reviews as needed and is the PI1 PO&C code being applied appropriately.
Generate follow-on assignments as needed if additional actions need to be taken.
i. PDC Modeling was discussed during an All-Hands Meeting.
ii. PDC was implemented on January 1, 2024, an effectiveness review of PDC was completed on March 11, 2024, and found only 1 CR that should have been screened as a CAQ.
The inspectors determined that these corrective actions were both appropriate and timely in addressing the licensees failure to identify and correct a CAQ for the EDG fuel oil system that left the system vulnerable to premature piping cracks in that all of the above corrective actions have been implemented to assist plant personnel in screening, tracking, and evaluation of CAQs.
4. The following CAs were implemented by the licensee to address CC-2, "There was
less than adequate management oversight to ensure appropriate technical conscience was applied to EDG Fuel System Leaks":
a.
CACC-9: "Perform an independent review of RCE11271696, RCE11316973, &
RCE12034701 for additional learnings, including common causes, organizational and programmatic deficiencies, and technical conscience. Generate new CRs or follow-on assignments as needed to address identified conditions."
b.
CACC-1: "Implementation of the PDC model and supporting actions per Contributing Cause 1 addresses this behavior."
c.
CACC-2: "Revision of ER-AA-PRS-1001, "Plant Health Committee," per Contributing Cause 1 addresses oversight of long-standing, repetitive, or chronic equipment issues. This revision:
i. Implemented an annual requirement to review long-standing/chronic or repeat issues in the PHSC.
ii. Revised supporting attachments to ensure long-standing/chronic or repeat issues are screened appropriately by the PHC."
The RCE states that implementation of PI-AA-300, "Cause Evaluation," and PI-AA-300-3007, "Level of Effort Evaluation," established supervisory and management oversight of Level of Effort Evaluations. The effective date for these procedures was February 3, 2022.
Objective: Ensure that planned corrective actions to preclude repetition direct timely and effective actions to address and preclude repetition of significant individual and collective performance issues.
Under this objective, the inspectors assessed the appropriateness and timeliness of the licensee's planned corrective actions.
NRC Assessment: The team concluded that this objective was met. The NRC recognizes some actions are not complete, but do not affect our determination that corrective actions are timely and adequate to prevent recurrence.
a. Planned Corrective Actions to Prevent Recurrence.
1. Corrective Action to Preclude Repetition (CAPR)-1: Implementation of PI-AA-200,
Corrective Action, and revise to Attachment 7, Risk and Uncertainty Model, to include Through-wall leakage in Safety-Related piping as an example of a Medium Risk event.
The RCE stated that the licensee will perform EFR-1 (an effectiveness review) in accordance with procedure PI-AA-300-3001, "Root Cause Evaluation." The effectiveness criteria will evaluate the following Effectiveness Goals:
- (1) Confirm PI-AA-200 procedure revision has occurred (completed).
- (2) Review of sample NCAQ CRs written since CAPR implementation to ensure proper screening with a success measure of greater than or equal to 95% (due by September 6, 2024).
- (3) Review CRs to ensure through-wall leak on Safety-Related components are being screened commensurate with Medium Risk with a success measure of greater than or equal to 95% (due by September 6, 2024). The inspectors determined that EFR-1 incorporates appropriate quantitative and qualitative measures of success due to that fact that PI-AA-200 has already been implemented, and that a 6-month evaluation (due September 6, 2024) to gage if a greater than or equal to 95% efficiency has been achieved are reasonable measures of success.
2. CAPR-2 & CAPR-3: Develop Emergency Diesel Generator redesign per VC-23-
00022, "Emergency Diesel Generator Fuel Piping Upgrade," that improves design margin and maintainability (CAPR-2). Implement the Emergency Diesel Generator redesign per VC-23-00022, Emergency Diesel Fuel Piping Upgrade (CAPR-3).
The licensee is actively performing document reviews of all related mod change work orders and will close them out once review is completed. Completion is scheduled for the spring / summer of 2024.
The RCE determined that the Licensee is to perform EFR-2 in accordance with PI-AA-300-3001, "Root Cause Evaluation." The effectiveness criteria will evaluate the following Effectiveness Goals:
- (1) Complete STP-125.002A/B "Diesel Generator A/B Operability Testing," with no quantifiable fuel oil leakage observed on the modified portion of EDG Fuel Oil Piping until EFR-2 is closed (due by March 10, 2025 (12 months after CAPR Implementation)).
- (2) Accumulate 24-hours of run time with no quantifiable fuel oil leakage observed on the modified portion of EDG fuel oil piping (due by March 10, 2025 (12 months after CAPR Implementation)). The inspectors determined that the effectiveness review incorporates appropriate quantitative and qualitative measures of success because the EDGs need runtime to ensure the design change was effective in preventing pipe cracking and leakage of the FO system.
b. Other Planned Corrective Actions.
1. The RCE identified that the current format of one of the Operating Experience
questions in the LEE Equipment Checklist could be misinterpreted. Documentation of the OE search and relevant OE should be performed regardless of the "Yes/No" response to the question. The RCE determined that a procedure revision was needed to address this. A corrective action was written as a Department Tracking Item (DTI)
(Due spring/summer of 2024).
a. DTI: Revise PI-AA-300-3007, "Level of Effort Evaluation," Attachment 7, Equipment Checklist Question H to read: Is there a deficiency in how OE applicable to this component has been addressed? Document the OE review including key words used, databases researched, or previous analysis performed. (Both internal and industry OE) If "yes," complete the Organizational/Programmatic Checklist.
The inspectors determined that the DTI was appropriate and timely in addressing the Licensees failure to identify and correct a CAQ for the EDG fuel oil system that left the system vulnerable to premature piping cracks because the revision to PI-AA-300-3007 clarified that an OE review was to be performed and documented, including method, which will decrease the likelihood of recurrence.
Conclusion The inspectors concluded the corrective actions to preclude repetition of the root and contributing causes (causal factors) of the White performance issue were effective and adequately prioritized considering safety significance and regulatory compliance. In addition, the inspectors determined that evaluations were documented at a sufficient level of detail, included relevant operating experience, and identified the root causes, extent of conditions, and extent of causes of the performance issue. Based on the results of the inspections, the inspectors concluded that the objectives of the inspection procedure were met and that the finding will be closed.
EXIT MEETINGS AND DEBRIEFS
The inspectors verified no proprietary information was retained or documented in this report.
- On March 21, 2024, the inspectors presented the 95001 supplemental inspection results and Mr. Ryan Taylor presented the regulatory performance meeting conclusions to Mr.
Robert Justice, Site Vice President, and other members of the licensee staff.
DOCUMENTS REVIEWED
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
DC05600-111
Pipe Stress Analysis for Emergency Diesel Generator Fuel
Oil Supply Lines VC Summer Unit 1
Rev. 0
DC05600-114
Flexible Hose Design Calculations for EDG Fuel Oil Supply
Piping
Rev. 0
Calculations
DC05610-013
Pipe Supports for Skid Mounted EDG Fuel Oil Supply Piping
Rev. 0
CA11328477
Level of Effort Evaluation for CR1211780
01/10/2023
CA12194078
Engineering Personnel to discuss with Site Engineering
10/26/2023
CA12194087
CAP personnel to discuss the causes, lessons and
documentation of OE review performed as part of Level of
Effort evaluations with CAP department.
10/26/2023
CA12194098
Perform 10 CFR 21 Evaluation for Reportability using LI-AA-
301
10/26/2023
CA12194130
Implement the Culture of Prevention Model
10/26/2023
CA12194132
Revise ER-AA-PRS-1001, Plant Health Committee
10/26/2023
CA12194140
Revise OAP-100.6, Control Room Conduct and Control of
Shift Activities, Enclosure B
10/26/2023
CA12194141
Revise the CRT Agenda
10/26/2023
CA12194142
Create PDC Model Posters and post around the site.
10/26/20223
CA12194143
Conduct Leadership Training on the Prevention, Detection,
and Correction Model
10/26/2023
CA12194144
Submit WOBS attribute updates as needed.
10/26/2023
CA12194157
EOC - Review the Emergency Diesel Generator engine skid
10/26/2023
CA12194158
EOC - EF System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194159
EOC - RH System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194160
EOC - AH System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194161
EOC - FW System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
95001
Corrective Action
Documents
CA12194164
EOC - SG System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
CA12194165
EOC - CS System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194165
EOC - CS System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194166
EOC - ES System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194167
EOC - SI System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194168
EOC - ED System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194170
EOC - SW System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194172
EOC - CC System Engineer to determine if any 1" or smaller
threaded connection Sch40 piping.
10/26/2023
CA12194173
EOCa - EF System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194174
EOCa - RH System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194176
EOCa - AH System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194177
EOCa - FW System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194178
EOCa - SG System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194179
EOCa - CS System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194187
EOCa - ES System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194188
EOCa - SI System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194189
EOCa - ED System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12194190
EOCa - SW System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
CA12194191
EOCa - CC System Engineer to review historical NCAQ CRs
and corrective work orders.
10/26/2023
CA12283467
Submit written notification for Part 21 evaluation in follow-up
2/21/2023
CR-14-04592
When performing WO 14011019 on XEG0001A-E, A Diesel
Generator, to repair a fuel oil leak, it was identified that the
pipe nipple (3/4" X 2") had a hairline crack in between two of
the threads. The pipe nipple is a part of the fuel oil piping
from the FO duplex filter outlet to the left bank (Cyl. 1-6_ fuel
oil supply header. The CR is written in accordance with
SAP-0999 to document the condition discovered during the
maintenance.
08/21/2014
CR-14-05797
Diesel Generator 'B' has a minor leak on the fuel oil supply
header inlet pipe connection, located on the southwest side
of the skid. The leak is approximately 2 drops per minute
with the DC Fuel Oil Pump running, and less than
measurable in standby condition. A similar leak occurred on
'A' DG in September of this year, with a magnitude of 120
drops per minute (CR-14-04564).
11/01/2014
CR-20-00712
While performing maintenance retest of work performed on
WO 1909952-002, fuel oil was observed still leaking around
a fitting on the cross. Reference CR-20-00735 (Closer
inspection found crack on pipe nipple) FM12650 3/9/2020.
03/05/2020
CR-20-00735
During pre-start checks on the A Diesel, Operations
identified fuel leakage at the pipe nipple on the top of the
cross that was recently worked under WO 1909952.
03/06/2020
CR1211780
Hour Run 'A' DG Fuel Oil Leak with increasing leak rate.
11/02/2022
CR1234635
NRC Identified Finding for the 'A' EDG 11/2/22 Run Failure
08/02/2023
CR1251237
Common Cause Evaluation of Three Recent Root Cause
Evaluations
2/19/2024
B
MRule Evaluation for CR1211780
Rev. 4
RCE#:
CA12034701
Failure to Identify and Correct 'A' Emergency Diesel
Generator (XEG0001A-E) Pipe Thread Crack Vulnerability
03/07/2024
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
Corrective Action
Documents
Resulting from
Inspection
CR1253957
Tubing and hose clamps do not have rubber softeners
03/18/2024
VC-EDWG-000-
23-00022-32-005-
2 SH-001
Fuel Oil System
Rev. 0
VC-EDWG-000-
23-00022-32-161-
001-A SH-001
Diesel Generator Air Start Tubing
Rev. 0
VC-EDWG-000-
23-00022-32-161-
001-B SH-001
Diesel Generator Air Start Tubing
Rev. 0
VC-EDWG-000-
23-00022-32-227-
001 SH-001
Fuel Pump to Duplex Filter Piping Isometric DGB Elevation
above 436'-0"
Rev. 0
VC-EDWG-000-
23-00022-32-227-
2 SH-001
Duplex Filters to Fuel Headers Piping Isometric DGB
Elevation above 436'
Rev. 0
VC-EDWG-000-
23-00022-32-228-
001 SH-001
3/4" Nom. Flexible Metal Hose Tag No. XPS0264A &
XPS0264B (Cylinder Bank 7-12)
Rev. 0
VC-EDWG-000-
23-00022-32-228-
2 SH-001
3/4" Nom. Hose Assembly Tag No. XPS0265A & XPS0265B
(Cylinder Bank 1-6)
Rev. 1
VC-EDWG-000-
23-00022-32-229-
001 SH-001
'A' & 'B' EDG North Fuel Oil Piping Support
Rev. 3
VC-EDWG-000-
23-00022-32-229-
2 SH-001
'A' & 'B' EDG North Fuel Oil Piping Support
Rev. 4
Drawings
VC-EDWG-000-
23-00022-32-230-
001 SH-001
Fuel Rail Bushing Fabrication Details
Rev. 0
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
VC-EDWG-000-
2-005 SH-001
Piping Schematics
Rev. 13
Engineering
Changes
VC-23-00022
EDG Fuel Piping Upgrade
Rev. 3
ETE-VC-2023-
0017
Operating Vibration Data for Existing Skid Fuel Oil Piping
Rev. 0
ETE-VC-2024-
0011
Operating Vibration Review for Modified EDG Skid Fuel Oil
Piping
Rev. 1
ETE-VC-2024-
0014
Document 'A' EDG Post-Modification Testing Vibration Data
Review
Rev. 0
Engineering
Evaluations
ETE-VC-2024-
0017
Document 'B' EDG Post-Modification Testing Vibration Data
Review
Rev. 0
DEMA Standard Practices, Standard Practices for Stationary
Diesel and Gas Engines, Sixth Edition, Diesel Engine
Manufacture's Association
1972
Electric Power Research Institute (EPRI), Vibration Fatigue
Testing of Socket Welds (PWRMRP-07)
SUMMER EDG IP 95001 Mock Inspection Table
Rev.
8.19.2021
CRT Agenda for CACC4
Prevention - Detection Poster
09/07/2023
Shift Manager 1300 Meeting Agenda OAP-100.6 (CACC1)
Leadership CT - INPO21 - 003
All Hands Meeting
09/13/2023
0845 Weekly Site Engineering Meeting Agenda (CACC1)
24 0104 VCS D-15 Driving Performance (CACC1)
11-870-767
Colt Industries, Design Specification for Fuel Oil System
Rev. 1
201714746
VC-23-00022 Remove and install Fuel Oil Piping, Tubing and
supports for replacement on XEG0001A-E
201714881
VC-23-00022 Remove and install Fuel Oil Piping, Tubing and
supports for replacement on XEG0001B-E
201721581
VC-23-00022 Prefab Diesel Generator (XEG0001A-E) Fuel
Piping and Tubing North and South bank (Shop)
Miscellaneous
201721789
VC-23-00022 Prefab two new Fuel Piping Supports for
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
(XEG0001A-E) (shop)
201721798
VC-23-00022 PMT Leakage Check of Piping and Tubing and
Perform Procedure MMP-180.006 (XEG0001A-E)
201722010
VC-23-00022 Machinist to Fab 1" nipples and 1-7/8" Hex
201722258
VC-23-00022 Prefab Diesel Generator (XEG0001B-E) Fuel
Piping and Tubing North and South Bank (shop)
201722259
VC-23-00022 Prefab two new Fuel Piping Supports for
(XEG0001B-E) (shop)
201722262
VC-23-00022 PMT Leakage Check of Piping and Tubing and
Perform Procedure MMP-180.006 (XEG0001B-E)
D-049-VC-SPEC-
000-SP-0337
Engineering Services Specification, SP-337, Pipeline
Specification for Conventional Piping
Rev. 17
D-050-VC-SPEC-
000-SP-0545
Engineering Services Construction Specification, SP-0545,
Pipe Line Specification for Nuclear Safety Class Piping
Rev. 14
DBD-DG
Design Basis Document, Diesel Generator Engine Support
and Control Systems
Rev. 16
CFR 21 Discovery and Evaluation Checklist
2/21/2023
Serial No.24-023
Dominion Energy South Carolina (DESC), Virgil C. Summer
Nuclear Station (VCSNS) Unit 1, 10 CFR 21 Report,
Emergency Diesel Generator Fuel Oil Piping
01/18/2024
VC-PROC-000-
PP-02
Design Engineering Guideline, PP-02, Pipe Support Design
Guide
Rev. 0
VC-SPEC-000-
DSP-544F
ASME Design Specification, Emergency Diesel Generator
Service System Piping and Pipe Supports
Rev. 5
VC-SPEC-000-
Engineering Services Construction Specification, SP-0546,
Diesel Generator Unit
Rev. 4
ER-AA-PRS-1001
Plant Health Committee
Rev. 15
Self-Assessments
Rev. 15
Corrective Action
Rev. 41
Cause Evaluation
Rev. 19
Root Cause Evaluation
Rev. 14
Procedures
Cause Evaluation Methods
Rev. 7
Inspection
Procedure
Type
Designation
Description or Title
Revision or
Date
Level of Effort Evaluation
Rev. 4
Corrective Action
Rev. 13