05000530/LER-2016-002
| Palo Verde Nuclear Generating Station (Pvngs) Unit 3 | |
| Event date: | 12-15-2016 |
|---|---|
| Report date: | 02-10-2017 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications 10 CFR 50.73(a)(2)(v)(D), Loss of Safety Function - Mitigate the Consequences of an Accident |
| 5302016002R00 - NRC Website | |
All times are Mountain Standard Time and approximate unless otherwise indicated.
1. REPORTING REQUIREMENT(S):
This Licensee Event Report (LER) is being submitted pursuant to 10 CFR 50.73(a)(2)(i)(B) to report a condition prohibited by Technical Specification (TS) Limiting Condition for Operation (LCO) 3.8.1, AC Sources — Operating and 10 CFR 50.73(a)(2)(v)(D) for a condition that could have prevented the fulfillment of a safety function of a system needed to mitigate the consequences of an accident. On December 15, 2016, the Unit 3 "B" train (3B) diesel generator (DG) (EllS Code: EK) experienced a significant mechanical failure during the performance of a monthly surveillance test. To support the repairs, Palo Verde Nuclear Generating Station (PVNGS) received two license amendments to extend the LCO 3.8.1, Condition B required action completion time for the 3B DG from 10 days to 62 days. PVNGS Unit 3 entered the extended required action completion time for LCO 3.8.1, Condition B on December 25, 2016. The extended required action completion time expires on February 15, 2017. Repairs are complete and testing is currently in progress.
Based upon preliminary results of the cause investigation, the 3B DG was likely not capable of performing the specified safety function for the entire 7-day mission time within the required action completion time specified in TS LCO 3.8.1, Condition B. Also, on October 4, 2016 and September 7, 2016, the 3B DG was likely not capable of performing the specified safety function for the entire 7-day mission time when the Unit 3 "A" train (3A) DG was inoperable for planned maintenance. Therefore, during time periods when the 3A DG was inoperable, the degraded condition of the 3B DG could have prevented the fulfillment of the safety function of a system needed to mitigate the consequences of an accident. Upon completion of the cause investigation, a supplement to this LER will provide additional information on the period of time the 3B DG's ability to perform the specified safety function was affected.
The failure of the 3B DG resulted in the declaration of an alert emergency classification on December 15, 2016 at 0410 in accordance with Emergency Action Level HA2.1 based on an explosion resulting in visible damage to a safety system required for safe shutdown. This alert was reported in Event Number 52435 pursuant to 10 CFR 50.72(a)(1)(i). The alert was terminated on December 15, 2016 at 0636.
2. DESCRIPTION OF STRUCTURE(S), SYSTEM(S), AND COMPONENT(S):
In each PVNGS unit, the class lE electrical distribution system AC sources consist of the preferred offsite power sources (normal and alternates) and the onsite standby power sources ("A" train and "B" train DGs). Offsite power is supplied from startup transformers (El IS Code: EA) through two engineered safety features (ESF) (El IS Code: JE) service transformers to the two class lE 4.16 kV buses. The design of the AC electrical distribution system provides independence and redundancy to ensure an available source of power to the ESF systems.
The onsite class lE AC distribution system in each unit is divided into redundant load groups (trains). Either of the load groups is capable of providing power for safe plant shutdown and event mitigation so that the loss of any one group does not prevent the safety functions from being performed. Each train has connections to two preferred offsite power sources (normal and alternate) and the related train DG.
The PVNGS DGs are Cooper-Bessemer Model KSV-20 diesel engines designed in a v-type configuration. The DGs are 20-cylinder engines that incorporate 10 pairs of right and left bank power cylinders. Each power cylinder includes a cylinder liner, head assembly, piston, connecting rod, and associated crankshaft throw and bearings. The connecting rods for each pair of power cylinders include one master rod and one articulating rod, each made of forged steel and configured as shown in Attachment 1, Figure of the Connecting Rod Configuration.
Each DG attains full load speed of 600 rpm and is ready to accept sequenced load within 10 seconds following an emergency start signal. The DG is designed to operate during and after a safe shutdown earthquake. The DG operates at a frequency of 60 Hz +/- 1.2 Hz when operating under any load and is rated for 5500 kW continuous output and 6050 kW output for 2 hours out of 24 hours. Each DG unit has its own independent and redundant auxiliary support systems.
3. INITIAL PLANT CONDITIONS:
On December 15, 2016, PVNGS Unit 3 was in Mode 1 (Power Operation) at 100 percent power, normal operating temperature, and normal operating pressure. There were no other structures, systems, or components out of service that contributed to this event.
4. EVENT DESCRIPTION:
On December 15, 2016, the 3B DG experienced a significant mechanical failure during the performance of a regularly scheduled monthly surveillance test. At 0356, the 3B DG was operating at approximately 2500 kW when a low lube oil pressure trip occurred. Operators responded to alarms and identified physical damage to the 3B DG based on oil and metal debris on the 3B DG room floor near the 9-right cylinder. The 3B DG trip functioned satisfactorily, and the DG shutdown immediately following the lube oil pressure trip without operator actions. The Unit 3 control room staff immediately declared the 3B DG inoperable and entered TS LCO 3.8.1, Condition B. The failure of the 3B DG resulted in the declaration of an alert emergency classification on December 15, 2016 at 0410 in accordance with Emergency Action Level HA2.1 based on an explosion resulting in visible damage to a safety system required for safe shutdown. The alert was terminated on December 15, 2016 at 0636. The plant continued to operate at 100% power. There were no automatic or manual safety system responses initiated as a result of the failure. No other systems were impacted.
An investigation was initiated to determine the cause and corrective actions for the failure. As part of the investigation, a damage assessment was performed which identified the following engine internal components had evidence of mechanical overload, plastic deformation, or impact marks:
- Power assemblies #9 right (9R) and #9 left (9L) (includes: pistons, cylinder liners, crankshaft counterweights, master and articulating rods, and associated rod bearings and bushings)
- Power assemblies #8 right (8R) and #8 left (8L) (includes: pistons, cylinder liners, crankshaft counterweights, master and articulating rods, and associated rod bearings and bushings)
- cylinder block and liner in the 9R location
- centerframe in the 9R and 9L locations
- crankcase floor under the #9 and #8 crankshaft throw locations
- lube oil headers and hoses in the #9 and #8 crankshaft throw locations PVNGS initiated a comprehensive disassembly, repair, and reassembly process to restore the DG to manufacturer specifications. Due to the extensive nature of the repair effort, PVNGS submitted two emergency license amendment requests (LARs) to request additional time to allow for completion of the repairs and testing. The NRC staff approved the two LARs that granted one-time extensions of the 10-day LCO 3.8.1, Condition B required action completion time.
1. License Amendment 199 (Agencywide Documents Access and Management System (ADAMS) Accession Number ML16358A676), which extended the required action completion time to 21 days using a deterministic approach 2. License Amendment 200 (ADAMS Accession Number ML17004A020), which extended the required action completion time to 62 days using a risk-informed approach PVNGS has completed the comprehensive disassembly, repair, and reassembly process. The disassembly efforts included removal of the generator, flywheel, engine rotating parts (crankshaft, connecting rods, and pistons), main bearings, cylinder liners, centerframe, head assemblies, turbocharger, intercoolers, and attendant engine instrumentation. This allowed for detailed centerframe and block measurements and inspections as well as crankshaft main bore alignment checks. Repair activities included centerframe metal repairs, machining of some centerframe and block mating surfaces, a centerframe precision line bore, and a base check after line bore. Line boring is an engine machining process to establish alignment of the crankshaft main bearing bores. This re-established crankshaft alignment within the centerframe in accordance with manufacturer specifications. Reassembly activities included replacement or refurbishment of damaged parts including the crankshaft, all master and articulating rods, and bearings. Pistons and cylinder liners were replaced as necessary. Although not damaged, the generator was also refurbished.
The damage assessment activities include non-destructive examination of critical parts, visual inspections, parts recovery, and analysis. A review of past engine analysis data and predictive maintenance histories for all six PVNGS DGs has been performed along with a detailed review of the maintenance history for both Unit 3 DGs. An extensive internal and external operating experience review has also been performed. A metallurgical analysis was conducted to determine the direct cause of the #9 master connecting rod failure. Industry experts were enlisted to assess the condition of the engine bearings. Engineering finite element analyses are being developed to generate a static and dynamic model of the engine and a flaw versus stress sensitivity analysis to aid in characterization of the failure. An engineering analysis is also being developed to evaluate the vibrational data and enhance future vibration monitoring capabilities for the 3B DG.
A detailed retest plan, including post-maintenance and surveillance tests, has been developed considering manufacturer recommendations, regulatory guidance, industry expertise, and operating experience. Testing is currently in progress.
5. ASSESSMENT OF SAFETY CONSEQUENCES:
This event did not result in a challenge to the fission product barriers or result in the release of radioactive materials. The event did not adversely affect the safe operation of the plant or health and safety of the public and did not result in a potential transient more severe than those analyzed in the Updated Final Safety Analysis Report. This event did not result in any personnel injuries or any damage to other safety-related equipment.
Because the 3A DG was removed from service while 3B DG was inoperable, this event represents a safety system functional failure. A final assessment of the safety consequences using risk insights will be provided in a supplement to this LER which will be submitted upon completion of the cause investigation.
6. CAUSE OF THE EVENT:
The cause evaluation is currently in progress. The root cause of this event will be provided in a supplement to this LER.
The direct cause of this event was high cycle fatigue of the #9 master rod ligament. The preliminary cause analysis determined the 3B DG had a misaligned crankshaft bore that resulted from a previous failure of the 3B DG that occurred in 1986. The misalignment of the crankshaft bore resulted in sufficient cyclic stresses at the #9 master rod ligament to initiate and propagate a fatigue crack. Refer to Attachment 1 for a figure of the master rod configuration and the location of the ligament crack.
Evidence indicates the 3B DG crankshaft bore misalignment was due to the previous 1986 connecting rod failure and subsequent repair, which did not include a check of the crankshaft main bore alignment. The other five DGs at PVNGS have not had a connecting rod failure or any other mechanical event that could have introduced such misalignment.
Therefore, the failure mechanisms that caused the 1986 and 2016 3B DG failures are not present in the 3A DG or any other PVNGS DG. The 1986 event is discussed further in the previous occurrences section of this report.
7. CORRECTIVE ACTIONS:
PVNGS has continuously pursued disassembly, inspection, and repair of the damaged 3B DG since initial failure on December 15, 2016. PVNGS established an outage control center to schedule, manage, and oversee the work activities needed for the repairs. Multi-discipline teams were formed to assess the extent of damage, inspect and recover parts, and determine the cause of failure.
The damaged equipment has been repaired or replaced, and the misalignment of the crankshaft bore has been corrected by precision line boring, which restored the crankshaft bearing alignment to within manufacturer specifications.
However, additional analytical work is in progress to validate the preliminary conclusions of the cause investigation team.
Prior to returning the 3B DG to service, post-maintenance and surveillance testing will be performed. The retest plan has received input from industry experts and a third-party independent review.
Upon completion of the cause investigation, the final corrective actions will be provided in a supplement to this LER.
8. PREVIOUS OCCURRENCES:
In 1986, the 3B DG experienced a significant mechanical failure of the #9 master rod during pre-operational testing. A manufacturing process flaw was identified as the root cause of the PVNGS 1986 master rod failure. The articulated rod pin bore on the #9 master rod was initially oversized during manufacturer machining and repaired by the manufacturer using an iron-plating process. The electroplated iron was more brittle than the base material and was found dis-bonded in some locations during the root cause investigation. A fatigue crack originated near the center oil hole of the articulated rod pin bore and propagated through the ligament into the crank pin bore of the master connecting rod precipitating the master connecting rod failure. This high cycle fatigue failure occurred after approximately 100 hours of engine runtime.
There was engine centerframe and block damage which was repaired in situ, and damaged parts were replaced, as necessary. However, the engine repairs did not include crankshaft main bore alignment checks.
As a corrective action and in addition to replacement of the failed #9 master rod, the 3B DG #2 master rod was also replaced during the engine repair as it had received an iron-plating repair. A 10 CFR Part 21 report was issued to the NRC documented under letter ANPP-40058, dated February 9, 1987, for the manufacturer connecting rod iron-plating repair process. Subsequent inspection of the other five PVNGS DGs identified that the #9 master rod in the Unit 2 "A" train DG also had an iron-plating repair, which was replaced in 1987. This issue of iron-plating repairs performed during connecting rod manufacturing was addressed for all PVNGS DGs and did not contribute to the 3B DG 2016 failure.
Attachment 1 Figure of the Connecting Rod Configuration 1. Piston Pin 2. Washer 3. Piston Pin Bolt 4. Articulated Rod 5. Articulated Rod Pin Bolt 6. Washer 7. Dowel 8. Oil Passage 9. Articulated Rod Pin 10. Bushing Dowel Pin 11. Bushing 12. Dowel 13. Connecting Rod Cap 14. Locknut 15. Stud 16. Bearing Shell 17. Master Rod 18. Nut Tightening Sequence 68-76A