05000335/LER-2012-007
| St. Lucie Unit 1 | |
| Event date: | 04-02-2012 |
|---|---|
| Report date: | 08-03-2012 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications |
| 3352012007R00 - NRC Website | |
U.S. NUCLEAR REGULATORY COMMISSION LER NUMBER (6) 2012 - - 0
Description of the Event
On April 2, 2012, St. Lucie Unit 1 was shutdown in Mode 3, HOT STANDBY, returning to service from the SL1-24 refueling/extended power uprate (EPU) outage. At approximately 1440 hours0.0167 days <br />0.4 hours <br />0.00238 weeks <br />5.4792e-4 months <br />, during the monthly Technical Specification (TS) surveillance run, the 1A emergency diesel generator (EDG) [EIIS:EK:DG] was declared inoperable based the 1A2 EDG tripping on high jacket water temperature.
The operators at the 1A2 EDG noted a small amount of water collecting under a conduit inspection box within the 1A2 EDG oil immersion heater terminal box conduit run, and called for electrical maintenance to troubleshoot the trip. Troubleshooting identified that the leakage was caused by the failure of an unused thermal well within the EDG immersion heater [EIIS:EK:DG:EHTR] pressure boundary, allowing a small coolant leak into the EDG alarm and trip circuits that shared the conduit run with the immersion heater. The wetted conduit contained a degraded water jacket temperature switch cable which led to the invalid EDG trip on high jacket water temperature.
The degraded water jacket temperature switch cable and immersion heater were replaced and the 1A EDG was returned to service on April 5, 2012 On April 20, 2012, during an NRC problem identification and resolution (PI&R) inspection exit, the NRC inspection team identified the April 2, 2012, 1A2 EDG coolant pressure boundary leak as a potential greater than green unresolved item. On June 7, 2012, the subsequent root cause had progressed to the point where FPL determined that the EDG was credited as being fully operable during the time period when the immersion heater pressure boundary was in a latent failed state. FPL started the 60-day reportability clock at this time.
FPL completed extent of condition electrical checks on all of the St. Lucie EDG immersion heaters ensuring that no EDGs were susceptible to the same failure mechanism that affected the 1A2 EDG.
Cause of the Event
The immersion heater pressure boundary failure was most likely initiated by operation with insufficient fluid coverage. The immersion heaters are typical electrical heating elements, constructed with a 5 inch, 150 pound SA-105 steel flange, six 0.475" diameter Incoloy 800 heating elements, and an Incoloy 800 thermowell. At St.
Lucie, the thermowell is not used. Overheating the heater by uncovering or operating in a vapor space caused a breach in the heated region of the uppermost heater element sheath. Subsequently, water intrusion caused degradation of the magnesium oxide dielectric surrounding the uppermost heater element nichrome wire and initiated electrochemical dissolution of the non-heated or cold wire end region of the sheath.
Continued attack eventually resulted in degradation of the adjacent heater element sheaths and central thermowell, thereby causing a breach of the immersion heater pressure boundary. A change in coolant chemistry, specifically rising iron and ammonia levels, resulted from the electrochemical dissolution mechanism.
The most important factor that contributed to the immersion heater pressure boundary failure was inadequate chemistry monitoring procedures. Although chemical testing of the EDG coolant is performed as part of health monitoring, no clear guidance or action levels exist for the chemistry samples obtained. The chemical monitoring program for the EDG coolant identifies jacket water iron and ammonia as potential LER NUMBER (6) 2012 - - 0 indicators of heater failure; however chemistry procedures designate these parameters as diagnostic without specific limits.
Analysis of the Event
The sequence of events related to the failure of the immersion heater pressure boundary is as follows:
On November 30, 2011 the EDG 1A2 was successfully operated for a 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> surveillance run. On December 7, 2011 the 1A EDG was removed from service for the "A" train refueling outage maintenance window and was returned to service on January 8, 2012 During the maintenance window the 1A2 EDG jacket water system was drained and refilled. FPL concluded that the immersion heater was intact (e.g., fully operable) up until January 5, 2012, at which time phase-to-phase resistance checks indicated that the heater elements were intact. The initial heater damage is believed to have occurred immediately following the restoration of the system because, during this period, the flow rate across the heaters would be limited to that caused by the natural circulation. The combination of the low flow rates and the increased operating cycle likely caused the development of a steam space surrounding the uppermost heater element. The resultant overheating caused a breach in the uppermost heater element sheath, allowing water intrusion into the surrounding magnesium oxide dielectric.
On January 27, 2012, the Operations logs identified that the 1A2 EDG immersion heater breaker had tripped and a condition report was initiated. FPL concludes this trip was caused by the uppermost heater element's nichrome wire shorting to ground.
On February 14, 2012, following the monthly surveillance run of the 1A2 EDG a chemistry sample was taken. This chemistry sample noted an increase in the ammonia and iron but attributed this to the recent maintenance and establishment of the new normal values; however, FPL now concludes that these sample results were caused by the electrochemical dissolution of the heater element sheaths and thermowell.
On March 11, 2012, the 1A2 EDG was again started for the conduct of the monthly surveillance and ran for approximately two and one half hours. Chemistry samples taken on this date noted that the coloring of the jacket water had changed and that significant levels of ammonia and iron were present in the sample. A condition report was generated to document the chemical issues and a work order was generated to replace the immersion heater on the 1A2 EDG.
On April 2, 2012, the decision was made to perform the monthly surveillance earlier than scheduled to facilitate the unit's return to service following the EPU outage.
Approximately 10 minutes after starting, the EDG tripped on high jacket water temperature.
Operations reported that the immersion heater breaker was found in the tripped condition. Electrical maintenance responded and verified the small amount of water collecting at the conduit inspection box area. Electrical maintenance obtained permission and removed the conduit covers on the inspection covers along the conduit run from the heaters to the control panel and observed a small amount of leakage described as a weep from the conduit joint to the inspection box. This confirmed that there was some wetting of the wires in the conduit. The electrical maintenance supervisor requested to open the control panel to inspect and that a tag out be applied to allow for additional troubleshooting. When the EDG clearance was authorized to be hung the operator in the field inadvertently placed the immersion US.NUCLEARREGULATORYCOMMMSION LER NUMBER (6) 2012 heater breaker in the on position at which time a loud "pop" was heard, the breaker tripped free, and a rapid flow of water was observed from the conduit inspection covers. The water flow lasted approximately 10 to 15 minutes at which time it returned to a weep. Although the breaker miss-position event was documented in a condition report, the significance of the event was not addressed. When the full scope of the post-event conditions was verified, FPL determined that the gross coolant loss assumed to occur coincident with the EDG trip on high jacket water temperature actually occurred when power was inadvertently applied to the partially flooded 1A2 immersion heater terminal box; the loud "pop" heard was the electrical arc that caused the catastrophic failure (i.e., blowout) of the thermowell. The initial weepage noted but not reported directly after the EDG trip would be consistent with the immersion heater failure mechanism.
Analysis of Safety Significance This condition is reportable under 10 CFR 50.73(a)(2)(i)(B) as operation of the facility in a manner prohibited by Technical Specifications. Based on the failure mechanism, the EDG was susceptible to this failure for a time period in excess of the allowed outage time. This condition is also reportable under 10 CFR 50.73(a)(v)(D) as the loss of a safety function necessary to mitigate the effects of an accident.
The inoperable 1A EDG was credited as the protected EDG during 1B EDG maintenance windows.
The EDGs at the St. Lucie plant each consist of two tandem mounted diesel engines driving a common electrical generator. Each of the two engines has a self-contained cooling system which consists of a forced circulation cooling water system which cools the engine directly, and an air cooled radiator system which removes the heat from the cooling water. The main components for the diesel engine cooling system include a cooling water pump (internal to the engine), temperature control valve, radiator, radiator fan, and expansion tank. However, in order to maintain the standby EDGs in a warm, ready to start condition, immersion heaters are energized to maintain engine cooling water temperature between 125 to 155 degrees Fahrenheit by circulating warm water through the lube oil heat exchangers. Auxiliary lube oil pumps circulate the warmed lube oil continuously.
The failure mechanism resulted in the relatively slow degradation and eventual failure of the unused thermowell. Once the thermowell was breached, EDG coolant would weep from the immersion heater pressure boundary, wetting the heater element connections in the terminal box, and result in tripping the immersion heater breaker.
This slow, yet un-quantified leakage, would affect the mission time of the EDG as the coolant level dropped in the system. Once the level got low enough (judged to take several hours) to affect the internal water pump and air bind the system, the EDG temperature would rapidly increase and result in loss of the machine.
There was no significant impact to the health and safety of public for several reasons. First, St. Lucie Unit 1 was in a long refueling outage during the window where the 1A2 EDG was susceptible to the failure mechanism. FPL judges that the EDG could have run for a period of time after the initial breach of the immersion heater pressure boundary, and non-safety makeup water is available to refill the radiator.
The downstream conduit run was reviewed and at the expected leak rate there was no circuitry that could have tripped the EDG while running in the emergency mode (note the high jacket water temperature trip is bypassed when the EDG is running in the emergency mode) In the unlikely event that the lA EDG failed due to the immersion heater pressure boundary leak, the 1B EDG would be available. In case the 1B EDG was US.NUCLEARREGULATORYCOMMMSION DOCKET LER NUMBER (6) NUMBER (2) 2012 unavailable due to failure or being out of service, was available.
the station blackout cross-tie Second, the St. Lucie low mode off normal procedures would direct the operators to use gravity feed from the refueling water tank to the reactor coolant system in the unlikely case where all electric power is postulated to be lost. These proceduralized and trained actions would prevent core uncovery. A probabilistic safety assessment was performed for this condition that conservatively assumed that the EDG failure window extended to November 30, 2011, at the completion of the last TS surveillance 24-hour run. For the plant conditions that existed at the time the assessment results show that the conditional core damage probability for the loss of power scenario was less than 1 X 10' (i.e., green), and therefore not risk significant.
Corrective Actions
The corrective actions listed have been entered into the site corrective action program (CAP) the CAP.
Any changes to the actions below will be processed in accordance with 1. Corrective Actions associated with the failed immersion heater pressure boundary:
All St. Lucie Unit 1 and 2 EDG oil immersion heaters were verified intact by phase-to-phase resistance checks and phase-to-ground megger checks.
Additionally, visual inspections were performed on accessible heater junction boxes to confirm the health of the heater pressure boundary.
Replacement heaters with sealed terminal ends and without thermowells are identified, ordered, and scheduled to be replaced for both St. Lucie Unit 1 and 2 The PMS for testing the immersion heaters were revised to include adding phase- to-ground meggers as well as the existing the phase-to-phase resistance checks.
The eSoms component ID for the immersion heater breakers was revised to include the requirement to ensure jacket water level is within the procedural requirements of the EDG prior to energizing the heaters by shutting the breaker.
2. Chemistry procedures were revised to establish limits for EDG jacket water iron and ammonia based on this event.
= LER NUMBER (6) 2012 Similar Events A search was conducted for the failures of the immersion heaters at St. Lucie and identified an event in 2003 In 2003 the 1B2 immersion heater was identified leaking and subsequently replaced. The failure mechanism noted in the heater repair was identified as similar in nature and the heater was replaced. Corrective actions were developed to replace the heaters and no other actions taken.
Failed Components
Component: Unit 1 Emergency Diesel Generator System (System 59), 1A2 Engine Immersion Heater in the EDG Cooling Water System (tag ID: DG ENG 1A2, IMRSN HTR) Manufacturer: Chromalox Part Number: P/N 155-500725-668, Cat. No. TMI-6E4XXE4 208V 1-3PH 15KW