05000361/LER-2008-004

LER-2008-004, Malfunctioning stator water cooling system check valve causes reactor trip

Docket Number
Event date:	6-05-2008
Report date:	8-1-2008
Reporting criterion:	10 CFR 50.73(a)(2)(iv)(A), System Actuation
3612008004R00 - NRC Website

Event Date: June 5, 2008 Reactor Vendor: Combustion Engineering Mode: Mode 1- Power Power: 97 percent

Background:

The Main Generator stator water cooling system [TJ] removes heat from stator windings and excitation rectifiers [TL] and includes two parallel cooling water pumps and two heat exchangers. The system is operated with one pump running and one pump in standby. The stator operating temperature is maintained by a minimum flow rate of approximately 595 gpm to the windings and the main exciter static rectifiers receive a minimum flow of 4 gpm. A sustained decrease in either rate results in an automatic main generator trip [TB] and a subsequent reactor trip.

Description of Event:

On June 5, 2008 (Event Date), with Unit 2 at approximately 97 percent power, plant personnel began a monthly test of the stator water cooling system. In accordance with the test procedure, Operators swapped the stator water pumps by starting the pump that was in standby and then stopping the operating pump. The operator in attendance reported that a check valve that prevents backflow through the non-running pump shut louder than expected. Three alarms were received in quick succession indicating a reduction in stator water flow and rectifier water flow. Approximately two minutes later, the Main Generator [TA] tripped from a "Rectifier Low Flow." At approximately 2256 PDT, Unit 2 reactor automatically tripped on a "Loss of Load" signal. On June 6, 2008 at 0148 PDT, Southern California Edison reported this occurrence to the NRC in accordance with 10CFR50.72(b)(3)(iv)(A) (NRC Event Log # 44273).

Cause of the Event:

SCE's cause evaluation concluded that a pump discharge check valve momentarily remained open following the stator water pump swap and then slammed shut. The resulting pressure spike was sufficient to lift one or both of the relief valves on the stator water heat exchangers. The discharge from the relief valve(s) reduced stator cooling water flow through the rectifier and resulted in a main generator trip and a reactor trip.

Inspection of check valve S21413MU055 revealed that the disc was contacting the valve body on the sides. Over time, this contact between the disc and valve body resulted in the disc briefly becoming stuck and slamming closed on reverse flow during system testing.

Corrective Actions:

SCE has completed the following corrective actions:

1. SCE repaired the malfunctioning check valve.

2. SCE inspected the check valve on the parallel pump revealing no similar wear.

3. The heat exchanger relief valves were removed and replaced by bench tested spares.

4. The removed cooler relief valves were tested and performed as expected.

5. SCE reviewed recorded temperatures on the outlet of the stator and determined that very little temperature changes were evident during the transient.

6. SCE performed the generator stator cooling water flow trip test, off-line before unit re-start, with no abnormal results.

The corresponding check valves at Unit 3 have not yet been inspected because Unit 3 was operating at full power at the time of the Unit 2 trip and remains at full power at the time of this report. SCE has suspended monthly stator water cooling flow testing at Unit 2 and Unit 3 pending further assessment.

Safety Significance

This event was bounded by the Updated Final Safety Analysis evaluation of a loss of load event.

Therefore, the significance of this event was minimal.

Additional Information:

The SONGS Plant Computer System (PCS) is a non-safety related system used to monitor and record plant parameters. During the review of the PCS-generated Sequence of Events (SOE) report after the June 5, 2008 Unit 2 trip, two anomalies were noted. The SOE indicated that six of eight reactor trip circuit breakers (RTCBs) opened before the initiating loss of load trip signal. SCE considers this indication to be non-credible. The SOE also indicated that the remaining two of eight RTCBs opened 149 milliseconds after the first six. SCE concluded the integrated clock in the SOE printed circuit card was unreliable and replaced the card. As a precaution, the relay that initiates the opening of the SOE-reported slow responding RTCBs was replaced and tested with satisfactory results.

Following this reactor trip, both main feedwater pumps exhibited undesired oscillation in output flow.

Operations personnel secured one pump and took manual control of the second pump. SCE determined the feedwater pump oscillations were caused by a recently installed digital feedwater control system.

SCE has changed a proportional gain setting in the digital feedwater control system to dampen this phenomenon.

In the past three years, there have been no reported occurrences related to degraded check valves or other plant trips caused by a loss of stator cooling water flow.