ML20153F383

Special Rept SR-88-003-01:on 850801-880404 Several Failures of Emergency Diesel Generators to Start Occurred.Caused by Actuation of Trips Which Bypassed in Emergency Mode.Trip Sensor Replaced

ML20153F383
Person / Time
Site:	Waterford
Issue date:	05/04/1988
From:	Carns N LOUISIANA POWER & LIGHT CO.
To:	NRC OFFICE OF ADMINISTRATION & RESOURCES MANAGEMENT (ARM)
References
SR-88-003-01, SR-88-3-1, W3A88-0047, W3A88-47, NUDOCS 8805100312
Download: ML20153F383 (10)

Text

..

Ref:

10CFR50.36(c)(2)

W?b LOUISIANA P O W E R & L I G H T! WATERFORD 3 SES

• PG BOX B e KILLONA. LA 70066-0751 maac May 4, 1988 W3A88-0047 A4.05 QA U.S. Nuclear Regulatory Commission ATTENTION:

Document Control Desk Washington, D.C.

20555

SUBJECT:

Waterford 3 SES Docket No.

0-382 License No. NPF-38 Reporting of Special Report Attached is Special Report Number SR-88-003-01 for Waterford Steam Electric Station Unit 3.

This Special Report is submitted per 10CFR50.36(c)(2) and Technical Specifications 3.8.1.1 and 6.9.2.

Very truly yours, j/

('%-

N.S. Carns Plant Manager - Nuclear NSC/WEM:rk Attachment cc:

R.D. Martin, NRC Resident Inspectors Office, INPO Records Center (J.T. Wheelock), E.L. Blake, W.M. Stevenson, D.L. Wigginton hgk51 DbC b5 N EQUAL OPPORTUNITY EMPLOYER" 0 82 A

S DCD

SPECIAL REPORT SR-88-003 4-Invalid Failures of Emergency Diesel Gene.ators Due to Spurious Operation of Trips Which are Bypassed in the Emergency Mode i

INTRODUCTION From August 1, 1985, to April 4, 1988, there were several failures of the l

Emergency Diesel Generators (EDGs) to start due to actuation of trips which are bypassed in the emergency mode. There were ten shutdowns due to actuation of l

the Turbocharger Low Lube Oil Pressure trip, one shutdown due to actuation of the Generator Fault trip, one shutdown-due to an air leak in the Pneumatic Control System (PCS), and two shutdowns due to actuation of Generator Bearing High Temperature trips.

The Turbocharger Low Lube Oil, Generator Fault, and Generator Bearing High Temperature trips actuate the Safety Trip Valve in the PCS to shut down the diesel. All the trips associated with the PCS are bypassed in the emergency mode. There were no failed starts in the emergency mode (i.e., valid failures) which have not been previously reported and corrected.

In each case, observation of engine parameters and subsequent troubleshooting demonstrated that the trip actuation was due to a malfunction of the trip mechanism rather than an actual out-of-range parameter. Thus, these events were not considered valid tests or failures in accordance with Regulatory Guide 1.108, Revision 1, Section C.2.e(2),

and there was no safety significance to these events.

1

..m.. _. _. _. _, _,.. _,.. - - -,..... _....,-, _,,,

NARRATIVE From August 1, 1985, to April 4, 1988, there were several failures of the l

Emergency Diesel Generators (EDGs) to start due to actuation of trips which are bypassed in the emergency mode. These trips cause the Safety Trip Valve in the Pneumatic Control System (PCS) to lift, venting air pressure from the Pneumatic Fuel Control Valve.

If this occurs when the diesel is running in the Test Mode, the diesel will shut down.

In the emergency mode, the Emergency Mode Fuel Control Solenoid Valves bypass the Pneumatic Fuel Control Valve and l

lock the fuel "on" to prevent a trip.

Only diesel "overspeed" or "generator differential" can cause a shutdown in thia mode.

l t

i At 1350 he<trs and 1409 hours0.0163 days <br />0.391 hours <br />0.00233 weeks <br />5.361245e-4 months <br /> on August 1, 1985, with the plant in cold shutdown,

(

(mode 5) EDG 'B' tripped during performance of procedure OP-903-068, "Emergency l

Diesel Generator Operability Verification," due to a broken connector on the EDC 'B' Connecting Rod Bearing High Temperature Switch, EG-ITS-3002BS.

The Connecting Rod Bearing High Temperature Detectors consist of spring-loaded fuse rods secured in place by a low melting-point alloy. When the temperature of a connecting rod bearing reaches 197 degrees Fahrenheit, the metal alloy melts, releasing the fuse rod.

The fuse rod, under spring pressure, unseats a l

ball valve thus venting off control air to shut down the diesel.

The EDG had just been started manually in the test mode when it tripped.

It was determined by examination of the trip device that no high temperature condition on the connecting rod bearings existed. The Connecting Rod Bearing Temperature Detector had not melted from a high temperature condition, but the Connecting Rod Bearing High Temperature Switch had broken. Tte switch was subsequently repaired.

The cause of the broken switch was attributed to personnel who had stepped on the associated instrumentation. The diesel was declared operable at 0552 hours0.00639 days <br />0.153 hours <br />9.126984e-4 weeks <br />2.10036e-4 months <br /> on August 3, 1985.

Since this trip is bypassed l

l in the emergency mode, there was no safety significance to this event and this 1

was not considered a valid test or failure.

l l

l 2

s.

\\

s

=

At 1551 hours0.018 days <br />0.431 hours <br />0.00256 weeks <br />5.901555e-4 months <br /> on January 30, 1936 with the plant operating at 100% power, EDG 'B' shutdown on actuation of the Turbocharger' Low Lube Oil Pressure trip during performance of OP-903-068. At 1608 hours0.0186 days <br />0.447 hours <br />0.00266 weeks <br />6.11844e-4 months <br /> on January 30,-1986, EDG 'B' was declared inoperable and the action statement of Limiting Condition for t

Operation (LCO) 3.8.1.1 was entered.

EDG 'A' was started at 1627' hours and

'A' train equipment was verified operable to comply with the LCO.

The turbocharger bearing was inspected and did not exhibit excessive wear. The

'B' EDG was satisfactorily retested and declared operable at 2354 hours0.0272 days <br />0.654 hours <br />0.00389 weeks <br />8.95697e-4 months <br /> on January 30, 1986.

Subsequent investigation under Condition Identification Work Authorization f

(CIWA) 11648 described several discrepancies with the

'B' EDG.

EDG lube oil was slightly diluted with fuel oil, the turbocharger was suspected of windmilling due to an imbalance of the Heating, Ventilation, and Cooling (HVAC) system in the room, and high crankcase pressure existed. Work performed per CIWA 25149 found the HVAC supply and exhaust f*"As were out of tolerance resulting in approxim'ately 20% more air being teaaved than supplied. According to the EDG manufacturer, low air pressure in the EDG room induces air flow from the EDG exhaust, through the turbochargers, and into the room via the ventilated crankcase, causing the turbocharger to' rotate while the EDG is at rest. Ventilation dampers were adjusted to bring these flows back into tolerance and prevent the EDG turbocharger from windmilling.

This also eliminated the high crankcase pressure which had been caused by the windmilling turbocharger. Lube oil and fuel oil filters were replaced under CIWA 25441.

A leaking fuel injector was replaced under CIWA 25738. The turbocharger bearing was inspected per CIWA 24737 and did not exhibit excessive wear.

These conditions would not have impacted the ability of the diesel to perform its design function.

A definite cause for the diesel trip could not be determined.

Since subsequent starts of EDG 'B" were successful, the diesel generator was declared operable at 2354 hours0.0272 days <br />0.654 hours <br />0.00389 weeks <br />8.95697e-4 months <br /> on January 30, 1986, after performing OP-903-068.

Since this trip is bypassed in the emergency mode and inspections revealed no abnormalities in the diesel, the shutdown was attributed to spurious operation of this trip. Thus, there was no safety r.ignificance to this event, and this was not considered a valid test or failure.

3

At 0446 hours0.00516 days <br />0.124 hours <br />7.374339e-4 weeks <br />1.69703e-4 months <br /> on March 7, 1986, with the plant operating at 98% power, EDG 'A' tripped after running fully loaded in the test mode for 42 minutes. The EDG was restarted at 0508 hours0.00588 days <br />0.141 hours <br />8.399471e-4 weeks <br />1.93294e-4 months <br /> but could not be stabilized at full load due to erratic load sharing in parallel.with the grid and was subsequently secured and declared inoperable.

The applicable action statement of Technical Specification (TS) 3.8.1.1 was satisfied within one hour.

Shortly before the unit tripped, it was in the "Test Mode" (Manual Local Control),

loaded to 4.4 MW (governor in droop mode) in parallel with the grid when the load started to hunt.

The "Generator Fault" annunciator was actuated, probably due to a trip of the reversa power relay.

There are two other causes that could have tripped this relay: generator excitation fault or a governor system fault.

i Vendor personnel reported on site on March 7, 1986, to determine if there was a governor stability probler on the 'A' EDG.

The diesel was run with no apparent problema, with the governor system.

On March 8, 1986, an attempt was made to duplicate the problem, but the EDG ran satisfactorily with no noted j

problems. A review of EDG operating data indicated no apparent degradation of l

the diesel due to the spurious trip. Upon completing OP-903-068, EDG 'A' was i

declared operable at 1310 hours0.0152 days <br />0.364 hours <br />0.00217 weeks <br />4.98455e-4 months <br /> on March 8, 1986.

This failure was classified as invalid since the diesel ran for less than the one hour required for a valid test by Regulatory Guide 1.108, section C.2.e(3), and the trip was attributed to malfunction of the paralleling and synchronizing circuitry, which is not used in the emergency mode.

i 4

l

1 At 1323 hours0.0153 days <br />0.368 hours <br />0.00219 weeks <br />5.034015e-4 months <br /> on May 8, 1987, with the-plant operating at 100% power, EDG 'A' failed to start satisfactorily while performing OP-903-068 due to an air leak from valve EGA-407A.

EDG 'A' was declared inoperable at 1330 hours0.0154 days <br />0.369 hours <br />0.0022 weeks <br />5.06065e-4 months <br /> and the applicable action statement of LCO 3.8.1.1 was entered. EDG 'B' was started at 1420 hours0.0164 days <br />0.394 hours <br />0.00235 weeks <br />5.4031e-4 months <br /> to comply with the action statement of the LCO.

I i

EGA-407A is a three-way valve in the PCS that directs air from the left or right banks of the Control Air Shutoff Panels to the Safety Shutdown Panel.

The Control Air Shutoff Panels are supplied by the Starting Air System.

The

)

Safety Shutdown Panel contains the Safety Trip Valve which vents off control air when a trip device is actuated. EDG-407A had developed a large air leak in the gasket which was sufficient to drop control air pressure and interrupt I

the manual mode starting sequence.

The leak was repaired and the valve was verified to operate properly. The diesel was declared operable at 1715 hours0.0198 days <br />0.476 hours <br />0.00284 weeks <br />6.525575e-4 months <br /> on May 10, 1987, after performing OP-903-068. The diesel was tested in the "emergency" mode before the leak was repaired with satisfactory results.

Since this failure was due to spurious operation of the Safety Trip System, which is bypassed in the Emergency Mode, there was no safety significance to this event, and it was not considered a valid test or failure.

I 1

i l

i j

5

1 At 1302 hours0.0151 days <br />0.362 hours <br />0.00215 weeks <br />4.95411e-4 months <br /> on June 22, 1987, with the plant operating at 100% power, EDG

'A' shut down on "Turbocharger Low Lube Oil Pressure" and ",Tacket Water Pressure Low."

EDG 'A' was being started to comply with the action statement of TS 3.8.1.1 which requires a successful start of the operable EDG every 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> when one EDG is out-of-service. The diesel was successfully started in the emergency mode 13 minutes later at 1315 hours0.0152 days <br />0.365 hours <br />0.00217 weeks <br />5.003575e-4 months <br /> and functioned as designed. At 1840 hours0.0213 days <br />0.511 hours <br />0.00304 weeks <br />7.0012e-4 months <br /> on June 23, 1987, EDG 'A' again failed to start in the test mode with the same trips indicated. Again, the diesel was successfully started nine minutes later in the emergency mode and functioned as designed. The cause of these trips was attributed to a setting of 10 seconds on a lockout bypass relay delay timer which did not allow sufficient time for the air trip system pressure to rise above the shutdown setpoint during the manual diesel start sequence.

Concurrence was obtained from the manufacturer, Cooper-Bessemer, to increase the setting of this timer to 15 seconds.

Station Modification (SM) 1961 imple-mented this change for both diesels.

Since the diesel was successfully started in the emergency mode in both cases and the shutdowns were due to actuation of a trip mechanis.n which is bypassed in the emergency mode, there was no safety significance to these events, and they were not considered valid tests or failures.

At 1938 hours0.0224 days <br />0.538 hours <br />0.0032 weeks <br />7.37409e-4 months <br /> on 1.ngust 15, 1987, with the plant operating at 100% power, the

'A' EDG would not start in the test mode during performance of procedure OP-903-094, "'ISFAS Subgroap Relay Test - Operating", due to a high temperature j

trip on the e,utboard bearing. The diesel did start satisfactorily in the emergency mode.

Operators verified that the outboard bearing temperature was normal.

6

Maintenance personnel discovered that a loose screw from terminal 174 in a junction box mounted on the diesel engine had fallen cut leaving the wire I

connected to the outboard bearing trip switch laying loosely against its terminal. V!brations caused the. connection to break and tripped the diesel.

Similar connections on the 'B' EDG were inspected and'no discrepancies were found, so the loose wire in the 'A' EDG was classified as an isolated case.

This shutdown was therefore due to spurious operation of a trip which is bypassed in the emergency mode, so it was not considered a valid test or failure. Work Authorization 01009095 was authorized to install internal tooth lock washers at wire-end terminal board connections of engine-mounted junction boxes. This action, to be completed during the current refueling outage.

l should preclude recurrence of this event.

Between October 19, 1987, and March 8,1988, there were six trips of the ' A' EDG during performance of OP-903-068.

Each shutdown was actuated by the "Turbocharger Low Lube Oil Pressure" trip which is bypassed in the emergency mode. Prior to the diesel engine overhaul during the second refueling outage, the worst case credible cause of the spuricus actuations of this trip was considered to be small air leaks in the PCS which contributed to the slow buildup of air pressure and thus resulted in spurious trips during manual i

starts.

i l

At 0315 hours0.00365 days <br />0.0875 hours <br />5.208333e-4 weeks <br />1.198575e-4 months <br /> on April 4, 1988, while in cold shutdown (mode 5), EDG "A" tripped 1

on "Turbocharger Low Lube Oil Pressi're".

The EDG was started just prior to placing it out-of-service for overhaul during the current refueling outage.

This test was performed to allow an inspection of operating diesel character-11 tics in order to verify all known discrepancies and document any previously i

4 undiscovered discrepancies prior to the overhaul, as well as warming up the engine to allow checking certain clearances during the initial inspection.

The diesel was successfully started at 0319 hours0.00369 days <br />0.0886 hours <br />5.274471e-4 weeks <br />1.213795e-4 months <br /> in the manual mode.

7

--v-

_ - r r,

,_,--3

,,.y,,,

,,,,,.,,,_,__,,,_,._w

,,,,,,,,,__m,,_,

i Inspection and maintenance activity on the "A" EDG during the current refueling outage showed that the trip setpoint of the Turbocharger Low Lube Oil Pressure trip seasor, EGL-IPDEV-3018A, had drifted to 5.15 psig. The sensor is calibrated to trip at 4 psig +0.5/-0.0 psig and reset by 5 psig.

Since the Turbocharger Lube Oil System Pressure Regulating Valve, EGL-217A, was set for 5 psig, the as-found setpoint of the low pressure trip sensor may not have allowed the 1

sensor to reset during the diesel starting sequence.

Additionally, examination of the ball check valve, EGA-422A, discovered a particle on the check valve ball which prevented the valve from fully opening.

This valve supplies control air pressure to reset the pressure switch, EGL-IPS-3017A. When the trip sensor, EGL-IPDEV-3018A, trips, control air pressure supplied by ECA-422A is vented through the sensor, causing the pressure switch to trip the Pneumatic Fuel Control Valve and shut down the diesel.

The air flow restriction in the check valve resulted in insufficient control air pressure to reset the pressure switch. The root cause of the Turbocharger Low Lube Oil Pressure trips has therefore been reclassified as due to a combination of the faulty check valve and the drifting low pressure trip sensor.

This is a less serious condition than that initially identified due to its lack of simultaneous effects on other trip parameters.

Check valve EGA-422A has been replaced. The trip sensor was replaced and calibrated.

Pressure regulation of the Turbocharger Lube Oil System was increased from 5 psig to 6 psig to provide additional margin to the low pressure trip setpoint.

This maintenance was part of a major overhaul of the

'A' EDG PCS which was completed on April 17, 1988.

This overhaul is part of a preventive maintenance program for the diesel which includes one, five, and ten year repetitive tasks.

The ten year tasks include maintenance, replacement, and calibration of PCS Instrument and Control (I&C) equipment including safety shutdown devices.

The yearly and five year tasks are being developed to include 1

the less extensive maintenance needed more frequently than the ten year overhaul.

This preventive maintenance program supplements existing periodic calibration j

requirements on diesel I6C equipment. The diesel was satisfactorily started in the manual mode after completion of this overhaul.

8

In all of the events attributed to spurious operation of the "Turbocharger Low Lube Oil Pressure" trip, the EDG tripped when first started in the test mode and started successfully on a second attempt.

A test was performed on March 3,1988, where the "A" EDG was successfts'.ly started in the emergency mode, secured, and then successfully started in the test mode. Throughout the period in question, the EDG would have started and run in the emergency mode when required to fulfill its intended safety function. Thus, there was no safety significance to these events and they were not considered valid tests or failures.

A thorough examination of the diesel control systems and the events during which the diesel is required to operate demonstrates that there is no increase in either the probability or consequences of any accident due to this condition.

It is evident, however, that there is some increase in core damage probability if the diesel is taken out-of-service for the time allowed by the action statement of the Limiting Condition for Operation to repair the condition.

It was therefore decided, in order to minimize the overall risk of adverse consequences, that the diesel should not be taken out-of-service until the plant was scheduled to be in a mode in which one diesel may be placed out-of-service for maintenance.

1 All events described in this report are classified as Invalid Tests.

There has been only one failure during a Valid Test of the 'A' EDG and none of the

'B' EDG since the operating license was issued. The failure of the 'A' EDG occurred on May 23, 1986, and was reported in Special Report 86-004.

The surveillance test interval required by TS Table 4.8-1 has always been "at least once per 31 days" since issuance of the operating license.

PLANT CONTACT D.E. Baker, Event Analysis Reporting & Response Manager, 504/464-3133 9

. - -