ML20006D961
| ML20006D961 | |
| Person / Time | |
|---|---|
| Issue date: | 02/28/1990 |
| From: | Wegner M NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| To: | |
| Shared Package | |
| ML20006D960 | List: |
| References | |
| TASK-AE, TASK-E90-01, TASK-E90-1 AEOD-E90-01, AEOD-E90-1, NUDOCS 9002150312 | |
| Download: ML20006D961 (19) | |
Text
f
~
AE00 ENGINEERING EVALUATION
! UNIT:'
Multiple EE REPORT N0.: AE0D/E90-01
' DOCKET NO.: Multiple DATE:
February,1990
- LICENSEE:
Multiple EVALUATOR / CONTACT:
M. Wegner NSSS/AE:
Multiple / Multiple 1
SUBJECT:
FAILURES OF ELECTRICAL SUPPLY AND POWER GENERATION EQUIPMENT WHICH DISRUPTED PLANT FUNCTION AT NUCLEAR POWER PLANTS
SUMMARY
This study was initiated because of the occurrence of several events in early 1989 involving major electrical equipment or offsite power problems. A search for similar events identified 79 events in the past 18 months. Twenty-two of those events were considered to be of major interest and are described in this
. report.
.I The issues addressed in this report concern disruptions of normal station-auxiliary alternating current (AC) power supply caused by grid, switchyard, and non-1E electrical system problems and major electrical equipment failures.
Failures in electrical supply or power generation equipment can impact safety equipment-in that they often cause scrams or require unit shutdown, they may cause engineered safety features systems to operate, they frequently cause interruptions of offsite power supply to station auxiliaries, they occasionally cause fires, and they complicate post-scram operations by rendering equipment unavailable or unreliable..
Interruptions of offsite power to 1E busses, while they are contributors to the risk of a station' blackout, generally cause transfer of the busses to the emergency diesel-generators (EDGs); while interruptions of offsite power to non-1E busses cause scrams, loss of forced circulation, loss of preferred heat sink, and. loss of service / instrument air.
An AE0D Engineering Evaluation, AE00/E905, " Electrical Bus Bar Failures", and an NRC information notice,'IN 89-064; and an ongoing AE0D study on main trans-former_ failures address the major equipment failures.
Fast bus transfer failure is also being addressed in an on-going AE0D study. The operating
-experience, particularly maintenance, surveillance, and procedural problems, should be utilized by NRR in developing procedures for the upcoming team inspections.
DISCUSSION
-From January 1 to February 7,1989, there were reports of a breaker and a trans-
).
former explosion and fire, a partial loss of offsite power, another breaker fire, another partial loss of offsite power, a fire in the main generator, a bus bar short, loss of offsite power, a second partial loss of offsite power at a startup plant, a loss of offsite power involving two units, turbine high vibration, another breaker fire, and a fire in the turbine area.
The quantity of the reports was large enough to warrant a look at the report data bases to determine the scope of the problem and an analysis of the safety impact.
At that time, a preliminary search was performed. All events involving electrical supply equipment up to (but not including) the startup transformer were included as well as events involving power generation equipment. The main turbine and generator were included but not their control systems, instruments, valves, or other attendant systems. Additionally, the initiator must also have caused or contributed to a complicated transient in which there was a turbine trip, reactor scram, engineered safety features (ESF) actuation, and/or other major complications.
Several events of significance were found and this study was undertaken.
The NRC's document control system (DCS) was searched for licensee event reports I
(LERs) which involved a turbine trip or loss of the main generator in the 12-month period from February 7,1988, to February 7,1989. Twelve significant events were found which showed the impact of electrical disruptions on plant function. A more thorough search of LERs for the period using the Sequence coding and Search System (SCSS) found 45 LERs.
From daily reports and 10 CFR 50.72 reports made from February 7,1989, through May 31, 1989, additional reports were selected.
From all these data bases, a total of 79 event reports involving interruptions of offsite power supply to the station auxiliaries or major electric equipment l
problems was identified. Twenty-two of these, considered to be events of interest, are detailed in the following section.
Events involving interruptions ofoffsitepowertotheplant(eitherpartialortotallossofoffsitepower) are shown in Table 2.
Appendix 1 is a tabulation of all the other events.
Search Strategies:
The events which are discussed were selected from the LERs written from 02/07/88 to 05/31/89. These events involve failures in electrical supply or power generation' equipment which disrupted plant function. These events were selected from a larger number of events found in a variety of Boolean searches of the DCS. The most fruitful search was DTC (document t T (term) ype code) TRLER (the code for LERs) + DA (date range) 880207. 890207 +
Additional events were found by using the same DTC and DA and combining with T GENERATOR and by scanning 10 CFR 50.72 reports. The loss of offsite power information for 1989 was taken from a scan of DCS listings of all LERs submitted in 1989, DTC TRLER and DA 890101. 890720 and 10 CFR 50.72 reports.
A Boolean search of the DCS for all LERs written in 1988 describing scrams found about 320 events. A Sequence Coding and Search System search of LERs written in the same period by a variety of strategies found about 141 LERs of which 45 were applicable to this study. The data bases queried were designed to have safety system information readily accessible. The keywords relevant to this study were few and non-specific; that is, a term such as " transformer" would locate a tiny device on a printed circuit as well as a station transformer.
Therefore, the strategies used were " reactor scram" and " turbine trip". LERs were then read to determine if they were pertinent.
Major Events of Interest:
The following 22 events were of major interest because of the complexity of the event, the safety consequences of the occurrence, the failures of the equipment involved, and the effects on plant systems - safety related and non-safety
3-L o
related.
In general,' automatic systems functioned as they were expected to
- function for the event described unless otherwise indicated. The events described in the following paragraphs are identied by plant name, LER number, date, and event descriptors as defined in Table 1.
Table 1.
Event Descriptors l-
'10PS Interruption of offsite power XFMR Transformer failure BRKR Breaker failure FIRE Fire ESF Engineered safety features actuation, including ECCS, AFW, EDG, PCIS, SRV SCRAM Reactor shutdown, manual or automatic, while critical RPS Reactor shutdown signal while shutdown T-G Turbine or generator non-electrical problems t
[
COMP Complications DUAL Both units at the site involved Kewaunee LER 88/001 03/02/88 SCRAM / FIRE A reactor scram and turbine trip occurred due to an undervoltage transient on l
.two 4160 volt busses which supply power to a reactor coolant pump (RCP) motor andamainfeedwater(MFW)pumpmotor. A section of the bus bar running from.
the main auxiliary transformer to the bus switchgear was badly damaged due to insulation failure and subsequent fault. The fault caused the voltage at the switchgear to decrease and_the undervoltage on both busses caused a reactor scram and turbine trip. The busses were automatically isolated from the main auxiliary transformer and transferred to the reserve auxiliary transformer as expected preventing the trip of the RCP and the MFW pump.
' Due to to amount of smoke in the turbine building, the shif t supervisor activated the emergency siren which required all the on-site personnel to assemble for accountability. The root cause of the event was a fault on the bus bar at the bus bar support. Poor housekeeping in the area of the bus bar lead to water and dirt accumulation which hastened the deterioration of the insulation and provided a path to ground.
Palo Verde 1 LER 88/010 07/06/88 10PS/XFMR/ FIRE /ESF/ SCRAM / COMP A B-phase ground fault occurred on a non-1E 13.8KV bus, ionizing the air in the vicinity, precipitating a three-phase fault to ground. The feeder breakers to this and another bus did not immediately trip because they had a 0.7 second time delay (42 cycles). In that time period, the unit auxiliary transformer which was connected to the faulted bus experienced a greater than 24,000 amp fault, exploded, and caught fire. The busses' supply-breakers and the generator output breakers opened. The unpowered RCPs coasted down causing the the reactor to scram as expected.
Fast bus transfer of the safety busses failed to occur because'the sync check relay, which compares the voltage of one of the safety busses with the faulted bus, found them both in sync, but at zero potential due due to the failed unit auxiliary transformer and the-faulted bus.
(They must be in sync and have the required voltage for the fast transfer to occur.)
i <
Several electrical equipment problems complicated the' post-scram transient. The nuclear cooling water system (non-safety related) was lost when the non-1E power was lost. The essential cooling water (ECW) system was cross-tied to the nuclear ccoling water system. One of the cross-tie valves was found to be shut when the instruments read both open and closed.
When the control room was notified that a fire was in progress, the fire area auxiliary operator proceeded to the area to check the equipment. Deluge flow to the failed transformer could not be verified because the panel was located inside a wall damaged by the transformer explosion. The operator activated the deluge valves to all the transformers and exited the area. Since electrical power to remotely operate the deluge valves had been lost, the manual actuation was necessary.
Emergency ventilation to the control room was initiated when normal ventilation was lost due to loss of power. The normal source of instrument air was not available due to the loss of power and the nitrogen system provided pneumatic backup according to design, s
EDGs were started and the safety busses were loaded onto the EDGs in order to isolate the safety busses from the wet transformers and non-safety busses.
The supply breaker to one.of the safety busses could not be opened remotely but was opened manually. To determine if the faulted bus could be re-energized, operations personnel proceeded to the switchgear room and noted that the only targets on the bus were undervoltage relays (the bus was de-energized) and the trip flags were recorded and reset prior to re-energized. The room was dart and smile was present. The smoke was thought to be coming from the transformer fire. An attempt was made to re-energize the bus, but the breaker tripped and the bus was reported afire.
Upon restarting RCPs A and B, the pumps tripped on an actuation of. the lock-out relay in the speed sensor. The cause of the relay actuation was low direct current (DC) voltage due to a discharged non-1E station battery.
Although all types of errors contributed to complicate the event, the initiator
- failed bus bar - was caused in a large part by poor house-keeping in the area of the bus bar. The dirt accumulation contributed directly to the evolution of a single phase fault into a three phase fault. The second fire was caused by personnel error, with the failure to maintain the emergency lighting as a strong contributor.
Ginna LER 88/006 07/16/88 10PS/BRKR/ SCRAM / COMP An electrical fault in the plant's main electrical substation caused five main breakers in the substation to open.
This caused loss of about one-half of the transmission capability of the substation and loss of ali normal off-site power
.to the plant. The four safety busses deenergized and were reenergized with the EDGs in about 30 seconds.
In that period, an instrument bus was lost momentarily causing a turbine runback. The plant was stabilized at 75 percent power.
The security department reported hearing an explosion at the substation. Upon investigation, it was discovered that a bushing on a 115KV oil-filled circuit breaker had failed and the breaker exploded.
A plant shutdown was commenced at the request of the power control dispatcher because of the damage to the
.c
-S-off-site power transmission equipment. The turbine was taken off-line and the reactor trip breakers were opened. After the turbine was off-line, an operator attempting to reset the feedwater isolation on the B-steam generator (S/G),
pressed the S/G isolation reset button instead and the MSIY closed.
Other problems reported were a rod which failed to insert manually but did insert when the trip breaker was opened, and an oil leak at the substation in the 115KV underground line from the plant to the substation.
Peach Botton 2&3 LER 88/020 07/29/88 XFMR/ FIRE /ESF/RPS/ DUAL 1he capacitors which connect the 500KV #1 bus tie line with the A-phase potential transformer failed and the transformer caught on fire. The potential transformer steps line voltage down to that used by equipment in the substation.
A voltage disturbance ensued which ultimately resulted in several expected ESF actuations on Units 2 and 3 and a partial loss of telephone services at the site.
The power supply which was established was permissible for the conditional which existed at the site. Both units were shutdown and defueled with equip-ment out of service for maintenance, u
These events occurred as expected while the plants were in off-normal configura-tions for maintenance outages.
River Bend LER 88/018 08/2!dd3 SCRAM /ESF/10PS/ COMP The generator tripped from loss of field excitation, resulting in a turbine l
trip which caused a reactor scram.
Prior to the scram, one of the main generator l
brushes had been sparking and maintenance procedures were being readied to l
address the problem.
During the post-scram transient, the recirculation pumps transferred to the slow speed motor / generator set on an end-of-cycle recircula-tion pump trip signal. Reactor pressure peaked high causing five safety / relief valves (SRVs)tolift. Turbine bypass valves opened. The pressure spike collapsed the voids causing reactor water level to decrease. All actuations I
were expected in an event of this type. An hydraulic perturbation on the wide range level instruments showed a low level spike greater than -29 inches.
High pressure core spray (HPCS) and reactor core isolation cooling (RCIC)
L initiated and injected for about 30 seconds. ' Subsequently, the HPCS injection l
I line upstream of the injection valve was found to be hot, due to backleakage I
from the reactor.
A non-safety related bus failed to transfer from the normal station service l
transformer and a second bus failed the fast transfer but did slow transfer.
L The first failure caused a loss of power to to the HPCS bus. The HPCS diesel-generator started and loaded the bus. The turbine building closed cooling water pumps tripped and the instrument air compressors subsequently tripped on high temperature. Power to the RPS bus A was lost resulting in an initiation of the standby gas treatment and annulus mixing systems and the trip of the annulus pressure control system. A spurious high drywell alarm also actuated.
l l
l
- 6'-
Catawba 2 LER 88/028 09/28/88 T-G/ COMP / SCRAM
-The generator stator water cooling system circulates high purity water through the stator coil hollow conductors.
Either of two AC motor-driven centrifugal pumps will produce the required flow. Pressure actuated switches cause the automatic startup of the reserve pump in the event that the pressure decreases.
If the stator cooling water is lost, the turbine runback circuitry will automa-tically reduce the generator output to the rated capability without stator cooling water circulation (about 23% turbine load).
In this event, maintenance personnel bumped the pump switch while removing masking tape after painting the switch panel. The switch was put in the OFF position. The running pump tripped and the reserve pump was prevented from starting by the switch position.
The loss of stator cooling caused a turbine runback as designed and reactor power was reduced from 95% to 35% in 3 minutes.
The turbine runback failed to stop at the rated capability without stator cooling water circulation but terminated when the operator restored the stator cooling water pump.
Following the runback, the steam dumps erratic operation caused an increase in the reactor coolant system (RCS) average temperature (Tave) which caused a S/G 1evel swell when the steam dumps failed to open properly. Then a steam dump iully opened, aggravating the level swell. The turbiretrippedonanindicationofS/Ghighlevel}toautostartbydesign.
isolating MFW and causing the MFW pumps to trip and auxiliary feedwater (AFW The reactor was then manually scrammed because of the loss of main feedwater with the reactor above 10% power.
Braidwood 1 LER 88/022 10/16/88 10PS/ COMP /ESF/ SCRAM / DUAL A loss of all off-site power to the unit occurred when the A-phase potential transformer for a 138KV line failed causing a current surge on the low side of a transformer at an off-site substation, causing its sudden pressure relay to actuate. A transfer trip signal was sent to the 345KV breakers associated with
.both the substation and the plant. The 345KV oil circuit breaker and the 345KV air circuit breakers opened, but the air circuit breaker took longer to open I
causing a pole disagreement actuation (all three phases not in the some state).
This. caused another air circuit breaker to open, which resulted in the power being removed from the high side of the station auxiliary transformers. Auto transfer of two 6.9KV busses occurred and EDGs started and loaded two 4KV busses.
An RCP supply breaker tripped on instantaneous overcurrent because of a piece of cardboard in the relay that bypassed the time delcy of the relay. The licensee speculates that the cardboard was inserted during the last maintenance on the relay and not removed. This caused a reactor scram on low RCP flow with the reactor above 30% power. The turbine and generator tripped as expected.
Voltage on the unit auxiliary transformers decayed, causing a loss of power on additional busses. Station air compressors tripped and instrument air pressure began to decrease to both units. Several attempts to restore equipment were thwarted by the pole disagreement. The pole disagreement was found to have been caused by an out-of-calibration circuit breaker.
m
?
- '[
Hope Creek LER 88/029 11/01/88 SCRAM /ESF/ COMP lA main generator lockout occurred on the loss of excitation which was caused by the failure of the exciter brush-collector ring assembly from undetermined
-causes followed by fast closure of the turbine control valves, main turbine trip, and reactor scram. The recirculation pumps tripped on end-of-cycle J
logic. Reactor pressure increased and the H-SRV lifted. Reactor feedwater
. pumps tripped on high level.
Highpressureecolantinjection(HPCI)andRCIC actuated and injected for about 10 minutes. Automatic actuations were as expected for this type of event. During the subsequent transient, the P SRV 1
(low-set) failed to lift because of a faulty pressure transmitter.
Although the licensee could not determine.the root cause of the exciter brush failure due to the extent of damage to the equipment, it was saeculated that a more rigorous inspection may have prevented the failure. To tais end, the utility is developing a new procedure.
Clinton LER 88/028 11/11/88 XFMR/ SCRAM / FIRE The C-phase main power transformer experienced a phase-to-ground overcurrent fault causing a generator-to-transformer differential relay trip of the main generator and a consequential turbine trip. The reactor scramed on turLine stop valve fast closure as designed. A fire had started from oil which had erupted from the transformer bushing. The fire protection deluge initiated in the transformer area and the fire brigade was dispatched. The fire was above the deluge, since the deluge was designed to preclude' spraying of the high voltage bushing. The fire brigade extinguished the fire in about 30 minutes.
.The cause of the event was an internal fault on the high voltage side of the transformer.
.Sequoyah 1 LER 88/045 11/18/88 ' CRAM /ESF/ COMP A reactor scram occurred as a result of the main turbine being tripped by the main' generator neutral overvoltage relay, which had detected a ground fault.
Steam dump valves opened, MFW isolated, and AFW actuated. All automatic actuations were as designed.
Both motor-driven and the turbine-driven AFW I.
pumps injected. The flow indicator for the turbine-driven AFW pump indicated
.off-scele,high(greaterthan1000gpm). The overspeed protection should have limited the flow to 880 gpm to prevent pump runout. Consequently the RCS cooled down below the no-load Tave. Also the volume control tank level decreased below 7 percent. Maintenance personnel determined that the cause of the event was a ground fault internal to the main generator. There was an insulation breakdown on a C-phase stator bar.
Oconee 1 LER 89/002 01/03/89 FIRE / SCRAM / COMP A fire began in the non-1E switchgear while escalating in power. Two RCPs tripped. After 30 minutes, the reactor was manually scrammed and the remaining 2 RCPs were tripped so that water could be safely used to fight the fire after carbon dioxide and dry chemicals had proven unsuccessful.
A failure of the integrated control system (ICS) to control steam generator level using the AFW nozzles (main feedwater is required to change injection from HFW nozzles to AFW nozzles) caused an RCS pressure transient and technical I
=--
8 specifications (TS) cooldown rates were exceeded while mitigating the pressure transient as a result of overfeeding the steam generators. The failure of the ICS was caused by fire damage to control cables _in the switchgear. The result-ant operation in the thermal shock operating range was not properly compen-sated. Other equipment failures and personnel errors complicated the post-scram transient.
South Texas 1 LER 89/005 01/20/89 FIRE /T-G/ SCRAM / COMP Alarms were received which indicated that there was high vibration in the turbine and a high temperature on the #8 and #9 bearings. A fire was reported at the #9 bearing and the deluge actuated. The turbine was manually tripped causing an automatic reactor scram. The fire brigade extinguished the fire in about 20 minutes and the generator was purged of hydrogen with carbon dioxide and the carbon dioxide with air in the usual manner. The problem with the generator began with a loss of stator cooling due to a loose wire in the temperature sensing unit which caused a temperature increase to go unnoticed and without compensation until the hydrogen seal at the #9 bearing was damaged by the heat causing pressure to increase and leakage. The source of ignition is not known. The visible flames around the bearing were probably caused by heat igniting grease or lube oil.
The generator was disassembled and the rotor removed for inspection. No significant damage was found.
The bearings' pedestals were inspected and no damage was found. The lube oil was drained and flushed.
WNP 2 LER 89/002 01/30/89 SCRAN/ESF/ COMP A buildup of a conductive film on the surface of an insulator on the output l
side of the main transformer caused the insulator to short to ground. The resultant high currents tripped the main generator output breakers. The load rejection tripped the generator and turbine and scramed the reactor as expected.
During the post scram transient, a level transient caused certain containment I
isolations on low level and subsequent pump trips on high level (due to over-compensation). Six SRVs lifted and reseated properly as expected. The scram discharge volume vent and drain valves did.not reopen on the scram reset.
Pilgrim LER 89/010 02/21/89 RPS/ESF/10PS Aircircuitbreakers(ACBs)openedduetoagroundfaultintheunderground portion of one of the C-phase power feeder cables between the secondary side of o
l the startup transformer and a non-safety related 4160 volt bus. The differen-l tial ground current relay detected the cable fault and caused the startup l
transformer to'be locked out which, in turn, tripped the ACBs. The EDGs started and loaded the safety busses as expected. An RPS actuation - scram L
signal, PCIS actuation, reactor building isolation, and SBGT actuation occurred as designed. Offsite power was restored 15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> later.
LaSalle 1&2 LER 89/009 03/02/89 10PS/ SCRAM /BRKR/ COMP / DUAL The C-phase lightning arrestor on the station auxiliary transformer (SAT) failed resulting in a phase-to-ground fault on the line to the unit 2 distribu-tion system. Oil circuit breakers (OCBs) and unit 2 feeder breaker from the
}
- . 9 -
i SAT opened to isolate the fault. Automatic transfer from the SAT to the unit auxiliary tranformer (UAT) was accomplished and an EDG started and loaded a bus as designed.
Because of the lightning arrestor failure, the Unit 1 main generator protective relays sensed a high differential current on phase A and locked out the generator. The load rejection caused a turbine trip and a reactor scram. An 008 failed while opening.
P Pre-existing conditions complicated the post-scram transient. The Unit 2 process computer was performing the primary data acquisition and safety parameter display system (SPDS) functions for both units, with the Unit 1 procese com> uter in standby. The Unit 2 process computer was powered from the Unit 1 SAT >ypassing the uninterruptible power system (UPS) because of a failed inverter in the UPS.
The computer used for core monitoring and off-site dose calculations had also bypassed UPS and was powered from Unit 2 SAT. Additionally, internediate range power ronitors (IRMs) D and F were out of service.
The loss of the process com> uter made the job of assuring that all rods went to full-in more difficult in t1st it had to be done manually for each rod by selecting the rod and observing the rod sequence control system panel " full-in" l
lights.
Service air was lost briefly on Unit 1 making it difficult to reset the A-scram L
channel.
The B-scram channel could not be reset because of the inoperable IRMs. Unit 2 service air was also lost briefly.
L The drywell chillers in Unit 2 were lost for about 15 minutes while the unit I
was in mode 1.
Drywell temperature rose to 213 degrees F and pressure rose to
+0.6 psig.
The 2A reactor feedpurp controller locked up causing a reactor level transient.
Yessel minimum was +25.0 inches and maximum was +53.0 inches. Operators took control of the pump to limit the transient. Reactor water cleanup system and the reactor building ventilation system isolated.
Palo Verde 3 LER 89/001 03/03/89 COMP /ESF/IOPS/ SCRAM A fault in a California switchyard caused a grid disturbance which actuated the I
subsynchronous oscillation protection relay for the unit 3 main turbine /
l generator. The generator output breakers opened and the steam bypass control system was called upon to dump steam to the condenser while reactor power was run back. Automatic actuations were as expected for this transient; however, four of the steam dumps cycled from 10% to 100% open about nine times while the remaining valves fluttered between 80% and 100% open.
Excessive steam removal caused the pressure in steam generator 2 to go low enough to scram the reactor, and to cause the main steam isolation valves to close. There was a safety injection on pressurizer low pressure as well as a containment isolation actuation.
Fast transfer did not occur because the house loads were connected to the generator which was isolated from the grid. (Reverse power signal from the grid would have dropped the house loads from the generator.) As the generator
1 coasted down, the frequency decayed to 30 Hz. When the house loads were dropped from the generator, they could not be synchronized to offsite power because non-1E busses 3E-NAN-501 and 3E-NAN-S02 were deenergized.
Two of the RCPs had been manually shutdown. Loss of power to the non-1E busses shut down the other two. Also the condenser circulating water pumps, air compressors, nuclear cooling water, and some control room displays were lost.
j The atmospheric durp valves malfunctioned for several reasons, one of which was directly related to the loss of IE power - no lighting in the area made the manual operation more difficult. One main steam safety valve opened lower than the TS limits. Normal pressurizer spray was unavailable because the RCPs were not running.
RCP IB seals were degraded when charging was secured.
The MSIV bypass valve could not be operated remotely.
Dresden 3 LER 89/001 03/25/89 10PS/ COMP / SCRAM / DUAL A 345 KV circuit breaker developed a phase-to-ground fault and tripped. Local J
L breaker backup logic tripped additional breakers, isolating busses 8 and 15 and de-energizing the reserve auxiliary transformer (RAT) 32. Bus 32, which was L
powered from RAT 32, transferred to RAT 31 in 14 seconds (slower than design).
l-Bus 32 developed an undervoltage condition because of this and the 3B reactor L
feedpump tripped and reactor feedpump 3A sped up to the point where runout finw control took control of the pump. The 3B recirculation pump also tripped and the 3C standby feedpump would not start until the 32 bus transfer was corpleted.
At that time, still in runout flow control mode, the 30 feedpump started and its flow increased rapidly until the high level trip setpoint was reached, whereupon the feedpump tripped and the turbine stop valves closed tripping the turbine and scramming the reactor.
Other problems were a loss of an annunciator panel - an alert level condition, a transfer of a-low pressure coolant injection system motor control center l
which should not have occurred, spurious breaker trips, failure of the isolation l
condenser supply valve to open, HPCI lube oil problemt, failure of a breaker to remain closed, HPCI turning gear motor failure, security electronics problems, oxygen analyzer failure, and loss of instrument air. Loss of instrument air also affected unit 2.
South Texas 2 LER 89/009 04/05/89 10PS/ SCRAM / COMP /ESF On initial synchronization, with a jumper missing between two terminals of the generator backup distance relay in the generator protection circuit causing an open circuit on the phase C. current transformer of the protection circuit, the breaker pole failure relay actuated causing a generator lockout.
(Another wiring error in the protection circuit on the negative phase sequence relay wouldhavealsocausedageneratorlockout.)
Two 345 KV switchyard breakers, the generator circuit breaker, the generator exciter field breaker, the generator voltage regulator, the main turbine, and the 13.8 KV feeder breakers to auxiliary busses 2F, 2G, 2H, and 2J tripped as designed. The tie breaker to the 13.8 standby bus from the auxiliary bus 2F opened as designed deenergizing the bus and 4160 volt bus E2A.
The RCPs lost power on the loss of the auxiliary busses and the undervoltage coils on their trip breakers generated a low flow signal to the solid state protection system which scransned the reactor. The d gital rod position indicator
-had lost power so all rods on bottom could not be v eified and the RCS was borated.
The EDG for the emergency bus started and loaded the emergency bus as expected.
When bus 2J was reenergized, RCP 2D restarted because its breaker failed to trip on loss of voltage. The additional flow caused a loss in S/G water level and an actuation of AFW.
RCS temperature was decreasing and the MSIVs were closed to prevent overcooling.
The non-ESF balance-of-plant diesel-generator could not be started in auto or manual. The Technical Support Center diesel-generator was out of servi a @
maintenance.
Nine Mile Point 2 LER 89/014 04/13/89 SCRAM /10PS/ESF/ COMP A disconnected wire in the main generator potential transformer cubir a signal to be sent to the main generator protective circuitry which..
a turbine trip. The reactor scrammed on the turbine trip. Fast tran:fr o
offsite pcwer was only partially successful: one of the 13.8 KV busses fn lu to transfer.
This caused a loss of feedwater and subsequent reactor level reduction to level 2.
HPCS and RCIC actuated and injected. After water level reached normal, HPCS and RCIC were shut down. Water continued to be injected into the vessel condensate booster pump could (pressure had been lowered to the point where the from the feedwater lines.
RCS and did) inject water into the vessel through a failed-open valve.
Due to a series of personnel errors and equipment failures, the remaining 13.8 KV bus was deenergized momentarily. This tripped the operating condenser circulating water pump and the decrease in water box level prevented the immediate restart of the pump.
In the face of falling condenser vacuum, the MSIVs were closed but auto-closure occurred before the manual closure could be accomplished.
The uninterruptible power supply to an instrument bus was interrupted causing loss of comunications in the control room and partial loss of emergency lighting. Water level was reported to have risen slightly above the lowest elevation of the main steam lines.
Oyster Creek LER 89/016 06/25/89 XFMR/ SCRAM /ESF A fault in one of the two main transformers caused the main generator to be tripped on a phase differential condition.
Subsequently, the turbine tripped and the reactor scrammed.
Rapid closure of the turbine stop valves caused a reactor pressure spike to 1067 psig. The isolation condensers auto-actuated, two safety / relief valves lifted, all five reactor recirculation pumps tripped, and the reactor water clean up system tripped and isolated. All automatic responses were expected for this transient. The cause of the transformer
-..,..-...---..e,..
i.,.o
.i.,....
i,.-y..y
, 1 ~.
p I
failure was attributed to a failure of an internal winding which caused the phase differential condition which cause the generator trip.
Oyster Creek LER 89/017 07/11/89 XFMR/ SCRAM The second of two main transformers failed due to a failed internal winding which caused a phase differential condition on the main generator. The generator tripped, the turbine tripped, the reactor scramed, and all antici-pated automatic responses occurred.
Since this was the second main transformer failure in a month, the licensee initiated a study to determine the root cause of the transformer failures.
Sumer LER 89/012 07/11/89 10PS/ESF/ COMP Technicians working inside the generator stator cooling water cabinet shorted the power leads on the temperature converter causing the AC power fuses to blow, giving a false indication of loss of stator cooling water. A turbine runback relay in the circuitry failed to operate; instead, the turbine tripped and the recctor scramed.
Sumer was supplying 860 megawatts of power to the grid at 440 megavolt-amperes, reactive (MVt.R)[89.7%powerfactor]. The McHeekin generating station and the Saluda hydro units (older units with their generator backup relays set below the utility's standard) tripped. The subsequent loss of voltage tripped the four Fairfield Pumped Storage units whose backup relays were set at the utility's standard. The cascading failures caused a degraded voltage to be sensed by the ESF busses at Sumer.
The ESF busses tripped on undervoltage and transferred to the EDGs as designed.
Non-1E busses remained on the grid and equipment supplied from them experienced the degraded voltage.
South Texas 2 LER 89/017 07/13/89 XFMR/ SCRAM /IOPS/ESF/ COMP An internal fault ocurred on the main transformer 2A causing the main transfor-mer differential relay and the primary side pilot wire differential relay to actuate the main transformer lockout relay which tripped the main generator breaker, the offsite power feeds from the switchyard, the auxiliary transformer, and the m61n turbine. The reactor scrammed on the. turbine trip. Loss of the auxiliary transformer caused loss of one IE bus and the non-1E busses which supplied power to the RCPs. The IE bus was loaded on its EDG. AFW initiated.
All responses to the incident were as expected.
Events Involving Interruptions of Offsite power:
Twenty-four events in the first 6 months of 1989 were found involving an interruption of offsite power to'the plant. These events are tabulated in Table 2 by plant, LER number, date, number and type of busses involved, and duration of the' loss. Note that the time begins at the loss of power and stops at its restoration. "Available" power which is not used to energize busses is not considered in computing the duration of the incident.
~4.
- 13 4
Table 2.-
Interruptions of Offsite Power in 1989:
Plant LER #
Date Description Time Pa k Verde 2 89/001 01/03/89 2 1E busses 21h8m Stuth Texas 2 89/001 01/06/89 2 1E busses Not known Catawba l' 89/012 01/07/89 1 1E bus-blackout
- 28m 4
. Fermi 2 89/003 01/10/89 2 1E busses 20m AND 1 89/002 01/20/89 1 non-1E bus Not known South Tuas 2 69/00r 01/21/89 All non-1E, 2h53m train A 1E Pilp'im 89/010 0?d1/89 All 15h45m Pa*v Verde 3 S9/001 03/03/89 2 aon-1E busses 30m/41m teuth Texas 2 89/005 03/20/89 2 1E busses 32m
-Oresden 3 89/001 03/25/89 All 6h35m/7h33m South Tu as 2 89/009 04/05/89 4 non-1E, 1 Brief IE busses Surry 1&2 09/010 04/06/89 1 1E,non-1E 4h5m bus ea unit t
Surry 182 89/013 04/13/89 1 IE,non-1E 3h17m bus ea unit
'Aine Nile Pt ?
89/014 04/13/89 1 non-1E lost, Not known 1 dropped North Anna 1&2 89/010 04/16/89 1 1E bus, each 40m/45n unit Soutn Texas 2 89/014 04/18/89 1 1E bus 51m Millstone 1 89/012 04/22/89 All Brief WNP 2 89/016-05/14/89 2 IE busses 50m Millstone 2 89/009 05/23/89 1 1E bus Brief River Bend EN 15855 06/13/89 1 1E bus Not known Ri' er Send EN 15858 06/13/89 2 non IE busses Not known Crystal Rher 3 EN 15886 06/16/89 All 1h6m Bumswict 2-EN 15895 06/17/89 All 9h45m crystal River 3 EN 15986 06/29/03 All (and 1 EDG)
Not known At!ALYSIS Tbc issues addressed by this study are disruptions of normal AC power as manifested by grid, switchv:rd, and non-1E electric systems (including the power generetion system) problems and major electrical equipment failures,
%e ecopment whose failures caused these events (the 22. detailed in the Major Eve n s of Interest. the 24 listed in Table 2, as well as the 57 in Appendix 1)
W re busses, circuit breakers, capacitors, potential transformers, switchgear, exciter brushes, neutral grounding transformers, lightning arrestors, stator cooling systems for the main generator, and main transformers as well as the main turbine and the main generator.
Firu have been caused by the catastrophic failure of transformers and circuit breakers, oil fil'cd devices which have both fuel (oil) and ignition source (electrical failure).
In other cases, the failure of devices such as switchgear
14 and bus bars provide the ignition source with' the fuel being provided by nearby combustible material sometimes the result of poor housekeeping. The results of fire are often equipment damare and personnel hazard.
A common factor to many of these events is maintenance. At Kewaunee and Pa'o Verde, poor housekeeping contributed to the breakdown of the bus insulation and complicated the event. Maintenance errors at Catawba initiated and complicated the event. At Palo Verde, poor maintenance of emergency lighting complicated the recovery from the event by contributing to personnel error that caused a second fire. A piece of cardboard left in a relay from a previous maintenance at Braidwood complicated recovery from the event by tripping the RCPs and causing a scram.
Peach Bottom was vulnerable to the problems they experienced because they were in an alignment for a maintenance outage.
The events at Kewaunee and Palo Verde 1 had virtually the same initiator, a bus failure caused by insulation degradation complicated by dirt in the cubicle; yet consequences et Kewaunee were normal and expected, while the consequences at Palo Verde I were complicated by eouipment failures, design errors, personnel errors, and pre-existing plant conditions. AE0D has issued a report AE0D/E905,
" Electrical Bus Bar Failures," based on these two events. An NRC information notice, IN 89-064, based on this report was also issued.
t River Bend had two events on the list, an exciter brush failure while prepara-tions for maintenance were being completed, and a fault on a neutral grounding transformer caused by a stray cat grounding the high side of the transformer.
i l
The latter event led to a rather " normal" load rejection and scram while the former was complicated by a partial failure to transfer to offsite power, several ESF actuations, and a coincidental, but potentially serious, backleakage i
l from the reactor into the high pressure core spray lines.
1.
/
l The Braidwood incident was caused by the failure of an off-site potential I
transformer and affected both units on the site. General Design Criterion 17 L
(10 CFR 50, Appendix A) calls for "two physically independent circuits...
designed and located so as to minimize to the extent practical the likelihood of their simultaneous failure under operating and postulated accident and i
environmental conditions." A common point is vulnerable to a failure that l
could cause simultaneous loss of more than one circuit.
l Loss of offsite power is identified as the main contributor to risk of station l
blackout. The staff states in NUREG 1109, " Regulatory Backfit Analysis for the Resolution of Unresolved Safety Issue A-44, Station Blackout," that "the estimated frequency of core damage from station blackout events is directly proportional to the frequency of the initiating event." Table 2, compiled from all LERs and 10 CFR 50.72 reports made in 1989, lists the events in the first 6 months of 1989 that have met parts of the blackout criteria (turbine trip or off-line, loss of offsite power to IE and non-1E busses, and unavailability of onsite emergency AC power).
Of the 24 events in Table 2, only one involved a IE bus blackout (a total loss of AC power) - a short duration event caused by a loss of offsite power to the bus with its EDG unavailable. Twenty events involved the loss of one or more IE busses and 14 events involved the loss of one or more non-1E busses.
The
i,:.
I duration of the events ranged from momentary to 21 hours2.430556e-4 days <br />0.00583 hours <br />3.472222e-5 weeks <br />7.9905e-6 months <br />.
Four plants reported more than one event. Three events involved both plants on the site. The 24 events involved 19 plants at 15 sites. The causes of the events are nearly equally divided between human error and electrical failure with weather being a minor contributor.
Six of the total events reviewed involved either total or partial interruptions of offsite power to the station auxiliary power system because of problems with the " fast bus transfer" scheme, Palo Verde 1 - 07/06/88; River Bend - 08/25/88;
-palo Verde 3 - 03/03/89; Dresden 3 - 03/25/89; Nine Nile Point 2 - 04/13/89; and South Texas 2 - 07/13/89.
In all cases, the EDGs provided power to the plants' safety busses.
The fast bus transfer scheme is designed to permit the station's auxiliary electric loads (both IE and non-1E) to be continued to be supplied from offsite i
power sources, the preferred source per GDC 17, following a generator / turbine trip with reactor scram. This is done by quickly transferring power to the various auxiliary busses from the VAT (connected to the generator) to the reserve or startup transformer (connected directly to the switchyard). The transfer is quick enough so that the associated busses and loads see virtually i
l uninterrupted power supply, improving the reliable operation of this scheme would, per GDC 17 " minimize to the extent )ractical, the likelihood of simul-taneous failure" of electrical power from tie transmission network to the onsite l
- distribution system.
The results of the loss of power to the 1E busses have, in all tut one case, been the immediate transfer to EDGs. The loss of non-1E busses have resulted in the loss of condenser circulating water pumps, consequent loss of condenser vacuum, and unavailability of the condenser as heat sink; the loss of one or more RCPs, subsequent reactor scram, and loss of forced circulation; and the loss of cooling water to air compressors and subsequent loss of service and
' instrument air, 1
FINDINGS 1.
The enumerated events show that failures in electrical supply or power l
generation equipment impact safety equipment in that they often cause scrams or q
necessitate unit shutdown in operating plants, they cause ESF systems to operate, they frequently cause interruptions in offsite power, they occasionally cause fires, and they complicate post-scram transients by rendering needed equipment unavailable or unreliable.
Table 3.
Sumary of findings Total 10PS Fire Comp.
-l Detailed Events 22 13 6
18 Other Reports 57 20 1
31 l
Total 79 33 7
49 l
I e
j The comparison of the number of interruptions of offsite power in 1989 in Appendix 1 (4) which was compiled by a variety of automated searches to the number in Table 2 (24) which was corspiled from reading expanded titles of each LER shows that the scope of the problem to be even greater than Appendix 1 would suopest. Expanding that idea, it may be said that for each failure in electrical supply systems and power generator equipment found by computer searches, there may be sever 61 others not found, i
2.
Interruptions of offsite pcwer to IE busses generally cause transfer of the busses to EDGs while interruptions of offsite pcwer to non-1E busses cause i
scrams, loss of forced circulation, loss of heat sink, and loss of service and instrument air.
From Table 2, it can be seen that several plants have multiple events and several events involve multiple units.
3.
Seven fires occurred which damaged equipment and caused complicated j
transients and involved exposure of personnel to fire and smoke.
)
4.
After a scram initiated by and coupled with electrical power problems, there riay be a failure of the fast bus transfer scheme and a resultant complica-tion of the post-scram transient in which the skill and training of personnel; the accuracy and completeness of procedures; and the design, construction, operation, and maintenance of nicoy systems cnd components in the plant can be challenged. As with every scram, additional duty cycles on the RCS pressure boundary due to the cool down and reheat also occur.
5.
a.
Several ESF actuations occurred as a consequence of the initietor, the nornial actuations (expected to occur as a result of the transient caused by the initiator) were - EDG autostart and load, PCIS actuations, HPCI and flCIC injection, and AFW initiation.
b.
The abnormal actuations (not an expected response to the initiator, occurred as a result of equipment failure caused by the initiator in the ensuing transient) were - EDGs manually started and loaded, emergency control room ventilation actuated, nitrogen backup for instrument air actuated, all area radiation monitors alarmed, standby service water actuated, standby gas treatment system actuated, annulus mixing system actuated, HPCS and RCIC i
actuated, and AFW autostart.
c.
Coincidtntal equipment failures (occurring in the transient but not caused by the initiator) include atmospheric steam dump malfunctions, valve position indicator malfunctica, flashing in the letdown line, pressurizer level transient, malfunctioning letdown isolation valve, operator errors, failure to remotely operate of various devices, backleakage of RCS into HPCS, and other instrument malfunctions.
l l
CMCLUS10h5 Failures in electrical supply or power generation equipment impact safety equip-ment. They of ten cause scrams or require unit shutdown, they may cause engineered safety features systems to operate, they frequently cause interruptions of offsite power supply to station auxiliaries, they occasionally cause fires, and they complicate post-scram operations by rendering equipment unavailable or unreliable.
Interruptions of offsite power to 1E busses, while they are potential contribu-tors to the risk of a station blackout, generally cause transfer of the busses to EDGs while interruptions of offsite sewer to non-1E busses cause major plant transients and complicate scrams. Of tie 24 events involving interruptions of offsite power supply, six were total losses of offsite power as considered in the blackout rule, but the remaining events are equally important.
Interruptions which are not recoverable from the offsite power source due to either equipment or system f ailures lead to the same consequences as a station blackout due to a total loss of offsite power. Therefore, inprovement in the o)eration and reliability of equipnent such as EDGs, transformers, busses, switcigear, fast bus transfer schehe, etc. are necessary.
Furthermore improvement of the reliability of the non-1E power supply and equipment has the potential of lessening the impact of problems with electrical supply on the safety systems by decreasing the number of scrams, ESF actuations, and complicated post-scram transients.
Recent examples of failure in the fast bus transfer schene point to the need to find the root cause of the failure and to make effective corrective actions so that the simultaneous failure of offsite power sources is truly minimized.
Failure of the fast bus transfer schene is being addressed in an on-going AE00
- study, rires are easier to prevent through surveillance and maintenance than to fight.
Fires occur when an electrical fault ignites a fuel source. personnel and touipment are at risk in fires, preventive mairu nance and good housekeeping practices can minimize both ignition sources and G el for fires.
The bus bar failures at palo Verde 1 and Kewaunee were addressed in AE00/E905,
" Electrical Bus Bar failures" and an NRC information notice, IN 89-004 Transformer failures which have occurred are being investigated in an on-going AE00 study.
The electrical maintenance, surveillance, and procedural problems identified in this report should be utilized in the development of the instructions to be used in the upcoming electrical team inspections being formulated by NRR.
m m
m m
z
4 APPENDIX 1 OTHER REPORTS INITIATOR DKT LER NO. DATE SCRAM 10PS FIRE #ESF COMP LOSS OF EXCITATION 029 88/008 880517 A
P N
Y Y
LOAD REJECT 155 88/002 880307 M
Y H
Y Y
GROUND FliULT ON BUS 219 88/022 881002 (SD)
N N
Y Y
LOW VOLTAGE 220 88/015 880725 A
N N
N N
PUS DEENERG12ED 237 88/021 881113 P
N Y
N 10PS 245 89/012 890429 P
N N
N BUS DEENERG12ED 2E9 88/045 881129 P
N Y
N GENERATOR GROUND 265 88/001 880127 A
N N
Y Y
TURPINE VIBRATION 271 88/008 880719 A
N N
Y N
ANTI-MOTORING RELAY 275 88/026 880930 A
N N
N N
- BREAKER FAILURE 280 89/005 890303 (SD)
P N
Y Y
LOSS OF EXCITATION 287 89/002 890306 A
N N
N Y
LOSS OF EXCITATION 295 88/011 880606 A
N N
l{
N DEGRADED BUS 295 88/015 880813 (SD)
Y N
N N
BREALER FAILURE 296 88/005 881108 (SD)
Y N
Y N
ARC IN IS0 PHASE BUS 298 88/019 880715 M
N N
Y N
TRANSFORMER F/! LURE 301 89/002 890329 A
P N
Y Y
GRID PERTURBATIONS 309 89/003 890505 A
N N
N N
FAULTED MAIN XFMR 309 88/006 880819 A
Y N
Y Y
PHASE YOLT IMBALANCE 312 88/015 881014 A
N N
Y Y
GENERATOR GND 321 88/003 880226 A
N N
N Y
LINE FAULT 322 88/016 E81011 (SD)
N N
Y N
LOAD REJECTION 323 89/005 890416 A
N N
N Y
LIGHTN!hG STRIKE 328 88/034 880815 N
D Y
Y Y
TURBINE VIBRATION 331 88/008 880724 A
N N
N N
10PS 333 88/011 881031 (SD)
Y N
Y Y
10PS 336 88/005 880304 (SD)
Y N
Y Y
LOSS OF VACUUM 338 88/002 880203 M
N N
Y Y
DUAL UNIT 10PS 338 89/010 890416 (SD)
Y N
Y Y
GROUND FAULT ON XFMR 341 88/019 880606 M
P N
Y Y
TURBINE VIBRATION 341 88/030 880912 A
N N
N Y
XFMR FAULT 358 88/021 880906 A
N N
Y N
L LOSS OF EXCITATION 369 88/001 880107 A
N N
Y Y
LOAD REJECT 387 88/006 880304 A
N N
N N
GROUND FAULT, LINE 387 88/010 880618 A
N N
N N
LOAD REJECT 410 88/012 880305 A
N N
Y Y
HAIN XFMR FAULT 416 88/002 880110 A
N N
N Y
GEN FIELD GND 424 88/006 880215 A
N N
Y N
HIGH STATOR COOLING 424 88/008 880407 A
N N
Y Y
OVEREXCITATION 424 88/022 880808 A
N N
Y N
FAULT ON DISCONNECT 424 88/024 880730 A
N N
N N
LOSS OF STATOR COOLING 425 89/018 890422 N
N N
N Y
TURBINE TRIP 440 88/026 880623 A
N N
N N
10PS 443 88/004 880810 (SD)
Y N
Y N
GRID INSTABILITY 454 88/005 880804 A
N N
Y Y
APPENDIX 1(CONT.)
OTHER REPORTS INITIATOR DKT LER NO. DATE SCRAM 10PS FIRE fESF COMP BUS DEENERGlZED 455 88/008 880714 A
N N
Y Y
GENERATOR LOCK 0UT 457 88/012 880620 A
N N
Y N
LOSS STATOR COOL 458 88/003 880128 A
N N
N Y
XFMR FAULT 458 88/005 880314 N
P N
Y N
PARTIAL 10PS 498 88/026 880330 A
P N
Y Y
STATOR COOLING LOW 498 88/049 880826 A
N N
Y Y
10PS 499 80/005 890320 N
P N
Y N
GEN PROT CIRCUIT ACT 499 89/014 890418 (SD)
N N
N N
10PS 528 88/003 880216 (SD)
Y N
Y Y
LOAD REJECT 528 88/011 880419 A
N N
N Y
LOW STATOR COOLING 528 88/021 880821 A
N N
N Y
10FS 530 88/004 880505 N
Y N
Y N
i-KEY:
SCRAM - A AUTOMATIC 10PS - P PARTIAL M MANUAL T TOTAL (SD) SHUTDOWN D DUAL UNIT
-