IR 05000269/1992026
| ML16148A712 | |
| Person / Time | |
|---|---|
| Site: | Oconee  |
| Issue date: | 11/25/1992 |
| From: | Shymlock M NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML16148A711 | List: |
| References | |
| 50-269-92-26, 50-270-92-26, 50-287-92-26, NUDOCS 9212040292 | |
| Download: ML16148A712 (57) | |
Text
pfk REGU UNITED STATES NUCLEAR REGULATORY COMMISSION
REGION II
101 MARIETTA STREET, N.W., SUITE 2900 ATLANTA, GEORGIA 30323 N1OV 2 7 1992 Report No.:
50-269/92-26, 50-270/92-26, and 50-287/92-26 Licensee:
Duke Power Company Docket No.:
50-269, 50-270, 50-287 License No.:
DPR-38, DPR-47, DPR-55 Facility Name: Oconee Nuclear Station Units 1, 2, and 3 Inspection Conducted: October 20 -
28, 1992 Team Members:
B. Desai, Resident Inspector, RII C. Payne, License Examiner, RH B. Raughley, Senior Mechanical Engineer, AEOD L. Wert, Senior Resident Inspector, RH L. Wiens, Project Manager, NRR Accompanying Personnel: S. Rudisail, RH Team Leader:
/Z-t fY/J M. ShymlocK/Chief Date Signed Plant Systems Section Division of Reactor Safety 9212040292 921127 PDR ADOCK 05000269
TABLE OF CONTENTS Paige INTRODUCTION Background..........................................
1 AIT Formation.........................................
2 AIT Charter...........................................
II. SEQUENCE OF EVENTS..........................................
II DETAILED DISCUSSION OF EVENT Initiating Event..................................9 Loss and Lockout of Red and Yellow Busses, Switchyard Isolation and Loss of Off-site Power..................
10 Initial Keowee Hydro Station Response.................. Restoration of Red Bus from Off-site Power............
14 Loss of Keowee Units While Restoring Yellow Bus to Grid..................................
14 Return of Electrical Distribution System to Normal Alignment.................................15 Overall Oconee Response to the Event..................
.16 Emergency Notification/Classification..................16 I
I EQUIPMENT MALFUNCTIONS OR ANOMALIES, INSPECTION, TESTING, AND MODIFICATION ACTIVITIES Equipment Malfunctions or Anomalies....................
1. 230 kV Switchyard Battery Charger SY-2............
17 kV Switchyard Breaker Failure Circuit.....
18 kV Switchyard Events Recorder................
18 Keowee ACB-7 Failure to Close...................
.18 Keowee ACB-8 Failure to Close............
...
9 Undervoltage Trip of Keowee Unit 2............... 20 Keowee Unit 2 Generator Field Breaker Failure to Close................................20 Keowee Telephone, Computer Alarm Typer and Alarm Panels.............................
21 Equipment Inspection, Testing and Modifications Activities............................................
21 Testing to Confirm the Initiating Event...........
21 Testing to Confirm Adequacy of the PC B Breaker Failure Circuit Modification....................
22 Inspection and Testing of PCB-24............
2.
Inspection and Testing of Keowee ACB-5............
22 Testing of the Keowee Auxiliary Bus!
Transfer Function...............................
2 LICENSEE ACTIONS PRIOR TO OCONEE UNIT 2 RESTART Confirmation of Action Letter.........................
23 Keowee Electrical System Realignment....................
25 Procedures and Training.................................
26 Keowee Special Test....................................
V ANALYSIS OF RELATED ISSUES Procedure Inadequacies.................................
29 Abnormal Procedures at Oconee Unit 2..............
29 Procedures at Keowee Hydro Station...............
30 Normal and Emergency Operating Procedures
at Oconee Unit 2.................................30 Personnel Knowledge.and Training Oconee Nuclear Station...........................
. Keowee Hydro Station.............................32 Complexity of Design..................................
VI ROOT CAUSE DETERMINATION.....................................
VIII. CONCLUSIONS.................................................
IX. EXIT........................................................
APPENDIXES AND FIGURES Appendix 1 Acronyms and Abbreviations Appendix 2 System Design Descriptions General Oconee Electrical System Red and Yellow Bus Lockout External Grid Protection System (Switchyard Isolation) Degraded Grid Protection System Main Feeder Bus Monitoring Panel System Emergency Power Switching Logic (EPSL) Retransfer To Startup Logic General Keowee Electrical System Appendix 3 Persons Contacted Figure 1 Oconee Electrical Distribution Figure 2 Keowee Electrical Distribution Figure 3 230 kV Switchyard DC Power III
.. INTRODUCTION Background Oconee Units 1, 2, and 3 are B&W pressurized water reactors with steel lined prestressed post tensioned concrete cylinder, hemispherical dome containments. Each nuclear unit is rated at 850 MWe. Secondary plant cooling water is supplied from and returned to Lake Keowee. Oconee Units 1 and 2 supply the transmission grid via a 230 kV switchyard while Oconee Unit 3 supplies the transmission grid via a separate 525 kV switchyar Emergency AC power is provided by two, dual use hydro generators located on-site and supplied by Lake Keowee. The station is located 8 miles north of Seneca, South Carolina in Oconee Count Unit 2 went critical in November 1973 and was commercially operational in September 1974. Units 1 and 3 went critical in April 1973 and September 1974 and were commercially operational in July 1973 and December 1974, respectivel On October 19, 1992, Oconee Units 1, 2, and3 were generating to the grid at 100% power with no significant equipment problem Normal Oconee control room staffing was present.. Maintenance was in progress per Temporary Installation TN/5/A/2863/00/AL2, Replace 230 kV Switchyard Battery SY-2, to replace one of the 230 kV switching station batteries. The Oconee Unit 1 Supervisor was present at the switchyard relay house to perform switch alignments in support of the TN. A routine fire drill was in progress at another building within the owner controlled area. The Oconee Unit 2 Supervisor and several auxiliary operators were involved in the dril The Oconee Unit 3 Supervisor was present in the Unit 1/Unit 2 control room. Keowee Unit 1 was operating, generating power to the transmission grid for load peaking and available to supply the overhead emergency power path. Keowee Unit 2 was operable and aligned to the underground emergency power pat Transformer CT-5 was energized and available to manually supply the Standby Busses from the Central switchyar At 9:21 p.m., a DC control power problem in the 230 kV switchyard caused a Red and Yellow Bus lockout and subsequent Switchyard Isolation. This lockout also caused an Oconee Unit 2 main generator transformer lockout producing a turbine trip and an ensuing reactor trip. The Oconee Unit 2 startup transformer breakers were also opened by the problem and AC power was restored to the units from the Keowee hydro station. Oconee Unit 1 continued generating to the grid because one of its output breakers was unaffected by the lockout. Oconee Unit 3 also continued generating to the grid since the 525 kV switchyard was not affected by the lockout. However, off-site power to both Oconee Unit 1 and Unit 3 startup transformers was lost. A Notice of Unusual Event was declared at 10:25 p.m., and the NRC Operations Duty Officer was notified at 10:44 ALT Formation On the morning of Tuesday, October 20, 1992, the acting Regional Administrator, after briefings by the Region II staff and with the concurrence of Senior NRC Management, dispatched an AIT to investigate-the events related to the Oconeel Unit 2 reactor trip and subsequent loss of emergency AC power. The regional team was augmented by individuals from the Office of Nuclear Reactor Regulation and the Office for Analysis and Evaluation of Operational Data. Members of the AIT began arriving at the Oconee site later that afternoon. The special inspection commenced with a preliminary entrance meeting followed by an event briefing conducted by the license AIT Charter Detailed instructions for the inspection were transmitted to the team in a memorandum from the Regional Administrator dated October 21, 1992. The objectives of the AIT were to communicate the facts surrounding the event to Regional and Headquarters management, and to document the findings and conclusions of the onsite inspection. The Charter for the AIT specified the following: Develop and validate the sequence of events associated with the October 19, 1992, Unit 2 reactor trip and subsequent loss of emergency AC power. This sequence should begin with plant.conditions immediately prior.to the event and extend until the plant had fully recovered from the transien.
Identify the initiating event that led to the Red and Yellow bus lockou.
Evaluate the safety significance of the event with regard to system performance and plant proximity to safety.limits as defined in the Technical Specification.
Evaluate operator actions during the event and the effectiveness of the procedures for the recovery from the loss of all AC powe.
Evaluate the accuracy, timeliness and effectiveness with which information on these events were reported to the NR Also, evaluate the adequacy of the event classificatio.
Determine the root cause of each equipment malfunction and personnel error. Include failure of breaker PCB-23, unexpected trip of Keowee Unit 2, and loss of auxiliary power to both Keowee unit.
Prepare a special inspection report documenting the results of the above activities within 30 days of the inspection completio II. SEQUENCE OF EVENTS The following sequence of events was developed from information gathered from control room logs, Keowee logs, sequence of events recorders, data printouts, interviews with individuals involved in the event, and from reviews of the licensee generated sequence of event TIME ITEM DESCRIPTION October 19, 1992 6:55 Keowee Unit 1 is started and aligned to supply power through the overhead power path to the transmission gri :00 TN/5/A/2863/00/AL2, 230 kV Switchyard Battery Replacement is in progress. Oconee Unit 1 Supervisor arrives to perform switch manipulations in support of the T :21:08 Oconee Unit 1 Supervisor locally opens the cross-tie breaker between SY-DC-1 and SY-DC-2 busses in the 230 kV switchyard relay house in accordance with the TN (see Figure 3).
This action results in DC bus SY-DC-2 being powered from only battery charger SY-2, previously the SY-2 battery had been electrically removed from the bu A DC voltage surge from battery charger SY-2 and/or the switching actuates the breaker failure circuitry in PCB-24 causing PCB-23 to open, the Oconee Unit 2 main generator transformer to lock out and the Yellow Bus to lock out (see Figure 1).
The Yellow Bus lockout opens PCBs-9, 12, 15, 18, 21, 24, 27 and 3 PCB-33 was tagged open for maintenanc The voltage surge also initiates the breaker failure circuitry of a PCB on the Red Bus causing a lock out of that bus as wel The Red Bus lockout opens PCBs-4, 7, 10, 13, 16, 19, 22, 26 and 28. PCB-31 was tagged open for maintenanc The External Grid Trouble Protection System initiates a Switchyard Isolation due to the loss of voltage on the Red and Yellow Busses. Switchyard Isolation opens
- 4 PCB-8. Keowee Unit 1 separates from the grid and ACB-1 opened (see Figure 2).
An emergency start signal is generated for both Keowee unit Switchyard PCBs-11, 14, and 20 are unaffected and remain closed. Neither of the lockout signals required these breakers to trip nor did the breaker failure circuit problem cause these PCBs to ope After Keowee Unit 1 separates from the grid, it overspeeds and a normal generator lockout is received.. The Keowee hydro station auxiliary busses fast transfer to their alternate power source, as designed. When the Switchyard Isolation temporarily de-energizes the overhead path and both Keowee units emergency start. The emergency start signal overrides the Keowee Unit 1 normal generator lockou The Oconee Unit 2 main generator transformer lockout produces a turbine trip and reactor trip. Oconee Unit 2 MFBs are de-energized due to the trip and are not automatically re energized from the startup transformer due to the switchyard busses being locked out.. A loss of off-site power occurs on Oconee Unit Sensing a loss of voltage, the MFBMPs begin to time ou Oconee Units 1 and 3 remain at 100% power with their MFBs being fed from their generator outputs. However, the 230 kV switchyard breaker alignment results in the startup transformers (CT-i,- CT-2, and CT-3) for each of the three units being unavailable to supply power to the MFB :21:09 Oconee Unit 2 RCPs trip due to loss of power, and the unit goes into natural circulatio Oconee Unit 2 condensate and feedwater pumps are lost when the MFBs are de-energized. Emergency feedwater is initially provided by the TDEFW Loss of the Oconee Unit 2 MFB also de-energizes the battery charger SY-2. Main condenser cooling is provided by gravity flow as designe A Switchyard Isolation Complete signal is generate :21:12 After opening on the Yellow Bus lockout, PCB-9 receives a close signal from the; Switchyard Isolation circuitry to align the overhead path
to the Yellow Bus. However, the receipt of simultaneous open and close signals actuates the breaker's anti-pump feature and PCB-9 remains ope :21:14 The Switchyard Isolation Complete signal allows ACB-1 to reclose. The overhead power-path is re-energized but the.Yellow Bus remains de energized since PCB-9 is open. The Keowee hydro station auxiliary busses retransfer to their normal power sourc :21:20 The Keowee operator enters the Keowee CR. He notes numerous annunciator windows flashing on the alarm panels but fails to recognize the Keowee units have received an emergency start signa Seeing indications of no load on Keowee Unit 1, the Keowee operator attempts to opens ACB-1 to protect the Keowee unit from a perceived proble The opening of ACB-1 removes normal power from the Keowee hydro station auxiliary busses again and they attempt to fast transfer to their alternate power source. ACB-5 opens to transfer the iX bus but a lockout occurs on the bu Keowee Unit 1 auxiliaries are lost but the unit continues to operate with its control functions being supplied by the DC batteries. ACB-6 opens to transfer the 2X bus but.ACB-8 fails to shu Consequently, Keowee Unit 2 auxiliaries are also lost but again the unit continues to run on its batterie After opening ACB-1, the Keowee operator then notes that Keowee Unit 2 has emergency started and is accelerating to speed and voltage. The Keowee CR lighting has extinguished and the alarm panels are dar The Oconee Unit 1 Supervisor recloses the switchyard cross tie-breaker between SY-DC-1 and SY-DC-2. The resulting DC voltage surge actuates the PCB-8 and/or PCB-9 breaker failure circuit relays and causes the Keowee main step up transformer to lock ou (This lockout will also cause ACBs-1, 5 and 6 to trip if not already open.)
9:21:21 Keowee Unit 2 is up to speed and voltage and energizes emergency transformer CT- :21:28 The MFBMPs generates an additional emergency start signal for-the Keowee unit :21:29 The MFBMPs initiate an Oconee Unit 2 load she Since Keowee Unit 2 has already energized CT-4, the SKI and SK2 breakers close to energize the Standby Busse :21:39 Oconee Unit 2 Standby Bus breakers S1 and S2 close to re-energize the Oconee Unit 2 MFB This provides power to the Oconee Unit 2 MDEFWPs and HPI pump :21:45 The Keowee operator attempts to phone the Oconee Unit 1/2 CR but cannot because the telephone is supplied from Keowee auxiliary bus power. He calls the Charlotte load dispatcher on the dedicated dispatcher line. He informs the dispatcher of a problem with the Keowee units and his need to contact the Oconee CR. The Keowee operator also requests that the dispatcher call the Keowee on-call technician and direct him to report to the Keowee hydro statio :24 The Oconee Unit 2 Supervisor contacts the Keowee operator, via the load dispatcher, and directs him not to take any action involving Keowee Unit 2 since it is providing power to Oconee Unit 2 MFB The Oconee operator resets the MFBMP/Load Shed signals and commences recovery action :49 The Keowee on-call technician arrives at the Keowee hydro station. He determines that a Keowee main transformer lockout, a Keowee Unit 1 normal generator lockout, and a 1X transformer lockout have occurred. He also notes that ACBs-5, 6, 7 and 8 are open with the IX and 2X auxiliary busses de-energize :58 The Keowee on-call technician informs the Oconee Unit 2 Supervisor of the Keowee situation (by dispatcher phone line) and obtains permission to reset the Keowee main transformer lockout (86T).
When 86T is reset, ACB-1 and ACB-6 automatically shut, re-energizing the 2X auxiliary bus. ACB-5 does not close because the 1X auxiliary bus is still locked.ou II
9:59 The Keowee on-call technician unsuccessfully attempts to manually-close ACB-5. He also unsuccessfully attempts to manually close ACB- :00 The Oconee Unit 1 Supervisor resets the Red and Yellow Bus lockout :01 The Oconee CR requests that the Charlotte dispatch office start and align the Lee gas turbines as a dedicated supply to transformer CT- :06 The Keowee on-call technician determines that the ACB-7 overcurrent device has tripped and resets i :12 The Keowee on-call technician resets the lX auxiliary bus lockout. The Keowee operator shuts ACB-7, restoring the 1X auxiliary bus and Keowee CR normal lighting. The Oconee CR is informed that AC power has been restored to both Keowee units' auxiliarie :13 The Oconee CR re-energizes the Red Bus from the grid by closing PCB-10. PCBs-7, 13, 16, 19 and 22 automatically reclos The Oconee operator attempts to close PCB-26 to re-energize CT-2. PCB-26 closes but then reopens because the Switchyard Isolation signal is still present. However, this action momentarily clears the Switchyard Isolation Complete signal and removes PCB-9 from anti-pump. PCB-9 automatically.shuts and re energizes the Yellow Bus from Keowee Unit Though energized, no loads are aligned to the Yellow Bu :16 The Keowee Supervisor arrives at the Keowee hydro statio :18 The Switchyard Isolation signal is rese PCB-26 is manually shut which re-energizes CT-2 from the Red Bu :24 The Lee gas turbines energize CT-5 through the dedicated lin :25 The Oconee Shift Supervisor declares a NOUE based on loss of off-site powe :37 Keowee Unit 1 normal lockout is reset at the Keowee Hydro Statio :44 The Oconee operators reset the Keowee emergency start signal to allow shutdown of Keowee Unit 1 so the Yellow Bus can be tied to the Red Bu The NRC Operations Duty Officer is notified of the Unusual Event at Oconee via the EN :50 The Keowee operator unsuccessfully attempts to manually close ACB-5 to restore the normal source of AC power to the Keowee Unit 1 auxiliarie :51 The Keowee Supervisor attempts to reset the ACB 5 Amptector spring and accidentally hits the lX bus lockout trip device. This trips ACB-7 and locks out the lX auxiliary bus again. Keowee CR lighting is again los :52 The ACB-7 overcurrent trip device is reset. The 1X transformer lockout is reset. ACB-7 is reclosed and 1X is re-energize :53:00 The Oconee operators direct shutdown of Keowee Unit 1. When Keowee Unit 1 is shutdown, Keowee Unit 2 trips resulting in a second de energization of the Oconee Unit 2 MFBs. Sensing the loss of voltage, the MFBMPs begin to time ou :53;20 The MFBMPs generate an emergency start signal for the Keowee unit Keowee Unit.1 emergency starts but since the Switchyard Isolation Complete signal had been removed during earlier restoration efforts, ACB-1 does not close. The Keowee 2X auxiliary bus fast transfers to and is.energized by its alternate sourc Keowee Unit 2 also emergency starts. However, the unit does not re-energize the Oconee Unit 2 MFBs because its field breaker does not close thus preventing the generator field from being flashe :53:21 The MFBMPs initiate an Oconee Unit 2 load she :53:30 The Retransfer to Startup circuitry re-energizes the MFBs from CT-2 and the Red Bus (off-site grid).
10:54 The Oconee operators reset the MFBMP/Load Shed signals and initiate actions for a second recovery of the Oconee electrical syste October 20, 1992 12:18 The Oconee operators reset the Keowee emergency start signal and both Keowee units are shutdow Reducing unit speed allows the Keowee Unit 2 Field Breaker X-Relay to rese :21 The Keowee operator places the 2X auxiliary bus automatic transfer switch in "manual" with the 2X bus aligned to its alternate power source. A 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> LCO on the Keowee auxiliaries is entere :24 The Keowee Unit Operator manually starts Keowee Unit 2 and energizes CT- :42 The Oconee operator manually shuts PCB-8, re energizing the Keowee overhead power path and the Yellow Bu :48-The Oconee operator manually closes PCBs-12, 15, 17, 12:57 18, 21,27, 28, and 30 completing restoration of the 230 kV switchyard. PCBs-23 and 24 are not reclosed due to Oconee Unit 2 reactor tri :14-All Oconee Unit 2 RCPs are restarte :29 03:25 The Oconee Shift Supervisor exits the NOU II DETAILED DISCUSSION OF EVENTS Initiating Event On the night of October 19, 1992, TN/5/A/2863/00/AL2 involving replacement of the 230 kV, 125 VDC switchyard batteries was in progress. The switchyard battery system alignment just prior to the event was such that battery SY-2 and charger SY-2 were disconnected, switchyard DC busses SY-DC-1 and SY-DC-2 were cross tied, and charger SY-1 and battery SY-1 were energizing busses SY DC-1 and SY-DC-2 (Refer to Figure 3).
After connecting charger SY-2 to the system, busses SY-DC-1 and SY-DC-2 were separated leaving charger SY-2 tied to bus SY-DC-2, and charger SY-1 and battery SY-1 tied to bus SY-DC-1. When the busses were separated,
a voltage surge in excess of 200 VDC was generated. This surge was apparently due to a battery charger failure. The voltage surge propagated through.the SY-DC-2 125 VDC syste The 230 kV PCBs are each provided with two electrically and mechanically separate tripping coils for redundancy. Either trip coil energized will trip the breaker. When an abnormal condition, such as a fault, is detected by protective relaying on a PCB, the protective relaying circuit (different from the.breaker failure circuitry) energizes the tripping coils to trip the breaker and isolate the fault. The breaker failure circuitry also senses the fault through the overcurrent detector, and starts a timer. If the faulted breaker fails to trip within a certain time, an AR relay within the breaker failure circuitry operates to isolate and de-energize the PCB by tripping the adjacent PCB The AR relay within each PCBs' breaker failure circuit is connected across a DC power supply through a transistor and a zener diode in paralle The AR relay normally operates through the transistor, and the zener diode functions as a surge protective device f6r the transistor. The zener diode acts as a switch and momentarily conducts when it experiences excessive voltages (the peak inverse voltage for this type zener diode is greater than.180 VDC for greater than two milliseconds) and re opens when the voltage subsides. The zener diode is also connected in series with the AR coi While it conducts to suppress the voltage and protect the transistor, it also momentarily connects the AR relay coil across the power supply allowing the AR coil to draw current and consequently causes the breaker failure circuit to actuat The loads assigned to the 125 VDC system are such that all 230 kV switchyard PCB breaker failure circuits are powered from bus SY-DC-The voltage surge during breaker manipulation on October 19, 1992, resulted in the zener diodes conducting and causing the AR coils for several PCBs to draw current, thus actuating the breaker failure circuitry for these 230 kV switchyard PCBs. Of the breaker failure circuits that were actuated, at least one was on a breaker connected to the Red Bus and at least one was on a breaker connected to the Yellow Bus. The total number of circuits actuated was not know Loss and Lockout of Red and Yellow Buses, Switchyard Isolation, and Loss of Off-site Power (Refer to Appendix 2, Figures 1 and 3)
Each 230 kV PCB connected to the Yellow Bus has a contact from the AR relay in the 230 kV Yellow Bus Differential Lockout Relay (Yellow Bus Lockout) circuit. When any one of the breaker failure circuits on a breaker connected to the Yellow Bus actuates, the AR relay.contact activates the 230 kV Yellow Bus lockout. While the
~11*
AR relay from any PCB connected to the 230 kV Yellow Bus also could have actuated the 230 kV Yellow Bus lockout, the 230 kV Switchyard Events Recorder initially indicated "23/24 Breaker Failure". It was later concluded that the AR relay associated with PCB-24 caused PCB-23 to trip as well as initiate the Yellow Bus.lockout. PCBs-9, 12, 15, 18, 21, 24, and 27 and 30 tripped as a result of the Yellow Bus lockou In a similar manner, the voltage surge operated at least one AR relay of a PCB that was connected to the Red' Bus and actuated the 230 kV Red Bus Differential Lockout Relay (Red Bus lockout).
PCBs-4, 7, 10, 13, 16, 19, 22, 26, and 28 tripped as a result of the Red Bus lockou PCBs-31 which normally trips on Red Bus lockout, and PCB-33, which normally trips on Yellow Bus lockout, were tagged open to support maintenance and modification activities on the Auto Transformer between the 230 kV and the 525 kV switchyard The AR relay in PCB-24 also caused an Oconee Unit 2 generator transformer lockout (the Unit 2 generator feeds to the 230 kV switchyard through PCBs-23 and 24), which resulted in a turbine trip, and a subsequent reactor trip. When the N1.and N2 breakers opened, the El and E2 breakers closed momentarily to provide power to Oconee Unit 2 loads. However, the El and E2 breakers opened immediately on undervoltage because PCBs-26 and 27 had opened due to breaker failure relay actuatio The External Grid Protective System detected loss of voltage and frequency conditions on both the Red and Yellow busses indicating a loss of off-site power. The system proceeded to isolate the 230 kV switchyard by tripping designated switchyard PCBs. This is referred to as Switchyard Isolation. The Switchyard Isolation also gave an emergency start signal to both Keowee unit Since PCBs-12, 15, 21, 24, 26, and 33 were already open on a Red and Yellow Bus lockout, Switchyard Isolation only opened PCBs-8 and 1 The Switchyard Isolation also sent close signals to PCBs-9, 18, 27, and 30 to complete an isolated electrical path from Keowee, via the Yellow Bus to each Oconee Units startup transforme PCBs-9, 18, 27 and 30 tried to close momentarily but tripped again because of the existing Yellow Bus lockout. As a result of having both a trip and a close signal, PCBs-9, 18, 27, and 30 remained open and in anti-pum Each PCB has anti-pump circuitry to prevent repetitive cycling (pumping) of a breaker if continuous close and trip signals are simultaneously applied to the breaker control circuits. The Yellow Bus lockout made the Keowee overhead power path inoperabl With PCBs-26 and 27 open, Oconee Unit 2 was in a condition where no automatic off-site power was available. Keowee Unit 2 was coming up to spee Therefore, Oconee Unit 2 MFBs 1 and 2, and
thus.all AC loads for Oconee Unit 2 temporarily lost power. The MFBMPs for Unit 2 sent another emergency start signal to the two Keowee units and initiated a loadshed to enelgize the MFBs from the Standby Busse Oconee Unit 1 continued to operate supplying the Central transmission line through PCB-20. PCB-20, like PCBs-11 and 14 was not affected by either the Red or Yellow Bus lockout, Switchyard Isolation, or the breaker failure circuit actuation. However since PCB-17 had opened, off-site power would not have been automatically available in the event of a generator trip. Oconee Unit 3 continued to feed the 525 kV switchyard which was not affected by the voltage spike. Since PCB-26 had opened, Unit 3 too would have been without automatic off-site power in the event of a generator tri Initial Keowee Hydro Station Response Just prior to the event, Keowee Unit 1 was operating in Automatic Governor Control and supplying power to the grid via the overhead power path. Keowee Unit 2 was shutdown and lined up electrically to provide Oconee emergency power via the-underground power path (Refer to Figure 2).
Upon receipt of the Switchyard Isolation signal, both Keowee units received an emergency start. Keowee Unit 1 continued to run but was removed from the grid when its output breaker (ACB-1) opened, as designed. When ACB-1 opened, power was removed from the 1X and 2X busses and they automatically fast transferred to their alternate power sourc Keowee Unit 2 emergency started on Switchyard Isolation and was up to speed in approximately thirteen seconds. The MFBMPs sensed an undervoltage condition on MFB1 and MFB2 and sent another emergency start signal to the Keowee units and initiated loadshed of non essential Oconee loads, as designed. With Keowee Unit 2 up to speed, the CT-4 to Standby Bus breakers SKI and SK2 closed, as designed. The Standby Bus breakers S1 and S2 closed 10 seconds later, and energized the Oconee Unit 2 MFBs bringing power back to essential loads including HPI and MDEFWP. In total, the Oconee Unit 2 MFBs were without power for approximately 31 second Six seconds after receipt of the Switchyard Isolation signal, the Keowee Unit 1 output breaker closed and re-energized the overhead power path,up to PCB-9. With the normal power supply again available, the normal power seeking transfer feature of the"Keowee auxiliary busses triggered the fast transfer of these busses back to their normal power source. Though the timing of these transfers was found to be deficient in later testing, as discussed in section IV.B below, it caused only a temporary loss of Keowee Unit 2 auxiliary load The Keowee operator proceeded to the Keowee CR. Since Keowee Unit 1 had been generating to the grid, the operator's first reaction
was to check the status of that unit. He noted many alarms on the annunciator panels were flashin The Keowee operator also noted indications of Keowee Unit 1 operating atno load and manually opened ACB-1 in order to protect the Keowee Unit 1 from what he perceived to be a problem with the grid. Again the auxiliary busses transferred to their alternate power source; however, this time several problems occurred. During the 1X bus transfer, a bus lockout was received on 1X, preventing ACB-7!from closing and the bus remained de-energized. Also, during the 2X bus transfer, ACB 8 failed to close and that bus remained de-energized as wel The Keowee operator noted that the Keowee CR overhead lights went off and the Keowee Unit 1 and Unit 2 annunciator panels went dar This plant response was consistent with a loss of the iX and 2X busses. The Keowee computer alarm typer (but not the computer)
was also lost with the loss of auxiliary bus power. With both 1X and 2X lost, the Keowee units continued to operate with their control functions being supplied by the batteries (Refer to Appendix 2H).
The Keowee operator determined that Keowee Unit 2 was running in the emergency mode. He attempted to call the Oconee CR but the commercial telephone was inoperable due to loss of Keowe auxiliary power. He then contacted the Charlotte LD on the dedicated dispatcher line. He requested the LD ask the Oconee CR to immediately contact him about problems with the Keowee units and asked that the Keowee on-call technician be called to come to the Keowee sit A few minutes later the Oconee Unit 2 Supervisor contacted the Keowee operator on the dispatcher line. The Keowee operator explained the status of the Keowee units and that the units were running with only DC power available. The Oconee Unit 2 Supervisor directed him not to take any action involving Keowee Unit 2 since that unit was supplying the Oconee Unit 2 MFBs. The Keowee operator then monitored the operation of the hydro units awaiting the arrival of the on-call technicia When the technician arrived he determined a Keowee main step-up transformer lockout (86T), a Keowee Unit 1 normal generator lockout and a lX transformer lockout had occurred. Post-event review indicated that these lockouts may have opened ACB-1 instead of operation of the control switch by the Keowee operator. The technician also noted that ACBs-5, 6, 7 and 8 were open with the 1X and 2X auxiliary busses de-energize The Keowee technician informed the Oconee Unit 2 Supervisor of the Keowee situation and the need to restore auxiliary powe Permission was obtained to reset the Keowee 86T lockout. When 86T was reset, ACB-1 closed, providing power to transformer 2X causing ACB-6 to shut, thus re-energizing the 2X bus and restoring auxiliary power to Keowee Unit 2. The technician attempted to reset the 1X bus lockout and was unsuccessfu He then attempted
to manually close ACB-7 and was also unsuccessfu Local investigation revealed that.an overcurrent trip had occurred on ACB-7 which he reset The Keowee operator then reset the 1X bus lockout and manually shut ACB-7. This action re-energized the lX bus and restored auxiliary power to Keowee unit 1..The Oconee CR was informed that auxiliaries were restored to the Keowee unit Restoration of Red Bus from Off-site Powe At this point, Red and Yellow Busses were still locked out, Keowee Unit 1 was running with ACB-1 closed but was not tied to the Yellow Bus (due to PCB-9 being in anti-pump). Oconee Unit 2 MFBs were energized by Keowee Unit 2 through CT-4. Oconee Unit 2 CR operators had reset the MFBMP/Load Shed signals. Due to the limited capacity of CT-4 and the desire to restore RCS forced circulation, plans were made to restore off-site power and energize MFB1 and MFB2 from the startup transformer CT-The Oconee Unit 1 Supervisor reset the Red and Yellow Bus lockout from the switchyard relay house. This action took the open signal away from PCBs-7, 10, 13,.16, 19, 22, 26, 28 on the RED Bus and PCBs-9, 12, 15, 18, 21, 24, 27 and 30 on the Yellow Bu Resetting the lockout does not automatically close the PCB The Red Bus was energized by closing PCB-10. Red Bus PCBs-7, 13, 16, 19, and 22 were also closed, thus tying the Red Bus to the majority of the transmission gri Having restored the Red Bus, the Oconee operator attempted to tie the Red Bus to CT-2 by closing PCB-26. PCB-26 closed momentarily but opened immediately because the Switchyard Isolation signal was still present. Although the personnel in the CR were aware of the Switchyard Isolation signal, the need to reset the signal prior to closing PCB-26 was not recognized. When PCB-26 was momentarily closed, it took away the Switchyard Isolation Complete signal, thus clearing the anti-pump signal on PCB-9. When PCB-26 tripped open again, Switchyard Isolation Complete signal.was again generated allowing PCB-9 to close. Keowee Unit 1 now energized the Yellow Bus through PCB-9. The Switchyard Isolation signal was then reset, allowing closure of PCB-26. PCB-26 was manually closed and CT-2 was energized by the Red Bu Loss of Keowee Units while Restoring Yellow.Bus to Grid The Yellow Bus was energized from Keowee Unit 1. A discussion was held between Oconee CR personnel and the Keowee Supervisor regarding the appropriate method to restore the Yellow Bus to the grid. It was decided that a dead bus transfer after shutting down Keowee Unit 1 would be conducte Coordinating with the Oconee CR, the Keowee Unit 1 normal lockout was reset. The Oconee operators then reset the Keowee emergency
start signa The Keowee Supervisor noted that the lX bus was operating on its alternate power source. Recognizing that normal power was available, the Keowee operator attempted to close ACB-5 but was unsuccessfu The Keowee Supervisor investigated the ACB-5 breaker and while attempting to reset an apparent overcurrent trip, accidentally generated another lX bus lockout signa Keowee Unit 1 auxiliary power was again lost. The IX bus lockout was quickly reset by the Keowee operator and the Keowee Unit 1 auxiliary bus re-energized by closing ACB-After resetting the Keowee emergency start signals, the Oconee CR directed a normal shut down of Keowee Unit 1. Upon shutdown of Keowee Unit 1, the main step-up transformer became de-energize Due to the de-energized overhead feeder interlock feature, which had been overlooked, an undesired shut down of Keowee Unit 2 occurred. Since Keowee Unit 2 was supplying the Oconee Unit 2 MFBs, Oconee Unit 2 again lost AC powe The Unit 2 MFBMPs sensed a loss ofvoltage and properly restarted the loss of power sequence. Both Keowee units received another emergency start signal and starte The Red Bus lockout had previously been reset and the Red Bus, connected to the grid. The Retransfer To Startup Logic actuated and tied the MFBs to the off-site grid by closing the El and E2 breakers. This loss of power on the MFBs was for 31 seconds as the EPSL logic reenergized the busse Keowee Unit 1 emergency started but ACB-1 did not close because a Switchyard Isolation condition was no longer present. When Keowee Unit 2 emergency started, its field breaker did not close and this condition lasted for approximately-1 1/2 hours. The field breaker stayed open due to the anti-pump relay. Refer to section IV. for detail Return of Electrical Distribution System to Normal Alignment The Keowee units continued to operate for 1 1/2 half hours in the condition discussed above. During this time period, Oconee Units 1 and 3 were without the automatic AC power, transfer of the startup transformer to off-site power in the event of a reactor trip. A decision was made not to energize the Standby Busses from CT-5 during this period. After the Lee gas turbine had energized transformer CT-5, the Oconee operators reset the emergency start signal after which both Keowee units were shut down. Shutdown of Keowee Unit 2 allowed the field breaker X-relay to automatically reset and Keowee Unit 2 was subsequently started to re-energize transformer CT-4 until the Oconee switchyard was restore The Oconee operator reset the MFBMP/Load Shed signals. The Yellow Bus was re-energized from the Red Bus by closing PCB-Subsequently, PCBs-18, 27, 30 21, 17, 28, 12 and 15 were close This completed the restoration of the 230 kV switchyar Overall Oconee Response -to the Event With the loss of power and reactor trip of Oconee Unit 2, Emergency Operating Procedure, EP/2/A/1800/01 was entered. The loss of power to the MFBs resulted in the loss of the RCPs, the condensate pumps and a subsequent loss of main feedwater pump The TDEFWP automatically started to maintain steam generator levels. Abnormal Operating Procedure AP/2/A/1700/11, Loss of Power, and AP/2/A/1700/19, Loss of Main Feedwater, were also entere Oconee Unit 2 remained in natural circulation flow with plant parameters within expected values. The team reviewed the post reactor trip package and concluded that most major plant equipment performed as expected during the transient. The 2A RBCU tripped while shifting to high speed and could not be restarted. An instrument air line break occurred on the operator of condenser discharge valve 2CCW-24, causing the valve.to go shu Pressurizer spray valve, 2RC-1, did not shut automatically at the required 2155 psig setpoint and was subsequently shut by the operators at 2145 psig. The team verified that Work Requests had been initiated for this equipment and prior to restart all repairs had been accomplishe Following recovery of off-site power, forced circulation was established by starting the RCPs. No safety limits were exceeded during this even Emergency Notification/Classification The licensee notified the NRC of an Unusual Event at 10:25 p.m.,
in accordance with 10 CFR 50.72. The Unusual Event condition had existed since 9:21 p.m. The licensee classified the event per Emergency Classification Procedure, RP/O/B/1000/01. The licensee also notified the resident inspector at approximately 10:35 The team concluded that the event was appropriately classified and communicated to the-NR IV. EQUIPMENT MALFUNCTIONS OR ANOMALIES, INSPECTION, TESTING, AND MODIFICATIONS ACTIVITIES Equipment Malfunctions or Anomalies The various equipment problems experienced during the event are discussed individually below.Section II, Sequence of Events and,Section III, Detailed Discussion of Event briefly discussed these failure.
230 kV Switchyard SY-2 Battery Charger Post-event testing of the SY-2 battery charger identified that the DC terminal voltage was fluctuating. These voltage fluctuations occurred quite rapidly and were not readily discernible by observation of the normal charger indications since they were dampened by the response time of the mete It is suspected that the battery charger SY-2 output voltage was oscillating when it was powering switchyard DC bus SY-DC-2 without the battery just prior, to the event. The malfunctioning of the battery charger was due to an internal defect discovered after the event. Additionally, the charger was not designed to function under this condition The manipulation of the cross-tie breaker between the DC busses SY-DC-1 and SY-DC-2 is also suspected to be a possible source of the voltage surge.Section IV. discusses testing that confirmed that voltage surges caused by the charger were sufficient to activate the breaker failure circuit relay The licensee inspected the SY-2 battery charger and found its reference voltage circuit to have a defect that rendered this feature functionally inoperable. Specifically, a cracked solder lead on a capacitor was found in the reference voltage circuit. The battery charger voltage control normally obtains its reference voltage from the battery. Following removal of the battery, the reference voltage circuit would provide a voltage reference from which the charger would regulate its output voltage. The failure of the reference circuit could explain the observed voltage fluctuations. The lead was re-soldered. While the charger output voltage was still fluctuating after the repairs, the magnitude of these fluctuations had been reduced. The battery charger investigation was continuing at the conclusion of the inspection. The spare charger was being used in the interi It was also noted that the charger was rated for 50 amperes, with a 125% current limiting feature ( i.e., an internal control system limits the output to a maximum of 62.5 amperes).
The capacity (sizing).of the battery charger was based on it being capable of fully recharging the battery in eight hours and concurrently supplying continuous load current. The battery load profile as.a function of time was 341 amperes from 0 to 1 minute, 54 amperes from 1 to 59 minutes and 304 amperes from 59 to 60 minutes. Thus the battery charger was not sized to fully replace the battery as it was expected to do in the configuration it was placed. It was noted by the team that a more thorough review of the battery modification package may have prevented the charger from being placed in an alignment for
which it was not sized. This error apparently contributed to the initiation of the even.
230 kV Switchyard Breaker Failure Circuit Numerous 230 kV switchyard PCB breaker failure circuit relays were operated as a result of the DC voltage surg Details of the breaker failure circuit operation are discussed in section III.A. Testing of the modification of this circuit is discussed in section IV.B.2. Westinghouse Product Reliability Letter 79-47, issued in 1979 discussed the susceptibility of the zener diode in the output of the breaker failure circuit to conduct if the applied voltage were too high. This could cause inadvertent operation of the breaker failure circuit. Duke Power engineering had reviewed the Westinghouse letter and specifically identified the need to modify to some circuits as recommended by Westinghouse. However, no action was taken to implement these change Subsequent to this event, the recommended modifications were completed on all safety-related PCBs prior to the restart of Oconee Unit.
230 kV Switchyard Events Recorder The 230 kV switchyard events recorder did not record all the data necessary for a detailed re-creation of the event due to capacity limitations. A modification is planned for implementation by December 31, 1992, which will replace the device with an upgraded mode.
Keowee ACB-7 Failure to Close During the event, the Keowee on-call technician found ACB-7 tripped and the lX auxiliary bus lockout relay actuated indicating an overcurrent situation had occurred. However, there were no overcurrent targets picked up in the breaker's Amptector thus indicating that the Amptector device did not see an overcurrent conditio ACB-7 is a Westinghouse type DB-50 breaker that provides power to the Keowee Unit 1, 600 VAC Load Center 1X from transformer CX,. The Amptector is an overcurrent sensing device within the breaker that electrically initiates the Amptector actuator, which in turn mechanically.trips the breaker. The Amptector also electrically releases a plunger which actuates the bus lockout relay. There is a separate tripping actuator on the breaker for other tripping operations and these would not normally interfere with the Amptector actuator mechanis After the event the licensee initially suspected two potential causes of the breaker failing to close. The first involved an overcurrent condition which caused the Amptector overcurrent protective device to operate and hold ACB-7 open. The other possible cause was that mechanical problems internal to the ACB caused the Amptector actuator mechanism to malfunction. The licensee investigated and dismissed the possibility of the overcurrent condition after inspection of the potentially affected components, fault current analyses, and successful re-energization of the bus. The licensee was in the process of reviewing the integrity of the Amptector actuator at the conclusion of the inspectio The most probable cause of ACB-7 failing open was that the Amptector actuator inadvertently actuated the lockout relay following a series of repetitive breaker operations, which occurred as load center IX lost and regained power during the event. The licensee was pursuing this investigation at the conclusion of the inspection. In the interim, ACB-5, 6, 7, and 8 have been placed in a configuration that will not unnecessarily challenge them with rapid, repetitive operations (i.e., automatic transfers are not required). Keowee ACB-8 Failure to Close During the event, ACB-8 failed to close when required on one occasion. ACB-8 is a Westinghouse type DB-50 breaker that feeds the Keowee Unit 2, 600 VAC Load Center 2X from transformer CX. The breaker failed to close because of a high resistance in a closed contact (close permissive) of the MG-6 relay in its control circuit. The previously recognized X-Relay problem may have also compounded the proble A MG-6 relay malfunctioned and failed to provide the permissive to close ACB-8 during the event. The relay is used to indicate the presence of supply voltage prior to permitting automatic closure of the normal breaker. The relay itself operated but the contact required to give the voltage permissive had excessive resistance. A similar problem had been identified by the licensee in September 199 Beginning on December 31, 1992,' a review of all safety-related applications of the relay will be conducte The X-Relays are the anti-pump relays used in Westinghouse type DB breakers. The anti-pump feature is an electrical mechanical device that allows the breaker to receive only one close signa This feature prevents the breaker from cycling repeatedly between the closed land tripped position The X-Relay is operated by a coil which is energized on any one of several closed signal (The coil pulls in a latch
arm assembly, which in turn pulls the-X-Relay contacts closed.)
The ACB close coil is operated through the X-Relay contacts. The breaker trip circuit.is activated and should the ACB subsequently receive a trip, with a close signal still present, the ACB will open and not close. Thus, the ACB is in anti-pump. When the close signal is removed, the coil is de-energized and the latch arm assembly is free to slide back, latch in its de-energized position (i.e.,
gravity resets the relay), and reactivate the close circui Dirt, dust, or lack of lubrication can cause the mechanism to bind and not reset. As this condition is not automati cally detectable, the Keowee operators have been verifying that the relay has reset after each operation of the Keowee units. Modifications had been completed.on Keowee Unit 1 to replace the electrical mechanical anti-pump feature with an electrical anti-pump scheme. These modifications are scheduled to be completed on Keowee Unit 2 by December 3, 199.
Undervoltage Trip of Keowee Unit 2 In the process of restoring the 230 kV switchyard, Keowee Unit 1 was intentionally shutdown to de-energize the Yellow Bus and Keowee Unit 2, which was supplying power to Oconee Unit 2 through the underground path, tripped unexpectedl This was an unanticipated system interaction caused by a design feature of which personnel were not awar A voltage relay for each Keowee unit senses the voltage on the overhead power path and when the voltage is lost it will trip each Keowee unit. This design feature (also known as the "de-energized overhead feeder interlock") will be bypassed when an emergency start signal is sent to the Keowee units. However, when it was decided to shutdown Keowee Unit I the emergency start signal had been rese Thus, Keowee Unit 2 tripped when Keowee Unit 1 was shutdown, because there was no voltage on the overhead path. The design function of this feature was to prevent the waste of the water in the reservoir, because the unit would be spinning but not generating power. This design did not consider that Keowee may be supplying emergency power through the underground power pat A modification was installed by the licensee to remove this design feature shortly after the conclusion of this inspectio.
Keowee Unit 2 Generator Field Breaker Failure to Close This ACB is.a Westinghouse DB-25 which provides the field to the generator. The breaker failed to close as a result of a
design feature involving a speed switch and anti-pump devic The Keowee Unit 2 generator field breaker has a speed switch in its close circuit. Provided the unit does not have an emergency start signal present, the speed switch closes at 52 rpm on start up and allows the generator field breaker to close. The switch also opens when the unit coasts down to below 52 rpm. An emergency start signal disconnects the speed feature and provides a close signal to the generator field breaker. Specifically, one set of relays in the emergency start system breaks the speed circuit and another set of relays provides a closed signa Following the trip of Keowee Unit 2 (as discussed in paragraph 6 above), its generator field breaker tripped, as designed on a unit trip. However, the generator field breaker close coil remained energized, because the unit was still above 52 rpm. Thus, with a trip and a closed signal present the breaker went into anti-pump and the generator field breaker remained ope A Keowee Unit 1 modification had been completed to correct this condition prior to the event. This same modification is to be implemented on Keowee Unit 2 by December131, 199.
Keowee Telephone, Computer Alarm Typer and Alarm Panels The Keowee telephone, computer alarm typer and alarm panels failed to function while electrical power was lost to,the Keowee AC auxiliary busses. The unavailability of this equipment hampered the Keowee operator in responding to the emergency start of the Keowee units. The operator had a dedicated phone line to the load dispatcher in Charlott The computer alarm typer was permanently connected to an uninterruptible power supply shortly after the event. The power supply to the telephone and the alarm panels will b evaluated by the license Equipment Inspection, Testing and Modification Activities Testing to Confirm the Initiating Event The team observed testing to support the determination of the initiating event. The SY-2 battery charger was tested in an attempt to duplicate the effects postulated during the event. Prior to testing, the licensee determined that the circuit had not been modified per Westinghouse Product Reliability Letter 79-4 A resistive load drawing approximately 20 amperes was placed on the charger and a strip chart recorder was used to measure the voltage output of the battery charge A voltage range of 70 to 215 volts was measured. After it was confirmed that the battery charger was producing voltage surges which could have led to the breaker failure circuit actuation, the charger was used to test the PCB-23 breaker failure circuit. As suspected, when the SY-2 battery charger was connected to the PCB-23 breaker failure circuit, the relay actuate.
Testing to Confirm Adequacy of the PCB Breaker Failure Circuit Modification A design modification was' developed in accordance with the recommendations of Westinghouse Product Reliability Letter 79-47. To demonstrate that the modification of the breaker failure circuitry would correct the problem, the licensee modified and tested the PCB-23 breaker failure circuitr This test was witnessed by team members. The SY-2 battery charger was used in order to test the-modified circuit under the same conditions as when the failure had occurred. The modified breaker failure relay circuit did not actuat Subsequently, modification package TN/A/OE/4688/00 was implemented which installed the zener diode modification on the remaining Red and Yellow Bus PCBs. This was accomplished on all safety-related PCBs prior to restart of Oconee Unit.
Inspection and Testing of PCB-24 Additional inspection and testing was conducted on PCB-24 prior to restart of Oconee Unit 2 due to the Oconee Unit 2 generator trip. This breaker was tested to assure that the PCB's insulation system had.not degraded. The PCB was visually inspected and preventive maintenance performe The inspection, maintenance and test of the breaker revealed no problems with the breaker and proper operation was demon strate.
Inspection and Testing of Keowee ACB-5 Keowee ACB-5 is a Westinghouse type DB-50 circuit breaker that feeds the Keowee Unit 1, 600 VAC Load Center IX from the Keowee main step-up transformer. Following the apparent overcurrent trip on breaker ACB-5 (as discussed in III.E), a test and inspection was conducted to ensure that no damage had occurred during the transient and establish continued operability of the circuit breaker. The breaker was operated both manually and electrically and satisfactory operation was demonstrated. These activities were witnessed
- by team members. The investigation revealed no damage or unsatisfactory condition in the breaker mechanism.
Testing of the Keowee Auxiliary Bus Transfer Function Team members witnessed the testing that was performed to verify the adequacy of the automatic transfers of the Keowee auxiliary busses. The transfer of lX and 2X to the alternate supply through ACB-7 and ACB-8 respectively performed as designe The transfer back to the normal power supply through the Keowee main step-up transformer did not occur as designe Following the re-energization of the normal power source, the design requires a 10 second time delay before making the fast transfer. For 1X, ACB-7 opened and ACB-5 closed instantaneously. For 2X, ACB-8 opened instantaneously and after 6 seconds ACB-6 closed. The 10 second time delay is intended to prevent rapid transfers which could potentially cause mechanical damage. It was the rapid transfer of lX that most likely caused the lockout of 1X during the even The undervoltage relays and transfer timers for Keowee breakers ACB-5, 6, 7 and 8 were individually tested. The 10 second timers were found to be working correctly. The MG-6 undervoltage relays were found to need contact adjustment and cleaning. The contacts on the MG-6 relay that activate the 10 second timer for the' bus transfer, were found to be out of adjustment and did not operate the 10 second timer for the 1X bus transfer. The contacts that provide a close permissive to ACB-8 were found to have excessive resistance (3000 ohms) across the closed contacts and could have caused the breaker not to clos The team noted that while the transfer function was periodically tested, the 10 second time delay was not included in the acceptance criteria for successful completion of the tes LICENSEE ACTIONS PRIOR TO OCONEE UNIT 2 RESTART Confirmation of Action Letter On October 21, 1992, a Confirmation of Action Letter was issued for Oconee Unit 2 by the Region II office of the NRC. This letter reiterated the commitments of a previous telephone conversation during which Duke Power Company had stated that they were actively pursuing the root cause of the event and that the followin actions would be taken. The licensee would: Determine the initiating cause of the even.
Determine the cause of any electrical malfunction including PCB-2.
Determine-the cause of the trip of Keowee Unit 2 when Keowee Unit 1 was taken off lin.
Determine the cause of the loss of auxiliary power to the Keowee generator The letter also stated that once an understanding of these events had been determined, a conference call would be held with the NRC prior to returning Oconee Unit 2 to power operation 'On October 26, 1992, a conference call was conducted during which the licensee summarized their response to the Confirmation of Action Letter. Following the conference call, Duke Power Company issued a letter dated October 27, 1992, regarding enhancements to the Keowee Hydro Station operator performance during emergency operations. This letter also documented commitments regarding the alignment of the Keowee auxiliary power system and the availability of an additional source of power to the Oconee Standby Busses. The process which is underway to enhance Keowee operator performance is summarized below: An abnormal procedure has been developed to provide operator response to a Keowee emergency start. Validation of the procedure has been performed. Job Performance Measures which establish training objectives and evaluation criteria are under development by the Oconee site training group for critical task.
The role and responsibilities of the Keowee operator will be documente.
A licensed Oconee operator will work, during the entire shift, with Keowee operators to stress watchstanding. This
"peer assessment" began on October 26, 1992. The licensed Oconee operator will remain until the Keowee exhibit adequate performance in this area as determined by Oconee site managemen.
Enhancements to the Oconee loss of power abnormal procedure are in development. Written operator guidance has been established to enhance the ability to recover from a Switchyard Isolation. Further-testing will be performed prior to implementing the changes to the procedur.
Senior Duke Power Company management will review the relationship of the Keowee Hydro Station organization to the Oconee site organization to determine the most effective organization for assuring the function of the Oconee
emergency power source. (The Keowee organization was realigned to report to Oconee Operations effective 11/9/92.) Keowee Electrical System Realignment As a result of the Oconee Unit 2 loss of power event on October 19, 1992, several weaknesses or deficiencies in the Keowee auxiliary load center automatic transfer circuitry were identified. On October 22, 1992, a conference call was conducted to discuss the proposed alignment of the Keowee auxiliary power system. The intent of the alignment was to meet the requirements of the TS. This alignment put one Keowee Unit and dedicates its auxiliaries to the overhead emergency power path and aligns the other Keowee Unit and dedicates its auxiliaries to the underground emergency power path. The Oconee TS permits 'operation in this configuration. This configuration is discussed below (refer to Figure 2):
-
Underground Path One Keowee hydro unit will be normally aligned and dedicated to supply.power to the underground power path after an emergency start. The auxiliary load center for this hydro unit was placed in manual and energized from the alternate power source (ACB-7 or ACB-8 closed).
Overhead Path The other Keowee hydro unit will be available to supply power to the overhead path after an emergency start. The auxiliary load center for this hydro unit was.also placed in manual and would be energized from the normal power source (ACB-5 or ACB-6 closed).
The above realignment removed the automatic transfer capability of the Keowee auxiliary load centers (IX and 2X) and effectively powered each load center from its respective hydro generato This arrangement of the Keowee power system was intended to be flexible and determination of which hydro unit would be dedicated to the underground path would depend on the needs of the sit A letter was provided to the NRC on October 23. 1992, which provided the basis for acceptability of this alignment, including consideration of the October 19, 1992, transient with regard to the revised alignment and included the following measures:
-
Procedures were developed, including the associated 50.59 safety evaluation, to direct the Keowee operators to.align the Keowee auxiliary power supplies as' described above. The Keowee operators were trained on the use of these procedures prior to exiting TS 3.7.2(a) II
Transformer CT-5 will be maintained energized and available to the Oconee standby busses from either the Central switchyard or the dedicated line from a gas turbine at the Lee steam station. CT-5 will be maintained in this condition until the relays, which may affect operation of Oconee, are modifie The Keowee Hydro Station will not resume normal operations (i.e., generate to the system grid), except for TS required testing, until further discussions with the NRC staff are hel Based on this information the Keowee electrical systems were realigned and the limiting condition for-operation of Oconee TS 3.7.2(a)2 was exited on October 22, 1992. It was also agreed that the Keowee Hydro Station could resume generating to the system gri Procedures and Training Before the October 19, 1992, event, Keowee hydro station procedures existed for maintenance activities, normal hydro generator operations and alarm response. Also, an abnormal procedure existed for natural disasters. However, the Keowee operators had-no specific procedure for responding to or verification of an emergency.start of the Keowee hydro units. The lack of such a procedure apparently contributed to problems that occurred in the operation of the Keowee units during the loss of power event (refer to III.C).
As part of the immediate corrective actions, Oconee with the help of Keowee developed abnormal procedure AP/O/A/2000/002, Keowee Hydro Station - Emergency Start, to provide guidance to the Keowee operators during such an even The new procedure provided operator guidance for Keowee emergency start symptoms, automatic system actions, immediate manual actions and subsequent actions. This guidance would allow the Keowee operator to recognize a Keowee emergency start condition and verify proper system operation. Additionally, the procedure provided direction should the units not respond in the expected manner. Because of the difficulties experienced with the Keowee auxiliary load centers and the associated transformer lockouts, the subsequent-actions section of the procedure provides response to problems associated with this equipmen This abnormal procedure was developed and approved, with training provided to the Keowee operators before Oconee Unit 2 was restarted. During the special prestartup test of the Keowee units (refer to V.D),.the new procedure was used as part of the tes It provided useful information for verification of proper system operation and for response to load center problems during a Keowee emergency start. However, while it provided proper guidance, the procedure was not actually used by the operator when Keowee Unit 1
tripped and auxiliary load center power problems developed during restoration from the tes The procedure was apparently not used because the emergency start signal had been reset to allow manual operator control of the hydro generator. Thus when the unit subsequently tripped, the entry condition for using the new procedure (Keowee Emergency Start) was no longer met. The licensee indicated that further improvements would be made-to the procedural guidanc Keowee Special Test Before restart of Oconee Unit 2, a special load rejection test was conducted with the Keowee Hydro Generators on October 25, 199 This test was witnessed and the procedure and results were reviewed by the team. The test was performed according to procedure TT/O/A/0620/02, Keowee Hydro Load Rejection Test. It was intended to verify that Keowee Unit 1 could continue to operate properly in the emergency mode following a full load rejection incident (similar to the event that occurred on October 19, 1992).
The test was also intended to test the new Keowee abnormal procedure and a new Oconee procedure for "live" bus transfer, from the Oconee CR, of the overhead power path loads back to the switchyard. The lack of a procedure for "live" bus transfers from the Oconee CR was the reason the Oconee operators directed the shut down of Keowee Unit 1 during the event (resulting in the unintentional automatic shut down of Keowee Unit 2). initial Conditions The 230 kV switchyard was energized from the Duke Power transmission system. Oconee Unit 1 was operating and tied to the grid with both the Red and Yellow switchyard busses energized. Oconee Unit 3 was also operating but was tied to the 525 kV grid. All three Oconee startup transformers (CT-1, CT-2 and CT-3) were in their normal alignment and tied to the switchyard. Transformer CT-5 was energized from the Lee gas turbine via the dedicated line. Keowee Unit 1 was aligned to the overhead power path and Keowee Unit 2 was aligned to the underground power path. Both Keowee units were shut down but available for emergency start. Also, both Keowee auxiliary busses were aligned to their respective hydro generator with the transfer switch in manual (refer to section V.B).
PCB-9 was open and the local DC control power breaker to its close coil was also open. This condition was established to prevent Keowee Unit 1 from actually tying to the Yellow Bus since the Yellow Bus would remain energized from the grid throughout the test and Keowee Unit 1 would not be synchronized with the gri Additionally, the 230 kV External Grid Protection System was
placed in test for all switchyard PCBs, except PCB-8, to simulate a Switchyard Isolation Complete signa.
Test Methodology The test procedure would align Keowee Unit 1 to the Duke Power transmission grid at full load. Oconee operators would then open PCB-8 to separate Keowee Unit 1 from the grid and initiate a Keowee Emergency Start signal from the Oconee CR. Since PCB-9 was already open, this sequence would result in a load rejection condition. Both Keowee units would emergency start. Since a Switchyard Isolation Complete signal had been simulated as an initial condition, Keowee Unit 1 would energize the overhead path up to PCBs-8 and 9 while Keowee Unit 2 would.energize the underground path to transformer CT-4. The Keowee Unit 1 auxiliary bus would be lost.for about six seconds until ACB-1 reclose System restoration was to be performed by resetting the emergency start and Switchyard Isolation Complete signals, synchronizing and tying Keowee Unit 1 to the grid through PCB-8 (from the Oconee CR) and then conducting a.normal shut down of the Keowee unit.
Test Results Generally, the Keowee units performed as expected during the test. Keowee Unit 1 handled the'load rejection without difficulty. Both Keowee units emergency started and tied to their respective paths. The Keowee auxiliary busses also performed as require Some weaknesses were noted during the test in the area of the Oconee operators' proficiency at remotely operating the Keowee units. The Oconee operators had difficulty in controlling Keowee Unit 1 while initially tying it to the grid and while paralleling the overhead path to the grid during system restoration after the test. These weaknesses led to the inadvertent tripping of Keowee Unit 1, and consequently a trip of Keowee Unit 2 (refer to section VI.B.1).
Additionally, several unexpected Keowee annunciators, which involved abnormal conditions, were received. While none directly affected the performance of the Keowee units, the Keowee operator did not demonstrate familiarity on how to handle these conditions (refer to section VI.B.2).
The system's response during the test and during the subsequent loss of Keowee Units 1 and 2 was similar to that experienced during the loss of power event on October 19, 1992, and served to validate many team conclusions regarding the sequence of event The "live" bus transfer procedure and training were not adequate as indicated by the problem In addition, during a review of portions of the Design Bases Document for the Keowee Emergency Power System, the team noted that the Keowee Overhead Emergency Path was not periodically tested to verify operabilit V ANALYSIS OF RELATED ISSUES Procedure Inadequacies The team reviewed procedures related to the required operator responses in the Oconee Control Room and at the Keowee Hydro Station, including normal, abnormal, annunciator and emergency operating procedure.
Abnormal Procedures at Oconee Unit 2 The primary abnormal procedure used in response to the loss of power, AP/2/A/1700/11, Loss of Power, did not provide adequate guidance to the operators for responding to this event. The procedure did not require verification of the proper operation of the Keowee hydro generators, either from available Oconee indications or from the on-shift Keowee operator. Although the procedure required verification of Switchyard Isolation, no guidance was provided for recovery from an improper lineup, such as the one that occurred during this event. No cautions or guidance existed in the procedure to ensure that power was available to the non affected units before resetting the emergency start signal and securing the Keowee units. In summary, the abnormal procedure provided some guidance in responding to the initiating event, but very limited guidance in recovering from the even The operators also entered AP/2/A/1700/19, Loss of Main Feedwater, and AP/2/A/1700/22, Loss of Instrument Air, during this event. Main feedwater was lost because of the loss of power. The instrument air pressure decreased when the instrument air compressors were de-energized on the loss of power. Both procedures were found adequate for responding to this event. The team noted that AP/2/A/1700/19 directed the operators to secure the TDEFWP when it was verified that both MDEFW pumps were operating satisfactorily with less than 500 gpm per header. No caution or note was included to ensure that a reliable and stable source of power was available to the MDEFWP prior to securing the TDEFW.
Procedures at Keowee Hydro Station The team reviewed procedures available to the operators at the Keowee hydro station. These procedures consisted of normal operating procedures, annunciator response procedures, and one abnormal procedure providing guidance for response to natural phenomena such as earthquake The normal operating procedures appeared to be adequate for the routine operation of the Keowee generators. No guidance was provided in these procedures for response to abnormal system configurations, nor was there additional guidance for action in case of an emergency start of the Keowee generators. Because all of the guidance in these procedures was part of the operators' daily routine, the Keowee operators normally did not refer to the normal operating procedures. Except for annunciator response procedures, these procedures were not available in the Keowee contro room but were located in the supervisors office. They operated from memory and their knowledge of the system Annunciator Response Procedures were available for most alarms in the Keowee Control Room. These procedures generally described the alarm and provide directions to acknowledge and log the alarm. However, the procedures did not usually provide definitive guidance as to appropriate operator actions in response to the alarm. Normally, the only action in response to an alarm was to inform the dispatche With the exception of the abnormal procedure for responding to naturally occurring events, there were no abnormal or emergency operating procedures at Keowe Additionally, there was no formal guidance documenting the role and responsibilities of the Keowee operators. Guidance on absence from the Keowee area, log keeping, turnovers and other watchstanding procedures were implemented by word of mouth. Although the operators appeared to understand their responsibilities, without documented guidance it would be difficult to ensure uniformity in carrying out management expectations in this are.
Normal and Emergency Operating Procedures at Oconee Unit 2 EP/2/A/1800/01, Emergency Operating Procedure, provided the emergency guidance in response to the reactor trip. The procedure was adequate for responding to this event. It included ensuring that adequate core cooling was being maintained during natural circulation.. A review of the principal normal operating procedures used during this event did not identify any significant deficiencies. The
procedures reviewed were OP/O/A/1107/03, 100 KV Power Supply; OP/0/A/1107/11, Removal and Restoration of Auxiliary Electrical Systems; and OP/0/A/1106/19, Keowee Hydro at Ocone Personnel Knowledge and Training Oconee Nuclear Station During the recovery portion of the event, several incidents occurred which suggested that personnel were not fully aware of some of the involved systems details. Most of these problems occurred after additional personnel had responded to the site. The fundamental knowledge of the systems possessed by key personnel and the general sound approach to restoration prevented more serious problems from developing as a result of the error To restore off-site power capability to the Oconee Unit 2 MFBs, an attempt was made to close switchyard breaker PCB-26 before the Switchyard Isolation signal was reset. This action resulted in the unintentional shutting of PCB-9 and re-energization of the Yellow Bus from Keowee Unit While no serious problems resulted from this error, personnel did not recognize that the Switchyard Isolation circuitry included an open signal to PCB-26. Given the existing conditions and the absence of procedural guidance, the general approach of attempting to realign "preferred" power was appropriat During attempts to restore power to the Oconee Unit 2 MFBs from a source other than CT-4, Keowee Unit 1 was shutdown by the operators. When Keowee Unit 1 was secured, Keowee Unit 2 shutdown unexpectedly resulting in a temporary loss of power to the Oconee Unit 2 MFBs. The trip of Keowee Unit 2 was due to the "de-energized overhead feeder" interloc The existence of this interlock was overlooked during formulation of the recovery plan following the initial event. The team noted that Oconee personnel had previously restored the availability of power from the off-site transmission grid (Red Bus to CT-2) before attempting to manipulate Keowee units. The Oconee Unit 2 MFBs were automatically re-energized from this source on the shutdown of.Keowee Unit 2. As with the above error, the absence of adequate procedural guidance played a significant role in this proble The team noted that the MFBs of Oconee Units 1 and 3 would not have been automatically re-energized during a significant portion of the recovery phase, if a Unit 1 or Unit 3 reactor trip had occurre (The Keowee units were not automatically available and the PCBs from the grid to
CT-1 and CT-3 had not been shut.)
Although a feature intended for degraded grid protection would have provided automatic power restoration-to the MFBs if a LOCA had occurred, a cognizant decision was made not to shut the PCBs from the Red Bus to the Oconee Unit 1 and Unit 3 startup transformers during this portion of the recovery phas Additionally, the SL breakers had not been closed in to energize the Standby Busses from CT-5 which would provide an automatic power path to the MFBs. Information provided to the team showed that the Oconee Unit I and Unit 3 operators had specifically discussed the fact that the PCBs from the grid to the startup transformers had been left open and that manual actions would be required to restore power if neede The team concluded that a lack of procedural guidance addressing the existing plant conditions and the overall complexity of the interacting systems contributed to the problem The team also noted that on-site Oconee personnel did not possess sufficient knowledge of the Keowee facility to provide assistance to the Keowee operators when unexpected problems occurred. Efforts to restore power to the Keowee auxiliary busses were delayed until the Keowee on-call technician arrived on-site. The team recognizes that resetting of electrical bus lockouts needs to be done with a deliberate decision.process. However, some ability to accomplish necessary actions to ensure the operability of the emergency power sources was not available on-sit The team concluded that the primary cause of these problems was the lack of appropriate procedural guidance and the overall complexities involved with the various electrical power systems interacting with each other. Given the circumstances, it appears that the general restoration approach of the Oconee operators was reasonable. The team concluded that the restoration action plan could have been more carefully planned, which could have prevented some of the problems encountere.
Keowee Hydro Station During the event, deficiencies involving the operation of the Keowee hydro units occurred. Several of these deficiencies were considered significant performance problems by the team. Following the initial Keowee emergency start signal, the Keowee operator opened ACB-At that time, he failed to realize that an emergency start signal had been initiated. This incorrect action could have interfered with the Keowee unit being able to perform its safety function. On October 25, 1992, the licensee completed installation of a modification which provided a distinct indication (large red light). of a Keowee emergency
start on the Keowee CR panels. Additionally, a new Abnormal Procedure (AP/O/A/2000/002, Keowee Hydro Station - Emergency Start) was developed to provide specific directions to the Keowee operator when an emergency start occur While the Keowee operator rapidly overcame the loss of phone communications by utilization of the dispatcher lines to contact the Oconee CR, the extent and :significance of the loss of Keowee auxiliaries busses were not adequately communicated to the Oconee CR personnel. The team concluded that Oconee CR personnel were not aware that the Keowee units had operated for a long period (Keowee Unit 1 about 43 minutes, Keowee Unit 2 about 30 minutes) without AC auxiliarie The length of time that the Keowee units can run without AC auxiliaries is limited by each unit's ability. to maintain hydraulic control oil pressure to the governor. This controls the wicket gates of the turbine. Without AC power, preloaded air pressure is used to pressurize the oi The time available depends on Keowee unit load changes which require wicket gate operation and thus consume oil pressur The licensee has stated in several LERs that one hour is the expected maximum time period of Keowee unit.operation without the support of AC auxiliary load The newly developed A/P contains specific guidance on actions regarding the Keowee AC auxiliaries. The restoration of the auxiliaries required resetting of several lockouts and was delayed until the Keowee on-call technician arrived. The technician responded rapidly and within approximately 25 minutes after being contacted, he arrived at the Keowee units. The team noted that the expertise and knowledge of on-site (Oconee and Keowee) personnel were limited and excessive reliance was placed on a 'few specific individual During the recovery portion of the event, Keowee Unit 2 was inadvertently tripped. During a subsequent emergency start sequence, the generator field did not flash due to the field breaker speed switch being in anti-pump. Despite the fact that this was a previously identified and well known problem, the speed switch was not reset until about one and a half hours late Several members of the team observed portions of a Keowee Special Test (refer to section V.D) from the Keowee C Shortly after Keowee Unit 1 received the emergency start signal, one of the inspectors observed that annunciator 1SA1/32, Generator Bearing Oil Level,Low, was in alarm. A normal generator lockout was also indicated but that was expected as part of the test and is bypassed on emergency
start. After several minutes, the inspector verified by review of the annunciator response procedures that the low oil level signal was providing a normal lockout signa The Keowee operator returned to the control panels after unsuccessfully attempting to reset the lockout. The low oil level annunciator was then brought to his attention by the inspector. Although the operator was extremely involved in the details of the test and numerous personnel were present observing the test, the out of normal condition were not identified by the operato The oil level in the Keowee Unit 1 thrust bearing sightglass was found to be at or just below the low level setpoin The alarm was cleared for continuation.of the test. Members of the team noted that the oil level of Keowee Unit 2 was slightly below the low level mark and its alarm had not actuated. After both units were shutdown, the inspectors noted the oil level on both units was slightly below the
"normal full cold level mark". Apparently, not running the units to the grid for several days had allowed the oil to cool off and the level had decreased. This oil level is required to be checked during operator rounds but the acceptable level band was very wide. The day after the test approximately 40 gallons of oil were added to each Keowee uni The overall performance of the Keowee station personnel was not up to the standards expected of personnel operating and maintaining a nuclear power station's emergency power system. The team did not identify any documentation or other information to indicate that Oconee management had specified their expectations or identified responsibilities for the Keowee station personne Complexity of Design The design of the electric power system, including the emergency power system at Oconee is both atypical and complex. During the event several problems were encountered and some due to the staff not being aware of specific design features. The overall complexity of the interacting systems contributed to the problems when operating these systems. These system interactions have not been fully identified nor are they fully understood by the operating staf VI ROOT CAUSE DETERMINATION The team determined that the root cause for the October 19, 1992 event was:
The failure of the switchyard battery charger SY-2, to properly regulate DC voltage to the SY-DC-2 bus when placed in that configuration,
initiated the event. It has been determined that voltage surges from this battery charger actuated breaker failure circuits on several switchyard PCBs resulting in an isolation of the swlitchyard from off site power and isolation of Keowee Unit 1 from the overhead power pat At the conclusion of the inspection the licensee was still investigating this piece of equipment and had not yet completely identify which component within the charger resulted in the voltage oscillations. The SY-2 battery charger is being replace.
CONCLUSIONS The Team made the following conclusions based on the findings and other information review as they related to this event: No TS safety limits were exceeded. The response to the reactor trip and maintaining adequate core cooling during natural circulation was goo.
The event was caused by a voltage surge from a battery charger in the 230-kV switching station DC control power system. This caused several power circuit breakers in the switchyard to open and an isolation of the Red and Yellow-Busses occurred causing a loss of Off-site AC power. It was not known that the battery chargers in the 230 kV switchyard were not designed (sized) to carry loads independent of the batter.
During the recovery phase of the event, lack of.procedural guidance and inadequate operator actions resulted in Oconee Unit 1 and 3 being without automatic transfer of off-site or emergency AC power to the MFB's in the event of a reactortrip. This condition existed for about 1 1/2 hours. Efforts were apparently focused on power supply restoration to Unit.
Oconee Abnormal Operating Procedure AP/2/A/1700/11, Loss of Power, did not provide cautions or guidance to ensure that power was available to the non-affected units before resetting the emergency start signal and securing the Keowee units. The procedure was inadequate to assist the operators in their recovery from the loss of Red and Yellow Busses. The Oconee A/P did not require verification of "normal" Keowee Units operation, that was available in the Oconee CR, during an emergency star.
Oconee CR personnel and management were not aware of the "de energized overhead feeder" interlock feature and this resulted in an inadvertent loss of a Keowee unit during the recovery phas.
At Oconee the response and recovery to this event were complicated by the lack of appropriate procedural guidance and the overall complexity involved with the various electrical power systems interaction.
The level and significance of the problems at Keowee, during the event, were not completely communicated to nor fully understood by Oconee operators. The Keowee units lost both 1X and 2X auxiliary busses and the unit continued to operate with their control functions being supplied by the batteries. The Keowee annunciator panels, computer alarm typer, and the telephones were not available due to them being powered from the' auxiliary busses which lost power. The Keowee operator was tbld to take no action involving Keowee Unit 2 because it was supplying power to the Oconee Unit 2 MFBs. The operator then just monitored the operation of the hydro units, while awaiting the arrival of the on-call technicia.
Knowledge of the.Ke'owee Station by the on-shift Oconee personnel was not sufficient to enable them to assist the Keowee operator when unexpected problems occurred. It was noted that expertise and knowledge of on-site (Oconee and Keowee),personnel were limited and excessive reliance was placed on a few specific individual.
At Keowee except for the abnormal procedure for-responding to naturally'occurring events, there were no abnormal or-emergency operating procedures at Keowe Procedural guidance was not provided in the normal operating procedures to respond to abnormal system configurations, and there was no additional guidance to the operator in case.of an emergency start of the.Keowee unit.
The Keowee operator initially was not aware that an emergency start of the Keowee units had occurred. The Keowee operator performed an action, by opening ACB-1, that 'could have interfered with the ability of Keowee unit to perform its safety functio.
Keowee operators overall performance was not up to the standards expected of personnel operating and maintaining a nuclear power station's emergency power system. However, performance expectations and/or identified responsibility for the Keowee station personnel have not been communicated to the Keowee staff by Oconee Managemen.
It was noted during the special test that the Oconee operators had difficulty in controlling the Keowee units from the Oconee control room. These difficulties le d to the tripping of Keowee' Units 1 and 2 during the tes Furthermore, the Keowee operator experienced difficulty in responding to abnormal conditions that occurred during this test despite just having completed-procedural and operational training as corrective-action following the even '.
Westinghouse Re liability Letter 79-47 which, discussed a potential problem with the zener diodes in the breaker failure circuitry, was evaluated by licensee engineering staff in April of 198 The, evaluation indicated that some corrective actions needed to be completed; however, actions-were not take.
The known communication problem between Oconee and Keowee, the loss of some communication during the event, and the lack of emergency lighting needs to be assessed by the license.
During normal testing of the Keowee auxiliary power system transfer scheme, the timing feature was not specified as part of the acceptance criteria. It was noted that the time delay feature malfunction may (due to rapid transfers) have contributed to the 1X transformer lockout during the even IX. EXIT The inspection scope and findings were summarized on October 28, 1992, with those persons indicated in Appendix 3. The NRC described the areas inspected and discussed in detail the inspection findings identified in this report. No proprietary material is contained in this report. No dissenting comments were received from the license APPENDIX 1 ACRONYMS AND ABBREVIATIONS AC Alternating Current ACB Air Circuit Breaker AIT Augmented Inspection Team AR Auxiliary Relay B&W Babcock and Wilcox CCW Condenser Circulating Water CFR Code of Federal Regulations CR Control Room DC Direct Current EFW Emergency Feedwater ENS Emergency Notification System EPSL Emergency Power Switching Logic GPM Gallons per Minute HPI High Pressure Injection kV Kilovolts KVA Kilovolt-amperes LCO Limiting Condition of Operation LER Licensee Event Report LOCA Loss of Coolant Accident LD Load Dispatcher LPI Low Pressure Injection MDEFWP Motor Driven Emergency Feedwater Pump MFB Main Feeder Bus MFBMP Main Feeder Bus Monitoring Panel MSRV Main Steam Relief Valve MWe Megawatts Electric NOUE Notification of Unusual Event NRC Nuclear Regulatory Commission PCB Power Circuit Breaker PSIG Pounds per Square Inch Gauge RBCU Reactor Building Cooling Unit RCP Reactor Coolant Pump RCS Reactor Cooling System RPM Revolutions Per Minute STA Shift Technical Advisor TDEFWP Turbine Driven Emergency Feedwater Pump TS TechnicalSpecification TN Temporary Installation (Modification)
VDC Volts Direct Current VAC Volts Alternating Current
APPENDIX 2A GENERAL OCONEE ELECTRICAL SYSTEM All three Oconee Units have the same generating capacity (850-MWe net)
and similar AC power systems. Output from the Oconee Unit I and Unit 2 generators feed power to the 230 kV switchyard via step-up transformers T1 and T2 respectively. This switchyard is connected to the 230 kV grid by eight outgoing transmission lines. The output of Oconee Unit 3 generator feeds into the 525 kV switchyard via step-up transformer T This switchyard is connected to the 525 kV grid by three outgoing transmission lines. These transmission lines also bring off-site power to the switchyard tofeed Oconee unit auxiliaries when normal power is not available. The 230 kV and 525 kV switchyards are normally connected through an auto transformer block (used to maintain a fixed voltage);
however, this had been disconnected before the event for a planned upgrade of the auto transformer and had no impact on the even The 230 kV and the 525 kV switchyards are divided into 2 busses designated as the Red Bus and the Yellow Bus. The switchyards are normally operated with both busses energized through a breaker-and-one half scheme to the grid. The Yellow Bus in the 230 kV switchyard is identified as safety-related. The Keowee hydro station supplies powe to the switchyard via an above ground (overhead) path. The overhead path is used to supply power to the Yellow Bus if the grid is los The operating Oconee units normally provide power to their own auxiliary loads through auxiliary transformers 1T, 2T, and 3T respectively. When a unit's generator.is unavailable, such as following a reactor trip or during outages, electrical power is automatically supplied from the switchyard through its respective startup transformer CT-I, CT-2, or CT-3. Though Oconee Unit 3 feeds to the 525 kV switchyard, the source of power for its startup transformer is through the 230 kV switchyar The power to RCPs for each unit is supplied by each units 6.9 kV switchgear TA and T Electrical power to TA and TB is supplied by either the operating unit through its own auxiliary transformer or from the 6.9 kV portion of its respective startup transforme The unit auxiliary power system for each.Oconee unit is designed as a dual-train cascading bus system. There are two 4160 volt main feeder busses, MFB1 and MFB2, with each supplying power to three 4160 volt load busses TC, TD, and TE. The power to MFB1 and MFB2 is supplied by either the unit's auxiliary transformer through the "N" breakers or the startup transformer through the."E" breakers. In~addition, MFB1 and MFB2 for each Oconee unit can be energized from the two Standby Busses SB1 and SB2 through the "S" breakers. Standby Bus 1 and Standby Bus 2 are common to all three Oconee units and can be energized automatically from the Keowee underground path through transformer CT-4, or manually from CT-S. Transformer CT-S can be supplied from the Lee steam station through a dedicated line or from the Central substatio Appendix 2A All safety and non-safety AC loads (except RCP's) are fed from either TC, TD, or T (e.g. The three HPI and LPI pump motors, the condensate booster pumps, the hotwell pumps are supplied by TC, TD, TE busses).
During a loss of power event, load shed circuits are provided to remove all non-essential loads, such as condensate booster pumps, from the MFBs of any unit prior to automatically tying to the Standby Busses due to the limited power capacity of CT-4 or CT-APPENDIX 2B RED AND YELLOW BUS LOCKOUT The 230 kV switchyard Red and Yellow Busses are each protected against faults by a bus protective relaying scheme. The 230 kV Yellow Bus differential lockout (Yellow Bus Lockout) and the Red Bus differential lockout (Red Bus Lockout) each protect their respective busses. The Yellow Bus scheme is described below. The-Red Bus scheme is simila The Yellow Bus lockout relay can be energized by either the bus differential relays or an output from the breaker failure circuit on any of the PCBs connected to the Yellow Bus. When the Yellow Bus lockout relay is energized, it trips PCBs-9, 12, 15, 18, 21, 24, 27, 30, and 3 The lockout also provides inputs to an annunciator and recorder and also initiates the breaker failure circuit logic of the PCBs listed abov The Yellow Bus lockout relay can be manually reset after the trip condition is remove The operation of the Red Bus differential lockout (Red Bus lockout) is similar except PCBs-4, 7, 10, 13, 16, 19, 22, 26, 28, and 31 are trippe APPENDIX 2C EXTERNAL GRID TROUBLE PROTECTION SYSTEM (Switchyard Isolation)
For an off-site power source to satisfy design commitments, special relay protection must be present to protect plant auxiliaries in the event of system voltage degradation. This protection will isolate the degraded off-site power source and allow the on-site emergency power sources to supply powe An External Grid Trouble Protection Relaying System is provided for the 230 kV switchyard. It consists of two redundant undervoltage and two redundant underfrequency relaying sub-schemes (channels). These relays monitor the voltage and frequency on the Red and Yellow Busses. Each channel can initiate the operation of the External Grid Trouble Protection Relaying System. Each channel has different hardware to preclude a common mode failure due to equipment malfunction. After detecting degraded conditions on both the Yellow and Red Busses, the system will start both Keowee hydro units and provide an isolated electrical path from one Keowee hydro unit to the Oconee startup transformers CT-1, CT-2, and CT-3 by closing PCBs-9, 18, 27, and 30 and tripping PCBs-8, 12, 15, 17, 20, 21, 23, 24, 26, 28 and 3 The following describes the system in detail.
Undervoltage Channel Number 1 The external grid trouble protection.relaying system channel number 1 consists of six undervoltage relays which are each connected to a phase of both the Yellow and Red Busses. When a degraded voltage condition (less than 67 percent nominal voltage),
is sensed on two out of three phases PCBs-9, 18, 27, and 30 close, PCBs-8, 12, 15, 17, 20, 21, 23, 24, 26, 28 and 33 trip and both Keowee units start. The computer, event recorder point, and status alarm are also actuate Underfrequency Channel Number 1 The external grid trouble protective system underfrequency channel number 1 consists of six underfrequency relays which are each connected to a phase of both Yellow and Red Busses. When degraded frequency is sensed on two out of three phases, PCBs-9, 18, 27, and 30 close, PCBs-8, 12, 15, 17, 20, 21, 23, 24, 26, 28 and 33 trip and both Keowee units are started. The computer, event recorder point, and status alarm are also actuate III
Appendix 2C Undervoltage and Underfrequency Channels Number 2 The External Grid Trouble Protective System undervoltage and underfrequency channels number 2 are similar-in operation and redundant to channel number 1. The major difference between the two channels is the fact that each channel utilizes different manufacturer components to minimize common-mode failures. Also, all wiring and panels are physically and electrically separate Switchyard Isolation Complete The Switchyard Isolation Complete circuitry.functions following an external grid trouble protective relay actuation. A Switchyard Isolation Complete signal is generated when the logic indicates that the 230 kV switchyard Yellow Bus has been separated from the external grid and that the switchyard PCBs are aligned to supply power to the Oconee startup transformers. This circuitry provides an automatic close permissive to PCB-9. If Keowee hydro station is generating to the grid at the time of the event, this logic initiates tripping of the ACBs aligned to the overhead path and allows reclosure of the output ACB of Keowee unit aligned to the overhead path after a 6 second time dela APPENDIX 2D DEGRADED GRID PROTECTION SYSTEM Each Oconee Unit has "degraded voltage protection" relays that monitor voltage on the primary side of the start-up transformer. If any two of the startup transformers experience degraded voltage either as a result of a grid condition, or as a result of not being energized from the switchyard, a nine second timer is activated. Following nine seconds, if an ES signal indicating a LOCA is present or subsequently obtained, the relays activate to initiate the External Grid Trouble Protection System (Switchyard Isolation). This enables energization of the Oconee Units MFBs from the Keowee overhead pat With the nominal grid voltage of 230 kV, the undervoltage relays are set to actuate at 22 k APPENDIX 2E MAIN FEEDER BUS MONITORING PANEL (MFBMP) SYSTEM The MFBMP system is a non-safety system designed to assure a reliable source of power to the main feeder busses (MFB1 and 2) during non-LOCA loss of power events. In addition, the MFBMP's assure the integrity of the RCP seals by insuring that seal injection and component cooling flows are regaine Voltage is constantly monitored on each phase of the two main feede busses MFB1 and 2. An undervoltage for 20 seconds on two of three phases of both MFB's will initiate a MFBMP signal., This in turn: (a)
emergency starts both Keowee units, (b) provides a.close -signal to the SKI and SK2 breakers (i.e., permit the Keowee underground path to energize the standby busses SB1.and SB2), (c) initiates a loadshed signal, and (d) starts the 'A' and 'B' HPI and CC pump The closing logic for the "E" and "S" breakers will energize the MFBs through the startup transformer by closing the"E" breakers or through CT-4 by closing the "S" breakers. Typically, power to essential loads is restored within 31 second APPENDIX 2F EMERGENCY POWER SWITCHING LOGIC (EPSL)
EPSL is a safety related system designed to assure availability of power to the MFBs following a LOCA. EPSL essentially carries out the function of the MFBMP but without the 20 second delay. Additionally, EPSL does not send start signals to the HPI and CC pumps (HPI pumps receive a start signal from the ES signal).
EPSL is actuated on: (a) an ES signal with undervoltage on MFBs with normal and startup breakers open, or (b) an ES signal and undervoltage on both startup and normal sources. Upon actuation, EPSL provides an input to the loadshed circuit to initiate loadshed. The loadshed signal in turn provides a signal to breaker logic to transfer to standby or retransfer to startup-circuitry. It should be noted that EPSL does not emergency start Keowee units but rather assumes that ES a signal has started the Keowee unit II APPENDIX 2G RETRANSFER TO STARTUP LOGIC This logic provides the capability to retransfer the essential loads back to the startup power source if power on the standby bus is lost, or if startup source becomes available before power is supplied by the standby bus. This is accomplished by sensing undervoltage on the MFBs and SB busses and by sensing voltage on the startup source. If power is sensed on the startup source, with no power on the MFB, EPSL will trip the S breakers and close the E breakers for the affected MFB APPENDIX 2H GENERAL KEOWEE ELECTRICAL SYSTEM The Keowee Hydro Station is located approximately three-fourths of a mile east-northeast of the Oconee Nuclear Station.' It consists of two hydro electric generators rated at 87,500 KVA each, which generate at 13.8 k The two Keowee hydro units serve the dual functions of generating commercial power to the Duke Power system grid through the Oconee 230-ky switchyard and providing emergency power to the Oconee Statio Upon loss.of power from the Oconee generating unit and 230-ky switchyard, power is supplied from both Keowee units through two separate and independent paths. One path is a 4000 ft underground 1 kV cable feeder to transformer CT-4 which supplies power to the 4160 volt standby busses through breakers SKi and SK2. The underground power path is connected at all times to one hydro unit on a predetermined basis by having either ACB 3 or ACB 4 locked-closed. The underground power.path and associated transformer are sized to carry full engineered safeguards auxiliaries of one Oconee unit plus auxiliaries for safe shutdown of the other two units. If a Keowee unit is to-provide power to an Oconee unit through the underground power path, due to the limited capacity of CT-4, loadshed of non-essential loads from the Oconee units MFBs occur The second route is a 230 kV transmission line through ACB 1 or ACB 2, via the Yellow Bus of the 230 kV Oconee switchyard to each Oconee unit's startup transformer. Both Keowee units share a Keowee main step-up transformer located just outside the Keowee hydro statio Emergency start signals to both the Keowee units can be supplied from either Engineered Safeguards Actuation, Main Feeder Bus Monitoring Panel Actuation, switchyard isolation, and manual emergency starts from Oconee control rooms or cable spreading room The Keowee unit's AC auxiliaries (1X and 2X bus) provide power to the AC hydraulic oil pumps, along with other loads. These pumps are used to pressurize the air pre-loaded accumulators that provide hydraulic oil pressure to the governor which controls the position (depending on load)
of the wicket gates on the Keowee water turbine. When AC auxiliary power is not available the air pre-loaded accumulators, oil pressure is used by the governor to position the wicket gates. The length of time that the Keowee units can run without AC auxiliaries is limited by the changing load for which the governor must respond. The licensee has indicated in several LERs that one hour is the expected maximum time period of Keowee unit operation without AC auxiliarie Appendix 2H
When a Keowee unit is generating to the grid and an emergency start occurs, it is separated from the,230 kV switchyard and continue to run in standby until neede Each Keowee unit is designed to accept full emergency power load within approximately 23 seconds from receipt of an emergency start signa The normal Keowee configuration, before the loss of power event on October 19, 1992, consisted of having either Keowee Unit 1 or Keowee Unit 2 available for generation to the system grid using the overhead path (via ACB-1 for Keowee Unit 1 or ACB-2 for Keowee Unit 2).
One unit also was aligned to supply the underground path with emergency power (either ACB-3 for Keowee Unit 1 or ACB-4 for Keowee Unit 2 closed).
The design of the Keowee control circuitry was to provide emergency power to-the underground power path from one unit for all emergency start situations while providing power to the overhead path from the other unit only if off-site power was los The Keowee Unit 1 and Unit 2 auxiliary busses normally were powered from the overhead path through their respective 1X and 2X transformers, the Keowee main step-up transformer and the 230 kV switchyard. Normal power was supplied to the 1X bus through ACB-5 while normal power was supplied to the 2X bus through ACB-6. These two load centers also had an alternate power source from the CX transformer that receives power from Oconee Unit 1 load center 1TC. Alternate power from the CX transformer for the lX bus was provided via ACB-7 and alternate power for the 2X bus was provided via ACB-8. An automatic transfer scheme would quickly switch these busses to their alternate power supply on loss of normal power. The transfer scheme was designed to be normal seeking so if normal power was restored for about 10 seconds, the bus would switch back to the normal suppl Keowee Emergency Start Normally on the initiation of a Keowee emergency start signal, the unit aligned to the underground power path would start and supply power to emergency transformer CT-4 as it came up to speed and voltage. The other unit supplying the overhead power path also would start but operate at no load with its breaker open. The Keowee unit would automatically power the overhead power path only if off-site power were lost (i.e., Switchyard Isolation). If the unit aligned to the underground power path was generating to the system grid, the emergency start signal would separate that unit from the grid and immediately supply power to the underground power path. The other Keowee unit, if also generating to the system grid, would separate from the grid as well and continue to operate at no load. Since the overhead power path would remain energized by off-site power, the Keowee auxiliary busses would remain continuously energized and aligned to their normal power sourc APPENDIX 3 PERSONS CONTACTED Licensee Employees
- S. Adams Public Affair *H. Barron Station Manager R. Beaver Electrical Engineer R. Brock Electrical Engineer
- L. Conley Site Communications Specialist
- D. Couch Keowee Plant Manager
- D. Coyle Systems Engineering Manager
- J. Davis Quality Assurance Manager
- D. Deatherage Operations Support Manager
- B. Dobson Supervisor, Electrical Engineering
- J. Hampton Vice President, Oconee Site
- D. Jamil Supervisor, Electrical Engineering
- C. McSwain Superintendent, Hydro Productions
- M. Patrick Compliance Manager
- J. Peele Engineering Manager P. North Compliance Engineer
- G. Ridgeway Shift Operations Manager
- G. Rothenberger Superintendent, Integrated Scheduling
- P. Stovall Director of Operator Training
- D. Sweigart Operations Superintendent
- T. Talbert Hydro System Manager
- R. Todd Onsite Safety Review Group Engineer
- L. Wilkie Training Manager
- H. Tucker Vice President Nuclear Operations L. Underwood Electrical Engineer Other licensee employees contacted during this inspection included engineers, operators, technicians, and administrative personne Other NRC employees:
- A. Belisle Division of Reactor Projects (DRP), Section Chief
- C. Berlinger NRR, Chief, Electrical Systems Branch
- 0. Chopra NRR, Electrical Systems Branch
- K. Clark Public Affairs Officer
- S. Ebneter Regional Administrator
- A. Gibson Director, Division of Reactor Safety
- P. Harmon Senior Resident Inspector
- K. Poertner Resident Inspector
- Attended Exit Meeting
(MO T11 62EENTLE)
(CENTRAL ENTRAL)
SOUFH ASBURAY DACUS OCONE CALHOUNA
'SVT/ ElfPORT MVCOSS m
W B W B 230 KV RED RED BUS TRANS-7-
-
BUS RE T
A'EO EE
~AUT01l
~~
eFR LEE AUTX WR-,1 EHYDRO TRAES UA YELLOW YEO uST PirN L: KV CE'TM'L oaSSSRTT 2U X
71 3 A3 7)
GEN~~~~~
K2 SLU AELO l
4/MDVBUUS US o
/..L ff2 E S/7I 52))
)l)2)/
TS CD
)5/)HV)E
)E
)S/ ) S B
ccF 2 lA U
AL2 4/go~O71 v 7TBY81S 222R S
)
H5 R
)I)
..
ALT )_)_ALT SC ITD /TE 2TT ZTD 2rc JTD 3TE AX/L/A2Y 4/7WUG/EERED SAFEGUARDS Bl/S1 WSsy
FIGURE 2 KEOWEE ELECTRICAL DISTRIBUTION rmH coc m
a mW
b0 mm 0~
-n C>
x.- x oc
>.
0D m-4 r-
0-4
zz 0Zr
ITEE I -
I SPA SPASP sy-I iisy-S ij SY-2A E CHARGER Y-CHARGER S-HRE cV)
____C I
-
-C OCD OYH