05000325/LER-2004-003
| Docket Number08 15 2004 2004 -- 003 - 00 10 13 2004 05000 | |
| Event date: | 08-15-2004 |
|---|---|
| Report date: | 10-13-2004 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications |
| 3252004003R00 - NRC Website | |
Energy Industry Identification System (EIIS) codes are identified in the text as [XX].
INTRODUCTION
On August 14, 2004, during Hurricane Charley, Unit 1 experienced a loss of 230 kV B bus [FK/BU], which was feeding the unit's startup auxiliary transformer (SAT) [FK/XFMR]. As a result, power was lost to both reactor recirculation pumps [AD/P] lA and 1B. In accordance with plant procedures, operators initiated a manual reactor scram followed by a manual turbine trip upon loss of the reactor recirculation pumps. This resulted in loss of the unit auxiliary transformer (UAT) [EL/XFMR] which was supplying power to the emergency buses [EK/BU]. All four emergency diesel generators (EDGs) [EK/GEN] had started upon loss of the SAT and supplied power to the emergency buses upon loss of the UAT. However, it was subsequently determined, on August 15, 2004, that load shed from emergency bus El did not properly occur prior to EDG No. 1 tying to the bus. This LER provides notification of operation prohibited by Technical Specification 3.8.1, "AC Sources Operating," in accordance with 10 CFR 50.73(a)(2)(i)(B). The manual reactor scram and loss of offsite power are reported in LER 1-2004-002.
EVENT DESCRIPTION
Initial Conditions Unit 1 was in Mode 3, at 0 percent rated thermal power, after having inserted a manual reactor scram at approximately 1259 Eastern Daylight Time (EDT) on August 14, 2004. Unit 2 was in Mode 1, at approximately 96 percent rated thermal power. EDG No. 1 was declared inoperable at approximately 2156 on August 15, 2004, after it was determined that a mispositioned cover on relay 1-E1-AE7-CL-B [94] prevented proper load shedding of emergency bus El prior to EDG No. 1 tying to the bus. Units 1 and 2 entered required actions, delineated in TS 3.8.1, for one inoperable EDG.
Discussion During post-trip investigations associated with the Unit 1 manual scram, it was determined that the following loads did not properly shed from emergency bus El prior to EDG No. 1 tying to the bus.
1B Conventional Service Water Pump 2C Conventional Service Water Pump lA Nuclear Service Water Pump lA Control Rod Drive Pump Trouble-shooting activities revealed that the dust cover for relay 1-E1-AE7-CL-B (i.e., subsequently referred to as CL-B), a General Electric model HGA surface mount relay, was found to be mispositioned.
The base of the cover was pushed into the relay base, causing its armature to be picked up and the EDG load shed contact to be closed. The cover had been modified to provide room for required wiring; however, EVENT DESCRIPTION (continued) it also made it more susceptible to mispositioning. This modification was most likely performed during original plant construction.
The following provides a simplified schematic diagram of the emergency bus El control logic associated with the CL-B relay.
(+) � 52S/b (RHR 1C) 52S/b (RHRSW IC) —I— VR-1A (EDG VOLT & FREO) 52S/b (RHR 2C) 27EX (BUS UNDERVOLTAGE)
EDG BREAKER
52S/b (RHRSW 2C) CLOSE SIGNAL 525/b (NSW 1A) 1_ T T CL-BCL-A 525/b (CSW 1B) 52S/b (CSW 2C) 525/b (CRD 1A)125 VDC � 52S/b (CSP 1A) 52S/b (E1-E2 TIE) � 52S/b (E1 -E3 TIE) 525/b (NORMAL INCOMING) (.) The CL-A and CL-B relays energize when all rotating load (i.e., motor) breakers on the 4.16 kV emergency bus, as well as the bus normal incoming power breaker and both emergency bus tie breakers are open. The relay output contacts are wired in the EDG output breaker control logic. Normally-open contacts from the two relays are wired in parallel with each other and then in series with the bus undervoltage signal (i.e., relay 27EX) and the EDG voltage and frequency permissive (i.e., relay VR-1A) to provide a close signal to the EDG output breaker. During the August 14, 2004, event the EDG was started by the loss of the SAT.
Therefore, the EDG was at rated voltage and frequency prior to losing power to the emergency bus via the UAT. This satisfied the voltage and frequency close permissive for the EDG output breaker. As discussed above, the mispositioned cover on the CL-B relay, which verifies all loads are stripped, resulted in the relay EVENT DESCRIPTION (continued) being maintained in the operated position. This prematurely satisfied the load shed close permissive for the EDG output breaker. Upon emergency bus El receiving an undervoltage condition, the 27EX relay provided the last permissive to close the EDG output breaker.
In addition to providing the EDG breaker close permissive, the loss of voltage relay 27EX concurrently initiates a load shed signal to all rotating loads on the emergency bus via auxiliary relays 27Y (i.e., one relay per motor load), as shown in the following diagram.
(+)
- /�27EX (BUS UNDERVOLTAGE) 27Y TRIP (RHR) R
PUMP
- � ..../." 52Seb (EDG BKR OPEN)
TRIP
PUMP
27Y �
- I TRIP I (CRD) �27Y
CRD
PUMP
- j Closing of the EDG breaker interrupted the load shed signal and prevented the load shed. When the EDG breaker began to close, the 52S/b contact in series with the 27Y relays opened faster than required for the relays to pickup and trip the motor breakers.
Past Operability The condition described above resulted in EDG No. 1 being inoperable due to the improperly functioning load shed circuitry. As part of the investigation of this condition, a past operability review was performed.
On February 28, 2004, maintenance personnel performed OMST-DG11R, "DG - 1 Loading Test." Section 7.4, Step 7.4.3 requires technicians to verify the contact state of the CL-A and CL-B relays. Verification of the contact state is accomplished by removing the relay covers. After verification, the covers are reinstalled per procedure. The individuals who performed this task were experienced technicians, and had previously EVENT DESCRIPTION (continued) performed the EDG load test. In addition, the technicians were aware of the modified covers and associated operating experience (OE) involved with the removal and reinstallation of the CL-A and CL-B relay covers; they fully recognized the potential of mispositioning the cover and the possible affect to the operation of the relay. Based on the technicians' skill and knowledge, task familiarity, awareness of the modified relay covers, and associated OE, it is not believed that the mispositioning of the CL-B relay cover resulted during the performance of OMST-DG11R.
The most credible opportunity for the CL-B relay cover to have been mispositioned was during emergency bus El preventive maintenance (PM) activities performed on March 29, 2004. These activities typically involve 10 to 12 individuals performing actions such as cleaning, termination checks, breaker removal, and overall inspections. There were no specific directions to remove or otherwise manipulate the CL-B relay.
However, as a routine calibration evolution during the bus PM the 27EX relay is replaced. This relay is located directly above and to the left of the CL-B relay. The process of performing an as-built, determinating and reterminating, and verification would have placed individuals in close proximity to the CL-B relay.
Although the potential exists that the cover became mispositioned either before or after the March 29, 2004, PM activities, no other activities took place during which personnel were required to be in a position near the CL-B relay. As such, March 29, 2004, has been established as the date when EDG No. 1 became inoperable. This condition was corrected, and EDG No. 1 returned to operable status on August 16, 2004, at approximately 2223 EDT.
Reportability EDG No. 1 was considered inoperable from March 29, 2004, until August 16, 2004. This exceeds the seven day Allowed Outage time of TS 3.8.1, resulting in operation prohibited by the plant's TSs. Therefore, this condition is being reported in accordance with 10 CFR 50.73(a)(2)(i)(B).
EVENT CAUSE
There are two root causes associated with this event. First, the relay was inadvertently struck or pushed causing the relay to become held in the operated state. Second, the relay cover sides were modified by cutting them back to a maximum extent which made the cover more susceptible to being mispositioned if struck. Both root causes were required to be present in order to cause EDG No. 1 to become inoperable.
CORRECTIVE ACTIONS
An operability assessment was performed which demonstrated that EDG No. 1 was not adversely affected by the load applied as a result of tying to the loaded emergency bus. The load applied is enveloped by existing loading scenarios and testing.
CORRECTIVE ACTIONS (continued) The following corrective actions to prevent recurrence of this condition have either been completed or are planned.
1. Engineering Change (EC) 58696 was written and approved to allow HGA surface mounted relay covers to be removed. The relay covers were removed from relays 1-E1-AE7-CL-A and 1-E1-AE7-CL-B.
EC 58696 also demonstrated that modified covers do not present a seismic / operability concern.
2. HGA surface mounted relays on all 4.16 kV emergency buses (i.e., El, E2, E3, and E4) were inspected for similar conditions. No additional operability concerns were identified on any of the emergency buses. However, some non-desirable cover conditions were identified. Degraded conditions associated with emergency buses El and E2 will be corrected prior to the end of the next Unit 1 refueling outage.
Degraded conditions associated with emergency buses E3 and E4 will be corrected prior to the end of the next Unit 2 refueling outage.
3. Training will be completed, by December 15, 2004, to ensure that all appropriate maintenance personnel are aware of modified HGA relay covers and their potential for mispositioning.
4. Review of this event will be incorporated into the I/C Electrical Continuing Training backbone schedule. This will be completed by December 15, 2004.
SAFETY ASSESSMENT
The safety significance of this occurrence is considered minimal. Although EDG No. 1 was considered to be inoperable, with respect to TS compliance, from March 29, 2004, to August 16, 2004, the actual unavailability with respect to design bases loss-of-coolant-accident (LOCA) / loss-of-offsite-power (LOOP) response is approximately 7.1 days; which is only slightly in excess of the existing TS allowed out of service time of 7 days. The 7.1 days represents the total period of time 4.16 kV emergency bus El was carrying motor loads in excess of 1650 hp. This is in addition to the motor load of 562 hp and static load of 288 kW that could have been connected to downstream 480 V unit substation E5 and associated motor control centers. EDG No. 1 availability at or below this load level is based on analysis, past testing, and industry operating experience. Furthermore, the EDGs' capacity is such that any three of the four EDGs can supply the required loads for safe shutdown of one unit and a design basis accident on the other unit without offsite power.
Response to a LOOP without a LOCA is bounded by the LOCA/LOOP scenario. The EDGs' capacity is such that both units can reach simultaneous safe shutdown with one EDG available per unit under LOOP conditions. Additionally, the station blackout (SBO) analysis demonstrates that one EDG can support safe shutdown of the non-blacked out unit while supplying the SBO coping loads of the blacked out unit.
PREVIOUS SIMILAR EVENTS
A review of events occurring within the past three years identified a previous similar occurrence which was reported in LER 1-2004-001, Emergency Diesel Generator No. 3 Condition Prohibited by the Technical Specifications," dated March 4, 2004. This LER discussed an event where maintenance activities associated with the EDG No. 3 jacket water system, rendered the EDG inoperable. However, the root cause and corrective actions associated with LER 1-2004-001 could not reasonably be expected to have prevented the condition reported herein.
COMMITMENTS
Those actions committed to by Carolina Power & Light Company, now doing business as Progress Energy Carolinas, Inc. (PEC), in this document are identified below. Any other actions discussed in this submittal represent intended or planned actions by PEC. They are described for the NRC's information and are not regulatory commitments. Please notify the Manager — Support Services at BSEP of any questions regarding this document or any associated regulatory commitments.
- Degraded conditions associated with HGA surface mounted relay covers on emergency buses El and E2 will be corrected prior to the end of the next Unit 1 refueling outage (i.e., B116R1, currently scheduled to begin in March 2006). Degraded conditions associated with HGA surface mounted relay covers on emergency buses E3 and E4 will be corrected prior to the end of the next Unit 2 refueling outage (i.e., B217R1, currently scheduled to begin in March 2005).
- Training will be completed, by December 15, 2004, to ensure that all appropriate maintenance personnel are aware of modified HGA relay covers and their potential for mispositioning.
- Review of this event will be incorporated into the I/C Electrical Continuing Training backbone schedule. This will be completed by December 15, 2004.