05000261/LER-2010-001

LER-2010-001, Emergency Diesel Generator Inoperable in Excess of Technical Specifications Allowed Completion Time

Docket Number
Event date:
Report date:
Reporting criterion:	10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications 10 CFR 50.73(a)(2)(v)(D), Loss of Safety Function - Mitigate the Consequences of an Accident
2612010001R01 - NRC Website

I. DESCRIPTION OF EVENT

At 0455 hours0.00527 days <br />0.126 hours <br />7.523148e-4 weeks <br />1.731275e-4 months <br /> EST on February 22, 2010, with H. B. Robinson Steam Electric Plant (HBRSEP), Unit No. 2, operating at approximately 99% power, Emergency Diesel Generator (EDG) 'B' [EB] was removed from service for scheduled maintenance. Technical Specifications (TS) Limiting Condition for Operation (LCO) 3.8.1 Condition B, "One DG inoperable," was entered. After completion of the maintenance, post maintenance testing was performed in accordance with procedure OP-604, "Diesel Generators 'A' and 'B'," breaker 52/27B [EB:BKR], EDG 'B' output breaker, failed to close.

An Auxiliary operator trainee attempted to close the output breaker using the local control switch on EDG 'B' Generator Control Panel [EK:33], and the breaker failed to close. Control room operators noted that the green indication light, indicating open, blinked at the approximate time of the switch closure. It was observed by the qualified Auxiliary Operator that when the trainee took the control switch to the closed position, the trainee immediately released the control switch prior to any change in the indication of the 'B' EDG output breaker. Following this occurrence, the qualified Auxiliary Operator and a third Auxiliary Operator who was the designated peer checker decided that a second attempt to close the output breaker should be made because it was believed that the trainee had failed to turn the control switch far enough and hold the switch in position long enough. The control room operators were not consulted on the decision reached by the operators at the Emergency Diesel Generator Control Panel. The Auxiliary Operator trainee attempted to close the breaker a second time unsuccessfully. On the second attempt control room operators reported the EDG 'B' circuit breaker Light Emitting Diode indicator changed to dual indication denoting a trip free position. A visual inspection was performed on breaker 52/27B by operations personnel with no abnormalities identified. The control room operators placed the breaker 52/27B Appendix R Isolation Switch [EB:33] in the, "Isolate," position in accordance with procedure OP-604.

On February 23, 2010, a test was performed on EDG 'A' as required by TS Action Statement 3.8.1.B.3.1 and in accordance with OP-604. The test was completed satisfactorily and concluded EDG 'A' was operable. In addition, breaker 52/27B was replaced on February 23, 2010, with a spare breaker (DB-100) and OST-401-2, "EDG B Slow Speed Start," was completed satisfactorily.

On February 24, 2009, EDG 'B' was declared operable. January 28, 2010, was the date breaker 52/27B was last successfully operated and as discussed below was the most likely time that EDG 'B' became inoperable. It was concluded that EDG 'B' was inoperable from January 28, 2010 to February 24, 2010, which is approximately 27 days. This event resulted in a failure to meet the required actions associated with TS Action Statement 3.8.1.B.4 and Condition C, which require a plant shutdown if an EDG is inoperable for seven days. During this period of time, EDG 'A' was declared inoperable on February 8, 2010, for approximately 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and 53 minutes for scheduled maintenance. This coincidental inoperability of EDG 'A' and 'B' could have prevented the fulfillment of the emergency power safety function.

II. CAUSE OF EVENT

The cause of the event was determined to be a vendor workmanship error, which allowed use of a defective Shunt Trip Attachment (STA) movable core in the breaker. The STA is a solenoid- operated device designed for remote breaker electrical tripping. The shunt trip is designed for intermittent service and the circuit must be opened by an auxiliary switch contact. The STA consists of a metal frame, coil, moving and stationary core, and trip lever. A trip signal to the STA energizes a coil on the STA, thereby causing its moving core to be pulled into the stationary core. The moving core can either rotate a trip lever to lift the trip bar or pull a trip paddle on the trip shaft, thereby rotating the trip shaft counter-clockwise (looking down the trip shaft on a DB-75/DB-100 breaker from the right side) to trip the breaker. As the breaker trips open, its auxiliary switch contact, in series with the STA coil, opens and de- energizes the coil. With the coil deenergized, the STA return spring pushes the moving core out of the stationary core to the reset position.

During the investigation HBRSEP, Unit No. 2, personnel and Westinghouse personnel disassembled the defective STA in an attempt to isolate the binding issue. The STA internal assembly was removed from the core as one complete unit. The internal STA assembly consists of the movable core, the stationary core, brass sleeve, guide pin, return spring, and anti-freeze washer. The internal STA assembly was dissembled into its individual components and each component was inspected and verified against Westinghouse design drawing requirements. The inspection of the movable core disclosed that the leading edge of the core was not chamfered to the required 1/32 inch. The leading edge was specified by Westinghouse manufacturing prints to have an outside diameter of 1.485 inches to 1.490 inches. On the leading edge of the core the diameter measured at a maximum of 1.494 inches. The moving core slides within the brass sleeve on the STA. The brass sleeve inside diameter was to be machined to a minimum of 1.492 inches and a maximum of 1.500 inches. These values are acceptable per the Westinghouse design print for the STA. The investigation revealed that internal binding of the movable STA occurred when the 1.492 inches inside diameter area of the brass sleeve aligned with the 1.494 inches outside diameter area of the movable core. When the two areas were not aligned the STA would operate properly; however, once the movable core rotated in such a way that the areas were aligned, then binding occurred and the STA would not operate as designed. This explains why the EDG 'B' output breaker, 52/27B, was able to successfully cycle 12 times over a period of 10 months because the internal binding condition of the STA is intermittent and had not occurred.

This vendor workmanship error resulted in the internal binding condition that prevented the STA from releasing the trip bar, resulting in the trip bar remaining in the "trip condition" and preventing the output breaker from closing.

III. ANALYSIS OF EVENT

The conditions described in this Licensee Event Report are reportable in accordance with 10 CFR 50.73(a)(2)(i)(B), "Any operation or condition which was prohibited by plant's Technical Specifications," and 10 CFR 50.73(a)(2)(v)(D), "Any event or condition that could have prevented the fulfillment of the safety function of structures or systems that are needed to mitigate the consequences of an accident.

This event was investigated using the HBRSEP, Unit No. 2, Corrective Action Program (CAP) and documented in Significant Adverse Nuclear Condition Report 382604. This reportable event and the associated significant adverse condition investigation was reviewed by the Plant Nuclear Safety Committee on April 7, 2010. The investigation for the inoperability of EDG 'B' due to breaker 52/27B failure to close determined that the STA contained internal binding that prevented the STA from releasing the trip bar, preventing the breaker from closing.

Additionally, the investigation determined that EDG 'B' was not able to fulfill its safety related function from January 28, 2010, to February 24, 2010. January 28, 2010, was the date breaker 52/27B was last successfully operated and the most likely time that the internal alignment of the STA caused it to become bound. Operations personnel declared the EDG 'B' operable February 24, 2010, after completion of repairs and testing. This resulted in approximately 27 days of inoperability and the TS Required Actions and Completion Times not being met. During this time period, EDG 'A' was declared inoperable for 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and 53 minutes for scheduled maintenance. The combined inoperability of EDG 'A' and 'B' could have prevented the fulfillment of a safety function.

During the investigation, an extent of condition was performed. Fourteen circuit breakers were identified as DB-75 and DB-100 currently in service and one spare DB-100 breaker. Of those fourteen breakers, six breakers are safety-related. The six safety-related breakers are 52/17B (EDG 'A' output breaker to Emergency Bus 1 [E1]), 52/18B (Station Service Transformer 2F to El), 52/22B (El supply to Safety Injection [SI] Pump 'B'), 52/27B (EDG 'B' output breaker to Emergency Bus 2 [E2]), 52/28B (Station Service Transformer 2G to E2), and 52/29B (E2 supply to SI Pump 'B').

While the SI Pump 'B' is not in service, breakers 52/22B and 52/29B are not required for operability. Caution tags are in place on 52/22B and 52/29B to verify the STA is in the correct position, in the event the breakers are closed.

Breakers 52/18B and 52/28B are currently in the closed position. These breakers do not have a safety-related function to close.

Scenarios were evaluated to consider a Loss of Power Event. Immediately following a loss of normal offsite electrical AC power, the control room operators would enter procedure PATH-1.

PATH-1 requires the verification of either El and/or E2 energized. As previously stated, even if the EDG 'B' output breaker 52/27B failed to close, EDG 'A' would have been capable of providing power to the necessary equipment, except during the approximately 7 hour8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> period on February 8, 2010, when EDG 'A' was also inoperable. If it is assumed that EDG 'A' had failed to operate, PATH-1 directs the start of End Path Procedure (EPP), EPP-1, "Loss of All AC Power." EPP-1 directs the control room to dispatch an operator to start and load the Dedicated Shutdown Diesel Generator (DSDG) to restore AC power. If the DSDG operates, sufficient equipment can be operated to safely shutdown the plant. EPP-1 then directs restoration of AC power to energize El and E2. Assuming EDG 'A,' EDG 'B,' and DSDG fail to operate, the control room would request assistance from Maintenance in restoring AC power. As a result of steps performed in EPP-1, it would be determined that 52/27B failed to close, and maintenance could use the following repair options:

1.Perform a localized inspection of the 52/27B, EDG 'B' output breaker. This inspection would reveal that the STA has positioned the trip bar in such a way which will not allow the breaker to close. Maintenance would then manually manipulate the trip bar allowing the breaker to close. The estimated time of completion for this evolution is approximately one hour.

2. EDG 'B' output breaker 52/27B could be replaced with EDG 'A' output breaker 52/17B.

The estimated time of completion for this evolution is approximately one hour.

3. Manual closure of 52/27B in accordance with EPP-22, "Energizing Plant Equipment Using Dedicated Shutdown Diesel Generator," Attachment 2. The estimated time of completion for this evolution is approximately two hours.

4. Retrieval of spare DB-100 breaker from stock and installing the spare breaker into the 52/27B cubicle. The estimated time of completion for this evolution is approximately four hours.

IV. CORRECTIVE ACTIONS

Completed Corrective Actions:

• EEDG output breaker, 52/27B, was replaced including a new STA assembly.

• Instructions were added to work requests for EDG 'A' and EDG 'B' to perform a visual inspection of the STA to ensure proper orientation following each operation.

• Work requests were initiated to inspect STA assemblies on the 'A' train during Refueling Outage 26, currently scheduled to end on June 25, 2010.

Planned Corrective Actions:

• Applicable EDG operating procedures are scheduled to be revised by July 10, 2010, to include instructions to visually inspect DB-100 circuit breakers for the correct STA orientation after completion of the test and with the breaker in the open position.

• Procedure, PM-402, "Inspection and Testing of Circuit Breakers for 480 Volt Bus El," is scheduled to be revised by July 10, 2010, to include instructions to physically inspect the internal binding of the STA on DB-75 and DB-100 circuit breakers.

• Procedure, PM-163, "Inspection and Testing of Circuit Breakers for 480 Volt Bus E2," is scheduled to be revised by July 10, 2010, to include instructions to physically inspect for internal binding of the STA on DB-75 and DB-100 circuit breakers.

• Procedure, SPP-032, "Breaker Receipt Inspection for New and Refurbished Breakers," is scheduled to be revised by July 10, 2010, to include instructions to physically inspect for internal binding of the STA on DB-75 and DB-100 circuit breakers.

Previous Similar Events:

Licensee Event Reports (LERs) for HBRSEP, Unit No. 2, were reviewed from the past 10 years.

The following event was identified as being similar to the event described in this LER:

• LER 2009-001-00, Emergency Diesel Generator Inoperable in Excess of Technical Specifications Allowed Completion Time. This event was a failure to meet the required actions associated with TS LCO 3.8.1, "AC Sources - Operating," due to the cotter pin used to retain the relay's mechanical lift linkage had rotated to a position that prevented complete return of the lift linkage to its normal position. The lift linkage maintained the control relay trip pin engaged and maintained the control circuits open preventing the breaker from closing. This event is relatable to the event described in