05000289/LER-2008-001

From kanterella
Jump to navigation Jump to search
LER-2008-001, Decay Heat River Water System Pump (DR-P-1B) Failed to Start
Three Mile Island
Event date: 02-12-2008
Report date: 06-09-2008
2892008001R00 - NRC Website

There were no structures, systems, or components out of service that contributed to this event.

Event:

On April 8, 2008, it was discovered that the "B" Decay Heat River Water System Pump (DR-P-1B) IBS/13] was inoperable from January 25, 2008 till February 13, 2008. This conclusion was reached following evaluation of test data on a failed X relay coil IBP/CLI for the circuit breaker of DR-P-1B 1BP/52].

At approximately 2250 on February 12, 2008, the "B" Decay Heat River Water System Pump (DR-P-1B) was being started in accordance with procedure OP-TM-533-402, "Operating DR-P-1B for Other Than Decay Heat Removal Operations," and the pump failed to start. DR-P-1B was declared inoperable, placing TMI-1 in an unplanned 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> shutdown Limiting Condition of Operation (LCO).

Troubleshooting determined that the coil for the DR-P-1B breaker control X relay was open circuited The function for the control X relay is to control the operation of the breaker closing solenoid and to provide the breaker anti-pumping control feature. The anti-pumping feature allows only one closing attempt for the breaker as long as a close signal is present. The control relay for the breaker was replaced and DR-P-1B was successfully returned to service at approximately 1032 on February 13, 2008.

The initial investigation by the electrical technician determined that one of the coil wires on the X relay was not firmly anchored to the coil, which may have allowed it to move when the breaker was cycled.

The electrical technician confirmed that the lead wire was not securely anchored by the coil outer layer of tape. When the relay was removed it was observed that the coil lead moved and the section of the lead under the tape was seen moving under the tape. The point where the wire exited the tape was acting as a pivot point. When the coil insulating tape was cut to inspect the coil lead wire, the coil strand wire solder joint and wire lead fell loose from the relay coil. It was evident that the wire was open approximately Yi" from the solder joint. The wire appeared to have burned opened due to current flow.

The X relay and the removed coil lead wire were quarantined and sent to Exelon Power Labs for failure analysis.

The submittal of this LER constitutes reporting to the NRC in accordance with 10 CFR 50.73 (a)(2)(i)(B), "a condition prohibited by Technical Specifications." Technical Specification (TS) 3.3.1.4.d requires two decay heat river water pumps to be operable when the reactor is critical. TS 3.3.2 allows a 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> timeclock to restore an affected pump, or the reactor must be placed in a hot shutdown condition within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />. Following completion of the evaluation of the Exelon Power Labs test data on April 8, 2008, it was determined that DR-P-1B was inoperable from January 25, 2008 till February 13, 2008, which exceeds the 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> timeclock allowed by TS 3.3.2, and thus constitutes a condition prohibited by TS lv The circuit breaker for DR-P-1B is a 480 VAC Westinghouse Type DB-25 electrically operated breaker.

The breaker was installed on September 11, 2007. This breaker was refurbished by Westinghouse in the October 2005 time frame as a safety related breaker. During a breaker refurbishment, the X relay is replaced due to previous identified X relay issues with other breakers. These issues are associated with X relay contacts/breaker mechanical timing issues and not open circuited relay coils. The breaker was initially inspected and tested at TMI-1 in December 2005, and then placed into a spare switchgear cubicle for storage. Based on interviews with electrical technicians, the X relay coil connections are typically verified by completing a resistance check between the coil lead termination points shown on the breaker­ wiring diagram. The inspection maintenance includes an X relay contact condition inspection and several functional checks associated with the X relay. The functional checks include breaker mechanism timing (releases the X relay contacts to de-energize the breaker closing solenoid at the proper point during the breaker closing cycle), the breaker anti-pumping feature, and breaker functional testing. The inspection does not include any checks that require manipulation of the X relay coil wires. During a typical inspection, a breaker is cycled a minimum of 23 times either manually or electrically. For electrical closing cycles the X relay is energized. There is a minimum of 10 breaker cycles/checks where the X relay is energized. A review of the work order did not identify any operational issues identified for this breaker.

When this breaker was setup for the DR-P-1B, the breaker maintenance completed would have cycled the breaker a minimum of 10 times. A review of the Control Room log determined the number of start/stop operations for DR-P-1B since the breaker was installed. The pump was started 23 times since the breaker was installed in September 2007.

Control X Relay Design and Condition Evaluation:

The X relay is a device that is supplied by a sub-vendor and is subjected to a rigorous commercial grade dedication process by Westinghouse. Westinghouse has Commercial Grade Instructions (CDI) for both the X relay coil and the X relay assembly. The coil CDI includes specific instructions to verify that magnetic wire must be fully covered with tape, the tape must not be frayed, torn or cut, the leads must be located as shown on the spool drawing, the lead insulation must not be cut, nicked or otherwise damaged, the tape must secure the leads in a manner that prevents stress on the lead to magnet wire connection. The CDI for they relay requires that the relay be disassembled to install an anti-freeze washer. The relay coil is removed by Westinghouse to install the anti-freeze washer then reassembled.

The relay coil is inspected for proper workmanship and manufacturing, for proper assembly, and that the tape and varnish must fully cover the magnetic wire. The coil resistance is measured and minimum pickup and drop out voltages are verified. The relay is cycled 50 times.

The Westinghouse DB Breaker engineer indicated that the dedication process used for the entire X relay assembly would be completed per the CDI for the X relay only and would not include the CDI for X relay coils. Therefore for a new X relay assembly there is no specific requirement to address stress relief for the coil wires.

The X relay coil is mounted on the top section of the relay within a bracket assembly. The coil is not a snug fit in the bracket assembly and is free to float vertically within the bracket approximately 1/16". One Coil lead to Auxiliary Switch assembly 4° 4r.orki Coil lead termination (failed lead DR-P-I B breaker) coil lead exits the coil next to the coil spool and connected to the breaker auxiliary switch assembly. The second coil exits the coil between the outer layer of tape wraps and is terminated to a terminal on the X relay assembly. This lead is formed with marginal slack when it is terminated on the relay connection terminal.

omparison Gail , magnet wire well tiirmed with smooth bends, lead asemblv was secured with tape under outer laver ol tape Failed coil lead, magnet wire appeared to be not as well formed, coil lead wile was observed as not being securely anchored by the tape

II/

The following conclusions were made by Exelon Power Labs. The relay failed to operate due to an open in the coil circuit. The area of the open circuit and associated damage was localized to an area of the coil near the point where the soldered termination of the coil wire to the coil is made. This is an area of the coil that is manually handled during the manufacturing process, in which the varnish is stripped from the coil wire and the soldered termination is made. Also, it was observed on a comparison coil (second sample) from the same lot of the subject coil, that bends of the wire were present in the area where the failure of the subject relay occurred. Thus, the associated arcing, melting, and consequential open circuit of the coil strand wire were mostly due to an operation associated with the manufacturing process.

Both wires of the comparison relay were individually utilized in lifting the entire relay assembly to ascertain whether this type of handling could have stressed the relay termination or internal magnet wire. In each case there was no indication the weight of the assembly stressed or damaged the relay connection or internal coil wire. The tape wrapping of the coil adequately supported the weight in each case.

Additional manufacturing details for the coil assembly were reviewed. The coil strand wire varnish is removed using a sanding process prior to completing the soldered connection to coil external lead wire. The coil lead is formed into a loop to provide stress relief during taping. The design specification requires that the connection be secured using epoxy and tape. For the failed coil it appeared that the coil lead was anchored under approximately '1/4" of the outer layer of tape. The coil lead for the comparison relay was anchored under approximately 1/z" of the outer layer of tape. Based on the Power Labs Report this wire was securely anchored.

NR-P-I A breaker X Relay Coil — Coil lead verified as being securely anchored.

Iv As an additional data point, the X relay coil leads for Nuclear Services River Water Pump "A" (NR-P-1A) were inspected during the breaker PM which was completed during March 2008. The leads were observed as being sufficiently anchored by the coil outer layer of tape to prevent stress on the lead to magnet wire connection. It is evident that the coils were not stressed either during the commercial grade dedication process or any wiring verification checks completed at TMI. The breaker counter showed 156 breaker operations since the X relay was replaced during refurbishment.

B. CAUSE OF EVENT

The apparent cause of the X relay open circuit was a coil manufacturing defect. The coil strand wire failure occurred approximately 1/4 " away from the solder connection to the coil lead wire. The coil wire appeared to have twists instead of being formed smooth with no twist like the comparison relay coil. Since the wire burnt open at a flaw it can not be determined if a defect was introduced when either the varnish was removed by sanding or when the wire was formed and prepared for making the solder connection. The coil lead connection assembly was secured using clear tape and epoxy during the manufacturing process. The Westinghouse CD! for X relays does not specifically address verification of the coil leads being secure to prevent stress on the magnet wire. From inspection of other X relays the coil lead anchoring for the affected manufacturing lot does not appear to be as robust as other relays. However the coil lead anchoring for the comparison relay from the same manufacturing lot was determined to be adequate based on Exelon Power Labs testing. This indicates that the failed coil lead was likely not adequately anchored during manufacture, since the Lab test showed that considerable force can be applied to a properly secured coil lead without the lead becoming loose and subjecting the magnet wire to stress. Since the failed relay outer tape was cut to inspect the lead connection in the field, a detailed evaluation could not be completed.

, G If the solder joint was not securely anchored at this point where the end of the magnet wire is secured by the clear tape, a stress point is introduced that could stress the wire during every breaker operation. Since the coil assembly can move within the mounting bracket, the impact of the breaker opening or closing could result in movement of the solder joint, resulting in stress to the magnet wire at the point where it is secured by the clear tape. After several breaker operations the stress could weaken to the point of causing a high resistance connection and eventually open circuiting.

C. ANALYSIS / SAFETY SIGNIFICANCE

There were no actual safety consequences associated with this event.

The Decay Heat Removal function is provided by two redundant trains, the "A" and "B" trains. Each train consists of a river water loop, a closed cooling water loop, and a low-pressure injection loop. A review of the "A" train availability during the affected time-period (January 25, 2008 to February 13, 2008) indicated the following: 100 percent availability of the river water loop and closed cooling water loop, and 0.82 hours9.490741e-4 days <br />0.0228 hours <br />1.35582e-4 weeks <br />3.1201e-5 months <br /> unavailability of the low pressure injection loop due to quarterly surveillance testing of the "A" LPI Pump.

During this quarterly surveillance, the "A" LPI Pump is declared inoperable because the return path from the pump discharge is open to its suction source, the Borated Water Storage Tank (BWST). Thus, if the "A" LPI Pump would have received an Engineered Safety Feature Actuation (ESAS) start, the minimum required injection flow into the Reactor Coolant System (RCS) might not have been immediately available from the "A" train. However, the "B" LPI pump was available and would have provided the required injection flow. Although the inoperability of DR-P-1B causes the entire "B" train to become inoperable, the low-pressure injection loop can provide its safety function until the closed cooling water loop overheats, due to the river water loop being inoperable. The time-period for overheating of the closed cooling water loop is conservatively estimated at greater than 30 minutes, which is the minimum time required to empty the BWST after which the LPI suction is switched to the Reactor Building (RB) sump. Heating of the closed cooling water loop would be rapid after the LPI suction becomes the RB sump. Therefore, since it is reasonable to conclude that the "A" LPI pump's configuration would be corrected in a few minutes, no loss of safety function would have occurred during this 0.82-hour window. Note that an operator is stationed in the vicinity of this return path valve and is in communication with the Control Room during this surveillance.

D. CORRECTIVE ACTIONS

This relay failure is considered an isolated occurrence. There have been no similar failure occurrences at TMI­ 1 and there is no OPEX for a similar X relay coil failure. Inspections to verify that no additional coil manufacturing defects exist will be performed. All of the X relays from the same manufacturing lot as the failed relay will be inspected, and a representative X relay from each other manufacturing lot will be inspected.

The site procedure for inspection and testing of this type electrical breaker has been revised to include X relay inspection criteria for coil resistance measurement.

Also, Westinghouse has evaluated this issue in their corrective action program. Westinghouse will revise their 1 v commercial grade dedication program for X relay assemblies to be the same as that used for control relay coils.

E. PREVIOUS OCCURENCES

There is no record of similar failure occurrences at TMI-1. A review of industry databases indicated no OPEX for an X relay coil failure.

  • Energy Industry Identification System (EIIS), System Identification (SI) and Component Function Identification (CFI) Codes are included in brackets, [SI/CFI] where applicable, as required by 10 CFR 50.73 (b)(2)(ii)(F).