05000250/LER-2006-004
| Docket Number Sequential Revmonth Day Year Year Month Day Yearnumber No. 05000 | |
| Event date: | 3-8-2006 |
|---|---|
| Report date: | 11-24-2006 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications 10 CFR 50.73(a)(2)(iv)(B), System Actuation 10 CFR 50.73(a)(2)(iv)(A), System Actuation |
| 2502006004R01 - NRC Website | |
FACILITY NAME (1) II NUMBER (2) LER NUMBER (6) PAGE (3) Turkey Point Unit 3 05000250 sus
DESCRIPTION OF THE EVENT
At the time of the event, Unit 3 was in Mode 5 with reactor [MS: AC] coolant system temperature at approximately 108 degrees F with the 3A residual heat removal (RHR) pump [EIIS: BP, P] supplying core cooling. The power supply to Unit 3 was from a back feed alignment via the Unit 3 main transformer [EIIS:
EL, XFMR] and auxiliary transformer [EIS: EA, XFMR]. Unit 4 was in Mode 1 at 100% power.
On March 8, 2006 at approximately 1553, a loss of the Unit 3 3A 4 kV electrical distribution bus [EIIS: EB, BUJ occurred during restoration of the 3C load center (LC) following outage maintenance. The 3A load sequencer performed bus load stripping and a loss of offsite power (LOOP) to the 3A bus occurred due to a degraded voltage condition that was sensed on the 3C LC. This was caused by a misaligned auxiliary switch contact [EIIS: CNTR] on the 3C 480V LC feeder breaker (30302) [EUS: ED, BKR]. The 3A emergency diesel generator (EDG)� EK, DG] automatically started and restored power to the 3A bus. LOOP loads were sequenced and core cooling was reestablished utilizing the 3B RHR pump on the 3B 4 kV bus which was unaffected by the loss of the 3A 4 kV bus. Core cooling via the 3B RHR pump was restored at approximately 1600 on March 8, 2006. The event was self evident as various alarms alerted operators.
During load sequencing, the 4 kV supply breaker (3AA14) [EUS: EB, BKR] to the 3C LC failed to close.
This was caused by a failed relay. One of two normally closed (NC) contacts was open with the relay de energized. Both NC contacts must be closed for the breaker to electrically close.
Operators were in the process of draining the reactor coolant system (RCS) to approximately 1.5 feet below the reactor vessel flange in preparation for refueling activities. The RCS was at approximately 60% drain down level at the time of the loss of the 3A 4 kV bus. When power to the 3A 4 kV bus was lost, RCS draining was stopped and level was stabilized via the chemical and volume control system (charging and letdown). Condition Reports 2006-7036 and 2006-7100 were initiated to evaluate the event and relay failure, respectively.
The automatic start of the 3A EDG is reportable in accordance with 10 CFR 50.73(a)(2)(iv)(A). The failure of 4 kV supply breaker 3AA14 to close is reportable in accordance with 10 CFR 50.73(a)(2)(i)(B). The 3A EDG actuation was also reported in accordance with 10 CFR 50.72(b)(3)(iv)(A) via event notification 42399.
BACKGROUND
During the refueling outage, when the Unit 3 startup transformer was removed from service, power to Unit 3 was via a back feed from the 240 kV switchyard through the Unit 3 main and auxiliary transformers.
Each Turkey Point unit has two associated EDGs. Technical Specification (TS) Limiting Condition for Operation (LCO) 3.8.1.1.b requires a unit's two EDGs and one of the opposite unit's EDGs to be operable to provide standby electrical power for required equipment in support of plant operation in Modes 1-4. The FACILITY NAME (1) NUMBER (2) LER NUMBER (6) PAGE (3) MMILISII=1611.1.
- safety related function of the EDGs is to automatically start and provide power to required safety related loads during a loss of offsite power in order to achieve and maintain safe shutdown of the reactor. TS LCO 3.8.1.2.b.1 requires one EDG associated with a unit to be operable to provide standby electrical power for required equipment in support of safe shutdown in Modes 5 and 6.
The RHR system is designed to remove residual and sensible heat from the core and reduce the temperature of the RCS during the second phase of plant cool down. During the first phase of cool down, the temperature of the RCS is reduced by transferring heat to the steam and power conversion system. Two RHR pumps and two heat exchangers perform the decay heat removal functions for the reactor. After RCS temperature and pressure have been reduced to 350 degrees F and 450 psig respectively, decay heat removal is initiated by aligning one RHR pump to take suction from the reactor outlet line and discharge through the heat exchangers and into the reactor inlet line. If only one heat exchanger is available, reduction of reactor coolant temperature is accomplished but at a lower rate.
The 3C LC powers various Unit 3 safety and non-safety related equipment including the spent fuel pit cooling pumps, pressurizer heater control group 3, various Emergency Core Cooling System (ECCS) valves, battery charger 3A1, emergency containment filter fan 3C, emergency containment cooler (ECC) fan 3C, and the 3A boric acid transfer pump.
CAUSE OF THE EVENT
The cause of the event was a vendor human error in the configuration of the auxiliary switch contacts on the 3C 480V LC feeder breaker (30302) which went undetected by the vendor test and inspection program and Turkey Point pre-installation checks. Actions from previous events invoked by the vendor were not successful in preventing this event. The barrier put in place was a verification of the as left auxiliary contact configuration against the customer schematic. The technician performing the work and two independent people performing the verification missed the condition. The vendor reported that the schematic was not part of the verification package for this breaker. However, the as found configuration was included and could have been used for final verification. Although this condition was not a vendor wiring error, the measure put in place should have detected and corrected this condition.
The cause of the failure of the 3C LC 4 kV supply breaker (3AA14) to close is due to the relay not having proper NC contact wipe. The lack of proper wipe caused intermittent failure of the NC contacts to make-up upon de-energization of the relay. The apparent cause of the failure was an inadequate dedication of a commercial grade relay by a vendor, which allowed the relay to be installed in the breaker without checking the contact wipe. The vendor failed to identify the wipe as a critical characteristic.
ANALYSIS OF THE EVENT
On March 7, 2006, the 3C 480V LC was removed from service for routine preventive maintenance and breaker exchanges. Included in this scope was the replacement of the 3C 480V LC feeder breaker with a eiFACILITY NAME (1) LER NUMBER (6) PAGE (3)NUMBER (2) newly refurbished breaker. On March 8, 2006, during the restoration of the 3C 480V LC a 3A load sequencer actuation occurred. The 3A sequencer actuation resulted in a loss of power to the 3A 4 kV and 3A 480V LC, and a 3A EDG automatic start. The EDG reenergized the 3A 4 kV bus and 3A 480V LC.
The initial conditions/sequence of events follows:
- The 3A sequencer was in service and enabled.
- The 3C 480V LC was de-energized.
- The 3C 480V LC feeder breaker was racked in and open.
Troubleshooting determined the cause of the loss of power to the 3A 4kV bus to be a misalignment of the auxiliary switch contact block in the newly refurbished 3C 480V LC feeder breaker 30302. The misalignment caused the auxiliary switch contacts to operate in reverse of their design state. The contacts required to be normally open were closed and vice versa. This condition, in conjunction with the 3C 480V LC being de-energized and the 3C 480V LC feeder breaker being closed, satisfied the sequencer actuation logic.
The vendor's final test and inspection procedure does not require the as left auxiliary contact configuration to be documented and compared with the as found configuration. The procedure only requires documentation of pass or fail. This does not provide a robust barrier to prevent breakers with incorrect configurations or wiring errors of the auxiliary switch contacts from being returned to the customer.
The pre-installation checks and tests at Turkey Point of refurbished 480V ABB LC breakers do not check or verify the proper operation of all auxiliary switch contacts.
The 3C LC 4 kV supply breaker 3AA14 was overhauled by a vendor in October 2003. As part of the overhaul, a new HMA relay was purchased, dedicated and installed without checking the contact wipe characteristics. The breaker was inspected on site and installed into the 3C LC 4 kV supply application in October 2004. Troubleshooting found that one of the two NC contacts was open. The relay was removed from the breaker and provided to Engineering for evaluation. It is a GE Model #12HMA11B6. Visual inspection and manual operation of the relay shows that there is very little wipe associated with the NC contacts. Generally, upon release during manual operation, the armature moves to a position where both NC contacts are closed. However, when the armature is moved all the way to one side, one of the NC contacts remains open (very small visible air gap).
A previous HMA relay failure due to inadequate contact wipe was noted during pre-installation testing at Turkey Point on March 9, 2005. An inspection program of potentially affected relays was ongoing at the time of this event. The relay in breaker 3AA14 was scheduled to be inspected during the Unit 3 refueling was in power operation, the inspection had been deferred to the refueling outage based on the successful demonstration of relay function during overhaul, post-maintenance and surveillance testing, and personnel FACILITY NAME (1) LER NUMBER (6) PAGE (3) ssAR I SEQUENTIAL I REVISION Turkey Point Unit 3 05000250 21106 MC =I= � safety concerns associated with the inspection while the breaker was energized. 3AA14 was one of two breakers of concern for Unit 3 whose inspections were deferred to the March 2006 Unit 3 refueling outage.
The other breaker was the 3A 480V LC supply breaker 3AA08. The breakers had passed site inspections, functional testing, post maintenance testing and safeguards testing. The breakers had cycled numerous times during the overhaul and testing with no apparent evidence of the inadequate wipe condition. It was concluded that there was reasonable assurance that operation of the 3AA08 and 3AA14 breakers were not affected and the inspections could be deferred.
Reportability A review of the reporting requirements of 10 CFR 50.72 and 10 CFR 50.73 and NRC guidance provided in "Event Reporting Guidelines," 10 CFR 50.72 and 10 CFR 50.73 (NUREG-1022, Rev. 2) was performed for the subject condition. As a result of this review, the condition is reportable as described below.
3A EDG Automatic Actuation 10 CFR 50.73(a)(2)(iv)(A) requires a licensee event report (LER) to be submitted as follows:
"(A) Any event or condition that resulted in manual or automatic actuation of any of the systems listed in paragraph (a)(2)(iv)(B) of this section, except when:
(1) The actuation resulted from and was part of a pre-planned sequence during testing or reactor operation; or (2)The actuation was invalid and; (i) Occurred while the system was properly removed from service; or (ii) Occurred after the safety function had been already completed.
(B) The systems to which the requirements of paragraph (a)(2)(iv)(A) of this section apply are:
(8) Emergency ac electrical power systems, including: emergency diesel generators (EDGs); hydroelectric facilities used in lieu of EDGs at the Oconee Station; and BWR dedicated Division 3 EDGs.
EDGs are included in 10 CFR 50.73(a)(2)(iv)(B) as a system whose actuation is potentially reportable.
Since the actuation of the 3A EDG in response to the loss of power to the 3A 4 kV bus did not result from being part of a pre-planned sequence during testing or reactor operation and was a valid actuation, the actuation did not meet the exceptions in 10 CFR 50.73(a)(2)(iv)(A). Therefore, the actuation of the 3A EDG is reportable.
3C LC Supply Breaker 3AA14 Failure to Close Evidence indicates that the GE HMA relay in the 3C LC supply breaker 3AA14 had insufficient NC contact wipe when it was installed in the breaker during overhaul. There is no reasonable assurance that the breaker would operate during repeated demands or after a seismic event. Even though it had passed functional FACILITY NAME (1) LER NUMBER (6) 2005 - D4 - 01 aMMEISI=IM surveillance testing, the 3C LC supply breaker is considered inoperable from the time it was installed on October 7, 2004 through March 8, 2006 when it was replaced.
The applicable TS LCO for Unit 3 in Modes 1-4 is 3.8.3.1 which states "The following electrical busses* shall be energized in the specified manner with the tie breaker open between redundant busses within the unit** and between the busses of Unit 3 and 4." The applicable TS LCO for Unit 3 in Modes 5-6 is 3.8.3.2 which states "As a minimum, the following electrical busses shall be energized in the specified manner.
These TS LCOs require the 3C LC to be energized during the applicable modes.
After installation of the breaker and for the period between the PT3-21 refueling outage in October 2004 and the PT3-22 refueling outage in March 2006, the 3C LC remained energized meeting the requirements of TS LCOs 3.8.3.1.a.1 and 2 for the 3A switchgear and 3C LC.
During the first week of the PT3-22 refueling outage (modes 5-6), the 3B train was energized satisfying TS LCO 3.8.3.2.
TS LCO 3.8.3.1, Action b, states "With any of the required LC's and/or MCC's associated with the opposite unit inoperable, restore the inoperable LC or MCC to OPERABLE status in accordance with Table 3.8-1 or Table 3.8-2 as applicable or place the unit in at least HOT STANDBY within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and in COLD SHUTDOWN within the following 30 hours3.472222e-4 days <br />0.00833 hours <br />4.960317e-5 weeks <br />1.1415e-5 months <br />.
TS LCO 3.8.3.1, Action b, was not met since the 3C LC supply breaker is considered inoperable from the time it was installed on October 7, 2004 through March 8, 2006. Since TS LCO 3.8.3.1, Action b, was not met, the condition is reportable as a "...condition which was prohibited by the plant's Technical Specifications..." under 10 CFR 50.73(a)(2)(i)(B).
ANALYSIS OF SAFETY SIGNIFICANCE
Based on the analysis described below, it is concluded that the health and safety of the public were not affected by this event.
Core cooling was interrupted for approximately seven minutes when the 3A RHR pump lost power due to the loss of the 3A 4 kV bus. Operations personnel entered the appropriate procedures to recover core cooling and restore power. Using the most conservative available indication, RCS temperature increased approximately 25 degrees F while core cooling was being restored by starting the 3B RHR pump. No inadvertent mode change occurred as sufficient margin was available to the 200 degree F limit for cold shutdown. The 3A EDG automatically started and supplied power to the 3A bus loads until power was restored via an offsite power back feed through the main and auxiliary transformers. The 3A RHR pump was operable and could have been loaded onto the 3A 4 kV bus after the 3A EDG started. The increase in risk due to loss of core cooling is judged to be very small given the availability of the redundant RHR pump and power source and the short period for restoration of cooling.
DOCKETFACILITY NAME (1) LER NUMBER (6) PAGE (3)NUMBER (2) "--..-.-
- =t=es The loss of the 3C LC as a result of the failure of supply breaker 3AA14 to close during the shutdown conditions at the time of the event did not result in a significant spent fuel pit temperature rise or damage to other plant equipment. In order to determine the impact of the loss of the 3C LC during plant operation, the case of a loss of coolant accident (LOCA) coincident with loss of the 3C LC was run on the Turkey Point simulator. The most significant effect is operation of the train A sequencer in the safety injection (SI)/LOOP mode. This results in 3A 4kV bus stripping and sequencing SI loads while being powered from the 3A EDG even though offsite power is available.
ECCS train B starts and functions normally in response to the LOCA, which alone provides adequate core cooling.
ECCS train A valves do not open and require local operation to be used for core cooling. Most are in parallel with train B ECCS valves/piping and can be left as is. Hot leg injection requires local operation of motor operated valve (MOV) MOV-3-869 to get flow as does alternate low head SI through MOV-3-872.
Train A containment spray flow can be restored by locally opening MOV-3-880A. The only Phase A containment isolation valve that may need closure is MOV-3-1426 if it was open for 3B SG sampling (normally closed).
Overall, in the event of loss of the 3C 480V LC, train B ECCS provides adequate cooling to prevent core damage. With some manual/local actions, most of train A can be restored as well. As a result, the increase in risk due to the loss of the 3C LC is judged to be very small given the availability of the redundant ECCS train.
CORRECTIVE ACTIONS
Auxiliary Switch Contacts (3C LC 480V Feeder Breaker 30302) 1. The auxiliary switch contact was re-configured and the breaker was returned to service.
2. The remaining refurbished breakers being installed during refueling outage U3C22 were reviewed. The review encompassed breakers that have functions performed by auxiliary contacts that would not be verified by normal breaker cycling. For these breakers, auxiliary contact verification was added to the applicable work orders. No additional discrepancies were found.
3. The critical functions provided by the auxiliary switches in safety related 4 kV and LC breakers were reviewed and verified.
4. Breaker refurbishment standards were revised to require the vendor to record as found and as left auxiliary switch configuration and compare the two to ensure proper configuration. This is required for 4 kV, GE, and ABB breakers and 480V ABB, Siemens and Westinghouse (DS, DB) breakers.
FACILITY NAME (1) DOCKET NUMBER (2) LER NUMBER (6) PAGE (3) Turkey Point Unit 3 05000250 a=m 5. The receipt inspection procedure at Turkey Point was revised to require a review of the overhaul report provided by the vendor for each refurbished breaker and ensure the as left configuration of the auxiliary switches matches the as found configuration or as specified in the purchase order. This is required for 4 kV, GE and ABB breakers and 480V ABB, Siemens and Westinghouse (DS, DB) breakers.
6. Checks of the auxiliary contacts at the secondary disconnects were added to the applicable plant procedures for refurbished breakers.
GE HMA Relay Contact Wipe (3C LC 4 kV Supply Breaker 3AA14) 1. The HMA relays in the 4 kV breakers installed in the 3A and 3B (Unit 3), and 4A and 4B (Unit 4) bus switchgear were inspected for adequate contact wipe. All relays that did not meet inspection acceptance criteria were replaced.
2. The procurement specification and applicable receipt inspection procedure for HMA relays have been revised to verify adequate contact wipe by vendor and receipt inspection personnel, respectively.
ADDITIONAL INFORMATION
(MC), second component function identifier (if appropriate)].
FAILED COMPONENTS IDENTIFIED:�GE relay Model #12HMA11B6 SIMILAR EVENTS:�A previous HMA relay failure due to inadequate contact wipe was noted during pre-installation testing at Turkey Point on 3/9/2005. An inspection program of potentially affected relays was ongoing at the time of this event. The relay in breaker 3AA14 was scheduled to be inspected during the Unit 3 refueling outage in March 2006. The breaker failed on demand during the outage prior to the inspection.