05000247/LER-2004-004

From kanterella
Jump to navigation Jump to search
LER-2004-004, 450 Broadway, GSB
P.O. Box 249Entergy Buchanan, NY 10511-0249
Tel 914 734 6700
Fred Dacimo
Site Vice President
Administration
January 6, 2005
Indian Point Unit No. 2
Docket Nos. 50-247
N L-05-003
Document Control Desk
U.S. Nuclear Regulatory Commission
Mail Stop O-P1-17
Washington, DC 20555-0001
Subject:L Licensee Event Report # 2004-004-00, "Emergency Diesel Generator Actuation
from 480 VAC Safety Bus Undervoltage/Blackout Signal Due to Missing Tie
Breaker Contact Fingers."
Dear Sir:
The attached Licensee Event Report (LER) 2004-004-00 is the follow-up written report
submitted in accordance with 10 CFR 50.73. This event is of the type defined in 10 CFR
50.73(a)(2)(iv)(A) for an event recorded in the Entergy corrective action process as Condition
Report CR-IP2-2004-05927.
There are no commitments made by the Licensee in the attached LER. Should you or your staff
have any questions regarding this matter, please contact Mr. Patric W. Conroy, Manager,
Licensing, Indian Point Energy Center at (914) 734-6668.
Sincerely,
....�0-,
Fred R. Dacimo
Site Vice President
Indian Point Energy Center
Docket No. 50-247
NL-05-003
Page 2 of 2
Attachment: LER-2004-004-00
CC:
Mr. Samuel J. Collins
Regional Administrator — Region I
U.S. Nuclear Regulatory Commission
475 Allendale Road
King of Prussia, PA 19406-1415
Resident Inspector's Office
U.S. Nuclear Regulatory Commission
Indian Point Unit 2
P.O. Box 59
Buchanan, NY 10511-0059
Mr. Paul Eddy
State of New York Public Service Commission
3 Empire Plaza
Albany, NY 12223-1350
INPO Record Center
700 Galleria Parkway
Atlanta, Georga 30339-5957
NRC FORM 366 U.S. NUCLEAR REGULATORY COMMISSION APPROVED BY OMB NO. 3150-0104 EXPIRES: 06/30/2007
(6-2004)
Estimated burden per response to comply with this mandatory collection
request: 50 hours. Reported lessons learned are Incorporated Into the
licensing process and fed back to Industry. Send comments regarding burden
estimate to the Records and FOIA/Privacy Service Branch (T-5 F52), U.S.
Nuclear Regulatory Commission, Washington, DC 20555-0001, or by InternetLICENSEE EVENT REPORT (LER) e-mail to infocollects@nrc.gov, and to the Desk Officer, Office of Information
and Regulatory Affairs, NEOB-10202, (3150-0104), Office of Management and
Budget, Washington, DC 20503. If a means used to impose an information
collection does not display a currently valid OMB control number, the NRC
may not conduct or sponsor, and a person is not required to respond to, the
information collection.
1. FACILITY NAME 2. DOCKET NUMBER 3. PAGE
INDIAN POINT 2 05000-247 16OF66
4. TITLE Emergency Diesel Generator Actuation from 480 VAC Safety Bus Undervoltage/Blackout
Signal Due to Missing Tie Breaker Contact Fingers
Docket Number
Event date: 11-09-2004
Report date: 0-6-2005
Reporting criterion: 10 CFR 50.73(a)(2)(iv)(B), System Actuation

10 CFR 50.73(a)(2)(iv)(A), System Actuation
2472004004R00 - NRC Website

Note: The Energy Industry Identification System Codes are identified within brackets {

DESCRIPTION OF EVENT

On November 9, 2004, at 2339 hours0.0271 days <br />0.65 hours <br />0.00387 weeks <br />8.899895e-4 months <br />, with the plant in a refueling condition and no core alterations in progress, two of three emergency diesel generators (EDG) of the emergency AC electrical power system {EK} actuated during a planned evolution, after a tie breaker {BKR} was used to tie two of three 480 VAC safety buses {ED} (bus 2A/3A, 5A, 6A) together. At 2338 hours0.0271 days <br />0.649 hours <br />0.00387 weeks <br />8.89609e-4 months <br />, 480 VAC tie breaker 52/3AT6A was operated to tie 480 VAC safety bus 3A to 480 VAC safety bus 6A. When the normal supply breaker (fed from 138 KV offsite power) to bus 6A was opened at approximately 2339 hours0.0271 days <br />0.65 hours <br />0.00387 weeks <br />8.899895e-4 months <br />, an unexpected loss of power to 480 VAC safety bus 6A occurred. The de-energized bus 6A caused an undervoltage/blackout signal to be generated which actuated the start and automatic loading of two of three EDGs (EDG-21 and EDG-22) in accordance with plant design. EDG 21 and 22 automatically started and powered 480 VAC safety buses. The EDG-23 for bus 6A was out of service for planned maintenance therefore bus 6A remained de-energized. At the time of the event, the 22 Residual Heat Removal (RHR) {BP} pump and 22 Spent Fuel Pool (SFP) {DA} pump were in service on bus 6A and were de-energized by the event. RHR cooling was restored in approximately 5 minutes and SFP cooling was restored in approximately 39 minutes. The redundant RHR and SFP pumps (21 RHR and 21 SFP pumps) were placed in service. RHR pump 21 was manually started because it only automatically connects with the 480 VAC safety bus on a Safety Injection signal. The SFP pump 21 was also manually started as the SFP pumps have no automatic start features. All fuel assemblies were installed in the core but the reactor vessel head and upper internals were removed and the reactor cavity was filled with water. On November 10, 2004, at 0058 hours6.712963e-4 days <br />0.0161 hours <br />9.589947e-5 weeks <br />2.2069e-5 months <br />, the plant was returned to its normal 480 VAC lineup. On November 10, at 0405 hours0.00469 days <br />0.113 hours <br />6.696429e-4 weeks <br />1.541025e-4 months <br />, an 8- hour non-emergency notification was made to the NRC for a valid actuation of the emergency AC electric power system under 10CFR50.72(b)(3)(iv)(A) (Event Log # 41187). The event was recorded in the IPEC corrective action program (CAP) as CR-IP2-2004-05927.

Tie breaker 52/3AT6A is a 480 VAC electrical bus tie breaker Model DB-75, manufactured by Westinghouse (W120). An investigation into the cause of the event identified the following chronology of activities that resulted in the event:

March 19, 2003: maintenance performs breaker closeout section of Preventive Maintenance (PM) procedure (BRK-P-003-A) which includes 8 major steps including reinstallation of the primary disconnects with 4 of the 8 steps requiring QC verification. However, the primary disconnect installation step is the only major step in this section of the procedure that does not require a "perform" signature or "verification" signature. The maintenance mechanic fails to install the primary disconnects as required by procedure BRK-P-003-A, step 8.7.2. A final inspection is conducted and the breaker covered prior to returning it to operations for installation in the breaker cubicle.

March 20, 2003: Maintenance Supervisor signs procedure step 8.8.1 to indicate that the procedure is complete and all data recorded even though the breaker cleaning and breaker lubrication sections had not been signed.

April 17-18, 2003: Operations removed the tag out for the tie breaker (52/3AT6A) and installs the breaker into the breaker cubicle. Operations installation procedure (OAP-037) check list contains an "as-found" inspection of the breaker cubicle, a visual inspection of the breaker and requires the breaker to be cycled. Under this check list, all DB-50 and DB-75 breakers, except for tie breakers, are required to be racked-in and its associated load started or energized. Operations backed the covered cart-mounted breaker into position in front of the breaker cubicle opening, removed the FME cover and failed to notice the missing primary disconnect contacts prior to inserting the breaker into the switchgear cubicle.

An extent of condition review was performed. Maintenance and Component Engineering determined that the condition only applies to 480 V tie breakers.

Except for tie breakers, failures of this type are self-revealing as a result of energizing the connected load during post-maintenance testing. The remaining two Unit 2 480 VAC tie breakers were inspected and verified to have their primary disconnects installed. Unit 2 spare breakers staged for installation in the 480 V Switchgear Room were inspected to verify the presence of the primary disconnects. No discrepancies were identified.

CAUSE OF EVENT

The direct cause of EDG-21 and EDG-22 actuation and start was an undervoltage/blackout signal on 480 VAC safety bus 6A due to missing tie breaker 52/3AT6A primary disconnects when 480 VAC bus 3A was tied to 480 VAC bus 6A and normal bus 6A supply power was removed in a planned evolution. The absence of the primary disconnects for the breaker prevented the electrical circuit from energizing bus 6A from bus 3A when the tie breaker was closed.

The root cause of the event was human performance (HP) errors/inadequate work practices. The HP errors were mainly due to the overconfidence of personnel performing repetitive tasks that resulted in ineffective application of error prevention tools to ensure attention to detail. The verification tools of self check, peer check, independent verification were not effective. The maintenance mechanic failed to verify the breaker maintenance was complete and that the primary disconnects were installed. The maintenance supervisor failed to verify that all steps in the work package were complete. Operations personnel failed to identify the missing breaker primary disconnects when the breaker was installed into the cubicle. Procedure place keeping techniques were not used. Maintenance mechanics did not use the recommended place keeping convention or any other equivalent method while using the breaker procedure. Questioning attitude challenges were not effective. The mechanics and supervisor failed to take appropriate actions to check the status of the assembled tie breaker when the primary disconnect parts were left over following completion of the breaker PM.

A primary contributing cause was inadequate procedures. DB-75 Breaker PM procedure BRK-P-003-A does not contain appropriate verification hold points (e.g., verify that all breaker deficiencies have been resolved and that the breaker has been reassembled, verify that all steps have been completed and that the breaker is complete and ready to be placed in service). In addition, the procedure does not have good procedure place keeping features (i.e., signoff provisions at each step).

Maintenance procedure BRK-P-003-A and Operations procedure OAP-037, "Operations Electrical Equipment Operating Guidelines," do not include specific precautions or additional enhancements for the tie breakers that may be appropriate because they cannot be retested in the conventional manner as all breakers.

CORRECTIVE ACTIONS

The following corrective actions have been or will be performed under the CAP to address the causes of this event and prevent recurrence.

  • Tie breaker 52/3AT6A was removed from the cubicle and the missing primary disconnects reinstalled, an inspection was performed and the breaker reinstalled and tested.
  • Maintenance performed a stand down meeting with maintenance staff to discuss the event and lessons learned and to reinforce management's expectations on use of HP tools (i.e., self-checking, verification, questioning attitude, place keeping).

The Maintenance Superintendent and Human Performance Manager interviewed the individuals responsible for the breaker task to review the human performance factors associated with the event and reinforce management's expectations by reminding the individuals of the importance of situational awareness, effective communication, and a questioning attitude. The Station Event-Free Clock was reset and an Entergy Red Memo issued to provide site wide notification of the event and the lessons learned.

  • Operations prepared a Shift Order discussing the lessons learned from the event with Operations crews emphasizing the role of the Nuclear Plant Operator (NPO) as the last line of defense in catching anomalies prior to breaker installation.
  • The Breaker Procedure involved in this event (BRK-P-003-A) was revised to include proper place keeping provisions for each step and signoff verification based on EPRI recommendations, including installation of the primary disconnects. The revision also recognized the inability to fully test the tie breakers and provides additional appropriate precautions and reference to the lessons learned from this event. The procedure revision was completed on December 29, 2004.
  • The appropriate breaker procedures will be revised to upgrade their format according to the latest procedure Writer's Guide. Procedure enhancement will include appropriate place keeping and verification features, incorporate lessons learned from the event and steps to verify installation of primary disconnects. Procedure revisions are scheduled to be complete by March 15, 2005.
  • Operation's procedure OAP-037, "Operations Electrical Equipment Operating Guidelines," will be revised to provide inspection of primary disconnects for proper installation, a visual check for missing parts, and a note to remind the NPO that a visual pre-installation inspection is critical for tie breakers since they can not be re-tested. Procedure revisions are scheduled to be complete by March 15, 2005.

EVENT ANALYSIS

The event is reportable under 10CFR50.73(a)(2)(iv)(A). The licensee shall report any event or condition that resulted in manual or automatic actuation of any of the systems listed under 10CFR50.73(a)(2)(iv)(B). Systems to which the requirements of 10CFR50.73(a)(2)(iv)(A) apply include the emergency AC electrical power systems (EDGs). This event meets the reporting criteria because two of three EDGs actuated from a undervoltage/blackout signal on 480 VAC safety bus 6A and energized their associated buses.

PAST SIMILAR EVENTS

A review of the previous two years of Licensee Event Reports (LERs) for events that involved an actuation of the EDGs due to undervoltage as a result of human error (HE) did not identify any applicable LERs. LER-2002-006 reported a TS prohibited condition caused by two inoperable EDGs due to EDG component failure. Component failures were not a direct result of HE. LER-2002-003 reported a loss of 138 KV preferred offsite power that de-energized 480 V safety buses 5A and 6A. Two of three EDGs started as EDG-21 was inoperable for maintenance. Cause of event was a HE for a worker performing unauthorized activities that resulted in grounding of the 138 KV C-phase conductor for normal off-site power. Corrective actions for these events would not have prevented this event.

SAFETY SIGNIFICANCE

This event had no effect on the health and safety of the public.

There were no actual safety consequences for the event because two of three EDGs were operable in accordance with plant Technical Specifications and automatically started per plant design and energized their assigned safety buses (480 VAC buses 5A and 2A/3A). Redundant safety components for forced core cooling and SFP cooling (21 RHR and 21 SFP pumps) were placed in service using their assigned safety power source. In addition, the decay heat load in the reactor core was reduced and the heat sink for core cooling increased as the reactor cavity was filled with water for refueling activities. The reactor core had approximately a third of the core with new fuel assembles which have no heat load and with the reactor vessel head still removed, the core had a large reactor cavity water inventory thereby increasing the time available to respond to the event. RCS forced core cooling was restored in approximately 5 minutes and SFP cooling was restored in approximately 39 minutes.

The event occurred at 2339 hours0.0271 days <br />0.65 hours <br />0.00387 weeks <br />8.899895e-4 months <br />, on November 9, 2004, and RHR pump 21 placed in service at approximately 2345 hours0.0271 days <br />0.651 hours <br />0.00388 weeks <br />8.922725e-4 months <br />. Reactor cavity water temperature was approximately 94 degrees F. On November 10, 2004, at 0018 hours2.083333e-4 days <br />0.005 hours <br />2.97619e-5 weeks <br />6.849e-6 months <br />, the 21 SFP pump was placed in service. The SFP temperature was 99 degrees. At the time of the event, approximate time to boil for the reactor coolant system (RCS) was 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br />, and for the SFP approximately 25 hours2.893519e-4 days <br />0.00694 hours <br />4.133598e-5 weeks <br />9.5125e-6 months <br />. This event is well below the temperature and time limits for restoring cooling to the RCS and SFP. The time that RCS cooling was restored (approximately 5 minutes) was well within the available time to restore forced RCS cooling (12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br />). FSAR Section 9.3.3.2.3, discusses the loss of SFP heat removal at a maximum SFP water temperature of 180 degrees, and concludes the time to rise to 212 degrees is 1.8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />. This FSAR time (1.8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />) and the time to boil (25 hours2.893519e-4 days <br />0.00694 hours <br />4.133598e-5 weeks <br />9.5125e-6 months <br />) bound the actual time the SFP was without cooling (approximately 39 minutes). Adequate operator response time was available to mitigate the event.

There were no significant potential safety consequences of this event under reasonable and credible alternative conditions. The loss of power to 480 VAC safety bus 6A would be bounded by a loss of off-site power (LOOP) event as analyzed in FSAR Section 14.1.12, "Loss of all AC Power to the Station Auxiliaries." The plant is designed for a LOOP and has safety related emergency AC power by EDGs to start on a 480 VAC safety bus undervoltage/blackout signal and power the 480 VAC safety buses (2A/3A, 5A, 6A). Plant operation in accordance with Technical Specification (TS) 3.8.1, AC Sources Operating (Modes 1, 2, 3 and 4), Limiting Condition for Operation (LCO) require two offsite AC sources and three operable EDGs, any two EDGs of which can supply power to one minimum required set of safeguards equipment (e.g., RHR pump). TS 3.8.9 Basis (Distribution Systems-Operating) requires tie breakers between redundant safety related AC, DC and 118 VAC instrument bus power distribution subsystems to be open (Modes 1,2, 3 and 4). A single failure is considered when evaluating the ability to meet the design of electrical power systems. The inability to power safeguards bus 6A is bounded by the single failure and therefore adequate power remained available for postulation of this type of event. During plant shutdown (Modes 5 and 6), TS 3.8.2 (AC Sources-Shutdown) requires one offsite AC power source and two EDGs to be operable. TS 3.8.2 Basis states it is acceptable for safeguards power trains to be cross tied during shutdown conditions and notes for the shutdown/refueling condition (Modes 5 and 6), that assuming a single failure and concurrent loss of all offsite or all onsite power is not required. One EDG has sufficient capacity to support required actions that may be needed in response to any event that might occur during refueling operations. For this event, the plant retained adequate emergency AC power with two operable EDGs and one offsite power supply.

ntiV 4.1141013g1.1,‘ I

  • -ZOO I J