05000445/LER-2007-001
| Docket Numbers11 21 2007 2007 - 001 - 00 01 17 08 05000 | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications 10 CFR 50.73(a)(2)(v)(D), Loss of Safety Function - Mitigate the Consequences of an Accident |
| 4452007001R00 - NRC Website | |
DESCRIPTION OF THE REPORTABLE EVENT
A. REPORTABLE EVENT CLASSIFICATION
10CFR50.73(a)(2)(i)(B) "Any operation or condition which was prohibited by the plant's Technical Specifications" and 10 CFR 50.73(a)(2)(v)(D) "Any event or condition that could have prevented the fulfillment of a safety function.
B. PLANT OPERATING CONDITIONS PRIOR TO THE EVENT
On November 21, 2007, CPNPP Unit 1 was in Mode 1, operating at 100% power.
C. STATUS OF STRUCTURES, SYSTEMS, OR COMPONENTS THAT WERE
INOPERABLE AT THE START OF THE EVENT AND THAT CONTRIBUTED TO
THE EVENT
There were no structures, systems, or components that were inoperable at the start of the event that contributed to the event.
D. NARRATIVE SUMMARY OF THE EVENT, INCLUDING DATES AND
APPROXIMATE TIMES
CPNPP has two Emergency Diesel Generators (EDGs) [EIIS: (EK) (DG)] per unit (one per safety related train) and they provide a backup power supply for their respective safety related busses. Each EDG automatically starts in emergency mode whenever an undervoltage condition is experienced on its respective emergency bus or a Safety Injection signal is received. The EDGs are 16 cylinder Delaval Enterprise Type DSRV-16-4 engines and are tested monthly per Technical Specification (TS) 3.8.1. The governor is connected to the control rack [EIIS: (EK) (DG) (MTR) (P) (FC)] of the fuel injection pump for each cylinder by a common mechanical linkage system. The governor controls engine speed by rotating the linkage causing the control rack to slide in and out of the pump body to regulate fuel delivery to each cylinder.
From October 24 to November 21, 2007 CPNPP Unit 1 was in Mode 1 operating at 100% power. During this time period, cosmetic painting activities involving the engine and other components were being conducted in the Unit 1 Train B EDG room. A thorough pre-job briefing was conducted each day prior to painting, which included review of a book with photos showing areas on the EDG that were not to be painted. This book was developed to address industry operating experience issues that involved painting activities that affected EDG operability at other plants. In addition, after each day's painting a visual inspection of the EDG was performed to identify any paint that had inadvertently spattered or dripped onto areas of the EDG that were not to be painted.
On October 24, 2007, the Unit 1 Train B EDG was started and successfully passed its monthly TS SR 3.8.1.2 surveillance test. This test verifies that the EDG starts from standby conditions and achieves the TS acceptance criteria for steady state voltage and frequency.
On November 21, 2007, the next monthly TS SR 3.8.1.2 surveillance test was being performed on the Unit 1 Train B EDG. At 1020 hours0.0118 days <br />0.283 hours <br />0.00169 weeks <br />3.8811e-4 months <br />, the EDG was given a slow start signal from the Main Control Room, but it failed to start as required. The "Unit Failed to Start" alarm and the "Unit Tripped" light were received. The EDG was declared inoperable and the applicable alarm procedure was performed to investigate the alarm indications.
Per Shift Manager (Utility, Licensed) direction, the fuel pump control racks were not manually manipulated during performance of the "check fuel racks free and max position" alarm procedure step because the EDG did not appear to have rolled. Investigation was initially centered on air start system alignment, alignment and fill of the fuel day tank, and venting of the fuel system. No problems were found with the EDG as a result of performing the alarm procedure.
At 1318 hours0.0153 days <br />0.366 hours <br />0.00218 weeks <br />5.01499e-4 months <br />, troubleshooting activities began. These troubleshooting efforts focused on the EDG's failure to roll. At 1649 hours0.0191 days <br />0.458 hours <br />0.00273 weeks <br />6.274445e-4 months <br /> Operations attempted a second slow start of the EDG. The engine rolled up to 90-100 rpm but the EDG did not start and the "Unit Failed to Start" alarm, both of the "Starting Air Low Pressure" alarms, and the "Unit Tripped" light were received. At this point, focus shifted to potential problems with the EDG fuel supply.
At 1802 hours0.0209 days <br />0.501 hours <br />0.00298 weeks <br />6.85661e-4 months <br /> Operations attempted a fast start of the EDG, and personnel (Utility, Non licensed) were assigned to watch the governor and fuel linkage to the fuel pump during the start. During this third start attempt the EDG did not start and both the "Unit Failed to Start" and "Voltage Regulator System Trouble" alarms were received. During this start attempt, the governor was observed to respond but there was no observed movement of the fuel pump control racks. At this point, focus shifted to potential problems with the fuel pump control racks.
At 1939 hours0.0224 days <br />0.539 hours <br />0.00321 weeks <br />7.377895e-4 months <br /> workers (Utility, Non-licensed) discovered that the 6L fuel pump control rack was bound solid and the 2L fuel pump control rack was restricted to approximately 50% of its movement. Based on these observations, the fuel pump control racks on the Unit 1 Train B EDG were manually stroked and the 2L and 6L fuel pump control racks became free. During this same time frame, the fuel pump control racks on the other three EDGs were also manually manipulated and found to be free and operated smoothly. After the fuel pump control racks on the Unit 1 Train B EDG were freed up, another slow start of the EDG was planned. Just prior to the start, a visual inspection of the fuel pump control racks was performed. During this inspection, very small paint spots were observed on fuel pump control racks 4R and 4L, and what appeared to be the residue from an approximately 3/16 inch diameter drop of paint was observed on the 6L fuel pump control racks (where the shaft enters the fuel pump).
Train B EDG was performed and the EDG was determined to be operable.
E. THE METHOD OF DISCOVERY OF EACH COMPONENT OR SYSTEM FAILURE,
OR PROCEDURAL OR PERSONNEL ERROR
During troubleshooting activities, a Maintenance Technician (Utility, Non-licensed identified paint residue on one of the EDG fuel pump control racks.
II. COMPONENT OR SYSTEM FAILURES
A. FAILURE MODE, MECHANISM, AND EFFECT OF EACH FAILED COMPONENT
Not applicable — there were no component failures associated with this event.
B. CAUSE OF EACH COMPONENT OR SYSTEM FAILURE
The most probable cause for this event was determined to be a paint drop that was not cleaned off of the 6L fuel pump control rack following painting activities on the EDG. This paint drop is believed to have prevented the operation of the control rack, which subsequently prevented the entire mechanical linkage from rotating. The lack of movement precluded sufficient fuel from being admitted to all of the cylinders which prevented the engine from starting.
C. SYSTEMS OR SECONDARY FUNCTIONS THAT WERE AFFECTED BY FAILURE
OF COMPONENTS WITH MULTIPLE FUNCTIONS
Not applicable - there were no component failures associated with this event.
D. FAILED COMPONENT INFORMATION
Not applicable - there were no component failures associated with this event.
III. ANALYSIS OF THE EVENT
A. SAFETY SYSTEM RESPONSES THAT OCCURRED
Not applicable — no system responses occurred during this event.
B. DURATION OF SAFETY SYSTEM TRAIN INOPERABILITY
As discussed above, the most probable cause for the failure of the Unit 1 Train B EDG to start during the monthly surveillance test was an approximately 3/16 inch diameter drop of Train B EDG room from October 15 to November 21, 2007, however, the only time that painting was conducted in the vicinity of the Unit 1 Train B EDG fuel pump control racks was from October 29 to November 1, 2007. Therefore, assuming that the drop of paint was inadvertently left on the 6L fuel pump control rack on October 29, Luminant Power believes the maximum time that the Unit 1 Train B EDG could have been inoperable was from October 29 to November 21, 2007 (24 days, approximately 576 hours0.00667 days <br />0.16 hours <br />9.523809e-4 weeks <br />2.19168e-4 months <br />). Similarly, assuming that the drop of paint was inadvertently left on the 6L fuel pump control rack on November 1, the minimum time that the Unit 1 Train B EDG could have been inoperable was 20 days (approximately 480 hours0.00556 days <br />0.133 hours <br />7.936508e-4 weeks <br />1.8264e-4 months <br />). During this time period the completion times of TS 3.8.1 Condition B were exceeded for the Unit 1 Train B EDG.
Consequently, this condition is reportable under 10 CFR 50.73(a)(2)(i)(B) as an operation or condition prohibited by Technical Specifications. Since the Unit 1 Train A EDG was inoperable for 65 minutes (while that EDG was being barred over in preparation for its monthly surveillance test) during the same time period that the Unit 1 Train B EDG was inoperable, this condition is also reportable under 10 CFR 50.73(a)(2)(v)(D) as an event or condition that could have prevented the fulfillment of a system safety function.
C. SAFETY CONSEQUENCES AND IMPLICATIONS OF THE EVENT
The safety function of the CPNPP EDGs is to automatically start and provide power in emergency mode whenever an undervoltage condition is experienced on their respective emergency bus or a Safety Injection signal is received. As noted above, the Unit 1 Train B EDG could have been inoperable from October 29 to November 21, 2007 (approximately 576 hours0.00667 days <br />0.16 hours <br />9.523809e-4 weeks <br />2.19168e-4 months <br />). During this period of time, the Unit 1 Train A EDG was also inoperable for 65 minutes on November 7, 2007 (while that EDG was being barred over in preparation for its monthly surveillance test). During this 65 minute period, two trains of emergency power may not have been available. Since at least one EDG per unit is required to perform the safety function, a loss of safety function for the system occurred during the 65 minutes that the Unit 1 Train A EDG was inoperable on November 7, 2007.
From October 29 to November 21, 2007, no events occurred requiring the EDGs to start and perform their safety function of providing power to the safety related busses. Since the period of time that both Unit 1 EDGs were inoperable was relatively short (65 minutes), and two offsite power sources were available, the potential safety significance is low.
Therefore, this event had minimal safety consequences and the health and safety of the public was not affected.
IV. CAUSE OF THE EVENT
Although a specific root cause for this event could not be conclusively determined, Luminant Power believes that the probable cause of the event is an approximately 3/16 inch diameter paint drop that was not cleaned off of the 6L fuel pump control rack following painting activities on the EDG. This paint drop is believed to have prevented the operation of the control rack, which subsequently prevented the entire mechanical linkage from rotating. The lack of movement precluded sufficient fuel from being admitted to all of the cylinders which prevented the engine from starting.
V. CORRECTIVE ACTIONS
The Unit 1 Train B EDG was immediately declared to be inoperable and troubleshooting was initiated to determine the cause for the failure to start. All painting activities in the EDG rooms were suspended until the cause could be determined and addressed. The fuel racks on the other three EDGs were manually manipulated to determine their condition. The paint residue was removed from the Unit 1 Train B EDG and it was successfully started, declared operable, and returned to service.
As a part of the CPNPP Corrective Action Program, the following corrective actions will be taken for this event:
The CPNPP General Plant Painting procedure will be revised to 1) require a post maintenance "pull test" of the fuel pump control rack mechanisms to ensure they are free to operate (in addition to the visual inspections), 2) require "as you go" inspection and cleanup when painting around sensitive components, and 3) add an attachment capturing the pictures and information presently contained in the pre-job briefing notebook used by painters.
This event will be included in the pre job briefing for painters to heighten their sensitivity to the problems paint drops and spatter can cause for mechanical linkages. The information in the pre job briefing notebook used by painters will be examined by System Engineering to ensure that it contains all sensitive areas on the EDG that should not be painted.
VI. PREVIOUS SIMILAR EVENTS
There have been no previous similar reportable events at CPNPP in the last three years.