05000483/LER-2009-004
| Docket Number | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications |
| 4832009004R00 - NRC Website | |
1.�DESCRIPTION OF THE REPORTABLE EVENT
A. REPORTABLE EVENT CLASSIFICATION
10CFR50.73(a)(2)(i)(B) requires reporting of any operation or condition prohibited by a plant's Technical Specifications (TS). Following the loss of one offsite power circuit, Required Action A.1 of the Limiting Condition of Operation (LCO) of TS 3.8.1 requires the performance of a surveillance requirement to verify the correct breaker alignment and power availability for the remaining Operable offsite circuit within one hour and once every eight hours thereafter, among other actions.
An unlit undervoltage indication light on the Main Feeder Breaker to a safety-related distribution bus, indicating the loss of one offsite power circuit, was discovered at 0927 on 4 July 2009. However, Required Action A.1 of TS LCO 3.8.1 was not performed until 1850 on 6 July 2009, which is beyond the Completion Time required by Required Action A.1. Consequently, Callaway was in a condition prohibited by Technical Specifications.
B. PLANT OPERATING CONDITIONS PRIOR TO THE EVENT
The plant was in MODE 1, Power Operation, at 100% reactor power at the time of the event.
C. STATUS OF STRUCTURES, SYSTEMS OR COMPONENTS THAT WERE INOPERABLE AT THE
START OF THE EVENT AND THAT CONTRIBUTED TO THE EVENT
No structures, systems, or components were inoperable at the start of the event which contributed to the event.
D. NARRATIVE SUMMARY OF THE EVENT, INCLUDING DATES AND APPROXIMATE TIMES
At 0617 on 4 July 2009, two switchyard breakers opened on protective relay commands in response to a lightning strike. The Control Room (CR) staff observed plant computer alarms due to plant equipment actuations caused by the lightning strike. Operating Technicians (OTs) were dispatched to investigate and restore normal plant conditions.
At 0927, during these investigations, an OT noted that an indication light on Main Feeder Breaker NB0112 [EllS System: EB; Component: BKR] to safety-related 4160V distribution bus NB01 [EllS System: EB; Component: BU] was unlit. This light is an undervoltage indication light and belongs to a circuit that monitors power available from the Train A Engineered Safety Feature (ESF) Transformer, XNB01 [EllS System: EB; Component: XFMR]. This is a unique circuit which receives power from two potential transformers. One potential transformer receives power from the 4160V A-B phase connections, and the other potential transformer receives power from the 4160V B-C phase connections. On the secondary side of the potential transformers, the B phase is treated as a common tap for both potential transformers.
In this event, the A-B phase potential transformer, NBPT0112F [El IS System: EB; Component: XPT], had failed. NBPT0112F provides voltage to the local undervoltage indication light and the bus synchronizing relay, among other components. The components receiving voltage from NBPT0112F do not have safety-related functions. However, TS Surveillance Requirement (SR) 3.8.1.16 requires verifying that each Emergency Diesel Generator (EDG) [EIIS System: EK; Component: DG] can synchronize with its offsite power source when the EDG is supplying the associated safety bus and is loaded with emergency loads.
Since the NB0112 synchronizing relay had lost power, it was not capable of performing this function.
Synchronization to the alternate source via alternate feeder breaker NB0109 could be credited for compliance with this TS requirement. However, this equipment alignment would leave both safety- related distribution buses NB01 and NB02 receiving offsite power from the Startup Transformer [EllS System: EA; Component: XFMR]. In this configuration, the normal offsite power source for NB01 must be declared inoperable. Consequently, TS LCO 3.8.1 Required Action A.1 should have been entered at the time the undervoltage indication light was discovered unlit. However, several factors led to the belief that only metering circuits were affected, and Required Action A.1 was not immediately entered.
Because the local bus indication lights have no labels to indicate which specific device they are associated with, the OTs had difficulty discerning indication lights associated with undervoltage indication. The OT recorded the condition in the OT logs as "white light out on the breaker" but in an incorrect log point that ultimately delayed the notification of the condition to the Control Room until 1730.
If the OT had logged the condition in the correct log point, he would have been prompted to contact the Field Supervisor (FS).
At 1820, the operating crew began to investigate the unlit voltage indication light. The light bulb of the unlit indication light was changed three times, confirming that the bulb itself had not failed. The FS and Reactor Operator (RO) reviewed the NB0112 breaker schematics, but incorrectly identified the affected equipment. The CR staff incorrectly concluded that the unlabeled light on NB0112 was an indication of control power to the closing springs. The breaker closing springs were subsequently verified to be charged, and since NB01 showed normal voltage, the condition was assumed to be only an indication issue resulting from the lightning strike.
After shift turnover, the oncoming Shift Manager (SM) and Shift Engineer (SE) continued to investigate the condition. Upon visual inspection, the SM correctly identified the unmarked, unlit indication light as an undervoltage indication light, not a closing spring control power indicator. The SE correctly identified the unlit undervoltage indication light on the system drawings and also noted that the indication light had no component identifier.
The SM and SE analyzed the circuit, both locally and via system drawings, for several hours. Based on this analysis, local indications, and the lack of computer alarms, the SM and SE concluded that 6-amp fuses that feed local meters and the undervoltage indication light had failed. However, the unique design of the circuit was misunderstood by the operating crew, and this conclusion was also incorrect.
The CR staff contacted the system engineer and discussed the SM's assessment. After an independent assessment, the system engineer came to the same incorrect conclusion. As a result, the operating crew felt that the repair could wait until the maintenance crews arrived on Monday, July 6.
Upon investigation by the maintenance crews on 6 July 2009, the assumed failure of the 6-amp fuses was eliminated as a possible cause of the loss of voltage indication. The failure of Potential Transformer NBPT0112F was then correctly identified. The operating crew determined that distribution bus NB01 was incapable of a synchronized bus transfer if the EDG was supplying the bus. At 1820 on 6 July 2009, the SM correctly declared the normal offsite power source inoperable and entered TS LCO 3.8.1 Action Statement A.1.
TS LCO 3.8.1 Required Action A.1 requires that the remaining offsite circuit is verified to be Operable within one hour and once every eight hours thereafter. Upon declaring the normal offsite circuit inoperable, the remaining offsite circuit was initially verified at 1850 on 6 July 2009. However, since the condition was first discovered by the OT on 0927 on 4 July 2009, initial verification of the remaining offsite circuit needed to have been performed by 1027 on 4 July 2009 (and again once every eight hours thereafter) to meet the requirements of TS LCO 3.8.1. Consequently, Callaway was in a condition prohibited by Technical Specifications from 1027 on 4 July 2009 through 1850 on 6 July 2009.
In addition, TS LCO 3.8.1 Required Actions G.1 and G.2 require the plant to be in Mode 3 within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and in Mode 5 within 36 hours4.166667e-4 days <br />0.01 hours <br />5.952381e-5 weeks <br />1.3698e-5 months <br />, respectively, when the Required Actions and associated Completion Times of Condition A are not met. Since it was not known that Callaway had not met the Completion Time of TS LCO 3.8.1 Required Action A.1, Required Actions G.1 and G.2 were also not performed within their associated Completion Times.
NBPT0112F was replaced, and the offsite source was restored to Operable status at 0439 on 7 July 2009. TS LCO 3.8.1 Required Action A.3 requires the offsite circuit to be restored to Operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. This Action was met within the allowed TS Completion Time.
E. METHOD OF DISCOVERY OF EACH COMPONENT, SYSTEM FAILURE, OR PROCEDURAL ERROR
The indication of component failure was first discovered during OT rounds at 0927 on 4 July 2009 when a white indication light was found unlit. The failure of NBPT0112F was not discovered until 1820 on 6 July 2009 during a job to further investigate the circuit associated with the unlit white indication light.
2.0 EVENT DRIVEN INFORMATION
A. SAFETY SYSTEMS THAT RESPONDED
No safety systems responded to this event. No safety systems were required to respond to this event.
B. DURATION OF SAFETY SYSTEM INOPERABILITY
No safety systems were inoperable for this event. Although the ability to synchronize an EDG powering its associated bus to an offsite power source is a TS requirement, the offsite source synchronizing relay does not perform a safety-related function.
C. SAFETY CONSEQUENCES AND IMPLICATIONS OF THE EVENT
The offsite source synchronizing relay does not perform a safety-related function. As such, the safety significance of this event is very low. The ability to synchronize an EDG powering its associated bus to an offsite power source is a requirement of TS SR 3.8.1.16 and Regulatory Guide 1.108.
3.� CAUSE(S) OF THE EVENT AND CORRECTIVE ACTION(S) Three Causal Factors (CF) have been identified for this event, and a Root Cause and Corrective Action to Prevent Recurrence (CATPR) have been determined for each Causal Factor. Further discussion of the Root Cause and CATPR is detailed below for each Causal Factor.
CF-1: THE MEANING OF THE UNLIT INDICATION LIGHT WAS NOT UNDERSTOOD The Root Cause of CF-1 is lack of written instruction for proper response to an unlit undervoltage indication light. Instructions such as startup, normal or alarm response procedures were never written because the function and TS significance of the indication light were not understood. None of the procedure development processes in place led to discovery or correction of the inadequate written instructions.
Had written instructions been written and posted locally, or had information labels been installed at the bus indication lights, the OT would have had sufficient guidance to inform the CR staff of the Required Actions associated with the circuit quickly enough to comply with Technical Specifications. Per CATPR-1, written instructions will be developed and posted locally at the distribution bus for OT use.
The electronic OT log will be enhanced to reference these new procedures, give direction in the event of an unlit indication light, identify the applicable Technical Specifications, and direct the OT to inform the FS of the issue. While reporting the indication light condition in the incorrect operator log location (which resulted in a CR notification delay) did not meet Callaway expectations for prompt CR notification, it was determined to not be causal to the event. However, revision of the OT logs will eliminate the possibility of logging the condition under a related but incorrect point as part of this CATPR.
CF-2: NO TRAINING WAS PROVIDED ON THE FUNCTION OR SIGNIFICANCE OF THE UNLIT
INDICATION LIGHT
The Root Cause of CF-2 is lack of training on the proper response to an unlit undervoltage indication light. Training to provide knowledge of the function or significance of the indication lights on the safety related distribution bus does not exist. Operations personnel knew the indication lights not connected to individual breaker controls were related to undervoltage relay metering. However, they did not know which relays or functions were served by the individual lights. Operations personnel could not quickly find the indication lights on the correct drawing or understand the potential transformer design unique to NB0112 well enough to make an accurate Immediate Operability Determination. Training development processes have not provided the training tools necessary to successfully cope with this event.
If the Operations crews had known the function or significance of the indication light, this event could have been avoided. CATPR-2 will implement training to teach the lessons learned from this event.
CF-3: DISTRIBUTION BUS INDICATION LIGHTS WERE NOT LABELED LOCALLY OR ON DRAWINGS The Root Cause of CF-3 is the absence of indication light labels on drawings and local equipment. As previously stated, component identification and labeling on local equipment and Class 1 E drawings did not exist for the NB01 indication lights. The OT communicated the unlit indication light as, "white light out on the breaker." Equipment identifiers did not exist to aid CR evaluation of the condition. During this event, a RO reviewed electrical drawings of NB0112 to find two white lights drawn. Neither light was labeled on the drawing, and neither one was the undervoltage indication light the OT found unlit.
No local equipment labels exist to allow staff to correctly identify the component or to reference a system drawing. The operating crew had no way to know that the unlit undervoltage indication light signified the plant was in TS LCO 3.8.1 Condition A with one hour to comply with Required Action A.1.
No component identification process has led to discovery or implementation of the component identifiers needed to successfully manage this event.
Had the indication lights on NB01 been labeled, and had the appropriate information labels been installed, the CR staff would have been able to correctly trace components and drawings in sufficient time to comply with TS 3.8.1. Per CATPR-3, component identifiers will be provided on Class 1 E drawings for the indication lights on NB01 and NB02 distribution buses that are not associated with breaker controls. In addition, component labels will be created and installed locally at the indication lights.
The extent of the condition in this event is limited to the unmarked indication lights associated with distribution buses NB01 and NB02 with a one-hour TS Action Statement. In addition to NB0112, three other breaker cubicles have unmarked white indication lights not related to breaker controls. As a Remedial Action, labels have been applied to these four cubicles that mark the indication lights associated with the safety-related buses and transformers. The labels instruct the operating crew to enter TS LCO 3.8.1 if found unlit. Operating crews have been informed of the purpose and significance of the labels.
These labels will augment the permanent labels upon their installation.
It should be noted that the unique circuitry of NBPT0112F allowed it to fail without a CR alarm or annunciation. Operating crews can successfully manage all other Required Action time requirements, including one-hour Action Statements, that become evident through CR annunciation because annunciator response procedures are available.
Because the discovery of the condition was coincident with numerous plant equipment alarms and actuations as a result of the lightning strike, the operating crew experienced high workload pressure which may have contributed to the event.
4.� PREVIOUS SIMILAR EVENTS A search of Callaway Operating Experience (OE) did not find a previous internal event that was directly applicable to this event.
The most applicable event at Callaway occurred in August 2000 during calibration of an offsite synchronizing relay.� In this event, the offsite source was not declared inoperable, causing an unrecognized entry into TS LCO 3.8.1 Condition A. However, the August 2000 event was not a missed learning opportunity because the event resulted from work being performed outside of the schedule. It could not have revealed the underlying faults that caused the event described in this report.
5.�ADDITIONAL INFORMATION NBPT0112F is a Potential Transformer manufactured by General Electric, Type JVM-3, Part Number 643X94. It is used in the 4160V Class 1E Medium Voltage Power System.
The cause of the NBPT0112F failure will be determined by an outside laboratory.
The system and component codes listed below are from the IEEE Standard 805-1984 and IEEE Standard 803A-1983 respectively.
System: EB, Medium-Voltage Power System, Class 1 E Component: XPT, Potential Transformer