05000348/LER-2013-002
| Joseph M. Farley Nuclear Plant, Unit 1 | |
| Event date: | 10-04-2013 |
|---|---|
| Report date: | 01-31-2014 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications 10 CFR 50.73(a)(2)(v), Loss of Safety Function |
| 3482013002R01 - NRC Website | |
Westinghouse — Pressurized Water Reactor Energy Industry Identification Codes are identified in the text as [XX]
Description of Event
On October 4, 2013 with Unit 1 in Refueling Mode (Mode 6), a test of B-Train safety injection actuation concurrent with a loss of off-site power was conducted per procedure FNP-1-STP-40.0B (Safety Injection With Loss Of Off-Site Power Test - B Train). At 1202 CDT operators manually disconnected B- Train electrical buses from off-site power and manually initiated a safety injection signal as directed by the procedure. In response to these actions the 1B Emergency Diesel Generator [DG] auto-started and auto-connected to the B-Train bus, however the Emergency Safeguards System (ESS) sequencer [44] did not actuate to auto-start required ESS loads as expected. Operators recognized the failure of the sequencer to actuate and immediately manually started service water pumps [P] to provide cooling water to the running diesel generator. Verification that a running diesel has supporting service water pumps in operation is a simple immediate operator action in the emergency response procedures that is performed at the main control board and trained on. An investigation of the failure of the sequencer to run was immediately initiated and promptly identified the failure of the Mechanism Operated Cell (MOC) switch [IEL] for the 1B Diesel Generator output breaker [BKR] to actuate upon closure of the diesel output breaker which resulted in the failed actuation of the sequencer.
There were no operator errors or test procedure deficiencies that contributed to this event and no equipment damage or adverse operational transient occurred as a result of this event. B-Train systems were not required by Technical Specifications to be operable during the test performance due to the existing plant conditions (Mode 6 with the refueling cavity filled to refueling level). A-Train components required by Technical Specifications to be operable remained operable and unaffected throughout this event.
The 1B Diesel Generator output breaker MOC switch, in addition to providing an input for actuation of the ESS sequencer, provides breaker position indication to the Integrated Plant Computer (IPC) [CPU].
A post-event review of historical IPC data for the 1B Diesel Generator output breaker position indication revealed that the IPC did not consistently indicate the proper breaker position when the 1B Diesel Generator output breaker was closed. Beginning in August of 2010 until the time of the event, proper breaker position indication on the IPC was not received for nineteen of sixty-one closures of the 1B Diesel Generator output breaker. Troubleshooting of the IPC indication circuit identified no other problems with this circuit. Consequently, the nineteen occurrences of improper IPC breaker position indication for the 1B Diesel Generator output breaker have been attributed to the MOC switch for that breaker not actuating. Due to the history of this MOC switch the 1B Diesel Generator is considered to have been inoperable for extended periods between August of 2010 and October of 2013. This represents a condition prohibited by Technical Specifications and is reportable under 10 CFR 50.73(a)(2)(i)(B). Due to opposite train (A-Train) equipment having been out of service on various occasions for maintenance activities during the periods of 1B Diesel Generator inoperability this is also reportable as a condition that could have prevented fulfillment of a safety function per 10 CFR 50.73(a)(2)(v)(B,C,D).
The MOC switch that failed to actuate is a type Q-10 MOC switch (16 stage) manufactured by Siemens.
The switch installed on the 1B Diesel Generator output breaker also performs a safety related function of actuating the loss of off-site power (LOSP) sequencer during an LOSP event. Following this event, the MOC switch and the breaker for the 1B Diesel Generator were replaced and tested per work order SNC524823 prior to Unit 1 entering a condition in which 1B Diesel Generator operability was required.
Cause of Event
The direct cause of this event was determined by examination and testing to be inadequate lubrication of the MOC switch. Causal analysis of the inadequate lubrication determined the root cause of this event to be an inadequate procedure review process in 2002 that resulted in MOC switch preventive maintenance procedures having no associated task directing the performance of the procedure. The procedure review process was subsequently corrected for reasons unrelated to this event to require cross disciplinary review when a maintenance task is changed. A contributing cause of this event was determined to be inadequate monitoring of MOC switch performance by station personnel.
Safety Assessment At the time of the event, Unit 1 was in Mode 6 with the refueling cavity filled to refueling level.
A-Train components were in an operable condition to meet the operability requirements of Technical Specifications for this mode. A-Train components were unaffected by and remained operable during the event. There were no adverse operational transients and no equipment damage that resulted from the actuation failure of the MOC switch. Public health and safety were unaffected by the test failure.
The reliability and availability of the 1B Diesel Generator and its output breaker were not affected by this condition. The condition affected the ability for the sequencer to automatically sequence loads on the bus in the event of an LOSP or an LOSP concurrent with a safety injection signal. The failure of the MOC switch is recoverable by simple operator action from the control room and is directed by procedure.
Additionally, during the performance of FNP-1-STP-40.0B on October 4, 2013, operators identified the failure and took the appropriate manual actions to start loads when the MOC switch failed to operate.
This gives further confidence that the event would be mitigated and the safety significance was low.
As reported in Farley Unit 1 LER 2013-001-00, an actual Unit 1 B-Train loss of off-site power event occurred on June 11, 2013. During this event, the 1B Diesel Generator MOC switch satisfactorily actuated and the associated LOSP sequencer operated as expected.
Corrective Action To restore the 1B Diesel Generator output breaker MOC switch to an operable condition the MOC switch and the output breaker were replaced and tested per work order SNC524823.
Historical breaker position indication data from the IPC for all other diesel generator output breakers for both units has been reviewed with no other anomalies identified. The output breakers for the remaining four diesel generators were also cycled following this event to verify proper MOC switch operation. IPC data and condition report history have also been reviewed to identify MOC switch failures in other plant breakers. A total of seven other MOC switches have been identified as not functioning properly when operated. None of these MOC switches have resulted in inoperable Emergency Safeguards Function (ESF) equipment.
A sampling of the MOC switches exhibiting unreliable operation, including the 1B DG MOC switch, were provided to a vendor for failure analysis. The vendor analysis identified the direct cause of the unreliable operation to be inadequate lubrication of the MOC switch, particularly of the shaft bushings. To address this condition, the station's MOC switch maintenance procedures were upgraded to include all vendor recommended maintenance, including lubrication of the shaft bushing and other components. The revised maintenance procedures are being performed on an expedited basis on all MOC switches that perform safety-related functions. This work was completed on all of these MOC switches as allowed by on-line plant conditions on January 26, 2014. Additionally, recurring tasks are being created to direct performance of the MOC switch maintenance procedure at vendor-recommended frequencies on all MOC switches that perform safety-related functions.
To address the root cause of an inadequate procedure review process, the review process had been corrected prior to this event for unrelated reasons. To address the extent of cause, Engineering personnel will review current preventative maintenance tasks for all plant ESF equipment and ensure vendor-recommended preventive maintenance strategies are implemented. If it is determined that vendor recommended strategies will not be implemented in certain cases, a technical basis for the variance will be required.
To address the contributing cause of inadequate monitoring of MOC switch performance, interim written instructions for monitoring MOC switch performance have been provided to station personnel. These instructions will be replaced by formal procedural requirements. Additionally, a case study of this event will be developed and presented to a targeted plant population.
Additional Information
The following LER's have been previously submitted regarding MOC switch operation:
a bent MOC switch contact.
error between the breaker and the MOC switch.