05000461/LER-2009-001

LER-2009-001, Safety Function Lost Due to Capacitor Failure on Circuit Card

Docket Number
Event date:
Report date:
Reporting criterion:	10 CFR 50.73(a)(2)(v)(D), Loss of Safety Function - Mitigate the Consequences of an Accident
4612009001R00 - NRC Website

PLANT OPERATING CONDITIONS

Unit: 1 Event Dates: 7/2/09 Event Time: 7/2/09, 0100 hours0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br /> Central Daylight Time Mode: 1 (Power Operation) Reactor Power: 97 percent

DESCRIPTION OF EVENT

At 0100 hours0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br /> on 7/2/09, control room operators received an alarm [ALM] indicating the Nuclear Systems Protection System (NSPS) [JE] Self Test System (STS) failed. Initial investigation identified the reason for the failure was a failure of Division 3 logic circuit card 1PAP663BA16A114: High Pressure Core Spray (HPCS) [BG] Error Code 1 (HPCS-1) system test. The operators attempted to restart the STS and it failed again for the same reason. In accordance with station procedures, the Instrumentation Maintenance Department was required to investigate the second failure, and this task was assigned to the day-shift.

Issue Report 938015 was initiated to track investigation of this issue.

The NSPS consists of four independent and redundant divisions of safety-related solid-state circuitry used to scram the reactor and to initiate emergency core cooling systems and engineered safety feature systems. The STS is a testing and surveillance system designed to automatically and continuously monitor the NSPS functional circuitry. The STS provides the means to continuously monitor the logic circuit integrity and the circuit continuity of the NSPS systems once every 40 minutes.

At about 1035 hours0.012 days <br />0.288 hours <br />0.00171 weeks <br />3.938175e-4 months <br />, initial troubleshooting at the station determined the Division 3 STS logic was locked­ up and the Division 3 HPCS logic circuit card was identified to be the problem.

At 1108 hours0.0128 days <br />0.308 hours <br />0.00183 weeks <br />4.21594e-4 months <br />, the Operations shift manager held a preemptive discussion with the Main Control Room team concerning a manual start of HPCS with logic inoperable due to concerns with the Division 3 logic circuit card.

At 1137 hours0.0132 days <br />0.316 hours <br />0.00188 weeks <br />4.326285e-4 months <br />, preparations commenced to allow on-site testing of the Division 3 logic circuit card and to develop a strategy to obtain vendor repair of the card if needed.

At 1230 hours0.0142 days <br />0.342 hours <br />0.00203 weeks <br />4.68015e-4 months <br />, a dedicated main control room operator was designated to manually start and initiate HPCS if required.

At 1415 hours0.0164 days <br />0.393 hours <br />0.00234 weeks <br />5.384075e-4 months <br />, troubleshooting concluded that the most likely cause of the STS failure was malfunction of the Power On Initialization (P01) circuitry on the Division 3 logic circuit card. This failure resulted in the output initiation signals from this card being blocked, preventing automatic HPCS initiation and automatic start of the Division 3 emergency diesel generator [EK] [DG] and the Division 3 Shutdown Service Water System (SX) [BI] pump [P]. At this time, operators declared the HPCS, Division 3 DG, and Division 3 SX systems inoperable but available and entered the applicable Technical Specification action requirements, requiring restoration of HPCS within 14 days. The failure of the Division 3 logic circuit card does not prevent manual start of the HPCS pump, opening of the injection valve, manual start of the Division 3 DG or manual start of Division 3 SX pump using hand switches in the Main Control Room. Operators verified the Reactor Core Isolation System (RCIC) [BN] was operable as required by Technical Specification required actions.

_ At about 1608 hours0.0186 days <br />0.447 hours <br />0.00266 weeks <br />6.11844e-4 months <br />, the Division 3 logic circuit card was removed for further site testing (using a GENRAD tester). During this site testing, the POI circuitry of the Division 3 HPCS logic circuit card failed the test.

Subsequently, the Division 3 logic circuit card was sent to the supplier for further troubleshooting and diagnostic testing. Supplier investigation of the Division 3 logic circuit card confirmed the card failure was due to degraded coupling ceramic capacitors [CAP] in the POI circuitry. The card was repaired by the supplier and returned to the station.

At 1350 hours0.0156 days <br />0.375 hours <br />0.00223 weeks <br />5.13675e-4 months <br /> on 7/7/09, following completion of installation of the card, post-maintenance testing and restoration of the automatic functions of the card, HPCS, and Division 3 DG and SX were declared operable.

No other inoperable equipment or components directly affected this event.

CAUSE OF EVENT

The cause of the Division 3 logic circuit card failure is attributed to a knit-line delamination and associated cracks that caused a low insulation resistance of the ceramic capacitor in the POI circuit of the card that was installed in 1987. The ceramic capacitor is one of two coupling capacitors in series with a resistor to form a time delay that prevents erroneous signals from actuating field devices during circuit card power-up.

The purpose of the POI circuit is to set latches to a predetermined state during power-up or during a circuit card removal, and to inhibit the outputs to the field instrumentation during power-up to allow logic setting time. The circuit analysis determined that one degraded ceramic capacitor on the POI circuit caused the HPCS-1 circuit card to malfunction, blocking initiation signals for HPCS, Division 3 DG and Division 3 SX systems. The vendor failure analysis of the capacitor was unable to identify the cause of the knit-line delamination and cracks; however, further industry research shows the most probable cause of the defective capacitor is a manufacturing anomaly that limited the expected life of the capacitor.

SAFETY ANALYSIS

This event is reportable under the provisions of 10 CFR 50.73(a)(2)(v)(D) due to a condition that could have prevented fulfillment of the HPCS safety function to mitigate the consequences of an accident.

The HPCS and Division 3 DG and Division 3 SX systems were inoperable but available from 0100 hours0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br /> on 7/2/09 until declaration of operable status at 1350 hours0.0156 days <br />0.375 hours <br />0.00223 weeks <br />5.13675e-4 months <br /> on 7/7/09, except for a period of 71 minutes from 1200 hours0.0139 days <br />0.333 hours <br />0.00198 weeks <br />4.566e-4 months <br /> to 1311 hours0.0152 days <br />0.364 hours <br />0.00217 weeks <br />4.988355e-4 months <br /> on 7/7/09. During this time, the HPCS and Division 3 DG and Division 3 SX systems were inoperable and unavailable while control power fuses for the HPCS pump were removed for installation of the repaired Division 3 logic circuit card.

During the times the HPCS and Division 3 DG and Division 3 SX systems were available, the manual start capability of the HPCS pump and the functions for opening the injection valve using hand switches in the main control room continued to be available for operator manual initiation if required. Additionally, the manual initiation functions for the Division 3 DG and SX systems were available in the main control room if required during this time.

The RCIC system was operable during this event. Although no credit is taken in the safety analysis for the RCIC System, it performs a similar function as HPCS but has reduced makeup capability. Nevertheless, it will maintain inventory and cool the core, while the Reactor Coolant System is still pressurized, following a reactor pressure vessel isolation. If HPCS fails to maintain reactor water level above Level 1, it is backed up by automatic initiation � of Automatic Depressurization System in combination with Low Pressure Coolant Injection [BO] and Low Pressure Core Spray [BM] systems; these systems remained operable during this event.

CORRECTIVE ACTION

The Division 3 logic circuit card that failed has been repaired and a spare Division 3 logic circuit card will be obtained.

A Performance Centered Maintenance template will be created and strategies for performing preventive maintenance activities will be evaluated for NSPS circuit cards.

PREVIOUS OCCURRENCES

The 7/2/09 event was a repeat failure of STS identified on 6/24/09 (in issue report 934532) that caused the STS system to stop testing. The STS 6/24/09 failure was an intermittent failure on the HPCS-1 circuit card that was reset and ran successfully in fully automatic test mode. This event was not reportable under the provisions of 10 CFR 50.73.

COMPONENT FAILURE DATA

Circuit Card Manufacturer: General Electric Nomenclature: HPCS-1 circuit card Manufacture Model Number: 147D8500G001 Part Number: 1PAP663BA16A114 Ceramic Capacitor Manufacturer: KEMET Electronic Corporation Nomenclature: Multi-Layer Ceramic Capacitor, one micro-Farad Manufacturer Model Number: 1CK06BX105K