05000440/LER-2009-002
| Docket Number06 22 2009 2009 - 002Dd- 00 08 20 2009 | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(ii)(B), Unanalyzed Condition |
| 4402009002R00 - NRC Website | |
Energy Industry Identification System Codes are identifiedPin the text as [XX].
INTRODUCTION
On June 22, 2009, it was discovered that a wiring error in a fire protection carbon dioxide (CO2) [LW] panel [PL] interface resulted in a condition in which a fire protection CO2 actuation signal for the Division 3 diesel generator [EK] room would cause the Division 2 diesel generator room ventilation [VJ] supply fans to isolate. Additionally, a fire protection CO2 actuation signal for the Division 2 diesel generator room would cause the Division 3 diesel generator room ventilation supply fans to isolate. The condition existed from January 15, 2009, to June 23, 2009, when the wiring error was corrected.
This issue is being reported in accordance with 10 CFR 50.73(a)(2)(ii)(B) as any event or condition that resulted in the nuclear power plant being in an unanalyzed condition that significantly degraded plant safety.
EVENT DESCRIPTION
All 3 diesel generator CO2 Fire Suppression System control panels were replaced in December of 2008 under an engineering modification. Testing was performed in accordance with the engineering modification testing requirements. The output wires from the CO2 panels to the diesel generator ventilation relays were lifted to maintain diesel generator operability during the testing.
Testing was completed on January 15, 2009 and the CO2 control panel was placed in 'lockout.' Plant Engineering recommended placing the CO2 Fire Suppression System control panels back into automatic service on January 19, 2009. On January 25, 2009 at approximately 1655 hours0.0192 days <br />0.46 hours <br />0.00274 weeks <br />6.297275e-4 months <br />, plant operations personnel removed the divisional CO2 systems from 'lockout,' thereby placing the CO2 Fire Suppression System control panels into automatic (i.e., normal) service.
A partial performance test was performed on the Division 3 CO2 panel on February 4, 2009. During the test it was noted that when a CO2 initiation occurred in the Division 3 diesel generator room, the associated fans did not trip. Testing was stopped and per the recommendations of the control room, all 3 Fire Suppression System control panels were placed into 'lockout.' The failed test was documented in the corrective action program and a troubleshooting order was created. On June 12, 2009, during troubleshooting of the cause of Division 3 fans and louvers not changing state during CO2 initiation, it was identified that the wires coming from the control panel to the trip relays in the control room for Division 2 and 3 fans were swapped and labeled incorrectly.
On June 22, 2009, a condition report was initiated to document that during a review of the Division 3 diesel generator CO2 Fire Suppression System control panel miswiring issue, the diesel generator system engineers identified a period of time where the wiring issue would have affected the Division 2 and Division 3 diesel generator ventilation supply fans and there were no follow-up actions to address the miswiring issue identified on June 12, 2009. In the discovered wiring configuration, from January 15, 2009, through June 23, 2009, a fire protection CO2 actuation signal for the Division 3 diesel generator room would cause the Division 2 diesel generator room ventilation to isolate. Additionally, a fire protection CO2 actuation signal for the Division 2 diesel generator room would cause the Division 3 diesel generator room ventilation to isolate. On June 23, 2009 at 1210 the crossed wires were corrected and the incorrect wire labels were removed.
�
CAUSE OF EVENT
The root cause was determined to be an inadequate post modification test which failed to identify the miswiring of two output wires from the diesel generator CO2 Fire Suppression System control panel. The specific failure mechanism in this event was the mislabeling and subsequent mislanding of two output wires from the diesel generator CO2 Fire Suppression System control panels. The mislanding of the two wires impacted the Division 2 and Division 3 diesel generators. This condition should have been identified by the post modification testing and corrected prior to returning the equipment to service. It was determined that the post maintenance/modification testing requirements do not require sufficient rigor to assure system functionality prior to returning equipment/systems to service. A contributing cause was a less than adequate cable tag/wire mark configuration control process that led to the mislabeling and eventual mislanding of the two output wires.
EVENT ANALYSIS
All 3 diesel generator CO2 Fire Suppression System control panels were replaced in December of 2008 under an engineering modification. The replacement of the obsolete CO2 fire protection panels was created to enhance reliability of the CO2 systems and minimize the possibility of inadvertent CO2 releases. The modification replaced the three sets of circuit boards with two new redundant panel control units, each capable of independently controlling each of the three systems separately. This was to maintain separate control of each system without disabling all three systems if a problem develops in a control unit.
On June 22, 2009, engineers identified a period of time where a CO2 Fire Suppression System wiring issue would have affected the Division 2 and Division 3 diesel generator ventilation supply fans. The issue was identified in the corrective action program on June 22 and was reviewed by the Senior Reactor Operator (SRO) on June 23, 2009. The SRO comments state that the wiring issue has been corrected in the plant and does not currently challenge operability of the Division 2 or 3 diesel generators.
The fire protection safe shutdown analysis requires the operation of one of the two HVAC fans in each divisional diesel generator room for support of the emergency diesel generator operation to achieve and maintain safe shutdown in the event of a plant fire. The SRO stated that if neither ventilation fan was able to be started, this could make the Division 2 diesel generator inoperable and requested a past operability review to determine if the Division 2 diesel generator was made inoperable during the testing performed. The reportability review was due at 1500 on June 30, 2009. At the time this event was determined to be reportable, the plant was at 100 percent power.
Over the duration of the miswiring condition, the plant was in every MODE 1-5, as well as defueled.
From February 23, 2009, until May 13, 2009, the plant was in a refueling outage.
The time period during which the diesel generators were affected by the wiring error began on January 15, 2009, with the completion of post modification testing and the relanding of all the lifted leads on the terminals of the new CO2 Fire Suppression System control panels following a design modification, and ended on June 23, 2009, when the wiring error was corrected. In the discovered wiring configuration, a fire protection CO2 actuation signal for the Division 3 diesel generator room would cause the Division 2 diesel generator room ventilation supply fans to isolate. Additionally, a fire protection CO2 actuation signal for the Division 2 diesel generator room would cause the Division 3 diesel generator room ventilation supply fans tolsolate. During troubleshooting, it was identified that this condition existed whether the CO2 Fire Suppression System control panels were in 'normal' (automatic) or 'lockout.' The identified wiring configuration did not affect Division 1 diesel generator. Additionally, over the duration of the miswiring condition, there were active fire impairments to perform an hourly fire watch of the affected diesel generator rooms.
Later during the root cause investigation, it was determined that the Division 2 diesel generator was inoperable for approximately 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> and 52 minutes during the February 4, 2009, testing when a Division 3 CO2 actuation signal was given a trip signal to the Division 2 diesel generator ventilation supply fans. During this period of time, the Division 1 diesel generator was operable.
A bounding probabilistic risk assessment was performed for the time period the miswiring condition existed between January 15, 2009, and June 23, 2009. For this condition, the probabilistic risk assessment calculated a change in Core Damage Frequency (CDF) to be 4.41E-07. The Large Early Release Frequency (LERF) is on the order of 1.0E-08. Configurations with changes in CDF of less than 1.0E-06 and a LERF of less than 1.0E-07 are not considered to be significant risk events. Based on the probabilistic risk assessment results, this condition is considered to be of low safety significance.
CORRECTIVE ACTIONS
The two Division 2 and Division 3 CO2 Fire Suppression System control panel wiring label errors were corrected and the wires relanded to their correct terminals.
Procedure guidance will be revised to provide more detailed expectations concerning the scope and rigor of post maintenance/modification testing requirements. Additionally, the cable tag/wire mark configuration control process guidance and wiring verification requirements will be enhanced.
The Operations Superintendent will review with all licensed SROs the importance of safety system functional testing when maintenance or modifications have been performed on portions of the system.
Corrective actions will track satisfactory completion of the Division 1, 2, and 3 diesel generators CO2 Fire Suppression System Detection/Operability Testing.
PREVIOUS SIMILAR EVENTS
A search of Licensee Event Reports and the corrective action program over the past 3 years at the Perry Nuclear Power Plant found two similar events had been reported.
panel that was found to be installed incorrectly. The jumper was identified as a result of surveillance testing. The switch contact has the function of isolating control room circuitry from the remote shutdown panel circuitry for the Reactor Core Isolation Cooling (RCIC) system turbine exhaust valve. Complete isolation of the control room circuitry for the RCIC valve would not have been established by transferring control switches to the emergency position. The cause of the condition was determined to be a wiring drawing error made during manufacture of the panel that �NRC FORM 366A (9-2007) PRINTED ON RECYCLED PAPER resulted in the switch being incorrectly wired. The cause of the wiring error was determined to be a less than adequate vendor drawing review that failed to discover a drawing error on a wiring diagram and less than adequate testing.
Initial corrective actions consisted of correcting the miswired jumper in the remote shutdown panel and contacting the vendor's Engineering Manager, and informing him of the drawing error and the wiring error in the vendor supplied remote shutdown panel. Further corrective actions included revising remote shutdown surveillances to include testing to verify correct isolation and transfer functions of the Normal/Emergency switches in the remote shutdown panel to ensure the circuits meet the unique testing requirements for double isolation of the Fire Protection Program, and revising the Updated Safety Analysis Report to clarify information that was difficult to locate and information that conflicted with the Supplement to the Safety Evaluation Report (SSER). Corrective actions from LER 2006-001 could not reasonably be expected to have prevented the condition documented in LER 2009-002.
revision, it was discovered that one circuit of the Division 1 Emergency Diesel Generator (EDG) Control Room Pull-To-Lock (PTL) Control Switch was not designed to isolate the Control Room from the local Division 1 EDG controls in the event of a Control Room fire. At 1430 hours0.0166 days <br />0.397 hours <br />0.00236 weeks <br />5.44115e-4 months <br /> on May 4, 2006, with the plant in Mode 1 at 100 percent power, it was determined that this condition violated the Perry Nuclear Power Plant Fire Protection Program and could adversely affect plant shutdown in the case of a control room fire. A potential fire induced hot short in the diesel generator logic circuit could have resulted in a spurious trip of the diesel generator, even if control was transferred to local control. This condition has existed since 1989. Interim actions in the form of procedure changes have been completed to address this issue. A final resolution to the issue was a design change to incorporate Appendix R Control Room isolation features to the diesel generator pull-to lock control switch circuit. Corrective actions from LER 2006-003 could not reasonably be expected to have prevented the condition documented in LER 2009-002.
COMMITMENTS
There are no regulatory commitments contained in this report. Actions described in this document represent intended or planned actions, are described for the NRC's information, and are not regulatory commitments.
NRC FORM 366A (9-2007) PRINTED ON RECYCLED PAPER