05000247/LER-2009-005
| Indian Point 2 | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(iv)(B), System Actuation 10 CFR 50.73(a)(2)(iv)(A), System Actuation 10 CFR 50.73(a)(2)(v), Loss of Safety Function |
| 2472009005R00 - NRC Website | |
Note: The Energy Industry Identification System Codes are identified within the brackets ().
DESCRIPTION OF EVENT
On November 02, 2009, at 22:39 hours, while at approximately 100% steady state reactor power, an automatic Reactor Trip (RT) {JC} was initiated as a result of a Turbine Trip (TT) {JJ} due to a Main Generator {TB} protective trip (primary lockout relay 86P) (RLY). All control rods (AA) fully inserted and all required safety systems functioned properly. The plant was stabilized in hot standby with decay heat being removed by the Steam Generators (SG) {AB} via the Atmospheric Steam Dump Valves. The Emergency Diesel Generators {EK} did not start as offsite power remained available.
The Auxiliary Feedwater System {BA} automatically started as expected due to SG low level from shrink effect. Systems that did not perform as expected after the RT included: 1) one turbine stop valve had indication that the valve did not fully close.
At 22:41 hours, the Main Steam Isolation Valves (MSIVs) {SB} were closed in accordance with procedures due to indication that a turbine stop valve did not indicate closed, 2) At 22:41 hours, entered Technical Specification (TS) 3.4.1 (Reactor Coolant System Pressure, Temperature, and Flow Departure from Nucleate Boiling (DNB) Limits) due to pressurizer pressure less than TS limit. TS 3.4.1 action statement exited at 22:44 hours after pressurizer pressure returned to above the TS limit, 3) At 22:45 hours, the lower left main turbine control valve (MS-HCV-127-1) did not indicate closed in the Control Room (CR) (NA). At 23:06 hours, Nuclear Plant Operators (NP0s) reported all main turbine stop valves are closed. On November 2, 2009, at 23:55 hours, a four hour non-emergency notification was made to the NRC (Log Number 45474) for a reactor trip while critical and the notification included the eight hour non-emergency notification for actuation of the AFW system. The RT was reported under 10 CFR 50.72(b)(2)(iv)(B) and the AFW actuation reported under 10 CFR 50.72(b)(3)(iv)(A).
The event was recorded in the Indian Point Energy Center Corrective Action Program (CAP) as condition report CR-IP2-2009-04530. An investigation into the cause of the event and a post transient evaluation was initiated.
Prior to the event on November 2, 2009, at approximately 19:12 hours, an investigation was in progress as a result of the receipt in the CR of an "Exciter Trouble Alarm," on panel FB for the Main Generator voltage regulator exciter cabinet. Investigation by nuclear plant operators (NP0s) found that the number 1 and 2 positive 15 volts DC (+15 volts on #1, and + 15 volts on #2) power supplies were both in "Low Alarm" status, signaling a low output voltage of both power supplies. Operators attempted to reset the alarms without success and the alarms remained locked in. A system engineering investigation discovered that the primary and redundant power supply voltages were oscillating approximately +/- 0.5 volts from their nominal setpoints. At no time was the observed voltage low enough to cause an alarm or require a generator trip. The +15 volt power supply setpoints are approximately 14.6 volts DC.
Unit 2 has a General Electric Excitor-Generator {TL} coupled to a Westinghouse Turbine {TB}. The Generrex is a main generator excitor regulating device for producing the main generator field and is manufactured by General Electric Company {G080}. The Generrex Exciter system has +/- 15 volt power supplies {RJX} that are independently regulated by means of their own circuit cards. The power supplies are used to operate voltage sensitive electronics in the Generrex system. The Generrex system is a compound potential source exciter that utilizes three transformers {XFMR} housed above the main generator to obtain excitation power from additional stator bars (P-Bars) in the generator. AC power flows from the P-Bars into the transformers which, when the generator is excited, supply enough voltage to fulfill the maximum possible field voltage at all times of operation.
Cause of Event
The direct cause of the reactor trip was a turbine trip from actuation of the generator primary (86P) lock-out relay. The primary (86P) lock-out relay actuation was due to a two out of two trip logic made up in the trip logic circuitry in the Exciter Cabinet as a result of a high resistance ground on a common ground wiring connection to terminal 102TB10 of the +15 volt power supply #1. This ground connection is composed of four wires: 1) the +15 volt/#1 negative return leg (which is at earth ground potential), 2) the auctioneered output leg of the -15 volt/#2 power supply, 3) the main outgoing ground connection from the power supply racks to the Exciter Cabinet, and 4) the main power supply chassis ground which travels from the terminal block back inside the power supply. The electrical ground was a single connection daisy chained in a series connection across Exciter power supplies #1 and #2. When the high resistance occurred it appeared as a loss of electrical ground or an open circuit to the alarm card comparators. Should both the primary and redundant power supplies of any voltage level fail then a direct trip signal is sent from the Exciter Cabinet to the 86P relay which will initiate a turbine trip. The vibration of the main generator was shown to have aggravated the condition causing an intermittent electrical connection. The natural vibration of the turbine-generator caused this connection to connect and disconnect multiple times a second. The speed of this transition was likely high enough to overcome the latency of the trip circuit, effectively cutting off the trip output voltage before it could build up to the necessary 24 volts to actuate the trip logic circuitry.
The intermittent connection is also the reason for the voltmeter oscillations since the outgoing wire also serves as the ground connection to the voltmeter panel for this power supply. This condition is also why both required trip signals did not propagate to the trip circuit. The card design allows a trip signal to clear, or reset without operator action but the alarm indication will lock in.
The root cause is a poor Original Equipment Manufacturer (OEM) design of the common ground wiring connections on the power supply racks. Current manufacturing practices preclude the use of more than three terminations on any single terminal stud due to increased potential for the loss of a common connection (single point vulnerability).
Engineering Standard ENN-EE-S-008-IP (Electric Cable Installation Standard) states; "No more than three terminal lugs shall be landed on a common terminal stud, unless otherwise indicated on design documents/drawings or approved by Engineering.
Corrective Actions
The following corrective actions have been or will be performed under Entergy's Corrective Action Program to address the cause and prevent recurrence:
- The Generrex power supply wiring, connections and terminal points were inspected and the terminal stud was verified to be tight and connections did not move. The high resistance connection was disassembled and cleaned. After re-assembly and testing for proper operation the system was returned to service.
- A second ground connection was installed between the #2 power supply rack and the central cabinet ground to eliminate the single point vulnerability.
- A Generrex system upgrade was prepared for installation in refueling outage 19 and includes upgrading to solid state power supplies and modifying its wiring to adhere to the IPEC Cable Installation Standards. During the upgrade installation the ground wiring going from terminal 102TB10 to the alarm card rack will be tested to verify wire integrity. Generrex system upgrade installation and wire testing is scheduled for the spring 2010 refueling outage.
- A sample of trip-risk related cabinets and identified Single Point Vulnerable (SPV) cabinets will be inspected (Lovejoy control cabinets, main transformer control cabinets, Stator Cooling Water control cabinets, main turbine front standard). Visual inspections will be performed during the spring 2010 refueling outage to identify any terminal studs with more than three wires terminated on them and actions taken as necessary to correct the condition.
Event Analysis
The event is reportable under 10CFR50.73(a)(2)(iv)(A). The licensee shall report any event or condition that resulted in manual or automatic actuation of any of the systems listed under 10CFR50.73(a)(2)(iv)(B). Systems to which the requirements of 10CFR50.73(a)(2)(iv)(A) apply for this event include the Reactor Protection System (RPS) including RT, AFW System (AFWS) actuation and manual or automatic actuation of multiple MSIVs. This event meets the reporting criteria because an automatic RT was initiated at 22:39 hours, on November 2, 2009, and the AFWS actuated as a result of the RT due to a SG low level from shrink effect. As a result of indication that a turbine stop valve did not close, operators manually closed the MSIVs in accordance with plant procedures. No primary system failed to function properly therefore, there was no safety system functional failure reportable under 10CFR50.73(a)(2)(v).
When the immediate notification (Log Number 45474) was provided under 10CFR50.72, the 8-hour non-emergency notification for the AFWS actuation was identified but the notification did not include the closure of the MSIVs which is also reportable under 10CFR50.72(a)(2)(iv). This condition was recorded in the IPEC CAP.
Past Similar Events
A review of the past three years of Licensee Event Reports (LERs) for events that involved a RT from a Generrex Main Generator protective trip was performed. The review identified LER-2006-005 which reported a RT on November 15, 2006, due to a Generrex protective trip. The cause was determined to be a momentary loss of electrical ground to the alarm cards that monitor the Generrex power supplies. High resistance connections caused unreliable operation of the Generrex regulator circuit card. The circuit card has tin plated steel sleeves that mate to connection pins on the main power supply chassis. The connection pins showed oxidation. A contributing cause was a loose screw terminal where the grounds are mounted due to mechanical vibration of the Generrex power supply. Corrective actions included replacement of the #1 power supply circuit card, cleaning and lubricating the pins, providing proper connections for the common ground, checking remaining +/- 15 volt pins and terminal points for proper operation. The 2006 event recognized the four wire termination but believed it was justified as it complied with the OEM design and was verified for tightness.
Safety Significance
This event had no effect on the health and safety of the public.
There were no actual safety consequences for the event because the event was an uncomplicated RT with no other transients or accidents and the plant safely shut down as designed. Actuation of the AFWS is an expected reaction to full power reactor trips due to SG shrink effect which causes SG level to drop below a SG level trip set point.
The closure of the MSIVs was in accordance with plant procedures in response to indication that a turbine stop valve did not close. The stop valve was subsequently determined to be closed and the condition was due to an indicator switch. The turbine control valve that was not indicating fully closed was determined to have a sheared anti-rotation pin that connects the valve stem to the actuator causing the valve to rotate down and contact the valve seat prematurely. The control valve was determined to be closed with its valve plug on its seat. The anti-rotational pin was replaced and the valve spring actuator was adjusted to center. All required safety systems performed as designed in response to the RT.
There were no significant potential safety consequences of this event.
The RPS is designed to actuate a RT for any anticipated combination of plant conditions including a direct RT on TT unless the reactor is below approximately 20% power (P-8).
The analysis in UFSAR Section 14.1.8 concludes an immediate RT on TT is not required for reactor protection. A RT on TT is provided to anticipate probable plant transients and to avoid the resulting thermal transient. If the reactor is not tripped by a TT, the over temperature delta temperature (OTDT) or over pressure delta temperature (OPDT) trip would prevent safety limits from being exceeded. The generator is protected by the generator protection system (GPS) which is designed to protect the generator from internal and external faults by tripping the output breakers. During this event the GPS functioned as designed and initiated a TT. This event was bounded by the analyzed event described in FSAR Section 14.1.8, Loss of External Electrical Load. The response of the plant is evaluated for a complete loss of steam load from full power without a direct RT and includes the acceptability of a loss of steam load without direct RT on turbine trip below 35 percent power. The analysis shows that the plant design is such that there would be no challenge to the integrity of the reactor coolant system or main steam system and no core safety limit would be violated. A low SG water level initiates actuation of the AFWS whose design has adequate capability to provide the minimum required flow. For this event, rod control was in Auto and all rods inserted upon initiation of the automatic RT. The AFWS actuated and provided required FW flow to the SGs. The reactor coolant system pressure remained below the set point for pressurizer PORV and code safety valve operation, and above the set point for automatic safety injection actuation. Following the RT the plant was stabilized in hot standby.