05000483/LER-2008-006

LER-2008-006, Reactor Trip Due To Turbine Trip During Power Reduction Following Loss Of 'C' Condensate Pump

Docket Numbersequential Revmonth Day Year Year Month Day Yearnumber No.
Event date:
Report date:
Reporting criterion:	10 CFR 50.73(a)(2)(iv)(B), System Actuation 10 CFR 50.73(a)(2)(iv)(A), System Actuation
4832008006R01 - NRC Website

I.�DESCRIPTION OF THE REPORTABLE EVENT

A. REPORTABLE EVENT CLASSIFICATION

10CFR50.73(a)(2)(iv)(A) requires reporting of any event or condition that resulted in a manual or automatic actuation of any of the systems listed in 10CFR50.73(a)(2)(iv)(B). The systems listed below are relevant to this LER:

(1) Reactor protection system (RPS) including: reactor scram or reactor trip; and (6) PWR auxiliary or emergency feedwater system.

B. PLANT OPERATING CONDITIONS PRIOR TO THE EVENT

The plant was in MODE 1, Power Operation, at 100% power.

START OF THE EVENT AND THAT CONTRIBUTED TO THE EVENT

No structures, systems, or components were inoperable at the start of the event which contributed to the event.

D. NARRATIVE SUMMARY OF THE EVENT, INCLUDING DATES AND APPROXIMATE TIMES

At 2254 on December 11, 2008 the 'C' condensate pump [EIIS system: SD, component: P] tripped due to a motor [EIIS system: SD, component: MO] ground fault failure (turn-to-turn failure on the stator). At 2255 the operating crew noticed computer point MAJ0001, Main Generator Power, had turned yellow and was indicating approximately 1314 MWe. The operators immediately noticed that the 'C' condensate pump had tripped, commenced a reduction in power, and entered Callaway off-normal procedure OTO-AE-00001, Feedwater System Malfunction. However, the manual load reduction rate was approximately 31% in one minute, exceeding the procedural guidance of 5% per minute. At 2256 the control rods [EIIS system: AA, component: ROD] automatically stepped in and the condenser steam dumps [EIIS system: SB, component: RV] opened as designed. Level transients subsequently occurred in the steam generators (S/Gs) [EIIS system: AB, component: SG], and level deviation annunciators actuated in the main control room for all four S/Gs. Meanwhile, the operators perceived wind-up, or slow response, in the controllers for the main feedwater regulating valves (MFRVs) [EIIS system: SJ, component: FCV] and therefore took the controllers from automatic to manual and then back to automatic in an attempt to reset the wind-up condition. As a result of high level in the 'B' S/G, a turbine [EllS system: TA, component: TRB] trip/reactor trip occurred at 2300.

The turbine tripped on a Hi-Hi S/G level (P-14) feedwater isolation signal (FWIS) [EIIS system: SJ, component: ISV] from the 'B' S/G, and the reactor trip immediately followed. Additionally, as a result of the P-14 FWIS, both main feedwater pumps (MFPs) [EIIS system: SJ, component: P] tripped and hence a motor-driven auxiliary feedwater actuation [EMS system: BA, component: P] occurred as expected. The Auxiliary Building sample station Chemical and Volume Control System (CVCS) Letdown Gamma Detector [EIIS system: IL, component: DET] went into Alert as a result of the transient (due to a pre-existing fuel defect) and a Loose Parts Monitor [EIIS system: II, component: MON] alarmed, as expected, due to the control rods dropping. The 'B' and 'C' atmospheric steam dumps (ASDs) [EIIS system: SB, component: RV] opened and re-closed at the time of the reactor trip.

Two root cause investigations were performed to determine causes and corrective actions for the pump motor failure and for the automatic actuations of the Main Feedwater Isolation, RPS & Auxiliary Feedwater System (AFWS). The pump motor failure which initiated the sequence of events was the result of one root cause associated with two causal factors. The investigation for the automatic actuations of the Main Feedwater Isolation, RPS & Auxiliary Feedwater System (AFWS) found that the event occurred due to four root causes with associated causal factors, as described in Section III below.

ERROR

The 'C' condensate pump trip was discovered by the Reactor Operator (RO) while the operators were investigating the cause of why the "MWe High" indication, i.e. MAJ0001, Main Generator Power, was yellow. No other components or systems failed during this event, and there were no errors in the procedural guidance. However, human error caused the Balance of Plant (BOP) operator's action to exceed procedural guidance for reducing power (load) subsequent to the condensate pump trip.

Subsequent to the December 11, 2008 event, the root cause team investigating the system actuations utilized an unrelated forced outage, from February 19, 2009 to March 3, 2009, to conduct dynamic testing of the steam generator level control circuits [EIIS system: SJ, component: LC]. Through this testing it was determined that the gain settings of the four MFRV flow controllers [EIIS system: SJ, component: TC] were at a value of 0.5 rather than 1.0 as analyzed and shown in plant design documentation. These "field-tuned" gain settings were the primary cause of the slow MFRV response (wind-up) observed by the plant operators during the December 11, 2008 event.

A. SAFETY SYSTEMS THAT RESPONDED

All safety systems functioned as designed. The Main Feedwater Isolation actuation (i.e. a FWIS), RPS actuation, and motor-driven Auxiliary Feedwater actuation all occurred as expected for such an event.

B. DURATION OF SAFETY SYSTEM INOPERABILITY

No safety systems were inoperable which contributed to this event.

C. SAFETY CONSEQUENCES AND IMPLICATIONS OF THE EVENT.

At no time was the plant in an unanalyzed condition. When the 'C' condensate pump tripped, both the 'A' and 'B' condensate pumps were available and capable of providing adequate flow and suction head for the feedwater pumps (notwithstanding the load reduction that was effected).

Had only one condensate pump remained running at greater than 45-percent reactor power, the reactor would have been manually tripped.

The field-tuned gain settings of the four MFRV flow controllers had an adverse impact on the ability of the plant to respond to the load reduction without a reactor trip. However, this did not impact safety functions credited in the Callaway accident analysis. As discussed in the Bases for Callaway Technical Specification 3.7.3, the specified function for the MFRVs is to close within a specified time to mitigate main steam line break or feedline break events. This is a diverse backup to the Main Feedwater Isolation Valves [EIIS system: SJ, component: ISV]. The field-tuned gain settings had no adverse impact on the specified safety function of the MFRVs. Therefore, the value of the gain settings did not represent a significant degradation of nuclear safety.

The events described in this LER can be best characterized by comparison with the Loss of Load heatup event analyzed in the Callaway Final Safety Analysis Report (FSAR).

The plant conditions described in this LER were bounded by the Current Licensing Basis accident analysis for Loss of Load transient. The Loss of Load transient is less severe than a Turbine Trip event which is also analyzed in the Callaway FSAR, so the Turbine Trip accident analysis bounds the LER event. The Turbine Trip transient is evaluated with the following conservative conditions:

1. The power is reduced from full power to zero power, and the reactor is in manual control.

2. There is no credit taken for operation of the steam dump system or the atmospheric steam dumps.

3. The reactor does not trip until RCS conditions result in a trip.

The event described in this LER occurred with a total load loss of approximately 36 percent, from full power. The reactor was in automatic control, allowing the control rod banks to step in before the trip occurred. The atmospheric steam dumps did function as designed, which caused the steam generator levels to rise to the P-14 level, resulting in a reactor trip to coincide with the turbine trip. The reactor trip coinciding with turbine trip is a result of the P-9 automatic protective function.

The events described in this LER are bounded by the Current Licensing Basis analysis in the Callaway FSAR. Therefore, the events described in this LER do not represent any degraded or unanalyzed conditions, nor were they safety significant.

III.�CAUSE(S) OF THE EVENT AND CORRECTIVE ACTION(S) This event was evaluated in two parts, each using a seven-step root cause analysis process. The first Root Cause Team, analyzing the pump motor failure, concluded that there was a single Root Cause with two Causal Factors. The second Root Cause Team, analyzing the human performance and plant response during the event, concluded that there were four Root Causes with associated Causal Factors.

The 'C' condensate pump motor failure: The condensate pump motors are Allis Chalmers VersaPac three-phase wye-wound vertical induction motors rated at 3500 hp, 1184 rpm, and 4000 volts.

Destructive examination performed by the vendor identified a causal factor for the failure as a winding failure in the 'C' condensate pump motor stator, along with the root cause of a voltage surge causing damage to the motor insulation. As described in Callaway LER 2008-008, following the 'C' condensate pump motor failure, the 'B' condensate pump motor failed on December 14, 2008, also due to a motor ground fault failure.

Voltage transients occur in most electrical systems and are usually caused by switching surges from breaker contactor switching. These surges often originate from normal switching actions within the plant and are less likely to occur from lightning strikes. Motors that run continuously have a higher probability of experiencing voltage surges caused by any type of electrical disturbance on the bus.

Fast rise time surges can cause severe voltage stress on the turn insulation due to the non-uniform distribution of voltage across the winding. Typically, the voltage becomes uniformly distributed across the turns for rise times that are longer than approximately 1 microsecond. Some equipment characteristics that may influence these effects include cable length, cable size, insulation condition, common bus electrical loads, and switching sequence. As such, it was identified that an additional causal factor is the close proximity (i.e., cable length —150 feet) from the supplying electrical bus to the motors, which makes them more susceptible to voltage surges than other motors on the same busses.

During the extent of condition investigation of the similar failures for the 'C' and 'B' condensate pump motors, the 'A' condensate pump motor was sent offsite to a vendor for testing. The 'A' condensate pump motor was determined to not have the same degradation as the 'C' and 'B' condensate pump motors.

This issue has been entered into Callaway's corrective action program. The Corrective Action to Prevent Recurrence (CATPR) was to have surge capacitors installed at the condensate pump motor lead boxes to minimize the slope of the voltage transient wave front. By elongating the rise time of the surges, less of the full surge voltage will appear across any one coil or any two turns in the winding, thereby providing protection to the motor from future voltage surges. This will maintain the winding integrity and prevent a turn-to-turn fault. This action is complete for all of the condensate pump motors.

The human performance and plant response: There were four root causes of the automatic actuations of the Main Feedwater Isolation, RPS and AFWS. When faced with quickly changing conditions, the operators did not clearly communicate by stating the objective and desired outcome to ensure mutual understanding and minimize mistakes. This led to taking actions outside of existing procedural guidance.

The procedural guidance for loss of a condensate pump is accurate and provides clear guidance on the actions to take. Furthermore, the training that operators received on a condensate pump trip demonstrated that a very small, if any, power reduction is required to ensure plant equipment is not damaged.

Additionally, the Root Cause Team investigated the Steam Generator Water Level Control (SGWLC) System operation during the event. The team found, through dynamic testing of the steam generator level control circuits, that the gain settings of the four MFRV flow controllers were at a value of 0.5 rather than 1.0 as analyzed and shown in plant design documentation. Historical reviews identified that the gain settings of these four controllers were modified, i.e. field-tuned, in 1999 following Refuel 10 (RF10) when SGWLC system performance concerns were identified during power ascension. The changes in system performance were the result of a modification of the MFRVs under Callaway modification package 98-2005 during RF10.

With the reduced gain setting of 0.5 the SGWLC system was not capable of automatically controlling level below the P-14 (FWIS) actuation setpoint during this event. As a result, operator intervention was required to maintain steam generator levels below this Engineered Safety Feature Actuation System (ESFAS) actuation setpoint. However, the action which was initially taken to reset the controller "wind­ up" by taking the controllers from automatic, to manual, and then back to automatic without the system being stabilized at the system setpoint (i.e., at program level with no feed flow/steam flow mismatch) had the impact of further slowing the response of this control circuit based upon the bumpless transfer feature utilized within these controllers. Thus, the initial attempts to speed the controllers' response during this event only further slowed the response of this already dampened (due to the field-tuned gain setting) control circuit.

Also, from the historical reviews, it was determined that no changes to the plant design documentation were generated to document the field-tuning that occurred. The current configuration control process allows this parameter to be changed without revising plant design documentation. Moreover, it appears that this field-tuning only took into account the impact of the adjustments on steady-state system operations and did not take into account any potential impact of these adjustments on plant transient analysis and response.

The four causal factors associated with root causes were identified as follows:

(1) The rate and magnitude of power reduction was not in compliance with procedural requirements.

(2) Operators have a knowledge deficiency regarding rapid load reductions and how to achieve them.

(3) Operating crew dynamics and teamwork (including the use of Event Prevention Tools) was inadequate to prevent this event.

(4) The SGWLC System did not operate correctly to restore SG levels following the power transient.

The corresponding root causes were:

(1) Clear expectations and standards have not been established regarding operator directed actions necessary to stabilize the plant versus procedurally directed actions. There is a conflict between the desire to implement verbatim compliance with written instructions and the authority and responsibility of a licensed operator to take actions necessary to stabilize the plant and protect the health and safety of the public.

(2) Lack of guidance on operation of the load limit potentiometer, combined with lack of experience performing rapid power reductions using the load limit potentiometer, caused load reduction in excess of procedure limits.

(3) Operational events have not been prevented due to gaps to excellence in the Operations Department Code of Conduct and re-enforcement of the current Code of Conduct.

(4) Gain settings for the SGWLC System flow controllers were field-tuned away from the analyzed value due to plant operating concerns. The field-tuning was performed in accordance with the current configuration control process which allows this parameter to be changed without revising plant design documentation.

This issue has been entered into Callaway's corrective action program, and the CATPRs which are in progress are as follows:

(1) CATPR: Establish clear expectations and standards regarding operator directed actions necessary to stabilize the plant versus procedurally directed actions.

(2) CATPR: Create guidance on operation of load limit potentiometer and reinforce skills in training.

(3) Robust Corrective Action (RCA) 3.1: Complete a performance analysis worksheet (PAW) and initiate a training request (TRRQ) to further enhance training. (This RCA is complete.) RCA 3.2: Benchmark Operations Department Code of Conduct (standards) to identify industry best practices.

RCA 3.3: Upgrade the Callaway Operations Department Code of Conduct to include industry best practices.

RCA 3.4: Develop and execute a change management plan to implement changes to the Operations Department Code of Conduct. This plan will include communication, training, observation, and re-enforcement.

(4) CATPR 4.1: Restore the gain settings to a value that is documented in the Plant Configuration Control Documentation and is supported by analysis. (This CATPR is complete.) CATPR 4.2: Revise Administrative Controls to prevent field-tuning parameters that are included as inputs in the plant transient analysis.

IV.�PREVIOUS SIMILAR EVENTS With respect to the 'C' condensate pump motor failure and field-tuning that affected an analyzed input parameter, no previous similar events have been identified. As for the human performance error root causes:� .

• Callaway LER 2004-001-00 addressed an event that had a similar cause of "failure to properly follow an approved plant procedure;" however, the contributing cause for that event involved identical name plate labels for two bus transfer selector switches, which does not apply to this event. The resulting discussion of the importance of human performance factors and attention to detail could not have been expected to prevent this event four years later.

• Callaway LER 2008-003-00 addressed an event that had similar issues with the operations crew clearly communicating and crew teamwork, but the individual coaching performed would not have prevented this event. Additionally, a remedial action taken to discuss the importance of crew teamwork and incorporate details of the event into future licensed operator training is not necessarily a CATPR and may not have been fully implemented as the event in LER 2008-003-00 only took place two months prior to this event.

• One event in the Callaway corrective action program documents an Institute of Nuclear Power Operations (INPO) Area for Improvement, from 2002, in which "Operations does not set and reinforce high standards in some activities. Specifically, Operations lags the industry in implementing tools to improve human performance." The CATPRs were to develop a 6-year plan for the performance of Self-Assessments and Benchmarking trips as well as conducting evaluations periodically with each crew to focus on maintaining and reinforcing high standards.

The 6-year plan has only gone through a single cycle and may not have had time to be effective.

V.� ADDITIONAL INFORMATION The system and component codes listed below are from the IEEE Standard 805-1984 and IEEE Standard 803A-1983, respectively.

System:� AA� Control Rod Drive System AB� Reactor Coolant System (PWR) BA� Auxiliary/Emergency Feedwater System (PWR) II� Loose Parts Monitoring System IL� Radiation Monitoring System SB� Main/Reheat Steam System (PWR) SD� Condensate System SJ� Feedwater System TA� Main Turbine System Component:� DET� Detector FCV� Valve, Control, Flow ISV� Valve, Isolation LC� Control, Level MO� Motor MON Monitor P� Pump ROD Rod RV� Valve, Relief SG� Generator, Steam TC� Control, Flow TRB� Turbine