05000247/LER-2004-001

LER-2004-001, Manual Reactor Trip Due to Oscillating Feedwater Flow and Steam Generator Level with Flow Perturbations Caused by a Degraded Feed Water Regulating Valve

Docket Number
Event date:
Report date:
Reporting criterion:	10 CFR 50.73(a)(2)(iv), System Actuation 10 CFR 50.73(a)(2)(iv)(B), System Actuation 10 CFR 50.73(a)(2)(iv)(A), System Actuation
Initial Reporting
2472004001R00 - NRC Website

Note: The Energy Industry Identification System Codes are identified within brackets }

DESCRIPTION OF EVENT

On September 1, 2004, at approximately 0005 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />, while at 100k steady state reactor power, Operations manually tripped {JC} the reactor {RCT} as a result of oscillating Feedwater (FW) {SJ} flow and 22 Steam Generator (SG) {SB} level with flow perturbations and FW pipe {P} movement in the Auxiliary FW (AFW) Pump Building {NF}. Prior to the transient, on August 31, at 2348 hours0.0272 days <br />0.652 hours <br />0.00388 weeks <br />8.93414e-4 months <br />, while operating at 100k reactor power, with SG level control {JB} in AUTO, 22 SG narrow range (NR) level records show two cycles of level changes of approximately 2% and correction in automatic between 2348 hours0.0272 days <br />0.652 hours <br />0.00388 weeks <br />8.93414e-4 months <br /> and 2356 hours0.0273 days <br />0.654 hours <br />0.0039 weeks <br />8.96458e-4 months <br /> with no operator action. Subsequently, at 2356 hours0.0273 days <br />0.654 hours <br />0.0039 weeks <br />8.96458e-4 months <br />, operators observed 22 SG NR level starting to decrease from a normal value of 49% to 30% with a 5% deviation alarm annunciated at 44%. CR operators observed oscillating FW flow and erratic behavior of the 22 Main FW regulating valve FCV-427 {FCV}. At 0001 hours1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, Operators entered Abnormal Operating Procedure 2AOP-FW-1 and placed the FW regulating valve (FCV-427) in manual and attempted to increase FW flow in 22 SG without success. Excessive FW flow oscillations continued. Operators then opened low flow bypass valve FCV-427L to increase SG level which started 22 SG level increasing at a level of 30%. At approximately 35% SG level valve FCV- 427L was returned to closed. At approximately 0004 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />, a Nuclear Plant Operator (NPO) in the AFW Pump Building reported to the control room loud noises due to flow perturbations and pipe movement. Based on plant conditions, the Control Room Supervisor (CRS) directed a manual reactor trip (RT) {JC}. All control rods {AA} fully inserted and all primary systems functioned properly.

The 22 FW regulating valve FCV-427 failed to fully close. Operators initiated FW isolation by closing FW motor operated isolation valves (MOV) BFD-5-1 {ISV} and BFD-90-1 (ISV). At 0021 hours2.430556e-4 days <br />0.00583 hours <br />3.472222e-5 weeks <br />7.9905e-6 months <br />, a 22 SG high level trip {JB} was actuated at 73% SG level, initiating automatic closure of the Main FW Pump motor operated discharge valves (BFD-2-21 and BFD-2-22), Main FW and Low Flow FW regulating and isolation valves, and trip of the turbine driven Main FW Pumps. The plant was stabilized in hot standby with decay heat being removed by the main condenser {SG}. Off site power remained available and therefore the emergency diesel generators {ER} did not start. The AFW System {BA} automatically started as a result of a SG low level normally experienced on trips from full power. FW regulating valve FCV-427 is a Copes-Vulcan {C635} globe valve {V} with Copes- Vulcan actuator Model D-1000-160. The valve has a positioner to perform its modulating function and 3 solenoids {SOL} attached to the actuator for fast closure.

CR operators observed the rod bottom lights, Reactor Trip (RT) First Out Annunciator (Manual Trip). The plant was stabilized in hot standby with decay heat being released to the main condenser via the steam dump valves {V}.

At 0746 hours0.00863 days <br />0.207 hours <br />0.00123 weeks <br />2.83853e-4 months <br />, an 8-hour non-emergency notification was made to the NRC for an AFW actuation and a reactor trip (RT) (Incident Log No. 42003) under 10CFR50.72(b)(3)(iv)(A). Subsequently, at 1120 hours0.013 days <br />0.311 hours <br />0.00185 weeks <br />4.2616e-4 months <br />, a corrected notification was made for a four hour non=emergency notification for a Reactor Protection System (RPS) actuation in accordance with 10CFR50.72(b)(2)(iv)(B) with a note that the 4-hour report was made late. Operations' recorded the RT event in the corrective action program (CAP) as Condition Report CR-IP2-2004-04043. A post transient evaluation was performed on September 1, 2004.

A non-intrusive inspection was performed of the remaining FW regulating valves (FCV-417, FCV-437, FCV-447) to verify that their valve cage had not unthreaded from the valve body web. The verification was done by obtaining the maximum stroke capability of the FCV and relating that to a point at which the valve stem is connected into the actuator yoke (Measurements of the FCVs exposed stem threads and actuator posts were,compared to the available actuator travel).

These measurements provided reasonable. assurance that the remaining FCV cages were properly threaded into their body webs. Following plant shutdown a walk down was performed of the four (4) FW lines inside containment {NH} and FW and AFW piping outside containment for any impacts of the FW flow perturbations.

There were no indications of excessive movement or damage to the insulation, supports or piping above the 95 foot elevation of containment nor was there any observed signs of excessive movements, support damage, support impacts/scarring, or insulation damage on FW lines to SG-21, SG-22, SG-23, SG-24 on any containment elevations. For FW and AFW piping outside containment, no piping or support damage was evident due to pipe movements from the flow perturbations.

FW piping inside and outside containment showed some light powder insulation dust on the floor indicative of pipe vibration.

CAUSE OF EVENT

The cause of the manual RT was oscillating FW flow and 22 SG level from an erratic FCV-427 with piping pulsations and,FW pipe movement. The cause of the oscillating FW flow and 22 SG level was a faulty FW regulating valve FCV-427. The flow perturbations and pipe movement in downstream piping to the 22 SG was a result of FW flow transients. The cause of the faulty FW regulating valve FCV-427 was a disengaged valve cage from the valve body web. The valve cage had loosened over time to the point where the cage had disengaged from the valve body allowing the cage to be free floating within the valve body and susceptible to movement about the valve plug and body. The cause of the valve cage loosening was improper installation in 1997 due to inadequate guidance in the maintenance procedure (A0V-B-012-A) used to verify that the cage was fully seated and properly torqued into the valve body web. The procedure (AOV-B-012-A) did not include a requirement to inspect or verify the cage was threaded into the valve body until the mating surfaces were metal to metal prior to using the HYTORC machine to apply the final pass at the specified torque value. The procedure only provided a torque value for the cage installation.

Contributing causes included the following: 1) Mechanical valve failure from the phenomenon of relaxation of a threaded fastener over time as a result of thermal cycles and normal forces imposed by system flow that assist in loosening the cage.

2) Lack of a questioning attitude by the supervisors and maintenance crew during the installation of the valve cage to question if the cage was properly torqued to its intended value. 3) Procedural weakness of scheduled preventive maintenance (PM) on FCV-427 in January 1999, to identify that the cage was not torqued into the valve body properly because the PM did not include a step for verifying that the torque was adequate. 4) Lack of adequate procedural guidance on the use of the HYTORC machine and inadequate training on inherent error traps that are associated with its use contributed to the improper use of the HYTORC machine. 5) Process weakness identified after review of M&TE. A calibrated HYTORC head that was used to torque FCV-427 in 1997 was retired within one calibration cycle after use based on the vendor's recommendation. There was no calibration data at the end of the calibration period since the equipment was retired. The normal process would be to perform an evaluation on everything that the equipment was used on in the past calibration cycle but no evaluation could be found.

The cause of FW pipe vibration and movement was due to flow perturbations as a result of FW hydraulic/harmonic resonance from the change in stroke length of the valve actuator and actuator spring constant and a change in FW flow pattern when the low flow pathway was opened and closed.

CORRECTIVE ACTIONS

The following corrective actions have been or will be performed under the CAP to address the causes of this event and prevent recurrence.

1. Performed troubleshooting, repair in accordance with revised guidance, testing of FW regulating valve FCV-427 and returned the valve to service. FCV-427 was disassembled and the valve cage/trim assembly replaced. During installation, the new cage was verified to be fully seated into the valve body web prior to torquing.

2. Revised procedure AOV-B-012-A to include steps to verify that the cage is fully engaged and torqued into the valve body. These steps as a minimum consist of manual torquing until metal to metal is achieved, a visual inspection of the seating surfaces, and a stack up measurement from the top of the cage to compare to the expected value. Also incorporated is a step to verify the cage torque whenever internal work is performed on the valve.

3. The need for a periodic PM to retorque the valve cages will be determined based on findings during the IP-2 Refueling Outage (October 22-November 19).

4. A meeting was conducted with the maintenance staff to review the event and reinforce management's expectations on a questioning attitude.

5. An assessment will be performed to determine the need for a procedure for using a HYTORC machine and to revise the associated training lesson plans to include common error traps and determine the need for any additional training or qualification on the use of a HYTORC machine. The assessment is scheduled for completion by December 31, 2004.

6. An evaluation will be performed to identify any equipment affected by HYTORC machine TW-039-91 used to torque FCV-427 in 1997 and a justification prepared or corrective action performed on the identified equipment. The evaluation is scheduled to be completed by December 31, 2004.

EVENT ANALYSIS

The event is reportable under 10CFR50.73(a)(2)(iv)(A). The licensee shall report any event or condition that resulted in manual or automatic actuation of any of the systems listed under 10CFR50.73(a)(2)(iv)(B). Systems to which the requirements of 10CFR50.73(a)(2)(iv)(A) apply for this event include the reactor protection system (RPS) including reactor scram or reactor trip, and AFWS.

This event meets the reporting criteria'because the RPS was actuated by manual trip and the AFWS actuated on low level due to steam generator level changes in response to the manual RT, which occurs after a RT from full power as a result of SG shrink.

After the RT a high SG level (SGL) actuation occurred for the 22 SG due to failure of FCV-427 to fully close. The FW/SGL system supplies the High-High level signal to the RPS for initiation of main FW line isolation, main FW pump trip with FW pump discharge isolation valve closure, and Main Generator Trip (86P and 86BU Relays). A TT will result from a generator trip which initiates a RT via the RPS: However, the reactor and turbine were already tripped as a result of the manual trip.

PAST SIMILAR EVENTS

A review of the past two years of Licensee Event Reports (LERs) for events that involved a RT caused by FW flow transients identified no LERs.

SAFETY SIGNIFICANCE

This event had no effect on the health and safety of the public.

There were no actual safety consequences for the event because the event was an uncomplicated reactor trip with no other transients or accidents. Required primary safety systems performed as designed when the manual RT was initiated. The AFWS actuation was an expected reaction as a result of decreasing SG water level due to the reduction of SG void fraction (shrink), which occurs after automatic RT/TT from full load.

There were no significant potential safety consequences of this event under reasonable and credible alternative conditions. The malfunction of the FW flow control valve caused an initial loss of SG level (SGL) then a SG high level condition. This event was bounded by the analyzed event described in FSAR Section 14.1.9, Loss of Normal Feedwater. A Low-Low SGL in any one SG initiates actuation of two motor-driven AFW pumps and one steam driven AFW pump. One motor-driven AFW pump is sufficient to provide the minimum required flow.

After reactor shutdown, the 22 SG reached the SG High-High level set point (73%) and in accordance with plant design the proper actuation signals were initiated to isolate FW addition. Plant design is for FW/SGL to automatically isolate to preclude excessive RCS cooldown, containment overpressure, and SG overfill.

FW/SGL isolation is initiated by a Hi-Hi SGL signal or a safety injection signal.

A SG high-high level signal to the FW/SGL control system on two-out-of-three high SGL in any one of four SGs initiates FW isolation. The protection signals provide redundant isolation. Redundant FW isolation is accomplished by automatically closing all main and bypass FW control valves and closing the FW Pump discharge isolation valves. The closure of the FW Pump discharge isolation valves will automatically trip the FW Pumps and close the motor-operated isolation valves upstream of the FW control valves. The SG Hi-Hi level trip also initiates Main Generator trip (86P and 86BU relays)/TT. For this event the manual RT initiated a TT/Main Generator trip therefore the RT/TT actuation had already been completed when the SGL Hi-Hi level actuation occurred. This event was bounded by the analyzed event described in FSAR 'Section 14.1.10, Excessive heat removal due to a FW system malfunction. The plant performed as expected and the event was bounded by the FSAR analysis. For this event rod control was in automatic and the reactor scrammed immediately upon a manual reactor trip. RCS pressure remained below the set point for pressurizer PORV or code safety valve operation and above the set point for automatic safety injection actuation.

� Following the RT, the plant was stabilized in hot standby.