05000296/LER-2009-003
| Browns Ferry Nuclear Plant | |
| Event date: | 03-22-2006 |
|---|---|
| Report date: | 07-29-2011 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications |
| 2962009003R03 - NRC Website | |
I. PLANT CONDITION(S)
On March 25, 2010, Tennessee Valley Authority (TVA) determined that the Reactor Core Isolation Cooling [BN] (RCIC) system had been previously inoperable. The inoperability event date was March 22, 2006. On March 22, 2006, Unit 1 was shutdown, Unit 2 was at 100 percent power, and Unit 3 was in Mode 2, commencing restart activities.
II. DESCRIPTION OF EVENT
A. Event
On March 14, 2006, during Unit 3 refueling outage 12, Browns Ferry Nuclear Plant (BFN) installed a replacement Electric Governor-Remote (EG-R) on the RCIC system as a scheduled preventative maintenance activity. Post-maintenance surveillance testing was satisfactorily completed using Condensate Storage Tank (CST) to CST test mode. On March 22, 2006, Unit 3 exceeded 150 psig while in Mode 2 commencing restart operations.
On February 9, 2007, Unit 3 received an automatic reactor scram from 100 percent power following a loss of condensate flow. RCIC auto-initiated and injected into the reactor vessel in response to the low water level resulting from the loss of condensate flow. On February 12, 2007, Operations personnel commenced restart operations with Unit 3 entering Mode 2. Specific details on the reactor scram can be found in LER 50-296/2007-001, Reactor Scram due to Low Reactor Water Level Caused by Loss of Feedwater, submitted to the Nuclear Regulatory Commission (NRC) on April 10, 2007.
On February 13, 2007, a post-scram review of the RCIC operating parameters revealed the unexpected level of instability in the system flow and turbine control system response that was experienced on February 9, 2007. During the injection sequence, RCIC system flow oscillated between approximately 300 gallons per minute (gpm) and 900 gpm. However, because the RCIC system only operated approximately 2 minutes and automatically shut down when the Reactor Pressure Vessel (RPV) high water level was attained, the instability was not noted by the Operations crew; therefore, no review of the system response was conducted prior to the startup of Unit 3. On February 15, 2007, a functional evaluation concluded that the RCIC system was capable of performing its design function. As part of the troubleshooting for the observed oscillations, maintenance was performed on a control system wiring terminal lug.
However, the post-maintenance testing was conducted using the routine quarterly surveillance procedure which operated RCIC in a CST recirculation mode, rather than aligned for RPV injection. Since no RCIC oscillations were identified during the post-maintenance test, it was concluded that the flow oscillation problem had been corrected.
On March 18, 2008, Unit 3 entered refueling outage 13. During the refueling outage, the EG-R needle valve was adjusted and the turbine governor valve was replaced.
However, the post-maintenance testing was conducted using the routine quarterly surveillance procedure which operated RCIC in a CST recirculation mode, rather than aligned for RPV injection. Since no RCIC oscillations were identified during the post-maintenance test, it was concluded that the flow oscillation problem had been corrected. On May 15, 2008, following the completion of outage activities, Operations commenced restart activities for Unit 3 Cycle 14 operation, entering Mode 2.
On August 24, 2009, Unit 3 was manually scrammed from 100 percent power due to the lowering of the water level in the RPV. Following the manual reactor scram, RCIC auto initiated and injected into the reactor vessel. On August 26, 2009, Operations personnel commenced startup operations with Unit 3 entering Mode 2. Unit 3 was returned to service on August 28, 2009, and remained at power until September 12, 2009. Specific details of the Unit 3 manual reactor scram can be found in LER 50-296/2009-001, Reactor Scram Due to Loss of Condensate Booster Pumps, submitted to the NRC on October 23, 2009.
On August 26, 2009, as part of a post-scram review and prior to restart, site engineering personnel again identified an unexpected level of instability in the RCIC system flow and turbine response. During the injection sequence, the RCIC system flow oscillated between approximately 230 gpm and 970 gpm. A functional evaluation dated August 26, 2009, concluded that the RCIC system was capable of performing its design function and Operations determined the RCIC system was operable.
Following each event, BFN Engineering personnel evaluated the RCIC system response, concluded the RCIC system was capable of performing its design function and Operations determined that the RCIC system was operable.
EG-R needle valve adjustments were made immediately prior to the Unit 3 shutdown in September 2009. The subsequent testing was inconclusive due to the Amphenol disconnection that occurred during testing. On September 12, 2009, Unit 3 was removed from service for scheduled maintenance activities not associated with the RCIC system. During the September 2009 maintenance outage, the RCIC EG-R was replaced and successfully tested. A failure analysis of the removed EG-R, conducted by ESI, determined the oscillations were caused by a missing buffer piston and springs within the EG-R.
On March 25, 2010, in response to questions from the NRC, WA notified the NRC via a telephone conference call that the RCIC system was inoperable since Unit 3 exceeded 150 psig while in Mode 2 on March 22, 2006, based on reevaluation of the impact of the non-conforming EG-R. Technical Specification (TS) 3.5.3 requires that the RCIC pump develop a flow rate greater than or equal to 600 gpm against a system head corresponding to reactor pressure. Operability with respect to the applicable TS requirements could not be concluded as a result of the observed instability.
WA has determined the RCIC system was inoperable from March 22, 2006, to September 12, 2009. This time period represents Unit 3 entering Mode 2 after installation of the EG-R to Unit 3 shutting down entering Mode 4 replacing the defective EG-R. March 25, 2010, was the determination date for the past inoperability of the RCIC system.
WA is submitting this report in accordance with 10 CFR 50.73(a)(2)(i)(B), as an operation or condition prohibited by the plant's Technical Specifications.
B. Inoperable Structures, Components, or Systems that Contributed to the Event None C. Dates and Approximate Times of Major Occurrences March 14, 2006 March 22, 2006 February 9, 2007 February 12, 2007 February 13, 2007 March 18 thru May 15, 2008 August 24, 2009 August 26, 2009 August 26, 2009 September 12, 2009 September 14, 2009 September 21, 2009 March 25, 2010 BFN installs replacement EG-R and it is successfully tested.
Unit 3 enters Mode 2, commencing restart activities.
Unit 3 received an automatic reactor scram. RCIC pump starts and injects into the reactor vessel on low water level.
Unit 3 enters Mode 2, commencing restart activities.
BFN personnel noted an unexpected level of instability during RPV injection on February 9, 2007.
BFN conducts Unit 3 Refueling Outage 13.
Unit 3 Operations personnel insert a manual scram on Unit 3. RCIC pump starts and injects into the reactor vessel on low water level.
BFN personnel noted an unexpected level of instability during RPV RCIC injection on August 24, 2009.
BFN Operations personnel commenced restart activities on Unit 3 by placing the mode switch in Startup position.
Unit 3 shut down for scheduled maintenance activities.
RCIC EG-R replaced.
RCIC system successfully tested using RPV injection.
WA informs NRC that RCIC was inoperable longer than allowed by TS.
D. Other Systems or Secondary Functions Affected
None
E. Method of Discovery
BFN personnel noted the instability in RCIC system operation during post-scram reviews of the RCIC system operating parameters.
F. Operator Actions
None
G. Safety System Responses
None
III. CAUSE OF THE EVENT
A. Immediate Cause
The immediate cause for the inoperable RCIC pump was the EG-R actuator non conformance and the resulting reduced stability of the RCIC governor control system during RPV injection. The EG-R was absent critical parts that would keep the RCIC pump from oscillating during RPV injection.
B. Root Cause
A failure analysis, performed by Engine Systems Incorporated (ESI), determined the oscillations observed during RPV injection were caused by a missing buffer piston and springs within the EG-R. However, the missing parts did not affect stable operation during the periodic surveillance testing, and therefore, inoperability was not detectable by routine surveillance testing of the RCIC system. Therefore, inoperability of the RCIC system was caused by omission of critical parts in the EG-R actuator during original manufacturing or during vendor repairs. Based on the failure analysis performed by ESI, this was considered an isolated occurrence.
C. Contributing Factors
None
IV. ANALYSIS OF THE EVENT
On February 9, 2007, and again on August 24, 2009, following the Unit 3 reactor scram, the RCIC system, along with the High Pressure Coolant Injection [BJ] (HPCI) system, auto-initiated and injected into the RPV restoring water level. Both the HPCI and the RCIC systems auto-stopped as expected on high RPV water level.
Subsequent review of the RCIC System operating flow parameters for both scrams revealed an unexpected level of instability in the RCIC system flow and turbine control system response. In both cases, the instability was not noted by the BFN Operations personnel in the main control room due to the short time the system operated (approximately 2.0 and 2.5 minutes respectively). With regard to the oscillations that occurred on February 9, 2007, a review of RCIC system operation was not conducted prior to Unit 3 restart. The troubleshooting activities noted in the description of the February 2007 event above were ineffective as corrective actions because they did not address the root cause, i.e., the missing buffer piston and springs within the EG-R. The post-maintenance test method used did not inject into the RPV, and therefore did not replicate actual system pressure conditions which induced the oscillations allowed by the missing EG-R components during the February 2007 RCIC injection. Thus, the corrective actions taken in 2007 did not prevent recurrence of flow oscillations.
The following discussion is specific to the August 24, 2009, event; however, the data is consistent with data from the event that occurred on February 9, 2007. During the injection event on August 24, 2009, flow data obtained from a high resolution source (100 samples per second from the plant Integrated Computer System [ID] (ICS)) indicated RCIC pump output flow was oscillating between 230 gpm and 970 gpm. A least-squares fit analysis of this event indicated that the RCIC system was providing an average flow rate of approximately 620 gpm.
The highest recorded speed of the turbine was 4610 RPM which is well below the over speed setpoint of 5625 RPM. Therefore, while the turbine speed was oscillating, the turbine did not approach the over-speed setpoint.
Another flow rate estimate was performed using a flow totalization method. The evaluation used high speed data (Dataware Program) to estimate the total injection during the 2 minute 29 second time period. The total volume obtained was 1573 gallons, which corresponded to 630 gpm during the injection period. A similar flow totalization estimate was performed using high resolution ICS data. This estimate calculated an average flow rate of approximately 623 gpm during the injection period.
Normal RCIC system flow testing is performed taking suction from the condensate header and discharging back to the CST. During the RCIC system testing activities, perturbations are introduced into the control system by operating the system with the flow controller in the manual mode and then placing the controller in the automatic mode with a flow setpoint different than the existing system flow rate. This method limits the severity of the perturbation. Additionally, due to the hydraulic difference between the CST to CST mode of operation and injection into a pressurized RPV, the instability on the Unit 3 governor control system during RPV injection was not detected until the RPV injection occurred.
V. ASSESSMENT OF SAFETY CONSEQUENCES
The safety consequences of this event were not significant.
The applicability statement for BFN TS Limiting Condition for Operation 3.5.3 requires the RCIC system be operable when the reactor is in Mode 1 and in Modes 2 and 3 with the reactor dome pressure greater than 150 psig. TS 3.5.3 Condition A and Required Actions A.1 and A.2 require immediate verification by administrative means that the HPCI system is operable and restoration of the RCIC system to operable status in 14 days. These Required Actions were not met. The extended period of RCIC system inoperability without verification of HPCI system operability constituted a condition prohibited by TS 3.5.3; had the condition been known, a unit shutdown would have been required by the TS. Mode changes prohibited by TS 3.0.4 were also made. During the time that the RCIC system was inoperable, the HPCI system was inoperable for short periods at three different occasions which are discussed below.
- On November 30, 2007, between approximately 1052 and 1435 hours0.0166 days <br />0.399 hours <br />0.00237 weeks <br />5.460175e-4 months <br /> Central Standard Time, during a controlled reactor shutdown, Unit 3 HPCI system was declared inoperable when a steam leak on a HPCI system condensate inboard drain valve increased. Details on the HPCI system inoperability can be found in a Steam Leak, submitted to the NRC on January 28, 2008.
- On July 24, 2007, at 1645 hours0.019 days <br />0.457 hours <br />0.00272 weeks <br />6.259225e-4 months <br /> Central Daylight Time (CDT), the Unit 3 HPCI system was declared inoperable when the Division II Emergency Core Cooling Systems [AD] Analog Trip Unit Inverter [EJ] failed due to a cleared fuse. On July 25, 2007, at approximately 0105 hours0.00122 days <br />0.0292 hours <br />1.736111e-4 weeks <br />3.99525e-5 months <br /> CDT, the HPCI system was declared operable. Details on the HPCI system inoperability can be found in LER 50-296/2007-002, Unplanned Inoperability of the Unit 3 High Pressure Coolant Injection System Due to Loss of 120 V-AC Instrument Power, submitted to the NRC on September 24, 2007.
- There was approximately 6.58 hours6.712963e-4 days <br />0.0161 hours <br />9.589947e-5 weeks <br />2.2069e-5 months <br /> of Maintenance Rule unplanned unavailability for the HPCI system during the period from March of 2006 thru September of 2009. Additional planned unavailability of the HPCI system occurred during the performance of surveillance tests and other maintenance activities. However, these instances would typically be less than a shift in duration.
To be considered operable in accordance with the applicable TS requirements, the RCIC system is assumed to deliver a minimum of 600 gpm to the RPV. Although the RCIC system was inoperable, during the period discussed in this LER, it was functional. That is, the RCIC system was capable of starting and injecting into the RPV delivering an average flow rate greater than or equal to 600 gpm to the RPV. During the RPV injection on February 9, 2007, and again on August 24, 2009, the RCIC system along with the HPCI system injected for approximately 2.0 and 2.5 minutes and injected an average of approximately 620 gpm for the period. For long term operation such as maintaining water level with the RPV isolated, Operating Instruction 3-01-71, "Reactor Core Isolation Cooling System," provides instructions for operating the RCIC system in a manual mode upon malfunction of the flow controller. Therefore, TVA concludes that there was no significant reduction in the health and safety of the public by this event.
VI. CORRECTIVE ACTIONS
A. Immediate Corrective Actions
On September 14, 2009, BFN replaced the Unit 3 RCIC system EG-R. Following the replacement of the Unit 3 RCIC system EG-R, a RPV injection test was conducted on September 21, 2009. The EG-R exhibited stable RCIC turbine speed and flow during the RPV injection. The EG-R that was in place during the period was sent to the vendor for failure analysis and refurbishment.
B. Corrective Actions
Safety related systems were reviewed for their flow setpoints specified in their respective Nuclear Engineering Setpoint and Scaling Documents relative to the TS limits. HPCI and RCIC were the only systems that were revised which had TS limits that coincided with their safety related setpoint.
C. Corrective Actions to Prevent Recurrence
The current vendor, ESI (used by TVA for dedication of the EG-R), is not the vendor that dedicated the failed EG-R for use at BFN. TVA relies on ESI to provide a fully dedicated EG-R for use at BFN. Implementation of the vendor's Appendix B Quality Assurance Program is expected to provide TVA with a fully dedicated EG-R and prevent the recurrence of this event. All new governors and/or actuators are subject to retesting while at ESI using the same test specifications as the manufacturer.
VII. ADDITIONAL INFORMATION
A. Failed Components
The failed component was the EG-R. The EG-R, serial number 12047729, was sold new by Woodward Governor Company to Dresser-Rand in March 1998.
Dresser-Rand dedicated the EG-R for use on Unit 3. Woodward Governor Company records show that it was returned in April 1998 for warranty by Dresser-Rand.
Woodward returned the EG-R to Dresser-Rand after correcting a warranty issue.
There have not been any other EG-R returns to Woodward Governor Company or ESI with this serial number.
B. Previous LERs or Similar Events A previous similar event occurred on the Standby Gas Treatment [BH] (SGT) Train.
An equipment issue associated with one of the three SGT Train's relative humidity heater power loss alarms was misdiagnosed by Operations, Maintenance, and Systems Engineering. Operations failed to initiate a Problem Evaluation Report (PER) when the problem was first identified and did not pursue timely and accurate evaluation and correction of this equipment issue. As a result, SGT Train A was declared inoperable approximately 5 months later.
C. Additional Information
Corrective action documents associated with this event are PERs 119628, 200183, 224614, 232668, and 246526. PER 246527 was generated to address the incomplete and inaccurate information provided in Revision 0 to this LER.
PERs 304722 and 329704 were generated to address the incomplete and inaccurate information provided in Revision 2 to this LER.
D. Safety System Functional Failure Consideration
This event is not classified as a safety system functional failure according to NEI 99-02.
E. Scram With Complications Consideration
This LER does not describe a complicated scram according to NEI 99-02.
VIII. COMMITMENTS
None