05000498/LER-2004-001, Regarding Reactor Trip on High Steam Generator Levels Initiated by Failure of Inverter 1201
| ML040860766 | |
| Person / Time | |
|---|---|
| Site: | South Texas  |
| Issue date: | 03/23/2004 |
| From: | Halpin E South Texas |
| To: | Document Control Desk, Office of Nuclear Reactor Regulation |
| References | |
| NOC-AE-04001695 LER 04-001-00 | |
| Download: ML040860766 (8) | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(iv)(A), System Actuation |
| 4982004001R00 - NRC Website | |
text
Nuclear Operating Company kb ff~~w PM av PO.a us28 XM 1
raw774f3 VA March 23, 2004 NOC-AE-04001695 10CFR50.73 U. S. Nuclear Regulatory Commission Attention: Document Control Desk One White Flint North 11555 Rockville Pike Rockville, MD 20852 South Texas Project Unit 1 Docket No. STN 50-498 Licensee Event Report 1-04-001 Unit 1 Reactor Trip On High Steam Generator Levels Initiated by Failure of Inverter 1201 Pursuant to 10CFR50.73(a)(2)(iv)(A), the South Texas Project submits the attached Licensee Event Report 1-04-001 regarding the Unit I reactor trip on high steam generator levels initiated by the failure of Inverter 1201.
This event did not have an adverse effect on the health and safety of the public. There are no commitments contained In this event report.
Resulting corrective actions will be handled in accordance with the STP Corrective Action Program.
If there are any questions on this submittal, please contact S. M. Head at (361) 972-7136 or me at (361) 972-7849.
E. D. Halpin v/4 v
Plant General Manager JaV Attachment: LER 1-04-001 (South Texas, Unit 1)
LER 1-04-001 (c).doc STI: 31711 765
NOC-AE-04001 695 Page 2 of 2 cc:
(paper copy)
(electronic copy)
Bruce S. Mallett Regional Administrator, Region IV U. S. Nuclear Regulatory Commission 611 Ryan Plaza Drive, Suite 400 Arlington, Texas 76011-8064 U. S. Nuclear Regulatory Commission Attention: Document Control Desk One White Flint North 11555 Rockville Pike Rockville, MD 20852 Richard A. Ratliff Bureau of Radiation Control Texas Department of Health 1100 West 49t Street Austin, TX 78756-3189 Jeffrey Cruz U. S. Nuclear Regulatory Commission P. O. Box 289, Mail Code: MN1 16 Wadsworth, TX 77483 A. H. Gutterman, Esquire Morgan, Lewis & Bockius LLP L. D. Blaylock City Public Service David H. Jaffe U. S. Nuclear Regulatory Commission R. L. Balcom Texas Genco, LP A. Ramirez City of Austin C. A. Johnson AEP Texas Central Company Jon C. Wood Matthews & Branscomb C. M. Canady City of Austin Electric Utility Department 721 Barton Springs Road Austin, TX 78704 LER 1-04-001 (c).doc STI: 31711765
Abstract
On 01/23/04 with Unit 1 at 100% power, the Class 1 E 7.5 KV Inverter 1201 failed, resulting in a reactor trip when steam generator level could not be controlled. The plant was stabilized, and after troubleshooting and repair, the reactor was restarted at approximately 2237 hours0.0259 days <br />0.621 hours <br />0.0037 weeks <br />8.511785e-4 months <br /> on 01/25/04.
The cause of the inverter failure was a Ferro-Resonant transformer short due to heat-related degradation of the transformer insulation. The technical root cause of the event is that the design of the Class 1 E Vital Instrument Power Supply (VA) System is not single train fault-tolerant and failure of the inverter causes significant operational challenges. Additionally, the organizational root cause is that the station did not aggressively pursue effective failure prevention strategies even though previous events have shown that the inverters are susceptible to failure and the Instrument Power Supply (VA) System is not single train fault-tolerant.
Corrective actions include the replacement/repair of all Class 1 E 7.5 KV Inverter Ferro-Resonant Transformers and design enhancements that will minimize inverter failures and the effects of initiating transients requiring significant operator rapid response.
This event resulted in no personnel injuries, no offsite radiological releases, and no damage to safety-related equipment. There were no challenges to plant safety.
NRC FORM 366 (7-2001)
(if more space Is required, use additional copies of SUMMARY OF THE EVENT, INCLUDING DATES AND APPROXIMATE TIMES At approximately 1615 hours0.0187 days <br />0.449 hours <br />0.00267 weeks <br />6.145075e-4 months <br /> on 1/23/04, Control Room Operators noted numerous alarms and the Shift Supervisor announced to the Operators that DP-1201 had failed.
The Secondary Reactor Operator identified main feedwater regulating valves for steam generators A" and "B" had full open demand signals and attempted to take manual control of the manuaVauto (M/A) station for each valve. The other two steam generators were not affected by the DP-1201 failure. The Primary Reactor Operator responded to automatic rod insertion caused by a failed reference temperature signal. The Primary Reactor Operator verified the inward rod motion was due to the failed channel and then placed rod control in manual as directed by the Shift Supervisor. While the Shift Supervisor and Primary Reactor Operator were addressing the failure related rod motion, the actions taken by the Secondary Reactor Operator were inadequate to regain control of the "An and ABE Steam Generator levels. At 1616, about 66 seconds after the inverter failure, high steam generator water levels initiated an automatic turbine trip/reactor trip signal. The plant was stabilized, and after troubleshooting and repair, the reactor was restarted at approximately 2237 hours0.0259 days <br />0.621 hours <br />0.0037 weeks <br />8.511785e-4 months <br /> on 01/25/04.
Failure of inverter 1201 caused the loss of all loads fed by distribution panel DP 1201, which includes the NSSS Process Cabinet Protection Set 1. The loss of Protection Set 1 caused both loop 1 and loop 2 steam generator level signals to fail low. These level signals are sent to NSSS control cabinets 01 and 02 as input signals to the steam generator level control loops. The steam generator level controls fully opened loop 1 and loop 2 feedwater regulating valves as a reaction to the false Steam Generator Low Level Signals. The operator attempted to take manual control at the manual/auto station, but turbine trip/reactor trip occurred on high water level in the loop 2 steam generator before full control could be achieved.
LER 1-04-001 (c).doc STI: 31711765 LER 1-04-001 (c).doc STI: 31711765
(if more space requred.use additional copies of NRC Fom 366A) (17)
Damage Assessment and Investigation Some board indications were noted to have erroneous indications such as very low, but not zero, indications. Subsequent investigation determined that the output of the 1201 inverter was at about 9 volts and had not failed to zero, causing the anomalous indications.
Troubleshooting determined that the "TI" Ferro-Resonant Transformer (FRT) had a tum-to-tum short in the secondary windings resulting in the 9-volt output, which is not enough to provide proper indication. The FRT was replaced and the failed FRT was sent to an offsite laboratory for failure analysis. The insulation varnish on the windings was found to be very brittle and fragile. The conclusion Is that the failure was the result of thermal aging. The FRT that failed was a part of the original equipment installation.
Summary of Root Cause The Technical Root Cause is that the STP design of the Instrument Power Supply (VA)
System is not single train fault-tolerant and failure of the inverter causes significant operational challenges. Failure of a Class 1 E instrument power supply bus Inverter causes multiple instrument failures and creates rapid response events for operations. The STP design incorporates a manual transfer switch that is not effective at providing a fast transfer from the failed Inverter to the bypass power source. Corrective actions in this area are centered on evaluating modifications to provide a more fault tolerant Instrument Power Supply (VA) System and on increasing the reliability of SG level controls and other instrumentation associated with bus 1 E vital Instrument power supplies.
The Organizational Root Cause was that the Station has not aggressively pursued effective failure prevention strategies even though previous events have shown that the inverters are susceptible to failure and the Instrument Power Supply (VA) System Is not single train fault-tolerant.
The root cause analysis of a previous inverter failure identified several design changes that were implemented at other stations to address inverter failure.
The focus during the investigation was centered on operational responses and addressing the direct cause of the failure. Modification to make the Instrument Power Supply (VA) System design more fault tolerant to Inverter failures was not pursued. The root cause investigation was conducted by an SCAQ qualified Investigator, reviewed by a division manager, and again reviewed by the Condition Review Group. The identification of a narrowly focused root cause and generic implications was either not questioned or proper follow-up was not performed to address these areas.
During a review by the Systems Engineering Equipment Reliability Team (EQRT), the team identified that failure of an inverter would cause a significant operational rapid response challenge. Credit for Operator action was used as justification for not initiating a Condition Report to address this issue.
Despite the fact that failures of FRTs have been responsible for the majority of inverter failures, no coping strategies such as initiating PMs for periodic replacement, obtaining a different FRT design, or developing a monitoring plan to be able to predict degraded FRTs LER 1-04-001 (c).doc STI: 31711765 LER 1-04-001 (c).doc STI: 31711 765
I (If more space is requied, use addift coes of (if more space Is required, use additional copies of NRC Fonn 366,A) (17)
Ill. CAUSE OF THE EVENT A. Technical Root Cause: The design of the Class 1E Vital Instrument Power Supply (VA)
System is not single train fault-tolerant and failure of the inverter causes significant operational challenges.
B. Organizational Root Cause: The Station has not aggressively pursued effective failure prevention strategies even though previous events have shown that the inverters are susceptible to failure and the Instrument Power Supply (VA) System is not fault tolerant.
C. Contributing Cause
Operations actions to mitigate the consequences of the failed inverter were inadequate.
IV. CORRECTIVE ACTIONS
A. Evaluate modifications needed to eliminate a significant plant transient or a plant trip as a result of inverter failures.
- Develop a list of all simulated malfunctions that necessitate prompt operator action (e.g.,
less than two minutes) to maintain plant control.
- Submit these scenarios to the LOR Curriculum Review Committee for incorporation into frequent LOR cycle simulator training.
B. Develop simulator training to include the following aspects:
Flawed success path to raise awareness of diagnostics
- Monitoring of critical operating parameters and reinforcement of critical communications (i.e., use of PVT, margin to trip)
- Single operator to diagnose flaw while monitoring and communicating margin to actuation
- Discussion of Conduct of Operations expectations and background for avoiding an automatic reactor trip when possible C. Review Off-Normal procedures that Include expected trip or actuation values for consistency with Operations Management expectations.
Initiate procedure change requests as appropriate.
D. Using the list of malfunctions that necessitate prompt operator action, determine if fault tolerant designs should be developed for any other control systems.
LER 1-04-001 (c).doc STI: 31711765(1-2001)
LICENSEE EVENT REPORT (LER)
- 1. FACILITY NAME
- 2. DOCKET
- 6. LER NUMBER
- 3. PAGE South Texas Unit 1 05000 498 YEAR SEQUREAL IREVISION 6
OF 6
Souh exa Uit 1NUMBER NUMB6E0 2004 01 00 NARRATVE (It more space Is required, use additional copies of NRC Foim 3 20A)
(00 70 E. Review all equipment failure related SCAQ root cause investigations completed in 2000, 2001 and 2002 to determine if appropriate root causes, corrective actions, and generic implications were evaluated.
F. Perform a reliability analysis of the Class I E inverters to identify changes to preventive maintenance activities to increase the reliability of the inverters.
G. Present a case study of previous inverter failures to SCAQ investigators, Plant Engineering personnel and to key Station Management.
V. PREVIOUS SIMILAR EVENTS
A. On December 30, 2003 at 1813, STP Unit 2 inverter 1202 failed resulting in Steam Generator Power Operated Relief Valve (PORV) 2D partially opening. PORV 2D opened to approximately 12% and would not close from the Control Room hand switch. Reactor power increased from 3853 MWth to 3859 MWth and was promptly lowered by reducing turbine load. Steam Generator 2D PORV was manually isolated and distribution panel 1202 was transferred to regulating transformer 1202.
B. On July 7, 2002 Unit 2 was operating in Mode 1 at 100% power. The Unit 2 main turbine generator tripped automatically due to a High-High level in the 2B steam generator (SG).
The reactor tripped automatically as a result of the main turbine trip. The trips occurred shortly after the Channel II inverter and distribution panel de-energized. The loss of the distribution panel and inverter resulted in the loss of power to the Instrumentation channels selected to control narrow range steam generator water level. This failure resulted in loss of SG level signal to all four SG Main Feedwater Regulating Valve (MFRV) control circuits because they were all selected to the same channel. This caused the MFRVs to go fully open. With the MFRVs fully open, water level increased in all four steam generators. Steam generator 2B reached its high-high level set point resulting In the main turbine trip and the feedwater isolation signal. The cause of the inverter failure and distribution panel loss of power was the blowing of the direct current (DC) input fuse which de-energized the inverter and power supply to the distribution panel. The second cause of the reactor trip was having all four steam generator level control switches aligned to a single control channel coupled with the loss of power to instruments on that channel.
VI. ADDITIONAL INFORMATION
None LER 1-04-001(c).doc STI: 31711765