05000382/LER-2017-002

LER-2017-002, Automatic Reactor Scram due to the Failure of Fast Bus Transfer Relays to Automatically Transfer Station Loads to Off- Site Power on a Main Generator Trip

Waterford Steam Electric Station, Unit 3
Event date:	07-17-2017
Report date:	09-18-2017
Reporting criterion:	10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications 10 CFR 50.73(a)(2)(iv)(A), System Actuation 10 CFR 50.73(a)(2)(vii), Common Cause Inoperability 10 CFR 50.73(a)(2)(v)(A), Loss of Safety Function - Shutdown the Reactor
Initial Reporting
ENS 52863	10 CFR 50.72(b)(3)(v)(A), Loss of Safety Function - Shutdown the Reactor, 10 CFR 50.72(b)(2)(iv)(B), RPS System Actuation, 10 CFR 50.72(a)(1)(i), Emergency Class Declaration, 10 CFR 50.72(b)(3)(iv)(A), System Actuation
3822017002R00 - NRC Website

LER 17-002-00 for Waterford, Unit 3, Regarding Automatic Reactor Scram due to the Failure of Fast Bus Transfer Relays to Automatically Transfer Station Loads to Off-Site Power on a Main Generator Trip

ML17261B215
Person / Time
Site:	Waterford, Waterford Steam Electric Station, Unit 3
Issue date:	09/18/2017
From:	Jarrell J P Entergy Operations
To:	Document Control Desk, Office of Nuclear Reactor Regulation
References
W3FI -2017-0069 LER 17-002-00
Download: ML17261B215 (7)

EVENT DESCRIPTION

A. Plant Status C. Event Causes B. Event Chronology At the time of this event Waterford 3 was operating at 100% reactor power. The site was experiencing a severe thunderstorm. No structures, systems or components were out of service that contributed to this event.

On July 17, 2017, at approximately 1555 CDT, Waterford 3 control room operators received indications of an electrical grid spike. The control room received several control panel alarms related to the Waterford 3 switching station [FK] and operators were dispatched to investigate the electrical components in the transformer yard [EL]. During this investigation field operators reported that the Isophase Bus duct to Main Transformer ‘B' [MT] was glowing orange and arcing and sparks were observed.

At 1606 CDT, control room operators manually tripped the main turbine [TA] to de-energize Main Transformer ‘B'. When the main turbine was tripped the main generator [TB] automatically tripped and Reactor Power Cutback [JD] initiated as designed. The transfer of electrical busses did not occur as expected and the site experienced a loss of off-site power to the safety and non-safety electrical busses [SWGR]. This loss of off-site power resulted in a loss of all four Reactor Coolant Pumps (RCP) [P] which then resulted in an automatic reactor scram [JC] on loss of forced circulation. Both emergency diesel generators (EDG) [DG] started and loaded to provide power to both safety busses. The automatic actuation of the reactor protection system [JC], reactor scram, and the automatic start of both EDG's, are reportable conditions pursuant to 10 CFR 50.73(a)(2)(iv)(A).

At 1617 CDT the Shift Manager declared an Unusual Event for a loss of off-site power for >15 minutes (EN 52863).

Due to the loss of power both Main Feedwater Pumps [SK] tripped resulting in levels in both Steam Generators [SG] lowering to the initiation setpoint of Emergency Feedwater (EFW) [BA] at the time of the automatic scram. At 1630 CDT, when reactor coolant system cold leg temperature lowered to 530 degrees Fahrenheit, plant operators took manual control of EFW to control Steam Generator levels as required in site Emergency Operating procedures. The automatic actuation of the EFW system is a reportable condition pursuant to 10 CFR 50.73(a)(2)(iv)(A).

At 1831 CDT off-site power was restored to the ‘A' train non-safety electrical busses.

At 1844 CDT off-site power was restored the ‘A' train safety electrical bus.

At 1854 CDT EDG ‘A' was secured and returned to standby, operating for two (2) hours and 47 minutes.

At 1944 CDT off-site power was restored to the ‘B' train non-safety electrical busses.

At 2001 CDT off-site power was restored to the ‘B' train safety electrical bus.

At 2015 CDT EDG ‘B' was secured and returned to standby, operating for four (4) hours and eight (8) minutes.

At 2056 CDT the Shift Manager exited the Emergency Plan and secured from the Unusual Event.

At 0116 CDT on July 18, 2017, control room operators started the Auxiliary Feedwater Pump and commenced feeding both SG's from the condensate system.

At 0128 CDT on July 18, 2017, the emergency feedwater actuation signal was reset. EFW pumps ‘A', ‘B', and ‘AB' were secured, operating for nine (9) hours and 21 minutes.

The direct cause of the Isophase Bus Duct overheating is the failure of a shunt assembly connection to the Phase B bus duct.

The most likely apparent causes are: (1) a grid transient recorded on July 17, 2017, produced a dynamic response between the isophase bus and the bus duct that resulted in the separation of the shunt assembly connection, and; (2) Isophase Bus Duct Overheating:

comments regarding burden estimate to the FOIA, Privacy and Information Collections Branch (T-5 F53), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by e-mail to used to impose an information collection does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, the information collection.

used to impose an information collection does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, the information collection.

The inadequate design of the fast bus transfer relays resulted in a common cause inoperability of both trains of off-site power. This condition is reportable pursuant to 10 CFR 50.73(a)(2)(vii).

CORRECTIVE ACTIONS

A. Isophase Bus Overheating:

B. Failure of Fast Dead Bus Transfer:

The direct cause of the failure of the fast dead bus transfer was the Struthers Dunn (S-D) 237 Series Direct Current (DC) Time Delay on Dropout (TDDO) relays [62] installed in the fast dead bus transfer circuitry instantaneously timed out when they were exposed to DC coil inductive kick, which prevented automatic transfer of the safety and non-safety electrical busses from the Unit Auxiliary Transformers [XFMR] to the Startup Transformers [XFMR].

The Root Cause of this event was design change procedures in effect during the development of the 1997 and 2017 modifications to the fast dead bus transfer circuitry did not include guidance that electronic devices have a greater susceptibility to DC coil inductive kick than electro-mechanical devices and did not require identification of critical characteristics for non-quality related plant changes.

The Contributing Cause of this event was the post-modification testing performed following change of the relays from Allen Bradley to Struthers Dunn did not exercise the fast dead bus transfer timing circuitry. This contributed to this condition by delaying detection of relay failure.

degraded connections between the laminated shunt assemblies and the bus duct resulted in formation of gaps between the contact surfaces. Electrical arcing across these gaps over time eroded the contact surfaces which increased resistance and weakened the connection.

Completed corrective actions include:

a. The Main Transformer ‘B', Phase ‘B' isolated phase bus duct was repaired by cutting out the damaged duct sections and welding in new aluminum material. All laminated shunt assemblies were replaced using new bolting hardware.

b. The Main Transformer ‘B' Phase ‘A' and ‘C' laminated shunt assemblies were removed, cleaned and re-installed with new bolting hardware.

c. Performed a complete inspection and torque check for all laminated shunt assemblies to identify any other degraded bolted connections and adjusted torque as needed.

Corrective actions planned include:

a. Revise preventive maintenance strategy for isophase bus based on EPRI and industry recommendations to include shunt assembly torque checks, regular bus inspections, thermography scans, and corona scans.

b. Revise the site procedure for inspection of the isophase busses and ducting based on engineering input.

Completed corrective actions include:

a. The installation of a suppression diode in parallel with fast bus transfer relay. (Corrective Action to Prevent Recurrence) b. The Struthers-Dunn relays were removed from service and replaced with Allen Bradley relays.

c. Entergy Nuclear Fleet has adopted the Standard Design Process in accordance with IP-ENG-0001, Revision 0, Standard Design Process.

d. Fast Dead Bus Transfer engineering change test procedure was developed during troubleshooting and was performed following re-installation of the Allen Bradley relays and the installation of flyback diodes.

Corrective actions planned include:

a. Verify that all Allen-Bradley 700RTC relays installed in the plant are immune to the effects of inductive kick. If any Failure of Fast Dead Bus Transfer:

used to impose an information collection does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, the information collection.

SAFETY EVALUATION

The inoperability of both trains of off-site electrical distribution is a reportable condition pursuant to 10 CFR 50.73(a)(2)(v)(A) and (D).

The non-compliance with Technical Specifications is a reportable condition pursuant to 10 CFR 50.73(a)(2)(i)(B).

The safety significance determination of this event is still being determined. The safety significance will be included as a planned update to this LER.

PREVIOUS OCCURRENCES

CR-WF3-2015-3566: Failure of ‘B' Fast Dead Bus Transfer during Plant Scram on June 3, 2015. (Reported under Technical Specification (TS) 3.8.1.1 requires two physically independent circuits between the off-site transmission network and the on-site Class 1E distribution system to be operable. The operability of the electrical distribution systems during operation ensures that sufficient power will be available to supply the safety-related equipment required for (1) the safe shutdown of the facility, and (2) the mitigation and control of accident conditions within the facility. The minimum specified independent and redundant A.C. and D.C. power sources and distribution systems satisfy the requirements of General Design Criterion 17 of Appendix A to10 CFR Part 50. The fast dead bus transfer system is required to be functional per the Waterford 3 FSAR to support the operability of off-site power.

The Struthers-Dunn relays were installed in the fast dead bus transfer circuit in May of 2017, and were required to be functional on June 2, 2017, at 1902, when the main generator was paralleled to the grid and loads were transferred to the Unit Auxiliary Transformers at the conclusion of Refueling Outage 21. Both trains of the offsite transmission network have been inoperable from that time until the time of the generator trip on July 17, 2017. Action ‘e' of TS 3.8.1.1 requires restoration of one of the inoperable offsite A.C. circuits to OPERABLE status within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> or be in at least HOT STANDBY within the next 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />. Because of this, Waterford 3 was not in compliance of TS 3.8.1.1 for 45 days.

Directly following a manual plant scram on June 3, 2015, due to a secondary system malfunction, a loss of off-site power occurred on the ‘B' Train due to the failure of the fast dead bus transfer relays on the ‘B' train. The Apparent Cause of this event was determined to be Unknown Equipment Cause (FEUU). A contributing cause was that the 1997 Plant Change was due to a Design Change That Was Not Adequate (FE18) because a latent design deficiency did not take into account the observed failure mode where a two coil design allowed one coil to fail preventing the fast dead bus transfer function (timed contacts) without actuating the contacts for the alarm circuit.

The corrective actions from this earlier event did not correct the issue and prevent recurrence because the nature of the relay failure was not determined conclusively. Corrective actions were assigned from the apparent cause without a solid understanding of the causal factors for the respective failure.

Allen-Bradley relays are discovered to not be immune to the effects of Inductive kick, generate additional actions to correct the lack of immunity.

b. Review and verify the existing EMC qualifications of currently installed Allen-Bradley 700RTC relays to ensure that they are appropriate for their operating environment. If any Allen Bradley relays are discovered to not be appropriate for their operating environment, generate additional corrective actions to correct the deficiency.

c. Anchor surge suppression/flyback diodes into design basis drawings and the Fast Dead Bus Transfer circuit Control Wiring Diagrams to ensure they are not removed if the Fast Dead Bus Transfer circuit is modified. (Corrective Action to Prevent Recurrence) From this event, all fast dead bus transfer relays were replaced with new Allen -Bradley relays. Preventative Maintenance strategies were revised to replace the relays on a three (3) year frequency.