05000333/LER-2003-001, Automatic Reactor Shutdown Due to Grid Instability Associated with the August 14, 2003 Transmission Grid Blackout and Related Plant Mode Change with the EDG Subsystem Inoperable

Automatic Reactor Shutdown Due to Grid Instability Associated with the August 14, 2003 Transmission Grid Blackout and Related Plant Mode Change with the EDG Subsystem Inoperable

ML032960125
Person / Time
Site:	FitzPatrick
Issue date:	10/14/2003
From:	Ted Sullivan Entergy Nuclear Northeast, Entergy Nuclear Operations
To:	Document Control Desk, Office of Nuclear Reactor Regulation
References
CR-JAF-2003-03930, CR-JAF-2003-04203, JAFP-03-0138 LER 03-001-00
Download: ML032960125 (8)

LER-2003-001, Automatic Reactor Shutdown Due to Grid Instability Associated with the August 14, 2003 Transmission Grid Blackout and Related Plant Mode Change with the EDG Subsystem Inoperable

Event date:
Report date:
Reporting criterion:	10 CFR 50.73(a)(2)(iv), System Actuation 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications
3332003001R00 - NRC Website

text

Entergy Nuclear Northeast Entergy Nuclear Operations, Inc.

James A. Fitzpatrick NPP R~~~~~~~~~~~~~~~~~~~~~~PO.

Box 110 Entergy Lycoming, NY 13093 Tel 315 349 6024 Fax 315 349 6480 T.A. Sullivan October 14, 2003 Site Vice President - JAF JAFP-03-0138 United States Nuclear Regulatory Commission Attn: Document Control Desk Mail Station P1-137 Washington, D.C. 20555

Subject:

Docket No. 50-333 LICENSEE EVENT REPORT: LER-03-001 -00 (CR-JAF-2003-03930; CR-JAF-2003-04203)

Automatic Reactor Shutdown Due to Grid Instability Associated With the August 14, 2003 Transmission Grid Blackout and related Plant MODE Change with the A EDG Subsystem Inoperable

Dear Sir:

This report is submitted in accordance with 10 CFR 50.73(a)(2)(iv)(a), "Any event or condition that resulted in manual or automatic actuation of any of the systems listed in paragraph (a)(2)(iv)(B) of this section." This report is also submitted in accordance with 10 CFR 50.73(a)(2)(i)(B), "Any operation or condition which was prohibited by the plant's Technical Specifications."

There are no commitments contained in this report.

Questions concerning this report may be addressed to Mr. John Hoddy at (315) 349-6538.

Very truly yours T. A. Sullivan TAS:JH:jh Enclosure cc:

USNRC, Region 1 USNRC, Project Directorate USNRC Resident Inspector INPO Records Center

Abstract

On August 14, 2003 at 1611 hours0.0186 days <br />0.448 hours <br />0.00266 weeks <br />6.129855e-4 months <br />, with the reactor operating at 100 percent power, a reactor scram occurred due to unstable conditions on the transmission grid causing rapid cycling of Turbine Control Valves (TCVs). This resulted in low turbine Electro-hydraulic Control System (EHC) oil pressure which caused a "Turbine Control Valve Fast Closure, EHC Oil Pressure -

Low" Scram as instrumentation sensed conditions similar to a TCV fast closure - generator load rejection. A Main Turbine trip and generator trip followed approximately thirty-five seconds later, accompanied by a normal fast transfer of plant 4 kV busses to the offsite 115 kV supply. A subsequent loss of offsite power followed approximately two minutes later. Plant Emergency Diesel Generators (EDG) started and provided power to safety related plant loads. A cooldown was conducted using Safety Relief Valves, High Pressure Coolant Injection, and Reactor Core Isolation Cooling. MODE 4 was achieved on August 15, 2003 at 2220 hours0.0257 days <br />0.617 hours <br />0.00367 weeks <br />8.4471e-4 months <br />.

On August 29, 2003 at 1015 hours0.0117 days <br />0.282 hours <br />0.00168 weeks <br />3.862075e-4 months <br />, during surveillance testing, the A EDG subsystem was determined to be inoperable due to out of tolerance no-load frequency, traceable to recovery from the August 14 event. A plant start-up had occurred subsequent to August 14, with an accompanying plant MODE change prohibited by Technical Specifications.

NRC FORM 366 (7-2001)

I I I

, I,

i

' I ".

I

'!

1.
I -U.S. NUCLEAR REGULATORY COMMISSION (6-1998)

LICENSEE EVENT REPORT (LER)

TEXT CONTINUATION FACILITY NAME (1) -

DOCKET (2) : -

LER NUMBER (6)

PAGE (3)

YEAR SEQUENTIAL REVISION 2

OF 7

NUMBER NUMBER James A. FitzPatrick Nuclear Power Plant 05000333 03 001 00 TEXT (If more space is required, use additional copies of NRC Form 366A) (17)

EIIS Codes in [ ]

Fvpnt Description Scram due to grid instability, with subsequent loss of offsite power On August 14, 2003 at 1611 hours0.0186 days <br />0.448 hours <br />0.00266 weeks <br />6.129855e-4 months <br />, with the reactor operating at 100 percent power, an automatic reactor shutdown (scram) occurred due to conditions associated with grid instability. Grid perturbations ultimately led to an area-wide transmission system blackout. The unstable conditions on the transmission grid caused rapid cycling of Turbine Control Valves (TCVs) [JJ] resulting in low Electro-hydraulic System (EHC) [TG] oil pressure.

Low EHC oil pressure caused the plant Reactor Protection System (RPS) [JC] instrumentation to sense conditions similar to a TCV fast closure (Reactor Thermal Power (RTP) >/= 29%; EHC oil pressure >= 500 psig, </=850 psig), resulting in a "Turbine Control Valve Fast Closure, EHC Oil Pressure -

Low" Scram. The Main Generator [TB] remained synchronized to the grid for approximately 35 seconds with transmission grid conditions continuing to fluctuate, at which point the Main Turbine [TA] tripped on high reactor water level, followed by a generator trip and a normal fast transfer of plant 4 kV busses (busses 10100 through 10600) [EA]

to the 115 kV offsite power supply [FK].

At approximately 1613 hours0.0187 days <br />0.448 hours <br />0.00267 weeks <br />6.137465e-4 months <br />, degraded conditions on the 115 kV transmission system resulted in an actuation of plant Loss of Power instrumentation [JE], causing an automatic start of plant Emergency Diesel Generators [EK] and an automatic transfer of plant emergency busses (busses 10500 and 10600) from the degraded offsite circuits to the EDGs. Degrading conditions on the 115 kV system initially actuated 4.16 kV Emergency Bus Undervoltage (Degraded Voltage) circuitry on undervoltage, starting a transfer sequence timer (nominal 45 second time delay). Shortly thereafter, continuing 115 kV voltage degradation actuated 4.16 kV Emergency Bus Undervoltage (Loss of Voltage) circuitry (nominal 2.5 second time delay), which timed to completion, resulting in the described transfer.

Following the transfer, the emergency busses (i.e. 10500 and 10600) were being supplied by EDGs. Plant non-safety related loads (busses 10100, 10200, 10300, and 10400) remained supplied from the 115 kV transmission system. Commencing shortly after 1615 hours0.0187 days <br />0.449 hours <br />0.00267 weeks <br />6.145075e-4 months <br /> and extending over a period of approximately forty-five seconds, plant non-safety related load breakers tripped on overcurrent and/or undervoltage such that by 1616 hours0.0187 days <br />0.449 hours <br />0.00267 weeks <br />6.14888e-4 months <br />, power was lost to all non-safety related loads. During this interval, Main Circulating Water Pumps [KE] were among those loads that tripped. With the loss of Main Circulating Water, operators manually closed Main Steam Isolation Valves (MSIVs) [SB]. A plant cooldown to MODE 4 was commenced using Safety Relief Valves (SRVs) [SB] and High Pressure Coolant Injection (HPCI) [BJ], later augmented by Reactor Core Isolation Cooling (RCIC) [BN]. MODE 4 was achieved on August 15 at 2220 hours0.0257 days <br />0.617 hours <br />0.00367 weeks <br />8.4471e-4 months <br />.

Reactor water level at the time of the event was being controlled in the normal band (196.5 - 206 inches above top of active fuel (TAF)). Following the Scram, level dropped rapidly due to shrink effects. This initial transient initiated HPCI on Reactor Vessel Water Level -

Low Low (Level 2). HPCI did not inject, as level recovered above the initiation setpoint before the injection valve permissive conditions were satisfied. RCIC did not start, due to the rapid nature of the level excursion and recovery transient. Level recovered quickly due to the combined contributions of Reactor Feed Pumps (RFPs) [SK], which responded rapidly to the transient, and transient conditions within the reactor vessel.U.S. NUCLEAR REGULATORY COMMISSION (6-1998)

LICENSEE EVENT REPORT (LER)

TEXT CONTINUATION FACILITY NAME (1)

DOCKET (2)

LER NUMBER (6)

PAGE (3) y.EAR I SEQUENTIAL RESION 3

OF 7

NUMBER NUMBER James A. FitzPatrick Nuclear Power Plant 05000333 03 001 00 TEXT (If more space is required, use additional copies of NRC Form 366A) (17)

Fvpnt Dprcription (nntinuld)e Level control was subsequently maintained with HPCI and RCIC and cooldown conducted using SRVs, HPCI, and RCIC.

All system isolations [D] and initiations associated with low Reactor Vessel Water Level occurred as expected.

These included isolations of Reactor Water Clean-up, Main Steam Line Drains, Primary Coolant Sampling Valves, Primary Containment Hydrogen and Oxygen Sensing lines, and Secondary Containment; and automatic initiation of the Standby Gas Treatment System [BH].

Offsite electrical systems were recovered over a period of time, commencing at 1900 hours0.022 days <br />0.528 hours <br />0.00314 weeks <br />7.2295e-4 months <br /> on August 14, 2003 with restoration of the 115 kV transmission system with an imposed load limit and ending at 2400 hours0.0278 days <br />0.667 hours <br />0.00397 weeks <br />9.132e-4 months <br /> on August 14, 2003 with restoration of 115kV transmission system to full capacity. EDGs continued to carry plant safety bus loads until busses were transferred to their normal qualified offsite circuits at 2307 hours0.0267 days <br />0.641 hours <br />0.00381 weeks <br />8.778135e-4 months <br /> (10600 bus) and 2328 hours0.0269 days <br />0.647 hours <br />0.00385 weeks <br />8.85804e-4 months <br /> (10500 bus) respectively. EDGs were shutdown by 2345 hours0.0271 days <br />0.651 hours <br />0.00388 weeks <br />8.922725e-4 months <br /> on August 14, 2003.

The site Emergency Plan was entered at the Unusual Event level (EAL 6.1.1 - loss of off-site power for greater than 15 minutes) at 1626 hours0.0188 days <br />0.452 hours <br />0.00269 weeks <br />6.18693e-4 months <br /> on August 14, 2003. The plant remained in an Unusual Event until exiting at 0039 hours4.513889e-4 days <br />0.0108 hours <br />6.448413e-5 weeks <br />1.48395e-5 months <br /> on August 15, 2003.

This event is reportable under the provisions of 10 CFR 50.73(a)(2)(iv)(a), "Any event or condition that resulted in manual or automatic actuation of any of the systems listed in paragraph (a)(2)(iv)(B) of this section," due to RPS actuation, containment isolation signal actuation involving multiple systems, HPCI and RCIC actuation, and EDG system auto-initiation, including associated Emergency Service Water System initiation.

MODE change with the A EDG subsystem inoperable At 1015 hours0.0117 days <br />0.282 hours <br />0.00168 weeks <br />3.862075e-4 months <br /> on August 29, 2003, during performance of regularly scheduled surveillance test ST-9BA, EDG A and C Full Load Test and ESW Pump Operability Test," frequency for the A EDG subsystem stabilized at 61.3 Hz under no-load conditions following auto-start and force paralleling. This is contrary to the required steady state frequency of >/= 58.8 Hz and </= 61.2 Hz required by Technical Specifications (TS) SR 3.8.1.2.b.

Surveillance test results were satisfactory with the exception of no-load frequency. These EDGs were last run to supply power during the August 14, 2003 transmission system blackout at which time plant operating procedures were used to set up EDGs for subsequent auto-start as part of the shutdown evolution of the EDGs.

The A EDG subsystem was thus previously set up for auto-start when securing the EDGs, following restoration of the 115 kV transmission system and normal offsite power during recovery from the August 14 transmission system blackout. This set-up during EDG shutdown occurred at approximately 2343 hours0.0271 days <br />0.651 hours <br />0.00387 weeks <br />8.915115e-4 months <br /> on August 14.

Evaluation has determined that the cause of the out of tolerance no-load frequency experienced during the August 29 test resulted from setting up the EDGs for auto-start on August 14 based upon adjustment and comparison with a still slightly off-normal transmission system, which was operating at slightly above normal frequency.

J,U.S. NUCLEAR REGULATORY COMMISSION (6-1998)

LICENSEE EVENT REPORT (LER)

TEXT CONTINUATION FACILITY NAME (1) l DOCKET (2)

LER NUMBER (6)

PAGE (3) yEA I SEQUENTIAL REVISION 4

OF 7

I NUMBER NUMBER James A. FitzPatrick Nuclear Power Plant 05000333 03 001 00 TEXT (If more space is required, use additional copies of NRC Form 366A) (17)

EvPnt DP-grrptenn (ontinuad)!

The A EDG subsystem had been declared inoperable for periodic testing at 0755 hours0.00874 days <br />0.21 hours <br />0.00125 weeks <br />2.872775e-4 months <br /> on August 29 and remained inoperable based upon failure to satisfy SR 3.8.1.2.b requirements. As part of ST-9BA, EDG frequency is adjusted and EDGs are paralleled with plant busses for the load testing portion of the surveillance test. At the completion of the test, EDGs are set up for subsequent auto-start in procedural steps analogous to those contained in plant Operating Procedures. The A EDG system was thus set up for auto-start following the performance of ST-9BA, with set-up based upon known, stable transmission system conditions.

At 1053 hours0.0122 days <br />0.293 hours <br />0.00174 weeks <br />4.006665e-4 months <br /> on August 29, ST-9R, EDG System Quick-Start Operability Test" was performed for the A EDG subsystem, a test which demonstrates compliance with the requirements of SR 3.8.1.2. Test results were satisfactory, demonstrating operability of the A EDG subsystem.

ST-9R was repeated on September 5 to assure that the A EDG subsystem performance had not degraded due to undetermined cause. Test results were satisfactory and consistent with those obtained on August 29.

Subsequent surveillance test results have also been satisfactory.

ST-9BB, BEDG B and D Full Load Test and ESW Pump Operability Test" was completed for the B EDG subsystem at 2348 hours0.0272 days <br />0.652 hours <br />0.00388 weeks <br />8.93414e-4 months <br /> on August 29. Test results were satisfactory, confirming that the extent of condition for the subject event was limited to the A EDG subsystem.

The discrepant condition existed from the time the system was set up for auto-start at 2343 hours0.0271 days <br />0.651 hours <br />0.00387 weeks <br />8.915115e-4 months <br /> during shutdown of the EDG subsystem on August 14, 2003. The A EDG subsystem is considered to have been inoperable from that time until satisfactory performance of ST-9R at 1053 hours0.0122 days <br />0.293 hours <br />0.00174 weeks <br />4.006665e-4 months <br /> on August 29. Plant start-up from the August 14 scram occurred during this time period and resulted in a MODE change, which is prohibited by TS LCO 3.0.4, even though the condition was not known at the time. This event is therefore reportable under the provisions of 1 OCFR 50.73(a)(2)(i)(B), Any operation or condition which was prohibited by the plant's Technical Specifications."

lCwuce of Ev~nt*

Scram due to grid instability, with subsequent loss of offsite power The event was caused by grid instability associated with the regional transmission system blackout that occurred on August 14, 2003.

MODE change with the A EDG subsystem inoperable The A EDG subsystem (A and C EDGs) was properly restored in accordance with plant procedures following the grid transient. The system operator was gradually restoring system loads in balance with available generation. Given the abnormal conditions associated with restoration of the grid as compared to normal steady state conditions, the A subsystem EDGs were synchronized to the grid while grid frequency was slightly above normal steady state conditions. This resulted in setting the A EDG subsystem to a slightly higher no-load frequency than normal.U.S. NUCLEAR REGULATORY COMMISSION (6-1998)

LICENSEE EVENT REPORT (LER)

TEXT CONTINUATION FACILITY NAME (1)

DOCKET (2)

LER NUMBER (6)

PAGE (3)

YEAR l SEQUENTIAL.

REVISION 5

OF 7

NUMBER NUMBER James A. FitzPatrick Nuclear Power Plant 05000333 03 001 00 TEXT (if more space is required, use additional copies of NRC Form 366A) (17) lCamp of Event (onntinuied)

The condition was undetected prior to startup as the EDGs were within their normal surveillance interval, the EDGs had exhibited normal performance during the August 14, 2003 event, the EDGs were properly restored in accordance with plant procedures, and the above normal grid frequency at the time of shutdown was not known or suspected prior to plant startup. The condition was thus not detected until the next regularly scheduled EDG subsystem surveillance testing, performed on August 29, 2003.

The event was compounded by individual subsystem characteristics; the A EDG subsystem tends to run at slightly higher no-load frequency than the B EDG subsystem, as discussed in the Event Analysis section.

Ent Analysis-Scram due to grid instability, with subsequent loss of offsite power Plant systems responded as designed during the event.

MODE change with the A EDG subsystem inoperable An engineering evaluation was performed on August 29 to evaluate the discrepant out of tolerance no-load frequency condition. This evaluation assessed that the A EDG subsystem would have been capable of performing its safety function throughout the out of tolerance interval with no loss of safety function.

The cause of the A EDG subsystem inoperability is a higher than normal set-up frequency associated with the August 14 EDG shutdown. This shutdown was atypical in at least two ways.

1. Normally, the 345 kV system, which is operated at an extremely constant frequency, is used as a reference for EDG set-up. In this instance, the 345 kV system had not yet been restored to service.

The 115 kV system was used as a reference instead, as allowed by procedure.

2. At the time of setting up the EDGs, the system operator had just given permission for various users, including FitzPatrick, to resume unrestricted 115 kV operations. FitzPatrick and other users were in the process of restoring major loads to the 115 kV system. Under such conditions, it would reasonable for the system operator to establish system frequency slightly above normal to accommodate the addition of large loads associated with system restoration without causing unacceptable voltage and frequency transients on the system.

It is postulated that the system operator was operating the 115 kV transmission system at a frequency slightly above 60 Hz at the time of setting up the A EDG subsystem. Frequency data on a timeline is not available.

However, this cause has been indirectly confirmed by setting up the A EDG subsystem under known, stable 60 Hz conditions and subsequently testing the EDG subsystem under fast start conditions. The A EDG subsystem has since demonstrated consistent satisfactory performance under those conditions.U.S. NUCLEAR REGULATORY COMMISSION (6-1998)

LICENSEE EVENT REPORT (LER)

TEXT CONTINUATION FACILITY NAME (1)

DOCKET (2)

LER NUMBER (6)

PAGE (3) yEAR I SEQUENTIAL REVISION 6

OF 7

NUMBER NUMBER James A. FitzPatrick Nuclear Power Plant 05000333 03 001 00 TEXT (if more space is required, use additional copies of NRC Form 3664) (17)

Fvent Anaiyvnig (rnntinteip The effects of EDGs running at above normal frequency were evaluated. Two areas of concern were addressed.

1. At above normal frequency, EDGs will carry increased load for the same connected plant components.

Evaluation determined that the increased load associated with the out of tolerance frequency was well within rated capacity for the EDG subsystem.

2. A higher initial frequency reduces the margin to overfrequency trip associated with rapid load rejection, such as a large motor trip. Evaluation determined that the EDG subsystem could accommodate a trip of the largest connected load under these conditions, with ample margin.

In summary, the elevated frequency did not compromise the ability of the A EDG subsystem to perform its safety function(s).

A contributing factor was noted in reviewing past EDG performance in conjunction with this evaluation. System characteristics are such that, under no-load conditions, the A EDG subsystem typically runs at a slightly higher frequency than the B EDG subsystem due to governor action. As a corrective action to this LER, the A EDG subsystem governor(s) will be tuned (adjusted) during Refueling Outage 16 so that the A EDG subsystem controls more closely to the midpoint of the allowable frequency band.

Fxtnt of Condition Scram due to grid instability, with subsequent loss of offsite power N/A MODE change with the A EDG subsystem inoperable The condition was limited to the A EDG subsystem. The B EDG subsystem remained operable throughout the period, except during testing. The B EDG subsystem governor tuning is such that the subsystem controls more closely to the midpoint of the allowable frequency band under no-load conditions, making the B EDG subsystem less susceptible to the effects of off-normal grid frequency than the A EDG subsystem.

Cnrrective Actions-Scram due to grid instability, with subsequent loss of offsite power NoneU.S. NUCLEAR REGULATORY COMMISSION (6-1998)

LICENSEE EVENT REPORT (LER)

TEXT CONTINUATION FACILITY NAME (1)

DOCKET (2) l LER NUMBER (6) l PAGE (3)

YEAR SEQUENTIAL REVISION 7

OF 7

NUMBER NUMBER James A. FitzPatrick Nuclear Power Plant 05000333 03 001 00 TEXT (if more space is required, use additional copies of NRC Forn 366A) (17)

Corrective Actions (continued):

MODE change with the A EDG subsystem inoperable

1. Revise Operating and Surveillance Test Procedures to include provisions for setting up EDGs for auto start, strengthening frequency set up verification and addressing potential off-normal transmission system conditions.

(Scheduled Completion Date: December 15, 2003)

2. Perform tuning of the A and C EDG governor control system (WR-02-07393-00, WR-02-07394-00.)

(Scheduled Completion Date: Refueling Outage 16)

Safety System Fitnetinnal Faihisrp ReviPw None Similar FventS None Failed Component dentification None NRC FORM 3866A (6-1998)