ML17227A424

Special Rept:On 920421,turbine Generator Failed to Trip Following Manual Reactor Trip.Caused by Failures in Redundant & Electrically Independent Turbine Tripping Schemes.I&C Disassembled & Trip Block Inspected

ML17227A424
Person / Time
Site:	Saint Lucie
Issue date:	05/14/1992
From:	Sager D FLORIDA POWER & LIGHT CO.
To:	NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
L-92-149, NUDOCS 9205180236
Download: ML17227A424 (15)

Text

ACCELERATED DEMONSTTION SYSTEM DISTRIBUTION REGULATORY INFORMATION DISTRIBUTION SYSTEM (RIDS)

ACCESSION NBR:9205180236 DOC.DATE: 92/05/14 NOTARIZED: NO DOCKET FACIL:50-389 St. Lucie Plant, Unit 2, Florida Power & Light Co. 05000389 AUTH. NAME . AUTHOR AFFILIATION SAGER,D.A. Florida Power & Light Co.

RECIP.NAME RECIPIENT AFFILIATION Document Control Branch (Document Control Desk)

SUBJECT:

Special rept:on 920421,turbine generator failed to trip following manual reactor trip. Caused by failures in redundant & electrically independent turbine tripping schemes. I&C disassembled & trxp block inspected.

DISTRIBUTION CODE: IE22D COPIES RECEIVED:LTR ENCL SIZE:

TITLE: 50.73/50.9 Licensee Event Report (LER), Incident Rpt, etc.

NOTES RECIPIENT COPIES RECIPIENT COPIES ID CODE/NAME LTTR ENCL 'ID CODE/NAME LTTR ENCL PD2-2 LA 1 1 PD2-2 PD 1 1 NORRIS,J 1 1 INTERNAL: ACNW 2 2 ACRS 2 2 AEOD/DOA 1- 1 AEOD/DS P/TPAB 1 1 AEOD/ROAB/DS P 2 2 NRR/DET/EMEB 7E 1 1 NRR/DLPQ/LHFB10 1 1 NRR/DLPQ/LPEB10 1 1 NRR/DOEA/OEAB 1 1 NRR/DREP/PRPB11 2 2 NRR/DST/SELB 8D 1 1 NRR/DST/SICB8H3 1 1 RR/ LB8D1 1 1 NRR/DST/SRXB 8E 1 1 REG FIL 02 1 1 RES/DSIR/EIB 1 1 RGN2 FILE 01 1 1 EXTERNAL: EG&G BRYCE,J.H 3 3. L ST LOBBY WARD 1 1 NRC PDR 1 1 NSIC MURPHY,G.A 1 1 NSIC POORE,W. 1 1 NUDOCS FULL TXT 1 1 NOTE TO ALL "RIDS" RECIPIENTS:

PLEASE HELP US TO REDUCE V'ASTE! CONTACT THE DOCUMENT CONTROL DESK, ROOiil Pl-37 (EXT. 20079) TO ELlilllNATEYOUR NN!E FROivI DISTRIBUTION LISTS FOR DOCUMENTS YOU DON T NEED!

TOTAL NUMBER OF COPIES REQUIRED: LTTR 32 ENCL -

32

'

P.O. Box 128, Ft. Pierce, FL 34854-0128 May 14, 1992 FPL L-92-149 U. S. Nuclear Regulatory Commission Attn: Document Control Desk Washington, D. C. 20555 Re: St. Lucie Unit 2 Docket No. 50-389 Event Date: April 21, 1992 Turbine Tri Failure U date Attached please find the updated version of a plant problem report which discusses an event which occured on April 21, 1992. Included is the current status of our analysis of the event. Pursuant to 10 CFR 50.73 a Licensee Event Report will be submitted within 30 days of the event date.

Very truly yours, D. A. S ger Vice Pr sident St. L e Plant DAS/JWH/kw Attachment cc: Stewart D. Ebneter, Regional Administrator, USNRC Region II Senior Resident Inspector, USNRC, St. Lucie Plant DAS/PSL 1697-92 9205180236 9205i4 PDR ADOCK 05000389 8 PDR an FPL Group company

ST. LUCIE PLANT IN HOUSE EVENT REPORT 92-023 REV. 9 TURBINE GENERATOR FAILED TO TRIP FOLLOWING MANUAL REACTOR TRIP UNIT' DATE: APRIL 21, 1992 I. INITIAL PLANT CONDITIONS On April 21, 1992, St. Lucic Unit 2 was shutting down for a scheduled refueling outage.

Reactor power was at approximately 12%.

II. EVENT DESCRIPTION A scheduled reactor-turbine shutdown was in progress on Unit 2. During the shutdown, power moved to the top of the reactor core. Existing procedural guidance on axial shape index (ASI) was being employed to minimize this effect. However, when three out of four local power density prc-trips on the reactor protection system actuated, thc nuclear plant supervisor (NPS) instructed the reactor control operators to manually trip the Unit. A manual reactor trip was initiated, but the turbine did not trip. Several unsuccessful attempts were then made to trip the turbine using the turbine trip push button. The nuclear watch engineer was then dispatched and tripped the turbine locally from the turbine front standard.

III. EVENT SEQUENCE St. Lucie Unit 2 was performing a scheduled reactor shutdown for refueling purposes. During the plant shutdown ASI approached its upper (negative) limit.

Coordination at controlling AS I within limits and maintaining reactor coolant temperature due to Xenon build up became increasingly difficult.. When reactor power was decreased to about 12%, control rod insertion no longei had any effect on making ASI less negative and three of the four reactor protection system local power density pre-trips alarmed. At 0238, the NPS instructed the reactor control operators to manually trip the unit and the manual reactor trip pushbuttons on RTGB (reactor turbine generator board) 204 and RTGB . 201 were depressed. Two seconds later the turbine trip pushbutton on RTGB 201 was depressed; however, thc turbine did not trip. Standard post trip actions from Emergency Operating Procedure (EOP) 1 were i m pl emcn ted.

When the turbine trip pushbutton was depressed, the turbine governor valves indicated closed but the throttle valves, reheat valves, and intercept valves indicated open. Auto stop oil and the emergency trip header indicated pressures of approximately 100 psi and 2000 psi, respectively. The megawatt recorder indicated ninety megawatts.

"

Several additional attempts were made to trip thc turbine using the pushbutton, but each was unsuccessful. At this point, the reactor control operator opened generator output breakers 8W52 and 8W49, secured the voltage regulator, stopped thc digital electro-hydraulic (DEH) pumps, closed the main steam isolation valves (MSIV), and broke condenser vacuum.

IHE 8 92-023, 5/14/92 1 of 5

I I I. EVENT SEQUENCE (cont'd)

The assistant nuclear plant supervisor instructed the NWE to trip the turbine from the Front Standard. At 0241 all the turbine valves closed.

EOP-2, Reactor Trip Recovery, was then carried out, two sets of satisfactory safety function status checks werc completed, the plant was stabilized, and the Turbine Shutdown Procedure was re-entered.

I V. EVENT EVALUATION A)

With the exception of the failure of the turbine to trip, the plant response to this event was as expected.

Upon initiation of the reactor trip signal, the following should have occurred. The four control element drive mechanism (CEDM) bus undervoltagc relays should dc-energize. This should energize (open) the 20 ET solenoid valve and drain the electro-hydraulic fluid from the emergency trip header ( See attachment ¹I). The 20 AST trip solenoid coil should simultaneously energize, pull down the mechanical trip lever, and dump the auto stop oil header. Decreasing auto stop oil pressure allows the interface valve to open and dump the electro-hydraulic fluid emergency trip header.

When auto stop oil prcssure decreases to 45 psi, pressure switches 63-1 AST and 63-2 AST should lock in, actuating a generator primary lockout which trips thc generator output breakers. The same result should be obtained when attempting to trip the turbine via the turbine trip pushbutton.

The overspeed protection system (OPC) did not and should not have actuated because it was not challenged. The maximum turbine rpm observed during this event was 1850 rpm. This is below the OPC setpoint of 1854 rpm.

A multi-discipline team is investigating to determine why the turbine failed to trip when the trip signals were initiated. Refer to Figures 1, 2, & 3 for more details.

B) FETY I IFf The plant response to this event is bounded by section 15.1.5 of the St. Lucie Unit 2 Final Safety Analysis Report (FSAR), "Increased Heat Removal by the Secondary System" as further described below..

Analysis of three scenarios similar to but with different parameters than the actual event were conducted. In each case the events were within the plant's design basis and arc summarized below.

1. 12% Power Trip, No Operator Action Scenario.

• Automatic MSIV closure at 600 psig. Cooldown terminated.

• Cooldown rate bounded by limiting FSAR cooldown event.

IHE ¹92-023, 5/14/92 2 of 5

I V. EVENT EVALUATION (cont'd)

B) NIFI

2. 100% Power Tr'ip, No Operator Action Scenario.

• Possible Safety Injection Actuation Signal (SIAS) but no actual injection.

• Automatic MSIV closure at 600 psig. Cooldown terminated.

~ Cooldown rate is bounded by the FSAR cooldown event as confirmed by in-house RETRAN analysis.

3. 100% Power Trip with FSAR Chapter 15 Accident Scenario.

• FSAR Chapter 15 Analysis considers single failure of turbine stop valves to

~

close.

Operators trained on the simulator for design basis accidents. Multiple failures are simulated and procedurally addressed by EOPs. These include "Excess Steam Demand".

C) ALY I LA A multi-discipline investigation team is performing a root cause analysis of this event. Members of this group include System Engineers, and representatives from Operations, Maintenance, Engineering, and the Turbine vendor. Thc redundant design of thc turbine trip scheme suggests that at least two failures werc necessary to explain the inability to trip observed during the 04/21/92 event. Accordingly the failure analysis plan is evaluating both the 20 ET and the 20 AST turbine trips.

Extensive turbine vendor input was obtained for the failure analysis plan. Particular attention was paid to the lessons lcarncd in the Salem overspeed event. A Letter of Instruction, 2-LOI-T-67, "Evaluation of Turbine Solenoid Trip Function," was written to evaluate the operation of all trip functions. Attachment 1 is the failure analysis.

8QQILJl5-The failure of the turbine to trip was caused by unrelated failures in the redundant and electrically independent turbine tripping schemes. (more details later.....)

Turbine trip 20 AST failed due to an intermittent loss of circuit continuity. Either of two deficiencies noted at relay 62 ASTX (item ¹6 of attachment 1 "Failure Analysis Plan" ) could cause such a failure. (more details later.....)

Results of thc 20 ET solenoid valve autopsy. (more details later.....)

Because these two unrelated failures could not be detected with thc existing system design and surveillance program, a total loss of trip capability resulted. (more details later.....)

IHE ¹92-023, 5/14/92 3 of 5

V. CORRECTIVE ACTIONS A) R A IVIT I

1. The original 20 ET, 20 OPC-1 and 20 OPC-2 Solenoid valves have been removed and were sent to an independent laboratory for analysis. Results are due back on 05/18/92

2. I&C is continuing to investigate thc root cause of the intermittent electrical problems of the 20 AST trip solenoid under PWO 7356. The circuits of the 20 ET and 20 OPC 1 and 2 solenoids are also being evaluated for similar problems

3. I&C will evaluate providing coil monitoring lights for 20 AST and 20 ET to indicate continuity of the field cable and solenoid coil.

4. Operations is evaluating additional guidance for EOP-1, "Standard Post Trip Actions", on actions to take for a reactor trip with no subsequent turbine trip.

5. Engineering will evaluate providing a latch-in circuit for the 20 ET solenoid valve. The latch-in will bc provided only across the PB710 contacts.

6. Engineering will evaluate providing a 30 second time delay between a turbine trip and a generator trip to activate the reverse power relays.

7. Engineering will evaluate providing the capability to exercise thc 20 AST solenoid valve while on line.

8. Operations will evaluate modifying their procedures to allow individual trip feature testing.

9. PCM 032-292M "Turbine Lube Oil Conditioner Connections" will install valves and fittings to allow connection of an up-graded lube oil conditioner.

10. I&C will disassemble and inspect the trip block under PWO 7356.

11. Engineering will evaluate providing improved 20 AST coil suppression to limit arcing across 62 ASTX relay contacts.

12. Nuclear Fuels is evaluating additional guidance for ASI control at the end of core life.

B) MA IVITI

13. Engineering will evaluate enhancements to the filtration system for the turbine EH fluid.

14. Engineering will evaluate the use of a stainless steel solenoid valve body to replace the existing carbon steel body.

IHE ¹92-023, 5/14/92 4of 5

V. CORRECTIVE ACTIONS (cont'd)

B) A I

15. Engineering will evaluate use of an alternative DEH fluid to reduce material incompatibilities.

16. Engineering will evaluate providing an additional 20 AST valve of diverse manufacture which can be tested on line by the use of the manual trip buttons.

17. Engineering will evaluate providing a means of testing the 20 ET solenoid valve while on line.

IHE ¹92-023, 5/14/92 5of5

Attachment FAILURE ANALYSIS PLAN of2 rfr 1 1 P F T FTII F

1. 20 ET (valve) -sticking, binding - removed from system in -results duc 05/18/92 "as found" condition to preserve any cvidencc of failure mode; sent to an independent lab.

2. 20 ET (circuit) -loose wire -continuity check -satisfactory / PWO 7356

-open coil -resistance check -no open / 2-LOI-T-67

-improper supply voltage ~

-voltage measured and -acceptable / 2-LOI-T-67 recorded

3. 20 AST (valve) -sticking, binding -valve operated during test -no indication of sticking or binding / 2-LOI-T-67

4. 20 AST (Trip Block) orifice 'plugged

-disassemble and inspect -scheduled 5/12/92

5. 20 AST (circuit) -loose wire -continuity check -loose terminal connection found at position ¹6 on 63 ASTX / PWO 7356

-open coil -resistance check -no open / 2-LOI-T-67

-improper voltage supply -voltage measured and -acceptable / 2-LOI-T-67 recorded

6. 62 ASTX (Relay) -set point drift -digital recorder -set point verified / 2-LOI-T-67

-loose wire -continuity check -loose terminal connection at position ¹6 / 2-LOI-T-67

• Potential Failure Cause

-degraded contacts -visual examination -slight arcing on both sets of contacts with burn residue evident

• Potential Failure Cause

Attachment 1 FAIL RE ANALY I PLAN 2of2

7. 20-1 OPC (valve) -sticking, binding -valve operated during test -no indication of sticking, binding / 2-LOI-T-67, valve sent to an independent lab, results duc 05/18/92

8. 20-1 OPC (circuit) -loose wire -continuity check -no opens / PWO 7356

9. 20-2 OPC (valve) -sticking, binding -valve sent to an -results due 05/18/92 independent lab

10. 20-2 OPC (circuit) -loose wire -continuity check -no opens / PWO 7356

11. FCV-22-148 (interface -sticking, binding -valve operated during test -no indication of sticking, binding / 2-LOI-T-67 valve)

-contacts -digital recorder and -acceptable / 2-LOI-T-67

12. 63-1 AST (pressure visually switch) -contacts -digital recorder and- -acceptable / 2-LOI-T-67 v isual ly

13. 63-2 AST (pressure

-contacts -switch operated during test -acceptable / 2-LOI-T-67 switch) I'continuity

-contact alignment checks, -top contacts required more

14. PB 710 disassembly, and inspection movement than bottom contacts, switch replaced /

PWO 7356

- Not a Cause of the Failure

-contamination -water and particulate -in specification / Predictive Maintenance periodic testing program

15. EFI fluid

-contamination -water and particulate -in specification / Predictive Maintenance periodic testing program

16. Turbine lube oil PL 1 of1 EEL LIEK

1. Manual Reactor trip due to a -Ineffective ASI control method -Improved ASI control procedure loss of ASI control

2. Loose lead at 62 ASTX -Lack of means to detect a loss of -Design changes to add circuit continuity monitoring lights

-Design changes to allow periodic on-line testing of both 20 AST and 20 ET

3. Burnt contacts on 62 ASTX -Inadequate arc suppression -Design change to upgrade arc suppression in 20 AST circuit (MORE LATER......

}IESTINGHOUSE ELECTRO-HYDRAULIC CONTROL SYSTEH AND AUTO-STOP OIL SYSTEtl EMERGENCY TRIPS

1. LOW AUTO-STOP OIL PRESSURE (2/3) aaettt c

2. REACTOR TRIP (0)

~ ee ee ~ atvttw 3. HANUAL TRIP

~ p+AApwo oa I

I 4. ttl-HI LEVEL lft ANY S/G SOLENOID TRIPS I

I REACTOR TRIP (A)

I-I

2. ELECTRICAL OVERSPEED I 3. LOH EH FLUID PRESSURE

~ wet we ~ a taatIW 3

I 4. flANUAL eel catv as I 5. ETC.

C I I eeccva

'f tie I

I I

I AUTO STOP OIL PRESSURE SKI TCHES ~ '~e ~ 20 AsT cecal ~ a toOtIW I

I I

~ aie eeeea ~ ~talll tag caaaea SL

~ electee M I I I (ll I I I

~ eee ~

~ el el

~ tt et tV Ia l a taa tlW I

I I

I I tee et ea tIW I

~e

~

~ 1 ~e

~ at~tv

~ I I wtfateca aeaac

~ tc 2o-i 0Pc la

~ aae aeeeeee fa Bo-2 opc QrtaftV OVERSPEED PROTECTION CONTROLLER

, Etf FLIIIOSTSfitt 103K OF NOR)lAL SPEED AUfO SfOPOILSYSflN

2. GENERATOR OUTPUT BREAKERS OPEN ABOVE A PRESET POMER LEVEL

3. CHANGE IN GENERATOR OUTPUT AND TURBINE INLET PRESSURE

TURBINE TRIPS I AUTO I STEAI4 I ETH DEH DC

<~ 'HOOP TAILURE i jsuez Z W4~ FZ-oag Manual ) GDIERATOR OVERSPEEO RECTOR [

SIOP ASTO) ~ ~ lOCKOUT OIL [ s.EASEL IIHP 2 25 r

0 6 6 TD I -4 I I ~ 63-1 I 63K IL-7IO X-(

%5 EHlXH (5ILlll) I AST 2

6 6 O 7.

PB 710.

2 86

~P '2C 2 86 GB AT R 0 IN- I t SH. ~. SH..

885 ~ d II 14L 5 74 R 4'r 3 II 74DITX Tppo b

63, 12 TDDO 0 6 14GADOO 85 SEC 6 25SEC 6 0.25 SEC.

4r DCF 0 63 t3 63 g E3 74 C 62A O ~ asses

~ 25SEC LV AST 0

osATa R (TYP) ~su -

4 cg-e, R lo9 109 A5T V 3 62ASTX III TPO IDgEC ISA.Lh&P3 234 is% 2SA ISC 23C Is'5 '23 Bs

.z LI 62ASTX ISC 2SC' A 0 00 IOSB:

In j

4 0 irv RB5 l5OA 2OOW ISP 23P SET AT I25Tl.

3 CSC . 1 CSB cs-p 5 6 0 4 Tie II los 4 d 6 6 d 6 LI d 6 74 gO 63X 74 S

74IBO S

74LV S

74TB EASTX R AST .. I3 ~

710 1IX 14 S<g 6 OS 6 r 6 6 TDPLI 6 B 5 H.TI I CWP L2 I4 4il5,6s20 883

~

ASTH .;~

'j rvvzasra+ vrll<'r AUTO STOP esapeerl Low Br8 Loss'tuust Q Slop Soleno lsI TR1P FAII.UIIE SCRAP Tdp Tslp Knes8

'C

~"

~ r ~

2 '-'."':2 ",r 2 '.I 1 1 I 6 'I dj" "'"'lS 6 6

~.."- rs st l8 74 7 74 7 rs rt P4R 63T iy63Y';,'.-"...

Cw0109 LV / EHTI( 7 10 R '~g'5 BO TB 6 ~sits I2 2

~TT""'"-'g'>>'vrs.rmrsrqvrvcsvrsrrrrrv6e.!l, rvss.rsv-svv~"-srrvrms 5

"'4'> ';.',t,"4>>....".;". (:I ":

i"

.i",'i(q "r'"."rt$ p ~+8OTESFOIC C5 70>+ PEVELOpQRI4Tg 5Q~R CV47 7~ la IEIeLATlaP jQL-tg+6Ip 8624'-4 IIl"k..t.: I . ..ss4i s 1

~

EbASCO S INCORPORA FLORIDA POWER I 485

'f5

@'f Cf I 63T V5 .V 6% 83f g',, '

r

'jul V VIA VI44 V>>s" '.

r 4

N ST. LUCE PLANT-EX

..'; (,.'SGHEMATll;Di,

.W$ Ilr 6;:e a Io .. v.Z-..q. OFQ:s'rI 1: IO-203 APPROV fP , ~

TURBIHE AUTO. STOP TNI e~r 'rrrr REV PATE APPROVEP,

~ CWP . Tr'-SNi al e p- - ~r "

~ ~

LTIr..i.-s; I;Ie>>r: y:pg:.g!"'.p gal@~rsrni~na)so-'gQ-les 9 re '8M)Wy~Aj~~~ rl,"'j~" P4'sr~'"g+~+~jRki

~>

Rigueur 3 Wh'> Pz=cvz~S T5- IO-ISAI TS- IO.I3A2 ~ %5-IO.I5A5 /

II I ET TK-O. IQl Q'

T6-I-13A2 0

d.o 0 0-I Q-2 C3-2 LIT-2 uv-4 js %2It K AAT lo-I5)I T5-e-IM2 ~

Nf tsINI; 7 p II IIV-I 'AV.3 8 IIC II t I 0 0 Q 74-2/T to Ta-0-ISSI Tf 1382 ~ IO-IMI) d t jr 74 74 II 3 -?I

!RH c5RlTk 7e Iee.Cve1O q t tqjt I i 2

r. rite Ptr "tr. );. .-'ItYjgiftg~OIH~ . I t t jR IllR III L I C

1% B BAt IS Akk'D P

IIB;I

%Nil '.

0 o o 5 *t Bg pg

~0 0 -I 20-2 0 f g(~m g, 7

5H.

E,T 0 PC OPC A

E7 EX T IIOOO TEMP. SWITCIIKS t

0 45I' T5-IO-I3IIabt.BI r a e ru B-ZZ~5 ftc-ZZ tII II I MI III III 111 10 MI~91O~III 9 Io 9 Q 9 0 0 63/G$ .6 63/IBS-5 63,a~.h I ' t I I;.Qg 0 0 d d '.

I >III 29-2t11 .9-31'lO 9-4(ie 63 63 63 74 ~ 74 33 4 tr

~

N I tti 8 2 5 iVIttIG 3 N/IIO 'LL7 LV TQ t."ttt>TI<

t RPB CtB.B RPB trB C RPm~9 Ic to 16 l7 23 r Wir C It'QC. 71:) { Tll I J ht'ti Q rtt~ VJc U-t'N:tt)-ZS4- I4n-mO79-28,3 Re- Fun-Z.u<=268- NUCLEAR SAFETY RELATEO

~OI

-%~5  ;~

Att0018%

7 il ll. 3 tBIV C DR ~

ESA5CO SERVlCKS INCORPORATED NEW YOIIK

~AP .r/ I '

FLORtDA POWER 81 LlGHT CO SK UJCtE PLANT-BPENStON4NlT 2 SCHEMATlC DIAGRAM EMfRQa "Qhtttf TRIP 4 UII+INfAI.ltRMS

$ OAAkrB.t

I 4