ML17227A424

Special Rept:On 920421,turbine Generator Failed to Trip Following Manual Reactor Trip.Caused by Failures in Redundant & Electrically Independent Turbine Tripping Schemes.I&C Disassembled & Trip Block Inspected

ML17227A424
Person / Time
Site:	Saint Lucie
Issue date:	05/14/1992
From:	Sager D FLORIDA POWER & LIGHT CO.
To:	NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
L-92-149, NUDOCS 9205180236
Download: ML17227A424 (15)

Text

ACCELERATED DISTRIBUTION DEMONSTTION SYSTEM REGULATORY INFORMATION DISTRIBUTION SYSTEM (RIDS)ACCESSION NBR:9205180236 DOC.DATE: 92/05/14 NOTARIZED:

NO FACIL:50-389 St.Lucie Plant, Unit 2, Florida Power&Light Co.AUTH.NAME.AUTHOR AFFILIATION SAGER,D.A.

Florida Power&Light Co.RECIP.NAME RECIPIENT AFFILIATION Document Control Branch (Document Control Desk)

SUBJECT:

Special rept:on 920421,turbine generator failed to trip following manual reactor trip.Caused by failures in redundant&electrically independent turbine tripping schemes.I&C disassembled

&trxp block inspected.

DISTRIBUTION CODE: IE22D COPIES RECEIVED:LTR ENCL SIZE: TITLE: 50.73/50.9 Licensee Event Report (LER), Incident Rpt, etc.NOTES DOCKET 05000389 RECIPIENT ID CODE/NAME PD2-2 LA NORRIS,J INTERNAL: ACNW AEOD/DOA AEOD/ROAB/DS P NRR/DLPQ/LHFB10 NRR/DOEA/OEAB NRR/DST/SELB 8D RR/LB8D1 REG FIL 02 RGN2 FILE 01 EXTERNAL: EG&G BRYCE,J.H NRC PDR NSIC POORE,W.COPIES LTTR ENCL 1 1 1 1 2 2 1-1 2 2 1 1 1 1 1 1 1 1 1 1 1 1 3 3.1 1 1 1 RECIPIENT'ID CODE/NAME PD2-2 PD ACRS AEOD/DS P/TPAB NRR/DET/EMEB 7E NRR/DLPQ/LPEB10 NRR/DREP/PRPB11 NRR/DST/SICB8H3 NRR/DST/SRXB 8E RES/DSIR/EIB L ST LOBBY WARD NSIC MURPHY,G.A NUDOCS FULL TXT COPIES LTTR ENCL 1 1 2 2 1 1 1 1 1 1 2 2 1 1 1 1 1 1 1 1 1 1 1 1 NOTE TO ALL"RIDS" RECIPIENTS:

PLEASE HELP US TO REDUCE V'ASTE!CONTACT THE DOCUMENT CONTROL DESK, ROOiil Pl-37 (EXT.20079)TO ELlilllNATE YOUR NN!E FROivI DISTRIBUTION LISTS FOR DOCUMENTS YOU DON T NEED!TOTAL NUMBER OF COPIES REQUIRED: LTTR 32 ENCL-32

'P.O.Box 128, Ft.Pierce, FL 34854-0128 FPL May 14, 1992 L-92-149 U.S.Nuclear Regulatory Commission Attn: Document Control Desk Washington, D.C.20555 Re: St.Lucie Unit 2 Docket No.50-389 Event Date: April 21, 1992 Turbine Tri Failure U date Attached please find the updated version of a plant problem report which discusses an event which occured on April 21, 1992.Included is the current status of our analysis of the event.Pursuant to 10 CFR 50.73 a Licensee Event Report will be submitted within 30 days of the event date.Very truly yours, D.A.S Vice Pr St.L ger sident e Plant DAS/JWH/kw Attachment cc: Stewart D.Ebneter, Regional Administrator, USNRC Region II Senior Resident Inspector, USNRC, St.Lucie Plant DAS/PSL 1697-92 9205180236 9205i4 PDR ADOCK 05000389 8 PDR an FPL Group company ST.LUCIE PLANT IN HOUSE EVENT REPORT 92-023 REV.9 TURBINE GENERATOR FAILED TO TRIP FOLLOWING MANUAL REACTOR TRIP UNIT'DATE: APRIL 21, 1992 I.INITIAL PLANT CONDITIONS On April 21, 1992, St.Lucic Unit 2 was shutting down for a scheduled refueling outage.Reactor power was at approximately 12%.I I.EVENT DESCRIPTION A scheduled reactor-turbine shutdown was in progress on Unit 2.During the shutdown, power moved to the top of the reactor core.Existing procedural guidance on axial shape index (ASI)was being employed to minimize this effect.However, when three out of four local power density prc-trips on the reactor protection system actuated, thc nuclear plant supervisor (NPS)instructed the reactor control operators to manually trip the Unit.A manual reactor trip was initiated, but the turbine did not trip.Several unsuccessful attempts were then made to trip the turbine using the turbine trip push button.The nuclear watch engineer was then dispatched and tripped the turbine locally from the turbine front standard.III.EVENT SEQUENCE St.Lucie Unit 2 was performing a scheduled reactor shutdown for refueling purposes.During the plant shutdown ASI approached its upper (negative) limit.Coordination at controlling AS I within limits and maintaining reactor coolant temperature due to Xenon build up became increasingly difficult..

When reactor power was decreased to about 12%, control rod insertion no longei had any effect on making ASI less negative and three of the four reactor protection system local power density pre-trips alarmed.At 0238, the NPS instructed the reactor control operators to manually trip the unit and the manual reactor trip pushbuttons on RTGB (reactor turbine generator board)204 and RTGB.201 were depressed.

Two seconds later the turbine trip pushbutton on RTGB 201 was depressed; however, thc turbine did not trip.Standard post trip actions from Emergency Operating Procedure (EOP)1 were i m pl emcn ted.When the turbine trip pushbutton was depressed, the turbine governor valves indicated closed but the throttle valves, reheat valves, and intercept valves indicated open.Auto stop oil and the emergency trip header indicated pressures of approximately 100 psi and 2000 psi, respectively.

The megawatt recorder indicated ninety megawatts.

Several" additional attempts were made to trip thc turbine using the pushbutton, but each was unsuccessful.

At this point, the reactor control operator opened generator output breakers 8W52 and 8W49, secured the voltage regulator, stopped thc digital electro-hydraulic (DEH)pumps, closed the main steam isolation valves (MSIV), and broke condenser vacuum.IHE 8 92-023, 5/14/92 1 of 5 I I I.EVENT SEQUENCE (cont'd)The assistant nuclear plant supervisor instructed the NWE to trip the turbine from the Front Standard.At 0241 all the turbine valves closed.EOP-2, Reactor Trip Recovery, was then carried out, two sets of satisfactory safety function status checks werc completed, the plant was stabilized, and the Turbine Shutdown Procedure was re-entered.

I V.EVENT EVALUATION A)With the exception of the failure of the turbine to trip, the plant response to this event was as expected.Upon initiation of the reactor trip signal, the following should have occurred.The four control element drive mechanism (CEDM)bus undervoltagc relays should dc-energize.This should energize (open)the 20 ET solenoid valve and drain the electro-hydraulic fluid from the emergency trip header (See attachment

¹I).The 20 AST trip solenoid coil should simultaneously energize, pull down the mechanical trip lever, and dump the auto stop oil header.Decreasing auto stop oil pressure allows the interface valve to open and dump the electro-hydraulic fluid emergency trip header.When auto stop oil prcssure decreases to 45 psi, pressure switches 63-1 AST and 63-2 AST should lock in, actuating a generator primary lockout which trips thc generator output breakers.The same result should be obtained when attempting to trip the turbine via the turbine trip pushbutton.

The overspeed protection system (OPC)did not and should not have actuated because it was not challenged.

The maximum turbine rpm observed during this event was 1850 rpm.This is below the OPC setpoint of 1854 rpm.A multi-discipline team is investigating to determine why the turbine failed to trip when the trip signals were initiated.

Refer to Figures 1, 2,&3 for more details.B)FETY I IFf The plant response to this event is bounded by section 15.1.5 of the St.Lucie Unit 2 Final Safety Analysis Report (FSAR),"Increased Heat Removal by the Secondary System" as further described below..Analysis of three scenarios similar to but with different parameters than the actual event were conducted.

In each case the events were within the plant's design basis and arc summarized below.1.12%Power Trip, No Operator Action Scenario.*Automatic MSIV closure at 600 psig.Cooldown terminated.

• Cooldown rate bounded by limiting FSAR cooldown event.IHE¹92-023, 5/14/92 2 of 5 I V.EVENT EVALUATION (cont'd)B)NIFI 2.100%Power Tr'ip, No Operator Action Scenario.*Possible Safety Injection Actuation Signal (SIAS)but no actual injection.

• Automatic MSIV closure at 600 psig.Cooldown terminated.

~Cooldown rate is bounded by the FSAR cooldown event as confirmed by in-house RETRAN analysis.3.100%Power Trip with FSAR Chapter 15 Accident Scenario.*FSAR Chapter 15 Analysis considers single failure of turbine stop valves to~close.Operators trained on the simulator for design basis accidents.

Multiple failures are simulated and procedurally addressed by EOPs.These include"Excess Steam Demand".C)ALY I LA A multi-discipline investigation team is performing a root cause analysis of this event.Members of this group include System Engineers, and representatives from Operations, Maintenance, Engineering, and the Turbine vendor.Thc redundant design of thc turbine trip scheme suggests that at least two failures werc necessary to explain the inability to trip observed during the 04/21/92 event.Accordingly the failure analysis plan is evaluating both the 20 ET and the 20 AST turbine trips.Extensive turbine vendor input was obtained for the failure analysis plan.Particular attention was paid to the lessons lcarncd in the Salem overspeed event.A Letter of Instruction, 2-LOI-T-67,"Evaluation of Turbine Solenoid Trip Function," was written to evaluate the operation of all trip functions.

Attachment 1 is the failure analysis.8QQILJl5-The failure of the turbine to trip was caused by unrelated failures in the redundant and electrically independent turbine tripping schemes.(more details later.....)

Turbine trip 20 AST failed due to an intermittent loss of circuit continuity.

Either of two deficiencies noted at relay 62 ASTX (item¹6 of attachment 1"Failure Analysis Plan")could cause such a failure.(more details later.....)

Results of thc 20 ET solenoid valve autopsy.(more details later.....)

Because these two unrelated failures could not be detected with thc existing system design and surveillance program, a total loss of trip capability resulted.(more details later.....)

IHE¹92-023, 5/14/92 3 of 5 V.CORRECTIVE

-ACTIONS A)R A IVIT I 1.The original 20 ET, 20 OPC-1 and 20 OPC-2 Solenoid valves have been removed and were sent to an independent laboratory for analysis.Results are due back on 05/18/92 2.I&C is continuing to investigate thc root cause of the intermittent electrical problems of the 20 AST trip solenoid under PWO 7356.The circuits of the 20 ET and 20 OPC 1 and 2 solenoids are also being evaluated for similar problems 3.I&C will evaluate providing coil monitoring lights for 20 AST and 20 ET to indicate continuity of the field cable and solenoid coil.4.Operations is evaluating additional guidance for EOP-1,"Standard Post Trip Actions", on actions to take for a reactor trip with no subsequent turbine trip.5.Engineering will evaluate providing a latch-in circuit for the 20 ET solenoid valve.The latch-in will bc provided only across the PB710 contacts.6.Engineering will evaluate providing a 30 second time delay between a turbine trip and a generator trip to activate the reverse power relays.7.Engineering will evaluate providing the capability to exercise thc 20 AST solenoid valve while on line.8.Operations will evaluate modifying their procedures to allow individual trip feature testing.9.PCM 032-292M"Turbine Lube Oil Conditioner Connections" will install valves and fittings to allow connection of an up-graded lube oil conditioner.

10.I&C will disassemble and inspect the trip block under PWO 7356.11.Engineering will evaluate providing improved 20 AST coil suppression to limit arcing across 62 ASTX relay contacts.12.Nuclear Fuels is evaluating additional guidance for ASI control at the end of core life.B)M A IVITI 13.Engineering will evaluate enhancements to the filtration system for the turbine EH fluid.14.Engineering will evaluate the use of a stainless steel solenoid valve body to replace the existing carbon steel body.IHE¹92-023, 5/14/92 4of 5 V.CORRECTIVE ACTIONS (cont'd)B)A I 15.Engineering will evaluate use of an alternative DEH fluid to reduce material incompatibilities.

16.Engineering will evaluate providing an additional 20 AST valve of diverse manufacture which can be tested on line by the use of the manual trip buttons.17.Engineering will evaluate providing a means of testing the 20 ET solenoid valve while on line.IHE¹92-023, 5/14/92 5of5 Attachment 1 P F T FAILURE ANALYSIS PLAN FTII 1 of2 F rfr 1.20 ET (valve)-sticking, binding-removed from system in"as found" condition to preserve any cvidencc of failure mode;sent to an independent lab.-results duc 05/18/92 2.20 ET (circuit)-loose wire-open coil-improper supply voltage-continuity check-resistance check~-voltage measured and recorded-satisfactory

/PWO 7356-no open/2-LOI-T-67-acceptable

/2-LOI-T-67 3.20 AST (valve)4.20 AST (Trip Block)5.20 AST (circuit)-sticking, binding'plugged orifice-loose wire-valve operated during test-disassemble and inspect-continuity check-no indication of sticking or binding/2-LOI-T-67-scheduled 5/12/92-loose terminal connection found at position¹6 on 63 ASTX/PWO 7356-open coil-improper voltage supply-resistance check-voltage measured and recorded-no open/2-LOI-T-67-acceptable

/2-LOI-T-67 6.62 ASTX (Relay)-set point drift-digital recorder-set point verified/2-LOI-T-67-loose wire-continuity check-loose terminal connection at position¹6/2-LOI-T-67

• Potential Failure Cause-degraded contacts-visual examination-slight arcing on both sets of contacts with burn residue evident*Potential Failure Cause Attachment 1 FAIL RE ANALY I PLAN 2of2 7.20-1 OPC (valve)-sticking, binding-valve operated during test-no indication of sticking, binding/2-LOI-T-67, valve sent to an independent lab, results duc 05/18/92 8.20-1 OPC (circuit)9.20-2 OPC (valve)-loose wire-sticking, binding-continuity check-valve sent to an independent lab-no opens/PWO 7356-results due 05/18/92 10.20-2 OPC (circuit)11.FCV-22-148 (interface-loose wire-sticking, binding-continuity check-valve operated during test-no opens/PWO 7356-no indication of sticking, binding/2-LOI-T-67 valve)12.63-1 AST (pressure switch)13.63-2 AST (pressure switch)14.PB 710-contacts-contacts-contacts-contact alignment-digital recorder and visually-acceptable

/2-LOI-T-67-digital recorder and-v isual ly-acceptable

/2-LOI-T-67 I'continuity checks, disassembly, and inspection-top contacts required more movement than bottom contacts, switch replaced/PWO 7356-switch operated during test-acceptable

/2-LOI-T-67

-Not a Cause of the Failure 15.EFI fluid 16.Turbine lube oil-contamination-contamination-water and particulate-water and particulate-in specification

/Predictive Maintenance periodic testing program-in specification

/Predictive Maintenance periodic testing program Attachment 2 EEL LIEK PL 1 of1 1.Manual Reactor trip due to a loss of ASI control-Ineffective ASI control method-Improved ASI control procedure 2.Loose lead at 62 ASTX-Lack of means to detect a loss of continuity-Design changes to add circuit monitoring lights-Design changes to allow periodic on-line testing of both 20 AST and 20 ET 3.Burnt contacts on 62 ASTX-Inadequate arc suppression-Design change to upgrade arc suppression in 20 AST circuit (MORE LATER......

}IESTINGHOUSE ELECTRO-HYDRAULIC CONTROL SYSTEH AND AUTO-STOP OIL SYSTEtl aaettt c~ee ee~p+AApwo oa~wet we eel catv as C cecal~electee M (ll~eee~~el el~tt et tV Ia~atvttw I I I I I-I I I~a taatIW 3 I I I I I I I I I~a toOtIW I I I I I I I I l a taa tlW I I I I tee et~ea tIW I~1~e~e~I I~tc SOLENOID TRIPS REACTOR TRIP (A)2.ELECTRICAL OVERSPEED 3.LOH EH FLUID PRESSURE 4.flANUAL 5.ETC.eeccva'f tie AUTO STOP OIL PRESSURE SKI TCHES~'~e~~aie eeeea~20 AsT caaaea SL~talll tag~at~tv wtf ateca aeaac EMERGENCY TRIPS 1.LOW AUTO-STOP OIL PRESSURE (2/3)2.REACTOR TRIP (0)3.HANUAL TRIP 4.ttl-HI LEVEL lft ANY S/G 2o-i 0Pc Bo-2 opc la~aae QrtaftV aeeeeee fa , Etf FLIIIOSTSfitt OVERSPEED PROTECTION CONTROLLER 103K OF NOR)lAL SPEED 2.GENERATOR OUTPUT BREAKERS OPEN ABOVE A PRESET POMER LEVEL 3.CHANGE IN GENERATOR OUTPUT AND TURBINE INLET PRESSURE AUfO SfOPOILSYSflN ASTO)Manual)GDIERATOR~~lOCKOUT TURBINE TRIPS OVERSPEEO I AUTO I STEAI4 I ETH DEH DC RECTOR[SIOP[<~'HOOP TAILURE OIL s.EASEL IIHP i jsuez Z W4~FZ-oag 0 O V III.z LI 0 In 2 6 IL-7IO 6 PB 2 710.25 TD I%5 6 86 2~P'2C SH.~.86 GB SH..885~14GADOO 6 85 SEC O R AT 5 6 r-4 I 0 IN-I d 74 R4'r 3 II~25SEC II 14L 12 Tppo 6 0.25 SEC.I EHlXH 7.(5ILlll)I t 74DITX TDDO DCF 4r asses~25SEC~su-4 cg-e, lo9 3 109 ISC 23C Is'5'23 Bs 0 RB5 l5OA 2OOW SET AT I25Tl.4 irv j ISP 23P osATa R (TYP)62ASTX TPO IDgEC ISA.Lh&P3 234 is%2SA 62ASTX ISC 2SC'A 00 IOSB: 0~63-1 I AST 2 6 63K 0 R A5T 0 63 t3 63 LV g E3 b 63, 74 C 62A AST X-(5 6 74 710 5 H.TI I ASTH 0 gO AST 3 4..I3 I4 CSC.1 CSB Tie II los 1IX CWP 4il5,6s20~.;~cs-p 4 14 d 63X S<g 6 883 LI 6 S 6 S d S 6 d 74~74IBO 74LV 74TB EASTX OS 6 r 6 6 L2 TDPLI 6 R 6 B'j rvvzasra+vrll<Žr Q'C AUTO STOP esapeerl Low Br8 Loss'tuust Tdp Tslp Slop TR1P FAII.UIIE Soleno lsI Knes8~" 2~r~iy';,'.-"...

~.."-63T 63Y Cw0109 I2 2'-'."':2 6'I LV/1 rs rt'.I 74 7 BO 1 6 74 7 TB 6 I 6 P4R",r 2 dj" l8 10"'"'lS EHTI(7~sits rsst R'~g'5 5la IEIeLATlaP i" jQL-tg+6Ip 8624'-4 IIl" k..t.:.I EbASCO S 1~IN CORPORA FLORIDA POWER I ST.LUCE PLANT-EX..';(,.'SGHEMATll;Di, TURBIHE AUTO.STOP TNI 485'f5 Cf I V5.V r'jul@'f 63T 6%83f g',,'.W$Ilr 6;:e a Io..v.Z-..q.r V VIA VI44 V>>s"'.OFQ:s'rI~CWP.Tr'-SNi 4 1: IO-203 ,~REV PATE APPROVEP, APPROV fP~e~r'rrrr~al e p--~r" LTIr..i.-s; I;Ie>>r: y:pg:.g!"'.p gal@~rsrni~na)so-'gQ-les 9 re'8M)W y~Aj~~~rl,"'j~" P4'sr~'"g+~+~jRki

~TT""'"-'g'>>'vrs.rmrsrqvrvcsvrsrrrrrv6e.!l, rvss.rsv-svv~"-srrvrms

"'4'>';.',t,"4>>....".;".

(:I":.i",'i (q"r'"."r t$p~+8OTESFOIC C5 70>+SCRAP PEVELOpQRI4Tg 5Q~R CV47 7~..ss4i s N

~>

T5-IO-ISAI TS-IO.I3A2~II I%5-IO.I5A5 Rigueur 3 Wh'>Pz=cvz~S/ET Q TK-O.IQl'lo-I5)I 0 T6-I-13A2 T5-e-IM2 II t I 7 8 js p II IIC d.o 0-I Q-2%2It K 0 C3-2 AAT LIT-2 IIV-I uv-4~'AV.3 Nf tsINI;0 Ta-0-ISSI tqjt I i r.rite Ptr"tr.);.IIB;I g(~m 0 Tf-6-1382 Ill R III 1%B BAt%Nil'.Q 74-2/T to~IO-IMI).-'ItY jgiftg~OIH~.IS 0~0 g, 7 E,T 5H.d 74 74 II 3-?I 7e o o 0-I 20-2 0 PC OPC 5 0 A E7!RH c5RlTk Iee.Cve1O q 2 t I t L I t jr t jR C Akk'D P*t Bg pg f B-ZZ~5 t 0 ftc-ZZ 45I'EX T IIOOO TEMP.SWITCIIKS T5-IO-I3IIabt.BI r a e ru tII II I MI III III 111 10 MI~91O~III 9 Io 9 Q 9 0 0 63/G$.6 63/IBS-5 63,a~.h I>III 29-2t11 N I tti 8 I'.9-31'lO 2 5 iVI ttIG t I I;.Qg 9-4(ie 3 N/IIO 0 0 63 63 63'LL7 LV TQ d d 74~74'.33 4 t."ttt>TI<

tr~t RPB CtB.B RPB trB C RPm~9 I J U-t'N:tt)-ZS4-I4n-mO79-28,3 C Ic to 16 l7 Re-Fun-Z.u<=268-23 It'QC.r Wir 71:){Tll ht'ti Q rtt~VJc NUCLEAR SAFETY RELATEO~OI-%~5$OAAkrB.t;~Att0018%7 il ll.3 ESA5CO SERVlCKS INCORPORATED NEW YOIIK tBIV DR~.r/I'C~AP FLORtDA POWER 81 LlGHT CO SK UJCtE PLANT-BPENStON4NlT 2 SCHEMATlC DIAGRAM EMf RQa"Qhtttf TRIP 4 UII+INf AI.ltRMS I 4