ML17304B037
| ML17304B037 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 02/06/1989 |
| From: | Haynes J, Shriver T ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR |
| To: | NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM) |
| References | |
| REF-PT21-89, REF-PT21-89-008-000 192-00449-JGH-T, 192-449-JGH-T, LER-89-004, LER-89-4, PT21-89-008-000, PT21-89-8, NUDOCS 8902240282 | |
| Download: ML17304B037 (28) | |
Text
ACCELERATED Dl +6BUTl 05 DEMON IST>I1 0% SYSTEM REGULATORY INFORMATION DISTRIBUTION, SYSTEM (RIDS)
ACCESSION NBR:8902240282 DOC.DATE: 89/02/06 NOTARIZED: NO DOCKET I FACIL:STN-50-530 Palo Verde Nuclear Station, Unit 3, Arizona Publi 05000530 AUTH. NAME AUTHOR AFFILIATION SHRIVER,T.D. Arizona Nuclear Power Project (formerly Arizona Public Serv HAYNES,J.G. Arizona Nuclear Power Project (formerly Arizona Public Serv RECIP.NAME RECIPIENT AFFILIATION R
SUBJECT:
LER 89-004-01:on 890104,emergency diesel generator rocker arm failure/ESF actuation.
N/8 - ltr.
DISTRIBUTION CODE: IEISD COPIES RECEIVED:LTR TITLE: Part 21 Rept (50 DKT)
L ENCL ( SIZE:
NOTES:Standardized plant. 05000530/
RECIPIENT COPIES RECIPIENT COPIES ID CODE/NAME LTTR ENCL ID CODE/NAME LTTR ENCL PD5 LA 1 0 PD5 PD 1 1 DAVIS,M.J. 1 1 DAVIS,M 1 1 INTERNAL: AEOD/DOA 1 1 AEOD/DS P/TPAB 1 1 ARM TECH ADV 1 1 NRR CRUTCHFIELD 1 1 NRR VARGA,S 1 1 NRR/DEST/ADE 8H 1 1 NRR/DEST/ADS 7E NRR/DOEA/GCB NUDOCS-ABSTRACT ll 1 1
1 1
1 1
NRR/DLPQ/QAB 10 REG FILE VIB 9D 01 1
1 1
1 1
1 RES/DSIR/EIB 1 1 G 1 1 RGN2 1 1 RGN3 1 1 RGN4 1 1 RGN5 1 1 EXTERNAL: INPO RECORD CTR 1 1 LPDR 1 1 NRC'DR 1 1 NSIC SILVER,E 1 1 NOTES: 1 1 THts DOCUMENT HAS BEEN SCANNED NOIR Q) ALL ltRIDS>> RECIP~,
PLEASE HELP US 'IO REDUCE WASTEl (XyG'ACZ 'IHE DOCX3MENI'3ÃHKL DESK, ROOM Pl-37 (EZI'. 20079) IO EZJMCNMIR YOUR NAME PRCN DISTEKK7TION LISTS FOR DOCUMENIS YOU DON'T NEED)
TOTAL NUMBER OF COPIES REQUIRED: LTTR 27 ENCL 26
NRC Form 355 US. NUCLEAR REOULATOAY COMMISSION (94)3)
APPAOVED OMB NO. 31500104 LICENSEE EVENT REPORT {LERI EXPIRES: Sl31lSS FACILITY NAME (1) DOCKET NUMBER l2) PA E 3 TITLE Ici Palo Verde Unit 3 0 s 0 0 053 0sor13 Emer enc Diesel Generator Rocker Arm Failure/ESF Actuation EVENT DATE (5) LFR NUMBER ISI REPORT DATE IT) OTHER FACILITIES INVOLVED IS)
MONTH OAY YEAR YEAR >%+: SNOVNNTIAL 5 rp RDV>DION MONTH DAY YEAR FACILITYNAMES DOCKET NUMBER(S)
NUMBER r. o NVMDER N/A 0 5 0 0 0 0 1 0 489 8 9 0 0 4 0 1 0 2 0.68 9 N/A 0 5 0 0 0 THIS REPORT IS SUBMITTED PURSUANT 7 0 THE AEOUIREMENTS OF 10 CFR (It iChrch onr or morr Of the lollowlnel l11 OPERATINO MODE (DI 20A02(D) 20.405 Ic) X 50.73(ol(2)(ir) 73.71(D)
POWER 20.405(o)(1) (i) SOW(c)(I) X 50.73(o)(2)(r) 73.71(c)
(.EYE(.
1 0 0 20AOS(o) (1)(NI 50.35(c) (2) 50.73(o) (2)(rN) OTHER iSi>>clfy ln Aetarct Orrow enrf ln TrcL HiDC Form 20.405(o)(1)(NII 50.73(o) (2) li) 50.73(o) (2) (rNI) (A) 3$ 5A) 20AOS(o) (I)lir) 50.73lol(2)(N) 50.73(o) (2)(riN) (5) 10CFR21 &
20.405(el (I)lrl 50.73 lel(2) (IN) 50.73(ol(2)lc) Special Report LICENSEE CONTACT FOR THIS LER (12)
NAME TELEPHONE NVMBEA AREA CODE Timothy D. Shriver, Compliance Manager 6 02 39 3 2 521 COMPLETE ONE LINE FOR EACH COMPONENT FAILURE DESCRISEO IN THIS REPOAT (13I CAUSE SYSTEM COMPONENT MANUFAC REPORTABLE 5$ 8k~R43%,. CAUSE SYSTEM COMPONENT MANUFAC. El'ORTABLE )
TURER TO NPROS g(>@g~IP.>~ TVRER TO NPRDS IMih <. 5 >~gh B E K D G C 6 3 4
%$R45 SUPPL'EMENTAL REPORT EXPECTED (14) MONTH DAY YEAR EXPECTED SUBMISSION YES ilfyrt, completr EXPECTED $ (fehttSStOH OATEl NO DATE (I SI 05 30 89 ASSTRACT (Limit to le00 u>>crt, I.A, ronrerrlmetrry fifteen Vnerr tprcr tyorwritNn finrtl IIDI On January ll, 1989 Palo Verde Unit 3 was in Hode 5 (COLD SHUTDOWN) when information was obtained while performing a root cause of failure evaluation on a failed Emergency Diesel Generator (EDG) rocker arm which indicated that the failure was a result of a manufacturing error. Consequently, the rocker arm failure was determined to be reportable pursuant to 10CFR21.
During routine testing of the RAR EDG on January 4, 1989, the exhaust rocker arm for the 8L cylinder failed resulting in an EDG trip. Subsequent investigation revealed that a crack, which had existed in the rocker arm prior to delivery to Palo Verde, resulted in the rocker arm failure. The remaining rocker arms on the HAR EDG were inspected, and another crack was identified on the exhaust rocker arm for the 9R cylinder.
Pursuant to Technical Specification ACTION requirements, Unit 3 was shut down and Hode 5 (COLD SHUTDOWN) was entered on January 7, 1989 at approximately 1032 HST. During the cooldown a spurious Hain Steam Isolation Engineered Safety Feature actuation occurred.
The rocker arms for the remaining Palo Verde Units 1, 2, and 3 EDG's have been inspected and no other deficiencies were noted. Long term corrective action will be developed in association with the original equipment manufacturer (Cooper-Bessemer).
8902240282 890206 PDR ADOCK 05000580 NRC ro<m 345 S PDC
NRC Form 344A ~ V.S. NUCLEAR REOULATORY COMMISSION (983)
LICENSEE EVENT REPORT ILERI TEXT CONTINUATION APPROVEO OM8 NO, 3150 0104 EXPIRES: 8/31/88 FACILITY NAME ()I OOCKET NUMSER )2)
LER NVMSER )4) PACE )3)
SEQUENTIAL :/)r'EVISION NUMSER NUMSSR Palo Verde Unit 3 o s o o o 5 3 0 8 9 0 0 4 0 1, 02 oF 1 3 TEXT /I/more eoece /e )e)/rkerL oee //ro'ooe//YRC Forrrr 3////A'e))12)
This report is also being provided pursuant to the provisions of (1) Technical Specification 4.8. 1. 1.3 for a Special Report concerning a Diesel Generator failure and (2) 10CFR21. The narrative below includes the information requested by 10CFR21.21(b)(3); however, it is being formatted to report this event in accordance with the requirements of 10CFR50.73.
DESCRIPTION OF WHAT OCCURRED:
A. Initial Conditions:
Diesel Generator Failure Shutdown Re uired b Technical S ecifications At approximately 0602 HST on January 4, 1989, Palo Verde Unit 3 was in Mode 1 (POWER OPERATION) at approximately 100 percent power when the rocker arm failed on the "A" Emergency Diesel Generator (EDG)(EK). Since the rocker arm failure, Palo Verde Unit 3 was shutdown pursuant to Technical Specification 3.8. 1. 1 ACTION UB" and entered Mode 5 (COLD SHUTDOWN) at approximately 1032 HST on January 7, 1989.
ENGINEERED SAFETY FEATURE ACTUATION At the time of the Hain Steam Isolation Engineered Safety Feature actuation (JE)(SB) at approximately .0640 HST on January 7, 1989, Palo Verde Unit 3 was in Mode 4 (HOT SHUTDOWN). Reactor Coolant System (AB) temperature and pressure were approximately 237 degrees Fahrenheit and approximately 390 pounds per square inch-absolute (psia). Steam generator pressure was approximately 23 psia.
B. Reportable Event Description (Including Dates and Approximate Times of Major Occurrences):
Event Classification: Plant shutdown required by the Technical
'Specifications. Condition which could have prevented the fulfillment of a safety function. Engineered Safety Feature Actuation. 10CFR21 Report.
Diesel Generator Failure Note: This section includes information requested by 10CFR21 concerning the nature of the defect and dates for which information was obtained/developed.
At approximately 0200 HST on January 3, 1989, the "AU Emergency Diesel Generator was declared inoperable for the performance of routine inspections and.maintenance. Following the maintenance and inspection, on January 4, 1989 at approximately 0250 HST the UA" 4RC IORM SESA (9 SSI
Jl ~J NRC Form 388A U.S NUCLEAR REOULATORY COMMISSION (943)
LICENSEE EVENT REPORT ILER) TEXT CONTINUATION APPROVEO OMS NO. 3150&10E EXPIRES: 8/3)/88 FACILITY NAME )1) OOCKET NUMSER )1)
LER NUMSER )8) PACE C))
gW SEQVENTIAL H~~) REVrSION NVMEE/I NvMEErr Palo Verde Unit 3 o s o o o 5 3 0 8 9 0 0 4 0 1 0 3 OF 1 3 TEXT /IImoro Eooco /E Err/krorI, rrro aEAEAe/orrol IIRC Forrrr 3EELE8/ )) 7)
Emergency Diesel Generator was started for a 4-hour engine performance analysis. At approximately 0602 HST on January 4, 1989, the "AR Emergency Diesel Generator tripped due to an indicated overspeed condition.
Subsequent investigation determined that the diesel trip resulted from excessive vibration which was caused by a failed exhaust rocker arm. The vibration caused the overspeed trip devices to actuate. It was determined that no actual overspeed condition existed. An engineering evaluation of the failed rocker arm was initiated.
A physical and visual examination of the SL cylinder valve train assembly was completed on January 4, 1989. Following this examination, the failed rocker arm was replaced and on January 6, 1989 the RA" Emergency Diesel Generator was started for post repair testing. During the testing, excessive vibrations were observed and the diesel tripped on a high vibration condition at approximately 2132 HST. Per discussion and agreement with Cooper-Bessemer, the diesel generator was again started for further troubleshooting and another trip occurred due to high vibration at approximately Olll HST on January 7, 1989. These trips were determined to be a result of turbocharger (EK)(BLO) damage which occurred as a result of the SL exhaust valve rocker arm failure.
Shutdown Re uired b Technical S ecifications Due to the inoperable RAR Emergency Diesel Generator trip, Palo Verde Unit 3 was shutdown pursuant to Technical Specification 3.8. l. 1 ACTION RbR. Technical Specification 3.8. 1. 1 ACTION "b" states, "With one emergency diesel generator...inoperable,...
restore the diesel generator to OPERABLE status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or be in at least HOT STANDBY within the next 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> and in COLD SHUTDOWN, within the fol.lowing 30 hours3.472222e-4 days <br />0.00833 hours <br />4.960317e-5 weeks <br />1.1415e-5 months <br />." As discussed above, the "AR Emergency Diesel Generator was declared inoperable at approximately 0200 HST on January 3, 1989. The Unit 3 shutdown was commenced at approximately 0140 HST on January 6, 1989. At approximately 0637 HST on January 6, 1989 Hode 3 (HOT STANDBY) was entered. At approximately 1032 HST on January 7, 1989 Mode 5 (COLD SHUTDOWN) was entered.
En ineered Safet feature Actuation During the Unit 3 cooldown, a spurious Hain Steam Isolation Engineered Safety Features (SB)(JE) actuation occurred at approximately 0640 HST on January 7, 1989 as the result of a Steam Generator (AB)(SG) Number 2 low pressure trip signal.. All components responding to the Hain Steam Isolation actuation functioned properly.
4rrc ro/IM EOOA (9 83r
4~ ~ '
NRC Form 344A U.S. NUCLEAR REOULATORY COMMISSION 19@$ 1 LICENSEE EVENT REPORT ILER) TEXT CONTINUATION APPROVEO OMS NO, 3150WI 04 EXPIRES: S/31/SS FACILITY NAME 111 OOCKET NUMSER (31 LER NUMSER 191 ~ AOE 131 SEOUENT>AL:,".o:: AEV>4>ON NUM 4>l ',>>> NUM44>l Palo Verde Unit 3 o s o o o 5 3 0 8 9 00 4 01. 040F 1 3 TEXT llfmore 4oece 4 n />>/re>L 4>> /A/4/ '>ne/H>IC For>>> 34SASl IITI As the Unit is being cooled down and steam generator pressure is decreasing, steam generator low pressure trip setpoints are periodically reduced to prevent inadvertent Engineered Safety Feature actuations. The variable trip setpoints are manually reduced by control room personnel (utility, licensed) at prescribed intervals during the cooldown. At approximately 2309 MST on January 6, 1989, the Steam Generator Number 2 Channel RC" low pressure trip setpoint reduction circuitry operated improperly.
The low pressure trip setpoint was observed to increase rather than decrease when it was adjusted by control room personnel. This caused a spurious Steam Generator Number 2 low pressure channel trip; however, a Hain Steam Isolation did not occur as the required two (2) of four (4) coincidence was not completed. Steam Generator Channel UCR was placed in "bypass", thus changing the Hain Steam Isolation actuation coincidence to two (2) of three (3). The cooldown continued and at approximately 0606 MST on January 7, 1989, the Main Steam Isolation Valves (ISV)(SB) were closed pursuant to procedural controls during the unit cooldown. At approximately 0640 MST on January 7, 1989, a Steam Generator Number 2'ow pressure Channel "AU trip was received which resulted in the Main Steam Isolation Engineered Safety Feature actuation. All components operated per design during the actuation.
C. Status of structures, systems, or components that were inoperable at the start of the event that contributed to the event:
Other than the RAR Emergency Diesel Generator and Steam Generator Number 2 Low Pressure Channel RCR as discussed above in Section I.B, there were no structures, systems, or components inoperable at the start of the event that contributed to the event.
D. Cause of each component or system failure, if known:
Note: This section includes information requested by 10CFR21 concerning the nature of the defect and dates for which information was developed.
Rocker Arm Failure ANPP removed the fractured rocker arm and visually inspected the part. ANPP discussed the failure with Cooper-Bessemer on January 4, 1989, and an evaluation of the rocker arm failure was, started. The 8L cylinder exhaust rocker arm failed when the end of the push rod arm fractured and separated from the main exhaust valve rocker arm (Figure 1). The push rod arm casting is machined to accept a round insert which comprises the push rod seat. The fracture occurred at the rocker arm wall on both sides of this insert and down through the base N>IC ~ a>IM 344A 19 93>
0 NRC form 34SA U.S. NUCLEAR REGULATORY COMMISSION (943 I LICENSEE EVENT REPORT ILER) TEXT CONTINUATION APPROVED OMB NO. 3150&104 EXPIRES: 8/31/88 FACILITY NAME (Il DOCKET NUMBER (2l LER NUMBER (Sl ~ AOE (3l YE*R ~2. I SKQVENTI*L NUMBER
- r. II REVISION NVM44R Palo Verde Unit 3 o 5 o o o 5 3 0 8 9 00 4 0 1 0 5 oF 1 3 TEXT /// mo/4 NMco /4 /Ntu/RNE uoo 4//40oo/HAC form 38$ 49/ (IT) of the push rod arm. Subsequent inspection of the fractured piece revealed evidence of paint on the fracture face. This indicates a crack had occurred prior to painting the rocker arm and the paint seeped into the crack. ANPP received the rocker arm painted from the factory. ANPP sent the failed rocker arm to Cooper-Bessemer for inspection'nd analysis.
On January ll, 1989, discussions with Cooper-Bessemer confirmed that the rocker arm failure resulted from an original defect prior to delivery to ANPP. The specific cause of the failure is still under investigation by Cooper-Bessemer. A supplement to this report will be submitted to provide the results of the final root cause analysis.
On January ll, 1989, ANPP removed the inspection covers and visually inspected the remaining intake and exhaust rocker arms on the UAR Emergency Diesel Generator. This 'inspection revealed a cracked exhaust rocker arm for the 9R cylinder.
An inspection of the fracture revealed evidence of paint on the fracture face. This rocker arm remained functional. The rocker arm had been cracked prior to receipt from the factory.
- 2. Turbochar er Failure On January 9, 1989, ANPP and Cooper-Bessemer conducted an investigation concerning the turbocharger (intake blower) damage. An inspection was made of the combustion air intake piping and turbocharger intake blower. The inspection of the combustion air intake piping included the piping both upstream and downstream from the turbocharger. During the inspection, various turbocharger pieces were found. This included pieces of the aluminum intake blower and a single vane from the turbocharger diffuser. A visual and microscopic examination of the turbocharger diffuser vane was conducted. This examination indicated that the vane attachment bolts failed under sudden loading conditions.
A preliminary examination of the damaged turbocharger blower fracture surface was made. It was observed that the fracture surface did not indicate the presence of beach marks or crack arrest lines typical of high cycle fatigue failures.
A visual inspection of the turbocharger main journal bearing assemblies was conducted. This included both the turbine exhaust and the blower intake ends of the turbocharger. The babbitt lining on the bearing pad inserts indicated abnormal signs of wear. Micrometer readings of the pad thickness indicated that approximately 4 mils of the babbitt had been worn away. No indication of discoloration or heat damage 4RC IORM 3444 I9 BSI
II 0 NR(( Form 388A'943 U.S. NUCLEAR REQULATORY COMMISSION l
LICENSEE EVENT REPORT ILER) TEXT CONTINUATION APPROVED OMB NO. 3150M(BE EXPIRES: 8/3l /88 FACILITY NAME (Il DOCKET NUMBER (2l LER NUMBER (8) PACE (3l SEQUENTIAL Wring REVISION NUMBER R!W NVMSER Palo Verde Unit 3 o s o o o 5 3 0 8 9 0 0 4 0 1 0 6oF 1 3 TEXT ///mart NMco 4 rt/O/rtrE vw //aarrt/P//IC %%drrn 38((ABI (ITI typical of a loss of lubrication failure was apparent. The abnormal wear was determined to be a result of rotor imbalance following the turbocharger damage.
The oil seals on both ends of the turbocharger had indications of damage from contact with the rotor labyrinth oil seal. The main thrust bearing exhibited indications of scoring from normal wear.
A review was conducted of the historical operating data for the turbocharger lube oil pressures and vibration monitoring. No unusual trends indicating .an impending bearing failure were found.
A visual inspection of the damaged turbocharger intake blower was made. Per discussions with Cooper-Bessemer, a blower overspeed condition is indicated by cracking initiated at the blower hub section bore. This cracking propagates outward from the hub bore toward the blower wheel outer diameter. No indications of this type of cracking were found in the blower hub bore during the visual inspections. Additionally, Cooper-Bessemer indicated that an overspeed failure would have resulted in a total fai.lure of the intake blower. The type of failure experienced at ANPP, the loss of a blade from the front section of the blower, is not typical of an overspeed failure.
It was determined that the damage to the turbocharger was not a result of high cycle fatigue failure, loss of bearing function, or, an overspeed of the turbocharger rotating element.
During the failure of the SL exhaust valve rocker arm, strong pulsations occurred in the combustion air intake piping.
These pulsations occurred because the exhaust valves would not open and combustion gasses flowed into the intake piping instead of the exhaust piping. These pulsations were observed by engineering and operations personnel at the DG building during the performance of surveillance testing.
These pulsations caused the turbocharger intake blower to surge. Surge is a phenomenon which can occur in radial flow compressors such as the diesel generator turbocharger intake blower. It can be described as a periodi'c reversal of flow through individual stages of the compressor blower.
Pulsating back pressure on the blower causes a reduction in the fluid mass flow rate at the compressor inducer inlet.
This causes a flow separation of the fluid from the blade resulting in a localized stall condition. Stall encourages a reversal of flow through the affected stages causing a system 4RC I QIIM 34OA (9 BSI
NRC Form 348A'9431 U.S. NUCLEAR REOULATORY COMMISSION LICENSEE EVENT REPORT (LER) TEXT CONTINUATION APPROVEO OMS NO. 3)50&104 EXPIRES: 8/31/88 FACILITY NAME (11 OOCKET NUMBER (3)
LER NUMSER (5) PACE (31 YEAR ~~ SEOUENTIAL '<r'r'1 IIEVISION NUMEER ~@ NUMeea Palo Verde Unit TEXT /// more eoece /4 te))I/rerL oee ~ 3
/V/EC Form 3/)Cl'sl (17) o s o o o 5 3 0 8 9 00 4 01. 07oF 1 3 instability described as surge. During surging, violent flow processes can occur causing periods of back flow through the compressor. Operation in surge condition not only drastically reduces compressor performance but can also result in compressor damage.
The blower surge combined with the pulsations caused severe bending stresses to occur in the blower vane. These bending stresses caused the blower vane to fail at the hub fillet near the front of the blower. The root cause of the turbocharger intake damage failure was determined to be pulsation disturbances of the combustion air intake piping.
These disturbances resulted directly from the 8L exhaust rocker arm failure.
E. Failure known:
mode, mechanism, and effect of each failed comp'onent, if The failed rocker arm resulted in the "AR Emergency Diesel Generator trip as described in Section I.B. The failed rocker arm also resulted in turbocharger damage which resulted in the subsequent diesel generator trip as described in Section I.B.
F. For failures of components with multiple functions, list of systems or secondary functions that were also affected:
Not applicable - the failed rocker arm and damaged turbocharger do not have multiple functions.
G. For failure that rendered a train of a safety system inoperable, estimated time elapsed from the discovery of the failure until the train was returned to service:
The failed rocker arm resulted in the "AR Emergency Diesel Generator (EDG) trip at approximately 0602 HST on January 4, 1989.
The RAR EDG was returned to service at approximately 0350 HST on January 14, 1989.
H. Hethod of discovery of each component or system failure or procedural error:
The rocker arm failure was detected as a result of a post trip visual inspection performed on the Emergency Diesel Generator. The actual method of detection was as a result of observing signs of heat blistering near the 8L cylinder intake manifold. Upon removal of the cylinder head inspection cover, a visual inspection determined that the exhaust rocker arm assembly on the 8L cylinders had failed.
NIIC r OEM 3444 I9 83 I
4!
NAC form 388A'943)
U.S. NUCLEAR AEOULATORY COMM)$8)ON LICENSEE EVENT REPORT ILER) TEXT CONTINUATION APPAOYEO OMS NO, 3)50-010O EXPIRES: 8/31/88 FACILITY NAME III OOCKET NULISEA )2)
LEA NUMSER IS) PACE 13) hajj. TEOVENTIAL II8 V ro IO N NVMOTA NVMOEA Palo Verde Unit 3 o s o o o 5 30 8 9 00 4 01 08 OF 1 3 TEXT litmoro )oooo lr n)lr/rorL ow c/roo'ono/HAC Form 388A'I) )17) f The turbocharger failure was detected as a result of a second post trip visual inspection performed on the RA" EDG. This visual inspection identified the turbocharger damage.
There were no procedural errors.
I. Cause of Event:
See Section I.D. for the discussion concerning the Emergency Diesel Generator exhaust rocker arm failure and turbocharger damage.
The cause of the improper operation of the Steam Generator Number 2 channel RCR low pressure trip setpoint reduction circuitry could not be determined. The improper operation discussed in Section I.B could not be recreated during troubleshooting. This type of circuit behavior has been experienced before at Palo Verde. There is a periodic problem with an asynchronous clock circuit card in the trip setpoint reduction circuitry. This is an infrequent phenomenon and is considered low risk since the clock circuitry is enabled only when the setpoint reduction reset button is pushed or when steam generator pressure is increasing. This problem does not adversely affect the ability of the Hain Steam Isolation System to be actuated when necessary.
The cause of the spurious Hain Steam Isolation System Engineered Safety Feature actuation. was determined to be a malfunctioning matrix relay (94) in the Steam Generator Number 2 Channel RBA low pressure circuitry. The resistance value between the relay contacts was discovered to be abnormally high. The high resistance Channel RBR resulted in more current flow through the Channel
'n RAR portion of the RA-BR matrix logic vice equally between channels RAR and RB". When Channel AAA tripped as steam generator pressure was decreasing (See section I.B), there was a momentary interruption in current flow since a short period of time was required for current flow to re-establish through the ABR portion of the matrix logic. This momentary interruption in current flow caused the Hain Steam Isolation Actuation. The high resistance problem is considered to have no adverse safety consequences since the Hain Steam Isolation System would be conservatively actuated on a one (I) of four (4) coincidence vice the normal two (2) of four (4) coincidence.
The relay is manufactured by Electro Hechanics, Inc. The model number is 33335//'ELHE.
J. Safety System Response:
Other than the Hain Steam Isolation system response described in Section I.B, no safety system responses occurred and none were necessary.
4IIC IOIIM 3OOA 19 8)r
II Ot 1
NRC, Form 344A U.S. NUCLEAR REOULATORY COMMISSION 1983 I LICENSEE EVENT REPORT ILER) TEXT CONTINUATION APPROVEO OMB NO. 3)SPWIP4 EXPIRES: 8/31/88 FACILITY NAME (I) OOCKET NUMBER (3)
LER NUMBER IS) PACE (3)
$ 4ovSNT/AL RSVlSIPN NVM SR .. Ir NVMSSR Pal o Verde Uni TEXT /// mole )ooce t 3 o s o o o 5 3 0 8 9 0 0 4 01 09 OF 1 3
/I /e//rkerL ow aAWonol /VRC Form 38843/ lll)
K. Failed Component Information:
Note: This section includes information requested by IOCFR21 concerning the identification of the firm supplying, the basic component and the number and location of the rocker arms at Palo Verde.
The rocker arms and turbochargers installed on the Palo Verde Diesel Generators were manufactured by Cooper-Bessemer. The rocker arm is a cast iron cylinder with three integral arms, two for actuation of the exhaust valves and one for the push rod to act on. The cylinder has a longitudinal hole, with oil lubricated bearings for the mounting shaft, about which the rocker arm pivots. The two valve actuation arms are fitted with hydraulic lifters and clearance adjusters. The push rod arm is fitted with a steel insert for the push rod to pivot in. (See Figure 1)
The damaged turbocharger model number is ET-24 and the failed rocker arm part number is KSV-25-IAO'I. Both of these parts are installed on the RAR Emergency Diesel Generator Engine Serial Number 7187, Model No. KSV-20-T.
The emergency diesel generators utilized at Palo Verde Units 1, 2, and 3 are manufactured by Cooper-Bessemer division of Cooper Energy Services. The model number is KSV-20-T. Each unit utilizes two (2) emergency diesel generators (six total). Each emergency diesel generator has twenty cylinders. Each cylinder has one intake rocker arm and one exhaust rocker arm. There are 120 exhaust rocker arms and'20 intake rocker arms installed at Palo Verde.
I I. ASSESSMENT OF THE SAFETY CONSEQUENCES AND IMPLICATIONS OF THIS EVENT:
Emer enc Diesel Generator Failure Note: This section contains the information requested by 10CFR21 concerning the nature of the safety hazard which is created or could be created.
There were no safety consequences as a result of the failed rocker arm on the "AR Emergency Diesel Generator as the RB" Emergency Diesel Generator remained operable and off-site power sources remained available. However, the emergency diesel generators serve as a back-up power supply in the event off-site power becomes unavailable. In the event of a loss of off-site power, the emergency diesel generators would be required to supply the safety related equipment necessary for (1) the safe shutdown of the facility and (2) the mitigation and control of accident conditions within the facility. As discussed in Section I.B:,
rocker arm failure results in emergency diesel generator unavailability.
4RC ~ PRM 3444 I9 831
NRC Forrrr 344A" U.S. NUCLEAR REOULATORY COMMISSION (9831 LICENSEE EVENT REPORT ILER) TEXT CONTINUATION APPROVED OMS NO. 3150M(94 EXPIRES! 8/31/88 FACILITY NAME (II OOCKET NUMSER (11 LER NUMSER (8) ~ AOE (31 SCQVCNTIAL ~i~ IICVISION NVM CA (i'4 NVM4CII Palo Verde Unit TEXT lllrrrrrre Mece /I eer(re44, eee ~ 3 HRC %%dnn 3(C(AS/ (17) o s o o o 53 08 9 0 0 4 01.10oF 13 En ineered Safet Feature Actuation There were no safety consequences or implications resulting from the Hain Steam Isolation Engineered Safety Feature actuation. The Hain Steam Isolation System (HSIS) rapidly terminates steam blowdown and feedwater flows by isolating each generator, should a significant loss of steam generator mass inventory or pressure occur. The HSIS is actuated'y receipt of a 2-out-of-4 high containment pressure, low steam generator pressure, or high steam generator water level signal. The steam generator pressure setpoints are variable. At steam generator pressures below normal operating conditions, control room personnel have the ability to manually decrease the setpoint to a fixed increment below existing system pressure. This is used during plant cooldown.
Prior to the HSIS actuation described in Section I.B, the steam generator low pressure trip setpoint had been reduced to zero (o) psia.
Due to normal instrument error associated with the steam generator low pressure trip circuitry, it is not unusual for steam generator trip signals to be actuated as steam generator pressure approaches atmospheric pressure during the cooldown. These low steam generator pressure trip signals are expected and are not indicative of an abnormal steam generator pressure. Control room personnel (ut'ility, licensed) verified that no actual condition requiring an HSIS actuation occurred (i.e., significant loss of steam generator mass inventory or pressure).
I I I. CORRECTIVE ACTIONS:
This section contains the information requested by 10CFR21 concerning the corrective action which has been, is being, and will be taken; the organizations responsible for the corrective action; and the length of time for accomplishing the corrective action.
A. Immediate:
Emer enc Diesel Generator Failure The failed rocker arm and damaged turbocharger components were replaced. The remaining rocker arms on the "A" Emergency Diesel Generator (EDG) have been inspected and one was replaced. These corrective actions were completed by January 12, 1989. Following the return of the "A" EDG to service, the "BN EDG rocker arms were inspected. No other failed rocker arms were identified. This action was completed by January 14, 1989.
The Unit 1 and 2 EDG intake and exhaust rocker arms were visually inspected on January 16, 1989. No additional defective rocker arms were identified.
4IIC ~ OIIM 3444 (9 43 I
II 'VS NRC FOIIII 344K 1943) U.S. NUCLEAR REOULATORY COMMISSION LICENSEE EVENT REPORT (LER) TEXT CONTINUATION APPROVFO OMS NO, 3150M)04 EXPIRES; 8/3'I/88 FACILITY NAME II) DOCK ET NUM8 E R l2)
LER NUMSER IS) ~ AOE IS)
SEOVENTIAL ~y REVISION YEAR gP NVMSER )Sr: NVM ER Palo Verde Unit 3 o s o o o 5 3 0 8 9 00 4 01 1 1 QF 1 3 TEXT //F IIRVO 4OOCO ir /OFMPed. VW I//OOOO////IC FORR 3/)//AS / l)T)
En ineered Safet Feature Actuation The malfunctioning matrix relay in the Steam Generator Number 2 Channel "BR low pressure circuitry was replaced.
B. Action to Prevent Recurrence:
Emer enc Diesel Generator Failure Until final corrective actions for the rocker arm failure are established, all new replacement rocker arms will be inspected by ANPP utilizing non-destructive examination techniques prior to installation. The non-destructive examination will provide assurance that the new rocker arms utilized at Palo Verde are free from cracks in the press fitted area. Further determination of corrective action to prevent recurrence will be developed in association with Cooper-Bessemer. A supplement to this report will be provided to describe the long term corrective actions.
En ineered Safet Feature Actuation As corrective action to prevent recurrence for the periodic improper operation of the. steam generator low pressure trip setpoint reduction circuitry, a design modification will be implemented to replace the appropriate variable setpoint card with an improved model. As an interim measure, additional procedural instructions will be provided to caution operations personnel about the problem which may occur and to delineate the appropriate measures for minimizing the probability of occurrence.
Concerning the malfunctioning matrix relay in the Steam Generator
.Number 2 Channel RBR low pressure circuitry, no further corrective actions are considered necessary at this time. However, be noted that similar relay malfunctions have previously occurred it should and an engineering evaluation of this problem is being conducted.
If substantial changes to the corrective actions described in this LER are implemented as a result of this event, a supplement to this report will be issued.
IV. PREVIOUS SIMILAR EVENTS:
Emer enc Diesel Generator Failure ANPP experienced one other broken rocker arm in September 1985. The failure occurred on the Unit 2, Train RBR Diesel Generator. The intake rocker arm failed at the junction of the push rod arm and rocker arm body. The September, 1985 root cause of failure was due to improper adjustment of the push rod clearances. This caused the push rod to hammer against the push rod seat eventually breaking the rocker arm.
4RC lORM 3444 IQ SSI
4 NRC Form 3SSA U.S. NUCLEAR REGULATORY COMMISSION I0 4)31 LICENSEE EVENT REPORT {LER) TEXT CONTINUATION APPROVED OMS NO. 3150&104 EXPIRES: S/31/88 FACILITY NAME 111 DOCKET NUMSER Ill LER NUMSER IS) ~ AOE LT)
SEOVENTIAL Re v rslo N NUMSEII rrvMeea Palo Verde Unit TEXT /I/ mare Nrece /1 n))reer/. rree 3
////)me////IC Farm 3//SAS) (lt) o s o o o 5 30 89 0 0 4 01.12 QF 1 3 The material and casting were not found to be deficient and proper tappet adjustments have precluded recurrence. This was determined to be reportable under 10CFR50.55e (
Reference:
Deficiency Evaluation Report 85-20). There have been no previous similar events reported pursuant to 10CFR50.73.
En ineered Safet Feature Actuation There have been no previous similar events reported pursuant to 10CFR50.73 for the Main Steam Isolation System actuation described above.
V. SPECIAL REPORT ADDITIONAL INFORMATION:
Pursuant to Technical Specification 4.8. 1. 1.3, the following is provided to include information concerning a valid Emergency Diesel Generator failure not described above and which is required in Regulatory Position C.3.b of Regulatory Position 1. 108.
The failure described above is the third failure in the last 100 valid tests. The failure described above is the first failure in the last 20 valid tests. The current surveillance testing interval is once per 31 days.
4aC SOAM 34eo (9 83r
44 H Arizona Nuciear Power Project P.O. BOX 52034 ~ PHOENIX, ARIZONA85072-2034 192-00449-JGH/TDS/DAJ February 6, 1989 U. S. Nuclear Regulatory Commission NRC Document Control Desk Washington, D.C. 20555
Dear Sirs:
Subject:
Palo Verde Nuclear Generating Station (PVNGS)
Unit 3 Docket No. STN 50-530 (License No. NPF-74)
Licensee Event Report 89-004-01 File: 88-020-404 Attached please find Supplement 1 to Licensee'vent Report (LER) No. 89-004-00 prepared and submitted pursuant to 10CFR 50.73. In accordance with 10CFR 50.73(d), we are herewith forwarding a copy of the LER to the Regional Administrator of the Region V office.
This report is also being submitted pursuant to Technical Specification 4.8. 1. 1.3 and 6.9.2 for a Special Report concerning a Diesel Generator failure and 10CFR21. This report includes the information requested in 10CFR21.21(b)(3) . In accordance with 10CFR21.21(b)(2), three copies of this report are being provided to the Director, Office .of Nuclear Reactor Regulation.
If you have any questions, please contact T. D. Shriver, Compliance Hanager at (602) 393-2521.
Very truly yours, J. G. Haynes Vice President Nuclear Production JGH/TDS/DAJ/kj Attachment cc: D. B. Karner (all w/a)
E. E. Van Brunt, Jr.
T. E. Hurley (3 copies)
J. B. Hartin T. J. Polich H. J. Davis A. C. Gehr INPO Records Center Cooper-Bessemer
0 r f