05000312/LER-1989-001, :on 890131,auxiliary Feedwater Pump P-318 Reached Overspeed Condition Resulting in Overpressurization of Auxiliary Feedwater Trains.Caused by Failure of Turbine Governor to Control Turbine Speed

:on 890131,auxiliary Feedwater Pump P-318 Reached Overspeed Condition Resulting in Overpressurization of Auxiliary Feedwater Trains.Caused by Failure of Turbine Governor to Control Turbine Speed

ML20247M321
Person / Time
Site:	Rancho Seco
Issue date:	07/24/1989
From:	Firlit J, Schumann D SACRAMENTO MUNICIPAL UTILITY DISTRICT
To:	NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
CEO-89-255, LER-89-001, LER-89-1, NUDOCS 8908020059
Download: ML20247M321 (8)

LER-1989-001, on 890131,auxiliary Feedwater Pump P-318 Reached Overspeed Condition Resulting in Overpressurization of Auxiliary Feedwater Trains.Caused by Failure of Turbine Governor to Control Turbine Speed

Event date:
Report date:
Reporting criterion:	10 CFR 50.73(a)(2)(1) 10 CFR 50.73(a)(2)(i)(A), Completion of TS Shutdown
3121989001R00 - NRC Website

text

. - - -

y;

^

y T?SMUD if i 'N

' flACRAMENTO MUNICIPAL UTIUTY DISTRICT C 6201 S Street. P.o. Box 15830, Sacramerrto CA 95852-1830,(916) 452-3211 AN ELECTRIC SYSTEM SERVING THE HEART OF CALIFORNIA

.CE0l89-255 July 24, 1989 U. S. Nuclear Regulatory Commission Attn: Document Control Desk

- Washington, DC 20555 Docket No. 50-312 Rancho Seco Nuclear Generating Station

' License No. OPR-54 LICENSEE EVENT REPORT 89-01, REVISION 1: TECHNICAL SPECIFICATION REQUIRED SHUTDOHN DUE TO INOPERABLE AUXILIARY FEEDHATER SYSTEM Attention: George Knighton

.In accordance with the requirement of 10 CFR 50.73(a)(2)(1)(A) and (B),-the Sacramento Municipal Utility District hereby submits Licensee Event Report 89-01, Revision 1.

This revision is being submitted to update results of additional investigations since the original submittal.

Members of.your staff with questions requiring additional information or

. clarification may contact Mr. Steve Rutter at (209) 333-2935, extension 4911.

Sincerely,

)-

Joseph F. Firlit Chief Executive Officer Nuclear Attachment cc w/atch:

J. B. Martin, NRC, Hainut Creek A. D'Angelo, NRC, Rancho Seco INP0 m

8908020059 890724 f

PDR ADOCK 05000312 S

PDC RANCHO SECO NUCLEAR GENERATING STATICN O 1444o Twin Cities Road, Herald, CA 95638 9799;(209) 333 2935

N C form 264 U $, NUCLEQR REGULQTOQY COMMIS$10N APPROVED OMB NO aflio 0104

[

LICENSEE EVENT REPORT (LER)

' *P'REs s 3i se f ACILITY NAME (11 DOCKET NUM9ER (21 PAGk s3' l2 i lor l 0 1 7 Rancho Seco Nuclear Generating Station oisio10101 3l i

Technical Specification Required Shutdown "7

Due to Inoperable Auxiliary feedwater System l

EVENT DATE (5)

LER NUMBf R del REPORT DATE (7)

OTHE R F ActLITIES INVOLVED (B) l uoNTH oAv vEAR vtAa

$6L%~j ;A' 07 n MONTH oAv vfAn r ac' tie v ~>uts oOestET NuMeEHts>

l 0 16101010 1 1 I d1 3l 1 89 8l9 0l 0l 1 0l 1 0l 7 2l4 8l 9 o isio 1 0: 0, I i D WWM M M MWWEW M M CM $ NA one w m v m 4w@ W OPE R ATING N

ro a2e 20 *i<'

so 72=H2He 78 "'a' POvvE R 20 406teH1Hil 60.36 teH i t 60 731 sit 2H=l 73.71 tc) 0 913 20 *'aH'Hai

$ow H2) so n H2"v'M g E gsa y, = rgr (101 20 405(sH1 Head X

so 72. H2Ho so 73i.H2a."*HAi asua>

20 406teH1Hiv) 60.73aal(2Hed 60 73teH2HvihH3) 20.4061aH1Hel 90./ Sia H2 Huel 60 7SieM2Hal LICENSEE CONT ACT FOR THl3 LER 112)

NAME TE LEPHONE NUMBER ARE A CODE 2 -i 3l 2 1l1 16 45 1 Dave Schumann, Nuclear Licensing Incident Analyst 91 1

1 1

1 COMPLETE ONE LINE FOR E ACH COMPONENT F AILURE DESCRIBED IN THit REPORT 113)

M I

M gf C'AUSE SYSTEM COMPONENT

CAUSE

S v 5T E M COMPONENT A

ORTA8 f p

ppd 04,2 Y

B Bi A ;i51 i H i 2191 0 Y

B Si B PC i V i

i B i 1

i Y

PC ! Vi V 1 08;Si D

BI A TIR I Bi Ti ll41 7 Y

X A l B I

EkPECTED YES tt! res compete EMPECTED SUBMIS$10N DATEI NO l

l } l G85TROCT ILomst to F400 suces u o sporeermoreiv r>rteen av soece trouwtren ronw (16l At 1725 hours0.02 days <br />0.479 hours <br />0.00285 weeks <br />6.563625e-4 months <br /> on January 31, 1989, a post maintenance test of auxiliary feedwater (AFH) pump P-318 was in progress. During the performance of this test, the pump reached an overspeed condition resulting in an overpressurization of both AFH trains. At 2156 hours0.025 days <br />0.599 hours <br />0.00356 weeks <br />8.20358e-4 months <br />, after an engineering evaluation of the incident, both AFH trains were declared inoperable.

In accordance with Technical Specifications, Control Room Operators began a plant shutdown at 2212 hours0.0256 days <br />0.614 hours <br />0.00366 weeks <br />8.41666e-4 months <br /> and transition to decay heat cooling.

The reactor was in hot standby at 0146 hours0.00169 days <br />0.0406 hours <br />2.414021e-4 weeks <br />5.5553e-5 months <br /> on February 1, 1989. At 0155 hours0.00179 days <br />0.0431 hours <br />2.562831e-4 weeks <br />5.89775e-5 months <br />, the reactor was manually tripped to assure that a greater than 1 percent shutdt,arn margin was achieved within the Technical Specification imposed 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> time limit.

At 1554 hours0.018 days <br />0.432 hours <br />0.00257 weeks <br />5.91297e-4 months <br />, Control Room Operators established decay heat cooling, I hour and 59 minutes after the specified Technical Specification time requirement.

The mandatory shutdown of the plant as required by Technical Specifications is reportable pursuant to 10 CFR 50.73(a)(2)(i)(A).

The failure to establish decay heat cooling within 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> subsequent to the reactor trip as required by Technical Specifications is reportable pursuant to 10 CFR 50.73(a)(2)(1)(B).

A slie team conducted a Root Cause Investigation of the pump overspeed and associated overpressurization of the AFH System.

The investigation disclosed that the turbine governor failed to control turbine speed and the mechanical overspeed trip mechanism failed to clnse the turbine steam inlet valve.

These failures resulted in pump overspeed and overpressurization of the AFH System.

g, 3

~

L

. ire tor-ma u.s. wucuaR RooutaToRv commission LICENSEE EVENY PEPOR1 (LER) TEXT CONTINUATION AreRovso oue so mo-oio4 e

EXPIRES: 8/31/b8 F ACILITV NAME (U DL 3ET NUMBLR Cr)

LER NUMBER 16)

PAGE (3)

"W.PP 1r*,0 viaa Rancho Seco Nuclear Generating Sta*. loc j 0 l5 l0 l0 lo 13 l 11 2 8l 9 0101 1

- 0110 l 2 oF 0 l '7 TEXT IH more spece a remnrod, une eMtmnet NRC Form ansket (11)

- Description of the Event At 1725 hours0.02 days <br />0.479 hours <br />0.00285 weeks <br />6.563625e-4 months <br /> on January 31, 1989, maintenance technicians were conducting a post j

maintenance installation test of the turbine governor for auxiliary.feedwater (AFH) pump P-318. During the performance of this test, the pump reached an overspeed condition resulting in overpressurization of both AFH trains. An engineering' evaluation was initiated to review the impact of the incident on the operability of the AFH trains. Based on the evaluation, both AFH trains were declared inoperable at 2156 hours0.025 days <br />0.599 hours <br />0.00356 weeks <br />8.20358e-4 months <br /> on January 31, 1989. With both AFH trains inoperable, Technical

. Specification 3.4.2.F.(2) specifies that the. plant be subcritical within 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> and on decay heat cooling within the next 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br />.

At 2212 hours0.0256 days <br />0.614 hours <br />0.00366 weeks <br />8.41666e-4 months <br /> Control Room Operaton began power reduction from 93% power in accordance with plant procedures. Control Room Operators tripped the main turbine at 0122 hours0.00141 days <br />0.0339 hours <br />2.017196e-4 weeks <br />4.6421e-5 months <br />. The reactor was in hot standby at 0146 hours0.00169 days <br />0.0406 hours <br />2.414021e-4 weeks <br />5.5553e-5 months <br />. Control Room Operators manually. tripped the' reactor at 0155 hours0.00179 days <br />0.0431 hours <br />2.562831e-4 weeks <br />5.89775e-5 months <br /> to assure a greater than 1 percent shutdown margin was achieved within-the 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> time limit. The powcr level of the reactor at the time of the trip was 10-8 amps and decreasing on intermediate. range Nuclear Instrumentation. A minor departure from the post-trip window occurred as a result of the trip in that the average reactor coolant temperature dropped.about 12 degrees F.

Before the trip the reactor was operating

- with an average temperature of 540 degrees F and pressure of 2180 psig.

. At approximately 0156 hours0.00181 days <br />0.0433 hours <br />2.579365e-4 weeks <br />5.9358e-5 months <br /> the Control Room Operators noted an abnormal rate of decrease in pressure for the 'A' steam generator. Control Room Operators determined L

that a failed open turbine bypass valve, PV-20561, was the cause of the abnormal

. pressure decrease. At approximately 0158 hours0.00183 days <br />0.0439 hours <br />2.612434e-4 weeks <br />6.0119e-5 months <br /> Control Room Operators closed the block valve for the 'A' loop turbine. bypass. valves and terminated the perturbation.

As a result of the PV-20561 malfunction, 'A' steam generator pressure peaked low at 692 psig before returning to the normal range.

At 0759 hours0.00878 days <br />0.211 hours <br />0.00125 weeks <br />2.887995e-4 months <br /> the Technical Support Center was partially activated.

This partial activation was done as a precaution to assist the Control Room in the event of main feedwater problems during the cooldown evolution.

At 0829 hours0.00959 days <br />0.23 hours <br />0.00137 weeks <br />3.154345e-4 months <br />, during plant cooldown, the alternate pressurizer spray valve PV-21509 failed to close.

Pressurizer spray block valve HV-21510 was closed per procedure.

PV-21509 was closed at 0856 hours0.00991 days <br />0.238 hours <br />0.00142 weeks <br />3.25708e-4 months <br />.

The reactor was in hot shutdown (subcritical) at 0155 hours0.00179 days <br />0.0431 hours <br />2.562831e-4 weeks <br />5.89775e-5 months <br />.

In accordance with Technical Specification 3.4.2.F.(2) the plant was required to be on decay heat cooling by 1355 hours0.0157 days <br />0.376 hours <br />0.00224 weeks <br />5.155775e-4 months <br /> on February 1, 1989. Control Room Operators established decay

- heat cooling at 1554 hours0.018 days <br />0.432 hours <br />0.00257 weeks <br />5.91297e-4 months <br />,1 hour 59 minutes beyond the Technical Specification time' requirement.

Plant Operatina Conditions Before the Event At the time of the post maintenance testing on pump P-318 the plant had been in operation for 20 days and was at 93% power.

Maintenance technicians had replaced the P-318 turbine governor.

Testing of the pump was being performed as part of the post maintenance installation testing.

w...

c m, a..- m m o m gc o.u =A

~RC form 306A m

U.6, NUCLE AR QEGULATORY COMMISElON LICENSEE EVENT REPORT (LER) TEXT CONTINUATION APPROvfD OMB NO,3150-010d EXPtRES; 8/31/88 '

'~

F ACILITY NAMk (Il.

. DOCKET NUMBER (2)

LER NUMGER (61 PAGE (3)

"MW 2 :::

~"

Rancho'Seco Nuclear Generating Station o l5 lo lo Io l3 l1l 2 8l 9 0l0 l 1

- 0l1 0l3 or 0l7 TEXT IIImore space An anywre4 une s#tenelNRC Form 306AW(111 Cause of the Manual Reactor Trio and Failure to Establish Decay Heat System Ooeration Within the Technical Specification Time Limits The manual. reactor trip was the teruit cf Control Room Operator action to assure that a greater than,1 percent shutdm margin was achieved within the 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> time limit required by Technical Specifications.

The cause of the 'A' steam generator pressure decrease to less than 700 psig was.the failurc (open) of valve PV-20561.

The cause of the valve malfunction was determined to be a failure of the positioner cam hub, an internal component of the positioner.

During extensive troubleshooting and testing of PV-21509 and its associated control circuit, technicians were unable to duplicate the malfunction.

Further investigation and testing confirmed satisfactory valve operation and did not disclose any malfunction.

The failure to establish decay heat cooling within the 12 hour1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> requirement resulted from several considerations:

Procedural requirement to borate the reactor to 300 degrees F concentration prior to initiation of cooldown.

Complications in the performance and subsequent revision of new procedure SP.520 "Special Frequency Test of the Low Temperature

. Overpressure Protection (LTOP) Circuitry, and Verification of EMOV Operability."

Investigation to resolve PV-20561 and PV-21509 response problems.

Management established a site team to conduct a Root Cause Investigat'on of the pump overspeed and associated overpressurization of the AFH System. The investigation disclosed that the turbine governor failed to control turbine speed, resulting in pump overspeed and overpressurization of the AFH System.

The direct cause of the governor failing to control turbine speed was an internal modification performed at the vendor's-facility which resulted in the governor being able to function only when rotating in the clockwise direction. When the governor rotated in the counterclockwise direction, it did not develop the required internal oil pressure and was therefore not able to perform its intended function. The direct cause of the AFH System overpressurization due to the excessive overspeed condition was the failure of the mechanical overspeed trip mechanism to close HV-30801 when turbine speed reached the overspeed trip setpoint (4450 rpm).

The Root Cause of the installation of a governor with improper rotation was that the implementation of the procurement process did not identify an unauthorized c:ange by the governor supplier or prevent subseouent installation of the governor.

h

.u... m, i o.. e 4.. -

gPmM m.

NRc Poem pe4A U.S. NUCLEAR GiEGULATDAY COMMISSION LICENSEE EVENT REPORT (LERITEXT CONTINUATION APFROVED OMB NO.3150-0104 j

ExmnEs enues f AC4LITY NAME (1)

DOCKLT NUMBER Cl LER NUMBER (6)

PACE (3)

"ets

- O*,T:

va^a 0l1 01 4 oF 0l 7 l

Rancho Seco Nuclear Generating Wtion o 15 Io lo lo l3 l 11 2 Eq 9 010l1 TEXT IN more spece in seqwmt, une eMannelNRC form 3654's)(17l.

The Root Cause of the overspeed trip mechanism not being effective was the lack of i

testing of the linkage mechanism and associated valve to ensure operability.

The Root Cause of the AFH overpressurization event was that the work request did not..

provide specific guidance and controls for post maintenance verification testing on 1

.the' newly refurbished governor.

The Enerav Industry Identification System (EIIS) Conoonent Function and System Identifier AFH pump turbine governor is'65.

AFH turbine is-TRB.

I AFH System is BA.

PV-20561 is PSV.

Main St,eam System is SB.

PV-21509 is PSV.

)

Reactor Coolant System is AB.

I I

Decay Heat System is BP.

Manufacturer and Model Number The manufacturer of the turbine governor is Woodward Governor Company.

The model is PGPL part number 9903-340.

The overspeed trip mechanism is included as a portion of the Terry Turbine, model GS-2, and the steam inlet valve HV-30801, manufactured by.

Gimpel Machine Works, vendor drawing P-2989.

The manufacturer of the positioner for PV-20561 is Bailey Controls.

The model is 5324090-2.

I I

i i

NRC POMM BesA

• 0.s. cpon 1968-520-589 D007D C438

I!

=ac re..a wu u.s suctsu nEcutArony coMassion i

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION uenovso oMe so 3 iso-oio4 EXPIRES: 8/3U88 FActLITV NAM 4 (1)

DOCKET NUMBEM (2)

LtR NUMBER {6)

PAot (31 SE i

AL R,f 3 g YEAR p

Danchn brn Nitrlpar Conprating Station o l5 jo j o j o I 31 112 81 9 01Ol 1 01 1 Ol 5 0F 01 7 TLKT W mwe space a requued. une modstenalNRC Fwm 306Kal 0h

Assessment of Safety Consequences

There were no adverse consequences affecting the health and safety of the public or Rancho Seco employees as a result of this event.

The failure of turbine bypass valve PV-20561 to close resulted in only a slight decrease of primary system temperature.

Prompt action by the Control Room Operators to isolate PV-20561 terminated the perturbation.

Corrective Actions Taken and Planned as a Result of the Event The initial investigation into the cause for the failure of alternate spray valve PV-21509 to close was not able to duplicate the malfunction. The valve was successfully cycled three times. During further investigations, technicians performed five motor load transducer tests to verify stroke times, limit switch function and smooth component operation.

Results indicated satisfactory valve operation. With the plant in hot shutdown conditions at normal operating pressure, the System Engineer performed Special Test Procedure STP.1222 " Pressurizer Spray Back-Up Valve (PV-21509) Stroke Test At Hot Shutdown" on March 9, 1989.

PV-21509 was cycled four times in manual and once in automatic control during the test.

Valve operation was satisfactory in all cases.

As a result of the positioner cam hub failure, the Plant Performance organization coordinated a further investigation. On March 15, Plant Performance engineers, I&C technicians and vendor representatives reviewed maintenance work procedures and observed field calibrations. No deficiencies were found in procedures or field work practices.

Technicians then replaced the failed cam hub and completed associated adjustments on the positioner and valve stroke length. On March 24, 1989, with the plant in power operation, Plant Performance engineers performed Special Test Procedure STP.1229 " Turbine Bypass Valve, PV-20561, Differential Pressure Stroke Test." The STP results confirmed smooth valve operation on three consecutive strokes with no component deficiencies. On March 28, a loss of main feedwater flow caused an automatic plant shutdown.

For this shutdown PV-20561 operation was satisfactory. The next day during reduction of main steam header pressure by Control Room Operators, PV-20561 was reported cycling from full open to full closed.

I&C technicians investigated and found that the positioner cam hub had 3

failed again.

Based on subsequent analyses, it is suspected that the cause of the l

recurring valve failures is an internal valve problem requiring higher than normal l

air pressure to obtain movement of the pilot plug.

This can result in higher i

accelerations and impact loads on the valve positioner, which in turn can result in cam hub failures. A corrective action plan has been formulated which requires disassembling PV-20561 to inspect the valve internals.

Valve PV-20561 will remain out of service until parts are available, and plant conditions allow inspection.

The plant Computerized Commitment Tracking System maintains the inspection as an open item but action will be taken only in the event of continued operation of Rancho Seco.

~ ^ ~ ~

C 1'8EI["

l^

l*

NRC FsemJa&A U.S. NUCLEAR REGULATORY COMMIS$10N LICENSEE EVENT REPORT (LER) TEXT CONTINUATION ApeRovEo oMe No siso#o4 EXPIRES: 8/31/RB F ACILITY NAML 11p DOCKET NUMBER (2)

LtR NUMBER (6)

PAGE (3)

,[,4 [u*,y,"

YEAR c

Rancho Seco Nuclear Generatina Station 0 151010lo13111 2 89 0101 1 Ol1 01 6 OF 01 7 TEXT fit more spece e neuned, une addotenet NRC Form 366Ksl(1h As a result of the failure to establish decay heat cooling within the 12 hour1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> requirement. the following corrective actions were proposed and completed:

l Modify procedures to add additional temperature curves for boron concentration.

o to reduce the time required to borate prior to initiation of cooldown.

Modify plant procedures to allow earlier performance of procedure SP.520 during o

power reduction to preclude impact on a Technical Specification time requirement.

Develop a method to review new procedures for impact on Technical Specification o

required Action times.

As a result of the Root Cause Investigation the District identified and evaluated deficiencies in several areas.

Major corrective actions completed prior to plant startup were:

Engineering evaluation with associated rework and testing of AFH components and o

system to ensure operability.

Clarification for District buyer personnel to formalize. supplier technical information requests.

Modification of the work planning program to ensure work requests are stand alone documents.

Revision of Maintenance procedures to address lack of precautions for testing l

o requirements.

Revision of Maintenance procedures to ensure incorporation of vendor warnings into the test plan.

l Implementation of periodic testing for the AFH overspeed component.

o Addition of System Engineers into the review process for component post o

maintenance testing.

l

.u... a v. u,..-sm m owe gRgroRM

.A

W 4RC Form.3660 U.S. NUCLESR REGULOTORY COMMIS$10N LICENSEE EVENT REPORT (LER) TEXT CONTINUATION APPaoveu ous wo. mo-oun EXPIRES: 8t31/88 PCCtLITV NAME (1)

DOCKET NUMBER (2) 5 LER NUMBER 16)

PAGk (3)

"=PP trJS3:

"aa Rancho Seco Nuclear Generating Station 01510 l o l o 13 l 1l 2 8l9 010l 1 0 l 1; 0l7,OF Ql 7 VEUT lif more spece Je required, une eddotene!NRC hovm 30M's)(116 Major corrective actions completed after plant startup were:

Notification to the governor' vendor reinforcing the requirement to obtain design information and change approval from the District.

Notification to all vendors emphasizing the requirement to explicitly indicate component changes on the appropriate procurement form.

. Training of System Engineers on the procurement process, specifically correspondence with the vendor.

Training to ensure the vendor procurement documents are routed to the cognizant System Design Engineer.

Revision to the procurement dedication procedures to emphasize transmission of implementation / testing requirements into the work planning process.

l Development of a new Post-Maintenance Testing program.

Previous _Similar Events at Rancho Seco Reportable Occurrence 77-2 reported both AFH pumps simultaneously out-of-service for 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br /> 9 minutes with the plant at power operation. Pump P-319 was inoperable for maintenance work.

Pump P-318 was discovered out-of-service during operator rounds due to a failure to reset the overspeed trip.

Reportable Occurrence 80-46 and LER 80-46, Revision 1 reported the discovery on two occasions of.the P-318 overspeed indication in the tripped condition.

No specific cause was determined and a review of tN incidents concluded that positive indication for the tripped condition was not readily apparent.

LER 83-41 reported the discovery cf a closed cooling water supply valve to the AFH turbine bearings.

LER 84-25 reported a reactnr trip due to a rapid change in turbine load.

Subsequent to the trip, Control Room Operators attempted to secure the AFH turbine, but the steam admission valve traveled to mid-position. On investigation after the i

malfunction, valve stroke time was found to be excessive and was reworked to restore normal timing.

ggaau m,

.v.,. m, mea sn mn