ML20113H054

Rev 1 to Final Deficiency Rept DER 83-61 Re LPSI Pumps Failing to Start & Produce Required Head.Initially Reported on 830914.Caused by Hard Contact Between Impeller & Case Ring

ML20113H054
Person / Time
Site:	Palo Verde
Issue date:	01/11/1985
From:	Van Brunt E ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR
To:	Kirsch D NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V)
References
ANPP-31676-TDS, DER-83-61, NUDOCS 8501240627
Download: ML20113H054 (11)

Text

4 1

• ' 0a- 50 Arizona Nuclear Pcwer Project P.O. BOX 52034

• PHOENIX. ARIZONA 85072-2034 . . . _LJ J W l5 l.7 g -

r January 11, 1984 , , .

ANPP-31676-TDS/TRB " '?O1 U. S. Nuclear Regulatory Commission Region V 1450 Maria Lane - Suite 210 Walnut Creek, CA 94596-5368 Attention: Mr. D. F. Kirsch, Acting Director Division of Reactor Safety and Projects

Subject:

Final Report, Revision 1 - DER 83-61 A 50.55(e) Reportable Condition Relating to LPSI Pumps Failure To Start And Do Not Produce The Required Head.

File: 84-019-026; D.4.33.2

Reference:

A) Telephone Conversation between P. Johnson and R. Tucker on September 14, 1983 B) ANPP-28001, dated October 13, 1983 (Interim Report)

C) ANPP-28627, dated January 16,1984 (Time Extension)

D) ANPP-28883, dated February 15, 1984 (Time Extension)

E) ANPP-29293, dated April 13, 1984 (Interim Report)

F) ANPP-29841, dated June 27, 1984 (Time Extension)

G) ANPP-30134, dated August 9, 1984~(Final Report)

Dear Sir:

Attached is Revision 1 of our final written report of the Reportable Deficiency under 20CFR50.55(e), referenced above. .This revision clarifies the Condition Description and provides updated information on.

the Corrective Action which has been taken.

Very truly you ,

EP. cut E. E. Van Brunt, Jr.

nJ H APS Vice President Nuclear Production ANPP Project-Director EEVB/TRB:nj 4

Attachment:

cc: See Page Two 8501240627 840111 i PDR ADOCK 05000528 S pyg .

1 e R'd 7

l' Mr. D. F. Kirsch DER 83-61 j Page Two ec: Richard DeYoung, Director Office of Inspection and Enforcement U. S. Nuclear Regulatory Commission-Washington, D. C. 20555 T. G.. Woods, Jr. I D. B. Karner (

W. E. Ide l D.-B. Fasnacht A. C. Rogers L. A. Souza D. E. Fowler i T. D. Shriver C. N.-Russo J. D. Houchen J. R. Bynum J. M. Allen A. C. Gehr W. J. Stubblefield W. G. Bingham R. L. Patterson.

.R. W. Welcher H. D. Foster D. R.' Hawkinson R. P. Zimmerman M. Matt T. J. Bloom D. N. Stover L. Clyde-D. Canady J. E. Kirby Records Center Institute of Nuclear Power. Operations 1100 circle 75 Parkway, Suite 1500 Atlanta, GA 30339 Mr. E. Licitra U. S. Nuclear Regulatory Commission Phillips-Building 7920 Norfolk Avenue Bethesda, MD 20814

'i FINAL REPORT - DER 83-61 DEFICIENCY EVALUATION 50.55(e).

-ARIZONA PUBLIC SERVICE COMPANY (APS)

PVNGS UNITS 1, ?> 3 t

I. Description of Deficiency The-Low Pressure Safety Injection (LPSI) Pumps are supplied by Combustion Engineering (C-E) and are manufactured'by JIngersoll-Rand-(LR) and include 500 hp Westinghouse motors.

4 . They are identified by tag numbers as follows:

4 1MSIA-P01' 2MSIA-P01 3MSIA-P01.

JMSIB-P01- 2MSIB-P01 3MSIB-P01 While performing preoperational' testing at Unit-1 on.

March 18, 1983, LPSI pump JMSIB-P01 was run for approximately three minutes with the pump suction valve IPSIB-UV-652 closed. A restart of the pump was attempted approximately three hours later and the motor failed to start (i.e., tripped out on the time overcurrent protection circuit which was set

'at 6 seconds). The pump was disassembled and inspection revealed surface damage (galling) to the pump lower case wear ring and mating surfaces or the impeller. Also, the lower-case ring was angularly displaced approximately 3/4". This incident was the subject of DER 83-21. The corrective action was to repair the rough surfaces by smoothing with a stoning operation and replace the lower case ring set screws.. The j' pump was then retested successfully.

-During the period of May 16 to May 19, 1983, two (2) more failures to start the same LPSI pump occurred due to motor.

l time overcurrent trips. -Disassembly and inspection revealed galling had_ occurred again on the lower case year ring and impeller mating surfaces and the lower case ring was angularly

, displaced approximately 3/4". As before,.the corrective action was to smooth the impeller and case ring by stoning.

Subsequently, this DER (83-61) documented that on June 14, 1983, LPSI pump 1MSIB-P01 again failed to start i during the continuation of the preoperational testing a'ter ,

running ' for approximately , twenty minutes. At this point, the l condition was attributed to a motor problem due to the fact i that after each failed start, :the pump was free to turn but i was not coming to speed. Therefore, the LPSI Unit 1 motor was ,

replaced with a Unit 3 LPSI pump motor.

On June 19, 1983, the same LPSI pump with the Unit 3 motor i

failed to start after running successfully for two bours. On  !

June 21, 1983,- with the'IR representative present, the pump was' disassembled and inspected. Galling of the lower' ring surfaces had occurred again and the lower case ring was

< angularly displaced approximately 3/4".

, l

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ , . _ _ . _ . __ - , . . , ,. _ _ ..m . __ . _ , _ -_ - . ,

. . - - - - ,.~. . -. . _ . . .. --

1 <

, J #,C ,

Mr. D. F. Kirsch '

ANPP-31676' Page Two

-Under the IR representative's direction, the pump was reworked to smooth all galled areas and reassembled. Ten successful test starts were recorded on June 26. On July 6,71983, the LPSI pump again' failed Lto start.

L On October 8,1983, Unit 2 LPSI pump 2MSIA-P01 failed to start (6-second overcurrent trip) after 41 succcessful starts and accumulation of 66 hours7.638889e-4 days <br />0.0183 hours <br />1.09127e-4 weeks <br />2.5113e-5 months <br /> of running time, reference

' NCR SM-3026. Disassembly revealed heavy galling at the lower impeller to case ring fit and the lower case ring was angularly displaced approximately 3/4", the same as observed after the Unit 1 pump failures to start.

~ Miscellaneous

1. NCRs SM-2756 and SM-2757 are also included in this DER (83-61) to document.that test data indicated pumps 1MSIA-P01 and IMSIB-P01 did not meet performance requirements for head-capacity (Ref. CESSAR Table 6.3.2-1)..

f 2. On December 3,1983, the impeller from LPSI pump 3MSIA-P01 was removed for installation in LPSI pump IMSIB-P01 to replace the damaged Unit 1 impeller. During inspection of the Unit 3 impeller, it was noted that the bore was out of tolerance per vendor drawing C-8X20A3DX1A. The bore should be 2.5005"+.0005". The actual bore measurement is 2.5050" L to 2.5015" (Ref. NCR SM-3418). All other LPSI pump impe11er bores were within tolerance.

3. Inspection rf running clearances on pump IMSIB-P01 on January 24, 1984~ after test runs to evaluate a preloaded bottom bearing and modified running clearances with Nitronic 60 case rings, revealed ~1ight contact (Ref. NCR-SM-3588),

Equipment Description The LPSI pumps are 8 x 20 type WDF which are single-6. age, i

centrifugal, vertical, overhung, end-suction type, j close-coupled'to Westinghouse frame 5010,. vertical solid shaft i (VSS) 500 hp, 1780 1qps 3/60/4000, squirrel cate induction i- motors.

The Containment Spra'y (CS) pumps are IR 8 x 23 type WDF which l- have a 23" diameter: impeller with matching diffuser fitted-

into the' esse 8 X 20 WDF casing as'the LPSI pumps. The 23"'

l impe11ers are optimized for a different head-capacity range than the LPSI pumps which have 20" diameter impe11ers and-Jarger diameter case ring fits.

o

__u. ._.-._;_. _ _ ._ ,

_ . - , _. m. _ _ _ _ _ _ _ . - _ _ ~ -

4 Mr. D. . F. : Kirsch '

ANPP-31676~

Page Three

. The motors are Westinghouse'frase 5809, VSS, 800 hp, 1780 rps, 3/60/4000, squirrel case induction type, with the same upper and lower bearings, mounting fit to pump, and shaft extension dimensions as the LPSI pump motors. The CS ~p ump / motor sets

~

are . supplied by C-E and are identified by tag numbers as follows:

IMSIA-P03 2MSIA-P03 3SMIA-P03 '

IMSIB-P03 2MSIB-P03 3SMIB-P03

.Because of the design similarities between the LPSI and CS pumps, the corrective actions address both LPSI and CS pumps, >

even though CS pumps have never failed' to. start.- i 4

II. Analysis of Safety Implications i

The Low Pressure Safety Injection (LPSI) pumps are used to provide core cooling flow in a shutdown cooling mode and also to provide emergency core cooling flow if there is a Loss of j Cooling Accident (LOCA). 6 The Containment Spray (CS) pumps are designed to remove Lest from the containment atmosphere in the event of a LOCA. They are also used to circulate reactor coolant to remove decay heat following plant shutdown.

• Based upon the above, the failure to start condition is  ;

evaluated as reportable under the -requirements of 10 CFR

~

Part 50.55(e) since, if left uncorrected, it could adversely.

affect the safety of plant operations.

Also, this condition is evaluated as reportable under the requiremente of 10CFR Part 21:since it constitutes a substantial safety hazard and, if left uncorrected, could adversely affect the capability to safely' shutdown the reactor.

III.A. Corrective Actions, LPSI and CS Pumps The cause of the failure to start was theorized to be hard t contact between the impeller and the lower case ring.

Accordingly, based on the recoseendation of'Coahustion ,

Engineering and Ingersoll-Rand'(Ref., 10 & 11)~and with Bechtel Engineering concurrence,;the following modifications were -implemented on the LPSI'and CS pumps for Units 1, 2, and 3'to.sitisate the' effects of the contact. These modifications

[ were accomplished in accordance with NCR SM-2156 for the Unit 1 LPSI pumps and DCPa ~ 2SM-SI-115'and 3CM-SI-115 for the Units

~

2'& 3.LPSI' pumps and DCPs ISM-SI-114, 2SM-SI-114, and

.MCM-SI-114 for the CS pumps.

P - p- dp y- W-? v- .-u.ei.., - e - w t - =m. m. , g ,p *p -s er. g-ce.t v gru.-e -

+-+-c y y.-g>-y-4

s  :

Mr. D. F.-Kirsch =

ANPP-31676-Page.Four Figure 1 attached, represents a generic cross sectional view

' of the LPSI 8 X 20 and CS X 23 WDF pumps, and is provided to identify, by ites number, the corrective action modifications.

1. The upper and lower K-500 Monel case-wear rings (Item 6) were replaced with new rings using Arsco Nitronic 60 material which is known for its gall-resistant properties (Ref. 1). The Nitronic 60 is more gall-resistant than K-M>nel and has a lower coefficient of friction. l

2. (a) The running clearances between the impeller (Ites 3) and casing rings (Item 6) on the LPSI pumps were increased to .029" .036" diametrical, (Ref. 2, Item 3). This was I

done to reduce startup contact friction forces.

(b) The running clearances between the impeller (Item 3) and the casing rings (Ites 6) on the CS pumps were increased to .025" .032" diametrical (Ref. 3), for the same reason as 2(a).

3. The impeller (Ites 3) upper and lower ring fit areas were serrated (grooved) to make them less sensitive to any contact and to minimize possible loss of pump head due to increasing the running clearances (Ref. 2, Item 4).

4. To more accurately assure centralization of the impeller in the case rings and to exclude a stackup of tolerances as a factor in the LPSI pump failure to start investigation, run-outs on the upper and lower sale rabbat fits of the stuffing box extension (Item 264) were decreased from .005" to .002" total indicated reading (TIR). Also, the run-out of the support head to motor fit was reduced from .004" to

.002" TIR and this fit doweled to limit movement after final alignment (Ref. 2, Item 2 and Ref.12). It was subsequently determined that tolerance stackups within the original ~.005" TIR are acceptable and would not contribute to a failure-to-start condition.

The root cause of the failure to start condition was determined to be associated with excessive startup shaft deflections and resonant characteristics peculiar to the particular frame 5010 500 hp motors as discussed in III.B.

During testing of LPSI IB (with its 500 hp motor and with above Items 1-4 implemented) another failure to start occurred on-January 29, 1984 (the unit was tripped manually after two seconds). As with previous failed starts, the shaft rotated

. . ~ . . . ._- _ _.. . _ __ . . _. __ ._ . -_ _

~

Mr. D. F. Kirsch p ANPP-31676 Page Five ,

i slowly-prior to the trip and was free to rotate by hand thereafter. Unlike previous failed starts, the unit was ,

successfully restarted six times on January 30, 1984. .

l Subsequent disassembly and inspection disclosed light contact L at the lower clearance, and the lower case ring was angularly l ,

displaced approrisately 1/4".

III.B. Corrective Action, LPSI Pump Motors ,

i As documented and presented in detail in the C-E report j' (Ref.'9), it was concluded that due to the shaft flexibility i in combination with transient electromagnetic starting forces, this particular 500 hp motor (frase 5010) is unsuited for reliable use with the WDF type pump, in spite of the i sitigating effects of Corrective Action Items 1-4 above.-

I Contact between impellers and case rings during startup does not in itself cause failure to start, but has been demonstrated to be a precondition of the failure-to-start '

, mechanism.

t Measurements of startup shaft end and impeller lower running i_ surface deflections in air (without case rings, stuffing box i bushing or shaft seals to restrain radial movement) have shown

> that ispeller-to-case ring contact would consistently occur with the original 500 hp LPSI pump' motors if started dry in an

, assembled pump, and would.probably also occur when started in a pump full of water. Comparable seasurements have shown that l the 800 hp CS pump motor, with its . stiffer shaft between l bearings,'1arger frame size and.different electromagnetic, starting forces does not exhibit startup deflections with either LPSI or CS. impellers which would cause contact either

' dry in an assembled pump or in a pump full of water, (Ref. 4 '

i and 6). Therefore, to minimize startup deflections, an 800 hp

! CS pump motor was installed on the LPSI pump.

It was--determined that a 100 start test with no failures would i demonstrate adequate reliability with 95% confidence for a.

l pump / motor set (Ref. 5). The following items were therefore implemented.

1. An 800 hp CS pump motor from Unit 3 was installed on the '

LPSI 1B pump (Ref. 6).

~ The pump / motor set was operated for 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> to stabilize '

f hearing temperatures. .It was then restarted 5 times (20  ;

4 ' minutes run time for the first four starts to satisfy motor l 4 . cooling requirements). Subsequent disassembly and .l inspection on March 3, 1984, revealed light contact at the 1

'Jover impeller clearance (360*)'with corresponding contact at approximately 270* of the lower case ring on the side away from the pump. discharge connection.

_ _ - _ , _ _ _ _ _ . . _ , _ _ _ . _ _ _ _ _ _, . _ _ _ _ - . , _ _ - , - ~ - _ _ - - - - - _

l L

Mr. D. F.~-Kirsch ANPP-31676 Page Six

2. The pump was reassembled and 100 additional starts were made. All runs were approximately 20 minutes duration and at 2000 to 2100 spe valve setting. The first run (after refilling and venting) started at approximately 100 spo through the minimum flow line, the same as in Paragraph I above. Subsequent disassembly and inspection (March 8, 1984) disclosed no evidence of contact at the upper ring clearance and no obvious additional . contact at the lower ring clearance, as noted in Ites 1 above. The contact surfaces of- the Nitronic 60 case rings were smooth with a light intermittent film transfer to the four impeller running surface lands.

At a March 9, 1984 meeting, APS, Bechtel, C-E, and IR agreed.that the light rubs were normal and within acceptable limits (Ref. 7). The significance of no evidence of contact on 900 of the lower case ring indicates startup deflections are not resulting in contact. The 270' contact resulted from the brief operation at minimum flow where pumps characteristically rua rougher than at higher flows. It was further agreed that the test showed this particular 800 hp motor (frame 5809 with a 6" disseter shaft between bearings) would operate reliably in combination with the WDF type pump (either 8 x 20 or 8 x 23 sizes).

3. All six LPSI pump motors from Units 1, 2, and 3 were 8 replaced by six new Westinghouse frame 5809 800 hp motors.

These new motors will be duplicates of and interchangeable with the original CS pump motors (Ref. 4). This motor replacement was accomplished in accordance w1th DCP .1SM, .l 2SM, and 3CM-SI-117.

The increased startup inrush current (800 vs 500 hp) is .

within the capacity of the diesel. generators. The LPSI

-breaker settings were changed to accommodate the GOO hp motor. In addition, the original current transformer (100/5 esp.) was replaced (150/5 asp.) and the'asseter was changed from a 0-100 to 0-150 scale meter.

4. SAR Change Notice 1180 has been initiated to revise affected sections of the FSAR for the change in LPSI pump-motors.

5. Since completion of the 100 start test and as of August 8, I 1984, Unit 1 LPSI pumps A and B havs been started 36 times and 46 times, respectively, without any. difficulties.

t Y-

, ~,.- - , . . . ,~

- - -- - .. -- - ~ _ - . . . _ . - - . .-

i- ,

' Mr. D. F. Kirsch ANPP-31676

.Page Seven

! Disassembly and inspection'of these pumps during the time peried of additional starts have not disclosed any abnormal ,

wear patterns.

Also, CS pump A has been started 48 times and CS pump B, 46 times. Inspection after additional operation revealed no abnormal wear patterns. These particular pump /sotor sets have .never had a failure-to-start problem.

i Since August 8, 1984, to date, the Unit 1 LPCI and CS pumps i as well as the Unit 2 LPSI and CS pumps have been started 1 '- numerous times with various valve alignments without any startup difficulties, i

III.C. Corrective Action, Miscellaneous

1. Regarding NCRs SM-2756 and 2757, additional field

i. performance tests after final sodifications of both Unit 1 LPSI pumps have verified that the head-capacity

! characteristics from minieue flow to; runout are acceptable, i.e., above ECCS minimus, (Ref. 8). These tests were i conducted to verify that the head loss due to increasing tba impeller-to-case ring clearances was within acceptable

! limits. These particular NCRs were assigned to this DER only because they related to LPSI pumps, and were not in any way related to the failure-to-start issue.

l 2. NCR SM-3418 was dispositioned to have the impeller returned to IR for rework to correct the bore dimension.

3. NCR SM-3588 was dispositioned to cleanup the impeller and replace the lower case ring after rubs had occurred while l, operating with a 500 hp motor on January 18, 1984. ,

-l 1

4 i

l a

i 4

. . - - . --..a

-s ,

if Mr. D. = F. Kirsch ANPP-31676

'Page Eight IV. References (1) V-CE-19299, dated Nov.-11,.1983 (2) V-CE-19218, dated Oct. 28,'1983 (3) -V-CE-19298, dated Nov.- 11, 1983 (4) V-CE-30175, dated Apr. 30, 1984 0

(5) V-CE-20584, dated Jan. 26,.1984 (6)V-CE-19784, dated Feb. 20, 1984 (7) V-CE-21628, dated Mar. 9, 1984 (8) V-CE-21710, dated June 11, 1984 (9) V-CE-30704, dated Aug. 2, 1984, Combustion Engineering Report CEN-285(V)-P, "Palo Verde Nuclear Generating Station Low Pressure Safety Injection Pumps Failure to Start" (10) V-CE-19348, dated Nov. 23, 1983 (11) V-CE-19389, dated Dec. 1, 1983 (12) V-CE-20536, dated Dec. 2, 1983 4

' 1  !,

.. -- 7

. l 1

i l  :

- ~

R ,

• E R W O

E L L

L ,

3 t t

A96

- 'AEP 3 1

RG N A R A 2T 8258.

2 26I 3M A3I 669 9 6 O 6 4 T 6 I 34t4 8 33 3 23 F 2 I 2

O - - S M A M C O 6 R

F '

N ,

O I I S ,l N

E D-T , -

X E . - '

T i

F t

i

/ / ~

S o / / .

t

{ .

/ \ 'l.

. }

/ 3, ',

i p g ._l 7 Y

/ = '

J (\ - .\

qN '

i'N r ,

- ,t -

\ \ M

- /

I

/ 4 .lq,

/

\ i

'l 1 '

\ '

/ '

3,N

/ 3 ./

l g N o

\ g -

f

_I

/ \

N =

O I

S N

E T _

XR EE P

XP 8 2 ta0U 2 6 B 8 4 5

- Ag6AI A

5 aGG l

t NN 3

3 85 2 A

5 6

4 8

2

1. A 2 8 3 II 3 2 1 FR -

F

. UE

.. TS SA -

- C

=

4 6 .

. = 2 o '

l Eb E lluh ,s9\* ,iu i l

l L

,T2- - o $$

.