ML17304A221

Justification for Continued Operation of Auxiliary Feedwater Pumps.

ML17304A221
Person / Time
Site:	Palo Verde
Issue date:	06/09/1988
From:	ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR
To:
Shared Package
ML17304A218	List: ML17304A217 ML17304A221
References
REF-PT21-88 NUDOCS 8806230164
Download: ML17304A221 (48)

Text

ARIZONA NUCLEAR POWER PROJECT PALO VERDE NUCLEAR GENERATING STATION UNITS 1, 2, AND 3 JUSTIFICATION FOR CONTINUED OPERATION AUXILIARY FEEDWATER PUMPS 8806230i64 880609 PDR ADOCK 05000528 8 PDR 1560K/0052K

ll il TABLE OF CONTENTS PAGE Executive Summary I. Equipment Description II. Safety Function III. History IV. South Texas Project 'Experience.

Engineering. Evaluation.

A. ANPP Analysis B. BW Analysis C. Pump Consultant Analysis.

D. Metallurgical Analysis.

E. Operating History F. Reliability Analysis. 10 Safety Analysis H. Root Cause Discussion 12 VI. Conclusion/Corrective Actions 14 VII. Justification For Continued Operation 15 Figure 1 - AFW Pump Cross Sectional View Figure 2 - Center Stage Piece (Enlarged)

Figure 3 - South Texas Project AFW Pump

k ~

EXECUTIVE

SUMMARY

On March 25 . 1988 the'PVNGS. Unit 1 B-Tra'in auxiliary feedwater pump failed to pass its surveillance:t'est. Subsequent inspecti'on of the pump revealed that the center stage shaft sleeve had cracked and machined the fourth stage impeller hub to the extent that the functional loss of the impeller had occurred. An extensive investigation, was .performed. by ANPP's Engineering, Safety Analysis, Licensing and Reliability staffs working in conjunction with the pump vendor, independent consultants and metallurgical analysis laboratories. The results of the, investigation indicate, that the root cause of failure is ,attributed to hydrogen 'embrittlement of the center stage shaft sleeve material. The investigation also identified that a similar ,material failure had occurred at the South Texas Project on February 28, 1988.

To prevent future AFM pump failures of this type, the pump's rotating and stationary parts will be changed to materials that are essentially immune to hydrogen embrittlement and more resistant to galling. These new materials ..are Type 410 stainless steel with a hardness in the range of 250 to 300 BHN for the rotating parts and Ni-Resist ¹2 for the stationary parts. This modification to the AFW pumps will be'ccomplished at the earliest available opportunity subject to constraints imposed by the vendors capability to support rework of the pumps and to provide the necessary supporting parts to allow rotating element replacement.

Il I

pl I il

JUSTIFICATION FOR CONTINUED OPERATION.

AUXILIARY FEEDWATER PUMPS I. E UIPMENT DESCRIPTION The equipment associated with this evaluation are the 'three Auxiliary Feedwater (AFW) pumps for each PVNGS unit. The AFW system for .each PVNGS unit consists of one Seismic Category I motor-driven pump, one Seismic Category I steam turbine-driven pump,. and one non-seismic motor-driven pump.

The nine AFW pumps at PVNGS were manufactured by Bingham Willamette (BW) of Portland, Oregon. The pumps are BW Model'x6x10-1/2 B MSD eight stage units. Each essential AFW pump, AFA P01 and AFB P01, has a rated capacity of 1010 gpm at 3280 .feet Total Dynamic Head'TDH). 'The nonessential AFW pump, AFN-P01, has a rated capacity of 1010 gpm at 2960 feet TDH. The pumps are eight stage, single suction, double volute with a horizontal split casing. The rotating element is supported with sleeve journal bearings at .both ends outboard of the stuffing box and the pump has an oil-lubricated ball thrust bearing assembly located opposite of the driver end. The configuration of the impellers on the pump shaft is commonly called the opposed impeller arrangement. The purpose of this arrangement is to reduce axial thrust loading. In th'i;s arrangement, the first four impellers are arranged on the .pump shaft facing in one direction with the remaining four impellers facing in the opposite direction. This design requires an interstage bushing assembly to separate the opposed impellers at the pump center between impellers four and eight. For additional insight into the pump construction, refer to the pump cross .sectional drawing shown in Figure 1.

The interstage bushing (center stage shaft, sleeve) is approximately 3 inches long and 0.25 inches thick. It is manufactured from. AISI 420 material, hardened to Rockwell C-50..5 hardness. The sleeve is shrunk-fit to the shaft to form a small interference fi't of 2 mils or less. The sleeve is also keyed to the shaft by a key that is common with the fourth stage impeller."'The sl'eeve is, surrounded by a stage piece forming, a hydrodynamic ,bearing."-'here isi.". a groove within the. center stage piece that spirals'round the 'center stage sleeve for pumped'luid flow. While the pump is at rest, the center shaft sleeve is resting on the center stage .piece. Upon startup, the rotating shaft is then supported by a film of pumped'lufd that is forced through. the annulus region between the center stage piece and'he sleeve by the pressure differential between the ei'ghth and fourth stages. Thus, upon startup, .there is usually contact between the center stage shaft sleeve and the center stage piece. An. enl'arged view of the center stage piece is shown in Figure 2.

II. SAFETY FUNCTION The AFW pumps provide feedwater to the steam generators for the removal of decay heat from the reactor coolant system following reactor .shutdown from any power level and until such time as heat removal via the shutdown cooling system may be initiated.

0 0 C;

f

III. HISTORY PVNGS has experienced two similar events that resulted in AFW pump degradation. The most recent event occurred on March 25, 1988 in Unit

1. During the performance of the monthly surveillance test on the B-Train motor-driven AFW pump (1M-AFB-P01), it was observed that the total delivered head was below the allowable value of 1682'.2 psid. The actual total delivered head was 1538.5 psid. This value was approximately 210 psid below the value of 1748 psid which was obtained in the previous surveillance test for this pump. The pump. was declared inoperable and disassembly was initiated. The System Engineer, who was present for the pump disassembly, made the following observations during the visual inspection of the pump's rotating assembly. First, the fourth stage impeller hub was no longer keyed to the shaft and the center stage sleeve was found underneath the fourth stage impeller. The fourth stage impeller hub was observed to be capable of rotating freely around the stationary pump shaft. There was also evidence of grinding on the center stage piece side of the fourth stage impeller hub. Secondly, the center stage shaft sleeve was also free to rotate about the pump shaft. By design, the center stage shaft sleeve is shrink fit onto the shaft and also keyed to the shaft. Visual observation showed a crack on the outside surface of the center stage shaft sleeve that went axially along the entire length of the shaft sleeve at the keyway location.

Additionally,, the key for the center stage shaft sleeve and fourth stage impeller hub could not be found during visual inspections.

The previous instance of AFW pump degradation of this type occurred on June 1, 1987 in Unit 1. This event affected 'the non-seismic, motor-driven AFW pump (1M-AFN-P01). The visual inspections of the rotating assembly for this pump yielded'any of the same observations that were described above. Specifically, the center shaft sleeve had an axial crack, the key had been sheared, and the sleeve was free to rotate about the shaft. The key had also been sheared at the fourth stage impeller hub allowing freedom of rotation of the fourth stage impeller hub about the shaft. The rotating assembly for the pump was replaced and the damaged rotating assembly was sent to the vendor for his evaluation and repair. The conclusion from the vendor's evaluation was that the damage was due to water hammer or excessive pulsations at the discharge side of the pump.

After the most recent failure, ANPP initiated a search within the industry to determine if any other plants with similar BW pumps had experienced problems of this type. The vendor was contacted and asked if they knew of any problems of this type that had been experienced by other users of their pumps. At that time, the vendor replied that he was unaware of any other failures of this type. Further investigation by ANPP revealed that South Texas Project (STP) Uni't 1 had recently experienced a failure of one of their eleven stage AFW pumps. Some of the damage reported by STP is similar to the damage experienced by the PVNGS pumps. ANPP also conducted a Nuclear Plant Reliability Data System (NPRDS) search to determine if this failure mechanism had been experienced previously at other utilities. There were no NPRDS reports of similar failures at other operating utilities.

0 IV. SOUTH TEXAS'ROJECT EXPERIENCE, ANPP has had several conversations with Bechtel 'Power Corporation (BPC) and STP personnel concerning the recent AFW pump failure at STP. A summary of ANPP's understanding of the STP experience is presented in this and the following paragraphs. The AFW pumps used at STP are eleven stage, horizontally split case, double volute, centrifugal type (see attached Figure 3) furnished by BW as their MSD model pump with a TDH of 3260 ft. at a design flow of 600 gpm. On February 28, 1988, the STP operators had been running the pump for approximately 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> when a reduction in turbine rotational speed was observed. The operators initially attributed the speed reduction to an improper governor setting and adjusted the governor to increase turbine speed. Information from STP indicates that the pump speed had decreased approximately 40 rpm over a period of one hour. This is a relatively small speed reduction on a pump that normally has a rotational speed of approximately 3600 rpm. The operators discontinued testing by manually stopping the pump when a high bearing temperature was observed. The throttle sleeve/bushing seizure was identified during a subsequent internal investigation of the turbine-driven pump.

The AFW ,pump at STP was inspected and damage to the throttle sleeve/bushing, the impeller and center stage shaft sleeve were observed. Bechtel and BW performed a detailed review to assess the extent of the damage. Their findings are as follows.

i) The throttle bushing (normally stationary) had rotated in the pump case resulting in a plastic flow of the case metal in the upper half of the case. A blue discoloration was noted on the case surface in contact with the bushing indicating that tempering of the case material had occurred. The bushing's spiral trash groove (approximately 1/16" design depth) had been ground down by <he sleeve outer surface rubbing on the inner surface of the bushing. Melted sleeve/bushing material was observed to have been extruded out to the high and low pressure ends of the sleeve/bushing pieces. The sleeve was found brazed to the bushing which was designed to have 0.004-0.006 inch radial clearance during normal operation of the pump. The remainder of the 3/8" anti-rotation pin associated with the bushing was found in place. The throttle bushing sleeve was reported to have cracked axially along the keyway, however, the key remained in place without incurring damage. It was noted that the sleeve retaining ring was not damaged and the pump shaft was not heat discolored where the shaft and sleeve are in contact.

ii) The center stage shaft sleeve was found to have cracked axially through the keyway and the key was sheared at the intersection of the sleeve inner surface and the shaft outer surface. A portion of the key remained in the keyway half of the shaft under the shaft sleeve. Additionally, the portion of the key associated with the fifth stage impeller was found to be intact and in place. Sleeve movement toward the impeller hub was P

41 0

estimated 'to be 5/16". 'The 'center. sleeve .and pump shaft had circumferential wear 'rooves (estimated depth of 1/64") at their interface. The sleeve was not found brazed to the center stage piece, but similar circumferential wear grooves. were observed at the interface of the sleeve and center stage piece where contact had occurred.

The fifth stage impeller hub was machined by the center stage shaft sleeve to a, depth of 5/16.". The .worn end of the sleeve matched the indentation on the impeller hub. The portion of the key enclosed by the impeller ,hub was not sheared and functional loss of the impeller did not occur.

STP tasked Bechtel's Materials and Quality. Services Group .with providing a metallurgical analysis of the center stage shaft sleeve and throttl'e bushing. Their findings are summarized below.

The shaft center sleeve and the throttle sleeve are made of Type 420 stainless steel,, heat treated to 51 HRC. The throttle bushing is Type 440A stainless steel', heat treated'o 32 HRC.

Both sleeves split axially in the keyway. The splits in these sleeves were caused by stress corrosion- cracking/hydrogen embrittlement, which initiated at or near a corner of the keyway.

iii) The spl'it in the throttle sleeve was not caused by metal particles from abraded 5th-stage impeller hub being 1'odged between the throttle bushing and the throttle sleeve.

iv) It is, reasonable,to conclude that the shaft center sleeve had, been cracked for some time before the throttle sleeve split.

The corrosion on the fracture faces of the shaft center sleeve supports .this. It is also possible, that the pump could have operated with the shaft center sleeve split but not seized. It is likely that the throttle sleeve split completely during the pump operation on February 28, 1988, leading to the complete seizure between the sleeve and 'the bushing and the friction heat damage to the bushing and its,housing.

v) The splits in the sleeves came first and the increase in diameter due to the splits caused the frictional heating. The 0.053 inch gap at the split of the shaft center sleeve is equi'valent to an increase in diameter of 0.017 inch, which is more than the design clearance- between the .sleeve and the shaft center stage piece. It is reasonable, .therefore, to believe that the initial friction between them was sufficient to shear the key off. Subsequently, the shaft center sleeve rotated around'he shaft inside the center stage piece, cutting into the 5th-stage impeller hub. The keyway length in the throttle sleeve is much longer than that in the shaft center sleeve.

Therefore, the key in the throttle sleeve remained in place, overcoming the frictional force that developed as a result of 4l. 0 the split. Instead, the friction between the throttle sleeve and the throttle bushing caused such an intense heat that the metals melted, squeezing some molten metal out. The sleeve and the throttle bushing were welded together. Since the 'key was stronger , than 'the 3/8-inch diameter anti-rotation pin for the throttl'e bushing, the.'atter sheared off, causing, the bushing to rotate inside the "casing. This would explain the large heat affected zone and the d'eformation around the bushing, collar.

BW has reviewed,,the 'Bechtel M&QS 'findings and'as concurred'ith the test results, conclusions and opinions for the metallurgical aspects of the STP failure. BW reported that their expe'rience in commercial applications of this,'type of pump has shown~ that .sleeve cracking does not inevitably lead to pump failures. Specifically, BW has reported'he following scenarios to STP:

"We (BW) have seen during normal maintenance one of these sleeves would be cracked with the key intact. Further, there would be little or no ,galling. The pump had continued'o operate normally since there was no galling and the recirculation losses due to the crack would be insignificant. We have also seen the situation where the sleeve cracked, then seized to,the stationary bushing and therefore, sheared the key and the shaft then became the running part inside the sleeve. The pump then continued to run normally since the sleeve was captured."

Currently, STP is implementing the following material modifications to their pumps.

i) Change the material of the stationary parts (case rings, center stage piece, throttle bushing and series stage pieces) to

'Ni-Resist ¹2.

ii) Change the material of the rotating parts to AISI 410 with a hardness of 250/300 BHN..

iii) Change the key material from 416 to 410 stainless steel.

ENGINEERING EVALUATION ANPP initiated a mul'ti-disciplinary review of the AFW .pump problems. The review involved several different groups within ANPP as well as many outside organizations. The Mechanical Group of ANPP's Engineering Evaluations Department (EED), took the lead in the investigative efforts which were aimed at determining a root cause of the AFW pump degradations. The Technical Support group of EED was involved to provide metallurgical analysis support and they contracted with the independent laboratory METL to perform the actual metallurgical testing of the center stage shaft sleeve. The Mechanical and Reliability Groups of ANPP's Engineering Department provided technical support and reliability analysis -to support this effort. ANPP's Nuclear Fuel Management (NFM)

Department was contacted to determine the impact of reduced AFW pump lh Il P

t

flows on the accident analyses. NFM also consulted with Combustion Engineering for their evaluation. ANPP also obtained'echnical support from Bechtel Power Corporation (BPC) and the pump vendor, BW, to aid in the evaluation of the ,AFW pump degradations. BPC then involved an independent pump consultant and BPC's Materials and Quality Services (MSQS) group. BW contracted with an independent laboratory, Oregon State University, (OSU), to perform metallurgical examination of the center stage shaft sleeve. This large team was assembled'y ANPP to thoroughly investigate the observed AFW pump degradations. The significant findings from this investigative effort are discussed below in the analysis section.,

V.A ANPP ANALYSIS After ANPP's visual inspection and engineering evaluation of the damaged AFW pump, the following failure scenario was developed'. The initiating event was the crack in the center stage shaft sleeve.. The existence of the crack allowed the,,shaft sleeve to expand which eventually resulted in the shearing of the key due to the frictional forces between the sleeve-and the center stage piece. The ,shearing of the key gave rotational freedom to the shaft sleeve. The differential pressure that is normally developed across. the shaft sleeve is approximately 800 psid. This is the pressure differential between eighth stage and fourth stage pressures.

This large differential pressure forced the center stage shaft sleeve into the fourth stage impeller hub. The shaft sleeve ground into the impeller hub removing hub and key material. Finally, the sleeve reduced the key in the impeller to such a dimension that the impeller torque sheared the remaining length of the key. The fourth stage impeller was then free from the rotation of the shaft and the developed head of the pump was reduced.

V.B BW ANALYSIS On April 4, 1988, BW met with the ANPP site representatives to discuss the indications found while disassembling the Unit 1 B-Train AFW pump.

taken of the damaged rotating assembly.

BW reviewed the video Discussions were then held's to the condition of the Unit 1 N-Train AFW pump rotating assembly from the failure of June, 1987. BW stated that the damage,to the two rotating assemblies~ is very'similar. BW was given half, of,,the center " stage, shaft sleeve for their failure analysis. BW then witnessed the starting of the Uni:t 1'-Train AFW pump which had'een instrumented on the suction side of the pump with pressure gauges.

During the starting of the B-Train AFW pump, swings from 0 to 60 psi for 3-5 seconds were observed on the .suction side pressure gauge until the pump steadied out. After the run, BW performed a walkdown of the suction piping for the AFW pump. The 'BW representative noted that there was nothing unusual about the 'PVNGS suction piping arrangement that would lead them to believe that this is a problem for mini-flow pump starts.

They were then taken to Unit 2 to inspect the N-Train AFW pump which had been disassembled for this review. The center stage shaft sleeve for this pump was noted to have moderate rotational marks, but no bluish tempering was found. The sleeve was in its design .position, and no Il t'/

visual signs oE any diEferential rotational movement between the shaft sleeve and the fourth stage impeller hub was found. The lower center stage piece revealed light to moderate signs of rubbing contact. The upper halE showed very light signs of contact.

The observations made by BW during their trip to PVNGS and the subsequent review of the B-Train AFW pump damage at their facility have led them to believe that abnormal pump operations may have contributed to the pump failures following the sleeve cracks. Specifically, BW postulates that the pumps, which have incurred the failure of the center sleeve, have either been subjected to dry starts (loss of suction pressure) or starts under conditions (two phase flow) where the auxiliary feedwater flow is aligned for steady state flows at or, near the design flow (1010 GPM) of the pump. To shear the key, a considerable torque must act on the sleeve. BW performed an analysis to estimate the maximum frictional forces which can occur during pump upsets. Particularly, the case of pumping a two phase fluid (water/vapor) was considered. A simplified model was used in which it was assumed'hat, in the worst case, 4 of the 8 impellers would be half filled with water and the remainder with vapor. The results are:

i) With a friction factor of 0.3 (dry steel against steel), the torque developed by this upset is only about 10$ of the torque necessary to shear the key.

ii) To shear the key the sleeve must at least temporarily gall. If this happens, the necessary torque can easily be reached, as the steady state motor torque is about 4 times larger than is needed to shear the key. The Eact that the center stage .pi:ece does not rotate, although held by a relatively small pin only, indicates that the shearing of the key happens rapidly with not enough time to accelerate the heavy center stage piece. This would support the argument that at least a temporary sudden galling of the sleeve is necessary for shearing the key.

BW indicated that galling marks on the center stage piece and center sleeve are strong evidence that dry starts or normal starts with subsequent two phase flow have occurred for the failed pumps. Each of these phenomena are discussed below.

) Normal Starts. BW estimates that the motor-driven pump starts in less than one second. The inertia of the water in the long suction line (approximately 200 feet) prevents the water from moving into the eye oE the first stage impeller at the same rate as the pump is removing water from the case.

.Consequently, the pressure at the suction nozzle of the pump drops below the vapor pressure oE the water in the suction line. This results in two phase flow through the .pump. The two phase flow can lead to one of the following two occurrences or a combination of both. First, the impeller(s) can contain a mixture of vapor and water which is not evenly distributed throughout the impel'ler. This imbalance condition causes the shaft to move erratically with the largest shaft deflections Ii

'J

taking place at the center stage sleeve. The sleeve contacts/rubs the 'enter stage piece leading to the observed galling. Secondly, a two phase condition inside the pump could exist over a large portion of the pump which reduces the hydrodynamic lift lift effect on the sleeve.

leads to the sleeve The reduction in contacting the center hydrodynamic stage piece which results in the galling of both pump parts,.

Compounding the above effects of two phase pump operation, the water column which separated at the pump suction nozzle will rejoin when the water in the suction line starts to move. The rejoining results in a water hammer, placing an additional load on the shaft which may be in a deflected state or moving erratically, thus causing or ampli.fying the radial forces which are necessary to deflect the sleeve enough to contact/rub the center stage piece.

ii) Dry Starts. BW postulates that the suction valves may have been closed when the pump was inadvertently started. This would lead to the loss of the hydrodynamic lift effect at the center sleeve. Heavy rubbing on the lower half of the center stage piece would occur with subsequent galling of the parts.

V.C PUMP CONSULTANT ANALYSIS An independent pump consultant was retained by BPC to perform an evaluation of the AFW pump degradation problem at PVNGS. This paragraph summarizes the consultant's significant findings. The consul'tant performed an inspection of the damaged rotating assembly of the B-Train motor-driven AFW pump 'before the assembly was shipped to BW. His evaluation corroborated the failure scenarios developed by ANPP.

Specifically, he stated that the end to end split of the shaft sleeve initiated the failure. Once the crack was fully developed, the sleeve would expand due to residual stresses to allow contact with, the bushing.

The resultant torque sheared off the key and the sleeve, under eighth stage pressure, began drilling, into the fourth stage impeller hub. After enough metal was removed from the impeller hub and shaft key, the torque sheared the shaft key which freed the impeller from positive drive.

Another interesting finding from this pump. consultant is that there is no reason to suspect that further mechanical problems would occur following the loss of the fourth stage impeller that would prevent the continued use of the pump at a reduced flowrate. The consultant also expressed concerns regarding the shaft sleeve design. In particular, the consultant was concerned by the thickness of the shaft sleeve, the hardness of the material, and the stress concentrations created by the 0.125 inch deep keyway in the 0.250 inch thick sleeve material. As a result of his concerns, the consultant recommended that the thickness of the center stage shaft sleeve be doubled and that the material should be replaced with a material that is not nearly as hard as that currently used. Further evaluation of this recommendation has led ANPP to conclude that increasing the shaft sleeve thickness is not practical due to the increase in axial loads on the sleeve that would result from the change.

Ik II V.D METALLURGICAL ANALYSIS, This .paragraph summarizes the metallurgical analysis results that have been obtained to date. ANPP metallurgical analysis of the cracked shaft sleeve has concluded that the failure is due to hydrogen embrittlement cracking. Evidence to support this include 'Scanning Electron Microscopy (SEM) showing intergranular cracking, an average material hardness of Rockwell C-50, and ,applied radial forces on the shaft sleeve. The susceptibility of martensitic stainless steels with high hardnesses to.

hydrogen embrittlement is well documented and supports ANPP's conclusion. There was no evidence to suggest the ductile overload or fatigue modes of cracking. Secondary heat induced cracks were also identified on the shaft sleeve inside diameter,, however, these cracks were a result of severe galling and intense localized heating. Bechtel M&QS has examined the failed shaft sleeves for PVNGS and STP. The examination of the PVNGS sleeve confirms the ANPP metallurgical examination results of hydrogen embrittlement cracking. Additionally, Bechtel M&QS has found significant similarities between the ANPP and STP sleeve failures that suggest that the fracture modes are the same. These similarities are summarized below:

i) The material is Type 420 stainless steel, which was hardened to a high hardness level (50/51 HRC). Steels, including chromium stainless steels, in a high hardness condition (higher than 38 HRC) are known to be susceptible to hydrogen embrittlement cracking.

ii) The fracture occurred in or around the keyway corners where the stress .,(residual tensile stress in the circumferential direction from heat treating :and shrink fit) would be the highest.

iii) The fractures initiated at the keyway, surface rather than at the outside surface of the sleeves.

iv) The sleeves show evidence of corrosion in the areas of fracture initiations.

v) The fracture faces of the splits are intergranular.

V.E OPERATING HISTORY ANPP conducted a review of the operating history of the AFW pumps in each unit. One of the factors that was investigated was how many full flow pump starts the B-Train AFW pump had experienced in each unit. The results of the review indicate that the Unit 1 pump had experienced an estimated 27 full flow pump starts,. the Unit 2 pump had 3, and the Unit 3 pump only had 2 full flow starts. The reason why the Unit 1 B-Train AFW pump has experienced more full flow starts is attributed to additional preoperational testing that was conducted only in the lead unit. Based upon the BW hypothesis concerning galling during full flow pump starts and the operating history data, we would expect to have considerably more galling on the Unit 1 pump than the other pumps. This has,been partially 0

confirmed by the inspections conducted'n the B-Train AFW pumps in Units 1 and 2. As an additional note, full flow starts of the A-Train and N-Train AFW pumps are not a concern for this investigation due to design and procedural considerations which preclude full flow starts of these pumps. Additional operating information has been gathered (and estimated) to aid in the reliability analyses that ANPP has performed.

The demand'and run times associated'ith the :Unit 1 pumps and the estimated time and'emand for'he" Units 2 and' A'FW,pumps . are summarized in the following table.

Unit 1 AFW'UMP Dt IRUN

~Pum 200 331 l'fter TIME STARTS

'A'Pum 'B'Pum THROUGH 03 28 88 249(2) 606(2) 147 3643

'N',

Pump

'N'ailure(l) 63 959 Ht Unit 2 (Est. ) Dt 123 109 143 Ht 200 330 1960 Unit 3 (Est.) Dt 31 124 27 Ht 80 390 560 TOTAL Dt 354 482 317 63 Ht 611 1326 6163 959 Dt Start/Stop Cycles Ht - Run Time (Hours)

Notes: (1) Includes starts and run hours that this pump received in Unit 3 prior to being moved to Unit 1 in June, 1987.

(2) Uni:t 1 AFW Pump B fai'led on 3/25/88'.

V. F RELIABILITY ANALYSIS'n order to evaluate the impact of the pump design deficiency on AFW system reliability, the PVNGS AFW pump data was systematically analyzed and the results input into an AFW system reliability model. This reliability analysis was based upon the AFW pump operating history information presented previously and the observed failures at PVNGS. The PVNGS AFW pumps have experienced two failures (i.e., degraded flow events) in approximately 1216 demands, with both failures having occurred in Unit 1. Note that the AFW pumps at PVNGS have not experienced any pump seizures due to the hydrogen embrittlement cracking phenomena. A statistical analysis of this data was performed assuming that hydrogen embrittlement cracking occurs randomly in time. The results of the

-10'-

4l analysis indicate that the best estimate incremental risk incurred from operating with this deficiency for a period of 18 months (i.e., a full cycle of operation) is. less than 'that which results from the removal of the B-Train AFW pump from service for 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. Technical Specification 3.7.1.2 allows one AFW pump to be removed from service for 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> without requiring a plant shutdown. ANPP believes .that this reliability analysis is conservative since failure of .the fourth stage impeller was conservatively assumed to result in the total loss of flow from the pump. In conclusion, the reliabi'lity analysis demonstrates that the AFW system reliability will not be significantly degraded by the AFW pump susceptibility to hydrogen embrittlement in the interim period until the replacements can be implemented. ANPP has previously committed (see referenced letter dated June 3, 1988) to replace the rotating elements in the AFW pumps as soon as the new rotating elements and necessary supporting parts become available from,the vendor. ANPP believes that this replacement work can be .performed in much less time than the 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> allowed by the Technical Specifications. This reduces the risk to the operating units of performing the replacement work.

V.G SAFETY ANALYSIS To complete the evaluation of the impact of the AFW pump deficiencies on plant operations, the impact of the AFW pump failures must be evaluated.

Both AFW pump failures at PVNGS have involved the loss of only the fourth stage impeller. ANPP's NFM Department was contacted to perform an evaluation of the affect of the degraded AFW pump performance on the relevant accident analyses. NFM first determined the impact of a loss of the pump's fourth stage on the amount of flow delivered to the steam generators. Information was also obtained from BW as to the impact on the pump's flowrate of the loss of the fourth stage impeller. The NFM analysis indicates that one AFW pump will deliver 559 gpm to the steam generators following the loss of the fourth stage impeller. This is compared to the 750 gpm which is assumed in the current accident analyses. It should be noted that the degraded AFW pump will still be capable of delivering this reduced flow to the steam generators under all conceivable operating pressures (i.e., up to the lift setting of the main steam safety valves). Once an estimate of the degraded pump flowrate was obtained, CE was asked to evaluate the impact of this reduced flowrate on the PVNGS accident analyses. CE responded that they had previously performed some calculations for a similiar 3800 MWt plant. The calculations concluded that an AFW flow of 440 gpm would be sufficient to maintain adequate heat removal for the limiting LOCA event and the limiting non-LOCA event. CE also reviewed the long term cooling function of the AFW pump for the natural circulation cooldown event. CE determined that a flowrate of not more than 450 gpm would be required after the first hour of the event. For the first hour of the event, the existing inventory of the steam generators along with the reduced AFW pump flow is sufficient for decay heat removal. In conclusion, the NFM evaluation has provided confidence that the plant will respond adequately to the postulated accidents assuming the degraded AFW pump performance associated with the loss of the fourth stage impeller. Additionally, ANPP and the independent pump consultant believe that the degraded pump will be capable of continuing to deliver the degraded flow for an

0 1

extended period of time following the loss of the fourth stage impeller.

This is supported by the fact that the B-Train AFW pump was started twice for troubleshooting immediately following the fourth stage impeller failure. The pump was operated for a total of 30 minutes and the System Engineer was present for both runs. The System Engineer did not observe any abnormalities, unusual noises, or increased vibrations from the pump.

V.H ROOT CAUSE DISCUSSION The relevant data from each of the different groups investigating this problem has been presented previously. This portion of the report discusses the root causes that have been identified during the investigation. Two separate root causes have been identified for the failure scenarios discussed previously. Before these root causes are discussed, it is noted that all parties involved in the investigation are in agreement on the end portion of the failure scenario. Specifically, the different groups looking into this event agree that galling between the center stage shaft sleeve and the center stage piece must exist in order to generate high enough forces to shear the shaft key. After the shaft key shears, the shaft sleeve is free to rotate which leads to machining of the fourth stage impeller hub. The fourth stage impeller hub eventually breaks free from,,the shaft. However, two failure mechanisms have been developed to explain how and why the galling of the shaft sleeve occurs. These two failure mechanisms are presented below.

i) Suction Transients The galling of the center stage shaft sleeve to the center stage ,piece could occur as a result of suction transients on the AFW pumps. Sucti'on transients can be caused by either plant operation or plant design. Normal starts of the motor-driven AFW pumps can lead to a condition where the pressure at the suction nozzle of the pump drops below the vapor pressure of the -water in the suction line. This is due to the inertia of the water in the suction line which prevents the water from moving, into the eye of the first stage impeller at the same rate that the pump is removing water from the case. This, condition can result in galling of the center stage sleeve by either causing an imbalance of the pump shaft or by reducing the hydrodynamic li;ft effect at the sleeve location.

Another plant operating problem that can lead to galling occurs if the non-seismic AFW pump (AFN-P01) is started with at least one of its two motor operated suction isolation valves closed.

Again, this leads to an inadequate suction water supply for the pump which could lead to galling of the shaft sleeve due to loss of the .hydrodynamic lift effect at the center stage shaft sleeve. One possible plant design problem that could lead to galling is long horizontal runs of suction piping impacting pump startup. If the piping is not adequately vented following system maintenance, then air could be ingested at the pump suction again leading to galling of the center stage shaft sleeve.

0 Il I

Hydrogen Embrittlement Cracking The galling of the center stage shaft sleeve could have been a direct result of the .observed axial crack in the shaft sleeve.

Specifically, the crack in the shaft sleeve was initiated by intergranular attack. When. the crack .had fully propagated, 'it reduced the diametral clearance between the center stage .piece and the shaft sleeve. 'This led to increased friction between the center stage, piece and the, shaft sleeve which resulted in galling.

ANPP believes that the root cause of failure is due to hydrogen embrittlement cracking or a combination of hydrogen embrittlement cracking and suction transients. From the evidence cited below, ANPP concludes that the dominant failure .mechanism is hydrogen embrittlement cracking and that sleeve .cracking is a prerequisite condition for suction transients to contribute to pump failures. ,Based on this conclusion, ANPP'ecommends that corrective actions to prevent further pump failures address the concerns of both failure mechanisms previously cited.

i) .Primary Root Cause (hydrogen embrittlement cracking).

'etallurgical analysis has indicated that the chief mode of cracking on the STP and PVNGS center, stage shaft sleeves is cracking due to hydrogen embrittlement. The 'ntergranular above conclusion is supported by fractographic examination along the fractured face of the, failed sleeve. The micro hardness measurements indicate that during the early sequence of the event, the sleeve had rubbed'r galled against the center stage piece. Therefore, it can be concluded that the sleeve had expanded sufficiently to reduce and/or possibly eliminate cooling/lube water, which would result in an upset condition causing the key to shear and the pump to fail.

The metallurgical results of the STP failure have shown that the center sleeve. contained a 0.053" wide crack. This crack size is, sufficient to allow the sleeve diameter to increase by approximately 0.017" which is more than the design diametral clearance between the sleeve and bushing.

During the April,, 1988 inspection of the non-seismic motor-driven.,pump (2M-AFN-P01), BW's inspection of the pump indicated that the lower half .of ,the center stage piece showed ,signs of, moderate rubbing."", BW believes that this is an indicat'ion that dry starts have occurred. Further investigation by Nondestructive Examinati'on (NDE) revealed that both the:center stage and throttle sleeves did not have any ID or OD cracks.

Sleeve cracking existed for the two observed PVNGS pump failures (1M-AFN-P01 .and 1M-AFB-P01).

Secondary Root Cause (hydrogen embri'ttlement cracking/suction transients):.

Il I

0

'h r~

'Sleeve" cracking, and suction transients- do not ,exist simultaneously 'for the PVNGS'umps that have been inspected

~

(2M-AFN-P01 and 2M-AFB-P01).

'During the BW trip to PVNGS in Apri'1 of 1988, a start (estimated to be one second or less) of the B-Train AFW pump, 1M'-AFB-P01, was performed; The .suction line was fitted with a compound gauge (vacuum/pressure) gauge and the pump was aligned for minimum flow operation. In this configuration, the pressure at the pump suction pegged the pressure gauge to less than zero. Based on BW's observations and subsequent analysis of the pump, start during minimum flow operation, BW indicated that pump starts with the pump aligned for flows at or near the design flow, would degrade suction pressure, to the extent that the pump would operate .under two phase conditions.

Interviews with ANPP personnel 'have, indicated that pump 1M-AFB-P01 has lost suction flow at least once over the operating history of the pump. During the performance of startup procedure 91HF-1AFOl on July 15, 1984, the pump's sucti'on was transferred from the Condensate Storage Tank (CST) to the Reactor Makeup Water Tank (RMWT). During the transfer sequence, the pump discharge pressure dropped and the test was discontinued. A subsequent investigation.

revealed that the suction line from the RMWT was not vented and the pump :had become air/vapor locked with the resulting loss of discharge pressure. To complete the test, the suction line from the RMWT was vented, the pump was restarted and proper discharge pressure established when testing continued. The pump was not inspected prior to the continuation of .testing. Additionally, Engineering is not

,aware of any events of this type in Units 2 and'.

BW has stated that a shaft sleeve crack by itself is,not always a sufficient condition for pump failures.

VI. CONCLUSION CORRECTIVE ACTIONS From the review of the STP failure, it is clear that this. type of failure (i.e., seizure of the throttle sleeve to the bushing) could occur at PVNGS'. Therefore, the corrective actions must address the STP failure.

Additionally, the recommendation from the root cause analysis section was that any design changes should consider both postulated'oot causes. BW and ANPP have conducted a comprehensive review of the pump design., The objective of this effort was to incorporate design changes which will eliminate the possibility of a failure due to the .two root causes identified. ANPP has completed an Equipment Change Evaluation (ECE-AF-020) to address material modifications for the stationary and rotating pump parts. The material modifications were selected to maximize the material',s resistance to the hydrogen embrittlement phenomena and lower its susceptibility to galling to produce a more forgiving design. In addition to the material changes, the keyway design for the throttle and center sleeves will be optimized to reduce the il 0 1

operating stresses inherent to the keyway. 'The modifications currently being implemented include:

i) Stationary parts (e.g., case rings, center stage piece, throttle bushing and series stage pieces) to be changed to Ni-Resist P2.

ii) Rotating parts -(e.g., throttle sleeve, center stage sleeve and impeller wear rings) to:be changed to 410 stainless steel with a hardness in the range of 250 to 300 BHN.

iii) Keyway design for the throttle and center sleeves is to be optimized. It is noted that "optimized" is intended .to mean that the fillet radii of the keyways are to be as large as possible without causing excessive bearing, stresses due. to, reduced contact area between the key and its mating parts.

iv) Change the key 'material from 416 to 410 stainless steel. This modification is being implemented's a design enhancement.

These modifications will be implemented in the nine AFW pumps as soon as

,the new rotating elements, and necessary supporting parts are available from the vendor. The repl'acement ,may. involve .entering, the Technical Specification action statements for units that are operating.

Additionally, one spare rotating element will be kept on-site at all times as a contingency. ANPP currently has three spare rotating elements for the AFW pumps. Two of these are currently at the vendor's facility for rework and the third one is on-site.. The replacement effort can be conducted in series by modifying two rotating elements at a time.

One of the potential contributing causes of the AFW pump degradation is full-flow starts of the ,AFW pumps. Presently, full-flow starts of the Train B AFW pump occur only during performance of Integrated Safeguards (ISG) testing. To eliminate this potential cause of degradation to the pump, the ISG test procedure will be revised to eliminate the full-flow pump start. The B-Train AFW pump will now be started on. minimum flow recirculation. The minimum flow start is also closer to the conditi'ons that the pump will see pose-accident due to the fact that the AFW pump discharge valves open slowly in relation to the acceleration of the pump.

VII. JUSTIFICATION'OR CONTINUED OPERATION ANPP has conducted an extensive investigation of the AFW pump failures experienced at ANPP. From, our investigation, we have concluded that the continued'perati'on of the PVNGS units does not pose .an undue risk to plant safety. This conclusion is based on the foll'owing considerations:

i) 'PVNGS has ,experi'enced two-events where separate .AFW pumps have.

lost their fourth stage imp'ellers. Calculations have shown that the reduced AFW pump flow, caused by the, potential loss of a fourth: stage impeller, i:s sufficient to meet safety analysis requirements for decay heat removal".

0 1

ii) To correct the material deficiencies identified wi'thin this report, ANPP will pursue the rework of the AFW pump rotating elements in an expeditious manner., This will reduce the amount of operating time until rotating element: ,

replacement is completed.

iii) During the interim period (until rotating element replacement is completed), AFW'yst:em reliability is: not significantly degraded by the material deficiencies due ,to the low, probability, that this event would occur simul'taneously in all three pump trains.

I ll

FIGURE l. ARl PUMP CROSS SECTIONAI VIW 4 X 6 X lOB MSD 8 STAGE

'I3 114 IrI I65 Ir>

S II3 2ss ~rs II3 Il3 I~255 208 Ill l47 255 II3 8 8 6 A ~8 F 6 H A A 8 8 0

~C Ir6 Ir4 r r/I/ rr J' /' J/r'rr'. r

/

A A

)

/ OIII so

~R;~fR To Avxzouvr DRY//I/6 Oe.SrI/Crag i,Box Dfr~cs.

//'

/ ///'r///

tl I

'I ~ I

~

I r

rg

'I ~

r r

'/r /Ca/

R4 I ~

r

~

r/

/

/I

~ /

r//

~

r

/

aa 1

I4

'( l~,

r.

aJ I '/

~ '

/ifffR TO DR~~e. Fm r

/4lx6/4aar S

5~my

'px.DZi

'2I0 O (g /.'/

, (2SS Oaa I

/

/

I ST6.

5 /

'TR./

'/

6

//'r r'.///,

'T6.

4. a I

7/

'R / I 262 8

///

rrr.

~

r/r

'62 4l'1 I/J aaaaar

~

A rl J "

Ir/ trar

.Cg SlV.

5,

/ r

~

~

'r,

,r Slm.;

2 J r

/J

~

Q I

/ SlR."

~

~vena/r I

I ..."'

I' I'

I, Q

0 1 PARTS liIST FOR FIGURE 1 PART NO. DESCRIPTION PART NO. ,DESCRIP TION 113-A* IMPELLER (Stg. 1) 1?4-C* RING, IMPELLER (Hub) 113-'8* IMPELLER (Stg. 2} 176-A*. RING, CASE (Stg. 1) 113-C* IMPELLER (Stg, 3} 176-8

• RING, CASE (Series) 113-D* IMPELLER (Stg. 4} 177

• SLEEVE, OIL RING MOUNTING'ASE, 113-E* IMPELLER (Stg. 5) 179 VOLUTE (Top Half) 113-F* IMPELLER (Stg. 6} 185-A* BUSHING, THROAT 113-G* IMPELL'ER (Stg. 7) 185-'8* BUSHING, THROTTLE 113-H* IMPELL'ER (Stg. 8} 201

• BEARING, THRUST 114 CASE, VOLUTE (Bottom Hal f) 202

• WASHER, BEARING LOCK 126-A* RING, OIL 203

• NUT, BEARING LOCK 126-8* RING, OIL 207-A* ,

GASKET, CASE (Not Shown) 140-A DEFLECTOR (Inboard} 207-Y* GASKET 140-8 . DEFLECTOR (Outboard} 207-Z* GASKET 147-A* STAGE PIECE (Series} 208-A* "0" RING (Center Stage

~

Piece) 147-8* STAGE PIECE (Center}

208-Z* "0" RING (Bearing Cover}

148

• BEARING, JOURNAL 210 SHAFT, PUMP 154-Z COVER PLUG'OUSING, 255-A'* KEY (Series Impellers) 163-A BEARING (Radial)=

.255-8" KEY (Stg. 4 Impel',ler E 163-8 HOUSING, BEARING (Thrust.} Center Shaft Sleeve) 164 COVER, BEARING HOUSING 255-C* KEY (Final Impeller}

171-A* SL'EEVE, SHAFT (Center) 255 KEY (Thrott'le Sleeve)  :

1?1-8 SLEEVE, SHAFT (Throttle) 255-E" KEY (Shaft Extension) 174-A" RING; IMPELLER (Stg. 1} 262-A PIN, 09'rl=l 174-8" RING, IMPELLER (Series) 26,2-8 P I tl, .00'r,'EL SPARE PARTS, FURBISHED

0 lk ,4 I k

PARTS LIST FOR FIGURE 1 '(contitIued)

PART NO. DESCRIPTION PART NO. DESCRIPTION 274-A

• SHIMS Va 274-B

• SHIMS (Bearing Adjust.)

284 NUT, LOCK (Shaft Extension) 308

• RING, BEARING LOCK 310-A

• RING, THRUST (Impellers) 310-8

• RING, THRUST (Throttle Sleeve) 327-A COOLING COIL 327-B COOLING COIL 338 CONSTANT LEVEL OILER (Not Shown}

345 PLUG, INSPECTION

r 0

4

'v. V P

FIGURE 2 CENTER STATE PIECE ENLARGED Center Shaft Sleeve Center Stage Piece 0 Rin 8th Sta e Ke wa Ke for Sleeve and 4th Stage 4th Sta e Im eller Center Sta e Piece Dowel Pin

0 ~i I't gl

FIGURE 3 SOUTH TEXAS.AFW PUMP SHAFT SLEEVE THROTTLE SHAFT SLEEVE CENTE~

THROT T L E BVSHING

$ 7AGE PIECE CENTER lOt I II% %If ~ XSE REFER TQ 5 "STAGE DISCHARGE AU X II. IARY DRAY/ING FOR STUFFING 60$

QE.TAILS 5 ST%

~

g /

l.~ p

'aea@

~ 5'"STAGE DISCHARGE F

r II

.*PRESS RELIEF LINE l

OUTBOARD END ELEVATION VIEW CENTER SECTION AUXILIARYFEEDWATER PUMP

II 4',

-t5 ~h