ML17229A057

Part 21 Rept Re 2 1878 Piston Check Valve That Failed to Close.Caused by Fretting Wear Damage Between Piston (Disc) & Valvo Body Caused by Pressure Pulsations of Reciprocating Pump

ML17229A057
Person / Time
Site:	Saint Lucie
Issue date:	09/19/1996
From:	Knecht W ANCHOR/DARLING VALVE CO.
To:	David Skeen NRC
References
REF-PT21-96 NUDOCS 9610040032
Download: ML17229A057 (17)

Text

CATEGORY 1

,REGULAT INFORMATION DISTRZBUTIONeSTEM (RIDE)

A ACCESSION NBR:9610040032 DOC.DATE: 96/09/19 NOTARIZED: NO FACIL:50-335 St. Lucie Plant, Unit 1, Florida Power 6 Light Co.

50-389 St.

Lucie" Plant, Unit 2, Florida Power

& Light Co.

AUTH.NAME AUTHOR AFFILIATION KNECHT,W.G.

Anchor/Darling Valve Co.

RECIP.NAME RECIPIENT AFFILIATION SKEEN,D.

NRC No Detailed AffiliationGiven DOCKET 05000335 05000389

SUBJECT:

Part 21 rept re 2" 1878 piston check valve that failed to cl'ose.Caused by fretting wear damage between piston (disc) valvo body caused by pressure pulsations of reciprocating pump.

DISTRIBUTION CODE:

IE19D COPIES RECEIVED:LTR i ENCL +

SIZE:

lQ TITLE: Part 21 Rept (50 DKT)

NOTES:

RECIPIENT ID CODE/NAME PD2-3 PD INTERNA

. PIbH<<SENTE~O NRR/DRPM/PECB RES/DSIR/EIB RGN2 RGN4 EXTERNAL: INPO RECORD CTR NRC PDR COPIES LTTR ENCL 1

1 1

1 1

1 1

1 1

1 1

1 1

1 1

1 RECIPIENT ID CODE/NAME WIENS,L.

NRR/DISP/PSIB/B PDR WARDF M.

RGN1 RGN3 NOAC SILVER,E NUDOCS FULL TXT COPIES LTTR ENCL 1

1 1

1 1

1 1

1 1

1 1

1 1

1 0

D 0

E N

NOTE TO ALL "RIDS" RECZPIENTS:

PLEASE HELP US TO REDUCE WASTE. TO HAVE YOUR NAME OR ORGANIZATION REMOVED FROM DISTRIBUTION LISTS OR REDUCE THE NUMBER OF COPIES RECEIVED BY YOU OR YOUR ORGANIZATION, CONTACT THE DOCUMENT CONTROL DESK (DCD)

ON EXTENSION 415-2083 TOTAL NUMBER OF COPIES REQUIRED:

LTTR '5 ENCL 15

0 4

'T

~

tg 1

SENT BYi ANCHOR DARLING) 9-19-96 11:22Aii; 717 327 4922

~>

0 301 5042279)

PkzI

%'73I AnchorlDarling Valve Company e

Wiiliamsport, PA 17701 w.G. KNEGHr Technical Dircclo)

S~pt;ember 19, l996 FlC TO:

Dave Skeen U.S.N.R.C.

301-415-2279 SUDTECT:

1)art 21 Reported Tncident nn FP&l), 2" Lift Check Valve ALtached:

Copy of p/Dv letter being sent t;o custorr)ers

+ho mre fmnisher3 2" lift check valves

COpy Of FP&L 1'inal Fngineel"ing Disposition

/rj,;fg)-,.,Jj william G. Knecht Technical Director Attachm'.nts 03009$

9~~004003~

~~09~9 PDR ADOCK 05000335 S

PDR

<pPt t

SENT BY: ANCHOR DARLING; 9-19-96 11:22AM) 717 327 4922

~~

301 5042279)

¹2 DATE TO:

Utilities that have been furnished 2" 1878 Piston Check Valves Gentlemen:

There was a reported failure of a 2" 1878 piston check valve at the Florida Power and Light, St.

Lucia site on July 18, 1996.

This valve failed.to close and it was determined that the valve disc was stuck in a partially open position.

It appears that this failure resulted from fretting wear damage between the piston (disc) and valvo body caused by pressure pulsations of the St. Lucio reciprocating pump.

Anchor/Darling Valve Company has furnished with similar valves of this size on the following order(s):

(Utility)

(Fill in A/DV S.O. and Customer Order ¹)

We suggest that the valve applications be evaluatod for potential pressure pulsation that could result in fretting of the valve body.

In addition, FP&L and A/DV need assistance to finalize the root cause of the St. Lucie problem.

If an A/DV 2" piston check is in pump service, we would greatly appreciate receiving the following

, information:

1

~

Pump type - centrifugal or reciprocating 2.

Forward flow rate(s) 3.

Approximate length of service time 4.

Description of any found anomalies Should you have any questions regarding this matter please call Mr. Floyd Bensinger, our Engineering Manager or Bill Knecht, our Technical Director.

Sincerely, Contract Administrator

SENT BY:

ANCHOR DARLING'9 96 1 1 '22Ahij 717 327 4922 301 5042279>

va vo-i I Is

~ Attach 8 Page 1 of 11 Scope:

Previous interim dispositions provided an operability assessment for 2'nchor Darling check valves installed in Unit 1 (Attachment 1) and Unit 2 (Attachment 7). Attachments 2,3,4 8 6 provided supporting documentation. was superseded by Attachment 7.

This attachment provides a final disposition for the CR.

Attachments 9-18 provide supporting documentation.

The additional information developed in ibis final disposition does not adversely affect the previous operability reviews.

Corrective actions ln this disposition supersede corrective actions listed in prior attachments,

Background:

CR 96-1 738 and In-House Event 96-059 identifled loss of Unit 2 charging flow associated with the potential failure of V2167. V2167, the check valve for charging pump 2C discharge, was suspected of sticking open which would result ln the charging flow recirculating back to the volume control tank rather than Into the RCS. The valve model is a 2'tainless steel Anchor Darling piston check which was installed in 1994. It is a piston check valve (straight/T-pattern) located downstream of the reciprocating pump, V2167 (serial ¹ E-T401-9-19) was disassembled for inspection and was found stuck open approxImately 1/2'. In addition, the body bore machining in the area where the 'disc is guided was abnormal (looking from top with discharge port at 12:OO, machining/casting anomalies were identified from 10:OO to 12:30). We first candidate replacement valve inspected also exhibited machining anomaties in the same location. A second spare for V2167 was inspected, found to be acceptable, installed and successfully tested.

Based on initia( review of the valves in stores, a working theory for the degradation cause was developed and used to disposition the operability of the Unit 1 valves per Attachment 1. The operablllty of the Unit 2 valves was subsequently reviewed in Attachment 7 based on a refinement of the same workfng theory.

This attachment provides a final CR disposition by documenting tasks performed to date, developing a root cause evaluathn, addressing regulatory reportability 8 industry notification and planning follow-up activities to be pursued under specific PMAI's.

Initial Review of Machining Anomalies and Stuck Open Check Valve By Manufacturer The machining anomalies associated with2'nchor Darting check valves and the study'pen check valve in the charging system were reviewed with Anchor Darting. Their initial review is provided in Attachment 2.

Anchor Darling states that the selection of the subject 2'iston check valve as the charging pump discharge valve is appropriate. They also state that pressure pulses experienced at the discharge of a reciprocating pump may tend to force the valve disc toward the downstream port. They conclude that this system application combined with the machining anomaly on the body bore could cause the valve to remain open (i.e., the disc cocked and caught on machining anomaly). They further stated that other applications may not be as susceptible.

Review of FPL's Surveillance Testing of Installed ADVValves

'wenty-eight SS 8 CS Anchor Darling piston check valves (2'nd smaller) are in-service in PSL 1 8 2 for periods ranging from 0 to 36 months (Attachment 1B). A review of the testing records (Attachment 3) shows approximately 450 valve-months of satisfactory operation (Exceptions are the Unit 2 V2167 and Y2168 failures)

~

The testing population includes 16 of the 17 two-inch piston check valves reviewed in Attachments 1 (Unit 1) and 7 (Unit 2). Note that V18193 is not included in the tested population as it is outside the LLRT boundary for its instrument air containment penetration.

'9-19-96 11:23AMi 717 327 4922

=>

301 5042279) 4O 00 I //4 Attach 8 Page 2 of 11 Review of Reportabillty of Discrepant Material Received from ADVUnder 10CFR21 Nuclear Engineering Quality Instruction ENG-Ql 2.2 governs evaluations conducted to determine whether an issue/concern could result in a 10CFR21 Substantial Safety Hazard.

This document states:

Note that SSH issues already reported (by FPI or a vendor to the NRC) do not require SSH evaluation (although operability evaluation may be warranted}." As Anchor Darling Valve Co reported the subject situation with their 2'iston check valves to the NRC on 7/25l96 (Attachment 9),

an SSH evaluation by FPL is not required.

FpL independently alerted the industry to the Loss of Charging event caused by the failure-to-close of the 2C charging pump discharge check valve via the Nuclear Network (Attachment 11).

SENT BY:

ANCHOR DARLING) fI4 t'nitial inspections of Valves ln Stores:

Inspections were performed on spare Anchor Darling 1' 2'arbon Steel (CS) 8 Stainless Steel (SS) piston check valves that were in the stores inventory for use with the small bore valve replacement program.

Similar machining/casting flaws were found in multiple valves as described ln more detail in Attachment 7.

Based on these inspections, Attachments 1 8 7 concluded:

1) 2 " CS 6 SS ADVpiston check valves in stores are susceptible to machining anomalies.

2) 1'nd under CS 8 SS ADVpiston check valves are not susceptible to machining anomalies.

3)

Allsizes of CS ADYpiston check valves in stores are susceptible to internal rusting.

Review of Avallabfe NPRDS Information The Nuclear Plant Reliability Data System was searched for information concerning Anchor Darting check valves with an 1878 model number. With the exception of the Shearon Harris event, all other events (see below) were deemed to involve foreign material within the valve. Information to support an NPRDS entry

. for the PSL event was develo ed and rovided to Joe Cimino Attachment 10.

NPRDS Search Crfterfa:

Subject:

Valve Selected Manufacturer. Anchor/ Darting Valve Co Words equal to or beginning with: check valve, ck vfv, chk vtv MirModel Numbers: 1878 Company, Plant, Service, Valve, Vatve 0 CPKL, Shearon Hanfs 1, CVCS 2 in piston check. 1CS-193 Northeast UUrities. Con Yankee 1, CVCS 1 in piston check. GH-GV~SC GPU. Oyster Creek 1, Containmenf Spray 2 in swing check. V40133 GPU, Oyster Creek 1, Containment Spray 2 in piston check, V-~133 GPU, Oyster Creek 1. Containment Spray 2 in piston check. V-3-1033 GPU, Oyster creek 1, confalnmeni spray 2 in piston check, V-3.0131 Niagra Mohawk. 9 M Poini, FW, 3141R 18 in swing check Niagara Mohawk, 9 M Point, FW,31-01R 18 in swing check Niagra Mohawk, g M Poini. FW, 31*02R 18 in swing check Portiand GE, Trojan 1, SA, SA-2005 2 in unknown style NYPA, Fitzpatrick 1, Combustible Gas Control. 27CAD 68. 1.5 in unknown style NYPA, Fifzpainck 1. Combustibfe Gas Coniroi. 27CAD -68, 1.5 in unknown style Noted Problem Failure to provide fuIIflow, failure fo seat 10CFR21 issued, This event is discussed separately.

Gross Leakage, 0-Ring seal came foose from disc, Valve pfaced back in service Unusual noises, Disc dhengaged due to missing nut/cotter pin, Valve replaced with swing check Unusual noises tchattering). leaking past seat: Telephoned to cfaNy NPRDS data:

ValveIin seawater senrice, no damaqe to valve, pads not degraded, normaify open

-150 qpm, debns behind seat (sift and sheNs), have chanqed valve design 8 manuracfufers sevorai ffmos. info per John Qaianto (609-371~

Stuck Open, Marine Growth and Foreign Mafeifaf, Valve placed back h seivice Engineering requested to evaiuato posshlo vafve'repfacement Siuck Open, Maifne Growih and Foreign Matenal, Valve placed back in service, Plan to install upgraded valve Faiied LLRT, Dirt and Debris Inside Valve, Valve piaced back in seivice Failed LLRT Flutter During Flow Condition inherent to Piping Design, Valve placed back in so/vice Failed I I RT, Flutter During Flow Condition inherent to Piping Design, Valve placed back in sennce Failed LLRT, Foreign Material on Seating Surfaces, wire and rubber gasket foieign maferiaf, Valve placed back in mrvice Failed LI.RT, Found no otf nomial conditions, piece oi dirt believed io be responsibie

/etumed to service Failed LLRT, Cloih like debris prevented closure, valve placed back in service

SENT BY:

ANCHOR DARLING) 96 11:23AIA) 717 327 4922

=>

301 5042279I II5 i 6 Jb 1//4 Attach 8 Page 3 of 11 Review of Shearon Harris 10CFR21 Report - CSIP Mlnlflow Check Valve Oeflclency A CP8L 10CFR21 report to the NRC (Reference 7) doscribes an event where their 1B Charging Safety Injection pump (CSIp) was placed in service when it was inoperable (due Io the inability of its mini-flow check valve to meet forward flow testing requirements).

One CSIP is normally operating in their system alignment to provide charging fiow. The subject ADVvalve failure was detected because it exhibited back leakage on at feast two different occasions.

The back leakage was caused by cocking of the piston toward the valve discharge port andlor damage to the resilient seat.

CP&L attributed the damage to the near-instantaneous opening of the piston check due to large hydraulic forces caused by pump startup.

Subsequent to the initialfailure, three different disc designs were installed (resilient seat, hard seat, fullbody guided) with no improvement. Originally, the resilient seat was suspected of causing the valve to stick open because it was found partially off the disc and degraded.

The valve was inspected, measured, and found to be within the vendor specifications. While no internal machining or casting anomalies were noted, CPKL was not specifically looking for the problem seen at PSL. The valves from CP8 L are not available for further inspection. After the problems with the ADVpiston check valves (stuck open, insufficient forward flow) were not remedied by subsequent disc modifications, CP8L ceased efforts to modify the ADV valve and reinstalled the original Y-pattern piston check valve. No subsequent probiems wel'e identified. GP&L did not develop the root cause conclusively, CP&L pt'ovided the following information regarding the CSIP recirculation check valves:

1.

CP&L changed valve from Rockwell Y-pattern piston check valve to ADVT-pattern piston check vaive in 1994 due to ALARAconcerns (eliminate seal welded bonnet).

ExperIenced problem viith ADVvalve (stuck open, Insufficient forward fiow) that was not remedied by the disc modifications (see below),

ceased efforts, and reinstalled Rocfcwell Y-pattern piston check valve.

2.

Changed the disc design (resilient seat, hard seat, full body guided) with no improvement.

Originally, the resilient seat was suspected of causing the valve to stick open because it was found partially off the disc and degraded.

The valve failed-towpen with the fullbody guided disc.

3.

X-Ray of valve showed disc was cocked with the bottom of the disc towards the discharge port. CP8 L believes disc was always cocked to discharge port (l.e. the first time the pump was started the disc would cock). CP&L presented no data to support or refute this contention.

4.

CP&Lstates the most probable root cause is signNcant hydraulic forces acting on the check valve when the pump starts; pump takes approximatety 3 sec to come up to speed, the valve doses within the first 1/2 second.

5.

The valve was rot originally backseat leak tested by the plant. They noticed the pipe was hot after the ADVvalves were installed and initiated backseat leakage testing.

6.

Inspected the body bore of suspect valve with no indication of roughness or machining marks.

Inspected intersection on body bore and discharge port and found no problems.

(Not dear if fretting damage was identified) 7.

Disc Dimensions were within vendor specification: Vendor 1.867, %.002, As-found 1.868 Bore Dimensions were within vendor specification: Vendor 1.875, +0.004, As-found 1.875 8.

Design PressurelTemperature:

2735 psig, 200'F; Operating:

2712 psig, 130'F; Flow 60 gpm CP&Lhas two other ADV2 piston check valves Installed (BATransfer Pumps Discharge Check Valves) that have not exhibited the noted problem. These valves are tested quarterly at 35 gpm (forward flow and backseat leakage) with no reported failures. The operating conditions of these valves are different from the CSIP check valves: operating pressure is lower (120 psig vs 2712 psig) and flaw is lower and variable (35 gpm vs 60 gpm). CP8 L provided the following information regarding the BAT recirculation check valves:

1.

2.

3.

Anchor Oafflng T-patiem piston check valve installed in 1994.No problems reported.

Valves tested quarterly for flow and backseat leakage at 35 gpm.

Design Pressure/temperature:

150 pslg, 250'F; Operating: 120 pslg, 250'F; Flow 140 gpm max, varies The information obtained from ADVand CP8L is insufficient to conclusiveiy determine the root cause for the Shearon Harris event. There is insufficient information on which to base any further conclusions.

ADV conducted flow testing but could not replicate the problem.

Further testing is planned for the CP8L scenario. and FPL has supplied a suspect valve which could also be tested.

When further information

SENT. BY:

ANCHOR DARLING) 96 11:24AM; 717 327 4922

~ ~

301 50422?9)

M CR96.1774 Attach 8 Page 4 of 11

¹6/13 becomos available, it willbe evaluated as appropriate.

Review of Engineering's records indicates JPN was notified of the Shearon Harris 9/29/95 everit by ccMaii from FPL Licensing on 10/2/SG due to GPL's Nuclear Network entry. JPN contacted CPL on 10/7/95 to obtain a copy of the Part 21 notice and to discuss the event.

The cause of the event was not well understood and no further action was appropriate at that time. It has been determined that the NRC did not make any formal disclosure of the Shearon Harris event. Recent conversation with the PSL site resident indicates that NRC issuance of an Information Notice noting both the CPL and FpL events is likely.

Inspection of V2167 After Sectloning The initialconclusion that machining anomalies in the 2'DV piston check valves were directiy resporisibie for the field failure is not supported by more recent inspection of V2167 following valve sectioning.

The inspection. as documented in Attachment 14, concludes that local fretting wear occurred in the body contact area with the disc which likely Ied to a local groove, sticking/disc cocking and the valve's failure-to-close, As the castirig flaw above the discharge port and the machining marks below the fretting zone did not exhibit any significant wear marks, they are not believed to have led to the failure of V2167 to dose.

The machining marks below the fretting zone. however, could well have contributed to piston sticking for different valve operating conditions. However, had the machining anomalies been located adjacent to the observed fretting location (ie., the valve's normal disc location during flow) they would have affected valve operability/reliability.

Review of the V2167 anomalies indicates the acceptability of casting/machining anomalies is largely dependent on their location. While such anomalies are certainly undesirable in a finished product, their presence can be accepted on a case-by-case basis based on their extent and location.

Inspection of V02534 After Sectlonlng V02134 was removed from service due to galling of the bonnet threads.

The valve was subsequently scrapped and sectioned for internal inspection.

The inspection of this valve, as documented in Attachment 16, was performed in the presence of ADV. It identified internal machining anomalies near the discharge port which are not believed to have affected valve operation. Wear bands in the guide surfaces (fretting) were identified at the disc land locations. No local grooving was identified as a result of fretting wear. Local scuffing (abrasive wear) of the guide area at the edge of the discharge bore was observed.

Review PWO V/ork Hlstoty for Trending Per FRG request, the work order history of V2167 and similar valves was reviewed to determine whether there was sufficient data to trend a developing problem with the Anchor Darling valve. 7 documents this review.

An initial report in December of 1994 indicated V2167 failed to close (flow decreased as recirculation valve opened} but problem could not be replicated.

Further inspections did not identify valve internal problems and testing was inconclusive. Engineering concludes there was insufficient data prior to the 7/13!96 event (with its determination the valve was cocked and stuck open) to identify an incipient problem.

SENT BY:

ANCHOR DARLING> 96 11:24AM> 717 327 4922

=>

301 5042279>

t'Vm-I//4 Attach 8 Page 5 of 11 II7/13 Disposition of ADVPiston Check Valves ln Stores Sased on the conclusions drawn from the inspection of V2167, the 2'iston check valves remaining in stores were re-inspected against a criteria that would allow raw casting surfaces and/or machine marks in valve guide areas not in contact during valve operation.

This review was based on the criteria depicted in 3 which provides the basis for accepting certain valves that would otherwise be returned to ADVas non-conforming. 2 lists all 2'iston check valves received from Anchor Darling and the remaining 1" and under piston check valves in Stores. This information for the 2'alves is summarized below:

Date==>

Total 2 Valves Received from Anchor Darling 7/13/96 7/19/96 66 7/31/96 66 8/9/96 8/1 6/96 66 Vatves Installed in Unit 1

-Uninspected

-inspected/Acceptable

- Inspected/Replaced Valves installed in Unit 2

-Uninspected

-Inspected/Acceptable

- Inspected/Replaced Valves In Stores

- Uninspected

- Inspected Acceptable

- Inspected Unacceptable Reduction of inventory

- Scrapped

- Returned to Anchor Dartina

- Return Credited (ENG use)

Some require rework prior to use.

11 11 0

49 49 49 14 29 11 10 1

47 14 4

28 11 9

1 1

0

~ 16 11 9

1 14 0

14'5 2

32 1

As described in Attachment 12, a number of packaging anomanes were ident5ed from review of the valves in Stores; these included many valves that were not yet unpacked from the initial factary shipment.

Anomalies included water droplets inside the majority of the piston check valves, internal rusting and absence of desiccant.

ADV indicates that disassembly of the valve following hydrotest for dry out and reassembly, with the valve pressure seal shipped as a loose part, would be an appropriate corrective action to the first item.

These inspections supparted the conclusions from the initial inspections (see page 2) that:

1) 2 " CS 8 SS ADVpiston check valves in stores are generally susceptible to machining anomalies.

2) 1" and under piston check valves are not susceptible to the machining anomalies notedin the 2'alves.

3)

Allsizes of CS ADVpiston check valves in stores have various levels of internal rusting.

The anomalies tound in the 1" 8 under piston check valves (a casting lip) were different from the generic problems associated with the 2 piston check valves and were deemed to be an isolated cases, These two valves willbe returned to ADV.

SENT BY:

ANCHOR DARLING)

Root Cause Evaluation: 96 11:25AM> 717 327 4922

~> ~ '01 5042279i Attach 8 Page 6 of 11

¹8/13 Plant Status When Event Occurred 100% Power Activityin which Event Occurred 2C Charging Pump Shutdown Type of inappropriate Action/Job Category Not Applicable How Human/Admln Error Occurred Not Applicable Plant Location V/here Event Occurred 2C Charging Pump Related Activityln Which Event Occurred 2B Charging Pump Startup after Oil Addition Type of Equipment 2'nchor Darling Piston Check Valve How Equipment Degraded/Palled Failed to Close Why Event Oc'curred Charging application developed fretting wear on disc/body interface leading to disc sticking open The failure-towiose event for the 2C Charging Pump discharge check valve {V2167) initiallyappeared to sed bY internal casting/machining anomalies.

Physical inspection of a spare 2'alve also indicated internal machining anomalies which required the consideration of the continued operability of all installed 2'DV piston check valves (Attachments 1 & 7).

Based on a series of verbal discussions with ADV, the cause of the machining anomalies is believed to be due to the past use of a drilling process to perform rough machining of the guide area and to develop the seat pocket recess.

This'process tends to follow the geometry of the rough casting.

Following brazing of the seat and heat treatment, a boring process is used for final machining.

The centerline differences between these two processes is believed to account for the observed anomalies.

The quality controls within ADV's manufacturing process identified the subject anomalies and these were subsequently accepted followingADV's internal review.

Further review of V2167 internal conditions (Attachment

14) has concluded that the noted internal casting/machining anomalies are not the primary cause of the valve's failure-to-close. This conclusion is based on the location of the casting/machining anomalies in V2167. However, the types of anomalies noted in other ADVpiston check valves {Attachment 12) may well affect valve operability/reliability.

The present understanding is that the 2C Charging Pump discharge check valve's (V2167) failure-torose was due to internal fretting damage within the valve. Fretting damage ls believed to be primaril caused by the valve's material combination and the charging system application {iong service times in a pulsating flow). increased toierances due to casting anomalies may also be a contributing factor. The machining anomalies in Y2167 would not affect valve function.

The charging pumps are triplex positive displacement piston pumps manufactured by Union Power Pump

. Company.

These pumps have a 2 1/16'ore & a 5 inch stroke and operate at 213 rpm. The number of pressure pulsations per minute of run-time is -640 at a nominal discharge pressure of 2335 psig.

Review of the run time hours for the 2 ADVcheck valves installed in charging system service does not indicate that runtime hours, in of itself. is a ood indicator of im ndin failure.

Charging Pump 1B Charging Pump 1C Charging Pump 2A Charging Pump 2B Charging Pump 2C Charging Pump Unit 2 Common Header Discharge Check Valve V02133 V02134 V2169 V2168 V2167 V2462 Check Valve Run Time 1,200 hours0.00231 days <br />0.0556 hours <br />3.306878e-4 weeks <br />7.61e-5 months <br /> 6,500 hours0.00579 days <br />0.139 hours <br />8.267196e-4 weeks <br />1.9025e-4 months <br /> (est) 3,100 hours0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br /> 9,700 hours0.0081 days <br />0.194 hours <br />0.00116 weeks <br />2.6635e-4 months <br /> (est) 5,700 hours0.0081 days <br />0.194 hours <br />0.00116 weeks <br />2.6635e-4 months <br /> 5,850 hours0.00984 days <br />0.236 hours <br />0.00141 weeks <br />3.23425e-4 months <br /> (est)

I

SENT BY:

ANCHOR DARLING) 1 9 96 1 1 25Ah1 j 71 7 327 4922

~

301 5042279 j Attach 8 Page?

ol 15

/I9/1 3 Review of the valve application, knowledge of the disc cocking and inspections ol valves V2167 (Unit 2),

VO2134 (Unit 1), and valves in Stores lead to the following failure description.

This description has been updated to reflect additional information; the basic explanation, as contained in Attachments 1 8 7, remains essentially unchanged.

Failure was due to a combination ot causal factors:

3)

The constant flow ot the charging pomp (44 gpm) results in the valve disc opening to move off its seat approximately 1/2'-5/8 '. The constant flow rate of the positive displacement pump results in a single disc position leading to the concentration of wear damage at a single'location.

2)

As the bottom edge ot the disc is not fully supported by the body bore due to the presence of the discharge port, the disc's bottom skirt is supported at two relatively wide points on its arc. At 68 gprn, the angle between the arc points (-14+ is at its maximum (44 gpm tor charging system/-110').

3)

The pulsating nature of the charging fhw (three piston, positive displacernent, reciprocating pump) results in a small continuous oscillation of the valve disc. The disc movements within the bore guide surfaces (vertically and side-to-side) result in wear bands that correspond to the disc land locations.

4) The raw casting of the body bore was apparently oversized/off-center which led to incomplete machining ot the bore and greater clearances between the disc and bore.

Absence of signicant wear marks-.

within these regions indicates the rough guide surfaces did not contribute to the disc wear. Greater clearances result in larger disc movements and may lead to higher disc velocities and impact forces.

5)

Disc movement led to wear of the disc and the body guide surfaces.

The wear band from the disc's upper end is distributed around the entire circumference but is more noticeable on the outkit side of the valve. A lower wear band is also present but more rapid wear occurs at the two lower land's contact points on the bore/discharge port intersection.

6)

Due to the long in-service time, 'constant flow. two point support and softer material of the body, grooving (due to fretting) and scuffing (due to abrasive wear) occurs at the body bore/discharge port intersection. Eventually, the fretting wear reaches a point where the damage results in a cocked disc that is stable under loss of Aow; the spring force is not sufficient to reclose the valve when the flow stops/reverses.

Based on the below review ot the failure description, a reciprocating pump application is expected to be more susceptible to fretting damage than a typical centrifugal pomp application.

Causal Factor Constant flow rate Pulsating ilow of a reciprocating pump Mlswast valve Piston & valve body materials Duration of valve's service at flow Criteria for Selection / Implications Results in a single check valve disc position - liftis a function of flowvehcity and density Results in concentration of wear at one discharge port kcation Maximum angle of contact occurs with -68 gpm or 12 ft/sec of water Results in small oscillation of the plug lo create fretting at the points of contact within the guide surface, flaw turbulence with a centrifugal pump may also produce fretting but is expected to result in a lower wear rate Mls~sting causes greater clearances; more disc movement may result in more rapid wea depending on location Harder disc provides tor more rapid wear at points ot contact in bonnet bore/discharge port intersection; item with greater wear (body) is non-replaceable.

Wear process is time related; Duration ot service at flow governs effect of wear mechanism.

More frequent inspections are warranted for valves normally at tlow Absence of one or more factors willincrease the time to failure and/or may prevent the failure mechanism.

lt ls difficultto state whether fretting willnot occur for a centrifugal pump application - given a sufticient time duration for the damage to be observed.

The duration ot time requii'ed for failure will also be dependent on the pre-existence of casting/machining anomalies.

Additional field data for centrifugal pump applications is needed to fullyaddress this potential concern.

SENT-BY:

ANCHOR DARLING) 96 11:26AM; 717 327 4922

~>

301 5042279; w<tvv >sr t AttaCh 6 Page 8 Of ] t if10/13 Fretting is an adhesive wear mechanism between two surfaces having oscillatory, relative motion of small amplitude.

It involves the repeated formation and fracture of cold welds between two surfaces.

Abrasive wear involves the plowing or cutting of grooves in a soft material by a harder one. Factors which affect both adhesive and abrasive wear include relative hardness, surface finish and microstructure.

Generally, harder materials are more wear resistant and greater hardness differences lead to increased wear. Rough surface finishes usually cause increased wear, however, very smooth finishes can increase the tendency for cold welding. Also, similar microstructures can increase the tendency for cold welding.

The cause of the fretting appears to be small disc oscillations due to the pulsating flow coupled with a less than optimum material combination of disc and body materials, Discussion of the material selection with ADVcentered on a rule of thumb of achieving a difference of more than 4 HRC between material couples.

This criteria is typically used to prevent the onset of galling, an adhesive wear mechanism. but does not address other wear mechanisms.

For the material combination of the resilient seated valves (SA-564 Gr 630-1075 disc and SA 351-CF8M or SA 216-WCB body) wear data indicates a less than optimum couple.

For the material combination of the hard seated valve (SA-638 Gr. 660 Type 1 disc and SA 351-CF8M or SA 216-WCB body) no wear data was found in the literature.

Based on the hardseated valve disc material hardness (SA-564 Gr 630:43 HRG) compared to resilient valve disc material hardness (SA-564 Gr 630-1 075: 32 HRC), it is reasonable to conclude that wear rates for the resilient seated valve body willbe less than those observed for the hardseated valve. Anchor Darling has been requested to review wear characteristics of their material combinations.

Anchor Darling has stated that piston check valve use in non-pulsating flow applications, may not be susceptible to the above degradation mechanism.

For a centrifugal pump application (assuming variable flowthrough the check valve), the periodic travel of the disc across the machining anomafy would tend to smooth the damaged area.

For larger flow applications, the greater stroke would result ln more avaflable spring closure force. Anchor Darling suggests that this force may be sufficient to overcome sticking due to fretting damage and coupled with the disc's dosing momentum may well result in closure of a piston check despite casting/machining anomalies at lower elevatlons.

ADVsuggests a smaller valve would be beneficial.

The root cause for the V2167's failure-torse is fretting damage between the disc and the valve body at two contact points adjacent to the valve's discharge port The raw casting surfaces and machining anomalies noted within the piston guide areas are not presently befleved to have significantly contributed to the failure of V2167 based on their location and the tack of significarit wear marks. However, these type of manufacturing anomalies may well contribute to failure-to-close/open in other valves depending on their location, degree and the valve flow rate application.

The fretting damage was likelycaused by the hrge difference in hardness between the valve disc (43 HRC) and valve body (78 HRB) and the pressure pulsations & constant flow application of the charging system.

These conditions would lead to small disc oscillations (due to pressure pulsations) at a single point (due to constant flow) in the valve discharge port.

Implications of Root Cause Determination Based on the above description of the failure causes, it would follow, but is not certain, that fretting damage would be significantly less pronounced for a centrifugal pump application. This presumption is based on pressure pulsations from a centrifugal pump being less than for a reciprocating pump and the variation in pump flow distributing any postulated fretting wear across a wider surface area.

The first presumption, though reasonable, may delay rather than prevent the onset of significant fretting damage.

The presumed beneficial effect of flow variation is reasonable for a centrifugal pump discharge check valve application where system flow rate varies with time. For a mini-recirculation line check valve serving a centrifugal pump that is normally operated in a near dead-headed application, the flow variation through the recirculation tine would be negligible, This latter application is descriptive of the Shearon Harris CSIP application.

SENg BY: ANCHOR DARLING) 96 11:26AM) 717 327 4922 301 5042279)

Attach 8 Pago 9 of 11 011/13 Based on the above, the Shearon Harris event may bear more similarity to the St. Lucie event than first realized. Based on discussions with ADVand Shearon Harris, fretting damage was not identified within the valve discharge bore.

At this point in time, Shearon Harris and ADVcontinue to believe the cause was related to the initial insurge of the flow from the CSIP pump.

Until further data is obtained, it would be prudent to inspect O'DV piston check valve internals on a more frequent basis for applications where valves see long periods of service at flow.

Based on the current level of understanding, O'DVpiston check valves are suitable for intermittent service or service in varying Aow regimes without restriction.

Further use of ADV piston check valves in near continuous use, constant fiow service must consider the potential for the valve to fail-to-close after long periods of service (2500-3500 hrs at constant flow).

An In-depth design review and in-service applications of the 2'iston check valve is warranted and has been requested by letter to Anchor Darling (Reference 8).

Specific long term corrective actions willbe required for the installed valves in constant fhwservice. These corrective actions are under review and willbe determined based on the work to be performed by Anchor Darling. The changes willlikely Indude a reduction in valve size of specific ADVpiston check vaives.

In the interim, the following specific countermeasures are recommended:

Perform yearly internal inspections of the ADV piston check valves in charging pump discharge check valve service until further notice, These inspections will develop trending information and should be sufficient given the rotation of the pumps.

2.

Perform an inspection each refueling outage of other ADYpiston check valves in charging service until further notice, These inspections should be based on GMP<1 and specNcally target fretting damage in the discharge bore.

Inspection by SCE personnel is recommended.

General actions have been specified below to prevent recurrence.

These Include a QA audit of ADV discrepancy identification, evaluation practices, 8 packaging practices to prevent recurrence at the manufacturing facility. Training of FPL weldem Is also planned so that similar potential valve problems would be Identified prior to implementation.

Corrective Actions:

Corrective actions in this disposition supersede corrective actions listed in prior attachments.

Inspect and. replace if necessaiy, ADV2'iston check valves installed in Unit 1 &2 per the schedule requirements of Attachment 15. AilWork Orders for these inspections have been written by SCE, No further action is required at this time for this item.

N 2.

Following completion of the initialvalve inspections under Attachment 15, perform periodic inspections of ADVvalves in charging service until further notice:

a)

Perform yearly internal inspections of the valves in charging pump discharge check valve service.

Recommend pump discharge valve inspection be tied to charging pump maintenance with instructions to disassemble valve and inspect if not already performed in last 6 months.

b)

Perform an inspection each refueling outage of other ADVpiston check valves in charging service.

Inspections should be based on GMP-01 and specifically target fretting damage in the discharge bore.

Localized fretting of the type seen in V2167 (a groove on each side of the discharge port) is cause for replacement.

Inspection by SCE valve component specialist is recommended.

Issue a PMAI to ENG-Systems Engineering to institute GMP-01 testing on AOV piston check valves in charging service on a more frequent basis. Recommend pump discharge check valve inspection be tied to charging pump seal maintenance with instructions to disassemble AOV discharge check valve and inspect if not performed in last 6 months.

Recommend other downstream Charging System ADV piston check valves be inspected during each refueling outage until further notice. Oue Date 10/$ 8/96.

SENT. BY:

ANCHOR DARLING)

~-19-96 11:27AII) 717 327 4922

~> ~

301 5042279>

vl t wvv t

~

~

Attach 8 Page 10 of 11

~

~

//1 2/13 Safety Evaluation JPN-PSL-SEMS-96-052 allows temporary manual isolation o( the Unit 2 recirculation valves while the associated charging pumps are operating-Implementation of the safety evaluation shall be implemented and controlled by approved cages to plant procedures in accordance with section 9.0 of the safety evaluation.

The recirculatlo>>alves for all 3 Unit 2 charging pumps shall be dosed untii V2462 is inspected and/or replaced as appro priate.

This action provides an additional compensatory measure to prevent backflow (intersystem LOCA), in the event V2462 fails to dose.

Closure of the manual stop valve on the discharge of a charging pump (V2336, V2464. V2339) I allow pump maintenance Is an equally acceptable compensatory measure for that specific pump. With the manual stop valve closed, the respective charging pump and its recirculation valve may be disassembled or tested as required The recirculation valve shall be closed prior to reopening the manual discharge stop valve, In addition to the above requirement:

a)

The recirculation valve for pump 2A (V2555} shalt be kept closed until valve V2169 has been inspected or replaced with a suitable spare.

Ckeure of V2336 is an equally acceptable compensatory measure.

b)

The recirculation valve for pump 28 (V2554) shaI be kept closed until vaive V2166 has been inspected or replaced with a suitable spare.

CIosun: of V2464 is an equally acceptable compensatory measure.

The above requirements have been accommodated by a permanent change to Op 021pp2p which provides for a normal charging system alignment with the three recirculation valves in the cio~

positions. This system alignment change is acceptabte for one fuel cycle per safety evaluat;on gpN.

PSL-SEMS-96-052.

5.

6.

7.

8, Issue a PMAI to Operations to revise OP 0210020 to return to the design basis use of the charging pump recirculation valves after the inspection and replacement, as appropriate, of Anchor Darting check valves V2462, V2167, V2168, and V2169. This PMAI is to include a Mode 4 h Id for the power ascension following the 1997 refueling outage (Cyde 15). Due Date S/1 S/97, rtgo/'7l Issue a PMA}to Nuclear Materials Management to return rejected ADVpiston check valves to the vendor as identified in CR 96-1774 Attachment 12. Vafve Serial ff E-T401-9-25 is to be released to ENG. Valve Serial ff E-T401-9-30 is to be tagged Fkstncted Use - requires Engineering Approvai (Reference CR 96-1774) Gordon McKenzie ext 7276'.

Valves desig d ap "Rework're to be segregated for rework by MM prior to field use.

Due Date 8/30/96.

Issue a PMAi to Mechanical Maintenance to perform rawor1t'f selected ADVvalves as identified in CR 96-1774 Attachment 12. Engineering willprovide support as required. pa~Da

/96.

vr/i issue a PMAI to Mechanica'I Maintenance to perform lrajring for welders to ensure they would identify and report casting and machining problems of the type noted within the 2'DV pjgon check valves.

Engineering will provide supporl as required.

Due Date grg7/96./p

+~+pc'ssue a PMAI to QA to perform an audit of ADV manufactunng and packaging process controls.

Manufacturing controls should have prevented shipmerit of valves with casting/machining anomalies.

A potential solution would be written guidelines governing evaluation of discrepant bore surfaces.

Packaging controls should have prevented the water observed within shipped valves.

Disassembly of the valve following hydrotest for dry out and reassembly 'he v lv rossure seal shipped as a loose part is a potential solution.

Due Date 10/18/96.

Issue a PMAI to Engineering to provide appropriate iong term corrective actions (valve replacements and/or procedure/inspection program changes) based on the root cause determination within CR 96-1774 and ADV's subsequent input requested by Reference 8. Review plant documents (eg., SOER B6-03 Inspection Database, GMP-01) for potential revision to address the root cause determination.

Due Date 11/15/96.

SENT BY: ANCHOR DARLING; 96 11:27AM) 717 327 4922

~>

301 5042279)

Attach 8 Page 11 of 11 0'13/13 Attachments:

Engineering Disposition related to Unit 1 Operability Review 2.

Anchor Darling l.etter dated July 19, 1996 3.

Inter-Office Correspondence.

"Testing of Anchor Dat1lng Check Valves", July 20, 1996 4.

Anchor Darling Piston Check Valves Summary of Inspection Findings, July 20, 1996 5.

Engineering Disposition related to Unit 2 Operability Review (Superseded) 6.

Anchor Darling Letter dated July 23, 1996 7.

Engineering Disposition related to Unit 2 Operability Review (Revisiori 1) 8.

Final Engineering Disposition 9.

ADV 10CFR21 Letter to NRC dated July 25, 1996 10.

NPRDS System Notification 11.

Nuclear Network Notification 12.

Disposition of ADV Piston Check Valves in Stores

13. Acceptance Criteria for Raw Casting/Machining Anomaly l ocations 14.

FPL Met Lab Report 96-170, dated August 5, 1996 15.

Inspection Results for installed Valves (8/16/96) 16.

FPL Met Lab Report 96-180, dated August 15, 1996 17.

Review of PWO History forTrending 18.

Installed Anchor'-Darling Check Valves. August 16,1996

References:

Unit 1 FSAR, Amendment 14 2.

Unit 1 Technical Specifications, Amendment 142 3.

Anchor Darling Drawing W9323936, Rev A 4.

Unit 2 FSAR, Amendment 9 S.

Unit 2 Technical Specifications, Arnendrnent 82 6,

Design Basis Document DSD-CVCS-2, Rev 0 7.

CP8 L Letter to NRC dated Sept 29, 1995 8.

FpL Letter to ADVdated August 16, 1996 9.

Union Pump Technical Manual 2998-3414, Rev 15 10.

Crane Technical Paper 410, 1991 Printing Prepared by Verified by Approved by Date 8- <<- P'