ML20059M829

Intervenor Exhibit I-MFP-27,consisting of 930727 Draft Rept, Wrong Oil Used in Asw Pump Motors

ML20059M829
Person / Time
Site:	Diablo Canyon
Issue date:	08/17/1993
From:	AFFILIATION NOT ASSIGNED
To:
References
OLA-2-I-MFP-027, OLA-2-I-MFP-27, NUDOCS 9311190364
Download: ML20059M829 (12)

Text

__ -

Mff' f4hobdKf 6o - 2 75 a u -O 9 - L pn e.f u s.

1-- MPP-2 7 e

e

(

NCR DCO-93-MF'if U""ev.

00 DRAFT:

Ju 27, 1993 l

'93 00T 28 P 6 :16 l

l MANAGEMENT

SUMMARY

l

.y _

n.vc:

l t.: t t

During the performance of work order R0101407, the incorrect oil for the ASW motors was found at the intake storage area.

This oil was found in a one gallon container labelled GST-32 oil for.

l l

ASW Motors. The correct oil f m ASW motors is AW machine-100 oil.

Test results of the oil sample indicated that some mixing of GST 32 and AW-100 was present.

This sample was primarily AW-100 with j

some GST-32 mixed in.

The lab could not determine the amount of l

GST in the sample.

Using the viscosity.of a new AW-100 sample l

and the results from the lab analysis, it is estimated'that the mix was less than 10% or approximately 1 quart of GST.

Per conversation with Chevron mixing of these amounts will not cause l

l lubrication problems.

Mixing of these oils wouM cause small I

amounts of precipitates.

Precipitates result from the interaction of the inhibitor additives in the two oils.

The amount of precipitation would be minimal and should not cause degradation of the bearing.

Westinghouse was contacted and could not provide any additional information.

The root cause and corrective actions will be determined by the TRG.

NUCLEAR RfCULATORY COMMISSION Docket No. 9.215-/1L3_ om ta na7TF P MI fn the matter of.D{Ciplo (= AS % {[J1C$ict(Q.

WI IDENTIFl!D /

A; punt _

sqq;ygg /

Nv":*14 _ ___ _ tacno-Cm i ;;if'r 9311190364 930817 N

bb PDR ADOCK 05000275 L*"

wam A t-_I

%:rtst _{Cjkt? 'I4i9t i G

PDR i

_ J 92NCRWP\\93MFN039.PGD Page 1

of 12 hl/

h

t-j NCR DCO-93-MF-NO39'Rev. 00 DRAFT:

July 27,.1993

')

NCR DCO-93-MF-N039

~

WRONG OIL USED IN ASW PUMP MOTORS I.

Elant conditions Modes and power of operation are not applicable to:this-condition.

II.

Description of Event A.

Summary:

During the performance of work order R0101407,'the incorrect oil for the ASW motors was found at the intake storage area.

This oil was'found in a one.

gallon' container labelled GST-32 oil for ASW M.R.

The correct oil for ASW motors is.AW machine-100 oil.

Contrary to Administrative' Procedure (AP).D-753

" control of plant lubricants" which lists'the applications and the recommendations for use of each lubricant, GST 32 was added in1the ASW pump. motor instead of the required AW machine-100-oil.

B.

Background:

This the latest of several instances of. procedure A.P D-753 violations were noted.

For instance:

1.

On March 26, 1987 a Notice of ' Violation' (NOV) was' issued to DCPP by the NRC fe failure <co comply with Administrative Procedure (AP D-753) " Control of Plants Lubricants".

Contrary to.the-requirements of this procedure a onefgallon-unlabelled container filled with oil was in the tool shed area of the intake structure,'another1 gallon unlabelled container filled with oil was in a storage cabinet in the new cold machine shop, and three unlabelled grease guns'were found in the hot-machine rhop tool room.

In' addition the log books i

were not being maintained in the bulk storage; areas or at any of the dispensing rooms.

2.

During a review of PM work order # R0051375.it was-

/

identified that on April;10, 1990 the wrong oil (GST-32) was added to the: heater-2 drain: tank pump 2-1.

The required oil should have been GST-68.

92NCRWP\\93MFN039.PGD Page 2

O f.. 1:2.

~l

L.

NCR DCO-93-MF-NO39 Rev. 00 DRAFT:

July 27, 1993 3.

During a review of the oil addition log at the aux.

control board, it was noticed that-the wrong oil (GST 32) had been added on November 3, 1991 to the CCWP 2-1 motor bearing instead of GST-68 required by AP D-753.

4.

On July 12, 1991 NCR DCO-91-MM-N061 was written to identify discrepancies noted in lubrication storage and handling.

5.

On January 8, 1992, approximately 8 oz. of GST-32 was taken from the lube oil. storage room for use'in the motor bearings for CCW pumps 11, 12 and 13.

GST 68 is designated for use in the motor bearings while GST 32 is designated for use in the pump bearings.

C.

Event

Description:

June 22, 1993:

during the' performance of a Preventive Maintenance (PM) on Aux Salwater pump 1-2 per-work order R0101407, a. half gallon of oil in a one. gallon-container labeled ASW MTR. GST-32 was found-at the intake storage area. The correct oil for ASW motors is AW MAHINE-100. A review of the oil log books did not reveal where or when this oil was obtained. A check with predictive maintenance indicated taht these oils are not compatible.

Sample of the oil for ASW pump 1-2 motor bearing was sent to the Herguth laboratories for testing. Test results indicated that this oil was not contaminated. However, it was then decided to change the oil in all the other'three ASW pumps and' send samples to the lab for testing.

June 23, 1993:

the bearing oil in the lower and upper bearings of ASW 1-1 pump motor was replaced in accordance with AP D-753.

Oil samples were taken and sent out for testing.

Test results' indicated that-the oil in ASW pump 1-1 motor bearing l

was contaminated.

Sample was primary AW-100 with some GST-32 mixed in.

The l

lab could not determine the amount of-l GST in the sample.

Using the viscosity of a new AW-100 sample and the results-from the lab analysis, it is estimated 92NCRWP\\93MFN039.PGD Page 3

of 12

NCR DCO-93-MF-NO39 Rev. 00

. DRAFT:

July 27, 1993 that the mix was less than 10% or approximately 1 quart of GST.

Per conversation with Chevron mixing of

• these amounts will not cause lubrication problems.

Mixing of these oils would cause small amounts of precipitates.

Precipitates result from the interaction of the inhibitor additives in'the two oils.. The amount of precipitation would be minimal and should not cause degradation of the bearing.

Westinghouse was contacted and could not provide any additional information.

June 29, 19:3:

Suspect bearing 1 oil wae found in. upper and lower bearing reservoirs in ASW 1-1 pump motor.

Oil drained and samples taken. Oil was replaced in accordance with AP D-753. Samples sent for analysis and found to be contaminated.

On July 20, 1993 samples suspect bearing oil taken from the and lower bearing reservoirs Lin ASW 2-1 and 2-2 pump motor were analyzed and found to have~a mixture of AW-100 and GST-32.

On July 22, 1993 contaminated bearing oil was changed in upper and lower bearing reservoirs in ASW 2-1 pump motor.

On July 23, 1993 contaminated bearing oi' was cha~nged in upper and lower bearing reservoirs in ASW 2-2 pump motor.

D.

Inoperable Structures, Components, or Systems that Contributed to the Event:

None.

E.

Dates and Approximate Times for Major Occurrences:

f 1.

March 26, 1987:

Notice of Violation (NOV) was issued to DCPP by the NRC for failure to comply with Administrative Procedure D-753.

4 2

April 10, 1990:

During a review of 144 work order ft R0051375 the wrong oil (oil GST-32) was added to 92NCRWP\\93MFNO39.PGD Page 4

of 12

lt-NCR DCO-93-MF-NO39 "~r.

00 l

l DRAFT:

July 27, 1993 l

the heater 4! drain tank pump _

2-1.

3.

November 3, 1991:

The wrong oil (GST 32) had been added on to the CCWP 2-

~

f 1 motor bearing instead of GST-68.

4.

July 12, 1991:

NCR'LCO-91-MM-N061 was written to identify discrepancies noted in lubrication storage and l

handling.

5.

January 8, 1992:

App. 8 oz. of_GST-32 was i

taken from the lube oil storage room for use in the motor bearings for CCW pumps l

11, 12 and 13.

6.

June 22, 1993:

Event date. during the performance of WO R0101407, the incorrect oil for the ASW motors was-found at the intake storage area.

F.

Other Systems or Secondary Functions Affected:

None.

i G.

Method of Discovery-On June 22, 1993 during the performance of work order R0101407, the incorrect oil for the ASW motors was found at the intake storage area.

H.

Operator Actions:

)

Not applicable.

I.

Safety System Responses:

None.

92NCRWP\\93MFN039.PGD Page 5

of 12

i NCR DC0-93-MF-N039 Rev. 00 DRAFT:

July 27, 1993 III.

Cause of the Event A.

Immediate Cause:

To be determined.

B.

Determination of Cause:

1.

Human Factors:

a.

Communications:

b.

Procedures:

c.

Training:

d.

Human Engineering:

e.

Management System:

2.

Equipment / Material:

a.

Material Degradation:

b.

Design:

c.

Installation:

d.

Manufacturing:

e.

Preventive Maintenance:

e f.

Testing:

g.

End-of-life failure:

C.

Root Cause:

To be determined.

D.

Contributory Cause:

To be determined.

IV.

Analysis of the Event A.

Safety Analysis:

On June 22, 1993 during the performance of work order R0101407, the incorrect oil for the ASW 1-1 motor was found at the intake storage area.

92NCRWP\\93MFN039.PGD Page 6

of 12

L*

'NCR DCO-93-MF-NO39 Rev. 00 DRAFT:

July 27, 1993 On June 23, 1993' the bearing: oiliin. the ' lower and upper bearings of ASW 1-2. pump motor.was' replaced.in accordance with AP D-753. Oil' samples:taken.

Test results of the oil sample indicated thatLsome mixing of.GST 32 and AW-100 was present.

This sample was primarily AW-100 with some GST-32 mixedfin.- The lab could not determine the-an.ount1of GST in the sample.

Using.theLviscosity of a new AW-100, sample and.

the results from the lab: analysis, it'wascestinated that the mix was less than 10% or approxinately 1 quart of GST.

Per conversation with Chevron mixing of these'

+

amounts will not-cause: lubrication problems..' Mixing:of-these oils would cause'snall amounts of precipitates.

due to the inhibitor additives in the oil.

The amount of precipitation wouldfbe' minimal and should-not cause r

degradation of the bearings.

^

l Subsequently, tests runion samples!taken from ASW 1-1,.

~

l 2-1 and 2-2 indicated that the oil had also been mixed in these pumps.

l Based on the above it appears that. 5 l pints i(.24 ; qts) l of GST-32 was added to-the reservoirs of.the pumps. The l

reservoir capacity has been determined:to'be 1 quart.

j Using viscosity blend charts with.75% of GST-68 and 25%

~

t of GST-32,-the average viscosity 11s reduced.from.335'to~

l 280 SUS e 100 degrees F.

t The vendor recommended oil viscosity'has:been-determined to be 270-325 SUS at 100 degrees.F.

i I

)

ASTM viscosity / chart temperature shows no' impact on?

L existing TI setpoints.

The mixture of GST-68 with the same family of mineral oil (GST-32) in the amount listed above,.would not affect the motor operation, TI setpoints, compatibilities, etc.

In addition Chevron's GST oils '(acidic additives) and machine oils (alkaline additives)~are-not compatible.

However, since the ASW motor initially contained 100%'

alkaline oil (AW machine oil) andithe acidic ~ (GST) oil added in topping off the reservoir, the ratio'of volumes of acidic / alkaline oil is?to small that the precipitate (if there was a reaction between the two oils) would be negligible.

According to Chevron, the alkaline oil is also capable of keeping lthe small amount of reaction material in_ suspension..,For?these-reasons Chevron does not require intermediate flushing 92NCRWP\\93MFN039.PGD Page

7) of 12

~

NCR DCO-93-MF-N039 Rev. 00 DRAFT:

July 27, 1993 if the oil was changed from acidic to alkaline (GST to machine oil).

In conclusion considering.the worst csJe scenario of a mixture of 50% of alkaline oil-and 50% of acidic oil, *he amount of precipitates in suspension would not affect the immediate operability of the pump.

Eventually, after a long period of time, the oil would brake down and the temperature of the bearing would increase which will cause failure of the motor bearing..

However, this would give ample. time for. replacement of tne bearings before the pump would fail.

Thus considering this worst case scenario,. mixture of the two oils would result in increase wear of the l

bearing and eventual failure af the pump which would be capable to operate long enough to pcrform its indented safety function.

Therefore, based on the' discussion above the health and safety of the public were not affected by this event.

B.

Reportability:

1.

Reviewed under QAP-15.B and determined to be non-confonaing in accordance with Section 2.1.7.

2.

Reviewed under 10 CFR 50.72 and 10 CFR 50.73 per l

NUREG 1022 and determined not to be reportable.

J 3.

Reviewed under 10 CFR Part 21 and determined that this problem will not. require a 10 CFR 21 report,-

1 since it does not involve defects in vendor-supplied services / spare parts in stock.

l 4.

This problem will not be reported via an INPO l

Nuclear Network entry.

]

5.

Reviewed under 10 CFR 50.9 and determined to be not l

reportable since this event does not have a i

significant implication for public health and safety or common defense and security.

6.

Reviewed under the criteria of AP C-29 requiring the issue and approval' of an OE and determined that i

an OE is not required.

V.

Corrective Actions A.

Immediate Corrective Actions:

To be determined.

92NCRWP\\93MFN039.PGD Page 8

of 12 r

NCR DCO-93-MF-NO39 Rev. 00 DRAFT:

July 27, 1993 B.

Investigative Actions:

1.

Rewrite the NCR description to change its scope from " Wrong Oil in ASW Pump Motors" to " Lack of Control of Lubricants" and have signed by the.

RESPONSIBILITY:

P. Dahan ECD:

08/04/93 DEPARTMENT:

Peulatory Compliance Tracking AR:

A0312554, AE #__

2.

Regulatory Compliance (Paul Dahan) to perform a time line of the events and a barrier analysis for this NCR and submit them to the TRG members for review.

RESPONSIBILITY:

P. Dahar. ECD:

08/04/93 DEPARTMENT:

Regulatory Compliance Tracking AR:

A0312554, AE 3.

Material director (David Oatley) to form an employee involvement team to develop corrective actions based on the root cause of this NCR.

4 RESPONSIBILITY:

D.

Oatley ECD:

08/04/93 DEPARTMENT:

Materials Tracking AR:

A0312554, AE i

C.

Corrective Actions to Prevent Recurrence:

1.

To be determined.

DEPARTMENT:

ECD:

RESPONSIBILITY:

Tracking AR:

A0 AE #__

"o OUTAGE: ( R_ ' or U_FO)

Outage Related?

OE Related?

' es/

)

NRC C'ommitment?

'eE," o

]

CMD Commitment?

q<- No D.

Prudent Actions (not required for NCR closure) e To be determined.

VI.

Additional Information A.

Failed Components:

None.

B.

Previous Similar Events:

92NCRWP\\93MFN039.PGD Page 9

of 12

'e NCR DCO-93-MF-NO39 Rev. 00 DRAFT:

July 27, 1993 1.

Notice of Violation 50-275/87-01. 50-323/87-01.

On March 26, 1987 a Notice of Violation (NOV) was issued to DCPP by the NRC for failure to comply with Administrative Procedure (AP D-753) " Control of Plants Lubricants".

Contrary to the requirements of this procedure a one gallon unlabelled container filled with oil was in the tool shed area of the intake structure, another gallon unlabelled container filled with oil was in a storage cabinet in the new cold machine shop, and three unlabelled grease guns were found in the. hot machine shop tool room.

In addition the log books were not being maintained in the bulk storage areas i

or at any of the dispensing rooms.

2.

NCR DCO-91-MM-N061 - Storace and Handlina of-Lubricants.

During walkdowns of the Turbine Building 85' elevation QA identified discrepancies in the lubricants storage areas.

This is documented in QA Assessment #91025I.

These discrepancies on storage and handling of lubricants include unlabelled grease guns, potential cross contamination of greases through the use of a single pump.

The root cause of this event has been determined to be &

lack of specific ownership of the lubric?nts storage area. The immediate corrective actions were as follows: (1) The PM's as required.by AP C-750 which require lubrication were either rescheduled or postponed pending evaluation of the lubricants; (2) evaluation of the mixing'of lubricants or the usage of the wrong lubricants were-conducted and no potential adverse effects on equipment were noted.

The corrective actions to prevent. recurrence are:

(1) Mechanical maintenance will appoint a foreman prior to 2R4 to oversee the lubrication storage area, to include a weekly surveillance of the area to insure requirements are met; (2)

Mechanical maintenance will perform an additional' monthly audit of the area; (3) Enhance and standardize'the markings on the grease-dispensing tools.

These corrective actions were intended to prevent' recurrence by appointing a foreman to oversee the lubrication storage area and perform a weekly surveillance of the area to insure that 92NCRWP\\93MFN039.PGD Page 10 cf 12

i-NCR DCO'-93'MF-NO39 Rev. 00 DRAFT:

July 27, 1993 requirements are met did not prevent 1 usage of the wrong lubricant.

C.

Operating Experience Review:

1.

NPRDS:

l Not applicable.

2.

NRC Information Notices, Bulletins,. Generic Letters:

To be determined.

3.

INPO SOERs and SERs:

To be determined.

D.

Trend Code:

Responsible department __,'and cause. code __.

E.

Corrective Action Tracking:

1.

The tracking action request is A0312554.

2.

Are the corrective actions outage related?- To be-determined.

F.

Footnotes and Special Comments:

l NRC CONCERNS:'.

Why an official and complete POA was not written.

(the words in the AR providing operability.

i l

assessment only include notifying.the' oil manufacturer, not the pump vendor) ? -

After receiving the test.results identifying the wrong oil in Unit 1,.why wasn't Unit ~2fsampled-immediately?

How long was the wrong oil in, Unit 1?

When the Unit 1 test resultsJidentified the' incorrect oil, why wasn't an NCR initiated?

f The oil manufacturer states'that precipitate's;will-form.

What are the characteristics of those precipitates (e.g.,

size, affinity'for surfaces)?'

92NCRWP\\93MFN039.PGD Page 11 off L12' l

l t

4 i

NCR DCO-93-MF-N039 Rev. 00 DRAFT:

July 27, 1993 Could they block ports in the motor-(it light) or will they sink to the sump (if heavy)'and wait for l

the next seismic event?

What are the critical characteristics of the oil mixture (e.g. density, viscosity, temperature characteristics)?

Do they resemble the profiles of the correct oil?

Violation of AP D-753, " Control of Plant' Lubricants," in that no log entry was made for'the oil.

Previous NOV on control of plant lubricants was in IR 87-01 (March 1987).

I G.

References:

1.

Initiating Action Request A0232586.

H.

TRG Meeting Minutes:

On July 28, 1993, the TRG convened and considered _the following:

Agenda.

Change the scope of the NCR.

Review of the voting members for this TRG. The voting members are electrical maintenance, mechanical

~

• maintenance, materials, QC and QA.

Review of the NCR concerns.

Regulatory Compliance to perform a time line of the events.

POA not written.

Did not know that we had I.

Remarks:

None.

92NCRWP\\93MFN039.PGD Page 12 of-12

_