ML20059D182

Intervenor Exhibit I-MFP-172,consisting of Mgt Summary, Rev 00,NCR DCO-91-MM-N049, Deg 1-3 Test Cock Valve, 911002

ML20059D182
Person / Time
Site:	Diablo Canyon
Issue date:	08/23/1993
From:	AFFILIATION NOT ASSIGNED
To:
References
OLA-2-I-MFP-172, NUDOCS 9401070050
Download: ML20059D182 (13)

Text

t I

\*= fg> .2 75l a23 -Oy)-n ~T-AlFP -/ 7Z /W W"M M r

/ j93 Deaa .

I NUCLEAR REGULATORY COMMISSION hebt No. 94/5-011 omu E,uur##-/'72  :

yca oco_91 g eo4, aoy, oo in the muer er 84d /f/C In9Fcl flNd"7XICLI,C' October 2, 1991 m,.1 varv u o v sb: . _ _ _ _ _ _ _ _ a3 ' __

i w V aiac; C: t t ; " " .

4 cau=.ukbbkfkh5%:t our ^ a:.as S'&? .

Reporter f# tr#

t' MANAGEMENT

SUMMARY

f During surveillance testing on diesel generator 1-3 performed as part of post-maintenance testing, Mechanical Maintenance attempted to tighten the testcock~ valve on cylinder BR but inadvertently worked the 2R testcock.

During the tightening of 2R, the testcock valve (at the testcock to head adapter) broke off in the body of the cylinder head.

(The issue of inadvertently working 2R instead of 8R will be addressed separately by QE Q0008915.)

The root cause is that the threaded adapter failed due to l

mechanical fatigue cracking. The most probable cause for this high cycle fatigue is loosening of testcock 2R in service resulting in a " looser" fit and allowing the high i cyM a vibration to occur at the testcock connection.

\

l l Corrective actions to prevent recurrence will include:

l l

1. Revise STP M-81 to include test cock tightening (or l check of tightness) to ensure that tightness is i checked on a periodic basis. l

2. Submit an FCT to the current vendor manual to specify l copper washer at the testcock connection, OR, incorporate new vendor information.

l l

l 91NCRWP\91MMN049.PSN Page 1 of 13 9401070050 930823 l PDR ADOCK 05000275

! O PDR l

e i

NCR DCO-91-MM-N049 Rev. 00 October 2, 1991 NCR DCO-91-MM-N049 DEG 1-3 TEST COCK VALVE I. Plant Conditions Unit 1 and Unit 2 were in Mode 1 (Power Operation) at 100% power (ref. 1).

II. Description of Event A. Event (ref. 3 and ref. 9):

On June 3, 1991, at 0403 hrn, Diesel Generator (D/G) 1-3 was declared inoperable in preparation for clearance request (C/R) 00032003.

On June 3, 1991, at 2130 hrs, work was. completed by Mechanical Maintenance. In addition to other maintenance items, replacement.of the testcock valve at cylinder 8R had been accomplished.

On June 3, 1991, at 2156 hrs, the Shift Foreman reported off C/R 00032003. On June 4, 1991, at 0155 hrs, the D/G was started to perform STP M-9A3 as part of post-maintenance testing (not specifically for the 8R testcock). . Due to lack of clear communication, Operations'was not aware that the new testcock valve at 8R would require a leak check and possible tightening during the post-

' maintenance diesel run.

During the surveillance test run, Operations noticed leakage at the threads of the testcock at cylinder 8R. The Shift Foreman generated an Action Request (AR A0232834) documenting the leakage and requesting repair of the leaking l testcock.

Mechanical Maintenance inspected and replaced the copper washers on test cock 8R. The.D/G was run again for a leak check. Mechanical Maintenance (day shift) performed an initial leak check / tightness check on 8R upon starting of the diesel.

l l

91NCRWP\91MMN049.PSN Page 2 of 13-l l

y p-+--y ., y -M -

-+w5- T ---

p

1, NCR DCO-91-MM-N049 Rev. 00 October 2, 1991 l After appropriate run time to heat up the engine, ,

a second " hot" leak check / tightness check was l initiated by Mechanical Maintenance (swing shift). j i

Mechanical Maintenance inadvertently performed the leak check / tightness check on testcock 2R instead of 8R. 'During the tightening of 2R, the testcock valve (at the testcock to head adapter) broke off l

at the face of the diesel head. Action Request A0232933 was generated documenting the inappropriate maintenance attempt and subsequent damage to testcock 2R.

A Mechanical Maintenance Engineer was called in to-support corrective maintenance to 2R. The engineer contacted the vendor, Fraser Boiler and Diesel, and their representative (who was coincidentally in town) to. support troubleshooting.

Testcock 2R and associated parts were removed l

(easily) from the diesel and the testcock was j replaced.

(The issue of inadvertently working-2R instead of 8R will be addressed separately by QE Q0008915.)

(The 6/26/91 TRG agreed with the Operations Manager and the Mechanical Maintenance Manager's

. position that a lack of clear communication between Ops and MM would (and did, in this case) cause an inconvenience in completing the PMT, but not a safety or operability concern. See 6/26/91 TRG minutes on p. 11 below and Footnote and i special Comment #1 on p. 10 below.)

, (As the diesel start was not considered a valid or j non-valid failure, no Special Report is required by Technical Specification 4.8.1.1.4 (ref. 10 and ref. 11).)

l B. Inoperable Structures, Components, or Systems that Contributed to the Event:

None.

! 91NCRWP\91MMN049.PSN Page 3 of 13 e

r i

NCR DCO-91-MM-N049 Rev. 00 October 2, 1991 C. Dates and Approximate Times for Major Occurrences:

1. June 3, 1991, 0403 PDT: D/G 1-3 declared inoperable for maintenance (ref. 1).

2. June 4, 1991: Event / Discovery date.

Testcock valve at cylinder 2R broken (ref. 2).

3. June 5, 1991, 1522 PDT: Testcock 2R replaced and D/G 1-3 returned to service (ref. 1).

D. Otl.r Systems or Secondary Functions Affected:

None.

E. Method of Discovery:

Mechanical Maintenance personnel, during the performance of corrective maintenance rework on the diesel, immediately identified the broken test cock valve (ref. 1, 2, 3).

F. Operators Actions:

None.

G. Safety System Responses:

None. l l

III. Cause of the Event -

A. Immediate Cause: l l

Mechanical Maintenance tightened the 2R testcock, causing it to break (ref. 2 and ref. 3).

B. Determination of Cause: i

1. Human Factors: QE Q0008915 will address the I human factors of working the 1

2R testcock instead of 8R.

See also Footnote and Special i

l Comment #1 on p. 10 below.

j 91NCRWP\91MMN049.PSN Page 4 of 13

I y

NCR DCO-91-MM-N049 Rev. 00 '

October 2, 1991

a. Communications: N/A.

b. Procedures: N/A.

c. Training: N/A.

d. Human Factors: N/A.

e. Management System: N/A.

2. Equipment / Material:

a. Material Degradation:

Loss of cross-sectional- area due tx) fatigue cracking. 'However, per discussion.

between TRG chair and TES,-there was no evidence of material' flaws'(ref. ;B ) ' .

b. Design:

N/A. No design deficiencies were'noted by the TES report or in the TRG.-

c. Installation:.

As discussed in the 6/26/91 TRG, lack of a copper washer may have allowed the-connection to work loose.

d. Manufacturing:

N/A. .The report'from TES does not indicate.any manufacturing deficiencies..

Confirmed per discussion between TRG: chair and TES (ref. 8) - no evidence of manufacturing flaws.

e. Preventive Maintenance: N/A.

f. Testing: N/A.

g. End-of-life failure: N/A.

C. Root Cause:

The threaded adapter failed due to mechanical fatigue cracking.

The most probable cause for this high cycle fatigue is11oosening of testcock 2R-in service-resulting in a " looser" fit and allowing the high cycle vibration to occur at the testcock connection. The apparent ease in removing the broken piece from the diesel supports this scenario (ref. 3).

91NCRWP\91MMN049.PSN Page 5 of 13 ,

NCR DCO-91-MM-N049 Rev. 00 October 2, 1991 D. Contributory Cause:

Looseness of the connection may have been due to the lack of a copper washer. Current vendor literature does not require the use of a copper washer; however, a forthcoming vendor maintenance instruction will specify one.

IV. Analysis of the Event A. Safety Analysis:

The function of the diesel generators is to supply emergency power in the event that offsite power is not available. As this event was discovered and immediately repaired while the diesel was on a scheduled clearance, it did not immediately affect the operability of the diesel or its ability to supply power in an emergency situation.

The immediate corrective action to perform a

" ring" test on all other test cock. valves provides assurance that no other fractures of this type exist. Looseness of the connection was determined to be an isolated event, and will be prevented in the future by adding copper washers and periodically checking the tightness.

As discussed in the TRG, a failure of this type

. under operating conditions would not create a i high-energy missile hazard, and would not affect '

the overall operability of the diesel generator.

Therefore, this event did not adversely affect the -

health and safety of the public. l B. Reportability:

l 1. Reviewed under QAP-15.B and determined to be j non-conforming in accordance with Section l

2.1.8.

2.

Reviewed.under 10 CFR 50.72 and 10 CFR 50.73 per NUREG 1022 and determined to be'not reportable. The event was determined to be an isolated incident that did not affect the operability of the diesel.

91NCRWP\91MMN049.PSN Page 6 of 13

e*

NCR DCO-91-MM-N049 Rev. 00 October 2, 1991

3. This problem will not require a 10 CFR 21 report. The report from TES does not indicate any manufacturing defects (ref. 7 and ref. 8).

4. This problem will not be reported via an INPO Nuclear Network entry.

5. Reviewed under 10 CFR 50.9 and determined to be not reportable since this event does not have a significant implication for public health and safety or common defense and security.

6. Reviewed under the criteria of AP C-22 requiring the issue and approval of a JCO and determined that no JCO is required, as no degraded condition exists.

7. As the diesel test run was not considered a valid or non-valid failure, no Special Report is required by Technical Specification 4.8.1.1.4 (ref. 10 and ref. .11).

V. Corrective Actions A. Immediate Corrective Actions:

1. Testcock 2R and associated parts were removed from the diesel and the testcock was. replaced (ref. 3).

2. The engineer and the vendor rep requested a check of the remainder of the testcocks using a " ring" test. This test is accomplished by tapping the testcock lightly with a ball-peen hammer and ensuring a " ring" is heard. If the tap has a " dead" sound,'this would indicate looseness or perhaps a crack.

This test revealed that 1R and SR testcocks sounded " dead." Testcock 1R was successfully tightened. As discussed in the 6/11/91 TRG, testcock SR required replacement because the tightened valve would have been in an l unsuitable position (up against the body of l the diesel). l 91NCRWP\91MMN049.PSN Page 7 of 13

e NCR DCO-91-MM-N049 Rev. 00-October 2, 1991

3. AR A0233074 was generated to request. l investigative work orders.for the other four diesels. This investigation included tightening of the testcocks then performing a

" ring" test. Investigation was" performed on one diesel at a time, sequentially. As- '

discussed in the 6/11/91 TRG, no other cracked testcocks were noted. '!

B. . Investigative Actions:  ;

1. Both parts of testcock 2R were examined.by_MM' Engineering, vendor, and TES representatives. ,

RESPONSIBILITY: S. Allen .. ECD:- " COMPLETE-2PARTMENT: Mechanical Maintenance Tracking AR: A0233337 , AE #01 Testcock SR.was subjectedLto a liquid

~

2.

! penetrant test by ISI to identify any ,

cracking.

l

.1 (Test cock SR PT results were negative for cracks or indications of any kind.)-

RESPONSIBILITY: S. Allen ECD:- COMPLETE DEPARTMENT: Mechanical-Maintenance Tracking AR: A0233337 , AE.#02.

. 3. Testcock SR and the 2R: parts were sent to Tr9 for analysis via special courier, RESPONSIBILITY: S. Allen ..

' ECD: COMPLETE )

DEPARTMENT. Mechanical Maintenance Tracking AR: A0233337 , AE #03 I C.

Corrective Actions to Prevent Recurrence:

l 1.

Revise STP M-81 to include test-cock tightening (or check of tightness)'to. ensure-that tightness 1:s checked onLa periodic basis.-

RESPONSIBILITY: L.-Price .

' ECD: COMPLETE DEPARTMENT:

Tracking AR: ' Mechanical A0233337 Maintenance

, AE #04 Outage Related? No JCO Related? No NRC Commitment? No 91NCRWP\91MMN049.PSN Page 8 of 13 l

i NCR DCO-91-MM-N049 Rev. 00 October 2, 1991 CMD Commitment? Yes

2. Submit an FCT to the current vendor manual to -

specify copper washer at the testcock .

connection, OR, incorporate new vendor information.

RESPONSIBILITY: L. Price ECD: COMPLETE DEPARTMENT: Mechanical Maintenance Tracking AR: A0233337 , AE.#05 Outage Related? No.

JCO Related? No NRC Commitment? No  !

CMD Commitment? No VI. Additional InformatiqD

-i A. Failed Components:

1. Test cock valve-2R on D/G 1-3 broke.'  ;

B. Previous Similar Events:

None (ref. 6). A search of RMS: for NCRs Ror QEs with the DSCR field containing keywords DIESEL, GENERATOR, and VALVE found no similar items.

C. Operating Experience Review:

l

, 1. NPRDS: i As discussed in the 6/11/91 TRG, there are no >

other documented failures. ,

l

2. NRC Information Notices, Bulletins, Generic

, Letters:

I None (ref. 6). A search of the Operating )

Experience Assessment database under keywords DG and CRACK (ING) foundino similar items.

3. INPO SOERs and SERs: l 1

A' search of-the Operating- j None (ref. 6).

Experience Assessment database under keywords i l DG and CRACK (ING) found no similar' items. )

91NCRWP\91MMN049.PSN Page 9 of 13 j

l

~

i

'.' -I NCR DCO-91-MM-N049'Rev. 00  ;

October 2, 1991 '

D. Trehd Code:

l Responsible department HM (Mechanical  ;

Maintenance), and cause code C6  !

(Material / Equipment Deficiency, Other). The' root j cause was fatigue cracking due to in-service vibration. 1 E. Corrective Action Tracking:

1. The tracking action request is A0233337. l l

2. Are the corrective actions outage related?

No.

F. Footnotes and Special Comments:- )

1. As illustrated by this event,_ coordination of post-maintenance testing'on the diesel generators is not always clearly defined.

This does not represent a safety concern; however, this TRG will discuss possible prudent actions to improve the situation. See Prudent Actions on p. 12 below.

G.

References:

1. Control Room Logs dated 6/3/91 - 6/5/91 l

2. Initiating Action Request A0232933;-

i ARs A0232834 and A0233074

3. Broken Testcock Valve (2R) - DEG 1-3 writeup

4. E-mails dated 6/4/91 - 6/5/91 re:.DEG-1-3

5. Photographs of broken testcock valve

! 6.

7.

Search _of RMS and OEA databases --6/12/91-Memorandum from TES to Mechanical Maintenance dated 6/25/91 re: Metallurgical Failure l

' Analysis of_ Threaded Coupling from Diesel Emergency Generator Petcock #R-2_.

8. June 25, 1991 Telephone Record by.S. Allen to .

D. Aguiar

9. Sequence of Events - writeup.from10ps

10. Surveillance Test Procedure (STP)LM-9I,

" Diesel Generator Testing Frequency.

Determination" dated 6/5/91

11. Technical Specification 4.8.1.1.4 91NCRWP\91MMN049.PSN Page 10 of 13

I l

0 w%

NCR DCO-91-MM-N049 Rev. 00 October 2, 1991 ;

l H. TRG Meeting Minutes:

On June 11, 1991, the initial TRG convened and considered the following:

1. The TRG discussed the event. As most of the i fracture surface was dull with only a small part shiny (ref. 5), it appears that the fracture existed prior to -the event. Root  :

cause will be discussed when the TES analysis is completed.

2. The personnel error issue.(working testcock 2R instead of 8R) will be addressed by a separate QE. The Mechanical Maintenance Manager.will be consulted to determine if the lack of communication issue (between MM and ops) will be addressed separately.

3. The TRG discussed that a failure of this type ,

under operating conditions would not create a high-energy missile hazard. Such a. failure would affect the power output of the affected cylinder, but would not impact the overall operability of the diesel.

4. This TRG will reconvene to discuss-rvv. cause on June 26, 1991, after results of the TES analysis are received.

On June 26, 1991, the TRG reconvened and considered the following: )

1. The TRG agreed with the Operations Manager and the Mechanical Maintenance Manager's position  ;

that a lack of clear communication between Ops '

and MM would (and did, in this case) cause an inconvenience in completing the PMT, but not a ,

safety or operability concern. Though the PMT )

was performed by Ops without MM input, the DEG ]

would not have been declared ~ operable without l an MM signoff. This TRG will not address the  !

entire PMT program; however, coordination of ]

PMT on the DEGs (which requires coordinating several organizations) will be discussed.

i 91NCRWP\91MMN049.PSN Page 11 of 13

l . l t

~

NCR DCO-91-MM-N049 Rev. 00 October 2, 1991

2. The TRG determined root cause and contributory causes, as noted in section III on p. 4 above.

u

3. The TRG determined corrective actions to prevent recurrence, as noted in section V on

p. 7 above.

4. This TRG will reconvene to discuss Post-Maintenance Testing, date to be determined.

l On July 9, 1991, the TRG reconvened and considered f

the following:

1. The TRG discussed revisions to the NCR write-t,: Description of Event and Root Cause.

2. The TRG agreed on Prudent Actions (see below).

! 3. The TRG discussed coordination of post-l maintenance testing on the diesels and agreed i

on Prudent Action #2 below.

4. ECDs for outstanding actions were discussed _

and will be finalized by the TRG chairman.  !

This TRG will not reconvene. Overall ECD for i the NCR will be 9/30/91.  !

(Subsequent to the meeting, the TRG chairman  !

. provided an ECD of 9/30/91 for Corrective Action j

1. 2 and an overall ECD of 11/1/91 for the NCR.) i I. Remarks: l t

Prudent Actions (not required for NCR closure): l

1. Replacement testcocks for all five diesels will be ordered and expedited to support a scheduled replacement of all old style testcocks with the newer style testcocks. The style change was only to simplify test  ;

j connection activities. However, upgrading of the testcocks is considered a conservative

" good practice" at this point.

Writt an AR for each diesel generator to replace all old style testcocks with the newer style testcocks.

91NCRWP\91MMN049.PSN Page 12 of 13 l

l

l 1

3 .. .

i i

$ h.

NCR DCO-91-MM-N049 Rev. 00:

i October 2, 1991:

s RESPONSIBILITY: L. Price ECD: COMPLETE l DEPARTMENT: Mechanical Maintenance '

j Tracking ARs: DEG 1-1 A0237516' DEG 1 A0237517 DEG 1-3 A0237519 DEG 2-1 A0235730 S DEG:2-2 A0235731 -

4 0 2. For'the six-month diesel generator maintenance:

windows, the system engineer will be.the HIT team leader'and will coordinate all maintenance and PMT testing for the window. ,

The System-Engineering Manager.will write:a memo describing the system engineer's responsibility.

RESPONSIBILITY: S. Allen: ECD: 9/30/91

~

DEPARTMENT: Mechanical ~ Maintenance i

L s

l 91NCRWP\91MMN049.PSN Page 13 of- 13

_. _ _ , . , , . _ , , - . . . . . - . _ . . _ . . - _ - . . _. . _ - . . _ . _ . . _