ML20062M735

Intervenor Exhibit I-MFP-62,consisting of DCO-93-SS-N007,dtd 930427

ML20062M735
Person / Time
Site:	Diablo Canyon
Issue date:	08/19/1993
From:	AFFILIATION NOT ASSIGNED
To:
References
OLA-2-I-MFP-062, OLA-2-I-MFP-62, NUDOCS 9401110163
Download: ML20062M735 (23)

Text

_ . .

" $ OL fn#in i ah, or PAeiR6 66 6d TME 00icial Exh. No.N ' -

l <

say ictm < - . ,

' "ic 9 m rw%

SS-N007  %

% ,,. two / _ " ' A'pril . 27 1993

,,. y v M WL - . _ . . -

[

m u.,t , it  ! SUI l w Wiawss ,

F . putter d J

MANAGEMENT

SUMMARY

I During piping modification activities, rust colored water was found in the fire protection carbon dioxide (CD,) pipirg that supplies suppression capabilities to the. Unit 2 diesel generators (DGs).

Da Jaros y 27 19H w*tte utc1 ting a flange conne: tion to install a blank in CO, line # K2-4540-3" to perfo*m tie-tn 'or the DG 2-3 CD, system, water leaked f rom the open flange. This flange is located in a vertical portion of the line. The blank flange installation was completed which isolated the Unit 2 DG flange f rom the CD, supply.

A continuous fire watch had been established in the Unit I and Unit 2 cable spreading room (CSR) before this evert, to c aper: sate for tFe automatic lo. pressure CO, system being out-of-service for an unrelated design issue. An hourly fire watch had been est.,blished in other areas as a fire protection progra'r geoo practice, fulfilling the reautrements of the equipment control guideltnes (ECGl.

Subseouently , durino the removal of a een e i the 2-%" line in the hallway at elevation 102',

apprc> ma's , t a g. ;rt cf water arent,ec out of the line.

The same day. subseweet to disccvery of water tr' the 2-W" line, water was also identified in the %"

line to the cilot cataret On January 28, 1993, approximately h galion of water was drained f rom the %" line to the pilot cabinet. The 3" header sverlying CO, to the Unit 2 DM and C5R were declared inoperable.

On Fetruaey !! 19L the 3" heade* that supplies CE tc b:th the Uaits I and 2 D5s and CSRs was l 1solated with spettacle flanges to facilitate investigative actions._ On February 12,'1993, the two l expansion joints irt the header were found to leak.

The root cause of this event was standing water trapped inside' the piping expansion assembly bellows that permitted galvantc corrosion to occur between the carbon steel flange and the Minless steel bellows, this process resulted in the formation of a crevice between the two galvanically dissimilar metals. Crevice corrosion (a particular type of p tting corrosion) followed and eventually caused the observed " pinhole *' leak The CO, piping has been repaired and returned to service.

On March 23, 1993, at 1:00 pm PST the TR3 reconvened in room 327 of the administration building to I discuss the status of identified actions. The TRG discussed various options for confirming the integrity of tne t : incn CO, system piping until its replacement (as part of ref. 2). Corrective-action V.C.2. is intended to acccmplish this.

The TRG deterrrineo that the identified actions are suf ficient to address the concerns of this NCR, Closure of these actions is further sufficient to complete this NCR, No reconvene of this TRG is planned unless ever.ts require it. The ECD for this MR~)s July 15, 1993.

s3NCad\93Ssh00*.JC Page 1 of 17 3 2 75 a z 3 -Ou - 2., 2 M PP- d L E /9 9'3 9401110163 930819 PDR ADOCK 05000275 O PDR ^

i DC0-93-SS-N007 April 27, 1993 D. Inoperable Structures, Components, or Systems that Contributed to the Event:

Based on the configuration and geometry of the system and the location of the water identified in the Unit-2 diesel generator cardox piping, the cardox piping header that supplies the DGs and the CSRs is considered inoperable.

E. Dates and Approximate Times for Major Occurrences:

1. January 27, 1993: Event / discovery date. Water leaked from the open flange in the CO, line

1. K2-4540-3" and approximately six

- gallons drained from the 2-%" line on the 102' elevation. The %" line to the pilot cabinet has water in I it, l

2. January 28, 1993: Approximately % gallon was drained from the line to the pilot cabinet.

Based on previous observations, the header was declared inoperable.

F. Other Systems or Secondary Functions Affected:

None.

G. Method of Discovery:

On January 27, 1993 while unbolting a flange connection to install a blind flange in CO, line # K2-4540-3" to pe. form tie-in for the DG 2-3 CO, system, water leakenzfrom the open flange. This flange is located in a vertical portion of the _m line.

H. Operator Actions:

i None required.

1. Safety System Responses:

None required.

111. Cause of the Event A. Immediate Cause:

u~: w s m .a u c~ Page 4 of 17

DC0-93-SS-N007 April 27, 1993 The piping expansion joints were found to have through-wall leaks that permitted standing water in the pipe trenches to leak into the cardox pipe.

B. Determination of Cause:

The potential for pipe degradation and leakage was previously identified in NCR DC2-92-TN-N028, "ASW Annubar Line/DF0T Line Corrosion," (ref. 2).

J C. Root Cause:

The root cause of this event was standing water trapped inside the piping expansion assembly bellows that permitted galvanic corrosion to occur between the carbon steel flange ,

and the stainless steel bellows. This process resulted in l the formation of a crevice between the two galvanically l dissimilar metals. Crevice corrosion (a particular type of 1 pitting corrosion) followed and eventually caused the

! observed " pinhole" leak.

D. Contributory Cause:

None, i

l IV. Analysis of the Event A. Safety Analysis:  ;

1. An evaluation was performed by Fire Protection and NECS-Engineering. The low pressure cardox' system consists of five separate headers with N/C solenoid valves:

a. Unit 1 and Unit 2 DG rooms and CSRs.

b. Unit 1 L0 Room. l

c. Unit 2 L0 Room.

l

d. CO, Hose reels stations. i l  !

e. Unit I and Unit 2 # 10 turbine bearings.

Based on the configuration and geometry of the system and the location of'the water identified in the Unit 2 DG cardox piping, the cardox piping header that supplies the DG and the CSR is considered inoperable.

l l . m. n m s~e . w. Page 5 of 17 i i

.- . . . -. . - .-~ - - . . . .- _. . . _ - .

.c  !

DCO-93-SS-N007 April 27, 1993' The compensatory measures for these areas are described in ECG 18.5 and require'a continuous-firewatch in areas  !

.where redundant systems or components could be damaged-and a fire watch patrol in other areas. These'

~

compensatory measures are in place. The remaining four headers are considered operable'because of the normally. '

closed isolation' valves and the results of STP M-39 series cardox testing that'has'been performed in the past, i

2. PROBLEM The DG CO, header- _ running along the Unit 2-turbine building in the trench below the 85' elevation in thel ,

buttress area was founc to be full of water at the' low'  !

point. Additionally,' the CO, header in the DG. hallway ,

i at 'he 102 elevation was found to ba full of water at the north end of that header. .These headers are t

' normally full: of C0j gas and are at atmospheric.-

pressure due to valve leakage.and intentional venting,  ;

except during an actuation' of -one of the' fol_ lowing C0,  :

zones:

Unit 1 Cable Spreading Room Unit 1 DG l-1 Room j Unit 1-DG l-2 Room :1 Unit 1 DG l-3 Room-  !

Unit 2 Cable Spreading Room I

Unit 2 DG 2-1-Room Unit 2 OG 2-2 Room could this water _ prevent or degrade the ability of the  ;

CO, systems to extinguish a fire. in tha Unit 2 DG' 2-1  !

or 2-2 rooms?

ASSUMPTIONS l a. A CO, system pipe full of water located in the.  !

l trench outside the turbine building'will not result in pipe, pipe hanger, valve or nozzle  ;

failure when that water is pushed through the system during any actuation.of the CO, system on? j that header. i

b. A CO, system pipe full of water located in the DG hallway inside the Unit 2 turbine building will not result in pipe, pipe hanger, valve or _ nozzle-fail're u when that water is pushed through the

.mm,3sse x Page 6 of 17-7

~

DCO-93-SS-N007 April 27, 1993 system during any actuation of the CO, system on that header.

ASSESSMENT In the event of an actuation of any CO, system zone other than the Unit 2 DG rooms, no appreciable effects would be noticed at the CO, tank, main header or actuating zone. since any water would be initially pushed away from the actuating zone toward the Unit 2 DG rooms and then reach virtually equilibrium pressure conditions of about 300 psig for the duration of the discharge. There would be insignificant return of the water toward the actuating zone after completion of the discharge, since the header.is allowed to depressurize slowly, and there would be only a small amount of gas trapped downstream of the water to expand and push the -

water back.

In the event of an actuation of either Unit 2 DG 2-1 or 2-2 room CO, system zones, the header would be pressurized upstream of any water with the opening of the master valve at the tank and, shortly thereafter, depressurized downstream of the water by the opening of the local selector valve. This would tend to entrain water in the flow of the CO, gas, and possibly push

" slugs" of water ahead of most of the CO, to be discharged. The distance of pipe run to be traversed by CO, from the tank to the syston zone is several hundred feet in length. During initial startup CO, discharge testing of the DG 2-1 room, it took about one minute of discharge to receive liquid CO, at that zone.

The initial gas discharged from the header is present at normal ambient temperatures of from 50 to 120*F. As gas from the tank reaches the discharge zone the temperatures will drop to well below freezing. By that time it can reasonably be expected that the water would have either have been. entrained in the gas flow or would have been discharged in " slugs" of water into the discharge zone. Some limited freezing of remaining water may occur and malfunctioning of the solenoid valve or local selector valve may occur. Initial action of the solenoid valve and local selector valve will likely occur before cold gas or liquid CO, reach the valves due to the long header distances. Any adverse malfunction of these valves would be limited to l f ailure-to-reposition-to-close the local selector valve, thus resulting in a somewhat extended discharge, l

. w. n,m.w . w. Page 7 of 17 l

l

DCO-93-SS-N007 April 27, 1993 or boundary failure allowing leakage or discharge from the header outside that discharge zone. An extended discharge would be limited by the master valve closing in its normal time cycle. The master valve is normally set to discharge only seconds more than the local selector valve, so this minor, extended discharge would not prevent the system from performing its design fire suppression purpose. If a boundary failure occurred and allowed CO, discharge outside that discharge zone, l some reduction of the final concentration of CO, in i that discharge zone might occur. A'significant amount of water in one location at a very low temperature would be required to cause this failure mechanism. The significant flow of gas through the line prior to low temperature or liquid CO, reaching the location of any water makes the collection of significant amounts of l water in one location unlikely. In addition, e long transit time for the liquid CO, to reach the zone (about 57 seconds) compared to the actual discharge time of the zone (about 75 seconds), limits the amount of CO, lost from the normal discharge. It can be.

reasonably concluded that significant damage and leakage that would materially affect the final discharge zone concentration would not occur, and the system would not be prevented from performing its design fire suppression purpose.

Water discharged onto the DGs and their auxiliary equipment has not been evaluated since the functioning of a particular DG and auxiliary systems would affect only the DG at the location of a fire. The Apper. dix R analysis demonstrates that the loss of all _ equipment in one fire zone will not prevent the ability to reach and maintain safety cold shutdown of the plant.

If a CO, discharge was due to a test discharge in one zone, water damage would be noted and other zones would not have been subsequently discharged. The analyses of Appendix R would remain valid.

Thus the health and safety of the public was not adversely affected_by this condition.

B. Reportability:

1. Reviewed under QAP-15 B and determined to be non-conforming in accordance.with Section 2.1.8.

.mmons~w o Page 8 of 17

DCO-93-SS-N007 April 27, 1993

2. Reviewed under 10 CFR 50.72 and 10 CFR 50.73 per NUREG 1022 and determined to be not reportable. Engineering evaluation determined that the cegradation of the CO, piping occurred over time and a particular date could not be assigned for when it became inoperable.

Although the CO, piping did become inoperable, further engineering evaluation has determined it was functional throughout this time. Compensatory measures have been in place and plant fire p otection was considered to be adequate.

3. Reviewed under 10 CFR Part 21 and determined that this problem will not require a 10 CFR 21 report, since it does not involve defects in vendor-supplied services / spare parts.in stock.

4. This problem will not be reported via an INP0 Nuclear Network entry.

5. Reviewed under 10 CFR 50.9 and determined to be not reportable since this event does not have a significant implication for public health and safety or common defense and security.

6. Reviewed under the criteria of AP C-29 requiring the issue and approval of an OE and determined that an OE is not required.

V. Corrective Actions i

A. Immediate Corrective Actions:

1. Replace the leaking expansion joints. Check system piping for leakage following replacement of the expansion joints. Complete PMT by 2/27/93.

l RESPONSIBILITY: Allen /Waltos COMPLETE Mechanical Maintenance (PGMA)

AR A0294740

2. Disassemble and inspect valves FP-2-FCV-98, DG room 2 C0, FCV, and FP-2-FCV-99, DG room 2-2 CO, FCV, and their respective pilot valves evidence of damage or.

degradation.

The valves were determined to requirement replacement, however, inspection established that these valves were exwom mo Page 9 of 17

1 l

~

I i

.DCO-93-SS-N007'

) April'27, 1993 operable prior to this inspection. New replacement valves will be installed.-

RESPONSIBILITY: S. Allen. ~,

RETURN ,

Mechanical Maintenance (PGMA) . .

AR A0291942, AE # 16.(PGMW, for W/0 preparation) j I

B. Investigative Actions: a

1. Request the DCPP chemistry department to analyze'the water sample from .the CO, lines. i RESPONSIBILITY:'D.-Powell . COMPLETE  !

Fire Protection'(PASE)

AR A0292169,:AE # 01-I j 2. Deve'op an inspection plan for the DG'/CSR h'eader, j taking in consideration:

! u

a. Pipe access,-

l

)

b. trench water,

c. elevation,

d. leak check,

e. boroscope inspection and

f. hanger / pipes. water hammer; RESP: R. Klimczak/R. Bruns COMPLETE NECS - OPEG/ Mechanica1LMaintenance AR A0292169, AE # 02 '

3. Research the maintenance history of.00,: lines. . . . '

Discussion in the February 5,--1993 TRG determined that the CO, system, . other than header.. # 1, ~ was last . " puff" tested-in October.1991.

RESPONSIBILITY: D. Powell. COMPLETE Fire Protection-(PASE)

AR A0292169' AE # 03,

4. Perform chemical analyses as. appropriate-by.DCPPL Chemistry and TES on water . sample provided to rule.out sources of. water.

e ne w sms~m . x. Page ' 10 of 17 ,

.+

, - - - - ~ ~ , . , , - rww , , . , ,w, n weg .

n y, en-' eys,y

' l v . DCO-93-SS-N007 April'27, '993 l RESPONSIBILITY:.J.~Knemeyer COMPLETE Chemistry (PGPC)

AR A0292169, AE # 04- ,

5. Perform water leakage investigation based on' the inspection plan developed in AE # 2,'to' include pressure drop test on three inch pipe-(CSR/DG header),

RESPONSIBILITY: R. Waltos -

COMPLETE-Mechanical Mair....iance '(PGMA)

AR A0292169, AE # 05.

6. Perform pressure drop test on two inch pipe:(Ho'se reels) located in the-pipe l trench.

I RESPONSIBILITYi R. Waltos - COMPLETE.

Mechanical Maintenance (PGMA) .

AR A0292169, AE # 06

7. Verify.no water is contained in _the Unit'l DG CO, .

-i system piping. Alsorequested;inAR-A0291942,AE#_.12-RESPONSIBILITY: R. Waltos COMPLETE- t Mechanical Maintenance (PGMA)

AR A0292169, AE.# 07

8. Walk down the Unit 2 CO, system piping with the vendor representative from Chemetron and scope the plan for-water removal from the system'.

RESPONSIBILITY: D. Powell: ' COMPLETE

' Fire Protection _(PASE)

AR A0292169, AE # 08'

9. Provide an analysis to demonstrate (or ' refute) that

" puff" testing of the CO, system: piping and. nozzles was the likely driving force to_cause water. accumulation in the higher elevations of the CO, system piping.

l RESPONSIBILITY: J.' Griffin COMPLETE NECS - Engineering (NCFN)

AR A0292169, AE.# 09'

10. Inspect Unit 2 CO, 3" piping low. point areas. with a -

boroscope.

RESPONSIBILITY: R. Waltos . COMPLETE Mechanical Maintenance (PGMA).

. x w m ss~e n . m Page 11 of 17 f

,, - -l, .n l

, - .,-ww, - ,, -,-w

DCO-93-SS-N007 -

April 27, 1993 AR A0292169, AE # 10

11. Perform inspection and analysis (" slice and. dice") of the removed CO, 3" piping expansion joints.

' RESPONSIBILITY: R. Klimczak ECD: 5/1/93 NECS - Engineering (Piping) (NCFD)

AR A0292169, AE # 11 C. Corrective Actions to Prevent Recurrence:

1. The corrective actions described in NCR DC2-92-TN-N028 (ref. 2) will replace the CO, piping during 1993.

2. Write and perform a monthly special test of the CO, hose reel header to locate any water in-leakage to verify the continuing integrity of the piping until 4

piping replacement is completed in accordance w th ref.

2.

4 .

RESPONSIBILITY. D. Powell ECD: 6/15/93 Fire Protection (PASE)

AR A0292169, AE # 12 Not outage related.

~ Not OE related.

Not an NRC commitment.

1 Not a CMD commitment.

3. Ensure the affected portions of the CO, system piping-are flushed as part of'the replacement of the valves described in immediate corrective' action V.A.2 (above).

Add details of this flush requirement to AR A0291942, AE # 26, including acceptance e-iteria.

RESPONSIBILITY. D. Powell ECD: 5/1/93 Fire Protection (PASE)

AR A0292169, AE # 13 Outage related, 2R5.

Not OE related.

Not an NRC commitment.

Not a CMD commitment.

D.

Prudent Actions (not required for NCR closure)

None.

VI. Additional Information

.w nim.m.s3 Page 12 of 17

.- ... - - - - . ._ . . ~ . - .- . . - . . . . _ - _

)

, Y

< i

'DCO-93-SS-N007-April 27, 1993 i

1 J

A. Failed Components: l l

None.

t-I B. Previous Similar Events:

The potential _for pipe degradation and leakage was . .

l l

previously identified-in-NCR Dr.2-92-TN-N028, "ASW Annubar.  ;

Line/DF0T Line Corrosion," (ref. 2). i i

C. Operating Experience Review:  !

'l. NPRDS:  :

Not applicable.

2. NRC.Information Notices, Bulletins, Generic Letters:

-None.  ;

-j

3. INPO SOERs and SERs:

1 None. >

D. Trend Code: j l YY - D (Other) - (Other, general = cor'rosion).

E. Corrective Action Tracking:  ;

1. The tracking action request is A0292169. ,

1 F. Footnotes and Special~ Comments: .i

.i None.

G.

References:

l l

1. Initiating Action Request A0291942.

2 NCR DC2-92-TN-N028.

H. TRG Meeting Minutes:

1. On January 29, 1993,- the.TRG' convened and' considered the following:

u~c w s.2ss~ m .;: Page 13 o f. 17  :)

l l

-l

8 DCO-93-SS-N007 April 27, 1993

)

l The three inch line running thru the trench is full.

Leakage from the trench is suspected.

j Description of the problem.

1 l Actions plan:

a. identify the leakage source,

b. confirm amount of water in the DG/CSR header,

c. check other four headers,

d. determine the inspection / cleaning repairs requirements,

e. determine the root cause,

f. post-maintenance testing of header piping, 9 description of the problem, Impact,

- Requirements / regulation,

h. develop a plan,

i. DG 2-3 installation,

j. chemical sample of water,

k. bring out vendor for review,

1. boroscope/ inspect.

m. reportability and

n. Consider 2 over 1. Class 2 equipment over class I safety-related equipment.

Description of the system was conducted.

There was a " puff" test on January 14, 1993 on DG l-2.

Nothing abnormal noted.

Mechanical Maintenance and Fire Protection to determine how water got into the CO, lines.

u,,: s.s 3ss~ocr.ac. Page 14 of 17

. i DCO-93-SS-N007 April 27, 1003 The water in the line was under pressure. No water was noted in the trench.

Three investigative actions were established as follows:

a. Fire Protection to request Chemistry department to analyze the water sample from the CO2 lines.

b. NECS/0 PEG tu develop an inspection plan for the DG/CSR header, taking in consideration:

i. pipe access, ii. trench water, iii. elevation, iv. leak check, j

v. boroscope inspection and vi. hanger / pipes water hammer,

c. Fire protection to research the maintenance history of C02 lines.

2. On February 5, 1993, the TRG reconvened in room 533 of the administration building at 9:00 am PST to discuss:

a. locate the source of the leak and the amount of water in CO, header # 1,

b. check the other four CO, aeaders for water intrusion and _,

c. determine the inspection / cleaning requirements for CO, header # 1.

It was agreed that CO, header # 1 has been inoperable since water intrusion existed in that header.

A representative of ASC0A (Chemetron) Fire Systems, a CO, systems vendor (Fred Hildebrandt), was contacted on the telephone and expressed his expert opinion that the source of the water in header # 1 was not water-carryover in the CO, itself but definitely from a source outside the CO, system.

.mm.m m .m. Page 15 of 17

! I

.x

-DCO-93-SS-N007 8 April 27,'1993 Work orders (W/0s), clearances, schedules, blank flange ..

fabrication and gasketing determination will be accomplished without tracking AEs. Testing of the CO, system in the t

immediate future will investigate the extent of the water.

• intrusion, i

The TRG plans to reconvene on or about February 12,'1993 in' the' afternoon to. discuss ~ the preliminary.results of. '

investigations. .

3. On February 12, 1993, the TRG reconvened'in room 302 of the administration building at 3:00 pm PST..to review the status; of current investigative actions and'to ide'tifyn additional I . actions as appropriate. The' intent'of the TRG was to: 1) l l

identify the leak (s), 2) develop a repair' plan and 3)

~

investigate for other leaks in_ the'CO,. system. Actions ~, as' +

l shown herein, were updated.and identified. The extent of l

the water contamination has yet to be' determined. Lcog term : '

repairs will replace all trench piping.(see reference three).

The TRG plans to reconvene on Fr'iday, February 19, 1993'at-2:00 pm PST to discuss the status of-actions underway. *

4. On February 19, 1993, the TRG reconvened in room 316 of the-administration building to discuss"the.. status'of immediate- ,

and investigative actions 1 identified in previous TRGs.

As noted herein a new immediate corrective action and two new investigative actions were identified by the~ TRG.

The TRG plans to reconvene.on or about February 26, 1993 at; 2:00 pm to update the status of ider'ified actions.

5. On February 26, 1993, the TRG reconvened in room 327 of the administration building to discuss the status of immediate and investigative actions identified in previous TRGs.

Updates to' existing actions were discussed!and agreed to as noted herein. No new actions were identified.. The reportability of this event will not be. determined until ,

after the piping expansion joint analysis is completed (investigative action 11. above).

The TRG plans to reconvene on or about March 8, 1993.to.

discuss the status.of ident.ified actions.. ,

n o n.m~m.x. Page .'16 of 17- "

i

, ,,_., , :- i -- - + ^ ~-

DCO-93-SS-N007 April 27, 1993

6. On March 8, 1993, the TRG reconvened in room 604 of the-administration building to discuss the status of identified actions. These actions are updated as noted herein.

Reportability has yet to be determined for this event; the preliminary evaluation of reportability continues to be that this event is not reportable. The TRG plans to reconvene on or about March 23, 1993.

7. On March 23, 1993, at 1:00 pm PST the TRG reconvened in room 327 of the administration building to discuss the status of identified actions. These actions are updated as noted herein. Additional corrective actions were identified as-noted herein.

The TRG discussed various options for confirming the integrity of the two inch CO, system piping until its replacement (as part of ref. 2). Corrective action V.C.2.

is intended to accomplish this.

I The TRG determined that the identified actions are sufficient to address the concerns of this NCR. Closure of these actions is further sufficient to' complete this NCR.

Additional, related actions, e.g., replacement of CO, system piping, will be tracked in other documents, e.g., ref. 2.

No reconvene of this TRG is planned unless events require it. The ECD for this NCR is July 15, 1993.

I. Remarks:

None.

n~:=.nnw.m o Page 17 of 17 i