ML20151C939

Package of Drafts Re Evaluations & Actions Completed Prior to Startup & Plant Evaluations of Single MSIV Slow Closure

ML20151C939
Person / Time
Site:	Perry
Issue date:	11/11/1987
From:	CLEVELAND ELECTRIC ILLUMINATING CO.
To:
Shared Package
ML20151C937	List: ML20151C933 ML20151C939
References
FOIA-88-165 NUDOCS 8807220260
Download: ML20151C939 (53)

Text

~

TOV 11 '87 16: [ P.2

.')

[ Enclosure Q .

The folloving evaluations and actions have been or vill be completed prior to plant startup:

1. As previously stated in FY-CEI/0II-0288 L, for the dual (fast closure) solenoids, the total air pack has been replaced for the 1821-F0280 valvggthe whole dual solanoid has been replaced on kv. An the 1821-F022D 4 valvey No other solenoids shoved significant degradation or requiredireplacement. All of the other M8IV dual solenoids have been rebuilt.

2. As previously stated in PT-CEI/0II-0288 L3 the single (slow closure) solenoid was replaced on the 1821-F028D valve sinca the whole air pack was replaced and on the 1821-F0285 valve due to 5 frayed wire. j Eased on the inspection results above, no other replacement:; vere necessary.

3. As previously stated in FT-CEI/0IE-0288 L, a evaluation has been performed of other ASCO solenoid Class 1E harsh environment applications in the plant, including these which may have been rJbject to the steam leak environment vtich affec .he MSIV solenoids. The review ident

• Nd two normally deenergized solenoids which vere med subject to the same conditions as the SIV solenoi s.

Since the solenoids are in a normally doenergised sta e, y action was considered necessary. Vork history review of all other ', y applications has shown no solenoid fail res indicating the ASCO O solenoid degradation appears to be limited to the NSIV solenoid valves.

erformed of other aquipment in the vicinity

4. An evaluation l of the 1821-F022D,1821-F028D, and 1521-F028B valves, to assess any I impact that the steam leaks may have had on cesponents. It was determined that no heat degradation af fecting equipment operability had occurred.

I l

8807220260 800630 l PDR FOIA MAXWELLOO-165 PDR l

y

, 5. Until the temporary to:perature monitoring baselin val /es have been  !

determined, the existing permanent temperature elen ill be  !

used. The historical readings of the existing permanant steam tunnel and dryvell temperature elements in the vicinity of the MsIva has been reviewed, and a baseline has been determined for each element (see Attachment 1). It has been determined that a 10% rise above these baselines values may be indicative of a localised steam 1 leak and would require investigation. This value is approximately one half of the temperature rise expected for the Technical specification trip value for leak detection.7A procedure vill be established specifying necessary actions to be taken upon exceeding these values. The corrective actions to be taken are as folloves o Reduce pover as necessary to perform a visual inspection to determine the equipment inspected.

I o lamediately repair the leakage or shield the adjacent Class 1E components to limit the impact until a repair is possible.

o Note components being affected and assess the thermal impact (30). Evaluatej / d determine the necessary time frame for I takingaddittofal tion.

o At least 1 temporary temperature element in the area of each MEIV vill be maintained in service in operating conditions 1, 2 and 3. *

6. Additional steam tunnel temporary temperature monitoring has been installed on the preselected sample points M the MSIM ow including the dual and the test solenoid bodies. Baseline data vill be obtained on the tamporary temperature elements in the steam tunnel during the next full operating period of sufficient duration to allow temperatures to stabiliza. Based on experience, this vill be several days af ter the plant is at full power. Inspections vill be perfereed during startup to assure that the initial temperature reading are not being effected by stea'n leaks. Once it has been deternised that the readings have stabilized, the procedure outlined '

in ites (5) above vill be revised to use the tamporary temperature elements in lieu of the permanent elements. I 2

, , . - . - - . - . - - - - . - ..,_. --.,__.... .,_.,,, -. . ~~m.,.-._,...-m.-...~_-,.m- ,

NOV 11 '57 16:49 P,4 1

. . 1 Af ter completion of the startup test progran,at least nine dryvell  !

temporary tesperature elements vill be installed on locations on nad l mesmed the inboard M51Va', typical of what was done with the I tosporary steam tunnel temperature elements. A baseline vill be established after the startup following this outage as described l

above for the temporary steam tunnel temperature elements. These l baseline values will then be incorporated into the program, along with the respective acceptance criteria. I 1

7. A test has been performed which verified that air does not flow L  :

between the air compressor reduction gear vents and the air '

compressor intake. Consequently, it was determi.ned.that g .. d there was no need for any equipment modification, or n ske filter replacement frequency.c 5 Following startugthese additional evaluations and actions vill be performedt j

1. To further substantiate the root cause, the laboratory analysis will be performed to determine the failure mechanisms of the EPDM degradation. A review of industry experiences and discussions with i l

various industry sources vill continue to be conducted in order to l input into our analysis plan. Der preliminary analyses plan which included these industry contacts is completed, and a summary is provided in Attachment 2.

M h6#t. g d #12$ N l>N R b oft h M [ f4pr M et,

• The ialtial industry review did not change our preliminary conclusion that the root cause of the problem was primarily localised elevated temperatures near the ABC0 solenoid valves.

Iovever, we have not eliminated the potential of hydrocarbons having a deleter'ious effect. We plan to use data obtained from other plant exportances as described in IIH 86-57, along vith our ovn analysis, <

to confirm the root cause.

Our pr nary schedule is to have initial results and analyses by ri end of , quarter 1988. Any further analyses required vill be determined at that tin,e. Ve plan to use a local research laboratsey, as our primary analyses contractor. DM MNgE (clkd eigth se gg 0.,g(ys/s girytten3 fossible ign improvementsj vill be evaluated' and a determination vill be ande on future actions, including replacement frequencies.

3

. - - . , . , - - - -.-,- ,,-- ,.e.- .- -----, ,---,r. -

- -- t:',y t#fil 187 %ma

2. Presently, in order to hE8N the pstential for introducing l .

hydrocarbons to the air systcgs prev:ntive maintenance requirc ent l vill be established for periodic replacement of the instrument air l systes pref 11ters. The asintenance frequency vill be consistent vfth replacesant of the instrument air systen after filters.

Additionally a generic precaution vill be added into air systen vork orders regarding the use of thread lubricants and sealants.-

If the outcome of the Chemical Analyses indicates the presence of l hydrocarbons, we vill establish an appropriate hydrocarbon sample and analysis program for the instrument air system. This progran vill be provided to the NRC.

Dev point and particulate sampling of the instrument air systen vill contgue in accordance with the existing plant administrative fp h 7 Any unacceptable results vill be evaluated and systaa blevdowns vill be conducted until satisfactory reruits are obtained.

3. Until the first refueling outagg the fast closure dual solenoids will be checked for proper operation duils the noothly slow closure check. This test vill not be performed duri g o(e months d d fo .

p;"e.Jugkquarterly fast closure test'.'bhis win be performed by fully closing each MSIV individually utilising the test solenoid, followed by taking the control switch to clon. This vill verify the proper operation of the dual solenoid, since the MsIV vill only reaala closed if the dual solenoid deenergisas and properly repositions. If any MSIV should reopen during this test indicating the failure of a dual solenoid, the associated MSIT vill be declared inoperable and the plant vill be in Rot Shutdown within 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> and Cold Shutdown within the following 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. The R C will be notified upon discovery of such a failure.

4

. - - . - - . . . . , . - . - - , - . . - . - , , . - - - - - _ . - - - . . - _ . _ _- - - - - - - . . ~ - - - - - - - . -

~ ~~

t<> 11 '87 16:51 P.6 Also during this time frame the NSIVs vill be cycled individually on a quarterly basis regardless of plant operating conditions, and the f ast closure time verified. Upon failure of this quarterly test due l to a problem with a dual solanoid, the plant vill be shutdown and the NRC vill be notified as described above.

l Prior to exceeding a six month period an inspection vill be l performed on the dual solanoid during experiencing the highest temperature profile during an outage of opportunity. This j inspection vill verify no degradation of the solenoid valve internals. If accelerated heat degradation is observed a complete investigation vill be initiated and the NRC notified.

A review has been completed of all knovn steaa leeks in the plant ,

l which could have affected class 1E equipment. These componenets vill be evaluated to determine if there has been any affect on their qualified life based en the environment under which they were I subj ected. The results of this evaluation vill be completed and subsitted to the NRC by November 30, 1987. 4 1

1 t

I I

l l

i h

(

m u w ms; o e

Attachment 1 TEMPERATURE MONITORING l FOR DETECTION OF ST EAM LEAK 5 l

TEMPERATURE NORMAL OPERATIONAL ACTION PLAN SENSOR IMPLEMENTATION NUM8ER TEMPERATURE:

TE RE D23 X102 A 140'F 154' F UPPER D23 K102 8 140' F 154' F DRYWELL M13 R110 2 150' F 165'F AREA M13 R11016 135'F 148' F D23 K112 A 135'F 148' F D23 K112 B 131' F 144'F MIDDLE M13-R110 3 136'F 150' F DR L M13 R110 4 124' F 136' F M13 R11014 136'F 150' F >

M13-R11015 127'F 140'F D23 K122 A 130' F 143'F D23 K122 8 128'F 141'F LOWER M13 R110 5 114' F 125'F DRYWELL M134110 7 122'F 134'F AREA M13 R110-8 122'F 134'F M13411011 110' F 121' F  !

M13 R11012 127'F 140'F E31 N604 A 125'F 138' F l STEAM E31 N604 8 134'F 147' F TUNNEL E31 N604 C 130'F 143'F MO E31 N604 D 128'F 141' F RS STEAM E31 N605 A 80' F 88' F TUNNEL E31 N605 8 80' F E31 N605 C 88'F DELTA T 82*F 90' F MONITOR 5 E31 N605 D 82' F 90'F 4

9 9

-..,----------,--.-----.----,,_,-.,,-_,,_n_n-,-,

, . . - , - ..,n,,n,...-enn-n,., -..-, n, __.--

' ~ ~ - ' ~ ' - ~

xovif >er 'isigt - -

Attachkont 1.

ANALYSIS PLAN FOR IPDM SCLENOID CONF 0lGNTE r

i I. INTRODUCTION l

To determine the cause for failure of solenoid pilot valves '

which resulted in the slow closing of MSIV's, two appecaches vill be taken. Both approaches involve analyses of the IPDW elastomer gasket material. The physical properties of the elastoretic material 3 l

vhich was in service vill be compared to new material to observe degradation, loss of material, deformation, anomolies in surface characteristics, and reduce & performance. In addition, the gasket material vill be subjected to chemical analyses to discover changes from original saterial at the molecular level. Data obtained from i the analysis regimen along vith data from a similar failure  !

experienced at Brunsvick in 1985 vill be used to determine cause.

1 II. PERSONNEL CONTACTED I

Interviews with the Harris Research Persor.nel and NRR provided information regarding analyses performed and resulting postulations.

PN?? analyses vill include methods to confirm or deny these failure postulates. The full Brunevick Failure Analysis Report has been sent and vill be used as guidance. A meeting with Ricerca, Inc.

Personnel regarding this failure analysis program resulted in the {

i following proposed course of testing. I III. ANALYSIS PROGRAM A. Samples

1. Unused Elastomer Gasket material

2. Used Ilastomer from pilot solenoids which did not fail.

3. Used, degraded Blastomer Materiti from failed pilot solenoids.

4. Pilot solenoid valve bodies with elastomer residue.

1

& 4

--.--,,-.--..-,.,,.,_c,. -,__,-#,.--_ ,, _ ,.,,,_._ , . . _,,y,m__,,,,-_.-.m., -,m_,e.,__y -.-..,.-_.,__,_,_m,,

~ ' '

Nov11'b716:53 P9 Attachment 2. ,

i

3. Physical Testing l

, 1. Profilinetric analysis to compare ledentations in ,

IPDM disco (sample nos. 3, and 2) l

2. Optical Microscopy to determine the presence of foreign satorial, or loss of material from surfaces. i

3. Hardness testing to compare with original specifications. j

4. Con.nrossion set to compare with unused material and note i performance degradation, j

C. Chemical Testing

1. Infrared survey to determine carbonile content. This vill provide information about mode of attack (organic acids from the presence pf hydrocarbons) and extent of oxidation.

2. Scanning Electron Microscopy /X-Ray dispersion Spectrometry to confirm or negate copper-catalysed accelerated oxidation. (Yhich was a postulated Failure Mode at Brunswick)

D. Environmental Testing Six new dual coil solenoids vill be sent to a laboratory for additional environmental testing. The solenoids vill be placed in three separate environmental chambers (two per chamber) at various elevated temperatures in an energised condition. The solenoids vill remain energised for predetermined times in an attempt to determine the temperature and continuously energised time at which the solenoids do not perform their function.

IV. St.MMARY l

The above analyses and their results vill provide evidence of  !

failure mode and vill describe any further confirming analyses which l may be caeded. In additica, recommendations vill be made in order  !

to preclude recurrence. 1 i

t

(

l

-.-n- , - - - , , ,-..- - - - - - - - - , , - - - - - ,, e m,--~ws -e--- - * - ~ - - - - - - - -

4 d.4 PERRY NUCLEAR Port.R PLANT EVALUATIONS OF SINGLE MSIV SLOW 4 ( f/3/Itt' '1 Th j to ked p rfor' seg'an a'aly is to 'alua % he saf ty si ni ica pe f .a ir o .e !m n'st !gois a* in the to er ini 48 4 s, tot.

~

t me B h E'an

/ as llinel all U 11b t v re u tohe} pint s an ysis.

Fint GE determined that two accident scenargosandthreetransientsdescribed 7 in the FSAR t i fWe'los~ure.)f^hi"?I" 21 ' IM fo11owing:

-4(eG(A be hM -b he+ t eb5( 0 M '^

FC-~-- M)W 'f o/*"^#'

~

> o enM n e,/w 1, t M T

1) Steamline break outside containment

2) Inside containment breaks which reach Lsvel 1

3) Pressure regulator f ailure transient

4) Loss of condenser vacuum transient

5) Loss of AC power transient i

&Whht & O # d" U

_ M - :!n;d that the h - Aing event ef th::: u^^"a m Nu Mh d'-"'bd e"' d h i steamline break outside containment, since this event would permit the largest

. amount of activity to reach the site boundcry. Therefore, GE was tasked with determining what the mass flow would be for a main steam line break outside containment given the as found condit; ions that existed on November 3,1987 4

(i.e. threemainsteamlinesisola'thwith'hropertimes,,andtheremainingmain wu  !

steam line isolat in 18 seconds). Enclosure 1 to the results of tMsth GL 1

, analysis. The analysis was done using the GE's SAFE 06 Code, a NRC approved  !

J de which has been previous 1 used by Perry in the ECCS performance analyses l ke , ..' 4 f% M .

Chapter 67j Note that theY mass s b.nl 6 e i release determined by this code were much j hA. -

less than the mass release discussed in FSAR 15.6.4.4 for the main steam line break outside containmen g Thie was due to Ihe conservative assumptions used in the FSAR analysis, eech-es (assuming that level rise time is 1.0 seconds, '

that mixture

% . s Vquality is a constant 7.0%, and that the system pressure remains constant at 1060psigthroughoutMSIVclosure),

9 Crk

Y L Ch h do nd CM-u mcever Qs.M CSM w, i t y Akkk k \%FVJ was decided that two calculstions would be done. The first r;;'.e s 4~~ J S GLU~# A#

t sA.

we the mass release given in the FSAR GISAL page 15.6-10) for the first 5.5 seconds, M t h n using the GE supplied flow data after 5.5 seconds when only one main steamline is open. The second calculation used the GE supp;ied data throughout the event. For each calculation two results were deterained. First the postulated amount of radiation which would be released in the 18 seconds it took forA@.g o 2.t b Mity e w

• uM c'Ir Mr.e to isolate on November 3, and secondly the total time it b  :

would take with one main stea line unisolated before 100FR Part 100 licits '

1/ w a s W I were exceeded. J conservative 'assum @ for these calculation that there would be no,plateout, or hold up time for the release. Enclosure 2 documents the results of these calculations.

To sum =arize the results of En:losure 2, the analyses was perfor=ed using realistic assumptions that no fuel f ailure would occur for r.he events.

Therefore, FSAR Table 15.6-17 values were used for isotopic content of the reactor coolant.

For the calculation using the TSAR mass release the following conclusions were dravn:

EB Iodine dose with 18 se:ond single MSIV closure - 192 Rem EB Iodine dose with 79 se:ctd single MSIV closure - 300 Rem For the calculation using the CE data the following conclusions were travn:

EB lodine dose with 18 second single MSIV closure - 82 Rem EB Iodine dose with 120 second single MSIV closure - 300 Rem As shown above for either calculation the slow closure (18 second) of the D MS1V line on November 3 would not have resulted in a release exceedhg 10CFR100

& /

guidelines. Also, depending g which calculationjused,12 - M a * .:d ;h.i the plant would have had between 79 and 120 seconds to isolate that line u er M Pr4- $thsL <W

. accident conditions prior to exceeding 10CFR100 guidelines. Em 6., th e, th 18 f

/

second slow closure of the D sain steam line penetration has be,en sh:.rn to be within the bounds of accident guidelines.

3 DRIfi il 87 Root CAUSE ANALYSIS EXECUTIVE SUl@!ARY This docu:nent describes the evaluations parformed to deter-mine the cause cf events on October 29 and Ncvember 2,1997 when Perry I Main Steam Isolation Valves (MSIVs) f ailed to fast close on com.and. The most probable' root cause, based on data current-ly available, is failure of an Automatic Switch Company (ASco)

Model 8323 3-way dual solenoid valve. The primary suspected cause is hardening and dimpling of the E P3' rubber disc seat material and other EPDM seals, causing the disc holder assembly to wedge in place when the solenoid was de-energized. Several mechanisms have been proposed that might lead to EPDM degradation, the most probable pof which is a local high temperature environment. ....

.,. + ^ c.

The d:cument is organ zed A~ckhe fodr section. Section 1 describes the most probabl, root cairce , . rand the basis for its selection as such. Sectio!. 2 gives an mverview of how the rcot cause analysts team reachec its concidsions, Secticn 3 describ>ts potential c:.penent failure modes that could Tead,to MSIV failure to close, and finally, Sr.rtion 4 describes specific failures within the Asco Model 832' valve that ,coul lead to the observed conditions, and discusses envirorunental conditions that could lead to the failure. v.

N g.:"

.~.

N t

i 1-1 SECTION 1 MOST PRCSA3LE ROC'" CAUSE __-

. ,n 51 The most probable rcot cause of the observed MS .ildie to close is failure of the Automatic Signal Company 00) 'Model 8323 3-way dual solen:id valve to shift frcm the e r rized tc de-energized position. Within the component, the Ethylene Propylene Liene Monerer (EPDM) rubber disc sest material was found te be defermed. A "dimple" (see figure 1 and 2) was found in the EPDM seat material on the disc holder. This is also indicative of a general hardening and degradation of the rubber seals withir the valve. If ,the disc holder sticks to the orifice the MsIV will n: cl:ss. Delayed. closure is censistent with de-energizing of the solenoid, followed by sticking of the' dis:

holder to the crifice for s:me period of ti me , when the disc i helder breaks leess e ..d allows the air pp ssure :: relieva j through the orifice. Cnce the air pressure is re2ieved, the MSIV  !

will close.

.. 1 Failure of this ccmponent is ,the only failura that is  !

consistent with the ch:trved f ailure. No other single ccmponent tailure will result in a delayed MSIV closure. l l

The EPDM degradatica is most probably caused by exceeding i the temnerature limits of the EPDM material. EPCM was :hesen fc- l this application because of its radiation resistance from c.n l equipnent quajificaticrc:andpoint. It is qualified te a temper ature of 140 F. Perry has experienced bulk drywell and steam I tunnel temperatures which have approached tech spec lin ts during l much of the startup test program. Additional'y, stear 'eaks ha.re.

_ 1 occurred in the vicini y 'c f the affected MSIV solenc.is. W'v_ le-no data exists to actually confirm that' the local t e. .pe rat ur e s have exceeded the capa':ility of the EPDM rubber, a geof correla-tion exists between the location of st;4m leaks and the a f fected valves.

Several cther mechanisms have been postulated for *.he EPDM 6egradation, and sufficient data does not currently exist to absolutely prove or disprove any hypothesis. It is true, howev-er, that the temperatures near the valves have been c1cse to the maximum allowable for EPDM material, and this is the mest likely cause.

\

4-

/

4

. , g IMNh

,.n.w w zs 3a S f bl Sci" M+i siE  :

m ym p(y#_ _'::'_"n - _

~

" - -- n M

-f_,,m;M' r : ..

y 7

1, 5

. - V i Ah  %

y

,[ C

^

G.

y3 __

jf -

. .Y .

. ** d

~ .

.r:-

i 1

i. n

~) -

1 l

, y s .

'R vu ., ..

j

.' ' h.N l

.c _ . m. : ...

4 . _;-

-,. *

• h,5* >t, tir n' ' s.

c.:Euc l;y 7

~

mMg

• ^

.g 2

,.;uq .

-Q Ju n id,. .$:~g.

t- "s* .j -

y , . hr6'.-

.b?i-br.

-. Qn m

Vkv,1 '~

. ,o g 5; . : 1 D u nn -' ~-

't W% 1

, .5$'b.y-

~

J .J , . .

-Q z

t. -

s W m;'w::ir

&f" ' l

.. Q .-y..b .~, -

.k.N ' N n

1 g,,.gg:y'* g- -

c' d, .~'+ d ..w; M -

.: Uhkhh.hh Y& ']> ..: \i. r. sffygy i -

_e 9.Mepgyu

n. nw y -

j M,g ,

i gI5,O N.I .

w ..

i' a

,,,3 dl Pa ,

L .. .;- 4 ap ,.. .a g

} ,

i' .

1

=

- s

.- ~. - no rn.c

< , i E4 ;i $Ui p ., . ,fM .

,, , , f r.i, ,g

@ $$ $ *9kY

T

~~

lM'?80 'h' p)sc H snercysava,xylp,gcp,co,qg_ _ _ ,

eaaud iv cgI;nda Rep.,d m, y a ' ~1 O nggg g ,. n i

~~~~ *

... SKETCH SH0uDWG CROSS.~: SECT /ON OF DISC IN IT.S SEATED fo.sJTION F1pa L

=- 1 l

f M t' I 2-1 lk  :. l l

l SI~IICN 2 ANALYSIS TECHNIQUES AND CVERV EW l

l l

Follcwing the failure of the 321-F022"B" and "D" Main Steam l s:1ation valves, a multi-discipline team was convened with the l charter to determine the most like'y cause cf the problem. This

_ l a :ivity would be useful prior to actuator disassembly and I inspecticn. The team censisted of senior engineers f rom the CEI l mechanical and electric engineering, and CEI technical depart- ,

- s .. s , as well as the architect engineer ( Gil.le rt ) and NSS5 I supplier (General Electric).

problems analysis prcceeded using standard Kepner-Tregoe (K ) Problem Analysis techniques. The initial thrust of the team was to determine which equipment failures would cause the failure of a MSIV to close in the delayed manner cbserved. An initial brainstorming session was held to determine potential component failure which might cause the observed behavior. These potential fsilures were then compared with known facts and design condi-tiens, using "is/is-nct" techniques to rate the postulated failures as to probability.

Twenty four (24) potential failures were initially postulat-ed as to component failures. Cf these , 19 were rated as unlike-ly, one (1) as potential, and four (4) as probable causes.- All five of the potential and highly likely candidates involved either the ASco Model 8323 3-way Eual Solenoid Valve, or the air su,: ply ' to these components. Specific work items and inspection steps were thus incorporated in other site action plans to address these components in detail.

Section 3 of this report docuents each of the 24 original c =ponent failures. It is crga..ized in order of highest to 1cwest probability. Each potential cause is described, discussed and conclusions drawn with regard to root component failure.

1 l

l

=_

DRAFf 1 1

2-2 l 1

Following disassembly of the actuator air packs and diagnos-tic tests on the air supply system, it was determined that the i

most likely f ailure mode was, in fact, the ASCO PAel 8323 3-way dual solenoid valve. The suspected cause was dimpling of the I?DM rubber disc seat material, causino the dis: holder ascambly to wedge in place when the solenoid was de-energized. The team was again ccnvinced, this time to evaluate the e..viror. mental and ,

design conditier.s which might be responsible fer the observed J c:mponent failure.

Analysis te:.niques similar to those utilized in the corp:-

-=-- = valuation were used to screen the p::ential cause:. i N5s31ute determinaticn of the rootscause is dif ficult, . however,,

the most likely ::nditions leading to the f ailure was local hign ,

- e- pe ra tures les: r.g to E?DM degradation. Analysis results art civer. in Se::icn 4, again describing e a c'.7. c f the nine (9) postulated roct cause conditions and discussion cf the evidence to confirm or deny the postulated condition as rcot cause.

i l

l DRAr

.c:r. . . '. O N 3 CO".IONENT FIJ'.'.3E DESCRIPTIONS .

, 4

• I 4 S ,

, .r-.7-.. .e, _ - , , - , , , ,- ,, ,. , . _ , , ~ . . - - , . , . y...,..,--.-~v.v..~.,..-......#e,#-._ m._-,w_.m,.,- ..e-__,,y ..,-, ve.____,,-.. - _ _

l l

l F" ~

3'  !. '

LL L '

i Potential Cause 1 l

Failure of the Part 44 ASCO Medel 8323 3-way Dual Solencif i Valve Discussion 1

Fa'. lure of the ASCO Model 8 323 3-way dual solenoid valve tc s':ft from the energized to de-energized position cou lc'.

cause the delayed closure event experienced by Perry.  !

This f ai'ure mcde has happened in the past due to varicur reascns as evidenced by TE Notices 85-17 and 85-57, and INPC S I.: 57-!!.

I

.s . . . , l conclusicr This failure mcde is the most likely candidate fc:: roc:

cc.T.po nen: failure of the prcblem. The post-disassembly i

nspection has found dimpling of the EPDM rubber disc seat j aaterial. This could cause the dis: holder assembly to i wedge i .- place when the scienoid is de-energized. This would in turn not allow air pressure to relieve thrcugh the 23 air pcrt, and preclude !!SIV closure.

l 4

/

3-2 ,. e-pctential cause_ T' Instrument Air System Quality L i L l (oils, moisture, particulates)

Discussion This potential cause has been experienced at other plants.

This is evidenced by IE Information Notices No. 86-57 and 85-17. -

In the likelihood that poor instrument air quality, such e ;

moisture, particulate, and/or oils been present, the pessi-bility of failure related to several Mainsteam Iscla tic:-

'/a l /s cenpenents would be highly likely. The main cencerns would resolve around the Automatic Switch Company (ASCC)

- solenoid valvest. Since the seal and . discs internal to these valves are Ethylene propylene, any intrusion of oil ir.to the instrument air system could cause degradation. Degradatic of the seals and discs would, in this case be caused by hydrocarbon contamination that would distort them and could result in sticking of the valves.- Although at Perry this is unlikely because of cur "oil f ree air system" . Disassembly and inspection of the ASCO NP8323-20E dual solenoid valve from MSIV F022D did not reveal any hydrocarbon substance which could have been borne frem the instrument air.

Additicnally, a sample of the instrument air shcwed ne signs cf hydrocarbon contamination.

Th'e poss'ibn intrusion' of wa t'e'r or moi >:ture' into the air' system could cause residue to form on the ASCO valve inter-nals and cause sticking cf the valves over a period of time.

The meisture may collect during outage periods and become residue during plant operation when the ambient temperatures are higher. Particulate int.usion gre'ter than the 40 micron allowables would be a major concera since they could plate out on the solenoid valve core and/or base sub-assembly resulting in slow operation of the s:lencid valve. The disassectly and inspection of the ASCO NP8323-20E dual solenoid valve revealed no traces of mois-ture or particulate contamination.

The concerns addressed above also apply to the C. A. Norgren Shuttle valves; however, the solenoid valves are much more susceptible to instrument air quality.

Conclusion This item was initially considered to be a high potential, but following analysis of air samples,, this item was changed to low probability.

3-3 - _ - - -

Poten*.ial cause P F-i >

n.: <

Chstructions/ Foreign Naterials k in Air Lines /Accumulatcrs iN i Discussion T'..is potential cause has been experienced at other plants as e.*idenced oy IE Information Notice 86-57 and 85-17. Ob-structions/ Foreign Materials in the air lines /accumulatcrs is a likely cause since it would permit valve failures cs experienced. The obstructions may permit periodic operaticn of the valves and depepding en the instru: rent air cycling c:uld :e.Tperarily baceme dislodped. This cculd result i-the same characteris:ics discussed in the write-up cr. " :- c c -

Air Quality".- ~ .

c e r.c lu s .'. c r This item was initially considered to have a high potential as rcot compcnent failure. Inspections of the air lines and accumulaters found no defecc that could cause the observed c;eraticnal pattern, however, so this potentia.' cause is ucl- kely to be a root compone..: failure.

I l

p5T l 3-4 (_

Potential Cause I l l

One er both of the pilct solenoid valves av& u .. v. d- 1 MSIVs failed to de-couple (mechanical ac-icn) upon l de-energization.

Discussion Electrical centrol circuits identify positive j de-energi:ation of the respective pilot solenoids. This is  !

verified via the indicating light and any meters as showr l per elementary diagrams per B-208-013 3011 anc F.036. The I testing secuence and visual verification has ide..tified that  !

the sciencids have been de-energized, although the valves failed to cpen or delayed opening. If either solenoids fail's to de-couple then the valves will not cperate .. No.; I method exists to remotely determine whether one or both of the solencids fer a particular valve failed to de-couple. l The mis-operation (erratic) closure or deferred closure may possibly be attributed to this occurrence. As such it may be a highly susceptible cause. Further evaluation identi-fled that each cf the pilct solenoids were sealef with Bisco LOCA Seal at the condui- entry point. This design change implemented per CCp 6.0618 is the cnly change initiated recently. The degradation and/or migratica cf foreign matter could , also be a cause to prevent de-ccu;'ing of thi solenoids. .

Cenclusion This item was initially classified as a high pc ential, and cendition of the Bisco LOCA seal was evaluated u;cn solenoid disassembly. Since no interference with the valve cperation was noted, this cause has been eliminated fren considera -

tion.

i l

. .. . . . - ... w

^m.. r l

&w

. f1 i. .

H; y  ?

' . i 3

j ;1

. j

- -r p e

I u

I _ '

.-0C 4 i

+

,. . l MNYM N

"J '

01

[

I N b -

s .- i 4 l v

A'*$Cf* 'a .

i

.a > 3f = 1

.,2 a-D J'

M I

l

. K C C '

< g si n25 9'

,pa p .p.a > i. >

i.e > 3. .  :: .a. M.

i

=; *J .. . #2 < '.2 =; .

1 O. n.,. ~ .. e . \

AQ>

i O.4=a P Q >.

s. => ^.

+ ,

, l

_. p s# I .2 1 l

,, r . -- .

l

~[ g # ,r 72 p

, } .,, - . . .

a _ _ . , . J .

. . . _ n ,

m +-

~. ,

F ,, ) (

e p._ .

~ , ,

j i ..

N . l  % ..

• • . l et l-

-l . . . . . , .

'= . ,...g i

.... m

• • w... ,. .. . ,-

e

-....~ _ . . l .'..;.i i .

i.- .s.. ... . ... . .. ..... ..r . / ,J f' l 7 r..

. t.

o g ,3 ,

l ..o .

y / un V

g 3Y>

.~.....e.... ,

b .

es

'- - -- D l-.

t

~ .==** -l @ < ..

= - - - -

.. 7, M .

1 M M e

O a* Q s-O-c

= 4 .-

q

.o no - --

(

5 ,. ( o .,

N E .!! -

. uW,E u '

d o.. .

r g.

O "

C '"f"  :! o j (g- .: ,- ' $,

~. .

8 0 e e

, , . , . . ~ , w. m . , . . , , . - , , . . . , . ,. ,, , , , , .

$@ cd l@l lQ!" 3, "j g i a  : x w -

-- 1 U,..-

(c) r- L.s-. .. . _ . _ . _........ .:. _ ... ... . . -

- l

.h. r;:

tu

< .~

)

1 3 c,- (LI '

(D 0~ i - LL

.) '-

t l

I t s vAo :.

n* .' ..- o 0

.x c: '

.*. g Q 7g OM .~?}, :s L.

w t h,,,

o< . _. a u=

o 7. x -

v f ,

am a 1.__ - ,

5 ,:

g  ; . - -

w .- I o e" z l# ' G30<

G B 1.t 5 ??

u=

o-L' S m e:

i a - ,, - O e  !

'10S I

+y c N

a o "

O y

o,<4 7

Sds s.

• O W e -

_- 1- H i

- ONV d

M3

. . r' U.J

._J O' C'd] d E ,

'p- O' w

0] , > (O -> ^<" , t b$ a .

r u h

,30 h' ._ e m m m ., o 1 l

Je

.4 .-e. .

p w

w c: ,

J 7t ew t,

h S: '- r ~2~

r (w w2 f, a,) ,5 >

h <D -- F. o -

N e < -

c2  :: ..

s  !.m

0

.: i,d A TL- j ',/

O c.: v .t O O l

c3 4 .' 0 " , s

• f. =1aN

(

I Cs N .J ' . - , i ch N N . L[:.- 4 .f) O < i > 8 I .-'* 1 O I - r- l N C3 - O. H @. -

a s .; . m~ ~

a i

. = ,=.= a .< n a '\ !. g 3 ]v I 9.. s _M 3- e h.

O (s i 8 - W~

.t l

l C.'3

x .

O

.e.-

.le' V(/J -

e

.c<Wn\

o =

u c3

~~

m ."s uJ a C '

'M

. M ~ + 4 *-

yp .-

u < u - ,') O ,,9 m . t u., m La O R, c i_. O

+ + = .

= ,

- D D W cs - 7' ' s j m o o n -

O u W w >- >- . m o O 1 Cd $ l

- < < = n e -

O , ' _, '

o a a i c3 o u. ,,_, O Il '

3" - L.') c3 c3 d g 1 l.,4 'I c a

x a o o = a x t/ ,4.a utaZu4 .

cm < t - . w 4

r.)

< o cm 3 - _j a e o w pd

.N, 5.

g

• t:

m i; m

=

6 b1 -

s1

" ' JI g a o _d c I II C o ud < c. ')

a=aa uJ G < o '

j J crua F w;y

< c= i o w .-2 o - < u h l e

.j seted,' P l, M W- - u- CD C p- -m u. C CC 14 0 i

.-, us

+ . .

m

- c.3

- n ,o to t -. . ,.

& k,

. . - , - . ~ - , c. ,

1

. - _ . , .. - .. -- ~~ -- ,

~

8 ed t

D S  % '

1q) v' e\ n r / t u o0 2 W /J t i  ! i  !

ul d W t g r \ .z g.. M r F- c 3 4 a:o <o -

2 g Zt I 4 2  % '

b

.. (/

J

<P O

rd f\-

i T'

~: i \

,.d F l jOw .

p .

g  ;

b  : a O g w \

/-

O O O @ N J qp I (o 9, w o N _

oOU

.<,0 y'_ OO. h ef U C'

u U tJ j G - --

N--

$9 b 5 3 N 13 Q ') 3 ' .< < N uo mm <f l'l <f <

. sj _ -.

UJ m( y m -

g4 .d; - - . .

o O N O ,

mm O LG J .O g g F' y '

E. :c'.I I  % wx fC 'o u x m .o

-.y < N -

m1 + 1 mx

-s . z_ :

l I d cff 32 qe N N , $d sco N N 1

l, _~

i -m -m

~

I,,

~ 7

; y f'%-Y'd d j

-- - ,,, * . , , ,.---,am- -----.-,----.,-,--r,.,v- .-,-,.-r *+ g .-w ,

3-5 ~~

Potential Cause Solenoid valve exhaust port blocked. g, jl L  ;

Discussion Blockage of the exhaust port could occur threugh internal or external centamination. The port is open to the ambient.

Particles may fall below the disc preventing shifting of the solenoid valve frem its normally energited t' normally de-energized position. Subsequent actuaticn could blow he blockage cut of the valve ellowing normal operation thereaf-ter. This is considerec a potential cause for the Pe rr:/

delayed MS:'. closure e::perience.

Cen:1usion This was initially considered to be a p:tential cause fer I the Perry delayed MSIV closure experience. Inspection for blockage was performed, and on one solencid a piece of tape was discevered to be blocking one port. Subsecuent testing de:er. Tined that this blockage was insufficient to precluds MSI? actuation.

s

c-3-6 Potential Cause DAFrf{ t Failure of the Part H Norgren Model B0004A 2-way shu ti:

valve.

Discussien The 2-way shuttle valve works in conjunction with the Part

1. 1 4-way shuttle valve to open and close the MSIV. The 4-way shuttle valve provides the primary logic f or pressur-itation and venting of the actuator cylinder. The pote..tial failure mcde description is the same as that fer the 4-way shuttle valve operatien.8 The 2-way shuttle valte cannot by itself open er main:al.-

the actuator in the' open position unless the 4-way valva. is energized er stuck in the energized positien.

Conclusion The delayad cicsure e tent experienced at Perry is unlikely to have been caused by the 2-way valve failure, s ir.:e it requires dual mede failure.

1. g g 9 6 g 4

4

~

3-7 ~~--~~~

potential cause i

)f

. '. i E .

k y

Hydraulic Speed Control Failure  ; i y,,', r, ,

{ g. r ', l a

Discussien The hydraulic cylinder function is tc slow the closing speed:

of the MSIV to specification limits under a wide variation of applied forces.

The closing speed cf the MSIV is accomplished through adjustment of the Monatrol needle ficw control valves parts h6 a..d n7 has shown in t'ne drawing 13560-01-4. hydraulic flow I logi: s che.T a t i c .

Should either or both flow control valve (s) ~become totally -

)

biccked and also all other fluid leak paths (e.g. ring gaps ' '

in pis en) motion would be prevented.

Such a situation is un'likely because:

1. The amount of contamination would need to be so large l tha: it would not disappear after one cycle. 1

2. The hydraulic fluid was installed under clean controlli-conditions. The system. is closed and pressurized, preventing contaminatich from external sou'ces. r

3. Such a failure mcchanism is not supported by histoticti

.. . . , experience. , ,, ,,,- ,

NOTE: The flow control valves are designed.to provide a  !

ilcw path even at the maximum choked condition. 1 i

-i conclusica I Unlikely to be occurring.

{

E 3-8 --

potential cause MSIV internal bindin;. ~j {

i Discussion Poppet binding agcinst the upper body ribs due t: poppet rotation is very unlikely due to poppet concentricity and long length of rib engagement. Binding of the ste. against the packing gland edge is considered extremely unlikely by the valve manufacturer. Potential for the lantern ring to cock and bind to the stem is a possibility with inadequate packing compression butJis also considered unlikely. The packing compression used in the reassembled valves is n :4-mated to he adequate :c prevent' lantern ring movement.

cenclusien The low probability of binding and lack of reported indus':ry cases, ir inconsistent with the multiple valve f ailures or-the timc facter seen in the f ree up of some valves. This is unlikely to be occurring.

'%=  % 8$ ,y *)'. $ g c,-,,nn ., - , , , - - - - ,, ~ , ~ , , , . , en . ..r-.

J l

l l l J.. Psb, .- f 3-9 * - '

I k. f Poten-ial Cause j _.

f /

l Swagelok fittings improper installation / assembly / u:ar.... _

7 1

Discussion Exressive fitting leakage would not cause an irregular cperation of the valve. This type of leakage would induce a censtant operational characteristic- i.e. slow rate of c'..ange .

Likewise if leakage exists accumulator would close ' valve 'in case of leakage en ASCO pilot control valve tubing.

3: '

Condiusion * '

c '.  :> n. ' . l

';.likely to be cccurring.

s' *e - . , t = s' , . . - . .. .. ..,r , , , . , , ,,, ,

. . , . . . . . . . _ . , . . , _ . _ ~ . , _ _ . _ . . - , _ . . , _ . , _ . _ _ - . . . . . . _ _ _ . _ , . _ , . _ , . . _ . _ _ . . . _ . - . , . . . .

3-10 l l'+fg- I e Potential cause h Failure of the Par: 4 5 ASCO Model 8320 ]-way s:' enc:.. . ' W ,,!

Discussion The model 8320 3-way solenoid valve is used to slowly stroke the MSIV (cl se MSIV when energized). When -he solenoid.

valve is energized (opened) pneumatic pressure is routed to the Part #2 3-way air valve. This causes -he 3-way air valve to vent -he rod side of the actuator through a flat cent 1 crifice, while blocking the inlet air f rem air valve ,

Part #1. The gradual loss of pressure fr-- beneath the  !

pisten allcws the actuator springs to slowly cicse the M3IV i (up to 60 seconds).

.. . w . . x. ~- . , . . .

The potential f ailure modes of the valve are:

a. Stuck open (failure to clcse when de-energized)

b. Stuck closed (failure to open when energized) l

c. Stuck partially opened'

d. Catastrcphic failure of val"e body The ef f ects of these f ailure modas are as foll ws: )

a. X stuck open ' valve preve5ts reopening ~cf the' MSIV. '

b. A stuck closed valve prevents operation of the t..:< i i...,...- .- -in'the slow closure mede. This is the normal (nontest) modW of the"~valV5 and'does ne't'affed't"the'-

^

normal closure functions of the othe- i subc:mponents. l

c. A partially cpened valve will tend t close the  ;

MSI; however more slcwly than the ncrmal f ully l opened conditicn. This affect can be visualized in the drawing 13560-01-H schtmatic. The 3-way ,

solenoid valve, partially opened, would bleed inlet j air f rcm the system exhausting it. Additionally it could pressurize the 3-way air valve resulting in further exhausting of both inlet and air pressure.

d. A catastrophic f ailure of the valve b:dy would result in loss of pneumatic pressure resulting in ,

MSIV closure. l l

None of the above f ailure modes support the delayed closure event at Perry.

Conclusion Unlikely to be occurring. ,

3-1; p -..

Fctential Cause ,

Valve packing tco tight.

Discussion Grafoil packing has replaced earlier asbestos packing on=7 of 8 MSIVs. While it is likely that the grafoil packing has greater breakaway f riction due to increased compression of the softer material, the cir:urstances of the events shcuing quick closure after initial release make this semewhac unlikely as the cause.

cenclusion

- - ..,'~f . s. , , . ,, ,

,s, , .

Eecause ether valves with grafcil packing and equal cackin:

ccmpressica requirement shewed no effect during fas.. c'r sic'.;

speed testing and the lack of industry experience of cn Ms:V being held up due to packing, this cause must be censiderei unlikely.- .

~

i 3-12 Potential Cause Failure of :he Norgren Model F0013A 4-way shu::le valve. ~I Discussien The 4-way shuttle valve is energized by the Part #4 3-way dual solen:i:i valve. Upon energization it routes pneumatic pressure :: the rod (bottem)- side of the a :uator cylinder pisten and vents the blind (top) side cf the piston. The resulting pressure dif ferential across the ;istcn force: the red up, cpe..ing the MSIV.

s The 3-way dual sciencid valve when de-anergi:sd, vente (de-ene r gi:e s ) the 4-way shuctie valve, ven-i..g the rod cide and' pressurizing the blind ' side'.

The resulting "pressure-dif f eren-ial acrcsa the piston in conjunction with the springs fc::es the M3:V closed.

~

The Part $2 2-way air valve is provided in the circui't t6 eliminate a single mcde f ailure of the 4-way valve.

The failurt mcde of interest concerns failure of the M3:V to close when the 3-way dual solenoid valve is de-energi:ed.

Should the pressure leg of ,the 4.-way . valve stick, the pressure is still ventei b'1 the Part 43 2-way valve. II'the' exhaus- le; sticks upon de-energi:ation cf the valve, the

. springs al:ne a 9 capable of c,1c. sing the, MSIVs although at. a slower rate.

If either 'eg partially

_ sticks, the inlet pressure ic e.xhausted, promoting closure of the MSIV.

Conclusion The only failure of the 4-way valve which can result in delayed c':sure of the MSIVs as ev.perienced at Perry is sticking ci the pressure leg with a concurrent failure of the Part 63 2-way air valve. This is unlikely as it is as I double mcfe failure -

requiring failure of two separate l subcomponents. Thus this is unlikely to be occurring.

1 I

l l

l l

m m em w ,

%eIL D [:

n l

3'3 f -*r -

Potential cause l

valve line-up of instrument air header syste:..

Discussion Had an impre;er valve line-up in the instru ent air header-system cccurred, numerous other air users throughout the plant would have been affected. Belcw are listed valves and the pessible consequences had they been advertently closed.

1) 1P52-F640 (manual drywell isolation) Improper line-up cf this ralve wouM have prevented repeated actuatier cf :-:1-F022A, B, C, and D. This valve would also isolate the MSR valves as well as the personnel air lock'at 599'-0" Elevati'on'. "'

^ P

)

2) 1P52- rl-7646 (drywell isolation) Hid this valve closed, it wculd have been indicated by status lig..t c on both H13-P601 and H13-P870 panels in the control room. ERIS points EC-007 'a nd' 008 would ha've alscl indicated closed.

3) 1 P 5 2 -M ~/-F 2 0 0 (containment isolation) (A) Had this va l>/ 2 heen closed the entire air supply into contain-ment wculd have been isolated which in turn would have

. affec.ted instrument air. supply to all the air users off of the' air distribution'~ manifolds PS2-J600', ~ 601' , 602, 603, 6~;, 605, 606, 607, 608, 609, 610, 611, and 612.

.. ,'- .. '(B.). Also had this valve been closed it would have been indicated by status lights 'on both t?.e ' Hif-P601' 'an6 ' '

R13-P87: panels in the control room.

4) Manuti talves PS2-F554 and F605 - Had these valves bee.

closed -hey would have isolated a large number of the air users throughout the containment. 3 With all of the discussion above the fact remains that the valves did cperate as observed. This would not have been the cause since the valves would not have repeatedly func- I tiened. l l

l Conclusion i Unlikely to be occurring. )

i 3-14 b 20 BA i

E Pctential Cause -

Air pack wiring and termination f ailure resulting in t hc:

short.

Discussion

'The air pack units are self contained for each solenoid and wired to a common junction box. This wiring and associatef hardware is provided by the .Tanuf acturer. The field wirin7 is terminated at the respective solenoid valve junctic..

boxes. Refer to drawings D-209-013 Sheets 2 through 9 for each of the MSIV assemblies.

per review of the in:er:cnnection wiring diagrams and corre'spendin'g elementary' schematics, the wiring'and t'ermint- '

tien infermation is correct.

The control schematic .for operation of. the. respective solenoids is "fail safe" by design basis, which requires the solenoid coil to be energized ' to prevent an isclation.

De-energi:ation would result in closure of the valve.

The wiring to cach valve is classified as Class lE. Al-though the,120VAC pcwer to each of the A & B pilot solenoid valves pairs if centinued in a common cable', each fcenductcf is properly sized and meets the separation requirements' .

The. cables.are rated for 6C0 volt. insulation, besides,havin; _ _, , .

minimum current draw. Therefore, the potential. for a he short is improbable.

References D-209-013 Sheets 2 through 9.

Conclusion Unlikely that wiring or het short is a potential cause.

(

p .

3-15 I I .v

-Potential Cause Glazed contacts on control and relay ccmdonents cW'M high resistance which would result in cuscontinuity and potential mis-operation of the MSIV circuitry.

Discussion contact integrity and circuit continuity of the respective solenoid valve coils is constantly monitored by measuring the coil circuit current, in addition to an indicating light (white) which relies on actuating contact integrity t; remain energized. Refer to attached partial of drawing 3-209-013, Shee: 10 and Sheet 11.

' The isolation "cont'rol' circuit ( s ) 'are< 'a' "f ail saf e" 'desigri,' -

which requires the solencid coil to be energized to prever :

ar. isoistica. If contact glazing had cccurred resulting ir

.a disconti.nuity (high .resis t,ance .at connecticn.or contac; ,, ,

pcints) ~ in the control' circult(s), the resulting effects

~

would cause the lack ' of voltage to the coil (s). This car.dition due to the "fail safe" design basis, would cause an undesirchle isciation (closure cf the MS:V valves) rather than a. failure.to isolate. .

References D-208, Sheets H05, E10, H11'a'nd i!f6'."'

Conclusion Evidence cf repetitive tasks to cycle these valves clon; with the proper configuration for power and control indica-tion does not suggest any potential failure. Also, the control circuitry and electrical components for each of the inboard and outboard MSIVs is identical. In that there is no past or present evidence to support this cause scencrio, it is highly unlikely that this is the root cause of the problem.

s.

~

3-16 Potential cause Relay failure or incorrect operation % :esultine i- i mis-operation of the MSIV valves.

Discussien The associated control and relay ccmponents are located in the PGCC which is designated as a non-harsh environment and is also seismically designed. Furthermore this area is centrclied for relative humidity and temperature. Thi likelihood of a failure or incorrecc operation due te ccm.:cnent failure is nighly impractical in that this failure log-would h2.ve to occur en three (3) different MSIV ic/ control circuits. The . proper operation and . closure of '

these valves and' repetiEive tiesting' positively indicates 9'-

thac relay failure is not the cause. Also, as shewn through testin; and verification the centrcl functions and indic a. -

tien.was correct. , ,

s conclusien Unlikely and highly ' impractical that relay failurs is i potential cause.

% w 0 0 ,

..._p%

i l

, , . _ _ - = - n.,

l 3-17 -

Potential Cause panel centrol switch failure or mis-operati on.

Discussion The control switches nos.- SlA-D and S2A-D are General.

Electric type CR2940, 3 position maintained' contact. All of which are located in the PGCC. The contrcl schematics as shown per drawing B-208-013 Sheet 10 (inboard)~and B-208-013 .

Sheet 11 (outboard) are-identical. No test data or evidence has been identified to suggest a failure of the switches.

Repetitive testing has demonstrated the proper operation of each cf these control switches.

s. . .- . .. . .

. . p: . . .

Refe'rences .

,, ., 3-2_08-013 Sheet H04, H10,.,and.Hil, . , , , . ,

Conclusien Evidence of repeated acceptable testing to cycle there -

valves does not suggest any potential failure. As such'1:

is highly unlikely that this is a potential root cause cf the problem.

n. - . .. ,. . . . . .

. > . r:. g 7-

. . , . .,. .. . . . - .n ..,,.s.. ,,. . , , . , , . . , . , .,

I l

l 1

j

3-18 - -

Potential cause ] '

Limit switch settings incorrect or inoperabl 3.

Discussion The limit switches (total of 6 each) for each of the MSIV inboard and outboard valves are NAMCO type as furnished by Atweed & Merrill company. These limit switches are not an active component in the control scheme which initiates opening or closure of the respective MSIV valves, rather they monitor and provide lccal indication in the control room for valve position. Refer tc elementary drawing-c B-203-013 Sheets H10, H;#, and H36.

INnit' ' switch 's'ettings' is '

~

.I.nacc~dra,. .te Trie ' 1 otsnti'al' for 5

possible, but c:her independent sources can verify and provide indication for closure or opening of the valves vic

. instantaneous steam flow. . and steam line , pre'ssure ..

Agai., t ..

this issue would not impact the actual operation of the valves.

F.eferences B-208-013 Sheets H10, H11, and H35.

• s ?, p .* s a e o.j.....~. , * .t.< .' e'. ~ - n ., . *-

conclusion In that the limit switches are not part of the.. cont ro:-

circuits, mis-operation would not affect valve closure.

\

1 I

l 3-19 5 Potential Cause- .f .N 4

t t

Mis-wirinc for indication of instrumen.ation or T E ..e

~

. I 4__

Discussion This potential cause wa.- r cently a problem wherein the "A-and "B" solenoid valve s were wired to a common Reactor protection System (RPS) bus. The basis of the design requires that each of the trip solenoids A and 3 for each ci the I.:SIVs be wired to different RFS buses. This issue wu corrected via the preparation and issue of Design change Fackage (DCF) 870414. As part cf this design package and c prerequisite for start-np, each of the MSIVs were verifiec and tested for applicable power sources and func:icnal ope rhtions' . ' Tho 'r robability' of additional'uiring er ors' if' -

.t highly unlikely in that repetitive testing of these valv6:

did not indicate mis-operation.

v- ., s -

, .. - , , , < . . a

References B-203-Cl3 Sheets H05, H10, Hil, and H36.

Conclusion

. .... Although . .this item .is a , problem previ.ously,- 1.t.,is highly ..

unlikely that a sim'isr type of . problem could be the root cause. The efforts .c resolve this RFS problem, retestin;

. .. - and . managementa expoeme. significantly.. rule, out, this pote.v ...,.. .

tial cause. Also, re ent testing of the specific valves in question indicate then the instrumentation and switches are correct.

. ._t -

3-20 *~"^"

Potential cause hbi 1 Data acquisition failure. . -

(

Discussion Failure in the data acquisition and recording system could

..ead to improper assumptions on closing speed being dre.wn.

Valve speed data is taken and recorded using the TR.'. subsys-tem of ERIS. - This system has the capability to' sample data from a wide variety of signals for later analysis. Data en-reactor power, steam fitw, reactor pressure, limi: switch

. , ~, "~

. ~ , . ~.,

, position, and solenoid current are all consistent.. Measure-dents exterior to" ERIS, 'm' aid 'condol" panel" Mnd. back ' panel

4' indicating lights, f or e:: ample, cre also censiste.nt with the ERis datz.. In . sun. mary; multiple ' concurrent f ailure's .taces-

..s - .. .

sary for this.sce'nario to. occur.make.it.;incrediole. . -< ,. .

Conclusion  ;

Highly unlikely to be occurring. . .

,t- - ,. .;:. .. .

- , . . t 6 . , g ,_ 4* *,.

S, ). q , , , . 'g g o

4 1

i I

l 1

l l

1 l

1 I

l "r ' -) ,

't

• 49 ** - g ~

e- -- ---..c-, , ,p 4 y,_,-, ,.p. ,m , , , . ,

.m.-

g h V e

Potential Cause 3-21 Pl!;I%g r b

_?

procedural error for testing. Most previous b Vt~ speed MS.~./

closures have been performed using SVI B21-T2001. The first failure was noted while performing the test per STI-B21-025A section 8.3 and the remaining failures were noted while performing the MSIV strokes using the system operating instruction (S.O.I.)

Discussien Althcugh most previcus tests have been performed usirg the SVI, thic is no the first time that an STI has been per-fermed. As early as 10912/36 an ST!-321-025A secticn i..'

Additionally, the ucs

' " ." was, of thsusedSOI to. fha's ast_ ~been" stroke. demonstrated'

' .the valves.befor'e' 'and after

• the failures. During the E21-F022D, 321-F0283, and. 321-F02??

failure on 10'/29/87 and the S21-F022D and B21-F028D failure

..on - lL1/ 3./ 8 7.. the - SOI was. .used, . however ,thi.s ,is..,the., same 50I .

that was used for the remaining valves which passed their stroke time.

_ Conclusion It is highly un.1.ikely that there is a procedure : roble.:. , .  :

i ,e g* j e*,9gt

• F. f e,% 'g *

• f y , ,,, , ,

e e g. * , ,

e 9

Potential cause High Steam Flow /High Reactor Power Interaction. All previ-ous low and high speed MSIV closure tests have been per-formed at low to medium reactor power. The potential exists that the higher steam flows associated with high reactor power could interfere with MSIV closure.

Discussion Althcugh all previous tests have been run at lw pcuer, th-valve design basis is closure at full ficw, and the capabil-ity of the valve to cicse under full power conditions has been demonstrated r.nerous times at nTnerous cperating 3'dRs .

The . valves that shcwed delayed closure . are identical in

~

~ 'desi~gn to' valve's ' that closed 'within' specifications , - and ' the-affected valves closed successf"lly follcwing cycling. The valve design is such that pressure drop ass:ciated steam flow.will. actually..a.ssis.t in.' closing the valve. ,

_Ocn. :lusier It is highly unlikely that .this_ is the. cause .of the problem.

.,. c. ., ,,

• '- e *, ...s . g .. e .- , 3 . , , , , , ,

+ .

' *\ . . , e 9

$ ~

g penP 3-23 pa . t b I Potential Cause 2

Incerrect reassembly and installatien of tu - 'r  ; n' Th[

air packs were all removed, however not disassembled, during the September 22, 1987 forced MSIV outage. The purpose for removing all of the air packs was to allow for temporary air supply to be installed and allow local stroking of the MSIV to check stroke measurements.

Discussien During the September 1987 outage all air packs were removed from the MS:Vs to facilitate local streking of each valve te set the strcke length. ; Af ter final reinstallatitn c2 the

.ai,r. packs there were several fast and slow strokes per-Tdese st'rokes' ' were" perf orrhe'd using SVI C71'-200 3 9 f o rmi. d '.

and s'/: I:1-T:001. Sven though SVI C71-T0039 (slew strcke testing) does not test the same valves as SVI B21T2001 (fast'

, , . . - s troke.., testing ) ' . the , same. si: . pack is used and the, ma. ting

' surf ace between the' air pack and' actua' tor remai'ns"the"s~ame as does all hose ~ connections.

Conclusion .

It is highly unlikely that this is the problem duc to thc number of .strc' ras perf cr:r.ed af ter reassembly.

.s.. . . set . . . . . . . . , , , , .. . n .. s.: , . , , ,, . , , , , , , , , , , ,, ,. , , , , , , . ,, , . , , , , , , , , . , . .

I 20tential cause Actuator binding / stem binding I Discussion Sinding of the actuator internals for both the hydraulic and pneumatic assemblies is highly unlikely. Neither assembly is subject to external loads to cause stem bending. The hydraulics are not subject to external particulate contami-nation and centamination within the main air cylinder me.;

~

score the cylinder but could not likely stop the movement by resisting the air pressu-a #^--e.

, 1,

'. . 4. - ., r - * ,

,. y. . _ , . . .. ,

, conclusion _ '

This cause would likely have shewn up during prior history

'. '- of"stroking the valves and would. not likely apply to multi- .

ple. valves at one time. Nor would such-binding.likely apply to the top of stroke only. Thus this cause is estimated to be highly improbable.

a .,  :  ;..... , , , , ,,,,,,.;,,

..o.-

. c, . .,. . c.. c . . ...s

. ,, s. s _, , ,, ,,

. . . . , . , - . s ., . . , .. .. ... ,

. , .; , . ,, , ,, ,; , ,,, , , , , , g .; .

l T

J ~

i l

l am r-R I  !

DP. n 1

S..~eeres, w a

• vii 4 2

~ *- . J.,  ; - , ..

FAILURE R.OGT CAUSE DESCRIPTIOlS -

<s . , .. . , . . . . . . .. . . . .

4

w. i ..r:..... ,. : . .; ,, ,; . . ; .; , _ , ..

= ; .* .,=* , < s,,#4 e e *,7 ,.4 , . , * , gs*

• ' ' " ' ' .". = 7g,k . * ,s . .g , , , ,g ,,g,, .i, , , , ,_

s .

1 1

1 1

l i

i e

i e

pe ce L

5 L

i l c.[. 'L h 4-1 l-Potential Cause:

High Temperature Discussion:

Localized high temperature conditions existed during the plant cycle due to reported steam leakage and eleveted area temperature indications. Steam leakage is known to have occurred in MSIV 1B21-F022B packing and the MSIV leakage centrol system isolation valves. This leakage vas in the direct vicinity of the MSIV's affected by slow closure. Steam in excess of 300 degrees F is suspected of being, directed toward the subject MSIV air packs.

The observed hardened. dimples. on the disc holder assembly and core assembly ,

h'ardened 'elastcmer seals in t'he"dual soleno'id valves is'e6nsistent with high-temperature, conditions.. ,0ther evidence of steam conditions include degradation of 't'he' solenoid' valve' 0-ring's and observed rust / moisture' . ^

.. discoloration;.of. the 1B21-F022D splenoid' coil, implying a s. team envitonment. . .

Conclusion:

The high temperature is considered to be highly probable in the vicinity of th.e 1B21-F,022D, IB.21,-F028B, and 1B21-F028D MSIVs. ,

. . ., . v .... ..

. . .>, :p i . .c.s . ., . .ys. . . .. . . . , . , . . , , .,;.,,

. . .. , - :e . .

s 4

A .

pasp P.

Potentia 1 Oause 4-2 D"

iI L Blockage of the dual solenoid valve exhaust port with tape.

i Discussion During the previous MSIV refurbishment where the air packs were removed, duct tape was used to cover exposed ports including the solenoid valve exhaust port. On F028D the-exhaust port tape had apparently not been rem:ved following the refurbishment. Blockage of the solenoid valve exhaust port could delay the closure of the MSIV.

However, the strength 6f the tape adhesi.ve is considered

, weak compared. to .the p,neunatic pressure .f.orces. . Typically the tape will bl6w '6utwsrd re'maining connected en ene side during de-ener.gization an3. .f al], back i . place.1.ike a flap. ,

Further ~ tests ofl"the F.02'80 Valve' has verified the tape i's-

.a . . . not an .ef f ective, block. .; .. .

.. ,,c .,- . .,.s..,;-..

, ?I '

Conclusion

,Very unlikely.to.be occurring.

~

, (* . .

.s s, w, .. . , c. :e -

c .. ; .~; , .:;p,. , . e, / . :. . p. , . . .

. ,.+ . , . . ; .. ,

s . ,; ._ , , ,. , , ,y. , , , , _ ,

c..-...

. ....s. .. .. .....,_......,,,,

8 Y

I i 4-3 D1 I

f l Potential cause i.

6 l l

Jamming of kinematic components.

i i

Discussion In order for the valve to shift to the de-energized condi-tion, both solenoid movable cores must slide within their i 1

guides. The disc holder assembly is also a guided component which must shift fc_ the valve to operata 1 Failure of the ccmponents to shif t may be caused by fcreign material contamination of the sliding surfaces, either particulate er fluid (adhesive in natQre), er by physice.'

. damage,to the,va],ve parts.

l l Ixamination of the . 7 0 2,2D vplve, and.the air ,, surely, . yst er..

has not identified any unusual substances er damage wh'ich '

could expl,airt .the jp.IV delayed closure 2 cond.ition. . Consi. der.

l ing the proportion of ' valves which demonstrated the delayed l closure (3 of 8), a n' extramely dirty system would be expected for this effect.

1

- Cer.clus!en .

Unlikely to be occurring. j

. . ...2., .- . -

. . u , c, - , e.,

. c. s. n ,.. g 3. , , . g. , . . , . , , , , ,

,, . ; s , , ,. ,

l l

l l

l

,. e- . . .

4-4 w Potential cause oxidation of EPDM rubber compound used in ga.sk e . ", I '-

_lh u.

disc seal materials.

Discussion Oxidation of EPDM rubber in the prescnce of a brass catalyst suggested as cause for a similar incident at has been Brunswick-2. This has been documented in II Bulletins 85-17 and 80-11, and in INPO Significant Event. Report 57-85.

Review of SER 57-85 indicates that although catalytic cxidation is a pctential cause for the Brunswick situation, that utility was never able to determine the exact cause tc.:

EPDM degradation. There ,i.s,,however, a relatively large.

, .-: . d'at'a ba'se for 'tise df"'EPDIO irlastomer in' brass' valve ' bodies **

. .v. .. .. '

with acceptable results. .

.. .. s ....,,.. .

.. . . .s . .. .. . ,, . , . . , , ... ,

.. ,. ,,.e, . ,, . ,,

Conclu..io. s n Catalytic oxidation of EPDM in the preseni:e of brass cannot be ccr.pletely ruled out as the root cause for pilot valve c failure. While postulated as .a failure. mechanism,..its validity has not been. proven. if catalytic oxidation doea play a part, it is most likely as a contributing f ae::r, i. -

the high ter.perature scenario, for exar.ple. -

ne. ...w. 2,,c , ,, g 3. .. , . y n .. c.;.. w.e u

..,., ,.,;p,.g . . ._c , ;. . .,a ws . . .a 4 . a .. . 9. .. -

. m. . .. . . .

t

~ ~ " ~ ~

4-5 -

potential Cause ,

Residual' magnetism f olic'. ling coil de-energi::at. .o L LE ll L Discussion sufficient residual magnetism of the ferritic steel materi-als in the region of the coil could cause the valve to remain open following de-energization.

The probability of this ir considered low. The valvs dee:

not shew repeatable effects which would be expe::sd f:c:

resi. dual magnetism.

2 i Al w similar experience.. *'

-- ~, . .

g,so,g,.

, h. i,l..e

,, f6und theorei:icallv siseQherk.possible,. no*

• Any ~ res'idu'a'1 mainet'ic

• ' force's '

would be ice c:mpared to the closure lo;ce unless additi:ns.1

magnetic mass was'added to'the coil vicini'ty.- -

.- - ,,. . . a. .< :.

. . ; ... ~; . .... - .. .;

Conclusion" -

Unlikely to be occurring.

t

, .z .

. . . , . . . . . 3 . n .. .

ei.. .a...-......:,,,  ;..y ...< w.y ,,....,..,w ,,,.-:.....yvie...:;.; .

. - ,,, m ,c....

a... .. - t . 3;. c. p. ,. g .

,a . .. , , . < . . . . . * .s. s

. . , . .. ./. .

. . . .+

E l

i i

l r I 1 .

4-6 ~' 7 Potential Cause  ? M

. t Wrong materials. 1.

L 1 e

i __

Discussion This failure root cause description considers the use cf wrong materials for the disc holder elastomer seal. The potential for wrong lubricant is considered separately.

Dimpling of the disc holder seal in the dua' sciencid valve is postulated to result in wedging of the seal in .the enhaus: to cylinder per:s. The use of a wr:ng materit2 could result in the obaerved dimpling. The proper dis:

'is. an ASCO proprietari/ EFDM, , utilized in . : heir

. . nucidar materia.1,faali'fi'ed

'i ' Valves'. 'Materi'sll problems 'may ~inelsdfr the" ~ ^ ' .' -"

• felle.iing: .

. . . .q . 7.~.. . yr,o,ng ... paterial. . .cf ., lower ls'tre ng.th or . hsr: al . . , . , ,

. capability.

Improperly cured EPDM.

s..-

Improperly..f armulated EpDM.- .. .. ..- .. s An analysis of the di! material may be perfermed to i(enti-

~

fy the m'atfial or ' f: emulati6ti,c. hewaver'/ it is.- uniikeOf ty

-ce : . p #:,,~. i.crdetermine,.the.. rela.Mw , curd..of thq..ggmpounds ,,, ..u,

...,..n.,c,,,,,,,.s Conclusion .

Moderate potential.

1 i

1 1

i l

i

~

f.

l 4-7 .

Pag- f l m l potential Cause ,

~

I al .. J .F LOCA seal vapors L-9 Discussion In order to seal the solenoid housings on the solenoid i valves a LOCA seal is poured in the opening and allowed to cure. The cure reaction reportedly produces hydrogen gas Hydrogen is a reducing agent which might result in softening cf some elastcmer materials under certain conditions.

Hydrogan's effect on IPDM is probably unlikely due to its hyd:ccarbon (Ithylene Prepylene) structure.

hydrogen . be.ing very . light would _ tend z A;iditionally, f rom its p'oint i ~ of Lapp 1'i~catio;n , ' which"is',~several 'inche,o. s a4ay~

rise , , . .,,

f rom .t. h. e val. ve bcdy . ,

. ., . . , .. s . .. . . . . . . .

. .. , . .. . , . . . . . ..... .. . . ,r. - . . . .x ,.-

... ..>... . .  :. c.

Conclusion .

Unlikely to be cccurring.

..s..- , .. , , .

,.3 ,. ,,

. ,.7,... ,, ;

..: ....+... ..8.-~ . s ,. . b .: + . . . :. . . . . :. : .a _

.,..a,.._

. . ;. ,. x ;-...... . . :., y , . ...~....

. .. . .s , .

t l

l I

l l

1 I

I l

Potential cause o-ring / lubricant interacticn _

Discussien During the disassembly and inspection of the ASCO dual solenoid valves, the three body gasket.- (o-rings) were found to be significantly degraded. Degradation included harden-ing, flattening and adherence to the mating valve body.

The cbserved condition of the gaskets is consistent with the effects of an improper, lubricant. The EPDM gaskets are susceptible to hydroca 5cn oils. Normally a silicons Oil e . - (Dowi Corning ' 55.0 ), is used~ as.. a .g,aske.t . lubricant,., . EPDM is . '.

compatible with silichne ' fluids. '

The degradation of tp.e gaske'ts c'ci.1Id ' not' ~ af'f e'ct' ~the 'va'ide ' '

- itself, ~as they are . located .away f rom- the moving.compcnents .. .

However,.. vapors' Irom the. lubricant (no . signs of. fluid migration were observed) could result.in softening of the disc pads resulting in the dimple effect suspected as being

t. h. e ph. ysi,...cal cau.se.. of.. sticking .

Other potential causes of the observed ga'sket degradation

.o-ring materiti (silicene "

. rubber are . high..is. heat

' a f fe'c.cenditi, t'ed s cts iir.i l,.i wrong"by- 1siliconei ' fidiifs1 'aFE

rl'

- c. b'rkss/EPDW*interactionP A /^4 " M9f ,' . ". .h . , n .. W -

. . o. ... . .

Cenclusion Possible. The o-rings should be investigated for p:cpe.::

material and lubricant.

I d _ .