05000302/LER-1989-001, :on 890105,discovered Solenoid Operated 3-way Air Valves Undersized W/Respect to Max Operating Pressure Differential Rating.Caused by Installation of Improperly Rated Solenoid Valves.Solenoid Valves Replaced
| ML20244B721 | |
| Person / Time | |
|---|---|
| Site: | Crystal River  |
| Issue date: | 06/07/1989 |
| From: | Moffatt L, Widell R FLORIDA POWER CORP. |
| To: | NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM) |
| References | |
| LER-89-001, LER-89-1, TF0689-04, TF689-4, NUDOCS 8906130240 | |
| Download: ML20244B721 (10) | |
text
.___-_-
m.
Florida Power C ORPOR ATION June 7, 1989 3F0689-04 U. S. Nuclear Regulatory N i m ion Attention: W = nt Control Desk Washington, D. C. 20555 Subject: Crystal River Unit 3 Ibcket No. 50-302 Operating License No. DPR-72 Licensee Event Report No. IER 89-01-02
Dear Sir:
Drlemd is Licensee Event Report (IER) 89-01-02 which is submitted in accordance with 10 CPR 50.73.
Should there be any questions, please contact this office.
Yours very trul',
j.
A Rolf C. Widell Director, Nuclear Operations Site Support WIR: mag Enclosure xc: Regional Administrator, PsJ on II i
Senior Resident Inspector bCF h$0h002 PDC POST OFFIGt ovX 219
- CRYSTAL RIVER, FLORIDA 326294219 + (904) 563-2943 k
A Florida Progress Company
9 m 388 U.S. NUCLEAA EtrWLATORY CORAsik APPROVED Dese suo 3130 0104 UCENSEE EVENT REPORT (LER)
' ""' 8 8"'
4 FAC6LfTV NAA8E til DOCEST IeWilWOR L36 NW CRYSTAL RIVER UNIT 3 o is l o t o i e l 3: 012 i lOFl 019 m
- Unknown Cause Leads to Installation of. Improperly Sized Three-Way Solenoid Air Valves and Results in Plant Being Outside Design Basis EvtNT DATE FBI LER NUMBER 10s AEPORT DATE 17)
OTH8R F ACILffets tervotvec ts)
MONPM OAV vtan vtAR
- $$$ L*'
7j[
MONTH DAY TEAR F AClketv seAMte DOCEST NUMSSAqsp N/A o i s i c i e g o.,, i l
0 l1 0l 5 8
9 8l 9 0l 0 l1 0l2 0l 6 0l7 8l9 N/A o,sio,o,o,,,
~
~
TMis mEPOnt a sueMitTED PunsuaNT To int asOuineMeNTs Or 10 Con 5. scana eae e, e, ra e,W itu Moot
- 5 30 0 m 30 i.)
.0.n.H HN) j 73.7 w (106 01010 20 4054sH1Hil 90.asieH1) 30.736sH3Het 73,714el n
30.ge4.HiHe e0.mi.Ha>
e0.naH:H.m 30.405(aHiHim to 73+nH2HI) to 75teH2He HHA) ggu-ggy.
JW8Af 20 4064eH1H6vi
{
30.731eH2He g
so.734eH3HvHiHB) 20.eWeeHtHvl I 90.73 eH2Him to.734eH2Hal LICEluSEE CONTACT FOR Ynis LEn Hal NAME TELEPMONE NuMSEM L. W. MOFFATT, NUCLEAR SAFETY SUPERVISOR 90 i 4 7 9:5 1 6:48 6 1
i 1
i i COMPLETE ONF LINE FOR EACM CO88PONENT F AILURE DESCRISED IN THIS REPOiq Hal
"($C' "foqpna omT p
La j
CAU$E tv8 TEM COMP 0NENT
"(hC-
CAUSE
system COMPONENT s
g I I I I I I i
1 i f I i i I
I I I i l l I
I l i I I I sue.uMENTat at.on? Ex,ECTEo ci.,
6NT.
oAv vta.
SUOMi$$80N VEB (I, v, com,ogne fMP,CTED $U8h,lS$10N DA T(1 NO A T a AC T m,~,,. un -.,....,,..,m..
,,, n.,
On Ja'1uary 5, 1989 Crystal River Unit 3 was In the midst of a maintenance outage with the phnt in WDE 5, CDID SEIDOWN. On this date it was determined two solenoid operated 3-way air valves (IGV-130-SV and MSV-148-SV) were undersized with r=wt to their Mwi== Operatiry Pressure Differential (mPD) rating.
Solenoid valves MV-130-SV and MSV-148-SV were replaced with properly sized valves prior to the plant enterirg EDE 4, HOT SHLTTDOWN.
A review for similar design hwnt deficiencies was initiated ard on January 11, 1989 similar deficiencies were found in the dmwntation for valves, GV-6-SV and GV-7-SV.
GV-6 and GV-7, were de-activated ard held in the closed position in accordance with the Action Statement of Limitirg condition for Operation. Solenoid valves CAV-6-SV and GV-7-SV were replaced by a model with the proper EPD rating and the applicable 'Ibchnical Specification Action Statement was exited.
A field verification was ocmpleted on March 13, 1989 for all safety related solenoid valves.
One additional solenoid valve was fourd to be urdersized and was replaced with an appropriate valve by May 9, 1989.
",N" "'
am 3s&A u 3 NUCLEAR REOULATORY -
UCENSEE EVENT REPORT (LER) TEXT CONTINUATION Aaaaovio ous No sino-o'c4 s xPengs 8.3145 F ACILITY hAME (U DOCKET NUMBER Gl LEtt NUMBER iS1
' AGE III CRYSTAL RIVER UNIT 3 o p jo jo jo l3l0l 2 8[9 0l0 l1 0 l2 0l2 0 l9 or ren nr
.m..
a.
m,c,a,, usa v av svar msm1Prlm i
)
On January 5, 1989 Crystal River Unit 3 was in the midst of a maintenance
)
outage with the plant in } ODE: 5, 00LD SWIDOWN. On this date it was determined
]
two solenoid operated 3-way air valves (}EV-130-SV and MSV-148-SV) [SB,PSV]
were undersized with r=wt to their Mavin = Operating Pressure Differential (POPD) rating.
MSV-130-SV aM MSV-148-SV control instrument air to air operated valves MSV-130 aM MSV-148 [SB,ISV). MSV-130 and MSV-148 are external Reactor miming (RB) [lE] isolation valves which isolate the A and B Once
'Ihrough Steam Generator (OISG)
[SB,SG]
secondary side blowdown lines respectively.
'Ihe installed or AS FOUND nodel (ASO3 Model 8320-A-24) of the 3-way solenoid valves had a }OPD rating of 40 PSID, whereas the design documents specified an instrument air operating pressure range of 80 to 125 PSI.
It appears these circumstances were never recognized because 1) the }OPD rating is not part of the nameplate data an:i can only be determined by researching the particular model number against the vendor catalog; and 2) the existing design documentation did not identify the required nodel number.
In reviewing the Instrument Data Sheets (IIE) for MSV-130-SV and MSV-148-SV, the model number was fouM to be missing.
'Ihe valves were field checked to determine the model number of the installed valves.
Since the design model number for these two valves could not be verified against the documentation, further errgineering evaluation was required.
On November 8, 1988 a Field Problem Report was written to resolve the apparent inconsistency between the IOPD rating of the AS FOUND nodel numbers and the known op&ating pressure.
It was during the engineering evaluation of the Field Problem Report (FPR) that the event was recognized and reported. 'Ihe IOPD rating is important because it rates the valve's ability to operate (i.e. position valve internals for isolating and venting associated air operators) as required under raximum differential prcssure canditions (static pressure drop actr;ss valve internal orifice).
On January 5, 1989 the event was drented as a Nanconforring Operations Report.
On January 9,1989 solenoid valves MSV-130-SV and MSV-148-SV were replaced with properly sized valves prior to the plant entering LODE 4, IUr SIUfDOWN.
As an immMiato action to the event, a reviev for similar design drent deficiencies a s initiated on all safety related air actuated valves which may have been affected.
As a result on January 11, 1989 similar deficiencies were found in the Av'entation for two other 3-way solenoid valves, GV-6-SV [lW,PSV] aM CAV-7-SV [lW,PSV].
GV-6-SV and GV-7-SV control instrument air to air operated valves CAV-6 and GV-7 [lW,ISV).
CAV-6 and CAV-7 are external RB isolation valves which isolate the A ard B OfSG secondary side sample lines respectively.
CAV-6 and CAV-7, were de-activated and held in the closed position in accordance with the Action Statem:mt of Limiting condition for j
Operation of 'Ibchnical Specification 3.6.3.1.
4 N
F ORW 3664
-*= me2 u a muctran uoutato*v commissiol.
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION amoveo ous No mo-oio.
EmpimEs 8/31/96 F Acatliv h4ME 111 pocs ET NUMtE R 62) ggn gyggga ig, pagg g, "CVP "No:
n="
CRYSTAL RIVER UNIT 3 ol5lolojoj3l0l2 8l9 0 l0 l1 0l2 0 l3 or 0 l9 rw m -,a.a.
m.-
- m,c r w,nm On January 13, 1989 the applicable Tochnt M Specification Action Statement was exited after solenoid valves CAV-6-SV and CAV-7-SV were replaced with the proper vraal.
The identification of additional valves with similarly deficient documentation iMicated there might be a pattern between inomplete h entation and lugve.r model substitution or replacement.
To substantiate such a e
possibility, five valves having cxmplete M=ntation were then selected aM field verified.
All five were found to be correctly equipped with the proper size of solenoid valve. Based upon these results additional field verification was not considered to be immediately r-wy; however, plans were developed for a systematic field verification of safety related valves.
As a result of investigation into the failure of the air control valve NV-253-SV [CB,PSV), it was determined an additional solenoid operated 3-way air valve was uMersized with respect to its }OPD rating.
NV-253-SV controls instrument air to the air operated valve WV-253 {}U,ISV] (Figure 1). N V-253 is an external Reactor Buildity (RB) (NH) isolation valve on the Reactor Ocolant PLmp seals [AB,P, SEAL] mntrolled bleed off line.
The installed or AS POUND model (ASO) Model NP 8320A197E) of the 3-way solenoid valve had a }OPD ratirg of 55 psid, where as the system design requires 110 psid.
The installation of the incorrect MOPD rated solenoid valve occurred in June, 1985, during a modification (MAR 82-05-24-09) to upgrade solenoid valves identified durirq inspections in accordance with !GC Bulletin 79-01B.
MUV-253 originally had two solenoid valves, SV1 and SV2, controlling the air operated valve (Figure 2).
SV1 provides full stroke control ard SV2 along with a pressure regulator provided for partial valve stroke capabilities.
During the modification development process one of the solenoid valves (SV2) was identified as having an inadequate temperature ratirg.
New upgraded models, "NP" as empared to "HT" or staMard types, were purchased to replace both of the existirg solenoid valves.
During reviews for installation of the now model solenoids, it was determined the modification instructions were in error in that they required SV:. to be a normally closM solenoid whereas the design for NUV-253 required SV1 to be a normally open solenoid. The SV2 solenoid was also determined to be the opposite design, and the two valves were switched as stipulated in a Field Charge Noti to the modification.
SV1 was retagged as SV2 ard visa versa. The required pressure ratings for the two solenoid valves, however were different; SV1 was a high delta pressure application valve and SV2 was a low delta pressure valve.
This requirement was apparently overlooked, aM only the valve function (i.e., energized to open or energized to close) was considered.
Therefore, the 1985 installed SV1 (ASCD Model NP8320A197E) had a rating cf 55 psid and SV2 (ASOD Model NP831655E) a }OPD ratirg of 125 psid, contrary to design requirements.
3;.o n...
9
em 3t44 u & NUCLEAR REGULAYQGV COMMrsh LICENSEE EVENT REPORT (LER) TEXT CONTINUATION araaovEo ove No >>so-oio4 E xa;RE5 8 0115 r C4 m haMEH3 DOCKET NUMBER (28 g,
" M."
- 3,T:
n="
CRYSTAL RIVER UNIT 3 o ls lo jo (o l3l0l 2 8 l9 0l0l1
- - 0l2 0 l4 0F 0 [9 1 EXT IM more space a mounwd, one eddeoner NMC Fem.N6A 's) M71 An additional system modification (MAR 87-07-25-02, removal of partial stroke capability) was performed in Har d, 1989 to renove MN-253-SV2 and the associated pressure regulator leaving only SV1 as the air control valve for NN-253 (Figure 1).
Shortly after the modification was field ccr.plete, MN-253-SV1 failed.
'Ihe failed solenoid operator was replaced ard failed again.
An investigation found the )OPD ratirg of MN-253-SV1 to be 55 psid as opposed to the required 110 psid.
'Ihis discrepancy was not found during the initial walk down of solenoid operators due to limited information on the solenoid valves provided on the 308 series drawirgs ard the physical inamibility of the valve due to its location in a high radiation area. '1he MOPD is inportant because it rates the solenoid valve's ability to cperate (i.e., position valve internals for isolatirg and venting the associated air operators) as required urder maximum differential pressure conditions (static pressure drop across the valve internal orifice).
On April 12, 1989, the event was rh't"ented on a Nonconforming Operations Report.
On May 9, 1989, the modification package to replace the solenoid valve, SV1, with the apptiately }OPD rated valve was ccupleted. An additional review of solenoid valve }OPD ratirgs and the Instrument Data Sheets was performed to identify any other discrepancy. None were identified.
CAUSE
'Ihe cause of the MSV ard CAV solenoid valves events was installation l (replacement or substitution) of improperly rated solenoid valves. 'Ibe lack of adequate design documentation (i.e.
model numbers) was a significant contributing factor.
'Ihe root cause of the event, the how or why it occurred, can not be accurately determined.
'Ibe followirq is a brief description based upon a historical search of documentation on MSV-130-SV.
During the design phase (1971), it appears the initial design for MSV-130-SV intended that the 125 psig supply air be filtered ard reduced to 40 psi.
Instrument Inta Sheets were developed for both a filter /mgulator ard a solenoid valve; however, the specific models and operatirg pressures were never identified.
In accordance with the verdor's records, the valve / actuator was delivered with l an AS SUPPIZED (ASCO Hodel 8320-A-33) solenoid valve rating of 80 to 125 psig.
'Ihe vendor drawirgs did specify the operatirg range but did not identify a specific model of solenoid valve.
Utility drawings were then developed specifyirg an operatirg rarge of 80 to 125 psig, but they did not identify a specific rodel of solenoid valve nor did they include any filter /rtgulator.
It appears the Instrument Data Sheet for the soleroid valve was also changed to reflect the new air pressure (still no model number) but the IDS for filter / regulator was never canceled.
A historical search of MSV-148-SV ard MSV-148 thwntation has provided similar inconclusive results.
go o., >..,
U s NUCLEAR KEGULATORY COW LICENSEE EVENT REPORT (LER) TEXT CONTINUATION
- eaovEc oua No mo+c4 E APiRES BtJt 85
@ LIT V NME 0 6 DOCKET NUMBER 12) g, pg,,
g "t'?,Po
%"llf:
"a CRYSTAL RIVER UNIT 3
,olslololol3l0l2 8l9 0l0l1 0 j2 0 }5 or 0 l9 nmu -. -. v~~<
.aa a anc r.
.n,se,>,,,,
Based upon stroke tim data involvirq diffemnt nodels of solenoid valves, it is an engineering judgemnt that the AS SUPPLIED solenoid valve nodel would l
have caused the valve / actuator time response to eM acceptable limits.
A historical search could not find any time msponse data prior to 1980.
'Ihe l
1980 time response dcta was acceptable ard based upon the engineering jt*ywnt, the model substitution or exdarge occurred scxnetime prior to 1980.
It would a be unmasonable to suspect the exchange may have taken place durirg original startup testing. 'Ihis is scmewhat supported by a review of the maintenance history for EV-130-SV, MSV-130, MSV-148-SV and MSV-148, as the amentation refers only to the AS FOUND model number or to no model number or uses 'like kind' terminology.
However, no start-up testirq dmentation can be found to support this conclusion.
A review of the historical documentation for CAV-6-SV, GV-6, GV-7-SV and CAV-7 did not provide additional information concernirg the cause of the improper installations.
'Ihe cause of the NUV-253 solenoid valve event was personnel ermr.
'Ihe Field Charge Notice to the modification which switted SV1 ard SV2 was not properly evaluated by utility engineers who failed to take into acocunt the different MOPD requirements for the solenoid valve functions.
Contributire factors to this event were 1) the WIO ratirg is not part of the nameplate data ard,an only be determined by researching the particular mode.1 nuaber againct the vendor catalog; ard 2) the valves had different energized functions which could have led the per hc involved to believe the different model numbers reflected only the valve function variations and, therefore, did not suspect additional design specification differences such as the NPD ratirgs.
EVINP ANALYSIS MSV-130-SV ard MSV-130 provides ErgineeIed Safeguards (ES) external RB isolation for RB penetration number 427.
MSV-148-SV and MSV-148 provides ES external RB isolation for RB penetration number 428.
CAV-6-SV ard CAV-6 provides ES external RB isolation for RB penetration number 440.
CAV-7-SV ard CAV-7 provides ES exterrpl RB isolation for RB penetration number 441.
Per Table 5-4 of the FEAR, each of the four affected penetrations (penetration 427, 428, 440 and 441) and isolation valves are categorized as a type III perubatiorVisolation; thus per figure 5-22 of the FSAR a type III penetration requires only a sirgle external isolatirg valve.
In the case of MSV-130 ard MSV-148, each penetration has a manual isolation valve immediately inside the RB ard both are normally required to be closed durirq power operation.
'Ihis requirement is established ard controlled administrative 1y by existirg operatirg procedures. Although the valves are not physically locked closed, RB access barriers provide cx2nparable controls against unauthorized openirg.
Nec somw sees
.:.c,, a.o v s Ucuaa noutaTon cwmssio~
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION areaovto oue No siso oio4 E X P:RES BrJI/95 F ACCITV NAME fil DOCKET NUMBER (2)
)
LgR huMBER tel PAot (31
)
'"w:W"
- :u:
l ns" CRYSTAL RIVER UNIT 3 l
o ls lo lo lo l 3l 0l 2 8l9 0 [0 [ 1 0l2 0 (6 or 0 19 l
,w,,.,.
.-m.,,,, mw.nm In the case of GV-6 ard CAV-7 each has a redundant automatic isolation valve imediately inside the RB, similar to a type I or type II penetration.
These valves are automatically closed by the redundant train of Engineered Safeguards actuaticrg/ isolation signal.
Even thcugh the MSV and GV solenoid valves were curside the intended design l ratings they centinued to perform their safety function. A Iuview of the irdividual valve leak rate data since 1983 indicates only two instanpos in which two cf the affected valves did not pass the individual leak rate tests and in both instances the cause was unrelated to the potential aspects of this event. In addition the Apperdix J leak rate testing data was reviewed and there were no instances of failure involving valves affected by this event.
MUV-253 provides Engincered Safeguards, external Reactac Building isolation for the Reactor Coolant Punp seal's controlled bleed off line. A review of MUV-253 leak rate test data frun 1985 to present irdicated no instances of failure or adverse trends for this valve to perform its safety function. This valve is a spring to close design ard the improperly rated solenoid valve should not affect the fail safe closure of MUV-253.
Based upon the above information there is no indication the event caused the affected valves to fail to perform their intended safety function, and therefore the event did not present any untie cafety risk to the general public.
00RRDC'.fIVE ALTICH Review of dccumentatica identifier) 148 safety related solenoids all of which required field wr.lkdown ard investigation.
The field walkdown was cxmpleted for all soleroids subject to inspection on March 13, 1989.
Design documentation for all safety related solenoid valves have Laen reviewd and updated as r==Jy.
The solenoid valve on MUV-253 was replaced with an appropriately !OPD rated valve by May 9,1989, while the plalit remained in }0DE 5.
A Human Performance Evaluatior. Syste review will be performed to determine any additional actions to be implemented which will aid in assuring a recurrence of this error is prevented.
An independent review was performed to verify the correctness of 10PD rating of all safety related solenoid valves against the instrument data sheet (IDS).
A field walk down of those air operators which are rated less than system pressure was performed to verify filter regulators are installed to reduce the air pressure in order not to exceed the 10PD rating of the solenoid valve.
1 c comw assa
J.,- naa -
u =uunaa noutaioav comm un LICENSEE MVENT REPORT (LER) TEXT CONTINUATION Areaoveo ous so 3 iso-oso.
E MPlaES 8'3t f85 F448 LIT i NAM Ill DOCKET NUMS R 433 LER NUMBER 46)
PAGE (31 "em!'t' "a*J:
saa CRYSTAL RIVER UNIT 3 0 l5 l0 l0 jo l3l 0l 2 8l9 0 l0 l1 0 l2 0l7 OF 0 l9 Tf.xT IMmore spece a reeuwed une endwenat NMC hwm J8Wsl(171 PREVIO[S SDEIAR DENIS A review of IERs indicates there are no pmvious similar events involvirq the installation of urdersized models of solenoid air valves. Based upon a br e r but more subjective perception of the term 'Similar Previous Events',
i.e.
shnilar personnel enurs relating to design deficiencies; a review of previous IERs has identified 12 similar previous events.
I l
ageoawaa.
NRC 7eem 3081 U S NUCLEAR Kf GuLaT02v COMMIS$som UCENSEE EVENT REPORT ILER) TEXT CONTINUATION Aaraovto ous =o siso-oio.
EkPtRf8 83fr86 FACILHY hAmt its W EttNUMSLR42)
LI A WUMBER 866 PAGE (2p
" O*.G
l' L*,'N vs'a CRYSTAL RIVER UNIT 3-o]stololol3l0l2 8 l9 0 l0 j '1 0l2 0 [8 jor o {9 r$xt<, - -..
., ~ e - -,,in, a
FlGUREI Aer Rest.
kM il is MUV-253-SY Air i
g EUPPly.
MUV 253 1
+..
=
=
3
F"ac eer-mea u a auctcan asautafoav couwisio=
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION avraevro ome sa no-ow.
- =,:ats etsi n paCsLITV NAME ill DDCRET NUMDER 82) ggg gygggp gg; PAGE m "Rty,;;*
Or*,t:
n'a CRYSTAL RIVER UNIT 3 olsjojoloj3l0l2 8l9 0l9 l 1 0l 2 0l9 or 0l9 TEXT ## miwo asece a roguerouf, use espooner AWIC 7sym JIEM's/117)
L n
FIGURE 2 Ane Aery.
(. Y
~
_S_
ps S V1 s /
t A
D
~ SUPPLY g
S v2
~~
N Pzr.
0*
Re s.
MU V-253 r
- .-o,
+4.-
<w o.,..