ML20055C731
| ML20055C731 | |
| Person / Time | |
|---|---|
| Issue date: | 06/15/1990 |
| From: | Novak T NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| To: | Spock T XOMOX CORP. |
| References | |
| NUDOCS 9006220287 | |
| Download: ML20055C731 (2) | |
Text
_
s 3lN15 ~C fir. Thomas Spock Xomox Corporation 4444 Cooper Road Cincinnati, Ohio 45242 Dear Mr. Spock-
Subject:
Preliminary Case Study Report on Solenoid Valve Problems at U.S. Light Water Reactors A preliminary AE0D case study report, " Solenoid Valve Problems at U.S. Light Water Reactors," is enclosed.
The study analyzes and evaluates operational experience and safety implications associated with failures and degradations of solenoid-operated valves (S0Vs) at U.S. LWRs.
It focuses upon the vulnerability of safety-related equipment to comon-mode failures or degradations of S0Vs.
The report presents information on more than 25 events in which comon-mode failures or degradations of over 600 S0Vs were affected, or had the potential to affect, multiple safety systems or multiple trains of individual safety systems. Although plant safety analyses do not address such common-mode failures or degradations of safety systems, operating experience presented in the report indicates that they have occurred and are continuing to occur.
A number of events in which safety systems have been adversely affected by degradations or failures of SOVs are considered sign ficant precursors. The case study notes that S0V problems permeate almost all U.S. nuclear power plants, and that they encompass many aspects of the SOVs' design, maintenance, and operdtion.
The case study also notes that individual S0V manufacturer's practices regarding guidance with respect to testing and maintenance contribute towards the observed problems. The report presents six recommendations which, if impiemented, should reduce reactor accident risks by reducing the likelihood for ccmon-mode failure or degradation of S0Vs affecting multiple safety systems or multiple trains of individual safety systems.
In accordance with our " peer review" process, prior to the finalization and distribution of our case study reports, we are providing you and other vendors who provided input to the case study with a copy of the preliminary report for review and comment. We request that you focus your review primarily on the accuracy and completeness of the technical details (i.e., comments are being solicited on the technical accuracy of the report). The findings, conclusions, and recommendations are provided for your information in order that you may understand the significance we place on these events and, therefore, obtain a more complete picture of the total report. Changes to the findings, conclusions, and recommendations will be considered only it the underlying information concerning the details cf plant design or systems operation is in error. We ask that coments be provided in writing.
%/
/
^
/
f,/
U NiiC FRE GEigm copy
~
iV Gi k i [.
o,
'J Mr.' Thomas _ Spock -
-12 l
Since we wish to finalize and issue the report shortly, we ask that any comments be received by us within 30 days from receipt of-this preliminary report.
If you require' additional-time beyond that point, please let us know.
If_ you or-your staff have'.any questions regarding-this. study, please feel free to contact me or Dr. Hal Ornstein at (301) 492-4439.-
~
Sincerely,,
Original signed by:
38:IEs 1[.' ffNak', Director Division of Safety Programs-Office for Analysis and-Evaluation of Operational Data-
Enclosure:
As stated Distribution:
-PDR-Central = File' JAE00 R/Fr
-DSP R/F ROAB R/F H0rnstein' SIsrael-Plam.,
JRosenthal
^
=EJordan 4
.l.
. Dross.
TNovak-
.MWilliams 3
-KBlack MTaylor,.EDO RSavio, ACRS-4.
i M
- 1r/
ROAB:DSP:AEOD ROAB:JDP:AE0D C: A :DSP:A@D D-H0rnstein:mk Plam vv JRosenthal T
k 6/g/90 6/g(/90 6/g90 6g/90
.. -......,,............ =,.............,,,,........... _........
.r,;
9 4
1 PRELIMINARY CASE STUDY REPORT SOLEN 0ID VALVE PROBLEMS AT U.S. LIGHT WATER REACTORS June 1990 Prepared by:
Dr. Harold Ornstein Reactor Operations Analysis Branch Office for Analysis and Evaluation of 0perational Data U.S. Nuclear Regulatory Comnission h
h \\ 0 ;y f5 {Ql1
e,
'e
-TABLE OF CONTENTS
.P. age EXECUTIVE
SUMMARY
vii 1
INTRODUCTION....................................................
1 2
DESCRIPTION OF EQUIPMENT........................................
3-3 USE OF SOLEN 0ID-OPERATED VALVES (SOVs)..........................
B 4
SOLEN 0ID-OPERATED VALVE FAILURE MODES: APPARENT AND ROOT CAUSES.
10 5
OPERATING EXPERIENCE:
SIGNIFICANT EVENTS INVOLVING COMON-MODE FAILURES OR DEGRADATIONS OF SOVs................................
11 5.1 Design Application Errors..................................
- 11 5.1.1 Ambient Temperatures................................
11 5.1.1.1 MSIVs at Perry - Excessive Heat From Steam Leaks................................
11 5.1.1.2 MSIVs at Crystal River 3 - Thermal Aging - Incorrect Estimation of Ambient Temperatures...............................
13 5.1.1.3 Millstone 2 - Thermal Aging - Localized
" Hot Spots" in Containment.................
13 5.1.2 Heatup from Energization............................
14 5.1.2.1 Grand Gulf 1 MSIVs - Thermal Aging (Sel f-Heating From Energization)...........
14 5.1.2.2 North Anna 1 and 2, and Surry 1 and 2 -
Thermal Aging (Self Heating Due To Energization)..............................
15 5.1.3 Maximum Operating Pressure Differential (MOPD) -
1 Multiple Plants.....................................
15 5.1.4 Directional SOVs....................................
20 5.1.4.1 Incorrect Valve Orientation at River Bend..
21 5.2 Maintenance................................................
21 5.2.1 Maintenance Frequency...............................
22 5.2.1.1 Dresden 3 - BWR Scram System --Primary System Leak Outside Containment............
22 5.2.1.2 Perry - Simultaneous Common-Mode Emergency Diesel Generator Failures........
24 5.2.2 Replacement Versds Rebuilding.......................
24-(
5.2.2.1 MSIVs at Perry - Inadequate SOV Rebuild.....
24 5.2.2.2 Brunswick 1 - Safety Relief Valves -
S0V Rebuilding Error:
Excess Loctite.......
25 iii
TABLE OF CONTENTS (Continued):
.P,in 5.2.2.3 Peach Bottom 3 - Scram System - SOV Rebuilding Error:
Excess Loctite...........
27-5.2.3 Co n tami na ti o n......................................... -
?4 5.2.3.1 Brunswick 2 MSIVs - Excessive Meat and Poor Air Quality (Hydrocarbons and Water)...
28 5.2.3.2 North Anna 1 and 2 - Multiple Systems Oil and Water Intrusion.........................
28 5.2.3.3 _Susquehanna 1 and 2 - Scram System: 011 and Water Contami nation....................
31 5.2.4 Lubrication..........................................
32 5.2.4.1 Multiple Plants - Manufacturing Error:
Residue-Producing Lubricant.................
32 5.2.4.2 Catabwa:
Poor Quality Air and Lubrication with Vaseline................
4..
34 5.2.4.3 Common-Mode Failure of 16 MSIVs at a Two Unit Station (BWRs):
Incorrect Lubrication.
35 5.L.4.4 Grand Gulf 1, LaSalle 1, and-River Bend MSIVs.- Sticking SOVs - Foreign Unidentified Sticky Substance (FUSS) - Lubricant Suspected...................................
36 5.3 Surve'11ance Testing........................................
40 5.3.1 Control Rod Timing Tests - Failed Scram Pilot SLVs - Perry.........................................
40 5.4 U se of Non-Quali f i ed SOVs...................................
40 5.4.1 Colt /~airbanks - Morse EDGs:
Repetitive Air Start Valve Failures.................................
40 6
ANALYSIS AND EVALUATION OF OPERATIONAL EXPERIENCE................
42 6.1 Common-Mode Failures........................................
42 6.2 SOV Failure Rates...........................................
43 6.3 Maintenance Problems........................................
44 6.3.1 Maintenance Problems - SOV Manufacturer's Contributions........................................
44 6.3.2 Maintenance Problems - Contribution of the Unrecognized S0Vs'....................................
46 6.3.2.1 Unrecognized SOVs in Emergency Diesel Generators..................................
46 6.3.2.2 Unrecognized SOVs in Auxiliary and Main Feedwater Systems......................
47 iv
4, TABLE OF CONTENTS (Continued)
Pa.21 1.
6.3.2.3 Unrecognized SOVs in BWR High Pressure CoolantInjectionSystems...................
47 l
6.3.2.4 Unrecognized SOVs in Instrument Air Dryers..
47 6.3.3 Maintenance Problems - Contributions of Utility Programs and Practices...............................
47 6.3.4 Rebuilding vs. Replacement...........................
48 7
FINMNGS.........................................................
50 7.1 Design Application Errors...................................
50 7.1.1 Ambient Temperatures.................................
50 7.2.2 Heatup from Energization.............................
50 7.1.3 Maximum Operating Pressure Differential..............
50 7.1.4 Unrecognized SOVs Used as Piece-Parts................
51 7.1. 5 Directional SOVs.....................................
51 7.2 Maintanance.................................................
51 7.2.1 Maintenance Frequency................................
51 7.2.2 Replacement Versus Rebuilding........................
52 7.2.3 Contamination........................................
-52 7.2.4 Lubrication..........................................
53 7.3-Surveillance Testing........................................
53 7.4 Verificaticn of the Use of Qualified SOVs...................
53
- 7. 5 Redundancy and Diversity....................................
53 7.6 Feedback of Operating Experience............................
54
.8 CONCLUSIONS......................................................
55 8.1 Safety Significance.........................................
55 8.2 Need For Action.............................................
56 9
RECOMMENDATIONS..................................................
57 9.1 Design Verification........................................
57-9.1.1 Ambient Temperatures.................................
57 9.1.2' Heatup From Energization.............................
57 9.1.3 Maximum Operating Pressure Differential..............
57 9.1.4 Unrecognized SOVs Used as Piece-Parts................
57 9.1.5 Directional S0Vs,.....................................
57 9.2 Maintenance.................................................
57 i
9.2.1 Frequency............................................
57 9.2.2 Replacement Versus Rebuilding........................
58 v
+ -
TABLE OF CONTENTS (Continued)
Page 9.2.3 Contamination........................................
58 9.2.4 Lubrication..........................................
58 9.3 Surveillance Testing........................................
58 9.4 Verification of the Use of Qualified SOVs...................
58 9.5 Redundancy and Diversity....................................
59 9.6 Feedback of Operating Experience............................
59 10 REFERENCES.......................................................
60 APPENDIX A SOV Failures Reported in LERS:
1984-1989................
A-1 B
Disposition of ASCO Dual-Coil 8323 SOVs Used for.........
8-1 MSIV Control C
Generic Communications on SOVs...........................
C-1 i
vi
e, EXECUTIVE
SUMMARY
The study atalyzes U.S. light water reactor (LWR) experience with solenoid-operated valves (SOVs).
It focuses upon the vulnerability of safety-related equipment to common-mode failures or degradations of SOVs.
The report presents information on over twenty events in which common-mode failures or degradations of over 600 SOVs affected, or had the potential to affect, multiple safety systems or multiple treins of individual safety systems. Although plant safety analyses do not address such common-mode failures or degradations of safety systems, operating experience presented in the report indicates that they have occurred, and are continuing to occur.
The events in which comon-mode failures of 30Vs have affacted multiple trains of safety systems or multiple safety systeme are important precursors.
-They indicate that actions are necessary to assure that important plant systems function as designed in accordance with plant safety analyses, and that plants are not subject to unanalyzed f ailure modes with the potential for serious consequences.
The report analyzes the operating experience and it outlines the root caus,es of comon-mode failures and degradations that have been observed, and provides -
recommendations to significantly reduce the occurrence of comon-mode SOV failures.
Analysis of operating data indicates that the underlying or root causes of many S0V failures are the users' lack of knowledge or understanding of SOVs' requirements or capabilities, such as:
SOVs' intolerance to process fluid contamination;sthe. necessity for preventive maintenance or changeout; and the propensity for rapid aging and deterioration when subjected to elevated tempera-tures.
Compounding the problem is the fact that some SOV manufacturers do not provide the users with adequate guidance regarding proper SOV maintenance and operation.
Further complicating the situation is the fact that many SOVs are
" unrecognized" i.e., they are provided as piece parts of larger components so that the end users-have a restricted knowledge of the 50Vs' operation and main-tenance requirements, or their useful design life.
The report addresses widespread deficiencies which were found in the. areas of:
design / application, maintenance, surveillance testing, and feedback of failure data.
It is recomended that for safety-related applications, licensees:
(1) verify the compatibility of SOV design and plant operating conditions; (2) ver-ify the adequacy of plant maintenance programs; (3) ensure that SOVs are not
-subjected to fluid contamination (e.g., instrument air); (4) review SOV surveil-lance testing practices; and (5) verify that all SOVs which are used in safety-related applications have been manuf actured, procured,-installed and maintained comensurate with their safety furiction to assure operation consistent with plant safety analyses.
Specific technical information supporting these broad recomendations is contained'throughout the report.
Detailed recommendations are provided in Chapter 9.
PRELIMINARY CASE STUDY vii
In addition, it is recommended that an industry group such as INPO take actica to improve the mechanism for: feeding back SOV failure data to the manu-factuiers for early detection and resolution of potential generic problems, PRELIMINARY CASE STUDY viii
c 1
INTRODUCTION All U.S. light water reactors (LWRs) rely upon solenoid-operated valves (SOVs) to perform safety-related and non-safety-related functions.
SOVs are used to operate with hydraulic and pneumatic fluids under a wide variety of con-ditions. They are used to control process fluid either directly, or indirectly as pilot controllers.
It has been estimated that the population of SOVs in safety systems at U.S. LWRs is between 1,000 and 3,000 per plant (Ref. 1).
Boiling water reactors (BWRs) usually have more SOVs than pressurized water reactors (PWRs), because of the extensive use of SOVs in BWR scram systems.
Many SOVs used in nuclear power plants are dedicated / qualified valves, which have undergone vigorous qualification testing to standards such as the Institute of Electrical and Electronics Engineers (IEEE) Standards 323, 344 and 382, and are manufactured in accordance with the Nuclear Regulatory Commission (NRC) requirements of Title 10 of the Code of Federal Regulations, Part 50 (10 CFR Part 50), Appendix B, and 10 CFR Part 21.
However, we have also found many cases in which plants use commercial, nonqualified SOVs to perform safety-related functions.
This study was initiated after several licensees experienced repetitive failures of SOVs at their plants and after the simultaneous failure of four SOVs at the Brunswick 2 plant on January 2,1988 (Ref. 2).
The Brunswick event resulted in a loss of containment integrity when two sets of redundant SOVs failed to close upon demand.
The NRC Office for Analysis and Evaluation of Operational Data (AE00) has reviewed and participated in follow up work that the licensees, the NRC regional inspectors, and the valve manufacturers have performed following the SOV-failures at Brunswick and several other plants.
A number of other significant operational events have occurred involving-malfunctioning SOVs.
Previous studies of SOV failures (Refs. 1, 3, 4, 5) dis-cussed SOV failure rates and provided a characterization of the degradations or failures.
This study addresses root causes and the generic nature of many of the observed failures.
Some of the significant events discussed in this report are:
Emergency diesel generator (EDG) failures at Perry and Catawba MSIV failures at Perry, Brunswick, Grand Gulf, LaSalle and River Bend AFW System degradation at Calvert Cliffs and North Anna Losses of containment integrity at Kewaunee, North Anna, and Brunswick BWR scram system component failures at Susquehanna, Brunswick and Dresden Safety Injection System degradation at Calvert Cliffs PRELIMINARY CASE STUDY 1
4
Chapters five and six of-this study provide comprehensive reviews and evaluations of operational experience and potential safety implications asso--
ciated with SOV problems at U.S. LWRs. This study provides several recommenda-tions to address the major deficiencies which were noted during the review of
--the_ operating experience.
fI 1
1 9
i PRELIMINARY CASE STUDY 2
1 i
~
V 2
DESCRIPTION OF EQUIPMENT There are many manufacturers and varietie's of SOVs used at nuclear power plants; SOV operation is based upon changing the electrical status of the valve's electro-magnetic coil, which in turn causes a shift of the position of an internal core.
The core acts to open or block the passageways inside the valve, changing the flow path within the valve.
A simplified version of a two-way SOV is illustrated in Figure 1.
Figures 2-through 4 illustrate other more complex SOVs which are made by three different manufacturers.
SOVs.are available for use over a wide range of temperature and pressure conditions for liquid and gas service.
They are available with the following formats:
normally open or normally closed fail open, fail closed,. fail as'is normally energized or normally de-energized ac'or de power, or both ac and de power two-way valves, three-way valves, four-way valves direct lift, pilot assist, balanced disc, gate, modulating control.
There is a wide range of sophistication and quality of SOVs.
For example, mass produced SOVs are available for home consumption for a few dollars each, ~
whereas a limited production of high quality SOVs are available at a much higher price.
SOVs that are qualified for Class 1E nuclear service (meeting IEEE Stan-dards 323, 344, 382, American National Standards Institute (ANSI) N45.2 and 10 CFR Part 50, Appendix B, and 10 CFR Part 21 requirements and having American Society of Mechanical Engineers (ASME)Section III "N" or "NPT" stamps) may cost several thousands of dollars.
l PRELIMINARY CASE STUDY 3
l
y I
1 E
sl
\\
t Lg 3
o
.I LLtLMVL d
c, i
mssms _ s hk p= > ~
El Q
g a
I A
a
\\
i S,,,,,,,
.s s
z a
N p LIlllkil C
/
g 1-]i wmwmwmN k
PRELIMINARY CASE STUDY 4
1 th;
[...
Solenoid "A"
\\,, ', '
N
,,,,/
/ Coil
's f
O
/
v
.s-i Solenoid Base Subassembly N g,,,,-
" s,
. -j Port 3 -
/fl Core Assembly
/
s j
Valve Body"N
' bi l
11 I I i
Port 1
,,,/
Port 2 y
g, f-Disc Holder
~lg#)
% 4#
Coil I
Solenoid Base
/
/
Subassembly
/
i Plug Nut Assembly h
'%., p Solenoid "B"
, Core' Assembly Stem Figure 2 Isometric Drawing of ASCO Dual Coil 8323 Solenoid Operated Valve PRELIMINARY CASE STUDY 5
1
g Magnetic A'ssembly.
j j
h
/
/
/
/
/
k-Terminal Plug w
/
s Vj i
m
~
. j
/
0:
/
/
m
~
solenoid Subassembly <
l g
g p
\\
dj )
coit 2
I N
N
! e i
I h!!
N m
[\\M N
stem A ssi Port
- valve
^
subassembly <
ws-- Port r _wm Port==s>
l:
h lilL y
Piston f
s amwwwxmug (sad k=J Figure 3 Schematic Drawing of a Valcor Solenoid Operated Valve PRELIMINARY CASE STUDY 6
i MaOnet Assembly Solenoid Coil Terminal #
Board
,m y Bonnet Assembly Fixed Core Return Spring Moveable Core Main Disc Pilot Disc f
r h)
~q %
%D
=-
c' j,(
(
. Figure 4 -
Schematic Drawing of a Target Rock Pilot Assisted Solenoid Operated Valve l
PRELIMINARY CASE STUDY 7
3 USE OF SOLEN 0ID-0PERATED VALVES In many applications SOVs are used as alternates t~o motor-operated valves
.(MOVs).
SOVs are frequently used as pilot operators to control air-operated valves (A0Vs).
The advantages of using SOVs instead of MOVs are that they generally have fewer moving parts, are compact and may be easier to mount.
They also have low power requirements and have fast response times.
Some 50V manufacturers' literature claim that SOVs have long qualified lives, have low initial and installed costs, and require low maintenance.
The use of A0Vs, MOVs and SOVs is a matter of preference of application that is determined by the utility, nuclear steam system supplier, and architect engineer; their specific utilization is not a licensing requirement.
Table 1 lists many of the systems that use SOVs at U.S. LWRs.
l r
PRELIMINARY CASE STUDY 8
~.
Table 1 Systems'Which Use S0Vs at U.S. LWRs 1.
- 2. - PWR Rod Control 3.
Reactor Coolant (RCP seal) 4.
Safety Injection 5.
Primary Conteinment Isolation 7.
High Pressure Coolant Injection / Reactor Core Isolation Cooling 8.
High Pressure Injection 9.
Automatic Depressurization 10.
Emergency Diesel Generator 11.
Instrument Air 12.
Chemical Volume Control / Charging and Letdown /Boration 13.
Pressurizer Control 14.
Steam Generator Relief (PORVs, ADVs) 15.
Low-Temperature Overpressurization Protection 16.
Decay Heat Removal / Residual Heat Removal 17.
Component Cooling Water.
18.
Service Water 19.
Reactor Head Vent 20.
Steam Dump 21.
Reactor Cavity / Spent Fuel / Fuel Handling 22.
Torus and Drywell/ Vent and Vacuum 23.
Emergency DC Power
- 24. Main Steam (Main Steam Isolation Valves / Auxiliary Boiler) 25.
Reactor Building / Auxiliary Building (Ventilation and Isolation)
- 26. : Main Feedwater 27.
Condensate
- 28. -Moisture Separation / Reheat.
29.
Containment Atmosphere / Containment Spray 30.
Standby Gas Treatment 31.
Floor / Sump Drain j32.
Sampling (normal and post-accident) 33.
Fire Suppression 34.
Turbine / Generator 35.
Reactor Building Purge 36.
Containment Air Lock 37.
Leak Detection 38.
Radwaste l
n PRELIMINARY CASE STUDY 9
l
4 SOLEN 0ID-OPERATED VALVE FAILURE MODES:
APPARENT AND ROOT CAUSES Previous studies (Refs. 1, 3, 4, 5) have noted that~ details of the failure mechanisms, the apparent causes, or the root causes of SOV failures were not provided in approximately half of the licensee event reports (LERs) and nuclear l-plant reliability data system (NPRDS) failure records for-years 1978 through 1984.
Appendix A of this report provides a listing of approximately 200 LERs describing 50V failures which occurred at U.S. LWRs between 1984 and 1989.
t The apparent and root causes of most (approximately 75 percent) of the SOV.
failures reported in LERs between 1984 and 1989 are given below:
L a.
Coil failure or burnout that was attributed to design or manufacturing deficiencies (early failure /end of life) or an error in application (type of current, voltage level, environmental conditions).- [11%)
b.
Valve body failure or leakage that was attributed to design or manufactur-ing deficiencies, such as excessive tolerances on internal parts; excessive 7
wear / degradation of gaskets, 0 rings, seals, or springs; or foreign particulates preventing proper sealing.
[13%)
c.
Passageway blockage / internal binding that was attributed to contaminants such as dirt, corrosion products, desiccant, water or moisture, incorrect lubricants, excessive lubrication, or hydrocarbons.
[9%)
d.
Electrical malfunctions that were attributed to faulty internal wiring, reed switch shorts or external wiring with inadequate connections, splices, or grounds.
[12%]
e.
Design errors or misapplications that were attributed to incorrect valve configuration (normally open vs. normally closed; normally energized vs.
normally de-energized); incorrect designation of " fail-safe" condition; incorrect electrical source (ac vs. de,-voltage level); incorrect desig-nation of environmental conditions (temperature, moisture, radiation);
incorrect designation of maximum operating pressure differential;-
incorrect material selection (incompatibility between elastomeric parts and process fluid contaminants); incorrect valve orientation (horizontal vs. vertical).
[13%]
f.
Installation errors that were attributed to incorrect physical orientation (backwards, upside-down), electrical source (ac vs. de, voltage level),
t
'or inadequate electrical connections (e.g., loose connections, incorrect grounds).
[7%)
g.
Maintenance errors that were attributed to incorrect determination of useful life or time between overhauls; inedequate preventive mainte-nance or incorrect preventive' maintenance.
[7%)
h.
Stio.ing that was the result of unidentified foreign substances coating j
valve internals, excessive use of lubricant, or foreign particulates.
[5%)
PRELIMINARY CASE STUDY 10
5 OPERATING EXPERIENCE: SIGNIFICANT EVENTS INVOLVING COMMON-MODE FAILURES OR DEGRADATION OF SOVS The events described below were chosen as a representative set.
They should not be construed as being a complete set of common-mode failures and degradations of SOVs.
Additional events are tabulated in Appendix A.
Many other SOV failures fall below NRC reporting requirements, and as a result are not captured in the LER data base.
Many individual SOV failures not reported in the LER data base are reported
'in the Nuclear Plant Reliability Data System (NPRDS) data base.
Reference 1 noted that for 1978-1984 data, all SOV failures reported in LERs were also reported in NPRDS.
5.1 Design Application Errors Representative operating experience illustrating design application errors associated with high ambient temperature, internal heatup from energization, incorrect maximum operating pressure differential and incorrect valve orienta-l tion are described below.
Based on this experience, findings and recommenda-tions relevant to design application errors are provided in Sections 7.1 and 9.1 i
respectively.
5.1.1 Ambient Temperatures 5.1.1.1 MSIVs at Perry - Excessive Heat From Steam Leaks On October 29, 1987, while performing stroke time testing, three of the plant's eight MSIVs failed to close within the plant Technical Specifications' allowable time of five seconds.
Two of the MSIVs were in the same main steam-J i
line.
During subsequent testing, each of the three valves closed within the L
Technical Specifications value.
l l
Since the valves all stroked satisfactorily subsequent to their initial l
failures, the licensee believed that the failures were due to the presence of l
impurities in the air pack SOVs controlling the MSIVs, and that the impurities were apparently discharged during subsequent MSIV operation.
As a result, the three MSIVs that had failed were declared operable.
. These MSIV air packs consist of a single-coil 4-way SOV (ASCO NP8320),
l a dual-coil 3-way SOV (ASCO NP8323) and three poppet type air pilot-operated valves-(2, 3, and 4-way CA Norgren Co.).
A photograph of one of the Perry plant's MSIV air packs appears in Figure 5.
In. response to NRC concerns, the licensee performed additional MSIV stroke testing.
As a result on November 3, 1987, the inboard and outboard MSIVs in the "0 line again failed to close,within the required 5 seconds (outboard MSIV closed in 2 minutes and 49 seconds and the inboard MSIV closed in 18. seconds).
'~
Additional MSIV stroke tests were performed, and both MSIV's again closed within the Technical Specification allowable times.
PRELIMINARY CASE STUDY 11 1
O ASCO Dual-Coil NP8323 SOV NORGREN 2-way dm. -
AOV x=3
)~_
- fi G
NORGREN
$hIf 4-w"
^ n'
~
NORGREN n
3-way AOV l
'[fi wa.
ASCO l
4 Single-Coil NP8320 SOV Figure 5 MSIV Air pack from Perry Nuclear Power Plant, November 1987 1
I l
l' s
Because of continued NRC concerns about MSIV reliability, the licensee shut down the plant and established a plan to find the root cause of the MSIV fail-I ures (Refs. 6, 7, 8).
Intense investigative efforts were conducted by the util-ity to determine the root cause of the MSIV failures.
The' failures of the MSIVs lon both October 29 and November 3, 1987, were attributed to the failure of the ASCO dual-coil Model NP8323 SOVs to shift position upon de-energization.
The l-SOVs failed to shift position because of degradation of their ethylene propylene dimer (EPDM) seats and discs.
The degradation was caused by high temperatures that had existed in tne vicinity of the SOVs as a result of several steam leaks.
Originally, hydrocarbon intrusion was suspected as. having contributed to the degradation of the EPDM seats and discs.
It was not until microscopic and spec-
. tral analyses were performed at an independent laboratory a month after the event that the possibility of impurities from hydrocarbon intrusion was elimi-cated as a root cause of these failures (Ref. 9).
However, as part of its cor-rective action to prevent future failures, the licensee took steps to improve the maintenance of the instrument air system.
In addition, the licensee under-took an aggre&sive program to review the effects of all known steam leaks that could affect other safety-related equipment.
I 5.1.1.2 MSIVs at Crystal River 3 - Thermal Aging - Incorrect Estimation of Ambient Temperatures In April 1989, NRC inspectors reviewed the environmental qualification of i
electrical equipment at the Crystal River 3 plant.
Their review found that errors had been made in the licensee's determination of the service life of 16 normally de-energized S0Vs that are used to pilot the plant's MSIVs (Ref. 10).
The licensee's determination of SOV service life was made assuming an ambient temperature equal to the weighted average of the temperature of the areas where the SOVs were located.
The licensee's calculations did not consider the localized elevated temperatures that the SOVs were subjected to as a result of hot process piping.
Recalculation of the service life of the SOVs using
.j representative ambient temperatures reduced the estimated service life of the S0Vs from 40 years to 8 years.
As a result, the licensee is replacing those SOVs-sooner than previously anticipated.
5.1.1.3 Millstone 2 - Thermal Aging - Localized " Hot Spots" in Containment In November 1938, an NRC inspection report (Ref. 11) noted that Millstone 2's environmental oualification program recognized a significant shortening of the qualified lifetime of eight Valcor SOVs that are used for pressurizer and reactor vessel head vents.
Originally the S0Vs were calculated to have quali-fied lives of 40 years based upon an ambient temperature of 120*F.
Although the plant's Ter.hnical Specifications require that the " primary containment average air temperature" does not exceed 120*F, the-licensee found localized " hot spots" of 157*F in the vicinity of the eight S0Vs.
The licensee determined that the
-increase in ambient temperatures from 120*F to 157*F shortened the lifetime of the SOVs from 40 years to 12 years. The problem of equipment degradation due to localized hot spots is not unique to Millstone 2.
Reference 12 lists several other plants that have experienced localized thermal " hot spots" inside contain-ment.
In addition, NRC Information h tice 89-30 (Ref. 13) noted that similar heating events have been reported since 1982.
The information notice alerted licensees to the potential for exceeding equipment's qualification specifications when the bulk temperatures are measured by a limited number of sensors that indicate acceptably low average temperatures.
PRELIMINARY CASE STUDY 13
!~
5.1.2 Heatup from Energization 5.1.2.1 Grand Gulf 1 MSIVs - Thermal Aging (Self-Heating From Energization)
On August 14, 1989, following a reactor trip, one MSIV (inboard "B" line) failed to close upon demand (Refs. 14,15,16). The MSIV did close about 30 minutes later. The failure of the MSIV to close was attributed to the faltare of an ASCO dual-coil NP8323 SOV, a piece part of the MSIV air pack.
The licen-see's investigation found a piece of EPDM from the SOV's disc on the 50V's out-let SOV' port screen. The licensee concluded that the piece had been lodged in the s internals, thereby keeping the SOV from venting control air and hence keeping the MSIV from closing.
It is believed that after the EPDM piece became dislodged from the internals, the MSIV closed.
g Subsequent inspections by the licensee of all eight ASCO dual-coil NP8323 SOVs piloting the MSIVs disclosed that all eight had degraded seats.
Initial visual inspection did not reveal the degradations, which became apparent under microscopic examination.
The EPDM seats of all eight SOVs had cracks.
- However, on six of them, the raised portion of the seat fo madebytheseatoftheexhaustport,wasmissing.rmedbytheannularimpression It appeared that six of the eight SOVs had experienced similar sloughing of material from the seat.
The August 14, 1989 failure is believed to have been caused by a piece of the EPDM disc material which had been extruded into the 50V's exhaust port vent hole.
The extruded material had separated from the disc as a result of the adhesive and frictional forces when the normally energized SOV was de-energized.
The frictional and adhesive forces eventually led to the tearing off of the extruded parts of the EPDM discs.
The extrusion of EPDM discs is discussed in GE Service Information Letter (SIL) 481 (Ref. 17).
SIL 481 notes that the intrusion of the disc into its ex-haust port may account for previous events involving the sticking of similar EPDM dual-coil SOVs, but tearing of the discs had not been observed previously.
It is believed that the tearing and overall degradation of the dual-coil SOVs' EPDM discs at Grand Gulf was symptomatic of thermal degradLtion resulting from the excessive time the EPDM materials were exposed to high service temperatures.
TheEPDMdiscshadbeenoperatingatelevatedtemperaturesduetotheenerilza-i tion of the dual coils.
The local temperatures inside the SOVs near the E DM discs were approximately 325'F inside the inboard SOVs in a 135'F drywell and 305'F inside the outboard 50V in a 125'F steam tunnel.
The SOVs had been in service for approximately 4.5 years.
However, the qualified lives of the degraded EPDM discs are estimated to have been 2.2 years for the inboards and 3.2 years for the outboards based upon environmental temperatures of 135'F for the inboard SOVs and 125'F for the outboard SOVs.*
The NRC issued an information notice on this event, noting the life shortening effects of self-heating from coil energization (Ref.18).
Subsequently, ASCO issued a service bulletin providing licensees with heat up data for all their nuclear qualified SOVs (NP series).
(Ref. 19).
- 0ther EPDM discs in the same 50V which were exposed to slightly higher temperatures were estimated to have had qualified lives of 1.58 and 2.28 years, respectively.
PRELIMINARY CASE GTUDY 14 i
40 m &' r Mt
1 1
1 0
-O 1
5.1.2.2 North Anna 1 and 2, and Surry 1 and 2 - Thermal Aging (Self Heating Due to Energization) 1 i
In December 1986, Virginia Electric & Power Co. (Vepco) requested ASCO to provide information regarding the effects of "self heating" in continuously
{
energized SOVs.
ASCO's response indicated that a significar,t increase in temp-erature would occur and that the temperature increase could result in a signi-ficant reduction in the qualified life of the SOVs. The licensee recognized i
that provinus estimates of 50V service life did not account for the effects of self heating (Refs. 20, 21).
The licensee evaluated the affected SOVs and determined that, contrary to previous analyses,125 SOVs would require replace-ment at North Anna 1 and C between the 1987 and 1989 refueling outages (Ref. 22).
The SOVs affected pilotr.d air-operated valves, many of which served containment isolation functions.
the systems affected were:
Safety Injection, Reactor Coolant, Main Steam, Component Cooling Water, Containment Vacuum, Radiation Monitoring, Sampling Systems, Instrument Air, Post Accident Hydrogen Removal, l
Heating and Ventilation, Steam Generator Blowdown, Gaseous Vent and Aerated Drains.
l The licensee recognized that Surry 1 and 2 were similarly affected, and Vepco engineering informed personnel at the Surry station of this problem.
t Similarly, Surry 1 and 2 required early replacement of 58 ASCO SOVs because of self heating.*
It is interesting to note that the licensee for North Anna station stated in a Deviation Report (Ref. 21) that these findings were non reportable because:
"NRC and utilities are aware of this issue to some extent." In Reference 20, the licensee noted that it had learned of this problem initiall with " industry representatives" at Equipment Qualification (EQ)y from discussions seminars in late 1986.
5.1.3 Maximum Operating Pressure Differential (M0PD) - Multiple Plants Many plants have experienced conditions in which SOVs failed or could have failed to perform safety-related functions because of exossive operating pres-sure differentials.
Figure 6 is a schematic diegram of an 50V, illustrating how an operating pr6ssure differential in excess of its maximum operating pres-I sure differential'(MOPD) can cause an 50V to malfunction.
When the SOV is in the de-energizcd position, pressurized fluid enters the valve at port 2 and is i
blocked by the core assembly.
If the pressure differential between ports 2 and 3 exceeds the POPD, the overpressure could lift the core assembly, resulting in
' leakage of fluid from port 2 to port 1 and port 3.
In the energized position the cere M se@ly is raised to block the exhaust port (port 3).
Howsver, the excess pressure would act to retard or core subassembly from dropping down (shifting) oon de-energization. prevent the As a re-sult, de-energi. ting the valve would not assure tne valve ach0eved its correct de-energized position (block off port 2).
l
- Telecopy comm nication between W. Murray, Vepce, and H. L. Ornstein, G RC, l
December 19, 1989, t
PRELIMINARY CASE STUDY 15
,.,_y-y-.
. _.,,,qn s
Port 3 Enhaust I
Solenoid Base Sub Assembly b
/
Coil Core j Assembly
/
/
(:@,
- 7y, Ifj>.
s'? +
.gggf%
To Control e Port 1 Port 2 e Flow Valve
, :. o m
at u >
w
.,,m
.,i.
Figure 6 Schematic of a Solenoid Operated Valve Illustrating Effect of Operating Pressure Differentials i
{
PRELIMINARY CASE STUDY 16 7
For many SOVs, the MOPD rating does not appear on the nameplate or in the installation and maintenance instructions. Vendor catalogs need to be consulted to determine those 50Vs' MOPD ratings.
In May 1988, the NRC issued Information Notice 88-24 " Failures of Air-Operated Valves Affecting Safety-Related Systems" (Ref. 23).
1; informed licensees of two SOV failures which were experienced at Kewaunee (Ref. 24) and of the potential for additional failures at Kewaunee and Calvert Cliffs I and 2 (Refs. 25-27).
Subsequently, several licensees informed the NRC of similar dis-coveries in their plants, where the potential for overpressurizing SOVs exists, which could prevent the SOVs from performing their safety-related functions.
At some plants, the task of verifying the potential for overpressurizing SOVs has been complicated by the fact that documentation is not readily available.
For example, Millstone 1 and 2 (Ref. 28), Crystal River 3 (Ref. 29), have re-ported that documentation to identify SOVs in containment is not readily available, and that containment walkdowns are necessary for their identification.
It is not clear that Information Notice 88-24 has been effective in eliminating the potential for SOV overpressurization.
Our concern is predicated upon Ref. 29 and a followup discussion in which the Crystal River 3 licensee stated that its review of the potential for SOV overpressurization assumed the properoperationofin-linepressureregulators,itdidnotaddresstheconse-quences of pressure regulator failures.
One of the events described in Infor-mation Notice 88-24 involved the discovery at Calvert Cliffs that several safety systems were vulnerable to single failures of pressure regulators in the air supply system.
The earliest 50V overpressurization failures that we found occurred in 1980 at the Pilgrim plant.
On October 7,1980 and again on October 31, 1980, a safety relief valve (SRV) spuriously opened while the reactor was at power.
On each occasion,-the SRV did not reclose until the reactor was shutdown and the reac-tor coolant system was depressurized.
The spurious valve openings were caused by excessive pneumatic (nitrogen) supply pressure to the 50V controlling the SRV.
The high nitrogen pressure exceeded the SOV's MOPD, causing the 50V to shif t position which caused the SRV to spuriously open.
The NRC issued an information notice and a bulletin on these events (Refs.
30,31).
Information Notice 80-40 (Ref. 30) indicated that two-stage SRVs with Target Rock SOVs are susceptible to such MOPD malfunctions, whereas older three-stage SRVs having ASCO or AVC SOVs are not.
Bulletin 80-25 (Ref. 31) required licensees to review and upgrade their SRV pneumatic supply systems and/or SOVs to assure that the SOVs operate within their maximum operating pressure. The bulletin required licensees to install protective devices (such as relief valves) to protect the SOVs against excessive supply pressures.
The issue of overpressurization failures of SOVs in other systems was not addressed in the information notice or the bulletin.
The discovery of the potential for overpressurizing multiple SOVs at the Vogtle plant was reported in Reference 32.
Reference 32 described a situation
- Telephone discussion between L. Kluit, Florida Power Corporation, and H. L.
Ornstein, USNRC, October 10, 1989.
PRELIMINARY CASE STUDY 17
mm i
\\
in which SOVs controlling the operation of all eight MSIVs could fail because of overpressurization due to overheating.
The MSIV manufacturer (Rockwell) had noted that a small steam-line break in the vicinity of the plant's MSIVs could cause an increase in the hydraulic fluid pressure in excess of the 50Vs' maximum operating pressure differential.
These SOVs were manufactured by the Keane Company.
As a iesult of SOV overpressurization, both MSIVs on one or more steam-linescouldallowuncontrolledblowdownofmorethanonesteamgenerator following a main steam or feedwater line break.
Essentially, if the MSIVs' hydraulic actuator fluid beated up 12'F, a condition not bounded by the plant's safety analyses could result, The licensee's corrective action was to replace the SOVs with others having higher MOP 0 ratings.
In November 1987, the Kewaunee plant actually experienced two SOV failures caused by overpressurization (Ref. 24).
During review of the two SOV failures, the licensee found that 58 additional SOVs could fail to perform their safety-related functions as a result of overpressurization.
In April 1988, the licensee of Calvert Cliffs 1 and 2 found th.t 40 SOVs could fail to perform their safety-related function as a result of overpressuri-zation (Ref. 25)
In the case of TMI-1, (Ref. 32) the SOVs were connected to line pressures in excess of the maximum dictated by the SOVs' MOPD.
In the case of Kewaunee and Calvert Cliffs 1 and 2, it was found that failure of a non-qualified pressure regulator under accident conditions could result in the SOVs being subjected to supply pressures in excess of the maximum allowed by the 50Vs' MOPD.
Eight reported events in which SOVs failed, or had the potential to fail, to perform their safety-related functions as a result of excessive operating pressure differentials are briefly described below.
(1)
Three Mile Island-1; October 17, 1980; (Ref. 32)
The following 11 containment isolation valves could have been prevented from achieving their safeguard positions:
2 makeup to core flood tanks 2 core flood tank sampling 1 reactor building vent 6 fan motor coolers for the reactor building cooling units.
(2) Vogtle-1; January 22, 1987; (Ref 33) 8 main steam isolation valves could have failed to perform their safety function.
(3) Kewaunee; November 28, 1987; (Ref. 24) 2 containment isolation valves failed to close 1 pressurizer relief tank makeup 1 RCDT pump discharge (its redundant SOV had the potential for similar failure) 58 other SOVs in safety-related applications were also found to be subject to overpressure failure.
PRELIMINARY CASE STUDY 18
g l ;. K (4)' Calvert Cliffs 1, 2; f.pril 14, 1988; (Refs. 25, 26, 27)
The following 40 SOVs, equally distributed between Units 1 and 2, had the potential to fail:
8 auxiliary feedwater system 8 steam generator blowdown isolation system 6 reactor coolant pump bleedoff isolation l
18 safety injection system (fill and vent)
(5) Pilgrim 1; July 19,1988; (Refs. 34, 35, 36)
The following six SOVs had the potential to fail due to overpressure:
4 control room high efficiency air filtration system damper controls (2 in each train)
I standby gas treatment system damper control l
1 primary containment system RCS sample line isolation valve (6) Millstone 2; October 8 1988; (Ref. 37) l One containment isolation valve failed as a result of an air pressure l
regulator that failed high.
l (7) Millstone 1, 2 and 3; November 8, 1988; (Ref. 28) l Unit 1:
The status of 16 SOVs in safety-related functions was unknown because of a lack of design information.
Unit 2:
A total of 24 " harsh environment safety valves and their installed EEQ (sic) solenoid valves" could have failed as a result of overpressure (one of the 24 had failed on October 8, 1988).
The licensee also noted that the status of an unspecified number ofsafety-relatedSOVswasundeterminedbecauseghe"databaseis l
incomplete as to solenoid make and model number.
Unit 3:
Approximately 20 SOVs installed in " safety valve configurations" could have failed because of overpressurization.
The specific applications of thesa SOVs were not listed.
However, the li-L censee indicated that there are Jany additional inaccessible SOVs that may also be susceptible to overpressure failure.
The licensee indicated that determination of such vulnerability would be made subsequent to future walkdowns when 50V nameplate data could be obtained.
(8) Crystal River 3; November 8,1988, January 5,1989 and January 11, 1989:
(Refs. 29, 38, 39, 40)
Five containment isolation valves had the potential to fail due to overpressure:
2 once through steam generator blowdown lines 2 once through steam generator sample lines 1 reactor coolant pump seal controlled bleed off line PRELIMINARY CASE STUDY 19
o.
5.1.4 Directional SOVs We are aware of seven plants that have observed spurious operation of safety-related Target Rock angle-type SOVs due to improper valve orientation.
As shown in Figure 3, upstream fluid pressure at the angle-type 50V's inlet port assists valve disc seating.
However, many licensees have also learned from their own operating experiences and from followup discussions with the SOV manufacturer, that several different models of Target Rock angle-type SOVs used for isolation purposes are " uni-directional" i.e., they will experience undesired seat lifting when the backpressure (pressure at the outlet port shown in Figure
- 3) is only 2 to 5 psi higher than the upstream or inlet pressure.
As noted in Target Rock Manual TRP 1571 (Ref. 41), the manufacturer has been aware of this problem at nuclear plants since 1978.
However in the late 1970s time-frame, Target Rock developed an 50V for use as a bi-directional isolation valve (would not open spuriously due to high backpressures). Tar 9et Rock considered the spurious seat lifting to be an Architect Engineer / Licensee " application problem"
--not an SOV problem.* The issue of uni-directional isolation SOVs is clearly addressed in some - but not all Target Rock SOV users manuals.
For example, Reference 42 noted that the uni-directional qualities of the Target Rock angle-type SOVs are stated in Target Rock manual TRP 1571 (Ref. 41) i.e.
"Most solenoid valves because of the nature of the operation of the valve, will stop flow in only one (1) direction.
By design, upstream pressure acts on the top of the disc, forcing it onto its seat, thereby creating a tighter seal.
However, if downstream pressure rises above upstream pressure, the disc will tend to lift off of its seat, thereby allowing flow."
Since Target Rock considered the spurious opening of uni-directional SOVs j
to be an application problem, not an SOV problem, Target Rock did not issue any field service notifications to alert owners of the affected SOVs to this problem.
Plants that have experienced spurious openings of safety-related Target Rock angle-type SOVs are:
H.B. Robinson 2 (1980)
(unspecified number of SOVs)
ANO-1 (1985)
(2 SOVs)
ANO-2 (1985)
(2 SOVS)
River Bend (1986) & (1989)
Harris 1 (1987)
(2 SOVs)
Hatch 2 (1988)
(12 SOVs)
The licensees' corrective actions were to re-orient the SOVs to assure that they would operate properly during accident conditions.
Section 5.1.4.1 describes the most recent events which occurred at River Bend.
- Telephone discussion between T. D. Crowley, Target Rock Corporation, and H. L. Ornstein, USNRC, January 24, 1990.
PRELIMINARY CASE STUDY 20
7 5.1.4.1 Incorrect Valve Orientation at River Bend i
In April and May 1989, during testing conducted in response to NRC Generic letters 88-14 (Ref. 43), the River Bend station found ten Target Rock SOVs used in safety-related applications which would spuriously open during accident con-ditions upon loss of instrument air.
The opening of those uni-directional SOVs would have resulted in the blowdown of safety-related accumulators and would have prevented safety-related equipment from performing their functions as assumed in plant safety analyses (Refs. 42, 44).
For example:
(1) Spurious actuation of six uni-directional SOVs upon loss of instrument air would result in bleed-down of safety-related accum-ulators in the control building, the auxiliary building and the fuel building.
The licensee postulated that rapid depletion of accumulators in the control building (in 3.7 minutes) would pre-vent proper operation of building dampers and would adversely affect cooling of safety-related equipment, control room cooling, and control room air filtration.
Depletion of accumulators in the auxiliary building would affect building dampers resulting in the loss of cooling of safety related switchgear.
Depletion of accumulators in the fuel building would affect building dampers and would impact air filtration and prevent tha maintaining of a negative building pressure.
(2) Two uni-directional S0Vs were found in the standby service water system (ultimate heat sink) which could spuriously open when subjected to accident conditions to prevent removal of heat through the ultimate heat sink.
(3) Two uni-directional SOVs were found in the instrument air system which could spuriously open upon loss of instrument air.
plant'pening would prevent long-term operability of all of the Such o s (16) ADS /SRVs.
In Reference 42, the licensee also noted that several years earlier (1986) it had found three other Target Rock SOVs which had to be re-oriented due to spurious opening which was discovered when they were subjected to leak rate testing.
Those three SOVs had served as containment isolation valves in the containment hydrogen sampling system.
The licensee did not consider that event to be reportable at that time.
5.2 Maintenance Representative operating experience illustrating maintenance problems associated with maintenance frequency, replacement versus rebuilding, contamina-tion, and lubrication are described below.
Based on this experience, findings and recommendations relevant to maintenance problems are provided in Sections 7.2 and 9.2 respectively.
PRELIMINARY CASE STUDY 21 l
5.2.1 Maintenance Frequency 5.2.1.1 Dresden 3 - BWR Scram System - Primary System Leak Outside Primary Containment During recovery from a reactor scram at 81 percent power on September 19, 1985, Dresden 3 experienced a leak of reactor coolant outside primary contain-ment. The leakage path was through the scram outlet valves and the SDV vent and J
drain valves (Refs. 45,46,47).
The NRC issued Information Notice 85-95 to 1
alert licensees to the potential for reacto. coolant leakage into the reactor building which could result from scram solenoid valve problems (Ref. 48).
The information notice indicated that c similar event had occurred at Dresden 2 in 1972; however, the licensec did not determine the root cause of that event.
After the reactor scrammed in September 1985, the control room operators attempted to reset the reactor protection system (RPS).
RPS channel A was successfully reset, but channel B could not be reset.* This channel configura-tion allowed the scrafn pilot SOVs to vent air, resulting in reduced air header pressure.
The reduced air header pressure (38 psig) was sufficient to allow the SDV vent and drain valves to open (opening pressure s8 to 15 psig), but it was not sufficient to enable the scram inlet and outlet valves to reclose (s42 psig required to close).
For approximately 23 minutes, reactor coolant leaked outside primary containment into the reactor building.
The high temperature reactor coolant flashed to steam.
The leak resulted in elevated radiation levels on the first three floors of the reactor building.
In addition to the anomaly associated with the half scram configuration, degraded scram pilot SOVs contributed to the event.
Testing showed that leaking scram pilot SOVs resulted in a combined SDV air header leak of 25 scfm.
The licensee found widespread wear, aging, and hardening of the 50Vs' 0-rings and diaphragms.
Maintenance records showed that some of the worst leaking valves had been rebuilt during the previous refueling outage.
After a reactor scram, the SDV and the scram instrument volume are in direct cor. tact with hot pressurized reactor water.
A common-mode failure of the pilot SOVs controlling the scram discharge system vent or the drain valves could result in an uncontrolled release of reactor water outside primary containment (see Figure 7) until the scram is reset.
Such an event occurred at Hatch 2 in August 1982 (Ref. 49).
Similarly, a sluggish SOV piloting an SDV drain valve caused a water hammer at Brunswick I which resulted in damaged pipe supports in the SDV drain system (Refs. 50, 51).
As noted in Reference 46, a severe water hammer in the SDV system could result in an uncontrolled leak of reactor water outside the primary containment, Discussion with GE** has indicated that since Information Notice 85-95 was i
issued, BWR owners have made improvements in their SDV systems so that there are redundant SDV vent and drain valves at all U.S. BWRs (vs. only one vent and l'
one drain valve per SDV header in the early 1980's).
However, it is not certain that all U.S. BWRs have manual handwheel overrides for the SDV vent and drain valves to limit reactor water leakage outside primary containment in the event of a common-mode failure of the SOVs piloting the SDV vent and drain systems.
- Channel B remained tripped because of stuck contacts on the reactor mode switch.
PRELIMINARY CASE STUDY 22 8
l e
i
)VMhhh/hhMhh//6ADHRhW//hh////HD/h//////h///4 W '///H/h///////H///////////////////6 NO CON 1RUL
/
AIR SU5Tt.Y j
"U M
m f
I, r-
/
REACTOR DRYWitt Y
Y VESSEL
=
'/
TORUS g g g g g, q ppg pq3 g-t (PHIMARY 1
z-CON T AtNME N D 120 VAC I
t y
/
jf T svMPS RfACTOR CHANNf L A N
4 RPS ~A" 4
y["
5 9
CORE l
2'
-=
Q
[
\\
,,ih,,
s AM g_]y,_____;
g} i ,, ) oy (A VALVES f. OOO in -4 f(' l RPS~B" l' t t C i h i Cp CONTROt ROD Scy:Au t SUBCHANNEL RELAYS DRIVf 5 ,{ ,[ 1 ,g, NOT It ygtygg, i t t 14 TYP. OF 1856 s-" ""-"/ g --{3 ^^ H-; -- 'q p t i , 7 _ h 0:0 000 N F .i- -.4 ri u =i SCPAM W VALVE SCRAM p , t g. / = mm p/ ^- VALVE k y N+t SCnAM Q i t 7-- g vatyg alh# 9 "ty X j g u N +7 CHANNEL 9 [ 3 Rr5 RuS r E p 120 VAC s A SDV Srstam Vene Va#wes ro 9 A to SCRAM instr. [ RE ACTOR RtMt DfNG Vcheme - _ _ g,~i a -J _4. _ Q,f. ::r =3__F- =-- __ C. - r - (SECONDARY \\ '-.3_ __2 -_.- : CON TAINMEN T) SOV System l man Vatwe \\ SCRAM OtSCHARGE i VOLUME HEADER /, To Clean Radovas e Open Onty Raetuveste j r r w O,- r o.a.n To cw aa* 0 __. Dw. -. 1 d e l V/####/#/3 Y/#/#/###/##/X flllllll"'.'"':llll... :..... :.'.'"'.'."MA VMm4 t4 A '/: R A / TORUS TORUS
- To fino, Dean CAtor Syvem r.'""
s5 >S- -CS RHR RHR HpCg RCIC antf CS i Gi [ F9 .lO [ cpl G_ -i.. ~ " - i / S / / / 11 1-11, l Figure 7 I BWR SCRAM System-Illustrating Leak Path Outside Containment t f t m
e a. 5.2.1.2 Perry - Simultaneous Common-Mode Emergency Diesel Generator Failures On February 27, 1987, the Perry Nuclear Plant experienced simultaneous common-mode failures of both emergency diesel generators (EDGs) (Ref. Sla). The failures were attributed to excessive air leakage through SOVs on each EDG's control panel. The SOVs were Humphrey Products Model No. T002El-3-10-35 which were supplied by Delaval as EDG piece parts. The SOVs are 3-ny air control valves which are continuously energized while the EDGs are in standby. The licensee had previously identified those SOVs for replacement due to observed air leakage. Work requests had been initiated for replacement of thoss SOVs but at the time of their failures, the work requests had not yet been implemented. Discussions with the licensee
- and the EDG manufacturer ** revealed the following information:
1 The failed SOVs had been in service for ovtr 10 years. 2 The analysis of the SOVs found that the elastomeric parts (Buna-N) were " dried up and cracked" 3 The failure was attributed to long-term operation at elevated temperatures 4 The Humphrey valves were purchased by Delaval as com-mercial valves and were upgraded / dedicated for nuclear service by Delaval. Delaval did not provide specific maintenance instructions for the SOVs. 5 The changeout frequency of the SOVs is not specified in the Delaval Operator's Manualj however, it could be implied from the manufacturer s control panel environmental qualification report. 6 Although the 50V manufacturer has stated that SOV failures have occurred because of incorrect use of lubricants on the Buna-N parts, the licensee was not provided with any such instructions. 7 The Perry plant upgraded the SOVs to ones with Viton instead of Buna-N; and more recently they-replaced the SOVs with electrical relays. 8 We are uncertain about the vulnerability of other nuclear power plants having Delaval EDGs with Humphrey SOVs similar to the ones that failed at the Perry plant in February 1987. 5.2.2 Replacement Versus. Rebuilding 5.2.2.1 MSIVs at Perry - Inadequate SOV Febuild After determining the cause of the MSIV failures of October 29 and Novem-ber 3, 1987, the licensee replaced or rebuilt the ASCO SOVs on the MSIV air packs. Due to the limited availability and long lead times for replacement
- Telecon H. L. Ornstein USNRC and R. DiCola, Cleveland Illuminating Co.,
May 29-30, 1990.
- Telecons H. L. Ornstein and D. Pesout and S. Owyoung, Cooper Industries (formerly Delaval) May 29-30, 1990.
PRELIMINARY CASE STUDY 24
\\ ~ i l parts (air packs and ASCO dual-coil NP8323 SOVs), the licensee had to rebuild some (rather than replace all) of the MSIV air pack SOVs. One entire air pack was replaced for the inboard D MSIV. One dual-coil NP8323 SOV was replaced for the outboard D MSIV air pack. One dual-coil ND8323 SOV was replaced for an inboard MSIV that had not failed previously. It was replaced upon inspection because it was observed to have sustained heavy damage to the electrical coils due to moisture intrusion. Five dual-coil NP8323 SOVs were rebuilt, including the inboard B MSIV which had failed on October 29, 1987. 1 The licensee conducted increased surveillance and testing of the MSIVs after repairing and replacing the air pack SOVs. The licensee initiated monthly operability testing of the MSIV air pack SOVs, quarterly fast closure timing tests, and inspections of the ASCO NP8323 dual-coil SOV experiencing the high-est temperatures. On November 29, 1987, while performing operability-testing, the ASCO dual-coil NP8323 50V, controlling the inboard B MSIV, faileii to change state when it was de-energized. Examination of the failed SOV found that the failure was i caused by several foreign particles in the SOV. Laboratory examination con-firmed that the particles were EPDM from the 50V's 0-ring which had been replaced during the 50V's rebuilding process subsequent to the November 3, 1987 failure (Refs. 8, 9). Apparently, during the original SOV rebuilding process, the licensee did not completely disassemble the ASCO dual-coil NP8323 SOV. As a result, one or more small particles remained in the valve and remained undetec?.ed until it (they) caused the 50V's failure.* To preclude additional failures due to foreign particits remaining from the rebuilding process, as had happened on November 29, 1987, the licensee re-placed all eight ASCO dual-coil NP8323 SOVs with new ones. Furthermore, the licensee stated that they were going to modify their preventive maintenance program: in the future, all Class 1E ASCO SOVs will either be replaced with new volves or undergo complete disassembly and cleanout to ensure that no particles remain or are introduced during the rebuilding process. 5.2.2.2 Brunswick 1 - Safety Relief Valves - SOV Rebuilding Ereor: Excess Loctite On July 1,1987, while attempting to control pressure follouing an unplanned automatic reactor trip, an SRV failed to open on demand. Following shut down, the licensee tested the SRVs that had r.ht cycled during the trip recovery and found another SRV that did not open on demand (Refs. 52, 53). The SRV failures were due to SOV failures. The two S0Vs that had failed (Targot Rock Model 1/2-SMS-A01) are used to port air to the SRVs' actuators,
- It is believed that one particle remained in the SOV, and that the particle l
broke up during subsequent 50V operation. l PRELIMINARY CASE STUDY 25
o. allowing remote-manual opening of the valves.
- he two SRVs that failed were part of the plant's Automatic Depressurization System (ADS).
The failure of both safety relief valves to open on demand was attributed to excess Loctite RC-620, which was found in the ir.ternals of the related SOVs. Although two additional valves were found to have excess Loctite on the 50V's internals, those valves did not exhibit signs of binding. The licensee determined, with the assistance of the SOV manufacturer, that Loctite RC-620 had been used by the 50V manufacturer's field service representa-tive while rebuilding the SOV during'a previous outage. In Reference 52, the licensee noted that the manufacturer s (Target Rock) field service representa-tive had rebuilt all of the Brunswick 1 SOVs that actuate all eleven of the plant's SRVs (seven ADS valves and four non-ADS valves). The licensee stated that the Target Rock field service representative had done SOV refurbishment work on the valves at Brunswick 1, but he had not done similar work on any SOVs which pilot SRVs at other plants. Target Rock field representatives service the SRVs for all U.S. BWRs (except for Browns Ferry 1, 2, and 3) at Wyle Laboratories during the plants' refueling outages. Most plants send their SRVs and SOVs to Wyle for refurbishment every refueling outage, but some only send half of their SRVs and SOVs to Wyle for such refurbishment each refueling outage. The problem encountered with Loctite RC-620 was one of excessive applica-tion. Loctite RC-620 is an anaerobic adhesive. Curing takes place in the absence of air. The SOV manufacturer's refurbishment procedure specifies that Loctite RC-620 be applied to a locknut assembly beneath the valve plunger. The procedure cautions against application of excessive amounts of the adhesive. The licensee concluded that the SOVs had excess amounts of Loctite RC-620 applied to them, and that curing did not occur until after the valves were placed in the inerted containment. The licensee believed that, prior to curing, the excess adhesive migrated to the interior of the valves, bonding the 50Vs' plungers to the bodies of the valves. The licensee concluded that even though only two ADS SOVs were found to malfunction, two other ADS SOVs had similar bonding due to excess Loctite RC-620; however, those bonds were broken during the initial removal and handling of the SOVs when they were removed from the drywell and bench tested. l The licensee's assessment of the event (Ref. 52) concluded that a common-mode failure, the inoperability of all 11 SRVs as a result of Loctite RC-620 bonding of all SOVs by one vendor field service representative, is a reasonably credible event. The occurrence of a design basis event under such conditions is beyond the bounds of the plant's final safety analysis report. The NRC staff issued Information Notice 87-48 (Ref. 53) to notify licensees of the July 1, 1987 event. A similar SRV failure occurred on July 25, 1980 at Pilgrim (Ref. 31). A Target Rock SRV failed to open on a manual demand signal. The failure was caused by excessive Loctite RC-620, which had caustd the SRV't solenoid plunger to stick to the valve's bonnet. In this case, tha excessive Loctite was used during the fabrication of the SRV (as opposed to the July 1, 1987 event at Brunswick in which the excess Loctite was applied daring refurbishing). PRELIMINARY CASE STUDY 26 1
.e 5.2.2.3 Peach Bottom 3 - Scram System - SOV Rebuilding Error: Excess Loctite On November 17, 1983, a control rod was observed to have an excessive insertion time during a reactor scram (Refs 54, 55). The sluggish control rod insertion was attributed to the failure of an SOV* to shift position to allow control air to be exhsusted from the control rod's hydraulic control unit. As a result, the licensee replaced the scram pilot SOVs associated with the control rod that did not scram promptly and sent the scram pilot SOVs to GE for failure analyses. On January 14, 1984, during a reactor scram, another control rod did not insert within the technical specification allowable time of 7 seconds. The second control rM had acted sluggishly during the November 17, 1983 scram. Howr.cr, W ause it was believed to have inserted within the technical specifi-cation allowable time on November 17, 1983, no maintenance was performed on its pilot SOVs at that time. Subsequent to the second failure (January 14, 1984), the~1icensee undertook an extensive investigation. That investigation revealed that, contrary to pre-vious-findings, the second control rod also had failed to meet its allowable scram insertion time limit on November 17, 1983. Laboratory analysis of the two pairs of SOVs associated with the slow inserting control rods revealed that one valve of each pair had a yellow varnish-like foreign substance on its core assembly. One of the SOVs which was found to have the foreign substance on it exhibited sticking during subsequent bench testing. The foreign substance was originally believed to be a silicone lubri-cant, but it was later identified to be Loctite 242. Loctite 242 had been intro-duced to the SOVs during the rebuilding process, in accordance with the supplier s (GE) recommendations. In a 1978 Service Information Letter, (SIL) 128, (Ref. 56), GE had recommended that when rebuilding CRD scram pilot valves Loctite 242 adhesive / sealant should be used to secure the " acorn nut" on the solenoid housing to prevent it from loosening. The Peach Bottom 3 failures were attributed to excess Loctite 242 which was used in the rebuilding process. It had appeared to be fully cured and the excess had not been wiped off. When the system returned to service, the Loctite i 242 migrated and hardened and bonded the 50V's core plunger to its base assembly. After determining the source of the sticking, the licensee eliminated the use of Loctite 242 from its rebuilding process. Subsequently, GE issued a supple-mentary service information letter, SIL 128 (Ref. 57) which recommended that all BWR owners discontinue using Loctite 242 or any other chemical adhesive thread lockers on the acorn nut of the pilot SOVs. GE had originally recommended using Loctite 242 to overcome loosening of the " acorn nut", and ASCO had agreed. Following the sticking problems at Peach Bottom 3, ASCO made a design change and replaced t5e acorn cut with a nylon-lined locking nut which would not require adhesive thread lockers to remain tight.**
- ASCO Model HVA-90-405, which is built by ASCO but procuimd from GE, it is similar to the ASCO Model NP8316 valve.
l
- Telephone discussion between J. Shank, ASCO, and H. L. Ornstein, USNRC, June 19, 1989.
1 PRELIMINARY CASE STUDY 27 l 9.
~ The common-mode failure potential for the scram system at some BWRs exists because some plants have used the same SOVs that are used to pilot the individ-ual Control rod hydraulic control units to pilot the scram discharge volume vent and drcin valves. In the case of Peach Bottom 3, the potential for multiple simultareous failure was compounded by the fact that the licensee had rebuilt all 370 :ontrol rod scram SOVs during the previous refueling outage. To reduce this common-mode failure potential, GE's SILs (Refs. 56, 57) recommended (not a L" ding requirement) that CRD pilot SOVs be rebuilt on a staggered basis from e "cistributed checkerboard pattern." 5.2.3 Contamination 5.2.3.1 Brunswick 2 MSIVs - Excessive Heat and Poor Air Quality (Hydrocarbons and Water) On September 27, 1985, during surveillance testing at Brunswick 2, three of the plant's eight pneumatically operated MSIVs failed to fast close (Refs. 58,59). There are two MSIVs in series in each of four parallel steam lines. Two of the valves that failed to fast close were on the same steam line. An l investigation of the failures found that the MSIVs failed to close because of i disc-to-seat sticking of the MSIV air pack SOVs (ASCO dual-coil Model NP8323). The internal 0-rings on the SOVs also were found to be degraded; they were brittle, and several 0-rings were stuck to the valve body. Several SOV discs came apart after becoming brittle: pieces of one 50V disc became wedged in the 50V's exhaust port, one disc stuck to the exhaust port, and another S0V lost a piece of its disc. i Laboratory analysis of the three failed SOVs showed the presence of a significant amount of hydrocarbon in them. The combination of hydrocarbons and elented temperature caused the EPDM discs to swell and fill the 50Vs' exhaust l ports, which blocked the discharge of air in the air actuator and increased the l frictional force opposing SOV core movement. The instrument air system was believed to have been the source of the hydrocarbon contamination. Because of the susceptibility of the SOVs' EPDM parts to hydrocarbon contamination, the licensee replaced all of the SOVs with the same model SOV having Viton discs and seals. Compared to EPDM, Viton is less susceptible to hydrocarbon contamination, but it is more susceptible to radiation damage. This event was reported to Congress as an abnormal occurrence. The abnormal occurrence report categorized the event as one which resulted in the " loss of plant capability to perform essential safety functions such that a potential release of radioactivity in excess of 10 CFR Part 100 guidelines could result from a postulated transient or accident" (Ref. 60). 5.2.3.2 North Anna 1 and 2 - Multiple Systems - Oil and Water Intrusion [ While performing maintenance operations at North Anna in the morning on April 24,1987,- an operator error resulted in a service water intrusion into the Unit 1 and 2 instrument air systems (Refs. 61-64).* The licensee quickly recognized that the service water intrusion affected SOVs and pneumatic con-tro11ers for auxiliary feedwater systems, primary and secondary pressure
- Telephone discussions between J. Lewis and J. E. Wroniewiez, Vepco, and H. L. Ornstein, USNRC, May 1989.
PRELIMINARY CASE STUDY 28
control systems, and the SOVs required for containment isolation (" trip valves") for both Units 1 and 2. At the time of the event, Unit I was in mid-loop oneration and Unit 2 was operating at 100 percent power. The licensee's immediate response to the event was to continue operating Unit 2 and to blow down thr. affected instrument air t l lines. About 2-1/2 hours after the intrusion occurr9d the licensee tested the Unit 2 "A" motor-driven AFW pump. The air-opera *.ed discharge valve and the back-pressure regulating valve for the AFW pump soth malfunctioned rendering the pump inoperable. About three hours later the licensee tested pump B satisfactorily. Throughout the evening of April 24, 1987, the licensee continued to blow L down instrument air lines until no moisture was observed. The "A" AFW pump's discharge and pressure regulating valves were repaired on the evening of April 24, 1987 and were satisfactorily tested around midnight. The cleanup procedure was not totally effective since there were low points in the instrument air system that had not or could not be drained. The residual water that remained in the low points of the instrument air system and the mois-ture and contaminants in the instrument air system resulted in widespread 50V failures for almost two years after the service water intrusion event. In addi-tion to failures of " freestanding" SOVs, there were dozens of control valve failure. The bulk of the control valves that failed were Fisher control valves. Integral to each Fisher control valve is an ASCO SOV. The Fisher control valve failures were essentially failures of the ASCO SOVs which are piece parts of the control valves. Examination of plant equipment failure records noted that between April 1987 and February 1989, there were approximately fifty Fisher control valve (ASCO SOV) failures. It appears that those failures resulted from poor quality air due to the April 24, 1987 water intrusion event and from poor maintenance of the instrument air system. In addition to these failure records, NRC inspectors noted (Ref. 62) many l ASCO SOV failures that had been observed during surveillance testing after April 24, 1987, were not reported and the SOVs were not repaired. The primary reason was that the SOVs that failed to operate during surveillance testing operated properly after being tapped (" mechanical agitation") by plant personnel. As a result of such practices, repetitive malfunctions were observed, the mal-l functioning SOVs were not fixed or replaced expeditiously, and the root causes were not found or corrected on a timely basis. Characterization cf the licen-see's in-service testing practices regarding SOVs was cited in Refetonce 61 as follows: "The process of tapping on solenoid valves and repeated cycling of valves prior to running a satisfactory surveillance was considered ar, acceptable practice by the licensee." Some of the systems that were affected by malfunctioning ASCO SOVs (freestanding or piece-parts of Fisher control valves) due to contamination of the instrument air system are listed in Table 2. PRELIMINARY CASE STUDY 29 .w
Table 2 Systems Impachd At North Anna By SOV/ Control Valve Failures Due to Service Water Intrusion / Instrument Air Contamination Unit I and Unit 2 Residual Heat Removal / Low Pressure Safety Injection Main Steam Relief (PORVs) Auxiliary Feedwater Component Cooling Water Unit 2 only Containment Isolation Containment Fan Conling Main Steam Isolation In a Fahruary 10, 1988 memorandum, the Chairman of North Anna Station's Nuclear Safety and Operating Committee stated that successful stroking of the SOVs is an appropriate corrective action to remove contaminants, because cycling the affected valves blows the contamination fror, the lines and returns the SOVs to operable status" (hefs. 65, 66). North Anna Station's approach to maintenance of malfunctioning SOVs contradicts the valve manufacturer's recommendations. ASCO's installation and maintenance instructions and the licensee's telephone discussions with ASCO or, r bruary 4 and 5,1988 advised the licensee that, after e SOV contamination, the NF Series SOVs should be inspected for corrosion, sediment or other contaminants, ano cleaned accordingly.* A meeting was held at NRC Region II offices on February 7,1989 to discuss repetitive AFW system control valve failures which occurred in January 1989, due to moisture in the instrument air system (Ref. 67). At the meeting, the licensee acknowledged that widespread failures of SOVs, control valves and air-operated valves had occurred during the 21 months from the time of the service water intrusion into the instrument air system (April 1987 through January 1989). A large number of repetitive SOV and control valve failures were attributed to poor quality instrument air (oil and moisture contamination in addition to the April 1987 service water intrusion). The licensee noted that attention had been f') cussed on the quantity of instrument air available without paying attention to iv.s quality and indicated that subsequent to a review of their instrumerit air system, a program was initiated to clean or replace the affected equipment. The lic9nsee also provided information on steps that were being taken to improve the inst.ument air system to assure delivery of clean, dry, oil free instrument air. We view the April 24, 1987 service water intrusion into the instrument air system as a significant precu. = r event. It resulted in widespread degra-dation of SOVs, controllers, and air-operated valves that had the potential for disabling many systems needed to achieve safe shutdown. If a design-basis event
- Telephone discussions between F. Maiden and W. Murray, Vepco, and K. Thomas, ASCO, February 4 and 5, 1988.
1 PRELIMINARY CASE STUDY 30 l
had occurred at Unit 2 on April 24, 1987. before removing the service water from the instrument air system, the operators' ability to bring the plant to a safe shutdown could have been seriously impaired. A large number of SOV and control valve failures occurred at Units 1 and 2 between April 24, 1987 and January 1989 as a result of water, corrosion products, and residue from the service water intrusion, and from impurities introduced by poor quality instrument cir. This event exemplifies the necessity for providing SOVs with clean, dry, oil free air, and the need to thoroughly clean and inspect the equipment if water or other contaminant intrusions occur. 5.2.3.3 Susquehanna 1 and 2 - Scram System: Oil and Water Contamination The Susquehanna plants have experienced common-mode failures of SOVs that resulted in multiple failures of control rods to insert, slow insertion of multi-pie control rods, and repetitive failures of scram discharge volume vent and drain valves. The 50V failures were linked to contaminants in the instrument air system (i.e., hydrocarbons, water, and particulates) and high temperatures.* Because both Susquehanna units share a common instrument air supply, the common-mode failure potential that existed for both unit 1 and unit 2 scram pilot SOVs also existed for the SOVs that actuate both units' backup scram valves. The backup scram valves are intended to provide diverse scram capability to protect against common-mode failures. Although Unit 1 experienced the failures, the potential for such failures also existed at Unit 2; both units' scram and diverse scram systems were vulnerable. The Susquehanna SOV failures illustrate the potential for multi-plant common-mode failures leading to events that are beyond the plant safety analyses (i.e. failure of multiple control rods to insert and unisolated primary leak outside containment via the scram discharge volume). A summary of the Susquehanna SOV failures are described below: On October 6, 1984, while Susquehanna 1 was operating at 60 percent power, two control rods failed to insert during individual rod scram testing. Further scram testing revealed that a total of four rods would not insert and nine additional rodt hesitated before inserting. A similar event occurred previ-ously at Susquehanna on June 13, 1984, when several control rods hesitated momentarily before inserting (Ref. 68). Two of the control rods that failed to insert on October 6 had not met the plant Technical Specifications scram time requirer.ents on June 13. The licensee did not become aware of the June 13 mal-functions until the October 6 failures were investigated. The October 6 failures were attributed to common-mode contamination of the instrument air system. The combination of contaminants (oil and/or moisture) and high temperatures (140*F) caused the SOV internals to degrade and become stuck. The 50V polyurethane disc holder subassembly seats were found to be stuck to the SOV exhaust port orifice. This prevented air from the scram inlet and outlet valve operators from bleeding off through the SOV exhaust ports, which prevented the scram inlet and outlet valves from opening.
- At Susquehanna each of the 185 control rods is piloted by one ASCO HV-176-816 SOV. Many other BWRs' control rods are piloted by other model ASCO SOVs, but two per control rod.
The ASCO SOVs used in U.S. BWR scram systems are typically procured from GE. PRELIMINARY CASE STUDY 31
Two independent laboratories examined the failed SOVs and concluded that l the polyurethane parts degraded because of a combination of contamination in the instrument air and elevated temperature (Ref. 69). The first laboratory (Franklin Institute) cited the failure mechanism as hydrolytic decomposition of the polyurethane seats due to a combination of moisture and elevated tempera-tures. The second laboratory (GE) indicated that polyurethane seat failure was caused by contaminstion of the instrument air with a synthetic diester oil (SDO, which is a plasticizer). Both Franklin Institute and GE recommended replacing the polyurethane seats with a seat material capable of operating at higher temperatures and having an improved resistance to contaminants. The recommended material was Viton. The licensee replaced all of the 50V polyurethane seats on Units 1 and 2 control rods and all the backup scram valves. About half of the 50V discs for the Unit 2 control rods had already been replaced in 1983 with Viton discs. The licensee's investigation found that the 50V for the scram discharge volume vent and drain valves on Unit I had a polyurethane disc that also was susceptible to the same type of failure. The SOVs for the vent and drain valves also were replaced with different SOVs having Viton discs.* The October 6, 1984 scram system degradation at Susquehanna was reported to Congress as an abnormal occurrence (Ref. 70). The NRC staff concluded that the event involved a " major degradation of essential safety-related equipment," and demonstrated the plant's susceptibility to common-mode failure. The failure caused a reduction in "the required ' extremely high probability' of shutting down the reactor in the event of an anticipated operational occurrence" (Ref. 70). Another scram discharge volume (SDV) system component failure attributed to contaminated air occurred at Susquehanna 1 on December 21, 1984 (Ref 71). Dur-ing surveillance testing, an SOV that controls the SDV vent and drain line isolation valves malfunctioned as a result of particulate matter that was lodged between the 50V's disc and seat. As a result, the SDV vent and drain valves were stuck open. Since the reactor was at power, if the SOV had failed to completely close after a scram, the potential for an unisolated primary leak outside containment would have significantly increased. 5.2.4 Lubrication 5.2.4.1 Multiple Plants - Manufacturing Error: Residue-Producing Lubricant The Kewaunee nuclear power plant experienced three SOV failures on May 28, 1988 during surveillance testing (Ref. 72). Two of the SOVs were redundant containment isolation valves piloting the reactor coolant drain tank discharge reader isolation valves. The third SOV.that failed served as the pilot for the pressurizer relief tank makeup isolation valve. All three failed SOVs were nu: lear qualified ASCO NP8314 DC valves that piloted air-operated valves. The were normally open, normally energized, and were designed to close (fail safe)y on loss of instrument air or electrical power. The failures of the SOVs to
- The SOV chosen was a larger size, made by another manufacturer.
The original Unit 1 valve was undersized and the replacement made was the same as the one on Unit 2. PRELIMINARY CASE STUDY 32
o .o shift position upon de-energization were attributed to an amber-colored residue inside the 50Vs. The residue was found at the location where the SOV core as-sembly (plug) contacts the 50V body (solenoid base sub-assembly) see Figure 4. The failed SOVs had been in service about 18 months prior to their failure. The local ambient temperature was about 110*F. The licensee inspected two other ASCO NP8314 SOVs from the same manufacturing lot which were installed adjacent to the three SOVs that had failed. They had been installed at the same time as the ones that failed, but were operated in the de-energized mode. The de-energized SOVs had performed satisfactorily. The licensee assisted by two independent laboratories (Wyle Laboratories and Akron Rubber Development Laboratory) and ASCO conducted an extensive inves-tigation to determine the root cause of the failures. On the basis of the investigation, the licensee and ASCO concluded that the 50V failures were most likely caused by the degradation of a lubricant (International Products Corporation, "P-80" rubber lubricant) which had been introduced during the manufacturing process. P-80 is a water-based rubber lubricant used by ASCO personnel to facilitate 50V assembly. Although P-80 was an approved lubricant for use at ASCO's manufacturing facility, its use for the assembly of the NP8314 SOVs was not an explicitly approved procedure. P-80 product literature states that it provides " temporary slip'periness" for assembling rubber parts, and that it is absorbed into the rubber leaving no residue or harmful effect on the rub-ber." Subsequent to SOV assembly (using the P-80 lubricant), the SOVs were cleaned however, minute amounts of the P-80 lubricant remained within the inter-nal cavities of the 50V. From the laboratory results, it was concluded that the small amount of lubricant, remaining in the SOVs, migrated subsequent to ener-gization, and the heating, due to energization, degraded the P-80 to form the amber-colored sticky residue which caused the 50V malfunctions. The investiga-tion discounted Dow Corning 550 lubricant as the source of the residue that had been found inside the NP8314 SOVs. ASCO has discontinued using P-80 in the assembly of SOVs as a result of the investigation. On October 18, 1988, ASCO issued a 10 CFR Part 21 notification regarding the potential failures of NP8314 SOVs (Ref. 73). The notification accounted for 231 suspect SOVs that were sent to 17 U.S. LWRs, 76 suspect SOVs that were sent to sup)11ers who most likely shipped them to unspecified plants as piece-parts of otler equipment between 1981 and 1988, and 9 suspect SOVs that were sent to Franklin Research Center (FRC) in 1986. The Fort Calhoun plant had received the largest number of suspect SOVs (79) in 198L Several of those SOVs failed at Fort Calhoun in 1981 and 1982. Three of the SOVs that failed at Fort Calhoun were returned to ASCO for investigation. ASCO's investigation of those valves, incident report IR 3604 - May 1982 (see NRC Vendor Inspection Report 99900369/88-01 (Ref. 74), noted that the failures were due to sticking caused by a varnish-like residue. At that time, neither ASCO nor the Fort-Calhoun licensee were able to identify the source of the " acrylate ester residue found on the plunger and sub-base assembly" of the energized NP8314 SOVs. Fort Calhoun experienced a similar failure of another energized NP8314 SOV in March 1982. It was cleaned and returned to service (Ref. 75). The licensee stated that it would replace the internals of all the NP8314 SOVs using new spare parts kits. Subsequently, Fort Calhoun donated 10 ASCO NP8314 S0Vs that had been in continuously energized service for 18 months to FRC for use in an PRELIMINARY CASE STUDY 33 q
+ o. NRC sponsored SOV aging research program (Ref. 71). FRC also purchased nine new NP8314 SOVs from ASCO, which were shipped in April 1986, to be used in FRC's SOV aging program (those SOVs were also listed in ASCO's 10 CFR Part 21 notifica-tion). Six of FRC's purchased SOVs, which were under aging, failed prematurely (failure to shitt position) going accelerated thermal as a result of organic deposits (" sticky substance"). After the deposits were " cleaned away" with acetone and the SOVs were reassembled, they performed successfully for the dura-tion of FRC's testing program. FRC's report (Ref. 76) also noted that organic deposits were found in the NP8314 SOVs received from Fort Calhoun. FRC believed that the sticky deposits that had prevented the SOVs from functioning were due however,ganic compound that was introduced during the assembly of the valves; to an or a detailed analysis and final determination of the source of the deposits were not pursued by FRC because of budgetary restraints of the program. In the course of the FRC's 50V aging research program, ASCO had been apprised of the sticking problem, however ASCO did not find the source of the residue (P-80) until after the Kewaunee failures in 1988. The failures of the NP8314 SOVs indicate that P-80 was used to assemble the NP8314 SOVs as early as 1981 and as late as 1988. A similar case, in which another SOV manufacturer used a lubricant to assist with SOV assembly, also resulted in subsequent SOV performance problems. As noted in Reference 77, Target Rock Corporation used castor oil as a lubricant to facilitate the assembly of its two stage safety relief valves (SRVs). After investigating several SRV failures, it was found that castor oil, which was used to lubricate silicone rubber 0-rings, caused swelling and accelerated degrada-tion of the 0-rings. Subsequently, Target Rock discontinued using castor oil as a lubricant. DAG-156 lubricant (carbon particles suspended in an alcohol base) was used to replace castor oil. We are not aware of any subsequent Target Rock SRV failures that have resulted from the use of DAG-156. Target Rock informed the author of this case study during a visit to their facility {mineraloils"tofacilitate50Vassembly. November 1988) that, paralleling the use had used This practice was dis-m tinued in the mid-1980s and DAG-156 was chosen as a replacement for mineral
- 5. 2. k Catawba:
Poor Quality Air and Lubrication with Vaseline The Catawba nuclear power plant experienced common-mode failures of EDG starting air system inlet valves (Refs. 78,79,80). The EDGs were manufactured by Delaval. The air start system inlet valves, model T-3618, were made by California Controls Co. (Calcon). These two-stage air-operated valves each have a solenoid pilot valve that is normally closed and requires de power to actuate the soleMid pilot to admit starting air into the EDG. The licensee has reported five instances of common-mode failure of these valves. The valves stuct open when a sticky, slimy substance formed inside the poppet portion of the valve. The licensee determined that the substance was the silicone lubricant Dow Corning 111 that was used on the valves. On five occa-sions the licensee cleaned the valves and replaced the Dow Corning 111 with Vasellne petroleum jelly. Calcon's recommended lubricant is GE Silicone fluid G-322-L, which is significantly different from DOW Corning 111. The licensee TRELIMINARY CASE STUDY 34 1_
did not check for the compatibility of Vaseline petroleum jelly with the Buna-N rubber used in the Calcon Valve. Low nitrile Buna-N rubber degrades when in contact with petroleum based products. After reviewing the EDG air start valve failures and other EDG pneumatic equipment failures (Calcon pressure sensors) the licensee concluded that the sticking was caused by moisture interacting with the Dow Corning 111 silicon lubricant. The source of the moisture was the starting' air system, the root cause was inadequate dryer maintenance (the licensee s failure to changeout the spent desiccant). Subsequently, the licensee upgraded its maintenance on the air dryers, thereby lowering the starting air moisture content. In addition, the licensee cleanedthevalvesandreplacedtheVaselinepetroleumjellywithDowCorning 111 lubricant. These actions in conjunction with more frequent changeout of the Calcon gas valve's elastomeric parts in accordance with the Delaval owners' grou? plant specific recommendations appear to have eliminated the valve sticdng problem. 5.2.4.3 Common-Mode Failure of 16 MSIVs at a Two Unit Station (BWRs): Incorrect Lubrication In July 1986, the licensee of a two-unit station reported excessive stroke time of the Unit 1 "C" outboard MSIV which resulted from a failure of an Auto-matic Valve Corporation (AVC) SOV (model C4988-8). The failure was attributed to " poor workmanship from the factory" and " improper lubrication, which would allow the valve piston to jam at a certain place in the valve." The failed AVC valve was replaced with a new one. Five months later (December 1986 thelicenseofoundthettheUnit2"B},inboardMSIVdidnotstrokeproperlyaswhile performing mo a result of a failurt of another AVC SOV. The licensee shut down both units from 100 percent power and inspected the SOVs piloting all 16 MSIVs. The licensee found that the AVC SOVs on all 16 MSIVs were damaged. The three-way and four-way valves and solenoid pilot valves on all 16 MSIVs had a hardened, sticky lubricant in their ports and on their 0 rings. As a result, motion of all the SOVs was imoaired, resulting in instrument air leakage and the inability to operate all of tie MSIVs satisfactorily. The licensee also examined unused spares in the warehouse and found that the lubricant had dried out in those valves, leaving a residue. Several of the warehoused spares were bench tested. They were found to be degraded and they also leaked. The original " approved" or " preferred" S0V lubricant (based upon equipment qualification testing) was Parker Super-0-Lube. However, later equipment quali-fication testing (1985) found that the Parker Super-0-lube could cause SOVs in the MSIV air pack to malfunction. The Parker Super-0-Lube was found to break down to an adhesive, powdery substance when exposed to radiation fields greater than 1x10E6 RAD. Because of the potential for breakdown of Parker Super-0-lube and binding of the S0Vs in the air packs, the licensee changed the SOV lubricant to E. F. Houghton SAFE 620. In separate telephone conversations the SOV manufacturer (AVC) told the AE00 staff that it had informed the utility that E. F. Houghton SAFE 620 PRELIMINARY CASE STUDY 35 i
c lubricant attacks and degrades the aluminum in the AVC valves.* Nonetheless, in accordance with utility purchase orders, AVC shipped SOVs lubricated with E. F. Houghton SAFE 620 to two different utilities. After the multiple failures occurred in December 1986, General Electric (GE) informed the licensee that the Parker Super-0-Lube is an acceptable lubricant 6 "if it is applied in a ' thin film'." AVC and GE had concluded that the problem experienced with Parker Super-0-lube in the 1985 qus11fication testing was due to " excess lubricant." On December 19, 1986, AVC issued a 10 CFR Part 21 notification (Ref. 81). The notification indicated that Commonwealth Edison had also purchased AVC valves lubricated with E.F. Houghton SAFE 620. Commonwealth Edison told AEOD staff ** that the AVC valves which contained E. F. Houghton 620 lubricant were replace-ments for older model AVC SOVs which had been discontinued. Before being noti-fied by AVC of the problem with E. F. Houghton SAFE 620 and before installing the valves, Commonwealth Edison replaced the SAFE 620 with Dow Corning Molykote 55M. The licensee had recognized that Parker Super-0-lube was the lubricant that had been used in earlier equipment qualification testing, and SAFE 620 was probably not an acceptable replacement. Justification for the use of Molykote SSM instead of Super-0-lube was based upon the licensee's engineering analysis that indicated the similarities between Molykote 55M and Super-0-lube. In retrospect, a detailed examination of these two lubricants reveals they may have ver and, under similar operating conditions,y different high-temperature behavior the Holykote 55M would be more suscepti-ble to dryou. *** Because of these differences, it is not clear that Holykote 55M is an acceptable " qualified" replacement for the Super-0-lube. With regard to problems of excessive lubricant and the application of " thin films" of lubricant, it is interesting to note that a Commonwealth Edison plant had sticking problems with a similar AVC SOV several years earlier. In that case, the sticking was attributed to not having enough lubricant applied to the AVC valve. L 5.2.4.4 Grand Gulf _1, LaSalle 1, and River Bend MSIVs - Sticking SOVs - Foreign Unidentified Sticky Substance (FUSS) - Lubricant Suspected Between February 1985 and December 1989, the Grand Gulf 1, LaSalle 1 and River Bend nuclear power plants experienced sticking of ASCO dual-coil 8323 S0Vs l l in the MSIV air packs (Refs. 8, 82 to 88). The SOV malfunctions were attributed to a sticky substance at the contact point of the plug nut / core assembly inter-face (see Figure 1). The 50V malfunctions impaired or prevented the MSIVs from closing within the times specified in the plant safety analyses. l
- Telephone discussions between T. Hutchins, Automatic Valve Corporation (AVC) l and USNRC (S. Israel - October 14, 1988 and H. L. Ornstein - April 12, 1989).
- Telephone discussion between M. Sievert, Commonwealth Edison Company, and H. L. Ornstein, USNRC, April 12, 1989.
- Super-0-lube consists of high molecular weight silicones whereas Molykote SSM is a lighter weight methyl silicone oil thickened with lithium soap having a l
- Telephone discussion between M. Sievert, Commonwealth Edison Company, and H. L. Ornstein, USNRC, April 12, 1989.
lower dropping point than Super-0-lube (where dropping point is an indication of the temperature limit at which the lubricant dries out). PRELIMINARY CASE STUDY 36
. c In the case of LaSalle, it was demonstrated that the cohesive / adhesive force caused by the foreign sticky substance between the plug nut and the core assembly of an ASCO dual-coil NP8323 SOV was significant and could have been the cause of its failure. After the core assembly was held vertically, the plug nut was pressed against the core assembly, and then the plug nut let go, the adhesive forces from the foreign substance between the two surfaces were able to support l the weight of the plug nut to prevent it from falling.* Because the licensee suspected the Dow Corning 550 lubricant (applied to the SOVs internals at the factory) to be the cause of the sticking, the licensee considered removing the factory installed lubricant from the 8 new NP8323 50Vs that were installed after the December 16, 1987 failure. In consideration of ASCO's concern that, without the internal lubricant, ac powered SOVs could suffer fretting damage, the licensee installed the 8 new NP8323-Viton SOVs as they were received from the manufacturer (without removing the lubricant). Those 8 replacement SOVs have operated successfully through 1989.** Subsequent to the September 30, 1988 failures of two ASCO dual-coil NP8323 SOVs at River Bend, the licensee replaced all 8 dual-coil NP8323 SOVs with new ones. However, prior to installing the new SOVs, the licensee removed the fac-tory coated lubricant (Dow-Corning 550) from their internal metallic parts. On December 1,1989 two of those replacement SOVs failed due to sticking. The licensee attributed the sticking to FUSS which was believed (but not confirmed by laboratory analysis) to be Dow Corning 550 lubricant. In following up the December 1, 1989 failures, the licensee reviewed the procedures which were used in September 1988 to remove the factory applied lub-ricant. The licensee's review of those procedures indicated that although the Dow Corning 550 lubricant was removed from the internal metallic parts of the SOVs, the cleaning and reassembly procedures included a step in which the elastomeric parts of the SOVs were relubricated with the same Dow Corning 550 lubricant. Because there was more FUSS on the cleaned SOVs that failed in December 1989 than on the factory assembled SOVs that had failed September 1988, the licensee believed that the root cause of the December 1989 failures was the licensee's reapplication of excessive lubricant during the 50V cleaning and reassembly process. Subsequent to the December 1, 1989 failures the licensee's corrective action was to replace all eight NP8323 dual-coil SOVs with new ones -- after removing all the factory applied lubricant from them, without reitibricating the l elastomeric parts. Table 3 sumarizes events where MSIV air pack SOVs have stuck at Grand Gulf, LaSalle, and River Bend.
- According to ASCO, the plug nut weighs about one ounce while the spring force is about two pounds.
ASCO indicated that after a similar NP8323 S0V failure at WNP 2, the licensee had performed a similar demonstration. The sticky sub-stance at WNP2 was believed to be from excess lubricant (Dow Corning S50) that i had been applied by the licensee when the SOVs were rebuilt.
- Telephone discussion between R. Lanksbury (USNRC Sr. Resident Inspector at LaSalle Station) and H. L. Ornstein (USNRC), December 22, 1989.
PRELIMINARY CASE STUDY 37
u.
- g Table 3 MSIV Air Pack SOV Failures (ASCO Dual-Coil 8323)
P 5 Other SOVs Having g Description Number of Foreign Unidenti-Q Plant / Event of 50V and Stuck SOVs fied Sticky i Date Corrective Action and Location Substance (FUSS) Comments n R Grand Gulf 1 ASCO HTX8323* (Viton). 2 outboard All others (5) In subsequent testing at ASCO m 2/10/85 Replaced all 8 SOVs with lines (A and only 1 of 4 additional valves g ASCO NP8323 (having EPDM C) 1 inboard malfunctioned (leakage). parts). See Section (D line) However the failure of the 5.2.1.1 for a discussion outboard C-line 50V was j of the subsequent fail-attributed to FUSS at the I ures of the replacement plug nut / core assembly l valves caused by thermal interface. aging from self-heating (August 1989) LaSalle 1 ASCO NP8323 (Viton). 1 outboard All others (7) 3 of the valves that did not 12/16/87 Replaced all 8 SOVs (C-line) fail in the plant, failed .m with like. during subsequent testing at ASCO, due to presence of FUSS at the plug nut / core assembly interface. River Bend ASCO NP8323 (EPDM). Re-2 inboard One unfailed Not all SOVs have been 9/30/88 placed all 8 SOVs with lines (B and irhoard 50V inspected. Some are being like - attempted to re-C) (1 in-inspected was held for archival purposes. move the factory coated spected, FUSS found to have lubricant (Dow Corning. found) FUSS. l 550) from SOVs, but ap-Two outboard Two outboard SOVS were in-plied excessive amount SOVs inspected spected at ASCO. The coil of lubricant to 0 rings found to have enclosures of both SUVs had while reassembling FUSS.** had evidence of moisture caused 2 subsequent intrusion, indicative of failures (December 1989) localized steam heating.** 4
- ASCO HTX 8323 is not a nuclear qualified 50V, it is a non qualified commercial valve similar to the NP8323.
- Telephone discussion between J. Shank, ASCO, and H. L. Ornstein, USNRC, May 8, 1989.
r )
e A Table 3 MSIV Air Pack 50V Failures (ASCO Dual-Coil 8323) (continued) P, i5 Other SOVs Having i Description Number of Foreign Unidenti-Q Plant / Event of 50V and Stuck SOVs fied Sticky Date Corrective Action and Location Substance (FUSS) Comments n 3; River Bend ASCO NP8323 (EPDM) 2 outboard 1 other 50V was Licensee believes FUSS from m 12/1/89 Replaced all NP8323's lines (A and inspected (in-was from excessive application 8 with new ones--but D), FUSS found board), It also of Dow Corning 550 which was removed factory or both had FUSS, but used by the licensee when installed lubricant less than what lubricating the 0-rings sub-from all internal was found on the sequent to removing the Dow parts of the SOVs. failed outboards Corning 550 from the 50Vs' internal metallic parts subsequent to the 9/30/88 failures.* l l
- Telephone discussion between V. Bacanskas, River Bend, and H. L. Ornstein, USNRC, December 12, 1989.
1 i
The inspection of the SOVs on the inboard and outboard MSIV air packs at all three plants indicated that in almost every case the SOVs, which had not failed, were degraded in a manner similar to the failed SOVs, but to a lesser degree. In each case, the licensee recognized the common-mode failure potential for compromising fast closure of inboard and outboard MSIVs on one or more steamlines and replaced all the 8323 SOVs on the inboard and outboard MSIV air packs. The valve manufacturer and several laboratories conducted extensive inspec-tions and tests on the 8323 SOVs which had been replaced. There are no simple explanations for these failures individually or as a group. The source (s) of the sticky substance (s) which resulted in multiple 50V failures is uncertain. There is major disagreement between the utilities, the 50V manufacturer, the reactor manufacturer and the laboratories regarding the root causes of the failures. Internal SOV lubrication (by the manufacturer and in one case by the licensee), and poor air quality are primary suspects.* 5.3 Surveillance Testina 5.3.1 Control Rod Timing Tests - Failed Scram Pilot SOVs - Perry On July 22 1989, during scram time testing, plant personnel observed two control rods falled to meet their scram time testing requirements on initial attempts -- however, when retested the rods operated satisfactorily. As a result, both control rods and their SOVs were declared to be operable. Subse-quently, on November 25, 1989, one of those rods failed its timing test twice but was retesteo satisfactorily twice. As a result, it was declared operable. When the second c9ntrol rod that had also failed twice on July 22, 1989, was retested on Novembtr 25, 1989, and failed, it was declared inoperable. At that time, the licent::s conducted an investigation to determine the root cause of the i test failures (Refs. 89,90,91) The licensee's root cause analysis found that a manufacturing error had been made at ASCO (failure to upgrade polyurethane seats of the scram pilot SOVs with vitc ' and that the Perry Plant had not responded to a product recall l notice th.,t ASCO had sent them (Ref. 91). It is significant that the licensee's surveillance testing program did not provide guidance to the plant staff regarding actions to be taken when unsatisfactory test results are encountered. 5.4 Use of Non-Qualified SOVs 5.4.1 Colt /Fairbanks-Morse EDGs: Repetitive Air Start Valve Failures One plant, having Colt /Fairbanks-Morse EDGs, experienced six air start SOV l failures during an 8-year period. There were five failures of one valve and one failure of an identical, redundant SOV. The SOVs were commercial grade valves, model X833-134, made by ASCO. The failures occurred from February 1, 1980,
- Failures of ASCO NP8314 SOVs which are geometrically similar to the 8323 SOVs have been traced to an assembly error during manufacture.
Conceivably, a similar error may have been introduced during the assembly of the 8323 SOVs (see Section 5.2.4.1). PRELIMINARY CASE STUDY 40
.o through March 28, 1988, and in each case the failures involved excessive air leakage. Four of the five failures of the same valve (DA-198) were attributed to the SOV core and spring assembly. The first failure was attribe+ed to wear of the core and spring assembly caused by excessive heat from the solenoid being constantly energized. The SOV was rebuilt (core and spring were replaced). The SOV's second failure was attributed to " wear of the core and spring assem-bly." The 50V was rebuilt again (core and spring assembly were replaced). The third malfunction of the same SOV occurred while attempting to start the diesel. The failure was attributed to " misalignment of soler. id header due to previous repairs." The licensee's corrective action was to realign the solenoid header. Three months later the same SOV was again found to be leaking air. This fourth failure was attributed to " wear of the core and spring assembly." The SOV was rebuilt again (core and spring assembly were replaced). Five months later a redundant air start SOV (DA-238) on the same diesel was found to be leaking air. It was rebuilt (spring and core were replaced). On March 28, 1988 the same SOV that had failed four times before (DA-19B) failed again. The fifth failure was attributed to a worn seat that resulted in air leakage. The valve was replaced rather that being rebuilt. We are unaware of any subsequent failure of this replaced SOV. Discussions with the licensee who's EDGs experienced these six failures, and other licensees with Colt /Fairbanks-Morse EDGs indicated that they have received little, if any guidance from the EDG supplier about preventive mainte-nance or replacement of the air start system SOVs. The SOVs that are used for the Colt /Fairbanks-Morse EDGs are commercial grade ASCOs which are supplied with limited maintenance or service life information. PRELIMINARY CASE STUDY 41
6-ANALYSIS AND EVALUATION OF OPERATIONAL EXPERIENCE l l6.1< Common-ModeFailures I Examination of the events discussed in Chapter 5 and many of the SOV i failures included in Appendix A clearly indicate a potential exists for comon-mode 50V failures that coulu compromise multiple trains of diverse safety systems. Such common-mode failures are not considered in plant safety analyses. It is not practical to perform safety analyses for all combinations of common-mode SOV failures. However, it is feasible to take actions to minimize the likelihood for encountering common-mode SOV failures that could affect safety systems. Chapter 9 presents recommendations that can be effectively used to minimize the potential for common-mode SOV failures affecting safety systems. The root causes of many common-mode SOV failures that have been observed thus far are given oelow. (1) Design / Application Deficiencies ? 1 Incorrect specification of operating parameters such as M0PD (e.g., Section 5.1.3.1) and valve orientation (e.g., Section 5.1.4.1); incorrect material selection such as incompatibility between SOV internal parts and fluids in contact with the SOV (e.g., Section 5.2.3.3);- incorrect specification of ambient (non-accident) conditions.(i.e., temperatures, radiation, and moisture) (e.g., Sections 5.1.1.2, 5.1.1.3); _ incorrect assessment of the life shortening effects of coil heating (e.g., Sections 5.1.E.1, 5.1.2.2). (2) Inadequate Maintenance Failure to replace or rebuild limited life piece parts of the SOVs (e.g., gaskets, seals, diaphragms, springs, and coils) on a timely basis (e.g.,' Sections 5.2.1.1, 5.2.1.2); failure to rebuila SOVs correctly (e.g., Section 5.2.2.1); failure to maintain clean, dry instrument air. Contaminants have caused long-term common-mode SOV degradation and failure (e.g., Sections 5.2.3.1,5.2.3.2); excessive lubrication of SOV internals have contributed to S0V failures -(e.g., Section 5.2.4.3). (3) Installation Errors Incorrect o-Jtation (backwards, upside-down) installation at an angle not in accordance with SOV qualification testing (e.g., Section 5.1.4.1, Appendix A); PRELIMINARY CASE STUDY 42 l 1
incorrect electric current-(dc vs. ac) (e.g., Appendix A); inadequate terminal or junction box connections as a result of inade-quate manufacturer's guidance or architect engineer's interpretation of manufacturer's guidance (e.g., Appendix A). (4) Manufactbrina Defects Lubrication errors (e.g., Section 5.2.4.1); defective materials - body, plug, springs, elastomers (e.g., Ref. 74); tolerance / assembly errors such as incorrect spring size or stiffness (e.g., Ref. 74, Appendix A); faulty wiring / coil defects (e.g., Appendix A). 6.2 SOV Failure Rates It is difficult to accurately quantify SOV fai b ro rates due to the following reasons: (1) Not all SOV failures are documentid. In many cases SOVs are viewed as-expendable items, and their failu es are simply viewed as end of life, and replacements are installed without any failure reports. (2) Many SOV failures not associated with reactor trips or complete train fail-ures of safety systems are not reported in the LER data base. (3) Many S0Vs that are subcomponents or piece-parts of other larger components or systems are not always reported as S0V failures in the' nuclear. plant reliability data system (NPRDS) for example, MSIVs, flow regulators, gov-ernors that fail to function properly because the related= S0Vs have failed are unlikely-to be reported as 50V failures. Hence, an= accurate estimate of SOV failure rates from NPRDS is not achievable. Coupling the difficulties of obtaining accurate S0V failure counts with the difficulty of accurately assessing the number of successful SOV challenges or surveillance tests can, at best, lead to a crude estimate of 50V failure rates. Nonetheless, Table 4 lists S0V failure rates from several sources, including the results of this study's query of the NPRDS data'for failures which occurred from January.1, 1985 through December 31, 1988. It is significant that assuming quarterly testing of SOVs, NPRDS data, for the~ years 1985 through 1988, indicate failure rates of 7 to 9.5 times higher than the estimates used in WASH 1400 and in the NUREG 1150 methodology. Exem-plary of item (3) above, the NPRDS failure records used for estimating S0V failure rates generally do not include the unrecognized S0Vs. It should be noted that publicly available SOV failure rate data does not distinguish between SOV size, energization mode, valve opening status, manufac-turer, model, or type. In view of the wide range of S0V variations, the avail-able failure data does not allow for accurately predicting individual SOV performance or failure rates. PRELIMINARY CASE STUDY 43
-. ~. h . Table 4 Estimates of SOV Failures to Operate Estimated . Source failure rate WASH 1400 1x10 8/ demand l l This study 7 to 9.5x10 8/ demand (NPRDS data Jan 85-Dec. 88) Assuming quarterly testing NUREG 1150 methodology NUREG/CR 4550 Vol. 1 1.0x10 8/domand (Seabrook PRA) 2.4x10 8/ demand L NUREG/CR 4550 Vol. 6 1.6x10 8/ demand (Grand Gulf PRA) NUREG/CR 4819, Vol. 1 7x10 8/hr (NPRDS data Sept 78-July 84) L This study 6.4 to 8.7x10 S/hr (NPRDS data Jan 85-Dec. 88) l L In view of the aforementioned problems of estimating single 50V failure rates, we find the task'of estimating the risk resulting from common-mode SOV. failure to be a difficult task, the results of which may have significant g uncertainty. Such an undertaking is beyond the scoviof the present study. E We know of no PRA which accounts for the contribution of connon-mode fail-L ures of SOVs. Omission of such cross system / cross train failens lead towards nonconservative results. 6.3 Maintenance Problems t l' 6.3.1-Maintenance Problems - S0V Manufacturers' Contributions Review of operating experience indicates that a substantive number of SOV failures are attributed to inadequate maintenancs or refurbishment. As evidenced l by several of the events discussed in Chapter 5, it is clear that utilities are L not fully informed of 50V maintenance requirements. Theneglectoroversightof S0V maintenance oftentimes comes from the SOV manufacturers failure to provide '50V maintenance requirements to the 50V users or second-level manufacturers-- 15 such as EDG manufacturers (ALCO, Colt /Fairbanks-Morse, General Motors, Delaval, Cooper-Bessemer), valve manufacturers (Xomox),~ controller manufacturers (Fisher,- Masoneilan),etc. Some SOV manufacturers are more prescriptive than others. Some manufacturers provide no guidance on preventive maintenance. One manufac-turer (Valcor) varies its recommendations depending on whether the purchaser bought the " full documentation package." t y PRELIMINARY CASE STUDY 44 1 1 .1
i Examples of the variation among SOV manutacturers maintenance recomenda-tions are discussed below. ASCO does not provide any specific recommendations for SOV maintenance or refur-bTsTaent. This is even true for their nuclear qualified 1E valves.. Quoting-ASCO's installation and maintenance bulletin for NP8323 SOVs'(Ref. 92). " Preventive Maintenance 1. Keep,the medium flowing through the valve as frea from dirt and foreign material as possible. Use instrument quality air, oil-free for Suffix "E." 2. While in service, operate valve periodically to insure proper opening and closing. 3. Periodic inspection (depending upon medium and service conditions) of internal valve parts for damage or exces-sive wear is recommended. Thoroughly clean all parts. Replace any parts that are worn or damaged. 4. The valves say require periodic replacement of the coils-and all resilient parts-during their installed life'to maintain qualification. The exact replacement period-will depend on ambient and service conditions. Spare parts kits and coils are ordered separately (see Ordering Information). Consult ASCO for specific recommendations in connection with the replacement of parts." Valcor provides specific recomendations for maintenance or refurbishment of its N-stamped S0Vs. However, it is possible to purchase the same valve without an N stamp. If it is purchased without an N stamp,~it can also be purchased without any documentation. Such a "no-doc" valve would not be provided with-any preventive maintenance or refurbishment recomendations. Target Rock - All of Target-Rock Corporation's SOVs appear to be supplied with specific preventive maintenance and refurbishment recommendations. Circle Seal, Ross and an Unspecified Foreign Manufacturer - Circle Seal and Ross make SOVs which are used in several different EDG air start systems. Those valves are not supplied with any preventive maintenance or refurbishment recomendations.- Lack of specific maintenance recomendations has contributed to multiple failures of a foreign manufacturer's SOVs used in the EDG air start . system of a foreign plant (see Section 6.3.2.1). Skinner Electric - SOVs manufactured by Skinner Electric Company which are used in Woodward Governors on BWR HPCI turbines are not provided with any preventive maintenance or refurbishment recomendations. >+ Sperry-Vickers - SOV's manufactured by Sperry-Vickers which are used in the hydraulic controllers used-for BWR recirculation pumps and main turbine-trip syste;t.s are not provided with preventive maintenance or refurbishment recomendations. l 1 PRELIMINARY CASE STUDY 45
6.3.2 Maintenance Problems - Contribution of the Unrecognized SOVs In many cases plant maintenance and operations personnel are unaware of the presence of, or maintenance requirements of S0Vs. This situation is common be-cause there are many cases in which SOVs represent oiily a small portion of a larger system or component, and the information available to plant staff does not identify the care required by the SOV which is " unrecognized" within the "overall system". Examples that we have observed are: Emergency diesel generators: air start systems, governors, and cooling water control systems. Auxiliary feedwater and main feedwater systems: flow control regulators. BWR high pressure cooling injection systems: remote shf.off controls, governors. Instrument air dryers: desiccant column regeneration at.d cycling control systems. 6.3.2.1 Unrecognized S0Vs in Emergency Diesel Generators The operation and maintenance manuals for the plants diesel engines, and operator and maintenance personnel training are heavily weighted by the engine manufacturer's literature which, at best, includes minimal information regarding the S0Vs used in the EDG's auxiliary systems. Specific examples observed included: A foreign reactor site where the air start S0Vs were not on any preventive maintenance program. Failure of one 50V due to aging of a Buna-N diaphram was undetected until its redundant backup failed from the same cause. As a result, the station added refurbishment or changeout of such resilient parts to all its EDG air start systems. Similar failures have been observed at numerous U.S. plants, e.g., Three Mile Island 1* (Refs. 93, 04), Ginna (Refs. 95, 96, 97), Duane Arnold (Ref. 98). During a trip to the Duane Arnold plant in reviewing S0V experience, AE0D staff learned that subsequent to the July 1982 failure (Ref. 98), the Duane Arnold staff recognized the 50V's limited lifetime and the need for 50V efurbishment or replacement. As a result the Duane Arnold staff added SOV changeout to their preventive maintenant.e program. However, several years later, plant maintenance personnel made a decision to eliminate changeout of that S0V from their preventive maintenance program. The rationale for dropping such pre-ventive maintenance was that the S0V was cycled only 7 seconds a month, and such limited use did not seem to require maintenance. The basis for implementing the 50V's preventive maintenance and the previous failure, which resulted from age related degradation, appeared to have been forgotten. Subsequently, the licensee stated that preventive maintenance on the aforementioned S0Vs would be reinstated. As a student in a recent TVA EDG training course applicable to seven plants, (Browns Ferry 1, 2, 3, Sequoyah 1, 2, Watts Bar 1, 2) the case study author learned that maintenance literature for the General Motors Electro-Motive Division (GM-EMD) diesel engine supplied by Morris-Knudsen, does not provide the iicensee with any instructions for refurbishment or changeout of the SOVs in the EDGs' air start and governor control systems.
- Telephone discussion, M. Schaefer, General Public Utilities, and H. L. Ornstein, USNRC, February 16, 1989.
PRELIMINARY CASE STUDY 46
6.3.2.2 Unrecognized SOVs in Auxiliary and Main Feedwater Systems As noted in Section 5.2.3.2, a review of failure data at North Anna Units 1 and 2 showed that poor quality air was the root of the SOV/ control valve fail-ures. As a result, the licensee initiated a program for repairing and replacing the SOVs and control valves, as well as upgrading the air system quality and enhancing plant personnel training and maintenance practices. 6.3.2.3 Unrecognized SOVs in BWR High Pressure Coolant Injection Systems In Reference 99 the Duane Arnold plant's licensee-reported the failure of the remote shutoff' control system which is part of the HPCI turbine's governor system. Discussion with plant aersonnel and the turbine manufacturer indicated a lack of communication between t1em regarding the potential for undetected fail-ures of the SOVs. The licensee's report noted that the failure was caused by aging of the elastomeric parts of the SOV. Such an undetected failure could result in failure to start the HPCI system. Apparently information provided by the turbine manufacturer (Dresser-Rand, formerly Terry Turbine) did not pro-vide adequate maintenance information about the 50V that is supplied as an internal part to the Woodward Governor (the 50V was manufactured by Skinner Electric Co.). The Skinner Electric maintenance instructions do not address preventive maintenance or service life requirements for the SOV. The Woodward Governor service manual does not address 50V preventive maintenance, or service life. The service information letters (SILs) provided by the NSSS vendor (GE) did address other aspects of HPCI turbine service, performance and maintenance, but discussion with plant personnel and GE personnel indicated that maintenance, refurbishment or replacement of the SOVs are not addressed in any of GE's SILS. 6.3.2.4 l Unrecognized SOVs in Instrument Air Driers Review of a leading instrument air drier manufacturer's operation and maintenance manual indicated minimal guidance with regard to SOV maintenance. The SOVs are required to cycle every five minutes to ensure that the air flows through the correct desiccant stack to assure proper air drying and acceptable outlet dew points. Failure of the SOVs could result in undetected high instru-ment air moisture content which could lead to degradation and malfunction of-equipment utilizing instrument air, including other SOVs that perform safety-related functions. 6.3.3 Maintenance Problems - Contributions of Utility Programs and Practices Review of S0V failure reports and follow up discussions with plant personnel, NRC inspectors, and 50V manufacturers showed that shortcomings in many utilities' SOV maintenance programs and practices were a major source of S0V failures. For example: (1) Reference 100. indicated that Brunswick plant staff stated that ASCO Class 1E SOVs with 30 year qualified lives did not require any preventive mainte-nance for 30 years. The licensee did not recognize the fact that the resilient, or elastomeric parts of the SOVs require more frequent replacement. PRELIMINARY CASE STUDY 47
--.....o., l!' (2) After-finding t' hat-SOVs would not shift their position on demand during surveillance testing, it was common practice for Brunswick and North Anna Stations' plant personnel to tap the SOVs (" mechanical agitation").- If a l SOV would change position when tested after the mechanical agitation, no further maintenance would be performed, and the SOV would be declared operable (Refs. 100,101). (3) ASCO's valve engineering department product engineering manager visited the Susquehanna plant to assist the utility in finding the root cause of the = failure of a rebuilt ASCO SOV which failed after being returned to service. i The ASCO manager's discussions with plant personnel revealed that subsequent to rebuilding the SOV, plant personnel bench tested the SUV with poor qual-ity service air instead of clean, dry instrument air. Inspection of the SOV revealed that oil from tne service air system had caused the 50V's second failure.* = (4) The Calvert Cliffs 1 and 2 plants' SOV maintenance is tracked by the sta-tion's reliability centered maintenance (RCM) program. The RCH program has found that instrument air dryer SOVs have a mean time between failure of 10 months,*but the plants' maintenance program replaces such SOVs on an annual basis.
- The failura of the instrument air dryer SOVs can cause serious instrument air system degradation leading to common-mode failures of many other S0Vs, including those that perform safety related functions.
6.3.4 Rebuilding vs. Rt. placement Review of S0V failure data indicates,that inadequate rebuilding of SOVs has been a significant cause of 50V failures. There is a broad range of com-plexity associated with rebuilding S0Vs, depending upon individual S0V manufac-turer and model number. To further complicate the issue, there are variations among SOV manufacturers with regard to providing test apparatus to check the soundness of rebuilt SOVs; for example, Target Rock Corporation has marketed a test fixture for licensees to test their rebuilt S0Vs. Although some manufacturers provide values of acceptable coil voltages, leakage rates, etc., to enable users to check the conditions of their SOVs some other manufacturers do not make such information available. Seriousquestlons arise about the acceptability of new S0Vs if acceptance criteria are not available. 3 In Reference 102, ASCO notified licensees that it has discontinued selling rebuild kits for its nuclear power plat SOVs (NP series). However, ASCO is continuing to sell rebuild kits for coinmercial S0Vs and SOVs used in BWR scram systems (purchased through GE). As noted in Chapter 5, there have been several events in which common-mode failures resulted from incorrect rebuilding of SOVs. The potential for common-i mode SOV failure resulting from rebuilding errors may be minimized by staggering
- Telephone discussion, J. Shank, ASCO, and H. L. Ornstein, USNRC, May 11, 1989.
]
- Telephone discussion, J. Osborne, Baltimore Gas and Electric Co., and H. L.
Ornstein, USNRC, April 21, 1989. PRELIMINARY CASE STUDY 48
- the rebuilding (if possible), or by limiting the amount of SOV rebuilding done by any one individual (see Sections-5.2.2.2,5.2.2.3). In addition to focussing attention on the useful life of SOVs being governed by the elastomeric parts, attention should be focused on the shelf life and on - the actual manufacturing date of the elastomeric parts in the rebuild kits. For example, because of elastomeric (Buna-N) degradation observed in SOVs used in BWR scram systems, GE recommended (Ref. 56) that BWR scram system SOVs having Buna-N parts be rebuilt periodically. The frequency of rebuilding should be governed by the "useful life" of the elastomer ("useful life" being defined as the sum of shelf life and in-service life). Controlled by the Buna-N parts, GE recom-mended a "useful life" of seven years for scram system SOVs. The seven years being from the time of kit manufacture (not from the time of rebuild). h 4 PRELIMINAP.Y CASE STUDY 49
= 7.0 FINDINGS The root causes of most 50V problems are traceable to the lack of understanding of the capabilities and requirements of S0Vs. Oftentimes plant operations and maintenance programs do not address the short lifetimes of the resilient elastomeric piece parts of the SOVs (gaskets, seals, diaphrages, etc.). l_
- Maintenancc programs also fail to address the low tolerance SOVs have for oper-ating under adverse conditions that are significantly different than those of l
the coKrolled laboratory environment under which they were originally tested.- In man / cases, the manufacturers have not provided the end users with a full-une rstanding of the sensitive nature of certain parts of the SOVs. Many users .f hate learned after using certain SOVs that they are unforgiving and finicky with L rt. gard to contaminants and local environmental conditions. Deficiencies in selection, operation, and maintenance of SOVs have ~resulted in hundreds of SOV failures, many of which were common-mode failures that cut across multiple trains ~of safety systems. Ourmajorfindingsregardingthe root causes of common-mode 50V failures are described below. (: l 7.1 Desian Application Errors 7.1.1 Ambient Temperatures 1 Many common-mode 50V failures have resulted from subjecting SOVs to ambient temperatures in excess of their original design envelope. Such common-mode failures have resulted from localized steam leaks (see Section 5.1.1.1), . incorrect estimates of ambient temperatures (see Sections 5.1.1.2,5.1.1.3),and failure to account for ventilation system malfunctions (Ref.103). Because the useful. qualified lives of the short lived parts of SOVs are halved by every 18*F l.. temperature rise (Arrhenius theory - Refs. 104,105), seemingly minor increases in ambient temperatures above those considered in the SOV design cannot be-allowedtoprevailforextendedtimeperiodswithoutrunningtheriskof L ' sustaining ' seemingly" premature failures. l 7.1.2. Heatup from Energization Many common-mode SOV failures have occurred because the est hated service lives did not properly include the life-shortening effects of heatup due to continuous coil energization (see Sections 5.1.2.1,5.1.2.2).- Many licensees have been unaware of this situation. For example, by incorrectly using the certificates of compliance provided with ASCO's NP-1 nuclear qualified valves, - ~1icensees have overpredicted the service life of continuously energized SOVs. Use of appropriate 50V heatup data in conjunction with Arrhenius theory (Refs. 104,105) has been found to be an acceptable (but not a 100 percent accurate) method for predicting SOV life. 7.1.3 Maximum Operating Pressure Differential Many licensees have found misapplications in which SOVs could be or were L subjected to operating pressure differentials that could or did prevent them L 1 PRELIMINARY CASE STUDY 50
V 1 0' frem operating. Although NRC issued Information Notice 88-24 (Ref. 23) about thit, problem, as noted in Section 5.1.3.1, it is not clear that all the licensees havt addressed the issue, of over pressure which could result from pressure regulator failures. 7.1.4 Unrecognized SOVs Used as Piece-Parts Many S0Vs used in safety-related equipment are not given prominent attention because they are used as piece parts of larger equipment. Specific preventa-tive maintenance requirements are not readily available for then. Many SOV failures have occurred as a result of the lack of maintenance or replacement of such unrecognized SOVs (see Section 6.3.2). 7.1.5 Directional SOVs Six' plants have reported experiencing undesirable spurious openings of safety-related S0Vs due to high backpressure. The licensees did not recognize or were not aware of the directional requirements of the valves (see Section 5.1.4.1).. In addition to reports of SOV malfunctions which occurred because they were installed backwards, there are also reports of S0Vs which were installed upside-down, or at improper angles (see Appendix A). 7.2 Maintenance Operating experience has confirmed that SOV maintenance deficiencies can incapacitate multiple safety systems. The pervasiveness of maintenance deft-ciencies highlight the need for implementing aggressive SOV maintenance pro-gramt % prevent widespread common-mode failures. Specific maintenance problem art C are discussed below. 7. 1 Maintenance Frequency Lack of timely preventive maintenance (complete 50V replacement or rebuilding of short-lived piece parts of S0Vs) has resulted in many SOV failures-(see Sec-tions 5.1.2.1, 5.2.1.2, 6.3.2.1). Many SOV manufacturers have failed to provide the users with definitive information on the useful lifetime of the SOVs inter-nal diaphrages,, gaskets, 0-rings, coils, etc. Some manufacturers indicate that periodically changing the elastomeric parts is necessary, without specifying the frequency of changes. Other manufacturers do not even mention that any changing is necessary. Similarly, there are wide variations among manufacturers with regard to specifying (or not specifying) the allowable shelf. lives of their S0Vs and.50V rebuild kits (see Sections 6.3,1,6.3.3,6.3.4). Because of the limited lives of their elastomeric or resilient parts, SOVs . should be replaced or refurbished in accordance with the manufacturers'.recom-mendations. ' In the absence of specific manufacturers' recommendations, andin absence of applicable failure data, changeout of short-lived elastomeric and resilient materials (or complete valve replacement) should be done on the basis of material shelf life, manufacture date and installation date. However, change-out of elastomeric parts or complete SOV replacement should be done more fre-quently if operating conditions exceed the originally envisioned design conditions or if field failure experience dictates. PRELININARY CASE STUDY 51 .9
l 7.2.2 Replacement Versus Rebuilding Rebuilding or refurbishing certain models of several manufacturers' SOVs is a difficult task that can be made even more difficult if it is done in place, requiring the workers to wear decontamination or protective clothing.
- However, removal and reinstalittion of N-stamped valves which are welded into the primary system are not simple, inexpensive tasks either.
Incorrect rebuilding or refurbishing of SOVs have caused many premature failures (e.g., see Sections 5.2.2.1,5.2.2.2). Contributing to the difficulty of rebuilding or refurbishing S0Vs correctly is the fact that many manufacturers do not provide the licensees with adequate 50V documentation or testing apparatus to verify the effectiveness of the rebuilt or refurbished S0V. As a result, post-rebuild testing at. many facilities merely involves cycling verification rather than performing appropriate tests normally performed by the manufacturer during initial 50V manufacture (see Section 6.3.4). Discussions with plant personnel have revealed that many licensees, (e.g., Perry, River Bend, Salem, Grand Gulf, Duane Arnold) have chosen to discontinue rebuilding certain SOVs because improper rebuilding can result /has resulted in many SOV failures-and costly down-times. In general, licensees have reacted favorably to ASCO's recent decision to discontinue supplying rebuild kits for their NP-1 nuclear qualified SOVs (Ref. 106, 107). ASCO's decision to niscon-tinue supplying S0V rebuild kits was based upon field experience which indicated that many-ASCO SOV failures were caused by inadequate rebuilding techniques. 7.2.3 Contamination Many common mode S0V failures have been caused by contaminants in the fluids which flow throu h SOVs; instrument air in particular (see Sections 5.2.3.1, 5.2.3.2, 5.2.3.3. -S0V contamination resulting from particulates, moisture, and hydrocarbons in the instrument' air system have been a major source of common-mode SOV fail-ures. In many plants contaminants were introduced during original construction. Many contamination problems have resulted from poor design or maintenance of the instrument air systems. Many SOV failures are clearly attributed to subjecting the SOVs to conditions beyond their design regarding particulates, moisture, hydrocarbons, etc. Contributing to the problem is the fact that some manufacturers have specified the need for clean air or instrument quality air without quantifica-tion (e.g., maximum allowable particle sizes and dew points). Although licensees are taking actions to improve the quality of their plants' air systems, there is concern for the residual effects of previous air system contamination (Section 5.2.3.2). Long-term SOV degradation such as deterioration of EPDM parts as a result of hydrocarbon intrusion, formation of varnish-like deposits from heatup of hydrocarbons, and residue formation from the interaction of moisture, silicone lubricant, and heat, are areas of concern. PRELIMINARY CASE STUDY 52
7.2.4 Lubrication Improper lubrication has resulted in many com a n-mode S0V failures. The improper lubrication has been attributed to manufactaring errors (see Section 5.2.4.1), as well as licensee errors. Errors inc'.ude the wrong choice of lubricant (see Sections 5.2.4.2,5.2.4.3), unauttorized use of incorrect lubri-cant (see Section 5.2.4.1), and use of excessive amounts of lubricant (see Section 5.2.4.4). 7.3 Surveillance Testina Several cases'(see Section 6.3.3) have been reported in which SOVs failed to actuate on demand during surveillance testing, however, subsequent tapping (" mechanically agitating") the SOVs would enable them to actuate. As a result, 'the SOVs were declared operable without addressing the cause of the original-failures, thus leaving the SOVs in degraded states vulnerable to future failures upon demand. Similarly, as noted-in Section 5.3.1, incorrect surveillance testing led L operators to operate a BWR with multiple failed scram pilot SOVs. 7.4 Verification of the Use of Qualified S0Vs The issue of environmental qualification of Class 1E electrical equipment and S0Vs has been addressed by utilities in response to Bulletins 79-01A and B. Nonetheless, there are many instances in which S0Vs that were assumed (in plant safety analyses) to operate to mitigate design-basis events, have been procured as " commercial grade" SOVs of questionable quality and are not being maintained in a manner commensurate with their intended safety function. l Examples have been found where commercially available, non-qualified SOVs-are being used in safety-related applications without appropriate verification l of product quality and design control. In many instances the SOVs lack verifi-I cation that they can withstand the accident conditions postulated in plant safety analyses. A common problem appears to be categorization of the SOVs for use in EDG air systems. In many cases the original equipment that contained SOVs as j piece parts was certified or qualified to meet 1E requirements, whereas the l individual replacement SOVs were not. (See'Section 5.4.1). 7.5 Redundancy and Diversity The root causes of many common-mode failures of safety related SOVs have eluded many. licensees' detailed failure analyses (see Section 5.2.4.4). In many such instances the search for the origins of foreign unidentified sticky substances (FUSS) have been inconclusive, and corrective actions were limited to cleaning or replacing the failed SOVs (e.g., Brunswick (Ref. 2), Franklin Institute-(Ref. 76)). In some cases, the licensees discounted instrument air system contamination (oil, water, dirt) as the cause of the FUSS, but plant operating history indicated a prior history of air system contamination which could have been a contributor to the problem. Similarly, the SOV manufacturing process (see Section 5.2.4.1) and the licensee's rebuilding process (see Sections 5.2.2.1, 5.2.2.2, 5.2.2.3, Section 6.3.3) have been found to be the sources of contaminants which caused common-mode S0V malfunctions. PRELIMINARY CASE STUDY 53 L
Staggering the maintenan:e, testing and replacement of redundant SOVs may represent a simple way of pretenting common-mode failures of redundant SOVs. In addition, if the root causis of persistent common-mode SOV failures cannot be found, or cannot be eliminated, the need for-SOV diversity (with regard to model, energization mode, failure mode, or manufacturer) becomes apparent. (See Appendix B for a discussion of an example of such a problem with the ASCO NP8323 SOVs used for MSIV control at many BWRs.) 7.6 Feedback of Operatina Experience Based upon-visits to several of the major SOV manufacturers' facilities (e.g., ASCO (June 1988), Target Rock (November 1988), Valcor (December 1988), AVC (February 1990)) discussions with other SOV manufacturers (e.g., Circle Seal, Skinner Electric), and extensive discussions with manufacturers who's equipment utilize SOVs as piece parts (e' g., Fisher Controls, Dresser-Rand / Terry Turbine, Xomox Valves, California Controls (Calcon), Colt /Fairbanks-Morse), it was found that SOV manufacturers have not been fully apprised by the utilities of many SOV failures that have occurred at nuclear power plants. 50V manufacturers are not aware of many widespread failures of safety-related equipment that may have been caused by generic manufacturing or design-deficiencies of the SOVs. Conversely, when licensees purchase SOVs commercially, without 10 CFR 50 Appendix B, and 10 CFR Part 21 requirements, they are not fully apprised by the manufacturers of generic defects that are discovered subsequent to delivery. In one case, a major SOV manufacturer did not feed back generic SOV defect information to the end user due to the manufacturer's failure to understand or properly implement the 10 CFR Part 21 requirements that were applicable to its SOVs (Ref. 74) (also see Sections 5.1.2.2,5.2.4.3). PRELIMINARY CASE STUDY 54
-~ l; y
8.0 CONCLUSION
S 1 . Operating experience has demonstrated that common-mode failures and degradations of SOVs can compromise multiple trains of multiple safety systems. - The fact that hundreds, and in many cases thousands, of SOVs permeate all impor-L~ tant systems at all U.S. LWRs highlights the necessity for eliminating common-mode SOV problems that jeopardize plant safety. I 8.1 Safety Significance Considering the application of the " single failure criterion," the application of " defense-in-depth," and the large population of SOVs used in E safety-related systems at U.S. LWRs, it appears that the number of individual random SOV failures that have been reported do not appear to present a safety i l concern. However, examination of the root causes of many SOY failures at many plants demonstrate error patterns in the design / applications, maintenance and testing of SOVs which have led to a multitude of widespread common-mode failures. L l Operating experience shows that SOVs are vulnerable-to numerous common-mode L failure mechanisms and their failures can adversely impact numerous safety sys-L tems. Some of the safety systems that were observed to be adversely impacted by common-mode failures of SOVs were: EDG air start system, BWR scram system, BWR main steam isolation system, PWR auxiliary feedwater system, PWR safety injection system, component cooling water system, containment isolation system, residual heat rer. oval system, containment cooling system. These safety systems are required to function in order to prevent and/or mitigate accidents and/or l to protect the public from release of radiation from design basis accidents. Therefore, we conclude that SOV problems represent a significant safety concern. Chapter 5 presents examples of over twenty recent events having the potential for common-mode failures or degradations of over 600 S0Vs in impor-tant plant systems.* The comon-mode failures and degradations cut across mul-tiple trains of safety systems as well as n:ultiple safety systems. The recur-l rence of comon-mode St)V failures or degradations highlights the gravity of the situation. Although p.cnt safety analyses do not address common-mode, multiple train / multiple safety system failures, operating experience indicates that they have occurred and continue to occur. The common-mode SOV failures and degrada-tions that have occurred which compromised front line safety systems such as emergency ac power, auxiliary feedwater, high pressure coolant injection, and 7 scram systems clearly demonstrate the safety significance of SOV problems. Chapter 6 presents estimates of SOV failun rates whici, were extracted from plant operating data (NPRDS). The estimates ina'cate failure.Mtes of almost one order of magnitude larger than those assumed u the WASH 1400 study and in the NUREG 1150 methodology for level one PRAs. Coupling such nonconservative treatment of S0V failures with the fact that level one PRAs do not address SOV failures that cut across multiple systems leads us to conclude that the risk contribution from SOVs may have been severely underestimated in previous risk assessments. IThere have Deen many other similar events. The events chosen here are intended to be illustrative. Surely they are not a complete set of all such events. PRELIMINARY CASE STUDY 55 I
8,2 Need for Action On the basis of our analysis of operating data, we conclude that the SOV problems outlined in this study need to be addressed to ensure that the margins of safety for all U.S. LWRs remain at the levels perceived during original plant 111 censing. l, L We note' that to date the NRC has issued 36 generic comuiications pertain-I ing to SOV problems (See Appendix C). Those generic comunications alerted licensees to specific S0V problems, Based on our study we believe that an inte-grated comprehensive program is needed now to address the-root causes of SOV problems described in this report. We conclude that integrated implementation of the recomendations provided in Chapter 9 will significantly reduce the ' likelihood for comon-mode SOV failures eroding the margins of safety Ot all 'LWRs, ? o l i-l 1 PRELIMINARY CASE STUDY 56 l t
.,j .. = 9.0 RECOME@ATIONS t-In order to minimize the potential for common-mode failures, attention should be focused upon certain aspects of SOVs. We recommend that the actions a l-discussed below be initiated in order to assure that the plants retain the ? margins of safety perceived in their original licenses. If SOVs are found to be inadequate, prompt corretive actions should be taken. 9.1 Design Verification a Licensees should review SOV design specifications and actual operating conditions to verify-that all SOVc assumed to operate in FSAR safety analyses are operating within their design service life. 9.1.1 Ambient Temperatures The aviews should assure that the lifeshortening effects of elevated ambient temperatures are cons ldered in the determination of SOV service life. l' 9.1.2 Heatup From Energization l-l- The reviews should assure that che lifeshortening effects of heatup due to. coil energization are appropriately accounted for in the determinations of S0V service life. l 9.1.3 Maximum Operating Pressure Differential The -reviews should assure that the potential for overpressure due to pressure regulator failure or hydraulic fluid heatup due to postulated accident conditions have been considered in the selection of the SOVs. l. 9.1.4 Unrecognized S0Vs Used as Piece-Parts In addition to verifying the adequacy of the high visibility SOVs as noted -above, similar verification should be made for unrecognized S0Vs which are used as piece parts of flow regulators, governors, emergency diesel generators, etc. 9.1.5 Directional SOVs Licensees should verify that directional SOVs are installed in orientations which will assure satisfactory operation of the safety-related equipment which g depend upon them. 9.2 Maintenance 9.2.1 Frequency Licensees should implement SOV maintenance programs to replace or refurbish S0Vs on timely bases. Replacement or refurbishment schedules should focus upon thermal aging due to elevated ambient conditions and heatup fmm continuous coil energization. PRELIMINARY CASE STUDY 57
L 9.2.2 Replacement Versus Rebuilding *- -Licensees should review their programs for rebuilding SOVs because certain SOVs are difficult to rebuild and test properly, and improperly rebuilt SOVs degrade plant safety. Numerous utilities canvassed have found that in most instances it is cost beneficial to replace S0Vs rather than to rebuild them. If licensees choose to continue to rebuild their SOVs, we recommend that they obtain or develop test equipment to enable verification that the rebuilt S0Vs meet all the performance specifications of the original S0Vs. 9.2.3 Contamination SOVs, ggressive actions should be taken to assure that fluids which flow through A instrument air in particular, are maintained free of contaminants. If operational experience indicates a pattern of SOV malfunctions resulting from contamination (such as water or hydrocarbon intrusion), the affected licensees should consider replacing S0Vs that have been subjected to previous air system degradation assuming that the root causes of the air system problems have been corrected (InaccordancewithGenericLetter88-14). 9.2.4 Lubrication S0V manufacturer's lubrication instructions should be adhered to. Sub-stitution of similar but not identical lubricants should be avoided.
- However, if substitutions are made, their compatibility with all associated hardware should be verified.
9.3 Surveillance Testing Operation and maintenance personnel training should emphasize the importance of surveillance testing, root cause failure analysis, and timely repair or replacement of malfunctioning S0Vs. Licensees should review, and if appropriate, modify their surveillance testing procedures. Procedures should expressly prohibit " tapping" or mechanical agitation of S0Vs as techniques to assist successful operation during surveil-lance testing. Procedures should include actions to be taken when unsatisfactory test results are encountered, as well as a requirement to analyze and evaluate the causes of the unsatisfactory results prior to decla.ing the component back in service (even though subsequent retest results may be satisfactory). 9.4 Verification of the Use of Qualified SOVs Licensees should review all S0Vs in safety-related applications, EDGs in particular, to ensure that they meet 10CFR 50 Part B and appropriate Class 1E requirements; and that they have been installed and maintained appropriately to assure they will operate in a manner consistent with the assumations of the plants' safety analyses. If there is doubt regarding the accepta)ility of any such S0Vs, they should be replaced with appropriately qualified ones.
- exclusive of coil replacement - coils are generally replacement items PRELIMINARY CASE STUDY 58
9.5 Redundancy and Diversity When operating experience indicates unexplained repetitive common-mode-SOV failures affecting redundant components - (such as BWR MSIVs and containment isolation valves), licensees should consider performing maintenance, testing and replacement of redundant SOVs on a staggered basis. Additional consideration should be given to using diverse S0Vs (different design or manufacturer). 9.6 Feedback of Operatina Experience-In order to improve-SOV reliability, an industry group such as the Institute of Nuclear Power Operations (INPO) should initiate an SOV failure feedback prog-ram. The program should alert SOV manufacturers to failures of their equiomant by providing them with complete failure records of their specific S0Vs suc1 as those found in NPRDS. l PRELIMINARY CASE STUDY 59
n
10.0 REFERENCES
1. V. P. Bacanskas, G. C. Roberts, G. J. Toman, " Aging and Service Wear of Solenoid-Operated Valves Used in Safety Systems of Nuclear Power Plants, Volume 1. Operating Experience and Failure Identification," Oak Ridge National' Laboratory, NUREG/CR-4819, ORNL/Sub83-28915/4/V1, March 1987. 2. Carolina Power & Light Company, Licensee Event Report (LER) 50-324/88-001 Rev. 5, Brunswick Steam Electric Plant Unit 2, February 19, 1990. 3. Burns and Roe, " Investigation of Valve Failure Problems in LWR Power: Plants," ALO-73, April 1980. 4. W. H. Hubble, C. F. Miller, " Data l Summaries of Licensing Event' Reports of Valves at U.S. Commercial Nuclear Power Plants," January 1, 1976 to December 31, 1978 EG&G Idaho, Inc. NUREG/CR-1363 EGG-EA-5125, June 1980. 5. "IEEE Maintenance Good Practices for Nuclear Power Plant Electrical-Equip-ment; Work-In-Progress Report, Evaluation of Maintenance and Related Prac-tices for Solenoid Operated Valves In Nuclear Power Generating Stations," IEEE Power Engineering Society, Nuclear Power Engineering Committee,- Subcommittee 3, Working Group 3.3, 1988.
- 6. -
U.S. Nuclear Regulatory Commission, Inspection Report 50-440/87-024, Perry Nuclear Power Plant Unit 1, January 22, 1988. 7. Cleveland Electric Illuminating Company, Licensee Event Report (LER) 50-440/87-073 Rev.1, Perry Nuclear Power Plant, Unit 1, June 3,1988. 8. U.S. Nuclear Regulatory Commission, Information Notice No. 88-43, " Solenoid Valve Problems," June 23, 1988. 9. U.S. Nuclear Regulatory Commission, Inspection Report 50-440/87-027, Perry Nuclear Power Plant. Unit 1, February 10,'1988. .10. U.S. Nuclear Regulatory Commission, Inspection Report 50-302/89-09, Crystal River 3, June 7, 1989, 11. U.S. Nuclear Regulatory Commission, Inspection Report Number 50-336/88-22, Millstone Nuclear Station, Unit 2, November 8, 1988. 12. Memorandum from J. W.- Craig, U.S. Nuclear Regulatory Commission, to A. C. Thadani,
Subject:
" Status of TI 2515/98 "High Temperature Inside Containment /Drywell In BWR and PWR Plants," dated March 13, 1989, 13. U.S. Nuclear Regulatory Commission, Information Notice No. 89-30, "High Temperature Environments At' Nuclear Power Plants," March 15, 1989. 14. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 16313, Grand Gulf Unit 1, August 14, 1989. 15. U.S. Nuclear Regulatory Commission Region II Daily Report, August 15, 1989. ~ PRELIMINARY CASE STUDY 60 _____x
4 -16. System Energy Resources, Inc,' Licensee Event Report (LER) 50-416/89-013, Grand Gulf 1, September 13, 1989. 17. General Electric Company, Service Information Letter (SIL) No. 481, " Category 2, Malfunction of ASCO Solenoid Valves for MSIVs," February 14, 1989. 18. U;S. Nuclear Regulatory Commission, Information Notice 89-66, "Qualifica-tion Life of Solenoid Valves," September 11, 1989,
- 19. - Automatic Switch Company (ASCO), " Field Notification Concerning the Qualified Life of ASCO Catalog NP-1 Valves," October 27, 1989.-
~20. Memorandum, P. T. Knutsen, Virginia Electric and Power Company, "Self Heating-Effects in ASCO Solenoid Valves," dated February 3, 1987. 21. North Anna Power Station Deviation Report Number 87-105, dated February 4, 1987. 22.~ Memorandum, P. T. Knutsen, Virginia Electric and Power Company", "ASCO S0Vs To Be Replaced at North Anna Power Station Units 1 and 2, dated March 16, 1987. 23. U.S. Nuclear Regulatory Commission, Information Notice 88-24, "Fai N res of Air-0perated Valves Affecting Safety-Related Systems," May 13, 1988.
- 24. Wisconsin Public Service Corporation, Licensee Event Report (LER),
50-305/87-012 Rev.1, Kewaunee Nuclear Power Plant, March 4,1988. 25. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report' Number 12013, Calvert Cliffs l'and 2, April 14~, 1988. 26. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 12015, 'Calvert Cliffs 1 and 2, April 14, 1988.~ 27. U.S. Nuclear Regulatory Commission, Inspection Report 50-317/88-07 and 50-318/88-08, Calvert Cliffs Nuclear Power Plant Units 1 and 2, June 3, L 1988. 28. Memorandum from S. E. Scrace, Northeast Utilities, to E. Abolafia, " Failure of Air Operated Valves Affecting Safety Related Systems, NRC IN 88-24, N0A 9673, CR 0488-24 Revision 0," November 8, 1988. l 29. Florida Power Cor) oration, Licensee Event Report (LER) 50-302/89-01 Rev. 2, Crystal River Jnit 3, June 7,1989. 30.- U.S. Nuclear Regulatory Commission, Information Notice 80-40, " Excessive L Nitrogen Supply Actuates Safety-Relief Valve Operation to Cause Reactor Depressurization," November 7, 1980.
- 31. -
U.S. Nuclear Regulatory Commission, Office of Inspection and Enforcement, Bulletin No. 80-25, " Operating Problems with Target Rock Safety Relief Valves at BWRs," December 19, 1980, PRELIMINARY CASE STUDY 61
'32. Metropolitan Edison Co., Licensee Event Report (LER) 50-289/80-018 Three-Mile Island Unit 1,-November 4, 1980._ 1 . 33. Georgia Power Company, Licensee Event Report (LER) 50-424/87-002, Vogtle Electric Generating Plant Unit 1, February 23, 1987. 34. U.S. Nuclear Regulatory Comission 10 CFR 50.72 Report Number 12890, Pilgrim Unit 1, July 19, 1988. ~l u - 35. U.S. Nuclear Regulatory Comission Region I Daily Report, July 20, 1988. 36. Boston Edison Company, Licensee Event Report (LER) 50-293/88-021, Pilgrim Nuclear Power Station, August 18, 1988. 37. U.S. Nuclear Regulatory Commission, -Inspection Report 50-336/88-22, Millstone Nuclear Power Station, Unit 2, November 8,1988. 38. U.S. Nuclear Regulatory Commission, Inspection Report 50-302/89-01, Crystal River Unit 3, April 13,1989, 39. U.S. Nuclear Regulatory Comission 10 CFR 50.72 Report Number 14442, 4 Crystal River Unit 3, January 7,1989. 40. U.S. Nuclear Regulatory Comission 10 CFR 50.72 Report Number 14467, Crystal River Unit 3, January 11, 1989. - 41. Target Rock Corporation Operation Manual TRP 1571J, May 23,1978. 42. Gulf States Utilities Company, Licensee Event Report (LER) 50-458/89-022, River Bend 1, June 1,1989. 43. U.S. Nuclear Regulatory Commission, Generic Letter 88-14, " Instrument Air Supply System Problems Affecting Safety-Related Equipment," August 8,1988.
- 44. Gulf States Utilities Company, Licensee Event Report (LER) 50-458/89-024, River Bend 1, June 19, 1989.
45. U.S. Nuclear Regulatory Comission, Preliminary Notification, PNO-III-85-84, September 20, 1985. 46. U.S. Nuclear Regulatory Comission, Region III Daily Report, September 24 1985. 47. Comonwealth Edison Company, Licensee Event Report = (LER) 50-249/85-018, Dresden Nuclear Power Station, Unit 3, October 1, 1985. 48; - U.S. Nuclear Regulatory Comission, Office of Inspection and Enforcement, Information Notice No. 85-95, " Leak of Reactor Water to Reactor Building: Caused by Scram Solenoid Valve Problem," December 23, 1985. 49. U.S. Nuclear Regulatory Commission, Office for Analysis and Evaluation of Operational Data, Case Study No. AE00/C403, "Edwin I. Hatch Unit No. 2 Plant Systems Interaction Event on August 25, 1982," May 1984. PRELIMINARY CASE STUDY 62
.3 50. Carolina-Power & Light Company, Licensee Event Report (LER) 50-325/79-074, Brunswick Steam Electric Plant, Unit 1, November 16, 1979. 51. U.S. Nuclear Regulatory Commission, Office of Inspection and Enforcement, " Bulletin.No. 80-14, " Degradation of BWR Scram Discharge Volume Capability, June 12, 1980.- Sla. Cleveland Illuminating Company, Licensee Event Report (LER) 50-440/87-009, Perry Nuclear Power Plant, March 27, 1987. 52. Carolina Power & Light Company, Licensee Event Report (LER) 50-325/87-020, Rev. 1, Brunswick Steam Electric Plant Unit 1, March 31, 1988. 53. U.S. Nuclear Regulatory Commission, Information Notice 87-48, "Information. Concerning the Use of Anaerobic Adhesive / Sealants," October 9, 1987. - 54. Philadelphia Electric Company, Licensee Event Report (LER) 50-278/83-018, Peach Bottom 3', February 10, 1984. 55. U.S. Nuclear Regulatory Commission, Office of Inspection and Enforcement, Information Notice No. 84-53, "Information Concerning the Use of Loctite 242 and Other Anaerobic Adhesive / Sealants," July 5, 1984. 56. General Electric Service Information Letter (SIL) No. 128, Revision 1, Supplement 1, Category 1, " Preventive Maintenance for CRD Scram Pilot Valves," August 1978. 57. General Electric Service Information Letter (SIL) No. 128, Revision 1, Supplement 1, Revision 2, Category 1, " Preventive Maintenance for CRD Scram Pilot Valves," dated March 2, 1984. 58. Letter from P. W. Howe, Carolina Power & Light Co., to J. N. Grace, U.S. Nuclear Regulatory Commission,
Subject:
Docket Nos. 50-325 and 50-324 Brunswick Steam Electric Plant, Units 1 and 2, " Failure of ASCO 8323 A36E Double Solenoid-Valves," October 15, 1985. 59. Carolina Power & Light Company, Licensee-Event Report (LER) 50-324/85-008, Brunswick Unit 2, October 25, 1985. 60. U.S. Nuclear Regulatory Commission, " Report to Congress on Abnormal Occurrences, October-December 1985," NUREG-0090, Vol. 8, No. 4, May 1986. 61. North Anna Station Deviation Report, 87-379, April 24,1987. 62.. U.S. Nuclear Regulatory Commission, Inspection Report 50-338/88-02, 50-339/88-02, North Anna Power Station Units 1 and 2, March 14, 1988. 63. Virginia Electric Power Company, Licensee Event Report (LER) 50-338/89-002, North Anna Unit 1,-February 7, 1989. 64. North Anna Power Station, memorandum, " Water Intrusion Into Instrument Air System Event - April 1987," by T. L. Porter, August 8, 1988. PRELIMINARY CASE STUDY 63
j 65. North ~ Anna Power Station, memorandum, " Justification for Continued Opera-tion, Unit' 2 for ASCO CCW Trip Valves," by M. L. Bowling, February 10, 1988. 66. North Anna Power Station, memorandum, " Justification for Continued Opera-tion, Rev. 1," by D. A. Heacock, February 9, 1988. 67. U.S. Nuclear Regulatory Commission, Region II, " Notice of Significant Meeting," February 2,1989. 68. U.S. Nuclear Regulatory Commission, Inspection Report No. 50-387/84-35; 50-388/84-44, Susquehanna Steam Electric Station, November 15, 1984. 69. U.S. Nuclear Regulatory Commission, Inspection Report No. 50-387/85-09, i 50-388/85-09, Susquehanna Steam Electric Station, April 15, 1985. t 70. U.S.' Nuclear Regulatory Commission, " Report to Congress on Abnormal Occurrences, October-December 1984," NUREG-0090, Vol. 7 No. 4, May 1985. 71. U.S. Nuclear Regulatory Comruission,-Inspection Report No. 50-387/84-38, 50-388/64-37, Susquehanna Steam Electric Statior., February 27, 1985. 72. Wisconsin Public Service Corporation, Licensee Event Report (LER) 50-305/ 88-07, Rev. 1, Kewaunee Nuclear Power Plant, May 19, 1989. 73. Letter from J. P. Weaver, Automatic Switch Co. (ASCO) to T. E. Murley, U.S. Nuclear Regulatory Commission, " Potential Failures of NP8314 Series Valves," October 18, 1988. 74. U.S. Nuclear Regulatory Commission, Inspection Report 99900369/88-01, Automatic Switch Company, August 30, 1988. 75. Omaha Public Power District, Licensee Event Report (LER) 50-285/82-006, Fort Calhoun Station Unit 1, April 7, 1982. 76. V. P. Bacanskas, G. J. Toman, S. P. Carfagno, " Aging and Qualification Research on Solenoid Operated Valves," Franklin Research Center, NUREG/CR-5141, August 1988. 77. General Electric Service Information Letter (SIL) No.196, Supplement 10, Category 1, "SRV Failures To Open On Manual Demand and Air Operator Seal Failures," dated April 1981. 78. Duke-Power Company, Licensee Event Report (LER) 50-414/87-031, Catawba Nuclear Station, Unit 2, January 20, 1988. 79. U.S. Nuclear Regulatory Commission, Inspection Report 50-413/88-20 and 50-414/88-20, Catawba Nuclear Station Units 1 and 2, June 7, 1988. 80. U.S. Nuclear Regulatory Commission, Inspection Report.50-413/89-07 and 50-414/89-07, Catawba Nuclear Station Units 1 and 2, April 20, 1989. 81. Letter from T. Hutchins Automatic Valve Corporation (AVC) to J. Keppler, U.S. Nuclear Regulatory Commission Region III, December 19, 1986. PRELIMINARY CASE STUDY 64
-x t 82. Mississippi Power & Light Company, Licensee Event Report (LER) 50-416/85-007 Rev. 2, Grand Gulf Nuclear Station, Unit 1,-October 2, 1985. 83. U.S. Nuclear' Regulatory Commission, Information Notice No. 85-17, Supplement 1, "Possible Sticking of ASCO Solenoid Valves," October 1, 1985. 84. Commonwealth Edison Company, LaSalle County Station, "30 Day Report on the Failure of. Main Steam Isolation Valve (MSIV) Pilot Solenoid Valve 1821-F028C," prepared by Sargent & Lundy, January 14,.1988. 85. U.S. Nuclear Regulatory Commission, Inspection Report 50-373/87-035, LaSalle County Station, Unit 1, January 1988.
- 86. Gulf States Utilities Company, Licensee Event Report (LER) 50-458/88-023, River Bend Station, Unit 1, October 31, 1988.
8'/. Os S h eiear Regulatory Commission 10 CFR 50.72 Report Number 13591, September 30, 1988.
- 88. Gulf States Utilities Co., River Bend Station, " Analysis of Components in an ASCO Solenoid Valve," Franklin Research Center Report, P-741-1, February 9, 1989.
89 .U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 17201, November 27, 1989. 90. U.S. Nuclear Regulatory Commission Region III Daily Report, December 4, 1989. 1 91. Cleveland Illuminating Company, Licensee Event Report (LER) 50-440/89-030, L Perry Nuclear Power Plant, December 26, 1989. 92. Automatic Switch Co. (ASCO), Installation and Maintenance Instructions, Bulletin 8323, Form No. V5972R1, 1981. 93. U.S. Nuclear Regulatory Commission, Inspection Report 50-289/88-28, Three Mile Island Nuclear Station Unit 1, February 2,1989. 94. Facsimile Transmission, J. Shank, ASCO, to H. L. Ornstein, USNRC, February 17, 1989. 95. U.S. Nuclear Regulatory Commission, Region I Daily Report, December 9, 1988. 96. Rochester Gas & Electric Company - Ginna Station Memorandum, " Failure of Solenoid Operated Valve 5933B "A" Diesel Generator Air Start Valve ASV-1" by B..Popp, December 14, 1988. 97. U.S. Nuclear Regulatory Commission, Inspection Report 50-244/88-25, Ginna Nuclear Plant, February 2, 1989. 98. Iowa Electric Light and Power Company, Licensee Event Report (LER) 50-331/ 82-039, Duane Arnold Energy Center, July 6, 1982. i PRELIMINARY CASE STUDY 65
i e
- 99. Iowa Electric Light and Power' Coinpany, Licensee Event Report (LER)'50-331/
i 85-002, Duane Arnold Energy Center, February 27, 1985, 100. U.S. Nuclear Regulatory Commission, Inspection Report, 50-325/88-25 and li 50-324/88-25, Brunswick 1 and 2, September 26, 1988. 101. U.S. Nuclear Regulatory Commission, Inspection Report,'50-338/88-02 and c [ 50-339/88-02, North Anna 1 and 2, March 14, 1988. 102. Automatic Switch Co. (ASCO) Service Bulletin, " Discontinuation of Rebuild Kits for ASCO "NP" Series Valves," May 23, 1989. 1 -103. Commonwealth Edison Company, Licensee Event Report (LER) 50-237/88-022, Dresden Nuclear Power Station, Unit 2, December 13, 1988, 104. D. V. Paulson, T. A. Shook, V. P. Bacanskas, S. Carfagno, " Equipment Qual-E ification Research Test Program and Failure Analysis of Class 1E Solenoid l Valves," Franklin Research Center, NUREG/CR-3424, F-C 5569-309/315, 1 November 1983.- 105.S.P.Carfap'no,R.J.Gibson,"AReviewofEquipmentAgingTheoryand-Technology, Franklin Research Center, EPRI NP-1558, September 1980, t 106. Automatic Switch Company (ASCO), " Field Notification of Discontinuation of NP8323 Valve Line," August 3,1989. 107. Automatic Switch Company (ASCO), " Revised Field Notification of the Discontinuation of NP8323 Valve Line," October 27, 1989. lI PRELIMINARY CASE STUDY 66
.s .c.. t i i 4 APPENDIX A SOV FAILURES REPORTED IN LERs: 1984-1989 I ( -I l i I I I
7-.- t .,g-, t Legend for Appendix A-DOC NO. = Docket Number REP FL- = Repetitive Failure - TP/0VT = Cause Reactor Trip or Plant Outage FC- '= Failure Category l p 1; 1 l l l l 1 y P -4 L l l 1
s APPENDIX A FAILURE CATEGORIES OTHER 00 COIL rAILURE 01 ' VALVE BODY FAILURE / LEAKAGE 02 0-RING / GASKET / PLUG / SEAT / DIAPHRAGM / SPRING FAILURES / LEAKAGE 03 LUBRICANT / LUBRICATION 04 " STICKING" 05 INTERNAL WIRING / REED SWITCH / CONTACTS 06 EXTERNAL WIRING 07 INSTALLATION / MAINTENANCE ERROR-PHYSICAL (BACKWARDS, UPSIDE-DOWN, etc.) 08 INSTALLATION / MAINTENANCE ERROR-ELECTRICAL (LOOSE CONTACTS, AC vs DC, etc.) 09 EXCESSIVE ENVIRONMENT TEMPERATURE .30 MOISTURE INTRUSION (ELECTRICAL SHORTS / GROUNDING /0 PEN CIRCUITS) 11 CONTAMINANTS (DIRT, WATER, RUST, HYDROCARBONS, DESICCANTS, etc.) 12 M0PD (MAXIMUM OPERATING PRESSURE DIFFERENCE) 13 ' DESIGN ERROR (OTHER THAN MOPD)- 14 EQUIPME"T 9!!ALIFICATION-SEISMIC 15 EQUIP 65-i yVALIFICATION-RADIATION 16 INADEQUATE MAINTENANCE / EXCESSIVE TIME BETWEEN REPLACEMENT OR OVERHAUL 17 "END OF LIFE"/ NORMAL WEAR 18 "STILL UNDER INVESTIGATION" 19 "UNKNOWH" 20 " UNSPECIFIED" 21 " PERSONNEL ERROR" 22 REQUIRED CLOSING /0PENING TIME SPECIFICATIONS-NOT MET 24 LEAKAGE UNSPECIFIED 26 ASSEMBLY ERROR (PLUG / DIAPHRAGM / SPRING etc.) 27 EQUIPMENT QUALIFICATION (ELECTRICAL) 28
.m M {'.d7 4 Page No. 1-06/07/90 SOLEmotD-OPERATED VALVE FAILURE DATA + DOC PLAhT EVENT LER NO. OF FAILED SYSTEM 8WWUFACT MODEL ROOT REP CORRECTIVE COMEWTS WETEWENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART NO. CAUSE Ft. ACTION DOCUMEWTS SJT 206 San Onofre 1 12/30/86 86-014-0* One Ground Feeduster & Not tot Moisture in to New jtartion Corrective LER 87-001 Wo 11 fault. Safety Specified Specifi isnction box box instetted action taken on moisture Injection ed feited jtmetion in System bom and seven junction other bon vulnerabie enes. 206 San onofre 1 01/17/87 87-001 One Ground Feedwter ine* quete Yes Eliminated Vibration no 07 fault instettation/v ground,tig. ten caused ibration ed connections toosening of terminet box concheit locking ring 206 San onofre 1 11/10/87 87-016 Five failures of Slug Containment ASCO 206-380 Lthricant Yes su ured Sovs Cause of Insp ppt wo 05 four valves sticking Isotation, susp Mted in safety sticking trder 89-24 Contairment position and investigation Spray initiated weekIy testing 206 San onofre 1 12/01/87 87-017 Two Not safety not mot Unknown no Repelred or SOV required mone so 19 Specified injection Specified Specifi reptoced 50V for venting SIS wnt ed to evoid water hemmer 206 San Onofre 1 12/16/87 87-018 One Ground Plant Not Not Loose screws Yes The ground wos The loose See No 11 fault cooling Specified Specift and ino+ quete eliminated by screws we e consumts moisture water ed seat. Root renewing the probably in SOV cause not weter inside stri ped from housing specified the solenoid excessive housing and tightening. reseeting the Ref. Docs. LERS housing. 206/86-014/01, and 361/87-001,031 206 San Onofre 1 02/15/88 88-004-02 One SOV &afety Target 80EE-00 Stitt under Yes SUV was $0V failure lee No 19 sterse Injection Rock 1 irwestigation reptoced. prevented bleed 206/81-020 and Modifled of f from dothie position maintenance disc gate vetve indicatio procedures (ine bonnet. n switch Luding laetementation of afr's' recomummd for new reed switch cat ibratten = -
Page No. 2 06/07/90 SOLEno!D-OPERATED VmLVE FAILURE DATA DOC PLANT EVENT LER No. OF FAILED SYSTEM MANUFACT MODEL POOT REP CORRECTIVE CDISEWTS WEFEREWCE TP/ FC No. #AME DATE NUMBER FAILtRES PART NO. CAUSE FL ACTION DOC 1M4TS tRff 206 San Ormfre 1 03/03/89 89-006 Containment Design errer Design Discovered thet 14 fire suodification e singte $0V stepression niede could degrade contelrument oprey system,resultin y in containment everpressure during a LOCA 206 San Onofre 1 08/23/89 89-026 One Teil'A to Recire ASCO 206-380 Suspnt Yes Reptoced SUV LER 87-016 de 05
- shift, system (thricant "stickino tsafety sis.;;"
injectiorg/co nteirument sprey) 213 Maddam Neck 11/02/84 85-005 Tuo Failed to Auxiliary ASCO NPfLT20 Unknown no Sov retested Sovs feited None Jo 05 shift Feetheater acceptably, during testing. " stuck" System dectered SUVs re wired operetionet, for suore frement auto-init 9tice. cycling tests of AFW pter m i 213 Haddam Neck 09/10/85 85-024 One ' Faited to Auniliary ASCO NP-8320 Unknown Yes septered Sovs. Cause of LER 85-005 wo 05 shift,"st Feeduster Initiated more sticking has utk" Systec frepent not been periodic deterwrined. cycting Sense Sovs as in LER 85-005 213 Maddam Neck 01/14/88 88-001 Four incipients Sov Conteirument mot not Desipi no Corrected Instetted 50Vs mene me 08 w erating Isolation - Specified Speciff Deficiency circuit close te m mode Steam ed design, rather deenergizing Generator then changing insteed of Stoudoun the Sovs opening upon deenergizing per design. Condition existed for seven years l l e b
M' Page No. 3 06/07/90 SOLEWOID-OPERATED VALVE FAlttl#E DATA DOC PLANT EVENT LER MO. OF FAILED SYSTEM muuFACT NTEL ROOT WEP CORWECTIVE - COMEESTS REFEsteCE TP/ FC NO.
- AME DATE NUp0ER FAILURES PART mo.
CAUSE FL ACit05 DOCUMENTS OUT 219 Oyster Creek 10/16/84 84-022 Three Diaphragm Scram wet mot Instetled no Instett Caused slow mene no 27 Discharge specified sper'fi diaphrage disperse closure of 3 vottsee ed beckverds. correctly and air-operated In@te SOY develop SDV vent and rebuilding and isproved drain volves inedMaste. post-mointenen post-mointenen ce testing ce test 220 Nine Mile *t 1 06/14/84 84-013 Three Seat Main steam Dresser /C 1525VM Weer and Yes 1 refurbished, actest of att 6 LER 84-014 No 03 teekege(2 line on:st. contaminants 2 reptoced velves found ),misposi Electrome suspected att to be tiened tie teeking due to wires meterial lodged in the seet eres (see LER 84-014) 220 Nine Mile Pt 1 06/17/84 84-014 Six 5 seat Main steen Dresser / 1525 VM Foreign Yes cleaned sauf metcst of att 6 84-013 wo 12 teekege / Conset. meterist refurbished SOVs (LFR 1 stuck Electrome intrusion SOVs 84-013; found open dtse tic (source not ett to !e to stated) teeking dse to foreign forelyt matt meteriet todged in the seet eree 220 Nine Mite Pt 1 11/01/85 85-021 One plus two Jansned Main steam Dresser /C 1525Vx weer N Reptoced etI sene
- o 03 incipients springs onsot.
three ve,ves Electrome tic 237 Dresden 2 07/17/57 87-023 One Internet Feedwater ASCO 8300 ueer Yes septoced SOY SOV is e Wone Yes 18 passagawa (FWRV) pieceport of y the FURV. restricti on 245 Mittstone 1 12/24/85 85-034-01 setween three 1 core Controt red Asco not Deterioration Yes Sovs rebuilt, Falture of Wone No 17 and six
- spring, drive specift of the Dune-N tygraded SPSV three contret many ed discs and a maintenance rods to scram discs detached program per GE was attributed spring.
SIL 128 to feiture of three to six essociated scron p' tot setenoid watves.
Page No. 4 06/07/90 SOLENOID-OPERATED WALVE FAILURE DATA DOC PLAuf EVENT LER NO. OF FRILED SYSTEM MenUFACT MODEL 900T REP CORRECTIVE CtysqENTS REFEaENCE TP/ FC. NO. NAME DATE NUMBER F AILtJP',S PART NO. CAUSE FL ACTIOu 90CtJMENTS tR#T 245 Mittstone 1 06/06/87 87-015-02 One Excessive Contaffusent Target Wet Pitmeer ttee no Replaced Wene Gene No 93 teskage isolation - Rock Sp wift scored pttegea tube post ed accident sanpling 247 Indian Point 2 01/04/84 84-001 One Failed Contairment ASCO mot not specified no Replaced S0W we.e sone no 21 ctosed purge Specifi ed 247 Indian Point 2 11/27/84 84-022 Two not AFW Steam not mot mot Specified so Reconnected SOVs centrol mone we 09 Specified Specified Specifi powe-leads to AFW turbine ed S0Ws intet steam isolation vetwes 247 Indian Point 2 02/02/87 87-003-01 One Sluggish condensate mot not Dest;n no Enlarged Scv SOV controls None No 24 perferinen (storage Specified Specifi deficiency orifice and ACV. Slow ce tank ed (siring) cleaned closure isolation) regulator attributed to crifice size. Debris cou'd have also contributed. 249 Dresden 3 01/12/85 85-001 One Manuet mein turbine Sperry FSDG454 Grease No replaced Sov SOV controls mone Yes 04 operator Vickers 012A contaimiration e.u d trip 249 Dresden 3 08/07/87 87-013 One Colt Feedwater ASCO 8300 Shorted coit no Replaced S0w S0w controts mene Yes 01 FWRV air eperetor 250 Turkey Point 3 12/02/84 C4-031 One not Contairement Asco Not Not Specified 40 Reptaced SOV LER250/94-we 03 Specified isolation specift vetve 09,020 (nitrogen ed s@pt y) 250 Turkey Point 3 12/13/84 84-034 One Not CVCS ASCO Not Yes Replaced S0W S0W controts See no 02 specified (isolati o Specifi A0w. Ref. Comummts valve) ed Doctsamtt: LER 250/84-032, 251/8?-009,84-0 20 e e h r, w s .,n a -ra a ~~ . ~. -..
e Page No. 5. 06/07/90 SOLEWOID-OPERATED WALVE $AILURE DATA DOC PLANT EVENT LER Wo. OF FAILED SYSTEM MANUFACT emEL #00T REP CORRECTIVE CtpeqENTS
- EFEWEWCE TP/ FC NO. NAME DATE NUMBER TAILURES PART u0.
CAUSE FL ACit0N 00 CIA 4ENTS Olff 250 Turkey Point 3 01/13/85 85-002 One Clogged not not mot not Specified so Cteened air simiter mene so 17 SOY sir Specified Specified Spwifi fitters en worse 46: fitters ed this and other LER 250-84-034, simiter Sovs LER 250-84-031, in both units LER 251-84-020, 3 and 4 LER 251-84-009, and LER 250-83-016 250 Turkey Point 3 01/27/86 86-005 Two not pain stems ASCO 8316 1 internet no Reptoced 1 2 f.a.a.; mone Yes 09 Spelfied (MStV) interference, SOV, fuse SOV failures 1 t m t contact block pins discovered pins et fuse were daring testing. block. straightened Msty couldn't on other Sov. be closad 250 Turkey Point 3 08/03/86 86-031 One not Auxiliary /em ASCO 206-381 Meter entering No SOV reptoced simiter See Yes 03 specified ergency the SOY-occurrences: coment feedwater LER 251-54-020, and LER 251-84-009 250 Turkey Point 3 01/03/87 87-002 One Coit Comonent ASCO 8316 mot Specified no Reptoced S0v wone no 01 Cooling Mater 250 Turkey Point 3 09/13/87 87-023 One Intemet Steen Target 300525-Fautty wires No not specified mone mone Yes 06 wiring Generator Rock 1 going to Reed 5toudoun switch 251 Turkey Point 4 07/15/87 87-015-01 One Ground Contairument mot mot Deterioration No Cteened and Sor is s mene No 18 foutt Isotetion Specified Specifi of insuteting retaped wiring piece-port of (pressurirer ed tare from comectiens A0W anormat sa mting) ageing" 254 cued Cities 1 02/05/85 85-001 Two Corv6-etio HPCI sarksdete 178250m Fautty Repeir Failure of MPCI No 07 n tt MfW C2D4 tenminst terurinet turbine tripend power comection and connections reset SOVs tend vibration and secure wires to 50V housing l i l
,,.J .i.... i. r Page No. 6 06/07/90 SOLEN 0!D-OPERATED VALVE FAILURE DATA 00C PLANY EVENT LER NO. OF FAILED SYSTEM RANUFACT NODEL ROOT REP CDilWECTIVE COMIENTS REFERENCE TP/ FC NO. NAME DATE NUNDER FA! LURES PART NO. CAUSE FL ACTION 00CL91ENTS im7T 254 Oued Cities 1 04/03/87 87-006-01 One viring Nigh sortsdele 1019433 V6bratiervined Yes neptoced coits NPCI LER 85-001 No 07 connectio Pressure ACP1 equete on failed SOV inoperstpte. n to colt Cootant connectfJn/ine end three Replaced $0V Injection dequate others aits with support reptsced at newer modet, imits 1 and 2 etso adc>d wiring restraint to ett four SOVs. Yes 12 255 Pelisades 04/10/86 86-017-01 Three fait + Vetve Reactor farget 808-001 Metat showings Yes repeired 50Vs three incipients seat Coolant - Rock in valve seat and system testege (head vent) area. flushed to remove remaining metet shevings 255 Patisades 01/14/87 87-001-01 Eight Inadequot Contairvaent Not Not AE design No Isoletion None None No 14 isoletion(hy Specified Spnifi error logic nodified e isolation drogen ed logic monitoring) 259 Browns Ferry 1 07/03/86 86-022 Six incipients ECCS Rockwett/ Design error Remove air Potentist for No.14 Atwood stwty to overpressurizin Morritt affected g low pressure actuator systees &e to use of non quetified SOVs (six in each of three Browns Ferry units) 260 Browns Ferry 2 08/31/87 87-007-01 Potentist Loss of Conteiroient Not Not Jesign error Yes septoce 50Vs Use of None No 14 faitures att 3 SOV Drywel1 Specified Speciff with quotified non-tpJetified 50VS coutd toits funetion Contret Air ed ones prevent primery conteirament isolation. All 3 Browns Ferry units offected. G e
e Page No. T 06/07/90 SOLEN 0!D-UPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORNECTIDE CDFBIENTS REFERENCE TP/ FC 40. maME DATE NUMeER FAILURES PART NO. CAUSE FL ACTION DOCUMENTS OUT -2tG Browns Ferry 2 06/06/89 89-018 Che Valve Emergmcy Setem 812-6 Corrosion Yes Reptoced SOV Licensee No 12 seats dieset debris from tegraded EDG generator startirg air air system and air start system perforwed maintenance en it prior to event but dabris was believed to be there from before .261 H.B. Robinson 2 05/13/87 87-007 Two Not Not ASCO Not Inadequate Yes Instatt incorrectly None No 14 Specified Spacified Specifi instattations correct seats instatted ed of ctvutit conduit seats seats at entrance to several harsh environumt 1E quetified SOVs. Potential for moisture intrusion 261 H.B. Robinson 2 07/15/87 87-020 One Electrica Feeduster Not Not Water trapped No Wire was SOY is None Yes 11 I short (FWRV) specified Specifi in 50V repaired and piece-part of ed condolet weter removed FURV from the condutet. Other S0Ws examined for similar problems. 261 H.8. Robinson 2 11/05/87 87-028-01 Two 50v Dieset Not Not Internet weer No Replaced S0Ws Sov failures None No 18 Internets Generator specified Specifi caused venting Starting Air ed of starting air 263 Monticetto 10/25/89 89-032 One Loose ' Main steam Tighten No 09 terminst (MSIV) terminet screw screw and inspect simiter 50Vs
Page No. 8 06/07/90 SOLEN 0!O-OPERATED VALVE FAILURE DATA 00C PLANT EVENT LER No. OF TAILED SYSTEM MANUFACT MODEL ROOT REP CORRECTIVE CENSEENTS REFERENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART No. CAUSE FL ACTION DOCUMENTS OUT 265 ound Cities 2 06/28/85 85-015 One Not Reector Verse See Not Specified No SOV reptoced VGS-4422-U-10-3 None No 20 specifled 81dg. Vent. comment 1-38C System 265 Quad Cities 2 02/18/87 87-004 One Not Contairment ASCO 8317 " Solenoid Reptoced SOY SOV is No 21 specified vacutse rusted and piece-port of corrocW P vacuum breeker (reeson/ source air test not stated) cylinder 265 ound Cities 2 09/18/87 87-012 One plus two Not contairment ASCO 8317 Unknows Yes Not Specified SOV is LER 87-004 Ro 20 incipients specified vacutsu piece-port of Relief vacutse breaker air test cylinder 265 Quad Cities 2 12/10/87 87-020 One Not Main Turbine Sperry F3-SDG4 Net Specified No Splaced 50V None None Yes 02 Specified Control Vickers 54-0124 Fluid 265 Quad Cities 2 04/06/89 89-001 Or.e Turbogenerst No Webuilt SOV Failed SOV LER 87-020 Yes 21 or controts turbine mester trip solenoid 266 Point Beach 1 06/01/89 89-003 One Contairment ASCO 8302 Reptoce 50V No 21 Isolation (SG blowdown sanpting) 271 Vermont Yankee 08/18/87 87-009-01 Ne>t Specified Seat Autosetic ASCO 206-381 Dirt / corrosion Yes SOV cycled None None No 12 teakage Depressurira products from tion the air s w y 272 Salem 1 12/31/84 84-029 one Faulty feeduster ASCO Not Not Specified Yes Replaced SCW SOW is a None Yes 09 electrica (FWRV) Speciff piece-part of 1 ed FWRV connectio n and seat teskage 272 Salem 1 01/31/86 86-003 One Seat Feeduster ASCO Not Probably Yes Two SOWS were SOV is e mone Yes 12 teekage (FWRV) Specifi contaminated reptoced piece-port of ed air the FWRV. Dirt and moisture were dete.ted in air Iines causir's other associated faltures 4 e 4 ~-
Page No. Y l '06/07/90 SOLEWO!D-0PERATED VALVE FAILURE DATA i l DOC PLANT EVENT LER WD. OF FAILED SYSTEM MANUFACT MODEL ROOT REP FtlRPECTIVE C0pWIENTS REFEWENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART NO. CAUSE Ft. " TION DOCUNENTS GUT 2 72 Salem 1 02/20/36 86-006 One Broken ' Feedwater Not mot Instattation No Replaced wire None None Yes 00. wire (FURV) specified Specifi error and and checited -- ed vibration simiter SOVs 272 Satem 1 04/08/86 86-007 Eighteen Etectrica Post Not Net Design /instalt No Instatt 18 SOVs on None No 14. incipients t accident Specified Specifi ation required units 1 and 2 connector senpling ed error,inadequa connectors had inadequate s te comectors instaltation procedures 275 Diablo Canyon 1 01/02/85 85-001 Two SOY Main turbine Not Not Not Specified No Reptaced SOY None Yes 21 l " stuck (overspeed Specified Specifi open protection) ed a 275 Diablo Canyon 1 07/24/87 87-011 None Containment Not Not Procedurst No Perform Failur* to None No 22: isolation Specivied Specifi inadecpacies necessary verify ed verification. penetration Upgrade isolation procedJres subsequent to SOV replacement. 277 Peach Bottom 2 04/27/84 84-008 One Not Containnent asco 8320 Not specified No Replaced SOV Potentist None No 19 Specified Isolation existed for a (58Gt) single failure to have prevented the fulfilment of the safety function of the S8GT system 277 Peach Bottom 2 01/24/86 86-003 Two DC colts Main Steam Automatic Not Under No The failed DC Falture of 2 DC mene Yes 19 (MSIV) valve specift investigation solenoids were says in 2 Conpany ed replaced. seperate lines (AVC) Caused Closure of MSivs 277 Peach Bottom 2 05/29/87 87-006 Three Control room Piping No Reconnected Sample lines to No 20 ventitation/ configuration tubing to S0Ws three SOWS had radiation error pyty been connected monitoring incorrectly. Affected control rooms at both units 2 and 3 I +- . h-. m . -.2m -u m
i 'Page No. 10 06/07/90 SOLEN 0!D-OPERATED VALVE FAILURE DATA l DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT Mt2EL ROOT REP CORRECTIVE CXpWENTS REFERENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION 00C1pE1TS OUT 277 Peach Bottom 2 10/05/89 89-023 One sinding Mein steam Automatic 6010-20 Inedegsete No Reptoced SOV Reference LERs See Yes 27 of SOV (MSIV) Vatwe manufacturer's and revised 277/86-003, comments stug Conveny instattotion instattotion 278/85-018, i (AVC) instructions and 278/86-016 . meintenance procedures 278 Peach Bottom 3 09/30/85 35-015-01 One Leaked ADS backte Terget Not Not Specified Yes Replaced SOV Previous See No 03 nitrogen Rock Speciff with an simiter Conuments ed upgraded one occurrences reported in cERs 277/85-01 7 l and 278/85-05 278 Peach Bottom 3 07/11/84 85-018 One DC colt Main steam Autcmetic Not Reason for Yes Task force DC SOV feiture None Yes 01 l (MSIV) Vatwe Co. specifi coil failure recomended coupled with l ed not specified testing of DC somentary toss ( solenoids more of AC power often and resulted in analyze cause MSIV closure of future failures. 278 Peach Bottom 3 07/19/86 86-016 One Colt Main Steam Automatic Not Reason for Yes The de colt on simiter reactor See Yes 01 (MSIV) Vatwe Specifi coit fatture each MSIV's scrams in 1985 comuments Corp. ed not specified 50V was and (AVC) reptoced. 1986(defective de colt coupled with oc power interrtetion): Leas 278/85-018, 277/86-03 280 Surry 1 03/28/84 84-007 None Unspeciff reeduster Maintenance No Recennected IA Instrument air No 06 ed (FWRV) had been done lines to lines were l without proper SOV connected to approved ports the wrong ports i l procedures of 5 SOVs at inadegJete Surry tmits 1 post and 2 maintenance testing I e 4 e S e e-
i.. Page No. 11 06/07/90 SOLENDID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL #007 NEP CORSECTIVE CopeqENTS REFERENCE TP/ FC NO. CAUSE FL ACTION 00CtpENTS OUT NO. NAME DATE NUMBER FAILURES PART 280 Surry 1 11/12/87 87-031 One SOV Contelruent Masoneite 3500 !sproper No Secured Sov wiring to No 09 wiring isolation - n (SOV series instettetton tawyeci8ied 50V caused blocked trispecif f usechanical isolation ed) binding of valve contairment operator isolation valve's operator 281 Surry 2 01/27/88 88-001-01 Two SOV Contairment Target 86V-001 Cause of SOV No Repeir or Electricians None No 26 teskage isolation (pr Rock /ASCO /206-38 teskage not replace SOVs trying to Isolate teeking essurizer 0 spm ified, Cause of wrong Scvs tifted vapor space Iead tifting: wrong Iceds sampting) electricat meintenance "personnet error" No 12 231 Surry 2 02/02/88 88-002-01 Two Seat teactor Vatcor VS26-56 Ispurities is 50Vs replaced teakage coolant 83-19 reactor sampting coolant systes isolation water prevented couplete seat closure 285 Fort Calhoun 05/01/86 86-003-01 Two Failure Weste ses Not Not Persomet Non Return 50Vs to fait closed None No 22 e correct Sovs had been positions Specified Specifi error ed fatture changed to fait positions open, resulting of SOVs reversed in volume control tank teekese to auxitlery building. 286 Indian Point 3 02/11/87 87-002 One Coit Containment ASCO 8308 Not Specified Yes The feited The design of LER Yes 11 solenoid vetve no. 34 static 85-001-00 teekege reptoced with inverter was controt one of e laproved to higher ottow isolation temperature of single design. 3 branch circuits simiter SOV if a short colts were circuit etso replaced. develops.
Page No. 12 06/07/90 SOLENDID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MWEL ROOT REP CORRECTIVE COMIENTS REFERENCE TP/ FC NO. NAME DAtE NUNSER FAILURES PART
- 0.
CAUSE FL ACTION DOCUNENTS GJT i 293 Pilgrim 07/19/88 88-021 Four incipients Potentist Primery ASCO 8320 Design error No Reptsee SOVs Feiture of Wene No 13 for conteirveent, and with ones pressure exceeding controt rm,+ NP8320 reted for regutetor would MOPD turb btdg higher MOPD result in timits NVAC/SGTS incperability of 4 SOVs due to exceeding MOPO timits 293 Pilgrim 01/27/89 89-004 contairunent ASCO NP8320 Repeired teoks Failure of 2 LER 89-002 Yes 21 isotation and reptoced 2 A0Vs due to air SOVs system teoks. 2 SOVs were replaced es a precaution against exceeding MOPD timits of the SUVs 293 Pitgrim 05/03/89 89-015 One Coit Main Steam Automatic 6910-02 " Random No Reptaced SUV Yes 01 (MSIV) valve O failure" assembly Corp. (AVC) 295 Zion 1 08/08/85 85-029 Two " Stuck" EDG building Not Not Not specified Yes Reptsced SOVs 40 such valves LER No 05 pitot ventitetion sp&ified specifi used in both 304/85-015 valve ed snits. CesunorMuode feitures foted during testire. Additional CMrs ocurred next doy et unit 2. 295 Zion 1 01/12/89 89-001 One failed to Ventitation ASCO 8320 Weekened coit Yes Reptoced SUV LER 89-001 No 01 shift (service water building) O e b n
Page No. 13 06/07/90 SCLEN0!D-OPERATED VALVE FAILtJRE DATA DOC PLANT EVENT LER NO. Of _ FAILED SYSTFM 9tueU7ACT MODEL ROOT REP CURWECTIVE Cop 5EENTS REFERENCE TP/ FC D00MENTS OUT No. CAUSE FL ACTION No. NAME DATE IIUMBER FAfLURES PART 298 Cooper 08/18/86 86-015 One Not Reactor mot Not Not Specified No Not Specified None mone eso 21 Specified Recirculatio specified Specifi n System ed Muttiple ASCO 8320/NP Design Yes Reptaced SOWS See section See No 13 302 rrystal River 3 01/05/89 89-001-02 None 8316/83 error-Moro with others 5.1.3 of this coments systems having higher report for 20 MOPD rating additional info. Reference documents: LER 78-054, 83-023, 88-013 Contai vamt ASCO 8320 Design error Replace SOV 8 S0Ws were See No 14 302 Crystal River 3 04/07/89 89-012 coils with affected. commts isotation colts having Reference (RX cavity correct docummts: LER cooling te v rature 78-054, 83-023, system) ratings 88-013, 89-001 15 Inadequate Modified SOV Reactor 302 Crystat River 3 04/18/89 89-015 seismic stworts coolant psp seat bleed instattation No 09 off Modified power Intermingting Design error Electrica MVAC, supplies of TE and 302 Crystal River 3 07/26/89 89-034 1 power containment nort-TE power supplies isolation, sourccw ks Sovs Main steam 304 Zion 2 07/11/64 84-015 Not Specified Internet Main steam Keane 51-170 Licensee could No Three SOWS to None mone No 26 (MSIV) not find cause be replaced teakage (MSiv) of failure with erwirementatt y w atified S0Ws 304 Zion 2 08/09/85 85-015 Two
- Staxk"pi EDG building Not Not Not specified Yes The valves Comunon-mode LER No 05 were replaced. failures foemd 295/85-029 tot valve vent specified speciff during testing.
ed Also occurred on unit 1 the previous day. 40 such valves on smits 1 and 2. I L
lg, Page No. 14 l 06/07/90 l SOLEWOID-OPERATED VALVE FAtttRE DATA i DOC PLANT EVENT LER NO. OF FAILED SYSTEM stWUFACT IEODE'. ROOT REP CORRECTIVE COIGEENTS REFEWENCE TP/ FC se0. NAME DATE NUMBER FAILtRES PART NO. CAUSE Fi. ACTION DOCUME4tS OUT l i l 304 Zion 2 02/03/S7 87-001 One 0-Ring Mein steen Chicago NSV1-16 seenufacturing no Reptsced SOW sone mene Yes es j (MSIV) Fluid -C-XP defect or Power demoge dJring j instettetton i 305 Kewaunee 07/02/84 84-013 One Colt Auxiliary J& nson V-24 not specified Yes The Johnson SOV feitures 82-03,28, no 01 building valves were to resulted ite 81-34 special be reptoced initleting ventitetton with ASCO safeguards leP8323 Sovs es e gipment. 59 they failed. such SOWS remaining would be reptoced with ASCOs.ed at next outage 305 Kewounce 12/16/84 84-020 One Coit Auxiliary Johnson V-24
- Burnt out" Yes The Johnson Due to LER 84-13 to 01 butIding coit, root SOV was repetitive specist cause not reptoced with failures of ventilation specified en ASCO these Johnson NP8320.
SUVs, they ware ett being reptoced with ASCO NP8320 SOWS on en es-fell besis 305 Kewaunee 02/11/85 85-005 One toit Auxiliary Johnson V-24 Coil " burnt Yes Reptoced SOV Due to LER No 01 building out," root with en Asco repetetive 84-013,020 specist cause not feitures of ventitation stated these Johnson SOWS, they were ett being reptoced with ASCO uP8320 t 50Vs on en es-fell besis. 305 Kewounee 11/28/87 87-012-01 Two fatted plus Tailed to Conteirment ASCO NP8314 Design error. Yes Reptoce SUVs See Section wone no 13 58 incipients shift Isolation-Pz Conditions and correct 5.1.3 of this r exceevied $0Vs' regulator report relief,melte-MOPD timits settings so up.RCDT that MOPD discharge ratings will not be exceeded 4 4 9 6
Page No. 15 06/07/90 SOLEN 0!D-CPERATED VALVE FAttt*E DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL R30T REP CONNECTIVE C0pu1ENTS REFEhtNCE TP/ FC NO. NAME DATE 8PJNSER FAILURES PART N0. CAUSE FL ACTION DOCUENTS IRif 305 Kewaunee 05/28/88 88-007-01 Three plus 7 Failed to Conteirusent ASCO NP831. Manufacturing No Cleaned and Initiated an LER No 05 incipients shift Isolation error refurbished extensive root 87-012-01 (pzr relief, (unauthor red the effected cause analysis. N eup use of SCVs See Section isolation) W orrect 5.2.4.1 of this subricant) report. 309 Maine Yankee 08/10/86 86-005-01 One Graund Cardon Fire Chemetron 5-020-0 Not Specified No Replaced SUV SOV falture No 21 'ault Protection 0 74-8 tripped Cardon system system power supply breaker, thereby disabling the Cardon system. 309 Maine Yankee 05/23/88 88-005-02 Four incipients Not pra /chargin R.C. 670WA24 Design error No Modified SCVs in high None No 16 Specified g pump Laurence DCSW system rad. fields not suction vent errviron. <F>at. failure could cause uncontrotled release of radioactivity to non qual. systems. 311 Setem 2 05/22/89 89-011-01 None Main steam inadegante No Modified Testing Yes 14 (isolation surveittonce testing deficlencies valve) testing circuitry would prevent 1 detection of 50V fallure Deficiency existed at unit 2 also 313 ANO 1 05/06/85 88-001 Two Lifting Post Target 80E'dO1 Design error No SUVs were Incorrectly LER No 06 of accident Rock /8tS-00 reoriented oriented SOVs 368/88-001 plunger sampting Corp. 64 correctly could open upon (spurious smatt increases actuation in ) backpressure. See Section 5.1.4 of this report
,w'b =tB Page No. 16 06/07/90 SOLENOID-OPERATED VALVE FAILURE D4tA DOC PLANT EVENT LER No. OF FAILED SYSTEM MANUFACT MODEl. ROOT REP CORRECTtvE COMENTS REFERENCE TP/ FC No. NAME DATE NUMBER FAILURES PART No. CAUSE FL ACTION 000MENTS OUT 317 Calvert Cliffs 1 04/01/87 87-007-03 Four incipients (Anquetiff Atmiliery Not Not Design error No Deficient Two SOVs on None Yes 28 ed Feedwater Specified Speciff electricot each unit foted electrica ed connections to have t were upgraded inedequate (EQ) connector with EQ electricot s quetified ones connectiens 317 Calvert Cliffs 1 08/22/89 89-015 0 todine Design error Reptoce with 50v failure No 15 fitter (o list seismicalty could prevent dousing ctessification quotified 50Vs iodine filters system ) from performing their function 317 Calvert Cliffs 1 11/13/89 89-020 0 Satt water Design error Reptoce with & Sovs in No 15 cooling (0 list seismicotty safety system classification qualified SOVs not able to ) and power withsta
- sources seismic event power sources for 5 safety-related SUVs not seismicatty quotified 318 Calvert Cliffs 2 09/05/86 86-006-01 One Seat Mein Steam ASCO 8300 Not specified No StN Int-rnets None None No 03 testege (atmospheric were replaced dtsp) 321 Match 1 12/07/85 85-043-01 Nu6er of falted Seat Contairment Not Not Normel Yes Leeking None LER 84-017 No 18 Sovs not spec tenkage isolation specified specifi equipment use volves in 42
-suttiple ed or wear penetrations systems repelred,rebul At, or replaced. 321 Hatch 1 04/15/87 87-004 One incipient Wein controt Not Not AE destyi No Redesign mein Single 50v None No 14 room Specified Specifi deficiency centrot room feiture could enyironmenta ed enyirormenta1 compromise t control control system control room hability e i e e 6 o rw - e ,-w-,
Page No. 17 06/07/90' $0LEN0!D-OPERATED VALVE FAILURE DATA DOC PLAWT EVENT LER
- 0. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORWECTIVE CGSIENTS ~
REFEREuCE TP/ FC NO. NAME DATE Wlm8ER FAILURES PART
- c.
CAUSE FL ACTION DOCUMENTS inff 321 natch 1 03/18/87 87-005 Two
- 1. Missing Contairment ASCO NPS321 Unspecified Yes 1. Instetted a 2 W LER Wo 90 lock ret ventilation arissing lock faltares. (1 85-015-01
- 2. Stuck nut./ 2. No cr A by plunger corrective missing tock action tak.,e nut on 50V, 1 on stuck SOV caused by stuck because it SOY plunger) tested okay subsegJent to failure.
322 Shoreham 11/15/87 87-009 0 Conteirnent ASCO 206-832 Design error, Reorient Sovs C-.O mo 08 isolation SOVs were to correct failures having (RX building 206-380 oriented positions potentist to stancby incorrectly (vertical vs. prevent ventitation) horizontat) fulfillment of safety function 323 Diablo Canyon 2 08/14/85 85-019-01 Three Incorrect Main Steam Not None Personnet Yes Replaced SOV Undetected SOV LER 85-014 No 07 wiring to (MSIV) Specified error (incorrec failure caused SOV t undocumented 5 month loss of wiring change) I train of ESTAS a;tetion of MSIVs 323 Diablo Canyon 2 12/21/85 85-022 One open Feedeater Not Not Improper No The wiring SOV is a LER Yes 09 t circuit specified Specifi wiring connection was piecepart of 275/85-030 ed instattation property the FWRV and tuuped reteriminated junctico box other simitar SUVs' teratinations were inspected. 324 Brunswick 2 09/27/85 85-008 Three Dise-to-s Main steam ASCO 8323 Mydrocarbon, no Reptaced Sovs comanon-mode more no 12 est (MSIV) water and high failures. See sticking temperatures Section 5.2.3.1 caused of this report. degradation of seat materlat.
Page No. 18 06/07/90 SOLEN 0!D-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORRECTIVE COBUIENTS REFERENCE TP/ FC Wo. NAME DATE NUMBER FAILURES PART 11 0. CAUSE FL ACTION 000pIENTS OUT 324 grunswick 2 10/15/85 ES-011-01 Two DC colt Mein Steen ASCO NP8323 Licensee No Reptoced SOVs. None None Yes 01 (MSiv) suspected Estensive chtoride fatture corrosion anetysis initiated. 324 Brunswick 2 01/02/88 86-001-05 Four Failed to Contairument ASCO Stitt under Yes Replace SOVs. Four previous Yes 19 shift isoL/drywel investigation. Performing similar i floor and Fotsid debris extensive failures had e@ mt drain and oil film feiture been steps on one 50V. onetysis experienced Suspect high tem eratures from setf heating of energized SOVs 324 Brunswick 2 06/17/89 89-009-01 One Falted to Drywett ASCO Not Suspected that No Reptaced SOV Entensive es 12 shift purge and speciff foreign anstysis of vent ed porticulates root cause was found in the not totally SOW had conclusive attacked elastomeric perts of the $0V 325 erunswick 1 02/28/87 87-005-02 Two Discs contairument Valcor VS2645-Not Specified No Reptoced SOVs SOV teskoge hone No 03 isotation 5683-14 fotrid during LLRT 325 Brunswick 1 07/01/87 87-019 one Stuck Main Steam Target 1/2-SMS Excess Lectite Yes Refurbished See Section LER No TT plunger (MSRV) Rock -A-01 used by SOV -5.2.2.2 of this 87-020-01 I manufocturer*s report fleid rep 325 erunswick 1 07/03/87 87-020-01 Four Stuck Main steam Target 1/2-545 Excess Loctite No Reptoced SOVs See Section LER 87-019 No 17 plunger (MSRV) Rock -A-01 used by 5.2.2.2 of this manufacturer *s report field rep e 4 -~ e w ---a
w, Page No. 17 06/07/90 SOLEN 0!D-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT M(BEL ROOT REP CORRECTIVE CtpHENTS REFEaENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART Wo. CAUSE FL ACTION DOCIDIENTS IRIT 327 Sequoyah 1 05/18/84 87-020 Not Specified Not Not Not Not Design erroe No Plant TE SOVs were mene No 14 - Specified Specified Specified Specifi modifcations not protected ed to protect from water vulnerable 1E spray which . equipment could emenete from pipas which were vulnerable to an SSE 328 Sequoyah 2 08/30/84 84-014-02 One Seat Feedwater ASCO 8320 Design Error No Replaced SOV An incorrectly None No 13 teskage selected SOV faited when put in service where its MOPD timits were exceeded 328 Sequoyah 2 06/11/88 88-026-01 Two Incorrect Auxiliary Not Not Inadequate Yes Recomected Incorrect None No 07 external feedwater Specified Specift maintenance SOVs correctly external wiring wiring levet ed configuration to 2 SOVs controt controt 328 Sequoyah 2 06/06/88 88-027-01 Not Auxiliary Not Not Inadequate Yes Reptaced None None No 07-Specified feedwater Specified Specifi electrical diodes missing ed mainte ence from externet circuitry connecting 2 SOVs 331 Duane Arnold 01/10/84 84-004 Two stockage Standby ASCO 8316 Foreign Air path No 12 of filtration meterlat in eteened internet instrument air passagewe y 331 Duane Arnold 01/28/85 85-002-00 One Diaphragm Migh Skimer L2D8515 End of No Replaced 50V None None No 17 pressure Electric 0 tife/ excessive coolant time between injection maintenance 4 .m .m. .-.m--. . m-m...
P page no. 20 06/07/90 l SOLEN 0!D-OPERATED VALVE FAILURE DATA I l I DOC PLANT EVENT LER N0. OF FAILED SYSTEM 9tdRJFACT MODEL ROOT REF CORRECTIVE COIR 4ENTS REFERENCE TP/ FC NO. NAME. DATE muMBER falLURES PART No. CAUSE FL ACTION 00CtmEWTS OUT 331 Duane Arnold 05/27/88 88-095 One Fire Electro-M 2010008 Design error No Reptoced SOY Licensee had mone so 14 h iffed Sippression enuet 3 and inodaquete tygraded SOY (Chemetro post with en n Corp.) maintenance incorrect one. I testing Deficiency was j not fotrid during post j maintenance testing. 331 cuene Arr.otd 03/05/89 89-006 One Colt Mein stenue ASCO WP3323 Moisture me Reptoced SOV. 7 other simiter Yes 11 (MSIV) intrusion from Tightened SOVs were i steem teek / enetosure rhject to inadequete covers of eoisture to w ing of other simiter intrusion i enetosure 50Vs. feiture due to festeners compon-mode torqueing deficiency 333 Fitzpatrick 08/20/85 85-022 One Electrice Msin steem ASCO wot Maintenance Nc Sovs reptoced AC colt had mone Yes 09 I foult (MSIV) Specift personnet and rewired been comected M error in correctty to DC source l enternet and DC colt hed wiring been connected to AC source l 333 Fi rpetrick 11/22/85 85-027-01 One SOV Mein stears ASCO NP8323 Brass stiver no Cleaned /refurb MSIV unable to mone no 12 snable to (MS!v) d.se to cross ished SOV close i seat threeding air check other property line fitting for simiter problem 333 Fitzpatrick 08/03/89 89-013 mone Contalment Design error Cerrect wiring no 07 isolation error 334 Beaver vetLey 1 06/07/88 88-007 One Not Dieset Johnson Not Not specified to Reptoced SOY EDG eir start None No 22 Specified generator Specifi Sov felled air start ed I 336 Mittstone 2 12/31/86 86-021 Two - Broten Reactor vetcor VS26-60 Suspect no Reptoced 17-7 Prior to event mene no 03 springs Cootent Read Engg 42-3A hydrogen PM springs of these SUVs had in SOWS vent Corp. embritttement eit simiter been teeking Volcor SOVs and had been isolated e a 4 m ~.
Page so. 21 06/07/90 SOLEWOID-OPERATED VALVE FAILtRE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT WODEL ROOT PEP CORRECTIVE CGUIENTS-WEFEnENCE TP/ FC NO. NAME DATE Nt#WER FAILURES PART mo. CAUSE FL ACTION DOCl#EWTS IRff Yes 82 336 Mittstone 2 01/02/87 87-002 One Diephrope main ASCO 8262 mot specified Yes Inspected and reptoced teekege feedwoter (FWRV) 338 North Amo 1 02/02/86 84-005 6 fatted and 54 Electrics Conteirment Veteor Volcor Inadequete Replaced 6 Sovs felled 09 incipients t isoletion and ASCO 526seri conduit felled Sovs and 54 sovs (moisture ) hydrogen es seating and sealed ett were instetted intrusion controt/pess methods did deficient incorrec^.ty in not meet ofrs cordsit seats both units ) ) specs to meet IEEE-324 quelifications 338 worth Anno 1 11/23/87 87-020 Two not mein Steen Ccpes-Vut Wet not specified so unter To prevent mone so 02 Specified (Atmospheric con Speciff induction recurrence of Dimp Volves) ed circuits were this type de-energized event, en in order to evolustion to stort the instatt condensete additionst pumps and tewet switches begin will be secondery performed. system recovery actlens. 338 worth Arm 1 01/08/88 88-002 One por Condenser mot not mot specified Yes septoced Sov sone None Yes 21 specified waterbox Specified specifi ed 338 North Anno 1 03/11/88 88-011 Nine sluggish Conteirwent AsCO WP-f Design error Yes Reworked Sovs feiture to LER mo 14 vacuum operation isolation series to meet fotlow 339/87 manufacturer's manufacturer's 01 instructions instettation instructions modifled the 50vs' performance end g etification.
Page No. 22 06/07/90 SOLEuctD-OPERATED VALVE FAILURE DATA DOC PLANT . EVENT LER NO. OF FAILED SYSTEM MauttACT NODEL ROOT
- c.7 CORWECT!vE CCIWEEWTS WEFEnEWCE TP/ FC NO.
NAME DA7 NUMBER FAILURES PART NO. CAUSE FL ACTION 00C1pIENTS (RFT 7 338 worth Anno 1 03/15/88 88-012 Ow Not Couponent ASCO mot Not Specified Yes SOW from None LER 88-011 to 02 specified Cooting speciff 1-CC-TV-1934 l Water ed mes instetted on 1-CC-TV-1038, and the 50V fren 1-CC-TV-1038 was refurbished and instetted en 1-CC-TV-103A 338 worth r.nna 1 07/19/89 89-014 1 0-ring turbogmerat Parker-Ne MRFN16M 0-ring pinched No Reptoce 0-ring SWesental LER 88-013 Yes 03 or (EMC) nnefin M0834 during 50V info obtained refurbishment freum licensee by turbine 5/16/90, M.L. uenufacturer's Ornstein/ meintenance C.W. Atten team 344 Trojan 04/16/87 87-009 Not Reactor mot mot Design /instatt No Reptaced None wone no 28 specified coolant Specified specifi ation error sptices which (PORV) ed did not meet EQ instettotion requireements 346 Davis-Besse 09/11/84 84-013-01 One mot noin steen Controt not mot specified Yes Reptoce or 30V is a wone so 21 Specified (Atmospheric CeW Specifi refurbish SUV piece-port of Vent) Internati ed the atmospheric onet vent welwe's air-eperated centrotier ~ 346 Davis-Besse 01/03/86 86-006-01 Thirty two colt Not ASCO mot Failure to Replaced SOW Colts en EO Wone no 17 incipients specified specifi perfonn coits SOVs had been ed preventive in service meintenance beyond their when required esotified lifetime s O 9 9 4 s m ._.2__ _ __._ _. _... _ _,
e Page No. 23 06/07/90 SOLEWO!D-OPERATED VALVE FAILUWE DATA DOC PLAWT EVENT LER WO. OF FAILED SYSTEM OWWRN ACT 8EODEL 900f PEP CONNECTIVE COMENts REFEREuCE TP/ FC DOCUMENTS OUT No. CAUSE FL ACTION NO. NAME DATE tRMBER FAILURES PART 346 Davis-8 esse 12/07/87 87-015 One Sov Instrument Asco 1179237 mot Specified no Reptoced SOV, Feiture of SOV mene Yes 21 instrument air caused toss of vented air dryer dryers instrument air reptoced with air /reoctor tJpgraded ones trip. 0-rings on severet 50Vs in turbine bypess system etto foted degraded 348 Farley 1 01/18/87 87-005 Two not conteirament ASCO 8316 unknown no 1 SOY closed Redurufent SOVs None No 20 on additlenet in one Specified Isolation ettewpts. penetration (conteirveent 19boerd 50V to failed to close susp be inspected discharge) sthsettaent to shutdcen. mot mot Root cause of no Att accessthte 84 Sovs M nh mone so 28 348 Farley 1 07/21/87 87-012 84 incipients et inadaquet mot SOWS'instettet imit vert fc M each init e specified Specified Specifi inedequate electrice ed splices and ions modified not to be terminatiens to en orproved instetted in not stated E0 splice and accordance with t instett. termination EQ requirements (splices / configuration (splices and terminets on a priority junction bon ) besis. connections) 352 Limerick
- 05/09/88 88-017 one teekege Reector 8tdg ASCO 8316 mot specified no Reptoced 50V Licensee could mone Le 20 not determine
-stug ventitetton coupe of $0V stuck in feiture. mid-posit Cetted e ion acceponent fatture of ta*ncert cause" 352 Limerick 1 03/14/89 89-019 0 Electrice RX building Design error seeted Potentist for no 07 1 ventitetion (EC). electricot commen-made inadequete conduits feitures failure /m conduit oisture seating for intrusien MELS potentist environsumt
Page No. 26 06/07/90 SOLENOID-OPERATED VALVE FAILURE DATA l i 00C PLANT EVENT LER NO. OF FAILED SYSTEM MenUFACT MODEL ROOT REP CDRRECTIVE CopWEENTS REFERENCE -TP/ FC NO. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION 000.9EENTS CUT 354 Hope Creek 08/28/86 86-063 12 incipients Not conteirment ASCO NP8316 Design error No Reptoced ett Felture of None No 13 Specified Atmosphere tuelve 50Vs non-0 Controt with ones regulators l j having a could have higher MOPD caused fattures rating. of the 50Vs. 354. Hope Creek 02/24/87 87-018-01 One Failed to Main Steam Automatic Not Foreign No Reptoced Foreign LER No 03 shift (MSIV) Valve Specift meterlat feited 50V arvi meterlat in 87-03T,038 Corp. ed inside SOV its manifold SOV. Plunger in (AVC)
- body, essenbty.
SOY not per manufacturing Replaced 7 design defect, and Sovs for other (incorrect inedequete MSivs. Sent length), instattation failed 50V to mounting screws supplier (GE) on jtmetion box for anstysis were loose. 354 Hope Creek 10/10/87 87-067 One Failed to Mein Steam Target Not Inodequete No The Failure caused None No '12 shift (MSRV) Rock Specifi protection of melfinctioning by intrusion of ed MSF't during SRV and its san 6 testing pt ar,c SOY piece-part grit which was construction were replaced used during in kind. plant construction 361 San Onofre 2 01/09/86 86-006 Two Coil Feeduster Not Not Moisture No The velves None None Yes 11 specified Speciff intrusion - were replaced ed fautty conduit and visuet connection inspectiens made of the i conduit I connections of simiter Sovs 361 San onofre 2 12/17/87 87-031-01 One Corrosion Mein Marotta Mv233C Inedequete Yes Reptaced SOV, Weter and Ltt Yes 12 of power Feeduster Scientiff / maintence terminst foreign 206/86-006 teads and MFIV) e PV2380 instructions block,and meteriet terminet Controts power teeds. Intrusion block Inc. Seeted conduit (inodequetely connections seated corukrit proper 1y. connection) 9 a-m. .m ,m
s Page No. 25 06/07/90 SOLENOID-OPERATED VALVE FAILtmE DATA DOC PLANT EVENT LER NO. OF FATTED SYSTEM MANUFACT 842EL ROOT REP CORRECTIVE CtpWEENTS-WEFEWENCE TP/ FC. No. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION 000.puENTS OUT I l 366 Match 2 09/21/84 84-021 One casket usin Steam ASCO Not Not Specified so v ptaced None None Yes 03 (RSIV) Speciff-gasket ed 366 Match 2 01/20/88 88-004 Numerous Leakage contairment Targat 75F-009 Inadensete No Reverse See Section LER No 08 isolation Rock /7567F instructions / orientation of 5.1.4 of this 366/86-020 (many normat use and many SOVs/ report systems) wear replace falted e-rings 366 Match 2 02/12/88 88-007 Twe!ve Not Conteirnent Terget 73K-001 Inadequate. No Reversed See Section None No 03 Specified Isolation - Rock /75F-00 instructions / orientation /fo 5.1.4 of this l Torus 9 design r unit one report Dr m it deficiency instatted Vacutse stronger Breaker springs 368 ANO 2 04/24/87 87-003 Two Seat Reactor Not Not Seat teakage No Replaced 50v Concern for None No 03 teskage Coolant Specified Specifi and installed teak causing (pressurizer ed a collector corrosion high point for any future damage to other vent) teskage ca p ts 368 ANO 2 04/29/85 88-001 2 Leakage contairment Target 80E-001 sackwerds Reinstatted See section No 08 isolation Rock instattation Sovs in 5.1.4 of this !pess) due to rec' sed report for inadequete a + wtr; ion additionet info instattation instructions 368 ANO 2 02/16/89 89-003 0 Containment Target 74F Design error-Ref tbished valve had No 14 isototion Rock incorrect Sr.. Checked exceeded EO (hydrogen assessment of r.hers ferr life 6 years anetyrer SOY similar design prior to sempting) life-falture error discovery of to account problem for heatup & e to energiration 369 McGuire 1 07/23/84 84-023 One Seat Mein . Borg mot Hydroutic No Adjusted SOY None None Yes 03 deformati Feedwater Warner Speciff fluid was and modified on ed teeking systein 1 m m s. - m
Pege No. 26 06/07/90 SOLENotD*0PERATED VALVE FAILURE DATA DOC PLAWT EVENT LER NO. OF FAILED SYSTEM MANUfACT MODEL WOOT REP CORRECTIVE ComEWTS REFERENCE TP/ FC NO. NAME DATE NLMBER FAILURES PART NO. CAUSE FL ACTION DOCUMECTS OUT 369 McGuire 1 09/19/85 85-028 One plus three Cable Post vetcor 526-529 Persomet so Att four Simiter volves mone no 11 incipients terminati accident 5-45 error vetves were checked at Unit on seepting (instellation
- repeired, 2, and found to seating not performed resented.
be okay per Wiring on ett instattation ether volcor specification) 526 series SOVs et station to be segraded and seats reptoced 369 McGuire 1 04/15/87 87-009 One system Main turbine not not modification no Change System mone 'es 00 perturbet Specified Specift of design and meintenance operation logic ton ed meintenance schedule to end time of evoid testing preventive iAile et meintenance had power. beenchanged. soth factors contributed to e reactor trip. 370 McGuire 2 06/24/85 85-318-01 Two (of the some Colt and Main sorg-Wern mot 1-colt no 1-repieced Second felture mone Yes 01 SOV) shert feedueter er Specifi fei1ure - not SOV. 2-dried occurred prfor circuit ed scw ified. 2-water from to cooptete short circuit
- SOV, instottotion of l
- weter sprey electricot box reptocement SOV onto open electricot box l 370 McGuire 2 08/27/86 86-017 One Coit mein sorg mot mot Specified Yes SOV colt was None LER Yes 01 Feedwater Werner Specift reptoced and 85-018-01 ed originnt coil was sent to the manufacturer for anstysis. 373 LaSette 1 08/29/84 84-051 One SOV (3 Electrice Main stese Crosby IMF-2 Cause of short no Reptoced SOV Caused SRv to mone no 11 i sent f tmetions) t ground (MSRV) Velve to ground not lift three spectfIed times O l l I-
..Page No. 27 06/07/90 SOLENOID-OPERATED VALVE FAILURE DATA rAC PLANT EVENT LER WO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORRECilvE C085EENTS REFEREaCE TP/ FC DoctpeEWTS OUT WO. CAUSE FL ACit0E NO. NAME DATE NUMOER FAILURES PART 3 73 LaSatte 1 02/02/85 85-008 Tour Diaphrase Reector ASCO 8316 Diaphregas Yes Rebuilt S W s, vitt change no 03 s . building tost their cycling Sovs to esseleer ventitation resitience frequency to quetIfied be increased NPS316 w.det 3 73 LaSatte 1 03/12/87 87-013 Six incipients not Main Steans mot not nigh drywett no anetyre Three Sovs mone no 10 Specified (MSRV) Specified Specifi temperature effects of declared ed high dryvett inoperable. temperature Three SOvs suspect due to high local teaperatures 374 LaSatte 2 06/08/84 84-033 one plus many Passagewa Containment ASCO 206-832 Sov was Repositioned Other simiterly no 08 incipients y blocked isolation impropert y SUV effected S0vs were positioned repositioned or replaced 374 LaSatte 2 11/20/84 84-076 One Colt Turbine Not mot Juretion box wo Replaced SOV wone Wone no 11 Steam Bypass Spacified Speciff was futt of ed water of unknown origin 374 taSatte 2 07/31/86 86-013 None - Many Electrics CRD, RCS ASCO See Design error Yes Repelred all 1E equipment LER 86-072 No 28 incipients t
- recire, consnent affected used electrical unquetified connectio RCIC, s
terminations electrical ns service to meet cormections. water, fIoor quotification 50v model nos. drain, air requirements Mv4-206, NP206, NP-8320, NP-8323 3 74 LaSatte 2 01/17/87 87-002 One Leskage Feeduster volcor v52660- Root cause of Yes Refurbished Sov body and mone no 12 5292-16 corrosion, Sov stem corroded, Sov fitted with dirt and o-ring dirt, and deformetion o-ring was not stated deformed
~Page No. 28 06/07/90 SOLENCID-OPER!TED WALVE FAILURE DATA 00C PLANT' EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORRECTIVE COMENTS REFERENCE TP/ FC WO. NAME DATE NUN 8ER FAILURES PART NO. CAUSE FL ACTION DOCUBENTS tRff 382 Waterford 12/11/87 87-028 one sov Mein steam fluid 7WRP477 Not specified No Reptoced sov sov feited None Yes 05
- stuck (MSIV)
Controt 4-600K8 during testing. opm" inc. 65 LER noted previous tnrelated SOY feiture due te open colI. 387 Susquehanne 1 02/25/84 84-010 one sov Main steam Not Not Not Specified No Reptsced sov sov stuck epen Mcne Yes 05 " stuck (MSRV) Specified Specifi causing SRV to open" ed remain open 387 Susquehanne 1 06/13/84 84-044 Severat
- Discs, Control Rod ASCO Nb-176-Contamination Yes Refurbished See Section hone No 12 repetetive seats Drive 81t of the air sovs, upgraded 5.2.3.3 of this failures system and disc meterlat report etevated from tenperatures polyurethene to viten 387 Susquehanna 1 07/06/87 87-023 One Colt Contairemmt Circle Not
" Burned open" Yes Reptsced coit open colt found Wene No 01 Vacuum Seat Spei(fi coit on same vacutse Reisef Controts ed breeter in 10/82. A unit 2 weevum breaker else had a simiter Circle Seat Sov colt feiture in 4/87 387 Susquehanne 1 02/04/89 89-006 Three m echanic St9pression Circle Root cause Yes R'.placed One sov failed, LER 87-023 Yes 19 atty ' chandser Seat analysis failed SOY and hourver two bountP dryvett planned but eight simiter simiter sovs i vacuum not complete enes had " problems
- breaker yet
(" problems" not specified) 388 Susquehanne 2 01/10/87 87-001 Two not R+ actor ASCO Not Not 56.mified No Reptoced Sov mone None Yes 02 Specified Building Specifi l ChiIted ed Meter I-e l
~~ -Pege No. 29- '06/07/90 SOLENOIP-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM ' MA8 ENACT MODEL ROOT REP CORRECTIVE - - ColelENTS REFERENCE TP/ FC.
- tC.
NAME DATE NUMBER FAILURES PART' WO. CAUSE FL ACTION DOCUuENTS OUT Contalrument ASCO Yes Replaced SOV Licensee shut LER 84-036 No,21 388 Susquehanna 2 02/27/89 89-003 One down ptont isotation instead of (recircutati cuntinuing on pum operation at-chitted reduced power water per tech specs 389 St. Lucie 2 08/16/89 89-006 One Not Hydrogen Valcor 52600-5 Not specified No Replaced SOV No. 21 specified samling 15 395 Summer 06/29/86 86-011 One ' Electric Feedwater Not Not Oxidation of' No Electrical None None Yes 07 comector (FWIV) Specified Specifi connector pins comector and ed SOV were replaced. 395 Sunener 12/02/88 88-012-01 None many - Ground Main Steam ASCO Not Design No Isoleted SOV Found that None No 14 incipients faults and Specifi deficiency contacts to ground faults Feedwater ed prevent could cause spurious spurious SOV sctuations actuations 395 Sumner 02/17/89 89-003-01 None, 3 Electrica Main steam incorrectly No Modified Comenon-mode LER 88-012 No 07 incipients t (MSIV) designed . wiring foiture isolation potentist for grounding att 3 MSIVs relay 397 WNP 2 03/22/84 84-027-02 Fifteen Ground Main steam Not Not SOV Yes Replaced Events at WMP LER No 14 faults (MSRV) Specified Specifi susceptibility defective. occurred during 84-027-01 ed to spurious SOVs. Tested startup actuation due potentletty. testing, to ground affected SOVs. Consuon-mode faults Voltage spike taiture suppression potential. diodes were Previous insts;ted on simitar events - att MSRV+ ADS.at La Satte + SOVs Susquehanna 397 WNP 2 07/23/85 85-050 Two faltures (1 Diaphragm Fire. Not Not Root cause of No 1-Replaced 'None done No 06 SOV) / seat protection Specified Speciff diaphragm diaphragWvatv leakage ed tenkage not e seat. 2-spectfied. backwards Backwards bomet bonnet due to " repaired *' inadequate maintenance s s-a qi ' ' qii ei y 'n "m' "iis N u 'i w.
s :- - ,g t Page No. 30 06/07/90 SOLENOID-OPE *ATED YALVE FAlltNtE DATA 1~ DOC PLANT EVENT LER NO. OF ' FAILED. SYSTEM MANUFACT MODEL ROOT REP CORRECTI'd COMENTS REFERENCE 'TP/ FC - f MO. NAME DATE NUMBER FAILURES FAR1 20. CAUSE. FL ACTION DOCUMENTS OUT i 400 Shearon Harris 1 02/08/88 88-006 Two Failed to Emergency Target 790-024 Source o' Yes The fatted e - sede~ mene- . no 12 close ' service Rock debris $0Vs were failure water pump accuaut ation repaired.- No affecting both seat water not specified statement made trains of ' supply about actione Emergency i taken for Service Water l removal of debris or prevention of j additienst' j debris 500 Shearon H:rris 1 05/13/88 88-012 Two F918ed to Emergency-Target 790-024 Cebris in Yes Repelred SCVs .14 shift service Rock water and Mocked water seal off source of water supply debris 400 Sheeron Harris 1 09/09/88 88-026 Eleven or more Internal Containment Target Eleven Manufacturing No Ung.setified C-.M Wone no 06 / reed isolation Rock models deficiency parts of 1E failure switch (many harth env. potentiet for wiring systems) 30Vs replaced
- 15. Sovs for with qualified hacsh
- enes, em ircrements.
Corrective SOUs for action for ex--containment non-harsh erre, also deficient. Sovs not specified. 409 La Crosse 12/03/84 84-022 One Seat Isolatim ASCO 8210 mot Specified Yes Reptaced SOY mone mene no 03 teskage condenser 409 La Crosse 04/20/85 85-006 One Colt Control Rod Royal not mot specified Yes Replaced 50V Wone LER 81-13 YES 01 Drive Industrie Specifi I s ed 409 La Crosse 05/17/85 85-012 One Seat contret Rod Royal Not Root cause of Yes Replaced SOV None mene Yes 12 Drive Industrie Specifi metal chip in s ed Sov seat not specified O O 4 ~.. g ---~, ..w ,,ne.., ,,w...-~ a w- ._m
-. ~.bf f (, .4 4: .z' -W 4 i r f i. ~' .? - Y &g _ ' :f. ' h}} ; \\1)L ( \\ _[ ~< y ~*
- ~ ;
Pag
- No, 31 06/07/90 SOLEN 0!D-OPEr4TED VALVE FAILURE DATA EOC FLAiff EVEitT LER NO. OF FAILED SYSTEM MANUFECT MODEL ROOT REP CORRECTIVE ColeqENTS REFERENCE TP/ FC No. NAME DATE NUMBER FAILURES PART-NO.
CAUSE FL ACTION 000DEENTS OUT 409
- s Crosse 07/08/86 86-020 one
' Coit. ' Cmtrot Reva royal Not tincerteln, Yes Reptaced SUV There have been LER 85-08 Yes 01 Oriv1r Industrie Specif f water 7 previous s ed intrusion or screas due to randam coil the scram faiture solenoid suspected shorting out. 409 in Crosse 07/19/86 86-024 One Electrica Reactor ASCO 8300 Personnet No Reptoced SOV ESFAS None-No 11 actuation, ( short cavity error-cascading event ventitation sptashed water on SOV 409 La Crosse 12/09/86 86-036-01 One Coit control Rod Royst Not (A1certain, Yes Reptsced There have been LER Yes 18~ Drive Industrie Speciff ageing or several Sovs. 8 previous 85-08,86-0 s ed moisture Replacement of scrams due to 20 intrusion SOVs will be these SOV suspected included in feitures. SOV CRDM that failed was preventive about 20 years maintenance old. program 410 Nine Mile Pt 2 06/22/88 88-025 Numerous Nydrautic Feedwater Keane 33896 Foreign object No Replaced SOV, SOV is None Yes Fe-internal parts Control in SOV, due to etso reptoced piece-part of manufacturing simiter SUVs tevet control Unit de.iciency or in other vetve failure to trains because instatt filter of sericus degradetion of screen their internets SOV Recomected SOV falture NO 08 414 Catawba 2 10/11/86 86-045 One Failed to AFW (steam .hift ednission to incorrectty SOV property defeated annuet turbine) instetled per start an incorrect capability of design drawing AFW turbine 416 Grand Gulf 1 02/10/85 85-007-02 Three Core-plug Main Steam ASCO 8323 FUSS No Replaced att 8 See section None
- Yes 05' MStV SOVs 5.2.4.4 nut (MSit) sticking
i ... y l - Page No. 32 l-06/07/90' SOLEN 0!D-OPERATED VALVE FAILtRE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORRECTIVE COMENTS REFERENCE 'TP/ FC _ NO.-NAME liATE NUMBER FAILtRES PART NO. CAUSE FL ACTION DOCUMENTS OUT 416 Grand Gulf 1 09/25/85 85-038-01 One Colt Drywett ASCO 8320 Excessive Failed SOV Licensee stated None. No ~ 11 No equipment - cerrosion replaced with. that the SUV drain within the a duplicate did not need to colt housing be believed ta be envireevnentally caused by seated water which entered during i ptont construction 416 i and Gulf 1 07/30/86 86-026-01 One Colt control Rod ASCO 1050602 Particulate No Replaced SUV, Porticulate None No 12 Drive SP1 acctsautation system filters acctsautation on the valve to be checked resulted in an seating 'and.aspted inevertent surface for control rod perticulates withdrawet 416 Grand Gutf 1 01/08/87 87-001 One Sov offgas ASCO 8320 Not specified No Not specified Modified system Wone No 00. failed in sanpling - specific mid posit actions taker. regerding SOV Ion - . not stated 416 Grand Gulf 1 03/15/89 88-010 One-Loose Control Rod ASCO Not Cause of toose No The loose Licensee to None Yes 07 l terminal Specifi connection not terminst eyetuote design box ed fourd connection was change to l connectio cleaned & improve i n to SUVs tightened. reliability of other SOV power leads terminet connections checkee, att l were okay._ 423 Mittstone 3 09/06/86 86-051 Not Specified " Failed Feedwater Not Not Intermittent No Att local None None Yes 01' electrica Specified Specifi open circuit, terminations tty" ed root cause on the SOV
- unknown, wiring to be suspect checked fer-vibration and tightness steam during the ispingement next shut hen.
from a packing teak O O e. - ~ y , - ~, ,,~m- .s*-, i-, W 6 e w ""'e
'A g. a- ~ Page No. 33 SOLENOID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER WO. OF FAILED SYSTEM MANUFACT MODEt. ROOT REP CORRECTIVE ComIENTS' REFERENCE'.TP/ FC ~ NO. NAME DATE NUN 8ER FAILURFS PART No. CAUSE FL ACTION Doct R NTS OUT 423 Mittstone 3 03/07/97 87-006 One Colt Feedwater Skinner v5N6620 C wse for open Yes Replaced SOV $0V was LER 86-051 Yes 00 0 circuit not operating (open .within its circuit) specified " design Life" 423 ptiltstone 3 05/06/87 87-024 One SUV would Emergency Cirete N299 P Not specified No failed air Faited SUV None No 20-not shift dieset ' Seat
- 617, start Sow and resulted in the diese!'s slow (out of witt,in generator redtrident SOV spec) EDG spec air start were replaced starting time with new ones 423 Mittstone 3 09/23/87 87-034 One Colt
-Feedwater Skinner v5N6620 Root cause of Yes Replaced 53V Sov controts LER Yes 01 Electric 0 colt falture hyurautic ott 87-06/36-0 flow to Fu!V $1 (open circuit) rot determined. ) Coit was within its "cpatsdied life" 424 Vogtte 1 01/22/87 87-002 Eight incipients Potential Main Steam Keane Not Design error No Instatted a Potential for None No 13 for MOPD specifi relief valve common-mode ed on each MOPD failures hydraulic due to heette system to of hydraulic - timit pressure fluid.. See to below MOPD Section S.T.3 timits of this report. 424 Vogtte 1 04/24/88 88-013 One Colt Feedwater Skinner V5N6559 Colt burnout No Reptaced SOV SOV is a None No. Of and similar piece-part of Electric 0 SOW on other ADV controtting train of FWIV FW1V control system k 440 Perry 06/30/86 36-030 one Seat Contairveent ASCO 8320 Dust from No Replaced SOV None Nw No 12 tenkage vesset and instrument air Drywett prevented Purge proper valve seating a
p h I i i I id I 'II .u. Page No. 34 06/07/90 SOLEN 0!D-CPERATED VALVE FAltuRE DATA DOC PLANT EVENT' LER NO. OF FAILED SYSTEN MANUFACT MODEL ROOT REP CORRECTIVE CopeqENTS REFERENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART No. .CAUSE FL ACTION DOCUMENTS OUT 440 Perry 02/27/87 87-009 Two Air Emergency Misphrey TOG 2E1-Falture due to Yes Replaced Beth Slauttaneous Ikme No 1T teakage Diesel Products 3-10-35 extended 50Vs. common-mode service with Retur 'I failure of both (through Generator elastomer Control Air high local failed SOVs to diesets. Delay Ic parts) temperatures EDG in repairing and continuous manufacturer tesking SOVs energiration. for analysis. contributed. Sovs in svc 10 wilt tygrade See section years and preventive -this report-never had PM neintenance and elastomers 440 Perry 10/29/87 87-073-01 Five SOVs on ?wo Etastomer Main steam ASCO NP8323 Nest and Yes Replaced or Common-mode Inso Rpt Yes 10-moisture from refurbished. failures. See 87-024 occasions-ic seats, (MSIV) steam teaks SOVs ' Section 5.1.1.1
- discs, of this report etc for additional-Information 440 Perry 03/10/88 88-010 One Core Auxiliary ASCO 8320 Inadequate No Replaced SOV. Failure of SOV None No 17 shaft Building (no)
Instituted a results in loss wear ventitation preventive preventive of RWCU room maintenance maintenance cooling for this SOV program (replace ashen upgrade to fait). Valve replace those I had been in SOVs every 2 service for years over 5 years 440 Perry 02/03/89 89-004 One -Auxiliary ASCO 8320 Yes Replaced SOV Licensee LER 88-010 No.19 investigsting twitding root cause ventitatfon 456 Braidwood 1 09/15/89 89-010 One Coit Containment valcor V526-53 Colt leads No Reptaced with Also reptaced 5 No 09 Isolation 95-1 tabeled different-other simitar (hydrogen backwards model SOW SOVs. Licensee investigating analyzer) source of mistabeling (manufacturer vs. plant) e e e
- p Page l'o.
7 06/07/90 . SOLEN 0!D-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORRECTIVE' C0puENTS REFERENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION DOCUMENTS OUT 458 River Bend 05/02/89 89-022 Affected Target 77kk-01 Beckesords Yes SUVs See section LER 89-024 No 14 many Rock 3 instaltation reinstatled in 5.1.4.1 f v systems. due to reverse additional See coment inadequate orientation details instattation instructions 458 River Bend 04/06/89 89-024 0 Affected Target 77KK-01 Backwards Yes Reversed Potentist LER 89-022 No 08 many Rock 3 installation - orientation of common-mode systems. design error. SUVs failures. 6 See coment Inadequate SOVs had the instattation same instructions, instattation deficiency. See section 5.1.a.1 of this report for info 461 Clinton 03/06/87 87-009 One SOV Fuet Not Not Not Specified No Replaced SOV None None No 03 failed in Building Specified Spectif mid wentitation ed position 461 Clinton 04/14/89 89-019 Electrica Main steam Seitz Design efror Instatt heat' Fall M to meet No 08 t .(MSIV) (EO). shrink tubing E0 instattation connectio inadequate per EO requirements ns electrical requirements connector sesting 461 Clinton 11/29/89 89-037 One 0-rings Vectam GPC LD240-4 Inadequate No Refurbished No scheduled No 03 relief Controts 20 preventive SOV, replaced preventive (SOV (CPE) maintenance 0-rings - maintenance tsispecifi program. ed) Falture discovered during stroke testing 483 Cattaway 01/02/85 85-001 One Not Feedwater Not mot Licensee Yes Replaced SOV Sov is a None Yes 00 Specified Specified Specifi considered piece-part of ed this to be a FWIV hydraulic random failure operator
4 Page No.
- 36 M/0W ~
SOLEN 0!D-OPERATED VALVE FAILURE DATA DOC ' PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL' ROOT REP CORRECTIVE COMENTS REFERENCE TP/ FC ' NO. NAME' DATE MtmOER F AILtRES ' PART-NO.' CAUSE FL ACTION DOCUMENTS OUT 483 Cattaway 02/20/86 86-002-01 None Electrice Reector head Not Not Construction Yes Not specified On 2 occasions None No 28 - t went and Specifled SpecifI and stertup . tIcensee foind connector chemical ed program it had not s votune deficiencies instatIed controt - environmentatty qualified connectors on S0Ws as required (3 Sovs) 528 Pato Verde 1 08/08/85 85-052 Two or more. potentist Post .Airmatic Not Design error No Affected S0Ws Sovs controt None No 14 incipients insulatio eccident
- Specifi, were shielded air-operated r.
. sampling ed - to reduce post sample flow breakdown accident control valves / shorts radiation to ground I I ~ .. t 6 4 ' k e 4 o 6 -my y m. ,,b=.w,a g 3* 7,p,e 5 9,w y w e--.- c +g v'f y, -r-- - - ,f.rg. +
- s -
t. APPENDIX B DISPOSITION OF ASCO DUAL-COIL 8323 SOVs USED FOR MSIV CONTROL u
l l APPENDIX B Disposition of ASCO Dual-Coil 8323 SOVs Used for MSIV Control Many plants have experienced problems with ASCO aual-coil 8323 SOVs which have been used for MSIV control. Several examples are provided in Chapter 5. ASCO issued two field notifications (Refs. 106,107) addressing NP8323 SOVs. The K notifications stated that the NP8323 SOVs have no defects, and that their mal-functions were primarily caused by foreign materials, aggrevated by adverse service conditions. Furthermore, because ASCO does not envision significant changes in the service conditions that the NP8323 S0Vs are subjected to, ASCO is phasing out the sale of those valves. As an alternative, ASCO recommends f the use of a pair of single-coil NP8320 SOVs. Two NP8320 S0Vs can be configured to perform the function of one NP8323. Because of the NP8320 50V's single-coil
- onstruction, ASCO anticipates that they will perform more satisfactorily than the NP8323 SOVs under adverse service conditions.
In anticipation of ASCO's discontinuance of the NP8323 SOVs,.the MSIV air pack manufacturer (R. A. Hiller Company) has initiated a program to select a suitable - replacement of the ASCO NP8323 SOVs.* The Hiller company has assembled five'MSIV l-air packs for baseline testing. The SOVs to be tested in the MSIV air packs are: ASCO: NP8320 V (2 valves configured as recommended by ASCO in Refs. 102,103). l AVC:04964 Target Rock: -fSMS-502(modified) Valcor: V70900-87V Zeiss: 629-60007 (assembly) l-GE and Hiller Company have noted that all of the American SOVs are IE qualified; L and that although the Zeiss assembly is not 1E qualified, it has been used successfully in Europe. . It should be noted that the choice of a ieplacement for the NP8323 SOVs can affect the qualification of the overall MSIV air packs (e.g. seismic / dynamic loading). Final ' selection of replacements for the NP8323 SOV should address this issue. In-the past, GE was actively involved in the qualification testing of MSIV air packs which were used at many plants. GE has indicated that as a result of ASCO's discontinuance of NP8323 SOVs they are trying to interest BWR owners to . support a consolidated effort with the Hiller Com;any to qualify MSIV air packs having suitable replacements for the ASCO NP8323.**
- Telephone discussion between J. Nanci, R. A. Hiller Company, and H. L.
.Ornstein, USNRC, December 8, 1989.
- Telephone discussion between C. Nieh, GE, and H. L.
Ornstein, USNRC, December 1989. PRELIMINARY CASE STUDY B-1 r
p-- 4 .e 5 t 7 s APPENDIX C GENERIC COMUNICATIONS ON SOVs lb l I l l l I li 4 --s
4
- ...n v
APPENDIX C Generic Communications on S0Vs Bulletin Number Date Title Bulletin 75-03 March 14, 1975 Incorrect Lower Disc Spring and Clearance Dimension in 8300 and 8302 ASCO Solenoid Valves Bulletin 78-14 December 19, 1978 Deterioration of Buna-N Components-in ASCO Solenoids Bulletin 79-01A June 6,'1979 Environmental Qualification of Class 1E Equipment (Deficiencies in the Environmental Qualification of ASCO Solenoid Valves) Bulletin 80-14 June 12,-1980 Degridation of BWR Scram Discharge Volume Capability Bulletin 80-17 July 3, 1980 Failure of 76 of 185 Control Rods to Fully Insert During a Scram-at a BWR Bulletin 80-17 July 18, 1980 Failure of 76 of 185 Control Rods Supplement 1 to Fully Insert During a Scram J at a BWR Bulletin 80-17 _ July 22, 1980 Failures Revealed by Testing Subse-Supplement 2 quent to Failure of Control Rods to Insert During a Scram at a BWR' Bulletin 80-23 November 14, 1980 Failures of Solenoid Valves Manu-factured by Valcor Engineering Corporation Bulletin 80-25 December 19, 1980 Operating Problems with Target Rock Safety Relief Valves at BWRs L l-l PRELIMINARY CASE STUDY C-1 I m..
o s Information Notice Numby Date-Title Information Notice 80-11 March 14,1980 Generic Problems with ASCO Valves in Nuclear Applica-tions Including Fire Protection Systems .Information Notice 80-39 October 31, 1980 Malfunction of Solenoid Valves Manufactured by Valcor Engineering Corporation Information Notice 80-40 November 7, 1980 Excessive Nitrogen Supply Pressure Actuates Safety-Relief Valve Operation to Cause Reactor Depressurization Information Notice 81-29 September 24 1981 EquipmentQuantification Testing Experience, Equip-ment Qualification Notice No. 1 Information Notice 81-38 Decembar 17, 1981 Potentially Significant Equipment Rieres Resulting from Contamination of Air-Operated Systems Information Notice 82-52 December 21, 1982 Equipment Environmental Qualification Testing Expe- -rience - Updating of Test' Summaries'Previously Published in IN 81-29 Information Notice 83-57 August 31, 1983 Potential Misassembly Problem with Automatic Switch Company-(ASCO) Solenoid Valve Model NP 8316 Information Notice 84-23 April 15, 1984 Results of NRC Sponsored Qualification Methodology Research Test on ASCO Solenoid Valves Information Notice 84-53 July 5, 1984 Information Concerning the Use of Loctite 242 and Other Anaerobic Adhesive-Sealants Information Notice 84-68 August 21, 1984 Potential Deficiency in Improp-erly Rated Field Wiring to Solenoid Valves PRELIMINARY CASE STUDY C2
k j y, Information Notice Number Date Title Information Notice 85-08 January 30, 1985 Industry Experience onlCertain Materials Used in Safety-Related Equipment. Information Notice 85-17 March 1,_1985 Possible Sticking of ASCO Solenoid Valves Information Notice 85-17 October 1, 1985 Possible Sticking of ASCO Supplement 1 Solenoid Valves Information Notice 85-47 June 18, 1985 Potential Effect of Line-Induced Vibration on Certain L-Target Rock Solenoid-Operated L Valves L Information Notice 85-95 December 23, 1985 Leak of Reactor Building I Caused by Scram Solenoid Valve Problem Information Notice 86-57 July 11, 1986 Operating Problems with Solenoid Operated Valves at Nuclear Power Plants Information Notice 86-72 August 19, 1986 Failure of 17-7_PH Stain-less Steel Springs in-Valcor Valves Due to Hydrogen Embrittlement q Information Notice 86-78' September 2, 1986 Scram Solenoid Pilot Valve (SSPV) Rebuild Kit Problems j Information Notice 87-48 October 9, 1987 Information Concerning the i Use of Anaerobic Adhesive / l Sealants -Information Notice 88-24 May 13, 1988 Failures of Air-Operated Valves Affecting Safety-1 Related Systems 'J Information Notice 88-43 June 23, 1988 Solenoid Valve Problems 1 Information Notice 88-51 July 21,1988 - Failure of Main Steam Isolation Valves i 1 .\\ Information Notice 88-86 March 31, 1989 Operating with Multiple Supplement 1 Gre'nds in Direct Current Di: vibution Systems l L L 1 l PRELIMINARY CASE STUDY C-3 l
i Information Notice Number; DateL Title Information Notice 89-30: March 15, 1989 High Temperature Environ-4 ments at Nuclear Power Plants Information Notice 89-66 September 11, 1989 Qualification Life of Solenoid Valves Information Notice 90-11 February 28, 1990 - Maintenance Deficiency i Associated with Solenoid- ~' Operated Valves Circular Number Date-Title Circular 81-14 -November 5, 1981' Main Steam Isolation Valve Failures to Close - .( t (~ i i s L j r L PRELIMINARY CASE STUDY C-4 u 1 -.}}