ML20055C861

Forwards Preliminary Case Study Rept:Solenoid Valve Problems at Us Lwrs, for Review & Comments.Response Requested within 30 Days from Receipt of Rept

ML20055C861
Person / Time
Issue date:	06/15/1990
From:	Novak T NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD)
To:	Kramer E DRESSER INDUSTRIES, INC.
References
NUDOCS 9006250277
Download: ML20055C861 (2)

Text

{{#Wiki_filter:; w e-( 3 JUti 15 1930 i 1 Mr. E.. Kramer > 1 Masonellan-Dresser i 7 c '85 Bodwell Street-Avon, Massachusetts 02322 l

Subject:

Preliminary Case Study Report on Solenoid Valve Problems att u.S. ' Light Water Reactors A preliminary AE0D case study re) ort, " Solenoid Valve Problems at U.S. Light Water Reactors," is enclosed. T1e study analyzes and evaluates operational { experience and safety implications associated with failures and degradations 1 .of solenoid-operated valves (SOVs) at U.S. LWRs. It focuses upon the vulnerability of safety-related equipment-to comon-mode failures or degradations of SOVs.. >The report presents information on more than 25 events in which comon-mode ' failures or degradations of over 600 SOVs were affected..or had the potential o .to affect,- multiple safety systems or. multiple trains of individual safety systems. Although plant safety analyses do not address such comon-mode failures or degradations of safety systems, operating experience presented in the. report indicates'that they have occurred and are continuing to occur.. i "A number of events in which' safety systems have been adversely affected by. degradations or failures of-S0Vs are. considered significant orecursors. The case study notes 1that 50V problems permeate almost all U.S. nuclear power i plants, and that they encompass many aspects of the.50Vs' desip, maintenance, and. operation. The case study also notes that indiviosal.SOV mai:4facturer's . practices regarding guidance with respect to testing and maintenance contribute 'towards the observed--problems. The report presents six recomendations which, if implemented, should reduce reactor accident risks by reducing the likelihood-f s for"comon-mode failure or degradation of SOVs affecting multiple safety systems or multiple. trains of individual safety. systems. In'accordance with our " peer review" process, prior to.the finalization and ' distribution of our case study reports, we are providing you and other vendors who provided input to the case study with-a copy of the preliminary report for review' and coment. We request'that you focus your review primarily on the T accuracy and completeness of the technical details (i.e., coments are being_ . solicited on the. technical accuracy of the report). The findings, conclusions, and recomendations are provided for your. information in order that you may understand the significance we place on these events and, therefore' obtain a @c more-complete picture of the total report. Changes to the findings,. conclusions, and,recomendations will be considered only if the underlying information concerning the details of plant design or systems operation is in error. We ask that'coments be provided in writing. A i bldcg7 J NRC FILE CENTER COPY '9006250277 900615 PDR ORO NEXO i

g.;, 7- ~-

• l Y^

{ y t j t u., ; i

c.

Mr.EfKramer,

Since we Wish to finalize.and issue the report shortly, we ask that any_.

connents be received.by us within 30 days from receipt of this preliminary.

report.1: If-you' require additional time beyond'that point, please let us knaw. 'If you or your staff have any questions'regarding this study, please feelt frer 1 to. contact me or Dr. -Ha1.Ornstein at (301) 492-4439. ) Sincerely.- I Original cisned by: Thomas M. Novak Thomas H. Novak, Director Division of Safety Programs, Office for Analysis and Evaluation .of 0lerational Data

Enclosure:

o: LAs. stated l LDistribution:: .PDR Central File- 'AE0D R/F DSP.R/F-ROAB R/F-H0rnstein-SIsrael', Plam. i JRosenthal l EJorda n' r a. Dross w.. f TNovak MWilliams: J .KBlack. b;: eMTaplor,iED0 t lRSavio,ACRS I y; + :p + ~_4.Y-1 lb _ y ' lg ; -;j l A &*jpths W) ,. 'ROAB:DSP: AE0D 'ROAB:gSP:AEOD C: 0 B:DSP:lE0D E0D

H0rnstein

mk >,

PlamS -JRosenthal k t Y 6////90; 6/lj/90 6/l(f/90 ' 6 90 .) r ~~ ) 3 (g: 3. 74 ^l x 4

l:l !): ? ).. g ir

7 1

i PRELIMINARY CASE STUDY REPORT .i SOLENOID VALVE PROBLEMS AT U.S. LIGHT WATER REACTORS i 1 I June 1990 j l-1.j r Prepared'by: L P -Dr. Harold Ornstein I I: L . Reactor Operations: Analysis Branch i' Office for Analysis and Evaluation of Operational Data U.S. Nuclear Regulatory Commission l uz., t

- ~ - m p 4 TABLE OF CONTENTS-Pasje EXECUTIVE SUMARY................................................... vii 1 INTRODUCTION.................................................... 1 t 2 DESCRIPTION OF EQUIPMENT........................................ 3 3_ USE OF SOLENOID-OPERATED VALVES (SOVs).......................... 8 4 SOLEN 0ID-OPERATED VALVE FAILURE MODES: APPARENT AND ROOT CAUSES. 10 5 OPERATING EXPERIENCE: SIGNIFICANT EVENTS INVOLVING CO MON-MODE FAILURES OR DEGRADATIONS OF SOVs................................ 11 5.1 Design Application Errors.................................. 11 5.1.1 Ambient Temperatures................................ 11 5.1.1.1 MSIVs at Perry - Excessive Heat From Steam Leaks................................ 11 5.1.1.2 MSIVs at Crystal River 3 - Thermal Aging - Incorrect Estimation of Ambient Temperatures............................... 13 5.1.1.3 Millstone 2 - Thermal Aging - Localized " Hot Spots" in Containment................. 13 5.1.2 Heatup from Energization............................ 14 5.1.2.1 Grand Gulf 1 MSIVs - Thermal Aging (Self-Heating From Energization)........... 14 5.1.2.2 North Anna 1 and 2, and Surry 1 and 2 - Thermal Aging (Self Heating Due To Energization).............................. 15 5.1.'3 Maximum Operating Pressure Differential (MOPD) - Multiple Plants..................................... 15 5.1.4 Directional SOVs.................................... 20 5.1.4.1 Incorrect Valve Orientation at River Bend.. 21 5.2 Maintenance................................................ 21 5.2.1 Maintenance Frequency............................... 22 5.2.1.1 Dresden 3 - BWR Scram System - Primary System Leak Outside Containment............ 22

5. 2.1. 2 Perry - Simultaneous Common-Mode Emergency Diesel Generator Failures........

24 5.2.2 Replacement Versu's Rebuilding....................... 24 ,x 5.2.2.1 MSIVs ~ at Perry - Inadequate SOV Rebuild..... 24 5.2.2.2 Brunswick 1 - Safety Relief Valves - S0V Rebuilding Error: Excess Loctite....... 25 iii

-.~, a .n. .(, .' 5 '1 e-TABLE OF CONTENTS (continued) j .PgLe = ~ l. l - 5.2.2.3' Peach Bottom 3 - Scrc9 System - SOV Rebuilding Error: Excess Loctite........... 27 5.2.3 Contamination........................................ 28 1 ', 1 ' 5.2.3.1 Brunswick 2 MSIVs - Excessive Heat'and l E Poor Air Quality (Hydrocarbons and Water)... 28 5.2.3.2 North Anna 1 and 2 - Multiple Systems Oil and Water Intrusion......................... 28 ^ 5.2.3.3 Susquehanna 1 and 2 - Scram System: Oil and Water Contamination..................... 31 '- 5.2.4 Lubrication.......................................... 32 5 2.4.1 Multiple Plants - Manufacturing Error: Residue-Producing Lubricant................. 32 5.2.4.2 Catabwa: Poor Quality Air and Lubrication with-Vaseline............................... 34 5.2.4.3 Common-Mode Failure of 16 MSIVs at a Two Unit Station (BWRs): Incorrect Lubrication. 35 5.2.4.4 6.and Gulf 1, LaSalle 1, and River Bend MS1Vs --Sticking SOVs - Foreign Unidentified Sticky Substance (FUSS) - Lubricant Suspected................................... 36 L 5.3 Surveillance Testing........................................ 40-5.3.1 Control Rod Timing Tests - Failed Scram Pilot SOVs - Perry......................................... 40 5.4 'Use of Non-Qualified SOVs................................... 40 l 5.4.1 Colt /Fairbanks.- Morse EDGs: Repetitive Air L Start Valve Failures................................. 40 1 L 6 ANALYSIS AND EVALUATION OF OPERATIONAL EXPERIENCE................ 42 l L - 6.1 Common-Mode Failures........................................ 42 - 6.2 SOV Failure Rates........................................... 43 ' 6.3 Maintenance' Problems........................................ 44-

6. 3.1: Maintenance Problems - 50V Manufacturer's Contributions........................................

44-6.3.2 Maintenance Problems - Contribution of the i Unrecognized S0Vs.................................... 46 1; 6.3.2.1 Unrecognized SOVs in Emergency Diesel i I Generators................................. 46 6.3.2.2 Unrecognized SOVs in Auxiliary and Mai n Feedwater Sys tems...................... 47 iv 1 ~

- - = .s .l TABLE-OF CONTENTS (Continued) Pan 6.3.2.3 Unrecognized SOVs in BWR High Pressure Coolant: Injection Systems................... 47 6.3.2.4 Unrecognized SOVs in Instrument Air Dryers.. 47 ,1 6.3.3 Maintenance Problems - Contributions of Utility Programs and Practices............................... 47 6.3.4 Rebuilding vs.-Replacement........................... 48 7-FINDINGS.......................................................... 50 7.1 Design Application Errors................................... 50 7.1.1 Ambient Temperatures................................. 50 7.1.2 Heatup from Energization............................. 50 7.1.3 Maximum Operating Pressure Differential.............. 50 7.1.4 Unrecognized SOVs Used as Diece-Parts................ 51

l 7.1.5 Directional SOVs....................................

51 7.2 Maintenance................................................. 51 ' 7.2.1 Maintenance Frequency...........................-..... 51 -7.2.2 Replacement Versus Rebui'idin 52 7.2.3 Contamination..............g 52 7.2.4 Lubrication.......................................... - 53 7.3 Surveillance Testing........................................ 53 7.4 _ Veri fication of the Use of Quali fied SOVs................... 53 7.5 Redundancy and Diversity.................................... 53 7.6-Feedback of Operating Experience............................ 54 8 CONCLUSIONS...................................................... 55 8.1 Safety Significance......................................... 55 8.2 Need For Action............................................. 56 9 RECOMMENDATIONS.................................................. 57 9.1 Design Verification-........................................ 57 9.1.1_ Ambi ent Temperature s................................. - 57 o l 9.1.2 Heatup From Energization............................. 57 i 9.1.3 Maximum Operating Pressure Differential.............. 57 9.1.4 Unrecognized SOVs Used as Piece-Parts................ 57 9.1. 5 Directional SOVs..................................... 57 L L 9.2 Maintenance................................................. 57 h 9.2.1 Frequency............................................ 57 h 9.2.2 Replacement Versus Rebuilding........................ 58 L u l V I f I l

TABLE OF CONTENTS (Continued) Pagg 9.2.3 Contamination........................................ 58 9.2.4 Lubrication.......................................... 58 9.3 Surveillance Testing........................................ 58 9.4 Verification of the Use of Qualified SOVs................... 58

9. 5 Redundancy and Diversity....................................

59 9.6 Feedback of Operating Experience............................ 59 10 REFERENCES....................................................... 60 APPENDIX A SOV Failures Reported in LERS: 1984-1989................ A-1 P. Disposition of ASCO Dual-Coil 8323 SOVs Used for......... 'B-1 MSIV Control C Generic Communications on SOVs........................... C-1 vi

4 EXECUTIVE

SUMMARY

The study analyzes U.S. light water reactor (LWR) experience with solenoid-operated valves (SOVs). - It focuses upon the vulnerability of safety-related equipment to common-mode failures or degradations of SOVs. The report presents information on over twenty events in which common-mode failures or degradations of over 600 SOVs affected, or had the potential to affect, multiple safety systems or multiple trains of individual safety systems. Although plant safety analyses do not address such common-mode failures or degradations of safety systems, operating experience presented in the report indicates that they have occurred, and'are continuing to occur. The events in which common-mode failures of SOVs have affected multiple trains of safety systems or multiple safety systems are important precursors. They indicate that actions are necersary to assure.that important plant systems function as designed in accordance with plant safety analyses, and that plants-are not subject to unanalyzed fai'.ute modes with the potential for serious consequences. The report analyzes the operating experience and it outlines the root caus.es of common-mode failures and degradations that have been observed, and provides recommendations to significantly reduce the occurrence of common-mode 50V failures. Analysis of operating' data indicates that the underlying or root causes of many SOV failures are the users' lack of knowledge or understanding of 50Vs' requirements or capabilities, such as: 50Vs' intolerance to process fluid contamination; the necessity for preventive maintenance or changeout; and the propensity for rapid aging and deterioration when subjected to elevated tempera-tures. Compounding the problem is the fact that some SOV manufacturers do not provide the users with adequate guidance regarding proper SOV maintenance and operation. Further complicating the situation is the fact that many S0Vs are " unrecognized" i.e., they are provided as piece parts of larger components so that the end users have a restricted knowledge of the 50Vs' operation and main-tenance requirements, or their useful design life. i The report addresses widespread deficiencies which were found in the areas of: design / application, maintenance, surveillance testing, and feedback of l failure data. l' It is recommended that for safety-related applications, licensees: (1) verify the compatibility of SOV design and plant operating conditions; (2) ver-ify the adequacy of plant maintenance programs; (3) ensure that SOVs are not subjected to' fluid contamination (e.g., instrument air); (4) review SOV surveil- ' lance testing practices; and (5) verify that all SOVs which are used in safety-related applications have been manufactured, procured, installed and maintained commensurate with their safety fun'etion to assure operation consistent with 4 plant safety analyses. Specific technical information supporting these broad recommendations is contained throughout the report. Detailed recommendations are provided in -Chapter 9. PRELIMINARY CASE STUDY vii i L

, 7 .I t i i t .-7n addition, it is recommended that an industry group such as INPO take action to improve the mechanism for feeding back SOV failure data to the manu-l' facturers for early detection and resolution of potential generic problems. e s l p l. l L (- I 1, i 1 l l' I + m L PRELIMINARY CASE STUDY viii L =

1 INTRODUCTION All U.S. light water reactors (LWRs) rely upon solenoid-operated valves (SOVs) to perform safety-related and non-safety-related functions. SOVs are-used to operate with hydraulic and pneumatic fluids under a wide variety of con-ditions. They are used to control process fluid either directly, or indirectly - as pilot controllers. It has been estimated that the population of SOVs in safety systems at U.S. LWRs is between 1,000 and 3,000 per plant (Ref. 1). Boiling water reactors (BWRs) usually have more SOVs than pressurized water reactors (PWRs), because of the extensive use of SOVs in BWR scram systems. Many SOVs used in nuclear power plants are dedicated / qualified valves, -which have undergone vigorous qualification testing to standards such as the Institute of Electrical and Electronics Engineers (IEEE) Standards 323, 344 and j 382, and are manufactured in accordance with the Nuclear Regulatory Commission (NRC) requirements of Title 10 of the Code of Federal Regulations, Part 50 (10.CFR Part 50) Appendix B, and 10 CFR Part 21. However, we have also found manycasesinwhIchplantsusecommercial,nonqualifiedSOVstoperform-safety' related functions, This study was initiated after several licensees experienced repetitive failures of SOVs at their plants and after the simultaneous failure of four SOVs at the Brunswick 2 plant on January 2,1988 (Ref. 2). The Brunswick event resulted in a loss of containment integrity when two sets of redundant SOVs failed to close upon demand. The NRC Office for Analysis and Evaluation of Operational Data.(AEOD) has reviewed and participated in follow up work that the licensees, the NRC regional inspectors, and the valve manufacturers have performed following the SOV failures at Brunswick and several other plants. A number of other significant operational events have occurred involving malfunctioning SOVs. Previous studies of SOV failures (Refs. 1, 3, 4,~5) dis-cussed SOV failure rates and provided a characterization of the degradations or failures. This study addresses root causes and the generic nature of many of the observed failures. Some of the significant events discussed in this report are: Emergency diesel generator (EDG) failures at Perry and Catawba MSIV failures at Perry, Brunswick, Grand Gulf, LaSalle and River Bend AFW System degradation at Calvert Cliffs and North Anna Losses of containment in'tegrity at Kewaunee, North Anna, and Brunswick BWR scram system component failures at Susquehanna, Brunswick and Dresden Safety Injection System degradation at Calvert Cliffs PRELIMINARY CASE STUDY l

_y i Chapters five and six of this. study provide comprehensive reviews and. + evaluations of operational experience and potential safety implications =asso - ciated with SOV problems at U.S. LWRs. This study provides.several recommenda-tions to address the major deficiencies which were noted during.the review of 1 the operating experience. [ I i i l s s 2 m P r .h' f VE 4 f I l . PRELIMINARY CASE STUDY 2

t I:

j.~' s 5 .2

3 2 DESCRIPTION OF EQUIPMENT 'There are many manufacturers and varieties of SOVs used at nuclear power plants. SOV. operation is based upon changing the electrical status of the valve's electro-magnetic coil, which in turn causes a shift of the position of-an internal core.. The core acts to open or block the passageways inside the i valve, changing the flow path within the valve. A simplified version of a two-way SOV is illustrated in Figure 1. Figures 2 through 4 illustrate other more complex SOVs which are made by. three different manufacturers. SOVs are available for use over a wide range of temperature and pressure i conditions for liquid.and gas service. They.are available with the following formats: normally open or normally closed fail open,. fail closed, fail as is normally energized or normally de-energized ac or de power, or both ac and de power two-way valves, three-way valves, four-way valves direct lift, pilot assist, balanced disc, gate, modulating control. There-_is a wide range of sophistication and quality of SOVs. For example, mass produced SOVs are available for home consumption for a few dollars each, ~ whereas a limited production of high quality SOVs are available at a much higher price. SOVs that are qualified for Class 1E nuclear service (meeting IEEE Stan-dards 323, 344, 382, American National Standards Institute (ANSI) N45.2 and .10 CFR Part 50, Appendix B, and 10 CFR Part 21 requirements and having American Society of Mechanical Engineers (ASME) Section III "N or "NPT" stamps) may cost several thousands of dollars. PRELIMINARY CASE STUDY 3

m h$- { E = r C i i5 I j =-< t i 9 X; y ring Spring ' i Sp m N ( 'h (' 'k b W W b I Core nun ruus) suuA vann Q ) Core inlet _ Outlet inlet c Outlet \\ / 'h il ,1,1,11. v1,1,1, r11,1,1, OPEN-CLOSED I i Figure 1 Simplified Diagram of a Two-Way Solenoid-Operated Valve .~-r,. -n .,.---m ~ m _. ,,.,.n, .;...~,.. ,+ e -m ,,e ,m.

.44 _...., --, a =.. _. _ 2 y.. . - ~,.

u.,.

1 I Solenoid "A" s...,, ' 5v Coil R / I l V j a ,e Solenoid Base Subassembly N Port 3 Y/l D 1 \\ Core Assembly cA'i( 2' Valve Body N b $jl$Q b IL Disc Holder e \\ / l '~ -) \\. 'y y$,,. ' Coil J Solenoid Base / Subassembly ,g i Plug Nut Assembly h ,f' 5 *.,,y Solenoid "B" .I Core Assembly Stem ,5 l Figure 2 Isometric Drawing of ASCO Dual Coll 8323 Solenold Operated Valve PRELIMINARY CASE STUDY 5 4

'l (( l . Magnetic Assembly e j l:: / ~ / / E A f t / Terminal I / Board Plug i W/ l x 6 !.=. / Y f h ,1 p i m 4 , /, a Solenoid lo Subassembly < . 3 6 p y g \\ ( p h/" i N R ! ^ ii i bl l s i ri N\\M U s ( N l l Port * - Valve -i Subassembly <

• Port X

y ;kQ 1 a ' Port

• M

.l _ad Piston w w Figure 3 j Schematic Drawing of a Valcor Solenoid Operated Valve. 1. PRELIMINARY CASE STUDY 6

Magnet Assembly Solenoid Coil Terminal S lenoid Board Assembly Bonnet Assembly Fixed Core Return Spring Moveable Core Main Disc Pilot Disc 4 r ~ <h 4D Port ort t J . Figure 4 Schematic Drawing of a Target Rock Pilot Assisted Solenoid Operated Valve 4 l-l PREt.IMINARY CASE STUDY 7 ~ s -ve

'l 3 USE OF SOLENOID-0PERATED VALVES In many applications SOVs are used as alternates to motor-operated valves (MOVs). SOVs are frequently used as pilot operators to control air-operated valves (A0Vs). - The advantages of using SOVs instead of MOVs are that they e nerally have fewer moving parts, are compact and may be easier to mount. They also have low power requirements and have fast response times. j Some SOV manufacturers' literature claim that SOVs have long qualified - lives, have low initial and installed costs, and require low maintenance. ~ The use of A0Vs, MOVs and SOVs is a matter of preference of application that is determined by the utility, nuclear steam system supplier, and architect engineer; their specific utilization is not a licensing requirement. Table 1-lists many of the systems th'at use SOVs at U.S. LWRs. 1 -PRELIMINARY CASE STUDY 8 l F

Table 1 Systems Which Use SOVs at U.S. LWRs 1. BWR Scram I 2. PWR Rod Control' . 3. Reactor Coolant (RCP seal) 4. SafetyInjection 1 5. Auxiliary Feedwater 6. Primary Containment Isolation 7. High Pressure Coolant Injection / Reactor Core Isolation Cooling 8. HighPressureInjection 9. Automatic Depressurization 10. Emergency Diesel Generator L 11. Instrument Air 12. Chemical Volume Control / Charging and Letdown /Boration 13. Pressurizer Control 14. Steam Generator Relief (PORVs, ADVs) 15. Low-Temperature Overpressurization Protection 16. Decay Heat Removal / Residual Heat Removal 17. Component Cooling Water 18. Service Water 19. Reactor Head Vent 20. Steam Dump 21. Reactor Cavity / Spent Fuel / Fuel Handling

22. Torus and Drywell/ Vent and Vacuum-23.

Emergency DC Power

24. Main Steam (Main Steam Isolation Valves / Auxiliary Boiler) 25.

Reactor Building / Auxiliary Building (Ventilation and Isolation)

26. -Main Feedwater 27.-

Condensate

28. Moisture Separation / Reheat 29.

Containment Atmosphere / Containment Spray 30. Standby Gas Treatment 31. Floor / Sump Drain 32. Sampling (normal and post-accident) 33. Fire Suppression

34.. TurM,ie/ Generator 35.

Reactor Building Turga 36. Containment Air Lock 37. Leak Detection 38. Radwaste PRELIMINARY CASE STUDY 9

4 SOLEN 0ID-OPERATED VALVE FAILURE MODES: APPARENT AND ROOT CAUSES Previous studies (Refs.1, 3, 4, 5) have noted that details of the failure mechanisms, the apparent causes, or the root causes of 50V failures were not provided in approximately half of the licensee event reports (LERs) and nuclear plant reliability data system (NPRDS) failure records for years IN8 through 1984. 7 Appendix A of this report provides a listing of approximately 20u LERs describing SOV failures which occurred at U.S. LWRs between 1984 and 1989. The apparent and root causes of most (approximately 75 percent) of the SOV failures reported in LERs between 1984 and 1989 are given below: a. Coil failure or burnout that was attributed to design or manufa;:turing deficiencies (early failure /end of life) or an error in application (type of current, voltage level, environmental conditions). [11%) b. Valve body failure or leakage that was attributed to design or manufactur- ? ing deficiencies, such as excessive tolerances on internal parts excessive rI. wear / degradation of gaskets, 0 rings, seals, or springs; or foreI'gn particulates preventing proper sealing. [13%) c. Passageway blockage / internal binding that was attributed to contaminants such as dirt, corrosien products, desiccant, water or moisture, incorrect lubricants, excessive lubrication, or hydrocarbons. [9%) d. Electrical malfunctions that were attributed to faulty internal wiring, reed switch shorts or external wiring with inadequate connections, splices, or grounds. [12%) e. Design errors or misapplications that were attributed to incorrect valve F configuration (normally open vs. normally closed: aormally energized vs. I normally de-energized); incorrect designation of

• tail-safe" condition; incorrect electrical source (ac vs. de, voltage level); incorrect desig-nation of environmental conditions (temperature, moisture, radiation);

incarrect designation of maximum operating pressure differential; incorrect material selection (incompatibility between elastomeric parts and process fluid contaminants); incorrect valve orientation (horizontal vs. vertical). [13%) f. Installation errors that were attributed to incorrect physical orientation 1 (backwards, upside-down), electrical source (ac vs. de, voltage level), or inadequate electrical connections (e.g., loose connections, incorrect grounds). [7%) g. Maintenance errors that were attributed to incorrect determination of useful life or time between overhauls; inadequate preventive mainte-nance or incorrect preventive maintenance. [7%) Sticking that was the result of unidentified foreign substances coating [5%) h. valve internals, excessive use of lubricant, or foreign particulates. ~ PRELIMINARY CASE STUDY 10

5 OPERATING EXPERIENCE: SIGNIFICANT EVENTS INVOLVING COMMON-MODE FAILURES OR DEGRADATION OF SOVS The events described below were chosen as a representative set. They should not be construed as being a complete set of common-mode failures and degradations of SOVs. Additional u nts are tabulated in Appendix A. Many other SOV failures fall below NRC reporting requirements, and as a result are not captured in the LER data base. Many individual SOV failures not reported in the LER data base are reporte'd in the Nuclear Plant Reliability Data System (NPRDS) data base. Reference 1 noted that for 1978-1984 data, all SOV failures reported in LERs were also reported in NPRDS. 5.1 Desion Application Errors Representative operating experience illustrating design application errors -- associated with high ambient temperature, internal heatup from energization, incorrect maximum operating pressure differential and incorr;ect valve orienta-tion are described below. Based on this experience, findings and recommenda-tions relevant to design application errors are provided in Sections 7.1 and 9.1 i respectively. 5.1.1 Ambient Temperatures 5.1.1.1 MSIVs at Perry - Excessive Heat from Steam Leaks On October 29, 1987, while performing stroke time testing, three of the plant's eight MSIVs failed to close within the plant Technical Specifications' allowable time of five seconds. Two of the MSIVs were in the same main steam-line. During subsequent testing, each of the three valves closed within the Technical Specifications value. Since the vth s all stroked satisfactorily subsequent to their initial - failures, the licensee believed that the failures were due to the presence of impurities in the air pack SOVs controlling the MSIVs, and that the impurities were apparently discharged during subsequent MSIV operation. As a result, the three MSIVs that had failed were declared operable. These MSIV air packs consist of a single-coil 4-way SOV (ASCO NP8320), a dual-coi) 3-way SOV (ASCO NP8323) and thrse poppet type.nir pilot-operated valves (2, 3, and 4-way CA Norgren Co.). A photograph of one of the Perry plant's MSIV air packs appears in Figure 5. In response to NRC concerns, the licensee performed additional MStV stroke testing. As a result on November 3, 1987, the inboard and outboard MSIVs in the "O line again failed to close,within the required 5 seconds (outboard MSIV closed in 2 minutes and 49 seconds and the inboard MSW closed in 18 seconds). Additional MSIV stroke tests were performed, and both MSIV's again closed within the Technical Specification allowable times. I PRELIMINARY CASE STUDY 11

I L ~ i ASCO Dual-Coil NP8323 SOV _ ;sj y. .28 NORGREN 2-ww e q., AOV i NORGREN 4-way NORGREN AOV i-3-way AOV ~ ^ 9;,. ASCO Single-Coil 3*. NP8320 SOV Figure 5 MSIV Air pack from Perry Nuclear Power Plant, November 1987

4 i Because of continued NRC concerns about MSIV reliability, the licensee shut ) down the plant and established a plan to find the root cause of the MSIV fail-i ures (Refs. 6, 7, 8). Intense investigative efforts were conducted by the util-I ity to determine the root cause of the MSIV failures. The failures of the MSIVs t on both October 29 and November 3,1987, were attributed to the failure of the ASCO dual-coil Model NP8323 SOVs to shift position upon de-energization. The SOVs failed to shift position because of degradation of their ethylene propylene diner (EPDM) seats and discs. The degradation was caused by high temperatures t that had existed in the vicinity of the SOVs as a result of several steam leaks. Originally, hydrocarbon intrusion was suspected as having contributed to the degradation of the EPDM seats and discs. It was not until microscopic and spec-tral analyses were performed at an independent laboratory a month after the event that the possibility of impurities from hydrocarbon intrusion was elimi-nated as a root cause of these failures (Ref. 9). However, as part of its cor-rective action to prevent future failures, the licensee took steps to improve i the maintenance of the instrument air system. In addition, the licensee under-took an aggressive program to review the effects of all known steam leaks that could affect other safety-related equipment. 5.1.1.2 MSIVs at Crystal River 3 - Thermal Aging - Incorrect Estimation of Ambient Temperatures In April 1989, NRC inspectors reviewed the environmental qualification of electrical equipment at the Crystal River 3 plant. Their review found that errors had been made in the licensee's determination of the service life of 16 normally de-energized SOVs that are used to pilot the plant's M51Vs (Ref. 10). The licensee's determination of SOV service life was made assuming an ambient temperature equal to the weighted average of the temperature of the areas where the SOVs were located. The licensee's calculations did not consider the localized elevated temperatures that the SOVs were subjected to as a result of hot process piping. Recalculation nf the service life of the SOVs using representative ambient temperatures reduced the estimated service life of the SOVs from 40 years to 8 years. As a result, the licensee is replacing those SOVs sooner than previously anticipated. 5.1.1.3 Millstone 2 - Thermal Aging - Localized " Hot Spots" in Containment In November 1988, an NRC inspect % n report (Ref. 11) noted that Millstone 2's environmental qualification program recognized a significant shortening of the qualified lifetime of eight Valcor 50/s that are used for pressurizer and reactor vessel head vents. Originally the SOVs were calculated to have quali-fied lives of 40 years based upon an ambient temperature of 120'F. Although the plant's Technical Specifications require that the " primary containment average air temperature" does not exceed 120'F, the licensee found localized " hot spots" of 157'F in the vicinity of the eight SOVs. The licensee determined that the increase in ambient temperatures from 120'F to 157'F shortened the lifetime of the SOVs from 40 years to 12 years. The problem of equipment degradation due to localized hot spots is not unique to Millstone 2. Reference 12 lists several other plants that have experienced localized thermal " hot spots" inside contain-ment. In addition, NRC Information Notice 89-30 (Ref. 13) noted that similar heating events have been reported since 1982. The information notice alerted licensees to the potential for exceeding equipment's qualification specifications when the bulk temperatures are measured by a limited number of sensors that indicate acceptably low average temperatures. PRELIMINARY CASE STUDY 13 )

i 5.1. 2 Heatup from Energization 5.1. 2.1 Grand Gulf 1 MSIVs - Thermal Aging (Self-Heating From Energitation) On August 14 failed to close up,on demand (Refs.1989, following a reactor trip, one MSIV (int,oard " line) i 14,15,16). The MSIV did close about 30 minutes later. The failure of the MSIV to close was attributed to the failure of an ASCO dual-coil NP8323 SOV, a piece-part of the MSIV air pack. The licen-see's investigation found a piece of EPDM from the SOV's disc on the 50V's out-let 50V' port screen. The licensee concluded that the piece had been lodged in the s internals, thereby keepin keeping the MSIV from closing. g the 50V from venting control air and henceIt is dislodged from the internals, the MSIV closed. Subsequent inspections by the licensee of all eight ASCO dual-coil NP8323 i SOVs piloting the MSIVs disclosed that all eight had degraded seats. Initial visual inspection did not reveal the degradations which became apparent under microscopic examination. TheEPDMseatsofalleIghtSOVshadcracks.

However,

) on six of them, the raised portion of the seat, formed by the annular impression made by the seat of the e.xhaust port, was missing. It appeared that six of the eight SOVs had experienced similar sloughing of material from the seat. The August 14, 1989 failure is believed to have been caused by a piece of the EPDM disc material which hed been extruded into the 50V's exhaust port vent hole. The extruded material had separated from the disc as a result of the adhesive and frictional forces when the normally energized SOV was de-energized. The frictional and adhesive forces eventually led to the tearing off of the extruded parts of the EPDM discs. The extrusion of EPDM discs is discussed in GE Service Information Letter (SIL) 481 (Ref. 17). SIL 481 notes that the intrusion of the disc into its ex-haust port may account for previous events involving the sticking of similar EPDM dual-coil SOVs, but tearing of the discs had not been observed previously. It is believed that the tearing and overall degradation of the dual-coil SOVs' EPDM discs at Grand Gulf was symptomatic of thermal degradation resulting from the excessive time the EPDM materialc were exposed to high service temperatures. The EPDM discs had been operating at elevated temperatures due to the energiza-tion of the dual coils. The local temperatures inside the SOVs near the EPDM discs were approximately 325'F inside the inboard SOVs in a 135'F drywell and 305'F inside the outboard SOV in a 125'F steam tunnel. The SOVs had been in i service for approximately 4.5 years. However, the qualified lives of the degraded EPDM discs are estimated to have been 2.2 years for the inboards and 3.2 years for the outboards based upon environmental temperatures of 135'F for the inboard SOVs and 125'F for the outboard SOVs.* The NRC issued an information notice on this event, noting the life shortening effects of self-heating from coil energization (Ref. 18). Subsequently, ASCO issued a service bulletin providin nuclear qualified SOVs (NP series)g licensees with heat up data for all their (Ref. 19). "Other EPDM discs in the same 50V which were exposed to slightly higher temperatures were estimated to have had qualified lives of 1.58 and 2.28 years, respectively. PRELIMINARY CASE STUDY 14

i 5.1.2.2 North Anna 1 and 2, and Surry 1 and 2 - Thermal Aging (Self Heating Due to Energization) ) In December 1986, Virginia Electric & Power Co. (Vepco) requested ASCO to i provide information regarding the effects of "self heating" in continuously energized S0Vs. ASCO's response indicated that a significant increase in temp-erature would occur and that the temperature increase could result in a signi-ficant reduction in the qualified life of the SOVs. The licensee recognized that previous estimates of 50V service life did not account for the effects of self heating (Refs. 20, 21). The licensee evaluated the affected SOVs and determined that, contrary to previous analyses,125 SOVs would require replace-ment at North Anna 1 and 2 between the 1987 and 1989 refueling outages (Ref. 22). The SOVs affacted piloted air-operated valves, many of which served containment isolation functions. The systems affected were: Safety Injection Reactor Coolant, Main Steam, Component Cooling Water, Containment Vacuum, Radiation Monitoring, Sampling Systems, Instrument Air, Post Accident Hydrogen Removal, Heating and Ventilation, Steam Generator Blowdown, Gaseous Vent and Aerated Drains. The licensee recognized that Surry 1 and 2 were similarly affected, and. Vepco engineering informed personnel at the Surry station of this problem. Similarly, Surry 1 and 2 required early replacement of 58 ASCO SOVs because of self heating.* It is interesting to note that the licensee for North Anna station stated in a Deviation Report (Ref. 21) that these findings were non-reportable because: "NRC and utilities are aware of this issue to some extent." In Reference 20, the licensee noted that it had learned of this problem initiall with " industry representatives" at Equipment Qualification (EQ)y from discussions seminars in late 1986. 5.1.3 Maximum Operating Pressure Differential (MOPD) - Multiple Plants Many plants have experienced conditions in which SOVs failed or could have failed to perform safety-related functions because of excessive operating pres-sure differentials. Figure 6 is a schematic diagram of an SOV, illustrating how an operating pressure differential in excess of its maximum operating pres-sure differential (MOPD) can cause an 50V to malfunction. When the SOV is in the de-energized position, pressurized fluid enters the valve at port 2 and is blocked by the core assembly. If the pressure differential between ports 2 and 3 exceeds the M0PD, the overpressure could lift the core assembly, resulting in leakage of fluid from port 2 to port 1 and port 3. In the energized position the core assembly is raised to block the ex..aust port (port 3). However, the excess pressure would act to retard or prevent the core subassembly from dropping down (shifting) upon de-energization. As a re-sult, de-energizing the valve would not assure the valve achieved its correct de-energized position (block off port 2).

• Telecopy communication between W. Murray, Vepco, and H. L. Ornstein, USNRC, December 19, 1989.

PRELIMINARY CASE STUDY 15

t Port 3 Exhaust P i Solenoid Base a Sub-Assembly 5 1 Coil i A AN Core A p ssembly / /

o;

,d 1 hd' h kl$uf 4-$k ;;R$(;T.' .,y:$;l;s; gj ;;}-f[ijg,y ' A, g / 7 q - y l To Control g>. Port 1 Port 2 e Flow Valve ? I i -h ', y;; [;ii'.p * +, & n, 'f t. ' O.

i ;

's I Figure 6 Schematic of a Solenold Operated Valve illustrating Effect of Operating Pressure Differentials 1 PRELIMINARY CASE STUDY 16

l For many SOVs, the MOPD rating does not appear on the nameplate or in the installation and maintenance instructions. Vendor catalogs need to be consulted to determine those 50Vs' MOPD ratings. In May 1988, the NRC issued Information Notice 88-24 " Failures of Air-Operated Valves Affecting Safety-Related Systems" (Ref. 23). It informed l licensees of two SOV failures which were experienced at Kewaunee (Ref. 24) and of the potential for additional failures at Kewaunee and Calvert Cliffs 1 and 2 (Refs. 25-27). Subsequently, several licensees informed the NRC of similar dis-coveries in their plants, where the potential for overpressurizing SOVs exists, which could prevent the SOVs from performing their safety-related functions. l At some plants, the task of verifying the potential for overpressurizing SOVs has been complicated by the fact that documentation is not readily available. For example, Millstone 1 and 2 (Ref. 28), Crystal River 3 (Ref. 29), have re-ported that documentation to identify SOVs in containment is not readily available, and that containment walkdowns are necessary for their identification. It is not clear that Information Notice 88-24 has been effective in eliminating the potential for 50V overpressurization. Our concern is predicated upon Ref. 29 and a followup discussion in which the Crystal River 3 licensee stated that its review of the potential for 50V overpressurization assumed the proper operation of in-line pressure regulators, it did not address the conse-quences of pressure regulator failures. One of the events described in Infor-mation Notice 88-24 involved the discovery at Calvert Cliffs that several safety systems were vulnerable to single failures of pressure regulators in the air supply system. The earliest SOV overpressurization failures that we found occurred in 1980 at the Pilgrim plant. On October 7, 1980 and again on October 31, 1980, a safety relief valve (SRV) spuriously opened while the reactor was at power. On each occasion, the SRV did not reclose until the reactor was shutdown and the reac-tor coolant system was depressurized. The spurious valve openings were caused by excessive pneumatic (nitrogen) supply pressure to the SOV controlling the

SRV, The high nitrogen pressure exceeded the 50V's M0PD, causing the SOV to shift position which caused the SRV to spuriously open.

The NRC issued an information notice and a bulletin on these events (Refs, 30,31). Information Notice 80-40 (Ref 30) indicated that two-stage SRVs with Target Rock SOVs are susceptible to such MOPD malfunctions, whereas older three-stage SRVs having ASCO or AVC SOVs are not. Bulletin 80-25 (Ref. 31) required licensees to review and upgrade their SRV pneumatic supply systems and/or SOVs to assure that the SOVs operate within their maximum operating pressure. The bulletin required licensees to install protective devices (such as relief valves) to protect the SOVs against excessive supply pressures. The issue of overpressurization failures of SOVs in other systems was not addressed in the information notice or the bulletin. The discovery of the potential for overpressurizing multiple SOVs at the Vogtle plant was reported in Reference 32. Reference 32 described a situation

• Telephone discussion between L. Kluit Florida Power Corporation, and H. L.

Ornstein, USNRC, October 10, 1989. PRELIMINARY CASE STUDY 17 . 1 ~ . ~

in which SOVs controlling the operation of all eight MSIVs could fail because of overpressurization due to overheating. The MSIV manufacturer (Rockwell) had noted that a small steam-line break in the vicinity of the plant's MSIVs could cause an increase in the hydraulic fluid pressure in excess of the 50Vs' maximum operating pressure differential. These SOVs were manufactured by the reane Company. As a result of 50V overpressurization, both MSIVs on one or more steam-lines could allow uncontrolled blowdown of more than one steam generator following a main steam or feedwater line break. Essentially, if the HSIVs' hydraulic actuator fluid heated up 12'F, a condition not bounded by the plant's safety analyses could result. The licensee's corrective action was to replace the SOVs with others having higher MOPD ratings. In November 1987, the Kewaunee plant actually experienced two 50V failures caused by overpressurization (Ref. 24). During review of.the two SOV failures, the licensee found that 58 additional SOVs could fail to perform their safety-related functions as a result of overpressurization. 4 In April 1988, the licensee of Calvert Cliffs 1 and 2 found that 40 SOVs could fail to perform their safety-related function as a result of overpressuri-zation (Ref. 25) In the case of TMI-1, (Ref. 32) the SOVs were connected to line pressures in excess of the maximum dictated by the 50Vs' MOPD. In the case of Kewaunee and Calvert Cliffs 1 and 2, it was found that failure of a non-qualified pressure regulator under accident conditions could result in the SOVs being subjected to supply pressures in excess of the maximum allowed by the SOVs' MOPD. Eight reported events in which SOVs failed, or had the potential to fail, to perform their safety-related functions as a result of excessive operating pressure differentials are briefly described below. (1) Three Mile Island-1; October 17, 1980; (Ref. 32) The following 11 containment isolation valves could have been prevented from achieving their safeguard positions: 2 makeup to core flood tanks 2 core flood tank sampling 1 reactor building vent 6 fan motor coolers for the reactor building cooling units. (2) Vogtle-1; January 22, 1987; (Ref. 33) 8 main steam isolation valves could have failed to perform their safety function. (3) Kewaunee; November 28, 1987; (Ref. 24) 2 containment isolation valves failed to close 1 pressurizer relief tank makeup 1 RCDT pump discharge (its redundant SOV had the potential for similar failure) 58 other SOVs in safety-related applications were also found to be subject to overpressure failure. PRELIMINARY CASE STUDY 18

t -(4) Calvert Cliffs 1, 2; April 14, 1988; (Refs. 25, 26, 27) The following 40 SOVs, equally distributed between Units 1 and 2, had the potential to fail: i i 8 auxiliary feedwater system 8 steam generator blowdown isolation system 6 reactor coolant pump bleedoff isolation 18 safety injection system (fill and vent) (5) Pilgrim 1; July 19, 1988; (Refs. 34, 35, 36) The following six SOVs had the potential to fail due to overpressure: 4 control room high efficiency air filtration system damper controls (2 in each train) 1 standby gas treatment system damper control 1 primary containment system RCS sample line isolation valve (6) Millstone 2; October 8 1988; (Ref. 37) One containment isolation valve failed as a result of an air pressure regulator that failed high. (7) Millstone 1, 2 and 3; November 8, 1988; (Ref. 28) Unit 1: The status of 16 SOVs in safety-related functions was unknown because of a lack of design information. Unit 2: A total of 24 " harsh environment safety valves and their installed EEQ (sic) solenoid valves" could have failed as a result of overpressure (one of the 24 had failed on October 8, 1988). The licensee also noted that the status of an unspecified number of safety-related SOVs was undetermined because the " data base is incomplete as to solenoid make and model number." i Unit 3: Approximately 20 SOVs installed in " safety valve configurations" could have failed because of overpressurization. The specific appli:ations of these SOVs were not listed. However, the li-censee indicated that there are many additional inaccessible SOVs that may also be susceptible to overpressure failure. The licensee indicated that determination of such vulnerability would be made subsequent to future walkdowns when SOV nameplate data could be obtained. (8) Crystal River 3; November 8,1988, January 5,1989 and January 11, 1989: (Refs. 29, 38, 39, 40) i Five containment isolation valves had the potential to fail due to overpressure: l 2 once through steam generator blowdown lines i l 2 once through steam generator sample lines l I reactor coolant pump seal controlled bleed off line PRELIMINARY CASE STUDY 19 l

b 5.1.4 Directional SOVs We are aware of seven plants that have observed spurious operation of safety-related Target Rock angle-type SOVs due to improper valve orientation. As shown in Figure 3, upstream fluid pressure at the angle-type 50V's inlet port assists valve disc seating. However, many licensees have also learu d l from their own operating experiences and from followup discussions with the SOV manufacturer, that several different models of Target Rock angle-type SOVs used for isolation purposes are " uni-directional" i.e., they will experience undesired seat lifting when the backpressure (pressure at the outlet port shown in Figure

3) is only 2 to 5 psi higher than the upstream or inlet pressure.

As noted in Target Rock Manual TRP 1571 (Ref. 41), the manufacturer has been aware of this problem at nuclear plants since 1978. H wever in the late 1970s time-frame, i Target Rock developed an 50V for use as a bi-directional isolation valve (would not open spuriously due to high backpressures). Target Rock considered the spurious seat lifting to be an Architect Engineer / Licensee " application problem" --not an SOV problem.* The issue of uni-directional isolation SOVs is clearly addressed in some - but not all Target Rock SOV users manuals. For example, l Reference 42 noted that the uni-directional qualities of the Target Rock angle-type SOVs are stated in Target Rock manual TRP 1571 (Ref. 41). i.e. "Most solenoid valves because of the nature of the operation of the valve, will stop flow in only one (1) direction. By design, upstream pressure acts on the top of the disc, forcing it onto its seat, thereby creating a tighter seal. However, if downstream pressure rises above upstream pressure, the disc will tend to lift off of its seat, thereby allowing flow." j Since Target Rock considered the spurious opening of uni-directional SOVs to be an application problem, not an SOV problem, Target Rock did not issue any field service notifications to alert owners of the affected SOVs to this problem. Plants that have experienced spurious openings of safety-related Target Rock angle-type SOVs are: H.B. Robinson 2 (1980) (unspecified number of SOVs) ANO-1 (1985) (2 SOVs) ANO-2 (1985) (2 SOVS) River Bend (1986) & (1989) (3 SOVs) & (10 SOVs) Harris 1 (1987) (2 SOVs) Hatch 2 (1988) (12 SOVs) 1 l The licensees' corrective actions were to re-orient the SOVs to assure that they would operate properly during accident conditions. Section 5.1.4.1 describes the most recent events which occurred at River Bend, I l l i

• Telephone discussion between T. D. Crowley, Target Rock Corporation, anc H. L. Ornstein, USNRC, January 24, 1990.

PRELIMINARY CASE STUDY 20 l'

1 5.1.4.1 Incorrect Valve Orientation at River Bend I In April and May 1989, during testing conducted in response to NRC Generic Letters 88-14 (Ref. 43), the River Bend station found ten Target Rock SOVs used in safety-related applications which would spuriously open during accident con-3 ditions upon loss of instrument air. The opening of those uni-directional SOVs would have resulted in the blowdown of safety-related accumulators and would have prevented safety-related equipment from performing their functions as assumed in plant safety analyses (Refs. 42, 44). For example: (1) Spurious actuation of six uni-directional SOVs upon loss of instrument air would result in bleed-down of safety-related accum-ulators in the control building, the auxiliary building and the i fuel building. The licensee postulated that rapid de accumulators in the control building (in 3.7 minutes)pletion of would pre-vent proper operation of building dagers and would adversely affect cooling of safety-related equipment, control room cooling, and control room air filtration. Dealetion of accumulators in the auxiliary building would affect 3uilding dampers resulting in the loss of cooling of safety-related switchgear. Depletion of accumulators in the fuel building would affect building dampers and would impact air filtration and prevent the maintaining of a negative building pressure. (2) Two uni-directional SOVs were found in the standby service water system (ultimate heat sink) which could spuriously open when subjected to accident conditions to prevent removal of heat through the ultimate heat sink. (3) Two uni-directional SOVs were found in the instrument air system which could spuriously open upon loss of instrument air. Such o plant'pening would prevent long-term operability of all of the s (16) ADS /SRVs. In Reference 42, the licensee also noted that several years earlier (1986) it had found three other Target Rock S0Vs which had to be re-oriented due to spurious opening which was discovered when they were subjected to leak rate testing. Those three SOVs had served as containment isolation valves in the containment hydrogen sampling system. The licensee did not consider that event to be reportable at that time. 5.2 Maintenance Representative operating experience illustrating maintenance problems associated with maintenance frequency, replacement versus rebuilding, contamina-tion, and lubrication are described below. Based on this experience, findings and recommendations relevant to maintenance problems are provided in Sections 7.2 and 9.2 respectively. PRELIMINARY CASE STUDY 21

o ) 5.2.1 Maintenance Frequency 5.2.1.1 Dresden 3 - BWR Scram System - Primary System Leak Outside Primary Containment During recovery from a reactor scram at 81 percent power on September 19, / 1985, Dresden 3 experienced a leak of reactor coolant outside primary contain-ment. The leakage path was through the scram outlet valves and the SDV vent and drain valves (Refs. 45,46,47). The NRC issued Information Notice 85-95 to alert licensees to the potential for reactor coolant leakage into the reactor building which could result from scram solenoid valve problems (Ref. 48). The information notice indicated that a similar event had occurred at Dresden 2 in 1972; however, the licensee did not detemine the root cause of that event. After the reactor scrammed in September 1985, the control room operators attempted to reset the reactor protection system (RPS). RPS channel A was 4 successfully reset, but channel B could not be reset.* This channel configura-tion allowed the scram pilot SOVs to vent air, resulting in reduced air header pressure. The reduced air header pressure (38 psig) was sufficient to allow the SDV vent and drain valves to open (opening pressure s8 to 15 psig), but it was not sufficient to enable the scram inlet and outlet valves to reclose (s42 psig required to close). For approximately 23 minutes, reactor coolant leaked outside primary containment into the reactor building. The high temperature reactor coolant flashed to steam. The leak resulted in elevated radiation levels on the first three floors of the reactor building. j In addition to the anomaly associated with the half scram configuration, degraded scram pilot SOVs contributed to the event. Testing showed that leaking scram pilot SOVs resulted in a combined SDV air header leak of 25 scfm. The licensee fand widespread wear, aging, and hardening of the 50Vs' 0-rings and diaphrages. Maintenance records showed that some of the worst leaking valves had been rebuilt during the previous refueling outage. After a reactor scram, the SDV and the scram instrument volume are in direct contact with hot pressurized reactor water. A common-mode failure of the pilot SOVs controlling the scram discharge system vent or the drain valves could result in an uncontrolled release of reactor water outside primary containment (see Figure 7)'until the scram is reset. Such an event occurred at Hatch 2 in August 1982 (Ref. 49). Similarly, a sluggish SOV piloting an SDV drain valve caused a water hammer at Brunswick 1 which resulted in damaged pipe supports in the SDV drain system (Refs. 50, 51). As noted in Reference 46, a severe water hammer in the SDV system could result in an uncontrolled *ieak of reactor water outside the primary containment. Discussion with Gi** has indicated that since Information Notice 85-95 was issued. BWR owners hr.ve made improvements in their SDV systems so that there are redundant SDV veit and drain valves at all U.S. BWRs (vs. only one vent and one drain valve per SDV header in the early 1980's). However, it is not certain that all U.S. BWRs have manual handwheel overrides for the SDV vent and drain valves to limit reactor water leakage outside primary containment in the event of a common-mode failure of the SOVs piloting the SDV vent and drain systems.

• Channel B remained tripped because of stuck contacts on the reactor mode switch.
  • Telephone discussion between G. Strombach and E. Giebo, GE, and H. L. Ornstein, USHRC, June 23, 1989.

PRELIMINARY CASE STUDY 22

i I o 1 'Y/!OWOl/* NA? /OAUNNA1}AjNNOA//NYllllN///////N///s 71 //N/////pffpffffffffffffffffffffG NO CONTROL h 4 / AIR StJPPLY p m r / M SCR AM AIR HE ADt R f j 1 ra t i r-- / REACTOR ] VESSE L DR YWit t l i TORtf5 u g - - RpS mpS A~ j i (PRIMARY i CONTAtNMENTI gygygg i k M Jt i POMPS fiE ACTOR p 5 CORE OOO RPS

• a-Q

! 9 h' \\ ,,h 'EE[,T -4 h m d, 'l h VAL /ES - -.;, i--- E f EbE I f, ~ OOO ~ M + --4 c-i o CONYROL ROD c;Rau ' 1 i I,, L RF S ~B 1 t t SUBCHANNEL RELAYS - t '911OT ~< DRtVE S 1 1 Lf 1 3 ' gg m s "",-"/ ' t to TYP. OF 189 s"" 1r_ t d p N I <l: , i ir-M-: ( t ""~ 2 OOO l r: e: ei b SCRAM w - f I VALVE SCRAM y g j# N VALVE I t t X N+t SCRAM J d f E VAtvE

=

g u N+2 3lr - # t h^ CHANNEL 8 9,.y m n ArS ouSr g A 4 120 VAC SDV SYs*m s u VmwVaswo N 8 A W SCF AM Insw. ( -1,~. *. i. Vh REACTOR BtHtOfNG ' C.-{r F- ~ ?.:-=. . _9.~=~~-83I8_ F. E CONTA8NMENil I 1 \\ (SECONDARY F -f ~ ~.*_C.- CC: SDV System h Dram,va% \\ l r SCH AM DfSCHAR iE To Cican Radwes e VOLUME HEADE9" i i Open De Raf=sste [ M NN. To C1m e Radwegee L "' N 1 vswannna vess/zzar::=::==::::..-...:::::_::: z_. ---us wm 8 I R p A f' gCRO 3. , TORUS

• *
• C"" S" l

-;.r.-

p. p S.,,,, e, [ RHR '"^ =CS ~ RHR upcp RCSC anus CS h ? ~ / .-. s

== mi c.- y / / ^ / -~ 1 Figure 7 BWR SCRAM System-litustrating Leak Path Outside Containment I i

-l 5.2.1.2 Perry - Simultaneous Common-Mode Emergency Diesel Genarator Failures On February 27, 1987, the Perry Nuclear 01 ant experienced simultaneous common-mode failures of both emergency diesel cenerators (EDGs) (Ref. Sla), i The failures were attributed to excessive air leakage through SOVs on each i EDG's control panel.. The SOVs were Humphrey Proitucts Model No. TOG 2El-3-10-35 i which were supplied by Delaval as EDG piece parts. The SOVs are 3-way air control valves which are continuously energbed while the EDGs are in standby. The licensee had previously identified thost SOVs for replacement due to observed air leakage. Work requests had been initiated for replacement of those SOVs but at the time of their failures, the work requests had not yet been implemented. Discussions with the licensee

• and the EDG manufacturer ** revealed the following information:

1 The failed SOVs had been in service for over 10 years. l 2 The analysis of the SOVs found that the elastomeric parts (Buna N) were " dried up and cracked" 3 The failure was attributed to long-term operation at elevated temperatures '4 The Humphrey valves were purchased by Delaval as com-mercial valves and were upgraded / dedicated for nuclear service by Delaval. Delaval did not provide specific maintenance instructions for the SOVs. 5 The changeout frequency of the SOVs is not specified in l theDelavalOperator'sManualjhowever,itcouldbe implied from the manufacturer s control panel environmental qualification report. 5 Although the SOV manufacturer has suted that SOV failu m have occurred because of incorrect use of 1 #.t nt.s on the Buna-N parts, the licensee was not mvided with any such instructions. 7 The Perry plant upgraded the SOVs to ones with Viton instead of Buna-N; and more recently they replaced the SOVs with electrical relays. 8 We are uncertain about the vulnerability of other nuclear power plants having Delaval EDGs with Humphrey SOVs similar to the ones that failed at the Perry plant in February 1987. 5.2.2 Replacement Versus Rebuilding 5.2.2.1 MSIVs at Perry - Inadequate SOV Rebuild l After determining the cause of the MSIV failures of October 29 and Novem-ber 3, 1987, the licensee replaced or rebuilt the ASCO SOVs on the MSIV air packs. Due to the limited availability and long lead times for replacement l

• Telecon H. L. Ornstein USNRC and R. DiCola, Cleveland Illuminating Co.,

1 May 29-30, 1990.

• • Telecons H. L. Ornstein and D. Pescut and S. Owyoung, Cooper Industries (formerly Delaval) May 29-30, 1990.

PRELIMINARY CASE STUDY 24

parts (air packs and ASCO dual-coil NP8323 SOVs), the licensee had to rebuild some (rather than replace all) of the MSIV air pack SOVs. One entire air pack was replaced for the inboard D MSIV. One dual-coil NP8323 50V was replaced for the outboard D MSIV air pack. One dual-coil NP8323 SOV was replaced for an inboard MSIV that had not failed previously. It was replaced upon inspection because it was observed to have sustained heavy damage to the electrical coils due to moisture intrusion. Five dual-coil NP8323 SOVs were rebuilt, including the inboard B MSIV which had failed on October 29, 1987. The licensee conducted increased surveillance and testing of the MSIVs after repairing and replacing the air pack 50Vs. The licensee initiated monthly operability testing of the MSIV air pack SOVs, quarterly fast closure timing tests, and inspecttons of the ASCO NP8323 dual-coil SOV experiencing the high-est temperatures. On November 29, 1987, while performing operability testing, the ASCO dual-coil NP8323 SOV, controlling the inboard B MSIV, failed to change state when it was de-energized. Examination of the failed SOV found that the failure was caused by several f oreign particles in the 50V. Laboratory examination con-firmed that the particles were EPDM from the 50V's 0-ring which had been replaced during the 50V's rebuilding process subsequent '.o the November 3, 1987 failure (Refs. 8, 9). Apparently, during the original SOV rebuilding process, the licensee did not completely disassemble the ASCO dual-coil NP8323 SOV. As a result, one or more small particles remained in the valve and remained undetected until it (they) caused the 50V's failure.* To preclude additional failures due to foreign particles remaining from the rebuilding process, as had happened on November 29, 1987, the licensee re-placed all eight ASCO dual-coil NP8323 SOVs with new ones. Furthermore, the licensee stated.that tney were going to modify their preventive maintenance i program: in the future, all Class 1E ASCO SOVs will either be replaced with new valves or undergo complete disassembly and cleanout to ensure that no particles remain or are introduced during the rebuilding process. 5.2.2.2 Brunswick 1 - Safety Relief Valves - SOV Rebuilding Error: Excess Loctite On July 1,1987, while attempting to control pressure following an unplanned automatic reactor trip, an SRV failed to open on demand. Following shut down, the licensee tu ted the SRVs that had not cycled during the trip recovery and found another SRV that did not open on demand (Refs. 52, 53), i The SRV failures were due to SOV failures. The two SOVs that had failed (Target Rock Model 1/2-SMS-A01) are used to port air to the SRVs' actuators.

• It is believed that one particle remained in the 50V, and that the particle broke up during subsequent SOV operation.

PRELIMINARY CASE STUDY 25 l ,c

i allowing remote-manual opening of the valves. The two SRVs that failed were part of the plant's Automatic Depressurization System (ADS). The failure of both safety relief valves to open on demand was attributed to excess Lottite RC-620, which was found in the internals of the related SOVs. Although two additional valves were found to have excess Loctite on the 50V's internals, those valves did not exhibit signs of binding. The licensee determined, with the assistance of the SOV manufacturer, that Loctite RC-620 had been used by the 50V manufacturer's field service representa-tive while rebuilding the 50V during'a previous outage. In Reference 52, the licensee noted that the manufacturer s (Target Rock) field service representa-tive had rebuilt all of the Brunswick 1 SOVs that actuate all eleven of the plant's SRVs (seven ADS valves and four non-ADS valves). The licensee statW that the Target Rock field service representative had done SOV refurbishment work on the valves at BrunMck 1, but he had not done similar work on any SOVs which pilot SRVs at other plants. Target Rock field representatives service the SRVs for all U.S. BWRs (except for Browns Ferry 1, 2, and 3) at Wyle Laboratories during the plants' refueling outages. Most plants send their SRVs and SOVs to Wyle for refurbishment every refueling oitage, but some only send half of their SRVs and SOVs to Wyle for such refurbishment each refueling outage. The problem encounterei with Loctite RC-620 was one of excessive applica-tion. Loctite RC-620 is an anaerobic adhesive. Curing takes place in the absence of air. The SOV manufacturer's refurbishment procedure specifies that Loctite RC-620 be applied to a locknut assembly beneath the val 4 plunger. The procedure cautions against opplication of excessive amounts of tce adhesive. The licensee concluded that the SOVs had excess amounts of Loctitt RC-620 applied to them, and that curing di,1 not occur until after the valves were placed in j the inerted containment. The licensee believed that, prior to curing, the excess J adhesive migrated to the interior of the valves, bonding the 50Vs' plungers to the bodies of the valves. The licensee concluded that even though only two ADS SOVs were found to malfunction, two other ADS SOVs had similar bonding due to excess Loctite RC-620; however, those bonds were broken during the initial removal and handling of the SOVs when they were removed from the dr>well and bench tested. The licensee's assessment of the evtnt (Ref. 52) concluded that a common-mode failure, the inoperability of all li SRVs as a result of Loctite RC-620 bonding of all SOVs by one vendor field service representative, is a reasonably credible event. The occurrence of a desige basis event under such conditions is beyond the bounds of the plant's final shf*+y analy

• repoet.

The NRC staff issued Information Notice 87-48 (Ref. 53) to notliy licensees of the July 1, 1987 event. A similar SRV failure occurred on July 25, 1980 at Pilgrim (Ref. 31). A Target Rock SRV failed to open on a manual demand signal. The failure was caused by excessive Loctite RC-620, which had caused the StV's solenoid plunger to stick to the valve's bonnet. In this case, the excessive Loctite was used during the fabrication of the SRV (as opposed to the July 1, 1987 event at Brunswick in which the excess Loctite was applied during refurbishing). l PRELIMINARY CASE STUDY 26 \\

5.2.2.3 Peach Bottom 3 - Scram System - SOV Rebuilding Error: Excess Loctite On november 17, 1983, a control rod was observed to have an excessive insertion time during a reactor scram (Refs. 54, 55). The sluggish control rod insertion was attributed to the failure of an SOV* to shift position to allow control air to be exhausted from the control rod's hydraulic control unit. As a result, the licensee replaced the scram pilot SOVs associated with the control rod that did not scram promptly and sent the scram pilot SOVs to GE for failure analyses. On January 14, 1984, during a reactor scram, another control rod did not insert within the technical specification allowable time of 7 seconds. The i second control rod had acted sluggishly during the November 17, 1983 scram, i However, because it was believed to have inserted within the technical specifi-cation allowable time on November 17, 1983, no maintenance was performed on its j pilot SOVs at that time. Subsequent to the second failure (January 14,1984), the licensee undertook an extensive investigation. That investigation revealed that, contrary to pre-vious findings, the second control rod also had failed to meet its allowable scram insertion time limit on November 17, 1983. Laboratory analysis of the two pairs of SOVs associated with the slow inserting control rods revealed that one valve of each pair had a yellow varnish-like foreign substance on its core assembly. One of the SOVs which was found to have the foreign substance on it exhibited sticking during subsequent bench 1 testing. The foreign substance was originally believed to be a silicone lubri-1 cant, but it was later it.entified to be Loctite 242. Loctite 242 had been intro-duced to the SOVs during the rebuilding process, in accordance with the supplier's (GE) recommendations. In a 1978 Service Information Letter, (SIL) 128, (Ref. 56), GE had recommended that when rebuilding CRD scram pilot valves Loctite 242 adhesive / sealant should be used to secure the " acorn nut" on the solenoid housing to prevent it from loosening. 1 The Peach Bottom 3 failures were attributed to excess Loctite 242 which was used in the rebuilding process. It had appeared to be fully cured and the excess had not been wiped off. When the system rett:ened to service, the Loctite 242 migrated and hardened and bonded the 50V's core plunger to its base assembly. After determining the source of the sticking, the licensee eliminated the use of Loctite 242 from its rebuilding process. Subsequsntly, GE issued a supple-mentary service information letter, SIL 128 (Ref 57) which recommended that all BWR owners discontinue using Loctite 242 or any other chemical adhesive thread lockers on the acorn nut of the pilot SOVs, GE had originally recommended using Loctite 242 to overcome loosening of the." acorn nut", and ASCO had agreed. Following the sticking problems at Peach Bottom 3, ASCO made a design change and replaced the acorn nut with a nylon-lined locking nut which would not require adhesive thread lockers to remain tight.**

• ASCO Model HVA-90-405, which is built by ASCO but procured from GE, it is similar to the ASCO Model NP8316 val /e.
  • Telephone discussion between J. Shank, ASCO, and H. L. Ornstein, USNRC, June 19, 1989.

PRELIMINARY CASE STUDY 27 l

The common-mode failure potential for the scram system at some BWRs exists because some plants have used the same SOVs that are used to pilot the individ-ual control. rod hydraulic control units to pilot the scram discharge volusse vent and drain valves. In the case of Peach Bottom 3, the potential for multiple t simultaneous failure was compounded by the fact that the licensee had rebuilt all 370 control rod scram SOVs during the previous refueling outage. To reduce this common-mode failure potential, GE's SILs (Refs. 56, 57) recommended (not a binding requirement) that CRD pilot SOVs be rebuilt on a staggered basis from a " distributed checkerboard pattern." 5.2.3 Contamination i I 5.2.3.1 Brunswick 2 MSIVs - Excessive Heat and Poor Air Quality (Hydrocarbons and Water) On September 27, 1985, duritg surveillance testing at Brunswick 2, three of the plant's eight pneumatically operated MSIVs failed to fast close (Refs. 58,59). There are two MSIVs in aeries in each of four parallel steam lines. Two of the valves that failed to fast close were on the same steam line. An investigation of the. failures found that the MSIVs failed to close because of I disc-to-seat sticking of the MSIV air pack SOVs (ASCO dual-coil Model NP8323). The internal 0-rings on the SOVs also were found to be degraded; they were brittle, and several 0-rings were stuck to the valve body. Several 50V discs came apart after becoming brittle: pie;;es of one 50V disc became wedged in the 50V's exhaust port, one disc stuck to the exhaust port, and another SOV lost a piece of its disc. Laboratory analysis of the three failed SOVs showed the presence of a significant amount of hydrocarbon in them. The combination of hydrocarbons and elevated temperature caused the EPDM discs to swell and fill the SOVs' exhaust ports which blocked the discharge of air in the air actuator and increased the frictIonalforceopposingSOVcoremovement. The instrument air system was believed to have been the source of the hydrocarbon contamination. Because of the susceptibility of the 50Vs' EPDM parts to hydrocarbon contamination, the licensee replaced all of the SOVs with the same model 50V having Viton discs and seals. Compared to EPDM, Viton is less susceptible to hydrocarbon contamination, but it is more susceptible to radiation damage. This event was reported to Congress as an abnormal occurrence. The abnormal occurrence report categorized the event as one which resulted in the " loss of plant capability to perform tw ential safety functions such that a potential release of radioactivity in excess of 10 CFR Part 100 guidelines could result from a postulated transient or accident" (Ref. 60). 5.2.3.2 North Anna 1 and 2 - Multiple Systems - Oil and Water Intrusion While performing maintenance operations at North Anna in the morning on April 24, 1987, an operator error resulted in a service water intrusion into the Unit I and 2 instrument air systems (Refs 61-64).* The licensee quickly recognized that the service water intrusion affected SOVs and pneumatic con-trollers for auxiliary feedwater systems, primary and secondary pressure

• Telephone discussions between J. Lewis and J. E. Wroniewiez, Vepco, and H. L. Ornstein, USNRC, May 1989.

PRELIMINARY CASE STUDY 28 .-. ~

P control systems, and the SOVs required for containment isolation (" trip valves") for both Units 1 and 2. At the time of the event, Unit I was in mid-loop operation and Unit 2 was operating at 100 percent power. The licensee's immediate response to the event was to continue operating Unit 2 and to blow down the affected instrument air lines. About 2-1/2 hours after the intrusion occurred the licensee tested the Unit 2 "A" motor-driven AFW pump. The air-operated discharge valve and the back-pressure regulating valve for the AFW pump both malfunctioned rendering the pump inoperable. About three hours later the licensee tested pump B satisfactorily. Throughout the evening of April 24, 1987, the licensee continued to blow down instrument air lines until no moisture was observed. The "A" AFW pump's discharge and pressure regulating valves were repaired on the evening of April 24, 1987 and were satisfactor11y tested around midnight. The cleanup procedure was not totally effective since there were low points in the instrument air system that had not or could not be drained. The residual water that remained in the low points of the instrument air system and the mois-ture and contaminants in the instrument air system resulted in widespread SOV failures for almost two years after the service water intrusion event. In addi-tion to failures of " freestanding" SOVs, there were dozens of control valve failures. The bulk of the control valves that failed were Fisher control valves. Integral to each Fisher control valve is an ASCO SOV. The Fisher control valve failures were essentially failures of the ASCO SOVs which are piece parts of the control valves. Examination of plant equipment failure records noted that between April 1987 and February 1989, there were approximately fifty Fisher control valve (ASCO SOV) failures. It appears that those failures resulted from poor quality air due to the April 24, 1987 water intrusion event and from poor maintenance of the instrument air system. In addition to these failure records, NRC inspectors noted (Ref. 62) many ASCO SOV failures that had been observed during surveillance testing after April 24, 1987, were not reported and the SOVs were not repaired. The primary reason was that the SOVs that failed to operate during surveillance testing operated properly after being tapped (" mechanical agitation") by plant personnel. As a result of such practices, repetitive malfunctions were observed, the mal-functioning SOVs were not fixed or replaced expeditiously, and the root causes were not found or corrected on a timely basis. Characterization of the licen-see's in-service testing practices regarding SOVs was cited in Reference 61 as follows: "The process of tapping on solenoid valves and repeated cycling of valves prior to running a satisfactory surveillance was considered an acceptable practice by the licensee." Some of the systems that were affected by malfunctioning ASCO SOVs (freestanding or piece-parts of Fisher control valves) due to contamination of the instrument air system are listed in Table 2. l PRELIMINARY CASE STUDY 29 \\

v + Table 2~ Systems Impacted At North Anna By SOV/ Control Valve Failures Due to Service Water Intrusion / Instrument Air Contamination Unit I and Unit 2 Residual Heat-Removal / Low Pressure Safety Injection Main Steam Relief (PORVs) Auxiliary Feedwater Component Cooling Water Unit 2 only Containment Isolation Containment Fan Cooling Main Steam Isolation In a February 10, 1988 memorandum, the Chairman of North Anna Station's Nuclear Safety and Operating Committee stated that successful stroking of the l SOVs is an appropriate corrective action to remove contaminants, because " cycling the affected valves blows the contamination from the lines and returns the SOVs to operable status" (Refs. 65, 66). North Anna Station's approach to maintenance of malfunctioning SOVs contradicts the valve manufacturer's recommendations. ASCO's installation and maintenance instructions and the licensee's telephone discussions with ASCO on February 4 and 5,1988 advised the licensee that, after SOV contamination, the NP Series SOVs should be inspected for corrosion, sediment or other contaminants, and cleaned accordingly.* 3 A meeting was held at NRC Region II offices on February 7,1989 to discuss repetitive AFW system control valve failures which occurred in January 1989, due to moisture in the. instrument air system (Ref. 67). At the meeting, the licensee acknowledged that widespread L m c of SOVs, control valves and air-operated valves had occurred during the n -'ths from the time of the service water intrusion into the instrument air sysb (April 1987 through January 1989). A large number of repetitive SOV and control valve failures were attributed to poor quality instrument air (oil and moisture contamination in addition to the April 1987 service water intrusion). The licensee noted that attention had been focussed on the quantity of instrument air available without paying attention to its quality and indicated that subsequent to a review of their instrument air i system, a program was initiated to clean or replace the affected equipment. The s licensee also provided information on steps that were being taken to improve the instrument air system to assure delivery of clean, dry, oil free instrument air. We view the April 24, 1987 service water intrusion into the instrument air system as a significant It resulted in widespread degra-dation of SOVs, controllers, precursor event.and air-operated valves that had the otential for disabling many systems needed to achieve safe shutdown. If a desi n-basis event

• Telephone discussions between F. Maiden and W. Murray, Vepco, and K. Thomas, ASCO, February 4 and 5, 1988.

PRELIMINARY CASE STUDY 30 ---_------_o

l had occurred at Unit 2 on April 24, 1987 the instrument air system, the operators, before removing the service water from ability to bring the plant to a safe shutdown could have been seriously impaired. A large number of 50V and control valve failures occurred at Units 1 and 2 between April 24, 1987 and January 1989 as a result of water, corrosion products, and residue from the service water intrusion, and from impurities introduced by poor quality instrument air. This event exemplifies the necessity for providing SOVs with clean, dry, oil free air, and the need to thoroughly clean and inspect the equipment if water or other contaminant intrusions occur. 5.2.3.3 Susquehanna 1 and 2 - Scram System: 011 and Water Contamination The Susquehanna plants have experienced comon-mode failures of SOVs that resulted in multiple failures of control rods to insert, slow insertion of multi-pie control rods, and repetitive failures of scram discharge volume vent and drain valves. The SOV failures were linked to contaminants in the instrument air system (i.e., hydrocarbons, water, and particulates) and high temperatures.* Because both Susquehanna units share a common instrument air supply, the common-mode failure potential that existed for both unit I and unit 2 scram pilot SOVs also existed for the SOVs that actuate both units' backup scram valves. The backup scram valves are intended to provide diverse scram capability to protect against comon-mode failures. Although Unit 1 experienced the failures, the potential for such failures also existed at Unit 2; both units' scram and diverse scram systems were vulnerable. The Susquehanna SOV failures illustrate the potential for multi plant common-mode failures leading to events that are beyond the plant safety analyses (i.e. failure of multiple control rods to insert and unisolated primary leak outside containment via the scram discharge volume). A summary of the Susquehanna SOV failures are described below: On October 6, 1984, while Susquehanna 1 was operating at 60 percent power, two control rods failed to insert during individual rod scram testing._ Further scram testing revealed that a total of four rods would not insert and nine additional rods hesitated before inserting. A similar event occurred previ-ously at Susquehanna on June 13, 1984, when several control rods hesitated momentarily before inserting (Ref. 68). Two of the control rods that failed to insert on October 6 had not met the plant Technical Specifications scram time requirements on June 13. The licensee did not become aware of the June 13 mal-functions until the October 6 failures were investigated. The October 6 failures were attributed to common-mode contamination of the instrument air system. The combination of contaminants (oil and/or moisture) and high temperatures (140'F) caused the 50V internals to degrade and become stuck. The SOV polyurethane disc holder subassembly seats were found to be stuck to the 50V exhaust port orifice. This prevented air from the scram inlet and outlet valve operators from bleeding off through the SOV exhaust ports, which prevented the scram inlet and outlet valves from opening.

• At Susquehanna each of the 185 control rods is piloted by one ASCO HV-176-816 SOV.

Many other BWRs' control rods are piloted by other model ASCO SOVs, but two per control rod. The ASCO SOVs used in U.S. BWR scram systems are typically procured from GE. PRELIMINARY CASE STUDY 31 .I~ ~

Two independent laboratories examined the failed SOVs and concluded that the polyurethane parts degraded because of a combination of contamination in the instrument air and elevated temperature (Ref. 69). The first laboratory i (Franklin Institute) cited the failure mechanism as hydrolytic decomposition of the polyurethane seats due to a combination of moisture and elevated tempera-tures. The second laboratory (GE) indicated that polyurethane seat failure i was caused by contamination of the instrument air with a synthetic diester oil (SDO, which is a plasticizer). Both Franklin Institute and GE recommended replacing the polyurethane seats with a seat material capable of operating at higher temperatures and having an improved resistance to contaminants. The recommended material was Viton. The licensee replaced all of the 50V polyurethane seats on Units 1 and 2 control rods and all the backup scram valves. About half of the 50V discs for the Unit 2 control rods had already been replaced in L 1983 with Viton discs. l The licensee's investigation found that the SOV for the scram discharge volume vent and drain valves on Unit I had a polyurethane disc that also was susceptible to the same type of failure. The SOVs for the vent and drain i valvec also were replaced with different SOVs having Viton discs.* The October 6, 1984 scram system degradation at Susquehanna was reported to Congress as an abnormal occurrence (Ref. 70). The NRC staff concluded that the event involved a " major degradation of essential safety-related equipment," and demonstrated the plant's susceptibility to commo n ode failure. The failure caused a reduction in "the required ' extremely high probability' of shutting down the reactor in the event of an anticipated operational occurrence" (Ref. 70). Another scram discharge volume (SDV) system component failure attributed to contaminated air occurred at Susquehanna 1 on December 21, 1984 (Ref. 71). Dur-ing surveillance testing, an SOV that controls the SDV vent and drain line isolation valves malfunctioned as a result of particulate matter that was lodged between the 50V's disc and seat. As a result, the SDV vent and drain valves were stuck open. Since the reactor was at power, if the SOV had failed to completely close after a scram, the potential for an unisolated primary leak outside containment would have significantly increased. I

5. 2. 4 Lubrication 5.2.4.1 Multiple Plants - Manufacturing Error:

Residue-Producing Lubricant The Kewaunee nuclear power plant experienced three SOV failures on May 28, 1 1988 during surveillance testing (Ref. 72). Two of the SOVs were redundant containment isolation valves piloting the reactor coolant drain tank discharge header isolation valves. The third SOV that failed served as the pilot for the pressurizer relief tank makeup isolation valve. All three fail q SOVs were nuclear qualified ASCO NP8314 DC valves that piloted air-operated valves. They were normally open, normally energized, and were designed to close (fail safe) on loss of instrument air or electrical power. The failures of the SOVs to 1

• The SOV chosen was a larger size, made by another manufacturer.- The original Unit I valve was undersized and the replacement made was the same as the one on Unit 2.

PRELIMINARY CASE STUDY 32

shift position upon de-energization were attributed to an amber-colored residue inside the 50Vs. The residue was found at the location where the SOV core as-sembly (plug) contacts the SOV body (solenoid base sub-assembly) see Figure 4. ) The failed SOVs had been in service about 18 months prior to their failure. The local ambient temperature was about 110*F. The licensee inspected two other ASCO NP8314 SOVs from the same manufacturing lot which were installed adjacent to the three SOVs that had failed. They had been installed at the same time as t the ones that failed, but were operated in the de-energized mode. The de-energized SOVs had performed satisfactorily. The licensee assisted by two independent laboratories (Wyle Laboratories and Akron Rubber Development Laboratory) and ASCO conducted an extensive inves-tigation to determine the root cause of the failures. On the basis of the investigation, the licensee and ASCO concluded that the 50V failures were most litely caused by the degradation of a lubricant (International Products Corporation, "P-80" rubber lubricant) which had been introduced during the manufacturing process. P-80 is a water-based rubber lubricant used by ASCO l personnel to facilitate 50V assembly. Although P-80 was an approved lubricant for use at ASCO's manufacturing facility, its use for the assembly of the NP8314 SOVs was not an explicitly approved procedure. P-80 product literature states i thatitprovides"temporarysligperiness"forassemblingrubberparts,andthat it is absorbed into the rubber leaving no residue or harmful effect on the rub-ber." Subsequent to SOV assembly (using the P-80 lubricant), the SOVs were cleaned; however, minute amounts of the P-80 lubricant remained within the inter-nal cavities of the SOV. From the laboratory results, it was concluded that the small amount of lubricant, remaining in the SOVs, migrated subsequent to ener-gization, and the heating, due to energization, degraded the P-80 to form the amber-colored sticky residue which caused the 50V malfunctions. The investiga-tion discounted Dow Corning 550 lubricant as the source of the residue that had been found inside the NP8314 SOVs. ASCO has discontinued using P-80 in the assembly of S0Vs as a result of the investigation. On October 18, 1988, ASCO issued a 10 CFR Part 21 notification regarding the potential failures of NP8314 SOVs (Ref 73). The notification accounted for 231 suspect SOVs that were sent to 17 U.S. LWRs, 76 suspect SOVs that were sent to sup) liers who most likely shipped them to unspecified plants as piece-parts of otler equipment between 1981 and 1988, and 9 suspect SOVs that were sent to Franklin Research Center (FRC) in 1986. The Fort Calhoun plant had receivedthelargestnumberofsuspectSOVs(79)in?981. Several of those S0Vs failed at Fort Calhoun in 1981 and 1982. Thret ;f the SOVs that failed at Fort Calhoun were returned to ASCO for investigation. ASCO's investigation of those valves, incident report IR 3604 - May ?982 (see NR0 Vendor Inspection Report 99900369/88-01 (Ref 74), noted that the failures were due to sticking caused by a varnish-like residue. At that time, neither ASCO nor the Fort Calhoun licensee were able to identify the source of the " acrylate ester residue found on the plunger and sub-base assembly" of the energized NP8314-S0Vs. u Fort Calhoun experienced a similar failure of another energized NP8314 SOV in March 1982. It was cleaned and returned to service (Ref. 75). The licensee l stated that it would replace the internals of all the NP8314 S0Vs using new spare parts kits. Subsequently, Fort Calhoun donated 10 ASCO NP8314 S0Vs that L had been in continuously energized service for 18 months to FRC for use in an PRELIMINARY CASE STUDY 33 r

NRC sponsored 50V aging research program (Ref. 71). FRC also purchased nine new NP8314 SOVs from ASCO, which were shipped in April 1986, to be used in FRC's SOV aging program (those SOVs were also listed in ASCO's 10 CFR Part 21 notifica-tion). Six of FRC's purchased SOVs, which were under aging, failed prematurely (failure to shift position) going accelerated thermal as a result of organic deposits (" sticky substance"). After the deposits were " cleaned away" with acetone and the SOVs were reassembled, they performed successfully for the dura-tion of FRC's testing program. FRC's report (Ref. 76) also noted that organic deposits were found in the NP8314 SOVs received from Fort Calhoun. FRC believed that the sticky deposits that had prevented the SOVs from functioning were due to an organic compound that was introduced during the assembly of the valves; however, a detailed analysis and final determination of the source of the deposits were not aursued by FRC because of budgetary restraints of the program. In the course of tie FRC's SOV aging research program, ASCO had been apprised of the sticking problem, however ASLO did not find the source of the residue (P-80) until after the Kewaunee failures in 1988. The failures of the NP8314 SOVs indicate that P-80 was used to assemble the NP8314 SOVs as early as 1981 and as late as 1988. A similar case, in which another SOV manufacturer used a lubricant to assist with SOV assembly, also resulted in subsequent 50V performance problems. As noted in Reference 77, Target Rock Corporation used castor oil as a lubricant to facilitate the assembly of its two stage safety relief valves (SRVs). After investigating several SRV failures, it was found that castor oil, which was used to lubricate silicone rubber 0-rings, caused swelling and accelerated degrada-tion of the 0-rings. Subsequently, Target Rock discontinued using castor oil as a lubricant. DAG-156 lubricant (carbon particles suspended in an alcohol base)wasusedtoreplacecastoroil. We are not aware of any subsequent Target Rock SRV failures that have resulted from the use of DAG-156. Target Rock informed the author of this case study during a visit to their facility hadused{mineraloils"tofacilitate50Vassembly. November 1988) that, paralleling This practice was dis-continued in the mid-1980s and DAG-156 was chosen as a replacement for mineral oils. 5.2.4.2 Catawba: Poor Quality Air and Lubrication with Vaseline The Catawba nuclear power plant experienced common-mode failures of EDG starting air system inlet valves (Refs. 78,79,80). The EDGs were manufactured by Delaval. The air start system inlet valves, model T-3618, were made by California Controls Co. (Calcon). These two-stage air-operated valves each have a solenoid pilot valve that is normally closed and requires de power to actuate the solenoid pilot to admit starting air into the EDG. The licensee has reported five instances of common-mode failure of these valves. The valves stuc( open when a sticky, slimy substance formed inside the poppet portion of the valve. The licensee determined that the substance was the silicone lubricant Dow Corning 111 that was used on the valves. On five occa-sions the licensee cleaned the valves and replaced the Dow Corning 111 with Vase 1Inepetroleumjelly. Calcon's recommended lubricant is GE Silicone fluid G-322-L, which is significantly different from DOW Corning 111. The licensee PRELIMINARY CASE STUDY 34

didnotcheckforthecompatibilityofVaselinepetroleumjellywiththeBuna-N rubber used in the Calcon Valve. Low nitrile Buna-N rubber degrades when in contact with petroleum based products. After reviewing the EDG air start valve l failures and other EDG pneumatic equipment failures (Calcon pressure sensors) the licensee concluded that the sticking was caused by moisture interacting with the Dow Corning 111 silicon lubricant. The source of the moisture was the starting air system, the root cause was inadequate dryer maintenance (the licensee's failure to changeout the spent desiccant). I Subsequently, the licensee upgraded its maintenance on the air dryers, thereby lowering the starting air moisture content. In addition, the licensee cleanedthevalvesandreplacedtheVaselinepetroleumjellywithDowCorning 111 lubricant. Theseactionsinconjunctionwithmorefrequentchangeoutof the Calcon gas valve's elastomeric parts in accordance with the Delaval owners' group plant specific recommendations appear to have eliminated the valve sticking problem, 5.2.4.3 Common-Mode Failure of 16 MSIVs at a Two Unit Station (BWRs): l Incorrect Lubrication In July 1986, the licensee of a two-unit station reported excessive stroke time of the Unit 1 '!C" outboard MSIV which resulted from a failure of an Auto-1 matic Valve Corporation (AVC) SOV (model C4988-8). The failure was attributed-to " poor workmanship from the factory" and " improper lubrication which would I allow the valve piston to jam at a certain place in the valve.",The failed AVC valve was replaced with a new one. Five months later (December 1986), while performing monthly closing tests, the licensee found that the Unit 2 "B" inboard MSIV did not stroke properly as a result of a failure of another AVC SOV. The licensee shut down both units from 100 percent power and inspected the SOVs piloting all 16 MSIVs. The licensee found that the AVC SOVs on all 16 MSIVs were damaged. The three-way and four-way valves and solenoid pilot valves on all 16 MSIVs had a hardened, sticky lubricant in their ports and on their 0-rings. As a result, motion of all the SOVs was impaired, resulting in instrument air leakage and the inability to operate all of the MSIVs satisfactorily. The licensee also examined unused spares in the warehouse and found that the lubricant had dried out in those valves, leaving a residue. Several of the warehoused spares were bench tested. They were found to be degraded and they also leaked. i The original " approved" or " preferred" S0Y lubricant (based upon equipment qualification testing) was Parker Su)er-0-lube. However, later equipment quali-fication testing (1985) found that tie Parker Super-0-lube could cause SOVs in the MSIV air pack to malfunction. The Parker Super-0-lube was found to break down to an adhesive, powdery substance when exposed to radiation fields greater than 1x10E6 RAD. Because of the potential for breakdown of Parker Super-0-lube and binding of the SOVs in the air packs, the licensee changed the SOV lubricant to E. F. Houghton SAFE 620. In separate telephone conversations 1.he SOV manufacturer (AVC) told the AEOD staff that it had informed the utility that E. F. Houghton SAFE 620 PRELIMINARY CASE STUDY 35

lubricant attacks and degrades the aluminum in the AVC valves.* Nonetheless, i in accordance with utility purchase orders, AVC shi E. F. Houghton SAFE 620 to two different utilities.pped SOVs lubricated with After the multiple failures occurred in December 1986, General Electric (GE) informed the licensee that the Parker Super-0-lube is an acceptable lubricant i "if it is applied in a ' thin film'." AVC and GE had concluded that the problem experienced with Parker Super-0-Lwe in the 1985 qualification testing was due L to " excess lubricant." On December 19, 1986, AVC issued a 10 CFR Part 21 notification (Ref. 81). The notification indicated that Commonwealth Edison had also purchased AVC valves lubricated with E.F. Houghton SAFE 620. Commonwealth Edison told AE00 staff ** that the AVC valves which contained E. F. Houghton 620 lubricant were replace-ments for older model AVC SOVs which had been discontinued. Before being noti-fied by AVC of the problem with E. F. Houghton SAFE 620 and before installing the valves, Commonwealth Edison replaced the SAFE 620 with Dow Corning Molykote 55M. The licensee had recognized that Parker Super-0-lube was the lubricant that had been used in earlier equipment qualification testing, and SAFE 620 was probably s not an acceptabie replacement. Justification for the use of Holykote 55M instead of Super-0-lube was based upon the licensee's engineering analysis that indicated tht similarities between Molykote 55M and Super-0-lube. In retrospect, a detailed examination of these i two lubricants reveals they may have very different high-temperature behavior j and, under similar operating conditions, the Holykote SSM m ld be more suscepti-ble to dryout.*** Because of these differences, it is not clear that Molyhte 55M is an acceptable " qualified" replacement for the Super-0-Lube. With regard to problems of excessive lubricant and the application of " thin l films" of lubricant, it is interesting to note that "ommonwealth Edison plant had sticking problems with a similar AVC SOV several years earlier. In that case, L' the sticking was attributed to not having enough lubricant applied to the AVC ' valve. 5.2.4.4 Grand Gulf 1, LaSalle 1, and River Bend MSIVs - Sticking SOVs - Foreign l Unidentified Sticky Substance (FUSS) - Lubricant Suspected l Between February 1985 and December 1989, the Grand Gulf 1, LaSalle 1 and River Bend nuclear power plants experienced sticking of ASCO dual-coil 8323 SOVs in the MSIV air packs (Refs. 8, 82 to 88). The SOV malfunctions were attributed to a sticky substance at the contact point of the plug nut / core assembly inter-face (see Figure 1). The 50V malfunctions impaired or prevented the MSIVs from closing within the times specified in the plant safety analyses.

• Telephone discussions between T. Hutchins, Automatic Valve Corporation (AVC) and USNRC (S. Israel - October 14, 1988 and H. L. Ornstein - April 12,1989).
  • Telephone discussion between M. Sievert, Commonwealth Edison Company, and l

H. L. Ornstein USNRC, April 12, 1989. l~

• • • Super-0-lube consists of high molecular weight silicones whereas Molykote SSM is a lighter weight methyl silicone oil thickened with lithium soap having a lower dropping point than Super-0-lube (where dropping point is an indication of the temperature limit at which the lubricant dries out).

PRELIMINARY CASE STUDY 36

In the case of LaSalle, it was demonstrated that the cohesive / adhesive force caused by the foreign sticky substance between the plug nut and the core assembly of an ASCO dual-coil NP8323 SOV was significant and could have been the cause of its failure. After the core assembly was held vertically, the plug nut was pressed against the core assembly, and then the plug nut let go, the adhesive forces from the foreign substance between the two surfaces were able to support the weight of the plug nut to prevent it from falling.* Because the licensee suspected the Dow Corning 550 lubricant (applied to the SOVs internals at the factory) to be the cause of the sticking, the licensee l considered removing the factory installed lubricant from the 8 new MP8323 SOVs that were installed after the December 16, 1987 failure. In consideration of ASCO's concern that, without the internal lubricant, ac powered SOVs could suffer fretting damage, the licensee installed the 8 new NP8323-Viton SOVs as they were received from the manufacturer (without removing the lubricant). Those 8 replacement SOVs have operated successfully through 1989.** Subsequent to the September 30, 1988 failures of two ASCO dual-coil NP8323 SOVs at River Bend, the licensee replaced all 8 dual-coil NP8323 SOVs with new ones. However, prior to installing the new SOVs, the licensee removed the fac-tory coated lubricant (Dow-Corning 550) from their internal metallic parts. On December 1, 1989 two of those replacement SOVs failed due to sticking. The licensee attributed the sticking to FUSS which was believed (but not confirmed by laboratory analysis) to be Dow Corning 550 lubricant. In following up the December 1, 1989 failures, the licensee reviewed the procedures which were used in September 1988 to remove the factory applied lub-ricant. The licensee's review of those procedures indicated tnat although the Dow Corning 550 lubricant was removed from the internal metallic parts of the SOVs, the cleaning and reassembly procedures included a step in which the elastomeric parts of the SOVs were relubricated with the same Dow Corning 550 lubricant. Because there was more FUSS on the cleaned SOVs that failed in December 1989 than on the factory assembled SOVs that had failed September 1988, the licensee believed that the root cause of the December 1989 failures was the licensee's reapplication of excessive lubricant during the SOV cleaning and reassembly process. Subsequent to the December 1, 1989 failures the licensee's corrective action was to replace all eight NP8323 dual-coil SOVs with new ones -- after removing all the factory applied lubricant from them, without relubricating the elastomeric parts. Table 3 summarizes events where MSIV air pack SOVs have stuck at Grand Gulf, LaSalle, and River Bend.

• According to ASCO, the plug nut weighs about ene ounce while the spring force is about two pounds.

ASCO indicated that after a similar NP8323 S0V failure at WNP 2, the licensee had performed a similar demonstration. The sticky sub-stance at WNP2 was believed to be from excess lubricant (Dow Corning 550) that had been applied by the licensee when the SOVs were rebuilt.

• • Telephone discussion between R. Lanksbury (USNRC Sr. Resident Inspector at LaSalle Station) and H. L. Ornstein (USNRC), December 22, 1989.

PRELIMINARY CASE STUDY 37

m mmm ~

g Table 3 MSIV Air Pack 50V Failures (ASCO Dual-Coil 8323)

.5 Other SOVs Having g . Description Number of Foreign 'Jnidenti-4 Plant / Event of 50V and Stuck SOVs. fied Sticky Date Corrective Action and Location Substance (FUSS) Cossents n 3; m Grand Gulf 1 ASCO HTX8323* (Viton). 2 outboard All others (5) In subsequent testing at ASCO-2/10/85 Replaced all 8 S0Vs with lines (A and only 1 of 4 additional valves g ASCO NP8323 (having EPDM C) 1 inboard malfunctioned (leakage). parts). See Section (D line) However the failure of the 5.2.1.1 for a discussion outboard C-line 50V was of the subsequent fail-attributed to FUSS at the ures of the replacement plug nut / core assembly valves ccused by thermal interface. aging from self-heating (August 1989) LaSalle 1 ASCO NP8323 (Viton). 1 outboard All others (7) 3 of the valves that did not 12/16/87 Replaced all 8 SOVs (C-line) fail in the plant, failed- .m with like. during subsequent testing at ASCO, due to presence of FUSS at the plug nut / core assembly interface. River Bend ASCO NP8323 (EPDM). Re-2 inboard One unfailed Not all SOVs have been 9/30/88 placed all 8 SOVs with lines (B and inboard 50V inspected. Some are being like - attempted to re-C) (1 in-inspected was held for archival purposes. nove the factcry coated spected, FUSS found to have lubricant (Dow Corning. found)' FUSS. 550) from SOVs, but ap-Two outboard Two outboard SOVS were in-plied excessive amount S0Vs inspected spected at ASCO. The coil of lubricant to 0 rings found to have enclosucas of both SOVs had while reassembling FUSS.** had evidence of moisture caused 2 subsequent intrusion, indicative of failures (December 1989) localized steam heating.**

• ASCO HTX 8323 is not a nuclear qualified 50V, it is a non qualified commercial valve similar to the NP8323.
  • Telephone discussion between J. Shank, ASCO, and H.- L. Ornstein, USNRC,.May 8,1989.

,..,,.m..e

r x a i

E Table 3 MSIV Air Pack SOV Failures (ASCO Dual-Coil 8323) (continued)

C Other SOVs Having 5 Number of' Foreign Unidenti-y Description 4 Plant / Event of 50V and Stuck SOVs fied Sticky Date Corrective Action and Location Substance (FUSS) Comments nD'"- River Bend ASCO NP8323 (EPDM) 2 outboard 1 other 50V was Licensee believes FUSS from. 12/1/89 Replaced all NP8323's lines (A and incoected (in-was from excessive application 5 with new ones--but 0), FUSS found boat d), Jt also of Dow Corning 550 which was . removed factory or both had FUSS, but used by the licensee when installed lubricant less than what lubricating the 0-rings sub-from all internal was found en the sequent to removing the Dow parts of the 50Vs. failed outocards Corning 550 from the 50Vs' internal metallic parts subsequent to the 9/30/88 failures.*

• Telephone discussion between V. Bacanskas, River Bend, and H. L. Ornstein, USNRC, December 12, 1989.

.a e - - - ~ - - " ' ' - - - - - '-Y

The inspection of the SOVs on the inboard and outboard MSIV air packs at all three plants indicated that in almost every case the SOVs, which had not failed, were degraded in a manner similar to the failed SOVs, but to a lesser i degree. In each case, the licensee recognized the common-mode failure potential for compromising fast closure of inboard and outboard MSIVs on ene or more steamlines and replaced all the 8323 SOVs on the inboard and outboard MSIV air packs. The va'1ve manufacturer and several laboratories conducted extensive inspec-tions and tests on the 8323 SOVs which had been replaced. There are no simple explanations for these failures individually or as a group. The source (s) of l the sticky substance (s) which resulted in multiple SOV failures is uncertain. There is major disagreement between the utilities, the SOV manufacturer, the reactor manufacturer and the laboratories regarding the root causes of the failures.- Internal SOV lubrication (by the manufacturer and in one case by the licensee), and poor air quality are primary suspects.* 5.3 Surveillance Testing 5.3.1 Control Rod Timing Tests - Failed Scram Pilot SOVs - Perry On July 22, 1989, during scram time testing, plant personnel observed two control rods failed to meet their scram time testing requirements on initial attempts -- however, when retested the rods operated satisfactorily. As a i result, both control rods and their SOVs were der,lared to be operable. Subse-quently, on November 25, 1989, one of those rods failed its timing test twice but was retested satisfactorily twice. As a result, it was declared operable, i When the.second control rod that had also failed twice on July 22, 1989, was i retested on November 25, 1989, and failed, it was declared inoperable. At that l . time, the licensee conducted an investigation to determine the root cause of the test failures (Refs. 89,90,91). The licensee's root cause analysis found that a manufacturing error had been made at ASCO (failure to upgrade polyurethane seats of the scram pilot SOVs with viton), and that the Perry Plant had not responded to a product recall notice that ASCO had sent them (Ref. 91), i It is significant that the licensee's surveillance testing program did not provide guidance to the plant staff regarding actions to be taken when unsatisfactory-test results are encountered. 5.4 Use of Non-Qualified SOVs s 5.4.1 Colt /Fairbanks-Morse EDGs: Repetitive Air Start Valve Failures t One plant, having Colt /Fairbanks-Morse EDGs, experienced six air start SOV failures during an.8 year period. There were five failures of one valve and one failure of an identical, redundant SOV. The SOVs were commercial grade valves, J model X833-134, made by ASCO. The failures occurred from February 1, 1980, i

• Failures of ASCO NP8314 SOVs which are geometrically similar to the 8323 SOVs have been traced to an assembly error during manufacture.

Conceivably, a similar error may have been introduced during the assembly of the 8323 SOVs (see Section 5.2.4.1). PRELIMINARY CASE STUDY 40

3. through March 28, 1988', and in each case the failures involved excessive air I L

leakage 1

Four of the five failures of the same valve (DA-198) were attributed to .the S0Y core and spring assembly. The first failure was attributed to wear of the core and spring assembly caused by excessive heat from the solenoid being constantly energized. The SOV was rebuilt (core and spring were replaced). The SOV's-second failure was attributed to " wear of the core and spring assem-bly." The 50V was-rebuilt again (core and spring assembly were replaced). The third malfunction of the same SOV occurred while attempting to start the diesel. The failure was attributed to " misalignment of solenoid header due to previous repairs." The licensee's corrective action was to realign the solenoid header. t Three months later the same SOV was again found to be leaking air. This fourth failure was attributed to " wear of the core and spring assembly." The SOV was-rebuilt again (core and spring assembly were replaced). Five months later a redundant air start SOV (DA-23B) on the same diesel was found to be leaking air. It was rebuilt (spring and core were replaced). On March 28, 1988 the same SOV that had failed four times before (DA-198) failed again. The fifth g failure was attributed to a worn seat that resulted in air leakage. The. valve l was replaced rather that being rebuilt. We are unaware of any subsequent failure of this replaced SOV. Discussions with the licensee who's EDGs experienced these six failures, and other licensees with Colt /Fairbanks-Morse EDGs indicated that they have . received little, if any guidance from the EDG supplier about preventive mainte-nance or replacement of the air start system-SOVs. The SOVs that are used for the Colt /Fairbanks-Morse EDGs are commercial grade ASCOs which are supplied with limited maintenance or service life information. I I7 1 PRELIMINARY CASE STUDY 41 '1 L .a...

6. ANALYSIS AND EVALUATION ~0F OPERATIONAL EXPERIENCE ] 6.1 Common-Mode Failures 1 1 Examination of the events discussed in Chapter 5 and many of the SOV failures included in Appendix A clearly indicate a potential exists for common-mode SOV failures that could compromise multiple trains of diverse safety systems. Such common-mode failures are not considered in plant safety analyses. It is not practical to perform safety analyses for all combinations of common-mode SOV failures. However, it is-feasible to take actions to minimize ~the likelihood for encountering common-mode SOV failures that could affect safety sys, ems. Chapter 9 presents recommendations that can be effectively used to minimize the potential for common-mode 50V failures affecting safety systems. The root causes of many common-mode SOV failures that have been observed thus far are given below. (1) Design / Application Deficiencies Incorrect specification of operating parameters such as MOPD (e.g., Section 5.1.3.1) and valve orientation (e.g., Section 5.1.4.1); incorrect material selection such as incompatibility between SOV L internal parts and fluids in contact with the 50V (e.g., Section 5.2.3.3); incorrect specification of ambient (non-accident) conditions (i.e., temperatures, radiation,andmoisture)(e.g., Sections.5.1.1.2, 5.1.1.3); i L incorrect assessment of the life shortening effects of coil heating E (e.g., Sections 5.1.2.1,5.1.2.2) (2) Inadequate-Maintenance Failure to replace or rebuild limitad life piece parts of the SOVs (e.g., gaskets, seals, diaphragms, springs, and coils) on a timely-basis (e.g., Sections 5.2.1.1 5.2.1.2); i failure to rebuild SOVs correctly (e.g., Section 5.2.2.1); l. failure to maintain clean, dry instrument air. Contaminants have caused long-term common-mode 50V degradation and failure (e.g., Sections 5.2.3.1,5.2.3.2); excessive lubrication of SOV internals have contributed to S0V failures (e.g., Section 5.2.4.3). (3) Installation Errors Incorrect orientation (backwards, upside-down) installation at an angle not in accordance with SOV qualification testing (e.g., Section 5.1.4.1, Appendix A); 1 PRELIMINARY CASE STUDY -42 l h H

incorrect' electric' current'(dc vs.-ac) (e.g., Appendix A); inadequateterminalorJunctionboxconnectionsasaresultofinade-quate manufacturer's guidance or architect engineer's interpretation of manufacturer's guidance (e.g., Appendix A). (4) Manufacturina Defects Lubrication errors (e.g., Section 5.2.4.1); defective materials - body, plug, springs, elastomers (e.g., Ref. 74); tolerance / assembly errors such as incorrect' spring size or stiffness (e.g., Ref. 74, Appendix A); faulty wiring / coil defects (e.g., Appendix A). 6.2 SOV Failure Rates It is difficult to accurately quantify S0V failure rates due to the following reasons: (1) Not all S0V failures are documented. In many cases SOVs are viewed as-expendable items, and their failures are simply viewed as end of life, and-replacements are installed without any failure reports. i (2) Many-SOV failures not associated with reactor trips or complete train fail-ures of safety systems are not reported in the LER data base. (3). Many: S0Vs that are subcomponents or piece parts of other larger components or systems are not alway (s reported as SOV failures in the nuclear plant reliability data system NPRDS) for example, MSIVs, flow regulators, gov-ernors that fail to function properly because the related'S0Vs have failed are unlikely to be reported as SOV failures. Hence, an accurate estimate of SOV failure rates from NPROS is not achievable. Coupling the difficulties of obtaining accurate S0V failure counts with.the difficulty of accurately assessing the number of successful SOV challenges or surveillance. tests can,- at best, lead to a crude estimate of 50V failure rates. Nonetheless, Table 4 lists SOV. failure rates from several sources, including the results of this study's query of the NPRDS data for failures which occurred.-from -January 1, 1985 through December 31, 1988. It is significant that assuming quarterly testing of S0Vs, NPRDS data, for the years 1985 through 1988, indicate failure' rates of 7 to 9.5 times higher than the estimates used in WASH 1400 and in the NUREG 1150 methodology. Exem-plary of item (3) above, the NPRDS failure records used for estimating S0V L failure rates generally do not include the unrecognized S0Vs. It should be noted that publicly available SOV failure rate data does not distinguish between'SOV size, energization mode, valve opening status, manufac-turer, model, or type. In view of the wide range of SOV variations, the avail- -able failure data does not allow for accurately predicting individual S0V performance or failure rates. PRELIMINARY CASE STUDY 43

v 4 Table 4 Estimates of SOV Failures to Operate Estimated Source; failure rate-WASH 11400 1x10 3/ demand This study 7 to 9.5x10 8/ demand (NPRDS data Jan 85-Dec. 88) Assuming quarterly testing NUREG 1150 methodology NUREG/CR 4550 Vol. 1 1.0x10 8/ demand (Seabrook PRA). 2.4x10 8/ demand NUREG/CR 4550 Vol. 6 1.6x10 3/ demand (Grand Gulf PRA) NUREG/CR 4819, Vol. 1 7x10 s/hr (NPRDS data Sept 78-July 84) ,This study 6.4 to 8.7x10 8/hr (NPRDS data Jan 85-Dec. 88) In view of the' aforementioned problems of estimating single SOV failure' rates, we find the. task of estimating the risk resulting ' rom common-mode 150V failure to be a difficult task the results of which may have significant uncertainty, Suchanundertaklngisbeyondthescopeofthepresentstudy. .We know of no PRA which accounts for the contribution of common-mode fail-ures of SOVs. Omission of such cross system / cross train failures lead towards nonconservative results. 6.3. Maintenance Problems 6.3.1. Maintenance Problems - S0V Marufacturers' Contributions - Review of operating experience indicates that a substantive number of SOV failures are attributed to inadequate maintenance or. refurbishment. As evidenced 'by several of.the events discussed in Chapter 5, it is clear that utilities are ' not fully informed of SOV maintenance requirements. Theneglectoroversightof SOV maintenance oftentimes'comes from the SOV manufacturers failure to provide ~ SOV maintenance requirements to the 50V users or second-level manufacturer - such as EDG manufacturers;(ALCO, Colt /Fairbanks-Morse, General Motors, Del val, Cooper-Bessemer), valve manufacturers (Xomox),1contro11er manufacturers (Fisher, -Masonellan),etc. ~ Some SOV manufacturers are more prescriptive than others. .Some manufacturers provide no~ guidance on preventive maintenance. One manufac-turer (Valcor) varies its recommendations depending on whether the purchaser bought the " full documentation package " PRELIMINARY CASE STUDY 44 l.

I p ^; I Examples of the variation among SOV manufacturers' maintenance recommenda- 'tions are discussed below.. i ASCO does not provide any specific recommendations for SOV maintenance or refur-BTsiiment.. This is even true for their nuclear qualified 1E valves. Quoting ASCO's installation and maintenance bulletin-for.NP8323 SOVs (Ref. 92). " Preventive Maintenance a 1. Keep the medium flowing through the valve as free from dirt and foreign material as possible. Use instrument quality air, oil-free for Suffix "E." 2. While in service, operate valve' periodically to insure i e proper opening and closing. 3. Periodic inspection (depending upon medium and service conditions) of internal valve parts for damage or exces-sive wear is recommended. Thoroughly clean all parts. Replace any parts that are worn or damaged. 4. The valves may require periodic replacement of the coils and all resilient parts during their installed life to maintain qualification. The exact replacement will depend on ambient and service conditions. period Spare parts kitt, and coils-are ordered separately (see Ordering .Information). Consult ASCO for specific recommendations in connection with the replacement of parts." Valcor provides specific recommendations for maintenance or refurbishment of its N-stamped SOVs. However, it-is possible to purchase the same valve without an N stamp. If-it is purchased without an N stamp, it can also be purchased .without any documentation. Such a "no-doc"1 valve would not be provided with any preventive maintenance or refurbishment recommendations. Target Rock - All of Terget Rock Corporation's SOVs appear to be supplied with specific preventive maintenance and refurbishment recommendations. Circle Seal, Ross and an Unspecified Foreign Manufacturer - Circle Seal and Ross make-50Vs which are used in several different EDG air start systems. Those valves are not supplied with any preventive maintenance or refurbishment . recommendations. Lack of specific maintenance recommendations has contributed to multiple failures of a. foreign manufacturer's SOVs used in the EDG air start system of a foreign plant (see Section 6.3.2.1). - Skinner Electric - SOVs manufactured by Skinner Electric Company which are used .in Woodward Governors on BWR HPCI turbines are not provided with any preventive maintenance or refurbishment recommendations. Sperry-Vickers - $0V's sanufactured by Sperry-Vickers which are used in the hydraulic contro11 era used for BWR recirculation pumps and main turbine-trip-systems are not 'prwided with preventive maintenance or refurbishment recommendations. [ PRELIMINARY CASE STUDV 45 m a

6.3.2 Maintenance Problems - Contribution of the Unrecognized SOVs In many cases plant maintenance and operations personnel are unaware of the presence of, or maintenance requirements of SOVs. This situation is common be-cause there are many cases in which S0Vs represent only a small portion of a larger system or component, and the information available to plant staff does not identify the care required by the SOV which is unrecognized" within the "overall system". Examples that we have observed are: j Emergency diesel generators: air start systems, governors, and cooling water control systems. Auxiliary feedwater and main feedwater systems: flow control regulators. BWR high pressure coriing injection systems: remote shutoff-l controls, governors. Instrument air dryers: desiccant column regeneration and cycling control systems. 6.3.2.1 Unrecognized SOVs in Emergency Diesel Generators The operation and maintenance manuals for the plants diesel engines, and operator and maintenance personnel training are heavily weighted by the engine manufacturer's literature which, at best, includes minimal information regarding the SOVs used in the EDG's auxiliary systems. Specific examples observed included: A foreign reactor site where the air start SOVs were not on any preventive maintenance program. Failure of one 50V due to aging of a Buna-N diaphram was undetected until its redundant backup failed from the same cause. As a result, the station added refurbishment or changeout of such resilient parts to all its EDG air start systems. Similar failures have been observed at numerous U.S. plants, e.g., Three Mile Island la (Refs. 93, 94), Ginna (Refs. 95,96,97), Duane Arnold (Ref. 98). During a-trip to the Duane Arnold plant in reviewing SOV experience, AE00 staff learned that subsequent to the July 1982 failure (Ref. 98), the Duane Arnold staff recognized the SOV's limited lifetime and the need for SOV refurbishment or replacement. As a result the Duane Arnold staff added SOV i changeout to their preventive maintenance program. However,.several years:later, plant maintenance personnel made a decision to eliminate changeout of that S0V from their preventive maintenance program. -The rationale for dropping such pre-ventive maintenance was that the SOV was cycled only 7 seconds a month, and such limited use did not seem to require maintenance. The basis for implementing the -SOV's preventive maintenance and the previous failure, which resulted from age related degradation, appeared to have been forgotten. Subsequently, the licensee stated _that preventive maintenance on the aforementioned SOVs would be reinstated. As a student in a recent TVA EDG training course applicable to seven plants, (Browns Ferry 1, 2, 3, Sequoyah 1, 2, Watts Bar 1, 2) the case study author learned that maintenance literature for the General Motors Electro-Motive Division (GH-EMD) diesel engine supplied by Morris-Knudsen, does not provide the licensee with any instructions for refurbishment or changeout of the SOVs in the EDGs' air start and governor control systems.

• Telephone discussion, M. Schaefer, General Public Utilities, and H. L. Ornstein, USNRC, February 16, 1989.

PRELIMINARY CASE STUDY 46 l l

i 4 6.3.2.2-LUnrecognized SOVs in Auxiliary.and Main Feedwater Systems o As noted in Section 5.2.3.2,'a review of failure data at North Anna Units j I and 2 showed that poor quality. air was the root of the SOV/ control valve fail-y ures. : As a result,.the licensee initiated a program for repairing and replacing the SOVs and control valves, as well as upgrading the air system quality and ' enhancing plant personnel training and maintenance practices. 6.3.2.3 Unrecognized SOVs in BWR High Pressure Coolant Injection Systems In. Reference 99 the Duane Arnold plant's licensee reported the failure of 'l the remote shutoff control system which is part of the HPCI turbine's governor system. Discussion with plant personnel and the turbine manufacturer indicated a lack of communication between them regarding the potential for undetected fail-ures of the S0Vs. 'The licensee's report noted that the failure was caused by aging of the elastomeric parts of the S0V. Such an undetected failure could 3 result in failure to start the HPCI system. Apparently information provided by the turbine manufacturer (Dresser-Rand, formerly Terry Turbine) did not pro-vide adequate maintenance information about the S0V that is supplied as an internal part to the Woodward Governor (the SOV was manufactured by Skinner Electric Co.). The Skinner Electric maintenance instructions do not address preventive maintenance or service life requirements for the SOV. The Woodward Governor service manual does not address 50V' preventive maintenance, or service . life. The service-information letters (SILs) provided by the NSSS vendor (GE) did address'other aspects of HPCI turbine service, performance and maintenance, but discussion with plant personnel and GE personnel indicated that maintenance, refurbishment or replacement of the SOVs are not addressed in any of GE's SILS. 6.3.2.4 Unrecognized SOVs in Instrument Air Driers Review of a leading instrument air *.er manufacturer's operation and maintenance manual indicated-minimal gu9 ace with regard to SOV maintenance. The SOVs are required to cycle every fir minutes to ensure that the air flows through the correct desiccant stack to assure proper air drying and acceptable outlet ' dew points. Failure of the S0Vs-could result in undetected high instru-ment air: moisture content which could lead to-degradation-and_ malfunction of equipment utilizing instrument air, including other S0Vs that perform safety-related functions. 6 '. 3. 3 Maintenance Problems - Contributions of Utility Programs and Practices Review of SOV failure reports and follow up discussions with plant personnel, NRC inspectors, and S0V manufacturers showed that shortcomings in many utilities' SOV maintenance programs and practices were a major source of S0V failures. For example: (1) Reference 100 indicated that Brunswick plant staf f stated that ASCO Class 'lE SOVs with 30 year qualified lives did not require any preventive mainte-nance for 30 years. The' licensee did not recognize the fact that the resilient, or elastomeric parts of the SOVs require more frequent replacement. PRELIh1 NARY CASE STUDY 47

3 _.. V '(2) -After finding.that'S0Vs would not shift their position on demand during surveillance testing, it was common practice for Brunswick and North Anna'- Stations'. plant personnel to tap the SOVs (" mechanical agitation").. If a SOV would change position when tested after the mechanical agitation, no further maintenance would be performed,'and the SOV would be declared operable (Refs.- 100,101). (3)' ASCO's valve engineering department product engineering manager visited the Susquehanna plant'to assist the utility in finding the root cause of the failure-of-a rebuilt ASCO S0V which failed after being returned to service. The ASCO manager's discussions with plant personnel revealed that subsequent to rebuilding the SOV, plant personnel bench tested the SOV with poor qual-ity service air instead of clean, dry instrument air. Inspection of the SOV revealed that oil from the service air system had caused the SOV's second-failure.* (4) The Calvert Cliffs 1 and'2 plants' S0V maintenance is tracked by the sta-- tion's' reliability centered maintenance (RCM) program. The RCM program has found that instrument air dryer SOVs have a mean time between failure of 10 months,*but the plants' maintenance program replaces such SOVs on an annual basis.

• The failure of the instrument air dryer SOVs can cause serious instrument air system degradation leading.to common-mode failures of many cther SOVs,-including those that perform safety-related functions.

6.3.4 -Rebuilding vs. Replacement l p Review of SOV failure data indicates that inadequate rebuilding of S0Vs has been a significant cause of SOV failures. There is a broad' range of com-plexity associated with rebuilding SOVs, depending upon individual SOV manufac-turer and model number. To further complicate the issue, there are variations p among SOV manufacturers with regard to providini test apparatus to check the soundness of rebuilt SOVs; for example,- Target l ock Corporation has marketed a test fixture for licensees to test their rebuilt S0Vs. Although some manufacturers provide values of acceptable coil voltages, H leakage rates, etc., to enable users to check the conditions'of their SOVs, some L other manufacturers:do not make such information available. Serious questions l' -arise about the acceptability of new SOVs-if acceptance criteria are not l .available. 'In Reference 102,-ASCO notified licensees that it has discontinued selling rebuild kits for its nuclear power plant S0Vs (NP series). However, ASCO is

~

continuing to sell rebuild kits for commercial SOVs and SOVs used in BWR scram = systems (purchased through GE). lr As noted in. Chapter 5, there have been several events in which common-mode failures resulted from incorrect rebuilding of SOVs. The potential for common-h mode SOV failure resulting from rebuilding errors may be minimized by staggering

• Telephone discussion, J. Shank, ASCO, and H. L. Ornstein, USNRC, May 11, 1989.
  • Telephone discussion, J. Osborne, Baltimore Gas and Electric Co., and H. L.

Ornstein, USNRC, April 21, 1989. PRELIMINARY CASE STUDY 48

_~ -..e the rebuilding (if possible), or by limiting the amount of SOV rebuilding done by any one individual-(see Sections 5.2.2.2, 5.2.2.3). -In addition to focussing attention on the useful life of SOVt being governed by the elastomeric parts,' attention should be focused on the shelf life and on the actual manufacturing date of the elastomeric parts in the rebuild kits. For example, because of _ elastomeric (Buna-N) degradation observed in SOVs used in BWR - scram systems, GE recommended (Ref. 56) that BWR scram system SOVs having Buna-N parts be rebuilt periodically. The frequency of rebuilding should be governed by the "useful life"_of the elastomer ("useful life" being defined as the sum - of shelf life and in-service life). Controlled by the Buna-N parts, GE recom-- mended a "useful life" of seven years for scram system SOVs. The seven years being from the time of kit manufacture (not from the time of. rebuild). 1 i 1 PRELIMINARY CASE STUDY 49

1 ' 7. 0 FINDINGS The root causes of most SOV problems are traceable to the lack of understanding of the capabilities and requirements of SOVs. Oftentimei plant operations and maintenance programs do not address the short lifetimes of the resilient elastomeric piece parts of the SOVs (gaskets, seals, diaphragus, etc.). Maintenance programs also fail to address the low tolerance SOVs have for oper-ating under adverse conditions that are significantly different than those of the controlled laboratory environment under which they were originally tested. In many cases, the manufacturers have not provided the end users with a full understanding of.the sensitive nature of certain parts of the SOVs. Many users have learned after using certain-S0Vs that they are unforgiving and finicky with regard to contaminants and local environmental conditions. Deficiencies in. selection, operation, and maintenance of SOVs have resulted in hundreds.of 50V, failures, many of which were common-mode failures that cut across multiple trains of safety systems. Our major findings regarding the root causes of common-mode SOV failures are described below. 7.1 Desian' Application Errors 1 7.1.1 Ambient Temperatures Many common-mode SOV failures have resulted from subjecting SOVs to ambient temperatures in excess of their original design envelope. Such common-mode failures have resulted from localized steam leaks (see Section 5.1.1.1), incorrect estimates of ambient temperatures (see Sections 5.1.1.2,5.1.1.3),and failure to account'for ventilation system malfunctions (Ref. 103). Because the useful qualified lives of the short lived parts of SOVs are. halved by every 18'F temperature rise-(Arrhenius theory.- Refs. 104,105), seemingly minor increases in ambient temperatures above those considered in the SOV design cannot be allowed to p'revail for extended time periods without running the risk of sustaining seemingly" pr: mature failures. 7.1.2 'Heatup from Energization Many common-mode SOV failures;have occurred because the estimated service lives did not properly includo the life-shortening effects of heatup due to - continuous coil energization i.see sections 5.1.2.1, 5.1.2.2). Many licensees have been unaware of this sitzcion. For example, by incorrectly using the certificates of compliance provided with ASCO's NP-1 nuclear qualified valves, licensees have overpredicted the service life of continuously energized SOVs. l: Use of appropriate S0V heatup data in conjunction with Arrhenius theory (Refs. 104,105).has been found to be an acceptable (but not a 100 percent accurate) method for predicting S0V life. 7.1.3 Maximum Operating Pressure Differential 4 Many licensees have found misapplications in which SOVs could be or were subjected to operating pressure differentials that could or did prevent them L PRELIMINARY CASE STUDY 50 t i n- -u-- w

L w from operat'ng. Although NRC issued Information Notice 88-24 (Ref. 23) about L this probl9m, as noted in Section 5.1.3.1, it is not clear that all the licensees have addressed the issue, of over pressure which could result from pressure -regulator failures. l. ~7.1.4 Unrecognized SOVs Used as Piece-Parts t, j' Many SOVs used in safety-related equipment are not given prominent attention because they are used as piece parts of larger equipment. Specific preventa-tive maintenance requirements are not readily available for them. Many SOV failures have occurred as a result of the lack of maintenance or replacement of such unrecognized SOVs (see Section 6.3.2). 7.1.5 -Directional SOVs Six plants have reported experiencing undesirable spurious openings of safety-related S0Vs due to high backpressure. The licensees did not recognize .or were not aware of the directional requirements of the valves (see Section 5.1.4.1). In addition to reports of SOV malfunctions which occurred because they were installed backwards, there are also reports of S0Vs which were installed upside-down, or at improper angles'(see Appendix A). 7.2 Maintenance Operating experience has confirmed that-S0V maintenance deficiencies can incapacitate multiple safety systems. The pervasiveness of maintenance defi-ciencies highlight the need for implementing aggressive S0V mainterance pro-grams.to prevent widespread common-mode failures. Specific maintenance problem areas are discussed below. 7.2.1-Maintenance Frequency Lack of timely preventive maintenance (complete 50V replacement or rebuilding of short-lived piece parts of SOVs) has resulted in many SOV failures (see Sec-tions 5.1.2.1, 5.2.1.2, 6.3.2.1). Many SOV manufacturers have failed to provide the users with definitive information on the useful lifetime of the SOVs inter-nal' diaphragms, gaskets, 0-rings, coils, etc. Some manufacturers indicate that ' periodically changing the elastomeric parts is necessary, without specifying the frequency of changes. Other manufacturers do not even mention that any changing is necessary. Similarly, there are wide variations among manufacturers with regard to specifying (or not specifying) the allowable shelf lives of their -SOVs and SOV rebuild kits (see Sections 6.3.1,6.3.3,6.3.4). Because of the limited lives of their elastomeric or resilient parts, SOVs

mendations. placed or refurbished in accordance with the manufacturers' recom-should be re In the absence of specific manufacturers' recommendations, and in absence of applicable failure data, changeout of short-lived elastomeric and resilient materials-(or complete valve replacement) should be done on the basis of material shelf life, manufacture date and installation date.

However, change-out of elastomeric parts or complete 50V replacement should be done more fre-quently if operating condities exceed the originally envisioned design conditions or if field failure operience a;:tates. PRELIMINARY CASE STUDY 51

~ a- -7.2.2 Replacement Versus Rebuilding Rebuilding or refurbishing certain models of several manufacturers' SOVs is a difficult task that can be made even more difficult if it is done in place, requiring the workers to wear decontamination or protective clothing.

However, removal and reinsta11ation of N-stamped valves which are welded into the primary system are not simple, inexpensive tasks either.

Incorrect rebuilding or refurbishing of SOVs have caused many premature failures (e.g., see sections 5.2.2.1,5.2.2.2). Contributing to the difficulty of rebuilding or refurbishing SOVs correctly is the_ fact that many manufacturers do not provide the licensees with adequate SOV documentation or testing apparatus j to verify the effectiveness of-the rebuilt or refurbished SOV. As a result, l post rebuild testing at many facilities merely involves cycling verification rather than performing appropriate tests normally performed by the manufacturer during initial SOV manufacture (see Section 6.3.4). Discussions with plant personnel fiave revealed that many licensees, (e.g., Perry, River Bend, Salem, Grand Gulf, Duane Arnold) have chosen to discontinue rebuilding certain SOVs because improper rebuilding can result /has resulted in i many SOV failures and costly down-times. In general, licensees have reacted -favorably-to ASCO's recent decision to discontinue supplying rebuild kits for their NP-1 nuclear qualified SOVs (Ref. 106,107). ASCO's decision to discon-i tinue supplying SOV rebuild kits was based upon field experience which indicated that many ASCO SOV failures were caused by inadequate reb'Jilding techniques. 7.2.3 Contamination Many common mode 50V failures have been caused by connaminants in the fluids which flow throu h SOVs; instrument air in particular (see Sections 5.2.3.1, 5.2.3.2, 5.2.3.3. S0V contamination resulting from particulates, moisture, and hydrocarbons in the instrument air system have been a major source of common-mode SOV fail- .ures. In manyl plants contaminants were introduced during original ccostru-tion. Many contamination problems have resulted from poor design or maintena ve of the instrument air systems. 1 Many SOV failures are clearly attributed to subjexting the SOVs to conditions beyond their design regarding particulates, moisture, hydrocarbons, etc. Contributing to the problem is the fact that some manufacturers have specified the need for clean air or instrument quality air without quantifica-tion (e.g., maximum allowable particle sizes and dew points). g Although licensees are taking actions to improve the quality of their plants' air systems, there is concern for the residual effects of previous air system contamination (Section 5.2.3.2). Long-term SOV degradation such as deterioration of EPDM parts as a result of hydrocarbon intrusion, formation of varnish-like deposits from heatup of hydrg arbons, and residue formation from the interaction of moisture, silicofie lubricant,'and heat, are areas of concern. PRELIMINARY CASE STUDY 52

m i -7.2.4 Lubrication Improper lubrication has resulted.in many common-mode SOV failures. The improper lubrication has been attributed to manufacturing errors (see Section 5.2.4.1), as well as licensee errors. Errors include the wrong choice of lubricant (see Sections 5.2.4.2, 5 2.4.3), unauthorized use of imorrect lubri-cant (see Section 5.2.4.1), and use of excessive amounts of lubrichst (see Section 5.2.4.4). 7.3 Surveillance Testing Several cases (see Section 6.3.3) have been reported in which S0Vs failed to actuate on demand during survtillance tetting, however, subsequent tapping (" mechanically agitating") the-S00s would ena51e them to actuate. As a result, the SOVs were declared operable without addrescing the cause of the original i failures, thus leaving the SOVs in degraded states vulnerable to-future failures-upon demand. Similarly, as noted in Section 5.3.1, incorrect surveillance testig led aperators to operate a BWR witt multiple failed scram pilot S0Vs. 7.4 Verification of the Use of Qualified SOVs The issue of environmental qualification of Class 1E electrical equipment and SOVs has been addressed by utilities in response to Bulletins 79-01A and B. Nonetheless,- there are m uy instances in which S0Vs that were assumed (in plant safety analyses) to operate to mitigate design-basis events, have been procured as " commercial grade" SOVs of questionable quality and are not being maintained in a manner commensurate with their intended safety function. Examples have been found where commercially available, non qualified SOVs are being used in safety related applications without appropriate verification of product quality and design control. In many instances the SOVs lack verifi-cation that they can withstand the accident coaditions postulated in plant safety -analyses. A common problem appears to be categorization of the SOVs-for use in EDG air systems. In many cases the origint.1 equipment that contained SOVs as piece parts was certified or qualified to meet 1E requirements, whereas the individual replacement SOVs were not. (See Section 5.4.1). 7.5 Redundancy and Diversity The root causes of many common-mode failures of safety-related SOVs have' eluded many licensees' detailed failure analyses (see Section 5.2.4.4). In many such instances the search for the origins of foreign unidentified sticky substances (FUSS) have been inconclusive, and corrective actions were limited to cleaning or replacing the failed SOVs (e.g., Brunswick (Ref. 2), Franklin Institute (Ref. 76)). In some cases, the licensees discounted instrument air system contamination (oil, water, dirt) as the cause of the FUSS, but plant operatiag history indicated a prior history of air system contamination which could ha<e been a contributor to the problem. Similarly, the 50V manufacturing - process (see Section 5.2.4.1) and the licensee's rebuilding process (see Sections 5.2.2.1, 5.2.2.2, 5.2.2.3, Section 6.3.3) have been found to be the sources of contaminanus which caused common-mode S0V malfunctions. PRELIMINARY CASE STUDY 53

Staggering the maintenance, testing and replacement of redundant SOVs may represent a simple way of preventing common-mode failures of redundant SOVs. In addition, if the root causes of-persistent common-mode SOV failures cannot be.found, or cannot be-eliminated, the need for SOV diversity (with regard to-model, energization mode, failure mode, or manufacturer) becomes apparent. (See Appendix B for a discussion of an example of such a problem with the ASCO NP8323 i SOVs used for MSIV control at many BWRs.) 7.6 Feedback of Operating Experience-Based upon visits to several of the major SOV manufacturers' facilities (e.g., ASCO (June 1988), Target Rock (November 1988), Valcor (December 1988),- 4 AVC (February 1990)) discussions with other SOV manufacturers (e.g., Circle Seal, Skinner Electric), and extensive discussions with manufacturers who's equipment utilize SOVs.as piece parts (e.g., Fisher Controls, Dresser-Rand / Terry Turbine, Xomox Valves, California Controls (Calcon), Colt /Fairbanks-Morse), it was found that SOV manufacturers have not been fully apprised by the utilities of many SOV failures that have occurred at nuclear power plants. SOV manufacturers are not aware of many widespread-failures of safety - related equipment that may have-been caused by generic manufacturing or design deficiencies of the 50Vs. Conversely, when licensees purchase SOVs commercially, without 10 CFR 50 Appendix B, and 10 CFR Part 21 requirements, they are not fully apprised by the manufacturers of generic defects that are discovered subsequent to delivery. In one case, a major 50V manufacturer did not feed back generic SOV defect information to the end user due to the manufacturer's failure to understand or properly implement the 10 CFR Part 21 requirements that were applicable to its SOVs (Ref. 74) (also see Sections 5.1.2.2,5.2.4.3). i l l L L 1 PRELIMINARY CASE SfUDY 54 + .o

- x

3. '

8.0 CONCLUSION

S Operating experience has demonstrated that common-mode failures and ' degradations of S0Vs can compromise multiple trains of multiple safety systems. The fact that hundreds, and in many cases thousands, of SOVs-permeate all impor-i tant' systems at all U.S. LWRs highlights the necessity for eliminating common-mode SOV problems that jeopardize plant safety. 8.1. Safety Significance Considering the application of the " single failure criterion," the L application of " defense-in-depth," and the large population of SOVs used in safety-related systems at U.S. LWRs, it appears that the number of individual random SOV failures that have been reported do not appear to present a safety concern. However, examination of the root causes of many SOV. failures at many plants demonstrate error patterns in the design / applications, maintenance and - testing of SOVs _which have led to a multitude of widespread comon-mode failures. Operating experience shovs that SOVs are vulnerable to numerous common-mode failure mechanisms and their failures can adversely impact numerous safety sys-tems. Some of the safety systems that were observed to be adversely impacted by common-mode failures of SOVs were: EDG air start system, BWR scram system, BWR main steam isolation system, PWR auxiliary feedwater system, PWR safety injection system, component cooling water system, containment isolation system, residual heat removal system, containment cooling system. These safety systems are required to function in order to prevent and/or mitigate accidents and/or to protect the_public from release of radiation from design basis accidents. Therefore, we conclude that S0V problems represent a significant safety concern. Chapter 5 presents examples of over twenty recent events having the potential for comon-mode failures or degradations of over 600 SOVs in impor-tant plant-systems.* The comon-mode failures and degradations cut across mul-tiple trains of safety systems as well as multiple safety systems. The recur-rence of comon-mode S0V failures or degradations highlights the gravity of the situation. Although plant safety analyses do not address comon-mode, multiple train / multiple safety system failures, operating experience indicates that they have occurred and continue to occur. The comon-mode SOV failures and. degrada-tions that have occurred which compromised front line safety systems such as emergency ac power, auxiliary feedwater, high pressure coolant injection, and t scram systems clearly demonstrate the safety significance of SOV problems. , Chapter 6 presents estimates of S0V failure rates which were extracted from plant operating. data (NPRDS). The estimates indicate failure rates of almost one order of magnitude larger than those: assumed in the WASH 1400 study and in

the NUREG 1150 methodology for level one PRAs.

Coupling such nonconservative treatment of SOV failures with the fact that level one PRAs do not address SOV failures that cut across multiple systems leads us to conclude that the risk contribution from S0Vs may have been severely underestimated in previous risk assesssents.

• There have been many other similar events.

The events chosen here are intended to be illustrative. Surely they are not a complete set of all such events. . PRELIMINARY CASE STUDY 55 J 1 ~

. _ ~. - _ \\ 8.2' Need for Action On the basis of our analysis'of operating data, we conclude that the SOV . problems outlined in this study need to be addressed to ensure that the margins of safety for all U.S. LWRs remain at the levels perceived during original plant licensing. We note thst to date the NRC has issued 36 generic communications pertain-ing to SOV problems (See Appendix C). Those generic communications alerted licensees to specific SOV problems. Based on our study we believe that an inte-grated comprehensive program is needed now to address the root causes of SOV problems described in this report. We conclude that integrated implementation of the recommendations-provided in Chapter 9 will significantly redut:e-the likelihood for common-mode SOV failures eroding the margins of safety at all LWRs. I i l' L i PRELIMINARY CASE STUDY 56

9.01 RECOMENDATIONS In order to minimize the potential for common-mode failures, attention 'should be focused upon certain aspects of SOVs. We recommend that the actions discussed below be initiated in order to assure that the plants retain the margins of safety perceived in their original licenses. If SOVs are found to be inadequate, prompt corrective actions should be taken. 9.1 Design Verification. Licensees should review SOV design specifications and actual operating conditions to verify that all SOVs assumed to operate in FSAR safety analyses are operating within their design service life. 9.1.1 - Ambient Temperatures The reviews should assure that the lifeshortening effects of elevated ambient temperatures are considered in the determination of 50V service life.. t 9.1.2 Heatup From Energization The reviews should assure that the lifeshortening effects of heatup due to coil energization are appropriately accounted for in the determinations of SOV service life. 9.1.3 Maximum Operating Pressure Differential i The reviews should assure that the potential for overpressure due to pressure regulator failure or hydraulic fluid heatup due to postulated accident conditions have been considered in the selection of the SOVs. 911.4 Unrecognized SOVs Used as Piece-Parts In addition to verifying the adequacy of the high visibility SOVs as noted above, similar verification should be made for unrecognized S0Vs which are used as piece parts-of flow regulators, governors, emergency diesel generators, etc. 9.1.5 Directional SOVs: Licensees should verify that directional SOVs are installed in orientations which will assure satisfactory operation of the safety-related equipment which depend upon them. l 9.2 Maintenance 9.2.1 Frequency l Licensees should implement 50V maintenance programs to replace or. refurbish SOVs on timely bases. Replacement or refurbishment schedules should focus upon thermal aging due to elevated ambient conditions and heatup from continuous coil energization. PRELIMINARY CASE STUDY 57

l l 9.2.2 Replacement Versus Rebuilding

• i Licensees should review their programs for rebuilding SOVs because certain SOVs are difficult to rebuild and test properly, and improperly rebuilt SOVs i

degrade plant safety. Numerous utilities. canvassed have found that in most instances it is cost beneficial to replace 50Vs rather than to rebuild them. 1 If licensees choose to continue to rebuild their SOVs, we recommend that they obtain or develop test equipment to enable verification that the rebuilt S0Vs meet-all the performance specifications of the original SOVs. i 9.2.3 Contamination Aggressive actions should be taken to assure that fluids which flow through SOVs, instrument air in particular, are maintained free of contaminants. If operational experience indicates a pattern of SOV malfunctions resulting from contamination (such as water or hydrocarbon intrusion), the affected licensees should co.' sider replacing 50Vs that have been subjected to previous air system degradatiot, assuming that the root causes of the air system problems have been - corrected (in accordance with Generic Letter 88-14). 9.2.4 Lubrication 1 SOV manufacturer's lubrication instructions should be adhered to. Sub-stitution of similar but not identical lubricants should be avoided.

However, if substitutions are made, their compatibility with all associated hardware should be verified.

9.3 Surveillance Testina Operation and maintenance personnel training should emphasize the importance of surveillance testing, root cause failure analysis, and timely repair or replacement of malfunctioning SOVs. Licensees should review, and if appropriate, modify their surveillance testing procedures. Procedures should expressly prohibit " tapping" or mechanical agitation of S0Vs as techniques to assist successful operation during surveil-lance testing. Procedures should include actions to be taken when unsatisfactory test results are encountered, as well as a requirement to analyze and evaluate the causes of the unsatisfactory-results prior to declaring the component back in service (even though subsequent retest results may be satisfactory). 9.4 Verification of the Use of Qualified SOVs Licensees should review all SOVs in safety-related applications _, EDGs in particular, to ensure that they meet 10CFR 50 Part B and appropriate Class 1E requirements; and that they have been installed and maintained appropriately to assure they will operate in a manner consistent with the assumptions of the plants' safety analyses. _If there is doubt regarding the acceptability of any such SOVs, they should be replaced with appropriately qualified ones.

• exclusive of coil replacement - coils are generally replacement items PRELIMINARY CASE STUDY 58

9. 5 Redundancy and Diversity When operating experience indicates unexplained repetitive common-mode 50V

. isolation valves)g redundant components - (such as BWR MSIVs and containment failures affectin , licensees should consider performing maintenance, testing and replacement of redundant SOVs on a staggered basis. Additional consideration should be given to using diverse SOVs (different design or manufacturer). - 9. 6 Feedback of Operating Experience In order to' improve SOV reliability, an industry group such as the Institute of Nuclear Power Operations (INPO) should initiate an SOV failure feedback prog-ram. The program should alert S0V manufacturers to failures of their equipment by providing them with complete failure records of their specific S0Vs such as-those found in NPRDS. l t I 1. i l L I i PRELIMINARY CASE STUDY-59

10.0 REFERENCES

1. V..P. Bacanskas, G.;C. Roberts, G. J. Toman, " Aging and' Service Wear of Solenoid-Operated-Valves Used in Safety Systems of Nuclear Power Plants, Volume' 1. 0perating Experience and Failure Identification," Oak Ridge National Laboratory, NUREG/CR-4819, ORNL/Sub83-28915/4/V1, March 1987. } 2. Carolina Power & Light' Company, Licensee Event Report (LER) 50-324/88-001 Rev. 5, Brunswick Steam Electric Plant Unit 2, February 19, 1990. 3. Burns and Roe, " Investigation of Valve Failure Problems in LWR Power l~ Plants," ALO-73, April 1980. 4. W. H. Hubble, C. F. Miller, " Data Summaries of Licensing Event Reports of Valves at U.S. Comercial Nuclear Power Plants," January 1,1976 to J December 31, 1978 EG&G Idaho, Inc. NUREG/CR-1363 EGG-EA-5125, June 1980. 5. "IEEE Maintenance. Good Practices for Nuclear Power Plant Electrical Equip-ment; Work-In-Progress Report, Evaluation of-Maintenance and Related Prac-tices for Solenoid Operated Valves In Nuclear Power Generating Stations," IEEE Power Engineering Society, Nuclear Power Engineering Committee, .Subcomittee 3. Working Group 3.3, 1988. l 6. U.S. Nuclear-Regulatory Comission, Inspection Report 50-440/87-024, Perry Nuclear Power Plant _ Unit 1, January 22, 1988. l 7. Cleveland Electric Illuminating Company, Licensee Event Report-(LER)- j 50-440/87-073 Rev. 1, Perry Nuclear Power Plant, Unit-1, June 3, 1988. 8. U.S. Nuclear Regulatory Commission, Information Notice No. 88-43, " Solenoid Valve, Problems," June 23, 1988. -9. .U.S. Nuclear Regulatory-Commission, Inspection Report 50-440/87-027, ' Perry Nuclear Power Plant Unit 1, February 10, 1988. 1 10. U.S.. Nuclear Regulatory Comission, Inspection Report 50-302/89-09, Crystal River 3, June-7, 1989. 11. U.S. Nuclear Regulatory Comission, Inspection Report Number 50-336/88-22, Millstone Nuclear Station,. Unit 2, November 8, 1988. l 12. Memorandum from J. W. Craig, U.S. Nuclear Regulatory Comission, to A. C. Thadani,

Subject:

" Status of TI 2515/98 "High Temperature Inside Containment /Drywell In BWR and PWR Plants,"'. dated March 13, 1989. -13. U.S. Nuclear Regulatory Commission, Information Notice No. 89-30, "High Temperature Environments At Nuclear Power Plants'," March 15, 1989. 14. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 16313, Grand Gulf Unit 1, August 14, 1989. 15. U.S. Nuclear Regulatory Commission Region II Daily Report, August 15, 1989. PRELIMINARY CASE STUDY 60 9

g ?. ~ X _16.-- System Energy Resources, Inc., ' Licensee Event Report (LER) 50-416/89-013-Grand Gulf 1, September 13, 1989. 17. General Electric Company, Service Information Letter (SIL) No. 481, " Category 2, Malfunction of ASCO Solenoid Valves for MSIVs," February 14, 1 1989. 18. U.S. Nuclear Regulatory Commission, Information Notice 89-66, "Qualifica-tion Life of Solenoid Valves," September 11, 1989. -19. Automatic Switch Company (ASCO), " Field Notification Concerning the Qualified Life of ASCO Catalog NP-1 Valves," October 27, 1989. ' 20. Memorandum, P. T. Knutsen, VirgiM a Electric and Power Company, "Self Heating Effects in ASCO Solenoid Vai m." dated S bruary 3, 1987, 21. North Anna Power Station Deviation Report Number 87-105, dated February 4, 1987.- {

22. Memorandum, P. T. Knutsen, Virginia Electric and Power Company", "ASCO SOVs To Be Replaced at North Anna Power Station Units 1 and 2, dated March 16, 1987.

23. U.S. Nuclear Regulatory Comission, Information Notice 88-24 " Failures of Air-0perated Valves Affecting Safety-Related Systems," May 13, 1988.

24. Wisconsin Public Service Corporation, Licensee Event Report (LER),

50-305/87-012 Rev. 1, Kewaunee Nuclear Power Plant, March 4, 1988. '25. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 12013, Calvert Cliffs 1 and 2, April 14, 1988. 1 26. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 12015, 1 Calvert-Cliffs 1 and 2, April 14, 1988. 27. U.S. Nuclear Regulatory Commission, Inspection _ Report 50-317/88-07 and 4 50-318/88-08, Calvert Cliffs Nuclear Power Plant Units 1 and 2, June 3, 1988.

28. Memorandum from S, E. Scrace, Northeast Utilities, to E. Abolafia, " Failure of Air Operated Valves Affecting Safety Related Systems, NRC IN 88-24, NOA 9673, CR 0488-24 Revision 0," November 8, 1988.

29. Florida Power Corporation, Licensee Event Report (LER) 50-302/89-01 Rev. 2, Crystal River Unit 3, June 7,1989. 30.- U.S. Nuclear Regulatory Commission, Information Notice 80-40, " Excessive Nitrogen Supply Actuates Safety-Rolief Valve Operation to Cause Reactor Depressurization," November 7, 1980. 31. U.S. Nuclear Regulatory Commission, Office of Inspection and Enforcement, Bulletin No. 80-25, " Operating Problems with Target Rock Safety Relief Valves at BWRs," December 19, 1980. PRELIMINARY CASE STUDY 61

H 32. Metropolitan Edison Co., Licensee Event Report (LER) 50-289/80-018, Three n Mile Island Unit 1, November 4, 1980. 33. Georgia Power Company, Licensee Event Report (LER) 50-424/87-002,. Vogtle Electric Generating Plant Unit 1, February 23' 1987. 34. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 12890, Pilgrim Unit 1, July 19,1988. 4 35. U.S. Nuclear Regulatory Comission Region I Daily Report, July 20,-1988. l 36. Boston Edison Company, Licensee Event Report (LER) 50-293/88-021, Pilgrim Nuclear Power Station, August 18, 1988. 37. U.S. Nuclear Regulatory Commission, Inspection Report 50-336/88-22, Millstone Nuclear Power Station, Unit 2, November 8, 1988, l 38. U.S. Nuclear Regulatory Commission, Inspection Report 50-302/89-01, Crystal River Unit 3, April 13, 1989. t 39. .U.S. Nuclear Regulatory Commission'10 CFR 50.72 Report Number 14442, l Crystal River Unit 3, January 7, 1989. 40. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 14467, i - Crystal River Unit 3, January 11, 1989. 41. Target Rock Corporation Operation Manual TRP 1571J, May 23, 1978. 42. Gulf States Utilities Company, Licensee Event Report (LER) 50-458/89-022, River Bend 1, June-1, 1989. 43. U.S. Nuclear Regulatory Comission, Generic Letter 88-14, " Instrument Air - Supply System Problems Affecting Safety-Related Equipment," August 8,1988. 44. Gulf States Utilities Company, Licensee Event Report (LER) 50-458/89-024, River Bend 1, June 19, 1989. 45. U.S. Nuclear Regulatory Commission, Preliminary Notification, PHO III-85-84, September 20, 1985. 46. U.S. Nuclear Regulatory Commission, Region III-Daily Report, September 24, i 1985. 47. Comonwealth Edison Company, Licensee Event Report (LER) 50-249/85-018, Dresden Nuclear Power Station, Unit 3, October 1, 1985. 48. U.S. Nuclear Regulatory Commission, Office of Inspection and Enforcement', -Information Notice No. 85-95, " Leak of Reactor Water to Reactor Building Caused by Scram Solenoid Valve Problem," December 23, 1985.

49..U.S. Nuclear Regulatory Commission, Office for Analysis and Evaluation of Operational Data, Case Study No. AEOD/C403, "Edwin I. Hatch Unit No. 2 Plant Systems Interaction Event on August 25, 1982," May 1984.

PRELIMINARY-CASE STUDY 62

-- - ~. ~. t 4 ' 50. Carolina Power & Light' Company, Licensee Event Report (LER) 50-325/79-074, Brunswick Steam Electric Plant, Unit 1, November 16, 1979. 51. U.S. ' Nuclear Regulatory Comission, Office of Inspection and Enforcement, 1 Bulletin No. 80-14, " Degradation of BWR Scram Discharge Volume Capability," June 12, 1980. Sla. Cleveland Illuminating Company, Licensee Event Report (LER) 50-440/87-009, Perry Nuclear Power Plant, March 27, 1987.-- 52. Carolina Power & Light Company, Licensee Event Report (LER) 50-325/87-020, Rev. 1, Brunswick Steam Electric Plant Unit 1, March 31, 1988. 53. U.S. Nuclear Regulatory Comission, Information Notice 87-48, "Information Concerning the Use of Anaerobic Adhesive / Sealants," October 9, 1987. 54. Philadelphia Electric Company, Licensee Event Report (LER) 50-278/83-018, i Peach Bottom 3, February.10, 1984. 55. U.S. Nuclear Regulatory Comission, Office of Inspection and Enforcement, Information Notice No. 84-53, "Information Concerning the Use of Loctite 242 and Other Anaerobic Adhesive / Sealants," July 5, 1984. 56. General Electric Service Information Letter (SIL) No. 128, Revision 1, Supplement 1, Category 1, " Preventive Maintenance for CRD Scram Pilot Valves," August 1978.

57. - General Electric Service Information Letter (SIL) No. 128, Revision 1, Supplement-1, Revision 2, Category 1, " Preventive Maintenance for CRD Scram Pilot Valves," dated March 2, 1984.

- 58. Letter from P. W. Howe, Carolina Power & Light Co., to J. N. Grace, U.S. Nuclear. Regulatory Comission,

Subject:

Docket Nos. 50-325 and 50-324 Brunswick Steam Electric Plant, Units 1 and 2, " Failure of ASCO 8323 A36E Double Solenoid Valves," October 15, 1985. l-59. Carolina Power & Light Company, Licensee Event Report (LER) 50-324/85-008, l- . Brunswick Unit 2, October 25, 1985. 60. U.S.: Nuclear Regulatory Comission, " Report to Congress on Abnormal Occurrences, October-December 1985," NUREG-0090, Vol. 8, No. 4, May 1986. L 61. North Anna Station Deviation Report, 87-379, April 24, 1987. 62. U.S. ~ Nuclear Regulatory Comission, Ins)ection Report 50-338/88-02, 50-339/88-02, North Anna Power Station Jnits 1 and 2, March 14,1988. 63. Virginia Electric Power Company, Licensee Event Report (LER) 50-338/89-002, l North Anna Unit 1, February 7, 1989. L l 64. North Anna Power Station, memorandum, " Water Intrusion Into Instrument l Air System Event - April 1987," by T. L. Porter, August 8, 1988. L 1 PRELIMINARY CASE STUDY 63

l -I 65. North Anna Power Station, memorandum, " Justification for Continued Opera-tion, Unit 2 for ASCO CCW Trip Valves," by M. L. Bowling, February 10, t 1988. '66. North Anna Power Station, memorandum, " Justification for Continued Opera-tion, Rev.1," by D. A. Heacock, February 9,1988. L 67. U.S. Nuclear Regulatory Commission, Region II, " Notice of Significant Meeting," February 2, 1989. 68.'.U.S. Nuclear Regulatory Commission, Inspection Report No. 50-387/84-35; 50-388/84-44, Susquehanna Steam Electric Station, November 15, 1984. m 69. U.S. Nuclear Regulatory Commission, Inspection Report No. 50-387/85-09, 50-388/85-09, Susquehanna Steam Electric Station, April 15, 1985. 70. U.S. Nuclear Regulatory Commission, " Report to Congress on Abnormal r Occurrences, October-December 1984," NUREG-0090, Vol. 7 No. 4, May 1985. 1 71. U.S. Nuclear Regulatory Commission, Inspection Report No. 50-387/84-38, 50-388/84-37, Susquehanna Steam Electric Station, February 27, 1985. 72. Wisconsin Public Service Corporation, Licensee Event Report (LER) 50-305/ 88-07, Rev. 1, Kewaunee Nuclear Power Plant, May 19, 1989. 73. Letter _ from J. P. Weaver, Automatic Switch Co. (ASCO) to T. E. Murley, U.S. Nuclear Regulatory Commission, " Potential Failures of NP8314 Series Valves," October 18, 1988.' l 74. U.S._ Nuclear Regulatory Commission, Inspection Report 99900369/88-01, ' Automatic Switch Company, August 30, 1988. 75. Omaha Public Power District,. Licensee Event Report (LER) 50-285/82-006, Fort Calhoun Station Unit 1, April 7, 1982. 76. V. P.-Bacanskas, G..J. Toman, S. P. Carfagno, " Aging 'ind Qualification L Research on Solenoid Operated Valves," Franklin Research Center, p NUREG/CR-5141, August 1988. I 77. General ~ Electric Service Information Letter (SIL) No. 196, Supplement 10, Category 1, "SRV Failures To Open On Manual Demand and Air Operator Seal Failures," dated April 1981. i : ; 78. Duke Power Company, Licensee Event Report (LER) 50-414/87-031, Catawba' i Nuclear Station, Unit 2, January 20.-1988. 79. U.S. Nuclear Regulatory Commission, Inspection Report 50-413/88-20 and 50-414/88-20, Catawba Nuclear Station Units 1 and 2, June 7, 1988. E i 80. U.S. Nuclear Regulatory Commission, Inspection Report 50-413/89-07 and 50-414/89-07, Catawba Nuclear Station Units 1 and 2, April 20, 1989. 81. Letter from T. Hutchins Automatic Valve Corporation (AVC) to J. Keppler, U.S. Nuclear Regulatory Commission Region III, December 19, 1986. -PRELIMINARY CASE STUDY 64

d Mississippi Pt,wer & Light Company, Licensee Event Report (LER) 50-416/85-007

82..

Rev. 2, Grand Gulf Nuclear-Station, Unit 1, October 2,1985. 83. U.S. Nuclear Regulatory Commission, Information Notice No. 85-17, Supplement 1, "Possible Sticking of ASCO Solenoid Valves," October 1 1985. 84. CoIPeonwealth Edison Company, LaSalle County Station, "30 Day Report on the ] Failure of Main Steam Isolation Valve (MSIV) Pilot Solenoid Valve L 1821-F0280," prepared by Sargent & Lundy, January 14, 1988. j 85. U.S. Nuclear Regulatory Commission, Inspection Report 50-373/87-035, LaSalle County Station, Unit 1, January 1988. l 86.'- Gulf States Utilities Company, Licensee Event Report (LER) 50-458/88-023, River Bend-Station, Unit-1, October 31, 1988. 87. U.S. Nuclear. Regulatory Commission 10 CFR 50.72 Report Number 13591, September 30, 1988. -l l

88. Gulf States Utilities Co., River Bend Station, " Analysis of Components in an ASCO Solenoid Valve," Franklin Research Center Report, P-741-1, February.9, 1989.

89. U.S. Nuclear Regulatory Commission 10 CFR 50.72 Report Number 17201, November 27, 1989. 90. U.S. Nuclear Regulatory Commission Region III P W - Report, December 4,.1989. 1 91. Cleveland Illuminating Company, Licensee Event Report (LER) 50-440/89-030, Perry Nuclear Power Plant, December 26, 1989. 92. Automatic Switch Co. (ASCO), Installation and Maintenance Instructions, ~ Bulletin 8323, Form No. V5972R1, 1981. 93. U.S. Nuclear Regulatory Commission, Inspection Report 50-289/88-28, Three Mile Island Nuclear Station Unit 1, February 2,1989. 94. Facsimile Transmission, J. Shank, ASCO, to H. L. Ornstein, USNRC, i February 17, 1989. 95. U.S. Nuclear Regulatory Commission, Region I Daily Report, December 9, 1988. 96. Rochester Gas & Electric Company - Ginna Station Memoran6Jm, " Failure of Solenoid Operated Valve 5933B "A" Diesel Generator Air Start Valve ASV-1" by B. Popp, December 14, 1988. 97. U.S. Nuclear Regulatory Commission, Inspection Report 50-244/88-25, Ginna Nuclear Plant, February 2, 1989. 98. Iowa Electric Light and Power Company, Licensee Event Report (LER) 50-331/ 82-039, Duane Arnold Energy Center, July 6, 1982. PRELIMINARY CASE STUDY 65

99. Iowa Electric Lfaht and Power Compkny, Licensee Event Report (LER) 50-331/

85-002, Duane Arnold Energy Center, February 27, 1985. 100. U.S. Nuclear degulatory Commission, Inspection Report, 50-325/88-25 at.d 50-324/88-25, Brunswick 1 and 2, September 26, 1988. 101. U.S. Nuclear Regulatory Commission, Inspection Report, 50-338/88-02 and 50-339/88-02, North Anna 1 and 2, March 14, 1988. 102. Automatic Switch Co. (ASCO) Service Bulletin, " Discontinuation of Rebuild Kits for ASCO "NP" Series Valves," May 23, 1989. 103. Commonwealth Edison Company, Licensee Event Report (LER) 50-237/88-022, Dresden Nuclear Power Station, Unit 2, December 13, 1988. 104. D. V. Paulson, T. A. Shcok, V. P. Bacanskas, S. Carfagno, " Equipment Qual-ification Research Test Program and Failure Analysis of Class IE Solenoid Valves," Franklin Research Center, NUREG/CR-3424, F-C 5569-309/315, November 1983. 105. S. P. Carfagno, R. J. Gibson, "A Review of Equipment Aging Theory and Technology, Franklin Research Center, EPRI NP-1558, September 1980. 106. Automatic Switch Company (ASCO), " Field Notification of Discontinuation of NP8323 Valve Line," August 3, 1989. 107. Au'Aomatic Switch Company (ASCO), " Revised Field Notification of the Discontinuation of NP8323 Valve Line," October 27, 1989. I j i ( PRELIMINARY CASE STUDY 66 I i

._i [ g 4 I s5 E I $

a. e ' 4p - c..m$. '. J h. '.~

4, ,'f-[;_...5 '3i ' N. ./.'t~l,- g -t e"-' r'.4' g.,- a n; 3 ,-h. .- t e 1, 3 I )'. 'g. ~;g>hq Is dv.o A;$ 0,f i ; y J' ! ' ' y, Pi ,f ( !_, -! J: _.g1-i ' a 26 + _ i - y ' d 11 ua y > o. et. - h , 3 ' ',i - s o O n 3.;N[L;04 - ^ N' D --di-( -r. i g; -1a -'6 cA - W o L 1- - e i =O 4

• i

-t,' "4

-pip e '3

~ > s gi.Ruib m s c' .**,wj% . ;mb,. 3 ~ 1 .. m, c, .#'.- J :e _q' .u 4 a, x, l C s e' ;. ~m. 4 f t . h i.' s I, y *. p i g ) 3 _ i ) -g v pP :: ,;:s,r uu .y v v:'i s n p-1 o - u t o' q;:.,. <c s V ,5t a r ,..,vg (. 97";m;gu = %pm' l,,p: r i. g 'c 't + .w+ 3 w-

3' t

s te O -3.. dQ$lD t r.t:i5 5 i % Ming... j M ' (- g W i 3:)w m.-c r " _w p - ,J.'.. -s s m.;;bt: w, 1pyt;.x "I s + ,4 s t i.O 'g m c ',6 ( d ~ N 3 ,T. el 3 e jiA<gy mau r9 a 1-D .l t. ~i ,N i . i1 1. t"

c. p.

_- 5, U ' _ _ri 4, 1A 1 m.* 3 U.t1. :. f, g ) \\ }._ f, ( 'f. ', m,i g r i-t a.. t N ff./- ~ - ( i, g .4 1* A i ' k- [ -' - I z NG ,' Y .1

i

e t s.',',g,,

$.y( s. 'l it + " Wa a,.' 1 APPENDIX!A + 4 . ~ . c MM ? t$0V FAILURES REPORTED IN LERsF '19844, 9H e w# m I ,t1, J'- (E jg- ' y)v 46 [G c a.- ~ ~ ~ g,4 Q_ n i %mw g y. 'i4 f ~1+ Bf','1 @;..[t. i 5 , i --, s. -9 N p'Q #64 P a .l'Ik ~, ; \\ I .',i f - 1 3 ['h g 7 L'-, sf 31p d ,3, q = 2 j 5 L 6 3 i.j h, 3 ,5 , ? I. 41 4 i L/j. -.J.r b W g }o,;; $'t s ' p ; 'e f f-t.: -a.s.. n -1 ( t, .i Li; .d' . "t : o

c. l : ~

tM. f;. e a ,m 34 ts Wx' q W.,,, ' >p ,,.j h > w i 4;. i 6:., b,, u ; '.y q-4 it-v 7, 5{ q q 1 p; N r-gm. V

c. b,

,.. 3 s y [' 4=, y,r t 's > g.,.,'.3 y, p s=i 45 t 1 1,,.- (.- ry A N4 i,', j., u{3)' s A d. 1: s .g g .~;4 , f- ' 4 8 b. ci o4M'f.'1El'3 6 j vTr- - o. < = ' ' 5 b i F+ t w (, a - ,. pl J k t i a [.$ t T.' Q c, [. )) V, C 't 4 L 'p h,O s.= G - .'., $, T r a.t

e 3'

4a J q i. 1 5 . -. g 1,.^1* I'.' t e '. g - Q9__ 'ji, &., s as,j - b-i$ &. i

• 4 I.

N ;j. 1

.f"-

.1 4> a, 1

• i

,1,'? y Ro w\\ c, 1 , D,$, u g) y s E '\\_ }'4y' 1..b 9: { (. l \\ - f L. i

3. : s

01

,t @.- . + .s P:,P v, .,y... (. jib 1' t -j, ,,ty + s, 3 j r.c.' <-)p 3.- y i $I _f L ' t' 's I 3)t.J w o%~ ,f,e t x

9. n,. ;N "./_

v e;s > i 4 '. k - J '~ mQ : N, n., C-i!A : ~ W},r . e.. ,,g,., .gn' ,1 t rE i g ' S t, .,<m I 9 r1

g. n 91h go i :1, "z

4 i or- 'h: ,f. o[,,, W8 @. f t to ? 9 .1 w po \\ ,, ) ; J .ty 'f 'l:. y \\ %, '[i s A,'h['*.NQ@if, ' '{ r}.,;

• .1'-.,

q.- 4 @h; H s . t y. it - t;_ g i

c;

, F s Tj ", 4 i j.-', 1-y er> m W h c @Q.,, p NHp J y a dMhp:. ' A s e... s pe 3jl sr. 4,

Gd-IMlW,WN m %,Id D C,

2 .,.4s , t.: i ,y$a.q ~r .t ; mt -a. ( Mi{sW %,yY . ~...nG .3.,;_ ,P. S L' r \\ Wj m. ~ 1 i p ty *, =.a.,' \\ ~. f ? - Q' %, j'h'. V. f< l}j. {; % k..[- ... s. s so.- N$- _;_va = en . k, 3 ,.i .,h, '.{. e g LL ) ' Y 2.s .a. I t., 5'f t'- j ,;4 ; y v

I g

Y y l'3. [' y -- Jg ( g ' c.a u' g ps , s' : ' "n , y, f_f f j ['l ' t a e e u e.s t._ , h k. v't g j w _[; n ju in n

l T i Legend for Appendix A DOC NO. = Docket Number REP FL = Repetitive Failure TP/0UT = Cause Reactor Trip or Plant Outage FC = Failure Category l I P W ti ? l i

APPENDIX A FAILURE CATEGORIES OTHER 00 COIL FAILURE 01 VALVE BODY FAILURE / LEAKAGE 02 0-RING /0.ASKET/ PLUG / SEAT / DIAPHRAGM / SPRING FAILURES / LEAKAGE 03 LUBRICANT / LUBRICATION 04 " STICKING" 05 INTERNAL WIRING / REED SWITCH / CONTACTS 06 EXTERNAL WIRING 07 INSTALLATION / MAINTENANCE ERROR-PHYSICAL (BACKWARDS, UPSIDE DOWN, etc.) 08 INSTALLATION / MAINTENANCE ERROR-ELECTRICAL (LOOSE CONTACTS, AC vs DC, etc.) 09 EXCESSIVE ENVIRONMENT TEMPERATURE 10 MOISTURE INTRUSION (ELECTRICAL SHORTS / GROUNDING /0 PEN CIRCUITS) 11 CONTAMINANTS (DIRT, WATER, RUST, HYDROCARBONS, DESICCANTS', etc.) 12 MOPD (MAXIMUM OPERATING PRESSURE DIFFERENCE) 13 DESIGN ERROR (OTHER THAN MOPD) 14 EQUIPMENTQUALIFICATION-SEISMIC 15 EQUIPMENT QUALIFICATION-RADIATION 16 INADEQUATE MAINTENANCE / EXCESSIVE TIME BETWEEN REPLACEMENT OR OVERHAUL 17 g "END OF LIFE"/ NORMAL WEAR 18 i "STILL UNDER INVESTIGATION" 19 " UNKNOWN" 20 " UNSPECIFIED" 21 " PERSONNEL ERROR" 22 j REQUIRED CLOSING /0PENING TIME SPECIFICATIONS MT MET 24 LEAKAGE UNSPECIFIED 26 ASSEMBLY ERROR (PLUG / DIAPHRAGM / SPRING etc.) 27 EQUIPMENT QUALIFICATION (ELECTRICAL) 28 i

t Page No. 1 06/07/90 SOLEmotD-OPERATED vatvE FAILURE DATA 4 DOC PLA4T EVENT LER

1. 0. OF FAILED SYSTEM nmnUFACT MODEL 2007 REP CORPECTIVE CDuuEE#TS REFEREWCE TP/ FC

..u. NAME DATE NUMBER TAllturES PART WO. CAtf5E FL ACTlau 80CUNF#TS SUT 206 San onofre 1 12/30/86 86-014-01 One Granad Feedueter & mot mot meisture in to sew J m etion Corractive LER 87-001 se 11

fault, Sefety Specified Specifi itriction tot im instatted action tenen on moisture Injection ed failed junction in System boa and seven junction other box vulnerable ones.

206 San Onofre 1 01/17/87 87-001 One Ground Feeduster Inodecysete Yes Etiminsted Vibration no 07 fautt installation /w gremd, tighten caused ibration ed comections loosening of terminst tm conduit tecking ring 206 San onofre 1 11/10/87 87-016 Five failures of Stug Containment ASCO 206-380 Lthricant Yes Secured Sovs Cause of Insp opt me 05 four valves sticking Isotation, suspected in safety sticking m der 89-24 Contai<uamt position aruf inv*stigation Spray initieted weekIy testing 206 San onofre 1 12/01/87 87-017 Two not Safety not mot Ur*nown no Repeired or Sov ragtsired sone no 17 Specified injection Specified Speeifi reptneed SOW for venting S1S vent ed to ovoid water hessmer 206 San onofre 1 12/16/87 87-018 One Gre ruf Plant mot mot Loose screws Yes The ground was The loose See no 11 fault cooling Specified speciff and inedequete eliminated by screws were consents moisture water ed ses'.. Root removing the probably in SOW tause not water inside strippad from housing specified the solenoid escessive housing and tightening. reseeting the Ref. Docs. LENS housing. 206/86-014/01, and 361/87-001,031 206 San onofre 1 02/15/88 88-004-02 One SOV Safety Terget 80EE-00 Stilt urufer Yes Sov was 50V feiture LER wo 19 sleeve injection acck 1 investigation replaced. prevented bleed 206/81-020 and modifled off from doulde position maintenance disc gate valve indicatio procedurestine bonnet. n switch tuding imptementation of mfr's recesumend for new reed switch cattbration ~. -.

e ge No. 2 s 06/07/90 SOLEWOID-OPERATED VALVE FAILIJRE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MmuuFACT MODEL a00T

1. EP COneECTIVE CtpuuENTS WEFEWEWCE TP/ FC 40.

NAME DATE NUMBEM FAILURES PART NO. CAUSE FL ACTION DoctpetTS enfT 206 San onofre 1 03/03/89 89-006 Conteirment Design error Desip Discovered thet 14 fire modification a single S0W stepression mode could degrade conteiruumt spray system,resuttin e in centa truent overgressure during a LOCA 206 San Onofre 1 08/23/89 89-026 Ore Failed to Rectre ASCO 206-380 Suspect Yes septoced SOV LER 87-016 to 05

shift, systee Itbricant

• sticking (sofety stug*

Injection /co nteirwumt sprey) 213 Maddam Neck 11/02/84 85-005 Tuo Failed to Auxiliary ASCO =P8320 Ur*nown no Sov retested Sovs feited mone no 05 shift Teeduster acceptably, dsring testing. " stuck" System dectered S0Ws reoJired g erationet, for pore fregsent outo-initietien cycting tests of AFU pterred 213 Heddam Neck 09/10/85 85-024 One ' Failed to Atatitiary ASCO NP-8320 Ur* noun Yes Reptoced Sovs. Cause of LER 85-005 wo 05 shift,*st Feeduster Inittsted more sticking nos uck" System frequent not been periodic determined. cycling Saee 50Vs as in LER 85-005 213 noddam ucck 01/14/88 88-001 Four incipients SOV Conteirment Not Not Design No Cor m ted Instetted S0Ws mene no 06 operating Isotation - Specified Specifi Deficiency _ circuit close upon mode Steam ed design, rather de=nergiring Generator then changing insteed of stoudoun the SOVs opening upon deenergizing per design. Condition existed for seven years e ~ n , - ~ - -

t Page No. 3 06/07/90 SOLEN 0!D-OPERATED VALVE FAtttfRE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEat MANUFACT MODEl. ROOT REP CURRECTIv5 CtpWIENTS REFZaEuCE TP/ FC NO.

1. AME DATE MUMBER FAILtRES PART NO.

CAUSE FL ACTION 000plEWYS Otif 219 Oyster Creek 10/16/84 84-022 Three Dieg*reru Scram hot Not - Instetted no Install Caused slow mene me 27 Discherge specified speciff diaphragm dia:Aram elesure of 3 Voltsee ed backwerds. correctly and air-operated Inede wete 50V develop SDV vent and rebuitding and leproved drain volves inadequate post-maintenen post-mointenen ce testir:g ce test 220 Nine Mite Pt 1 06/14/84 84-013 Three Seat Main steam Dresser /c 1525vX Wear and Yes 1 refurbished, Retest of att 6 LER 84-014 no 03 teekage(2 line onset. con *aninents 2 replaced volves fotnr. ),misposi Electrome suspected att to be tiened tic teaking due to wires meteriel lodged in the seat eres (see LER 84-014) 220 Nine Mile Pt 1 06/17/M 84-014 Six 5 seat Mein steam Dresser / 1525 Vx Foreign Yes cleaned and Retest of att 6 84-013 me 12 teskoge / Consol. meterlat refurbished 50Vs (LER 1 stuck Electrome intrusion Sovs 84-013) foemd open due tic (source not ett to be to stated) teeking dJe to foreign foreign mett meteriet todged in the sent eree 220 Nine Mile Pt 1 11/01/85 85-021 One plus twe Janned Main steam Dresser /c 1525vr Weer Yes Reptoced ett mene no 03 incipients springs onsot. three velves Electrome tic 237 oresden 2 07/17/87 87-023 One Internet Feeduster ASCO 8300 Weer Yes Reptaced SOY SOV is a mene Yes 18 passagewa (FWRV) pieceport of the FWRV. y restricti on 245 Mittstone 1 12/2C/85 85-034-01 8etween three 1 core Ctetrol rod Asco Not Deterioration Yes SOWS rebuilt, Failure of None No 17 and six

spring, drive speciff of the Dune-N spgraded SPSV three controt many ed discs and a meintenance rods to scram discs detached program per GE was attributed sprits.

SIL 128 to feiture of i three to sin essociated seram pitet solenoid vel m.

Page No. 4 06/07/90 SOLEWO!D-OPERATED VALVE FAILURE DATA DOC PLAN' EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORRECT!WE CXpWEEWYS REFEWEWK TP/ FC NO. NAME DATE NUMBER FAILURES PART 20. CAUSE FL ACTION DottpENTS mff 2&5 MitIstone 1 06/06/87 87-015-02 One Excessive Contairment larget mot Plteper tube so Replaced

1. ene mene sa 93 ieekage isetation - Rock Specifi scored pleger tube post ed accident saspting 247 Indian Point 2 01/04/84 84-001 One Faited Contairveent ASCO mot Not Specified so Repisced SOY mone wone no 21 cIosed porge Specifi ed 247 Indian Point 2 11/27/84 84-022 Two not AFU Steam Not Not Not Specified No Recorvincted SOVs contret None No Of Specified Specified Specifi power leads to AFW turbine ed SOVs intet steam isolation vatwes 247 Indian Point 2 02/02/87 87-003-01 One Sluggish Condensate Not Not Design Eo Enterged 50V SOV controts mone
2. o 24 performan (storage Specified Specifi deficiency orifice and A0V. Slow ce tank ed (sizing) cleaned closure isolation) regulator attributed to orifice sire.

Debris could have etso centributed. 249 Dresden 3 01/12/85 85-001 One Manual Main turbine Sperry FSDG&S4 Grease No Replaced SOV SOV controts mone Yes 04 operator Vickers 012A containinetton a.u-M trip 249 Dresden 3 08/07/87 87-013 One Colt Feeduster ASCO 8300 Shorted coit No Replaced 50V SOV controts mone Yes 01 FWRV sir operater 250 Turkey Point 3 12/02/84 84-031 One mot Contalement Asco not mot Specified no Reptsced SOV LER250/84-wo 03 Specified IsoIation specifi vatwe 09,020 tnitrogen ed stspty) 250 Turkey Point 3 12/13/84 84-034 One Wot CvCS ASCO mot Yes Replaced SOV Sov controts See so 02 specified (isolation Specifi A0V. Ref. Conusmts watwe) ed Docuuments: LER 250/84-032, 251/84-009,84-0 20 4 ec ....m .m.

Page no. 5 06/07/90 SOLENDID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER No. OF FAILED SYSTEM MANUFACT MODEL WOOT REP CORWECTIVE CtpeEEWIS REFERENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION DOCUMENTS Olff 250 Turkey Point 3 01/13/85 85-002 One Clogged not Wot mot mot specified so Cleened air simiter mene no 17 4 30V sir Specified Specified Speciff fitters en occurrences: fitters ed this and other LER 250-84-034, similar Sovs LER 250-84-031, in both units LER 251-84-020, 3 ard 4 LER 251-84-009, and LER 250-83-016 250 Turkey Point 3 01/27/86 86-005 Two not Main steam ASCO 8316 1 internet so Replaced 1 2 i.e a.t mene Yes 09 Specified (MSiv) interference, SOV, fuse SOV failures 1 bent contact block pins discovered pins at fuse were during testing. block. straightened Mstv couldn't en other 50V. be closed 250 Turkey Point 3 08/03/86 86-031 One Not Auxiliary /em ASCO 206-381 Water entering No SOV replaced similar See Yes 03 specified ergency the 50V occurrences: consumt feedwater LER 251-84-020, and LER 251-84-009 250 Turkey Point 3 01/03/87 87-002 One Colt Convenes t ASCO 8316 Not Sp& ified No Replaced SOV mene NO 01 Cooting Water 250 Turkey Point 3 09/13/87 87-023 One Internet Steam Target 300525-Fautty wirt No Not SpeCffled None None Yes 06 wiring Generator Rocit 1 going to Reed 8towdown switch-251 Turkey Point 4 07/15/87 87-015-01 One Ground Contairement not not Deterioration No Cleened and 50V is e mene -- No 18 - fault isolation Specified Specifi of insulating retaped wiring piece-port of (pressurizer ed tape from connections ?.0V normet sagting) a ageing" 254 Ouad Cities 1 02/05/85 85-001 Two Comectio MPCI serksoste 1782504 Faulty Repair Failure of MPCI No 07 n to SOV C2D4 terminet terminet turbine tripend power comection and comections reset SOVs lead vibration and secure wires to SUV housing , - ~

Page No. 6 06/07/90 SOLENOID-OPERATED VALVE FAILtRE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT NODEL ROOT REP CURRECTIVE COBUEENTS REFERENCE TP/ FC NO NAME DATE NUN 8ER FAILtRES PART NO. CAUSE .FL ACTION 90CMENTS tRif 254 oued Cities 1 04/03/87 87-006-01 One Wiring Nigh serksdete 1013433 vibration /ined fes Reptoced coils NPCI LER 85-001 No 07 connectio Pressure ACP1 equete on feited SOV inoperable. n to coit Coolant connection /ine and three Reptoced SOV Injection dequate others coils with support reptoced et newer mo&t, inits 1 and 2 also added wiring restraint to att four SOVs. 255 Patisades 04/10/86 86-017-01 Three felt + vetve Reactor Target 808-001 Metal shavings Yes Repelred SOVs Yes 12 three incipients seat Coolant - Rock in vetve seat .and systeur leakage (head vent) area. flushed to remove remaining metet shavings 255 Palisades 01/14/87 87-001-01 Eight Inst >quet Containment Not Not AE design No Isolation None None No 14 e isolation (hy specified specifi error togic modified isolation drogen ed logic monitoring) 259 Browns Ferry 1 07/03/86 86-022 Six incipients ECCS Rockwett/ Design error Remove air Potentist for No 14 Atwood steply to overpressurizin Morritt affected g low pressure actuator systems due to use of non quelifled SOVs (sin in each of three growns Terry units) 260 Browns Ferry 2 08/31/87 87-007-01 Potennat loss of Containment Not Not Design error Yes Replace SOVs Use of None No 14 faltures att 3 SOV Drywett specified SpecifI with quelified non-quetifled units ftnetion Contro1 Air ed ones SOWS coutd prevent primary conteirwent isolation. Att 3 srowns Ferry imits affected. 4 .a

e . cr .Page No. 7 06/07/90 SOLENOID-OPERATED VALVE FAILtRE DATA 4 DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORGECTIVE CapetENTS REFERENCE TP/ FC No. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION 000pENTS OUT 260 Browns Ferry 2 06/06/89 89-018 One Valve Emergency Setem 812-6 Corrosion Yes Replaced SUV Licensee No 12 seats diesel debris from tegraded EDG generator starting air air system and air start system performed maintenance on it prior to event but debris was believed to be there from before 261 H.B. Robinson 2 05/13/87 87-007 Two Not Not ASCO Not Inadaquete Yes Instatt incorrectly None No 14 Specifled Spectfied Specifi instatIations correct seats instatled ed of conduit conduit seats seats at entrance to several harsh emironment IE qualifled 50Vs. Potentist for moisture intrusion 261 H.B. Robinson 2 07/15/87 87-020 One Electrica Feedwater Not Not Water trapped No Wire was SUV is None Yes 11 I short (FWRV) specified Specif f in SOV fepaired and piece-part of ed condotet water removed FWRV from the condutet. Other SOVs examined for similar problems. 261 H.B. Robinson 2 11/05/87 87-028-01 Two SOV Dieset Not Not Internet wear No Replaced SOVs SOV failures None No 18 internets Generator Specified Specifi caused venting Starting Air ed of starting air 263 Monticelto 10/25/89 89-032 One Loose Main steam Tighten No 09 t terminal (MSIV) terminal screw screw and inspect simiter SOVs

.w s Page No. 8 06/07/90 SOLENCID-OPERATED VALVE FAILtRE DATA DOC' PLANT EVENT LER WO. OF FAILED SYSTEM-MANUFACT MtBEt. ROOT REP CORRECTIVE CtseENTS REFEWENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART WO. CAUSE FL ACTION DOCtmEWTS OUT 265 Quad Cities 2 06/28/85 85-015 One Not Reector verse See not Specified No SOV replaced VGS-4422-U-10-3 mene no 20 Specified 8tdg. Vent. comment 1-3ec System 265 ound Cities 2 02/18/87 87-004 One mot Containnent ASCO 8317 " Solenoid Reptoced SOV SOV is no 21 specifled vectase rusted and piece-port of corrodee' vacuum breaker (reeson/ source air test not stated) cylinder 265 ound Cities 2 09/18/87 87-012 One plus two Not Contairunent ASCO 8317 Unknown Yes Not Specified SOV is LER 87-004 No 20 incipients specified vacuun piece-port of Retief vacuus breaker air test cylinder 265 ound Cities 2 12/10/87 87-020 One not Main Turbine Sperry F3-SDG4 Not Specified No Rptacej SOV None None Yes 02 Specified Controt Vickers 54-0124 Fluid 265 ound Cities 2 04/06/89 89-001 One Turbogenerat No Rebuilt SOV Failed SUV LER 87 020 Yes 21 or controts turbine mester trip solenoid 266 Point Beach 1 06/01/89 89-003 One containment ASCO 8302 Reptace Sov wo 21 isolation (SG bloudoun sanpting) 271 Vermont Yankee 08/18/87 87-009-01 wot specified Seat Automatic ASCO 206-381 Dirt / corrosion Yes SOV cycted mone mone no 12 teskage Depressurira products frone tion the air supply 272 Salem 1 12/31/84 84-029 One Faulty Feedwater ASCO Not Not Specified Yes Replaced SOV SOV is a None Yes 09 electrica (FWRV) Speciff piece-part of 1 ed FWRV connectio n and seat teoksge 272 Salem 1 01/31/86 86-003 One Seat Feedseter ASCO mot Probably Yes Two SOVs were SOV is a mone Yes 12 ieekage (FWRV) Specifi centaminsted reptseed piece-port of ed air the F1dRV. Dirt and moisture were detected in air lines causing other associated failures c 4 -,., e .n_, --w

Page No. O 06/07/90 SOLEmotD-OPERATED VALVE FAILt*E DATA DOC PLANT EVENT LER No. OF FAILED SYSTEM MANUFACT MMt. - RCOT REP CORRECTIVE COMENTS REFERENCE TP/ PC NO. NAME DATE NUNDER FAlltRES PART NO. CAUSE FL ACTION DoctpENTS inff 272 Salem 1 02/20/86 86-006 One Broken Feed ster Not Not Installation No Reptsced wire None None Yes 09 wire (TWRV) specified Spgifi error and and checked ed - vibration similar SOVs 2 72 Salem 1 04/06/86 86-007 Eighteen Electrica Post Not Not Design / install No Instatt 18 SOVs on None No 14 incipients t accident Specified Specifi ation required units 1 and 2 connector sampling ed error,inadeque ccvectors had inadequate s te-connectors instattation procedures 275 Diablo Canyon 1 01/02/85 85-001 Two SOV Main turbine Not Not Not Specified No Reptaced SOV None . Yes 21 " stuck (overspeed Specified Speciff open" pretection) ed 275 Diablo Canyon 1 07/24/87 87-011 None containment Not Not Procedurst No Perform Failure to None No 22 isolation Specified Speciff inadequacies necessary verify ed verification. penetration Upgrade isolation procedures subsequent to SOV replacener't. 277 Peach Bottom 2 04/27/84 84-008 one Not Containment Asco 8320 Not specified No Reptsced Sov Potentiat Wone. No 19 Specified Isolation existed for a (58GT) single fatture to have pcevente3 the fulfilment of the safety ftmetion of the $8GT system 277 Peach Bottom 2 01/24/86 86-003 Two DC colts Main Steam Automatic Not Under No The failed DC Faiture of 2 DC None Yes 19 (MSIV) valve speciff investigation solenoids were SOVs in 2 Cougany ed replaced. separate lines (AVC) caused closure of MStVs 277 Peach Bottom 2 05/29/87 87-008 Three Controt room Piping No Reconnected Sample lines to No 20 ventitation/ configuratio.. tihing to Sovs three Sovs had radiation error property been connected monitoring incorrectly. Affected control rooms at both units 2 and 3 L s , ~.. c ,__m 2 m m .m.

w-Page No. 10 - 06/07/90 SOLENOID-OPERATED VALVE FAILtJNE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTER MANUFACT MODEL ROOT REP CORRECTIVE COWIENTS REFEREWCE TP/ FC NO. NAME 'DATE NUMBER FAILURES PART No. CAUSE FL ACTION DOCUNENTS OUT 277 Peach Sottom 2 10/05/89 89-C23 One sinding Mein steam Automatic 6910-20 Inadequete to Replaced SOV Reference LERS See Yes 27 of SOV (Msty) Valve manufacturer's and revised 277/86-003, causents stog Company instattation instattation 278/85-018, (AVC) instructions and 278/86-016 maintenance procedures 278 Peach Bottom 3 09/30/85 85-015-01 One Leaked ADS backup Terget Not Not Specified Yes Replaced 50V Previous See No 03 nitrogen Rock Speciff with an simitar comuments ed upgraded one occurrences reported in LERs 277/85-01 and 278/85-05 278 Peach Bottose 3 07/11/84 85-018 one DC coit main steam Automatic Not Reason for Yes Task force DC SOY failure None Yes 01 (MSIV) Vatve Co. speciff colt failure recomended coupled with ed not specified testing of DC momentary loss solenoids more of AC power often and resulted in anatyre cause MStY closure of future failures. 278 Peach Bottom 3 07/19/86 86-016 One Colt Main Steam Automatic Not Reason for Yes The de colt on Simiter reactor See Yes 01 (MSIV) valve Specifi colt faiture each MSIV's screes in 1985 comunents Corp. ed not specified SOY was and (AVC) replaced. 1986(defective de colt coupled with oc power interrtetlon): LERs 278/85-018, 277/86-03 280 Surry 1 03/28/84 84-007 None Unspecifi Feeduster Maintenance No Recomected IA Instrtsment air No 08 ed (FWRV) had been done lines to lines were without proper SOV comected to approved ports the wrcng ports procedures of 5 SOVs at inadequate Surry units 1 post and 2 neintenance te= ting G 1 m-y y

R Page No. 11 06/07/90 SOLEN 0ID-OPERATED VALVE FAILtNE DAt*_ DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT n0 DEL ROOT REP CORWECTIVE CtpWENTS REFERENCE TP/ FC NO. NAME DATE NUMBER FAILtRES PART

1. 0.

CAUSE FL ACTION DOCtpENTS IRIT 280 Surry 1 11/12/87 87-031 One SOV conteirament Masoneite 3500 laproper No Secured 90V Wiring to No - 09 wiring isolation n (SOW series instettation tswpecified SOV blocked unspecifi caused isolation' ed) sechanical vetve binding of operator centeinment isotetion vetwe's operator - 231 Surry 2 01/27/88 88-001-01 Two SOV Conteirument Target 86V-001 Cause of SUV No Repair or Electricians None No 26 teskoge isolation (pr Rock /ASCO /206-38 teekoge not keptoce SOWS trying to essurizer 0 specified. Isolete teeking vapor space Cause of wrong SOVs tifted sempting) tend lifting: wrong teeds electrical meintenance "personnet error" 281 Surry 2 02/02/88 88-002-01 Two Seat Reactor vetcor V526-56 tapurities in SUVs reptoced No 12 teskege cootent 83-19 reactor sempting coolent system isotetion water prevented complete seat closure 285 Fort Calhoun 05/01/86 86-003-01 Two Feiture Weste gas Not Not Personret Non Return SOVs to Fail closed m No 22 positions Specified Speci75 error e correct SOVs had been of SOWS ed feiture changed to fait reversed positions open, resulting in voltsee coratrol tank teekoge to auxillery building. 286 Indian Point 3 02/1*/87 87-002 One Co8; conteirument ASco 8308 Not Specified Yes The failed The design of LER Yes 11 teekage solenoid vetve no. 34 static 85-001-00 control reptoced with irwerter was - one of e - improved to higher sitow isotation temperature of siruJ e design. 3 bronch circuits simiter $0V if a short colts were circuit also reptoced. develops. r

1. c m

• m.

m. m. m.m.. .m. +

Pope No. 12 06/07/90 SOLENDID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER-NO. OF FAILED SYSTEM MANUFACT MtBEL WOOT REP CORGECTIVE Ctp5EEWTS REFERENCE TP/ FC NO. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION 00C15EENTS TWT 293 Pilgrim 07/19/88 88-021 Four incipients Potentist Primary ASCO 8320 Design error No acptoce 30Vs Fellure of None-me 13 for conteirwent, and with ones pressure exceeding contret rm,+ NP8320 rated for reputator would MOPD turb btdg higher MOPD resutt in timits nVAC/SGTS inoperability of 4 SOWS dte to exceeding-MOPD limits 293 Pilgrie Ot/27/89 89-004 Conteirwent ASCO NPS320 Repelred teoks Failure of 2 LER 89-002 Yes 21 isolation and reptoced 2 A0ve due to air SOVs swtem teoks. 2 SOVs were replaced es a precaution against exceedirig MOPD a limits of the SOVs 293 Pilgrim 05/03/89 89-015 One Colt Main Steam Automatic 6910-c2 M andom po Reptoced Sov Yes 01 (MSIV) valve O faiture" assembly Corp. (AVC) 295 Zion 1 08/Or/85 85-029 Two Mtuck" EDC building Not Not Not specified Yes Replaced SOVs 40 such volves LER No 05 pilot ventitetion specified specif9 used in both 304/85-015 valve ed units. Common-mode fattures foted dring testing. Additionel DeFs ocurred next day et seit 2. 295 Zion 1 01/12/89 89-001 One Faited to Ventitetion ASCO 83M Weekened colt Yes Reptoced Sov LER 89-001 wo 01 shift (service unter building) i o 9 .-y

= Page No.. 13 06/07/90 SOLEmotD-OPERATED VALVE FAILtJRE DATA DOC PLANT EVENT LER WO. OF FAILED SYSTEM MAutNACT MODEL ROOT REP CORRECTIVE CSSIEETs REFEWEeCE TP/ FC 40. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION DOCtpEWYS Off 298 Cooper 08/18/86 86-018 one not Reector mot mot not specified No #ct specified mone mone no 21 specified Recircutatio specified speciff n system ed 302 Crystal River 3 01/05/89 89-001-02 mone Muttiple Asco 8320/uP Design les Reptaced sovs r-w ' dien see no 13 systems 8316/83 error-MorD with others

• of this cons =mts 20 having higher r w t for McPD rating additlenaL info.

Reference docu=mts: LER 78-054, 83-023, 88-C13 302 Crystat River 3 04/07/89 89-012 conteirment ASCO 8320 Design error Neptace 50v 8 SOVs were see

1. o 14 coits with effected.

comments isolation coils having Reference (RM cavity correct docteents: LER cooting tenperature 78-054, 83-023, system) ratings 88-013, 89-001 302 trystat River 3 04/18/89 89-015 Reactor inadequate Modified 50V 15 cootant ptsup seismie steports seat ble-d instattation off 302 Crystal River 3 09/26/89 89-034 Electrica MVAC, Design error Modified power Intermingling No 09 1 power containment sigpties of 1E aM non-1E power stypt ies isolat f ort, sources to s"Afs Main steam (MSIV) 304 Zion 2 07/11/84 84-015 mot specified Internet Main steam Keane 51-170 Licensee could no Three sovs to sone wone no 26 teskage (MS!v) not find cause be replaced of failure with envircrementalt y quatifled 50Vs 304 Zion 2 08/09/85 85-015 Two

• stuck *pi EDG building Not Not Not specified Yes The vatwes Coasuon-mode LER mo 05 tet watwe vent specified specifi were reptaced. faltures fotsw! 295/85-029 ed during testing.

Also occurred on init 1 the l previous day. l 40 such valves 1 on units 1 and 2. } l

d ve 4 Page No. 14 ' 06/07/90 SOLENOID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORNECTIVE-CUNIENTS REFEWENCE TP/ FC NO. NAME DATE NLREBER FAILURES PART NO. CAUSE FL ACTION DoctpfMTS IRff - l 304 Zion 2 02/03/87 87-001 One 0-Ring Rain stees Chicago NSv1-16 Manufacturing No Reptaced SUV None None Yes 98 (MSIV) Fluid -C-NP defect or Fower damage during instattation 305 Kewaunee 07/02/84 84-013 One Coit Aunitiary Johnson V-24 Not Specified Tes The Johnson 50W failures 82-03,28, No 01 building vetwes were to resulted in 81-34 special be replaced initiating ventitation with ASCO safeguards NP8320 S0Ws es equipment. 59 they failed. such S0Ws ressaining would be replaced with ASCOs.ed at next outage 305 Kewaunee 12/16/84 84-020 One Coit Auxiliary Johnson v-24

• eurnt out" Yes The Johnson Oue to LER 84-13 No 01 butIding colI, root SUV was repetitIwe special cause not replaced with fattures of wentilation-specified an ASCO these Johnsor.

WP8320. S0Ws, they were att being replaced with ASCO NP8320 St#s en an os-fait basis 305 Kewaunee 02/11/85 85-005 One Coit Aunillary Johnson V-24 Colt

• burnt Yes Replaced 50V Due to LER No 01 building out,' root with an Asco repetetive 84-013,C20 special cause not fattures of ventitation stated these Johnson S0Ws, they were ett being replaced with ASCO NP8320 S0Ws on an t

es-fait basis. 305 Kewaunee 11/28/87 87-012-01 Two falted plus Failed to Conteirunent ASCO NP8314 Design error. Yes Replace S0Ws See Section None No 13 58 incipients shift Isolation-PZ Conditions and correct 5.1.3 of this r exceeded SOWS' regulator report relief,make-MOPD timits settings so up,RCDT that MOPO discharge ratings will not be exceeded d w 4 - w r a + ,..-a.,

cs Page No. 15 06/07/90 SOLENotD-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. Cf FAILED SYSTEM MANUFACT MODEL ROOT REP CORWECTIVE COMEENTS REFERENCE TP/ FC NO. NAME DATE NUMBER FAlttRES. PART NO. CAUSE FL ACTION DOCIMENTS OUT 305 Keuounce 05/28/88 88-007-01 Three plus 7 Failed to conteirament ASCO NP8314 Montef acturing No Cteened and Initteted an-LER No 95 incipients shift Isolation error refurbished extensive root 87-012-01 (p2r relief, tuneuthorized the effected cause anetysis. makety use of SOVs See Section isotation) incorrect 5.2.4.1 of this ttbricant) report. 309 Maine Yankee 08/10/86 86-005-01 One Ground Cardon Fire Chemetron 5-020-0 mot Specified No Reptoced SOV SOY failure No 21 fault Protection 0 74-8 tripped Cardon system system power stsyty breaker, thereby disabling the Cordon system. 309 Maine Yankee 05/23/88 88-005-02 Four incipients Not MPSI/chargin R.G. 620uA24 Design error No Modified SOVs in high None No 16 Specified g ptmp Laurence DCSU system red. fields not suction vent environ. quet. Falture could cause secontretled reteese of radioectivity to nan quet. systems. 311 Salem 2 05/22/89 89-011-01 None Mein steam Inedeemte No Modified Testing Yes 14 (isolation surveillance testing deficiencies volve) testing circuitry would prevent detection of SOW failure Deficlency-existed at imit 2 etso 313 ANO 1 05/06/85 't-001 Two Lifting Post Target 80E-001 Design error No S0W were Incorrectly LER No 06 of accident Rock /81P-00 reoriented oriented 50Vs 368/88-001 plunger sanpting Corp. 64 correctly could open spon (spurious smelt increeses actuation in ) beckpressure. See Section 5.1.4 of this report

-- -Q Page No. 16 '06/07/90 SOLEWOID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER WO. OF FAILED SYSTEM MANUFACT MODEL WOOT REP CORRECTIVE CGeIENTS REFERENCE TP/ FC N0. NAME DATE NUMBER FAILURES PART

1. 0.

CAUSE F t. ACTION 000pENTS OUT 317 Calvert Ctfffs 1 06/01/87 87-007-03 Four incipients thiquet f

• f Atatillery not not Design error. no Deficient Two Sovs on mone Yes 28 ed Feeduster specified specift electricot coch unit found electrica ed comections to have 1

were segreded inadequete (EC) connector with EG electrical s quetifled ones comections 317 Calvert Cliffs 1 08/22/89 89-015 0 todine Design error Replace with 50v falture no 15 fitter (o 1ist seisariestty coutd prevent dousing ctassification quetifled sovs iodine filters system ) from performing their function 317 Calvert Cliffs 1 11/13/89 89-020 0 satt water Design error Reptsee with 4 sovs in no 15 cooling (Q list seismicatty safety system classification quellfied sovs ret able to ) and power withstand sources seismic event power sources-for 5 nafety-related sovs not seismicetty quetified 318 Catvert CtIffs 2 09/05/86 86-006-01 One seat Main steem ASCD 8300 mot specified no sov internets mone mone no 03 teskage (atmospheric were replaced disy) 321 Hatch 1 12/07/85 85-043-01 atster of failed seat Contairunent mot not normat Yes Leefting Wone LER 84-017 No 18 sovs not spec Ia dage isolation specified specifi equipment use volves in 42 -atet tiple ed or weer penetrations Systems repeired, rebut it, or replaced. 321 Match 1 04/15/87 87-004 One incipient Main contret Fot Not AE design No pedesign mein Single 50v Wone No 14 room specified specif f deficiercy controt room failure could environmente ed envirorumental comprosrise t contret control system contret room 1 ability b .w~ =n . m w- + w - _ _, _ _. -. _ _ _ _. _. -

o s . Page no. 17 06/07/90 SOLEWOID-OPERATED VALVE Fi.ILURE DATA DOC PLAuf EVENT LER NO. OF FAILED SYSTEM ManufACT MODEL ROOT REP CORGECT*VE CapelEWTS REFTWENCE TP/ FC NO. NAME DATE NupWER FAILURES PART

1. 0.

CAUSE FL ACTION DO M WTS IRFT 321 Match 1 03/18/87 87-005 Two

1. Missing Containment ASCO WP8321 Unspecified Yes 1. Instelled a 2 dumper LEt No SS tock nut ventitetion missing lock failures. (1 85-015-01

2. Stuck nut./ 2. no caused by plunger corrective missing tock action taken nut on SUV,1 on stuck SOV caused by stuck because it SUV plunger) tested ehey sthsegsent to failure.

322 Shoreham 11/15/89 89-009 0 Conteirment ASCO 206-832 Design error, Reorient SOVs Cassion-mode no 08 isolation SOVs were to correct f ailures having (Rx building 206-380 oriented positions potentist to stoney incorrectly (vertiest vs. prevent ventitation) horizontal) fulfitteent of safety f metion 323 Diablo Canyon 2 08/14/85 85-019-01 Three Incorrect Main Steam Not

1. ene Personnet Yes Replaced SUV Undetected SOV LER 85-014 no 07 wiring te (MSIV)

Specified error (incorree failure caused SOV t tsidocumented 5 amnth loss of wiring change) 1 train of ESFAS actuation of MStVs 323 Diablo Canyon 2 12/21/85 85-022 One Open Feedwater

1. ot Not Isproper No The wiring 50V is e tER Yes 09 circuit specified Speciff wirino connection was pieceport of 275/85-030 ed instettation property the fury and tweed reterimineted itsiction box other simiter S0Ws' terminations were inspected.

124 Brunswick 2 09/27'J5 85-006 Three Dise-to-s Main steam ASCO 8323 u,. - twi, so Replaced SOVs Casuman-mede mene no 12 eet (MSIV) water and high feitures. See sticking temperatures Section 5.2.3.1 caused of this report. dagradetion of seet materiet. l

4 Page No. 18 06/07/90 SOLEN 0ID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED STsTEM MMANACT 8RBEL ROOT REP CORRECTIVE COIWEENTs mEFEWEWCE TP/ FC No. NAME DATE NUMBER FAILURES PART No. CAUSE FL ACTION 000DEEWTs (Rff 324 Brunswick 2 10/15/85 85-011-01 Two DC colt noin steem ASCO uP8323 Licensee me septoced S0vs. None mone Yes 91 (MSIV) suspected Entenstwe chloride failure corrosion-onetysis initfeted. 324 Brunswick 2 01/02/88 38-001-05 Four Failed to Contelevamt AsCD stitt ts der Yes Reptoce Sovs. Four previous Yes 19 shift isol./drywel investigation. Performing simiter t floor and Fomd debris extensive feitures had e<gsat drain and ett film fatture been stsups on one 50V. enetysis experienced suspect high tegeratures from self heeting of energized SOVs 324 Brunswick 2 06/17/89 89-009-01 One Failed to Drywett ASCO Not suspected that so Reptoced SUV Extensive No 12 shift purge and speciff foreign enetysis of wsnt ed perticutetes root cou!r" was found in the not tot.ely S0W had cenet Alve attacked etestomeric ports of the S0W 325 Brunswick 1 02/28/87 87-005-02 Two Discs conteirunent vetcor v52645-not specified so Reptoced S0Ws 50V teekage mone No 03 isolation 5683-14 foted during LLRT 325 Brunswick 1 07/01/87 87-019 one stuck noin steen Target 1/2-SMS Encess Lectite Yes Refurbished see section LER to 17 plunger (MSRV) Rock -A-01 used by S0r 5.2.2.2 of this 87-020-01 manufacturer's report field rep 325 Brunswick 1 07/03/87 87-020-01 Four stuck Main steen Terpet 1/2-SRs Excess Lectite No Reptoced S0Ws see section LER 87-019 no IT pitmger (MSRV) Rock -A-01 used by 5.2.2.2 of this manufacturer's report field rep i s v, -,c- .y, y .e

a Page No. 19 06/07/90 SOLEDDID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT ~ LER NO. OF FAILED SYSTEtt 9tANUFACT 91WEL ROOT REP ComeECTIVE COIWIENTS REFEWEWCE TP/ FC WO. NAME DATE IRDe9ER TAILURES PART NO. CAUSE FL ACTION 000ustEWTS SUT 327 Secysoyah 1 05/18/84 87-020 Wot Specified not mot Wet tot Desipi error No Plant 1E S0Ws were teone to 14 Specified Specified Specified Specifi sedifcetions not protected ed to protect from water vulnerobte 1E sprey which equipment could emenete from pipes which were vutnerable to en SSE 328 Sequoyah 2 08/30/84 84-014-02 One Seet Feedwater ASCO 8320 Design Error so Reptoced SOW An incorrectly mone No 13 teskage selected 50V

• ailed eAen put in service where its POPD timits were exceeded 328 Sequoysh 2 06/11/88 88-026-01 Two incorrect Aunitiary not mot insecysete Yes Reconnected Incorrect mene No 07 externet feedwater Specified Specifi meintenance SOVs correctly externet wiring wiring levet ed configuration to 2 S0Ws control controt 328 S m ysh 2 06/06/88 88-027-01 mot Auxiliary not mot inedequete Yes Reptoced Isone soone suo 07 Specified feedwater specified Specifi electrical diodes arissing ed maintenance freur exterrel circuitry connecting 2 SOWS 331 Duane Arnold C1/10/84 84-004 Two Blockage St an*y ASCO 8316 Foreign-Air peth so 12 of filtration meteriet in eteened intemet instrummt air possegewe Y

331 Duane Arnold 01/28/85 85-002-00 One Diophrsga Nigh Skimer L2D8515 End of No Reptoced SOV #ene None llo IT pressure Electric 0 tife/encessive coolent time between injection maintenance

Page No. 20 06/07/90 SOLEWO!D-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER WO. OF FAILED SYSTEM puuluFACT MODEL ROOT REP CIIRRECTIVE COBUIEWTS REFERENCE TP/ FC NO. NAME DATE NLp"*e FAILUWES PART NO. CAUSE FL ACTION 00CtpENTS OUT 331 Duere Arnold C5/27/88 88-005 One not Fire Electro-M 2010008 Design error No Replaced Sov Licenece had mene no 14 Specified Sw pression anuet 3 ed inedecFiete sqmgraded SOW (Chemetro post with an n Corp.) meintenance incorrect one. testing Deficiency was not found daring post meintenance testing. 331 Duane Arnold 03/05/89 89-008 one Coit Main steam ASCO NP8323 Moisture no Reptaced S0t. 7 other similar Tes 11 (Mstv) intrusion from Tightened Sovs were steem teak / enclosure subject to inedeQJ8te Covers of Solsture to w ing of c'her similar intrusion enclosure SOVs. foiture dse to festeners c_.4 j torqueing deficiency 333 Fitzpatrick 08/20/85 85-022 One Electrice Main steam ASCO mot Maintenance No S0Ws replaced AC colt had mone Yes 0 7 t fault (MstV) Specifi persomet and rewired been connected ed error in correctly to DC source esternet and DC colt had wiring been connected to AC source 333 Fitzpatrick 11/22/85 85-027-01 One Sov Main steam ASCO uP8323 grass stiver no Cleaned /refurb MstV unable to mone no 12 enable to (MSIV) dse to cross ished SOW close seet threeding air check other property line fitting for sleiter prohtem 333 Fitzpatrick 08/03/89 89-013 None containment Design error Correct wiring no 07 isolation error 334 Beaver Valley 1 06/07/88 88-007 One not Dieset Johnson mot mot specified we septoced SOW EDG air stort mene no 22 Specified generator Speciff Sov failed air start-ed 336 Mittston= 2 12/31/86 86-021 Two Broken Reector Veteor VS26-60 Suspe t no Reptoced 1T-7 Prior to event mene so el springs Cootant Head Engg 42-3A hydrogen PM springs of these SOWS had in SUVs Vant Corp. embrittIcomt alt simiter been ieeking Vatcor SOWS and had been isolated O g,- -m +,., = -

e Page No. 21 06/07,M SOLEN 0!D* OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER No. OF FAILED SYSTEM MANUFACT M(BEL ROOT - REP CURRECTIVE CtpWEENTS REFEREeICE TP/ FC NO. NAME DATE IUEBER FAILURES PART NO. CAUSE FL ACTION DOCUMENTS iRff 336 Mittstone 2 01/02/87 87-002 One Diaphrope main ASCO 8262 Not specified Yes inspected and Yes 82 teskage feeduster replaced' (FURV) 338 North Anna 1 02/02/84 84-005 6 failed and 54 Electrica Containment Valcor volcor Inadequate Replaced 6 SOVS failed 09 incipients t isolation and ASCO 526wri coneksit faited SOVs and 54 Scvs (moisture ) hydrogen es seating and seated att were instatled intrusion centrot/ pass methods did deficient incorrectly in ) ) not meet afrs conthsit seats both units specs to meet IEEE-324 qualifications 338 North Anna 1 11/23/87 87-020 Two Not Main Steam Copes-Vut Not mot specified No Water To prevent mone No 02~ Specified (Atmosg*eric can Specifi induction recurrence of Dump Valves) ed circuits were this type de-energized event, en in order to evaluation to start the install condensate additional ptmps and levet switches begin will be secondary perforwed. system recovery actions. 338 North Ar.w 1 01/08/88 88-002 One Not Condenser Not Not mot Specified Yes Replaced SOV None None Yes 21 Specified waterbox Specified Specift vacutse ed 338 North Anno 1 03/11/88 88-011 Nine Stuggish Containment ASCO NP-1 Design error Yes Rewortred SUVs Failure to LER No 14 operation isotation series to meet foitow 339/87 manufacturer's manufacturer's 01 instructions installation instructions modified the 50Vs' performance and gietification. ~

Pege No. 22 06/07/90 SOLEW0!D-OPERATED VALVE FAILtRE DATA DOC PLANT EVENT LER WO. OF FAILED SYSTEM .MANUFACT MtBEL ROOT WEP CORRECT!WE CEp8ENTS REFENEWCE TP/ FC NO. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION DottmENTS OUT 338 North Anno 1 03/15/88 88-012 One Not component ASCO Wot Not Specified 'Yes SOY from None LER 88-011 No 02 Specified Cooting Speciff 1-CC-TV-1034 Water ed esos instetted en 1-CC-TV-1035, and the S0W ' from 1-CC-TV-1038 ises refurbished and instetted en 1-CC-TV-103A 338 North Anna 1 07/19/89 89-014 1 0-ring TV-miat Parker-Ma 95tFN16M 0-ring pinched No Reptoce 0-ring Supplementet LER 88-013 Yes 03 or (EMC) mefin X0834 dJring SOW info obtained refurbishment from Iicensee ty turbine 5/16/90, M.L. manufacturer's Ornstein/ maintenance C.W. Atten tese 344 Trojen 04/16/87 87-009 Not Reactor Not Not Design /instatt No Reptoced None None No 28 Specified coolant Specified Speciff ation error sptices ediich (PORV) ed did not meet EQ instettetlen requirements 346 Davis-8 esse 09/11/84 84-013-01 One Not Mein steau controt Not Not specified Yes Reptoce or S0W is a None No 21 Specified (Atmos #teric Component specif f refurbish SOV piece-part of Vent) Internati ed the atmospheric enet vent vetwe's air-eperated controtter 346 Davis-Besse 01/03/86 86-006-01 Thirty tieo Colt Not ASCO Not Failure to Reptoced 50V Colts on EO None No 17 incipients specified specifI perfora coiis S0Ws had been ed preventive in service maintenance beyond their ishen required quellfied Iifetime i O r = .c m

e Page No. 23 06/07/90 SOLEm0!D-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER NO. OF FAILED SYSTEN MANUFACT R) DEL WOOT REP CORRECTIVE CON S TS REFEREsM TP/ FC - Wo. NAME DATE WL918ER FAILURES PART NO. CAUSE FL ACTION 00QpIENTS (R7T ' 346 Davis-sesse 12/07/87 87-015 One SOV Instrument ASCO 1179237 set specified wo eeplaced sov, Feiture of S0W mone Yes 21 instrtament ofr caused loss of vented air dryer dryers . Instrument air reptoced with air / reactor terreded ones trip. 0-rings on severet 50Vs in turbine bypass systese s'.-o found degraded 348 Farley 1 01/18/87 87-005 Two Not Conteirus-t ASCD 8316 Unknown no 1 sov closed Redundant sovs mone wo 20 on additionet in one Specified isolation (conteirunent ettewpts. penetration inboerd SOV to feited to close stmp be inspected discharge) subsequent to shutdown. 348 Farley 1 07/21/87 87-012 84 incipients et Inodaquet mot mot Not

1. cot cause of no Att accessible 84 sovs et each mone no 28 each unit e

specified specified Specift inadequate 50vs'instattet unit were found electrica ed splices and ions modified not to be terminations to en approved instetted in i instel1. not stated E0 splice and accordance with termination EO requirements (sptices/ configuration (splices ord terminets on a priority junction bon ) basis. connectierus) 352 Limerici-1 05/09/88 88-017 One teskoge Reector Bldg ASCD 8316 Not Specified No Reptoced S0W Licensee could Wone No 20 not detensine -stug ventitetton cause of 50v stuck in fatture. mid-posit Cetled e ion ccaponent a feiture of te* noun cause" 352 Limerick 1 03/14/89 89-019 0 Electrice RK tw.sfiding Desi p error seated Potentist for no 07 1 ventitation (EC). electricot 4:omeon-mode failure /o Inadequate conduits fat tures oisture conduit intrusion seating for MELB potentist envirorueent -

M.. m 1 Page No. '24 06/07/90 SOLEN 0!D-OPERATED VALVE FAILURE CAT' 00C PLANT EVENT LER NO. OF FAILED SYSTEM MANLt 4T MODEL ROOT REP CORRECTIVE CIBOIENT% REFERENCE TP/ FC bO. NAME DATE NURDER FAILURES PART NO. CAUSE FL ACTION 000mENTS Otf7 354 nope Creek 08/28/86 86-063 12 incipients Not Conteirament ASCO mP8316 Design error No Replaced ett Fellure of-None No 13 Specified Atmosphere tuetwe Sovs non-e Control with ones regulators having a could have higher MOPD caused failures rating. of the 50Vs. 354 Mope Crack 02/24/87 87-018-01 One Failed to Mein Steam Automatic Not Foreign No Replaced Foreign LER No 03 shift (MSIV) Vetwe Specifi meteriet failed SOW and meterlat in 87-037,038 Corp. ed inside SOW its manifold SOV, Plunger in (AVC)

body, assembly.

50V not per manutecturing Replaced 7 design-defect, and Sovs for other (incorrect inadequate MSIVs. Sent length), instattation failed SOV to mounting screws supplier (GE) on jwetion box for anstysis were tcose. 354 Hope Creek 10/10/87 87-M7 One Faited to Main stems Target Not Inadequate No ?** Fellure caused mone no 12 shift (MSRV) Rock-Specifi protection of notfunctioning by intrusion of ed MSRVs during SRV and its santbtesting plant SOV piece-port grit which was construction were reptoced used dring in kind. ptant constructten 361 San onofre 2 J1/09/86 86-004 Two Colt Feeduster Not Not moisture No The velwes mene mone Yes 11 specified Specifi intrusion - were reptoced ed faulty conduit and visuet connection inspections mode of the conthsit connections of simiter SOWS 361 San Onofre 2 12/17/87 87-031-01 One corrosion Main Marotta KV233C Inodequete Yes Reptoced Sov, Weter and LER Yes 12 of power Feedweter Scientifi / maintence terminet fJreign 206/86-004 teeds and MFIV) e MW238C instructions block,and meteriet terminet Controts power teeds. Intrusion block Inc. Seeted conesit (iradequetely cormeetions meeted conduit proper 1y. comection) 9 .-~- m. - -, ~ =-.e- - - - ~ - - - - - - - - ~ - -

~ y '2 s Page ko. ' 25 06/07/90 SOLEN 0!D-OPERATED VALVE FAILURE DATA - 3.. DOC PLANT EVENT L2R NO. OF FAILED SYSTEM MANU7ACT MODEL. ROCT REP CORRrCT!WE C3mENTS REFERENCE' f*/ FC NO. NAME. DATE NUMBER FAILURES PART NO. CAUSE FL ACTION' 000 DENTS OUT 366 Natch 2 09/21/84 84-021 One casket Main Steam ASCO - Not Not Specified No Replaced None . No.) Yes 03 (MS!?). ' Specifi gasket ed 366 Natch 2 01/20/88 88-004 Numerous Leakage Contairunent Target 75F-009 Inadegiate No Reverse See Section LER No 08 ' isolation Rock /7567F instructions / orientation of 5.1.4 of this-366/86-020 nonnat use and . many SOV:/ report (many. systems; wear.. replace failed o-rings ~366 Hatch 2 02/12/88 88-007 Twelve Not contairunent Target 73K-001 Inadeg; ate No Reversed See Section None No 08 Specified Isotation - Rock /75F-00 instructions / orientation /fo 5.1.4 of this Torus 9 design r unit one report DrywetI deficiency installed vacutan stronger Bresker. springs 368 ANO 2 04/24/87 87-003 Two Seat Reactor mot Not Seat teakage No Replaced SOV Concern for mone No 03 teakage Coolant Specified Specifi and instatted teek causing (pressurizer ed a cettector corrosion high point for any future damage to other vent) teekage components 368 ANO 2 04/29/85 88-001 2 Leakage containment Target 80E-001 sackwards Reinstatted See section No 08 isotation Rock instattation SOVs in 5.1.4 of this (pass) dJe to reversed report for inadequate orientation additional info instattation Instructions 368 ANO 2 02/16/89 89-003 0 Conteirunent Target 74F Design error-Refurbished valve had No 14 ' isolation Rock incorrect 50V. Checked exceeded EQ (hydrogen assessment of others for life 6 years anatyrer SOV similar design prior to tife-failure error discovery of sanpting). to account problem for heatup due to - energiration 369 McGuire 1 07/23/84 84-023 One Seat Main sorg met Pydraulic No Adjusted SOV Wane None Yes 03 deformati Feedwater Warner Specifi fluid was and modified ed leaking system on 6m

t: Page No. 26 06/07/90 SOLEN 0!D-OPERATED VALVE FAILURE DATA-DOC PLANT EVENT LER NO. OF FAILED SYSTEM' MANLfFACT MODEL ROOT WEP CORRECTIv* CDeqENTS REFERENCE TP/ FC-NO. N#9E DATE NUMBER' FAILURES PART NO. CAUSE. FL ACTION 00CIMNTS Otff 369 McGuire 1 09/19/85 85-028 One plus three cable. Post Valcor 526-529 Personnet - No. Att four. Simitar vetves-None-No 11 incipients terminati accident 5-45 error vetves were checked at Unit e-sa mling (instattation

repelred, 2, and fourd to seating not performed resented.

be okay - per Wiring on att iristat tation other volcor specification) 526 series SOVs at station to be tegraded and seats replaced 369 McGuire 1 04/15/87 87-009 One System Main turbine Not Not Modification No change System None Yes 00: perturbat Specified Specift of design and maintenance operation logic ion ed meintenance schedute to and time of avoid testing preventive white at maintenance had power. beenchanged. Both factors conti-ibuted to a reactor trip. 3 70 McGuire 2 06/24/85 85-018-01 Two (of the same Coit and Main Borg-Wern Not 1-colt No 1-replaced Second failure None Yes 01 SOV) short feedwater er Specifi failure - not SOV. 2-dritd occurred prior circuit ed specifled. 2-water from to co m tete short circuit

SOV, instetlation of

- water spray electrical bon replacement SOV ento open electrical box es cuire 2 08/37/86 86-017 One Colt Main Sorg N >t Not specified Yes SOV colt was None LER Yes 01 370 e Feedwater Werner Specifi reptoced and 85-018-01 ed originst coil was sent t6 the manufacturer for anetysis. 373 LaSatte 1 08/29/84 84-051 One SOV (3 Electrica Main steam Crosby IC-2 Cause of short No Reptaced SOY Caused SRV to None No 11 matfunctions) i ground (MSRV) Valve to mid not lift three specified tiers -.d.,- ..m m ,,y. -.L -m#-mr.. - - -. -, - -n-.a m- --e--

e _; X 'Page No. 27 '~ 06/07/90 SOLENOID-OPERATED VALVE FAlltAFE DATA DOC PLANT 1 EVENT LER NO. OF FAILED SYSTEM _ 'MANOFACT MODEL ROOT RE't CORRECTIVE ColeqENTS IIEFERENCE TP/ FC No. NAME 'DATE NUMBER FAILURES PART NO. CA'USE F. ACTION DOCUMENTS OUT 313 LaSatte 1 02/02/85 85-006 Four Diaphragm Reactor ASCO 8316 Diaphraspas Yes Rebuilt Sovs,.Witt change No 03 s building lost their cycling SOWS to nucteer ventitation resitlence frequency to quatifled be increased NP8316 modet 373 taSatts 1 03/12/87 87-013 Six incipients Not ' Main Steam Not Not Nigh dryvest No Anatyre - Three 50Vs None. No ' 10 Specified (MSRV) Specified Specifi tenversture effects of declared ed high drywell inoperable. tenperature Three SOVs suspect due to -high local tenperatures 3 74 LaSatle 2 06/08/84 84-033 One plus marty Passagewa Contalryent ASCO 206-832 Suv was Repositioned other similarly No 08 incipients y blocked isolation improperty SOY affected SOVs positioned were-repositioned or.. replaced 374 Lasatte 2 '11/20/84 f34-076 One Colt Turbine Not not Jmetion box No Replaced SOY None None No 11 . Steam Bypass Specified Specifi was fatt of ed water of unknown origin 374 tasatte 2 07/31/86 86-013 None - Many Electrica CRD, RCS ASCO See Design error Yes Repaired att 1E egsipment LER M-012 No 28 irtipients i

recire, comunent affected used ~

corw:ectio RCIC, s' electrical unqualifled ns. se v! e terminations electrical water, ftoor to meet cormections. drain, air qualification Soy model nos. regJirements Nv4-206 NP206, NP-8320 NP-8323 374 LaSatte 2 01/17/87 87-002 One leakage Feedwater Valcor V52660- Root cause of Yes Refurbished Sov body and N3ne No 12 -5292-16 corrosion, Sov stem corroded, dirt and Sov fitted with o-ring dirt, and deformation o-ring was not stated deformed

m ~ Page No. 28 ~ 06/07/90~ SOLENOID OPERATED 'WLLVE FAILURE DATA DOC PLANT EVENT ~ LER NO.' 0F FAILED SYSTEM MANUFACT MODEL ROOT REP CORRECTIVE C0pWENTS REFERENCE. TP/ FC

NO.'

NAME-DATE NUleER FAILURES PART NO. CAUSE FL ACTION D0QWENTS OUT 382 Waterford 12/11/87 87-028 one SOV Main Steam Fluid 17WKP477 Not Specified No Replaced SUV SOV felled None. Yes 05 " stuck-(MSIV) Controt 4-600K8 -during testing. open" Inc. 65 LER noted previous w reteted SOV feiture due to open coil. 387 Susquehanna 1 02/25/84 84-010 One SOV Main steen Not Not Not Specified No Reptoced SOV SOV stuck open None Yes 05 " stuck (MSRV) Specified Specifi causing SRV to open" ed remain open 387 Susquehanna 1 06/13/84 84-044 Several

Discs, Control Rod ASCO HV-176-Contamination Yes Refurbished See Section None No 12 repetetive seats Drive 816 of the air SOVs, sporeded 5.2.3.3 of this failures system and disc material report etevated from tesperatures polyurethene to Viton 387 Susquehanna 1 07/06/87 87-023 One

. Colt Containment Circle Not. "surned open" Yes Reptaced coit open colt found None No 01 Vacuum Seat Speciff coil on same vacuum Relief Controts ed breaker in 10/82. A unit 2~ vecuum breaker etso had a simiter Cirete Seet SOV coit failure in 4/87 387 Susquehanno 1 02/04/89 89-006 Three " Mechanic Suppression Circle Root cause Yes Reptoced one 50V failed, LER 87-023 Yes 19 atty chamber Seat onetysis feited SOV and however two bound" dryvett planned but eight simiter simiter SOWS vacuum not couplete ones had "probtensa i breaker yet (" problems" not specified) 388 Susquehanna 2 01/10/87 87-001 Two Not Reactor ASCO Not Not Specified No Reptoced SOV None None Yes 02. Specifled Butiding SpecifI Chitted ed Water O s ..s. - *~-, m-~ .., - ~, r, .g~-, .--3. ~,. +,. e-i. O ,y

-~ 'Page No.

29

06/07/90 c

SOLENOID-OPERATED VALVE FAILURE DATA ~ DOC PLANT' EVENT LER NO. OF FAILED SYSTEM MANUFACT MNEL ROOT REP CORRECTIVE CONENTS REFERENCE TP/ FC No. NMIE DATE NUMBER FAILURES PART NO. CAUSE FL ACTION DOCtpENTS OUT 386 Susquehanno 2 02/27/89 89-003 .One-Conteirament ASCO Yes Reptoced SOW Licensee shut LER 84-036 No 21 isolation down ptant (recirculati instead of en pump continuing chitled operation at water reduced ',mwer per tera specs 389 St. Lucie 2 08/16/89 89-006 One Not Nydrogen Valcor 52600-5 Not specified Ne staced Sov No 21 specified sampling 15 '395 Stsoner 06/29/86 86-011 One Electric Feedwater Not Not-Oxidation of-No,- strical Nr* None Yes 07 connector (FWIV) Specified Specifi connector pins +.snector and ed SOV were reptaced. 395 Supuner 12/02/88 88-012-01 None many Ground Main Steam ASCO Not Design No Isolated SOV Foisid that None No ' 14 incipients faults and Specifi deficiency contacts to grossid feutts Feeduater ed prevent could cause spurious spurious SOV octuations actuations 395 Stsoner 02/17/89 89-003-01 None, 3 Electrica Main steam Incorrectly No Modified Conunon-mode LER 88-012 No 07 incipients I (MStV) ' designed wiring failure grounding isolation potentlet for relay att 3 MSIVs 397 WNP 2 03/22/8? 84-027-02 Fifteen Grossid Main steam Not Not SOV Yts Replaced Events at WMP LER No 14 - faults (MSRV) Specified Specifi susceptibility defective occurred during 84-027-01 ed to spurious SOVs. Testeci startup actuation due potentietty testing. to ground af fected SUVs. Common-1 mode faults Voltage spike felture' stgression potentist. diodes were Previous instelled on similar events att MSRV+ ADS et La Sette + SOVs Susquehanna 397 WNP 2 67/23/85 85-050 Two fattures (1 Diaphragm Fire Not Not Root cause of Mo 1-Replaced None None No 08 SOV) / seat protection Spectfled specifI diaphragm diaphreyst/volv tenkage ed teekage not e seat. 2-specified. beckverds sockwerds bonnet bonnet due to " repaired" inedequate maintenance = . ~: _, _ = - a w - - --c

s + . Page No.. 30 - 06/07/90 SOLEN 010-OPERATED VALVE FAILtNtE DATA DOC PLANT EVENT LER- . No. OF. FAILED SYSTEM-INusuTACT MODEL ROOT REP CORRECTIVE CGOENTS REFERENCE TP/ FC 4 -No. NAME DATE ' NUMBER FAILURES PART NO. CAUSE FL ACTION DOCUNENTS Otff ' 400 Shearon Marris 1 02/08/88 88-006. Two Failed to Emergency Target 790-024 source of ' Yes The falted '. rm-sede None No 12 close ' service Rock debris 50Vs were falture ' water puup occumulation repelred.' No effecting both seat water not specified statement mode trains of supply about actions Emergency taken for Service Water reamvel of debris or prevention of stkfitional debris-400 Shearon Harris 1 05/13/88 88-012 Two Failed to Emergency Target 790-024 Debris in Yes Repelred Sovs 14 shift service Rock water and blocked water seat off source of water supply debris 400 Shearon Harris 1 09/09/88 88-026 Eleven or more Internet Containment Target Eleven Manufacturing No Unquetified consnon-mode None No 06 ' / reed isolation Rock models deficiency parts of 1E failure - switch (many harsh env. potential for wiring systems) Sovs reptoced 1E SOVs for with quotified harsh ones. environments. Corrective SOVs for action for ex-containment non-hersh env. also deficient. S0Ws not spectfled. 409 La Crosse 12/03/84 84-022 One seat Isolation ASCO 8210 Not specified Yes Replaced SOY None None No 03 teskage Condenser 409 La Crosse 04/20/85 85-008 One Colt Control Rod Royet Not Not Specified Yes Replaced 50V None LER 81-13 YES 01 Drive Industrie Specifi s ed 409 La Crosse 05/17/85 85-012 One seat control Rod Royet Not Root cause of Yes Reptoced 50v Wone None - Yes 12 Drive Industrie Specift metal chip in - s ed SOV seat not specified 4 s ' L r - ~ 8 --~--# r-AN w e <~N

,I Page No. -31 06/07/90 SOLENOID-OPERATED VALVE FAILURE DATA DOC PLANT EVENT LER -No. OF FAILED SYSTEM MANUFACT MODEL ROOT REP CORRECTIVE Co mENTS'- REFERENCE TP/ FC .NO. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION DOCIDENTS ;, GUT 409 La Crosse 07/08/86 86-020 One Colt control Rod Royet Not Uncertain, Yes Replaced SOV There have been LER 85-08.Yes 01 Drive Industrie Speciff water 7 previous s ed intrusion or scrans due to random colt the scram faiture solenoid suspected shorting out. 409 La Crosse 07/19/86 86-024 One Electrica Reactor ASCO 8300 Persomet No Replaced SOV ESFAS . None No 11 -t short cavity error-actuation, ventitation sptas.hed warer cascading event on SOV 409 ta Crosse 12/09/3% B5-036-01 One Colt Control Rod Royet Not Unorrtain, Yes Replaced There have been LER Yes 18 Drive Industrie Sp m rt egeing or. several SUVs. 8 previous 85-05,86-0 s. ed moisture Rectacement of scrans due to 20 intrusion SOVs will be these SOV suspected included in fattures. -SOV CROM that failed was preventive about 20 years maintenance old. program 410 kine Mile Pt 2 06/22/88 83-025 Ntsnerous Hydreutic Feedwater Keane 33896 Foreign object No Replaced SUV, SOV is None Yes 03 internal parts Controt in SOV, due to also reptoced piece-part of Urit manufacturing simiter SOVs level control deficiency or in other valve failure to . trains because instatt fitter of serious screen 'degradetion of their internets 414 Catawba 2 10/11/86 86-045 One Failed to AFW (steam SOV Reconnected Sov failure No 06 - shift odnission to incorrectly SUV property defeated manuet turbine) installed per stort an incorrect

• apability of design drawing AFW turbine 416 Grand Gutf 1 02/10/85 85-007-02 Three Core-plus Main Steam ASCO 8323 FUSS No Reptoced ett 8 See section None Yes 05 nut (MSIV)

MSIV Sovs 5.2.4.4 sticking a b l B .~ ,y. ,y m. _-rg . - -.,.,, ~.,., -., ~ y ,y,

at .e: a Page No.' 32 06/07/90 SOLENDID-OPERATED VALVE FAILURE DATA DOC' PLANT EVENT 'LER NO. OF FAILED SYSTEM MANUFACT-MODEL ROOT REP CORRECTIVE CapOEENTS RLFERENCE TP/ FC NO. NAME DATE NUMBER FAILURES 'PART NO. CAUSE FL ACTION DOCLNENTS (Rff 416 Gram Gulf 1 09/25/85 85-038-01 One Colt Drywell ASCO 8320 Encessive No Failed SOV Licensee stated None' No 11 equipment corrosion replaced with that the SOV. drain within the a duplicate. did not need to -coil housing be believed to be environmentalty caused by seated water which ~ entered during. plant construction 416 Grand Gulf 1 07/30/86 86-026-01 One Colt control Rod ASCO 105D602 Particulate No Replaced SOV, Particutete None No 12 Drive SP1 accumuletion system filters accumulation on the valve to be checked resulted in an seating and sanpted inovertent surface for control rod porticulates withdrawat 416 Grand Gutf 1 01/08/87 87-001 One SOV Offges ASCO 8320 Not specified No Not specified Modified system None No ' 00 - failed in sampling - specific mid-posit actions taken ion regarding SOV not stated 416 Grand Gutf 1 03/15/88 88-010. One Loose Control Rod ASCO Not cause of toose No The loose Licensee to None Yes 07 terminst Speciff connection not terminst evetuste design box eo found connection was change to connectio cteened & improve n to SOVs tightened. rettability of Other 50V power Leeds terminst connections checked, att were okey 423 Millstone 3 09/06/86 86-051 Not Specified " Failed Feedwater Not Not Intere'ttent No Att local None None Yes 01' electrica Specified Specifi open cirevit, terminations (ty" ed root cause on the SOV

unknown, wiring to be suspect checked for vibration and tightness steem during the ispingement next shutdown.

from a packing teak u

t . o: O. ~ Page No. 33-06/07/90' ' SOLENOID-OPERATED VALVE FAILURE DATA r DOC PLANT EVEN LER NO.'0F' ' FAILED SYSTEM' MANUFACT. MODEL ROOT REP CORRECTIVE COIUEENTS REFERENCE TP/FC$ NO. NAME DATE . NUMBER FAILURES PART - NO. CAUSE ' FL ACTION DOCtNENTS. OUT i. 423 Mittstone 3 03/07/87 87-008- ' One Colt feedwater Skinner V5H6620 Cause for open Yes Replaced 50V, SOV was LER 86-051 Yes 00, (open 0 circuit not operating circuit) specified within its " design life" 423 Mittstone 3 05/06/87 87-024 One SOV would Emergency -Circle N2990-9 Not specified No Failed air . Failed SOV None No : 20 not shift dieset Seat 617 start SOV and resulted in within generator the dieset's slow (out of spec air start redundant SOV spec) LDG were rectaced starting time with new ones 423 Mittstone 3 09/23/87 87-034 One Colt reedwater Skinner V5H6620 Root cause of Yes Reptoced SOV SUV controts LER Yes 01 Electric 0 colt failure hydrautie oit 87-08/86-0 (open circuit) flow to FWIV 51 not determined. [ Coit was within its " qualified life" 424 Vogtte 1 01/22/87 87-002 Eight incipients Potential Main Steam Keane Not Design error No Instatted a Potential for None No 13 for MOPD specifi retief vaive common-mode ed on each MOP 0 failures hydraulic due to heatup system to of hydraulic limit pressure fluid. See to below MOP 0 Section 5.1.3 timits of this report. 424 Vogtte 1 04/24/88 88-013 One Colt Feedwater Skinner V5H6559 Coit burnout No Replas M SOV SOV is a Mone No 01 Electric 0 and . iter piece-part of SOV o other A0V centrolling tras; of FWlV FWiv - control system 440 Perry 06/30/86 86-030 One Seat Containment ASCO 8320 Dust from No Replaced SOV None mone No 12 teskage Vesset and instrument air Drywell prevented Purge proper valve seating ~,

'F = Page No. 34 06/07/90 50LEN0!D-OPERATED VALVE FAILURE DATA DOC' PLANT EVENT LER NO. OF FAILEO SYSTEM MANUFACT MODEL-ROOT REP CORRECTIVE CopOIENTS REFERENCE -TP/ FC NO. NAME-DATE NUpWER FAILURES PART NO. CAUSE 'FL ACTION DOCUMENTS OUT _.. 440 Perry 02/27/87 87-009 Two Air: -Emergency Ntauphrey ~ TOG 2E1-F5flureduetoYesReplacedtoth Simultaneous !None No ' 17 temkage Oleset Products 3-10-35 entended Sovs.' commen-sode 'I (through. Generator service with Returned felture of both' elastomer Control Air high local fatted SOVs to diesets.' Delay-ic parts) temperatures EDG in repairing and continuous manufacturer teaking SOVsc energization. for anetysis. contributed. SOVs in svc 10 Will segrade See Section years arvi preventive this report never had PM. meintenance and elastomers 44C Perry 10/29/87 87-073-01 Five SOVs on two Elastomer Main steam ASCO NP8323 Heat and Yes Replaced or Comenon-mode _. Insp Rpt - Yes 10 occasions ' ic seats, (MSIV) moisture from refurbished failures. See 87-024

discs, steam teaks SOVs Section 5.1.1.1-etc of this report for additionet information 440 Perry 03/10/88 38-010 One Core Auxiliary ASCO 8320 Inadequate No Replaced SOV. Failu e of SUV None.

No 17' shaft Building (no) Instituted a results in toss wear -Ventitetion preventive preventive of RWCU room meintenance meintenance cooting for this SOV program (reptoce when tygrade to fait). valve replace those had been in SOVs every 2 service for years over 5 years 440 Perry 02/03/89 89-004 One Auxiliary ASCO 8320 Yes Replaced SOV ticensee ' LER 88-010 No 19 butIding investiseting ventitation root cause 456 Braidwood 1 09/15/89 89-010 One Colt Containment vetcor v526-53 Colt leads No Replaced with Also replaced 5 - No 09 ' Isolation 95-1 tabeled - different other simitar (hydrogen backwards modet SOV SOVs. Licensee anetyrer) investigating source of mistabeting (manufacturer vs. plant) k n ~ .u.. +

~ Page No. -35 ' 06/07/90 SOLENDID-OPERATED VALVE FAILURE DATA ~ ' DOC PLANT. EVENT LER NO. OF FAILED-SYSTEM MANUFACT MODEL ROOT . REP CORRECTIVE ^ COWNTS - . REFERENCE TP/ FC1 NO. NAME DATE NUMBER FAILURES PART NO. CAUSE FL ACTION DOCUMENTS OUT-458 River send 05/02/89 89-022-Affected . Target 77kk-01 eackuerds Yes 30Vs See section LER 89-024 No' 14.. ~ many Rock 3 instattation reinstatted in 5.1.4.1 for systems. &e to. reverse additionet see cwnent inadequate orientation details instattation instructions 458 R;ver Bend 04/06/89 89-024 0 Affected Target 77KK-01 Backwards Yes Reversed Potential LER 89-022 No 08 many Rock 3 instattation - orient.4 tion of connon-mode -systems. design error. SOVs failures. 6 See conment inedequate Sovs had the instattation same instructions. .instatistion defIclency. See section 5.1.4.1 of this report for info 461' Clinton 03/06/87 87-009 One SOV - Fuel Not Not Not Specified No Replaced SOV None None No: 03 failed in Building Specified Spectif mid Ventitation ed position 461 Clinton 04/14/89 89-019 Electrica Main steam Seitz Design error Instatt heet Failed to meet'. ~ No 08-1 (MSIV) (EQ). shrink tubing ' EQ instattation connectio Inadequate per EQ requirements ns electrical requirements corriector seating 461 Clinton 11/29/89 89-037 One 0-rings Vacuum GPE LD240-4 Inedequate No Refurbished No scheduled No 03 relief Controts 20 preventive : SOV, replaced preventive (SOV (GPE) maintenance 0-rings maintenance unspec1fI program. ed) . Failure - discovered during stroke testing 483 Cattaway 01/02/85 85-001 One Not Feedwater Not Not' Licensee. Yes Replaced SOY SOV is a - None Yes 00 Specified specified Speciff considered-piece-part of. ~ ed this to be a FWIV hydraulic random failure operator s ',,a e.n-- s, n s- ~ .~, N e- ~ -v-g ~

-.en

-n .~.n.

q.. p a 2:s.- _;.- 7:3; e. ~

Page No.'

36 .-2 06/07/90: ~ ~ ..'SOLENDID-OPERATED VALVE. FAILURE DATA-DOC PLANT EVENT-LER N0; 0F ' FAILED SYSTEM MANUFACT fe0 DEL. ROOT REP CURRECTIVE CofGENTS - NEFERENCE 2TP/ FC;- i_ No. NAME DATE' NUMBER-FAILURES. .PART NO. CAUSE FL ACTION 00c5EENTS. GUT'.,e _.l.. ; - 483 Callaeasy ' 02/20/86 86-002-01 None .Etectrica Reactor head Not not ' Construction Yes Not Specified on 2 occastens None: No 28 1 vent and Specifled SpecifI and sterttp - tIcensee found - connector chemicat- ' ed program s . volume deficiencies . It had not instatteo: controt ~ environmentatty". ~ - qualified - connectors on t SOVs as respired (3 Sovs) SM Pato Verde 1 08/08/85 85-052 .Two'or more potential Post Airmatic ' Not Design error No Affected SOVs ~ SOVs control _ None'- ho 14 - ':incipients insulatio accident Specifi were shletded air-operated n. sampling ed to reduce post sample flees breakdown accident control valves / shorts radiation - to ground i a d 4 e. e y ,e- . ~.. - +, _ _. ~, - e.---g',, -n. - y

,,,t.,. ,g s 'l , L' t. l V e o,: ..1 4 4 ' APPENDIX B 1 DISPOSITION OF ASCO DUAL-COIL 8323 SOVs-USED FOR MSIV CONTROL i s 'I 1 4 0 4! i r 2 ? t j :- 'i!;. iy ,30 2 s E'-- -I ___m__. _. _--. _ __ _ _ _ _ _ _ _ _ _ _. _ _ _.__ _ _ _ _ _ _ _.,

4! I APPENDIX B Disposition of ASCO Dual-Coil 8323 SOVs Used for MSIV Control. Many plants have experienced problems with ASCO dual-coil 8323 SOVs which have been used for MSIV control. Several examples are provided in Chapter 5. ASCO issued two field notifications (Refs. 106,107) addressing NP8323 SOVs. The notifications stated that the NP8323 SOVs have no defects, and that their mal-functions were primarily caused by foreign materials, aggrevated by adverse service conditions. Furthermore, because ASCO does not envision significant changes in the service conditions that the NP8323 SOVs are subjected to, ASCO is phasing out the sale of those valves. As an alternative, ASCO recommends the use of a pair of single-coil NP8320 SOVs. Two NP8320 SOVs can be configured to perform the function of one NP8323. Because of the NP8320 50V's single-coil construction, ASCO anticipates that they will perform more satisfactorily than '3 the NP8323 SOVs under ad"erse service conditions. -In anticipation of ASCO's discontinuance of the.NP8323 SOVs, the MSIV air pack' manufacturer'(R. A. Hiller Company) has initiated a program to select a suitable replacement of the ASC) NP8323 S0Vs.* The Hiller company has assembled five MSIV i air packs for baseline testing. The S0Vs to be tested in the MSIV air packs are: 2 ASCO: NP8320 V (2 valves configured as recommended by ASCO in Refs. -102,103). .AVC:C4964 i Target Rock: fSMS-S02(modified) ~ 4 Valcor: V70900-87V Zeiss: 629-60007 (assembly) GE and Hiller Company have noted that all of the American SOVs are 1E qualified; and that although the Zeiss assembly is not 1E qualified, it has been used successfully in Europe. It should be noted that the choice of a replacement for the NP8323 SOVs can affect the qualification of the overall MSIV air packs (e.g. seismic / dynamic loading). Final selection of replacements for the NP8323 S0V should address this issue. In the past, GE was actively involved in the qualification testing of MSIV air packs which were used at many plants. GE has indicated that as a result of ASCO's discontinuance of NP8323 S0Vs they are trying to interest BWR owners to l support a consolidated effort with the Hiller Company to qualify MSIV air packs having suitable replacements for the ASCO NP83?3.**

• Telephone discussion between J. Nanci, R. A. Hiller Company, and H. L.

Ornstein, USNRC, December 8, 1989.

• • Telephone discussion between C. Nieh, GE, and H. L.

Ornstein, USNRC, December 1989. L PRELIMINARY CASE STUDY B-1

a.. ] i.. f" r f .g'.- 'I!..

. 0

-

/

I !.) - ] i {p .., e ; 4 APPENDIX C GENERIC COMUNICATIONS ON SOVs s}r dh NL f i a- ,l 1 - i ~ 1 .:l

l l

l \\ -l 1 ou . m g r-w -4

o 7 -( APPENDIX C Generic Communications on SOVs Bulletin Number Date Title Bulletin 75 March 14,.1975 Incorrect Lower Disc Spring and Clearance Dimension in 8300 and 8302 ASCO Solenoid Valves Bulletin 78-14 December 19, 1978 Deterioration of Buna-N Components in ASCO Solenoids Bulletin 79-01A June 6, 1979 Environmental Qualification of Class 1E Equipment (Deficiencies in the Environmental Qualification of ASCO Solenoid Valves) Bulletin 80-14 June 12, 1980-Degradation of BWR Scram Discharge Volume capability l 4 Bulletin 80 July 3, 1980 Failure of 76 of 185 Control Rods to Fully Insert During a Scram at a BWR Bulletin 80-17 July 18, 1980 Failure of 76 of 185 Control Rods Supplement 1 to Fully Insert During a Scram at a BWR Bulletin 80-17 July 22, 1980 Failures Revealed by Testing Subse-L Supplement 2 quent to failure of Control-Rods L to Insert During a Scram at-a BWR Bulletin 80-23 November 14, 1980 Failures of Solenoid Valves Manu-factured by Valcor Engineering Corporation Bulletin 80-25 ' December.19, 1980 Operating Problems with' Target p j Rock Safety Relief Valves at BWRs l-L l 1 PRELIMINARY CASE STUDY C-1 L l L

C Information Notice'Humber Date-Title Information Notice 80-11 March 14, 1980 Generic Problems with ASCO Valves in Nuclear Applica-i tions Including Fire J Protection Systems ll Information Notice 80-39 October 31, 1980 Malfunction of Solenoid ~ Valves Manufactured by-Valcor Engineering Corporation Information Notice 80-40 November 7, 1980 Excessive Nitrogen Supply 3 Pressure Actuates Safety-Relief Valve Operation to Cause i Reactor Depressurization. l 'Information Notice 81-29 September 24, 1981 Equipment.Quantification k Testing Experience, Equip-ment Qualification Notice No. 1 Information Notice 81-38 December 17, 1981 Potentially Significant Equipment Failures Resulting i from Contamination _of Air-Operated Systems Information Notice 82-52 December 21, 1982 Equipment. Environmental Qualification Testing Expe-rience - Updating of Test Summaries Previously Published in IN 81-29 Information Notice 83-57 August 31,'1983 Potential Misassembly Problem a with Automatic Switch Company 1 (ASCO) Solenoid Valve Model NP 8316 Information Notice 84-23 April 15, 1984 Results of NRC Sponsored Qualification Methodology Research Test on ASCO Solenoid Valves-Information Notice 84-53 July 5, 1984 Information'Concerning the Use of Loctite 242 and Other-Anaerobic Adhesive Sealants Information Notice 84-68 August 21, 1984 Potential Deficiency in Improp-erly Rated Field Wiring to Solenoid Valves ' PRELIMINARY CASE STUDY C-2 1 (

E L so

Information' Notice Number hte Title

IhformationNotice85-08 January 30, 1985 Industry Experience on Certain.

L Materials Used in Safety-5 Related Equipment j Information Notice 85-17 March 1, 1985 Possible Sticking of ASCO Solenoid Valves Information Notice 85-17 October 1, 1985 Possible Sticking of ASCO ' I Supplement 1 Solenoid Valves Information Notice 85-47 June 18, 1985 Potential Effect of Line-l Induced Vibration on Certain Target Rock Solenoid-Operated ~ Valves Information Notice 85-95 December 23, 1985 Leak of Reactor Building-Caused by~ Scram Solenoid Valve Problem Information Notice 86-57 July 11, 1986 Operating Problems with Solenoid Operated Valves at Nuclear Power Plants Information Notice 86-72 August 19, 1986 Failure of 17-7 PH Stain-less Steel Springs in Valcor Valves Due to Hydrogen Embrittlement 'Information Notice 86-78 September 2, 1986 Scram Solenoid Pilot Valve (SSPV) Rebuild Kit Problems Information Notice 87-48 October 9, 1987 Information Concerning the Use of Anaerobic Adhesive / Sealants Information Notice 88-24 May 13, 1988 Failures of Air-Operated Valves Affecting Safety-Related Systems Information S tice 88-43 June 23, 1988 Solenoid Valve Problems Information Notice 88-51 July 21, 1988 Failure of Main Steam Isolation Valves Information Notice 88-86 March 31, 1989 Operating with Multiple Supplement 1 Grounds in Direct Current Distribution Systems PRELIMINARY CASE STUDY C-3 r e..-

l e t .Infornation 3, A Wumber Date Title y Information Notice 89-30 March 15, 1989 High Temperature Environ-ments-at Nuclear Power i Plants i Information Notice 89-66 September 11, 1989 Qualification Life of Solenoid Valves 1 ~Information Notice 90-11 February 28, 1990 Maintenance Deficiency .- i Associated with Solenoid-Operated Valves j -1 Circular Number Date Title-Circular 81-14 November 5, 1981 Main Steam Isolation Valve Failures to Close \\ i \\ 1 1 PRELIMINARY CASE STUDY C-4 _ - - - - _ - -.}}