ML17329A683
| ML17329A683 | |
| Person / Time | |
|---|---|
| Site: | Cook  |
| Issue date: | 10/31/1992 |
| From: | Gore B, Isom J, Lloyd R, Moffitt N, Vo T Battelle Memorial Institute, PACIFIC NORTHWEST NATION, NRC |
| To: | Office of Nuclear Reactor Regulation |
| References | |
| CON-FIN-L-1310 NUREG-CR-5832, PNL-7782, NUDOCS 9211240215 | |
| Download: ML17329A683 (32) | |
Text
92X1ke02>5 ~V<03Z l
PDR 'DQCK 050003l'5 8
PDR j-~~ r NUREG/CR-5832 PNL-7782 Au&liamFeedwater System
. Risk-Based Inspection Guide for the D.C. Cook Nuclear Power Plant Manuscript Completed: September 1992 Date Published: October 1992 Prepared by R. C. Lloyd, N. E. Moffitt,B. F. Gore, T. V. Vo. J. A.
Isom'aciTic Northwest Laboratory Richland, WA 99352 Prepared for Division of Radiation Protection and Emergency Preparedness Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Washington, DC 20555 NRC FIN L1310
'U.S. Nuclear Regulatory Commission
Abstract In a study sponsored by the U.S. Nuclear Regulatoiy Commission (NRC), PaciTic Northwest Laboratory has developed and applied a methodology for deriving plant-specific risk-based inspection guidance for the auxiliary feedwater (AFW)system at pressurized water reactors that have not undergone probabilistic risk assessment (PRA). This methodology uses existing PRA results and plant operating experience information. Existing PRA-based inspection guidance information recently developed for the NRC forvarious plants was used to identify generic component fail-ure modes. This information was then combined with plant-speciTic and industry-wide component information and failure data to identifyfailure modes and failure mechanisms for the AFWsystem at the selected plants. D. G Cook was selected as one ofa series ofplants forstudy. The product ofthis effort is a prioritized listing ofAFWfailures which have occurred at the plant and at other PWRs. This listing is intended foruse by NRC inspectors in the preparation ofinspection plans addressing AFWrisk-important components at the D. G Cook plant.
Contents Abstract Summary.
1 Introduction 2
D. C. Cook AFW.
2.1 2.1 System Description.
2.2 Success Criterion 2.3 System Dependencies 2.4 Operational Constraints 3
Inspection Guidance for the D. GCook AFWSystem 3.1 Risk Important AFWComponents and Failures Modes 3.1.1 MultiplePump Failures Due to Common Cause 3.1.2 'Ibrbine Driven Pump Fails to Start or Run.
3.1.3 Motor Driven Pump "E"or "W"Fails to Start or Run 3.1.4 Pump Unavailable Due to Maintenance or Surveillance.
3.1.5 AirOperated Control Valves Fail 3.1.6 Motor Operated Isolation Valves Fail Closed 3.1.7 Manual Suction or Discharge Valves Fail Closed 3.1.8 Leakage ofHot Feedwater through Check Valves 2.1 2.2 2.2 2.2 3.1 3.1 3.1
~
~
~
~
~
32 3.3 3.4 3.4 3.5 3.5 3.6 3.2 Risk Important AFWSystem Walkdown lhble 3.6 4
Generic Risk Insights From PRAs 4.1 4.1 Risk Important Accident Sequences InvolvingAFWSystem Failure 4.1 4.1.1 Loss ofPower System 4.1.2 1lansient-Caused Reactor or 'Ibrbine tulip.
4.1.3 Loss ofMain Feedwater 4.1.4 Stcam Generator 'Ibbe Rupture (SGTR).
4.1 4.1 4.1 4.1 4.2 Risk Important Component Failure Modes.
5 Failure Modes Determined From Operating Experience 5.1 D. C Cook Experience 5.1.1 MultiplePump Failures 5.1.2 Motor Driven Pump Failures 4.1 5.1 5.1 5.1 5.1 NUREG/CR-5832
5.1.3 'Ibrbine Driven Pump Failures 5.1.4 Flow Control and Isolation Valve 5.1.5 Check Valve Failures 5.1.6 Human Errors 5.1 5.1 5.1 5.1 5.2 Industry Wide Experience 5.2 5.2.1 Common Cause Failures 5.2.2 Human Errors.
5.2.3 Design/Engineering Problems and Errors 5.2.4 Component Failures 5.2 5.4 5.4 5.5 6
References 6.1 NUREG/CR-5832
Figures 2.1 D. C. Cook auxiliary feedwater system 2.3
'Ihbles 3.1 Risk importance AFWsystem walkdown table................................................
3.7 vll NUREG/CR-5832
Summary This document presents a compilation ofauxiliary feedwater (AFW)system failure information which has been
~
screened for risk significance in terms offailure frequency and degradation ofsystem performance. It is a risk-priori-tized listing offailure events and their causes that are signiTicant enough to warrant consideration in inspection plann-ing at the D. C. Cook plant. This information is presented to provide inspectors with increased resources for inspec-t 'tion planning at D. C. Cook.
The risk importance ofvarious component failure modes was identified by analysis ofthe results ofprobabilistic risk assessments (PRAs) for many pressurized water reactors (PWRs). However, the component failure categories identi-fied in PRAs are rather broad, because the failure data used in the PRAs is an aggregate ofmany individuals failures having a variety ofroot causes.
In order to help inspectors focus on specific aspects ofcomponent operation, main-tenance and design which might cause these failures, an extensive review ofcomponent failure information was per-formed to identify and rank the root causes of these component failures. Both D. C, Cook and industry-wide failure information was analyzed. Failure causes were sorted on the basis offrequency ofoccurrence and seriousness ofcon-sequence, and categorized as common cause failures, human errors, design problems, or component failures.
This information is presented in the body ofthis document. Section 3.0 provides briefdescriptions of these risk-important failure causes, and Section 5.0 presents more extensive discussions, with specific examples and references.
The entries in the two sections are cross-referenced.
An abbreviated system walkdown table is presented in Section 3.2 which includes only components identiTied as risk important. This table lists the system lineup for normal, standby system operation.
I This information permits an inspector to concentrate on components important to the prevention ofcore damage.
However, it is important to note that inspections should not focus exclusively on these components.
Other com-ponents which perform essential functions, but which are not included because ofhigh reliabilityor redundancy, must also be addressed to ensure that degradation does not increase their failure probabilities, and hence their risk importances.
1 Introduction This document is one of a series providing plant-specific inspection guidance for auxiliary feedwater (AFW) sys-tems at pressurized water reactors (PWRs). This guid-ance is based on information from probabilistic risk assessments (PRAs) for similar PWRs, industry-wide operating experience with AFWsystems, plant-speciTic AFWsystem descriptions, and plant-specific operating experience. It is not a detailed inspection plan, but rather a compilation ofAFWsystem failure information which has been screened for risk signiTicance in terms of failure frequency and degradation ofsystem perfor-mance. The result is a risk-prioritizcd listing offailure events and the causes that are significant enough to warrant consideration in inspection planning at D. C.
Cook.
This inspection guidance is presented in Section 3.0, followinga description ofthe D. C. Cook AFWsystem in Section 2.0. Section 3.0 identifies the risk important system components by D. C. Cook identiTication num-ber, followed by briefdescriptions ofeach of the various failure causes ofthat component.
These include specific human errors, design deficiencies, and hardware fail-ures. The discussions also identify where common cause failures have affected multiple, redundant components.
These briefdiscussions identify speciTic aspects of system or component design, operation, maintenance, or testing for inspection by observation, records review, training observation, procedures review, or by obser-vation ofthe implementation ofprocedures.
An AFW system walkdown table identifying risk important com-ponents and their lineup for normal, standby system operation is also provided.
The remainder ofthe document describes and discusses the information used in compiling this inspection guid-ance. Section 4.0 describes the risk importance infor-mation which has been derived from PRAs and its sources.
As review ofthat section willshow, the failure events identiTicd in PRAs are rather broad (e.g., pump fails to start or run, valve fails closed). Section 5.0 addresses the speciTic failure causes which have been combined under these broad events.
AFWsystem operating history was studied to identify the various specific failures which have been aggregated into the PRA failure events. Section 5.1 presents a sum-mary ofD. C. Cook failure information, and Section 5.2 presents a review ofindustry-wide failure information.
The industry-wide information was compiled from a variety ofNRC sources, including AEOD analyses and reports, information notices, inspection and enforce-ment bulletins, and generic letters, and from a variety of INFO reports as well. Some Licensee Event Reports and NPRDS event descriptions were also reviewed indi-vidually. Finally, information was included from reports ofNRC-sponsored studies ofthe effects ofplant aging, which include quantitative analyses ofreported AFW system failures. This industry-wide information was then combined with the plant-speciTic failure infor-mation to identify the various root causes ofthe broad failure events used in PRAs, which are identiTicd in Section 3.0.
'2 D, C. CookAFW System This section presents an overview ofthe D. C. Cook AFWsystem, including a simpliflicdschematic system diagram. In addition, the system success criterion, system dependencies, and administrative operational constraints are also presented.
2.1 System Description
The AFWsystem provides feedwater to the steam generators (SG) to allow secondary-side heat removal from the primary system when main fccdwater is unavailable. The system is capable offunctioning for extended periods, which allows time to restore main feedwater flowor to proceed with an orderly cooldown of the plant to less than 350 degrees F from normal operating conditions in event oftotal loss ofoff-site power, to where the residual heat removal (RHR) sys-tem can remove decay heat. AsimpliTied schematic diagram ofthe D. C. Cook AFWsystem is shown in Fig-ure 2.1. AFWsystem valve numbers are the same in both units. However, itshould be noted that this is not the case for all other systems.
The system consists ofone Condensate Storage Tank (CST), two motor-driven (MD)AFW pumps, one turbine-driven (TD) AFW pump, associated piping, valves and instrumentation for each unit. The system is designed to start up and establish flowautomatically.
Allpumps start on receipt of a stcam generator low-low level signal or ATWS MitigationSystem Actuation Cir-cuitry (AMSAC). The motor-driven pumps start on low-lowlevel in one SG, whereas, two SG low-lowlevel signals are required for a turbine-driven pump start.
The motor-driven pumps also start for the following conditions: loss ofnormal voltage to the 4KVbus, trip ofboth fecdwater pumps, and a safety injection signal.
The turbine-driven pump also starts on an undervoltage condition on 2 of4 RCP buses.
The preferred source ofAFW pump suction is from each unit's CST. Acommon header supplies water to both the motor-driven and turbine-driven pumps through a sealed open isolation valve and a check valve to each pump. The CST for each unit can be cross connected through a normally closed, air operated valve (CRV 51). An additional back-up source ofwater for the AFW pumps is provided from the essential service
'ater system (ESW) through normally closed, motor operated isolation valves (WMO744, 753, 754). The down stream manual isolation valve must be opened before ESW is supplied through these valves.
Power, control, and instrumentation associated with each train are independent from each other. In addi-tion, each unit has its own battery system ("N"train) as an emergency electrical supply to ensure maximum reliabilityin any type ofplant emergency. Stcam for the turbine driven pump is supplied through MCM221 and 231 from steam generators 1 and 4, from a point up-stream ofthe main steam isolation valves. Each AFW pump is equipped with an emergency leakoff system which prevents pump deadheading.
The discharges ofthe motor-driven pumps for each unit are normally aligned so that the West "W"pump sup-plies the 1 and 4 steam generators and the East "E" pump supplies the 2 and 3 steam generators. Note that there is no possible cross-connection, for the discharges ofthe Eand W pumps in the same unit. However, the discharge piping for each unit's "E" pump contains a cross-connect valve (FW 129) which connects to the dis-charge piping of the opposite unit's "W"AFW pump.
Cross-tie operation is only to be used ifAFWflowcan' be achieved from the affected unit. The cross-connect valve is sealed closed and administratively controlled.
The turbine-driven pump feeds all four steam genera-tors, but through separate lines. Steam generator inlet isolation valves are sealed open manual valves and the flowdischarge isolation valves are motor operated. Each line also contains check valves to prevent leakage from the feedwater lines.
The Condensate Storage Tank (CST) has a 500,000 gal-lon capacity and is required to store a minimum of 175,000 gallons forAFWsystem use, to maintain the reactor coolant system (RCS) at hot standby for nine hours with steam discharge to atmosphere, followed by a cool down to 350 F.
2.1 NUREG/CR-5832
D. C. Cook AFW System 2,2 Success Criterion System success requires the operation ofat least one pump supplying rated flowto two steam generators.
2.3 System Dependencies The AFWsystem depends on ACpower for motor-driven pumps and some motor-operated isolation valves, DC power for control power to pumps, valves, and automatic actuation signals, and instrument air for AFWemergency leakoffvalves. Each air operated valve is designed to failin its safe condition on loss of instrument air. In addition, the turbine-driven pump also requires steam availability.
2.4 Operational Constraints When the reactor is in Modes 1,2, or 3, the D. C. Cook
'Ibchnical Specifications require that all three AFW pumps and associated flowpaths are operable with each motor-driven pump powered from a different emergency bus and the turbine-driven feedwater pump capable of being operated from an operable steam supply. Also, at least one auxiliary feedwater flowpath in support of the other unit's shutdown functions must be available. If one AFWpump becomes inoperable, it must be restored to operable status within72 hours or the plant must be shut down to hot standby within the next six hours and to hot shutdown within the followingsix hours. Iftwo AFWpumps are inoperable, the plant must be shut down to hot standby withinsix hours and to hot shutdown withinthe followingsix hours. With three AFWpumps inoperable, corrective action to restore at least one pump to operable status must be initiated immediately.
The D. C. Cook 1bchnical Spedfications requires a minimum volume of 175,000 gal. ofwater to be stored in the Condensate Storage Tank With thc CST inopera-ble, it must be restored to operable status within four hours or the plant must be shut down to hot shutdown within the next twelve hours. Ifthe essential service water system is demonstrated to bc operable, it may serve as a backup AFWsupply for seven days before plant shutdown is required.
NUREG/CR-5832 2.2
CONDEHSATK 81onAOK 1ANN DAFN FlI 159 10 OPPOSITE UNIT SNO 243 NO 1 l'-I I-I STN OEN C-259 ESW 243 545 Fnv FN FN 168 174 NF>>
I Fl I 1
4 132-4 8TH CEN ESSENTIAL SEAV ICE NATEA TO OPPOSITE DNIT S/6 184 Fll 153 rnv7 Fll 175 IIj 4 If8-4 NFII 12-cnv Fll 122 Fll 123 rll 131 2 t1 Fll 132-2 S Ttl C EH OPPOSITE UNIT CST fgLI L~"
DAFN 138X Ig 2 138-2 N
Fll tlFll 9/0 2 H
8/0 3 NCN 231 84 8-2 NS 88-3 TDAFIII Fll 136 FNO 232 ru tl Fll 131-3 132 3 I
N Rl 7-3 130 3 NFII S1t1 0CII Figure 2.1 'D.C. Cook auxTIiary feehvater system
3 Inspection Guidance for the 0; C. Cook A&VSystem In this section the risk important components ofthe D. C. Cook AFWsystem are identified, and the impor-tant failure modes for these components are brieflydes-cribed. These failure modes include specific human errors, design deflciencies, and types ofhardware fail-ures which have been observed to occur for these com-ponents, both at D. G Cook and at PWRs throughout the nuclear industry. The discussions also identify where common cause failures have affected multiple, redundant components.
These briefdiscussions identify speciTic aspects ofsystem or component design, opera-tion, maintenance, or testing for inspection activities.
These activities include: observation, records review, training observation, procedures review, or by obser-vation of the implementation of procedures.
Table 3.1 is an abbreviated AFWsystem walkdown table which identiTies risk-important components.
This table lists the system lineup for normal (standby) system operation. Inspection ofthe identified components dresses essentially all ofthe risk associated withAFW system operation.
3.1 Risk Important AFW Components and 5'ailure Modes On the basis ofgeneric PWR risk insights, the plants with similar design features indicated that common cause failures ofmultiple pumps are the most risk-important failure modes ofAFWsystem components.
These are followed in importance by single pump fail-ures, level control valve failures, and individual check valve leakage failures.
Thc followingsections address each of these failure modes, in decreasing order ofrisk-importance. They present the important root causes ofthese component failure modes which have been distilled from historical records.
Each item is keyed to discussions in Section 5.2 where additional information on historical events is presented.
3.1.1 MultiplePump Failures Due to Common Cause The followinglisting summarizes the most important multiple-pump failure modes identified in Section 5.2.1, Common Cause Failures, and each item is keyed to entries in that section.
~
Incorrect operator intervention into automatic sys-tem functioning, including improper manual start-ing and securing ofpumps, has caused failure ofall pumps, including overspeed trip on startup, and ina-bilityto restart prematurely secured pumps.
CC1.
Inspection Suggestion - Observe Abnormal and, Emergency Operating Procedure (AOP/EOP) simu-lator training exercises to verifythat the operators comply with procedures during observed evolutions.
Observe surveillance testing on the AFWsystem to verifyit is in strict compliance with the surveillance test procedure.
t
~
Valve mispositioning has caused failure ofall pumps. Pump suction, steam supply, and instru-ment isolation valves have been involved. CC2.
Inspection Suggestion - Verifythat the system valve alignment, air operated valve control and valve actuating air pressures are correct using 3.1 Walk-down ihble, the system operating procedures, and operator rounds logsheet. Review surveillance pro-cedures that alter the standby alignment ofthe AFWsystem. Ensure that an adequate return to normal section exists.
~
Steam binding has caused failure ofmultiple pumps.
This resulted from leakage ofhot fccdwater past check valves and motor operated valves into a com-mon discharge header. (Sce item 3.1.8 below.)
CC10. Multiple-pump steam binding has also resulted from improper valve lineups, and from run-ning a pump dcadheaded.
CC3.
3.1 NUREG/CR-5832
Inspection Guidance for the D. C Inspection Suggestion - Verifythat the pump dis-charge temperature is withinthe limits specified on the operator rounds logsheet. Assure any instru-ments used to verifythe temperature by the utility are ofan appropriate range and included in a calibration program. Verifyaffected pumps have been vented in accordance with procedures to ensure steam binding has not occurred. Verifythat a maintenance work request has been written to repair leaking check valves.
~
Pump control circuit deficiencie or design modi-fication errors have caused failures ofmultiple pumps to auto start, spurious pump trips during operation, and failures to restart after pump shut-down. CC4. Incorrect setpoints and control circuit calibrations have also prevented proper operation ofmultiple pumps. CC5.
Inspection Suggestion - Review design change implementation documents for the post mainte-nance testing required prior to returning the equip-ment to service. Assure the testing verifies that all potentially impacted functions operate correctly, and includes repeating any plant start-up or hot functional testing that may be affected by the design change.
~
Loss ofa vital power bus has failed both the turbine-driven and one motor-driven pump due to loss of control power to steam admission valves or to tur-bine controls, and to motor controls powered from the same bus. CC6. AtD.C. Cook, electrical power to the trip and throttle valve which opens to start the TDAFWpump, is powered by the N train bat-tery system, making such failure unlikely.
Inspection Suggestion - The material condition of the electrical equipment is an indicator ofprobable reliability. Review the Preventative Maintenance (PM) records to assure the equipment is maintained on an appropriate frequency for the environment it is in and that the PM's are actually being performed as required by the program. Review the outstanding Corrective Maintenance records to assure the deficiencies found on the equipment are promptly corrected.
~
Simultaneous startup ofmultiple pumps has caused oscillations ofpump suction pressure causing multiple-pump trips on lowsuction pressure, despite the existence ofadequate static net positive suction head (NPSH). CC7. Design reviews'have identifled inadequately sized suction piping which could have yielded insufficient NPSH to support operation ofmore than one pump. CC8. At D. C. Cook, a low pressure suction trip rendered a motor driven pump inoperable while the turbine driven pump was out ofservice for testing.
Inspection Suggestion - Assure that plant condi-tions which could result in the blockage or degrada-tion ofthe suction flowpath are addressed by sys-tem maintenance and test procedures.
Examples include, ifthe AFWsystem has an emergency source from a water system with the potential for bio-foul-ing, then the system should be periodically treated to prevent buildup and routinely tested to assure an adequate flowcan be achieved to support operation ofall pumps, or inspected to assure that bio-fouling is not occurring. Design changes that affect the suc-tion flowpath should repeat testing that-verified an adequate suction source forsimultaneous operation ofall pumps. Verifythat testing has, at sometime, demonstrated simultaneous operation ofall pumps.
Verifythat surveillances adequately test all aspects ofthe system design functions, for example, demon-strate that the AFWpumps willtrip on lowsuction pressure.
3.1.2 1brbine Driven Pump Fails to Start or Run
~
Improperly adjusted and inadequately maintained turbine governors have caused pump failures. HE2.
Problems include worn or loosened nuts, set screws, linkages or cable connections, oflleaks and/or con-tamination, and electrical failures ofresistors, tran-sistors, diodes and circuit cards, and erroneous grounds and connections.
CFS. Improperly ad-justed governors have occurred at D. C. Cook.
Inspection Suggestion - Review PM records to assure the governor oil is being replaced within the designated frequency. During plant walkdowns NUREG/CR-5832 3.2
Inspection Guidance for the D. G carefully inspect the governor and linkages for loose fasteners, leaks, and unsecured or degraded conduit.
Review vendor manuals to ensure PM procedures are performed according to manufacturer's recom-mendations and good maintenance practices.
~
'ibrbines with Woodward Model PG-PL governors have tripped on overspeed when restarted shortly after shutdown, unless an operator has locally exercised the speed setting knob to drain oilfrom the governor speed setting cylinder (per procedure).
Automatic oildump valves are now available through ibrry. DE4. Overspeed trip events have occurred at D. G Cook Inspection Suggestion - Observe the operation of the turbine driven Aux Feed pump and assure that the governor is reset as directed in STP.17T. Assure the turbine is not coasting over, which can result in refillofthe speed setting cylinder.
~
Condensate slugs in steam lines have caused turbine overspeed trip on startup.
Ibsts repeated right after such a trip may failto indicate the problem due to warming and clearing ofthe steam lines. Surveil-lance should exercise all steam supply connections.
DE2.
Inspection Suggestion - Verifythat the steam traps are valved in on the stcam supply line. For steam traps that are on a pressurized portion ofthe steam line, check the steam trap temperature (ifunlagged) to assure it is warmer than ambient (otherwise it may be stuck or have a plugged line). Ifthe stcam trap discharge is visible, assure there is evidence of liquid discharge.
~
'll'ipand throttle valve (VIV)problems which have failed the turbine driven pump include physically bumping it, failure to reset itfollowingtesting, and failures to verifycontrol room indication ofreset.
HE2. Whether either thc ovcrspced trip or VIV trip can be reset without resetting the other, indi-cation in the control room ofTIVposition, and unambiguous local indication ofan overspeed trip affect the likelihood of these errors. DE3. TTV problems have occurred at D. G Cook.
Inspection Suggestion - Carefully inspect the ITV overspeed trip linkage and assure it is reset and in good physical condition. Assure that there is a good steam isolation to the turbine, otherwise continued turbine high temperature can result in degradation ofthe oil in the turbine, interfering with proper ovcrspeed trip operation. Review PM records to ensure that the TTVhas been adequately lubricated to prevent binding (LER 1983-101). Review train-ing procedures to ensure operator training on reset-ting the TIVis current.
~
Design and/or calibration errors on systems can void pump run out protection. This happened at the D. C. Cook plant when run out protection was lost due to an improperly calibrated flowretention orifice. 'IItis problem had not been solved at the time ofpublication ofthis document.
Inspection Suggestion - Review LER 1989-017 to ensure corrective actions have been properly imple-mented.
Observe the TDAFWpump surveillance test to verify test line flowand process flowindi-cation agree. Verifyflowretention actuation sct-point is set where proper actuation willoccur.
3.1B MotorDriven Pump "E"or '%P Fails to Start or Run
~
Control circuits used for automatic and manual pump starting are an important cause ofmotor driven pump failures, as are circuit breaker failures.
CF7. Control circuit failures have prevented auto-matic pump starts at D. C. Cook.
Inspection Suggestion - Review corrective mainte-nance records when control circuit problems occur to determine ifa trend exists. Every time a breaker is racked in a PMTshould be performed to start the pump, assuring no control circuit problems have occurred as a result ofthe manipulation ofthe breaker. (Control circuit stabs have to make up upon racking the breaker, as well as cell switch dam-age can occur upon removal and reinstallation of the breaker.)
3.3 NUREG/CR-5832
Inspection Guidance for the D. C.
~
Mispositioning ofhandswitches and procedural deficiencies have prevented automatic pump start.
HE3. Mispositioning ofhandswitchcs has occurred at D. C. Cook.
Inspection Suggestion - Confirm switch position using Table 3.1. Review administrative procedures concerning documentation ofprocedural deficien-cies. Ensure operator training on procedural changes is current.
3.1.4 Pump Unavailable Due to Maintenance or Surveillance valve listed for each train is a normally closed AOVin the AFWpump test flowline. These valves failclosed on loss ofinstrument air or loss ofpower.
~
Control circuit problems have been a primary cause offailures, both at D. G Cook and elsewhere.
CF9.
Valve failures have resulted from blown fuses, failure ofcontrol components (such as current/
pneumatic convertors), broken or dirty contacts, misaligned or broken limitswitches, control power loss, and calibration problems. Degraded operation has also resulted from improper air pressure due to
'ir regulator failure or leaking air lines.
~
Both scheduled and unscheduled maintenance re-move pumps from operability. Surveillance requires operation with an altered line-up, although a pump train may not be declared inoperable during testing.
Prompt scheduling and performance ofmainte-,
nance and surveillance minimize this unavailability.
Inspection Suggestion - Review the time the AFW system and components are inoperable.
Assure all maintenance is being performed that can be per-formed during a single outage time frame, avoiding multiple equipment outages.
The maintenance should be scheduled before the routine surveillance test, so credit can be taken for both post mainte-nance testing and surveillance testing, avoiding excessive testing. Review surveillance schedule for frequency and adequacy to verifysystem operability requirements per Kchnical Specifications.
3.1.5 AirOperated Control Valves Fail TD Pum Agin: FRV-258256 MDPum "E"Train; FRV-257255 MD Pum "W"'Rain: FRV-247245 The first valve listed for each train is a normally-open air operated valve (AOV)that controls AFWpump emergency lcakoff(ELO) to the CST. They fail open on loss of Instrument Airor loss ofpower. The second Inspection Suggestion - Check for control air system alignment and air leaks during plant walkdowns.
(Regulators may have a small amount ofexternal bleed to maintain downstream pressure.)
Check for cleanliness and physical condition ofvisible circuit elements.
Review valve stroke time surveillance for adverse trends, especially those valves on reduced testing frequency. Review air system surveillances moisture content ofair is withinestablished limits.
~
Out-of-adjustment electrical flowcontrollers have caused improper valve operation, affecting multiple trains ofAFW. CC12.
Inspection Suggestion - Review PM frequency and records, only upon a trend offailure ofthe controllers.
~
Leakage ofhot feedwater through check valves has caused thermal binding offlowcontrol MOVs.
AOVs may be similarlysusceptible.
CF2.
Inspection Suggestion - Covered by 3.1.1 bullet 3.
~
Multipleflowcontrol valves have been plugged by clams when suction switched automatically to an alternate, untreated source.
CC9.
Inspection Suggestion - Covered by 3.1.1 bullet 6.
NUREG/CR-5832 3.4
Inspection Guidance for the D. C 3.1.6 Motor Operated Isolation, Valves Fail Closed MDPum Dischar eIsolation:
TDPum Dischar e Isolation:
FMO-211 221 231 241 Essential Service Water Suction Isolation:
WMO-744 754 753 Inspection Suggestion - Review the administrative controls fordocumenting and changing the settings ofthermal overload protective devices. Assure the information is available to the maintenance planners.
~
Out-of-adjustment electrical flowcontrollers have caused improper discharge valve operation, affect-ing multiple trains ofAFW. CC12.
These MOVs isolate flowto the steam generators and provide AFWpump suction isolation f'rom the ESW system. The discharge isolation valves are normally open and the essential service water suction valves are normally closed. They all failas-is on loss ofpower.
Failure ofmotor operated isolation valves at D.C. Cook causing AFWsystem failure is minimized by supplying power from the N-train battery system which is backed up by two chargers. The system is also designed so that valves would failin the correct operating position.
~
Common cause failure ofMOVs has occurred at D. C. Cook and elsewhere, from failure to use elec-trical signature tracing equipment to determine proper settings oftorque switch and torque switch bypass switches. Failure to calibrate switch settings for high torques necessary under ~desi n basis acci-dent conditions has also been involved. CC11.
Inspection Suggestion -Review PM frequency and records, only upon a trend offailure ofthe controllers.
~
Grease trapped in the torque switch spring pack of the operators ofMOVs has caused motor burnout or thermal overload trip by preventing torque switch actuation.
CF8.
Inspection Suggestion - Review this.only ifthe MOVtesting program reveals deficiencies in this area.
~
Manually reversing the direction ofmotion of operating MOVs has overloaded the motor circuit.
Operating procedures should provide cautions, and circuit designs may prevent reversal before each stroke is finished. DE7.
Inspection Suggestion - Review the MOVtest records to assure the testing and settings are based on dynamic system conditions. Overtorquing ofthc valve operator can result in valve damage such as cracking ofthe seat or disc. Review the program to assure overtorquing is identified and corrective actions are taken to assure valve operability follow-ing an overtorque condition. Review the program to assure EQ seals are renewed as required during the restoration from testing to maintain the EQ rat-ing of the MOV.
~
Space heaters designed for prcoperation storage have been found wired in parallel withvalve motors which had not been environmentally qualiTied with them present. DE8.
Inspection Suggestion - Spot check MOV's during MOVtesting to assure the space heaters are physi-cally removed or disconnected.
3.1.7 Manual Suction or Discharge Valves Fail Closed
~
Valve motors have been failed due to lack of, or improper sizing or use, ofthermal overload pro-tective devices.
Bypassing and oversizing should be based on proper engineering for ~desi n basis conditions. CF4.
TDPum 1?ain: FW-133136'137-1-2-3-4 MDPum "E" 1?ain: FW-123 130 131-2 131-3 MDPum "W"'I?ain: FW-162 158 131-1 131-4 3.5 NUREG/CR-5832
Inspection Guidance for the D. C.
These manual valves are normally locked open. For each train, closure ofthe firstvalves would block pump suction, closure ofthe second valves would block pump discharge and closure ofthe third set ofvalves would block discharge to the steam generators.
Valve mispositioning has resulted in failures of multiple trains ofAFW. CC2. Ithas also been the dominant cause ofproblems identified during operational readiness inspections. HE1. Events have occurred most often during maintenance, cali-bration, or system modifications. Important causes ofmispositioning include:
Failure to provide complete, clear, and specific procedures for tasks and system restoration Failure to promptly revise and validate procedures, training, and diagrams followingsystem modifications Failure to complete all steps in a procedure Failure to adequately review uncompleted procedural steps after task completion Failure to verifysupport functions after restoration Failure to adhere scrupulously to administrative procedures regarding tagging, control and tracking ofvalve operations Failure to log the manipulation ofsealed valves system restoration followingmaintenance, valve labeling, system drawing updating, and procedure revision, for proper implementation.
3.1.8 Leakage ofHot Feedwater through Check Valves MD Pum "E"'Ilain: FW-132-2 FW-132-3 FW-128 MDPum "W"'Rain: FW-132-1 FW-132-4 FW-159 TD Pum Bain: FW-138-1 3 A FW-135
~
Leakage ofhot feedwater through several check valves in series has caused steam binding ofmultiple pumps. Leakage through a closed level control valve in series with check valves has also occurred, as would be required for leakage to reach the motor driven or turbine driven pumps.
CC10.
Inspection Suggestion - Covered by 3.1.1 bullet 3.
~
Slow leakage past the final check valve of a series may not force the check valve closed. Other valves in series may leak similarly. Piping orientation and valve design are important factors in achieving true series protection. CF1. D. C. Cook has experienced check valve leakage.
Inspection Suggestion - Covered by 3.1.1 bullet 3.
3.2 Risk Important AFW System Walkdown Ihble Failure to followgood practices ofmitten task assignmcnt and feedback of task completion information Failure to provide easily read system drawings; legible valve labels corresponding to drawings and.
procedures, and labeled indications oflocal valve position H
Inspection Suggestion - Review the administrative controls that relate to valve positioning and sealing, Table 3.1 presents an AFWsystem walkdown table in-cluding only components identiTied as risk important.
This information allows inspectors to concentrate their efforts on components important to prevention ofcore damage.
However, it is essential to note that in-spections should not focus exclusively on these com-ponents.
Other components which perform essential functions, must also be addressed to ensure that their risk importances are not increased.
Examples include the (open) stcam lead stop check valves and ensuring an adequate water level in the CST.
NUREG/CR-5832 3.6
Inspection Guidance for the D. C.
Table 3.1 Risk importance AFWsystem walkdown table Component Number Component Name Required Position-losed Actual Position Electrical Motor Driven Pump Racked In/
Closed 12-CRV-51 Motor Driven Pump Unit 1-Unit 2 AUXFP Suction Supply Cross Tie Racked In/
Closed Closed FW-162 FW-122 FW-123 FW-133 WM0-744 WM0-754 WM0-753 ESW-243 ESW-145 ESW-240 FRV-247 FRV-257 FRV-258 AFPs Suet from CST "W"MDAFPSuet Isol CST to "W"MDAFP "E"MDAWPSuet Isol CST to TDAFP Suet Isol ESW Supply to "W"MDAFP ESW Supply to "E"MDAFP ESW Supply to TDAFP ESW Supply to "W"MDAFP ESW Supply to "E"MDAFP ESW Supply to TDAFP "W"MDAFP Emergency Leakoff Valve "E"MDAFP Emergency Leakoff Valve TDAFP Emergency Leakoff Valve Sealed Open Scaled Open Sealed Open Sealed Open Sealed Open Closed Closed Closed Locked Closed Locked Closed Locked Closed Auto/Open Auto/Open Auto/Open 3.7 NUREG/CR-5832
Inspection Guidance for the D. G Table 3,1 (Continued)
Component Number FW-174 FW-175 FW-127 FW-158 FW-130 FW-136 FW 131-1 Component Name "W"MDAFP Emergency LeakoffIsol "E"MDAFP Emergency LeakoffIsol TDAFP Emergency Leakoff Isol "W"MDAFP Disch Isol to S/G 1 &4 "E"MDAFP Disch Isol to S/G 2 &3 TDAFP Disch Isol "W"MDAFP Disch to S/G 1 Ctrl Inlet Isol Required Position-losed Locked Open Locked Open Locked Open Sealed Open Sealed Open Sealed Open Sealed Open Actual Position FW 131-4 "W"MDAFP Disch to S/G 4 Ctrl Inlet Isol Sealed Open FW 131-2 "E"MDAFP Disch to S/G 2 Ctrl Inlet Isol Sealed Open FW 131-3 FW 137-1 FW 137-4 FW 137-2 FW 137-3 FW-129 FMO-212 "E"MDAFP Disch to S/G 3 Ctrl Inlet Isol TDAFP Disch to S/G 1 Ctrl TDAFP Disch to S/G 4 Ctrl TDAFP Disch to S/G 2 Ctrl TDAFP Disch to S/G 3 Ctrl "E" MDAFP Cross-tie to Opposite Unit S/G 1 and 4 "W"MDAFP Flow to S/G 1 Sealed Open Sealed Open Inlet Isol Sealed Open Inlet Isol Sealed Open Inlet Isol Sealed Open Inlet Isol Sealed Closed Auto/Open NUREG/CR-5832 3.8
Inspection Guidance for the D C
'Ihble 3.1 (Continued)
Component Number FM0-242 FMO-222 FMO-232 FM0-211 FM0-241 FM0-221 FMO-231 MCM-221 MCM-231 FW 132-1 FW 132-2 FW 132-3 FW 132'W 138-1 FW 138-2 FW 138-3 FW 138-4 Component Name "W"MDAFP Flow to S/G 4 "E"MDAFP Flow to S/G 2 "E"MDAFP Flow to S/G 3 TDAFP Flow to S/G 1 TDAFP Flow to S/G 4 TDAFP Flow to S/G 2 TDAFP Flow to S/G 3 S/G 2 Mainsteam to TDAFP S/G 3 Mainsteam to TDAFP Piping Upstream ofCheck Valve Piping Upstream ofCheck Valve Piping Upstream of Check Valve Piping Upstream of Check Valve Piping Upstream of Check Valve Piping Upstream ofCheck Valve Piping Upstream of Check Valve Piping Upstream ofCheck Valve Required Position-losed Auto/Open Auto/Open Auto/Open Open Open Open Open Open Open Ambient Ambient Ambient Ambient Ambient Ambient Ambient Ambient Actual Position 3.9 NUREG/CR-5832
4 Generic Risk Insights From PRAs PRAs for 13 PWRs were analyzed to identifyrisk-important accident sequences involving loss ofAFW, to identify and risk-prioritize the component failure modes involved. The results ofthis analysis are described in this section. They are consistent with results reported'y INELand BNL(Gregg et al. 1988, and 1iavis et al.
1988).
4.13 Loss ofMain Feedwater
~
A fecdwater line brcak drains the common water source for MFWand AFW. The operators fail to provide feedwater from other sources, and failto initiate feed-and-bleed cooling, resulting in core damage.
4.1 Risk Important Accident Sequences InvolvingAIVV System Failure 4.1.1 Loss ofPower System
~
Aloss ofoffsite power is followed by failure of AFW. Due to lack ofactuating power, the power operated reliefvalves (PORVs) cannot be opened preventing adequate feed-and-bleed cooling, and resulting in core damage.
~
Astation blackout fails all AC power except VitalAC (Bus T21C) from DC invertors, and all decay heat removal systems except the turbine-driven AFWpump. AFWsubsequently fails due to battery depletion or hardware failures, resulting in core damage.
~
ADC bus fails, causing a trip and failure ofthe power conversion system. One AFWmotor-driven pump is failed by the bus loss, and the turbine-driven pump fails due to loss ofturbine or valve control power. AFWis subsequently lost completely duc to other failures. Feed-and-bleed cooling fails because PORV control is lost, resulting in core damage.
4.1.2 'Itansient-Caused Reactor or 'Ibrbine Trtp
~
Atransient-caused trip is followed by a loss ofthe power conversion system (PCS) and AFW. Fced-and-bleed cooling fails either due to failure ofthe operator to initiate it, or due to hardware failures, resulting in core damage.
~
A loss ofmain feedwater trips the plant, and AFW fails due to operator error and hardware failures.
The operators failto initiate feed-and-bleed cooling, resulting in core damage.
4.1.4 Steam Generator 'Ibbe Rupture (SGTR)
~
ASGTR is followed by failure ofAFW. Coolant is lost from the primary until the refueling water stor-age tank (RWST) is depleted. High pressure injec-tion (HPI) fails since recirculation cannot be estab-lished from the empty sump, and core damage results.
4.2 Risk Important Component Failure Modes The generic component failure modes identiTied from PRAanalyscs as important to AFWsystem failure are listed below in decreasing order ofrisk importance.
1.
Turbine-Driven Pump Failure or Start or Run.
2.
Motor-Driven Pump Failure to Start or Run.
3.
TDP or MDP Unavailable due to'ibst or Maintenance.
4.
AFW System Valve Failures
~
steam admission valves
~
trip and throttle valves 4.1 NUREG/CR-5832
Generic Risk
~
flowcontrol valves
~
pump discharge valves
~
pump suction valves
~
valves in testing or maintenance.
5.
Supply/Suction Sources
~
condensate storage tank stop valve
~
hot well inventory
~
suction valves In addition to individual hardware, circuit, or in-strument failures, each of these failure modes may result from common causes and human errors. Common cause failures ofAFWpumps are particularly risk important. Valve failures are somewhat less important due to the multiplicityofsteam generators and con-.
nection paths. Human errors ofgreatest risk impor-tance involve: failures to initiate or control system operation when required; failure to restore proper system lineup after maintenance or testing; and failure to switch to alternate sources when required.
NUREG/CR-5832 4.2
5 Failure Modes Determined From Operating Experience This section describes the primary root cause ofAFW system component failures, as determined from a review ofoperating histories at D. G-Cook and at other PWRs throughout the nuclear industry. Section 5.1 describes experience at D. G Cook, from 1981 to 1990. Some ap-
. plicable Licensee Event Report numbers (LERs) are included after each entry which inspectors may review.
Section 5.2 summarizes information compiled from a variety ofNRC sources, including AEOD analyses and reports, information notices, inspection and enforce-ment bulletins, and generic letters, and from a variety of INFO reports as well. Some Licensee Event Reports and NPRDS event descriptions were also reviewed.
Finally, information was included from reports ofNRC-sponsored studies ofthe effects ofplant aging, which include quantitative analysis ofAFWsystem failure reports. This information was used to identify the various root causes expected for the broad PRA-based failure events identified in Section 4.0, resulting in the inspection guidelines presented in Section 3.0.
5.1 D. C. Cook Experience The AFWsystem at D. C. Cook has experienced failures ofthe AFWpumps, pump Qow control and discharge isolation valves, turbine trip and throttle valves, essential service water backup supply valves, and numerous system check valves. Failure modes include electrical, instrumentation and control, hardware failures, and human errors.
5.1.1 MultiplePump Failures pump suction gauge, out ofbalance pump bearings, and worn pump seals. (LERs81-002, 83-055,83-100, 85-058) 5.19 'Ihrbine Driven Pump Failures Twelve events have occurred that have resulted in de-creased operational readiness or spurious starting ofthe turbine driven pump. Failure modes involved failures in instrumentation and control circuits, pump hardware failures, corrosion, mechanical wear, and human failures during maintenance activities. The fTVand associated linkage were the cause ofseveral ofthe TDAFP failures.
(80-003,80-017, 81-032,82-012, 84-004,84-019, 85-003,85-019, 85-048,85-065) 5.1.4 Flow Control and Isolation Valve Failures
'pproximately fortyevents have resulted in impaired operational readiness ofthe air operated emergency leakoff valves, motor operated flowcontrol valves, and motor operated isolation valves. Principal failure causes were equipment wear, corrosion, instrumentation and control circuit failures, valve hardware failures, and human errors. Valves have failed to operate properly due to blown fuses, failure ofcontrol components (such as I/P convertors), broken or dirtycontacts, misaligned or broken limitswitches, control power loss, and opera-tor calibration problems. Human errors have resulted in improper control circuit calibration, limitswitch adjust-ment, and connection to wrong phase power. (80-015,80-027, 81-002,81-004, 82-023,82-101, 83-101,84-020, 85-015)
One incident has occurred in which a MDAFP tripped while the turbine driven pump was out ofservice. The MDAFPwas placed in service within the allotted time period so that a reactor shutdown was not required.
(LER 85-058) 5.1.5 Check Valve Failures More than ten events ofcheck valve failure have occurred. The failure mode cited in all cases was normal wear and aging. (81-002,81-032, 81463)
'5.1.2 Motor Driven Pump Failures 5.1.6 Human Errors There have been six events that have resulted in failure ofthe motor driven pumps. Failure modes involved control circuit problems, circuit breaker problems, dirty There have been approximately fifteen events affecting the AFWsystem. Personnel have inadvertantly actuated 5.1 NUREG/CR-5832
Failure Modes the AFWpumps during testing, bumped switches, mis-used air tubes, and mispositioned control switches during operation. Both personnel error and inadequate procedures have been involved. Misunderstanding of operability requirements has resulted in equipment exceeding 'Ibchnical Specification limits. (80-024,82-087, 84416) 5.2 Industry Wide Experience Human errors, design/engineering problems and errors, and component failures are the primary root causes of AFWSystem failures identifiied in a review ofindustry wide system operating history. Common cause failures, which disable more than one train ofthis operationally redundant system, are highly risk significant, and can result from all ofthese causes.
This section identifies important common cause failure modes, and then provides a broader discussion of the single failure effects ofhuman errors, design/
engineering problems and errors, and component fail-ures. Paragraphs presenting details ofthese failure modes are coded (e.g., CC1) and cross-referenced by inspection items in Section 3.
5.2.1 Common Cause Failures The dominant cause ofAFWsystem multiple-train fail-ures has been human error. Design/engineering errors and component failures have been less I'requent, but nevertheless significant, causes ofmultiple train failures.
CC1. Human error in the form ofincorrect operator intervention into automatic AFWsystem functioning during transients resulted in the temporary loss ofall safety-grade AFWpumps during events at Davis Besse (NUREG-1154, 1985) and iiojan (AEOD/I'416, 1983).
In the Davis Besse event, improper manual initiation of the steam and fecdwater rupture control system (SFRCS) led to overspeed tripping ofboth turbine-driven AFWpumps, probably due to the introduction of condensate into the AFWturbines from the long, un-heated steam supply lines. (The system had never been tested with the abnormal, cross-connected steam supply lineup which resulted.) In the 11ojan event the operator incorrectly stopped both AFWpumps due to misinter-pretation ofMFWpump speed indication. The diesel driven pump would not restart due to a protective feature requiring complete shutdown, and the turbine-driven pump tripped on overspeed, requiring local reset ofthe trip and throttle valve. In cases where manual intervention is required during the early stages ofa transient, training should emphasize that actions should be performed methodically and deliberately to guard against such errors.
CC2. Valve mispositioning has accounted for a signifi-cant fraction ofthe human errors failingmultiple trains ofAFW. This includes closure ofnormally open suction valves or steam supply valves, and ofisolation valves to sensors having control functions. Incorrect handswitch positioning and inadequate temporary wiring changes have also prevented automatic starts ofmultiple pumps.
Factors identified in studies ofmispositioning errors include failure to add newly installed valves to valve checklists, weak administrative control oftagging, restoration, independent verification, and locked valve logging, and inadequate adherence to procedures.
Illeg-ible or confusing local valve labeling, and insufficient training in the determination ofvalve position may cause or mask mispositioning, and surveillance which does not exercise complete system functioning may not reveal mispositionings.
CC3. AtANO-2, both AFWpumps lost suction due to steam binding when they were lined up to both the CST and the hot startup/blowdown demineralizer effluent (AEOD/C404, 1984). AtZion-1 steam created by runn-ing the turbine-driven pump deadheaded for one minute caused trip ofa motor-driven pump sharing the same inlet header, as well as damage to the turbine-driven pump (Region 3 Morning Report, 1/17/90). Both events were caused by procedural inadequacies.
CC4. Design/engineering errors have accounted for a smaller, but significant fraction ofcommon cause fail-ures. Problems with control circuit design modifications at Farley defeated AFW pump auto-start on loss of main feedwater. At Zion-2, restart ofboth motor driven pumps was blocked by circuit failure to deencrgize when NUREG/CR-5832 5.2
Failure Modes the pumps had been tripped with an automatic start signal present (IN82-01, 1982). In addition, AFWcon-trol circuit design reviews at Salem and Indian Point have identified designs where failures ofa single com-ponent could have failed all or multiple pumps (IN87-34, 1987).
CC5. Incorrect setpoints and control circuit settings resulting from analysis errors and failures to update procedures have also prevented pump start and caused pumps to trip spuriously. Errors ofthis type may re-main undetected despite surveillance testing, unless surveillance tests model all types ofsystem initiation and operating conditions. Agreater fraction ofinstru-mentation and control circuit problems has been identi-fied during actual system operation (as opposed to sur-veillance testing) than for other types offailures.
CC6. On two occasions at a foreign plant, failure ofa balance-of-plant inverter caused failure oftwo AFW pumps. In addition to loss ofthe motor driven pump whose auxiliary start relay was powered by the invertor, the turbine driven pump tripped on overspeed because the governor valve opened, allowing fullsteam flowto the turbine. This illustrates the importance ofassessing the effects offailures ofbalance ofplant equipment which supports the operation ofcritical components.
The instrument air system is another example ofsuch a system.
CC7. MultipleAFWpump trips have occurred at Millstone-3, Cook-l, Zlojan and Zion-2 (IN87-53, 1987) caused by brief, low pressure oscillations of suction pressure during pump startup
. These oscilla-tions occurred despite the availabilityofadequate static NPSH. Corrective actions taken include: extending the time delay associated with the low prcssure trip, remov-ing the trip, and replacing the tripwith an alarm and operator action.
CC8. Design errors discovered during AFWsystem re-analysis at the Robinson plant (IN89-30, 1989) and at Millstone-1 resulted in the supply header from the CST being too small to provide adequate NPSH to the pumps ifmore than one of the three pumps were operating at rated flowconditions. This could lead to multiple pump failure due to cavitation. Subsequent reviews at Robinson identiTied a loss of feedwater transient in which inadequate NPSH and flows less than design values had occurred, but which were not recognized at the time. Event analysis and equipment trending, as well as surveillance testing which duplicates service conditions as much as is practical, can help iden-tifysuch design errors.
CC9. Asiatic clams caused failure of two AFWflow
'control valves at Catawba-2 when lowsuction pressure caused by starting ofa motor-driven pump caused suc-tion source realignment to the Nuclear Service Water system.
Pipes had not been routinely treated to inhibit clam growth, nor regularly monitored to detect their presence, and no strainers were installed. The need for surveillance which exercises alternative system opera-tional modes, as well as complete system functioning, is emphasized by this event. Spurious suction switchover has also occurred at Callaway and at McGuire, although no failures resulted.
CC10. Common cause failures have also been caused by component failures (AEOD/C404, 1984). AtSurry-2, both the turbine driven pump and one motor driven pump were declared inoperable due to steam binding caused by leakage ofhot water'hrough multiple check valves. AtRobinson-2 both motor driven pumps were found to be hot, and both motor and steam driven pumps were found to be inoperable at different times.
Backleakage at Robinson-2 passed through closed motor-operated isolation valves in addition to multiple check valves. AtFarley, both motor and turbine driven pump casings were found hot, although the pumps were not declared inoperable. In addition to multi-train failures, numerous incidents ofsingle train failures have occurred, resulting in the designation of"Steam Binding ofAuxiliaryFeedwater Pumps" as Generic Issue 93.
This generic issue was resolved by Generic Letter 88-03 (Miraglia, 1988), which required licensees to monitor AFWpiping temperatures each shift, and to maintain procedures for recognizing steam binding and for re-storing system operability.
CC11. Common cause failures have also failed motor operated valves. During the total loss offeedwater event at Davis Besse, the normally-open AFWisolation valves failed to open after they were inadvertently closed. The failure was due to improper setting of the torque switch bypass switch, which prevents motor trip on the high torque required to unseat a closed valve. Previous prob-lems with these valves had been addressed by increasing 5.3 NUREG/CR-5832
Failure Modes the torque switch trip sctpoint - a fix whic failed during the event due to the higher torque required due to high differential pressure across the valve. Similar common mode failures ofMOVs have also occurred in other sys-tems, resulting in issuance ofGeneric Letter 89-10, "Safety Related Motor-Operated Valve Gating and Sur-veillance" (Partlow, 1989). This generic letter requires licensees to develop and implement a program to pro-vide for the testing, inspection and maintenance ofall safety-related MOVs to provide assurance that they will function when subjected to design basis conditions.
CC12. Other component failures have also resulted in AFWmulti-train failures. These include out-of-adjustment electrical flowcontrollers resulting in improper discharge valve operation, and a failure ofoil cooler cooling water supply valves to open due to silt accumulation.
5.2.2 Human Errors HE1. The overwhelmingly dominant cause ofproblems identified during a series ofoperational readiness evaluations ofAFWsystems was human performance.
The majorityofthese human performance problems resulted from incomplete and incorrect procedures, particularly with respect to valve lineup information. A study ofvalve mispositioning events involving human error identiTied failures in administrative control of tagging and logging, procedural compliance and comple-tion ofsteps, veriTiication ofsupport systems, and inadequate procedures as important. Another study found that valve mispositioning events occurred most often during maintenance, calibration, or modification activities. Insufficient training in determining valve position, and in administrative requirements for con-trollingvalve positioning were important causes, as was oral task assignment without task completion feedback.
HE2. 'Ibrbine driven pump failures have been caused by human errors in calibrating or adjusting governor speed control, poor governor maintenance, incorrect adjust-ment ofgovernor valve and overspeed trip linkages, and errors associated with the trip and throttle valve. TfV-associated errors include physically bumping it, failure to restore it to the correct position after testing, and failures to verifycontrol room indication ofTTVposi-tion followingactuation.
HE3. Motor driven pumps have been failed by human errors in mispositioning handswitches, and by procedure deficiencies.
5.23 Design/Engineering Problems and Errors DE1. As noted above, the majority ofAFWsubsystem failures, and the greatest relative system degradation, has been found to result from turbine-driven pump fail-ures. Overspeed trips of'krryturbines controlled by Woodward governors have been a significant source of these failures (AEOD/C602, 1986). In many cases these overspced trips have been caused by slow response of a Woodward Model EG governor on startup, at plants where fullsteam flowis allowed immediately. This over-sensitivity has been removed by installing a startup steam bypass valve which opens first, allowing a control-led turbine acceleration and buildup ofoil pressure to control the governor valve when fullsteam flowis admitted.
DE2. Overspecd trips of'Ibrry turbines have been caused by condensate in the steam supply lines. Con-densate slows down the turbine, causing the governor valve to open farther, and overspced results before the governor valve can respond, after the water slug clears.
This was determined to be the cause of the loss-of-all-AFWevent at Davis Bcsse (AEOD/602, 1986), with condensation enhanced due to the, long length ofthe cross-connected steam lines. Repeated tests followinga cold-start trip may be successful due to system heat up.
DE3. Turbine trip and throttle valve (TIV)problems are a significant cause ofturbine driven pump failures (IN84-66). In some cases lack of7IVposition indica-tion in the control room prevented recognition of a tripped TIV. In other cases itwas possible to reset either the overspecd trip or the TIVwithout reseting the other. This problem is compounded by the fact that the position of the ovcrspecd trip linkage can be mis-leading, and thc mechanism may lack labels indicating when it is in the tripped position (AEOD/C602, 1986).
DE4. Startup ofturbines with Woodward Model PG-PL governors within 30 minutes ofshutdown has rc-sultcd in overspeed trips when the speed setting knob was not exercised locally to drain oil from the spccd NUREG/CR-5832 5.4
Failure Modes setting cylinder. Speed control is based on startup with an empty cylinder. Problems have involved turbine rota-tion due to both procedure violations and leaking steam.
'Ibrry has marketed two types ofdump valves for auto-matically draining the oil after shutdown (AEOD/C602, 1986).
AtCalvert Cliffs,a 1987 losscf-offsite-power event required a quick, cold startup that resulted in turbine trip due to PG-PL governor stability problems. The short-term corrective action was installation ofstiffer buffer springs (IN88-09, 1988). Surveillance had always been preceded by turbine warmup, which illustrates the importance of testing which duplicates service condi-tions as much as is practical.
DE5. Reduced viscosity ofgear box oil heated by prior operation caused failure ofa motor driven pump to start due to insufficient lube oil pressure. Lowering the pres-sure switch setpoint solved the problem, which had not been detected during testing.
DE6. Waterhammer at Palisades resulted in AFWline and hanger damage at both steam generators.
The AFW spargers are located at the normal steam generator level, and are frequently covered and uncovered during level fluctuations. Waterhammers in top-feed-ring steam generators resulted in main fccdline rupture at Maine Yankee and feedwater pipe cracking at Indian Point-2 (IN84-32, 1984).
DE7. Manually reversing the direction ofmotion ofan operating valve has resulted in MOVfailures where such loading was not considered in the design (AEOD/C603, 1986). Control circuit design may pre-vent this, requiring stroke completion before reversal.
DE8. Ateach ofthe units of the South 'Ibxas Project, space heaters provided by the vendor for use in prein-stallation storage ofMOVs were found to be wired in parallel to the Class 1E 125 V DC motors for several AFWvalves (IR 50-489/89-11; 50-499/89-11, 1989). Thc valves had been environmentally qualified, but not with the non-safety-related heaters energized.
5.2.4 Component Failures Generic Issue II.E6.1, "InSitu Zbsting OfValves" was divided into four sub-issucs (Beckjord, 1989), three of which relate directly to prevention ofAFWsystem component failure. Atthe request ofthe NRC, in-situ testing ofcheck valves was addressed by the nuclear in-dustry, resulting in the EPRI report, "Application Guidelines for Check Valves in Nuclear Power Plants" (Brooks, 1988). This extensive report provides infor-mation on check valve applications, limitations, and inspection techniques.
In-situ testing ofMOVs was addressed by Generic Letter 89-10, "Safety Related Motor-Operated Valve 1bsting and Surveillance" (Partlow, 1989) which requires licensees to develop and implement a program for testing, inspection and main-tenance ofall safety-related MOVs. "Thermal Overload Protection for Electric Motors on Safety-Related Motor-Operated Valves - Generic Issue II.E6.1
. (Rothbcrg, 1988)" concludes that valve motors should be thermally protected, yct in a way which emphasizes system function over protection ofthe operator CF1. The common-cause steam binding effects ofcheck valve leakage were idcntiTied in Section 5.2.1, entry CC10. Numerous single-train events provide additional insights into this problem. In some cases leakage ofhot MFWpast multiple check valves in series has occurred because adequate valve-seating prcssure was limited to the valves closest to the stcam generators (AEOD/C404, 1984). AtRobinson, the pump shutdown procedure was changed to delay closing the MOVs until after the check valves were seated. At Farley, check valves were changed from swing type to lifttype. Check valve re-work has been done at a number ofplants. Different valve designs and manufacturers are involved in this problem, and recurring leakage has been experienced, even after repair and replacement.
CF2. AtRobinson, heating ofmotor operated valves by check valve leakage has caused thermal binding and fail-ure ofAFWdischarge valves to open on demand. At Davis Besse, high differential pressure across AFW 5.5 NUREG/CR-5832
Failure Modes injection valves resulting from check valve leakage has prevented MOVoperation (AEOD/C603, 1986).
CF3. Gross check valve leakage at McGuire and Robinson caused overpressurization ofthe AFWsuc-tion piping. Ata foreign PWR it resulted in a severe waterhammer event. AtPalo Verde-2 the MFWsuction piping was overpressurized by check valve leakage from the AFWsystem (AEOD/C404, 1984). Gross check valve leakage through idle pumps represents a potential diversion ofAFWpump flow.
CF4. Roughly one third ofAFWsystem failures have been due to valve operator failures, with about equal failures for MOVs and AOVs. Almost halfof the MOV failures were due to motor or switch failures (Casada, 1989). An extensive study ofMOVevents (AEOD/C603, 1986) indicates continuing inoperability problems caused by: torque switch/limitswitch settings, adjustments, or failures; motor burnout; improper sizing or use ofthermal overload devices; premature degrada-tion related to inadequate use ofprotective dcviccs; damage due to misuse (valve throttling, valve operator hammering); mechanical problems (loosened parts, im-proper assembly); or the torque switch bypass circuit improperly installed or adjusted. The study concluded that current methods and procedures at many plants are not adequate to assure that MOVs willoperate when needed under credible accident conditions. Specifically, a surveillance test which the valve passed might result in undetected valve inoperability due to component failure (motor burnout, operator parts failure, stem disc separation) or improper positioning ofprotective devices (thermal overload, torque switch, limitswitch).
Generic Letter 89-10 (Partlow, 1989) has subsequently required licensees to implement a program ensuring that MOVswitch settings are maintained so that the valves willoperate under design basis conditions for the
'ife ofthe plant.
CF5. Component problems have caused a significant number ofturbine driven pump trips (AEOD/C602, 1986). Onc group ofevents involved worn tappet nut faces, loose cable connections, loosened set screws, improperly latched TIVs, and improper assembly.
Another involved oil leaks due to component or seal failures, and oil contamination due to poor maintenance activities. Governor oil may not bc shared with turbine lubrication oil, resulting in the need for separate oil changes.
Electrical component failures included transis-tor or resistor failures due to moisture intrusion, erroneous grounds and connections, diode failures, and a faulty circuit card.
CF6 Elcctrohydraulic-operated discharge valves have performed very poorly, and three ofthe five units using them have removed them due to recurrent failures.
Failures included oil leaks, contaminated oil, and hydraulic pump failures.
CF7. Control circuit failures were the dominant source ofmotor driven AFWpump failures (Casada, 1989).
This includes the controls used for automatic and manual starting ofthe pumps, as opposed to the instru-mentation inputs. Most of the remaining problems werc due to circuit breaker failures.
CF8. "Hydraulic lockup" ofLimitorque SMB spring packs has prevented proper spring compression to actuate the MOVtorque switch, duc to grease trapped in the spring pack. During a surveillance at 1lojan, failure ofthe torque switch to trip the VIVmotor resulted in tripping ofthe thermal overload device, leaving the turbine driven pump inoperable for 40 days until the next surveillance (AEOD/E702, 1987). Prob-lems result from grease changes to EXXONNEBULA EP-0 grease, one ofonly two greases considered environmentally qualiTied by Limitorque. Duc to lower viscosity, it slowly migrates from the gear case into the spring pack. Grease changeover at Vermont Yankee affected 40 of the older MOVs ofwhich 32 were safety related. Grease reliefkits are needed for MOVopera-tors manufactured before 1975. AtLimerick, additional grease reliefwas required for MOVs manufactured since 1975. MOVrefurbishment programs may yield other changeovers to EP-0 grease.
CF9. For AFWsystems using air operated valves, almost halfof the system degradation has resulted from failures of the valve controller circuit and its instrument inputs (Casada, 1989). Failures occurred predominantly at a few units using automatic electronic controllers for the flowcontrol valves, with the majority of failures duc to electrical hardware. AtTurkey Point-3, controller malfunction resulted from water in thc Instrument Air system due to maintenance inoperability of the air dryers.
NUREG/CR-5832 5.6
Failure Modes CF10. For systems using diesel driven pumps, most of the failures were due to start control and governor speed control circuitry. Halfofthese occurred on demand, as opposed to during testing (Casada, 1989).
CF11. For systems using AOVs, operability requires the availabilityofInstrument Air,backup air, or backup nitrogen. However, NRC Maintenance 'Imam Inspec-tions have identified inadequate testing ofcheck valves isolating the safety-related portion ofthe IAsystem at several utilities (Letter, Roe to Richardson).
Generic Letter 88-14 (Miraglia, 1988), requires licensees to verifyby test that air-operated safety-related com-ponents willperform as expected in accordance with all design-basis events, including a loss ofnormal IA.
5.7 NUREG/CR-5832
6 References Beckjord, E S. June 30, 1989. Closeout ofGeneric Issue IIE.6.1, "In Situ Testing ofValves." Letter to V. Stello, Jr., U.S. Nuclear Regulatory Commission, Washington, DG Brooks, B. P. 1988. Application Guidelines forCheck Valves in Nuclear Power Plants. NP-5479, Electric Power Research Institute, Palo Alto, CA.
Casada, D. A. 1989. AuxiliaryFeedwater System Aging Study. Volume 1. Operating Experience and Cunent MonitoringPracnces.
NUREG/CR-5404. U.S. Nuclear Regulatory Commission, Washington, DG AEOD Reports AEOD/C404. W. D. Lanning. July 1984. Steam Binding ofAuxiliaryFeedwater Pumps.
U.S. Nuclear Regulatory Commission, Washington, DG AEOD/C602. C. Hsu. August 1986. Operational Experience InvolvingTurbine Overspeed Trips. U.S.
Nuclear Regulatory Commission, Washington, DG AEOD/C603. E.J. Brown. December 1986. A Revie~
ofMotor-Operated Valve Performance.
U.S. Nuclear Regulatory Commission, Washington, DG Gregg, R. E and R. E. Wright. 1988. Appendix Review forDominant Generic Contributors. BLB-31-88. Idaho National Engineering Laboratory, Idaho Falls, Idaho.
Miragli,'.J. February 17, 1988. Resolution ofGeneric Safety Issue 93, "Steam BindingofAuxiliaryFeedwater Pumps" (Generic Letter 88-03). U.S. Nuclear Regulatory Commission, Washington, DC.
Miraglia, F. J. August 8, 1988. Instrument AirSupply System Problems AffectingSafely-Related Equipment (Generic Letter 88-14). U.S. Nuclear Regulatory Commission, Washington, DC.
Partlow, J. G. June 28, 1989. Safety-Related Motor-Operated Valve Testing and Surveillance (Generic Letter 89-10). U.S. Nuclear Regulatory Commission, Washington, DG Rothberg, O. June 1988. Thermal Overload Protection forElectric Motors on Safety-Related Motor-Operated Valves-GenericlssueII.E.6.1.
NUREG-1296. U.S.
Nuclear Regulatory Commission, Washington, DC.
Travis, R. and J. Ihylor. 1989. Development of Guidance forGeneric, Functionally Oriented PRA-Based Team Inspections forBWR Plants-Idennftcation ofRisk-Important Systems, Components and Human Actions.
TLR-A-3874TGABrookhaven National Laboratory, Upton, Ncw York.
AEOD/E702. E.J. Brown. March 19, 1987. MOV Failure Due to Hydraulic Lockup From Excessive Grease in Spring Pack. U.S. Nuclear Regulatory Commission, Washington, DG AEOD/I'416. January 22, 1983. Loss ofESF Auxiliary Feedwater Pump Capability at Trojan on January 22 1983. U.S. Nuclear Regulatory Commission, Washington, DG Information Notices IN 82-01. January 22, 1982. AuxiliaryFeedwater Pump Lockout Resulting from Westinghouse W-2 Switch Circuit Modification. U.S. Nuclear Regulatory Commission, Washington, DG IN84-32. E. LJordan. April18, 1984. Auxiliary Feedwater Sparger and Pipe Hangar Damage.
U.S.
Nuclear Regulatory Commission, Washington, DC.
IN 84-66. August 17, 1984. Undetected Unavailabilityof the Turbine-Driven AuxiliaryFeedwater Train. U.S.
Nuclear Regulatory Commission, Washington, DC.
IN87-34. G E. Rossi. July 24, 1987. Single Failures in AuxiliaryFeedwater Systems.
U.S. Nuclear Regulatory Commission, Washington, DC.
6.1 NUREG/CR-5832
References IN87-53. C. E. Rossi. October 20, 1987. Auxiliary Feedwater Pump Trips Resulting from LowSuction Pressure.
U.S. Nuclear Regulatory Commission, Washington, DG IN88-09. C. E. Rossi. March 18, 1988. Reduced ReliabilityofSteam-Driven AuxiliaryFeedwater Pumps Caused by InstabilityofWoodward PG-PL Type Governors.
U.S. Nuclear Regulatory Commission, Washington, DG IN89-30. R.A.Azua. August 16,1989. Robinson Unit 2 Inadequate NPSH ofAuxiluuyFeedwater Pumps. Also, Event Notification 16375, August 22, 1989. U.S.
Nuclear Regulatory Commission, Washington, DG Inspection Report IR 50M9/89-11; 50499/89-11. May 26, 1989. South Texas Project Inspection Report. U.S. Nuclear Regulatory Commission, Washington, DG NUREG Report NUREG-1154.
1985. Loss ofMain and Auxiliary Feedwater Event at the Davis Besse Plant on June 9, 1985.
U.S. Nuclear Regulatory Commission, Washington, DC.
NUREG/CR-5832 6.2
NUREG/CR-5832 PNL-7782 Distribution No. of
~Co ies OFFSITE U.S. Nuclear Re lato Commission B. K Grimes OWFN 9 A2 F. Congel OWFN 10 E4 A. C Thadani OWFN SE2 R. J. Barret OWFN 13 Dl
'. D. Holahan OWFN SE2 K Campe OWFN 1 A2 J. A. Isom OWFN 9A1 10 J. Chung OWFN 10 A2 J. N. Hannon OWFN 13 E21 2
B. Thomas OWFN 12 H26 U.S. Nuclear Re lato Commission-
~Re 'on 3 H. J. Miller E. G. Greenman W. D. Shafer No. of
~Co ies 4
D. C Cook Resident Ins tor Office J. H. Taylor Brookhaven National Laboratory Bldg. 13 Upton, NY 11973 R. 'Davis Brookhaven National Laboratory Bldg. 130 Upton, NY 11973 R. Gregg EG&G Idaho, Inc.
P.O. Box 1625 Idaho Falls, ID 83415 D. R. Edwards Prof. of Nuclear Engineering University of Missouri - Rolla Rolla, MO 65401 ONSITE 22 Pacific Northwest Laborato L R. Dodd B. F. Gore (10)
N. E. Maguire-Moffitt R. C. Lloyd B. D. Shipp F. A. Simonen T. V. Vo Publishing Coordination
'Ibchnical Report File (5)
Distr.1
NRC FOAM 335 12W91 NRCM 1102.
2201, 2202
- 2. TITLE AND SUBTITLE US. NUCLEAR REGULATORY COMMISSION B(BLIOGRAPHIC DATA SHEET ISaainsrnrcsrons on lna ioeorsrl I. REPORT I4UMBEP Iaessenen Oy NRC. AOO Vol.. Ss>>o.. Rey
~no Anoenoem Nsrmoers, It eny,s NUREG/CR"5832 PNL-7782 Auxiliary Feedwater System Risk-Based Inspection Guide for the D. C.
Cook Nuclear Power Plant 3.
DATE REPORT PUBLISHED MONTN YEAR October 1992
- 5. AU THO R IS)
R.C. Lloyd, N.E. Moffitt, B.F. Gore, T.V. Vo, 3.A.
Isom*
- 6. TYPE OF REPORT Techni cal
- 7. PERIOD COVERED nnasen>> Oasrn 7/91 9/92 B. PERFORMING ORGANIZATIONNAMEAND ADDRESS illiyRC pror>>r One>>n, price or neo>>n, IES, Fr~~n~i~ C ~ ano~ ~~lsl ~sr~sar p>>~
none ono masssnF aoaresnl Pacific Northwest Laboratory
- U.S. Nuclear Regulatory Commission P.O.
Box 999
- Richland, WA 99352
- 9. SPONSORING ORGANIZATIONNAMEAND ADDRESS llfrrRC. Iype "San>>as aooee":ilconiraasor proesoerrnCOire>>n. Otleeorneo>>n, u> rrsresearneaerasory Corn>>re>>n.
ann rnailesF aooreeLI Division of Radiation Protection and Emergency Preparedness Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Washington, DC 20555 TO. SUPPLEMENTARY NOTES 11, ABSTRACT (200>>eros or srssl In a study sponsored by the U.S. Nuclear Regulatory C'ommission (NRC), Pacific Northwest Laboratory has developed and applied a methodology for deriving plant-specific risk-based'nspection guidance for the auxiliary feedwater (AFW) system at pressurized water reactors that have not undergone probabilistic risk assessment (PRA).
This methodology uses existing PRA results and plant operating experience information.
Existing PRA-based inspection guidance information recently developed for the NRC for various plants was used to identify generic component failure modes.
This information was then combined with plant-specific and industry-wide component information and failure data to identify failure modes and failure mechanisms for the-AFW system at the selected plants.
D. C.
Cook was selected as one of a series of plants for study.
The product of this effort is a prioritized listing of AFW failures which have occurred at the plant and at other PWRs.
This listing is intended for use by NRC inspectors in the preparation of inspection plans addressing AFW tisk-important components at the D.C. Cook plant.
- 12. KEY WOROSIDESCRIPTORS lLes scorns or pnreees met <<coarser researeners sn rocassnF Ine reoors,l Inspection,
- Risk, PRA, D.C. Cook, Auxiliary Feedwater (AFW)
UL AVAILABILITYSTATEMENT Unlimited Ia. SECURITY CLASSIFICA'IIOrs IToe l'aael Unclassified IThe Reposes Unclassified
- 15. NUMBER OF PAGES I6. PRICE