ML20113H105: Difference between revisions
StriderTol (talk | contribs) (StriderTol Bot insert) |
StriderTol (talk | contribs) (StriderTol Bot change) |
||
Line 18: | Line 18: | ||
=Text= | =Text= | ||
{{#Wiki_filter:}} | {{#Wiki_filter:- . . = | ||
: , Ef4 CLOSURE NUREG/CR-P!it - | |||
i AUXILIARY FEEDWATER SYSTEM RISK-BASED INSPECTION GUIDE FOR THE VIRGIL 0. SUMMER NUCLEAR POWER PLAtiT R. C. Lloyd N. E. Moffitt B. F. Gore T. V. Vo November 1991 Prepared for Division of Radiation Protection and Emergency Preparedness Office of Nuclear Regulatory Regulation U.S. Nucleer Regulatory Commission Washington, DC 20555 NRC FIN L1310 Pacific Northwest Laboratory Richland, Washington 99352 9207070372 920629 'S P Di< ADOCK 0500 F | |||
. l SUu m Y This document pre.ents a compilation of auxiliary feedwater (AFW) system failure information which has been screened for risk significance in terms of failure frequency and degradation of system performance. It is a risk-prioritized listing of f ailure events and their causes that are significant enough to warrant consideration in inspection planning at the Virgil C. Summer | |||
. plant. This information is presented to provide inspectors with increased resources for inspection planning at Virgil C. Summer. l The isk importance of various component failure modes was identified by analysis of the results of probabilistic risk assessments (PRAs) for many pressurized water reactors (PWRs). | |||
However, the component failure categories ! | |||
identified in PRAs are rather broad, because the failure data used in the PRAs 3 is an aggregate of many individual failures having a variety of root causes. ; | |||
In order to help inspectors focus on specific aspects of component operation, ' | |||
maintenance and design which might cause these failures, an extensive review of component failure information was performed to identify and rank the root causes of these component failures. Both Virgil C. Summer and industry Hde failure information was analyzed. Failure causes were sorted on the basis of frequency of occurrence and seriousness of consecuence, and categorized as common cause failures, human errors, design problems, or component f ailures. | |||
This information is presented in the body of this document. Section 3.0 provide brief descriptions of these risk important failure causes, and Section 5.0 presents more extensive discussions, with specific examples and references. The entries in the two sections are cross-referenced. | |||
An abbreviated system walkdown table is presented in Section 3.2 which includes only components identified as risk important. This table lists the system lineup for normal, standby system operation. | |||
This information permits an inspector to concer,trate on components important to the prevention of core damage. However, it is important to note-that inspections sh;ald not focus exclusively on these components. Other components which perform essential functions, but which are not included because of high reliability or redundancy, must also be addressed to ensure that degradation does not increase their failure probabilities, and hence their risk importance. | |||
iii I | |||
6 | |||
[pNTENTS n | |||
==SUMMARY== | |||
.............................................................. iii , | |||
==1.0 INTRODUCTION== | |||
....... ............................................ 1 2.0 VIRGIL C. SUMMER AFW SYSTEM...................................... 2 i | |||
2.1 SYSTEM DESCRIPTION ......................................... 2 2.2 SUCCESS CRITERION .......................................... 4 2.3 SYSTEM DEPENDENCIES ........................................ 4 2.4 OPERATIONAL CONSTRAINTS .................................... 4 3.0 INSPECTION GUIDANCE FOR THE VIRGIL C. SUMMER AFW SYSTEM.......... 5 3.1 RISK IMPORTANT AFW COMPONENTS AND FAILURE MODES ............5 3.1.1 MULTIPLE PUMP FAILURES DUE TO COMMON CAUSE .......... 5 3.1.2 TURBINE DRIVEN PUMP FAILS TO START OR RUN ..................................... 6-3.1.3 MOTOR DRIVEN PUMP A OR B FAILS TO START OR RUN ..... .................................. 7 3.1.4 PUMP UNAVAl'LABLE DUE TO MAINTENANCE OR SUPVEILLANCE .................. 7 3.1.5 AIR OPERATED ISOLATION AND FLOW CONTROL VALVE FAILURE ....................................... 7 3.1.6 MOTOR OPERATED VALVE FAILURE ........................ 8 3.1.7 MANUAL SUCTION OR DISCHARGE VALVES Fall CLOSED ......................................... 9 3.1.8 LEAKAGE OF HOT FEE 0 WATER THROUGH CH VALVES ............................ECK .................. 9 3.2 RISK IMPORTANT AFW SYSTEM WALKDOWN-TABLE ................... 10 4.0- GENERIC RISK INSIGHTS FROM PRAs ................................, 14 4.1 RISK IMPORTANT ACCIDENT SEQUENCES INVOLVIN AFW SYSTEM FAILURE .......................,.................. 14 4.2 RISK IMPORTANT COMPONENT FAILURE MODES ..................... 15 V | |||
, , - - , . _ . . . ~ . , _ _ _ . _ . . _ . -- | |||
m . | |||
CONTENTS (continued) 5.0 FAILURE MODES DE1 ERMINED FROM OPERATING EXPERIENCE .............. 16 1 5.1 VIRGIL C. SUMMER EXPERIENCE................................. 16 , | |||
l i 5.1.1 MOTOR DRIVEN PUMP FAILURES .......................... 16 5.1.2 TURBINE DRIVEN PUMP FAILURES ........................ 16 5.1.3 FLOW CONTROL AND ISOLATION VALVE FAILURES ........... 16 5.1.4 CHECK VALVES ........................................ 17 1 5.2 INDUSTRY WIDE EXPERIENCE ................................... 17 5.2.1 COMMON CAUSE FAILURES ............................... 17 5.2.2 HUMAN ERRORS ........................................ 20 5.2.3 DESIGN / ENGINEERING PROBLLMS AND ERRORS .............. 20 5.2.4 COMPONENT FAILURES .................................. 21 REFERENCES ........................................................... 25 I-i I I I vi l | |||
i | |||
~ | |||
==1.0 INTRODUCTION== | |||
This document is one of a series providing plant-specific inspection guidance for auxiliary feedwater (AFW) systems at pressurized water reactors (PWRs). This guidante is based on information from probabilistic risk assessmentt (PRAs) for similar PWRs, incustry wide operating experience with AFW systems, plant specific AFW system descriptions, and plant specific | |||
, operating experience, it is not a detailed inspection plan, but rather a compilation of AFW system failure information which has been screened for risk significance in terms of failure frequency and degradation of system performance. The result is a risk-prioritized listing of failure events and the causes that are significant enough to warrant consideration in inspection planning at Virgil C. Summer. | |||
This inspection guidance is presented in Section 3.0, following a description of the Virgil C. Summer AFW system in Section 2.0. Section 3.0 identifies the risk important system components by Virgil C. Summer identification number, followed by brief descriptions of each of the various failure causes of that component. These include specific human errors, desion deficiencies, and haroware failures. Tne discussions also identify where common cause failures have affected multiple, redundant components. These brief discussions identify specific aspects of system or component design, operation, maintenance, or testing for inspection by observation, recoros review, training observati e procedures review, or by observation of the implementation of procedur e An AFW system walkdown table identifying risk important components and their lineup for normal, standby system operation is also previoed. | |||
The remainder of the document describes and discusses 14 information used in compiling this inspection guidance. Section 4.0 describes the risk important information which has been derived from PRAs and its sources. As review of that section wlll show, the failure events identified in PRAs are rather broad (e.g., pump f ails to start or run, valve f ails closed). Section 5.0 addresses the specific failure causes which have been combined under these broad events. | |||
AFW syster operating history was studied to identify the various specific failures which have been aggregated into the PRA failure events. Section 5.1 presents a sunc,ary of Virgil C. Summer f ailure information, and Section 5.2 presents a review of industry-wide failure information. The industry-wide information was compiled from a variety of HRC sources, including /E0D analyses and reports, information notices, inspection and enforcement bulletins, and generic letters, and from a variety of INPO reports as well. | |||
Some Licensee Event Reports and NPRDS event descriptions were also reviewed. | |||
Finally, infort.ation was incluoed from reports of HRC-sponsored studies of the effects of plant aging, which incluoe cuantitative analyses of reported AFW system failures. This industry-wide information was then combined with the plant-!pecific failure infurmation to identify the various root causes of the broad failure events used in PRAs, which are ioentified in Section 2.0. | |||
1 | |||
2.0 VIRGIL C. SV%' tJW SYSTEM This section presents an overview description of the Virgil C. Summer AFW system (Westinghouse 3 loop plant), including a simplified schematic system diagram, in addition, the system success criterion, system dependencies, and administrative operational constraints are also presented. | |||
2.1 System Description The ATW system provides feedwater to the steam generators (SG) to allow secondary-side heat removal from the primary system when main feedwater is unava ilable. The system is capable of functioning for extended periods, which allows time to restore main feedwater flow or to proceed with an orderly cooldown of the plant to where the residual heat removal (RHR) system can remove decay heat. A simplified schematic diagram of the Virgil C. Summer AFW system is shown in Figure 2.1. | |||
The AFW system consists of two motor driven (MD) pumps and one steam-driven (TD) pump along with the associated piping, valves and instrumentation normally connected to the Condensate Storage Tank (CST). It is designed to start up and establish flow automatically. All pumps start on receipt of a steam generator low low level signal. (The motor criven pumps start on low level in one SG, whereas, two low level signals are reouired for the steam-driven pump to start.) Also, the motor driven pumps start en a trip of main feedwater pumps (MFW) pumps, a safety injection signal, undervoltage on either ESF Bus DA or DB, or on ATWS Mitication System Circuit Activation (AMSAC). ine single turbine-driven (TD) pump also starts on undervoltage o.1 the ESF busses DA and DB, and on an AMSAC SIGNAL. | |||
A suction line from the CST provides a common header that supplies water to the turbine-driven pump and to both motor driven pumps. Isolation valves in these lines are locked open. Power, control, and instrumentation associated with each motor-driven pump are independent from one another. | |||
Steam for the turbine-driven pump is supplied by steam generators B and C, from a point upstream of the main steam isolation valves, through valve 2865. | |||
Each AFW pump is equipped with a continuous recirculation flow system, which prevents pump deadheading. | |||
- The discharges of the motor driven pumps are cross connected and each pump can feed all three steam generators. The turbine-driven pump also feeds all three steam generators througn separate lines. Each of these six lines contains an air operated discharge flow control valve. Motor driven and turbine driven pump flow control valves are 3531,3541,3551, and 3536, 3546, 3556, respectively. Safety class air accumulators for-each flow control valve provide sufficient air capacity to permit remote valve closure for isolation of a seconcary system break. Each of the lines from the motor driven pumps also contains a manually operated discharge isolation valve,1017 A,B,C. | |||
Discharge isolution valves in the lines from the turbine-driven pump are manually operated valves,101E A.E,C, AFW containment isolation valves are provided by air assist-spring operated check valves 1009 A, E, C. Each AFW 2 | |||
i A- | |||
~ | |||
{ '1 i | |||
i i: | |||
1017A to.9A ORC f IRC | |||
- g/gs / | |||
l 1009A | |||
- J , | |||
CorMf. | |||
1018A 1020A lL"i e 'x-#:-t+ | |||
l | |||
~ | |||
3536 !j | |||
.L. ' | |||
1007 %1010 - | |||
i ~~ | |||
10250 / | |||
101TB 10198 # | |||
* 03 A 001A , | |||
( ' fMi 1018R 101 El | |||
( S""* ] A _ | |||
102tA f t 3546 / . | |||
wasos l a | |||
v M h 1026 tot 1C 1019C / | |||
toop s + . _p'l _N \_M IN $ | |||
j tr 1B 10018 s, | |||
'M10210 3551 # | |||
s 1018C 1020C ,, | |||
B # | |||
ep 3556 f | |||
/T 1006 | |||
._Q - | |||
Cham H M-OfD) | |||
/ | |||
[ | |||
Il P / | |||
S am 1012 2002A 20T8 p G D) g,C C) staan M | |||
tsoterm Gowwcor ; | |||
Desun ' Vdve Vda 2813 & | |||
E 2ees i i | |||
FIGURE 2.1 V.C. SUMMER EMERGENCY FEEDWATER SYSTEM | |||
_I | |||
[ | |||
; \ | |||
line also contains several check valves to prevent leakage from the feedwater lines. | |||
The condensate storage tank (CST) is the normal source of water for the AFW System and is required to store sufficient demineralized water to maintain the reactor coolant system (RCS) at hot standby conditions for 11 hours (172,000 gallons). All tank connections except those required for instrumentation, auxiliary feedwater punp suction chemical analysis, and tank | |||
, drainage are located above this minimum level. Backup AFW supply is automatically provided by the service water system. On a low suttion pressure condition of 11.0 psig for 5 seconds, service water valves, 1001A & B and 1037 A & B will open automatically to supply Loop A and B. | |||
I 2.2 Success Criterien l System success requires the operation of at least one pump supplyin; | |||
' rated flow to at least two of the three steam generators. | |||
2.3 Svsten_pecendencies Tne AFW system depends on At and DC power at various voltage levels for motor operation, valve control, monitor and alarm circuits, and valve / motor control circuits. Instrument Air is reouired for governor speed control and flow control valve operation. Stet availability is required for the turbine-driven pump. | |||
2.4 Ooerational Constreints The Virgil C. Summer Technical Specifications recuire that all three AFW pumps and associated flow paths are operable with each motor driven pump powered from a different vital bus and one turbine driven pump enpable ef being powered from an operable steam supply system. If one AFW pump becomes inoperable, it must be restored to operable status within 72 hours or the plant must shut down to hot standby within the next six hours and in Hot Standby within the following six hours. If two AFW pumps are inoperable, be in hot standby within six hours and in het shutdown within the following six hours. If three AFW pumps are inoperable, immediately initiate corrective action to restore one pump to operable status as soon as possible. | |||
The Virgil C. Sum.ner Technical Specifications require a minimum supply of 172,000 gallons of water to be storea in the CST to maintain the RCS at het standby condition for 11 hours with steam discharge to atmosphere concurrent with total loss of offsite power. | |||
4 l | |||
3.0 INSoECT10N GUIDul:E FOR THE V100lt C. SUMMEP Arp SYSTEM in this section the risk important components of the Virgil C. Summer AfW system are identified, and the important failure modes for thesa components are briefly described. These f ailure modes include specific human errors, design deficiencies, and types of hardware f ailures which have been observed to occur for these components, both at Virgil C. Sumer and at PWRs throughout the nuclear industry. The discussions also identify where common cause f ailures have affected multiple, redundant components. These brief discussions identify specific aspects of system or component design, operation, maintenance, or testing for inspection activities. These activities include; observation, records review, training observation, procedures review, or by observation of the implementation of procedures. | |||
Table 3.1 is an abbreviated ATW system walkdown table which identifies risk-important components. This table lists the system lineup for normal (standby) system operation. Inspection of the components identified in the AFW walkdown table adoresses essentially all of the risk associated with AFW system operation. | |||
3.1 Risk leportant ArV Co nonenti and ra ilure Mod u Common cause f ailures of multiple pumos are the most risk-important failure mod: of AFW system components. These are followed in importance by 1 | |||
single pump ''ilures, level control valve f ailures, and individual check valve leakage failures. | |||
The following sections address each of these f ailure modes, in decreasing order of risk-importance. They present thc- important root causes of these component failure modes which have been distilled from historical records. | |||
Each item is keyed to discussions in Section 5.2 where additional information on historical events is presented. | |||
3.1.1 Multiple Plime Failures due to Comon Cause The following listing summarizes the most important multiple-pump failure modes identified in Section 5.2.1, Common cause Failures, and each item is keyed to entries in that section. | |||
- Incorrect operator intervention into automatic system functioning, including improper manual starting and securing of pumps, has caused failure of all pumps, including overspeed trip on~startup, and inability to restart prematurely secured pumps. CC1. ' | |||
Valve mispositioning has caused failure of all pumps. Pump suction, | |||
, steam supply, and instrument isolation valves have been involved. | |||
CC2. | |||
* Steam binding has caused failure of multiple pumps. This resulted from leakage of hot feeowater past check valves and a motor-operated valve into a common discharge neader CC10. Multiple c amp steam 5 | |||
I s-.. . ~ . . ,r-,-- -,r_-,wm,m._,m.___ _ ._--_,,..,..,,w%,,,, ,m,,,,-..my,y._,,,-,,r.,_,,,,,,..,,,,-,.,,,-m-,,,m-.-g,-,,,-y_,,,, - | |||
binding has also resulted from improper valve lineups, and from running a pump deadheaded. CC3. | |||
. Pump control circuit deficiencies or design modification errors have caused f ailures of multiple pumps to auto start, spurious pump trips during operation, and failures to restart after pump shutdown. CC4. | |||
Incorrect setpoints and control circuit calibrations have also prevented proper operation of multiple pumps. CCS. | |||
t | |||
. Loss of a vital power bus has failed both the turbine-driven and one motor driven pump due to loss of control power to steam admission valves or to turbine controls, and to motor controls powered from the same bus. CC6. | |||
. Simultaneous startup of multiple pumps has caused oscillations of pump suction pressure causing multiple-pump trips on low suction pressure, despite the existence of adequate static net positive suction head (NPSH). CC7. At H.B. Robinson, design reviews have identified inadequately sized suction piping which could have yielded insufficitnt NPSH to support operation of more than one pump. CCS. | |||
3.1.2 Turbine Driven Port re nt te Start or Run | |||
. Improperly adjusted and inadeouately maintained turbine governors have caused pump failures. HE2. Problems include worn or loosened nuts, set screws, linkages or cable connections, oil leaks and/or contamination, and electrical f ailures of resistors, transistors, diodes and circuit cards, and erroneous grounds and connections. | |||
CF5. Governor problems, bearing wear, low oil, and human error in making proper settings have caured failure of the turbine driven pump at Virgil C. Summer. | |||
~ | |||
. Terry turbines with Woodward Model EG governors have been found to overspeed trip if full steam flow is allowed on startup. | |||
Sensitivity can De reduced if a startup steam bypass valve is sequenced to open first. del. | |||
. Turbines with Woodward Model PG-PL governors have tripped on overspeed when restarted shortly after shutdown, unless an operator has locally exercised the speed setting knob to dr.in oil from the governor speed setting cylinder (per procedure). Automatic oil dump valves are now available through Terry. DE4. | |||
. Condensate slugs in steam lines have caused turbine overspeed trip on startup, lests repeated right after auch a trip may fail to indicate the problem due to warming and clearing of the steam lines. | |||
Surveillance should exercise all steam supply connections. DE2. | |||
. Trip and throttle valve (TTV) problems which have failed the turbine driven pump include physically bumping it, f ailure to reset it following testing, and f ailuces to verifv control icor indication of 6 | |||
s reset. HE2, Whether either the overspeed trip or TlV trip can be reset without resetting the other, indication in the control room of TTV position, and unambiguous local indication of an overspeed trip affect the likelihood of these errors. DES. The TTV at the Virgil C. | |||
Summer plant has failed to reset due to misalignment of pins which prevented reengagement of the trip lever. | |||
3.1.3 Motor Driven Pumo A or B Fails to Start or Run | |||
. Control circuits used for automatic and manual pump starting are an important cause of motor driven pump failures, as are circuit breaker failures. CF7 Control circuit problems and a blown fuse due to overload have occurred at Virgil C. Summer. - | |||
. Hispositioning of handswitches and procedural deficiencies have prevented automatic pump start. HE3. | |||
. Low lubrication oil pressure resulting from heatup due to previous operation has prevented pump restart due to failure to satisfy the protective interlock. DE5. | |||
3.1.4 Puro Unavailable Due to Maintenance or Surveillance | |||
. Both scheduled and unscheduled maintenance remove pumps from operability. Surveillance requires operation with an altered line-up, although a pump train may not be declared inoperable during testing. Prompt scheJuling and performance of mainte:.ance and surveillance minimize this unavailability. | |||
3.1.5 Air Ooerated Isolation and Flow Control Valve Failure TD Pumn Train: 3536.3546.3556. 1009A.B.C 2030 MD Pumon Trains A: B: 353].3541.3551.1009 A.B.C These normally open air operated valves (A0Vs) isolate and control flow to the steam generators. They f ail open on loss of instrument air. | |||
. Control circuit problems have been a primary cause of failures, both at Virgil C. Summer and elsewhere. CF9. Valve ft lu es have resulted from blown fuses, failure of control components (st;h as current / pneumatic convertors), broken and dirty contacts, misaligned or broken limit switches, control power loss, and calibration problems. Degraded operation has also resulted from improper air pressure due to air regulator failure or leaking air lincs. | |||
Out-of-adjustment electrical flow controllers have caused improper valve coeration, affecting multiple trains of AFW, CC12 7 | |||
l | |||
. 1 Leakage of hot feedwater through check valves has caused thermal binding of flow control MOVs. A0Vs may be similarly susceptible. CF2. ; | |||
. Multiple flow control valves have been plugged by clams when suction l switched automatically to an alternate, untreated source. CC9 i I | |||
3.1.6 Motor Ooerated Valve Failure Loon A Blekuo Suction Sources: 1037A.1001A.1008 Loco B Backun Suction Sources: 10378.10018. l qQ2 These MOVs control or isolate flow of the service water to the AFW pumps. | |||
They fail as is on loss of power. , | |||
Commen cause failure of MOVs has resulted from failure to use electrical signature tracing equipment to determine proper settings | |||
- of torque switch and torque switch bypass switches. Failure to calibrate switch settings for high torques necessary under design j basis accident conditiuns has also been involved. CCli. Diaphragm , | |||
failure, packing leakage, air leakage, electrical component f ailure and seat leakage have been the main causes of valve failure at Virgil C. Summer. | |||
Valve motors have been failed due to lack of, or improper sizing or use of thermal overload protective devices. Bypassing and oversizing should be based on proper engineering for desion basis conditions. CF4. | |||
At Virgil C. Summer, heating of motor operated valves by check valve leakage has caused thermal binding and failure of AFW discharge valves to open on-demand. CF2. | |||
Out-of-adjustment electrical flow controllers have caused improper discharge valve operation, affecting multiple trains of AFW. CCl2, Grease trapped in the torque switch spring pack of Limitorque SMB motor operators has caused motor burnout or thermal overloed trip by preventing torque switch actuation. CF8. | |||
' Manually reversing the direction of motion of operating MOVs has overloaded the motor circuit. Operating procedures should provide | |||
- cautions, and circuit designs may prevent reversal before each stroke is finished. DE7. | |||
Space heaters designed for preoperation storage have been found wired in parallel with valve motors which had not been environmentally qualified with them present. DEB. | |||
8 c-- - - + - , . , w. ,.,y.- --n-, ,me- _ ,-,,o,~.,- | |||
~ | |||
3.1.7 Manual Suction or Discha oe Valves Faii Closfj CST Suction and Recirculation Valves: 1007.1010.1025A.B.1Mi TD Pume Train: 1012.1036. 1018A.B.C HD Pume Train A: 1011A.102]A. 1017A B.C MD Puer Train B: 10))B.102]E. 1017A.B.C These manual valves are all normally locked open. For each train, closure | |||
, of the first valve listed would block pump suction and closure of the second valves would block pun discharge. | |||
. Valve mispositioning has resulted in failures of multiple trains of AFW. CC2. It has also been the dominant cause of problems identified during operational readiness inspections. HEl. Events | |||
-have occurred most often-during maintenance, calibration, or systcm modifications, important causes of mispositioning include: | |||
. Failure to provide complete, clear, and specific procedures for tasks and system restoration | |||
. Failure to promptly revise and v:lidate procedures, training, and diagrams following system modifications | |||
. Failure to complete all steps in a procedure | |||
. Failure to adeouately review uncompleted procedural steps after task completion | |||
. Failure to verify support functions after restoration | |||
. Failure to adhere scrupulously to acministrative | |||
_ procedures regard,ng tagging, control and tracking of valve operations | |||
! . Failure to log the manipulation of sealed valves Failure to follow good practices of written task assignment and feedback of task completion information Failure to provide easily read system drawings, legible valve labels corresponding to drawings and procedures, and labeled indications of local valve position i | |||
3.1.8 Leakace of Hot Feedwater throuch Check Valves: | |||
i i MD Pump A: 1019A.B.C l MD Pumo B: 1019 A . B .,1 | |||
! TD Pumet 1020A.B.C l- Containment lsolation Stop Check Valver 1009A.B.C Leakage of hot feedwater throuah several check valves in series has caused steam binding of multipic pumps. Leakage through a closed level control valve in series with check valves has also occurred at Virgil C. Summer, as would be recuired for leakage to reach the motor driven pumps A and B. CC10. | |||
i Slow leakage past the final check valve of a series may not force l | |||
the check valve closed. Other check valves in seri.es may leak similarly. Piping orientation and valve design are important fatters in achieving true series protection. CFl. | |||
9 | |||
t 3.2 Pisk 1reertant tJV Syster Walkdown Table-Table 3.1 presents an AfW system walkdown table including only components identified as risk important. This information allows inspectors to concentrate their efforts on components important to prevention of core damage. However, it is essential to note that inspections should not focus exclusively on these components. Other components which perform essential | |||
- functions, but which are absent from this table because of high reliability or | |||
, redundancy, must also be addressed to ensure that their risk importance are not increased. An example would include ensuring an adequate water level in the CST exists. | |||
4 10 grec +e.es -,= 3 -,mw-rter--a.ww%,..-se %y.r.3 y .-.w ...,--yyp-a..-,.-- +,y-w.- -.ew.w=,-m---*--y--~- ,-w,-c..=-~,,---4,-;.,---,.-c-<-v- - | |||
,e,r y ,,sw s- | |||
~ . | |||
t 1 | |||
i 4 | |||
TABLE 3.1. Risk Important Walkdown Table for j Virgil C. Summer AfW System Components Required Actual Component a Component Name Position Position IIectrical i | |||
A Hotor Driven Pump Racked In/ | |||
Closed l B Motor-Driven Pump Racked In/ _ | |||
Closed - | |||
Valve 1010 CST Outlet Valve Locked Open 1007 CST Outlet Bypass Valve Locked Open 1009A Containment Isolation to S/G A Closed / Norm 1009B Containment Isolation to S/G B Closed / Norm 3009C Centainment Isolation to S/G C Closed / Norm 1012 TDP Suction Valve Locked Open 1036 TD FWP Discharge Valve Locked Open 1018A TD FWP Supply to S/G A Locked Open 1018B TD FWP Supply to S/G B Locked Open - | |||
1018C TD FWP Supply to S/G C Locked Open ._ | |||
1026 TDP Recire Isolation Locked Open 3536 TDP Flow Control to S/G A Open 3546 TDP Flow Control to S/G B Open 3556 TDP Flow Control to S/G C Open 1011A MDp *A* Suction Locked Open 1011B MDP *B* Suction Locked Open 1025A MDP *A* Recirculation Locked Open IC25B MDP *E' Recirculation Locked Open 22 l | |||
i | |||
..~ . . . - . _ . . . - . - - . - . - - _ - . . _ . _ . _ - - - _ . - _ . - . . - . . - . _ | |||
~ . | |||
JABLE 3.1. Risk Important Walkdown Table for Virgil C. Summer AFW System Components (Continued) | |||
MDP A Discharge Locked Open 1022A 1021B MDP B Discharge Locked Open r 1017A MDP Supply to S/G A Locked Open 1017B MDP Supply to S/G B Locked Open 1017C MDP Supply to S/G C Locked Open 3531 MDP Flow Control to S/G A Open 3541 MDP flow Control to S/G B Open 35$1 MDP Flow Control to S/G C Open 2802A MS Supply to TDP Open/ Auto 2802B MS Supply to TDP Open/ Auto 2030 Steam Isolation Valve Closed / Auto 2034 TDP Governor Valve Open __ _ | |||
2865 Steam Supply Throttle Valve Open 1037A SW Loop A Supply Valve Closed / Auto _ | |||
1037B SW Loop B Supply Valve Closed / Auto 2001A SW Loop A to MDP A Closed / Auto __ | |||
1001B SW Loop B to MDP B Closed / Auto 1008 SW Loop A to TDP Closed / Auto 1002 SW Loop B to TDP Closed / Auto 1019A MDP Supply Stop Ck to S/G A Locked /Open 1019B liDP Supply Stop Ck to S/G B Locked /Open 1019C MDP Supply Stop Ck to S/G C Locked /Open 1020A TDP Supply Stop Ck to S/G A Locked /Open 1020B TDP Supply Stop Ck to S/G B Locked /Open | |||
! 12 | |||
o TABLE 3.1. Risk Important Walkdown Table for Virgil C. Summer AfW System Components (Continued) 1020C TDP Supply Stop Ck to S/G C Locked /Open 1009A Piping Upstream of Check Valve Cool i 1009B Piping Upstream of Check Valve Cool 1009C Piping Upstream of Check Valve Cool 13 | |||
--_ ~ - _ . - .- | |||
8 h | |||
4.0 GENERIC RISK INSIGHTS FROM PRAs PRAs for 13 PWRs were analyzed to identify risk important accident sequences involving loss of AFW, and to identify and risk-prioritize the component failure modes involved. The results of this analysis are described in this section. They are consistent with results reported by INEL and BNL (Gregg et al 1988, and Tr evis et al,1988). | |||
4.1 Risk Important Accident Seauences involvina AFW System Failure Loss of Power System | |||
. A loss of offsite power is followed by failure of AFW. Due to lack of actuating power, the power operated relief valves (PORVs) cannot be opened preventing adequate feed-and bleed cooling, and resulting in core damage. | |||
. A station blackout fails all AC power except Vital At from DC invertors, and all decay heat removal systems except the turbine. | |||
driven AFW pump. AFW subsecuently fails due to battery depletion or hardware failures, resulting in core damage. | |||
. A DC bus faili, causing a trip and failure of the power conversion system. One AFW motor-driven pump is failed by the bus loss, and the turbine-driven pump f ails due to loss of turbine or valve control power. AFW is subsequently lost completely due to other failures. Feed-and bleed cooling f ails because PORV control is lost, resulting in core damage. | |||
Transient-Caused keactor or Turbine Trir | |||
. A transient-caused trio is followed by a loss of the power conversion system (PCS) and AFW. Feed-and-bleed cooling f ails either due to f ailure of the operator to initiate it, or due to hardware failures, resulting in core damage. | |||
Less of Main Feedwater | |||
. A feedwater line break drains the common water source for MFW and The operators fail t: . ' vide feedwater from other sources, AFW. | |||
and fail to initiate feed-a. sleed cooling, resulting in core damage. | |||
. A less of main feedwattr trips the plant, and AFW fails due to operator error and haroware failures. The operators fail to initiate feed-and-bleed cooling, resulting in core damage. | |||
Steam Generator Tube Ruoture (SGTP) | |||
. A SGTR is followed by failure of AFW. Coolant is lost from the primary until the refueling water storage tank (RWST) is cepleted. | |||
14 | |||
= | |||
e High pressure iniection (HPI) f ails since recirculation cannot be established from the empty sump, and core damage results. | |||
4.2 Risk Important Comoonent Failure Modes The generic component failure modes identified from PRA analyses as important to AFW system failure are listed below in decreasing order of risk i importance. | |||
: 1. Turbine-Driven Pump Failure to Start or Run. | |||
: 2. Motor-Driven Pump failure to Start or Run. | |||
: 3. TDP or HDP Unavailable due to Test or Maintenance. | |||
: 4. AFW System Valve Failures | |||
- steam admission valves a trip and throttle valve | |||
- flow control valves | |||
. pump discharge valves | |||
. pump suction valves | |||
- valves in testing or maintenance. | |||
: 5. Supply / Suction Sources | |||
. condensate storage tank stop valve | |||
. hot well inventory | |||
. suction valves. | |||
In addition to individual hardware, circuit, or instrument failures, each of these failure modes may result from common causes and human errors. | |||
Common cause failures of AFW pumps are particularly risk _important. | |||
Valve f ailures _ are somewhat less important due to the multiplicity of steam generators and connection, paths. Human errors of greatest risk importance involve: failures to initiate or control system operation when required; f ailure to restore proper system lineup after maintenance or testing; and f ailure to switch to alternate sources when required. | |||
15 | |||
s . . | |||
a e | |||
5.0 FAltVRE HDDES DETERMINED FROM OPERATING EXPERifNC[ | |||
This section describes the primary root causes of AFW system component failures, as determined from a review of operating histories at Virgil C. | |||
Summer and at other PWRs throughout the nuclear industry. Section 5.1 describes experience at Virgil C. Summer. Section 5.2 summarizes information compiled from a variety of NRC sources, including AEOD analyses and reports, e information notices, inspection and enforcement bulletins, and generic letters, and from a variety of INPO reports as well. Some Licensee Event Reports and NPRDS event descriptions were also reviewed individually. | |||
Finally, information was included from reports of NRC-sponsored studies of the effects of plant aging, which include quantitative analysis of AFW system failure reports. This information was used to identify the various root causes expected for the broad PRA based failure events identified in Section 4.0, resulting in the inspection guidelines presented in Section 3.0. | |||
5.1 Viroil C. Summer Exnerience The AFW system at Virgil C. Summer has experienced failures of the AFW pumps, pump discharge flow control valves, the turbine steam admission and supply valves, turbine trip and throttle valve, pump discharge isolation valves, service water backup supply valves, and numerous system check valves. | |||
Failure modes include electrical, instrumentation and control, hardware failures, and human errors. | |||
5.1.1 Motor Driven Pump Failures There have been three events from 1983 to 1991 which involved failure of the motor driven pumps during several modes of operation. Failure modes involved instrumentation and control circuit failures, blown fuse, and human failurts during maintenance attivities. | |||
5.1.2 Turbine Driven Pump Failures Eight events from 1983 to 1991 have resulted in decreased operational readiness of the turbine driven pump. Failure causes were attributed to governor trip valve problems, bearing failure, low lube oil pressure, and operator error and procedural deficiencies in setting speed control. , | |||
5.1.3 Flow Control and Isolation Valve Failures More than sixteen events between 1983 and 1991 have resulted in impaired operational readiness of the air operated flow control and motor operated isolation valves. Principal failure causes were equipment wear, instrumentation and control circuit failures, valve hardware failures, and human errors. Valves have failed to operate properly due to failure of control components, dirty seat, overload and air leaks, Human errors have resulted in valve binding and packing *:.51 ems, i | |||
16 | |||
* I E : + | |||
4 . | |||
l 5.1.4 Check Valves l five events of check valve f ailure have occurred from 1983 to 1991. | |||
Normal wear and aging, poor design, lack of proper maintenance and human error were cited as the failure mode, resulting in leakage. | |||
p 5.2 1ndustry Wide Evoerience Human errors, design / engineering problems and rrors, and cumponent SN- < ilures are the primary root causes of AFW Syster ailures identified in a y gview of industry wide systen operating history. Common csuse failures, y | |||
t N.ch disable more than one train of this operationally redundant sy tem, are jgp | |||
: tigh1.y risk significant, and can result from all c these causes. | |||
This section identifies important common cause failuu modes, and then | |||
?rovides a broader discussion of the single failure effects cf human errors, iesign/ engineering proMems and errors, and component f ailures. Paragrapns aresenting details nf inese failure modes are coded (e.g., CCl) and etoss-j referenced by inspection items in Section 3.0. | |||
5.2.1 O mmon Cause railures The dominant cause of AFW syster multiple-train f ailures has been human error. Design / engineering errors and component f ailures have been less fre@cnt, but nevertheless significant, causes of multiple train failures. i | |||
^ f,CL Human error in the form of incorrect operator intervention it.to l autcmat '4FV >/ stem functioning during transients resulted in the temporary loss of U w ety-grade AFW pumps during events at Davis Besse (NUREG-1154, 1985) ano frojan (AE0D/T416, 196.'. Ir the Davis Besse event, improper manual initiation of the steam and feedwater M .Jre Control system (SFF.CS) led to everspei ? tripping of both turbine-driven AFW pumpt, probably due to the introduction of condensate into the AFW turbines from the long, unheated steam supply lines. (The system had never been tested with th e abnorr~1, cross-connected steam supply lineup whkh resulted.) In the Trojan event the operator incorrectly stopped both AFW pumps due to misinterpretation of MFW pump speed indication. The diesel driven pump would not restart due to a protective feature requiring complete shuaNn, and the turbine-driven pump tripped on overspeed, requiring local reset of the trip and throttle valve. In cases where manual intervention is required during tna early stages of a transient, training shccid emphasize that actions aould be oerformed methodically and deliberately to guard against * . n errors. | |||
R Valve mispes;tinring hts accounted for a significant fract wr of the hun.an errors f ailing r.uhiple trains of AF',I. This incluces closure of normally open suction valves e steam supply valves, and of isolation valves to sensors having control fun. ions. Interrect handswitch positioning and inadequate temporary wiring changes have also prevented automatic starts of multiple pumos. Factors identified in studies of mispositioning errors include failure to add newly installed valves to valve checklists, weak administrav /e control of tagging, restoration, independent verification, and locked valve logging, and inacequate adnerence to procedures. Illegible or 17 sw _ _ . . . . .. | |||
so , | |||
t confusing local valve labeling, and insufficient training in the determination of valve position may cause or mask mispositioning, and surveillance which does not exercise complete system functioning may not reveal mispositionings. | |||
E At ANO 2, both AFW pump. 'nst suction due to steam binding when they were lined up to both the CST ai,e the hot startup/ blowdown demineralizer effluent (AE00/C404, 1984). At Zion 1 steam created by running the turbine-driven pump deadheaded for one minute caused trip of a motor-driven pump sharing the same inlet header, as well as camage to the turbine-driven pump (Region 3 Morning Report, 1/17/90). Both events were caused by procedural inadequacies. | |||
E Design / engineering errors have accounted for a smaller, but significant fraction of common cause failures. Problems with control circuit design modifications at Farley defeated AFW pump auto-start on loss of main feedvater. At Zion-2, restart of both motor driven pumps was blocked oy circ:it failure to deenergize when the pumps had been tripped with an autometic start signal present (IN 8?-01, 1952). In addition, AFW control circuit design reviews at Salem and Indian Point have identified designs where failures of a single component could have failed all or multiple pumps (IN 87 34,1987). | |||
CCS. Incorrect setpoints and control circuit settings resulting from analysis errors and failures to update procedures have also prevented pump start and caused pumps to trip spuriously. Errors of this type may remain undetected despite surveillance testing, unless surveillance tests model all types of system initiation and operating conditions. A gretser fractic'1 of instrumentation and contiol circuit problems has been .dentifud during actual syst,'' operation (as. opposed to surveillance testing) than for other types of f ailu, .:5. | |||
E On two occasions at a foreign plant, failure of a balance-of. plant inverter caused f ailure of two AFW pumps. In- addition to loss of the motor driven pump whose auxiliary start relay was powered by the invertor, the turbine driven pump tripped on overspeed because the governor valve opened, allowing full steam flow to the turbine. This illustrates the importance of assessing the effects of f ailures of balance of plant ecuipment which supports the operation of critical components. The instrument air system is another example of such a system. | |||
E Multiple AFW pump trips have occurred at ! listone-3, Cook-1, Trojan and Zion-2 (IN 87-53, 1987) caused by brief, low pressure oscillations of suction pressure during pump startup . These oscillnions occurred despite the availability of adequate static NPSH. Correttive actions taken include: | |||
extending the time delay associated with the low pressure trip, removing the trip, and replacing the trip with an alarm and operator action. , | |||
CC8, Design errors discovered during AFW system reanalysis at the Robinson plant (IN 89-30,1959) and at Millstone-1 resulted in the supply header from the CSI being too small to provide adecuate N?SH to the pumps if more than one of the three pumps were ,perating at rated flow conditions. This could lead to multiple pump failure due to cavitation. Suosecuent review:. at Robinson 15 l | |||
l | |||
g as-A, -La-L_- a r-n sg L-W--Ma >4 D 4, A A | |||
- - - -- a- x a-4 t | |||
identified a loss of feedwater transient in which inadequate NP',H and flows less than design values had occurred, but which were not recognized at the time. Event analysis and equipment trending, as wc11 as surveillance testing which duplicates service conditions as much as is practical, can help identify such design errors, gpoa Asiatic clams caused failure of two AFW flow control valves at Catawba-2 when low suction pressure caused by starting of a motor-driver pump caused i suction source realignment to the Nuclear Service Water system. Pipes had not been routinely treated to inhibit clam growth, nor regularly monitored to detect their presence, and r4 strainers wera installed. The need for surveillance which exercises alternative sy' m operational modes, as well as complete system functioning, is emphasized b3 his event. Spuricut s9: tion switchover has also occurred at Callaway and at McGuire, although no f ativres resulted. | |||
CC10. Common cause f ailures have also been caused by component failures (AEOD/C404,1984). At Surry-2, both the turbine driven pump and one motor driven pump were declara. inoperable due to steam binding caused by leakage of hot water through multiple check valves. At Robinson-2 both motor driven pumps were found to be hot, and both motor and steam driven pumps wert: found to be inoperable at different times. Backleakage at Robinson 2 passed through L closed motor-operated isolation valves in addition to multiple check valves. | |||
At farley, both motor and turbine driven pump casings were found hot, although the pumps were not declared inoperable. In addition to multi-train failures, numerous incidents of single train f ailures have occurred, resulting in the designation of " Steam Binding of Auxiliary Feedwater Pumps" as Generic issue i 93. This generic issue was resolvec by Generic Letter 88-03 (Firaglia, 1988), | |||
; which required licensees to monn 'or A"W piping temperatures eacn shift, and to maintain-procedures for recognizing steam b' ding and for restoring system operability. | |||
CC11. Common cause failures have also failed mo ar operated valves. During the total lon of feedwater event at Davis Besse, the normally-open AFW i isolation valses failed to open after they were inadvertently closed. The l failure was due te improper setting of the to"que switch bypass switch, which prevents motor trip on the high torcue required to unseat a closed valve. | |||
Previous problems with these valves had been addressed by increasing the torque switch trip setpoint - a fix wnich failed during the event due to the higher torque required due to high differential pressure across the valve. | |||
l Similar common mode failures of MOVs haw also occurred in other systems, I resulting in issuance of Generic Letter 99-10, " Safety Related Motor-Operated l Valve Testing and Surveillance (Partlow,1929)." Ttis generic letter requires l licensees to develop and implement a program to provide for the testing, inspection and maintenance of all safety-related MOVs tr provide assurante that they 'ill function when subjected to oesign basis conditions. | |||
; CCl?. Other component f ailures have also resulted in AFW multi-train l failures. These include out-of-adjustment electrical flow controllers resulting in improper discharge valve operation, and a failure of oil cooler cooling water supply valves to open due to silt accumulation. | |||
19 L | |||
l | |||
. . . .-- _ _ . - . .- - -. =. | |||
~ | |||
s . | |||
t 5.2.2 Human Errors E ine overwhelmingly dominant cause of problems identified during a series of operational readiness evaluations of AFW systems was human performance. The majority of these human performance problems resulted from incomplete and incorrect procedures, particularly with respect to valve lineup information. | |||
A study of valve mispositioning events involving human error identified failures in administrative control of tagging. and loggihg, procedural | |||
, compliance and completion of steps, verification of support systems, and inadequate procedures as important. Another study found that valve mispositioning events occurred most often during maintenance, calibration, or modification activities. Insufficient training in determining valve position, and in administrative requirements for controlling valve positioning were important causes, as was oral task assignment without task completion feedback. | |||
R Turbine driven pump failures have been caused by human errors in calibrating or adjusting governor speed control, poor governor maintenance, incorrect adjustment of governor valve and overspeed trip linkages, and errors associated with the trip and throttle valve. TTV-associated errors include physically bumping it, f ailure to restore it to the correct position after testing, and failures to verify control room indication of TTV position following actuation. | |||
E Motor driven pumps have been failed by human errors in mispositioning handswitches, and by procedure deficiencies. | |||
5.2.3 Desion/Encineerine Problems and Errors D E As noted above, the majority of AFW subsystem failures, and the greatest relative system degradation, has been found to result from turbine-driven pump f ailures. Overspeed trips of Terry turbines controlled by Woodward governors hi e been a significant source of these failures (AE0D/C602, 1986). In many cases these overspeed trips hcve been caused by ' slow response of a Woodward Model EG governor on startup, at plants where full staam flow is allowed immediately. This oversensitivity has been removed by installing a startup steam bypass valve which opens first, allowing a controlled turbine acc.eleration and buildup of oil pressure to control the governor valve when tull steam flow is admitted. | |||
PRI Oversoeed trips of Terry turbines have been caused by condensate in the steam supply lines. Condensate slows down the turbine, causing the governor valve to oren farther, and overspeed results before the governor valve can respond. after the water slug clears. This was determined to be the cause of the _lcss-of-all-AFW event at Davis Besse ( AEOD/602,1986), with condensation enhanced due to the long length of the cross-connected steam lines. Repeated tests following a cold-start trip n.ay De successful due to system h at up. | |||
DI?L Turbine trip and throttle valve (TTV) problems are a significant cause of turbir.e driven pump failures (IN 84-66). In some cases lack of TTV position indication in the control room prevented recognition of a tripped TTV. In other cases it was possible to reset either the overspeed trip or the 20 | |||
a ., m, p---+4 s..mem a 2- .- 4 4 _aJ. JA e A- J--- a w -= _ s i- . a-_ iaaA. | |||
o , .. | |||
.0. | |||
,TTV without resetting the other. This problem is compounded by the fact that the positior, of the overspeed trip linkage can be misleading, and the mechanism may lack labels indicating when it is in the tripped position (AE0D/C602,1986). | |||
DE Startup of turbines with Woodaard Model PG-PL governors within 30 minutes of shutdown has resulted in overspeed trips when the speed setting knob was not exercised locally to drain oil from the speed setting cylinder. | |||
Speed control is bastd on startup with an empty cylinder. Problems have involved turbine rotation due to both procedure violations and leaking steam. | |||
Terry has marketed two types of dump valves for automatically draining the oil after shutdown (AEOD/C602, 1986). | |||
At Calvert Cliffs, a 1987 loss-of. ,ffsite-power event required a cuick, cold startup that resulted in turbine t;ip due to PG-PL governor stability problems. The short-term corrective action was installation of stiffer buffer springs (IN 88-09, 1958). Surveillance had always been preceded by turbine warmup, which illustrates the importance cf testing which duplicates service conditions as much as is practical. | |||
0 1 Reduced viscosity of gear box oil heated by prior operation caused f ailure of a motor driven pump to start due to insufficient lube oil pressure. | |||
Lowering the pressure switch setpoint solved the problem, which had not been detected during testing. | |||
p.1 Waterhammer at Palisades resulted in AFW iine and hanger damage at both steam generators. The AFW spargers are located at the normal steam generator j level, and are frequently covered and uncovered during level fluctuations. | |||
I Waterhammers in top-feed-ring steam generators resulted in main feedline l rupture at Maine Yankee and feedwater pipe cracking at Indian Point-2 (IN 84-l 32,1984). | |||
l DE Manually reversing the direction of motion of an operating valve has | |||
; resulted in MOV failures where such loading was not considered in the design l (AE00/C603,1986). Control circuit design may prevent this, requiring stroke y completion before reversal. | |||
D R At each of the units of the Sruth Texas Prosect, space heaters provided l by the vendor for use in preinstallation storage of MOVs ware found to be I | |||
wired in parallel to the Class IE 125 V DC motors for several AFW valves (IR l 50-489/89-11; 50-499/89-11, 1999). The valves had been environmentally qualified, but not with the non-safety-related heaters energized. | |||
5.2.4 Component Faibres Generic Issue II.E.6.1, *In Situ Testing Of Valves" was divided into four sub-issues (Beckjord,1989), three of which re.-te directly to prevention of AFW system component failure. At the request u the NRC, in-situ testing of check valves was adoressed by the nuclear indust.y, resulting in the EPRI report, " Application Guidelines for Check Valves in Nuclear Power Plants (Brooks,1988)." This extens1ve report provides information on check valve applications, limitations, and inspection techniques. in-situ testing of MOVs 21 | |||
q . - | |||
fi , | |||
. e : i was addressed by Generic letter'89-10, " Safety Related Motor-Operated Valve __ | |||
-Testing and Surveillance * (Partlow, 1989) which requires licensees to develop and implement =a program for test _i_ng,- inspection and maintenance of all safety-related MOVs. ' | |||
Thermal Overload Protection for Electric Motors on Safety-Related Motor operated: Valves - Generic issue II.E.6.1 (Rothberg,1988)* | |||
: concludes that valve motors _ should be. thermally protected - yet 'in a way which emphasizes system function over prctection of the operator. | |||
E The common cause steam binding effects of check valve leakage were identified in Section 5.2.1, entry CC10. humerous single-trait, svents previde | |||
-additional insights into this problem. In some cases leakage of hot MFW past multiple check valves.in series has occurred because adequate valve-seating-pressure was limited to the valves closest to the steam generators (AE00/C404, 1984). At Robinson,-the pump shutdown procedure was changed to delay closing the MOVs 'until after it.e check valves were seated. At Farley, check valves were changed from swing type to lift type. Check valve rework has bo n done at a number of plants. Different valve designs and manufacturers are involved in this problem, and recurring leakage hae been experienced, even after repair and replacement. | |||
A At Robinson, heating of motor operated valves by check valve leakage has caused thermal binding and failure of AFW discharge valves to open on demand. ' | |||
At Davis Besse,-high. differential presst're across AFW injection valves resulting from check valve leakage has prevented MOV operation (AE0D/C603, 1986). | |||
E Gross check valve leakage at McGuire and Robinson caused . | |||
-overpressurization of the AFW suction' piping. At a foreign PWR it resulted in-a severe waterhammer event. At Palo Verde-2 the MFW suction piping was overpressurized by check valve leakage from the AFW system (AEOD/C404, 1984). | |||
Gross check valve leakage through idle pumps represents a potential diversion | |||
-ef- AFW pump flow. | |||
CF4. Roughly one third of AFW system failures have been due to valve operator failures, with about equal failures -for MOVs and A0Vs. Almost half of the MOV failures were due to motor or switch failures _(Casada, 1989). An extensive study of MOV events-(AE0D/C603, 1986) indicates continuing inoperability | |||
_ problems caused by: _ torque switch / limit switch settings, adjustments, or failures; motor burnout; improper sizing or use of thermal overload devices; premature degradation related to inadequate use of. protective devices; damage E due tol misuse (valve throttling, valve operator hammering); mechanical problems (loosened parts, improper assembly); or the torque switch bypass - | |||
l circuit improperly installed or adjusted. The study concluded that current methods and procedures at trany plants are not- adequate to assure that MOVs | |||
-willtoperate when needed under credible accident conditions. Specifically, a surveillance test which the. valve passed might result in undetected valve inoperability_due to component' failure (motor burnout, operator parts failure, sten. disc separation) or improper positioning of protective-devices (thermal overload, _ torque switch, limit switch). Generic Letter 89-10 (Partlow, 1989) has subsequently recuired licensees to implement a program ensuring that MOV switch settings.are maintained so that the valves will cperate under design | |||
-basis conditions for the life of the plant. | |||
22 p ens 3 w-e--y- m-w- -- -y | |||
*7 yy-- ty -i-w T-- w f--e'w--e- u re" -- '='e'** ,r-+w 1~1 y e- --= -'w ---- | |||
sa.-_. | |||
E Component problems have caused a significant number of turbine driven pump trips (AEOD/C602, 1986). One group of events involved worc. tappet nut faces, loose cable connections, loosened set screws, improperly 'atched TTVs, and improper assembly. Another involved oil leaks due to component or seal failures, and oil contamination due to poor maintenance activities. Governor oil may not be shared with turbine lubrication oil, resulting in the need for separate oil changes. Electrical component failures included transistor or resistor failures due to moisture intrusion, erroneous grounds and connoctions, diode failures, and a faulty circuit card. | |||
& Electrohydraulic-operated discharge valves have performed very poorly, and three of the five units using them have removed them due to recurrent failures. Failures included oil leaks, contaminated oil, and hydraulic pump failures. | |||
.(172 Contral circuit failures were the dominant source of motor driven AFW pump failures (Casada,1989). This includes the controls used for automatic and manur1 starting of the pumps, as opposed to the instrumentation inputs. | |||
Most of the remaining problems were due to circuit breaker failures. | |||
CF8. " Hydraulic lockup" of Limitorque SMS spring packs has prevented proper spriag r.ompression to actuate the MOV torque switch, due to grease trapped in ine spring pack. During a surveillance at Trojan, failure of the torque switch to trip the TTV motor resulted in tripping of the thermal overload device, leaving the turbine driven pump inoperabh for 40 days until the next surveillance (AF%E702,1987). Problems result from grease changes to EXXON NEBULA EP-0 grease, one of only two greases considered environmentally qualified by Limitorque. Due to lower viscosity, it slowly migrates from the gear case into the spring pack. Grease changeover at Vermont Yankee affectec 40 of the older MOVs of which 32 were safety related. Grease relief kits are needed for MOV operators manufactured before 1975. At Limerick, additional grease relief was required for MOVs manufactured since 1975. MOV refurbishment programs may yield other changeovers to EP-0 grease. | |||
CF9. For AFW systems using air operated valves, almost half of the system degradation has resulted from failures of the valve controller circuit and its instrument inputs (Casada, 1989). Failures occurred predominantly at a few units using automatic electronic controllers for the flow control valves, with the majority of failures due to electrical hardware. At Turkey Point-3, controller malfunction resulted from water in the Instrument Air system due to maintenance incperability of the air dryer:;. | |||
CF]O. For systems using diesel driven pumps, most of the failures were due to start control and governor speed control circuitry. Half of these occurred on demand, as opposed to during testing (Casada, 1989). | |||
CFil For systems using A0Vs, operability requires the availability of Instrument Air (IA), backup air, or backup nitrogen. However, NRC Maintenancc l- Team Inspections have identified inadequate testing of check valves isolating l- the safety-related portion of the IA system at several utilities (Letter, Roe i to Richardson). Generic Letter 88-14 (Miraglia,1988), recuires licensees to verify by test that air-operated safety-related components will perform as 23 | |||
.o - | |||
, g c | |||
expected in accordance with all design basis events, including a loss of nortnal IA. | |||
f l 24 l | |||
~ | |||
[ -1 C.,* , | |||
l= | |||
==6.0 REFERENCES== | |||
Beckjord, E. 5. Jane 30, 1989. | |||
Testina of Valves". Letter to V.Closecut of Generic Issue II.E.6.1. "In Situ Stello, Jr., U.S. Nuclear Regulatory Commission, Washington, DC. | |||
Brooks, B. P. 1988. | |||
Aeolication Guidelines for Check Valves in Nuclear Power Plants. NP-S479, Electric | |||
, Power Research Institute, Palo Alto, CA. | |||
Casada, D. A. '1989. Auxiliary Feedwater System Aoine Study. | |||
Volume 1. | |||
Doeratino Experience and Current Monitorina Practices. NUREG/CR-5404. U.S. | |||
Nuclear Regulatory Commission, Washington, DC. | |||
Gregg, R. E, and R. E. Wright. 1988. Aroendix Review for Dominant Generic Contributprt. BLB-31-88. Idaho Nati)nal Engineering Laboratory, Idaho Falls, Idaho. | |||
Miraglia, F. J. February 17, 1988. Resolution of Generic Safetv Issue 93, | |||
" Steam Bindino of Auxiliarv Feedwater Pumps * (Generic Letter 88-03). U.S Nuclear Regulatory Commission, Washington, DC. | |||
Miraglia, F. J. August 8, 1988. Jnstrument Air Sucolv System Problems Affectina Safetv-Related Ecuigme-t (Generic letter 88-14). U.S. Nuclear Regulatory Commission, Washington, DC. | |||
Partlow, J. G. June 28, 1989. Safetv-Related Fotor-Operated Valve Testino and Surveillance (Generie Letter 89-10). U.S. Nuclear Regulatory Commission, Washington, DC. | |||
Rothberg, O. June 1988. | |||
Thermal Overload Protection for Electric Mocers on Safety-Rel.ated Motor-Ocerated valves - Generic issue II.E.6.1. NUREG-1296. | |||
U.S. Nuclear Regulatory Commission, Washington, DC. | |||
Travis, R. and J. Taylor. -1989. Develooment of Guidance for Generic. | |||
Functional v Oriented PRA-Based Team Inspections for BWR Plants-Jdentification of Risk-1moortant Svstems. Components and Human Actions. TLR-A-3874-TGA Brookhaven National Laboratory, Upton, New York. | |||
AE00 Reoorts AE0D/C404. W. D. Lanning. July 1984. Steam Bindinc of Auxiliarv Feedwater Pumps. | |||
U.S. Nuclear Regulatory Commission, Washinoton, DC. | |||
AE0D/C602. C. Hsu. August 1985. Operational Exoerience Nvolvine Turbine Oversceed Trios. U.S. Nuclear Regulatory Commission, Wasnington, DC. | |||
AEOD;T605. E. J. Brown. December 1986. A Review of Motor-Operated Valve Performante. U.S. Nuclear Regulatory Commission, Washington, DC. | |||
25 | |||
e - | |||
~ | |||
O . ,a | |||
+ | |||
L. ' | |||
O AEOD/E702. E. J. Brown. March 19, 1987. MOV Fiilure Due to Hydraulic locku-From Excessivt Grease in Sprino Pack. U.S. huclear Regulatory Commission, Washington, DC. | |||
AEOD/T416. January 20, 1983. Loss of ESF Auxiliary Feedwa(tt. Pump Capability at Tro.ian on January 22. 1963. U.S. Nuclear Regulatory Commission, Washington, DC. | |||
Information Notices IN 82-01. January 22, 1982. Auxiliarv Feedwater Pumn Lockout Resultino from Westinchouse V-2 Switch Circuit Modification. U.S. Nuclear Regulatory Commission, Washington, DC. | |||
IN 84-32. E. L. Jordan. April 18, 1984. Auriliary Feedwater Sparcer and Pioe Hancar Damace. U.S. Nuclear Regulatory Commission, Washington, DC. | |||
IN 84-66. August 17, 1984. Undetected Unavailabilitv of the Turbire-priven Auxiliary Feedwater Train. U.S. kuclear Regulatory Commission, Washington, DC. | |||
IN 87-34. C. E. Rossi. July 24, 1987. Sincle Failures in Auxiliarv Feedwater Systers. U.S. Nuclear Regulatory Commission, Washington, DC. | |||
IN 87-53. C. E. Rossi. October 20, 1957. Auriliarv Feodwater Pume Trios Resultino from Low Suction Pressure. U.S. Nuclear Regulatory Commission, l Wasnington, DC. | |||
IN 88-09. C. E. Rossi. March 18, 1938. Reduced Reliab41itv of Stear-Driven | |||
! Auxiliarv Feedwater Pumos Caused bv Instabilitv of Wooaware PG-PL Type Governors. U.S. Nuclear Regulatory Commission, Washington, DC. | |||
IN 89-30. R. A. Azua. August 16, 1989. Robinson Unit 2 Inadeouate NPSH of A0xiliarv Feedwater Puros. Also, Event Notification 16375, August 22, 1989, i U.S. Nuclear Regulatory Commission, Washington, DC. | |||
l l | |||
l Inspection Report | |||
~ | |||
sR 50-489/89-11; 50-499/89-11. May 26, 1989. South Texas Prniett Inspection Report. U.S. Nuclear Regulatory Commission, Washington, DC. | |||
NUREG Report 1 | |||
! NUREG-ll54. 1985. Loss of Mair and Auriliarv Feedwate* Event at the Davis Besse Plar* <.* June 9. 1985. U.S. Nuclear Regulatory Commission, Washington, 26}} |
Latest revision as of 06:51, 24 July 2020
ML20113H105 | |
Person / Time | |
---|---|
Site: | Summer ![]() |
Issue date: | 11/30/1991 |
From: | Gore B, Lloyd R, Moffitt N Battelle Memorial Institute, PACIFIC NORTHWEST NATION |
To: | Office of Nuclear Reactor Regulation |
Shared Package | |
ML20113H085 | List: |
References | |
CON-FIN-L-1310 NUDOCS 9207070372 | |
Download: ML20113H105 (26) | |
Text
- . . =
- , Ef4 CLOSURE NUREG/CR-P!it -
i AUXILIARY FEEDWATER SYSTEM RISK-BASED INSPECTION GUIDE FOR THE VIRGIL 0. SUMMER NUCLEAR POWER PLAtiT R. C. Lloyd N. E. Moffitt B. F. Gore T. V. Vo November 1991 Prepared for Division of Radiation Protection and Emergency Preparedness Office of Nuclear Regulatory Regulation U.S. Nucleer Regulatory Commission Washington, DC 20555 NRC FIN L1310 Pacific Northwest Laboratory Richland, Washington 99352 9207070372 920629 'S P Di< ADOCK 0500 F
. l SUu m Y This document pre.ents a compilation of auxiliary feedwater (AFW) system failure information which has been screened for risk significance in terms of failure frequency and degradation of system performance. It is a risk-prioritized listing of f ailure events and their causes that are significant enough to warrant consideration in inspection planning at the Virgil C. Summer
. plant. This information is presented to provide inspectors with increased resources for inspection planning at Virgil C. Summer. l The isk importance of various component failure modes was identified by analysis of the results of probabilistic risk assessments (PRAs) for many pressurized water reactors (PWRs).
However, the component failure categories !
identified in PRAs are rather broad, because the failure data used in the PRAs 3 is an aggregate of many individual failures having a variety of root causes. ;
In order to help inspectors focus on specific aspects of component operation, '
maintenance and design which might cause these failures, an extensive review of component failure information was performed to identify and rank the root causes of these component failures. Both Virgil C. Summer and industry Hde failure information was analyzed. Failure causes were sorted on the basis of frequency of occurrence and seriousness of consecuence, and categorized as common cause failures, human errors, design problems, or component f ailures.
This information is presented in the body of this document. Section 3.0 provide brief descriptions of these risk important failure causes, and Section 5.0 presents more extensive discussions, with specific examples and references. The entries in the two sections are cross-referenced.
An abbreviated system walkdown table is presented in Section 3.2 which includes only components identified as risk important. This table lists the system lineup for normal, standby system operation.
This information permits an inspector to concer,trate on components important to the prevention of core damage. However, it is important to note-that inspections sh;ald not focus exclusively on these components. Other components which perform essential functions, but which are not included because of high reliability or redundancy, must also be addressed to ensure that degradation does not increase their failure probabilities, and hence their risk importance.
iii I
6
[pNTENTS n
SUMMARY
.............................................................. iii ,
1.0 INTRODUCTION
....... ............................................ 1 2.0 VIRGIL C. SUMMER AFW SYSTEM...................................... 2 i
2.1 SYSTEM DESCRIPTION ......................................... 2 2.2 SUCCESS CRITERION .......................................... 4 2.3 SYSTEM DEPENDENCIES ........................................ 4 2.4 OPERATIONAL CONSTRAINTS .................................... 4 3.0 INSPECTION GUIDANCE FOR THE VIRGIL C. SUMMER AFW SYSTEM.......... 5 3.1 RISK IMPORTANT AFW COMPONENTS AND FAILURE MODES ............5 3.1.1 MULTIPLE PUMP FAILURES DUE TO COMMON CAUSE .......... 5 3.1.2 TURBINE DRIVEN PUMP FAILS TO START OR RUN ..................................... 6-3.1.3 MOTOR DRIVEN PUMP A OR B FAILS TO START OR RUN ..... .................................. 7 3.1.4 PUMP UNAVAl'LABLE DUE TO MAINTENANCE OR SUPVEILLANCE .................. 7 3.1.5 AIR OPERATED ISOLATION AND FLOW CONTROL VALVE FAILURE ....................................... 7 3.1.6 MOTOR OPERATED VALVE FAILURE ........................ 8 3.1.7 MANUAL SUCTION OR DISCHARGE VALVES Fall CLOSED ......................................... 9 3.1.8 LEAKAGE OF HOT FEE 0 WATER THROUGH CH VALVES ............................ECK .................. 9 3.2 RISK IMPORTANT AFW SYSTEM WALKDOWN-TABLE ................... 10 4.0- GENERIC RISK INSIGHTS FROM PRAs ................................, 14 4.1 RISK IMPORTANT ACCIDENT SEQUENCES INVOLVIN AFW SYSTEM FAILURE .......................,.................. 14 4.2 RISK IMPORTANT COMPONENT FAILURE MODES ..................... 15 V
, , - - , . _ . . . ~ . , _ _ _ . _ . . _ . --
m .
CONTENTS (continued) 5.0 FAILURE MODES DE1 ERMINED FROM OPERATING EXPERIENCE .............. 16 1 5.1 VIRGIL C. SUMMER EXPERIENCE................................. 16 ,
l i 5.1.1 MOTOR DRIVEN PUMP FAILURES .......................... 16 5.1.2 TURBINE DRIVEN PUMP FAILURES ........................ 16 5.1.3 FLOW CONTROL AND ISOLATION VALVE FAILURES ........... 16 5.1.4 CHECK VALVES ........................................ 17 1 5.2 INDUSTRY WIDE EXPERIENCE ................................... 17 5.2.1 COMMON CAUSE FAILURES ............................... 17 5.2.2 HUMAN ERRORS ........................................ 20 5.2.3 DESIGN / ENGINEERING PROBLLMS AND ERRORS .............. 20 5.2.4 COMPONENT FAILURES .................................. 21 REFERENCES ........................................................... 25 I-i I I I vi l
i
~
1.0 INTRODUCTION
This document is one of a series providing plant-specific inspection guidance for auxiliary feedwater (AFW) systems at pressurized water reactors (PWRs). This guidante is based on information from probabilistic risk assessmentt (PRAs) for similar PWRs, incustry wide operating experience with AFW systems, plant specific AFW system descriptions, and plant specific
, operating experience, it is not a detailed inspection plan, but rather a compilation of AFW system failure information which has been screened for risk significance in terms of failure frequency and degradation of system performance. The result is a risk-prioritized listing of failure events and the causes that are significant enough to warrant consideration in inspection planning at Virgil C. Summer.
This inspection guidance is presented in Section 3.0, following a description of the Virgil C. Summer AFW system in Section 2.0. Section 3.0 identifies the risk important system components by Virgil C. Summer identification number, followed by brief descriptions of each of the various failure causes of that component. These include specific human errors, desion deficiencies, and haroware failures. Tne discussions also identify where common cause failures have affected multiple, redundant components. These brief discussions identify specific aspects of system or component design, operation, maintenance, or testing for inspection by observation, recoros review, training observati e procedures review, or by observation of the implementation of procedur e An AFW system walkdown table identifying risk important components and their lineup for normal, standby system operation is also previoed.
The remainder of the document describes and discusses 14 information used in compiling this inspection guidance. Section 4.0 describes the risk important information which has been derived from PRAs and its sources. As review of that section wlll show, the failure events identified in PRAs are rather broad (e.g., pump f ails to start or run, valve f ails closed). Section 5.0 addresses the specific failure causes which have been combined under these broad events.
AFW syster operating history was studied to identify the various specific failures which have been aggregated into the PRA failure events. Section 5.1 presents a sunc,ary of Virgil C. Summer f ailure information, and Section 5.2 presents a review of industry-wide failure information. The industry-wide information was compiled from a variety of HRC sources, including /E0D analyses and reports, information notices, inspection and enforcement bulletins, and generic letters, and from a variety of INPO reports as well.
Some Licensee Event Reports and NPRDS event descriptions were also reviewed.
Finally, infort.ation was incluoed from reports of HRC-sponsored studies of the effects of plant aging, which incluoe cuantitative analyses of reported AFW system failures. This industry-wide information was then combined with the plant-!pecific failure infurmation to identify the various root causes of the broad failure events used in PRAs, which are ioentified in Section 2.0.
1
2.0 VIRGIL C. SV%' tJW SYSTEM This section presents an overview description of the Virgil C. Summer AFW system (Westinghouse 3 loop plant), including a simplified schematic system diagram, in addition, the system success criterion, system dependencies, and administrative operational constraints are also presented.
2.1 System Description The ATW system provides feedwater to the steam generators (SG) to allow secondary-side heat removal from the primary system when main feedwater is unava ilable. The system is capable of functioning for extended periods, which allows time to restore main feedwater flow or to proceed with an orderly cooldown of the plant to where the residual heat removal (RHR) system can remove decay heat. A simplified schematic diagram of the Virgil C. Summer AFW system is shown in Figure 2.1.
The AFW system consists of two motor driven (MD) pumps and one steam-driven (TD) pump along with the associated piping, valves and instrumentation normally connected to the Condensate Storage Tank (CST). It is designed to start up and establish flow automatically. All pumps start on receipt of a steam generator low low level signal. (The motor criven pumps start on low level in one SG, whereas, two low level signals are reouired for the steam-driven pump to start.) Also, the motor driven pumps start en a trip of main feedwater pumps (MFW) pumps, a safety injection signal, undervoltage on either ESF Bus DA or DB, or on ATWS Mitication System Circuit Activation (AMSAC). ine single turbine-driven (TD) pump also starts on undervoltage o.1 the ESF busses DA and DB, and on an AMSAC SIGNAL.
A suction line from the CST provides a common header that supplies water to the turbine-driven pump and to both motor driven pumps. Isolation valves in these lines are locked open. Power, control, and instrumentation associated with each motor-driven pump are independent from one another.
Steam for the turbine-driven pump is supplied by steam generators B and C, from a point upstream of the main steam isolation valves, through valve 2865.
Each AFW pump is equipped with a continuous recirculation flow system, which prevents pump deadheading.
- The discharges of the motor driven pumps are cross connected and each pump can feed all three steam generators. The turbine-driven pump also feeds all three steam generators througn separate lines. Each of these six lines contains an air operated discharge flow control valve. Motor driven and turbine driven pump flow control valves are 3531,3541,3551, and 3536, 3546, 3556, respectively. Safety class air accumulators for-each flow control valve provide sufficient air capacity to permit remote valve closure for isolation of a seconcary system break. Each of the lines from the motor driven pumps also contains a manually operated discharge isolation valve,1017 A,B,C.
Discharge isolution valves in the lines from the turbine-driven pump are manually operated valves,101E A.E,C, AFW containment isolation valves are provided by air assist-spring operated check valves 1009 A, E, C. Each AFW 2
i A-
~
{ '1 i
i i:
1017A to.9A ORC f IRC
- g/gs /
l 1009A
- J ,
CorMf.
1018A 1020A lL"i e 'x-#:-t+
l
~
3536 !j
.L. '
1007 %1010 -
i ~~
10250 /
101TB 10198 #
- 03 A 001A ,
( ' fMi 1018R 101 El
( S""* ] A _
102tA f t 3546 / .
wasos l a
v M h 1026 tot 1C 1019C /
toop s + . _p'l _N \_M IN $
j tr 1B 10018 s,
'M10210 3551 #
s 1018C 1020C ,,
B #
ep 3556 f
/T 1006
._Q -
Cham H M-OfD)
/
[
Il P /
S am 1012 2002A 20T8 p G D) g,C C) staan M
tsoterm Gowwcor ;
Desun ' Vdve Vda 2813 &
E 2ees i i
FIGURE 2.1 V.C. SUMMER EMERGENCY FEEDWATER SYSTEM
_I
[
- \
line also contains several check valves to prevent leakage from the feedwater lines.
The condensate storage tank (CST) is the normal source of water for the AFW System and is required to store sufficient demineralized water to maintain the reactor coolant system (RCS) at hot standby conditions for 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> (172,000 gallons). All tank connections except those required for instrumentation, auxiliary feedwater punp suction chemical analysis, and tank
, drainage are located above this minimum level. Backup AFW supply is automatically provided by the service water system. On a low suttion pressure condition of 11.0 psig for 5 seconds, service water valves, 1001A & B and 1037 A & B will open automatically to supply Loop A and B.
I 2.2 Success Criterien l System success requires the operation of at least one pump supplyin;
' rated flow to at least two of the three steam generators.
2.3 Svsten_pecendencies Tne AFW system depends on At and DC power at various voltage levels for motor operation, valve control, monitor and alarm circuits, and valve / motor control circuits. Instrument Air is reouired for governor speed control and flow control valve operation. Stet availability is required for the turbine-driven pump.
2.4 Ooerational Constreints The Virgil C. Summer Technical Specifications recuire that all three AFW pumps and associated flow paths are operable with each motor driven pump powered from a different vital bus and one turbine driven pump enpable ef being powered from an operable steam supply system. If one AFW pump becomes inoperable, it must be restored to operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or the plant must shut down to hot standby within the next six hours and in Hot Standby within the following six hours. If two AFW pumps are inoperable, be in hot standby within six hours and in het shutdown within the following six hours. If three AFW pumps are inoperable, immediately initiate corrective action to restore one pump to operable status as soon as possible.
The Virgil C. Sum.ner Technical Specifications require a minimum supply of 172,000 gallons of water to be storea in the CST to maintain the RCS at het standby condition for 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> with steam discharge to atmosphere concurrent with total loss of offsite power.
4 l
3.0 INSoECT10N GUIDul:E FOR THE V100lt C. SUMMEP Arp SYSTEM in this section the risk important components of the Virgil C. Summer AfW system are identified, and the important failure modes for thesa components are briefly described. These f ailure modes include specific human errors, design deficiencies, and types of hardware f ailures which have been observed to occur for these components, both at Virgil C. Sumer and at PWRs throughout the nuclear industry. The discussions also identify where common cause f ailures have affected multiple, redundant components. These brief discussions identify specific aspects of system or component design, operation, maintenance, or testing for inspection activities. These activities include; observation, records review, training observation, procedures review, or by observation of the implementation of procedures.
Table 3.1 is an abbreviated ATW system walkdown table which identifies risk-important components. This table lists the system lineup for normal (standby) system operation. Inspection of the components identified in the AFW walkdown table adoresses essentially all of the risk associated with AFW system operation.
3.1 Risk leportant ArV Co nonenti and ra ilure Mod u Common cause f ailures of multiple pumos are the most risk-important failure mod: of AFW system components. These are followed in importance by 1
single pump ilures, level control valve f ailures, and individual check valve leakage failures.
The following sections address each of these f ailure modes, in decreasing order of risk-importance. They present thc- important root causes of these component failure modes which have been distilled from historical records.
Each item is keyed to discussions in Section 5.2 where additional information on historical events is presented.
3.1.1 Multiple Plime Failures due to Comon Cause The following listing summarizes the most important multiple-pump failure modes identified in Section 5.2.1, Common cause Failures, and each item is keyed to entries in that section.
- Incorrect operator intervention into automatic system functioning, including improper manual starting and securing of pumps, has caused failure of all pumps, including overspeed trip on~startup, and inability to restart prematurely secured pumps. CC1. '
Valve mispositioning has caused failure of all pumps. Pump suction,
, steam supply, and instrument isolation valves have been involved.
CC2.
- Steam binding has caused failure of multiple pumps. This resulted from leakage of hot feeowater past check valves and a motor-operated valve into a common discharge neader CC10. Multiple c amp steam 5
I s-.. . ~ . . ,r-,-- -,r_-,wm,m._,m.___ _ ._--_,,..,..,,w%,,,, ,m,,,,-..my,y._,,,-,,r.,_,,,,,,..,,,,-,.,,,-m-,,,m-.-g,-,,,-y_,,,, -
binding has also resulted from improper valve lineups, and from running a pump deadheaded. CC3.
. Pump control circuit deficiencies or design modification errors have caused f ailures of multiple pumps to auto start, spurious pump trips during operation, and failures to restart after pump shutdown. CC4.
Incorrect setpoints and control circuit calibrations have also prevented proper operation of multiple pumps. CCS.
t
. Loss of a vital power bus has failed both the turbine-driven and one motor driven pump due to loss of control power to steam admission valves or to turbine controls, and to motor controls powered from the same bus. CC6.
. Simultaneous startup of multiple pumps has caused oscillations of pump suction pressure causing multiple-pump trips on low suction pressure, despite the existence of adequate static net positive suction head (NPSH). CC7. At H.B. Robinson, design reviews have identified inadequately sized suction piping which could have yielded insufficitnt NPSH to support operation of more than one pump. CCS.
3.1.2 Turbine Driven Port re nt te Start or Run
. Improperly adjusted and inadeouately maintained turbine governors have caused pump failures. HE2. Problems include worn or loosened nuts, set screws, linkages or cable connections, oil leaks and/or contamination, and electrical f ailures of resistors, transistors, diodes and circuit cards, and erroneous grounds and connections.
CF5. Governor problems, bearing wear, low oil, and human error in making proper settings have caured failure of the turbine driven pump at Virgil C. Summer.
~
. Terry turbines with Woodward Model EG governors have been found to overspeed trip if full steam flow is allowed on startup.
Sensitivity can De reduced if a startup steam bypass valve is sequenced to open first. del.
. Turbines with Woodward Model PG-PL governors have tripped on overspeed when restarted shortly after shutdown, unless an operator has locally exercised the speed setting knob to dr.in oil from the governor speed setting cylinder (per procedure). Automatic oil dump valves are now available through Terry. DE4.
. Condensate slugs in steam lines have caused turbine overspeed trip on startup, lests repeated right after auch a trip may fail to indicate the problem due to warming and clearing of the steam lines.
Surveillance should exercise all steam supply connections. DE2.
. Trip and throttle valve (TTV) problems which have failed the turbine driven pump include physically bumping it, f ailure to reset it following testing, and f ailuces to verifv control icor indication of 6
s reset. HE2, Whether either the overspeed trip or TlV trip can be reset without resetting the other, indication in the control room of TTV position, and unambiguous local indication of an overspeed trip affect the likelihood of these errors. DES. The TTV at the Virgil C.
Summer plant has failed to reset due to misalignment of pins which prevented reengagement of the trip lever.
3.1.3 Motor Driven Pumo A or B Fails to Start or Run
. Control circuits used for automatic and manual pump starting are an important cause of motor driven pump failures, as are circuit breaker failures. CF7 Control circuit problems and a blown fuse due to overload have occurred at Virgil C. Summer. -
. Hispositioning of handswitches and procedural deficiencies have prevented automatic pump start. HE3.
. Low lubrication oil pressure resulting from heatup due to previous operation has prevented pump restart due to failure to satisfy the protective interlock. DE5.
3.1.4 Puro Unavailable Due to Maintenance or Surveillance
. Both scheduled and unscheduled maintenance remove pumps from operability. Surveillance requires operation with an altered line-up, although a pump train may not be declared inoperable during testing. Prompt scheJuling and performance of mainte:.ance and surveillance minimize this unavailability.
3.1.5 Air Ooerated Isolation and Flow Control Valve Failure TD Pumn Train: 3536.3546.3556. 1009A.B.C 2030 MD Pumon Trains A: B: 353].3541.3551.1009 A.B.C These normally open air operated valves (A0Vs) isolate and control flow to the steam generators. They f ail open on loss of instrument air.
. Control circuit problems have been a primary cause of failures, both at Virgil C. Summer and elsewhere. CF9. Valve ft lu es have resulted from blown fuses, failure of control components (st;h as current / pneumatic convertors), broken and dirty contacts, misaligned or broken limit switches, control power loss, and calibration problems. Degraded operation has also resulted from improper air pressure due to air regulator failure or leaking air lincs.
Out-of-adjustment electrical flow controllers have caused improper valve coeration, affecting multiple trains of AFW, CC12 7
l
. 1 Leakage of hot feedwater through check valves has caused thermal binding of flow control MOVs. A0Vs may be similarly susceptible. CF2. ;
. Multiple flow control valves have been plugged by clams when suction l switched automatically to an alternate, untreated source. CC9 i I
3.1.6 Motor Ooerated Valve Failure Loon A Blekuo Suction Sources: 1037A.1001A.1008 Loco B Backun Suction Sources: 10378.10018. l qQ2 These MOVs control or isolate flow of the service water to the AFW pumps.
They fail as is on loss of power. ,
Commen cause failure of MOVs has resulted from failure to use electrical signature tracing equipment to determine proper settings
- of torque switch and torque switch bypass switches. Failure to calibrate switch settings for high torques necessary under design j basis accident conditiuns has also been involved. CCli. Diaphragm ,
failure, packing leakage, air leakage, electrical component f ailure and seat leakage have been the main causes of valve failure at Virgil C. Summer.
Valve motors have been failed due to lack of, or improper sizing or use of thermal overload protective devices. Bypassing and oversizing should be based on proper engineering for desion basis conditions. CF4.
At Virgil C. Summer, heating of motor operated valves by check valve leakage has caused thermal binding and failure of AFW discharge valves to open on-demand. CF2.
Out-of-adjustment electrical flow controllers have caused improper discharge valve operation, affecting multiple trains of AFW. CCl2, Grease trapped in the torque switch spring pack of Limitorque SMB motor operators has caused motor burnout or thermal overloed trip by preventing torque switch actuation. CF8.
' Manually reversing the direction of motion of operating MOVs has overloaded the motor circuit. Operating procedures should provide
- cautions, and circuit designs may prevent reversal before each stroke is finished. DE7.
Space heaters designed for preoperation storage have been found wired in parallel with valve motors which had not been environmentally qualified with them present. DEB.
8 c-- - - + - , . , w. ,.,y.- --n-, ,me- _ ,-,,o,~.,-
~
3.1.7 Manual Suction or Discha oe Valves Faii Closfj CST Suction and Recirculation Valves: 1007.1010.1025A.B.1Mi TD Pume Train: 1012.1036. 1018A.B.C HD Pume Train A: 1011A.102]A. 1017A B.C MD Puer Train B: 10))B.102]E. 1017A.B.C These manual valves are all normally locked open. For each train, closure
, of the first valve listed would block pump suction and closure of the second valves would block pun discharge.
. Valve mispositioning has resulted in failures of multiple trains of AFW. CC2. It has also been the dominant cause of problems identified during operational readiness inspections. HEl. Events
-have occurred most often-during maintenance, calibration, or systcm modifications, important causes of mispositioning include:
. Failure to provide complete, clear, and specific procedures for tasks and system restoration
. Failure to promptly revise and v:lidate procedures, training, and diagrams following system modifications
. Failure to complete all steps in a procedure
. Failure to adeouately review uncompleted procedural steps after task completion
. Failure to verify support functions after restoration
. Failure to adhere scrupulously to acministrative
_ procedures regard,ng tagging, control and tracking of valve operations
! . Failure to log the manipulation of sealed valves Failure to follow good practices of written task assignment and feedback of task completion information Failure to provide easily read system drawings, legible valve labels corresponding to drawings and procedures, and labeled indications of local valve position i
3.1.8 Leakace of Hot Feedwater throuch Check Valves:
i i MD Pump A: 1019A.B.C l MD Pumo B: 1019 A . B .,1
! TD Pumet 1020A.B.C l- Containment lsolation Stop Check Valver 1009A.B.C Leakage of hot feedwater throuah several check valves in series has caused steam binding of multipic pumps. Leakage through a closed level control valve in series with check valves has also occurred at Virgil C. Summer, as would be recuired for leakage to reach the motor driven pumps A and B. CC10.
i Slow leakage past the final check valve of a series may not force l
the check valve closed. Other check valves in seri.es may leak similarly. Piping orientation and valve design are important fatters in achieving true series protection. CFl.
9
t 3.2 Pisk 1reertant tJV Syster Walkdown Table-Table 3.1 presents an AfW system walkdown table including only components identified as risk important. This information allows inspectors to concentrate their efforts on components important to prevention of core damage. However, it is essential to note that inspections should not focus exclusively on these components. Other components which perform essential
- functions, but which are absent from this table because of high reliability or
, redundancy, must also be addressed to ensure that their risk importance are not increased. An example would include ensuring an adequate water level in the CST exists.
4 10 grec +e.es -,= 3 -,mw-rter--a.ww%,..-se %y.r.3 y .-.w ...,--yyp-a..-,.-- +,y-w.- -.ew.w=,-m---*--y--~- ,-w,-c..=-~,,---4,-;.,---,.-c-<-v- -
,e,r y ,,sw s-
~ .
t 1
i 4
TABLE 3.1. Risk Important Walkdown Table for j Virgil C. Summer AfW System Components Required Actual Component a Component Name Position Position IIectrical i
A Hotor Driven Pump Racked In/
Closed l B Motor-Driven Pump Racked In/ _
Closed -
Valve 1010 CST Outlet Valve Locked Open 1007 CST Outlet Bypass Valve Locked Open 1009A Containment Isolation to S/G A Closed / Norm 1009B Containment Isolation to S/G B Closed / Norm 3009C Centainment Isolation to S/G C Closed / Norm 1012 TDP Suction Valve Locked Open 1036 TD FWP Discharge Valve Locked Open 1018A TD FWP Supply to S/G A Locked Open 1018B TD FWP Supply to S/G B Locked Open -
1018C TD FWP Supply to S/G C Locked Open ._
1026 TDP Recire Isolation Locked Open 3536 TDP Flow Control to S/G A Open 3546 TDP Flow Control to S/G B Open 3556 TDP Flow Control to S/G C Open 1011A MDp *A* Suction Locked Open 1011B MDP *B* Suction Locked Open 1025A MDP *A* Recirculation Locked Open IC25B MDP *E' Recirculation Locked Open 22 l
i
..~ . . . - . _ . . . - . - - . - . - - _ - . . _ . _ . _ - - - _ . - _ . - . . - . . - . _
~ .
JABLE 3.1. Risk Important Walkdown Table for Virgil C. Summer AFW System Components (Continued)
MDP A Discharge Locked Open 1022A 1021B MDP B Discharge Locked Open r 1017A MDP Supply to S/G A Locked Open 1017B MDP Supply to S/G B Locked Open 1017C MDP Supply to S/G C Locked Open 3531 MDP Flow Control to S/G A Open 3541 MDP flow Control to S/G B Open 35$1 MDP Flow Control to S/G C Open 2802A MS Supply to TDP Open/ Auto 2802B MS Supply to TDP Open/ Auto 2030 Steam Isolation Valve Closed / Auto 2034 TDP Governor Valve Open __ _
2865 Steam Supply Throttle Valve Open 1037A SW Loop A Supply Valve Closed / Auto _
1037B SW Loop B Supply Valve Closed / Auto 2001A SW Loop A to MDP A Closed / Auto __
1001B SW Loop B to MDP B Closed / Auto 1008 SW Loop A to TDP Closed / Auto 1002 SW Loop B to TDP Closed / Auto 1019A MDP Supply Stop Ck to S/G A Locked /Open 1019B liDP Supply Stop Ck to S/G B Locked /Open 1019C MDP Supply Stop Ck to S/G C Locked /Open 1020A TDP Supply Stop Ck to S/G A Locked /Open 1020B TDP Supply Stop Ck to S/G B Locked /Open
! 12
o TABLE 3.1. Risk Important Walkdown Table for Virgil C. Summer AfW System Components (Continued) 1020C TDP Supply Stop Ck to S/G C Locked /Open 1009A Piping Upstream of Check Valve Cool i 1009B Piping Upstream of Check Valve Cool 1009C Piping Upstream of Check Valve Cool 13
--_ ~ - _ . - .-
8 h
4.0 GENERIC RISK INSIGHTS FROM PRAs PRAs for 13 PWRs were analyzed to identify risk important accident sequences involving loss of AFW, and to identify and risk-prioritize the component failure modes involved. The results of this analysis are described in this section. They are consistent with results reported by INEL and BNL (Gregg et al 1988, and Tr evis et al,1988).
4.1 Risk Important Accident Seauences involvina AFW System Failure Loss of Power System
. A loss of offsite power is followed by failure of AFW. Due to lack of actuating power, the power operated relief valves (PORVs) cannot be opened preventing adequate feed-and bleed cooling, and resulting in core damage.
. A station blackout fails all AC power except Vital At from DC invertors, and all decay heat removal systems except the turbine.
driven AFW pump. AFW subsecuently fails due to battery depletion or hardware failures, resulting in core damage.
. A DC bus faili, causing a trip and failure of the power conversion system. One AFW motor-driven pump is failed by the bus loss, and the turbine-driven pump f ails due to loss of turbine or valve control power. AFW is subsequently lost completely due to other failures. Feed-and bleed cooling f ails because PORV control is lost, resulting in core damage.
Transient-Caused keactor or Turbine Trir
. A transient-caused trio is followed by a loss of the power conversion system (PCS) and AFW. Feed-and-bleed cooling f ails either due to f ailure of the operator to initiate it, or due to hardware failures, resulting in core damage.
Less of Main Feedwater
. A feedwater line break drains the common water source for MFW and The operators fail t: . ' vide feedwater from other sources, AFW.
and fail to initiate feed-a. sleed cooling, resulting in core damage.
. A less of main feedwattr trips the plant, and AFW fails due to operator error and haroware failures. The operators fail to initiate feed-and-bleed cooling, resulting in core damage.
Steam Generator Tube Ruoture (SGTP)
. A SGTR is followed by failure of AFW. Coolant is lost from the primary until the refueling water storage tank (RWST) is cepleted.
14
=
e High pressure iniection (HPI) f ails since recirculation cannot be established from the empty sump, and core damage results.
4.2 Risk Important Comoonent Failure Modes The generic component failure modes identified from PRA analyses as important to AFW system failure are listed below in decreasing order of risk i importance.
- 1. Turbine-Driven Pump Failure to Start or Run.
- 2. Motor-Driven Pump failure to Start or Run.
- 3. TDP or HDP Unavailable due to Test or Maintenance.
- 4. AFW System Valve Failures
- steam admission valves a trip and throttle valve
- flow control valves
. pump discharge valves
. pump suction valves
- valves in testing or maintenance.
- 5. Supply / Suction Sources
. condensate storage tank stop valve
. hot well inventory
. suction valves.
In addition to individual hardware, circuit, or instrument failures, each of these failure modes may result from common causes and human errors.
Common cause failures of AFW pumps are particularly risk _important.
Valve f ailures _ are somewhat less important due to the multiplicity of steam generators and connection, paths. Human errors of greatest risk importance involve: failures to initiate or control system operation when required; f ailure to restore proper system lineup after maintenance or testing; and f ailure to switch to alternate sources when required.
15
s . .
a e
5.0 FAltVRE HDDES DETERMINED FROM OPERATING EXPERifNC[
This section describes the primary root causes of AFW system component failures, as determined from a review of operating histories at Virgil C.
Summer and at other PWRs throughout the nuclear industry. Section 5.1 describes experience at Virgil C. Summer. Section 5.2 summarizes information compiled from a variety of NRC sources, including AEOD analyses and reports, e information notices, inspection and enforcement bulletins, and generic letters, and from a variety of INPO reports as well. Some Licensee Event Reports and NPRDS event descriptions were also reviewed individually.
Finally, information was included from reports of NRC-sponsored studies of the effects of plant aging, which include quantitative analysis of AFW system failure reports. This information was used to identify the various root causes expected for the broad PRA based failure events identified in Section 4.0, resulting in the inspection guidelines presented in Section 3.0.
5.1 Viroil C. Summer Exnerience The AFW system at Virgil C. Summer has experienced failures of the AFW pumps, pump discharge flow control valves, the turbine steam admission and supply valves, turbine trip and throttle valve, pump discharge isolation valves, service water backup supply valves, and numerous system check valves.
Failure modes include electrical, instrumentation and control, hardware failures, and human errors.
5.1.1 Motor Driven Pump Failures There have been three events from 1983 to 1991 which involved failure of the motor driven pumps during several modes of operation. Failure modes involved instrumentation and control circuit failures, blown fuse, and human failurts during maintenance attivities.
5.1.2 Turbine Driven Pump Failures Eight events from 1983 to 1991 have resulted in decreased operational readiness of the turbine driven pump. Failure causes were attributed to governor trip valve problems, bearing failure, low lube oil pressure, and operator error and procedural deficiencies in setting speed control. ,
5.1.3 Flow Control and Isolation Valve Failures More than sixteen events between 1983 and 1991 have resulted in impaired operational readiness of the air operated flow control and motor operated isolation valves. Principal failure causes were equipment wear, instrumentation and control circuit failures, valve hardware failures, and human errors. Valves have failed to operate properly due to failure of control components, dirty seat, overload and air leaks, Human errors have resulted in valve binding and packing *:.51 ems, i
16
- I E : +
4 .
l 5.1.4 Check Valves l five events of check valve f ailure have occurred from 1983 to 1991.
Normal wear and aging, poor design, lack of proper maintenance and human error were cited as the failure mode, resulting in leakage.
p 5.2 1ndustry Wide Evoerience Human errors, design / engineering problems and rrors, and cumponent SN- < ilures are the primary root causes of AFW Syster ailures identified in a y gview of industry wide systen operating history. Common csuse failures, y
t N.ch disable more than one train of this operationally redundant sy tem, are jgp
- tigh1.y risk significant, and can result from all c these causes.
This section identifies important common cause failuu modes, and then
?rovides a broader discussion of the single failure effects cf human errors, iesign/ engineering proMems and errors, and component f ailures. Paragrapns aresenting details nf inese failure modes are coded (e.g., CCl) and etoss-j referenced by inspection items in Section 3.0.
5.2.1 O mmon Cause railures The dominant cause of AFW syster multiple-train f ailures has been human error. Design / engineering errors and component f ailures have been less fre@cnt, but nevertheless significant, causes of multiple train failures. i
^ f,CL Human error in the form of incorrect operator intervention it.to l autcmat '4FV >/ stem functioning during transients resulted in the temporary loss of U w ety-grade AFW pumps during events at Davis Besse (NUREG-1154, 1985) ano frojan (AE0D/T416, 196.'. Ir the Davis Besse event, improper manual initiation of the steam and feedwater M .Jre Control system (SFF.CS) led to everspei ? tripping of both turbine-driven AFW pumpt, probably due to the introduction of condensate into the AFW turbines from the long, unheated steam supply lines. (The system had never been tested with th e abnorr~1, cross-connected steam supply lineup whkh resulted.) In the Trojan event the operator incorrectly stopped both AFW pumps due to misinterpretation of MFW pump speed indication. The diesel driven pump would not restart due to a protective feature requiring complete shuaNn, and the turbine-driven pump tripped on overspeed, requiring local reset of the trip and throttle valve. In cases where manual intervention is required during tna early stages of a transient, training shccid emphasize that actions aould be oerformed methodically and deliberately to guard against * . n errors.
R Valve mispes;tinring hts accounted for a significant fract wr of the hun.an errors f ailing r.uhiple trains of AF',I. This incluces closure of normally open suction valves e steam supply valves, and of isolation valves to sensors having control fun. ions. Interrect handswitch positioning and inadequate temporary wiring changes have also prevented automatic starts of multiple pumos. Factors identified in studies of mispositioning errors include failure to add newly installed valves to valve checklists, weak administrav /e control of tagging, restoration, independent verification, and locked valve logging, and inacequate adnerence to procedures. Illegible or 17 sw _ _ . . . . ..
so ,
t confusing local valve labeling, and insufficient training in the determination of valve position may cause or mask mispositioning, and surveillance which does not exercise complete system functioning may not reveal mispositionings.
E At ANO 2, both AFW pump. 'nst suction due to steam binding when they were lined up to both the CST ai,e the hot startup/ blowdown demineralizer effluent (AE00/C404, 1984). At Zion 1 steam created by running the turbine-driven pump deadheaded for one minute caused trip of a motor-driven pump sharing the same inlet header, as well as camage to the turbine-driven pump (Region 3 Morning Report, 1/17/90). Both events were caused by procedural inadequacies.
E Design / engineering errors have accounted for a smaller, but significant fraction of common cause failures. Problems with control circuit design modifications at Farley defeated AFW pump auto-start on loss of main feedvater. At Zion-2, restart of both motor driven pumps was blocked oy circ:it failure to deenergize when the pumps had been tripped with an autometic start signal present (IN 8?-01, 1952). In addition, AFW control circuit design reviews at Salem and Indian Point have identified designs where failures of a single component could have failed all or multiple pumps (IN 87 34,1987).
CCS. Incorrect setpoints and control circuit settings resulting from analysis errors and failures to update procedures have also prevented pump start and caused pumps to trip spuriously. Errors of this type may remain undetected despite surveillance testing, unless surveillance tests model all types of system initiation and operating conditions. A gretser fractic'1 of instrumentation and contiol circuit problems has been .dentifud during actual syst, operation (as. opposed to surveillance testing) than for other types of f ailu, .:5.
E On two occasions at a foreign plant, failure of a balance-of. plant inverter caused f ailure of two AFW pumps. In- addition to loss of the motor driven pump whose auxiliary start relay was powered by the invertor, the turbine driven pump tripped on overspeed because the governor valve opened, allowing full steam flow to the turbine. This illustrates the importance of assessing the effects of f ailures of balance of plant ecuipment which supports the operation of critical components. The instrument air system is another example of such a system.
E Multiple AFW pump trips have occurred at ! listone-3, Cook-1, Trojan and Zion-2 (IN 87-53, 1987) caused by brief, low pressure oscillations of suction pressure during pump startup . These oscillnions occurred despite the availability of adequate static NPSH. Correttive actions taken include:
extending the time delay associated with the low pressure trip, removing the trip, and replacing the trip with an alarm and operator action. ,
CC8, Design errors discovered during AFW system reanalysis at the Robinson plant (IN 89-30,1959) and at Millstone-1 resulted in the supply header from the CSI being too small to provide adecuate N?SH to the pumps if more than one of the three pumps were ,perating at rated flow conditions. This could lead to multiple pump failure due to cavitation. Suosecuent review:. at Robinson 15 l
l
g as-A, -La-L_- a r-n sg L-W--Ma >4 D 4, A A
- - - -- a- x a-4 t
identified a loss of feedwater transient in which inadequate NP',H and flows less than design values had occurred, but which were not recognized at the time. Event analysis and equipment trending, as wc11 as surveillance testing which duplicates service conditions as much as is practical, can help identify such design errors, gpoa Asiatic clams caused failure of two AFW flow control valves at Catawba-2 when low suction pressure caused by starting of a motor-driver pump caused i suction source realignment to the Nuclear Service Water system. Pipes had not been routinely treated to inhibit clam growth, nor regularly monitored to detect their presence, and r4 strainers wera installed. The need for surveillance which exercises alternative sy' m operational modes, as well as complete system functioning, is emphasized b3 his event. Spuricut s9: tion switchover has also occurred at Callaway and at McGuire, although no f ativres resulted.
CC10. Common cause f ailures have also been caused by component failures (AEOD/C404,1984). At Surry-2, both the turbine driven pump and one motor driven pump were declara. inoperable due to steam binding caused by leakage of hot water through multiple check valves. At Robinson-2 both motor driven pumps were found to be hot, and both motor and steam driven pumps wert: found to be inoperable at different times. Backleakage at Robinson 2 passed through L closed motor-operated isolation valves in addition to multiple check valves.
At farley, both motor and turbine driven pump casings were found hot, although the pumps were not declared inoperable. In addition to multi-train failures, numerous incidents of single train f ailures have occurred, resulting in the designation of " Steam Binding of Auxiliary Feedwater Pumps" as Generic issue i 93. This generic issue was resolvec by Generic Letter 88-03 (Firaglia, 1988),
- which required licensees to monn 'or A"W piping temperatures eacn shift, and to maintain-procedures for recognizing steam b' ding and for restoring system operability.
CC11. Common cause failures have also failed mo ar operated valves. During the total lon of feedwater event at Davis Besse, the normally-open AFW i isolation valses failed to open after they were inadvertently closed. The l failure was due te improper setting of the to"que switch bypass switch, which prevents motor trip on the high torcue required to unseat a closed valve.
Previous problems with these valves had been addressed by increasing the torque switch trip setpoint - a fix wnich failed during the event due to the higher torque required due to high differential pressure across the valve.
l Similar common mode failures of MOVs haw also occurred in other systems, I resulting in issuance of Generic Letter 99-10, " Safety Related Motor-Operated l Valve Testing and Surveillance (Partlow,1929)." Ttis generic letter requires l licensees to develop and implement a program to provide for the testing, inspection and maintenance of all safety-related MOVs tr provide assurante that they 'ill function when subjected to oesign basis conditions.
- CCl?. Other component f ailures have also resulted in AFW multi-train l failures. These include out-of-adjustment electrical flow controllers resulting in improper discharge valve operation, and a failure of oil cooler cooling water supply valves to open due to silt accumulation.
19 L
l
. . . .-- _ _ . - . .- - -. =.
~
s .
t 5.2.2 Human Errors E ine overwhelmingly dominant cause of problems identified during a series of operational readiness evaluations of AFW systems was human performance. The majority of these human performance problems resulted from incomplete and incorrect procedures, particularly with respect to valve lineup information.
A study of valve mispositioning events involving human error identified failures in administrative control of tagging. and loggihg, procedural
, compliance and completion of steps, verification of support systems, and inadequate procedures as important. Another study found that valve mispositioning events occurred most often during maintenance, calibration, or modification activities. Insufficient training in determining valve position, and in administrative requirements for controlling valve positioning were important causes, as was oral task assignment without task completion feedback.
R Turbine driven pump failures have been caused by human errors in calibrating or adjusting governor speed control, poor governor maintenance, incorrect adjustment of governor valve and overspeed trip linkages, and errors associated with the trip and throttle valve. TTV-associated errors include physically bumping it, f ailure to restore it to the correct position after testing, and failures to verify control room indication of TTV position following actuation.
E Motor driven pumps have been failed by human errors in mispositioning handswitches, and by procedure deficiencies.
5.2.3 Desion/Encineerine Problems and Errors D E As noted above, the majority of AFW subsystem failures, and the greatest relative system degradation, has been found to result from turbine-driven pump f ailures. Overspeed trips of Terry turbines controlled by Woodward governors hi e been a significant source of these failures (AE0D/C602, 1986). In many cases these overspeed trips hcve been caused by ' slow response of a Woodward Model EG governor on startup, at plants where full staam flow is allowed immediately. This oversensitivity has been removed by installing a startup steam bypass valve which opens first, allowing a controlled turbine acc.eleration and buildup of oil pressure to control the governor valve when tull steam flow is admitted.
PRI Oversoeed trips of Terry turbines have been caused by condensate in the steam supply lines. Condensate slows down the turbine, causing the governor valve to oren farther, and overspeed results before the governor valve can respond. after the water slug clears. This was determined to be the cause of the _lcss-of-all-AFW event at Davis Besse ( AEOD/602,1986), with condensation enhanced due to the long length of the cross-connected steam lines. Repeated tests following a cold-start trip n.ay De successful due to system h at up.
DI?L Turbine trip and throttle valve (TTV) problems are a significant cause of turbir.e driven pump failures (IN 84-66). In some cases lack of TTV position indication in the control room prevented recognition of a tripped TTV. In other cases it was possible to reset either the overspeed trip or the 20
a ., m, p---+4 s..mem a 2- .- 4 4 _aJ. JA e A- J--- a w -= _ s i- . a-_ iaaA.
o , ..
.0.
,TTV without resetting the other. This problem is compounded by the fact that the positior, of the overspeed trip linkage can be misleading, and the mechanism may lack labels indicating when it is in the tripped position (AE0D/C602,1986).
DE Startup of turbines with Woodaard Model PG-PL governors within 30 minutes of shutdown has resulted in overspeed trips when the speed setting knob was not exercised locally to drain oil from the speed setting cylinder.
Speed control is bastd on startup with an empty cylinder. Problems have involved turbine rotation due to both procedure violations and leaking steam.
Terry has marketed two types of dump valves for automatically draining the oil after shutdown (AEOD/C602, 1986).
At Calvert Cliffs, a 1987 loss-of. ,ffsite-power event required a cuick, cold startup that resulted in turbine t;ip due to PG-PL governor stability problems. The short-term corrective action was installation of stiffer buffer springs (IN 88-09, 1958). Surveillance had always been preceded by turbine warmup, which illustrates the importance cf testing which duplicates service conditions as much as is practical.
0 1 Reduced viscosity of gear box oil heated by prior operation caused f ailure of a motor driven pump to start due to insufficient lube oil pressure.
Lowering the pressure switch setpoint solved the problem, which had not been detected during testing.
p.1 Waterhammer at Palisades resulted in AFW iine and hanger damage at both steam generators. The AFW spargers are located at the normal steam generator j level, and are frequently covered and uncovered during level fluctuations.
I Waterhammers in top-feed-ring steam generators resulted in main feedline l rupture at Maine Yankee and feedwater pipe cracking at Indian Point-2 (IN 84-l 32,1984).
l DE Manually reversing the direction of motion of an operating valve has
- resulted in MOV failures where such loading was not considered in the design l (AE00/C603,1986). Control circuit design may prevent this, requiring stroke y completion before reversal.
D R At each of the units of the Sruth Texas Prosect, space heaters provided l by the vendor for use in preinstallation storage of MOVs ware found to be I
wired in parallel to the Class IE 125 V DC motors for several AFW valves (IR l 50-489/89-11; 50-499/89-11, 1999). The valves had been environmentally qualified, but not with the non-safety-related heaters energized.
5.2.4 Component Faibres Generic Issue II.E.6.1, *In Situ Testing Of Valves" was divided into four sub-issues (Beckjord,1989), three of which re.-te directly to prevention of AFW system component failure. At the request u the NRC, in-situ testing of check valves was adoressed by the nuclear indust.y, resulting in the EPRI report, " Application Guidelines for Check Valves in Nuclear Power Plants (Brooks,1988)." This extens1ve report provides information on check valve applications, limitations, and inspection techniques. in-situ testing of MOVs 21
q . -
fi ,
. e : i was addressed by Generic letter'89-10, " Safety Related Motor-Operated Valve __
-Testing and Surveillance * (Partlow, 1989) which requires licensees to develop and implement =a program for test _i_ng,- inspection and maintenance of all safety-related MOVs. '
Thermal Overload Protection for Electric Motors on Safety-Related Motor operated: Valves - Generic issue II.E.6.1 (Rothberg,1988)*
- concludes that valve motors _ should be. thermally protected - yet 'in a way which emphasizes system function over prctection of the operator.
E The common cause steam binding effects of check valve leakage were identified in Section 5.2.1, entry CC10. humerous single-trait, svents previde
-additional insights into this problem. In some cases leakage of hot MFW past multiple check valves.in series has occurred because adequate valve-seating-pressure was limited to the valves closest to the steam generators (AE00/C404, 1984). At Robinson,-the pump shutdown procedure was changed to delay closing the MOVs 'until after it.e check valves were seated. At Farley, check valves were changed from swing type to lift type. Check valve rework has bo n done at a number of plants. Different valve designs and manufacturers are involved in this problem, and recurring leakage hae been experienced, even after repair and replacement.
A At Robinson, heating of motor operated valves by check valve leakage has caused thermal binding and failure of AFW discharge valves to open on demand. '
At Davis Besse,-high. differential presst're across AFW injection valves resulting from check valve leakage has prevented MOV operation (AE0D/C603, 1986).
E Gross check valve leakage at McGuire and Robinson caused .
-overpressurization of the AFW suction' piping. At a foreign PWR it resulted in-a severe waterhammer event. At Palo Verde-2 the MFW suction piping was overpressurized by check valve leakage from the AFW system (AEOD/C404, 1984).
Gross check valve leakage through idle pumps represents a potential diversion
-ef- AFW pump flow.
CF4. Roughly one third of AFW system failures have been due to valve operator failures, with about equal failures -for MOVs and A0Vs. Almost half of the MOV failures were due to motor or switch failures _(Casada, 1989). An extensive study of MOV events-(AE0D/C603, 1986) indicates continuing inoperability
_ problems caused by: _ torque switch / limit switch settings, adjustments, or failures; motor burnout; improper sizing or use of thermal overload devices; premature degradation related to inadequate use of. protective devices; damage E due tol misuse (valve throttling, valve operator hammering); mechanical problems (loosened parts, improper assembly); or the torque switch bypass -
l circuit improperly installed or adjusted. The study concluded that current methods and procedures at trany plants are not- adequate to assure that MOVs
-willtoperate when needed under credible accident conditions. Specifically, a surveillance test which the. valve passed might result in undetected valve inoperability_due to component' failure (motor burnout, operator parts failure, sten. disc separation) or improper positioning of protective-devices (thermal overload, _ torque switch, limit switch). Generic Letter 89-10 (Partlow, 1989) has subsequently recuired licensees to implement a program ensuring that MOV switch settings.are maintained so that the valves will cperate under design
-basis conditions for the life of the plant.
22 p ens 3 w-e--y- m-w- -- -y
- 7 yy-- ty -i-w T-- w f--e'w--e- u re" -- '='e'** ,r-+w 1~1 y e- --= -'w ----
sa.-_.
E Component problems have caused a significant number of turbine driven pump trips (AEOD/C602, 1986). One group of events involved worc. tappet nut faces, loose cable connections, loosened set screws, improperly 'atched TTVs, and improper assembly. Another involved oil leaks due to component or seal failures, and oil contamination due to poor maintenance activities. Governor oil may not be shared with turbine lubrication oil, resulting in the need for separate oil changes. Electrical component failures included transistor or resistor failures due to moisture intrusion, erroneous grounds and connoctions, diode failures, and a faulty circuit card.
& Electrohydraulic-operated discharge valves have performed very poorly, and three of the five units using them have removed them due to recurrent failures. Failures included oil leaks, contaminated oil, and hydraulic pump failures.
.(172 Contral circuit failures were the dominant source of motor driven AFW pump failures (Casada,1989). This includes the controls used for automatic and manur1 starting of the pumps, as opposed to the instrumentation inputs.
Most of the remaining problems were due to circuit breaker failures.
CF8. " Hydraulic lockup" of Limitorque SMS spring packs has prevented proper spriag r.ompression to actuate the MOV torque switch, due to grease trapped in ine spring pack. During a surveillance at Trojan, failure of the torque switch to trip the TTV motor resulted in tripping of the thermal overload device, leaving the turbine driven pump inoperabh for 40 days until the next surveillance (AF%E702,1987). Problems result from grease changes to EXXON NEBULA EP-0 grease, one of only two greases considered environmentally qualified by Limitorque. Due to lower viscosity, it slowly migrates from the gear case into the spring pack. Grease changeover at Vermont Yankee affectec 40 of the older MOVs of which 32 were safety related. Grease relief kits are needed for MOV operators manufactured before 1975. At Limerick, additional grease relief was required for MOVs manufactured since 1975. MOV refurbishment programs may yield other changeovers to EP-0 grease.
CF9. For AFW systems using air operated valves, almost half of the system degradation has resulted from failures of the valve controller circuit and its instrument inputs (Casada, 1989). Failures occurred predominantly at a few units using automatic electronic controllers for the flow control valves, with the majority of failures due to electrical hardware. At Turkey Point-3, controller malfunction resulted from water in the Instrument Air system due to maintenance incperability of the air dryer:;.
CF]O. For systems using diesel driven pumps, most of the failures were due to start control and governor speed control circuitry. Half of these occurred on demand, as opposed to during testing (Casada, 1989).
CFil For systems using A0Vs, operability requires the availability of Instrument Air (IA), backup air, or backup nitrogen. However, NRC Maintenancc l- Team Inspections have identified inadequate testing of check valves isolating l- the safety-related portion of the IA system at several utilities (Letter, Roe i to Richardson). Generic Letter 88-14 (Miraglia,1988), recuires licensees to verify by test that air-operated safety-related components will perform as 23
.o -
, g c
expected in accordance with all design basis events, including a loss of nortnal IA.
f l 24 l
~
[ -1 C.,* ,
l=
6.0 REFERENCES
Beckjord, E. 5. Jane 30, 1989.
Testina of Valves". Letter to V.Closecut of Generic Issue II.E.6.1. "In Situ Stello, Jr., U.S. Nuclear Regulatory Commission, Washington, DC.
Brooks, B. P. 1988.
Aeolication Guidelines for Check Valves in Nuclear Power Plants. NP-S479, Electric
, Power Research Institute, Palo Alto, CA.
Casada, D. A. '1989. Auxiliary Feedwater System Aoine Study.
Volume 1.
Doeratino Experience and Current Monitorina Practices. NUREG/CR-5404. U.S.
Nuclear Regulatory Commission, Washington, DC.
Gregg, R. E, and R. E. Wright. 1988. Aroendix Review for Dominant Generic Contributprt. BLB-31-88. Idaho Nati)nal Engineering Laboratory, Idaho Falls, Idaho.
Miraglia, F. J. February 17, 1988. Resolution of Generic Safetv Issue 93,
" Steam Bindino of Auxiliarv Feedwater Pumps * (Generic Letter 88-03). U.S Nuclear Regulatory Commission, Washington, DC.
Miraglia, F. J. August 8, 1988. Jnstrument Air Sucolv System Problems Affectina Safetv-Related Ecuigme-t (Generic letter 88-14). U.S. Nuclear Regulatory Commission, Washington, DC.
Partlow, J. G. June 28, 1989. Safetv-Related Fotor-Operated Valve Testino and Surveillance (Generie Letter 89-10). U.S. Nuclear Regulatory Commission, Washington, DC.
Rothberg, O. June 1988.
Thermal Overload Protection for Electric Mocers on Safety-Rel.ated Motor-Ocerated valves - Generic issue II.E.6.1. NUREG-1296.
U.S. Nuclear Regulatory Commission, Washington, DC.
Travis, R. and J. Taylor. -1989. Develooment of Guidance for Generic.
Functional v Oriented PRA-Based Team Inspections for BWR Plants-Jdentification of Risk-1moortant Svstems. Components and Human Actions. TLR-A-3874-TGA Brookhaven National Laboratory, Upton, New York.
AE00 Reoorts AE0D/C404. W. D. Lanning. July 1984. Steam Bindinc of Auxiliarv Feedwater Pumps.
U.S. Nuclear Regulatory Commission, Washinoton, DC.
AE0D/C602. C. Hsu. August 1985. Operational Exoerience Nvolvine Turbine Oversceed Trios. U.S. Nuclear Regulatory Commission, Wasnington, DC.
AEOD;T605. E. J. Brown. December 1986. A Review of Motor-Operated Valve Performante. U.S. Nuclear Regulatory Commission, Washington, DC.
25
e -
~
O . ,a
+
L. '
O AEOD/E702. E. J. Brown. March 19, 1987. MOV Fiilure Due to Hydraulic locku-From Excessivt Grease in Sprino Pack. U.S. huclear Regulatory Commission, Washington, DC.
AEOD/T416. January 20, 1983. Loss of ESF Auxiliary Feedwa(tt. Pump Capability at Tro.ian on January 22. 1963. U.S. Nuclear Regulatory Commission, Washington, DC.
Information Notices IN 82-01. January 22, 1982. Auxiliarv Feedwater Pumn Lockout Resultino from Westinchouse V-2 Switch Circuit Modification. U.S. Nuclear Regulatory Commission, Washington, DC.
IN 84-32. E. L. Jordan. April 18, 1984. Auriliary Feedwater Sparcer and Pioe Hancar Damace. U.S. Nuclear Regulatory Commission, Washington, DC.
IN 84-66. August 17, 1984. Undetected Unavailabilitv of the Turbire-priven Auxiliary Feedwater Train. U.S. kuclear Regulatory Commission, Washington, DC.
IN 87-34. C. E. Rossi. July 24, 1987. Sincle Failures in Auxiliarv Feedwater Systers. U.S. Nuclear Regulatory Commission, Washington, DC.
IN 87-53. C. E. Rossi. October 20, 1957. Auriliarv Feodwater Pume Trios Resultino from Low Suction Pressure. U.S. Nuclear Regulatory Commission, l Wasnington, DC.
IN 88-09. C. E. Rossi. March 18, 1938. Reduced Reliab41itv of Stear-Driven
! Auxiliarv Feedwater Pumos Caused bv Instabilitv of Wooaware PG-PL Type Governors. U.S. Nuclear Regulatory Commission, Washington, DC.
IN 89-30. R. A. Azua. August 16, 1989. Robinson Unit 2 Inadeouate NPSH of A0xiliarv Feedwater Puros. Also, Event Notification 16375, August 22, 1989, i U.S. Nuclear Regulatory Commission, Washington, DC.
l l
l Inspection Report
~
sR 50-489/89-11; 50-499/89-11. May 26, 1989. South Texas Prniett Inspection Report. U.S. Nuclear Regulatory Commission, Washington, DC.
NUREG Report 1
! NUREG-ll54. 1985. Loss of Mair and Auriliarv Feedwate* Event at the Davis Besse Plar* <.* June 9. 1985. U.S. Nuclear Regulatory Commission, Washington, 26