ML14188B164

Auxiliary Feedwater System RISK-BASED Inspection Guide for the H.B. Robinson Nuclear Power Plant

ML14188B164
Person / Time
Site:	Robinson
Issue date:	08/31/1993
From:	Garner L, Gore B, Lloyd R, Moffitt N, Vo T Battelle Memorial Institute, Pacific Northwest National Laboratory, NRC Office of Inspection & Enforcement (IE Region II)
To:	Office of Nuclear Reactor Regulation
References
CON-FIN-L-1310 NUREG-CR-5833, PNL-7907, NUDOCS 9308160303
Download: ML14188B164 (36)

Text

NUREG/CR-5833 PNL-7907 Auxiliary Feedwater System Risk-Based Inspection Guide for the H. B. Robinson Nuclear Power Plant Prepared by N. E. Moffitt, R. C. Lloyd, B. F. Gore, T. V. Vo/PNL L. W. Gamer/NRC Pacific Northwest Laboratory Operated by Battelle Memorial Institute Prepared for U.S. Nuclear Regulatory Commission 9308160303 930831 PDR ADOCK 05000261 10 PDR

AVAILABILITY NOTICE Availability of Reference Materials Cited in NRC Publications Most documents cited in NRC publications will be available from one of the following sources:

1.

The NRC Public Document Room, 2120 L Street. NW, Lower Level, Washington, DC 20555-0001

2.

The Superintendent of Documents, U.S. Government Printing Office, Mail Stop SSOP, Washington, DC 20402-9328

3.

The National Technical Information Service, Springfield, VA 22161 Although the listing that follows represents the majority of documents cited In NRC publications, it is not Intended to be exhaustive.

Referenced documents available for inspection and copying for a fee from the NRC Public Document Room Include NRC correspondence and Internal NRC memoranda; NRC Office of inspection and Enforcement bulletins, circulars, Information notices, Inspection and Investigation notices; Ucensee Event Reports; ven dor reports and correspondence; Commission papers; and applicant and licensee documents and corre spondence.

The following documents in the NUREG series are available for purchase from the GPO Sales Program:

formal NRC staff and contractor reports, NRC-sponsored conference proceedings, and NRC booklets and brochures. Also available are Regulatory Guides, NRC regulations in the Code of Federal Regulations, and Nuclear Regulatory Commission Issuances.

Documents available from the National Technical Information Service include NUREG series reports and technical reports prepared by other federal agencies and reports prepared by the Atomic Energy Commis sion, forerunner agency to the Nuclear Regulatory Commission.

Documents available from public and special technical libraries include all open literature items, such as books, journal and periodical articles, and transactions. Federal Register notices, federal and state legisla tion, and congressional reports can usually be obtained from these libraries.

Documents such as theses, dissertations, foreign reports and translations, and non-NRC conference pro ceedings are available for purchase from the organization sponsoring the publication cited.

Single copies of NRC draft reports are available free, to the extent of supply, upon written request to the Office of Information Resources Management, Distribution Section, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001.

Copies of industry codes and standards used in a substantive manner in the NRC regulatory process are maintained at the NRC Library, 7920 Norfolk Avenue, Bethesda, Maryland, and are available there for refer ence use by the pubilc.

Codes and standards are usually copyrighted and may be purchased from the originating organization or, if they are American National Standards, from the American National Standards Institute, 1430 Broadway, New York, NY 10018.

DISCLAIMER NOTICE This report was prepared as an account of work sponsored by an agency of the United States Government.

Neither the United States Government nor any agency thereof, or any of their employees, makes any warranty, expresed or implied, or assumes any legal liability of responsibility for any third party's use, or the results of such use, of any information, apparatus, product or process disclosed in this report, or represents that its use by such third party would not infringe privately owned rights.

NUREG/CR-5833 PNL-7907 Auxiliary Feedwater System Risk-Based Inspection Guide for the H. B. Robinson Nuclear Power Plant Manuscript Completed: June 1993 Date Published: August 1993 Prepared by N. E. Moffitt, R. C. Lloyd, B. F. Gore, T. V. Vo L. W. Garner*

J. Chung, NRC Project Manager Pacific Northwest Laboratory Richland, WA 99352 Prepared for Division of Systems Safety and Analysis Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Washington, DC 20555-0001 NRC FIN L1310

• U.S. Nuclear Regulatory Commission Washington, DC 20555-0001

Abstract In a study sponsored by the U.S. Nuclear Regulatory Commission (NRC), Pacific Northwest Laboratory has developed and applied a methodology for deriving plant-specific risk-based inspection guidance for the auxiliary feedwater (AFW) system at pressurized water reactors that have not undergone probabilistic risk assessment (PRA). This meth odology uses existing PRA results and plant operating experience information. Existing PRA-based inspection guid ance information recently developed for the NRC for various plants was used to identify generic component failure modes. This information was then combined with plant-specific and industry-wide component information and failure data to identify failure modes and failure mechanisms for the AFW system at the selected plants. H. B. Robinson was selected as one of a series of plants for study. The product of this effort is a prioritized listing of AFW failures which have occurred at the plant and at other PWRs. This listing is intended for use by NRC inspectors in the preparation of inspection plans addressing AFW risk-important components at the H. B. Robinson plant.

iii NUREG/CR-5833

Contents Abstract.

Summary ix Acknowledgments xi I Introduction....................................................................

1.1 2 H. B. Robinson AFW System 2.1

2.1 System Description

2.1 2.2 Success Criterion 2.1 2.3 System Dependencies....

2.1 2.4 Operational Constraints..........................................................

2.2 3 Inspection Guidance for the H. B. Robinson AFW System 3.1 3.1 Risk Important AFW Components and Failure Modes 3.1 3.1.1 Trbine Pump Unavailable Due to Test or Maintenance 3.1 3.1.2 Thrbine Driven Pump Fails to Start or Run 3.1 3.1.3 Latent Human Action Failure Resulting in Valve Mispositioning.........................

3.2 3.1.4 Multiple Pump Failures due to Common Cause 3.2 3.1.5 Manual Suction or Discharge Valves Fail.....

3.4 3.1.6 Motor Operated Isolation Valve Failure.....

3.4 3.1.7 Electro-Hydraulic Flow Control Valves Fail..

3.5 3.1.8 Motor Driven Pump A or B Fails to Start or Run 3.5 3.1.9 Leakage of Hot Feedwater through Check Valves 3.5 3.2 Risk Important AFW System Walkdown 'Phble 3.6 4 Generic Risk Insights From PRAs 4.1 4.1 Risk Important Accident Sequences Involving AFW System Failure 4.1 4.2 Risk Important Component Failure Modes 4.1 5 Failure Modes Determined From Operating Experience 5.1 5.1 H. B. Robinson Experience 5.1 5.1.1 Motor Driven Pump Failures 5.1 5.1.2 Thrbine Driven Pump Failures 5.1 5.1.3 EHC Flow Control and Motor Operated Isolation Valve Failures 5.1 5.1.4 Check Valves 5.1 v

NUREG/CR-5833

5.2 Industry Wide Experience 5.1 5.2.1 Common Cause Failures 5.2 5.2.2 Human Errors 5.4 5.2.3 Design/Engineering Problems and Errors............................................

5.4 5.2.4 Component Failures 5.5 6 References 6.1 NUREG/CR-5833 vi

Figure 2.1 H. B. Robinson Auxiliary Feedwater System 2.3 Tables 3.1 Risk important walkdown table for H. B. Robinson AFW system components 3.7 4.1 Risk prioritized comparison of H. B. Robinson and generic PRA results 4.3 vii NUREG/CR-5833

Summary This document presents a compilation of auxiliary feedwater (AFW) system failure information which has been screened for risk significance in terms of failure frequency and degradation of system performance. It is a risk prioritized listing of failure events and their causes that are significant enough to warrant consideration in inspection planning at the H. B. Robinson plant. This information is presented to provide inspectors with increased resources for inspection planning at H. B. Robinson.

The risk importance of various component failure modes was identified by analysis of the results of probabilistic risk assessments (PRAs) for many pressurized water reactors (PWRs). However, the component failure categories identi fied in PRAs are rather broad, because the failure data used in the PRAs is an aggregate of many individual failures having a variety of root causes. In order to help inspectors focus on specific aspects of component operation, main tenance and design which might cause these failures, an extensive review of component failure information was performed to identify and rank the root causes of these component failures. Both H. B. Robinson and industry-wide failure information was analyzed. Failure causes were sorted on the basis of frequency of occurrence and seriousness of consequence, and categorized as common cause failures, human errors, design problems, or component failures.

This information is presented in the body of this document. Section 3.0 provide brief descriptions of these risk important failure causes, and Section 5.0 presents more extensive discussions, with specific examples and references.

The entries in the two sections are cross-referenced.

An abbreviated system walkdown table is presented in Section 3.2 which includes only components identified as risk important. This table lists the system lineup for normal, standby system operation.

This information permits an inspector to concentrate on components important to the prevention of core damage.

However, it is important to note that inspections should not focus exclusively on these components. Other compo nents which perform essential functions, but which are not included because of high reliability or redundancy, must also be addressed to ensure that degradation does not increase their failure probabilities, and hence their risk importance.

ix NUREG/CR-5833

Acknowledgments We wish to thank G. L. Comer, D. T Gudger, E. M. Shoemaker, and I. J. Zarzar of the Carolina Power and Light Company for reviewing and validating this document. Their input to sections 2, 3, and 4 make this report a more useful inspection tool. Their co-operation is greatly appreciated.

xi NUREG/CR-5833

1 Introduction This document is one of a series providing plant-specific The remainder of the document describes and discusses inspection guidance for auxiliary feedwater (AFW) sys-the information used in compiling this inspection guid tems at pressurized water reactors (PWRs). This guid-ance. Section 4.0 describes-the risk important informa ance is based on information from probabilistic risk as-tion which has been derived from PRAs and its sources.

sessments (PRAs) for similar PWRs, industry-wide As review of that section will show, the failure events operating experience with AFW systems, plant-specific identified in PRAs are rather broad (e.g., pump fails to AFW system descriptions, and plant-specific operating start or run, valve fails closed). Section 5.0 addresses experience. It is not a detailed inispection plan, but the specific failure causes which have been combined rather a compilation of AFW system failure information under these broad events.

which has been screened for risk significance in terms of failure frequency and degradation system performance.

AFW system operating history was studied to identify The result is a risk-prioritized listing of failure events the various specific failures which have been aggregated and the causes that are significant enough to warrant into the PRA failure events. Section 5.1 presents a sum consideration in inspection planning at H. B. Robinson.

mary of H. B. Robinson failure information, and Sec tion 5.2 presents a review of industry-wide failure in This inspection guidance is presented in Section 3.0, fol-formation. The industry-wide information was compiled lowing a description of the H. B. Robinson AFW system from a variety of NRC sources, including AEOD in Section 2.0. Section 3.0 identifies the risk important analyses and reports, information notices, inspection system components by H. B. Robinson identification and enforcement bulletins, and generic letters, and from number, followed by brief descriptions of each of the a variety of INFO reports as well. Some Licensee Event various failure causes of that component. These include Reports and NPRDS event descriptions were also re specific human errors, design deficiencies, and hardware viewed individually. Finally, information was included failures. The discussions also identify where common from reports of NRC-sponsored studies of the effects of cause failures have affected multiple, redundant compo-plant aging, which include quantitative analyses of re nents. These brief discussions identify specific aspects ported AFW system failures. This industry-wide in of system or component design, operation, maintenance, formation was then combined with the plant-specific or testing for inspection by observation, records review, failure information to identify the various root causes of training observation, procedures review, or by observa-the broad failure events used in PRAs, which are identi tion of the implementation of procedures. An AFW sys-fled in Section 3.0.

tem walkdown table identifying risk important compo nents and their lineup for normal, standby system operation is also provided.

1.1 NUREG/CR-5833

2 H. B. Robinson AFW System This section presents an overview description of the equipped with a continuous recirculation flow system, H. B. Robinson AFW system (Westinghouse three loop which prevents pump deadheading.

plant), including a simplified schematic system diagram.

In addition, the system success criterion, system depen-The discharges of the motor driven pumps are cross con dencies, and administrative operational constraints are nected, and they feed all three steam generators. The also presented.

turbine-driven pump also feeds all three steam genera tors through a connection into the main feedwater reg ulating valve bypass line, downstream of the bypass

2.1 System Description

control valve. A flow control valve at the discharge of each pump ensures AFW flow will automatically main The AFW system provides feedwater to the steam gen-tai a desired flowrate to each steam generator. Each of erators (SG) to allow secondary-side heat removal from the lines from the motor driven pumps contains a motor the primary system when main feedwater is unavailable.

operated discharge isolation valve, V2-16A,B,C. Dis The system is capable of functioning for extended per-charge isolation valves in the lines from the turbine iods, which allows time to restore main feedwater flow driven pump are motor operated valves, V2-14A,B,C.

or to proceed with an orderly cooldown of the plant to Flow control valves FGV-1424, 1425, 6416 are electro where the residual heat removal (RHR) system can re-hydraulically operated. Each line contains check valves move decay heat. A simplified schematic diagram of the to prevent leakage from the feedwater lines.

H. B. Robinson AFW system is shown in Figure 2.1.

H. B Roinsn AW sste is how inFigre.1.The condensate storage tank (CST) is the normal source The AFW system consists of two motor-driven (MD) of water for the AFW System and is required to store pumps and one turbine-driven (SD) pump along with enough demineralized water at a minimum level suffi the associated piping, valves and instrumentation cient to maintain the reactor coolant system (RCS) at normally connected to the Condensate Storage Tank hot standby conditions for 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />. All tank connec (CST). It is designed to start up and establish flow auto-tions except those required for instrumentation, auxil matically. All pumps start on receipt of a steam genera-iary feedwater pump suction, chemical analysis, and tank tor low-low level signal. (The motor-driven pumps start drainage are located above this minimum level. AFW on low level in one SG, whereas, two low level signals suction may also be manually switched to the service are required for the turbine-driven pump start.) Also, water system or deep well water as a second and third the motor-driven pumps start on a trip of main feed-source of water.

water pumps (MFW) pumps, a safety injection signal, or on a blackout signal (total loss of all AC power). The single turbine-driven (SD) pump also starts on under-2.2 Success Criterion voltage on the 4160 V busses 1&4.

System success requires the operation of at least one

'ITWo separate suction lines off of the CST join to form a pump supplying rated flow to at least one of the three common header that supplies water to the turbine-steam generators.

driven pump. A separate line off of the common header supplies water to both motor-driven pumps. Isolation valves in these lines are locked open. Power, control, 2.3 System Dependencies and instrumentation associated with each motor-driven pump are independent from one another. Steam for the The AFW system depends on AC and DC power at van turbine-driven pump is supplied by steam generators A, ous voltage levels for motor operation, valve control, B and C, from a point upstream of the main steam isola-monitor and alarm circuits, and valve/motor control tion valves, through valve MIS 154. Each AFW pump is circuits. Instrument Air is normally used, but not 2.1 NUREG/CR-5833

AFW System required for governor speed control. Steam availability six hours. If two AFW pumps are inoperable, restore at is required for the turbine-driven pump.

least one inoperable AFW pump to operable status within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> or the plant must be cooled down to below 350'F within an additional 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br />.

2.4 Operational Constraints 2.4 peraionl CostrantsThe H. B. Robinson Technical Specifications require a When the reactor coolant is heated above 3500 F the minimum supply of 35,000 gallons of water to be stored H. B. Robinson Technical Specifications require that all in the CST (2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> operation) and an unlimited water three AFW pumps and associated flow paths are oper-supply from the lake via either leg of the plant Service able with each motor-driven pump powered from a dif-Water System. With the CST inoperable for greater ferent vital bus. If one AFW pump becomes inoperable, than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, the plant must be placed in the hot shut it must be restored to operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or down condition.

the plant must shut down to hot standby within the next NUREG/GR-5833 2.2

To A" Steam Generator FW~~S 6A6W To "B" Steam Gonerator Storag Ta To "C" FCV 4788 FW-8A F-A FW-V2-6A ro Steamte Fiur.1H B obnsnAuiiayFewter System FW-6A FW-9A FCV 4988 FW-89 V2-14A FW-V2-6B From-8 FW-7B SG dwmte FW-6B FW-9B P

S22 IG"B IC 9

-l8 To '" Stam S-14 C,

MS26I Gonertor G -I I~

2, V2-16BW-1 V2-16A 8Fo V2-208 KASevc unles othewiseaoted 70

~ ~

C4 489 4

FW-80 V21BMF 2 -6S-I8 MS-1544SG "C

All ~~~~~

~2 84ve haeA dsgainCeiaWum unles oterwse nted.Fe4 FCV 641 Recir Figue 21 H B.

obisonAuxiiar Fedwatr SstD

3 Inspection Guidance for the H. B. Robinson AFW System In this section the risk important components of the 3.1.1 Thrbine Pump Unavailable Due to Test H. B. Robinson AFW system are identified, and the im-or Maintenance portant failure modes for these components are briefly described. These failure modes include specific human Both scheduled and unscheduled maintenance re errors, design deficiencies, and types of hardware fail-move pumps from operability. Surveillance requires ures which have been observed to occur for these com-operation with an altered line-up, although a pump ponents, both at H. B. Robinson and at PWRs through-train may not be declared inoperable during testing.

out the nuclear industry. The discussions also identify Prompt scheduling and performance of mainte where common cause failures have affected multiple, re-nance and surveillance minimize this unavailability.

dundant components. These brief discussions identify specific aspects of system or component design, opera-Inspection Suggestion - Review the time the AFW tion, maintenance, or testing for inspection activities.

system and components are inoperable. Assure all These activities include: observation, records review, maintenance is being performed that can be per training observation, procedures review, or by observa-formed during a single outage time frame, avoiding tion of the implementation of procedures.

multiple equipment outages. The maintenance should be scheduled before the routine surveillance Ible 3.1 is an abbreviated AFW system walkdown table test, so credit can be taken for both post mainte which identifies risk-important components. This table nance testing and surveillance testing, avoiding ex lists the system lineup for normal (standby) system op-cessive testing. Review surveillance schedule for eration. Inspection of the components identified in the frequency and adequacy to verify system operability system lineup table addresses essentially all of the risk requirements per Technical Specifications.

associated with AFW system operation.

3.1.2 'flrbine Driven Pump Fails to Start or 3.1 Risk Important AFW Components Run and Failure Modes Improperly adjusted and inadequately maintained turbine governors have caused pump failures. HE2.

PRA analysis at H. B. Robinson has identified the Problems include worn or loosened nuts, set screws, turbine-driven 'ump unavailable due to maintenance or linkages or cable connections, oil leaks and/or con surveillance as the most risk-important failure mode for tamination, and electrical failures of resistors, tran AFW system components. This is followed in impor-sistors, diodes and circuit cards, and erroneous tance by single turbine pump failure, human errors in grounds and connections. CF5. Governor problems valve mispositioning, common cause failure of all three have caused failure of the turbine driven pump at AFW pumps, manual suction valve failures, and motor H. B. Robinson.

driven pump failures.

Inspection Suggestion - Review PM records to assure The following sections address each of these failure the governor oil is being replaced within the desig modes, in decreasing order of risk-importance. They nated frequency. During plant walkdowns carefully present the important root causes of these component inspect the governor and linkages for loose faste failure modes which have been distilled from historical ners, leaks, and unsecured or degraded conduit. Re records. Each item is keyed with a three digit code (e.g.

view vendor manuals to ensure PM procedures are CF2) to discussions in Section 5.2 where additional in-performed according to manufacturer's recommen formation on historical events is presented.

dations and good maintenance practices.

3.1 NUREG/CR-5833

Inspection Guidance

• Condensate slugs in steam lines have caused turbine calibration, or system modifications. Important causes overspeed trip on startup. Tests repeated right after of mispositioning include:

such a trip may fail to indicate the problem due to warming and clearing of the steam lines. Surveil-

- Failure to provide complete, clear, and specific lance should exercise all steam supply connections.

procedures for tasks and system restoration DE2.

- Failure to promptly revise and validate proce Inspection Suggestion - Verify that the steam traps dures, training, and diagrams following system are valved in on the steam supply line. For steam modifications traps that are on a pressurized portion of the steam line, check the steam trap temperature (if unlagged)

- Failure to complete all steps in a procedure to assure it is warmer than ambient (otherwise it may be stuck or have a plugged line). If the steam

- Failure to adequately review uncompleted trap discharge is visible, assure there is evidence of procedural steps after task completion liquid discharge.

- Failure to verify support functions after Iip and throttle valve (TTV) problems which have restoration failed the turbine driven pump include physically bumping it, failure to reset it following testing, and

- Failure to adhere scrupulously to admini failures to verify control room indication of reset.

strative procedures regarding tagging, con HE2. Whether either the overspeed trip or TTV trol and tracking of valve operations trip can be reset without resetting the other, indica tion in the control room of TTV position, and un-

- Failure to log the manipulation of sealed ambiguous local indication of an overspeed trip af-valves fect the likelihood of these errors. H. B. Robinson has on occasion experienced worn and damaged

- Failure to follow good practices of written parts in the overspeed trip mechanism that has pre-task assignment and feedback of task coi vented it from being reset. DE3.

pletion information Inspection Suggestion - Carefully inspect the TTV

- Failure to provide easily read system draw overspeed trip linkage and assure it is reset and in ings, legible valve labels corresponding to good physical condition. Assure that there is a good drawings and procedures, and labeled podi steam isolation to the turbine, otherwise continued cations of local valve position turbine high temperature can result in degradation of the oil in the turbine, interfering with proper Inspection Suggestion - Review the administrative overspeed trip operation. Review training proce-controls that relate to valve positioning and sealing, dures to ensure operator training on resetting the system restoration following maintenance, valve TIV is current.

labeling, system drawing updating, and procedure revision, for proper implementation.

3.1.3 Latent Human Action Failure Resulting in Valve Mispositioning 3.1.4 Multiple Pump Failures due to Common Cause

-Valve mispositioning has resulted in failures of mul tiple trains of AFW-CC2. It has also been the The following listing summarizes the most important dominant cause of problems identified during multiple-pump failure modes identified in Section 5.2.1, operational readiness inspections. HE. Events Common Cause Failures, and each item is keyed to en have occurred most often during maintenance, tries in that section.

NUREG/CR-5833 3.2

Inspection Guidance Incorrect operator intervention into automatic sys-operation, and failures to restart after pump tem functioning, including improper manual start-shutdown. CC4. Incorrect setpoints and control ing and securing of pumps, has caused failure of all circuit calibrations have also prevented proper pumps, including overspeed trip on startup, and in-operation of multiple pumps. CC5.

ability to restart prematurely secured pumps. CCL.

Inspection Suggestion - Review design change imple Inspection Suggestion - Observe Abnormal and mentation documents for the post maintenance test Emergency Operating Procedure (AOP/EOP) ing required prior to returning the equipment to simulator training exercises to verify that the opera-service. Assure the testing verifies that all tors comply with procedures during observed evolu-potentially impacted functions operate correctly, tions. Observe surveillance testing on the AFW sys-and includes repeating any plant start-up or hot tem to verify it is in strict compliance with the functional testing that may be affected by the design surveillance test procedure.

change.

Valve mispositioning has caused failure of all Loss of a vital power bus has failed both the turbine pumps. Pump suction, steam supply, and instru-driven and one motor-driven pump due to loss of ment isolation valves have been involved. CC2.

control power to steam admission valves or to turbine controls, and to motor controls powered Inspection Suggestion - Verify that the system valve from the same bus. CC6. At H. B. Robinson, this is alignment, air operated valve control and valve actu-not expected to be a problem. Upon loss of power, ating air pressures are correct using 3.1 Walkdown the turbine driven AFW pump can be manually Table, the system operating procedures, and opera-operated.

tor rounds logsheet. Review surveillance proce dures that alter the standby alignment of the AFW Inspection Suggestion - The material condition of the system. Ensure that an adequate return to normal electrical equipment is an indicator of probable reli section exists.

ability. Review the Preventative Maintenance (PM) records to assure the equipment is maintained on an Steam binding has caused failure of multiple pumps.

appropriate frequency for the environment it is in This resulted from leakage of hot feedwater past and that the PM's are actually being performed as check valves and a motor operated valve into a com-required by the program. Review the outstanding mon discharge header. CC10. Multiple-pump Corrective Maintenance records to assure the defi steam binding has also resulted from improper valve ciencies found on the equipment are promptly lineups, and from running a pump deadheaded.

corrected.

CC3.

Simultaneous startup of multiple pumps has caused Inspection Suggestion - Verify that the pump dis-oscillations of pump suction pressure causing multi charge temperature is at ambient temperature. As-ple-pump trips on low suction pressure, despite the sure any instruments used to verify the temperature existence of adequate static net positive suction by the utility are of an appropriate range and in-head (NPSH). CC7. At H. B. Robinson, design re cluded in a calibration program. Verify affected views have identified inadequately sized suction pumps have been vented in accordance with proce-piping which could have yielded insufficient NPSH dure OP-402 to ensure steam binding has not oc-to support operation of more than one pump. CC8.

curred. Verify that a maintenance work request has This problem was corrected by design modifications been written to repair leaking check valves.

which increased the size of the AFW pump suction line.

Pump control circuit deficiencies or design modif ication errors have caused failures of multiple Inspection Suggestion - Assure that plant conditions pumps to auto start, spurious pump trips during which could result in the blockage or degradation of 3.3 NUREG/CR-5833

Inspection Guidance the suction flow path are addressed by system main-Common cause failure of MOVs has resulted from tenance and test procedures. Examples include, if failure to use electrical signature tracing equipment the AFW system has an emergency source from a to determine proper settings of torque switch and water system with the potential for bio-fouling, then torque switch bypass switches. Failure to calibrate the system should be periodically treated to prevent switch settings for high torques necessary under de buildup and routinely tested to assure an adequate sign basis accident conditions has also been flow can be achieved to support operation of all involved. CC11. Improper torque setting, dirty pumps, or inspected to assure that bio-fouling is not contacts and seat leakage have been the main causes occurring. Design changes that affect the suction of valve failure at H. B. Robinson.

flow path should repeat testing that verified an ade quate suction source for simultaneous operation of Inspection Suggestion - Review the MOV test rec all pumps. Verify that testing has, at sometime, ords to assure the testing and settings are based on demonstrated simultaneous operation of all pumps.

dynamic system conditions. Overtorquing of the Verify that surveillances adequately test all aspects valve operator can result in valve damage such as of the system design functions, for example, demon-cracking of the seat or disc. Review the program to strate that the AFW pumps will trip on low suction assure overtorquing is identified and corrective ac pressure.

tions are taken to assure valve operability following an overtorque condition. Review the program to as 3.1.5 Manual Suction or Discharge Valves Fail sure EQ seals are renewed as required during the restoration from testing to maintain the EQ rating CST Suction Valves: AFW-1,104 of the MOV TD Pump fain: AFW-4; AFW-20,17; FW-6A,B,C MD Pump tain A: AFW-22,28; AFW-44,53,54; Valve motors have been failed due to lack of, or im AFW-62,63 proper sizing or use of thermal overload protective MD Pump hain B: AFW-29; AFW-55,64 devices. Bypassing and oversizing should be based Backup Suction Sources: SW-24,118; DW-19,21 on proper engineering for design basis conditions.

C174.

These manual valves are normally locked open except for the backup suction sources from the deep well and Inspection Suggestion - Review the administrative service water systems, which are normally locked closed.

controls for documenting and changing the settings of thermal overload protective devices. Assure the Valve mispositioning has resulted in failures of mul-information is available to the maintenance tiple trains of AFW. CC2. It has also been the planners.

dominant cause of problems identified during op erational readiness inspections. HEL. Events have 0

Grease trapped in the torque switch spring pack of occurred most often during maintenance, calibra-Limitorque 8MB motor operators has caused motor tion, or system modifications.

burnout or thermal overload trip by preventing torque switch actuation. CF8.

Inspection Suggestion - See 3.1.3 bullet 1.

Inspection Suggestion - Review this only if the MOV 3.1.6 Motor Operated Isolation Valve Failure testing program reveals deficiencies in this area.

SD Pump 'ain:

V2-14ABC Manually reversing the direction of motion of op MD Pump hains A;B: V2-16AB,C erating MOVs has overloaded the motor circuit.

MD Pump Cross-Connect Valves: V2-20 A,B Operating procedures should provide cautions, and circuit designs may prevent reversal before each These MOVs isolate flow to the steam generators. They stroke is finished. DE7 At H. B. Robinson, circuit fail as-is on loss of power.

design precludes this type of failure.

NUREGICR-5833 3.4

Inspection Guidance Inspection Suggestion -Review operator training on 3.1.8 Motor Driven Pump A or B Fails to Start MOV operation to ensure this topic is adequately or Run addressed. Review operating procedures to ensure adequate precautions are identified.

Control circuits used for automatic and manual Space heaters designed for preoperation storagecause of motor Spac hetersdesgne forprepertionstoagedriven pump failures, as are circuit breaker failures.

have been found wired in parallel with valve motors CF7. Control circuit problems have occurred at which had not been environmentally qualified with H. B. Robinson.

them present. DE8.

Inspection Suggestion - Spot check MOV's duringmainte MOV testing to assure the space heaters are phys-to dern iaen exts. Ert beaker ically removed or disconnected.

t eemn fatedeit.Eeytm rae icaly reove or iscnneced.is racked in a PMT should be performed to start the pump, assuring no control circuit problems have oc 3.1.7 Electro-Hydraulic Flow Control Valves curred as a result of the manipulation of the Fail breaker. (Control circuit stabs have to make up upon racking the breaker, as well as cell switch dam SD Pump Rain: FCV-6416 age can occur upon removal and reinstallation of MID Pump Rains AB: FCV-1424; FCV 1425 the breaker.)

These electro-hydraulic valves control flow to the steam

• Mispositioning of handswitches and procedural generators. FCV-6416 on the steam driven pump dis-deficiencies have prevented automatic pump start.

charge fails open, FCV-1424 and FCV-1425 on the HE3.

motor-driven pumps discharge fail closed.

Inspection Suggestion - Confirm switch position EHV performance has been poor at other facilities, using Tble 3.1. Review administrative procedures primarily due to hydraulic problems. CF6. H. B.

concerning documentation of procedural deficien Robinson experience has been much better.

cies. Ensure operator training on procedural changes is current.

Inspection Suggestion - Review the Preventative Maintenance (PM) records to assure the equipment 3.1.9 Leakage of Hot Feedwater through is maintained on an appropriate frequency for the Check Valves environment it is in and that the PM's are actually being performed as required by the program. Re-MD Pump A: AFW-40,68,69 view the outstanding Corrective Maintenance rec-MD Pump B: AFW-41,70 ords to assure the deficiencies found on the equip-SD Pump: AFW-84,FW-8ABC ment are promptly corrected.

PRA analysis at H. B. Robinson does not indicate that Leakage of hot feedwater through check valves has check valve leakage is a significant AFW system failure caused thermal binding of normally closed flow con-mode. However, check valve leakage has occurred at trol MOVs. EHVs may be similarly susceptible.

H. B. Robinson causing failure of AFW discharge valves CF2.

to open on demand, therefore it is prudent to consider Inspctin Sggetion-Cverd b 3.14 blle 3.the following root causes in inspection planning at Inspection Suggestion - Cove re vetiveRobinson.

3.5 NUREG/CR-5833

Inspection Guidance

• Leakage of hot feedwater through several check Inspection Suggestion - Covered by 3.1.4 bullet 3.

valves in series has caused steam binding of multiple pumps. Leakage through a closed level control 3.2 Risk Important AFW System valve in series with check valves has also occurred at Walkdown Table H. B. Robinson, as would be required for leakage to reach the motor driven pumps A and B. CC10.

This problem has been corrected by system modifi-le 3.1 presen n AF eyte askowptan cations that have d vle andireplaced t This information allows inspectors to concentrate their valves with double disc valves and replacedr efforts on components important to prevention of core checkment damage. However, it is essential to note that inspec arrangement.tions should not focus exclusively on these components.

Inspection Suggestion - Covered by 3.1.4 bullet 3.

Other components which perform essential functions, but which are absent from this table because of high reli Slow leakage past the final check valve of a series ability or redundancy, must also be addressed to ensure may not force the check valve closed. Other check that their risk importance are not increased. Examples valves in series may leak similarly. Piping orienta-include the (locked open) steam lead isolation valves tion and valve design are important factors in (MS-262ABC) upstream of the main steam isolation achieving true series protection. CF1.

valves, and an adequate water level in the CST Inspection Suggestion - Covered by 3.1.4 bullet 3.

" At H. B. Robinson, heating of motor-operated valves by check valve leakage has caused thermal binding and failure of AFW discharge valves to open on demand. CF2. This problem has been cor rected by system modifications that have replaced the original flow control valves with double disc valves and replaced the check valves with bonnet and spring plunger arrangement.

NUREG/CR-5833 3.6

Inspection Guidance Table 3.1 Risk important walkdown table for H. B. Robinson AFW system components Actual Component #

Component Name Required Position Position Electrical A

Motor-Driven Pump Racked In/Open B

Motor-Driven Pump Racked In/Open Valves RTGB Control Board V2-20A AFW Header Section Isolation Open V2-20B AFW Header Section Isolation Open V2-16A AFW Header Discharge to S/G "A" Closed V2-16B AFW Header Discharge to S/G "B" Closed V2-16C AF Header Discharge to S/G "C" Closed V2-14A SD FWP Discharge to S/G "A" Closed V2-14B SD FWP Discharge to S/G "B" Closed V2-14C SD FWP Discharge to S/G "C" Closed FCV-1424 MDP "A" Flow Control Valve Operable/Closed FIC-1424 MDP "A" Flow Controller Energized FCV-1425 MDP "B" Flow Control Valve Operable/Closed FIC-1425 MDP "B" Flow Controller Energized FCV-6416 SDP Flow Control Valve Operable/Closed FIC-6416 SDP Flow Controller Energized CCW Pump Room AFW-24 Service Water Supply Valve Locked Closed SW-1 18 Service Water Supply Valve Locked Closed Auxiliary Feedwater Pump Room AFW-22 AFW Pumps "A" & "B" Suction Locked Open AFW-28 MDP "A" Suction Isolation Locked Open AFW-29 MDP "B" Suction Isolation Locked Open AFW-42 AFW Pump "A" Recirculation Isol.

Open AFW-43 AFW Pump "B" Recirculation Isol.

Open AFW-44 AFW Recirculation Isolation Open FCV-1424 MDP "A" AFW Flow Control Closed FCV-1424 MDP "A" AFW Flow Control Manual Override Locked Disengaged FCV-1425 MDP "B" AFW Flow Control Closed FCV-1425 MDP "B" AFW Flow Control Manual Override Locked Disengaged 3.7 NUREG/CR-5833

Inspection Guidance Table 3.1 (Continued)

Actual Component #

Component Name Required Position Position AFW-53 V2-16A Inlet Isolation Locked Open AFW-54 V2-16B Inlet Isolation Locked Open AFW-55 V2-16C Inlet Isolation Locked Open AFW-62 V2-16A Outlet Isolation Locked Open AFW-63 V2-16B Outlet Isolation Locked Open AFW-64 V2-16C Outlet Isolation Locked Open Thrbine Building AFW-1 AFW Pumps Suction From CST Locked Open AFW-104 AFW Pumps Suction From CST Locked Open AFW-4 SD AFW Pump Suction Locked Open AFW-17 SD AFW Pump Recirculation Isolation Open AFW-20 SD AFW Pump Discharge Isolation Locked Open FCV-6416 SD AFW Flow Control Valve Open FIC-6416 SD AFW Flow Control Valve Manual Override Locked Disengaged MS-262A MSV1-8A Inlet Isolation Locked Open MS-262B MSV1-8B Inlet Isolation Locked Open MS-262C MSV1-8C Inlet Isolation Locked Open MS-VI-8A SDP Steam Shutoff Valve Closed MS-VI-8B SDP Steam Shutoff Valve Closed MS-VI-8C SDP Steam Shutoff Valve Closed MS-154 Steam to AFW Pump Isolation Locked Open TDP Throttle-Trip Valve Open FW-6A SG "A" Feed Reg Bypass Outlet Isolation Locked Open FW-6B SG "B" Feed Reg Bypass Outlet Isolation Locked Open FW-6C SG "C" Feed Reg Bypass Outlet Isolation Open Condensate Storage 'nk Area DW-19 Deepwell Emergency Backup Tb AFW Suction Isolation Locked Closed DW-21 AFW Suction Isolation From Deepwell Emergency Backup Locked Closed Check Valves AFW-40 Piping Upstream of Check Valve Cool AFW-41 Piping Upstream of Check Valve Cool AFW-68 Piping Upstream of Check Valve Cool AFW-69 Piping Upstream of Check Valve Cool AFW-70 Piping Upstream of Check Valve Cool AFW-84 Piping Upstream of Check Valve Cool NUREG/CR-5833 3.8

Inspection Guidance Table 3.1 (Continued)

Actual Component #

Component Name Required Position Position FW-8A Piping Upstream of Check Valve Cool FW-8B Piping Upstream of Check Valve Cool FW-8C Piping Upstream of Check Valve Cool 3.9 NUREG/CR-5833

4 Generic Risk Insights From PRAs PRAs for 13 PWRs were analyzed to identify risk-Feed-and-bleed cooling fails either due to failure of important accident sequences involving loss of AFW, the operator to initiate it, or due to hardware and to identify and risk-prioritize the component failure failures, resulting in core damage.

modes involved. The results of this analysis are de scribed in this section. They are consistent with results Loss of Main Feedwater reported by INEL and BNL (Gregg et al. 1988, and Travis et al., 1988).

A feedwater line break drains the common water source for MFW and AFW The operators fail to H. B. Robinson PRA analysis indicates a slightly differ-provide feedwater from other sources, and fail to ent risk-priority for AFW system component failure initiate feed-and-bleed cooling, resulting in core modes. H.B. Robinson plant specific information is damage.

contained in Section 3.0. Thble 4.1 shows a comparison of H. B. Robinson PRA results to the generic insights A loss of main feedwater trips the plant, and AFW identified from the analysis of 13 PWRs.

fails due to operator error and hardware failures.

The operators fail to initiate feed-and-bleed cooling, resulting in core damage.

4.1 Risk Important Accident Sequences Steam Generator Tube Rupture (SGTR)

Involving AFW System Failure Loss A SGTR is followed by failure of AFW. Coolant is Loss f Poer Sstemlost from the primary until the refueling water stor

• pwage tank (RWST) is depleted. High pressure injec

• AFW loss ac of offsiten power isflowdb hfiue pofe tion (HPI) fails since recirculation cannot be estab AFW Due to lack of actuating power, the powerlihdfoteemysupancreaag operated relief valves (PORVs) cannot be opened ls.

preventing adequate feed-and-bleed cooling, and re sulting in core damage.

• A station blackout fails all AC power except Vital 4.2 Risk Important Component Failure AC from DC invertors, and all decay heat removal Modes systems except the steam-driven AFW pump. The steam driven AFW pump subsequently fails due to The generic component failure modes identified from hardware failures, resulting in core damage.

PRA analyses as important to AFW system failure are listed below in decreasing order of risk importance.

• A DC bus fails, causing a trip and failure of the power conversion system. One AFW motor-driven

1. Thrbine-Driven Pump Failure to Start or Run.

pump is failed by the bus loss, and the turbine driven pump fails due to loss of turbine or hardware

2. Motor-Driven Pump Failure to Start or Run.

failure. AFW is subsequently lost completely due to other failures. Feed-and-bleed cooling fails because

3. TDP or MDP Unavailable due to Thst or PORV control is lost, resulting in core damage.

Maintenance.

Transient-Caused Reactor or Turbine Trip

4. AFW System Valve Failures A transient-caused trip is followed by a loss of the steam admission valves power conversion system (PCs) and AFW.

4.1 NUREG/CR-5833

Generic Risk trip and throttle valve In addition to individual hardware, circuit, or instru ment failures, each of these failure modes may result

• flow control valves from common causes and human errors. Common cause failures of AFW pumps are particularly risk im

• pump discharge valves portant. Valve failures are somewhat less important due to the multiplicity of steam generators and connection

• pump suction valves paths. Human errors of greatest risk importance in volve: failures to initiate or control system operation

• valves in testing or maintenance.

when required; failure to restore proper system lineup after maintenance or testing; and failure to switch to

5. Supply/Suction Sources alternate sources when required.

condensate storage tank stop valve suction valves.

NUREG/CR-5833 4.2

Generic Risk Table 4.1 Risk Prioritized Comparison of H. B. Robinson and Generic PRA Results H. B. Robinson PRA Generic PRA

1. Turbine-driven pump unavailable due to test or

1. TUrbine-driven pump failure to start or run maintenance.

2. Turbine-driven pump failure to start or run.

2. Motor-driven pump failure to start or run.

3. Operator failure to restore turbine-driven pump valve 3. TDP or MDP unavailable due to test or maintenance.

position after test and maintenance.

4. Common cause failure of all three AFW pumps.

4. AFW system valve failures.

- steam admission valves

- trip and throttle valves

- flow control valves

- pump discharge valves

- Pump suction valves

- valves in testing or maintenance

5. Turbine-driven pump flow transmitter miscalibrated.

5. Supply/suction sources.

- Condensate storage tank stop valves

- hot well inventory

- suction valves

6. Backup AFW suction valves fail to open.

7. Other miscalibrations and valve mispositioning.

8. Motor-driven pump failures.

4.3 NUREG/CR-5833

5 Failure Modes Determined From Operating Experience This section describes the primary root causes of AFW 5.1.2 Tbrbine Driven Pump Failures system component failures, as determined from a review of operating histories at H. B. Robinson and at other Three events have resultedin decreased operational PWRs throughout the nuclear industry. Section 5.1 de-readiness of the turbine driven pump. Failure causes scribes experience at H. B. Robinson from 1980 to 1991.

were attributed to steam binding of the turbine-driven Section 5.2 summarizes information compiled from a pump resulting from motor-operated isolation valve variety of NRC sources, including AEOD analyses and leakage and worn parts in the overspeed trip reports, information notices, inspection and enforce-mechanism.

ment bulletins, and generic letters, and from a variety of INPO reports as well. Some Licensee Event Reports 5.1.3 EHC Flow Control and Motor Operated and NPRDS event descriptions were also reviewed.

Finally, information was included from reports of NRC sponsored studies of the effects of plant aging, which in-More than fifteen events have resulted in impaired op clude quantitative analysis of AFW system failure re ports. This information was used to identify the various er ation aWsst fl contr ad root causes expected for the broad PRA-based failureinstrumentation and control cir events identified in Section 4.0, resulting in the inspec-cuit failures, valve hardware failures, and human errors.

tion guidelines presented in Section 3.0.

Valves have failed to operate properly due to failure of control components, broken or dirty contacts, mis H. B Robnso erincealigned or broken limit switches. Human errors have re 5.1 H. B. Robinson Experience sulted in improper wiring. Since 1973, flow control and isolation valves have been extensively modified or re The AFW system at H. B. Robinson has experienced placed to improve AFW system performance. Hydro failures of the AFW pumps, pump discharge flow con-motors were installed on the flow control valves and V2 trol valves, the turbine steam admission and supply 16ABC and V2-14ABC valves were replaced with valves, turbine trip and throttle valve, pump discharge double disc valves, which corrected system back flow isolation valves, service water backup supply valves, and problems, and limitorque MOV operators were modi numerous system check valves. Failure modes include fied to prevent hydraulic lockup.

electrical, instrumentation and control, hardware fail ures, and human errors. Since the mid-1970's, the AFW 5.1.4 Check Valves system has undergone extensive modifications to in crease system reliability. As a result of these modifica-Five events of check valve failure occurred during the tions, many of the earlier problems identified within the period analyzed. Normal wear and aging was cited as AFW system have been rectified.

the failure mode, resulting in leakage. In 1982, check valves were replaced with a bonnet and spring plunger 5.1.1 Motor Driven Pump Failures type arrangement to help prevent system back leakage.

There have been nine events which involved failure of the motor driven pumps during several modes of opera-5.2 Industry Wide Experience tion. Failure modes involved instrumentation and con trol circuit failures, pump hardware failures, and human Human errors, design/engineering problems and errors, failures during maintenance activities. Three cases of and component failures are the primary root causes of arcing within the motor required motor replacement.

AeW System failures identified in a review of industry 5.1 NUREG/CR-5833

Failure Modes wide system operating history. Common cause failures, sensors having control functions. Incorrect handswitch which disable more than one train of this operationally positioning and inadequate temporary wiring changes redundant system, are highly risk significant, and can re-have also prevented automatic starts of multiple pumps.

sult from all of these causes.

Factors identified in studies of mispositioning errors in clude failure to add newly installed valves to valve This section identifies important common cause failure checklists, weak administrative control of tagging, res modes, and then provides a broader discussion of the toration, independent verification, and locked valve log single failure effects of human errors, design/

ging, and inadequate adherence to procedures. Illegible engineering problems and errors, and component fail-or confusing local valve labeling, and insufficient train ures. Paragraphs presenting details of these failure ing in the determination of valve position may cause or modes are coded (e.g., CC1) and cross-referenced by in-mask mispositioning, and surveillance which does not spection items in Section 3.0.

exercise complete system functioning may not reveal mispositionings.

5.2.1 Common Cause Failures CC3. At ANO-2, both AFW pumps lost suction due to The dominant cause of AFW system multiple-train fail-steam binding when they were lined up to both the CST ures has been human error. Design/engineering errors and the hot startup/blowdown demineralizer effluent and component failures have been less frequent, but (AEOD/C404,1984). At Zion-i steam created by nevertheless significant, causes of multiple train failures.

running the turbine-driven pump deadheaded for one minute caused trip of a motor-driven pump sharing the CC1. Human error in the form of incorrect operator in-same inlet header, as well as damage to the turbine tervention into automatic AFW system functioning dur-driven pump (Region 3 Morning Report, 1/17/90). Both ing transients resulted in the temporary loss of all safety-events were caused by procedural inadequacies.

grade AFW pumps during events at Davis Besse (NUREG-1154, 1985) and 1Iojan (AEOD/T416, 1983).

CC4. Design/engineering errors have accounted for a In the Davis Besse event, improper manual initiation of smaller, but significant fraction of common cause fail the steam and feedwater rupture control system ures. Problems with control circuit design modifications (SFRCS) led to overspeed tripping of both turbine-at Farley defeated AFW pump auto-start on loss of driven AFW pumps, probably due to the introduction of main feedwater. At Zion-2, restart of both motor driven condensate into the AFW turbines from the long, un-pumps was blocked by circuit failure to deenergize when heated steam supply lines. (The system had never been the pumps had been tripped with an automatic start sig tested with the abnormal, cross-connected steam supply nal present (IN 82-01, 1982). In addition, AFW control lineup which resulted.) In the 'Liojan event the operator circuit design reviews at Salem and Indian Point have incorrectly stopped both AFW pumps due to misinter-identified designs where failures of a single component pretation of MFW pump speed indication. The diesel could have failed all or multiple pumps (IN 87-34, driven pump would not restart due to a protective fea-1987).

ture requiring complete shutdown, and the turbine driven pump tripped on overspeed, requiring local reset CC5. Incorrect setpoints and control circuit settings re of the trip and throttle valve. In cases where manual in-sulting from analysis errors and failures to update proce tervention is required during the early stages of a tran-dures have also prevented pump start and caused pumps sient, training should emphasize that actions should be to trip spuriously. Errors of this type may remain unde performed methodically and deliberately to guard tected despite surveillance testing, unless surveillance against such errors.

tests model all types of system initiation and operating conditions. A greater fraction of instrumentation and CC2. Valve mispositioning has accounted for a signif-control circuit problems has been identified during icant fraction of the human errors failing multiple trains actual system operation (as opposed to surveillance test of AFW This includes closure of normally open suction ing) than for other types of failures.

valves or steam supply valves, and of isolation valves to NUREG/CR-5833 5.2

Failure Modes CC6. On two occasions at a foreign plant, failure of a CCIO. Common cause failures have also been caused by balance-of-plant inverter caused failure of two AFW component failures (AEOD/C404,1984). At Surry-2, pumps. In addition to loss of the motor driven pump both the turbine driven pump and one motor driven whose auxiliary start relay was powered by the invertor, pump were declared inoperable due to steam binding the turbine driven pump tripped on overspeed because caused by leakage of hot water through multiple check the governor valve opened, allowing full steam flow to valves. At Robinson-2 both motor driven pumps were the turbine. This illustrates the importance of assessing found to be hot, and both motor and steam driven the effects of failures of balance of plant equipment pumps were found to be inoperable at different times.

which supports the operation of critical components.

Backleakage at Robinson-2 passed through closed The instrument air system is another example of such a motor-operated isolation valves in addition to multiple system.

check valves. At Farley, both motor and turbine driven pump casings were found hot, although the pumps were CC7. Multiple AFW pump trips have occurred at not declared inoperable. In addition to multi-train fail Millstone-3, Cook-1, Trojan and Zion-2 (IN 87-53, ures, numerous incidents of single train failures have 1987) caused by brief, low pressure oscillations of suc-occurred, resulting in the designation of "Steam Binding tion pressure during pump startup. These oscillations of Auxiliary Feedwater Pumps" as Generic Issue 93.

occurred despite the availability of adequate static This generic issue was resolved by Generic Letter 88-03 NPSH. Corrective actions taken include: extending the (Miraglia, 1988), which required licensees to monitor time delay associated with the low pressure trip, remov-AFW piping temperatures each shift, and to maintain ing the trip, and replacing the trip with an alarm and procedures for recognizing steam binding and for restor operator action.

ing system operability.

CC8. Design errors discovered during AFW system re-CCI 1. Common cause failures have also failed motor analysis at the Robinson plant (IN. 89-30, 1989) and at operated valves. During the total loss of feedwater Millstone-I resulted in the supply header from the CST event at Davis Besse, the normally-open AFW isolation being too small to provide adequate NPSH to the valves failed to open after they were inadvertently pumps if more than one of the three pumps were op-closed. The failure was due to improper setting of the erating at rated flow conditions. This could lead to mul-torque switch bypass switch, which prevents motor trip tiple pump failure due to cavitation. Subsequent re-on the high torque required to unseat a closed valve.

views at Robinson identified a loss of feedwater tran-Previous problems with these valves had been addressed sient in which inadequate NPSH and flows less than by increasing the torque switch trip setpoint - a fix which design values had occurred, but which were not recog-failed during the event due to the higher torque required nized at the time. Event analysis and equipment trend-due to high differential pressure across the valve. Sim ing, as well as surveillance testing which duplicates ser-ilar common mode failures of MOVs have also occurred vice conditions as much as is practical, can help identify in other systems, resulting in issuance of Generic Letter such design errors.

89-10, "Safety Related Motor-Operated Valve Testing and Surveillance (Partlow, 1989)." This generic letter CC9. Asiatic clams caused failure of two AFW flow requires licensees to develop and implement a program control valves at Catawba-2 when low suction pressure to provide for the testing, inspection and maintenance caused by starting of a motor-driven pump caused suc-of all safety-related MOVs to provide assurance that tion source realignment to the Nuclear Service Water they will function when subjected to design basis system. Pipes had not been routinely treated to inhibit conditions.

clam growth, nor regularly monitored to detect their presence, and no strainers were installed. The need for CC2. Other component failures have also resulted in surveillance which exercises alternative system opera-AFW multi-train failures. These include out-of tional modes, as well as complete system functioning, is adjustment electrical flow controllers resulting in im emphasized by this event. Spurious suction switchover proper discharge valve operation, and a failure of oil has also occurred at Callaway and at McGuire, although cooler cooling water supply valves to open due to silt no failures resulted.

accumulation.

5.3 NUREG/CR-5833

Failure Modes 5.2.2 Human Errors controlled turbine acceleration and buildup of oil pres sure to control the governor valve when full steam flow HEL. The overwhelmingly dominant cause of problems is admitted.

identified during a series of operational readiness eval uations of AFW systems was human performance. The DE2. Overspeed trips of Rrry turbines have been majority of these human performance problems resulted caused by condensate in the steam supply lines. Con from incomplete and incorrect procedures, particularly densate slows down the turbine, causing the governor with respect to valve lineup information. A study of valve to open farther, and overspeed results before the valve mispositioning events involving human error iden-governor valve can respond, after the water slug clears.

tified failures in administrative control of tagging and This was determined to be the cause of the loss-of logging, procedural compliance and completion of steps, all-AFW event at Davis Besse (AEOD6O2,1986), with verification of support systems, and inadequate proce-condensation enhanced due to the long length of the dures as important. Another study found that valve mis-cross-connected steam lines. Repeated tests following a positioning events occurred most often during mainte-cold-start trip may be successful due to system heat up.

nance, calibration, or modification activities. Insuf ficient training in determining valve position, and in DE3. Tirbine trip and throttle valve (TIV) problems administrative requirements for controlling valve posi-are a significant cause of turbine driven pump failures tioning were important causes, as was oral task assign-(IN 84-66). In some cases lack of TTV position indica ment without task completion feedback.

tion in the control room prevented recognition of a tripped 'TTV In other cases it was possible to reset HE2. Turbine driven pump failures have been caused by either the overspeed trip or the TFV without resetting human errors in calibrating or adjusting governor speed the other. This problem is compounded by the fact that control, poor governor maintenance, incorrect adjust-the position of the overspeed trip linkage can be mis ment of governor valve and overspeed trip linkages, and leading, and the mechanism may lack labels indicating errors associated with the trip and throttle valve. TIV-when it is in the tripped position (AEOD/C602,1986).

associated errors include physically bumping it, failure to restore it to the correct position after testing, and DE4. Startup of turbines with Woodward Model failures to verify control room indication of TIV posi-PG-PL governors within 30 minutes of shutdown has re tion following actuation.

sulted in overspeed trips when the speed setting knob was not exercised locally to drain oil from the speed set HE3. Motor driven pumps have been failed by human ting cylinder. Speed control is based on startup with an errors in mispositioning handswitches, and by procedure empty cylinder. Problems have involved turbine rota deficiencies.

tion due to both procedure violations and leaking steam.

Tebrry has marketed two types of dump valves for auto 5.2.3 Design/Engineering Problems and matically draining the oil after shutdown (AEOD/C602, Errors 1986).

DEL. As noted above, the majority of AFW subsystem At Calvert Cliffs, a 1987 loss-of-offsite-power event re failures, and the greatest relative system degradation, quired a quick, cold startup that resulted in turbine trip has been found to result from turbine-driven pump fail-due to PG-PL governor stability problems. The short ures Ovrsped rip ofTerr tubins cntrlle byterm corrective action was installation of stiffer buffer ures. Overspeed trips of lbrry turbines controlled by srns(N8-9 98.Srelac a lasbe Woodward governors have been a significant source of prings (I 88-09, 1988). Srich als e

these failures (AEOD/C602, 1986). In many cases these prce by trbin which illstates he overspeed trips have been caused by slow response of a ioranc of tsingcwichli Woodward Model EG governor on startup, at plants where full steam flow is allowed immediately. This over-DE5. Reduced viscosity of gear box oil heated by prior sensitivity has been removed by installing a startup operation caused failure of a motor driven pump to start steam bypass valve which opens first, allowing a NUREG/CR-5833 5.4

Failure Modes due to insufficient lube oil pressure. Lowering the pres-protected, yet in a way which emphasizes system func sure switch setpoint solved the problem, which had not tion over protection of the operator.

been detected during testing.

CF1. The common-cause steam binding effects of check DE6. Waterhammer at Palisades resulted in AFW line valve leakage were identified in Section 5.2.1, entry and hanger damage at both steam generators. The AFW CC1O. Numerous single-train events provide additional spargers are located at the normal steam generator level, insights into this problem. In some cases leakage of hot and are frequently covered and uncovered during level MFW past multiple check valves in series has occurred fluctuations. Waterhammers in top-feed-ring steam because adequate valve-seating pressure was limited to generators resulted in main feedline rupture at Maine the valves closest to the steam generators (AEODC4O4, Yankee and feedwater pipe cracking at Indian Point-2 1984). At Robinson, the pump shutdown procedure was (IN 84-32, 1984).

changed to delay closing the MOVs until after the check valves were seated and hydromotors were installed on DE7. Manually reversing the direction of motion of an the flow control valves. At Farley, check valves were operating valve has resulted in MOV failures where changed from swing type to lift type. Check valve re such loading was not considered in the design (AEOD/

work has been done at a number of plants. Different C603, 1986). Control circuit design may prevent this, re-valve designs and manufacturers are involved in this quiring stroke completion before reversal.

problem, and recurring leakage has been experienced, even after repair and replacement.

DE8. At each of the units of the South Thxas Project, space heaters provided by the vendor for use in prein-CF2. At Robinson, heating of motor operated valves by stallation storage of MOVs were found to be wired in check valve leakage has caused thermal binding and fail parallel to the Class 1E 125 V DC motors for several ure of AFW discharge valves to open on demand. At AFW valves (IR 50-489/89-11; 50-499/89-11, 1989). The Davis Besse, high differential pressure across AFW in valves had been environmentally qualified, but not with jection valves resulting from check valve leakage has the non-safety-related heaters energized.

prevented MOV operation (AEODwC6a3, 1986).

5.2.4 Component Failures CR3. Gross check valve leakage at McGuire and Robinson caused overpressurization of the AFW suc Generic Issue II.E.6.1, "In Situ 'hsting Of ValvesC was tion piping. At a foreign PWR it resulted in a severe divided into four sub-issues (Beckjord, 1989), three of waterhaNmer event. At Palo Verde-2 the MFW suction which relate directly to prevention of AFW system com-piping was overpressurized by check valve leakage from ponent failure. At the request of the NRC, in-situ test-the AFW system (AEOD/C404, 19814). Gross check ing of check valves was addressed by the nuclear valve leakage through idle pumps represents a potential industry, resulting in the EPRI report, "Application diversion of AeW pump flow.

Guidelines for Check Valves in Nuclear Power Plants (Brooks, 1988)." This extensive report provides in-CF4. Roughly one third of AFW system failures have formation on check valve applications, limitations, and been due to valve operator failures, with about equal inspection techniques. In-situ testing of MOVs was ad-failures for MOVs and AOVs. Almost half of the MOV dressed by Generic Letter 89-10, "Safety Related Motor-failures were due to motor or switch failures (Casada, Operated Valve tsting and Surveillance" (Partlow, 1989). An extensive study of MOV events (AEOD/

1989) which requires licensees to develop and imple-C603, 1986) indicates continuing inoperability problems ment a program for testing, inspection and maintenance caused by: torque switch/limit switch settings, adjust of all safety-related MOVs. "Thermal Overload Protec-ments, or failures; motor burnout; improper sizing or tion for Electric Motors on Safety-Related Motor-use of thermal overload devices; premature degradation Operated Valves - Generic Issue II.E.6.C1 (Rothberg, related to inadequate use of protective devices; damage 1988)" concludes that valve motors should be thermally due to misuse (valve throttling, valve operator 5.5 NUREG/CR-5833

Failure Modes hammering); mechanical problems (loosened parts, im-actuate the MOV torque switch, due to grease trapped proper assembly); or the torque switch bypass circuit im-in the spring pack. During a surveillance at lojan, fail properly installed or adjusted. The study concluded that ure of the torque switch to trip the TFV motor resulted current methods and procedures at many plants are not in tripping of the thermal overload device, leaving the adequate to assure that MOVs will operate when turbine driven pump inoperable for 40 days until the needed under credible accident conditions. Specifically, next surveillance (AEOD/E702, 1987). Problems result a surveillance test which the valve passed might result in from grease changes to EXXON NEBULA EP-0 grease, undetected valve inoperability due to component failure one of only two greases considered environmentally (motor burnout, operator parts failure, stem disc sep-qualified by Limitorque. Due to lower viscosity, it aration) or improper positioning of protective devices slowly migrates from the gear case into the spring pack.

(thermal overload, torque switch, limit switch). Generic Grease changeover at Vermont Yankee affected 40 of Letter 89-10 (Partlow, 1989) has subsequently required the older MOVs of which 32 were safety related. Grease licensees to implement a program ensuring that MOV relief kits are needed for MOV operators manufactured switch settings are maintained so that the valves will op-before 1975. At Limerick, additional grease relief was erate under design basis conditions for the life of the required for MOVs manufactured since 1975. MOV re plant.

furbishment programs may yield other changeovers to EP-0 grease.

CF5. Component problems have caused a significant number of turbine driven pump trips (AEOD/C602, CF9. For AFW systems using air operated valves, 1986). One group of events involved worn tappet nut almost half of the system degradation has resulted from faces, loose cable connections, loosened set screws, im-failures of the valve controller circuit and its instrument properly latched TTVs, and improper assembly.

inputs (Casada, 1989). Failures occurred predominantly Another involved oil leaks due to component or seal at a few units using automatic electronic controllers for failures, and oil contamination due to poor maintenance the flow control valves, with the majority of failures due activities. Governor oil may not be shared with turbine to electrtcal hardware. At Thrkey Point-3, controller lubrication oil, resulting in the need for separate oil malfunction resulted from water in the Instrument Air changes. Electrical component failures included transis-system due to maintenance inoperability of the air tor or resistor failures due to moisture intrusion, erro-dryers.

neous grounds and connections, diode failures, and a faulty circuit card.

CF1O. For systems using diesel driven pumps, most of the failures were due to start control and governor speed CF6. Electrohydraulic-operated discharge valves have control circuitry. Half of these occurred on demand, as performed very poorly, and three of the five units using opposed to during testing (Casada, 1989).

them have removed them due to recurrent failures.

Failures included oil leaks, contaminated oil, and hy-CF11. For systems using AOVs, operability requires the draulic pump failures.

availability of Instrument Air (LA), backup air, or back up nitrogen. However, NRC Maintenance Team In CF7. Control circuit failures were the dominant source spections have identified inadequate testing of check of motor driven AFW pump failures (Casada, 1989).

valves isolating the safety-related portion of the IA sys This includes the controls used for automatic and man-tem at several utilities (Letter, Roe to Richardson).

ual starting of the pumps, as opposed to the instrumen-Generic Letter 88-14 (Miraglia, 1988), requires licen tation inputs. Most of the remaining problems were due sees to verify by test that air-operated safety-related to circuit breaker failures.

components will perform as expected in accordance with all design-basis events, including a loss of normal IA.

CF8. "Hydraulic lockup" of Limitorque SMB spring packs has prevented proper spring compression to NUREG/CR-5833 5.6

6 References Beckjord, E. S. June 30, 1989. Closeout of Generic Issue AEOD Reports II.E.6.1, "In Situ Testing of Valves". Letter to V Stello, Jr., U.S. Nuclear Regulatory Commission, Washington, AEOD/C404. W D. Lanning. July 1984. Steam Binding D.C.

of Auxiliary Feedwater Pumps. U.S. Nuclear Regulatory Commission, Washington, D.C.

Brooks, B. P. 1988. Application Guidelines for Check Valves in Nuclear Power Plants. NP-5479, Electric AEOD/C602. C. Hsu. August 1986. Operational Exper Power Research Institute, Palo Alto, California.

ience Involving Turbine Overspeed Trips. U.S. Nuclear Regulatory Commission, Washington, D.C.

Casada, D. A. 1989. Auxiliary Feedwater System Aging Study. Volume 1. Operating Experience and Current AEOD/C603. E. J. Brown. December 1986. A Review Monitoring Practices. NUREG/CR-5404. U.S. Nuclear of Motor-Operated Valve Performance. U.S. Nuclear Regulatory Commission, Washington, D.C.

Regulatory Commission, Washington, D.C.

Gregg, R. E. and R. E. Wright. 1988. Appendix Review AEOD/E702.

. J. Brown. March 19,1987. MOVail for Dominant Generic Contributors. BLB-3 1-88. Idaho ure Due to Hydraulic Lockup From Excessive Grease in National Engineering Laboratory, Idaho Falls, Idaho.

Spring Pack. U.S. Nuclear Regulatory Commission, Washington, D.C.

Miraglia, FA J. February 17, 1988. Resolution of Generic Safety Issue 93, "Steam Binding of Auxiliary Feedwater AEOD/T4I6. January 22, 1983. Loss of ESFAuxiliary Pumps" (Generic Letter 88-03). U.S. Nuclear Regulatory Feedwater Pump Capability at Trojan on January 22, Commission, Washington, D.C.

1983. U.S. Nuclear Regulatory Commission, Washington, D.C.

Miraglia, E

J. August 8,B1988. InstrumentAir Supply System Problems Affecting Safety-Related Equipment Information Notices (Generic Letter 88-14). U.S. Nuclear Regulatory Com mission, Washington, D.C.

IN 82-01. January 22, 1982. Auxiliary Feedwater Pump Lockout Resulting from Westinghouse W2 Switch Circuit Partlow, J. G. June 28, 1989. Safety-Related Motor-Modification. U.S. Nuclear Regulatory Commission, Operated Valve Testing and Surveillance (Generic Letter Washington, D.C.

89-10). U.S. Nuclear Regulatory Commission, Washington, D.C.

IN 84-32. E. L. Jordan. April 18,1984. Auxiliary Feed water Sparger and Pipe Hangar Damage. U.S. Nuclear Rothberg, 0. June 1988. Thermal Overload Protection Regulatory Commission, Washington, D.C.

for Electric Motors on Safety-Related Motor-Operated Valves - Generic Issue 8I.E.61. NUREG-1296. U.S.

IN 84-66. August 17, 1984. Undetected Unavailability of Nuclear Regulatory Commission, Washington, D.C.

the Turbine-Driven Auxiliary Feedwater Train. U.S.

Nuclear Regulatory Commission, Washington, D.C.

Travis, R. and J. Agtylor. 1989. Development of Guid ance for Generic, Functionally Oriented PRA -Based Team IN 87-34. C.. Rossi. July 24,1987. Single Failures in Inspections for BR Plants-Identification of Risk-Auxiliary Feedwater Systems. U.S. Nuclear Regulatory Important Systems, Components and Human Actions.

Commission, Washington, D.C.

TLR-A-3874-TGA Brookhaven National Laboratory, Upton, New York.

6.1 NUREG/CR-5833

References IN 87-53. C. E. Rossi. October 20, 1987. Auxiliary Inspection Report Feedwater Pump Trips Resulting from Low Suction Pres sure. U.S. Nuclear Regulatory Commission, IR 50-489/89-11; 50-499/89-11. May 26, 1989. South Washington, D.C.

Texas Project Inspection Report. U.S. Nuclear Regula tory Commission, Washington, D.C.

IN 88-09. C. E. Rossi. March 18, 1988. Reduced Reli ability of Steam-Driven Auxiliary Feedwater Pumps NUREG Report Caused by Instability of Woodward PG-PL Type Gover nors. U.S. Nuclear Regulatory Commission, NUREG-1154. 1985. Loss of Main and Auxiliary Feed Washington, D.C.

water Event at the Davis Besse Plant on June 9, 1985.

U.S. Nuclear Regulatory Commission, Washington, IN 89-30. R. A. Azua. August 16, 1989. Robinson D.C.

Unit 2 Inadequate NPSH ofAuxiliary Feedwater Pumps.

Also, Event Notification 16375, August 22, 1989. U.S.

Nuclear Regulatory Commission, Washington, D.C.

NUREG/CR-5833 6.2

Distribution No. of No. of Copies Copies OFFSITE 4

H. B. Robinson Resident Inspector Office U.S. Nuclear Regulatory Commission J. H. Taylor Brookhaven National Laboratory B. K. Grimes Bldg. 130 OWFN 9 A2 Upton, NY 11973 F Congel R. Travis OWFN 10 E4 Brookhaven National Laboratory Bldg. 130 A. C. Thadani Upton, NY 11973 OWFN 8E2 J. Bickel G. M. Holahan EG&G Idaho, Inc.

OWFN 8E2 P.O. Box 1625 Idaho Falls, ID 83415 W T Russell OWFN 12 E23 Dr. D. R. Edwards Professor of Nuclear Engineering H. N. Berkow University of Missouri - Rolla OWFN 14H22 Rolla, MO 65401 K. Campe ONSITE OWFN 1 A2 27 Pacific Northwest Laboratory 10 J. Chung OWFN 10 A2 S. R. Doctor L. R. Dodd 2

B. Thomas B. F Gore (10)

OWFN 12 H26 R. C. Lloyd N. E. Moffitt (5)

U.S. Nuclear Regulatory Commission - Region 2 B. D. Shipp F A. Simonen S. D. Ebneter T V. Vo A. F Gibson Publishing Coordination E. W Merschoff Technical Report File (5)

L. W Garner Distr.1 NUREG/CR-5833

NRd FORM 335 U.S. NUCLEAR REGULATORY COMMISSION

1. REPORT NUMBER (2-89)

(Assigned by NRC. Add Vol., Supp., Rev.,

NRCM 1102, and Addendum Numbers, if any.)

3201,3202 BIBLIOGRAPHIC DATA SHEET NUREG/CR-5833 (See instructions on the reverse)

PNL-7907

2. TITLE AND SUBTITLE Auxiliary Feedwater System Risk-Based Inspection Guide for the H. B. Robinson Nuclear Power Plant

3.

DATEREPORTPUBLISHED MONTH YEAR August 1993

4. FIN OR GRANT NUMBER L1310

5. AUTHOR(S)

6. TYPE OF REPORT N. E. Moffitt, R. C. Lloyd, B. F. Gore, T. V. Vo L. W. Garner*

Technical

7. PERIOD COVERED (inclusive Dates) 11/91 to 6/93

8. PERFORMING ORGANIZATION -

NAME AND ADDRESS OfNRC, provide Division, Office orRegion, U.S. Nuclear Regulatory Commission, andmailing address; ifcontractor, provide name and mailing addessJ Pacific Northwest Laboratory

• U.S. Nuclear Regulatory Commission Richland, WA 99352 Washington, DC 20555-0001

9. SPONSORING ORGANIZATION -

NAME AND ADDRESS Of NRC, type "Same as above"; if contractor, provide NRC Division, Office or Region, U.S. Nuclear Regulatory Commission, and mailing address.)

Division of Systems Safety and Analysis Office of Nuclear Reactor Regulation U.S. Nuclear Regulatory Commission Washington, DC 20555-0001

10. SUPPLEMENTARY NOTES

11. ABSTRACT (200 words or less)

In a study sponsored by the U.S. Nuclear Regulatory Commission (NRC), Pacific Northwest Laboratory has developed and applied a methodology for deriving plant-specific risk-based inspection guidance for the auxiliary feedwater (AFW) system at pressurized water reactors that have not undergone probabilistic risk assessment (PRA).

This methodology uses existing PRA results and plant operating experience information.

Existing PRA-based inspection guidance information recently developed for the NRC for various plants was used to identify generic component failure modes. This information was then combined with plant-specific and industry-wide component information and failure data to identify failure modes and failure mechanisms for the AFW system at the selected plants. H. B. Robinson was selected as one of a series of plants for study.

The product of this effort is a prioritized listing of AFW failures which have occurred at the plant and at other PWRs. This listing is intended for use by NRC inspectors in the preparation of inspection plans addressing AFW ri'sk-important components at the H. B. Robinson plant.

12. KEY WORDS/DESCRIPTORS (List words or phrases that will assist researchers in locating the report.)

13. AVAILABILITY STATEMENT Unlimited Inspection, Risk, PRA, H. B. Robinson, Auxiliary Feedwater (AFW)

14. SECURITY CLASSIFICATION (This Page)

Unclassified (This Report)

Unclassified

15. NUMBER OF PAGES

16. PRICE NRC FORM 335 (2-89)

Printed on recycled paper Federal Recycling Program