NUREG/CR-5833, Forwards Draft NUREG/CR-5833, Auxiliary Feedwater Sys Risk-Based Insp Guide for Hb Robinson Nuclear Power Plant. NRC Planning to Send Contractor Pnl to Visit Plant on 930414-15

Forwards Draft NUREG/CR-5833, Auxiliary Feedwater Sys Risk-Based Insp Guide for Hb Robinson Nuclear Power Plant. NRC Planning to Send Contractor Pnl to Visit Plant on 930414-15

ML14184B046
Person / Time
Site:	Robinson
Issue date:	03/31/1993
From:	Mozafari B Office of Nuclear Reactor Regulation
To:	Watson R Carolina Power & Light Co
References
RTR-NUREG-CR-5833 NUDOCS 9304050204
Download: ML14184B046 (26)

Text

Docket No. 50-261 Marth 31, 1993 Mr. R. A. Watson Senior Vice President Nuclear Generation Carolina Power & Light Company Post Office Box 1551 Raleigh, North Carolina 27602

SUBJECT:

RISK-BASED INSPECTION GUIDE - H. B. ROBINSON STEAM ELECTRIC PLANT, UNIT NO. 2 This is in reference tothe development of a Risk-Based Inspection Guide (RIG) to be published as a NUREG report under an NRC Technical Assistance contract with Pacific Northwest Laboratory (PNL).

This RIG is intended to provide useful guidance for NRC inspection activities.

The purpose of this letter is to inform you that we are planning to send our contractor, Ms. Nancy Moffitt of PNL, and Dr. John Schiffgens of NRC to visit the H. B. Robinson Steam Electric Plant, Unit No. 2, on April 14 and 15, 1993.

The visiting contractor will accompany the Senior Resident Inspector in a system walkdown and verify the accuracy of the information in the RIG. During this visit, the contractor will be available to meet with your staff and receive their comments regarding the RIG in order to reflect your plant status accurately. However, we would like to emphasize that your participation during the visit will be strictly voluntary.

We are enclosing a draft copy of the RIG. If you choose to participate during the visit, please inform me and the RIG Project Coordinator, Dr. Jin Chung at (301) 504-1071.

Sincerely, ORIGINAL SIGNED BY:

Brenda L. Mozafari, Project Manager Project Directorate II-1 Division of Reactor Projects -

I/II

Enclosure:

Office of Nuclear Reactor Regulation Draft RIG cc:

See next page DISTRIBUTION:

Docket Filel OGC NRC/Local PDRs ACRS (10)

PD II-1 Reading E. Merschoff, RII S. Varga J. Mitchell G. Lainas B. Mozafari J. Chung P. Anderson L. Garner H. Christensen cc:

Plant Service list A

OFFICE LA: D :

DRPE AD:PD21:DRPE PRAB NAME PAn son BMozafari:dt JMitchell JChung\\

( '4 O

DATE 3 / O/93

/ /93 3 /30/93 3 /J /93 Document Name: ROBBRIG.LTR 9304050204 930331* ~7~r Ty Reag PDR ADOCK 05000291 Mi 1 B

PDR

Mr. R. A. Watson H. B. Robinson Steam Electric Carolina Power & Light Company Plant, Unit No. 2 cc:

Mr. H. Ray Starling Mr. Dayne H. Brown, Director Manager -

Legal Department Department of Environmental, Carolina Power & Light Company Health and Natural Resources Post Office Box 1551 Division of Radiation Protection Raleigh, North Carolina 27602 Post Office Box 21687 Raleigh, North Carolina 27611-7687 Mr. H. A. Cole Special Deputy Attorney General Mr. Robert P. Gruber State of North Carolina Executive Director Post Office Box 629 Public Staff -

NCUC Raleigh, North Carolina 27602 Post Office Box 29520 Raleigh, North Carolina 27626-0520 U.S. Nuclear Regulatory Commission Resident Inspector's Office Mr. C. R. Dietz H. B. Robinson Steam Electric Plant Vice President Route 5, Box 413 Robinson Nuclear Department Hartsville, South Carolina 29550 H. B. Robinson Steam Electric Plant Post Office Box 790 Regional Administrator, Region II Hartsville, South Carolina 29550 U.S. Nuclear Regulatory Commission 101 Marietta St., N.W., Ste. 2900 Mr. Heyward G. Shealy, Chief Atlanta, Georgia 30323 Bureau of Radiological Health South Carolina Department of Health Mr. Ray H. Chambers, Jr.

and Environmental Control General Manager 2600 Bull Street H. B. Robinson Steam Electric Plant Columbia, South Carolina 29201 Post Office Box 790 Hartsville, South Carolina 29550 Mr. R. B. Starkey Public Service Commission Vice President State of South Carolina Nuclear Services Department Post Office Drawer 11649 Carolina Power & Light Company Columbia, South Carolina 29211 Post Office Box 1551 Raleigh, North Carolina 27602

NUREG/CR-5833 PNL-7907 AUXILIARY FEEDWATER SYSTEM RISK-BASED INSPECTION GUIDE FOR THE H. B. ROBINSON NUCLEAR POWER PLANT N. E. Moffitt R. C. Lloyd B. F. Gore T. V. Vo August 1991 Prepared for Division of Safety Systems and Analysis Office of Nuclear Regulatory Regulation U.S. Nuclear Regulatory Commission under Contract DE-ACO6-76RLO 1830 NRC FIN L1310 Pacific Northwest Laboratory Richland, Washington 99352

SUMMARY

This document presents a compilation of auxiliary feedwater (AFW) system failure information which has been screened for risk significance in terms of failure frequency and degradation of system performance. It is a risk prioritized listing of failure events and their causes that are significant enough to warrant consideration in inspection planning at the H. B. Robinson plant. This information is presented to provide inspectors with increased resources for inspection planning at H. B. Robinson.

The risk importance of various component failure modes was identified by analysis of the results of probabilistic risk assessments (PRAs) for many pressurized water reactors (PWRs). However, the component failure categories identified in PRAs are rather broad, because the failure data used in the PRAs is an aggregate of many individual failures having a variety of root causes.

In order to help inspectors focus on specific aspects of component operation, maintenance and design which might cause these failures, an extensive review of component failure information was performed to identify and rank the root causes of these component failures. Both H. B. Robinson and industry-wide failure information was analyzed. Failure causes were sorted on the basis of frequency of occurrence and seriousness of consequence, and categorized as common cause failures, human errors, design problems, or component failures.

This information is presented in the body of this document. Section 3.0 provide brief descriptions of these risk-important failure causes, and Section 5.0 presents more extensive discussions, with specific examples and references. The entries in the two sections are cross-referenced.

An abbreviated system walkdown table is presented in Section 3.2 which includes only components identified as risk important. This table lists the system lineup for normal, standby system operation.

This information permits an inspector to concentrate on components important to the prevention of core damage. However, it is important to note that inspections should not focus exclusively on these components. Other components which perform essential functions, but which are not included because of high reliability or redundancy, must also be addressed to ensure that degradation does not increase their failure probabilities, and hence their risk importance.

CONTENTS

SUMMARY

1:0 INTRODUCTION..................................................

2.0 ROBINSON AFW SYSTEM............................................2 2.1 SYSTEM DESCRIPTION........................................

2 2.2 SUCCESS CRITERION.........................................4 2.3 SYSTEM DEPENDENCIES........................................4 2.4 OPERATIONAL CONSTRAINTS....................................4 3.0 INSPECTION GUIDANCE FOR THE ROBINSON AFW SYSTEM.................

5 3.1 RISK IMPORTANT AFW COMPONENTS AND FAILURE MODES............

5 3.1.1 MULTIPLE PUMP FAILURES DUE TO COMMON CAUSE..........

5 3.1.2 TURBINE DRIVEN PUMP FAILS TO START OR RUN.....................................6 3.1.3 MOTOR DRIVEN PUMP A OR B FAILS TO START OR RUN.......................................

7 3.1.4 PUMP UNAVAILABLE DUE TO MAINTENANCE OR SURVEILLANCE..................

7 3.1.5 MOTOR OPERATED CONTROL AND ISOLATION VALVES FAIL CLOSED..................................7 3.1.6 MANUAL SUCTION OR DISCHARGE VALVES FAIL CLOSED........................................8 3.1.7 LEAKAGE OF HOT FEEDWATER THROUGH CHECK VALVES............................................. 9 3.2 RISK IMPORTANT AFW SYSTEM WALKDOWN TABLE...................

9 4.0 GENERIC RISK INSIGHTS FROM PRAs.................................13 4.1 RISK IMPORTANT ACCIDENT SEQUENCES INVOLVING AFW SYSTEM FAILURE........................................13 4.2 RISK IMPORTANT COMPONENT FAILURE MODES.....................

14 V

CONTENTS (continued) 5.0 FAILURE MODES DETERMINED FROM OPERATING EXPERIENCE..............

15 5.1 ROBINSON EXPERIENCE.......................................15 5.1.1 MOTOR DRIVEN PUMP FAILURES..........................

15 5.1.2 TURBINE DRIVEN PUMP FAILURES........................ 15 5.1.3 FLOW CONTROL AND ISOLATION VALVE FAILURES...........

15 5.1.4 CHECK VALVES.......................................16 5.2 INDUSTRY WIDE EXPERIENCE..................................16 5.2.1 COMMON CAUSE FAILURES...............................16 5.2.2 HUMAN ERRORS.......................................

19 5.2.3 DESIGN/ENGINEERING PROBLEMS AND ERRORS..............

19 5.2.4 COMPONENT FAILURES.................................20 REFERENCES........................................................24 vi

1.0 INTRODUCTION

This document is one of a series providing plant-specific inspection guidance for auxiliary feedwater (AFW) systems at pressurized water reactors (PWRs). This guidance is based on information from probabilistic risk assessments (PRAs) for similar PWRs, industry-wide operating experience with AFW systems, plant-specific AFW system descriptions, and plant-specific operating experience. It is not a detailed inspection plan, but rather a compilation of AFW system failure information which has been screened for risk significance in terms of failure frequency and degradation system performance.

The result is a risk-prioritized listing of failure events and the causes that are significant enough to warrant consideration in inspection planning at H.

B. Robinson.

This inspection guidance is presented in Section 3.0, following a description of the H. B. Robinson AFW system in Section 2.0. Section 3.0 identifies the risk important system components by H. B. Robinson identification number, followed by brief descriptions of each of the various failure causes of that component. These include specific human errors, design deficiencies, and hardware failures. The discussions also identify where common cause failures have affected multiple, redundant components. These brief discussions identify specific aspects of system or component design, operation, maintenance, or testing for inspection by observation, records review, training observation, procedures review, or by observation of the implementation of procedures. An AFW system walkdown table identifying risk important components and their lineup for normal, standby system operation is also provided.

The remainder of the document describes and discusses the information used in compiling this inspection guidance. Section 4.0 describes the risk important information which has been derived from PRAs and its sources. As review of that section will show, the failure events identified in PRAs are rather broad (e.g., pump fails to start or run, valve fails closed). Section 5.0 addresses the specific failure causes which have been combined under these broad events.

AFW system operating history was studied to identify the various specific failures which have been aggregated into the PRA failure events. Section 5.1 presents a summary of H. B. Robinson failure information, and Section 5.2 presents a review of industry-wide failure information. The industry-wide information was compiled from a variety of NRC sources, including AEOD analyses and reports, information notices, inspection and enforcement bulletins, and generic letters, and from a variety of INPO reports as well.

Some Licensee Event Reports and NPRDS event descriptions were also reviewed individually. Finally, information was included from reports of NRC-sponsored studies of the effects of plant aging, which include quantitative analyses of reported AFW system failures. This industry-wide information was then combined with the plant-specific failure information to identify the various root causes of the broad failure events used in PRAs, which are identified in Section 3.0.

2.0 H. B. ROBINSON AFW SYSTEM This section presents an overview description of the H. B. Robinson AFW system (Westinghouse three loop plant), including a simplified schematic system diagram. In addition, the system success criterion, system dependencies, and administrative operational constraints are also presented.

2.1 System Description

The AFW system provides feedwater to the steam generators (SG) to allow secondary-side heat removal from the primary system when main feedwater is unavailable. The system is capable of functioning for extended periods, which allows time to restore main feedwater flow or to proceed with an orderly cooldown of the plant to where the residual heat removal (RHR) system can remove decay heat. A simplified schematic diagram of the H. B. Robinson AFW system is shown in Figure 2.1.

The AFW system consists of two motor-driven (MD) pumps and one turbine driven (SD) pump along with the associated piping, valves and instrumentation normally connected to the Condensate Storage Tank (CST). It is designed to start up and establish flow automatically. All pumps start on receipt of a steam generator low-low level signal.

(The motor-driven pumps start on low level in one SG, whereas, two low level signals are required for the turbine driven pump start.)

Also, the motor-driven pumps start on a trip of main feedwater pumps (MFW) pumps, a safety injection signal, or on a blackout signal (total loss of all AC power).

The single turbine-driven (SD) pump also starts on undervoltage on the 4160 V busses 1&4.

Two seperate suction lines off of the CST join to form a common header that supplies water to the turbine-driven pump. A separate line off of the common header supplies water to both motor-driven pumps.

Isolation valves in these lines are locked open. Power, control, and instrumentation associated with each motor-driven pump are independent from one another. Steam for the turbine-driven pump is supplied by steam generators A, B and C, from a point upstream of the main steam isolation valves, through valve MS 154.

Each AFW pump is equipped with a continuous recirculation flow system, which prevents pump deadheading.

The discharges of the motor driven pumps are cross connected, and they feed all three steam generators. The turbine-driven pump also feeds all three steam generators through a connection into the main feedwater regulating valve bypass line, downstream of the bypass control valve. A flow control valve at the discharge of each pump ensures AFW flow will automatically maintain a desired flowrate to each steam generator. Each of the lines from the motor driven pumps contains a motor operated discharge isolation valve, V2-16A,B,C.

Discharge isolation valves in the lines from the turbine-driven pump are motor operated valves, V2-14A,B,C. Flow control valves FCV-1424,1425,6416 are motor operated. Each line contains check valves to prevent leakage from the feedwater lines.

2

To "A Steam Generator FW-6A FW-8B 2-14A M V2-6B3 From Feedwater System FW-6B FW-6C MS-VI-8BW MS-154 EPMS-VI-8C M

To 'B" Steam Generator V2-14C Steam Driven 20 84 AFW Pump Inside CV FCV-6416 to ir 17 CS)t t --

CST From 9A 109 04 Condensate

,44 Storage Tank FCV-142422 DW-21 DW-19 FC-44Chemical Or From Deep 9 654 2

AA Feed Well Pumps V21B V2-20A KA Service Water To '"

68 62

-M 53 Dan System Steam Motor Driven Dri Generator V2-16A V22BAFW Pumps a

70 64 M43 41 V2-160 B

FCV-1425 Chemical S

Feed rarIURE 2.1 HRB ROBINSON AUXILIARY FEEDWATER SYSTEM V.

w~-

-:44~

The condensate storage tank (CST) is the normal source of water for the AFW System and is required to store enough demineralized water at a level sufficient to maintain the reactor coolant system (RCS) at hot standby conditions for 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />. All tank connections except those required for instrumentation, auxiliary feedwater pump suction, chemical analysis, and tank drainage are located above this minimum level.

AFW suction may also be manually switched to the service water system or deep well water as a second and third source of water.

2.2 Success Criterion System success requires the operation of at least one pump supplying rated flow to at least one of the three steam generators.

2.3 System Dependencies The AFW system depends on AC and DC power at various voltage levels for motor operation, valve control, monitor and alarm circuits, and valve/motor control circuits.

Instrument Air is required for governor speed control.

Steam availability is required for the turbine-driven pump.

2.4 Operational Constraints When the reactor coolant is heated above 350 degrees F the H. B. Robinson Technical Specifications require that all three AFW pumps and associated flow paths are operable with each motor-driven pump powered from a different vital bus. If one AFW pump becomes inoperable, it must be restored to operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> or the plant must shut down to hot standby within the next six hours. If two AFW pumps are inoperable, restore at least one inoperable feedwater pump to operable status within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> or the plant must be shut down to hot standby within six hours.

The H. B. Robinson Technical Specifications require a minimum supply of 35,000 gallons of water to be stored in the CST (2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> operation) and an unlimited water supply from the lake via either leg of the plant Service Water System. With the CST inoperable, the service water system may serve-as backup supply for seven days before plant shutdown is required.

4

3.0 INSPECTION GUIDANCE FOR THE H. B. ROBINSON AFW SYSTEM In this section the risk important components of the H. B. Robinson AFW system are identified, and the important failure modes for these components are briefly described. These failure modes include specific human errors, design deficiencies, and types of hardware failures which have been observed to occur for these components, both at H. B. Robinson and at PWRs throughout the nuclear industry. The discussions also identify where common cause failures have affected multiple, redundant components. These brief discussions identify specific aspects of system or component design, operation, maintenance, or testing for inspection activites. These activities include:

observation, records review, training observation, procedures review, or by observation of the implementation of procedures.

Table 3.1 is an abbreviated AFW system walkdown table which identifies risk-important components. This table lists the system lineup for normal (standby) system operation. Inspection of the components identified in the system lineup table addresses essentially all of the risk associated with AFW system operation.

3.1 Risk Important AFW Components and Failure Modes Common cause failures of multiple pumps are the most risk-important failure modes of AFW system components. These are followed in importance by single pump failures, level control valve failures, and individual check valve backleakage failures.

The following sections address each of these failure modes, in decreasing order of risk-importance. They present the important root causes of these component failure modes which have been distilled from historical records.

Each item is keyed to discussions in Section 5.2 where additional information on historical events is presented.

3.1.1 Multiple Pump Failures due to Common Cause The following listing summarizes the most important multiple-pump failure modes identified in Section 5.2.1, Common Cause Failures, and each item is keyed to entries in that section.

Incorrect operator intervention into automatic system functioning, including improper manual starting and securing of pumps, has caused failure of all pumps, including overspeed trip on startup, and inability to restart prematurely secured pumps. CC1.

Inspection Suggestion - Observe Abnormal and Emergency Operating Procedure (AOP/EOP) simulator training exercises to verify that the operators comply with procedures during observed evolutions. Observe surveillance testing on the AFW system to verify it is in strict compliance with the surveillance test procedure.

5

Valve mispositioning has caused failure of all pumps. Pump suction, steam supply, and instrument isolation valves have been involved.

CC2.

Inspection Suggestion - Verify that the system valve alignment, air operated valve control and valve actuating air pressures are correct using 3.1 Walkdown Table, the system operating procedures, and operator rounds logsheet. Review surveillance procedures that alter the standby alignment of the AFW system. Ensure that an adequate return to normal section exists.

Steam binding has caused failure of multiple pumps. This resulted from leakage of hot feedwater past check valves and a motor operated valve into a common discharge header. CC1O. Multiple-pump steam binding has also resulted from improper valve lineups, and from running a pump deadheaded. CC3.

Inspection Suggestion -

Verify that the pump discharge temperature is within the limits specified on the operator rounds logsheet. Assure any instruments used to verify the temperature by the utility are of an appropriate range and included in a calibration program. Verify affected pumps have been vented in accordance with procedures to ensure steam binding has not occurred. Verify that a maintenance work request has been written to repair leaking check valves.

Pump control circuit deficiencies or design modification errors have caused failures of multiple pumps to auto start, spurious pump trips during operation, and failures to restart after pump shutdown. CC4.

Incorrect setpoints and control circuit calibrations have also prevented proper operation of multiple pumps. CC5.

Inspection Suggestion - Review design change implementation documents for the post maintenance testing required prior to returning the equipment to service. Assure the testing verifies that all potentially impacted functions operate correctly, and includes repeating any plant start-up or hot functional testing that may be affected by the design change.

Loss of a vital power bus has failed both the turbine-driven and one motor-driven pump due to loss of control power to steam admission valves or to turbine controls, and to motor controls powered from the same bus. CC6.

6

Inspection Suggestion - The material condition of the electrical equipment is an indicator of probable reliability. Review the Preventative Maintenance (PM) records to assure the equipment is maintained on an appropriate frequency for the environment it is in and that the PM's are actually being performed as required by the program. Review the outstanding Corrective Maintenance records to assure the deficiencies found on the equipment are promptly corrected.

Simultaneous startup of multiple pumps has caused oscillations of pump suction pressure causing multiple-pump trips on low suction pressure, despite the existence of adequate static net positive suction head (NPSH). CC7. At H.B. Robinson, design reviews have identified inadequately sized suction piping which could have yielded insufficient NPSH to support operation of more than one pump. CC8.

Inspection Suggestion - Assure that plant conditions which could result in the blockage or degradation of the suction flow path are addressed by system maintenance and test procedures. Examples include, if the AFW system has an emergency source from a water system with the potential for bio-fouling, then the system should be periodically treated to prevent buildup and routinely tested to assure an adequate flow can be achieved to support operation of all pumps, or inspected to assure that bio-fouling is not occurring. Design changes that affect the suction flow path should repeat testing that verified an adequate suction source for simultaneous operation of all pumps.

Verify that testing has, at sometime, demonstrated simultaneous operation of all pumps.

Verify that surveillances adequately test all aspects of the system design functions, for example, demonstrate that the AFW pumps will trip on low suction pressure.

3.1.2 Turbine Driven Pump Fails to Start or Run Improperly adjusted and inadequately maintained turbine governors have caused pump failures. HE2.

Problems include worn or loosened nuts, set screws, linkages or cable connections, oil leaks and/or contamination, and electrical failures of resistors, transistors, diodes and circuit cards, and erroneous grounds and connections.

CF5. Governor problems have caused failure of the turbine driven pump at H. B. Robinson.

Inspection Suggestion - Review PM records to assure the governor oil is being replaced within the designated frequency.

7

During plant walkdowns carefully inspect the governor and linkages for loose fasteners, leaks, and unsecured or degraded conduit. Review vendor manuals to ensure PM procedures are performed according to manufacturer's recommendations and good maintenance practices.

Turbines with Woodward Model PG-PL governors have tripped on overspeed when restarted shortly after shutdown, unless an operator has locally exercised the speed setting knob to drain oil from the governor speed setting cylinder (per procedure). Automatic oil dump valves are now available through Terry. DE4.

Inspection Suggestion - Observe the operation of the turbine driven Aux Feed pump and assure that the governor is reset as directed in accordance with procedure. Assure the turbine is not coasting over, which can result in refill of the speed setting cylinder.

Condensate slugs in steam lines have caused turbine overspeed trip on startup. Tests repeated right after such a trip may fail to indicate the problem due to warming and clearing of the steam lines.

Surveillance should exercise all steam supply connections. DE2.

Inspection Suggestion - Verify that the steam traps are valved in on the steam supply line. For steam traps that are on a pressurized portion of the steam line, check the steam trap temperature (if unlagged) to assure it is warmer than ambient (otherwise it may be stuck or have a plugged line). If the steam trap discharge is visible, assure there is evidence of liquid discharge.

Trip and throttle valve (TTV) problems which have failed the turbine driven pump include physically bumping it, failure to reset it following testing, and failures to verify control room indication of reset. HE2. Whether either the overspeed trip or TTV trip can be reset without resetting the other, indication in the control room of TTV position, and unambiguous local indication of an overspeed trip affect the likelihood of these errors. DE3.

Inspection Suggestion - Carefully inspect the TTV overspeed trip linkage and assure it is reset and in good physical condition. Assure that there is a good steam isolation to the turbine, otherwise continued turbine high temperature can result in degradation of the oil in the turbine, interfering with proper overspeed trip operation.

Review training procedures to ensure operator training on resetting the TTV is current.

8

Low lubrication oil pressure resulting from heatup due to previous operation has prevented pump restart due to failure to satisfy the protective interlock. DE5.

Inspection Suggestion - Low oil pressure is a trip that is in service at all times for the turbine driven AFW pump.

Normally the low oil pressure occurs at approximately 1400 rpm and serves to protect the pump from low RPM operation, however low oil pressure due to a plugged filter will also cause a trip. Review PM records to assure the filter is replaced on the designated frequency.

3.1.3 Motor Driven Pump A or B Fails to Start or Run

• Control circuits used for automatic and manual pump starting are an important cause of motor driven pump failures, as are circuit breaker failures. CF7. Control circuit problems have occurred at H. B. Robinson.

Inspection Suggestion - Review corrective maintenance records when control circuit problems occur to determine if a trend exists.

Every time a breaker is racked in a PMT should be performed to start the pump, assuring no control circuit problems have occurred as a result of the manipulation of the breaker. (Control circuit stabs have to make up upon racking the breaker, as well as cell switch damage can occur upon removal and reinstallation of the breaker.)

• Mispositioning of handswitches and procedural deficiencies have prevented automatic pump start. HE3.

Inspection Suggestion - Confirm switch position using Table 3.1.

Review administrative procedures concerning documentation of procedural deficiencies. Ensure operator training on procedural changes is current.

3.1.4 Pump Unavailable Due to Maintenance or Surveillance Both scheduled and unscheduled maintenance remove pumps from operability. Surveillance requires operation with an altered line up, although a pump train may not be declared inoperable during testing. Prompt scheduling and performance of maintenance and surveillance minimize this unavailability.

Inspection Suggestion - Review the time the AFW system and components are inoperable. Assure all maintenance is being performed that can be performed during a single outage time frame, avoiding multiple equipment outages. The maintenance should be scheduled 9

before the routine surveillance test, so credit can be taken for both post maintenance testing and surveillance testing, avoiding excessive testing. Review surveillance schedule for frequency and adequacy to verify system operability requirements per Technical Specifications.

3.1.5 Motor Operated Isolation and Control Valves Failure SD Pump Train:

FCV-6416, V2-14A,BC MD Pump Trains A;B: FCV-1424;1425, V2-16ABC MD Pump Cross-Connect Valves: V2-20 AB These MOVs isolate and control flow to the steam generators. They fail as-is on loss of power.

• Common cause failure of MOVs has resulted from failure to use electrical signature tracing equipment to determine proper settings of torque switch and torque switch bypass switches.

Failure to calibrate switch settings for high torques necessary under design basis accident conditions has also been involved. CC11. Improper torque setting, dirty contacts and seat leakage have been the main causes of valve failure at H. B. Robinson.

Inspection Suggestion - Review the MOV test records to assure the testing and settings are based on dynamic system conditions. Overtorquing of the valve operator can result in valve damage such as cracking of the seat or disc. Review the program to assure overtorquing is identified and corrective actions are taken to assure valve operability following an overtorque condition. Review the program to assure EQ seals are renewed as required during the restoration from testing to maintain the EQ rating of the MOV.

• Valve motors have been failed due to lack of, or improper sizing or use of thermal overload protective devices. Bypassing and oversizing should be based on proper engineering for design basis conditions. CF4.

Inspection Suggestion - Review the administrative controls for documenting and changing the settings of thermal overload protective devices. Assure the information is available to the maintenance planners.

• At H. B. Robinson, heating of motor-operated valves by check valve leakage has caused thermal binding and failure of AFW discharge valves to open on demand. CF2.

10

Inspection Suggestion - Covered by 3.1.1 bullet 3.

• Grease trapped in the torque switch spring pack of Limitorque SMB motor operators has caused motor burnout or thermal overload trip by preventing torque switch actuation. CF8.

Inspection Suggestion - Review this only if the MOV testing program reveals deficiencies in this area.

• Manually reversing the direction of motion of operating MOVs has overloaded the motor circuit. -Operating procedures should provide cautions, and circuit designs may prevent reversal before each stroke is finished. DE7.

Inspection Suggestion - Review operator training on MOV operation to ensure this topic is adequately addressed. Review operating procedures to ensure adequate precautions are identified.

• Space heaters designed for preoperation storage have been found wired in parallel with valve motors which had not been environmentally qualified with them present. DE8.

Inspection Suggestion - Spot check MOV's during MOV testing to assure the space heaters are physically removed or disconnected.

3.1.6 Manual Suction or Discharge Valves Fail Closed CST Suction Valves: AFW-1,104 TD Pump Train: AFW-4; AFW-20,17; FW-6A,B.C MD PumD Train A: AFW-22,28; AFW-44,53,54; AFW-62,63 MD PumD Train B: AFW-29; AFW-55,64 Backup Suction Sources:

SW-24,118; DW-19,21 These manual valves are normally locked open except for the backup suction sources from the deep well and service water systems, which are normally closed.

• Valve mispositioning has resulted in failures of multiple trains of AFW. CC2. It has also been the dominant cause of problems identified during operational readiness inspections. HE1.

Events have occurred most often during maintenance, calibration, or system modifications. Important causes of mispositioning include:

• Failure to provide complete, clear, and specific procedures for tasks and system restoration

• Failure to promptly revise and validate procedures, training, and diagrams following system modifications

• Failure to complete all steps in a procedure

• Failure to adequately review uncompleted procedural steps after task completion

• Failure to verify support functions after restoration 11

• Failure to adhere scrupulously to administrative procedures regarding tagging, control and tracking of valve operations

• Failure to log the manipulation of sealed valves

• Failure to follow good practices of written task assignment and feedback of task completion information

• Failure to provide easily read system drawings, legible valve labels corresponding to drawings and procedures, and labeled indications of local valve position Inspection Suggestion - Review the administrative controls that relate to valve positioning and sealing, system restoration following maintenance, valve labeling, system drawing updating, and procedure revision, for proper implementation.

3.1.7 Leakage of Hot Feedwater through Check Valves:

MD Pump A: AFW-40,68,69 MD Pump B: AFW-41,70 SD Pump: AFW-84,FW-8ABC

• Leakage of hot feedwater through several check valves in series has caused steam binding of multiple pumps.

Leakage through a closed level control valve in series with check valves has also occurred at H. B. Robinson, as would be required for leakage to reach the motor driven pumps A and B. CC10.

Inspection Suggestion - Covered by 3.1.1 bullet 3.

• Slow leakage past the final check valve of a series may not force the check valve closed. Other check valves in series may leak similarly. Piping orientation and valve design are important factors in achieving true series protection. CF1.

Inspection Suggestion - Covered by 3.1.1 bullet 3.

3.2 Risk Important AFW System Walkdown Table Table 3.1 presents an AFW system walkdown table including only components identified as risk important. This information allows inspectors to concentrate their efforts on components important to prevention of core damage. However, it is essential to note that inspections should not focus exclusively on these components. Other components which perform essential functions, but which are absent from this table because of high reliability or redundancy, must also be addressed to ensure that their risk importances are not increased. Examples include the (locked open) steam lead isolation valves upstream of the main steam isolation valves, and an adequate water level in the CST.

12

TABLE 3.1.

Risk Important Walkdown Table for H. B. Robinson AFW System Components Required Actual Component #

Component Name Position Position Electrical A

Motor-Driven Pump Racked In/

Closed B

Motor-Driven Pump Racked In/

Closed Valve AFW-1 CST Outlet Valve Locked Open AFW-104 CST Outlet Valve Locked Open V2-16A AF Header Discharge to S/G "A" Closed V2-20A MDP Cross Connect Valve Closed V2-20B MDP Cross Connect Valve Closed V2-16B AF Header Discharge to S/G "B" Closed V2-16C AF Header Discharge to S/G "C" Closed V2-14A SD FWP Discharge to S/G "A" Closed V2-14B SD FWP Discharge to S/G "B" Closed V2-14C SD FWP Discharge to S/G "C" Closed AFW-17 SDP Recirculation Isolation Open 13

Risk Important Walkdown Table for..

H. B. Robinson AFW System Components (continued)

MS-154 SDP Main Stream Supply Valve Locked Open AFW-4 SDP Suction Valve Locked Open AFW-22 MDP "A" & "B" Suction Locked Open AFW-28 MOP "A" Suction Isolation Locked Open AFW-29 MDP "B" Suction Isolation Locked Open AFW-42 MOP "A" Recirculation Open AFW-43 MDP "B" Recirculation Open AFW-44 AFW Recirculation Valve Open AFW-53 V2-16A Inlet Isolation Locked Open AFW-54 V2-16B Inlet Isolation Locked Open AFW-55 V2-16C Inlet Isolation Locked Open AFW-20 SD AFP Discharge Isolation Locked Open FCV-1424 MDP "A" Flow Control Valve Operable/Closed FCV-1425 MDP "B" Flow Control Valve Operable/Closed FCV-6416 SDP Flow Control Valve Operable/Closed MS-VI-8A SOP Steam Shutoff Valve Closed MS-VI-88 SDP Steam Shutoff Valve Closed MS-VI-8C SOP Steam Shutoff Valve Closed TDP Throttle-Trip Valve Open AFW-24 Service Water Supply Valve Locked Closed SW-118 Service Water Supply Valve Locked Closed DW-19 Deepwell Water Supply Valve Locked Closed 14

TABLE 3.1.

Risk Important Walkdown Table for H. B. Robinson AFW System Components (Continued)

DW-21 Deepwell Water Supply Valve Locked Closed V2-14A Piping Upstream of Check Valve Cool V2-14B Piping Upstream of Check Valve Cool V2-14C Piping Upstream of Check Valve Cool V2-16A Piping Upstream of Check Valve Cool V2-16B Piping Upstream of Check Valve Cool V2-16C Piping Upstream of Check Valve Cool 15

4.0 GENERIC RISK INSIGHTS FROM PRAs

-s a

PRAs for 13 PWRs were analyzed to identify risk-important accident sequences involving loss of AFW, and to identify and risk-prioritize the component failure modes involved. The results of this analysis are described in this section. They are consistent with results reported by INEL and BNL (Gregg et al 1988, and Travis et al, 1988).

4.1 Risk Important Accident Sequences Involving AFW System Failure Loss of Power System A loss of offsite power is followed by failure of AFW. Due to lack of actuating power, the power operated relief valves (PORVs) cannot be opened preventing adequate feed-and-bleed cooling, and resulting in core damage.

A station blackout fails all AC power except Vital AC from DC invertors, and all decay heat removal systems except the turbine driven AFW pump. AFW subsequently fails due to battery depletion or hardware failures, resulting in core damage.

A DC bus fails, causing a trip and failure of the power conversion system. One AFW motor-driven pump is failed by the bus loss, and the turbine-driven pump fails due to loss of turbine or valve control power. AFW is subsequently lost completely due to other failures. Feed-and-bleed cooling fails because PORV control is lost, resulting in core damage.

Transient-Caused Reactor or Turbine Trip A transient-caused trip is followed by a loss of the power conversion system (PCS) and AFW. Feed-and-bleed cooling fails either due to failure of the operator to initiate it, or due to hardware failures, resulting in core damage.

Loss of Main Feedwater A feedwater line break drains the common water source for MFW and AFW. The operators fail to provide feedwater from other sources, and fail to initiate feed-and-bleed cooling, resulting in core damage.

A loss of main feedwater trips the plant, and AFW fails due to operator error and hardware failures. The operators fail to initiate feed-and-bleed cooling, resulting in core damage.

Steam Generator Tube Rupture (SGTR)

A SGTR is followed by failure of AFW. Coolant is lost from the primary until the refueling water storage tank (RWST) is depleted.

16

High pressure injection (HPI) fails since recirculation cannot be established from the empty sump, and core damage results.

4.2 Risk Important Component Failure Modes The generic component failure modes identified from PRA analyses as important to AFW system failure are listed below in decreasing order of risk importance.

1. Turbine-Driven Pump Failure to Start or Run.

2. Motor-Driven Pump Failure to Start or Run.

3. TDP or MDP Unavailable due to Test or Maintenance.

4. AFW System Valve Failures steam admission valves trip and throttle valve flow control valves pump discharge valves pump suction valves valves in testing or maintenance.

5. Supply/Suction Sources condensate storage tank stop valve hot well inventory suction valves.

In addition to individual hardware, circuit, or instrument failures, each of these failure modes may result from common causes and human errors.

Common cause failures of AFW pumps are particularly risk important.

Valve failures are somewhat less important due to the multiplicity of steam generators and connection paths. Human errors of greatest risk importance involve: failures to initiate or control system operation when required; failure to restore proper system lineup after maintenance or testing; and failure to switch to alternate sources when required.

17

5.0 FAILURE MODES DETERMINED FROM OPERATING EXPERIENCE This section describes the primary root causes of AFW system component failures, as determined from a review of operating histories at H. B. Robinson and at other PWRs throughout the nuclear industry. Section 5.1 describes experience at H. B. Robinson from 1980 to 1991. Section 5.2 summarizes information compiled from a variety of NRC sources, including AEOD analyses and reports, information notices, inspection and enforcement bulletins, and generic letters, and from a variety of INPO reports as well.

Some Licensee Event Reports and NPRDS event descriptions were also reviewed individually.

Finally, information was included from reports of NRC-sponsored studies of the effects of plant aging, which include quantitative analysis of AFW system failure reports. This information was used to identify the various root causes expected for the broad PRA-based failure events identified in Section 4.0, resulting in the inspection guidelines presented in Section 3.0.

5.1 H. B. Robinson Experience The AFW system at H. B. Robinson has experienced failures of the AFW pumps, pump discharge flow control valves, the turbine steam admission and supply valves, turbine trip and throttle valve, pump discharge isolation valves, service water backup supply valves, and numerous system check valves.

Failure modes include electrical, instrumentation and control, hardware failures, and human errors.

5.1.1 Motor Driven Pump Failures There have been nine events which involved failure of the motor driven pumps during several modes of operation. Failure modes involved instrumentation and control circuit failures, pump hardware failures, and human failures during maintenance activities. Three cases of arcing within the motor required motor replacement.

5.1.2 Turbine Driven Pump Failures Two events have resulted in decreased operational readiness of the turbine driven pump. Failure causes were attributed to steam binding of the turbine-driven pump resulting from motor-operated isolation valve leakage and governor trip valve problems.

5.1.3 Flow Control and Isolation Valve Failures More than fifteen events have resulted in impaired operational readiness of the motor operated flow control and isolation valves.

Principal failure causes were equipment wear, instrumentation and control circuit failures, valve hardware failures, and human errors. Valves have failed to operate properly due to failure of control components, broken or dirty contacts, misaligned or broken limit switches. Human errors have resulted in improper wiring.

18

6.0 REFERENCES

Beckjord, E. S. June 30, 1989. Closeout of Generic Issue II.E.6.1, "In Situ Testing of Valves".

Letter to V. Stello, Jr.,

U.S. Nuclear Regulatory Commission, Washington, DC.

Brooks, B. P. 1988. Application Guidelines for Check Valves in Nuclear Power Plants. NP-5479, Electric Power Research Institute, Palo Alto, CA.

Casada, D. A. 1989. Auxiliary Feedwater System Aging Study. Volume 1.

Operating Experience and Current Monitoring Practices. NUREG/CR-5404. U.S.

Nuclear Regulatory Commission, Washington, DC.

Gregg, R. E. and R. E. Wright. 1988. Appendix Review for Dominant Generic Contributors. BLB-31-88.

Idaho National Engineering Laboratory, Idaho Falls, Idaho.

Miraglia, F. J. February 17, 1988. Resolution of Generic Safety Issue 93.

"Steam Binding of Auxiliary Feedwater Pumps" (Generic Letter 88-03).

U.S.

Nuclear Regulatory Commission, Washington, DC.

Miraglia, F. J. August 8, 1988. Instrument Air Supply System Problems Affecting Safety-Related Equipment (Generic Letter 88-14).

U.S. Nuclear Regulatory Commission, Washington, DC.

Partlow, J. G. June 28, 1989. Safety-Related Motor-Operated Valve Testing and Surveillance (Generic Letter 89-10).

U.S. Nuclear Regulatory Commission, Washington, DC.

Rothberg, 0. June 1988. Thermal Overload Protection for Electric Motors on Safety-Related Motor-Operated Valves - Generic Issue II.E.6.1. NUREG-1296.

U.S. Nuclear Regulatory Commission, Washington, DC.

Travis, R. and J. Taylor. 1989. Development of Guidance for Generic, Functionally Oriented PRA-Based Team Inspections for BWR Plants-Identification of Risk-Important Systems, Components and Human Actions. TLR-A-3874-TGA Brookhaven National Laboratory, Upton, New York.

AEOD Reports AEOD/C404. W. D. Lanning. July 1984. Steam Binding of Auxiliary Feedwater Pumps. U.S. Nuclear Regulatory Commission, Washington, DC.

AEOD/C602. C. Hsu. August 1986. Operational Experience Involving Turbine Overspeed Trips. U.S. Nuclear Regulatory Commission, Washington, DC.

AEOD/C603. E. J. Brown. December 1986. A Review of Motor-Operated Valve Performance. U.S. Nuclear Regulatory Commission, Washington, DC.

27

AEOD/E702. E. J. Brown. March 19, 1987. MOV Failure Due to Hydraulic Lockup From Excessive Grease in Spring Pack. U.S. Nuclear Regulatory Commission, Washington, DC.

AEOD/T416. January 22, 1983.

Loss of ESF Auxiliary Feedwater Pump Capability at Trojan on January 22, 1983.

U.S. Nuclear Regulatory Commission, Washington, DC.

Information Notices IN 82-01. January 22, 1982. Auxiliary Feedwater Pump Lockout Resulting from Westinghouse W-2 Switch Circuit Modification. U.S. Nuclear Regulatory Commission, Washington, DC.

IN 84-32. E. L. Jordan. April 18, 1984. Auxiliary Feedwater Sparger and Pipe Hangar Damage. U.S. Nuclear Regulatory Commission, Washington, DC.

IN 84-66. August 17, 1984. Undetected Unavailability of the Turbine-Driven Auxiliary Feedwater Train. U.S. Nuclear Regulatory Commission, Washington, DC.

IN 87-34. C. E. Rossi.

July 24, 1987. Single Failures in Auxiliary Feedwater Systems. U.S. Nuclear Regulatory Commission, Washington, DC.

IN 87-53. C. E. Rossi.

October 20, 1987. Auxiliary Feedwater Pump Trips Resulting from Low Suction Pressure. U.S. Nuclear Regulatory Commission, Washington, DC.

IN 88-09. C. E. Rossi.

March 18, 1988. Reduced Reliability of Steam-Driven Auxiliary Feedwater Pumps Caused by Instability of Woodward PG-PL Type Governors. U.S.

Nuclear Regulatory Commission, Washington, DC.

IN 89-30. R. A. Azua. August 16, 1989. Robinson Unit 2 Inadequate NPSH of Auxiliary Feedwater Pumps. Also, Event Notification 16375, August 22, 1989.

U.S. Nuclear Regulatory Commission, Washington, DC.

Inspection Report IR 50-489/89-11; 50-499/89-11. May 26, 1989. South Texas Project Inspection Report. U.S. Nuclear Regulatory Commission, Washington, DC.

NUREG Report NUREG-1154. 1985. Loss of Main and Auxiliary Feedwater Event at the Davis Besse Plant on June 9, 1985. U.S. Nuclear Regulatory Commission, Washington, 28