Text

Possible Common Mode Failure of Aux Feed Recirculation Lines, RCEOOO191, Report Date: Draft 1
	ML030800553
Person / Time
Site:	Point Beach
Issue date:	11/19/2002
From:	Bennett K, Bosacki W, Flessner R, Freels J, Hettick D, Peterson L, Schmidt E; Nuclear Management Co
To:	; Office of Nuclear Reactor Regulation
References
	CAP029952, FOIA/PA-2003-0094 RCE000191, Draft 1
	Download: ML030800553 (47)
	v • d • e

NMC--'

Co*mi"tted to Nuclear Point Beach Nuclear Plant Possible Common Mode Failure of Aux Feed Recirculation Lines RCE000191 (CAP029952)

Event Date: October 29, 2002 Report Date: Draft 1 Principal Investigators:

Richard Flessner - Team Lead Kevin Bennett William Bosacki Eric Schmidt Approved By:

Group Lead - Larry Peterson Date Issue Manager - James Freels Date CAP Manager - Dennis Hettick Date N---3Lý

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines Table of Contents I. Executive Summary ............................................................................. 3 II. Event Narrative ............................................................................... 4 III. Extent of Condition Assessment .................................................... 12 IV. Nuclear Safety Significance .......................................................... 12 V. Report to External Agencies ......................................................... 12 VI. Data Analysis ................................................................................. 13 Information & Fact Sources ................................................................. 13 Evaluation Methodology & Analysis Techniques ................................ 35 Data Analysis Summary ......................................................................... 35 Failure Mode Identification ................................................................. 35 VII. Root Causes & Contributing Factors ............................................. 35 VIII. Corrective Actions ...................................................................... 36 IX. References ...................................................................................... 38 X. Attachments .................................................................................... 41 Attachment A - Event Timeline .......................................................... 42 Attachment B - Barrier Analysis .......................................................... 45 Attachment C - Charter ........................................................................ 47 2

Possible Conznon Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines I. Executive Summary

Purpose:

The purpose of this investigation is to determine the root and contributing causes of why a potential existed for a common mode failure where all AFW pump recirculation lines could have restricted flow rates (due to orifice plugging) resulting in eventual pump failure, and why these inadequacies where not identified previously.

Event Synopsis:

[Briefdescifption of event]

==

Conclusions:==

[Brifdiscussion offindings]

Nuclear Safety Significance:

(XEplainnuclearsafely significanceof event)

Root Cause:

[Explainroot cause & significantcontributors. Use bold & bullets]

Corrective Action Synopsis:

[Briefy describe major comctive actions]

3

Possible Common Mode Failure of RCEO00191 Aux FeedwaterRecirculationLines I1. Event Narrative

Background

On May 5, 1988 NRC issued Bulletin 88-04, Potential Safety-Related Pump Loss. This bulletin requested licensees to investigate and correct as appropriate two mini-flow design concerns. The first concern was the potential for deadheading one or more pumps that have a common mini-flow line. The second concern is whether or not the installed mini-flow capacity is adequate to prevent damage to safety related pumps. In a response dated June 28, 1988, we acknowledged that each of the pumps in the Auxiliary Feedwater (AFW) System have their own recirculation lines with an AOV isolation valve and an orifice upstream of the common return line to the condensate storage tank (CST). We discussed the logic of the recirculation valves to open or shut dependent on AFW system forward flow. We also acknowledged that the flow orifices for the pumps needed to be replaced with higher flow orifices to ensure sufficient flow for indefinite pump cooling via the recirculation lines.

On July 7, 1988 modifications MR 88-099*A-C were initiated to increase the recirculation line flows to prevent pump degradation due to hydraulic instability. The minimum pump flow prior to this MR was 30 gpm. The MR increased this to minimum flow to 70 gpm for the MDAFPs and 100 gpm for the TDAFPs. The replacement orifices were installed during 1991.

During 1996 and 1997 several condition reports were written regarding high noise levels associated with operation of the AFW pumps while on mini-recirc. EWR 99-031 was initiated on 12/8/98 to determine if the orifices were incurring damage from cavitation and needed replacement. During 1998 and 1999 weld cracks were discovered in the recirculation piping associated with the MDATW pumps. On July 9, 1999 the Engineering Advisory Committee (EAC) reviewed three options proposed to address EWR 99-031. The options were to install a multi-stage restricting orifice (RO) of the same type as currently installed but with 10 stages instead of 2 stages used in the present design, to install a new design involving a pressure-reducing flow element in a valve body, or to install a multi-stage pressure-reducing trim in the recirculation line AOVs.

The EAC did not select an option, but did recommend implementing modifications for the MDAFW pumps during 2000 and evaluate performance to determine if TDAFW pumps should be modified during 2001. Modifications MR 99-029*A-D were initiated during 1999 to replace the AFW recirculation line restricting orifices. RCE 99-081 evaluated the socket weld failures in the AFW recirculation lines and concluded that vibrations induced by the RO cavitation caused the cracking. The RO modification scope was expanded to address weld and pipe replacement and installation of oversized sockets.

The option to install a new design involving a pressure-reducing flow element in a valve body was chosen and the new orifices were installed in the recirculation lines for the MDAFW pumps during November 2000. Some difficulty was encountered in achieving the desired flow rates through the orifices and it was necessary to drill additional holes in the trim. The design was changed for the TDAFW pump orifices to include a movable 4

Possible Common Mode Failureof RCEO00191 Aux FeedivaterRecirculationLines plug that could be adjusted to vary the flow. The Unit 2 TDAFW pump orifice was replaced in April 2002 and the Unit I TDAFW pump orifice was replaced in September 2002.

Event On 10/23/2002 at 0400, Operations removed MDAFW pump P-38A from service for scheduled maintenance. During the day shift, Maintenance personnel conducted scheduled corrective maintenance on P-38A. Work orders 9945610, 0205651 and 9949098 were performed. These WOs consisted of changing the operating diaphragm on Aux Feed discharge control valve, AF-4012, repacking the inboard and outboard stuffing boxes (pump seals) on P-38A, and replacing service water drain valve AF-38.

While repacking the pump, Maintenance found only six packing rings on both the inboard and outboard sides of the pump instead of the expected seven rings.

Maintenance put 7 rings in each seal and documented these findings on CAP029923.

The physical work with replacing AF-38 involved cutting the socket weld, manually removing the valve, inspecting the piping visually, prepping the area for reinstallation, and then installing the new valve with a socket weld.

The work on AF-4012 involved replacing the valve's operating diaphragm. Following completion of the physical work, I&C performed a drop test on the AOV diaphragm.

The drop test failed and day shift turned this work over to night shift for completion.

On night shift, Maintenance changed the stem diaphragm gasket on the AOV early in the shift. Re-performance of the drop test was successful, testing on the instrumentation was completed by I&C, and the pump was turned over to Operations for post-maintenance testing.

Operations filled and vented the pump suction line, casing, and discharge line per 01 62A, Motor-Driven Auxiliary Feedwater System (P-38A & P-38B). The venting procedure requires the suction line to be vented first until a solid stream of water is observed, followed by the casing vent and finally the discharge vent.

On 10/24/2002 at 0159 hours0.00184 days 0.0442 hours 2.628968e-4 weeks 6.04995e-5 months , Operations started P-38A for post maintenance testing per IT 10, Test of Electrically Driven Auxiliary Feed Pumps and Valves (Quarterly). When the pump started, a Maintenance supervisor observed that the suction relief valve (AF 4028) momentarily lifted, reseated, but continued to drip water. The relief valve has WO 9948850 associated with it. The relief valve was also observed to not be sucking air.

This was determined by the Maintenance supervisor holding his hand over the discharge tailpiece to feel for suction.

Maintenance personnel proceeded to adjust the pump packing as expected. It was observed that the recirc flow was 64.5 gpm. The shift manager was informed of the matter. The Shift Manager directed the crew to first vent the flow indicator. This resulted in no improvement in flow. He then had the crew monitor the pump casing for 5

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines overheating. The pump was reported to be cool to the touch. The shift manager then directed the crew to secure the pump. The pump was secured at 0212 hours0.00245 days 0.0589 hours 3.505291e-4 weeks 8.0666e-5 months . The crew observed a normal coast down of the pump. The Shift Manager stated he felt assured that the pump was not degraded or damaged from low recirc flow based upon the short run time (13 minutes) and the above listed parameters. IT 10 requires 70 gpm recirc flow.

Minimum flow to prevent damage to the motor-driven auxiliary feedwater pumps is 50 gpm.

After the pump was secured, the Shift Manager discovered that I&C had calibrated the flow indicator the previous day. He requested that I&C perform a calibration check to verify proper operation. This was performed with different MTE than the original calibration. The calibration check was satisfactory (-0440).

The Shift Manager also requested that I&C review the instrument venting procedure with the auxiliary operator who performed the earlier vent. The auxiliary operator and I&C technician concluded that the transmitter was vented correctly. The crew also re-vented the pump suction piping, casing, and discharge piping. During each of the vents, solid streams of water were observed with no air. An auxiliary operator verified the valve lineup satisfactory.

CAP029908 was initiated by Operations at 0323 to document the inadequate flow obtained during performance of IT 10.

The AFW System Engineer was notified at 0400 on 10/24 ht which time an investigation was commenced. Between 0500 and 0700 hours0.0081 days 0.194 hours 0.00116 weeks 2.6635e-4 months , the potential causes were defined in order of likelihood:

1. Restriction in the piping.

a. Recirculation orifice AF-4008

b. Check valve degradation: AF-1 12 or AF-1 15

c. Manual valve problem: AF-39 or AF-27

d. Recirc AOV stem-to-disc separation: AF-4007

e. FE-4050A degradation

2. Instrumentation: FIT--4050A.

3. Degradation of the pump rotating assembly.

4. Valve leakage to the discharge path.

5. Pump P-38A was not adequately vented.

At about 0603, the BOP/NSSS Mechanical Systems Engineering Supervisor was established as the overall lead for Engineering for this issue.

At 0830 hours0.00961 days 0.231 hours 0.00137 weeks 3.15815e-4 months on 10/24/02, a meeting was held to discuss the issue. The BOP/NSSS Mechanical Systems Engineering Supervisor led the meeting, with representatives from Engineering, Maintenance, and Operations in attendance. It was decided to perform the following actions:

6

Possible Common Mode Failure of RCEO00191 Aux FeedwaterRecirculation Lines

1. I&C - Re-vent the transmitter once again (Engineering to observe) and verify the position of the equalizing valve.

2. Operations - Vent all lines and the pump casing with Engineering observing.

3. Engineering - Use a UT flow transmitter to verify installed indication.

4. Engineering to take vibration data.

5. Discharge pressure will be monitored.

6. A work plan to open RO-4008 will be prepared in parallel to be worked if adequate flow cannot be achieved.

The venting was performed by Operations in accordance with 01 62A, Section 7.1. No air was noted during the vent; there was a solid stream of water from all three vents. The UT flow device was attached and the flow transmitter was vented. A pre-job brief was held by Operations with the control operator, auxiliary operator, and supervision. The brief included the direction that if recirc flow was not >70 gpm, then the pump would be secured immediately.

At approximately 1100 hours0.0127 days 0.306 hours 0.00182 weeks 4.1855e-4 months , Operations started P-38A. Equipment and components responded as expected. Indicated flow on the recirc transmitter went to about 64 gpm.

The UT device flow stabilized at 60 gpm. The on-scene SRO observed these indications and directed that the pump be secured after running for less than one minute. During the run, Engineering took vibration data, which indicated, normal. P-38A was then danger tagged and drained to allow removal and inspection of the flow orifice.

Maintenance removed the orifice. The Mechanic noted several of the holes on the outside sleeve were plugged. There was no evidence of debris in the pipe (a boroscope was used to verify this). The orifice was taken to the maintenance shop where Engineering mapped the plugged holes and took photographs of the orifice. Rust blockage was noted in 24 of the 54 outer holes. Maintenance milled off the spot welds on the retaining pins to facilitate removal of the sleeves. The pins were driven out with a punch. The nested sleeves had to be tapped out using a block and hammer. The mechanic-electrician who disassembled the orifice stated that all of the sleeves except the outermost sleeve were "spotless" with no evidence of debris. Material did fall out of and off the orifice during disassembly. Four small particles were collected for further analysis.

Following cleaning and inspecting, the nested sleeves were reassembled and reinstalled using a new bonnet gasket (spiral wound) and the old seat gasket (spiral wound but already compressed because a new gasket was not available). Installation was completed by 1800 hours0.0208 days 0.5 hours 0.00298 weeks 6.849e-4 months .

During night shift on 10/24 Operations cleared tags, and filled and vented the system per 01 62A. Maintenance, Operations and Engineering met to discuss the next test run. The Maintenance supervisor requested that the high points be re-vented prior to the run.

Operations re-vented the suction (solid stream) and discharge (slight amount of air) piping. Operations then started P-38A pump per 01 62A at 2115 hours0.0245 days 0.588 hours 0.0035 weeks 8.047575e-4 months . Recirc flow was 7

Possible Comnnion Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines observed to be within specification at 75 gpm. Following satisfactory observance of key parameters, the pump was secured.

Earlier in the shift, at approximately 1700 to 1800 hours0.0208 days 0.5 hours 0.00298 weeks 6.849e-4 months , the Shift Manager directed Chemistry to sample all four AFW pump suctions and both condensate storage tanks.

The purpose of these samples was to determine the water quality as it related to possible plugging of the recirc orifices. The communication received by the chemistry technician was to obtain and analyze samples from Unit 1 and Unit 2 auxiliary feedwater and from the Unit I and Unit 2 condensate storage tanks.

The Chemistry Supervisor stated that the Unit 1 and Unit 2 auxiliary feedwater suction samples were taken at 1C26 and 2C26, feedwater sample control panels. These were aligned to the 1&2 P29 turbine-driven auxiliary feedwater pump suctions, and that is where the samples were drawn from. The chemistry technician also took samples from both condensate storage tanks (CSTs) at their normal sample points on El. 26'. The CST sample points tap into the CST are, 6" to 1' above the elevation that auxiliary feed suction piping exits the CSTs. The samples were analyzed for total suspended solids (TSS), chlorides, fluorides, and sulfates. The Chemistry Supervisor stated that the results showed nothing abnormal.

The Shift Manager then made preparations to perform IT 10, Test of Electrically Driven Auxiliary Feed Pumps and Valves (Quarterly), for both the post-maintenance return to service test and the scheduled quarterly run on both motor-driven pumps. A Unit 2 power reduction to meet IT 10 requirements of 98% power was completed at 2200 hours0.0255 days 0.611 hours 0.00364 weeks 8.371e-4 months .

The shift manager decided to turn over IT 10 to the mid shift.

Testing of P-38A was completed satisfactorily at 0216 hours0.0025 days 0.06 hours 3.571429e-4 weeks 8.2188e-5 months on 10/25/02; the recirc flow indicated 75.2 gpm on FIT-4050A, mini-recirc flow, at which time AFW pump P-38A was returned to service and the TSAC was exited. Testing of P-38B was completed satisfactorily at 0450 hours0.00521 days 0.125 hours 7.440476e-4 weeks 1.71225e-4 months on 10/25/02; the recirc flow indicated 74 gpm on FIT 4050B, mini-recirc flow. A partial performance of IT 8A to test 1P29 recirc flow was performed and IT 9A was performed to verify 2P29 operation. All testing was completed and all four pumps were back in service by 1206 hours0.014 days 0.335 hours 0.00199 weeks 4.58883e-4 months on 10/25.

ACE001023 was initiated on 10/25 at 1030 to evaluate the inadequate recirculation flow on P-38A.

On 10/25, the Operations Manager and the Assistant Operations Manager discussed the event. The Assistant Operations Manager stated that we needed a short-term follow up action plan to address the orifice plugging issue. He wanted to know if contingencies needed to put in place, such as: increase our testing frequencies; do call-ups to run the pumps more often; perform the Service Water flush more often; institute a sampling plan; etc. The Operations Manager stated that the conversation focused on the additional actions needed to done to finalize extent of condition/cause for the P-38A recirc line plugging. Late in the afternoon they paged the BOP/NSSS Mechanical Systems Engineering Supervisor who called back from home. The BOPINSSS Mechanical 8

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines Systems Engineering Supervisor, Operations Manager, and Assistant Operations Manager had a conference call. They acknowledged that an Apparent Cause Evaluation had already been assigned to System Engineering to look into the cause of the plugging.

Because ACEs have 30 day due dates, the Operations Manager and Assistant Operations Manager wanted System Engineering to develop any recommended short-term actions more quickly than 30 days. They mutually agreed to a one week due date (11/01/02) for Engineering to recommend what, if any, short-term actions should be taken. During the conversation the BOP/NSSS Mechanical Systems Engineering Supervisor asked if Operations was questioning operability, and they said they were not. The BOP/NSSS Mechanical Systems Engineering Supervisor stated that he did not have any operability concerns at this point in time. The purpose of the action plan was to find the source of the foreign material and to determine what other testing or flushing would be required to assure that future plugging does not occur. The Operations Manager stated that they also discussed the fact that we needed to look at service water to ensure we didn't have a potential to plug the orifice from service water.

The BOP/NSSS Mechanical Systems Engineering Supervisor later stated that since neither he nor the Operations Manager had any concerns with AFW operability, no immediate response was required and that over the weekend (10/26-1027) he considered the actions that might be required and formulated potential questions that needed to be answered that would be discussed at the meeting he would hold on 10/28. The Operations Manger has also stated that he didn't ask for weekend evaluation of the issue because he didn't think it was realistic that the recently designed and installed orifices would be susceptible to plugging from service water.

On 10/28/02 at 1100 hours0.0127 days 0.306 hours 0.00182 weeks 4.1855e-4 months , Engineering held a meeting to discuss the event. The meeting included the AFW System Engineer, the Engineering Programs Supervisor, the Engineering Analysis Supervisor, the PRA Supervisor, the BOP/NSSS Mechanical Systems Engineering Supervisor, a Maintenance Support Engineer, and the SW System Engineer. The outcome of this meeting identified a need to research the following items:

What is the material?

)What is its origin?

What are the tolerances in the pump?

What is the size of holes in the main SW Zum strainer?

Can we perform an operability determination on the SW issue from a realistic approach?

Explore the silt study.

Get detailed information on the recirc orifice.

Early on 10/29/02, the BOP/NSSS Mechanical Systems Engineering Supervisor convened a meeting to review the issue in detail with other members of Engineering. He stated that his purpose in convening the meeting was to communicate his conclusions that he had a reasonable degree of certainty that the issue of potential orifice clogging from service water debris was valid. The meeting was convened at 0900 hours0.0104 days 0.25 hours 0.00149 weeks 3.4245e-4 months . Personnel in attendance were the Director of Engineering, the BOPINSSS Mechanical Systems 9

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines Engineering Supervisor, the PRA Supervisor, the Auxiliary Feedwater System Engineer, the Service Water System Engineer, and the Engineering Analysis Supervisor. The Engineering Director inquired whether there was a concern regarding the ability of the system to perform its design functions. Most, if not all persons in attendance stated that they had the concern that in a situation where AFW was required to take suction from the SW system, the running pumps' recirculation orifices would likely become plugged and non-functional. The Regulatory Affairs Manager was then briefed on the issue. The BOP/NSSS Mechanical Systems Engineering Supervisor immediately informed Operations of the concern.

On 10/29/02 at 0945 hours0.0109 days 0.263 hours 0.00156 weeks 3.595725e-4 months , a meeting with Operations and Engineering personnel was held to discuss the internal tolerances of the RO with respect to potential plugging.

Attendees at the meeting were the Operations Manager, a Shift Manager, the Operations Procedure Coordinator, the Acting Plant Manager, the Regulatory Affairs Manager, the CVCS and RCS System Engineer, the Service Water System Engineer, the Auxiliary Feedwater System Engineer, the BOP/NSSS Mechanical Systems Engineering Supervisor, the PRA Supervisor, the Senior PRA Engineer, and two Operating Supervisors. The presentation basically expressed a concern with the recirc orifice plugging if SW flow was initiated to the pumps. The Operations Manager stated that the discussion centered around the fact that the service water basket strainer mesh is 1/8" and the orifice is much finer (-15mil x 90 mil). He further stated that they had extensive discussion on the fact that the AFW safety related suction source is service water. They also talked about whether we could call AFW operable from the Condensate Storage Tanks (CSTs). Most people in the room were confident that the CSTs were acceptable as a suction source, however two engineers questioned that. The Operations Manager stated that he felt that the two engineer's questions were based on uncertainty. The attendees discussed whether we could do an evaluation (OPR) on the service water suction and engineering personnel concluded that we could not in the short term. The Operations Manager stated the since service water is the safety related suction source, and we had a condition where the recirc line could fail from potentially poor water quality, and there was no indication of recirc flow in the control room, we did not have reasonable confidence that AFW could perform its safety function under all accident conditions and subsequently declared AFW out of service.

At 1027 hours0.0119 days 0.285 hours 0.0017 weeks 3.907735e-4 months on 10/29/02, all four auxiliary feedwater pumps were declared out of service due to AFW recirc orifice flow issues. Upon declaring AFW out of service, the attendees discussed options for return to service. These included shutting down the plant.

Per Technical Specification 3.7.5, when all AFW is out of service, we maintain stable plant conditions until we return a pump to service. We then take actions as directed by the other TSACs. The attendees also discussed compensatory measures and concluded that we needed a combination of administrative controls, briefs and training to essentially eliminate the need for the recirc line.

The Operations manager stated that the short-term actions were crew briefings and temporary information tags in the control room at the flow indicators to maintain 50175 10

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines gpm forward flow for motor/turbine driven pumps. Once the b~riefings and temporary information tag placement were complete, AFW could be declared operable.

Briefing of the on-shift crew on the AFW recirc flow orifice issue was completed at 1100. Posting of temporary information tags for all 4 AFW pumps with requirements to secure AFW if minimum flow is not maintained was completed at 1210.

At 1241 a CAP was initiated by the Systems Engineering Supervisor on the concern for a potential common mode failure for the AFW pumps due to plugging of the recirculation line orifices.

All 4 AFW pumps were returned to service at 1305, based on the compensatory actions taken.

At 1410, the Engineering Director established a root cause team comprised of two Engineers from the Process Engineering Group, the AFW System Engineer, and a Design Engineer.

At 1525 the Engineering Director discussed the need for an operability determination with the Regulatory Affairs Manager and the Operations Manager because of the possible operable but nonconforming condition of the AFW pumps.

Simulator runs were made from about 1530 to 1630 to evaluate the risk impact of the compensatory measures taken.

At 1700, the Operations Manager accepted Safety Screening SCR 2002-0458 for the procedure changes needed due to the potential for AFW orifice plugging.

The 8-hour NRC notification required by 10 CFR 50.72(b)(3)(v)(D) was made using the Emergency Notification System (ENS) telephone at 1711. Event number EN 39330 was assigned to this notification.

The Operations Manager requested an OPR for CAP029952 concerning the common mode failure of the AFW pumps from Engineering at 1830.

At 2030, the Safety Monitor transitioned to Yellow for both units based on PRA Group input.

On 10/30 at 0700, the initial meeting of the Root Cause Team was held to discuss scope and responsibilities.

At 1030, the Engineering Director suspended the qualifications for design work for three individuals involved with the AFW orifice modifications.

At 1100 the site was notified that an NRC Special Inspection Team would be responding to the Event Notification.

11

Possible Common Mode Failureof RCEO00191 Aux Feedwater RecirculationLines The Engineering Director approved the charter for the Root Cause Team at 1250.

OPROO0031 was completed and approved at 1850. OPROO0031 concluded that the AFW pumps were operable but nonconforming because the AFW pump recirculation paths described in the FSAR were not available. Compensatory actions specified included shift briefings and placement of temporary information tags. An additional action specified was to implement operating procedure changes. The condition where compensatory actions could be terminated was stated as an engineering evaluation or modification to restore the function of the recirculation lines.

On 10/31, the Site Vice President established an event resolution team to provide management oversight to drive resolution of the issue involving the potential to have a common mode failure due to orifice plugging in the AFW recirculation lines. The organization consisted of:

"* Team Lead - Nuclear Oversight Manager (temporarily relieved of job responsibilities)

"* Incident Investigation - Radiation Protection Manager

"* On-Line Risk Management - Production Planning Manager

"* Interim Corrective Actions - Operations Manager

"* Issue Resolution Team and Root Cause Evaluation - Engineering Director Ill. Extent of Condition Assessment

[Written summary of the generic imAphcations and the breadth (symptoms 6- causes) oftheproblem.]

- SG blowdown lines

- Radwaste steam?

IV. Nuclear Safety Significance

[Written summagy ofactualandpotentialnuclearsafty imptications. This may require inputfrom j'censinb EngineeringorProbabilisticRisk Assessment depending upon the event.]

Under development - depends on plugging probabilities. On line risk is yellow with procedural restrictions in-place.

V. Report to External Agencies This event was determined to reportable to the NRC in accordance with 10 CFR 50.72(b)(3)(v) as a condition that could have prevented the fulfillment of the safety function of structures or systems that are needed to:... (D) Mitigate the consequences of an accident. This is an eight-hour non-emergency notification. The NRC notification was made using the Emergency Notification System (ENS) telephone at 1711 on October 29'h. Event number EN 39330 was assigned to this notification.

12

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines A Licensee Event Report (LER 266/2002-003-00) will be submitted within 60 days of this event as required by 10 CFR 50.73.

CA026903 has been initiated to issue an operating experience notice to the industry if appropriate.

CAXXXXX has been initiated to report this event to KNPP.

[This section subject to revision based on conclusions regarding plugging probabilities.]

VI. Data Analysis Information & FactSources

[Interview Information]

[Questionnaires/Surveys]

[DataSources (Data,Logs, Records, Procedures,Etc.]

[DirectEvidence (Parts,Photos,Etc.]

[Relevant Industry & Station OperatingExperience]

Document Review Results EWR 99-031 The EWR was initiated on 12/8198 to address Control Room noise resulting from operation of the AF pumps in the recirculation mode. The EWR was to determine if restricting orifices in the recirculation lines should be replaced with multi-stage restricting orifices. An action item was created on 4/15/99 and assigned to Mechanical Design Engineering. In an update to this action item made on 614/99, Design Engineer 1 concluded that most of the noise was due to cavitation in the restricting orifice, and that the orifice did not have an adequate number of stages. It was recommended that an EAC evaluation be performed to look at three options to resolve the issue. The options were installation of a multi-stage pressure- reducing orifice, replacement of the present orifice with a pressure-reducing flow element in a valve body, and retrofitting the AOV control valve with a multi-stage pressure-reducing trim.

On July 9, 1999 the Engineering Advisory Committee reviewed three options proposed to address EWR 99-031. The EAC did not select an option, but did recommend implementing modifications for the MDAFW pumps during 2000 and then evaluate performance to determine if TDAFW pumps should be modified during 2001.

Modification requests MR 99-029*A-D were initiated to replace the AFW recirculation line restricting orifices and the EWR action item was closed on 7/29199.

OD CR 99-1391 Rev. 0 of this OD was issued on 5/21/99 and dealt with three pinhole leaks that had developed on the MDAFW pump recirculation lines in the past year. The safety function for the recirculation line is described as designed to ensure a minimum flow through the 13

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines AF pumps to protect from adverse effects of hydraulic instability at low flow rates. The minimum flow rate for the MDAFW pumps (P-38A and P-38B) is 70 gpm, and for the TDAFW pumps (IP-29 and 2P-29) it is 100 gpm.

In the section of the OD that discusses the basis for declaring the SSC Operable But Degraded or Nonconforming, reference is made to the Westinghouse LONF/LOAC Analysis and that it requires the AFW recirc AOV to close within 60 seconds after the flow setpoint has been reached. DBD-01 describes the delay in reaching full capacity for MDAFW pumps as being 35 seconds and for the TDAFW pumps it is 39 seconds. It is then concluded that the recirculation line AOV will not be open longer than 100 seconds.

The discussion then describes that various PBNP EOPs require the use of AF to maintain SG levels, and that this may require operation at low flows such that the recirculation AOV could be open. Part II of the OD specifies that implementation of MR 99-029*A-D will replace the existing orifices and that build up of the socket welds will occur during installation of those modifications.

The OD was revised on 1/7/00 to combine it with an OD from CR 99-1844, which dealt with the cause of the weld failures. The OD was revised again on 3/6/01 to reflect completion of MR 99-029*A&B for the MDAFW pumps, which restored them to a fully operable condition. The OD was closed on 10/22/02 after MR 99-029*C and D were completed.

EvaluatorObservations:

The OD clearly recognizes the needfor having AFW recirculationcapability during an accident.

MR 99-029*A/B (MDAFW Pump ROs)

The purpose of the proposed modifications was to minimize piping line noise and vibration when operating the MDAFW pumps in the recirculation mode. This would also eliminate the socket weld failures caused by cycle fatigue created by this vibration.

The scope of the modifications was to replace the existing restricting orifices with new pressure reducing orifices. In addition, a portion of piping was being replaced which included oversized weld sockets.

Installation and testing were completed during November 2000. MR 99-029*B was accepted during November 2000; MR 99-029*A was accepted in March 2001, following drilling of additional holes in the innermost stage of the orifice to achieve the desired flow rate.

The final design description contains the following relevant information:

DBD-01, Auxiliary Feedwater System, is listed as a design input

The replacement RO will have the same function as the existing orifice RO-4008 (RO-4014), which is to provide pressure reduction and act as a pressure boundary for the AF system piping.

14

Possible Common Mode Failureof RCEO0191 Aux FeedwaterRecirculationLines It (Bechtel Specification No. 6118-M-6 Rev. 3 dated 10/28/68) specifies that, "Each pump shall be furnished with a pressure reducing orifice to be used in conjunction with the on-off control valve in the pump recirculation piping."

The design of the new RO is different than the presently installed orifice.

The replacement RO is heavier than (the) existing one.

None of the above changes is introducing new, unknown equipment to PBNP.

The proposed modification is located in the Control Building on Elevation 8", and adheres to the requirements of the Fire Protection Evaluation Report. A fire protection analysis, for the affected area, has been performed and the Fire Protection Conformance Checklist, PBF-2060 has been completed and approved.

The design input checklist (PBF-1584 Rev. 6 3/31/99) contained the following:

"* Item A.6, Incorporate new types/models of equipment not presently used at PBNP? - APPLIES TO DESIGN? - NO

"* Item A. 10, Consider failure effects on structures, systems, and components:

APPLIES TO DESIGN? - NO

F.2.b, Affect fire protection requirements? - APPLIES TO DESIGN - YES

F.2.f, Based on Tables 6.7-1 to 6.7-4 and Figures 6.6-1 to 6.6-8b and 6.9-1 to 6.9 2d, will the change add to, delete from, or revise the listed systems and components? - APPLIES TO DESIGN - YES The modification request checklist (PBF-1606 Rev. 5 7/24/98) had the following regarding documentation updates:

Item B.4, Component Instruction Manuals - N/A

Item B. 14, EPIX Update - N/A

Item B.18, PSA Models and Documentation - N/A

Item C.5, Spare parts stocking and scrapping inputs into CHAMPS - N/A

Item D.1, Abnormal Operating, Normal Operating, and Refueling Procedures N/A

Item D.5, EOPs, ECAs, CSPs - N/A

Item E.4, Preventive Maintenance - initiate/revise CHAMPS callups - N/A

Section I, ECRs - blank The MR 99-029*A package has an addendum with a revised IWP to drill 6 additional holes in the innermost stage of the orifice. The MR 99-029*B package utilized the existing 1WP and revised it to drill 4 additional holes in the innermost stage of the orifice.

There is a pen and ink change to the IWP revising the number of holes from 4 to 6.

EvaluatorObservations:

Thefinal design descriptiondoes not discuss the potentialfor orifice plugging from service water

Thefinal design descriptiondoes not discuss service water strainersize

DBD-01 describes a safety-relatedfunction of the orifice to provide sufficient flow to prevent low-flow instabilitiesand excessive fluid temperature rise in the AFW pumps - this is not discussed in the final design description 15

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines The basisfor concluding that the RO is not new or unknown equipment does not address the new orifice size being less than the service water strainer

The basisfor not performing a FMEA is not provided

The FireProtection Conformance Checklist answer to question 5.8, "Will the revised shutdown component continue to perform itsfunction requiredby AOP JOA, AOP JOB, AOP IOC, andAOP IOD?" is marked YES with no basis provided.

SE 2000-0055 (MR 99-029*A/B)

The safety evaluation screening for MR 99-029*A/B concluded that there was a difference in the design of the replacement RO, so the proposed activity would constitute a change to the facility as presently described in the current CLB.

The approved safety evaluation contains the following relevant information:

"* The ROs do function to support operation of the AF system pumps P-38A and P 38B and these pumps are involved in accidents (Section 2.A.1)

"* The changes that will be implemented by the proposed modifications will not affect the overall performance of the AF System and operation or function of the AF pumps P-38A and P-38B to perform their intended function (Section 2.A.1)

"* During accident conditions, the safety related functions of each AF pump recirculation line orifice is as follows: (in part) These ROs must provide adequate flow to prevent low-flow instabilities and excessive fluid temperature rise in the AF system pumps (Section 2.A.2)

"* The components are passive in nature when the system is operational and will be designed, installed and tested in accordance with the existing procedures and controls. Therefore, they do not introduce any new failure mechanisms not already considered for the area. (Section 2.A.2)

"* The modified recirculation lines will function identically to the currently installed recirculation lines (Section 2.A.4)

"* The oversized socket welds and replacement ROs do not change the function, method(s) of operation, or introduce any new credible failure mechanisms to the AF pumps P-38A and P-38B and their recirculation lines (Section 2.A.5)

"* The flow passage area of replacement ROs could possibly lead to reduced pump recirculation flow during operation of the pumps with SW since particles/debris in the SW could be filtered by the RO's trim. To preclude this, the RO's design directs flow through the outside of the trim. The outside cartridge of the trim contains the smallest size flow passage area. The flow passage area then becomes progressively larger. Therefore, the smallest flow passage areas are located at the zone of highest differential pressure. This design feature reduces the potential of debris accumulation on the RO's trim. (Section 2.A.5)

"* The recirculation line AOV automatically closes approximately 45 seconds after the pump discharge flow is approximately 95 gpm and increasing. Failure to pass flow through the recirculation orifice during the 45 seconds would be conservative since flow to the SGs would be delivered sooner. The recirculation line AOV is also designed as a failed closed valve to ensure that recirculation 16

PossibleCommon Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines flow is not diverted from the SG in the event of a loss of instrument air. (Section 2.A.5)

The recirculation line flow path is not require to support AF system in its response to the design basis accidents since the pumps' discharge valves will automatically open fully in response to the accident and provide a flow path for the pump. Failure to pass flow through the recirculation orifice due to potential of SW debris accumulation on the ROs trim would be conservative since flow to the SGs would be delivered sooner. (Section 2.B)

The Manager's Supervisory Staff reviewed the safety evaluation at a meeting on 4/18/00.

The version of the safety evaluation presented to MSS had different wording that generated some discussion. The MSS Chairman questioned a phrase in Section 2.A.5 that stated "The AF pumps have 9 stages with 0.009" to 0.014" diametrical clearances and a minimum 0.4375" impeller vane path. Since the pumps have multiple stages and small clearances, they will reduce larger particles size contained in SW to less than 0.015"." The MSS Chairman did not feel the statement was entirely true because he felt that particles could go through the vane paths. The AFW System Engineer said the pump had 9 stages and the particles would have to clear all 9 clearances in order to pass through, and this was unlikely to occur.

The Mechanical Design Supervisor said the discussion that could be included in that section is that of plant configuration and the fact that SW is used. He also did not like the quantitative discussion and recommended discussing the recirculation line flow path.

The Design Engineer subsequently removed the phrase from the safety evaluation and included a different discussion including the phrase "The recirculation line flow path is not required to support this function since the pump discharge valves will automatically open fully in response to the accident and provide a flowpath for the pump. The recirculation line AOV automatically closes at approximately 95 gpm increasing. Failure to pass flow through the recirculation orifice during the 45 seconds would be conservative since flow to the SGs would be delivered sooner. The recirculation line AOV is also designed as a failed closed valve to ensure that recirculation flow is not diverted from the SG in the event of a loss of instrument air."

With those changes made, MSS recommended approval of the SE. The Operations Manager approved the document in the absence of the Plant Manager.

The JOSRC 50.59 Subcommittee reviewed and approved the SE without comment at its meeting held on 6/27/00.

In an earlier, undated version of the draft safety evaluation, the following information was provided:'

In summary, based on the large differential pressure across the ROs and the effect of pump discharge clearances on SW sample particle size, the replacement ROs design will not result in acute reduction in AF pumps recirculation flow, therefore, causing the pumps to fail. In addition, the body of the replacement ROs 17

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines contain a bolted blind flange that can be removed to facilitate cleaning of the trim in the event of long-term debris accumulation. This long-term accumulation can be identified in advance by noticing a changing pattern of recirculation flow values that are recorded during IT-I0/10A/10B.

EvaluatorObservations:

"* The safety evaluation does not discuss the Appendix Rfunction (partof the CLB) of the recirculationline as a safe shutdown flow path.

"* The safety evaluation only discusses the recirculationlinefunction duringAFW pump start and does not addresspump shutdown or the needfor operationon recirc during the other stages of accident response.

"* The safety evaluation states that the modified recirculationlines willfunction identically to the existing lines even though it is assumed that the orifices may plug.

"* The safety evaluation does not consider that loss of the recirculationline due to loss of instrumentair is recoverable via actions specified in AOP-5B.

"* The safety evaluation discussion concerning the orifice design thatprecludes plugging is extractedfrom the vendor manual, where the information is discussingwhy the internalthrottle plug (on a different design orifice) will not stick or gall.

"* Neither MSS nor JOSRC raisedthe aforementionedissues.

MR 99-029*C/D (TDAFW Pump ROs)

The purpose of the proposed modifications was to minimize piping line noise and vibration when operating the TDAFW pumps in the recirculation mode. This would also eliminate the socket weld failures caused by cycle fatigue created by this vibration.

The scope of the modifications was to replace the existing restricting orifices with new pressure reducing orifices that had a movable plug. In addition, a portion of piping was being replaced which included oversized weld sockets.

Installation and testing of the Unit 1 TDAFW pump RO was completed during September 2002, and the modification was accepted during October 2002. Installation and testing of the Unit 2 TDAFW pump RO was completed during April/May 2002, and the modification was accepted during May 2002.

The final design description contains the following relevant information:

DBD-01, Auxiliary Feedwater System, is listed as a design input (*D only)

The replacement RO will have the same function as the existing orifice, which is to provide pressure reduction and act as a pressure boundary for the AF system piping.

It (Bechtel Specification No. 6118-M-6 Rev. 3 dated 10/28/68) specifies that, "Each pump shall be furnished with a pressure reducing orifice to be used in conjunction with the on-off control valve in the pump recirculation piping."

The design of the new RO is different than the presently installed orifice.

The replacement RO is heavier than (the) existing one.

18

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines

None of the above changes is introducing new, unknown equipment to PBNP.

A Fire Protection Conformance Checklist, PBF-2060 is listed as a design output

(*C only).

The design input checklist (PBF-1584 Rev. 8 6/8/01 for *D and Rev. 9 11/5/01 for *C) contained the following:

"* Item A.6, Incorporate new types/models of equipment not presently used at PBNP? - APPLIES TO DESIGN? - NO

"* Item A. 10, Consider failure effects on structures, systems, and components:

APPLIES TO DESIGN? - NO

Item F.2.b, Affect fire protection requirements? - APPLIES TO DESIGN - NO

Item F.2.e, Based on Section 2 and Appendix A of the SSAR, will the change add to, delete from, or affect the performance of safe shutdown systems or equipment?

- APPLIES TO DESIGN - YES (for *C) and NO (for *D)

The modification request checklist (PBF-1606 Rev. 5 7/24/98 for *D and Rev. 6 10/2/01 for *C) had the following regarding documentation updates:

PSA Models and Documentation - N/A

Spare parts stocking and scrapping inputs into CHAMPS - N/A

Abnormal Operating, Normal Operating, and Refueling Procedures - N/A

EOPs, ECAs, CSPs - N/A

Preventive Maintenance - initiate/revise CHAMPS call-ups - N/A

ECRs - blank EvaluatorObservations:

The final design descriptiondoes not discuss the potentialfor orifice plugging from service water

The final design descriptiondoes not discuss service water strainersize

DBD-01 describesa safety-relatedfunction of the orifice to provide sufficient flow to prevent low-flow instabilitiesand excessive fluid temperature rise in the AFW pumps - this is not discussed in thefinal design description

The basisfor concluding that the RO is not new or unknown equipment does not address the new orifice size being less than the service water strainer

The basisfor not performing a FMEA is not provided

Applicability of affect on FireProtection is not addressedcorrectlyfor the *D modification.

For the *C modification, the Fire ProtectionConformance Checklist answer to question 5.6, "Does the modification affect operation of a system relied upon for post-fire safe shutdown (e.g. changes is system flow rate, change in normal position, etc. ?" is marked NO with a statement that "Operationof the auxiliary feedwater system will not be affected. The replacementRO will be set to the same flow rate as the current RO." This is not consistent with assumptionsfrom the previous modifications to the MDAFW pump ROs that the recirculationline may not be availabledue to plugging.

19

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines SCR 2001-0981 (MR 99-029*C/D)

The safety evaluation screening for MR 99-029*C/D was performed under the new 10 CFR 50.59 rule and concluded that the activity did not adversely affect the design function of an SSC credited in safety analysis, and did not adversely affect the method of performing or controlling the design function of an SSC credited in safety analysis.

Supporting information included:

"* The restricting orifices design functions affected by the modifications are: (in part) They ensure adequate flow and pressure drop through the AFW pumps when they are operated in the recirculation mode, thus preventing low flow instabilities and excessive fluid temperatures. (Section 1.2 and II)

"* These orifices are not explicitly required in an accident to be able to pass service water, since the recirculation control valve would be closed when the pump is aligned to the steam generator. However, it is possible that when the pump is aligned to the service water system supply after the condensate storage tanks have been drained, service water could be pumped through the recirculation lines.

(Section II.1)

"* To preclude the chance of clogging the orifice trim, the flow is directed from the outside of the stages inward. The holes in the outer stage are the smallest and they get progressively larger in the inner stages. This causes the largest differential pressure to exist at the outer stages at locations with the smallest hole, which will reduce the potential for debris accumulation inside the orifice.

(Section 11D.1)

The MSS and JOSRC did not review this information because it was determined to only be a screening under the new rule.

EvaluatorObservations:

"* The safety screeningdoes not discuss the Appendix R function (credited in the safety analysis) of the recirculationline as a safe shutdownflow path.

"* The safety screeningdoes notfully evaluate the recirculationline function while supplying service water to the SGs. The ability to startand stop the AFW pumps while on service water and to throttle back AFWflow to the SG is not addressed.

"* The safety screening raisesthe issue of the potentialforthe recirculationorifices to accumulate debrisfrom SW but does not provide any definitive basisfor the conclusions drawn regardingits impact on AFW pump operation.

"* The safety screening discussionconcerning the orifice design thatprecludes plugging is extractedfrom the vendor manual, where the information is discussingwhy the internalthrottleplug will not stick or gall.

"* There is no discussionexplaining the change in conclusions drawnfrom SE 2000 0055 regardingnot needing the recirculationline.

MR 02-029 (Upgrade safety function of AFW recirculation AOVs and line)

The purpose of this modification was to upgrade AFW minimum flow recirculation AOVs to have a safety related function to open.

20

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines The scope of the modification included removal of the internals for check valve AF-1 17 to prevent a common mode active failure of all AFW pumps due to isolation of the mini recirculation line. The AF-1 17 valve is non-QA, non-Seismic, and non-ASME. The modification will document the upgraded design basis of the AFW recirculation line AOVs and piping to support the safety-related function to provide a flow path for the AFW pumps to prevent overheating and hydraulic instabilities.

There was not EAC review of this modification because it was scoped as a Level of Effort Minor Plant Change.

This modification was initiated on 8/20/02, and was installed, tested and accepted on 9/12/02.

The final design description contains the following relevant information:

"* The minimum recirculation flow AOVs have a safety-related function to close to ensure adequate flow to the steam generators during several events.

"* FSAR Section 10.2 also discusses the effects of a failure of a mini-recirc AOV to close and gives the flow that is diverted from the steam generators through the recirculation lines as limited by the flow restricting orifices.

"* These min-flow recirc AOVs have never been classified as having a safety related function to open to prevent pump damage. This has been described as a non-safety related function only, since the AFW pumps will always have forward flow to the steam generators on auto-start.

"* These recirculation lines AOVs have an augmented quality function to be opened for Appendix R fires to support AFW pump operation, per SSAR 2.3.1.4.

"* Letter NRC 2002-0068 dated 8/12/02 states that PBNP will classify the open function for the AFW pump minimum flow recirculation valves as safety-related.

The letter also states that because not all of the recirculation flow path is safety related, operability of the AFW pumps will not be dependent on the availability of that flow path. However, it has been conservatively decided to tie AFW pump operability to this recirculation line. Therefore, even though the recirculation line downstream of the orifices is not safety-related, it is required to be in-service to consider the AFW pumps fully operable per TS 3.7.5.

"* The current safety-related boundary for the recirculation lines is at the flow restricting orifices.

"* This line can be credited to support a safety-function while not being classified safety-related because failure of the piping would be conservative in terms of AFW pump protection.

"* The only credible failure of the piping that would cause AFW pump damage would be if check valve AF-1 17 failed to open (an active failure). Therefore, the internals for this check valve will be removed.

"* All other non-conservative failure modes for the recirc line are passive in nature.

Several manual valves exist in the recirculation lines, and all of these valves are currently red-locked open. Mispositioning is not credible due to procedural controls in place (red locks), and a disk separation failure of a manual valve is considered passive.

21

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines

" (Installation) Both Units may be in any condition, but the common recirculation line must be isolated to remove the AF- 117 internals. While this line is out of service, the AFW pumps will be considered fully operable. Manual operator action will be credited to prevent pump damage by stationing a level 3 dedicated operator at the AF-4035 relief valve. If the minimum flow recirculation line is needed, and the relief does not open automatically while the line is isolated, then the dedicated operator will notify the control room that the recirculation flow path is not available.

"* This action for the dedicated operator will be required only when the control operator has taken action to reduce AFW flow.

The design input checklist (PBF-1584 Rev. 10 8/19/02) contained the following:

"* Item A. 10, Consider failure effects on structures, systems, and components.

APPLIES TO DESIGN - YES.

"* Item A.1O.a, The design discusses those events/accidents which the system/components are to withstand? - APPLIES TO DESIGN - YES. The AFW recirculation line is required to support AFW pump operation during events where AFW is required to provide reactor heat removal. Most of the recirculation piping is non-Seismic, as are the CSTs, but a failure would be conservative in terms of recirc flow.

"* Item A.10.b, The failure effect of the system/components: - APPLIES TO DESIGN - YES. AF-1 17 internals are being removed to prevent a single active failure from making all AFW pumps inoperable.

"* Item F.2.e, Based on Section 2 and Appendix A of the SSAR, will the change add to, delete from, or affect the performance of safe shutdown systems or equipment? - APPLIES TO DESIGN - YES. The AFW system is a safe shutdown system credited for Appendix R. The mini-recire AOVs are credited to be opened in this scenario to provide AFW pump cooling. Upgrading the safety function of the AOVs and removing the AF-1 17 internals improves this capability.

"* Item G. 1, Installation requirements/plant conditions have been determined?

APPLIES TO DESIGN - YES. Additionally, a level 3 dedicated operator will be present to monitor the relief valve, and notify the control roorfi if it fails to open.

The modification request checklist (PBF-1606 Rev. 6 10/2/01) had the following regarding documentation updates:

Item B.3, Component Instruction Manuals - N/A

Item D.5, Spare parts stocking and scrapping inputs to CHAMPS - N/A

Item J, ECRs - blank EvaluatorObservations:

"* Design evaluation does not evaluate impact of safety function upgrade on all components in the AFW recirculationline (didnot evaluate ROs or electrical circuitfor AOVs)

"* Design evaluationdoes not evaluate impact of safety function upgrade on past AFW modifications 22

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines Design evaluation does not evaluate impact of safeýyfunction upgrade on AFW modifications in progress (MR 99-029*C)

SCR 2002-0359 (MR 02-029)

The safety evaluation screening for MR 02-029 was performed under the new 10 CFR 50.59 rule and concluded that the activity did not adversely affect the design function of an SSC credited in safety analysis, and did not adversely affect the method of performing or controlling the design function of an SSC credited in safety analysis. Supporting information included:

"* The scope of the screening involves: (in part) State in the FSAR (FSAR 10.2) and Technical Specification Basis (B.3.7.5) that (1) the open safety function for all AFW pump mini-recirc valves is safety-related, and (2) the recirculation line downstream of the flow restricting orifices have a safety function and is required for AFW operability, but the line is not safety-related since failure of the line is conservative. (Section 1.1)

"* The AFW system has the following functions described in the licensing basis (in part):

"o To provide sufficient feedwater to remove decay heat from both units for one hour during a station blackout (SBO) event (TDAFP only)

"o To provide sufficient flow to the steam generators to remove decay heat to achieve cold shutdown within 72 hours8.333333e-4 days 0.02 hours 1.190476e-4 weeks 2.7396e-5 months following a plant fire (Appendix R)

"o To provide flow to the steam generators during plant startup and shutdown, and during hot shutdown or hot standby conditions for chemical additions and when operation of the main feedwater and condensates systems is not warranted. (Section 1.2)

"* The AF-1 17 check valve has an implicit function to open to allow minimum recirculation flow from the AFW pumps to return to the condensate storage tank(s). (Section 11.1)

"* The safety-related flow restricting orifices in the recirc line from each AFW pump limit the flow and pressure from each pump. (Section 11.1.b)

"* The minimum recirculation valves for the AFW pumps have the design functions to isolate the minimum recirculation line to ensure that the AFW pumps deliver the required flow to the steam generators as needed to support the mitigation of accidents or events. (Section 11.2)

"* The only function identified for the AF-1 17 check valve was to open to allow mini-recirculation water from AFW pumps to return to the condensate storage tanks. It has no design function to close. (Section In.1)

"* The change in designation of the open function of the mini-recirculation valves (AOVs) as safety-related has no adverse affect on the valve's function to open or close.

Evaluator Observations:

"* The descriptionof designfunctions in Section Ilfocuses on selected components and does not discuss the overallifunction of the line.

"* The discussion of the ROfunction in Section II.l.b does not include its need to passflow for pump cooling.

23

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines

The potentialfor plugging of the restrictingorifices is not discussed.

The discussions of specific designfunctions in Section H and III do not include the Appendix Rfunction.

SCR 2002-0377 (AFW Operability During MR 02-029)

The safety evaluation screening for AFW operability was performed under the new 10 CFR 50.59 rule and concluded that the activity did not adversely affect the design function of an SSC credited in safety analysis, and did not adversely affect the method of performing or controlling the design function of an SSC credited in safety analysis.

Supporting information included:

"* The Level 3 Dedicated Operator will remain in constant radio communications with the control room, and the operator's only function is to monitor actuation of AF-4035, and to notify the control room if the valve fails to open after AFW pumps start. If AF-4035 fails to relieve (open), the dedicated operator will notify the control room immediately while the pumps are still being cooled by forward flow, and the control room operators will know that when they reduce AFW flow to control steam generator level they will have to maintain the minimum forward flow in AFW pumps or secure pumps as necessary as directed by the EOPs as discussed above. (Section 1.1)

"* The minimum recirculation lines for the auxiliary feedwater pumps and the recirculation header for the AFW pumps have a function of providing recirculation flow paths from the AFW pumps to prevent hydraulic instabilities and to dissipate pump heat. Hydraulic instabilities are prevented by the presence of flow restricting orifices in the individual AFW pump recirculation lines, so the function of maintaining individual pump cooling and the impact of diverted CST water are the concerns. (Section I1)

EvaluatorObservations:

"* The responsibilitiesfor the dedicated operatorin the safety screening only addressAF-4035 monitoring and notification associatedwith AFW pump starts, there is no discussion of monitoringwhen flow is reduced to less than required minimum flow values.

"* The potentialfor plugging of the restrictingorifices is not discussed.

DBD-01, Auxiliary Feedwater System The DBD section on AFW Pump Recirculation Line Orifices (Section 3.16 in the current Revision 3 of the DBD) describes the Safety-Related Functions as:

1. These orifices shall provide passive flow resistance in the recirculation line of each AFW pump; thereby establishing the required min-recirc flow and pressure drop from the AFW pump discharge pressure to CST pressure. These orifices must provide sufficient flow to prevent low-flow instabilities and excessive fluid temperature rise in the AFW pumps [REF 9.5.17].

2. These orifices shall limit the recirculation flow in the event that the recirculation control valve fails to close during the AFW operation response to an accident

[REF 9.5.117].

24

Possible Common Mode Failureof RCE000191 Aux FeedwaterRecirculationLines

3. These orifices shall passively maintain the AFW system pressure boundary integrity.

These functions have remained essentially the same since the original DBD was issued in 1994. However, as part of CA003702 from Root Cause Evaluation 01-069, the description of the AFW recirculation line in the FSAR, DBD-01, and the IST program was reviewed for consistency and accuracy, and required changes were initiated. One change that was identified was that the function regarding providing sufficient flow (Item 1 above) was not a Safety-Related Function because the recirculation AOVs did not have an open safety function. The original DBD-01 did identify the recirculation AOVs as having an open safety function based on the same reference provided for the ROs, REF 9.5.117, which was MR 88-099*A-C for increasing the recirculation line flow rate in response to NRC Bulletin 88-04. CR 97-3363 evaluated the open safety function of the recirculation AOVs and concluded the DBD needed to be revised to reflect that there was no open safety function. That change was made in Revision 1 of DBD-01 on 3/21/00.

As a result of CA003702, a marked-up draft of the DBD was created that showed moving the RO function for providing sufficient flow from the Safety-Related Function section to the Non-Safety-Related/Non-QA Function section. CA003702 was completed on 8/13/02.

While this change was pending, the decision was made to upgrade the open function of the recirculation AOVs to safety-related. As part of the DBD-01 changes needed to support MR 02-029, the proposed revision to the RO safety function for providing sufficient flow was abandoned.

A related issue was identified regarding the statement in the DBD regarding the RO function to limit flow. This statement was also attributed to MR 88-099*A-C.

CAP029983 was initiated on 11/2/02 to resolve the actual flow limit values for the ROs.

EvaluatorObservations:

After it was recognized that the RO did not have a safety function to provide sufficientflow, there was no additionalevaluationperformed to restore that function to support MR 02-029.

NP 1.6.5, Manager's Supervisory Staff and Qualified Reviewer Revision 3 (11/8/99) of NP 1.6.5 was reviewed to see what guidance was in effect when MSS reviewed SE 2000-0055 on 4/18100. Section 3.2.4 lists the responsibilities of an MSS Member as:

"* Review all proposed changes or modifications to plant systems or equipment where changes affect nuclear safety.

"* This includes 10 CFR 50.59172A8 evaluation reviews and a review of the modification request design scope.

Qualified Reviewer responsibilities are listed as:

25

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines

"* Review procedures or changes thereto, which affect nuclear safety as designated by the Plant Manager.

"* Review temporary changes to procedures described in Attachment B within two weeks.

In general, the MSS Members must meet the ANSI N18.1-1971 qualifications for their discipline. There are also requirements for a Qualified Reviewer to either have 10 CFR 50.59 Safety Evaluation qualifications or be designated by the plant Manger based on plant experience.

Evaluator Observations:

"* There is no requirementfor MSS Members that review modification safety evaluations to be 10 CFR 50.59 qualified.

"* The procedure specifies MSS Members are to review the modification request design scope, but the practice it to only distribute the safety evaluation document for review.

NP 7.2.1, Plant Modifications NP 7.7.10, Q-List Nuclear Safety Classification for Structures, Systems, and Components Recirculation Line Potential Failure Modes Review On 11/6/02 a review of the AFW PRA Notebook, associated with the AFW pump recirculation lines, was performed as part of the RCE000191 charter. An experienced KNPP Design Engineer who possessed no previous PRA knowledge or experience in the development of PRA assumptions or bases performed this review. The intent of this technical review was to identify any additional potential component failures and/or postulated failure modes not already addressed in the Point Beach AFW PRA model.

The Point Beach Senior PRA Engineer provided a brief overview of the AFW PRA Notebook structure and use of the PRA Notebook drawings and section tables.

The applicable AFW Notebook drawings and section table failure modes, associated with the AFW recirculation system, were reviewed against applicable portions of the following documents:

AFW system P&ID drawings - BECH 6118 M-217, Sheets 1, IA, and 2.

Fire Water P&ID drawings - BECH 6118 M-208, Sheets 1 and 2.

Service Water P&ID drawing - BECH 6118 M-207, Sheet IA.

DBD-01, Auxiliary Feedwater

DBD-12, Service Water System, Sections 3.15, 3.20, and 3.20.2.

FSAR, Sections 10.2,10.3, and 14.

CR 99-1391, Operability Determination, Rev. 0, dated 5/21/99.

26

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines

"* CHAMPS data for valves I-AF-04002, 2-AF-04002, AF-04014, AF-04007, 1 AF-00 114, 2-AF-00 114, AF-00 115, AF-00 116 and orifice flow elements I-FE 4049, 2-FE-4049, FE-4050A, and FE-4050B.

"* Design Changes MR 99-029*A, B, C, D and MR 02-029.

The review consisted of reviewing the PRA data and assumptions against the data and assumptions located in the above listed documents. This was performed on a component by-component basis for the components associated with the AFW pump recirculation lines. This included review of the main air/backup air supply circuits for the recirculation line AOVs and cooling water supply to the AFW pump bearings.

A summary of findings and items to be addressed were identified in Point Beach CAP030045 initiated on 11/7/02. No significant Failure Modes and Effects issues were identified under this review.

Interview Results Civil/Structural Design Engineer He recalled that a meeting was held to discuss making the recirculation line safety related (modification MR 02-029*). This meeting was held approximately 1.5 to 2 weeks prior to the installation of modification MR 99-029C* which, installed the last restricting orifice in the aux feedwater pump recirculation line. It was believed that Design Analysis arranged this meeting. This meeting was held after the decision was made to make the lines safety related but prior to the issuance of a letter to the NRC stating this position. It is believed that representatives from System Engineering, Mechanical Design, Design Analysis, and Design Drawings participated in this meeting. No meeting notes were taken to document the discussion.

The purpose of the meeting was to discuss the relief and check valves. The discussion focused on the common portion of the line because the rest were already safety related.

A review of the recirculation lines in their entirety for the ability to provide flow was not preformed nor was the orifice discussed. The discussions included questioning if forward flow could always be credited and therefore preclude inoperability of the pumps if recirculation flow was lost. No resolution of this question was stated during the interview.

It was stated that the attendees were apprehensive about declaring a safety function for these lines since it was not designed or installed to meet this qualification. . A discussion as to whether it was necessary to remove the relief valve or any other component that may cause problems in the future also took place. No components were identified during this discussion. All participants had reservations about the check valve modification (MR 02-029) and giving the recirculation line a safety function. He stated that the participants felt that the decision to make the line safety related would eventually "bite" us.

27

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines Civil/Structural Engineering stated the Engineering felt left out of the decision process to "do the right thing" to get past the red finding. They felt that senior management dictated an action without proper input.

Design Engineer 1 The modification engineer for MR 99-029*A/B no longer works at PBNP and was contacted by telephone for this interview. The engineer was asked why this particular design of orifice was chosen. His reply was that this orifice was selected because it was the cheapest available design that would fit in the space available. The engineer stated adamantly that he knew that the orifice would plug instantly if exposed to service water.

He also stated that this was not a problem because flow would always be present to the steam generators because it would not be throttled to lower values. He stated that this was the accepted function of the line at the time (no safety function/not needed). He also stated that the system engineer agreed with the conclusion that the recirculation line did not have a safety function and was not required for the pump to be operable.

When asked why the design description did not include a discussion of orifice plugging, the engineer stated that he felt placing the discussion in the safety evaluation was acceptable because the safety evaluation was a part of the design package and it didn't matter where plugging was discussed.

Design Engineer 2 (Still needs to be condensed)

1. What is your recollection of the thought process for upgrading the safety function of the AFW Recirc AOVs?

I was not involved in the discussions related to that decision. The information just came to me. The pneumatic back-up supply for the MDAFW recirc valves was always going to be safety-related. For the TDAFW system, a whole new back-up system was needed; it began as safety-related. In the middle of the mo d (Dec 01/Jan

02) the direction changed to making the mod augmented quality. Based on second hand information (Novak), Mende made the decision. The components were bought as safety-related anyway, and the MDAFW mod was already installed. During installation (Apr 02) it was decide to call the mod safety-related, just prior to the NRC Meeting (4/29/02). I think by second hand information that this came from Warner (SVP). An ECR was issued for the upgrade. Fred Cayia sent out a memo explaining the upgrade decision (NPM 2002-0228).

In the August 2002 timeframe a decision was made to upgrade the safety function of the AFW recirc AOVs to have an open safety function. I was not involved in those discussions. Lori Armstrong was involved in making the decision. We had one short

(-1 minute) discussion in the hallway regarding the topic. There was an August 12 letter (NOV response NRC 2002-0068) signed by Warner deciding to upgrade the safety function to open, but the recirc line doesn't have to be available. That didn't make sense to the engineers. I was told we needed to do the mod before the NRC inspection (9/23/02). Operations raised concerns about not declaring the AFW system inop when the line was taken OOS. Further review resulted in linking AFW 28

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines operability to the availability of the recirc line. I was not involved in any discussions until after the letter went out.

If a physical change to the plant wasn't needed, the upgrade probably wouldn't have been done via a mod. I understood that the mod involved both a physical change and a function change. I considered that the mod impacted AOV testing (IST program),

and that the operability/availability decision required a change to the Tech Spec basis.

In retrospect, I should have looked at more. I didn't see a problem with the RO at that time. I was under the gun to get the mod done. I also had to develop a contingency mod for the reactor head inspection. Fred Cayia told me the AFW mod was more important.

2. What direction did you get from your supervisor (Dick Homak)? Was there any pre job brief or coaching?

He told me its a lot to do, and if I can't do it to let him know and he would get someone to help. That's his management style, and its OK with us (his group).

3. What is your recollection regarding the last AFW RO mod (MR 99-029*C)?

It was released after the NRC Red Finding came to light. It was a copycat mod. I was the reviewer on the other ones. I co-signed as preparer for the safety evaluation with Alex because he wasn't qualified.

4. What do you remember about any discussions on plugging of the orifice?

The evaluation stated that the flow path made it difficult to plug. That was taken from the vendor information. I wasn't involved with the MSSM discussion for the evaluation. The fact that I retained was that the orifice could pass SW without plugging.

5. The safety evaluation seems to have conflicting information regarding the importance of the recirc line.

The wording in the SE and SCR does not mean that the line or RO has a safety function, but that it has a design function. The DBD was wrong. It was marked up to change the function of the RO to a non-safety function. That change was never made because the decision to upgrade the AFW recirc AOVs to have an open safety function changed what the RO needed to do.

6. Did you make any connection with the upgrade of the recirc line to the RO mod that was still being installed?

No I didn't. The RO mod had already been done 3 times - it was a no-brainer.

29

Possible Common Mode Failure of RCEO00191 Aux FeedwaterRecirculation Lines

7. The NRC Red Finding had implications regarding assumptions about only needing forward flow to the S/Gs - were you aware of this?

I was aware of the finding and it's implications. I didn't relate it to the SCR for the RO mod. I did the SCR and moved on to other work.

8. Was it appropriate to do this upgrade mod as a plant minor change?

The paper work is the same, only it doesn't go to EAC. EAC isn't being used much now. This was another mod were I had to drop everything to do it. Most of my mods in the last year have been that way - AFW mods, SI mods, reactor head mod.

9. Were you concerned with the level of independence for the RO mods, since you were the preparer on one and reviewer on others?

Some, but we have a small group - independence is a luxury. Being the tech reviewer makes me the best person to prepare the last RO mod.

10. Is there anything else you think I should know that is relevant?

Like I said before, everything is hurried and sporadic in nature, which is difficult to work with. If we had more time then there is a higher probability we may have caught this - I was uncomfortable with this.

11. Did you raise this concern with anyone?

I talked about it with some people (other engineers). I did not raise it officially with my supervisor.

Design Engineer 3 The mechanical design engineer interviewed stated that his involvement with the MR 99 029* modifications started with performing the technical review for the last modification being installed (MR 99-029*C). He also aided with the installation of MR 99-029*A/B when additional holes had to be added in the trim assembly to obtain the required flow in the recirculation lines for the P-38A and B auxiliary feedwater pumps.

The engineer stated that the issue of plugging was that if the orifice would plug it would be good because more water was supplied to the steam generator, if the orifice was to plug. This was the accepted site position on the issue at this time. He also stated that he believed from a legal stand point the site was better off if the AOV didn't open because the plant was designed for forward flow only and recirculation flow only lessened the flow that could be supplied to the steam generators. It was stated that the Engineering Director at the time MR 99-029*A/D was being installed absolutely refused to consider the need for the recirculation line. The engineer also stated that there was a 1985 letter to the NRC stating that even though we tested the AOV in the recirculation line in both the open and closed direction the valve had no safety function in the open direction.

30

Possible Common Mode Failureof RCEOO0191 Aux FeedwaterRecirculation Lines The mechanical design engineer stated that he had no direct involvement in modification MR 02-029. The design review for this modification was after the root cause was performed and before the regulatory conference for the red finding associated with auxiliary feedwater. He stated that he felt that inadequate rigor was applied to the modification, that the time line was too short. He felt that the message was "Pull the check valve out and call the line safety related".

The design engineer stated that the recirculation flow was not an acceptance criterion for pump operability. This was confirmed by reviewing the procedure revisions for IT 10.

The recirculation flow was for information only and was not an acceptance criterion for operability until September 9, 2002. He felt that the site upgraded parts/pieces but not the whole system when deciding to assign a safety function to the recirculation lines. As part of the upgrade process, the backup pneumatic supply to the AOVs was upgraded to safety related. The engineer stated that he originally wanted to install this modification (MR 01-144) as safety related and that somewhere above the acting Engineering Director this proposal was not approved. This occurred in January 2002. Though the engineer disagreed with this position it was consistent with our understanding of the regulatory requirements and he could not argue the point.

Design Engineer 4 (Independent Reviewer of MR 02-029)

Mechanical Design Supervisor When asked about the safety evaluation for MR 99-029*A/B, the supervisor stated he believed that the wear ring would grind up the particles, but the MSS Chairman did not accept this argument. Therefore, the evaluation was changed to a less technical approach.

This was based on not needing the recirculation line because of the AOV failure mode.

He stated that the safety screening done for MR 99-029*C/D had similar thoughts but was stated somewhat differently. He did not think the orifices would plug and not requiring the recirculation line was an additional conservative position.

For the MR 02-029, he said there were a lot of other discussions on upgrading the line that he wasn't involved in. He was given the modification and had to rationalize how to do it. He recalled that the modification to install the backup pneumatics was originally going to be safety related, but he was told not to make it safety related. Later that was changed to be safety related to support the open safety function. He was given verbal directions for MR 02-029, and then had internal Engineering discussions to clarify the design specifics. He said there was nothing in the licensing basis review that pointed them to look at past decisions.

When asked if there was anything in the modification process that initiated a review of a modification that had been open for a long time pending installation, he said there wasn't.

He said the 50.59 for old open modifications were reviewed with operations (on an honor system) to see if anything had changed..

31

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines AFW System Engineer 1 The engineer that had AFW system responsibilities at the time of MR 99-029*A/B was interviewed. He was asked when did the plugging issue get raised, and he recalled that it came up late in the design process. He could not recall if he was the one that brought it up. He said in the first version of the safety evaluation they had included the results of samples of service water taken from the AFW suction and compared it to the orifice size.

Staff (MSS) didn't like that and the Operations Manager said it doesn't make a difference because the AFW recirc AOV goes closed. He said he believed that the build up on the orifice would be gradual, not acute, but staff wanted a yes/no answer. The position suggested seemed conservative and he had no problem with the change.

When asked how the orifice option was selected, he said RCE 99-081 concluded that the existing orifice was undersized, but an orifice with more stages wouldn't fit easily in the existing space. An isolation valve would have to be moved and that would involve a freeze seal. He wasn't sure how the choice was made between the remaining options. He thinks it may have been because the channel stream orifice was more likely to knock down the noise problem.

He stated that he did not make a connection between the outcome of the AFW Red Finding (importance of proper recirculation line operation) and the last RO modification being installed.

AFW System Engineer 2 Fire Protection Engineers Two engineers with fire protection responsibilities were interviewed. One of the individuals participated on the modification design team for MR 99-029*A/B and the other participated on the design team for MR 99-029*C/D.

The engineer on the design team for MR 99-029*A/B stated that there was little if any discussion of orifice plugging associated with their design. He personally felt that plugging was not an issue with the proposed modification. He stated that the installation of the orifices was a non-outage work package fire protection issue and not an Appendix R operability issue.

He also stated that he never saw the safety screening or evaluation for the modification.

The only information he would review was the information in the detailed design description and the fire protection checklist contained in the modification package.

The engineer stated that the orifice was an integral part of the piping system. To his knowledge, the change in orifice design did not change the fit/form/function of the component. To his knowledge the orifice remained a passive component in the system and did not require a QA code for Appendix R or fire protection. He also stated that he would like to see Appendix R and fire protection broken up into their own separate QA codes.

32

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines Discussions with the second fire protection engineer (MR 99-029*C/D) revealed the following. It was assumed that the orifice was a piece of pipe. He had no idea that it could plug and nothing in the design description told him other wise. "The orifice worked yesterday and it will work tomorrow unless someone tells me different and no body did."

The air accumulators used to open the recirculation line AOVs are installed to buy the site time to manually gag open the valves and make AOP 10A work, that is why there is a QA code 20 associated with the valves and air tanks. We do not need to manipulate the orifice therefore there is no QA 20 code associated with them.

MSSM Chairman When interviewed, the MSS Chairman stated that the purpose of the MSS was to review safety evaluations and to either concur or disagree with the conclusions reached. It was not their responsibility to review the design package. The individual stated that he has not (at least recently) read the procedure governing the functions of the MSS. He stated that he is not qualified to perform safety evaluations and does not know if anyone on MSS is qualified to perform safety evaluations. It was stated that the MSS does not take a vote on the safety evaluation but a consensus is reached. If any member of the MSS has a strong dissenting opinion, the individual has the right to bring it to the attention of the Plant Manager. He stated that this has happened in the past.

The safety evaluation (SE 2000-0055) for modifications MR 99-029A/B stated the orifice might plug if service water was used as the suction source. The staff did not believe that the pump was capable of grinding any sand/silt into small enough particles to allow them to flow through the orifice. The staff member stated that staff recommended that service water be sampled to determine the size of the grit entrained in it. [It should be noted that this was performed prior to the safety evaluation being presented to MSS.] He believes that Engineering's opinion was that the orifice would not plug but did not present a conclusive argument to back the opinion.

Staff stated: there was no safety function for the recirculation line; they were aware that instrument air was needed for the recirc line AOV to work and instrument air is not safety related. Therefore, there was no guarantee that the AOV in the recirculation line would open. Consequently, since you could not rely on the AOV to open to provide a recirculation flow path, it did not matter if the orifice plugs and prevents recirculation flow. It was also stated that MSS has been criticized in the past for getting too involved in the details associated with safety evaluations; therefore, they didn't require this issue to be resolved operationally, nor did they assess the installation of a "filter" in the line.

Operations Representative An interview was conducted with the Operations Department Auxiliary Feed Water System owner. He stated that he has been the owner of this system for in excess of four years.

33

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines When questioned about his involvement with modification MR 99-029*A-D, he made the following statements. He was unaware of any consideration as to the size of the service water strainer mesh when selecting the restricting orifice. He also stated that he was not aware of the size of the flow channels in the restricting orifice until November 2002. He does not remember seeing the safety evaluation or screenings associated with the implementation of this modification.

The Operations system owner was aware that an operability determination (OD 99-139 1) had been written on the Auxiliary Feedwater System when weld cracking occurred on the recirculation lines. However, there were no compensatory actions required by this OD.

There is no routine call up for review of operability determinations; therefore, the issue of the need for the recirculation line to be operable when throttling flow to the steam generators was lost. The system owner was also aware that there was a IOCFR50 App. R function for the air-operated valve in the recirculation line. He however, did not associate this requirement with the need for the recirculation line to be operable for the pump to be able to perform its safety function.

It was stated that per procedure, any time that the indicated recirculation flow did not meet the minimum requirements stated in IT 10 the Auxiliary Feedwater Pump was inoperable. This requirement was instituted when the flow meters were installed on the recirculation lines in the early 1990s. (As stated above in the Design Engineer 3's interview, a review of IT 10 procedure revisions shows that recirculation flow was for information only until September 2002.)

Modification MR 02-029* was initiated to remove the internals of check valve AF-1 17 and give the recirculation a safety related function. Operations involvement was to identify and implement any necessary procedure changes to implement this modification.

It is his understanding that no one performed a comprehensive review of the Auxiliary Feedwater system or the modifications that had taken place to that system as part of design review of MR 02-029.

Thermal/Hydraulic Design Supervisor He recalled that the issue about upgrading the AOV and recirc line came up during the NRC SSDI time frame, and he was only briefly involved. He participated in a brainstorming session on what would have to be done to upgrade the recire AOVs to have an open safety function. He believed that upper management made the decision.

On about 8/19/02, he was assigned oversight of the upgrade modification. He said the schedule was very pressured with several aspects of the normal design development process being circumvented. The design engineer had some reservations with the modification scope and direction. This is contrary to normal practice where the design engineer evaluates various options and chooses an approach. He recalled that NMC Corporate Staff were brought in to perform an independent critique of the proposed changes out of concern for the rushed and extraordinary circumstances. During the course of the modification, at least one major scope change was made from simply 34

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines upgrading the open function of the AOV and removing a possible common mode failure (AF-1 17), to upgrading the line to pass flow.

Acting Engineering Director Site Vice President Industry and Station Operating Experience CAP013812 (CR 01-1259) Foreign Material Exclusion (FME) for Aux Feed Work EPIX - None INPO OE - PBNP OE13861 on Foreign Material Exclusion (FME) for Aux Feed Work Evaluation Methodology & Analysis Techniques The primary analytical techniques used in this investigation were interviewing, document review, and barrier analysis. A timeline of the event is presented in Attachment A, and the barrier analysis is presented in Attachment B.

DataAnalysis Summary

[Summarize Data Analysis and Facts in Orderto Draw Conclusions to Support Failure Mode Identification]

FailureMode Identification

[FailureMode IdentificationMust be Supported Through Facts and ConclusionsDrawn From DataAnalysis.]

[IM03] [Reflex - InappropriateAction (Inadequate Work Practice) Taken Based IUpon the Wrong Instincts.] _I

[BriefDiscussionof FactsSupportingFailureMode #1]

[POll I [Insufficient Detail - InadequateGuidance Containedin Procedure.]

0 [BriefDiscussion of FactsSupporting FailureMode #2.1 VII. Root Causes & Contributing Factors 35

Possible Common Mode Failureof RCEOO0191 Aux Feedwater Recirculation Lines

[BriefNarrativeof Conclusions]

[Identify specific Root Cause(s) and Significant ContributingCause(s)]

Conclusions Root Cause Contributing Causes Viii. Corrective Actions

[Require ComretiveActionsforEach Root Cause (CorrctiveActions to Prevent Recurrence) andSignificant ContributingFactors. Corrective actionsshould ident01 responsibk workgroup, actionpriority,and anticipatedcompletion date. Identif4 any Conrective Actions Which Have Already Been Implemented]

Corrective Actions to Restore (broke - fix)

CA #1 Responsible Group: [GroupName], Priority: [#], Completion Due Date: [Date]

1. CA026900: Ensure that the PRA work to evaluate the risk significance of this potential common mode failure is completed expeditiously. Priority 2, Due Date 1/29/03.

2. CA026901: Review for impact on aux feed pump unavailability for NEI cornerstone indicator. Priority 3, Due Date: 12/20/02.

Interim Corrective Actions (mitigation)

CA #1 Responsible Group: [GroupName], Priority: [#], Completion Due Date: [Date]

1. OPR00003 1: OPR requested to strengthen basis of Aux Feed Recieculation Lines, Priority 2, Due Date: 10/30/02: Completed, 10/31/02.

2. CA026897: Write and hang temp info tags on the pump hand switches, requiring that a minimum forward aux feed flow of greater than 50 gpm (P-38A/B) or 75 gpm (1/2 - P29) to be maintained, if not, stop affected pump; brief the Ops crews on the issue, Priority 2: Completed, temporary information tags written and place next to AFW flow indicators 10/30/02.

3. CA026898: Make changes to affected AOPs, EOPs, and other critical procedures to ensure that minimum forward flow is maintained, or the affected pump is stopped Priority 2, Due Date 11/11/02: Completed: 117/02.

4. CA026902: Redesign the recirc line orifices to make use of an orifice design that has aperture size >1/8", Priority: 1, Due Date: 11/22/02.

36

Possible Common Mode Failureof RCEOO0191 Aux FeedwaterRecirculation Lines

5. CA026908: Complete the interim administrative controls. Operator training and EOP and AOP changes to establish appropriate operator guidance for all accident sequences of interest. Priority 2, Due Date: 11/27/02.

6. CA026909: Independently evaluates the procedure changes being made to the EOPs and AOPs and assess their adequacy to provide adequate operator guidance.

Additionally, evaluated the adequacy of the temporary information tags and objectively evaluate our decision to declare the AFW pumps operable after taking interim actions to address this issue on Tuesday, October 29, 2002. Priority 2, Due Date: 11127/02.

7. CA0269 10: Independently evaluate the briefings and training conducted or planned for operating crews. We would like an objective look to ensure that the training was adequate and well documented. Priority 2, Due Date: 11/27/02.

8. CA02691 1: Evaluate hydraulic system response to determine if the orifices could become plugged. Priority 2, Due Date: NONE this item is being worked, to be updated by T Kendall

9. CA026912: Develop a test plan to evaluated plugging of a spare orifice. Use the vendor utilized by Kewaunee to evaluate the AFW suction orifice issue as appropriate. Priority 2, Due Date: NONE

10. CA026914: Evaluate the auxiliary feedwater system to determine the sources and quality of potential corrosion products. Priority 2, Due Date: 1/20/03

11. CA026915: Contact vendor of the orifice to obtain test data and other relevant information that could be used as input to the PRA work. Priority 2, Due Date:

11/4/02. Completed.

12. CA026917: Identify all other applications of this type of orifice in the Point Beach units and determine applicable operating history. Priority 2, Due Date:

11/15102. Completed

13. CA026918, Ensure that modifications to the orifice design or a replacement orifice are pursued expeditiously to resolve any questions. Priority 2, Due Date 12/13/02.

14. CA026958, Ensure compliance with procedure NP 10.3.7, "On-line Safety Assessment," while at elevated risk due to the potential for common mode plugging of AFW recirc orifices. Focus should continue with daily updates to Resolution Team Leader until interim corrective actions have restored on-line risk to a "green" condition. Priority 2, Due Date: 12/4/02.

15. CA026962, Perform an incident investigation. Scope is to conduct an investigation to collect facts surrounding the maintenance conducted on the P 38A motor driven AFW pump, the post maintenance testing, the evaluation of the corrosion products discovered in the recirculation orifice and conclude with the decision to declare the AFW pumps inoperable on Tuesday, October 2002. Use applicable portions of NP 5.3.3, "Incident Investigation and Post-Trip Review,"

for guidance. Priority 2, Due Date 11/21/02 Corrective Actions to Prevent Recurrence (CATPRs) 0 CA #1 Responsible Group: [Group Name], Priority: [#], Completion Due Date: [Date]

37

Possible Common Mode Failureof RCEO00191 Aux Feedwater RecirculationLines 0 CA #2 Responsible Group: [Group Name], Priority: [#], Compleuon Due Date: [Date]

IX. References

1. NP 7.7.10. Q-List Nuclear Safety Classification for Structures, Sustems, and Components. Revision 1.

2. NP 7.2.1, Plant Modifications, Revision 6

3. NP 7.2.1, Plant Modifications, Revision 10

4. NP 7.2.6, Engineering Change Process, Revision 2

5. NP 7.2.2 Design Control, Revision 5

6. NP 1.6.5, Manager's supervisory Staff and Qualified Reviewer, Revision 3

7. NP 7.7.12, Safety Related and QA Scope Classification Upgrade or Downgrade Process, Revision 1

8. OM 3.12, Control of Equipment and Equipment Status, Revision 8

9. OM 3.26, Use of Dedicated Operators, Revision 8

10. DG-G06, Guidelines for System, Component and Part Classification, Revision 6

11. PBF-1551, Q-List Classification Document, Revision 6

12. RCE 01-069, Increased CDF in AFW PRA Model Due to Procedural Inadequacies Related to Loss of Instrument Air, Revision 1

13. NUREG-0737, Clarification of TMI Action Plan Requirements

14. MR 97-038A/B, AFW Discharge Valves AF-4012/4019 Modification, Revision 0

15. 10CFR 50.59/72.48 Screening and Safety Evaluation, SE 97-085

16. FCR 97-033, Final Safety Analysis Report Change Request.

17. CR 96-317, FSAR Has Inaccurate AFW Flow Rate Assumptions

18. CR 97-3363, IST Program Design Basis for AFW Minimum Flow Recirculation Valves

19. CR 01-3654, Development and Revision of DBE-01 (AFW Design Basis Document) Appears to Have Been a Missed Opportunity to Identify a Design Weakness in the AFW System

20. CR 96-574, Alarm Indication Audio Levels Do Not Meet DG-G01

21. CR 97-0720, High Background Noise in Control Room

22. CR 99-2460, Control Room Noise and Pump Wear During AFW Pump Use During Plant Shutdown

23. CAP 001625 (CR 99-1391), SCAQ Action -Potential common Mode Failure Mechanism Affecting Welds in AFW Pumps

24. CAP025033 (CR 99-1844), Socket Weld Survey of Aus Feedwater Pump Recirc Piping

25. RCE 99-081, Socket-Weld Failures in Auxiliary Feedwater Pump Recirculation Piping, Revision 1

26. CAP 027667 (CR 99-1163), Water Spraying From a Cracked Weld On the Aux Feed Pump Recirc Line

27. CAP024973 (CR 99-1368), Weld Crack in Motor-Drive Aux Feed Pump Recirc Line

28. CAP 030045, Discrepancies in AFW PRA Model and Notebook.

29. FCR 98-068, Final Safety Analysis Report Change Request 38

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculation Lines

30. QCR 99-0115, Code Testing Conflicts with the Aux Feedwater Mini-Flow Recirc Check Valves

31. PBNP FSAR Section 10.2 (06/01)

32. FCR 99-049, Final Safety Analysis Report Change Request

33. DBD-01, Auxiliary Feedwater System, Final Draft 12/17/92

34. Validation of Design Basis Attributes Auxiliary Feedwater System, Sargent and Lundy, April 1993

35. DBD-01, Auxiliary Feedwater System, Revision 0

36. DBD-01, Auxiliary Feedwater System, Revision 1

37. SER 91-025 Safety Evaluation for JWP 88-099*A

38. IWP 88-099*A, Revision 0

39. NPL 97-0186, Licensee Event Report 97-014-00, Auxiliary Feedwater System Inoperability Due to Loss of Instrument Air.

40. NPM 93-0450, Auxiliary Feedwater Design Basis Document Review

41. NRC-92-085, In-Service Testing Pump and Valve Program Third 10-Year Program Safety Evaluation Report

42. NRC-30-130 (VPNPD-90-500), Inservice Testing Program for Pump and Valves, 12/21/90

43. NRC Letter to PBNP, Inservice Testing, Third 10-Year Program, Including Relief Requests (TAC Nos. M79386 and M79387), dated 4/17/92

44. NRC-93-031 (VPNPD-93-054), Pump and Valve Inservice Testing Program Third 10-Year Program Safety Evaluation Report

45. PBNP Inservice Testing Background Valve Data Sheet, page 21 of 96, Revision 3

46. PBNP Inservice Testing Background Valve Data Sheet, pages 11 and 21 of 95, Revision 4

47. PBNP Inservice Testing Background Valve Data Sheet, page 66 of 96, Revision 4

48. PBNP Inservice Testing Background Valve Data Sheet, page 67 of 96, Revision 3

49. Inservice Testing Program Relief Request No. VRR-28, Revision 1

50. Inservice Testing Program Relief Request No. VRR-28, Revision 2

51. Inservice Testing Program Relief Request No. VRR-28, Revision 3

52. MSS Subcommittee Meeting minutes, March 28,1991

53. PBM 92-0663, AFW Mini-Recirc Calve Gag Operation

54. SCR 98-1583, Inservice Testing Program Third Ten-Year Interval; Revision 5'

55. SCR 98-1732. Inservice Testing Program Background Document

56. Permanent Procedure and Procedure Revision Review and Approval for Revision 23 to IT 20, Auxiliary Feedwater System Check Valves and Flow Indicators

57. NPM 2002-0228, Designation of Backup Pneumatics for AFW Mini Recirculation Valves as Safety-Related

58. NRC 2002-0068, Reply to a Notice of Violation

59. NPM 2002-0594, Oversight of Plant Modifications

60. NPM 2002-0599, AFW Recirculation Line Orifice Evaluation

61. NPM 99-0749, Engineering Advisory Committee Meeting Minutes of 6130/99

62. EWR 99-031, AF Pump Recirculation Noise in Control Room

63. Memo to R Flessner from B Bosacki detailing discrepancies in PRA Notebook 5.9

64. PRA 5.9 Auxiliary Feedwater System Notebook, Revision 0 39

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines

65. FCR 01-050

66. FCR 02-002

67. SE 2001-0059, Operation of the Auxiliary Feedwater Pumps Without Cooling Water

68. SCR 2002-0007, Revise FSAR Table 9.6-1, "Essential Service Water Loads", To List The SW Supply To the AFW Pump Suctions

69. FAX to JP Schroeder from FLOWSERVE, 8/25/99

70. NRC Bulletin NO. 88-04

71. NRC-88-062, PBNP Response to NRC Bulletin 88-04

72. Kewaunee Nuclear Power Plant RCE 0 1-003, Auxiliary Feedwater Pump Suction Strainer Configuration Not as Expected

73. Kewaunee Nuclear Power Plant Calculation C 1197, Auxiliary Feedwater System Water Quality Following Seismic Event. Revision 1

74. Operability Determination, OPR 000031

75. Operability Determination, CR 99-139 1, Revision 0

76. Operability Determination, CR 1391/1844, Revision 1

77. Operability Determination, CR 1391/1844, Revision 2

78. Nuclear Oversight Quarterly Assessment Report of Point Beach Nuclear Plant, October 30, 2001

79. Operating Experience Event Number 455-010412-1

80. NRC Information Notice 98-45, Cavitation Erosion of letdown Line Orifices Resulting in Fatigue Cracking of Pipe Welds

81. NRC Information Notice No. 90-65, Recent Orifice Plate Problems

82. NPM 99-1323, IR 99-016 Evaluation

83. FAX to JP Schroeder from FLOWSERVE dated 12/09/99

84. Report From Laboratory to JP Schroeder, analytical results from Service Water Samples, dated, 10/19/99

85. Letter to P Gonzales, Flowserve Corporation from JP Schroeder, Auxiliary Feedwater Pump Operation With Service Water, 12/01/99

86. NRC Inspection Report 50-266/99013

87. NUTRK item IR 99-013

88. Unofficial Station Logs 10/23/02 Mid Shift

89. Unofficial Station Logs 10/24/02 Mid Shift

90. Unofficial Station Logs 10/24/02 Day Shift

91. Unofficial Station Logs 10/24/02 Swing Shift

92. Unofficial Station Logs 10/25/02 Mid Shift

93. Unofficial Station Logs 10/25/02 Day Shift

94. Unofficial Station Logs 10/29/02 Day Shift

95. Unofficial Station Logs 10/29/02 Swing Shift

96. Work order 0212107

97. Work order 9934148

98. Work order 0205651

99. Work order 9945610 100. Work order 9949098 101. Work order 9934148 102.

40

Possible Common Mode Failureof RCEO00191 Aux Feedwater RecirculationLines X. Attachments Attachment A: Event Timeline Attachment B: Barrier Analysis Attachment C: Charter 41

Possible Common Mode Failureof RCE00191 Aux FeedwaterRecirculation Lines Attachment A - Event Timeline Time Description 0400 P-38A OOS for scheduled maintenance. Enter TS LCO 3.7.5 for both units.

Day Shift Maintenance repaired pump seal per WO 0205651; replaced diaphragm on valve operator AF-4012-O per WO 9945610 (diaphragm fails drop test); and replaced valve AF-38 per WO 9949098 Evening Shift Diaphragm replaced on AF-4012 and passes drop test; performed ICP 06.086A per WO 0202494; Operations filled and vented pump suction line, casing, and discharge line per OI-62A 0110 Commenced IT-10 (Quarterly Test of Electrically Driven Auxiliary Feed Pumps and Valves) for P-38A. P-38A is still OOS per IT-10 and TS LCO 3.7.5 is not met for both uniti.

0159 Started P-38A per IT-10; suction relief valve observed to lift momentarily and then continue to drip 0212 Secured P-38A due to mini-recirc flow of 64.5 gpm. Required flow is 70 gpm.

-0215-0500 I&C contacted and flow transmitter was re-vented and recalibrated; ran P-38A and flow was still 65 gpm 0323 CAP029908 initiated for P-38A, MDAFW Pump having inadequate recirc flow during IT-10

-0400 Engineering contacted; Operations informed pump is OK and investigation will begin.

0440 Completed calibration of P-38A flow transmitter; instrument found to be in calibration

-0500-0700 Engineering postulated potential failure modes

-0630 BOP/NSSS Mechanical Systems Engineering Supervisor established as overall lead for the issue' 0830 Meeting with Engineering, maintenance, and Shift Manager to establish an action plan for the investigation

-1030-1100 Completed venting of the flow transmitter and AFW pump and started P-38A per OI-62A; P-38A was run on recirc only for about one minute - flow remained at -65 gpm on installed FE and -60 gpm on ultrasonic flow meter. Vibration data was taken and was normal.

-1100-1800 Maintenance removes RO-4008 and finds rusty looking material in openings of outer ring (24 of 54 holes had some degree of blockage),

disassembles, cleans and reassembles RO. Boroscope inspection performed of piping upstream of RO-4008 up to and including FE 4050A and found no material. Very small amount of debris retained.

1700 50.59 screening completed for one time temp change to IT-10 to lower acceptance criteria for data collection purposes only 42

Possible Common Mode Failureof RCEOO00191 Aux FeedwaterRecirculation Lines Time Description 1710 Engineering e-mails recovery plan for comment 1830 Meeting with Engineering, Maintenance, and Shift Manager to discuss plans associated with P-38A

-1900 - 2100 Ops clearing tags; Maintenance, Engineering and Operations met.

Ops reported tags cleared and system was filled and vented.

Maintenance requested high points to be re-vented prior to run. Ops re-vented suction (solid stream) and discharge (slight amount of air) 2115 Started P-38A per OI-62A 2117 Secured P-38A per OI-62A. Flow was 75 gpm. P-38A is now considered available for Safety Monitor and Maintenance Rule 2336 Commenced IT-10 Evening Samples taken of both CSTs and suction of AFW pumps looking for evidence of rust - results did not show evidence of rust.

0008 Started P-38A per IT-10.

0050 Stopped P-38A per IT-10 0129 P-38A is available for operation per IT-10 0155 Commence IT-10C for P-38A 0216 IT-10C is complete for P-38A only. P-38A RTS. TS LCO 3.7.5 now met. Exit TSAC entered at 0400 on 10/23/02 0309-0450 Test P-38B per IT-10 0512-0608 Test 1P-29 per IT-08A 0756-1121 Test 2P-29 per IT-09A 1030 Initiated ACE001023 to evaluate inadequate recirc flow on P-38A 1132-1206 Test 1P-29 per IT-08B 1222-1255 Test P-38B per IT-10D

-1600 Operations informs Engineering that short term action plan needs to

AF be in their hands by operability Fridayexisted concern this time. confirmed that no (11/1); atparticipants rWeekendW BOP/NSSS Mechanical Systems Engineering Supervisor considered the actions that might be required and formulated potential questions that needed answering and that would be discussed at the meeting he would hold on 10/28. The Operations Manger didn't ask for weekend evaluation of the issue because he didn't think it was realistic that the recently designed and installed orifices would be susceptible to luging from service water.

1100-1230 System Engineering Meeting to discuss AFW recirc orifice fouling issues. Potential for SW fouling was raised as a concern. Actual I plugging event was considered a one-time occurrence.

Afternoon Operations and Engineering management briefed on issue.

43

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines 0900 Meetinig with Engineering Director held to review issue. Consensus was that the system would become dysfunctional when taking suction from SW. Licensing was contacted and Operations was briefed.

0945 Management Meeting held to identify immediate actions needed 1015 EDT group began scoping out a modification to replace existing orifices 1027 All AFW pumps declared OOS due to recirc flow orifice issue 1110 Crew briefed on AFW recirc flow orifice issue 1210 Posted temporary information tags for all 4 AFW pumps for requirements to secure AFW if minimum flow is not maintained 1241 CAP029952 was initiated for Potential Common Mode Failure of AFW Recirculation Lines 1305 All 4 AFW pumps RTS based on compensatory measures taken to brief crew and post temporary information.

1410 Engineering Director forms RCE Team 1525 Engineering Director discussed need for operability determination with Licensing and Operations - concern is being in an operable but non-conforming condition 1530- 1630 Simulator runs made on comp measures to assist in risk determination 1700 SCR 2002-0458 for procedure changes accepted by Operations Manager 1711 NRC 8 hour9.259259e-5 days 0.00222 hours 1.322751e-5 weeks 3.044e-6 months notification (EN#39330) made regarding AFW system 1830 OPR requested for CAP029952 concerning common mode failure for AFW pumps. OPR is due at 1500 on 10/30/02 2030 Safety Grou Monitor in ut transitioned to Yellow for both units based on PRA 0700 RCE Team Meeting held to define roles and responsibilities 1030 Qualifications suspended for design work for 3 individuals 1100 Notified of NRC special inspection team 1250 RCE Charter approved 1530 RCE Team Meeting 1850 OPR com leted on AFW Pum s Day Shift SVP forms event resolution team 44

Possible Common Mode Failureof RCEOO00191 Aux FeedwaterRecirculationLines Attachment B - BarrierAnalysis Energy/Hazard Barrier Assessment Target Modification Procedure Good AFW design - no MR 99-029*A/B unknown failures modes Design Engineer No information in design description on potential to plug (DEl) RO; no discussion of orifice size being less than SW strainer size. Failed to apply FMEA to design.

Design Team Fire Protection and Operations members do not recall (FPEI, OP) seeing safety evaluation that had information on plugging. Inadequate communications among organizations.

Independent Independent reviewer was a co-preparer of the safety Review evaluation. Agrees that FMEA does not apply to (DE2) design. Lack of independence. .....

Supervisor Approves design Fails to ensure adequate FMEA Approval performed.

(MDS)

EAC Specific details of RO not presented to EAC. Plugging not discussed. EAC does not select specific option to be pursued. Inadequate communications within an organization.

Safety Procedure No unreviewed safety question Evaluation SE 2000-0055 Preparer (DEI/DE2)

Reviewer (AF SE1) 45

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines Energy/Hazard Barrier Assessment Target MSS JOSRC '

Modification Procedure Good AFW design - no MR 99-029*C/D unknown failures modes Design Engineer

(*C-DE2 *D DEI)

Design Team (FPE2, OP)

Independent Review

(*C-DE3

D-DE2)

Supervisor Approval (MDS)

EAC Safety Screening Procedure No unreviewed safety question SCR 2001-0981 Preparer (DE2)

Reviewer (AF SE I)

MSS JOSRC 1 f 1 1- t +

A. ____________ L_____________________________________ .1.

46

Possible Common Mode Failureof RCEO00191 Aux FeedwaterRecirculationLines Attachment C - Charter Root Cause Investigation Charter CAP029952 RCE000191 Issue Manager:

Jim Freels Problem Statement:

Discovery during the evaluation of CAP029908 (P-38A, MDAFW Pump had inadequate recirc flow during IT-10) that the recirculation line restricting flow orifices had become plugged during plant operation causing a reduced flow (but above minimum required) and that a potential existed for a common mode failure where all AFW pump recirculation lines could have restricted flow rates resulting in eventual pump failure.

Investigation Scope:

Determine the following:

"* Timeline of key events

"* The root and contributing causes of why the condition exists, including any potential human performance issues

"* Why the problem was not identified previously Make recommendations for:

"* Correcting the problem, including any remedial actions

"* Preventing recurrence of the problem

"* Applicability of the root cause to other areas (extent of condition), including verification that a safety-related AFW recirculation flow path exists for the postulated failure modes Team Members:

Team Leader - Richard Flessner, Engineering Processes Team Member - Kevin Bennett, Engineering Processes Team Member - Eric Schmidt, System Engineering Team Member - William Bosacki, KNPP Design Engineering Milestones:

Status Update - 11/4/02 Status Update - 11/11/02 Draft Report- 11/18/02 Final Report - 11/26/02 Approved: (oriainal signed by J. Freels) Date: 10/30/02 Jim Freels, PBNP Engineering Director 47

ML030800553

Text

Purpose:

Background

Navigation menu

Search