IR 05000498/1993038

Insp Repts 50-498/93-38 & 50-499/93-38 on 931018-22.No Violations Noted.Major Areas Inspected:Effectiveness of Licensee Actions to Improve Reliability & Testing Methodology of turbine-driven Auxiliary Feedwater Pumps

ML20058C421
Person / Time
Site:	South Texas
Issue date:	11/18/1993
From:	Johnson W NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV)
To:
Shared Package
ML20058C404	List: IR 05000498/1993038 ML20058C402
References
50-498-93-38, 50-499-93-38, NUDOCS 9312020451
Download: ML20058C421 (23)

Text

.

,

!

'

-

l

!

APPENDIX

'

'

U.S. NUCLEAR REGULATORY COMMISSION

REGION IV

NRC Inspection Report Nos.:

50-498/93-38 50-499/93-38

'

Licenses: NPF-76 NPF-80

,

Licensee: Houston Lighting & Power Company P.O. Box 1700 Houston, Texas 77251 f

Facility Name: South Texas Project Electric Generating Station (STPEGS),

Units 1 and 2

,

Inspection At: Matagorda County, Texas Inspection Conducted: October 18-22, 1993 Inspectors: Mark A. Satorius, Project Engineer, Project Section A, Division of Reactor Projects

// /8 M Approved:

/

. D. Johnson /, Chief, Project Section A Da'te /

.

,

Inspection Summary l

Areas Inspected (Units 1 and 2): Routine, announced inspection to determine l

the effectiveness of the licensee's actions to improve reliability and testing methodology of turbine-driven auxiliary feedwater pumps (TDAFWPs).

l Results (Units 1 and 2):

The preventive maintenance (PM) program has been re-written, with

,

!

enhanced maintenance procedures that incorporated the latest revisions

!

of the turbine, governor, and trip / throttle valve vendor manuals (Section 2.1.1).

Enhancements to the condensate removal system have been completed and l

'

tested to ensure adequate operation, and monitoring instrumentation installed to alert operators and engineers of potential system degradation (Section 2.1.2).

The licensee has established a good mechanical maintenance training

program that includes the latest PMs and a training TDAFWP with the i

installed governor, trip / throttle valve, overspeed trip mechanism and l

associated linkage (Section 2.1.6).

9312020451 931122 PDR ADOCK 05000498 G

PDR

- - - -

-

-

_ _ = _ _. _

_.

__

_

-

_

_.

_ _ _

'

.

'

i i

!

..

i l

i i-2-l h

,

-

Acceptable repairs have been accomplished on both unit _'s TDAFWPs to

!

adequately address material deficiency issues identified in NRC f

l Inspection Reports 50-498/93-05; 50-499/93-05 and 50-498/93-07; l

50-499/93-07 (Section 2.1.7).

j

"

i

.

The licensee's surveillance testing procedures have been revised _in -

!

order to address the testing inadequacies identified in NRC Inspection i

j Reports 50-498/93-05; 50-499/93-05 and 50-498/93-07; 50-499/93-07.

Specifically, these enhancements should provide assurance that future

,

}

TDAFWP deficiencies that could degrade reliability will not be masked by

,

,

an inadequate surveillance testing program (Section 2.2).

!

~

Readiness Review Committee activities were conducted in a thorough

!

Division Managers that constituted the TDAFWP Readiness Review i

manner.

Committee were appropriately critical and circumspect with respect to

!

j system status and the acceptability of proposed' deferral of maintenance'

act;vities (Section 3.2),

Pending the satisfactory completion of MODE 3 testing of the Unit 1

TDAFWP, the inspector concluded that no further review was required-

'

l prior to the restart of Unit I and that Restart Issue No. I could be considered resolved (Section 4.11).

'

!

STPEGS management's receptiveness to identifying and correcting problems

'!

with respect to the TDAFWP issues, were considered to have improved

!

.

since the original problems were identified and documented in NRC i

'

Inspection Reports 50-498/93-05; 50-499/93-05 and 50-498/93-07; 50-499/93-07 (Section 6).

l l

Summary of Inspection Findinos:

- 1 Violation 498;499/9305-05 war. statused and remained open pending the

+

completion of TDAFWP testing on Unit I during MODE 3 (Section 4.1).

Inspection Followup Item (IFI) 498;499/9331-06 was statused concerning

.

TDAFWP issues and remained open (Section 4.2).

IFI 498;499/9331-07 was statused concerning TDAFWP issues and remained

.

open (Section 4.3).

IFI 498;499/9331-08 was statused concerning TDAFWP issues and remained

'

open (Section 4.4).

IFI 498;499/9331-09 was statused concerning TDAFWP issues and remained

.

open (Section 4.5).

IFI 498;499/9331-10 was statused concerning TDAFWP issues and remained

open (Section 4.6).

.

-

- - -

m--

mi

-

,ye_,

ef

..e

.

-

s

.--y-

- _ _m4ym-p c,__

=

. _ _

.

(

,

.'

-3-IFI 498;499/9331-35 was statused concerning the TDAFWP system

certification and remained open (Section 4.7).

-

IFI 498;499/9331-50 was statused concerning the installation of

permanent flow instrumentation on the AFW system and remained open (Section 4.8).

IFI 498;499/9331-52 was statused concerning the installation of

permanent flow instrumentation on numerous safety-related systems and i

remained open pending followup inspections on other systems

.

(Section 4.9).

i Licensee Event Report (LER) 498/93-07 remained open pending the

completion of TDAFWP testing on Unit I during MODE 3 (Section 4.10).

'

LER 499/93-04 remained open pending the completion of the corrective

action on the problems identified with the startup steam generator feedwater pump, this item will be closed (Section 4.11).

Example 3 of Violation 498;499/9235-02 was closed (Section 5.1).

>

Violation 498;499/9305-04 was closed (Section 5.2).

e Violation 498;499/9305-07 was closed (Section 5.3).

.

IFI 498;499/9331-43 was closed (Section 5.4).

.

IFI 498;499/9331-70 was closed (Section 5.5).

• IFI 498;499/9331-71 was closed (Section 5.6).

.

IFl 498;499/9?31-72 was closed (Section 5.7).

.

Attachments:

Attachment 1 - Persons Contacted and Exit Meeting

Attachment 2 - Unit 1 Turbine-Driven Auxiliary Feedwater Pump Schematic

.

,

.

.

--

.

..

-..

.-

. - - - -

-.

,

.

.

'

,

.

c.

i-4-

.

DETAILS

1 BACKGROUND Both units at STPEGS were shut _down in early February 1993, and remain i

shutdown as a result of numerous broad scoped problems identified by the NRC

and the licensee.

,

,

NRC_ Inspection Report 50-498/93-31; 50-499/93-31, issued on October 15, 1993, identified 16 Restart Issues that required resolution prior to the restart of.

_

Unit 1.

In addition to these Restart Issues, a number of items related to

!

'

these Restart Issues were identified. The. purpose of this inspection was to determine the licensee's effectiveness _in resolving' Restart Issue No. 1,

" Turbine-Driven Auxiliary Feedwater Pump Reliability and Testing Methodology,"

r

-

and to establish a basis for concluding that this Restart'_ Issue has been l

!

adequately resolved by the licensee.

2 ACTIVITIES ASSOCIATED WITH IMPROVING TDAFWP RELIABILITY (92720)

2.1 Resolution of Identified Material Deficiencies During the Augmented Inspection Team (AIT) inspection (refer to NRC Inspection

.

Rep::rt 50-498/93-07; 50-499/93-07) conducted at STPEGS following the repeated i

overspeed trips of both unit's TDAFWPs, the following' root causes were i

identified:

,

For the Unit 1 TDAFWP, the AIT considered the misadjustment of the -

,

governor valve to be the most probable cause of the overspeed trips.

This misadjustment, which occurred during the previous plant refueling-

'

outage, reduced the governor.'s ability to control turbine speed.

,

For the Unit 2 TDAFWP overspeed trip, the root cause was~ determined to

be collected condensate upstream of the-trip / throttle motor-operated

!

valve (MOV) 514 caused by an incorrect valve lineup' combined with an l

inoperable or degraded steam trap in the drain line for the steam admission line. This caused a slug of water to enter,the turbine and resulted in a turbine overspeed.

Contributing causes for these trips that affected both TDAFWPs included:

.

The use of MOV-514 as the steam admission valve in. lieu of MOV-143, a containment isolation valve located upstream of MOV-514, and originally designed to serve as the steam admission valve. This usage created an opening time coordination problem between MOV-514 and the governor valve.

It also created additional demands on the steam admission line drain system due to condensate buildup in this line.

l t

l i.

-

- -

-

c

,

_

..

.

-

..., _. _ _

_

__

__

.

-

.

.

i I *

L.

l-5-Excessive seat

.ge past MOV-514 which had the potential of reducing the gou or control margin and the fcrmation of j

condensate in the 1DAFWP turbine casing.

<

,

The licensee had addresse'd each of these material deficiencies, in addition to.

performing additional _ enhancements to both TDAFWPs. These enhancements i

included installing monitoring systems that were designed to provide

indication of system performance in order to alert operators and _ engineers of j

degraded performance.

t

!

!

2.1.1 Governor and Governor Valve Refurbishment, Adjustment, and Maintenance

l Procedure Improvements

'

'

Following the problems with overspeed trips of both TDAFWPs, the licensee

removed the governors, Woodward Model MSD 4X6X9C PGA, from both pumps and shipped them to the vendor for refurbishment. The Unit I work was documented

.

t l

under Service Request (SR) AF-1-172874 and Unit 2 under SR AF-2-161263.

,

l Following the vendor work, the governors were returned to the licensee and.

-

installed on their respective TDAFWFs. A number of problems were encountered

'

during the initial testing of the Unit 2 TDAFWP in February 1993'(refer to NRC l

Inspection Reports 50-498/93-07; 50-499/93-07 and 50-498/93-11; 50-499/93-11).

-

However, these problems were resolved by the installation of stronger buffer -

l l

springs bt a vendor representative and by the licensee discovering and subsequently removing a shipping plug that was blocking an unused instrument air port. Following these actions, the Unit 2 TDAFWP governor performed

i acceptably. Acceptable governor performance was confirmed by Unit 1 TDAFWP

testing in August 1993 (refer to NRC Inspection Report 50-498/93-24;

50-499/93-24).

'

!

In addition to the governor refurbishment, the licensee performed maintenance

-

on both unit's governor valves. The AIT determined in February 1993, that a

,

misadjusted governor. valve was the root cause of-the overspeed TDAFWP_ trips

on Unit 1.

A review of Preventative Maintenance Work Instruction

~

MM-1-AF-89003576, the procedure that performed governor adjustment during the Unit 1 outage (mid-September to late December 1993), revealed that_.there was insufficient quantitative and qualitative acceptance criteria to demonstrate that the governor valve would perform satisfactorily in service (refer to NRC

'

Inspection Report 50-498/93-05; 50-499/93-05).

As a result of the identified ~

deficient maintenance procedure, the licensee developed Preventative l

Maintenance Work Instructions MM-1-AF-93000609 and MM-2-AF-93000610 for Units-i

' and 2, respectively. These maintenance procedures were developed with the j

assistance of the TDAFWP vendor, and their purpose was to provide specific l

guidance for the disassembly, inspection, lubrication, and reassembly of the TDAFWP's governor valve. The inspector reviewed both these maintenance procedures and determined that they provided sufficient direction for_ governor valve disassembly and suitable acceptance criteria for the adjustment of the valve.

-

.

.

. -..

. - - -

.. -. >

.

.

.

-6-In addition to these maintenance procedures, the licensee has

,

developed / revised the following maintenance procedures:

i Procedure Procedure Purpose

]

OPMP04-AF-0002 Major TDAFWP Turbine Maintenance

~

Associated with Refueling Outages OPMP04-AF-0003 TDAFWP MOV-514 Maintenance Guidance MM-1-AF-90002003 Mechanical Maintenance Guidance for the Turbine Overspeed Test-Unit 1

'

MM-2-AF-90002004 Mechanical Maintenance Guidance for the Turbine Overspeed Test-Unit 2 MM-1-AF-93000907 Mechanical Maintenance Guidance for Lubrication, Inspection, and

Adjusting MOV-514 and Trip l

Mechanisc-Unit 1 MM-2-AF-93000906 Mechanical Maintenance Guidance for i

Lubrication, Inspection, and Adjusting MOV-514 and Trip Mechanism-Unit 2 MM-1-AF-93001284 Mechanical Maintenance Guidance for Acoustic Te; ting of the Leakby MOV-514-Onit 1 MM-2-AF-93001285 Mechanical Maintenance Guidance for Acoustic Testing of the Leakby MOV-514-Unit 2 MM-1-AF-93000609 Mechr >. 31, 'ntenance Guidance for Lubr-2 tion and Inspection of TDAFk Jni' 1 MM-2-AF-93000610 Mechanical Maintenance Guidance for Lubrication and Inspection of TDAFWP-Unit 2 Concurrent with these procedure updates, the TDAFWP vendor manual was significantly revised and updated utilizing the following design change notices (DCNs) and plant change forms (PCFs):

DCN MM-1461 DCN MM-1454 PCF-189795A DCN MM-1417 PCF-158219A PCF-158008A DCN MM-1407 PCF-169758A DCN MM-1412 PCF-179428A

- _.

.-

.

-

-_.

..

-

,

_

.

.i

,

,

-

i

-7-l 2.1.1.1 Conclusions l

The inspector reviewed these procedural updates and determined that collectively, they represented a significant improvement of the leve of-i guidance provided to maintenance craft.

Further, the inspector concluded that-

!!

future maintenance activities conducted on the governor and governor valve

!

would, at minimum, have adequate procedures available.

2.1.2 Steam Trap and Condensate Drain System Improvements (refer to Attachment 2)

l i

The AIT concluded that the root cause for the overspeed trips of the Unit 2

TDAFWP was collected condensate upstream of MOV-514. caused by an incorrect.

t valve lineup combined with an inoperable or degraded steam trap.in the drain j

-

line for the steam admission line.

'

,

In an effort to solve the known problems and prevent any future condensate.

j drainage problems with either of the TDAFWPs, the licensee performed many i

hardware enhancements.

-l i

t 2.1.2.1 Steam Trap Removal In order to preclude condensate buildup problems due.to steam trap failure or I

,

degradation, the licensee removed the steam traps from the condensate' drain

I

'

piping on both units and replaced the traps with spool pieces. This activity was performed in Unit I under SR MS-1-179511 and Unit 2 under SR MS-2-179512 and was completed on both units on February 12, 1993 (refer to-NRC Inspection.

Report 50-498/93-07; 50-499/93-07).

i

!

In addition to the installation of the spool pieces, the licensee revised the valve line-up contained in OPOP02-AF-0001,. " Auxiliary Feedwater," to require

.

that operators position Valves MS 515, 516, and 517 (the inlet, outlet, and l

bypass to the spool piece) in the open position when aligning the auxiliary

!

feedwater (AFW) for operation. This action was taken to ensure.that a direct

,

path to the condenser was available whenever the AFW system was required to be i

operable.

,

During the testing that was conducted on the' Unit'l TDAFWP in August 1993, the performance of the spool pieces and the valve line-up changes was:

.'

satisfactorily demonstrated by thermography and acoustical monitors.

,,

.-

-

-

...

--

- -. -.-

-

i

!

'

,

e

!

I-8-

,

2.1.2.2 Condensate Drain Enhancements The inspector reviewed and physically verified the installation of the following licensee installed enhancements to the Unit 1 TDAFWP condensate drain system:

Work Package Description AF-1-174401 Reroute Turbine Casing Drain Line

'

I MS-1-310259 Reroute Steam to Turbine Exhaust MS-1-310255 Reroute Drain to Room Sump AF-1-3102E4 Extend Turbine Casing Drain to Room Sump AF-1-310256 Split Governor Valve and MOV-514 Leak Off Drains AF-1-174398 Install Drain Hole in Condensate Orifice

,

PCF-189425 Provide Additional Drainage Enhancements l

l These enhancements consisted of providing M0V-514 leak off, governor valve l

leak off, turbine casing, and gland drains with a single, unshared path to either the TDAFWP exhaust stack or the room sump. This action was taken to

'

ensure that no two drain paths would share a common drainage route, an action that could potentially hamper free condensate flow, and to simplify any future i

troubleshooting of the condensate drain system.

The performance of these drain system enhancements was satisfactorily demonstrated by thermography and acoustic monitoring during testing conducted on the Unit 1 TDAFWP in August 1993.

Although all of these enhancements had yet to be completed on Unit 2, they were scoped and scheduled for completion during the present outage.

2.1.2.3 Condensate Drain Performance Monitoring Enhancements i

In order to provide operators and engineers a method of monitoring the performance of the TDAFWP condensate drain system, the licensee developed PCF 189776A. This modification involves moving Flow Orifice (F0) 7537E, which was currently located upstream of MS 515 and resistance thermal detector Temperature Element (TE) 7537, to a position between MS 515 and MS 516 and downstream of TE 7537. The modification would also install a heated junction thermocouple in the location that Flow Orifice 7537E was previously located.

This thermocouple would provide an input into the emergency response facilities data acquisition and display system (ERFDADS) computer and would i

provide operators an alarmed indication of whether the drain system was passing steam or condensate, based on the indicated temperature.

PCF 189776A also installed a second thermocouple downstream of MOV-514.

This thermocouple also inputted into ERFDADS and provided operators an alarmed indication of MOV-514 leakby.

The inspector considered the addition of this plant

_

._.

.

.

.

/

.

.

-9-l

!

modification to be a good enhancement, that could provide operators with an i

early indication of TDAFWP drain system and M0V-514 degradation. The i

degradation of these two TDAFWP components were instrumental in the overspeed i

trips of the TDAFWPs in February 1993.

The licensee will install this modification prior to restart in Unit 2, and j

plans to defer the work in Unit I until the next scheduled refueling outage in

approximately 18-months. The inspector was concerned that this valuable r

system monitor would not be available for essentially an entire fuel cycle to Unit 1 plant personnel and that TDAFWP degradation could occur without

,

'

operators and engineers being alerted. Subsequent conversations with the system engineer revealed that based on data recorded during the Unit 1 TDAFWP testing in August 1993, the presently installed TE 7537, which also provides a non-alarmed input to the PROTEUS computer, would be available to provide the system engineer a method of monitoring the performance of the TDAFWP drain

system. Although not located in the optimum monitoring position downstream of F0 7537E, this instrument would be capable of indicating if the drain system

was not passing steam and condensate to the condenser.

Further, Step 2.17.1 i

of the data sheet to the TDAFWP monthly operability surveillance

'

,

Procedure IPSP03-AF-0007, "Auy.iliary Feedwater Pump 14 Inservice Test,"

i required that operators record the temperature reading from TE 7537 prior to commencing the testing. The system engineer stated that since he was' required.

to review the monthly testing results, the review would constitute an adequate -

,

interim monitoring program for the performance of the TDAFWP drain system.

Based on the extensive hardware enhancements made to the drain system in i

conjunction with the monthly temperature monitoring program, the inspector concluded that Unit I drain system performance monitoring would be adequate i

'

until the permanent installation of PCF 189776A.

a

'

The inspector further questioned.the system engineer whether an interim program had been established for monitoring leakby of Unit 1 MOV-514 because of the deferral of the installation of PCF 189776A. As stated earlier, this-plant modification, in addition to monitoring the performance of the TDAFWP drain system, would provide an alarmed indication of M0V-514 leakby. The

'

system engineer provided the inspector with the acoustic data taken downstream of MOV-514 during the Unit 1 TDAFWP testing in August.1993. This baseline data was to be utilized in future acoustic testing..in' order to monitor the performance of MOV-514. The inspector reviewed MM-1-AF-93001284 and MM-2-AF-93001285, which were in draft, and determined that these semi-annual planned maintenance procedures adequately provided guidance for monitoring the leakby of both unit's MOV-514 valve. The system engineer stated that both of these procedures would be approved and in place such that the first maintenan.e requirement would occur six months following Unit I restart.

2.1.2.4 Conclusions The inspector concluded that adequate actions had been taken to enhance the TDAFWP drain systems and that appropriate monitoring programs were in place to provide a reasonable assurance that any future drain system inadequacies could be identified prior to TDAFWP system degradation.

i

-

_

_

-

.

_

-, _ _ _.. _ _ -. _ _ _. _ _. _...,, _ _,

-

-.

--

. - - -

.-

.

-

.

.

'

,

b

,

.

-10-J t

2.1.3 Resolution of MOV-514 as the Steam Admission Valve l

!

The AIT concluded that a contributing cause for the trips of both units TDAFWPs included the use of MOV-514 as the steam admission valve in lieu of l

M0V-143, a containment isolation valve located upstream of MOV-514, and

.

originally designed to serve as the steam admission valve. This usage created

.

an opening time coordination problem between M0V-514 and the governor valve.

In order to provide a greater margin for the governor valve to react to the start rate of the TDAFWPs, the licensee performed a modification to slow the

'

opening time of MOV-514. This modification was accomplished under SRs

AF-1-189788 and AF-2-189795 for Units 1 and 2,'respectively. The inspector reviewed both of these SRs and the 10 CFR 50.59 review of the modification.

l These SRs were identical, and involved changing MOV-514's motor-operated

,

actuator by replacing the motor pinion and worm shaft gears. The original

!

actuator gear ratio was a reduction of 47.85:1, which fully opened MOV-514 in approximately 10 seconds; the SRs modified this ratio to 90.5:1, which essentially doubled the opening time to approximately 20 seconds.

Actual test data collected during the TDAFWP testing conducted in February and

August 1993, indicated that the modification had.been successful; both governor valves were able to more effectively control.the rate of TDAFWP speed increase.

2.1.3.1 Conclusions

'

.

,

,

The inspector concluded that the modification to slow the opening times of both units MOV-514 valves had been successful and were an enhancement to the

,

ability of the TDAFWP to reliably start and operate.

2.1.4 MOV-514 Refurbishment I

The AIT concluded that excessi e seat leakage past M0V-514 was a contributing l

'

cause of the overspeed trips of both units TDAFWPs and had the potential of reducing the governor control margin and the formation of condensate in the TDAFWP turbine casing.

,

Following the TDAFWP overspeed trip events in February 1993, the licensee a

,

removed both unit's MOV-514 valves and returned them to a vendor,

'

representative for refurbishment. These activities were completed under SRs MS-1-189509 and MS-1-189510 for Units 1 and 2, respectively (refer to NRC Inspection Report 50-498/93-07; 50-499/93-07).

Following installation, these valves were tested during February and August

!

1993. Both valves performed acceptably, with the leakby within the acceptance i

criteria specified by the valve vendor.

In addition to the valve rework, acoustic data was obtained to provide baseline information to monitor the

,

future performance of the valves, as mentioned in paragraph 2.1.2.3.

The

I i

-, - - _. -.

.-

.

.. -

-.

.

... -. -. - -....

-. -. -

.

.

.

,

.

.

-11-

.

installation of PCF 189776A in both units should furtiier enhance the licensee's ability to monitor the performance of both MOV-514 valves.

2.1.4.1 Conclusions The inspector concluded that appropriate repairs had been made to both unit's MOV-514 valves and that adequate programs were in place to monitor valve performance.

2.1.5 Overspeed Tappet / Lubricating Oil Improvements A separate item, not addressed in the AIT, concerned an issue identified in NRC Inspection Report 50-498/92-35; 50-499/92-35, involving the licensee's

failure to take adequate corrective action to preclude recurrence of repeated problems with resetting the TDAFWP overspeed plunger following manual tripping of the pump.

The licensee determined that the cause of the problem was due to the type of lubricating oil being used in the TDAFWPs and defective overspeed tappet molded heads.

Prior to the NRC inspection documented by Inspection Report 50-498/92-35; 50-499/92-35, the licensee had used Mobil Vaprotec Light turbine oil in both TDAFWPs. This oil contained a rust inhibiting additive that at elevated temperatures (greater that 120of) would evaporate, creating a varnish-like substance that would coat portions of the TDAFWP's overspeed tappet causing the mechanism to stick. This condition did not preclude the TDAFWP from tripping on an actual overspeed condition, but increased the out-of-service time for the TDAFWP because maintenance personnel were required to clean the tappet to ensure proper operation.

Because both unit's MOV-514 valves were leaking-by, temperatures were exceeding 120*F and causing the varnishing problem.

As a result, when the TDAFWP tripped on an overspeed condition, or was routinely tripped following monthly operability surveillances, operators were not always able to reset the overspeed trip due to the mechanism sticking.

As a result of this condition, the licensee changed to Mobil DTE 797, a vendor approved oil that was compatible to Mobil Vaprotec, but did not contain the additive that could varnish onto components. The inspector reviewed all of the maintenance procedures listed in paragraph 2.1.1, and determined that all required replacement TDAFWP turbine oil was specified as Mobil DTE 797.

A second potential cause of the overspeed trip mechanism binding was determined to be the molded heads on some of the overspeed tappet assemblies.

l This potential condition was identified by the vendor, who informed the licensee. This issue involved some lots of tappet assemblies having a molded j

polyurethane head that could swell and cause tappet binding. The licensee's j

action to resolve this potential problem was to place all affected replacement i

tappets in the warehouse on hold, in addition to developing and revising maintenance procedures OPMP04-AF-0003, MM-1-AF-93000609, and MM-2-AF-93000610 to inspect for the absence of binding in the overspeed tappet assemblies.

The i

latter two maintenance procedures were conducted on a monthly frequency.

.

.

-

.

-

,

.

-12-2.1.5.1 Conclusions The inspector concluded that the licensee had taken adequate measures to resolve the previously recurring problems with the sticking of the TDAFWP overspeed tappets.

2.1.6 Maintenance Training Improvements The inspector reviewed the licensee's activities associated with the development of a program to train mechanical maintenance personnel on TDAFWP turbine maintenance. The licensee had developed the program with the assistance of a visiting engineer from another utility. This visiting engineer had extensive TDAFWP experience and was active in the Terry Turbine User's Group that had been organized by several utilities to pool infc mation and share lessons-learned.

Lesson Plan MMT928.01LP, " Auxiliary Feedwater Pump Turbine Controls," and Lesson Plan MMT928.02LP, " Auxiliary Feedwater Pump Turbine Maintenance," were reviewed by the inspector.

Both lesson plans were extensive and well-prepared documents, with the former course of instruction being designed for 40-hours and the latter for 30-hours of classroom and laboratory type training. An

initiative taken by the licensee consisted of purchasing a Terry Turbine and installing a governor, MOV-514, and associated linkage that match the as-installed equipment in the plant. This mockup was used in conjunction with the course of instruction and permitted trainees to disassemble the turbine and all associated support equipment and practice corrective and preventive maintenance procedures. The inspector considered this training program to be excellent.

At the close of this inspection, the licensee had completed training eight

>

mechanical maintenance personnel, and planned to train an additional four during the coming year.

.

2.1.7 Conclusions on Identified Material Deficiencies The inspector concluded that the licensee had taken appropriate actions to resolve the material deficiencies identified by the NRC and the licensee,

'

developed adequate maintenance procedures and training, and established adequate programs to monitor and assess TDAFWP system performance.

2.2 Improvements to TDAFWP Testina Methodology

,

NRC Inspection Report 50-498/93-05; 50-499/93-05 determined that TDAFWP surveillance testing was not being performed consistently under appropriate pump standby conditions, such that degraded conditions affecting operability were masked. The inspector reviewed the initiatives that the licensee had taken to correct this identified problem.

2.2.1 Operability Surveillance Procedural Changes l

_ _ _ _

.

'

,

.

-13-The licensee revised IPSP03-AF-0007, " Auxiliary Feedwater Pump 14 Inservice Test," and 2 PSP 03-AF-0007, " Auxiliary Feedwater Pump 24 Inservice Test," for Unit 1 and 2, respectively.

These procedures provided detailed instructions to verify that the TDAFWPs were operable as required by Technical Specifications (TS) 4.0.5, 4.7.1.2.1.a.2, and 4.7.1.2.1.a.4.

The revisions incorporated changes that were intended to improve the surveillance methodology to ensure that consistent, reliable operability tests were performed on the TDAFWPs.

One of the specific findings of NRC Inspection Report 50-498/93-05; 50-499/93-05, was that TDAFWP testing was being conducted immediately following previous local manual or unsuccessful remote start attempts. The effect was that degraded TDAFWP conditions, in this case condensate buildup that would subsequently be carried into the TDAFWP and cause an overspeed condition, were masked because the unsuccessful start attempt had removed the discrepant condition by freeing the steam piping of condensate. The licensee's revision to the in-service testing requirements provided a period of time that operators would be required to allow the TDAFWP to return to normal, ambient, and standby conditions.

The time requirements delineated in step 2.17.2 of IPSP03-AF-0007 and 2 PSP 03-AF-0007 required that prior to conducting an operability test, a period of 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> must have elapsed since MOV-514 was last cycled open, regardless of whether the valve was opened while attempting a manual or remote start. The basis of this time interval was temperature data that the licensee gathered

,

during the Unit 2 TDAFWP testing conducted in February 1993, and confirmed during the Unit 1 TDAFWP testing in August 1993. During this testing, the licensee determined that all the drains associated with r a ving condensate from the turbine casing had returned to approximately ambient conditions within a two hour period. The data also revealed that the TDAFWP turbine casing required nearly 12-hours before returning to an ambient condition. The

,

inspector considered that for the TDAFWP system to be completely returned to an ambient and standby condition the turbine casing would be required to ambient temperature; however, the inspector concluded that it would not be unreasonable with the enhancements accomplished on the TDAFWP material condition, in addition to the system performance monitoring program that had been installed, the two hour wait period was adequate.

In addition, during testing of the both units TDAFWPs, condensate was collected and measured from the turbine casing drains following the completion of testing, in order to confirm satisfactory drain system operation. The results of the testing indicated minimal condensate was being passed.

The licensee concluded that the enhancements performed on the condensate drain

'

lines from upstream of MOV-514 (removal of the steam trap and maintaining all these drain valves open) and the refurbishment of MOV-514 had effectively halted any condensate formation downstream of MOV-514 in the turbine casing.

l An additional requirement that was incorporated into Step 2.17.3 of IPSP03-AF-0007 and 2 PSP 03-AF-0007 required that a period of 4-hours must

-

-_

._.

-

i

.

l

.

!

-14-

!

I have elapsed since MOV-143 was last cycled open. MOV-143 was the containment t

isolation valve located approximately 97-feet upstream of MOV-514. This valve

!

was periodically stroked to meet ASME Section XI and TS 4.0.5 requirements.

!

In the past, the licensee had experienced problems following stroke. testing of MOV-143 because when the valve was closed, the 97-foot length of pipe would

cool. After restoring MOV-143 to its normal open position, a significant ~

-l volume of condensate was formed while the length of pipe.was reheated to its j

normal, hot, standby condition. During the testing conducted in February

,

1993, and August 1993, the licensee acoustically measured the condensate flow

!

through F0 7537E, in' order to determine at what time condensate was clear and only steam passed; which would be the time that the steam pipe downstream of l

MOV-143 was considered. in a normal, hot, standby condition. The licensee

!

consistently recorded this time as less than 10 minutes. The inspector i

concluded that the four hour wait requirement was conservative and that the l

piping would be considered in its normal condition following that period'of

'

time.

Steps 2.2.1 and 2.2.2 of IPSP03-AF-0007 and 2 PSP 03-AF-0007 added further'

requirements to ensure that consistent and reliable surveillance testing;was

'

obtained. Step 2.2.1 requires that following any maintenance outage on the TDAFWPs, the inservice test (Step 5.8 of the procedure).must be satisfactorily

,

performed, followed by the operability test.(Step 5.7 of the procedure). The j

two hour wait was required between these two tests. Step 2.2.2 adds the

!

additional requirement that any time the TOAFWP was started manually and then subsequently declared operable after successful completion of Step 5.7 of l

IPSP03-AF-0007 or 2 PSP 03-AF-0007, an additional operability check shall be

,

performed a minimum of 48-hours and a maximum.of 168-hours following the

!

TDAFWP's return to an operable status. The inspector considered this

'

additional licensee required operability check an appropriate initiative.

!

2.2.2 Augmented Surveillance Testing Program

!

In order to assure that both TDAFWPs remained capable of performing their f

safety-related function following their return to service and entry into a

{

MODE that requires the pumps be operable, the licensee developed an augmented i

surveillance testing program that will be implemented utilizing the

>

operability test section (step 5.7) of IPSP03-AF-0007 and 2 PSP 03-AF-0007.

This testing will be performed in accordance with the following schedule:

j

,

'

Every 72-hours for 3 weeks,

Every 7-days for 4 weeks,

Every 14-days for 6 weeks, and

Every 30-days thereafter (normal periodicity).

• 2.2.3 Conclusions The inspector concluded that the actions taken by the licensee to improve TDAFWP testing methodology to be adequate and appeared to address'previously identified weaknesses in their surveillance program.

In addition, the

--

,-

,

_. _. _ -, _ _

,.

. _._,_

. _. _

,

...

.

-.

.

.

.

-15-inspector considered the augmented surveillance testing program appropriate to ensure operability following restart.

3 ACTIVITIES ASSOCIATED WITH OTHER TDAFWP INITIATIVES (92720)

3.1 Permanent Flow Indicators The licensee, in response to observations during the Diagnostic Evaluation Team Inspection, had initiated Modification 88269 (Unit 1) and 88270 (Unit 2),

a modification originally approved in 1988, but not worked. This modification ir. stalled AFW mini-recirculation flow meters to facilitate gathering data for the in-service testing program. The previous method of recording this data involved installing temporary flow detection instrumentation.

Modification 88270 had been completed, and Modification 88269 was scheduled to be completed prior to the restart of Unit 1.

>

3.2 Readiness Review Panel Activities The inspector observed the AFW System Readiness Review Panel that was conducted on October 19, 1993. This activity was being conducted in accordance with OiGP03-ZA-0005, Revision 0, " System Readiness." This procedure had been recently approved and established the methodology for evaluation of selected systems to support an orderly startup and maintain safe and reliable full power operations of both units.

In addition, the system readiness review was an activity that the licensee had committed to perform in the Operational Readiness Plan.

The scope of this activity involved the formal review of system issues that had the potential to affect safe component, subsystem, or system operations and to disposition the outstanding work remaining to be completed during the current outage.

The panel consisted of division managers from operations, maintenance, and plant engineering, with the system engineer presenting his

,

system and making recommendations on what pending outage work should be

)

deferred.

The panel conducted their review in a thorough manner. The AFW system had 29 items that the system engineer was recommending for deferral from the outage. The inspector reviewed this list of recommended deferrals, which included SRs completions, station problem report (SPR) corrective action resolutions, temporary modification installations, quality assurance issues, planned maintenance deferrals, and design changes pending. As a result of this review, the inspector determined that none of the deferrals appeared to be of sufficient significance to affect the reliability and operability of the system.

The panel did not accept all of the deferral recommendations, and directed the system engineer to place five of the items on the outage scope to be completed prior to restart of Unit 1 The inspector considered this action to be indicative of a healthy review process and proactive steps by licensee managers to correct problems and deficiencie.

.

.

-16-3.3 Corrective Action Taken to Inadequate Maintenance Practices NRC Inspection Report 50-498/93-05; 50-499/93-05 identified an example of licensee personnel conducting unauthorized maintenance on a safety-related component without the use of appropriate procedures. The root cause of this

,

!

event was determined to be improper judgement by the Unit Supervisor, General Maintenance Supervisor, and the Unit 1 Operations Manager.

The licensee addressed this example of inadequate maintenance implementation in SPR 930342. The action taken by the licensee to correct this problem and the steps taken to preclude recurrence consisted of counseling the individuals j

involved on the proper and procedurally acceptable manner that maintenance activities will be performed on plant equipment.

Additional action taken by the licensee consisted of directing the Unit 1

'

Operations Manager to conduct training to crew leaders, maintenance foremen, and all operators on the acceptable level of maintenance activities that operations personnel were permitted to perform without an authorized i

procedure. This training was completed and the SPR actions closed on April 29, 1993.

4 STATUSING OF ITEMS RELATED TO RESTART ISSUES (92701)

The following -items related to kestart Issues were statused concerning the

,

manner in which the licensee had resolved the issue within the scope of improving TDAFWP reliability and testing methodology.

Several of these issues were generally broad scoped and covered numerous licensee systems or programs.

They will remain open pending further NRC inspection effort to completely i

'

resolve the items during future Restart Issue inspections.

j 4.1 (0 pen) Violation 498:499/9305-05: Neither Unit's TDAFWP had been consistently tested under suitable environmental conditions to identify deficient conditions that affected operability.

Following the successful MODE 3 testing of the Unit 1 TDAFWP, this item will be closed.

4.2 (0 pen) Inspection Followup Item 498:499/9331-06:

Management Support to Correct Program and Component Problems was not Always Effective.

,

The inspector determined that management's support to correct the program and component problems associated with TDAFWP issues was effectiv.

'

o

.

!

-17-4.3 (0 pen) Inspection Followup Item 498:499/9331-07:

Maintenance and Testina Weaknesses Reduced the Reliability of Safety-related and Balance-of-Plant Equipment.

The inspector determined that the licensee's actions to improve maintenance

'

and testing of the TDAFWPs should increase reliability of that safety-related

'

component.

,

4.4 (0 pen) Inspection Followup Item 498:499/9331-08:

Ineffective corrective

!

and weak preventive maintenance significantly contributed to poor eauipment performance.

The inspector determined that the licensee had taken adequate steps to improve the preventive maintenance procedures associated with the TDAFWPs and should provide adequate guidance for personnel to perform future pump maintenance.

4.5 (0 pen) Inspection Followup Item 498:499/9331-09:

Ineffective corrective maintenance, caused by inadeauate root cause analysis, poor prioritization of work, and poor craft performance, adversely affected safety-related equipment performance.

The inspector determined that the licensee had taken adequata steps to improve the corrective maintenance procedures associated with the TDAFWPs.

4.6 (0 pen) Inspection Followup Item 498:499/9331-10:

Surveillance and postmaintenance testina did not always verify equipment operability: and Inspection Followup Item 498:499/9331-13:

Numerous weaknesses in the implementation and programmatic requirements for postmaintenance testing reduced assurance that eauipment was operable upon return to service.

The inspector concluded that the enhancements made to the surveillance and postmaintenance testing procedures for the TDAFWP were adequate and appeared to address previously identified weaknesses in their surveillance program.

4.7 (0 pen) Inspection Followup Item 498:499/9331-35:

The certification and acceptance process will be procedurally controlled and documented by two procedures. One procedure will define a comprehensive package that demonstrates each key system has been adeauately reviewed and any outstanding items have been appropriately evaluated and aispositioned. A second procedure will be developed that will require a comprehensive walkdown followed by acceptance of the system by the plant manager.

The inspector considered the system readiness review panel portion of the system certification process for the TDAFWPs a healthy review process and that proactive steps had been taken by licensee managers to correct TDAFWP problems and deficiencies.

,.

.

.

,

!-

-18-i 4.8 (0 pen) Inspection Followup Item 498:499/9331-50: The plant modification for permanent flow instrumentation in the auxiliary feedwater system will be installed.

Following the completion of the installation of Modification 88269 on Unit 1, this IFI will be closed. Modification 88270, which installed permanent flow instrumentation on the Unit 2 auxiliary feedwater system has been completed.

4.9 (0 pen) Inspection Followup Item 498:499/9331-52: Design Ch3nges or PCFs i

have been initiated to eliminate the use of temporary flow instruments or temporary pressure pages in the surveillance tests for the following systems:

Essential Chilled Water. High Head Safety Injection. Spent Fuel Pool Cooling, and Screen Wash Booster Pumps. The status of these changes will be evaluated as part of the assessment.

Following the completion of the installation of Modification 88269 on Unit 1, this IFI will be considered closed for the AFW system.

4.10 (0 pen) Licensee Event Report (LER) 498/93-07: Turbine-driven auxiliary feedwater pump is inoperable due to repetitive overspeed trips.

Following the successful MODE 3 testing of the Unit 1 TDAFWP, this LER will be considered closed.

4.11 (0 pen) LER 499/93-04:

Reactor trip due to the startup steam generator feedwater pump failing to operate because of recurrent problems with water intrusion into the pump's lube oil system.

The aspects of this LER associated with TDAFWP corrective actions have been completed.

Following the completion of the corrective action en the problems identified with the startup steam generator feedwater pump, this item will be closed.

5 CLOSED ITEMS RELATED TO RESTART ISSUES (92701)

The inspector determined that the licensee's actions to address the following issues was adequate. These items were considered closed.

5.1 (Closed) Violation 498:499/9235-02 (Example 3): The licensee had not identified the cause for the repetitive problems with the Unit 2 TDAFWP overspeed trip mechanism and had taken no corrective actions to preclude recurrence of the problem.

This item was closed based on the licensee's corrective action described in paragraph 2.1.5 of this report.

)

1

-.

.

.

.

.

.

- -.-

-

.

.

-

!

!

.

!

~r.

!

!

j

'

i-19-j

,

,

5.2 (Closed) Violation 498:499/9305-04: One of four violations concerning

?

the inoperability of the TDAFWP and the emergency diesel generators:

specifically, a failure to follow procedures that-resulted in

.

unauthorized maintenance being conducted by unaualified personnel on the i

Unit 2 TDAFWP.

Based on the corrective action taken by the licensee (refer to paragraph 3.3

,

of this report), this item was closed.

j 5.3 (Closed) Violation 498:499/9305-07: One of four violations concerning.

the inoperability of the TDAFWP and the emeraency diesel cenerators:-

!

specifically. the failure to have adeauate procedures for the adjustment

>

of the Unit 1 TDAFWP novernor valve.

,

i Based on the actions taken by the licensee (refer to paragraph 2.1.1 of this -

report), this item was closed.

)

5.4 (Closed) Inspection Followup Item 498:499/9331-43: The turbine-driven f

auxiliary feedwater pumps will be subjected to an augmented surveillance program that will confirm the reliability of the eauipment.

i

,

As a result of the licensee's commitment to adhere to the enhanced schedule i

for augmented surveillance testing (refer to paragraph 2.2.2 of this report)-

!

i of the TDAFWPs following entry into.a MODE that requires they be operable =

this issue was closed.

l 5.5 (Closed) Inspection Followup Item 498:499/9331-70:

Concerns the

!

licensee's commitment to revise the TS that reauire specific levels of

!

boron concentration in shutdown margin calculations.

!

The Office of Nuclear Reactor Regulation (NRR) issued TS Amendment Nos. 54. and

43 on October 4, 1993, which revised the boron concentration in shutdown.

margin calculations, which completed the action ~ required to close this issue.

.]

5.6 (Closed) Inspection Followup Item 498:499/9331-71: Concerns the licensee's commitment to revise the TS concerning the surveillance reauirements of turbine-driven auxiliary feedwater pump testina.

The licensee has concluded that a TS change to support the completion of the remaining TDAFWP testing in MODE 3 was not necessary and will not be requested.

5.7 (Closed) Inspection Followup Item 498:499/9331-72:

Concerns additional:

information requested by NRR in the licensee's initial response to Generic Letter 93-04.

In a letter to NRR dated October 4,1993, the licensee provided the additional

,

requested information per the Generic Letter.

<

,

.

,

i

).

-20-

>

c 6 ASSESSMENT OF MANAGEMENTS RECEPTIVENESS TO IDENTIFYING AND CORRECTING PLANT PROBLEMS (92720)

,

l The inspector determined that licensee management had responded in a proactive l

manner to the problems identified with the TDAFWPs.

In addition, several

'

activities, such as the enhanced training programs, purchase of a training TDAFWP, material enhancements installed on the TDAFWP, and installation of performance monitoring instruments downstream of MOV-514 and in the condensate drain lines, were indicative of management driven initiatives that should

'i result in improved reliability of the TDAFWPs.

t

>

,j I

!

!

l

!

!

!

i e

',

i

>

._

- -.

_

-..

_, _.

___

_

_.

.

.

__

n

a

.

A ATTACHMENT 1 1 PERSONS CONTACTED 1.1 Licensee Personnel H. Bergendahl, Manager, Technical Services J. Blevins, Supervisor, Records Management and Administration B. Brown, Supervisor, Mechanical Fluid Systems J. Calloway, Participant Services M. Chambers, System Engineer, Plant Engineering Department

J. Conly, Licnesing Engineer, Nuclear Licensing i

T. Cloninger, Vice President Nuclear Engineering W. Cottle, Group Vice President, Nuclear

,

D. Daniels, Administrator, Corrective Action Group

J. Groth, Vice President, Nuclear Generation

!

E. Halpin, Systems Manager, Plant Engineering Department

A. Harrison, Supervising Engineer, Nuclear Licensing -

l S. Head, Deputy, General Manager Nuclear Licensing

!

T. Jordan, General Manager, Nuclear Engineer

W. Jump, Assistant to Group Vice President, Nuclear.

!

M. Kanavos, Manager, Mechanical and Nuclear Engineering, Design Engineering

>

Department i

D. Keating, Director, Independent Safety Engineering Group-t A. Khosla, General Supervisor, Chemical Operations

J. Labude, Supervisor, Fire Protection

M. Ludwig, Manager, Nuclear Training-l L. Martin, General Manager, Nuclear Assurance f

L. Myers, Plant Manager, Unit 1

!

M. Pacy, Manager, Design Engineering

'

G. Parkey, Plant Manager

~

P. Parrish, Senior Specialist, Nuclear Licensing E. Stansel, Division Manager, Plant Engineering Department C. Stephenson, Licensing Engineer, Nuclear Licensing

J. Sheppard, General Manager, Nuclear Licensing

!

S. Thomas, Assistant, Vice President Nuclear Engineering i

D. Tower, Supervisor, Quality Assurance

L. Walker, Licensing Engineer, Nuclear Licensing i

W. Waddell, Manager, Operations Support

'

K. Wissman, Engineer, Material Technical Service Nuclear In addition to the personnel listed above, the inspectors contacted other l

personnel during this inspection period.

1.2 NRC Personnel

.j

!

M. Satorius, Project Engineer, Project Section A, Division of Reactor i

Projects (DRP)

!

J. Whittemore, Reactor Inspector, Plant Support Section, Division of Reactor i

Safety (DRS)

i M. Murphy, Reactor Inspector, Plant Support Section, DRS

D. Garcia, Resident Inspector, Project Section A, DRP

'}

The above listed licensee and NRC personnel attended the exit meeting.

i

l

!l

_.

-

_

._ _

_

.

e

'

<

.

,

J.

-2-2 EXIT MEETING An exit meeting was conducted on October 22, 1993. During this meeting, the inspectors reviewed the scope and findings of this report. The licensee did not take exception to any of the inspection findings, and did not identify as proprietary any information provided to, or reviewed by, the inspectors.

I s

~.,

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _, _.. _ _.. _ _... _ _. _ _ _. _ _ _ _ _. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _.. _ _ _ _ _

.__ _ ____ _ _ _ _____._ _.___ _ ___

( 88m.

-

%

.

TURBINE DRIVEN AUXILIARY FEEDWATER PUMP (UNIT 1 CURRENT CQNElGiURAT!QN1 ik

r 0 is @

01 3 MAIN SiLAnd

'

I f

"

^

>

>

=

,

__

U l

l lif-U U

.

.

M_

N'si' TIE E'd*mU'o's

'

covrawon is'

E

~

' " " ' '

u

{'l l4

_

tws %

"W5

-

"

Fo 7537E

-

g g

==

" ' ' '

cLTO.

A y

,,

fy A

Y

,_

,

I

'

i i

>

~1

, =. -

m v v

.

>

>

sui..

m o-12 : 3A owc

'

REV O

_

--

.

-., - -