IR 05000277/1998003

Insp Rept 50-277/98-03 on 980212-0303.Violations Noted.Major Areas Inspected:Reviewed Events,Significance,Reportability & CAs for 2 a RFP Turbine High Level Trip Events Involving 3 1997 Failures & 1 Trip on 980206

ML20217N377
Person / Time
Site:	Peach Bottom
Issue date:	03/30/1998
From:	NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I)
To:
Shared Package
ML20217N342	List: IR 05000277/1998003 ML20217N336 ML20217N365
References
50-277-98-03, 50-277-98-3, NUDOCS 9804090089
Download: ML20217N377 (13)

Text

, - -

..

.

U.S. NUCLEAR REGULATORY COMMISSION REGION l Docket Nos.

50-277 License Nos.

DPR-44 Report No.

98-03 Licensee:

PECO Energy Company Facility:.

' Peach Bottom Atomic Power Station Unit 2 Dates:

February 12 through March 3,1998 Inspectors:

' A. McMurtray, Sensor Resident inspector M. - Buckley, Resident inspector

.s Y

'

y

,, -

.-)

,

'

9904090089 990330 PDR 'ADOCK 05000277 G

PDR.

'

..

r-

.(,

,

,

_

i

.

., -

\\

TABLE OF CONTENTS EX EC UTIVE S UMM A RY, ;.......................,,,,,,,,,,,,,,,,,,,,, ji.

o 111. Eregineering -...............................,,,,,,,,,,,,,,,,,,,,, 3

'

E2 Engineering Support of Facilities and Equipment

.......................1

.

E8., Miscellaneous Engineering issues.................................. 7 E8.1 Unresolved item No. '50-277/97-07 04:2 'A' RFPT Trip Mechanism (Closed)

....................................................7 E8.2 ~ Unit 2 Reactor Scram Resulting from a Generator Lockout Condition &

Subsequent Turbine Trip Failure of the 2 'A' Reactor Feedpump Turbine to Trip on Demand during Recovery _ Activities. LER 50-277/97-009(Closed) and

,

Failure of the 2 'A' Reactor Feedpump Turbine to Trip LER 50-277/97-010

(Closed)

..............................................7 V. ' M anage ment Meetings ;........................................... 7

X 1 Exit Meeting Sum m ary............................................ 7

.

X2 Review of Updated Final Safety Analysis Report (UFSAR) Commitments......... 8

.

'

H

.,-

t

. <

'

s t

,I"'"

'

).

..:

,

.

..

EXECUTIVE SUMMARY Peach Bottom Atomic Power Station, Unit 2 NRC Inspection Report 50-277/98-03 This special inspection was conducted to review events, significance, reportability, and corrective actions for the 2 fA' Reactor Feedwater Pump Turbine high level trip events

. involving three 1997 failures and one slow trip on February 6,1998.

~

Corrective actions from previous events failed to prevent noncompliance with Technical

- Specification 3.3.2.2 requirements to maintain operability of the Digital Feedwater Control System high water level trip capability for the 2.'A' Reactor Feedwater pump. This -

condition existed for an indeterminate period for up to 9 months. This noncompliance resulted in an apparent violation of technical specification 3.3.2.2. Of particular concern-was the dec sion not to perform test ng o tf his trip function at power following the second i

i

. event in November 1997.

The failure to submit a Licensee Event Report (LER) for the noncompliance with Technical

- Specifications due to the 2'A' Reactor Feedwater Pump Turbine high water level trip function event on April 1,1997, was considered a violation of 10 CFR 50.73 -

requirements.

The delay to change the status of the 'A' feedwater train to (a) 1 until February 1998, by the Maintenance Rule Expert Panel was not proactive given the technical specification

- significance of these events. However, the change in status met the maintenance rule procedural and regulatory requirements.

-

__

ii

,

,,

'

s

q

,.

,

m<

k Report Details i

111.. Engineering E2'

' Engineering Support of Facilities and Equipment

)

- E2.1 Inonerable Reactor Feedwater Pumo Hiah Water Level Turbine Trio a.

Inspection Scone (37551. 62707. & 92701)

The insl.ectors reviewed events, significance, reportsbility, and corrective actions -

for the ? 'A' Reactor Feedwater Pump Turbino (RFPH Ngh level trip events involving three 1997 failures and one slow trip on aw my 6,'1998. The c

feedwate r system performance was also reviewed for compliance with the Mainten 4nce Rule criteria.

b.

Observ stions and Findinas The Eeactor Feedwater system for each unit consists of three 50% flow capacity RFP'/s. The Digital Feedwater Control High Water Level Trip System (DFCS) and mr.n turbine high water level trip instrumentation is designed to detect a potential folure of the Feed Level Control System that causes excessive feedwater flow.

'With excessive feedwater flow, the water level in the reactor vessel rises toward.

the high water level set point, causing the trip of the three feedwater pump turbines and the main turbine. A trip of the feedwater pump turbines limits further increase -

in reactor vessel water level by limiting further addition of feedwater to the reactor vessel. This prevents moisture carryover into the main steam lines. A trip of the main turbine and closure of the stop valves protects the turbine from damage due to water entering the turbine. As described in the Peach Bottom UFSAR, the reactor high water level condition would trip the main turbine causing a reactor scram thus avoiding unacceptable safety results and ensure that the Minimum Critical Power Ratio (MCPR) safety limit is not exceeded. MCPR limits are imposed to avoid fuel damage due to severe overheating of the fuel cladding.

During the normal shutdown and testing evolutions in 1997 and early 1998 several failures were noted during testing of the 2'A' Reactor Feedwater Pump Turbine high water level trip function. A description of these events is provided below and in

. inspection report 97-07.

Aoril 1.1997: During a response to an EHC problem, operators reduced power to 40% and attempted to trip the 2 'A' RFPT from the control room. The RFPT failed to_ trip. An attempt to_ trip locally using the manual trip also failed to trip the RFPT.

The RFPT finally tripped after some mechanical agitation of the trip dump valve. In

_

October 1996, the licensee had successfully functionally tested all three unit 2

-

RFPTs using digital computer trip inputs signals.

' The licensee's troubleshooting found debris in the oil with a sludge consistency in L and around the trip ' ump valve. The manufacturer verified that this condition could d

cause a failure to trip and was consistent with failures described by other industry

_.

..

]

operating experience. - Maintenance removed the sludge, reassembled the trip mechanism, and verified the RFPT would trip both from the control room and :

locally. The Ikensee chose to bring the 2'A' RFPT back into service with only static s

testing as pmof of trip capability.' Static testing provided only. verification of the.

l control oil system trip and no steam was admitted to the turbine. During this testing, operations personnel performed two trips with the push-button in the control room and two local trips at the turbine. All trips occurred within one second.

as described in section 14 of the UFSAR.

November 9,1997: Following a reactor scram, the Unit 2 reactor operator attempted to trip the 2'A' RFPT from the control room. The RFPT failed to trip and a subsequent local attempt failed. Again, light physical force directed to the dump -

-

valve resulted in a trip.

During troubleshooting, maintenance personnel disassembled the trip mechanism and identified intermittent binding of the manual trip rod spring cup against the trip device housing due to deformation of the spring cup. A slight misalignment was also identified between the trip lever, trip rod and reset piston above the trip dump housing. This misalignment was scheduled to be corrected during the next refueling outage. The manual spring cup, spring, and trip rod were replaced and the trip mechanism was reassembled and reinstalled.

The inspectors observed the technicians in the shop duplicating the failure of the trip mechanism to dump oil. The mechanism would hang up so that the trip dump.

valve would not move. The inspectors observed that the spring cup was deformed and the spring rod was scored, gouged, and out of round. The inspectors also observed that these parts had damage that was dull in appearance in some locations indicating that the damage had occurred over an extended period of time.

Other locations were brighter indicating that the damage was ongoing. The inspectors determined that these deficiencies could have caused the RFPT failure to trip when required and that this condition had existed for some time.

n Following the November 9 event, the licensee returned the 2'A' RFPT back to service with static testing consisting of three trips with the push-button in the control room and three local trips. All trips occurred within one second as described

. in the UFSAR. No subsequent testing during operation occurred that would have

. assured the cause had been identified and corrected.

The inspectors noted similar symptoms for both the April and November events.

Following this event, the inspectors questioned the lack of at-power verification -

testing and opened Unresolved item (URI) 50-277/97-07-04.

December 29I1997: During a normal shutdown evolution for EHC system repairs, the Unit 2 reactor operator. attempted to trip the 2 'A' RFPT from the control room.

~

.

' The RFPT failed to trip at this time. Subsequently, the 2 'A' RFPT tripped during a transient that occurred when all the turbine bypass valves inadvertently opened.

' The licensee initiated troubleshooting and repair activities for both the electrical and mechanical components to the trip function.

[

,-

._,

'*b

.

,

' Following the third trip failure, maintenance and engineering personnel rigorously investigated possible root causes, both mechanical and electrical. The main focus of these investigations was on the electrical portion of the trip mechanism, since the turbine had mechanically tripped during the December 29 transient.

Initial resistance checks during troubleshooting activities on the trip solenoid did not reveal any unusual readings. Further troubleshooting identified an intermittent short by vibrating the trip solenoid. The licensee's investigations indicated that the '

probable cause of failure to trip on December 29 was the result of an intermittent short in the trip solenoid coil.

An analysis by Valley Forge laboratories confirmed an intermittent short existed when the trip dump valve solenoid was vibrated. Laboratory testing and analysis determined that the short likely was caused by long term vibration. Laboratory.

- results revealed there was no change in pull force even with the short present. A contributing factor to the intermittent short was that the trip coil mounting -

assembly / bracket was missing two hold down bolts. Maintenance replaced the trip dump valve trip solenoid coil.

Other out-of-specification or anomalous conditions found during the inspections ws,re repaired or replaced. Maintenance and engineering personnel found and resolved the following conditions that could have contributed to the trip failure:

Trip dump piston scored on lands.

• Many holes drilled in trip valve piston in piston travel set area.

• Trip lever out of alignment.

• Trip dump valve anti-rotation pin galled.

• SV-12 trip coil plug out of alignment, and undersized.

• SV-12 trip coil indicated intermittent shorts under vibration and missing 2

hold down bolts.

Lockout valve binding in the bore and land scoring.

• 1.ength of service for springs on the trip dump, trip valve, and trip rod.

• '

Control Room manual trip push-button reliability verification.

The licensee performed a visual verification on the trip coils for the Unit 3 RFPTs and tested the coils for the other Unit 2 RFPTs. This testing and verification indicated no shorts or similar mounting conditions.

Upon completion of repairs, the post-maintenance testing included 2 trips with tho'

control rcom push-button and 2 trips locally in the static condition. All trips occurred within one second as described in the UFSAR. The licensee decided after

. an engineering review to initiate a periodic testing program. Testing included tripping the turbine wh% running norma?ly but isolated from the reactor vessel and

. recirculating to the condenser.1his test was performed just before restoring 2'A'

RFPT from standby to service, durieg a c own power approximately ten days later, and every subsequent thirty days. The ',icensee previously chose not to perform at-

" -

power testing as recommended in Technical Information Letter (TIL) 83-4 " Lockout =

' Suppressed Overspeed Trip Test Frequency-Utility Turbines."

.

,

.

..

The inspector noted that periodically at power testing with the turbine in operation

,

provided better verification that the repairs resolved the problem. This addressed

' the concerns identified in URI 50-277/97-04regarding testing of the 2'A' RFPT during power operations to ensure that all failure mechanisms from the previous events had been identified and corrected.

February 6.1998: During the performance of an at-power test, the Unit 2 reactor operator attempted to trip 2'A' RFPT from the control room. The operator had to hold the push button for three to five seconds, while the expected trip should have occurred immediately ( l.a., less than one second).

- After the slow response of the RFPT trip system, the licensee instrumented the control oil system and performed testing both from the control room and locally.

The slow response could not be duplicated during the testing. Engineering change request (ECR) 98-00329 and 10 CFR 50.59 evaluation indicated that no thermal limits would be exceeded for a feedwater pump trip delay of less than ten seconds

,

provided the other RFPTs and the main turbino trip within one second. Further, the

.

10 CFR 50.59 evaluation concluded that there were no safety impact and no unreviewed safety question for this trip delay. The inspectors reviewed the NCR p

- and the 10 CFR 50.59 and identified no concerns with the analysis presented.

The licensee restored the 2'A' RFPT back to service on February 11 and performed a satisfactory at power test on February 21,1998. The licenses planned to perform an at-power trip test of the 2'A' RFPT around March 21,1998.

Following the February 1998 slow trip, the inspectors reviewed the unit 2 'A'

feedwater train Maintenance Rule classification. During the monthly Maintenance Rule Expert Panel (MREP) meeting for February 1998, the system manager presented a corrective action plan for the Unit 2 'A' feedwater train with the recommendation to change the status to an (all system. The expert panel agreed

'

with this recommendation and accepted the action plan which included inspection of the 2 'B' and 'C' RFPT durir.g the Fall 1998 outage. Although a MREP meeting

" had been held in January 1998, the 'A' feedwater train remained an (a)2 system.

The performance criteria that would initiate consideration of the feedwater trains into (all status were:

more than one maintenance preventable function failure (MPFF)

%

unavailability greater than 2%.

'Although the unavailability of the 'A' feedwater train had been between 5% and 6% since January 1996, the train was not changed to (all status because of previous modification and operational activities. The inspectors determined that this met the licensee's administrative requirements for the Maintenance Rule. However, the inspectors questioned the timeliness of the status change of the 2'A' feedwater train to (a)1. After review of the MREP activities with respect to the 2'A' RFPT events in April, November and December, the inspectors had no overall concerns.

,.

,

.

-

J L __

v

.

3-

1 i

'

At the end of this inspection the licensee was still reviewing this issue and was performing additional analysis. The licensee had still not determined the root cause and was re-evaluating the operability determinations for the prior failures to trip.

The inspectors noted that, based on the UFSAR section 14.5.2.2 and figure 14.5.5, all RFPT trips are expected within one second of a trip demand. Following the three to five second delay of the 2'A' RFPT tripping on February 6,1998, the licensee -

evaluated the impact of a 10 second trip delay.

Throughout these events, the main turbine trip function and the trip functions on the two other reactor feed pumps remained operable. The licensee's analysis showed that the fuel thermal and MCPR limits were protected. However, moisture carryover into the main steam piping would increase with the feedwater puinp trip delay time. The licensee identified that even for the cases where the main steam-

_

pipe filled with water; the design criteria for the piping was maintained and the safety relief valves remained operable. Also, the reactor trip and reactor high level procedures require operator action to limit and prevent increasing reactor water level. The inspectors reviewed the documentation for the trip failures and the safety evaluation, ECR, and supporting analysis from the February 6, slow trip and independently concluded that these events were of limited safety significance.

However, these events constitute a departure from the regulatory requirements.

Peach Bottom Atomic Power Station, Unit 2, Technical Specification 3.3.2.2,

"Feedwater and Main Turbine High Water Level Trip instrumentation," requires that the Digital Feedwater Control System high water level trip capability be maintained operable with reactor power 225%. If the Digital Feedwater Control System high water trip capability is inoperable, reactor power shall be reduced to < 25% within

'

six hours. The basis information for technical specification 3.3.2.2 describes the trip of the RFPTs as part of the logic functional testing for the Digital Feedwater Control System.

.

Based on the following, the inspectors determined that the requirement to maintain the RFPT trip function or restore the function within two hours had not been met for an indeterminate period for up to nine months:

4*-

the failures to trip on April 1, November 9, and December 29,1997.

the long term mechanical binding indications observed by the inspectors.

• k

,o lack. of preservation of original parts for future evaluation.

• -

.;.

• I

the detailed analysis required for the trip' failures after February 6,1998, for operability justification.

-

,

'

l?

,

s

-] $ l a.

$

.:

..

H

Therefore, from April to December the 2 'A' RFPT high water trip function was not maintained operable. The failure to maintain the Digital Feedwater Control System high' water level trip capability operable for the 2 'A' reactor feedwater pump when -

-

reactor power was equal to or greater than 25% is an apparent violation of TS 3.3.2.2. (eel 50-277/98-03-01)

During the review'of these events, the inspectors noted that no Licensee Event Report (LER) had been submitted for the failure to trip of the 2'A' RFPT on April 1,

.

1997.10 CFR 50.73 requires that an LER be submitted within 30 days for any operation or condition prohibited by Technical Specifications. ' NUREG-1022, Rev.1,

" Event Reporting Guidelines 10 CFR 50.72 and 50.73" discussed the need for firm evidence that the condition existed prior to the time period prohibited by technical specifications. However, the inspectors determined based on the suspected cause of the failure and history of failure that the condition existed for some time.' After discussion with the system manager and manufacture representative, the inspectors

"

determined that the cause of the failure to trip on April 1 had existed for longer than the two hours limit specified in section 3.3.2.2 of Technical Specifications, prior to the attempt by the operator to trip the 2 'A' RFPT. Therefore, not reporting the April 1,1997 failure to trip of the 2 'A' RFPT within 30 days is contrary.to '.0 CFR 50.73. (VIO 50-277/98-03-02)

The inspectors noted that LERs were submitted for the failures that occurred on November 9 and December 29,1997. The LER for the event on November 9 also discussed the April event and the corrective actions for both events. Based on the fact that a full discussion of this issue was presented in this LER, no additional submittal of an LER for the April event is required.

c.

Conclusions This special inspection was conducted to review events, significance, and corrective actions for the 2 'A'. Reactor Feedwater Pump Turbine high level trip events

'

involving three 1997 failures and one slow trip on February 6,1998.

Corrective actions from previous events failed to prevent noncompliance with technical specification requirements to maintain operability of the Digital Feedwater Control System high water level trip capability for the 2 'A' Reactor Feedwater pump. This noncompliance from April to December 1997 resulted in an apparent violation of technical specification 3.3.2.2. Of particular concern was the decision not to perform testing of this trip function at-power following the second event in November 1997. _ The failure to submit a Licensee Event Report (LER) for the -

noncompliance with Technical Specifications due to the 2'A' Reactor Feedwater Pump Turbine high water level trip function event on April 1,1997, was considered

~ a violation of 10 CFR 50.73 requirements. The delay to change the status of the

"A" feedwater train to (a)1 until February 1998, by the Maintenance Rule Expert Panel was slow and not proactive to the problem.

s'

y

[

,

.,

E8 Miscellaneous Engineering lasues -

' E8.1 Unresolved item No. ' 50-277/97-07-04:2 'A' RFPT Trio Mechanism (Closed)

This unresolved item questioned the decision by th' licensee, following the e

November 9,1997 failure to trip,'not to perform at-power testing of the 2 'A' RFPT trip mechanism to verify that the cause of the failure was identified and corrected.

.The safety significance of this event was also unresolved.

As discussed above in Section E2.1, the inspectors determined that the licensee'can perform at-power testing to identify further failures or deficiencies. The February 6, 1998 slow trip of the' RFPT was identified due to at-power testing initiated after the December 29,1997, failure to trip. The inspectors also documented the safety significance of these events in Section E2.1 Licensee' action on this issue will be tracked by the apparent violation No. eel 50-277/98-03-01.

l E8.2 Unit 2 Reactor Scram Resultina from a Generator Lockout Condition & Subseauent Turbine Trio Failure of the 2 'A' Reactor Feodoumo Turbine to Trio on Demand durino Recovery Activities. LER 50-277/97-OO9(Closed) and Failure of the 2 'A'

Reactor Feodoumn Turbine to Trio LER 50-277/97-010(Closedl The inspectors reviewed PECO's rcot cause and corrective actions for the reactor scram' documented in LER 50-277/97-009. This scram occurred when an equipment operator failed to follow procedure while swapping a.125 volt DC system battery charger. This event was documented in inspection Report 50-277(278)/97-

' 07 and resulted in a violation for failure to properly implement procedures. This issue will be tracked under VIO 50-277(278)/97-07-02.

The inspectors reviewed LERs 50-277/97-009and 50-277/97-010for completeness and corrective actions taken in response to the 2'A' reactor feed

,

"

pump turbine high water level trip failure. Further licensee action on this issue will be tracked via apparent violation No. eel 50-277/98-03-01.

V. Management Meetings

~X1-Exit Meeting Summary The inspectors presented the inspection results to members of licensee management at the conclusion of tha inspection on March 4,1998. The licensee acknowledged the findings presented, but disagreed with the inspectors conclusions regarding previous operability of the feedwater trip function.

The inspectors asked the licensee whether any materials examined during the inspection Lshould be considered proprietary. No proprietary information was identified.

.

i

- - - _ -- -

sprA e

,

.

8'

X2 Review of Updated Final Safety Analysis Report (UFSAR) Corrnntments

- A discovery of a licensee operatire their facility in a manner contrary to the Updated Final

'

Safety Analysis Report (UFSAR) description highlighted the need for a special focused review that compares plant practices, procedures and/or parameters to the UFSAR descriptions. While performing the inspections discussed in this report, the inspectors

.' reviewed the applicable portions of the UFSAR that related to the areas inspected. The inspectors verified that the UFSAR wording was consistent with the' observed plant practices, procedures and/or parameters.

-,

i

,

b

.

-

e i

'

.

, *

LIST OF ACRONYMS USED digital feedwater control high water level trip system (DFCS)

'

electro-hydraulic' control (EHC)

,

escabAed enforcement item (EEI)

licensee event report (LER)

maintenance rule expert panel (MREP)

minimum critical power ration (MCPR)

' Peco Energy (PECO)

public document room (PDR) _

reactor feed pump turbine (RFPT)

safety relief valve (SRV)

surveillance requirement (SR)

technical information letter (T!LL)

' t techr ical specification (TS)

unresolved item (URI)

. updated final safety analysis report (UFSAR)

,

i

-

i i

.

i

..

4.. ; '.

INSPECTION PROCEDURES USED IP 37551:

Onsite Engineering Observations IP 62707:

Maintenance Observation IP 92700:

Onsite Follow-up of Written Reports of Nonroutine Events at power Reactor Facilities ITEMS OPENED, CLOSED, AND DISCUSSED Ooened 50-277/98-03-01 eel 2 'A' RFPT High Water Level Trip Failure per TS 3.3.2.2 50-277/98-03-02 VIO Failure to Submit LER for TS Non-compliance Closed 50-277/97-07-04 URI

'A' RFPT Trip Mechanism

.50-277/97-009 LER Reactor Scram and 2 'A' RFPT Trip Failure 50-277/97-010 LER 2 'A' RFPT Trip Failure

<

+

n-jf /

i

_.