IR 05000313/1989003

Insp Repts 50-313/89-03 & 50-368/89-03 on 890124-25.Major Areas Inspected:Circumstances Following 890120 Reactor Trip Leading to Reactor Coolant Backleakage Through HPSI Check Valve Which Failed to Seat

ML20235M670
Person / Time
Site:	Arkansas Nuclear
Issue date:	02/15/1989
From:	Chamberlain D NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV)
To:
Shared Package
ML20235M668	List: IR 05000313/1989003 ML20235M666
References
50-313-89-03, 50-313-89-3, 50-368-89-03, 50-368-89-3, NUDOCS 8902280360
Download: ML20235M670 (21)

Text

. ____ _ _ ___ _ ______-___ - ______

-,

~

.

'

, .-

t i

APPENDIX U.S. NUCLEAR REGULATORY COMMISSION REGION.IV NRC Inspection Report: 50-313/89-03 Operating Licenses: DPR-51 50-368/89-03 NPF-6 Dockets: 50-313 50-368

Licensee: Arkansas Power _and Light Company (AP&L)

P.O. Box 551 Little Rock, Arkansas 72203

.; , Facility Name: Arkansas. Nuclear One (AN0), Units 1 and 2 Inspection At: ANO, Russellville, Arkansas Inspection Conducted: January 24-25, 1989 e

~

. Team Leader: V) 2/ 8 /87 D. D. Chamberlain, Chief," Project Section A Date Division of Reactor Projects i

Team Members: C. Harbuck, Project Manager, NRR R. Kendall, Senio: ilectrical Engineer, NRR C. Liang, Senior Reactor Systems Engineer, NRR R. Azua, Reactor Inspector, RIV T. McKernon, Reactor Inspector, RIV

{

8902280360 DR 890217 r'

g ADOCK 05000323 i PNU l

_ _ ___ _ _ _ l

_ _ __

.

.

,

1.0 General Background Information The NRC has established a policy to provide for the timely, thorough, and systematic inspection of significant operational events at nuclear power plants. This includes the use of an Augmented Inspection Team (AIT) to determine the causes, conditions, and circumstances relevant to an event and to communicate its findings, safety concerns, and recommendations to NRC management. An AIT was sent to Arkansas Nuclear One (ANO) Unit 1 on January 23, 1989, to review an event which occurred following a reactor trip on January 20, 1989. A description of the event and the AIT tasks is provided belo .1 Description of Event Following a reactor trip on January 20, 1989, a series of occurrences led to an event where reactor coolant flowed through a high pressure injection (HPI) system crossover line outside the reactor containment building. A detailed overview and time sequence for the occurrences is provided in Section 3.1 of this report. Conditions were created which allowed reactor coolant systems (RCS) flow through the "B" HPI line outside the reactor containment building into a crossover line to the "C" HPI line back to the reactor vessel. A diagram showing the HPI system interface with the RCS, and the back flow path is provided in Figure The concern with this event was the pressure and temperature effects of the RCS flow in this crossover line outside the reactor containment building. The AIT was dispatched to ANO on January 23, 1989, to gather facts regarding licensee actions and plant response to this even .2 AIT Tasks The AIT dispatched to ANO was composed of a team leader from NRC Region IV, two regional inspectors, the ANO Project Manager from Nuclear Reactor Regulations (NRR), and two technical specialists from NRR. The AIT tasks were specified in a memorandum from the Region IV Director of the Division of Reactor Projects to the team leader. These tasks included:

" A thorough review of all of the sequence of events and subsequent NRC notification . A review of the operator response which may have contributed to the check valve backleakage (i.e., start and stop of high presure injectionpump). An investigation of the contribution of reactor coolant pumps (RCP)

trips to the check valve back leakag l

' The adequacy of licensee preliminary evaluation of effect of leakage on piping and conta hment integrity and evaluation of system design."

Although the AIT concentrated efforts on the event of concern, all aspects surrounding the plant trip and response were reviewed for any other safety

.

.

,

-

concerns. The primary focus of the AIT was on fact finding; any potential enforcement matters will be the subject of subsequent inspection effor .0 AIT Inspection 2.1 Sequence of Occurrences Leading to the Event 2.1.1 Overview On Friday, January 20, 1989, at approximately 9:58 p.m., the Arkansas Unit 1 (ANO-1) nuclear power plant experienced a lockout / trip of the main generator and subsequent main turbine trip and reactor trip. The plant was operating at 100 percent power prior to the reactor trip,. and power to plant / house loads was being supplied from the main generator via the unit auxiliary transformer (UAT). Following the reactor trip, the source of power to 6.9 kV Bus H1 failed to fast transfer (from the UAT to the startup transformer) as designed. This caused the two reactor coolant pumps (RCP) powered from bus H1 to trip upon sensing bus undervoltag There are a total of four RCPs at ANO-1, one in each of the two return lines (cold legs) from each once through steam generator (OTSG) back to the reactor vessel. Power is supplied to the startup transformer from the offsite power syste Main feedwater (MFW) system flow to one of the OTSGs failed to auto-matically runback / decrease to post reactor trip values (because of several problems / failures), causing a slight (11 F) overcooling of the reactor coolantsystem(RCS). A high pressure injection (HPI) system pump was manually started (to maintain the level in the pressurizer above the heaters) when the operators observed pressurizer level decreasing because of RCS shrinkage from the overcooling. The HPI pump was started and HPI was injected for approximately 2 minutes through the HPI injection path No automatic safety injection setpoints were reache Shortly after the HPI pump was secured, a fire alarm (received in the control room) was activated from a smoke detector located in the upper north piping penetration room (UNPPR). The HPI system injection lines enter / penetrate the reactor containment building from this room. An operator who was dispatched to the UNPPR to investigate the alarm discovered that the "B" and "C" HPI injection lines were hot (the temperature of these lines was well in excess of the expected borated water storage tank water temperature). The high temperature resulted from RCS flow through a failed check valve (MU-34B) located in the "B" HPI injection line inside the reactor containment building. The leakage flow path was from the RCS and outside the reactor containment building through MU-34B, through the crossover pipe to the "C" HPI injection line, and then back inside the reactor containment building to the RCS. The existing RCP configuration (i.e., two pumps running and two pumps tripped) created a differential pressure across Check Valve MU-34B that caused backflow through the valve when it failed to reseat after the HPI pump was secure i Subsequent review by the licensee revealed that the HPI crossover line was not qualified for RCS temperatures. Analysis by the licensee has shown that the maximum allowable stress limit for certain locations on the I

-___ ___ ___

.

.

,

crossover line was exceeded during the event. A diagram showing the HPI system interface with the RCS, and the backleakage flow path is provided in Figure 1. A detailed sequence of occurrences is provided belo )

2.1.2 Sequence of Occurences l The licensee provided the Augmented Inspection Team (AIT) with a hand written sequence of occurrences listing that was compiled from data obtained from the following sources:

plant computer sequence of events printout,

plant computer alarms and status messages printout, safety parameter display system (SPDS) printouts (these include equipment status information and 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> 'tistory data for critical parameters),

shift supervisor logs, and plantperformanceanalysissystem(PPAS) data (thePPASmonitorsthe integrated control system, including inputs and outputs; the nonsafety-related integrated control system is designed to automatically control steam flow, feedwater flow, and reactor power level within appropriate limits to maintain desired power generation levels).

The sequence of occurrences listing below was compiled from the licensee's sequence of events listing as supplemented by additional information that has since become available. Times followed by "(?)" are best estimates based on available infomation, but the exact times are unknow Initial conditions: 100 percent power; normal operating temperature and pressur Janua ry 20, 1989 20:30 (?)

(Exact time unknown) Perturbations in the form of voltage swings /

spikes are observed by the operators on a control room meter that displays the output voltage from the main generator exciter voltage regulator (i.e., generator field voltage). The normal generator field voltage is 50 Vdc. The voltage spikes occur fairly regularly (at approximately four minute intervals) with peak values of near 90 Vdc at first, and becoming more severe with time until the meter pegged high at 150 Vdc just prior to losing generator field voltag :58:11 The automatic voltage regulator is placed in the off position by the control room operator. Loss of main generator field voltage occurs due to a failed electrical connection on an exciter field windin ___ _ _______- _____ _-_-__ __-__ _ _

- -

- _ _ - -

'

s

'

'[

-

gp .

-*-

,

. r

";p L This causes a generator; lockout via the generator protection circuits (i.e., the generator _ field breakers and output breakers open, electrically disconnecting the generator).

,

' The main turbine trips on generator lockou . The reactor trips upon trip.of. the main turbine via the safety related anticipatory reactor trip (ART) circuits. The plant

. operators proceeded to bring the unit to hot shutdown condition The power source to 'nonsafety-related 6.9kV Bus H1 fails- to automatically fast transfer from the unit auxiliary transformer (supplied from. the main generator) to the startup transformer (supplied from the offsite power system).. An automatic fast transfer.-

' does occur to provide power to the other nonsafety-related buses.,

! 6.9kV' Bus'H2 and 4.16.kV Buses Al and A2. 'All safety-related buses transfer as designed following the tri '

. 21:58:15 A trouble alann is received in the control room on 6.9kV Bus H1 loss

-

of' voltag Reactor Coolant. Pumps "A" and "C trip on undervoltage.' These pumps are powered from Bus H Two channels' of the emergency, feedwater initiation and control (EFIC)

. system spuriously trip upon sensing once through steam generator-(OTSG) low level. Actual OTSG level' was well'above the emergency feedwater initiation setpoint. .

The main feedwater (MFW) system startup flow control valves (SUCVs CV-2623 and CV-2673) and low load flow control valves (LLCVs CV-2622 '

.

and CV-2672) fail to close.as designed following the reactor trip, allowing. continued MFW flow paths to each OTS The."B" MFW pump (P1B)' fails to runback to minimum speed as' designed followhg the reactor tri The "B" MFW block valve (CV-2675) fails to close as designed following the reactor trip. The valve starts to close, but stops when the valve torque switch actuates before the valve close An automatic " slow dead bus transfer" occurs to provide power to 6.9kV Bus H1 from the startup transformer, restoring power to the

,

bus'. . Reactor Coc,lant Pumps ."A" and "C" remain shutdow :59':08:

.The operators manually start high pressure injection (HPI) System Pump P36A to provide additional makeup flow to maintain pressurizer level above.the heater cutoff point. Pressurizer level was decreasing due m

- - _

..

.

,

to reastor coolant system shrink from excessive MFW flow to OTSG "B" and the slight overcooling (reactor coo _lant-temperature dropped about 11*F).

'

21:59:25 OTSG_"B" high level alarm at:92 percent.is received in the contro room.- 1

~ 21:59:38 The "A" MFW isolation valve (CV-2680) and the 'd" MFW isolation valve (CV-2630) are manually closed by the operators from the control roo '

21:59:40 .

The "B" MFW main block valve (CV-2675) is manually closed by the

_

operators from the control roo The."B" MFW pump (P18) runs back to minimum speed on its ow :59:44

'

Level in the "B" 0TSG begins decreasing from a high value of-99 percent on the operating range. Level in the "A" OTSG is less than, and paralleling, the level in the "B" OTS The MFW isolation valves are reopened by the operators from the control ~ roo :00:25-

"B" OTSG 1evel begins increasing from 91 percent on the operating rang :01:33 HPI Pump P36A is secure "B" 0TSG 1evel increases to near 100 percent on the operating range (actual level may have gone slightly offscale above 100 percent).-

22:01:40

"B" 0TSG 1evel begins decreasing from 100 percen :02:33 The "B" MFW pump is secured, and Motor Operated Valve CV-2827 in the cros-tie line between the discharge of the two turbine driven MFW pumps (PIA and PIB) is opened. MFW Pump "A" is now providing MFW flow to both OTSGs via the SUCVs and LLCV :03:53 The "A" and "B" MFW isolation valves are closed by the operators from .

the control room in response to the increasing level in the OTSG I I

I Control of the SUCVs and the LLCVs for both OTSGs is transferred from automatic control to manual control, and these valves are manually closed by the operators.

p-

- - -_-----_-------_-_____-_____a

.-- - _____ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

.

'

-

.

22: (?)

(Exact time unknown). A fire alarm is received in the control roo The alarm is activated from a smoke ~ detector located in the upper north piping penetration room (UNPPR). There is no fire water system flow to the UNPPR which indicates no actual fire in the area. An operator is dispatched to the UNPPR to investigate the cause for the alar ~

22:31 The NRC Operations Center is notified of the reactor trip in accordance with 10 CFR 50.72. The call was initially misclassified by the licensee as a courtesy call instead of a required notification. The call was made by a shift administrative assistant, not an operato :38(?)

(Exacttimeunknown) The "A" and "B" MFW isolation valves (and SUCVs) are reopened by the operator :50(?)

(Exacttimeunknown) The operator dispatched to the UNPPR reports back to the control room by telephone that the temperature of the "B" and "C" HPI system injection lines, and the crossover line that connects them, is excessively high (more indicative of RCS temperature than the expected temperature of the borated water used for HPI). The smoke detector is believed to have been actuated when tape attached to the HPI piping began to melt and smolder / smoke. It was noted that this event could have gone undetected if the tape had not smoldered and set off the smoke alarm. In fact, this event may have occurred previously and gone undetected. The plant operators suspected that the high temperature in the HPI piping was caused by failure of Check Valve MU-348 to reseat after HPI Pump P36A was secured at 22:01:33. Check Valve MU-348 is located inside the reactor containment building. The leakage flow path was in the )

reverse direction through Check Valve MU-34B and outside the reactor containment building via the "B" HPI injection line, then through the crossover line to the "C" HPI injection line, and back inside the reactor containment building to the RCS. The upstream check valves (MU-1214 and MU-1215 in the "B" and "C" HPI injection lines respectively)

performed as designed to prevent further backflow of reactor coolant into the HPI system pipin :59:41 AnEFICsysteminitiationofemergencyfeedwater(EFW)occursupon unsing low level in the "B" 0TSG. The operators were aware that OTSG levels were decreasing, and were beginning to increase MFW flow at the time of the ED.C system initiation of EF :00:00 EFW is secure Very little, if any, EFW system flow was injected into the OTSGs.

- _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ _ _ _ _ 3

'

. .

'

h H -- ,

,

h

!

p ,

l'

['.

i' 23:05'(?). . .

(Exact' time unknown)' Auxiliary Feedwater Pump P75 (motor driven) is manually started by the operators from the control room.

l 23i15-(?)

! (Exacttimeunknown) The "A" MFW pump is secure January 21,'1989 1:00.(?) . .

, (Exact time unknown) The oncoming shift waste control operator is dispatched to check the piping in the UNPPR but because_of confusion-.

he missed the hot HPI crossover piping.- He erroneously-reported to

<

the~ control room that the pipe appeared to be cooling dow iO2 . .

The NRC Operations Center is notified of the EFIC system.. initiation of EFW in accordance with 10 CFR 50.72. This notification was made

.

' by the unit shift superviso :00. (?) ~.

(Exacttimeunknown) The waste control operator is again dispatched to the UNPPR with better instructions and he correctly determines

that the HPI crossover piping is still ho

5 Reactor Coolant Pumps "A" and "C" are restarted. It appears that restarting the pumps causes Check Valve MU-34B to reseat and/or removed the differential, pressure which was driving the backflow, stopping'the RCS backleakage into the HPI. system piping. The waste control operator confirms that the piping was cooling off at this tim :00 A routine post reactor trip walkdown of the reactor containment -

. building identifies possible reactor coolant system leakag :56 The 1sakage is confirmed to be from an elbow weld in a 11/2-inch drain line off the "B"_ reactor coolant pump suction line. The leaka is believed to be small'(approximately 10 to 20 ml per minute)g An- rate Unusual Event (UE) is. declared by the licensee due to unisolable RCS-pressure boundary leakage. (TS 3.1.6.3 requires cooldown to begin t

'Athin M hours of ident ify ng

.

i t eh leakage.)

,

.13:06 Operators begin the proc'ess;of taking <:he reactor to a cold shutdown conditio ,

__._._--.m.._m_.______.._m__.m _ _ _ ________ - ______ _ _ __ - ______

-

o

'-

,

,

u

13:10 The NRC Operations Center is notified of the declaration of an UE in accordance with 10 CFR 50.72. This notification-is made by the shift administrative assistan . January 22, 1989 17:30 The reactor is at cold shutdown, and the UE is terminate .2 Equipment Problems / Failures Review of the event revealed a number of equipment failures or improper system responses. The equipment problems are listed here and are discussed in detail belo *

The main generator exciter failed due to a 'oroken (fatigued)

electrical connectio *

6.9kV Bus H1 failed to automatically fast transfer from the unit auxiliary transformer to the startup transformer following trip of the main generato '

Two channels of the EFIC system spuriously actuated on low steam generator -level .

The MFW system SUCVs and LLCVs failed to close as designed following the reactor tri *

The "B" MFW pump (P18) failed to runback to minimum speed as designed following the reactor tri *

The "B" MFW main block valve (CV-2675) failed to close as designed following the reactor tri *-

Check Valve MU-34B in the "B" HPI system injection line failed to reseat following termination of HPI flow, allowing RCS to flow into j the HPI system pipin A " pinhole" leak was found in a weld at an elbow in a 1 1/2-inch I diameter drain line connected to the "B" reactor coolant pump suction I'

lin . Main Generator Exciter The cause of the main turbine trip was main generator lockout from a  !

failure of the exciter used to generate the field for the main generator. The generator field was lost when an electrical connection consisting of a thin metal tab broke into two pieces. The metal tab was approximately 4 inches long (welded to the exciter  ;

stationary field windings at one end and bolted to a lead wire at the

,

i

n -- . , .

p,;

o -

-

-

,

})

,

,4 , ,

'

-O t + e

'

l;

, f :., . '

)~ 1

'

[

10-

[ , ,

, ,

' 1

,

s

,

.

, j P other end), 1 1/4 inches wide, and .03; inches thick. It' appears that-

!.1 a stress induced crack in.the tab, ~most likely due to the weight of L' the. lead. wire on one-end in conjunction with the' constant' low level

- vibration that exists when the generator is running.. led to its eventual failure. - Other similar electrical connections in the

'

.

exciter' appear' to be provided with better' support, and therefore l . would be less susceptible to this failure mode. Field service

'. - engineers from Westinghouse. (the exciter manufacturer) were onsite to

!! ensure proper installation and support of the new connection. The exciter is designated as not being a safety-related component.

L 12.2.2- : 6.9kV Bus H1 b; - Following lockout'of the main generator and the loss of power to-plant loads from the unit auxiliary transformer (UAT), 1 nonsafety-related 6.9kV Bus H1 failed to automatically fast' transfer from the UAT to the startup transformer (SUT). The.SUT receives power from the.offsite power system. An automatic fast transfer did occur as designed for the.other 6.9kV bus (H2), and for.the two

' . nonsafety-related 4.16kV Buses Al'and A2. It is.noted that-power to the two. safety-related 4.16kV Buses A3 end A4 is provided from .1 nonsafety-related Buses Al and A2, respectively. .Should power have ' l e been lost to Bus A1 or A2, the corresponding safety-related bus woul : have been isolated, and then powered from its associated emergency y diesel-' generator. . j

,

o The circuitry that' performs the' automatic fast transfer function is j essentially identical for Buses H1, H2, .A1, and A2. Tk. fast transfer J circuits for Bus H1 have been tested several times since the event, J and have performed as designed. The licensee suspects that a General Electric Type IJS: synchronism-check relay' actuated at approximately

>

the same time as the generator lockout and prevented an automatic -

. fast transfer of Bus Hl.. The function of the synchronism-check relay is to complete the associated breaker closing circuit to. permit g 3 transfer only if the frequency difference between the existing bus

- voltage and the incoming / transfer voltage is negligible or zero. The

_

licensee believes that perturbations caused by the exciter failure

• ~ actuated the relay.-

The synchronism-check relays have reset times that can be adjusted by a relay time dial setting (TDS). A TDS of.2 (corresponding to a reset

- time of approximately 2 cycles) was found to exist for the synchronism-check relays in the. transfer circuits for Buses H2, A1,

. and A2. The TDS for-the synchronism-check relay in the transfer circuit for. Bus H1 was found to be 10 (corresponding to a reset time of approximately 20 cycles). The licensee believes that the excessive reset time contributed to the failure to fast transfe An automatic "s1'ow dead bus transfer" of Bus H1 did occur approximately 3 seconds,later. However, by this time Reactor Coolant

,

Pumps "A" and "C," which are powered from Bus H1, had tripped on i

- - --- -

. - - - _ - - - - _ _ ,

.

'

-

.

,

L j undervoltage. The licensee is analyzing the synchronism-check relay reset times to determine the appropriate TDS. The AIT concurs with the licensee's assessment of the probable cause fior failure of nonsafety-related bus H1 to fast transfer. The emergency power source to the safety-related buses were not challenged during the event. .There appear to be no significant safety implications 4 associated with the failure to transfer under normal condition . Spurious EFIC Channel Actuation The sequence of events printout indicates that two redundant emergency feedwater initiation and control (EFIC) system instrument channels spuriously actuated approximately 3 seconds following the turbine trip. The channels tripped on OTSG low water level, although the actual level was above the low level setpoint for EFIC system initiation of emergency feedwater (EFW). The EFW system was not actuated since.the combination of channels that tripped did not satisfy the EFIC/EFW actuation logic. Spurious EFIC system channel actuations have occurred at AN0-1 and at other B&W designed reactors

'in the past. The spurious actuations have been found to be caused by pressure oscillations induced in the main steam lines when the turbine stop valves or main steam isolation valves are closed while the reactor is operating at high power levels. The pressure oscillations propagate back to the OTSGs and affect the level sensing i instrumentation in such a manner to cause the spurious actuation The EFIC system OTSG low level EFW initiation circuits contain 2-second time delays to allow time for any pressure oscillations in the main steam lines to dampen to the point where spurious actuations will not occur. The licensee believes that either the time delays did not function properly, or that the duration of the pressure oscillation exceeded the 2-second delay tim The licensee indicated to the AIT their intent to reanalyze the appropriateness of the 2-second delay time value. It is noted that several different combinations of spurious EFIC system channel actuations would have resulted in automatic initiation of the EFW system. For this event, which involved slight overcooling due to excessive post reactor trip MFW flow, the additional contribution of EFW flow would have made the overcooling more sever . SUCVs and LLCVs The ANO-1 plant design includes circuitry designed to close the MFW system startu valves (LLCV)pflowcontrolvalves(SUCV)andlowloadcontrol upon sensing that the reactor has tripped. Circuitry i is also provided to close the MFW system main block valves and to run the turbine driven MFW pumps back to minimum speed when the reactor trips. The function of these circuits is to stop the continued addition of MFW flow to the OTSGs immediately following a reactor trip to prevent overcooling transients. Once the levels in the OTSGs 4 l

'

>

,

, , .

i h,

g ,

.

C 12 have dropped sufficiently,.the flow control . valves reopen to control 0TSG 1evel at the low level limit # - During normal plant operation, the MFW system pumps and flow control valves are automatically controlled by the integrated control

>

system (ICS). The circuits that sense when the reactor has tripped, and provide signals to the ICS to cause MFW pump runback and SUCV/LLCV closure are the rapid feedwater reduction (RFR) circuit ,

The RFR circuits were added as part of the Babcock & Wilcox Owners Group (BWOG) Safety and Performance Improvement Program.(SPIP) effort to reduce the frequency of reactor trips and reduce the severity of-post trip transients at B&W reactors. Installation of the RFR circuits at AN0-1 was completed during the last refueling outage. A ,

'

simplified: diagram, showing the major components of the MFW system and their. interface with the RFR/ICS is provided.in Figure 2. - ,

Auxiliary contacts operited by the. reactor trip breakers are used to provide inputs to the RFR circuits. Upon sensing that the reactor-has tripped, relays in the.RFR circuits become energized and operate contacts in the ICS MFW control circuits to: . (1) interrupt / block the -

normal demand signal to the SUCV/LLCV electric-to-pneumatic (E/P)

-valve positioners, and (2) to transfer in a new constant demand signal.to the valve positioners that causes the valves to go. close It appears that during the event .the. RFR relays and contacts-functioned properly. However, because of a wiring error.during installation of the RFR modification, the RFR contacts in the ICS were effectively bypassed by the existing ICS control. signals and the valves remained open. The.same wiring error was made twice, once fo each train of MFW system control valve It appears that the-ICS. functioned as designed during the event-except for the RFR function. The cause for.the 'RFR wiring error appears to be related to the existing ICS installation. The ICS consists of several cabinets containing multiple rows of back-piane wired interconnected modules, where the various modules can be physically wired in one of several. different configurations that are electrically the same. This results in an installed ICS for which plant-specific as-installed wiring diagrams do.not exist. The ICS electrical schematic / elementary diagrams were used to install the RFR modification.. The RFR modification as shown on the schematic diagrams is correct; however, using the schematic diagrams in place of wiring diagrams ~ can be difficult when trying to determine all the necessary wiring changes for a modification, and eppears to have led to the wiring error. The cause for the wiring error not being detected is that the post modification testing only consisted of l

continuity checks across the RFR' circuit contacts (added to the ICS)

to verify their proper operation. A system functional test to verify that an open SUCV/LLCV would go . closed upon a simulated reactor trip ,

was not performe ,

- - - - - . _ - - ---

.. . .

__ -

,

~

'

, , -

w

.The AIT team considers.the extent of.the post modification testing that was performed on the RFR modification, and the absence of

'as-installed wiring diagrams to be unacceptable practices for ,

safety-related systems. The .'.IT team recognizes that the MFW system, the..ICS, and the RFR circuits. are all considered to be -

nonsafety-related systems. The RFR modification,1 which enhances the system ability to prevent a serious overcooling event, was one part of an overall modification designed to simplify-and improve the

1 performance of the ICS. The post modification test procedure for the overall ICS simplification modifications' involved a series of i different tests. The test procedure did . undergo several stages of review prior to final approval in accordance with the review process l

for safety-related. systems. The licensee plans-to correct the wiring .

error prior to plant. restart following the even '

2. .MFW Pump I

The RFR circuits also provide-inputs to the ICS to cause the turbine driven MFW pumps.to runback to minimum speed following a reactor

,

trip. ' During the event, the "B" MFW pump (P1B) failed to runback per i

'

design'(which in conjunction with the failed open SUCVs, LLCVs, and

"B" MFW main block valve led to excessive MFW flow to the OTSGs and slight RCS overcooling). The licensee has been unable to determine the cause for this failure. PlantPerformanceAnalysis. System (PPAS)

data shows that the output signal from the ICS to the pump controller

,

'

was correct'(i.e., corresponded to minimum speed), indicating that

. the RFR and ICS circuits. functioned properly. Later in the event, PIB did eventually runback to minimum speed on its own (runback

. occurred approximately 1.1/2 minutes late). The licensee suspects that the malfunction' occurred either within the pump controller or the turbine steam supply. However, these components have been tested several times since the event and were found to be working satisfactorily. The'11censee is continuing their investigation of the failur Although there appear to be no significant safety concerns regarding failure of PIB to runback to minimum speed, it did contribute to operator challenges during this event. Flow from either train of MFW can be manually isolated by the operators via motor-operated isolation valves independent of the ICS/RFR, and either MFW pump can supply flow to both OTSGs via the crosstie line connecting the two train . MFW Block Valve

' The "B" MFW block valve (CV-2675) started to close per design following the reactor-trip, but stopped when the closing torque switch actuated to interrupt the valve control circuit upon sensing mechanical overload while trying to close. The torque switch is adjustable ~ over a' range of torque settings, depending on the valve application. The as found setting of the closing torque switch for G , _

)

.

.

.

1 CV-2675 corresponded to a differential pressure across the valve of approximately 285 psi. The licensee has calculated that the actual differential pressure across the valve at the time of the event was between 285 and 300 psi, thus requiring a torque of greater than the torque switch setting to close the valve. Based on this failure, the licensee is revising the criteria used to calculate torque switch settings and will revise the torque switch settings for the MFW system main block valves. .The licensee stated that they are reviewing other differential pressure calculations used to determine torque switch settings for similar problems. It was noted that the licensee had taken the initiative to include this valve along with several other balance of plant motor operated valves in their valve testing program (not limited to safety-related).

2. Check Valve Check Valve MU-34B in the "B" high pressure injection (HPI) system injection line failed to reseat following termination of HPI flo This valve is the downstream check valve in the "B" injection line and is located inside the reactor containment building. The HP1 injection lines are 21/2 inches in diamete Failure of the valve to reseat allowed RCS flow through the HPI line outside the reactor containment building. The consequences of RCS flow outside the reactor containment building are discussed in Section 2.5 of this repor Following the event, Check Valve MU-34B was removed from the HPI line for inspection. When the valve disc was lifted open by hand and released, the valve would not close solely from gravity acting on the weight of the disc as expected. Additinnal force had to be applied to the top of the disc to cause the valve to close. Disassembly of the valve revealed wear on the bushings that attach the disc hanger to the hanger bracket. This appears to have caused mechanical binding of the disc hanger, preventing its rotation to the closed position. A licensee review of operational history revealed that the HPI check valves had not been subjected to extensive operational use and, therefore, the degree of wear found was not expecte While the AIT was onsite, a second identical check valve (MU-34A from the "A" HPI system injection line) was removed, tested, disassembled and inspected. Although the valve seemed to function properly, close inspection of the valve revealed loose internal parts that could have resulted in eventual mechanical binding. This valve, however, did not show the degree of wear as found on MU-348. The licensee plans to disassemble and inspect the other HPI system check valves relied on for containment isolation prior to plant restart. The licensee deferred identification of corrective actions pending the results of the testing and inspection of the remaining valves. Subsequent to the AIT inspection, the licensee has implemented repairs and vendor recommended modifications for these valve m

.

'

-

..

.

2. RCS Leakage

>

During a routine post reactor trip walkdown of the reactor building, the licensee discovered a " pinhole" leak in a 11/2-inch drain line attached to the suction line of the "D" reactor coolant pump (P32D).

The walkdown is' performed by procedure and must be completed before plant restart. The leak was found in a weld on an elbow of the drain line, u valves)pstream

.

of valve The licensee RBD-8D has (one stated that ofleak the two occurred series locked from aclosed drain transverse crack at the weld' location. The licensee is performing an investigation of the cause of the crack'at the weld. It is believed to be a weld defect caused by lack of weld fusion at this tim .3 Operator Response Contributing to the Event Based on the AIT. team review of this event, it appears that the response of the operators throughout the event was generally good. In fact, prompt operator response prevented a significant event that could have occurred if the OTSGs had overfilled to the point of water carryover to the main steam lines. No operator actions / errors were noted which added to the severity of the RCS back flow event effect on the HPI piping. Although not a' contributor to the event, it was noted that operator logs and shift turnover documentation did not discuss the HPI line high temperature condition. However, a condition report was initiated by the operations l staff to document the condition and to cause e corrective action evaluatio .4 Contribution of Reactor Coolant Pump Trip to the Event During the inspection, the AIT reviewed the sequence of events print out, contacted key licensee operations personnel, reviewed vendor documentation (B&W), and witnessed simulator event scenarios. Through discussion with'

licensee engh eers and a review of vendor documentation it was determined that the two RC pump operation contributed to the backflow of reactor coolant through the unseated check valve, MU-34B. Review of ANO  ;

specification for design RCS flow indicated that for one pump running per j OTSG loop configuration, the differential pressure between loops is 4 j

approximately 15-19 psid. This differential pressure would provide the motive force to create a backflow through the unseated check valv l However, the pressure differential was probably not sufficient to reseat t the Check Valve MU-34B. With all four RCP pumps operating, the differential pressure is negligibl .5 Licensee Evaluation of Event on Piping and Reactor Containment Building

Integrity In response to the January 20, 1989, event at ANO-1, the licensee has developed an action plan the purpose of which is to evaluate the significance and the root cause of the event and to determine the necessary corrective actions to minimize the potential for recurrence of this even I

!

_ _____

e

-

.

The licensee's action plan includes an assessment of the effects of the event on piping and reactor _ containment building integrity. This is because the piping system upstream of the failed Check Valve MU-34B has experienced temperature higher than its design temperature during this even In a walkdown for visual inspection of this piping, the licensee had not identified any evidence of. deformation or damage of concern in the piping '

system. The licensee asserts that the piping system upstream of the Check Valve MU-34B may have experienced a temperature less than 300 F during this event. This estimation is based on the circumstantial evidence including the data obtained.from the equipment qualification test report on some flexible conduit found touching the crossover piping. However, the licensee assumed a temperature of 545'F which was the cold leg temperature at the time of the event initiation, and also assumed rigid piping supports to estimate the potential effects on piping and reactor containment building penetration integrity. The results of licensee' rough calculations indicate that there are several locations of this piping system which have exceeded the ASME code allowable limits. The licensee is in the process of. performing a detailed piping stress calculation for its assessment of the piping integrit During its evaluation of the piping stress calculation, the licensee has found that the actual temperature assumed in the existing piping stress calculation was 145 F. This deviates from the design temperature of 300'F documented in Table 6-3 of FSAR for ANO-1. The licensee stated that they will correct this problem in a later amendment to the FSA The licensee is also in the process of performing a steady-state heat transfer calculation to bound the piping temperature profile at various check valve leakage rates. In response to the AIT concern regarding reactor containment building integrity, the licensee has stated that this effort will include an evaluation of reactor containment building penetrations of the high pressure safety injection lines:to verify the effects of this i event on reactor containment building integrit The licensee is continuing evaluation of the event on piping and reactor containment building integrity. This effort was not completed at the end of the AIT inspection. Subsequent to the AIT inspection, the licensee determined that certain high stress locations of the HPI and crossover piping would be replaced. This includes several pipe elbows and one tee, but no actual piping runs are being replace .6 System Desion Features Which Contributed to the Event l ANO-1 plant design incorporates three primary makeup pumps (P36 A, B, andC). Pump B provides nonnal RCS makeup and seal water supply to all four reactor coolant pumps (RCPs) during normal plant operations. Pumps A and C are normally in standby to serve the high pressure safety injection function. Pump A is aligned to inject borated water to the RCS at the discharge side of RCP C and D. Pump C is aligned to inject borated water

.

,

.

I to the RCS at the discharge side of RCP A and B. In each of the high pressure injection lines, there is a nonnally closed motor operated control valve, a check: valve and a globe throttling valve outside the containment and a check valve inside containment upstream of the nozzle and RCS loop. The standby primary makeup pumps and the motor operated control valves are designed to automatically actuate upon a safety-injection signal or~to be manually initiated by operator actions for high pressure safety injectio The high pressure injection (HPI) discharge piping-configuration was modified in 1979 as a resolution to an emergency core cooling system t performance concern.- In order to mitigate a loss of coolant accident (LOCA) considering ~a break location at the RCP discharge piping coincident with a single failure of a class IE power _ supply train, cross connect lines were installed between the high pressure injection lines to assure sufficient. injection flow paths for reactor cooling. These cross ,

connect lines are located outside the reactor containment building in the penetration room upstream of the globe throttling valves. One crossover-line connects high pressure injection for RCS Loops B and C and the other crossover line connects high pressure injection for RCS Loops A and ~

The design features which the AIT team believes were major contributors to this event are summarized below:

The lack of a monitoring system to detect excessive check valve -

leakage contributed to this event.- Also, the check valves (MU-34A, B, C, and D)' do not have leak test requirements. The previous visual inspection and vertical stroke of the valve disc during the last outage did not reveal the excessive wear problem and subsequent leakage of the valv The cross connect lines of the HPI discharge piping upstream of a single check valve allow a flow path for reactor coolant through a leaking check valve to piping outside the reactor containment building. Plant operation with less than four reactor coolant pumps operating creates the necessary differential pressure for the back flow event.

l HPI piping system design at ANO-1 did not provide for the high temperature condition created by the back flow even .7 Discussion on Potential for Reoccurrence of This Event The AIT-team believes that the present plant / system design provides a high potential for recurrence of this event. Design modifications are believed necessary to prevent recurrence of this even .0 Findings of Fact The AIT members reached the following findings of fact:

• A reactor trip occurred at ANO-1 on January 20, 1989, from a turbine tri _ - _ _ _ _ _ ____-___

,

'

.

An installation error was made during modification of the integrated control system (ICS) for rapid feedwater reduction (RFR) at the last refueling outag A calculation assumption error was made for motor operated valve torque switch settings of a main feedwater block valve which resulted in the vcive failing to close following the tri The above noted errors contributed to main feedwater control problems l following the reactor tri *

Feedwater control problems resulted in an overfill of one steam generator and slight overcooling of the RC High pressure injection (HP1) was initiated manually by the operators following the event and injection occurred for about 2 minutes through the safety injection flow path A nonsafety-related electrical bus failed to fast transfer causing two of four reactor coolant pumps to tri When the HPI flow was stopped, a check valve in the "B" HPI line i failed to resea !

The open check valve and the reactor cooling pumps running configuration caused a back flow of reactor coolant outside the reactor containment building through an HPI crossover lin The licensee made the required notifications to the NRC during the series of occurrences following the reactor trip. However, the notification of the reactor trip was initially misclassified by a licensee shift administrative assistant as a courtesy call instead of a required notificatio '

"

The HPI piping upstream of the check valve inside of the reactor containment building and the crossover line was not designed for the RCS temperature experienced during the back flow even * The licensee's initial evaluation of the HPI piping indicates that certain locations were stressed beyond ASME code allowables by the high temperatur No visible damage to the HPI piping or supports was found by the license Licensee evaluation of this event is continuing. Piping system repairs and/or modifications may be required based on the licensee evaluatio _ - - _ - _ _ - _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

p, m.

o ,

'

.

'

=

.

'

'

I.

l'

4.0 Conclusions

'

Operator actions.following the trip were generally ' good and prompt L  ; action prevented a; serious overfill event where the. main steam lines could have been floode The-AIT members concluded that this backflow event could occur again

,at ANO-1 with the present plant desig *

The AIT concludes that design modifications will be required to prevent this back flow event from recurrin *

-The AIT concludes that followup inspections should be performed to evaluate the: adequacy of licensee actions to resolve the problems-identified. Also, NRR review and approval of licensee proposed

~

actions, including proposed design modifications, should occur

. prior to unit restar .0J Persons Contacte A. C11nkingbird, Senior _ Reactor Operator

'*A. Cox,.0perations Superintendent

!*M. Durst, Project Engineering Supervisor

• B. Eaton, Manager, Mechanical' Civil, Structural Desig *E. Ewing,' General Manager, Plant Support

• D. Howard.-Licensing Manager

*L. Humphrey, General Manager, Nuclear Quality

_

• G. Jones, General Manager, Design Engineering

• R. Lane, Manager, Engineering

• J. Levine, Executive Director, Nuclear Operations _

• D. Lomax, Supervisor, Plant Licensing

• J. McWilliams, Manager, Maintenance

• P. Michalk, Licensing Engineer D. Olson, Senior Reactor Operator ,

• S.- Quennoz, General Plant Manager-L. Rushing, Lead Shift Technical Advisor

• S. Smith, Installation Superintendent

• L. Taylor, Plant Licensing

'C. Thompson, Shift Supervisor

• J. Vandergrift, Manager, Operations

• J. Veglia, Senior Startup Engineer R. . Walter, Reactor Operator

• R. Wewers, Manager. Work Control Center

• C. Zinnerman, Operations Technical Supervisor

• Denotes those persons who attended the exit interview on January 25, 198 The NRC Senior Resident Inspector and Resident Inspector also attended the exit intervie _ _-__-___ - -_ _ _ __- - - _ _ _ _ - _ _ _ _ _ _ _ _ - -__ __-_ -____ _ ___ __ . - - - _ - _

- 6 6 GS

_

3 I S g P P HE R

T O

' , %3 P H ) L gL T M J 3A AA P R G

A 4 O

'h I

T 0 D N L 2 I GE F 1 NM D '

V

- EI N E C GPI M I AIA E F 9 0 7 8 KPT T I 1 2 2 2 A N S L 2 42 EMO

'2 I2 Y P S M I

V

- >V 1

- P UW 17- 1 V

r-V LEC KT CSE AYD S C C EO C C N (

KL 4 5 BSI O AF S 1 1 1 M SIT 1 2 '

M 2 1 1 CPU C

E E

T 1

-

,

A' U

-

U

-

' RHO J S U 2 M M N Y M 1 I S

E R

U T

N A

2h U

-

M '

S L VA " l

"

T S O

'E R C O t" N E

M N

P _

I L[ -

l l l l I l! l 1 l l g l l l I R C D 8 A A H O 4 4 ' 4 8 4 T G T 3 3 3 N I

H A C

U

- h' 3U -

,

U

-

U

- OB C

E M M M M R

-

O E -

N H A,T +

-

-

"

) ) )

D G E N ' N E CP D BI AP R

U 2P 2N - 2N 2P I

F G 3I PR T

(

C 3N P

C R

( 3N PU R

(

3I PR (

O T

C A

E R

L A

QA B G

G S S T T O O

-

-

l l

!f I.j) Ii !' ,1 l ;);I1I! ,l!

--

.

' ' -

-

-

G G

-

S S T A T B P O O MI OR RT F S -

-

RR SOE -

R TTK RN _

E UCA EO -

RT PAE 0 TIS 0 EA NER 8 NATE 3 TW IRB 6 IWAV 6 AD 2 ADLL 2 _

WE -

MEOA -

DE V ESV V EF C FI C E _

F ' O

-

s i g i a , : 8 l , e ' a I s 8 8 I

,ei'

N) _

I M_ _, ! , : I i g , I l I i j:

s_

DA _

,i , e 1e I MR ___ s_ _

P G __

_ _

_

A A RN _

_

RHI EOS __ _ _

_

TD TIT __ _

_

_

_

I ATI 2 2 DW DWCU __ _ 2 _ 7 N D IDUC

__

6 _ 6 A E PEDR _ 2 _

EI AEEI ___ - -

CF RFRC V _ V MAI ___ C _ C EFL ___ _

TRP 3 5 3 5 SEM __ _ 2 2 7 7 YTI __

_ 6 6 6 6 SNS _ 2 2 2 2 I ( __ - - -

-

_ V

__ = V V V L _ C C C _ C OTS __ _ _

_

RI T __ _

.

.

TUN NCE

_

_

4_ 4 _

-

._

__ 2_ 7 -

ORN CI O _

_

_

_

6_ 6_ .

_

2 _

CP M

___

__

_ 2_- -

V _

-

D O _ _

_

V_ C _

ENC .l _ _ _ C_

_

TO _ _

AI il

.lsg u_ _ _ _

RTM l

_

_ _

_

GCE .' : 8 :

8 : t i i i 1 L

_

EUT .' _

_

TDS D  : 8 I e i 8 i i8 I : t NEY E . ' 8 e 8 l , , 1 I t 1 ,! * llIt _

I RS T . j e I i e i g 8 il 8 ' I e ,! s s I s 8 ' 8 l m AL ROM _

- GRE) _

ETTS . ' e ' 8 ' 8 I e g l t ' ,,! e ' i'

fU

TNSC NOYI ICSC

k J

-

-

_

E 2 R . ',,!

- 8 U - 2 G .la - - _

I

- - V F C

- -

= ?i i:- -

-

-

_ ,

-

_

-

N _

ra e e I - ,i I i '8 8 8 e I O _

8 8 ,!

J, I R ,

ST r's , i i g I i ' 8 8 ; ,

E TUA NTM # TS AP

/' B AAR A 1 M 1 LTO P DU P PSF P N E I F

_

_

_

_

_

_

_

_