ML17311B121

LER 95-005-00:on 950717,RT on Low SG Water Level Was Result Following Degradation of MFW Flow.Completed Evaluation of Event

ML17311B121
Person / Time
Site:	Palo Verde
Issue date:	08/16/1995
From:	Grabo B ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR
To:
Shared Package
ML17311B120	List: ML17311B119 ML17311B121
References
LER-95-005-01, LER-95-5-1, NUDOCS 9508230041
Download: ML17311B121 (11)

Text

LlCENSEE EVENT REPORT (LER)

ACIUTYNAME (1) DOCKET NUMBER (2) PAGE (3) oF Palo Verde Unit 2 0 5 0 0 0 5 2 9 1 0 7 ITLE (6)

Reactor Tri Followin the De radation of Main Feedwater Flow EVENT DATE 5 LER NUMBER 6 REPORT DATE 7 OTHER FACILITIES INVOLVED 8 MONTH DAY TEAR YEAR SEOUENTIAL REVISION MONTH DAY TEAR FACIUTYNAMES 1 NVMSERS NUMBER tIVMBER NIA 0 5 0 0 0 0 7 1 7 9 5 9 5 0 0 5 0 0 0 8 1 6 9 5 NIA 0 5 0 0 0 REPQRT Is sUBMITTED PURsUANT To THE REQUIREMENTs oF 10 cFR C (check one or mora ot the toltovvrno) (11)

MODE (0) 20A02(b) 20.45(c) 50.73(a)(2)(rv) 73.71(b)

POWER 20.405(a)(t) 0) 50.36(c)(1) 50.73(a)(2)(v) 73.71(c) 20.45(a)(1)(ri) 50.35(c)(2) 50.73(a) (2)(vti) OTHER (Specify in Abstract LEVEL<<0) 0 0 20,45(a)(1) (iii) 5073(a)(2)O) 50.73(a)(2)(vib)(A) beiovv and in Text, NRC Form 20.405(a)(t) Ov) 50.73(a)(2) 50.73(a) (2) (vtii)(B) 366A) 20.45(a)(1) 50.73(a)(2)(x)

<i'0.73(a)

(v) (2) (iii)

LICENSEE CONTACT FOR THIS LER (1 2)

AME E NUMBER Burton A. Grabo, Section Leader, Nuclear Regulatory Affairs 6 0 2 3 9 3 - 6 4 9 2 COMPLETE ONE UNE FOR EACH COMPONENT FAILURE DESCRIBED IN TI6S REPOR T(13)

MANUFAG. REPORTABLE . MANUFAG. REPORTABLE TURER TO NPRDS TURER TO NPRDS MPX No SUPPLEMENTALREPORT EXPECTED(te) EXPECTED DAY YEAR SUBMISSCN YES ol yea. comptete EXPECTED SUBMISSCN DATE)

TllxcrNJlel e N00 aopN, I ~, wwrelewy r On July 17, 1995, le ~~ ~at Inelr (ei approximately 2331 MST, Palo Verde Unit 2 was in Mode 1 DATE (1 5)

(POWER OPERATION), operating at approximat:ely 100 percent power when a reactor trip occurred when Steam Generator Number 2 (SG-2) water level reached the Reactor Protection System (RPS) trip setpoint for low SG water level following the degradation of main'feedwater (FW) flow. Immediately following the reactor trip an Engineered Safet:y Feature Actuation System (ESFAS) actuation of both Auxiliary Feedwater Actuation Systems (AFAS) was received on SG-1 and SG-2 low levels. The loss of FW flow was the result of a momentary power loss (from NNN-Dll) to the FW control system (FWCS) during a switching operation of the non-Class 1E 13.8 kV (NAN-S05) switchgear from the normal power supply (startup transformer X01) to the 'alternate power supply (startup transformer X02) in order to take startup transformer X01 out of service for maintenance.

When the switching operation was carried out, the breaker indications for startup transformers X01 and X02 did not change. Plant multiplexer (PMUX) indication problems led the operating crew to believe that the switching operation did not occur. Concerned about an inadvertent breaker closure, the operating crew reset the alternate supply breaker X02. This resulted in the deenergization of NAN-S05, NAN-S03, and the class 1E 4.16 kV bus (PBA-S03).

The root cause for the Unit 2 reactor trip was determined to be a failure of the initial design to consider FWCS responses to momentary power interruptions.

An evaluation of the adequacy of NNN-Dll to support the FWCS and steam bypass control system (SBCS) is ongoing. Any corrective actions identified will be tracked under the APS Commitment Action Tracking System.

Previous similar events were reported pursuant to 10 CFR 50.73 in LERs 528/95-008, 530/94-007, 530/94-005, 530/93-001 and 529/92-001.

9508230041 950816 PDR 'ADO(."K 0500052tyi S PDR

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION ACIUTYNAME DOCKET NUMBER LER NUMBER PAGE SEQUENTIAL EVISIO NUM8ER  ; NUMSER Palo Verde Unit 2 0 5 0 0 0 5 2 9 9 5 0 0 5 0 0 0 2 OF 0 7 EXT 1 ~ REPORTING REQUIREMENT:

This LER 529/95-005 is being written to report an event that resulted in an automatic actuation of an Engineered Safety Feature (ESF), including the Reactor Protection System (RPS) as specified in 10 CFR 50.73(a)(2)(iv).

Specifically, on July 17, 1995, at approximately 2331 MSTg Palo Verde Unit 2 was in Mode 1 (POWER OPERATION), operating at approximately 100 percent power when a reactor trip occurred when Steam Generator Number 2 (SG-2) (AB) water level reached the Reactor Protection System (RPS)(JC) trip setpoint for low SG water level following the degradation of main feedwater (FW)(SJ) flow. Immediately following the reactor trip an Engineered Safety Feature Actuation System (ESFAS)(JE) actuation of both Auxiliary Feedwater Actuation Systems (AFAS)(JE, BA) was received on SG-1 and SG-2 low levels. The loss of FW flow was the result of a momentary power loss to the FW control system (FWCS) during a switching operation of the non-Class 1E 13.8 kV (NAN-S05)(EA) switchgear from the normal power supply (startup transformer (XH4R) X01) to the alternate power supply (startup transformer X02) in order to take staztup transformer X01 out of for maintenance. 'ervice When the switching operation was carried out, the breaker indications for startup transformers X01 and X02 did not change. Plant multiplexer (PMUX) indication problems led the operating crew (utility, licensed) to believe that the switching operation did not occur. Concerned about an inadvertent breaker closure, the operating crew reset the alternate supply breaker X02.

This resulted in the deenergization of NAN-S05, NAN-S03, and the class 1E 4.16 kV bus (PBA-S03)(EB). The loss of power to the class 1E 4.16 kV bus resulted in a valid ESFAS signal starting the Train A Emergency Diesel Generator (EDG)(EK) ~ Required plant equipment and safety systems responded to the event as designed. No other safety actuations occurred and none were required. By approximately 0000 MST on July 18, 1995, the plant was stabilized in Mode 3 (HOT STANDBY).

2 ~ EVENT DESCRIPTION:

On July 17, 1995, at approximately 2315 MST, Unit 2 was in Mode 1 (POWER OPERATION) at approximately 100 percent power when Control Room personnel (utility, licensed) were preparing to transfer the non-Class 1E 13.8 kV switchgear (NAN-S05) from the normal power supply (startup transformer X01) to the alternate power supply (startup transformer X02) in order to take startup transformer X01 out of service for maintenance. When the, Reactor Operator (RO) (utility, licensed) closed the alternate breaker, there was no indication from the main control board that either breaker position had changed, (normal or alternate). After conferring with the Control Room Supervisor (CRS) (utility, licensed), believing that the alternate breaker had not closed and concerned about an inadvertent breaker closure, the RO opened the alternate breaker which resulted in the deenezgization of NAN-S05, NAN-.S03, and the Class 1E 4.16 kV bus (PBA-S03). The loss of power to the class 1E 4.16 kV bus resulted in a valid ESFAS signal starting the Train A EDG which restored power to PBA-S03.

LICENSEE EVENT REPORT (LER) TEXT CONTINuATION AGILITYNAME DOCKET NUMBER LER NUMBER PAGE

'EQUENTIAL EVISIO NUMBER NUMBER Palo Verde Unit 2 0 5 0 0 0 5 2 9 9 5 - 0 0 5 0 0 0 3 OF 0 7 EXT When the alternate power supply breaker X02 was opened, NNN-D11 (power supply for FWCS and steam bypass control system (SBCS)(JI)) recognized the undervoltage condition on bus PBA-S03 and the undervoltage relay dropped out which forced the transfer switch to transfer to the "Normal" power source. (Note: NNN-D11 was lined up to the "Emergency" power supply per Operation's procedures, as the preferred power source.) The automatic transfer switch is a break before make switch which caused the output voltage to drop for approximately 0.5 second.

This voltage drop caused the FWCS to go to manual operation and SBCS to go to emergency off without any automatic functions. The FWCS went to zero output which resulted in the main FW pumps going to minimum speed and the economizer flow control valves closing.

At approximately 2331 MST on July 17, 1995, the Unit 2 reactor (AC) tripped when SG-2 water level reached the RPS trip setpoint foz low SG water level following the degradation of main FW flow.

With the SBCS in emergency off (without any automatic functions) reactor coolant system (RCS) temperature and SG pressure were controlled by a main steam safety valve (MSSV)(SB, RV) and atmospheric dump valves (ADVs)(SB) until the SBCS was available for use. Required plant equipment and safety systems responded to the event as designed. No other safety system actuations occurred and none were required.

The SS diagnosed the event as an uncomplicated reactor trip. At approximately 0000 MST on July 18, 1995, the plant was stabilized in Mode 3 (HOT STANDBY).

3. ASSESSMENT OF THE SAFETY CONSEQUENCES AND IMPLICATION OF THIS EVENT:

This Unit 2 reactor trip can be classified as a Loss of Normal Feedwater which is an infrequent event of the "decreasing heat removal by the secondary system" category and is bounded by the limiting event for this category which is the Loss of Condenser Vacuum (LOCV). Additionally, equipment and systems, assumed in Safety Analysis, were functional and plant response was normal for the situation that occurred. Scenarios defined in Updated Final Safety Analysis Report (UFSAR) Chapter 15 and design assumptions of the reactor protection system are bounding for this Unit 2 reactor trip. Scenarios defined in UFSAR Chapter 6, concerning Loss of Coolant Accidents (LOCA), were not challenged during this transient.

The reactor coolant system (RCS)(AB) pressure increased to approximately 2275 pounds per square inch absolute (psia) during this event. The peak pressure criteria of 110 percent of design (2750 psia) was not challenged during this RCS pressure transient. The steam generator peak pressures of 1258 and 1260 (SG-1 and SG-2 respectively) psia were reached at approximately 2332 MST. The main steam safety valve operated as designed to maintain SG pressure until the atmospheric dump valves (ADVs) were used to maintain SG pressure. The ADVs were used until the SBCS became available to maintain SG pressure. The maximum SG pressure was below the 110 percent of design pressure (1397 psia),

I

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION AGILITYNAME DOCKET NUMBER YEAR

'ER NUMBER SEQUENTIAL EVISIO PAGE NUMBER ~ NUMBER Palo Verde Unit 2 0 5 0 0 0 5 2 9 9 5 0 0 5 0 0 0 4oFO 7 EXT The transient did not cause any violation of the Specified Acceptable Fuel Design Limits (SAFDLs). This event did not result in any challenges to the fission product barriers or result in any releases of radioactive materials'herefore, there were no adverse safety consequences or implications as a result of this event. This event did not adversely affect the safe operation of the plant or the health and safety of the public.

CAUSE OF THE EVENT:

An incident investigation for the Unit 2 reactor trip is being performed in accordance with the APS Corrective Action Program. The cause for the Unit 2 reactor trip was that the initial design did not consider the FWCS responses to momentary power interruptions (SALP Cause Code B: Design, Manufacturing, Construction/Installation). The FWCS design does not ~

provide immediate transfer of power upon loss of power. The current plant configuration is in accordance with design; however, the 500 milliseconds that it takes to transfer power is not adequate to ensure that the FWCS is not interrupted.

In addition to the root cause there were two contributing factors that led to the loss of power on NNN-D11 and subsequent reactor trip and AFAS:

1. The failure of the PMUX system provided inaccurate breaker information to the control room staff. The temperature sensitive equipment in PMUX=

had failed due to high temperatures in the 25RMT cabinet, which was the initiating cause. There are no control room alarms associated with a RMT failure and the Operations Computer Systems personnel, responsible for the PMUX equipment, were unaware of the high temperatures in the 25RMT cabinet.

2. When the expected response was not achieved during the switching operation, PMUX system was providing inaccurate information, the RO conferred with the CRS (believing that the breaker might inadvertently close), and took actions with the intention to prevent parallel sources for the bus and personnel injury of the Auxiliary Operator sent out to inspect the breaker. However, had the RO checked additional indications that exist in the control room, the true condition of the power source for the bus could have been ascertained. P No unusual characteristics of the work location (e.g., noise, heat, or poor lighting) directly contributed to this event. There were no procedural errors which contributed to this event.

If the evaluation results differ from this determination or if information is developed which would affect. the readers understanding oz perception of this event, a supplement to this report will be submitted.

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION ACIUTY NAME DOCKET NUMBER LER NUMBER PAGE YEAR SEOUENTIAI. EVISIO NUMBER NUMBER Palo Verde Unit 2 0 5 0.0 0 5 2 9 9 5 0 0 5 0 0 0 5 OF 0 7

5. STRUCTURES, SYSTEMS, OR COMPONENTS INFORMATION:

On July 14, 1995, at approximately 1530 MST, remote multiplexer terminal (RMT) cabinet 25 failed due to excessive heat in the 25RMT cabinet. This failure froze all of the NAN-S05 input/output signals in their present state without the control room operators knowledge. This failure was determined to be a causal factor in the reactor trip. A description of the PMUX system is provided in Section 8 "Additional Information" of this LER.

On July 11, 1995, one of the air conditioning units that provide cooling for 25RMT was identified to be out of service. A work request was written and repairs were scheduled to begin on July 18, 1995. The 25RMT room has two air conditioning (AC) units to provide cooling, however, recognized that during this time one AC unit was unable to provide it was not sufficient cooling.

On July 13, 1995, an OCS technician found that 25RMT (multiplexer for NAN-S05) had experienced a communication error which was caused by a intermittent failure of the PCM. The technician performed a download and was able to clear the communication error, and 25RMT was brought back on line.

Following the reactor trip, the investigation showed that on July 14, 1995, 25RMT had another communication error at 1430 and 1530 MST. Apparently 25RMT was able to perform a download at 1430 successfully and reboot itself, however, at 1530 25RMT failed due to high temperature in the 25RMT cabinet.

Subsequently, all of the NAN-SOS input/output signals froze in their present state without the control room operators knowledge. This led to the control zoom staff believing that the breaker manipulations taken did not occur when in fact they did.

The investigation conducted as a result of this event revealed that the overall PMUX systems availability since 1992 is 98.5 percent. In 1994 and 1995 the PMUX systems availability was 99.5 percent.

There are no indications that any other structures, systems, or components were inoperable at the start of the event which contributed to this event.

No failures of components with multiple functions were involved. All safety system actuations that were required actuated as designed.

6. CORRECTIVE ACTIONS TO PREVENT RECURRENCE:

The evaluation of the event has not been completed to date. Any corrective actions identified will be tracked under the APS Commitment Action Tracking System.

On July 18, 1995, the PMUX equipment malfunctions were troubleshot. The supply breaker to 25RMT's air conditioner was repaired, PCM-99 card was replaced, and 25RMT was placed in service. A walkdown of all the air conditioners for RMTs was completed and found no additional concerns.

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION ACILITYNAME DOCKET NUMBER LER NUMBER PAGE YEAR . SEQUENTIAL EVISIO NUMBER NUMBER Palo Verde Unit 2 0 5 0 0 0 5 2 9 9 5 0 0 5 0 0 0 6 OF 0 7 I'EXT As an interim corrective action, a functional check of the PMUX status will be performed during the normal working day by Operations Computer System (OCS) technicians (utility, nonlicensed).

Long term corrective actions for the PMUX system being considered are:

~ Develop a remote indication monitoring system accessible to the Unit 1 Control Room Operators. To be completed by October 17, 1995.

~ Establish local monitoring of temperature in the PMUX cabinets.

To be completed by September 19, 1995.

~ OCS will install a monitor of the PMUX system in the OCS maintenance shop to ensure that switchyard PMUX systems are properly monitored. To be completed by October 17, 1995.

On July 19, 1995, a night order was issued to inform all Operations personnel that Auxiliary Operators are to be dispatched to the switch yard to observe evolutions of the nature until it can be demonstrated that the multiplexer problems have been corrected. Additionally, when anticipated results are not achieved following an "Action" step, and plant conditions permit, an investigation should be initiated prior to taking additional actions.

On July 21, 1995, a night order was issued to inform-all licensed Operations personnel that described the potential for a loss of function from both the PCS and SBCS during a transfer of NNN-D11.

This event will be presented to operators during the requal cycle in industry events. Additionally, Nuclear Training is to review the lesson plans and clarify how the automatic bus transfer for NNN-D11 works. To be completed by November 17, 1995.

Long term corrective actions for NNN-Dll being considered aze:

~ Evaluate existing design adequacy and modification options to support FWCS and SCBS during momentary power interruptions in accordance with the existing PVNGS Design Modification Process.

Present any proposed design modifications to the Plant Modification Committee (PMC) for approval by December 29, 1995.

The PMC will determine modif ication.

if the evaluation warrants a design Zf the evaluation results differ from this determination or if information is developed which would affect the readers understanding or perception of this event, a supplement to this report will be submitted.

LICENSEE EVENT REPORT (LER) TEXT CONTINUATION AGILITYNAME DOCKET NUMBER LER NUMBER PAGE YEAR S MUEN IIAI EVISIO NUMBER NUMBER Palo Verde Unit 2 0 5 0 0 0 5 2 9 9 5 0 0 5 0 0 0 7 OF 0 7 7 ~ PREVIOUS SIMILAR EVENTS:

Reactor trips attributed to a Feedwater Control System (FWCS) malfunction have been previously reported in LERs 528/95-008, 530/94-007, 530/94-005, 530/93-001 and 529/92-001. However, in the previous events, the cause of the specific FWCS component failure and corrective actions taken were not the same as in this event and would not have prevented this event.

However, there is some indications that had previous issues been properly addressed this event may have been prevented.

8. ADDITIONAL INFORMATION:

On July 19, 1995, at 1344 MST, Palo Verde Unit 2 was in MODE 1 (POWER OPERATIONS) and synchronized to the grid.

The following is a description of the PMUX system manufactured by Teledyne Controls.

The design function of remote multiplexer terminal (RMT) cabinet 25 in the plant multiplexer (PMUX) system is to gather information of various plant analog/digital instrument, input signals of the switchyazd and transmit them to the local multiplexer terminal (LMT) so that indications of switchyard breakers can be monitozed by Control Room personnel.

There are no fails, control room alarms associated with a RMT failure.

the central master control unit simply drops that RMT out of the If a RMT loop and continues operation with the remaining RMTs. There aze a total of five RMTs per unit, plus an additional RMT that provides switch yard information to the Transmission Control Center and Salt River Pro)ect (load dispatchers).

Three of the five RMTs are'sed for indication and control of the cooling tower (KE) fans (one RMT per cooling tower), the remaining RMTs are used for indications on NAN-S05 and NAN-S06 (one RMT pez bus).

It should be noted that the PMUX system is configured such that control signals for the switchyard breakers are transmitted through normal cabling while switchyard indications such as breaker position, bus voltage, and amperage rely on multiplex signals.

V 4