Text

Insp Repts 50-498/93-22 & 50-499/93-22 on 930628-0702. Violations Noted.Major Areas Inspected:Circumstances Surrounding Loss of Spent Fuel Pool Cooling on 930613-14
	ML20056C976
Person / Time
Site:	South Texas
Issue date:	07/23/1993
From:	Johnson W; NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV)
To:	;
Shared Package
ML20056C974	List: ML20056C973; ML20056C975; ML20056C976;
References
	50-498-93-22, 50-499-93-22, NUDOCS 9307300175
	Download: ML20056C976 (10)
	v • d • e

__ ___ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ . __

. .. .

APPENDIX B

.U.S. NUCLEAR REGULATORY COMMISSION

REGION IV

NRC Inspection Report: 50-498/93-22

50-499/93-22

Operating Licenses: NPF-76

r,PF-80

Licensee: Houston Lighting & Power Company

P.O. Box 1700

Houston, Texas 77251

Facility Name: South Texas Project Electric Generating Station,

l

Units 1 and 2

l

l Inspection At: Matagorda County, Texas

Inspection Conducted: June 28 through July 2, 1993

Inspectors: Mark A. Satorius, Project Engineer

Approved: /b I bI 7/C /93

W. D. JohnsonT Chief, Project Section A Dafe

l

Inspection Summ3ry

Areas inspected: A special inspection was conducted to determine the

circumstances surrounding the loss of spent fuel pool (SFP) cooling on

June 13-14, 1993.

( Results:

!

Two examples of operators failing to follow station procedures were

identified:

o An oncoming operations shift failed to conduct an adequate review

of plant status prior to assuming the shift (Section 3.2.1).

o A reactor plant operator failed to note that the noise level in

the area of the SFP pumps and heat exchangers was significantly

reduced following the isolation of component cooling water to the

SFP heat exchanger (Section 3.2.2).

These two examples of failing to follow procedures resulted in the

13-hour duration of a loss of SFP cooling event,

,

9307300175 930723

PDR

P"

ADDCK 05000498 5

G PDR f-. J'

..- . -. . - - . . _- . . -

-

.

4

-2- -i

,

I

The failure of both the off-going and oncoming shifts to identify the

loss of SFP cooling during routine control board walkdowns were

considered to be significant operator performance weaknesses.

The failure to initiate a station problem report and to take adequate

corrective action following deficiencies identified during a previous ,

safety-related Class IE distribution panel power supply transfer was a

violation of the requirements of 10 CFR Part 50, Appendix B,

Criterion XVI (Section 3.2.3). ;

t

Summar_y of Inspectinn Findings:

Violation 499/9322-01 was opened (Sections 3.2.1 and 3.2.2).

L

Violation 499/9322-02 was opened (Section 3.2.3).

Attachments and/or Enclosures:

Attachment 1 - Persons Contacted and Exit Meeting I

l

i

F

1

i

1

)

.- . .. -- - .=. _ . - - _ . - -

.

'l

,

-3-

P

DETAILS

!

i

1 BACKGROUND .

t

On June 13,1993, Unit 2 was in Mode 6, with all fuel off-loaded from the core

and stored in the SFP. The SFP cooling pump and Heat Exchanger 2A was in i

service cooling the SFP and was being supplied cooling water by component '

cooling water Train A. SFP temperature was 99oF, which was 41oF less than the

Final Safety Analysis Report limit of 140 F, and had bcan steady for several ;

weeks. Operating crews were on an outage rotation that consisted of 12-hour

shif ts, with shift changes taking place at 7 a.m. and 7 p.m. '

2 EVENT DESCRIPTION

The inspector reviewed the operational aspects of the events surrounding the

loss of SFP cooling.

2.1 Electrical Power Supply Shifts

f

A corrective maintenance activity scheduled for the day shift on June 13,

1993, consisted of shifting the power supply for 120 volt AC class lE

Distribution Panel DP001, from its normal _ safety-related '

Inverter / Rectifier EIV001, to the standby Regulated Transformer ERV001A.

Because of the configuration of Distribution Panel _DP001 and its two sources

of power, the transfer involved a break-before-make operation that left

Distribution Panel DP001 de-energized for a short period before power could be i

restored. This activity was planned following the earlier identification-by i

licensee personnel of a hot terminal lug on Inverter EIV001 through a routine !

thermography inspection. f

i

Prior to the power supply shift taking place, operators reviewed !

Procedures OPOP02-AE-0004, "120 VAC ESF Vital Distribution Power Supplies,

Revision 0, OPOPO4-EE-0002, " Loss of 120 VAC Class-lE Vital Bus," Revision 0, .

and SE542EL8000 " Unit 2 Electrical Load List," Revision 0, in order to ensure {

that any unexpected electrical perturbations resulting from the break-before-- l

make transfer would be identified. In addition to reviewing these procedures,

the shift supervisor also reviewed two uncontrolled database information t

systems available to operators on- personal computers: (1) unit outage notes ;

that utilize the database in the licensee's equipment clearance order-program- 3

to permit operators to search previous equipment clearance orders to determine ;

'

if past problems have been identified and documented when performing

maintenance, and (2) computer database load lists that document plant

-

distribution panels and loads. After conducting the review, operators

'

determined that the only equipment that would be affected was reactor }

containment ventilation, the fuel handling building heating and air ,

conditioning system, and Train C of the control room heating and air l

conditioning system. Operators took actions to ensure that this equipment was l

properly aligned to prevent either an inadvertent actuation or isolation. i

i

, , , _ _ - _ -

-

__ _ _ _.

. l

. ;

.

,

-4-

L At 2:15 p.m on June 13, 1993, Distribution Panel DP001 was transferred to its l

J standby power source by the mechanical / electrical auxiliary building reactor '

plant operator. Distribution Panel DP001 was de-energized for approximately -

5 seconds while the feeder breaker from Inverter EIV001 opened and the feeder

breaker from Transformer ERV001A was shut, re-energizing Distribution Panel

DP001. The control room operators acknowledged all the annunciator alarms as

received and did not note any abnormal alarms or conditions. !

Following successful repairs to the terminal lug on Inverter EIV001, it was -

re-energized and Distribution Panel DP001 was transferred back to its normal i

source of power at 4:35 p.m. on the same day. At approximately 5:15 p.m., the

mechanical / electrical auxiliary building reactor plant operator _ entered the

fuel handling building to conduct a routine 8-hour tour that consisted of l

'

recording SFP temperature and level and observing satisfactory operation of

the operating SFP pump. During that tour, the reactor plant operator did not

note any abnormalities. Shift turnover was conducted between 6 p.m. and

7 p.m. on June 13, 1993. During that turnover, all off-going control room l

.

operators discussed with their reliefs the earlier power supply shift, the

alarms received, and actions taken to repair Inverter EIV001. In addition,

all oncoming operators conducted a control board walkdown with the off-going

operators. The operators noted no abnormalities.

2.2 SFP Cooling Loss Identification

!

At approximately 3 a.m. on June 14, 1993, while performing a logtaking tour,

the mechanical /e"ectrical auxiliary building reactor plant operator noted that

the in-service Component Cooling Water Pump 2A discharge pressure was reading

abnormally high. The expected pressure was 110-115 psi; however, the

,

discharge pressure was approximately 136 psi. The reactor plant operator

informed the control room of the abnormal condition. After being informed of ;

the local component cooling water pump discharge pressure, operators checked -

the control room indication of Train A component cooling water flow and

discovered that flow was approximately 5500 gallons per minute (gpm) when the

expected flow was 12,000 gpm. At this same time, control room operators

observed that the component cooling water Train A common isolation supply and

return valves, CC-M0V-0316 and CC-MOV-0052, were indicating shut. With these ,

valves shut, component cooling water flow was isolated from the SFP heat !

excharger. The operators immediately opened Valves CC-MOV-0316 and '

CC-MOV-0052, to restore SFP cooling and directed the mechanical / electrical

auxiliary building reactor plant operator to proceed to the fuel handling

'

building and report the SFP temperature. The reactor plant operator entered '

the fuel handling building and SFP temperature was 117 F. !

c

Based on a review of the logs and'a component cooling water flow trace j

retrieved from the plant computer, the inspector determined that component :

cooling water had been isolated from the SFP from June 13 at approximately !

2:15 p.m. until June-14, 1993, at approximately 3:15 a.m., a period of !

13 hours1.50463e-4 days 0.00361 hours 2.149471e-5 weeks 4.9465e-6 months . This would equate to a linear heat-up rate of about 1 1/2 F per ;

hour.

I

h

i

_- , _ _ _ _ __

. . _ _ _ __

.

'

4

.

-5-

3 FOLLOWUP (92701)

The inspector conductad a review of the events that lead to the loss of SFP

cooling and the licensee's immediate and longer term corrective action.

3.1 Repositioning of Valves CC-MOV-0316 and CC-M0V-0052

The inspector reviewed the events associated with the repositioning of

Valves CC-MOV-.0316 and CC-MOV-0052. These valves are located in the component ,

cooling water system in order to isolate non-Technical Specification governed

equipment from components governed by the Technical Specifications.

Non-Technical Specification equipment supplied by the component cooling water

system included both SFP heat exchangers, the letdown and seal water heat'

exchangers, the boron thermal regenerative system chiller, and the boron

recycle evaporator package. Valves CC-MOV-0316 and CC-MOV-0052 received an

automatic closure signal on a low level in the component cooling water surge

tank. Following the automatic closure, the valves received an open permissive

signal that permits operators to reopen the valves after the component cooling

water surge tank low level condition clears. The basis for this signal. was to

isolate these nonessential loads from the essential Technical Specification j

loads in the event of a low component cooling water surge tank level that

would result from a break in the component cooling water piping. - If.the ;

postulated break occurred in the nonessential portion of the component cooling

water system, the closure signal would prevent component cooling water

inventory loss and the potential of loss of the component cooling water train. l

When the operators initially de-energized Distribution Panel DP001 from !

Inverter EIV001, the component cooling water surge tank low level alarm was :

received and the circuit logic was completed to shut Valves CC-MOV-0316 and ;

CC-MOV-0052; however, the valves did not shut because-ie-energizing :

Distribution Panel DP001 removed control power to-operate these valves. When

Distribution Panel DP001 was re-energized from Transformer ERV001A, a relay ;

race occurred such that control power to operate Valves CC-MOV-0316 and

CC-MOV-0052 was restored and the valve's control. power circuit hold-in contact '

closed and remained closed after the component cooling water surge tank low

level circuit cleared. This relay race resulted in Valves CC-MOV-0316 and .

CC-MOV-0052 shutting and isolating flow to the SFP heat exchangers. The l

inspector determined that numerous ' annunciators alerting operators to the

condition would have illu.ninated during the 5-second period that Distribution. .

Panel DP001 was de-energized; however, following re-energizing the panel,

these alarms cleared.

After the repairs were completed on Inverter EIV001, Distribution Panel DP001

was transferred back to the primary power source at 4:35 p.m. on June 13,

1993. During this transfer, which was also a break-before-make evolution,

similar alarms to the those received on the earlier transfer at 2:15 p.m. l

annunciated in the control' room. During this transfer, since

,

' Valves CC-MOV-0316 and CC-MOV-0052 were already closed and required operator. -

action to re-open, the component cooling water surge tank low level did not - ,

cause any adverse plant conditions.

'

,

I

l

~

.

- - , , w

._. . . . . _ - _ _ _ . _

,

'

.

- .

.

i

,

-6-

3.2 Operator Performance issues

The inspector reviewed operator logs, security logs that recorded entry and

egress into Unit 2 areas, and operator statements that the licensee gathered

following the event. In addition, the inspector interviewed selected members

of the operations staff who were present during the period that the SFP was

isolated on June 13-14, 1993.

3.2.1 Control Room Performance .

l

The inspector determined that there were three separate indications available

to control room operators to indicate that component cooling water had been

isolated from the SFP. These included the valve position indications for

Valves CC-M0V-0316 and CC-MOV-0052, which are normally-open and were shut

following re-energizing Distribution Panel DP001; Component Cooling Water 2A

heat exchanger flow, which normally indicates approximately 12,000 gpm and was

approximately 5500 gpm following the component cooling water isolation; and ,

Component Cooling Water 2A heat exchanger outlet pressure, which normally wa

in the range of 100-105 psi and was indicating approximately 115 psi during

the event. None of these parameters were required to be logged by control

room operators. '

,

During the period that component cooling water was isolated to the SFP, a

shift turnover occurred at 7 p.m. on June 13, 1993. During that turnover, the

multiple indications of the loss of SFP cooling went unnoticed by both the

off-going and oncoming shift. Technical Specification 6.8.1.a states that

written procedures shall be established, implemented, and maintained covering

the activities referenced in Appendix A of RG 1.33, Revision 2, February 1978.

Paragraph 1.9 of RG 1.33, Appendix A, recommends that administrative

procedures addressing shift and relief turnover should be implemented. ,

Paragraph 3.2.5 of Procedure OPOP01-ZQ-0022, " Plant Operations Shift

Routines," Revision 2, states, in part, that each individual relieving a

watchstation SHALL be fully aware of plant status prior to assuming the watch. j

The failure of the oncoming shift to be fully aware of plant status, in that

component cooling water was isolated from the SFP, was the first example of a ,

violation of Technical Specification 6.8.1.a (50-499/9322-01).

During interviews conducted with the control room operators, the inspector

determined that a perception of a relatively low safety risk existed in the :

control room. This operator perception was based on the plant condition of

being 'defueled, with no decay heat removal requirements to remove residual j

heat from the core. This perception by the operators and the failure of both

the off-going and oncoming shifts to identify the loss of SFP cooling during

routine control board walkdowns were considered to be significant operator

performance weaknesses.

3.2.2 Reactor Plant Operator Performance

The inspector conducted a tour of the fuel handling building, tracing the path

that the mechanical / electrical auxiliary building reactor plant operators

_ . _ . .. . _. . . . _ . . _ _ _ .____ _ __ _______.._

.

. .

.

-7-

follow during their normal tours. In addition to the normal fuel handling

building logs that the reactor plant cperator was required to take each shift,

plant operations had invoked an additional temporary logsheet that required

recording local observation of the running SFP pump and SFP level- and

temperature. Although the normal fuel handling building logs required

readings to be taken each shift (every 12 hours1.388889e-4 days 0.00333 hours 1.984127e-5 weeks 4.566e-6 months ), this temporary logsheet

required reading to be recorded every 8 hours9.259259e-5 days 0.00222 hours 1.322751e-5 weeks 3.044e-6 months , which necessitated a second

fuel handling building entry per shift by one reactor plant operator during

each 24-hour period.

In the area of the SFP pumps and heat exchangers, hearing protection was

required, due to the high noise level from component cooling water flow.

Based on this observation, the inspector concluded =tnat the noise levels would

be expected to be significantly lower if the component cooling water flow were

isolated, as was the case after Valves CC-M0V-0316 and CC-M0V-0052 were

inadvertently closed.

The mechanical / electrical auxiliary building reactor plant operator entered

the fuel handling building at approximately 2 p.m. on June 13, 1993, to

conduct his routine tour shift of the building and to record the second 8-hour

temporary logsheet readings on the SFP parameters. During this tour,

component cooling water supply to the SFP heat exchangers was not isolatea.

SFP temperature and level were normal, with temperature recorded at 99 F and

level at 66-feet,1-inch. SFP Pump 2A was in service, with no pump

deficiencies observed.

The same reactor plant operator entered the fuel handling building again at-

approximately 5:15 p.m., to record the third 8-hour temporary logsheet

readings on the SFP parameters. During this tour, component cooling water

supply to the SFP heat exchangers had been isolated for approximately 3 hours3.472222e-5 days 8.333333e-4 hours 4.960317e-6 weeks 1.1415e-6 months .

SFP temperature and level were recorded at 99oF and level at 66-feet,1-inch.

SFP Pump 2A was in service and with no pump deficiencies noted.

The inspector questioned the licensee concerning the 5:15 p.m. recorded SFP

temperature. Assuming a linear heat-up rate, the expected SFP temperature

would be approximately 103 F, and not 99 F as recorded by the reactor plant

operator. The licensee responded that the heatup rate would not be expected

to be linear. Following component cooling water isolation to the SFP heat

.

exchanger, the shell side of the SFP heat exchanger would contain a large

volume of component cooling water. This volume of water would require a

period of time to reach an equilibrium temperature prior to-the SFP

temperature increasing and that a reading of 99 F taken at 5:15 p.m. would not

be considered unrealistic. Although based on a deterministic evaluation, the

inspector concluded that this argument was not unreasonable.

During the interview with the reactor plant o'perator that recorded the

2:15 p.m. and 5:15'p.m. fuel handling building parameters, the inspector

questioned the individual concerning the level of component cooling water flow ,

'

noise that was present in the area of the SFP pumps and heat exchangers. The

reactor plant operator acknowledged that there was a high level of flow noise ,

?

I

,-

- . . . . . - -- - . ..

.

-

. ;

.

\

-8- :

.

in that area and he acknowledged that the component cooling water flow noise

would be expected to be significantly reduced after component cooling water

isolation. However, the reactor plant operator stated that he did not recall.

whether there was a reduction of flow noise during the 5:15 p.m. fuel handling

building tour. Technical Specification 6.8.1.a states that written procedures

shall be established, implemented, and maintained covering the activities f

referenced in Appendix A of RG 1.33, Revision 2, February 1978. Paragraph 1 9

of RG 1.33, Appendix A, recommends that administrative procedures addressing

shift and relief turnover should be implemented. Paragraph 5.3.1.11 of i

Procedure OPOP01-ZQ-0022, " Plant Operations Shift Routines," Revision 2, i

states, in part, that while conducting local operator rounds, individuals ,

should inspect all areas and equipment, ensuring all noise and vibration

levels are normal. The failure of the reactor plant operator to note that the

'

noise level in the area of the SFP pumps and heat exchar.gers was significantly .

reduced following the isolation of component cooling water the SFP was the !

~

second example of a violation of Technical Specification 6.8.1.a

(499/9322-01).

3.2.3 Operations Staff Support l

The inspector reviewed the systems in place to provide information to

operators when equipment was required to be de-energized for corrective or ,

'

preventive maintenance. As mentioned in Paragraph 2.1, the control room

operators, prior to shifting Distribution Panel DP001 from its normal power 7

supply to its alternate power supply, reviewed Procedures OPOP02-AE-0004, ;

OPOP04-EE-0002, SE542EL8000 and the uncontrolled unit outage notes and !

computer database load lists. These systems appeared to be unwieldy and

difficult for the operator to utilize in order to extract useful information l

to assist them in operating plant equioment. During an interview with a shift ;

supervisor, the inspector was informed that the formal, controlled sources of '

information were less useful than the uncontrolled load list and outage notes. i

i

During interviews with other control room operators, the inspector identified

a similar event that occurred during the Unit 2 second refueling outage. In !

order to perform maintenance on safety-related Class IE Distribution f

Panel DP002, the licensee de-energized the panel in a manner similar to the i

Distribution Panel DP001 event. When Distribution Panel. DP002 was !

re-energized, a relay race occurred which resulted in several valves changing !

position unexpectedly. This event was not documented with a station problem l

report.

The inspector determined that the condition concerning the unexpected valve !

re-positioning during the transfer of Distribution Panel DP002 constituted a l'

condition adverse to quality and that the licensee failed to take prompt and

effective actions to ensure that the cause of this identified deficiency was j

identified, corrected, and steps taken to preclude recurrence. This was a ;

violation of the requirements of 10 CFR Part 50, Appendix B, '

Criterion XVI (499/9322-02).

I

'

.. . . . - . . .

-

. _ _ _ - . . _ .

.

,

-9-

'4 LICENSEE CORRECTIVE ACTION

i

As a result of this event, the licensee had taken several corrective actions.

Station Problem Report 932030 was generated to investigate the hardware l

deficiencies. The scope of this station problem report was increased to :

address the nonhardware issues, such as procedural weaknesses and personnel

performance inadequacies that were identified.

Although licensee personnel had not completed their review of the event at the -I

end of the inspection, they had identified a number of causes and had

developed a corrective action plan to address these deficiencies.

l

The licensee has revised Procedure OPOP02-AE-0004, "120 VAC ESF Vital ,

Distribution Power Supplies," to alert station personnel to the possibility of

inadvertent valve repositioning due to circuit relay races or other electronic !

perturbations when shifting power supplies using a break-before-make transfer.

The Operations Manager has conducted crew briefings with all shifts (licensed

and nonlicensed operators), and operations personnel involved in the event -

have received discipline in accordance with the Houston Lighting & Power

Company Constructive Discipline Program. ;

The licensee's training department will implement improved simulator scenarios

for use in operator requalification training. These will include t

demonstrations of operator awareness of mispositioned/ misaligned components

during shift turnover and normal control board walkdowns. In addition, the ,

training department will revise initial licensed operator training and

operator requalification training to increase operator awareness that the loss

of SFP cooling when the core is defueled to the SFP constitutes a loss of

decay heat removal. These actions were scheduled for completion by September

15, 1993.

.

'

The licensee's design engineering department has initiated a modification to

add an annunciated alarm in the control room to alert operators when component l

cooling water flow to the SFP cooling heat exchangers or SFP cooling flow to

the SFP has degraded. In addition, the design engineering department is

evaluating a reduction of the' current.SFP temperature control room annunciated .

alarm from 154af to a lower temperature that would provide operators a more :

timely indication of SFP temperature abnormalities. These actions were. !

scheduled for completion by-October- 31,1993. A longer term corrective action i

assigned to the design engineering department was validation and

implementation of formal control of the computer databases and other reference

material currently utilized by operations personnel to identify electrical.

loads. This task was scheduled for completion by March 31, 1994.

!

7

.

i

+

, - - -- . - - . . .-- . - --. .

.

3

..

.

ATTACHMENT 1

.

1 PERSONS CONTACTED

1.1 Licensee Personnel

H. Bergendahl, Manager, Technical Services

J. Calloway, Consultant, Participant Services ,

M. Chakravorty, Executive Director, Nuclear Safety Review Board

K. Christian, Manager, Plant Operations

W. Cottle, Group Vice President

J. Groth, Vice President, Nuclear Generation

S. Head, Deputy General Manager, Nuclear Licensing

.

T. Jordan, General Manager, Nuclear Engineering :

D. Leazar, Manager, Plant Engineering i

L. Ledgerwood, Consulting Engineering Specialist

M. Ludwig, Manager, Nuclear Training ,

,

G. Parkey, Plant Manager

P. Parrish, Senior Specialist, Licensing

K. Poling, Assistant to Manager, Nuclear Training

D. Ruthven, Shift Supervisor :

S. Walker, Manager Public Affairs !

G. Weldon, Manager, Operations Training !

M. Wigginton, Associate Engineer, Nuclear Training

1.2 NRC Personnel

R. Evans, Resident Inspector

J. Keeton, Resident Inspector l

M. Satorius, Project Engineer i

The personnal listed above attended the exit meeting. In addition to the i

personnel listed above, the inspectors contacted other personnel during this

inspection period.

2 EXIT MEETING

'

An exit meeting was conducted on July 2, 1993. During this meeting, the

inspector reviewed the scope and findings of the report. The licensee did.not

identify as proprietary any information provided to, or reviewed by, the

inspector.

_ . . _

_ _ _ _ . _ _ _

ML20056C976

Text

Navigation menu

Search