ML17311A981

From kanterella
Jump to navigation Jump to search
Insp Repts 50-528/95-10,50-529/95-10 & 50-530/95-10 on 950409-0520.Violations Noted.Major Areas Inspected:Response to Plant Events,Operational Safety,Maint & Surveillance Activities,Onsite Engineering & Employee Concerns Program
ML17311A981
Person / Time
Site: Palo Verde  Arizona Public Service icon.png
Issue date: 06/16/1995
From: Wong H
NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV)
To:
Shared Package
ML17311A980 List:
References
50-528-95-10, 50-529-95-10, 50-530-95-10, NUDOCS 9507030010
Download: ML17311A981 (60)


See also: IR 05000409/2005020

Text

ENCLOSURE 2

U.S.

NUCLEAR REGULATORY COMMISSION

REGION IV

Inspection

Report:

50-528/95-10

50-529/95-10

50-530/95-10

Licenses:

NPF-41

NPF-51

NPF-74

Licensee;

Arizona Public Service

Company

P.O.

Box 53999

Phoenix,

Arizona

85072-3999

Facility Name:

Palo Verde Nuclear Generating Station,

Units 1, 2,

and

3

Inspection At:

Maricopa County, Arizona

Inspection

Conducted:

April 9 through

May 20,

1995

Inspectors:

Approved:

K. Johnston,

Senior Resident

Inspector

J.

Kramer,

Resident

Inspector

A.. MacOougall,

Resident

Inspector

D. Garcia,

Resident

Inspector

F.

Huey, Technical Assistant

ow

ong,

C i

,

eactor

Jects

rane

~ /~AM

ate

Ins ection

Summar

Areas

Ins ected

Units

1

2

and

3

Routine,

announced

inspection of onsite

response

to plant events,

operational

safety,

maintenance

and surveillance

activities, onsite engineering,

and the

Employee

Concerns

Program.

C

Results

Units

1

2

and

3

Plant

0 erations

Overall, performance

in the area of plant operations

was good.

The Unit

1

midloop operation

went very smoothly.

Inspectors

observed

a cautious

approach

to the evolution, with good crew briefings

and well established

command

and

control.

Additionally, operators

responded

well to the loss of reactor

coolant

system

letdown resulting

from a failed instrument fitting on

a

common

charging line.

t

However,

the licensee's

evaluation of operation's

use of an out of calibration

boronometer to obtain

a Technical Specification required reactor coolant

system

sample described

in an

LER did not adequately

address

the cause of the

95070300i0

950627

PDR

ADOCK 05000528

8

PDR

0

event

and did not identify corrective actions which were applicable to the

problem.

The licensee's

internal

review was also incomplete in that it did

not address

the barriers

in place which should

have alerted the operators

to

the uncalibrated

boronometer.

Maintenance

Maintenance

work was observed

to be good.

There

was noted progress

in the

licensee's

efforts to address

valve packing leaks

by implementing

a

comprehensive

valve packing program.

However, there

were

some indications

that maintenance

personnel

did not place

much emphasis

on work instructions.

In one example,

work was performed

on

an air operated

valve without work

instructions at the job site.

In another

example,

an inverter maintenance

procedure

was issued for Unit I.work with numerous

pen

and ink changes

which

were over two years old.

The inspectors

found that

one of the

pen

and ink

changes

was missing

due to a duplication error.

The changes

made t'e

procedure difficult to follow.

En ineerin

The technical

support to plant operations

and maintenance

continued to be very

good.

The licensee

had addressed

weaknesses

in environmental

qualification

(Eg) monitoring, allowing them to compare

actual plant environmental

conditions with the previously calculated

Eg basis

documents.

It was noted

that

some aspects

of this monitoring lacked engineering rigor.

The engineering

evaluation of the failure of Unit 2 containment

spray Valve

SIA-664 during testing in August

1994

was found to be inadequate.

The

licensee

missed

a number of opportunities

to identify that the valve was

degraded,

including indications in motor operated

valve testing data.

The

inspector's

questions

resulted

in the licensee

determining that the

containment

spray

system

was inoperable for a period of 19 days,

which was

a

violation of plant Technical Specifications.

Mana ement Oversi ht

J

While the licensee

had

made progress

in their Employee

Concerns

Program

(ECP)

and the Management

Issues

Tracking

and Resolution

(MITR) programs

and

employees

expressed

satisfaction with the efforts of the

ECP, the inspector

identified

some programmatic

weaknesses.

These

weaknesses,

which primarily

involved the thoroughness

of evaluations

and formal communications

to the

concerned

employee,

could erode'he

cohfidence

employees

have in management's

concern for their issues if not addressed

promptly.

The two issues

in the enclosed

report that are the subject of the Notice of

Violation involved concerns that

had

been raised previously by inspectors,

but

licensee

followup was weak

and not thorough.

Only after questions

and

prompting

by the inspector

were comprehensive

evaluations

conducted.

e

'

Summar

of Ins ection Findin s:

~

One violation was identified (529/9510-01)

regarding the use of an

uncalibrated

boronometer to satisfy

a Techn'ical Specification

(Section

7. 1).

Both

LER 529/94-008

and Unresolved

Item 529/9431-06,

concerning

the

same subject,

were closed.

~

One violation was identified (529/9510-02)

regarding the failure to

comply with the Technical Specification action statement for an

inoperable

containment

spray

system valve (Section 7.2).

Unresolved

Item 529/9426-02,

concerning

the

same subject,

was closed.

Unresolved

Item 528/9431-01,

concerning

a leaking Unit

1 letdown

isolation valve,

was reviewed

and left open pending the completion of a

licensee

assessment

(Section 8. 1).

Unresolved

Item 528/9426-04,

concerning

the environmental

qualification

of auxiliary feedwater

system valves,

was closed

(Section 8.2).

.

Attachments:

1.

Persons

Contacted

and Exit Meeting

e

j

e

DETAILS

1

PLANT STATUS

1.1

Unit

1

Unit

1 began

the inspection period in a refueling outage with core offload in

progress.

On April 9, the licensee

completed

core offload.

On April 27, the

licensee

commenced

core reload

and

on April 30

a fuel assembly

got lodged

as

it was lowered into the core.

The licensee

requested

and

was granted

a Notice

of Enforcement Discretion to allow a higher refueling bridge overload limit.

On May 1, the licensee

successfully lifted the fuel assembly

and

recommenced

core reload

(Section

2. 1).

On May 20, the unit entered

Mode

3 and the

inspection

period

ended with

a reactor

cooTant

system

heatup

in progress.

1.2

Unit 2

Unit 2 began

the 'inspection

period at

100 percent

power.

On April 11,

operators

reduced

power to 40 percent to repair

a condenser

tube leak'hat.

developed

in the

2C condenser

hotwell.

The licensee

repaired

the leaking

tubes

and returned

the plant to

100 percent

power on April 11.

On May 10,

power was reduced to 40 percent

due to

a large intrusion of impurities into

the steam generators

(Section 2.2).

- The licensee

restored

steam generator

chemistry to normal

and returned to plant to 100 percent

on May 11.

1.3

Unit 3

Unit 3 started

and

ended .the inspection period at

100 percent

power.

'2 ONSITE RESPONSE

TO EVENTS (93702)

2. 1

Fuel

Assembl

Stuck Durin

Core Reload

Unit

1

On April 30, during the Unit

1 core reload,

personnel

on the refueling bridge

r'eceived

an underload

alarm as they were lowering

a fuel assembly into core

location E-12.

During the subsequent

attempt to raise the assembly,

a

refueling bridge overload interlock was received.

Visual examination

revealed

that the assembly

in adjacent

core location,

F-12,

had not been positioned

properly

and

was preventing the movement of the assembly

in position E-12.

The fuel assembly

in E-12 was approximately

2 feet

above the core support

plate

and

was restrained

on three sides

by previously loaded fuel assemblies.

The licensee

made

an attempt at moving the assembly

in a horizontal direction

by operating

the refueling machine manually.

The refueling operators

were

only able to move it approximately

one inch before the load cell indicated

that the fuel assembly

was stuck.

After uns'uccessful

efforts to raise the

assembly,

the licensee

put the refueling operations

on hold pending further

evaluation.

e

0

l

I

Early in the morning

on Hay 1, the licensee

placed

a restraining device

on the

fuel assembly

in the

F-12 core location.

The restraining device

was designed

to prevent the F-12 fuel assembly

from tipping while efforts were

made to

dislodge the fuel assembly

in the E-12 core location.

Refueling operators

again attempted

to move the stuck assembly

manually in the horizontal

direction in which it was not blocked

by another

assembly.

The overload limit

was reached

and the movement

was

suspended.

The licensee

subsequently

requested

a Notice of Enforcement Discretion from

Region

IV to allow raising the Technical Specification

(TS) refueling machine

hoist overload limit from 1600 lbs to 1800 lbs.

The licensee

and the fuel

vendor

had determined that the additional

200 lbs would not impact the core

internals or the pressure

vessel.

They suspected

that there could

be damage

to the fuel assembly grid straps,

but did not anticipate fuel rod damage.

Region

IV granted

the

one time limit adjustment

at 8:45 a.m.

(HST)

on Hay 1,

1995,

based

on

a review of the licensee's

assessment.

On Hay 1, at 1:30 p.m.

(HST), refueling personnel

were able to free the stuck

fuel assembly with a force of 150 lbs over the

TS limit,

The licensee

proceeded

to move the fuel assembly

and adjacent

assemblies

to the spent fuel

pool for a detailed inspection.

The inspectors

observed

most of the significant attempts

at lifting the fuel

assembly. on April 30 and

Hay 1.

The inspectors

observed that the refueling

operators

proceeded

with caution

and in accordance

with pre-established

strategy

and limits.

The cause of the fuel assembly to become

stuck appears

to have

been the

failure to properly seat the fuel assembly

in core location F-12.

The

licensee initiated

a condition report/disposition

request.(CRDR)

to perform

a

review of the failure to seat

the fuel assembly.

The assembly

which was stuck

was

one of the last

25 assemblies

to be put into the core.

Earlier in core

reload,

the licensee

used

a high powered light and

a video camera

located

low

in the core to provide good monitoring of the reload.

However,

as the core

was filled, the camera

and light had to be removed

due to its proximity to

fuel being

moved

and the high radiation fields which could damage

the video

camera.

As

a result, refueling operators verified that the final fuel

assemblies

were properly set using binoculars to see if the assembly

was

located

on the core support pins

and

by reading the "Z" coordinate

on the

refueling machine to verify that the assembly

was fully lowered.

In the case

of the fuel assembly

in location F-12, the "Z" coordinate

indicated the

assembly

was fully lowered.

A refueling operator's 'isual

examination

by

binocular inappropriately

concluded that the assembly

was properly seated.

The inspector

reviewed past

performance

in refueling to determine if there

had

been previous

problems during later stages

of the core reload,

The inspector

noted that there

had not been previous

problems.

The licensee

stated that

they planned to develop

a high powered light designed

so 'that it could

be kept

in the core to the last stages

of the fuel loading

and that this will be in

e

place for the Unit 3 refueling outage

scheduled

for October

1995.

The

inspector

found this to be appropriate.

No violati'ons of NRC requirements

were identified.

2.2

Power Reduction

Due to Increased

Sodium in the Steam Generators

- Unit 2

On Nay 10, at approximately

10:45 a.m.,

operators

observed

a sharp

increase

in

the IA hotwell sodium level

and

a corresponding rise in steam generator

sodium

levels.

Within minutes,

the steam generator

sodium levels increased

from

normal levels of less

than

one part per billion (ppb) to over 300 ppb.

The

operators

entered

the condenser

tube rupture procedure

and

began

reducing

power to 40 percent.

At approximately

12:30 p.m., the plant was stabilized at 40 percent

power. and

the IA condenser

shell

was isolated.

The licensee

performed

a tube inspection

and did not identify any leaking circulating water

(CW) tubes.

They suspected

that the source of the impurities was the auxiliary steam condenser

tank that

returns to the

1A hotwell.

The licensee

secured

the auxiliary steam

condenser

tank to the hotwell

and was able to reduce

the sodium

and sulfate levels in

both steam generators.

On Hay II, the licensee

returned

the unit to 100 percent

power.

The inspector

observed

portions of the downpower from the control

room and noted very good

command

and control

from both the shift supervisor

and the control

room

supervisor.

The inspector

also noted that the operators

responded

quickly to

the event

and

had all the condensate

demineralizers

in service within 15

minutes of detecting

the increase

in hotwell sodium levels.

As a result,

the

operators

limited the subsequent

peak of the steam generator

sodium levels.

The inspector

concluded that the operators

were very sensitive to the

importance of steam generator

chemistry

on the integrity of steam generator

tubes.

The licensee

was investigating the exact

source of the impurities in the

auxiliary steam

system

and suspected

a leaking isolation valve between

the

auxiliary steam

and the condenser

hotwell

and

a tube leak in the liquid

radioactive

system evaporator.

The inspector will review the licensee.'s

corrective actions during future routine inspections.

2.3

Loss of Letdown

Unit 2

On Nay 19,

a swage lock connector for the Unit 2

common charging line pressure

transmitter,

CHA-PT-212, failed resulting in

a charging

header to atmosphere

leak in the "E" charging

pump room.

At the time of the event,

the "E" and the

"8" charging

pumps were in operation

and the "A" charging

pump had

a freeze

seal

applied to the discharge line to allow repairs to the discharge

isolation

valve.

The operators

had conducted

a briefing on the contingency

actions to

isolate

seal

injection and stop the remaining charging

pumps if the freeze

seal failed.

e

g

f

At approximately

10:50 p.m., control

room operators

received

a charging

system

trouble alarm

and noticed that both the charging

header

pressure

and flow

instruments

were reading zero.

Within a minute,

letdown automatically

isolated

on high regenerative

heat

exchanger outlet temperature.

The

operators

implemented the previously briefed contingency actions

and stopped

the running charging

pumps

and isolated

seal

injection.

The leak was stopped

when the operating

charging

pumps were stopped.

The licensee

estimated that

about

300 gallons of charging

system water was

pumped into the

"E" charging

pump room.

The operators

did not detect

any reactor coolant

system

leakage

during the event.

The inspector

reviewed the applicable

alarm response

and operating

procedures

and discussed

the event with the control

room operators.

The inspector

noted

that the operators

appropriately

entered

TS 3.0.3

due to the loss of all

charging

pumps

and the loss of an operable

boration flow path.

The licensee

was in TS 3.0.3 for about

45 minutes.

The inspector

also noted that the

licensee initiated

a

CRDR to evaluate

the root cause of the

swage lock fitting

failure.

The inspector

concluded that the operator

response

to the event, was

good and'that

the licensee's

corrective actions

were appropriate.

The.

licensee

planned to submit

a Licensee

Event Report

(LER).

The inspector will

review the licensee's

evaluation of the event's

cause

and proposed

corrective'ctions

in

a future inspection report.

3 OPERATIONAL SAFETY VERIFICATION

(71707)

3. 1

Reactor Coolant

S stem Midloo

0 erations

Unit

1

On May 12, the inspector

observed

the licensee

drain the

RCS to midloop to

remove

steam generator

nozzle

dams.

The inspector

reviewed the prerequisites

for draining to midloop and noted

no discrepancies.

The inspector

noted that

the operations staff conducted excellent briefings

and maintained

good

command

and control throughout the evolution.

The inspector

noted that the operators

used all available level indications

as the level

approached

a midloop

condition

and that the operators

often verified the levels were within

acceptable

deviations.

The inspector

observed

active participation of the

shift technical

advisor

and the nuclear

assurance

evaluator.

The inspector

concluded that the licensee

performance

during the midloop evolution was good.

4

MAINTENANCE OBSERVATIONS

(62703)

4. 1

125

VDC Vital Inverters

Unit

1

On April 26,

1995,

the inspector

observed

portions of maintenance

procedure

32MT-9ZZ58,

"Maintenance of Inverters."

The electricians

indicated that the

procedure

was difficult to follow and the inspector

noted that the procedure

had numerous

steps

and

pages that were crossed

out and

hand written steps

were

inserted'he

changes

resulted

from a temporarily approved

procedure

action

~

~

~

~ (TAPA) written in April 1993.

The licensee

approved

the

TAPA, but did not

formally incorporate

the

TAPA into the procedure.

e

The inspector

reviewed the completed

procedure

and noted that the content

was

technically accurate;

however,

the inspector

noted

a controlled copy of the

procedure

appeared

to have

a hand written step missing at the bottom of the

page

due to reproduction.

The inspector notified'lectrical maintenance

about

the discrepancy.

The licensee

reviewed the original

TAPA, and recovered

the

missing step.

The inspector

noted that the step directed the user to

a

subsequent

section in the procedure

and did not contain technical

information.

The licensee

determined that the missing step

had not impacted the performance

of work.

The licensee

subsequently

re-issued

the procedure

which included all

the

hand written steps.

Nuclear Assurance

issued

a

CRDR to evaluate

the problem.

The licensee

looked

at all of the electrical

maintenance

procedures

and determined this was the

only procedure

which had

pen

and ink changes.

The inspector

noted that the

technicians

continued to use the procedure

although the procedure

did not work

as written.

In addition,

the inspector

noted the technicians

did not

adequately

document

steps

marked

NA (not applicable).

The inspector 'discussed

the maintenance

workers'eaknesses

with the electrical

maintenance

department

leader.

The electrical

maintenance

department

leader

acknowledged

the

weaknesses,

indicated that the technician

performance

did not meet his

expectations,

and stated

he would discuss

the weaknesses

with the technicians.

The inspector

concluded that the technicians

were knowledgeable

about the

maintenance

task

and followed a logical progression

through the procedure

and

correctly performed the task,

although

one step

was missing.

4.2

Air 0 crated

Valve Dia hra

m Re lacement

On April 21, the inspector

observed

a preventative

maintenance

task to replace

internal

components

in the operator for Valve SIA UV 560, the inside

containment isolation valve to the reactor drain tank.

]he inspector

observed

the mechanic

change

the o-rings

on the stem bushing

and install

a

new

diaphragm for the air operator.

The inspector

noted that the mechanic lubricated

and installed the o-rings

and

installed the

new diaphragm,

but did not have the work order available at the

job site.

The mechanic stated that another

member of his team

had taken the

package

back to the

shop to get

a tool.

The inspector.

noted that the mechanic

could not review the actions

he completed

because

he could not refer to the

work order instructions.

The inspector discussed

this observation with the responsible

mechanical

maintenance

section leader

who agreed that not having the

WO available at the

job site did not meet management's

expectation.

The inspector also reviewed

the

WO instructions

and discussed

the scope of the job with the technician.

The inspector

concluded that the mechanic

completed

the job in accordance

with

the

WO instructions.

The section leader

counseled

the mechanic

concerning

management's

expectations

for having the work order available at the job site.

The inspector

concluded

e

0

0

that the licensee's

corrective actions

were appropriate

and that this appeared

to be

an isolated incident.

4.3

Valve Packin

Pro

ram Review

The inspector

conducted

a review of the licensee's

valve packing program to

determine

how the licensee

was identifying and correcting old packing

configurations

in safety-related

valves.

The inspector wanted to determine

the number of other safety-related

motor operated

valves

(NOVs) which could

have

had obsolete

packing configurations like the

10 braided rings found in

Valve SIA 664 that subsequently

prevented

the valve from closing

(Section 8.2).

The inspector

reviewed the licensee's

valve packing procedure

and

specification,

interviewed the maintenance

engineer

responsible for the valve

packing program,

observed

repacking of several

valves,

and conducted

walkdowns

to assess

the condition of the valve packing.

4.3. I

Program Description

The licensee's

valve packing program required valves to be repacked

using

a

standard five ring packing configuration consisting of three die-formed rings

and two braided

end rings.

The Maintenance

Department

planned to repack

safety-related

valves

as part of a preventive maintenance effort instead of

repacking

in response

to identified leaks

as

has

been

past practice.

The

licensee's

program allowed

some variation in this configuration depending

on

the depth of the stuffing box and the existence

and location of a leak off

port.

The inspector

also noted that valves with unique packing configurations

and/or materials

were individually exempted

from the valve packing program.

The inspector

concluded that the licensee's

packing program

was adequately

described

in the procedure

and packing specification.

The inspector

noted that the mechanical

maintenance

engineering

was the

process

owner of the licensee's

valve packing program

and was generating

a

valve packing data

base that contained specific packing information for every

valve in the plant.

The data

base

was generated

from information gathered

in

the field during valve packing replacements.

The valve specific information

was recorded

on

a valve survey data

sheet

(VSDS).

The

VSDS included

information such

as the dimensions of the stuffing box, the type,

number

and

order of packing rings

and spacers,

and the as-left packing gland torque.

The

inspector

concluded that the licensee's

effort to generate

an accurate

data

base of the field condition of packing

was

a strength.

4.3.2

Observation of Work in Field

The inspector

observed

mechanics

repack Valve SIA 647,

a motor-operated

high

pressure

safety injection system valve in Unit 2,

and Valve

CHB 337,

a manual

charging

system valve in Unit I,

The inspector

noted that the work was

performed

by contractors,

The Atlantic Group.

The inspector

concluded that

the valve repacking activities were appropriately

conducted.

e

-10-

The Director of Maintenance

hired

The Atlantic Group to repack the valves in

the plant

as part of the "Level I" action to reduce

the number of packing

leaks

in the plant.

The inspector

noted that during the Unit

1 refueling

outage

there

were

a total of 82 valves that were repacked

and

16 of these

valves

were safety-related

MOVs.

During the Unit 2 refueling outage

completed

in March 1995,

a total of 95 valves

were repacked

and

32 were safety-related

MOVs.

The inspector

concluded that the licensee

was aggressively

implementing

the valve repacking

program to improve the material condition of the plant

and

the performance of the valves.

The inspector

also noted that the licensee

was

placing

a high priority on repacking

the safety-related

MOVs.

4.3.3

Plant

Walkdown

The inspector identified three

MOVs with apparent

packing leaks in Unit 2 on

May 3 during

a routine plant tour.

The inspector

informed the valve services

group

and they conducted

a walkdown of the safety-related

MOVs in Unit 2 and

.

identified nine more valves with apparent

packing leaks.

The licensee

subsequently 'determined that eight of the

12 leaking valves

had

been

identified two weeks earlier during operations

and engineering

system

~

walkdowns.

The inspector determined that four of the leaking valves

had

been

repacked

during the Unit 2 outage

in March 1994.

The inspector

noted that

Valve CH-524 had

a significant leak and appeared

to be actively leaking.

The

other three valves

had small

amounts of boron

on the

stem but did not appear

to be actively leaking.

The inspector

reviewed the

VSDS for Valve CH-524

and noted that the valve was

properly packed

and that the packing gland

was properly tightened,

The

inspector

noted that for motor-operated

valves the mechanics

tighten the

packing gland towards the low end of the required torque range.

The valve

services

group subsequently

performs

an as-left static diagnostic test

and

ensures

that the packing load is within the proper range.

The diagnostic test

procedure

has general

guidance that the running load should

be around

1000 pounds for each

inch of stem diameter.

The procedure

also includes

guidance that the packing should

be tightened if the running load is low.

The

inspector

concluded that the licensee's

procedures

provided adequate

guidance

to ensure that the packing of MOVs was not tightened

too much, which would

prevent operation of the valve, or not tightened

enough,

which would allow the

valve to leak.

The inspector

noted that the as-left diagnostic test

was performed with the

system filled, but not at normal operating

pressure

and temperature.

As

a

result,

the packing

may have loosened

causing

the valve to leak when the

normal

system pressure

was reached.

The licensee

agreed

to evaluate

optimizing when as-left diagnostic tests

were conducted

to minimize the

potential for subsequent

packing leaks.

The inspector

concluded that the

I

'icensee's

actions

were appropriate.

0

)

j

4.4

Other Maintenance

Work Observed

The inspector

observed

portions of the work listed below:

Feedwater

isolation valve accumulator nitrogen pre-charge - Unit l.

Disassembly of turbine driven auxiliary feedwater governor valve

Unit'.

5

SURVEILLANCE OBSERVATION

(61726)

5.1

Hain Steam Isolation Valve Testin

- Unit 2

On Hay 9, the inspector

observed

a portion of main steam isolation valve

(HSIV) partial stroke testing in Unit 2.

The inspector

observed

good

command

and control of the test

by the control

room supervisor

and the reactor

operator performing the test.

For example,

the operators

had briefed

contingency actions

in the event

a MSIV went closed

and were closely

monitoring the HSIV accumulator

pressure

for any abnormal

trends during the

test.

The inspector also observed

good communications

between

the auxiliary

'perator

and the control

room operators.

t

6

EMPLOYEE CONCERNS

PROGRAM REVIEW

The inspector

performed

a review of the licensee's

Employee

Concerns

program

(ECP)

and Management

Issues

Tracking

and Resolution

program

(MITR).

The fCP

provided employees -a method for independent

review of .technical

and safety

concerns

which they did not consider

would be adequately

resolved

through

normal

processes.

The MITR provide employees

similar review for personnel

and

administrative

concerns.

The inspector

noted the following concerns:

The HITR program

was not covered

by

a formal administrative

procedure.

As

a result,

there

were

no consistent

requirements

for the content of

MITR files or how they were to be closed.

Review of Files 94-01,

94-06,

94-09,

and 94-10 identified the following concerns:

~

The files did not include appropriate

documentation of evaluation

conclusions.

~

The files did not include final closure

correspondence

to the

concerned

employee

as to how each of his concerns

had

been

resolved.

~

The files did not indicate

any cause

evaluation or corrective

action for problems that

had resulted

in val'id employee

concerns.

~

HITR 94-43 addressed

a discrimination concern involving an engineer

having raised

safety concerns

about

ADV problems

in

a report to the

NRC.

It was not clear from the file that the concern

received

an

1

-12-

appropriately

independent

evaluation,

in that the concern

was evaluated

by the Vice President

to whom the alleged discriminating manager

was

a

direct report.

The file did not clearly .indicate the basis for Human

Resources

department

conclusion that this

I'0 CFR 50.7 concern

received

a

proper

independent

evaluation.

~

The inspector identified the following concerns

in a review of ECP

administrative guidelines

ECPOI,

ECP02,

ECP03,

and

ECP Files 94-07-03,

94-08-05,

94-09-02,

94-10-03,

and 95-01-07:

ECP Administrative Guideline

ECP02

(Handling

ECP Concerns)

does

not require

an initial letter to the concerned

employee

which

clearly documents

the scope of his concerns,

the plan of action to

evaluate

the concerns,

or. the target

schedule for completing the.

evaluation,

nor does

the guideline require

a final closure letter

to the employee

which documents

the conclusions

and

actions'esulting

from the

ECP evaluation.

~

File 95-01-07 did not indicate

any cause

evaluation or corrective

action for the mishandled

CRDR investigation

problem identified

during the

ECP evaluation.

ECP Administrative Guideline

ECP03

(ECP File Open Action Tracking)

does

not address

completion of file closure followup with the

employee.

~

The cl'osure followup for File 94-10-03 indicated that the employee

considered

that the hostile work environment

in his work group

had

"drastically improved," however,

NRC interview with the concerned

employee indicated that the employee still considers that

supervisors

in his work group are hostile to employees

raising

safety concerns.

~

The inspector interviewed six employees

who recently

had discrimination

concerns

evaluated

by the

ECP

and

MITR programs.

These

employees

consistently

expressed

satisfaction with the performance of the

ECP

program;

however,

some stated that they were not confident with the

MITR

program,

indicating that the

Human Resources

department

was not viewed

as

a credible organi'zation for representing

employee's

best interests.

The employees

also stated that senior licensee

managers

had increased

their credibility in recent

months

such that employees

were confident

that they could

come to senior managers

and receive fair treatment,

without fear of retribution.

However,

several

of the

same

employees

stated that they did not have similar confidence

in lower levels of

licensee

management

and supervision.

While progress

had

been

made in the

MITR and

ECP

and employees

seemed

generally satisfied with the

ECP,

some programmatic

weaknesses

were

identified.

The inspector discussed

these

findings with licensee

senior

~

'

0

-13-

management

responsible

for the

ECP

and

MITR programs.

They concurred with the

inspector's

observations

and planned to implement appropriate corrective

actions.

7

FOLLOWUP MAINTENANCE (92902)

7. 1

Closed

LER 529 94-008

and

Closed

Unresolved

Item 529 9431-06:

Use

Of Uncalibrated

Boronometer

Caused

a

TS Action to be Missed

This

LER involved operator's

use of an uncalibrated

boronometer to verify

reactor coolant

system

boron concentration

in order to comply with the

compensatory

action requirements

of TS 3. 1.2.7 which applied in February

and

March

1994

when

a startup

channel

was

removed

from service.

7.1.1

Background

Instrumentation

and Controls

(I&C) technicians

began

a routine 18-month

calibration of the Unit 2 boronometer

in February

1993.

They found that

one

of the instrument's

power supplies did not meet the calibration

specifications.

The

ILC technicians

stopped

work, left the boronometer

in

service, notified the Shift Supervisor,

and proceeded

to attempt to procure

a

new power supply.

The boronometer

appeared

to be operating

adequately

in that

it closely tracked reactor coolant

system

boron concentration.

The

ILC group performed work on the boronometer sporadically until November

1994.

Near the time the calibration

was completed

in November

1994,

the

licensee identified that the boronometer

may have

been

used

by operators,

during the Unit 2 refueling outage in February

and March 1994, to satisfy the

compensatory

action requirements

of TS 3. 1.2.7 which apply when

a startup

channel

was

removed

from "service.

The licensee

concluded this review in

January

1995,

having found that the boronometer

was relied

on in three

occasions.

On one of the occasions, if no credit is provided for the

use of

the boronometer to meet the

TS action,

the

TS action

was not complied with.

The inspector

noted that, despite

the questionable

calibration status of the

boronometer,

during the period it was

used for TS compliance, it remained

within

1 to

2 percent of the

RCS samples.

The inspector

reviewed the

LER and found that there were significant

weaknesses

in the evaluation of the cause of the event

and the corrective

actions

taken.

Further,

the inspector

found that the licensee's

internal

evaluations

which support the

LER were weak.

7. 1.2

Evaluation of the

Cause of the Event - Operations

The

LER statement

of cause

states,

in part:

"An evaluation

was performed

in accordance

with the

APS Incident

Investigation

Program.

The evaluation

concluded that the apparent

cause

of the boronometer

past calibration

was

a lack of concern for the

instrument

being calibrated correctly."

I

e

-14-

The inspector

noted that the cause, statement

did not address

the barrier

necessary

to prevent

an operator

from using

an boronometer of questionable

calibration to satisfy

a

TS requirement.

The inspector determined,

through

discussions

with operations

personnel,

that the appropriate

administrative

control would have

been for operators

to have

made

a control

room deficiency

log

(CROL) entry with a

CRDL tag placed

on the control board.

The licensee's initial CROR (2-4-343) evaluation identified that

I&C had

informed the shift supervisor that the calibration of the boronometer

had not

been completed.

The

CROR noted that

a second

CRDR (2-4-4?1)

was initiated for

operations

to evaluate

how the boronometer

was allowed to be used while it was

being calibrated.

CROR 2-4-471 identified that the issue

was reportable

under

10 CFR 50.73,

but did not provide any root cause

review.

The inspector

found

that,

CROR 2-4-343

was closed

based

on the evaluation to be performed in

CROR 2-4-471.

However,

CRDR 2-4-471

was subsequently

closed without a

documented

evaluation of cause

and without further corrective actions

based

on

the evaluation

provided in

CRDR 2-4-343.

As

a result,

there

was

no documented

evaluation of operations failure to identify and tag the boronometer

'eficiency.

7, 1.3

Evaluation of the

Cause of the Event - Maintenance

The licensee

had

made

some effort to identify what the inspector considered

to

be

a contributing cause

concerning

the lack of a timely calibration of the

boronometer.

The licensee identified that the calibration

had not been given

priority due to the fragmentation of the responsibility for the resolution of

equipment reliabili.ty problems.

The inspector

reviewed this evaluation

and found that it lacked rigor.

The

preventive maintenance

(PH) task to calibrate the boronometer

was considered

to be

an "operations surveillance test no-waive

PH."

The

PH task,

which had

an 18-month frequency,

had last

been completed

on September

26,

1991,

and

was

due

on March 26,

1993.

The licensee

allowed

a grace period to August 23,

1993.

The boronometer

was recognized

in its

PH basis

documentation

as

providing post-accident

indication in accordance

with Regulatory

Guide 1.97:

On July 29,

1993,

an

I&C foreman performed

a

PM work order disposition report

to delay the calibration of the boronometer.

No justification was provided

and the due date

was extended

to September

26,

1993.

The work was not

completed

in September

1993

and

no further delay requests

were documented.

In September

1994,

a Shift Technical Advisor discovered that the

PM had not

been completed.

The

I&C section leader

performed

a

PH work order disposition

report to waive the original

PH task.

CRDR 2-4-343

was initiated

and the

calibration

was completed

in November

1994 under

a new work order.

The waiver

of a "no-waive"

PH was considered

an Unresolved

Item in Inspection

Report

94-31.

The evaluation

in

CRDR 2-4-343,

which was included in the

LER, discussed

the

lack of priority placed

on calibrating the boronometer.

However, it did not

f

address

the specifics of what barriers

are provided to ensure that priority is

appropriately

placed

and

how these barriers

may have

been defeated.

Specifically:

~

Regarding

the disposition

on July 29,

1993,

how could

a "no-waive"

PM be

delayed with no documented justification for the delay?

~

After the first delay expired

on September

26,

1993,

why were there

no

subsequent

"no-waive"

PM dispositions

documented?

~

Why were there

no controls in place to ensure that post-accident

monitoring instrumentation

referred to in the Updated Final Safety

Analysis Report

be returned to service

in

a expeditious

manner?

7. 1.4

Evaluation of Corrective Acti'ons

The inspector

reviewed the actions to prevent recurrence

discussed

in the

LER.

The

LER stated that the following step

had

been

added to the

PM Process

and

.

Activities procedure:

" 18C Section/Team

Leaders

shall refer out-of tolerance results

found

during Operations

Surveillance

Tests

"NO WAIVE" PMs

on installed plant

equipment to the duty

STA or other appropriate

engineering

personnel.

The STA/engineer shall

perform

a documented

evaluation of the

significance of the problem or deficiency

and ensure that

a

CRDR is

generated, if required."

The inspector

found that this step did not address

any of the problems

noted

in the

LERs or the

CRDRs associated

with the boronometer.

The inspector

interviewed

a Shift Technical Advisor (STA),

a Control

Room Supervisor,

a

Shift Supervisor,

an

I8C Team Leader,

and

an

18C Section leader

and determined

this step.was

intended to ensure that out-of-tolerance results

are

appropriately

reviewed for their impact

on tests for which they may have

been

relied

upon

and to trend

and assess reliability of the instrument.

Typically,

as-found out-of-tolerance

results

are quickly resolved

and the instrument

placed

back in service with no impact

on ongoing plant operations.

Therefore,

IKC typically refers

subsequent

out-of-tolerance

reviews to

IKC maintenance

engineers

and not the STAs.

Furthermore,

the inspector

noted that the boronometer calibration procedure

included

two steps requiring

18C to notify the control

room of out-of-

tolerance

findings

and that this was done for the subject calibrations.

The

LER concludes

by stating that the evaluation of the event

had not been

completed.

The

LER was issued

on February

7,

1995.

All open evaluations

of

the issue

were closed

by February

14,

1995, without further documented

review.

0

I

l

'I

-16-

7. 1.5

Conclusion

The failure to assure

that

an instrument

used to meet

TS requirements

was

properly calibrated is

a violation of 10

CFR Part .50,

Appendix B,

Criterion XII, concerning

the control of measuring

and test equipment

(Violation 529/9510-01).

The inspector

noted that the licensee identified

this violation in December

1994-.

However,

the inspector considered

that the

licensee

had not performed

an adequate

cause

review nor identified appropriate

corrective actions to prevent recurrence.

The inspector considered

this to be

potentially significant in this case

since it appeared

that both licensee's

corrective actions

and event report programs failed to ensure that

an adequate

review was performed.

7.2

Closed

Unresolved

Item 50-529 94-26-02:

Failure of Motor-0 crated

Valve to Close

This unresolved

item involved the failure of a Unit 2 containment

spray

(CS)

miniflow isolation Valve SIA-664 to close during

a surveillance test

.on

September

5,

1994.

The valve also failed .to close

on August

17,

1994,

when

'perators

attempted

to verify the valve was

open

by closing the valve.

The licensee

conducted

a root cause of failure analysis

and determined that

t

the failure of Valve SIA-664 to close

was caused

by an excessive

running load.

The licensee

determined that the excessive

running load was caused

by an

obsolete

packing configuration

(10 braided rings

and

a lantern ring) .that

was

originally installed in the valve.

The valve was repacked

using low

resistance

packing,. manufactured

by Argo,

and returned to service.

The

licensee

also

changed

the bill of materials to specify the Argo packing

and

initiated work requests

to repack the Unit

1

CS and low pressure

safety

injection (LPSI)

pumps miniflow valves during the Unit

1 refueling outage

in

April 1995.

The inspector initiated the unresolved

item to assess

whether the torque

switch (T(S)

was correctly set

and if the licensee's

response

to the initial

failure to close

on August

17 was appropriate.

The licensee initiated

CROR 2-4-0301 to address

the inspector's

questions.

The inspector

reviewed the licensee's

CROR evaluation,

reviewed all the

previous diagnostic traces for Valve SIA-664,

and

had

numerous

discussions

with members of the valve services

group.

In summary,

the inspector

noted the

following weaknesses

with the licensee's

overall

response

to the problems with

Valve SIA-664:

~

The licensee

did not conduct

a thorough review of the issue

in response

to the unresolved

item.

The inspector

had to prompt

a more thorough

review which eventually led to discovering that Valve SIA-664 was

inoperable

between

August

17 and September

5,

1994.

0

~,

-17-

The licensee

did not perform

a thorough, critical review of past

diagnostic traces

to assess

the future performance of the Valve SIA-664.

As

a result,

the licensee failed to identify an anomaly in the

diagnostic traces of Valve SIA-664 which wa's

a precursor to the failures

on August

17 and September

5,

1994.

The licensee

did not have

any requirements

for the technicians

analyzing

diagnostic traces

to perform

a qualitative assessment

of the quality and

shape of the traces.

The inspector pointed out to the valve services

group the potential

problems with not performing this type of

qualitative assessment

of the diagnostic traces

in Inspection

Report 50-528/94-13

and the licensee

did not provide any additional

tools or training to help the technicians

recognize trace

anomalies.

The inspector

noted that the response

by the valve services

group to this

problem was poor,

but not typical.

For example,

the licensee

identified: and

promptly corrected

anomalies

in butterfly'valve traces.

This was noted

as

a

strength

in Inspection

Report 50-528/93-32.

However,

the inspector

concluded

that the weaknesses

described

above highlighted the

need for improvement in

the use of diagnostic trace information and the thoroughness

of evaluations

in

response

to unresolved

items.

t

7.2. I

Licensee's

Response

to the Unresolved

Item

The licensee initiated

a

CROR evaluation

in response

to the unresolved

item

and concluded that the T(S was properly set

and that the operator

had

sufficient available thrust to close the valve.

The licensee

reviewed

April 1992 diagnostic test data

and noted that Valve SIA-664 had about

4800 pounds of available thrust

and

a running load of 1800 pounds.

The

September

5,

1994,

as-found diagnostic test data

showed

2000 pounds of

available thrust

and

a running load of 2500 pounds.

The licensee

had

an upper

limit on running load of 2580 pounds.

The licensee

subsequently

concluded

that the

TOS was properly set

and that the higher than normal running load

caused

the close

T(S to interrupt the travel of the valve.

The inspector

reviewed the

CRDR and

had the following observations:

~

The licensee

had not evaluated

the operability of Valve SIA-664 between

the two apparent failures to close

on August

17

and September

5,

1994.

~

The licensee

had not determined

what would have

caused

the close

TgS to

trip if there

was still 2000 lbs of available thrust to shut the valve.

The inspector

asked

the licensee

to perform

an evaluation of the operability

of Valve SIA-664 and to review previous diagnostic traces of Valve SIA-664 to

explain the apparent

close

TgS trip on August

17 and September

5.

The

licensee

subsequently

determined that Valve SIA-664 would not'ave

been

able

t

to close

between

the two failures

and was, therefore,

inoperable for about

19 days.

e

7.2.2

Operability Evaluation

-18-

e

The inspector

reviewed the design

basis function of Valve SIA-664 to determine

the safety significance of the valve being inoperable

and to determine

the

subsequent

impact

on the overall operability of the

CS system.

Valve SIA-664

is normally open to provide

a flow path to the refueling water tank

(RWT) when

the

pump is started with the discharge

isolation valves shut.

Valve SIA-664

receives

a recirculation actuation

signal

(RAS) to close during the

recirculation

phase of a loss of coolant accident

(LOCA).

As described

in the licensee's

Updated

Final Safety Analysis Report,

Valve

SIA-664 received

a close signal to prevent the transfer of radioactive fluid

to the

RWT, mitigating

a potential

release

in excess of 10 CFR Part 100

limits.

Additionally, closing SIA-664 would prevent

a loss of inventory in

the containment

sump that eventually could challenge

the operation of the.

safety injection pumps.

TS 3.6.2.

1 required that two independent

CS systems

be operable with the

capability to automatically transfer suction to the containment

sump'n

a RAS.

As described

above,

a specified function of the

RAS is to close the

CS min'imum

flow recirculation valves.

The action statement

of TS 3.6.2. 1 allowed the

licensee

to have

one train of CS inoperable for 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />.

Train A of CS was

inoperable for 19 days,

from August

17 to September

5, l994, in violation of

TS 3 .6 .2 . I (Violation 529/9510-02)

.

The inspector

noted that operators

would receive

an alarm indicating that the

Valve SIA-664 did not move to the closed position

on

a

RAS.

Manual operation

of the valve would be prevented

due to the

assumed

high radiation levels in

the auxiliary building.

However, operators

could secure

the operating

CS

pump

and line-up the

LPSI

pump to provide containment

heat removal.

The licensee's

probability risk assessment

(PRA) group ranked the failure of

Valve SIA-664 as having low safety significance during the scoping of the

generic letter

(GL) 89-10

MOV program.

The

PRA group did not model

a failure

'of Valve SIA-664 because

they assumed

operation of a redundant

valve in the

miniflow line to the

RWT (Valve SIA-660) that also received

a signal to close

on

a

RAS.

Based

on this information, the inspector

concluded that the safety

significance of having Valve SIA-664 inoperable

was low.

The licensee

determined that Valve SIA-664 being inoperable

was reportable

and

planned to submit

an

LER.

The licensee

stated that they would perform an

evaluation of the impact

on the Part

100 dose limit calculations

due to the

known failure of Valve SIA-664 and

an

assumed

single failure of the redundant

Valve SIA-660.

e

-19-

7.2.3 Diagnostic Trace Anomalies

The licensee

reviewed the previous diagnostic traces of Valve SIA-664 in

April 1995

and determined that there

was

an anoma'ly in the springpack

displacement

curves at the beginning of the close stroke that

had not been

previously identified and evaluated.

The licensee typically measures

the deflection of the

HOV springpack during

diagnostic tests.

The deflection of the springpack is directly proportional

to the output torque of the motor operator.

The beginning of a typical spring

pack deflection curve

shows

a steady

increase until the valve stem begins to

move through the valve packing.

At this point, the springpack deflection

should stay at

a relatively constant

value corresponding

to the running load.

The magnitude of the running load is largely determined

by the dynamic

friction between

the packing

and the valve stem.

The spring pack deflection

curve stays

at the running load .value until the valve plug begins to enter the

valve seat..

At this point the spingpack displacement

rapidly'increases

until

the operator

reaches

the point where the torque switch trips

(TST)

and the

valve is fully seated.

The licensee

reviewed the springpack deflection curves for Valve SIA-664 and

noted that the springpack displacement

curves

had

a large spike at the

beginning of the valve stroke that

was significantly larger than the

displacement

corresponding

to the running load.

The licensee

called the

magnitude of the spike the breakaway

torque

and initially thought that it was

caused

by

a combination of the higher than

normal

packing friction and the

unique rotating rising stem

(RRS) valve.

In the

RRS valve, the stem not only

moves vertically, but it also rotates

through the packing.

In most other

HOVs, the valve stem goes vertically through the packing with no rotating

motion.

The inspector

reviewed the previous diagnostic traces for Valve SIA-664 and

noted the value of the as-found

and as-left

breakaway

torque,

the running load

torque,

and the torque at TST.

The inspector

noted that the breakaway

torque

spikes

were consistently larger than the running load

and

on several

occasions

were close to challenging

the close

TgS

and operation of the valve.

The

torque values

are recorded

as

inches of springpack displacement

and were

as

follows:

I

0

l

0

0

Hay 1989

As-Found

As-Left

-20-

Hay 1991

As-Found

As-. Left

September

1994

.

As-Found

As-Left

Break-away

torque

Running

torque

Torque

Switch Trip

0.1335"

0.1490"

0.0785"

0.0610"

0.1685"

0.1645"

0.1771"

0.1103"

0.1674"

0.0820"

0.1362"

0.0171"

0.0425"

0.0834"

0.0132"

0.1923"

0.1537"

0.1771"

7.2.4

Inadequate

Corrective Actions

The inspector

concluded that the licensee

did not evaluate

the impact of'the

breakaway

torque spikes

on operation of Valve SIA-664 in 1989,

1991,

and

1994;

Additionally, the inspector determined that the licensee

was only trending the

minimum available thrust to close the valve

and

had not recognized

the spikes

as

an anomaly that needed

to be trended.

The inspector

noted that after valve

SIA-664 was repacked

in 1994 with the Argo packing,

the breakaway

torque spike

was not present

in the subsequent

traces.

t

The inspector

concluded that the licensee

had several

opportunities to

identify and correct the high breakaway torque which challenged

the

operability of Valve SIA-664 and did not take appropriate corrective actions.

As

a result,

the valve subsequently

failed to perform on two separate

occasions.

The previous opportunities to identify and correct the condition

are outlined below:

7.2.4.

1

Initial Review of 1989

and

1991 Traces

The inspector

noted that the large

breakaway

torque existed since the i'nitial

1989 trace

and could have

been identified during the initial implementation of

the

GL 89-10 program.

For example,

the

May 1991 as-found trace

showed

the

breakaway

torque

as high as the torque at TST,

a condition which should

have

caused

the valve not to stroke.

This fact was not identified and trended.

The inspector

concluded that the

NOV program

was still maturing at that time

and'he

licensee

apparently did not have

enough

experience

to recognize

the

torque spike

as

an anomaly.

7.2.4.2

August

1994 Failure to Close

On August

17,

a control

room operator

attempted

to close Valve SIA-664 and

an

auxiliary operator

observed

the valve stem rotate

about

I/O of a turn.

The

control

room operator

observed

both the

open

and close valve position lights

indicating the valve

had stopped

in mid-position.

Control

room operators

t

subsequently

went to close

again

and Valve SIA-664 went fully closed.

The

valve was then successfully

stroked five more times.

0

0

-21-

The inspector

noted that Unit 2 operators

had not informed the valve services

group of the failure of Valve SIA-664 to close

(Inspection

Report 50-528/94-31).

The inspector

subsequently

informed the valve services

group

and

asked for their

assessment

of the potential

cause of the problem."

The valve services

group

determined that

once the valve began to move only the

MOV torque or limit

switches

could have

stopped

the valve from fully closing.

The valve services

group did not perform any subsequent

testing or inspection of the valve because

they

assumed

that

any evidence of the problem was destroyed

by repeatedly

stroking the valve.

The inspector

concluded that

as

a minimum the licensee

should

have evaluated

the

past diagnostic traces of Valve SIA-664 after the August

17 apparent failure to

close.

Had they reviewed the traces,

they could have identified the abnormally

high running loads

and possibly the breakaway

torque

anomaly.

The inspector

also noted that the licensee

had problems with unexplained

losses

in the available closing torque with similar

RRS valves in Units

2 and 3.

In

some of these

cases,

there

was unusually high running loads

caused

by friction

between'the

valve plug and the valve seat.

The inspector

concluded that the "

licensee

should

have performed

a diagnostic test after the August

17 problem

based

on the

known unpredictable

performance of RRS valves.

The inspector

b'ased

this conclusion

on the fact that the only real indicator of future performance of

an

MOV is

a diagnostic test

and the extent of controls

on the use of diagnostic

equipment

should

be commensurate

with the degree of uncertainty

in the

performance of the valve.

The inspector further noted that the licensee

had

a procedure for troubleshooting

MOV failures in August

1994.

The stated

intent of the procedure

was to "provide

a guideline for use in troubleshooting

MOV actuator failures,

and to ensure that

the cause of the failure determined,

corrected,

and documented".

The inspector

reviewed the procedure

and noted that there

was

a section for troubleshooting

a

TST which would have eventually led the technicians

to perform

a diagnostic test.

7.2.4.3

September

1994 Failure to Close

The inspector

noted that the licensee

had another opportunity to identify and

evaluate

the breakaway

torque spikes after the September

1994 failure to close

and subsequent

root cause of failure analysis.

The inspector

noted that

a review

of the as-found trace

on September

5 clearly shows the break

away torque spike

and the as-left trace after the packing replacement

does not have the breakaway

torque

anomaly.

The inspector

noted that although the valve services

technicians

were analyzing the diagnostic traces,

they were not critically reviewing the

quality of the trace

and were not sensitive to changes

in the characteristics

of

the curves.

The inspector

concluded that if the licensee

had done

a qualitative

review of the

shape of the as-left curve to the as-found

curve they should

have

recognized

the breakaway

torque

anomaly.

The inspector

had

a similar concern with the adequacy of diagnostic trace reviews

following the failure of another

RRS valve to close during

a differential

pressure

test

in March

1994

( Inspection

Reports

50-528/94-09,

Section

2. 1,

and

0

-22-

50-528/94-13,

Section S.l).

This particular failure was caused

by having the

wrong motor pinion gear ratio that caused

the va'lve to stroke too fast.

A

qualitative review of the as-left

and as-found traces clearly showed the

significant drop in stroke time.

The inspector

subsequently

concluded that the

licensee

did not have appropriate

acceptance

criteria because

they did not review

the relative

shape of the curves.

A noncited violation was issued

based

on the

licensee's

indication that

a qualitative assessment

of the shape of the

diagnostic trace would be included in the diagnostic test procedure.

The inspector

reviewed the current diagnostic test procedure

and noted that the

licensee still did not require

a qualitative review of the

shape

and quality of

the trace

and

a comparison of the as-left to the as-found trace.

The licensee

subsequently

determined that there

was

a

CRDR action to add the qualitative

.

assessment

to the diagnostic testing procedure,

but that it was incorrectly

closed without the action being completed.

7.2.5 Transportability of Breakaway

Torque Anomaly

The inspector

asked

the licensee

what were the most susceptible

valves to the

increased

packing loads

and if they had the

new low resistance

Argo 'packing

installed.

The licensee

determined that the most susceptible

valves

were the

RRS

valves with Limitorque SMC-04 actuators.

There are

55 of these

valves

in the

high pressure

safety injection,

LPSI,

CS and reactor coolant

systems

in all three

units.

. The licensee

had repacked all but eight of these

valves with the

new Argo

packing.

The licensee

could not find any documentation

that two of the eight

valves

had ever

been

repacked.

The licensee

evaluated

the traces for the eight

MOVs that did not have the

new

Argo packing

and noted that the spikes existed

in three of these

valves.

The

licensee

concluded that. if the value of the spike exceeded

50 percent of the

TST-.

value then the operability of the valve needed

to be evaluated.

The licensee

determined that the Unit 3 LPSI miniflow isolation Valve SIA-669 had

a spike of

about

55 percent of the

TST value.

The licensee

determined that the packing load

trended

down during maintenance

intervals

and there

was not

an operability

concern with Valve SIA-669.

The inspector

reviewed the licensee's

transportability evaluation

and agreed with

the licensee's

assessment

that there

was not

an

immediate operability concern

with any other

MOVs.

7.2.6

Corrective Actions

The licensee

planned to repack the eight valves that

had not been

packed with the

low resistance

Argo packing in the next available outage.

The licensee

also

agreed

to trend the breakaway

torque

on the valves with the old packing

and

a

sample of the

pew Argo packing to evaluate future performance.

At the exit meeting,

the licensee

agreed to evaluate

what additional tools could

be provided to technicians

performing diagnostic trace

reviews to help them

0

0

e

-23-

identify anomalies

in the traces.

The inspector will review the licensee's

corrective actions

in the response

to the notice of violation.

8

FOLLOWUP ENGINEERING/TECHNICAL SUPPORT

(92903)

8.1

Unresolved

Item 528 9431-01

OPEN:

Letdown Isolation Valve Would Not

Close

In Inspection

Report

( IR) 50-528/94-31,

the inspector identified concerns

regarding

the licensee's

evaluation

and resolution of significant seat

leakage

past the Unit

1 letdown isolation Valve CHB-UV-515.

The valve had initially been

identified as leaking at approximately

40

gpm when closed

in December

1992

and

had not been repaired

when the inspector

observed

a deficiency tag

on the control

room switch associated

with Valve CHB-UV-515 in September

1994.

As noted in

IR 50-528/94-31,

the inspector identified that the licensee

had not evaluated'he

degraded

condition

on the system operability in December

1992 and,

when

an

evaluation

was performed in October '1994, failed to consider all

appropriate'esign

basis.

At the time of the exit meeting for IR 528/94-31,

the licensee

planned to perform

diagnostic testing of Valve CHB-UV-515 during the refueling outage

scheduled

to

begin in April 1995.

In addition,

the licensee

had initiated corrective actions

to address

weaknesses

in the operability evaluation

process.

8. 1. 1 Review of Outage

Maintenance

on Letdown Isolation Valves

Valve CHB-UV-515 is- the

B train air-to-open,

spring to close,

2" globe valve

located

on the reactor coolant

system letdown line to the chemical

and volume

control

system

(CH), located inside containment

upstream of the regenerative

heat

exchanger.

It receives

close signals

on high regenerative

heat

exchanger outlet

temperature

and

a SIAS.

Two similar valves,

CHA-UV-516 and CHB-UV-523, are

located

downstream

and provide inside

and outside containment isolation

respectively.

During the Unit

1 refueling outage,

the licensee

performed diagnostic testing

on

Valve CHB-UV-515.

The licensee

determined that the

"bench set" of the

actuator

was not providing adequate

seating

force to ensure that the valve would remain

closed against reactor coolant

system operating pressure.

The bench set

determines

the

amount of spring pressure

to close

and seat the valve.

Although

the

bench set for Valve CHB-UV-515 was below the values specified for the

actuator,

the licensee

determined that even set at the appropriate

values,

the

valve would have leaked if called

on to close against

a design basis differential

pressures

equivalent to the reactor coolant system relief valve settings.

The deficient seating

force was determined

to also apply to Valves

CHA-UV-516 and

CHB-UV-523.

The licensee

took action to evaluate

the deficiency in accordance

with the operability determination

process for the letdown isolation valves in

the operating units, identify the cause of the deficient design,

and perform

repairs

on the letdown isolation valves in Unit 1.

The inspector

reviewed the

operability determination

and discussed

the repairs with the licensee.

l

0

J,

t

0

8. 1.2

Potential

Generic

Issue

-24-

The licensee

discussed

the finding regarding

low bench set with the vendor of the

actuator,

Fisher Controls.

The vendor determined that the valve actuator

had not

been appropriately

sized for the application.

The vendor found that they had not

properly accounted for valve packing friction in the sizing of the actuator.

At

one point in their design process,

they stopped

using low friction teflon packing

material

and switched to higher friction graphite packing material.

However,'or

some period after the switch, they had not factored the higher friction into the

actuator sizings

The vendor determined that this error was applicable to all of

the letdown isolation valves in all three

Palo Verde units.

NRC Information Notice

( IN) 88-94, "Potentially Undersized

Valve Actuators,"

dated

December

2,

1988,

discussed

the failure of the Fisher Controls to consider

packing friction in val've actuators.

IN 88-94 concluded that valve actuators

shipped after January I, 1977,

had appropriately

accounted for valve packing

.

friction.

The licensee

stated that their initial review of IN 88-94. determined

that applica'ble valves

had all been

shipped after January I, 1977,

and in the

case of 'the letdown isolation valves,

had

been

shipped

in 1978

and

1979.

The discovery that the letdown isolation valve actuators

were not properly s>zed

indicates that the conclusions

in IN 88-94

may not have completely accounted for

all undersized

actuators.

At the

end of the inspection period,

the licensee

was

communicating with the vendor

and planned to perform

an audit.

Preliminarily, it

appeared

that valve actuators

ordered

before

January I, 1977,

and subsequently

shipped,

may have not been appropriately sized.

8. 1.3

Valve Actuator Repairs

The licensee

repaired

the letdown isolation valves

by providing

a stronger

actuator spring,

which allowed

a greater

bench set.

In communication with the

vendor,

they concluded that other valve components

did not need to be replaced.

Subsequently,

the licensee

determined that the actuator

handwheel

could not

support the stronger spring.

As

a result,

the licensee

performed

a safety

evaluation

and determined that the handwheel

was not necessary

for the valve

design functions.

The licensee

performed operability determinations

for the Unit 2 and

3 letdown

isolation valves.

They determined that it was necessary

for two valves to close

to isolate the letdown line against

design basis differential pressure.

They

reviewed the design basis

events for the letdown line valves

and determined that

two valves would be available in all cases

to provide letdown isolation.

The

inspector

reviewed the operability evaluation

and determined that the evaluation

appropriately

addressed

the design conditions identified.

The inspector identified weaknesses

in the operability evaluations

performed

prior to November

1994

and included this

as

an unresolved

item.

In their

t

~ evaluation

in April 199S,

the licensee

determined that the letdown isolation

valves inside containment

provided high energy line break protection

and that

this had not been previously reviewgd.

The licensee

planned to perform

a self-

f

0,

0

-25-

assessment

of the operability review process

as applied to the letdown isolation

valves.

The inspec'tor will review the assessment

in a future inspection report.

8.2

Closed

Unresolved

Item 50-528 94-26-04:

E ui ment

uglification of

Turbine Driven Auxiliar

Feedwater

Pum

Hain Steam

Su

1

8

ass

Valves

This unresolved

item involved an

Eg issue with the solenoid coil of the main

steam

suppl'y bypass

Valves

MSSBVs,

SG-134A and

138A.

The

HSSBVs are solenoid

operated

valves,

normally de-energized,

and are subjected

to high process fluid

temperatures

of 600 degrees

F,

In August

1994 the licensee

determined that the coils

had exceeded

the qualified

life of 20 years

since the actual field temperature

of the coil was about

150

degrees

F more than the temperature

assumed

in the qualification binder for the

coil (the calculated

Eg life was

based

on 204 degrees

F and the actual field

temperature

was

350 degrees

F).

Based

on

a coil temperature

of 350 degrees

F,

the qualified life was reduced to

18 days.

The inspector

was concerned that 'the

licensee

was collecting data for over

a year before they evaluated

the impact to

the plant.

The inspector

was also concerned

about the rigor of the

Eg

evaluations

since there

had

been

several

iterations of the

Eg life of these

solenoids.

~

~

~

8.2. 1

Licensee's

Evaluation

The licensee initiated

a

CRDR to address

the inspector's

concerns

and to

determine

the most accurate

Eg life of the

HSSBV solenoid coils.

The licensee's

evaluation explained the history of the

Eg issues

associated

with the

HSSBVs

and

the basis for the various values for the

Eg life of the valve coils.

The licensee

determined that the calculated

Eg life of 18 days

used the most

limiting activation energy for all the components

in the

HSSBV solenoid coil.

The licensee

contacted

the solenoid valve vendor

and determined that the most

critical component of the solenoid

was the polymide insulation

on the coil wires.

The enginoers

used the activation energy

and aging information for the polymide

insulation

and the service

temperature

of 350 degrees

F and calculated

an

Eg. life

of 9.3 years.

The licensee

subsequently

updated

the

Eg binder for the

HSSBVs

and

the

PM frequency in the

PH basis

database

to reflect the 9.3 year

Eg life.

The inspector

reviewed the licensee's

evaluation

and

had the following

observations:

The licensee

performed

a thorough evaluation

and

had

a good engineering

basis to support

an

Eg life of 9.3 years for the

HSSBV solenoid coil.

Eg engineering

collected temperature

data

on the

MSSBVs for almost

a year

that clearly

showed

in service coil temperatures

that were significantly

higher than the temperature

used in the qualification report

and did not

perform

a relatively simple calculation to determine

the qualified life

based

on these

temperatures.

l

I

l

J

'

-26-

~

The engineer

was collecting data

and did not have

an acceptance

criteria,

or upper threshold,

at which point an evaluation

would be conducted

to

determine if there

may be

a qualification issue.

The inspector

noted that the licensee

recently

implemented

an "attributes of

engineering

excellence"

program to address

the timely and complete resolution of

engineering

issues.

The inspector

concluded that application of these principles

by all levels of the engineering

organization

would help prevent

two year long

evaluations

similar to the

HSSBVs.

The inspector also noted that nuclear

assurance

engineering

(NAE) performed

an

audit in December

1994

and reviewed the adequacy of the licensee's

overall

thermal monitoring program.

8.2.2

Thermal Monitoring Program

The

NAE audit team noted that the licensee

committed to have

a thermal monitoring

program

and that

a formal program did not exist.

The audit team found that

a

1991 audit

had identified the

same

weakness

and the corrective actions

from th'at

audit were ineffective.

The audit team also identified that formal

EQ training

for engineering,

maintenance,

and operations

personnel

was significantly below

the industry average.

The audit team issued

CRDR 9-4-Q184 to the

EQ group to

develop

and

implement

a formal thermal monitoring program.

The inspector

reviewed

EQ engineering's

response

to

CRDR 9-4-Q184

and discussed

the proposed

implementation of the thermal monitoring program with the

EQ

Department

Leader.

. The inspector

noted that the

EQ group reviewed the various

EQ

zones

in the plant

and determined

what areas

were susceptible

to process fluid

heating

and then identified the

EQ components

in these

areas with qualified lives

less

than

15 years.

The licensee

determined that the majority of these

components

were in the main steam support structure

(HSSS)

around the main steam

and main feedwater lines.

The licensee

was in the process of determining the components

to be monitored in

these

areas

and then selecting

a temperature

recording device to install

on these

components.

The monitoring instruments

were scheduled

to be installed

by the

end

of 1995.

The inspector

conducted

a walkdown of the auxiliary building and the

HSSS to

determine

where the process fluid could possibly increase

the temperature

of

EQ

components.

The inspector

agreed with the licensee's

assessment

that the areas

of concern

were primarily in the

HSSS.

The inspector

concluded that the

licensee's initial evaluation to determine

the scope of the thermal monitoring

program

was appropriate.

The inspector

also took field temperature

readings of -the atmosphere

dump valve

(ADV), main steam isolation valve (MSIV), and the feedwater isolation valve

(FWIV) lower limit switches

and compared

them to the temperatures

used in the

qualified life calculation.

The inspector

noted that the qualified life of the

r

-27-

HSIVs and

ADVs was

based

on

a service

temperature

of 150 degrees

and the actual

temperature

in the field was about

150 degrees.

The inspector

noted that 'the

licensee

had

changed

the service temperature for the

MSIV and

ADV lower limit

switches

in the

Eg binder from 130 to

150 degrees'n

January

1994

based

on

measured field temperatures

of 150 degrees.

The licensee

subsequently

changed

the

Eg life of the switches

from 149 to 58 months.

The inspector

noted that field temperature

readings

on the

FWIV lower limit

switches

were reading

around

150 degrees

and that the qualified life was still

based

on

a service temperature

of 130 degrees.

The inspector

asked

the licensee

if they had

been monitoring the temperatures

of the

FWIVs limit switches

and

evaluated

the impact of the higher temperatures

on the life of the switches.

The licensee

informed the inspector that

an

Eg engineer

had

been monitoring the

temperature

of all the limit switches

in the

HSSS

once

a month for the last five

months

and

had also observed

some temperatures

as high

as

150 degrees

on the

lower FWIV limit switches.

The inspector

asked

the

Eg section leader

what field

temperature

would trigger an evaluation of the limit switch qualified life and

how long they planned to collect the temperature

data before

an initial

evaluation

was performed.

The

Eg section leader

informed the inspector that the

Eg engineer did not suspect

a problem with the existing temperatures

of the lower FWIV limit switches

because

the temperatures

were not significantly greater

than the

assumed

value.

Additionally, the

Eg engineer

was

aware of conservatism

in the activation energy

and post accident

environment

used

in the qualification of the limit switches

that would mitigate the small

increase

in service temperature.

The inspector

was

concerned

that the engineer

had not done

any calculations

or modeling of the

field conditions to substantiate

his engineering

judgement

used to arrive at this

conclusion.

The licensee

subsequently

performed

an assessment

of the temperature

data

and

determined that there

was not

a qualification concern if the average

FWIV limit

switch temperature

remained

less

than

140 degrees.

The engineer

had reco. ded

an

average

FWIV lower limit switch temperature

of 137 degrees.

This number

was then

adjusted

to account for a

2 month outage

time with a temperature

of 80 degrees

every

18 months which reduced

the av'erage

temperature

to 133 degrees.

The

inspector

reviewed the evaluation

and agreed with the licensee's

conclusion that

there

was not

an immediate qualification concern with the

FWIV lower limit

switches.

8.2.3

Conclusion

The inspector

concluded that the licensee

should

have

bounded

the temperatures

at

which

a more detailed

review of the qualified life of the components

in the

NSSS

was required.

The inspector

noted that relying exclusively

on engineering

judgement

was not consistent

with the attribute of engineering

excellence that

requires

a "rigorous application of engineering principles".

The stated

value of

this attribute is "the complete

and demonstrated

resolution of an issue with

sufficient technical justification, graphs,

calculations

and associated

analysis

0'

-28-

techniques

so the issue is 'engineered'o

completion

and not just 'justified'o

a conclusion.

This approach

allows for engineering

judgement;

however, it

requires

an in-depth application of calculations,

cost benefit,

and risk

analysis."

The inspector discussed

this apparent

over-reliance

on engineering

judgement

during the collection of the temperature

data with the

Eg section leader

who

agreed with the inspector.

At the exit meeting,

the Director of System

Engineering

stated that they would establish

levels of temperature

differences

between

measured

in-service temperatures

and the temperature

used to qualify each

critical component that would trigger the performance of various levels of

evaluations prior to implementing the formal thermal monitoring program.

The

inspector

concluded that these

actions

were appropriate.

4

V

0

ATTACHMENT 1

1

Persons

Contacted

1. 1

Arizona Public Service

Com an

  • J. Bailey, Vice President,

Nuclear Engineering

  • S. Bauer,

Acting Department

Leader,

Nuclear Regulatory Affairs

  • S. Coppock,

Engineering Supervisor,

Maintenance

Valve Services

  • B. Eklund, Regulatory Consultant,

Nuclear Regulatory Affairs

  • D. Garchow, Director, Site Engineering
  • W. Ide, Director, Operations
  • J. Levine, Vice-President,

Nuclear Production

R. Lucero,

Department

Leader,

Electrical Maintenance

  • D. Hauldin, Director, Maintenance
  • J. Hinnicks, Department

Leader,

Maintenance

Valve Services

M. Muhs, Section

Leader,

System Engineering

M. Radspinner,

Section

Leader,

Design Engineering.

F. Riedel

Department

Leader,

Operations

Unit 2

M. Salazar,

Section

Leader,

Maintenance

Valve Services

  • C. Seaman,

Director, Nuclear Assurance

.

D. Smith,

Department

Leader,

Operations

Unit

1

B. Simpson,

Vice-President,

Nuclear Support

  • W, Stewart,

Executive Vice President,

Nuclear

  • R. Stroud,

Regulatory Consultant,

Nuclear Regulatory Affairs

J. Taylor, Department

Leader,

Operations

Unit 3

"P. Wiley, Department

Leader,

Operations

1.2

NRC Personnel

  • K. Johnston,

Senior Resident

Inspector

  • D, Garcia,

Resident

Inspector

J,

Kramer,

Resident

Inspector

  • A. HacDougall,

Resident

Inspector

1.3

Others

  • J. Draper, Site Representative,

Southern California Edison

  • F. Gowers, Site Representative,

El

Paso Electric

  • R. Henry,

Site Representative,

Salt River Project

  • Denotes those present

at the exit interview meeting held

on May 19,

1995.

The inspector

also held discussions

with and observed

the actions of other

members of the licensee's

staff during the course of the inspection.

2

EXIT MEETING

An exit meeting

was conducted

on May 19,

1995.

During this meeting,

the

inspectors

summarized

the scope

and findings of the report.

The licensee

acknowledged

the inspection findings documented

in this report.

The licensee

did

not identify as proprietary

any information provided to, or reviewed

by, the

inspectors.

J

0

ATTACHMENT 2

1

LIST OF ACRONYMS

ADV

APS

ATTN:

CS

CFR

CRDL

CRDR

CW

ECP

EQ

FWIV

GL

I&C

IN

LER

LOCA

LPSI

HITR

MOV

MSSBV

HSSS

HSIV

NA

NAE

NRC

PM

ppb

PRA

RAS

RCS

RRS

RWT

SIAS

STA

TAPA

TQS

TST

TS

VSDS

WO

atmospheric

dump valve

Arizona Public Service

attention

containment

spray

system

Code of Federal

Regulation

control

room deficiency log

condition report/disposition

request

circulating water system

Employee

Concerns

program

equipment qualification

feedwater isolation valve

Generic Letter

instrumentation

and controls

Information Notice

Licensee

Event Report

loss of coolant accident

low pressure

safety injection system

management

issues. tracking

and resolution

program

.

motor operated

valve

main steam supply bypass

valve

main steam support structure

main steam isolation valve

not applicable

Nuclear Assurance

Engineering

Nuclear Regulatory

Commission

preventive maintenance

parts

per billion

probabilistic risk assessment

recirculation actuation

signal

reactor coolant

system

rotating rising stem

refueling water tank

.

safety injection actuation

signal

shift technical

advisor

temporarily approved

procedure

action

torque switch

torque switch

trip'echnical

Specification

valve survey data

sheet

work order

C

0

0