ML17311A981
| ML17311A981 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde |
| Issue date: | 06/16/1995 |
| From: | Wong H NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV) |
| To: | |
| Shared Package | |
| ML17311A980 | List: |
| References | |
| 50-528-95-10, 50-529-95-10, 50-530-95-10, NUDOCS 9507030010 | |
| Download: ML17311A981 (60) | |
See also: IR 05000409/2005020
Text
ENCLOSURE 2
U.S.
NUCLEAR REGULATORY COMMISSION
REGION IV
Inspection
Report:
50-528/95-10
50-529/95-10
50-530/95-10
Licenses:
NPF-51
Licensee;
Arizona Public Service
Company
P.O.
Box 53999
Phoenix,
85072-3999
Facility Name:
Palo Verde Nuclear Generating Station,
Units 1, 2,
and
3
Inspection At:
Maricopa County, Arizona
Inspection
Conducted:
April 9 through
May 20,
1995
Inspectors:
Approved:
K. Johnston,
Senior Resident
Inspector
J.
Kramer,
Resident
Inspector
A.. MacOougall,
Resident
Inspector
D. Garcia,
Resident
Inspector
F.
Huey, Technical Assistant
ow
ong,
C i
,
eactor
Jects
rane
~ /~AM
ate
Ins ection
Summar
Areas
Ins ected
Units
1
2
and
3
Routine,
announced
inspection of onsite
response
to plant events,
operational
safety,
maintenance
and surveillance
activities, onsite engineering,
and the
Employee
Concerns
Program.
C
Results
Units
1
2
and
3
Plant
0 erations
Overall, performance
in the area of plant operations
was good.
The Unit
1
midloop operation
went very smoothly.
Inspectors
observed
a cautious
approach
to the evolution, with good crew briefings
and well established
command
and
control.
Additionally, operators
responded
well to the loss of reactor
coolant
system
letdown resulting
from a failed instrument fitting on
a
common
charging line.
t
However,
the licensee's
evaluation of operation's
use of an out of calibration
boronometer to obtain
a Technical Specification required reactor coolant
system
sample described
in an
LER did not adequately
address
the cause of the
95070300i0
950627
ADOCK 05000528
8
0
event
and did not identify corrective actions which were applicable to the
problem.
The licensee's
internal
review was also incomplete in that it did
not address
the barriers
in place which should
have alerted the operators
to
the uncalibrated
boronometer.
Maintenance
Maintenance
work was observed
to be good.
There
was noted progress
in the
licensee's
efforts to address
valve packing leaks
by implementing
a
comprehensive
valve packing program.
However, there
were
some indications
that maintenance
personnel
did not place
much emphasis
on work instructions.
In one example,
work was performed
on
an air operated
valve without work
instructions at the job site.
In another
example,
an inverter maintenance
procedure
was issued for Unit I.work with numerous
pen
and ink changes
which
were over two years old.
The inspectors
found that
one of the
pen
and ink
changes
was missing
due to a duplication error.
The changes
made t'e
procedure difficult to follow.
En ineerin
The technical
support to plant operations
and maintenance
continued to be very
good.
The licensee
had addressed
weaknesses
in environmental
qualification
(Eg) monitoring, allowing them to compare
actual plant environmental
conditions with the previously calculated
Eg basis
documents.
It was noted
that
some aspects
of this monitoring lacked engineering rigor.
The engineering
evaluation of the failure of Unit 2 containment
spray Valve
SIA-664 during testing in August
1994
was found to be inadequate.
The
licensee
missed
a number of opportunities
to identify that the valve was
degraded,
including indications in motor operated
valve testing data.
The
inspector's
questions
resulted
in the licensee
determining that the
containment
spray
system
was inoperable for a period of 19 days,
which was
a
violation of plant Technical Specifications.
Mana ement Oversi ht
J
While the licensee
had
made progress
in their Employee
Concerns
Program
(ECP)
and the Management
Issues
Tracking
and Resolution
(MITR) programs
and
employees
expressed
satisfaction with the efforts of the
ECP, the inspector
identified
some programmatic
weaknesses.
These
weaknesses,
which primarily
involved the thoroughness
of evaluations
and formal communications
to the
concerned
employee,
could erode'he
cohfidence
employees
have in management's
concern for their issues if not addressed
promptly.
The two issues
in the enclosed
report that are the subject of the Notice of
Violation involved concerns that
had
been raised previously by inspectors,
but
licensee
followup was weak
and not thorough.
Only after questions
and
prompting
by the inspector
were comprehensive
evaluations
conducted.
e
'
Summar
of Ins ection Findin s:
~
One violation was identified (529/9510-01)
regarding the use of an
uncalibrated
boronometer to satisfy
a Techn'ical Specification
(Section
7. 1).
Both
and Unresolved
Item 529/9431-06,
concerning
the
same subject,
were closed.
~
One violation was identified (529/9510-02)
regarding the failure to
comply with the Technical Specification action statement for an
containment
spray
system valve (Section 7.2).
Unresolved
Item 529/9426-02,
concerning
the
same subject,
was closed.
Unresolved
Item 528/9431-01,
concerning
a leaking Unit
1 letdown
isolation valve,
was reviewed
and left open pending the completion of a
licensee
assessment
(Section 8. 1).
Unresolved
Item 528/9426-04,
concerning
the environmental
qualification
system valves,
was closed
(Section 8.2).
.
Attachments:
1.
Persons
Contacted
and Exit Meeting
e
j
e
DETAILS
1
PLANT STATUS
1.1
Unit
1
Unit
1 began
the inspection period in a refueling outage with core offload in
progress.
On April 9, the licensee
completed
core offload.
On April 27, the
licensee
commenced
core reload
and
on April 30
a fuel assembly
got lodged
as
it was lowered into the core.
The licensee
requested
and
was granted
a Notice
of Enforcement Discretion to allow a higher refueling bridge overload limit.
On May 1, the licensee
successfully lifted the fuel assembly
and
recommenced
core reload
(Section
2. 1).
On May 20, the unit entered
Mode
3 and the
inspection
period
ended with
a reactor
cooTant
system
heatup
in progress.
1.2
Unit 2
Unit 2 began
the 'inspection
period at
100 percent
power.
On April 11,
operators
reduced
power to 40 percent to repair
a condenser
tube leak'hat.
developed
in the
2C condenser
hotwell.
The licensee
repaired
the leaking
tubes
and returned
the plant to
100 percent
power on April 11.
On May 10,
power was reduced to 40 percent
due to
a large intrusion of impurities into
the steam generators
(Section 2.2).
- The licensee
restored
chemistry to normal
and returned to plant to 100 percent
on May 11.
1.3
Unit 3
Unit 3 started
and
ended .the inspection period at
100 percent
power.
'2 ONSITE RESPONSE
TO EVENTS (93702)
2. 1
Fuel
Assembl
Stuck Durin
Core Reload
Unit
1
On April 30, during the Unit
1 core reload,
personnel
on the refueling bridge
r'eceived
an underload
alarm as they were lowering
a fuel assembly into core
location E-12.
During the subsequent
attempt to raise the assembly,
a
refueling bridge overload interlock was received.
Visual examination
revealed
that the assembly
in adjacent
core location,
F-12,
had not been positioned
properly
and
was preventing the movement of the assembly
in position E-12.
The fuel assembly
in E-12 was approximately
2 feet
above the core support
plate
and
was restrained
on three sides
by previously loaded fuel assemblies.
The licensee
made
an attempt at moving the assembly
in a horizontal direction
by operating
the refueling machine manually.
The refueling operators
were
only able to move it approximately
one inch before the load cell indicated
that the fuel assembly
was stuck.
After uns'uccessful
efforts to raise the
assembly,
the licensee
put the refueling operations
on hold pending further
evaluation.
e
0
l
I
Early in the morning
on Hay 1, the licensee
placed
a restraining device
on the
fuel assembly
in the
F-12 core location.
The restraining device
was designed
to prevent the F-12 fuel assembly
from tipping while efforts were
made to
dislodge the fuel assembly
in the E-12 core location.
Refueling operators
again attempted
to move the stuck assembly
manually in the horizontal
direction in which it was not blocked
by another
assembly.
The overload limit
was reached
and the movement
was
suspended.
The licensee
subsequently
requested
a Notice of Enforcement Discretion from
Region
IV to allow raising the Technical Specification
(TS) refueling machine
hoist overload limit from 1600 lbs to 1800 lbs.
The licensee
and the fuel
vendor
had determined that the additional
200 lbs would not impact the core
internals or the pressure
vessel.
They suspected
that there could
be damage
to the fuel assembly grid straps,
but did not anticipate fuel rod damage.
Region
IV granted
the
one time limit adjustment
at 8:45 a.m.
(HST)
on Hay 1,
1995,
based
on
a review of the licensee's
assessment.
On Hay 1, at 1:30 p.m.
(HST), refueling personnel
were able to free the stuck
fuel assembly with a force of 150 lbs over the
TS limit,
The licensee
proceeded
to move the fuel assembly
and adjacent
assemblies
to the spent fuel
pool for a detailed inspection.
The inspectors
observed
most of the significant attempts
at lifting the fuel
assembly. on April 30 and
Hay 1.
The inspectors
observed that the refueling
operators
proceeded
with caution
and in accordance
with pre-established
strategy
and limits.
The cause of the fuel assembly to become
stuck appears
to have
been the
failure to properly seat the fuel assembly
in core location F-12.
The
licensee initiated
a condition report/disposition
request.(CRDR)
to perform
a
review of the failure to seat
the fuel assembly.
The assembly
which was stuck
was
one of the last
25 assemblies
to be put into the core.
Earlier in core
reload,
the licensee
used
a high powered light and
a video camera
located
low
in the core to provide good monitoring of the reload.
However,
as the core
was filled, the camera
and light had to be removed
due to its proximity to
fuel being
moved
and the high radiation fields which could damage
the video
camera.
As
a result, refueling operators verified that the final fuel
assemblies
were properly set using binoculars to see if the assembly
was
located
on the core support pins
and
by reading the "Z" coordinate
on the
refueling machine to verify that the assembly
was fully lowered.
In the case
of the fuel assembly
in location F-12, the "Z" coordinate
indicated the
assembly
was fully lowered.
A refueling operator's 'isual
examination
by
binocular inappropriately
concluded that the assembly
was properly seated.
The inspector
reviewed past
performance
in refueling to determine if there
had
been previous
problems during later stages
of the core reload,
The inspector
noted that there
had not been previous
problems.
The licensee
stated that
they planned to develop
a high powered light designed
so 'that it could
be kept
in the core to the last stages
of the fuel loading
and that this will be in
e
place for the Unit 3 refueling outage
scheduled
for October
1995.
The
inspector
found this to be appropriate.
No violati'ons of NRC requirements
were identified.
2.2
Power Reduction
Due to Increased
Sodium in the Steam Generators
- Unit 2
On Nay 10, at approximately
10:45 a.m.,
operators
observed
a sharp
increase
in
and
a corresponding rise in steam generator
levels.
Within minutes,
the steam generator
sodium levels increased
from
normal levels of less
than
one part per billion (ppb) to over 300 ppb.
The
operators
entered
the condenser
tube rupture procedure
and
began
reducing
power to 40 percent.
At approximately
12:30 p.m., the plant was stabilized at 40 percent
power. and
the IA condenser
shell
was isolated.
The licensee
performed
a tube inspection
and did not identify any leaking circulating water
(CW) tubes.
They suspected
that the source of the impurities was the auxiliary steam condenser
tank that
returns to the
1A hotwell.
The licensee
secured
the auxiliary steam
condenser
tank to the hotwell
and was able to reduce
the sodium
and sulfate levels in
both steam generators.
On Hay II, the licensee
returned
the unit to 100 percent
power.
The inspector
observed
portions of the downpower from the control
room and noted very good
command
and control
from both the shift supervisor
and the control
room
supervisor.
The inspector
also noted that the operators
responded
quickly to
the event
and
had all the condensate
demineralizers
in service within 15
minutes of detecting
the increase
in hotwell sodium levels.
As a result,
the
operators
limited the subsequent
peak of the steam generator
sodium levels.
The inspector
concluded that the operators
were very sensitive to the
importance of steam generator
chemistry
on the integrity of steam generator
tubes.
The licensee
was investigating the exact
source of the impurities in the
auxiliary steam
system
and suspected
a leaking isolation valve between
the
auxiliary steam
and the condenser
hotwell
and
a tube leak in the liquid
radioactive
system evaporator.
The inspector will review the licensee.'s
corrective actions during future routine inspections.
2.3
Loss of Letdown
Unit 2
On Nay 19,
a swage lock connector for the Unit 2
common charging line pressure
transmitter,
CHA-PT-212, failed resulting in
a charging
header to atmosphere
leak in the "E" charging
pump room.
At the time of the event,
the "E" and the
"8" charging
pumps were in operation
and the "A" charging
pump had
a freeze
seal
applied to the discharge line to allow repairs to the discharge
isolation
valve.
The operators
had conducted
a briefing on the contingency
actions to
isolate
seal
injection and stop the remaining charging
pumps if the freeze
seal failed.
e
g
f
At approximately
10:50 p.m., control
room operators
received
a charging
system
trouble alarm
and noticed that both the charging
pressure
and flow
instruments
were reading zero.
Within a minute,
letdown automatically
isolated
on high regenerative
heat
exchanger outlet temperature.
The
operators
implemented the previously briefed contingency actions
and stopped
the running charging
pumps
and isolated
seal
injection.
The leak was stopped
when the operating
charging
pumps were stopped.
The licensee
estimated that
about
300 gallons of charging
system water was
pumped into the
"E" charging
pump room.
The operators
did not detect
any reactor coolant
system
leakage
during the event.
The inspector
reviewed the applicable
alarm response
and operating
procedures
and discussed
the event with the control
room operators.
The inspector
noted
that the operators
appropriately
entered
due to the loss of all
charging
pumps
and the loss of an operable
boration flow path.
The licensee
was in TS 3.0.3 for about
45 minutes.
The inspector
also noted that the
licensee initiated
a
CRDR to evaluate
the root cause of the
swage lock fitting
failure.
The inspector
concluded that the operator
response
to the event, was
good and'that
the licensee's
corrective actions
were appropriate.
The.
licensee
planned to submit
a Licensee
Event Report
(LER).
The inspector will
review the licensee's
evaluation of the event's
cause
and proposed
corrective'ctions
in
a future inspection report.
3 OPERATIONAL SAFETY VERIFICATION
(71707)
3. 1
S stem Midloo
0 erations
Unit
1
On May 12, the inspector
observed
the licensee
drain the
RCS to midloop to
remove
nozzle
dams.
The inspector
reviewed the prerequisites
for draining to midloop and noted
no discrepancies.
The inspector
noted that
the operations staff conducted excellent briefings
and maintained
good
command
and control throughout the evolution.
The inspector
noted that the operators
used all available level indications
as the level
approached
a midloop
condition
and that the operators
often verified the levels were within
acceptable
deviations.
The inspector
observed
active participation of the
shift technical
advisor
and the nuclear
assurance
evaluator.
The inspector
concluded that the licensee
performance
during the midloop evolution was good.
4
MAINTENANCE OBSERVATIONS
(62703)
4. 1
125
VDC Vital Inverters
Unit
1
On April 26,
1995,
the inspector
observed
portions of maintenance
procedure
"Maintenance of Inverters."
The electricians
indicated that the
procedure
was difficult to follow and the inspector
noted that the procedure
had numerous
steps
and
pages that were crossed
out and
hand written steps
were
inserted'he
changes
resulted
from a temporarily approved
procedure
action
~
~
~
~ (TAPA) written in April 1993.
The licensee
approved
the
TAPA, but did not
formally incorporate
the
TAPA into the procedure.
e
The inspector
reviewed the completed
procedure
and noted that the content
was
technically accurate;
however,
the inspector
noted
a controlled copy of the
procedure
appeared
to have
a hand written step missing at the bottom of the
page
due to reproduction.
The inspector notified'lectrical maintenance
about
the discrepancy.
The licensee
reviewed the original
TAPA, and recovered
the
missing step.
The inspector
noted that the step directed the user to
a
subsequent
section in the procedure
and did not contain technical
information.
The licensee
determined that the missing step
had not impacted the performance
of work.
The licensee
subsequently
re-issued
the procedure
which included all
the
hand written steps.
Nuclear Assurance
issued
a
CRDR to evaluate
the problem.
The licensee
looked
at all of the electrical
maintenance
procedures
and determined this was the
only procedure
which had
pen
and ink changes.
The inspector
noted that the
technicians
continued to use the procedure
although the procedure
did not work
as written.
In addition,
the inspector
noted the technicians
did not
adequately
document
steps
marked
NA (not applicable).
The inspector 'discussed
the maintenance
workers'eaknesses
with the electrical
maintenance
department
leader.
The electrical
maintenance
department
leader
acknowledged
the
weaknesses,
indicated that the technician
performance
did not meet his
expectations,
and stated
he would discuss
the weaknesses
with the technicians.
The inspector
concluded that the technicians
were knowledgeable
about the
maintenance
task
and followed a logical progression
through the procedure
and
correctly performed the task,
although
one step
was missing.
4.2
Air 0 crated
Valve Dia hra
m Re lacement
On April 21, the inspector
observed
a preventative
maintenance
task to replace
internal
components
in the operator for Valve SIA UV 560, the inside
containment isolation valve to the reactor drain tank.
]he inspector
observed
the mechanic
change
the o-rings
on the stem bushing
and install
a
new
diaphragm for the air operator.
The inspector
noted that the mechanic lubricated
and installed the o-rings
and
installed the
new diaphragm,
but did not have the work order available at the
job site.
The mechanic stated that another
member of his team
had taken the
package
back to the
shop to get
a tool.
The inspector.
noted that the mechanic
could not review the actions
he completed
because
he could not refer to the
work order instructions.
The inspector discussed
this observation with the responsible
mechanical
maintenance
section leader
who agreed that not having the
WO available at the
job site did not meet management's
expectation.
The inspector also reviewed
the
WO instructions
and discussed
the scope of the job with the technician.
The inspector
concluded that the mechanic
completed
the job in accordance
with
the
WO instructions.
The section leader
counseled
the mechanic
concerning
management's
expectations
for having the work order available at the job site.
The inspector
concluded
e
0
0
that the licensee's
corrective actions
were appropriate
and that this appeared
to be
an isolated incident.
4.3
Valve Packin
Pro
ram Review
The inspector
conducted
a review of the licensee's
valve packing program to
determine
how the licensee
was identifying and correcting old packing
configurations
in safety-related
valves.
The inspector wanted to determine
the number of other safety-related
motor operated
valves
(NOVs) which could
have
had obsolete
packing configurations like the
10 braided rings found in
Valve SIA 664 that subsequently
prevented
the valve from closing
(Section 8.2).
The inspector
reviewed the licensee's
valve packing procedure
and
specification,
interviewed the maintenance
engineer
responsible for the valve
packing program,
observed
repacking of several
valves,
and conducted
walkdowns
to assess
the condition of the valve packing.
4.3. I
Program Description
The licensee's
valve packing program required valves to be repacked
using
a
standard five ring packing configuration consisting of three die-formed rings
and two braided
end rings.
The Maintenance
Department
planned to repack
safety-related
valves
as part of a preventive maintenance effort instead of
repacking
in response
to identified leaks
as
has
been
past practice.
The
licensee's
program allowed
some variation in this configuration depending
on
the depth of the stuffing box and the existence
and location of a leak off
port.
The inspector
also noted that valves with unique packing configurations
and/or materials
were individually exempted
from the valve packing program.
The inspector
concluded that the licensee's
packing program
was adequately
described
in the procedure
and packing specification.
The inspector
noted that the mechanical
maintenance
engineering
was the
process
owner of the licensee's
valve packing program
and was generating
a
valve packing data
base that contained specific packing information for every
valve in the plant.
The data
base
was generated
from information gathered
in
the field during valve packing replacements.
The valve specific information
was recorded
on
a valve survey data
sheet
(VSDS).
The
VSDS included
information such
as the dimensions of the stuffing box, the type,
number
and
order of packing rings
and spacers,
and the as-left packing gland torque.
The
inspector
concluded that the licensee's
effort to generate
an accurate
data
base of the field condition of packing
was
a strength.
4.3.2
Observation of Work in Field
The inspector
observed
mechanics
repack Valve SIA 647,
a motor-operated
high
pressure
safety injection system valve in Unit 2,
and Valve
CHB 337,
a manual
charging
system valve in Unit I,
The inspector
noted that the work was
performed
by contractors,
The Atlantic Group.
The inspector
concluded that
the valve repacking activities were appropriately
conducted.
e
-10-
The Director of Maintenance
hired
The Atlantic Group to repack the valves in
the plant
as part of the "Level I" action to reduce
the number of packing
leaks
in the plant.
The inspector
noted that during the Unit
1 refueling
outage
there
were
a total of 82 valves that were repacked
and
16 of these
valves
were safety-related
MOVs.
During the Unit 2 refueling outage
completed
in March 1995,
a total of 95 valves
were repacked
and
32 were safety-related
MOVs.
The inspector
concluded that the licensee
was aggressively
implementing
the valve repacking
program to improve the material condition of the plant
and
the performance of the valves.
The inspector
also noted that the licensee
was
placing
a high priority on repacking
the safety-related
MOVs.
4.3.3
Plant
Walkdown
The inspector identified three
MOVs with apparent
packing leaks in Unit 2 on
May 3 during
a routine plant tour.
The inspector
informed the valve services
group
and they conducted
a walkdown of the safety-related
MOVs in Unit 2 and
.
identified nine more valves with apparent
The licensee
subsequently 'determined that eight of the
12 leaking valves
had
been
identified two weeks earlier during operations
and engineering
system
~
walkdowns.
The inspector determined that four of the leaking valves
had
been
repacked
during the Unit 2 outage
in March 1994.
The inspector
noted that
Valve CH-524 had
a significant leak and appeared
to be actively leaking.
The
other three valves
had small
amounts of boron
on the
stem but did not appear
to be actively leaking.
The inspector
reviewed the
VSDS for Valve CH-524
and noted that the valve was
properly packed
and that the packing gland
was properly tightened,
The
inspector
noted that for motor-operated
valves the mechanics
tighten the
packing gland towards the low end of the required torque range.
The valve
services
group subsequently
performs
an as-left static diagnostic test
and
ensures
that the packing load is within the proper range.
The diagnostic test
procedure
has general
guidance that the running load should
be around
1000 pounds for each
inch of stem diameter.
The procedure
also includes
guidance that the packing should
be tightened if the running load is low.
The
inspector
concluded that the licensee's
procedures
provided adequate
guidance
to ensure that the packing of MOVs was not tightened
too much, which would
prevent operation of the valve, or not tightened
enough,
which would allow the
valve to leak.
The inspector
noted that the as-left diagnostic test
was performed with the
system filled, but not at normal operating
pressure
and temperature.
As
a
result,
the packing
may have loosened
causing
the valve to leak when the
normal
system pressure
was reached.
The licensee
agreed
to evaluate
optimizing when as-left diagnostic tests
were conducted
to minimize the
potential for subsequent
The inspector
concluded that the
I
'icensee's
actions
were appropriate.
0
)
j
4.4
Other Maintenance
Work Observed
The inspector
observed
portions of the work listed below:
isolation valve accumulator nitrogen pre-charge - Unit l.
Disassembly of turbine driven auxiliary feedwater governor valve
Unit'.
5
SURVEILLANCE OBSERVATION
(61726)
5.1
Hain Steam Isolation Valve Testin
- Unit 2
On Hay 9, the inspector
observed
a portion of main steam isolation valve
(HSIV) partial stroke testing in Unit 2.
The inspector
observed
good
command
and control of the test
by the control
room supervisor
and the reactor
operator performing the test.
For example,
the operators
had briefed
contingency actions
in the event
a MSIV went closed
and were closely
monitoring the HSIV accumulator
pressure
for any abnormal
trends during the
test.
The inspector also observed
good communications
between
the auxiliary
'perator
and the control
room operators.
t
6
EMPLOYEE CONCERNS
PROGRAM REVIEW
The inspector
performed
a review of the licensee's
Employee
Concerns
program
(ECP)
and Management
Issues
Tracking
and Resolution
program
(MITR).
The fCP
provided employees -a method for independent
review of .technical
and safety
concerns
which they did not consider
would be adequately
resolved
through
normal
processes.
The MITR provide employees
similar review for personnel
and
administrative
concerns.
The inspector
noted the following concerns:
The HITR program
was not covered
by
a formal administrative
procedure.
As
a result,
there
were
no consistent
requirements
for the content of
MITR files or how they were to be closed.
Review of Files 94-01,
94-06,
94-09,
and 94-10 identified the following concerns:
~
The files did not include appropriate
documentation of evaluation
conclusions.
~
The files did not include final closure
correspondence
to the
concerned
employee
as to how each of his concerns
had
been
resolved.
~
The files did not indicate
any cause
evaluation or corrective
action for problems that
had resulted
in val'id employee
concerns.
~
HITR 94-43 addressed
a discrimination concern involving an engineer
having raised
safety concerns
about
ADV problems
in
a report to the
NRC.
It was not clear from the file that the concern
received
an
1
-12-
appropriately
independent
evaluation,
in that the concern
was evaluated
by the Vice President
to whom the alleged discriminating manager
was
a
direct report.
The file did not clearly .indicate the basis for Human
Resources
department
conclusion that this
I'0 CFR 50.7 concern
received
a
proper
independent
evaluation.
~
The inspector identified the following concerns
in a review of ECP
administrative guidelines
ECPOI,
ECP02,
ECP03,
and
ECP Files 94-07-03,
94-08-05,
94-09-02,
94-10-03,
and 95-01-07:
ECP Administrative Guideline
ECP02
(Handling
ECP Concerns)
does
not require
an initial letter to the concerned
employee
which
clearly documents
the scope of his concerns,
the plan of action to
evaluate
the concerns,
or. the target
schedule for completing the.
evaluation,
nor does
the guideline require
a final closure letter
to the employee
which documents
the conclusions
and
actions'esulting
from the
ECP evaluation.
~
File 95-01-07 did not indicate
any cause
evaluation or corrective
action for the mishandled
CRDR investigation
problem identified
during the
ECP evaluation.
ECP Administrative Guideline
ECP03
(ECP File Open Action Tracking)
does
not address
completion of file closure followup with the
employee.
~
The cl'osure followup for File 94-10-03 indicated that the employee
considered
that the hostile work environment
in his work group
had
"drastically improved," however,
NRC interview with the concerned
employee indicated that the employee still considers that
supervisors
in his work group are hostile to employees
raising
safety concerns.
~
The inspector interviewed six employees
who recently
had discrimination
concerns
evaluated
by the
and
MITR programs.
These
employees
consistently
expressed
satisfaction with the performance of the
program;
however,
some stated that they were not confident with the
MITR
program,
indicating that the
Human Resources
department
was not viewed
as
a credible organi'zation for representing
employee's
best interests.
The employees
also stated that senior licensee
managers
had increased
their credibility in recent
months
such that employees
were confident
that they could
come to senior managers
and receive fair treatment,
without fear of retribution.
However,
several
of the
same
employees
stated that they did not have similar confidence
in lower levels of
licensee
management
and supervision.
While progress
had
been
made in the
MITR and
and employees
seemed
generally satisfied with the
ECP,
some programmatic
weaknesses
were
identified.
The inspector discussed
these
findings with licensee
senior
~
'
0
-13-
management
responsible
for the
and
MITR programs.
They concurred with the
inspector's
observations
and planned to implement appropriate corrective
actions.
7
FOLLOWUP MAINTENANCE (92902)
7. 1
Closed
LER 529 94-008
and
Closed
Unresolved
Item 529 9431-06:
Use
Of Uncalibrated
Boronometer
Caused
a
TS Action to be Missed
This
LER involved operator's
use of an uncalibrated
boronometer to verify
system
boron concentration
in order to comply with the
compensatory
action requirements
of TS 3. 1.2.7 which applied in February
and
March
1994
when
a startup
channel
was
removed
from service.
7.1.1
Background
Instrumentation
and Controls
(I&C) technicians
began
a routine 18-month
calibration of the Unit 2 boronometer
in February
1993.
They found that
one
of the instrument's
power supplies did not meet the calibration
specifications.
The
ILC technicians
stopped
work, left the boronometer
in
service, notified the Shift Supervisor,
and proceeded
to attempt to procure
a
new power supply.
The boronometer
appeared
to be operating
adequately
in that
it closely tracked reactor coolant
system
boron concentration.
The
ILC group performed work on the boronometer sporadically until November
1994.
Near the time the calibration
was completed
in November
1994,
the
licensee identified that the boronometer
may have
been
used
by operators,
during the Unit 2 refueling outage in February
and March 1994, to satisfy the
compensatory
action requirements
of TS 3. 1.2.7 which apply when
a startup
channel
was
removed
from "service.
The licensee
concluded this review in
January
1995,
having found that the boronometer
was relied
on in three
occasions.
On one of the occasions, if no credit is provided for the
use of
the boronometer to meet the
TS action,
the
TS action
was not complied with.
The inspector
noted that, despite
the questionable
calibration status of the
boronometer,
during the period it was
used for TS compliance, it remained
within
1 to
2 percent of the
RCS samples.
The inspector
reviewed the
LER and found that there were significant
weaknesses
in the evaluation of the cause of the event
and the corrective
actions
taken.
Further,
the inspector
found that the licensee's
internal
evaluations
which support the
LER were weak.
7. 1.2
Evaluation of the
Cause of the Event - Operations
The
LER statement
of cause
states,
in part:
"An evaluation
was performed
in accordance
with the
APS Incident
Investigation
Program.
The evaluation
concluded that the apparent
cause
of the boronometer
past calibration
was
a lack of concern for the
instrument
being calibrated correctly."
I
e
-14-
The inspector
noted that the cause, statement
did not address
the barrier
necessary
to prevent
an operator
from using
an boronometer of questionable
calibration to satisfy
a
TS requirement.
The inspector determined,
through
discussions
with operations
personnel,
that the appropriate
administrative
control would have
been for operators
to have
made
a control
room deficiency
log
(CROL) entry with a
CRDL tag placed
on the control board.
The licensee's initial CROR (2-4-343) evaluation identified that
I&C had
informed the shift supervisor that the calibration of the boronometer
had not
been completed.
The
CROR noted that
a second
CRDR (2-4-4?1)
was initiated for
operations
to evaluate
how the boronometer
was allowed to be used while it was
being calibrated.
CROR 2-4-471 identified that the issue
was reportable
under
but did not provide any root cause
review.
The inspector
found
that,
CROR 2-4-343
was closed
based
on the evaluation to be performed in
CROR 2-4-471.
However,
CRDR 2-4-471
was subsequently
closed without a
documented
evaluation of cause
and without further corrective actions
based
on
the evaluation
provided in
CRDR 2-4-343.
As
a result,
there
was
no documented
evaluation of operations failure to identify and tag the boronometer
'eficiency.
7, 1.3
Evaluation of the
Cause of the Event - Maintenance
The licensee
had
made
some effort to identify what the inspector considered
to
be
a contributing cause
concerning
the lack of a timely calibration of the
boronometer.
The licensee identified that the calibration
had not been given
priority due to the fragmentation of the responsibility for the resolution of
equipment reliabili.ty problems.
The inspector
reviewed this evaluation
and found that it lacked rigor.
The
preventive maintenance
(PH) task to calibrate the boronometer
was considered
to be
an "operations surveillance test no-waive
PH."
The
PH task,
which had
an 18-month frequency,
had last
been completed
on September
26,
1991,
and
was
due
on March 26,
1993.
The licensee
allowed
a grace period to August 23,
1993.
The boronometer
was recognized
in its
PH basis
documentation
as
providing post-accident
indication in accordance
with Regulatory
Guide 1.97:
On July 29,
1993,
an
I&C foreman performed
a
PM work order disposition report
to delay the calibration of the boronometer.
No justification was provided
and the due date
was extended
to September
26,
1993.
The work was not
completed
in September
1993
and
no further delay requests
were documented.
In September
1994,
a Shift Technical Advisor discovered that the
PM had not
been completed.
The
I&C section leader
performed
a
PH work order disposition
report to waive the original
PH task.
CRDR 2-4-343
was initiated
and the
calibration
was completed
in November
1994 under
a new work order.
The waiver
of a "no-waive"
PH was considered
an Unresolved
Item in Inspection
Report
94-31.
The evaluation
in
CRDR 2-4-343,
which was included in the
LER, discussed
the
lack of priority placed
on calibrating the boronometer.
However, it did not
f
address
the specifics of what barriers
are provided to ensure that priority is
appropriately
placed
and
how these barriers
may have
been defeated.
Specifically:
~
Regarding
the disposition
on July 29,
1993,
how could
a "no-waive"
PM be
delayed with no documented justification for the delay?
~
After the first delay expired
on September
26,
1993,
why were there
no
subsequent
"no-waive"
PM dispositions
documented?
~
Why were there
no controls in place to ensure that post-accident
monitoring instrumentation
referred to in the Updated Final Safety
Analysis Report
be returned to service
in
a expeditious
manner?
7. 1.4
Evaluation of Corrective Acti'ons
The inspector
reviewed the actions to prevent recurrence
discussed
in the
LER.
The
LER stated that the following step
had
been
added to the
PM Process
and
.
Activities procedure:
" 18C Section/Team
Leaders
shall refer out-of tolerance results
found
during Operations
Surveillance
Tests
"NO WAIVE" PMs
on installed plant
equipment to the duty
STA or other appropriate
engineering
personnel.
The STA/engineer shall
perform
a documented
evaluation of the
significance of the problem or deficiency
and ensure that
a
CRDR is
generated, if required."
The inspector
found that this step did not address
any of the problems
noted
in the
LERs or the
CRDRs associated
with the boronometer.
The inspector
interviewed
a Shift Technical Advisor (STA),
a Control
Room Supervisor,
a
Shift Supervisor,
an
I8C Team Leader,
and
an
18C Section leader
and determined
this step.was
intended to ensure that out-of-tolerance results
are
appropriately
reviewed for their impact
on tests for which they may have
been
relied
upon
and to trend
and assess reliability of the instrument.
Typically,
as-found out-of-tolerance
results
are quickly resolved
and the instrument
placed
back in service with no impact
on ongoing plant operations.
Therefore,
IKC typically refers
subsequent
out-of-tolerance
reviews to
IKC maintenance
engineers
and not the STAs.
Furthermore,
the inspector
noted that the boronometer calibration procedure
included
two steps requiring
18C to notify the control
room of out-of-
tolerance
findings
and that this was done for the subject calibrations.
The
LER concludes
by stating that the evaluation of the event
had not been
completed.
The
LER was issued
on February
7,
1995.
All open evaluations
of
the issue
were closed
by February
14,
1995, without further documented
review.
0
I
l
'I
-16-
7. 1.5
Conclusion
The failure to assure
that
an instrument
used to meet
TS requirements
was
properly calibrated is
a violation of 10
CFR Part .50,
Appendix B,
Criterion XII, concerning
the control of measuring
and test equipment
(Violation 529/9510-01).
The inspector
noted that the licensee identified
this violation in December
1994-.
However,
the inspector considered
that the
licensee
had not performed
an adequate
cause
review nor identified appropriate
corrective actions to prevent recurrence.
The inspector considered
this to be
potentially significant in this case
since it appeared
that both licensee's
corrective actions
and event report programs failed to ensure that
an adequate
review was performed.
7.2
Closed
Unresolved
Item 50-529 94-26-02:
Failure of Motor-0 crated
Valve to Close
This unresolved
item involved the failure of a Unit 2 containment
spray
(CS)
miniflow isolation Valve SIA-664 to close during
a surveillance test
.on
September
5,
1994.
The valve also failed .to close
on August
17,
1994,
when
'perators
attempted
to verify the valve was
open
by closing the valve.
The licensee
conducted
a root cause of failure analysis
and determined that
t
the failure of Valve SIA-664 to close
was caused
by an excessive
running load.
The licensee
determined that the excessive
running load was caused
by an
obsolete
packing configuration
(10 braided rings
and
a lantern ring) .that
was
originally installed in the valve.
The valve was repacked
using low
resistance
packing,. manufactured
by Argo,
and returned to service.
The
licensee
also
changed
the bill of materials to specify the Argo packing
and
initiated work requests
to repack the Unit
1
CS and low pressure
safety
injection (LPSI)
pumps miniflow valves during the Unit
1 refueling outage
in
April 1995.
The inspector initiated the unresolved
item to assess
whether the torque
switch (T(S)
was correctly set
and if the licensee's
response
to the initial
failure to close
on August
17 was appropriate.
The licensee initiated
CROR 2-4-0301 to address
the inspector's
questions.
The inspector
reviewed the licensee's
CROR evaluation,
reviewed all the
previous diagnostic traces for Valve SIA-664,
and
had
numerous
discussions
with members of the valve services
group.
In summary,
the inspector
noted the
following weaknesses
with the licensee's
overall
response
to the problems with
Valve SIA-664:
~
The licensee
did not conduct
a thorough review of the issue
in response
to the unresolved
item.
The inspector
had to prompt
a more thorough
review which eventually led to discovering that Valve SIA-664 was
between
August
17 and September
5,
1994.
0
~,
-17-
The licensee
did not perform
a thorough, critical review of past
diagnostic traces
to assess
the future performance of the Valve SIA-664.
As
a result,
the licensee failed to identify an anomaly in the
diagnostic traces of Valve SIA-664 which wa's
a precursor to the failures
on August
17 and September
5,
1994.
The licensee
did not have
any requirements
for the technicians
analyzing
diagnostic traces
to perform
a qualitative assessment
of the quality and
shape of the traces.
The inspector pointed out to the valve services
group the potential
problems with not performing this type of
qualitative assessment
of the diagnostic traces
in Inspection
Report 50-528/94-13
and the licensee
did not provide any additional
tools or training to help the technicians
recognize trace
anomalies.
The inspector
noted that the response
by the valve services
group to this
problem was poor,
but not typical.
For example,
the licensee
identified: and
promptly corrected
anomalies
in butterfly'valve traces.
This was noted
as
a
strength
in Inspection
Report 50-528/93-32.
However,
the inspector
concluded
that the weaknesses
described
above highlighted the
need for improvement in
the use of diagnostic trace information and the thoroughness
of evaluations
in
response
to unresolved
items.
t
7.2. I
Licensee's
Response
to the Unresolved
Item
The licensee initiated
a
CROR evaluation
in response
to the unresolved
item
and concluded that the T(S was properly set
and that the operator
had
sufficient available thrust to close the valve.
The licensee
reviewed
April 1992 diagnostic test data
and noted that Valve SIA-664 had about
4800 pounds of available thrust
and
a running load of 1800 pounds.
The
September
5,
1994,
as-found diagnostic test data
showed
2000 pounds of
available thrust
and
a running load of 2500 pounds.
The licensee
had
an upper
limit on running load of 2580 pounds.
The licensee
subsequently
concluded
that the
TOS was properly set
and that the higher than normal running load
caused
the close
T(S to interrupt the travel of the valve.
The inspector
reviewed the
CRDR and
had the following observations:
~
The licensee
had not evaluated
the operability of Valve SIA-664 between
the two apparent failures to close
on August
17
and September
5,
1994.
~
The licensee
had not determined
what would have
caused
the close
TgS to
trip if there
was still 2000 lbs of available thrust to shut the valve.
The inspector
asked
the licensee
to perform
an evaluation of the operability
of Valve SIA-664 and to review previous diagnostic traces of Valve SIA-664 to
explain the apparent
close
TgS trip on August
17 and September
5.
The
licensee
subsequently
determined that Valve SIA-664 would not'ave
been
able
t
to close
between
the two failures
and was, therefore,
inoperable for about
19 days.
e
7.2.2
Operability Evaluation
-18-
e
The inspector
reviewed the design
basis function of Valve SIA-664 to determine
the safety significance of the valve being inoperable
and to determine
the
subsequent
impact
on the overall operability of the
CS system.
Valve SIA-664
is normally open to provide
a flow path to the refueling water tank
(RWT) when
the
pump is started with the discharge
isolation valves shut.
Valve SIA-664
receives
a recirculation actuation
signal
(RAS) to close during the
recirculation
phase of a loss of coolant accident
(LOCA).
As described
in the licensee's
Updated
Final Safety Analysis Report,
Valve
SIA-664 received
a close signal to prevent the transfer of radioactive fluid
to the
RWT, mitigating
a potential
release
in excess of 10 CFR Part 100
limits.
Additionally, closing SIA-664 would prevent
a loss of inventory in
the containment
sump that eventually could challenge
the operation of the.
safety injection pumps.
1 required that two independent
CS systems
be operable with the
capability to automatically transfer suction to the containment
sump'n
a RAS.
As described
above,
a specified function of the
RAS is to close the
CS min'imum
flow recirculation valves.
The action statement
of TS 3.6.2. 1 allowed the
licensee
to have
one train of CS inoperable for 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />.
Train A of CS was
inoperable for 19 days,
from August
17 to September
5, l994, in violation of
TS 3 .6 .2 . I (Violation 529/9510-02)
.
The inspector
noted that operators
would receive
an alarm indicating that the
Valve SIA-664 did not move to the closed position
on
a
RAS.
Manual operation
of the valve would be prevented
due to the
assumed
high radiation levels in
the auxiliary building.
However, operators
could secure
the operating
pump
and line-up the
pump to provide containment
heat removal.
The licensee's
probability risk assessment
(PRA) group ranked the failure of
Valve SIA-664 as having low safety significance during the scoping of the
generic letter
(GL) 89-10
MOV program.
The
PRA group did not model
a failure
'of Valve SIA-664 because
they assumed
operation of a redundant
valve in the
miniflow line to the
RWT (Valve SIA-660) that also received
a signal to close
on
a
RAS.
Based
on this information, the inspector
concluded that the safety
significance of having Valve SIA-664 inoperable
was low.
The licensee
determined that Valve SIA-664 being inoperable
was reportable
and
planned to submit
an
LER.
The licensee
stated that they would perform an
evaluation of the impact
on the Part
100 dose limit calculations
due to the
known failure of Valve SIA-664 and
an
assumed
single failure of the redundant
Valve SIA-660.
e
-19-
7.2.3 Diagnostic Trace Anomalies
The licensee
reviewed the previous diagnostic traces of Valve SIA-664 in
April 1995
and determined that there
was
an anoma'ly in the springpack
displacement
curves at the beginning of the close stroke that
had not been
previously identified and evaluated.
The licensee typically measures
the deflection of the
HOV springpack during
diagnostic tests.
The deflection of the springpack is directly proportional
to the output torque of the motor operator.
The beginning of a typical spring
pack deflection curve
shows
a steady
increase until the valve stem begins to
move through the valve packing.
At this point, the springpack deflection
should stay at
a relatively constant
value corresponding
to the running load.
The magnitude of the running load is largely determined
by the dynamic
friction between
the packing
and the valve stem.
The spring pack deflection
curve stays
at the running load .value until the valve plug begins to enter the
valve seat..
At this point the spingpack displacement
rapidly'increases
until
the operator
reaches
the point where the torque switch trips
(TST)
and the
valve is fully seated.
The licensee
reviewed the springpack deflection curves for Valve SIA-664 and
noted that the springpack displacement
curves
had
a large spike at the
beginning of the valve stroke that
was significantly larger than the
displacement
corresponding
to the running load.
The licensee
called the
magnitude of the spike the breakaway
and initially thought that it was
caused
by
a combination of the higher than
normal
packing friction and the
unique rotating rising stem
(RRS) valve.
In the
RRS valve, the stem not only
moves vertically, but it also rotates
through the packing.
In most other
HOVs, the valve stem goes vertically through the packing with no rotating
motion.
The inspector
reviewed the previous diagnostic traces for Valve SIA-664 and
noted the value of the as-found
and as-left
breakaway
the running load
The inspector
noted that the breakaway
spikes
were consistently larger than the running load
and
on several
occasions
were close to challenging
the close
TgS
and operation of the valve.
The
torque values
are recorded
as
inches of springpack displacement
and were
as
follows:
I
0
l
0
0
Hay 1989
As-Found
As-Left
-20-
Hay 1991
As-Found
As-. Left
September
1994
.
As-Found
As-Left
Break-away
Running
Torque
Switch Trip
0.1335"
0.1490"
0.0785"
0.0610"
0.1685"
0.1645"
0.1771"
0.1103"
0.1674"
0.0820"
0.1362"
0.0171"
0.0425"
0.0834"
0.0132"
0.1923"
0.1537"
0.1771"
7.2.4
Inadequate
Corrective Actions
The inspector
concluded that the licensee
did not evaluate
the impact of'the
breakaway
torque spikes
on operation of Valve SIA-664 in 1989,
1991,
and
1994;
Additionally, the inspector determined that the licensee
was only trending the
minimum available thrust to close the valve
and
had not recognized
the spikes
as
an anomaly that needed
to be trended.
The inspector
noted that after valve
SIA-664 was repacked
in 1994 with the Argo packing,
the breakaway
torque spike
was not present
in the subsequent
traces.
t
The inspector
concluded that the licensee
had several
opportunities to
identify and correct the high breakaway torque which challenged
the
operability of Valve SIA-664 and did not take appropriate corrective actions.
As
a result,
the valve subsequently
failed to perform on two separate
occasions.
The previous opportunities to identify and correct the condition
are outlined below:
7.2.4.
1
Initial Review of 1989
and
1991 Traces
The inspector
noted that the large
breakaway
torque existed since the i'nitial
1989 trace
and could have
been identified during the initial implementation of
the
GL 89-10 program.
For example,
the
May 1991 as-found trace
showed
the
breakaway
a condition which should
have
caused
the valve not to stroke.
This fact was not identified and trended.
The inspector
concluded that the
NOV program
was still maturing at that time
and'he
licensee
apparently did not have
enough
experience
to recognize
the
torque spike
as
an anomaly.
7.2.4.2
August
1994 Failure to Close
On August
17,
a control
room operator
attempted
to close Valve SIA-664 and
an
auxiliary operator
observed
the valve stem rotate
about
I/O of a turn.
The
control
room operator
observed
both the
open
and close valve position lights
indicating the valve
had stopped
in mid-position.
Control
room operators
t
subsequently
went to close
again
and Valve SIA-664 went fully closed.
The
valve was then successfully
stroked five more times.
0
0
-21-
The inspector
noted that Unit 2 operators
had not informed the valve services
group of the failure of Valve SIA-664 to close
(Inspection
Report 50-528/94-31).
The inspector
subsequently
informed the valve services
group
and
asked for their
assessment
of the potential
cause of the problem."
The valve services
group
determined that
once the valve began to move only the
switches
could have
stopped
the valve from fully closing.
The valve services
group did not perform any subsequent
testing or inspection of the valve because
they
assumed
that
any evidence of the problem was destroyed
by repeatedly
stroking the valve.
The inspector
concluded that
as
a minimum the licensee
should
have evaluated
the
past diagnostic traces of Valve SIA-664 after the August
17 apparent failure to
close.
Had they reviewed the traces,
they could have identified the abnormally
high running loads
and possibly the breakaway
anomaly.
The inspector
also noted that the licensee
had problems with unexplained
losses
in the available closing torque with similar
RRS valves in Units
2 and 3.
In
some of these
cases,
there
was unusually high running loads
caused
by friction
between'the
valve plug and the valve seat.
The inspector
concluded that the "
licensee
should
have performed
a diagnostic test after the August
17 problem
based
on the
known unpredictable
performance of RRS valves.
The inspector
b'ased
this conclusion
on the fact that the only real indicator of future performance of
an
MOV is
a diagnostic test
and the extent of controls
on the use of diagnostic
equipment
should
be commensurate
with the degree of uncertainty
in the
performance of the valve.
The inspector further noted that the licensee
had
a procedure for troubleshooting
MOV failures in August
1994.
The stated
intent of the procedure
was to "provide
a guideline for use in troubleshooting
MOV actuator failures,
and to ensure that
the cause of the failure determined,
corrected,
and documented".
The inspector
reviewed the procedure
and noted that there
was
a section for troubleshooting
a
TST which would have eventually led the technicians
to perform
a diagnostic test.
7.2.4.3
September
1994 Failure to Close
The inspector
noted that the licensee
had another opportunity to identify and
evaluate
the breakaway
torque spikes after the September
1994 failure to close
and subsequent
root cause of failure analysis.
The inspector
noted that
a review
of the as-found trace
on September
5 clearly shows the break
away torque spike
and the as-left trace after the packing replacement
does not have the breakaway
anomaly.
The inspector
noted that although the valve services
technicians
were analyzing the diagnostic traces,
they were not critically reviewing the
quality of the trace
and were not sensitive to changes
in the characteristics
of
the curves.
The inspector
concluded that if the licensee
had done
a qualitative
review of the
shape of the as-left curve to the as-found
curve they should
have
recognized
the breakaway
anomaly.
The inspector
had
a similar concern with the adequacy of diagnostic trace reviews
following the failure of another
RRS valve to close during
a differential
pressure
test
in March
1994
( Inspection
Reports
50-528/94-09,
Section
2. 1,
and
0
-22-
50-528/94-13,
Section S.l).
This particular failure was caused
by having the
wrong motor pinion gear ratio that caused
the va'lve to stroke too fast.
A
qualitative review of the as-left
and as-found traces clearly showed the
significant drop in stroke time.
The inspector
subsequently
concluded that the
licensee
did not have appropriate
acceptance
criteria because
they did not review
the relative
shape of the curves.
A noncited violation was issued
based
on the
licensee's
indication that
a qualitative assessment
of the shape of the
diagnostic trace would be included in the diagnostic test procedure.
The inspector
reviewed the current diagnostic test procedure
and noted that the
licensee still did not require
a qualitative review of the
shape
and quality of
the trace
and
a comparison of the as-left to the as-found trace.
The licensee
subsequently
determined that there
was
a
CRDR action to add the qualitative
.
assessment
to the diagnostic testing procedure,
but that it was incorrectly
closed without the action being completed.
7.2.5 Transportability of Breakaway
Torque Anomaly
The inspector
asked
the licensee
what were the most susceptible
valves to the
increased
packing loads
and if they had the
new low resistance
Argo 'packing
installed.
The licensee
determined that the most susceptible
valves
were the
valves with Limitorque SMC-04 actuators.
There are
55 of these
valves
in the
high pressure
safety injection,
LPSI,
CS and reactor coolant
systems
in all three
units.
. The licensee
had repacked all but eight of these
valves with the
new Argo
packing.
The licensee
could not find any documentation
that two of the eight
valves
had ever
been
repacked.
The licensee
evaluated
the traces for the eight
MOVs that did not have the
new
Argo packing
and noted that the spikes existed
in three of these
valves.
The
licensee
concluded that. if the value of the spike exceeded
50 percent of the
TST-.
value then the operability of the valve needed
to be evaluated.
The licensee
determined that the Unit 3 LPSI miniflow isolation Valve SIA-669 had
a spike of
about
55 percent of the
TST value.
The licensee
determined that the packing load
trended
down during maintenance
intervals
and there
was not
an operability
concern with Valve SIA-669.
The inspector
reviewed the licensee's
transportability evaluation
and agreed with
the licensee's
assessment
that there
was not
an
immediate operability concern
with any other
MOVs.
7.2.6
Corrective Actions
The licensee
planned to repack the eight valves that
had not been
packed with the
low resistance
Argo packing in the next available outage.
The licensee
also
agreed
to trend the breakaway
on the valves with the old packing
and
a
sample of the
pew Argo packing to evaluate future performance.
At the exit meeting,
the licensee
agreed to evaluate
what additional tools could
be provided to technicians
performing diagnostic trace
reviews to help them
0
0
e
-23-
identify anomalies
in the traces.
The inspector will review the licensee's
corrective actions
in the response
to the notice of violation.
8
FOLLOWUP ENGINEERING/TECHNICAL SUPPORT
(92903)
8.1
Unresolved
Item 528 9431-01
OPEN:
Letdown Isolation Valve Would Not
Close
In Inspection
Report
( IR) 50-528/94-31,
the inspector identified concerns
regarding
the licensee's
evaluation
and resolution of significant seat
leakage
past the Unit
1 letdown isolation Valve CHB-UV-515.
The valve had initially been
identified as leaking at approximately
40
gpm when closed
in December
1992
and
had not been repaired
when the inspector
observed
a deficiency tag
on the control
room switch associated
with Valve CHB-UV-515 in September
1994.
As noted in
IR 50-528/94-31,
the inspector identified that the licensee
had not evaluated'he
degraded
condition
on the system operability in December
1992 and,
when
an
evaluation
was performed in October '1994, failed to consider all
appropriate'esign
basis.
At the time of the exit meeting for IR 528/94-31,
the licensee
planned to perform
diagnostic testing of Valve CHB-UV-515 during the refueling outage
scheduled
to
begin in April 1995.
In addition,
the licensee
had initiated corrective actions
to address
weaknesses
in the operability evaluation
process.
8. 1. 1 Review of Outage
Maintenance
on Letdown Isolation Valves
Valve CHB-UV-515 is- the
B train air-to-open,
spring to close,
2" globe valve
located
on the reactor coolant
system letdown line to the chemical
and volume
control
system
(CH), located inside containment
upstream of the regenerative
heat
exchanger.
It receives
close signals
on high regenerative
heat
exchanger outlet
temperature
and
a SIAS.
Two similar valves,
CHA-UV-516 and CHB-UV-523, are
located
downstream
and provide inside
and outside containment isolation
respectively.
During the Unit
1 refueling outage,
the licensee
performed diagnostic testing
on
Valve CHB-UV-515.
The licensee
determined that the
"bench set" of the
actuator
was not providing adequate
seating
force to ensure that the valve would remain
closed against reactor coolant
system operating pressure.
The bench set
determines
the
amount of spring pressure
to close
and seat the valve.
Although
the
bench set for Valve CHB-UV-515 was below the values specified for the
actuator,
the licensee
determined that even set at the appropriate
values,
the
valve would have leaked if called
on to close against
a design basis differential
pressures
equivalent to the reactor coolant system relief valve settings.
The deficient seating
force was determined
to also apply to Valves
CHA-UV-516 and
CHB-UV-523.
The licensee
took action to evaluate
the deficiency in accordance
with the operability determination
process for the letdown isolation valves in
the operating units, identify the cause of the deficient design,
and perform
repairs
on the letdown isolation valves in Unit 1.
The inspector
reviewed the
and discussed
the repairs with the licensee.
l
0
J,
t
0
8. 1.2
Potential
Generic
Issue
-24-
The licensee
discussed
the finding regarding
low bench set with the vendor of the
actuator,
The vendor determined that the valve actuator
had not
been appropriately
sized for the application.
The vendor found that they had not
properly accounted for valve packing friction in the sizing of the actuator.
At
one point in their design process,
they stopped
using low friction teflon packing
material
and switched to higher friction graphite packing material.
However,'or
some period after the switch, they had not factored the higher friction into the
actuator sizings
The vendor determined that this error was applicable to all of
the letdown isolation valves in all three
Palo Verde units.
NRC Information Notice
( IN) 88-94, "Potentially Undersized
Valve Actuators,"
dated
December
2,
1988,
discussed
the failure of the Fisher Controls to consider
packing friction in val've actuators.
IN 88-94 concluded that valve actuators
shipped after January I, 1977,
had appropriately
accounted for valve packing
.
friction.
The licensee
stated that their initial review of IN 88-94. determined
that applica'ble valves
had all been
shipped after January I, 1977,
and in the
case of 'the letdown isolation valves,
had
been
shipped
in 1978
and
1979.
The discovery that the letdown isolation valve actuators
were not properly s>zed
indicates that the conclusions
in IN 88-94
may not have completely accounted for
all undersized
actuators.
At the
end of the inspection period,
the licensee
was
communicating with the vendor
and planned to perform
an audit.
Preliminarily, it
appeared
that valve actuators
ordered
before
January I, 1977,
and subsequently
shipped,
may have not been appropriately sized.
8. 1.3
Valve Actuator Repairs
The licensee
repaired
the letdown isolation valves
by providing
a stronger
actuator spring,
which allowed
a greater
bench set.
In communication with the
vendor,
they concluded that other valve components
did not need to be replaced.
Subsequently,
the licensee
determined that the actuator
handwheel
could not
support the stronger spring.
As
a result,
the licensee
performed
a safety
evaluation
and determined that the handwheel
was not necessary
for the valve
design functions.
The licensee
performed operability determinations
for the Unit 2 and
3 letdown
isolation valves.
They determined that it was necessary
for two valves to close
to isolate the letdown line against
design basis differential pressure.
They
reviewed the design basis
events for the letdown line valves
and determined that
two valves would be available in all cases
to provide letdown isolation.
The
inspector
reviewed the operability evaluation
and determined that the evaluation
appropriately
addressed
the design conditions identified.
The inspector identified weaknesses
in the operability evaluations
performed
prior to November
1994
and included this
as
an unresolved
item.
In their
t
~ evaluation
in April 199S,
the licensee
determined that the letdown isolation
valves inside containment
provided high energy line break protection
and that
this had not been previously reviewgd.
The licensee
planned to perform
a self-
f
0,
0
-25-
assessment
of the operability review process
as applied to the letdown isolation
valves.
The inspec'tor will review the assessment
in a future inspection report.
8.2
Closed
Unresolved
Item 50-528 94-26-04:
E ui ment
uglification of
Turbine Driven Auxiliar
Pum
Hain Steam
Su
1
8
ass
Valves
This unresolved
item involved an
Eg issue with the solenoid coil of the main
steam
suppl'y bypass
Valves
MSSBVs,
SG-134A and
138A.
The
HSSBVs are solenoid
operated
valves,
normally de-energized,
and are subjected
to high process fluid
temperatures
of 600 degrees
F,
In August
1994 the licensee
determined that the coils
had exceeded
the qualified
life of 20 years
since the actual field temperature
of the coil was about
150
degrees
F more than the temperature
assumed
in the qualification binder for the
coil (the calculated
Eg life was
based
on 204 degrees
F and the actual field
temperature
was
350 degrees
F).
Based
on
a coil temperature
of 350 degrees
F,
the qualified life was reduced to
18 days.
The inspector
was concerned that 'the
licensee
was collecting data for over
a year before they evaluated
the impact to
the plant.
The inspector
was also concerned
about the rigor of the
Eg
evaluations
since there
had
been
several
iterations of the
Eg life of these
solenoids.
~
~
~
8.2. 1
Licensee's
Evaluation
The licensee initiated
a
CRDR to address
the inspector's
concerns
and to
determine
the most accurate
Eg life of the
HSSBV solenoid coils.
The licensee's
evaluation explained the history of the
Eg issues
associated
with the
HSSBVs
and
the basis for the various values for the
Eg life of the valve coils.
The licensee
determined that the calculated
Eg life of 18 days
used the most
limiting activation energy for all the components
in the
HSSBV solenoid coil.
The licensee
contacted
the solenoid valve vendor
and determined that the most
critical component of the solenoid
was the polymide insulation
on the coil wires.
The enginoers
used the activation energy
and aging information for the polymide
insulation
and the service
temperature
of 350 degrees
F and calculated
an
Eg. life
of 9.3 years.
The licensee
subsequently
updated
the
Eg binder for the
HSSBVs
and
the
PM frequency in the
PH basis
database
to reflect the 9.3 year
Eg life.
The inspector
reviewed the licensee's
evaluation
and
had the following
observations:
The licensee
performed
a thorough evaluation
and
had
a good engineering
basis to support
an
Eg life of 9.3 years for the
HSSBV solenoid coil.
Eg engineering
collected temperature
data
on the
MSSBVs for almost
a year
that clearly
showed
in service coil temperatures
that were significantly
higher than the temperature
used in the qualification report
and did not
perform
a relatively simple calculation to determine
the qualified life
based
on these
temperatures.
l
I
l
J
'
-26-
~
The engineer
was collecting data
and did not have
an acceptance
criteria,
or upper threshold,
at which point an evaluation
would be conducted
to
determine if there
may be
a qualification issue.
The inspector
noted that the licensee
recently
implemented
an "attributes of
engineering
excellence"
program to address
the timely and complete resolution of
engineering
issues.
The inspector
concluded that application of these principles
by all levels of the engineering
organization
would help prevent
two year long
evaluations
similar to the
HSSBVs.
The inspector also noted that nuclear
assurance
engineering
(NAE) performed
an
audit in December
1994
and reviewed the adequacy of the licensee's
overall
thermal monitoring program.
8.2.2
Thermal Monitoring Program
The
NAE audit team noted that the licensee
committed to have
a thermal monitoring
program
and that
a formal program did not exist.
The audit team found that
a
1991 audit
had identified the
same
weakness
and the corrective actions
from th'at
audit were ineffective.
The audit team also identified that formal
EQ training
for engineering,
maintenance,
and operations
personnel
was significantly below
the industry average.
The audit team issued
CRDR 9-4-Q184 to the
EQ group to
develop
and
implement
a formal thermal monitoring program.
The inspector
reviewed
EQ engineering's
response
to
CRDR 9-4-Q184
and discussed
the proposed
implementation of the thermal monitoring program with the
Department
Leader.
. The inspector
noted that the
EQ group reviewed the various
zones
in the plant
and determined
what areas
were susceptible
to process fluid
heating
and then identified the
EQ components
in these
areas with qualified lives
less
than
15 years.
The licensee
determined that the majority of these
components
were in the main steam support structure
(HSSS)
around the main steam
and main feedwater lines.
The licensee
was in the process of determining the components
to be monitored in
these
areas
and then selecting
a temperature
recording device to install
on these
components.
The monitoring instruments
were scheduled
to be installed
by the
end
of 1995.
The inspector
conducted
a walkdown of the auxiliary building and the
HSSS to
determine
where the process fluid could possibly increase
the temperature
of
components.
The inspector
agreed with the licensee's
assessment
that the areas
of concern
were primarily in the
HSSS.
The inspector
concluded that the
licensee's initial evaluation to determine
the scope of the thermal monitoring
program
was appropriate.
The inspector
also took field temperature
readings of -the atmosphere
dump valve
(ADV), main steam isolation valve (MSIV), and the feedwater isolation valve
(FWIV) lower limit switches
and compared
them to the temperatures
used in the
qualified life calculation.
The inspector
noted that the qualified life of the
r
-27-
HSIVs and
ADVs was
based
on
a service
temperature
of 150 degrees
and the actual
temperature
in the field was about
150 degrees.
The inspector
noted that 'the
licensee
had
changed
the service temperature for the
MSIV and
ADV lower limit
switches
in the
Eg binder from 130 to
150 degrees'n
January
1994
based
on
measured field temperatures
of 150 degrees.
The licensee
subsequently
changed
the
Eg life of the switches
from 149 to 58 months.
The inspector
noted that field temperature
readings
on the
FWIV lower limit
switches
were reading
around
150 degrees
and that the qualified life was still
based
on
a service temperature
of 130 degrees.
The inspector
asked
the licensee
if they had
been monitoring the temperatures
of the
FWIVs limit switches
and
evaluated
the impact of the higher temperatures
on the life of the switches.
The licensee
informed the inspector that
an
Eg engineer
had
been monitoring the
temperature
of all the limit switches
in the
HSSS
once
a month for the last five
months
and
had also observed
some temperatures
as high
as
150 degrees
on the
lower FWIV limit switches.
The inspector
asked
the
Eg section leader
what field
temperature
would trigger an evaluation of the limit switch qualified life and
how long they planned to collect the temperature
data before
an initial
evaluation
was performed.
The
Eg section leader
informed the inspector that the
Eg engineer did not suspect
a problem with the existing temperatures
of the lower FWIV limit switches
because
the temperatures
were not significantly greater
than the
assumed
value.
Additionally, the
Eg engineer
was
aware of conservatism
in the activation energy
and post accident
environment
used
in the qualification of the limit switches
that would mitigate the small
increase
in service temperature.
The inspector
was
concerned
that the engineer
had not done
any calculations
or modeling of the
field conditions to substantiate
his engineering
judgement
used to arrive at this
conclusion.
The licensee
subsequently
performed
an assessment
of the temperature
data
and
determined that there
was not
a qualification concern if the average
FWIV limit
switch temperature
remained
less
than
140 degrees.
The engineer
had reco. ded
an
average
FWIV lower limit switch temperature
of 137 degrees.
This number
was then
adjusted
to account for a
2 month outage
time with a temperature
of 80 degrees
every
18 months which reduced
the av'erage
temperature
to 133 degrees.
The
inspector
reviewed the evaluation
and agreed with the licensee's
conclusion that
there
was not
an immediate qualification concern with the
FWIV lower limit
switches.
8.2.3
Conclusion
The inspector
concluded that the licensee
should
have
bounded
the temperatures
at
which
a more detailed
review of the qualified life of the components
in the
was required.
The inspector
noted that relying exclusively
on engineering
judgement
was not consistent
with the attribute of engineering
excellence that
requires
a "rigorous application of engineering principles".
The stated
value of
this attribute is "the complete
and demonstrated
resolution of an issue with
sufficient technical justification, graphs,
calculations
and associated
analysis
0'
-28-
techniques
so the issue is 'engineered'o
completion
and not just 'justified'o
a conclusion.
This approach
allows for engineering
judgement;
however, it
requires
an in-depth application of calculations,
cost benefit,
and risk
analysis."
The inspector discussed
this apparent
over-reliance
on engineering
judgement
during the collection of the temperature
data with the
Eg section leader
who
agreed with the inspector.
At the exit meeting,
the Director of System
Engineering
stated that they would establish
levels of temperature
differences
between
measured
in-service temperatures
and the temperature
used to qualify each
critical component that would trigger the performance of various levels of
evaluations prior to implementing the formal thermal monitoring program.
The
inspector
concluded that these
actions
were appropriate.
4
V
0
ATTACHMENT 1
1
Persons
Contacted
1. 1
Arizona Public Service
Com an
- J. Bailey, Vice President,
Nuclear Engineering
- S. Bauer,
Acting Department
Leader,
Nuclear Regulatory Affairs
- S. Coppock,
Engineering Supervisor,
Maintenance
Valve Services
- B. Eklund, Regulatory Consultant,
Nuclear Regulatory Affairs
- D. Garchow, Director, Site Engineering
- W. Ide, Director, Operations
- J. Levine, Vice-President,
Nuclear Production
R. Lucero,
Department
Leader,
Electrical Maintenance
- D. Hauldin, Director, Maintenance
- J. Hinnicks, Department
Leader,
Maintenance
Valve Services
M. Muhs, Section
Leader,
System Engineering
M. Radspinner,
Section
Leader,
Design Engineering.
F. Riedel
Department
Leader,
Operations
Unit 2
M. Salazar,
Section
Leader,
Maintenance
Valve Services
- C. Seaman,
Director, Nuclear Assurance
.
D. Smith,
Department
Leader,
Operations
Unit
1
B. Simpson,
Vice-President,
Nuclear Support
- W, Stewart,
Executive Vice President,
Nuclear
- R. Stroud,
Regulatory Consultant,
Nuclear Regulatory Affairs
J. Taylor, Department
Leader,
Operations
Unit 3
"P. Wiley, Department
Leader,
Operations
1.2
NRC Personnel
- K. Johnston,
Senior Resident
Inspector
- D, Garcia,
Resident
Inspector
J,
Kramer,
Resident
Inspector
- A. HacDougall,
Resident
Inspector
1.3
Others
- J. Draper, Site Representative,
Southern California Edison
- F. Gowers, Site Representative,
El
Paso Electric
- R. Henry,
Site Representative,
Salt River Project
- Denotes those present
at the exit interview meeting held
on May 19,
1995.
The inspector
also held discussions
with and observed
the actions of other
members of the licensee's
staff during the course of the inspection.
2
EXIT MEETING
An exit meeting
was conducted
on May 19,
1995.
During this meeting,
the
inspectors
summarized
the scope
and findings of the report.
The licensee
acknowledged
the inspection findings documented
in this report.
The licensee
did
not identify as proprietary
any information provided to, or reviewed
by, the
inspectors.
J
0
ATTACHMENT 2
1
LIST OF ACRONYMS
ADV
ATTN:
CFR
CRDL
CRDR
GL
IN
LER
HITR
MSSBV
HSSS
HSIV
NA
NAE
NRC
ppb
TAPA
TQS
TS
VSDS
atmospheric
dump valve
Arizona Public Service
attention
containment
spray
system
Code of Federal
Regulation
control
room deficiency log
condition report/disposition
request
Employee
Concerns
program
equipment qualification
feedwater isolation valve
Generic Letter
instrumentation
and controls
Information Notice
Licensee
Event Report
loss of coolant accident
low pressure
safety injection system
management
issues. tracking
and resolution
program
.
motor operated
valve
main steam supply bypass
valve
main steam support structure
not applicable
Nuclear Assurance
Engineering
Nuclear Regulatory
Commission
preventive maintenance
parts
per billion
recirculation actuation
signal
system
rotating rising stem
refueling water tank
.
safety injection actuation
signal
shift technical
advisor
temporarily approved
procedure
action
torque switch
trip'echnical
Specification
valve survey data
sheet
work order
C
0
0