ML17312A554
| ML17312A554 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 02/22/1996 |
| From: | Kirsch D NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV) |
| To: | |
| Shared Package | |
| ML17312A552 | List: |
| References | |
| 50-528-95-25, 50-529-95-25, 50-530-95-25, NUDOCS 9602260382 | |
| Download: ML17312A554 (62) | |
See also: IR 05000528/1995025
Text
ENCLOSURE
2
U.S.
NUCLEAR REGULATORY COMMISSION
REGION IV
Inspection
Report:
50-528/95-25
50-529/95-25
50-530/95-25
Licenses:
NPF-51
Licensee:
Arizona Public Service
Company
P.O.
Box 53999
Phoenix,
Facility Name:
Palo Verde Nuclear Generating
Station,
Units 1,
2,
and
3
Inspection At:
Haricopa County,
AZ
Inspection
Conducted:
December
17,
1995,
through January
27,
1996
Approved:
Inspectors:
K. Johnston,
Senior Resident
Inspector
D. Garcia,
Resident
Inspector
J.
Kramer,
Reside
spector
>rsc
, , ie
,
e
r
r
s
anc
Z ZZ- Q
ate
Ins ection
Summar
Areas
Ins ected
Units
1
2
and
3
Routine,
announced
inspection of onsite
review of an event,
operational
safety verification, maintenance
and
surveillance
observations,
onsite engineering,
plant support activities,
inspection
followup items,
and review of licensee
event reports
(LERs).
Results
Units
1
2
and
3
Plant
0 erations
Inadequate
communications
between
the operations,
maintenance
and
engineering
organizations
resulted
in the failure to perform
a required
(OD) evaluation for a degraded
condition of
the Train
B essential
chilled water system
(EC) chiller.
Unit 3
operators
had considered
the chiller operable for a week even
though it
was in
a degraded
condition which would have required
manual
actions to
assure
continued operability during
an event
(Section 4.2).
'7602260382
960223
ADOCK 05000528
8
l
i
~
Operations failed to critically assess
an incomplete operability
determination
evaluation
provided by the engineering
organization
regarding safety related solenoid valves.
Plant operations
approved
an
OD evaluation for several
safety-related
even
though the
engineering
evaluation did not thoroughly address
the effects of
potential
heat
damage
on the Technical, Specification functions of some
of the valves
(Section
7. 1).
~
Plant operations
implemented
an informal posttrip action which had
operators
begin boration before the action
was prescribed
by operating
procedures.
This guidance
was not formally reviewed to assure
consistency
with established
operating
procedure
requirements
(Section
3. I).
~
Plant operators
failed to follow the shutdown margin procedure
requirements
to continue boration until reactor coolant
system
concentration
was confirmed to meet
shutdown margin requirements.
This
was
a licensee identified noncited violation (Section 3. 1).
Operators failed to ensure that adequate
precautions
were taken prior to
attempting to return
a Unit 2 condensate
pump to service.
As
a result,
a loss of pump suction
caused
the trip of both main feedwater
pumps
and
a subsequent
reactor trip (Section
2. I).
A Unit 3 shift supervisor
demonstrated
a good questioning attitude prior
to
a moderator
temperature
coefficient
(MTC) test.
The indepth
questioning contributed to the identification of excessively
conservative
shutdown margin calculations
and
an inaccuratelj
drawn
insertion limit (TIL) curve (Section 6.1).
In addition,
the
shift supe> visor responded
promptly to concerns
regarding
a deficiency
with an
EC valve which contributed to the discovery of other system
control setpoint deficiencies
(Section 4.3).
Maintenance
Maintenance
technicians
demonstrated
strong
equipment
knowledge
and
proper maintenance
procedure
usage during the performance of mainten'ance
work (Section 5).
Maintenance
personnel
only corrected
an immediate deficiency in the
level
oF refrigerant in
a Unit 3
EC chiller without documenting
the
problem,
as prescribed
by the corrective action process.
This resulted
in
a missed opportunity to assess
the potential
impact
on system
performance
and operability (Section
4. 1).
Maintenance
did not thoroughly understand
the impact that
a chiller oil
leak could have
on the system operability
and the
need for an
Maintenance
personnel
failed to adequately
assure
that operations
personnel
were thoroughly aware that operation of
I
1
l
an
EC chiller with a significant oil leak would require
manual
actions
and of the implications of those actions
(Section
4.2).'n
ineerin
~
Haintenance,
system
and design engineers,
responsible
for the
EC system,
demonstrated
weak communications with each other
and did not ensure that
parameters
important to the operability of the chillers were
communicated
to maintenance
technicians
and the operations staff.
For
example:
~
Following a chiller trip in which refrigerant level
was
an
apparent contributor,
engineers
did not ensure that technicians
were fully cognizant of previous refrigerant level discrepancies
(Section
4. 1).
~
Haintenance
engineers
did not involve system'r
design
engineers
in the decision to consider
a chiller operable with a significant
oil leak (Section 4.2).
~
System
and design engineers
did not ensure
maintenance
and
operations
personnel
were aware that
a water leak rate of
approximately
2 drops per second
could impact the
EC system
operability (Section 4.3).
~
Nuclear Assurance
performed
a thorough
assessment
of a Unit I
EC chiller
trip and identified that the evaluation of the trip by maintenance
and
system engineering
had not been thorough.
In addition,
Nuclear
Assurance
identified that engineering
had
been
slow to resolve design
issues
regarding
the impact of cold spray
pond temperatures
on the
operability of the chillers (Section 4. 1).
~
An engineering
team,
established
to review emergency diesel
generator
cooldown trips,
completed
an excellent investigation.
They established
that several
deficiencies contributed to the problems, initiated
appropriate corrective actions,
and developed
a presentation
for the
diesel
owners
group (Section 7.2).
Plant
Su
ort
~
Contractor personnel
displayed
poor housekeeping
practices
during the
platform construction
in the mainsteam
support structure.
Hanagement
aggressively
assured
that the contractor
understood
and
implemented
their expectations,
which resulted
in improved conditions in the area
(Section 8).
f
'
~Summar;e
Ins ection Findin s:
~0en
Items
One violation was identified (530/9525-01)
involving the failure to
identify a condition adverse
to quality (Section
F 1).
One violation was identified (530/9525-02)
involving failure to follow
procedure
(Section 4.2).
~
One violation was identified (528/9525-03)
involving the failure to
perform leak rate testing of the
EC system
(Section 4.3).
~
One noncited violation was identified involving failure to follow the
shut
down margin
(SDN) procedure
(Section
3. 1).
~
Inspection
Followup Item 530/9525-04
(Section 6.1)
I
Closed
Items
~
Unresolved
Item 528/9521-01
(Section
4)
~
Violation 529/9431-02
(Section 9.1)
~
Violation 529/9431-05
(Section 9.2)
~
Violation 530/9438-01
(Section 9.3)
~
LERs 528/94-05,
Revision 2; 528/94-07,
Revision
1; 528/94-09,
Revision
1; 528/94-10,
Revision
1; 529/94-04,
Revision 2; 529/94-08,
Revision
1 (Section
10)
Attachment:
1.
Persons
Contacted
and Exit Heeting
2.
List of Acronyms
DETAILS
1
PLANT STATUS
1.1
Unit
1
Unit
1 began
the inspection period at
100 percent
power.
On December
29,
1995,
the unit reduced
power to 40 percent for repairs to the condenser
hotwell.
On December
31, the unit returned
to 100 percent
power and remained
there throughout the inspection period.
1.2
Unit 2
Unit 2 began
the inspection
period at
100 percent
power.
On January
21,
1996,
a reactor trip occurred following an attempt to return
a condensate
pump to
service
(see Section
2. 1).
On January
23, the unit was returne'd to
100 percent
power operation
and operated
at this power for the remainder of
the inspection period.
1.3
Unit 3
Unit 3 operated
at full power for the duration of the inspection period.
2
ONSITE RESPONSE
TO EVENTS
(93702)
2. 1
Loss of Feedwater
and Reactor Tri
Unit 2
On January
21,
1996, Unit 2 tripped from 100 percent
power due to low steam
generator level.
Prior to the trip, operators
were attempting to place the
Condensate
Pump
C, which had
been out of service for several
weeks,
in
service.
In the process of aligning the
pump, air was introduced into the
suction of the operating
condensate
pumps
as
a result of a failure to have
explicit procedural
steps
to require filling and venting of the condensate
line,
The transient
caused
a lowered suction pressure
to the two running
condensate
pumps
and
a loss of suction to both of the main feedwater
pumps.
The main feedwater
pumps tripped causing
a loss of feedwater
and
a subsequent
The steam generator levels continued to decrease
until
an
automatic signal
started
both auxiliary feedwater
pumps.
Posttrip plant
recovery
was normal
and the licensee classified
the event
as
an uncomplicated
The inspector
responded
to the unit trip and noted that the operators
were
responding
well to the event
and all required safety
equipment
was operating
as designed.
Operators
properly controlled primary plant parameters,
recovered
inventory,
and secured
the unloaded
operating
emergency diesel
generators
in
a timely manner.
The inspector
concluded that
the crew's
use of procedures
was strong.
I
ff
,f
l,
The inspector
noted that the licensee
investigation
olanned to evaluate
further corrective actions
including proceduralizing
the filling and venting
of the condensate
pump
and piping during plant operations.
The inspector will
evaluate
the corrective actions during
LER closure.
The inspector
observed
the reactor startup
on January
22 and noted
good
communications
and procedure
usage.
The inspector
concluded that the
operating
crew demonstrated
an overall strong performance.
3
OPERATIONAL SAFETY VERIFICATION
(71707)
3.1
SDM and Boration
On March 3,
1995,
the licensee identified
a condition where the
1 percent
SDM required in Technical Specification
3. 1. 1. 1 did not provide adequate
SDH for the limiting license basis
steam line break event in Mode
3
As
a result,
the licensee
established
administrative
controls to apply the requirement of Technical Specification 3. 1. 1.2 for
Technical Specification 3. 1. 1. l.
Technical Specification 3. 1. 1.2 required
between
a
4 and 6.5 percent
SDM, dependent
on the reactor coolant
system
temperature,
and
was analyzed for the limiting license
basis
steam line break
event.
At the
end of the inspection period,
the licensee
was preparing to
submit
a licensee
amendment
to address
the adequacy of the requirements
of
The shift technical
advisors
developed
a reactivity control worksheet to
provide information to the operators
on reactivity worth and minimum boron
concentration
to satisfy the 6.5 percent
SDM requirement.
The inspector
noted
that several
crews
used
the information as part of the crew brief at the
beginning of the shift to addrc"s the
SDM requirements
should
a unit shut
down
occur.
As
a result,
the worksheet often indicated that the posttrip
SDH was
inadequate.
The amount of boric acid addition indicated
by the worksheet
was
more than what was actually required
because
the assumptions
associated
with
the worksheet
were overly conservative.
As
a result,
the worksheet often
indicated that the posttrip
SDH was inadequate.
The licensee
had issued
a night order
on December
7,
1995, discussing posttrip
SDH.
The night order explained that following a reactor trip, the operators
would need to borate to maintain
adequate
SDH.
The night order further
explained that the operators
should review the reactivity control worksheet
on
a shiftly basis
in order to have
an idea of how much borated water must
be
added following a unit trip.
3.1.1
Reactor Trip Event
Unit 2
On January
21,
1996, shortly following the unit
trip, the operators
injected
800 gallons of borated water consistent
with the
reactivity control worksheet
information.
The licensee
subsequently
obtained
sample
and performed
Procedure
"Shutdown
Hargin," and determined
that the boron concentration
was
16 parts
per million
f
less
than the minimum required
boron concentration
for adequate
SDM.
The
operators
injected
an additional
400 gallons of borated water to compensate
for this deficiency
and stopped
the boration.
The shift technical
advisor,
during review of the
SDM calculation, identified
that Procedure
72ST-9RX09 required continuous
boration until the
SOM was
satisfied.
The shift technical
advisor informed operations
management
present
in the control
room of the continuous
boration requirement.
Operations
management
directed the operators
to re-establish
boration until the
SOM was
verified with a reactor coolant
sample.
The operators
injected
an additional
200 gallons of borated water before the shift technical
advisor verified the
SDM acceptance
criteria were met.
3. 1.2
Licensee
Evaluation
During the posttrip evaluation,
the licensee
concluded that operators
had
a
flawed understanding
that
SDM requirements
would be met by adding the amount
of boron indicated
on the reactivity control worksheet.
The inspector
agreed
with the licensee's
conclusion.
On January
22, the licensee
issued
a night order
and held crew briefings for
the operators
to clarify management's
expectation
regarding reactivity
control.
The briefing indicated that the reactivity control worksheet
provided useful
and accurate
information but would not
be used
as direction to
start borating in order to meet
SDM requirements.
During posttrip conditions,
the emergency
operating
rocedures
would provide direction to operate
the
plant, including the verification of SOM.
In addition,
the briefing indicated
that, if a boration
was started
to meet
SDM requirements,
the boration would
continue until
SDM requirements
were verified by chemistry sample.
The inspector
noted that the licensee
issued
a revision to
Procedure
on January
26, to clarify the contingency actions to
maintain
a continuous
boration until
SDM was satisfied.
In addition,
the
licensee
performed calculations,
which concluded for all cases
of the current
unit cycles that
a
5 percent
SDM would be established
and that the
5 percent
SOM would be suFficient to compensate
for the reactivity addition of the
limiting steam line break.
The inspector
noted that the licensee
planned to
further evaluate
SDM corrective actions
as part of the trip investigation.
The inspector
noted that the'uidance
to operators
to borate
immediately
following a reactor trip had not been
included in the emergency
operating
procedures
and
had not received
a formal review.
The inspector
concluded that
the use of this informal guidance
was not consistent with the philosophy of
the emergency
operating
procedures.
The inspector
discussed
the informal
guidance
concern with pl,ant management.
Plant
management
agreed with the
inspector's
concern
and indicated that they would evaluate to ensure that
a
guidance
developed for operators
to mitigate plant deFiciencies
is formally
reviewed.
The inspector
concluded that the licensee's
corrective actions
were
appropriate.
I(
3.1.3
Conclusions
The inspector
concluded that the licensee's
corrective actions to address
the
boration
issues
were appropriate.
Although the operating
crew failed to
follow the
SDH Procedure
72ST-9RX09 for boration requirements,
this licensee
identified and corrected violation is being treated
as
a noncited violation,
consistent
with Section VII of the
NRC Enforcement
Policy.
4
ESSENTIAL CHILLED WATER (EC)
SYSTEM
(37751,
62703,
71707,
92720,
92902)
In
NRC Inspection
Report 50-528/95-21;
50-529/95-21;
50-530/95-21,
the
inspector
opened
Unresolved
Item 528/9521-01
to perform further review of the
problems
experienced
in both the Units
1 and
3
EC systems
on November
27,
1995.
The Unit
1 Train
B chiller had tripped unexpectedly
and the Unit 3
chiller was declared
due to oil pump cavitation
and then declared
operable after the addition of more oil.
During the inspection period,
the
inspector identified concerns
regarding
EC system sensitivity to water
inventory loss.
In addition,
the inspector considered
Unresolved
Item 528/9521-01
closed.
4.1
Refri erant Levels
The licensee initiated Condition Report/Disposition
Report
(CRDR) 1-5-0215 in
response
to the Unit
1 Train
B chiller trip that occurred
on November 27.
Prior to completing
a root cause
evaluation for the trip, the licensee
suspected
that low refrigerant level in the cooler
may have
been
a
contributing factor.
The licensee identified that the Unit
1 Train
B chiller
refrigerant level
was low in its band.
I
In 1990,
HVAC engineering
had "stablished that refrigerant levels in the
cooler should
be maintained
between
3 and
7 inches in accordance
with
Engineering
Evaluation Report
(EER) 90-EC-031.
The licensee
had translated
these
values into a weekly preventive maintenance
task.
The high and low
levels
had
been defined
as the maximum and minimum operability limits.
The
inspector
noted that, following the Unit
1 Train
B chiller trip,
technicians
increased
the
shutdown refrigerant levels in all three units
as
necessary
to slightly greater
than
6 inches to prevent
a trip on "low
refrigerant temperature."
4. 1. 1
Control of Refrigerant
Levels
On December
20,
1995, during
a system walkdown to verify the licensee's
corrective actions,
the inspector identified that the refrigerant level in the
Unit 3 Train
B chiller was
above
7 inches.
The inspector notified the shift
supervisor,
who in turn contacted
a
HVAC technician.
The
HVAC technician
confirmed that the refrigerant level
was
7 3/8 inches,
lowered the level to
6 1/4 inches,
and documented
this activity in a weekly preventive
maintenance
work order.
The shift supervisor
concluded that the chiller had
been operable with the
reFrigerant level
ab'ove
7 inches
based
on
a discussion
with HVAC maintenance
personnel.
On January
4,
1996,
the inspector determined that
a
CRDR had not been
initiated to document
and resolve the refrigerant level discrepancy.
The
inspector obtained
the completed weekly chiller preventive maintenance
tasks
for November
and December.
The inspector
noted that
a week prior to
December
20,
1995,
the refrigerant level
was recorded
at
4 3/4 inches.
The
week Following December
20, the refrigerant level
was recorded
at
3 3/8 inches.
The inspector determined
that these
changes
in level represent
up to
25 percent of the
1 ton refrigerant
volume.
The inspector concluded that the high refrigerant level
and the abnormal
trend
represented
a condition adverse
to quality which, according to the licensee's
corrective action program,
required that
a
CRDR be initiated,
The inspector
concluded that the failure of the maintenance
technicians
to initiate a
CRDR
demonstrated
that they were not fully cognizant of refrigerant level issues,
despite
the recent trip of the Unit
1 Train
B chiller.
The inspector further
concluded that maintenance
engineering,
who had responsibility for the chiller
preventive maintenance
program
and
had
been
involved in the review of the
chiller trip, had not ensured
that the maintenance
technicians
were
sufficiently aware of previous refrigerant level discrepancies.
The failure
to identify a condition adverse
to quality is
a violation of 10 CFR Part 50,
Appendix B, Criterion XVI. (530/9525-01)
On February
1,
1996,
the licensee initiated
CRDR 9-6-0078 to address
this
issue.
HVAC 'technicians
suspected
that the level
changes
may be due to
a
leaking isolation valve between
the refrigerant storage
tanks
and the cooler
and scheduled
a work order to investigate.
4.1.2
Assessment
of the Chiller Trip Investigation
The inspector reviewed the progress
of CRDR 1-5-0215
and noted
tha~ the pace
of the investigation
had not been aggressive.
The inspector
noted that the
classification
oF the
CRDR had
been
changed
from "potentially significant" to
"adverse."
The root cause of the Unit
1 chiller trip had not been determined.
As of January
16, the critical setpoints
on the chiller had not been verified
and
assessed
by the
system engineer.
The inspector
noted that Nuclear Assurance
had
been performing
a concurrent
evaluation
and
had arrived at
a similar conclusion.
Nuclear Assurance
had
determined
that the
CRDR should
have
been classified
as "significant" based
on
the potential applicability of this problem to all essential
chillers.
On
January
19, Nuclear
Assurance initiated
CRDR 9-6-(019.
In addition,
Nuclear
Assurance
had identified that
"Concerns
Regarding
Essential
Chiller Reliability During Periods of Low Cooling Water
Temperature,"
concerning similar issues,
had not received
a timely or thorough
review.
l
-10-
The inspector questioned
the system engineer
about the effects of operating
essential
cooling water without operating
the chiller.
The system engineer
stated that, if the
EC temperature
is lowered due to cool weather
and it is
operated without the chiller operating,
the refrigerant
may migrate into the
condenser.
This would decrease
refrigerant level in the cooler,
thus
increasing
the risk of a trip on "low refrigerant temperature"
when the
chiller is called
upon to start.
The inspector
noted that
a modification had
been
proposed
to address
this concern.
However,
the licensee
had not
evaluated
the existing configuration nor implemented
interim actions.
The inspector
noted that Nuclear Assurance
had developed similar concerns
documented
in CRDR 9-6-f019.
Nuclear Assurance
referenced
six separate
EC chiller events
since
1994 which could
be attributed to control of
refrigerant levels.
Nuclear Assurance
had identified that modifications to
install
a three-way mixing valve to control refrigerant level
had
been
canceled
twice, once in 1988
and once in 1993.
Nuclear Assurance
also
identified that engineering
had not adequately
resolved
whether the chillers
could reliably operate with spray
pond temperatures
less
than 49'F.
In response
to these
issues,
HYAC maintenance
revised
the weekly chiller
preventive
maintenance
task to include seasonal
operating refrigerant levels
and the need to write
a
CRDR if the levels are not within its desired
range.
In addition,
the licensee
planned to enter the results of the weekly
preventive maintenance
into a data
base for trending purposes.
System
engineering
would have the responsibility for long term trends while
maintenance
engineering
would have the short term trending responsibilities.
The licensee
planned to implement this action in March of 1996:
4.2
Lubricatin
Oil Levels
On November 27,
1995, Unit 3 operators
declared
the Unit 3 Train A chiller
inoperable after
an auxiliary operator
noted that there
was
no observable oil
level in its oil reservoir.
Operators
subsequently
declared
the chiller
operable after adding
5 gallons of oil and performing the chiller surveillance
test.
The inspector
reviewed the licensee's
basis for operability
and
concluded that the procedure for operability determinations
had not been
implemented
and found that the basis for operability had not been well
established.
4 '.1
Background
The chiller compressor
and motor are refrigerant cooled
and oil lubricated.
A
motor-driven,
compressor
lubricating oil pump
and reservoir
are located in the
compressor
base.
The oil pump takes
suction
on the reservoir
and discharges
through
a filter and heat
exchanger
to the compressor
and motor.
The
lubricating oil is returned to the reservoir
by means of an oil return system.
There
are
two protective trips associated
with the lubricating oil system,
"Compressor
High Bearing Oil Temperature"
and
"Compressor
Low Oil Pressure."
I
There are
two sightglasses,
an upper
and lower bullseye,
that represent
the
maximum and minimum oil levels.
If the oil in the
eservo
~ 'ceeds
the
maximum level, the chiller can trip on high bearing oil temperature.
Additionally, the chiller can trip on low oil pressure
because
of low oil
levels
caused
by the low differential pressure
across
the compressor.
Since the chillers are closed
systems,
oil leaking from the seals
enters
the
refrigerant cycle.
The chillers were designed
to operate with some
amount of
oil in the refrigerant
and
a small oil return system
was installed to return
oil from the refrigerant.
However,
the oil return system
does not work
efficiently when the chiller is operated
under low loaded conditions.
As
a result of a chiller trip in 1987, engineering
performed
an evaluation,
documented
in EER 87-EC-019.
The licensee
had established
that the chiller
should not be operated
with the oil level
below the lower sightglass,
which
corresponds
to approximately 7.5 gallons of oil, or with a total oil volume in
excess of 25 gallons.
4.2.2
Timeline of Unit 3 Operation of the Train A Chiller
On October
29,
1995,
the Unit 3 Train A chiller was returned to service
following the replacement
of the rear motor bearing seal.
The
HVAC technician
initially charged
the chiller with 15 gallons of oil.
During the period of
November
10 through
November 20,
HVAC technicians
added
approximately
7 gallons of oil to the chiller in order to maintain acceptable oil levels in
the oil reservoir sightglass.
The chiller had
been
in service to support the
operation of safety-related
equipment during the Unit 3 refueling outage.
On November 26, operations
personnel
started
the Train A chiller in support of
testing Train
pump, which had
been out of service for
approximately
60 hours6.944444e-4 days <br />0.0167 hours <br />9.920635e-5 weeks <br />2.283e-5 months <br /> of a 72 hour8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> Technical Specification action statement.
Shortly after the chiller was started,
operators
observed
low oil levels in
the oil reservoir.
HVAC technicians
added
5 gallons of oil, bringing the
total calculated oil volume to 27 gallons, after discussing
the addition with
an
HVAC maintenance
engineer.
The
HVAC maintenance
engineer
suspected
that
an
oil leak had developed
on the rear motor bearing seal.
On November 27, at approximately 4: ll a.m., Unit 3 operations
personnel
declared
Train A chiller inoperable after
an auxiliary operator
heard
the
lubricating oil pump cavitating
and noted that the oil level
was not visible
in the reservoir.
The site shift manager,
shift supervisor,
and the
HVAC maintenance
team leader
discussed
the operability of the Train A chiller.
The licensee
added
5 additional gallons of oil to the chiller and performed
a chiller
surveillance test.
Maintenance
logs identified that this addition brought the
total oil in the chiller to approximately
32 gallons,
7 gallons
above the
limit established
by engineering
in
EER 87-EC-019.
Both the Train A chiller
pump were declared
at 6:55 a.m.
The chiller
was subsequently
shutdown,
placed
in standby,
and remained
On
~ I
-12-
Novemb r 28, the
HVAC maintenance
engineer initiated
CRDR 9-5-1185
and
developed
an action
pl-an to determine
why the chiller was losing oil from the
reservoir.
On December
4, operations
removed
the Train A chiller from service
and
technicians
replaced
the rear motor seal o-ring as part of their corrective
maintenance.
The
HVAC technicians
determined that the o-ring had
been
damaged
during the previous maintenance
outage.
The oil recovered
from the chiller
was approximately
37 gallons,
5 gallons
more than
had
been
recorded
in
maintenance
logs.
Following the maintenance activity, operations
personnel
tested
the chiller for 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />.
After the test
showed
no apparent oil leaks,
operations
personnel
declared
the chiller operable.
4.2.3
Assessment
of Interim Actions
On January
18,
1996,
the inspector
met with the
HVAC maintenance
team,
including the
HVAC maintenance
engineer,
to discuss their basis for
considering
the chiller operable
on November 27,
1995.
The
HVAC maintenance
engineer
had recognized that
32 gallons
exceeded
the established criteria and
that the majority of the oil was in the refrigerant,
The maintenance
engineer
stated
that the primary concern
was that, if the chiller operated
in
a full
load condition,
the oil return system would flood the oil reservoir.
He noted
that, without manual
action to remove the recovered oil, there would be
an
increased risk of a chiller trip on compressor
high bearing oil temperature.
The
HVAC maintenance
team stated that they had provided
21 hours2.430556e-4 days <br />0.00583 hours <br />3.472222e-5 weeks <br />7.9905e-6 months <br /> of onsite
coverage
from November
27 through
December
1, to be available if the Unit 3
Train A EC system
was called
upon, to prevent the chiller from tripping and
maintain chiller operability.
For the remainder of the time from December
2
through 4, the technicians
were available
by telephone
and
a team leader
was
also available
on backshift.
The inspector
subsequently
reviewed the unit and shift technical
advisor logs
and observed that the log entries did not clearly establish
that continued
operability of the chiller depended
on manual
action
by
a
HVAC maintenance
technician.
In addition,
the inspector determined that the Unit 3 operations
personnel
had not recognized
the fact that manual
action
was required
by the
HVAC technician to maintain operability.
Licensee
Procedure
provided guidelines
and instructions for
evaluating
the operability of a system
when
a degraded
or nonconforming
condition has
been identified.
Appendix C, to the procedure
provided several
actions
which must
be met before taking credit for manual action.
These
actions
were consistent with the guidance
in
concerning operability and included
a
10 CFR 50.59 evaluation, written
instructions that prescribe
the manual
actions,
and ensuring that necessary
communications
are established.
Operations
did not implement the operability
determination
procedure,
nor were
any of the actions called out in Appendix
C
documented prior to declaring the Train A chiller operable.
The failure to
f
-13-
follow the prescribed operability evaluation
procedure
is
a violation.
(530/9525-02)
4.2.4
Assessment
of Technical
Basis
The inspector
inquired whether the maintenance
engineer
had evaluated
whether
the oil loss to the refrigerant could exceed
the oil returned to the
reservoir.
The maintenance
engineer
found that this was possible if the
chiller was operated
in
a low-load condition.
He stated that compensatory
actions
could have
been
taken to artificially load the chiller, increasing
the
rate of oil return.
The inspector
noted that these
compensatory
actions
were
not prescribed
in existing instructions.
The inspector further inquired whether the maintenance
engineer
had evaluated
the impact of the additional oil in the refrigerant
on the efficiency of the
chiller.
The maintenance
engineer
stated that, while he was confident that
the chiller efficiency would not have
been significantly impacted,
he did not
have specific basis for this conclusion,
The inspector
noted that this
confidence
may have
been
acceptable
for establishing
a reasonable
assurance
for operability on November
27.
However,
the inspector considered
that
a more
formal evaluation
was warranted to support continued operability.
The
inspector
also inquired whether
a trend of oil addition
had
been
made to
assess
whether the oil leak was degrading.
The maintenance
engineer
had not
evaluated
the trend.
The inspector determined
in discussions
with the
licensee
that the maintenance
engineer
had developed
the basis for chiller
operability without substantive
support
from either the system or design
engineers.
finally, the inspector questioned
whether the licensee
had developed
a
reasonable
basis for considering
the chiller operable
on November
27.
The
licensee
concurred that the operability determination
process
had not been
implemented
and the arguments
to support operability had not been well
developed.
On February
2,
1996,
the licensee
presented
a well developed
basis
for considering
the chiller operable
from November
27 through
December
4,
1995.
The inspector critically reviewed the licensee's
determination
and
found the basis
acceptable.
4.3
fC
S stem
Leaka
e
On January
19,
1996,
the inspector identified
an
EC chilled water system leak
which was in excess
of the maximum system leak rate identified in the system
design basis
manual
(DBH).
The inspector
subsequently
noted that past
system
configuration changes
had
made the system's operability sensitive
to
a loss
oF
inventory
as
small
as
15 gallons in the
1100 gallon system.
However,
the
licensee
had not initiated adequate
compensatory
actions to ensure that
operations
and maintenance
personnel
were
aware of this sensitivity.
In
addition,
the inspector identified that the licensee
had missed
subsequent
opportunities
to identify that the compensatory
actions
were not adequate
during their
DBN project
and in
a followup of a
1994 inspection
issue.
I
-14-
4.3.1
Original
EC System
Design
Two trains of
EC circulate through their respective refrigeration (chiller)
units,
through safety-related
equipment
room coolers,
and back through
an
EC
pump.
The
EC system
pumps, chiller units,
and chilled water expansion
tank
are located in the lowest level of the control buildings with room coolers
located at higher elevations.
The
EC system
was designed
as
a closed
loop
system
and its expansion
tank was provided with a pressurized
nitrogen cover.
Makeup to the expansion
tank was provided
by demineralized
water
and from the
condensate
storage
tank through the condensate
transfer
pumps.
The condensate
transfer
pumps,
normally in standby,
were designed
to start
on the
same
initiation signals
as the
EC system.
A solenoid-operated
valve, actuated
by
expansion
tank level instrumentation,
was designed
to open
on low expansion
tank level,
and close
on high and low-low expansion
tank level.
Nonsafety-related
instrumentation
was provided for local expansion
tank level
and pressure
and control
room annunciation of high and low expansion
tank
levels
and pressures.
The licensee's
Final Safety Analysis Report stated,
that
.
.
. "critical
conditions of the tank level
and pressure
are
alarmed
in the control
room for
leak detection."
The NRC's safety evaluation report stated that the
EC system
had met
10 CFR Part 50, Appendix A, General
Design Criteria
(GDC) 44,
45,
and
46 concerning
the design,
inspection,
and testing of cooling water
systems.
GDC 44 states
that "Suitable
.
.
. leakage
detection
.
.
. shall
be provided to assure that
.
.
. the system safety function can
be
accomplished
.
. ."
GDC 46 states
that
"The cooling water systems
shall
be
designed
to permit appropriate
periodic testing to assure
.
.
. the leaktight
integrity of its components
.
4.3.2
System Configuration
Chages
Two
EC system configuration changes
occurred
between
1990
and the time of the
inspection.
In 1990,
the licensee
removed the
EC expansion
tank
instrumentation,
which provided control
room annunciation of critical level
and pressure
parameters,
from service.
In addition,
in 1992, the licensee
concluded that it would no longer take credit for a safety-related
EC expansion
tank makeup
from the condensate
storage
tank.
The action to remove the control
room annunciation
from service
had
been
prompted
by licensee
concerns with the code boundary
excess
flow check valves,
which separated
the seismically qualified instrumentation
from nonseismically
qualified instrumentation.
As documented
in
EER 86-XH-046, the licensee
determined
that the excess
flow check valves could leak following a seismic
event with an adverse
impact
on the
EC system.
To resolve this concern,
the
licensee
closed valves
upstream 'of the excess
flow check valves.
As
a
compensatory
measure,
the licensee
established
an operator
round to read
levels
by opening the isolation valves every shift.
The licensee
stopped
taking credit for
EC system
makeup
from the condensate
storage
tank in
a design basis
event
when they recognized that the operability
ilr
-15-
of the
EC system would be dependent
upon the operabi>ity of the condensate
transfer
system.
The inspector determined that the licensee's
review of these configuration
changes
was not thorough.
The
EC system
has
a relatively small
expansion
tank,
making the system sensitive to small
volume changes.
With a relatively
small
system leak
and the loss of the nonsafety-related
makeup to the
expansion
tank,
the expansion
tank could drain
and introduce nitrogen into the
system at the suction of the
pump.
The inspector determined that both the
isolation of expansion
tank level instrumentation
and the changes
to the
classification of the condensate
storage
tank makeup
reduced
the ability of
operators
to detect
a leak
and reduced
the reliability of the system
makeup.
The inspector determined that the licensee's
10 CFR 50.59 evaluation,
performed in 1990 to address
removing the expansion
tank level
and pressure
annunciation
from service,
did not discuss
the
GDC 44 requirements
for cooling
system
leakage detection
and did not account for the
GDC 46 requirement for
leak rate testing.
At that time, the licensee
did not establish
a maximum
leak rate for the
EC system
and
had not established
a program for
EC system
leak rate testing.
The inspector determined that the compensatory
measures
to include
EC
expansion
tank level
and pressure
readings
on operator
rounds
were
appropriate.
However,
the inspector
noted that the operator
rounds
procedure
did not address
a need to monitor the system for leaks.
In addition,
maintenance
procedures '; re not implemented
which assured
that leakage
was
minimized.
While the safety evaluation
performed in 1990 was flawed,
the inspector
concluded that,
since
1992,
the licensee
has
improved the quality of
10 CFR 50.59 evaluations
to address
these
types of weaknesses.
However,
as
discussed
below, the licensee
subsequently
missed additional opportunities to
identify the weaknesses
with the
EC system configuration changes.
4.3.3
Licensee's
DBM and Associated
Calculations
In May 1994,
the licensee
issued
Revision
1 to the
EC system
DBM.
The
DBM identified that
GDC 44 had
been applied to the
EC system to establish
the system leak rate.
The
DBM provided
a system specific leakage
criteria of 9 gallons per day or 0 '75 gph
and referenced
EC hydraulic
Calculation
13-MC-EC-200 for the basis of this leak rate.
The
DBM also
identified that the
EC system
should
be tested
to assure
the overall
system
leakage did not exceed
design values calculated
in Calculation
13-MC-EC-200.
The licensee
did not initiate any immediate actions
to ensure that
a leak rate
of 0.375
gph would be detected
by operators
or
HVAC maintenance
personnel.
In
addition,
the action to establish
a system leak performance test
was captured
in one of the final phases
of the design basis reconstitution
program to be
implemented
by mid 1996.
l
I
l
-16-
The inspector determined that
a leak rate of 9 gallons per day represented
a flow of approximately
2 drops per second.
The inspector considered
that
a
leak rate this small would not be identified as
an operability concern
by
plant operators
and maintenance
personnel
unless specifically brought to their
attention.
The inspector determined that licensee
engineering
had not
identified this
as
a concern.
The inspector
found that,
in general,
the licensee's
plan to implement testing
in accordance
with a schedule for implementation
in mid 1996 appeared
acceptable.
However,
the inspector
was concerned that the plan did not
prioritize the implementation of tests
required to meet
a regulatory
requirement.
The inspector reviewed the portion of Calculation
13-MC-EC-200 pertaining to
the maximum allowed
EC water leak rate.
The inspector
noted that the basis
for the
maximum leakage
rate for the
EC system
was to maintain
adequate
expansion
tank pressure
for a period of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />
assuming
no operator action
and the loss of nitrogen
makeup.
The inspector identified the following
two errors:
The calculation
assumed
that level in the
EC expansion
tank was at the
low-level makeup valve actuation setpoint
versus
the high level
makeup
valve-close setpoint.
Since the limiting condition of the calculation
was the lowest allowed nitrogen cover pressure,
using the low level
setpoint
provided
a nonconservatively
high starting
volume of nitrogen.
Assuming
a higher setpoint,
the calculation would have allowed
a leak
rate of only 0.25 gph.
The calculation using the low-level setpoint credited
expansion
tank
volume below the surge
1-:ne to the
EC pump suction.
Subtracting
the
unusable
volume would also
have provided
a leak rate of approximately
0.25 gph.
The licensee
revised their calculation
on January
24,
1996.
The revised
calculation identified that
a loss of volume of 15.2 gallons,
versus
the
originally calculated
27 gallons,
could impact system operability.
However,
the licensee
took credit for identifying a low-level condition within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />
oF
an event versus
the
72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />
assumed
in Calculation
13-MC-EC-200.
This
change
increased
the maximum leak rate to 0.63 gph.
The inspector
found this
change
acceptable
based
on the operator
rounds
performed every 12-hour shift.
Following the inspection period,
the licensee
developed
and started to
implement
an
EC leak rate test.
The licensee
also initiated
a review of other
08Ms to identify if there
were similar tests
which required priority
implementation.
j
I0
-17-
4.3.4
Concerns
Identified by the Inspector in March
1994
In March 1994,
the inspector
performed
a walkdown of the
EC system
(see
NRC
Inspection
Report 50-528/94-09;
50-529/94-09;
50-530/94-09)
and observed that
the
EC expansion
tank instrumentation
had
been
removed
from service.
As
documented
in
NRC Inspection
Report 94-09,
the inspector
noted that there
was
no mention in the annunciator
response
procedures
that this instrumentation
was out of service.
The inspector also noted that this change
had not been
reflected in the Final Safety Analysis Report
(FSAR).
In response
to the concerns
discussed
in
NRC Inspection
Report 94-09,
the
licensee initiated
CRDR 9-4-0157.
As resolution for CRDR 9-4-0157,
the
licensee initiated
an evaluation of the existing
EC configuration.
Additionally, they initiated
an evaluation to determine if a modification to
the
EC expansion
tank level
and pressure
alarms
were necessary.
The licensee
also revised the annunciator
response
procedures
to note that the expansion
tank instrumentation
had
been valved out of service.
The licensee's
evaluation of the expansion
tank instrumentation
configuration
concluded that the 12-hour operator
rounds
was
an adequate
compensatory
measure.
The evaluation
recognized that the system design leak rate
was
9 gallons per
day.
However,
the evaluation
took credit for plant personnel
correcting
a
system leak without identifying how they would know the leak represented
a
problem.
The inspector considered
that the evaluation of CRDR 9-4-0157
represented
a missed opportunity to identify the lack of guidance to operators
and maintenance
personnel
on the limits to
EC system leakage.
In August
1994,
the Technical
Review Committee,
responsible
For reviewing all
plant change
requests,
approved
a modification which would provide seismically
qualified instruments for level
and pressure
alarms.
The committee determined
that the modification was justified and would eliminate
an operator action to
compensate
for the isolated,
nonseismically qualified level
and nressure
instrumentation.
The licensee
implemented
the modification in the Unit 2 Train
B
EC system.
At
the time of this inspection,
the licensee
had not implemented the modification
in the other five
EC trains.
On December
15,
1995,
the licensee's
work
schedule
indicated that the modifications would be completed
in by August
1996.
These modifications will restore
the chill water leak monitoring to
full conformance with the
FSAR system description.
The inspector
found that the modification performed
on Unit 2 Train
B was
appropriate
and that the plans for completion of the modifications were
acceptable.
l
\\
1
-18-
4.3.5
Deficiencies
Impacting
EC System Inventory
During the inspection period,
the inspector identified various deficiencies
that could have
impacted the
EC system inventory.
On January
10,
1996,
the inspector identified
a substantial
water leak
past
a thermowell
in the Unit 2 Train
B
EC chiller cooler.
The
magnitude of the leak had increased
recently
and could have developed
following the previous
12-hour auxiliary operator
round.
The inspector
notified the control
room supervisor
and corrective actions
were taken
by
HVAC maintenance
during the shift.
The inspector
noted that
no
attempt
had
been
made to quantify the leak and that operators
had not
considered
the impact on chiller operability.
The repair was documented
in the weekly preventive maintenance
task.
~
On January
10, the inspector
noted seal
leaks in both the Unit
1 Train
B
EC pump
and the Unit 3 Train
B
EC pump.
The Unit
1
pump seal
leak was
approximately
80 drops per minute
and the Unit 3
pump seal
leak was
approximately
120 drops per minute.
The inspector
noted that neither
pump
had
a work request
tag identifying the leak.
The inspector
subsequently
determined that there
was
an
open work request for the leak
on the Unit 3 Train
B
EC pump.
However,
the work request
was cancelled
on January
18 by the
EC maintenance
team leader
based
on
a walkdown he
had performed in December
1995.
~
On January
10,
1996,
the inspector
noted that, while there
was
no active
leak on the Unit
1 Train A EC pump, it did have
a work request
tag which
identified
a leak rate of "2-3 drops per second."
The .inqpector
subsequently
identified that this work request
had
been cancelled
in
August
1995.
~
On January
12,
1996,
the inspector
noted that the Unit 3 Train A
EC
expansion
tank makeup Valve EC-LV-015 had
a Work Request
Tag 900965,
which stated that valve would not make
up to the surge tank with the
valve open.
The inspector discussed
this work request with the Unit 3
shift supervisor.
On January
14, technicians
determined that
deficiencies
in the setpoints for the expansion
tank nitrogen regulator
and the expansion
tank level instrument controlling Valve EC-LV-015
contributed to prevent
makeup flow with the
makeup valve open.
In
essence,
the stackup of instrument setpoint deficiencies
had resulted
in
an expansion
tank pressure
that exceeded
the pressure
of demineralized
water makeup pressure.
The shift supervisor initiated
CRDR 3-6-0013 to
have the condition evaluated.
The inspector
noted the shift supervisor
was aggressive
in pursing
and ensuring resolution of these
concerns.
On January
16, the inspector
met with HVAC maintenance
management
and
team
members,
the
EC system engineer,
and the
EC design engineer.
The inspector
noted the findings discussed
above
and the current configuration of the
EC
system instrumentation.
The inspector
noted that the
EC design engineer
was
1
(I
I
l
-19-
not aware that
an
EC leak rate calculation existed.
Suhs
quent to the
meeting,
the inspector
reviewed the
DBH for the
E
system
and noted that it
specified
a maximum leakage criteria of 0.375 gph.
The inspector
informed the
licensee of the criteria.
On January
19, the inspector
performed
a rough calculation of the size of the
Units
1 and
3
EC pump seal
leaks.
The inspector estimated that the Unit 3
Train
8
EC
pump seal
leak exceeded
the 0.375
gph criteria.
The inspector
informed the licensee of this estimate.
The licensee
subsequently
collected
seal
leakage for an extended
period
and determined
the leak rate to be
approximately 0.42 gph.
The licensee
removed the
pump from service shortly after the leak rate
measurement
and repaired
the seal.
The licensee
also performed
a review of
the maximum leak rate criteria (see Section 4.3.3)
and revised it to the
0.63 gph.
The licensee
scheduled
a repair of the leaking Unit
1
EC pump seal
for the next Train A outage.
4.3.6
Conclusions
The inspector
concluded that the
EC system configuration
changes
had not been
appropriately
reviewed
and that the c'ompensatory
actions
had not been
adequate.
In addition, the licensee
had subsequent
opportunities to identify
that the
EC system
was sensitive to
a system leak.
In particular,
in
February
1994,
the licensee
performed
a calculation which determined that the
system
maximum leak rate
was
9 gallons per day.
In Hay 1994, the licensee
recognized
a regulatory requirement that this leak rate
be verified with
testing. 'owever,
the licensee
had not established
testing
and
had not made
operations
or maintenance
personnel
aware that
a relatively small
system leak
could exceed
design basis calculations.
The inspector
subsequently
identified
a system leak which exceeded
the design basis calculation.
The failure to perform
EC system leak rate testing
was identified as
a
violation of 10 CFR Part 50, Appendix B, "Test Control."
(528/9525-03)
The inspector considered
that the licensee
had identified this violation in
Hay 1994.
However,
the inspector considered
that the licensee
had not
corrected
the violation in a reasonable
time.
This conclusion
was
based
on
the relatively small size of the maximum leak rate of the system,
the
consequences
of a loss of
EC system
volume during
a design basis
event,
the
initial lack of recognition of the
HVAC maintenance
and engineering
team of
the potential
impact of
EC system leakage,
and the presence
during the
inspection of a leak which exceeded
this leak rate.
4.4
EC
S stem Review Conclusions
The inspector discussed
these
findings with plant management
at the exit
meeting
and in subsequent
discussions.
The licensee
recognized that the
performance
of the
HVAC team,
including maintenance,
system engineering,
and
design engineering,
had
been
weak.
In addition,
they concurred that
~
1
'
-20-
manaevent
had not been sufficiently critical in thei- evaluation of the
team's
performance
in the resolution of the
EC system issues.
The licensee initiated
an
EC system task force modeled after the successful
emergency diesel
generator
task force, discussed
in Section 7.2, to address
problems identified by the line organization,
Nuclear Assurance,
and the
inspectors.
5
MAINTENANCE OBSERVATIONS
(62703)
5. 1
Essential
Chiller Haintenance
Unit
1
On January
10,
1996,
the inspector
observed
the
HVAC technicians verify the
setpoint
and operation of the low refrigerant temperature
switch for the
Train
B chiller.
The inspector
noted that the technicians
were knowledgeable
about the mechanics of the chiller.
The technicians
performed the component
manipulations
as directed
by the maintenance
task.
The inspector
concluded
that the technicians'erformance
of the maintenance
task
was good.
5.2
Lon
Term Coolin
Instrument
Loo
Calibration - Unit 2
On January
17, the inspector
observed
the instrument
and calibration
technicians
perform
an instrument
loop calibration for the Train
B high
pressure
safety injection long term cooling flow transmitter.
The inspector
noted that the technicians
exhibited
good knowledge of the Rosemount
transmitters
and strong
knowledge of the maintenance
task
and calibration
equipment.
The inspector
concluded that the technicians'erformance
of the
maintenance
task was good.
5.3
Other Maintenance
Observations
The inspectors
observed
the following maintenance
activities
and determined
that they were performed
by knowledgeable
technicians
using appropriate
procedures:
~
Breaker
and cubicle alignment of a Class
4160 volt breaker
Unit
1
~
Inspection of Limitorque
SMC valve motor operator
Unit 2
6
SURVEILLANCE OBSERVATION
(61726)
6.1
HTC Test at Power - Unit 3
On January
13, the inspector monitored the preparations
for the
HTC
surveillance test.
The inspector
noted that the shift supervisor
questioned
what the effect of inserting the control element
assemblies
(CEAs) below
transient insertion limit (TIL) would have
on
SDH.
height required to assure
that,
at
any power level, the minimum shutdown
margin is available,
maximum core peaking factors
are not exceeded,
and the
~
I
a
l
i
I
-21-
reactivity inserted
by the postulated
ejection
oF '.he highest worth
CEA is
acceptable.
The inspector
noted that the shift supervisor's
proactive
questioning
before starting the
HTC test revealed
several
deficiencies.
6. 1. 1
SOH Verification
The shift supervisor directed the shift technical
advisor to calculate
the
with the
The shift technical
advisor utilized the
operator assistance
program aid to calculate
the
SDM and initially reported
that the
SOH would be adequate.
Upon further review,
the shift technical
advisor identified that the incorrect operator
assistance
program aid was
used.
Using the correct program,
the shift technical
advisor determined that
the
SDN would be inadequate if the
CEAs were inserted
below the TIL.
The
licensee
issued
a
CROR to evaluate this error.
The inspector inquired whether the shift technical
advisor verified adequate
SDM using the operator assistance
program aid with all the
CEAs fully
withdrawn (150 inches).
The operator assistance
program aid indicated that
the
SDH was inadequate
because it was overly conservative.
The inspector
and
the shift supervisor discussed
the operator assistance
program
SDN results.
The shift supervisor
concluded,
and the inspector
agreed,
that the unit did
not have
an immediate safety concern since
adequate
SDH existed
as long as
CEAs remained
above the TIL curve.
The inspector
noted that the detailed
SOM
calculation contained significant conservatisms
to account for uncertainties
introduced
by the simplicity of the operator
assistance
program aid.
The
shift supervisor reverified with nuclear fuels management
that
SDH was
adequate.
The licensee
issued
a night order to all three units explaining that,
iF the
CEAs are
above the TIL curve,
the
SDN was assured
by the saFety analysis
calculations
performed prior to each cycle.
The licensee
issued
a
CRDR to
evaluate
the
SDN concerns.
The inspector
concluded that the licensee's
initial corrective actions
were appropriate.
6.1.2
TIL Curve
The licensee identified that the TIL curve in the Unit 3 Cycle
6 core
operating limits report was incorrectly drawn.
The TIL curve indicated that
for 100 percent
power, the TIL was at regulating group five at
120 inches.
The licensee
reviewed the core reload analysis
and noted that the correct
value for the curve was regulating group five at
108 inches
as it has
been in
the past for all three units.
On January
14, the licensee
issued
a revision
to the core operating limits report with the correct TIL curve.
In addition,
the licensee initiated
a
CRDR to evaluate
how the curve was changed
and not
identified.
The inspector
concluded that the licensee's initial corrective
actions
were appropriate.
-22-
On January
13, the licensee identified that, durirg the performance of several
procedures,
operators
missed
the opportunity to identify the incorrectly drawn
TIL curve.
Specifically, during the performances
of 40ST-9ZZ23,
"CEA Position
Data Log;" 40ST-92Z16,
"Routine Surveillance Daily Hidnight Logs;" and
"Control
Room Data Sheet Instructions;" all of which refer to the
TILs, the operators
did not identify the discrepancy
that
some
CEA positions
were being recorded
below the incorrectly drawn TIL curve.
The operators
withdrew the regulating group fi,ve CEAs one step
and verified
all position indications
were greater
than
120 inches.
The licensee
issued
a
memo to all shift supervisors
to remind crew members
to exercise attention to
detail
when consulting reference
material utilized to satisfy surveillance
test procedures.
The licensee initiated
a
CRDR to further evaluate
the event.
Although there were
some instances
where
CEAs were inserted
below 120 inches,
the inspector
noted that
CEAs were never inserted
beyond the
108 inch limit of
the corrected
TIL curve.
However,
the operators
demonstrated
inattention to
detail
by not verifying the limits specified
in the procedures.
The inspector
concluded that the initial corrective action to withdraw the
CEAs one step to
ensure
compliance with the original TIL curve until issuance
of the revised
TIL curve was appropriately conservative.
In response
to the issues
identified during the
HTC test,
the licensee
initiated
a total of four CRDRs to evaluate
and disposition the issues.
The
inspector will review the licensee's
corrective actions during
a future
inspection
(Inspection
Followup Item 530/9525-04).
6. 1. 3
Conclusions
The inspector
concluded that the
hift supervisors
questioning attitude prior
to the performance of the
HTC test contributed to the identification of
several
issues.
These
issues
included:
(1) errors
by reactor engineering
relative to the TIL curve;
(2) missed opportunities
by the
ope} ators to
identify the errors in the TIL curve
due to inattention to detail;
and
(3) the
use of an outdated
operator assistance
program
by the shift technical
advisors.
The inspector
noted that the licensee
had demonstrated
a sufficient
level of concern for these reactivity control
issues
and that their immediate
corrective actions
were appropriate.
6.2
EC and Ventilation
S stems
Ino erable Action Surveillance
Unit
1
On January
10, the licensee
removed the Train
8
EC system
from service.
The
inspector
noted that the operators
did not place
an entry in the Technical
Specification
component condition record computer to evaluate
the opposite
train components
every
4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />
when
an
EC train was
removed
from service.
The
inspector
noted that this entry had
been routinely placed
in the computer in
the past to alert operators
to evaluate
the opposite train components.
The inspector discussed
this observation with the shiFt supervisor.
The shift
supervisor directed
a reactor operator to place
the entry into the computer.
I
i
I
-23-
The inspector discussed
the event with operations
management.
The operations
department
leader
noted that the procedure directs
an operator to make the
computer entry "if desired."
The operations
department
leader
issued
an
instruction
change
request
to clarify the procedure
and operations
management's
expectation
that the entry into the computer shall
be performed.
The inspector
noted that, while the initial procedure
was weak, it had not
resulted
in
a performance
problem.
The inspector
concluded that the
corrective actions
were appropriate.
7
ONSITE ENGINEERING
(37551)
7. 1
Solenoid Valve 0 erator
De raded Internal Wirin
On January
3, the inspector
noted during
a routine review of the Unit
1 logs,
that
had
been
performed
on December
22,
1995,
concerning
the operability of several
solenoid-operated
valves inside
containment.
On January
4,
1996,
the inspector
reviewed the operability
determination with both environmental qualifications
(Egs)
and valve services
engineers.
The inspector identified that the operability determination,
as
documented
on December
22,
1995, did not provide sufficient basis for
operability.
On January
5,
1996,
the licensee
revised
the operability
determination
and initiated investigation actions to provide additional
information.
7. l. 1
Actions Taken
h-
the Licensee Prior to January
4
On November
16,
1995, during maintenance activities
on Unit 3 pressurizer
steam
space
sample line containment isolation Valve SS-UV-205, valve services
technicians
identified that the internal wiring f.r the valve had
been
severely
heat
damaged.
The valve was
an environmentally qualified
solenoid-operated
valve.
The heat
damage
was to the lead wires spliced to the
electrical
conduit seal
assembly
(ECSA) which established
the environmentally
qualified seal for the solenoid housing.
The insulation of several
wires was
found to be cracked,
in some instances
exposing
the conductor,
and the
fiberglass
sleeving
over the wires had lost its elastomer coating.
Additionally, the
ECSA sealing material
appeared
to have melted.
On November 21, valve services
engineering initiated
a
CRDR which was
subsequently
assigned
to the
E() group.
The
CRDR was determined
to be
potentially significant
and
an evaluation
was completed
by the
Eg group
on
December
20.
Although
a history search
performed for the initial evaluation,
had not identified any similar failures, valve services
personnel
identified
a
similar failure in 1993
on Unit 2 Valve SS-UV-204 to the
Eg group
on
December
21.
This prompted the licensee
to consider
an operability
determination for the following valves:
I
sample valve for the hot leg, pressurizer
surge line,
and pressurizer
steam
space
(SS-UV-203,
-204 and -205, respectively) for
all three units.
Six steam generator
sample valves for all three units.
~
Two auxiliary pressurizer
spray Valves
CH-UV-203 and -205 in Units
2
and 3.
The Unit
1 valves were
a different model.
Plant operations
completed
on December
22 in
accordance
with Procedure
The licensee
had determined that the condition represented
a significant
condition adverse
to quality and performed
a root cause
review on both the
ECSAs found on Unit 3 SS-UV-205
and Unit 2 SS-UV-204.
In addition, the
Eg
group performed
a Justification for Continued Operation
as required
by
7.1.2
Assessment
of the
On January
4,
1996,
the inspector questioned
the adequacy of the
OD based
on
the following observations:
~
.
The
OD identified that the degraded
fiberglass
sleeving
guarded
against
shorting
and that it was not expected
to further degrade significantly.
The inspector
found that this statement
was not well supported.
The
inspector
noted that lead wires were relatively long and would have
been
tightly packed
inside the housing.
Although the SS-UV-205,performed
adequately prior to its disassembly, it appeared
that similar
degradation
at other point". in the wiring could lead to
a fault.
In
addition,
the wiring from Unit 2 SS-UV-204
had faulted.
The
OD stated that in a review of the operating circuits,
a failure
could only either cause
a valve to close or result in a loss of position
indication.
The valves
are de-energize
to close.
The inspector
questioned
whether
a fault between
a solenoid wire and
an indication
wire could either cause
a valve to open or keep
an open valve from
closing.
The licensee
subsequently
determined that it was possible for
an open valve to remain
open
due to
a fault.
The
OD stated that the safety function of the reactor coolant
and
steam
generator
sample valves
was to close to provide containment isolation.
However,
the
OD did not address
the Technical Specification 3.3.F 1
function for these
valves to open to support postaccident
sampling.
The
OD noted that the safety function of the pressurizer
spray valves
was to open.
However, it discussed
the availability of redundant
means
called out in the emergency
operating
procedure
to perform the
same
i
0
f
-25-
Function.
The
OD did not address
the Technical Specification 3.4.3.2
requirement that both auxiliary pressurizer
spray valves
be operable.
7.1.3
Licensee Actions Taken After January
4,
1996
On January
5, the licensee
performed
more investigation
and revised the
OD.
The licensee
discussed
their conclusions
that the subject valves
remained
operable with the inspector,
Region
IV personnel,
and the
NRR project manager
during
a conference call.
The revised
OD was
based
on additional
information
and analysis
gathered
since January
4.
The inspector
concluded that this
information and analysis
provided reasonable
assurance
that the subject
valves
remained
In addition,
the licensee
identified that there
was
a timing circuit which,
following the opening of one of these
dropped
the voltage
applied to the solenoid
from the opening voltage of 125 Vdc to
a holding
voltage of 40 Vdc.
They noted that this circuit was not tested
and its
failure would not directly be observed.
They suspected
that the failure of
this timing circuit could have
caused
the
damage to Unit 3 SS-UV-205.
The licensee
subsequently
initiated testing of the circuits to the subject
valves to determine if there
were
any existing faults
and to check the
functioning of the timing circuits.
The licensee identified four timing
circuits that were not functioning properly, including the circuit for Unit 3
Valve SS-UV-205.
The licensee
also identified two other conditions which
contributed to the excess
heat to Valve SS-UV-205.
The valve operator
housing
had
been insulated,
increasing
the heat retained
in the valve operator.
In
addition,,in,.1994
a modification had
been
made to this family of valves.
The
licensee
had lowered the point where the
ECSA entered
the valve operator
housing with respect
to the valve body, increasing
the heat to which the
was exposed.
The licensee
determined that, while none of these conditions
alone could have resulted
in the damage,
the three
combined would have.
The licensee
inspected all of the subject valves
and
removed insulation in the
instances
where it was identiFied.
The licensee
also evaluated
the as-found
conditions of all the subject valves
and concluded that they were operable.
7. 1.4
Licensee
Followup Actions
The licensee initiated
a significant condition investigation which was
underway at the
end of the inspection period.
In addition,
they planned to
perform
a lessons
learned evaluation of problems identified during their
investigation.
The inspector
discussed
some preliminary findings with the
engineering
department
leader responsible for Eg.
He noted that the following
problems
would be reviewed:
~
The
CRDR initiated for the Unit 3 SS-UV-205 deficiency was prematurely
assigned
to the
Eg group.
This assignment
presupposed
that the cause
t
was environment related.
~
I
I
J
l
-26-
The history search did not reveal similar problems with ECSAs.
However,
a similar failure was identified by the valve services
group.
~
The licensee
had originally planned
to perform
a root cause
evaluation
for the
1993 failure of Unit 2 SS-UV-204.
This evaluation
had not been
performed.
~
As discussed
above,
the configuration
and environmental
conditions for
many of these
valves did not match the
Eg analysis.
7. 1.5
The inspector discussed
the findings regarding-the
OD for the
as well
as the failure to document
an
OD for the Unit 3 Train A
EC chiller (see
Section 4.2), with the operations
department
leader responsible for the
process.
The inspector
noted that the evaluation contained
in the
addressed
the operability of several
valves,
each with more than
one
safety-related
function.
The
OD had not been structured
such that
a reader
could easily determine
what deficient condition was being addressed
for a
given function.
The operations
department
leader concurred with the
inspector's
assessment.
In addition,
the operations
department
leader recognized
that the
contained substantially
more information than the process
intended
and did not
capture
what additional
information was needed
to further develop the
determination.
He noted that these
weaknesses
had
been previously identified
and that
he planned to address
these
issues
with further training and
a
procedure
revision planned for mid 1996.
The inspector
noted that the
problems
observed,
during this inspection period regarding
OD, concerned
the
implementation of the procedure
and not its content.
The inspector considered
that the planned actions to be appropriate if an emphasis
was placed
on the
program implementation.
7.1.6
Conclusions
In summary,
the inspector identified that operations
failed to critically
assess
an incomplete operability determination
evaluation provided
by the
engineering
organization regarding safety related
Plant
operations
approved
an
OD evaluation for several
safety-related
solenoid
valves
even though the engineering
evaluation did not thoroughly address
the
effects of potential
heat
damage
on the Technical Specification functions of
some of the valves.
The inspector
observed
that when the weakness
in the
evaluation
was brought to the licensee's
attention,
they took prompt actions
to re-perform the evaluation
and provided
a thorough analysis.
Additionally,
the licensee
took prompt action to inspect for and correct deficient plant
conditions.
The inspector
noted that the licensee
had initiated
a lessons
learned evaluation of the issue
and
had identified weaknesses
in the
implementation of the corrective actions,
the
Eg,
and the
OD programs.
-27-
7.2
Emer enc
Diesel
Generator
Cooldown Tri
Evaluation
In
NRC Inspection
Report 50-528/95-21;
50-529/95-21;
50-530/95-21,
the
inspector
noted that the licensee
had established
a multi-discipline task
force to determine
the cause of several
emergency diesel
generator
cooldown
trips.
During this inspection period,
the task Force completed its
investigation.
The task force determined that minor degradation
of several
different components
in the nonsafety-related
maintenance
run protective trip
circuitry had apparently
combined to cause
the spurious trips.
They
identified that
some of the degradation
was due to aging
and that
some of the
degradation
had
been
induced
by maintenance
practices.
The task force presented
their findings to plant management
and initiated
corrective actions to address
the deficiencies identified.
They also
presented
their findings to the Cooper-Bessemer
diesel
owners group.
The inspector
noted that the evaluation
performed
by the task force was
an
exceptional
product.
The investigation to identify the deFiciencies
had
been
based
on reviews of industry experience,
equipment
performance
trending, field
observations
and troubleshooting,
interviews with operators,
and laboratory
testing.
The results
were documented
in a clear format describing
the
conditions
found, the apparent
causes
of the deficiencies,
and the corrective
actions that were initiated.
8
PLANT SUPPORT
(71750)
Hain Steam
Su
ort Structure
Platform Installation
Unit I
On December
21,
1995,
the inspector
observed
poor housekeeping
and
construction practices
during the installation
oF
a platform in the main
steam
support structure.
Specifically, the inspector
noted that welding
equipment
was left energized
during
a lunch break
and minimal barriers
were
present
to contain metal filings from drilling, grinding,
and welding.
The
inspector discussed
the observations
with the shift supervisor,
and the
director of site maintenance,
and modifications.
On January
3,
1996,
the
inspector
noted that the construction
area
housekeeping
had greatly improved.
At the exit meeting,
the inspector
presented
the concern that the maintenance
foreman
and workers
had
a low sensitivity towards
housekeeping
when working in
safety-related
spaces
in an operating unit.
The site maintenance
director
agreed with the inspector's
concerns
and indicated that the licensee's
expectations
for housekeeping
were re-enforced
to the contractors
performing
the platform installation.
In addition,
the director indicated that the
prejob briefings for the contractors
would address
what equipment could be
affected
by the work being preformed.
The licensee
informed the contractor's
management
that the licensee's
standards
for housekeeping
were not being met.
The inspector
concluded that the licensee's
response
to the issue
was
excellent.
\\
0
-28-
9
FO'LOWUP - OPERATIONS
(92901)
9. 1
Closed
Violation 529 9431-02:
Boron Concentration
Not Verified Ever
2 Hours
This violation involved the failure to determine
the reactor coolant
system
boron concentration
at the frequency specified in the core operating limits
report
when
a startup
channel
high neutron flux alarm was
removed
from
service.
The licensee
determined
that the cause of the violation was that the shift
supervisor
and control
room supervisor did not correctly determine
the
appropriate
actions for an inoperable startup
channel
and the crew did not
question
the assessment
of the required actions.
In addition, the maintenance
task
and surveillance
logs did not provide adequate
guidance for the action
required with a startup
channel
The licensee initiated several
corrective actions.
The licensee
added
a note
to the routine surveillance
logs to perform 4XST-XZZ24, "Startup
Channel
High
Neutron Flux Alarm Inoperable
3. 1.2.7,"
when
a startup
channel
is out of
service.
The licensee
changed
"Excore Startup
Channel
Calibration," to add the requirement
to perform 4XST-XZZ24 prior to removing
startup
channels
from service.
The licensee
issued
a night order to all
three units discussing
the event
and reinforcing management's
expectations
on
performance of surveillance tests.
The inspector
reviewed the procedures
previously mentioned
and noted that the
changes
were
implemented.
The inspector
concluded that the licensee
conducted
a thorough review of the issue
and performed appropriate corrective actions.
9.2
Closed
Violation 529 9431-05:
Failure to Follow CVCS Procedure
Dilution Event
This violation involved the failure to ensure
the automatic
makeup valve
closed after the desired
volume of water
had
been
added to the reactor coolant
system.
The inspector verified the corrective actions described
in the
licensee's
response letter,
dated
January
6,
1995, to be reasonable
and
complete.
The inspector
noted that
no similar problems
were identified.
In
addition,
the inspector questioned
several
licensed
operators
about the
performance of the automatic
makeup valve
and all responded
that the valve
exhibited
no deficiencies.
9.3
Closed
Violation 530 9438-01:
Incom lete Lineu
Durin
Reduced
Inventor
0 erations
This violation involved the failure to align the required reactor coolant
system
makeup
flow paths prior to entering
a reduced
inventory condition.
As
a result,
the licensee
entered
an event with high safety significance.
jl,'fl
-29-
The licensee
determined
the root cause of the event
was
a personnel
error on
the part of the control
room supervisor.
The licensee
noted that the control
room supervisor lost
command
and control of the evolution in that
he did not
direct the alignment or verify the alignment of the makeup flow paths.
The
licensee
concluded that the error by the control
room supervisor
was
an
isolated
performance error and the control
room supervisor
was returned to
onshift duties.
The licensee initiated several
dedicated
midloop crews consisting of a senior
reactor operator 'and
a reactor operator
to perform the midloop evolutions.
The crews were in addition to the normal
crew complements
The inspector
assessed
the performance of the licensee
in several
midloop
conditions.
The inspector
noted
a minor weakness
in the use of the sightglass
for level verification
(NRC Inspection
Report 50-528/95-06;
50-529/95-06;
50-530/95-06).
The inspector
noted strengths
in the performance of the most
recent
midloop evolutions
(NRC Inspection
Reports
50-528/95-10;
50-529/95-10;
50-530/95-10
and 50-528/95-21;
50-529/95-21;
50-530/95-21).
The inspector
concluded that the licensee's
performance of the midloop evolutions
have
significantly improved since the occurrence of the violation.
10
IN OFFICE REVIEM OF LERs
(90712)
The following LER revisions
were reviewed inoffice and determined
to be
acceptable.
The
LERs were issued to correct typographical
errors
and
add
component identification codes.
Revision 2:
Core Protection Calculator,
Delta-T Power
Fluctuations
Revision
1:
Surveillance
Requirement
Hissed for
Containment
Purge Isolation Valves
Revision
1:
Letdown Isolation Valve Leakage
Impact
On
Appendix
R Requirements
Revision
1:
Hisalignment of Limitorque Torque Switch
Contact
Bar Prevented
Remote Operation of HOVs
Revision 2:
Class
1E Batteries
in
a Degraded
Condition
Revision
1:
Use of Uncalibrated
Boronometer
Causes
a
TS
SR to
Be Hissed
ATTACHHENT 1
1
Persons
Contacted
1.1
Arizona Public Service
Com an
- T. Cannon,
Department
Leader,
Nuclear Engineering
and Projects
- R. Flood, Department
Leader,
System Engineering
- B. Grabo,
Section
Leader,
Compliance,
Nuclear Regulatory Affairs
- W. Hartley, Offsite Review Committee
Member
- R. Hazelwood,
Engineer,
Nuclear Regulatory Affairs
- M. Hypse,
Section
Leader, Electrical Maintenance
Engineering
- A. Krainik, Department
Leader,
Nuclear Regulatory Affairs
- J. Levine, Vice-President,
Nuclear Operations
- R. Lucero,
Department
Leader,
Electrical Maintenance
- D. Mauldin, Director, Maintenance
- W. Montefour, Senior Representative,
Strategic
Communications
- G. Overbeck,
Vice President,
Nuclear Support
- C. Russo,
Department
Leader,
Nuclear Assurance
- C. Seaman,
Director, Nuclear Assurance
- W. Stewart,
Executive Vice-President,
Nuclear
1.2
NRC Personnel
- D. Kirsch, Chief, Reactor Projects
Branch
F
- K. Johnston,
Senior Resident
Inspector
- D. Garcia,
Resident
Inspector
- J. Kramer, Resident
Inspector
1.3
Others
- F. Gowers, Site Representative,
El
Paso Electric
- K. Slagle,
Manager,
San Onofre N<<clear Oversight
- Denotes those
present
at the exit interview meeting held
on January
24,
1996.
The inspectors
also held discussions
with and observed
the actions of other
members of the licensee's
staff during the course of the inspection.
2
EXIT MEETING
An exit meeting
was conducted
on January
24,
1996.
During this meeting,
the
inspectors
summarized
the scope
and findings of the report.
The licensee
acknowledged
the inspection findings documented
in this report.
The licensee
did not identify as proprietary
any information provided to, or reviewed by,
the inspectors.
r
t[
ATTACHMENT 2
LIST OF ACRONYMS
CRDR
DBM
EC
Eg
GDC
LER
MTC
control element
assembly
condition report/disposition
report
design basis
manual
essential
chilled water
electrical
conduit seal
assembly
engineering
evaluation report
environmental qualification
final safety analysis report
general
design criteria
heating, ventilation,
and air conditioning
licensee
event reports
moderator
temperature
coefficient
shut
down margin
insertion limit
i
'
k