ML17313A907
| ML17313A907 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 04/29/1999 |
| From: | NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV) |
| To: | |
| Shared Package | |
| ML17313A906 | List: |
| References | |
| 50-528-99-04, 50-528-99-4, 50-529-99-04, 50-529-99-4, 50-530-99-04, 50-530-99-4, NUDOCS 9905040294 | |
| Download: ML17313A907 (38) | |
See also: IR 05000528/1999004
Text
e
ENCLOSURE
U.S. NUCLEAR REGULATORYCOMMISSION
REGION IV
Docket Nos.:
License Nos.:
Report No.:
Licensee:
Facility:
Location:
Dates:
Inspectors:
Approved By:
50-528
50-529
50-530
NPF-51
50-528/99-04
50-529/99-04
50-530/99-04
Arizona Public Service Company
Palo Verde Nuclear Generating Station, Units 1, 2, and 3
5951 S. Wintersburg Road
Tonopah, Arizona
February 7 through March 20, 1999
J. H. Moorman, III, Senior Resident Inspector
D. R. Carter, Resident Inspector
N. L. Saigado, Resident Inspector
G. W. Johnston, Senior Project Engineer
P. Harrell, Chief, Project Branch D
ATTACHMENT: Supplemental Information
0
9905040294
990429
ADOCK 05000528
9
EXECUTIVE SUMMARY
Palo Verde Nuclear Generating Station, Units 1, 2, and 3
NRC Inspection Report No. 50-528/99-04; 50-529/99-04; 50-530/99-04
~Oerations
~
Misdiagnosis of plant conditions and unnecessarily hurried operator actions in response
to a failure in the main turbine electrohydraulic control system caused a Unit 1 reactor
trip on high pressurizer pressure.
Posttrip operator actions were good (Section 04.1).
~
A violation of Technical Specification 4.5.2.d.3 was identified for the failure to perform
the required surveillance test on the trisodium phosphate baskets.
This Severity
Level IVviolation is being treated as a noncited violation per the guidance provided in
Appendix C of the Enforcement Policy. This issue is in the licensee's corrective action
program as Condition Report/Disposition Request 9-8-Q047 (Section 08.1).
Maintenance
Observable material condition of the three units was good. During a posttrip walkdown
of the Unit 1 containment, the licensee discovered a moderate amount of boron crystals
on carbon steel components of Reactor Coolant Pump 2A. The licensee's actions to
address the boron accumulation were good (Section M2.1).
The licensee failed to take actions to ensure that a deficient condition was appropriately
corrected on all affected components.
As a result, the deficiency was not corrected for
all turbine-driven auxiliary feedwater pumps in all units. This deficiency was identified
again by an overspeed trip of the Unit 2 turbine-driven auxiliary feedwater pump. This is
a violation of 10 CFR Part 50, Appendix B, Criterion XVI. This Severity Level IV
violation is being treated as a noncited violation consistent with Appendix C of the NRC
Enforcement Policy. The licensee took prompt actions to assess
transportibility and
correct the conditions. This issue is in the licensee's corrective action program as
Condition Report/Disposition Request 2-9-0019 (Section M3.1).
The licensee failed to provide sufficient design basis information in the appropriate
procedures.
As a result, missing and/or loose bolts were identified on the Units 1,2,
and 3 emergency diesel generator air-start headers.
The torque value for the bolts was
increased from 25 to 50 foot-pounds, and the bolts that required torqueing were not
identified in the appropriate maintenance instructions.
This is a violation of 10 CFR Part 50, Appendix B, Criterion III, for not implementing design basis information. This
Severity Level IVviolation is being treated as a noncited violation consistent with
Appendix C of the NRC Enforcement Policy. This issue is in the licensee's corrective
action program as Condition Report/Disposition Request 3-9-0026.
During routine testing of a containment isolation valve for the hydrogen control system,
the valve failed to function, as designed.
The failure was caused by the valve wiring
being improperly installed following maintenance.
The condition was not detected by
postmaintenance
testing because
the procedure, which specified the testing
requirements for the valve actuators, was inadequate.
This is a violation of 10 CFR
-2-
Part 50 Appendix B, Criterion XI; however, this Severity Level IVviolation is being
treated as a noncited violation, consistent with Appendix C of the NRC Enforcement
Policy. This issue is in the licensee's corrective action plan as Condition
Report/Disposition Request 3-9-0010 (Section M4.1).
~
Inattention to detail led to a failure to followprocedures while retrieving and verifying
replacement 480-Vac circuit breakers.
This resulted in the installation of two
nonsafety-related circuit breakers into safety-related motor control center cubicles
affecting two high pressure safety injection valves. This is a violation of Technical
Specification 5A.1 for the failure to followprocedures.
Postwork reviews also failed to
prevent the discrepancies.
This Severity Level IVviolation is being treated as a
noncited violation, consistent with Appendix C of the NRC Enforcement Policy. This
issue is in the licensee's corrective action program as Condition Report/Disposition
Request 1-9-0030 (Section M4.2).
~En ineerin
r
A Y2K readiness plan had been developed and was being implemented by the licensee.
The plan was organized and contained the necessary elements to address current and
potential problems from the Y2Kbug. A Y2Kcontingency plan has been developed, but
not finalized. The licensee was well positioned to complete Y2K remediation prior to the
end of the year (Section E1.1).
A violation of Criterion!II was identified for not specifying the correct type of seal fittings
for conduits.
As a result, during flooding of a portion of the auxiliary building, water
entered the conduits. This affected the operability of safety-related equipment.
This
Severity Level IVviolation is being treated as a noncited violation consistent with
Appendix C of the enforcement policy. This issue is in the licensee's corrective action
program as Condition Report/Disposition Request 1-60236 (Section E8.2).
~
The radiological protection program was effectively implemented in those areas
reviewed (Section R1.1).
e
l
Re ort Details
Summa
of Plant Status
Unit 1 operated at 100 percent power until March 10, 1999. The unit experienced a reactor trip
on high pressurizer pressure.
See Section 04.1 for details. The unit was returned to
100 percent power on March 15, 1999, and remained at that power level for the duration of this
inspection period.
Units 2 operated at 100 percent power until March 19, 1999, at which time the unit began a
coastdown for the planned eighth refueling outage.
Unit 3 operated at essentially 100 percent power for the duration of this inspection period.
04
Operator Knowledge and Performance
04.1
-Reactor Tri
Due to Hi h Pressurizer Pressure
Unit 1
a.
Ins ection Sco
e 71707 93702
On March 10, 1999, at 1:26 p.m., the Unit 1 reactor tripped from 100 percent power on a
valid high pressurizer pressure signal. The inspectors responded to the control room to
observe operator actions and assess
plant conditions. The inspectors also conducted
interviews with operators and reviewed personnel statements and the printout of alarms.
b.
Observations and Findin s
The Unit 1 reactor was operating at 100 percent power when a failure in the throttle
pressure limiter circuit of the main turbine electrohydraulic system caused all turbine
control valves to stroke shut over an approximately 6 second period. Both the steam
bypass control system (SBCS) and the reactor power cutback (RPCB) system properly
responded by opening the steam bypass valves. The secondary operator observed that
the steam bypass valves were open and reported this to the crew. In the span of
approximately 6 seconds,
the control room supervisor (CRS) erroneously diagnosed that
the valves were open due to a failure of the SBCS and directed the secondary operator
to shut all steam bypass valves using Emergency Off Switch HS-1010.
The secondary
operator responded promptly to the order by the CRS, after a cursory review of plant
parameters.
When all steam bypass valves went shut, some of the main steam safety
valves lifted and reseated.
This transient resulted in a high pressurizer pressure reactor
trip. A short time after the reactor trip, the SBCS was returned to automatic.
The
operators responded to the reactor trip by using the correct procedures, and they
appropriately classified the trip as uncomplicated.
Approximately 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> after the trip, a timer card in the steam generator (SG) level
control circuitry failed. This caused the economizer feedwater control valve to SG B to
open slightly. Despite operator actions to isolate feedwater to the SG, level increased to
the high SG level main steam isolation signal setpoint.
After the main'steam isolation
f
!
f
1
!
-2-
signal setpoint was reached, the operators continued the plant cooldown by using the
atmospheric dump valves.
The operators successfully stabilized the plant in Mode 3 at
normal operating pressure and temperature.
From review of personal statements taken from the shift crew and the control room
alarm printout, the inspectors determined that the initial diagnosis was conducted in the
first few seconds of the transient without observation of turbine load or control
valve'osition.
Although the operators looked for the RPCB, the decision to take the SBCS to
emergency off was made just prior to receiving the RPCB alarm.
Lowering turbine load
and control valve position was not observed by either the control room supervisor or the
secondary operator prior to making the diagnosis.
These indications would have
confirmed to the operators that the SBCS was functioning properly in response to
closure of the control valves. The inspectors determined that operator response to the
transient was conducted without the proper diagnosis, unnecessarily hurried, and
inappropriate for plant conditions at the time. The licensee was further assessing
operator performance through a formal human performance evaluation to determine
how the event willbe incorporated into operator training.
C.
Conclusions
Misdiagnosis of plant conditions and unnecessarily hurried operator actions in response
to a failure in the main turbine electrohydraulic control system caused a Unit 1 reactor
trip on high pressurizer pressure.
Posttrip operator actions were good.
08
Miscellaneous Operations issues (92901)
08.1
Closed
Licensed Event Re ort LER 50-529/98-001:
Surveillance Test Deficiency
Found During Quality Assurance Audit Leads to Technical Specification (TS) 3.0.3/4.0.3
Entry.
On March 1, 1998, Unit 2 control room personnel declared both trains of the emergency
core cooling system inoperable due to exceeding, the specified 18 month surveillance
interval of TS Surveillance Requirement (SR) 4.5.2.d.3, "ph of Trisodium Phosphate,"
pIus the maximum allowable extension of 25 percent.
The licensee entered limiting
condition for operation TS 3.0.3 and invoked the provisions of TS SR 4.0.3 to allow up to
24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> to permit the completion of the missed surveillance.
The licensee satisfactorily
completed TS SR 4.5.2.d.3 analysis within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> and exited TS limiting condition'for
operation TS 3.0.3.
The failure to perform the SR within its required interval is a violation of TS 4.5.2.d.3.
This Severity Level IVviolation is being treated as a noncited violation, consistent with
Appendix C of the NRC Enforcement Policy. This violation is in the licensee's corrective
action program as condition report/disposition request (CRDR) 9-8-Q047
(50-29/9904-01).
-3-
Conclusions
A violation of TS 4.5.2.d.3 was identified for the failure to perform the required
surveillance test on the trisodium phosphate baskets.
This Severity Level IVviolation is
being issued as a noncited violation per the guidance provided in Appendix C of the
Enforcement Policy. This issue is in the licensee's corrective action program as
Condition Report/Disposition Request 9-8-Q047.
08.2
Closed
LER 50-530/99-001:
Loss of Automatic Closure of Containment Isolation
Valve (CIV)
This LER was issued to discuss the events described in Section M4.1 of this report.
II. Maintenance
M1
Conduct of Maintenance
M1.1
General Comments on Maintenance Activities Units 2 and 3
a.
Ins ection Sco
e 62707
The inspectors observed all or portions follwing activities performed per the listed work
document:
874321
"Troubleshoot and Rework the Problem With Valve 2JSIBUV0659
Stroking Slow" (Unit 2)
823838
"Perform 6 Month Inspection of Charging Pump A" (Unit 3)
856616
"Replace Charging Pump A Suction and Discharge Valves" (Unit 3)
b.
Observations and Findin s
The inspectors found the work performed under these activities to be properly
performed.
Allwork observed was performed with the work package present an in
active use. Work and foreign material exclusion practices observed were good.
Technicians were experienced and knowledgeable of their assigned tasks.
C.
Conclusions
Knowledgeable technicians used approved procedures to perform routine maintenance
activities in a safety conscious manner.
Good work and foreign material control
practices were observed.
I
j
II
g
tl
f
e
-4-
M1.2
General Comments on Surveillance Activities Units 2 and 3
a.
Ins ection Sco
e 61726
The inspectors observed all or portions of the following activities performed per the
listed surveillance procedures:
"Control Room AFU Air Flow Capacity and Pressurization Test,"
Revision 5 (Unit 2)
"AFA-P01 - Inservice Test," Revision 10 (Unit 2)
"ESFAS Train B Subgroup Relay Functional Test," Revision 19 (Unit 3)
b.
Observations and Findin s
The inspectors found that knowledgeable personnel performed these surveillances
satisfactorily, as specified by applicable procedures.
c.
Conclusions
Knowledgeable technicians used approved procedures to conduct surveillance activities
in a satisfactory manner.
M2
Maintenance and Material Condition of Facilities and Equipment
M2.1
Review of Material Condition Durin
Plant Tours
Units 1 2 and 3
a.
Ins ection Sco
e 62707
During this inspection period, routine tours of all units were conducted to evaluate plant
material condition. The inspectors also reviewed the licensee's assessment
of boric
acid accumulation on Unit 1 Reactor Coolant Pump 2A.
C
b.
Observations and Findin s
Observations of plant material condition during this inspection period identified no major
observable material condition deficiencies.
Minor deficiencies brought to the attention of
the licensee were documented with work requests.
The licensee conducted a posttrip walkdown of the Unit 1 containment to assess
the
reactor coolant system for boron accumulation that would result from minor system
leaks. This walkdown revealed that a moderate amount of boron crystals had formed on
Reactor Coolant Pump 2A. The boron was on the bolted joint between the reactor
coolant pump seal housing adapter (commonly known as the "top hat") and the seal
housing.
There was also a large quantity of boric acid buildup on the stainless steel
seal housing surface, carbon steel clamp ring, and main closure nuts. The licensee
determined that the leakage that led to the boric acid precipitation was approximately
~
'
e
-5-
4 drops per second from the "top hat" of the reactor coolant pump seal. After the boric
acid had been cleaned off of the components, an inspection was conducted to assess
the components for wastage.
The inspection revealed some wastage of the carbon
steel components.
The amount of corrosion was assessed
by the licensee, and it was
determined that the structural integrity of the clamp ring and main closure nuts was not
affected.
The condition was documented and dispositioned in deficiency Work Order
(WO) 874896 and CRDR 9-9-0326.
The inspectors reviewed the licensee's assessment
and agreed with the conclusions.
Conclusions
Observable material condition of the three units was good.
During a posttrip walkdown
of the Unit 1 containment, the licensee discovered a moderate amount of boron on
carbon steel components of Reactor Coolant Pump 2A. The licensee',s actions to
address the boron accumulation were good.
M3
Maintenance Procedures and Documentation
M3.1
Auxilia
AFW Pum
Turbine Overs
eed
Units1 2 and 3
Ins ection Sco
e 62707
On February 18, 1999, during postmaintenance
testing of AFW Pump A, the pump
turbine tripped on overspeed.
The inspectors evaluated the licensees response to the
turbine trip, conducted interviews with the system engineer, and reviewed related
documentation.
Observations and Findin s
On February 17, the licensee declared AFW system Train A inoperable to perform
planned maintenance on Valves AFA-HV33and AFA-UV37, discharge valves to SG 2B.
On February 18, the licensee operated AFW Pump A in accordance with Surveillance
Procedure 73ST-9AF02, "AFA-P01-Inservice Test," Revision 10, to perform
postmaintenance
testing for Valves AFA-HV33and AFA-UV37. After approximately
19 minutes of operation, AFW Pump A reached the electrical overspeed setpoint and
tripped.
The pump was quarantined for root cause investigation, and the licensee initiated
CRDR 2-9-0019 to document and evaluate the cause of the pump trip. The licensee
initiated WO 872823 to troubleshoot the cause of the overspeed.
Investigation of the
turbine control system by mechanics found that the governor linkage had an
approximate 1/4-inch gap between the governor linkage fork clevis and the spring seats
that are attached to the governor valve stem.
Valve stem position is controlled via a pivoted linkage fork that connects the valve stem
to a remote servomechanism.
The governor valve stem is prevented from rotating by an
antirotation block threaded onto it and held in place by a set screw. The governor
linkage fork clevis fits over the antirotation block and is held onto the valve stem by a set
e
e
-6-
of spring seats on either side of it. The spring seats were threaded onto the valve stem
and held in place by a set screw. A 0.010- to 0.015-inch clearance is required to be
maintained between the linkage fork clevis and the spring seats.
The additional 1/4-inch clearance resulted in the remote Servomechanism
being able to
travel its full length and bottom out before the governor valve could fullyseat to control
turbine speed.
As steam quality improved with turbine operation, the governor valve
could not be closed enough to compensate for the increased efficiency: This resulted in
a gradual increase in turbine speed over a 3-minute period prior to the trip.
The licensee postulated that the excessive clearance could have resulted if: (1) both
the antirotation block and the inner spring seat set screws were either not sufficiently
tightened or had worked loose, allowing the antirotation block and inner spring seat to
change their location on the valve steam, (2) steam flow eddy currents within the
governor valve caused the valve plug and stem to rotate, and (3) the outer spring seat
remained firmlysecured to the stem.
A similar event was documented on Field Change Request (FCR) 65.359-M, dated
October 1983.
In this event, loose set screws on the Unit 1 AFW Pump A turbine
governor inner spring seat allowed the valve stem to turn. This resulted in
uncontrollable excessive turbine speed.
The FCR corrective actions added a second
set screw to the inner spring seat and required the spring seats to be lockwired
together.
During the Unit 2 refueling outage in September 1997, WO 795078 directed the
performance of Maintenance Instruction (Ml) MAF-00008, "Disassemble/Inspect
Turbine
Governor Valve." This procedure did not contain steps to tighten set screws for the
spring seats or reference that two set screws were required.
The procedure did not
provide instructions to lockwire the spring seats together, but did have a step to tighten
the set screw on the antirotation block. The inspectors determined that FCR 2235
corrective actions were never incorporated into all affected plant maintenance
instructions.
The licensee inspected the turbine governor valves on the turbine-driven AFW pumps
for all three units. The outer spring seat on the Units 1, 2, and 3 valves had one set
screw, and only the inner spring seat on the Unit 2 valve had two set screws.
None of
the spring seats in any unit were lockwired. Spring seat clearances
in Units 1 and 3
were inspected and found to have the correct clearance between the linkage fork clevis
and the spring seats.
Set screws in the Units 1 and 3 spring seats were tight. The
licensee had replaced the turbine governor valves on Units 1 and 3 in 1992 and 1994,
respectively.
When the governor valves were replaced, the updated design information
was not applied to the new valves. This resulted in the governor valve spring seats
being installed without two set screws.
The failure to incorporate FCR 2235 actions into maintenance instructions and to
maintain design control measures commensurate
with those applied to the modified
design is a violation of 10 CFR Part 50, Appendix B, Criterion III, "Design Control." This
Severity Level IVviolation is being treated as a noncited violation, consistent with
Appendix C of the NRC Enforcement Policy. This violation is in the licensee's corrective
action program as CRDR 2-9-0019 (50-528,-529,-530/9904-02).
\\
-7-
Conclusions
The licensee failed to take actions to ensure that a deficient condition was appropriately
corrected on all affected components.
As a result, the deficiency was not corrected for
all turbine-driven auxiliary feedwater pumps in all units. This deficiency was identified
again by an overspeed trip of the Unit 2 turbine-driven auxiliary feedwater pump. This is
a violation of 10 CFR Part 50, Appendix B, Criterion XVI. This Severity Level IV
violation is being treated as a noncited violation consistent with Appendix C of the NRC
Enforcement Policy. The licensee took prompt actions to assess
transportibility and
correct the conditions.
This issue is in the licensee's corrective action program as
Condition Report/Disposition Request 2-9-0019.
Emer enc
Diesel Generator
EDG Air-Start Manifold Bolts Not Pro erl Tor ued
Units1
2 and3
Ins ection Sco
e 62707
On February 18, 1999, during a routine surveillance, the licensee identified loose and
missing bolts on the Unit 3 EDG A air-start manifold. The inspectors evaluated the
circumstances surrounding the loose and missing bolts, conducted interviews with the
system engineer, and reviewed documentation related to the event.
Observations and Findin s
During performance of Surveillance Procedure 40ST-9DG01, "Diesel Generator A Test,"
Revision 6, an auxiliary operator noted that, of the two bolts holding the air-start
manifold to Cylinder 9R, one was missing and the other bolt was loose. The air-start
manifold bolts on Cylinder 7R were also loose. The licensee initiated CRDR 3-9-0026 to
document the problem and to evaluate the transportability of the issue.
Licensee
management
did not consider the EDG to be inoperable, because the EDG started
within its required time limits and the air-start header was still held in place by the other
cylinders on the right side.
Licensee inspection of all EDGs (two in each of the three
units) revealed that Unit 1 EDG A was also missing a bolt on Cylinder 3R. The licensee
initiated WRs 955623, 955638, 955639, 955640, 955641, and 955642 to verify the
torque on the air-start manifold bolts of all EDGs. All EDG manifold bolts required
tightening approximately 1/4 turn to satisfy the c'urrent specification value of 50 ft-lbs.
CRDR 1-6-0030, dated February 20, 1996, described loose and missing bolts on the
air-start manifold of several cylinders on Unit 1 EDG A and Unit 3 EDG B. An operability
determination associated
with this CRDR concluded that the EDGs remained operable
with the loose or missing bolts, due to the rigidityof the manifold and the number of
bolts that remained tight on the adjacent cylinders. The CRDR root cause determination
identified a low torque value of 25 ft-Ibs as the primary reason for the bolts becoming
loose.
In addition, the CRDR identified that, in order to remove the air-start manifold
from one of the cylinder heads for maintenance,
it was necessary to loosen the bolts on
adjacent cylinders to allow removal of the manifold flange from the head being worked.
However, this portion of the task was not in the maintenance procedure.
f
i
e
-8-
CRDR 1-6-0030 immediate corrective actions included the verification and retorque of
the manifold bolts on both EDGs in all units. Long-term corrective actions included:
~
Increasing the torque value of the manifold bolts from 25 to 50 ft-lbs, and
~
Adding a step to Ml MDG-00046, "Remove and Replace Cylinder Head," to
check the torque on adjacent cylinder head air-start manifold bolts.
The licensee modified Drawing M018-519, "AirStarting Header," on May 29, 1996, to
increase the air-start manifold bolt torque value from 25 to 50 ft-lbs. However, the
licensee did not change any Mls, which mechanics use to perform maintenance on the
EDGs, to incorporate the increased torque value.
In addition, step 4.8.1A was added to
Ml MDG-00046 to torque the manifold bolts on the adjacent heads to the head being
worked. However, the licensee did not add the additional step to Ml MDG-00065,
"Piston Modification," the Ml used to remove the Cylinder 8R head on Unit 3 EDG A.
The failure to implement design basis information into appropriate procedures
is a
violation of 10 CFR Part 50, Appendix B, Criterion III. This Severity Level IVviolation is
being treated as a noncited violation, consistent with Appendix C of the NRC
Enforcement Policy. This violation is in the licensee's corrective action program as
CRDR 3-9-0026 (50-528,-529,-530/9904-03).
The inspectors also determined that Mls MDG-00046 and MDG-00065 did not contain
steps that instructed mechanics to loosen air-start manifold bolts on cylinders adjacent
to those which are to be worked. The inspectors interviewed a maintenance planner,
.who indicated that, if the adjacent manifold bolts needed to be loosened, the Ml would
be modified in the field to reflect the change.
Ml MDG-00065 used to perform work on
Unit 3 EDG A Cylinder 8R did not document the additional steps to loosen the adjacent
manifold bolts.
Licensee corrective actions detailed in CRDR 3-9-0026 included changing the Mls to
increase the torque values of the air-start manifold bolts to 50 ft-lbs, as specified on
Drawing M018-519.
In addition, steps were added to all Mls associated with EDG head,
removal to torque the adjacent cylinder air-start manifold bolts. Steps were also added
to provide for the loosening of adjacent manifold bolts to facilitate the removal of the air-
start manifold on a cylinder head requiring work. Long-term corrective actions required
Maintenance Engineering to screen and identify critical engine systems or components
that have a potential for bolts to become loose and to use this data to change
appropriate Mls. The inspectors determined that these actions were appropriate.
Conclusions
The licensee failed to provide sufficient design basis information in the appropriate
procedures.
As a result, missing and/or loose bolts were identified on the Units 1, 2,
The torque value for the bolts was increased from 25 to
50 ft-lbs, and the bolts that required torqueing were not identified in the appropriate
maintenance instructions. This is a violation of 10 CFR Part 50, Appendix B,
Criterion III, for not implementing design basis information. This Severity Level IV
violation is being treated as a noncited violation consistent with Appendix C of the NRC
Enforcement Policy. This issue is in the licensee's corrective action program as
Condition Report/Disposition Request 3-9-0026.
-9-
Maintenance Staff Knowledge and Performance
Loss of Automatic Closure of a CIV Unit 3
Ins ection Sco
e 62707
On January 23, 1999, the licensee identified that inboard containment hydrogen control
supply isolation Valve'3JHPAUV0001 failed to close during routine functional testing.
The inspectors reviewed the circumstances surrounding the failure of the valve to
operate properly, observed troubleshooting activities, and reviewed LER 99-001-00 and
CRDR 3-9-0010.
Observations and Findin s
On January 23, during the performance of Surveillance Procedure 36ST-9SA01,
"ESFAS Train A Subgroup Relay Functional Test," Revision 19, Valve 3JHPAUV0001
failed to close during testing of containment isolation actuation signal (CIAS) Relay
K213. The licensee initiated CRDR 3-9-0010 to document the valve failure and to
evaluate transportability of the event.
The licensee initiated WO 870644 to troubleshoot and repair the valve. Troubleshooting
revealed that the wiring for Valve HPAUV0001 was not in accordance with
Diagram 3-E-HPB-002, "Containment Hydrogen Control System Hydrogen Control
Containment Isolation Valve 3JHPAUV-1," Revision 6. The improper wiring
configuration not only prevented the valve from closing on a CIAS but, in the as-found
configuration, the valve would have opened on a CIAS. This represented
a partial loss
of the ability to isolate containment.
The wiring was returned to the proper configuration
and the licensee successfully completed Surveillance Procedures 73ST-9XI08, "HC 8
HP Valves - Inservice Test," Revision 2, and 36ST-9SA01 and then declared Valve
3HPAUV0001 operable.
The licensee's investigation determined that on September 24, 1998, during the Unit 3
refueling outage, WO 834623 required the actuator for Valve 3JHPAUV0001 to be
removed for static diagnostic testing per Procedure 32MT-9ZZ56, "Motor Operator
Testing Using MOVATS3500 System," Revision 20. When wiring was removed from
the actuator, Wire 52, which was removed from Terminal 20, was documented on the
determination/retermination sheet as having been removed from Terminal 21. During
reassembly of the actuator on September 26, Wire 52 was reianded per the
determination/retermination
sheet on the wrong terminal (Terminal 21 instead of 20). A
second party verification was conducted, as required by plant procedures, for both the
determination and retermination of all wiring associated
with the actuator maintenance.
The licensee also determined that the wire was configured correctly prior to this
maintenance by successful performance of Surveillance Procedure 36ST-9SA01,
completed on August 2, 1998, and Procedure 73ST-3DG01, "Class 1E Diesel Generator
and Integrated Safeguards Surveillance Test - Train A," Revision 8, completed on
September 21, 1998.
Both surveillances ensured that the entire actuation circuit
operated properly.
f
e
-10-
The retest of the valve specified by WO 834623 was a full-stroke test per
Procedure 32MT-9ZZ49, '-'PM Inspection of Rotork Operators," and a local leak rate test.
These tests functionally verified operation of the valve, but did not test the CIAS portion
of the engineered safeguards feature actuation signal (ESFAS) actuation circuitry. Had
either Surveillance Procedure 36ST-9SA01 or 73ST-3DG01 been identified as a retest,
the wiring error would have been identified.
The licensee determined, as part of CRDR 3-9-0010 root cause determination, that only
Rotork type motor-operated valve actuators are susceptible to having the safety function
disabled during actuator maintenance.
Unique to the 20 Rotork actuators installed in
the three units was the issue of hammering (cyclically energizing/deenergizing
the valve
into its seat) due to the CIAS being a locked in signal. To ensure that the hammering
did not occur for Rotork actuators, the ESFAS control wiring was routed directly to the
actuator, and a limitswitch contact was put in series with the ESFAS actuation to
interrupt the main ESFAS signal prior to seating the valve. The licensee determined
that only 20 valves with Rotork actuators would require ESFAS functional testing after
actuator maintenance.
A review of the test history found that Valves 1JHPBUV002 and
3JHPBUV002 had maintenance performed without a proper ESFAS retest.
The
licensee performed Surveillance Procedure 36ST-9SA01 on the two valves in question,
and both were found to be operable.
Valve 3JHPAUV0001 was incorrectly rewired during maintenance, and this condition
was not detected by the required maintenance
retest.
The valve was inoperable from
September 29, 1998, through January 23, 1999, during which it could not perform its
automatic isolation function. The failure to include adequate
retest requirements in
Procedure 39DP-9ZZ04 for Rotork-type valve actuators is a violation of 10 CFR Part 50,
Appendix B, Criterion XI. This Severity Level IVviolation is being treated as a noncited
violation, consistent with Appendix C of the NRC Enforcement Policy. This violation is in
the licensee's corrective action program as CRDR 3-9-0010
(50-528,-529,-530/9904-04).
c.
Conclusions
During routine testing of a CIV for the hydrogen control system, the valve failed to
function as designed.
The failure was caused by the valve wiring being improperly
installed following maintanance.
The condition was not detected by postmaintenance
testing because
the procedure, which specified the testing requirements, was
inadequate.
This is a violation of 10 CFR Part 50 Appendix B, Criterion XI, for not
performing adequate testing of a valve. This Severity Level IVviolation is being treated
as a noncited violation, consistent with Appendix C of the NRC Enforcement Policy.
This issue is in the licensee's corrective action program as Condition Report/Disposition
Request 3-9-0010.
e
M4.2
Nonsafe
-Related Circuit Breakers Installed In Safe
-Related A
lications Unit 1
a.
Ins ection Sco
e 71707 37551
62707
On February 19, 1999, the licensee identified two nonsafety-related circuit breakers
installed in 480-Vac safety-related Motor Control Center (MCC) Cubicles 1EPHBM3417
and 1EPHBM3419 for high pressure safety injection (HPSI) Valves 1JSIBUV616 and
1JSIBUV636. The inspectors reviewed applicable logs, WOs, procedures, and
CRDR 1-9-0030.
b.
Observations and Findin s
On February 19, a procurement engineer identified, while conducting a routine audit of a
quarterly failure data trending report, that two nonsafety-related circuit breakers were
possibly installed in two safety-related MCC cubicles.
As documented
in the unit logs,
Valves 1JSIBUV616 and 1JSIBUV636 were deenergized to verify breaker qualification.
The inspectors verified that operators had entered TS 3.6.3 Condition A for CIVs and
TS 3.5.3 Condition B for HPSI B. Once the licensee confirmed the discrepancies, WO 872905 was generated and implemented to replace the nonsafety-related
breakers.
The licensee's evaluation of the affect on the safety function of the two HPSI valves
during the time that the nonsafety-related breakers were installed, indicated that the
breakers would have performed their required design function for the following reasons:
~
Overcurrent protection requirements were tested by Procedure 32MT-9ZZ74.
~
MCC cubicles were located in an area classified as a mild environment.
~
Southern California Edison performed seismic tests on the breakers, which
indicated that the breakers remained functional during and after the tests.
Also,
structural integrity of the breakers was verified after the seismic tests.
In March 1998, the licensee performed Plant Change Work Orders 803385 and 803397,
which changed the overload heaters and reset the breakers installed in MCC
Cubicles 1EPHBM3417 and 1EPHBM3419. The existing breakers failed to trip within
the allowed time per Procedure 32MT-9ZZ74, "Molded Case Circuit Breaker Test,"
Revision 14. The licensee initiated Maintenance Instruction Work Orders (MIWO)
834070 and 834067 to procure and test the new breakers.
It was during the
performance of these MIWOs that the licensee incorporated the wrong replacement
parts.
The failure to followprocedures for retrieving and verifying proper replacement parts, as
described in the equipment list associated
with MIWOs 834070 and 834067, is a
violation of TS 5.4.1. This Severity Level IVviolation is being treated as a noncited
violation, consistent with Appendix C of NRC Enforcement Policy. This violation is in the
licensee's corrective action program as CRDR 1-9-0030 (50-528,-529,-530/9904-05).
4
-12-
Completed MIWOs 834070 and 834067 documented the nonsafety-related equipment
identification numbers on several steps.
Thorough postwork reviews would have
provided a second barrier to identify and prevent these discrepancies.
The inspectors
assessed
that the postwork reviews performed for both MIWOs as poor.
The inspectors considered the storage of nonsafety- and safety-related equipment in the
Level B electrical storage area as a contributing factor. The maintenance personnel
retrieved the incorrect replacement breakers from this storage area in Unit 2. While the
storage facilitywas acceptablly maintained per plant procedures, the licensee
subsequently determined that this storage area would only store nonsafety-related
components.
The licensee initiated six WOs to inspect all safety-related MCC cubicles in all units to
verify installation of the proper breakers.
At the conclusion of this inspection period, the
licensee had inspected approximately 50 percent of the MCC cubicles without identifying
any additional discrepancies.
c.
Conclusions
Inattention to detail led to a failure to followprocedures while retrieving and verifying
replacement 480-Vac circuit breakers.
This resulted in the installation of two
nonsafety-related, circuit breakers into safety-related MCC cubicles affecting two high
pressure safety injection valves. This is a violation of TS 5.4.1 for the failure to follow
procedures.
Postwork reviews also failed to prevent the discrepancies.
This Severity
Level IVviolation is being treated as a noncited violation, consistent with Appendix C of
the NRC Enforcement Policy. This issue is in the licensee's corrective action program
as Condition Report/Disposition Request 1-9-0030.
M8.1
Closed
Ins ection Followu
Item 50-528 -529 -530/9812-05:
Fire potential in the
Emergency Diesel Generator rooms.
The inspections performed an in-office review of this item and determined that no
additional inspection was required since the item was captured by the licensee's
condition reporting program as Condition Report/Disposition Request 98-0267.
E1
Conduct of Engineeriag
E1.1
Year 2000 Y2K Readiness
Units 1 2 and 3
a.
Ins ection Sco
e 37551
On February 24, 1999, the inspectors met with personnel responsible for implementing
the Y2K Readiness
Plan to determine if progress was being made in efforts to cope with
the Y2K computer bug. The inspectors reviewed procedures and documentation of the
program status and held discussions with licensee personnel.
e
-13-
b.
Observations and Findin s
The inspectors determined that the Y2K Readiness
Plan was being implemented by a
dedicated Y2K Project Manager, who reports directly to the Vice President of Nuclear
Engineering.
The Y2K Project Manager was supported by a Y2K Core Team, as well as
other station personnel who were assigned collateral Y2K duties for their departments.
The inspectors determined that the Y2K project had adequate
resources and sufficient
management support to accomplish the Y2K program.
The inspectors reviewed, "Year 2000 Readiness
Plan for the Palo Verde Nuclear
Generating Station," Revision 3. This document defined the roles and responsibilities of
the personnel involved with the Y2K program and contained the guidelines for the
implementation plan. The inspectors determined that the licensee had a well planned
approach to Y2K readiness.
The inspectors verified that the licensee had developed a
contingency plan, although at the time of the inspection, the plan was not finalized.
The inspectors reviewed the completion status of the Y2Kprogram. The inventory,
assessment,
and remediation planning steps were 100 percent complete.
The licensee
was approximately 42 percent complete in remediation of department and
information-technology supported systems and was approximately 94 percent complete
in remediation of embedded systems.
The inspectors determined that the licensee was
well positioned to complete Y2K remediation.
c.
Conclusions
A Y2K Readiness
Plan had been developed and was being implemented by the
licensee.
The plan was organized and contained the necessary elements to address
current and potential problems from the Y2K bug. A Y2K contingency plan had been
developed, but not finalized. The licensee was well positioned to complete Y2K
remediation prior to the end of the year.
E8
Miscellaneous Engineering Issues (92903)
E8.1
Closed
Violation VIO 50-528 50-529 50-530/9617-02:
Failure to Implement Work
Instructions.
This violation involved the failure of plant personnel to reference Piping Installation
Specification 13-PN-304 during installation of a modification to the essential chilled
water system chillers.
The modification was intended to reduce the overcooling of the essential chilled water
system condensers
in the winter, when low essential cooling water temperatures
and
system loading have resulted in low condenser pressure.
During the installation, a
pressure control valve was the final component installed to complete the piping portion
of the modification. The valve was installed using bolted flange connections.
Downstream of the pressure control valve was an elbow, followed by a box-style hanger.
Upstream of the pressure control valve was a piping tee with one end going to the
t
1
I
II
e
-14-
bypass valve and the other end going to a flanged connection to the chiller. The bolted
flange connection, when installed, would have resulted in cold spring of the piping,
causing stress in a hanger and the piping. Specification 13-'PN-204 stated that all
situations involving cold spring should be evaluated by engineering.
Design engineering initiated CRDR 9-6-1371 to evaluate this concern.
An estimate
determined that bolting the flanges would provide roughly 0.1 inches of spring.
The
calculated additional stress that would be added to the hanger, the additional
stress'dded
to the piping, and any impact this condition had on the response to a seismic
event and to expected thermal expansion and contraction corresponded
to roughly half
of the code allowable initial stress limits. The inspectors determined that completion of
corrective actions by the licensee to address CRDR 9-6-1371 were satisfactorily
accomplished.
E8.2
Closed
LER 50-528 -529 -530/97-003:
Construction Design Deficiency Resulted in
Inadequate Protection Against Floodwater Migration
On September 22, 1996, while in Mode 5, an overfillof an SG resulted in flooding on the
100-foot elevation of th'e main steam support structure (MSSS). Inspections of the
80-foot elevation found some electrical junction boxes with water present inside. The
cause of the leakage was determined to be from the submergence
of Erickson fittings in
conduits leading from the 100-foot elevation to the 80-foot elevation in the MSSS.
Investigation by engineering personnel revealed that, unless specifically ordered as
such, Erickson fittings are not designed to be watertight. The licensee further
determined that, with a maximum worst case feedwater line break, a height of 8.65 feet
could be reached in the MSSS. The flooding during the steam generator overfill event
reached a height of approximately 6 inches.
Testing of an Erickson fitting flooded to a
height of 8.65 feet resulted in a leakage rate of 6.8 quarts in 10 minutes.
Engineering
concluded that the Trains A and B AFW pumps would not have been able to perform
their intended safety function if a design basis flooding event had occurred.
Having
determined that the potential existed during a design basis flooding event for water
leakage to migrate to lower elevations through conduit, the licensee initiated
CRDR 1-6-0236 and conducted a survey to identify all possible leakage paths from
conduits throughout the three units. As a result, all identified Erickson fittings below
projected flood heights in the MSSS, diesel generator, auxiliary, and control buildings in
all three units were sealed.
Further, 14 other conduit fittings that required a seal to be
installed were identified. The inspectors determined this review adequately addressed
, the transportability of the design deficiency to all the units and buildings.
Electrical equipment qualification requirements for submergence
are addressed
in
Two guidance documents were referenced by the regulation,
"Guidelines for Evaluating Environmental Qualification of Class 1E Electrical Equipment
in Operating Reactors," November 1979 (DOR Guidelines), and NUREG-0588, "Interim
Staff Position on Environmental Qualification of Safety-Related Electrical Equipment."
10 CFR Part 50, Appendix B, Criteria III, requires the adequacy of design be verified by
-15-
the performance of design reviews. Contrary to these requirements, the licensee had
not verified by design review or by another method, such as testing, the adequacy of
conduit penetrating flooding barriers to resist water intrusion. This resulted in the
potential for rendering the redundant trains of AFW unavailable following a design basis
flooding event in the MSSS.
This Severity Level IVviolation of Criterion Ill is being treated as a noncited violation,
consistent with Appendix C of the NRC Enforcement Policy. This violation is in the
licensee's corrective action program as CRDR 1-60236 (50-528,-529,-530/9904-05).
Conclusions
A violation of Criterion lil was identified for not specifying the correct type of seal fittings
for conduits. As a result, during flooding of a portion of the auxiliary building, water
entered the conduits.
This affected the operability of safety-related equipment.
This
Severity Level IVviolation is being treated as a noncited violation consistent with
Appendix C of the enforcement policy. This issue is in the licensee's corrective action
program as Condition Report/Disposition Request 1-60236.
IV. Plant Su
ort
R1
Radiological Protection and Chemistry Controls
R1.1
General Comments on Radioio ical Protection Controls Units 1 2 and 3
a.
Ins ection Sco
e 71750
The inspectors monitored radiological protection activities during routine site tours.
b.
Observations and Findin s
I
The inspectors observed radiation protection personnel, including supervisors, routinely
touring the radiologically controlled areas.
Licensee personnel working in radiologically
controlled areas exhibited good radiation work practices.
Contaminated areas and high radiation areas were properly posted.
Area surveys
posted outside the room were current. The inspectors checked a sample of doors,
required to be locked for the purpose of radiation protection, and all were in accordance
with requirements.
C.
Conclusions
The radiological protection program was effectively implemented in those areas
reviewed.
w
-16-
V. Mana ement Meetin s
X1
Exit Meeting Summary
The inspectors presented the inspection results to members of licensee's staff at the
conclusion of the inspection on March 25, 1999. The licensee acknowledged the
findings presented.
The inspectors informed the licensee that some material'examined during the inspection
was considered proprietary.
ATTACHMENT
PARTIALLIST OF PERSONS CONTACTED
Licensee
P. Brandjes, Department Leader, Electrical Maintenance Engineering
F. Gowers, Site Representative,
El Paso Electric
J. Hesser, Director, Engineering
R. Henry, Site Representative,
Salt River Project
W. Ide, Vice President, Nuclear Engineering
S. Jones, Section Leader, Engineering Systems
D. Kanitz, Engineer, Nuclear Regulatory Affairs
P. Kirker, Unit 3 Department Leader, Operations
A. Krainik, Department Leader, Nuclear Regulatory Affairs
J. Levine, Senior Vice President, Nuclear
D. Marks, Section Leader, Nuclear Regulatory Affairs
G. Overbeck, Vice President, Nuclear Production
T. Radke, Director, Outages
G. Shanker, Department Leader, Speciality Engineering
D. Smith, Director, Operations
M. Sontag, Section Leader, Nuclear Assurance
E. Sterling, Department Leader, Nuclear Assurance
S. Terrigrino, Manager, Strategic Communications
P. Wiley, Unit 2 Department Leader, Operations
INSPECTION PROCEDURES USED
37551
61726
62707
71707
71750
92901
93702
92903
Onsite Engineering
Surveillance Observations
Maintenance Observations
Plant Operations
Plant Support Activities
Plant Operations Followup
Event Followup
Engineering Followup
~Oened
50-529/9904-01
50-528,-529,
-530/9904-02
50-528,-529,
-530/9904-03
50-528,-529,
-530/9904-04
NCV
NCV
-2-
ITEMS OPENED AND CLOSED
Missed 18-month surveillance of trisodium phosphate baskets
(Section 08.1)
Failure to correct a deficient condition in the auxiliary
feedwater pump governors (Section M3.1)
Failure to provide sufficient instructions for torqueing the EDG
air-start header bolts (Section M3.2)
Inadequate postmaintenance
testing of valve actuators
(Section M4.1)
50-528,-529,
-530/9904-05
50-528,-529,
-530/9904-06
Closed
50-529/98-001
50-530/99-001
50-529/9904-01
50-528,-529,
-530/9904-02
50-528,-529,
-530/9904-03
50-528,-529,
-530/9904-04
50-528, -529,
-530/9812-05
50-528,-529,
-530/9904-05
Failure to followprocedure for identification of correct
replacement parts (Section M4.2)
Failure to conduct an adequate design review of conduit
penetrating flooding barriers (Section E8.2)
LER
Surveillance Test Deficiency Found During QA Audit Lead to
TS 3.0.3/4.0.3 Entry (Section 08.1 )
LER
Loss of Automatic Closure of a CIV (Section 08.2)
Missed 18-month surveillance of trisodium phosphate baskets
(Section 08.1)
Failure to correct a deficient condition in the auxiliary
feedwater pump governors (Section M3.1)
Failure to correct a deficient condition for torqueing the EDG
air-start header bolts (Section M3.2)
Inadequate postmaintenance
testing of valve actuations
(Section M4.1)
IFI
Fire potential in diesel rooms (Section M8.1-)
Failure to followprocedure for identification of correct
replacement parts (Section M4.2)
50-528,-529,
-530/9904-06
50-528,-529,
-530/9617-02
Failure to conduct an adequate design review of conduit
seals in flood areas (Section E8.2)
Failure to Implement Work Instructions for a Modification
(Section E8.1)
50-528,-529,
-530/97-003
LER
Construction Design Deficiency Resulted in Inadequate
Protection Against Floodwater Migration (Section E8.2)
I
1
-3-
LIST OF ACRONYMS USED
CFR
CIAS
CRDR
FCR
ft-Ibs
IFI
LER
Ml
MIWO
MSSS
NRC
RPCB
SBCS
SR
TS
Y2K
Code of Federal Regulations
containment isolation actuation signal
containment isolation valve
condition report/disposition request
control room supervisor
engineered safeguards feature actuation signal
field change request
foot-pounds
high pressure safety injection
inspector followup item
licensee event report
motor control center
maintenance instruction
maintenance instruction work order
main steam support structure
noncited violation
Nuclear Regulatory Commission
public document room
reactor power cutback
surveillance requirement
Technical Specification
violation
work order
Year 2000
J
'L
I
I