ML031390384
| ML031390384 | |
| Person / Time | |
|---|---|
| Site: | Palisades  |
| Issue date: | 05/19/2003 |
| From: | Grant G Division Reactor Projects III |
| To: | Cooper D Nuclear Management Co |
| References | |
| IR-03-005 | |
| Download: ML031390384 (35) | |
See also: IR 05000255/2003005
Text
May 19, 2003
Mr. Douglas E. Cooper
Site Vice President
Palisades Nuclear Plant
Nuclear Management Company, LLC
27780 Blue Star Memorial Highway
Covert, MI 49043-9530
SUBJECT: PALISADES NUCLEAR GENERATING PLANT
NRC SPECIAL INSPECTION REPORT 50-255/03-05
Dear Mr. Cooper:
On April 4, 2003, the U.S. Nuclear Regulatory Commission (NRC) completed a special
inspection at your Palisades Nuclear Generating Plant to review the circumstances surrounding
two Alert emergency declarations on March 18, and March 25, 2003. The enclosed report
documents the inspection findings which were discussed on April 4, 2003, with you and other
members of your staff.
The inspection examined activities conducted under your license as they relate to safety and
compliance with the Commissions rules and regulations and with the conditions of your license.
The team reviewed selected procedures and records, observed activities, and interviewed
personnel.
On March 18, 2003, a fire started in a safety-related breaker cubicle in the cable spreading
room. An emergency plan Alert (the second lowest of four emergency classification levels)
condition was declared due to a fire that could impact safety-related equipment. Your staff
secured from the Alert after about 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> when the fire was extinguished and the source of the
fire was identified.
On March 25, 2003, plant maintenance workers were installing signposts in the parking lot to
designate parking spaces. One of the signposts was driven into a conduit and damaged a
cable which contained protective relay circuitry for all sources of offsite power. An Alert was
declared due to the loss of offsite power combined with the loss of shutdown cooling. The Alert
was downgraded to an Unusual Event (the lowest of four emergency classification levels) after
about 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> when shutdown cooling was restored. Your staff secured from the Unusual Event
on March 27, 2003, when offsite power was restored.
Both of these events could have been avoided. Inadequate breaker maintenance procedures
coupled with a number of human performance errors in the operations and maintenance areas
resulted in the March 18, 2003, cable spreading room fire. In addition, a weakness in the
implementation of your corrective action program was a primary contributor to the
March 25, 2003, loss of offsite power event. Both the human performance and problem
identification and resolution areas were previously identified as substantial cross-cutting issues
D. Cooper -2-
and were discussed in our March 4, 2003 annual assessment letter to you. The human
performance substantial cross-cutting issue was closed in our letter since no findings involving
human performance had been identified since the first quarter of the 2002 assessment period.
However, the human performance errors which occurred on March 18, 2003 indicates that this
area may again be a challenge which warrants your immediate attention. The problem
identification and resolution area was a substantial cross-cutting area which remained open in
our letter since improvements in this area had only recently been implemented. The
March 25, 2003 event indicates that these improvements have not been fully effective and that
further improvements are necessary before this area can no longer be considered a substantial
cross-cutting issue.
Based on the risk and deterministic criteria specified in Management Directive 8.3, NRC
Incident Investigation Program, and Inspection Procedure 71153, Event Followup, and due to
the equipment performance problems which occurred, a Special Inspection was initiated in
accordance with Inspection Procedure 93812, Special Inspection, to evaluate the facts and
circumstances surrounding the events as well as the actions taken by your staff in response to
the unexpected system performance issues encountered. The inspection focused on: (1) the
sequence of events for each Alert; (2) the adequacy of your evaluation of the events and
corrective actions; (3) any common causes or relationship between the two events; (4) the
operational performance issues associated with the repeated attempts to restart charging pump
P-55A; (5) any equipment performance issues during the two events; (6) maintenance
performance issues associated with the missing arc chutes for the charging pump P-55A
breaker; (7) design issues associated with the offsite power configuration and any changes to
address the event; and (8) the emergency plan actions to address the events.
Based on the results of this inspection, two self-revealed findings of very low safety significance
which involved violations of NRC requirements were identified. However, because these
violations were non-willful and non-repetitive and because they were entered into
your corrective action program, the NRC is treating these findings as Non-Cited Violations in
accordance with Section VI.A.1 of the NRCs Enforcement Policy.
The NRC identified one finding for which the final risk significance remains to be determined at
a later date. The finding concerned the failure of site management to take adequate corrective
actions after a series of events during digging and excavating on station property between the
protected area and the switchyard. This finding did not present an immediate safety concern
because compensatory measures were put in place while long-term corrective actions were
being determined and implemented.
If you contest the subject or severity of a Non-Cited Violation, you should provide a
response within 30 days of the date of this inspection report, with the basis for your denial, to
the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington DC
20555-0001; with copies to the Regional Administrator, U.S. Nuclear Regulatory Commission -
Region III, 801 Warrenville Road, Lisle, IL, 60532-4351; the Director, Office of Enforcement,
U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident
Inspector at the Palisades facility.
D. Cooper -3-
In accordance with 10 CFR 2.790 of the NRCs "Rules of Practice," a copy of this letter
and its enclosure will be made available electronically for public inspection in the NRC Public
Document Room or from the Publicly Available Records (PARS) component of NRCs
document system (ADAMS). ADAMS is accessible from the NRC Web site at
http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).
Sincerely,
/RA/
Geoffrey E. Grant, Director
Division of Reactor Projects
Docket No. 50-255
License No. DPR-20
Enclosure: Inspection Report 50-255/03-05(DRP)
cc w/encl: J. Cowan, Executive Vice President
and Chief Nuclear Officer
R. Fenech, Senior Vice President, Nuclear
Fossil and Hydro Operations
L. Lahti, Manager, Regulatory Affairs
J. Rogoff, Esquire, Nuclear Management Company, LLC
A. Udrys, Esquire, Consumers Energy Company
R. Remus, Plant Manager
D. Malone, Site Director
S. Wawro, Nuclear Asset Director, Consumers Energy Company
W. Rendell, Supervisor, Covert Township
Office of the Governor
Michigan Department of Environmental Quality -
Waste and Hazardous Materials Division
Department of Attorney General (MI)
DOCUMENT NAME: C:\ORPCheckout\FileNET\ML031390384.wpd
To receive a copy of this document, indicate in the box: "C" = Copy without attachment/enclosure "E" = Copy with attachment/enclosure "N" = No copy
OFFICE RIII RIII RIII RIII
NAME EDuncan for EDuncan SReynolds for
CPhillips:dtp GGrant
DATE 05/16/03 05/16/03 05/19/03
OFFICIAL RECORD COPY
D. Cooper -4-
ADAMS Distribution:
WDR
JHE
RidsNrrDipmIipb
GEG
JAL3
C. Ariano (hard copy)
DRPIII
DRSIII
PLB1
JRK1
U.S. NUCLEAR REGULATORY COMMISSION
REGION III
Docket No: 50-255
License No: DPR-20
Report No: 50-255/03-05
Licensee: Nuclear Management Company, LLC
Facility: Palisades Nuclear Generating Plant
Location: 27780 Blue Star Memorial Highway
Covert, MI 49043-9530
Dates: March 24 through April 4, 2003
Inspectors C. Phillips, Senior Operations Engineer, Team Leader
J. Lara, Senior Resident Inspector, Kewaunee
C. Brown, Resident Inspector, Clinton
J. Lennartz, Senior Resident Inspector, Palisades
R. Krsek, Resident Inspector, Palisades
H. Gonzales, Nuclear Safety Intern
Approved by: Geoffrey E. Grant, Director
Division of Reactor Projects
SUMMARY OF FINDINGS
IR 05000255-03-05; Nuclear Management Company; 03/24 - 04/04/2003; Palisades Nuclear
Generating Plant; Special Inspection - March 18, 2003, Alert due to cable spreading room fire
and March 25, 2003, Alert due to loss of offsite power and temporary loss of shutdown cooling.
This report covered a 2-week period of special inspection by Region III inspectors and resident
inspectors. Two Green findings and one finding with a significance which is yet to be
determined were identified. The significance of most findings is indicated by their color (Green,
White, Yellow, Red) using Inspection Manual Chapter (IMC) 0609, Significance Determination
Process (SDP). Findings for which the SDP does not apply may be Green or be assigned a
severity level after NRC management review. The NRCs program for overseeing the safe
operation of commercial nuclear power reactors is described in NUREG-1649, Reactor
Oversight Process, Revision 3, dated July 2000.
A. NRC-Identified and Self-Revealed Findings
Cornerstone: Initiating Events
- Green. A finding of very low safety significance was self-revealed during an event when
an operator failed to adhere to a procedure for operating the chemical volume control
system and repeatedly attempted to close a charging pump breaker after the breaker
tripped. In addition, the operator failed to trip primary coolant pumps before primary
coolant system pressure dropped below the minimum pressure for primary coolant
pump operation. The primary cause of this finding was related to the cross-cutting area
of Human Performance.
The finding was more than minor because it could be reasonably viewed as a precursor
to a significant event. The repeated operation of an electrical breaker contrary to
procedural requirements was a contributing cause to the March 18, 2003, cable
spreading room fire. The finding was determined to be of low safety significance
because the failure to follow the procedure did not result in a loss of shutdown cooling or
loss of reactor inventory. This issue was determined to be a Non-Cited Violation of
Technical Specification 5.4.1, which required the implementation of written procedures
covering the chemical volume control system and the reactor coolant system.
(Section 04.1)
- Green. A finding of very low safety significance was self-revealed during an event when
the licensee failed to have adequate maintenance procedures in place to ensure that
when an electrical breaker was removed to be refurbished, that the arc chutes were
reinstalled before the breaker was placed back in service.
The finding was more than minor because it could be reasonably viewed as a precursor
to a significant event since a fire resulted in the P-55A charging pump breaker when the
arc chutes were not reinstalled after the breaker had been refurbished. The finding was
determined to be of low safety significance because the failure to follow the procedure
did not result in a loss of shutdown cooling or loss of reactor inventory. This issue was
determined to be a Non-Cited Violation of 10 CFR 50, Appendix B, Criterion V,
Instructions, Procedures, and Drawings. (Section 07)
- TBD. The licensee failed to take effective corrective actions to address a series of
events involving digging and excavating between the protected area and the switchyard.
The finding was more than minor because it could be reasonably viewed as a precursor
to a significant event in that a maintenance technician drove a signpost into the ground
and damaged an electrical cable that resulted in a loss of offsite power and a loss of
shutdown cooling. No violation of regulatory requirements was identified since the act of
driving the signpost into the ground was not an activity affecting quality. (Section 02.2)
B. Licensee-Identified Violations
No findings of significance were identified.
2
REPORT DETAILS
Summary of Plant Events
Synopsis of Events
On March 18, 2003, a fire started in a safety-related breaker cubicle in the cable spreading
room. Although the fire did not spread beyond the cubicle, the room filled with smoke. The
licensee declared an Alert based on the existence of a fire that could affect safety-related
equipment. The licensee secured from the Alert about 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> later when it was determined
that a threat no longer existed.
On March 25, 2003, while shutdown for a refueling outage, a maintenance worker installing a
signpost in the parking lot struck an underground cable and cut and shorted together
conductors for protective circuitry that impacted circuit breakers for all sources of offsite power.
The licensee declared an Alert based on the complete loss of offsite power and the loss of
shutdown cooling. About 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> after the Alert declaration, the licensee downgraded the event
to an Unusual Event based on the recovery of shutdown cooling with both emergency diesel
generators supplying power to the safety-related buses. The licensee secured from the
Unusual Event on March 27, when power from a qualified offsite power source was restored to
the safety-related buses.
March 18, 2003 Cable Spreading Room Fire
On March 18, 2003, the plant was in Mode 5 (Cold Shutdown) and in a solid plant condition.
Two charging pumps were running at maximum speed with maximum letdown flow to expedite
the chemical cleanup of the primary coolant to reduce the coolant activity levels in preparation
for the refueling outage. The night shift operating crew was assigned to increase the
concentration of boric acid in the primary coolant by gravity feeding boric acid solution from the
B boric acid storage tank to the suction of the two running charging pumps. After the
operators realigned the boric acid suction path from the volume control tank to the B boric
acid storage tank, the P-55A charging pump breaker tripped. An operator attempted to restart
the P-55A charging pump three times. Arcing within the P-55A charging pump breaker caused
the failure of the wiring connections to the breaker anti-pump relay. Therefore, the breaker
attempted to re-close more times than the number of times the operator manipulated the
breaker control switch.
The rapid opening and closing of the breaker caused an arc to form within the breaker cubicle.
The breaker design included an arc chute installed on each phase to extinguish any arc that
may form upon breaker opening. However, due to the combination of an inadequate
maintenance procedure and human performance errors when the breaker was previously
refurbished, the P-55A charging pump breaker did not have the arc chutes installed. The
absence of the arc chutes resulted in a phase-to-phase arc forming which increased in
magnitude and ultimately damaged the breaker. The arc caused the cable spreading room to
fill with smoke.
3
The arc current caused the breaker supplying power to load control center (LCC)-12 to trip to
isolate the fault. The fire brigade responded promptly and the problem was quickly brought
under control.
Shutdown cooling continued to operate during the event. The primary coolant system
depressurized and due to human performance errors, the primary coolant pumps were not
secured before plant pressure dropped below the minimum pressure for primary coolant pump
operation.
March 25, 2003 Loss of Offsite Power
On March 25, 2003, with the plant in Mode 6 (Refueling), plant maintenance workers were
installing signs in the parking lot designating parking spaces. One of the signposts was driven
into a conduit and damaged a cable which contained a combination of energized indication
circuitry and de-energized protective relay circuitry. The metal signpost cut and shorted
together several of the conductors within the cable generating a fault signal to the breakers
supplying offsite power to the 345 kilovolt (kV) Rear (R) bus.
The R bus was supplying power to nonsafety-related loads through the startup transformer.
About 30 seconds later, the breaker between the safeguards transformer and the safeguards
bus tripped open resulting in a temporary loss of power to the safety-related buses. The
emergency diesel generators started and energized the safety-related 1C and 1D buses.
Shutdown cooling was lost, but was restored after about 20 minutes when the emergency
diesel generators started and shutdown cooling pumps were re-energized.
An Alert was declared based on the loss of offsite power combined with the loss of shutdown
cooling. The Alert was subsequently downgraded to an Unusual Event after shutdown cooling
was restored. A temporary modification was installed which re-routed the conductor for the
protective relaying of the startup transformer. The licensee secured from the Unusual Event on
March 27, 2003, when offsite power was restored.
Inspection Scope
Based on the risk and deterministic criteria specified in Management Directive 8.3, NRC
Incident Investigation Program, and Inspection Procedure 71153, Event Followup, and due to
the equipment performance problems which occurred, a Special Inspection was initiated in
accordance with Inspection Procedure 93812, Special Inspection.
The purpose of the inspection was to evaluate the facts and circumstances surrounding the
events as well as the actions taken by licensee personnel in response to the unexpected
system performance issues encountered. In particular, the inspection focused on the following:
(1) the sequence of events for each Alert; (2) the adequacy of the licensees evaluation of the
events and corrective actions; (3) any common causes or relationship between the two events;
(4) the operational performance issues associated with the repetitive attempts to restart
charging pump P-55A; (5) any equipment performance issues during the two events; (6)
maintenance performance issues associated with the missing arc chutes for the charging pump
P-55A breaker; (7) design issues associated with the offsite power configuration and any
changes as a result of the event; and (8) the emergency plan actions to address the events.
4
1 REACTOR SAFETY
Cornerstones: Initiating Events and Mitigating Systems
01 Sequence of Events (93812)
a. Inspection Scope
The inspector reviewed logs, alarm printouts, and other documentation; interviewed
licensee personnel; and developed the following sequence of events for the March 18,
2003, cable spreading room fire and the March 25, 2003, loss of offsite power:
March 18, 2003 Cable Spreading Room Fire
Day Time Event Description
3/18 19:00 The plant was in Mode 5, water solid condition with primary
coolant system (PCS) temperature at 120 degrees
Fahrenheit (EF). A chemical soak of the PCS was in progress to
reduce activity levels. Charging pumps P-55A and P-55B were
running with increased letdown flow to aid in PCS cleanup.
Primary Coolant Pumps (PCPs) 1A and 1D were running.
The operating crew made plans to increase boron concentration
in the PCS to that required for refueling activities by realigning the
suction of the charging pumps from the volume control tank (VCT)
to Boric Acid Storage Tank (BAST) T-53B.
20:36 A nuclear control operator (NCO-1) realigned the charging pump
suction from the VCT to BAST T-53B. NCO-1 unexpectedly
observed that BAST level did not change and PCS pressure was
decreasing. NCO-1 attempted to reduce letdown flow by closing
the controller for the letdown backpressure regulator, however
PCS pressure continued to decrease. NCO-1 noticed that
charging flow had decreased from about 107 gallons per minute
(gpm) to about 40 gpm and that charging pump P-55A had
tripped. Licensee personnel later determined that when the
suction source to the charging pump was changed, the suction
pressure decreased. The charging pump low suction pressure
switch had a time delay, but the switch had stuck in the tripped
position.
NCO-1 did not announce the charging pump trip or any other
actions, either before or after they were taken. A second
operator, NCO-2, announced the trip of charging pump P-55A.
NCO-1 attempted to restart P-55A unsuccessfully, but failed to
verify the on/off indication lights for the pump on the control panel.
NCO-1 cycled opened and closed the suction valve to the BAST
5
and again attempted to restart charging pump P-55A. During this
attempt, NCO-1 held the control switch closed for a longer period
under the belief that the control switch had not been closed long
enough during the previous attempt, but again did not visually
verify the pump on/off indication lights. NCO-2 observed that the
charging pump on/off indication lights cycled three or four times.
NCO-1 attempted to start charging pump P-55A unsuccessfully a
third time.
The control room supervisor (CRS) and shift supervisor (SS)
noticed the charging pump low suction pressure alarm and
increased activity at the control panel. The SS noticed that PCS
pressure had decreased below the minimum pressure for PCP
operation and directed the CRS to trip the PCPs. The CRS then
made a statement to trip the PCPs, but did not specifically direct
NCO-1 to trip the pumps. NCO-1 did not hear the statement by
the CRS and therefore did not repeat it back. The CRS was
distracted by a fire alarm from the cable spreading room and did
not ensure his direction to trip the PCPs was carried out. The SS
observed that the direction to trip the PCPs was not carried out
and directed NCO-1 to trip the PCPs. The NCO-1 then tripped
the PCPs. This entire sequence of events occurred over about 37
seconds.
Decreasing PCS pressure required the trip of the PCPs at 235
pounds per square inch absolute (psia) per Standard Operating
Procedure (SOP)-1, Primary Coolant System. However, the A
and D PCPs were not secured until reactor pressure was about
135 psia.
Off Normal Operating Procedure (ONP)-25.1, Fire in Safety
Related Area, was entered due to the cable spreading room fire
alarm.
20:39 The fire brigade was dispatched to respond to the cable spreading
room fire alarm.
20:47 An Alert was declared per the station Site Emergency Plan due to
a fire with the potential to affect safety-related equipment.
22:20 Procedure ONP-25.1 was exited when the fire brigade determined
that the fire was no longer present.
22:38 The Alert was terminated after the licensees investigation
determined that the fire was extinguished and was isolated to the
circuit breaker for charging pump P-55A.
6
March 25, 2003 Loss of Offsite Power
Day Time Event Description
3/25 11:15 The plant was in Mode 6. The 345 kilovolt (kV) switchyard
front (F) bus was supplying power to the safeguards transformer
and through Breaker 152-401 to the safeguards bus (see the last
page of this report for a simplified diagram of the electrical
distribution system). All safety-related 2400 volt alternating
current (Vac) buses were energized from the safeguards bus.
The 345 kV switchyard rear (R) bus was supplying power to the
startup transformer, which was available as a backup power
source to the safety-related 2400 Vac buses. All 4160 Vac buses
and nonsafety-related 2400 Vac buses were energized from
startup transformer feeder breakers. The main generator output
breakers were closed. Motor-operated disconnect M26H5 was
open, interrupting backfeed power to the main transformer. The
reactor vessel head was removed and the reactor cavity was
partially flooded to about 6 feet above the reactor vessel flange.
Nozzle dams were installed in the reactor coolant loops. Service
air was supplied to the nozzle dams to prevent leakage from the
reactor vessel to the coolant loops. Primary coolant system
temperature was 94EF.
11:16 Building and grounds workers replaced two signposts for parking
signs alongside the site access road. The signposts were
installed by pounding them directly into the ground using an
impact tool. The second signpost installed was inadvertently
driven into a buried conduit and shorted together protective relay
conductors. This caused the protective relays to actuate and
resulted in a loss of offsite power and a loss of shutdown cooling.
The control room received numerous alarms indicating that all
switchyard circuit breakers feeding the R bus had opened.
Shortly thereafter, Breaker 152-401 also opened, resulting in a
loss of offsite power.
Operators entered ONP-2.1, Loss of AC [Alternating Current]
Power, and ONP-17, Loss of Shutdown Cooling.
Both emergency diesel generators (EDGs) started and closed
onto their respective buses. Service air was initially lost and was
subsequently realigned to backup nitrogen bottles to provide air to
the nozzle dams.
11:21 Control room operators verified that the 1C and 1D buses were
re-energized from their respective EDGs.
7
11:26 The shift supervisor declared an Alert per the Site Emergency
Plan for the loss of offsite power and loss of shutdown cooling
event.
11:36 Control room operators restored shutdown cooling to service.
Primary coolant system temperature increased from 94EF to
104EF while shutdown cooling was unavailable.
11:57 Control room operators stabilized PCS temperature. The highest
PCS temperature reached was 104EF.
12:06 Control room operators authorized re-opening containment
penetration M7-10 service air to restore air to the nozzle dams.
12:22 Operators restored the service air lineup to containment.
12:31 The site emergency director downgraded the emergency
classification from an Alert to an Unusual Event with shutdown
cooling restored and the EDGs providing power to safety-related
buses 1C and 1D.
13:37 Control room operators exited ONP-17 and continued actions to
restore offsite power.
14:25 Operators reset relay 486 S-X1 in order to re-energize the R bus.
Restoration of the R bus was necessary in order to restore offsite
power transmission line reliability.
14:27 Operators opened disconnect 24R2 to isolate the startup
transformer from the R bus. Control room operators then restored
the R bus. Control room operators did not restore power to the
startup transformer because the cause of the loss of offsite power
remained unknown.
17:30 Operators closed Breaker 152-401 and energized non-safeguards
loads. The cause of Breaker 152-401 to trip remained unknown.
However, licensee management concluded that there was no
existing electrical bus or breaker fault and closed
Breaker 152-401 to supply power to nonsafety-related bus 1E.
The EDGs continued to supply power to safety-related buses 1C
and 1D.
3/27 5:39 The Shift Supervisor authorized the installation of temporary
modification TM-2003-012 for R bus relay 486 S-X1. This
temporary modification replaced conductors associated with
R Bus relay 486 S-X1 with wires in another cable. This allowed
for the realignment of offsite power to safety-related buses 1C
and 1D through the startup transformer.
8
14:47 Operators energized nonsafety-related buses 1E, 1B, 1A, 1G and
1F from the startup transformer.
15:30 Operators paralleled the startup transformer to EDG 1-1 to supply
power to the 1C bus.
15:46 Emergency diesel generator 1-1 was secured.
17:23 Operators paralleled the startup transformer to EDG 1-2 to supply
power to the 1D bus.
17:35 Operators exited ONP-2.1.
17:37 Emergency diesel generator 1-2 was secured. The Unusual
Event was terminated following the restoration of offsite power.
4/4 6:13 Work Order (WO) 24321140, implementing engineering action
request (EAR) 2003-0086, was completed. Control circuits for
offsite power protective relays were transferred to alternate
conductors and the offsite power protective relays were fully
restored.
b. Findings
No findings of significance were identified.
02 Adequacy of Licensee Evaluation of Events and Corrective Actions (93812)
02.1 March 18, 2003 Cable Spreading Room Fire
a. Inspection Scope
On March 18, 2003, charging pump breaker 52-1205 failed and caught fire. The
breaker was contained in LCC-12 in the cable spreading room. The site emergency
director declared an Alert due to the potential impact of the fire on other safety-related
equipment. The team assessed the licensees root cause investigation efforts for this
event. The team interviewed control room operators, electrical maintenance workers
and licensee personnel involved with the investigation. As part of the inspection effort,
the team reviewed the licensees root cause investigation results and performed
independent reviews of the licensees troubleshooting activities. The team also
reviewed design drawings, design basis documents, and the Updated Final Safety
Analysis Report. The team reviewed maintenance controls practices including previous
breaker maintenance and calibration activities.
b. Findings
No findings of significance were identified.
9
The team determined that the licensees troubleshooting efforts were structured and
methodical. A failure mode analysis approach was used to establish a timeline and
potential failure modes chart. This approach facilitated the structured review of potential
failure modes for both the charging pump P-55A supply breaker as well as the low
suction pressure trip of charging pump P-55A which initiated the event. Licensee
personnel demonstrated a focus on gathering as-found data for the sequence of events,
maintenance records, pressure transmitters, operator actions, and operator statements.
Additionally, the licensee performed an extensive as-found investigation into the extent
of damage to LCC-12 that housed the failed breaker. Similarly, the licensee performed
a rigorous extent of condition review to determine if any other breakers were missing arc
chutes and whether problems encountered with the charging pump low suction pressure
trip switches were isolated or generic.
At the end of the inspection, the licensees root cause investigation was not complete
and therefore corrective actions were not yet formalized. The licensee identified that the
cause the cable spreading room fire was the failure to reinstall arc chutes in the P-55A
breaker when the breaker was reinstalled into LCC-12 after it was refurbished in
May 2002. Following the event, the licensee sent the breaker to the manufacturer for a
complete failure mode analysis. The manufacturer identified that the breaker failure
was caused by an arc forming between phases on the breaker and then moving over
the backing insulator to form an arc directly between the breaker stabs entering LCC-12.
The missing arc chutes resulted in a normal arc, formed when a breaker opens, not
being extinguished as designed. A combination of the operator attempting multiple
motor starts in a short time, a stuck low suction pressure trip switch, and burnt wires to
the anti-pump coil in the breaker, caused the breaker to rapidly close and trip open
multiple times when the operator held the control switch in the close position. This
allowed a continuous arc to form, and initiated the fire in the breaker cubicle.
The team concluded that although the root cause investigation was not complete at the
end of the inspection, the licensees evaluation of the event was proceeding
satisfactorily and the root cause investigation efforts were adequate.
02.2 March 25, 2003 Loss of Offsite Power
a. Inspection Scope
On March 25, 2003, the plant experienced a loss of offsite power to the nonsafety-
related and safety-related buses providing power to equipment necessary to maintain
shutdown cooling of the PCS. The team reviewed the licensees root cause
investigation results and performed independent reviews of the licensees offsite power
system design and troubleshooting activities. The team also reviewed design drawings,
design basis documents, and the Updated Final Safety Analysis Report. The team also
interviewed licensee personnel involved with the root cause investigation.
10
b. Findings
Introduction
The team identified a performance deficiency in that the licensee failed to address a
repetitive problem of weak controls over excavation and digging activities. The finding is
greater than minor, but is unresolved pending completion of a significance determination
review. No violation of regulatory requirements was identified.
Description
The licensee used a failure mode analysis approach to establish a timeline and potential
failure modes chart. This approach facilitated the structured review of potential failure
modes, which could have resulted in the loss of offsite power. The licensee focused on
gathering as-found data, and identified the circuit breakers and relays that were
impacted. The licensee reviewed all potential failure modes to identify the likely cause
of the event. The licensee identified several conductors within a common cable which
were routed to various relays that could have caused the loss of offsite power to occur.
The licensee identified that a single cable carried protective relay circuitry for all the
affected breakers. Through interviews and information provided by plant personnel, the
licensee identified that the event was caused by cables being damaged during the
installation of a signpost in a parking lot outside the facilitys protected area. The metal
signpost breeched the plastic conduit wall and penetrated the outer jacket and insulation
of one of the cables routed in the conduit. This conduit was routed between the plant
and the switchyard about 30 inches underground. The damaged cable carried the
conductors associated with the protective relaying scheme for the offsite power system.
The damaged cable actuated protective relaying associated with the switchyard. These
actuations resulted in the opening of various circuit breakers including Breaker 152-401,
main generator breakers, and all four circuit breakers supplying the switchyard R bus.
Collectively, the opening of these circuit breakers removed all offsite power to the plant .
However, there were two instances during the licensees recovery efforts that should
have received more careful consideration. First, licensee personnel made the decision
to close Breaker 152-401 prior to completing a physical inspection of the breaker. At
the time of this decision, the root cause for the breaker trip was unknown. Licensee
personnel determined there was no clear indication of an existing electrical fault and
decided to close the breaker and re-energize nonsafety-related buses from the
safeguards bus. The team concluded that although there was no obvious indication of
an electrical fault, more consideration should have been given to the possibility of a
mechanical failure. The team concluded that licensee management had missed an
opportunity to identify a mechanical problem within the breaker, had one existed.
Second, when the licensee determined that the loss of offsite power occurred at the
same time a signpost was driven into the ground, it was also known that two signposts
were involved. The licensee had excavated the first signpost hole and found minor
damage to an underground conduit. The licensee had also developed a temporary
modification (TM-2003-012) to relocate the protective relay conductors. The licensee
planned to move ahead with the implementation of the temporary modification before
excavation of the second signpost hole. Regional NRC management questioned why
11
the second signpost hole was not excavated prior to proceeding with the temporary
modification. The licensee then identified that it was the second signpost that caused
the damage to the protective relay circuitry cable.
Licensee personnel identified that a contributing cause to the loss of offsite power to the
safety-related buses was the use of a common cable to provide controls for the R bus
load shed relays as well as the F bus load shed/fast transfer relays. Part of the
licensees corrective actions included separating the conductors for protective relays to
different cables.
The licensees root cause investigation assessed the adequacy of existing plant policy
regarding excavation and digging activities within the licensees property. Similarly the
team performed independent reviews of the licensee administrative controls in this area.
During those reviews, the team independently identified that there were several previous
corrective action program (CAP) documents pertaining to problems encountered during
excavating or digging activities. Documents reviewed by the team included the
following:
- CAP 19522, Unidentified Cable Severed During Excavation, dated June 1999;
- CAP 08000, Unknown Cables Cut While Digging to Install Fire Protection Piping,
dated August 2000;
- CAP 26496, Equipment Operator Damaged an Underground Telephone Cable
While Excavating, dated October 2000;
- CAP 14634, During Excavation Work Digging Equipment Damaged Two Plastic
Conduits, dated July 2001;
- CAP 30724, Severed Power Cable to the Meteorological Tower, dated
May 2002;
- CAP 31300, Underground Cable Hit by Municipal Project Resulting in Power
Outage, dated September 2002; and
- CAP 31378, Unmarked Phone Line Cut During Installation of Domestic Water,
dated September 2002.
Based on discussions with plant personnel and a review of corrective actions associated
with the above CAP documents, the team concluded that the licensee had failed to
address a repetitive problem of weak controls of excavation and digging activities. The
lack of established controls in the form of administrative policies and procedures in this
area contributed to the lack of awareness and sensitivity to potential safety
consequences which could arise during such activities. The licensees preliminary root
cause evaluation similarly concluded that the root cause of the event was that the plant
did not have a written policy or process for excavating and trenching activities.
12
The team concluded that although the root cause investigation was not complete at the
end of the inspection, the licensees evaluation of the event was proceeding
satisfactorily and the root cause investigation efforts were adequate.
Analysis
The team determined that the licensees failure to develop and implement corrective
actions for a repetitive problem of excavating and digging activities damaging buried
components was a performance deficiency warranting a significance evaluation. The
team concluded that the finding was of greater than minor risk significance in
accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue
Disposition Screening. This conclusion was based on the licensees failure to develop
and implement corrective actions for several previous similar events. When the loss of
offsite power event occurred the plant was shutdown with the reactor vessel head
removed and the reactor cavity flooded to about 6 feet above the reactor vessel flange.
Inspection Manual Chapter 0609, Significance Determination Process, Appendix A,
SDP Phase 1 Screening Worksheet for IE [Initiating Events], MS [Mitigating Systems],
and B [Barrier Integrity] Cornerstones, states that if the finding is assumed to degrade
the safety of a shutdown reactor then use Appendix G, Shutdown Operations
Significance Determination Process. Utilizing IMC 0609, Appendix G worksheet, PWR
[Pressurized Water Reactor] Cold Shutdown and Refueling Operation RCS [Reactor
Coolant System] Open and Refueling Cavity Level <23 Feet, the team determined this
was a finding that increased the likelihood of a loss of offsite power and therefore
required a Phase 2 analysis. In addition, a Phase 3 analysis was required since the
potential for this event to occur also existed when the plant was operating at power.
This is an Unresolved Item (URI 50-255/03-05-01) pending the completion and review of
these analyses. The licensee entered this issue into their corrective action program as
CAP 034500.
Enforcement
The maintenance technician driving the signpost into the ground was not performing an
activity affecting quality. Therefore, no violation of regulatory requirements was
identified.
03 Event Common Cause Review and Assessment (93812)
a. Inspection Scope
The team interviewed individuals involved in both events, and reviewed pertinent logs,
information, and procedures to identify any common causes or relationships between
the two events.
b. Findings
No findings of significance were identified.
The team did not identify any common causes or relationship between the two events.
13
04 Operator Performance Issues (93812)
04.1 March 18, 2003 Cable Spreading Room Fire
a. Inspection Scope
The team reviewed operator performance during the March 18, 2003, cable spreading
room fire through interviews of the on-shift operating crew, a simulator training
instructor, and licensee management. The team also reviewed operating logs,
operating procedures, abnormal operating procedures, alarm response procedures,
annunciator alarm printouts, system drawings, and operator training records.
b. Findings
Introduction
A Green finding associated with a self-revealed event was identified when a control
room operator repeatedly attempted to start charging pump P-55A and failed to trip the
Primary Coolant Pumps when required. One Non-Cited Violation of Technical
Specification 5.4.1 was identified.
Description
On March 18, 2003, the plant was in Mode 5 (Cold Shutdown) and in a solid condition
with charging and letdown flow increased to aid in PCS cleanup. Operators were
directed to increase the boric acid concentration in the PCS by realigning the charging
water suction from the VCT to BAST T-53B.
Prior to the evolution, the CRS and NCO-1 discussed the activity, but did not involve the
operating crew in a crew briefing. The CRS did not use a pre-job checklist during the
discussion, nor did NCO-1 review or discuss the procedure requirements in SOP-2A,
Chemical and Volume Control System. Instead, with the CRSs knowledge, NCO-1
used a placard intended for use during emergency boration as guidance to perform the
evolution. Since the valves to be operated were the same as those identified in the
placard, the CRS and NCO-1 saw no immediate problem.
Operators calculated that 67.4 gallons of high concentration boric acid solution from the
BAST were required to be added to raise PCS boron concentration to the desired level.
However, operators did not calculate how long this evolution should require or recognize
that the computer screen used to monitor BAST level updated only every 10 seconds.
NCO-1 initiated actions to borate the PCS by shifting the charging pump suction from
the VCT to BAST T-53B. The CRS performed a peer check of the valve manipulations
and then proceeded with other duties. NCO-1 turned away to verify a decrease in BAST
level, but did not observe an immediate change. NCO-1 then turned his attention back
to the control panel and noted that PCS pressure was unexpectedly rapidly decreasing.
Operations management expectations were that operators announce problems and
obtain supervisory approval prior to operating equipment unless the action is an
immediate action within a procedure, which was not the case for this situation.
14
However, without informing anyone else in the control room about the rapidly decreasing
PCS pressure, NCO-1 attempted to arrest the pressure drop by adjusting the
backpressure regulator to reduce the letdown rate from the PCS. As PCS pressure
continued to decrease, NCO-1 again attempted to adjust the backpressure regulator
without verifying charging pump flow. Charging pump P-55A tripped on low suction
pressure about 40 seconds after the VCT discharge valve was closed. At that point, a
second operator, NCO-2, noted that the charging pump had tripped and announced the
trip. NCO-1, without supervisor approval, then reopened the VCT discharge valve, shut
the BAST suction valve, and repeatedly attempted to restart charging pump P-55A. In
9 seconds, NCO-1 unsuccessfully attempted to start pump charging pump P-55A three
times.
Step 5.2.3 of Procedure SOP-2A limited the attempted starts of P-55A to one attempt.
Step 7.5.5.f of SOP-2A required verifying charging flow after shifting to gravity feed.
The licensees investigation later revealed that P-55A charging pump Breaker 52-1205
had no arc chutes installed. In addition, the licensees investigation identified that the
breakers anti-pump relay had its wires burned off, likely from an arc either before or
during an earlier attempt to close the breaker and start the pump. The licensees
investigation also identified that the pump low suction pressure trip switch had stuck
closed so that the low suction pressure trip was locked in. With the anti-pump relay
damaged and the breaker trip signal locked in, the actions of NCO-1 to hold the breaker
control switch in the close position resulted in electrical arcing within the breaker.
Without the arc chutes installed, a fire started within the charging pump P-55A breaker
cubicle. The arc was extinguished when the upstream supply breaker to LCC-12,
Breaker 52-1202, opened as designed on overcurrent.
With NCO-1 attempting to restart the charging pump, PCS pressure continued to
decrease. Neither NCO-1 nor the CRS noticed that PCS pressure was trending toward
the minimum pressure for PCP operation. The SS did notice the trend in PCS pressure
and directed the CRS to trip the PCPs. The CRS made a statement to trip the PCPs,
but did not use effective three-way communications to ensure that NCO-1 understood
and complied with the direction. NCO-1 did not hear the direction given. As a result,
the PCPs were not tripped until the SS directed NCO-1 to trip the PCPs. Consequently,
PCS pressure decreased to below the minimum operating pressure for PCP operation,
a condition that led to minor damage to the 1D PCP seal.
The other running charging pump, P-55B, tripped when LCC-12 bus supply
breaker 52-1202 tripped on high current. Operators verified that adequate shutdown
margin existed, that shutdown cooling remained in operation, and that PCS cooldown
rates had not been exceeded. Operators secured PCS letdown and controlled bleedoff
flow to maintain primary coolant system inventory.
Analysis
The team determined that the control room operator failed to follow procedure
requirements by repeatedly attempting to restart charging pump P-55A, and to secure
the PCPs prior to PCS pressure dropping below the minimum pressure for PCP
operation. The team concluded that the finding had more than minor risk significance in
accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue
15
Disposition Screening, because it could reasonably be viewed as a precursor to a
significant event. Inspection Manual Chapter 0609, Significance Determination
Process, Appendix A, SDP Phase 1 Screening Worksheet for IE, MS, and B
Cornerstones, states that if the finding is assumed to degrade the safety of a shutdown
reactor then use Appendix G, Shutdown Operations Significance Determination
Process. Utilizing IMC 0609, Significance Determination Process, Appendix G
worksheet, PWR Cold Shutdown Operation and RCS Closed and S/Gs [Steam
Generators] Available for DHR [Decay Heat Removal], the team determined that since
the event did not result in an inadvertent change in PCS temperature or a loss of level,
the finding was of very low safety significance (Green).
Enforcement
Technical Specification 5.4.1 requires that written procedures be established,
implemented, and maintained covering the activities specified in Regulatory Guide 1.33,
Appendix A. Item 3.a of Appendix A included procedures for the operation of the reactor
coolant system, and item 3.n included procedures for the operation of the chemical and
volume control system. Step 5.2.3 of Procedure SOP-2A, Chemical and Volume
Control System, Revision 50, limited the attempted starts of charging pump P-55A to
one attempt. Procedure SOP-1, Primary Coolant System, Attachment 2, Revision 51,
Step 3, stated, The plant shall be maintained to the left and above the Minimum
Pressure for PCP Operation Curve whenever the Primary Coolant Pumps are
operating. Contrary to the above, on March 18, 2003, a nuclear control operator
attempted to start charging pump P-55A three times after it tripped and failed to trip the
PCPs prior to PCS pressure dropping below the minimum pressure for PCP operation.
However, because of the very low safety significance and because this issue was
entered into the corrective action program, it is being treated as a Non-Cited Violation,
consistent with Section VI.A.1 of the Enforcement Policy (NCV 50-255/03-05-02). This
issue was entered into the licensees corrective action program as CAP 034027.
04.2 March 25, 2003, Loss of Offsite Power
a. Inspection Scope
On March 25, 2003, the resident inspectors responded to the control room after the
offsite power supply to the plant was lost unexpectedly. The team observed the control
room operators response to the event. The team verified that ONP-17, Loss of
Shutdown Cooling, General Operating Procedure (GOP)-14, Shutdown Cooling
Operations, and the Emergency Plan were implemented in a timely and accurate
manner to address the event.
The team walked down control room panels to monitor key plant parameters including
primary coolant system heatup rate. The team also verified that the emergency diesel
generators were operating properly to provide power to plant equipment necessary to
re-establish shutdown cooling.
The Alert was downgraded to an Unusual Event after shutdown cooling was restored
and terminated on March 27, 2003 when offsite power was reliably re-established.
16
b. Findings
No findings of significance were identified.
The team concluded that control room operators responded effectively and in
accordance with plant procedures to the loss of offsite power and resultant loss of
shutdown cooling. Consequently, the event was mitigated in a timely manner. The CRS
demonstrated positive command and control while directing operator actions and the
team noted that communications between the control room operators were clear and
concise while addressing the event.
The CRS immediately entered ONP-17 and methodically initiated actions to re-establish
shutdown cooling which limited primary coolant system heatup to only 10EF.
The control room and auxiliary operators verified that the necessary plant equipment to
support shutdown cooling operations was available and then re-established cooling flow
to the reactor within 20 minutes of the initiation of the event.
05 Equipment Performance Issues - March 18, 2003 Cable Spreading Room Fire (93812)
a. Inspection Scope
The team reviewed the licensees efforts to restore the facility following the March 18,
2003, cable spreading room fire. Additionally, the team reviewed the performance of
equipment during the event.
b. Findings
No findings of significance were identified.
One equipment performance deficiency occurred when the P-55A charging pump
breaker low suction pressure trip switch stuck in the tripped position and contributed to
the event.
The team reviewed the circumstances surrounding the failure of the P-55A charging
pump low suction pressure trip switch including a review of the maintenance and
calibration records for this switch and other similar switches. No other instances of the
P-55A charging pump low suction pressure trip switch or other similar pressure switches
failing closed after actuating were identified.
The team reviewed the selective tripping design for the power sources to the P-55A
charging pump. The team determined that the upstream power supply breaker to
LCC-12, breaker 52-1202, functioned as designed and interrupted power to minimize
the loss of power to plant components while isolating the fault.
The licensee examined LCC-12 and determined that other than smoke and soot, the
damage was isolated to the charging pump breaker enclosure. The team also reviewed
the results of the testing performed to identify potential cable damage or degradation.
The licensees investigation attributed all damage to LCC-12 to the arcing on charging
17
pump P-55A breaker 52-1205. The teams independent review did not identify any
contradictory evidence.
Licensee corrective actions for this event included the installation of new wiring from
LCC-12 to the breaker enclosure and the installation of a replacement breaker. Post
maintenance testing on the new breaker was satisfactory. No concerns were identified.
06 Equipment Performance Issues - March 25, 2003 Loss of Offsite Power (93812)
a. Inspection Scope
The team reviewed the licensees efforts to restore offsite power to the facility.
Additionally, the team reviewed the performance of equipment during protective relay
actuations.
b. Findings
No findings of significance were identified.
There were no significant equipment performance deficiencies during the event. The
team reviewed the fast transfer design for the power sources to the safeguards buses.
In accordance with the facility design, were the safeguards power source to be lost, a
fast transfer would occur to the alternate supply from the R bus through the startup
power transformer 1-2. This fast transfer, however, was dependent on the availability of
the standby source, as indicated by available voltage. During this event, there was no
transfer of power source to the startup power transformer since the R bus was the first
power source lost. The team concluded that the electrical system fast transfer design
functioned as expected in that a fast transfer did not occur to a de-energized bus.
The team reviewed the licensees cable testing results to identify those with potential
cable damage or degradation. Through megger testing of numerous conductors, the
licensee identified several conductors which had less than nominal resistance readings.
Cable MISC-1, which was the only cable actually damaged by the signpost, had five
conductors with less than acceptable megger readings.
Licensee corrective actions to restore offsite power to the safety-related buses and re-
establish system integrity included the repair of the damaged conductors and the use of
existing spare conductors on other cables routed between the plant and switchyard. On
March 27, the Shift Supervisor authorized installation of Temporary Modification
TM-2003-012, Restoration of Startup Power Transformers (1-1, 1-2, and 1-3) Protective
Relaying, for R bus relay 486 S-X1. The team observed portions of the installation of
temporary modification TM-2003-012. The modification was installed to provide for the
use of undamaged conductors in existing cables and lifting of wires routed between the
switchyard and the plant. The team reviewed the modification documentation, including
the associated 10 CFR 50.59 screening. This temporary modification allowed the
station to realign offsite power to safeguards buses 1C and 1D through the startup
transformer. The team also reviewed the completed work order which installed cable
splices to repair the damaged conductors in cable MISC-1. Based on a review of the
records and an observation of activities, no concerns were identified.
18
07 Maintenance Performance Issues (93812)
a. Inspection Scope
The team reviewed the maintenance performance issues associated with the discovery
that charging pump breaker 52-1205 was installed on May 24, 2002 without arc chutes.
The licensee identified that the lack of arc chutes was the primary cause of the cable
spreading room fire. The team reviewed work orders, maintenance procedures,
calibration records, vendor technical manuals, electrical maintenance worker training
and qualifications, clearance order practices, and selected corrective action documents.
The team also interviewed the workers that installed the breaker without the arc chutes
and other electrical maintenance personnel.
b. Findings
Introduction
A finding of very low safety significance was self-revealed during an event when the
licensee failed to have adequate maintenance procedures in place to ensure that when
electrical breakers were removed to be refurbished that the arc chutes were reinstalled
before the breaker was placed back in operation. The finding was determined to be of
low safety significance because the failure did not result in a loss of shutdown cooling or
loss of reactor inventory. One Non-Cited Violation of 10 CFR 50, Appendix B,
Criterion V, Instructions, Procedures, and Drawings, was identified.
Description
The team reviewed the work orders that were used to remove and reinstall P-55A
charging pump breaker 52-1205 prior to the event. Work Order (WO) 2491-2422
removed the breaker (Serial Number 42924-A14-2-4D) for refurbishment on
October 18, 2001, in accordance with the requirements of Procedure Station Power
System (SPS) E-17, Temporary Installation and Removal of Spare Circuit Breakers,
Revision 1. Procedure SPS-E-17, Step 4.2.1 required, During performance of
maintenance, removed parts are to be adequately packaged, identified, and stored so
they are not lost, damaged, or lose traceability to the component from which they were
removed. An entry in the WO remarks section indicated that the arc chutes had been
removed, tagged, and stored in the electrical maintenance shop. Work Order
2411-3520, which reinstalled the breaker on May 24, 2002, made no mention of the arc
chutes. However, there was no procedurally required certification of arc chute re-
installation.
The team reviewed 16 WOs that had removed or installed similar breakers without
finding any discernible pattern of recording the removal or re-installation of the arc
chutes. Five WOs recorded arc chute removal for shipping, five WOs recorded arc
chute installation prior to installation into an LCC, six WOs made no mention of arc
chutes, and only three breakers had both the removal and re-installation of arc chutes
recorded. There was no procedural requirement in SPS-E-17 to remove or install arc
chutes. The SPS system engineer could not recall any specific action or incident that
19
required the removal of the arc chutes from K-Line breakers for shipping. The apparent
intent was to prevent damage to the arc chutes during shipping.
Step 5.4 of Maintenance Procedure SPS-E-6, ITE 480 Volt Breaker Inspection and
Repair, Revision 11, contained instructions for removal, inspection, and re-installation
of arc chutes. However, neither procedure SPS-E-6 nor any other procedure was used
for removal or re-installation of the K-Line breaker arc chutes. The licensee completed
a walkdown of all installed K-line breakers on March 18, 2003, and ensured that no
other breakers were missing arc chutes.
The team reviewed the training records, training course contents, and qualifications of
both of the workers that installed breaker 52-1205 without arc chutes. No issues were
identified. When interviewed by the team, neither of the workers could identify a reason
for not installing the arc chutes although both were aware of the function and
importance of the arc chutes for proper breaker function. The workers had used a
pre-job briefing checklist prior to reinstalling the breaker and both workers were very
experienced. From the work instructions and interviews, the team noted that at one
point the spare breaker and the refurbished breaker were side-by-side for comparison of
the auxiliary contacts, but neither worker identified the absence of the arc chutes.
The team reviewed the work control processes for removing and reinstalling the
breakers. The licensee staff informed the team that separate WOs were routinely used
for breaker removal and reinstallation to facilitate declaring the installed spare and re-
installed original breaker operable following WO closeout. The team reviewed
Procedure 5.01, Processing Work Requests/Work Orders, Revision 27, and found no
requirement for separate work orders. The licensee staff agreed that no identified
process or procedure required closing out the original WO. The team noted that
information on the removal WO had not been carried forward to the re-installation WO
(specifically the removal and storage of the arc chutes) and the licensee staff confirmed
that there was no requirement to carry the information forward to the installation WO.
The team also found that there was no process or procedure that addressed control of
partial equipment shipments for repair and positive controls of the remainder of the
equipment. Likewise, no processes existed for receipt of equipment components that
also addressed control of the remainder of the equipment. Procedure 5.01 stated that,
The assigned supervisor or repair worker is responsible for control of parts associated
with Work Orders; however, the team found that there was no assigned supervisor or
repair worker for closed work orders. Consequently, there was no assurance that
information noted in a work order upon removal of an item would be available for
reference during re-installation.
The team reviewed the operational history of breaker 52-1205 and identified that the
breaker cubicle door had been opened numerous times, after the breaker had been
installed without the arc chutes, in support of eight work orders for protective tagging.
At least one of those work orders included a wiring inspection by electrical maintenance
workers. In addition to the above work orders, electrical maintenance technicians
opened the breaker door to perform an inspection of the breaker dust shields. The
licensee staff agreed that multiple opportunities for both operations and electrical
maintenance personnel to identify that the arc chutes were missing had existed.
20
However, inspection of breakers for such missing components was not a normally
performed activity.
Analysis
The team identified that the licensee performed activities affecting quality without a
procedure or work instructions, specifically removing and/or replacing breaker arc
chutes without instructions to do so. In addition, no procedural guidance existed to
control equipment after it had been partially disassembled and the work order was
closed out. The team concluded that the finding had more than minor risk significance
in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue
Disposition Screening, because inadequate maintenance procedures and the
installation of non-conforming equipment in safety-related components could be
reasonably viewed as a precursor to a significant event. Inspection Manual Chapter 0609, Significance Determination Process, Appendix A, SDP Phase 1 Screening
Worksheet for IE, MS, and B Cornerstones, states that if the finding is assumed to
degrade the safety of a shutdown reactor then use Appendix G, Shutdown Operations
Significance Determination Process. Utilizing IMC 0609, Appendix G worksheet, PWR
Cold Shutdown Operation and RCS Closed and S/Gs Available for DHR, the team
determined that since the event did not result in an inadvertent change in PCS
temperature or a loss of level, the finding was of very low safety significance (Green).
The team concluded that the failure to install the breaker arc chutes also increased
the probability of the initiation of a fire. Inspection Manual Chapter 0609, Significance
Determination Process, Appendix F, Determining Potential Risk Significance of Fire
Protection and Post-Fire Safe Shutdown Inspection Findings, states that one of the
fire protection defense in depth elements is the prevention of fires from starting. The
team determined that the failure to install arc chutes increased the likelihood of a fire
in the cable spreading room. As such, further analysis was required as specified by
IMC 0609, Appendix A, SDP Phase 1 Screening Worksheet for IE, MS, and B
Cornerstones, The team reviewed licensee engineering analysis EA-PSA-FIRE-IE-
03-05 and concurred with the results which calculated a fire initiating event frequency of
4.7 x 10-4 per year. To calculate this frequency, the licensee determined that the
frequency of a standing trip signal for breaker 52-1205 was 5.5 x 10-2 per year based on
failure rates associated with one control switch, five external relays, and three internal
breaker relays. In the March 18, 2003 event, the upstream breaker for breaker 52-1205
tripped and the fire was limited to the breaker cubicle. For their analysis, the licensee
assumed that such a fire would not be limited to the breaker cubicle if the upstream
breaker failed to trip. The team noted that this assumption was conservative because
even if the upstream breaker failed to trip, a breaker further upstream would likely trip
thereby limiting the duration of and energy contribution to the fire. The licensee
identified the failure probability associated with the upstream breaker to trip open upon
demand was 8.49 x 10-3.
The team used the Phase 2 process outlined in IMC 609, Appendix F, to determine the
significance of the event given the fire initiating event frequency developed by the
licensee. The team noted that the Phase 1 process outlined in IMC 0609, Appendix F
was not applicable because the finding did not involve fire protection features. For this
evaluation, the team assumed that a fire which was not limited to the breaker cubicle
21
could develop into a widespread fire in the cable spreading room requiring plant
shutdown from outside the control room. The team noted that such an assumption was
conservative since the nearest exposed cables were more than 4 feet above the cubicle
and there was a supply ventilation duct nearby with a design flow rate of 2000 cubic feet
per minute which would tend to disperse hot gases from a cubicle fire before the gases
came in contact with the exposed cables. Using the guidance of IMC 0609, Appendix F,
and the licensee developed fire initiating event frequency, the team determined that the
fire mitigation frequency (FMF) was -5.08. This FMF was calculated based on no credit
for fire barriers or separation, moderate degradation for automatic suppression, and full
credit for manual suppression outside of the control room. Based on a review of
IMC 0609, Appendix F, Table 5.4, the team determined that the FMF correlated to an
approximate frequency of 1 per 105 to 106 years. The failure to have arc chutes in place
was greater than 30 days. Therefore, based on a review of IMC 0609, Appendix F,
Table 5.5, the inspectors determined that the estimated likelihood rating was F. Based
on a review of the text associated with IMC 0609, Appendix F, Figure 4-3, and
IMC 0609, Appendix F, Attachment 1, Example 1C, the inspectors determined that a -1
point credit for post-fire safe shutdown operation was applicable. Therefore, based on a
review of IMC 0609, Appendix F, Table 5.6, Risk Significance Estimation Matrix, the
team concluded that the finding associated with the failure to install arc chutes was of
very low safety significance (Green).
Enforcement
10 CFR 50, Appendix B, Criterion V, Instruction, Procedures, and Drawings, states, in
part, that activities affecting quality shall be prescribed by documented instructions,
procedures, or drawings of a type appropriate to the circumstances and that work shall
be accomplished in accordance with these instructions, procedures, or drawings.
Contrary to the above, on October 19, 2001, licensee personnel removed the arc chutes
from charging pump breaker 52-1205 without procedural guidance, controls, or
documentation, and had no adequate measures in place to prevent the non-conforming
breaker from being returned to service on May 24, 2002.
However, because of the very low safety-significance and because this issue was
entered in the corrective action program, it is being treated as a Non-Cited Violation,
consistent with Section VI.A.1 of the Enforcement Policy (NCV 50-255/03-05-03). This
issue was entered into the licensees corrective action program as CAP 034187.
08 Design Issues Associated With Offsite Power Configuration (93812)
a. Inspection Scope
The team reviewed the offsite and onsite electrical distribution system as described in
various licensing and design bases documents such as the Updated Final Safety
Analysis Report. In particular, the team focused on the facilitys design with respect to
electrical and cable separation requirements. This issue was of concern in light of the
event being caused by damage to multiple conductors in a single cable. This cable
contained protective relay circuitry which affected the normal and alternate power
sources to both safety-related buses.
22
b. Findings
No findings of significance were identified.
The team concluded that the design of the electrical circuitry was in conformance with
the licensing basis. From a review of the licensees Updated Final Safety Analysis
Report and licensing documents, the team determined that the plant was designed and
constructed prior to IEEE-308, Standard Criterion for Class 1E Power Systems for
Nuclear Power Generating Stations, requirements. Therefore, the facility may not meet
all design criteria and testing requirements contained in IEEE-308. The adequacy of the
licensees electrical power system was reviewed by the NRC as part of the Systematic
Evaluation Program. The results of these reviews were documented in NUREG-0820,
Integrated Plant Safety Assessment-Systematic Evaluation Program. The team
concluded that the licensees offsite power system, and conformance to existing design
requirements relating to electrical separation and physical separation, had been
previously reviewed and approved by the NRC as part of the evaluation program. No
additional concerns were identified by the team.
Since the facility was designed prior to issuance of 10 CFR 50, Appendix A, General
Design Criteria (GDC) 17, Electrical Power Systems, the installation of a safeguards
transformer in 1989 was considered a modification to improve the capability for reliable
offsite power. Additionally, the design was considered to enhance conformance with,
but not commit compliance to GDC 17.
The original design of the offsite power improvement project was to install the new
control circuits through an existing duct bank between the switchyard and the plant.
Due to difficulties in replacing some existing cables, a new conduit run was installed and
routed between the plant and switchyard. By design, the conduit was installed about
30 inches underground. It was this conduit that was damaged by the signpost being
driven into the ground. Based on the fact that the licensee was not required to provide
physical separation between circuits associated with the R and F buses, the licensee
designed the circuits to use separate conductors within the same cable for the various
protective relay circuitry. This design contributed to the loss of offsite power event.
The licensees interim and permanent repairs to the damaged cables and modifications
to the existing design included provisions to provide additional physical separation
between the circuits for the safeguards transformer and the backup source through the
R bus.
09 Review of Emergency Plan Response Actions (93812)
a. Inspection Scope
The team interviewed members of the control room crew, and reviewed the licensees
Site Emergency Plan, operating logs, technical support center narrative logs, and
applicable event notification forms to determine if the licensee correctly classified the
event and made the proper notifications in a timely manner.
23
b. Findings
No findings of significance were identified.
10 Exit Meeting Summary
On April 4, 2003, the team presented the preliminary inspection results to
Mr. D. Cooper and other members of the Palisades Plant management and staff. The
licensee acknowledged the information presented. The team asked the licensee
whether any materials examined during the inspection should be considered proprietary.
No proprietary information was identified.
24
KEY POINTS OF CONTACT
Nuclear Management Company
T. Blake, Emergency Planning Manager
D. Cooper, Site Vice President
B. Dotson, Licensing Analyst
P. Harden, Engineering Director
N. Haskell, Nuclear Oversight Manager
L. Lahti, Regulatory Affairs Manager
D. Malone, Site Director
M. Moore, Nuclear Oversight Assessor
T. OLeary, Business Support Manager
G. Packard, Operations Manager
R. Remus, Plant General Manager
U.S. Nuclear Regulatory Commission
S. Reynolds, Deputy Director, Division of Reactor Projects
J. Lennartz, Senior Resident Inspector
R. Krsek, Resident Inspector
LIST OF ITEMS OPENED, CLOSED AND DISCUSSED
Opened
50-255/03-05-01 URI Corrective Actions to Address Digging and Excavating Events
50-255/03-05-02 NCV Failure to Follow Operating Procedures
50-255/03-05-03 NCV Failure to Have Adequate Maintenance Procedures
Closed
50-255/03-05-02 NCV Failure to Follow Operating Procedures
50-255/03-05-03 NCV Failure to Have Adequate Maintenance Procedures
25
LIST OF ACRONYMS USED
B Barrier
BAST Boric Acid Storage Tank
CAP Corrective Action Program Document
CRS Control Room Supervisor
EAR Engineering Action Request
EDG Emergency Diesel Generator
EST Eastern Standard Time
FMF Fire Mitigation Frequency
GDC General Design Criteria
GOP General Operating Procedure
gpm gallons per minute
IP Inspection Procedure
kV Kilovolt
LCC Load Control Center
LPSI Low Pressure Safety Injection
NCO Nuclear Control Operator
NCV Non-Cited Violation
MC Manual Chapter
NMC Nuclear Management Company
NRC Nuclear Regulatory Commission
ONP Off Normal Operating Procedure
PCP Primary Coolant Pump
PCS Primary Coolant System
PSIA Pounds Per Square Inch Absolute
PWR Pressurized Water Reactor
SDC Shut Down Cooling
SDP Significance Determination Process
S/G Steam Generator
SOP Standard Operating Procedure
SS Shift Supervisor
TBD To Be Determined
URI Unresolved Item
Vac Volts Alternating Current
VCT Volume Control Tank
26
LIST OF DOCUMENTS REVIEWED
Documents Reviewed
Palisades Administrative Procedure 4.28, Control of Palisades Switchyard Activities, Revision 0
Palisades Drawing SK-EAR-203-0086-2, Safeguards and Startup Offsite Power Source
Between Plan and Switchyard, Revision 0
Plant Review Committee Overview of Offsite Power Recovery
NRC Evaluation of SEP Topic VII-3, Systems Required for Safe Shutdown, December 1981
Final Safety Analysis Report, Chapter 8
Palisades Offsite Reliability Improvement - Functional Description, GWO 8303, File 114.2,
Palisades Offsite Reliability Improvement - Design Plan, GWO 8303, File 110.2, FC-800
Palisades Nuclear Plant, Design Basis Document, 2400 VAC System, July 2001
Palisades Nuclear Plant, Design Basis Document, 345 kV Switchyard, January 1999
Work Request 294427, Perform Circuit Verification on Scheme Associated with Cable MISC-1
Integrated Plant Safety Assessment- Systematic Evaluation Program, NUREG-0820,
October 1982
Integrated Plant Safety Assessment- Systematic Evaluation Program, NUREG-0820,
Supplement 1, November 1983
Work Request 300477, Check and Megger Test Conductors for Cable MISC-1
Temporary Modification TM-2003-012, Restoration of Startup Power Transformers (1-1, 1-2,
and 1-3) Protective Relaying
CAP034185, Emergency Assembly Area Over Capacity.
CAP034187, Alert Declared Due to Fire in Cable Spreading Room.
CAP034198, A and D Primary Coolant Pumps Tripped Below Minimum Operating Pressure.
CAP034237, Capture Operations Lessons Learned following LCC-12 Breaker Fire.
CAP034241, PC-0218A (Charging Pump P-55A Low Suction Trip) Out of Specification.
27
CAP034245, PC-0218B (Charging Pump P-55B low suction trip) Found Out of Specification.
CAP034399, Conduct Common Cause Evaluation on the Response to Alert Declaration.
CAP034622, Missed Opportunity to Update FSAR Text During SOP Procedure Revision.
CAP019120, Alert Declared Due to Fire in Cable Spreading Room.
CAP034538, P-55C Potentially Run Without Suction or Discharge Path.
CAP034623, Design Bases Calculation for Charging Flow Not Updated for Operational Change.
Procedure SOP-2A, Chemical and Volume Control System, Revision 50.
Procedure ONP-25.1, Fire Which Threaten Safety - Related Equipment, Revision 12.
Procedure 24912222,Temporary Installation and Removal of Spare Circuit Breakers, Revision 1
Procedure SPS-E-17,Temporary Installation and Removal of Spare Circuit Breakers, Revision 2
Procedure 4.14, Conduct of Operations, Revision 0.
Procedure 4.09, Control of Operator Aids, Revision 7.
Procedure 10.53, Use and Adherence of Procedures and Other Forms of Written Instruction,
Revision 12.
Procedure 4.00, Operations Organization, Responsibilities and Conduct, Revision 23.
Procedure QO-27, Inservice Testing of CVCS Control, Motor-Operated and Check Valves,
Revision 9.
Procedure 5.01, Processing Work Requests/Work Orders, Revision 27.
Procedure QO-17, Inservice Test Procedure - Charging Pumps, Revision 18.
Procedure 1.10, Plant System, Structure, and Component Labeling, Revision 2.
WO SPS 24912222 7, Charging Pump P-551 Breaker
WO SPS 24912222 7, Removed, Tagged and Stored Arc Chutes in Shop.
WO SPS 24113520 1, Charging Pump P-55A Breaker.
P&ID M-202 SH, 1B, Chemical & Volume Control System.
VTD-2881-0009, Installation Maintenance Instructions For Low Voltage Power Circuit Breakers.
28
VTD-2881-0010, Installation Maintenance Instructions For Low Voltage Air-Magnetic Power
Circuit Breakers For K-Line 225A Thru 2000A.
GOP-14, Shutdown Cooling Operations, Revision 17
ONP-17, Loss of Shutdown Cooling, Revision 28
Operations Log Entries, March 25, 2003
EA-PSA-FIRE-IE-03-05; Fire Initiating Event Frequency for Missing Arc Chutes on 480VAC
Circuit Breaker 52-1205; Revision 0
29
30