ML20149G638
| ML20149G638 | |
| Person / Time | |
|---|---|
| Site: | Zion File:ZionSolutions icon.png |
| Issue date: | 07/17/1997 |
| From: | NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III) |
| To: | |
| Shared Package | |
| ML20149G625 | List: |
| References | |
| 50-295-97-12, NUDOCS 9707230382 | |
| Download: ML20149G638 (19) | |
See also: IR 05000295/1997012
Text
[
'
. .
,
U.S. NUCLEAR REGULATORY COMMISSION
REGION lli
.
Docket No: 50-295
License No: DPR-39
Report No: 50-295/97-12
Licensee: Commonwealth Edison Company
Facility: Zion Nuclear Plant, Unit 1
Location: 101 Shiloh Boulevard
Zion, IL 60099
Dates: March 11 through April 25,1997
Inspectors: E. W. Cobey, Resident inspector
D. R. Calhoun, Resident inspector
A Vegel, Senior Resident inspector
.
Approved by: Michael E. Parker, Acting Chief
Reactor Projects Branch 2
!
j.
i
'
, ,
! !
l
9707230382 970'717
G ADOCK 05000295 )
-
PDR !
._. _-
'
__
. .
EXECUTIVE SUMMARY
Zion Nuclear Plant, Unit 1
NRC Inspection Reports 50-295/97-12
Thit special inspection included aspects of licensee operations, maintenance, and
engineering associated with the Unit 1 loss of offsite power event that occurred on
March 11,1997.
Ooerations
-
Although the licensee satisfactorily recovered from the event with no actual
consequences to the public health and safety, some deficiencies in the licensee's
,
performance unnecessarily hampered and slowed the licensee's response to the
- evcnt. These performance issues involved inconsistent communication and
coordination between departments, inefficient work planning, plant equipment and
operating procedure problems, and insufficient control of operability assessments. '
(Section 01.1)
-
Licensee management's discretionary activation of the Technical Support Center,
pre-evolu' tion walkdowns of procedures, and the licensee's increased emphasis .
placed on exhibiting a questioning attitude and performing activities in a controlled l
manner demonstrated a conservative operating philosophy focused on safety.
(Sections 01.1 and 03.1)
-
Inconsistent quality of pre-evolution walkdowns of operating procedures and several
deficient operations procedures contributed to the delay in restoring offsite power.
A violation involved the failure to have an existing procedure for responding to a
loss of offsite power event. (Section 03.1)
The Independent Safety Engineering Group (ISEG) performed a thorough review of
-
the Unit 1 loss of offsite power event; however, the inspectors identified that
licensee management did not initiate action to address the issues or
recommendations of the associated ISEG report in a timely manner. (Section 07.1)
Maintenance
-
The emergency diesel generators successfully started on demand and provided the
required electrical power to the Unit 1 engineered safety feature loads: however,
the inspectors also concluded that numerous material condition problems extended
the recovery time frorn the event by diverting resources and licensee management
focus to resolve the issues. (Section M2.1)
-
The inspectors identified a violation involving the failure to implement timely and
effective corrective actions to prevent the recurrence of malfunctions of sequence
timers that control engineered safety feature bus loading. (Section M2.1)
2
. . . .
.
Enaineerinn
-
The inspectors identified that the periodic testing of a nonsafety-related sudden
pressure relay for the system auxiliary transformer did not demonstrate that the I
relay was within calibration and placed the plant at an increased' risk of a spurious
loss of offsite power. (Section E2.1)
1
!
-
The inspectors identified that between February 27 and March 5,1997, due to l
operations management not having ownership of operability assessments, the l
operating shift was unaware of the requirements imposed by the operability !
assessment for emergency diesel generator cylinder liner cracking. (Section E2.2) l
1
-
Poor commur'ication and documentation of the basis and the compensatory actions
l
required to support thB operability assessment for emergency diesel generator l
cylinder liner cracking, unnecessarily complicated the recovery plans and resulted in
delays in the restoration of offsite power. (Section E2.2)
4
3
~ _ _ _ . - _ _ . _ ~ ~ . . - - .. __ .~ _ _ _ _ _ _ . _ . _ _ _ - . . _ _ . _ _ . _
l
.
l .
l
l
!
!
Report Details
- - 1. Operations
01 Conduct of Operations
l
01.1 Loss of Offsite Power to Unit 1
l a. Insoection Scooe (93702 and 71707)
>
,
'
On March 11,1997, Unit 1 experienced a loss of offsite power. As a result, the '
inspectors responde~d to the site and observed licensee activities throughout the
event recovery, in addition, the inspectors performed a follow-up inspection into
the circumstances surrounding the event which included evaluating the licensee's
root cause investigation, interviewing licensee personnel, and reviewing applicable
procedures and documentation.
b. Observations and Findinas
Secuence of events
Initial Conditions:
Unit 1 -- Cold Shutdown, depressurized with offsite power via the Unit 1
system auxiliary transformer
Unit 2 -- Cold Shutdown, depre:,surized with offsite power via the Unit 2
system auxiliary transformer i
h1 arch 11,1992
1553 A loss of offsite power occurred on Unit 1 as a result of the isolation of
the system auxiliary transformer (SAT) due to a sudden pressure trip. The
Unit 1 SAT deluge activated and the 1 A,18, and O emergency diesel
generators (EDGs) started and energized their respective buses as designed.
Operators entered abnormal operating procedure (AOP) 6.3, " Loss of RHR
[ Residual Heat Removal] Shutdown Cooling." The commercial phone system
was lost.
1602 Operators restarted the 1B RHR pump to re-establish shutdown cooling in
accordance with AOP-6.3.
1603 Upon restoration of shutdown cooling, operators exited AOP-6.3.
L
i
1618 The licensee declared an Unusual Event in accordance with Emergency
- Action Level MU-1 (offsite power being unavailable for greater than
l 15 minutes).
- 4
I
i
. . - . .- - . - . . . = - .. ..
~.,
-
-
. l
l
!
1640 The licensee completed the initial emergency notification system (ENS)
i report for the declaration of an Unusual Event.
l
l. 1700 Operators restored spent fuel pool cooling.
1720 Control room personnel requested that the Technical Support Center (TSC) .
be activated to assist in restoration activities from the loss of offsite power. I
The licensee's intent was to power each engineered safety feature (ESF) bus
from its reserve feed on the other unit versus the present power sources
from each respective EDG.
l
1730 Operators energized bus 147 from its reserve feed (Unit 2 SAT)in
accordance with Emergency Support Procedure (ESP) 6.2, " Energize 4KV j
ESF Buses from Reserve Feed" and secured the O EDG.
l
1806 The OB instrument air (!A) compressor tripped on high temperature. I
Operators entered AOP-3.3, " Loss of Instrument Air."
l
1813 Operators restarted the OB IA compressor and exited AOP-3.3.
1819 The commercial phone system was restored.
1850 The licensee conducted a follow-up ENS report to update the NRC of the
current plant status and priorities.
2110 The TSC personnel assumed command and control of the event. '
March 12.1997
0200 Operators completed system operating instruction (SOI) 63BB, "Deenergizing
4KV Non-ESF Buses," in preparation for performing SOI 63A, "345KV
Operations," Section 5.8, "Deenergizing the 345KV System Auxiliary
Transformer." The licensee isolated the SAT so that the ring bus could be
restored.
0305 Operators attempted to close breaker OCB-67 to restore the ring bus in
accordance with SOI-63A; however, the breaker would not close since the
86 relay and the 142 lock out relay were in a tripped condition.
0420 Operators reset the 86 relay and the 142 lock out relay.
- 0428 Operators restored the ring bus in accordance with sol-63A.
0436 The licensee conducted a follow-up ENS report to update the NRC of the
current plant status and priorities.
0910 Operators shifted loads ; @ ration for the transfer of bus 149 to reserve
l. feed in accordance with - '
F, " Transferring 4KV ESS [ Engineered
i
!
5
i
t - - . -- _ - . , . - . , - - --,
. .
Safeguards System] Buses from D/G [ Emergency Diesel Generator) to
Reserve Feed with Unit Trip and System Auxiliary Transformer Trip."
0955 Operators identified a needed procedural change to sol-63F regarding the '
battery charger feed breaker position.
1045 Operators placed the transfer of bus 149 to reserve feed on hold, pending
the resolution of questions regarding the maximum current rating of the
Unit 2 SAT.Z winding power feed cabling and the resolution of seal oil
system problems.
1110 Operators backed out of sol-63F.
.
1130 Design engineering personnel set a current limit of 1200 amps on the Unit 2
1610 On the recommendation of the Plant Operations Review Committee (PORC).
TSC personnel d..ected that the ESF buses remain powered by their
respective EDG rather than transfer the buses to reserve feed until the sta*us
of the Unit 1 SAT repairs could be determined (expected in approximately
seven hours).
March 13,1997
0350 The licensee conducted a follow up ENS report to update the NRC of the
current plant status and priorities.
.1700 System engineering personnelidentified that the bus 147 division 17-2 safe
shutdown timer was not fully reset.
2040 Maintenance personnel replaced the SAT sudden pressure relay.
March 14.1997
0420 The licensee conducted a follow up ENS report to update the NRC of the
current plant status and priorities.
0735 Operators energized the Unit 1 SAT in accordance with SOI-63A.
1012 Operators energized all 4KV non-ESF buses (bus 142,143,144, and 145) in
accordance with SOI-638, " Energizing 4KV Non-ESF Buses from the SAT."
1154 While operators attempted to energize bus 135 in accordance with SOI-63H,
"480V Non-ESF Bus Operations," breaker 1351 did not close.
1253 Operators energized bus 132 in accordance with sol-63H.
6
.
l .
,
l
1340 Operators energized bus 133 in accordance with sol-63H; however, the
l phase voltages on bus 133 were not within 10 volts as required.
1352 Operators opened bus 133 feed breaker in accordance with SOI-63H due to
the phase t/oltage difference.
1433 Operators ener0 ized bus 134 in accordance with SOI-63H.
1718 Operators secured reserve feed and energized bus 147 from normal feed.
Operators reset the division 17-2 safe shutdown timer. l
l
1832 Operators closed the main feed breaker to bus 148 in accordance with sol- l
63E (bus 148 powered from normal feed and the 1 A EDG). ~
l
l
2214 Operators unloaded the 1 A EDG and opened the output breaker. I
2354 Operators closed the main feed breaker to bus 149 in accordance with
SOI-63E (bus 149 powered from normal feed and the 1B EDG). 1
[Aarch 15.1997
0021 All ESF buses were being powered from normal feed and the licensee
terminated the Unusual Event.
0033 The licensee completed the ENS report for the termination of the Unusual
Event.
0125 Operators energized bus 135 in accordance with SOI 63H.
0137 The licensee transferred command and control to the control room.
0242 Operators unloadt.d the 1B EDG and opened the output breaker.
0405 Operators energized bus 133 in accordance with sol-63H. Electrical
maintenance personnel verified that the phase to phase voltage difference
was less than 10 volts. The licensee initiated an action request to calibrate
the control room indication.
1000 The licensee assigned all restoration items for completion and suspended
TSC operation.
Evaluation of Licensee's Event Resoonse
The inspectors observed licensee event response activities in the control room,
, TSC, and the plant, talked with involved licensee personnel, and attended related
i licensee meetings. Through the observation of licensee interactions and activities
over the course of the event, the inspectors arrived at various insights regarding the
quality and timeliness of the licensee's event response. Although the licensee
7
.- - . .- ...~ - - . - .. -. . -- . . - - - - - - - - - - - . . -
l' . .
-
L
l
l
l
satisfactorily recovered from the event with no actual consequences to the public
health and safety, some deficiencies in the licensee's performance unnecessarily
hampered and slowed the licensee's response to the event.
Communication and coordination between departments was inconsistent, '
For example, inspectors observed mis-communication regarding the status of
safe shutdown timers, operator confusion regarding engineering guidance on
loading the EDGs, differing licensee interpretations of the licensee's EDG
operability assessment, and a security guard unnecessarily slowing the shif t
engineer's access to a vital switchgear room. ,
'
-
The licensee's work planning process did not always ensure recovery work
was accomplished in an expeditious manner. For example, inspectors
observed that pertinent parties were unaware that needed personnel would
not be available to support SAT maintenance activities when planned.
Slow recovery from the event can be partially attributed to the licensee's increased
emphasis on demonstrating a questioning attitude and performing activities in a
controlled manner. Related inspector observations and the licensee's discretionary
activation of the TSC to remove the burden of managing restoration activities from
control room personnelindicated strong licensee management support for a
. conservative operating philosophy focused on safety. However, the organizational !
and process problems noted above further extended event recovery time.
c. Conclusions
The inspectors concluded that the licensee's response to the event demonstrated
clear licensee management efforts to encourage a safe and controlled approach to
plant operations. However, the inspectors also concluded that the recovery time of
offsite power was extended, unnecessarily increasing the timeframe during which
the plant was at increased risk,- due to:. (1) various organization and process
problems (noted above): (2) several operations procedures needing revision prior to
use (see Section 03.1); (3) numerous equipment issues diverting licensee efforts
from recovery of offsite power (see Section M2.1); and (4) confusion regarding the
basis for the conclusions and the compensatory actions specified in the EDG
operability assessment on cylinder liner cracking (see Section E2.2).
03 Operations Procedures and Documentation
. 03.1 Ooerations Procedures '
a,- Insoection Scooe (71707)
L The inspectors interviewed licensee personnel and reviewed selected procedures
utilized during recovery from the loss of offsite power event.
!
8
t
4
- y-- 3 -,, , r, - -
.v,, -
r -
.. -- - -- - - . - - .- - .- .- ---- .- - -
l
-
.
-
b. Observations and Findinas
As a result of the increased focus on conservative operations, which had been
prompted by the reactivity management event on February 21,1997, the licensee .
-
had been reviewing and performing walkdowns of all operating procedures prior to
their performance except under emergency conditions. The licensee also
implemented this practice during the loss of offsite power for procedures used in
responding to the event. Consequently, the licensee revised four procedures
l
(SOI-63A, "345KV Operations:" sol-63B, " Energizing 4KV Non-ESF buses from the
SAT;" SOI-63D, " Transferring 4KV ESS Buses from Reserve Feed to Normal Feed;"
i and SOI-63E, " Transferring 4KV ESS Buses from D/G to Normal Feed") to
incorporate procedural enhancements that the shift determined were necessary
prior to the performance of the respective evolution. In addition, the licensee
identified that SOI-11K, " Transfer of Fuel Oil from Tanker Truck to Fuel Oil Storage
l Tanks," Section 5.3, " Filling Heating Steam Boiler Fuel Oil Storage Tank Using
Tanker Truck Fuel Oil Transfer Pump," could not be performed as written without
resulting in a fuel oil spill. The licensee also revised this procedure.
Even though the licensee was performing pre-evolution walkdowns of procedures,
the licensee did n6t identify needed procedural enhancements to SOI-63F,
" Transferring 4KV ESS Buses from D/G to Reserve Feed with Unit Trip and System
Auxiliary Transformer Trip," until the procedure was actually being performed. A
delay la performing the evolution resulted while the procedural concerns were being
resolved. The licensee concluded that this oversight was apparently due to an
!
operator's lack of understanding of licensee management's expectation for walking
down procedures prior to implementation. The operator responsible for walking
down the procedure had reviewed it versus actually walking it down, which
! contributed to the failure to identify the discrepancies.
!
Prior to the performance of SOI-63F, " Transferring 4KV ESS Buses from D/G to
Reserve Feed with Unit Trip and System Auxiliary Transformer Trip," engineering
personnel recognized that an open item existed from the validation of the Auxiliary
Power Design Basis Document in April 1996. The open item addressed the current
limit on the Z winding which could allow the cross-tie power feed cables to exceed
l their ampacity limit. Specifically, ESP-6.2, " Energize 4KV ESF Buses from Reserve
l
Feed," contained a caution statement which stated: "Do NOT exceed 1800 amps
on Z winding of the System Auxiliary Transformer when loading buses that are
energized from reserve feed." However, this limit was not sufficient to prevent the
ampacity limit for the cross-tie power feed cables from being exceeded, as sol 63F
l did not contain any guidance on the ampacity limits of the cross-tie power feed
'
cables for the Z winding. As a result, the licensee delayed the performance of
sol-63F pending the resolution of this issue. Subsequently, the licensee specified a
, conservative administrative current limit on the Z winding of 1200 amps. On
i
March 21,1997, design engineering personnel determined that the allowable
current limit on the Z winding of the SAT was 1250 amps when cross tied to the
opposite unit.
,
9
f
l
_ - . _ . _ _ _ _. 4 _._ _ _ _ _ _ - . - _ _ . _ _ _ . . _ . _ _ _ . _ . - _ _ . , _
. <
.
l
In addition, the licensee identified that no procedural guidance existed for
. responding to a loss of offsite power event. Although portions of existing
- procedures covered various operating evolutions conducted during the event, no
7
integrated procedure existed to delineate overall strategies in responding to the
!
!
event. ' As a result, the licensee initiated two problem identification forms, 97-1374
and 971386, to address this issue. At the end of the inspection period, the
licensee's investigation was stillin progress.
c. Conclusions
l
The inspectors concluded that the increased emphasis placed on reviewing the
adequacy of procedures prior to their use was conservative. However, the
inspectors also concluded that inconsistent quality of the pre-evolution walkdowns ,
of operations procedures and required procedure revisions contributed to the
extensive time required for restoration of offsite power.
The failure to have an existing procedure for responding to a loss of offsite power
event is considered a violation of Technical Specification 6.2.1.a
(50-295/97012-01), as described in the attached Notice of Violation. 1
L
07 Quality Assurance in Operations
07.1 Site Quality Verification (SOV) Oversicht
Subsequent to the recovery of offsite power to Unit 1, the independent Safety ]
Engineering Group (ISEG) performed an independent review into the event (OVL
22-97-026). This evaluation was thorough and contained several pertinent issues
and recommendations. The ISEG review was completed on March 24,1997,and
distributed to senior licensee management for comment. - However, as of
April 28,1997, no response had been received by the ISEG, and the ISEG had not
- yet initiated any additional action. The inspectors concluded that the licensee's
response to the evaluation did not appear timely.
..
08 Miscellaneous Operations issues
08.1 (Closed) LER 50-295/97007: System auxiliary transformer trip caused by spurious
operation of the sudden pressure relay with minimalimpact to the plant. This issue
is the subject of this inspection report. No new information was provided in the
LER. The LER is closed.
l
1
l
!.
>
I
!.
I
10
l l
!
i
l l
- -- -- -. . . - - - ,
l
. .
!
11. Maintenance
M2 Maintenance and Material Condition of Facilities and Equipment
M2.1 Material Condition ~
a. Insoection Scone (62707)
The inspectors interviewed operations, m'aintenance, and engineering personnel,
inspected selected equipment, and reviewed the licensee's corrective actions to
previous sequence timer malfunctions,
b. Observations and Findinas
On March 11,1997, in response to the Unit 1 loss of offsite power, the EDGs
started and the required essentialloads sequenced onto their respective ESF buses;
however, the recovery from the event was complicated by the numerous material
condition issues that arose during the event. Specific examples included: (1) the
~ failure of the division 17 2 safe shutdown timer to reset: (2) the failure of the
commercial phone system when offsite power was lost; (3) the loss of instrument
air (IA) due to the OB IA compressor tripping on high temperature and the 2A lA
dryer failure: (4) the rising pressure in the cardox tank which resulted in a personnel
safety concern due to the relief valve being vented into the turbine building: (5) the
clogging of the auxiliary building floor drain tank system lines: (6) the overheating
of the generator seal oil pump; (7) the f ailure of breaker 1351 to close: (8) the
inaccurato control room phase voltage indication on bus 133; and (9) the f ailure of
several breaker cubicle shudder mechanisms to operate properly. While none of
these individually compromised the safety of the plant, collectively they contributed
to the extended recovery from the event due to the diversion of resources and
licensee management focus to resolve the issues.
The malfunction of sequence timers has been a long standing equipment problem at
Zion Station. On December 8,1995, all of the bus 149 ESF loads, except the
reactor containment fan coolers, simultaneously loaded onto bus 149 during
testing. This event was documented in NRC Inspection Report 50-295/95020,
50-304/95020. On January 20,1996, two separate safety injection EDG sequence
timer malfunctions occurred.
As a result of these three events, the licensee conducted a level 2 root cause
investigation, Nuclear Tracking System (NTS) No. 295-200 96-CAT 2-003. This
investigation concluded that the necessary immediate corrective actions to prevent
recurrence were in place. These actions included: (1) system engineering interim
guidance for increased cam shaf t clearance; (2) initiated and scheduled action
requests / nuclear work requests implementing the interim guidance; and (3)
maintenance procedure E0231, " Eagle Signal Timer inspection and Lubrication,"
was updated. However, the work requests to perform these maintenance activities
were not completed prior to the failure of the division 17-2 safe shutdown timer to ;
! reset on March 11,1997. This was apparently due to the Unit 1 work requests not !
'
i
11
!
1
- - -
l
. . - - _ - -. - - - - ... . . . - = . .. . ~ . -
,, . .- .
.
!
. s
,
i
! being planned until October and Novernber 1996, approximately eight months after
the completion of the root cause investigation. Consequently, an opportunity to
perform the maintenance activities was missed during Z1F22 (August 27.through
September 18,1996).
.
The licensee's root cause investigation team also specified nine additional corrective
actions. The inspectors reviewed these corrective actions and determined that one
of the nine actions, NTS No. 295-200-96-CAT 2-00306, was never entered into
NTS and apparently was never completed. This corrective action specified that the
Central Receipt, inspection, and Testing Facility's dedication procedure be revised
as necessary to include all applicable critical attributes identified in the sequence
timer manufacturer's adjustment procedure. In response to the inspectors'
concern, the licensee initiated a problem identification form, 97-2179, to
investigate the failute to track identified corrective actions to completion,
c. Conclusions
The inspectors concluded that the EDGs successfully started on demand and
provided the required electrical power to the Unit 1 ESF loads. However, the
inspectors also concluded that the numerous material condition problems
contributed to the extended time for recovery from the event due to the diversion
of resources and licensee management focus to resolve the issues.
The failure to implement timely and effective corrective actions to prevent the
recurrence of sequence timer malfunctions, a significant condition adverse to
quality, is a violation of 10 CFR Part 50, Appendix B, Criterion XVI
(50-295/97012-02). as described in the attached Notice of Violation.
Ill. Enaineering
E2 Engineering Support of Facilities and Equipment
E 2.1 Licensee's Root Cause Investination
a. Insnection Scone (37551)
The inspectors interviewed licensee personnel and reviewed the licensee's root
cause evaluation of the loss of offsite power event.
b. Observations and Findinas
On March 11, the licensee initiated an investigation into the cause of the loss of
offsite power event. Based on the results of this investigation, the licensee could
not conclusively determine the cause for the Unit 1 SAT trip. However, through
j the elimination of other possib:e causes, the licensee concluded that the most
!
probable cause of the event was the spurious operation of the SAT sudden pressure
relay. Also, the licensee's investigation noted previous instances of spurious-
,
( 12
l
. _
. . - - - -.
. .
i
operation of these relays in the industry; however, the licensee was unable to
obtain any documentation of these previous failures. The inspectors reviewed the
licensee's investigation and determined that the conclusion was reasonable.
Additionally, the inspectors identified that the licensee did not calibrate these
sudden pressure relays. However, periodic testing was performed on a refueling
outage interval. This testing was considered within the craf t capability of the
maintenance staff and it roughly corresponded to Test Procedure I as described in
the relay vendor information. The testing demonstrates that the relay will change
state; but, it does not demonstrate that it will do so at the correct rate. The vendor
information described four test procedures of varying complexity, two of which
demonstrate that the relay will change state at the specified rate.
.
The sudden pressure relay was a transformer protective device utilized to detect
sudden pressure increases in the transformer resulting from an internal fault. If the
relay was actuating at a relatively low rate, the transformo, would have been
susceptible to being tripped by spurious actuations of the relay. While such a
setpoint would be conservative from a transformer protection standpoint, spurious
actuations could result in a loss of offsite power (and hence loss of shutdown
cooling), unnecessarily subjecting the plant to additional transients. The inspectors
were concerned that the periodic testing of the sudden pressure relay did not
demonstrate that the relay was within calibration.
c. Conclusions
The inspectors concluded that the licensee's investigation into the cause of the loss
of offsite power was sound and the conclusions were reasonable. Also, the
inspectors concluded that the failure to demonstrate that the sudden pressure relay,
a nonsafety-related component, was within calibration was a deficiency in the test
program.
E2.2 Emeroency Digsel Generator (EDG) Onerability Assessment for Cylinder Liner
Crackina
a. Insoection Scone (37551)
The inspectors interviewed licensee personnel and reviewed each revision of the
EDG operability assessment for cylinder liner cracking, #ER9700909.
b. Observations and Findings
On January 27,1997, the 2A EDG experienced a significant drop in crankcase tube
oil level during a maintenance run. The licensee's investigation into the event
l determined that the loss of tube oil was caused by a through-wall crack in the 1L
l cylinder liner. The crack allowed a sufficient amount of cooling water to
I
accumulate in the cylinder to cause the piston to catastrophically fail during engine
startup. The review of the licensee's root cause investigation will be the subject of
NRC Inspectic.i Report 50-295/97-05, 50 304/97-05.
,
13
l
l
i
- , .
-
i
i
i
On February 8,1997, the licensee approved the initial operability assessment.
were operable. The licensee subsequently inspected and identified additional
examples of cylinder liner cracking on the 28 EDG. As a result, on February 27,
1997, the licensee completed an operability issue re-assessment. This re-
I
assessment specified compensatory actions to be performed to ensure that a
cylinder liner through-wall crack did not exist. The re-assessment required these
compensatory actions to be performed prior to the next scheduled start of each
EDG and approximately 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> after each demand or series of demands
thereafter until the issue was resolved.
On March 5,1997, the inspectors identified that the operating shift personnel were
planning to perform the monthly operability surveillance test for the O EDG on the
day-shif trhowever, operating shift personnel were unaware of the requirement to
perform the compensatory action specified in the operability re-assessment prior to
engine operation. The licensee had apparently failed to route the re-assessment to
i
the operating shift. The inspectors were concemed that even though operations
l management had attended the Plant Operations Review Committee (PORC) meeting
which reviewed the operability re-assessment, the compensatory action
j requirements were not communicated to the control room operators.
On March 6,1997, the inspectors discussed this concern with the Unit 1 Plant
Manager. He indicated that the problem had occurred due to operations
management's lack of ownership of operability assessments and due to
programmatic weaknesses in the operability assessment process. Specifically, the
operability assessment process did not require revisions to operability assessments )
to be approved by the operating shif t. He also indicated that corrective actions for I
the problems were being developed and would be implemented as soon as practical.
,
On March 11,1997, Unit 1 experienced a loss of offsite power and the 1 A,18,
l- and O EDGs started and essentialloads were sequenced onto their respective ESF
buses. This successful start of these EDGs provided assurance that a through-wall
crack, of sufficient size to prevent the fulfillment of the EDG's safety function, did
not currently exist. The licensee's initial response to the loss of offsite power event
was to pursue shifting the ESF buses' power feeds from the EDGs to reserve feed
l from the Unit 2 SAT. Since the licensee's operability assessment stated that crack
(
'
propagation typically occurred during the thermal transient of the EDG cooldown,
the inspectors were concerned that the EDGs would be secured without evaluating
the potential impact of the cooldown on the EDGs. The licensee had yet to develop
a procedure to perform the specified compensatory actions in a manner that did not j
require the EDGs to be taken out-of service. In addition, transfer to the reserve l
l feed required disconnecting of the applicable EDG from its ESF bus prior to
) energizing the bus from the reserve feed (a dead bus transfer).
l
1.
i
l On March 12,1997, a conference call between the NHC and the licensee was I
'
conducted to discuss the licensee's plan for recovery ef offsite power to Unit 1.
l
The inspectors observed a subsequent PORC meeting regarding the EDG operability
assessment and the recovery plan Based on the inspectors observations and i
!
14 i
'
i
l
l
. _.
.
l
discussions with senior plant management, the inspectors concluded that senior
plant management did not fully understand the deta!is supporting the EDG
operability assessment conclusions. This lack of understanding was apparently due
to poor communication and documentation of the basis of the specified
compensatory actions. As a result of the PORC's recommendations, the licensee -
decided to keep the ESF buses powered by their respective EDGs and suspended
ongoing SOI-63F actions until the status of the SAT could be determined.
On March 15,1997, the licensee approved revision 2 of the operability assessment
which modified the compensatory action requirements. This revision specified that
the action, to check for the existence of a cylinder liner through wall-crack, be
performed 24 to 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> following each EDG demand or series of demands.
However, the inspectors were concerned with the soundness of the engineering
judgement utilized in determining the time interval, in the case of the 2A EDG
failure on January 27,1997, the machine had been started within 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> and
sufficient water had accumulated in the cylinder to result in a catastrophic failure of
the 1L piston. The inspectors subsequently raised this concern to senior plant
management.
On March 18,1997, the PORC reviewed revision 2 of the operability assessment
and determined that it needed to be revised. On March 19,1997, the licensee
approved revision 3 without addressing all of the concerns raised by the PORC. In
addition, the inspectors noted that this revision omitted the requirement to perform
the compensatory action prior to the next scheduled start of each EDG, although
that start had not yet been attempted for the 2A EDG. Based on discussions with
operations and engineering personnel, the inspectors determined that the omission
had been inadvertent. Af ter addressing issues previously raised by the PORC, the
licensee approved revision 4 on March 26,1997.
c. Conclusions
The inspectors concluded that between February 27 and March 5,1997, the
operating shift was unaware of the requirements imposed by the EDG operability
assessment due to poor communications and a lack of operations ownership of
operability assessments. The inspectors also concluded that poor communication
and documentation of the basis and the compensatory actions required to support
the operability assessment unnecessarily complicated the recovery plans and
resulted in delays in the restoration of offsite power.
V. Manaaement Meetinas
X1 Exit Meeting Summary
l The inspectors presented the inspection results to members of licensee
l management at the conclusion of the inspection on May 7,1997. The licensee
acknowledged the facts presented, but objected to the characterization of licensee
performance related to the timeliness of the event response. Specifically, the
15
. .
licensee did not agree with the inspectors' conclusion that licensee performance
deficiencies unnecessarily hampered and slowed the event response.
The inspectors asked the licensee whether any materials examined during the
inspection should be considered proprietary. . No proprietary information was
identified.
.
4
16
i
. . . . .
. .
,
Partial List of Persons Contacted
Licensee .
J. Mueller, Sito Vice President -
R. Starkey, Plant General Manager
.R..Godley, Regulatory Assurance Manager
M. Weis, Support Services Director
R. Zyduck, Site Quality Verification Director
G. Vanderheyden, Operations Manager
M. Schimmel, Unit 2 Maintenance Manager
D. Beutel, Regulatory Assurance
M. Wiesneth, Regulatory Assurance
fLRC -
A. Vegel, Senior Resident inspector
E. Cobey, Resident inspector
.;
17
. .. . . .
- _ - _ >
.. . .
-
c
.
)'
.
List of Inspection Procedures Used
IP 37551 Engineering
IP 62707 Maintenance Observation
IP 71707 Plant Operations
- IP 93702 Prompt Onsite Response to Events at Operating Power Reactors
List of items Opened. Closed, and Discussed
Oooned
50-295/97012-01 VIO - Failure to have an existing procedure for responding to a loss >
-
of offsite power event
50-295/97012-02 VIO Failure to implement timely and effective corrective actions to
prevent the recurrence of sequence timer malfunctions
Closed
50-295/97007 LER System auxiliary transformer trip caused by spurious operation
of the sudden pressure relay with minimalimpact to the plant
l- 18
-- -
. . . . .
. . . . . . .
,
. .
,
List of Acronyms
AOP Abnormal Operating Procedure
EDG Emergency Diesel Generator
ENS Emergency Notification System
ESF Engineered Safety Feature
ESP Emergency Support Procedure
ESS Engineered Safeguards System
lA Instrument Air
LER Licensee Event Report
NRC Nuclear Regulatory Commission
NTS Nuclear Tracking System -
PORC Plant Operations Review Committee
SAT System Auxiliary Transformer
SOI System Operating Instruction
19
8
.-