ML20216C260
ML20216C260 | |
Person / Time | |
---|---|
Site: | Quad Cities ![]() |
Issue date: | 03/06/1998 |
From: | NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III) |
To: | |
Shared Package | |
ML20216C246 | List: |
References | |
50-254-97-28, 50-265-97-28, NUDOCS 9803130354 | |
Download: ML20216C260 (32) | |
See also: IR 05000254/1997028
Text
4
.
U. 8. NUCLEAR REGULATORY COMMISSION
REGION 111
Docket Nos: 50-254;50-265
Report No: 50-254/97028(DRP); 50-265/97028(DRP)
Licensee: Corr,aonwealth Edison Company (Comed)
\
Facility: Quad Cities Nuclear Power Station, Units 1 and 2
Location: 22710 206th Avenue North
Cordova, IL 61242
Dates: December 24,1997 - February 10,1998
Inspectors: C. Miller, Senior Resident inspector
C. Lipa, Senior Resident inspector (DAEC)
K. Walton, Resident inspector
L. Collins, Resident inspector
M. Holmberg, Reactor inspector
R. Ganser, Illinois Department of Nuclear Safety
Approved by: Mark Ring, Chief i
Reactor Projects Branch 1
1
i
9803130354 980306
PDR ADOCK 050002t
G PDR
..
.
EXECUTIVE SUMMARY
Quad Cibes Nuclear Power Station, Units 1 & 2
NRC inspection Repost No. 50254/g7026(DRP); 50-265/g7028(DRP)
This inspection included aspects of licensee operations, engineering, maintenance, and a' ant
support. The report covers a six-week period of resident inspection.
Operations
. Technician error, inadequate communication and the misinterpretation of a procedure led
to the unexpected start of the Unit 1 emergency diesel generator (EDG). Licensee
investigation of operator performance during the event was insightful (Section 01.2).
. Two optors, using the licensee's independent verification methods, failed to establish
correct cooling watsr valve position for the Unit 2 EDG, renc'ering it inoperable during
some of the time that the Unit 1 EDG was also inoperable (Section 01.3). In addition, .
two individuals preparing a retum to service package caused a Unit 2 EDG air start valve
to be out of the reauired position. These events were indicative of a weakness in
independent verification (Section O2.1).
- The licensee did not proactively prepare for the effects of cold weather on equipment
deemed important to safety (Section 01.4).
. The licensee did not compiete a 10 CFR 50.5g evaluation for the change in the status of
the normal ventilation fan to the shared EDG room. This was considered a failure to
maintain the plant configuration in accordance with the Updated Final Safety Analysis
Report (UFSAR) (Section O2.1).
. The inspectors were concemed with the quality of system engineering walkdowns which
could result in conditions adverse to quality not being identified. (Section O2.1).
Maintenance
2 During a surveillance test, the inspectors noted the procedure adherence policy was not
followed. The inspectors also found problems with the use of independent vertfication
methods (Section M1.2).
. The licensee d!scovered that all three EDGs had out-of-tolerance time delay relay
settings. The maintenance work history showed that preventive maintenance to calibrate
the relays was not performed per the established schedule, and relays in the past had
exhibited a high failure rate (Section M1.3).
. Lack of control of foreign material (electrical tape) resulted in a condition which could
have adversely affected EDG performance and may have contributed to 'he failure of the
EDG to start. The licensee's investigation of this particular aspect of the event was
initially incondusive and did not include recommended corrective actions to prevent >
recurrence (Section M4.1).
2
4
'
.
- 1he inspectors identified that weaknesses existed in documenting work history
informatum previously provided by a vendor. In addition, wiring errors resulted in a
safsty-is;eted system being inoperable longer than scheduled (Section M4.2).
- The quality control (QC) organization implemented overview inspections to replace non-
mandatory hold points in maintenance activities. The Quad Cities Maintenance staff did
not substitute the verificaten of hold point activities with maintenance supervisor
verifications as initially intended by management (Section M7.1).
Engineerina
- The lack of follow-through on long-term EDG improvement plans was a weakness made
more significant by recent EDG failures (Section E1.2).
- The inspectors concluded there were instances where engineering evaluations were not
completed or were not of sufficient quality in areas such as snubber requirements, EDG
ventilation, and electromatic relief valve vibration issues. However, the inspectors noted
instances where engineering support of maintenance activities for the radweste system
was good and some aspects of troubleshooting for an electromatic relief valve failure
were good (Section E2.1).
I
I
I
3
.
.
Report Details
Summary of Plant Status
Both Unit 1 and Unit 2 were in cold shutdown during e.:!nspection period. The licensee
continued to develop the safe shutdown analysis and implementing procedures for both units
associated with 10 CFR Part 50, Appendix R " Fire Protection."
1. Operations
01 Conduct of Operations
01.1 General Comments (71707)
During the period, both the inspectors and the licensee identified configuration control
deficiencies, human performance issues, and continued problems with emergency diesel
generator (EDG) dependability and operability. All three EDGs experienced relay failures.
The Unit 1 EDG failed to start due to unkncwn causes.
Plant configuration was not maintained in accordance with operating valve lineups or in
accordance with the Updated Final Safety Analysis Report (UFSAR) in some cases. One
configuration error involved tagging the shared EDG ventilation room fan out of service
without evaluating the condition in accordance with 10 CFR 50.5g. Some of the
configuration problems were attributed to human errors; such as the Unit 2 EDG
becoming inoperabie due to a mispositioned cooling water valve. A second example '
involved a valve out of position in both the Unit 2 and shared EDGs, which did ed affect !
system operability. Both the inspectors and the licensee identified weaknesses in 1
operators implementing independent verifications. Also, operator control of system lineup !
configuration and use of system lineup procedures was weak in one instance. !
01.2 C Jerator Performance Durina Unit 1 Ememency Diesel Generator Autostart
a. Inspection Scope fg2700)
'
The inspectors reviewed the circumstances surrounding an unexpected start of the Unit 1
EDG. The initial investigation revealed that the EDG had initially failed to start after
receiving a start signal, but then unexpectedly started after operators cleared the alarms.
The inspectors reviewed the human performance aspects of this event. Other issues
regarding the EDG failure to start are covered under Sections M4.1 and E1.2.
b. Observations and Findanas
On January 5,1998, during the completion of logic testing involving the core spray
system, an electrician inae.ortently bumped the 1-1430127B relay which provided a start
signal to the Unit 1 EDG. Operators in the control room received two alarms immediately,
"DG [ diesel generator) 1 Trouble" and "DG 1 Autostart/Autostart Block," which were
indicative of an autostart demand. Sixteen seconds later operators received the alarm,
"DG 1 Fall to 8 tart," indicating that the EDG had failed to start.
4
_ _ _ _ _ _ _ _ _ _ _ _ _
.c
.
)
The coniiGI rut. operator directed an equipment operator (EO) to reset the EDG while
unaware that a start signal remained in the circuitry. The reset caused an unplanned
start of the EDG. Following initiation of the start signal, some operators were not aware
that the diesel had attempted to start, but failed to start. At this point, all actMties in the
control room were not stopped for a briefing as expected by operations departmord
management for abnormal events. Stopping and investigating the cause of the alarm I
condition, rather than resetting the EDG alarm, would have helped operators avoid an
inadvertent EDG start and might have helped to identify the cause of the EDG start
failure.
The licensee's root cause investigation into operator performance during this event waJ
thorough and the licensee determined that communications between the unit supervisor,
licensed operators, and electricians performing the core spray system logic testing was
inadequate. For example, control room operators were unaware that an electrician had '
bumped the relay causing the EDG start signal until approximately 15 minutes after the
alarms were received. Also, communications between the control room operator and the '
unit supervisor were not sufficiently clear to prevent the autostart of the EDG. The root
cause of the inadvertent start was determ8ned to be misinterpretation of a procedure by
the control room operator. The abnormal procedure for an EDG start failure used by the
operators covered " valid" start signals, whe eas the signal generated was " invalid." The ]
root cause team concluded that no procedure covered this situation, and the correct
response was to stop and initiate actions to investigate the source of the alarms.
Corrective actions included training, ccunseling, and procedure revisions.
Earty in the event, the unit supervisor indicated that alarms received at the onset of this
event were similar to an r.larm pattom operators had previously observed during auxiliary
.
power transfers. This alarm pattom falsely indicated an EDG autostart and had been
observed oy operators bth in the plant and in the simulator. During the unit supervisor's
response to the similar alarm pattom observed in this event, he incorrectly believed that
the EDG should not have attempted to start. The inspectors questioned whether the
licensee had performed an assessment of the alarm circuitry to evaluate whether the
erroneous alarm pattem could be corrected. The licensee had rd done so, but rather
had accepted that this misleading alarm pattem had been adequately assessed in the
past. Subsequent to the inspectors' question, the licensee initiated an effort to determine
the validity of the existing circuit condition.
c. Conclusion
inadequate communications between the control room operators and test personnel and
the misinterpretation of a procedure step by a control room operator led to the
unexpected start of the Unit 1 EDG. The licensee's investigation of operator performance
during the event was good and revealed these problems. The root caue investination
team did not examine the erroneous alarm pattom normally associated with ine transfer
of auxiliary power which contributed to improper operator response in this event.
4
5
,
9
.
01.3 Emernancy Diesel Generator Coolina Water Valve Mispositionina
a. Inspection Soone
The inspectors reviewed the circumstances surrounding an EDG cooling water valve
found in an incorrect position.
b. Observations and Findinos
On December 10,1997, Unit 1 was operating at rated power and Unit 2 was in cold
shutdown when the Unit 2 EDG cooling water heat exchanger supply valve (2-3999-87)
was retumed to service (RTS). This valve controlled flow to the EDG heat exchanger and
was specified in the RTS checklist to be throttled 18.5 tums from the full open position.
Verification of the minimum cooling water flow of 900 gpm, as required by Quad Cities
Operating Surveillance (QCOS) Procedure 5750-09, " Room and DGCWP [ dies,el
generator cooling water pump) Cubicle Cooler Monthly Surveillance," was not directed in
the RTS. On December 23,1997, during performance of the monthly surveillance,
QCOS 5750-09, the cooling water flow was found to be about 600 gpm, or about
300 gpm less than the required 900 gpm. As a result, the Unit 2 EDG was declared
inoperable. The licensee issued Licensee Event Report (LER) 1-98-003 and Problem
Identification Form Q1997-04970 to report and address this issue.
The licensee's investigation found that the throttled Unit 2 heat exchanger supply valve
had been incorrectly positioned during performance of the RTS on Decerser 10,1997.
On December 16,1997, the Unit 1 EDG was made inoperable for routine maintenance.
On January 10,1998, the licensee discovered that neither the Unit i nor the Unit 2 EDGs
were operable from December 16,1997, through December 21,1997. The inoperability
of both unit EDGs rendered both standby c::s treatment (SBGT) subsystems inoperable
from December 16,1997. to December 21,1997. Unit 1 was placed in cold shutdown on
December 21 due to safe shutdown concems. Unit 2 remained in cold shutdown during
this period.
Technical Specification (TS) 3.7.P.2 stated that with both SBGT subsystems inoperable
in Operational Modes 1,2, or 3 restore at least one subsystem to operable status within
one hour, or be in at least hot shutdown within the next 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> and in cold shutdown
within the following 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />. Following the concurrent inoperability of the Unit 1 and
Unit 2 EDGs and consequently both SrGT subsystems on December 16, failure to place
Unit 1 in hot shutdown within 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br />, and in cold shutdown within the following
24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> was a Violation (50-254/9702841) of TS 3.7.P.
The licensee identified the cause of this event as inadequate procedure development and
review in that a flow test was not performed to verify that the correct flow was established i
following repositioning of the throttled valve. Corrective actions included correctly
positioning the cooling water valve, revising the EDG operating procedure and the RTS >
procedures to require direct verification of correct flow in accordance with QCOS 5750-09 ,
following repositioning of the valve, conducting a search for similar positioning '
discrepancies, providing direction when conducting out-of-service and retum-to-service
activities to verify correct flow of throttle valves, and providir:g training to affected
personnel. The licensee found that the EDG cooling water throttia valves were the only
6
~
..
'
.
1
throttled valves that were specified in the out of servios system as requidng repositioning 1
to a reco! red number of tums rather than to catisfy a particular parameter (firm, pressure !
etc.).
The inspectors identified that this problem could apply any time a throttled valve was
taken out of the normal position and may not be limited to the out of service systwn.
Therefore corrective actions aimed at the out of service system alone, did not appear
complete. At the end of the period, the licensee had not addressed the need to control
throttled valves which were not being taken out of service but were being repositioned. A
training session for operators was planned for May 1998, but that would not preclude
problems in the interim. This issue of controlling valve lineups for important systems
applied to other problems with controlling configuration of system parameters seen in this
period (Section 02.1.b.1).
The inspectors also found that improper positioning of the throttle valve by an initial
positioner and an independent vedfier also contributed to the event. Better attention to
detail by either of the two operators could possibly have prevented this condition. Other
problems with independent verification activities are identified in Sections 02.1 and M1.2.
c. ConclusiQD
Two oporators using the licensee's independent verification methods failed to establish
correct cooling water valve position for the Unit 2 EDG. The Unit 2 EDG and the
equipment supported by it were thus rendered inoperable during some of the time that the
Unit 1 EDG was also inoperable. In addition, operators declared the Unit 2 EDG operable
without verification that the required cooling flow to satisfy EDG operab"3 had been
established. Adctional problems with independent verification and syraem inneup control
are identified in other sections of this report.
01.4 Inadeouste Actions for Addressina Cold Weather Effects on Eauipment
a. Insoedion Scope (71707. 92700)
The inspectors reviewed the licensee's problem identification forms addressing cold
weather effects on equipment. The inspectors spoke to knowledgeable individuals,
toured the affected areas and reviewed the licensee's corrective actions.
b. Observations and Findinas
With both units shut down, discharged circulation water was not warm enough to ensure
the circulating water discharge canal remained ice-free. The altemate fire pumps took
suction from the circulating water discharge canal. The licensee identified the discharge
canal had iced over and could not positively ensure the availability of these pumps during
a postulated fire. On January 12, operators declared the suction to the attemate fire
pumps (also known as " Rainbow" pumps) inoperable.
As corrective actions, the licensee installed a portable air compressor on the rainbow
pump suction lines to ensure the suction lines remained free of ice. The licensee
changed the rainbow pumps operating procedures and winterization program to
incorporate the corrective actions.
7
<
.
'
.
Operations supervisors required operators to record hourty the temperature in both units'
battery rooms to ensure the room temperature would not decrease below 65 degrees F.
Due to decreasing room temperatures, operators requested maintenance personnel to
repair room heaters and requested that engineering personnel develop a temporary
alteration to install portable heaters in the battery rooms. However, on January 14,
operators identified that the temperature in the Unit 2125 Vdc battery room had dropped
to below 65 degrees F. The licensee declared the Unit 2125 Vdc batteries inoperable
due to low battery electrolyte temperature. The licensee concluded that operations
supervisors failed to take ownership of the issues and failed to resolve the issues with
other organizations in a timely manner.
For corrective actions, the licensee installed room heaters for both units' safety-related
125 Vdc battery rooms. The celd weather checidist incorporated the use of this
temporary alteration for future cold weather operations. ;
c. Concluuons
The inspectors concluded the licensee did not proactively prepare for the effects of cold
weather on equipment deemed important to safety in two cases. In a previous inspection
report (Inspection Report No. 50-254/97021; 50-265/97021) the inspectors concluded
station support for the cold weather program was lacking.
O2 Operational Status of Facilities and Equipment
O2.1 Enaineered Safety Feature System Walkdowns
a. lagooction Scope (71707)
The inspectors used inspection Procedure 71707 to walk down accessible portions of
EDG and supporting auxiliary systems; an engineered safety feature system. The
inspectors compared the as-found condition of the EDG and supporting auxiliary systems
and components with the system drawings and the UFSAR. Identified discrepancies
were reported to the responsible organizations for disposition. Potential conditions
adverse to quality were documented on problem identification forms (PlFs).
b. Observations and Findinas
!
The inspectors identified various configuration-related discrepancies during the walkdown )
of the EDG and auxiliary systems. Some of these configuration discrepancies were
attributed to human performance problems, including improper independent verification.
The inspectors identified the following items during a walkdown of the EDG systems. i
b.1 Valves Found Out of Position in Ememency Diesel Generator Air Start System
The inspectors identified that two valves in EDG air start systems were out of position.
Both the Unit 2 and Unit % EDG air start system moisture separator drain valves (% and
2-4699-200) were ide.'4ified by the inspectors to be in the closed position. The Quad
Cities Operations M9anical (QOM) Procedure 6600-1 and mechanical drawings for the
system, required the valves to be open. Operators repositioned both Unit % and Unit 2
8
..
'
.
4699-200 valves to the proper position and venfied EDG supporting system valve
positions. No other discrepancies were noted.
The licensee documented the condition on PlF Q1998-00506 and commenced an
investigation. The investigation revealed that on October 28,1997, two reactor
operator-qualified individuals independently verified a retum to service position for
Valve 2-4699-200 in the closed (incorrect) position. The error was not identified until a
walkdown of the system by the inspectors on January 29,1998. The investigation could
not determine the cause of the %-4699-200 valve being out of the required position.
Engineering personnel later determined the system was operable with the valves in the
closed position. The inspectors determined Valves %-4699-200 and 2-4699-200 being
out of position was a violation of minor significance and is being treated as a Non CNed
Violation (50-265/97028 02) consistent with Section IV of the NRC Enforcement Policy.
The inspectors also noted there were no requirements to periodically perform valve line
ups for all system valves. Typically before startups from refuel outages, operations I
supervisors would authorize valve lineups to be performed in accordance with system l
operating procedures. However, these procedures did not include all system valves such
as the two 4699-200 valves. Other problems with control of system lineups were
identified in Section 01.3. As a result of this and other events in operations,
management discussed recent operator errors with all operations personnel.
b.2 Chance to the Plant Without a Completed Safety Evaluation
The inspectors identified an out-of-service (OOS 15580), initiated on August 30,1994,
removed the normal ventilation fan to the % EDG room from service. However, the 4
UFSAR, Section 9.4.5.B stated the fan supplied air to the room at 1200 ft'/ minute. The l
package associated with OOS 15580 had a 10 CFR 50.59 safety evaluation screening
initiated, but not completed, with a note stating that a full 10 CFR 50.59 safety evaluation
needed to be completed. A nuclear tracking system (NTS) number was initiated to
ensure the engineering commitment was completed. However, the NTS item was closed
out without completing the 10 CFR 50.59 safety evaluation. As a result, the licensee
changed the plant from what was described in the UFSAR for over three years without a j
complete safety evaluation.
The inspectors considered the closure of the NTS item without the actions being
completed to be an example of poor corrective action follow-up by both the
engineering and operations organizations. The licensee documented the concem on
PlF Q1998-00531. The inspectors considered this to be a Violation (50-265/97028-03)
of 10 CFR 50.59.
b.3 Inadeounte Documentation of Emeraericy Diesel Generator Fuel Oil Pipina Support !
The inspectors identified the fuel oil retum line to the day tank from the % EDG did not
appear to be adequately supported. A subsequent review of all EDG fuel oil retum lines
by a structural engineer revealed that there were no supporting calculations to ensure the
safety-related piping could remain intact after a seismic event (PlF Q1998-00559). The
inspectors consider this an Unresolved item (50-254/97028 04; 50-265/97028-04)
pending review of the licensee's operability evaluation of this condition.
9
.
.
b.4 Tamper Seals Missino From Relief Valves
The inspectors identified a relief valve on an EDG sir receiver without a tamper seal. The
licensee similarly identified various other relief valves that also were missing tamper seals
(PIF Q1998-00527). The licensee was addressing this potential programmatic problem at
the close of the inspection period.
b.5 System Engineering Walkdowns
The licensee administratively required periodic walkdowns of systems by system
engineers in an attempt to detect potential conditions adverse to quality. The issues in
b.1 - b.4 were identified by the inspectors and had not previously been identified by
system engineering walkdowns. As a result, the inspectors questioned the effectiveness
of system engineering walkdowns.
c. Conclusions
The inspectors concluded errors by two individuals conducting a retum to service resulted
in Valve 2-4699-200 being out of the required position. The cause of Valve %-4699-200
being out of position was unknown, but the inspectors found that a lack of system line-up
procedure requirements may have contributed to both valve lineup problems. The EDG
and the associated air start system were dctermined to be operable with this condition.
This event was indicative of a weakness in independent verification of required valve
positions. Other examples of weak independent verification are discussed in
Sections M1.2 and O1.3.
The inspectors concluded the licensee did not complete a 10 CFR 50.59 evaluation for
the change in the status of the shared EDG room ventiistion fan. This failure to maintain
the plant configuration also revealed poor follow-up by operations and engineering staff
on design related questions.
The inspectors were c ancemed with the effectiveness of system engineering walkdowns
in identifying conditions adverse to quality.
.08 Miscellaneous Operations issues (92700)
08.1 (Closed) Inspection Follow-up item (50-254/94004-57: 50-265/940QtJ_7J; integrated
Reporting Program. This program was well established, generating several thousand
problem identification forms per year. Poor rat cause identification and poor trending of
issues had been identified in several 1997 NRC inspections such as the maintenance rule
team inspection (Inspection Report No. 50-254/97017; 50-265/97017). These issues will :
be reviewed generically in following up on the items generated by those inspections and
as part of the core resident inspection program. This item is closed.
08.2 (Closed) InsMion Follow-up item (50-2-6 Site Quality
Verificat on Organization and Effectiveness. The licensee made a number of changes to
the organization in attempting to improve effectiveness, including a recent change to the
reporting structure. This was approved by the NRC as Revision 65 H to the quality ,
10
..
'
.
assurance topical report. Some problems still exist with effectiveness of the organization
in identifying and forcing corrective actions. Rose issues are better tracked by the NRC
using inspection Procedure 40500 and other core inspections. His item is closed.
08.3 (Closed) Violation (50-254/94029-01a and 01b: 50-265/94029-01a and 01b): Inadvertent
Reactor Vessel Draining. An out of service error caused inadvertent draining of the
reactor vessel when a scram was inserted with the unit shutdown. Out of servios errors
have decreased at the station since this event occurred and a revised out of service
program provided better controls to prevent such an occurrence. The inspectore
continued to monitor 008 errors at the station (see Section O2.1.b.1). This violation is
closed.
08.4 (Closed) Inspection Follow-yo item (50-254/96002-04: 50-265/96002-04): Weak Log
Reviews. Weak log reviews resulted in decreased safety system availability. The control
room emergency ventilation system (CREV) was unavailable due to Freon leaks. The
operator log reviews did not detect the problem in a timely manner. Since this ,
occurrence, the licensee completed maintenance achvities and modifications to improve j
system performance. The CREV system performance was the subject of enforce.,wnt in
inspection Report No. 50-254/96017; 50-265/96017. No similar failures have occurred
since that time. This item is closed.
08.5 (Closed) LER 50-254/96005-00: The CREV System inoperable Due to Low Outside Air
Temperature. The outside air temperature at -29 degrees Fahrenheit was found to be
below the TS limit of -28.1 degrees Fahrenheit for the CREV filtration heater operabikty.
Outside air temperature increased above this record low temperature and the system was
declared operable. The licensee also changed th: TS method for determining heater
operability (measure heater power versus an absolute low temperature limit when the
upgraded TS was implemented). This item is closed.
08.6 (Closed) LER 50-254/96017-00: Manual Scram During Reactor Startup. The scram was
followed by an unplanned opening of all main turbine bypass valves. The bypass valves
opened when the condenser backpressure permissive was met and the Electrohydraulic
Control (EHC) system pressure setpoint was effectively above reactor pressure.
Operators did not fully understand the inherent inaccuracies in the calibration of the EHC i
system pressure transducers at low pressures and the procedure was inadequate. The !
inspectors previously reviewed this event in inspection Report No. 50254/96012; I
50-265/96012 and concluded that no v'alations had occurred. Since the event, the I
startup procedure was revised to direct operators to establish condenser vacuum prior to ;
decreasing the EHC pressure setpoint. This would allow for the controlled opening of l
bypass valves at low pressures. Operators and engineers were also trained on the event
and the details of the EHC system operation at low pressures. No other similar events
have occurred during subsequent reactor startups. This item is closed.
08.7 (Closed) LER 50-254/97022-00. The Control Room Emergency Ventilation System
(CREVS) Inoperable. The CREVS was declared inoperable due to inadequate cooling "
!
water flow which was caused by a system perturbation. Cooling water flow rate through
the refrigeration condensing unit was recorded as 115 gpm and the performance
acceptance criterion was 120 gpm. The licensee established the probable cause as
11
'
..
,..
lifting of a relief valve during the test. The procedure was changed to check the status of
the relief valve dudng the test. The surveillance was satisfactorily performed and the
problem has not recurred. This item is closed.
ILKalntenance
M1 Conduct of Maintenance
M1.1 Oganfal Comments
During the inspection period, the inspectors noted problems associated with maintaining
the EDGs. Time delay relay problems contributed to inoperability of all three EDGs. The
'
Unit 1 EDG was inoperable for longer than expected due to problems with planned
maintenance. The Unit 2 EDG was also inoperable at the same time due to a cooling
water flow control valve being out of position.
Additionally, maintenance workers failed to verify removal of foreign material exclusion
barriers on an EDG component which could have affected operability. At the same time,
1,1e inspectors found that responsibility for verifying such maintenance hold points as
cleanliness was being transferred from Quality Control inspectors to the maintenance i
staff, without the maintenance staff taking over the function in some cases.
M1.2 Surveillance Observations
a. inspection Scope (61726)
The inspectors observed or reviewed portions of the following surveillance tests:
'
- Standby Liquid Control (SBLC) Check Valve Operability Test at Cold Shutdown,
QCOS 1100-03 (Unit 1)
. Main Steam Line Radiation Monitor Functional Test, QCIS 1700-01 (Unit 1)
b. Qhservations and Findinos
1
The inspectors noted two concems during the SBLC check valve testing for Unit 1 on ,
'
January 6,1998. During restoration of the system per QCOS 1100-03, the inspectors
questioned why operators performed Step I.3.h before Steps 1.3.e, f, and g were
completed. The licensed operator and unit supervisor explained that system restoration
(Steps 1.3.b, c, and d) and independent verification of valve lineup (Step I.3.h) were done
at the same time. Therefore, Step I.3.e, f, and g were done after Step 1.3.h was
completed.
The inspectors were concemed that the procedure steps were performed out of
sequence. Quad Cities Administrative Procedure (QCAP) 1100-12, " Procedure Use and
Adherence," Revision 13, SeWion D.4.c(5) specified that procedural steps must be
followed in sequence unless deviations are allowed by the procedure. Also,
Section D.4.c(7) specified that procedural revisions be obtained when the existing
procedure was found to be incorrect. Althcugh there were no apparent adverse
12
.
.
consequences associated with performing the steps out of sequence in this case, the
inspectors were concemed that the procedure adherence policy was not followed. After
further discussion with plant staff, the licensee reperformed Step 1.3.h and documented
the occurrence on PlF Q1997-00051. This minor violation is considered to be a Non-
Cited Violation (50-254/9702845) consistent with Section IV of the NRC Enforcement
Policy.
The inspectors were also conoemed that the opemtors worked together during the valve
lineup restoration and independent verification. Quad Cities Administrative
Procedure 0230-05, " Independent Verification," Revision 5, Section D.1.b specifies,
" Verifier independence must be maintained to ensure the integrity of the independent
verification. When possible, then actual separation of the individuals should be utilized."
Other sections of QCAP 0230-05 allow the use of either " Apart-in-Action" or
" Apart-in-Time" ve.tfication. The inspectors were concemed with the effectiveness of
independent verification since the procedure did not clearly state which type of verification
to use for which types of evolutions. Also there appeared to be some conflict within the
procedure regarding when separation of individuals should be utilized.
c. Conclusions
The inspectors noted two concoms during the SBLC check valve testing. The procedure
adherence policy was not followed, and the effectiveness of independent verification was
riuced when actions were performed and verified together. Other independent
verification problems this period are identified in Sections 01.3 and O2.1.b.1.
M1.3 Relav Failures Render Multiple Ememency Diesel Generators Inoperable
I
a. Inspection Scope (62707)
The inspectors reviewed the circumstances surrounding a December 18,1997, ENS
report regarding the inoperability of EDGs due to out of tolerance as-found settings of
time delay relays,
b. Observations and Findinas i
During preventive maintenance on the Unit 1 EDG on December 18,1997, the TD-2 relay
was found to be out of tolerance. This relay was intended to limit the stait crank cycle of
the EDG to 15 seconds. The acceptance criterion was 15.0 - 16.5 seconds. The UFSAR
and TS required that the EDG start and accelerate to full speed within 10 seconds. The
UFSAR also stated that if the engine does not reach 200 rpm in 15 seconds that the EDG
start failure logic actuation will occur. The concem with an out of tolerance TD-2 relay ;
was that premature actuation could trip the EDG when a successful start may have
occurred, and late actuation could allow starting air to deplete or damage to occur from
excessive cranking.
The initial test timed the relay at 6 seconds. A second and third test timed the relay at
12.61 seconds and greater than 50 seconds, respectively. A new relay was tested and
installed, and PlF Q1997-04891 was generated. Electricians then tested the same relay
13
,
.
on the shared EDG. The shared EDG TD 2 relay timed at 11 seconds. The Unit 2 EDG
TD-2 relay had been previously tested on December 9,1997, and found to be out of
tolerance at 13.85 seconds.
An ENS call was required due to the simultaneous inoperability of the Unit 1 and the
shared EDGs. However, the inspectors noted the TD 2 relays for all three EDGs were
found outside the acceptance criterion. For some unknown period of time, until the relays
were replaced in December 1997, all three EDGs may not have been able to perform
their safety function due to a common failure mode of EDG relays.
Prior to 1995, the calibration check of the TD 2 relay was performed via a work request
during refueling outages. Since 1995, a procedure, Quad Cities Electrical Preventive
Maintenance (QCEPM) 0700-18, " Calibration of Diesel Generator Time Delay Relays,"
was used and the calibration check was scheduled to be performed once every 18
months. The work history indicated that since 1991, all but one of the checks were past
due when performed. The longest period between calibrations was 34 months. The
longest period before calibration since the recent failures, was 25 months. The relay
vendor manual referenced yeady checks of the relays. Of the 12 test results reviewed,
8 relays failed the test and required calibration, or in most cases, replacement. In one
recent case, the corrective action for a TD relay failure was to replace the relay and close
the PlF for trending; even though an adverse trend was already established. No other ;
corrective action to address the root cause of the failure was specified until later failures
occurred.
The inspectors reviewed the LER (50-254/97027-00) for this event. The root cause was
determined to be relay drift due to lack of proper setup testing. Prior to December 1997,
the test was conducted as a bench calibration rather than under normal operating
conditions. Corrective actions included sending the relays offsite for further testing and
failure analysis, and testing the installed relays monthly to trend results and determine if
the frequency of the preventive maintenance (PM) needed to be changed. These
corrective actions appeared to be appropriate, but were not planned to be completed until
after the current inspection period.
c. Conclusion
The licensee discovered that all three EDGs had out of tolerance time delay relays. The
likely cause was an improper test setup. However, maintenance work history showed
that preventive maintenance to calibrate the relays was not performed per the established
schedule, and that the relays in the past had exhibited a high failure rate that was not
recognized by plant personnel. Later, corrective actions, including monthly testing to
trend performance and reestablish the PM frequency, appeared to be appropriate.
However, some eariier failures did not receive rigorous root cause reviews.
M1.4 Maintenance Observations
a. Inspection Scope (61726)
The inspectors observed mechanical maintenance department personnel during the
overhaul of the high pressure coolant injection system air operated Valve AO 1-2301-29
in accordance with Nuclear Work Request 970125804-01.
14
i
!
..
.
b. Observations and Findinas
The work statNm Was orderty, and workers performed the valve overhaul activities in
accordance with a work procedure. The inspectors observed that a maintenance
foreman frequented the work site to coach personnel conducting the overhaul activities.
During review of the work package, the inspedors identified th:d an attachment to the
work package, called a " red sheet," was inadequately completed. The " red sheet" was an
administrative control, used in the maintenance work package, to assure an adequate
level of management review was done prior to work being performed on critical plant
systems and components. This was an administrative weakness previously observed by
the inspedors and documented in NRC Inspection Report No. 50-254/95006; .
i
50-265/95006. This example indicated that the licensee had not achieved a high level of
attention to adminis'.rative detail. There were no adverse consequences resulting from
these infrequently observed omissions, and no impad to safety.
c. Conclusions
Supervisory oversight was good during the observed work. Some lack of attention to risk
reduction administrative detail was observed (inadequate completion of a " red sheet").
]
M4 Maintenance Staff Knowledge and Performance
M4.1 Electrical Taoe Found on Ememency Diesel Generator Air Start Solenoid
a. Inspection Scope (62707)
The inspectors reviewed the root cause evaluation (RCE) for the EDG start failure.
b. Observations and Findinas
During the RCE, the system engineer identified black eledncal tape protruding from the
air start solenoid. Upon disassembly, tape was found covering the air ports. The
solenoid had been replaced on December 16, igg 7. Subsequent to the solenoid
replacement, the EDG had been successfully run five times. The tape was found
punctured from the force of air during previous start attempts. Electricians performed
bench testing of a similar solenoid with tape covering the ports and found that the force of
the air punctured 'w tape during all attempts. The tape, while not ruled out as the cause
of the EDG failure, was not considered to be the likely cause. Further tests of the
solenoid were planned.
Although the presence of the tape was not determined to be the likely root cause of the
EDG failure, the potential safety effeds of the failure to property control foreign material
on this device were serious. Foreign material fully blocking the por#2 of the air start
solenoid, would result in an EDG failure to start on demand. Addit 3nally, the RCE was
initially inconclusive regarding how and when the tape was installed and did not address
foreign material exclusion (FME) requirements and possible FME weaknesses exhibited
during the work. Therefore, corrective actions to prevent recurrence were delayed.
15
l
l
..
'
.
c. Conclusion
Lack of control of FME (electrical tape) resulted in a condition which could have adversely
affected EDG performance and may have contributed to the failure of the EDG to start.
The licensee's investigation of this particular aspect of the event was initially inconclusive
and did not include recommended corrective actions to prevent recurrence.
M4.2 Maintenance Problems with the Toxic Gas Analyzer
a. Insocction Scope (g2700)
The inspectors reviewed problem identification forms and spoke to various engineering
and rr.aintenance personnel associated with maintenance activities with the control room
emergency ventilation system (CREVS) toxic gas analyzer (TGA).
b. Observations and Findinos
The licensee identified problems with the TGA. Maintenance personnel identified the
TGA chopper motor needed to be replaced. The new chopper motor was installed but
wired incorrectly. Troubleshooting by maintenance personnel could not identify the
problem. A vendor was summoned to the site to assist in troubleshooting efforts.
Previously, vendors assisting maintenance personnel provided knowledge about TGA
system operation and preventive maintenance. A maintenance worker documented the
vendor findings in work history, but did not notify work analysts for inclusion into future
work packages. When the recent chopper motor failure occurred, the work analyst failed
to include the previous work history in the packags. The analyst indicated the reason
was because the package was destined for fix-it now (FIN) team work, and history was
noi usually included in those packages. The licansee counseled the individual to ensure
work history was included in FIN team work ps.* ages. This event resulted in the CREV
TGA being inoperable longer than anticipated.
c. Conclusions
The inspectors identified that weaknesses existed in documenting work history
information previously provided by a vendor. In addition, wiring errors resulted in a j
safety-related system being inoperable longer than scheduled. l
M7 Quality Assurance in Maintenance Activities
M7.1 Quality and Safety Assessment involvement in Maintenance Activities
a. Inspection Scope (62707)
The inspectors reviewed the involvement of the Quality and Safety Assessment (Q&SA) _
department in two maintenance activities.
16
..
,.-
b. Observations and Findinas
A Quality and Safety Assessment inspector generated a PlF documenting three concoms
during an overview of maintenance work on the shared EDG TD 2 relay (see
Sechon M1.3). The concems included the method used to change acceptance criteria in
the work package, lack of proper work package documentation, and differences among
procedures regarding the acceptance criteria. The licensee determined that the PlF
required an apparent cause evaluation (ACE). The inspectors reviewed the ACE and
noted that it was closed with corrective actions being subsumed by those described in the
LER that was required due to multiple EDGs affected by TD 2 relay problems. The LER
did not address any of the original concems documented by the Q&SA inspe clor. The
Q&SA organization ultimately wrote a second PlF to document the inadequa e ACE
closure following the inspector's discussion of the concem with licensee management.
While reviewing work packages on the EDG, the inspectors noted that the quality control
(QC) hold points had been deleted from the procedure via a procedure field change
(PFC) to Quad Cities Mechanical Maintenance Surveillance (QCMMS) 6600-03,
Revision 7, " Emergency Diesel Generator Periodic Pmventive Maintenance inspection."
The QC organization revised the QC inspection plan to delete all but mandatory hold
points required by various procedures, codes, or regulations. In lieu of hold points, the
QC organization planned to perform more overview inspections. Station management's
intention was to convert these QC hold points to maintenance verifications performed by
supervisors. However, procedures had been changed to allow QC hold point deletion,
but did iot add the maintenance supervisor verification. The work package associated
with the EDG had several examples where the verification was performed by the same
individual who performed the work, one example where the supervisor did the verification,
and one example where no verification was documented.
While in this case, no problems appeared to result from the removal of the QC hold
points, the inspectors noted that one function designed to prevent conditions adverse to
quality had been removed rather than transferred to a different organizabon. Events such
as the tape found on the air start solenoid (Section M4.1) indicated the continued need
for such checks of critical maintenance work.
c. Gnaglualgn
The QC organization implemented overview inspections to replace non-mandatory hold
points in maintenance activities. The hold points were intended to be replaced with !
maintenance supervisor verifications, bet on one occasion, QC concems identified during
an overview were not addressed by the station. The inspectors also noted that former
QC hold points were not translated into maintenance supervisor verifcations during EDG ,
work.
M8 Miscellaneous Maintenance issues (g2g02) ,
!
M8.1 (ClosedH.ER 50-254/93025-01: "A" Loop Main Steam isolation Valves (MSIVs)
Exceeded TS Leakage Limits. The cause of the excessive leakage was steam erosion
on the pilot and main disc. The fasteners which retain the main valve disc to the valve
stem were also found to be loose. The inspectors verified the licensee commitments to
develop a maintenance procedure for disassembly and reassembly of MSIVs, to review
17
.
'
.
the existing prevenGwe maintenance program, and to ensure that all disassembled M81Vs
have new locidng devices installed, were complete. Recent local leak rate test results
were satisfactory. This item is closed.
M8.2' (Closed) Inspection Follow-up item 50-254/94004-05: 50 265/94004-05- Repetitive
Oxygen Analyzer Problems. The safety-related containment atmosphere monitor (CAM)
system was used for de-inerting the drywell because of repeated problems with the non-
safety related oxygen analyzers. The oxygen analyzer sample pumps were replaced.
Over the last two years oxygen analyzer reliability had improved. No outstanding work
requests existed for the system. This item is closed.
MS.3 (Closed) Inspection Follow-up Item 50-254/94004-06: 50 265/94004-06: Back Leakage
Through 1-RHR-7B Valve. The licensee replaced or repaired three of four of the RHR-7
[ residual heat removal] valves. In January 1998 control room logs indicated that a
leakage of about 1 to 2 gallons per minute into the Unit 1 torus through a RHR torus
cooling valve had been observed. This was being tracked in the licensee's corrective
action system. This item is closed.
M8.4 (Closed) Inspection Follow-uo item 50-254/94004-27: 50-265/94004-27: Core Spray
Pump Discrepancy. The licensee minimized leakage into the torus through repairs and
replacements cf the 1A Core Spray pump, and brought pump performance more in line
with the 1B Core Spray pump performance. High pressure coolant injechon pump
performance was also an issue, and was evaluated in the System Operational
Performance inspection 50-254/97022; 50-265/97022. This item is closed.
M8.5 (Closed) Inspection Follow-up item 50-254/94004-34: 50-265/94004-34: Work Package
Preparation, Utilization and Processing. The licensee implemented a number of
initiatives to improve maintenance work package quality, completion, and processing.
These initiatives included a Fix-It-Now team concept for minor maintenance which had j
been successful in minimizing the time to process minor maintenance items. A work
control center concept for centralizing work packages, out of services and problem
identification forms was also implemented. Quad Cities switched to an electronic work
control system in 1997 to provide better tracking of work items. This system was
implemented with some flaws which affected the completion of surveillances. In addition,
not all fields for equipment were complete when the system was tumed on. Work
package improvement efforts also took place in 1996 and 1997. In late 1997, some l
problems still existed in the amount of data being recorded by technicians during work,
and the amount of material history available for certain components (see Section M4.2). l
These issues were being reviewed as part of routine resident core inspections. This item i
is closed.
M8.6 (Closed) LER 50-254/95001-00: During Unit 1 Startup The Reactor Core Isolation
Cooling (RCIC) System Govemor Valve Did Not Respond Property Due to Valve Stem
Corrosion. Dusing testing, the RCIC govemor valve remained full open and would not
respond to flow controller signals. The RCIC system was declared inoperable and the
plant was shut down in accordance with TSs. The l6censee concluded that with the
govemor valve full open, the RCIC system could have performed its design basis
function; and therefore, submitted the LER on a voluntary basis. Corrective actions
included valve stem replacement and changes to operating procedures to reduce the
18
-_-_ _ ___
.
.
exposure to moisture. The inspectors verified that these actions were completed. No
similar RCIC valve failures had occurred sinos this event. This item is closed.
M8.7 (Closed) LER 50-254/95005-00: Control Room Emergency Ventilation inoperable Due To
Refrigerant Leak. A damaged joint in the system was repaired and operator surveillance
procedures were changed to perform trending of compressor refrigerant pressure each
shitt. This item is closed.
M8.8 (Closed) LER 50-254/96004-00 Unit 1 High Pressure Coolant injection (HPCI) System
inoperable Due To Gland Exhauster Breaker Trip. The breaker was replaced and the
HPCI system tested satisfactorily. Other corrective actions included a procedure change
to include breaker trip checks in the preventive maintenance procedure and a
reevaluation of the replacement schedule for the 250 Vdc breakers. The inspectors
vertfied that QCEPM Procedure 400-2, Revision 10, had been revised to incorporate the
trip checks _ The licensee reevaluated the replacement schedule, but determined that
acceleration of the schedule was not necessary. As of January 1996, five Unit 1 breakers
for reactor core isolation cooling system components were scheduled for replacement
during Q1R15 (beginning September 1998). No additional failures of these breakers have
occurred since this event. This item is closed.
M8.9 (Closed) LER 50-254/96014-00 Electrical Distribution Surveillance Did Not Document
Voltages in Accordance With TS 4.9.E. Revised TSs were implemented on
September 23,1996, which required verification of the proper voltage on safety-related
busses. On October 2, a unit supervisor discovered that the surveillance was not
satisfactorily completed as required by TS 4.9.E since there was no documentation that
proper voltages had been verified. In accordance with TS 4.0.C, which allowed 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />
to complete a missed surveillance before taking actions specified by the TS action
statement, operators revised the procedure and completed the surveillance on October 3.
The inspectors confirmed that the procedure had been revised and that operations
personnel were briefed on the event as committed to in the LER. The failure to perform
the required surveillance was a violation of TS 4.0.B. This non-repetitive, licensee-
identified and corrected violation is being treated as a Non-Cited Violation
(50-254/97028 06; 50-265/9702846) consistent with Section Vll.B.1 of the NRC
Enforcement Policy. This item is closed.
M8.10 (Closed) Unresolved item 50-254/97002-07: 50-265/97002-07: Unit 2 Containment
Atmosphere Monitoring (CAM) System Maintenance Rule implementation. During the
maintenance rule baseline inspection, the inspectors reviewed the maintenance rule
implementation for the CAM system and identified several examples of apparent j
violations. Refer to inspection Report No. 50-254/97017; 50-265/97017 for further
details. This item is closed. l
M8.11 (Closed) Unresolved item 50-265/97027-02(DRS). On November 7,1997, the inspectors
identified an unresolved item pertaining to the adequacy of the visual test (VT-2)
examination of the Class 1 system boundary based on the short time taken and methods
used during the June 22,1997, leakage test. The licensee subsequently concluded that
this VT-2 examination was inadequate with respect to Code requirements, based on an
examination re-enactment on January 3,1998. This re-enactment demonstrated that
leakage (if present) would not have been detected in several areas of the Class 1 system
19
_
.
.-
boundary. On January 6,1998, the licensee issued PlF Q1998-0052 to document this
issue.
Technical Specification 4.0.E, required in part " inservice inspection of ASME [American
Society of Mechanical Engineers] Code Class 1,2, 3 components...shall be performed in
accordance with Section XI of the ASME Boiler and Pressure Vessel Code..." The 1989
Edition of the ASME Code Section XI, IWA-5242(a) " insulated Components" required that
"For other components, visual examination VT-2 may be conducted without the removal
of insulation by examaing the accessible and exposed surfaces and joints of the
insulation. Essentially vertical surfaces of insulation need only be examined at the lowest
elevation where leakage may be detectable." Section IWA-5242(b) " Insulated
Components" required that "When examining insulated components, the examination of
surrounding area (including floor areas or equipment surfaces located undemeath the
components) for evkience of leakage, or other areas to which such leakage may be
channeled, shall be required." Contrary to these requirements, during the June 22,1997,
Class 1 system leakage test, the licensee had not completed a complete examination of
accessible sudaces and joints of the insulated Class 1 system boundary. Failure to
meet Code (IWA-5242) requirements for the VT-2 examination of the Class 1 system
boundary performed on June 22,1997, is considered an apparent violation
(FEl 50-265/97028 47(DRS)) of TS 4.0.E. The licer'see root cause evaluation and
corrective actions for this issue were presented to the NRC at the January 9,1998,
predecisional enforcement conference. Licensee corrective actions for this issue were
under NRC review, and no additional response is required at this time. This unresolved
item is closed.
Ill. Enoineerina
E1 Conduct of Engineering
E1.1 General Comments
Problems with safety-related equipment continued, and engineering department plans to
resolve the issues were not completed as planned in some cases. The Unit 1 EDG fr Nd i
to start on demand, with a root v o pursued but not found by the licensee at the en 9f
the inspection period. Several oh, c EDG failures to start on demand have occurred
since 1993. The inspectors identified that previous corrective action plans were not fully
implemented. In addition, electromatic relief valve (ERV) vibration problems from 1993
have not been fully resolved, and the inspectors identified that action plans to replace the
valves were only partially completed. A recent failure of the 3E ERV was attributed to
vibration. Some problems with safety evaluations and operability evaluations were also
noted.
E1.2 Unit 1 Emeroency Diesel Generator Fr"ure to Start on Demand '
a. Inspection Scope
The inspectors reviewed the circumstances surrounding a January 5,1998, ENS call
reporting the unexpected start of the Unit 1 EDG. The initialinvestigation revealed that
the EDG had initially failed to start after receiving a start signal, but then unexpectedly
'
20
,.
started when operators cleared alarms. Section 01.2 describes the inadvertent start. In
order to evaluate the failed start, the inspectors attended root cause team meetings,
reviewed PlFs and root cause reports, evaluated previous corrective actions to EDG
failure events, and spoke with engineers and operators,
b. Observations and Findinas
An inadvertent EDG start signal was generated by an electrician intdvertently bumping a
core spray relay. The starting of the Unit 1 EDG would have been the proper system
response. The cause of the failure to start was not determined. However, the
investigation revealed several problems with the EDG that could have contributed to the
failure to start. Among the problems noted during the investigation were:
. Tape was found covering the air ports of the air start solenoid. The tapo had
probably been punctured by the force of the air during the five successful starts of
the EDG prior to this failure. This issue is discussed further in Section M4.1.
. The air start motor main check valve, Valve 1-4699-309, was found to be stuck
closed.
. The lower air start motor pinion gear was found to be intermittently binding when
rotated by hand.
. One of the upper air start motor mounting bolts was found to be loose.
Because the investigation did not reveal the root cause of the failure to start, the licensee
used an independent EDG expert to review the problem and the investigation to
determine if any possible causes were overlooked. This review did not indicate that any
major issues were missed, but provided some suggestions for further testing.
The inspectors concluded that this problem was a continuation of a high number of EDG
failures, for each of which the licensee had not determined a definitive root cause. As a
result of this concem, the inspectors reviewed the licensee's corrective actions for the
shared EDG failure on January 17,1997, and the Unit 2 EDG failures on May 8,1997,
September 26,1995, and October 24,1995.
The inspectors found that a number of longer term corrective actions established in
response to the failures and as a result of the EDGs becoming a Maintenance Rule (a)(1)
system were delayed or had not been initiated. In particular, a corrective action item after
the May 1997 failure was to evaluate the need for EDG modifications to tolerate air start ,
motor pinion gear abutment. The original due date of November 1997 was extended to l
June 1998. Little progress was made on the modification options until this most recent l
EDG failure event when the licensee decided to more aggressively pursue redundant air !
start motors. (NOTE: Recommendations to enhance air start system performance were
included in a report to the licensee from Sargent and Lundy, and acknowledged in a i
September 22,1993, report by the licensee. See Inspection Reports No. 50-254/97022;
50-265/97022, Section E8.2). Corrective actions from the January 1997 failure event
involved obtaining additional maintenance and inspection guidance on the air start motors
from the vendor, co" sidering additional air start motor inspection requirements, and
21
i
- \
.
evaluating replacement frequency of the air start motors. These actions, originally due to
be completed in July 1997, were still open pending additional testing of the air start
motors by the vendor.
Also, the Maintenance Rule (a)(1) action plan first developed in April 1997 had many
action items which were overdue or had not been started, including such activities as
determining system performance improvement areas and developing action plans to
address those areas. Essentially none of the tasks of the action plan which may have led
to improved EDG performance were completed or even initiated.
!
c. Conclusion
The EDGs had experienced a number of start failures over the past several years and
root cause evaluations did not always identify the root cause of failures. Notwithstanding
I
the lack of definitive root causes, the licensee developed several long-term corrective
action plans designed to improve system performance. However, the plans were often
not completed, indicating a lack of follow-through on long-term EDG improvement plans.
E2 Engineering Support of Facilities and Equipment
E2.1 Enaineerina Support to Operations
a. Inspection Scoce (71707)
The inspectors reviewed PlFs, engineering evaluations, and troubleshooting plans. The
inspectors spoke to engineering and operations personnel and toured the facility.
b. Observations and Findinas
b.1 On December 30,1997, engineering personnel identified and replaced a damaged
snubber on the Unit i decay heat removal system piping. On January 10,1998, the
damaged snubber was tested and failed in the acceleration mode. To meet TS 3.8.F
requirements, operations personnel tasked the engineering staff with performing an
engineering evaluation to determine the cause for the failure within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />.
Engineering personnel presented the evaluation to operations personnel on January 13,
but the operations personnel were not satisfied with the quality of the engineering
evaluation since it did not include a failure analysis of the snubber. The extra time taken
by engineering staff to include the failure analysis in the evaluation resulted in exceeding
the 72-hour TS criteria and required operations personnel to notify the NRC via the ENS.
b.2 On August 30,1994, operations personnel removed the normal ventilation fan to the
shared EDG room from service. Administrative procedures required completing a
10 CFR 50.59 evaluation for the condition since the out of service was to remain in effect
for greater than three months. However, engineering staff did not complete the
10 CFR 50.59 safety evaluation. An NTS item was initiated to ensure the engineering
commitment was completed. However, the NTS item was closed out without completing
the 10 CFR 50.59 safety evaluation (see Section O2.1b.2).
b.3 On December 20,1997, operators tested five automatic depressurization system (ADS)
relief valves by operating a test switch from the control room However, operators
22
_-
-
l
..' )
determined relief Valve 1-0203-3E failed to open (PlF Q1997-04938). The other ADS i
valves operated property. Engineering personnel developed an action plan to determine !
the most likely cause of failure. Subsequent troubleshooting by engineering staff
determined the adjusting screw on the leverxrm on the pilot assembly had vibrated out of
position. Proposed corrective actions included locking the aquating screw and applying a .
stiffening agent to the threads to ensure the adjusting screw would not move after it had i
been set. Engineering personnel determined this was the first time vibrations from
the steam piping had resulted in a movement of the relief valve adjusting screw.
Section 6.3.3.1.4 of the UFSAR (emergency core cooling system (ECCS), Performance !
evaluation of ADS) stated only four of the five ADS valves were required to be operable in
the loss of coolant analyses.
In the past, the licensee had detected other vibration-related failures of ADS valves. The l
licensee replaced the Unit 2 ADS valves during a previous refuel outage but had not
replaced the ADS valves in Unit 1. The inspectors were concemed that the root cause of
ADS valve failures had not been corrected since system vibrations continued to result in
degraded equipment.
b.4 Operators identified indications of a leak in the waste collector tank. A tour of the l
associated room indicated the tank developed an 18-inch crack at a weld joint towards
the bottom of the tank. Engineering personnel developed an action plan to drain and
repair the tank. Engineering personnel proactively addressed similar concems with the
floor drain collector tank, which was a twin to the waste collector tank.
c. Conclusions
The inspectors concluded there were instances where engineering evaluations were not
completed or were not of sufficient quality in areas such as snubber requirements, EDG
ventilation, and electromatic relief valve vibration issues. However, the inspectors noted
instances where engineering support of maintenance activities for the radweste system
was good and some aspects of troubleshooting for an electromatic relief valve failure
were good.
E8 Miscellaneous Engineering issues (92902)
E8.1 (Closed) Inspection Fol19w-up item 50-254/92018: Control Room Emergency Ventilation
Train Surveillance Failure. This item documented inspector concems with CREV system
performance and the inspectors' intention to follow up on a CREV system modification
that was documented as a commitment in LER 50-254/94002-00, in inspection Report i
No. 50-254/97014; 50-265/97014 a deviation was issued for the licensee's failure to
install the modification. This item is closed, i
E8.2 (Closed) LER 50-254/93023-00- Engineered Safety Feature Actuation Caused by Main
Turbine Control L;g c Relay Failure. On December 16,1993, a relay failure in the main
turbine control logic resulted in a turbine trip, reactor trip and a main steam isolation valve
closure. Corrective actions were to replace the failed relay and to investigate options for
improving reliability and possibly adding redundancy to the turbine trip logic. The
inspectors verified that the engineers had proposed several modifications to the logic
23
,4
which were approved for installation during the Q1R16 and Q2R15 refueling outaces.
The inspectors also reviewed reactor trips since this occurred in 1993 and found that no
other trips were caused by failures in the turbine control system. This item is closed.
E8.3 (Closed) Inspector Follow-uo item 50-254/94004-43: 50-265/94004-43: System
Engineering Weaknesses. The licensee initiated several improvement efforts and
managemnt changes to address system engineering weaknesses, including those
incorporated into The three year Course Ot' Action. Improved qualification cards, some
improved trainleg initiatives, and hands on training on system walkdowns were
implemented in 1994 and 1995. Recent problems indicated continued deficiencies in
system engineer understanding of expectations and roles and system engineer support to
other parts of the station. System Engineering improvements were planned by Comed at
Quad Cities in order to bring the system engineers into a system manager role. These
efforts will be evaluated as part of the core program using inspection Procedure 37550.
This item is closed.
E8.4 (Closed) Insoefon Fobow-up Item 50-254/94004-45: 50-265/94004-45: Single Element
Feedwater Regulaung Valve Control. The licensee repaired and tuned the feedwater
regulating valve control system for both units and installed improved feedwater regulating
vanes. Both units are now capable of operation with three element feedwater control.
This item is closed.
E8.5 (Closed) Inspection Follow-uo item 50-254/94004-47: 50-265/94004-47: Potential Low
Pressure Coolant injection Swing Bus Deficiencies. This issue received Nuclear Reactor
Regulation (NRR) Electrical Engineering Branch rcview and was subsequently addressed
in the review of LER 50-254/93003-00. The LER was closed in Inspection Report
No. 50-254/96011; 50-265/96011. This item is closed.
E8.6 (Closed) Inspection Follow-up item 50-254/94004-51: 50-265/94004-51: Design Basis
Documentation (DBD). The licensee initially intended 22 system and/or topical reports to
be issued by December 1995. Many of these DBDs were issued for "information only" as
they had not been validated. The licensee was recently using outside resources to
perform a design basis initiative review and validation which was scheduled to extend into
1999. The licenses was also performing a line item validation of the UFSAR as part of a
10 CFR 50.54(f) request for information. This effort was scheduled to be completed in
October 1998. Design basis information availability was reviewed and discussed as part
of the System Operational Performance inspection 50 254/97022; 50-265/97022. Further
reviews will take place as part of routine enginesring inspections in accordance with
inspection Procedure 37550 and in conjunction with other regional and headquarters
inspection initiatives. This item is closed.
E8.7 (Closed) Violation 50-254/94004-54: 50-265/94004-54: Title 10 CFR 50.59 Evaluations.
The licensee evaluated the specific examples and either corrected the equipment
problem so that a safety evaluation was not needed (pumpback compressor), verified the
existing safety evaluation had addressed the concem noted (1 A RHR torus cooling and
test retum valves), or improved the safety evaluation (1 RHR 36 B valve.) Examples of
poor safety evaluations continue even thaugh the licensee took some corrective action to
trein engineers on better safety evaluation techniques in 1995. Ars enginecring
assessment group was formed in 1997 to address poor quality engineering work including
safety evaluations and screenings. Nevertheless, poor safety evaluations continued,
24
- _ _ _ _ _ _ _ _ - - _ _
,
,.'
iridis.;,,g the effectiveness of these actions had been limited. An apparent violation for
an ir+f+F safety evaluation was identified in inspection Reports No. 50-254/97027;
50-265/97027 as item 6. Corrective action for continued problems with safety evaluations
will be tracked along with licensee response to the 50065/97027-06 Item. This item is
closed.
<
E8.8 (Closed) Unresolved item 50-254/95009-02: 50-265/95004 02: Failure of the EDG to
Operate. The root cause of the failure was determined to be swelling of the air start
motor vanes due to improper control of moisture levels during storage. The NRC issued
a violation regarding the improper storage in Inspection Report No. 50-254/9600P
50-265/96002. This URI was created to follow up on the management initiatives to
improve root cause evaluations and troubleshooting methodology. Since this event, the
station has conducted root cwse training and established various root cause evaluation
experts. Additionally, more rigor in the troubleshooting process had been displayed as
root cause evaluations were conducted in accordance with the Nucioar Station Work
Procedure (NSWP-A-13). Although improvements in the process had been made, the
inspectors remained cc acomed with the number of EDG failuras and the lack of a
definitive root cause (see Section E1.1) and will continue te evaluate root cause
investigations during the normalinspection process. Th!s item is closed.
E8.9 (Closed) Unresolved item 50-254/96006-08: Secondary Containment Deficiencies. This
issue was addressed in inspection Report No. 50-254/96019; 50-265/96019 and resulted
in three Severity Level ill violations. This item is closed.
E8.10 (Closed) Unresolved item 50-254/96008-08: Reactor Building Siding issue. This issue
was addressed in inspection Report No. 50-254/96019; 50-265/96010 and resulted in
three Severity Level ill violations. This item is closed.
E8.11 (Clanad)_LER_50-25Mifa10: Gallery Steel above Primary Containment Equipment
installed improperty. Engineers preparing a modification noted that the gallery steel cross
bracing identified in drawings was not installed in the plant. The cross bracing would
auctify the gallery steel for seismic loading. T;w deficiency had existed since original ;
construction. The licensee concluded that during a postulated seismic event, the gallery
steel would fail. Some systems and components could be affected but, redundant
systems would be available for safe shutdown of the reactor. Corrective actions included
installing the cross bracing to achieve the seismic qualification. This item is closed.
E8.12 (Closed) Unresolved item 50-254/96012-07: Reactor Building Blowout Panel Bolts ,
Broken. This issue was addressed in IR 50-254/96019; 50-265/96019 and resulted in
three Severity Level lit violations. This item is closed.
E8.13 (Closed) LER 50-254/96012-00. Diesel Fuel Oil Transfer Piping Was in an Unenalyzed
Condition. The fuel oil transfer piping supplying the diesel fire pump day tank was not
safety-related. No isolation existed between this non-safety related piping and the safety-
l related piping to the EDG fuel oil system. Therefore, a failure in the non-safety portion of
the piping could have adversely affected the EDG safety function. This was discovered
during a station review of open items in the component classification program. Corrective
actions included isolating the two systems by closing two valves and reviewing other
open items within the component classification program to ensure that no similar issues
existed. This item is closed.
25
_ _ _ _ - _ _ _ _ _ _ _ _
- _ _ _ _ _ - _ _ _ - _ _ _
!
E8.14 (Closed) Unresolved item 50-254/9601444: 50-265/96014-04: The CREV System
Refrigerant Crankcase Heater Was Fed from Non-Safety Related Power Supply. The
system engineer identified the deficiency and the licensee took corrective actions to
provide a safety-related power supply to the heater. This item is closed.
E8.15 (Closed) Violation 50-254/96017-06: 50-265/9601/-06: Failure to incorporate a TS
Requirement. Technical Specification 4.8.D.4 required a laboratory analysis of the control
room HVAC charcoal adsorber after 720 hours0.00833 days <br />0.2 hours <br />0.00119 weeks <br />2.7396e-4 months <br /> of operation. The test procedure did not
track the hours of operation. This was a new requirement to the TSs and an oversight
resulted in not tracking the hours of operation. Licensee estimates cosecluded that it was
unlikely the 720-hour limit was exceeded. Procedures were changed to have operators
track the run times. This violation is closed.
E8.16 (Closed) Inspection Follow-up items 50-254/96017-07. 08. 09. 50-265/96017-07. 06. 09:
Control Room Dose Calculation. The inspectors questioned the licensee's operability
assessment which used different assumptions and parameters than the calculation of
record in the UFSAR. The licensee subsequently revised the calculation and the entire
t abitability study. The CREV system operability was the subject of enforcement as
documented in inspection Report No. 50-254/96017; 50-265/96017. These items are
closed.
E8.17 (Closed) Violations 50-254/96017-10.11: 50-265/96017-10,11: The CREV System
Could Not Maintain Required Positive Pressure. The CREV test program failed to verify
the system could maintain greater than or equal to 1/8 inch water gauge positive pressure
as specified in the UFSAR and per TS 3.8.D. The licensee repaired the system and
successfully completed testing that verified the control room would be maintained at the
required positive pressure. These violations are closed.
E8.18 G2339 Violation 50-254/96020-03: 50-265/96020-03- ImproperTesting of CREVs. The
inspectors identified that the CREV system was not adequately tested per TS 4.8.D.5.b.2.
Subsequent testing was performed to verify system operability. The test procedure was
changed. This violation is close,d.
E8.19 (Closed) LER 50-254/96024-00 The CREV System inadequately Tested. This issue
was identified by the NRC inspectors and documented as a violation in inspechon Faport
No. 50-254/96020; 50-265/96020. The inspectors confirmed that the proper test was
ultimately performed. As part of the corrective actions, the licensee performed a review
of new surveillances required as a result of the technical specification upgrade program. l
Although this review was completed in earty 1997, the inspectors noted that the review
'
did not identify a number of deficiencies and missed surveillances that were found in
late-1997. This item is closed.
E8.20 (Closed) LER 50-254/97020-00 The B Train CREVs Air Handling Unit Breaker Cycled
and Tripped. The licensee conducted testing to identify the cause but could not
reproduce the event. The monthly surveillance was performed successfully. No similar
failures have occurred since this event. This item is closed.
26
..
.- _ _ _ _ _ _ _ _ _ _
_
g=
{ 4
.
IV. Plant Succod
R1 Radiologios! Protection and Chemistry Controfs
R1.1 Uconsee's Actions to Address Past Hiah Radwiion Area Violations
The inspectors observed that the licensee had implemented new measures to assure
adequate attention was given to control access to high radiation areas. This was
accomplished through the use of high radiation area entry slips and a special computer
sign-in process to remind all personnel entering high radiation areas of their
responsibilities in maintaining proper cantrol of access to these areas. The results
observed thus far have been positive and no high radiation area access problems have
been observed since the implementation of these measures.
V. Manaaement Mootinas
X1 Exit Meeting Summary
The inspectors presented the inspection results to members of licensee management at the
conclusion of the inspection on February 10,1998. The licensee acknowledged the findings
presented.
i
!
j
27
1
=
.'
PARTIAL LIST OF PERSONS CONTACTED
Licensee
E. Kraft Site Vice President
L Pearce Station General Manager
C. Holbrook Engineering Manager
M. Wayland Maintenance Manger
B. Svaleson Operations Manager
G. Powell Acting Radiation Protection / Chemistry Manager
F. Famulari Quality and Safety Assessment Manager
28
c*
..'
INSPECTION PROCEDURES USED
IP 61726: Surveillance Observations
IP 62707: Maintenance Observations
IP 71707: Plant Operations
IP 92700: Onsite Follow-up of Written Reports of Nonroutine Events at Power Reactor
Facilities
IP 92902: Follow-up - Engineering
ITEMS OPENED, CLOSED, AND DISCUSSED
Opened
50-254/97028-01 VIO EDG cooling water valve mispositioning
50-265/97028-02 NCV valves found out of position in EDG air start system
50-265/97028-03 VIO out of service existing greater than three months
without completed safety evaluation
50-254/97028-04; 50-265/97028-04 URI inadequate documentation of EDG fuel oil piping
support
50-254/97028-05 NCV steps performed out of sequence during SBLC )
check valve test 8ng 1
50-254/97028-06; 50-265/97028-06 NCV failure to perform required surveillance )
50-265/97028-07(DRS) eel failure to meet Code requirsments for the VT-2
examination of the Class 1 system boundary
Closed 1
50-254/94004-57; 50-265/94004-57 IFl integrated reporting program
50-254/94004-58; 50-265/94004-58 IFl site quality verification organization and
effectiveness
50-254/94029-01a, 01b VIO inadvertent reactor vessel draining
50-265/94029-01a, 01b
50-254/96002-04; 50-265/96002-04 IFl weak log reviews
50-254/96005-00 LER the CREV system inoperable due to low outside air
temperature
50 254/96017-00 LER manual scram during reactor startup
50-254/97022-00 LER t% CREVS inoperable
50-254/93025-01 LER "A" loop MSIVs exceeded TS leakage limits
50-254/94004-05; 50-265/94004-05 IFl repetitive oxygen analyzer problems
50-254/94004-06; 50-265/94004-06 IFl back leakage through 1-RHR-78 valve
50-254/94004-27; 50-265/94004-27 IFl core spray pump discrepancy
50-254/94004-34; 50-265/94004-34 IFl work package preparation, utilization, and
processing
50-254/95001-00 LER during Unit 1 startup the RCIC system govemor j
valve did not respond property due to valve steam
corrosion
50-254/95005-00 LER CREV inoperable due to refrigerant leak
29
!
._
.E
50 254/96004-00 LER Unit i HPCI system inoperable due to gland
exhauster breaker trip
50-254/96014-00 LER electrical distribution surveillanos did not document
voltages in accordance with TS 4.9.E
50-254/97002-07; 50-265/97002-07 URI - Unit 2 CAM system maintenance rule
implementation
Go doiw/027-02(DRS) URI URI portaining to the adequacy of the VT-2 exam of
Class 1 system boundary
6y 42016-01 IFl CREV Train surveillance failure
56 64/93023-00 LER engineered safety feature actuation caused by main
turbine controllogic relay failure
50-254/94004-43; 50-265/94004-43 IFl system engineering weaknesses
50-254/94004-45; 50-265/94004-45 IFl single element feedwater regulating valve control
50-254/94004-47; 50-265/94004-47 IFl . potential LPCI swing bus deficiencies
50-254/94004-51; 50-265/94004-51 IFl design basis documentation
50-254/94004-54; 50-265/94004-54 VIO Title 10 CFR 50.59 evaluations
50-254/95009-02; 50-265/95009-02 URI failure of the EDG to operate
50-254/96006-08 URI secondary containment deficiencies
50-254/96008-08 URI reactor building siding issue
50-254/96010 LER gallery steel above primary containment equipment
installed improperty
50-254/96012-07 URI reactor building blowout panel bolts broken
50-254/96012-00 LER diesel fuel oil transfer piping was in an unanalyzed
condition
50-254/96014-04; 50-265/96014-04 URI the CREV system refrigerant crankcase heater was
fed from non-safety-related power supply
50-254/96017-06; 50-265/96017-06 VIO failure to incorporate a TS requirement
50 254/96017-07; 50 265/96017-07 IFl control room dose calculation
50-254/96017-06; 50 265/96017-08 IFl control room dose calculation
50 254/96017-09; 50-265/96017-09 lFl control room dose calculation
50-254/96017-10; 50 265/96017-10 VIO the CREV system could not maintain required
positive pressure
50-254/96017-11; 50-265/96017-11 VIO the CREV system could not maintain required
positive pressure
50 254/96020-03; 50-265/96020-03 VIO improper testing of CREVs l
'
50-254/96024-00 LER the CREV system inadequately tested
50-254/97020-00 LER the B Train CREVs air handling unit breaker cycled
and tripped
i
I
e
i
,.
./
LIST OF ACRONYMS AND INITIALISMS USED
ACE Apparent Cause Evaluation
ADS Automatic Depressurization System
ASME American Society of Mechanical Engineers
CAM Containment Atmosphere Monitor
CFR Code of Federal Regulations
Comed Commonwealth Edison Company
CREVS Control Room Emergency Ventilatum System
DBD Design Basis Docuraentation
DG Diesel Generator
) DGCWP Diesel Generator Cooling Water Pump 1
'
CCCS Emergency Core Cooling System
EDG Emergency Diesel Generator
EHC Electrohydraulic Control
ENS Emergency Notification System
EO Equipment Operator 1
ESF Engineered Safety Feature
FME Foreign Material Exclusion
GL Generic Letter j
HPCI High Pressure Coolant injection System )
HVAC Heating, Ventilation, and Air Conditioning
IDNS lilinois Department of Nuclear Safety
IFl Inspection Follow-up item
LER Licensw Event Raport
LPCI Low Pressure Coolant injection
MSIV - Main Steam Isolation Valves
NTS Nuclear Tracking System !
OOS Out of Service
PDR Public Document Room
PFC Procedure Field Change
PlF Problem identification Form
PM Preventive Maintenance
QAP Quad Cities Administrative Procedure
QC Quality Control
QCAP Quad Cities Administrative Procedure
QCEPM Quad Cities Electrical Preventive Maintenance
QCMMS Quad Cities Mechanical Maintenance Surveillance
QCOP Quad Cities Operating Procedure
QCOS Quad Cities Operating Surveillance Procedure l
QOM Quad Cities Operating Mechanical Procedure !
RCE Root Cause Evaluation
RCIC Reactor Core isolation Cooling System
RG Regulatory Guide
RTS Retum to service
SAR Safety Analysis Report l
SBLC Standby Liquid Control j
TD Time Delay ;
'
TGA Toxic Gas Analyzer
31
4*
! .s '
TS Technical Specification l
UFSAR Updated Final Safety Analysis Report i
URI Unresolved item
Vdc Volt direct current '
VT Visual Test -
VIO Violation
<
'
l
l
t
l
32
J