IR 05000321/1989008
| ML20246N857 | |
| Person / Time | |
|---|---|
| Site: | Hatch  |
| Issue date: | 08/17/1989 |
| From: | Jape F, Mellen L, Peebles T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML20246N806 | List: |
| References | |
| 50-321-89-08, 50-366-89-08, NUDOCS 8909080285 | |
| Download: ML20246N857 (62) | |
Text
,,,
.;_
,.
_ _ _ _. _ - - _ _ - - -. _, _.
37$ y ihW f.1 9--
-
,
s
Eto
>
+
s
--
..;
e r
-
N ',,' y'
.,
r
,
y N
S,a C.
m.90'
.
.
...
UNITED STATES
.
,
-
'
y,!$ if
}? 1 ".-
iNUCLEAR REGULATORY COMMISSION
,g
.,
'*
(f ( f.
" REGION il -
, ' ", "
~
,
O+e j4 (101 MARIETTA STREET.N.W..
~
a
.-
'3 t T Q"
,s
- ATLANTA, GEORGIA 30323 '
iT
-
f
'
[ m},, '?Q
'
l
'
+
,
,.. s
.
,
.
y
.
.
.
&,
17:
'
'
~*
e
,
,
,
'N Q
.,
N
~
'
,
..
-
o
.
'
~"
'
[ReportNo.s.:. 50-321/89-08'and 50-366/89-08 w.
L
'
>
a
,
s
<
,
,
,
,,
, in -J.Libensee: ; Georgia Power: Company:
A P
O. Box 1295 '
P-
' i
.
u
<-
@
' Birmingham, AL 35201
"
'
c
,
D5cAet Nos.:. 50-321 an'd 504366 License Nos : DPR-57,and NPF-5
.
. :,".
.
,.
L' r Facility'Name: / Hatch 1 and 2
.
.
[
' Inspection-Conducted:.. May.15-19; June 5-9, and June I19,1989-
'
-
Inspectors.
Ah
>
F; JapeqTeam Leader-g? //
Date Signed kb ca -
9 t h! 69
"L. S. Melleri, Assistant Tear / Leader
/Date Sfgned
.
,
E i Team Members:
S.,SpaYks
,
C. Hughey-R.; Wright RC Moore
'
,
M. Thomas
' '
y
' Contiractor Personnel:
~
- T. DelGaizo, Main' Line Engineering Associates C Crane, Main Line Engineering Associates E. Spilker, EBASCO Engineers W. Ludgren, EBASCO Engineers ApprovSd By:
f-/7M
'
Thomis Peebliis~, Chief Date Signed r
_ Operations Branch-
,.
'
Division of Reactor Safety
.
SUMMARY s
Scope:
This special,' announced Sa'fety System Functional Inspection (SSFI)'was performeditojassess the operational readiness of the Emergency Diesel.
,
"
Generators and associated support systerrs to meet their-. intended design-function under all postulated conditions. The licensee'~s. operational and
.
4.. '
, management' controls were evaluated in the following functional areas:
r
,
,
i
'~
8'09080285 890818
$,.,x
sc
'
'
PDR ADOCK 05000321
>
_.
Q FDC
~
>
. g
>
Ail.___._____.__m____
.,_
. -
_ _ _, _ _ -
_ _ - _ _ _ _ - _.-_
. _ - _
_ -
-
_ _ - _
_
--
._
-_
_ _ - _ _ _ _ _ _
.
.
' Design Control Operations Maintenance.
Surveillance QA/QC Inspection Objective:
The inspection objective at Hatch was to assess the operational readiness of the Emergency Diesel Generators and associated support systems. The assessment included the determination of the following:
- capability of the systems to perform their safety functions as required by the design basis adequacy of operations to ensure the systems are being operated
-
properly adequacy of maintenance to ensure the systems are being maintained
-
properly
- adequacy of surveillance to ensure the systems are being tested properly
- adequacy of QA/QC activities to ensure the systems are being reviewed properly.
Acronyms used.throughout this report are listed in the Appendix B.
Results:
The results of this inspection indicate that the emergency diesel generators and support systems are capable of achieving their design functions.
The various concerr:s identified by the inspection team do not severely impact the overall functionality of the systems, however these concerns do require attention. Generally, concerns were identified with the design, fuel chemistry, surveillance, and maintenance areas. These concerns are enumerated u
as the violation and Inspector follow-up items which follow.
Several items i
are particularly notable. The relatively high incidence of corrective main-tenance on the Emergency Diesel Starting Air System indicates a question as to
'
the overall' reliability of the system.
Surveillance activity, as specified by techical specifications, was performed as required, however a disparity existing between surveillance for similar equipment on differing units was a concern.
Also within the surveillance functional area, it was noted that the inservice testing program for emergency diesel generators and support systems did verify operability but did not provide a means of identifying degraded performance.
t L _ _ _____-__ _ _-
_
___-_-
..
..
_ _ _ _ _ -
_.
i
!
..
.I"
!
One. violation., no deviations, no URIs, and 25 IFIs were identified as follows:
Violation -
50-321/366-89-08-01i Failure to perform timely corrective actions, relative to Emergency Diesel Generator fuel oil problems identified in various QA audits. (paragraph 4. A.3.a)
IFI
-
50-321/366-89-08-02: Seven Day Fuel Storage Capacity. (paragraph 4. A.1.. a)
l IFI
-
50-321/366-89-08-03: PSW System Design Pressure. (paragraph 4.A.1.b)
L IFI.
50-321/366-89-08-04: Common 10 Inch PSW Discharge Line.
-
(paragraph 4.A.I f)
IFI
-
50-321/366-89-08-05: Documentation of Design Parameters.
-(paragraph 4..A.1.g)
L IFI
-
50-321/366-89-08-05: PSW Pump Pit Design. (paragraph 4.A.1.k)
IFI
-
50-321/366-89-08-06: Revised Emergency Diesel Generator Load.
(paragraph 4.A.2.b)
50-321/366-89-08-07: Emergency Diesel Generator CARDCX System.
IFI
-
(paragraph 4.A.2.c)
IFI
-
50-321/366-89-08-08: Seismic Qualification of EDG Low Lubricating 011 Switches. (paragraph 4.A.2.d)
IFI
-
50-321/366-89-08-09:
Ventilation Louver Motors in Diesel Rooms.
(paragraph 4.A.2.d)
IFI
-
50-321/366-89-08-10: Electro-Thermal Links on Diesel Room Roll-up Doors and Fire Dampers. (paragraph 4.A.2.d)
IFI
-
50-321/366-89-08-11: Diesel Battery Rack Mounting. (paragraph 4.A.2.d)
IFI
-
50-321/366-89-08-12: DC Battery Sizing. (paragraph 4.A.2.e)
50-321/366-89-08-13: Diesel Battery Specification. (paragraph IFI
-
4.A.2.f)
50-321/366-89-08-14: Diesel Battery Over Voltage. (paragraph IFI
-
4.A.2.g)
50-321/366-89-08-15: Transient Load for Unit 1. (paragraph IFI
-
4.A.2.h)
IFI 50-321/366-89-08-16:
Diesel Generator Loading Calculations.
-
(paragraph 4.A.2.j)
- - - - - _ - _ _ _ _ - - - - _ _ _ - _ _ - _ - _ _ -
. -
-.
_ __
_ _ - _ _ _. - _ _ _ _ _ _ _ _ _ - - -.
'
.
.
'IFI
-
50-321/366-89-08-17.:
Diesel Fuel Oil Storage Tank Design.
(paragraph 4.A.3.c)
IFI 50-321/366-89-08-18:
Fuel Oil Sampling Program. (paragraph
-
4.A.3.f)
50-321/366-89-08-19:
Gaitronics Communications. (paragraph l
IFI
-
4.B.3.a)
50-321/366-89-08-20:
Access to Vital Areas. (paragraph IFI
-
l 4.B.3.b)
IFI.
-
50-321/366-89-08-21:
Emergency Diesel Generator 1B PSW Supply.
(paragraph 4.B.3.c)
IFI 50-321/366-89-08-22: Accurary of Local Frequency Meter.
-
.(paragraph 4.C.2.a)
L IFI 50-321/36(.-89-08-23:
Reliability of Emergency Diesel Generator
-
Starting Air System. (paragraph 4.C.2.b)
,
IFI
-
50-321/366-89-08-24:
Comparison of Unit 1 and Unit 2 Technical Specifications. (paragraph 4.D.I.d)
l
.IFI
-
50-321/366-89-08-25: Emergency Diesel Generator Comr:onent Inservice Testing. (paragraph 4.D.2.b)
IFI-
-
50-321/366-89-08-26: Plant Service Water Valve Lineup.
(paragraph 4.D.2.c)
l i
l l
I l
_ _ _ _ _
_ _ _ _ _.
.-. _
j
_ _ _ _ _ _ _ _ _
. _ _ - _ _ _ _ _ _ _ _ _ _ _
__ _ _ _ _ _ _ _ _ _ - _ _ _ -
. _ _ _
. - _ _ _ _ _
.
.
l l'
TABLE OF CONTENTS FOR REPORT DETAILS P_ age 1.
Persons Contacted.............................................
'
2.
Acronyms and Initialisms............................................
3.
System Description.................................................
1-4.
Detailed Inspection Findings..
..................................
a.
Design Control..............................................
(1) Mechanical Design..................
....................
(a) Seven Day Fuel Storage Requirement (b) PSW System Design Pressure (c) PSW Pressure Switch Seismic Documentation (d) PSW System Valve Line-Up (e) Design Basis Documentation (f) Common 10 Inch PSW Discharge Line (g) Demineralized Water Fill Header (h) System Functionality (1) PSW Pump NPSH (j) River Level Indication (k) Vital Equipment Located in PSW Pits (1) Accident Simulation (2) Electrical Design.....................................
(a) Ambient Compensation on Thermal Overload Relays (b) Emergency Diesel Generator Loading Calculations (c) Design Basis For Emergency Diesel Generator Room Fire Protection System (d) Seismic Qualification Documentation (e) DC Battery Sizing (f) Emergency Diesel Generator Battery Specificati~on (g) [mergency Diesel Generator Battery Over Voltage (h) Unit 1 Emergency Diesel Generator Transient Analysis (1) Design Basis Review (j) Design Control (k) Hardware l
_-_ -.
_
__
_
. _ _ _ _ _ _ - - _ - _ _ _ _ _
__.
-_ _ _ _ _ - _
- _ - - -
_-
_ _ _
. _ _.
_ _ _ _
-
.
$-
(3) Chemical Design..........................................
(a) Emergency Diesel Generator Fuel Oil Sa:npling and Analysis
.(b). Heat Exchanger Performance (c) Fuel Oil Storage
"
(d) Existing Procedures ( f). Proposed Procedures or Procedures in the Validation Process (4) Configuration Control...................................
l '
(5) 0ff-site Engineering.....................................
L l
b.
Operations....................................................
(1) Final Safety Analysis Report and Vendor Requirements.....
1.
l (2) SORS and. Deficiency Cards...............................
36-
L (3) Simulator Demonstration................................. 38 (4).
Performance Indicators.................................
c.
Maintenance................................
..................
(1) Review of Maintenance in Progress........................
(2) Review of Completed MW0s.................................
d.
Surveillance.................................................
(1) Emergency Diesel Generator..............
................
(2) Emergency Diesel Generator Support Systems...............
e.
QA/QC and Procurement.........................................
50-(1) Quality Assurance...............................
......
(2) Procurement and QC Activities.............................
5. Exit Interview....................................................
APPENDICES
.
Appendix A - Persons Contacted Appendix B - Acronyms and Initialisms ii
- _ - __
__- _ _
_- ____-_-__ __- ___--_______ _ __ ______-_ ---___ _ -____-- _ - -___ _ - _ ___ _--_--__ - -_ _ a
-.
--
__-
..
.,
1.
Persons' Contacted Refer.to Appendix A 2.
Acronyms and Initialisms Refer to Appendix.B 3.
' System Description The Emergency Diesel Generator System at the Hatch Nuclear Plant consists of 5 turbo charged diesel engines, each with 4160 volt AC generators rated between 2850 KW (conti.nuous) and 3250 KW (peak).
HNP Units I and 2 are each provided with two dedicated engine generators. The fifth.
l'
. engine generator is. normally lined up to Unit 1, but automatically swings to the accident Unit upon receipt of an accident signal. The generators are designed to reach rated speed (900 rpm) and voltage within 12 seconds I
of receiving a start signal and supply the standby AC power for the 4160-V Emergency Service buses.
i The Emergency Diesel Generators cannot be paralleled with each other i
through the startup transformer's bus supply breakers. To prevent parallel operation.of the Emergency Diesel Generators, redundant loss of voltage signals pick up lock out relays to trip the transformer supply breakers. The only engine performance trips that are not bypassed during emergency operation are lubricating oil pressure and engine over speed.
Each engine is provided with a 40,000 gallon underground fuel oil storage
. tank, a 1000 gallon day tank, an independent fuel oil transfer system, two starting air receivers with two air compressors, and a DC battery for generator excitation and voltage regulation. The engines are supplied with a closed-loop demineralized water cooling system, using plant service water'as a heat sink.
Under accident conditions, the four dedicated engines are independently provided with service water by four of the eight PSW pumps, while the swing Emergency Diesel Generator has its own service water pump.
l-Each Emergency Diesel Generator System consists of a diesel engine, l
generator, and auxiliaries mounted on a common base.
Two independent air l
starting systems are furnished for each Emergency Diesel Generator, either of which is capable of starting the diesel engine.
Each of the air starting systems has adequate capacity to start a single diesel engine five times without recharging. Two motor driven air compressors are available for each Unit.
l The engine lubricating oil system for each Emergency Diesel Generator consists of an engine driven oil pump, a full flow oil filter, an oil cooler, a full flow strainer, a motor driven circulating pump and a motor i
driven prelubrication pump.
l'
l l,
n.___________
_ _ _ _ _ _ _ _ _ _. _ _ _ _ _ _
__
_
-
__
_ - _ _ _ _ - _ _ - _ _ _ _ _
...
.,
l l
The engine generators are located in separate rooms within a single seismically-designed Emergency Diesel Generator Building.
Each room is provided with ventilation to maintain ambient temperature below 122 degrees Fahrenheit when the Emergency Diesel Generator is operating. The rooms are also provided with automatic fire protection which isolates and floods the room with Carbon Dioxide upon detection of a fire.
4.
Detailed inspection Findings a.
Design Control (1) Mechanical Design The mechanical portion of the SSF1 inspection dealt primarily with the ability of the Emergency Diesel Generator System and selected support systems to perform their safety functions as required by the design basis. During the SSFI the Emergency Diesel Generators and the following mechanical systems which supported the Emergency Diesel Generators were reviewed and evaluated:
i
'
Plant Service Water Lubricating 011 Fuel Oil Service Water Cooling Scavenging Air Fuel Oil Fill and Transfer Starting Air Emergency Diesel Generator Building Ventilation Emergency Diesel Generator Building Fire Protection This review resulted in the following find'ngs:
(a) Seven Day Fuel Storage Requirement Unit 1 Technical Specifications require a total of 80,'000 gallons of diesel fuel oil storage in the two Unit I tanks and the Swing Emergency Diesel Generator fuel oil storage tank.
Unit 2 Technical Specifications require 32,000 gallons (minimum) in each Unit 2 diesel fuel oil storage tank and the Swing Emergency Diesel Generatur fuel oil storage tank. Taken together, the Technical Specifications require a total of approximately 149,000 gallons (IA-25K, IB-32K, IC-26K, 2A-32K, 2C-32K).
However, an accumulation of sludge in the tanks and the heat rating of the fuel oil can further reduce the available volume. Hatch Emergency Diesel Generators consume approximately 235 gph or 6640 gallons per day at full load.
l
_ _ - _ _ _ _ _ _ - _ _ _ _ _ _. - - -
. - _ - _
_ - _
.
- ____-
-
.- _
_ __
(%s
3 The basis of the seven day requirement in the Final Safety Analysis Peport, as indicated in the Technical Specifica '
tions, was not clear.
For. example, the available fuel oil, per the Technical Specifications, would;not be sufficient to run all five engines at full power for seven days (requires approximately 198,000 gallons), but would be sufficient to run three Emergency Diesel Generators at minimum safeguards loads.
The fuel oil required to run all available Emergency Diesel Generators at expected loadings following a design basis accident was not known.
The Final Safety Aralysis Report states that fuel oil for approximately seven days of post-accident Emergency Diesel Generator operation will be provided.
There are accident scenarios that do not consider the loss of a single Emergency Diesel Generator. The inspectors reviewed the Emergency Diesel Generator fuel oil consumption during these postulated events. When considering the reduction in storage capacity due to fuel oil sludge accumulations within the storage tanks and Emergency Diesel Generator fuel oil heat ratings, the inspectors determined the the
-
minimum fuel oil storage specified in the Technical i
Specifications was not sufficient to provide for seven day operation of all five Emergency Diesel Generators with loading limited only to full load.
Subsequent investigation by the Licensee indicated that the minimum required Technical Specificat.on fuel oil was not sufficient to operate all five Emergency Diesel Generators for seven days, considering uncontrolled loading of Emergency Diesel Generators on the non-accident Unit and the reduction of fuel oil available due to potential sludge in the storage tanks and fuel oil heat ratings.
Consequently, the Licensee committed to change the Technical Specifications to increase the fuel oil requirement and also to apply administrative controls to loading of the Emergency Diesel Generators on the non-accident unit until additional fuel oil is obtained.
This observation is of limited safety significant since seven days of Emergency Diesel Generator operations at minimum safeguards is available.
Nevertheless, the Licensee's commitment to increase the Technical Specifica-tion limits is considered appropriate because it is not realistic to assume that only the minimum safeguards equipment will be operated throughout the seven-day period.
Until the licensee's commitments are completed, this is identified as inspector follow-up item 50-321/366,89-08-02:
Seven Day Fuel Storage Capacity.
_
__
. -
[
(b) PSW System Design Pressure The plant service water system of each Unit consists of two divisions of safety related equipment (Division I and Division II), each division containing two PSW pumps.
Under normal operations, three PSW pump's are running (two in one division, and one in the other), with the fourth pump in standby -and the divisions cross-connected within the turbine building.
In this mode, the three pumps provide between 6000 and 7000 gpm each, with the majority of the flow (approximately 15000 to 17000 gpm) being provided to the turbine building. The total. developed head of the pumps under these conditions is approximately 130 to i
l 140 psig, at normal river level.
System design pressure is 180 psig.
!
Upon receipt of an accident signal (loss of coolant, loss l
of off-site power, or a pipe break in the turbine buildir.g,
'
as indicated by high flow).the turbine building is isolated and each individual division is supplied by one PSW pump.
In this mode, the flow in each division is greatly reduced (to approximately 2000 to 3500 gpm), resulting in pump TDH of approximately 170 to 180 psig (with an additional 10 to 15 psi at extremely high river levels).
The 150 psig coolers of the Emergency Diesel Generator are adequately protected from these higher pressures by pressure reducing orifices in the supply lives. The NRC inspection team, however., expressed a concern that the supply lines to the reactor building did not contain similar orifices, and l
therefore components in the reactor building could be l
exposed to pressures greater than 150 psig.
Subsequent investigation by the Licensee identified a number of coolers which can exceed their design pressures.
The following is a summary of the Unit 2 coolers. The Unit I coolers are similar except that calculated inlet pressures i
for Unit I are generally lower than those of Unit 2 and the j'
RHR pump seal coolers of Unit I were supplied by a different vendor and are designed for 150 psig (and were subjected to a single hydrostatic test at 225 psig).
!
Calculated Design Description Inlet Pressure Pressure (Psig)
(Psig)
CRD Pump Room Coolers 172 150
'
HPCI Pump Room Coolers 185 150 RHR Corner Room Coolers 175 150 RHR Pump Seal Coolers 180
RCIC Room Coolers 171 150 i
--
_ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ - _ - _ _ - _ _ _ _ _ _ _ _ _ _ _ _ - - _ _ _ _ _.
_ _ _ _ _ _ _ _ _ _ _ _ _.
_
_ _ _ _ - _ - _ _
_ _ _ _ - _ _ _ _ _ _ _. - _ - _
.
- s-
.
The Licensee committed to long-term code compliance with respect to these coolers by (I) revised vendor documentation qualifying components to the higher pressures (2) system modifications to meet the design pressure of the components-or (3)' replacement of components as necessary.
The Licensee l-also justified continued operation as follows:
~I'
In the case of all coolers other than the Unit 2 RHR pump seal-coolers, operation was justified based on the exemption in the ASME Code which permits 20 percent higher stress allowables for system abnormal pressures during less than one percent of the operating period, hydrostatic test pressures of 225 psig, or a combination of the two.
In the case of the Unit 2 RHR pump seal coolers,
. continued operation was justified based on a factor of safety of 2.5 between expected pressure and_ failure pressure.
In addition, these coolers are not needed for the safety injection mode of RHR (needed only for the shutdown cooling mode), in the event of PSW.
leakage room sump pumps can accommodate the expected 20 GPM maximum leakage, and all PSW requirements will be met.
No other instances were'found where design pressures were exceeded. However, a possible root cause for this problem involves a breakdown in the interface between the equipment vendor and the system design organization.
Similar vendor / designer interface problems were ider.tified relative to the documentation of seismic qualification of a number of equipment items.
(discussed in Paragraph 4.A.2.d and 4.A.2.1 below).
Design pressure requirements for the plant service water system are set forth in the ASME Code,Section III, Subsection ND.
This matter is safety significant because the integrity and reliability of piping components, such as coolers, is assured by safety margins provided in ASME Code allowables.
Items which are over stressed, such as by exceeding design pressure, may leak and subsequently rupture.
In the case of these
'
components, the licensee has shown that sufficient safety margin remains to I eclude leaks and ruptures, particularly since the condition which gives rise to the increased pressures is limited to a very small
-
_ _ _ - _ _ _ _ _ _ - _ _ _ _.
. _ _ _
. - - _
.
._--
'
R c..
, m.%
/
l i
percentage of the operating' period.
Nevertheless, these. items should be: brought into full compliance
,
with ASME Code requirements, as the Licensee has committed. This item will be tracked as inspector p
follow-up item 50-231/366,89-08-03: PSW System Design.
Pressure.
"
(c) PSW Pressure Switch Seismic Documentation-PSW DPIS N522 to N525 operate to isolate PSW (both divisions)'from the turbine building in case of high flow (line break) in the turbine building. These l
switches require seismic qualification.
In the.
'
initial review of. seismic documentation, the following concerns were identified:
L
.1 The purchase order (PEH-6653) specified Barton model 288..The test report (ITT Barton R1-288A-10) tested a Barton model 288A.
Similarity between the two models was not-documented.
2~
Two switchas were tested. One switch had no problems.
The other exhibited relay chatter at 38 Hz resonant frequency and 39 acceleration applied. This part of l-the test was suspended after one minute.
(The first I
switch was configured for TVA; the second had standard
'
Barton relays )
The Barton cover letter to Southern Services,' dated
-
December 3, 1975, provided for testing the Hatch switches if the test report was insufficient to meet the Hatch specifications. No' record of additional testing was found.
There is no indication of a technical review and acceptance of this report for Hatch.
Subsequently, the Licensee provided technical informa-tien sufficient to demonstrate qualification of the switches, including similarity of the installed and tested devices and technical basis for accepting the test as conclusive for Hatch. While this information was sufficient to resolve technical concerns relative to the equioment, the initial documentation was not l
sufficient to demonstrate qualification.
Device specific seismic qua'lification documentation could not' be provided for Emergency Diesel Generator low lubricating oil pressure switches or Emergency Diesel Generator governors.
Upon review of the initial documentation provided to the NRC team, it was
- - -.._ _.-..--
-
._
.
-
___
_.
-
.
__
-
_- -_
_ _ _ -
ha
,
! y,
'
' not clear as to how these devices were certified as L
part of the overall Emergency Diesel Generator seismic L
qualification test or.how this information was -accepted
'
for Hatch.
In the case of the PSW pressure switches, actual hardware concerns were subsequently resolved to the satisfaction of the NRC team.
'
The amplitude of the seismic response. spectra for Plant Hatch is relatively low and there is little.
. concern of actual component failures as a result of.a seismic event.
Furthermore, in each instance where seismic. qualification concerns were identified ~
,
hardware concerns were satisfactorily resolved.
Neverthe'.ess, documentation weaknesses point toward potential design process problems, including a possible g
'
generic problem relative to the review and approval of vendor documents during plant design and construction.
p (d) PSW System Valve Line-Up l
l During observation of a simulated LOCA/ LOOP event at the Unit 2 simulator, loss of the standby service water pump was simulated. Within minutes, loss of this pump led to a l
loss'of the IB Emergency Diesel Generator. Although the IB Emergency Diesel Generator is designed with alternative L
sources of cooling-water from both Division'I and Division II PSW, neither nf these. sources could be lined up to the coolers in time to prevent loss of the Emergency Diesel Generator because the motor operated valves are normally locked shut.
In addition to unlocking two motor operated velves, lining up an alternate source of cooling water to the IB Emergency Diesel Generator also involves installing an orifice plate in the discharge line.
Valves P41-F401A, F402A, F402B, and F403B are motor operated valves which are locked shut to preclude inadvertent cross-connection of Division I and Division II.
If two of these valves remained locked shut while the other two were normally shut, but unlocked, inadvertent cross-connection would still be avoided but an alternate source of cooling
..
water to the IB Emergency Diesel Generator would be immediately available to the control room operator, in the event of loss of the standby service water pump.
This is not a concern but an observation that the opera-tional flexibility in the design of the PSW system is not being used to full advantage.
Prior to initiating changes L
to existing nrocedures, the original reasons why the valves were locked shut should be revisited and the necessity for t he orifice (under emergency conditions) should be evaluated.
..
reduction of pressure from Division I and Division II to the IB Emergency Diesel Generator coolers is essential, permanent orifices upstream of the locked closed valves could be installed.
- - __ _ __ - _ _-_-__-___-__- _ -__
_ _ - _ _ _ _ - _ _ _ - -
._. _
_
_ _ _ - _ _ _ _ _ _
..
..-
-_
-
= -_
.
_.
.
-
-.
_
.
--. _. _ _.
-
j
'
o v..
The current configuration.is satisfactory..The observation
' is submitted for consideration by the Licensee to take better acivantage of the design flexibility of the PSW.
system.
z (e) Design Basis Documentation There are a number'of examples where a documented design
. basis was not readfly available.
In the case of the examples listed below calculations, walkdowns, or other engineering' evaluations were needed to establish a design basis or to. demonstrate specific aspects of the design:
PSW system design pressure Seven day Emergency Diesel Generator fuel oil storage commitment-DC over-voltage Unit I transient over voltage DC battery sizing Seismic support of non-safety pipe In addition, the engineering group used superseded calculation (calculation number 117) to determine pump TDH
- for the standby service. water pump.
The calculation had been superseded by calculation number 156.
Furthermore, when specifically requested to provide the design basis of the standby service water pump, a number of iterations and questions were needed before a complete design basis'was provide; With exception of the PSW design pressure issue, the examples above were documentation matters. The lack of.a documented design basis is safety significant when changes are made to the plant (hardware, software, procedures, etc.) wherein the lack of design basis information can result in inadvertent and undesirable modification of a system or equipment safety function.
(f) Common 10 Inch PSW Discharge Line The Unit 2 Final Safety Analysis Report, Section 3.1 u
l Criterion five, stctes systems are not shared between 1.
Units unless it can be shown the sharing will not impair the safety function.
Criterion 45 states that cooling water system are designed to the extent practical, to permit inspections to assure integrity and capability.
In spite of these two statements, the PSW cooling water discharge from all five Emergency Diesel Generators discFarges to the river through a common 10 Inch buried pipe l'
_
- ___ -___ - _ -______--_______-_ __ ___ _ _____-___ - ____ _ ___________ __.
_ _ _ _ _
.________ ______ _______________-__:
-
- - --_ - _ - _.
__ -_. _ _ _ _
_ _ _
_ _ - - - - -
- __ - - - -_ _____ _ -_- _ _,
.
9 The licensee reported that no inspections of the'10 Inch line have occurred to date; however, a six. Inch line that.
feeds into the 10 Inch line had been visually inspected in conjunction with other. maintenance activities. The purpose of this inspection was to determine the general condition of the line with respect to erosion / corrosion.
No signifi-cant degradation had occurred; a mild oxidation film existed and there was no evidence of pitting.
This inspection, along with videotaped inspections on similar sections of larger diameter piping, led to the conclusion that the 10 inch pipe would have internal conditions similar to the six Inch pipe.
The licensee also reported participating in the EPRI Service Water Working Group. The licensee anticipated periodic service water inspections as part of an upcoming NRC generic letter. An internal inspection device (video camera attached to a crawler) will probably be needed to inspect the 10 Inch pipe, cinal Safety Analysis Report Section 3.1, Criterion 5, and Criterion 45, discuss shared cooling water systems.
The safety. significance of this item involves a potential loss of cooling water to all five Emergency Diesel Generators.
In order for such a failure to occur, the walls of the piping would have to erode or corrode to the point of collapse, such that the earth would prevent flow ~of cooling water. This is clearly an extremely low probability event, however, the severity of the consequences are such that precautions should be taken. The licensee'has committed to a program to periodically confirm the integrity of the i
common pipe either as part of a general service water improvement program or separately.
This item will be tracked as inspector follow-up item 50-321/366-89-08-04:
Common 10 Inch PSW Discharge Line.
(g) Demineralized Water Fill Header The demineralized water header in the corridor of the Emergency Diesel Generator building is of non-safety and non-seismic design. The demineralized water provides make-up water to the heat exchangers on the Emergency Diesel Generators. The branch piping off this header has been designed as seismic as a result of a non-seismic over seismic (II/I) analysis.
The demineralized water header runs in close proximity to the Emergency Diesel Generator air intake filter boxes for all five Emergency Diesel Generators.
i
_ _ _ _ _ _ _ _ _. _
. _ _ _ _ _ _ _.
_
_
_ _ _ _ _. - - -
__
_ --
I:
..
L*
10
. The Final Safety Analysis Report and Technical Specifications l
identify the minimum s'4feguard loads.for loss of off. site
'
power with loss of coolant accident as two Emergency Diesel Generators supplying the accident Unit and 1 Emergency Diesel Generator sunplying the non-accident Unit.
If a postulated crack in the demineralized water header could disable a Emergency Diesel Generatcr, then this event
_
coupled with the failure of one Emergency Diesel Generator to start could lead to a condition where only one. Emergency Diesel Generator might be available to the accident Unit.
A postulated critical crack in the demineralized water pipe was reviewed which indicated that approximately 64 percent of that flow would have to pass through the filter and into the engine to reach a 15 percent noncombustible pollutant level (this level was identified by the Emergency Diesel Generator manufacturer in other unrelated documents as a critical level for Emergency Diesel Generator performance).
Based on the increased pressure drop'through a wetted filter media 'mbined with the geometry of the filter box, this was not onsidered as a credible accident by the Licensee. The inspection team concurred with this finding.
The following concerns were noted:
Results of tue II/I analysis could not be located for Hatch Unit 1.
There was no formal calculation or analysis which documents the validity of the review and assumptions made in dismissing the effects of a postulated break in the demineralized water header on the performance of the Emergency Diesel Generator.
The Final Safety Analysis Report identifies the need for two Emergency Diesel Generators to supply minimum safeguard electrical loads on an accident Unit.
Documentation requirements are found in ANSI N45.2.11 The frequencies for all three orthogonal components of seismic excitation input motions at the piping support points in the Emergency Diesel Generator building occur at approximately nine Hertz. The demineralized water piping would be rigid with respect to the frequency amplification due to seismic motion and maximum stresses appear to be well within allowable stresses. Therefore, there is little concern of actual piping failure as a result of a seismic event.
Nevertheless, failure to produce the results of the II/I analysis for Unit 1 indicate a possible breakdown in documentation requirements.
l l
_. _ - - _ _ - _ _ _ - _ _ _ _ - _ _ -
..
.
_ - _ _ _ _ _
l j
The effects of the postulated demineralized water line break on Emergency Diesel Generator performance was not considered a credible event based on a number of considerations, some of which were documented by vendor letter and others that were based on engineering judgment. The engineering judgment considered that 64 percent of the flow' f rom a critical crack would not pass through the filter and, given the geometry of the filter box, would not be carried'into the air intake of the Emergency Diesel Generator.
While this engineering judgment appears to be reasonable, it should be considered an unverified assumption since the consequences of an error in judgment could lead to the unnecessary loss of a Emergency Diesel Generator.
There is no formal calculation or analysis available, which may be indicative of a lack of attention to ANSI N45.2.11 documentation and verification requirements. This item will be tracked as Inspector follow-up item 50-321/366-89-08-05: Documentation of Design Parameters.
(h) System Functionality Throughout the course of the inspection, the NRC team conducted in depth reviews in a number of areas specifically to determine the potential for common-mode failure of the Emergency Diesel Generator. These reviews involved the PSW system, numerous seismic issues, the CARD 0X system, Diesel Building HVAC, and non-safety piping in the Emergency Diesel Generator building.
In each
'
instance, the basic design of the system proved to be satisfactory and no common-mode failures were detected (i) PSW Pump NPSH By reviewing design NPSH calculations, it was discovered that PSW pump flow must be throttled at river levels below 61.7 feet MSL to assure minimum required net positive suction head is provided to the PSW pumps.
It is not uncommon for this type of design information to be lost in-the interface between engineering and operations during the design of the plant.
However at HNP, the requirement is accurately reflected in both the Technical Specifications and alarm response procedures and is considered an indica-tion of positive interface control.
(j) River Level Indication Control room river level indication is needed to assure certain actions are taken in accordance with Technical Specifications at low river levels.
The HNP river level i
'~
___.__________________a
_ -
-
- _
_
__
.
..
- 12 detectors are non-Q devices.
Nevertheless, the cross-checking and maintenance of these devices, along with a once per-shift visual check of a river level stick, are such that there is adequate assurance that operators are provided with timely and reliable river levels.
(k) Vital Equipment Located in Below Grade PSW Pits A number of safecy significant motor operated valves and pressure switches are located outdoors _in open below grade PSW pits. A number of questions were raised as to the ability'of this~ equipment to function in various adverse conditions, including _ seismic events and design basis flood. Through a combination of design features and operations ' procedures, all of these questions were resolved to the satisfaction of the inspection team. Although the equipment appears adequate, the licensee has agreed to review the design of the PSW pits and determine if the curbing is adequate. This is identified as inspector follow-up' item 50-321/366-89-08-05: PSW Pump Pit Design.
(1) Accident Simulation l
During the inspection, the NRC team observed the simulation I
of a loss of off site power accident on the plant simulate. At the request of the NRC team, an Emergency Diesel Generator was simulated unavailable and a failure of the standby service water pump (and therefore the swing Emergency Diesel Generator) was postulated.
The NRC team was impressed with the ability of the control room crew to manage the shutdown with only one available Emergency l
Diesel Generator and also the capability of the simulator p
to generate the accident scenario.
The entire simulation and the post simulation critique were handled by the operators with strict procedural adherence, and a high degree of professionalism.
(2) Electrical Design A broad review was conducted of the electrical power design process associated with the Emergency Diesel Generators.
The following list is a sample of electrical items reviewed:
Diesel Generator Loading Generator Load Sequencing Diesel DC Battery Sizing DC Component Voltages l'
Thermal Overload Relays Equipment Seismic Qualification Fire Protection i
l:
l
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_
_
l
_ _ _ _
_ _ _ _ _
,-
.-
Inspector concerns were provided to the Licensee for response or comment. The following is a summary of concerns in the electrical area.
They have been categorized as being related to design, design control, or hardware.
(a) Ambient Compensation on Thermal Overload Relays The effect of a motor overload is a rise in temperature of the motor winding; all overloads tend to shorten motor life by deteriorating the insulation system of the motor winding.
Small overloads of short duration cause little damage, however, if the overload is sustained the damage -
to the motor insulation system could be significant.
Therefore, thermal overload relays are installed in motor control centers to protect motors from sustained overloads.
The thermal overload relay consists of a current sensing unit plus a trip mechanism to interrupt the circuit.
Some types of thermal overload relays are sensitive to ambient-temperature of the room resulting in changes to the trip characteristic of the device (i.e., the trip will occur sooner at higher temperatures), therefore ambient tempera-ture compensating thermal overload relays are available for these applications.
The NRC_ Inspection Team reviewed the design with respect to motor thermal overload protection on 600V AC Motor Control Center MCC-1A (MPL Number 1R24-S025). The NRC Inspection Team noted that the maximum indoor ambient temperature of the Emergency Diesel Generator room is 122 degrees Fahren-heit based on Calculation Number SNH-70-012, E. I. Hatch Unit 1, Diesel Generator Building Heating and Ventilating, Revision 0, dated June 12, 1970. The Licensee was asked if LSe installed thermal overload relays for the Diesel Gener-ator IA room fan (1X41-C002A), or any other class IE equip-ment fed frcm the Emergency Diesel Generator room MCC, was an ambient temperature compensated device.
Preventive Maintenance Procedure 52PM-R24-001-05 (Allis Chalmers Low Voltage MCC Inspection, Revision 2, dated May 5, 1989)
provides thermal overload relay selection data for non-compensated and compensated devices, however this procedure does not specify the use of ambient compensated devices for the Emergency Diesel Generator room motor control centers.
In response, the Licensee performed a walkdown on all of the Units 1 and 2 Emergency Diesel Generator building MCC's.
More than 150 MCC compartments feeding equipment were inspected; one thermal overload relay associated with the battery room 1A fan was found to be of the non-compensated type and improperly sized.
In the event that the first fan failed to operate an alternate fan is available with an ambient compensated thermal overload.
Corrective action was initiated through administrative channels in the form of a Deficiency Card.
_ _ _ - - - _ _ _ _ - _ _ _ _ _ - _ _ _ _ _.
-
_
-
-
-
_
-
_
.;
k l
ANSI N45.2.11 requires that technical documentation be sufficient to trace the' design back to its input and be such that it can be verified by competent reviewers without recourse to'the original designers.
In this case, Procedure 52PM-R24-001-05 does notuspecifically state that ambient temperature compensated thermal overload relays are required to be installed. Also, the Licensee performed a walkdown in order to confirm the as-installed configura-tion. This indicates a lack of documentation regarding installed devices.
Further, an incorrect thermal overload relay was found indicating an undetected installation error.
This observation is not safety significant because only a single isolated case of an incorrect thermal overload relay was discovered.
In addition, an alternate fan is available in'the event the thermal overload relay tripped prematurely. This observation appears to be a documentation. type problem because the design requirements for these thermal overload relays are clearly specified and the installed configuration is not easily discernible.
(b) Diesel Generator Loading Calculations The Emergency Diesel Generators are sized to automatically start and carry a maximum steady state electrical load of 3250 KW for the 0 - 10 minute or 10 - 60 minute time interval following a Loss of Coolant Accident in conjunction with a Loss of Offsite Power.
The NRC Inspection Team reviewed the Emergency Diesel Generator loading calculations SEN 89-009, Hatch Unit 1, Steady State Loading Emergency Bus IE, IF and 1G, Revision 0, dated May 13, 1989 and Number 95, Hatch Unit 2, Loads on 4160V Emergency Busses 2E, 2F and 2G During LOOP and Post-LOCA Conditions and Station Blackout.
Assumption number two of Calculation Number 95 (Sheet 4) and assump-tion number two of Calculation SEN 89-009 state that motor operated valves are not considered continuous loads and
'
therefore are not accounted for'in the Emergency Diesel Generator loading calculation.
The NRC Inspection Team
,
l was concerned that the loading of these MOV's may be appreciable, especially in the 0 - 10 minute time interval.
In fact, the Emergency Diesel Generator specification l
(Hatch Unit 2, Specification List for Diesel Generator l
Sets, Inquiry Number SS-2123-12) indicates (page 15) that the MOV load is 150 hp.
l
1
\\
,
_ _. - _ _
. - _
,
(-
,'
'
L 1.-
L
15 In response to the team's concern, the Licensee performed an evaluation which demonstrated that the_MOV loading is as follows:
DIESEL GENERATOR MOV LOAD (hp)
1A 14.03 1C-12.43 2A 17.87 2C 16.33 The Licensee concluded that the_ total worst case combina-tion of MOV loads on the Emergency Diesel Generator occurs-during a LOCA. These loads, shown above, have a small impact on the Emergency Diesel Generator loading.
Each Unit 2 Emergency Diesel Generator _ room is heated by three 12.5KW room heaters.
These heaters are correctly shown as 12.5KW loads on the~ respective single line diagram, Drawing H-23315, Single Line Diagram for Motor Control Center 2A, Revision 15, dated February 22, 1989.
However, the corresponding Emergency Diesel Generator Loading Calculation, Calculation Number 95, sheet 17, incorrectly indicates that the Emergency Diesel Generator room 2A heaters (X41-B003A, X41-8003B, and X41-B003C) are rated at 9.0KW each. A similar situation exists for the room heaters associated with Motor Control Center 2C (X41-B003D, X41-B003E and X41-B003F).
These examples show that there are.various inaccuracies with respect to maintaining proper documentation of the exact loading on the Emergency Diesel Generator.
During a review of Unit 2 Calculation Number 88, Loading on Emergency Diesel Generator 2C for Path II Loads During Fire in Common Areas, Revision 1, dated December 6, 1988, it was determined that the use of a pump motor load of 500 brake horsepower for plant service water pump 2B was in error and that the correct value is 520 brake horsepower. The Licensee committed to revise the calculation by June 30, 1989, to reflect the correct brake horsepower value. The revised load on the Emergency Diesel Generator will now be 3212.21KW which is below the maximum design limit of 3250KW. This is identified as inspector follow-up item 50-321/366-89-08-06: Revised Emergency Diesel Generator Load.
ANSI N45.2.11 requires that technical documentation be sufficient to trace the design back to its input and be j
such that it can be accessed and verified by competent j
reviewers without recourse to the original designers.
In this case, the subject calculations contained an assumption l
_-
_ _ _ _ _ _ _ _ _ _ - _ _ - --
-
,
,.
which can be characterized as undocumented engineering-judgement.
The assumption was not supported by technical data or an engineering analysis.
In other examples, the load calculations contained small errors indicating a lack of proper maintenance of design data associated with Emergency Diesel Generator loading.
This observation is not safety significant.
The observa-tion is a documentation type deficiency.
The additional.
loading contribution on the Emergency Diesel Generator from the MOV's is small. This is primarily due to the electrical design wherein the RHR Injection and Minimum Flow MOV's, and the Reactor Recirculation pump suction and discharge MOV's are powered off the essential 250V DC to 600V AC inverters.
The other examples also indicate various errors with respect to accurately accounting for Emergency Diesel Generator loads.
None the less, accurate tracking of the Emergency Diesel Generator loads is necessary during the life of the plant so that modifications will not result in un;nticipatt.d loads which could degrade the performance of the emergency power system.
The NRC Inspection Team concluded that the loading calculations are presently deficient because they do not account for the MOV loads on the Emergency Diesel Generators and there are various'other load tabulation errors which need to be corrected.
It is essential that accurate and up to date Emergency Diesel Generator loading tables be maintained so that the Emergency Diesel Generator will not become overloaded and appropriate machine rating are not exceeded. When cases of small load changes are viewed by themselves the concern may be thought to be minimal, however, the changes should be I
viewed on a cumulative basis because the potential for exceeding equipment rating is increased.
(c) Design Basis For Diesel Room Fire Protection System The Emergency Diesel Generator building critical areas such as the Emergency Diesel Generator room and the day tank rooms are protected by a five Ton CARD 0X low pressure fire extinguishing system.
Fire detectors in the ceiling sense the heat from a fire and actuate circuitry for a particular area which provides an alarm, melts the Electrothermal links to close the fire doors and dampers, and releases Carbon Dioxide to extinguish the fire.
The NRC Inspection Team reviewed the design and the design basis of the CARD 0X fire protection system in the Diesel Generator rooms to determine if there were any mechanisms, particularly common mode failure mechanisms, which could
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _.
i
_ _ _ _
_ _ _ _ _ _
_
m i
result in a scenario which challenges or fails the Emergency Diesel Generators and the emergency power system.
Since a seismic event is a design basis accident for the Hatch plant and the Emergency Diesel Generators are required to operate during and after a seismic' event, the Licensee was asked several questions-regarding the capability of-the CARDDX system to withstand a design basis earthquake.
The NRC Inspection Team was concerned that a postulated failure of the CARD 0X system fire detectors, due to inability to survive a seismic event, could lead to common mode failure wherein all Emergency Diesel Generator rooms are flooded
'
with Carbon Dioxide, possibly affecting Emergency Diesel Generator operation.
In response to the teams concerns, the Licensee stated:
Fire is not postulated as a credible event following an earthquake, therefore the fire detector switches were purchased non-seismic.
These. switches are not classified as Q-List items.
~
The fire detector switches, Fenwall -DETECT-A-FIRE
27121 switches, are seismically mounted.
The CARD 0X piping is seismically mounted. Mounting is under II/I requirements and criteria for components and piping seismically analyzed to prevent damage to safety grade equipment.
The fire detector switches are inherently seismically rugged devices.
A seismic test, Fenwall E. L. Report Number 6607, Report of Test on Seismic Vibration Test-ing of Fenwall Temperature Controllers for Fenwall, Inc., under Purchase Order Number 79KJ3-823777-2, Revision 1, dated February 8,1980, of moderately high l.
acceleration magnitudes and dwell points over a l
frequency range of interest support that the detector will perform correctly and will not produce an unwanted signal.
The NRC Inspection Team reviewed Fenwall E. L. Report Number 6607 and found that the test specimen equivalent to the installed switches was not operationally monitored during
!
the vibration testing (i.e., switch contact operation or contact closure " bounce" was not monitored.
The apparent purpose of the vibration test was to determine the ability to withstand vibration without e idence of mechanical
'
damage or loss of ability to operate properly as a result q
of a seismic event.
However, the test did not demonstrate q
that the switch contact would not momentarily close during j
a seismic event thereby inadvertently initiating Carbon Dioxide release.
._____________._________________J
_ _ _ _
_
_ _ _ _ _ _ _ _ - - _ _ _ _ _ _ _ _
..
'e:
Based on a review of this issue, the team concluded that the design basis of the CARD 0X system in the Emergency Diesel Generator rooms is deficient because the basis did not consider the ability of the fire detector switches to withstand a seismic event.
This is evidenced by the fact that these switches are not classified as Q-list items and
,
. were purchased non-seismic.
In addition, the seismic test report did not demonstrate that these switches.;ould main-tain their integrity (i.e., contacts remain ~ open) during a seismic event.
Failure to clearly define and implement-the design basis and demonstrate seismic integrity of the i -:
fire detection devices has resulted in' insufficient documen-tation to substantiate that the Emergency Diesel Generator will perform as required by the design.
Final Safety Analysis Report Chapter 8, page 8.3-22, states in part that the Class IE AC power systems are designed to ensure that any design basis event does not cause either i
l loss of' electric power to more than one load group, surveillance devices, or protection system devices suffi-cient to jeopardize the safety of the Unit or loss of electrical power to equipment that could result in a reactor power transient capable of causing significant damage to the fuel or to the reactor coolant system.
The NRC Inspection Team agreed with the Licensee that the Fire Detection switches appeared to be rugged devices and the design basis seismic motions are low. Therefore, this issue is most likely a documentation type problem rather than an actual equipment problem wherein the design basis for the CARD 0X system is not completely defined and the seismic integrity of the switches is not fully substantiated, Additionally it is unclear whether a postulated release of Carbon Dioxide (i.e., a common mode failure) in all Emerg-ency Diesel Generator rooms would degrade Emergency Diesel Generator operation below acceptable levels.
Regardless, this observation appears to indicate a lack of design review for the CARD 0X system.
This is identified as inspector follow-up item 50-321/366-89-08-07: Emergency Diesel Generator CARD 0X System.
l (d) Seismic Qualification Documentation The Diesel Generators provide a highly reliable source of emergency on-site AC power to essential electrical busses.
Several abnormal conditions will initiate a diesel engine i.-
shutdown: an exciter shutdown, or a trip of the Emergency Diesel Generator supply breaker. When the Emergency Diesel l
Generator is placed in the standby mode (i.e., the mode switch is in the " auto" position) low lubricating oil i
pressure switch actuation will shutdown the Emergency I
Diesel Generator upon sensing low lubricating oil pressure.
- _ _ _ _
_
_
_
_
n
.j * -
l
'
Two lubricating oi_1 pressure switches have been installed.
' and wired in series so that two signals are necessary: to
-
trip the Emergency' Diesel Generator. The engine overspeed mechanism will also shutdown the Emergency Diesel Generator upon sensing an overspeed condition.
The NRC Inspection Team reviewed the Emergency Diesel Generator System design and _ the design 'of. support systems i
such as Emergency Diesel Generator room ventilation to determine if there were any mechanisms which could result.
,
'
L in a scenario which challenges or fails the Emergency Diesel Generators and the emergency power system. With
'!
respect to seismic qualification of equipment, the team attempted to determine if there was adequate and sufficient technical evidence which clearly demonstrates that the equipment will perform as required in the event of a design basis earthquake.
Seismic qualification is simply the demonstration by technical evidence (i.e., test reports, analysis and calculations) to substantiate that the equip-ment is capable of meeting performance requirements as specified in the design basis.
Lack of seismic qualifica-tion documentation allows the possibility of. common mode i~
failure for a system since redundant equipment can be assumed to fail in its worst case state.
The inspectors asked the Licensee to provide the seismic qualification documentation for the Emergency Diesel Generator overspeed trip mechanism and the the low lubrica-ting oil pressure switch, both of which provide Erergency Diesel Generator trip signals.
The Licensee provided Colt /Fairbanks Morse Report S-70479, Seismic Analysis and Worksheets - Diesel Generator, Purchase Order PEH-961, Engineering Report of Seismic Documentation F.M. Contract 205781 and 205971, March 25, 1980. The NRC Inspection Team observed that the seismic analysis for the engine overspeed trip mechanism is appropriately contained in the report.
!
However, the seismic report did not contain technical
,
evidence (i.e., test data, analysis or calculations) which demonstrated that the Emergency Diesel Generator low
,
I lubricating oil pressure switches were seismically qualified.
The low lubricating oil pressure switches on Hatch Unit 1 Emergency Diesel Generators are Allen - Bradley bulletin 836, Colt Drawing Number 11906076; switches on Hatch Unit 2 Emergency Diesel Generators are Detroit Switch model 222-10 high shock, Colt Drawing Number 11906729.
Based on conver-sations, the Licensee acknowledged that the evidence of seismic qualification for these low lubricating oil pressure switches (i.e., the documentation) is not specifi-cally contained in the seismic report; however, the Licensee points out that Colt stated in the summary of the report that the equipment furnished on the purchase order is l
l
_ _ _ _ _ _ -
._--_-__
.
,
. seismically qualified.
The report summary identifies these switches as simply "0PLS".
" Additionally, Colt states on page 7, of the report summary " In normal operation the strecture and components of the model'38TD8-1/8 engine see i
forces that are orders of magnitude larger than those which would be imposed by a seismic event".
In view of the above details, the team concluded that-seismic qualification, for the Emergency Diesel Generator low lubricating oil pressure' switches has not been estab-lished based on a lack of the available documentation and L
technical data. The Licensee committed to request the following information from Colt:
The seismic qualification of the engine and associated equipment referenced in " Summary of Seismic Qualifica-l tion,-dated March 25, 1980, " which referred'to a shock
'
test that included the low lubricating oil pressure switches.
Clarification of the make and model of the pressure switches utilized in the test.
If the low lubricating oil pressure switches provided
,
on the engines supplied to Georgia Power Company were L
'
different than those tested, determine and provide the type of evaluation performed by Colt.
This is identified as inspector follow-up item
,
50-321/366-89-08-08: Seismic Qualification of EDG Low
'
l Lubricating 011 Switches.
l The Emergency Diesel Generator and its support systems L
are classified as Seismic Category I.
In an effort to i
confirm that equipment is capable of performing design basis functions during and after a seismic event, it was determined that the seismic qualification documen-tation was inadequate in many cases.
The following are equipment items which have non-existent or insufficient documentation to substantiate that the equipment will perform as required:
Ventilation Louver Motors in Diesel Rooms The Vendor Certification from Construction Specialists dated August 1, 1973, states that the louvers for Specification S5-6914-19 c.re seismically qualified for Hatch Units 1 and 2.
The NRC Inspection Team requested that' the Licensee provide the seismic qualification for the Honeywell louver motors.
motors model M-436A and 445A-1001 are installed.
In
'
response, the Licensee performed Calculation SCN-89-026,
- _ _ _ _ _ _ _ _ - _
_ _ - - - _ _ _ _ _ - _ _ _
_
_
_ - _ - - - - -
,
..
a.
!
Seismic Evaluation of Motor, Revision 0, dated May 18,.
1989, which reviewed and' justified a Seismic Report for Honeywell dampers and motors for Farley Nuclear plant.
The NRC Inspection Team concluded that the seismic' qualification documentation was deficient in several areas: the documentation and the calculation was performed af ter the. start of this inspection indicating that there was a lack of documentation, documentation for the Honeywell M-436A louver motors is not available, and documentation for the Hatch louvers.is not available.- This is identified as inspector follow-up item 50-321/366-89-08-09:
Ventila-tion Louver Motors in Diesel Rooms ~.
Electrothermal Links on Diesel Room Roll-up Doers and Fire Dampers The NRC Inspection Team asked the Licensee to provide the seismic qualification documentation which demon-strates the seismic capability of the electrothermal link for the Emergency Diesel Generator room roll up
. door and fire dampers.
In response, the Licensee performea Calculation SCN 89-029, Diesel Generator Roll Up Doors, Revision 0, dated May 25, 1989, which shows that the roll up door will not fall during-and after a seismic event.
The roll up door and electro-thermal links were purchased without seismic qualifi-cation.
The Licensee also performed Calculation SCN-89-030, Fusible Links for Diesel Generator Roll Up Doors, Revision 0, dated June 3,1989, which shows the electrothermal link will withstand the stresses of a seismic event.
The NRC Inspection Team concluded that the seismic qualification documentation was deficient in several areas: the calculations demonstrating the seismic capability of the roll up door and the roll up door electrothermal fusible link were performed after the start of this inspection indicating that documentation was not available prior to the inspec-tion; and seismic qualification documentation for the fire damper electro thermal links'is not available.
This is identified as inspector follow-up item 50-321/366-89-08-10: Electrothermal Links on Diesel Room Roll-up Doors and Fire Dampers.
Diesel Battery Racks
,
Documentation is not available to demonstrate that the as-installed bolting pattern of the Emergency Diesel Generator battery racks to the floor is consistent with the mounting requirements in the seismic test report for the batteries. This is identified as inspector follow-up item 50-321/366-89-08-11: Diesel Battery Rack Mounting
_
__ _ _ - __ __- - - _ ___ _ -____.
.
_ _ _ _ _
_
--
w
'
.
- ,W
4 Diesel Generator Room Ventilation-Fan Thermostats and Heater Thermostats-Irtresponse to the teams request for seismic qualifi-cation documentation for the Emergency Diesel Generator p
room thermostats, the Licensee provided a Honeywell report, Report EXC 4659, dated June -14,1974, which seismically qualifies the Unit I thermostats, The Licensee also prepared Calculation SCN-89-028, Seismic Qualification of Penn Thermostats, Revision 0, dated May 25, 1989, which seismically qualifies the Unit 2 Penn Controls thermostats.
However, the Unit 2
"
thermostats were purchased non-seismic and documenta-
,
tion was again prepared after the start of-this inspection.
The Emergency Diesel Generators'are classified as seismic category I equipment.
Final Safety Analysis Report, section 3.7A.A.1, states that seismic category I equipment must withstand the postulated seismic occurrence and that IEEE Standard 344-1971, Guide for-Seismic Qualification of Class IE electrical Equipment-for Nuclear Power Generating Stations, is used for seismic qualification. Section 4 of IEEE 344,.1971, requires documentation to demonstrate equipment performance. Additionally, Final Safety Analysis Report, section 8.4.4, page 8.4-8A, states in part that the Emergency Diesel Generator low lubricating oil pressure interlocks were analyzed as part of the seismic qualification.
This observation is a documentation type problem which appears to be generic in nature at the Hatch plant.
Throughout the course of the inspection, the Licensee obtained or developed technical information or rationale which indicated the equipment would most likely perform as required, particularly in view of the low seismic motions at Hatch.
However, it is apparent that the documentation to substantiate the design was weak in many cases.
(e) DC Battery Sizing Room temperatures that were used in load study calculations to size the Emergency Diesel Generator batteries for j
Units 1 and 2 are as follows:
'
Battery Temperature j
1A 58 Fahrenheit 1B 55 Fahrenheit 1C
Fahrenheit 2A 65 Fahrenheit 2C 65 Fahrenheit i
__-_..__._..___-._______.,.-__.-.______s-
-
- - -
_ _ _ _ -_ -
_-
..
i
However, the design basis minimum temperature for these battery rooms is 45 Fahrenheit,. since the battery room heaters are sized to maintain 45' Fahrenheit within the-
' rooms when outside temperature.is at its design minimum of 20 Fahrenheit.
Battery sizing should have been based on 45 Fahrenheit rather than the temperatures listed above.
The Final Safety Analysis Report for both Units states that the batteries are designed to Class IE requirements.
IEEE 485-1983 " Recommended Practice for Sizing Large Storage Batteries for Generating Stations" provides appropriate-temperature correction factors for sizing batteries where the temperature is less than 77 Fahrenheit.
This standard is listed as reference 1 in the Unit 1 battery sizing calculation.
The capacity of a lead acid battery decreases below 100 percent when temperature is less than 77 Fahrenheit.
Where battery rooms are not maintained at 77 Fahrenheit, compensation for the lower temperatures must be included when sizing the battery.
In this case, sufficient battery capacity is not assured when room temperature drops below the temperature compensation of the calculation (e.g.
between 65 Fahrenheit and 45 Fahrenheit for battery 2C).
The Licensee has agreed to revise the battery sizing calculations for both Unit I and Unit 2 to compensate for a design basis temperature of 45 Fahrenheit.
Engineering activity REA-HT-9673 has been initiated to address this task, with a scheduled completion date of September 30, 1989.
Final Safety Analysis Report Table 2.3-1 indicates there are zero normal heating degree days at a base of 65 Fahrenheit between the months of May and September.
Therefore the batteries are considered operable during this period. This is identified as inspector follow-up item 50-321/366-89.08-13: DC Battery Sizing.
(f) Diesel Battery Specification In response to an NRC inspection team request, the Licensee provided a copy of Purchase Order G-52508, dated May 12,
'
1981, for the Unit 1 125V DC Emergency Diesel Generator batteries.
This purchase order did not refer to an equipment specification nor was the Licensee able to provide a copy of an equipment specification for the Unit 1 batteries.
Equipment specifications define the necessary operating and technical requirements that the vendor must address in providing equipment to the purchaser.
Equipment ratings and capacity as well as operating parameters, both normal
-_
_ _ - _ _ _ _ _ _ - _ - _ _ _ _ - - - _ _ - _ _ _ _ - - - _ - _ - _ _ _ _ _ _ _ - _ _ - _ _ _ _
. _ - _ - _ _ _ _ _ - _ _ - _ _ _ _ _ _
__-___--_____a
-_-
,
24'
and emergency, should be defined. Also, the proper-quality control requirements must be in place to assure appropriate manufacturing and material selections-Without a specifi-
.
cation, it'is not clear what quality and technical require-ments were met by the vendor.
In addition, the Unit I calculation for sizing of the batteries, Number 82, Revision 0; Load Study of 125V Dr Emergency Diesel Batteries IA, IB,.and IC, is dated March 25, 1988.
This indicates the calculation tp size the battery was performed after purchase and installation of the batteries.
The Final Safety Analysis Report states that the batteries and battery racks are Class'1E equipment.
Design documenta-tion requirements are contained in ANSI N45.2.11.
Design documentation in the case of these batteries does not provide the assurances that performance will be in-accordance_with design requirements.
Nevertheless, this concern is not considered to be safety significant unless in gathering the appropriate documentation, information is obtained to suggest that design requirements may not be met. This is identified as inspector follow-up item 50-321/366-89-08-13: Diesel Battery Specification.
(g) Diesel Battery Over Voltage Hatch Nuclear Plant contains five Emergency Diesel Generators, three of which are associated with Unit 1, although one of these Emergency Diesel Generators can supply Unit 2 loads, and two associated with Unit 2.
Each Emergency Diesel Generator has its own auxiliary 125V DC safety related battery that is located in a separate room next Eto its l
respective Emergency Diesel Generator.
The Unit'l diesel batteries are lead calcium while the Unit 2 batteries are lead antimony type.
Each Unit also has its own safety related 125-250V DC station battery system.
The Unit 1 Emergency Diesel Generator batteries are floated continuously on the Unit 1 DC bus at 131V DC, while the Unit 2 Emergency Diesel Generator batteries are floated on i
the Unit 2 DC bus at a voltage of 135V DC.
Due to the L
close physical proximity of the batteries and supported DC l
loads, or end devices, there is minimal cable voltage drop.
This results in the end devices being connected to a voltage that is above their nameplate ratings of 125V DC.
Further, during periods of equalizing charges to the batteries, battery-voltage increases to 140.5V DC.
This results in subjecting the end devices to additional over voltage.
_. _-- __
..
3:
The Licensee could not provide adequate information that the end devices could sustain the over voltages that are L
present during either the float condition or the equalizing charges.
It is also noted that the Licensee via REA-HT-4622,
.
March 14, 1984, requested Southern Company Services (SCS)
to i.nvestigate reducing the DC power system voltage for the Emergency Diesel Generator auxiliary batteries.' Avail-able documentation reviewed by the NRC inspection team indicated that'SCS noted that in their opinien, the most-acceptable solution was to float and equalize voltages at L
the proper levels.
There was no mention of voltages that were above equipment rated design voltages of the end devices.
IE Information Notice Number 83-08, Component Failures Caused by Elevated DC Control Voltage, was issued March 9, 1983. While no specific action or response was required, licensees were expected to review the information for applicability.
Events noted in IEN 83-08 showed that DC safety related control components and indicating circuit components which operate for a sustained period of time at elevated voltages or voltages above their rated design voltage are subject to accelerated degradation which may have some impact on plant safety. A careful balance of rated voltage for components in DC systems must be maintained to assure maximum voltage during equalizing charging and float charging does not adversely affect the components.
The Licensee is currently conducting further investigations to determine the acceptability of the end devices when subjected to over voltage. This is identified as inspector i
follow-up item 50-321/366-89-08-14: Diesel Battery Over Voltage.
(h) Unit 1 Diesel Generator Transient Analysis In November 1987., the architect-engineer firm for Unit 2, Bechtel Eastern Power Corporation, discovered that, as result of a deficiency in a design change package, the
.
Emergency Diesel Generators for Unit 2 could become over I
loaded during a loss of coolant accidenu combined with a i.
L coincident loss of off site power. The worst case loading sequence for the potential over load condition was investi-gated and the Emergency Diesel Generator vendor subsequently confirmed the capability of the Emergency Diesel Generators to perform their intended safety function.
!
i I
I l
- - _
_
- _ _ _ _ _ - - _ _ _ _ - _ _ - _ _ _ _ _ _ _ _ _ _ _ -
_ -_-_-____ _ -
.
.
- ,.
The root cause of this event was determined to'be a lack ~of full understanding by design personnel of the impact of load sequencing on the 30 minute. rating of the Emergency L.
Diesel Generator..
Further investigation indicated that the L
Emergency Diesel Generator vendor had not communicated the importance of load sequencing to either BEPC or the AE for Unit 1, Southern Company Services.
l In reviewing this matter, the NRC inspection team' asked the I
licensee to provide information and documentation as to how the knowledge of the impact of sequencing loads at the L
upper end of the Emergency Diesel Generator rating had been l
incorporated on Unit 1.
Steady state loading calculations for Units 1 and 2 were provided along with a transient loading analysis for Unit 2.
A transient loading' analysis for Unit I was being prepared and was not available for review.
Further, at the time of. this inspection the report fror the vendor was not available for review. This is identified as inspector follow-up item 50-321/366-89-08-15:
Transient Load for Unit 1.
The Units 1 and 2 FSARs state that the Emergency Diesel Generators have the capability of starting and accelerating all ESF and safe shutdown loads to rated speed and demand L
power in the required times.
The Unit 1 equipment specifi-cation also contains the following paragraph:
The continuous output rating of the generators shall
'
be approximately 2850 KW, 3570 KVA at 0.8 pf. The Units shall have a two hour over load rating of 100 percent of the continuous rating. The Units shall-be sized to start and drive continuously the loads listed in Table A, page 2.
Momentary voltage on starting any individual load shall not drep below 2800 volts at the generator terminals and return to 90 percent of rated voltage in one second.
Generator over voltage shall not exceed 110 percent.
If manufacturer can not meet these limitations, he shall state the voltage dip and over voltage which can be guaranteed.
The sudden large increase in current drawn from tt.?
Emergency Diesel Generator by the start of large induction motors in rapid succession can result in substantial voltage reduction.
The lower voltage could prevent a motor from starting, i.e. accelerating its load to rated speed in the required time, or the reduced voltage could cause a
'
running motor to slow down or stall. Other loads., because of low voltage, might be lost if their contactors drop out.
Recovery from the transient caused by starting large motors or from the loss of a large load could cause Emergency
- _ _ - _ _ - _ - _ _ _ _ _ _ - _ - - _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ _ _ _ _ - _ _ - _ _ _ _ _ - _ - - - _
_ - _. - - _ _ _ _ _ _.
__
. _ _ _ _ _ _
- _ -. - _ = - - _
' d,
"
lx 27-
,
Diesel Generator over speed which, in the extreme,'might trip the Emergency Diesel Generator on over speed.
The same consequences could also result from the cumulative
effect of a sequence of more moderate transients if the; system is not permitted to recover sufficiently between l
successive steps in the. loading sequence.
On Unit 2, a transient loading analysis or calculation
.
. exists.
A study to address the resultant effects on the L
electrical auxiliary system during a loss of off site -
l
' power, based upon the input from the transient analysis, has been performed. A similar analysis and study for Unit L
does not exist.
The Licensee has stated that such a.
transient analysis is being developed by the Unit I vendor.
Upon receipt of the analysis, review of the report and its effect on 'the Unit 1 electrical auxiliary system by the Licensee or by the AE for Unit I will be needed.
p (i) Design Basis Review The design and design basis of the Emergency Diesel
,
l Generator system was carefully reviewed specifically to determine if there were any mechanisms which could cause 1.
the Emergency Diesel Generators to fail to perform their
'
design function.
Specifically, mechanisms which could possibly result in a common-mode failure of the Emergency Diesel Generators or the emergency power system were investigated, as discussed below:
l
CARD 0X System The CARD 0X fire protection system in the Emergency Diesel Generator building has thermal sensors in each of the five Emergency Diesel Generator rooms. The design basis of the CARD 0X system did not consider a seismic event.
Hence, the sensors wcre purchased as non-seismic devices. The
)ncern involved inadvertent actuation of the sensors during a seismic event result-ing in flooding of the Emergency Diesel Generator rooms, any one to all five, with Carbon Dioxide, to the detri-ment of Emergency Diesel Generator operation.
Since the sensors are rugged devices and since seismic motions at HNP are low, inadvertent actuation is unlikely'and this is primarily a documentation problem.
It does point to a weakness, however, in the design review and design verification process associated with the CARD 0X system.
Seismic Qualification Documentation The Emergency Diesel Generator system and its support systems, such as building ventilation, are classified Seismic Category I.
In an effort to confirm that the
_ _ _ _ _ _ _ _ _ _
I
_
_
.
S F
U
'c
<
.
equipment was capable of performing design basis func-tions during.and after a seismic event, it was deter-mined that seismic qualification documentation was inadequate in many cases to support design requirements.
The following are equipment items where weak documenta-tion was discovered:
Ventilation Louver Motors in Emergency Diesel Generator rooms Electro-thermal links on roll up fire doors and dampers Diesel battery racks Diesel room thermostats Emergency Diesel Generator low lubricating oil pressure switches Turbine building high flow isolation pressure switches As in the case of the CARD 0X sensors discussed above, it is unlikely that this equipment will fail in a seismic event.
Nevertheless, the documentation presented did not support design requirements.
(j) Design Control The following con: erns are related to the design control process:
Diesel Generator Loading Calculations Emergency Diesel Generator loading calculations assumed that electrical loads from motor-operated valves were insignificant and therefore were not included in the loading tabulations.
No justification was provided to support this assumption.
Actual MOV loads were determined to be:
Unit 1 17HP Unit 2 14HP The NRC team stated that the calculations should have accounted for the MOV loads.
In addition to MOV loads, the single line diagram for Unit 2 indicates Emergency Diesel Generator room heaters to be 12KW.
The applicable loading calculation, Number 95, identifies these heaters as 9KW.
The calculation should be revised to reflect actual load. This is identified as inspector follow-up item 50-321/366-89-08-16:
Diesel Generator Loading Calculations.
_ _ _ _ _
.
__
_
_ _ - - _ _ _ - _. ___ _ __
t p
- g
2 Unit 1-Transient Load Analysis For Unit 2, the. Emergency Diesel Generator vendor provided a transient. analysis that addressed voltage and frequency values on the generators due to load block sequencing for a loss of coolant accident coincident with a loss of off-site power.
No such analysis was performed on Unit 1.
.3 Diesel Battery Sizing Calculations Unit 1 and Unit 2 Emergency Diesel Generator battery sizing calculations, Numbers 82 and 83, include minimum room temperatures of.58 degrees Fahrenheit and 65 Fahrenheit respectively.
This value is above the minimum ambient temperature in the rooms of 45 Fahrenheit. The Licensee committed to revise the calculation to demonstrate the adequacy of the batteries and to complete this work prior to September 1989 (i.e. prior to the on set of cutdoor temperatures below 65 degrees Fahrenheit).
Diesel Battery Specifications Unit 1 Emergency Diesel Generator batteries were purchased in May 1981.
No equipment specification was provided to the vendor for this purchase. The NRC team was concerned as to'the lack of documentation of the requirements, technical and quality, imposed on l
the vendor.
( k) Hardware The'following concerns were identified which could impact the performance of installed equipment:
Diesel Generator Battery Over Voltage Emergency Diesel Generator batteries are exposed to normal float voltages and equalizing charge voltages as follows:
Unit Float Equalize
131V 140.5V
135V 140.5V These voltages exceed the nameplate ratings of 125V of a number of the DC loads or end devices supplied by the DC bus.
It has not been determined if all of the end devices can tolerate the additional voltage over a sustained period of time.
... _ _ _ _ - _- - _ _ - __
--;,- =-
-
l,
.
T..
l
..
_
3 0..
i
Ambient Compensation of Thermal Overload Relays
'
Emergency Diesel Generator room design maximum tempera-ture is 122 Fahrenheit. The Licensee was requested to confirm that the installed thermal overload relays for Emergency Diesel Generator.1A room fan (1X41-C002A)
was an ambient temperature compensated device.
The l
Licensee's review indicated it was not.
However, a l
walkdown of 150 MCC compartments feeding equipment for l-all Units 1 and 2 Emergency Diesel Generators. revealed l
that this particular fan was the only non-compensated l
device.
Although this. concern resulted in an isolated I
undetected installation-error, the fact that a walkdown was needed indicates a lack of documentation regarding installed devices.
(3) Chemical Design This part of the inspection focused on the Emergency Diesel Generator fuel oil sampling and analysis program.
A comparison was made of Technical Specification requirements, Final Safety i
Analysis Review requirements, procedural requirements, system l
design'and_ industry good practices for Emergency Diesel Generator
' fuel oil sampling and analysis, against the program that the licensee had in place at the time the inspection was announced, April 12, 1989. As a result of the inspection several program deficiencies were noted and are discussed below:
,
(a) Diesel Generator. Fuel Oil Sampling and Analysis Sampling and analysis requirements for the Emergency Diesel Generator fuel oil storage tanks were specified in Unit 1 (section 4.9. A.2.d) and Unit 2 (section 4.8.1.1.2.c)
,
l Technical Specifications and state.that operability of the Emergency Diesel Generators shall be maintained "At least once per 92 days by verifying that a sample of Emergency Diesel Generator fuel oil from the fuel oil storage tank,
,
l obtained in accordance with ASTM-D270-65, is within the l
acceptable limits specified in Table 1 of ASTM D975-74 when checked for viscosity, water and sediment." At the time the inspection was announced, sampling of the Emergency Diesel Generator fuel oil storage tanks was performed in accordance with procedure 62CH-SAM-003-05, 011 Sampling, Revision 2, dated December 1, 1986. This procedure describes the sample locations, sampling techniques and specific limits for viscosity, water and sediment in Emergency Diesel Generator fuel oil.
The inspectors reviewed quarterly sample results from the first quarter of 1987 through the first quarter of 1989.
These samples were analyzed as required by Technical Specifi-cations for viscosity, water and sediment and found to be within the limits of 62CH-SAM-003-OS and ASTM D975-74, Table 1, Standard Specifications for Diesel Fuel Oils.
l
_ - - _ _ _. _ _ - _ _ _. _ _ _ _ _. -
_
.I
,_
_-_
-_ -__.
__- --.
-.
- -
-
-
_ - _ -
_- _
--
- _ _ _ _. -
i l
l
.
l
'
o Procedural requirements specified samples to be collected from a sample line on the discharge side of one of two fuel oil transfer pumps.used for recirculation'after a'30 minute recirculation period.
The capacity of each transfer pump was approximately 30 gallons per minute; therefore, only about 900 gallons of fuel oil could be recirculated prior to sampling.
The capacity of each fuel oil tank is approximately 40,000 gallons.
This short recirculation time did not assure a representative sample of the fuel oil because of insufficient mixing of the fuel oil in the tank
,
I.
prior to sample collection. The licensee could not justify l
the sampling method in place at the' time of the inspection and also indicated that the sampling method did not comply I
with methods described in ASTM D270-65, Sampling' Petroleum and Petroleum Products, as required by Technical Specifica-tions. This was identified as an apparent violation of Units 1 and 2 Technical Specifications. The licensee identified some of the sampling program deficiencies in QA
.
Audit 88-MNT-2; (during November 1988) however, the correc-l tive actions for these findings had not been completed.
This is collectively combined with other examples and-constitutes violation 50-321/366-89-08-01, Failure to Perform Timely Corrective Actions.
Section 9.5.4.5 of the Unit 2 Final Safety Analysis Report stated that " Samples of the fuel oil from all tanks are analyzed periodically to ensure that the fuel oil quality requirements of the Emergency Diesel Generator manufacturer i
are met."
Fuel oil specifications as stated in the vendor manual, Fairbanks-Morse, included requirements for viscosity, carbon residue, sulphur, flash point, bottom sediment, water, ash, pour point, distillation, potential gum, cetane number, high heat value and A.P.I. gravity. Procedure Number 62CH-SAM-003-05, Revision 2, and the Technical Specifications only address requirements for viscosity, water and sediment. This discrepancy between the procedure, Technical Specifications and Final Safety Analysis Report was also identified during a quality assurance audit conducted during November 1988, 88-MNT-2. At the time of this inspection, corrective actions had not been finalized.
A previous fuel oil starvation incident during a surveill-ance run of the 2C Emergency Diesel Generator during November 1986, caused by the accumulation of sludge in the 2C fuel oil day tank and plant commitments in response to IEN 87-04, Diesel Generator Fails Test Because of Degraded Fuel, March 6, 1987, had also identified deficiencies in the sampling and analysis program for the Emergency Diesel Generator fuel oil storage and day tanks. At the time this inspection was announced, no procedural changes had been completed to upgrade the fuel oil sampling and analysis
>
___________m
_-_. _ _ _ _ _.__
__ --._._..__. _
-,_ - _
__.
___
_ _
-
. _ _.
_ _ _ - _.
._
_
_ - - _ _ _ _ -. --
-_
.. -
..
-32 program. The last revision to 62CH-SAM-003-OS, Oil Sampling, was; dated December 1, 1986, Revision 2.
In response to IEN 87-04, tne licensee had scheduled to " develop a program to consume all Emergency Diesel Generator fuel oil stored for a defined period of time and/or develop a program'to sample, inspect and clean the Emergency Diesel Generator
,
fuel oil storage and day tanks to prevent excess particulate l-I and sludge build-up by December 31, 1987." A Quality Assurance audit finding, LR-QAM-012-1188, dated November 29, 1988, stated that only portions of the sampling program were in place and that the schedule of December 31, 1987 i
I was not being adhered to as discussed in the response to IEN 87-04. This delay in corrective actions was not appropriate, in that the identified deficiencies in the fuel oil sampling and analysis program could have rendered all'five Emergency Diesel Generators inoperable.
10 CFR 50, Appendix B, Section XVI, Corrective Actions, specifies that condition adverse to quality should be promptly identified and corrected. The previously described scenario does not conform to these requirements and is a violation of these
!
l requirements. The licensee identified some of the sampling program deficiencies in QA Audit 88-MNT-2; however, the corrective actions for these findings had not been completed and in some cases the corrective actions were not adequate.
This is collectively combined with other examples and constitutes violation 50-321/366-89-08-01, Failure to
.
Perform Timely Corrective Actions.
l (b) Heat Exchanger Performance The Emergency Diesel Generators are cooled while running by a closed water jacket cooling system associated with each individual Emergency Diesel Generator. Heat is removed from each cooling water system via heat exchangers to the plant service water system which is a raw water system.
Heat from the lubricating oil system and the turbocharger air is also removed to the plant service water via heat exchangers. The licensee indicated that there were no apparent problems in the area of Emergency Diesel Generator heat exchanger fouling or integrity. All heat exchangers were inspected during 1988 outages and no signs of biolcgical fouling was noted on the service water side of any heat exchangers.
Also, no tubes had been plugged to date. The tube material was phosphorized admiralty brass.
The heat exchangers are inspected every 18 months in accordance with surveillance procedure 52SV-R43-001-05 and are cleaned and tested every five years in accordance with preventative maintenance procedure 52PM-R43-015-05.
L
_ -
_ _ _ _
_ _ _ - _
.-
_ _ _
_.____ - _
-
._
. -
_
-
-_.
f
-
j
%
(c) Fuel Oil-Storage
'
During a review of the design of the fuel oil storage tanks, the inspectors observed that there was no design feature of the tanks that allowed fuel oil contaminants, degradation products and water / sediment to be easily removed from the bottom of.the tanks.
Because of the November 1986, fuel oil starvation incident, all the fuel oil storage and day tanks were emptied and cleaned to remove accumulated sludge, but at the time of the inspection.no procedures.were in place to remove accumulated sludge from the bottom of the tanks. The licensee indi ated; however, that chemistry procedures were being upgraded to include periodic sampling of the storage tank bottoms and options were being considered and evaluated for periodic sludge removal.
As discussed in paragraph 3.b., the fuel oil storage tanks are recirculated by using one of two fuel oil transfer pumps on each of five tanks. The storage tanks are installed horizontally.
The transfer pumps' suction bases are approximately 1.75 inches from the bottom of the tanks near one end of the tanks. The recirculation returns are also at the same end of the tanks as the pump suction bases. Because of this design, the storage tanks appeared to be recirculated properly at one end only. The licensee was aware of this problem and was evaluating potential solutions. This is item will be tracked by the NRC as inspector follow-up item 50-321/366-89-08-17:
Diesel Fuel 011 Storage Tank Design.
- (d) Existing Procedures As part of the validation process for a new upgraded revision to chemistry procedure 62CH-SAM-003-OS, Oil Samp1-ing, discussed in section 3.f., samples from all five fuel l
oil storage tanks were collected on May 18, 1989, and I.
analyzed for the parameters recommended by the Emergency Diesel Generator manuf acturer.
Representative composite l
samples were collected in accordance with methods recom-E mended in ASTM D270-65. The sample results were within
,
limits for all parameters except high heat value.
The acceptance criteria for high heat value was 19350 Btu /lb and the results from all tanks showed values slightly
!'
l l owe r.
The licensee indicated that if the lowest of these l
values, 19224 Btu /lb, was used to recalculate previously assumed Emergency Diesel Generator fuel oil consumption rates at 3250 KW, an increase of approximately 1.8 gallons per hour would be noticed.
The effect of this slight increase in consumption rates is addressed in Seven Day Fuel Storage Requirement section of this report.
-
-
.. _ _ _ _ _ _ _ _ -
_ _ _ _
__
-.
,_ _
- _ _ - - _ - - _ -
- - -. _
_ - - -
~'
..
.
-(f) Proposed Procedures or Procedures in the Validation Procsss.
During-the inspection, the licensee was validating
= for future use, -Revision 3.of chemistry procedure
.62CH-SAM-003-05, Oil Sampling.
Revision 3, included the foilowing' improvements:
Upgraded; chemical specifications for Emergency Diesel Generator fuel.' oil to include the requirements of the Emergency Diesel Generator manufacturer
Sampling and analysis requirements for fuel oil delivery trucks prior to off-loading to the main storage tanks, previously covered by procedure 62CH-SAM-029-OS
Instructions for obtaining representative samples from'
the fuel oil storage tanks in accordance with ASTM D270-65, as required by Technical Specifications 4_
Sampling and analysis requirements for fuel oil day tanks
Instructions for the addition of biocide to the main fuel oil storage tanks, previously covered by Chemistry Standing Order 50-CHM-05-1288, December 21, 1988
Sampling and analysis requirements.for the fire pump and Security Emergency Diesel Generators l
Implementation and final approval of this procedure and other corrective actions' associated with the Emergency Diesel Generator fuel oil sampling and analysis program will be reviewed during subsequent inspections.
This is identified as inspector follow-up item 50-321/366-89-08-18:
Fuel Oil Sampling Program.
(4; Configuration Control
.,
Prior to this inspection the Licensee embarked on a self initiated Configuration Management Program.
The first phase of this program was completed during this inspection. The licensee defined the scope of this program as follows:
Configuration management is the integration of the set of processes which ensure that the plant's physical and furc-tional characteristics conform to the design requirements as provided by the plant's design basis, and the plant's license conditions.
In addition, this integration includes I
those processes which support the operation and maintenance l
_ _-__ _
_
__ _ _
...
m.
of the plant-such that they also are integrated with the design and license requirements.
Finally, the integration
formalizes the mechanism to transfer relevant information
for the status quo, i.e. day-to-day activities and. change (for example, a change to the physical' plant, procedure, design requirement) or a change to any process itself.
When the plant, plant documents, design basis, or license conditions are modified, this integration of processes provides management and staff a method for reviewing,.
approving and implementing the change in an integrated, controlled and auditable manner.
This program will in part reconstitute the design basis for systems important to safety. The concerns addressed in the Mechanical and Electrical Sections of this report, relating to design control and design basis documentation, will be addressed-in this program.
The Licensee has completed phase one of this program which involved: determining the scope of the program, approval for resource expenditures, and an initial look at the design basis for portions of the plant. The second phase of the program has considerable resources allocated and is scheduled to begin before the issuance of this report.
(5) Off-site Engineering The licensee has limited engineering support and resources available on site; however, the engineering support available through corporate engineering, the architect engineer, and contractors appears adequate. The Licensee's engineering support for this inspection was well coordinated and the efforts of the engineering staff contributed significantly to the timely completion of this inspection.
L b.
Operations Inspection of the operations functional area included review of procedures, operator / procedure interface, Significant Occurrence Reports., generic communications, obser"ance of Emergency Diesel Generator operations, and Emergency Diesel Generator performance indicators.
Performance in this functional area was generally good.
Procedures were adequate, operators were knowledgeable of system operating requirements and familiar with system hardware.
SORS and generic communications were generally reviewed in a prompt and adequate manner and Emergency Diesel Generator performance indicators suggested an improved reliability of the Emergency Diesel Generators over the previous three years.
(1) Final Safety Analysis Report Commitments and Vendor Requirements Incorporation of vendor manual requirements and Final Safety Analysis Report commitments into procedures was adequate and the
- _ _ _ _ _ - _ _ _ - - _ _
_ - _ _ _ _ - - _ _ - _ _ _ _ - _ _ _ _ - _ -
.. - - _
. _ _ _
_ -__. _ _ _ _ _ _ - _ - _ - _ _ -
_ -
_ _ _ _ _ _ _ _ _
.
..
operator / procedure interface was good. The following procedures were reviewed for conformance to Final Safety Analysis Report commitments and vendor manual requirements:
3450-R43-001-IS, Diesel Generator Standby AC System Operating Instructions, Revision 5 3450-R22-001-IS, 4160 VAC Systems 0peration, Revision 2 31E0-EOP-001-15, Inside Control Room Generic Electrical-System Recovery Procedure, Revision 1 31EO-EOP-001-15. Inside Control Room Generic Plani Service Water Recovery Procedure, Revision 1 31E0-EOP-002-15, Outside Control Room Electrical System Recovery, Revision 0 31E0-EOP-002-15, Outside Control Room Diesel Generators Generic System Recovery Procedure, Revision 0 The procedures adequately incorporated Final Safety Analysis Report commitments and vendor manual requirements for Emergency Diesel Generator operations. A walkthrough of the outside control room recovery procedure, which included a local Emergency Diesel Generator start simulation, demonstrated the operators familiarity with the procedure and related hardware.
The procedure provided adequate guidance to start and monitor Emergency Diesel Generator operation as well as local loading and unloading.
The plant practice of locally starting the Emergency Diesel Generator for periodic maintenance provided assurance that operators were familiar with local start activity This walkthrough included operator actions for Emergency Diesel Generator operation in the event of a control room fire.
The operator was cognizant of the additional action requirements and locations of all hardware to be manipulated.
Observation of a monthly surveillance of Emergency Diesel l
Generator IC which required starting, loading, and unloading from the control room demonstrated adequate use of the procedure by the operator.
Manipulation of controls and utilization of main control board instrumentation to monitor system parameters was good.
(2) SORS and Deficiency Cards l
The following SORS and deficiency cards related to Emergency
,
l Diesel Generator systems were reviewed:
SOR 2-87-198/065 DC 1-88-4429 SOR 1-87-1962/418 SOR 1-88-2833/153 DC 1-88-5477 DC 2-88-3683 u___
_ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - - - _ _ _ _ _ _ _ _ _ - - - -. __
_ _ - - -
-___ _ _
_________--_____
.
_
_
--
_ _ _--- - -. _-- ---____ _
.
'
,
SOR 1-87-0914/273-SOR 1-87-0899/266 SOR 2-87-0029/005 SOR 2-87-0436/155 SOR 2-87-0383/141 SOR 1-87-312/084 30R 1-87-0135/030 SOR 1-87-1610/387
- SOR 1-87-1613 /387 SOR 1-87-1612/387 The majority of these reports were adequately evaluated and documented,:however there were some exceptions.
SOR 1-87-1962/418, addressed the coils on the IF switchgear LOCA timer which " appeared to be burned up".
The documentation did not identify the impact, if any, of this deficiency on the operability or reliability of the Emergency Diesel Generator system to perform its function, i.e. sequence on loads from the 1B/2B Emergency Diesel Generator.
Since the Emergency Diesel Generator successfully completed operability tests prior.to and following identification of this deficiency it. appears'no oper-ability impact was involved.
DC 1-87-4429 addressed the calibration of IC Emergency Diesel Generator lubricating oil
' temperature switch under laboratory conditions rather than field conditions which resulted in improper switch operations.
Corrective action was weak in that the technicians were
" advised" of th' need to calibrate under field conditions. A more effective resolution would have been to enter the field condition equivalent calibration as a precaution in the admini-
.strative calibration procedure.
SOR 1-88-2833/153, addressed a trip of the IB Emergency Diesel Generator one minute after start No root cause or recurrence control was determined.
Although maintenance promptly performed troubleshooting actions, no hardware cause was identified.
The responsibility for root cause determination was transferred to operations with no evident action.
Approximately eight months later the SOR was
{
-
closed. 50R 1-87-0914/273, addressed a missed holdpoint en QC maintenance.
A bearing cap clearance measurement could have been' accomplished in the shop or field but was not performed.
The report did not clearly address hew this maintenance task l
was continued beyond the QC holdpoint without the required performance verification or if this might indicate a more broad based problem with maintenance procedure compliance.
SORS 1-87-1610, 1611, 1612/387 identified a failure of Fuel Oil Day Tank high level alarms to activate on Unit 1 Emergency Diesel Generators.
No root cause determination was accomplished.
Discussion with plant personnel indicated no safety problems were ri.ated to this issue.
However, this represents a weakness in providing a complete documentation of deficiencies parti-cularly with respect to actions taken prior to declaring a root cause indeterminate.
The following generic communications associated with the Emergency Diesel Genert ar system were reviewed:
SER 68-82-1 SER 68-83-1 SER 45-85 SER 19=87 SOER 83-06 IEB 83-03 IEN 86-73 IEN 87-09 IEN 87-42 IEN 89-07
__-____-___-________-__D
-
ju
,
s;
.. -
L
l Staff response to.these issues was generally adequate and timely.
Applicability screenings were prompt and well. documented. A sample of completed corrective actions demonstrated that actions were correct and the tracking system fer generic communications was effective. Those communication issues still open were active, i.e. no items were inactive due to lack of response.
(3) Simulator Demonstration The operations staff provided a simulator demonstration encom-passing a small LOCA on Unit 2 in conjunction with a LOOP. The purpose of this demonstration was to observe the operator use of E0Ps, manipulation of Emergency Diesel Generator controls, familiarity with limiting Emergency Diesel Generator operation conditions, and changing load on the Emergency Diesel Generator to accommodate plant conditions.
The demonstration indicated
~the operators could deal effectively with this scenario. The additional constraint of providing only one Emergency Diesel Generator for Unit 2 did not result in a compromise of reactor core integrity for this scenario.
The E0P flowpath and proce-dures provided adequate guidance for Emergency Diesel Generator operations and limiting conditions.
The operators were l
cognizant of Emergency Diesel Generator loading limitations and competent at manipulation of Emergency Diesel Generator controls.
The following obser etions were made with regard to general actions related to this event scenario:
a.
Communications were effective, however operators reliance on the Gaitronics could result in a communi-cations impairment on certain scenarios of LOOP when the power supply is lost.
E0P training or procedures L
may need to address this factor.
This is identified
'
as inspector follow-up item 50-321/366-89-08-19:
L Gaitronics Communications.
'
b Loss of the security systems power supply would impact operator access to vital areas.
Determine what method other than security personnel " piggy back" is available for operations personnel to enter vital areas in this situation.
This is identified as inspector follow-up item 50-321/366-89-08-20: Access to Vital Areas.
c If the swing Emergency Diesel Generator IB lost its associated PSW pump the PSW supply from the' Unit dedi-cated PSW systems would not provide adequate cooling for the swing Emergency Diesel Generator without changing out the orifice in the supply line.
Dctermine if the cooling flow allowed by the original orifice would allow operation of the swing Emergency Diesel Generator with restricted load. A similar scenario with a large b.eak LOCA could result in greater initial l
i l
l
- _ _ _ _ _ _ _ - _ - _ - - _ _
- _ _ _ _ - _ - _ _ _ _ - _ - _ - - -,
l
,
s'
y lc.
39'
l load' requirements to maintain core integrity and the shorter recovery time for the swing Emergency Diesel Generator could be an important factor.
This is e
identified as insp+ctor follow-up item 50-321/366-89-08-21:
Emergency Diesel Generator IB PSW Supply.
(4)
Performance'Indicaters Performance indicators for 1986 to the present indicate improved reliability of Emergency Diesel Generators.
The indicators reviewed weie start failures and load failures. The cumulative failures for the Emergency Diesel Generators decreased from 1986, to 1989.
There were seven failures in ~1986, five in 1987, two in 1988, and none yet in 1989. Additionally, with one exception, failure causes were not repeated indicating effective corrective action for Emergency Diesel Generator failures. The exception was personnel error in the load adjustment port'On of the startup.
There were three occurrences.
Review of corrective action 3 for failures indicated a comprehen-sive approach to corrective actions which included evaluation of all Emergency Diesel Generators for common mode failure. A specific example was the prompt replacement of the cylinder plugs in all Emergency Diesel Generators following a blowout of a plug on Emergency Diesel Generator 20 on April 14, 1987. The
. reduction in overall Emergency Diesel Generator failures and effectiveness of corrective actions to prevent redundant failures indicates an increased reliability of Emergency Diesel Generator systems.
c.
Maintenance The scope of the inspection focused on maintenance activities for the Emergency Diesel Generators and supporting auxiliaries. Activities reviewed included observing work in progress and associated documen-tation; review of completed MW0s; component trending data; and post m&;ntenance testing.
Specific findings are addressed in the following paragraphs:
(1) Review of Maintenance in Progress Tae inspector observed the monthly and quarterly preventive maintenance performed on starting air compressor 2R43-C005A for Emergency Diesel Generator 2A.
The PMs were being performed in accordance with procedure 52PM-R43-005-05, Diesel Generator Starting Air Compressor Maintenance.
Sections 7.4, 7.5, and 7.6 of the procedures were performed for the monthly PM under MWD 2-89-1883.
Section 7.7 of the procedure was performed for the quarterly PM under MWO 2-89-1858.
The inspector verified that the activities performed in the PM procedure were in accordance with the vendor manual.
The inspector observed maintenance
-
_ _ _ _ _ _ _ _ _ - _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _
_ -_ _.
-
- ,
d
personnel.to verify that. the work performed was in accordance-with administrative controls. Maintenance personnel performed the required prerequisites prior to initiating the work.
While verifying system alignment via Clecrance Number 2-89-561, the maintenance personnel noticed that the control switch for the air compressor was incorrectly labeled as Unit 1 instead of l
Unit 2.
Maintenance personnel notified operations personnel and the correct clearance tag was placed onLthe control switch prior to maintenance personnel proceeding with the PM.
It was stated by maintenance personnel, verified by maintenance personnel, and verified by the inspector that the control switch was in the correct position and that it was correctly labeled. No other discrepancies were observed during performance of the PMs.
(2) Review of Completed MW0s Completed MW0s were selected based on the importance of the component to Emergency Diesel Generator reliability and operab-ility and to provide a cross sectional overview of all types of maintenance activities. MW0s reviewed ranged from pressure switch recalibration to replacement of a starting air compressor and motor.
The following MW0s were reviewed:
1-86-9543 1-86-9948 1-88-1309 2-88-1889 3-86-9525 1-88-8155 1-88-1288 2-87-1600 1-86-9603 1-87-7504 1-89-2338 2-87-5229 1-87-1409 1-87-7640 1-86-3991 2-88-2011 1-87-1445 1-88-3777 1-86-7919 2-88-2229 1-87-2355 1-88-5128 1-88-8393 2-88-2947 1-87-2526 1-88-6454 1-88-4402 2-88-3785 1-88-4728 1-89-0928 1-88-2356 2-87-5000 1-88-6455 1-87-4912 1-89-0965 2-88-2178 1-88-6218 1-87-5768 1-89-0963 2-87-5482 1-88-7182 1-87-7719 1-89-0393 2-88-2679 1-88-8034 1-87-1774 1-89-4654 2-87-1874 1-89-375 1-88-2314 1-88-3901 2-88-2492 1-89-2395 1-88-1302 1-88-3470 2-88-0445
,
L 2-88-3013 2-87-1643 2-88-4145 2-87-1795 l
2-87-0814 2-86-7395 2-86-6993 2-86-8409 l
2-88-3287 2-87-0349 L
The inspector discussed the following concerns with licensee personnel based on the review of completed MW0s:
a There were a number of problems with starting air compressor pressure switches drif ting out of tolerance which caused improper operation of the starting air compressors and resulted in the switches having to be a
_ _ _ _ _ __._ -. - - _ _ - - - -
.1
- _ _ _ -
- _ _ -
..
al recalibrates. The pressure switches were on a 24-month recalibration schedule for Unit I and a 60-month recalibration schedule for Unit 2.
Licensee personnel. stated that the'recalibration schedule for the pressure switches would be changed to.every 18-months.
It was further stated that Maintenance Engineering was developing a program to trend instrument calibration data.
The inspector questioned the cal-:bration schedule for the Emergency Diesel Generator panel instruments which were currently on a 60-month schedule. The inspection team raised a question while observing Emergency Diesel Generator-IC operation when the Emergency Diesel Generator frequency
_
meter in the main control room indicated 60 Hertz, but. the Emergency Diesel Generator frequency meter in the~ Emergency Diesel Generator room IC was oscillating between approximately 52-58 Heitz.
Review of the.
calibration records for the Emergency Diesel Generator.
frequency meter in question indicated that the meter was recalibrates on April 19, 1989.
This is identified as inspector follow-up item 50-321/366-89-08-22:
Accuracy of Local Frequency Meter.
b The inspector determined from the review of completed-corrective maintenance MWDs that the Emergency Diesel Generator support system which had experienced the most problems was the Emergency Diesel Generator starting air system.
The problems included compressor unloader failures, relief valve failures, sta'rting air compressor replacement, compressor motor replacement, pressure' switches drifting outside setpoint,' etc.
Most of the problems were identified by licensee personnel during PM and/or surveillance activities.
The inspector questioned the adequacy of-the licensee's root cause identification and corrective actions because some of the component problems were
_
repetitive. The licensee had various programs for trending component failures (e.g., Deficiency Card trands, Maintenance History trends, NPRDS component group reviews, etc.).
The licensee did not appear to have an effective program for reviewing the individual component failures together as a group for a particular system in order to determine their overall effect on the reliability and operability of the Emergency Diesel Generator starting air system. Although the various trends of starting air system components indicated that the failure rate for Hatch was below the industry average, when the components were grouped for the starting air system as a whole it ipeared
!
that the overall system reliability was questionable.
Licensee personnel acknowledged the inspector's concerr s and stated that the concern would be reviewed and appropriate actions taken as necessary. This is iL__
_ _ _ _ _ _ _ _
_.
.
. - - - -
__
__
_
___ ____ _ __ _ _ _
.
..
identified as inspector' follow-up item i
50-321/366-89-08-23:
Reliability c.f Emergency Diesel Generator Starting Air System.
c During the review of completed MW0s, the inspector raised a question over the adequacy of post maintenance testing stated in some of the MW0s..There were examples where the MW0s stated that' operations would perform the post maintenance test and verify operability of the specific component.
Howevei, there were no further details stated in the MWO of the testing performed or procedure used. The concerns over post maintenance testing was based on reviewing MW0s completed prior to August 1988. The inspector discussed this item with licensee personnel who stated that additional administrative controls addressing post maintenance testing were implemented in August 1988, because of previous findings by INPO and the licensee's site QA organization.
The inspector did not identify any concerns over post maintenance testing for MW0s reviewed that were completed after the administrative controls were implemented in August 1988.
d There appeared to be good communications and interface between the maintenance department and the plant engineering staff, even though.the administrative controls in this area were not very formal in some cases.
This did not appear to affect the adequacy of the work performed.
d.
Surveillance The inspection team conducted a review of surveillance and inservice testing as implemented on the Emergency Diesel Generator and asso-ciated auxiliary support systems.
The review included an evaluation of technical adequacy of associated procedures and results, and a review of the licensee's implementation of surveillance requirements and commitments contained in Technical Specifications and the Final Safety Analysis Report.
(1) Emergency Diesel Generator The team reviewed the technical adaquacy and related data
,
l packages for the following test procedures:
l 42SV-R43-007-25, Diesel Generator 2A-18 Month Surveillance Test Procedure - Part 1, Revision 1, dated March 17, 1988
- - -
_ _ _ - _ _ _ _ _
-.
. _ _ -. _ _ _.
-
.__
_ _ _ _
- _
_
-
--
- _ _ _ _ _ _.
_ _ _ _ _
-_
- _.
- _
__
_ _ _ - _ _ - _ _
. _ - _ - _ - _ _ _
_
_
- -.
._
.
..
- ...
L,.
l I
42SV-R43-008-2S, Diesel Generator.2A-18 Month Surveillance Test Procedure - Part 2, Revision 1, dated May 17, 1988 42SV-R43-009-2S, Diesel Generator 2A-18 Month Surveillance Test Procedure - Part 3, Revision 0, dated November 25, 1986 42SV-R43-010-2S,, Diesel Generator IB 18-Month Automatic Load Sequence Tin'er Operability Test, Revision 1,' dated February 20,-1988 42SV-R43-011-2S, Diesel Generator 1B-18 Month Surveillance Test Procedure - Part 1, Revision 0, dated March 17, 1988
.42SV-R43-012-2S, Diesel Generator 1B-18 Month Surveillance..
Test Procedure - Part 2, P.evision 1, dated March 17, 1988
$
42SV-R43-013-2S, Diesel Jenerator IB-18 Month Surveillance
. Test Procedure - Part 3, Revision 1, dated March 17, 1988 42SV-R43-014-2S, Diesel Generator 2C 18-Month Automatic Load Sequence Timer Operability Test, Revision 2, dated February 20, 1988 34SV-R43-007-25 Diesel Generators 2A and 2C Simultaneous Start, Revision 1, cated March 8, 1988 42SV-R43-021-15, Diesel Generator IA LOCA/ LOOP Testing, Revision 0 42SV-R42-003-05, Battery Inspection, Revision 0 42SV-R42-006-05, Battery Service Test, Revision 4 52SV-R42-002-25, Battery / Individual Cell Surveillance, Revision 3 The above procedures and test data satisfied applicable Emerg-ency Diesel Generator surveillance testing requirements and operability tests contained in Technical Specifications, and were performed at the proper frequencies. Where testing discre-pancies were identified, corrective action was initiated.
In addition, the procedures had received appropriate levels of management review and approval.
Prerequisites, precautions, and test equipment were sufficient to support the detailed sections of the test.
In addition, the inspectors witnessed locally and from the control room, the performance of surveillance test 34SV-R43-006-15, Diesel Generator IC Monthly Surveillance Test Procedure, conducted May 17, 19-89.
The operators were familiar with the I
c_-_____-_-____-__-_______
_ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _.
__
_________-_______________________________-___i
- - _ -
_ _ _. _
..
[-
V.
lC acceptance criteria. in' the test procedure, and were cognizant of operating limitations and test precautions. The inspectors noted a slightly greater than recommended crankcase vacuum reading. and found that Deficiency Card 1-89-2253 was appr:p-
.
riately. generated. However, during the test the frequency
,
meter reading on the local control panel was fluctuating
'
between 55 and 60 hertz., while the control room meter was reading a constant 60 hertz.
The licensee did not. identify.
.
this as a discrepancy, and no Deficiency Card was written. The-
~
local" frequency meter may have to be used during an emergency, possibly upon control room evacuation, and thus the inspectors believe the licensee's attention to this detail was lacking.
All procedural ~ acceptance criteria were satisfied.
The inspectors'also performed a comparison.of current Unit 2 Emergency Diesel Generator surveillance testing as required.by Unit 2 Technical Specifications to.the recommendations of-Regulatory Guide 1.108, Periodic Testing of Diesel Generator
. Units Used A-Onsite Electric Power Systems At Nuclear Power Plants', '
.is >n 1, dated August 1977.
The licensee is committed to RG 1.108'for demonstrating operability of the Unit 2 Emergency Diesel Generators per Unit 2 Technical Specification Bases Section B 3/4 8-1 (a similar commitment has not been made for Unit 1). A comparison of the two documents revealed the following differences:
(a) RG 1.108 Section C.2.a.4 states that testing should demonstrate proper Emergency Diesel Generator operation, including a test of the loss of the largest single load.
This test is performed to demonstrate the Emergency Diesel Generators capability to sustain a single load trip and still maintain voltage and frequency requirements.
Table -
8.3-11 of the Unit 2 Final Safety Analysis Report identi-fies the RHRSW pump,1220 HP, as the largest possible load on the emergency buses, occurring 10 minutes post-LOCA with a LOOP and loss of one bus.
The Unit 2 Technical Specifications require a single load reject test of greater than 798 KW, performed every 18 months.
The licensee's Technical Specification load, greater than 798 KW, is approximately equivalent to an RHR pump, 1080 HP, at 100 percent efficiency. The RHR pump is the largest single load which is auto-connected the Emergency Diesel Genera-tore.
However, using the appropriate conversion factor and a 9u percent pump efficiency for the RHRSW pump, the
-
Emergency Diesel Generator could carry a load of 1011 KW.
(b) RG 1.108 section C.2.a.4 states that testing should demonstrate proper Emergency Diesel Generator operation, including a complete loss of load test.
The purpose of this test is to demonstrate the capability of the Emergency
- - _ _ _ _ _ - _ -
__ -
_ - _ - -
- - _ _ _ - __- _ _ ____ - -_- ___ _ _ ___-___ _______- --______ _ __ _ __ - __-- -
_- _
-_
_ - - _ _ _ -
.
'
.
l
!
L I
Diesel Generators to sustain a complete load reject without tripping on the engine overspeed limit. Typically the complete loss.of load.1s-performed at the Emergency Dies'l e
Generators continuous rating, or at a load equivalent to that which the Emergency Diesel Generators may be required to carry during a LOCA and/or LOOP. The licensee's-complete load reject per Unit 2 Technical Specification is consistent with the Emergency Diesel Generator loading
.during the first 10 minutes following a LOCA with LOOP..
The licensee stated that their complete loss of load test-is believed to be consistent with the requirements of RG 1.108, and adequately demonstrated the proper function-ing'of the Emergency Diesel Generators as intended.
However, the licensee's complete loss of load test is less than the continuous rating of the Emergency Diesel Generators, and is less than the maximum Emergency Diesel Generator loading per the conditions stated on Unit 2 Final Safety Analysis Report Tables 8.3-11 through 8.3-16, occurring 10 minutes post-LOCA.
'(c) RG 1.108 Section C.2.a.3 requires the demonstration of full load carrying capability for greater than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, of which 22 hours2.546296e-4 days <br />0.00611 hours <br />3.637566e-5 weeks <br />8.371e-6 months <br /> are at the continuous rating of the Emergency Diesel Generator and 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> at the two hour rating of the Emergency Diesel Generators.
The two hour rating of the Emergency Diesel Generator discussed in the RG is that rating above the continuous rating, overload condition, which the Emergency Diesel Generator may sustain for 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> and not require any increase in maintenance activities.
The Unit 2 Technical Specification requirements for Emergency Diesel Generators 2A and 2C satisfy the 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> test, however, the IB Emergency Diesel Generator is tested at loads less than the continuous rating and the two hour rating. The licensee stated that the Emergency Diesel Generator 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> test is in accordance with vendor recommendations.
The inspectors discussed these differences with plant management, who agreed to perform a comparison of current Unit 2 Technical Specification surveillance requirements w:th those stated in RG 1.108. Where differences are identified, the licensee will provide a technical basis, and will evaluate the need to modify current testing based on a prudent technical evaluation.
-.
- _ _ _ _ _ _ _ - - _ - _ _ - _ _ _ _
_
.
- _ _ _ _ - _ - _ -
.
'
.
..
The inspectors also discussed with the licensee differences between emergency diesel generator surveillance testing as required by Technical Specifications.
Although the licensee is'
committed to testing Unit 2 diesels 2A, 2C, and IB in accordance with RG 1.108, a similar commitment was not made for Unit I diesei testing.
The. inspectors noted the following tests, performed on Unit 2 diesels, which are not performed on Unit I diesels:
single load reject testing, complete loss of load reject testing, and a continuous 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> diesel run test.
These tests are indicative of conditions which the emergency diesel generators would be subjected to during an emergency, and thus
'
are typically performed on a periodic. basis to demonstrate operational readiness.
Since the Unit I diesels perform a similar safety function as the Unit 2 diesels, the team considers an assessment of the adequacy of Unit 1 emergency
' diesel generator testing to be prudent prior to the issuance of-the revised Unit 1 Technical Specifications.
This is ideatified as inspector follow-up item 50-321/366-89-08-24:
Comparison of Emergency Diesel Generator Testing Requirements with RG 1.108, and comparison of Unit 1 and Unit 2 Technical Specifications Surveillance Requirements.
(2) Emergency Diesel Generator Support Systems
>The team reviewed the surveillance _ testing ic
.m Emergency Diesel Generator support systems, and the followin, comments and concerns are provided:
(a) Emergency Diesel Generator Building Ventilation, Heating, and Cooling System Testing The team reviewed the licensee's implementation of periodic testing of the Emergency Diesel Generator building ventila-tion system. The ventilation system is designed to provide temperature and air movement control to support Emergency Diesel Generator operation, such that Emergency Diesel Generator building design requirements are not exceeded.
Commitments for periodic testing are contained in Section 9.4.5.4 of the Unit 2 Final Safety Analysis Report.
The licensee currently does not perform specific periodic tests for the Emergency Diesel Generator building HVAC system.
Functional observation is conducted as part of the periodic Emergency Diesel Generator testing. In addition, cali' ration of heater and fan thermostats, and firestats a
for the Emergency Diesel Generator rooms, battery rooms, switchgear rooms, and day tank rooms are performed or an as
- _ _ _ - _ _ _ _ _ - - -
_
-_
_..
_ _ _ _ - - _ _ _ _ _ _ - _ - _ _ _ _ _ _ _ - _ _ _ _ - _ _ - - _ _ _
_ - _ -
k
[
.:-
L
,
l l
needed basis. The licensee developed Inspection and Test Procedures 57IT-X41-001-IS, and'57IT-X41-001-25,- Diesel lL Generator Building Heating and Ventilation Test, in late l
1988.to address the need for periodic testing _of the HVAC system. These two tests have not been performed to date, and their current status is Validation, i.e. to validate the original performance of the procedure as written. Once validated, the tests will be performed on a periodic 36 month basis, and will provide calibration for all thermo-stats as well as functional testing of the system.to i
satisfy the commitments of.Section.9.4.5.4 of the Unit 2
'
.F.inal Safety Analysis Report.
However,.the inspectors consider the licensee's current testing of the Emergency Diesel Generator building HVAC system to be inadequate as implemented at the time of the inspection.
Periodic calibration of HVAC equipment as stated in the Unit 2 Final Safety Analysis Report is needed to insure operational readiness and demonstrate system capabilities to control temperature and air movement in the Emergency Diesel Generator building such that design requirements are not exceeded. Testing of the HVAC system during Emergency Diesel Generator periodic surveillance only insures that the system will operate for the conditions which exist during the Emergency Diesel Generator test, and does not demonstrate automatic activation of thermostats, heaters, fans, etc., at their proper setpoint.
This area was previously discussed in NRC inspection report 50-321,366/88-17, dated July 21, 1988.
Violation 88-17-01 was issued involving failure to classify procedures as safety-related, procedural inadequacy, and failure to align exhaust fan switches per procedure.
In addition, Deviation 88-17-02 was issued against the Final Safety Analysis Report involving thermostat calibration. These issues are still open, thus no additional items will be opened.
However, until testing is performed to fully satisfy the commitments covered in Unit 2 Final Safety Analysis Report section 9.4.5.4, this area continues to be a weakness in the licensee's Unit 1 and Unit 2 Emergency Diesel Generator HVAC surveillance program.
(b)
Inservice Testing of Emergency Diesel Generator Pumps and Valves The inspectors reviewed the implementation of the licensee's IST program with regard to pumps and valves
'
associated with the Emergency Diesel Generator and related subsystems.
Section XI, Subsections IWV and IWP of the ASME Code, provide the requirements for IST of pumps and valves which are needed to mitigate the consequences of an accident.
The licensee includes the following Emergency
-
-_
_ _
i
_
.
48 Diesel Generator related components in the IST prograa.,
submitted February 24, 1988: fuel oil transfer pumps, fuel oil transfer check valves, air start system check valves, air start solenoid valves, and air receiver relief valves.
IST of the air receiver relief valves is performed in accordance with Section X1 per surveillance procedure 42SV-SUV-004-05.
The licensee has submitted relief requests for the remaining Emergency Diesel Generator.
components, however, review and approval by the Office of Nuclear Reactor Regulation nas not been completed.
Generic Letter 89-04, Guidance on Developing Acceptable Inservice Testing Programs, dated April 3, 1989, indicated that plant Hatch should receive a Safety Evaluation Report soon.
Plant management stated that full compliance with the submitted IST Program regarding Emergency Diesel Generator components would be completed nine months after receipt of the SER.
In.the interim, the licensee stated they are revising surveillance procedures, as part of the Procedures Upgrade Program, to incorporate where practical the IST relief requests.
Although not specifically delineated by procedure, the fuel oil transfer check valves and air start solenoid valves are observed as part of the Emergency Diesel Generator air start test, performed monthly.
However, IST of the fuel oil transfer pumps is not performed in accordance with the submitted IST Program relief request.
The relief request describes an alternate testing method involving a measurement of the elapsed time between two elevations in the Emergency Diesel Generator day tank.
In addition, testing of the air receiver inlet check valves is not currently performed on a component basis and are not specifically leak tested. The licensee stated that monthly Emergency Diesel Generator testing, performed in accordance with Technical Specifications, provided adequate assurance of check valve exercising and fuel oil transfer pump operation.
The team is concerned with the licensee's lack of adequate IST with regard to the fuel oil transfer pumps and the air receiver inlet check valves. These Emergency Diesel Generator system components are included in the IST program l
due to their importance to plant safety, and thus IST
'
should be performed in accordance with the submitted ISI/IST Program.
The licensee's position to fully
implemert the ISI/IST Program upon receipt of the SER is l
I unacceptable.
The current method to accurately quantify the extent of component degradation is unacceptable, and is a weakness in the licensee's IST program.
This is identified as inspector follow-up item 50-321/366-89-08-25:
Emergency Diesel Generator Component Inservice Testing.
!
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _ _ _ _ _ _ _. _ _ _ _ _ _. _ _ _ _ _ _ _.. _ _. _ _. _ _. _
,_,_-
-.
_ _ _
_
-
l0-
(c) Plant Service Water Val.ve Lineup Licensee procedure 345V-SUV-012-2S, Plant Service Water, Residual Heat Removal, and Standby Service Water Subsystem Valve Position Verification, confirms the proper alignment
,
l of plant service water valves as required by Unit 2 l
Technical Specifications Section 4.7.1.2.b.
The procedure,
. performed at least once per 31 days, verifies that plant service water discharge valves for Emergency Diesel Generator 2A (F339A), Emergency Diesel Generator 2C (F339B), and Emergency Diesel Generator IB (F340) are closed prior to Emergency Diesel Generator starts. These valves are air operated, and open upon an Emergency Diesel Generator start signal to provide cooling water to the Emergency Diesel Generators, and thus the correct valve position is vital to Emergency Diesel Generator operation.
Conversely, Unit 1 Emergency Diesel Generator 1A and IC plant service water discharge valves, F553A and F553C respectively,'are manual valves.
These valves are normally open before a Emergency Diese1' Generator start, and are required to be open after an Emergency Diesel Generator start signal to allow Emergency. Diesel Generator cooling.
Emergency Diesel Generator operating procedure-34S0-R43-001-IS, Diesel Generator Standby AC System Operating Instructions, together with Plant Service Water Operating Procedure 34S0-P41-001-15 provide valve alignment verification for F553A and F553C.
However, these procedures are system operating procedures, and thus do not provide a periodic verification of valve position.
Valves F553A and F553C are unique in that they are manual valves which are not locked open, and as such do not receive any actuation signals to open. -A misalignment of either valve to the closed position and subsequent Emergency Diesel Generator start signal would result in loss of that Emergency Diesel Generator after approximately three minutes due to lack of cooling water.
The inspectors were concerned that valves F553A and F553C, which are required to be open to preclude Emergency Diesel Generator 1A and IC overheating and subsequent failure, are not part of a periodic valve alignment surveillance procedure.
The inspectors would consider it prudent to evaluate the necessity of monthly valve position verifica-tion.
In addition, the inspectors also questioned the licensee on the need for these valves to be locked open.
The licensee stated that these valves are not checked monthly because Unit 1 Technical' Specifications do not require it, and normal plant service water operating procedures adequately assure proper alignment for valves
-
-
. _ _ _.
_-__-____-
_
__ A
_ - _ _ _ _.
..
F553A and F553C.
However, the licensee agreed to evaluate whether these valves should be on a monthly valve position verification procedure, and the necessity for these valves to be locked open. -This.is identified as inspector follow-up item 50-321/366-89-08-26: Plant Service Water Valve Lineup.
i (d)
Instrument Calibration The inspectors reviewed the licensee's calibration proce-dures for the isolation of PSW valves P41-310A, B, C, D (Unit 1) and P41-316A, B, C, D (Unit 2).
These valves close to isolate the P5W supply to the turbine building such that sufficient cooling water is routed to the Emergency Diesel Generators. The inspectors reviewed P& ids-and elementary diagrams with licensee personnel, who were knowledgeable of the importance of these valves upon plant safety and their effects on the Emergency Diesel Generators.
The following signals were verified to initiate isolation of the associated valves: emergency bus undervoltage, downs'ceam high flow signal, LOCA, and condenser bay high wuter level.
The inspectors also reviewed calibration procedures 57CP-CAL-013-15 and 57CP-CAL-013-2S, Barton Differential Pressure Indicating Switch Calibration, which are used for calibration of the pressure switches to detect high flow downstream of the isolation valves.
The inspectors concluded that the licensee has adequately addressed this area.
e.
Quality Assurance, Quality Control, and Procurement (1) Quality Assurance Quality organization activities were examined to determine the organizations ability to identify technical problems, and to specifically review QA/QC activities in the systems encompassed by the SSFI.
Review of the following 1987, 1988, and 1989 audits / surveil-lances conducted to date on the Emergency Diesel Generator and supporting systems indicates that the Hatch QA organization has provided better that average coverage of those systems during the interim period.
Audit Surveillance No.
Dates Title 59-MNT-1 1/9-2/6/89 QA Audit of Maintenance
___. _. _ _ _ _
_ _ _ - _ _ - _ _ _ _ _ _ _
is
+
Audit Surveillance No.
Dates Title 88-MNT-2-10/17-11/17/88 QA Audit of Main-ten.nce 88-MNT-1 6/15-17, 20-24, QA Audit of Main-7/5/88 tenance 89-0RA-15 3/9/89 IEB 88-03, Emer-gency Diesel Generator Raw Cooling Water Check Valve Failures Surveillance
'88-ORA-45 8/10/88 IB Emergency Diesel Generator System Surveillance 87-ORM-40 4/9/87 QA Surveillance of Diesel Building and Intake Structure 87-ORM-23 2/26/87 QA Activity Surveillance of 125 Volt DC System Station Batteries The following specific SSFI technical findings and programmatic problems were identified in the above listed audits / surveil-lances by the QA organization:
.(a) AFR Number 9. of Audit 89-MNT-1 identified after performing walkdowns of the Unit 1 and 2 Emergency Diesel Generator switchgear panels and the main control room panels, that there are no controls in place, outside of Appendix R related fuses, to ensure that fuses with the correct rat,ngs and characteristics are installed in the plant.
The QA audit discrepancies identified were corrected on May 3, 1989, and the programmatic controls appear to be on schedule.
(b) Audit 88-MNT-2 identified three SSFI related audit findings. AFR Number 96 identified that contrary to the Unit 2 Final Safety Analysis Report and NRC IEN 87-04 corrective action response, the Emergency Diesel Generator i
fuel oil in the day tanks is not sampled, the tanks are not properly equipped for sampling, and only portions of the
- - - _ - ___-__ _ _ - ___-___ _
- - - _ _ _ _ _ _ -
p-
'
,
l[
.4
'52-commitment to the sampling program are in place. The subject AFR was distributed to the audited organization on November 29, 1988.- The initial corrective action response was required by December. 29, 1988, however, was not received by QA until January 9,1989, af ter a Manage-l<
ment Attention letter dated January 5, 1989, was written.
L A revised schedule, Revision 1, was submitted February 6,
'
1989, and an update schedule submitted on March 20, 1989.
This Revision established a milestone schedule for an effective oil sampling procedure, a validated 64 Chemistry Series Procedure, to be in the works by' August 8, 1989.
Discussions with the chemistry laboratory supervisor.
'
disclosed the 64 Chemistry Series Procedures are being developed to replace the existing 62 Series Procedures; however, the PUP requires all interrelated procedures and procedure groups to be reviewed and approved at the same-time. To cover the interim period until the 64 Chemistry Series oil sampling procedure is approved the current 62 Chemistry Series procedure has been revised incorporat-ing the same oil sampling techniques that will be incorpo-rated in the 64 Chemistry Series Procedure. The subject AFR was reviewed by NSC on January 19, 1989, and determined to be not reportable under 10 CFR 21. The PRB, Minute Meeting Number 88-164, reviewed Audit 88-MNT-2 on December 1, 1988 and concluded that none of the items identified consti-tuted an unreviewed safety question nor were there any other unaddressed concerns involving the AFRs identified by this audit.
. (c) AFR Number 98 identified that the current Emergency Diesel Generator surveillance procedure that is used to meet the Technical Specification surveillance requirement does not include all the required manufacture's recommendations.
Additionally, document control was not the central receiv-ing point for information received from the Emergency Diesel Generator vendor.
Corrective action is on schedule.
Applicable PM and SUV procedures were reviewed against vendor manuals and letters and discrepancies identified that need correction, deletion or incorporation in the Emergency Diesel Generator PM/SUV procedures. These procedures are scheduled to be revised June 15, 1989.
Respective site and Colt Industries personnel were notified that the Hatch Document Control facility is to be the central recovery point for all vendor information.
(d) AFR Number 100 identified that Hatch Emergency Diesel
Generator run data reflected temperatures up to 940 Fahrenheit in the area of the exhaust piping which was rated at 150 pounds at 500 Fahrenheit.
The vendor representative who eventually addressed this finding concluded that the existing exhaust pipe and flanges were
- _ _ _ _ _ _ _ - - _ _ _ - _ _ - _ _.
_ _ _ _ -_ _ - _
k
- . :
.>
.53 adequate to withstand the existing pressure and temperature conditions.
He determined that the auditor incorrectly l
interpreted the temperatures taken at the individual l
cylinders at the exhaust port to be exhaust piping tempera-tures.. This was a valid finding until the vendor's-evaluation concluded that no problem existed.
In any-event, this finding clearly demonstrates that the QA
.
organization will. tenaciously pursue questionable technical l
issues until they are resolved to their satisfaction with the assurance that no safety concerns exist in the plant.
(e) Audit 88-MNT-1 identified a procedural safety concern L
which was.immediately corrected involving maintenance personnel performing battery surveillance without wearing-required safety equipment, such as goggles, aprons, and gloves.
(f) Surveillance 87-0RM-23 identified minor corrosion on two battery cells and the floor under the battery racks needed cleaning.
Corrective action was implemented concurrent to the findings.
(g) Surveillance 87-0RM-40 identified:
(1) loose COND-0-LET cover screws on the IC Emergency Diesel Generator starting air compressor 1R43-C010C; (2) diesel 1R43-S001B had a rusty pipe.and flange, and cracked insulation on the pipe from the scavenger heat exchanger to the east wall.
Item (1) was corrected and a MWO initiated to handle item (2)
deficiencies.
(h) Surveillance 88-ORS-45 identified: (1) three valves shown on an air start system P&ID that could not be located in the field; (2) two air start system valves were improperly labelled; (3) two valves in the clean and two valves in the dirty fuel drain tank systems were not labelled; (4)
valve 1R43-F300SB had a broken handle; (5) no connection was indicated on the P&ID H11638 Revision 2 from 1R43-B003B to valve IR43-F3004B; (6) pumps IR43-C004B, C007B, and C008B were not labelled; (7) lack of proper bolt / nut thread engagement for the support at valve IR43-F094B; (8) the back panel plate for Panel 1R43-P003B was not secure; (9) housekeeping was needed under the IB Emergency Diesel Generator air receivers and the floor trough which transfers fuel oil from the day tank to the IB Emergency I
Diesel Generator; (10) two loose piping clamps and some components were found not labelled in the 40,000 gallon storage tank R43-A002C pit.
Review of Georgia Power Company interoffice correspondence to QA dated September 21, 1988, and September 26, 1988, and referenced documenta-tion, revealed the above items were satisfactorily resolved, secured, repaired, replaced, or cleaned as necessary to correct these deficiencies.
--_-_ _ -
-.
___
_ _ -
...
-
..
.
,
,
(1) Surveillance 89-0RA-1S:
Georgia ~ Power Company's initial response to IEB 83-03, identified three check valves in the Emergency. Diesel Generator plant service cooling-water lines and committed to a visual inspection of each check valve during the next Unit I or 2 scheduled outage.- This
,
surveillance confirmed the above corrective action was complete, examined the two revised procedures that assure periodic visual inspection of the three valves in a regular IST program.
Additionally, NRC inspector follow-up item IER 87-01-02 regarding check valves, and INPO SOER 86-03, " Check Valve Failures or Degradation" concerns were also verified-to be incorporated into these two IST procedures.
QA has been effective in examining prior licensee commit-ments made to the NRC and in verifying that they are continuously being implemented.
The above SSFI system audits / surveillance and checklists examined are primarily performance based, were accomplished
- .
through observation of work in progress, evaluation of l
plant conditions through physical walkdowns and interviews with personnel at many levels.
Considerable auditing time was performed during the back-shift hours.
The SSFI QA audits / surveillance and checklists were well organized of adequate technical scope and depth resulting in significant problems being identified in both the technical and procedural areas.
l In general, corrective actions to the SSFI audit />urve-l 111ance deficiencies identified have been prompt, appropri-l-
ate for the fix, well documented, and sufficient to prevent recurrence When milestones were passed without a L
required written response being received from the audited organization, QA has not hesitated to issue Management Attention letters which in turn has provided prompt responses to the subject deficiencies.
The inspector examined the overall OA audit program effectiveness by performing a thorough review of years 1987, 1988, and the first quarter of 1989 completed audit schedules.
In addition to performing the required Technical Specificati n and Final Safety Analysis Report, Chapter 17 audits, tP inspector found that many additional special audits were performed on evolving plant conditions, generic industry identified problems, items of management concern and on NRC inquires.
Some special audits conducted were; Tubline Corporation Pipe Fittings Special Audit of Operational Up-Grade Program, Valve Line-Up Verification, Root Cause, and Operating Experience Review.
The QA organization is able to accomplish this because open time
_
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _ _ __
_
_
_ _
-.
-.______a
_.
_- - -
_
.__-
- __ _-_
_ - _.
._ __ _-_-___- ____ _ _ -
l '.
>
.
l's
.;
is scheduled throughout their annual audit schedules to Conduct these special audits, they maintain an adequate QA i.
!
staffing level which is assisted by corporate auditors as necessary, and prepare well conceived realistic audit schedules, checklists, and scope of work.
The inspector examined a representative sample of 10 audits with pertinent checklists conducted between 1987 to 1989, and determined they appeared to be of sufficient scope and detail to adequately address the audited elements and l
areas. Many of these audits examined contained FAR items that were evaluated.
FAR items are suggested future audit items that QA accrues over time by placing them in a log and eventually adding them to relevant audit scopes for evaluation.
FAR items include, but are not limited to, checks for licen>ee continuance of commitments, problems identified by the NRC, INPO, or other licensee's that may affect HNP.
The QA organization augments the audit program with an excellent Surveillance Program, not required by the Hatch QA Program, which greatly enhances the QA program by conducting more performance based inspection activities.
QA has continued to maintain a well diversified audit staff of varying expertise, qualification and experience.
Hatch QVFI identified a potential weakness, Item Number 87-31-02, in that the single QA auditor with plant operators experience at that time was leaving the site.
The QA department more than rectified this loss by obtain-ing a licensed SRO who is OSOS qualified at Hatch and has 18 years nuclear utility experience.
Discussion with the site QA Manager revealed he is working toward the goal of providing audits that are 80 percent activity oriented, 20 percent documentation type, with 20 percent of the audits occurring during back-shift coverage.
(2) Procurement and QC Activities The inspector selected P0 D-15535 dated October 7, 1986, for the procurement of 48 Emergency Diesel Generator fuel oil injection nozzles of which 24 were installed in Emergency Diesel Generator IB, MPL/ Tag Number 1R43S001B, to verify that the procurement process and all required QC activities were adequately covered. The subject nozzles were procured commercial grade, PL-4B, from Colt Industries, Beloit, W1.
Procurement levels are defined in the NPPM.
This manual is i
unique in that it describes the entire procurement process and l
Georgia Power Company quality requirements in one concise l'
manual.
The numbers 1, 2, and 3 indicate the suppliers QA program complies with applicable requirements of 10 CFR 50, Appendix B and 10 CFR 21.
l
n-p
[}
-
L.;
--
Procurement Level 4 applies to items that are " Commercial Grade" and will.be used as basic components.
This suffix B indicates l
this item is an identical replacement item of the same make, model. number, manufacture etc., as the equipment, devise or
'
L component to be replaced.
Items procured commercial grade l:
should be dedicated prior to use in a safety related application
"
and their dedication based or one or more of the acceptance processes listed below:
(a) Special tests and/or inspections to verify critical characteristic of the item and or verification by post-installation testing (b) Audit of_the commercial grade supplier with supplier compliance documented by a C0C (c) Source verification surveillance activity verifying that the critical characteristics of the CGI are satisfactorily controlled by witnessing quality activities of the suppliers.
Colt industries is an approved PL-3 vendor based on a Bechtel led survey of their facility conducted on May 5-6, 1987. At the inspector's request Colt's latest revision of their QA Manual and a. copy of the above survey were sent from SON 0PCOs corporate office to the site for examination.
Review of these documents and telephone discussions with Colt's QA Manager verified that the PL-3 Quality Program was being applied to off-the-shelf, PL-4, items also.
Review of POD-15535 reveals it specifies the following:
The items procurement level and storage level.
The item was originally supplied for Diesel Assembly Model 38TD8-1/8 Ref. F/M S/0 Number 35-205781/971.
Materials shall be provided in accordance with the requirements of the suppliers product quality program approved for procurement-by Georgia Power Company.
Any recent significant changes to controls established for product quality or changed manufacturing location were to be reported to Georgia Power Company.
The supplier at each tier of procurement was required to provide access to its facilities for QA inspection and/or audit of supplier's QA program by Georgia Power Company.
o_
__
_
i
- .
.
q,
I 1.
E
The original P0 design specification number and the
. vendor was instructed that if the design.or fabrica-tion / materials have changed since the original part or component, the supplier shall notify the purchaser before proceeding with the order.
o l
h
Tbat a COC accompany the' nozzles certifying that the material / component meets the requirements of F/ti S/0 35-205781/971 and contain the following:
l~
l a
Identification of purchased material by Georgia Power Company PO number and item number.
b State the material / design specifications met by the items.
c Identify any procurement requirements not met.
d The signature of the person responsible for QA.
.
The subject nozzles were QC receipt inspected by MIRs 86-2915,87-880, 87-1830, and 87-755 and examined for identification and markings, protective covers and seals, coatings and preservatives, physical damage, cleanliness, and workmanship.~ The documentation review performed by NPRG personnel was checked acceptable; however the design specifications mentioned in item (2) above were not specified on the C0C as directed by the P0.
Discussions with NPRG personnel revealed the F/M Shop Order Number specified is the original detailed design specification to which the original part/ component was manufactured and was of primary importance. When Shop Order Nos are specified, no other specifications are usually required to be on C0Cs and this design requirement was superfluous and unnecessary in their opinion. The inspector examined design specification SS2123-012 mentioned in the P0 and found it to be a high tier Emergency Diesel Generator specification that did not adequately identify the design characteristics and parameters of the fuel oil nozzles procured.
Obviously, this documentation oversight had no safety significance nor did it make the nozzles, question-able. However, the inspector informed the NPRG supervisor and responsible document reviewer that whatever was specified for a C0C on the P0 should have been documented on the vendor's C0C unless a change order or deficiency card was written to formally document and resolve the
'
documentation omission.
-_ -__ _ ___ _ _
-
- - - _ _ _
_
-. - _ _. _.
__
--
..
x i
..
l MWO-1-88-0344 was initiated January 25,'1988, to begin the replacement of the Emergency Diesel Generator 1B fuel oil injection nozzles with new nozzles.
Post installation testing of these fuel oil' injection nozzles was conducted
.under MWO 18705104 by performing Surveillance Procedure 52SV-R43-001-05, " Diesel, Alternator and Accessories Inspec-tion." Section 7.2.1 of the procedure specifically addresses-fuel oil injector nozzle testing.
Review of documentation associated with materials and work necessary.for the nozzle replacements and the Emergency Diesel Generator surveillance testing revealed these materials received adequate QC receipt inspection, QC material issuance approval, proper QC review for inspection hold points, adequate QC inspection during the work involved and post installation testing, and formal system cleanliness acceptance.
In summary, it ap' pears this SSFI CGI procurement was properly dedicated in that the vendor's QA program was-audited and the vendor approved as a CG supplier, the purchased nozzles were backed by the vendor's C0C, receipt inspection verified some critical nozzle characteristics and licensee post installation testing of the nozzles was.-
performed.
Likewise, the work involved had adequate effective QC coverage to ensure safe operation.
The inspector randomly sampled and examined the qualifica-tions and training records of six QC inspectors and found these inspectors appropriately certified in accordance with ANSI N45.2.6 requirements.
Discussions with QC supervision, QC personnel and examination of QC inspection records and findings revealed that Hatch's QC personnel are not restricted to " tunnel vision". Most inspectors are cross trained in multiple disciplines and have the ability to identify problems, and do so, outside of their specialty area.
The QC department has been assisting the PUP by establishing built-in QC hold points for craft repair and replacement procedures thereby eliminating the need for separate QC inspection plans lessening paperwork and the chance for missed hold point violations.
Likewise, pro-cedural upgrading is evident in the training and certifica-tion program for inspectors. Although Hatch's program in this area is satisfactory, efforts are underway to make both Hatch and Vogtles' QC training and certification
requirements compe.tible.
Procedure 45QC-PQL-001-OS,
.
" Qualification and Personnel", Revision 4, Draft, represents a movement in this direction and an example of the effort being expended to continually improve the QC program.
-
_ _ _ _ _ - _ _ _ - _ - _ _ _ _ _ _ _ _ _ - _
- _ _ _ _ _ - _
f e.
.,.
L
The' QC ' department has an average experience level of '10 years and the inspectors are being sent to INPO accredited craft training programs, for repair of. pumps, valves, motors etc.,'when space is available.
Although QC is primarily a compliance oriented function, occasionally technical problems have been identified by this department.
Several examples similar to DC 1-88-514a have surfaced where QC has identified discrepancies between design approved drawings, design guidelines and tolerances.
l:
These problems are returned to. engineering for resolution.
L Recently, a QC inspector performing inspection of MWO 1-88-5546 questioned DCR 88-131 which permitted the welding of A36 class, carbon steel material into a Class I boundary, reactor vent head piping system which is not. acceptable per ASME Section III. A resulting DC 1-88-5320 was written, and FCR 88-131-005 initiated to restore acceptable material to the Class I boundary.
On another occasion, QC refused to accept design's valve L
gasket thickness recommendation which did not consider
'
spring tension considerations for proper seating and closure of the repaired valve.
Design was asked to reanalyze the Viton gasket and did so providing a specific design material thickness appropriate to provide the desired tension.
. 5.
Exit Interview The inspection scope and findings were summarized on June 19, 1989, with those persons indicated in Appendix A.
The inspectors described the areas inspected and discussed in detail the inspection findings previously listed. The licensee did not identify as proprietary any of the material provided to or reviewed by the inspectors during this inspection.
Dissenting comments were not received from the licensee.
One violation was identifed concerning corrective actions regarding diesel fuel oil issues. Many of the issues were licensee identified by QA audits, but corrective actions had not been completed in a timely manner.
(paragraph 4.A.3.a).
Also 25 IFis were identifed concerning many different aspects of the inspection.
Each item was discussed and the licensee has either completed or initiated action to reslove the item.
Concern was expressed regarding the current inservice testing program for emergency diesel generator pumps and valves, and dymanic testing for load changes on the emergency diesel generator, (IFIs 50-321/366-89-08-23, 24, and 25).
The current program does not adequately test the fuel oil transfer pumps, the starting air check valves, or the load rejection capability for unit 1 emergency diesel generators. The licensee committed to review these issues and initiate action to adequately surveil performance.
_ - _-__ _ __ -
_ _ - - _ _ - - _ _ _ _ _ _ _.
.*
i g.
l
'.
APPENDIX A l
. Licensee Employees B. Arnold, Chemistry Laboratory Supervisor H. Baker, QC Receipt Inspector
"G. Barker, Superintendent of I&C l.
- J. Beckham, Vice-President Plant Hatch Project
- S. Bethay, Project Engineer
- J. Branum, Senior Engineer
- K. Breitenbach, BOP Engineering g
G. Brinson, QC Superintendent l
1.
- C. Coggin, Training Manager l
- P. Fornel, Maintenance Manager
- 0. Fraser, QA Site Manager Quality Services
- B. Garner, Engineering Project Manager
- G. Goode, Manager Engineering Support
- L. Gucwa, Manager Neclear engineering and Licensing
- J. Hammonds, N&SC Supervisor
- W. Harvey, Acting General. Support Manager
- J. Heidt, Licensing Manager M. Horinka, QA Manager, Colt Industries
- J. Lewis, Operations Manager G. Lito Nuclear Specialist, Quality Services T. Metzler, NSC Supervisor
- H. Mirzakhani, System Engineer T. Mitchell, Senior. Engineer I, Safety, Audit and Engineer Review
- C. Moore, Assistant General Manager - Plant Services
- H. Nix, General Manager - Plant Hatch
.J. Payne, Senior. Plant Engineer Nuclear Safety & Compliance
- P. Roberts, Special Projects S. Piedia, Technical Support Supervisor, NPRG-GO
- D. Self, OPC
- S. Tipps, Nuclear Safety and Compliance Manager
- E. Wahab, Engineering Support K. Williams., Nuclear Procurement Review Group Supervisor J. Yaun, QC Supervisor, Mechanical and I&C Other licensee employees contacted included engineers, operators, technicians, maintenance personnel, and office personnel.
NRC Resident Inspectors:
- J. Menning, Senior Resident Inspector
- R. Musser, Resident Inspector Other NRC Personnel:
"E.
Merschoff, Deputy Director, Division of Reactor Safety
- D. Matthews, Project Manager, NRR
- Attended Exit Interview L___-_____-___
-
-
_.
$'
i
';g
-
c?
,
APPENDIX _B t
ACRONYMS'AND'INITIALISMS-
Alternating' Current AC
-
AE
-
Architect Engineer Audit Finding Report AFR
-
ANSI -
American National Standards Institute ASME -
American' Society of Mechanical Engineers Brake Horse Power BHP
-
CFR Code of Federal Regulation
-
Commercial Grade Item CGI
-
Certificate of Conformance COC
-
Direct Carrent DC
-
DPIS -
Differential Pressure Isolation Switch DR'
-
Deviation Report Engineered Safety Features ESF
~
ESW Emergency Service Water
-
EWR -
Engineering Work Request FAR -
Future Audit Required FCR
'
Fuel Change Request
-
F/M -
Fairbanks Morse Engine Division GPM -
Gallons Per Minute HNP -
Hatch Nuclear Plant Horse Power HP
-
HVAC -
Heating, Ventilation and Air Conditioning I&C -
Instrument and Control Inspection and Enforcement (NRC) Bulletin IEB
-
IEEE -
Institute of Electrical and Electronics Engineers Inspection and Enforcement (NRC) Information Notice IEB
-
IER -
Inspection and Enforcement (NRC) Follow-Up Item Inspector Follow-up Item IFI
-
INPO.-
Institute of Nuclear Power Operations Inservice Inspection ISI
-
Inservice Test (Program)
-
Limited Condition of Operation LCO
-
KVA -
Kilovolt-Ampere Leeds and Northrup L&N
-
LER Licensee Event Report
-
LOCA -
Loss.of Coolant Accident LOOP -
Loss' of Offsite Power MCB -
Main Control Board Motor Control Center MOV - Motor Operated Valve MCC
-
MSL Mean Sea Level
-
MIR -
Material Inspection Request MWO -
Maintenance Work Order NPPM -
Nuclear Procurement Policy Manual NPRG -
Nuclear Procurement Review Group NPSH -
Net Positive Suction Head NRC --
Nuclear Regulatory Commission NRR -
Nuclear Reactor Regulation NSC -
Nuclear Safety and Compliance OP
-
Operating Procedure OSOS -
On Shift Operation Supervisor
- _ - - _
_
y.
em
l
- +
,0-
{
Ippendix'.B'
.
pf
'
-
Pounds Per Square Inch PSI
-
PSIG -:
Pounds Per Square Inch Gauge.
PL Procurement Level
-
Preventive ~ Maintenance PM1
-
PO'
'
Procedure Order-
-
PRB -
Plant Review Board PSW -
Plant Service Water
~
-
Periodic Test PUP Procedure Upgrade Program
'
Quality A:surance QA
-
Quality Control QC
-
- QVFI -
Quality Verification Function Inspection RHR -
' Residual Heat Removal Regulatory Gaides RG'
-.
Reactor Operator-RO
-
Revolutions Per Minute
!
-
Reactor' Protection System RPS
-
SER Safety Evaluation Report
.
SOER -
Significant Operations Event Report SONOPC0 - Southern Nuclear Operating Company
'SOR
.Significant Occurrence Report
-
'SRO Senior Reactor Operator
-
-
Safety System Function Inspection SW.
-
Service Water SUV -
Surveillance TDH
'
Total Developed Head
-
l
_ - -. _ - - - _ _ _