IR 05000346/1978019
| ML19329A960 | |
| Person / Time | |
|---|---|
| Site: | Davis Besse  |
| Issue date: | 09/13/1978 |
| From: | Knop R, Streeter J, Tambling T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III) |
| To: | |
| Shared Package | |
| ML19329A940 | List: |
| References | |
| 50-346-78-19-01, 50-346-78-19-1, NUDOCS 8001270216 | |
| Download: ML19329A960 (19) | |
Text
,
U.S. NUCLEAR REGULATORY COMMISSION OFFICE OF INSPECTION AND ENFORCEMENT
REGION III
,
Report No. 50-346/78-19 Docket No. 50-346 License No. NPF-3 Licensee: Toledo Edison Company Edison Plaza 300 Madison Avenue Toledo, OH 43652 Facility Name: Davis-Besse Nuclear Power Station, Unit 1 Inspection At: Davis-Besse Site, Oak Harbor, OH Inspection Conducted: June 6-8, 13-15, 20-23, and July 17-19, 1978 7.d YcMdb 1klI
Inspectors:
T. N. Tambling
!
Hb1bT 6/7-8/78)
7z
//
M /3/M
.
J. Foster (6/13-15/78)
/
/
/
Menni g (6/13-15/78)
%i f
Other Accompanying Personnel:
J. Pulsipher (6/6-o B)
n
/
( ( C~}Vss-vp
/
R.)C. Knop,Chibf
2,77,f Approved By:
Reactor Projects Section 1
Inspection Summary I
Inspection on June 6-8, 13-15, 20-23, and July 17-19, 1978 l
(Report No. 50-346/78-19)
[
Areas Inspected: Special unannounced inspection and investigation associated with the failure of diesel generator sequencer circuit during the 18-month surveillance testing of the Safety Feature Actuation System (SFAS); review of the associated design installation and preoperational testing errors;
['
review of licensee's corrective action and review of retesting of the SFAS.
N The inspection and in restigation involved 166 inspector-hours onsite by four NRC inspectors.
8001270
.
.-
-
- _. _._......... _..___ _ _ ____.____ _ _ _ _ _ _
_ _. _ _ _ - -. _
-_ _ _ _ _..
-
.__ _ _.... _
. _._.,
i
1 I
.i
..
>
-
cy
.
i
-
i (
'
i i
Results: Three apparent items of noncompliance were identified (Operation
'
I with an inoperable sequencer logic - paragraph 7, failure to adequately l
test - paragraphs 5 and 6, and failure to follow procedures - paragraph
10a.
!
l
a h
!
f
!
$
!
I-l
.
k
.
'
I i
i i
1
'
!
!
l
!
i l
\\
'
I
!'
{
l-l l.
l F
.
i o
.
- -
.'
,
,
.
l.
l
)
l
'
l
~ \\
i O
2-
-
,
r eWa r w+se r * w hww eg e t-v v r*,
e w+m e or.
w e-e-e-wee-m,---,.www*wewr
- wwem emeew-e e,-
e+tr-+=.--=--
-*w--
m- -wee-em*-'*+
-e-w^---"-
- " = = + =
__.
.
A I
\\
- 'd DETAILS 1.
Persons Contacted
- J. S. Grant, Vice President, Energy Supply
- T. Murray, SLs~_on Superintendent
- G. Novak, Superintendent Power Engineering and Construction
- B. Beyer, Assistant Station Superintendent
- C. Domeck, Nuclear Project Manager, DB-1
- W. Green, Assistant to Station Superintendent
- L.
Stalter, Technical Engineer
- J. Buck, Operations Quality Assurance Manager D. Hitchens, I&C Engineer
- F. Miller, Plant Nuclear System Engineer
.
- M. Derivan, Operations Supervisor G. Grime, Reliability Engineer
- J.
Lingenfelter, Nuclear Engir.eer
'
- D. Lee, Test Coordinator (B&W)
- J. Albert, Test Engineer (B&W)
- U. Marathe, Senior Engineer (Bechtel)
- A. Friltag, Senior Engineer (Bechtel)
- S.
Jain, System Engineer J. Zell, Operations Support Engineer J. Nelson, Assistant Engineer D. Mitchell, Assistant Engineer B. Smith, Computer System Coordinator s
T. Beeler, Assistant Engineer F. Johonson, Maintenance Supervisor A. Horvath, Maintenance Foreman E. Cousino, Control Systens Foreman K. Yarger, I&C Group Leader C. Cousino, I&C Group Leader The inspectors also talked with and interviewed other licensee employ-ees, including members of the technical, maintenance, operations, L
- Denotes those attending the exit interviews.
2.
General Background a.
Notification on June 5, '1978, the licensee notified Region III by telephone that during the performance of Surgeillance Test (ST) 5031.07, "18-month
,
SFAS Surveillance Test", they discovered design and installation
,
deficiencies in the circuitry for sequencing electrical loads on the diesel generators during a loss of offsite power in conjunction
,
with a loss of coolant accident. This verbal report was subsequently i
followed up with a 14-day written report to Region III, dated June l
,s s\\
16, 1978, LER 78-055.
!
l (J
.
-3-
- - -
-
.
.
.
i b.
Immediate-Action Letter NRC inspectors were at the site June 6,1978, and through discus-sions with representatives of licensee and review of testing, design and other documentation established four basic problem areas.
1.
Inadequate control of tenninal slide links in the safety-related panels.
2.
Inadequate testing methods to demonstrate operability of control systems.
3.
Conflict between design drawings and the scheme and vendor as-built drawings.
4.
Compromise of safety systems design feature by authorized pro-cedure operator actions.
Based upon these problem areas a corrective action program was estab-lished to insure that the circuit problems associated with the diesel generator sequencer were corrected and did not exist in other safety-related systems.
this corrective action program was confirmed in writing in a letter to Toledo Edison Company from the Director of Region III, dated June 12,1978 (see paragraph 9 for corrective
'~'
action).
'%d c.
Basic Design Features of the Diesel Generator Sequencer The plants emergency core cooling systems (ECCS) are actuated by the Safety Feature Actuation System (SFAS). An integral part of of the SEAS is the sequential addition of ECCS equipment electrical loads on the emergency diesel generators (DG's) to prevent possible overloading of DG's if a loss of coolant accident (LOCA) occurred in conjunction with a loss of offsite power. The SEAS was designed to cover the following combinations of a LOCA in conjunction with a loss of offsite power.
1.
Simultanecus SFAS actuation and loss of offsite power.
2.
SFAS actuation followed at later time by a loss of offsite power.
3.
SFAS actuation and the DG's were later shut down manually and then there was a loss of offsite power, the DG's would restart and be sequentially loaded.
(Note: DG's always start on a SFAS actuation, but are not loaded
~
on the 4160 volt essential buses unless there is a loss of power
'
to the buses).
4.
Loss of offsite power followed at a later time by an SFAS p,_
g
-
actuation.
\\v-4-
,.
_
-_.
,
.
-A
.
U Design features 3. and 4. were added from the result. of a system reevaluation in late 1976 to (1) provide automatic restart and sequentially loading of the DG's and (2) provide sequential loading of the DG's and prevent the loading of ECCS equipment on the DG's at one time. The field design change came out in January 1977 and installations were completed February 14, 1977.
d.
Basic Consequences of the Design Defect and Installation ',rrors Under the conditions in which the circuitry was found June 2,1978 the sequencer would not run. Therefore, the consequences for the
- four design features in 2.c above are respectively:
1.
The DG's would start and tie to the two 4160 volt essential buses. Only the ECCS equipment on sequencer step 1 would be actuated. This would include such equipment as the component cooling water pumps, emergency vent fans, containment isolation valves (close), BWST valves (open), and spray isolation valve (open). Manual starting of other equipment was not defeated.
2.
The DG's would start but all 4160 volt equipment on the essen-tial buses would be stripped. This would include such equipment as the high pressure injection pumps (HPI), low pressure injec-tion pumps (LPI), and service water pumps. Manual restart of the equipment was not cefeated.
3.
The DG's would restart and loads would be stripped as in 2.
4.
The DG's would have started on loss of offsite power and the component cooling water and service water pumps would be running (these pumps start automatically when the DG's start and there is no SFAS actuation). Upon a SFAS actuation the other ECCS equipment would have loaded upon the DG's depending upon the i
SFAS Incident Level. The three major loads would be the.HPI pump (incident level 2),.LPI pump (incident level 3), cor.tain- -
ment spray pumps (incident level 4) (approximately 1200 KW/DG).
For a SFAS actuation without a loss of offsite power, the protective SFAS circuitry was not affected.
I For a loss of offsite power without a SFAS actuation, the protective
-
loss of offsite power circuitry was not affected.
3.
Chronology The general chronology of design, installation, modifications, testing
',
trouble shooting and retesting is as follows:
a.
The SFAS logic diagram, Bechtel Company drawing No. E-16 sheet, was issued for bid September 22, 1971.
The SFAS hardware design and i
fabrication' contract was let to Consolidated Controls Corporation.
'Y-5-
..
.
..
.
.
O-A' )4 b.
On December 12, 1976, Systems Revision Notice (SRN) No. 221E was issued by Bechtel Power Corporation. The purpose of this revision was to (1) provide a means for the sequencer to sequence the safety loads if loss of offsite power has occurred prior to a LOCA and (2) to ensure that the LOCA signal to the sequencer is unblocked if an under voltage cendition occurs and the DG is stopped af ter a LOCA. This design deficiency was reported to Region III by telephone on December 23, 1976 with a followup 10 CFR 50.55e report dated January 21, 1977.
c.
On January 25-27, 1977, construction work permits 24-E-51, 48-E-12, and 48-E-13 were issued by Bechtel and approved by Toledo Edison Company (TECo) operstions group. These CWP's covered pulling of cables, meggering the cables, terminating the leads and installation of the new control relays. The CWP's required a scheme check of the installed circuitry to meet final construction testing requirements, d.
On February 2-7, 1.977, permission to commence work on the CWP's was approved by TECo's shift foreman.
'
e.
On February 10-12, 1977, the work was signed as completed by the contractor representatives and that construction testing was complete by the contractors Q.C.
/'~'s f.
On February 14, 1977, the Bechtel startup engineer signed that start-(
)
up checkout was complete. TECo shift foreman signed that all TECo safety tags were renoved.
g.
February 19-23, 1977, the preoperational test T.P. 310.02, "SFAS Integrated Test" was performed.
h.
February 22, 1977, a copy of SRN 221E was received by the startup test group. The startup test engineer for TP 310.02 reviewed the SRN on March 1,1977 stating that no preoperational retesting was required. The SRN was further reviewed by the Station Review Board (SRB) on March 9,1977 concurring that no retesting was required (i.e., the design modifications of SRN 221E were completed prior to performance of TP 310.02).
1.
March 4-7, 1977, the CWP's were reviewed by the Test Program Mangager and signed that no preoperational retesting was required.
j. A scheme check covering SRN 221E modifications was filed dated March 15, 1977.
k.
The facility operating license No. NPF-3 was issued April 22, 1977.
-
1.
On August 12, 1977, the reactor achieved initial criticality.
i m.
On June 2,1978, ST 5031.07, SFAS 18-Month Test was conducted. This I
/N surveillance test was basically a repeat of TP 310.02 performed (
)
February 19-23, 1977. The sequencer on SFAS channel 1 failed to start
\\/
on a loss of 4160 volt essential bus voltage in conjunction with a SFAS actuation signal. As a consequence, the DG in SA channel 1 L
l-6-
_,
[h V
.
started and loaded on the essential 4160 volt bus, but the loads for sequence steps 2, 3 and 4 did not load on the bus, n.
During trouble shcoting between June 2-4, 1978, the following prob-lems were identified:
(1) Open slide links were found in the sequencer circuitry for channel 3 and 4.
,
(2) The similar slide links for channel 1 and 2 were found closed.
(3) The external wiring for two SFAS relay contacts in the sequencer circuitry were not completed. This manufacturer error in itself did not prevent the operation of the sequencer since a parallel contact in the other SFAS channels performed the same function.
o.
On June 5,1978, Power Engineering personnel confirmed a design deficiency existed in sequencer circuitry. As designed and properly installed (slide links closed) the design would prevent the opera-tion of the sequencer for all four design features (paragraph 2c)
rather than allow proper sequencing of ECCS electrical loads for loss of offsite power followed at a later time by a SFAS actuation (design feature 4 paragraph 2c).
p.
On June 7, 1978, during subsequent review it was determined that part of the memory circuitry of design feature 4 (paragraph 2c)
,
could be defeated by operator actions through the use of emergency
'
procedure (EP) 1202.02, Station Blackout. This memory circuit was an integral part of design feature 4 and would provide the feedback to the sequencer circuit that the 4160 volt essential buses were being supplied by the diesel generators. This memory circuit prevented the simultaneous loading of ECCS loads on the DG's.
q.
On June 17-22, 1978, as a result of SFAS retesting and review of SFAS circuitry, the licensee identified the following additional-problem areas:
1.
The SEQX relay in channel 2 of SFAS was improperly wired.
2.
The No. 3 component cooling water pump was getting a trip signal
on undervoltage w'en it should not have.
3.
The containment spray pump in SA channel 2 would not always start in required time frame.
4.
Design Defect in SFA'S Sequencer Circuit
-
.
The inspectors held discussions with representatives of the licensee and
- reviewed logic and scheme drawings associated with design modifications covered under SRN 221E dated December 12, 1976. Representatives of the licensee stated that the modifications were initiated as a result of a k-7-
..
,.
,
-
...
. - - - - - -
..
D(v)
reevaluation of the assumed time of occurrence of ECCS actuation in relationship to the assumed time of occurrence of a offsit'e power.
Modifications were intended to correct identified design deficiencies
, by (1) -providing a means for the sequencer to sequence safety loads if loss of offsite power has occurred prior to a LOCA and (2) ensuring that the LOCA signal to the sequencer is unblocked if undervoltage condition occurs af ter the DG's are stopped following a LOCA.
Representatives of the licensee stated that the modifications were made based on logic drawing E-16, sheet 1, SEAS Logic Diagram. However, in the as-built SFAS hardware the K04 relay state (energized vs. deenergized)
is opposite that indicated on the logic drawing.
The difference in relay state resulted in the SRN 221E modifications preventing the DG sequencer from performing as designed.
The problem with the SFAS logic diagram (drawing E-16, sheet 1) not agreeing with as-built conditions was iden-fied as a major problem area.
In the exit interview, the inspector discussed this problem and it was the basis of the corrective action items e and g of the Immediate Action Letter (paragraph 9).
Facility Change Request (FCR)78-268 was initiated June 5, 1978, to correct the above design defect. The change was implemented and the
'
SFAS 18-Month Surveillance Test was re-run June 16-20, 1978.
5.
Post Modification Scheme Check b
The inspectors reviewed the licensee's documentation associated with g\\~-
SRN 221E and CWP's Nos. 24-E-51, 48-E-12 and 48-E-13 used to implement the design modification to the SFAS sequencer and DG circuits required by Startup Administrative Procedure 10-A " Construction Work Permit Procedure," and Calibration and Functional Testing Procadure 1-C,
" Scheme Verification Procedure."
CWP 24-E-51 (Startup Syctem 24 - Diesel Generator) was initiated January 25, 1977 and involved the termination of cables, meggering of the cables, and installation of the new control relays (SEQX's). The work was per-
~
formed by the electrical contractor Fischbach & Moore /Colgan Electric Co.
Permission to commence work was approved by the TECo shif t foreman February 7,1977 and the work was signed off as complete by the contrac-tor February 12, 1977. Bechtel startup engineer signed the startup checkout complete on February 14, 1977 and the system was returned to TECo on February 14, 1977.
I CWP's 48-E-12 and 48-E-13 (Startup System 48-SFAS) were initiated January 26, 1977 and involved the terminating of cables and cable meggering. CWP 48-E-12 was performed by Johonson Service involved terminating of four cables at the SFAS cabinets in the control room
-
(Johanson Service has been detailed all work involving termination
-
of cables at the SEAS cabinets).
CWP 48-E-13 was performed by Fischbach
& Moore /Colgan Electric. Permission to commence work was approved by TECo shift foreman on February 3, 1977 for CWP 48-E-12 and February 4, fN 1977 for CWP 48-E-13 and the work was signed as completed by the con-(w/)
tractor on February 10, 1977 and February 12, 1977 respectively.
-8-l l
l l
.
_
_ _.
_
~
,
- (O)
v Bechtel Startup Engineer signed the startup checkout complete on February 14, 1977 and the system was returned to TECo on February
,
'
14, 1977.
l On each of the three CWP's the Bechtel Startup Engineer designated that a post installation scheme check was required and that these scheme checks were completed February 14, 1977.
A search of the licensee records was made to locate the scheme check
,
performed. The only scheme check record found associated with the modifications was one dated March 15, 1977 (see paragraph 8 for other details).
The scheme check performed involved " yellow lining" the circuit modifi-cation associated with SRN 221E.
Each part of the circuit checked is yellow lined as a record that it had been checked per procedure Cali-bration and Functional Testing Procedure 1-C, " Scheme Verification Procedure", Revision 1, dated April 5, 1976.
The objectives of this procedure are to (1) provide a method to verify the proper functioning of a schematic for electrical and mechanical schemes and (2)' to verify that all components depicted on the schematics operate as intended.
The specific procedure requirements in performance of a scheme check are in part:
/~%
/
\\
-
Operate the individual components in the scheme to demonstrate the
\\s_s/
circuits ability to perform as shown on tha schematic drawing.
-
Try all combinations of logic to uncover sneak circuits and ensure proper control of the end device.
-
Record progress of tests by " yellow lining" the schematic diagram as each device or combination of devices is tested.
-
Af ter each scheme has been functionally tested, the drawing from which the testing was conducted will be signed and dated by the person responsible for the test.
The signature of the person responsible for the test on the drawing shall indicate his verification that the acceptance criteria has been met.
i f
The acceptance criteria, in part, are:
The scheme under-test, functions as intended per applicable schematic
-
l l
drawing.
-
All applicable components are operational.
.
.
The requirements of this procedure were not met as evidence from the following:
-9-v)
- -
.,
-
-
.. _ -
-
.
-
%
.
.
i
..
-
Slide links N41 and N1 for SEAS channel 3 were found open in cabi-net RC 3605 and slide links N41 and N1 for SFAS channel 4 were found open in cabinet RC 3703 durir.g the period June 2-4, 1978 (see paragraph 3n and 8).
-
Slide links N41 and N1 for SFAS channels 1 and 2 had to be open February 19-23, 1977 during the performance of the preoperational test IP 310.02, SFAS Integrated Test (see paragraph 6 for further details).
-
The SEQX relay socket for SFAS channel 2 was found incorrectly wired June 17-22,1978 (see paragraph 10.c).
This failure to adequately test the modified circuit installed per SRN 221E according to Procedure 1-C resulted in reactor operscion without the required protection for a SFAS actuation in conjunction with a loss of offsite power as required by Technical Specification section 3. 8.11, 3.8.1.1.b and 3.3. 2.1 (Table 3.3-3 item 4).
This failure to test the modified circuits in accordance to Procedure IC is considered an item of noncompliance with the requirements of 10 CFR Part 50, Appendix B, Criterion XI.
gg 6.
Preoperational Test, TP 310.02, Integrated SFAS Test (-')
The inspectors reviewed test procedure TP 310.02 and associated documen-tation performed February 19-23, 1977, to test the SEAS and its sequencer used to sequence ECCS loads on the DG's for a LOCA in conjunction with a loss of offsite power.
Phase I and II of the test procedure were written to test performance of the sequencer in conjunction with loss of power to the essential 4160 volt buses C-1 and D-1.
Phase I tested SFAS channel 1 and 3 in conjunction with a loss of voltage on the C-1 bus.
Phase II tested SFAS channels 2 and 4 and the D-1 bus.
Phase I and II involved two tests.
The first test was designed to test the simultaneous SFAS actuation with loss of offsite power. The second test tested the restart of the DG af ter it had been blocked.
(Design feature 3, see paragraph 2.c).
This second test was added to test the design features added under SRN 221E.
(See paragraph 4 for description of SRN 221E.)
The inspector verified that the test acceptance criteria were met for the tests performed.
-
It was also verified that the testing would have detected the design error had the design modification installed under SRN 221 been properly installed.
This is evident in part in that ST 5031.07, SFAS 18-Month
/'~'g Test, performed June 2, 1978 was conducted in the same manner as TP t
)
310.02 and detected the design defect. A review of the circuitry also
,
\\~/
confirms this (paragraph 4).
- 10 -
_
.
.
.
jr_
,
d
'
,
s-
-
The most probable reason that TP 310.02 did not detect the design defect was that slide levels were also open in SFAS channel 1 at the time of the test.
(Similar to the links found open in channel 3 and 4 June 2-4, 19 78). The open' slide links would defeat the erroneous circuit permitting the sequencer operation for design features 1 through 3.
I Based upon these facts, it was concluded that there must have been a change in slide link positions (from open to close) subsequent to the performance of TP 310.02 in February 1977 and prior to running TS 5031.07 on June 2,1978.
Whether the slide links were open in SEAS channel 2 also, cannot be ascertained because the wiring error on the SEQX relay in that channel i
also resulted in an open circuit (see paragraph 10.c).
i The control of slide links was discussed with the licensee in the exit l
interview and was one of the corrective action items implemented by the licensee (paragraph 9).
,
The inspector through discussions with representatives of the licensee and review of control schemes verified that had TP 310.02 been written to specifically test a loss of offsite power followed by a SFAS actua-tion, the design defect and installation errors would have been un-
['~'
covered during the properational testing.
SRN 221E used to implement the control circuit modifications designated two separate design g
features (see paragraph 4).
TP 310.02 incorporated only one of these for testing.
10 CFR Part 50, Appendix B, Criterion XI, the Toledo Edison Quality Assurance Manual QAP 2110 and Section 17.2.11 of the FSAR state in part that a test program shall be established to assure that all testing required to demonstrate that structure, systems and components will
,
perform i satisfactorily in service is identified and performed in
'
accordance with written test procedures which incorporate the require-
'
ments and acceptance limits contained in applicable design documents.
These requiremena were not met for the performance of TP 310.02 on February 19-23, 1977 as evidence from the following:
-
At the time the test was performed the SFAS sequencer logic circuitry was designed to cope with four different combinations of SFAS actua-tions, in conjunction with loss of offsite power. Only two'of these were specifically tested.
-
The test performed did not detect the design and installation errors
-
associated with the modification performed under SRN 221E, which
-
were discovered during the performance of ST 5031.07 on June 2, 19,;
and June 17-23, 1978.
- 11 -
ss-./
.
- - -
-
. - -
-.
,
- -
--.
, --
-
- _ _
.
-~
_.-
- s_s/
This failure to adequately test the SFAS sequencer control circuits re-
,
sulted in reactor operation without required protection for a'SFAS ac-tuation in conjunction with a loss of offsite power as required by Technical Specifications sections 3. 8.1.1, 3. 8.1.1.b and 3. 3. 2.1 (Table 3.3-3 item 4).
This failure to adequately test that applicable design features prior to reactor operation is considered an item of noncompliance with the
'
requirements of 10 CFR Part 50, Appendix B, Criterion XI.
)
7.
Facility Operation Without Sequencer Logic
The licensee made a search of jumper-lif ted wire logs and work orders
,
associated with both the SFAS and DG control circuits to determine if any work was performed subsequent to the performance of TP 310.02 in February 1977 that might have resulted in a change in slide link
'
status. A representative of the licensee stated that no evidence was found as to when the status was changed.
!
Further reviews by the inspector indicate that the slide link status may have changed during the scheme check but no documented evidence
'
l has developed to support that conclusion (paragraph 8).
Section 3.8.1.1 and 3.8.1.lb of the Technical Specifications requires in part that two separate and independent A.C. diesel generators be operable.
Inclusive in the definition of operable, section 4.8.1.1.2.c.3 states in part that the diesel generator shall start on a loss of off-
.
site power on conjunction with a safety injection signal, shall de-I energize and load shed the essential busses and energize the auto-
connected essential load through the load sequencer.
Section 3.3.2.1 Table 3.3-3, item 4 of the Technical Specifications requires sequence logic channels of the SFAS to be operable.
l These requirements of the Technical Specifications were not met in that the reactor was operated from August 12,1978 (initial criticality of
<
the reactor) to April 28,1978 (beginning of the outage in which ST
5031.0'i was performed) without the full function of the sequence logic.
'
This conclusion is based upon the fact that at some time between the
performance of TP 310.02 testing the operation of the sequencers for essential bus C-1 and D-1 on February 19-23,1977, and ST 5031.07 on June ^2, 1978, the status of the SFAS sequencer was changed. This status change resulted in the loss of sequencer function in channel 1
'
and 3 (SA channel 1) for all' four design features.
SFAS channel 2 i.
and 4 (SA channel 2) would have performed the intended function for design' features 1, 2 and 3 (see paragraph 2.c for description of design
,
features). These conclusions are evident from:
,
1'
The SFAS sequencers in channel 1 and 3 did not function during the
-
performance of ST 5031.07 on June 2, 1978.
(The testing was sus-
,-
pended prior to testing channel 2 and 4).
!
/.-
- 12 -
,-.
- __
.
.
_
_,
.
.
-
..
..
- ---- -.
-
.
__-_ ________ _ _ - ___-___________
-s
.
-
The open slide links found in channel 4 and improperly-wired SEQX found in channel 2 resulted in the same conditions under which the system was tested under TP 310.02.
-
No evidence was developed that would show that the status of the slide links was changed af ter April 28, 1978. And even if that status had changed af ter April 28, 1978, the full design function of the sequencer logic was not available.
This failure to meet Technical Specification requirements is considered an item of noncompliance.
8.
Investigation of Allegation
!
On June 5, 1978, a representative of a local television station tele-phoned RIII and advised that he had been contacted by an anonymous individual who stated that some Davis-Besse plant personnel had been aware of problems with the SFAS system prior to the reported discovery of the problems on June 2,1978. He was unable to provide any specific information regarding the allegation, or enable RIII personnel to con-tact the individual who had made the allegation. During a subsequent telephone conversation on June 9,1978, he was able to clarify the allegation in that the anonymous individual had not indicated that plant management was aware of the defect, but that some individual
[
(or individuals) at a lower level had known about the defect.,He (,
agreed to request the anonymous individual to contact RIII and provide additional information to facilitate investigation of the allegation, but the anonymous individual did not contact RIII.
During the period of June 13-15, 1978, an investigation of the above allegation was performed by RIII personnel during a visit to the Davis-Besse site. During the investigation, discussions were held with plant management personnel, the equipment in question was inspected, plant personnel were interviewed,. records related to work on the SEAS system were reviewed, and telephone contact was made with an engineer involved with the scheme testing of the SFAS system. On June 20, 1978, further discussion was held with this engineer during a conference call between RIII, licensee, and Bechtel Power Corporation representatives.
Documents reviewed indicated that a modification to the SFAS system had been made under System Revision !!otice SRN 221E, and Construction Work Permits Nos. 24-E-51, 48-E-12 and 48-E-13.
Documents reviewed and state-s ments from personnel involved with the modification indicated that the modifications had been installed prior to the preoperational testing of the SEAS system.
,
Site records reviewed and cos?ents received during interviews indicated
.
that a successful preoperational test of the SFAS system was performed on February 23, 1977. The adequacy of the test proccdures utilized in the preoperational test is addressed elsewhere in this report.
7-_
v
_ 1, _
- -.
_
,
-
- -.
--..
.,. -
_. -
-
.
t (
Licensee personnel indicated that the SFAS 18-month surveillance test was unsuccessfully performed on June 2, 1978. Statements from reactor operators ind.fcated that the test was attempted seven times, without
,
success.
Site documents indicated that following the unsuccessful surveillance test, it was found that circuit opening / closing devices (slide links)
were in the open position in two SFAS channels out of four. Slide links N41 and N1 for SFAS channel 3 were found open in cabinet RC3605, and slide links N41 and N1 for SFAS channel 4 were found open in cabinet RC3703.
In addition, a review of the SFAS circuitry by licen-see personnel indicated that a design deficiency existed as a result of the modifications made under SRN 221E.
The design deficiency in-volved a relay in circuit E-64B (sheet 18) which had normally "open"
,
contacts, where a normally " closed" relay is necessary for the circuit
'
to function as intended. This deficiency disabled the circuit as detailed elsewhere in this report.
Discussion with plant personnel indicated that the slide links for channels 1 and 2 were found to be in the closed peaition. Licensee personnel stated that due to the design deficiency noted above, the preoperational tests of the SFAS system would have been unsuccessful
if these links had been in the closed position during the tests. As fq a successful preoperational test of the SFAS system was performed, it must be assumed that the slide links for channels 1 and 2 were closed (s 'J -
after the preoperational test was performed.
,
Interviews with plant electrical, instrument, and operations personnel indicated that there had been no prior knowledge of the defects in the SFAS system. Further, discussion 3 with plant electrical personnel in-dicated that the changes made to correct the identified defects were
minor and easily made, and could have been made 'during plant operation.
These discussions indicated that there would have been little motive
'
for concealing the defects in the SFAS system.
An attempt to identify when the slide links for channels 1 and 2 were
,
i
- closed was made through a review of records and interviews with plant personnel.
It was not possible to identify any particular work order or modification which would have required work on the portion of the SEAS system in question. However, as previously noted, the slide links could have been closed at any time.
I l
During the record review, it was found that the circuit scheme check
[
(a verification of circuit function)' for modifications made to the SEAS system'under SRN 221E was dated after the date of the preoperational
l SFAS ' test, 'and was Revision 2 of the scheme check. No copy of Revision
.
1 of the scheme check could be -located by _ the licensee. Revision 2 to the scheme check changed only the numbering of control cab' nets, without circuit modifications.
- 14 -
sm-l-
_ _
.
..
. - - -
,
,..
.
...
-
.
..
.
/O
!
l V
Discussion with the Bechtel engineer who performed-the circuit scheme check indicated that the date of-the scheme check was uncertain, and that he may have waited until issuance of Revision 2 of the circuit diagram to document cr. earlier scheme check. As circuits are manipu-laced during the scheme check procedure, it is possible that the slide links for channels 1 and 2 were closed during the scheme check, but no evidence was developed to support that conclusion.
Individual "A", the individual who had anonymously contacted the tele-vision station representative was successfully identified, and was con-tacted on June 18, 1978. Discussion with Individual "A" indicated that he was not familiar with the defects which had been identified in the SFAS system, nor with their location. Individual "A" was advised of the results of the NRC inspection and investigation into the SFAS defects, and was asked to provide any additional information that he could supply.
Individual "A" stated that he could not provide any additional informa-tion, identify any individual who had known of the SFAS defects, nor suggest other avenues of investigation.
Individual "A" indicated that he believed that the plant operating staff and management had no know-ledge of the SFAS defects, or they would not have attempted the SFAS surveillance test several times.
No information was developed to indicate that any Davis-Besse personnel
were knowledgeable of defects in the SFAS system prior to the performance
'N _ /
of the 18-month surveillance test.
No items of noncompliance with NRC regulations were identified within the scope of this investigation.
9.
Corrective Action The licensee committed to a program of review, inspection and testing to assure that.the problems found in the SFAS sequencer circuitry did not exist in other safety-related systems. This program was confirmed in writing in a letter to the Toledo Edison Company from J. G. Keppler, Director. of Region III, dated June 12, 1978.
This corrective action program is outlined below, a.
Slide Links (1) Establish and bnplement a system to verify by visual examination that slide links are in the correct position and screws are tight in the following areas:
(a) Q relay cabinets (b) all safety-related cabinets SFAS, SFRCS, RPS (c) switchgear performing essential functions (2) Slide Link Management Controls s
\\_ /
(a) Review slide link management controls for adequacy and revise as required.
- 15 -
.-
. _
-
ik,I (b) Emphasize to Station personnel the importance of the management controls for slide links.
(3) Document the inspections and any incorrectly positioned links found. Resolve any discrepancies found by documenced evalua-tion and/or testing.
b.
Technical Review of Test Procedures Establish and implement a program including acceptance criteria that verifies adequacy of procedures used to determine equipment opera-bility.
(1) From sample of systems * determine by detailed comparison of design control features to applicable surveillance procedures that the surveillance test does verify system operability in accordance with design. Design features that are not tested-will be documented and justification for not testing vill be provided.
(2) Where design control changes have been made to any safety-related system after the applicable test procedure was written, ensure that the change has been tested ** (intent of change) and ST's are revised to reflect the change re-(gg)
quired. This testing may be TP, ST, temporary mod to ST -
or specific procedure to test the new feature. If segmented x-m tests are used, insure that pr.ser overlap exists to completely test the change.
(3) Review Safety System design drawings from a standpoint of log 4.c diagrams (used as specification drawing) compared with scheme drawings or vendor drawings to ensure there is no conflict.
Establish a priority of review.
(4)
If any significant deficiencies are uncovered as a result of these reviews and tests, the scope of the review shall be in-creased to other Safety Systems.
If any additional deficiencies are found in those other systems, the scope shall be increased to include all Technical Specificat1on systems.
(5) Retest SFAS and verify all design features work.
(6) Review all safety-related plant procedures related to the onsite and offsite power systems and verify that design features are not inadvertantly defeated or compromised by operator action.
.
(7) Establish additional administrative controls to insure that changes or modifications to systems utilize drawings verified correct in accordance with (3) above.
C-(
- The " sample" systems will be SFAS, containment spray, diesel generators,
\\s_
and RPS.
- Testing will be done to the depth to demonstrate that any modified circuit has the proper redundancy, logic and other features of the individual channels.
- 16 -
,
,
aN'
,
(8) These items shall be completed according to the f'ollowing schedule:
(a) Items (1), (2), (5) and (6) shall be completed prior to l
entering the mode in which the system is required by
. Technical Specifications.
i i
(b)
Item (',) shall be completed by September 'i 1978.
(c) Item (7) shall be implemented prior to going into mode 5.
10.
Review of Corrective Action _
The licensee's corrective actions as listed in paragraph 9 were reviewed by the inspector.
a.
Slide Link Review The slide link inspection was performed per Maintenance Procedure
1410.28, Slide Link Inspection (approved June 10, 1978). Attach-ment 1 to the procedure provided verification tL-* the personnel performing the inspections were trained. Attachment 2 provided the list of the electrical equipment and cabinets to be inspected.
p Attachr.ent 3 provided the checklists for documenting the inspec-tions.
(
The inspection was completed June 16, 1978 and the results were pre-sented to the Station Review Board (SRB). Thirteen slide links were found open. Each was evaluated and dispositioned.
Two slide links found open would have prevented valves MS 106 and MS 106A from closing in the event of line rupture. These links had lifted wire tags on them installed in March 1977 for testing.
Ad-ministrative Procedure (AD) 1823.00, Jumper and Lift Wire Control Procedures requires in part a monthly review by the Operations Engineer or his representative of the Jumper and Lifted Wire Log to prevent carrying entries for a long period. Failure to pre-viously identify during the monthly reviews and remove these lif t wire tags is considered an item of noncompliance with regard to the requirement of AD 1823.00.
AD 1823.00 was revised June 12, 1978 and June 20, 1978 emphasizing the control of slide links, jumpers or lif ted wires used in test procedures and the placement of " critical" jumpers or lifted wires.
" Critical" was defined as important safety and reliability of station
-
equipment.
-
An intra-company memorandum was issued June 20, 1978 to all station personnel by the Station Superintendent emphasizing the importance of the control of slide link and the effect it had on the SFAS.
s
- 17 -
,
,
,
- -.
.--
-
-,,
-
-
-
, ~.
(
/-
b.
Tecnnical Review of Test Procedures
,
Special Order No. 9 was issued June 14, 1978 by the Station Super-intendent to implement review of test procedures.
The design feature of all safety-related systems was first tabulated for each system. Reviewers then reviewed routine surveillance test proce-dure (ST) and preoperational tests (TP) to determine whether each design feature was adequately tested.
The design feature of each system were divided into three categories:
-
List of design features tested in ST's list of design features "NOT" tested in ST's, but tested in
-
TP's
-
list of design features "NOT" tested in ST's or TP's The results of the reviews were submitted to the SRB for final evaluation dispositioning and documentation. Where necessary the licensee revised ST's to incorporate the specific testing or a one time test was performed to demonstrate operability.
c.
Retest of SFAS
[ ~ \\
ST 5031.07, 18-month SFAS test was revisa and rerun June 17-23, (,)
1978. During the performance of the rerun the licensee identified
,
several other problem areas.
-
The SEQX relay in SFAS channel 2 had been wired incorrectly.
- The socket wiring for the plug-in relay was found to be re-versed (mirror image). This wiring error resulted in an open circuit for contacts 2 and 8 in the actuation of the K6 relay.
The consequences of this wiring error along with the open slide links in channel 4 is SA channel 2 would have performed its intended function for design features 1, 2 and 3 (paragraph 2c).
>
-
Found that the No. 3 component cooling water (CCW) pump when used as the standby pump for either CCW pump No. 1 or 2 would trip on the loss of voltage. The pump breaker should have stayed closed on loss of voltage. Although the sequencer provides a start signal, it was supposed to be a confirmatory signal. An earlier design modification to remove this trip function from the CCW pumps was incorrect for the No. 3 pump.
.
-
Discovered that the containment spray pump in SA channel 2 would
,
,
not always start in the required time frame for an incident 4
,
,
SFAS actuation (Hi Hi containment pressure).
The manufacturer was to have removed an early design feature that provided a 5-second time delay on incident 4 level actuations. He failed to remove a wire in SFAS channel 4.
The presence of this wire
~~s
' ' ' '
i
- 18 -
!
i
!
,
. -
- -,
-,,
-
--
.
... -
.
-.
- _ _ _ - - - _ _ _ -
/'~'S
'G'1 caused channel 4 sequencer to get out of phase with channel 2 (the sequencers must work in phase). When the phase shift
,
between.the two sequencer exceeded a 3-second window, the i
containment spray pnsp woulo n.. start at the required time.
. This problem involved only sequence step 5 and not 1, 2, 3 and 4 steps.
Af ter correction of the design and installacion, and equipment problems, the test rerun was completed and the acceptance cri-teria was met.
During the review of the test results, the inspector noted several apparent inconsistent alarm printouts on the alarm typewriter.
Some of these had been previously identified by the licensee and were being corrected. Correction and identi-fication of.these alarms was discussed with the licensee in the exit interview, d.
Administrative Control on Future Nuclear Safety-Related Systems Design changes and modifications are performed for TECo under con-tract by Bechtel Company. A Bechtel interoffice memorandum was issued June 16,1978 outlining additional administrative control for Facility
,
Change Requests.
i '
11.
Exit Interview s
The inspectors met with licensee representatives (denoted in paragraph 1)
June 8, 15, 23, and July 19, 1978 to discuss the scope and findings of the inspection.
June 8, 1978 i
The licensee in conjunction with the inspectors outlined the corrective action prueram (paragraph 9).
June 15, 1978 The licensee acknowledged the results of the investigation made in regards to the allegation (paragraph 8).
June 23, 1978 Acknowledged the apparent items of noncompliance (paragraphs 5, 6, 7 and 10a)
.
Acknowledged the inspectors remarks in the review of corrective actions
'
(paragraphs 10a and 10b).
July 19, 1978 y,,)
Stated that a program had been started to identify, correct and checkout questionable alarm inputs to the alarm typewriter. Their program would be given a-high priority at the conclusion of the present outage.
(paragraph 10c).
L
'
- 19 -
l
+wa
-
-
--a--
-.
,,
--e.--
-
c
,w-r-we