ML20235W474
| ML20235W474 | |
| Person / Time | |
|---|---|
| Site: | Davis Besse  |
| Issue date: | 10/09/1987 |
| From: | Shelton D TOLEDO EDISON CO. |
| To: | Davis A NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III) |
| References | |
| 1429, CAL, NUDOCS 8710160248 | |
| Download: ML20235W474 (23) | |
Text
_
t g;:*
USNRC-DS E6YSON IE OCT 17 A O OLI DONALD C. SHELTON Vce Presdent-Nucier Docket No. 50-346' id'818d m 89 License No. NPF-3 Serial No. 1429 October 9, 1987 Mr. A. B. Davis Regional Administrator Region III 799 Roosevelt Road
. Glen Ellyn, IL 60137
Subject:
Response to NRC Confirmatory Action Letter
Dear Mr. Davis:
Titts letter is submitted in response to the Confirmatory Action Letter (CAL), dated September 9, 1987 (Log No. 1-1656), regarding the events surrounding the reactor trip at Davis-Besse Unit No. 1 on September 6, 1937.
Following the' September 6, 1987 trip, Toledo Edison established a Manage-ment Trip Assessment Team whose responsibility included reviewing the trip for appropriate plant response, evaluating equipment malfunctions, and assessing operator performance during and after the trip.
Additionally, Toledo Edison formed a special management team, which included Company Nuclear Review Board members and senior engineering staff, to perform reviews of the event and root cause determination. Although the trip and post trip actions involved some malfunctions and maloperation, the plant was shut down and maintained in a safe shutdown condition without difficulty.
On September 9, 1987, the NRC dispatched an Augmented Inspection Team (AIT), led by the NRC Region III Deputy Director, Division of Reactor Safety, Mr. N. J. Chrissotimos, and supported by NRC Headquarters personnel, to the Davis-Besse site. A CAL from NRC Region III documented Toledo
, Edison's agreement that Davis-Besse would not be made critical without the concurrence of Region III Regional Administrator.
8710160248 071009 PDR ADDCK 05000346 S
PDR j
THE TOLEDO EOISON COMPANY EDISON PLAZA 300 MAOISON AVENUE TOLEDO, OHIO 43652 i
b
g.-
L I
fDock'et No. 50-346.
3 License No. NPF-3' Serial No. 1429
-October 9, 1907-
'Page 2 On September. 11, 19873 the AIT exited with recommendations to allow restart after Toledo Edison had implemented necessary procedural changes and equipment modifications. Subsequently, on September 15,.1987,. Toledo Edison management met with senior NRC Staff in.Bethesda, Maryland to discuss the investigation results' surrounding the event. Approval for restart and return to normal operation was provided by the. Regional Administrator by letter dated September 16, 1987 (Log No. 1-1667).
In the letter the NRC Staff concluded that Toledo Edison's investigation into the circumstances surrounding the reactor. trip and subsequent equipment failures was_ thorough, the investigation identified root causes associated with the various occurrences, and Toledo Edison had established an appro-priate action plan to support plant restart. Therefore, on September 16, 1987,.upon completion of.the necessary training, procedural changes, and equipment modifications, Davis-Besse commenced plant startup.
.Provided'in Attachment 1 is the description of the event and chronological sequence of events for the September 6, 1987 reactor trip. Attachment 2 provides a response to each item discussed in the September 9, 1987 Confirmatory Action Letter.
Should you have any questions, please contact us.
-Sincerel
- ours, Q
GH:EBS:dem Attachments cc: DB-1 NRC Resident Inspector A. W. DeAgazio l
N. J. Chrissotomos j
Document Control Desk 1
1 I
l 1
I I
I i
i'
?
11 L
I
^
ATTACHMENT'1 Description of the Event and Chronological Sequence of Events I,
. (September 6,1987 Reactor Trip) 4 4
- 1 1
2 l
i i
i i
.1 l
J 1
1 I
l j
l 1
a l-t C______-
Docket No. 50-346.
" License No. NPF-3 Serial No. 1429 l'
Page 1 September 6, 1987 Reactor Trip - Event Description TRIP On September 6, 1987.at approximately 1142 hours0.0132 days <br />0.317 hours <br />0.00189 weeks <br />4.34531e-4 months <br />, with reactor power at approximately 100 percent, the Unit tripped when the Reactor Protection
' System (RPS) high flux trip setpoint was reached (see chronological sequence of events, Attachment 1, pages '4-o). The transient was initiated by a Main Feedwater (MFW) flow transmitter failure. The failure of the MFW flow transmitter resulted in an indication of lower than desired feedwater flow to the No. 2 Steam Generator (SG).
This false indication caused the Integrated Control System (ICS) to compensate for the sensed low flow condition by increasing feedwater flow to SG No. 2.
The increased feedwater flow to the SG caused Reactor Coolant System temperature to decrease., thereby adding positive reactivity and increasing reactor power.
The ICS functioned as expected and stabilized reactor power at 103 percent.
The operators monitoring the transient attempted to reduce reactor power by manually inserting control rods. This attempt to reduce power was unsuccessful and the rod movement caused power to increase to the RPS high flux trip setpoint, tripping-the Unit. The attempt by the operator was unsuccessful because the Group 8 rods, Axial Power Shaping Rods (APSR),
had previously been selected for manual control. This selection enabled post maintenance testing for intermittent indication of the Rod Transfer Confirm Light.
Insertion of these rods caused a positive reactivity addition rather than the expected negative addition which would have occurred if the Rod Group Selector Switch had been in the OFF position allowing normal rod sequence control.
Following the reactor trip, a transformer breaker (HX01A) failed to automatically transfer the power from the Unit Auxiliary Transformer to the Startup Transformer (01), and power was lost to the 13.8 kV A Bus (see simplified 4160V single line diagram, Attachment 1, page 7). The loss of a portion of nonessenti.al power (offsite power) caused a trip of two Reactor Coolant Pumps (RCPs) and a failure to automatically isolate steam flow to the Moisture Separator Reheater (MSR) No. I second stage reheat (MS-199). The steam flow to the MSR was subsequently manually isolated.
Emergency Diesel Generator No. 1 started as required due to loss of power to Bus A, and the output breaker closed to supply power to Essential Bus C1.
All essential electrical loads but one started when the Diesel loaded. Service Water (SW) Pump No. I did not automatically start and was manually restarted from the Control Room within minutes.
r~.. a Docket No. 50-346-License No.~ NPF-3
' Serial No. 1429
. Page 2
. Additionally, when the Usin Steam Safety Valves (HSSV) lifted following i
the trip, one failed to fully reseat. This decreased reactor coolant temperature (within allowable cooldown rate) and decreased SG pressures.
Subsequently, the Rapid Feeduater Reduction (RFR) circuit (Feedwater Control Subsystem of the Integrated Control System) drove the Startup j
Feedwater (SUFW) control valves to an initial target position. However, this left a higher than desired water level at the release of'RFR onto low level limit control. This caused the startup feedwater control valves to close with a large error signal which caused SG 1evels to fall below the low level limits. An operator promptly placed the startup feedwater control valves in manual to increase and stabilize SG No. I water level.
After SG No. 1 level stabilized, Startup Feedwater Control Valve No. I was placed in' automatic, and it functioned' properly.
Startup Feedwater Control Valve No. 2 did act require operator intervention to stabilize level. control in SG No. 2.
POST TRIP About I hour and 45 minutes after the reactor trip, at 1319 hours0.0153 days <br />0.366 hours <br />0.00218 weeks <br />5.018795e-4 months <br />, while restoration of offsite power to Bus A was in progress, RCP 2-1 was inadvertently tripped by an operator. The operator restarted the tripped RCP approximately one minute after its shaft stopped. The loss of the third.RCP had no effect on plant recovery. The Bus A power was restored to normal (from'offsite power) via the second startup transformer (02) at approximately 1343 hours0.0155 days <br />0.373 hours <br />0.00222 weeks <br />5.110115e-4 months <br /> and Emergency Diesel Generator No. I was unloaded 1
at approximately 1423 hours0.0165 days <br />0.395 hours <br />0.00235 weeks <br />5.414515e-4 months <br />.
In accordance with the trip recovery procedure, the Motor Driven Feedwater Pump (MDFP) was placed in service approximately 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> and 15 minutes after the trip. As expected, placing the pump in service caused the Main Feedwater System pressure to increase. However, the increased system pressure and resulting pressure differential across SUFW Control Valve No. 2 increased control valve leakage and caused an increase in the level of SG No. 2.
SG No. 1 began a much slower level increase. Both SUFW Control Valves were manually isolated and flow was controlled via the
" mini-feed" bypass lines. This action was successful in decreasing the water level in the SGs.
Approximately 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> and 45 minutes after the trip, restart of the 2 idle RCPs (RCP 1-1 and RCP 2-2) was commenced to allow normal plant cooldown.
j
\\
Turbine Bypass Valve SP13A3 exhibited erratic behavior and subsequently j
failed open approximately 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> and 30 minutes (~1510 hours) after the i
I reactor trip. The failed open position of the valve allowed main steam to dump directly to the condenser.
SG No. 2 level decreased to the Steam and Feedwater Rupture Control System (SFRCS) setpoint of 26.5 inches and continued to decrease to approximately 23 inches.
SFRCS actuation occurred at 26.5 inches, and both Auxiliary Feed Pumps started as required.
Since l
the level in SG No. I was already at 60 inches, Auxiliary Feed Pump No. 1 was secured shortly after starting.
Auxiliary Feed Pump No. 2 was secured l
after SG No. 2 reached a le,el of approximately 60 inches.
l l
l i
' Docket No.-50-346
' License No'. NPF-3 Serial No. 1429
' Attachment.1
.Page 3 On September 7, 1987, atfapproximately 4:30 a.m., the plant had been cooled to the point of shifting to the Decay Heat System. Two successive attempts to place Decay Heat (DH) Loop No. 2 in service resulted in
. pressurizer level drops of 10-and 6 inches, respectively.
The loss of pressurizer level was indicative of a gas bubble in the pressurized Decay Heat System.
Subsequent venting confirmed that a void existed in the
~ Decay Heat piping. The void was most likely caused by nitrogen coming out of solution af ter leaking from the Core Flood Tank past check valve DH76 to the low pressure' side of the system.
Pressurized nitrogen is used to maintain Core Flood Tank pressure. After the gas was vented, Decay Heat Pump No. 2 was started to continue the plant'cooldown to Mode 4.
At 11:15 a.m., operators attempted to initiate service water flow through the CCW Heat Exchanger No. 2.
The service water outlet valve would.not open.
This occurred when establishing DH System operation. As a result, CCW Pump No. 2 and heat exchanger were taken out of service and the " swing" CCW Pump No.-3 with associated heat exchanger was placed in service, i
l l
l l
l l
J Docket No. 50-346 License,No. NPF-3 Serial No. 1429 Page 4 h
)
1
' CHRONOLOGICAL SEQUENCE OF EVENTS The times used in the sequence of events are based on control room logs, operator interviews, Validyne computer printouts, and alarm printouts.
1 s
Initial conditions:
100 percent power, normal operating temperature and pressure.
September 6, 1987 Trip 11:39 Amplifier card fails in Main Feedwater Flow Loop FTSP2A1 causing an indication of low feedwater flow to Steam l
Generator No. 2.
Main Feedwater Flow Control Valve No. 1 l
begins to open in response to indicated low flow.
11:39:07 ICS indicates reactor power is limited by the indicated low feedwater flow.
11:41:02 Operator attempts to drive in Group 7 rods but because Rod Group Selector Switch is in Group 8 for troubleshooting he drives in the Axial Power Shaping Rods.
11:41:34 Reactor Protection System (RPS) Channel 4 Hi-Flux trip 11:41:41 RPS Channel 1 Hi-Flux trip. Reactor trips on two of four coincident protection signals.
11:41:42 High Pressure Turbine Stop and Governor valves close and Main Turbine trips in response to reactor trip. Main Steam pressure begins to rise.
11:41:45 Main Steam Safety Valves (MSSV) lift 11:42:08 Busses A, E6, E3, E2, and essential busses El and C1 are deenergized when auto-transfer from the auxiliary to startup transformer does not occur due to failure of breaker HX01A to close.
Reactor Coolant Pumps No. 1-1 and No. 2-2 trip when Bus A is lost.
MS-199, Moisture Separator Reheater No. 1 Second Stage Reheat Stop Valve, fails to close due to loss of Bus A.
11:42:18 Emergency Diesel No. I has started, comes up to speed, and 4160VAC Essential Bus C1 is reenergized.
[
L Docket No. 50-346 f-
-License No. NPF-3 Serial No. 1429 Attachment I d
Page 5 l
a 11:42:19 Main Steam pressure is reduced but MSSV SP17B3 fails to fully reseat. Valve continues to pass approximately 5 percent of its rated steam flow.
11:42:30 Rapid Feedwater Reduction " targets" and the Startup Feedwater Control valves are closed with a large level error signal due to high initial steam generator levels.
11:42:58 Service Water Pump No. 1-1 fails to auto-start.after Bus C1 is reenergized.
' 11:47:10 Service Water Pump No. 1-1 is started manually j
. 11:49:21 Channel 1 SFRCS low level trip i
11:49:40 Operacor takes manual control of Startup Control Valve No. I and raises level in Steam Generator No. I to-avoid full SFRCS trip.
11:55:48:
Startup Feedwater Control valves control steam generator levels on ICS low level limits.
POST TRIP 13:19:34 Operator inadvertently trips RCP No. 2-1.
l
' 13:22 RCP No. 2-1 is restarted.
13:43:21 Bus A voltage is restored.
13:56 Motor Driven Feed Pump is started, feeding through the Startup Feedwater Control valves to the steam generators.
Steam Generator levels begin to trend upward.
l l
13:57:09 RCP No. 2-2 is restarted.
14:00:49 RCP No. 1-1 is restarted.
14:08:49 SG No. 2 reaches level of approximately 100 inches.
Startup Feedwater Control valves are isolated and feed flow is controlled by " mini-feed" valves.
I 14:27:56 Operators shut down EDG N0.1.
14:30:17 SG 1evels are returned to normal low level limits.
15:08:13 Turbine Bypass Valve SP13A3 fails open.
15:11:43 SG No. 2 reaches low level alarm point.
+-_____-u---
g b
, Docket No.-50-346
' License No.-NPF c r:,
Serial No.,1429 Attachment l'-
Page 6 f
15:12 Channel 1! SFRCS low level trip.
15:12:03 Channel 1 SFRCS low level trip.
SFRCS full trip. Both Turbine Driven Auxiliary Feedwater
. pumps. start.
.15:13:02
'SG No. 2 restored to low level limits.
15:16:48 Shutdown AFW Pump'No. 2.
15:16:49 Shutdown AFW Pump No. 1.
September 7, 1987 i
05:00 Pressurizer level drops a total of 16 inches while placing Decay Heat Removal Loop No. 2 in service.
05:01 Plant is-in Mode 4.
11:15.
SW-1434, ComponentiCooling Water Heat Exchanger Service Water Outlet Valve, will not open when operators attempt to. initiate service water flow through CCW Heat Exchanger No. 2.
I i
l 1
I 4
f
i Docket No. 50-346 License No. NPF-3 To 345 kV Switchyard Serial. No. 1429
- Attachment 1 J L Page 7 M*I" wW Tratu.fermer my j
1
)
From 345 kV S witchyard y
From 345 kV 3C i
--)(
Start up Unit Auxiliary D "I""
Start up mm WW Transformer 02 Transforme 01 Main Generator 25 kV 30 60 H:
HX01A))wo)W
) NC
)No
)No No 13.8 ky SWGR Bus B 13.8 kV SWGR Bus A O
)))
+ (CwP's )
e- -
ACP's
)))
(
)
ww Bus Tie Bus Tie ww u
mmTransformer BD Transformer AC mm 4
- Buses N
~ * " " " " ' " '
Emergency Emergency 4160V SWGR Bus 02 4160V SWGR Bus C2 Diesel nerator 2l
'"'" 1 4160V Transfer SWGR Q
l S.,.me
....t Pump No. 3
_) Ho t. ).__
._ ) NC.,)
)l,__)HC No)
)Wo No 4160V Esan SWGR Bus D1 l 4160V Essen SWGR l Bus C1 1
1
)
i I
l u
og Compor'ent Coolirq
- p u
l Pump No. 3 i
s...
.s l
l Single Line Diagrun 4160V i
J
.I I
ATTACIRENT 2 l-Response to September 9, 1987 Confirmatory Action Letter Items
- _ =.
4 s
w
\\
Docket No. 50-346 License,No..NPF-3
)
Serial No. 1429 I
Page 1 Toledo Edison Response to Confirmatory Action' Letter (CAL) Items-CAL Item 1-Determine the cause of the reactor trip.
i Response to CAL Item 1 The cause of the high flux trip was the positive reactivity addition
' caused by the manual insertion of the Axial Power Shaping Rods (APSR).
Rods were being inserted due to a feedwater transient that had brought reactor power to 103 percent.
Normally, the Rod Group Selector Switch would have been in the OFF position which would have allowed normal rod sequence control. However, post maintenance testing being performed on the transfer confirm light had resulted in the Rod Group Selector Switch heing placed in the Group 8 (APSR) position. Therefore, positive reactivity was added due to insertion of the APSRs rather than the expected negative reactivity which would have resulted from driving in Group 7.
(Also, see response to CAL Item 6, Attachment 2, page 10) l CAL Item 2.
Determine the cause of each malfunction which occurred coincident with or i
subsequent to the' September 6, 1987 trip, including an assessment of historical reliability.
Response to CAL Item 2 Main Feedwater Flow Transmitter Failure The cause of the decrease in flow indication was found to be the failed amplifier board. This transmitter design, which uses an encapsulated amplifier card, is presently used in the complementary flow channel (FT SP2A2) as well as several level transmitter appli>
cations.
The encapsulation of the amplifier card is the standard Bailey improvement to accommodate harsher environments.
This type of transmitter has been in service in the plant ior many years and is very reliable; however., the encapsulated amplifier board in the transmitter has only been used for approximately 1 year.
The specific cause of failure was a shorted capacitor that loaded down the output and caused the transformer to overheat.
This is considered a random failure not indicative of a trend or design flaw.
The amplifier board did not have any historical or repetitive reliability problems due to its limited time in service. The failed q
amplifier board was replaced and the transmitter was returned to l
service after appropriate calibration and testing.
l l
L I
t i
L L - __ - _ _-___-
Dohket No. 50-346 t
. License No.-NPF-3 i
. Serial No. 1429
' Attachment'2 s Page 2-l Auto Bus A Fasc Transfer Failure
.Upon initiation of the Bus A fast transfer sequence, Auxiliary Transformer Breaker HX11A opened and Startup Transformer Breaker HX01A failed to.close.
Two probable causes have.been identified with no clear indication of which is the actual root cause.
1.
The floor tripper was found to have inadequate clearance.
Previous experience has shown that the. floor tripper interlock can prevent a breaker from closing. The as-found clearance for the floor tripper on HX01A was insufficient to maintain margins necessary for variations in clearances in the breaker cubicle and breaker. operation induced vibration.
The floor tripper interlock was readjusted.
2.
A position switch for Main Disconnect 34620 was found to have failed. The position switch provides a permissive interlock to the transfer scheme between the Auxiliary Transformer and the i
Startup Transformer. The linkage that connects the position switch assembly to. the operating mechanism was readjusted.
1
'The System Review and Test Program (SRTP) reviewed the historical performance of the fast transfer of the 13.8 kV busses from their normal sources (Auxiliary Transformer) to' their backup sources (Startup Transformer) and determined that the. transfers functioned, as designed, upon each reactor / main generator trip.
Since SRTP testing, all other operational trips functioned correctly. Toledo Edison checked other floor trippers clearances and found them within acceptable limits.
Service Water Pump No. 1 Failure to Automatically Restart Upon Emergency Diesel Generator Loading When 4.16 kV Bus C1 experienced undervoltage, Emergency Diesel Generator No. I titarted and began supplying power to C1.
Service Water (SW) Pump No. 1 Breaker AC107 tripped as required for C1 bus undervoltage, stopping the pump.
SW Pump No. 1 breaker failed to i
auto close and start the pump after its prescribed time delay. An operator manually started the pump by closing Breaker AC107 from the Control Room.
1
Docket No. 50-346 l
License No. NPF-3 Serial No.1429 Page 3 i
Troubleshooting revealed that control circuit wiring in the AC107 breaker cubicle was deficient,(i.e., one wire was missing and another was mislanded). The deficiencies existed in the undervoltage closing circuit and not in the manual or safety features actuation closing circuits to Breaker AC107. Testing was performed to verify these' findings. The time delay relay controlling the undervoltage auto start feature was tested with "as-found" readings in tolerance.
A functional test of the undervoltage control circuit was performed to verify that the appropriate wiring changes did correct the problem.
The SW pump's failure to auto start upon loading of the diesel was caused by the missing wire. The cause of the missing wire could not
)
be determined, although it is suspected that the wire was removed-when a 10CFR50 Appendix R fire disconnect swit-S was installed in July, 1986.
The undervoltage portion of the controls for the SW Pump No. 1 breaker was last successfully tested in June, 1980.
Some time between the successful June, 1980 test and the September 6, 1987 transient, the control circuit was modified.
l This undervol tage (non-SFAS) automatic start feature was intended to be included in the -scope of the SRTP for all SW pumps. SRTP testing 1
of this auto start feature was performed in January-February,1986 for two of three SW pumps.
SW Pump No. I was out of service and not tested, The procedure did not require all SW pumps to be tested as part of the acceptance criteria.
The wiring was corrected and a functional test was performed to verify that this resolved the problem. The remaining SW pump control circuits were inspected and no similar problems were identified.
Main Steam Safety Valve Failure to Reseat Main Steam Safety Valve SP-PSV-17B3 (B3) failed to fully reseat after lifting. Attempts to reseat the valve by manually lifting and by gagging were unsuccessful. On September 7, 1987, visual inspection of the valve, while on the header, confirmed that B3 had undergone a disc collar failure.
This was verified upon disassembly and inspection of B3 on September 10, 1987. Visual inspections of all other MSSVs were also conducted on September 7, 1987.
Valve k
USSV-B1 was removed as a result of this inspection and replaced with a valve from the A header.
Each header is therefore short one valve which is an acceptable condition governed by the Technical Specifications.
-x),
T Docket No. 50-346-1 License.No. NPF-3' Serial No.'1429
~ Attachment 2
~'
- Page.4 l
Currently the most probable causes involve:
I
.1.
Vibration.'of. upper support structure of the MSSV resulting in galling of,the spindle against the compression. screw.
'This vibration may be excited by response of the valve to the frequency response of the header and pressure fluctuations in the header during MSSV lifts.
2.
' Structural inadequacy of the disc collar design and material 1 for the loads experienced during MSSV lifts.
-The hause of MSSV-B3 failing to fully reseat was a failure of the
'I disc collar. Disc collar failures have been the s abject-of an ongoing' engineering study since the March 13, 1987 MSSV disc collar' failure. This study includes:
1.
evaluation of margins of capacity for multiple MSSV
, failures
]
2.
disassembly and inspection of MSSV R3 and B1 3.
' consultation with B&W on current ongoing MSSV materiv.1:
s evaluation.
This is expected.to be complet.ed by November 30, 1987. B&W's l
y preliminary. report confirms that the disc collar failures were due to high temperature ductile fracture.
This type of. fracture was apparently due to-overloading the threads between the spindle and l
disc collar.
1 The MSSVs were rebuilt and refurbished during the System Review and j
Test Program (SKfP) foll.owing the Jane 9,1985 event. After this 4
'refutbf theent, one disc collar failure occurred during testing at Wyle Laboratory, t
. Investigation of dis.c collhr failures was initiated following 1
- the Narch '13,198*/ MSSV failure to seat (a disc collar failure
/
causing the valvo disc to cock and not seat squarely on valve l
rescating). During discussions and investigation with the valve vendor about the Macch 13, 1987 failure, a previously unidentified
]
t failure of the disc collar that occurred in Septeraber,1981 was
{
discovered.
n Blank flanges have been installed at the locations of the two valves that were removed. The high flux reactor trip setpoint has been
)
W readjusted for one inoperable safety valve' on each steam generator j
in accordance with Technical Specification 3.7-1.
MSSVs will be J
n furbished during the fifth refueling outage with new disc collars, i
aade of stron.ger material, and new stems or will be replaced with
{
different MSSVs.
1 1
z../4 I
L j
y x
Docket No. 50-346.
License No. NPF-3 l Serial,No. 1429
Attachment:
2 Page 5 Startup Feedwater Control' Valve Problems a) liigher than Desired Steam Generato'r Levels Upon RFR Release-Following the reactor trip, the RFR circuit of the ICS properly..
j drove the startup feedwater valves to their targeted post trip
.1 positions.. However, the steam generators were left.at a higher than desired water level at the release of RFR onto low level l
' limit control. This caused the startup valves to close with a.
large error signal which in turn resulted ultimately in a SG 1evel undershoot condition. The control room operator intervened 4
to prevent.a possible SFRCS actuation.
The' higher than desired SG levels upon'RFR release were due, in part,'to slower than desired ramp rates to target values. A longer than desired time countant held the SUFW control valves open longer, resulting in more water flow into the SG because of the delay in achieving target positions..
\\
b)
Startup Feedwater Control Valve Leakage
'l
- Approximately two hours ufter the trip, leakage by the seats of Startup Feedwater Control Valves SP7A and SP7B le.dlto SG control
~
problems. The problems occurred shortly after placing the Motor Driven Feed Pump (MDFP) in service and resulted in' manual isolation of both SP7A and SP7B, Computer data indicates that both SG 1evels increased after placint; the MDFP in service. The inlet pressure on SP7A and
.SP7B increased from about 960 to 1380 psia, as expected, when the 11DFP was running.
SG No. 2 (SP7A side) exhibite'd the j
greates, rate of level increase'at approximately 4 inches per 1
minute or approximately 160 gpm after placing the MDFP in l
service.
.A r view of Data Acquisition Display System (DADS) i data and ICS ignals being recorded at the time indicates that SP7A was not iully closed but approximately two percent open.
l The rate of level increase in SG No. 1 (SP7B side) was much
{
slower at approximately one-half inch per minute or approximately 20 gpm.
Leak testing of SP7A and SP7B was performed in Mode 4 with essentially full MDFP differential pressure across the valves..
Although the leak check indicated SP7B had a higher Icakage rate
.than SP7A, during the September 6,1987 transient, its associated SG 1evel was much Icwer. This was due to SP7A not being fully closed (approximately 2 percent open) and more steam load on SG No. I since it was supplying auxiliary steam loads (i.e., steam was being provided to MSR-1 reheat due to difficulty in initially isolating MS199A, and the leaking MSSV (B3) was on this side).
I i
F 1
_- [ t ' '
I:fg
' Docket No.;50-346 LLicense;No; NPF-3
- Serial No. 1429 Page 6f SG level' control' problems after placing the MDFP in service were caused.by excessive' flow through SP7A and SP7B. This excessive
. flow has been attributed to a higher than design valve leakage,
.SP7A not fully closing (approximately 2 percent open) as commanded, s
,and-the higher head of the MDFP.
{
l A coping strategy has been added to the Trip Recovery Procedure (PP1102.03) which allows the operator to take appropriate action to limit overfeed of the.SGs upon SUFW valve leakage.
Toledo Edison 'is currently ' reviewing the original Facility Change Request (FCR) analysis for the MDFP in application with the startup'feedwater valves and is evaluating the scope of the refurbishment for the valves for the upcoming fifth refueling i
outage.
Turb,ine Bypass Valve Failure Approximately 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> after the trip, Turbine Bypass Valve '(TBV)
SP13A3' failed open causing SG No. 2 level to decrease to the SFRCS actuati6n setpoint. This SFRCS actuation'was a result of both the TBV failure and the intentional isolation of the startup feedwater valves due to leakage. The TBV failure was caused by overhanging the positioner cam and linkage. This overhanging was attributable to improper valve. settings which allowed the valve to exceed the maximum stroke setting.
In addition, entrained water in the upstream piping i
flashing to steam as it passed the valve plug is thought to have contributed to the overhanging by creating excessive forces in the upward direction on the valve stem. Further evaluation and review of system configuration, maintenance, and operation practices is being performed to validate the root cause determination.
Investigations of the TBV SP13A2 actuator failure during the June 9, 1985 event determined that overtravel of the valve occurred such that
]
mechanical fracture due to overload of the cast aluminum and cast l
iron components of the actuator resulted.
The Course of Action j
(Action Plan 9A/9B) provided the results of the investigation and corrective actions related to this failure.
j 1
1 Continued concerns regarding TBV operation ensued following restart
.{
in December,1986 and prior to the September 6,1987 trip.
During j
June, 1987, the TBV actuator positioner was discovered in a condition which may have resulted from and continued to cause erratic operation.
Corrective maintenance and calibration were performed.
During the August 21, 1987 trip, operation of the TBV resulted in the fracture of a portion of the actuator similar to that which was experienced during the June 9, 1985 event.
..--_______ _ A
Docket No. 50-346 I
License.No. NPF-3 Serial No. 1429 l
Attachment.2'
'Page 7 Subsequent to the September 6, 1987 trip, disassembly of the TBV
. actuator positioner revealed a condition similar'to that found in June, 1987.
The September'6, 1987 failure has been attributed to the actuator positioner condition.
Investigation determined that assembly and calibration techniques and procedures may have contributed -
to the pusitioner malfunction and subsequent TBV failure, j
The positioner for SP13A3 was repaired and refurbished. The mechanical and electrical limits of all TBVs were checked and reset as necessary. A full-flow stroke test of the TBVs was performed at full main steam pressure while in Mode 3.
Prior to the September 6, 1987 event the valves and their positioners had been rebuilt under the guidance of a vendor technical represen-tative.
1 CCW Heat Exchanger Service Water Control Valves There were two separate concerns identified with the Component l
Cooling. Water (CCW) Heat Exchanger Service Water Control Valves. The Service Water Temperature Control Valve (SW1434) for CCW Heat Exchanger No. 2 failed to open either from the control room or locally. This occurred when starting CCW Loop No. 2 to establish DH System operation.
The valve operated adequately when isolated.
As a result, CCW Pump No. 2 and heat exchanger were taken out of service and the " swing" CCW Pump No. 3 with associated heat exchanger was placed in service.
SW1429, the temperature control valve for CCW Heat Exchanger No. 3, did not properly control temperature.
The cause of SW1434 failure is that a cavity in the valve body and resulting leakage past the seat caused pressure gradients across the j
valve disc opposing the opening forces of the operator sufficient to I
hold the valve closed.
SW1434 was declared inoperable and will
]
remain out of service until appropriate corrective maintenance has q
been performed.
j 1
SW1429 failed to properly control temperature due to a known j
inadequacy of its valve controller. The controller was not capable 1
of performing over a range as large as the service water temperature j
varies with seasonal or climatic changes. This condition is common to all of the CCW Heat Exchanger Service Water Outlet Valves.
l l
The Service Water SRTP report addressed control problems experienced with SW1429, SW1424, and SW1434. The control problems were viewed I
from the normal operational control mode (modulating SW flow to j
control CCW temperature), and also the accident mode (the SW valve j
failing wide open for design flow). The valves' failure history was j
that of inconsistent CCW temperature control; the SRTP required the i
valves to be fully stroked prior to restart by performing ST5075.01.
l l
L i
l L
t
'DocketNo.i50-346
]
-License No, NPF '
l Serial No.-1429.
Attaciunent ' 2 -
Page 8 I
I 1
This was to provide assurance that the valves would go full open and L
closed and therefore adequately perform their safety function.
The-difficulties in controlling SW flow to maintain CCW temperature
.during varying _ climatic conditions were identified as "not required for restart" action.
It was determined that the current controller is not_ capable of accepting a one time calibration and must be calibrated'for seasonal changes:in service water temperature.
Appropriate calibration frequencies will be incorporated into the i
Preventative Maintenance Program.
The problems with SW1434 reviewed during the SRTP were with regards to the valve's poor throttling capability.
The problem as experienced following the September 6, 1987 shutdown, the valve failing to open, has no history of being a repetitive problem and was not identifled as.such during the SRTP review.
Decay Heat System Void Formation Subsequent'to the transient, while cooling down to Hot Shutdown (Mode 4), the pressurizer level dropped 10 and 6 inches respectively on two attempts to place DH Loop No. 2 in service. The loss of pressurizer level was indicative of a gas bubble in the pressurized Decay Heat (DH) System.
Subsequent. venting confirmed that a void existed in the DHR System piping.
The probable cause of the DHR System void was that nitrogen was coming out of solution as water from the Core Flood Tank leaked through a check valve (DH76) into the DH dischaz a piping. Vent ing l
confirmed the presence of a gas void in the DH discharge piping.
Valve DH 76 is currently scheduled for rework during the fifth refueling outage; Prior to restart, a temporary modification was installed which allows continuous venting of the DH discharge piping to prevent void j
buildup.
The DH System review was within the scope of the SRTP. DH System check valve leakage was not addressed during the SRTP review.
Since the valve did not have historical leaking problems, a gas void caused by DH Syscem (Core Flood Tank) valve leakage was not considered, l
CAL-Item 3 4
Take those actions necessary to ensure that complete documentary evidence of the "as found" condition of equipment which malfunctioned is maintained.
{
Docket No.'50-346 I
License No. NPF-3 Serial No. 1429 Page 9 j
- Response to CAL Item 3-Toledo Edison performed a thorough investigation of equipment malfunctions-prior.to manipulation of the affected equipment with the exception of that required to be placed in service for trip recovery.
Part of that-investigation was to review the "as-found" condition. Documentary evidence of."as found" conditions were preserved via the Trip Assessment Team report, videotapes,: Maintenance Work Orders (MW0s), and normal document contro1' methods.
CAL Item 4-Not disassemble the failed MSSV without the prior concurrence of the AIT leader, N. J. Chrissotimos, or W.-
G. Guldemond of my staff.
Response to CAL Item 4 Toledo Edison complied with this requirement. The valve was disassembled on September 10, 1987 in the presence of Mr. J. Harper of the NRC
- Augmented Inspection Team (AIT).
CAL Item 5 Except as dictated by plant safety, advise the NRC AIT leader prior to conducting any troubleshooting activities.
Such notification should be
'provided soon enough to allow time for the team leader to assign an inspector to observe activities.
Response to CAL Item 5 Toledo Edison complied with this requirement.
CAL Item 6 Review operator and shift personnel actions following the event to determine if those actions were in accordance with your procedures and I
policies.
' Response to CAL Item 6 Operator and shift personnel actions following the trip were in accordance with' Davis-Besse' procedures and policies with the exception of a) securing RCP 2-1 and b) inserting Group 8 control rods.
i a)
During the recovery phase following the trip, a Reactor Operator tripped an operating Reactor Coolant Pump (RCP 2-1).
The operator was concerned that since the two non-running Reactor Coolant Pump Motor Control Switches showed red flags, the RCPs might restart when Bus A was reenergized. He thought that these switches should be operated to the trip state or " flagged" to green. His actions were 1
________m._
l
/
Docket. No. 50-346 License,No. NPF-3 Serial No. 1429 Page 10-performed quickly, and a running Reactor Coolant Pump was inadvertently tripped. A memorandum to all operators terminating the practice of flagging breakers had been issued in January, 1987.
The Reactor
. Operator did not adhere to the instruction. His actions were performed quickly without matching flags to the '.ndicating light. The personnel error was addressed by Toledo Edison ranagement. The operator involved was removed from duty and placed in an evaluation program.
This error was reviewed by all operators during startup training to reemphasize the policy on flagging breakers and review the electrical operation associated with a breaker that trips when the electrical bus trips.
b)
Control-Rod Drive'(CRD) Transfer Confirm Light post maintenance testing was an operational concern since this created an abnormal configuration on CRD operation. The direction in the PM notes provided-inadequate administrative controls over the period of time to select Group 8.
Thus, at times the Rod Group Selector Switch was left in Group 8 well beyond the 5-10-minutes specified.
Prior to the reactor trip, the Rod Group Selector Switch had been placed in Group'8 by one Reactor Operator and announced to the second Reactor Operator.
Shortly thereafter, the Assistant Shift Supervisor temporarily relieved the second Reactor Operator. Although he received turnover information, the out of position-switch was not discussed. The first operator responded to the feedwater flow transient because he was closost to those controls. The Assistant Shift Supervisor assumed the CRD controls, and when he believed he was manually inserting the Group 7 regulating rods, the Group 8 (APSRs) actually moved resulting in positive reactivity addition.
This resulted in a reactor trip on high flux.
Use of PM notes for post maintenance testing is inappropriate. Toledo Edison is currently revising the appropriate procedure (DB-0P-00006,
" Night Order, Standing Order Log") to preclude recurrence. As an interim measure, the operations superintendent is approving all PM notes prior to use. The cause of the third or temporarily relieving operatot not knowing the position of the Rod Group Selector Switch is partially due to an inadequate turnover provided by the operator he relieved.
The plant equipment and operater response to the event did not produce any safety concerns. Overall, adequate procedural guidance existed and operator responses were those expected.
The combination of equipment malfunctions during the trip transient created ra unusual demands beyond the operators capabilities to control and ensure safe plant operation.
CAL ltem 7 Evaluate the adequacy of the normal, abnormal, and emergency procedures used during the event and subsequent plant stabilization.
l l
t
9 i
-Docket No. 50-346
~
' License No. NPF-3<
~
' Serial No. 1429-1 I
~
Page 11 I
Response to CAL Item 7 There were multiple equipment failures during the trip transient. The
.13.8 kV Bus A failed to transfer with a subsequent loss of two Reactor Coolant Pumps and other. electrical loads; the MSSV failed to fully reseat; the Moisture Separator 2nd Stage Reheat Valve did not isolate due to being i
in manual; RFR reacted slower than desired; and SW Pump No. 1 failed to automatically start following an EDG loading. EP 1202.01, "RPS, SFAS,
'SFRCS Trip or SG Tube Rupture," addresses the possibility of all these-events occurring either. singularly or at the same time and prioritizes the response to the.a.
Electrical sources were verified by the operators. They discovered i
that Bus A had not transferred and that Diesel Generator No. 1 had reenergized Class 1E Bus C1, as designed. Upon verifying that correct loads had restarted on C1, operators discovered that SW Pump No. 1 had not restarted and manually started it.
I Overcooling was checwed by procedure. Potential sources of overcooling 4
were examined. "One of the MSR 2nd stage pressures was quickly discovered to be much higher than the other.
An equipment operator manually isolated the steam load.
Concurrently, personnel were dispatched to the MSSVs and j
identified that a MSSV was blowing steam.
I Procedural guidance was provided for monitoring RFR performance. The j
operator recognized that the SG levels ended up high and monitored their decrease to low level limits.
As one SG level decreased below its control setpoint, the operator appropriately intervened to control the overshoot and restored automatic control.
-Adequate procedural guidance was provided for the above responses and all operator responses were appropriate.
The multiple failures during the transient created no excessive operator demand beyond the operators capability to safely operate the plant.
Inadequate procedural controls were exhibited during events leading to the SFRCS actuation. When the MDFP was placed in service, leakage through SUFW control valves resulted in increased SG 1evels although SUFW valves l
indicated closed, After verifying locally the SUFW valves were closed, l
they were isolated and levels controlled on the SUFW valve bypasses. The equipment operator assigned responsibility for manually operating the j
valves was not required to remain in the vicinity.
Subsequently, a TBV on i
the header for SG No. 2 failed open, creating a steam load that reduced the level in SG No. 2 below the trip setpoint for SFRCS actuation. The
)
equipment operator could not be dispatched quickly enough to manually increase SG 1cvel or isolate the open TBV. Although the SFRCS trip may or may not have been prevented by an operator stationed locally at the j
SUFW Bypass Valves, control of SG levels would have been better assured.
There was inadequate procedural control over this abnormal condition, j
l l
I:
e Docket No'.'50-346'
,)
License No. NPF F
~ Serial No. 1429 Page 12 CAL Item 8 Evaluate the. timeliness and completeness of the information reported to
.the NRC pursuant to 10CFR50.72 regarding this event.
I Response to CAliItem 8 Toledo Edison has concluded that a timely'and adequate accounting of the reactor trip was provided pursuant to 10CFR50.72 reporting' requirements.
Initial notification of the event was made via the Emergency Notification System (ENS) at1approximately 1324, 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> and 42 minutes after the trip.
ENumerous. followup' calls were made to both the NRC Operations-Center (via ENS), Region III,'and-the Senior Resident Inspector. All aspects of the trip were specifically discussed with.the possible exception of the EDG start and the SW pump. failure to automatically restart.. Toledo Edison did
- report that Bus A did not transfer (EDG No. 1 is' designed to. start on loss of Bus A), but it appears that no mention of an EDG start (with subsequent SW' Pump failure'to auto start) was made.
. Toledo' Edison reviewed recordings of telephone calls related to the NRC notification _and transcripts of ENS communications in an attempt to
. determine whether the NRC was properly notified of the events. This review wasfinconclusive since all Dahis-Besse outgoing calls are not recorded and Tcledo Edison's call at 1520 hours0.0176 days <br />0.422 hours <br />0.00251 weeks <br />5.7836e-4 months <br /> via ENS could'not be' located on NRC
~
tapes (i.e., NRC did not have a recording of the call).
Toledo Edison is currently. assessing the need for strengthening and providing more structure to the method of communication to the NRC via the Emergency Notification System.
I i
I
_-