IR 05000324/1988027
| ML20151Z760 | |
| Person / Time | |
|---|---|
| Site: | Brunswick  |
| Issue date: | 08/17/1988 |
| From: | Fredrickson P NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML20151Z754 | List: |
| References | |
| 50-324-88-27, 50-325-88-27, NUDOCS 8808300128 | |
| Download: ML20151Z760 (54) | |
Text
'
w
-
D..
.
ii
-
-l TABLE 0. CONTENTS
,
PAGE NO.
I.
INTRODUCTION - FORMATION AND INITIATION OF AUGMENTED INSPECTION TEAM (AIT)
A.
Background
.
B.
Formation AIT
.
1-C.
AIT Charter - Initiation of Inspection
,
D.
Persons Contacted
'
E.
Design Description
1.
High Pressure Coolant Injection (HPCI)._ System
2.
Rod Sequence Control System (RSCS)
,
3.
'
4.
Rod Select Switch
5.
Residual' Heat Removal (RHR) System
6.
Condenser Circulating Water System
II.
DESCRIPTION OF EVENT:
SHUT 00WN OF UNIT 1
,
A.
Event Description
B.
Detailed Sequence of Events
III.
EQUIPMENT STATUS AND EVALUATION
!
A.
DC Motor Operated Valves
f 1.
Eauipment Failure Description
2.
Equipment History
3.
Corrective Action
4.
Safety Significance and Conclusion
B.
l
1.
Equipment Failure Description
j 2.
Equipment History
3.
Corrective Action
4 4.
Conclusion
C.
,
a 1.
Equipment Failure Description
i
'
2.
Equipment History
3.
Corrective Action
4.
Conclusion
1
'
esos300128 880817
,
PDR ADOCK 05000324
.
Q PDC
-.
,
y
_
_
.
I
%
L iii.
D.
Rod Select Switch 24-1.
Equipment Failure Description
2.
Corrective Action and Conclusion
E.
RHR Injection Valve, F017B-
,
1.
Equipment Failure Description
2.
Equipment History
3.
Corrective Action
4.
Conclusion
F.
Condenser Circulating Water Traveling Screen
1.
Equipment Failure Description
2.
Equipment History
3.
Corrective Action
~34 4.
Unit 2 Effect
5.
Conclusion
G.
Equipment Safety Significance and Operability
,
1.
Safety Significance / Safety System Performance
2.
Operability and Reportability
3.
Conclusion
IV.
PERSONNEL ACTIONS
A.
Operations Involvement
1.
2.
3.
Rod Select Switch
4.
Loss of Shutdown Cooling
5.
Circulating Water Screens
i 6.
Conclusion
B.
Management Involvement /0versight on F0178 Valve Failure
j 1.
Senior Management
2.
Mid-level Management
3.
First-Line Supervision
4.
Decision Process
5.
Conclusion
V.
OVERALL CONCLUSION
VI.
EXIT
Attachment 1 - Condenser Circulating Water
'
Attachment 2 - RHR Shutdown Cooling
.
-
. -
.
.
REPORT DETAILS I.
INTRODUCTION - FORMATION AND INITIATION OF AIT A.
Background Brunswick Units 1 and 2 are General Electric (GE) boiling water reactors (BWR)
IV with Mark IC containments.
The units are located 20 miles south 'of Wilmington, N.C. at the mouth of the Cape Fear River in Brunswick County, N.C.
Unit 2 went critical in March 1975 and was commercially operational in November 1975.
Unit I went critical October 1976 and commercial operations began in March 1977.
l On July 13, 1988, a shutdown of Unit 1 commenced in order to conduct work related to the HPCI injection valve,1-E41-F006, DC motor.
In the attempt to conduct an orderly shutdowr, and place the unit in RHR shutdown cooling, several equipment failures occurred between July 13 and 15 which extended the shutdown and challenged all levels of the operations, maintenance and technical support
,
i staff.
These failures involved the Rod Worth Minimizer, the Rod Sequence Control System, a rod select switch, the RHR injection valve (1-E11-F0175) and the traveling screens of the Condenser Circulating Water System.
B.
Formation of All
'
On the af ternoon of Friday, July 15, 1988, the Region II Administrator, after briefing by the Regional and Resident staff and consultation with senior NRC management, directed the dispatch of an AIT to review the Unit I shutdown.
This team was headed by the Division of Reactor Projects Section Chief respon-sible for project management of the Brunswick facility.
Subsequent to this decision, a Preliminary Notification was issued. The team included participa-tion by NRR and AE00. The decision to send the AIT was not based solely on the significance of the individual equipment failures, but rather on the cumulative effect of the number of equipment failures during this shutdown in conjunction with the excessive number of safety system failures during the last two years.
,
Prior to the AIT arrival on site, the licensee submitted a Confirmation of
'
Corrective Actions letter on July 15, specifying completed interim corrective
.
actions and unit restart issues related to sizing of DC motors on HPCI and l
Reactor Core Isolation Cooling valves. This letter also confirmed that Unit 1 would not restart without concurrence from the NRC.
Subsequently, the NRC issued a Confirmation of Action letter on July 15, confirming the licensee's l
commitments.
C.
AIT Charter - Initiation of Inspection The Charter for the AIT was drafted on July 16, 1988, modified on July 20 to reflect the addition of DC motors and issued by the Regional Admtnistrator on July 21. Two team members arrived on site on July 16 and received a briefing at 0900 on the status of Unit 1 and also received a turnover briefing from i
-
-
.
%
I
members of an Operational Perforruance Assessment Team who reviewed the licensee's actions through the shutdown and immediate recovery.
Over the weekend, assistance was also provided by the resident inspection staff.
The remaining members arrived on July 17 and 18.
An AIT entrance meeting was conducted at 0730 on July 18. Subsequent to the AIT arrival on site, several telaphone discussions were conducted between senior licensee and NRC personnel j
j involving the sufficiency of short and long term management involvement to be maintained at Brunswick in order to resume Unit 1 operation and to provide the confidence that the equipment failure problems at Brunswick would be brought l
under control.
Based on these conversations, the licensee provided a letter dated July 18, 1988, describing several management initiatives to improve
,
Brunswick performance. Also, at the time, the NRC began twerty four hour site coverage for the restart of Unit 1 and the scheduled shutdown and subsequent i
restart of Unit 2.
The Charter for the AIT specified that the following tasks be completed:
,
l
'
!
1.
Develop and validate a detailed sequence of event, associated with
'
the shutdown of Brunswick Unit 1 on July 13-15, 1988, and review the equipment failure that led to the shutdown and those failures that occurred during the shutdown.
2.
Evaluate the significance of the equipment failures with regard to iadiological consequences, safety system performance, safety s,onificance, and plant proximity to safety limits as defined in the Tetanical Specifications.
I 3.
Ew.lvate the accuracy, timeliness, and effectiveness with which in1'r.,ation on these failures were reported to the NRC.
4.
For eacn equipment malfunction, to the extent practical, determine:
j a.
Root cause; b.
If the equipment was known to be deficient prior to the event; v
'
c.
If equipment history would indicate that the equipment had j
either been historically unreliable or if maintenance or (
modifications had been recently performed; l
l d.
Any equipment vendor involvement prior to or after the event; e.
Pre-event status of surveillance, testing and/or preventive maintenance; and f.
The extent to which the equipment was covered by existing corrective action programs and the implication of th.e failures with respect to program effectiveness.
5.
Evaluate the effect of the failures on Unit 2, if any, and the i
licensee's response.
l
,
'
6.~
Evaluate the licensee's action taken to verify equipmen't operability on Unit 2.
7.
Identify any human factors / procedural' deficiencies related to the
failures.
8.
Through operator and technician interviews, determine if'any of the following played a significant ~ role in each failure; plant material condition; the quality of maintenance; or the responsiveness -of engineering to identified problems.
'
I 9.
Evaluate operator action during the Unit i shutdown and subsequent equipment recovery.
10.
Evaluate management involvement during the Unit 1 shutdown and the subsequent recovery.
11.
Confirm root cause analyses for the problems associated with the D.C.
,
controlled MOVs.
I 12.
Provide a Preliminary Notification upon initiation of the inspection and an update on the conclusion of the inspection, j
13.
Prepare a special inspection report documenting the results of the
above activities within 30 days of the start of the inspection.
D.
Persons Contacted
,
t
!
K. B. Altman, Manager of Maintenance l
l W. M. Biggs, Engineering Supervisor i
l C. F, Blackmon, Manager, Operations
'
J. S. Boone, Engineering Supervisor
>
G. F. Booth, Project Scientist
'
S. Boyce, Project Systems Engineer
'
l A. J. Canterbury, Mechanical Maintenance Supervisor l
G. C. Cloninger, Mechanical Maintenance Engineer C. R. Dietz, General Manager t
J. Disosway, Electrical Engineer l
K. E. Enzor, Director, Regulatory Compliance T. Groblewski, Senior Maintenance Engineer J. L. Harness, General Manager (Arriving)
i A. S. Hegler, Superintendent, Operations i
R. E. Helme, Manager, Technical Support P. W. Howe, Vice Presidert, Brunswick Nuclear Project H. Mayes, Senior Maintmance Specialist J. L. Pearson, Systems Engineer P. O. Musser, Project Maintenance Engineer
,
!
J. O'Sullivan, Team Leader / Project Manager, Valves R. M. Poulk, Project Specialist, Regulatory Compliance D. E Quidley, Project Maintenance Engineer
.
-
-
-
-
.
,
.
-
~.
-
.
'
o i
K. Scott, Nuclear Engineer C. L. Schacher, Project Maintenance Enginee.-
.S. Tabor, Project Maintenance Engineer
.E.
Design Description E.1. High Pressure Conlant Injection (HPCI) System (System Description S0-19)
The HoCI System,-an Engineered Safeguards System, serves to provide sufficient
core cooling to prevent excessive fuel cladding temperatures in the event of a small line break of any unisolatable line directly associatect with the nuclear boiler.
The HPCI System is. comprised of a 100 percent capacity turbine and pump assembly, with associated piping, valves, instrumentation, controls, and accessories. The pump assembly consists of a main and booster pump to provide sufficient discharge pressure to inject make-up water into the reactor pressure vessel. This make-up water is required to maintain sufficient reactor water inventory since steam generation will continue, af ter the reactor has scrammed,
due to the core fission product decay heat. A turbine driven pumping system is used to supply demineralized maks-up water from the Condensate Storage Tank (CST) to the reactor.
Remote manual operator action may be taken to change pump suctio-from the CST to the suppression pool.
,
The HPCI system is designed to start and deliver design flow within 20 seconds upon receipt of an initiation signal, and is independent of AC power, plant
service air, and external cooling water systems.
However, the inboard steam supply isolation valve is AC powered but this valve is open in the standby readiness mode and is not required to operate for HPCI automatic initiation.
i Steam for the HPCI turbine is supplied from main steam line
"A" upstream of the Main Steam Isolation Valves (MSIVs).
The HPCI steam supply line isolation valves are normally kept in the open position to maintain the line at operating temperature up to the steam supply valve (which is within a few feet of the
!
turbine).
Exhaust steam from the turbine is discharged to the suppretsion pool.
A steam supply line drain pot and steam trap drains accumulated condensate to the clean Radwaste System when HPCI is in the Standby Readiness Mode.
It is automatically isolated when HPCI is initiated (Emergency Mode).
The turbine steam exhaust line drain pot and steam trap normally drain
,
accumulated ccndensate to the suppression pool, ex:ept that on a drain pot high i
water level signal the condensato is automatically drained to the barometric cordenser.
The normal supply of water to the HPCI System is from the CST. In the event the CST level decreases to a predetcrmined level or the suppression pool level rises above a predetermined level, pump suction will automatically be transferred to the suppression pool.
Cooling water for the gland seal exhauster barometric condenser and the turbine lube oil cooler is taken from an
,
j intermediate pressure connection on the main pump and returned to the suction side of the booster pump.
l
,
,,
.--
,r-
_
--,,,_--..y-ro
.
-__,,,,.m.,--
-,-%
,. -
,,-..----w-,.-
- -, -
-,e
.,
,e.
r
.
_
.
.
.
.
.-
.
,
The HPCI System when in operation discharges into the "A" feedwater line and is distributed through the feedwater sparger thereby obtaining uniform mixing with the hot water in the reactor pressure vessel.
A design flow functional test of the HFCI System may be performed during normal
'
plant operation by taking suction from the CST and discharging through a full
.
ficw test return line back to the CST.
The discharge valve to the feedwater line remains -closed during this test so normal plant - operation is no'
d'sturbed.
The control system is designed such that it will automatically transfer from the test to the operate mode in the event system initiation is required. The electrical components of the HPCI Syster, required for system operation, are supplied by DC power from the Station Battery System.
E.2. Rod Sequence Control System (RSCS) (FSAR Section 7 6.1.4)
>
The RSCS prevents the operator from moving a' control rod out of sequence. The RSCS consists of circuitry which allows control rods to be moved only in a
'
predetermined sequence and by predetermined amounts.
These limits are in effect below a preset power level in order to limit maximum control rod worths, which ensures that the enthalpy of the fuel will not exceed design limits as a result of the occurrence of a drop of the maximum worth rod.
In performi.1g this function, the RSCS is redundant to the rod worth minimizer which serves
,
the same function.
Rod density is defined as the percent of control rods fully inserted in the core (100 percent rod density = all rods fully inserted; zero percent rod density = all rods fully withdrawn). In the range of 100 percent to 50 percent
'
rod density, each control rod is moved from the full in to the full out
,
position according to a preset sequence.
From 50 percent rod density to 22
'
percent power, the group notch mode of operation (notch control)-essentially restricts the rod movement to one notch per selection, thus ensuring that all j
designated rods belonging to a group will be within one notch of each other while in this operating range.
'
A rod drop accident is of minimal concern when the reactor is operating above 22 percent of rated power; therefore, the RSCS does not restrict rod movement above this power level. While coming down in power, when the 27 percent power level is reached, the 9.SCS will light the Rod Select pushbuttons to assist the operator in attaining mandatory rod position patterns before the 22 percent power level is reached. Although 20 percent power is the calculated safe power level, the actual level to which the switching point is adjusted is 22 percent.
This ensures that, with all tolerances included, the switching point will never fall below 20 percent.
The system provides circuitry which inhibits rod movement that would result in high reactivity worths from the 100 percent rod density to 50 percent rod
,
density range.
In this range, the four RSCS groups of assigned rods for :Se i
sequences A and B are hard-wired. The circuit either inhibits or permi; yll l
of the rods assigned to a sequence unit to be moved.
This permit /i: %M. t condition is determined by logic circuits wnich monitor the sequence -e rod movemerts.
4 l
-
-
-
.
.
.
.
_.
.
4
The logic receives its inputs principally from the full-in and full-out switches in the rod position indicator probes and from the rod sequence selector switch controlled by the operator.
These full-in and full-out switches are used for indications only, and are not used as in-out to the RWM.
In addition to hard-wired controls, hard-wire indication ensures the operator that the correct rod sequence is being followed.- When the operator selects a particular rod sequence, all of the pushbuttons on the rod select panel of the RSCS group illuminate dimly. Any single rod selected from that RSCS group
illuminates brightly.
Any out-of-sequence rod selected remains-dark, indicating that an out-of-sequence rod has been selected.
Thus, the operator knows that only rods controlled by illuminated switches can be moved at that time. When all rods of the chosen sequence have been moved to their correct
,
position (ell full-in or insert, or all full-out or withdrawn), all of the
'
dimly-lit pushbuttons extinguish, indicating that the operator is permitted to continue with the next correct sequence.
For rods that have to be valved out of service, or for rods with defective switches in the indicator probe, remote bypass switches (under keylocked access) are provided. These switches are under supervisory control and are not activated until it has been determined that no high worth rod pattern would result from bypassing the logic signal to the control logic circuitry.
,
,
In the region between 50 percent rod density and the preset power level, any l
rod not full-out at 50 percent density can only be moved a single notch at a
.
time.
Furthermore, hard-wired logic is used to restrict all the rods assigned
to a notch group to be within one notch of each other.
In this operating range, the pushbuttons of the first selected rod and its associated notch group of rods are illuminated by dim lights on the rod select panel which displays all of the rods.
The dim lights indicating the chosen notch group automatically go out when insert or withdrawal notching of this group is complete and every rod of this notch group is at identical notch positions. The dim lights of the chosen notch group also go out if a rod of another notch group is chosen, and pushbuttons indicating the new notch gr.ep are dimly lit. The notch positions of the previously chosen notch group of rods are "memorized" to ensure that they are within one notch of each other at any time in this operating range.
Once the preset power level is achieved, the RSCS is automatically out of service in the startup operation.
Conversely, the RSCS is automatically in service when the power is decreased to the preset power level. At 35 percent J
j power level during power descention, annunciation is provided to aid the operator in preparing the RSCS for operation.
]
E.3. Rod Worth Minimizer (RhK) (F.qAR Section 7.7.1.6.2.3)
The RWM function assists and supplements the operator with an effective backup
,
control and monitoring routine that enforces adherence to established startup,
.
shutdown, and low power level control rod procedures.
The computer prevents
]
the operator from establishing control rod patterns that are not consistent
'
-
.
.
with pre-stored RWM sequences by initiating appropriate rod select block, rod withdrawal block, and rod insert block interlock signals to the reactor manual control system rod block circuitry.
The RWM sequences stored in the computer memory are based on control rod withdrawal procedures designed to limit. (and thereby minimize) individual control rod worths to acceptable levels as determined by the design basis rod drop accident.
The RWM function does not interfere with normal reactor operation, and in the
'
event of a failure does not, itself, cause rod patterns to be established. the RWM functions may be bypassed and its block function disabled only by specific procedural control initiated by the oparator.
E.4. Rod Select Switch (System Description 50-7)
There are 137 magnetically latched double pole, double throw pu.thbutton switches arranged in the same pattern as the rod display unit. Each switch is
>
back lighted to show the selected rod. Only one rod can be selected a'. a time.
by pressing the corresponding pushbutton.
Pushbutton selection accorsplishes Power Range Monitoring control over rod movement, and activates the companion four-- 4 display group to provide position data.
It also gives a permissive for o.. ;.tional control valve solenoid selection and fuli core display select indication.
The pushbutton array is furnished 28 VDC power through the Rod Select Power Switch.
E.5. Residual Heat Removal System (System Description 50-17)
.
E.5.a low Pressure Coolant Injection (LPCI)
The Low Pressure Coolant injection (LPCI) mode operates to restore and maintain the water level in the reacter core at an adequate height for cooling during a Loss of Coolant Accident which results in reactor depressurization.
This engineered safeguard subsystem operates in conjunction with the HPCI System, Automatic Depressurization System, and the Core Spray System to ensure that the
<
core receives adequate cooling under all postulated accident conditions, How3ver, LPCI, or HPCI, or Core Spray is capable of providing adequate core cooling under specific accident conditions for which it is designed.
Af ter an accident, coolant is lost from the core and the reactor vessel pressure decreases.
The HPCI System which is a high pressure system, pumps water into the reactor vessel when the nuclear system it at a high pressure to preclude a harmful loss of coolant and to reduce the reactor vessel pressure.
If the HPCI System cannot perform its function of maintaining water level and controlling system pressure, the Automatic Depressurization System functions to reduce the nuclear system pressure.
When the pressure in the reactor vessel has decreased to a preset value (410 psig), both Core Spray and LPCI are put into operation to ensure core cooling.
The HPCI System flow ceases and the automatic depressurization is stopped after the LPCI System is put into operation. When the pressure differential between the reactor vessel and the primary containment is 20 psi, the LPCI System is
-
.
.
capable of delivering rated flow. After the core has been flooded to at least two-thirds core neight, only one pump is required to maintain this level.
During LPCI operation, the RHR pumps take suction from the suppression pool through valves E11-F020A and E11-F004A (pumps A and C) and E11-F020B and.
E11-F004B (pumps B and D). The pumpage will be discharged directly into the reactor vessel via both of the recirculation loops through valves E11-F015A and F017A or E11-F015B and F017B, bypassing the RHR heat exchanger (through bypass valve E11-F048 A/B).
During the interval of time when the RHR pumps are operating to restore the reactor vessel level, heat removal is not necessary. Once the core is flooded, one RHR or core spray pump is normally required to make up for shroud leakage.
The other pumps are stopped so that the emergency power-(if there is a loss of off-site power) that was being required by +hese pun.ns may be shif ted to other plant loads including RHR service water ps.1ps, and an RHR pump and heat exchanger in the containment cooling mode.
E.5.b.
Shutdown Cooling The shutdown cooling mode removes the residual heat from the reactor primary system to maintain the reactor 'in a cold shutdown condition and to cool it for refueling and servicing.
The system is placed into operation during the normal reactor shutdown, after a normal cooldown, using the main condenser when the reactor vessel pressure reaches 125 psig or less.
It has the capability of completing cooldown to 125'F in approximately 20 hours2.314815e-4 days <br />0.00556 hours <br />3.306878e-5 weeks <br />7.61e-6 months <br /> af ter the control rods have been inserted and maintaining the nuclear system temperature at or below 125'F so that the reactor can be refueled and serviced.
In the shutdown cooling mode, reactor water is pumped from reactor recirculation Loop A suctiot, through isolation valves E11-F008 and E11-F009 and branches to the suction of the RHR pumps (valves E11-F006A through D).
The reactor water is passed through the RHR heat exchangers where it is cooled by the RHR service water. The coolant is returned into the reactor vessel via the recirculation loop.
In order to prevent possible high thermal stresses in the pipe wall of the tee connection where the PHR line joins the recirculation piping during initiation
,
!
of the shutdown cooling mode, a warmup procedure is utilized.
The procedure involves filling the RHR System with hot water (vessel temperature - 110*F)
before beginning shutdown.
With the entire shutdown cooling loop at an elevated temperature, the RHR pump can be started in the heat exchanger bypass mode at full rated flow without exceeding the RHR to Recirculation Syste.n tee
>
thermal gradients.
Preparations for shutdown cooling are handled as follows:
i a.
The reactor water level is permitted to rise during cooldown so a net loss
1 of vessel inventory can be accommodated without tripping any of the low i
level switches, i
e r
,. - -.
m.
r,
,. _.
-..--_,._,r--
-, ~.,
,---_.--_-,,--,y,
,
.
_
_
_
.
.
.
.
.
-L 9-
b.
When the vessel pressure is low enough so that the LpCI mode is no longer
a
'
required, the RHR System is drained and the heat exchanger flushed to condensate quality, c.
With the RHR heat exchanger isolated, the RHR System suction is opened to the recirculation loop suction pressure cnd the cold water in the RHR piping is forced out through E11-F040.and F049 to Radwaste.
'
d.
The RHR pump is started and full flow established from the recirculation
,
pump suction line through the RHR pump, through the heat exchanger bypass,
'
and back to the recirculation pump discharge line.
,
e.
The RHR heat exchanger is then slowly phased in, and the vessel cooled down.
.
,
E.6. condenser Circulating' Water System (FSAR Section 10.-4.5.2).
The flow diagram for this system is contained in Attachment 1.
The ~ condenser circulating water system is designed to nrovide a flow of 624,000 gpm to the condensers of each unit.
Each of four 25 percent capacity vertical pumps per unit takes suction from the intake canal and discharges through motor operated
!
shut-off valves into an 8 foot diameter *, teel cement-lined pipe leading to the condensers. The mouth of the intake caral is screened by a h inch resh wire
.
fence which serves to divert fish fror, entering the canal. Water flowing'to r
i the circulating water pumps first passts through the trash,acks and traveling
'l
)
screens to prevent entrance of relativnly large objects into the pump sump bay.
,
j There are four traveling screens per utit to prevent small debris from entering i
,
the system. Three of the traveling screens on each unit are equipped with fine
,
}
mesh (Imm) screen to reduce the numbet of fish and fish larvae entrained.in the i
condensers. All of the traveling screens are also equipped with fish pans and
'.
a screen wash system to remove impinged debris and fish or fish larvae from the screens. A collection trough, return sluiceway and return sump are provided to transport fish washed from the screens into a holdup pond for eventual return to their natural habitat. A typical traveling screen is shown in Attachment 1,
There are self-cleaning debris filters installed into each circulating water
condenser inlet line just below the inlet condenser waterboxes. These filters
with a 3/8 inch diameter hole size prevent any clogging of the one inch
condenser tubes.
In addition, to improving condenser officiency and eliminating waterbox cleaning, the filters eliminate tube failuras due to-erosion caused by tube blockages.
'
From the condenser outlet water boxes, the circulating water flows through 84
inch diameter steel pipes which discharge into reinforced concrete tunnels at the west wall of the Turbine Building, The concrete tunnels terminate with a
'
weir approximately 1000 feet south of the Turbine Building.
Four ocean i
discharge pumps pump the water from the discharge canal through two 13 feet
,
diameter reinforced concrete pipes which discharge 2003 feet beyond the shore line. The pipes have a minimum of 3 feet cover and discharge horizontally for j
maximum mixing.
!
.
,, _ - - _ _
_-
-, - _ -,,
,
,
- _ _. _ _ - _ - _ -, _. _ -
-
- - -
l
-
l
.
<
i
[
II.
DESCRIPTION OF EVENTS:
SHUTDOWN OF UNIT 1 The following Event Description and Sequence of Event covers that period from the commencement of shutdown of Unit I until the unit is tied to the grid after
'
restart.
Many equipment problems occurred during the shutdown and restart.
<
Those issues beyond the problems identified in section I of this report, were l
not formally developed by the AIT.
A. Event Des qi_ption July 13, 1988 A decision was made to begin shutdown of the Unit I reactor at 2000 hours0.0231 days <br />0.556 hours <br />0.00331 weeks <br />7.61e-4 months <br /> on July 13, 1988 to allow for work on the HPCI injection valve 1-E41-F006.
The operators reduced reactor power using recirculation flow in accordance with the
,
General Procedure (GP-5). Excess equipment-was removed from service as power was reduced.
At approximately 40% reactor power (2107 hrs.) the operators were warned by annunciations and indications that the RWM had sensed an abnormal control rod pattern and had prohibited control rod movement.
This was an erroneous indication.
The RhN is designt) to only inform of abnormal control rod patterns at this reactor power level and not block cont *ol rod movement. Not i
until approximately 27% reactor power should control rod blocks be received.
J l
The operators bypassed the RWM to defeat this erroneous control rod block. A l
second licensed operator was dedicated to verify proper control rod patterns
!
while the RhH was defeated.
The Senior Reactor Operator (SRO) on shift, in anticipation of entering the enforcemeat zone of the RSCS, reviewed the trouble tickets associated with RSCS.
This was done to preclude experiencing difficulties with the system due to previous malfunctions.
One trouble ticket existed and was cleared out.
The operators continued to reduce recirculation flow until at 2220 hrs. RSCS initialized at approximately 20*4 power. Control rod selection was prohibited.
The cause for the control rod select block was unknown at this time.
The operators verified the proper control rod positions.
Instrument and Control technicians were dispatched to investigate the problem.
July 14, 1988 The Operations Manager was notified (0145 hrs., by tM Shift Operating Super-l visor of the delay in the unit shutdown. Discussions centered on contingencies l
'
for obtaining Condition 3 priur to (xpiration of the HPCI Limiting Condition j
for Operations (LCO) at 1102 hrs. A decision was reached to manually scram the i
reactor if RSCS was not repaired by 0330 hrs.
The Plant Manager concurred on i
this matter.
Technicians identified the problem as a failure of RSCS to confirm all
"A" sequence control rods full-out.
The SRO recognized the association of an existing trouble ticket on control rod 22-03 full-out indication and ordered
l l
l
'
?
l
i
--
. -
_
-
.
l l
t
l
l this control roJ to be bypassed to the full-out position in the Rod Position Information System (RPIS),
c Selection of control rods for movement was now permitted by RSCS (0250 hrs.).
-
,
l The operators continued with a normal shutdown.
l l
l While centinuing with normal control rod insertion, at 12% power, the control
operater was unable to select control rod 34-03 (0645 hrs.). The latching coil l
in thw pushbutton seleutor switch had failed preventing selection of the
'
cont l?ol rod. Instrument and Control technicians were dispatched to correct the
,
l-preblem. At 0745 hrs, maintenance was completed on the pushbutton and control
!
rta insertion continued.
At 0831 hrs. the reactor was manually scrammed from 9.5% power to allow sufficient time to cooldown to Mode three prior to the expiration of the HPCI LCO. The operators followed E0P-01 path 3.
Delays in the shutdown due to the various equipment malfunctions had depleted the time margin to expiration of the LCO, therefore, to ensure normal cooldown the decision to manually scram the reactor was made, t
!
At 0839 hrs the reactor scram was manually reset by the operators.
l
!
One control rod "bounced out" to position 02 after the scram.
To allow l
reinsertion of this control rod (by removing the shutdown mode rod block) the i
operators positioned the mode switch to refuel and inserted the control rod
'
(0848 hr s.).
Subsequently the mode switch was returned to the shutdown position (0849 hrs.) and the shutdown mode switch scram was reset.
The operators initiated a depressurization to 113 psig.
l l
In accordance with the General Procedure for Unit Shutdown, the operators prepared the RHR system for the shutdown cooling mode of operation (1050 hrs.).
,
l Once all prerequisites for shutdown cooling alignment were met, the control i
operator started RHR pump "B" and attempted to establish the flow path to the i
recirculation loop by opening valve RHR F0178.
The valve failed to open.
After repeated attempts to open the valve failed the control operator secured l
the RHR pump (1750 hrs.). Unit I remained on the main condenser for decay heat
'
removal. Loop A RHR was available for shutdown cooling, however, it remained aligned as the LPCI injection path.
July 15,1988 Circulating Water Intake Pumps (CWIP) IC and 1D were in operation, IA and IB
were in standby. (pumps 1C and 10 had fine mesh traveling screens and 1A had
!
I course mesh traveling screens). Debris filtee flushing was scheduled.
Unit 2 was at 100% power running all CWIPs (A-D)
(pumps 2A, 28, and 2C had fine mesh screens 2D had course mesh screens).
Dabris filter flushing had been scheduled on Unit I and the 1B CWIP was started to support this activity (0112 hrs.). At 0249 hrs. IB CWIP was secured after i
the completion of debris filter flushing.
!
l
- -
-
-
'
i
,
i
The control operator on Unit I received a "High Screen Differential or Stop",
.
'
alarm for 1C CWIP (0249 hrs.) One minute later the pump tripped alarm was
'
received. This indicated a major blockage of flow through the screen and that
.
the blockage occurred suddenly. The pumps were automatically tripped due to l
suction pressure degradation. At 0251 hrs, a similar alarm actuated for the ID
!
CWIP and the pump tripped in the same manner.
l
5 t
The operator started the 1A CWIP (0252 hrs.), restarted the IB CWIP (0253
{
hrs.). The 1A had been in standby therefore it appeared safe to attempt a start
'
on it.
Likewise, the 1B pump had been running without difficulty therefore it appeared safe to start this pump also. An outstde auxiliary operator (AO) was dispatched to check the condition of the screens. The report from the A0 was that there appeared to be no flow blockage of the screens, and the screen wash system was operating properly.
l The 2A and 2B CWIPs tripped at 0255 hrs. and 0256 hrs. respectfully. The loss of circulating water flow immediately affected condenser vacuum on Unit 2 (0257). The operators commenced a power reduction by reducing recirculation flow to 35 E+6 lb/hr. In addition, they initiated actions to avoid the region of core flow / power instability. At this time 2C CWIP tripped.
J Unit 2 was brought to 40% power at 0300 hrs. Operations began diagnosis of the CWIP trips to ascertain the cause of the high differential pressure condition.
One possibility was that a low tide condition had contributed to the problem.
To either substantiate or disprove this theory the operators attempted restart of the CWIPs during high tide hours (high tide was to occur at 1100 am).
In addition, the Shif t Operation Supervisor, Technkal Support and Haintenance i
Supervisors were at the intake area to formulate a trouble shooting plan. The plan would include calibration of instruments and jumpering the high differential pressure interlocks and starting the pumps.
These activities
consumed the morning hours.
Several unsuccessful starts were attempted on 2A,B,and C CWIPs over the next several hours. At 1400 hrs. the 2B CWIP was started with the high differential pressure trip bypassed.
Jumpers were instailed on the trip to ascertain the root cause of the event (no visible i
evidence of flow blockage existed). The differential pressure increased to 40 J
inches and the pump was secured (pump trip is normally 36 inches). As a result
of the excessive differential pressure experienced the 2B screen drive shear
,
pin sheared.
'
'
Several sections of the 2B CWIP screen were removed for inspection. The punp was then successfully started (1657 hrs.).
The subject screens were observed
'
to be fouled with skeleton shrimp. These shrimp had attached themselves to the
,
screen and severely restricted flow.
Now 2B and 20 CWIPs were running. Vacuum had recovered and power ascension began (1705 hrs.). At 1810 hrs. power ascension was halted to maintain vacuum J
while awaiting removal of the fine mesh screens from the 2A CWIP.
2A CWIP was successfully started at 2003 hrs. and power ascension continued. At 2340 hrs, reactor power had reached 95%.
l i
i
-
- - -
- - - - -
--
- - - - - -
_ - - - - _ - _ - _ - - - - - ;
-
13
-
i July 16, 1988 I
i A mechanic working on the screens reported that the 2B screen was being damaged-(0005 hrs).
In anticipation of a loss of vacuum, the operators initiated a-
!
power reduction. Tnree minutes later the 28 screen was secured and the 2B pump was manually tripped at 0010 hrs.
Maintenance personnel completed removal of the fine mesh screen on 2C CWIP and it was successfully started at 0403. Reactor power again was increased. At 0415 hrs, power was at 70% and increasing. CWIPs 2A, 2C, and 20 were in service on Unit 2. -CWIPs IA and 18 running on Unit 1.
At 0600, the modification work was completed on the HPCI _1-E41-F006 valve.
Work on this valve precipitated the outage. DC motor replacement was the major modification.
Reactor power is returned to 99.6% at 1900 hrs.
July 17, 1988
On Unit 1, repairs were completed on RHR F0178 and operability verified. Loop
"B" RHR was placed in LPCI standby mode at 0243 hrs.
The "A" loop RHR system was placed in shutdown cooling mode at 2318 hrs.
July 18, 1988 A flange gasket fails on the RHR Service Water (RHRSW) side of "A" loop RHR.
The loop was removed from service. (0144 hrs.). Investigation into the failure began.
At 0507 hrs, the "B" loop of RHR was placed in the shutc.own cooling mode on Unit 1.
RHR loop A on Unit I was returned to LPCI standby alignment after repairs to RHRSW flange gasket.
Shutdown cooling loop B was secured in preparation for unit startup.
July 19,1988 LPCI loop B is returned to service.
In preparation for unit startup the licensee conducted PT-01.6.1 RSCS operability.
There were indications of abnormal control rod movement during this test. Diagnosis of the problem began and while moving control rods the rod selection capability was lost.
July 20, 1988 The Reactor Mode Switch was placed in the Shutdown position to complete repairs
)
of the RSCS.
At 1334 hrs the Mode Switch was placed in the Star'.up/ Hot Standby position to test RSCS ar.d the RWM.
Additional malfunctionr. of RSCS were observed. The Reactor Mode Switch was returned to Shutdown (1400 hrs.),
while repairs were completed,
-
_
-
-
.
.
--.
'l
,
At 1738 hrs, the licensee had repaired the RSCS and placed the Reactor Mode Switch in the Startup/ Hot Standby position. Licensee began venting of control
,
rods which had exhibited difficulty in movement.
]
July 21, 1988
,
I At 1523 hrs, control rod venting and timing were completed, F
' RWM were operable. The licensee commence startup of the Unit I reactor.
The Unit i reactor was brought critical at 1740 hrs.
!
July 22, 1988 Operators tied the main turbine generator to the grid at 0625 ht s.
HPCI i
operability was declared at 1848 hrs.
-l B.
Detailed Sequence of Events The sequence of events was developed from discussions with operations personnel, review of computer alarm typer data recorded during the ev2nt and review of plant logs and investigation packages.
i
.
,'
July 13, 1988 J
Initial Conditions: Unit I at 95.4% power
!
i HPCI System Inoperable l
i
"A" North Side Condenser Waterbox
out-of-service
i 2000 Commenced shutdown to less than 113 psig to replace motor for HPCI i
l 1-E41-F006 injection valve.
j i
2107 At approx. 20*4 power experienced erroneous rod blocks from RWM. Operators bypass RWM, second licensed operator used to verify rod movement.
2200 Cleared Trouble Ticket relating to Rod Sequence Control System (RSCS) in
i anticipation of RSCS initialization.
2220 Recirculation flow decrease d '
the point that RSCS initialized. Control
.
'
rod selection was prevented due to unknown cause.
I&C was called in to investigate the probicm.
!
July 14, 1988
!
j 0145 On-call supervision (operations manager) notified of condition.
j
~
0250 Control rod 22-03 bypassed to full out in Rod Position Indication System.
Selection of control rods fc movement was now permitted by the RSCS, l
-
_-
-_-
-
- -
.-
-
, - - -
.
..
.
.
0645 Unable to select control rod 34-03 from control panel.
0745 Rod selection difficulty corrected, continued with control rod insertion.
0831 Manual scram inserted to allow sufficient time for cooldown to Mode 3 prior to expiration of HPCI LCO.
Operators followed E0P-01 path 3.
0839 Reset reactor scram.
0848 Placed mode switch to refuel to insert control rod 18-31 from 02 to 00.
0849 Placed mode switch to shutdown, reset reactor scram. Cooldown to 113 psi initiated, l
1050 Auxiliary operator filled and vented "B" loop RHR in preparation for shutdown cooling.
l 1055 Reactor pressure at 113 psig.
1750 While attempting to place
"B" loop of RHR in shutdown cooling, the E11-F017B RHR injection valve failed to open.
Continued using the condenser for decay heat removal.
July 15, 1988 Unit I continued in hot shutdown at 95 psig reactor pressure.
l 0112 18 CWIP was started to provide suf ficient flow for debris filter flushing,
'
0249 1B CWIP was shutdown after completion of debris filter flushing.
,
0249 Received "High Screen Differential or Stop" alarm for 1C CWIP Within the minute received 1C pump tripped alarm.
0251 Received the same alarms and subsequent pump trip for ID CWIP.
0252 Started 1A CWIP.
0253 Restarted IB CWIP; an A0 was dispatched to investigate conditions.
0255 Received the same alarms and subsequent pump trip for 2A CWIP.
0256 Received the same alarms and subsequent pump trip for 2B CWIP.
0257 Vacuum decreasing on Unit 2, commenced power reduction. 2C CWIP trips.
0300 Unit 2 at 40*. power.
0302 Start attempted on 2B CWIP was ut. successful due to high screen differential pressure.
-
.
.
.
0304 Start attempted on 2A CWIP was unsuccessful due to high. screen differential pressure.
'
'
0326 Unsuccessful start attempt on 2A CWIP.
-
-
0337 Unsuccessful start attempt on 2C CWIP, i
0408 Unsuccessful start attempt on 2B CWIf, 0820 Unit 2 reactor power is 43%, another unsuccessful attempt was made to start 2A CWIP.
'
0920 Unsuccessful start attempt on 2B CWIP.
1137 Unsuccessful start attempt on 2C CWIP.
I 1306 Unsuccessful start attempt on 2C CWIP.
i 1400 Attempted 2B CWIP start with high ' differential pressure trip bypassed.
Manually secured pump at 40" d/p, Shear pin broken on 2B screen.
l 1657 Several sections of 28 CWIP screen removed, the pump was successfully i
started.
Removed screens were observed to be fouled with skeleton shrimp.
,
1705 Commenced power increase on Unit 2.
28 and 20 CWIPs running.
1810 Stopped power increase at 67% power to maintain vacuum awaiting removal of fine mesh screen from 2A CWIP.
2003 Started 2A CWIP.
2340 Unit 2 reactor power r.t 95%.
j July 16,1988
)
0005 Mechanic reports 28 screen being damaged, operators began reducing power.
i 0008 At 70% power 28 screen is secured.
,
0010 Tripped 2B CWIP manually.
]
0403 Started 2C CWIP after fine mesh screen removal.
0415 Reactor power at 70% and increasing.
CWIPs 2A, 2C, and 2D running on Unit 2.
CWIPs 1A and IB running on Unit 1.
0600 Motor replacement, diagnostic testing and torque switch settings completed on Unit 1 HPCI F006 valve.
1900 Unit 2 reactor power 99.6%.
-
-.
.
-
-
,
!
l
17-i
,
July 17, 1988
0243 Unit 1 RHR F017B valve repairs completed and Unit I loop "B" shutdown
~
cooling.in standby.
I 2318 "A" loop shutdown cooling pla'ced in service.
2350 Gasket fails on RHRSW flange, secured "A" loop shutdown cooling.
July 18, 1988 0507 Unit 1 "B" loop shutdown cooling placed in service.
1153 "A" loop LPCI returned to service.
'
2300 Stopped "B" Icop oc shutdown cooling, in preparation for reactor startup,
,
.
l July 19, 1988
,
0110 "B" loop LPCI returned to service.
'
.
'
0248 Indications of abnormal control rod movement during Performance Test 01.6.1, RSCS.
<
'
,
2200 While attempting to time the movement of two control rods, selection j
capability is lost.
l July 20, 1988 l
0157 Reactor Mode Switch placed in Shutdown to complete repairs on RSCS.
<
I 1334 Reactor Mode Switch placed in the Startup/ Hot Standby position per GP-01
,
to test RSCS and RWM, reactor is now in mode 2.
i 1400 Difficulties experienced with RSCS.
Mode switch returned to shutdown.
i 1738 RSCS repairs completed, placed Reactor Mode Switch to the Startup/ Hot
!
i Standby position.
Licensee began venting and timing of control rods.
{
July 21, 1988
.
1523 Control rod venting and timing completed, RSCS and RWM operable.
j Commencing Unit 1 startup, withdrawing control rods.
!
I
j 1740 Unit I reactor is critical.
July 22, 1988
'
!
i 0625 Unit I turbine tied to the grid.
'
'
1340 Commenced operational performance test on Unit 1 HPCI.
i
-
,
!
-.
-
-
-
_
.
.
.;
<
1848 Completed tese and HPCI declared operable.
III. EQUIPMENT STATUS AND EVALUATION A.
OC Motor Operated Valves W
'
A.1.
Equipment Failure _ Description l
l The circumstances surre Jnding design deficiencies associated with motor powered valve actuators were investigated during this inspection.
This 'section
'
describes two separate problems associated with 250 VDC valve actuator motors
and *, heir control circuits.
A.I.a.
Switching Surge Problem t
In December 1987, valve 1-E41-F001, the steam supply valve to the HPCI turbine, failed to open during a test. This event, reported in LER 87-023, was caused by damaged motor windings. A special consultant determined that the motor
,
,
windings were most probably damaged by a switching surge (voltage impulse)
~
d induced in the separate shunt field. It was postulated that a switching surge stressed the insulation of the shunt field beyond its impulse withstand strength. A Part 21 report was issued on May 6,1988. The failed insulation caused a short between the shunt field conductors and the series field conductors which resulted in high currents and visible damage.
Shorting of a portion of the field winding would result in speed problems. Shorting of the series field would affect armature current and motor torque.
The design deficiency is that a path for discharge of field current upon de-energization
of the shunt field was not provided.
The deficiency applies to all the DC motor operated valves at the site.
,
,
A.1.b.
Insufficient Torque Problem
!
On July 1, 1988, valve 1-E41-F001 again failed to open during a test. On July 2, 1988, the valve manufacturer (Anchor-Darling Co.) and the actuator
.
manufacturer (Limitorque Co) as well as an NRC inspector were at the site to
)
investigate the problem. NRC Report 50-325, 324/88-21 documents the activities conducted at the time to determine the cause of the F001 failure, and identi-
,
fies an unresolved item.
The cause of the July 1 failure was thought to be thermal binding. While studying Motor Actuator Characterizer (MAC) graphs, the
licensee identified a separate problem. The actuator drive motor for F001 was stalling during the period of time (about 2 seconds) that the starting
,
resistor, located in the motor control center, was in the circuit,
i Calculations indicated that the F001 motor could not supply sufficient torque to operate the valve during certain design basis events when the voltage would
be significantly reduced below normal operation.
'
A.2.
Equipment History i
!
A.2.a.
Switching Surge Problem i
-
_ _ _ _ _ __
_ _.. _ _ _
-.. _ _ _ _ _ _ _ _ _ _ _,.
_ _ _
_ _ _,, ~.. _..
..
-
.
.
-
O
'
,
,
!
i
,
As stated earlier, the switching surge problem was basically a design problem
in that a field discharge resistor (or equivalent) was not provided.
The
-
design at Brunswick is somewhat unusual in that the shunt ~ field is continuously energized.
Examination of Drawing LL-9272, Rev. O, dated 11/10/72, Control Wiring Diagram for the 2-E41-F012 valve, shows that the shunt field was wired
in the same manner as the original design.
At least seven DC valve actuator motors failed due to shorted windings since
,
1981.
Four of these failures occurred between January 1981 and January 1982.
The next failure occurred in December 1987, and this failure precipitated the'
,
root cause analysis mentioned earlier. Two failures occurred in 1988 that were
'
t shown to be a result of the switching surge problam.
A.2.b.
Insufficient Torque Problem l
I As a result of poor coordination of design activities, the 240 VDC valve actuator motors were sized without regard to the fact that the motor control
-
center incorporated resistor type reduced voltage starters.
Testing did not t
l reveal the design problem of insufficient notor torque, because the HPCI tests
'
are conducted during periods when the battery voltage is at normal levels. MAC
charts of torque and current contained the information that motors were occasionally stalling during the period of time that the starting resistor was in the circuit, however, the licensee did not focus on this aspect of the MAC charts until the July 1, 1988 event.
l The licensee performed a two-month-internal review of the HPCI system.
The inspection itself was concluded on May 15, 1987. One of the design questions that was generated during the HpCI internal review was:
"Does use of reduced
!
(versus full) voltage motor starters affect IEB-85-03 calculations?" Resolu-l tion of this question was given top priority, and the Technical Support Group submitted a finding on July 17, 1987.
Their finding was that the motors
provided sufficient torque despite the use of starting resistors. They reached i
this erroneous conclusion because they failed to realize the fact that, if the motor stalls af ter the "hammer blow" has occurred, the efficiency of the actuator is significantly reduced.
The E41-F006 valves had marginal torque even without a starting resistor in the circuit. Apparently, attention was not
focused on the F006 valve at that time.
i A.3.
Corrective Action
A.3.a.
Switching Surge Problem j
The corrective action for this problem primarily consists of installing a metal oxide varistor (fused) across the terminals of the shunt winding at the MCC.
i l
The purpose of the metal oxide varister would be to limit the magnitude of
voltage at the winding terminals to 620 Volts which is well below the basic
impulse strength of the winding, The modification packages are scheduled to be i
forwarded to the site on August 15 and September 15, 1988 for Units 2 and 1 i
respectively.
Installation completion is scheduled for January 1989.
In the I
interim, the licensee is manually installing a metal oxide varistor across
'
the terminals before each deenergization of the shunt field.
Also, plant
i
- - -
-
I
.
.
instructions call for verification of DC valve operability after closing the circuit breakers. In addition, procedures will require periodic checks of fuse status.
A.3.b.
Insufficient Torque Problem Motor sizing calculations to support tne adequacy of proposed modifications i
and/or justification for continued operation were reviewed for methodology.
'
Sources of information were reviewed. Specific corrective actions include the following:
,
Plant modification PM-88-019 increased the feeder cable size, bypassed the start resistor, ~ replaced the motor (same size) and changed the gear ratio for 1-E41-F001.
This PM also bypassed the start resistor for both i
'
E41-F006 valves and valve 2-E41-F001.
- Plant modification PM-88-025 replaced the 100 ft-lb motor for 1-E41-F006
with a 150 f t-lb motor. The Unit 2 F006 motor was also scheduled for
!
replacment with a 150 ft-lb motor.
Motor sizing calculations for nine other valves (per unit) mentioned in
'
the Confirmation of Action Letter dated July 15, 1988 were completed, j
Battery sizing calculations were revised in light of the fact that starting resistors were bypassed and motor sizes increased.
- The licensee committed to complete analysis of all DC motor operated valves by October 1, 1988.
This analysis will include determination and i
consideration of actual ambient temperatures at the motors.
- In general, each of these corrective actions apply to Units 1 and 2, but each unit requires a separate analysis. There may be minor differt;nces in required modifications between the two units.
A.4.
Safety Significance and Conclusion The licensee's program was effective in identifying and resolving problems with 240 VDC valve actuator motors and their control circuits during the period of July 1, 1988 to the time of the inspection.
Immediate and short-term correc-tive actions were appropriate to the circumstances.
It is expected that the addition of metal oxide varistors (fused) across the shuat field will solve the switching surge problem. Tests have shown that the surge wave peaked at 4KV or above.
Oscillograms to be made after the modifications are completed are expected to verify that the peak is limited to 620 volts.
In addition, industry experience with this solution gives confidence that it will be effective in solving the problem. The cause of the insufficient torque problem lies in errors made during the original design stage. Reduced voltage starters are not appropriate for motor operated valve applications because of the high starting torque requirement __
~
..
.
.
'
I
The HPCI system is required by the Technical Specification to be OPERABLE when W
the Unit is at power. Because of design problems mentioned herein (inoperable i
valves) the HPCI system may have been unable to perform its intended function i
during certain design basis accidents. There are apparently five possibilities where the licensee could have identified this valve operability problem earlier and declared the HPCI system inoperable.
They are as follows:
!
The actuator vendor has stated that if-it had known.of the starting
-
resistors during plant design and construction, a larger motor would have
'
.
'
been provided
Previous failures of DC motors could have been caused by the undersized
-
'
motor phenomena MAC graphs, if studied and analyzed closely, could have revealed the
-
relationship between motor stalling and the "hammer blow" effect.
.
A 1984, 125/250 Volt DC Load Study identified problems with DC motors as
-
they relate to starting resistors
The 1987 internal HPCI Safety System Functional Inspection (SSFI)
-
l identified concerns over undersized DC motors.
Collectively, all of these issues should have resulted in the licensee identi-fying this problem earlier.
The major contributors to this pre-knowledge
appear to be the lack of vendor knowledge of starting resistors, the DC load
In consideration of the above facts, the DC motor
'
)
problem is identified as a violation, inadequate Corrective Action for Problems Identified in DC Motor Operated Valves.
Tracking of this violation will be a
conducted, using a subsequent inspection report.
B.
B.I. Equipment Failure Description I
J At about 2107 on July 13, during the shutdown of Unit 1, the control room j
received erroneous control rod blocks from the Honeywell rod worth minimizer j
(RhN) at about 40 percent power. The RWM had obviously failed because, at 40
percent power, the proper response of the RhN is to provide annunciation and not blocks when deviations from the prescribed rod withdrawal / insertion pattern are detected. The system is designed to provide rod blocks only when reactor power is less than 23 percent.
The system was bypassed, and as per the j
Technical Specifications, a second licensed operator was stationed to j
independently verify
.,d position.
After the RhN was bypassed, its operation was monitored as the shutdown
'
proceeded. The RhN only provided annunciation signals below 23 percent reactor power instead of rod blocks as expected. The system's responses were directly
opposite of what they should have been, s
.
-
--
-
.
.
_-.
.
..
-
-
.
.
.
t l
<
'
)
i i
l j
The failure of the RWM during the shutdown was attributed to a faulty 6 VOC l
t power supply.
This was determined following the shutdown when troubleshooting
'
i of the system revealed a high level of electrical noise in the power supply.
!
B.2. Equipment His. tory
The Operations and Instrumentation and Control (I&C) staffs stated that the RWM
has had an extensive history of mechanical and computer software failures
'
during the plant's life. These personnel indicate, without exception, a lack i
of confidence in the system.
[
!
The inspector reviewed Brunswick work orders dated from 1984, reports submitted
i since 1985 under 10 CFR 50.72, and the NRC Document Control System data base l
l which contains records from 1979.
Review of these data bases produced five records describing RWM failures on Unit 2.. One record was found which
'
described a failure of the RWM on Unit 1.
This was the most recent failure of f
the RWM system prior to the July 13, 1988. failure. This failure occurred June
,
,
6, 1988 on Unit 1.
While proceeding to shutdown, the RWM would not annunciate
)
!
or block during movement of rods which were out of sequence.
l
B.3. Corrective Action
[
i i
i Short-term corrective action for the July 13 failure included changeout of the
!
faulty power supply. An additional problem with the RWM occurred subsequent to
!
i the shutdown of Unit 1 and required maintenance.
However, this problem was i
i beyond the scope of the AIT's review charter. The long-term corrective action
'
includes replacement of the current RWM with an updated design which has a
i dedicated computer.
This modification will take place during the next
!
scheduled refueling outage for Unit 1 in November, 1988.
The new system, the i
'.
GE designed NUMAC, was installed on Unit 2 during the unit's last refueling
'
outage.
The experience with the system to date has been positive.
The j
d licensee believes the new system will eliminate the types of problems caused by the old RWM system.
,
!
i l
B.4. Conclusion
'
)
The actions of I&C personnel in trouble shooting and repairing the RWM ll were reviewed by the inspector and found to be adequate.
.
,
i
The replacement of the old RWM system with the new NUMAC system should, at least, eliminate concerns regarding cornponent aging.
Given the positive
,
>
l experience to date with the system on Unit 2, its placement on Unit 1 should reduce or eliminate problems which have been experienced during low i
J power operations and restore personnel confidence in the RWM system.
i i)
C.
Rod Sequence Control System C.1.
Equipment Failure Description
i f
At 2220 on July 13, after initiation of rod block enforcement, RSCS began to
'
prevent the operator from selecting control rods for movement while completing
{
l t
t
-
-
-
-
-
-- -- - --
-
-
--
--
.
-
-
-
-
. -
.
.
'
e i
the controlled shutdown.
Following unsuccessful efforts of operating personnel to determine the reason for the problem, an I&t., technician was called in to
- ,
troubleshoot the system.
.
'
i'
<
'
C.2. Equipment History
!
l The inspector reviewed Brunswick work orders as well as reports submitted to the NRC by the licensee under 10 CFR 50.72 and 10 CFR 50.73.
There have been
'
!
numerous failures of equipment which provide input to the RSCS, However, there have been few reported failures of the RSCS itself.
j Licensee work orders dating back to 1984 and related to the Rod Manual Control
'
System (RMCS) indicate that, of a total of forty-seven records reviewed, 3 records were related to corrective maintenance required to be performed on the
RSCS for Unit 1, Of the fifty-six work orders completed on the RMCS of Unit 2
- ~
during the past four y. trs, nine involved the repair of components in the Unit
,
i 2 RSCS. A :ubstantial number of the remaining work orders were related to the
repair or re,slacemn'. of rod select switches.
Completed work orders also j
traced many eqcip.
a1 functions to failed or inoperable reed switches used
'
'
,
in control rod tot
'ldication.
C.3. Corrective A;
,
)
The I&C technician reviewed outstanding trouble tickets and work orders
!
j recently initiated on the Rod Control System, and conferred with operations
'
personnel with respect to the problem.
Initial troubleshooting involved review
,
of the full out indication on the full core display.
This resulted in
'
j replacement of several burned out bulbs and concluded with the I&C technician
asking if there were any other burned out bulbs.
These efforts proved
-
'
unproductive.
Therefore, the I&C technician began a comprehensive examination
!
of RSCS to determine the cause for the rod select block. After about two and
-
j one half hours into his troubleshooting procedure, the technician found that RSCS was not sensing all the appropriate RPIS interlocks for control rod full out indication. The technician informed operating personnel of his discovery.
Thus, when the technician informed them of his findings, operations personnel
,
stated that they immediately recalled the relationship between the RPIS and the
'
RSCS. There was a tracking LCO on control rod 22-03 related to a failed reed switch which provides the "full-out" indication for the control rod. The rod
,
position indication was immediately bypassed to fullout and, as a result the
<
rod select block was removed.
Shutdown efforts then proceeded, j
C.4 Conclusion The inspector reviewed the events leading to the determination that the RSCS
had failed.
The inspector also reviewed the events during and subsequent to
the I&C investigation of the purported failure. The history of RSCS failures j
at Brunswick were also reviewed. As a result of these reviews, the inspector concluded the following:
'
!
The RSCS responded properly during the shutdown.
The failure of
{
operations personnel to recall the relationship between lack of fullout
,
J
.
O
indication for control rod 22-03 and the red block inserted by the RSCS accounted for the conclusion on the part of operating personnel that the RSCS system had failed. Had the review of LCOs been more successful, the time required to analyze the RSCS system would not have been expended.
- The fact that both Operations and I&C personnel failed to notice that rod 22-03, on the edge of the core, was unlit, by not methodically checking each ligh1 associated with the
"A" rods, contributed to the delay in shutting down the reactor.
- The inspector reviewed the licensee's corrective acticns with respect to troubleshooting the RSCS system.
The inspector believes thet the amount of time which was required to isolate the failed component was not inordinate.
The I&C technician responsible for the effort displayed adequate knowledge and ability in troubleshooting the system.
- A review of recent operating experience with the RSCS indicated that there l
have been a number of failure events involving components which have functions that are important to the operation of the RSCS.
Of the failures reported, a substantial number appear to be related to the failure of reed switches used in rod position indication.
D.
Rod Select Switch 0.1. Equipment Failure Description At 0645 on July 14, an attempt was made to select control rod 34-03 for insertion during the shutdown sequence. The rod select switch (RSS) for this rod was depressed several times; however, the operators were unable to select l
the rod. I&C personnel were requested to determine the cause of the problem.
l A review of recent failures involving RSS at Brunswick Unit 1 indicate a number of failures of rod select switches. Most of these failures can be attributed to the normal wear which might be expected at end of life.
D.2. Corrective Action and Conclusion l
l I&C personnel determined that the failure 7f the switch was due to a burned l
inductor (coil) in the RSS circuitry. The switch was repaired and normal rod insertion continued.
,
The failure of the RSS appears due, primarily, to the effect of normal use and i
aging. Interviews with personnel at the plant reveal that failures of RSSs are
'
not uncommon and work orders reviewed by the inspector tend to support this statement. Often, operators find it necessary to depress switches more than once and with differing degrees of pressure and at various points on the switches to have them f unction properly.
The licensee is contemplating the
.
l t
l l
- - - - - - - -
-
.
.
installation of a new modular _ RSS system which employs "plug in" components which can be changed in a fraction of time currently required for the existing system's components.
The inspector believes that a change to a newer, more easily maintainable system would also increase the reliability of the system by replacing components which are approaching the end of their useful life.
Replacement of the system would substantially reduce the frustration operators experience during their efforts to select rods with faulty switches.
E.
ftHR Injection Valve, F0178
_
E.1. Equipment Failure Description On July 14, 1988, while lining up loop B of the residual heat removal (RHR)
system to enter the shutdown cooling mode for Unit No.1 in accordance with Section 5.4 of Procedure OP-17, valve 1-E11-F017B failed to open on demand due to misalignment of the clutch mechanism assembly in the valve operator.
There are two identically arranged RHR system loops in each of the two Brunswick Units, and each loop has a 20" x 24" angle valve to allow RHR flow into the recirculation loops. The RHR outboard injection valve 1-E11-F017B is the valve on loop B for this purpose. The F017B valves are routinely used to return flow to the reactor vessel during the cool down process in the plant shutdown process. All F017 valves were made by Rockwell and have Limitorque model SMB-ST operators. Attachment 2 contains a sketch of the RHR shutdown cooling mode.
Approximately 15-20 minutes prior to the event, Valve 1-E11-F017B was successfully closed per Step 5.4.B.58 of Procedure OP-17 in preparation for the starting of the RHR 1B pump.
This was later followed by six unsuccessful attempts to open the valve per step 5.4.B.59 of Procedure OP-17. An auxiliary operator ( AO) stated that: "while proceeding to check the operation of the pump j
[1B RHR), a grinding noise was noted coming from the 1-E11-F017B. When I looked up at the valve, the handwheel was noted spinning smoothly at a high rate of speed as if the clutch was engaged." The operator in the control room stated that he "attempted to throttle the valve open to establish the RHR flow.
The indication light flickered but the valve never opened.
Did not receive dual indication...., no indication of RHR flow, and no decrease in Rx level.
Held switch in open for [approximately) 20 seconds.
Since there was no indication of flow the RHR pump was tripped at this time....
Attempted to open valve again with same results....
Tried to open valve with same results.
,
The A0 checked the valve locally again and manually disengaged the clutch. We attempted to open the valve again but it wouldn't. We then closed the F0158 valve and attempted to stroke the valve again with no success.
The valve indication flickered each time when the switch was operated."
As can be seen from the operator's report, several attempts were made to open valve 1-E11-F017E without success. The licensee initiated immediate action to determine the cause of the failure without definitive results.
Upon disassembly of the operator it was discovered that the mating lugs on the sliding gear clutch and the flexible jaw clutch housing were badly damaged and would not engage. In addition, the sliding gear clutch was found jammed into
,
.
.
'
!
,
the solined insert.
Attachment 2 contains copies of pictures of the damaged
'
components.
The operator of valve 1-E11-F017A was subsequently partially disassembled and inspected to determine if similar problems existed.
Inspection results indicated that degradation of the clutch mechanism had not I
occurred. At the tiine of the AIT debrief on July 21, 1988, the reason for the i
damaged. clutch assembly-had not been determined, possible root causes
-
e i
identified by the licensee (in order of probability) are listed as follows-
Material failure over time, aggravated by a full torque backseating event l
'
,
that took place in 1987 when a jumper had been installed to bypass the limit switch,
Clutch mechanism assembly parts not meeting design criteria,
Clutch compression spring failure, and
'
Operator error.
-
'
Initial reviews performed by the licensee have eliminated the last three pos sibili tie t,.
The first cause may be an adequate description of the failure but does not constitute the root cause for the failure,
,
To determine possible root causes of the operator failure, the AIT conducted
]
interviews, reviewed maintenance histories, studied inservice test data, l
~
Limitorque maintenance procedures, and literature available concerning aging and
!
~
service wear of Limitorque operators. The results of the efforts are included below, i
,
!
E.2 Equipment History
,i
,
j E.2.a Maintenance History l
Maintenance of individual equipment, including preventive maintenance work,
,
requires the issuance of a work request and authorization, and a job number to
get started.
Typical examples would include packing leakage, which would be
,
!
requested by the operating personnel, and greasing and lubrication, which would
.
be initiated by the maintenance department.
The maintenance department l
performs the requested work on a priority oriented basis.
Examples of
'
maintenance history records for the four E11-F017 angle valves can be high-
!
l lighted as follows:
1-E11-F017A i
l Local position indicator was out of adjustment.
'
)
Adjusted limit switch to reflect problems associated with backseating of valve 1-E11-F017B event in 1987.
l I
i i
I t
,
__
.
__ _.
.
.
-.
,
i l
1-E11-F0178
,
Long history of packing leakage due to galled stem. The galled stem has
!
'
never been repaired or replaced since its discovery sometime prior to 1982.
'
A modification was performed in 1987 which jumpered out the limit switch.
The jumper was not removed, consequently, when the valve was stroked open the motor continued to run, backseating the valve and burning out the
motor.
Based on schedular constraints and discussions with limitorque, the licensee chose not to open and inspect the operator for possible l
stress damage.
- Local position indicator was out of adjustment.
- Clutch mechanism failure while attempting to open valve on July 14, 1988.
2-E11-F017A
Local position indicator found missing in July, 1988.
l 2-E11-F0178
,
'
l Motor drive gear slipped out of alignment causing the operator to be inoperable.
'
'
'
Installed replacement operator gear parts.
Motor leads replaced.
l
Maintenance history indicates that the F0178 valves have had more service
!
oriented maintenance problems than the F017A valves. This probably was caused
by the fact that F017B valves were used to initiate the shutdown cooling flow
'
, during plant shutdowns, while the F017A valves were infrequently used.
It should also be noted that an-NRC inspector cbserved a plant craftman trying to
,
reposition the 1-E11-F017B declutch lever to the "motor" position during a
.
repair evolution following the July 14, 1988 event.
The Limitorque operating
'
instruction manual forbids this action. Although the mechanical arrangement of the operator should preclude this action, the observation reveals that other inappropriate manipulations of the operator could have caused damage prior to the event in question.
E.2 b Surveillance Test History The four LpCI F017 valves are included in the Brunswick Inservice (IST)
Program, which requires periodic verification of their operational readiness in accordance with the ASME Boiler and Pressure Vessel Code Section XI. The tests required are 1) local leakage rate tests to verify the seat tightness of the valves, and 2) stroke timing tests to verify the degree of degradation of the operating mechanism of the valves.
Results from local leakage rate tests are not relevant to the present failure problem; but stroke timing could provide sufficient indication to warrant inspection for degradation of the operator.
,
-
-
.
- -
-
-
-
-
_
. _. _
_
. _.
.
.
t I
.
The stroke timing tests were performed by the operating personnel at the l
control room without the benefit of local observation of the valve and operator (
l during the test.
Results of stroke timing tests were delivered to the IST l
group for evaluation to determine their acceptability including any required'
corrective actions. The results of stroke timing tests for the four valves during the past four years showed that:
1-E11-F017A
,
i
,
Opening stroke times have been sporadic, varying from 20.0 to 27.2
,
seconds.
.
L Closing stroke times have been fairly consistent with the exception of the
last recorded surveillance which indicated a stroke time of 27.6 seconds
on June 11, 1988.
The licensee is currently evaluating the stroke time
'
variance.
'
1-E11-F0178 Opening stroke times have been relatively consistent since 1984 as they
varied between 19.8 and 24 5 seconds.
Two exceptions were readings of i
26.5 seconds on December 29,1985, and 18.92 seconds on July 16, 1988, following the recent operator repairs. The latest stroke time was over 4 j
seconds les> than the previous stroke time.
The licensee is currently evaluating the variance of this last reading.
Closing stroke times also have been relatively consistent during the same
time period, varying between 20.9 seconds and 23.9 seconds with the i
exceptions of three earlier readings in 1984-1985 period ranging from 24.4 I
to 25 seconds.
These readings exceeded the IST maximum limit of 24 i
seconds in effect at that time.
l
!
!
l 2-E11-F017A i
Opening stroke times have been consistent, ranging between 19.6 seconds I
and 20.3 seconds, with the exception of one reading of 26 seconds obtained
.
on June 26, 1986.
!
Closing stroke times have been consistent, ranging between 22 seconds and
.
23.6 seconds.
2-E11-F017B
Opening stroke times have been consistent, ranging between 19 seconds and 20 seconds, with the exception of one reading of 25 seconds obtained on l
August 8, 1987.
- Closing stroke times have been consistent, ranging between 22.7 seconds and 25 seconds, with the exception of one reading of 19.5 seconds obtained on June 3, 198.
..
Results from stroke timing tests gave no indication of pending failure of the 1-E11-F017B valve. However, as discussed above, the licensee performed stroke timing tests from the control room without the benefit of local observation.
This minimized the effectiveness of the IST program.
If the failure was caused by gradual wearing, as suspected, a local observer would notice the unusual noise caused by the grinding (slipping) of clutch lugs, and early discovery could have been made.
E.2.c Maintenance Procedures for Limitorque Operators Generic Limitorque operator preventive maintenance procedures developed by the licensee were reviewed to determine adequacy.
The licensee has committed to accomplishing preventive maintenance every 18 months as recommended by Limitorque.
The following procedures are for "Q" (safety-related) operators:
MI-10-25 - Electrical inspection of torque and limit switches.
MI-10-511 A-F - Gear case lubrication and miscellaneous visual inspections of valve and operator exterior.
The electrical and mechanical preventive maintenance procedures listed above appear to be adequate for their intended purposes.
However, additional preventive maintenance procedures involving selected valve operators (based on maintenance history, results from surveillance tests or importance to safety)
did not exist.
Limitorque does not recommend any routine maintenance that would include disassembly of the operator unless an operational deficiency of some kind exists or is suspected.
Because of the failure history of the clutch mechanism assembly, Limitorque established a limit en the number of allowable cycles for switching from motor to manual drive and back of 1000 cycles.
Clutch mechanism degradation occurs due to the random nature of clutch assembly lug engagement and impact loading on the lugs. The random nature of clutch assembly lug engagement is caused by the following:
When the operator is in the manual mode, the sliding gear clutch is locked into rosition with the splined insert and is separated from the flexible jaw clutch housing.
When the motor is started, a cam on the flexible jaw clutch housing contacts the clutch trippers which release the clutch fork, allowing the sliding gear clutch to come into contact with the flexibe jaw clutch housing.
The lug engagement can vary from direct lug contact (lugs not synchronized) to full lug engagement. With time, the clutch lugs will wear to the point of intermittent engagement and eventual failure.
This condition is further aggravated if the sliding gear clutch is not absolutely free to move quickly on the hollow drive shaft and engage the flexible jaw clutch housing once the motor has been energized. Degradation of the clutch mechanism assembly could be diagnosed by having an operator stationed at the valve during inservice testing.
E.3 Corrective Action E.3.4 Unit No. 1 Valve 1-E11-F017B operator was repaired and passed the post-maintenance
.
.
testing.
Repairs consisted of clutch mechanism assembly replacement. While Unit No. I was shutdown, the licensee inspected valve 1-E11-F017A operator internals to determine if excessive wear existed.
Only normal wear was observed.
The licensee was performing a root cause analysis of the valve operator failure. The analysis was not complete at the end of the inspection.
Hardness testing was performed on the failed clutch parts to determine material adequacy. No defiencies were noted.
E.3.b Unit No. 2 The licensee had committed to opening and inspecting both 2-E11-F017A & B valves during the next outage, scheduled for July 22.
Particular attention will be given to the parts which failed in the 1-E11-F0178 valve operato*.
Additional corrective actions will be formulated based on the results of inspections completed on Unit No. 2.
E.4. Conclusion
,
The only other F017 valve experiencing operator problems (based on maintenance history) was 2-E11-F017B which had some internal gear parts j
replaced.
'
'
Valve 1-E11-F017B has experienced chronic packing leakage due to a galled stem. This condition has existed for over 6 years and is not scheduled for repair until the next outage.
- The original FSAR and the latest revision to the system description both require the F017 A & B valves to open and close within 24 seconds. This
,
'
limit was revised to 30 seconds in Amendment No. 4 to the Updated FSAR, dated June 2, 1986. However, a review of IST records showed that prior to June 2, 1986, the 24 second valve stroke limit was violated several times.
There are no ir.dications that the valves were declared inoperable. ASME Section XI required action was not completed, nor was any corrective action performed.
Inservice Testing of valve 1-E11-F0178 failed to predict valve degradation and ultimate failure of the clutch mechanism assembly.
The Limitorque operator is designed for a finite number of applications, and is subject to normal degradation and deterioration.
The licensee should consider a selective PM based on usage and maintenance history.
- During normal accomplishment of IST of notor operated valves, the licensee does not have a mechanic stationed at the valve to observe valve operation. The purpose of IST is to determine the readiness of pumps and valves for their intended function, and this individual is to listen for
'
any abnormal valve, motor or operator noises. Without the benefit of local verification, the IST loses most of its intended value, and becomes a formality. Misalignment of the clutch assembly may have been discovered
,
l
-
..
.-
. -
-.
.
..
J
'
J
.
l
31 l
t
!
'
earlier during testing and could have prevented the operator failure, The inspectors are aware of several licensees who use this approach for
-
inservice testing.
.;
i In summary, valve 1-E11-F0178 failed to open because of lack of engagement of j
the operator clutch assembly caused by deformation of the clutch lugs.
This l
event was allowed to happen because of 1) ineffective inservice testing, and 2)
'
,
lack of knowledge concerning maintenance and operational limitation of the operator. The largest single contributor to the failure was the ineffective
,
!
testing.
The purpose of performing inservice testing of valves is to be
assured that the valve is able to perform its safety-related function.
Stroke i
time readings within Technical Specifications simi+s is one indicator of valve
'
!
operability.
This alone is not sufficient, howev.r, shce there are other
!
{
signs and indicatioas associated with the testing V at show that the deteriori-
-
'
{
zation process has started, which cannot be realized without direct observa-
,
!
tion.
To meet the IST program minimum requirements only would reduce the l
effectiveness of the program. Corrective actiors proposed by the licensee must j
not be limited to the E11-F017 valves but should have a more generic J
implication.
f F.
Condenser Circulating Water
i i
F.1. Equipment Failure Description
.
As detailed in the Sequence of Events section of this report, multiple CWIP i
i trips occurred starting at approximately 0250 on Friday July 15, 1988.
Prior to the CWIP trips, the pumps were aligned as follow:
Unit 1 (Hot Shutdown)
Unit 2 (100*4 Power)
,
i
- CWIP 1C, 10-Operating CWIP 2A, 2B, 2C, 20-Operating i
i
- CWIP 1A, IB-Off L
The Unit 1 CWIP 18, 1C, 10, and Unit 2 CWIP 2A, 2B, 2C traveling screen panels l
j are equipped with fine men screen material with approximately one millimeter l
square openings, and resemble fine mesh porch screen. The CWIP 1A and CWIP 2D i
traveling screens are equipped with coarse mesh screen material with
approximately 3/8 inch square openings.
l
,
l The IB CWIP was started at 0112 on July 15, for the purpose of conducting a
{
routine flush of the debris filters installed in each inlet line to the l
condenser water boxes.
,
,
t At 0249 the IB CWIP was secured and approximately one minute later a h!gh D/P
trip was received on the IC CWIP traveling screen.
High traveling screen 0/P I
trips continued to be received at one minute intervals and the pumps tripped in
!
the following sequence:
10, 2A, 2B, 20.
Unit I was without CWIP for
)
l approximately one minute when the operators started the 1A CWIP at 0252
followed by starting 1B CWIP.
The 2D CWIP remained in operation for Unit 2.
Unit 2 power level was reduced to approximately 40*4 power to accommodate heat
i transfer capability of one CWIP cooling water flow through the condenser.
l
!
I
,
.
.
.
i
...
.
_
-
.
From 0302 to 1657 on July 15, several attempts were made to start Unit 2 CWIPs.
The pumps continued to trip ca high D/P across the traveling screens. Inspec-tion of the traveling screens at approximately 0400 did.not indicate the screens to be clogged.
Maintenance activities were formulated during a planning meeting held on the morning of July 15.
The maintenance activities were initiated at apprximately 1100. The tubing associated with the traveling screen D/P cells were checked to ensure they were. not clogged.
Air is continously bubbled through the tubes to assist in keeping debris out and the tubing was found to be clear.
Calibration checks of the alarm and trip set point for the Unit 2 D/P trip set point was found to be low at 24 inches of water in lieu of the required 36 inches.
Unit I high D/P trip set p; int was found to be satisfactory at 36 inches of water.
After removing several_ fine mesh screens from the traveling screen panels, the licensee _ was able to start the 28 CWIP.
Between 1615, July 15 and 0515, July 16, the fine mesh sc een removal was completed for CWIP 2A, 2B, 2C and 10. During their removal licensee personnel discovered that iery large quantities of (which is later identified to be)
"skeleton shrimp." All fine mesh screens were scraped and flushed to restore then to a clean status.
At 1900, on July 16,'Jnit 2 was returned to 100% power. Unit i remained in hot shutdown to affect repairs on various plant equipment, Operating status of the CWIPS and fine mesh screens after screen changeout is as follows:
U1 U2 CWIP 1A -operating-coarse screen CWIP 2A -operating-coarse screen 18 -operating-fine screen 20 -operating-coarse screen 1C off - fine screen 2C -operating-coarse screen 10 off - coarse screen 2D -operating-coarse screen F.2. Equipment History The inspector examined operator legs, plant procedures, and maintenance procedures and conducted interviews with plant operators, maintenance engineers and environmental specialist to determine the cause for the multiple CWIP trips and to assess licensee maintenance activities, modifications.and environmental activities.
During the removal of the fine mesh screens from the pump traveling screen panels on July 15, licensee personnel noticed large quantities of skeleton shrimp on the screens.
Due to the transparency of the skeleton shrimp they are not readily visible when wet, which explains why they were not detected by the operators when visual inspections were made while the traveling screens were in operation. The licensee indicated that scraping methods had to
be used to affect their removal.
lushing with water under pressure would not j
dislodge the creatures. Attachment 1 contains a sketch of the creature and is identified scientifically as a "Capre11a."
j
.
,
-
_, -.
- -, - -
_-y,,
, - - -.,.
-
.
%[
. -.
9'
.
The environmental specialist initiated sampling at approximately 1730 on July 15. The sample was taken over a five minute period using a large net with a sample bottle attached.
The sample showed a skeleton shrimp count of approximately 126,000. Typically previous 5 minute samples showed a count of approximately 232. The sample was taken on the discharge from the condensers.
.The size of the skeleton shrimp were noted to range in size from a few tenth of an inch to a half an inch.
The licensee environmental personnel worked with the University of North -
Carolina which has a specialist on its staff with expertise on the Caprella and its habits. Caprella do not thrive in water where salinity-is below 24 parts
~
per thousands (PPT). They require salinities above 30 PPT.
The plant intake structure is fed from the Cape Fear River.
Various creeks and the Atlantic Ocean provide water mixing into the Cape Fear River. The drought season has cause the salinity in the river to run between 32 and 34 PPT. Salinity reading.
recorded during the past several years generally show salinity to be above 26 PPT. Caprella are not good swimmers and normally attach to seaweed, the bottom and sides of the river and on growth attached to concrete.
Routine samples indicate a Capre11a count of 232-235 per 5 minute sample.
The inspector determined that the only operation identified taking place prior to the CWIP trips was the securing of the 1B CWIP.
In addition, walkdowns of the CWIP, traveling screen and the plant intake structures were made.
The service water system did not experience traveling screen clogging during the event primarily for two reasons. The service water traveling screens all have coarse mesh screens (3/8 inch mesh) and the water velocity is toward the much larger CWIPs which have much greater velocity than the service water pumps.
These stronger current flows toward the circulating water intake structure tends to keep debris away from the service water intake structure. The plant has experienced only infrequent multiple CWIP trips on both units, the most notable being that due to jellyfish in 1984.
The inspector reviewed the following plant procedures for technical content, sufficient detail and ci tri ty with respect to operator actions.
No discrepancies were identif ed.
Operating Procedure - 29, Circulating Water System
-
Operating Instruction - 03.4, Operator Daily System Check Sheets
-
Annunciator Procedure - UA-01, CW Screen Differential Pressure High
-
Operating Procedure - 29.1, Screen Wash System
-
The inspector reviewed maintenance activities associated with the CWIPs, intake structure traveling screens and D/P 1 strumentation associated with the traveling screens.
Event 4 <e Maintenance instruction, OPM-LUB500 schedules at various intervals n..pection and lubrication requirements for the CWIPs motor and shaft bearings, l
traveling screen link chain joints, bearing and other moving parts. A detail schedule is presently being planned where refurbishing of the traveling screens I
as a unit will be conducted on a two year cycle. Maintenance Instruction, MI-03-3A and instrument routing schedules are used to calibrate instrumentation installed in the circulating water system at specified frequencies.
The
'
_
._,
_ __ _
....
,,, _ _ _
- -,, - _. _ _,, _ _ _ _. _.
__
~,..
.---, _
.
.
.
i
'
,
.
licensee noted as a result of the calibration checks conducted on Unit 2 after the CWIPs trips, that traveling screen high D/P setpoints were set low at 24 inches of water in lieu of 36 inches. A review of plant modifications showed that modificat$on 81-224A-D (approved February 1982) was outstanding on Unit 2 with regards to upgrading the traveling screen D/P pressure switches to a 36 inch trip setpoint. The licer.see did not provide any specific reason for this modification remaining open.
In addition, calibration frequency for the D/P instruments were assigned and placed into the instrument routing = schedule program.
F.3. Corrective Action The environmental group has increased its sample frequency from weekly to daily and increased the number of locations where samples are taken.
In addition, a i
rig is being devised to scrape and sample the bottom and sides of the intake canal, intake structure, and river.
Engineering and operation groups are trending traveling screen D/P readings and changes that occur during routine operations of the circulating water system.
F.4. Unit 2 Effect The loss of three out of four CWIP required an immediase reduction of reactor power and steam flow to the unit condenser to compensate for the decrease in condenser vacuum.
The reduction of plant power to approximately 40% was accomplished in a two to thice minute time peri-Condenser vacuum changes were minimal, decreasing from 28" Hg to apprt-stely 26.5" Hg and were returned to normal values once power was reduced.
F.5. Conclusion Based on the finding as discussed in this report the CWIP were tripped on high D/P across the traveling screens due to massive clogging of the fine mesh screen for those pumps so equipped.
In all cases where coarse mesh screens were used, the CWIPs remained in operation.
In addition, af ter maintenance
,
personnel become directly involved, the time that thcy took to troubleshoot the high screen D/P CWIP trips, affect modifications, calibrations and the subse-
'
quent removal of the fine mesh screens (approximately 200 panels per traveling screen) from four CWIP appears reasonable and effective.
j
,
The effects of not having plant modification 81-224A-D completed on Unit 2 did not appear to be adverse, however modifications should not be allowed to remain oper, for sucn a long period of time without a formal re-examination and manage-ment approval.
The National Pollutant Discharge Elimination Permit (NPOEP) that the licensee has with the State will require that the fine mesh screens be reinstalled on three of the four CWIP for both units. Since this tripping of the CWIP could potentially reoccur at a future date, the inspectors believe the licensee should make final assessment in this area and develop the following:
Predictability of the skeleton shrimp to return in massive numbers.
-
.
a
.
-
Provide information concerning this event to plant operators and establish guidance as a future reference in a form deemed appropriate.
G.
Equipment Safety Significance and Operability G.I. Safety Significance / Safety System Performance G.1.a.
During the shutdown of Unit 1 the operators bypassed the RWM functions. The RWM control system is not required for safety.
The RWM function does not interfere with normal reactor operation, and in the event of failure does not, itself, cause unanalyzed rod patterns to be established. The RWM functions may be bypassed and its block tunction disabled only by specific procedural control initiated by the operator.
Specifically, the RWM shall be operable when thermal power is less than 20% of rated thermal power. In the event RWM.is not operable at less than 20% power, Technical Specifications allows continued control rod movement provided that a second licensed operator or other quali-
. f ed member of the technical staf f is present at the reactor control cont le and verifies compliance with the prescribed control rod pattern.
Through interviews of personnel and review of operating logs the team concluded that-this requirement was met.
The system itself did not interfere with normal operation, nor cause unanalyzed rod patterns.
G.1.b.
Rod Sequence Control System (RSCS)
RSCS prevented the eperator from selecting control rods for movement while completing a controlled shutdown.
The system is required for safety.
The function of RSCS is to prevent the operator from moving a control rod out of sequence.
At the time the operators were prevented from selecting control
,
rods, the reactor was being operated in the "50 percent rod density to the Preset Power Level" range.
In this operating range, the RSCS is interlocked to
allow only one specific control rod sequence to be manipulated; i.e., either the A or B sequence of control rods. The circuitry either inhibits or-permits t
all of the rods assigned to a sequence unit to be moved.
In this case only sequence 8 rods could be moved, and sequence A rods must remain in the full-out position. Due to the failure of the full-out reed switch on control rod 22-03 (an "A" sequence control rod), the circuitry was not satisfied and a select block applied to the Reactor Manual Control System (RMCS). This action by RSCS was appropriate and the system rssponded correctly by preventing the operator from moving any control rods. Thus no failures of the RSCS occurred. The reed switch failure had minimal safety consequences.
.
G.1.c.
Rod Select 5 witch
'
The Rod Select Switch allows the operator to select a control rod for movement assuming all prerequisites are satisfied.
The switch is part of the Reactor Manual Control System which is not safety related. The failure of the select switch for control rod 34-03 prevented the operator from selecting and moving this control rod by normal means.
The RMCS does not include any of the circuitry or devices used to automatically or manually scram the reactor; this
- -
-
.
.
means of inserting the control rod was still available and not impeded by the failure.
This event had minimal safety consequences, nor were procedure controls violated.
G.1.d.
RHR Injection Valve F017B The failure of RHR injection valve F0178 prevented the operators from using loop B RHR in the shutdown cooling /LPCI modes.
The shutdown cooling and LPCI mode of RHR are safe shutdown functions. Alternate methods of shutdown cooling included using the main condenser, or loop A of RHR.
Operations personnel elected to use the main condenser as the heat sink allowing loop A of RHR to serve as the LPCI path.
For most single failures which could result in the loss of shutdown cooling, no unique safety actions are req 11 red; in these cases, shutdown cooling is simply reestablished using other, normal shutdown cooling equipment. This configuration, by itself, posed minimal challenge to the safety status of plant shutdown cooling modes.
The ability to reach a cold shutdown condition remained.
No Technical Soecifications were violated.
G.1.e CWIP Trips While Unit I was in Hot Shutdown and Unit 2 operating in Condition 1, a loss of several CWIPs occurred.
The Condenser Circulating Water system is not safety related. The consequence of a loss of CWIPs is essentially a inss of vacuum transient or in a more severe case, a loss of auxiliary power. The reduction or loss of vacuum in th: main turbine condenser will trip the main and feed-water turbines and close the main steam line isolation valves and bypass valves. The resulting transient is dependent upon t'e rate at which the vacuum is lost and the operating condition of the reactor.
For Unit 1, in
't Shutdown, the condenser was being used as the ultimate heat sink for decay heat removal. Loss of this heat sink would require alignment of the A loop of RHR for shutdown cooling (the B loop was inoperative with F017B out of service).
At the time of the CWIPs tripping, the A loop of shutdawn cooling had not been prepared for service.
This evolution would have taken considerable time and issulted in heat up of the reactor due to the loss of decay heat removal.
The operators were, however, able to maintain a minimum amount of Circulating Water flov 'a the condenser thereby preventing this condition and challenge to the Hot Shutdown status of the reactor.
For Unit 2 the failure was of a more immediate concern. The rate of vacuum loss was controllable by initiating a turbine load reduction.
Recirculation flow was decreased to lower the load of the turbine to within the capacity of the remaining CWIP.
FSAR analysis shows that an instantaneous loss of all CWIPs causes condenser vacuum to drop to the turbine trip setting in six, seconds.
The partial loss of CWIPs which occurred on Unit 2 does not approach the severity of the instantaneous loss of all CWIPs, thus this event was bounded by the safety analysis.
The combination, though, of inoperable CWIPs with a degraded shutdown cooling capability and with HPCI inoperable, did place the unit in a condition where additional circulating water and shutdown cooling
problems could have resulted in significant challenges to the operating staff.
'
l
!
..
r
-
.
G.2. Operability and Reportability The AIT evaluated the events of.the shutdown of Unit I for applicablity.of 10 CFR 50.72 and 50.73 reports. The licensee has made' the following reports'
i associated with the events leading to and. involving the ' shutdown of Unit 1 on July 13, 1988:
-
A 10 CFR 50.72 report to the NRC on July -1,1988, (0308 hrs.) concerning
'
inoperability of HPCI due to motor failure of HPCI E41-F001 Steam Supply Valve. A written report will be submitted to the NRC (LER-1-88-17),
A 10 CFR 50.72 report to the NRC on July 5, 1988, (1313 hrs.) concerning
-
HPCI inoperability due to motor failure of HPCI E41-F006 Injection valve.
.
A written report will be submitted to the NRC (LER-1-88-19).
These reports were reported under the conditions of 10 CFR 50.72.b.2.iii. No other reports to the NRC were inititated (or required) by the licensee for the events which occurred during the shutdown of Unit 1.
The 10 CFR 50.72 reports were initiated in a timely manner.
No difficulties were noted in the communciation of these events to the NRC.
i G.3. Conclusion The shutdown of Unit I was complicated by numerous failures of plant systems j
that required extensive maintenance and operator expertise to overcome.
The operational considerations associated with these failures' were significant in i
that the operators were required to mitigate the effects of multiple malfunc-tions; e.g., HPCI being inoparative during a shutdown, an inability to esta-blish normal shutdown cooling when required, failure of the RWM and its backup RSCS, using the condenser for decay heat removal and a subsequent loss of CWIPs. Multiple failures can perturb the situation to an extent that normal evolutions become quite complex with a resulting increased challenge to the operating staff.
In this event the failures did not prevent the safe shutdown of the facility and did not violate the safety analysis. No safety system injection or actua-tion occurred during the planned shutdown of the reactor, nor were safety limits violated. The effects of the CWIP trip on Unit 2 were within the bounds of the FSAR safety analysis for loss of condenser vacuum and loss of auxiliary power.
The potential existed though for serious degradation of the Hot Shutdown status of the reactor. The loss of CWIPs while using the main condenser as the decay heat removal source coupled with the failure of F017B created a condition which mandated expeditious and effective diagnosis and repair to avoid possible additional challenges to the operators.
- -
-
-
-
-
-
-
-
_.
.
o
IV.
PERSONNEL ACTIONS A.
Operations Involvement During the shutdown of Unit 1 on July 13, 1988, operations personnel responded to numerous unexpected failures for which deliberate action was net.essary to continue with a controlled shutdown. Their goal was to' safely shuti.own Unit 1 prior to the expiration of the Limiting Condition for Operation for HPCI that expired at 1102 on July 14.
The actions of operations personnel during this evolution were reviewed by the AIT. The review concentrated on a performanced based evaluation of their responses to five failures of plant systems: 1) RWM, 2) RSCS, 3) Rod Select Switch, 4) RHR F0178, and 5) CWIP trips. In addition, the response of Unit 2 operations personnel to the CWIP trip was also reviewed.
The AIT considered several areas of performance in the review:
Diagnosis of Events / Conditions Understanding of Plant / System Response Response to Alarms / Failures Manipulation of Controls Use of Procedures and Technical Specifications Supervision Communications Crew Interactions A.1. Rod Worth Minimizer During the snutdown the RWM exhibited improper performance by enforcing control rod blocks at 40% power.
This was indicated by the control rod withdrawl permissive light extinguishing and indications / alarms on the RWM.
The failure was immediately identified and appropriate action was taken to defeat the RWM.
A second licensed operator was used to verify control rod movement in accordance with procedures.
Historically, the RWM has been a problem for the operators, the latest event occurring on June 8, 1988, when a similar failure occurred. Reoccurring failurts of thir system have led the operators to anticipate the necessity for bypassing its function and continuing with control rod movements.
CP&L has replaced this system on Unit 2 and intends to replace the Unit 1 RWM during the next refueling outage. The operator response to this event was both timely and proper.
A.2. Rod Sequence Control System While continuing with the unit shutdown RSCS initiated and blocked the selection of centrol rods for movement.
The problem was not imcediately obvious to the operators primarily due to the lack of diagnostic aids provided for RSCS. The SR0 was aware of an existing tri uble ticket associated wi'.h the Rod Position Indication System (RPIS), however, he did not correlate this with the interface of RPIS and RSCS for full out indication.
The operators continued to diagnose the problem by checking for full out position indication
-
-
-
._
_
_
.
.
on the full core matrix and plant process computer.
Maintenance technicians were dispatched to trouble shoot.
Approximately one hour into the trouble shooting efforts, they suggested the possibility of _ a full out indication problem.
Miscommunication between the control operator and SRO may have prolonged maintenance activities. The control operator apparently assumed the-SR0 was aware of the problem with control rod 22-03 and did not reiterate the problem with RPIS on this particular control rod.
The time expended by 'the Instrument and Control personnel to isolate the problem added to the delay of the shutdown.
Ultimately, the delays necessitated a manual scram of the reactor to ensure shutdown and cooldown could be accomplished prior to the LCO deadline.
The Shift Operating Supervisor had notified the on-call manager (Manager of Operations) and had discussed contingencies with respect to the cooldown.
It was decided that if RSCS was not repaired by 0330 hrs. the unit would be manually scrammed.
Concurrence was obtained from the plant manager.
RSCS was restored prior to 0330 hrs. and normal control rod insertion continued without difficulty.
No apparent training deficiencies are noted in this event.
The operators were aware of both the failed reed switch and the full out interlock, the lack of correlation of these two appears to have been a cognitive oversight by the operators. The inspectar discussed the feasibility of maintaining failed reed switches in the "bypassed" position during normal operations with operations management. Operations is reconsidering the feasibility of maintaining the switches in bypass, but as of now no changes in policy have occurred. Several problems could result from this manipulation, including masking of additional
,
"bypassed" switches, and a continuous annunciation of this condition on the control board.
A.3. Rod Select Switch After RSCS was returned to service at 0250, selection of control red 34-03 was prevented by a failed rod select switch. Several techniques were attempted to allow selection of the control rod; e.g.,
ensuring deselection of all other rods, holding in the selection pushbutton and attempting to move rod 34-03.
The operators ascertained that the rod select pushbutton for rod 34-03 had
.
failed and maintenance was initiated. No further control rod movement was I
initiated; doing so may have resulted in abnormal control rod patterns.
This I
type of failure is recurring and random in nature. Only minimal preventive i
maintenance can be conducted on the subject switches.
These switches are
bardwired and CP&L has considered upgrading the rod select matrix to a newer i
model which would allow quick replacement of failed switches. Initially some
'
confusion may have existed with the operating crew on the cause of the inability to select the control rod, however, t'.e overall operator response to this event was appropriate.
A.4. Loss of Shutdown Cooling The reactor was eventually scrammed and cooldown to the point where RHR I
shutdown cooling was to be initiated. While aligning RHR for this mode the control operator attempted to open the RHR injection valve F017B and the valve failed to open.
Flow had not been observed by the control operator and the valve indicated closed af+.er 20 seconds of operation. After two unsuccessful
.
.
40.
attempts to open the valve the pump was secured. The open signal was causing damage to the limitorque actuator, however, the operator ce;1d not be aware of this condition from the contral rtom. An auxiliary operater did identify the malfunction of the valve and notified the control operator of a grinding noise.
After manual manipulation o: the valve and resetting the tnermal overloads by the auxiliary operator the control operator attempted to open the valve four more times. Further investigation by the auxiliary operator reported the valve motor very hot to the tauch.
Repeated attempts to operate the valve were unsuccessful. This actian contributed to serious degradation to _the actuator of RHR F0179.
During this evolution the 9HR pump operates without minimum flow protection.
Prompt shutdown of the RHR pump is necessary to prevent pump damage due to operation under "dead-head ' conditions.
The control operator secured the RHR pump in a sufficient amount of time to preclude any damage to the pump itself.
Subsequent to the loss of RHR loop B for LPCI and shutdown cooling the unit remained on the condenser for decay heat removal.
i.oop A of RHR was dedicated to the LPCI injection path in accordance with the Te:bnical Specification requirement for one coerable loop of LPCI while in Hot Standby. In the event the main condenser became unavailable, operations would have aligned Loop A RHR i
into the shutdown cooling mode of operation and entered the action statement l
for LPCI.
The diagnosis of the event by operations was accurate and their response to the loss of one loop of shutdown cooling was prudent. Mani*culation of controls for the F017B should have been more conservative in that continued attempts to operate the valve, after diagnosis by the auxiliary operator, may have contributed to the degradation of F017B.
A.S. Circulating Water Screens Both Unit 1 and Unit 2 operations personnel had to respond '- the multiple loss of the CWIPs. Annunciations were received identifying the
.use of the pumps trippin9; however, the root cause was less obvious.
The operators considered that low tide had somehow contributed to the pumps tripping.
Their' initial decisiuns were effected by a similar event which had occurred previously. The
'
previous event occurred approximately at low tide. Being within two hours from low tide and the observation by an auxillary operator that the screens appeared clean led the operators to believe low tide was again a factor. Contributing to this belief was the fact that repeated attempts to start fine mesh screens were un:urcessful during the hours immediately preceding and following low tide. Not until well past the next high tide and after removal of several screen sections, was it determined that screen plugging was the root cause.
Due to the innocuous nature of the failure, diagnosis of this event was extremely difficult.
Its impact was mitigated by t'ie operator's ability to maintain in operation those PWIPs with course mesh 3creens.
-
- -
- - -
.
.
The Unit 2 operators responded to the event by reducing recirculation flow, avoiding the region of instability for core power / flow.
Reduction in power enabled the unit to remain within the capacity of the running CWIP.
The control operator took action to mitigate the loss by shifting the Steam Jet Air Ejector to full load thereby assisting in. maintaining condenser vacuum.
In addition, the rapid reduction in recirculation flow initiated a feedwater transient which prompted the operator to take manual control of 2B Reactor i
Feedwater Pump. The operators continued their efforts to restore the CWIPs by attempting multiple restarts.
Trouble shooting efforts by maintenance
'
personnel was initiated to assist in restoration of the CWIP.
j i
The operators on both units accurately diagnosed the event and understood the effects of the pumps tripping. Timely action was taken on Unit 2 to respond to the loss.
Recognizing the degraded shutdown cooling condition, operators should have requested that maintenance personnel troubleshot the CWIPs while waiting for high tide conditions, instead of relying solely on the t1aai effect posssibility. Although an inordinate amount of time was expended in restora-tion of the CWIPs, ultimately, effective trouble shooting stratagies were used
- o locate and repair the system. In the review of this event no violation of procedures was noted, nor any training deficiencies.
The operators on both units responded satisfactorily to the event.
j A.6. Conclusion Operator dependence on rel bole equipment is crucial to their safe performance.
Many of the failures d.aring this event were associated with known or reoccurring equipment malfunctions.
Management must be aggressive in the correction of these cypes of deficiencies to ensure operator performance is not j
hampered by knowr. and/or recurring equipment malfunctions.
The operators had
'
to contend witn ar. inordinate number of malfunctions during this shutdown.
Overall, the operttion department's performance on these events was satisfactory.
Several areas where the need for improvement is indicated are:
1) Status of failed full-in/out RPIS switches (operations has no policy on the permanent status of these switches; i.e.,
should the bypass switches remain
,
bypassed or returned to normal), 2) Operator awareness of indications for failed rod select switches (diagnosis of Reactor Manual Control System
,
failures), 3) Operator awareness of the consequences of motor operatea valve failure; e.g., operating a valve that responds abnormally.
Improvement is also needed in the area of response to equipment failures.
Plant and operations management must improve their resolve to ensure equipment
'
malfunctions are adequately responded to in a timely manner. As evidenced in the investigation of the RSCS and CWIP functions, the burden of resolution to these problems rested with the operations staf f.
Operations was required to either operate around, or otherwise experience long delays in meeting their expected goal of shutdown by 1102, on July 14.
This situation places additional stress en the operators that could b: alleviated by aggressive attention to equipment problems by plant managemen.
.
B.
Management Involvement /0versight on F0178 Valve Failure Interviews were conducted with senior, mid-level management, supervisory and engineering and maintenance personnel involved in repair of the motor operated valve (F017B) which had failed to open at 1750 on July 14.
The purpose of these interviews was to ascertain the degree of management involvement in the oversight and the decision making process associated with the activities in the evaluation, repair and post-maintenance testing of the valve.
B.1. Senior Management The Vice President, Brunswick Nuclear Project, was briefed by the Plant General Manager on July 15.
During mid-morning, the Maintenance Manager and. support engineers showed him the damaged parts and with the aid of drawings explained how the declutching mechanism works.
Also discussed were actions to be taken for the three other similar valves onsite. Another meeting at about 1600, on July 15 with cognizant engineers and their supervision dealt with inspection of other gearing in the actuator for damage. He emphasized that all damage-had to be identified and corrected and also asked if outside expertise from other CP&L organizations or Limitorque were required.
The purpose was to ensure that people were not taking short cuts and resources were available.
On the morning of July 16, the valve problen was discussed with the Senior Executive Vice President during a routine plant status call.
On July 16 and 17, the Unit i F017B and A were also included in telephone conversations with the Senior Vice President, Nuclear Genert. tion and the Senior Executive Vice President.
'
The Plant Manager was informed of the valve problem on a July 15 morning briefing. During this briefing, he inspected the damaged parts and indicated he listened attentively to the dialogue between the Manager of Maintenance, maintenance and engineering personnel to insure that the right questions were being asked, concerns were being enveloped and that actions were being undertaken to resolve valve concerns and that activities were under way to investigate the root cause of the problem.
The Plant Manager : stated that he issued no specific directives since he had quite a few discussions with the Manager of Maintenance and he felt satisfied with the direction and the manner in which the Manager of Maintenance was vigorously pursuing the resolution of the valve issue.
B.2. Mid-Level Management i
The Manager of Maintenance was notified of the F0378 problem when he contacted
'
the plant for a status at approximately 2130, en July 14.
He had been in Washington D.C. earlier that day to discuss 9alve problems with the NRC.
Earlier attempts to contact the plant hao been unsuccessful because telephone lines into the Southport area had been cut by county workers.
During the conversation he verified tha+ the right people were involved.
Normally he is briefed every mo,'ning by his principal sub;rdinates at 0730 on the status of major work efforts in progress. However because of the significance of having another safety related valve failure, he was briefed on its status upon arrival onsite, before 0700, on July
'5, by the maintenance support staff supervisor.
..
.-_
..-
- -.
_
_,
- - -
.-
.
_.
_ _ _ _ _ _ _ - _ _ _ - _ _ _ _ _
.
.
He indicated that he was basically in the receive mode versus the direct mode; however, he did instruct his staff to be sure to adequately. document the inspection results and activities associated with the inspection of the F0178 valve.
He was informed tha t MP-57, Limitorque Valve Failure Analysis and Troubleshooting Procedure,.as being implemented. -Immediately prior to the 1130 PNSC meeting, he was briefed on the current status by the maintenance support engineer and hi; associates and supervisors involved in the F017B problem.
He was shown the damaged parts.
They discussed potential failure modes, availability of spare parts and emount of disassembly and inspection required of the actua'cor and valve.
From this meeting, it was decided that the 1-E11-F017A and similar unit 2 valves be looked at.
Also the ~ Manager of Maintenance directed that technical support management concurrence be sought on the technical support engineer's position concerning the reuse of a spring.
In general on July 15 and 16, he was continuously and informally updated of the valve status as new information became available.
Other specific actions associated with the overview of the F0178 problem included briefings of the Plant General Manager on the morning of July 15 and-periodically thereaf ter; discussion of the issue at the 0815 management meeting and the 1130 PNSC meeting on July 15; and tour of the reactor building on July
,
16, specifically to ensure that personnel were at the F017B when it was being stroked and tested after reassembly.
In addition, the: inspector observed the Maintenance Manager witnessing the disassembly and initial inspection of the declutching mechanism on the Unit 1 F017A.
B.3. First-Line Supervision q
An engineering supervisor was in the control room when this incident occurred.
Maintenance personnel were informed of the valve problem at approximately 1900, or one hour after the valve failed to open.
Activity continued during the night involving preparation for and the actual partial disassembly of the valve actuator to identify damaged components.
On July 15, a maintenance valve specialist and a technical support engineer were assigned to resolve this issue. These individuals interfaced well in evaluating the most probable cause of failure, determining what valve parts required replacement, discussing issues with control operators and the auxiliary operator who observed the malfunction of the valve as well as reading the control room and auxiliary operator's write-up of this event. A Limitorque representative was contacted to discuss possible or probable failure scenarios.
Mechanical and electrical maintenance craftsmen were briefed on the details of the job prior to work.
The inspectors observed that foremen witnessed performance of the craft work.
In addition, major evolutions were also observed by the maintenance support engineer and/or by the technical support engineer.
B.4. Decision Process The troubleshooting and repair decision making process was determined by the inspectors to be a group interactive process of cognitive members.
Members involved included mechanical support personnel, mechanical maintenance personne', I&C personnel and their supervision up to and including the Manager
-
-
-
.
-
..
.
-
-.
.
44 of Maintenance.
In addition, engineering support'was provided by Technical Support. Decisions tende1 to be shared among groups of personnel and usually _
evolved from discussions among two or more levels of management with their cognizant subordinates. Th?s sharing of information made it very difficult for the inspectors. to determine what levels of management made the many decisions on actions taken during the re:overy from this failure. However, the ' approach worked effectively in this it. stance. becaase, of the experience level and personal working relationships developed by the "team members" gained from addressing similar types of valve / actuator problems ~ in the past.
The inspectors recognized that the NRC and CP&L sensitivity to this ftilure necessitated the involvement of all levels of site management in the details of this particular probicm. Using this type of management style, routinely, would tend to shift the technical-decisions responsibility and accountability to the senior manager involved.
This approach is effective when management has technical expertise and experience in the technical area of the problem but could be a pitfall if there is weakness in the area.
For example,' management may be mostly mechanical, hence could function well for mechanical. types of
,
'
problems but could be ineffective for electrical type problems.
Determination of the overall effectiveness of maintenance management in other than the scope involved with the F0178 valve problem was not evaluated by this inspection.
B.S. Cy clus-Failure of FL.8 was immediately recognized by management as a significant event, as another safety related valve f ailure and as 'a critical path activity for the outage.
There was an appropriate level of senior management j
participation in the resolution of this issue with strong participation by q
mid-level management.
While informal, there was very good and real time coordination between the maintenance and technical support staff who also exhibited careful deliberation and attention to detail in addressing this issue.
V.
OVERALL CONCLUSION Although the shutdown of Urit i from July 13 through July 15 was a difficult abnormal evolution, the overall effort was handled in a satisf actory, effective manner. With respect to the specific equipment failures, the full out reed
switch problem affecting RSCS could have been identified sooner through better operator awareness of system interaction and maintenance work.
The actual i
troubleshooting and repair was conducted ir; an appropriate manner.
As the
'
F0178 failure was such a highly unusual failure and on the critical path for Unit I restart, it received an extensive amount of technical and management attention.
The effort was both aggressive and effective in identifying the
,
'
deficiency and making the necessary corrective actions.
Even so, a more purposeful IST effort on this valve may have identified the degradation earlier. The Unit 2 transient, caused by the skeleton shrimp in the circulat-ing water system traveling screens, was handled quite well by the operators.
,
'
The time frame, though, between the first pump trip and the recognition that the screens were clogged appears excessive in light of Unit 2 being at 40*;
power and Unit 1, steaming on the condenser, with a degraded shutdown cooling capability.
. - ~
.
_
_.
.
Also, the HPCI DC Motor operated valve issue, which precipitated the shutdown appears to be a problem that should have been identified and-corrected by the licencee sooner. These equipment failures reflect a continuing trend that not only places a heavy burden on the maintenance staff but also appears to send a message from Plant and Operations management to the operators that they are expected to safely operate the plant with less than fully reliable equipment.
VI.
EXIT A formal exit for the AIT has been schedule for August 18, 1988, at the CP&L Company office in Raleigh, North Carolina. Results of the formal exit will be dccutented in the Meeting Summary, issued subsequent to the meeting.
Two debriefs on the AIT issues were conducted with licensee personnel at the Brunswick site on July 21, and 22.
Althouah the AIT departed the site on July 22, several members continued to review licensee documents in-office.
This effort continued through July 29.
Based.on discussion with the licensee on July 18, 1988, and an initial review-from the AIT, the NRC issued a Confirmation of NRC Concurrence to Restart
- Brunswick Unit 1 letter on July 25, 1988. This letter expressed some reserva-tions over the extent of management oversight at the site but did concur with the restart of Unit 1 with continued NRC 24 hour coverage.
On July 29, CP&L sent a letter to the NRC providing a stav.is of actions to date, short-term actions in process and plans for long-term actions.
l l
l l
L
_ _ _. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _ _ _ _ _ _. _. _ _ _ _ _. _ _ _ _ _ _ _ _
_
__
_ _. _
__
_ _ _ _ _ _ _.. _ _ _. _ _ _
-- -_
.
'
l
'
.,
.
ATTACHMENT 1 CONDENSER CIRCULATING WATER S'fSTEM Page 1 of 3
.
I t
,
!
o OISCMaAQE *uMNEL l
t 4 a
l
a a
4 i
i
.
.
g
- t.
vie
@, =l C[o.UT.',=l 2
'..../
d
@ =l 'niD = @
i
"
<W W
$,, h
.........
,
,
,
,
a
-.....
....
_.....
v,..
._.
.........
IIT Tl l FCr Pf l
@
W d
j
,....
.....
i....
_...._
l. L
__
,
LL CONDENSER 28 CCNCENGER 2 A TT T "'
l..,. g'
'g r.
en g
' g r.
l
" :; ) 2.
"
{
Lc.;L..
..L Ld,,z. ? " 2.o )..L
.
.
..
-
v.
e
.
@
.
.
.
e o..........
- .,,C..... e,_@ + e C.......>v.... C..... i A-9
..
,
,
, _ s... L v n,.
Tr,.. # v
'
f
_
vi I
-
N...
N.., --
H.N
A N...
i
=
a
__
.
a a
_
_
_-
e..u......
s P
-
t nacawaa# zoms
t eacawsen zone l
.
..
.,
.,
i.1
.
.
ee.e see 1,
.,
ve vP v.
ve
[
osvamfisse zome ses sit
,,
a
-}
o
,,
a
,,
a
,,
see
"w'! ND o
n,-....
vna was
"
yet vit
"
..
"
.:1
..
-
..
vs vs
.
v.
vi
/
b
/\\
INTAKE STRUCTURE
[.
A f W
LNTAK E C Amal.
- .e
_
-P-
,
. _..
,. - -
.
- _ _.. - _ _ - -, _ ~ -
.. - - - - -.
.
!
,
.
ATTACHMENT 1 TYPICAL TRAVELING SCREEN Page 2 of 3
.
DRIVE ls0 TOR
>
d%
d SCREEN 1 ASH
>
$ PRAY PIPE DalyEN $PROCKET I't>
DI*
>
'
[
>
I
'
SCREEN PANELS %
>
-.
..
,
SCREEN CHAIN
-
)
,
-
e l
f'
<
x
CulCE $PROCKET f
l lI
,
\\
BOOT
[ SECTION
-
.
...
$.'
,
,
.
ATTAC1DIENT 1 SKETCH OF SKELETON SHRIMP-Page 3 of 3
.(
<
.
'
a
/
.6 el
'
gU iv (
A
=
%?
,".
N e
/
$
/
T
)
c w
E F
O
'
I J
K t
b M
N O
P Fig. 21.38. Crustacea, Arr phipoda, Capeellidea. A. A<giatIls sp., head region and detail of mandible and palp; helle op'.', whole animal; C, Argiartle spaese, abdomen; D, Aeginine leagicwah, abdomen; E, FAtisite marias, leg from segment Ill; F, luenecie incerte, segment V; G, Peterepelle unan, segment V; H, if<mie<gine minice, thoracic aegment la dorut view; I, Capelle paeatis, esphalon; j, same, part of leg.from segment V; K, CedcII, eatreet, part ofleg from segnxnt V; L, CapcIIe esics (left) and Cepelle eguih6re (right), part ofleg from segment V oc VI; Af, CepcIIe afvihere, insertion of gnathopod !!;
N, Cepelle srpratrienehs, detail of gnathopod II; O, same, gnathopod II: P. CaptI!.
lia<eris, gnathopod !!. Abbeeviations are as follow: a, antennae; b, basis; e, cephalon; gl, gnathopod I; gII, gnathopod ll; d, gill; e, abdornen; f, manilliped; leg bearing seg-ments (perconites) numbered 1-Vil.
l
.
.
b
%
.
I
.. : ;, x n...,.
,~2e
.
I.._ _ j
.
,
. *........
-
_,
.
r Page 1 of.3 RHR SYSTEM (SHUTDOWN COOLING MODE)
ATTAcerENT 2 mi..
... s.--
--< -
... ___.
_.
. _
-
lt..s. g.e.4 -
'T
aw.........
f n;
f...
.
.............
,,. g
_
._
- J-.
.
..
.
-
a_.-
.m
..y
'....
..
.-
>u
s
.,7
,
-
.
.
n
[
.
'j~-
'
w
-
=,
.
,
_ _ _ _ -
-
8E
-....
..
~7: J1 -
-
- ~7 hf.
,
-
"
.}
~
-
.
-
$--@
-
.
mt 2h *
E
's
--
.
a rr
lM *
.f.~
'
=
F017B l
'
~
l
- =- w - -
- * * *
-
-.,
- "(==='
,. _ _ _. _
-~
-
-
i
e r
l
_.
l A
-
i -
..
~
'
(&
li L g-
.mx
_
..
-
.
i
- .
__
--
i
In
4 M
lk
i
...
,r
--
-
- ::
~
L~ '
I m
.
l
,
,
ji
_.
il U 5 fi h_. 6 l-'
tfi t J... M
=
_i
. _ _. _
_.
..,..
"
ifif$ifil f7
'
-x
-
=_._
.
y*. g. _.
...
,, -
_)
-
,
...
,
t
.
.
i
~
_ -. _ _ _. - - _ _, _ _ _
.
. m
..
.
.
.
,_.
s
.
ATTACHMENT 2 F017B Valve Actuator Page 2 of 3
'N%
3.
y te
4.
f Y
i.
t,,
'
'O a
!! (k
~~
,
d
"
.
{b? h
)
f,
'..
'. $,<
L
..
1. Flexible Jaw Clutch Housing
- I',
4,,7* g,
,, 43, 2. Sliding Gear Clutch
,
,
..
,
.
3. Splined Insert
'
'.,, ~.., ~.;;_
,
.
.,,
..
4. Clutch Compression
'
.
J, 4 j,' a Spring
-
.s,
,
,. *[ '.' '
j
'
,
) ;GdN$
I
'
,
Damaged Mating Lug y,
,
-
-
,
_y--,m
...,.e.
- - - - -
- - - " " " " " " " - - ^
' ^
~ ~ ^^
... _ _ _.
. _ _
. _ _
_
_ _ _. - _ _
A-
'
.
,
ATTAC1DIENT 2 F017B Valve ~ Actuator Page 3 of 3
!
~-
kn i
~ ~ ~
'
.r,.
>3
~-
.w
-
.
.-
.
- /...
-
.
.
\\\\\\\\\\\\\\\\\\\\
A
\\\\\\ \\\\\\\\
g gg h i( h IS Danaged Gear,ceth
'
.
,
'
._
.. t..
~.
f i
.
- l
-.. -
...
-