IR 05000324/1988027
| ML20151Z760 | |
| Person / Time | |
|---|---|
| Site: | Brunswick  |
| Issue date: | 08/17/1988 |
| From: | Fredrickson P NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML20151Z754 | List: |
| References | |
| 50-324-88-27, 50-325-88-27, NUDOCS 8808300128 | |
| Download: ML20151Z760 (54) | |
Text
'
w -
1 .
ii -
-l TABLE 0. CONTENTS ,
!
PAGE N l INTRODUCTION - FORMATION AND INITIATION OF AUGMENTED INSPECTION TEAM (AIT) 1 l Background .
1 Formation AIT .
1- l AIT Charter - Initiation of Inspection 1 , Persons Contacted 3 '
E. Design Description 4 l High Pressure Coolant Injection (HPCI)._ System 4 1 Rod Sequence Control System (RSCS) 5 ,
' Rod Worth Minimizer (RWM) 6 I Rod Select Switch 7 Residual' Heat Removal (RHR) System 7 Condenser Circulating Water System 9 I DESCRIPTION OF EVENT: SHUT 00WN OF UNIT 1 10 , Event Description 10 B. Detailed Sequence of Events 14 II EQUIPMENT STATUS AND EVALUATION 18
! DC Motor Operated Valves 18 Eauipment Failure Description 18 f Equipment History 18 Corrective Action 19 Safety Significance and Conclusion 20
- Rod Worth Minimizer 21 l
- Equipment Failure Description 21 j Equipment History 22 l 4 Corrective Action 22 l Conclusion 22
, Rod Sequence Control System 22 a Equipment Failure Description l 23 i Equipment History 23
' Corrective Action 23 Conclusion 23
'
, esos300128 880817 PDR ADOCK 05000324 .
Q PDC
-. ,
.
y _
_
I
%
L ii Rod Select Switch 24- Equipment Failure Description 24 Corrective Action and Conclusion 24 RHR Injection Valve, F017B- 25
, Equipment Failure Description 25 Equipment History 26 Corrective Action 29 Conclusion 30 Condenser Circulating Water Traveling Screen 31 Equipment Failure Description 31 Equipment History 32 Corrective Action ~34 Unit 2 Effect 34 Conclusion 34 Equipment Safety Significance and Operability 35
, Safety Significance / Safety System Performance 35 Operability and Reportability 37 Conclusion 37 I PERSONNEL ACTIONS 38 Operations Involvement 38 Rod Worth Minimizer 38 Rod Sequence Control System 38 Rod Select Switch 39 Loss of Shutdown Cooling 39 Circulating Water Screens 40 i Conclusion 40 ; Management Involvement /0versight on F0178 Valve Failure 42 j Senior Management 42 Mid-level Management 42 First-Line Supervision 43 Decision Process 43 Conclusion 44 OVERALL CONCLUSION 44 V EXIT 45 ;
I'
Attachment 1 - Condenser Circulating Water Attachment 2 - RHR Shutdown Cooling
l
.
.
.
REPORT DETAILS INTRODUCTION - FORMATION AND INITIATION OF AIT Background Brunswick Units 1 and 2 are General Electric (GE) boiling water reactors (BWR)
IV with Mark IC containments. The units are located 20 miles south 'of Wilmington, N.C. at the mouth of the Cape Fear River in Brunswick County, Unit 2 went critical in March 1975 and was commercially operational in November 1975. Unit I went critical October 1976 and commercial operations began in March 1977.
l On July 13, 1988, a shutdown of Unit 1 commenced in order to conduct work related to the HPCI injection valve,1-E41-F006, DC motor. In the attempt to conduct an orderly shutdowr, and place the unit in RHR shutdown cooling, several equipment failures occurred between July 13 and 15 which extended the shutdown
,
and challenged all levels of the operations, maintenance and technical support i staf These failures involved the Rod Worth Minimizer, the Rod Sequence Control System, a rod select switch, the RHR injection valve (1-E11-F0175) and the traveling screens of the Condenser Circulating Water Syste Formation of All
'
On the af ternoon of Friday, July 15, 1988, the Region II Administrator, after briefing by the Regional and Resident staff and consultation with senior NRC management, directed the dispatch of an AIT to review the Unit I shutdow This team was headed by the Division of Reactor Projects Section Chief respon-sible for project management of the Brunswick facility. Subsequent to this decision, a Preliminary Notification was issued. The team included participa-tion by NRR and AE00. The decision to send the AIT was not based solely on the significance of the individual equipment failures, but rather on the cumulative effect of the number of equipment failures during this shutdown in conjunction
, with the excessive number of safety system failures during the last two years.
'
Prior to the AIT arrival on site, the licensee submitted a Confirmation of Corrective Actions letter on July 15, specifying completed interim corrective
.
actions and unit restart issues related to sizing of DC motors on HPCI and l Reactor Core Isolation Cooling valves. This letter also confirmed that Unit 1 would not restart without concurrence from the NRC. Subsequently, the NRC issued a Confirmation of Action letter on July 15, confirming the licensee's l commitment AIT Charter - Initiation of Inspection The Charter for the AIT was drafted on July 16, 1988, modified on July 20 to reflect the addition of DC motors and issued by the Regional Admtnistrator on July 21. Two team members arrived on site on July 16 and received a briefing i
at 0900 on the status of Unit 1 and also received a turnover briefing from
.
%
I
members of an Operational Perforruance Assessment Team who reviewed the licensee's actions through the shutdown and immediate recovery. Over the weekend, assistance was also provided by the resident inspection staf The remaining members arrived on July 17 and 18. An AIT entrance meeting was conducted at 0730 on July 18. Subsequent to the AIT arrival on site, several j
telaphone discussions were conducted between senior licensee and NRC personnel j involving the sufficiency of short and long term management involvement to be maintained at Brunswick in order to resume Unit 1 operation and to provide the confidence that the equipment failure problems at Brunswick would be brought l under control. Based on these conversations, the licensee provided a letter dated July 18, 1988, describing several management initiatives to improve ,
Brunswick performance. Also, at the time, the NRC began twerty four hour site ;
coverage for the restart of Unit 1 and the scheduled shutdown and subsequent i restart of Unit 2. The Charter for the AIT specified that the following tasks i be completed: ,
'
l
!
' Develop and validate a detailed sequence of event, associated with the shutdown of Brunswick Unit 1 on July 13-15, 1988, and review the equipment failure that led to the shutdown and those failures that occurred during the shutdow . Evaluate the significance of the equipment failures with regard to iadiological consequences, safety system performance, safety s,onificance, and plant proximity to safety limits as defined in the ;
Tetanical Specification I Ew.lvate the accuracy, timeliness, and effectiveness with which in1'r.,ation on these failures were reported to the NR . For eacn equipment malfunction, to the extent practical, determine: j Root cause; l If the equipment was known to be deficient prior to the event; 1 v
' 1 If equipment history would indicate that the equipment had j either been historically unreliable or if maintenance or !
( modifications had been recently performed; l
l l Any equipment vendor involvement prior to or after the event; Pre-event status of surveillance, testing and/or preventive maintenance; and The extent to which the equipment was covered by existing corrective action programs and the implication of th.e failures with respect to program effectivenes . Evaluate the effect of the failures on Unit 2, if any, and the i licensee's response.
l
,
'
6 .~ Evaluate the licensee's action taken to verify equipmen't operability on Unit 2.
- Identify any human factors / procedural' deficiencies related to the failure . Through operator and technician interviews, determine if'any of the following played a significant ~ role in each failure; plant material condition; the quality of maintenance; or the responsiveness -of engineering to identified problem '
I Evaluate operator action during the Unit i shutdown and subsequent equipment recover . Evaluate management involvement during the Unit 1 shutdown and the subsequent recover . Confirm root cause analyses for the problems associated with the D.C. ,
controlled MOVs.
I 1 Provide a Preliminary Notification upon initiation of the inspection and an update on the conclusion of the inspection, j 1 Prepare a special inspection report documenting the results of the
- above activities within 30 days of the start of the inspectio Persons Contacted
,
t
! K. B. Altman, Manager of Maintenance l l W. M. Biggs, Engineering Supervisor i C. F, Blackmon, Manager, Operations '
l
>
J. S. Boone, Engineering Supervisor G. F. Booth, Project Scientist '
S. Boyce, Project Systems Engineer '
l A. J. Canterbury, Mechanical Maintenance Supervisor l G. C. Cloninger, Mechanical Maintenance Engineer t
C. R. Dietz, General Manager J. Disosway, Electrical Engineer l K. E. Enzor, Director, Regulatory Compliance T. Groblewski, Senior Maintenance Engineer J. L. Harness, General Manager (Arriving)
i A. S. Hegler, Superintendent, Operations i
R. E. Helme, Manager, Technical Support P. W. Howe, Vice Presidert, Brunswick Nuclear Project H. Mayes, Senior Maintmance Specialist J. L. Pearson, Systems Engineer
,
P. O. Musser, Project Maintenance Engineer
! J. O'Sullivan, Team Leader / Project Manager, Valves R. M. Poulk, Project Specialist, Regulatory Compliance D. E Quidley, Project Maintenance Engineer
._ - - _ _ - _ _ _ _ - _ _ _ .
, . -
~. - .
'
o i
- 4 K. Scott, Nuclear Engineer C. L. Schacher, Project Maintenance Enginee.-
- .S. Tabor, Project Maintenance Engineer
. Design Description E.1. High Pressure Conlant Injection (HPCI) System (System Description S0-19)
The HoCI System,-an Engineered Safeguards System, serves to provide sufficient
- core cooling to prevent excessive fuel cladding temperatures in the event of a small line break of any unisolatable line directly associatect with the nuclear boile The HPCI System is . comprised of a 100 percent capacity turbine and pump assembly, with associated piping, valves, instrumentation, controls, and accessories. The pump assembly consists of a main and booster pump to provide sufficient discharge pressure to inject make-up water into the reactor pressure vessel. This make-up water is required to maintain sufficient reactor water inventory since steam generation will continue, af ter the reactor has scrammed, *
due to the core fission product decay heat. A turbine driven pumping system is used to supply demineralized maks-up water from the Condensate Storage Tank (CST) to the reactor. Remote manual operator action may be taken to change
,
pump suctio- from the CST to the suppression poo The HPCI system is designed to start and deliver design flow within 20 seconds upon receipt of an initiation signal, and is independent of AC power, plant 4 service air, and external cooling water systems. However, the inboard steam supply isolation valve is AC powered but this valve is open in the standby readiness mode and is not required to operate for HPCI automatic initiation.
i Steam for the HPCI turbine is supplied from main steam line "A" upstream of the Main Steam Isolation Valves (MSIVs). The HPCI steam supply line isolation valves are normally kept in the open position to maintain the line at operating temperature up to the steam supply valve (which is within a few feet of the
!
turbine). Exhaust steam from the turbine is discharged to the suppretsion pool. A steam supply line drain pot and steam trap drains accumulated condensate to the clean Radwaste System when HPCI is in the Standby Readiness Mod It is automatically isolated when HPCI is initiated (Emergency Mode).
,
The turbine steam exhaust line drain pot and steam trap normally drain i accumulated ccndensate to the suppression pool, ex:ept that on a drain pot high water level signal the condensato is automatically drained to the barometric cordense The normal supply of water to the HPCI System is from the CST. In the event the CST level decreases to a predetcrmined level or the suppression pool level rises above a predetermined level, pump suction will automatically be transferred to the suppression poo Cooling water for the gland seal
, exhauster barometric condenser and the turbine lube oil cooler is taken from an ;
j intermediate pressure connection on the main pump and returned to the suction side of the booster pum l
,
, , .-- ,r- _
--,,,_--..y-ro . -__,,,,.m.,-- r -,-% , . - ,,-..----w-,.- - - , - -,e . , , . _ . . . .
.-
.
,
The HPCI System when in operation discharges into the "A" feedwater line and is distributed through the feedwater sparger thereby obtaining uniform mixing with the hot water in the reactor pressure vesse A design flow functional test of the HFCI System may be performed during normal '
. plant operation by taking suction from the CST and discharging through a full ficw test return line back to the CST. The discharge valve to the feedwater line remains -closed during this test so normal plant - operation is no'
d'sturbe The control system is designed such that it will automatically transfer from the test to the operate mode in the event system initiation is required. The electrical components of the HPCI Syster, required for system operation, are supplied by DC power from the Station Battery Syste E.2. Rod Sequence Control System (RSCS) (FSAR Section 7 6.1.4)
>
'
The RSCS prevents the operator from moving a' control rod out of sequence. The RSCS consists of circuitry which allows control rods to be moved only in a predetermined sequence and by predetermined amount These limits are in effect below a preset power level in order to limit maximum control rod worths, which ensures that the enthalpy of the fuel will not exceed design limits as a result of the occurrence of a drop of the maximum worth rod. In performi.1g this function, the RSCS is redundant to the rod worth minimizer which serves ,
the same functio :
Rod density is defined as the percent of control rods fully inserted in the core (100 percent rod density = all rods fully inserted; zero percent rod
'
density = all rods fully withdrawn). In the range of 100 percent to 50 percent rod density, each control rod is moved from the full in to the full out ,
position according to a preset sequence. From 50 percent rod density to 22 '
percent power, the group notch mode of operation (notch control)-essentially restricts the rod movement to one notch per selection, thus ensuring that all j designated rods belonging to a group will be within one notch of each other while in this operating rang '
A rod drop accident is of minimal concern when the reactor is operating above 22 percent of rated power; therefore, the RSCS does not restrict rod movement above this power level. While coming down in power, when the 27 percent power level is reached, the 9.SCS will light the Rod Select pushbuttons to assist the operator in attaining mandatory rod position patterns before the 22 percent power level is reached. Although 20 percent power is the calculated safe power level, the actual level to which the switching point is adjusted is 22 percen This ensures that, with all tolerances included, the switching point will never fall below 20 percent.
- The system provides circuitry which inhibits rod movement that would result in high reactivity worths from the 100 percent rod density to 50 percent rod
, density rang In this range, the four RSCS groups of assigned rods for :Se i
sequences A and B are hard-wired. The circuit either inhibits or permi; yll l of the rods assigned to a sequence unit to be move This permit /i: %M . t condition is determined by logic circuits wnich monitor the sequence -e rod movemerts.
4 l
. .
4
The logic receives its inputs principally from the full-in and full-out switches in the rod position indicator probes and from the rod sequence selector switch controlled by the operator. These full-in and full-out switches are used for indications only, and are not used as in-out to the RW In addition to hard-wired controls, hard-wire indication ensures the operator that the correct rod sequence is being followed.- When the operator selects a particular rod sequence, all of the pushbuttons on the rod select panel of the RSCS group illuminate dimly. Any single rod selected from that RSCS group
illuminates brightl Any out-of-sequence rod selected remains- dark, indicating that an out-of-sequence rod has been selected. Thus, the operator knows that only rods controlled by illuminated switches can be moved at that time. When all rods of the chosen sequence have been moved to their correct ,
'
position (ell full-in or insert, or all full-out or withdrawn), all of the dimly-lit pushbuttons extinguish, indicating that the operator is permitted to continue with the next correct sequenc For rods that have to be valved out of service, or for rods with defective switches in the indicator probe, remote bypass switches (under keylocked access) are provided. These switches are under supervisory control and are not activated until it has been determined that no high worth rod pattern would
,
result from bypassing the logic signal to the control logic circuitry.
,
In the region between 50 percent rod density and the preset power level, any l rod not full-out at 50 percent density can only be moved a single notch at a time. Furthermore, hard-wired logic is used to restrict all the rods assigned
.
to a notch group to be within one notch of each othe ;
In this operating range, the pushbuttons of the first selected rod and its associated notch group of rods are illuminated by dim lights on the rod select panel which displays all of the rod The dim lights indicating the chosen notch group automatically go out when insert or withdrawal notching of this group is complete and every rod of this notch group is at identical notch positions. The dim lights of the chosen notch group also go out if a rod of another notch group is chosen, and pushbuttons indicating the new notch gr.ep are dimly lit. The notch positions of the previously chosen notch group of rods are "memorized" to ensure that they are within one notch of each other at .
any time in this operating rang Once the preset power level is achieved, the RSCS is automatically out of service in the startup operatio Conversely, the RSCS is automatically in service when the power is decreased to the preset power level. At 35 percent j power level during power descention, annunciation is provided to aid the J
operator in preparing the RSCS for operation.
] E.3. Rod Worth Minimizer (RhK) (F.qAR Section 7.7.1.6.2.3)
, The RWM function assists and supplements the operator with an effective backup control and monitoring routine that enforces adherence to established startup,
. shutdown, and low power level control rod procedures. The computer prevents
] the operator from establishing control rod patterns that are not consistent
'
l
-
.
.
with pre-stored RWM sequences by initiating appropriate rod select block, rod withdrawal block, and rod insert block interlock signals to the reactor manual control system rod block circuitr The RWM sequences stored in the computer memory are based on control rod withdrawal procedures designed to limit. (and thereby minimize) individual control rod worths to acceptable levels as determined by the design basis rod drop accident.
'
The RWM function does not interfere with normal reactor operation, and in the event of a failure does not, itself, cause rod patterns to be established. the RWM functions may be bypassed and its block function disabled only by specific procedural control initiated by the oparato E.4. Rod Select Switch (System Description 50-7)
There are 137 magnetically latched double pole, double throw pu.thbutton
>
switches arranged in the same pattern as the rod display unit. Each switch is back lighted to show the selected rod. Only one rod can be selected a'. a tim by pressing the corresponding pushbutton. Pushbutton selection accorsplishes Power Range Monitoring control over rod movement, and activates the companion four-- 4 display group to provide position dat It also gives a permissive for o.. ;.tional control valve solenoid selection and fuli core display select indicatio The pushbutton array is furnished 28 VDC power through the Rod Select Power Switc E.5. Residual Heat Removal System (System Description 50-17)
.
E. low Pressure Coolant Injection (LPCI)
The Low Pressure Coolant injection (LPCI) mode operates to restore and maintain the water level in the reacter core at an adequate height for cooling during a Loss of Coolant Accident which results in reactor depressurization. This engineered safeguard subsystem operates in conjunction with the HPCI System,
<
Automatic Depressurization System, and the Core Spray System to ensure that the core receives adequate cooling under all postulated accident conditions, How3ver, LPCI, or HPCI, or Core Spray is capable of providing adequate core cooling under specific accident conditions for which it is designe Af ter an accident, coolant is lost from the core and the reactor vessel pressure decrease The HPCI System which is a high pressure system, pumps water into the reactor vessel when the nuclear system it at a high pressure to preclude a harmful loss of coolant and to reduce the reactor vessel pressur If the HPCI System cannot perform its function of maintaining water level and controlling system pressure, the Automatic Depressurization System functions to reduce the nuclear system pressur When the pressure in the reactor vessel has decreased to a preset value (410 psig), both Core Spray and LPCI are put into operation to ensure core coolin The HPCI System flow ceases and the automatic depressurization is stopped after the LPCI System is put into operation. When the pressure differential between the reactor vessel and the primary containment is 20 psi, the LPCI System is
-
.
.
capable of delivering rated flow. After the core has been flooded to at least two-thirds core neight, only one pump is required to maintain this leve During LPCI operation, the RHR pumps take suction from the suppression pool through valves E11-F020A and E11-F004A (pumps A and C) and E11-F020B an E11-F004B (pumps B and D). The pumpage will be discharged directly into the reactor vessel via both of the recirculation loops through valves E11-F015A and F017A or E11-F015B and F017B, bypassing the RHR heat exchanger (through bypass valve E11-F048 A/B).
During the interval of time when the RHR pumps are operating to restore the reactor vessel level, heat removal is not necessary. Once the core is flooded, one RHR or core spray pump is normally required to make up for shroud leakag The other pumps are stopped so that the emergency power-(if there is a loss of off-site power) that was being required by +hese pun.ns may be shif ted to other plant loads including RHR service water ps.1ps, and an RHR pump and heat exchanger in the containment cooling mod E. Shutdown Cooling The shutdown cooling mode removes the residual heat from the reactor primary system to maintain the reactor 'in a cold shutdown condition and to cool it for refueling and servicin The system is placed into operation during the normal reactor shutdown, after a normal cooldown, using the main condenser when the reactor vessel pressure reaches 125 psig or les It has the capability of completing cooldown to 125'F in approximately 20 hours2.314815e-4 days <br />0.00556 hours <br />3.306878e-5 weeks <br />7.61e-6 months <br /> af ter the control rods have been inserted and maintaining the nuclear system temperature at or below 125'F so that the reactor can be refueled and service In the shutdown cooling mode, reactor water is pumped from reactor recirculation Loop A suctiot, through isolation valves E11-F008 and E11-F009 and branches to the suction of the RHR pumps (valves E11-F006A through D). The reactor water is passed through the RHR heat exchangers where it is cooled by the RHR service water. The coolant is returned into the reactor vessel via the recirculation loo In order to prevent possible high thermal stresses in the pipe wall of the tee
, connection where the PHR line joins the recirculation piping during initiation
! of the shutdown cooling mode, a warmup procedure is utilized. The procedure involves filling the RHR System with hot water (vessel temperature - 110*F)
before beginning shutdown. With the entire shutdown cooling loop at an elevated temperature, the RHR pump can be started in the heat exchanger bypass
>
mode at full rated flow without exceeding the RHR to Recirculation Syste.n tee thermal gradients. Preparations for shutdown cooling are handled as follows:
i 4 The reactor water level is permitted to rise during cooldown so a net loss
of vessel inventory can be accommodated without tripping any of the low i level switches, i
e r , . - - . m. r, , . _ . -..--_,._,r-- - , ~ . , ,---_.--_-,,--,y, ,
. _ _ _ . .
.
. .
-L 9-
a When the vessel pressure is low enough so that the LpCI mode is no longer 1
'
required, the RHR System is drained and the heat exchanger flushed to condensate quality, With the RHR heat exchanger isolated, the RHR System suction is opened to the recirculation loop suction pressure cnd the cold water in the RHR :
piping is forced out through E11-F040.and F049 to Radwast ' The RHR pump is started and full flow established from the recirculation ,
'
pump suction line through the RHR pump, through the heat exchanger bypass, and back to the recirculation pump discharge lin , The RHR heat exchanger is then slowly phased in, and the vessel cooled
.
,
dow E.6. condenser Circulating' Water System (FSAR Section 10.-4.5.2).
The flow diagram for this system is contained in Attachment The ~ condenser circulating water system is designed to nrovide a flow of 624,000 gpm to the condensers of each unit. Each of four 25 percent capacity vertical pumps per unit takes suction from the intake canal and discharges through motor operated !
shut-off valves into an 8 foot diameter *, teel cement-lined pipe leading to the condensers. The mouth of the intake caral is screened by a h inch resh wire
.
fence which serves to divert fish fror, entering the canal. Water flowing'to r i the circulating water pumps first passts through the trash ,acks and traveling 'l
) screens to prevent entrance of relativnly large objects into the pump sump ba ,
j There are four traveling screens per utit to prevent small debris from entering i
, the system. Three of the traveling screens on each unit are equipped with fine ,
} mesh (Imm) screen to reduce the numbet of fish and fish larvae entrained.in the i
'.
condensers. All of the traveling screens are also equipped with fish pans and a screen wash system to remove impinged debris and fish or fish larvae from the screens. A collection trough, return sluiceway and return sump are provided to transport fish washed from the screens into a holdup pond for eventual return to their natural habitat. A typical traveling screen is shown in Attachment 1, 1 There are self-cleaning debris filters installed into each circulating water 1 condenser inlet line just below the inlet condenser waterboxes. These filters
- with a 3/8 inch diameter hole size prevent any clogging of the one inch
- condenser tube In addition, to improving condenser officiency and eliminating waterbox cleaning, the filters eliminate tube failuras due to-erosion caused by tube blockages.
'
From the condenser outlet water boxes, the circulating water flows through 84
- inch diameter steel pipes which discharge into reinforced concrete tunnels at l
'
the west wall of the Turbine Building, The concrete tunnels terminate with a l weir approximately 1000 feet south of the Turbine Buildin Four ocean i
, discharge pumps pump the water from the discharge canal through two 13 feet diameter reinforced concrete pipes which discharge 2003 feet beyond the shore line. The pipes have a minimum of 3 feet cover and discharge horizontally for j maximum mixin ! l
.
, , _ - - _ _ _- - , - _ - ,, , , - _ _ . _ _ - _ - _ - , _ . _ - - - - -
l
-
l
.
10 <
i
[
I DESCRIPTION OF EVENTS: SHUTDOWN OF UNIT 1 The following Event Description and Sequence of Event covers that period from the commencement of shutdown of Unit I until the unit is tied to the grid after '
restar Many equipment problems occurred during the shutdown and restar <
Those issues beyond the problems identified in section I of this report, were l not formally developed by the AI l
A. Event Des qi_ption July 13, 1988 A decision was made to begin shutdown of the Unit I reactor at 2000 hours0.0231 days <br />0.556 hours <br />0.00331 weeks <br />7.61e-4 months <br /> on July 13, 1988 to allow for work on the HPCI injection valve 1-E41-F006. The operators reduced reactor power using recirculation flow in accordance with the ,
General Procedure (GP-5). Excess equipment-was removed from service as power was reduce At approximately 40% reactor power (2107 hrs.) the operators were warned by annunciations and indications that the RWM had sensed an abnormal control rod pattern and had prohibited control rod movement. This was an erroneous indication. The RhN is designt) to only inform of abnormal control rod patterns at this reactor power level and not block cont *ol rod movement. Not ;
i until approximately 27% reactor power should control rod blocks be receive J l The operators bypassed the RWM to defeat this erroneous control rod block. A l second licensed operator was dedicated to verify proper control rod patterns
!
while the RhH was defeate The Senior Reactor Operator (SRO) on shift, in anticipation of entering the enforcemeat zone of the RSCS, reviewed the trouble tickets associated with RSC This was done to preclude experiencing difficulties with the system due to previous malfunctions. One trouble ticket existed and was cleared ou The operators continued to reduce recirculation flow until at 2220 hrs. RSCS initialized at approximately 20*4 power. Control rod selection was prohibite The cause for the control rod select block was unknown at this tim The operators verified the proper control rod position Instrument and Control technicians were dispatched to investigate the proble July 14, 1988 The Operations Manager was notified (0145 hrs., by tM Shift Operating Super-l
'
visor of the delay in the unit shutdown. Discussions centered on contingencies l for obtaining Condition 3 priur to (xpiration of the HPCI Limiting Condition j for Operations (LCO) at 1102 hrs. A decision was reached to manually scram the i reactor if RSCS was not repaired by 0330 hrs. The Plant Manager concurred on i this matte Technicians identified the problem as a failure of RSCS to confirm all "A" sequence control rods full-ou The SRO recognized the association of an existing trouble ticket on control rod 22-03 full-out indication and ordered ;
l l
l
'
? l i
-- . - _
-
.
l t
l
l
this control roJ to be bypassed to the full-out position in the Rod Position l
Information System (RPIS), c
,
Selection of control rods for movement was now permitted by RSCS (0250 hrs.). -
l The operators continued with a normal shutdow l l
l While centinuing with normal control rod insertion, at 12% power, the control
- operater was unable to select control rod 34-03 (0645 hrs.). The latching coil l in thw pushbutton seleutor switch had failed preventing selection of the cont l?ol rod. Instrument and Control technicians were dispatched to correct the
'
,
l- preblem. At 0745 hrs, maintenance was completed on the pushbutton and control
!
rta insertion continue At 0831 hrs. the reactor was manually scrammed from 9.5% power to allow sufficient time to cooldown to Mode three prior to the expiration of the HPCI LCO. The operators followed E0P-01 path 3. Delays in the shutdown due to the various equipment malfunctions had depleted the time margin to expiration of the LCO, therefore, to ensure normal cooldown the decision to manually scram the reactor was made, t
!
At 0839 hrs the reactor scram was manually reset by the operator l
!
One control rod "bounced out" to position 02 after the scra To allow l reinsertion of this control rod (by removing the shutdown mode rod block) the i
'
operators positioned the mode switch to refuel and inserted the control rod (0848 hr s.). Subsequently the mode switch was returned to the shutdown position (0849 hrs.) and the shutdown mode switch scram was reset. The operators initiated a depressurization to 113 psi l l In accordance with the General Procedure for Unit Shutdown, the operators !
,
prepared the RHR system for the shutdown cooling mode of operation (1050 hrs.). !
l Once all prerequisites for shutdown cooling alignment were met, the control i operator started RHR pump "B" and attempted to establish the flow path to the i recirculation loop by opening valve RHR F0178. The valve failed to open.
1 After repeated attempts to open the valve failed the control operator secured l
'
the RHR pump (1750 hrs.). Unit I remained on the main condenser for decay heat removal. Loop A RHR was available for shutdown cooling, however, it remained aligned as the LPCI injection pat July 15,1988 Circulating Water Intake Pumps (CWIP) IC and 1D were in operation, IA and IB I
were in standby. (pumps 1C and 10 had fine mesh traveling screens and 1A had !
course mesh traveling screens). Debris filtee flushing was schedule I Unit 2 was at 100% power running all CWIPs (A-D) (pumps 2A, 28, and 2C had fine mesh screens 2D had course mesh screens).
Dabris filter flushing had been scheduled on Unit I and the 1B CWIP was started i
to support this activity (0112 hrs.). At 0249 hrs. IB CWIP was secured after
! the completion of debris filter flushin l l
'
i
- l
,
i
I
The control operator on Unit I received a "High Screen Differential or Stop", .
'
alarm for 1C CWIP (0249 hrs.) One minute later the pump tripped alarm was '
received. This indicated a major blockage of flow through the screen and that
.
the blockage occurred suddenly. The pumps were automatically tripped due to ;
l suction pressure degradation. At 0251 hrs, a similar alarm actuated for the ID !
1 CWIP and the pump tripped in the same manne l 5 t The operator started the 1A CWIP (0252 hrs.), restarted the IB CWIP (0253 {
hrs.). The 1A had been in standby therefore it appeared safe to attempt a start '
on i Likewise, the 1B pump had been running without difficulty therefore it appeared safe to start this pump also. An outstde auxiliary operator (AO) was dispatched to check the condition of the screens. The report from the A0 was that there appeared to be no flow blockage of the screens, and the screen wash system was operating properl l The 2A and 2B CWIPs tripped at 0255 hrs. and 0256 hrs. respectfully. The loss of circulating water flow immediately affected condenser vacuum on Unit 2 (0257). The operators commenced a power reduction by reducing recirculation flow to 35 E+6 lb/hr. In addition, they initiated actions to avoid the region of core flow / power instability. At this time 2C CWIP tripped.
J Unit 2 was brought to 40% power at 0300 hrs. Operations began diagnosis of the CWIP trips to ascertain the cause of the high differential pressure conditio !
One possibility was that a low tide condition had contributed to the proble To either substantiate or disprove this theory the operators attempted restart of the CWIPs during high tide hours (high tide was to occur at 1100 am). In addition, the Shif t Operation Supervisor, Technkal Support and Haintenance i Supervisors were at the intake area to formulate a trouble shooting plan. The plan would include calibration of instruments and jumpering the high
- differential pressure interlocks and starting the pumps. These activities consumed the morning hour Several unsuccessful starts were attempted on 2A,B,and C CWIPs over the next several hours. At 1400 hrs. the 2B CWIP was started with the high differential pressure trip bypasse Jumpers were instailed on the trip to ascertain the root cause of the event (no visible i evidence of flow blockage existed). The differential pressure increased to 40 J
inches and the pump was secured (pump trip is normally 36 inches). As a result
- of the excessive differential pressure experienced the 2B screen drive shear
,
pin sheare '
'
l Several sections of the 2B CWIP screen were removed for inspection. The punp l was then successfully started (1657 hrs.). The subject screens were observed '
, to be fouled with skeleton shrimp. These shrimp had attached themselves to the screen and severely restricted flo Now 2B and 20 CWIPs were running. Vacuum had recovered and power ascension began (1705 hrs.). At 1810 hrs. power ascension was halted to maintain vacuum J while awaiting removal of the fine mesh screens from the 2A CWI A CWIP was successfully started at 2003 hrs. and power ascension continued. At 2340 hrs, reactor power had reached 95%. l l
i i I
_----_-_-_-----;
l
-
13
-
i July 16, 1988 I i
A mechanic working on the screens reported that the 2B screen was being damaged-(0005 hrs). In anticipation of a loss of vacuum, the operators initiated a- !
power reduction. Tnree minutes later the 28 screen was secured and the 2B pump l was manually tripped at 0010 hr Maintenance personnel completed removal of the fine mesh screen on 2C CWIP and it was successfully started at 0403. Reactor power again was increased. At 0415 hrs, power was at 70% and increasing. CWIPs 2A, 2C, and 20 were in service on Unit 2. -CWIPs IA and 18 running on Unit ;
At 0600, the modification work was completed on the HPCI _1-E41-F006 valv Work on this valve precipitated the outage. DC motor replacement was the major modificatio Reactor power is returned to 99.6% at 1900 hr July 17, 1988
On Unit 1, repairs were completed on RHR F0178 and operability verified. Loop
"B" RHR was placed in LPCI standby mode at 0243 hr The "A" loop RHR system was placed in shutdown cooling mode at 2318 hr July 18, 1988 A flange gasket fails on the RHR Service Water (RHRSW) side of "A" loop RH The loop was removed from service. (0144 hrs.). Investigation into the failure bega At 0507 hrs, the "B" loop of RHR was placed in the shutc.own cooling mode on Unit RHR loop A on Unit I was returned to LPCI standby alignment after repairs to RHRSW flange gasket. Shutdown cooling loop B was secured in preparation for unit startu July 19,1988 LPCI loop B is returned to servic In preparation for unit startup the licensee conducted PT-01.6.1 RSCS operability. There were indications of abnormal control rod movement during this test. Diagnosis of the problem began and while moving control rods the rod selection capability was los July 20, 1988 The Reactor Mode Switch was placed in the Shutdown position to complete repairs )
of the RSCS. At 1334 hrs the Mode Switch was placed in the Star'.up/ Hot Standby position to test RSCS ar.d the RWM. Additional malfunctionr. of RSCS were observed. The Reactor Mode Switch was returned to Shutdown (1400 hrs.),
while repairs were completed,
- _ - - .
.
.
- --
'l l
,
At 1738 hrs, the licensee had repaired the RSCS and placed the Reactor Mode
,
Switch in the Startup/ Hot Standby position. Licensee began venting of control rods which had exhibited difficulty in movemen ]
July 21, 1988 ,
I At 1523 hrs, control rod venting and timing were completed, F ' RWM were I operable. The licensee commence startup of the Unit I reacto The Unit i reactor was brought critical at 1740 hrs.
! July 22, 1988 I Operators tied the main turbine generator to the grid at 0625 ht HPCI i operability was declared at 1848 hr l Detailed Sequence of Events !
The sequence of events was developed from discussions with operations personnel, review of computer alarm typer data recorded during the ev2nt and review of plant logs and investigation package l
. i July 13, 1988
,' l J
- Initial Conditions
- Unit I at 95.4% power !
i HPCI System Inoperable l i "A" North Side Condenser Waterbox ;
out-of-service ;
i 2000 Commenced shutdown to less than 113 psig to replace motor for HPCI i l
1-E41-F006 injection valv j i 2107 At approx. 20*4 power experienced erroneous rod blocks from RWM. Operators bypass RWM, second licensed operator used to verify rod movement.
2200 Cleared Trouble Ticket relating to Rod Sequence Control System (RSCS) in i anticipation of RSCS initialization.
.
2220 Recirculation flow decrease d ' the point that RSCS initialized. Control rod selection was prevented due to unknown cause. I&C was called in to
'
investigate the probicm.
! July 14, 1988
!
j 0145 On-call supervision (operations manager) notified of conditio j 0250 Control rod 22-03 bypassed to full out in Rod Position Indication Syste Selection of control rods fc movement was now permitted by the RSCS,
~
!
l l
l l
-
i l
_- -_- - - - .- -
, - - - .
..
.
.
0645 Unable to select control rod 34-03 from control pane Rod selection difficulty corrected, continued with control rod insertio Manual scram inserted to allow sufficient time for cooldown to Mode 3 prior to expiration of HPCI LC Operators followed E0P-01 path Reset reactor scra Placed mode switch to refuel to insert control rod 18-31 from 02 to 0 I 0849 Placed mode switch to shutdown, reset reactor scram. Cooldown to 113 psi l initiated, l 1050 Auxiliary operator filled and vented "B" loop RHR in preparation for l shutdown coolin l l
1055 Reactor pressure at 113 psi l 1750 While attempting to place "B" loop of RHR in shutdown cooling, the E11-F017B RHR injection valve failed to open. Continued using the l condenser for decay heat remova July 15, 1988 Unit I continued in hot shutdown at 95 psig reactor pressur l
'
0112 18 CWIP was started to provide suf ficient flow for debris filter flushing, 0249 1B CWIP was shutdown after completion of debris filter flushin ,
0249 Received "High Screen Differential or Stop" alarm for 1C CWIP Within the minute received 1C pump tripped alar Received the same alarms and subsequent pump trip for ID CWI Started 1A CWI Restarted IB CWIP; an A0 was dispatched to investigate condition Received the same alarms and subsequent pump trip for 2A CWI Received the same alarms and subsequent pump trip for 2B CWI Vacuum decreasing on Unit 2, commenced power reduction. 2C CWIP trip Unit 2 at 40*. powe Start attempted on 2B CWIP was ut. successful due to high screen differential pressur .
.
.
0304 Start attempted on 2A CWIP was unsuccessful due to high . screen differential pressur '
'
0326 Unsuccessful start attempt on 2A CWI Unsuccessful start attempt on 2C CWIP, i 0408 Unsuccessful start attempt on 2B CWIf, 0820 Unit 2 reactor power is 43%, another unsuccessful attempt was made to start 2A CWI '
0920 Unsuccessful start attempt on 2B CWI Unsuccessful start attempt on 2C CWI Unsuccessful start attempt on 2C CWI I i
1400 Attempted 2B CWIP start with high ' differential pressure trip bypasse Manually secured pump at 40" d/p, Shear pin broken on 2B scree l 1657 Several sections of 28 CWIP screen removed, the pump was successfully i started. Removed screens were observed to be fouled with skeleton shrim ,
1705 Commenced power increase on Unit and 20 CWIPs runnin Stopped power increase at 67% power to maintain vacuum awaiting removal of fine mesh screen from 2A CWI ;
2003 Started 2A CWI Unit 2 reactor power r.t 95%. j July 16,1988
)
0005 Mechanic reports 28 screen being damaged, operators began reducing powe i 0008 At 70% power 28 screen is secure ,
0010 Tripped 2B CWIP manuall ]
0403 Started 2C CWIP after fine mesh screen remova Reactor power at 70% and increasing. CWIPs 2A, 2C, and 2D running on Unit 2. CWIPs 1A and IB running on Unit Motor replacement, diagnostic testing and torque switch settings completed on Unit 1 HPCI F006 valv Unit 2 reactor power 99.6%.
- -. . - -
,
! l
17 -i
,
July 17, 1988 1 0243 Unit 1 RHR F017B valve repairs completed and Unit I loop "B" shutdown
~
- cooling.in standby.
I 2318 "A" loop shutdown cooling pla'ced in servic Gasket fails on RHRSW flange, secured "A" loop shutdown cooling.
July 18, 1988 0507 Unit 1 "B" loop shutdown cooling placed in service.
1153 "A" loop LPCI returned to servic '
,
2300 Stopped "B" Icop oc shutdown cooling, in preparation for reactor startup,.
July 19, 1988 l ,
'
0110 "B" loop LPCI returned to service.
.
'
0248 Indications of abnormal control rod movement during Performance Test 01.6.1, RSC <
'
,
2200 While attempting to time the movement of two control rods, selection j capability is los ;
l July 20, 1988 l 0157 Reactor Mode Switch placed in Shutdown to complete repairs on RSC <
I 1334 Reactor Mode Switch placed in the Startup/ Hot Standby position per GP-01 ,
to test RSCS and RWM, reactor is now in mode :
i 1400 Difficulties experienced with RSC Mode switch returned to shutdow i 1738 RSCS repairs completed, placed Reactor Mode Switch to the Startup/ Hot !
i Standby position. Licensee began venting and timing of control rod {
. July 21, 1988 1523 Control rod venting and timing completed, RSCS and RWM operabl j Commencing Unit 1 startup, withdrawing control rod !
I ;
j 1740 Unit I reactor is critica ; July 22, 1988
' !
!
i 0625 Unit I turbine tied to the gri '
'
- 1340 Commenced operational performance test on Unit 1 HPC !
i ;
-
,
!
_
.
.
18 .;
<
1848 Completed tese and HPCI declared operabl III. EQUIPMENT STATUS AND EVALUATION OC Motor Operated Valves W
Equipment Failure _ Description
' l l The circumstances surre Jnding design deficiencies associated with motor powered
'
valve actuators were investigated during this inspectio This 'section 4 describes two separate problems associated with 250 VDC valve actuator motors and *, heir control circuit A. Switching Surge Problem t
In December 1987, valve 1-E41-F001, the steam supply valve to the HPCI turbine, failed to open during a test. This event, reported in LER 87-023, was caused
, by damaged motor windings. A special consultant determined that the motor ,
windings were most probably damaged by a switching surge (voltage impulse)
~
d induced in the separate shunt field. It was postulated that a switching surge stressed the insulation of the shunt field beyond its impulse withstand strength. A Part 21 report was issued on May 6,1988. The failed insulation caused a short between the shunt field conductors and the series field conductors which resulted in high currents and visible damage. Shorting of a portion of the field winding would result in speed problems. Shorting of the series field would affect armature current and motor torqu The design deficiency is that a path for discharge of field current upon de-energization :
of the shunt field was not provide The deficiency applies to all the DC !
motor operated valves at the sit ,
,
A. Insufficient Torque Problem :
! On July 1, 1988, valve 1-E41-F001 again failed to open during a test. On
. July 2, 1988, the valve manufacturer (Anchor-Darling Co.) and the actuator I manufacturer (Limitorque Co) as well as an NRC inspector were at the site to )
investigate the problem. NRC Report 50-325, 324/88-21 documents the activities conducted at the time to determine the cause of the F001 failure, and identi- ,
fies an unresolved item. The cause of the July 1 failure was thought to be
- thermal binding. While studying Motor Actuator Characterizer (MAC) graphs, the l licensee identified a separate problem. The actuator drive motor for F001 was
, stalling during the period of time (about 2 seconds) that the starting resistor, located in the motor control center, was in the circuit, i Calculations indicated that the F001 motor could not supply sufficient torque l to operate the valve during certain design basis events when the voltage would 1
- be significantly reduced below normal operation.
' Equipment History i
! A. Switching Surge Problem i
1 l
-
_ _ _ _ _ __ _ _ . . _ _ _ -. . _ _ _ _ _ _ _ _ _ _ _ , . _ _ _ _ _ _ , , ~ . . _ . .
.. - . . -
O
'
,
,
!
i 19 :
,
As stated earlier, the switching surge problem was basically a design problem :
-
in that a field discharge resistor (or equivalent) was not provide The design at Brunswick is somewhat unusual in that the shunt ~ field is continuously energized. Examination of Drawing LL-9272, Rev. O, dated 11/10/72, Control Wiring Diagram for the 2-E41-F012 valve, shows that the shunt field was wired ;
in the same manner as the original desig At least seven DC valve actuator motors failed due to shorted windings since ,
198 Four of these failures occurred between January 1981 and January 198 The next failure occurred in December 1987, and this failure precipitated the' ,
t root cause analysis mentioned earlier. Two failures occurred in 1988 that were '
shown to be a result of the switching surge probla A. Insufficient Torque Problem l I
As a result of poor coordination of design activities, the 240 VDC valve actuator motors were sized without regard to the fact that the motor control
-
center incorporated resistor type reduced voltage starters. Testing did not t
are conducted during periods when the battery voltage is at normal levels. MAC
charts of torque and current contained the information that motors were occasionally stalling during the period of time that the starting resistor was in the circuit, however, the licensee did not focus on this aspect of the MAC charts until the July 1, 1988 even l The licensee performed a two-month- internal review of the HPCI system. The l inspection itself was concluded on May 15, 1987. One of the design questions
- that was generated during the HpCI internal review was: "Does use of reduced
! (versus full) voltage motor starters affect IEB-85-03 calculations?" Resolu-l tion of this question was given top priority, and the Technical Support Group
submitted a finding on July 17, 198 Their finding was that the motors 1 provided sufficient torque despite the use of starting resistors. They reached i this erroneous conclusion because they failed to realize the fact that, if the motor stalls af ter the "hammer blow" has occurred, the efficiency of the actuator is significantly reduce The E41-F006 valves had marginal torque even without a starting resistor in the circuit. Apparently, attention was not
- focused on the F006 valve at that tim i Corrective Action
- A. Switching Surge Problem j The corrective action for this problem primarily consists of installing a metal i oxide varistor (fused) across the terminals of the shunt winding at the MC l The purpose of the metal oxide varister would be to limit the magnitude of
- voltage at the winding terminals to 620 Volts which is well below the basic
- impulse strength of the winding, The modification packages are scheduled to be i forwarded to the site on August 15 and September 15, 1988 for Units 2 and 1 i respectively. Installation completion is scheduled for January 1989. In the I
'
interim, the licensee is manually installing a metal oxide varistor across the terminals before each deenergization of the shunt field. Also, plant
i
I
.
. ;
l
!
instructions call for verification of DC valve operability after closing the circuit breakers. In addition, procedures will require periodic checks of fuse statu A. Insufficient Torque Problem Motor sizing calculations to support tne adequacy of proposed modifications i and/or justification for continued operation were reviewed for methodolog '
Sources of information were reviewed. Specific corrective actions include the following: ,
Plant modification PM-88-019 increased the feeder cable size, bypassed the start resistor, ~ replaced the motor (same size) and changed the gear ratio for 1-E41-F001. This PM also bypassed the start resistor for both i
'
E41-F006 valves and valve 2-E41-F00 *
Plant modification PM-88-025 replaced the 100 ft-lb motor for 1-E41-F006 :
with a 150 f t-lb motor. The Unit 2 F006 motor was also scheduled for !
replacment with a 150 ft-lb moto '
Motor sizing calculations for nine other valves (per unit) mentioned in l the Confirmation of Action Letter dated July 15, 1988 were completed, j
- Battery sizing calculations were revised in light of the fact that starting resistors were bypassed and motor sizes increase *
The licensee committed to complete analysis of all DC motor operated valves by October 1, 1988. This analysis will include determination and i consideration of actual ambient temperatures at the motor l
- In general, each of these corrective actions apply to Units 1 and 2, but each unit requires a separate analysis. There may be minor differt;nces in required modifications between the two unit Safety Significance and Conclusion The licensee's program was effective in identifying and resolving problems with 240 VDC valve actuator motors and their control circuits during the period of July 1, 1988 to the time of the inspectio Immediate and short-term correc-tive actions were appropriate to the circumstance It is expected that the addition of metal oxide varistors (fused) across the shuat field will solve the switching surge problem. Tests have shown that the surge wave peaked at 4KV or above. Oscillograms to be made after the modifications are completed are expected to verify that the peak is limited to 620 volt In addition, industry experience with this solution gives confidence that it will be effective in solving the problem. The cause of the insufficient torque problem lies in errors made during the original design stage. Reduced voltage starters are not appropriate for motor operated valve applications because of the high starting torque requirement __ ~ ..
.
.
21 '
I
- The HPCI system is required by the Technical Specification to be OPERABLE when l W the Unit is at power. Because of design problems mentioned herein (inoperable i valves) the HPCI system may have been unable to perform its intended function i during certain design basis accidents. There are apparently five possibilities where the licensee could have identified this valve operability problem earlier and declared the HPCI system inoperabl They are as follows
! -
The actuator vendor has stated that if-it had known .of the starting ;
.
resistors during plant design and construction, a larger motor would have '
'
been provided
- -
Previous failures of DC motors could have been caused by the undersized '
motor phenomena
-
MAC graphs, if studied and analyzed closely, could have revealed the relationship between motor stalling and the "hammer blow" effect.
.
-
A 1984, 125/250 Volt DC Load Study identified problems with DC motors as
- they relate to starting resistors
-
The 1987 internal HPCI Safety System Functional Inspection (SSFI)
l identified concerns over undersized DC motor Collectively, all of these issues should have resulted in the licensee identi-fying this problem earlie The major contributors to this pre-knowledge ;
appear to be the lack of vendor knowledge of starting resistors, the DC load ;
study and the HPCI SSFI. In consideration of the above facts, the DC motor '
) problem is identified as a violation, inadequate Corrective Action for Problems a Identified in DC Motor Operated Valves. Tracking of this violation will be conducted, using a subsequent inspection report.
- B.I. Equipment Failure Description I
J At about 2107 on July 13, during the shutdown of Unit 1, the control room j received erroneous control rod blocks from the Honeywell rod worth minimizer j (RhN) at about 40 percent power. The RWM had obviously failed because, at 40 1 percent power, the proper response of the RhN is to provide annunciation and not blocks when deviations from the prescribed rod withdrawal / insertion pattern are detected. The system is designed to provide rod blocks only when reactor power is less than 23 percent. The system was bypassed, and as per the j Technical Specifications, a second licensed operator was stationed to j independently verify .,d positio '
After the RhN was bypassed, its operation was monitored as the shutdown proceeded. The RhN only provided annunciation signals below 23 percent reactor power instead of rod blocks as expected. The system's responses were directly
opposite of what they should have been, s
. _ _ _ -____ _--_
- . . _-. . .. - -
.
.
. ;
t l
<
'
) i i l j The failure of the RWM during the shutdown was attributed to a faulty 6 VOC l t
power suppl This was determined following the shutdown when troubleshooting '
i
of the system revealed a high level of electrical noise in the power suppl ! B.2. Equipment His. tory ;
1 The Operations and Instrumentation and Control (I&C) staffs stated that the RWM
- has had an extensive history of mechanical and computer software failures
'
during the plant's life. These personnel indicate, without exception, a lack i
- of confidence in the syste [
! The inspector reviewed Brunswick work orders dated from 1984, reports submitted :
i since 1985 under 10 CFR 50.72, and the NRC Document Control System data base l l which contains records from 1979. Review of these data bases produced five
'
records describing RWM failures on Unit 2. . One record was found which described a failure of the RWM on Unit 1. This was the most recent failure of f
,
the RWM system prior to the July 13, 1988. failure. This failure occurred June ,
- 6, 1988 on Unit 1. While proceeding to shutdown, the RWM would not annunciate )
! or block during movement of rods which were out of sequenc l
- B.3. Corrective Action [
i i i Short-term corrective action for the July 13 failure included changeout of the !
!
faulty power supply. An additional problem with the RWM occurred subsequent to i the shutdown of Unit 1 and required maintenance. However, this problem was i
'
i beyond the scope of the AIT's review charter. The long-term corrective action i includes replacement of the current RWM with an updated design which has a :
dedicated computer. This modification will take place during the next !
scheduled refueling outage for Unit 1 in November, 1988. The new system, the i
'
GE designed NUMAC, was installed on Unit 2 during the unit's last refueling
'.
d outage. The experience with the system to date has been positiv The j licensee believes the new system will eliminate the types of problems caused by ;
the old RWM syste ,
! i'
l B.4. Conclusion
) * The actions of I&C personnel in trouble shooting and repairing the RWM were reviewed by the inspector and found to be adequat .
ll ,
i * The replacement of the old RWM system with the new NUMAC system should, at ,
least, eliminate concerns regarding cornponent aging. Given the positive >
l experience to date with the system on Unit 2, its placement on Unit 1 ;
J should reduce or eliminate problems which have been experienced during low i power operations and restore personnel confidence in the RWM syste i i
) Rod Sequence Control System Equipment Failure Description
i f
'
At 2220 on July 13, after initiation of rod block enforcement, RSCS began to I prevent the operator from selecting control rods for movement while completing {
l t
t
- _ _ - _ _ _ _ ___ - ___ - _ ____-_ __--_--_-_-- _ -__-_ -- _ _ _ _ _ _ _ _ _ _ _ _ -- . _ _ - _ - _ - _ _ _ - . - _ _ _
.
.
- 23
'
e the controlled shutdown. Following unsuccessful efforts of operating personnel i to determine the reason for the problem, an I&t., technician was called in to .
'
- , troubleshoot the syste <
i' '
C.2. Equipment History
!
l' The inspector reviewed Brunswick work orders as well as reports submitted to the NRC by the licensee under 10 CFR 50.72 and 10 CFR 50.73. There have been
numerous failures of equipment which provide input to the RSCS, However, there !
have been few reported failures of the RSCS itsel j Licensee work orders dating back to 1984 and related to the Rod Manual Control
'
System (RMCS) indicate that, of a total of forty-seven records reviewed, 3
records were related to corrective maintenance required to be performed on the
during the past four y. trs, nine involved the repair of components in the Unit i 2 RSCS. A :ubstantial number of the remaining work orders were related to the ;
repair or re,slacemn'. of rod select switches. Completed work orders also j traced many eqci '
a1 functions to failed or inoperable reed switches used ,
'
in control rod tot 'ldicatio C.3. Corrective A; ,
) The I&C technician reviewed outstanding trouble tickets and work orders !
j recently initiated on the Rod Control System, and conferred with operations '
personnel with respect to the problem. Initial troubleshooting involved review ,
'
- of the full out indication on the full core displa This resulted in j replacement of several burned out bulbs and concluded with the I&C technician
'
asking if there were any other burned out bulb These efforts proved -
unproductive. Therefore, the I&C technician began a comprehensive examination !
7 of RSCS to determine the cause for the rod select block. After about two and -
j one half hours into his troubleshooting procedure, the technician found that RSCS was not sensing all the appropriate RPIS interlocks for control rod full out indication. The technician informed operating personnel of his discover l
, Thus, when the technician informed them of his findings, operations personnel stated that they immediately recalled the relationship between the RPIS and the
'
RSCS. There was a tracking LCO on control rod 22-03 related to a failed reed switch which provides the "full-out" indication for the control rod. The rod position indication was immediately bypassed to fullout and, as a result the
,
<
rod select block was remove Shutdown efforts then proceeded, j C.4 Conclusion The inspector reviewed the events leading to the determination that the RSCS
- had failed. The inspector also reviewed the events during and subsequent to 1 the I&C investigation of the purported failure. The history of RSCS failures j at Brunswick were also reviewed. As a result of these reviews, the inspector
'
concluded the following:
!
- *
The RSCS responded properly during the shutdown. The failure of
{ operations personnel to recall the relationship between lack of fullout
,
J !
.
O
indication for control rod 22-03 and the red block inserted by the RSCS accounted for the conclusion on the part of operating personnel that the RSCS system had failed. Had the review of LCOs been more successful, the time required to analyze the RSCS system would not have been expende * The fact that both Operations and I&C personnel failed to notice that rod 22-03, on the edge of the core, was unlit, by not methodically checking each ligh1 associated with the "A" rods, contributed to the delay in shutting down the reacto * The inspector reviewed the licensee's corrective acticns with respect to troubleshooting the RSCS system. The inspector believes thet the amount of time which was required to isolate the failed component was not inordinate. The I&C technician responsible for the effort displayed adequate knowledge and ability in troubleshooting the syste * A review of recent operating experience with the RSCS indicated that there l have been a number of failure events involving components which have functions that are important to the operation of the RSC Of the failures reported, a substantial number appear to be related to the failure of reed switches used in rod position indicatio Rod Select Switch 0.1. Equipment Failure Description At 0645 on July 14, an attempt was made to select control rod 34-03 for insertion during the shutdown sequence. The rod select switch (RSS) for this rod was depressed several times; however, the operators were unable to select l the rod. I&C personnel were requested to determine the cause of the problem.
l A review of recent failures involving RSS at Brunswick Unit 1 indicate a number of failures of rod select switches. Most of these failures can be attributed to the normal wear which might be expected at end of lif D.2. Corrective Action and Conclusion l
l I&C personnel determined that the failure 7f the switch was due to a burned l
inductor (coil) in the RSS circuitry. The switch was repaired and normal rod insertion continued.
,
The failure of the RSS appears due, primarily, to the effect of normal use and i aging. Interviews with personnel at the plant reveal that failures of RSSs are
'
not uncommon and work orders reviewed by the inspector tend to support this statement. Often, operators find it necessary to depress switches more than once and with differing degrees of pressure and at various points on the switches to have them f unction properl The licensee is contemplating the
.
l t
l l
_ - - - - - - - - _ _ -
.
.
installation of a new modular _ RSS system which employs "plug in" components which can be changed in a fraction of time currently required for the existing system's components. The inspector believes that a change to a newer, more easily maintainable system would also increase the reliability of the system by replacing components which are approaching the end of their useful lif Replacement of the system would substantially reduce the frustration operators experience during their efforts to select rods with faulty switche ftHR Injection Valve, F0178
_
E.1. Equipment Failure Description On July 14, 1988, while lining up loop B of the residual heat removal (RHR)
system to enter the shutdown cooling mode for Unit No.1 in accordance with Section 5.4 of Procedure OP-17, valve 1-E11-F017B failed to open on demand due to misalignment of the clutch mechanism assembly in the valve operator. There are two identically arranged RHR system loops in each of the two Brunswick Units, and each loop has a 20" x 24" angle valve to allow RHR flow into the recirculation loops. The RHR outboard injection valve 1-E11-F017B is the valve on loop B for this purpose. The F017B valves are routinely used to return flow to the reactor vessel during the cool down process in the plant shutdown process. All F017 valves were made by Rockwell and have Limitorque model SMB-ST operators. Attachment 2 contains a sketch of the RHR shutdown cooling mod Approximately 15-20 minutes prior to the event, Valve 1-E11-F017B was successfully closed per Step 5.4.B.58 of Procedure OP-17 in preparation for the starting of the RHR 1B pump. This was later followed by six unsuccessful attempts to open the valve per step 5.4.B.59 of Procedure OP-17. An auxiliary operator ( AO) stated that: "while proceeding to check the operation of the pump j
[1B RHR), a grinding noise was noted coming from the 1-E11-F017B. When I looked up at the valve, the handwheel was noted spinning smoothly at a high rate of speed as if the clutch was engaged." The operator in the control room stated that he "attempted to throttle the valve open to establish the RHR flo The indication light flickered but the valve never opened. Did not receive dual indication...., no indication of RHR flow, and no decrease in Rx leve Held switch in open for [approximately) 20 second Since there was no indication of flow the RHR pump was tripped at this time.... Attempted to open valve again with same results... . , Tried to open valve with same result The A0 checked the valve locally again and manually disengaged the clutch. We attempted to open the valve again but it wouldn't. We then closed the F0158 valve and attempted to stroke the valve again with no success. The valve indication flickered each time when the switch was operated."
As can be seen from the operator's report, several attempts were made to open l valve 1-E11-F017E without success. The licensee initiated immediate action to determine the cause of the failure without definitive result Upon disassembly of the operator it was discovered that the mating lugs on the sliding gear clutch and the flexible jaw clutch housing were badly damaged and would not engage. In addition, the sliding gear clutch was found jammed into ,
.
.
'
26 !
,
the solined insert. Attachment 2 contains copies of pictures of the damaged '
components. The operator of valve 1-E11-F017A was subsequently partially disassembled and inspected to determine if similar problems existe ;
Inspection results indicated that degradation of the clutch mechanism had not I
occurred. At the tiine of the AIT debrief on July 21, 1988, the reason for the i damaged . clutch assembly- had not been determined,
-
possible root causes e identified by the licensee (in order of probability) are listed as follows- i 3 :
,
'
Material failure over time, aggravated by a full torque backseating event l
that took place in 1987 when a jumper had been installed to bypass the limit switch,
Clutch mechanism assembly parts not meeting design criteria,
Clutch compression spring failure, and
- '
Operator erro '
Initial reviews performed by the licensee have eliminated the last three pos sibili tie t,. The first cause may be an adequate description of the failure but does not constitute the root cause for the failure,
,
To determine possible root causes of the operator failure, the AIT conducted !
]~
interviews, reviewed maintenance histories, studied inservice test data, l
~
Limitorque maintenance procedures, and literature available concerning aging and !
service wear of Limitorque operators. The results of the efforts are included
,
below, i
!
E.2 Equipment History
,i
,
j E. Maintenance History l
Maintenance of individual equipment, including preventive maintenance work, ,
requires the issuance of a work request and authorization, and a job number to ,
- get starte Typical examples would include packing leakage, which would be !
requested by the operating personnel, and greasing and lubrication, which would 3
. be initiated by the maintenance department. The maintenance department
'
l performs the requested work on a priority oriented basi Examples of maintenance history records for the four E11-F017 angle valves can be high- !
l lighted as follows:
1-E11-F017A i
Local position indicator was out of adjustmen '
l
) Adjusted limit switch to reflect problems associated with backseating of valve 1-E11-F017B event in 198 l I
i i
I t
,
__ . __ .
.
-. ,
i l
1-E11-F0178 ,
- Long history of packing leakage due to galled stem. The galled stem has !
'
never been repaired or replaced since its discovery sometime prior to '
198 * A modification was performed in 1987 which jumpered out the limit switc The jumper was not removed, consequently, when the valve was stroked open the motor continued to run, backseating the valve and burning out the *
motor. Based on schedular constraints and discussions with limitorque, the licensee chose not to open and inspect the operator for possible l
stress damag *
Local position indicator was out of adjustmen * Clutch mechanism failure while attempting to open valve on July 14, 198 E11-F017A
- Local position indicator found missing in July, 1988.
l 2-E11-F0178 ,
l
'
Motor drive gear slipped out of alignment causing the operator to be inoperabl '
'
'
Installed replacement operator gear part * Motor leads replace l Maintenance history indicates that the F0178 valves have had more service !
oriented maintenance problems than the F017A valves. This probably was caused ;
by the fact that F017B valves were used to initiate the shutdown cooling flow
'
, during plant shutdowns, while the F017A valves were infrequently used. It should also be noted that an-NRC inspector cbserved a plant craftman trying to ,
reposition the 1-E11-F017B declutch lever to the "motor" position during a repair evolution following the July 14, 1988 event. The Limitorque operating
.
'
instruction manual forbids this action. Although the mechanical arrangement of the operator should preclude this action, the observation reveals that other inappropriate manipulations of the operator could have caused damage prior to the event in questio E.2 b Surveillance Test History The four LpCI F017 valves are included in the Brunswick Inservice (IST)
Program, which requires periodic verification of their operational readiness in accordance with the ASME Boiler and Pressure Vessel Code Section XI. The tests required are 1) local leakage rate tests to verify the seat tightness of the valves, and 2) stroke timing tests to verify the degree of degradation of the operating mechanism of the valves. Results from local leakage rate tests are not relevant to the present failure problem; but stroke timing could provide sufficient indication to warrant inspection for degradation of the operato !
,
I
_ . _ . _ _ . _ . . .
t I
.
The stroke timing tests were performed by the operating personnel at the l control room without the benefit of local observation of the valve and operator (
l during the tes Results of stroke timing tests were delivered to the IST l group for evaluation to determine their acceptability including any required'
corrective actions. The results of stroke timing tests for the four valves during the past four years showed that:
1-E11-F017A ,
i
- Opening stroke times have been sporadic, varying from 20.0 to 2 ,
,
second .
L
- Closing stroke times have been fairly consistent with the exception of the last recorded surveillance which indicated a stroke time of 27.6 seconds ;
'
on June 11, 1988. The licensee is currently evaluating the stroke time varianc '
1-E11-F0178
- Opening stroke times have been relatively consistent since 1984 as they varied between 19.8 and 24 5 seconds. Two exceptions were readings of i 26.5 seconds on December 29,1985, and 18.92 seconds on July 16, 1988, following the recent operator repairs. The latest stroke time was over 4 j seconds les> than the previous stroke tim The licensee is currently evaluating the variance of this last readin *
Closing stroke times also have been relatively consistent during the same i time period, varying between 20.9 seconds and 23.9 seconds with the I
exceptions of three earlier readings in 1984-1985 period ranging from 2 !
to 25 seconds. These readings exceeded the IST maximum limit of 24 i seconds in effect at that tim l
! !
l 2-E11-F017A i
Opening stroke times have been consistent, ranging between 19.6 seconds I and 20.3 seconds, with the exception of one reading of 26 seconds obtained .
on June 26, 198 !
Closing stroke times have been consistent, ranging between 22 seconds and .
23.6 second E11-F017B
- Opening stroke times have been consistent, ranging between 19 seconds and 20 seconds, with the exception of one reading of 25 seconds obtained on l August 8, 198 l
- Closing stroke times have been consistent, ranging between 22.7 seconds and 25 seconds, with the exception of one reading of 19.5 seconds obtained on June 3, 198 .
..
Results from stroke timing tests gave no indication of pending failure of the 1-E11-F017B valve. However, as discussed above, the licensee performed stroke timing tests from the control room without the benefit of local observatio This minimized the effectiveness of the IST program. If the failure was caused by gradual wearing, as suspected, a local observer would notice the unusual noise caused by the grinding (slipping) of clutch lugs, and early discovery could have been mad E. Maintenance Procedures for Limitorque Operators Generic Limitorque operator preventive maintenance procedures developed by the licensee were reviewed to determine adequacy. The licensee has committed to accomplishing preventive maintenance every 18 months as recommended by Limitorque. The following procedures are for "Q" (safety-related) operators:
MI-10-25 - Electrical inspection of torque and limit switche MI-10-511 A-F - Gear case lubrication and miscellaneous visual inspections of valve and operator exterio The electrical and mechanical preventive maintenance procedures listed above appear to be adequate for their intended purpose However, additional preventive maintenance procedures involving selected valve operators (based on maintenance history, results from surveillance tests or importance to safety)
did not exist. Limitorque does not recommend any routine maintenance that would include disassembly of the operator unless an operational deficiency of some kind exists or is suspecte Because of the failure history of the clutch mechanism assembly, Limitorque established a limit en the number of allowable cycles for switching from motor to manual drive and back of 1000 cycles. Clutch mechanism degradation occurs due to the random nature of clutch assembly lug engagement and impact loading on the lugs. The random nature of clutch assembly lug engagement is caused by the following: When the operator is in the manual mode, the sliding gear clutch is locked into rosition with the splined insert and is separated from the flexible jaw clutch housin When the motor is started, a cam on the flexible jaw clutch housing contacts the clutch trippers which release the clutch fork, allowing the sliding gear clutch to come into contact with the flexibe jaw clutch housing. The lug engagement can vary from direct lug contact (lugs not synchronized) to full lug engagement. With time, the clutch lugs will wear to the point of intermittent engagement and eventual failur This condition is further aggravated if the sliding gear clutch is not absolutely free to move quickly on the hollow drive shaft and engage the flexible jaw clutch housing once the motor has been energized. Degradation of the clutch mechanism assembly could be diagnosed by having an operator stationed at the valve during inservice testin E.3 Corrective Action E. Unit No. 1 Valve 1-E11-F017B operator was repaired and passed the post-maintenance
.
.
testing. Repairs consisted of clutch mechanism assembly replacement. While Unit No. I was shutdown, the licensee inspected valve 1-E11-F017A operator internals to determine if excessive wear existed. Only normal wear was observed. The licensee was performing a root cause analysis of the valve operator failure. The analysis was not complete at the end of the inspectio Hardness testing was performed on the failed clutch parts to determine material adequacy. No defiencies were note l E. Unit No. 2 The licensee had committed to opening and inspecting both 2-E11-F017A & B valves during the next outage, scheduled for July 2 Particular attention will be given to the parts which failed in the 1-E11-F0178 valve operato*.
Additional corrective actions will be formulated based on the results of l inspections completed on Unit No. E.4. Conclusion ,
The only other F017 valve experiencing operator problems (based on !
maintenance history) was 2-E11-F017B which had some internal gear parts j replace '
'
Valve 1-E11-F017B has experienced chronic packing leakage due to a galled stem. This condition has existed for over 6 years and is not scheduled for repair until the next outag *
The original FSAR and the latest revision to the system description both
,
'
require the F017 A & B valves to open and close within 24 seconds. This limit was revised to 30 seconds in Amendment No. 4 to the Updated FSAR, dated June 2, 1986. However, a review of IST records showed that prior to June 2, 1986, the 24 second valve stroke limit was violated several time There are no ir.dications that the valves were declared inoperable. ASME Section XI required action was not completed, nor was any corrective action performe Inservice Testing of valve 1-E11-F0178 failed to predict valve degradation and ultimate failure of the clutch mechanism assembly. The Limitorque i operator is designed for a finite number of applications, and is subject I to normal degradation and deterioration. The licensee should consider a selective PM based on usage and maintenance histor *
During normal accomplishment of IST of notor operated valves, the licensee does not have a mechanic stationed at the valve to observe valve operation. The purpose of IST is to determine the readiness of pumps and l valves for their intended function, and this individual is to listen for
'
any abnormal valve, motor or operator noises. Without the benefit of local verification, the IST loses most of its intended value, and becomes ,
a formality. Misalignment of the clutch assembly may have been discovered )
l l l
l
.. .- . - -. . . .
'
J ;
J .
l
31 l t
earlier during testing and could have prevented the operator failure, The !
'
inspectors are aware of several licensees who use this approach for -
inservice testin .;
i In summary, valve 1-E11-F0178 failed to open because of lack of engagement of j the operator clutch assembly caused by deformation of the clutch lugs. This l event was allowed to happen because of 1) ineffective inservice testing, and 2)
'
,
- lack of knowledge concerning maintenance and operational limitation of the
, operator. The largest single contributor to the failure was the ineffective :
! testing. The purpose of performing inservice testing of valves is to be 1 assured that the valve is able to perform its safety-related functio Stroke i time readings within Technical Specifications simi+s is one indicator of valve
'
! operabilit This alone is not sufficient, howev.r, shce there are other !
{ signs and indicatioas associated with the testing V at show that the deteriori-
-
'
{ zation process has started, which cannot be realized without direct observa- ,
! tion. To meet the IST program minimum requirements only would reduce the l 4 effectiveness of the program. Corrective actiors proposed by the licensee must j not be limited to the E11-F017 valves but should have a more generic J implication.
f Condenser Circulating Water ;
i i F.1. Equipment Failure Description .
As detailed in the Sequence of Events section of this report, multiple CWIP i i trips occurred starting at approximately 0250 on Friday July 15, 1988.
- Prior to the CWIP trips, the pumps were aligned as follow
,
Unit 1 (Hot Shutdown) Unit 2 (100*4 Power) ;
i - CWIP 1C, 10-Operating CWIP 2A, 2B, 2C, 20-Operating i i - CWIP 1A, IB-Off L
The Unit 1 CWIP 18, 1C, 10, and Unit 2 CWIP 2A, 2B, 2C traveling screen panels l j are equipped with fine men screen material with approximately one millimeter l
square openings, and resemble fine mesh porch screen. The CWIP 1A and CWIP 2D i traveling screens are equipped with coarse mesh screen material with ;
approximately 3/8 inch square openings.
, l l The IB CWIP was started at 0112 on July 15, for the purpose of conducting a
{ routine flush of the debris filters installed in each inlet line to the l
, condenser water boxe ,
t At 0249 the IB CWIP was secured and approximately one minute later a h!gh D/P 1 trip was received on the IC CWIP traveling screen. High traveling screen 0/P I trips continued to be received at one minute intervals and the pumps tripped in ;
! the following sequence: 10, 2A, 2B, 20. Unit I was without CWIP for )
l approximately one minute when the operators started the 1A CWIP at 0252 1
- followed by starting 1B CWIP. The 2D CWIP remained in operation for Unit l 1 Unit 2 power level was reduced to approximately 40*4 power to accommodate heat ;
i transfer capability of one CWIP cooling water flow through the condense l
! I
,
________________________________.___.____.______________________________________________i
. . .
-
. _
.
From 0302 to 1657 on July 15, several attempts were made to start Unit 2 CWIP The pumps continued to trip ca high D/P across the traveling screens. Inspec-tion of the traveling screens at approximately 0400 did .not indicate the screens to be clogge Maintenance activities were formulated during a planning meeting held on the morning of July 15. The maintenance activities were initiated at apprximately 1100. The tubing associated with the traveling screen D/P cells were checked to ensure they were . not clogged. Air is continously bubbled through the tubes to assist in keeping debris out and the tubing was found to be clear. Calibration checks of the alarm and trip set point for the Unit 2 D/P trip set point was found to be low at 24 inches of water in lieu of the required 36 inche Unit I high D/P trip set p; int was found to be satisfactory at 36 inches of water. After removing several_ fine mesh screens from the traveling screen panels, the licensee _ was able to start the 28 CWI Between 1615, July 15 and 0515, July 16, the fine mesh sc een removal was completed for CWIP 2A, 2B, 2C and 10. During their removal licensee personnel discovered that iery large quantities of (which is later identified to be)
"skeleton shrimp." All fine mesh screens were scraped and flushed to restore then to a clean statu At 1900, on July 16,'Jnit 2 was returned to 100% power. Unit i remained in hot shutdown to affect repairs on various plant equipment, Operating status of the CWIPS and fine mesh screens after screen changeout is as follows:
U1 U2 CWIP 1A -operating- coarse screen CWIP 2A -operating- coarse screen 18 -operating- fine screen 20 -operating- coarse screen 1C off - fine screen 2C -operating- coarse screen 10 off - coarse screen 2D -operating- coarse screen F.2. Equipment History The inspector examined operator legs, plant procedures, and maintenance procedures and conducted interviews with plant operators, maintenance engineers and environmental specialist to determine the cause for the multiple CWIP trips and to assess licensee maintenance activities, modifications.and environmental activitie During the removal of the fine mesh screens from the pump traveling screen panels on July 15, licensee personnel noticed large quantities of skeleton shrimp on the screen Due to the transparency of the skeleton shrimp they are not readily visible when wet, which explains why they were not detected by the operators when visual inspections were made while the traveling screens were in operation. The licensee indicated that scraping methods had to 1 be used to affect their remova lushing with water under pressure would not j dislodge the creatures. Attachment 1 contains a sketch of the creature and is identified scientifically as a "Capre11a." j I
!
l
.
, -
_ , - . - - , - - _-y,, , - - - . , . - .
%[
. - .
9'
.
The environmental specialist initiated sampling at approximately 1730 on July 15. The sample was taken over a five minute period using a large net with a sample bottle attache The sample showed a skeleton shrimp count of approximately 126,000. Typically previous 5 minute samples showed a count of approximately 232. The sample was taken on the discharge from the condenser .The size of the skeleton shrimp were noted to range in size from a few tenth of an inch to a half an inc The licensee environmental personnel worked with the University of North -
Carolina which has a specialist on its staff with expertise on the Caprella and its habits. Caprella do not thrive in water where salinity-is below 24 parts ~
per thousands (PPT). They require salinities above 30 PPT. The plant intake structure is fed from the Cape Fear River. Various creeks and the Atlantic Ocean provide water mixing into the Cape Fear River. The drought season has cause the salinity in the river to run between 32 and 34 PPT. Salinity readin recorded during the past several years generally show salinity to be above 26 PPT. Caprella are not good swimmers and normally attach to seaweed, the bottom and sides of the river and on growth attached to concret Routine samples indicate a Capre11a count of 232-235 per 5 minute sampl The inspector determined that the only operation identified taking place prior to the CWIP trips was the securing of the 1B CWIP. In addition, walkdowns of the CWIP, traveling screen and the plant intake structures were mad The service water system did not experience traveling screen clogging during the event primarily for two reasons. The service water traveling screens all have coarse mesh screens (3/8 inch mesh) and the water velocity is toward the much larger CWIPs which have much greater velocity than the service water pump These stronger current flows toward the circulating water intake structure tends to keep debris away from the service water intake structure. The plant has experienced only infrequent multiple CWIP trips on both units, the most notable being that due to jellyfish in 198 The inspector reviewed the following plant procedures for technical content, sufficient detail and ci tri ty with respect to operator action No discrepancies were identif e Operating Procedure - 29, Circulating Water System
-
Operating Instruction - 03.4, Operator Daily System Check Sheets
-
Annunciator Procedure - UA-01, CW Screen Differential Pressure High
-
Operating Procedure - 29.1, Screen Wash System The inspector reviewed maintenance activities associated with the CWIPs, intake structure traveling screens and D/P 1 strumentation associated with the traveling screen Event 4 <e Maintenance instruction, OPM-LUB500 schedules at various intervals )
n..pection and lubrication requirements for the CWIPs motor and shaft bearings, l traveling screen link chain joints, bearing and other moving parts. A detail l schedule is presently being planned where refurbishing of the traveling screens I as a unit will be conducted on a two year cycle. Maintenance Instruction, MI-03-3A and instrument routing schedules are used to calibrate instrumentation installed in the circulating water system at specified frequencie The
'
l
_ ._, _ __ _ . . . . , , , _ _ _ - - , , - _ . _ _ , , _ _ _ _ . _ . __ ~ , . . .--- , _
. . .
'
i
,
.
licensee noted as a result of the calibration checks conducted on Unit 2 after the CWIPs trips, that traveling screen high D/P setpoints were set low at 24 inches of water in lieu of 36 inches. A review of plant modifications showed that modificat$on 81-224A-D (approved February 1982) was outstanding on Unit 2 with regards to upgrading the traveling screen D/P pressure switches to a 36 inch trip setpoint. The licer.see did not provide any specific reason for this modification remaining open. In addition, calibration frequency for the D/P instruments were assigned and placed into the instrument routing = schedule progra F.3. Corrective Action The environmental group has increased its sample frequency from weekly to daily and increased the number of locations where samples are taken. In addition, a i rig is being devised to scrape and sample the bottom and sides of the intake canal, intake structure, and rive Engineering and operation groups are trending traveling screen D/P readings and changes that occur during routine operations of the circulating water syste F.4. Unit 2 Effect The loss of three out of four CWIP required an immediase reduction of reactor l power and steam flow to the unit condenser to compensate for the decrease in !
condenser vacuum. The reduction of plant power to approximately 40% was ;
accomplished in a two to thice minute time peri- Condenser vacuum changes l were minimal, decreasing from 28" Hg to apprt -stely 26.5" Hg and were returned to normal values once power was reduce F.5. Conclusion Based on the finding as discussed in this report the CWIP were tripped on high D/P across the traveling screens due to massive clogging of the fine mesh screen for those pumps so equipped. In all cases where coarse mesh screens were used, the CWIPs remained in operation. In addition, af ter maintenance ,
personnel become directly involved, the time that thcy took to troubleshoot the l high screen D/P CWIP trips, affect modifications, calibrations and the subse- '
quent removal of the fine mesh screens (approximately 200 panels per traveling screen) from four CWIP appears reasonable and effectiv j
,
The effects of not having plant modification 81-224A-D completed on Unit 2 did not appear to be adverse, however modifications should not be allowed to remain oper, for sucn a long period of time without a formal re-examination and manage-ment approva The National Pollutant Discharge Elimination Permit (NPOEP) that the licensee has with the State will require that the fine mesh screens be reinstalled on three of the four CWIP for both units. Since this tripping of the CWIP could potentially reoccur at a future date, the inspectors believe the licensee should make final assessment in this area and develop the following:
-
Predictability of the skeleton shrimp to return in massive number .
a
.
-
Provide information concerning this event to plant operators and establish guidance as a future reference in a form deemed appropriat Equipment Safety Significance and Operability G.I. Safety Significance / Safety System Performance G. Rod Worth Minimizer'(RWM)
During the shutdown of Unit 1 the operators bypassed the RWM functions. The RWM control system is not required for safet The RWM function does not interfere with normal reactor operation, and in the event of failure does not, itself, cause unanalyzed rod patterns to be established. The RWM functions may be bypassed and its block tunction disabled only by specific procedural control initiated by the operato Specifically, the RWM shall be operable when thermal power is less than 20% of rated thermal power. In the event RWM.is not operable at less than 20% power, Technical Specifications allows continued control rod movement provided that a second licensed operator or other quali-
. f ed member of the technical staf f is present at the reactor control cont le and verifies compliance with the prescribed control rod patter Through interviews of personnel and review of operating logs the team concluded that-this requirement was me The system itself did not interfere with normal operation, nor cause unanalyzed rod pattern G. Rod Sequence Control System (RSCS)
RSCS prevented the eperator from selecting control rods for movement while completing a controlled shutdown. The system is required for safety. The function of RSCS is to prevent the operator from moving a control rod out of sequenc At the time the operators were prevented from selecting control ,
rods, the reactor was being operated in the "50 percent rod density to the l Preset Power Level" range. In this operating range, the RSCS is interlocked to 1 allow only one specific control rod sequence to be manipulated; i.e., either l t
the A or B sequence of control rods. The circuitry either inhibits or- permits 1 all of the rods assigned to a sequence unit to be moved. In this case only sequence 8 rods could be moved, and sequence A rods must remain in the full-out position. Due to the failure of the full-out reed switch on control rod 22-03 (an "A" sequence control rod), the circuitry was not satisfied and a select block applied to the Reactor Manual Control System (RMCS). This action by RSCS was appropriate and the system rssponded correctly by preventing the operator from moving any control rods. Thus no failures of the RSCS occurred. The reed switch failure had minimal safety consequence .
l G. Rod Select 5 witch '
The Rod Select Switch allows the operator to select a control rod for movement assuming all prerequisites are satisfie The switch is part of the Reactor Manual Control System which is not safety related. The failure of the select switch for control rod 34-03 prevented the operator from selecting and moving this control rod by normal means. The RMCS does not include any of the circuitry or devices used to automatically or manually scram the reactor; this
.
.
means of inserting the control rod was still available and not impeded by the failur This event had minimal safety consequences, nor were procedure controls violate G. RHR Injection Valve F017B The failure of RHR injection valve F0178 prevented the operators from using loop B RHR in the shutdown cooling /LPCI modes. The shutdown cooling and LPCI mode of RHR are safe shutdown functions. Alternate methods of shutdown cooling included using the main condenser, or loop A of RHR. Operations personnel elected to use the main condenser as the heat sink allowing loop A of RHR to serve as the LPCI path. For most single failures which could result in the loss of shutdown cooling, no unique safety actions are req 11 red; in these cases, shutdown cooling is simply reestablished using other, normal shutdown cooling equipment. This configuration, by itself, posed minimal challenge to the safety status of plant shutdown cooling mode The ability to reach a cold shutdown condition remaine No Technical Soecifications were violate G. CWIP Trips While Unit I was in Hot Shutdown and Unit 2 operating in Condition 1, a loss of several CWIPs occurred. The Condenser Circulating Water system is not safety related. The consequence of a loss of CWIPs is essentially a inss of vacuum transient or in a more severe case, a loss of auxiliary power. The reduction or loss of vacuum in th: main turbine condenser will trip the main and feed-water turbines and close the main steam line isolation valves and bypass valves. The resulting transient is dependent upon t'e rate at which the vacuum is lost and the operating condition of the reacto For Unit 1, in 't Shutdown, the condenser was being used as the ultimate heat sink for decay heat removal. Loss of this heat sink would require alignment of the A loop of RHR for shutdown cooling (the B loop was inoperative with F017B out of service). At the time of the CWIPs tripping, the A loop of shutdawn cooling had not been prepared for servic This evolution would have taken considerable time and issulted in heat up of the reactor due to the loss of decay heat remova The operators were, however, able to maintain a minimum amount of Circulating Water flov 'a the condenser thereby preventing this condition and challenge to the Hot Shutdown status of the reacto For Unit 2 the failure was of a more immediate concern. The rate of vacuum loss was controllable by initiating a turbine load reduction. Recirculation flow was decreased to lower the load of the turbine to within the capacity of the remaining CWIP. FSAR analysis shows that an instantaneous loss of all CWIPs causes condenser vacuum to drop to the turbine trip setting in six, second The partial loss of CWIPs which occurred on Unit 2 does not approach the severity of the instantaneous loss of all CWIPs, thus this event was bounded by the safety analysi The combination, though, of inoperable CWIPs with a degraded shutdown cooling capability and with HPCI inoperable, did place the unit in a condition where additional circulating water and shutdown cooling ;
problems could have resulted in significant challenges to the operating staf '
l
!
..
r -
.
G.2. Operability and Reportability The AIT evaluated the events of .the shutdown of Unit I for applicablity.of 10 CFR 50.72 and 50.73 reports. The licensee has made' the following reports' i associated with the events leading to and. involving the ' shutdown of Unit 1 on July 13, 1988:
-
A 10 CFR 50.72 report to the NRC on July -1,1988, (0308 hrs.) concerning ;
inoperability of HPCI due to motor failure of HPCI E41-F001 Steam Supply
'
Valve. A written report will be submitted to the NRC (LER-1-88-17), ;
-
A 10 CFR 50.72 report to the NRC on July 5, 1988, (1313 hrs.) concerning HPCI inoperability due to motor failure of HPCI E41-F006 Injection valv .
A written report will be submitted to the NRC (LER-1-88-19). l These reports were reported under the conditions of 10 CFR 50.72.b.2.iii. No other reports to the NRC were inititated (or required) by the licensee for the events which occurred during the shutdown of Unit 1. The 10 CFR 50.72 reports I were initiated in a timely manne No difficulties were noted in the l communciation of these events to the NR !
i G.3. Conclusion The shutdown of Unit I was complicated by numerous failures of plant systems j that required extensive maintenance and operator expertise to overcome. The l operational considerations associated with these failures' were significant in i that the operators were required to mitigate the effects of multiple malfunc-tions; e.g. , HPCI being inoparative during a shutdown, an inability to esta-blish normal shutdown cooling when required, failure of the RWM and its backup RSCS, using the condenser for decay heat removal and a subsequent loss of CWIPs. Multiple failures can perturb the situation to an extent that normal evolutions become quite complex with a resulting increased challenge to the operating staf In this event the failures did not prevent the safe shutdown of the facility and did not violate the safety analysis. No safety system injection or actua-tion occurred during the planned shutdown of the reactor, nor were safety limits violated. The effects of the CWIP trip on Unit 2 were within the bounds of the FSAR safety analysis for loss of condenser vacuum and loss of auxiliary powe The potential existed though for serious degradation of the Hot Shutdown status of the reactor. The loss of CWIPs while using the main condenser as the decay heat removal source coupled with the failure of F017B created a condition which mandated expeditious and effective diagnosis and repair to avoid possible additional challenges to the operator .
o
I PERSONNEL ACTIONS Operations Involvement During the shutdown of Unit 1 on July 13, 1988, operations personnel responded to numerous unexpected failures for which deliberate action was net.essary to continue with a controlled shutdown. Their goal was to' safely shuti.own Unit 1 prior to the expiration of the Limiting Condition for Operation for HPCI that expired at 1102 on July 14. The actions of operations personnel during this evolution were reviewed by the AIT. The review concentrated on a performanced based evaluation of their responses to five failures of plant systems: 1) RWM, 2) RSCS, 3) Rod Select Switch, 4) RHR F0178, and 5) CWIP trips. In addition, the response of Unit 2 operations personnel to the CWIP trip was also reviewe The AIT considered several areas of performance in the review:
Diagnosis of Events / Conditions Understanding of Plant / System Response Response to Alarms / Failures Manipulation of Controls Use of Procedures and Technical Specifications Supervision Communications Crew Interactions A.1. Rod Worth Minimizer During the snutdown the RWM exhibited improper performance by enforcing control rod blocks at 40% powe This was indicated by the control rod withdrawl permissive light extinguishing and indications / alarms on the RWM. The failure was immediately identified and appropriate action was taken to defeat the RW A second licensed operator was used to verify control rod movement in accordance with procedure Historically, the RWM has been a problem for the operators, the latest event occurring on June 8, 1988, when a similar failure occurred. Reoccurring failurts of thir system have led the operators to anticipate the necessity for bypassing its function and continuing with control rod movement CP&L has replaced this system on Unit 2 and intends to replace the Unit 1 RWM during the next refueling outage. The operator response to this event was both timely and prope A.2. Rod Sequence Control System While continuing with the unit shutdown RSCS initiated and blocked the selection of centrol rods for movement. The problem was not imcediately obvious to the operators primarily due to the lack of diagnostic aids provided for RSCS. The SR0 was aware of an existing tri uble ticket associated wi'.h the Rod Position Indication System (RPIS), however, he did not correlate this with the interface of RPIS and RSCS for full out indicatio The operators continued to diagnose the problem by checking for full out position indication
._ _ _
.
.
on the full core matrix and plant process computer. Maintenance technicians were dispatched to trouble shoo Approximately one hour into the trouble shooting efforts, they suggested the possibility of _ a full out indication proble Miscommunication between the control operator and SRO may have prolonged maintenance activities. The control operator apparently assumed the-SR0 was aware of the problem with control rod 22-03 and did not reiterate the problem with RPIS on this particular control ro The time expended by 'the Instrument and Control personnel to isolate the problem added to the delay of the shutdow Ultimately, the delays necessitated a manual scram of the reactor to ensure shutdown and cooldown could be accomplished prior to the LCO deadlin The Shift Operating Supervisor had notified the on-call manager (Manager of Operations) and had discussed contingencies with respect to the cooldown. It was decided that if RSCS was not repaired by 0330 hrs. the unit would be manually scramme Concurrence was obtained from the plant manager. RSCS was restored prior to 0330 hrs. and normal control rod insertion continued without difficulty. No apparent training deficiencies are noted in this event. The operators were aware of both the failed reed switch and the full out interlock, the lack of correlation of these two appears to have been a cognitive oversight by the operators. The inspectar discussed the feasibility of maintaining failed reed switches in the "bypassed" position during normal operations with operations management. Operations is reconsidering the feasibility of maintaining the switches in bypass, but as of now no changes in policy have occurred. Several
, problems could result from this manipulation, including masking of additional
"bypassed" switches, and a continuous annunciation of this condition on the control boar A.3. Rod Select Switch After RSCS was returned to service at 0250, selection of control red 34-03 was prevented by a failed rod select switch. Several techniques were attempted to allow selection of the control rod; e.g., ensuring deselection of all other rods, holding in the selection pushbutton and attempting to move rod 34-0 The operators ascertained that the rod select pushbutton for rod 34-03 had .
failed and maintenance was initiated. No further control rod movement was I initiated; doing so may have resulted in abnormal control rod pattern This I type of failure is recurring and random in nature. Only minimal preventive i maintenance can be conducted on the subject switche These switches are 1 bardwired and CP&L has considered upgrading the rod select matrix to a newer i model which would allow quick replacement of failed switches. Initially some
'
confusion may have existed with the operating crew on the cause of the inability to select the control rod, however, t'.e overall operator response to this event was appropriat A.4. Loss of Shutdown Cooling The reactor was eventually scrammed and cooldown to the point where RHR I
shutdown cooling was to be initiated. While aligning RHR for this mode the control operator attempted to open the RHR injection valve F017B and the valve failed to ope Flow had not been observed by the control operator and the valve indicated closed af+.er 20 seconds of operation. After two unsuccessful
.
.
4 attempts to open the valve the pump was secured. The open signal was causing damage to the limitorque actuator, however, the operator ce;1d not be aware of this condition from the contral rtom. An auxiliary operater did identify the malfunction of the valve and notified the control operator of a grinding nois After manual manipulation o: the valve and resetting the tnermal overloads by the auxiliary operator the control operator attempted to open the valve four more times. Further investigation by the auxiliary operator reported the valve motor very hot to the tauc Repeated attempts to operate the valve were unsuccessful. This actian contributed to serious degradation to _the actuator of RHR F017 During this evolution the 9HR pump operates without minimum flow protectio Prompt shutdown of the RHR pump is necessary to prevent pump damage due to operation under "dead-head ' conditions. The control operator secured the RHR pump in a sufficient amount of time to preclude any damage to the pump itsel Subsequent to the loss of RHR loop B for LPCI and shutdown cooling the unit remained on the condenser for decay heat remova i.oop A of RHR was dedicated to the LPCI injection path in accordance with the Te:bnical Specification !
requirement for one coerable loop of LPCI while in Hot Standby. In the event
)
the main condenser became unavailable, operations would have aligned Loop A RHR i into the shutdown cooling mode of operation and entered the action statement l for LPCI. The diagnosis of the event by operations was accurate and their i response to the loss of one loop of shutdown cooling was prudent. Mani*culation i of controls for the F017B should have been more conservative in that continued attempts to operate the valve, after diagnosis by the auxiliary operator, may have contributed to the degradation of F017 l I
A.S. Circulating Water Screens Both Unit 1 and Unit 2 operations personnel had to respond '- the multiple loss of the CWIPs. Annunciations were received identifying the .use of the pumps trippin9; however, the root cause was less obviou The operators considered I that low tide had somehow contributed to the pumps tripping. Their' initial l decisiuns were effected by a similar event which had occurred previously. The '
previous event occurred approximately at low tide. Being within two hours from I low tide and the observation by an auxillary operator that the screens appeared clean led the operators to believe low tide was again a factor. Contributing to this belief was the fact that repeated attempts to start fine mesh screens were un:urcessful during the hours immediately preceding and following low tide. Not until well past the next high tide and after removal of several screen sections, was it determined that screen plugging was the root caus '
Due to the innocuous nature of the failure, diagnosis of this event was extremely difficul Its impact was mitigated by t'ie operator's ability to maintain in operation those PWIPs with course mesh 3creen .
.
The Unit 2 operators responded to the event by reducing recirculation flow, avoiding the region of instability for core power / flo Reduction in power enabled the unit to remain within the capacity of the running CWI The control operator took action to mitigate the loss by shifting the Steam Jet Air Ejector to full load thereby assisting in . maintaining condenser vacuu In addition, the rapid reduction in recirculation flow initiated a feedwater transient which prompted the operator to take manual control of 2B Reactor i Feedwater Pump. The operators continued their efforts to restore the CWIPs by 1 attempting multiple restart Trouble shooting efforts by maintenance '
personnel was initiated to assist in restoration of the CWI j i
The operators on both units accurately diagnosed the event and understood the !
effects of the pumps tripping. Timely action was taken on Unit 2 to respond to l the los Recognizing the degraded shutdown cooling condition, operators should have requested that maintenance personnel troubleshot the CWIPs while waiting for high tide conditions, instead of relying solely on the t1aai effect posssibility. Although an inordinate amount of time was expended in restora-tion of the CWIPs, ultimately, effective trouble shooting stratagies were used ,
- o locate and repair the system. In the review of this event no violation of ;
procedures was noted, nor any training deficiencie The operators on both !
units responded satisfactorily to the even j A.6. Conclusion Operator dependence on rel bole equipment is crucial to their safe performanc Many of the failures d.aring this event were associated with known or reoccurring equipment malfunctions. Management must be aggressive in the !
correction of these cypes of deficiencies to ensure operator performance is not j hampered by knowr. and/or recurring equipment malfunction The operators had '
to contend witn ar. inordinate number of malfunctions during this shutdow Overall, the operttion department's performance on these events was satisfactor Several areas where the need for improvement is indicated are:
1) Status of failed full-in/out RPIS switches (operations has no policy on the permanent status of these switches; i.e., should the bypass switches remain ,
bypassed or returned to normal), 2) Operator awareness of indications for l failed rod select switches (diagnosis of Reactor Manual Control System ,
failures), 3) Operator awareness of the consequences of motor operatea valve !
failure; e.g., operating a valve that responds abnormall l Improvement is also needed in the area of response to equipment failure Plant and operations management must improve their resolve to ensure equipment '
malfunctions are adequately responded to in a timely manner. As evidenced in i the investigation of the RSCS and CWIP functions, the burden of resolution to these problems rested with the operations staf Operations was required to either operate around, or otherwise experience long delays in meeting their expected goal of shutdown by 1102, on July 1 This situation places additional stress en the operators that could b: alleviated by aggressive attention to equipment problems by plant managemen .
.
42 i Management Involvement /0versight on F0178 Valve Failure Interviews were conducted with senior, mid-level management, supervisory and engineering and maintenance personnel involved in repair of the motor operated valve (F017B) which had failed to open at 1750 on July 14. The purpose of these interviews was to ascertain the degree of management involvement in the oversight and the decision making process associated with the activities in the evaluation, repair and post-maintenance testing of the valv B.1. Senior Management The Vice President, Brunswick Nuclear Project, was briefed by the Plant General Manager on July 15. During mid-morning, the Maintenance Manager and. support engineers showed him the damaged parts and with the aid of drawings explained how the declutching mechanism works. Also discussed were actions to be taken for the three other similar valves onsite. Another meeting at about 1600, on July 15 with cognizant engineers and their supervision dealt with inspection of other gearing in the actuator for damage. He emphasized that all damage-had to be identified and corrected and also asked if outside expertise from other CP&L organizations or Limitorque were require The purpose was to ensure that people were not taking short cuts and resources were availabl On the morning of July 16, the valve problen was discussed with the Senior Executive Vice President during a routine plant status cal On July 16 and 17, the Unit i F017B and A were also included in telephone conversations with the Senior Vice President, Nuclear Genert. tion and the Senior Executive Vice Presiden '
The Plant Manager was informed of the valve problem on a July 15 morning briefing. During this briefing, he inspected the damaged parts and indicated he listened attentively to the dialogue between the Manager of Maintenance, maintenance and engineering personnel to insure that the right questions were being asked, concerns were being enveloped and that actions were being undertaken to resolve valve concerns and that activities were under way to investigate the root cause of the proble The Plant Manager : stated that he issued no specific directives since he had quite a few discussions with the Manager of Maintenance and he felt satisfied with the direction and the manner in which the Manager of Maintenance was vigorously pursuing the resolution of the valve issu B.2. Mid-Level Management i l
The Manager of Maintenance was notified of the F0378 problem when he contacted '
the plant for a status at approximately 2130, en July 1 He had been in Washington D.C. earlier that day to discuss 9alve problems with the NR Earlier attempts to contact the plant hao been unsuccessful because telephone lines into the Southport area had been cut by county workers. During the I conversation he verified tha+ the right people were involve Normally he is briefed every mo,'ning by his principal sub;rdinates at 0730 on the status of major work efforts in progress. However because of the significance of having another safety related valve failure, he was briefed on its status upon arrival onsite, before 0700, on July '5, by the maintenance support staff superviso .. .-_ ..- _ _, - - - .-
_ _ _ _ _ _ _ - _ _ _ - _ _ _ _ _
. .
.
He indicated that he was basically in the receive mode versus the direct mode; however, he did instruct his staff to be sure to adequately. document the inspection results and activities associated with the inspection of the F0178 valv He was informed tha t MP-57, Limitorque Valve Failure Analysis and Troubleshooting Procedure, .as being implemented. -Immediately prior to the 1130 PNSC meeting, he was briefed on the current status by the maintenance support engineer and hi; associates and supervisors involved in the F017B proble He was shown the damaged part They discussed potential failure modes, availability of spare parts and emount of disassembly and inspection required of the actua'cor and valv From this meeting, it was decided that the 1-E11-F017A and similar unit 2 valves be looked at. Also the ~ Manager of Maintenance directed that technical support management concurrence be sought on the technical support engineer's position concerning the reuse of a spring. In general on July 15 and 16, he was continuously and informally updated of the valve status as new information became availabl Other specific actions associated with the overview of the F0178 problem included briefings of the Plant General Manager on the morning of July 15 and-periodically thereaf ter; discussion of the issue at the 0815 management meeting and the 1130 PNSC meeting on July 15; and tour of the reactor building on July ,
16, specifically to ensure that personnel were at the F017B when it was being i stroked and tested after reassembly. In addition, the: inspector observed the Maintenance Manager witnessing the disassembly and initial inspection of the declutching mechanism on the Unit 1 F017 B.3. First-Line Supervision q
An engineering supervisor was in the control room when this incident occurre l Maintenance personnel were informed of the valve problem at approximately 1900, l or one hour after the valve failed to ope Activity continued during the !
night involving preparation for and the actual partial disassembly of the valve l actuator to identify damaged component On July 15, a maintenance valve specialist and a technical support engineer were assigned to resolve this issue. These individuals interfaced well in evaluating the most probable cause of failure, determining what valve parts required replacement, discussing issues with control operators and the auxiliary operator who observed the malfunction of the valve as well as reading the control room and auxiliary operator's write-up of this event. A Limitorque representative was contacted to discuss possible or probable failure scenario Mechanical and electrical maintenance craftsmen were briefed on the details of I the job prior to wor The inspectors observed that foremen witnessed performance of the craft wor In addition, major evolutions were also observed by the maintenance support engineer and/or by the technical support enginee B.4. Decision Process The troubleshooting and repair decision making process was determined by the inspectors to be a group interactive process of cognitive members. Members involved included mechanical support personnel, mechanical maintenance personne', I&C personnel and their supervision up to and including the Manager
.. . - -.
.
44 of Maintenance. In addition, engineering support'was provided by Technical Support. Decisions tende1 to be shared among groups of personnel and usually _
evolved from discussions among two or more levels of management with their cognizant subordinates. Th?s sharing of information made it very difficult for the inspectors. to determine what levels of management made the many decisions on actions taken during the re:overy from this failure. However, the ' approach worked effectively in this it. stance . becaase , of the experience level and personal working relationships developed by the "team members" gained from addressing similar types of valve / actuator problems ~ in the pas The inspectors recognized that the NRC and CP&L sensitivity to this ftilure necessitated the involvement of all levels of site management in the details of this particular probicm. Using this type of management style, routinely, would tend to shift the technical-decisions responsibility and accountability to the senior manager involved. This approach is effective when management has technical expertise and experience in the technical area of the problem but could be a pitfall if there is weakness in the are For example,' management may be mostly mechanical, hence could function well for mechanical . types of ,
problems but could be ineffective for electrical type problem Determination '
of the overall effectiveness of maintenance management in other than the scope involved with the F0178 valve problem was not evaluated by this inspectio B.S. Cy clus-Failure of FL .8 was immediately recognized by management as a significant event, as another safety related valve f ailure and as 'a critical path activity for the outag There was an appropriate level of senior management j participation in the resolution of this issue with strong participation by q mid-level managemen While informal, there was very good and real time coordination between the maintenance and technical support staff who also exhibited careful deliberation and attention to detail in addressing this issu OVERALL CONCLUSION Although the shutdown of Urit i from July 13 through July 15 was a difficult l abnormal evolution, the overall effort was handled in a satisf actory, effective I
manner. With respect to the specific equipment failures, the full out reed switch problem affecting RSCS could have been identified sooner through better operator awareness of system interaction and maintenance wor The actual i troubleshooting and repair was conducted ir; an appropriate manne As the '
F0178 failure was such a highly unusual failure and on the critical path for Unit I restart, it received an extensive amount of technical and management attention. The effort was both aggressive and effective in identifying the '
,
deficiency and making the necessary corrective action Even so, a more purposeful IST effort on this valve may have identified the degradation earlier. The Unit 2 transient, caused by the skeleton shrimp in the circulat-ing water system traveling screens, was handled quite well by the operator ,
The time frame, though, between the first pump trip and the recognition that '
the screens were clogged appears excessive in light of Unit 2 being at 40*; l power and Unit 1, steaming on the condenser, with a degraded shutdown cooling i capability.
l
_ _ .
.
Also, the HPCI DC Motor operated valve issue, which precipitated the shutdown appears to be a problem that should have been identified and-corrected by the licencee sooner. These equipment failures reflect a continuing trend that not only places a heavy burden on the maintenance staff but also appears to send a message from Plant and Operations management to the operators that they are expected to safely operate the plant with less than fully reliable equipmen V EXIT A formal exit for the AIT has been schedule for August 18, 1988, at the CP&L Company office in Raleigh, North Carolina. Results of the formal exit will be dccutented in the Meeting Summary, issued subsequent to the meeting. Two debriefs on the AIT issues were conducted with licensee personnel at the Brunswick site on July 21, and 2 Althouah the AIT departed the site on July 22, several members continued to review licensee documents in-offic This effort continued through July 2 Based.on discussion with the licensee on July 18, 1988, and an initial review-from the AIT, the NRC issued a Confirmation of NRC Concurrence to Restart
- Brunswick Unit 1 letter on July 25, 1988. This letter expressed some reserva-tions over the extent of management oversight at the site but did concur with the restart of Unit 1 with continued NRC 24 hour coverage. On July 29, CP&L sent a letter to the NRC providing a stav.is of actions to date, short-term actions in process and plans for long-term action l l
l l
L _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ _ _ _ . _ . _ _ _ _ _ . _ _ _ _ _ _ _ _ _ __ _ _ . _ __ _ _ _ _ _ _ _ . . _ _ _ . _ _ _
-- -_
.
'
l
'
.,
.
ATTACHMENT 1 CONDENSER CIRCULATING WATER S'fSTEM Page 1 of 3
.
,
I t
!
o OISCMaAQE *uMNEL l 4 a l
t 4
4 i
a a 4 i
g . .
- vie
'..../ d
@,
" =l C[o.UT.', <W
=l W2 @ =l 'niD = @
$,, h i
, ,
,
.........
,
-. .... .... a _. .... v,.. .........
IIT Tl l FCr Pf
. l @
,....
W d
..... j
__
, i.... ...._
l. L
_
CONDENSER 28 LL CCNCENGER 2 A TT T "'
l ..,. g'
l
"
'g en g 'g Lc.; " :; ) { .
.
e
. ..L Ld,, ..
-
? " v. 2.o ) ..L
@ e o..... .*.,,C..... ....v n ,.e,_@ + e C. ......>v
.
,
. .
, _ s... L .. ,
T r ,.. # v
' _
f
....viC..... i A-9
I
N. . , --
-
i N... A : a =
__ .
N...
a a
_ _- e ..u .. .. . .
_
s
-
P t nacawaa# zoms 1
. ..
t eacawsen zone l
., .,
i
. .1 .
., 1, ee.e see ve vP ve
[ osvamfisse zome o a -}
,,
,, a a ses sit
,,
,,
41 see
"w'! ND n,-.... o vna was
" yet vit
"
.. ..
" *
- .:1 ..
vs . vs vi b /
/\ INTAKE STRUCTURE [. A f W
LNTAK E C Ama * .e
_
-P- . _ . . , . - - - _ _ . . - _ _ - - , _ ~ - . . - - - - - .
. !
,
.
ATTACHMENT 1 TYPICAL TRAVELING SCREEN Page 2 of 3
.
DRIVE ls0 TOR
> d%
SCREEN 1 ASH
$ PRAY PIPE
>
d DalyEN $PROCKET I't>
DI**
- >
'
[
>
'
I SCREEN PANELS %
>
-.
.
.
,
SCREEN CHAIN
-
, )
-
e l
<
l f'
x
CulCE $PROCKET
,
f
\ l lI BOOT
[ SECTION l
.
-
. . $
.'
, ,
.
ATTAC1DIENT 1 SKETCH OF SKELETON SHRIMP- Page 3 of 3
.( <
.
' a
/ el .6
'
gU iv ( A =
%? ,".
N e
/ $ / T c 0 )
w E
F 0 O I J
'
K t
b M N O
P Fig. 21.38. Crustacea, Arr phipoda, Capeellidea. A. A<giatIls sp., head region and detail of mandible and palp; helle op'.', whole animal; C, Argiartle spaese, abdomen; D, Aeginine leagicwah, abdomen; E, FAtisite marias, leg from segment Ill; F, luenecie incerte, segment V; G, Peterepelle unan, segment V; H, if<mie<gine minice, thoracic aegment la dorut view; I, Capelle paeatis, esphalon; j, same, part of leg.from segment V; K, CedcII, eatreet, part ofleg from segnxnt V; L, CapcIIe esics (left) and Cepelle eguih6re (right), part ofleg from segment V oc VI; Af, CepcIIe afvihere, insertion of gnathopod !!;
N, Cepelle srpratrienehs, detail of gnathopod II; O, same, gnathopod II: P. CaptI!.
lia<eris, gnathopod !!. Abbeeviations are as follow: a, antennae; b, basis; e, cephalon; gl, gnathopod I; gII, gnathopod ll; d, gill; e, abdornen; f, manilliped; leg bearing seg-ments (perconites) numbered 1-Vi I
l l
. 1
.
I b % . . . : ;, x n. .., .
. * . . . . . . ..
, ,~2e .
. I I . ._ _ j
-
_,
.
r Page 1 of .3 ATTAcerENT 2 RHR SYSTEM (SHUTDOWN COOLING MODE)
. ..
--< -
mi . s.-- .:
'T faw.........
n; lt..s. g .e.4 -
-
f . ..
.
. _
. . . __ .............
-
a_.- :J-...
. ..
.m
.
,, . g _ ._
.-
..y >u ' . .::
.. s
. -
. .,7 ,
n
[ ****
.
'
'j~-
-
w
. =, _ _ _ _ - -
,
-
..
"
8E h . . . .
J1 -
-
~7: -
- ~7 ,
~
-
.
.}
. mt 2h * E --
's . -------------$--@
-
a rr
'
lM * .f.~ F017B l
' = ~
- =- w - - - ,.__ *
l -., :
"(===' -~
-
-
i e r
l ..
A - i - ~
l
'
g-
.
_
(& .mx
-
li *. --
i __
i
In
4 i
. . .
M ~
lk :
L~ ' : ::
m
-
,r --
I .
l
,
,
ji
= l-' il U fi t J..._i
. _ _ . _
.. ,.. "
M t
' -x ifif$ifil f7 5 fi h .
y* . g. = . . .
-
, , -
,
,
...
.
_)
t .
i
___._. _ _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ ~ _ _ __ _ _ __ _ - . _ _ _ . - - _ _ , _ _ _ _ _ _ _ _ . _ _ _ _ _ _ _ __ _____._m
. , s
.
ATTACHMENT 2 F017B Valve Actuator Page 2 of 3
'N% y te 1 f Y t,,
'
~~
'O a !! ,
4 .
(k
"
d
{b?
f, h '. .
)
'. $ ,<
L *
..
1. Flexible Jaw Clutch Housing
- I', 4,,7* g, , ,
, , 43, 2. Sliding Gear Clutch
..
,
.
3. Splined Insert l
- '
.. 4. Clutch Compression
' . , , J, ~ .. , ~ .;;_ .,,
.
,
'
.
-
4 j ,' a Spring
, .s ,
, , . *[ ' .' '
- '
j I
'
) ;GdN$
, Damaged Mating Lug y,
, -
- ,
_y--,m
.. . _ _ _ . . _ _ . _ _ _ _ _ _ . - _ _
A-
'
.
,
ATTAC1DIENT 2 F017B Valve ~ Actuator Page 3 of 3
!
~-
kn i
'
~ ~ ~
~-
.r,. >3
.w - .
. .-
- /. . . -
.
.
- \\\\\\\\\\ A g \\\ \\\\
IS gg h i( h
'
,
. Danaged Gear ,ceth
'
._ .. t . . ~.
f i
.
- l
- . . - .. . - _