ML20062M540
| ML20062M540 | |
| Person / Time | |
|---|---|
| Site: | Diablo Canyon  |
| Issue date: | 08/18/1993 |
| From: | AFFILIATION NOT ASSIGNED |
| To: | |
| References | |
| OLA-2-I-MFP-056, OLA-2-I-MFP-56, NUDOCS 9401070273 | |
| Download: ML20062M540 (14) | |
Text
I
& 6e-2 nrj'm oc A MlFPf!% %f S6f
& N P P- % _ _ ,
jf*~ Ml
[ /[N '
NC" NCR DC1-91-EM-N046 June 10, 1991 93 Or,~ 28 0 / :3 l
MANAGEMENT
SUMMARY
l l
On April 24, 1991, plant operators initiated a manual l reactor trip to terminate an increase in reactor power from j 2.0% to 2.5%. The immediate cause of the power increase was l an urgent failure of the rod control system which rendered manual control rod movement inoperable. -The immediate cause of this event was the failure of a fuse in the bus duct disconnect to the rod. control power supply cabinet. Upon l investigation twelve of fifteen fuses in similar locations
! were found to be the wrong type. -
History: NCR DC1-87-TI-N109 identified.the failure of l ceramic type 30 amp fuses as a generic problem t
with the rod control system. _All' rod control power fuses were to be replaced with new fiberglass style fuses (same part #, similar appearance).
Root Cause: The fuse that failed in this' event was the
'old' style which was to have been replaced as a corrective action to prevent' recurrence of DC1-87-TI-N109. The root cause of the wrong fuses is unknown, however, the mostLprobable root cause is personnel error, in that a contract electrician replaced the wrong fuses (in the rod control cabinet rather than-the requesteo (and documented) bus duct) , -leaving-the low reliability 'old' style fuse in place.
Corrective Action:
- 1. The bus duct disconnect panels will be uniquely identified.
- 2. The Electrical Maintenance Manager will provide a tailboard briefing to maintenance personnel' regarding this event emphasizing how informal communication contributed to this event.
A review of p:. ant procedures determined that' adequate procedural cutrols are in place to prevent this type of problem frce recurring.
91NCRWP\91emn046.ddm Page 1 of 13 9401070273 930818 PDR ADOCK05000g5 ,
O
2 l l '*. i i
NCR DC1-91-EM-N046 June 10, 1991 NCR DC1-91-EM-N046 Reactor Trip Due to Blown Fuses I. Plant Conditions Unit 1 was in Mode 2 (Startup) at 2.5 percent power.
II Description of Event !
A. Event:
On April 24, 1991, at 1819 PDT, Unit 1 was tiden critical after an automatic trip earlier that day. At 1819 PDT, power was stabilized at 10 E-08 amps to record critical data. Data was obtained, and the l control operator then began te withdraw the control rods to establish a positive start-up rate to take power to the point of adding heat (ref. 1).
At 1822 PDT, Rod Control Urgent Failure PK03-17 l alarmed. This alarm locked up the rod control drive '
j mechanism and prevented control rods from being moved in or out to terminate the power ramp (ref. 1).
Personnel were sent to the rod control cabinets to
' determine the failure if possible. I&C assistance was requested. The senior control operator reputted that power cabinet 1AC had the urgent failure, but he did not open the cabinet to determine the type of failure.
By this time, reactor power had passed the point of adding increase.heat and reactor power level was beginning to It did not appear that the rate of power increase was being damped by a negative moderator coefficient. It was agreed to attempt to insert Control Bank D in individual bank select. This was unsuccessful. The shift foreman and shift supervisor concluded that a manual reactor trip _was warranted, and the shift foreman ordered the control operator to manually trip the reactor. The control operator tripped the reactor at 1827 PDT at a peak indicated reactor power of approximately 2.5 percent (ref. 1).
On April 24, 1991, at 2054 PDT, a four-hour non-emergency notification was made in accordance with 10 CFR 50. 72 (b) (2) (ii) (ref. 4).
B.
Inoperable Structures, Components, or Systems that 91NCRWP\91emn046.ddm Page 2 of 13 i
i
1 e
i i
NUCLEAR REGULATORY C0YM13SION Meket No.bo&YS'C b A_ oma Espa?l? h $h
\~ n ,r.: cs hW' 'C &&S . M Etae7b e Oc.
w:. w:n v
.r.. .'., /-
s ,
__ ,, _ is . W d
{p e,
- 9Y_ _
n ;. %-+. .
' av ,,.- - . ,
l ,e aw _ .....
Re;. cit:t C 4i 2
/
P e
O y4,
+M g l
l l
t t
i
e' NCR DC1-91-EM-N046 June 10, 1991 Contributed to the Event:
None.
C. Dates and Approximate Times for Major Occurrences.
- 1. April 24, 1991, at 1822 PDT: A rod control urgent failure alarm was received in,the control room.
- 2. April 24, 1991, at 1827 PDT: Unit 1 was manually tripped after unsuccessful attempts to regain positive rod control.
- 3. April 24, 1991, at 2054 PDT: The four-hour non-emergency report was made in accordance with 10 CFR 50.72 (b) (2) (ii) .
D. Other Systems or Secondary Functions Affected:
None.
E. Method of Discovery:
The event was immediately known to plant operators due to alarms received in the control room.
F. Operators Actions:
Plant operators initiated a manual reactor trip to terminate the event as directed by the shift foreman.
G. Safety System Responses:
l The reactor trip breakers opened and rods were inserted into the core terminating the event.
III. Cause of the Event A. Immediate Cause:
91NCRWP\91emn046.ddm Page 3 of 13 l
,- i NCR DC1-91-EM-N046~
~ June 10, 1991
~
- ]
One of the three phase fuses . (AA)~(FU) inside one of _ . l the three1 bus duct fused. disconnect panels for theirod i control power supply cabinet- (PN1AC)' failed. ')
)
B. Root Cause:
- i Upon inspection of the-fuses.it.was found that.three-of.the nine fuses were of an 'old' styleTfuse.that were known.to plant operators:and technicians.to have i been.a-problem identified in 1987 (LER:1-87-016). .
-Resolution of this problem required asca corrective action that'all 'old' styles 30 amp fuses'in'the rod:
control system.be replaced with 'new' style fuses. '
The corrective ~ action work orders that replaced the- i
'old' style fuses were reviewed to-determine how this j required action did not replace;the recently failed bus. duct fuses. -
l A review of the corrective: action ~ work order-C0038363' ,
l for DC1-87-TI-N109 found that the bus duct fuses were-documented as properly replaced on October: 19,.1989. ;
The root cause of the' failure to replaceithe._ bus' duct: I fuses is unknown. However,-three possible-causes were- 3 identified:
- 1. The wrong part may have been withdrawn:from. >
warehouse stock. l
- 2. The wrong part may'have been obtained'and l installed from other stock.
- 3. The wrong fuses.(the rod. control powericabinet .!
l Instead of the bus. duct' disconnect) were4 replaced l l
t leaving the old style fuses'in place.
Each of the possible causes, except' scenario no. 3 was eliminated as follows: ,
i i
1.a. Corrective action taken for the initial problem identified in 1987.(DC1-87-TI-N109)-
included the removal of.all 'oldf style fuses from warehouse stock and-receipt. inspection of' i l
'new' style replacement fuses. .Also, an~ I examination ~of the presentiwarehouse stock.
found no further evidence of 'old' style fuses.
91NCRWP\91emn046.ddm Page J4 of- 13
{
L r 1 i l l
L _ _ . _ . _ -. l
l
.- I NCR DCl-91-EM-N046 l June 10, 1991 l 2.a. The replacement work order specified the correct part for the failed location and this was confirmed by an attached warehouse withdrawal slip from a receipt inspected 'new' style fuse purchase order.
3.a. A review of the replacement WO (C0038363) identified misleading information provided to the contract electricicn.
3.a.1. The scope of the WO described the replacement of the bus duct disconnect fuses 'above the rod control cabinet PN1AC'; however, the work plan activity line only stated ' replace fuses in fused disconnects.'
3.a.2. The job only required replacement of three fuses; however, nine fuses were provided with the work order (ref. 6).
Coincidentally, nine fuses identical to those provided with the WO (C0038363) are located within panel PN1AC.
Based upon the above considerations PG&E believes the most probable root cause is personnel error, in that the contract electrician performing the work on WO C0038363 replaced nine fuses i" PN1AC ratner than the bus duct fused disconnect.
C. Contributory Cause:
e e a Corrective actions to prevent recurrence for nonconformance DC1-87-TI-109 required the :
instrumentation and controls (I&C) department to l locate and replace any additional 'old' style fuses. ]'
The bus duct fused disconnect location was identified as containing 'old' style fuses which required replacement. A verbal request to replace the 'old' l style fuses between I&C and the' electrical maintenance l work planning department personnel resulted in the issuance of work orders for bus duct fuse replacement during the Unit 1 third refueling outage in 1989.
PG&E believes that a miscommunication lead the work planning department to stage nine 'new' style fuses rather than the required three fuses.
An additional contributory cause was identified in 91NCRWP\91emnO46.ddm Page 5 of 13 i
e l
i l
NCR DC1-91-EM-N046 June 10, 1991 that the bus duct fused disconnect panels were not uniquely identified. This.may have led the contract electrician to believe that he had replaced the fuses at the correct location.
IV. Analysis of the Event A. Safety Analysis:
The rod control system is a design class II system for positioning of the_ reactor control rods for reactor power modulation by manual or automatic control of control rod banks in a preselected sequence, and for manual operation of individual, banks. The urgent failure alarm is actuated when any one of the preset parameters is actuated. This alarm, when received due to a power supply failure Inhibits further rod motion until the alarm condition is resolved. This control feature is provided to stop further rod action that may result in an unintended rod position. The rod control system is provided electrical power from the non-safety related motor generator set through-the safety-related reactor trip breakers to the rod l control system and the control rod grippers. The l failure described in this LER had no effect on any safety-related portion of the reactor trip system.
l Therefore, the manual reactor trip initiated during-this event represents a conservative course of action.
In the event of an increase in neutron flux at low power, the nuclear instrumentation system is designed
_to provide three automatic reactor trip, signals.- Two of these signals were available to terminate the unintended power increase if in the unlikely event that manual operator action were not taken. The l intermediate range neutron flux reactor trip is designed to trip the reactor when one of two l intermediate range channels measures a power level greater than 25 percent RTP as stated in FSAR Update Section 15.2.1.1.2. The power range'high neutron flux reactor trip (low setting) is designed to trip the l
reactor when two of four power range channels indicate a power level greater than 25 percent RTP as stated in FSAR Update Section 15.2.1.1.3. Both of these-reactor trips require manual operator action to block the trips in order to intentionally increase RTP above 25 percent. Therefore, in the event that plant operators had not tripped the reactor manually, the reactor 91NCRWP\91emn046.ddm Page 6 of 13
l NCR DC1-91-EM-N046 June 10, 1991 I
would have automatically tripped when the 25 percent RTP threshold was exceeded.
Therefore, based upon the above analysis, the health and safety of the public were not adversely effected, and there were no adverse consequences or safety implications resulting from this event. ;
l B. Reportability:
i
- 1. This event is nonconforming in accordance with QAP 15.B Section 2.1.2 in that a manual reactor trip was initiated to terminate the event. Therefore, this event requires reporting to the NRC.
- 2. This event was reviewed in accordance with 10 CFR !
50.72 and 10 CFR 50.73, and guidance provided in NUREG 1022, and determined to be reportable in accordance with the requirements of 10 CFR ,
l 50.72 (b) (2) (ii) and 10 CFR 50.73 (a) (2) (iv) in that a reactor protection system function was manually l l
initiated.
- 3. The fuses that caused the problem are being investigated to determine if they were purchased under the applicability of.10 CFR 21.
- 4. This event was determined not to be reportable in accordance with 10 CFR 50.9 because the event is being reported under 10 CFR 50.73.
- 5. This event does not require a network entry.
V. Corrective Actions A. Immediate Corrective Actions:
All the buss duct fuses for the unit i rod drive control cabinets were replaced.
B. Investigative Actions:
- 1. Document the implementation of the corrective actions for NCR DC1-87-TI-N109. The corrective actions for this NCR required the change out of all ceramic fuses found in the control rod drive cabinets with newer fiberglass fuses.
91NCRWP\91emn046.ddm Page 7 of 13
t i NCR DC1-91-EM-N046 June 10, 1991 RESPONSIBILITY: H. Phillips' ECD: COMPLETE.
Tracking AR: A0229469, AE Ol#-.
' DEPARTMENT: Electrical Maintenance Outage Related? NO JCO Related? NO i NRC Commitment? NO CMD Commitment? NO
- 2. Document the as-found condition of'the fuses and
.the control-rod cabinets, and the. actions taken by Electrical Maintenance personnel in-troubleshooting the problem on. April 24, 1991.
RESPONSIBILITY: H..Phillips ECD: COMPLETE .
Tracking AR: A0229469, AE 02#
DEPARTMENT: Electrical Maintenance Outage Related? NO ,
JCO Related? NO NRC Commitment? NO CMD Commitment? NO
- 3. Document the inspection of the warehouse'to assure that all old style ceramic fuses like'those found' in the control rod cabinet were removed.from the warehouse stock.
RESPONSIBILITY: H. Phillips ECD: . COMPLETE Tracking AR: A0229469, AE 03#
DEPARTMENT: Electrical Maintenance' Outage Related? NO JCO Related? NO t NRC Commitment? NO CMD Commitment? NO i
! 4. I&C will determine the' appropriateness of the response of the control rod drive system to the l buss duct fuse failure.
l RESPONSIBILITY: D. Weatherby . ECD: COMPLETE Tracking AR: A0229469, AE 04#
i DEPARTMENT: I&C l Outage Related? NO JCO Related? NO !
NRC Commitment? NO CMD Commitment? NO
- 5. Determine the. applicability of 10 CFR Part 21 to the ceramic buss duct fuses that failed in power 91NCRWP\91emnO46.ddm Page 8 of 13 1
1
NCR.DC1-91-EM-N046 June 10, 1991 cabinet 1AC.
RESPONSIBILITY: G. Cecchi/PSG ECD: COMPLETE Tracking AR: A0229469, AE 05#
DEPARTMENT: EM/PSG Outage Related? NO JCO Related? NO NRC Commitment? NO CMD Commitment? NO l
- 6. Investigate panel ID. Determine the panel code or identifier for fused disconnects located above PN1AC.
RESPONSIBILITY: H. Phillips ECD: COMPLETE Tracking AR: A0229469, AE 07#
Di cARTMENT: Electrical Maintenance l
Outage Related? NO l JCO Related? NO NRC Commitment? NO CMD Commitment? NO
- 7. Personnel interview - contact work planner G.
Morrison and B. C. Williams regarding their work performed as documented in work order C0038363.
RESPONSIBILITY: H. Phillips ECD: COMPLETE Tracking AR: A0229469, AE 08#
DEPARTMENT: Electrical Maintenance Outage Related? NO JCO Related? NO
! _ NRC Commitment 2.NO , ,
l CMD Commitment? NO l
- 8. Request warehouse stock code 73-2238 be deleted (this is a duplication of 93-7457).
RESPONSIBILITY: H. Phillips ECD: COMPLETE Tracking AR: A0229469, AE 09#
l DEPARTMENT: Electrical Maintenance l Outage Related? NO JCO Related? NO NRC Commitment? NO l CMD Commitment? NO l C. Corrective Actions to Prevent Recurrence:
91NCRWP\91emn046.ddm Page 9 of 13 l
l l
i 9
I NCR DCl-91-EM-N046 June 10, 1991 l
l I i
- 1. The fuses in the Unit 2 control rod drive cabinets will be inspected to assure that they are not the old style fuse specified for the application.
RESPONSIBILITY: H. Phillips ECD: 11/15/91 Tracking AR: A0229469, AE 06#
DEPARTMENT: Electrical Maintenance l Outage Related? 2R4/U2FO l JCO Related? NO NRC Commitment? YES CMD Commitment? NO
- 2. Provide unique panel ID's. Provide input to NECS and obtain unique panel identification for each j panel location in the rod control system.
RESPONSIBILITY: H. Phillips ECD: 11/15/91 Tracking AR: A0229469, AE 10#
DEPARTMENT: Electrical Maintenance Outage Related? No JCO Related? No NRC Commitment? YES CMD Commitment? No
- 3. Conduct work order expectations tailboard. Review the specific circumstances surrounding this event and communicate management expectations regarding the need to seek clarification when necessary.
RESPONSIBILITY: H. Phillips ECD: 08/09/91 Tracking AR: A0229469,'AE 11#
_, -. DEPARTMENT: Electrical Maintenance Outage Related? No JCO Related? No NRC Commitment? YES CMD Commitment? No +
VI. Additional Information A. Failed Components:
Fuse (Shawmut, Amp Trap, Type 1, Form 101 P, Gould Inc., Newbury Port, Ma.)
B. Previous Similar Events:
NCR DC1-87-TI-N109 Control Rod Power Fuses 91NCRWP\91emn046.ddm Page 10 of 13
I h
L,' ,
NCR DC1-91-EM-N046 June 10, 1991 This NCR addressed the failure of power. fuses used in-control rod drive cabinet 2AC. This caused control l bank A to lock up. The cause of the failure of the fuse was poor connection of the end cap'with the' fusible link.
The corrective actions for the problem included ,
replacing the fuses with a newer designed' fuse.
C. Operating Experience Review: l l
- 1. NPRDS:
No specific failures identified.
l
- 2. NRC Information Notices, Bulletins, Generic Letters
- IEIN-8762, " Mechanical Failure of Indicating-Type Fuses," identified a similar cold solder joint j problem with similar fuses. This information was- i l known at the time the TRG for DC1-87-TI-N109 l required replacement of the 'old' fuses with the newer higher reliability fuses.
None identified. 'j i D. Trend Code Responsible department is electrical maintenance, and-l root cause personnel error, lack of mental attention.
(cause code EM-0A3).
E. Corrective Action Tracking:
! The tracking action request is A0229469.
! F. Footnotes and Special Comments:
None.
G.
References:
l
- 1. E-mail from Jim Dye to Steve Fridley dated 4/24/91 i
- 2. Initiating Action Request'A0229094.
- 3. Licensee Event Peport 1-91-008-00.
91NCRWP\91emnO46.ddm Page 11 of 13 i l l
NCR DC1-91-EM-N046 June 10, 1991
- 4. Control room logs dated 4/24/91. '
- 5. Event Response Plan 91-05.
- 6. Corrective maintenance work order C0038363.
- 7. Initial Non-Conformance Report DC1-87-TI-N109.
H. TRG Meeting Minutes:
l On May 1, 1991, the TRG convened and considered the following:
- 1. The TRG discussed a postulated root cause of the event being incomplete replacement of old style ceramic power fuses with new fiberglass fuses in 1989 as a result of NCR DC1-87-TI-N109. EM will document the replacement of the fuses.
l
- 2. The TRG assigned several investigative actions to determine the root cause.
- 3. It was determined that the event was reportable due to the manual reactor trip.
- 4. The TRG agreed that the fuses on-Unit 2 should not be inspected until the unit is shutdown or trips off line. The fuses will be expected at the first available opportunity.
- 5. The TRG will reconvene on 5/8/91 to dirruss the results of the investigative actions.
The TRG reconvened on 05/08/91 and reviewed the evidence collected documenting the replacement of the fuses in 1989 by a contract electrician. It was concluded that the most probable cause was personnel error in that the electrician replaced the fuses in PN1AC rather than the bus duct disconnect. The implications from the initiating documents of NCR DCl-87-TI-N109 will be reviewed further.
The TRG reconvened on 05/10/91 and considered the additional facts that none of the intended (15) 'old' style fuses had been replaced in 1989. Fifteen' fuses were replaced by I&C on April 24, 1991, twelve.of which were of the 'old' style. An additional work order (C0035512) was discovered which accounts'for the replacement of the 'old' style fuses in the bus duct disconnect for rod control cabinet 2BD in July 1988.
With this new information the TRG concluded that 91NCRWP\91emn046.ddm Page 12 of 13
I I
e*
NCR DCl-91-EM-N046 June 10, 1991 multiple personnel errors were.made by three contract electricians who performed the work.in October 1989.
Also, work planning was present and reviewed the.
discrepancies regarding'the work' order.i
~
The TRG reconvened on 6/12/91 to sign off the NCR.
All in attendance agreed that the NCR is ready for-PSRC concurrence.
I. Remarks:
None i
J h
l
,,,. a,. , .e.
91NCRWP\91emn046.ddm Page 13' of. 13
.. . . _ _