ML20133D220
| ML20133D220 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 12/06/1996 |
| From: | Bersak C, Rajkowski L FARWELL & HENDRICKS, INC. |
| To: | |
| Shared Package | |
| ML18102A725 | List: |
| References | |
| NUDOCS 9701090072 | |
| Download: ML20133D220 (33) | |
Text
- .. _- . .. . _ - - - . . - . _ . _ . . _ . . . - . - .-
RECElVED DEC 3 01996 Root Cause Analysis of Struthers Dunn Operate-Reset Relay
- Latch Failures l for Salem Generating Station l
l Condition Reports CR 960906195 CR 961101269 CR 961105229 December 6,1996 l
Principal Investiaator i
Craig Bersak l
l l
Prepared by: Md Craig Bers ngineering Assurance Approved by: // .
L. Rajkowskparfagifrf - Bailey Replacement Project l
l l
l l
9701090072 970102 PDR ADOCK 05000272 ,
S PDR l
l
_ _ _ . _ _ _ ~ _ _ _ _ _ _ _ _. . _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ ___ _.. . _ __m 1
l "Roet Cruss Antlysis cf Struthsrs-Dunn Op: rats.Rasst Rslay Letch Fellurss CR# 960906195 fcr Srl m G:n2rsting Stati:n -
TABLE E CONTENTS M EX EC U TIVE S U M M ARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 M I N TR O D U CTIO N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 B a c k g ro u n d . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Identification of Occurrence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Conditions Prior to Occurrence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Root Cause Analysis Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . 5 M DESCRIPTION E OCCURRENCE /SCAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Chronology of Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Troubleshooting, Remedial, and Compensatory Actions Taken . . 7 l .
Notification of Authoritieslindustry . . . . . . . . . . . . . . . . . . . . . . . . . . 8 l Related Failure Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
[
4.0 AtlALYSIS & CONCLUSIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Issue #1: l Seismic adequacy of B255 style relays for Salem Station. . 10 '
Description of the issue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 issue #2: Failure of relay to latch into OPERATE upon a valid demand.
................................................. 10 Description of the issue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Causal Factor Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 issue #3: Spurious uniatching of the relay without a valid demand. . 14 Description of the issue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Causal Factor Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 issue #4: Unsultability of new relays. . . . . . . . . . . . . . . . . . . . . . . . . . 15 Description of the issue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 l .
Causal Factor Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Prior Similar Events and Operating Experience . . . . . . . . . . . . . . . . . . . . 16 Generic implications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 l Safety Significance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 5.a CO R RECTIVE ACTIO N S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Corrective Actions and Accountabilities . . . . . . . . . . . . . . . . . . . . 19 Measures to Evaluate CA Effectiveness and Accountabilities . . . 20 6J OTH ER ISSUES AND IMPACTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Issues Beyond the Scope of Investigation . . . . . . . . . . . . . . . . . . . 21 Common Mode Failures, as Appropriate . . . . . . . . . . . . . . . . . . . . 21
) .
Repo rta bility . . . . . . . . . . . . . . . . . . . . . . . . . . , . . . . . . . . . . . . . . . . 21
) . Industry Information Dissemination . . . . . . . . . . . . . . . . . . . . . . . 21 7.0 SOU RCES E IN FORMATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 i
a ,
- Rost Causo Analysis cf Struth:rs Dunn Opsrats.R2sst Rstay LItch Failuras CR3 960906195 f ar Sslem G:nnr: ting Station ,
C o n t a c ts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Ref e re n c e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ............ 22 4
8.0 INVESTIGATOR COM M ENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 9& APPEN DIC ES &_. ATTACH M ENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 1
i 1 I i
i l
l l
l e
l I
i I
l 11
" Rect Causs Anslysis of struthsrs Dunn Opsrats Rasst Ralry Latch Feitures CR# 960906195 for Sal:m Ganarcting staticn j 1A EXECUTIVE
SUMMARY
l Description of Event l
This root cause evaluation addresses failures of Struthers-Dunn B255 style latching relays to latch on demand and to reset without a valid demand signal. Subsequent to I the identification of failures at Salem Station, PSE&G was notified by its vendor, Farwell & Hendricks, Inc., of its submission of a 10CFR21 notification of a potential defect and of the premature failure of a relay during seismic evaluation. Condition Reports 961101269 and 961105229 were written to document these concerns.
.Q_auses l
The root cause of the relay latching problems was a reduction in the elongation of the l coil spring for the latching mechanism. This reduction occurred during revisions to the design of the relay frame and reset coil bracket. As a result of the reduced spring load the impact of the operate armature upon the latch could result in its movement away from the latched position, allowing the relay to return to the reset configuration. I Latched relays would require the addition of energy from some outside source, i.e., I vibration or jarring, to become unlatched. !
Seismic concerns developed when PSE&G was notified that a verification relay of l similar manufacture dates had become unlatched during seismic testing above SG l broadband acceleration when mounted in the floor mount configuration. The Salem applications of the relays were evaluated and found to have limiting seismic ;
requirements below this level. As a result their seismic qualification was and is suitable i
for Salem Station applications. l l
Corrective Actions Coordination with the relay manufacturer and vendor, Magnecraft/Struthers-Dunn l (MSD) and Farwell & Hendricks, Inc.(F&H), has identified a replacement spring to l restor the latching force to levels consistent with the 1970 vintage relays. Relays i purchased from F&H are being returned to F&H to have the replacement spring i installed and become re-dedicated for PSE&G's use. MSD has incorporated the l 1 replacement spring into the manufacture of this series relay. New folios have been I established to identify refurbished and newly manufactured relays from the suspect -
population. l l
A formal relay bench testing protocol has been established and proceduralized to l identify potential relay failures prior to their installation into the plant. This testing j procedure has been aligned with the F&H testing procedure to assure consistency. l L .
l l A Bailey Replacement Project has been initiated to identify and replace the suspect relays prior to entering a mode where the component impacts plant safety. Safety l
1 1
l l
' Rect Causs Anslysis of struthsrz-Dunn Op3 ret:-Rasst Relay Lttch Friluras CRC 960906195 fer Sil:m G:n:rtting stition l Evaluations in accordance with 10CFR50.59 are being performed to assure that the unchanged relays do not impact safety requirements for the mode to be entered.
l Safety Sianificance The relays involved with this evaluation are utilized in the control systems for both safety-related and non-safety related components throughout both Salem units.
I All of the failures identified to date have involved DC powered relays, predominantly of the 24-28 VDC ratings. In addition to the 24-28 VDC relays, Salem utilizes 125 VDC relays (two failures) and 110 VAC relays (0 failures) of the B255 style. All of the recent failures have been with relays staged for or installed in the Bailey Control Cabinets.
The 24-28 VDC relays typically are used in the interfacing circuitry between the operator's control station pushbuttons and the subsequent actuation of the actual device. Within these applications the relays typically serve to establish and maintain the last selected AUTO / MANUAL mode of operation (83 device)
Breaker demand position (checking relay (3 device))
Alarm acknowledge (74 device) 4 The 125 VDC relays typically provide breaker controls for the actuated device.
Per the UFSAR SER Section 7.3.4, "the automatic operation of the ESF equipment is independent of any action in the 28V circuitry."
Nonetheless, improper operation of the relays imposes additional demands and l
d:stractions on the operators. Additionally, failures, such as the RH29s spurious shift '
to MANUAL, can adversely impact the operation of the controlled equipment. In the case of the RH29s, a spurious shift to MANUAL can result in operation of the associated RHR pump with less than minimum pump flow. A complete search of all affected control circuits was not conducted to determine the extent r* this problem.
i Plant mode specific safety evaluations in accordance with 10CFR50.59 are being performed prior to the entry into any defined mode to ascertain the impact of any l component which may have a failure prone relay within its control circuitry. These safety evaluations will be attached to this report as they become available.
Generic Issues l
None I
2 l - -.
_ , __ . . _ _ _. _ _ _ . _ _ _ ._ . . __ _ . _ . ~ . _ _ . _ _ _ _ _ ..__._ ____ . . _ . _ .._. . . _ . .
=
Roet Cause Antlysis cf Struthars Dunn Opsrato-Raast Rafay Latch Failurss CR# 960906195 f ar S:lem G3n3 rating St ti:n l
Reoortability These occurrences have been reported to the NRC by means of a four hour report on November 5,1996 and LER 272/96-031-00, i
Part 21 notification was initiated on November 1,1996 by F&H.
i i
1 l
l l
1 i
i l
l 1
i t
f f
3
! ' Rect Causo Antlysis of Struthsrs Dunn Opsrats-Rasst Rsisy Litch Frilurcs CR# 960906195
) for Sahm Gan:rcting Stdian l 2e INTRODUCTION
. Background This root cause evaluation addresses failures of Struthers-Dunn B255 style latching relays to latch on demand and to reset without a valid demand signal. Subsequent to the identification of failures at Salem Station, PSE&G was notified by its vendor, Farwell & Hendricks, Inc., of its submission of a 10CFR21 notification of a potential defect and of the premature failure of a relay during seismic evaluation. Condition Reports 961101269 and 961105229 were written to document these concerns.
Identification of Occurrence During post-maintenance testing of 22 Residual Heat Removal (RHR) pump, the 22 loop Heat Exchanger Bypass valve,22RH29, was observed to shift from Automatic to Manual mode without a demand from the operator. During the PMT activity, RHR pump flow was being reduced to open 22RH29. When the low-flow setpoint,. 500 gpm, was reached the valve spuriously swapped to MANUAL, resulting in a failure of the valve to open as required.
Separately, the Slow Speed start for the 22 Boric Acid Transfer Pump failed to latch in when selected. When the operator released the push-button, the pump stopped without generating an alarm.
Additional component failures due to latching relay problems are listed in Attachment 3.
Conditions Prior to Occurrence i
At the time of discovery, both Salem Unit 1 and Unit 2 were defueled. An extended
{
maintenance / refueling outage was on-going for Unit 2 with outage activities suspended '
on Unit 1. l
. Root Cause Analysis Methodology Failure Modes Analysis was utilized in the root cause evaluation of this event. A Failure Modes Chart can be found in Attachment 1. Additionally, Change Analysis was performed (by FPI international) comparing the recently purchased relays to the 1970s vintage relays installed as original equipment. The problems encountered with these 1 relays was discussed and reviewed with both the relay manufacturer, Magnecraft/Struthers-Dunn (MSD), and the qualifying vendor, Farweh & Hendricks, Inc.
(F&H).
l I l 4
1
Rest Causo An: lysis cf Struthsrs-Dunn Opsrata-Rss:t Rslay Latch Feituras CRS 960906195 frr sti:rn G:n:rcting Ststion
3.0 DESCRIPTION
OF OCCURRENCE /SCAQ
. Chronology of Events Prior to the commencement of the current Salem Unit 1 ;.nd 2 cutage, the degradation of relays within tne Bailey Relay cabinets was identified. The primary aspect of the observed degradation was an increace in contact resistance due to the formation of a cadmium chloride salt within the relay case. As a result of this adverse trend, a Bailey Relay inspection / Replacement Project was initiated with the commitment to perform a -
100% inspection of each unit's Bailey cabinets prior to the unit's return to service. A work standard (BAILEY 001) was developed providing the technician with criterion for evaluating the relays via visual inspection. Relays which were judged by the technician to be degraded were replaced with recently pruchased relays. During the Post Maintenance Testing and following return of the component to the Operations Department, failures of latching style relays were observed at an unacceptable frequency. Additionally, during bench testing of the relays, prior to their installation, an 1 approximate 10% latching failure rate was observed. As s result of the field and bench testing failures, condition report 960906195 was initiated to identify the root cause of I the relay problem.
In 1996,15 instances of Struthers-Dunn Operate-Reset Relays failing to latch into the OPERATE state or spuriously resetting have been identified on Salem Unit # 2. !
Representative of these failures are the following occurrences:
l On June 6,1996, the " LOCAL MANUAL" alarm for the 28 Emargency Diesel Generator did not lock in when acknowledged by the operator.
On June 30,1996,22RH19, the 22 Residual Heat Removal Discharge l
valve, would return to the closed pos! tion after initially responding to an l OPEN demand; i.e., stroke open, reach its open limit and then promptly l
close without a close demand from the operator.
l On August 301996 Post Maintenance Testing (PMT) was performed on 22 Residual Heat Removal (RHR) pump following completion of corrective maintenance work order 960828275. This work order was initiated due to excessive oil leakage. During the PMT, 22 RHR loop flow was being reduced to cause its heat Exchanger Bypass valve,22RH29, to automatically open on the resulting low flow (<500 gpm) signal. As the low flow setpoint was approached the bypass valve spuriously swapped from AUTO to MANUAL mode of operation.
There were three separate occurrences of problems with the 22 Boric Acid Transfer (DAT) Pump. On August 19,1996, the pump stopped when its mode was shifted by the operator from AUTO to MANUAL. It subsequently failed to restart on a manual demand. On September 6, 5
l l
'Rc:t Csuso An: lysis cf Struth:r:-Dunn Op3 rata-Rssat Rslay Latch Failur:s CR# 960906195 l far S:!;m G:n: rating Strtian i
1996 during PMT of the 22 BAT Pump Control Circuit, the pump failed to remain operating in slow speed when the "22 SLOW START" pushbutton was released. Similarly, during a separate PMT, the 22 BAT pump failed to operate in " BORATE".
All of these problems were diagnosed as a failure of the associated Operate-Reset relay to latch, or remain latched, in the Operate condition.
Troubleshooting, Remedial, and Compensatory Actions Taken Troubleshooting On each occurrence troubleshooting of the affected circuitry was performed by the Salem Maintenance Department. In each case, one or more Operate-Reset Relays was determined to be operating improperly and replaced with a like relay which had been successfuily bench tested.
A relay testing protocol was established and proceduralized in SC.MD-PM.ZZ-0204(O),
Bailey Relay inspection, to establish consistency in relay testing and provide an arduous standard to eliminate potential relay failures prior to their installation in the plant. This procedure cycled each latching relay through a minimum of 10 cycles to verify its performance. This procedure was issued September 13,1996.
Communications with the relay manufacturer, MSD, were initiated to investigate and resolve the problem. A sample of the failed relays was brought to MSD's Darlington, SC facility for evaluation by MSD's Product Engineering Manager. A PSE&G Root Cause Engineer accompanied the relays during their evaluation. Only one of the relays repeated the latching failure. That relay was found to have its latching coil spring stretched at an attachment loop. When the spring was replaced the relay functioned properly.
FPI International was commissioned to perform a Change Analysis of the relays to determine the cause of failure. Their report is included as attachment 5.
F&H was contacted regarding the observed failures. Their Systems Engineering Manager and Sales Manager visited the Salem facility at which time the failures were discussed in detail. At this visit, F&H identified that they had not performed qualification testing of the relays in the " floor mounted" orientation utilized within the Bailey Relay Cabinets. (Adequacy of the relays in the floor mounted configuration at Salem had previously been established by PSE&G's evaluation of the Hope Creek l Bailey Cabinet Seismic Report.)
6
Rect Caus2 An: lysis of struthers Dunn Oparsta-Rasst Relay Latch Failures CR# 960906195 fer sal:m G:n:rcting station I
l Remedial Actions i
Following identification of the root cause of the relay's unlatching, the reduction in spring elongation, a design change to the relays wac proposed by MSD replacing the existing coil spring with a shorter spring. The relay latching mechanism with the replacement spring had been utilized in other MSD relay designs.
Compensatory Actions A Bailey Relay Project team was formed by the NBU's Senior Vice President - Nuclear Engineering to evaluate the relay problem and initiate the corrective actions. The safety significance of the failure prone relays is being evaluated by a 10CFR50.59 evaluation for the then existing condition and for each subsequent mode change until the replacement of the suspect population. A prioritized listing of components, keyed to the modes where the component was required to be operable, was established to focus the replacement of the suspect relays with upgraded relays. Additional components l
were added as necessary based on the 50.59 evaluations. Changes in plant mode t were delayed until the necessary relays could be replaced. I l
Notification of Authorities / industry l
The Nuclear Regulatory Commission was notified by F&H of the potentially degraded component on November 1,1996. Their final report, including their root cause evaluation, was due to be issued November 30,1996 and has been delayed.
PSE&G made a four-hour report upon notification of the seismic testing failure by F&H.
A Licensee Event Report,272/96-031-00, was issued discussing PSE&G's response to the seismic, Part 21, and functional failures.
Related Failure Modes Previous relay problems have been identified related to high contact resistances and the formation of " white deposits" within the relay case for other styles of Struthers-Dunn relays. As a result of this a 100% inspection of Bailey Relay Cabinets and replacement of degraded relays was initiated during the current outage. Subsequent analysis by the 1 PSE&G Maplewood Test Labs identified the deposits as "similar to a corrosion product l of cadmium containing chlorine such as found in cadmium chloride." '
An INPO Operating Experience Notice, OE 5537, identified degraded performance (premature failures) of 219 style relays with Polypropylene relay coils. During the aforementioned inspection, relays having this type coil are replaced with bench-tested relays.
A sample of 12 relays was sent to Wyle Laboratories for failure analysis of the high resistance condition. Wyle Laboratories report no. 45053R95, " Failure Analysis of 7
l
' Root Cause Analysis of Struthtrs Dunn Opsrats Ras:t Rslay Lact h Failures CR# 96090619s for Sal:m Gsn: rating St:tisn 4
Struthers-Dunn Relays From Public Service Electric and Gas Co.," concludes that "the high contact resistance appeared to be the result of the depositing of residual flux and/or its reaction products on the contacts."
Operation of the relay coils at higher than 110% of rated voltage can result in the
" premature failure of the relay coil. The 28 VDC,125 VDC, and 115 Vac systems are typically maintained withing voltage ranges that approach or slightly exceed the 110%
values. This condition can result in premature coil failure, especially among continuously energized relays.
l
' These failure mechanisms result in high contact resistances or operating coil failures.
They do not impair the relay's ability to latch if the operate coil is sufficiently energized.
1 The failures identified within this evaluation are not the result of any of these related l failure modes. -
8 l
l
l
~ Rest Cause Analysis cf Struthsrs-Dunn oparata-R:sst Rslay Latch Failur s CR# 960906195 fcr Sti:m G2n:rsting stiti:n 1
i
, 4.0 ANALYSIS & CONCLUSIONS Issue #1: Seismic adequacy of B255 style relays for Salem Station.
l
. Description of the issue On November 5,1996, F&H notified PSE&G that one of two verification relays had l failed seismic qualification by becoming unlatched at acceleration levels above SG l
broadband when tested in a floor mounted configuration. !
A walkdown by System Engineering was performed to identify the locations where the latching relays are used in the floor mounted configuration. Plant Applications Group evaluated the various locations identified and determined that none were subject to seismic levels in excess of SG. As a result, the relays were determined, as a group, to be qualified for the seismic spectra at Salem station.
Issue #2: Failure of relay to latch into OPERATE upon a valid demand.
. Description of the issue The Struthers-Dunn 255 style Operate-Reset Relay, Figure 1, consists of two coils, a spring-loaded mechanical latch and a set of contacts operated off of the armature associated with the OPERATE coil. With the coil initially in the reset condition and both coils not energized, the spring force resulting from four of the contact leaves pulls the OPERATE coil armature via the contact slider to the reset condition. When the OPERATE coil is energized its armature is drawn to the coil. The movement of the armature results in the ccatact slider changing the state of the relay contacts and allows the latch pawl to descend capturing the armature in the operate condition. With the operate armature caught by the latch, power to the OPERATE coil can be removed without changing the state of the relay contacts. The relay is reset by energizing the RESET coil. When the RESET coil energizes the latcii pswl is lifted allowing the OPERATE armature to be returned to the reset condition and changing the state of the relay contacts. Simultaneously energizing both coils will result in the relay attaining its OPERATE state without latching the OPERATE armature. If both coils are then simultaneously de-energized, the final state of the relay will depend upon which coil's magnetic field collapses first; either latching the OPERATE armature or allowing it to return to the reset state.
The majority of the failures previously identified resulted when the relay failed to maintain its latched condition when the OPERATE coil was de-energized. This allowed the relay's contacts to be returned to their RESET configuration.
I 9
'Ro:lt Causs Antlysis cf Struthsrs Dunn Oparata-Reset Rstay Latch Failuras CRS 960906195 for S lim Gsnrrcting strtion Causal Factor Analysis Three potential areas of failure were evaluated for this issue: Mechanical failure of the relay, electrical failure and human error.
1 Mechanical Failure A mechanical failure of the relay to latch can result from the following means:
Latching Pawl or Operate Armature prevented from moving freely Each of the failed relays was examined and operated to determine if there was any blockage or impairment to the motion of the moving parts. Each of the relays operated at minimum voltage with successful latching. The cases of each relay was examined with no foreign material noted that may have prevented the latch or armature from traveling freely.
Mis-formed Latch Pawl The latch pawl on the retay is a square cut piece of solid nylon. Each of the latches was examined under a 15X magnifier to determine if the trailing (latching) edge had been mis-formed or worn in a manner that would allow the OPERATE armature to return to the reset position. No deformation of any latch was identified.
Mis-formed Armature " Catch Step" The portion of the armature that is " captured" by the latch pawl has a
" catch step" pressed into it during manufacture. A relay returned by Farwell & Hendricks, the qualifying distributor from whom PSE&G purchases the relays, to MSD was noted as having a misformed " catch step." The misformed step allowed the tip of the armature to slip past the latch pawl, resetting the relay without demand.
The recent PSE&G failed relay were examined for repetition of this failure.
None of the relays was observed to have a misformed armature " catch step."
Insufficient spring loading of the latch pawl One failed relay, removed from the 22RH29 control circuitry, was noticed as having the coil spring which applies the latch preload deformed. One of the attachment loops of the coil spring had become extended l increasing the overalllength of the spring by 3/64". This elongation i effectively removed the spring force seating the latch when the OPERATE 10
i l . .
'Rost Causs An: lysis cf struthsrs-Dunn Opsrata Rasst Rstsy Latch Failuras CR# 960906195 fcr Salern G:n:r ting ststicn coil was energized and allowed the armature to repeatedly return to the l
reset condition. The elongated spring was replaced with a properly formed spring and the relay was observed to function without subsequent problems.
Change Analysis, performed by FPI International, identified that the current relay design had reduced the elongation of this spring and resulted in a reduction of latching force being applied. Review of FPI's finding with MSD indicated that the distance between the attachment anchors for the coil spring had reduced when the reset coil attachment bracket was changed sometime in 1993. This reduction in spring elongation, coupled with variations in spring constants with a given set of springs, resulted in some of the relays having marginallatching capability.
When the OPERATE coil is energized, its armature is drawn toward it beyond the amount required to seat the latch. When the coil deenergizes the contact springs attempt to return to the reset state. N.ormally the latch prevents the return to the reset state. However, in those relays where the spring is weak, the impact of the operate armature hitting the latch results in the latch " bouncing" out of position and the relay returning to the reset condition. It was noticed by the Technicians that relays which had failed were susceptible to unlatching when the case was lightly tapped.
Electrical Failure An electrical failure of the relay to latch can result from the following means:
Insufficient voltage being applied to the operate coil The DC and AC control power busses at Salem typically are above their nominal design voltages. None of the failures occurred during a degraded voltage condition. Nominal Salem 28 VDC bus voltage is 29 VDC. Each of the failed relays was tested at minimum allowable voltage per the relay specification, i.e.,19.2 VDC (80% of 24 VDC), repeatedly without an observed latching failure. One of the failed relays was subsequently tested at a highly degraded voltage of 10 VDC and did not demonstrate a latching failure.
t l Energizing both the OPERATE and RESET coils simultaneously.
! When both coils are energized the relay's contacts will achieve the operate condition, but the latch will be raised preventing it from capturing 1
11
. __ . _ . . , _ _ . . . - . . . _ _ . - . . _ . __ -_-..m ._ _ _ _ .-
\
i 'Roct Cause Analysis cf struthars Dunn Oparata-Rasst Rslay Latch Failuras CR# 960906195 !
j for Selsm G:nsrcting Statisn i i l
the armature. If power is simultaneously removed, the final state of the j
relay will depend upon which coil's magnetic field collapses first.
4 L
The control circuits for the Boric Acid Transfer pump,28 Emergency l j Diesel Generator Alarm, and 21RH19 were evaluated for a means of inadvertently energizing both coils. Each potential path was determined i
to be protected by a surge suppression diode which would prevent current j flow through the RESET coil when the OPERATE coil was selected.
1 On October 9,1996, testing of the 22 BAT pump circuit was performed i
(WO# 960924247) to determine whether any unwarranted energization of a RESET coil existed. Voltage with respect to the negative bus was j obtained across each coil and diode as each control pushbutton was i depressed. Voltage at the Relay Cabinet was determined to be 27.21 i VDC. No improper voltage was seen the RESET coils when any control j console pushbutton was pressed.
Residual Magnetism of the RESET coil core 9 During the evaluation of a relay, its RESET coil was energized I j continuously at 24 VDC for 5 minutes. When the RESET coil was de-energized, the latch remained affixed to the coil. This was sustained for i approximately 25 minutes. Momentary reenergizing the RESET coil would re-establish sufficient residual magnetism to again hold the latch away from the armature. Nonetheless, with power removed from the
- l RESET coil and promptly applied to the OPERATE coil the latch was !
j returned to its proper position and maintained a positive latch. The l
- OPERATE coil was verified to have sufficient magnetic flux to restore the latch by blocking the armature's motion preventing the latch from i becoming dislodged by the impact of the armature hitting the OPERATE
! coil.
1 Operator Error a
} Human error can result in a failure of the relay to latch can result from the following
! means:
4
, Simultaneous demand signals to both coils i
j The OPERATE and RESET coils are manually actuated off of separate j control pushbuttons. Simultaneous operation of multiple pushbuttons is j contrary to normal plant operating practices. Additionally, the observation i were primarily made while performing Post-Maintenance Testing which is i
done in an intentionally controlled and deliberate manner to observe any j abnormalities with the component.
]
l 12 l
J _- , _ , _._ _ , _ _ _ _ _ . . _ _ _ _. - __, . _ ~ . .
Root caus2 Analysis cf struth:rs Dunn Opsrsts-R: set R:lty Letch Failur:s CR2 960906195 fcr Scl:m Gsnarcting st:tien l Insufficient duration.of demand signal it was observed that when the operate coil of the relay for the 2B EDG LOCAL MANUAL acknowledge function was momentarily energized a l failure to latch condition could be repeated. The product specification gives 25 ms as the relay operate time. This cause is unlikely to have
' occurred due to the manner in which the operators are trained to operate the controls. Console pushbuttons are held long enough to observe a direct effect of the pushbutton, e.g., valve position indication change, pump start, etc., the inherent delays would result in the pushbutton being depressed for much greater than minimum time.
Jarring of the relay 1
Mechanically agitating the relay was observed to cause the relay to unlatch. This required directly striking the relay. The relay is normally protected by the structure of the relay cabinet, preventing the necessary access.
In February 1996, PSE&G Hudson Generating Station identified to MSD a problem with 255 style relays becoming unreliable following a 30 inch drop to a carpeted floor. MSD identified that this was a result of how the operate coil was mounted to the relay frame. A drop or severe shock resulted in a change of alignment between the armature and latch. A change in construction implemented with date code 9240 (fortleth week of 1992) provided resolution. None of the recent Salem failures involved relays dated prior to 9501.
Issue #3: Spurious unlatching of the relay without a valid demand.
. Description of the issue l
The Struthers-Dunn 255 style Operate-Reset Relay is designed to maintain its last demanded condition, OPERATE or RESET, even upon loss of power and subsequent reenergizing of the relay. On separate occasions, the Loop 21 and 22 Residual Heat ,
Removal Heat Exchanger Bypass Valves,21(22)RH29, have spuriously changed from !
AUTO to MANUAL modes of operation. Each of these occurrences have been i coincident with the ffow in the respective RHR Loop being reduced to the Low Flow setpoint. With the valve in AUTO and the associated pump in service, a low flow condition is designed to open the associated RH29 valve to preserve a minimum flow flowpath for its pump.
l Causal Factor Analysis I i
i 13
l
'Roet Causo An lysis cf Struthers Dunn Op2rsta-Resst Retzy Latch Failuras CR2 960906195 for Salem Generating station '
l l Three potential causes of the spurious reset of the relays were identified: mutual i interference (Magnetic interaction) between adjacent relays, jarring of the relay, and degradation of the latch pawl.
I Degradation of the latch was eliminated due to visualinspection of the relays and i successful bench testing subsequent to the failure. No visible wear of the latch was observed nor other degradation that would weaken its performance.
Magnetic Interaction between relays is suspected due to the coincident actuation of the low flow relay, co-located within the same relay can, at the time that the change in '
operating mode was seen. The relay can containing the AUTO-MANUAL (83) relay, also contains two 115 VAC 219 style relays actuated on High and Low RHR Loop flow respectively. All three relays are mounted vertically, base down, in a line along their ;
long axis, each relay base adjacent to the next.
l This hypothesis was tested by mounting the 22RH29 255 style relay which failed in a relay module with two 219 relays. With the 255 relay latched, the 219 relays were energized without causing the 255 relay to reset. Additionally, a 255 relay was latched )
and placed alongside a 219 relay with the 219 relay coil adjacent to each side of the 255 relay. The 219 relay was energized without being able to cause the 255 relay to reset.
This potential cause was discussed with the vendor Magnacraft/Struthers-Dunn (MSD)
(Mr. Tom Mahaffey). He commented that the 219 relay design limited the magnetic flux outside of its case.
FPI international, Inc. confirmed that the magnetic field outside the case of an energized relay was too small (< 1 Gauss) to have impacted the operation of an adjacent relay.
1 Jarring of the relay is the most probable cause of the relay's unlatching. Due to the reduction of the spring's latching force, only a small external force is necessary to unlatch a failure prone relay. While the failures in this mode were observed to be concurrent with the energization of an adjacent relay, it is unlikely that any significant force was transmitted to the latching relay. The most probable initiator was opening or closing a cabinet door or floor borne vibration due to movement of a heavy load in the ,
vicinity. However, neither of these two conditions has been positively confirmed to l have occurred concurrent with the relay failure. '
Issue #4: Unsuitability of new relays.
. Description of the issue .
As pa.t of the current Salem 1 and 2 outages, a 100% inspection of the Bailey Cabinets and relays is being performed. In preparation for these activities a large quantity of 14
r .
- Rcot Causa Antlysis of struth2rs-Dunn Opsrats-Rasst Rclay Latch Failures CR# 960906195 for Sct:rn Ger. crating strtien replacement relays were purchased from Farwell and Hendricks (F&H). F&H, in turn purchases the relays from MSD and then qualifies the relays for nuclear applications.
Prior to a replacement relay's installation in the field, it is bench tested to verify its suitability. During the bench testing, a number of new relays were found to exhibit high contact resistance and/or crimped leads along the chassis of the relay.
Causal Factor Analysis The cause of the wire crimping is due to poor workmanship during the final assembly steps of the relay as the clear cover is placed over the relay assembly. This was discussed with MSD during a site visit on September 12,1996. They attribute this due to the short lead time provided in manufacturing the relays for PSE&G.
A relay with a crimped lead to the RESET coil was tested with the wire shorted to the chassis. No impairment of the relay function was observed.
High contact resistances is also attributed to poor workmanship controls during manufacture and subsequent qualification of these relays. F&H during its qualification of the relays for PSE&G identified a small number of problems with relay performance to MSD for their resolution. Five relays were returned by F&H to MSD for evaluation.
MSD subsequently reported that one did not latch reliably and two exhibited problems with contacts. The remaining two exhibited no problems. Prior to March 1996,255 series relay contacts were not 100% tested by MSD. As a result of the F&H identification,100% contact testing was initiated. MSD noted that " cleaning the contacts restored satisfactory operation. It a virtual certainty these relays would have functioned flawlessly under normal load conditions."
The single relay that did not latch reliably was attributed to a misformed armature
" catch step." The misformed armature was believed to be one that was used to set up l the press that forms the step and should not have been reused in manufacture of a I relay.
The concern with high contact resistance was determined to be a result of flux residue from the attachment of the contact pads to the contact leaf spring. " Burning in" the contact by operating the contact through repeated cycles under a 1 amp load has been shown to eliminate this residue and restore proper contact resistance. A requirement to " burn in" each relay has been added to the purchase order specification for the relays.
Prior Similar Events and Operating Experience The MMIS, TextSearch and NPRDS Databases were utilized to identify prior Similar Events and Operating Experience pertinent to this evaluation.
1 15
1
'-Rc t Causs Analysis of struthsrs Dunn Opsrata-Rasst Rslay Lttch Failures CR# 960906195 fer Salsrn Gen 2 rating station MMIS work orders in current, history, and archive status were searched for occasions where folio X 40 0407 or folio X 40 0408 parts were issued to identify replacements due to similar failures. Attachment 3 is a tabulation of pertinent occurrences.
TextSearch was queried for keywords " relay" and "Struthers" or "Dunn" or "MSD", and
" relay" and " latch" within 25 words of " failure." INPO OE 5537 was the sole instance of a reported problem with Struthers-Dunn relays. This OE notice reported that 219 style relays manufactured prior to 1983 exhibited abnormal discoloration resulting from out-gassing of the coil encapsulant. This condition, as well as any other unusual discoloration of the relay, is being identified during he current Bailey Relay inspection Program. Any relays identified as discolored are being replaced. This OE does not pertain to the current condition.
NPRDS was searched for industry experience with Struthers-Dunn relays. This search was initiated by querying for Component = RELAY, Manufacturer = MSD (formerly i Struthers Dunn). Twenty three records were identified satisfying these criteria. Four of these records involve 255 style relays. All four of these instances occurred at Salem
~
and were captured within the MMIS search. l Contact was made with the NPRDS coordinator at any facility which reported via NPRDS a problem with MSD manufactured relays. None of the facilities identified any l usage of B255 style relays. I l
As noted previously, F&H identified problems, predominantly with relays produced early in the PSE&G order, to MSD during their qualification testing. For their qualification of the relays F&H tests 100% of the relays prior to shipment to PSE&G.
PSE&G Hudson Generating Station identified a problem with relays easily coming out of alignment due to dropping. MSD responded by stating that the problem had been l previously identified and corrected in 1992. l l
Generic implications These relays are used throughout the control circuitry at both Salem units. Similar relays are used at Hope Creek Station. Following formation of the Bailey Relay Project team, an evaluation of relays used at Hope Creek was performed. This evaluation I determined that no relays manufactured after September 1992 had been installed at l Hope Creek. Therefore, none of the Hope Creek relays were susceptible to this latching failure due to the reduction in spring elongation. l Safety Significance The relays involved with this evaluation are utilized in the control systems for both safety-related and non-safety related components throughout both Salem units.
16 l l
1 Root Causa Analysis of struthsrs-Dunn Operats Ros2t Rafay Latch Failurss CR# 960906195 for salem Generating station All of the failures identified to date have involved DC powered relays, predominantly of 1 the 24-28 VDC ratings. In addition to the 24-28 VDC relays, Salem utilizes 125 VDC l relays (two failures) and 110 VAC relays (0 failures) of the B255 style. All of the recent I failures have been with relays staged for or installed in the Bailey Control Cabinets.
The 24-28 VDC relays typically are used in the interfacing circuitry between the operator's control station pushbuttons and the subsequent actuation of the actual device. Within these applications the relays typically serve to establish and maintain the last selected
' AUTO / MANUAL mode of operation (83 device) !
Breaker demand position (checking relay (3 device)) l Alarm acknowledge (74 device) I The 125 VDC relays typically provide breaker controls for the actuated device.
Per the UFSAR SER Section 7.3.4, "the automatic operation of the ESF equipment is independent of any action in the 28V circuitry."
Nonetheless, improper operation of the relays imposes additional demands and distractions on the operators. Additionally, failures, such as the RH29s spurious shift to MANUAL, can adversely impact the operation of the controlled equipment. In the case of the RH29s, a spurious shift to MANUAL can result in operation of the associated RHR pump with less than minimum pump flow. A complete search of all affected control circuits was not conducted to determine the extent of this problem.
Plant mode specific safety evaluations in accordance with 10CFR50.59 are being performed prior to the entry into any defined mode to ascertain the impact of any 1 component which may have a failure prone relay within it? control circuitry. These safety evaluations will be attached to this report as they t;;come available.
l l
l l
17 l 1
I
'Rost Ceuss Analysis cf Struthsrs Dunn Opsrata Rasst Relay Latch Failuras CRS 960906195 for S: Ism G2n: rating Station la CORRECTIVE ACTIONS Corrective Actions and Accountabilities Comoensatory Actions Upon identification of the failure of a relay in service troubleshooting and repairs are performed per the Work Control Program (NAP-9). Identification of the suspect relay, bench testing and installation of a replacement relay is performed. Following l replacement, testing of the affected circuit is conducted prior to its being returned to j service. Failed relays are turned over to the System Manger for evaluation.
l Interim Measures Bailey Relay inspection procedure, SC.MD-PM.ZZ-0204(Q) Rev. O, was issued on l 9/13/96. This relay provides consistent direction for the performance of the relay inspections and establishes formal acceptance criteria for the evaluation of the relays as well as a permanent record of the findings.
Salem System Manager has established a database to compile the details of relay problems. This database, if maintained, will facilitate trending for future problems.
Corrective Actions to Prevent Recurrence Resolution of the latching problem has been developed by the replacement of the letch coil spring with a shorter spring. This replacement spring was identified by MSD an:1 1 has been used with the B255 style latching motor in other MSD manufactured relays.
The replacement of the spring restores the latching force to levels consistant with the 1970 vintage relays. A two phase approach has been established to acquire replacement relays with the spring modification. Initially, post September 1992 manufactured relays have been returned from folio to F&H. F&H has committed to replace the spring and rededicate the relays for PSE&G's use. Additional relays are being purchased via F&H from MSD which incorporate both the different spring and a change in the wire gauge used in the RESET coil. The larger diameter wire in the RESET coil provides additional manufacturing margin to assure the relay's performance under degraded voltage conditions. Both the refubished relay and the newly manufactured relay have been assigned new folio numbers to aid in the tracking of their installation in the plant.
Bailey Relay inspection procedure, SC.MD-PM.ZZ-0204(Q) has been issued and upgraded to incorporate the findings of this root cause investigation. Additionally,
- alignment of the relay testing protocol between PSE&G and F8H has been established
! to improve the acceptability of relays received by PSE&G.
l l
l l 18 I
t- - . . -- - . . - - - - - - -
. -__ .=_ .- . .. . _ _. .. . ..
~ Root Causs Analysis of Struthers Dunn Oparata Ras:t Rafay Latch Failures CR# 960906195 for Silim Gsn:irating station Continued 100% augmented receipt inspection of Bailey relays will continue until adequate quality levels have been demonstrated by the vendor.
CRCA # Q Responsibility: Dept.: xxxxx Due Date: 12/31/97 Replacement of B255 Style Latching Relays manufactured after September 1992 is to be performed in accordance with the schedule developed by the Bailey Relay Project and the requirements of each Mode's Safety Evaluation.
CRCA # Q Responsibility: Dept.: SMD Due Date: 2/1/96 Safety Evaluation of the potential impact of the suspect 8255 relays upon the upcoming mode shall be performed prior to mode change until the suspect relay population has been replaced with upgraded / newly manufactured relays CRCA#Q Responsibility: Dept.: ESSA Due Date: 2/1/96 Measures to Evaluate CA Effectiveness and Accountabilities Evaluation of the in-plant performance of the upgraded / newly manufactured B255 relays will be performed by trending relay data from both the system manager's relay database and maintenance rule component performance data.
CRVR # Q1 Responsibility: Dept.: ESSA Due Date: 12/31/97 19
l l
Root Ceuss An !ysis of struthsrs-Dunn Opsrats-Rasst Rslay Latch Failuras CR# 960006195 for salem Generating station 6A OTHER ISSUES AND IMPACTS Issues Beyond the Scope of Investigation Several procurement issues appear to have contributed to the bailey relay problem.
These are indicated by The time pressures applied to the manufacturer and subsequent lapses in quality of the relays received.
Miscommunication of the qualification requirements for the relays to F&H.
Differences in testing protocols / requirements between PSE&G and F&H.
Nuclear Procurement and Materials Management recently completed a level 1 Root Cause investigation into the " Effectiveness of the NBU Materials Management Process", CR 960729163. This report addresses similar procurement 'concerns identified over the same interval as the purchase of these relays. As a result, this Bailey Relay investigation did not pursue the procurement issues.
Common Mode Failures, as Appropriate The degradation in the functionality of the relay latching mechanism established a means of common mode failure within the various systems that utilize these relays in their control scheme. As a result, mode changes have been delayed until the potential impact have been evaluated and necessary equipment for the forth coming mode restored to operability by the installation of an upgraded / newly manufactured relay.
. Reportability These occurrences have been reported tot he NRC by means of a four hour report on November 5,1996 and LER 272/96-031-00.
Part 21 notification was intiated on November 1,1996 by F&H.
. Industry Information Dissemination Industry dissemination will be provided by the Part 21 notification initiated by F&H.
I i
20
. e Rcot Causs Analysis of Struthsrs Dunri Opsrato-Rosat Rslay Latch Failuras CR# 960906195 for Salem Generating Station i
7.0 SOURCES OF INFORMATION l
. Contacts
- a. Thomas Mahaffey, Product Engineering Mgr., Magnecraft/Struthers-Dunn, (803) 395-8512
- b. Larry Bowman, Regional Sales Mgr., Magnecraft/Struthers-Dunn, (717) 764-698
- c. Mike Wooldridge, Systems Engineering Manger, Farwell and Hendricks, (513) 528-7900
- d. Matt Artz, Contract Engineer, Farwell & Hendricks, (513)528-7900
- e. Brian Miracle, Test Engineer, Farwell & Hendricks, (513)528-7900
- f. Mike Bell, Test Engineer, Farwell & Hendricks, (513)528-7900
- g. Len Rajkowski, Bailey Relay Replacement Manager (x-5142)
- h. Sandra Jannetty, System Manger (x2704)
- 1. Mike Panko, System Manger (x-7192)
- j. Ken Staring, System Manger (x-2157) !
l
= References
- a. Maintenance and Application Guide for Control Relays and Timers, NMAC Report TR-102067
- b. Protective Relay Maintenance and Application Guide, NMAC Report NP-7216
- c. Engineer's Relay Handbook, revised second edition, National Association of Relay Manufacturers
- d. Struthers-Dunn Commercial / Industrial Relays Catalog
- e. Wyle Laboratories Report, Report Number 45053R95, " Failure Analysis of
, Struthers-Dunn Relays From Public Service Electric And Gas Co.", February 19 l 1996 i f. Vendor Telcon Summary, Tom Mahaffey, MSD Magnacraft and Ken Staring, l j Salem System Engineering,6/13/1995 l 21
-- - I
_ _= - . . _ . _ _ ~ . -. -_ - -- . -. - .- .
'Rost Caus2 Analysis cf Struthars-Dunn Optr:ta-Ras:t R: lay Litch Failuras CR# 960906195 tor Sal;rn Gintrtting Statisn
. g. Maplewood Research and Testing Laboratory Report, Report # 77299, " Energy-Dispersive X-ray Chemical Analysis of Particles Found Inside Struthers Dunn Type 219 ABAP Relay from 21RH29 Circuit (63LX/RBA), No. 2 Unit - Salem Generating Station", July 12,1996
- h. Maplewood Research and Testing Laboratory Report, Report # 72668,
" Examination of Relay from Reactor Control System, Salem Generating Station",
June 15,1989 i i. Salem Significant Event Response Team (SERT) 95-02 Report, " Salem Unit 2 -
Reactor Trip on Partial Loss of Switchyard June 7,1995", July 18,1995
, J. FPI International, "Struthers-Dunn Model 255XCXP Relay Root Cause Evaluation for Salem Nuclear Generating Station", October 20,1996, FPI 96-829 8A INVESTIGATOR COMMENTS None
,o_0 APPENDICES &_ ATTACHMENTS
- 1) Attachment 1 Failure Mode Chart
- 2) Attachment 2 NPRDS Data
- 3) Attachment 3 MMIS Data
- 4) Attachment 4 Work Standard BAILEY 001
- 5) Attachment 5 FPI Root Cause Report 22
~~ Root Causs Analysis of Struthsrs-Dunn Operata Resst Relay Latch Failuras CR# 960906195
! for Salem Generating Station l
Figure 1 l
l Struthers-Dunn J
255 Style Operate Reset Relay l 1
i i
Reset Coil
\ m 6
7 T- N.
! ?' , ?
g= gi a
5
, ,1 1
ha crQ
,F Latch / Pawl
- p < >
i Contact Slider #
GOOD .
mn
-', ( < -Operate Coil
. l - -
[ .
Armature / Fr- 'tr -
i
%m
\
g t .i 1. l I * - < . - .. r- _
i l
l
( t ) -
! l C D 0 0 C
/
l 1
i f
i 23
l
~
Root Ceuse Anal) sis of Struthers.Dunn Operate-Reset Relay Latch Failures CR# 960906195 for Salem Generating Station Attachment i 1
OPERATE-RESET Relay I falls to maintain latched condition if 1 f 1( 1 f
Mechanical fa0ure of Other Electrical failure relay.
1f Latch Pawl Does Not Other Engage.
i 8 t t t t t t Weak / Latch Pawl Latch Pawl Operate Distorted Operate Travel Misformed / Latch Pawi Armature Armature Latch Blocked Wom Misaligned Spnng. Cocked Blocked ,
I FPI report No debris No wear of Alignment Maint. Tech No debris identfied noted. pawl fixed at two .9eted that noted.
reducton in Latch observed. points by cocking Armature applied observed to Latching l coil screw armature observed to latching be free to edge is and bracket could cause be free to force due to travel. square with tab. Anti. failure to travel, insuffi ent " Freeze sharp tamper paint latch. I elongaton of Plug
- 90* comer applied at Cocking is cod spring.1 installed to factory self l preclude correcting.
i galling to 1 coil. 1 1 f l Y Y t 1r Residual Magnetism "
of eset
- Sneak Circuit
- NoTNear y elay Pushbutton Staton Energeing Reset Coil. Operaton Problem.
Failures Reset coil down to 0.2 Able to influence. PB staton Problem mainly on can attract VDC. maintain Able to problem would not new relays. latch at Able to latch lifted replicate noted on clear by No a24 VDC replicate with outside some some replacing magnetism (-9 VOC some magnet on failures on completed relay.
noted dunng observed at f ailures on top of case. bench in work orders.
inspection. factory) bench in Adjacent isolaton.
remains isolaton. relay cod too attracted far to l Figure 2 24
e Root Csese A sttysis ofStruthers-Duna Operat:-Reset Relay Latch Fdlures CR# 960906195 for Sdem Genertting Striion Attachment 3 ,
Y I Y op.e.a.g een.h yeet Veenage Out operogen est 88Spe8 appropriate.
I I V Y
.Y I I Y. vennee 4,,ned se, High U" h* ehert
- senery Ememe geen,geee duremen Veaage Lhe Leesee Connects Lew Seneh Teeens wee redewed and found seneintent seen me genery specetcasene end requeements vestege es te releye. kaprovements of Reeted et ueegured et
-30 WDC gie sneeng and easement wh Besey FaH have been hcarporated noe.ees Ceteset has une hopectea precedure.
W ee encePeene rease in 24 - et 27.2 VOC 28VDC t t p*
C A **4d
..e,, e. - # a=
toe ehert e Operate hohn Y Y Pushbumene were e egnamined b foley. for entended pertos fouewhg h sksulky hw Wo. Smh3 demande rapidly did met escut.
Measured Idenamed weltage et gus residuo cet in 26.7 as sense et VDC high resistense sentmem.
- Bern h* et senteem ramewee remasue and essesree proper seatect reelstens o.
25
Root Cruse Anilysis of Struthers-Dnn Opertta-Reset Riley Latch Frillres CR# 960906195 for Srlem Gezerrting Striion Attachmeat 3 ,
WO# Component Remarks Aeid 920925171 21CW115 3A-4 Replaced relay (History 1/93) 950209135 21RH29 63LX/RBA
- Appears degraded" replaced (History 8/95) 950216204 2BGGD 69/MB Did not get Mimic Bus interlock when PB pressed. Found high resistance contacts.
Also found faulty Push-button station. (WIP) 950402084 21 PriWtr Pp 8W ' Sluggish operation ribserved 6 (History 6/95) 950617073 23CFCU 52Z-2/3 No problem identified. Could not reproduce failure. (History 7/95) 63LX, 63
- 950704047 22RH29 Replaced relays (History 8/96) 8 52A-2 960226072 Failed Operate-Reset relay, Valve failed to open position. Repeated after control 22SJ33 3/22SJ33 (CR# 960808049) bezel swapped. (RDYRT) 960226083 (CR# 960907137, 22 BAT Pp Failed to latch 83- 2 (WOCPT) 960829221) 960616106 28EDG 74-1 Would not remain latched. Failure observed with 27VDC applied, consistently would not latch at 25VDC. (History 8/96) 960630090 22RH19 3/22RH19 Failed to latch. Repeated twice on bench, then functioned properly (History 7/96) 960827264 (CR#960830239, 22RH29 83 Swapped to Manual while decreasing flow 960906195) 69 Emerg Lube Oil Alarm flashes with pump OOS. Replaced relays 891006093 22 SGFP 72X (History 11/89) 890421066 21 BAT Pp 18SS Pump would not start in Slow Speed. Replaced Relay (History 5/89) 20
Root Cctse Anlysis orStruthers-Dmm Oper:.t> Reset Rel y Latch Failures CR# 960906lb * ';
ror Salem Generating Station Attachement 3 ,
gg Component Remarks S 'd 900601202 74-1/FL-2 Alarm would not acknowledge. Repalced Relay (History 7/90) t an eve 74 I alann cannot M h@. RepW Relay (Histm 880803143 alarm ;
880808143 2 ECAC 52-Z Tripped. Replaced Relay (History 9/88) 910605079 I 52Z 3
Pump j 900620101 OHA E-44 6Z Will not alarm. Replaced relay (History 8/90) 890912154 3-3 Will not start. Replaced relay. (History 10/89)
Su Fan 12 Ckc Breaker will not close. Found bad NC contact on relay, replaced 891207122 3 Breaker relay. (History 12/89)
~
Found contact 4 & 5 welded together. Replaced Relay (History 900910172 12 SGFP 6 10/90) ,
1 FHB !
910106091 3 Couldn't stop form control Room. Replaced Relay (History 1/91) y S
Fan 910503102 74H Found open reset coil. Replaced relay (History 10/92) j Al ,
880803087 11 Chg Pp 52Z-2 No STOP indication. Replaced relay I
27
- - - - - - ~
, ~ , , -.,-n r~
Root Cexse Antlysis ofStrzthers-Dun Opertt> Reset Rel y Latch Failures CR# 960906 L.
ror Salem Generating Station Attachment 3 .
Component Remads WO# Affected 13 Vac Impr Per indication. Found relay not fully inserted, corrected 890605115 52/Z Pmp (History 7/89) l 910916112 52Z-1/1 Breaker cycled. Found bad relay. Replaced (History 12/91)
Ven an 911030113 12 CFCU 52Z-1 Contact not making up. Replaced (History 6/92) 11 BSCE 890207151 52-Z incorrect Indication. Found bad 52-Z relay, replaced (History 1/90) pp 13 Pp 910610158 52Z Found burned relay, replaced (History 1/93) 910610160 12 SW PP 52Z/2 Found open coil, replaced relay (History 10/92) 910619088 11PW Pp Faulty Indication. Found burnt relay replaced (History 10/92)
Slugg sh operation, found high resistance contacts, replaced relay 940826094 1PR1 T/ POPS (History 5/95) 12 ABV 950222058 SUP F" 74-3 Could not acknowledge. Found bad relay, replaced (History 6/95) ep Alarm Isol Found bad contacts on relay (high resistance), replaced relay.
950814358 3-3 Damper (History 10/95)
Fan w uld cycle without demand. Replaced start-stop relay (37) 930927180 o e Sup Fan 1
28
Root Cctse Andysis orStruthers-Dn:a Opertt> Reset R lay Latch Fdlures CR# 960906195 ror Salem Generating Station Attachment 3 Component Rernads WO# c ted 920601196 CVC M/U Swaps to STOP when PW Pp start acknowledged. Found sticking 83-1/MU Mode Sel contacts on relay, replaced. (History 10/92) 1 Demin START indication with pump OOS. Found bad 69 relay; replaced 930726085 69 Trf Pp (History 10/92) 1B EDG 931031082
^ 74-1 Would not acknowledge. Replaced Relay (2/22/96)
M UA Alarm 12 RHR Out Could not acknowledge Replaced relay due to high contact 950419125 74-2 Low Flow resistance. (History 11/95)
Alarm 950802386 CAV 3-3 Found stuck contacts on relay, replace (History 8/96) 18 920815078 74-4 Relay not staying latched. Replaced (History 10/92) 921227103 1EPX Brk 52-Z Breaker will not close from CR. Replaced relay (History 2/93) 950624085 13CW26 52Z-2 Valve cycling open - closed found bad operate coil (History 8/95)
Pump failed to start. Found bad relay contacts replaced. (History 960105092 15 SW 52Z-1 9/96)
V Ive returned to close position after stroking open. Relay replaced 961119257 21SJ113 ?
as part of Bailey Relay Project.
Valve returned to close position aller stroking open. Relay replaced as part 961119226 22SJl13 ?
of Bailey Relay Project.
I' 29
Root Cw.se Arlysis of Struthers-Dunn Opeat> Reset Relay Latch Failures CR# 960906195 for Salein Generating Station Attachsment 3 .
961126189 2DAEl1 86ADX-3 Failed Relay. Relay replaced under wo 961126189 l
e l
n r
i l
l l
4 30 i
_ - _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _