Information Notice 1992-04, Potter and Brumfield Model MDR Rotary Relay Failures

Potter and Brumfield Model MDR Rotary Relay Failures

ML031200770
Person / Time
Site:	Beaver Valley, Millstone, Hatch, Monticello, Calvert Cliffs, Dresden, Davis Besse, Peach Bottom, Browns Ferry, Salem, Oconee, Mcguire, Nine Mile Point, Palisades, Palo Verde, Perry, Indian Point, Fermi, Kewaunee, Catawba, Harris, Wolf Creek, Saint Lucie, Point Beach, Oyster Creek, Watts Bar, Hope Creek, Grand Gulf, Cooper, Sequoyah, Byron, Pilgrim, Arkansas Nuclear, Three Mile Island, Braidwood, Susquehanna, Summer, Prairie Island, Columbia, Seabrook, Brunswick, Surry, Limerick, North Anna, Turkey Point, River Bend, Vermont Yankee, Crystal River, Haddam Neck, Ginna, Diablo Canyon, Callaway, Vogtle, Waterford, Duane Arnold, Farley, Robinson, Clinton, South Texas, San Onofre, Cook, Comanche Peak, Yankee Rowe, Maine Yankee, Quad Cities, Humboldt Bay, La Crosse, Big Rock Point, Rancho Seco, Zion, Midland, Bellefonte, Fort Calhoun, FitzPatrick, McGuire, LaSalle, Fort Saint Vrain, Shoreham, Satsop, Trojan, Atlantic Nuclear Power Plant
Issue date:	01/06/1992
Revision:	0
From:	Rossi C E Office of Nuclear Reactor Regulation
To:
References
IN-92-004, NUDOCS 9112300138
Download: ML031200770 (10)

.gUNITED STATESNUCLEAR REGULATORY COMMISSIONOFFICE OF NUCLEAR REACTOR REGULATIONWASHINGTON, D.C. 20555January 6, 1992NRC INFORMATION NOTICE 92-04: POTTER & BRUMFIELD MODEL MDR ROTARY RELAYFAILURES

Addressees

All holders of operating licenses or construction permits for nuclear powerreactors.

Purpose

The U.S. Nuclear Regulatory Commission (NRC) is issuing this informationnotice to alert addressees of failures experienced with MDR series Potter& Brumfield (P&B) rotary relays installed in safety-related systems at certainnuclear power plants. It is expected that recipients will review the informa-tion for applicability to their facilities and consider actions, as appropri-ate, to avoid similar problems. However, suggestions contained in thisinformation notice are not NRC requirements; therefore, no specific action orwritten response is required.

Description of Circumstances

On January 14, 1986, September 17, 1987, and December 8, 1987, an emergencydiesel generator (EDG) failed an operability surveillance test at the LaSalleCounty Station, Units 1 and 2. In each case, while the Commonwealth EdisonCompany (CECO) attempted to synchronize the EDG to its bus, the EDG outputbreaker would not close. CECO replaced all P&B MDR relays in the outputbreaker closing circuits with General Electric HFA relays. The NRC staff hasreceived no reports of relay failures at LaSalle affecting EDGs since thesewere replaced.On October 10, 1988, the Arizona Public Service Company (APSC), the licenseefor the Palo Verde Nuclear Generating Station, submitted a report in accordancewith Title 10 of the Code of Federal Regulations, Part 21 (10 CFR Part 21).This report documented 18 instances over a 2-year period in which P&B MDRrelays failed to change position.APSC detected these failures during either routine surveillance testing oractuation of the engineered safety features (ESF) actuation system or thereactor trip switchgear. After replacing all P&B MDR series relays, APSCexperienced only two failures; improperly sized coils or contamination in theinsulating material of the switch caused these two failures.9112300138 PDK SV(E to<tr 1a*~~~ 9/&, 1 IN 92-04January 6, 1992 On July 19, 1991, during a monthly surveillance test, the River Bend Stationexperienced ESF actuation of containment isolation valves, control room filtertrains, the standby gas treatment system, and the fuel building filter trainsbecause of an MDR relay failure.On July 23, 1991, during a monthly surveillance test at the River Bend Station,the failure of an MDR-5111-1 relay caused an ESF isolation of a reactor watersample valve. The Gulf States Utilities Company (GSU), the licensee, committedto replace all P&B MDR relays.DiscussionP&B MDR relays are used in various safety-related applications in commer-cial nuclear power plants with reactors manufactured by the Babcock and WilcoxCompany; Combustion Engineering, Incorporated; the General Electric Company;and the Westinghouse Electric Corporation. Industry records identify over60 instances of P&B MDR rotary relays failing to operate properly since 1984.An MDR relay failure may cause the loss of a train-of the ESF actuation system,the emergency core cooling system', or the reactor protection system. A common-mode failure may result in the loss of one or more of these systems. GSUperformed a probabilistic risk assessment (PMA) of the reactor protectionsystem at River Bend, based on plant-specific, P&B MDR relay failure rates thatwere greater than the generic failure rates by a factor of 5.1. This PRAshowed that the reactor protection system failure rate increased by a factor of25 to 3.3E-4.The principal failure mechanism of P&B MDR rotary relays'appears to be mechani-cal binding of the rotor'caused by deposits from coil varnish outgassing andchlorine corrosion products. , A secondary failure mechanism appears to be theintermittent continuity of electrical' contacts. A number of variables contributeto these failure mechanisms and cause the relays to fail at random mostlywithin 2 to 5 years of the in-service date. 'Failures may occur regardless of.current or wattage, the use of ac or dc power, or whether normally energized orde-energized. It is also important to note that a relay rotor can bind immedi-ately 'after a surveillance test.Attachment 1 provides a detailed description of the failure mechanisms, con-tributing causes, and failure investigations. Attachment 1 also discussesmodifications made to P&B MDR series relays by the manufacturer to reducesusceptibility to the failure mechanisms discussed above.,ae ..

IN 92-04January 6, 1992 This information notice requiresyou have any questions about theof the technical contacts listedReactor Regulation (NRR) projectno specific action or written response. Ifinformation in this notice, please contact onebelow or the appropriate Office of Nuclearmanager.Charles E. Rossi, DirectorDivision of Operational Events AssessmentOffice of Nuclear Reactor RegulationTechnical contacts:K. R.(301)Naidu, NRR504-2980R. A.(301)Spence, AEOD492-8609

Attachments:

1. Potter & Brumfield Model MDR Rotary Relays2. Figure 1, Potter-Brumfield Model MDR Rotary Relay3. Figure 2, MDR Non-Latching Relay4. List of Recently Issued NRC Information NoticesCamsjl He6LS,'

Attachment 1IN 92-04January 6, 1992 Potter & Brumfield Model MDR Rotary RelaysDescription of the MDR Rotary RelayPotter & Brumfield (P&B) manufactures two types of MDR rotary relays: latchingand non-latching. Various series of these relays are provided for service at28 and 125 volts (V) dc and 115 and 440 Vac, with from 4 to 24 pole doublethrow (PDT) contacts. While each series has a different number of contactstacks and has a different coil, power, and current capacity, each of theseries is similarly constructed and exhibits similar failure mechanisms.Non-Latching RelaysThe non-latching MDR relay has two coils connected in series inside the relaywhich, when energized, rotate the relay rotor shaft, which operates the con-tacts through a shaft extension. The stator faces and stop ring limit therotor movement to a 30-degree arc. Two springs return the rotor to the stopring and the contacts to their normal positions when the coils arede-energized. The non-latching MDR relays have two positions: 'energized" and"de-energized." (See figures 1 and 2).Latching RelaysEach relay in the MDR latching series has two sets of series coils, whichprovide a latching two-position operation. When one set of coils is energized,the rotor shaft rotates through a 30-degree arc, changing the state of thecontacts. The other set of coils must be energized to return the relay to itsoriginal position.Failure InvestigationsThe Commonwealth Edison Company (CECO) determined that the three events at theLaSalle County Station resulted from the failure of P&B MDR-137-8 or MDR-138-8,125 Vdc normally energized relay contacts to close. CECO performed diagnostictesting after the earlier events but could not repeat the failure. This lackof repeatability is typical of MDR intermittent failures.The Arizona Public Service Company (APSC) found that three of the P&B MDR relayrotors at Palo Verde Nuclear Generating Station (PVNGS) would not move morethan 12 degrees of the complete 30-degree arc. The failed relays, locatedin cabinets without forced ventilation were in an ambient temperature of95 to 104OF (the design limit is 1490F5 and had an external surface temperatureof 1570 Attachment 1IN 92-04January 6, 1992 APSC detected no relay failures ii cabinets with forced ventilation whichprovided an ambient temperature of 81'F or less. Such relays had a temperatureof 112'F on their external surfaces. APSC determined that it had applied up to39.8 Vdc to the 28 Vdc coils. APSC tested 7 of the 18 failed relays on an18-month frequency and 10 on a 62-day frequency. APSC had the relay failuresanalyzed and determined that varnish on the relay coils outgassed, condensed,and accumulated between the rotor shaft and the end-bell bearings, binding therotor and the bearings together. The outgassing was due to excessive coiltemperatures that occurred when the coils were continuously energized atvoltages above their nominal ratings. The heat also may have caused therelease of chlorine from (1) the PVC coating on the fiberglass tubing coveringthe solder joint between the magnet wire and the Teflon coated lead wire, and(2) the Neoprene rubber grommet through which the coil lead wires penetrate thebase of the relay. The chlorine corroded brass parts inside the relay. P&B andAPSC concluded that long intervals between de-energizing of the relays may havealso contributed to the failures.In May 1989, APSC installed replacement P&B relays at PVNGS that were manufac-tured with coils'coated with epoxy instead of varnish. APSC conducted tests andfound that 5 of the 42 relays tested would not rotate to their de-energizedposition and that 5 other relays operated slowly. Two independent laboratoriesobserved that; (1) the relays' epoxy was not properly cured, (2) uncured epoxycontaminated the rotor and (3) P&B did not de-aerate the epoxy prior to use,contrary to the manufacturer's recommendations. This caused the rotor andstator surfaces to bond together, preventing the rotor from rotating freely.P&B informed the NRC that APSC returned the 42 relays and that P&B rebuiltthem.-On September 10, 1990, the General Electric Nuclear Energy Division (GENE)issued Rapid Information Communication Services Information Letter 053 toaddress P&B MDR relay failures reported at two GE boiling water reactors. P&Bbelieved that chlorine released from rubber grommets and polyvinyl chloridesleeves caused corrosion and that varnish on the-coils outgassed while therelays were continuously energized.' Both chlorine-corrosion products andvarnish accumulated in the bottom end-bell bearing and caused the rotor shaftto bond to the bearing. P&B suspected that the failed relays were exposed tohigh ambient temperatures and could have been exposed to high coil voltages orcould have been rarely cycled.On November 2, 1990, GENE-issued Potentially Reportable Condition 90-11 inwhich it stated that both 24 Vdc and 120 Vdc'coils had lower coil powers thanthe 125 Vdc relays and were therefore not vulnerable to this failure mode.GENE concluded that no substantial safety hazard existed. However, uponinvestigating the failed MDR relays at River Bend as discussed below, the NRCobtained results that may contradict these conclusions.On July 19, 1991, a high resistance on one set of contacts on a P&B 24 Vdc,MDR-5111-1 rotary relay, which should have been closed, caused a voltage dropto the downstream relays which opened their contacts and resulted in an ESF Attachment 1IN 92-04January 6, 1992 actuation at the River Bend Station. The Gulf States Utilities Company (GSU),the licensee, later performed bench testing of this failed relay and verifiedthat the relay actuated properly and all contacts changed state properly, andexhibited proper continuity. The coil was meggered and found to be acceptable.The contacts all appeared to be clean and shiny, with no evidence of pitting orresidue. GSU found no foreign material in the relay or on the rotor shaft andfound nothing that may have contributed to the high resistance across thecontacts.On July 23, 1991, GSU investigated another MDR relay failure at River Bend andfound two MDR-5111-1 relay contacts open that should have been closed when thecoil was energized. GSU also found that the contacts operated intermittentlywith some contacts closing several minutes after the coil was energized orsometimes not at all.Both River Bend failed relays had been in service within tightly-regulateddesign voltage and temperature conditions and were mounted inside stainlesssteel isolation cans for divisional separation. GSU measured the temperatureinside the isolation can at 1130F, while the ambient cabinet temperature was920F. In each case, the failed relay had been recently cycled because of ashort loss of power to the coil that had occurred a few days before the relayfailure was discovered, and it appears that not all contacts engaged properlywhen power was restored.Failure MechanismsThe primary failure mechanism of the P&B model MDR rotary relay appears to bea mechanical binding of the rotor caused by organic outgassing and depositionof contaminants and corrosion particles on the relay rotor shaft. The contamin-ants are deposited in the end bell bearings and sleeves and cause the rotorshaft to bond or stick to the bearing, preventing the rotor shaft from fullyrotating when the relay coils are energized or de-energized. The principalcontaminant is outgassed material emitted from the brown enamel varnish used tocoat the relay coils. This contamination may not be apparent to the naked eye.The corrosion results from chlorine released from the rubber grommets and thepolyvinyl chloride sleeves. Gulf States and P&B disassembled six operable andtwo failed relays that had been in service since December 1984. The thicknessand color of the deposits on the rotor, sleeve, and end-bell bearings of therelays varied widely among the eight relays, indicating varnish outgassing.A secondary failure mechanism appears to be intermittent continuity of the elec-trical contacts. High resistance and intermittent continuity may result fromchemical reactions on the fixed and movable silver contacts. P&B tested aMDR-5112-1, 125 Vdc relay that had been in service at River Bend and foundintermittent continuity on a set of clean, unused contacts.A number of variables contribute to these failure mechanisms and reduce thelength of the operating life of the complex P&B MDR rotary relays. Thesevariables include coil wattage, applied ac or dc voltage, normally energized orde-energized coil, manufacturing tolerances, ambient and coil temperatures,varnish thickness, mounting configurations and enclosures, cabinet ventilation, eAttachment 1IN 92-04January 6, 1992 relay breathing, testing frequency, operational cycling, the number of contactdecks, and the amperage and voltage of the contact load. These contributoryfactors cause an apparent random failure history. While each of the MDRrelays failed between 1 month to 13 years after it was placed in service, mostfailed within 2 to 5 years.Modifications to MDR RelaysP&B has made the following design changes to MDR series relays:Changed the movable contacts from silver to silver-cadmium-oxide inOctober 1985. However, P&B recommends against using MDR relays witheither silver or silver-cadmium-oxide in low current circuits.Changed the coil coating from varnish to Dolphon CC-1090 epoxy resin inFebruary 1986. This reduced the coil outgassing rate. However, P&B doesnot de-aerate Dolphon CC-1090 prior to use, contrary to Dolphon's recommen-dations. P&B informed the NRC that the epoxy manufacturer plans to ceaseproduction of this currently used and tested epoxy. The NRC is unaware ofwhen P&B will change to a new epoxy. Licensees may wish to determine ifPAB has examined the replacement epoxy for susceptibility to outgassingafter aging. Licensees may also wish to determine if P&B applies theepoxy in accordance with the manufacturer's recommendations.Replaced the brass switch studs in medium size MDR relays with stainlesssteel studs in November 1986.Began lubricating end-bell bearings in July 1988.Changed chloride-containing materials to chloride-free materials inJune 1989.Changed the rotor spacers from brass to stainless steel in May 1990.Changed the brass spring retainer in small size MDR relays from brass tostainless steel in May 1990.Changed shims from brass to phosphor bronze in May 1990.P&B had implemented all these modifications to its MDR rotary relay design byMay 1990.When APSC reported having problems with MDR relays at Palo Verde in 1988, P&Bbelieved that only relays normally energized with excessive voltage and operat-ed infrequently were susceptible to the corrosion and outgassing failure mode.PAB did not notify other licensees about these problems since this conditionappeared to occur only at plants with reactors manufactured by CombustionEngineering, Incorporated. P&B informed the NRC that since 1988 it has onlysupplied MDR relays as commercial grade components without accepting thereporting requirements of 10 CFR Part 2 Attachment 3IN 92-04January 6, 1992 MDR NON-LATCHING RELAYSPACERFigure 2 POTTER-DRUMWIELD MODEL 14DR ROTARY RELAYROTOR INDEENERGIZEDPOSITIONROTOR INENERGIZEDPOSITION(0AF-A5. 75'IIIII" .I ..COILHOUSINGROTOR ROTOR t QXSTOPS ASSIIBLY F t0 byED- N%-'I-.'3& I ..I Attachment 4IN 92-04January 6, 1992 LIST OF RECENTLY ISSUEDNRC INFORMATION NOTICESInformation Date ofNotice No. Subject Issuance Issued to92-0392-0292-0191-8791-8691-8591-8491-83Remote Trip FunctionFailures in General ElectricF-Frame Molded-Case CircuitBreakersRelap5/Mod3 Computer CodeError Associated with theConservation of EnergyEquationCable Damage Caused byInadequate Cable Installa-tion Procedures and ControlsHydrogen Embrittlement ofRaychem Cryofit CouplingsNew Reporting Requirementsfor Contamination Events atMedical Facilities(10 CFR 30.50)Potential Failures ofThermostatic Control Valvesfor Diesel Generator JacketCooling WaterProblems with CriticalityAlarm Components/SystemsSolenoid-Operated ValveFailures Resulted inTurbine Overspeed01/06/9201/03/9201/03/9212/27/9112/27/9112/26/9112/26/9112/20/91All holders of OLs or CPsfor nuclear power reactors.All holders of OLs or CPsfor nuclear power reactors.All holders of OLs or CPsfor nuclear power reactors.All holders of OLs or CPsfor nuclear power reactors.All licensees authorizedto use byproduct materialsfor human use.All holders of OLs or CPsfor nuclear power reactors.All Nuclear RegulatoryCommission (NRC) fuelcycle licensees, interimspent fuel storage licens-ees, and critical masslicensees.All holders of OLs or CPsfor nuclear power reactors.OL = Operating LicenseCP = Construction Permit