ML17306A334
| ML17306A334 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 12/13/1991 |
| From: | Tritch S WESTINGHOUSE ELECTRIC COMPANY, DIV OF CBS CORP. |
| To: | Murley T NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM), Office of Nuclear Reactor Regulation |
| References | |
| REF-PT21-91 ET-NRC-91-3627, NUDOCS 9112230189 | |
| Download: ML17306A334 (26) | |
Text
ACCELERATED DISTRIBUTION DEMONSTRATION SYSTEM REGULATORY INFORMATION DISTRIBUTION SYSTEM (RIDS)
ACCESSION NBR: 9112230189 DOC. DATE: 91/12/13 NOTARIZED: NO DOCKET g FACIL:STN-50-528 Palo Verde Nuclear Station, Unit 1, Arizona Publi 05000528 STN-50-529 Palo Verde Nuclear Station, Unit 2, Arizona Publi 05000529 STN-50-530 Palo Verde Nuclear Station, Unit 3, Arizona Publi 05000530 AUTH. NAME AUTHOR AFFILIATION TRITCH,S.R. Westinghouse Electric Corp. R RECIP.NAME RECIPIENT AFFILIATION MURLEY,T.E. Office of Nuclear Reactor Regulation, Director (Post 870411 y
SUBJECT:
Followup Part 21 rept re pote'ntial for certain Westinghouse D products, utilizing dc coil assemblies,to malfunction. Caused by epoxy compound becoming semi-fluid during svc.Initially reported on 910624.Epoxy check will be incorporated.
DISTRIBUTION CODE: IE19D COPIES RECEIVED:LTR ENCL SIZE:
TITLE: Part 21 Rept (50 DKT)
A NOTES: STANDARDIZED PLANT 05000528 Standardized plant. 05000529 D Standardized plant. 05000530 D
RECIPIENT COPIES RECIPIENT COPIES ID CODE/NAME LTTR ENCL ID CODE/NAME LTTR ENCL PD5 LA 1 0 PD5 PD 1 1 TRAMMELL,C 1 1 THOMPSON,M 1 1 INTERNAL: AEOD/DSP/ROAB 1 1 RR/4)QEA OGCB11 2 2 NRR/DRI S/RVI B9 D 1 1 EG FILE 01 1 1 RES/DSIR/EIB 1 1 RG 1 1 RGN2 1 1 RGN3 1 1 RGN4 1 1 RGN5 1 1 SECY VANDERMEL 1 1 EXTERNAL: INPO RECORD CTR 1 1 NRC PDR 1 1 I
NSIC S LVER g E 1 1 1 1 D
D D
NOTE TO ALL "RIDS" RECIPIENTS:
PLEASE HELP US TO REDUCE WASTE! CONTACT THE DOCUMENT CONTROL DESK, ROOM Pl-37 (EXT. 20079) TO ELIMINATEYOUR NAME FROM DISTRIBUTION LISIE POR DOCUMENTS YOU DON'T NEED!
TOTAL NUMBER OF COPIES REQUIRED: LTTR ~~ / ENCL QO
~9
Westinghouse Nuclear and Advanced Energy Systems Technology Oivision Electric Corporation Box 355 Pittsburgh Pennsylvania 15230 0355 December 13, 1991 ET-NRC-91-3627 Document Control Desk US Nuclear Regulatory Commission Washington, DC 20555 Attention: Dr. Thomas E<< Hurley, Director Office of Nuclear Reactor Regulation
Dear Dr. Hurley:
The following information is provided to supplement that communicated to you via NS-NRC-91-3600 (June 24, 1991) which reported a substantial safety hazard under the requirements of 10CFR Part 21. The issue concerns the potential for certain Westinghouse products utilizing DC coil assemblies to malfunction due to an epoxy compound becoming semi-fluid while in service.
BACKGROUND Westinghouse was informed during 1991 of a few instances of ARD relay failures due to melting epoxy. The first was from Brookhaven National Laboratory (BNL) and based on testing from date codes slightly before and after'the BNL shipment, Westinghouse assumed this was batch related. A subsequent request from Palo Verde nuclear plant on a similar failure with a much earlier date code indicated that the occurence was not isolated to a single batch. A search of available data combined with recent reports yielded only a total of about 15 which have been reported to have failed in service due to the same phenomenon over the last ten years. Based on the process specification this epoxy was introduced for use in DC products (relays and contactors) in 1975. Both the process s'pecification and the epoxy specification were reviewed and found acceptable for application in the specified products. These products are widely used in safety systems in plants with W and other vendor NSSS. For devices that deenergize to perform a safety function, it is possible that such functions would be significantly delayed or not performed at all. For other functions, where the device is normally deenergized but could be energized for an extended period, it might not reset when demanded or it might not perform the next time.
A'r 2000i3.
y ~p Purl')
ET-ttRC-91-3627-12/13/91 -
9ii2230189 9ii2i3 1
V t
'E J
(f
Periodic testing of these functions serves to verify performance and provides confidence that the devices will perform upon demand. The majority that failed in service were discovered during initial system testing or during the first surveillance period.
The small number of identified relay malfunctions due to this issue combined with the large population of such devices in safety related service demonstrates that the malfunction of individual devices is unlikely.. The malfunction of a redundant pair of such devices is even less likely. Therefore, it is very unlikely that a complete safety function would be unavailable on demand due to this matter.
Westinghouse presented information to the NRC regarding this subject on July 16, 1991. At that time, part of the recommended corrective action was based on energizing the devices and probing the epoxy for any evidence of softening. Since then, Westinghouse has conducted confirmatory testing which resulted in a change in the recommended corrective action.
SUPPLEMENTARY INFORMATION Confirmator Test Results Infrared and Mass Spectroscopy tests confirmed the epoxy family. These tests also demonstrated that no significant impurities were present and provided a gross ratio check of the two parts (resin and hardener) that make up the epoxy compound. Differential Scanning Calorimetry and Thermal Mechanical Analysis determined the temperature for various mix ratios at which the bonds begin to break down and showed a very forgiving epoxy with regard to mix ratio. A Nitrogen Content test was used to determine the exact mix ratio of the epoxy in returned relays. Through Gelation Characteristic testing, the time at which bonding begins was determined for various mixtures and temperatures. This confirmed that the cure temperature and time utilized in the manufacturing process of the DC coil assemblies was acceptable.
In addition to the above testing, many samples of various mix ratios were cured and then subjected to increasing temperature conditions to determine at what point the mixture would become soft or flow. The results of these tests indicated that the mixture would have to contain 70% or more excess hardener before it would flow. The minimum temperature to promote flow when the excess hardener is 70% was determined to be 250 F. Compounds with more than 70X excess hardener will flow at lower temperatures.
Mixtures with too little hardener would either not cure (flow at ambient temperature) or, if cured, would not flow with increasing temperature.
Samples taken from returned and stock relays were tested for nitrogen content and indicated a hardener content between 20X and over 200X. This indicated that our original assumption of poor mixing was correct.
Inadequate mixing could lead to pockets of poor compound being encapsulated in acceptable material. A few relays were sectioned to ET-NRC-91-3627-12/13/91 - 2
I I
examine this behavior. This showed that the most likely path for epoxy to reach the surface was from between the first and second pours. The second pour for these relays is only about 0.25 inches for cosmetic purposes.
Also, due to the method of pouring, the less viscous (excess hardener) mixes would rise to the top. Pockets of poor epoxy further down in the encapsulation tend to be trapped. The sectioning also demonstrated that the BNL relays had been filled with a very poorly mixed compound as excess resin was found at the bottom of the encapsulation.
These results show that it is necessary to promote flow when testing the DC Coil Assemblies to ensure that the epoxy compound is acceptable. When checking this compound at elevated temperature, it may appear soft but it is acceptable if no flowing is visible and if it doesn't show signs of separation when probed.
Coil Assembl Testin Westinghouse determined that if a coil assembly is subjected to 250 F for 2.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> in a preheated oven the entire assembly would stabilize at 250 F for over one hour. Based on the most likely path for flowing epoxy to reach the surface, it was judged that this would be sufficient time to detect suspect assemblies. Further proof was demonstrated in our commercial dedication testing which yielded 16 instances of separating or flowing epoxy from the first 122 tested. Subsequent to these conclusions, one utility subjected thirteen relay coil assemblies to 250 F for an extended period of time. Four of these did not exhibit flowing epoxy until after five hours. These assemblies were from relays that had been in service for 18 months without exhibiting any epoxy flow. Westinghouse retested these four assemblies at 250 F and the epoxy did not reflow.
This, along with in service data, indicates that the epoxy mix was marginal and once the mixture reached the surface enough hardener evaporated to cause the mixture to cure.- It was also noted that the amount of material that flowed was small and not comparable to those that failed in service. The utility also tested more assemblies and any flow that occurred did so within 2.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />. Westinghouse subjected eight relay coil assemblies that had passed the 2.5 hour5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> test to a ten day test at 250 F and although some cases cracked, no flowing or separation of epoxy was observed. Prior to the recommended 2.5 hour5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> test, Westinghouse had subjected seven different coil assemblies with various date codes to 250 F for 5 to 12 days with no occurence of epoxy failures.
Based on the initial failure rate (16 of 122) from the testingduring the commercial dedication process, Westinghouse investigated the machine mixing process used by the manufacturer for possible malfunctioning. It was determined that a check valve was sticking which would cause all of one part of the mixture to be transmitted to the mixing nozzle occassionally. This extreme "mixture" would flow at room temperature and would easily be detected in a 2.5 hour5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> test.
ET-NRC-91-3627-12/13/91 - 3
Igloo The continuation of the commercial dedication testing has resulted in a determination of flowing or separating epoxy in 40 out of 571 coil assemblies. Also, four of 96 assemblies returned from customers for similar testing were discovered to have questionable epoxy.
Surve Results The Westinghouse Owners Group conducted a survey of several sites to determine the number of installed devices and the number of additional failures due to poor epoxy. Reports from 21 plants indicated over 3000 devices installed with no epoxy failures determined in addition to those included in the original 15.
CONCLUSIONS The small number of identified relay malfunctions due to this issue combined with the large population of such devices in safety related service demonstrates that the malfunction of individual devices is unlikely. The malfunction of a redundant pair of such devices is even less likely. Therefore, it is very unlikely that a complete safety function would be unavailable on demand due to this issue.
Westinghouse also believes, based on the analysis and testing to date, that testing coil assemblies at 250 F for 2.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> will detect a high percentage of those assemblies that might fail in service due to poor epoxy.
Determining questionable epoxy by this conservative testing methodology does not mean that the device will fail in service. Based on failures reported in service to date, it is apparent that they are random in nature. Verifying the performance of normally energized applications through at least two surveillance'eriods also reduces the concern over common mode failures to the extent that subsequent failures could be considered random.
Intermittent operation for less than five minutes will not lead to sufficient heating to cause flowing to occur. Other normally deenergized applications may already have been through several surveillance periods which would create confidence in future operation.
It is also possible that rack temperatures could cause the internal temperature to exceed 250 F so that they are verified in-situ not only for normal conditions but for potential loss of ventilation. For racks that operate at lower temperatures, it is a remote possibility that a loss of ventilation could lead to coil temperatures increasing and reaching 250 F and the epoxy separating or flowing if an installed relay had more than 70N excess hardener. Surveillance testing may detect a problem before the function is required.
Based on the failure randomness, it is unlikely that redundant functions would be affected. Operator awareness can be relied on to reset or actuate necessary functions.
ET-NRC-91-3627-12/13/91 - 4
far CORRECTIVE ACTION Recommended Customer Actions A. It is recommended that all spares be tested at 250 F for 2.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />.
This testing will serve two purposes: I) provide confidence regarding operability before installation and 2) provide a data base to be utilized in establishing confidence in those assemblies that are installed. To help ensure consistency of the data received, procedures for testing each device are attached to this letter. The evaluation of the data will consist of the following:
- l. A failure rate less than 5X would indicate that it would not be necessary to risk disturbing existing installations to verify the epoxy compound.
- 2. If the failure rate of the tested spares is higher than 5X then specific date codes of the failed units should be reviewed to determine if the machine malfunction caused the high failure rate.
- a. A date code past August, 1990 would indicate that the epoxy was probably mixed by machine. If so, then only those installed units that were machine mixed would have to be reviewed. Based on the probable cause of machine malfunctioning only the applications that are normally deenergized would have to be tested. Normally energized functions would have provided sufficient heating to cause flowing in this extreme mix situation.
- b. If the date codes of the failed units are scattered then all that are normally deenergized devices (and normally energized less than two surveillance periods) should be considered for testing.
B. Regardless of the results of the spares testing, Westinghouse recommends that those installed devices that have machine mixed epoxy and are normally deenergized be considered for testing.
C. The results of the testing should be sent to Westinghouse.
Westinghouse, working with the Westinghouse Owners Group, will collect and evaluate all data from sites that perform testing on spare devices.
Westin house Action A. Westinghouse will incorporate the recommended epoxy check at 250 F for 2.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> into the commercial dedication instructions for each device that utilizes a DC coil assembly. All assemblies tested at Westinghouse will be coded with the letter N.
ET-NRC-91-3627-12/13/91 - 5
B. Westinghouse will evaluate the results of the testing of the spare devices.
PRODUCTS INVOLVED The original report (NS-NRC-91-3600) identified ARD, BFD and NBFD relays and A200DC and DPC 250 contactors as devices that utilize this epoxy A200, A210 and 250 (reversing) and A960, A970, A980 (non-reversing, two speed) starters; and A201, A211 and A251 (reversing), and A202 (lighting) contactors. The DPC 250 is also listed as DPCK 250. The same epoxy process specification is also used in larger AC devices. These devices have significantly different design features such that the contactor action would not be impeded should the epoxy flow.
If you have any questions regarding this matter, please contact Mr. R. B.
Hiller of my staff at (412) 374-5953, or myself.
S. R. Tritch, Manager Engineering Technology Department RBH/sa Attachment ET-NRC-91-3627-12/13/91 - 6
J l
To properly inspect and test coils of each relay, it is necessary to remove the relay from it's installed position. The coil must be removed from the assembly to allow for proper inspection and testing.
DISASSEMBLY:
- CAUTION ***
Do not mix basic components from various relays. To maintain critical relay/contact overtravel and ensure integrity of relay operation, steps should be taken to reinstall relay bodies onto coils and bases as received.
Remove two (2) machine screws which mount relay body to metal relay base. Screws extend through body of relay coil into threaded base. After removal of these two screws, access is gained to kick-out spring/kick-out spring cavity and moving armature areas by lifting body off coil. The coil can then be lifted from stationary poles/metal base.
No further disassembly of component parts will be necessary.
INITIALCOIL INSPECTION:
- 1. Reject any coil which exhibits potting compound flow.
- 2. Reject any coil which has an obstruction or crack in the bore of coil spool.
- 3. Reject any coil which has a crack on any surface of outer case.
COIL POTI'ING CHECK:
- 1. Preheat an oven to a temperature between 250 and 275'F.
- 2. Position coils in oven upside down (with connection terminals and white spools facing downward) and spaced so that any flow from a coil will not contaminate an adjacent coil.
- 3. Maintain the coils/oven temperature for a period of two and a half to three hours.
- CAUTION ***
Use gloves as the coils are HOT.and must be handled for the inspection.
~g
~I
Remove the coils from the oven one at a time. Some cooling of the coils is expected but it is necessary to perform the inspection while coils are still hot.
Inspect the coils per the following:
- 1. Reject any coil which exhibits potting compound flow.
- 2. Reject any coil where potting compound exhibits a softening such that material separating can be made with a pointed or moderately sharp object such as a paper clip or pencil. Pay particular attention to kick-out spring cavity area at center of the coil.
Probe each of the four outer corners and center cavity area. Surface stickiness is acceptable if material separation does not occur.
- 3. Reject any coil which has an obstruction or crack in the bore of coil spool.
4 Reject any coil which has a crack on any surface of outer case.
It is recommended that all acceptable coils be marked with an identifier on the existing coil label or ~
outer surface.
RELAY ASSEMBLY:
Relay bodies, coils and bases are symmetrical Install relay coil over armature stationary poles down against metal base. Install kick-out spring into kick-out spring cavity. On installation, caution should be-taken to ensure large diameter end of kick-out spring is pressed into kick-out spring cavity. This is best accomplished by compressing spring from small diameter end with thumb or finger and pushing spring against bottom of cavity in coil.
- 2. Align holes in relay body with holes in relay coil and threaded holes in metal base. Insert two mounting screws down through holes in body and coil. Start threaded ends of two mounting screws into threaded holes of base. Tighten mounting screws to compress lockwashers only.
DO NOT OVER-TIGHTEN!
- 3. Manually manipulate relay plunger bar. Relay action should be quick/immediate. If action is acceptable, proceed. If action is sluggish or questionable, disassemble relay and inspect for proper location of respective components and presence of kick-out spring. If all parts are positioned as required, or after relocating spring, assemble relay as instructed above.
Prior to return to service, functional/operational tests of relays should be performed after relay assembly. Relays that are judged acceptable should be reinstalled following normal plant maintenance
.procedures. When reinstalling relays and connecting leads to contact cartridge terminals, contact screws should be torqued to no more than 10 inch pounds. DO NOT OVER-TIGHTEN!
The system should be thoroughly tested per plant maintenance procedures after all relays are installed.
4 To properly inspect and test the coils of each suspect relay, it is necessary to remove the relay from it' installation position. The coil must be removed from the assembly to allow for proper inspection.
DISASSEMBLY:
- CAUTION ***
Do not mix basic components from various relays. To maintain the critical overtravel characteristics of each relay, keep the major components together.
Remove the four machine screws securing the upper contact assembly to the lower magnet frame.
Note that two of the screws have lockwashers while the two screws through the terminal block do not.
The entire contact/coil assembly should now be free to lift up from the lower magnet assembly.
Remove the screws retaining the coil wires to the contact assembly. Note the relative location of the wires to allow re-assembly and then remove the wires from the contact assembly. The contact assembly should now be free to lift free of the coil. Keep the two terminal blocks with the upper contact assembly.
TEST AND INSPECTIO¹ Visually inspect all coils, particularly at the squared off base area and through the center bore. Reject any coil where the epoxy appears to have flowed. Small bubble type voids in the surface are not reason for rejection. Any coil where the bore is not smooth and clear of obstructions or that has a crack in the outer case should be rejected.
Preheat an oven to between 250 and 275'F. Place the coils in the hot oven. They should be positioned vertically with the base of the coil facing downward and spaced such that if the epoxy should flow from any coil, it will not contaminate adjacent coils. Maintain the coils in the oven at the temperature for two and a half to three hours.
- CAUTION ***
Use gloves as the coils are hot and must be handled during this inspection. Remove the coils from the oven one at a time. Some cooling of the coils during the inspection is allowed but it is desirable to perform the examination while the coils are still hot.
4
- 1. Reject any coil where the epoxy has flowed.
- 2. Reject any coil where the epoxy exhibits a softening such that a material separation can be made with a moderately sharp object such as a paper clip or pencil. Probe each of the four corners on the bottom of the coil.
NOTE: Coils where the epoxy surface is sticky or tacky are acceptable as long as the potting material does not separate.
- 3. Reject any coil where the center bore is swollen or obstructed. Cracks are not acceptable in the bore or on the outer surface of the case.
It is recommended that all acceptable coils be marked to identify those that pass this inspection.
ASSEMBLY:
Reassemble the coil on to the lower magnet frame by sliding the center of the coil over the stationary magnet pole. The coil wires should be laid along the inside of the magnet frame without being pinched. (Use another relay that has not been disassembled as a pattern.)
The contact assembly may now be slid down onto the lower magnet frame while guiding the magnet armature into the hole in the coil and the coil wires into the slots in the upper magnet plate.
The magnet plate should sit on the top of the lower frame. Reconnect the wires to the terminals that they are removed from. Align the terminal blocks on the upper magnet plate and start (do not tighten until later), the two machine screws that attach the terminal blocks to the upper plate. Be careful that the wires are not pinched in the slots through the upper magnet plate. Now start the two machine screws with lockwashers. These screws and the two in the terminal blocks should be tightened to 7.5 inch-pounds.
With the contact assembly reattached to the lower magnet assembly, manually manipulate the plunger assembly by depressing and releasing the actuation button on top of the contact assembly several times.
Any abnormal resistance, sluggishness, or non-snap return of the button on release should be considered evidence that the assembly is not correct.
Relays that are judged acceptable should be reinstalled following normal plant maintenance procedures. During the installation, the following should be considered.
It is important that on inserting the various wires at each normally closed (N.C.) contact, the wire should not be pushed in such a way that it touches the moving contact assembly or the individual contact assembly springs.
- 2. The contact terminal screws should be tightened to insure that the wires are secure. All terminals should have a screw installed even if a connection is not being made at the terminal to prevent possible loose parts from falling into other assemblies.
The system should be tested thoroughly after all relays have been reinstalled.
SIZE 00, 0, 1 AND 2 - COIL REMOVAL AND REPLACEMENT'irst turn off ali power. Refer to Figure 2,
- 2. Loosen. the appropriate wiring to allow'the'top section of the contactor to be removed.
- 3. Loosen the two screws (6) located between the two center poles of the Size 00, 0, or 1 contactor, or at the sides of the arc box of the Size 2 contactor, and lift off the'op section of the contactor assembly.
- 4. Remove the kickout springs (2) and the two screws located in the cavity (12) adjacent to the kickout springs and withdraw the molded coil structure (3).
- 5. Replace the coil and parts in the opposite order, making sure that the auxiliary contacts, if used, are properly seated, and that with the arc box (11) mounted, the crossbar (9) moves freely without. binding. when. manually. actuated. Take, special precaution when replacing the top section of. the contactor. The springs must be located in their spring seats properly to ensure correct contactor operation.
Springs or spring ends must not be distorted. Torque screws to 8-9 in-lb.
SIZE 3 AND 4 - COIL REMOVAL AND REPLACEMENT
- 1. First turn off all power. Refer to Figure 2.
- 2. Loosen the appropriate wiring to allow the top section of the contactor to be removed. =
- 3. Loosen the two. screws (6) located at the sides of the arc box (11) and lift off the top section of the contactor.
- 4. Remove the four screws (2) attaching the coil (3) to the base (1), and withdraw the molded coil.
Replace the coil and parts in the opposite order making sure that the auxiliary contacts, if used; are properly seated, and that with the.arc box (11) mounted, the crossbar (9) moves freely without binding when manually actuated. Take special precaution when replacing the top section of the contactor. The springs must be located in their spring seats properly to ensure correct contactor operation.
Springs or spring ends must not be distorted. Torque screws to 8-9 inAb. Tighten arc box- screws until- lockwashers compress.
I h
I il I'
Potting Coll Check Place coils in a temperature. controlled oven at 250'F for 2 1/2 hours. Coils should be',
oriented. bottom down (exposed epoxy down). Immediately after removal from oven, check for flowing. or, soft epoxy.. A wire probe (paper. clip) can be. used to check, for soft epoxy.
Allow. coils to cool before-reassembly.
I 0 C
OC ACTUATED CON ACTORS f.L. 13629D 10 SIZES 00, 0 8 AND 1 4
4 3 2 5
'2
'IZE 2 8 3 2
,"pc SIZES 3 AND 4 10 9
Fig. 2 OC Actuated Contactors (Exploded Views)