ML20151M127

Insp Rept 99900824/97-01 on 970616-19.Noncompliance Noted. Major Areas Inspected:Insp Summary,Status of Previous Insp Findings & Mfg Defects Associated w/safety-related Pressure Switches

ML20151M127
Person / Time
Issue date:	08/06/1997
From:	NRC (Affiliation Not Assigned)
To:
Shared Package
ML20151M117	List: ML20151M114 ML20151M121 ML20151M127
References
REF-QA-99900824 NUDOCS 9708080210
Download: ML20151M127 (13)

Text

.--. .. .- . . . . .. -. . . _

U.S. NUCLEAR REGULATORY COMMISSION OFFICE OF NUCLEAR REACTOR REGULATION I

i Report No: 99900824/97-01 Organization: SOR, Inc. (SOR) i Lenexa, Kansas f

j

Contact:

Colbert 0. Turney l

L Vice President, Quality Assurance 913/888-2630 l

Nuclear Industry Pressure, vacuum, and temperature switches l,

Activity:

Dates: June 16-19, 1997 L

j Inspector: Anil S. Gautam, Senior Engineer Approved by: Gregory C. Cwalina, Chief Vendor Inspection Section '

Special Inspection Branch ,

Division of Inspection and Support Programs i

i i

Enclosure 2 9700080210 970806 PDR GA999 ENVSORNC 99900824 PDR l l

1 INSPECTION

SUMMARY

I l

During this inspection, the NRC inspector assessed the adequacy of the actions taken by SOR to correct manufacturing defects associated with certain safety-related pressure, vacuum, and temperature switches (hereafter referred to as switches). The defects included (1) cracked insulation of lead wires for .

switches, (2) leakage of 0-ring seals in switches exposed to radiation and I elevated temperatures, and (3) leakage of epoxy seals in switches. The inspector assessed specific attributes of SOR's quality assurance program and reporting of defects under 10 CFR Part 21, and licensees' monitoring of S0R's control of quality.

l The inspection bases were as follows:  !

=

10 CFR Part 21, " Reporting of Defects and Noncompliance"

=

Appendix B, " Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants," to Part 50 of Title 10 of the Code of Federal Reaulations (10 CFR Part 50)

=

SOR Nuclear Quality Assurance Manual (QAM) 8303-100, Revision 9, dated April 14, 1993, and associated implementing procedures During this inspection, the inspector noted one instance in which S0R failed  :

to conform to NRC requirements imposed upon it by NRC licensees. This  !

nonconformance is discussed in Section 3.1 of this report. In addition, the inspector observed that during the January 1995 Nuclear Utilities Procurement <

Issues Committee (NUPIC) audit, licensees did not evaluate SOR's corrective l actions regarding certain manufacturing defects reported by SOR in information notices to customers. Licensees' monitoring of SOR's control of quality is  !

discussed in Section 3.2 of this report. I 2 STATUS OF PREVIOUS INSPECTION FINDINGS  !

Open Item 99900912/93-01-06 (Closed) I l

During a June 1993 NRC inspection of National Technical Systems (NTS), Inc.,

in Acton, Massachusetts, the inspector assessed qualification testing of SOR i pressure switches and observed (1) a pressure leak in one sample during a I high-energy-line break (HELB) test, and (2) excessive leakage current in another sample during a dielectric withstand test. The inspector found no  !

documented evaluation by NTS or SOR of the root cause of the test failure nor i pertinent corrective action. The inspector considered this an open item.

Following the NTS inspection, SOR gave the NRC documentation regarding the test failures. On the basis of the documents, the inspector determined that l

the pressure leak was due to a leakage path provided by unsealed mounting '

bracket screws for the microswitch (switching element) mounted in the switch housing. SOR believed that the screws had not been resealed after the microswitch was readjusted during factory calibration. Failure to reseal the screws allowed the switch diaphragm (seal) to be overpressurized during the test and caused it to leak. S0R stated that since other switches with the 2

same type of housings did not suffer a similar test failure, the test failure was attributable to a random occurrence, not to an inherent weakness in the design. SOR's corrective action consisted of (1) resealing the microswitch mounting screw threads if the microswitch was readjusted during factory calibration, and (2) applying a primer to the microswitch bracket screws to improve the curing of the thread sealant in stainless steel housings. SOR confirmed that applicable safety-related pressure switches installed in plants were not compromised because the corrective measures had been instituted before production. This issue is closed.

Regarding excessive leakage current in one sample: the inspector observed that the SOR pressure switch test specimens passed the dielectric withstand test at 1500 Vac for 1 minute, except for one sample that experiencm about 2 milliamps (mA) of leakage current at 900 Vac. SOR could not find a root cause and believed that the 2 mA leakage current was a random anomaly and not indicative of a common failure mode. SOR's basis was that the other specimens (1) passed the test at 1500 Vac, (2) had adequate insulation resistance at 500 Vde, and (3) had sufficient margin for service conditions because the switches were rated for 250 Vac and typically energized for 120 Vac or 125 Vdc applications. The inspector determined that on the basis of information provided by SOR, and because leakage current from moisture intrusion would have been higher than 2 mA, the anomaly was satisfactorily addressed. This issue is closed.

3 INSPECTION FINDINGS AND OTHER COMMENTS 3.1 Ouality Assurance Proaram

a. Insoection Scope The inspector examined the adequacy of S0R's Part 21 evaluations, corrective actions, conformance to procurement documents, and self- ,

assessment of performance. '

i

b. Observations and Findinas The inspector observed that SOR's QA program was based on the policies l and criteria of 10 CFR Part 50, Appendix B. The QA program staff was I comprised of the Quality Arsurance Vice President (QAVP) and 2 quality control (QC) inspectors. The QAVP reported directly to the SOR's President / CEO. The QC inspectors were authorized to stop production of a nonconforming item until the nonconforming conditions were corrected.

The inspector observed that SOR had posted sections of the Federal Reo/ ster, dated September 19, 1995, concerning the latest changes to l 10 CFR Part 21 but had not posted the complete Part 21 regulation, as is l required by 10 CFR 21.6. During the inspection, the QAVP posted copies l of the complete regulation in appropriate locations. No further concerns were identified.

3

The inspector assessed SOR's Part 21 reports and corrective actions for manufacturing years. defects associated with SOR switches during the past 5 Defects included cracked lead wire insulation, leaking 0-ring seals, and leaking epoxy seals in the switch conduit seal. The inspector's review is summarized below:

(1) Cracked Insulated Lead Wires Insulated laad wires for the switch enter and exit an epoxy seal in ,

the conduit adapter of the switch housing. The conduit adapter is potted (sealed) with epoxy to keep moisture from entering the switch housing.

i In September 1994, Nebraska Public Power District (for the Cooper Power Station) and Connecticut Yankee Atomic Power Company (for the Haddam Neck Plant) notified S0R of eight defective switches that had cracks in the insulated lead wires. SOR determined that the cracks had been caused by SOR's misapplication of the epoxy on the insulation, subsequent hardening of the insulation, and cracking and X

tearing of the insulatinn when it was bent.

' On October la, 1994, SOR sent a 10 CFR Part 21 report to the NRC and customers about the cracking of the lead wire insulation in SOR's neclear-qualified switches (the NRC also issued event notification 27902 on October 14, 1994, to inform licensees that the switches posed 4

a potential risk of failure of safety-related equipment). Subsequently, appraximately 11 licensees returned their switches to SOR for repair.

I l During this inspection, SOR provided a written response (Attachment 1) to the inspector, dated. June 19, 1997. SOR stated that it inadvertently ,

applied the epoxy on the insulation "due to poor workmanship" and that '

"the condition was undetected because SOR quality inspectors did not l notice the coating of the epoxy on the (insulated) wires." SOR also told the inspector that it had not prepared instructions to ensure that epoxy was not applied to the wire insulation. The inspector concluded that SOR's failure to prescribe instructions or procedures to ensure that epoxy was not applied on the insulated lead wires of the switches, or that quality inspectors examined the switches properly, as required by Criterion V, " Instructions, Procedures, and Drawings," of Appendix B to 10 CFR Part 50, constituted Nonconformance 99900824/97-01-01.

The inspector observed that, in 1994, SOR revised its work procedures I to preclude application of epoxy on insulated wires, and to reject any insulated wire that may have been covered with epoxy. SOR added shrink tubing to the insulated wires where they entered the conduit adapter epoxy seal to protect the wire insulation during shipping and handling. SOR also recommended not exceeding a minimum bend radius for the insulated wires. The inspector determined that 50R's actions to correct the misapplication of epoxy and prevent recurrence were adequate. No further response is required. '

4

, (2) Leakage of 0-Ring Seal The vacuum switch included an 0-ring installed in a triangular gland to seal the process air or fluid between the vacuum screw and the piston, and between the vacuum piston and the primary diaphragm. In 1992 and 1993, SOR implemented a new program to qualify the vacuum switches (SOR test report 9058-102), and discovered that the 0-ring seal in the vacuum switch was not capable of retaining the required maximum pressure after exposure to high radiation, high temperature, and hydrostatic pressure greater than 150 psi. SOR concluded that -,

leakage between the vacuum screw and the 0-ring had occurred during testing. '

' On April 1,1993, SOR issued "Information Notice Concerning Vacuum 0-Ring Seal in SOR Nuclear-Qualified Vacuum Switches" to applicable customers regarding potential leakage in vacuum switches, and

' suggested to customers that all switches be replaced if exposed to pressures greater than 150 psi. The affected switches were those

' designated by a 54N6, 54TA, 52N6, or S2TA in the first section of the model number and JJTTX6, JJTTX7, JJTTX13, or JJTTX14 at the end of the model number. i 4

i During this inspection, on June 19, 1997, SOR provided a written response (Attachment 2) to the inspector. SOR stated that it had not 4

discovered the condition described above earlier "because of inadequate engineering testing and analysis of the vacuum switch." On May 20, 1993, SOR took corrective action to eliminate the leak path by j

(1) welding the vacuum screw to the vacuum piston and (2) replacing )

the triangular 0-ring seal with a face seal of the same material. The face seal was qualified by analysis (SOR test report 9058-102, Section i 14, Appendix 4, Analysis 8923-219) to retain a pressure of 750 psi after exposure to radiation and elevated temperatures. SOR reported that it replaced applicable switches sold to licensees. No further concerns were identified.

(3) Leakage of Conduit Seals I

j The switch lead wires pass through the outer nipple of the conduit '

seal connector, through the epoxy seal potted in the nipple, and through a glass seal which is soldered inside the nipple. In May l

1994, during routine testing, SOR discovered leakage of pressure ~

through the conduit epoxy seal of "NQ" switches. On June 10, 1994, SOR issued "Information Notice Concerning Conduit Seals in SOR  !

Nuclear-Qualified Pressure, Vacuum, and Temperature Switches" to  !

" inform the NRC and customers about the potential leak. In the notice, i SOR stated that the leak could lead to reduced insulation resistance or loss of function of the switch during or after a HELB. SOR suggested to customers that all switches be returned to the SOR l factory for inspection if they were subject to HELB conditions during '

or after an event, or if subject to conditions in which condensate may form inside the conduit, or if subject to any other conditions in which moisture could penetrate the conduit seal. l 5

1 I

During the inspection, on June 19, 1997, SOR provided a written response (Attachment 3) to the inspector. SOR stated that the problem went undetected because (1) the leakage was a random problem, (2) l SOR's inspection steps were not adequate to identify the faulty condition seals, and (3) there was a manufacturing error in the heat j cure of the epoxy because manufacturing personnel had not followed procedures.

SOR also determined that the work format was not adequate because it did not require manufacturing personnel to record the actual heat cure temperature and the cure time for each batch.

In June 1994, S0R took measures to prevent recurrence by implementing I more stringent testing on all conduit seals, including requiring (1) an insulation resistance test for conduit seals, (2) a 100 psi leak test for the completed conduit seal assembly, (3) a housing leak test for the conduit seal after completing all assembly steps and all

thermal testing, and (4) test results to be approved by manufacturing '

and QA personnel for every order of switches. In addition, SOR took measures to record the cure temperature and time for the epoxy to ensure that the correct heat cure was used. This activity is required to be approved by manufacturing and QA personnel for every order of conduit seals. No further concerns were identified.

The inspector observed that S0R did not ask customers to identify any chemicals that the switch components would be exposed to during installation or operation to ensure that the switch was not compromised i in the performance of its function. Chemicals in the process (e.g., I ammonia) could degrade switch components (e.g., seals). SOR indicated that they assessed any process chemicals if identified by the licensee.

The inspector assessed SOR's implementation of licensee purchase order requirements in 50R's design documents (SOR assembly drawings 8520-264 Revision 2, 8520-506 Revision 1, and 8215-659 Revision 2).

No concerns were identified.

The inspector observed that SOR's General Instructions did not address the protection of switch components during handling (e.g., debris entering the switch housing, damage to lead wires) which could affect the operation of the microswitch. The QAVP added a cautionary statement to the General Instructions.

The inspector assessed SOR's internal audit report 7701-128, revision 4, dated December 30, 1996. The audit, in part, assessed the results of nonconformance reports and corrective actions. No concerns were identified.

c. Conclusions In general, SOR's QA manual and its implementation were in compliance with the requirements of Appendix B to 10 CFR Part 50, except for the nonconformance described herein. SOR took adequate corrective actions and steps to prevent recurrence of identified manufacturing defects.

6

l l

3.2 Review of Licensee Monitorina of SOR l

a. Insoection Scone The inspector evaluated licensee monitoring of SOR's control of quality )

for safety-related items purchased by licensees, including Part 21 reports and associated corrective actions.

b. Observations and Findinos l In January 1995, NUPIC - represented by Southern California Edison (SCE),

Baltimore Gas & Electric Company, and Yankee Atomic Electric Company -

audited SOR's QA program. Part of the scope of the audit was to verify whether SOR had established and effectively implemented a QA program in compliance with the requirements of 10 CFR Part 50, Appendix B and other industry standards. The NUPIC audit team identified SOR deficiencies in the areas of (1) control of purchased materials, (2) test control, (3) corrective actions, and (4) control of measuring and test equipment.

NUPIC considered the findings to be " administrative" and believed there was no adverse impact on the quality of SOR's completed products. NUPIC accepted SOR's corrective actions for the above findings and closed the findings on April 11, 1995. NUPIC concluded that 50R's QA program was adequate and that implementation was satisfactory.

The inspector observed that during its audit, NUPIC reviewed S0R's 10 CFR Part 21 report, dated October 14, 1994, regarding cracking of the insulation of switch lead wires. NUPIC verified S0R's corrective actions by observing in-process assembly of pertinent switches and associated documentation. The inspector noted that during the NUPIC audit, licensees did not evaluate S0R's 1993-1994 corrective actions regarding leakage of 0-ring seals in switches exposed to radiation and elevated temperatures, and leakage of epoxy seals in switches. These manufacturing defects were reported by 50R in information notices to customers (see section 3.lb of this report). After a telephone discussion with SCE's procurement quality staff, the inspector confirmed that NUPIC had not included these issues in the scope of the audit.

In February 1997, NUPIC, represented by Omaha Public Power District, examined the application of SOR's QA program to all phases of the design and manufacture of SOR switches. NUPIC noted that S0R had issued a 10 CFR 21 report in 1993 (no details are noted in the NUPIC report).

NUPIC recommended, in part, that SOR should clearly document its methods of verification of critical characteristics, and develop a checklist of specific inspection criteria for items purchased. No findings were identified,

c. Conclusions In general, licensee monitoring of SOR's quality was in accordance with proper criteria, procedures, and checklists. NUPIC did not evaluate SOR's corrective action for two manufacturing defects reported by SOR to its customers.

7

3.3 Entrance and Exit Meetinas At the entrance meeting on June 16, 1997, the NRC inspector discussed the scope of the inspection, outlined the areas to be inspected, and established interactions with SOR management. In the exit meeting on June 19, 1997, the inspector discussed his finding and observations.

4 PARTIAL LIST OF PERSONNEL CONTACTED 103 Lew Goetz, President and CEO Colbert Turney, Vice President (VP), Quality Lind Coutts, Coordinator, Nuclear Engineering Joseph Modig, Engineer, Nuclear Engineering Landen Tuggle, Director, Manufacturing Harold Moddy, VP Sales Charisse Smith, VP Finance Tim Ceillesen, Product Manager Richard Johnson, QC Engineer Southern California Edison Jeff Larson, Supervisor, Procurement. Quality ITEMS OPENED, CLOSED, AND DISCUSSED Ooened 99900824/97-01-01 Para 3.1 b NON inadequate instructions and procedures Closed 9990091)/93-01-06 Para 3.10 of Open Item inadequate information inspection report regarding test anomalies 99900912/93-01 8

1 NEB E "--"

14685 W.105th Street

• Lenera. Kansas 66215 5964 USA Tel 913 888 2630

• Fcx 913 888 0767

,,,,,s , , ,, o Nir. Anil S. Gautam. NRC

Subject:

10CFR21 dated October 14,1994

Dear hir. Gautam:

In July of 1993 SOR began to manufacture pressure, vacuum. and temperature switches that were qualified by SOR test report 9058-102. In September of 1994 SOR was notified of a manufacturing defect by Nebraska Public Power (Ref. RGA 2125, seven defective units) and Connecticut Yankee (Ref. RGA 2117. one defective unit). In addition SOR assembly personnel had identined the same defect (Ref. h1RR 1479. one defective unit). The defect was identified as a crack in the lead wire insulmion. This prompted SOR to issue a 10CFR21 and investigate the cause of the defect. The cause of the cracked insulation was a heavy coating of epoxy on the wires outside of the potted i area and was due to poor workmanship. This condition went undetected because SOR quality inspectors did not notice the coating of epoxy on the wires.  !

As noted in the Part 21 Notification, the following corrective action was taken in October 1994:

i

1. The Work Order formats for the conduit seals were revised to include specific i instructions not to allow epoxy on the wires. In addition, there is an inspection step at

{

the end of the Work Order that instructs the inspector to examine the wire and reject l any that have epoxy on the wire. Each of these steps must be signed off on the Work Order by hianufacturing and QA personnel for each order of conduit seals.

2. Shrink tubing was added to the lead wires where they exit the conduit seal. The j

purpose of this tubing is to protect the wire insulation during shipping and handling.

This step is signed off on the assembly procedure by hianufacturing and it is reviewed by QA penonnel for eve y switch.

3. The wire manufacturers recommended minimum bend radius was added to the SOR General Instructions that are provided to the customer with each switch.

The conduit seals are manufactured as a sub-assembly in a separate environmentally controlled room. Therefore, there is no danger of epoxy contamination on any other parts of the switches. .

Reference Corrective Action Report 0357.

Regards, 0

cafa%

Colbert Turney,y Q." 547 V.P. Quality Joseph G. hiodig, Engineer

yggg MDGtGT2 WWWm.

14685 W.105th Street

• Lenexa. Kansas 66215-5964 USA Tel. 913 888 2630

• Fax 913 888 0767

,, ,,,,,,g Mr. Anil S. Gautam, NRC

Subject:

Information Notice of April 1.1993

Dear Mr. Gautam:

Prior to April 1,1993 SOR had been manufacturing nuclear qualified vacuum switches for approximately 10 years. These switches were qualified by a combination of testing and analysis as listed below:

AETC Test Report 17344-82N-D. Rev.1 AETC Test Report 18441-83N, Rev.1 AETC Test Report 17344-82N-C. Rev. 3 e AETC Test Report 18577-83N. Rev.1 AETC Test Report 18878-84N-2 Rev. 2 e SOR Analysis 8215-959 in 1992 and 1993 SOR underwent a new qualification program (SOR Test Report 9058 102) and discovered that the o-ring gland design on the vacuum piston was not capable of retaining maximum operating pressure after exposure to radiation, aging, and cycling.

SOR informed the NRC and the utilities of thic condition on April 1,1993. The qualification test specimens were left in the test program with no modifications and continued to function properly and passed all tests with the exception of the hydrostatic test at the conclusion of the HELB and LOCA. SOR redesigned this seal to meet hydrostatic requirements and qualified it by analysis (Ref. SOR Test Report 9058-102.

Section 14. Appendix 4, Analysis 8923-219).

This condition v as not discovered earlier because ofinadequate engineering testing and analysis of the vacuum switch.

As noted in analysis 8923-219, examination of the test specimens revealed that the o-ring was still sealing between the vacuum piston and the diaphragm, but leakage was occurring between the vacuum screw and the o-ring. This is attributed to the triangular gland design and a combination of compression set, volumetric swell, and shrinkage which occurs from exposure to elevated temperatures and irradiation. All of these factors contributed to the loss of the line of contact between the vacuum screw and the o-ring, and the resultant leakage at high hydrostatic pressures.

The redesign, which was released by an Engineering Order on May 20.1993, eliminates the leak path mentioned above because the vacuum piston is now welded to the vacuum

screw. In addition, the triangular o-ring gland was changed to a face seal configuration.

A face seal is utilized on the pressure port o-ring of the vacuum switch and has successfully retained 750 PSI hydrostatic pressure after exposure to radiation and thermal aging. The pressure port o-ring and the vacuum screw o-ring are made of the exact same material (Parker compound E740 for option "M9"; Parker compound V709 for option "M4") and differ only in size. The o-ring gland dimensions are in accordance with the Parker 0-Ring llandbook for a static face seal gland.

Regards, coes/a.a- gbb&

Colbert Tumey \(P. Quality  ; '7 Joseph G. Modig Engir.eer F

1 l

14685 W 105th Street

• Lenera Kansas 66215-5964 USA Tel 913 888 2630 + Fax 913 688 0767

,,o , g , , , , r3 Mr. Anil S. Gautam. NRC 1

Subject:

Information Notice of June 10.1994

Dear Mr. Gautam:

In July of 1993 SOR began to manufacture pressure, vacuum, and temperature switches that were quali6ed by SOR test report 9058-102. In May of 1994 SOR discovered a potential leakage problem in the conduit seals of these switches during routine testing and  !

reported this discovery to the NRC and the affected utilities on the subject Information i Notice. This problem went undetected by SOR for three reasons:

1. The leakage was a randon problem. I

2. Inspection steps were not adequate to identify faulty conduit seals.

3. There was a manufacturing error in the heat cure of the epoxy. This error was due to manufacturing personnel not following procedures. In addition the Work Order format was not adequate because it did not require manufacturing personnel to record 1

the actual heat cure temperature and cure time for each batch. '

As noted in the Information Notice, more stringent testing was instituted immediately (June.1994). This included the fbilowing steps:

1. An insulation resistance test was added to the Work Order format for conduit seals.

This step is signed off by the manufacturing personnel and reviewed and signed off by QA personnel for every order of switches.

2. The insulation resistance test procedure was changed to include testing of wire to wire (all combinations) in addition to the standard wires to case test.

3. A 100 PSI leak test was added to the Work Order fbrmat for the completed conduit seal assembly. This step is signed oft by the manufacturing personnel and ieviewed and signed off by QA personnel for every order ofconduit seals.

4. A housing leak test was added to the assembly procedures in order to test the conduit seal after all assembly steps and all thermal testing is complete. The test pressure is equivalent to the llELB or LOCA pressure as applicable. This step is signed off on the assembly procedure by the manufacturing personnel and reviewed and signed off by QA personnel for every order of switches.

In addition to the above steps, the following corrective action was taken in August 1994:

1

1. The Work Order formats for the conduit seals were changed to requ to record the cure temperature and time for the epoxy. This will insure that the correct heat cure is used. This step is signed off by the manufacturing perscit:v. a reviewed and signed off by QA personnel for every order of conduit seals.

Reference Corrective Action Report 0338.

Regards.

edif5-y Colbert Turney. V.P. Quality

• 1 A(3.bO< /9[97 Joseph G. Modig. Engineer l

.