ML20154Q872

Insp Rept 99900728/98-01 on 980908-1002.Nonconformance Noted.Major Areas Inspected:Selected Portions of Valcor QA Program & Implementation Re Supply of Solenoid Operated Valves to Nuclear Industry

ML20154Q872
Person / Time
Issue date:	10/21/1998
From:	NRC (Affiliation Not Assigned)
To:
Shared Package
ML20154Q780	List: ML20154Q776 ML20154Q858 ML20154Q872
References
REF-QA-99900728 NUDOCS 9810260097
Download: ML20154Q872 (14)

Text

..

1 U.S. NUCLEAR REGULATORY COMMISSION OFFICE OF NUCLEAR REACTOR REGULATION i

Report No: 99900728/98-01 Organization: Valcor Engineering Corporation i

Contact:

Jim Shieh, Quality Assurance Director (973) 467-8400 l F.'aclear Activity; Manufacturer and supplier of flow control devices, including '

safety-related solenoid operated valves and replacement parts used in nuclear applications. ,

i 1

Dates: September 8-10 and October 2,1998 Inspectors: Gregory C. Cwalina, Senior Operations Engineer l Joseph J. Petrosino, Quality Assurance Specialist i Harold L. Ornstein, Senior Reactor Systems Engineer Approved by: ' Robert A, Gramm, Chief - ,

Quality Assurance and Safety Assessment Section '

Quality Assurance, Vendor inspection and Maintenance Branch Division of Reactor Controls and Human Factors Enclosure 2 9810260097 981021 00 8 phE

y _ _ _ _._ _ . - _ . _ . _ . _ - . _ . _ - ._ -.- _ .. . _ _ . . _ _ . . _ _ _

?. .

[>

?- 'l, 1 . INSPECTION

SUMMARY

l l

On September 8-10 and October 2,1998, the Nuclear Regulatory Commission (NRC) performed an inspection of the Valcor Engineering Corporation (Valcor). The inspection  ;

reviewed selected portions of the Valcor quality assurance program, and its  !

implementation, as it relates to the supply of solenoid operated valves to the nuclear industry. Specifically, the inspection reviewed activities related to Valcor's review and ]

E analysis of reported valve failures at the Pennsylvania Power and Light Company's  ;

l (PP&L) Susquehanna Steam Electric Station (SSES), and Valcor's evaluation ,

performed in accordance with 10 CFR Part 21, including the susceptibility of similarly l designed valves. ]

y 1 The inspection bases were:

. 10 CFR Part 50, Appendix B, " Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants."

!  := 10 CFR Part 21, " Reporting of Defects and Noncompliance."

l' During this inspection, one nonconformance was identified and is discussed in Section l 3.6 of this report.

2 STATUS OF PREVIOUS INSPECTION FINDINGS l l

Violation 99900728/91-01-01(Closed)

During a February 1991 inspection of Valcor, the inspectors found that Valcor failed to adopt appropriate procedures to provide for evaluating deviations or informing licensees or purchasers of the deviations.

l During this inspection, the inspectors reviewed Valcor's Part 21 implementing I

procedures (see Section 3.3) and found them to meet the requirements of 10 CFR Part ,

21 with regard to customer notificatiorw. I l

Violation 99900728/91-01-02 (Closed)

During a February 1991 inspection of Valcor, the inspectors found that Valcor failed to

' post copies of Section 206 of the Energy Reorganization Act of 1974 and failed to post ]

l its 10 CFR Part 21 procedure in a conspicuous location. H During this inspection, the inspectors observed that the documents which Valcor had displayed were conspicuously located and met the requirements of 921.6, " Posting y requirements," of 10 CFR Part 21.

t b

I

! 2 I~

l4 _ _ . _ ..

3 INSPECTION FINDINGS AND OTHER COMMENTS 3.1 Backaround On September 20,1997, the Unit 2 Reactor Recirculation Pump Cooling Water inboard containment isolation valve failed its stroke time surveillance at the Pennsylvania Power and Light Company's (PP&L) Susquehanna Steam Electric Station (SSES) due to a reported failure of the associated solenoid operated valve (SOV). PP&L replaced the SOV, Valcor model V70900-65-11, and successfully stroked the valve. PP&L also initiated a Condition Report (CR-3173) to investigate the cause of the apparent failure of the Valcor SOV (serial number (S/N) 43).

Valcor SOV S/N 43 had been in operation and continuously energized since April 1997.

Following removal from service, PP&L was unable to duplicate the apparent failure.

Therefore, the SOV was returned to Valcor for their examination. Valcor reported that all internal components were in satisfactory condition and within tolerances. Stroke testing was performed successfully, and the stroke and gap of the valve were dimensionally within their acceptable range.

PP&L investigated several proposed causal factors, including misinstallation, misalignment and the possibility of internal binding due to manufacturing defects or the presence of adhesive particles from installation. The PP&L root cause analysis was reviewed by an independent testing laboratory, Altran Corporation, who also performed laboratory investigations and testing of the Valcor SOV. On March 20,1998, PP&L and Altran representatives met with Valcor to present the results of their root cause analysis.

Following the meeting with PP&L and Altran, on March 25,1998, Valcor issued Nuclear Field Report (NFR) #017. This began Valcor's evaluation in accordance with 10 CFR Part 21 requirements. Valcor issued a Part 21 notification to the NRC on May 19,1998 (see Section 3.4).

Altran issued Revision 0 of Technical Report No. 98115-TR-01, "RBCW Containment Isolation Valve HV-28792A2 Stroke Time Surveillance Failure Root Cause

!nvestigation," in April 1998. Revision 0 concluded that the most probable root cause for the failure was, " residual magnetism which existed in the Valcor Pilot SOV S/N 43 following de-energization."

The PP&L final root caus3 analysis identified several design deficiencies associated with j the valves, as well as some internal PP&L issues. PP&L,in conjunction with Altran, concluded that the root cause of the valve failure was, " Defective Design with High inherent Residual Magnetism, Unknown to the Vendor.. " Four contributing factors were also identified:

l 1- Inadequate control of the gap between the plunger and backstop 2- Less than adequate qualification testing of a new design l

l 3 l

l

l 3- Normalin-service creep deformation (of the backstop o-ring) 4- Particulate debris in the small clearance areas of the valve Based upon their evaluation and the independent root cause analysis performed by I

Altran, PP&L concluded' that all 48 Valcor solenoids, Models V70900-65-11 and 65-12, l were defective due to a loss of o-ring resiliency which was necessary to overcome the l effects of residual magnetism. As a result, Valcor, in consultation with PP&L, made the f following changes to the design and manufacture of the 65-11 valve: a residual washer was added to prevent the plunger from coming in direct contact with the stop (Model 65-11A) and, in order to meet SSES operational needs (i.e., oilin the air system which could attack the EPDM o-rings), Valcor changed the o-ring material to Viton (Model 65-118)in some of the valves.

SSES experienced further problems with Valcor valves in the May/ June time frame.

First, on May 23,1998, a model 65-11A failed to stroke when energized. An l examination of the valve performed by Altran identified some foreign substances within  ;

the valve. However, Altran concluded that the failure was most likely caused by the i residual washer "providing wedging action resistance between the coils of the spring and the backstop." Later, on June 3,1998, SSES reported that 5 of 20 model 65-11B valves had faile.1 bench testing. In this case one valve did not stroke upon demand, one exhibited unar.ceptable leakage, one showed a sluggish response and two exceeded the required pick-up voltage. PP&L again concluded that some of the failures were attributable to the residual washer interference.

Although Valcor testing could not duplicate the failures, Valcor widened the inside diameter of the residual washer to allow more clearance with the spring (Model 65-11C).

Then, on September 15,1998, during bench testing at SSES,5 of 8 spare model 65-11C valves failed to stroke upon demand. On September 18,1998,4 of the 5 valves again failed testing. The latter test was witnessed by Valcor, Altran and NRC employees. All 8 valves were sent to Altran for root cause analysis (see Section 3.5).

3.2 Valve Desion and Manufacturina

a. Scope The inspectors reviewed design and manufacturing documents associated with the Model V70900-65 valves to identify potential design and manufacturing deficiencies and whether other models within the series may be subject to the same operational failures as the SSES valves Documents reviewed included design drawings and qua:ification and test reports.

' During the investigation of this issue, several more interrelated condition reports were issued by PP&L. This inspection report will deal with the overall conclusions developed from all of the PP&L Condition Reports.

l l

4

.. . - . - . - . - . . . . . . - - - . .- . .~

l l.

l

b. - Observali.Qas and Findinas The Model 70900-65 is a low weight, 3-way, direct acting, balanced poppet, nuclear-qualified solenoid operated valve (see Figure 1). According to Valcor product literature, it can be operated in a normally open or normally closed configuration, in any attitude.

Each valve is hand assembled, adjusted, and tested prior to final shipment.

yrin'u nin- Backstop

( /

'//////fv ,M<- _; M NM; '

Y Plunger / Backstop Air Gap l --[ -

(location of g#jfr' --

N

/4 // m M: ,

residual washer) l X T

l DE-ENERGIZED N \ N

! \ '

CONDITION  : R l\  :,- Plunger Spring /:;,

N \ \

/ 1 ht-yK, j- P.

, _ in

& N

~ '

Backseat /

O-Ring w "

C Pon BPori ,

i

/ Poppet A Port /

MODEL V70900-65-11 SERIES l Figure 1 5

L

.-. . ~- - - - - . . -- . . - . - - - -

The inspectors reviewed valve assembly procedure AP70900-65-4, " Assembly Procedure Solenoid Valve, 3-Way Valcor P/N V70900-65-4," which was used to assemble the valves. This procedure was originally developed for the V70900-65-4 valves, which were the first of this model series to go into production. All subsequent valves in the series were constructed using this procedure. However, since the 65-11 is an AC powered valve, the assembly procedure was rmt completely applicable. Valcor augmented the procedure with Engineering Order (EO) 4113 to produce the model 65-  ;

11 valves. Subsequently, Revision D (January 9,1997) incorporated the assembly steps from EO 4113.

The assembly procedure required taking measurements with the valve in both the energized and de-energized positions. The assembler then calculated the number of l shims to be added to achieve the correct amount of poppet travel. Further measurements were taken to determine optimum plunger length. The plunger was then machined and the valve assembled and tested. The inspectors noted that Valcor did not consider the gap dimension as critical to valve operation, believing the assembly procedure resulted in a sufficient air gap to prevent contact between the plunger and backstop. Valcor did not recognize that the assembly procedure could allow the gap to shrink to zero, based upon dimensional tolerances and o-ring compression. It was Valcor's position that the post assembly testing assured a proper dimensional fit.

Otherwise, improper operation or leakage would have been detected.

Followino the problems identified at SSES, in April 1998 Valcor created procedure AP70900 65-11 A-1,' " Assembly Procedure Solenoid Valve,3-Way Valcor P/N V70900- i 65-11 A and P/N V70900-65-11B", for the assembly of the AC valves (initially for the model 65-11 A and subsequently revised to address the model 65-11B). This procedure was developed in consultation with PP&L and addressed the potential tolerance errors identified in the previous procedure. The inspectors noted that the model 65-11C is not addressed. Valcor explained that since the 65-11C valves were retrofitted from existing model 65-118 valves, and not built from scratch, a new assembly procedure was not necessary. Valcor stated that the procedure would be revised if model 65-11C valves are purchased in the future.

l The inspectors noted that there are sevc al valves within the V70900-65 series. The Model V70900-65-11 series is the only model that operates on AC voltage. All other models (65-4,5,6,10,12,16 and 17 have been produced) operate on DC voltages.

Although Valcor is not convinced that the DC models are susceptible to the residual magnetism problem, Valcor prudently decided to include the residual washer on all subsequent valves in the model 65 series. Procedure AP70900-65-4 was revised in May 1998 to delete it's applicability to the AC valves and identify all DC v.alves that are applicable. The procedure title was also modified to, " Assembly Procedure Solenoid Valve, 3-Way Valcor V70900-65-XX Series (D.C. Powered)."

Following the September 1998 bench testing failures and subsequent root cause analysis, the inspectors again reviewed Valcor's valve assembly procedures. A concern regarding the control of a dry film lubricant which is applied to the valve plunger and sometimes to the plunger tube was identified (see Section 3.6).

6 l

c. Conclusions The inspectors found that Valcor did not treat the air gap as a critical dimension in the

( original assembly procedures. Based upon a review of documents and discussion with the valve design and manufacturing personnel, the inspectors concluded that Valcor's basis for assuming the assembly and testing procedure provided an adequate air gap was reasonable at the time. However, Valcor did not recognize that aging of the o-ring )

. in the energized state could reduce its resiliency and increase its compression, thus potentially affecting valve operation. Following the PP&L and Altran investigation, Valcor revised the procedure to assure the existence of a positive air gap.

3.3 Review of Valcor's 10 CFR Part 21 Proaram and its Imolementation -l

a. Insoection Scooe l

The NRC inspectors reviewed the program that Valcor established to implement the provisions of 10 CFR Part 21, and verified selected aspects of the Part 21 program implementation.

b. Observations and Findinas 10 CFR Part 21 Procedure The inspectors reviewed Valcor Procedure S2110 "10CFR21 Defects and Non-Compliance Reporting Procedure," Revision D, dated February 7,1997. The practice for the evaluation of deviations and failures to comply and the reporting of defects within the required time frames were found to be generally i acceptable in S2110. The inspectors noted that procedure S2110 also required customer problems to be identified on Valcor " field reports," and dispositioned in accordance with S2110. The responsibility for documenting the results of deviation i evaluations were found to be included in procedure S2110. In addition, S2110 contained the necessary documentation forms, i.e., Form A, Nuclear Field Reports (NFRs); Form B, Field Problem Data Sheets; and Form C,10 CFR Part 21 Reporting Evaluation Forms (10CFR21 Evaluation Form).

However, the inspectors noted some minor changes that were needed in procedure S2110 to remove ambiguities and provide additional specificity. For example, although

" Form C" was provided for the documentation of the results of an evaluation, an explanation of the evaluation process2 was not established in S2110. It was noted that the S2110 procedure used the Part 21 term, " defect" as defined in $21.3 of 10 CFR Part 21, in some sections of S2110 instead of the correct Part 21 term of " deviation."

Additionally, the inspectors noted that Valcor did not define the meaning of several 10 CFR Part 21 words which have a specific meaning that are unique to 10 CFR Part 21.

The weaknesses were identified and discussed with the Valcor Quality Assurance e 2 10 CFR Part 21, $21.3, " Definitions," states that " evaluation" means the process of

, determining whether a particular deviation could create a substantial hazard or determining

[ .whether a failure to comply is associated with a substantial safety hazard.

i 7 l . - .

Director, and the QA Director stated that appropriate revisions would be incorporated in S2110 within 90 days after the completion of the exit meeting.

Program Imolementation The inspectors reviewed records related to the Part 21 program implementation including: deviation / failure to comply evaluation packages, completed NFRs, Field Problem Data Sheets; and 10CFR21 Evaluation Forms. The completed NFRs dating back to 1990 indicated that Valcor was reviewing nuclear customer component problems to determine whether any issues were potentially I

generic, or potential deviations or failures to comply that would need to be evaluated in accordance with Valcor's Part 21 program. All of the NFRs reviewed reflected an adequate and reasonable disposition of the matter except for NFR #017, "V70900-65-11 Valve Reportably Failed to Drop-Out when De-energized," dated March 25,1998.

A review of NFR #017 and its associated evaluation package, including Valcor's 10CFR21 Evaluation Form, indicated that Valcor did not perform an evaluation as defined in 921.3 of 10 CFR Part 21. That is, although @21.3 of Part 21 requires an evaluation to be performed, Valcor only reviewed the issue to determine whether it applied to other products that it produced. However, since Valcor discussed the matter with the only domestic customer that used that particular series of SOVs, the intent of

$21.21(b) was addressed. Therefore, no violation was cited in this area. (See further discussion in Section 3.4)

c. Conclusions The inspectors determined that Valcor's S2110 procedure was generally well written and addressed the salient aspects of 10 CFR Part 21. However, some weaknesses involving a lack of specificity may have caused an inadequate evaluation to be performed for NFR #017.

3.4 Part 21 Evaluation of SSES Valves

a. B999g The inspectors reviewed Valcor's Part 21 report and evaluation relating to the failures of the SSES valves to assure that their activities met applicable regulatory requirements.

In addition, the inspectors examined whether Valcor properly considered the failure j mode's impact on other similarly designed valves.

b. Observations and Findinos l Following the initial PP&L problem in September 1997, Valcor was not convinced of the l potential for residual magnetism. This was based upon their inability to duplicate the failure as documented in SKA 17979, " Failure Analysis of Valcor Valve V70900-65-11."

On March 19,1998, PP&L and Altran met with Valcor to present the results of the root cause analysis. During that meeting, PP&L was able to demonstrate the possibility of residual magnetism to Valcor. Subsequently, on March 25,1998, Valcor initiated NFR 1

l 8 i

~

i

1. 017, which documented the potential residual magnetism problem and started their l Part 21 evaluation process. NFR #017 stated that Valcor would review the design to verify suitability and, if necessary, modify the design to increase the valve's margin to close. (The design modification is discussed in Section 3.2, above.)

On May 19,1998, Valcor determined the problem was reportable and issued a Part 21 report to the NRC on the same day. The Part 21 report stated that, "Following 6 to 18  !

months of continuously energized service,3 units of Valcor Model V70900-65-11 air I pilot valves have reportedly failed to stroke closed immediately upcn de- ,

energization...Despite best efforts, the delays in closing have not been able to be replicated outside of the plant system. Delays in closing have ranged from 1 to 5 minutes (emphasis added)." The report went on to state, "The V70900-65-11 is an AC-powered version and is the only version (emphasis added) of the V70900-65 air pilot series subject to this effect."

The inspectors reviewed Valcor's Part 21 report and associated documentation. As stated above, Valcor was not able to duplicate the initial failure and did not consider j residual magnetism as a viable failure mode until the March 19,1998 meeting. At that l point, Valcor initiated their Part 21 process. The report to the NRC was made within the Part 21 timeliness guidelines.

The inspectors examined the basis for Valcor's statement that the closing delays ranged from one to five minutes. Valcor stated that the report was based upon failures later ,

reported by PP&L, not the September 1997 failure (which Valcor still questions). Valcor l informed the inspectors that the time delays were based on verbal information supplied by PP&L. The inspectors were not able to identify any documentation at Valcor to support this statement. However, a review of PP&L condition reports related to the later failures supports the times reported by yalcor.

The inspectors also reviewed the basis for Valcor's determination that only the AC valves were affected by the residual magnetism issue. Valcor explained that the AC and DC valves are essentially identical in construction with the exception of the solenoid coil (verified by the inspectors). Further, the model 65-11 utilizes a rectifier to convert the AC input voltage to DC at the coil. This reduces the effective solenoid strength.

Since the solenoid strength is reduced, AC valve construction typically results in a lorer plunger to meet the valve's operating requirements. Valcor believed at the time tha, 3 longer plunger resulted in a smaller air gap in the AC valves. In addition, as mentioned above, the assembly procedure for the AC valve was slightly different, using different construction steps and different dimensions. 9erefore, Valcor concluded that the DC valves would typically contain shorter tubes,12..ger air gaps and would not be susceptible to the effects of residual magnetism.

The Duke Power Company (Duke) had bought a number of model 65-16 (DC) valves from Valcor through R. E. Hiller. Upon hearing of the Part 21 report, Duke contacted Valcor to determine if the problem existed in their valves. Valcor informed Duke that they believed the problem was restricted to the AC valves, but offered to provide the valve modification (residual washer and verify air gap). Valcor personnel modified the 9

l l

l l

valves at Duke's McGuire facility. During the modifications, Valcor measured the air l gaps and found that some negative air gaps (contact between plunger and backstop) l existed. Valcor performed a Part 21 evaluation, " Engineering Report Evaluation for Model Number V70900-65-4 Series D.C. Solenoid Valves," and concluded that the i potential for residual magnetism did not constitute a significant safety hazard and was '

not reportable under Part 21. However, as a precaution, Valcor modified the assembly l procedure to incorporate the residual washer and assure a positive air gap in all DC valves. At the time of the evaluation, the only DC valves affected were the model 65-12 valves sent to SSES and the model 65-16 valves sent to Duke. Valcor did not formally l inform either customer for the following reasons: PP&L had informed Valcor that this failure mode would not result in a safety problem at SSES due to their particular  :

application for the model 65-12 valves and PP&L was already aware of the potential for l residual magnetism; the Duke valves had already been modified. Valcor incorporated the assembly change to their model 65-17 valves, which were in production.

The inspectors reviewed Valcor's Part 21 evaluation and discussed the issue with responsible staff. The inspectors determined that Valcor still believes the original design of the valve, particularly considering the strength of the return spring, sufficient to overcome the effects of residual magnetism, despite the strong argument provided in l the Altran report . The inspectors also determir ed that Valcor took a conservative approach and modified the valve design and assembly procedure to negate the potential

'o a residual magnetism problem. In addition, all affected parties were aware of the potential problem. Further, valves under construction were manufactured using the new design and assembly procedure. Although Valcor's acceptance of the concem for residual magnetism may have been nonconservative, their actions taken to preclude future occurrences were conservative and met the intent of NRC regulations.

Subsequently, on September 15,1998, five of eight spare model 65-11C valves failed bench testing at SSES. The bench test was repeated on September 18,1998 and witnessed by NRC inspectors, Valcor representatives and an Altran representative.

These failures are discussed below.

c. Conclusions The inspectors concluded that, based upon the information available at the time, Valcor's determination that the DC valves were not affected by residual magnetism was reasonable. In addition, when Valcor became aware of the potential for residual magnetism in the DC valves, they performed a Part 21 evaluation and modified the valve design and assembly procedures.

Altran's investigation included an analysis of the spring capacity and the stored compression energy and resilience of the backseat o-ring.

l l 10

L ,

! ' 3.5 - Observations of Testina at Susauehanna and Altran Corooration

~a. Scope I- -

l The NRC staff inspectors witnessed subsequent testing of several Valcor model I'

.V70900-65 series SOVs which failed to stroke at the SSES facility during a bench test  ;

on September 15,1998. The' subsequent testing was conducted on September 18,

~ 1998, at the SSES facility and on September 23-25,1998, at the Altran Corporation for diagnostic / root cause analysis.

I

- b. Observations and Findinas ~

I Backaround in July 1998, Valcor modified eight V70900-65-11 SOVs to address ,

- operational concerns that were identified at SSES by PP&L personnel. The modification L included adding Viton O-Rings, changing the residual magnetism washer with one which ,

i had a larger inside diameter, and regapping the valves to achieve better operation  !

(Model 65-11C). A review of an SSES Source Verification Report,98-026, regarding  ;

j .Valcor assembly / modification of four of these SOVs indicated that the Vendor Quality inspector requested that an "as received test be performed at SSES to determine the acceptability of the valves prior to being placed into stock." However, instead of functional testing upon receipt at SSES, the SOVs were placed in a QC-hold status until they were functionally tested on September 15,1998.

Testina On September 15,1998, SSES personnel tested all eight Valcor Model 65-11C  :

~

SOVs. Five of the eight valves failed to open, or stroke regardless of the various applied pressure and voltage. SSES personnel contacted the PP&L valve engineering l

personnel and a second test was scheduled for September 18,1998. The PP&L valve 1 engineering personnel contacted a testing laboratory, Altran Corporation, to perform additional SOV testing in case the SOVs continued to fail to stroke.. The valve engineerr orsonnel also' contacted Valcor for their assistance in developing test I

methodok , identifying possible failure mechanisms, and to witness the September 18, l 1998, testing at SSES and any subsequent testing at Altran. NRC personnel also witnessed the testing.

On September 18,1998, the five SOVs that previously failed were tested again. Four of the five again failed to stroke when energized, and the fifth did stroke (opened) but it did not meet the pull-in voltage acceptance criteria. Manual force was applied to one of the SOV valve poppet assemblies to determine whether it would stroke, but the plunger j assembly could not be moved. After the testing was completed, Valcor, Altran and j PP&L personnel discussed the results of the tests and planned diagnostic and root l cause actions that would follow at the Altran facility.

L Starting on September 22,1998, testing was performed at the Altran facility using an Altran Test Plan that was developed using input from Valcor and PP&L. The four SOVs l which failed to stroke on September 18,1998, again failed. Some of the failed SOVs were disassembled and inspected by various methodsc inspection of a plunger assembly under a scanning electron microscope (SEM) revealed a thin coat of the l l

[ l1 l

l y q m y -- -

ee,--- , - - ,

.. . . - . . - .- . _ - - - . ~ .- -- - - - - - - - ~.

i

! graphite base lubricant on one side of the plunger; while the other side showed a thicker l coating. The lubricant is a commercially obtained graphite in alcohol suspension called l Dag@ 156. Disassembly of other SOVs showed the same disparity in the lubricant coating on the plunger and cylinder of the solenoid. Testing with a load cell determined that it took approximate 25 pounds of force to break the piunger assembly free in the l l

SOV cylinder.

I Based upon the tests and observations, Altran, PP&L and Valcor concluded that the l most credible failure mechanism was adhesion of the plunger to the wall of the solenoid l l cylinder. The cause appears to be assembly of the SOV without allowing adequate drying time for the Dag@ 156. Further testing and observations confirmed that this condition was also apparent on other failed SOVs.

c. Conclusions 1

The inspectors observed testing at Altran and agreed with the conclusion that the cause of the SOV failure-to-stroke was a result of assembly of the SOV without allowing adequate drying time for the alcohol. Further testing and observations suggest that this condition may have been the cause of previous failures at the SSES originally thought to l be caused by the residual washer binding.

3.6 Solenoid Ooerated Valve Assembiv i

a. Insoection Scoce l The NRC inspectors reviewed records related to the assembly of V70900-65-11 series, 3-way,120-volt, alternating current solenoid operated valves (SOVs) to identify methods employed by Valcor to control the application of lubricants.

b. .Qhgafvations and Findinos The NRC inspectors reviewed the 3-way solenoid valve assembly procedures for Valcor V70900-65-11 series SOVs and conducted discussions with assembly, engineering and quality personnel regarding Valcor's SOV lubrication methodologies. The inspectors noted that three different lubricants for the V70900-65-11 assembly operations are required. Two of the lubricants are used for the elastomers in the SOVs and the third lubricant', Dag@ 156, is usud for metallic part application. The inspectors noted that the application of Dag@ 156, applied to the plunger and solenoid cylinder, was not procedurally required or controlled.

The inspectors noted that although Dag@ 156 was used on the V70900-65-11 series SOV components, Valcor had not established written guidance for its use. The l

• The NRC inspector noted that Valcor used Acheson Colloids Company (Acheson) dry film graphite lubricant (Dag@ 156) on certain metallic components on its V70900-65-11 series j SOVs.

12

e inspectors found that Valcor's assembly procedures and associated drawings did not address the application or use of the lubricant. Additionally, the inspectors noted that a cautionary note contained in the technicalliterature of the Dag@ 156, recommended a I five minute drying time, was neither relayed to, nor taken into consideration by the Valcor assemblers. The inspectors determined.that Valcor's failure to prescribe written guidance within its procedures and associated drawings to control the application of Dag@ 156 on its V70900-65-11 series solenoid operated valves constituted a nonconformance with 10 CFR Part 50, Appendix B requirements. (Nonconformance-j, 99900728/98-01-01)

The inspectors noted that, prior to the October 2,1998, exit meeting, Valcor had .

developed and implemented revised procedures to address the discrepancy.

c. Conclusions i The inspection identified that Valcor failed to prescribe written guidance within its procedures and associated drawings to control the application of dry film graphite lubricant on its V70900-65-11 series solenoid operated valves.

,/

<'.d,tli, Valcor Enoinserino Dana F. Shave, Bt ,c,d w

(MA4 g' ' f g) -a W Yi Jim Shieh, Quality ,

Barry W. Matiez, E g i vj gre, #

/ g gg, 4 Ravi Rustagi, Mans Anthony Sirianni, A f /

i Jose A. Vega, Jr.,

Joseph E. Sheride-Altran Corporation William J. McBrine, Manager, Engineering Mechanics & Materials Van Christie, Materials Scientist 1t Pennsylvania Power & Lioht Comoany Michael H. Rose, Supervising Engineer L Laurence M. Olson, Senior Engineer-Valve Design 13 i

m. _ _ -

o-l.

l: ITEMS OPENED, CLOSED AND DISCUSSED l* Closed 99900728/91-01-01 VIO Failure to adopt appropriate Part 21 procedures 99900728/91-01-02 VIO . Failure to post Section 206 '

. 99900728/98-01-01 NON ' Failure to adopt appropriate procedures i.

i

!~

i

t I .'

J 14

, . . . ., . - _ - - . _ - . - - - _ _ _