05000528/LER-1992-013, :on 920902,determined That Three Economizer Feedwater Isolation 4-way Valve Failures Probably Due to Mfg Defects.Caused by Improper Machining & Less than Acceptable Dedication Program.Also Reported Per Part 21
| ML17306B120 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 11/08/1992 |
| From: | Bradish T ARIZONA PUBLIC SERVICE CO. (FORMERLY ARIZONA NUCLEAR |
| To: | |
| Shared Package | |
| ML17306B119 | List: |
| References | |
| REF-PT21-92 LER-92-013, LER-92-13, NUDOCS 9211180011 | |
| Download: ML17306B120 (14) | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(x) 10 CFR 50.73(a)(2) |
| 5281992013R00 - NRC Website | |
text
UCENSEE EVENT REPORT'(LER)
FACILITYNAME(I)
Palo Verde.Unit 1
DOCKET HVMBER(2)
PACK 0
5" 0 0
0 5
2 B
1 QF 0 7
TITLE(d)
Teled ne-Re ublic Feedwater Isolation Valve Actuator s 4-wa Valve Bod Problems EVENT DATE (5)
R NUMBER S)
REPORT DATEP)
OTHER FA S SIVOLVED(5)
MONTH DAY 0902 9
2 YEAR
<<j,'
2 NUMBER 013 xa: NUMBER MONTH DAY YEAR FACIUTYNAMES P
n'lo Verde U it 2 Q
0 1
1 08 92.
Palo Verde'Unit 3
DOCKET NUMBER(3) o.sooo529 osooo53Q OPER ATINO MCDKgt) 1 POWER (Io) 1 0 0 NAME 20A02(b) 20A05(a)(1) g)
'0A05(a)(1g) 20A05(a)(1)gI) 20A05(a)(1)gx) 20AOS(aX IX')
50.73(aN2)gr).
SIL73(a)(2)(x)
SIL73(a)(2)(VT) 50 73(aX2)(VIIXA) 50 73(aX2)b~XB) 50.73(a)(2)(x) 20AOS(c) 50.35(c)(I) 50.35(c)(2) 50.73(a)(2)g) 50.73(a)(2)gi) 50.73(a)(2)git)
UCKNSEE CONTACTFOR T)BS LER (I2)
THIS REPORT IS SVBMITTEDPURSUANT TO THE RKOVIREMENTSOF 10 CFR ft (Check ont ot mort ot the~) (11) 73.71(b) 73.71(c)
OTHER (Speclly InAbetract betcrr anilIn Text NRC penn JddA)
Voluntary Re ort TELEPHONE NUMBER Thomas R. Bradish,
- Manager, Nuclear Regulatory Affairs 60 2393 5421 COMPLETE ONE UHE FOR EACH COMPONENT FAILUREDESCRIBED IN TIES REPORT (13)
CAUSE
SYSTEM COMPONENT MANUFAC.
TUBER EPORTABLE To NPRDS CAUSE SYSTEM COMPONENT MANUFAD.
TVRER SJFCVA391Y SUPPLEMENTAL REPORT EXPECTED (Id)
MONTH DAY YEAR YES (IIyea, complete EIIPEC TED SUBMISSION DATE)'
NO EXPECTED SUBMISSION DATE (15)
ABSTRACT(ffm'tb 1400 epacea I e., approximately sheen elnSIe-apace typerrrttten SneeJ (Id)
On September 2,
1992', at approximately 1200 MST, Palo Verde Units 1, 2, and 3
were in Mode 1 (POWER OPERATION) when APS Engineering personnel performing an equipment root cause of failure evaluation of.an economizer feedwaeer isolation valve (FWIV) 4-way. valve, determined that three FWIV 4-way valves'ailures were probably due to manufacturing defects and that fureher investigation to validate this conclusion would 'be pursued.
During the ongoing investigation, six additional 4-way valves were 'found to be similarly out of specification (body seal port cavities were found to be too large and
'out of round exceeding the maximum diametric clearance).
The investigation was completed on October 15, 1992.
Based on the results of the investigation, the cause of the FWIV 4-way valve failures was determined to be due to inadequate vendor supplied components, specifically, due to improper machining, improper manufacturing, and less than acceptable vendor dedication program.
The vendor and manufacturer have concurred with the equipment root cause of failur'e analysis and the vendor has advised the NRC and APS of a 10CFR21 reportable incident relaeed to the 4-way valve failures.
This is being reported as an information-type LER (i.e., voluntary LER).
A previous similar event was reported in LER 530/90-005-00.
9211180011 921108 PDR ADOCK 05000528 S
<Qi
LICENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILIIYNAME Palo Verde Unit 1
osooo52892
. BEQUENTIAL g
NUMBER 01-3
% REVIBION
% NUMBER
- PAOE, OF 0
7 I.
DESCRIPTION OF WHAT OCCURRED:
A.
Initial Conditions:
B.
At 1200 MST on September 2, 1992, Palo Verde Units 1, 2, and 3
were in Mode 1
(POWER OPERATION).- Units 1 and 2 were operating at approximately 100 percent power, and Unit 3 was operating at approximately 92 percent power.
Reportable Event Description (Including Dates and Approximate Times of Major Occurrences):
Event Classification:
Voluntary LER.
At approximately 1200 MST on September 2,
- 1992, APS Engineering personnel (utility, non-licensed) performing an equipment root cause of failure analysis (ERCFA) of economizer feedwater isolation valves (FWIV) (SJ)(ISV) in Palo Verde Units 1, 2, and 3, first concluded that three FWIV 4-way valves'ailures were probably due to manufacturing defects and that further investigat'ion to validate this conclusion would be pursued.
Prior to the event, on May 18, 1992, Unit 3 FWIV-174-M 4-way valve suddenly lost hydraulic accumulator (ACC) pressure.
The top and bottom 'p'ort o-rings and backup rings were found to be tom and pieces of the rings were missing.
This valve experienced a-similar failure in June, 1991.
On June 9,
1992, Unit 1 FWIV-132-M 4-way valve failed to exercise properly (i.e., failed to exercise closed) during routine surveillance testing.
An ERCFA was
" initiated in accordance with,the APS Incident Investigation Program.
As part of the
- ERCFA, PVNGS work history data was acquired, multiple FWIV 4-way valves were disassembled, inspected, bench tested, and laboratory analyses were performed, the vendor and other utilities were consulted, and an inspection of the manufacturing facility was conducted.
During the ongoing investigation, on July 21, 1992, Unit 2 FWIV-174-N 4-way valve failed duri'ng an attempt to repressurize the hydraulic accumulators 'following an accumulator dump to change out the hydraulic fluid (planned corrective maintenance).
Although the specific causes and conditions related to the three failures differ, the same subcomponent from each of the three FWIV 4-way valves was found to be out of specification.
The FWIV 4-way valve body seal port cavities were found to be too large and out of round (i.e., elongated bores) exceeding the maximum diametric clearance.
In addition, the FWIV 4-way valve body surface for Unit 1 FWIV-132-M contained a depression which resulted in
i'
UCENSEE EVENT REPORT (LER) TEXT CONTINUATION FACILITYNAME PaIo Verde Unit 1
osooo 528'92 SEOUENTIAL NUMBER 013
~ REVISION
.'UMBER 0
0 PAOE 030F07 insufficient compression of the o-rings.
'During the ongoing investigation, six additional 4-way valves were found to be out of specification (body seal port cavities were found to be too large and out of round exceeding the maximum diametric clearance).
Because of the extent and type of FWIV 4-way valve discrepancies identified and the continuing problems being encountered with 'FWIV 4-way valves at
Although the operational event does not meet the 10CFR50.73 reporting requirements; APS believes that the infor'mation provided in this voluntary LER'ight prove useful and be of generic interest to the nuclear industry.
C.
Status of structures,
- systems, or components that were inoperable at the start of the event that contributed to the event:
As discussed in Section I.B, following the discovery that nine FWIV 4-way valves were out of specification due to manufacturing
- defects, APS is, submitting this voluntary LER to provide information to the nuclear industry.
- However, no structures,
- systems, or components were inoperable at the start of the event which contributed to this event.
D.
Cause of each component or system failure, if known:
As discussed previously, nine FWIV 4-way valves were out of specification due to manufacturing defects.
The cause of the FWIV 4-way valve failures was determined to be due to inadequate vendor supplied components, specifically, due to improper machining, improper manufacturing, and less than acceptable vendor dedication program.
E.
Failure mode, mechanism, and effect of each failed component, if known:
As discussed in Section I.'B, the FWIV 4-way valve body seal port cavities were found to be too large and out of round exceeding the maximum diametric clearance.
During normal operation, the excess clearance caused by the elongated bores may permit extrusion of the locking ring, and eventually the o-ring, resulting in hydraulic fluid leakage and depressurization of the accumulator.
In addition, the FWIV 4-way valve body surface for Unit 1 FWIV-132-M contained a depression. which resulted in insufficient compression of the o-rings.
Both of these conditions could result in improper operation of the FWIVs in that the FWIVs would be unable to perform their safety function to,FAST-CLOSE (see Section II. ASSESSMENT OF THE SAFETY CONSEQUENCES AND IMPLICATIONS OF THIS EVENT for additional information).
LICENSEE EVENT REPORT. (LER) TEXT CONTINUATION FACLlTYNAME Palo Verde Unit 1
DOCKET NUMBER 0
5 0
0 0
.YEAR '~
<.o BEOUENTlAL NUMBER o)
REYIQON NUMBER PAOE QF07 F.
For failures of components with multiple functions, list of
,systems or secondary functions that were also affected:
Not applicable
'- no failures'f components with multiple functions were involved.
G.
For a failure that rendered
.a train of a safety system inoperable, estimated time elapsed from the discovery of the failure until the train was returned to service:
Not applicable
- - no failures that rendered a train of a safety system inoperable were involved.
H.
'ethod of,discovery of each component or system failure or procedural error:
As discussed in Section I.B, the Unit 3 FWIV 4-way valve failure was discovered during operation when the valve suddenly lost
'hydraulic accumulator pressure.
The Unit 1 FWIV 4-way valve failure was discovered during the performance of a routine, survellance testing.
The remaining valve failures were discovered during the ERCFA.
There have been no procedural errors identified.
Cause of Event
An ERCFA'was initiated in accordance with the APS Incident Investigation Program.
As part of the
- ERCFA, PVNGS work history data was acquired, multiple FWIV 4-way valves were disassembled; inspected, bench tested, and laboratory analyses were performed, the vendor and other utilities were consulted, and an inspection of the manufacturing facility was conducted.
Although the specific causes and conditions related to the three failures differ, the same subcomponent from each of the three FWIV.4-way valves was found to be out of specification.
The FWIV 4-way valve body seal port cavities were found to be too large and out of round (i.e., elongated bores) exceeding the maximum diametric clearance.
In addition, the FWIV 4-way valve body surface for Unit 1 FWIV-132-M contained a depression which resulted in
'insufficient compression of the o-rings.
During the ongoing investigation, six additional 4-way valves were found to be out of specification (body seal port cavities were found to be too large and out of round exceeding the maximum diametric clearance).
The investigation was completed on October 15, 1992.
Based on the results of the investigation, the cause of the FWIV 4-way valve failures was determined to be due to inadequate vendor supplied components, specifically, due to improper machining, improper f
li 0
LICENSEE EVENT REPORT,(LER) TEXT CONTINUATION FACIUTYNAME Palo Verde Unit 1
DOCKET NUMBER osooo52892 M'EQUENTlAl "< REVISION NUMBER
.4 NUMBER 01 3
0.0 PACE 0 50F 0 7'anufacturing, and less than acceptable vendor dedication program (SALP Cause Code B: Design, Manufacturing, Install'ation 'Error).
Other factors contributing to the FWIV 4-way valve failures included less than 'adequate maintenance rebuild and bench testing instructions,,and less than adequate failure data trending and root cause of failure analysis.
No unusual characteristics of the work location (e.g.,
noise, heat, poor lighting) directly contributed to this event.
There were no personnel or procedural errors which contributed to this event.
Safety System Response:
Not applicable
- - there were no.safety system responses and none were necessary.
K.
Failed Component Information
The FWIV 4-way valves are supplied by Anchor/Darling Valve Company.
The Anchor/Darling part number is W-19488.
Teledyne-Republic is the actual manufacturer of the 4-way valve, model number 23104-7001-2853.
Anchor/Darling believes that the discrepancy is only applicable to 4-way valves having serial numbers less than 1600.
II.
ASSESSMENT, OF THE SAFETY CONSEQUENCES AND IMPLICATIONS OF THIS EVENT:
The safety function of the FWIVs is to FAST CLOSE when a Main Steam Isolation System (MSIS)
(SB) Engineered Safety Features, Actuation System (ESFAS)
(JE) actuation occurs.
An MSIS is. actuated by receipt of a '2-out-of-4 high containment (NH) pressure, low steam generator (AB)
- pressure, or high steam generator water level signal.
The FAST CLOSE action prevents containment overpressurization,and excessive reactor (AC) cooldown following a main steam line break, main feedwater (SJ) line break, or loss of coolant (AB) accident.
The MSIS rapidly terminates steam flow and feedwater flow by isolating each steam generator.
There are four FWIVs, two in series in each feedwater line.
One valve in each line is actuated-from Train A and the other valve is actuated from Train B, providing completely redundant functions.
This configuration ensures that a single failure in the power source or a mechanical failure of one FWIV cannot result in the loss of the isolation function for the economizer (SJ) lines.
In the three cases, the redundant FWIV actuators in series were operable.
Therefore, both feedwater lines were still capable of'being isolated in the event that a MSIS would occur.
This condition did not adversely affect plant safety or the health and safety of the public.
The event did not result in any
i5 il
LICENSEE EVENT REPORT (LER) TEXTCONTINUATION FACILITYHALK Palo Verde Unit 1 DOCKET NUMBER' 5
0 0
0 5
2 8
9 2
SEQUENTIAL W NUMBER 013 REVISION NUMBER 0 006 PAGE op 07 challenges to the fission product barriers or result in any releases of radioactive materials.
Therefore, there were no other adverse safety consequences or implications as a result, of this event.
III.
CORRECTIVE ACTION
A'.
Immediate:
Following the discovery of the FWIV 4-way valve out of specification, an analysis of the installed FWIV 4-way valves (eight per unit) was performed, Four FWIV 4-way valves of the twenty-four were considered at risk only because insufficient data was available to support their longevity and/or acceptability.
Of the four FWIV 4-way valves at risk, valve failure would be immediately indicated in.the Control Room for three of the valves.
These valves are in their normally safety actuated position and annunciation is provided'o alert Control Room personnel of hydraulic fluid pressure loss.
The fourth valve is not in its normally safety actuated position and there would be no indication that the valve would not perform its safety function until valve failure occurred.
The fourth valve was replaced on September 4,
1992 Action.to Prevent Recurrence:
- - An ERCFA was conducted in accordance with the APS'nci'dent Investigation Program.
Based on the root cause determination, as discussed in Section I.I, APS has initiated actions to prevent recurrence (i.e.,
comprehensive 4-way valve disassembly, inspection,.
and reassembly instructions, FWIV 4-way valve replacement in the three units, improvement of the FWIV performance monitoring program, etc.).
These actions are being tracked to completion under the PVNGS Corrective Action Tracking System.
The vendor and manufacturer have concurred with. the equipment root cause of failure analysis and the vendor has advised the NRC and APS of a 10CFR21 reportable incident related to the 4-way valve failures.
IV.
PREVIOUS SIMILAR EVENTS
A previous similar event has been reported pursuant to 10CFR50.73 and 10CFR21 in LER 530/90-005-00.
APS determined that FWIV 4-way valve rebuild kits contained non-qualified backup ring material (non-Vi.'ton).
On August 17, 1990, in accordance with the APS Vendor Quality
ii 0
UCENSEE EVENT REPORT (LER) TEXTCONTlNUATION FACIUTYHAL%
Palo Verde Unit 1
DOCKETNUMBER osooo528 YEAR 8
9 2
0 1 3, REVIQON NUMBER 00 07 PAOE OF 07 Engineering
- Program, APS issued a Vendor Corrective Action Report
{VCAR) to Anchor/Darling Valve, Company.
During the 4-way valve disassembly/assembly to replace the questionable backup rings, APS identified several deficiencies and noted these conditions in the VCAR.
The VCAR requested that Anchor/Darling provide a root cause analysis and detail what actions would be taken to prevent recurrence.
In the latter part of 1990,, Anchor/Darling developed a dedication program which included a disassembly, inspection, and reassembly procedure and a static and dynamic testing procedure.
Based on the. APS Incident Investigation performed related to.this voluntary LER (528/92-013-00),
APS has determined that Anchor/Darling's dedication program is less. than acceptable and that it requires re-assessment and correction.
The disassembly, inspection, and reassembly procedure does not include verification of critical parameters and dimensions.
The dynamic testing performed on these valves did not simulate the operational characteristics of the valves.
0 0