IR 05000327/1990036
| ML20217A611 | |
| Person / Time | |
|---|---|
| Site: | Sequoyah  |
| Issue date: | 11/09/1990 |
| From: | Girard E NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML20217A609 | List: |
| References | |
| 50-327-90-36, 50-328-90-36, NUDOCS 9011210186 | |
| Download: ML20217A611 (46) | |
Text
{{#Wiki_filter:_ _ - , , , , . . - . - . . . . _ -
.e s ' '
v.
E UNIT ED ST ATES pa ntog#o, NUCLL' R REGULATORY COMMISSION 3 ~* REGION li
#
h* 101 MAnlETTA sTRE[T. ,5 - 2 AT LANT A, GioRGL A 30323
' , \...+
Report Nos.: 50-327/90-36 and 50-328/90-36 Licensee: Tennessee Valley Authority 6N 3BA Lookout Place 1101 Market Square Chattanooga, TN 37402-2801
-
Docket Nos.: 50-327 and 50-328 License,Nos.: DPR-77 and DPR-79 Facility Name: Sequoyah Units 1 and 2 Inspection Conducted: October 11-19, 1990
' ;
Team Leader: 4 E Gifard, Reactor Inspector
//h O Date' Signed Materials and Processes Section -
Division of Reactor Safety Team Members: M. Branch, Watts Bar Senior Resident Inspector J. Donohew, Sequoyah Senior Project Manager, NRR R. Woodruff, Senior Reactor Systems Engineer, NRR EXECUTIVE SUMMARY This special inspection was conducted by an Augmented Inspection Team (AIT) to provide the NRC staff with important factual information regarding the circumstances of multiple Main Steam Check Valve (MSCV) f ailures reported at the Sequoyah Plant on October 8,1990. There are four of these valves per unit and their safety function is to prevent reverse flow of steam into the containment in the event of a steamlin; break that is upstream of the Main Steam Isolation Valves (MSIVs) and inside the containment. The MSCV design is depicted in simplified form in Figure 1 and the locations of each unit's four
-
MSCVs relative to the MSIVs and containment are depicted in Figure Three complete MSCV failures were identified fur Unit 1 and one incipient MSCV failure was identified for Unit } The Unit 1 f ailures resulted from f atigue fracture of the post that attaches the disc to the swing arm of these swing check valves. The fracture occurred at a corner radius where the post transitions in size from a section threaded into the disc to a larger diameter unthreaded section which passes through the swing arm. In all three failed valves the diset separated f rom the post and swing arm. Two of the discs were carried downstream by steam flow and the other was lodged in the valve body. The principal physical cause of the failures was a design modification involving a weld buildup of the posts, performed in May 1990. The principal root cause was the failure of the licensee to assess the deleterious effects of the welding stresses in their h.12101869011oo a' a mcx osooo m _ PDC _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ -
_ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - o . ,
* , . .- :
design evaluation. The valve vendor had concurred with this change prior to i its implementation and it appeared that the licensee placed excessive reliance ' on this concurrence in lieu of detailed evaluation of the design chang Another contributing factor was apparent deficiencies in the original design ,
(e.g., the close proximity of the MSCVs to MSIVs which results in undesirable turbulent flow and cyclic stresses on the MSCV components).
r The Unit 2 incipient failure, like the 1990 Unit 1 f ailures, occurred as fatigue cracking of the valve post. However, it differed from the 1990 Unit I failures in that the cracking had not completely severed the post and it occurred in the end of the post near a weld that joined the end of the post to the valve disc to prevent unthreading. Two similar f ailures occurred on - Unit 1 MSCV posts in 1982, one incipient and the other a complete failure which allowed unthreading of the disc-to-post connection and permitted the disc to be carried downstream by steam flo The 1982 failures were attributed
. principally to valve counterweights having moved out of position and to lack of any specified torquing of the post into the disc to preload the joint in-original f actory assembly. Both were considered to have resulted in increased cyclic motion and stresses. The. licensee torqued the replacement posts for the 1982 failures into their discs but did not make this change to other MSCV They relied instead on corrections to assure against counterweight movement and ;
nondestructive examinations (at refueling outages) to detect cracking. The nondestructive examinations were discontinued prior to the last refueling outage. The 1990 Unit 2 incipient failure involved a valve which apparently had not experienced' counterweight movement. The physical cause of this failure was considered'to be inadequate preload combined with excessive cyclic stresses ; inherent to the installed piping and valve design. There had been no weld repairs on any of the Unit 2 posts prior to their crackin Had the licensee not been prompted to investigate the cause of a loud- noise that was apparently caused by a disc movement, the Unit 1 MSIV failures might have gone undetected until at' least the next Unit I refueling. outag No : similar noise was reported or investigated when the 1982 disc loss occurre Further, the licensee had not- been performing required periodic inservice , testing to assure the continued. operability of the MSCVs. (Note: The 1982 1
. failures were discovered during a refueling outage as a result of investigatio .
of out-of position counterweight arms.)
l Actions now being taken by the licensee to assure against further failure's of ,
'
the MSCVs include: l
-
Replacement of- previous posts with posts that do not have weld deposits and are identical to the original design except for an increased radius 'to' reduce a~ stress riser at the location of the 1990 Unit 1 fractures ,
-
i~ Addition of two disc stops to each valve to aid in reducing cyclic disc motion '
, -
Inservice acoustic monitoring to detect valve degradation l -
-Torquing the threaded posts into the discs to preload the connection and provide rigidity -
Evaluation of other options for long term correction
,. _ .-. . - _ _ .
p, - ."*-i
,
n ,
- ,. ;
U~ ;
'
e ! f i ; I f P t
!'
. f i
l TABLE OF CONTENTS i Page - Introduction - Formation and Initiation of AIT. . . . . . . . . . . . . . . . . . . I a l
'
,. Background................................................... ~1
. Formation of AIT..........................................., 1-
'
"L
= AIT Charter - Initiation of Inspection...................... 2 !
2. - . Description of Events............................................ 3 >
.
.
B verview.................................................... 3
/F 2,2 Chronolgy of Event - Indication.of Failure.................. 3 Timeliness of Licensee Actions.............................. 8 '
a o i L Fa'ilure Description.............................................. 9
:
,
.,
.
.
;
L 3.1. Design Description.......................................... 9 . L '
.3.2 Unit.1 Failures Discovered October 1990..................... Il >
p 3.3' Unit 2: Incipient Failure Discovered October 1990............ 13 .
,
g . . Significance of Failures......................................... 13 .
! System-Performance......................................... 13 ; '4.2 Damage'to Piping;and~ Supports.............................. ~
j 13 ! p 4.3 SafetyfSignificance- 14 ,
'
L .
.
-t L .
. Equipment Hi story - Check Valve Hi story. . . . . . . . . . ... . . . . . . . . . . . .
. 16' -:
r i 5.1 Mai ntenance and Modi fication s . . . . . . .' . . . . . . . . . . . . . . . . . . . . . . 16 - : 5.2 ' Examination s and Te st s Hi story. . . . . . . . .. . . . . . . . . . . . . . . . . . .'. . . 20 '
. Failure Evaluation............................................... 24= $
'
!1 6 ' Metallurgical Examination... ..............................
. 24 a Nondestructive Examination................................. 25' l 6.3: Wear Examinations,........................................ 25 6;4' Comparison with 1982 Failures............................. .
25-16 . 5 : Sy s tem De s i g n E f f ec t s , . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . : 26 - , 6.6: Mockup Test.....................................-........... 271 6.7- .Summarycof Physical-Causes...................... ........... 28- ; t ' Root Cause....................................................... 29 q i
'
7.1 Genera 1,............................................,,,....... 29 E 7.2-' Other. Contributing Factors.................................. 29 ,
>
.
+
L
.6 i i
!
'
L 1 1 _ . _ . _ ..
I
,, . . ~ , .
."
..
. . , _ . . - - ..- . . - . ~ . . . - . . - . . . . ~ . _ - . . - -. .- ,$mh #.
d -5
?j
: , ;F
;b~u , . ', j . ,
, + s
,
,
4 .. -,
..
. o
,
,
a e
.. ,
4 1
;
, , . ,
.- , .
. .- .
. ,.
s . ~ . .
,' '~ l ' _.'
. ,
, ,
.. , t
,..-
o- ,
,
j ( - e .
{
l
- !8; Corrective Actions............................................ . ... 30' :;
,
- - i M
I,.,-
'
8.1 : Main' Steam System Design................................... 30
' t . 1: -
B'],' 8.2: MSCV Design.................................................. 30 -!
!t<
'
n N' 8.3 MSCV Monitoring'and Surve111ance................-............ 31 Fact..............................'...................-32
' '
' Findings of
'
~9.
- +
. : f- , .. ..i j '
10 .L . C o n c l u s i o n s . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 4 :
.
a +.. f. ) t :
.3 .:
r_a >
: 11. - Recommendations..................................................-35:
e . , Exit Intent ew with Licensee Management. . . . . . . . . . . . . . . . . . . . . . . . . . 36
. , s d k-
-l s L c-r f' .
$
A
.g E
}.,. I !
'
,
~ . 'l i '
*
[[
w " ' . . - iI / r e a .f,
, ".' I<
'.,.g - '
_. <
' <
i g i ,'{%
% }
#
&
,1
. f :'.
'f ' .$ -,
g,
,
' '
.Y
) } ' z (P,'
y,**_ . _ sg '
" #
4-'4 l a
'
,
' ; . '._V
_ f
<
-
}-
h j .y _, s .t . i h.
p
.h t ,' . .
i3a, ~ e I _i-'^ --p !.,' .,
'i ,
1 s
( 'ya L b
_{y - . g ,'. '1 ' } :J
. , . *
I
.
k y
,
I h l
'
r h:
._ .
--
.
' - ,
s
./ ,
-n, .. x
'
. ,
-
..
a 3g ,; n ,
-
14 i. .
],- -
t -
' [.,$ ,( . '1: ..
I' { 7, ;V
>3 y: . .. ' .\
i;h c. , {'
, " "
APPENDICES
.s- 4 g-
-
i
-
.
.. .
i < . Appendix-1 .. Persons' Contacted-
"
j
'
Appendix 2' fAcronyms and Initialisms-n; h, - FIGURE > f~ _; Figure'l Main Steam Check Valve t- ' Figure 2 - Main Steam Valve Vcults p '- , ^ Figure' = MSCV. Disc Travel.Pavh i . ~ ,' Figure'4 Preparation of;the Ptst for Weld Overlay:
, - . -k ll '
pt
. .
I f I
, T , .
1 i Tjl f l'- t f
' ' .p,-, 'i- ~
n ~[ , j l' f '.', g
-
-
c 09 f i ..> ,e . .9, . ' '
'
,
.* 'y 1 ,$_'.
fk) g () ' *.,
' 1 L
4 . j '4_
,..
.
.
...
,. '
'
~t t '
r i'Y r
',.t.-
k ' 4 .I - t
'
')
.;. ; n r l ~ {I[.
-
. ,
'
, ,' t . - v .
,
. - ,
<
.,i)
'
.-
I j,- , 6
,
- k '- e .'-(,
'l'
-
'
s i t
, .
' . r.i '. s .. , t-
,
t
-
i-f .s-
. ,=
k
-.,., ',
f
' <.',. 4 ]'? 7 y,[ - y.]::
_
' W 3). .
' . 1 .. 4 ) '
m
'l
> M
.
'
'#
'
:u
'-
}f l -1 >
3 . t
}
- p. g )
, 1 ,.,.-L ., ,a . : ; - , .. . u. -.. .m a v, . s .~ -~ u . . -,- a ..a
'
.+...s-,.-,~., a-<....a,,,- .,-,,,...a-.,-,a.,,;-,% :n ,L~ %;, ;+ - ~
- _ _ . . . . . . .
' -'
,
, ,
.
.
1. INTRODUCTION - FORMATION AND INITIATION OF AIT 1.1 Backr.ound S'.quoyah Units 1 and 2 are virtually identical four loop Westinghouse
' ressurized Water Reactors with ice condenser containments. The units are located 10 miles northeast of Chattanooga, TN. Unit 1 went critical in July 1980 and was commercially operational in July 198 Unit 2 went critical November 1981 and commercial operation began in June 198 Both units were shutdown for about 3 years between 1985 and 1988 while various safety issues were resolved. T h r. most recent refueling outages for the two units were the Unit 1 Cycle a refueling outage from March .o June 1990 and the Unit 2 Cy'le * refueling vutage from
.
September to November 199 On Octobea 8,1990, at 7:21 a.m. , the licensee declared a Notice of Unusual Event (NOVE) due to commencing a forced shutdown of Unit I as required by Technical Specifications. The shutdown was required because 1 of 4 Main Steam System check valves (valve -624) was found to have experienced a failure of the disc to swing arm connectio These check valves prevent reverse flow of steam into the containment in the event of a steamline break in the containment upstream of the Main Stream Isolation Valves (MSIVs). A simplified depiction of this valve is given in Figure The licensee exited the NOVE on October 8, 1990 at 3:34 Subsequent inspection of all four Unit 1 Main Steam Check Valves (MSCVs) found that the partially threaded stuJ (more commonly referred to as a " post") which attached the disc to the swing arm on each was broken in three of the valves and intact in the fourth. The valve discs on two of the valves with broken posts were missing and assumed to be somewhere in the steam lines. The disc from the third was found in the valve bod The Unit 2 MSCVs were already disassembled for the current Unit 2 refueling outage and inspections revealed they did not have the same proble However, several did have significant post wear; and one had cracking in its post at a different location, with potential for incipieut failure and release of its disc similar to a Unit I failure that occurred in 198 .2 Formation of AIT On Octobea 9, 1990, the Regional Administrator, after briefing by the regional end resident staff and consultation with senior. NRC management, directed the formation of an Augmented Inspect'on Team (AIT) to invastigate the circumstances of the Sequoyah MSCV failure The team was to be composed of Region 11 and NRR personnel and to be headed by a Re31 on 11 Reactor Inspector / Metallurgis .3 AIT Charter - Initiation of Inspection
. . . _ _ _ _
,
. .
. . !
!
'
c
l
!
i i The.. Charter for the AIT was prepared on October 10, 1990. The team l 1eader. and one other AIT member had been performing preliminary ; inspection activities onsite since October 9 and were joined by the ; remaining team members on October 10. The Augmented Inspection J commenced with an Entrance Meeting and briefing of licensee i management on October 11, 199 The Charter for the AIT specified i that the following tasks be completed: l Develop and validate a detailed sequence of events associated with the failure of three main steam line check valves at
,
;
Sequoyah Unit 1, reported on October.8, 199 : Determine the cause (if possible) of the valve failures and evalua'e their significance with respect to system performance and safety significance as defined in the Technical Specifications and FSAR.- 3, Develop the testing, inspection, modification, maintenance and o;2 rating history of the valve . Compare the failure mode with that which occurred in 1982 - 1983. Evaluate the appropriateness of TVA's corrective actions prior to the recent failures, in view of' the above developed information. Evaluate the- basis for and timeliness of the repairs done in May 199 . Evaluate- any damage that the valve disc may have done af ter valve failur +
' Evaluat'e TVA involvement with the valve ve.idor.-
:
'
. Evaluate the TVA's implementation of the recommendations in the-INP0 .SOERs, the EPRI Application Guidelines for Check Valves ,
(EPRI Np-5479), and NRC Bulletin .
t , Evaluate operational history -' steam flow, turbine performance or other possible indicators of failur . Evaluate the licensee's communication of this event to the NRC, and determine if appropriate notification was made for all ; reportable occurrence . Evaluate the timeliness of TVA's actions from the time that the original noises were heard in the valve vault until Unit 1-was' . shut down, * 1 prepare a special inspection report documenting the results of the above activities within 30 days of the start of the-inspection.
l
!
_
- . _ _ _ _ _ _ _ . ' ' ., , . ! .
. .
1 Evaluate the effectiveness of the radiography for detecting this type of valve failur . DESCRIPTION OF EVENTS 2.1 Overview On September 19,1990, Unit 1 was made critical in its restart from a reactor trip. On September 20, 1990, the generator was put online to the grid and on September 21, 1990 a loud noise was heard in the East Steam Valve Vault. On October 8,1990, Sequoyah Unit I was shutdown-from 100 percent power af ter it was determined by radiography that the disc of Main Steam Check Valve (MSCV) 1-624 was detached from the arm and lodged in the valv The MSCVs are needed for the MSIV (i.e., Technical Specification valves) to perform their function in , the event of a main steam line break (MSLB) upstream of the MSIV Af ter the plant was cooled down, inspection of the internals of the four MSCVs revealed that the disc. post had failed on valves 1-623, 1-624 and 1-625 but not for valve 1-626. The disc for valve 1-624 was lodged in the body of the valve but the discs were missing from ( valves 1-623 and 1-62 The missing discs, which weighed approximately 600 pounds each, were later found lodged in the main steam line (MSL) piping approximatelv 500 feet downstream from the
~
MSCV.. The 28 inch diameter discs were lodged at the 36X28 inch i reducer in two of the four main stean supply lines between.the mixing tee-and turbine throttle valves 1 anc The licensee had been investigating ehe potential loss of a MSCV disc since September 21, 1990, when seve/al plant personnel reported loud noises (i.e.-, " sounded. like a fre ght train and it moved down the d i pipe toward the turbine"). Inspections of counterweight arm position of the four check valves;was inconclusive in determining if a disc ! was missing. However, information noted,as to arm position of_ valve
*
1-624 and noise attributed to the disc hitting the stop' for valve 1-626 later proved to be indicators of'the failures. Additionally, ; the licensee performed acoustic monitoring of the four valves to l determine if they could establish whether the disc was attach:d or not. Again the data was determined to be inconclusive even though ; the 1-624 valve was drastically different from the rest Radiography of' the valves was ,:anned but was delayed because of the
'
unavailabild iy of a special linear accelerator (MINAC) needed to
>. provide radiation sufficient to penetrate the heavy metal valve body and h<cause of specici controls necessary to -limit the potential esposure of. individuals onsit _
2.2 Chronology of Event - Indication o. Failure The chronology of events described below was based on interviews with involved individuals;: review and verification of information provided by the licensee; and information contained in operating, maintenance, and modification records. Information gathered through interviews t
,
. - . ,
- *
, , . .
was corroborated by other sources or noted as a statement attributed to one individua In the late afternoon of September 21, 1990, Senucyan un't I was in the process of escalating reactor power to 100 percent after startup f rom a reactor trip on September 19 1990. At approxir-tely 18:45 loud noises were heard coming from the Unit 1 East Steam Valve Vault by a Refuel Floor Coordinator who was outside the room. He described the noises as sounding like something fell vertically near the main steam reliefs, then something heavy hitting a wall several times. He also described it as sounding like a f reight train moving away from him which lasted a few seconds. He had never heard the noise,before, was scared by it, and believed a steam line had rupture At the same time a secu-;ty guard in the turbine building heard a loud
" boom" with a retal-to-metal scraping sound like rubbing brakes and as if sometFing was moving thruugh the inside of the large pipes in the overhead above the security access porta He stated that the noise lasted a few seconds and scared hi Both of the employees reported their observations to the control room. The security guard noted that whatever it was, it caused several of the large pipes in the turbine building to shak The control room operators believed -
the reports were from a noise only in the East Steam Valve Vault and sent an Auxiliary Unit Operator (AUD) to investigate the noise in that vault. The AVO reported back to the control room that the cause could not be identified. There were no indications in the control room of any abnormal event on the secondary steam side. None of the above accounts were recorded in the control room logs for September 21, 1990. The AUD on duty in the Turbine Building on this Friday evening stated that he heard nothing unusual that evenin On September 23, 1990, af ter being off for one day, the Refueling Floor Coordinator returned to work and discussed the event of September 21, 1990 with the Work Control Group (WCG) Shif t Manage At this time, the Refueling Floor Coordinator and the WCG manager were not aware of the report to the control room by the guard on September 21, 199 The WCG manager, recalling the 1982 MSCV disc separation event, ques.ioned whether the noise heard on September 21, 1990 could have been caused by the leas of a MSCV disc. He requested that system engineering personnel investigate the noise and attempt to determine if there was anything unusual about the position of the counterweight arms for the MSCVs. A group was sent to the East Steam Valve Vault to observe the two MSCVS. They were told not to move the counterweight arms on the valves. The group reported that nothing abnormal was seen. At the same time the WCG manager, recognizing a Technical Specification (TS) tie between the operability of the MSCV and- MSIV, requested that engineering process a Justification for Continued Operation (JCO) with a safety evaluation to allow operation with a missing MSCV disc. In order to conclusively determine if loss of a MSCV disc had occurred, the system engineering group recommended radiography of the two valves in the East Steam Vaul Standard radiographic machines and sources would not work due to ,
, , , , , , , , , , , , ,
__ _ _-
' * ; , . ,
C
' > '
their inability to provide radiation sufficient to penetrate the thickness of the valve body. The licensee owned a special linear accelerator X-ray machine termed MINAC capable of providing the necessary radiation but neither it nor the few other similar machines that existed could be obtained and set up in less than a few week Engineering understood that the delay allowed them additional time to develop the JC In addition to his discussion with the WCG manager (described above) on September 23, the Refueling Floor Coordinator also contacted the t ' Assistant Shif t Operating Supervisor (ASOS) regarding the noises he had previously reported. It was at this time that the control room L became aware that the noises reported on Friday evening were heard to travel from the East steam Valve Vault to the Turbine Building. The ASOS stated that he could only think that the noises indicated that something in the MSCV had failed and parts had travelled down the Main Steam Lin He made an entry in his log on September 23, 1990 to the effect that on September 21, 1990 he had received notification from several people that they had heard loud noises with
' '
metal-to-metal rubbing inside the MSLs from the East Steam Valve Vault to the Turbine Building. The entry stated that he _infumed the
,
Shift Operating Supervisor (SOS) of these fact , t-The SOS. stated to the AIT that, following the additional expressions of concern by the Refueling Floor Coordinator and WCG manager on September 23, he highly suspected the problem to be loss of an MSCV disc. He indicated he did not recognize the importance of this loss to. MSIV operability, but that he notified his management- of the noises and of his belief that it indicated an MSCV disc los On September 25, 1990, an inspection of the counterweight arms for MSCVs.1-624 and 1-625 found that the arm for 1-624 was ? inches lower i than the rest._ Also, one counterweight arm could be noved and one - , could not. No conclusions were made as to whether the orientation of # l the arm indicated the absence' or presence of the dis However, a JC0 for Unit 2 operations in a 1982 event where a MSCV dise was found
~
missing for valve .1-623, was based on the inspection of the , counterweight arm position of each MSCV (i.e. , the a' ms were- at. the l 'same height and the licensee concluded that the _d.scs were still attached).
The WCd manager indicated that he believed one disc was missing because of the difference in the movement of the arms but that.this evidence was not conclusive. He stated.that he informed the plant manager that MINAC radiography would be performed to determine conclusively if a disc was missin .On September 25, 1990, the Plant Manager notified the NRC Senior Resident Inwtor of the ongoing questions and evaluations for the l MSCV _j l
i i
_- . . _ _ _ . .
, e+ , ,
.
On September 26, 1990, the counterweight arms for all four MSCVs were inspected with licensee personnel attempting to pull against the arms. The valves were determined to exhibit similar responses to the ' pulling. Unlike the other MSCVs, 1-626 made a distinct contact sound when the arm was released. The licensee subsequently contacted the vendor on September 27, 1990 and the vendor indicated that the above . test could not be relied on to conclusively determine disc ! separatio On September 27, 1990, an acoustic analysis was performed on each t valve. The results indicated that valve 1-624 was noisier than the res The licensee determined that the results were inconclusive in identifying disc separation due to difficulties in assessing the data without previous baseline dat A Problem Event Report (PER) was-initiated to document the noises previously heard in the East Steam Valve Vaul The licensee stated that the MINAC linear accelerator was received onsite on October 2,1990. Operational checks of the - device were completed by - October 4, 1990. The Plant Manager and the Vice President, Nuclear Operations decided on October 4,.1990 to delay the radiography until midnight October 7,1990, to minimize the potential for inadvertent exposure of onsite personnel. . The Plant Manager informed the NRC Senior Resident Inspector of this decision on the same da On October 4,1990,- the licensee sent their JC0 on MSCV dise loss to ! the NRC Project Manager and requested a discussion with the NRC staff ' on the possibility of operating with an inoperable MSC ; On October 5, 1990, at approximately 01:50 loud noises were heard during turbine throttle valve testing. A licensee investigation attributed the noise to linkage rubbing on the number 3 turbine { throttle valve during valve closur The operators log indicated ? that several noises were heard during the closing of the valve and again during opening of. the valve, that the . turbine high vibration alarm was received and cleared, and vibration of the steam valves was also noted. The AUD who performed the test but who was not interviewed by the licensee investigation team, indicated that there were four distinct noises heard and that they appeared to be internal to the valve. He stated that they caused significant vibration which stirred up so much dust he first thought that there was a steam' lea The operators log also' indicated that the governor valve position j after.the throttle valve test was approximately 20 to 25 percent-more ' open for the same power leve This condition existed for , approximately one hou I On October 5,1990, -the licensee discussed with the NRC staf f the possibility of operating Unit I with an inoperable MSC The NRC staff did not agree with the JC0 but did agree that operations could continue until radiography of the two MSCVs was performed on
* *0 j - , . , .
L
i i October 8. .199 The 4RC Staff was not informed of the turbine vibration or noises a ssociated with throttle valve testing on October 5,1990, and de scribed abov The valves were radiographed on October 8,1990 and the radiographs indicated that the 1-624 disc was unattached. However, the evaluation of the radiographs for valve 1-625 determined that the ' disc appeared to be still attache The licensee began radiography at midnight. There were four shots for a composite picture of a - t MSC The first composite picture of valve 1-624 was completed at L 02:30 but was confusing. The second composite picture of valve 1-624 was completed at 05:00. It showed the arm in the MSCV clearly and apparently without a disc but had a confusing shadow to one side. It was decided to radiograph valve 1-625 for comparison. The composite picture of valve 1-625 was completed at 07:00 and appeared to show a disc was below the arm. The licensee stated that the decision to shutdown Unit I was based on comparing the last two composites.
, On October 8, 1990 the unit was taken to hot shutdown from 100 i percent power as required by the actions of Technical Specification 3.7.1.5 for an inoperable MSI Entries in the: control room log indicate the following on the shutdown of Unit 1 on October 8, 1990: (1) 07:10, Lagergren informed the control room to shutdown the unit in accordance with TS 3,7.1.5, (2) 07:44, NRC notified of shutdown and declaration of NOVE,
.(3) 07:13, shutdown of the unit had commenced, (4) 11:02, the unit had entered Mode 3, and (5) 15:35, the unit had entered Mode On October 9 and 10, 1990, after the plant was cooled down, all four of the MSCVs were opened for-internal inspection. The inspection revealed that the' disc post had failed on valves 1-623, 1-624 and 1-625 but not for valve 1-626. The disc for valve 1-624 was lodged in the body of the valve but the discs were missing from valves 1-623 and 1-62 . On October :ll,1990 the missing discs, which weighed approximately 600 pounds each,. were found lodged in the main steam piping approximately 500 feet downstream from the MSCVs. The 28 inch diameter discs were lodged at the 36X28 inch reducers in two of the four main steam supply lines between the mixing tee and the turbine throttle. valves 1 and Unit 1-Cycle 4 Operating History The purpose of this section is to detail operational transients 'or events that could have caused perturbations in the steam' supply system and might have had some relation to the MSCV failures. -Unit I had been returned to power on June 3,1990, following the cycle 4 refueling outage. During that outage unexpected wear was noted on I three of the four MSCVs and modifications .per Design Change Notice j
y_
*-
c. ' o a i: I
..
,
f 8 )
!
-
i (DCN) M-3281-A were accomplished. There_were 2 reactor and 3 turbin l trips between the time the unit was returned to power on June 3,1990 !' and September 14, 1990. The summary of events below starts with the September 14, 1990, reactor trip from 100 percent power and continues ' through the October 8, 1990 plant shutdown and cooldown.- _
'
09/14/90 1614 Rx trip from 100 percent power due to S/G Lolo level following vital invertor I-II failure 09/16/90 -0552 Rx critical ' 2005 Rx power at 14 percent 2235 Turbine trip from main transformer electrical problem, Note: Power was below
.-
the P-9 (Reactor Trip / Turbine-Trip) setpoint 09/17/90 1852 Generator online 09/19/90 0340~ Holding power at approximately 62 percent
-0357 Turbine trip /Rx trip from main transformer electrical problem
, 2113 Rx critical 09/20/90 -1446 . Generator online 09/21/90 1845 Loud noise detected by. employees Two employees notified Main Controi Room (MCR) MCR'did not detect any abnormalities
.
ASOS dispatched operator to East Valve-Vault, no abnormalities.were found 10/05/90: 0150 A loud noise was heard at the turbine during.- a throttio valve stroke: test-10/08/90 '0710 . Started shutting downL Unit 1 from 100 percent power due to an inoperable MSCV w '0744 NRC ' notified of shutdown in. accordance with 10 CFR 50.72 1102- Mode 3 entered 1535 Mode 4 entered e
- * , ,
, + .
2.3 Timeliness of Licensee Actions The licensee actions in response to the noises heard on September 21 and October 5, 1990 are given in detail in the sequence of events in Section 2.2 above. These noises are the only ones which the staff is aware of that could be, or were associated with a separated disc travelling inside a steam line from a MSCV in the East Steam Valve Vault to the riser lines beneath two of the throttle valves in the turbine buildin The licensee's first formal documentation of the events of September 21, 1990, were noted on PER 50900061, This report indicated that the licensee suspected that the cause of the noises reported on September 21,'1990, were from a separated MSCV disc, The report further indicated that the licensee was trying to conclude beyond a doubt that there was an actual f ailure prior to initiating the actions of TS 3/4.7.1.5, Additionally, Ali interviews of plant personnel and review of operating logs indicated that a missing check valve disc was the primary focus of the licensee's investigation of the noise Interviews of upper level plant management indicated that, although they were pursuing a course of actiou directed at either proving or disproving the check valve disc failure, they also had considered that the noises heard could have resulted f rom other causes (e.g. , spring hanger release or water hammer).
The AIT determined through record review and personnel interviews that verbal information flow between the operating crew and management was not fully successful in characterizing the type or possible source of the noises heard on September 21, 199 Additionally, the graphic description of the noises provided to the AIT were not as well characterized in the written accounts documented in the PER review. The team further determined that several inoividuals key to developing an accurate account of the actual events were not interviewed by the licensee (e.g. , the AVO who performed the throttle valve test on October 5, when loud noises were heard), The licensee worked from September 23 to Octioer 4,1990 to get the MINAC radiography machine onsite and operational. The licensee had concluded that radiography of the MSCVs in the East Steam Valve Vault-would conclusively prove whether a disc had separated from a MSCV or not, At this time, no one had assumed that there was a possibility that more than one disc could have separated from its MSC The licensee began making radiographs of the MSCVs at midnight, October 7,1990 to minimize the exposure of individuals onsite from the MINAC. Composite sets of radiographs for both MSCVs in the East Steam Valve Vault were compared at 07:00 on October 8,1990. At 07:13, the licensee began shutting down the unit. Once the licensee
__
' * , , . .
.
had conclusively determined MSCV failure they acted in accordance with timeliness requirements in (1) shutting down the unit in accordance with the action statements in TS 3.7.1.5, (2) reporting the initiation of a unit shutdown required by the TSs in 10 CFR 50.72(b)(1)(A), and (3) reporting entry into one of the emergency classes of the Sequoyah Emergency plan in 10 CFR 50.72(a)(ii)(3).
3. FAILURE DESCRIPTION i 3.1 Design Description Each unit has two Main Steam Valve Vaults which are contiguous with but outside of the containment building as shown in Figure 2. Two main steam lines penetrate containment and traverse each vault. The piping diameter in the vaults is 32 inches. At the exterior wall there is a transition in pipe size from 32 inches to 36 inches. Each pipeline in the vaults has a short radius elbow, then an isolation valve (MSIV), and a check valve (MSCV).
The MSCVs in both units are 32-inch, articulated, swing check valves and were manuf actured by Atwood and Morrill. Three of the MSCV bodies in each unit are butt welded to the adjacent MSIV bodies. The fourth MSCV is separated from its MSIV by a pipe spool that is pipe diameters lon The MSCV design is depicted in Figure Its major nternal components are the valve disc, a post or stud which cornects the valve disc to the tee or swing arm, a hinge pin which ex'. ends through the disc arm and both sides of the valve body around vnich the arm rotates, and a disc stop that limits the swing of the disc and is welded to the valve bod The major external components are counterweights that are attached to extension rods that are connected to each end of the hinge pins. Packing glands limit leakage from the valves through the hinge pin bearings. Bonnet plates provide access to the MSCV internal . The MSCV post is machined from an A105-71, carbon-steel forging and has an overall length of 13 inches. The middle part of the post is 3.25 inches in diameter and is approximately 5.8 inches long. The diameter of the ends is 2.5 inches, and both ends are threaded with eight threads per inch. A 1/16-inch, 180-degree, circumferential fillet at each shoulder between the middle part of the post and the threaded ends reduces the stress concentration at this transitio The disc is approximately 3.5 inches thick. The post is screwed into a mating hole in the center of the disc and the lower shoulder is seated against the disc. When seated, the end of the post is in the plane of . the bottom of the dis A 360-degree, locking weld is accommodated by a 3/8 inch chamfer on the end of the post and on the
. _ - __ _ _ _ _ _ _ _ _ _ _ _ - - _ _ _ _ - - _ _ ' ' , -4 .
o
dis When initially assembled by the vendor, no torque was specified for installatior of the pos During assembly, the bore in the arm is slipped over the post onto a bottom washer and a washer and nut are installed on the upper thread of the post. With the washer seated on the upper shoulder of the post, 1/32 inch of clearance is provided between the washer and the upper face of the arm. The nominal diametral clearance between the middle part of the post and the bore in the arm is 40 mil The lower f ace of the arm has a 15 degree circumferential bevel. The clearances and the bevel permit some artuulation of the disc relative to the arm. The maximum motion possible at the periphery of the disc is approximately 1/4 inc This allows fer better valve seatin A ringle disc stop is welded to the interior surface of the downstream side of the valve body. .The disc stop is located in the vertical plane through the axis (longitudinal plane) of the valv With the disc forced against the disc stop, articulation in the longitudinal plane is not possible but may be possible in the-transverse plan .2 Unit 1 Failures Discovered October 1990
'
MSCVs 1-623. -624, and -625 experienced similar failures of the post which connects the check valve tee or swing arm to its disc. In each case the failure consisted of fatigue fracture of the post. For each of the failed posts, a primary and secondary initiation point was found at the surface of the pos These initiation points were diametrically opposite from each other. . When the discs were viewed from the.. downstream side with 12 o' clock marking the top of the disc when seated, then the primary initiation point-was at 10:00 for MSCV 1-623, at 4:00 for MSCV 1-624, and at 3:30 for MSCV 1-62 These primary initiation points correlate with the; orientation of the upstream elbows. The elbow for 1-623 turns .to the right in the downstream direction, while the elbows for MSCVs 1-624 and 625 turn to the lef t when viewed from above:. The turbulent steam flow pattern caused fluttering, clattering and rocking of the discs in a plane 90 degrees from the pivot and support axis of the valv The 13 inch long post is basically a stud threaded at each end with-an unthreaded larger diameter -middle section, One end threads into the disc up'to the shoalder of the larger diameter unthreaded middle section. The middle section passes through and rides in a hole 'in' the swing arm, -and the other threaded section receives a nut and washer to complete connection of the swing arm to the' dis Failure occurred through the post at the corner radius (1/16 inch) I where the section threaded into the disc transitions to the larger l- diameter unthreaded section of the pos This is just below the j ..houlder of the unthreaded section of the oost that seats against the ,'
o *
,, j ,
l
.
12 )
.
disc. In all three instances the fracture surfaces had been exposed to the normal steam environment of the valve for a sufficient period of time to have oxidized - resulting in a black fine-grained surface i coating which made the f atigue striations difficult to distinguis > The fatigue crack was seen to have progressed almost through the !
'
entire cross section of each post prior to the final overload failure. Additional details of the failure are described in Section 6 belo Following separation from their posts, two of the almost 600 l discs were carried thruugh the downstream piping until they finally became wedged where the piping diameter reduced at a location just beneath the steam throttle-valves. (This travel is shown in Figure 3.) There was evidence that these discs were in the MSL mixing tee or header before entering the riser lines to the throttle valve The third disc became lodged in its valve case. None of the three was capable.of performing its safety functio .3 Unit 2 Incipient Failure Discovered October 1990 Examination during the 1990 Unit 2 refueling outage revealed that MSCV 2-624 exhibited cracking similar- to that which led to disc detachment and loss downstream in a 1982 event. Cracks were' not found in other Unit 2 valve posts, .The cracks were observed in the end of'the post threaded into the disc. In 1982.such crackin0 led to failure of the weld connection between the post end and disc face for '
- Unit 1 MSCV 1-623, permitting .the post to unthread and release the'
-disc. Inspections of other MSCVs at the time of 1982 event revealed cracking and incipient failure in valve 1-624, 4. SIGNIFICANCE OF FAILURES
:l 4.1 ~ System Performance p
,
There were no indications in the cor. trol room from the secondary side steam conditions .that any of the discs had separated from the three failed MSCVs. In normal plant cond tions, the MSL . steam flow ' holds the disc and arm in the upper. part- of valve body. With the separation of a disc from its arm, the disc will become lodged in the MSCV ~ or travel down the MSL to become lodged in the mixing tee, the . dump header, or the riser lines just below a turbine throttle valv i The disc is smaller than the inside diameter of the MSLs except .for the riser' lines just below the throttle valves. With the separation of a disc, there would be a drop in resistance to steam flow through the valve but this drop would be negligible compared to the total resistance of the valve and would not result in a measurable change in the steam flow of the af fected MSL ' The arm without the disc , would remain in the upper pr.rt of the MSCV, supported by the steam i flow and the counterweight arm. Therefore, .the separation of the , disc from the arm wouid not have an effect on secondary side [ performance unlesc 6he disc became lodged in the riser line in an j
, ,
. < .. ,
, ' - ,-
!
t
.
, .
I
'
orientation to significantly reduce steam flo The discs found wedged in the riser lines were parallel to the steam flow and, therefore, offered little resistance to the steam flo If a disc
'
did significantly disrupt flow in a riser line, the steam flow would significantly drop in that line and the flow in the three other riser
.
lines would increase because all the riser lines to the four turbine throttle valves come from the common mixing tee. This change in MSL ; steam flow would be seen in the control roo .2 Damage To Piping and Supports Aside from the post failures themselves and the in1bility of the valves to perform their functions as a result of the disc separa-tions, the damage in the 1990. event did not prove very significan > It consisted primarily of dings, dents, and scrapes inside the' piping caused by the discs as they were carried by steam flow. Depths and lengths of the damage were determined through direct inspection by ; licensee personnel (including a metallurgist) who travelled through the pip The maximum damage depth in terms of percent of wall , thickness was 9.4 percent. Most damage indications were oriented 45 degrees or less-to the longitudinal axis of the pipe. No significant indications were oriented transversely,
! ,
Based' on interviews- with the personnel who examined the internal piping damage and observation of damage at the location where the piping was cut to facilitate disc removal, the AIT was satisfied that a the damage was properly measured and recorded- An assessment of the
. -
damage performed by APTECH Engineering Services under contract to TVA' determined that the piping damage indications were acceptable. This assessment, which was reviewed by the AIT, was based on ASME Section . XI. criteria and conservatively treated the damage indications as. if they were crack ; Based on the ASME Section XI evaluation by APTECH and the fact that: the indication depths were less than the +/- 12.5% permitted by the-material specification, the l',censee determined that it was no necessary to blend out or otherwise repair the dents. dings and -
~
.
scrapes. . An exception was at locations where the damage resulted in , protrusions of metal that could (1) interrupt steam flow at the
'
.
surface and lead to erosion or (2) release metal chips into the steam system and damage the turbines. Criteria for identification - and removal of these were established and the unsatisfactory material was removed or blended out. It should be noted that the piping through which the discs travelled is non-safety-relate A check of sample lines and instrumentation within the piping found two nor.-safety-related isckenetic sample tubes that .were severely bent by disc impacts, but there.was no significant damage to the associated . penetration welds. The licensee determined that the function of the sample tubes was unnecessary and that repair was not required.
L ,
1 lic 1
,, t 9 F ..
,
.
A walldown of piping supports by the licensee revealed no damag , The inspection results, which were reviewed by the AIT, were documented in a memorandum from the Lead Civil Engineer to the Operational Support Manager dated October 17, 199 A 6 inch OD by 3 3/8 inch ID washer, formerly used in one of the disc-to-post connections, was carried away by steam following one of the failures and could not be locate This was documented and evaluated in Corrective Action Report SQP901523, which was reviewed by the AI The AIT concurred with the licensee's conclusion that the washer would n:t travel to any location where it might adversely affect safety-related equipment or preclude safe shutdown of = the l plan .3 Safety Significance The event was safety significant because a Main Steam Line Break (MSLB) in ene of three MSLs and the failure of a MSIV to close
,. under rever.e flow conditions would have resulted in an unanalyzed ;
accident. T.so event was an unmonitored, common cause, multiple failure. .The three failures occurred within the first operating cycle after mo,iifying the three MSCVs with no monitoring or inservice testing of the salves during the cycl ! The failures of the three MSCVs would prevent the valves from performing their. function.. The function of the MSCV is to reduce the reverse steam flow through the MSIV on a MSLB in containmen This is to assure the MSIV will.close under reverse steam flow. The MSIV is not designed to close under reverse steam flow conditions and, therefore, with only the MSIV operable, i MSLB between the MSIV'
.
and the SG could result in two SGs blowing Cown into containmen Sequoyah -was not analyzed for this situatio The containment was reviewed and the plant was licensed fo'r only ole SG blowdown for any MSLB. The safety significance of the failed MSCVu is that a two
.SG blowdown could occur for which the plant was not reviewed and ' licensed. This would be outside the licensing basis of the plan TS 3/4.7.1.5 requires the MSIVs to be operable in Modes 1, 2 and There is no TS that specifically mentions the MSCVs. However, with a failed MSCV, the corresponding MSIV must be considered inoperable, ;
since it would not be capable of performing its function under
'
reverse flow conditions. TS 3/4.7.1.5 addresses inoperable MSIV The purpose of the MSCV is discussed in Section 10.3.2.1 of the.FSA For the break in the MSL from one SG inside containment, the FSAR l states that the MSCV provides the redundancy required to allow for a single failure of an active component, the MSIV, to prevent reverse l, flow in the MSL through the break from an un'affected SG.
l l
.' . !
. --.. .
-l i
.
,
i
'
In Section 6.2.1.3.11, the FSAR simply states that the reverse steam flow from the other SGs is prevented by the cher( valves and should these valves f ail, the other SGs can blow down antil the respective MSIVs are close ' I As explained in the sequence of events, the licensee developed a JC0 i of Sequoyah with an inoperable MSCV. The po.ential accident is a two SG blowdown from a MSLB inside containment. Because Sequoyah has only been evaluated for a one SG blowdown inside containment, this new situation would be outside the licensing basis for the units. In the discussion held with the licensee on October 5,1990 at the staff's - headquarters, the NRC staff stated that this JC0 was not sufficient for the staff to conclude that operation with an inoperable MSIV was acceptable. In accordance with 10CFR50.59, the licensee could not operate Sequoyah with an inoperable MSCV without approval of NR The JC0 was reviewed during the inspection. The JC0 discussed the following effects of two SG blowdown: (1) effect on containment pressure analysis, (2) effect on subcompartment pressure analysis, (3) effect on valve vault peak pressure analysis, (4) effect on departure .from nucleate boiling ratio, (5) effect on temperature and ] environmental qualification inside containment, (6) effect on-temperature and environmental qualification in the valve- vault, (7) effect on steam generator tube rupture, and (8) probability of a two: generator blowdown. The basis for the licensee's conclusions in items (1) to (7) were discussed with the licensee. Item (8) was not discussed because it would not have-been considered in deciding if operation with an inoperable MSCV was acceptable. The calculations-for. the basis of the conclusions were not reviewed. -It' was deter-
'
mined during the discussion. that. the' analysis for - the effect of two SG blow down on the containment pressure analysis was not properly documented. The licensee stated that they only had informal > data, not a properly reviewed, signed, and verified formal calcula-tion.- Based on the discussion of the basis for the licensee's conclusions,.the AIT concluded that there was reason to believe that .
'
the - formal plant specific analyses that the staff would require to accept operation of Sequoyah with inoperable MSCVs would- show such
.
.
operation to be safe and acceptable. A more detailed review was not 1 warranted because the staff. had already concluded that the JC0 was not' acceptable and theilicensee had shut dowr Unit 1 after concluding * one MSCV was inoperable, , 5-. EQUIPMENT HISTORY - CHECK VALVE HISTORY 5.1 Maintenance.and Modification History All four (4) of the MSCVs have been plagued with packing leaks at the hinge pin packing gland since initial operation including problems identified during pre-operational (Hot Functional) testing. There have been many Maintenance Requests (MRs) issued to correct these
rec ..
+ < :. . < .. , . !
l'
< , .
packing leaks as well as to correct bonnet leaks by retorquing while l the valve body was hot or by Furmanite repairs if necessary. In 1982
: the disk; for Unit I valve 1-623 was found missing during a valve ' inspection. Additionally, the stud post for valve 1-624 was found :
to be cracked. The f ailures were attributed to high-cycle f atigue ! which cracked the stud post-to-disk wel The fix was to torque
(preload) the stud post-to-disc threads and reweld the post to the '
disc. Also, the nuts were tack welded on to The following is a list of-. the major maintenance, modification and inspection work that has been performed on the MSCVs since discovery > of the-valve failure in 198 o a VALVE ~1-626 Dat Work performed 4< _ 11/82 Based on the 1982 failure of valve 1-623 perform' the following:
-
Inspection of valve internals
- LPT of disc post -
UT of. post from seal weld end 10/85' Continue valve monitoring based on 1982 failure:
-
Inspect valve internals
-
LPT end of stud
-
UT of post.both ends-8/86- . Modify counter balance arms-4/90 Inspect valve due to problem note on sample valve
> , inspected for PM05014-1-002- tack weld nut.
. Install EPRI packing (Mod) l l VALVE 1-625 11/82 Based on the 1982 failure of valve 1-623 perform' the
- <
following:
-
Inspect valve internals
-
UT Post
-
PT Post i
* ' , . ,
, .
12/83 Continue with valve monitoring based on the 1982 failures of valve 1-623 by performing the following:
-
Inspect valve internals
-
LPT Post
-
UT Post 4/90 Inspect valve due to problems noted on sample valve inspected for PM05019-1-002, Due to excessive wear perform the following modifications:
- Repair weld post and arm with harder material (Mod) -
Pin nut (Mod)
-
Eliminate lower washer (Mod) Add set screws to pivot arm (Mod) Install EPRI packing (Mod) VALVE 1-624 11/82 Based on the 1982 f ailure of valve 1-623 the following internal inspection resulted in:
-
Determining that the old stud post / disc weld was cracked
-
Making of a new replacement stud post
-
Making of a new replacement washer
-
Installing and torquing replacement post to 2800ft-lbs
-
Rewelding end of post to disc to prevent unthreading 12/83 As a followup to the f ailures noted in 1982 perform the following:
-
Inspect internals
-
LPT stud post weld
-
UT stud post i
f'(,*,,
- i'
18~
~ o
.
k , 4/90 Inspect 1 valve internals due to problems noted on sample-valve that was inspected per PM05014-1-002
, ,
Due to unexpected wear: C - Repair' weld post and arm with harder material (Mod)' ol - Pin' nut (Mod) Add set screws to pivot arm.(Mod)
' -
- '
Is a_,. Install EPRI . Packing (Mod) , u., VALVE 1-623 a l11/82 ,Due <to visual indication > (i .e. , counter balance larm
, position) perform.the following: , ' - i- Inspect valve.internais- ,
Inspection resulted in the following:
--
Valve disc. missing
* - ' Seat. required repair -
Making ~of a new~ stud' post-
-^ ' Replace' cracked stud post and torque to'2800 ft-lbs , -
Weld end of' post to disc.to: prevent unthreading-
, , -
Reinstall disc .. g ,
-
UT'Stuu Post
- 'PTsStud'dost. weld -
Make new washer
'12/83~ As a followup to the : 1982 valve failure perform . the 'following: ~ -
Inspect internals
-
UT Stud Post
-
LPT Stud Post Weld
< >
$ 1
c >
..
L .: i -* f !
.
l 19 j L: j' , 10/85 As a continued followup to the 1982 valve failure perform l the following: '
;
g
- Inspect internals
.
:
'
-
UT Stud Post
-
LPT Stud Post Weld 11/85 Replace cracked weld on disc post nut 3/90 Inspect valve internals per PM05014-1-002 4/90 Due to unexpected wear perform the following:
-
Repair weld post and arm with harder material (Mod)
-
Pin Nut (Mod)
-
Add set screws to pivot arm (Mod)
-
Install EPRI packing (Mod) 5.2 Examinations and Tests History i In' LER 82-126, Re , the- licensee : documented the 1982 MSCV failures and committed to inspect at Itast 2 valves during the next outage to determine the effectiveness if the prescribed corrective actions.forLthe above problem. The valves were checked'several-times over the' subsequent 7 years. However,; the inspections' generally did not look for problems other than the previously identified-failures, only stud post weld' integrity and general valve condition was
-
, = verified and this was accomplished without internal disassembly or-dimensional . checks. These inspections were discontinued prior to the-last refueling outag In.1989 PM 05014-1-002 was developed to implement the INPO SOER 86-03-recommendations to improve check valve performance at nuclear power plants. This PM required inspection of the MSCVs on a frequency of
-
one valve each refuel outage. For the Unit 1. Cycle 4 outage, valve-1-623 was chosen for inspection. The engineer who perforne l the PM described the inspection as standing on the disc and rocking back and= forth (i.e., the-teeter-totter test). The PM did not LPT or UT the stud post and weld as in past outage examinations of valve internal Additionally, the PM lacked specific acceptance criteria and did not require the internals to be disassembled or dimensionally inspecte _
_ _ _ _ _ _ _ .. .
, , .
t
'
i t 20 The PM for valve 1-623 failed due to missing washers, so the other ; three Unit I valves were inspected. Excessive wear was noted on the l post and arm of all except the valve 1-626. The licensee modified the pnst and swing arm by weld buildup (harder and stronger 8018-C3 > material) on 3 of the 4 valves (1-626 was not repaired). ; Additionally, the post nuts were pinned on the three valves. Hinge pin set screws and new packing was installed on all 4 valves with the ; post nut tack welded on valve 1-62 i In September 1990, the 1-623 and 1-625 valve discs separated and were
. carried down the steam header. The valve 1-624 also failed but its disk lodged into the valve body. Valve 1-626 remained intac l The AIT questioned the licensee as to whether the MSCVs were included in the ASME Section XI,~ Article IWV, valve inservice testing progra i The licensee stated that the valves were originally in the program but were removed in 1982 revision of the program. The licensee indicated. that their interpretation of Regul6 tory Guide (RG) 1.26; ;
" Quality Group Classifications and Standards for Water, Steam, and t
"
Radioactive Waste Containing Components of Nuclear Power Plants", permitted dropping the valves from the IWV program. They considered - that Section C.I.d of RG 1.26 allowed the nuclear quality class of , components to be stopped at the MSIV, even though the MSIV cannot + perform its safety function of isolation during a break of the upstream piping without closure of the MSCV. Generic Letter 89-04, -
" Guidance- on Developing Acceptable Inservice Testing Program" indicates-in Attachment 1, Part 3, that the main steam non-return val,es (i.e., the check valve) are ASME code class . valves that r should be in the IWV program but may not always be back flow teste Additionally, NRC Information Notice No. 88-70, " Check Valve Inservice Testing Program Deficiencies", pointed out the fact that the IWV program was to ensure that all safety-related check valves i should be included in the program ano that testing is to prove that the valve can perform its safety functio In addition to the above bases for including the MSCVs and connecting piping in the inservice testing and inservice inspection programs, TS 3.7.1.5 and its bases require the MSIVs to be tested to the require-
'
ments of Section XI. Since the MSCVs are needed for the MSIVs to perform their safety function they too should be teste ' 5.3 Vendor Recommended Modifications The licensee was the architect-engineer for the Sequoyah plant and the Main Steam System was included in the architect-engineer's design scope. As architect-engineer, the licensee was responsible for the , location of the MSCVs relative to other components in the syste Problems with the stability of the valves resulting in part from the location of the valves, contributed to failures of the valve Subsequent to purchasing the MSCVs, the licensee maintained their interface with the vendor, Atwood and Morrill (A&M), t (
;
- - _ _ _ _ ___
' *
,
,
,
.
The AIT reviewed the licensee's correspondence from A&M and found that it generally related to proposed changes to the MSCVs to reduce
'
valve chattering and improve the capability of the valves to withstand the severe operating conditions being experience In several instances, A&M proposed lengthening the disc stops to hold the discs lower in the steam flow. This changt was never fully evaluated or implemented by the license One of the letters reviewed involved the May 1990 modification that is considered to have been a principal cause of the three subsequent 1990 Unit 1 MSCV failures. To repair post and arm wear, A&M recommended that t.he licensee machine 1/8 inch of material off the diameters of the posts and swing arm bores, replace the material with a weld overlay of harder material, and machine the posts and bores to their original dimensions and tolerances. In addition, A&M advised the I'censee to add a piece to each of the discs which would in effect lengthen the disc stops. The basis for the recommendations for lengthening the disc stops was not give Contrary to A&M's written recommendations, the licensee's Design Change Notice (DCN) M-03?L1-A, approved for issuance on April 17, 1990, as indicated on p.ge 7 of the DCN, included machining a minimum of 1/4 inch of material off the diameters of the posts and the disc arm bores, in accordarce with A&M's advice, the DCN required the use of a harder weld over'ay material and machining the posts and bores to their original Jimensions and tolerances af ter weldin The , licensee machined 1/2 inch from the diameter of the posts (i.e., 1/4 iach from the radius). Removing the additional material resulted in weld material being placed closer to the notch in the post, as shown l in Figure 4 with attendant. ef fects caused by welding and welding hea The licensee stated that the posts were not annealed after i welding and that the consequences of welding on the internal stress distribution and the condition of the notch were not analyzed by them. They also stated that they were not aware of A&M's bases for their recommendation Further, no analysis was found which addressed the effects of reducing the diameter of the post by 1/2 , inch instead of 1/8 inch and installing weld overlay, j In a letter dated. April 26, 1990 A&M provided the licensee with the ) dimensions to be used for the upper and lower washers on the post (page 30 of DCN M-03281-A). The washers had been severely damaged by fluttering of the valve disc ! The AIT found that in some instances, design control was inadequate and informal. For example, A&M reportedly did concur in the licensee's removal of 1/2 inch of material f rom the disc post; however, that concurrence was oral. Further, neither the licensee nor A&M analyzed the affect of weld overlay on the post or the stress distribution in the post.
- - _ - _ - _ - _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ - _ - _ _ _ _ _ _ _ _ - - - - _ - _ _ _ _ _ _ - _ _ _ _ _ _ _ -
_ _ . . . ___
' ,, * , . -
.
'
.
s FAILURE EVALUATION _ 6.1 Metallurgical Examination An evaluation of metal'1urgical factors in the 1990 Unit I f ailures did not reveal any chemical or physical characteristics of the f ailed - valve posts which might account for their (unexpectedly premature)
'
f ailure except for - the implied presence of high welding-induced residual ,tresse>. Each post was originally specified as ASTM A105 carbon stee ' - 1.
- The failed post in valve 1-625 was of material originally purchased, machined and installed by the manufacturer. .The
~ .. !E posts in valves 1-623 and 1-624 were posts replaced following '- the 1982 failure and had been purchased, machined and installed ? by the licensee. Chemical analyses on the failed post materials were within the limits specified by ASTM A105 except that the
{ posts installed by the licensee-co, tained excessive manganese (1.43 versus 1.05 maximum spec. Tied by ASTM A105). ' The licensee - had previously evaluated ano accepted this discrepancy. The AIT concurred with a licensee conclusion that the high manganese in
-
these posts played no significant part in the post failure . Microstructural examinations .of valves 1-624 and -625 post materials - revealed dif forences in material processing between .
- -
the vendor and licensee installed posts, but no metallurgical abnormalities. The -weld buildup applied to the_ OD of the posts
-
in the May 1990 modification was readily discerne The weld and its heat =affected zone appeared about 1/4 inch thick and contained no significant defects. The post fracture initiation was located at least 1/8 inch away' from t5e heat affected 7 zone of the weld. The. post shoulder surfact.that should seat
-
squarely " against the disc af ter torquing ir,to the disc was observed to be tilted away from the disc surface toward the 0 This trey have been due to residual tensile stresses at the post OD from weldin .
- ' Microhardness traverses through weld, heat af fected zone, and
_ base material yielded results consistent with the expected physical properties of the material ' Similar fracture locations and fracture surf ace conditions were observed for all three Unit 1 posts. ' Although f atigue beach marks present were difficult to discern . because of oxide _ buildup, the fractures 6ppeared characteristic of high cycle fatigu The progression of beach marks observed in a detailed examination of one fracture surface indicated crack initiation- _
"
at two opposite points on the OD surf ace with propagation through reverse bending load __
-
, . . . .. . . . . . .
. . . . _ _ .
. . . . . . ... _ ..... _ _ __ _ _ _ _ ... ._,.... _ _____
__ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . . _
'3 . . , .
.
Post cracking observed on the disc post end of Unit 2 MSCV 2-624 was not metallurgically evaluated prior to completion of the NRC Augmented Inspectio It appeared similar to MSCV post cracking that occurred and permitted separation and loss of a disc from Unit 1 MSCV 1-623 in 1982. The Unit 2 posts were original vendor installa-tions that had been assemLled without any specified requirement for torquing the post into the dis It is considered an incipient failur See 6.2 and 6.4 below for further discussio .2 Nondestructive' Examination
.
During October 1990, visual examination revealed cracks in the disc end of the post in Unit 2 MSCV 2-624. This cracking occurred near the opposite face of the di sc f rom the Unit 1 fractures (see Figure 1). Penetrant examinations of the remaining Unit 2 and the Unit 1 disc post ends detected no other crack The MSCV 2-624 cracking had propagated into the disc materia Based on a review of the licensee's report of 1982 post failures which occurred at the disc. end and discussions with licensee metallurgists, the AIT considers that the valve 2-624 end cracks represent an incipient post-to-disc connection failure. - As with the 1982 Unit 1 failures, the valve.2-624 failure occurred in an unmodified assembly installed during original manufactur .3 Wear Examinations Wear patterns resulting from swing arm / disc, post / arm hole, swing arm / disc stops and -dist/back .stop contact indicate eccentric deflection and non-uniform loading which likely contributed to initiation and- propagation of fatigue cracking it the post Measurements on the MSCV.2-623 post in the-current outage indicatec some small permanent bending of'the post by the flow 'nduced forces, This post exhibited severe wear (up to about 3/16 metal removal in some locations). The forces experienced in this valve were observed to have caused the nut washer to form around the nut and an impression of the nut was forged into the swing arm face at least 1/2 inch deep - and this was a valve that'did not experience post crack With the exception of the _MSCVs 1-626 and 2-626, all valves experienced severe wear between post and swing arm ID. Reduced wear at the -626 valves appears due' to the -reduction in turbulence which results because -it is further downstream from the MSIVs than the other MSCVs (2 1/2 pipe diameters).
6.4. Comparison with 1982 Failures i Unit 1 MSCVs 1-623 and 1-624 were f ound to have post cracking in November 1982 af ter only about two years operatio In the case of the 623 valve the cracking had resulted in complete separation of the disc f rom the post and the disc had been carried through the l i i in siiini m--iii-umiimumme-s- - - - e
._m-i-_ i__m
_ . . . . , , . -, .
. - ? .
.
downstream piping by steam flow, in the case of the 624 fracture the post-disc connection f ailure' was incomplete and it was considered an incipient failure. The licensae's analysis of the failure identified the failure mechanism as having been fatigue fracture which initiated and propagated in the post materici of the two MSCVs near the disc-to post end weld. In one instence the cracking had progressed sufficiently to completely f ail tne end connection, permitting the post to unthread and separate from the disc. The f atigue f racture was considered to have originated due to (1) external disc counterweights having moved out of position, resulting in increased cyclic loading of the disc onto and off the stop; (2) lack of preload on the post, permitting the operating forces to be transferred to the-disc / post end weld area rather than the post shoulder and upper threads; and, as a secondary factor, (3) degradation of the weld heat affected zone properties by tin contamination (0.15 percent) discovered in the pin material. Licensee metallurgical personnel and the AIT agreed that the 1990 Unit 2 MSCV 2-624 disc post end cracking appeared identical to the post end cracking experienced in Unit 1 MSCVs 1-623 and 1-624 in 198 The AIT considers it doubtful that tin contamination in the welds-played any part in the failures. A comparison of significant factors in the 1990 failures at the post shoulder radius with the failures experienced in 1982 -(and 1990) at post ends (disc side) is given below,. Similarities
-
Fatigue failures
-
Occurred in post
-
Lack of preload considered a significant contributor-to failures
-
No cracking was experie 'ced in the -626 valves (probably due to greater separation. distance from MSIV resulting in less severe turbulence) Differences
-
Fracture ac different post locations
-
Weld buildups on 1990 posts with shoulder radius failures but no weld buildups for failures that occurred at post end
-
Counterweight movement to improper location for 1982 failures (but not for similar 1990 #ailure)
,
.e ..
.
,> ..
, 3 i
.i *
r i 6.5 System Design Effects !
'At full power, the- steam in the main steam system piping in the '
valve vaults (See Figure 2) is flowing at 113 feet per second, the Reynolds; number -is 1.arge, and the flow is turbulent. The elbows - , disrupt and skew the symmetry of the -velocity distributio l According ;to EPRI Report NP-5479, symmetry of the flow distribution ; is reestablished in about 5 pipe diameters. -The velocity. distribu- -) tion at the' entrance to the MSIVs is not fully symmetric : and is +
- further disrupted by the MSIV internal components, the - shape of- t - the valve body, and the orientation of the seat. According to EPRI *
NP-5479, symmetry is reestablished in about 10 pipe diameters af ter . flow passes through a control valve. Although the MSIVs.are not flow ! control 1 valves, the internal configuration of the valves does. disrupt' l the-flow distribution downstream from the valve i , The- MSCV disc =is articulated and has a single disc stop located in
- the vertical.- (longitudinal) plane through the axis of.- the ~ pipe and- l valve body. Asymmetric, turbulent flow impinging on. the articulated s tMSIV; discs ' probably. causes the disc to oscillate :in two modes, with * - one mode being . in the : longitudinal plane and the other in the- ?
l transverse plane, Guidance from EPRI indicates that extending the j
= discYstop: might , be ef fective in= eliminating oscillations in .the longitudinal plan ] '
As shown- by licensee examinations of the Unit 1 ~ and ' 2 valve internals;twear increases the clearances between the post and bore in- ;
.the: disc : arm' and between the downstream surf ace of. the disc and--the ;
upstream' surface of the disc arm. As the clearances increase,c the
'
-
,
amplitude of oscillation, andtbending stresses in the -post increas ; 16.6 ' LMockup cTes.t L
'
j Th.e licensee prepared a strain' gage instrumented . mockup of? a
' post /disciassembly to simulate the May, 1990 welding modifications 3 and assess the stresses caused by the-welding. Components used in
'
the mockup;were a disc from valve 1-624 and a previously unrepaired-oostH f romm italve 2-626. 'The strain gages were located in three
-
- Dongitudinal! machined groves equally spaced around the; circumference ;
of the post 'at a depth 1/16 in below the root of ^ the disc ' end! F
thread The strain gages were . used to measure the - stresses obtained in simulated: assembly steps. One failed prior to'providing any results.'
:
!
- _ e 1)
, . - . .
.26 \
.
Details of the mockup along with results obtained with the two strain gages are as follows: Two of the three failed Unit 1 posts had been torqued into the discs (in their 1982 replacement by the licensee) at 2800 ; ft.lb The mockup post was torqued- to this value and the ! strain gages indicated a resultant average preload tensile l stress of about 14 ksi, j
:! The post end was elded in place af ter torquing simulating j previous assembly. The average tensile stress measured was ;
reduced to 11.4 ksi by the welding. That is, welding caused the ! stress to decreas . Several layers of weld buildup were added to the machined OD of the post using welders, weld procedures and sequences, and ! welding: material (8018-C3) simulating the previous- I modification / repair in-so-f ar as could be determined. The first layer of weld deposit resulted in residual bending _ stresses as indicated by differences between the stresses measured at each . of the two operable gages. One indicated +1.4 ksi and the other l
+14.5 ksi. .In its final welded and machined condition, !
-simulating: the previous return-to-service conditions of the modified Unit 1 MSCVs, both strain gages on 'the mockup indicated-l
,
compraive strese For one the compressive stress measured i was -20.5 ksi and tor the other it was -23.0 ksi. Correcting- [ the stress values .by an extrapolation to account for their _ { 1/16-in. depth below .the thread root depth, the licensee t estimated very low, possibly ' compressive stresses at the corrar f radius. This would suggest little affect of residual welding l stress on crack initiation and failure. However, the licensee' j metallurgist informed the AIT he believed ~that an actual tensile- i bending stress r.iight exist on the side of: the post where the i strain gagt had failed in setup of the mocku j
The post / disc weld was cut following the above measurement and it was l found that the post could be rotated out of the' disc by hand - little or none of the original preload torque still existed, The benefits of preloading had been negated by the weldin ; i 6.7 Summary.of Phy.dcal Causes
]
The AIT concluded that the physical cause of the 1990 Unit 1 MSCV failures, . based on the above information, was weaknesses in 'the j licensee's original (as the architect-engineer) design exacerbated by 8 the May 1990 modification. The original design, utilizing a : swing check valve located-in close proximity to the MSIV, appears to result u
'
in high non-uniform fluctuating stresses, as evidenced by wear noted on the discs, washers, et {
.
1 B>..o u: -. 7
-
, .sa
&
W,'
.
e
,
27 , l
:The AIT found' that ' the May 1990 modification played an important' -i though not' fully understood role in the failures. There is evidence that welding reduced any pre existing preload resulting in concen- ,
'
tration of bending stresses. The welding may also have contributed residual tensile stresses which, added .to. the normal- operatin :
- stresses, may have initiated the fatigue .. failures. The welding may - , - have also distorted the post shoulder resulting in a less rigid join The- absence of welding stresses in -the = shoulder radius is . believed to- account for' the 1982 Unit 1 -and 1990 Unit 2 failures n .that occurred at the post ends.(disc side) rather _ than in the ,
shoulder radius. These latter f ailures apparently suffered' from a ^ lack of preload due to-the failure of.the manufacturer to originally specify an , 1 ROOT CAUSE-7. 1 General TE ,
.The AIT determined .that the root cause of the 1990 MSCV-failures was-
'
;
inadequate- engineering evaluation and correction ' of deficiencies in [
-
the design = of the~ valves and associated piping- arrangement. .I n 1
~s particular, theLthree Unit 1 MSCV failures were due to the= licensee's- !? inadequate analysis of a design -modification-implemented with vendor 1 ,
l concurrence in May 1990. The. licensee failed to adequately consider ;
'theLeffectsDcf. welding stresses producedo in' implementation of : the design change. ; As a contributing- factor,- the licensee appeared to '!
T have oplaced too: much reliance on the manufacturer's concurrence .
<- with the modification process without> recognizing the-limited bases E fort the concurrence. The: licensee's ' responsibility. (as architect ,
"
- engineer)1to assure its adequacy lwas'not prop _erly performed,
The Unit 2 < incipient failure was on na . valve (2-624)L that had been-determined -not. to require modif.ication to prevent failures of uthe .; type that occurred-in 1982. The 1990 failure demonstrates the error, ;
.
of.this determinatio ,
' - Other Contributing' Factors j In 1986, the Institute for Nuclear Power Operations (INPO) issued to j their1 members.SOER 86-03, which addresses failures of check valvesLin l , ,
main steam and other systems and recommends actions to reduce the- 1 number of failures. ~In January 1988, the Electric Power LResearch :i Institute (EPRI)~ issued- EPRI NP-5479, a report titled " Application
. Guidelines for Check Valves in Nuclear Power Plants". EPRI's Eintent
-
in issuing-the report was to assist utilities in developing program in.. response to SOER 86- Both documents indicate that' instability 4 of check valves can be caused by installing a valve that is too large i and has . insufficient flow ~ through it to force the disc against the i disc stop.with enough force to prevent flutter. SOER 86-3 indicates that by installing a valve that is too small and has excessively
i
-
.
a za ( :. .'.
* ..
'
.
a turbulent flow may cause the disc to flutte The licensee stated-that they'have not applied the EPRI equation because of doubts about its validit Their doubt is based on work done by the Nuclear * Industry Check Valve Group at Utah State University. In a licensee trip . report dated March -14,1990, by R. Simmons, TVA, states that testing indicates that the EPRI equation give resuits that are 200 to - 400 percent low and that researchers ~at the Utah Water Research - Laboratory have developed a more accurate equatio ER 86-3 recommended that utilities determine whether check valves are properly : sized . and provided some guidance on how that can be , ' done. Additional guidance was provided by EPRI in NP-5479. EPRI & suggests that utilities determine the minimum flow by multiplying-a constant obtained from the venior for the specific valve model and-size by the square root of the toecific volume of the fluid. If the-vendor cannot supply this infornation, then EPRI suggests that an equation set forth in NP-5479 be ; sed to calculate the minimum flo That equat' ion relates minimum flow to the weight of the disc assembly, fluid density, disc aret , and the impingement: angle. EPRI 1
. .NP-5479 also indicates that instability can result from installing the check' valve .less;than ten pipo diameters ' downstream from a flow =
control valve or less than five pipe diameters downstream from an elbow because.of the disturbance that is caused by those component J
.EPRI 'NP-5479 recommended that utilities identify check . valves tha are less than ten pipe diameters downstream from control valves or-less'than five pipe diameters downstream from elbows ~ and ' determine whether internal components in these check valves are degrading. If o the check valves are degrading, EPRI; recommended that utilities j
,
evaluate: design modifications, such as lengthening:the disc stop,L to eliminate disc oscillations, or.-if this is not possible,, evaluate the feasibility' of relocating the valve or modifying the piping to-provide 10 pipe diameters of straight pipe upstream from the check , valve. If'neither of these options are feasible, EPRI recommended . , that:an inspection program be established to measure degradation of
.the valves so that the valves can be repaired before they fail in ' servic Prior to the last Unit I refueling outage, .he licensee had been -
aware that. the' MSCV discs -were unstable and that f ailures could t result. The licensee considered -the opticas of -lengthening the t disc stop or relocating the' MSCVs and reje:ted both in favor of i establishing an inspection program as recomn.*nded by EPRI. Their intent was to open one MSCV each- refueling outage _ for inspectio j , When an MSIV was opened- during that outage, serious degradation of - _ the surface of the post, bore 'of the disc arm, and washers .:: found, j Because of this discovery, the licensee opened-the .emaining MSCVs
'and found similar damage to three of four MSCVs for each unit. The damage-correlated to the proximity of the MSIV :
i
- - - . . .s . . *- . , .
<
s Apparently because the damage occurred over three fuel cycles, the licensee decided to face the damaged surfaces of the post and bore of the swing arm- with harder material rather than eliminate the disc stability proble This decision contributed to the recent f ailure of the MSCV . CORRECTIVE ACTICNS 8.1 Main Steam tystem Design The main steim system has not beea modifie .2 MSCV Design TVA has removed and replaced the posts in all of the MSCVs for both unit The new posts are identical to the original posts except , that the 1/16 inch radius (See Figure 4) has been replaced with a 3/16 inch radius at the lower shoulder. The purpose of this modification is to reduce the stress riser at the fille During assembly of the post and disc, the licensee used 2400 foot pounds of torque to preload the connection instead of the 2800 foot pounds used previously when installing replacement posts for valves 1-623 and -624 following their 1982 f ailures. Until now, no posts other than for 1-623 and -624 had received any specified torquin The purpose of this modification was to reduce the-inte.nal cycling stresses at the fille In addition, the. licensee installed two additional disc stops in aach MSCV. The additional stops are on either side of the existing stop and were machined to- the same level as the existing ~ stop. Although
-
the additional stops will not prevent oscillation in- the vertical ~
. longitudinal plane, they may, to some extent, limit oscillation in the transverse plan Unit I has been restarted and returned to full powe Audible chattering can be heard and may be somewhat louder-as a result of installing the additional stop The licensee is attempting to reduce chatter by adjusting . the counterweights. The licensee-had declined to lengthen the stops, as suggested by EPRI, because of the uncertainty in the accuracy of guidance for determining the minimum flow isquired to hold the disc ; gainst disc stop without chatte The same corrective actions are planned for both unit .3 MSCV Monitoring cnd Surveillance The licensee has stated that they will acoustically monitor the MSCVs on a periodic basis during operation to tide in prompt identification of any significant degradation or failure. Visual examination of a
_
. .. , . r-O
, valve or valves is planned next refueling outage with valve selection to be besed on acoustic monitoring results. Detailed criteria for acoustic monitoring have not been establishe , Similar monitoring is planned for Unit 2 upon return to servic In addition to the periodic acoustic monitoring and refueling outage examination described above, the licensee plans to establish baseline i or reference radiography with their M!MAC nn all of the MSCVs. The baseline radiography will be used to aid in veMfying disc presence should it be necessary during operation. The use cf radiography for observing disc position in other valves during operation can only be determined on a case-by-case basis. Factors that must' be considered include metal thicknesses to be penetrated, availability of suitable radiographic machine or source, accessibility to the valve, ability ) to ohtsin baseline radiographs, presence of moisture in the valve, potential for personnel exposure, disc oscillation, degree.of detail needed, etc., For the MSCVs _ the practicality of using MINAC radio-
. graphy to determine' disc integrity has been establishe . FINDINGS OF FACT The ' three 1990 - Unit 1 MSCV failures resulted ' from f atigue- f racture 4 of the. post that connects the disc to the -swing arm. The failures occurred in service less than four months after performance of a modification (to only these three) involving welding overlay to . correct and. reduce wear of the post and ar Examinations of the posts following failurefindicated the' weld buildup was over 1/4 inch thick and covered the unthreaded middle portion of the post (about 6 inches-long). The resultant weld heat -affected zone came within about 1/8 inch of the failure locatio The buildup was performed with the post still threaded into the disc and restrained by a disc-to post. end weld on the opposite side of the disc. No post weld -
stress relief was performed and there wasJ no examination of base material adjacent to the weld for cracking. Weld porosity, possibly 4 due to moisture in - the 8018-C3 weld material, had been observed in initial ' welding but had been removed and replaced with sound deposits. Weld preheat of about 205 F was reported to have been use . The potential effects of the stresses-produced by the welding'modifi-cation referred to above were not evaluated by the licensee in their design change analysi The vendor concurred in the. repair method ~ and the licensee _ states that they relied on this concurrence as sufficient basis for acceptance of the change, without ensuring-adequate base . Licensee communication with the vendor was sometimes informal and undocumented. Undocumented telephone calls were reportedly used in i obtaining vendor concurrence in increasing the thickness of the planned weld buildup and in performing the repair with the post still threaded into the dis ,
., , ..
. . ,
k v
4 .' Radiography successfully disclosed the Unit I valve failures, but performance of the radiography was delayed due to needed setup time, unavailability of alternative radiographic machines, and the need to assure against personnel exposure . There were numerous considerations which may have affected the timelinas, of the licensee's actions in determining the MSCV failures. These are enumerated in Section 2.3 abov . In three instances, MSCVs have experienced fatigue cracks in their posts near the post-to-disc wold. Of these, one actually resulted in a disc loss. while the other two (one Unit 1 and one Unit 2) were incipien Two occurred (one complete failure and one incipient) in 1982 after less than one operating cycle and the other (Unit 2) was detected at the . end of the' fourth operating cycle in 199 The post-to-disc assemblies on all three were as installed by the vendo 'The licensee's analysis attributed the 1982 failures principally to counterweights having moved out of position and absence of preloa The vendor's original assembly requirements had not specifie preload. In discussions with the licensee following the 1982 failures, the vendor agreed that preloading should be performed to 4 avoid further failures. The. licensee performed preload torquing on i the failure replacement posts onl Modifications were made to' prevent counterweight movements on all MSCVs and, therefore, this is not a factor in the 1990 failure The licensee demonstrated that preload was eliminated by weld repairs on the two replacement post . Altogether, six failures have been experienced in MSCVs. :All have occurred through c acking of the disc posts at either the . shoulder radius where the pos' is-threaded into and bears against the disc (three-failures) or at the welded. connection between the. disc and-
. post end that is used to prevent-unthreadin . The -626 valves in both units have experienced no cracking and'
significantly less wear than the other MSCVs. This appears to be due to their location. The other MSCVs are welded directly to their upstream MSIVs while -626s are separated from their MSIVs by a spool piece about 2 1/2 pipe diameters in length, i The MSCVs are not periodically tested and are not designed for suc They are not included in the ASME Section -XI Inservice Testing-Program. Information Notice '88-70 and Generic Letter 89-04 indicate- ( that the MSCVs should be included in an ASME Section XI Valve Testing Progra . The MSCVs, due to their location, are subjected to rough service (i .e. , ' oscillation and chattering) which has caused excessive wear and damage to the disc post'and swing ar __
._ .
. . . . . . . . . . . . . . .- . . . .. . --
, , ...
,,. . +
..
.
1 The Preventive Maintenance Procedure developed to implement INP0 SOER 86-03 did not provide sufficient acceptance criteria for evaluating incipient valve failures of the type previously experience Additionally, the PM program did not consider different service of the valve when establishing the inspection grouping or frequency of inspection (e.g., one valve receives significantly less wear than the other three).
1 Several vender recommendations associated with improving MSCVs have not been implemented by the licensee. An example'is extending the disc stop to force the disc further down in the steam flo . The licensee has installed or plans to install new posts in all eight Unit.1 and 2 MSCVs. The new posts are of similar design and installation to the original posts which operated for several cycles without failur . The discs in the MSCVs for both units (including the -626 valves) oscillate and impact the disc stops when the reactors are operating
'at full powe , Although noise was an indicator of complete . failure (i .e. , disc separation) for one MSCV, three others occurred without any apparent ,
indication or warning to tFe license . CONCLUSIONS The detailed Sequence of Events developed by the licensee ' generally agreed with that developed by the. AIT. However, several key plant personnel were not interviewed by the licensee. -Additionally, information provided to the AIT by several personnel was more graphic than that which they provided in writing to their own managemen This communications gap may have delayed corrective action . The disc failures to date have been caused by a combination of poor valve design for - the intended application, poor system design (location of the check valves) and improper design control and performance monitorin . The lack of periodic testing, ineffectise preventive maintenance, and the f ailure to include the MSCVs in th9 ASME Section XI inservice testing program may have. contributed to the failure . Post failures similar to those experienced in 1982, due to cracked weld as a result of unacceptable preload of the threads, may continue to occur for other A&M valves that have not been modifie , The licensee aggressively worked to conclusively prove if a disc had separated from its MSC O
!
hm
-. s, ,
.,-- , , .
33 Except for . one SOS, licensee personnel showed that they currently understood the safety significance of the MSCV However, the licensee's failure to include the MSCVs in their inservice testing program _ indicates a past misunderstanding of the safety _ function of these valve . Once the licensee knew a MSCV was inoperable, they were timely in deciding to shutdown the unit and in reporting this to the NRC i accordance with the TSs and.10CFR50,7 . .There were no indications from the secondary side steam conditions that any of the discs had separated from the three failed MSCVs and none_would normally be expecte . The failure of a MSCV is significant because a MSLB in containment and the failure of a MSCV to close under reverse flow with the single failure of an MSIV would result in an unanalyzed acciden . The principal physical contributors to the 1990 Unit 1 multiple valee failure are considered to be the loss-of preload and the addition of , tensile stresses which occurred as a result of the May 1990 modificatic ' 1 Radiography mey be an acceptable means for establishing the position and condition of valve internals but- its use must be assessed on a ' case-by-case basis which takes into consideration f actors such as availability of.a suitable radiographic machine or source, presence of moisture in the- valve, accessibility to the valve, degree of detail needed, oscillation of: valve parts, and ability to obtain-baseline radiographs, 1 The licensee's relation with the vendor was too informal and' failed -
-
to establish the adequacy of the bases for vendor. recommendations;and-concurrenc . -As related to-~ these valves, the licensee failed . to adequately
-
recognize and respond to industry experience and. recommendations stemming from SOER 86-03 ~ (and the associated guidelines of EPRI Report 5479), NRC IN 88-70 and NRC Generic' Letter 89-04. In regard to 50ER; 86-03, they did respond to the specific recommendations but not entirely - to the intent (e.g., in taking into account past failures for use in establishing examination requirements), 1 Based on a review and assessment of the licensee's previous operating i experience, the- apparent causes of the MSCV failures, the current .- modificas ans to - the MSCVs, and the planned monitoring of MSCV performance in service, the AIT concluded the MSCVs will provide ; reliable service for at least one operating cycl ' t i
e m s,' y a m ,: , >^ e s .j
:, ;
-
.34 .i
:
* .
-r
- 11', RECOMMENDATIONS
' The licensee should evaluate the- proper counterweight and its position ~on the counterweight arm for the optimun position of the !
, <
disc in the steam flow to eliminate' valve disc chatter, j
-
_ The licensee-should evaluate the optimum prelsading value of the post a in the disc to. minimize the cyclic stresses that may cause fatigue [
, failure of the pos j The 'NRC -staf f should evaluate 'the licensee's monitoring of the disc as to its effectiveness' and to the number of operating cycles this -!
should continu ; The NRC staff s'hould evaluate the licensee's inspections of the'MSCV in--futuref refueling outaget and/or unit shutdowns as to its , effectiveness to determine in..pient failures.similar to the 982 and-1990 MSCV. failure .1
! The NRC staff should generally' evaluateL the relationship between- U g.- licensees and evendors in deciding on' modifications toL equipme_nt ,to
.
V ' determineJ zif . there ' isL adequate- assurance that1 this relationship d y # always requires. proper evaluations of such modification it Y- 6i: Because one - . SOS: : stated t' hat he did - noti underst$nd: the safety ~ isignificance: of the; MSCVs and the training material reviewed lacked
' <
1 ,
* '
specificity, the training of- Sequoyah control room operators should y x 1be: reviewed .concerning the :importance of ;these valve . 12 '. , EXIT; INTERVIEW WITH-LICENSEE MANAGEMENT- J
,
The inspection::scopeY and findings' werensummarized. on! October 19,,1990,. ,
' ,
Lwith those-persons, indicated.inAppendixF1. The L AIT described cthea areas inspectediand ' discussed in, detaillthe? inspection: findings, No dissenting:
' : comments' were received from: the licensee; j
,a .
'
t . , .It
'
l4 '
,
'
.;
513 h ? , a A y
'If
'i
. 4
.
.,.
;j-i i
'
. .
1 > m
' " , .v n, [ . , ,1 ]
y a. - .
,
9 '-
, ,
q
'
(:
*
}
;
APPENDIX 1 e PERSONS CONTACTED Liconsee Employees j . y -E M arels, Mechanical: Engineer'
*S. : 3argerstock, _ Manager, Procurement Engineering R.' Bryan, Nuclear Engineer, Knoxville Office . * '*J. Bynum, Vice President, Nuclear Operations (also Interim Site Director) it *N.'Catron, Manager, Emergency Preparedness Planing 1 Ll Coe, Auxiliary Unit Operator 1 *M.; Cooper Lead Engineer-Balance of Plant, Technical Support j :*M.. Cooper, Site Licensing Manager i "S. Crowe,-Human Performance Engineering Systems Coordinator- -
._W Eberly, Mechanical. Engineer, Knoxville Office E. Freemane Level lIII Radiographer
,J. Gentry, Public Safety
,
t W. Gions, Engineering Specialist + pq ?P. Guthrie, Engineering Specialist
: E. c.Hyden,- Reactor Operator *C, Johnson,:.QAJSpecialist y *S.' Johnson, Communications _ Manager q ' '
J0. Jones, Shift 0utage Manager i
:*N;lKazanas, Vice, President.-Operations Services' 'i W. Lagergrent Operations Manager . '*R,1Lumpkin, QualityJ Assurance Manager 0; Lundy,qLead Civil Engineer '
nB'. M111 holland, Refueling fFloor-. Coordinator 1
% 1T. Nuhay,; Work l Control: Group Shif t Manager R,c.Phillips, Metallurgical.-Engineer '*S f Piercy, Senior Nuclear. Evaluator: " *G.'_Pitzl,_ Field Support. Manager, Coporate Maintenance' 4 *C.?Pratt, Valve Section=,-Corporate Nuclear Maintenance ' . J;fPsoffitt, Compliance Licensing Manager * ' ; ;J.-Qualls, Shift.0perating Supervisor T LJ.fRochelle,l Civil Engineering Specialist *Hi ' Rogers,rProgram Maanager, Technical Support-g'
K.1Seidle' Civil; Engineer, Knoxville Office
,
1*R,'; Thompson, Compliance: Licensing * j , ~* P: Trudel, Site Engineering Manager 7!
'*C'.:Vondra, Plant Manager #
_ FC Weller,OAssistant_ Shif t-Operating Supervisor . ~'
.t ' *W.?Wilburn,~Baiance offPlant' Supervisor, Technical Suppor G; Yeatts, Auxiliary Unit Operator, Turbine-Building 3 , ,7 ~Co. n tractors, * ,
l P. ' LaPointe, Engineering Section Manager,' Atwood and Morrill Company, In I GJ Rao, Failure Analyst, Westinghouse Electric Corporation .j
'S. Shields,cVice President, Atwood and Morrill Company, In m I) '
, ., >. 1
"
. .> :
, ;
Appendix 1 2
.
NRC Personnel
* Gibson, Di 2: tor, Division of Reactor Safety, Region II * Little, Section Chief, Division of Reactor Projects, Region II *P. Harmon, Senior Resident Inspector *S. Shaffer, Resident Inspector * Attended Exit Interview f
l
.
. p .. ..; .-
'
>
. . , "-
..
..
o APPENDIX 2 ACRONYMS AND INITIALISMS AIT Augmented Inspection Team ASME American Society of Mechinical Engineers ASOS Assistant Shift Operating Supervisor ASTM American Society for Testing and Materials AVO Auxiliary Unit Operator CFR Code of Federal Regulations DCN Design Change Notice EPRI Electric Poser Research Institute FSAR Final Safety Analysis Report ID Inside Diameter G INPO Institute for Nuclear Power Operations JC0 Justification for Continued Operation o
"
ksi Thousand Pounds per Square Inch LER- Licensee Event Report LPT ~ Liquid Penetrant Test MCR Main Control Room MSCV Main Steam Check Valve MSIV Main' Steam Isolation Valve MSL Main Steam Line MSLB Main Steam Line Break
:NOUE Notice of Unusual Event
.NRC -Nuclear Regulatory Commission NRR Office of Nuclear Reactor Regulation OD Outside Diameter PER Problem Evaluation Report PM Preventive Maintenance psia Pounds per Square Inch Absolute psig Pounds per Square Inch Gauge QA Quality Assurance RG Regulatory Guide Rx Reactor !
SG Steam Generator-50ER Significant Operating Experience Report SOS Shift Operating Supervisor n TB Turbine Building TS- Technical Specifications UT Ultrasonic Testing WCG Work Control Group , u;3 -- l _ - _ _ _ _ _ _ _ - - _ _ _ _ _ _ _ - _ _ _ _ _ _ _ _ _ _ . _ _ _ _ _ _ ___ _______ ________ ____ - - _-_ _ ___ _ __ - _ -
_ -
.- < s., .
_
- ., . . . ". '
g4 M . M OCWTERWEIGff . IXX X X X Xxxxxxx y
, 7 /
t plKNNNNNNW-
/ / / DISC STOP SE UNIT 1 / DISC ARM FAILUR J/ ' ' %
i
// A SEQUOY H UNIT 2
'
FAILURE p.., /
< $ g / - 1ar
'
-
DISC
-
FOST- -
' (
I
-
_ unum -
=
FIGURE 1. MAIN STEAM CHECK VALV =
- #
i
_ - - - _- - _
,
,
.
_ _ r
"o
.
* ,'
.
' .
.
*;,\
.
,
.
. ..s .:.1 .s
/ ~.1'
- l ii
.- : .-
, .: .: }, .% ! T 1)
r,
.
o
-
w
-
- .
O _
\ % .
Id_ _
-, \_ e 626 4 .. v 624 : .'
625 > 623 %
\':,
~h 7 P P / . .' ?,g, ..s
,
EAST VAULT WEST VAULT FIGURE MAIN STEAM VALVE VAULT .._ _ , .- - - - - - .-- - - - - . - - - - . -- . _ . . _ ~
+s.ae, ---
p_ y
"
_gj s ie d O
.* .. ' < , a .. _,
a n fh-
,,
V v v V \/
^ n n q ex 4 $a M'9 et T
M D- ,b nn
\/ V y a 4e *
w %t I N j, i
= ?
.ti 'k . a L :l
+r
<w
.
~ .
;
?
$$-, ~
=
*
5 si a
*
s ' N 1- a n Y k{ - g
~
-'O O O O O ,
o
'
# ,. W
%
$%
i5 V V V
/\ /\
Q l
* L -J
'>
c
= M -
wg *O kn t i
- I s u b- [' -' ' 'OO \ $ l
N: %. h k " N, kb
<. $s ,
at .
,,
u ' g x C ha61g^
^
-
y
> -
/
,
4 av . . ~ 4 (. O *
*
'
,
4 3 l/4 DIA i I
:p f 1/4 + 4 a
1 / 6 ---I> 4 h ,
'
I/16 - + :: h
,
I/8 > g-U '
)
a
-
,
-
1/8
[ ,
DIMENSIONS ARE IN INCHES
-
2-1/2 x 8 TISEAD - =J @ AaM RECCMAENDED THAT 1/8 INCH OE TAKEN OFF THE DIAMETER.
_ THE DCN REOUIRED THAT I /4_ INCH MINIMLN BE TAKEN OFF THE DI AMENTE h I/2 INCH WAS ACTUALLY TAKEN OFF THE DIAMETER,
=
PREPARATlON OF THE POST FOR VELD OVERLAY,
-
FlGURE a }}