IR 05000206/1990010

Insp Repts 50-206/90-10,50-361/90-10 & 50-362/90-10 on 900226-0427.Violations Noted.Major Areas Inspected:Licensee Performance on Implementing QA Program Re Control Design Changes & Mods for Check Valves

ML20043C131
Person / Time
Site:	San Onofre
Issue date:	05/18/1990
From:	Huey F NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V)
To:
Shared Package
ML20043C128	List: IR 05000206/1990010 ML20043C127 ML20043C130
References
50-206-90-10, 50-361-90-10, 50-362-90-10, NUDOCS 9006040099
Download: ML20043C131 (11)

Text

.- - x

.

__

o

,,q

),

,

e (

,.

.

L.

'

-

_

'

_

[,

V p,;

l:3

,

U. S. NUCLEAR REGULATORY COMMISSION 4

.

,~##

'

'

'

+

q 4

'

REGIONV C' >

. h

'

!

,

,

<

,

-

.

y

L Report Nos. 50-206/90-10,.50-36D90-10and50-362/90-10 i>

(

,

4 3 s

>

-

<

.

k Docket Nos. OPR-13, NPF-10 and NPF-15 i

<

,

.,,

,

,

Licensee: Southern California Edison Company

,'n w

'^

F P. O. Box 800

)

2244 Walnut Grove Avenue Rosemead, California 91770

. Facility Name:- San Onofre Nuclear Generating Stations Units 1, 2, and 3 '

Inspection at:

San Clemente, California Inspection Conducted:

February 26 - April 27, 1990 Inspector:

C. ClarE p m t r Ir,spector

~ Approved by:

A.

FIF/90

~

l-uef,Chie" A

Date~ Signed r

'

Inspection Summary:

. Inspection During the Period February 26 - April 27, 1990 (Report Nos.

50-206/90-10, 50-361/90-10 and 50-362/90-10 w

Areas Inspected:. A routine unannounced inspection by one regional ins)ector of the licensee performance on implementing a QA program relating to tie control of. design changes end modifications for check valves.

Inspection

-

Procedures Nos. 30703 and 37702 were used as guidance for the inspection.

l Results:

s Genera 11 Conclusions and Specific Findings:

}

4-Licensee personnel are still failing to accept responsibility for

-

~'

documenting nonconforming conditions, by initiating an NCR. A repeat

-

violation was issued, since a~similar violation was issued in 1989.

It l

appears that ineffective licensee management corrective action was taken on the previous violation.

~'

It appears the licensee does not have a fomal program to ensure the

>

latest vendor instructions are reviewed and incorporated into' applicable

'

licensee instructions and procedures in a timely manner.

'

The licensee has not established a specific chec_k valve program to address

the latest check. valve concerns in the industry.

' '

.,

,

e

{

-

..

.

,

9006040099 900518

"

C ik POR ADOCK 05000206

-

...

O FDC

<

s

,

e

-

.nz

,

,

_

,..

er/m, t%g s. m4 -k ;

-

.

-

.~

-

4 z -

-

• p 'Wp,

-

.,-

ht? :) ~

,, y p ; q} {..ljj. y f ', -

h

'

,

w,

-

..f,_

<

,.s'

i, 6_J.

' ;4 ','

,,

j=s, As

.

..

r. : s

q- } '

.

,;i

,

- *

,

+.w y,

,

.

.

.

Th

'tf.=

k f,.. U.'. * I,2 [

ye O

-

A, sample.reviewofMaintenanceOrders(M.O.'s)andproceduresidentified:

'

-, -.

5(1). Several documents that did not~ have~ a' tolerance on-torque values.

1;

,

%'

I

...

.

.

.

i

.(2)L Several documents that,did not identify the' latest applicable vendor '

manuals / instructions.

J,

&-

,

Significant Safety Matters: None

j,

..

-

-

-.:

"

' Summary 'of Violation: One violation. (50-361/90-10-04) was identified in-

~

y iParagra)h 2.E; a nonconfoming condition was not ' properly documented on-an!

'

LNCR.; :T11s item was closed in this report.

,,

,

'

.

i Open Items Sumary: -Two followup items were opened.

i i

,

c

.

- a

,

.t f

r s

s

!.-

'I

_{

a

!

>

%

,

.;

'

i sf I

t

.t

'. {

x

,.

.

f (

) i

't,.-

'

(

. _,

_

.hl

t ', -

,

s

1. '

%...

,

d

.g:

I

,

%

[?

[

-(

g,

.

$ 5:n

• ?

y t

.m,.

<#

.

..g f e

g.:

I

'

,

,

s-y

.

{J

~

&

2 g

-!

'

'

'

,. __,j :

t..

h

, s (,. 4

• 'i t,

.,

y,,

'

m;

,

3,-

. ;

.

4 7; v g:.;j M -

'm,-

.

.

O.

,,

f s,s

.y

,

q

<

' 4

'3

1. -

..y ;

t.-

<<'d,

,

,

'

,,4.

,

>

+

(

t'.:

6

,

$

,f f 4*,

-

.

,_}

-

?

-

e

+

s

'

' k.?

-

W

-

.

.

. '.

g

,

!

r w.

s f

~

'

A

.;

-

';f v

t 4..

.Y

'

,

}.Q '

_.

,

.,

,

'

'f

~

.-

x-

-

_

-

.

~

,

'

-

.,.

.

,

)

DETAILS ~

)

,

1..

Persons Contacted

,

Southern California Edison Company l

• H. Mor an. Station Manager f

'*K. S1agle. OMD

-

• L. Cash, Maintenance Manager

• B. Katz, N00 Manager

• J. Patterson,'MT&E Manager, Units 2/3

<

• R. Plappert, Technical Support & Compliance (TS&C) Supervisor)

• C, Brandt, OA

'

• D.Brevig,OnsiteNuclearLicensing(ONL) Supervisor-

-

• M. Speer, ONL i

• G. Gibson, ONL

'

• D.-Werntz, Engineering Representative

• M. Herschthal, STEC

• N. Quigley, STEC

.,'

• A. Molina, STEC

• W. Strom, ISEG

.

• J. Martin, ISEG l'

R. Clark, Station Technical

-

~S. Genschaw,' Mechanical Procedures

!

.

• C Brendel, CDM-San Diego Gas and Electric Company

' *G. Erickson, Site Representative

!

City'of Riverside

_

,

.,

.

,

,

• C. Harris, Site Representative 3e

.,,

,

'

• Denotes those attending the March 23, 1990 exit meeting.-

,

The inspector also held discussions with-other licensee and contractorA

'

personnel during the course of'the inspection..

2.

Design Changes and Modifications Program (37702) ^

-

$

A.

Selection of Inspection Area

'

In selecting areas to inspect; the inspector considered.the' guidance provided in the following documents:

- Available probabilistic Risk Assessinent (PRA) Data, to focus

'

,

' inspection activities into areas of highest potential ^ safety impact.

,

NUREG/CR-4674, ORNL/NO AC-232, Vol. 7, Precursors to Potential

Severe Core Damage Accidents.

'

'

~

• T

.

'

'

.

-

-

.

,

'

0'

.

NUREG-1275, Vol. 2; Operating Experience feedback Report - Air

.,

Systems Problems Check valves in the Safety Injection System in Unit I and the

Emergency Core Cooling System in Units 2 and 3 were initially

,

,

selected for' review during this inspection.

.

.

B._

Recent Historical Check Valves Problems,and Information In response to the Unit 1 November 21,1$85' wa'ter hanner event and the associated failure of six check valves, the~1icensee:took-the

.j

following actions:

-

'

-

,

'

7-

• -

Replaced the six. failed Unit 1 Padifib check valves with,.- n l

'

-

Atwood Morrill check valves.

'

'

-

,

,

Evaluated Units 1, 2 and 3 programs in'the areas of maintenance

\\

'

and design for check valves, with. respect to the reconnendation.

~

of Significant Operating Experience Report (SOER) 86-3, " Check'

'

Valve Failures er Degradation", issued October:16, 1986 by the 4 Institute of Nuclear Power Operations (INPO). The results ofs this licensee evaluation were documented in Independent' Safety,' F

.

'

Evaluation Group (ISEG) Evaluation' Report 87-ISEG-187 dated

April 30, 1987.

,

C.

PresentLicenseeCheckValvelrogramActivities During this inspection the inspector reviewed how the licensee is

',"

addressing check valve concerns in 1990.- A review of available documents and discussions with licensee personnel, identified the following-

,

The licensee has not implemented any new procedures or

'

'

(

programs since 1987, to address the current check valve concerns raised in the industry..The licensee considered.

that their existing valve programs (IST, etc.)-were acceptable, and identified that they were informally following current-check valve concerns.

The licensee is relying on the Inservice Testing (IST) Program

.for detecting safety related check valve degradation.

'

If check valve degradation is detected, the licensee stated -

corrective action would be implemented through the Nonconformance Report (NCR) program for safety related valves andtheSiteProblemReport(SPR)programfornon-safety

,

related valves.

The licensee does not appear to have an effective formal program

for trending check valve failures. During this inspection the.

licensee could not identify to the inspector the number of

<

e

h

rh,

-

,

'

Pf

'

xt +.

check valve failures identified in Units 1, 2 and 3 over the

-

W[-

last tWo years.

,

~r

.

,

<

,

' '

D.

Sample Survey of Check Valve Surveillance Procedures

.

y e

>

4'

'

~

" identified check valves are disassembled and inspected on a sampling

-

,

basis. This inspection work is performed.in accordance with

.'

instructionsissuedinstand-alonemaintenanceorders(M.O.'s),which

,

.

.

'

/ specify work to be accomplished using all or portions of existing licensee procedures. During this inspection the inspector reviewed

-

O sample of check valve M.O.'s, maintenance procedures, and

E

..

,

'

'4

" identified the following concerns:

.

,

(1) Torque Value Discrepancies:

,

' N.O. No. 87103749000, for low pressure Safety Injection p' ump"

,

P016 suction check valve S31204MUO77, identified that work shall-

,-

be accomplished in accordance with the latest revision and

-

temporary change notice (TCN) of maintenance procedure 5023-I-6.19.

Page 4 of the M.0., " work plan inquiry" identified a bonnet nut torque of.275 LB. ft. Attachment 6 to Revision 2 of maintenance procedure 5023-I-6.19 (torque table) identified a bonnet nut torque of 420 lb-ft, this was 145 lb-ft higher than that identified in the M.0.

A review of

- M.0.'s for similar valves in Units 2 and 3, using procedure 5023-I-6.19 as a reference, found the same bonnet nut torque discrepancies.

.

Also, the torque values identified in M.O.s and maintenance

.

procedures did not identify a tolerance.

It is difficult, U

if not impossible, to torque to a specific torque 'value. The

'

licensee acknowledged this, and identified that they were reviewing this item and had already added a tolerance to torque values in some procedures.

The licensee also stated that the subject bonnet nuts should always be torqued to the correct value of the latest revision /TCN of the identified maintenance procedure.

According to the licensee, the craftsman is required to verify at the start of the job, every shift and even after a lunch-break, that he is working with the current maintenance procedure.

If the currer maintenance procedure identifies a different torque value than the M.O., then the bornet nuts would not be torqued until this discrepancy had been resolved.

A review of maintenance procedure S0123-I-1.7, " Maintenance Order-Preparation, Use and Scheduling", Revision 2, TCN 2-6, identified that subsection 6.13.2.2 stated in part:

"The Journeyman shall verify the procedure revision and TCH is current once per shift while the procedure is in use by the methods listed in the procedure." While this

'

instruction did not cover checking after a lunch break, it did require a once per shift check.

'h.

,

-s

--

a

.r 4-t 2'.

u :

i

. >

i. -

,

'I-

'>

.

,.

,

,

,.

.,.

I

,

,.

L.

'

.The licensee did not have documentation that identified the acceptability of the use of lower bonnet nut' torque values during the September 1989 valve reassembly work, when:two vendor manuals identified higher torque values. This item

,

remains as a Followuo item No. 50-361/80-10-01, pending licensee

'

confirmation that the lower torque valve remains acceptable, i

,

'

(2)- Licensee Does Not Have A Formal Program To Ensure The Latest

,

Vendor Instructio_n_s Are Reviewed and Incorporate'd Into l

Applicable Licensee Instructions and Procedures In A

,

Timely Manner

~The existing torque values in maintenance procedure I

S023-I-6.19, Revision 2. TCN 2.6, Attachment 6 were revised

',

because the previous torque values were considerably lower than shown in current vendor manuals. A review of the history of this revision to the torque values and completed valve work, identified the following:

The current Revision 2 to Procedure 5023-I-6.19, was issued November 20, 1989 to revise the Attachment 6 values to agree with vendor instructions in an Anchor Darling-L Swing Check Valve Manual, licensee identification No.

S023-408-1-6-303. This vendor manual was received in the

<

licensee Corporate Documentation Management (CDM) file e

July 25, 1985, approximately four years earlier.

The latest Anchor Darling Swing Check Valve Manual

(MMSCV25001/88) was identified by the licensee during this

inspection as applicable for these valves

,

(IdentificationNo.S023-408-1-6-317-0). This document was received in COM May 5, 1988, approximately 18 months prior to the November 20, 1989 issue date for procedure

S023-I-6.19, but was not referenced in this procedure.

The torque values in Table 2 of this manual, for.a 1-8 stud size, agreed with the torque values identified in Revision 2 of Maintenance Procedure 5023-I-6.19,'except that attachment 6 of this procedure did not identify A534

,

stud / nut material. This letest vendor manual identified the torque values as maximum torques that shall not damage the designated bolting material, and identified that nominal torque would be 20 percent less than these values.

.

,

The latest vendor instructions in S023-408-1-6-317-0

identified the torq)ue for a 1-8 bonnet stud / nut (A453/660 material 'as 420~ft lb. This is a maximum i

torque value, and the nominal torque value was identified as 20 percent less, orf 336 ft. Ib.

During'the lastr September 1989 Unit 2 outage some. applicable Anchor Darling Check Valves were reassembled per the earlier revision of 5023-I-6.19, to a torque value of 275 lb? ft.

The 275 lb. ft. torque for-these 1-8 bonnet nuts,

.

l.

L T

a

.

,

-S-N.

.

...

was 145 ft. Ib. under the maximum value of 420 ft lb. and s'

61 ft. Ib. ander the nominal valve.of 336 ft. Ib.

Licensee personnel stated that they believe the valve bonnet stud / nut higher torque values were only

'

identified by.the vendor to aid in the reduction of bonnet.

to body gasket leaks.

I Per discussions with licensee personnel, it appears that the licensee did not have a formal documented vendor

- manual / instruction review program, to ensure the latest vendor manuals / instructions were incorporated into K

licensee procedures in a timely manner. The licensee h

acknowledged that administrative procedure S0123-VI-1.0.2, " Annual / Biennial Review of site orders,

-

procedures and instructions," Revision 3, TCH 3-2, did

.

not appear to be effective in the examples identified above. During this inspection, the licensee identified they were already reviewing this concern, buticould not provide any documentation on this vendor instruction review. The subject of vender interface to ensure that vendor information is complete, current and. controlled throughout the life of the plant,:and appropriately

'

~ referenced or incorporated in-plant instructions and

,

procedures, has been discussed in the fo,llowing;NRC

",

c

'

',

documents:

>w (

.

,

.

i, GenericLetter83-28,datedJuly,8,)1983.[

)',

.

,

,

r

,

,

"~

Generic Letter 90-03, dated March 20, 1990.'

!c~

"

s c

.

i

,

The-licensee did not have documentation that ensuresithe

'

latest vendor instructions will bet implemented into

,

licensee site orders, procedures and instructions {in a timely manner. Also, in the sample of.M.O.'s andl

r

~

maintenance procedures reviewed. it appeared that vendor.

q

'

manuals were not identified in these documents. The inspector reviewed licensee procedures mpg-001, " SONGS Maintenance Procedure Writers Guide," Revision 4 and Maintenance Procedure S0123-I-1.7, " Maintenance Order Preparation, Use and Scheduling," Revision 2, TCN 2-6.

These documents did not appear to require identification

'

of the reference applicable vendor manuals for the valves being worked.

Since there appears to be no identification of the

'

applicable vendor manual / instructions in work procedures, e

and a new or revised vendor manual can be in COM for 18 months to four years prior to incorporation into licensee

,

instructions and procedures, it appeared that the licensee could not assure that personnel are performing work in accordance with latest vendor instructions. The licensee g

stated that the maintenance planners nonnally perform an

informal review prior to the start of work. This concern is identified as Followup Item 50-361/90-10-02, and will be further reviewed during a. future inspection.

,

"

>j'h

.

,

,

..

<

'

"

.

.

E.

Sample Survey of Completed Check Valve Work

.

During review of completed check valve maintenance and surveillance

'

activities, the inspector identified the following:

During a September 1989 refueling outage, a Containment

Emergency Sump outlet check valve $21204MU003 was disassembled forInserviceTesting(IST) surveillance. A loose taper pin

"' '

was found inside the valve body and removed. Licensee t

personnel identified that the taper pin did not come from the I*

M'

check valve, and speculated that it came from upstream

' '

,

butterfly valve 2HV9303 or its companion butterfly valve

-

"

L 2HV9305. This information was documented in M.0. No.

L i 89032529000 on September 20, 1989. A Nonconformance Report

'

.s

}

(NCR) was not written at that time.

'

'

'

O*

During February 1990 a licensee review of M.0. No. 890325290005

-

.

,

identified that an NCR had not been issued on the loose taper'

'

pin. A Root-Cause Evaluation Nonconformance report (R-NCR)1No, i j,i

.

2R-0139 was issued February 28, 1990, five months later, to t

'

identify that the loose taper pin had been found in the check'

f'"

-

.

< valve. This R-NCR was issued against check valve S21204 MUOO3,

"

'

,

and under " apparent cause" the licensee had identified

"cause s

,

'

not apparent - issue M.O.'s to investigate".

(

As of March 22, 1990 the licensee had not taken any documented

.

action on this R-NCR. The inspector asked the licensee when2

+

'

they planned on taking action on this R-NCR, and was told that'

during the next refueling outage the two upstream butterfly-valves would be checked to see if the loose taper pin came

"

,

from one of them.

The licensee could not identify'the dimensions of the-taper pin

removed from the check valve, since they had not measured the taper pin and recorded its dimensions. The Itcensee could not provide the actual taper pin removed from the valve and bagged by the maintenance personnel working the check valve. The licensee staff stated that they believed the loose taper pin was approximately 4-inch long by a 1-inch diameter.

The licensee could not provide a drawing or dimensions of any

taper pins installed in the two upstream 24 inch butterfly valves.

On March 22, 1990 the inspector took the following action:

Asked the licensee to identify its documented justification for.

• continued operation of Unit 2, with the operability of two

24-inch motor-operated butterfly valves in the Safety Injection j

(SI)Systeminquestion, i

s a

-

,

.,

.

~

t.

s.

!

<

,

Identified that the fact that the licensee failure to issue an

i

,

NCR imediately in Septen.ber 1989, when the -loose taper was firstfound,wasanapparentviolation30-361/90-10-04_).

!

Identified that this failure to issue an NCR immediately after

identification of a nonconforming condition, was a repeat

..

violation similar to violation 50-206/89-03-02: issued in;

!

February 1989.

'

,

[

.

.

.

t

.

At the exit on March 23, 1990 the inspector dis. cussed his findings again with licensee management, and identified additional concerns:.

Licensee Personnel fail To Accept REssonsibility for

"

'

~

Documenting Problem By FnTtiating WRR.

From the time the loose taper pin was found, documented in the t

M.O.andidentifiedbytheNRC,severalindividuals.(Craft,,

Planners, Inspectors, Engineers,VariousSupervisor,etc.)in the licensee staff were aware of the nonconforming condition, held discussions on it, and all failed to:

<

(1) Recognize that each was responsible to assure that the problem was documented on an NCR and brought to the attention of higher management for evaluation and resolution.

(2) Recognize that this responsibility could not be informally delegated or transferred.

!

Ineffective Previous Licensee Management Action

A similar violation 50-206/89-03-02 was issued in February 1989 for not properly issuing an NCR " hen a nonconforming condition was first identified. The licensee March 9, 1989 r

response to that violation, was "to ensure the uniform application of existing site policy on when to initiate an NCR, a memorandum from the station manager will be prepared to reemphasize NCR requirements to appropriate station personnel."

-

This licensee memorandum on NCR's was issued April 10, 1989.

t Based on this new violation within approximately five months of issuing the licensee NCR memorandum, it appears the licensee management corrective action on the previous violation was

ineffective.

AvailabilityOfDetailedDrawing[InformationForValves

During this inspection the licensee's engineering staff could not provide the inspector with the drawings or dimensions for the subject taper pins installed in the applicable butterfly valves.

It appears that detailed valve information/ drawings are not readily available or identifiable to the system engineers in all cases. Various licensee personnel identified that they normally obtain detailed valve drawing identification,

'

H dimensions, etc. from the maintenance planners. -It appears-this same information should be readily available to the system engineers in a timely manner.

,

j t

%

.

g o-g 7,;

-

-

,

.

7

,;,r v

,

y (y li'

p 1m

.

",[+ y

<

p a-

,

x

'

,

c)

+ - -

'

y

.

.-

,

,

'

y

.

,

L In reply to the inspector request for an evaluation of the L

operability cf motor operated butterfly valves 2HV9303 and 2HV9305, i.

the licensee took the following actions

Performed a' visual inspection of both valves and identified

p

- that valve 2HV9303 was missing one taper pin. Three taper pins are normally installed to secure the valve disc to the stem.

l These pins transfer in shear the valve operating forces from the stem to the disc. The nonconforming condition identified.

[,

was that only two taper pins were installed.

• Performed a calculation showing that operation of the valve t

with two taper pins, instead of three as intended, did not pn reduce the margin for operation of the valve. This is because in both cases the highest stress levels during operation, under r

nonnal and accident conditions, are in the valve stem itself (

and not in the taper pins. Also, the highest stress levels, which are developed when valve.* break-away loads" are applied,

'

are well below allowable limits with two taper pins in place.

~

OnApril2,1990thelicenseeisvedalettertoRegionV(H.B. Ray to D.F. Kirsch), on the subject of'the failure to properly identify r

a nonconforming condition when the. loose taper pin was first found.

This letter discussed the following:

The results of a visual inspection of the two 24-inch butterfly

i valves, which was discussed above in this section.

t The results of a licensee calculation for valve 2HV9303.

• operation with two ta)er pins installed, instead of three, which was discussed a>ove in this section.

When tne licensee found the loose taper pin, they did not

properly identify the nonconforming condition which it represented. As a result, the licensee did not perform an

. assessment of the operability of valve 2HV9303.

Previous licensee corrective action for a similar

,

violation in 1989 was not effective.

,

(

-

A special licensce training module is scheduled to be

" developed, implemented, and completed in the next 60 days to:

L (1)s Require all engineering and other technical personnel to'

.

attend training on timely creation'of NCRs, whenever

%"c'*'

...

_ appropriate.

"

'

,

.

.

,

J

',

,

1 (2) Require. management' personnel to participate in the

<

^

3-

,

,s

'

presentation of this training, to ensure effective communication of its policy.

.

,

. -

F e

a

,

<

r

<

(-

i

ga

-

a 6E

.

la;

.o.

,

V t

e

,

.

By April 16, I?90 the licensee NCR process wil1 ~ be

c

' implemented using electronic redia, similar to that used currently for M.O.s.

This should make the task of initiating j

and processing an NCR easier, and reduce the incentive to

.

transfer this responsibility to others.

!

i

,

After reviewing the above information, it appears that the licensee

!

"6 has taken appropriate action for this item at this time,

'

Thisl item is closed.

,

.

I

,(

3.

Followup Items

~

L, p

Followup items are matters that have been discussed with-the licensee, i

k

.that will be reviewed further by the inspector, and that involve some f

,

action on the part of the NRC, the licensee, or both.

'

.

4.

Exit Interview r

"-

i

)

The inspector met with licensee management representatives-denoted in

. !

"

i

-

paragraph I on March 23, 1990. The scope of the inspection and the

,

.;

inspector's finding up to-the tine of the meeting were discussed. - The'

.

licensee acknowledged the scope and content of the inspection > findings.

~,

At this meeting the inspector requested that additional information be-forwarded to him for review in the NRC' regional office. The infornation -

was reviewed and the findings included in paragraphs 2.D. and 2.E. of

this report.

,

t t

t D

I t

l

!

.

,

+

i t

i ii