ML20073D630

Reactor Trip Breakers

ML20073D630
Person / Time
Site:	San Onofre
Issue date:	04/12/1983
From:	ANAHEIM, CA, SAN DIEGO GAS & ELECTRIC CO., SOUTHERN CALIFORNIA EDISON CO.
To:
Shared Package
ML13309B248	List: ML20073D630
References
IEB-83-01, IEB-83-1, NUDOCS 8304140074
Download: ML20073D630 (52)

Text

.

4

~

i 4- 4/12/83 4

i 1

d SAN ON0FRE NUCLEAR GENERATING STATION UNITS 2 AND 3 DOCKET N0s. 50-361 and 50-362

]

i REACTOR TRIP BREAKERS APRIL 1983 1

Southern California Edison Company San Diego Gas & Electric Company i

! City of Anaheim

- City of Rivers'ide i

C304140074 830413 PDR ADOCK 05000361 p PDR

I c l

. \

s.

)

i SAN ONOFRE UNITS 2 AND 3 RETURN TO POWER REPORT REACTOR TRIP BREAKERS DUTLINE 1 I. Executive Summary II. Scope and Purpose III. Introductory and Background Information

A. Reactor Protection System (RPS)

{ l. Design Criteria

2. Design Description B. Reactor Trip Breakers (RTB's)

, 1. Design Criteria

2. Design Description

]

C. RTB Initial Procurtment and Testing

1. Procurement History

2. Acceptance Tests - Vendor

3. Acceptance Tests - Startup -

4. Startup and Surveillance Tests r

D. March 1 and 8,1983 RTB Surveillance Test i 1. Description of Tests

2. Test Results 1

IV. Investigative Programs i

A. RTB Investigative Tests

1. Description of Tests

( 2. Test Results B. Evaluation of Licensee Administrative Procedures

1. Control of Vendor Infomation, Tech. Manuals

a. Vendor Supplied Documentation

b. Yendor Supplied Services

c. i Vendor Recommendations on RTB l

-f

2. Control of Hardware Configuration'

a. Spare Part Programmatic Controls

b. Reactor Trip Switchgear Spare Parts l

i I

u .

3. Maintenance Program 4

a. Procedures and QA/QC Requirements

b. Maintenance History and Records

c. Vendor Maintenance Activities

4. Surveillance Program

a. Technical Specification Requirements

b. Procedures and QA/QC Requirements

c. Surveillance History and Records l S. Reporting Requirements

a. Technical Specification Requirements

b. Reporting of March 1 and 8,1983 Surveillance Results

6. Non-Conformance Reports

7. Compliance With IE Bulletins and Circulars 4 a. Initial Reviews

b. Follow-up Procedures

8. Post-Trip / Restart Reviews 4

C. Evaluation of Capability to Mitigate ATWS

1. Procedures for Mitigating ATWS

2. Control Room Layout / Design

3. Operator Training / Knowledge D. Findings / Conclusions Regarding RTB Failure V. Proposed Corrective Actions A. Short Term Corrective Action Leading to Restart

1. Control of Hardware, Vendor Information, Vendor Personnel

2. Maintenance Procedures and QA/QC Requirements

3. Surveillance Procedures

4. Technical Specification

5. Operator Training B. Long Tern Corrective Actions Subsequent to Restart

1. Follow On Activities

2. Evaluation of RTB Improvements VI. Conclusions 11

. l s

I. Executive Summary ,

As a result of failure of reactor trip circuit breakers (RFBs) to function at Salem 1 (IE Bulletin 83-01), SCE perfomed its 18 month surveillance test at San Onofre Units 2 and 3 which independently tests undervoltage (UV) and shunt trip functions of the RTB. This was done even though the RTB are of a different design than Salem and were not required to be tested by the bulletin. The surveillance test was perfomed in early March 1983 when San Onofre Units 2 ar.d 3 were both shutdown and in different stages of their respective startup test programs. Four of the total of eighteen RTBs tested failed to trip following actuation of their UV devices. All eighteen tripped following actuation of the shunt devices.

SCE responded promptly to the failure of the UY devices to trip the RTB by conducting a comprehensive investigation which is discussed in this report. The purposes of this investigation, and resulting findings and corrective action, are to ensure that the UV devices operate reliably and to ensure that the SCE startup and operating programs include any lessons learned from the failure of the CV devices to operate reliably. However, it is important that the following facts be considered in reviewing the experience with RTB UV devices at San Onofre Units 2 and 3:

o The UV device is one of two diverse methods used to trip the RTBs.

Unlike other designs, the UV device is not required to function for the reactor protection system to complete its design basis protective action at San Onofre Units 2 and 3. Reactor trip is initiated by opening the RTBs automatically or manually, and both signals actuate both UY (deenergi o to operate) and shunt (energize to operate) trip devices. During our investigation the shunt device always functioned to trip the RTD, and, therefore, the RTBs at San Onofre Units 2 and 3 have perfomed so as to meet their design basis function.

o Failures of UV devices to function, and work perfomed in response to these failures, have occurred during startup testing at San Onofre Units 2 and 3. As a result, the initial failures in March and July 1982 were interpreted as problems with set up and adjustment which are typical of startup testing. The failures during surveillance testing in March 1983 were recognized by the operating phase surveillance program as symptomatic of other potential problems and as requiring in depth investigation.

The comprehensive SCE investigation discussed in this report has identified factors contributing directly to the failure of the.UV device. The report tem corrective actions required to obtain satisfactory breaker performance have been identified and implemented.

,. ~~ -g- op

The UV device has a much smaller force margin to trip the RTB than has the shunt device. Nevertheless, with upgraded maintenance and surveillance testing, the UV device will function reliably. The shunt device reliability for tripping the RTB has always been very high in the San Onofre Units 2 and 3 design, and operation of this device alone fully satisfies the reactor protection system design basis.

4 J

f 4

4 A

l 4

i I

4 I

- + -, - .w- -

l l

l

II. Scope and Purpose i This report provides a summary of the comprehensive review perfomed by the Southern California Edison Company (SCE) subsequent to discovery of the reactor trip breaker undervoltage trip mechanism operability problem on San Onofre Units 2 and 3. The purpose of the report is to describe the corrective actions that have been implemented as a result of our review.

Following a brief discussion of the reactor protection system (RPS),

, reactor trip bmakers (RTB), and surveillance tests identifying the operability problem, the point is made that the UV trip device is not required to function for the reactor protection system to perfom and

! complete its design basis protective action. The report then describes 1

SCE's investigative program, including the specific breaker tests that were perfomed on the test bench (NRC witnessed) and subsequent testing conducted by SCE at our Electrical Test Laboratory. Discussions are provided summarizing our evaluation of administrative procedures, including; control of vendor data, control of hardware configuration, maintenance and surveillance programs, reporting requirements, non-confomance reports and compliance with IE Bulletins and Circulars.

i The report also discusses evaluation of post-trip and restart reviews as

. well as capabilities to mitigate the consequences of Anticipated ,

! Transients Without Scram (ATWS). Based on the RTB investigative test '

results, SCE has identified enhanced maintenance and surveillance procedures for the RTB. Based on the evaluation of administrative procedure, SCE has also identified both short and longer tem corrective actions which we believe are appropriate to strengthen our administrative system.

The report concludes that with the implementation of the enhanced procedures for maintaining the RTBs, San Onofre Units 2 and 3 can safely j resume operation and startup testing.

i

=

I f

s III. Introductory and Background Infonnation NRC IE Bulletin No. 83-01, Failure of Reactor Trip Breakers (RTB's)

(Westinghouse DB-50) to Open On Automatic Trip Signal, issued on February 25, 1983, discussed the February 25, 1983 failure of the DB-50 RTB's to open automatically upon mceipt of a valid trip signal at Salem Unit 1. The reactor was manually tripped from the control room about 30 seconds later, and the event was successfully tenninated without core damage.

The Salem Unit 1 RPS is designed to automatically open the RTB's using only the undervoltage trip device; manual actuation of the RTB uses both the UV and shunt trip devices. The failure of the RTB's to trip automatically was attributed to sticking of the undervoltage trip device.

In contrast with Salem Unit 1, the San Onofre Units 2 and 3 RPS provides both automatic and manual signals to actuate both the UY and shunt trip devices of each of the RTB's and the design configuration utilizes eight channelized RTB's. This section of the report discusses the design criteria and function of the RPS, the design and function of the RTB as part of the overall mactor protection action and the role of the undervoltage trip device as part of the RTB.

Even though IE Bulletin No. 83-01 did not require testing of the RTB's at San Onofre Units 2 and 3, the UV trip devices on the RTB s at San Onofre Units 2 and 3 were independently tested on March 8 and 1,1983 respectively. The results of these tests are discussed in Section III.D of this mport.

j.

4 j s t

A. Reactor Protection System (RPS) l j 1. Design Criteria I The design bases for the Reactor Protection System are presented in FSAR Section 7.2.1.2 and are summarized as follows:

l The RPS is designed to assure adequate protection of the fuel, fuel l cladding, and RCS pressure boundary during antic' pated operational

occurrences. In addition, the system is designed to assist the ESFAS j in limiting the consequences of accident conditions.

l a. The system is designed in compliance with the applicable NRC i General Design Criteria of 10 CFR 50 Appendix A, IEEE 279-1971, 1 IEEE 338-1971 and is consistent with the recommendations of i Regulatory Guide 1.53 and Regulatory Guide 1.22.

b. The system is designed to alert the operator when any monitored

. plant parameter is approaching a condition that would initiate protective action.

i c. The system is designed so that spurious protective action will i not be initiated during normal operation of the plant.

i j d. Four measurement channels are provided for each plant parameter monitored by the protection system, with the exception of the control rod positions.

e. The four measurement channels are independent and isolated from i each other atilizing: separate sensors monitoring the channel

[ parameters, separate wire trays or conduits for the channel i interconnecting cabling, separate channels for the RPS cabinet mounted signal processing equipment and separate instrument ac j power buses backed up by separate batteries for each channel.

j f. The four measurement channels provide trip signals to six i

independent logic matrices, arranged to effect a two-out-of-four coincidence logic, each having outputs to four independent trip paths for each actuation signal.

l l g. When one of the four measurement channels is taken out of j service, the protection system logic can be changed to a

two-out-of-three coincidence logic for actuation of plant l

protective action.

h. After initiation, manual reset of the actuation _ output signal is j possible following the clearing of its input signals.

! i. System functions requiring operator attention or action during routine plant aperations are displayed and/or controlled at the l operators main control board,

j. Annunciation is provided at the main control board of all-l operations at the RPS cabinet that could affect the function of

the system.

l

I s l- .

i l 2. Design Description i

i Figum III.A.2-1 is a simplified functional diagram of the reactor

! protection system. As can be seen the RPS is a four channel system i and can be divided into several areas defined as: Measurement

! Channels, Bistables, Logic Matrices, Logic Matrix Relays, Trip Paths

! and Trip Circuit Breakers. FSAR 3ection 7.2 provides a detailed i discussion of each of these areas. The following is a summary j discription of the RPS operation. 4 Measurement channels consist of sensors and signal conditioning

equipment whose purpose is to convert the parameters being measured

! (pmssure, temperature, etc.) into signals usable to the RPS i

bistables or calculators. These signals are provided in the form of analog voltages.

f Signals from the measurement channels are sent to voltage comparator i circuits (bistables) where the input signal is compared to i predetermined setpoints. Whenever the measurement channel signal j reaches the setpoint, the bistable output will de-energize associated

bistable relays.

Tripping of a bistable results in a channel trip which is j characterized by the de-energization of three bistable trip relays.

l Contacts from the bistable relays of the same parameter in the four j protective channels are arranged into six logic ANDS, designated AB, i AC, AD, BC, BD, and CD, which represent all possible two-out-of-four i combinations. To form an AND circuit, the bistable trip relay j contacts of two like protective measurement channels are connected in

! parallel (e.g. , one from A and one from B). This process is

) continued until all combinations have been fonned.

i

! Since there is more than one parameter than can initiate a reactor j trip, the parallel pairs of bistable trip relay contacts for each i monitored parameter are connected in series (Logic OR) to fonn six

! logic matrices. The six matrices are designated AB, AC, AD, BC, BD, i and CD.

j This logic requires two or more bistables monitoring the same

parameter to be in a tripped condition before a reactor trip can be

{ generated.

1

Each logic matrix is connected in series with a set of four logic

! matrix miays. Each logic matrix relay is associated with a trip

path. The trip path is made up of six series contacts (one from each~

logic matrix). The contacts are also in series with a trip circuit j breaker control relay (initiation relay).

For each actuation signal, the above logic causes the de-energizing i of the four trip path initiation relays whenever any one of the logic matrices is de-energized. '

i 1

. . . _ . . _. - . _ , . . .- -. . . _ -, _ _ _ . _ , _ . . _ _ . . ~ . . - - ,

1 INPUTS FROM NSSS I L 1 2 3 4 5 6 7 8 9_N 12 3 4 5 6 7 8 9_N 1 2 3 4 5 6 7 8 9 -N l' 2 3 4 5 6 7 8 9_N 111n1111 111111111 111111111 i iii GISTABLES l CHANNELS A l CHANNELS B h l CHANNELS C' h l CHANNELS D 1 1 1 1 1 1 A RICES l A.B LOGIC h l A.C LOGIC l] l A D LOGIC h l B-C LOGIC h l B.O LOGIC h l C D LOGIC h AB4 , , AC4 , ,

AD4 , ,

BC4 , ,

604 _ ,

CD4 A AB1 AB2 AB3 ACI AC2 AC3 AD1 D2 AD3 BC1 BC2 BC3 B01 BD2 B03 CD1 CD2 CD3 TO 120 Vac TO 120 Vac TO 120 Vac TO 120 Vac VITA L INSTRUMENT VITAL INSTR UMENT VITAL INSTRUMENT VITAL INSTRUMENT BUS No.10 BUS No.1 A AB1 g  ;

AB2 BUS No. I B BUS No. Iq , AB3 g l AB4 AC1 1 ZAC2 AC3 1 Z AC4 AD1 1 ZAD2 AD3 Z 1 AD4 BC1 2 ZBC2 BC3 Z T BC4 TRIP PATHS BD1 B02 B03 Z 1 BD4 TRIP CD1 CD2 CD3 CD4 CIRCUIT BREAKER i i CONTRO L R ELAYS K11 i i I I IK2 ,, K31 Il i i IK4 ,

(INITIAT loM 480 Vac 3d 480 Vac-3d '*

BUS No.1 BUS No.2 d

N o.1 d) MAIN BREAKERS MOTOR CIRCUIT No.2 )

GENERATORS No.1 No.2 240 Vac-3/

~

hSVNCHRONIZER h)

+125 Vdc BUS No.3

+125 Vdc BUS No.2] _

1f o u hh K2 BUS TIE 9

1 _1 " '

_ __ [r_-

4_

MANUAL J o c c h""_TE TRIP No.1 g_ _'_ _" _ .

_ _ _ _. _Z_ _ _ _ _ _ _ _ _ _ . _ _ _ __

_ g _g.g

/__UV

_ /_S T, ,,,

,6,) ,_

3

- 7 g-/_V ___

__/

U ST

+125 Vdc BUS No.1 g =_7

_1, _L."

j 4 _j,4 p +125 Vdc BUS No. 4 MANUAL -

-h b K1 TRIP CIRCUIT K4 h

TRIP No. 2 l BR E A K E R *,

Q_ __

/-_

-_i____ __ ____________

5 go <-

uv/__/_ /_ _ _ _ _) ,3 4 luv]ST I

a g

g. ..

UV

_g ST i

CEDM PCWER INOlvlCUAL CEOM 3,CEDM CONTROL INOlVIDUAL CEOM SUPPLIES POWER SUPPLIES y SYST EM POWER SUPPLIES A 4 4 d I h ' CONTR OL CEOM 'CEOM CEOM CEDM CEDM ' ELEMENT COILS s COILS ' COILS , COILS , COILS , bMO lLS DRIVE

, MECHANISMS FIGURE III.A.2-1 SIMPLIFIED FUNCTIONAL DIAGRAM OF THE REACTOR' PROTECTION SYSTEM

)

i. .

la <

4 The trip path initiation relays then transmit trip signals to the RTBs. The design criteria and function of the RTB using these actuation signals is described in the next section.

'l i

i i

l i

b 1

}

I i

l i

l i

)

i J

t 1

}

~

l

~

B. Reactor Trip Bmakers (RTB's)

1. Design Criteria The reactor trip switchgear cabinet assembly, including the reactor trip breakers, is specified in SONGS 2 and 3 FSAR Table 3.2-1 as:

Quality Class 1 (safety related)

Seismic Category 1 (design basis earthquake)

Electrical Class lE It should be noted that the seismic qualification test plan does not mquire the bmaker closing circuit to function during a DBE; however, it must not interfere in any way with the trip function of the bmaker.

The reactor trip signals are generated in the reactor protection system through the six logic matrices as described in 1 Section III.A.l.2. The reactor trip signals are in turn transmitted i

to both the undervoltage and shunt trip devices on each RTB. Each of the four trip path initiation relays (K relays) sends an actuation signal to both the UV and shunt trip devices on a pair of RTBs. The pair of RTB receive de power from the same vital bus that provides ac power for their associated initiation relay. This can be seen schematically in Figure III. A.2-1 where the Kl relay provides the actuation signal to RTB numbers 1 and 5.

The reactor trip breaker design is such that failure of either the undervoltage trip device or the shunt trip device does not eliminate the safety function of the reactor trip breaker. The trip circuit bmakers will complete their protective action of interrupting power to the control rods using either the UV or shunt trip devices.

The RTB undervoltage device and the shunt trip device compliment each other in that the undervoltage device trips the breaker upon loss of control voltage while the shunt trip device trips the breaker upon application of control voltage (see Figure III.B.1-1).

1 4

l 1

- 1 i

l

, s e -

/ t

. . . . . . . ~ . .

-CI' ru w y oro

.........................e.

g yu usu 3LSE oTS *******"**'*****.".#''l**

.o = v r- -

. .n .z x,d< OTS scga e= .......

g w

~~l nTEST y 4 n TtsT

_-, , s. , ms . R G. .. m

><- ,, ,. =A. , m ,L, i . . . . . . -

< mm +,TR.

ma T<,., - - -Tc itS v o-C 3p" n u,, y oTB u&,hoTB sat <.

>g w SOURC E.*1 ENTERS

": g 073 R

g 3g U* C 1

. 3g T I-I MT I-I F3 ---

- - To

. , , . . S HT. fm N 4 2 f., ar< UPJIT2 PANE t_

f[.Q 52 0 #8 foTB 3 '- 4 UMIT 3 PANEt.'

BREAKER. CbOSINb oac"

g t..

s ss APPROVED

_ ,'6 t

a ,U 10 Y

. H 1" --+

• REFERENCE

-a ,e .

& DESIGN w B;; .

s u.cwaH 078 CTS OTB 07 8 0T5 CTS OTS o18 a o B- outgoing TERMINAL BLDCM 8P3 at 11/3t,33 ss s. ss ss .ss ss ss ss RC- PLAN T* COM PUTER

.,$2;, ,,,5,,.f 3 . , . $,1, ., S,1_. . 31,, .% 3L 2. AIR CIRCUlf BREAKER-T.COV 600A.

Tb &(b A, 7 ,2 fb 2 LEMERGENCY TRIP PUSHBUTTON AT SWITC64GF. AR TD HAVE A CLt Ait PLASTsc

?; ,,  ?; ,, - J ,, re ois

' q{TI-l y {tucae,. Hruc ac, p ,37ggy,ygccyg,, ,,7gp,oy,3,oy, FOR LoCKtNG WITH WIRE LEAt5.

tTa o oTs ooTs g eva o ova o oTa g ets .

T n.i r . v n.o-

-tv- 1,0'S - -u As**

y

. 1O TS styrr Z.

pc NC riucLEAR SAFETY f1 ELATED m /,r g g y l Do CAtg av. l omscaimon n. ,.,1)

.. ,. ,,.; new, ocse=enion .c>.,

.. w .. .

.... . . . <"> a w

• a v.

cmcu a E1.EMENTARYwiRtNG OtAGR.AM

. , .,. . ma. u u t -m0 <L i+" . , . . *~-' f pu ETEDBontacT3 5efeAJ e.,., /,) v -

w/, \ - .u.v m.7.scan:fr.. /,+h2 ,,...s .; w ;..,ov .w. '-

m. s w. s.e tre...y. xrj/ REACTOR Trip circuit BREAVER iv ,,

p.,m m \ erm str.v.nto esnn er -. ., ,c.is 9/..i y, . .  ! I m m, g,,,7 'T;.3

" CV ilCO % 31C.

• S i{.. ..

t

.'1,.,,.,,,,;.

. ,. - + - =

g 9 , ,, -,,-a, . . , m. ces.,c_ : <s. , t s. 5 Ci()

TcY iMUl%2El,':*G';s'**^

c,c.,l.,

~ T T:,- gun:cE'X *Ui ~g ; $'../J7c 4I2 3 3 O v->$ ~ . ',. 1

= ~

• l ' -

cri es por a o f: t .a t.m .-u-a c.. l - . -

lC4

_'o n,.. t

\n '

...<.... ,. m . ..

.i <,. ...r.....<-.

t itt rtco. t"310- V* C

+=N

j. ff.,..,v e 4 %

. .e a  :. . . . - ~ <."x..w. *lruser -

, 3% .".

e ls.

v- en.s...s..v

...s c.nar.Anat.or s.- . v i .a w.sw n a rs.n n* s e

• • A'af.

1 er S FICUP.T. III.r,.1-1 .-

s

_g_

2. Design Description The details of the breaker latching mechanism and the function of the tripping devices are shown pictorially in the following sketches:

a. Breaker Assembly, Trip Bar and Latch Figures III.B.2-1 through 8 provide simplified representations of the AK2-25 latch mechanism operation.

b. Shunt Coil Assembly Figure III.B.2-9 depicts the AK2-25 shunt trip device in both the

" tripped" and "not tripped" positions. The device requires 125 Vdc and is energized to trip by actuating the tripper bar in a counter clockwise direction.

c. Undervoltage (UV) Coil Assembly Figure III.B.2-10 depicts the AK2-25 undervoltage trip device in both the " tripped" and "not tripped" positions. This device requires 125 Yde and is normally energized. Upon a loss of voltage the device actuates to rotate the tripper bar in a counterclockwise direction.

-- , w

d. Existing Design Modifications The RTB in use at SONGS Units 2 and 3 have one additional

< modification to the foregoing design description. CE recommended a design change which SCE implemented to place a diode in 4 parallel with the UV coil to provide a discharge path for the 4 collapsing field and prevent contact damage to the RPS "K" relay. This diode is shown in Figure III.B.1-1. It should be l

noted that the diode is not installed on the breaker, but in the

. switchgear cabinet assembly, so that when the breaker is installed or in the " racked out" position, the diode is in the UY coil circuit, but when the breaker is removed from the cabinet, it is not. The requirement for simulating this diode when preventive maintenance and testing is performed is discussed under test results in Section IV.A of this report. This diode also results in a slower collapse time for the UV coil magnetic

, field, causing the UY trip device operating time to increase hy about 30 msec. This ci:ange may also result in a slight loss of UY device tripping force by reducing the kinetic energy of the UV device armature; however, when the enhanced breaker maintenance is performed as described in this report, the overall RPS response time remains within acceptable limits.

4 i

, . . I l

' 1 FIGURE'In.B.T-1 '

. GE AK-2-25 CIRCUIT BREAKER UDPER STUD BREAKER BASE -

STATIONARY CONTACTS MOVABLE C0tlTACT A r

2 IflSULATED LINKS '

STOP."A" 1 -~~

\

O ) \

\

s -

OUTPUT CRANK \

\. \

s.

'\

CLOSIt:G SPRING \

_g___\ l 1 PIVOT

\ '. \

S' \\ '\_.\ T I f,1 s~s\ a i

, i I

,! l-TRIP LATCH PIVOT PIN I

l l

STOP "B" (BLOCKItiG CAM) +

1 ' I LOWER STUD-

[i 1 I

.) CLOSING SOLEN 0ID arf 1ATURE LOSING-SOLEN 0ID

\ t \

l I THE CIRCUIT BREAKER f1ECHANISit HAS BEEN SI!1PLIFIED CONSIDERABLY 3

, 1 I ! IM ORDER TO CLARIFY ITS OPERATI0ft.

I I! STOP "A" REPRESENTS A PORT.10tl 0F THE !!ECHANIS?! THAT PREVENTS THE LINK FR01 fiOVING BEYOND TIlE POI!iT I:lDICATED.

i i STOP "S" IS A BLOCKING CA*1 THAT PREVENTS THE tiECHANIS!1 i FRO:1f:0VING AND CLOSI:lG THE.COITACTS UNTIL THE CLOSING l

I g S? RING IS CHARGED.

{ THE TRIP LATCH LOCKS THE !!ECHA:!IS!1 IN THE CLOSED C0f! TACT POSITION UNTIL RELEASED BY THE TRIP PUSHBUTT0ft, SHUNT TRIP DEVICE, OR THE UNDERVOLTAGE TRIPPING DEVICE. IT IS SHO'.!!!

ON THIS SKETCH AS A DICTORIAL REPRESENTATION OF ITS FU:lCTION.

t

_ - - = -

~ *

. . I..

FIGURE III.B.2.2 l

UPPER STUD

!!0VABLE BREAKER BASE \

C0flTACT 37 STATIONARY CONTACTS 1 i f

i INSULATED

' LINKS STOP "A" \ -L t.

_ [3_

b L--

CLOSING - . OUTPUT CRANK S0l

.,a._53,m$

/

. 1

/

i CLOSING SIEI"b k P!V0T

~

\'.

\ h W

. o -

l Q\

3 r

\M. '

1 I TRIP LATCH PIVOT PIN-

/

{ LO' DER STUD BLOCKING CAli

• 1_

l 1 1 l - CLOSING CONDITION 1 g

i SOLEN 0ID g THE CIRCUIT IS OPEN AND THE OPERATING l SEOUENCE IS ABOUT TO START.

I 1 1 g ) I a THE SPRING LOAD KEEDS THE CONTACTS OPEN.

I I I I

i .

\

FIChRE Ill.B.2-3 UPPER STUD BREAKER BASE-MOVABLE STATIONARY CONTACT CONTACTS CLOSI!!G SOLEN 0ID arf 1ATURE I

i CLOSING SPRING INSULATED k LINKS i 1 - -

i

, STOP "A"N m i

\'

L_ _ _ ___

s

{ .

k \

' OUTPUT CRA?:K

\\

\

/

- ~

s

- PIVOT 7 1

.- ~

m . 'r \ '

-~-

,;'s,f.fs{',' .

I r

(f '

TRIP LATCH PIVOT PIN 1

l i l BLOCKING CAM LOWER STUD

\ . .

E i k ik }\ CLOS:NGC0t:DITIC : 2 I I SOLEN 0ID THE CLOSING SOLEN 0ID IS ENERGIZED AND THE

} arf 1ATURE EXTENDS. ENERGY IS STORED IN THE I

CLOSING SPRING BUT THE BLOCKING CA!! PREVENTS g g g THE OUTPUT CRANK FROM l*0VING.

\ .

\\.

h .

l

~

FIGUPI III.B.2-4 UPPER STUD -

BREAKER BASE-STATIONARYn CONTACTS it0VABLE CONTACT h,

-CLOSIt'G SOLEN 01.D ' '

1

.. 1 1 ,

AR!!ATURE t*** 's L.

r .

s g

s, a

i /

': I \ '

rP CLOSING SPRING INSULATED \3 ',

( LINK \ ', ,

STOP "A" '

, l

\ ', ,..... =;--

/ws'. b. - - - - - - - - - - - - 9\- u- y A l

//

--./: .,/....'._..._...L-

, s

. ,y ^

'+ . \-s i

, /

e

.l

\,

s 7.

s

\

I i.. ,

lf. \ '

's <

> l'..s, ,

,'f , ,

-0UTPUT CRANK \, , 'i

' s

,/f ,

,/

\ s

\,

14 ,

l

, /

l Q[-- -

L h \.- )

-- CRANK PIVOT L

'3

, \

/ ,. ,1 l. ~- g g/ / .

/ ""I ..,'d' '

PIVOT PIM L TRIP LATCH BLOCKItiG LO'IER STUD

\ ' 1 q

CAM +

$ k CL OSING CONDITION 3 -

{ SOLEN 0ID

{

THE BLOCKING CNi RELEASES THE OUTPUT CRAtlK AtlD

.k. 1 THE SPRING TENSION CAUSES THE CRANK TO ROTATE j l ABOUT ITS PIVOT'AND IT DRIVES THE H0VABLE CONTACTS 1 g AGAINST T}lE STATIONARY CONTACTS.

i.I 1

THE TRIP LATCH 110LDS THE !'ECHANISit IN POSITION AND THE CIRCUIT CET'.lEEN THE UPPER AND L0tlER STUDS Li -

IS C0ftPLETED.

e

,. .. - ,. I'

' * ~ '

FIGURE III.B.2-5 l UPPER STUD BREAKER BASE STATIONARY -

• CONTACTS

]/

It0VABLE

CLOSI!!G SOLEliOID AR.'tATURE CONTACT \

,l 9 O ,

I

!' 1 i  :

1 I IflSULATED CLOSIrlG LINKS f SPRING ,

l O

d /

i

- OUTPUT CRAtlK

'l e

{ e .

P PIVOT  ;

1, i PIVOT PIN

. k

~

^" L0UER STUD t i ,-

{ s I _

r I

' i CONDITION 4 -

. \ \

I

) THE CLOSING SOLEil01D IS DE-ENERGIZED AND

'i j THE ARiiATURE REiAIXS IN ITS POSITION

, DUE TO THE OVER-CEilTER TOGGLE ACTION.

i , I I I

: I t I f '

CLOSING SOLEN 0ID Le\

1

- l -

FICURE III.B.2-6 UPPER STUD BREAKER BASE STATIONARY CONTACTS NOVABLE CONTACT .h

, i I, ,

i .

l I

i 2,

I

, -r i, \ ,

'iP: . INSULATED

' i i

, i

\.[;s, STOP."A"

..____ ," lI

!'!c...

g a i

s

,,. ?

, f ~s~-  :

1l.,

~r t ,/ ,,,

!*~

d '

. ' _ 1_' ,' . . .'

I

, i A ' ' ; ,,, , . 7T '

i 4' ~e s' l i ^', i ~.

,g t . ',,

/

' OUTPUT CRANK

^

, e

, j i i l ,, -

ie

^</ l

', I i ,

, l ' ' '

i '. CLOSING / e' l I

i. 5

/ / /

PIVOT h [

M, SPRING' , , , '

" ~

J

' l PIVOT PINJ

. CLOSING -TRIP LATCH 50LE'10ID ARitATURE LO'lER STUDJ

{

N BLOCKING CAM ,9 1 1 i q

E g i l C0tlDITION 5 g I i

g l THE TRIP LATCH IS TRIPPED BY THE TRIP PUSHBUTTON, THE SHUNT TRIP g DEVICE, OR THE UNDERVOLTAGE TRIPPING DEVICE. THE OVER-CENTER

.I - TOGGLE ACTION COLLAPSES AND THE MECHANISM IS NOU FREE TO MOVE.

I

' I 1 THE SPRING TENSION CAUSES THE OUTPUT CRAtiK TO ROTATE ABOUT ITS I

i

, II J PIVOT, PULLING THE IttSULATED LillKS AND CAUSING THE CONTACTS TO t i .1 OPEN.

I THE SYSTE!! UILL REMAIN IN THIS DE-ENERGIZED POSITION UNTIL THE l CLOSIN- TRIPPING CONDITION IS RECTIFIED AND THE CLOSING SOLEN 0ID IS SOLENOID ENERGIZED.

m O

e l

l

t

,, , FIGURk III.B.2-7 TRIP LATCH i  : ' '

_/

ROLLER PIVOT 2 TRIP SHAFT

- t TRIP LATCH TRIP LATCH ROLLER

\

THE TRIP LATCH AND TRIP LATCH ROLLER ARE IN THIS POSITION WHEN THE CLOSING SOLEN 0ID IS NOT ENERGIZED AND THE TRIP LATCH IS IN THE RESET POSITION. THERE IS A GAP BETWEEN THE ROLLER AND THE LATCH.

WHEN THE CLOSING SOLEN 0ID IS ENERGIZED, THE TRIP LATCH ROLLER IN ITS HOLDER TURNS

. COUNTER CLQCKHISE, CLOSES THE GAP, AND IS RESTRAINED BY THE TRIP LATCH. i THE TRIP LATCH AND ROLLER REf!AIN IN THIS POSITION UNTIL THE BREAKER IS TRIPPED. ,

a

.. 's, t, O '![ _' _ _ _ _ _ __ ~ ~

WHEN ANY OF THE TRIPS ARE ACTIVATED, THE TRIP SHAFT ROTATES IN A COUNTER CLOCK!!ISE DIRECTION AND THE TRIP ROLLER ROLLS ALONG THE CONTACTIrlG SURFACE OF THE TRIP LATCH UNTIL THE TRIP LATCH SWINGS OUT OF THE WAY, LEAVING THE ROLLER AND ITS HOLDER TO ROTATE FREELY ABOUT ITS PIV0T. THIS CAUSES THE OVER-CENTER TOGGLE ACTION IN THE MECHANISM TO COLLAPSE AND THE CONTACTS OPEN.

I THE ROLLER RETURNS TO ITS ORIGINAL POSITION WHEN TfiE C0:lTACTS OPEN AND THE f1ECHAtlIsli IS DE-ENERGIZED. .THE TRIP LATCH WILL REMAIN IN THE POSITIO:1 INDICATED UNTIL THE CAUSE FOR TRIPPIi.J'IS RECTIFIED AND THE TRIP LATCH IS RESET.

7- _ _ _ . . .- -

o .

' o?

FIGURE III.B.2-8.

i l

f1ANUAL TRIP PADDLE UNDERVOLTAGE TRIP PADDLE Ns SHUNT TRIP PADDLE g TRIP SHAFT s,

s. TRIP SHAFT BEARING

( TRIP LATCH

%. Yp '

4

'N TRIP LATCH ROLLER TRIP SHAFT SKETCH NOTE: THIS SKETCH DOES NOT PORTRAY THE ACTUAL SHAFT OR THE .

LOCATION OF THE VARIOUS TRIPPING PADDLES WITH RESPECT TO EACH OTHER OR TO THE SUPPORTING BEARINGS. IT IS A PICTORIAL REPRESENTATION OF THE TRIP SHAFT AND ITS FUNCTIONS.

o l

4

• • . g .

FIGURE III.B.2-9 r 9h *

.p .u 4

s- W ' -::

'%c < .

A,6

.e br l, *

. f 8= 9 i

~

. t  :  :

+  ! i

• i j

. N..I

==

- 1 ,

a fl0T TRIPPED i -

' (*-

~

3- y -

1 t , .

.my [-

g, , .....

.. v

_ . n .

4- .

_, ~ J -u- . .e

~am - rs. o.

$ T" W. lii u su

.8 4 i i* e

. q:  : .

'ig i  :  !, , & h

,W, e- -

.....Q r .

-- , TRIPPED .

. Figure 9. ( 895C161) Shunt Trip Device

1. Nut s

6.  !!Icrews 11. Armature Arm Trip Paddle

~

2. Frame 7. Magnet 12.

l 3. Spring 8. Coil 13. Mechanism Frame

! 4. Rivet 9. Clamp 14. Trip Shaft Clamp

[ 5. Weight 10. Armature l

1 ,

l .

l

. .-_...-----.a. -

- - .p, ; ,

. 'r t'J

, FIGURE III.B.2-10 Il

1Y' \S

\

4 I .

% g--' .

~

- 3 _

4 G- 7] is ..

.,j % [ g N

~

- ~

m 6

s QO'@ aj :3 S

%.C v + 'l '4;w .

zm

~

-E-

-l I

9 l

'# h/

g .

. 4,>- :o N '

9

. TRIPPED, --

20 I$/ 1. Mounting Screw

2. Frame w

is . 3. Armature I 4. spring

, 5. Shading Ring 2 ' 7 6. Adjus*.ng Screw s

Tl gY

.I

,, 7. TeW Nut

8. BusW

~

(b._ + -

+' _ -/ s)>-

• 6

9. Clamp

10. Magnet

' Wi ----

lI c

.10 s is li: E"""

13. Rivet '

14. Adjusting Screw

• I

-m  :: 15. T eW! Wire i s

16. Mounting Nut i 6 - N ' "O " O!, v(1- :3 17. Mechanism Fra se

% 18.* Trip Paddle Cla=g.s

'. - ~

F ' --. .

19. Trip Paddle 7 q l G

C 9l l

'h 12

20. Adjusting Screw

)

8- .

l L h> l, , , ,

1-@- -

l 4 ,> - :o

\ 9 NOT TRIPPED -

., N. ..

Fig.:re 28.. (0152C9206) Undervoltage Tripping Device -

C. RTB Initial Procumment and Testing i

1. Procurement History The San Onofre Units 2 and 3 Reactor Trip Switchgear (RTSG) Systems were procurred as a part of a general procurement package intended for several CE plants. A procurement history of this generic package and the documents that confim compliance with SCE's project specifications am described below. This documentation is for the RTSG that included the original AK2-25-2 RTB's, and is available for audit. Spare parts procurement would have to verify compliance with these specifications as well as certify complete functional equivalency.

1. A Request for Quotation (RFQ) was issued by CE for a generic order of four (4) RTSG systems which invoked CE General Specification 00000-ICE-3008 Rev. 01 and CE general RTSG design drawi ngs.

2. Unit Electric Control, Inc. (UEC) among other vendors provided a response to CE Request for Quotation which was received during i,

the first quarter 1975.

3. CE perfomed a Bid Evaluation of the bidders that responded to the CE RFQ recommending Unit Electric Control, Inc. as the preferred vendor for the subject equipment.

4. CE issued a Master Purchase Order to Unit Electric control invoking CE General Specification 00000-ICE-3008 Rev. 01.

5. A supplemental quotation request was issued by CE to Unit Electric Control, Inc. modifying the requirements for packaging, shipping, receiving, storage and handling (i.e., the ANSI 4

N45.2.2-1972 requirements now imposed on the vendor).

6. Unit Electric Control, Inc. issued a response to the supplemental request and implemented the requirements of same.

7. Purchase Order Release #1 against the Master P.O. for San Onofre Units 2 and 3 was issued by CE to Unit Electric Control. This

Release Order invoked CE Project Specification 1370-ICE-3008 Rev.

! 01 in addition to the general requirements and requirements for seismic / environmental qualification.

8. Supplement 1 to the Master Purchase Order was issued by CE requiring deletion of coil status indication lights (which were deleted to enhance breaker control circuit reliability) and changing the RTSG heaters from 120V AC to 240V AC.

9. The following Request for Approval or Review (RAR), Technical

Change Request (TCR), Deviation of Contract Request (DCR) submittals were.made by Unit Electric Control against the SCE Unit 2 and 3 RTSG contracts.

a. RAR's 1 through 16 were submitted hy UEC during the time period October 1975 to April 1976. These RAR submittals pertained to Inspection and Test Plans, Drawings, IMQP, Qualification Plan and Results, Tech Manual, and Function Test Procedures and Results. All RAR's were approved hy CE

, for compliance with CE specifications.

b. TCR's 1 through 4 were submitted by UEC in October 1975.

TCR's 1 through 3 were not approved by CE and the requested changes were not accomplished. TCR 4 was approved and allowed a change in the CE specified painting requirements.

i c. DCR's 1 through 3 were submitted by UEC during the time period of October 1975 to February 1976. DCR 1 was disapproved by CE and the requested change was not allowed.

DCR 2 allowed the deletion of terminal block cover wire numberi ng. DCR 3 allowed UEC not to submit Progress Reports on RAR's. DCR's 2 and 3 were approved by CE.

10. A conditional Certificate of Equipment was issued in December

, 1975 wnich formed the basis for shipment of the RTSG's for both San Onofre Units 2 and 3. An unconditional Certificate of Equipment was subseqently issued in June 1976 to include CE review of the as-certified drawings, instruction manual and test data.

11. The following lists the as-shipped breaker configuration for SCE Units 2 and 3 hy breaker serial number and location.

Unit 2 Breaker Serial No. Location 256A4002-656-3 TCB-1 256A4002-656-15 TCB-2 256A4002-656-14 TCB-3 256A4002-656-29 TCB-4

, . 256A4002-656-1 TCB-5 256A4002-656-8 TCB-6 256A4002-656-26 TCB-7 256A4002-656-33 TCB-8 l

256A4002-656-18 TCB-9 Unit 3 Breaker Serial No. Location 256A4002-656-5 TCB-1 256A4002-656-14 TCB-2 256A4002-656-25 TCB-3 256A4002-656-17 TCB-4 256A4002-656-2 TCB-5 4

256A4002-656-7 TCB-6 256A4002-656-34 TCB-7 256A4002-656-45 TCB-8 i -

256A4002-656-12 TCB-9 The above are the original- AK2-25-2 breakers, shipped with the RTSG.

i

2. Acceptance Tests - Vendor Reactor Trip Switch Gear acceptance tests required and performed by the RTSG vendor were as follows:

A. As required by CE General Specification No. 00000-ICE-3008 Section 5.10.3 Bench Tests were performed by the vendor of the San Onofre RTSG prior to installing equipment into the RTSG cabinet. This testing included:

1. Tests to check defects in mechanical assembly

2. Tests to check defects in electrical wiring

3. Checks for grounds, shorts or open circuits

4. Actuation checks to ensure:

a. Proper closing operation

b. Proper tripping operation

c. Proper status indication as defined by breaker position

d. Proper functioning of auxiliary relay contacts The above testing was satisfactorily completed and documented.

B. In addition to the individual equipment bench tests described in (A) above the RTSG system underwent additional testing as required by CE General Specification No. 00000-ICE-3008 Section 5.10. This testing included:

1. Hi-Potential Testing,

2. Point-to-Point Continuity Testing,

3. System operational testing (this procedure is incorporated in the CE Instruction Manual to.' the reactor trip circuit breaker switchgear for San Onofre Units 2 and 3).

C. Quality Assurance inspection of workmanship as required by Combustion Engineering Quality Control Specification (00000-WQC-ll.1, Rev. C) was performed.

The above testing and inspection were satisfactorily completed, a  ;

Cercificate of Equipment was issued hy CE, and the RTSG was I delivered to SCE. l The following summary of the RTSG equipment qualification data and references reflects testing and analyses performed prior to delivery I of the RTSG to SCE. The follow-up seismic study hy Wyle Laboratories (Item 6) confirmed the adequacy of the previous seismic qualification.

(1) The RTSG was seismically qualified to the requirements of IEEE-344, Seismic Qualification of Safety Related Electric Equipment for Nuclear Power Generating Station,1971.

-_m . - --.m - -

l*

I'.

i

., l l' (2) The RTSG environmental test was documented in accordance with the requirements of IEEE-323, Trial Use Standard, General Guide for ,

Qualifying Safety Related Electrical Equipment for Nuclear Power Generating Station,1971.

4 i (3) San Onofre Units 2 and 3 Reactor Trip Circuit Breaker Switchgear >

l was seismically and environmentally tested / analyzed by Wyle j Laboratories, Huntsville, Alabama.

(4) The msults of the testing / analysis were approved by CE following

! the requirements of Combustion Engineering " Sellers Procedure for l Submitting Approval Review (RAR)" NPD-WI-6 Appendix.

! (5) Test results were defined in CEN(94)-S Data Sheet No. 3 for the RTSG seismic test documentation and in CEN(95)-S Data Sheet No. 3 for the RTSG Environmental Test documentation as referenced in Section 3.10 and 3.11 of the SONGS FSAR for NSSS equipment.

i (6) Additional Analysis information was provided to SCE by Wyle I l Laboratories Analysis Report No. 26321 Section 6.11 as msponse j to the NRC Seismic Qualification Review Team (SQRT) general j concerns for San Onofre Units 2 and 3.

1 j The RTB's am intended to be used in the reactor trip switchgear cabinets which are located in a non-harsh radiation and temperature envi ronment. Consequently, environmental testing includes only the i upper limit of the nonnal room temperature and does not test or i analyze for radiation effec The RTB are in an area that is j expected to be less than 10gs. rads total 40 year dose, so their

! components are considered qualified for this service. Room i temperature is mgulated by the plant HVAC. Any potential effect of

temperature or radiation on maintainable parts (such as lubricant) i will be detectable through the enhanced surveillance program 4 described later in this report.

I

3. Acceptance Tests - Startup l As described in the San Onofre Units 2 and 3 FSAR, the Startup Test Program includes requirements for premquisite and preoperational 1

testing of equipment including the reactor trip switchgear. The 2

preoperational testing is discussed in Section III.D.6 of this

re ) ort. The prerequisite testing of Reactor Trip Breakers was as i follows

1

a. All type AK2-25-2 breakers (18 total) were successfully tested i with Startup gen ric.. test GT-400-05. This was a generic ' air j circuit breaker test,'and. contained the following elements:

) 1. Meggering of insulation resistance to >50M ohms

! 2. Recording of X, Y, and trip coil resistances.

! 3.

j Recording)of (g_100 VDC ' minimum voltages to trip -(<J0'VDC) and close 4

i

.t

~.

. 4. Verification of correct functional operation of auxiliary and test switches.

5. Verification of correct mechanical adjustments of contact wipe and latch adjustment.

6. Visual inspection of an: chutes, barriers, control switches and wiring, racking and locking mechanism, all switches and wiring and hardware.

7. Closing solenoid resistance was recorded.

Although not specifically required by GT-400-05, it was SCE's practice to record undervoltage device pickup and dropout voltages during this test.

b. Equivalent prerequisite testing was perfomed on the four breakers which were procured as spares and subsequently installed i

as replacements on Unit 3.

The fifth spare breaker is untested and has never been installed.

4. Startup and Surveillance Tests The acceptance tests discussed above have been perfomed prior to placing RTB's (including RTB's initially supplied as spares) into service. In addition, the following preoperational, surveillance and maintenance tests were perfomed prior to the March 1 and March 8, 1983 surveillance tests:

a. Preoperational testing:

RTB's were tested as part of the Plant Protection System (PPS) preoperational test, PE-357-01, and the PPS response time test, PE-358-01, on both San Onofre Units 2 and 3. All RTB's installed at the time of the tests responded properly. These tests, however, trip the bmakers on combined UV and shunt device operation (as would occur during RPS protective action) and thus would not reveal UV problems.

1) The PPS preoperational test is a comprehensive test of the entire system through RTB operation and Engineered Safety Featums actuation (pump starts, etc.). The RTB test in i Section 8.10.2 of PE-357-01 is a functional trip test which '

looks for bmaker operation only, and does not time breaker opening or otherwise detail its operation. The Unit 2 preoperational test was perfomed on May 30, 1981, and tne Unit 3 preoperational test on August 25, 1982. No RTB failures occurmd.

2) The PPS response time test was perfomed on both Units 2 and 3 to meet the requirements of FSAR 14.2.12.72S. This test i recorded electronic response times from the sensor to the i RTB's or ESFAS subgroup relays, as appropriate. Mechanical response times of pumps and valves were measured in other

~

l.

}

i tests and added on to these results to yield total ESFAS response times. In the case of RTB's, the time from the de-energization of the matrix relays to the opening of the i RTB's was measured as a subcomponent of total RPS response i time. The acceptance criterion was <120 ms. The longest 3

time (last RTB to open) was recorded in each case; Unit 2 was tested in November 1981 with the longest time being 60 ms.

4 and Unit 3 was tested in August 1982, recording a maximum

) time of 100 ms. No breaker failures occurred during these

tests.

b. Surveillance testing i Surveillance testing of Reactor Trip breakers is perfomed in accordance with S023-II-1.1, " Plant Protection System - Channel Function Test," S023-II-3.1, -3.2, -3.3, -3.4, '?lant Protection System Channel Response Time Tests," and S023-II-11.161, " Reactor

. Breaker Undervoltage and Shunt Device Circuit Test." These tests j are perfomed on the RTB's installed in the switchgear cabinets, i and on replacement RTB's as discussed below:

i

, 1) The PPS Channel Functional Test is a comprehensive test of i the protection system from sensor output through RTB

] operation and Engineered Safety Feature actuation similar to i the PPS preoperational test discussed previously in this i report. This test is perfomed at least once per month in accordance with the Technical Specifications, and for the i RTB's is a functional trip test using combined undervoltage i and shunt trip operation. As operation of the shunt or UY

! device satisfies protection system design criteria, this test

{ verifies RTB function in accordance with design requirements. The RTB's have never failed to trip during this test.

4 l 2) The PPS Channel Response Time Tests measure electronic

] msponse times from the sensor to the RTB's or ESFAS subgroup relays, as appropriate. Mechan'ical response times of pumps 4 and valves were measured in other tests and added on to these

! results to yield total ESFAS response times. These four test l procedures (one per Channel) are essentially identical to the preoperational response time test discussed previously in

, this report. As stated previously no RTB failures were

experienced during response time testing.

3) The RTB Undervoltage And Shunt Device Circuit Test separately tests the UV and shunt device trip functions. This test is

! required to be perfomed at least once per 18 months and

following maintenance or adjustment of the RTB's, in l accordance with the Technical Specifications. On Unit 2,

! this test was initially performed monthly and after any

! repair or replacement of RTB's. Following correction of UV trip failures on 4 Unit 2 RTB's in early 1982 and successful

,n.~

l

l. .

completion of 2 more monthly tests on each RTB, SCE reduced the frequency of this test to that specified by CE and the Technical Specifications. Two additional Unit 2 test

' failures were observed in July 1982 following UV coil adjustment. An RTB functional test was performed to verify function in accordance with the criteria for plant protection system design. The separate UV and shunt device surveillance was not completed before resuming the rod drop tests and LER 82-176 was submitted to document this occurrence. The surveillance test was successfully completed in July 1982 (Unit 2) and in August,1982 (Unit 3). An additional 4

surveillance test was perfomed on the Unit 3 RTB in October 1982. There were no further UV device surveillance tests until the March 1 and 8,1983 surveillance tests discussed i later in this report.

c. Maintenance testing Maintenance, including preventive maintenance (PM), corrective maintenance and testing as part of maintenance is perfomed in

accordance with Work Orders. The maintenance relating to Reactor Trip Breakers is discussed below

1) The preventive maintenance program was developed from vendor, NSSS and Engineer-Constructor (CE and Bechtel) experience and recommendations. MPEG-025 is a generic overhaul procedure 4

for 600 volt and below air circuit breakers. Work in l accordance with this procedure has not been perfomed on the

, RTB's. WES-008 is a specific maintenance procedure for the RTB's developed in response to IE Bulletin 79-09. This procedure addresses overall lubrication and other general maintenance items as well as specific steps pertaining to UV design in accordance with the guidance in IE Bulletin 79-09.

MPES-008 was perfomed on the RTB's originally installed in Unit 2, in March through May 1981 but has not been perfomed on the RTB's originally installed in Unit 3 or on RTB's supplied as spares. Preventative maintenance planning had

scheduled this to be done at the first refueling on the basis

! that the RTB's, as delivered, would not require PM before j that time. Maintenance procedure S023-I-4.36 was developed from MPES-008 for use at first refueling but has been cancelled because, through an administrative error, S023-I-4.36 was initially issued with key infomation from W ES-008, deleted. This infomation has been added to a new procedure (S023-I-4.66) which also incorporates the recommendations in Sections IV.D.2 and V.A of this report.

Although there is no Technical Specification requirement to perfom a baseline PM for all equipment, it is SCE's pracitee to do this and it was done for the Unit 2 RTB's. For the Unit 3 and spare RTB's, however, SCE relied on the delivered condition of the RTB's and did not specify PM until the first refueling. -

~.

. 2) Corrective maintenance on the RTB's was performed in response to problems documented in Non-Conformance Reports (NCR's); a Work Order is then written against the NCR to correct the problem. For the Unit 2 RTB undervoltage trip failures in early 1982, the NCR was dispositioned to correct the problera per vendor direction, based on a perception that failure of j the UY device to function reliably made SCE procedure i

MPES-008 suspect (it had been used earlier for PM on these RTB's) and that vendor assistance was needed to ensure that

no unusual problems existed with these safety related components. The Work Order specified that the undervoltage coil pickup voltages be reset to vendor technical manual (S023-944-352) criteria by SCE electricians under vendor

. (General Electric) supervision. For the July 1982 RTB adjustment to facilitate breaker closing, SCE electricians reset the undervoltage coil adjustments on all Units 2 RTB's to vendor technical manual criteria without vendor assistance. When two RTB's subsequently failed the July 1982 undervoltage surveillance tests, CE recommended that a vendor service representative check the UY coil adjustments for all Unit 2 RTB's. The UV coil adjustments were then reset (some outside the vendor technical manual limits) by the vendor representative to obtain suitable RTB (close and UV trip) operation and the NCR was dispositioned to accept these settings as-is, based on the perception that the vendor representative-determined settings were the best technical information available. For the Unit 3 and spare breakers, a Work Order had been written to perform both MPEG-025 and MPES-008 maintenance procedures, but was then cancelled in favor of work orders to adjust per vendor technical direction consistent with Unit 2. Following these activities, the surveillance test was repeated on all RTB's for Units 2 and 3 with satisfactory results.

Further discussion of the above maintenance process is provided later in this report. ,

1 l

4 e

y ,,,- - e-- p- --, .,n --- -e , --,~,--,,--g-------- ,,-y,-,. n ~-,y-,m---- --,~w--

. __ _ . ~ _ . _ _ -_ _ . _ . - . _ _ _ - _

~

j. s

-1 9-l ..

l D. March 1 and 8,1983 RTB Surveillance Tests i

j 1. Description of Tests j

The NRC issued IE Bulletin No. 83-01 " Failure of Reactor Trip i Breakers (Westinghouse DB-50) To Open On Automatic Trip Signal" on

! February 25, 1 983.

I Even though IE Bulletin No. 83-01 did not require testing of the

! General Electric RTB's installed at San Onofre Units 2 and 3,

! surveillance testing was perfomed in accordance with station

! procedure S023-II-11.161 " Reactor Breaker Undervoltage and Shunt

Device Circuit Test." This procedure tests the UV device and shunt j trip device independently as described in Section III.D.4 of this report.

j i 2. Test Results

! Testing was perfomed on all Unit 3 RTB's on March 1,1983. One breaker, RTB 4, type AK2-25-2 serial #256A4002-656-17, failed to trip

on its undervoltage test. It perfomed properly on shunt trip.

NCR-3-243 was written to document the nonconfomance. Since this was viewed as an isolated failure, and the CEDM cabinets were not energized, the failure was not considered to be reportable pursuant i to the Technical Specifications at this time. Testing on Unit 2 was j delayed by the presence of a failed power supply in the PPS. After the power supply was repaired, testing of Unit 2 RTB's connenced on March 8,1983. This testing revealed three RTB's (all type AK2-25-2) which did not trip on undervoltage:

RTB# Serial #

1 256A4002-656-3

4 256A4002-656-24 6 256A4002-656-8 All breakers successfully passed their shunt trip test. NCR-2-163 was written to document these failures and the NRC was notified of 8

the surveillance failures.

4 3

i a

l .

f

• l IV. Investigative Programs As a result of the March 1 and 8,1983 surveillance failures of the GE Type AK2-25 RTB at San Onofre Units 2 and 3, SCE conducted an investigation to determine the cause and ramifications associated with the surveillance failures. This section of the report discusses SCE's efforts in the following specific areas to detennine the root cause of the problem and to identify specific corrective actions and follow-up activities for resolution of RTB concerns and related programmatic deficiencies:

A. RTB Investigative Tests

1. Description of Tests Following the Reactor Trip Breaker surveillance tests of March 1 and 8, in response to IE Bulletin 83-01, SCE initiated further investigation and testing of San Onofre Units 2 and 3 Reactor Trip Circuit Breakers. This further investigation and testing has been perfonned in three parts: (a) initial investigative tests March 12-17, 1983 at San Onofre, (b) in-depth testing March 26 - April 1, 1983 at the SCE Electrical Test Laboratory in Alhambra, (c) test results of nine RTB's at San Onofre Unit 2 obtained during baseline preventative maintenance and independent testing by Franklin Research Center (requested by NRC). A discussion of investigative test parts a, b, and c follows; part d will be provided directly to the NRC by Franklin Research Center.

a. Initial Investigative Tests (March 12-17, 1983)

1) Following an introductory meeting with representatives of NRC and Franklin Research Center, SCE initiated an investigative plan for detennining problems with Reactor Trip Circuit.

Breakers. A procedure was developed and made available for review and comments by NRC and Franklin Research Center.

Comments were provided by the above organizations for SCE consideration and were factored into the final investigative procedure.

2) A General Electric Company field service engineer, who had previously provided vendor assistance on Reactor Trip Breakers, was called in. In addition, Combustion Engineering provided contact with the General Electric Company to request the assistance of a factory expert in the investigation of the Reactor Trip Breakers. The General Electric Company provided Mr. Max B. Fornwalt, Senior Project Engineer, who arrived on Sunday afternoon, March 13, 1983. It should be noted that Mr. Fornwalt authored the General Electric Service Letter that subsequently became part of IE Bulletin 79-09.

l l

3) Following the assessment of comments, a final investigative procedure was prepared. Work Orders were generated to implement the investigation. Reactor Trip Breaker TCB2 (functioning breaker) was tested to verify the adequacy of the investigative procedure. Reactor Trip Breakers TCB1 and TCB6 (two of the three malfunctioning Unit 2 breakers) were then tested with the same procedure. Each of these breakers )

was visually inspected and tested for undervoltage trip while  :

still installed in the switchgear cubicle, then removed and tested for trip shaft torque, undervoltage trip response time, and undervoltage device pickup voltage. The RTB lubricant was revitalized as necessary and changes were made in the UV pickup voltage levels.

4) The observations of the initial investigative tests are as follows:

a) Two breakers were missing the locking wire for undervoltage coil pickup voltage adjustment. One of these (malfunctioning breaker TCB1) had a lower than nominal pickup voltage (101 VDC vice 106 VDC) and the other (functioning breaker TCB2) a higher than nominal pickup voltage (112 VDC vs.106 VDC).

b) The as-found trip shaft torques all exceeded the 1.50 pound-inches specified by IE Bulletin 79-09. The as-found torque was slightly higher in TCB2 (functioning UV device trip) than TCB1 (malfunctioning UV device trip) but, as noted above, TCB2 also had a higher than nominal pickup voltage; TCB2 also exhibited erratic behavior (slow trip) when its pickup voltage was lowered to 100 VDC.

c) Both malfunctioning breakers tripped satisfactorily on repeated undervoltage trip tests with as-found or lower undervoltage pickup adjustments when the bearings lubricant was revitalized to reduce trip shaft torques to less than the 1.5 pound-inches specified by IE Bulletin 79-09.

d) The shunt trip device successfully tripped the breaker if the undervoltage device failed to do so.

e) The GE Factory Representative stated that he could find no evidence of improper handling or mechanical damage to the Reactor Trip Breakers examined. Further, all mechanical adjustments were satisfactory with the exception of undervoltage device pickup voltage and a ,

minor increase needed in the TCB6 overtravel adjustment 1 (made after successful testing of the breaker).

T ,-

i.

. 5) Preliminary conclusions of the initial investigative tests were as follows:

a) The major contributing factor to improper Reactor Trip Brear.er operation on undervoltage was due to insufficient or degraded lubricant in the trip shaft and latch roller bearings.

b) The secondary contributing factor was the undervoltage device amature pickup voltage being set below the recommended 106 VDC.

b. In-Depth Tests (March 26 - April 1,1983)

1) Based on the results of the initial investigative tests discussed in Section IV. A.1.a above, a more detailed inspection was perfomed on one of the Reactor Trip Breakers to obtain as much quantitative infomation as possible with regard to the dynamic operation of this breaker under various conditions. The breaker subjected to this in-depth test and -

inspection was Reactor Trip Breaker TCB-4 from Unit 2, the third of the three breakers which malfunctioned during surveillance testing in early March,1983. The other two malfunctioning breakers (TCB1 and TCB6) had been previously tested and readjusted as discussed in Initial Investigative Tests,Section IV. A.1.a of this report.

2) The work was perfomed by SCE at our Electrical Test Laboratory with the aid of a General Electric service representative. The breaker. was tested on a bench and when operational tests were made, the breaker was secured to the bench by the same breaker flanges which support the breaker -

when installed in the cabinet. High speed photography was used to assess breaker trip perfomance. A magnetic oscillograph was also used to record the following parameters as required for dynamic tests on the breaker, o Breaker main contacts (3) o Shunt trip coil current o Shunt trip coil voltage o Closing coil current 4

o Undervoltage trip device current o Undervoltage trip device voltage o Auxiliary "b" switch contact

3) The breaker was visually inspected and. tested in the as received condition and baseline measurements were obtained 1 for all parameters, including undervoltage coil pickup and i

dropout voltage, trip shaft torque, trip response time, undervoltage coil armature air gap, and all electrical component resistances. Ad,justments were varied and tests repeated to determine optimum settings and limitations. The

m _ _ _ . . . ___ _. - _ __ . . . , - __ _ _

, s

., I investigation also included inspecting, cleaning, and

revitalizing the trip shaft and latch roller bearings, and

cleaning and adjusting the undervoltage device. UV device pickup voltage was also investigated.

4) The results of the in-depth investigative tests were as l follows:

a) As-received trip shaft torque was greater than the 1.5 inch-pound limit and as-received undervoltage device pickup voltage lower than the 106 VDC limit specified by i IE Bulletin 79-09. Cleaning and revitlizing the trip

shaft and roller bearings reduced the trip shaft torques to less than 1.5 inch-pound and successful UV operation was obtained. This confims the preliminary conclusions of the initial investigative tests discussed in Section

IV. A.l.a of this report.

! b) Considerable variation of the UV device pickup voltage setting will result from variations in the UV device coil

, temperature during pickup voltage adjustment. A minimum of 30 minutes is required for the UV device coil to reach j a stable themal state.

3 c) The optimum adjustment for undervoltage device amature

, pickup voltage is 106 + 2 VDC at a " cold" UV device coil 4 temperature of 700 to ISOF.

i

d) The diode installed across the UV device coil for surge l protection of the PPS relays delays the breaker response a

time (nominal 30 millisecond difference) although it i

remains within allowable values. This diode is installed

! in the reactor trip breaker cubicle wiring and is not present on a removed breaker. Therefore, a diode is l required during Reactor Trip Breaker bench testing.

j e) Excessive clearance between the UV device armature magnet

! and restraining rivet reduced the effective throw of the i

amature by pemitting it to move up against the rivet i

rather than rotating. The as-found clearance of O.018 inches exceeded the 0.001 to 0.010 range recently

recommended by the manufacturer; a somewhat narrower l range (0.003 to 0.006 inches) will provide improved

! perfomance and is consistant with the G.E. factory range i of 0.001 to 0.010 inches.

I J

f) As-received condition of the trip latch roller bearing

, (rough operation and excessive clearance) may have i

resulted in variation of trip shaft torques with roller position but did not affect trip reliability when pickup 4

voltage and trip shaft torque were within desired range.

m,-. , _ , , . __.m y--._ -,-___%- _ - , _ , _ , . . - , - . , _ , . , , . ~ _ _ _ . ._,,,m., , . , , , _ . _ - , , . , , , _ . , . . . _ , . . . , . , _ . _ - . , , , , , _ , , . . - - - ...-.-.m-

g) There is ample design margin in the shunt trip device.

Operation of the breaker with the shunt trip was satisfactory down to approximately 30 VDC; the voltage available is nominally in excess of 130 VDC.

h) The undervoltage response time of the breaker is faster and more consistent for a well lubricated breaker than one with degraded lubricant.

5) The conclusions of the in-depth investigative tests were as follows:

a) Two major contributing factors to improper Reactor Trip Breaker operation on undervoltage are degraded lubricant and undervoltage device amature pickup voltage adjustment; this confims the preliminary conclusions of the initial investigative tests discussed in Section IV.A.l.a above, b) A third major contributing factor to improper RTB operation on undervoltage is armature / magnet / rivet clearance in the UY device. (This may have been isolated to this breaker).

c) A minor contributing factor to breaker response time variation could be damage to the trip latch roller bearing.

d) The undervoltage device amature pickup voltage should be adjusted to 106 + 2 VDC at a " cold" UV coil temperature of 700 to 850F. --

e) The UV device dropout voltage should be measured (after pickup voltage adjustment) with the coil energized a minimum of 30 minutes.

f) The Reactor Trip Breaker response time should be measured on clean, lubricated and properly adjusted breakers to establish a baseline. Further response time tests during breaker surveillance would detect any degradation of

, breaker perfomance from the baseline, thus identifying potential incipient failures.

g) The Reactor Trip Breaker response time on the bench should be measured with a test diode connected across the UV device coil (testing in the cubicle would have the installed diode in the cin:uit).

h) With preventative maintenance as dictated by the results of UV response time tests during enhanced surveillance, there is adequate design margin in the UV devices; no changes are needed to the undervoltage trip design.

. 1) There is ample design margin in the shunt trip device; no changes to the shunt trip design are needed.

c. During the period April 6th through April 10th,1983 preventative maintenace (PM) procedure S023-I-4.66 was implemented to maintan the breakers and establish the base line preventative maintenance data for Unit #2 reactor trip breakers.

1) As Found Yalves a) On six of the nine breakers, the undervoltage device armature to rivet clearance was not within the required range.

b) Seven of the nine undervoltage device pickup voltages were less than the required 104 to 108 volts.

c) Six of the nine trip shaft torque valves exceed the required 1.5 in-lbs.

2) Final Post-Maintenance Base Line Valves a) All trip torque valves were less than 1.26 in-lb. (most were in the range of 1.0 to 1.1 in-lbs. )

b) All RTB's trip times were less than 70 msec.

c) No other bearing problems were found. It is therefore concluded that the bad bearing found on TCB-4 during the in-depth investigation was an isolated occurance.

d) All TCB's operated satisfactorily in all respects following completion of the preventative maintenance.

2. Results of Investigative Tests The conclusions of the investigative tests performed by SCE are discussed in Sections IV. A.1.a and IV. A.l.b of this report.

Pertinent recommendations based on these conclusions are provided in Sections IV.D, V. A.1 and V. A.2 of this report.

l B. Evaluation of Licensee Administrative Procedures (Later) i 1

1 4

J h

I

l'

. C. Evaluation of Capability to Mitigate ATWS

1. Procedures for Mitigating ATWS Prior to the March 1 and March 8,1983, events regarding the Reactor Trip Breaker undervoltage trip function, Emergency ,

Operating Instruction (E0I) S023-3-5.1 " Emergency Plant Shutdown" (Revision 8) (this procedure is for any plant trip situation and l includes the plant's ATWS procedural steps), included an l instruction in the first step of the "Immediate Operator Action" section requiring operators to verify that all Reactor Trip Breakers are open and reactor power is decreasing. If the reactor is not tripped, the procedure then requires that all four manual reactor trip pushbuttons be actuated. In addition, the

following specific ATWS actions are called for:

a. De-energizing load centers B15 and B16. This interrupts power to the control element drive mechanism motor generator sets and therefore, removes power from the CEDM's regardless of Reactor Trip Breaker position.

i

b. Manually initiate emergency feedwater actuation signals.

c. Initiate emergency boration.

Revision 9 of E0I S023-3-5.1 was issued on March 25,1983, and implemented additional ATWS steps which require the initiation of a manual reactor trip when an automatic trip set point is being approached or has been reached regardless of whether the reactor has tripped automatically and to manually trip the turbine.

2. Control Room Layout / Design The control room 1 gout drawing (Figure IV.C.2-1) identifies control board locations of indications and instrumentation which can be used to identify whether or not an ATWS event.is occurring and controls available to aid in the mitigtfon or prevention of such an event. All locations depicted on Figure IV.C.2-1 are numbered relative to the sequence in which they are contained in the "Immediate Operator Action" section of Revision 9 of E0I S023-3-5.1, " Emergency Plant Shutdown."

Figure IV.C.2-1, as drawn, shows only the San Onofre Unit 2 side of the Control Room, but with the "same hand" arrangement of the control room indication and controls identified in this drawing are located as shown in either the Unit 2 or Unit 3 control room area.

l As shown on Figure IV.C.2-1, the indications and instrumentation I which can be used by operators to detect and initiate ATWS, are )

easily accessable to operators.

, l

4 89 * '

e' E 4

h RpS Remote Operntnt's Hodule-2 or sore channels indicate trip Depress Ry trip pushbuttons

. . . . . . . . . . . . . . . . 82A 2n Depress RX trip pushbuttons

, 5

,h h- - - - - - - h------ *

's, '

h Renctor t rip bkr status panel verify all 5 TCBs open i3 ,' s, 's, h Excore Ifnear power recorders-

! s, '. verify power <6% T.p.

\,

,' f' , , h CEAverify Bottomed Indication -

all N bttom W u on

' HA ' ,

',' nnuncintoi

, Bh 's ~~--"~~, De. energize HCC Bl5 l , s

' - -- pan e.l.. .

.- De-energize HCC B16 UM NC' ME I-T e  ! Initiate EFAS 1 & 2 I 8 e Initinte EFAS I & 2 tmlT 2 I l h Initiate emergency boration:

, e

- Pumps

! by h- CVCS Valves & Blend Cont rol e h Emerg. boration path

's, h Cravity feed valves be bo miion

.  %, ', Manually trip turbine

.'s Verify feedvater flow to S/Cs (MTNi M/ev ATW) s 's.

's '

• Annunciator panel -

• [ . \ ' 5DA38 par Safety vv aut. Temt.HI

^ 50A49 CEDHCS Bus Undervoltage

's ~, JLesk .

^

' . . kU)--. .. kGn) --.. . . .

- h Baronometer - verify RCS between

.tancL .. __.. ___ ... .-. 1750 and 2250 ppmB before stopp-

. Inr. emergency boration

. Amum FIGilRE IV. C. 2-1 CONTROL ROOM LAY 0llT/ DESIGN ATWS INSTRUMENTATION /CONTR01.S

\~

O

. 3. Operator Training / Knowledge 1 i

Following the issuance of IE Bulletin 83-01, on-shift training sessions were conducted for each operating shift reviewing the

. ATWS operations included in E0I S023-3-5.1 " Emergency Plant Shutdown." In addition, each licensed operator was required to review the IE Bulletin referenced above and IE Bulletin 83-04, when it was issued. After the revision of E0I S023-3-5.1 on l March 25,1983, all licensed reactor operators were required to review and acknowledge, in writing, this review of the revision to the procedure. In addition, the five operating shifts will receive, as part of the requalification training program, formal classroom training and discussion relative to the revised procedure. This will occur over a five-week period, which started the week of March 28, 1 983.

i 1

.l j

e 1

4 4

~

i i  !

1

-2 9-D. Findings / Conclusions Regarding RTB Failure

1. The reactor trip breaker (RTB) undervoltage (UV) trip device is not required for the San Onofre Units 2 and 3 reactor protection system (RPS) to perform its design basis protective function; failure of the UV trip device does not affect the ability of the San Onofre Units 2 and 3 RPS to perfom its protective system function due to the presence of the shunt trip.

2. From the results of the investigation of four RTB's discussed in Section IV.A of this report, it is concluded that the cause of the unreliable operation of the undervoltage (UV) trip function for the Reactor Trip Breakers was a combination of the small design margin in the forte provided by the UV trip device and the following:

o Degraded lubricant on the trip shaft bearings or latches.

t o Incorrect setting of the UY device pickup voltage.

o Excessive clearance in UY device armature hinge area.

3. Despite the small design margin in the force provided by the UV trip device, the UY trip feature of the RTB can be made to operate reliably when enhanced maintenance and surveillance techniques are

! used. This conclusion is based on the results of the tests and l inspections of three RTB's at San Onofre, the results of in-depth testing of one RTB at the SCE Electirtal Test Laboratory and the results of the review of RTB maintenance and surveillance history.

The enhanced maintenance and surveillance needed to ensure reliable operation of the UV trip feature, based on these results, are as follows:

i . .

a. Ma'iritenance The enhanced maintenance procedures for RTB's should include the following:

1) Trip shaft torques UV device pickup voltages, and UV opening

times should be detemined for the breakers in their as-found j condition:

a) Measure and record the pickup voltage of the UV device with its coil surface temperature within 70 to 850F.

Also, measure the pickup and dropout voltages after the UV device coil has been energized at 130 volts DC for

thirty minutes. Ambient and coil surface temperatures

should also be measurep and recorded.

?

l.

1

b) Measure the opening time of the circuit breaker with full loss of voltage to the UV device. This time is to be measured from the loss of DC control voltage and to be breaker main contact part. Three operations and measurements are to be made using an oscillagraph recording device, with a test diode connected across the i UV device coil.

c) Measure and record the trip shaft torque which is required to trip the circuit breaker. This measurement is to be made three times using a specially made tool to 4 fit on the trip shaft. s

2) Prior to the current plant restart, the RTB should be disassembled to the point where the operating mechanism can be removed. The trip latch roller should be inspected for damage and all bearings and bearing points should be checked for mechanical wear and freedom of rotation. Clean all parts with isopropyl alcohol. Apply CRC 5-56 to all bearings and

, bearing points. All excess CRC 5-56 should be removed to i

prevent the collection of dust and contaminants.

3) Remove the UV device to pemit inspection and to check the adjustment of the clearance between the UV device armature 4

and the rivet (0.003 to 0.006 inches). The mechanism portion is to be cleaned with isopropyl alcohol and then apply CRC 5-56 to all pivot points. Excess CRC 5-56 should be removed.

~

4) After reassembly of the circuit breaker, the mechanical adjustments should be checked and adjustments made as required.

5) Repeat the circuit breaker perfomance measurements:

a) Check and adjust the pickup voltage of the UV device when its coil surface temperature is within 70 to 850F.

This should be set to 106+ 2 VDC. Also, measure the pickup voltage after the lh device coil has been energized at 130 volts DC for thirty minutes.

b) Measure the opening time of the cin:uit breaker with full loss of voltage to the UY device. This time is to be measured from the loss of DC control voltage and to the breaker main contact part. The criteria for the main contact part shall be approximately 70 milliseconds or less subject to confimation by baseline testing of all breakers. Three operations and measurements are to be made.

c) Measure and record the trip shaft force to trip the

circuit breakers. This force shall not exceed 1.5 pounds-inches. This measurement is to be made three times using a specifically made tool to fit on the trip shaft.

i .

i.'..

i~-  !

I

b. Surveillance

! Enhanced surveillance for RTB's will be made to monitor the performance of the circuit breakers for indications of degrading l

a UV trip operation, in order to detennine the need for (and to i pennit perfonnance of) restorative maintenance before UV trip

! reliability is affected. The enhanced surveillance for RTB's l includes the following:

i 4

1) After the circuit breaker maintenance is completed, a surveillance test should be made. This should include three UV opening time tests. The opening times for the three tests

should not exceed the allowable limits shown in Item 5b under a maintenance, above.

j 2) Repeat the surveillance teet four more times at approximately 30-day intervals (i.e. , monthly).

j 3) After four sequential months of UV trip function surveillance i testing without UV trip failure or degradation, the i surveillance testing interval may be increased to at least j once per 62 days (i.e., bimonthly).

1

4) After eight more sequential months of successful UV trip testing, the surveillance testing interval may be increased to at least one per 92 days (i.e., quarterly).

5) After 12 more consecutive months of successful UV trip testings, the surveillance testing interval may be increased to at least once per 184 days (io.e., semi-annually).

6) If the average of the three UV trip times or the scatter of j the three trip times during each test increases significantly a from the baseline surveillance tests, or exceeds the l baseline-adjusted limits, the maintenance procedures should

be repeated and new surveillance tests initiated.

i

7) Surveillance and maintenance frequency should continue to be adjusted on the basis of surveillance and maintenance test results. The minimum RTB maintenance frequency should be one per 12 months (i .e. , annual PM). Unit 3 surveillance (and 1 maintenance) should be initiated at the frequency in effect j for Unit 2 at that time.

i 4. The reactor trip breaker UV device surveillance failures were due to the small design margin in UV trip force and the hardware condition

,resulting from the absence of the enhanced surveillance and inaintenance techniques which had not been developed or identified by the industry as needed at that time. This conclusion is based on the

review results discussed in paragraphs 2 and 3 above. Based on this l conclusion and the. fact that equipment difficulties are commonly experienced during the startup and construction phases, it must be i

) *,

I.% l

{ further concluded that incorporation of the potential improvements to d

SCE administrative procedures and implementations which have been j identified as a result of the reviews discussed in Section IV.B of this report might have reduced the likelyhood but would not have

precluded the UV trip function malfunctions observed.

! 5. A procedure which addresses ATWS was in place at San Onofre Units 2 l and 3 prior to the Salem RTB trip failure. This procedure, l S023-3-5.1, " Emergency Plant Shutdown" has been revised to improve operator response to an ATWS event by requiring immediate manual reactor trip and then manual turbine trip when an automatic trip i setpoint is being approached, regardless of whether the reactor has tripped automatically. The San Onofre Units 2 and 3 Control Room is i

already configured such that the instrumentation and controls to detect and mitigate ATWS are easily accessible to the operators.

6. The existing SCE operator training / retraining program contains sufficient provisions for promptly notifying operators of important

, procedural changes and for additional reinforcement through follow-on l training and retraining.

i 1

1 i

i I

j t i

! l

i .

IS l4  :

f V. Proposed Corrective Actions j Short tem and longer tem corrective actions are sumarized below based

! on discussions provided elsewhere in this report.

l A. Short Tem Corrective Action Leading to Restart j i

j 1. (Later)  :

i j 2. (Later) ,

I

3. Surveillance Procedures i

l As discussed in Section IV.A.1, surveillance testing of the RTB will i incorporate conclusions identified during the investigative tests j conducted by SCE. The details of this surveillance testing will be as described in Section IV.D.3.b of this report.

l If at any time during this surveillance program, degradation or failure of the undervoltage device occurs, the failure will be analyzed, corrected, and used to enhance the preventative maintenance procedure to prevent recurrence. Additionally, the surveillance testing program will be re-zeroed to the beginning of the interval i phase in progress to ensure successful correction of the problem prior to extending the test interval.

{

When Unit 3 attains initial criticality, its testing and maintenance will be initiated at the same periodicity in effect for Unit 2.

i The monthly P.P.S. testing under S023-II-1.1, which tests the RTB shunt and undervoltage trips together, will be unchanged.

4. Technical Specification i

SCE will review the need for Technical Specification changes after i the appropriate maintenance and surveillance frequencies have been

identified. ,

j 5. Operator Training l As discussed in Section IV.C. of this report, the " Emergency Plant t

Shutdown" procedure for San Onofre Units 2 and 3 operating i instruction for any plant trip situation including ATWS procedural steps has been modified to require the initiation of a manual reactor trip when an automatic reactor trip set point is being approached or j has been reached regardless of whether the reactor has tripped-automatically and to manually trip the turbine.

All licensed reactor operators have been reviewed and acknowledged in writing, review of the revision to this procedure. Additionally the five operating shifts will receive, as part of the requalification training program, -fomal classroom training and discussion relative i to the revised procedure. This will occur over a five-week period

which started the week of March 25, 1983.

l

T l *i -

34-l 4 SCE is confident that the procedural revision along with the review of the revised " Emergency Plant Shutdown" procedure which is being reinforced through the operator requalification training program will enabic operatcrs to detect and mitigate ATWS.

8 1

I i

i i

. .,--,--,,-v-- -, - , - ~ - . - -.-y-- - .--- - . ,. _ , ,__,-_y ,- - ., - , . , --c ------. . _ , .-- . . -_e ry_.-v ---- --

6 0-t 5 -

35-B. Long Tem Corrective Actions Subsequent to Restart (Later) l b

.e lw

a o l VI. Conclusions

! 1. From the standpoint of the reactor protection system design, the UV j coils are not required to operate in order for the breakers to trip i

under any design basis condition. In addition, the conservatice reactor protection system design at San Onofre provides added assurance (compares to similar vintage plants of other designs) that an ATWS event will not occur.

2. Exhaustive breaker testing in SCE's laboratories was performed on one i of the breakers that exhibited unsatisfactory performance. SCE was l able to do this evauation because of its experience and capabilities for testing of this nature that are largely unique to SCE. The experienced staff and facilities for such testing do not exist in most utilities throughout the country. This testing identified three key areas where additional attention needs to be devoted during maintenance and survillance. These are

! a) Lubrication of key components b) Perfoming UV coil adjustments at a known temperature c) Maintaining the UY amature to rivet clearance within recommended i

close tolerances Based on the results of the investigative testing, it is clearly understood how to adjust and maintain the breakers to provide a high degree of reliability for the UV trip function.

3. The proposed maintenance and surveillance programs will provide this high degree of reliability. The initial conservative frequency will be adjusted based on actual experience.

4. Based upon the above, and the fact that the shunt coil has never failed to trip the breaker, there is a very high degree of assurance that the RTB will operate if called upon to do so. Therefore, the health and safety of the public is assured and any and all restrictions limiting operation of San Onofre 2 and 3 associated with the RTB should be removed.

J 4

l

_