ML11242A081

MEng-SEng Report

ML11242A081
Person / Time
Site:	San Onofre
Issue date:	07/17/2008
From:	Southern California Edison Co
To:	Office of Information Services
References
FOIA/PA-2011-0157
Download: ML11242A081 (34)

Text

ISOUTHFMCAI IFORtNIA 7/17/2008 -2:19:33 PM EDISON SONGS - MEna-SEnq. Report:

ASSESSMENT OF THE IMPACT OF THE DEGRADED CONNECTIONS BETWEEN BREAKER 2D201 AND BATTERY 2B008 ON THE FUNCTIONALITY-OPERABILITY OF THE BATTERY 2B008 Executive Summary:

During a weekly surveillance test of battery 2B008 and normal charger 2B002 carried out on March 25, 2008, two degraded connections (one per polarity) were discovered between the battery and breaker 2D20 I.

Subsequent investigations yielded the following conclusions:

I. The probable cause for the degraded connections is the under-torquing of the connection bolts during the replacement of the breaker (2D201) carried out on 3/17/2004.

2. There is reasonable assurance that the degraded connections did not compromise the capability of the battery to perform its design-function until the end of the normal charger test on 3/21/-08. From 3/17/08 thru 3/21/08 the swing-charger was in operation while the normal charger was out of service for testing. This conclusion is supported by plant data which indicates the connection resistance likely changed state upon completion of the 3/21/08 normal charger testing. This conclusion is supported by test data previous to this date which includes weekly battery surveillances, two successful Integrated ESF surveillance tests, two successful LOVS surveillance tests, as well as data from the four day period when the swing charger supplied 112 amps to the 2D2 bus through the degraded connections (from 3/17/08 to 3/21/08).

3. Although there is no plant data to evaluate the condition of the connections during the period from 3/21/08 until the failed surveillance test on 3/25/08, it is likely that the degraded connections would have been capable of performing their design function from 3/21/08 until the degraded condition of the connection was discovered on 3/25/08. This conclusion is based on the fact a very thin layer of oxidation across the joint and/or a small gap would have been readily breached by the 120 VDC applied to it during a real event. The degraded connections may have impacted battery capability for coping with Station Blackout (SBO) design basis. Due to the lack of plant data, SCE conservatively considers the battery to have been nonfunctional sometime during this period.

4. Seismic events would not have prevented the battery from meeting its design basis as a result of the degraded connections.

File: Degradationof DC connection D201 I of 34

7/17/2008 - 2:19:33 PM JDISrAO An IOt$3 IMURA4'10f4l1. C "paný

Background:

On 03/25/2008 a weekly surveillance voltage test on 125 volt battery 2B008 failed its acceptance criteria. The test consists of verifying the battery voltage is at least 129 VDC when measured at the battery terminals. During the test, the measured voltage was approximately 121 VDC. During subsequent troubleshooting activities, degraded connections (one per polarity) were found on the battery side of the 2D201 breaker which connects the 2B008 battery to the 2D2 bus. Specifically, the bolts connecting the terminals of the 2D201 breaker to the aluminum buses leading to the battery were not fully tight. The battery terminal voltage recovered to 131 VDC when a voltage probe handled by the electricians was touched to one of the under-torqued connections during troubleshooting.

This report evaluates the impact of the under-torqued connections on the associated DC system, and the ability of the 213008 battery to perform its credited safety functions during the period from 3/17/04 to 3/25/08. This report is not intended to address human factors, which are discussed in a separate RCE.

Definition of Terms:

LOVS: Loss of Voltage Signal ESF: Engineered Safety Features SIAS: Safety Injection Actuation Signal DGVSS: Degraded Grid Voltage with SIAS Signal SBO Station Blackout ADC DC amperes VDC DC volts Factual Information:

1. The surveillance test that failed was carried out on 03/25/08 (MO #080301117). The measured value was 121 VDC. The required minimum is 129 VDC.

2. Previous surveillance tests were carried successfully on 3/18/08 and every week before that. A complete table of weekly surveillance tests is included in Attachment 2.

3. Two Integrated ESF Surveillance Tests and two LOVS Surveillance tests were successfully completed during the four year period between 3/17/04 when the 2D201 breaker was replaced, and 3/25/08 when the weekly surveillance test was failed.

These tests are designed to closely mimic the actual accident loading conditions on the battery bus. Successful completion of these tests provides a measure of confidence that the battery was capable of performing its design function. During approximately the first 15 seconds of these tests, the battery charger power is removed, so the battery is required to supply all required power to the bus through the impacted connection.

The data below shows how the voltage decreased on the 125 VDC supply at switchgear 2A06 during the period of time when the battery charger was off. Graphs File. Degradationof DCconnection D201 2 of 34

7/17/2008 - 2:19:33 PM l

EDISON SM!11111-RNCAI tFUR~NIA An EVIS N I*%TNA 47t0'4. COmpmn) obtained during the tests are included (Figures 1-3). To see where the voltage drop is measured, see Attachment 6.

Data is included from two ESF tests prior the 2D201 breaker replacement and two tests after the replacement. The data shows there was no significant change in voltage drop after the breaker was replaced. This supports a determination that the connection at the 2D201 breaker was sufficient to allow the battery to perform its safety function.

Data is also included from one LOVS test prior to the 2D20I breaker replacement and two LOVS tests after the replacement. The data shows one case with a slight increase in voltage drop (0.7 to 2.3 VDC). This occurred after the breaker was replaced. The measurement is affected by inverter-noise and has an uncertainty of approximately +/-

I volt, rendering the data inconclusive. It is judged that there would be sufficient margin in the battery design to accommodate a I to 2 volt change in voltage drop, based on the fact that the battery capacity test in 2008 determined a capacity near 100% while the calculation is based on a capacity of 80%. In the table below, all tests prior to the change of breaker 2D201 on March 2004 are noted as "TIGHT". Those after the breaker replacement are noted as "NOT TIGHT" (Refer to figure in Attachment 6).

Charger-off Connection's MO Date Test VDC Drop Time (sec) Condition 06122943 11/29/07 ESF 11.04 13.7 NOT TIGHT 05081829 1/6/06 ESF 10.97 13.4 NOT TIGHT 03031098 2/11/04 ESF 11.42 14.5 TIGHT 01061449 5/24/02 ESF 11.26 14.6 TIGHT 08010611 1/9/08 LOVS 12.32 15.0 NOT TIGHT 06011546 1/21/06 LOVS 10.70 14.8 NOT TIGHT 03031098 2/20/04 LOVS 9.98 15.8 TIGHT

4. The breaker 2D201 was replaced on 03/17/04.

5. The last time the bolts on the impacted connection were disturbed was during the breaker replacement on 03/17/04 [8].

6. The affected bolts were tightened to their specified torque during the repair activities following the failed surveillance test of 03/25/08, and according to the maintenance personal involved, each bolt was tightened using a torque-wrench.

7. Each bolt is fastened with two flat washers and a lock-washer.

8. Interview with the two craftsman that tightened the bolts after the problem was discovered revealed the following facts:

a. Each bolt was touched by hand while wearing insulated rubber gloves. Only one bolt was found completely loose.

b. Two of four bolts on each connection were snug enough that they couldn't be moved without the application of torque with a wrench. The remaining bolts could be barely moved by hand.

File: Degradationof DC connection D201 3 of 34

SOMI11RN CAI IFIONIA 7/17/2008 -2:19:33 PM EDISON

c. No evidence of sparking or pitting of bolts, washers and buses (restricted visual access only permitted a limited view of the connection).

d. All eight nuts were fully threaded in their bolts.

e. The condition of the grease in the contact area was not inspected.

f. Did not count number of turns applied on the nuts to effectively apply the specified torque. The ratcheting torque-wrench requires a number of strokes to move the nut a full turn.

g. Bench tests previous to the interview show that it takes a full turn of the nut to flatten a lock-washer.

h. When asked what is the typical manner they will use to tighten the bolts, one of them said:

- First by hand till the lock-washer is engaged,

- Then with a regular wrench,

- Finally with a torque-wrench.

9. Phone interview with the electrician that applied the Fluke probe to the degraded connection, revealed the following facts:

a. Several minutes after the weekly surveillance test failed (battery voltage measured below acceptable limit), and following a phone call, he reached the charger room to perform troubleshooting tests.

b. After taking a couple of readings on the charger side with satisfactory results, he touched the side of the head of one bolt of the degraded connection.

c. He applied little pressure with the probe and cannot say for certain that the bolt moved, although he thinks he perceived a very minor movement.

d. At the time the probe touched the head of the bolt, the sound from the charger or inverter changed, apparently indicating the loading changed.

e. Then he returned to the battery and now the voltage readings were satisfactory.

f. Looking at the busses and bolts with the panels removed, he could not see any signs of discoloration or other symptom related to high temperature on the busses and/or fasteners.

g. At about the same time an operator came to say that there was a Trouble Alarm on 2D2. The alarm was gone immediately (likely caused by a momentary ground imbalance when the battery was reconnected to the circuit).

10. Breaker 2D201 has a magnetic trip circuit, rendering it insensitive to temperature rise of its terminals due to abnormal heat generated at the degraded connection (if any)

(MO #04021613).

11. Swing charger 2B022 continuously fed 112 ADC thru the degraded connection between 3/17/2008 and 3/21/2008 (MO #08011467000).

12. The joint in question is designed for 1,200 continuous amperes of current (Reg. Dwg.

S023-302-5A-2). This is about four times the level of current this joint must carry continuously during an accident.

File: Degradationof DC connection D201 4 of 34

7/17/2008 - 2:19:33 PM J An 01(X0$%

SURIMtR 54 CA1IFORNIA SO

p. Y.R%4110% 41.

ON

13. A Ductor test on the re-torqued connections upstream of 2D201 carried out on 7/15/08 (NMO #800117881), show acceptable contact resistance of no more than about 10 Analysis:

The following questions are addressed in this section:

1. Most probable cause(s) for the degraded connection.

2. Was the degradation the result of a single event, or of a developing nature?

3. Did the degraded connection impact the ability of the DC System to perform its credited safety functions?

4. How would have a seismic event affect the integrity of the affected connections?

1. Most probable cause(s) for the degraded connection A bolted connection can become loose over time by a number of mechanisms. The table in Attachment I identifies the most probable mechanisms for looseness, and their relevance to the present situation: As shown therein, SCE concludes that the most probable cause is that the bolts were not fully tightened during the breaker replacement of March 2004, as required by the relevant procedures.

2. Was the degradation the result of a single event, or of a developing nature As explained in the previous section, the root-cause for the final onset of the degraded connection can be said to be due to a single event: the incorrect bolting activity on March 2004. However, due to the original oversight, there also was a potential for deterioration of the connection, given proper conditions; for instance, the drying out of the conductive grease applied between the busses during the breaker replacement. As explained below, a connection may lose pressure and still have a very low resistance. However, a connection which is not properly tightened may provide an opportunity for oxidation of the aluminum surfaces, suddenly increasing the contacts resistance to low voltage, which appears to be the case in the incident addressed herein.

The following provides the most probable explanation for the final failure of the weekly surveillance test:

During the March 2004 breaker replacement activity the electricians applied to the affected bolts enough torque to put some compression on the lock-washers, without fully tightening them. In doing so, they might have used a regular wrench, based on the fact at File: Degradationof DC connection D201 5 of 34

7/17/2008 - 2:19:33 PM J An frll,.S EDISON

'*%'TK'4f1(1\I.,

least two bolts per connection were found tight enough they could not be moved by hand (see item 8 under "Factual Information", above). The incomplete activity would have left the connections with a pressure less than required by the pertinent procedure. If a proper surface preparation was made, and anti-oxidant grease applied, this activity would have resulted in good contact (low contact resistance). In Figure 4, the anti-oxidant grease is visible on the edge of the bus joints. Also, as indicated in Item 12 under "Factual Information", the contact surface is designed with a current-carrying capacity of about four times the required under an accident, providing ample margin to carry the required current, even if torquing of all or some of the bolts did not meet existing procedures.

Between 3/17/08 and 3/21/08, the swing charger was placed 'in-service on 2D2 which caused about 112 amps to be supplied through the degraded connections. This resulted in increased heating of the connections for about 4 days, followed by a cooling of the connections when the swing charger was removed from service. Increased heating can accelerate the rate of oxidation on an under-torqued joint allowing the formation of a very thin layer of oxidation on the aluminum surfaces (about 50-120 angstroms thick [5]).

Increased heating can also accelerate the drying of the conductive grease, which can increase joint resistance. The normal charger output voltage is about 131 VDC and the.open circuit battery voltage is 120 VDC, so a developing layer of oxide or dried grease at the breaker connections would be exposed to a maximum of II VDC. This voltage may not be high enough to puncture a developing surface layer of oxidation or grease. However, if a real safety event were to occur, the entire 120V DC would be applied to the oxide layer. This potential is sufficient to break the thin layer of insulation and/or small gap recreating the conducting joint. Contact theory indicates that there is a threshold voltage, called the "fritting" voltage, associated with the breakdown of surface oxides or other contaminants between conducting surfaces. Below the fritting voltage, the surface materials act as an

.electrical insulator preventing current flow. Above the fritting voltage, electrical stress causes an avalanche breakdown of the surface insulator and conductivity is restored [4]. The fritting voltage necessary to break contact resistance can vary from several volts to several hundred volts depending on the thickness of the surface buildup, the pressure between the surfaces, the materials involved and other environmental factors. From relay applications, contact surfaces with less than 24 volts across them are known to be susceptible to high contact resistance from surface oxidation or contamination.

Based on calculations and instrument tolerances, the joint resistance until the end of the swing charger test of 3/17-21/08 is calculated to be between 0 to 8 mQ. The calculations are included in Attachment 3. The acceptable joint resistance value of 5.4 mQ (Attachment 4) is statistically within the range 0-8 mQ. A change in the total resistance of the circuit (attributable to the degraded connection) can be established as happening after 3/21/08.

Using "as-left" data for the charger voltage and voltage measurements taken on the battery terminals, and conservatively assuming (based on the fact the battery was fully charged) a charging current of no more than 0. 1 ADC, the calculation for the resistance of the degraded connection yields a value in the range of 0-5.7 Q at the time the regular charger took over from the swing charger (See Attachment 3). Statistically, this resistance is orders of magnitude higher than the connection resistance of 3/17/08.

File: DegradationofDC connection D201 6 of 34

7/17/2008 -2:19:33 PM I An FOISN EDISON~~

%' 4 M) (If . pan The change that happened with the reversal of current from the swing charger to the normal charger can be explained as follows:

Following the 4-day operation of the swing-charger (3/17/08 thru 3/21/08), the joint degraded. We believe the current flow thru the connection dried the grease on the contact surfaces and allowed the formation of an aluminum oxide layer. This layer of oxidation created enough resistance to withstand the very low potential differential between the charger and the battery. As explained above, this oxidation layer, and its corresponding fritting voltage would have been breached by the full battery voltage during a real event. A second possible cause (or in addition to the previous one), is that a small gap was created due to the geometry change following cool-down of the unloaded joint). The interview with the electrician that restored the circuit integrity by just lightly touching one bolt of the connection demonstrated that the oxidation layer or/and gap was indeed very weak/small.

3. Did the degraded connection impact the ability of the DC System to perform its credited safety functions?

The safety function of the 2B008 Battery is to provide power to the loads on the 2D2 Bus during three types of accident scenarios: Safety Injection Actuation Signal with Loss of Voltage Signal (SIAS/LQVS), Degraded Grid Voltage with SIAS Signal (DGVSS) and Station Blackout (SBO).

The design-basis bus loading during the various applicable scenarios is:

CONNECTION LOAD DURING SIAS/LOVS or DGVSS (normal charger on):

0-1 MIN = 474 ADC 1-90 MIN = 0 ADC CONNECTION LOAD DURING SIAS/LOVS or DGVSS (swine charmer on):

0-1 MIN = 474 ADC 1-90 MIN = 89 minutes of 193 ADC with one minute spike of 312 ADC SBO:

0-1 MIN = 341 ADC 1-30 MIN = 193 ADC with one minute spike of 312 ADC 30-240 MIN = 155 ADC with one minute spike of 239 ADC Evaluation of SIAS/LOVS and DGVSS scenarios (with normal charger in service):

During a SIAS/LOVS event or a DGVSS scenario, the battery is designed to provide adequate voltage to the 2D2 Bus loads for 90 minutes. Technical Specifications require periodic verification of the ability to supply the bus loads for 90 minutes by performing a Service Test on the battery.

File: Degradationof DC connection D201 7 of 34

7/17/2008 - 2:19:33 PM I i*

EDISON SOMMIl" (AI IFDIRNIA*

An if~(~JJrR1NVfA.4ff%41.- CMwnpv' During these scenarios, offsite power is lost and it takes approximately 10 seconds for Diesel Generator 2G003 to restore power to the 4KV system which will restore AC power to the battery charger. Once AC power is restored, the charger takes approximately 5 seconds to develop a DC output current due to the soft-start circuitry in the charger. Based on this, the 2B002 battery charger will be unable to support the 2D2 bus loads for approximately 15 seconds during SIAS/LOVS and DGVSS accident scenarios. During this 15 second period, the 2B008 battery would be required to supply the 2D2 bus loads through the 2D201 breaker with the degraded connections. During this 15 second period, the maximum design loading through the degraded connection is 474 amps for the first second and 340 amps for the remaining 14 seconds. Following this 15 second period, the battery charger will be restored to service. The charger is designed to carry 325 amps of load. The charger current supplied to the 2D2 bus will not pass through the degraded connections on 2D201 because the charger is connected to the 2D2 bus through a different breaker. All bus loading, up to 325 amps, will be supplied by the charger rather than the battery, once the charger recovers AC power.

During the period from 15 seconds to 60 seconds, the maximum bus loading is 338 amps.

Since the charger would assume 325 amps, only 13 amps would be supplied by the battery through the degraded connection. During the period from 1 minute to 90 minutes, the maximum bus loading is 245 amps so all of the current would be supplied by the charger and no current would be supplied by the battery through the degraded connection.

Based on this, the 2D2 DC System can perform its safety function if the degraded connections on the 2D201 breaker can carry the following current profile without adverse impacts: 464 amps for 1 second, followed by 340 amps for 14 seconds, followed by 13 amps for 45 seconds. Based on the short duration of this loading, it is concluded the degraded connection were capable of carrying the accident loading without significant degradation.

The connections passed all weekly surveillance tests (Ref Attachment 2) and four current-carrying tests [2] during the 4 year period of interest. The Integrated ESF tests and LOVS tests are designed to closely mimic the accident loading conditions on the bus This provides confidence that the degraded connections did not impact the ability of the battery to supply adequate voltage to the 2D2 loads during accident conditions.. From the list of tests indicated above, traces from three of the tests are presented herein:

• Figure 1 shows a trace from an ESF test carried out on 2/11/2004.

• Figure 2 shows a trace from an ESF test carried out on 11/29/2007, almost three years after the 2D002 breaker replacement.

" Figure 3 shows a trace from an LOVS test carried out on 1/28/2008, just a few weeks before the discovery of the degraded condition of the connections.

All three tests show time durations - when the battery charger was de-energized for approximately 15 seconds, and the degraded connections were conducting all of the 2D2 bus loads under simulated accident conditions. Item 3 in the Factual Information Section of this report contains a table which summarizes the results of the Integrated ESF Tests and LOVs File: Degradationof DC connection D201 8 of 34

7/17/2008 -2:19:33PM Ji ED ISON tests. The data supports a determination that adequate voltage is being maintained on 2D2 when accident loading is being supplied through the degraded connections.

Satisfactory weekly surveillance tests from the time of the breaker replacement on 3/17/04 until the one on 3/18/08 (Ref Attachment 2) indicate the affected connections, although not fully torqued during the maintenance activity of March 2004, were able to carry charging current without any problem until the last satisfactory weekly battery surveillance test, on 3/18/08. This indicates that the battery remained fully charged during this 4 year period.

Evaluation of SIAS/LOVS and DGVSS scenarios (with swing charger in service):

There was a 4-day period between 3/17/08 and 3/21/08 when the 2D2 bus was being supplied by swing charger 2B022 rather than the normal charger 2B002. The swing charger is connected at the battery terminals, so its output current would have been supplied to the 2D2 bus through the degraded connections on the 2D201 breaker. In this alignment, if an accident were to occur, the degraded connections would have to carry the entire bus load for the full duration of the event because the charger current goes through the degraded connections. The load profile is 474 amps for 1 minute, followed by 193 amps for 89 minutes with a 1-minute spike to 312 amps during the second period (due to a consideration of random loads). It is noted that during the 4-day period between 3/17/08 and 3/21/08 when the swing charger was in-service, it was supplying approximately 112 amps through the degraded connections continuously. Subsequent inspections of the connections during re-torquing on 3/25/08 and subsequent ductor testing performed on 7/16/08, revealed no evidence of significant degradation of the connections. Although the average accident loading is about twice as high as the 112 amp loading carried over the 4-day period, the accident loading lasts only 90 minutes. Based on the above, it is concluded that the degraded connections were capable of carrying the accident loading without significant degradation.

Evaluation of SBO Scenario:

During an SBO event, the battery is designed to provide adequate voltage to the 2D2 Bus loads for 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />. Verification testing for this scenario is not required by Tech Specs.

Analyses are performed to ensure the battery has adequate capacity to supply the 2D2 loads for 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />.

During this scenario, the battery charger (normal or swing) is not expected to be reenergized until the end of the 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> period so the battery would have to supply the bus loads for a 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> period. During this scenario the battery must supply the SBO load profile current (itemized above) through the degraded connection. Due to the 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> duration of this event, there is some possibility that heating of the connections would occur. It is concluded that this would not result in tripping of the 2D201 breaker within the four hour period given it has a magnetic trip not directly affected by the temperature at its terminals.

File: Degradationof DC connection D201 9 of 34

7/17/2008 - 2:19:33 PM IEDISON A. FM$(JV P~rFK%4j10%4. ,.pý It is noted that during the 4-day period between 3/17/08 and 3/21/08 when the swing charger was in-service, it was supplying approximately 112 amps through the degraded connections continuously. Subsequent inspections of the connections during re-torquing on 3/25/08 and subsequent ductor testing performed on 7/16/08, revealed no evidence of significant degradation of the connections. During an SBO condition the average current through the degraded connections would be approximately 161 amps. Although this is about 45% higher than the current carried during the 4-day period, the SBO current lasts only for 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />. Based on the above, it is concluded that the degraded connections were capable of carrying the SBO accident loading without significant degradation.

File: Degradationof DC connection D201 10 of 34 "5

J U)VIHFRN(AIFOM"iA 7/17/2008 - 2:19:33 PM Ani 11111% ASf1.%A1t0S.4L Ciifflri

f. q .M. FPI F'r W~fs:,. t-e34 I 2E~I4oaN3Ic'2I.5rI ZjLJ.20~. 314

• 0 ;LX09 .0,S kEE5T Ki-~

F M

- Ii 6t00.-

MC.A-I ?J*. ,'*1

,k0 I

off, ..uwd.f 11 154-s 1

.~ ft-r wt.w IF~-= Ia, 1 [3-I - 22rAW 2AfVA QL-4 2M607 PD24 PYfi

~ -$ 2A1W4 P016 Ma2 6O?AUCe L.:'dS PESFr 1430.0- J

1. ACWNiPD18 300.0- M~C- []2-2 2A~fli I PI 14 RA.1AC131335

£ M0.0-H2-6 11;-S ZAM4.1SIV5 START 02 7 2AO61E UAIDWSr P1

_ SUrIJ 1:.s SJ-0 ZA13618 UATCfMsr M 0.-1 X., rG-,

fli.j 2Na.0- H 31 EXI) -4x G 7.1i, J-'

3-7 SP.'BE

-m - x-A2ACf.4 w~rp.:Cý& vs HiHA21.-(C S i47 I2EK-C- --

4 16 1E4 I &2EH 4 2H.32-40T LASCCC J54i20 IGPDDX LO.W.,~ ISXO64.00 000 UbO.M 17.ZQ0 172C? 174OI~r MAN0 S B3MCE)%1X.TS I

1%.-7 iZ I E 133 64. SPR Fin:

22.3 ST8. P.!.. PC: ~ O -e Figure 1 - ESF test on affected circuit on 2/11/2004 before the breaker replacement. Test duration of the test about 15 seconds.

File: Degradationof DC connection D201 i11 of 34

7/17/2008 - 2:19:33 PM I EDISON SMtII4IRS (Al IOI1iA An ftQ.0f J%1:J#r%.41(0AL- C* pl 4.in " Matb FFT Fie tcisL . ~t MM1*I

_____PW 2A61 RAT 2AD61-5T-test-PD.D12rrG3SC1 JIM=7*D~. I:23 a ___________ ar. IS ..."dig 6=6 5ý.. d. T. VI.. fF-2Dj 1-0 JM1.W.3V501i 1-32ADNG P1375 ~ L-32AGV.1 P916 L-3ZAGVA PW S 4 2 AXA E4 ERO ~L-4 IAMW0 P525 L~2AMI P524 200.a-B 20 0AW 1.ZMA2 P12 RFSF-

[12.2 JAMf IP'134 U 23 2.AL.22 W1135 6 20*

a:wu []- AMI3J SIASSrAaR 0.0-H-2M0CGS OAT O$PST Ph 3 4 072 P 32 Me101214 setvei A i .342 36EI T432 ,

• 3CA.- .1-5 2806219071 V 11 Ik 400.0-

'42EP6faý2.32

,ffi AWL S628%43 MCC ID VXTS aRC I I 61.,V W~ V-4 WAA ?0.0C 112.00 K~oce 16.000 YsX 3Cx0 62.13W KfOM ft0.=

z32~ ~ *4~I I is se;;n-ýi- i Lý -

12.- &c2 & X lened . :e..Oa Figure 2 - ESF test on affected circuit on 11/29/2007 after breaker replacement. It is important to note no significant changes in the test results between this test after breaker replacement and the one in Figure 1, before breaker replacement. Test duration about 15 seconds.

File: Degradationof DCconnection D201 12of 34

7/17/2008 - 2:19:33 PM IEDISON 4.M ,n Haf f FT l Fir.kxel :A "m n~uecurmr - ~ ~'

Seic~

r~TIE~iJ LI fl..L4JflS~Afl.3LI

-LJ-jQV!S.sz1rL a:a L.~,

00 2X6£. LZA ETW.T I.- ;AV 3 rGRtK 2..2AR63 SW STAR Ii L3609 .Ov oys I'U r CA

~

-~~ 4 2XJIC43Lr#: gPE£ 4 R:I:1I 2 C 2.ACS04 LC1 FESE-

-12AOW4 PD13 Li-4 2MscT P011

'£0.0-122AC1L P114 J-. 242ACASS I.WTOHW£T P%

J- 1AW8PT CFF5TF%

Ajoo-0 a 20AE I V4 ALLA 3-5 250 is MY]

2 IRKc- r:-U ALL - )42"t'CC0) !g7 nD DC 22.15X J5.000 2? Xa .1J0A.0 U2.100 1S.G0B 34.3A .40.6 92AW0 1'U'6 -47D W.600 S4td

~3~2 __

lu-ýi:!A 1

.9 s-C I t

r... <8> 1 CaOD£140X ~ £..a~'

Figure 3 - LOVS test on affected circuit on 1/21/2008 performed a few weeks before the degraded condition of the connection was discovered. The test results were satisfactory. Test duration about 15 seconds.

File."Degradationof DC connection D201 13 of 34

7/17/2008 - 2:19:33 PM I EDISN mtI

,UihF%"I~ IFORNIA I 4ToI, * .flIPral J%t.1.1.P Y~%rM Figure 4: One bolt on the affected connection.

All the aforementioned are in accordance with the assumption based on existing knowledge (as shown graphically in Figure 5), that once a clean contact was established on March 2004, losing pressure did not immediately result in an increase of contact resistance.

This only happened after the 4-day period of swing-charger current flowing thru the impacted joint.

File: Degradationof DC connection D201 14 of 34

7/17/2008 - 2:19:33 PM I EDISON SOUHF " (Al An fOI.ON 1.%1h#%ATC)10.iL lIOCtNIA 4L.Mpan)

Figure 5 [4] - Contact theory says that contact points are actually cold welds.

If the high contact force is decreased, the contact resistance remains constant and does not increase until a much lower force is reached.

4. How would have a seismic event affect the inte2ritv of the affected connections A seismic event would have no adverse impact on the integrity of the connection [9].

First of all, from inspection of Vendor's drawings and Bill of Material, the breaker is attached by dedicated bolts and channels to the panel. The connections to the buses upstream and downstream are not designed for support purposes, but to provide electrical continuity to the rest of the circuit. From an electrical point of view, all the bolts belonging to the two affected connections are fully threaded into their nuts and some of them quite tight (from interview with the craftsman that tightened the bolts - Item 8 under "Factual Evidence" above). This means that there was little or no increase in risk that a seismic event would separate the buses breaking the physical continuity of the circuit. Second, the violent shaking of the connection would have help in breaching the very thin layer of oxidation between the busses further reducing (perhaps to zero) the fritting voltage of the connection. Once a DC current starts, it is well known that current flow will be maintained over small gaps due to the lack of zero-current crossing inherent to DC circuits. This is particularly true in an inductive circuit, such as the one being fed by 2B008.

The following statement results from seismic calculations performed specifically to address the situation on hand [9]:

File: Degradationof DC connection D201 15 of 34 NI

7/17/2008 - 2:19:33 PM J EDSOVHC$1ONA An ISO JOiT~ftK%A704). - (Unpn~

"The connections will not lose contact because there will be very little differential movement between the breaker tabs and the bus bars. Since the bolts have sufficient strength to withstand the seismic loads, lateral movements will be limited to about 1/16 inch which is the difference between the 3/8 inch diameter holes and the 5/16 inch bolt diameters. As shown in the figures (Figure 6 below), the breaker tabs are sandwiched top and bottom by the bus bars. Due to the out-of-plane flexibility of the bus bars and having at least two tight bolts per connection, the bus bars will move with the tabs vertically and thus, existing gaps would not be expected to become greater in a seismic event."

Figure 6: Physical support of breaker 2D201

==

Conclusions:==

1. The cause for the degraded bus connections between 2D201 and 2B008 was insufficient torque applied to the eight bolts (four per polarity) holding the buses and breaker terminals together, during 2D201 replacement in March 2004.

2. Reasonable assurance is provided by surveillance tests and data from the normal-charger test period that the connections were able to perform their design function for the entire period starting with the replacement of 2D201 until the last successful surveillance test a week before the failed one.

3. Based on the physics of aluminum-to-aluminum contact and fritting voltage, it is likely that the 120 DC-volts circuit would have exceeded the fritting voltage during a real event allowing the connections to carry current without abnormal voltage drop from 3/21/08 up to the time of the failed surveillance test.

4. As shown by calculations [9], a seismic event would have no deleterious effect on the physical integrity of the breaker or in the capability of the connections discussed herein to perform their design-duty.

File: Degradationof DC connection D201 16 of 34

7/17/2008 - 2:19:33 PM J A-, F/,*/

EDISON

$M II'%

SO ll I%,YAMK%

N;"

r'.AII}f(M'IfA I It0%I L- t mriin,

5. For a LOVS/SIAS or DGVSS event with normal charger 2B002 in service, there is high confidence that the 2B008 battery would have supplied the required power to the 2D2 loads, even with the degraded connections at the 2D201 breaker. This is based on the following facts:

a) The 120 VDC available at the battery would be expected to be above the fritting voltage for the connection so it would punch through any surface buildup of oxidation.

b) Once DC current begins to flow, it tends to be self sustaining as there is no zero crossing of the current & voltage, c) The battery would need to support the loads for only 15 seconds until the EDG starts, and then 2B002 charger recovers, so there would not be any significant heat buildup at the connection that would impact the connection or the breaker d) Two successful ESF tests and two successful LOVS tests were performed between 2004 and 2008. This corroborates the conclusions above.

6. For a LOVS/SIAS or DGVSS event with the swing charger 2B022 in service, there is reasonable confidence that the 2B008 battery would have supplied the required power to the 2D2 loads, even with the degraded connections at the 2D201 breaker. This is based on the following facts:

a) The 120 VDC available at the battery would be expected to be above the fritting voltage for the connection so it would punch through any surface buildup of oxide or contaminants, b) Once DC current begins to flow, it tends to be self sustaining as there is no zero crossing of the current & voltage, c) The battery would need to support the loads for 90 minutes, however during a recent (3/2008) normal charger test, the swing charger supplied 112 ADC thru the under-torqued connections for approximately four days.

7. For an SBO event, the degraded connection would have to support the 2D2 bus loads for a 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> period and there is some possibility that heating of the connections would occur. But, this would not result in tripping of the 2D201 breaker within the four hour period given it has a magnetic trip not directly affected by the temperature at its terminals. Calculations from data obtained during the swing charger test of 3/17/08 to 3/21/08 indicate the resistance before the current reversal was no more than 0-8 mf2. For an 8 mfl resistance, a current of 170 ADC (SBO average current) will produce a loss in the connection of about 250 watts - not enough to cause any deleterious heating over a four hour period.

The calculations in Attachment 3 and 4, independent from all the above, show the resistance of the joint prior the end of the 4-day normal charger test is statistically within the maximum allowable value, i.e.: the connection would have allowed the battery and swing charger to deliver their charger/load as required.

File: Degradationof DC connection D201 17 of 34

. S.I.F CMIFORNIA 7/17/2008 - 2:19:33 PM EDISON A, f I*C* 1ArMRS4T 0.'4. Compan9

References:

I. Attached Timeline. (Attachment 5)

2. Graphs obtained during the ESF and LOVS tests noted in Item 3 of the section titled "Factual Information", below.

3. Engineering Report: Bolting Techniques and Practices Aluminum / Copper, by Columbia MFG, LLC, Astoria, Oregon.

4. Creating Reliable Electrical Connections, by Norman Shackman, P.E., Maintenance World, Posted on Internet on 8-25-03.

5. Conductive Grease and Electrical Contact Lubricant "NO-OX-ID A-Special Electric Grade, The Sanchem Company.

6. SONGS - CalculationNo. E4C-0I 7.1 Rev. 3, sheet # 83.

7. MO #07042009000 (battery charger 2B002 test).

8. MO #03100406 (2D201 replacement).

9. SONGS Seismic study of degraded connections - July 2008.

10. Ductor test on re-torqued connections performed on 7/15/08 (NMO #800117881).

1I. Notification 200055175 (As found test of 2EYD2 computer readout for 2D bus voltage).

File: Degradationof DC connection D201 18 of 34

7/17/2008 - 2:19:33 PM I A- tamo Ci EDRISON SOU1141R CAI IFC)WdIA

~rim.%4ou

)

ATTACHMENTS File: Degradationof DCconnection D201 19 of 34

)

7/17/2008 - 2:19:33 PM A F .

IEDISON J VJVIIIPR%. CAI IFOHNIA FMS0 4!

N4110V41 CLmr*.n, ATTACHMENT 1 Potential Cause(s) for the Marginal Connection Found on 3/25/08 Cause Relevance External surfaces of bolts found not corroded Corrosion of the bolts' threads(1)

Over-tightening (taking the bolt material The looseness found on some of the bolts is beyond its yield point) more than it would have been found if the bolts were torqued over their yield point (1).

Vibration Equipment is not subject to severe and continuous vibration.

There is no visible indication of deformation Deformation of the bolted buses (1)

Continuous lateral movement of the bolt No such wearing of the buses/bolts/washers wearing the bus and/or washer was found on their outside surface (1),

Most probable cause.

Possible scenario: Maintenance personnel tightened the bolts with the common wrench, Under-tightening but forgot to finalize the torquing activity with the torque wrench. This conclusion is based on the interview with the craftsmen (see Item 8 under "Factual Information," above.-

Note (1): The degraded connections have not been disassembled and inspected.

File: Degradationof DC connection D201 20 of 34

7/17/2008 - 2:19:33 PM I EDISON

%OUHFR% CAl IICWNIA A. f AOr)%

iTFir.7 %4rrm C ompa ATTACHMENT 2 Weekly Battery (2B008) Test Data - 2004-2008 SpGr Cat A

Limits Pilot Cell Battery Met? A Pilot Cell B Date Volts (>= 1.202) Corr SpGr Corr SpGr MO Remarks 1.229 1.224 06/10/08 131.76 Yes 08060144 06/03/08 131.75 Yes 1.225 1.218 08052367 05/27/08 08051932 MO not imaged 05/20/08 131.75 Yes 1.232 1.224 08051182 05/13/08 08050425 MO not imaged 05/06/08 131.87 Yes 1.224 1.218 08042805 04/29/08 131.46 Yes 1.234 1.225 08042026 04/22/08 131.80 Yes 1.236 1.223 08041266 04/15/08 131.86 Yes 1.234 1.225 08040720 04/08/08 131.86 Yes 1.225 1.223 08040112 04/01/08 131.74 Yes 1.237 1.231 08031922 03125/08 131.88 Yes 1.223 1.225 08031473 2D201 Event 03/18/08 131.52 Yes 1.237 1.225 08030752 03/11/08 131.85 Yes 1.240 1.224 08030184 03/04/08 131.73 Yes 1.225 1.219 08022321 02/26/08 131.60 Yes 1.226 1.219 08021808 02/19/08 131.94 Yes 1.225 1.217 08020977 02/12/08 131.91 Yes 1.226 1.220 08020457 02/05/08 131.85 Yes 1.225 1.217 08012307 01/29/08 131.84 Yes 1.227 1.220 08011542 01/22/08 131.82 Yes 1.226 1.209 08011162 01/15/08 131.93 Yes 1.227 1.218 08010585 01/08/08 131.76 Yes 1.234 1.228 08010076 01/01/08 07121685 MO not imaged 12/25/07 131.40 Yes 1.227 1.221 07121328 12/18/07 131.22 Yes 1.226 1.221 07120824 12/11/07 131.29 Yes 1.229 1.224 07120307 12/04/07 131.37 Yes 1.226 1.223 07111436 11/27/07 131.50 Yes 1.230 1.223 07112430 11/20/07 131.35 Yes 1.227 1.221 07110810 11/13/07 131.62 Yes 1.227 1.221 07110397 11/06/07 131.45 Yes 1.225 1.220 07101717 10/30/07 131.61 Yes 1.227 1.221 07101477 10/23/07 131.69 Yes 1.229 1.223 07101022 10/16/07 131.37 Yes 1.229 1.224 07100642 10/09/07 131.31 Yes 1.226 1.225 07100110 10/02/07 131.23 Yes 1.229 1.217 07091415 File: Degradationof DC connection D201 21 of 34

7/17/2008 - 2:19:33 PM J

IEDISON l

CAI IFORNIA SO* 114FRIN 09/25/07 131.27 Yes 1.229 1.226 07091049 09/18/07 131.45 Yes 1.228 1.223 07090784 09/11/07 131.33 Yes 1.229 1.223 07090267 09/04/07 131.41 Yes 1.230 1.223 07081640 08/28/07 131.36 Yes 1.231 1.226 07081180 08/21/07 131.48 Yes 1.229 1.225 07080810 08/14/07 131.44 Yes 1.220 1.222 07080393 08/07/07 131.41 Yes 1.223 1.218 07080101 07/31/07 131.42 Yes 1.223 1.218 07071141 07/24/07 132.04 Yes 1.212 1.214 07070851 07/17/07 131.99 Yes 1.223 1.220 07070424 07/10/07 131.99 Yes 1.224 1.220 07070128 07/03/07 132.00 Yes 1.223 1.222 07061477 06/26/07 131.87 Yes 1.223 1.219 07061065 06/19/07 131.86 Yes 1.224 1.223 07060693 06/12/07 131.81 Yes 1.223 1.222 07060287 06/05/07 131.80 Yes 1.222 1.222 07051854 05/29/07 131.74 Yes 1.222 1.217 07051514 05/22/07 131.81 Yes 1.222 1.213 07051093 05/15/07 131.49 Yes 1.220 1.215 07050550 05/08/07 131.58 Yes 1.210 1.208 07050145 05/01/07 131.77 Yes 1.215 1.213 07041721 04/24/07 131.57 Yes 1.219 1.218 07041298 04/17/07 131.60 Yes 1.221 1.217 07040841 04/10/07 131.67 Yes 1.215 1.209 07040377 04/03/07 131.66 Yes 1.217 1.212 07033435 03/27/07 131.66 Yes 1.215 1.211 07032955 03/20/07 131.69 Yes 1.210 1.213 07032416 03/13/07 131.64 Yes 1.212 1.210 07030537 03/06/07 131.65 Yes 1.212 1.211 07022492 02/27/07 131.60 Yes 1.220 1.218 07021801 02/20/07 131.18 Yes 1.219 1.209 07021348 02/13/07 132.05 Yes 1.209 1.217 07020750 02/06/07 131.54 Yes 1.228 1.225 07011942 01/30/07 131.96 Yes 1.214 1,218 07011363 01/23/07 131.06 Yes 1.221 1.216 07011107 01/16/07 131.14 Yes 1.219 1.213 07010592 01/09/07 131.21 Yes 1.218 1.212 070100097 01/02/07 131.15 Yes 1.218 1.213 06121575 12/26/06 131.16 Yes 1.218 1.213 06121303 12/19/06 131.12 Yes 1.222 1.215 06150848 12/12/06 131.13 Yes 1.218 1,215 06120337 12/05/06 131.21 Yes 1.217 1.214 06111608 11/28/06 131.23 Yes 1.223 1.222 06111257 11/21/06 131.30 Yes 1.230 1.217 06110903 11/14/06 131.27 Yes 1.222 1T218 06110429 11/07/06 131.27 Yes 1.223 1.217 06110184 10/31/06 131.29 Yes 1.223 1.216 06101425 10/24106 131.28 Yes 1.223 1.216 06100933 10/17/06 131.31 Yes 1.223 1.217 06100741 File: Degradationof DC connection D201 22 of 34

7/17/2008 - 2:19:33 PM IEDISON MnFjjXj 5J SMI111 RN~LAI 1101041A

1. rpp!%4T0 11. - Compa')

10/10/06 131.28 Yes 1.218 1.214 06100111 10/03106 131.27 Yes 1.218 1.214 06091284 09/26/06 132.22 Yes 1.222 1.216 06090865 09/16/06 131.79 Yes 1.218 1.216 06090189 09/12/06 131.91 Yes 1.226 1.219 06090189 09/05/08 06081588 MO not imaged 08/29/06 131.60 Yes 1.222 1.217 06081261 08/22/06 131.59 Yes 1.225 1.217 06080908 08/15/06 131.58 Yes 1.220 1.217 06080586 08/08/06 131.59 Yes 1,229 1.218 06080107 08/01/06 131.68 Yes 1.224 1.219 06071235 07/25/06 131.76 Yes 1.223 1.219 06070854 07/18/06 131.66 Yes 1.224 1.219 06070606 07/11/06 131.61 Yes 1.219 1.218 06070160 07/04/06 131.64 Yes 1.226 1.220 06062026 06/27/06 131.63 Yes 1.224 1.221 06061477 06/20/06 131.59 Yes 1.223 1.219 06060926 06/13/06 131.49 Yes 1.223 1.219 06060596 06/06/06 131.57 Yes 1.224 1.219 06060174 05/30/06 131.53 Yes 1.222 1.221 06051947 05/23/06 131.29 Yes 1.223 1.222 06051339 05/16/06 131.47 Yes 1.226 1.221 06050784 05/09/06 131.45 Yes 1.228 1.225 06050300 05/02/06 131.48 Yes 1.218 1.217 06041634 04/25/06 131.46 Yes 1.227 1.223 06041324 04/18/06 131.34 Yes 1.228 1.222 06040718 04/11/06 131.36 Yes 1.226 1.222 06040424 04/04/06 131.41 Yes 1.227 1.223 06032246 03/28/06 131.37 Yes 1.227 1.224 06031753 03/21/06 131.35 Yes 1.220 1.218 06031102 03/14/06 131.24 Yes 1.229 1.223 06030667 03/07/06 131.44 Yes 1.233 1.225 06021785 02/28/06 132.56 Yes 1.220 1.223 06021408 02/21/06 131.44 Yes 1.238 1.230 06020927 02/14/06 131.65 Yes 1.220 1.217 06020532 02/07/06 131.66 Yes 1.221 1.218 06020011 01/31/06 131.67 Yes 1.218 1.217 06011826 01/24/06 131.74 Yes 1.219 1.216 06011303 01/17/06 131.58 Yes 1.220 1.217 06010560 01/10/06 131.71 Yes 1.221 1.218 06010091 01/03/06 131.54 Yes 1.218 1.215 05121705 12/27/05 131.56 Yes 1.220 1.217 05121316 12/20/05 131.63 Yes 1.219 1.217 05120898 12/13/05 131.63 Yes 1.220 1.218 05120406 12/06/05 131.59 Yes 1.222 1.219 05111528 11/29/05 131.64 Yes 1.220 1.219 05111259 11/22/05 131.72 Yes 1.221 1.218 05110868 11/15/05 131.62 Yes 1.219 1.217 05110596 11/08/05 131.68 Yes 1.221 1.218 05110066 11/01/05 131.71 Yes 1.222 1.218 05102804 File: Degradationof DC connection D201 23 of 34

-d

7/17/2008 - 2:19:33 PM IEDISON A. F11SO O.J SXI4111%

'TNOW44l ON CAI IFORNIA

- C mp~ft 10125/05 131.67 Yes 1.221 1.219 05102429 10/18/05 131.67 Yes 1.223 1.219 05100637 10/11/05 131.71 Yes 1.221 1.218 05100194 10/04/05 131.67 Yes 1.218 1.217 05091712 09/27/05 131.67 Yes 1.220 1.218 05091222 09/20/05 131.70 Yes 1.219 1.217 05090683 09/13/05 131.70 Yes 1.223 1.219 05090306 09/06/05 131.71 Yes 1.219 1.218 05082109 08/30/05 131.82 Yes 1,221 1.217 05081531 08/23/05 131.78 Yes 1.221 1.216 05081047 08/16/05 131.73 Yes 1.220 1.217 05080627 08/09/05 131.79 Yes 1.220 1.218 05080126 08/02/05 131.79 Yes 1.220 1.219 05071506 07/26/05 131.87 Yes 1.220 1.218 05070985 07/19/05 131.66 Yes 1.220 1.217 05070626 07/12/05 131.68 Yes 1.217 1.217 05070246 07105/05 131.63 Yes 1.217 1.216 05062441 06128/05 131.59 Yes 1.218 1.216 05061796 06/21105 131.50 Yes 1.217 1.217 05061234 06/14/05 131.61 Yes 1,219 1.218 05060690 06107/05 131.61 Yes 1.220 1.216 05052313 05/31/05 131.63 Yes 1.218 1.216 05052099 05/24/05 131.63 Yes 1.222 1.218 05051459 05/17/05 131.76 Yes 1.214 1.214 05050915 05/10/05 131.63 Yes 1.220 1.217 05050250 05/03/05 131.58 Yes 1.220 1.217 05041899 04/26/05 131.62 Yes 1.217 1.218 05041362 04/19/08 131.56 Yes 1.221 1.216 05040766 04/12/05 131.57 Yes 1.219 1.217 05040467 04/05105 131.54 Yes 1.217 1.215 05032192 03/29105 131.61 Yes 1.218 1.217 05031618 03/22/05 131.56 Yes 1.219 1.216 05031095 03/15/05 131.52 Yes 1.217 1.215 05030661 03/08/05 131.60 Yes 1.222 1.216 05030091 03101/05 131.56 Yes 1.219 1.216 05022033 02/22/05 131.59 Yes 1.220 1.217 05021455 02/15/05 131.52 Yes 1.219 1.216 05020661 02/08/05 131.54 Yes 1.220 1.216 05020116 02/01/05 131.55 Yes 1.220 1.216 05012154 01/25/05 131.63 Yes 1.218 1.219 05011488 01/18/05 131.67 Yes 1.218 1.216 05011156 01/11/05 131.64 Yes 1.219 1.217 05010189 01/04/05 131.42 Yes 1.219 1.216 04123087 12/28/04 131.76 Yes 1.221 1.218 04122690 12/21/04 131.76 Yes 1.220 1.215 04122402 12/14/04 131.63 Yes 1.220 1.215 04120542 12/07/04 131.57 Yes 1 221 1.216 04112002 11/30104 131.49 Yes 1.222 1.217 04111735 11/23/04 131.45 Yes 1.222 1.218 04111198 11/16/04 131.56 Yes 1.220 1.215 04110704 File: Degradationof DC connection D201 24 of 34

7/17/2008 - 2:19:33 PM IEDISON J OtJIHF'% C;Al AnM IXO 1%Tf.#?%.I IFORNIA UO.4L - (umpan 11/09/04 131.51 Yes 1.222 1.216 04110222 11/02/04 131.62 Yes 1.220 1.215 04102097 10/26/04 131.63 Yes 1.216 1.216 04101499 10/19/04 131.66 Yes 1.215 1.214 04100924 10/12/04 131.60 Yes 1.219 1.218 04100488 10/05/04 132.00 Yes 1.217 1.216 04091788 09/28/04 131.83 Yes 1.217 1.219 04091336 09/21/04 131.78 Yes 1.217 1.215 04090929 09/14/04 131.92 Yes 1.219 1.219 04090272 09/07/04 131.89 Yes 1.216 1.217 04082120 08/31/04 131.81 Yes 1.220 1.222 04081618 08/24/04 131.80 Yes 1.220 1.222 04081084 08/17/04 131.84 Yes 1.217 1.218 04080658 08/10/04 131.87 Yes 1.221 1.223 04080244 08/03/04 131.83 Yes 1.218 1.221 04072035 07/27104 131.85 Yes 1.220 1.219 04071416 07/20/04 131.89 Yes 1.218 1.218 04070933 07/13/04 131.86 Yes 1.219 1.219 04070422 07/06/04 131.82 Yes 1.219 1.220 04062326 06/29/04 131.72 Yes 1.216 1.219 04061791 06/22/04 131.77 Yes 1.215 1.220 04061223 06/15/04 131.76 Yes 1.218 1.219 04060698 06/08/04 131.76 Yes 1.217 1.220 40600810 06/01/04 131.75 Yes 1.218 1.221 04052007 05/25/04 131.70 Yes 1.218 1.219 04051494 05/18/04 131.79 Yes 1.218 1.220 04050904 05/11/04 131.74 Yes 1.218 1.218 04050260 05/04/04 131.90 Yes 1.213 1.221 04042120 04/27/04 131.31 Yes 1.224 1.216 04041552 04/20104 132.28 Yes 1.227 1.229 04041001 04/13/04 131.93 Yes 1.225 1.225 04040483 04/06/04 131.87 Yes 1.222 1.221 04032496 2D201 Breaker 03/07/04 Replacement 03/30/04 131.48 Yes 1.220 1.224 04031814 03/23/04 131.25 Yes 1.226 1.227 04031252 03/16/04 131.28 Yes 1.227 1.225 04030721 03/09104 131.57 Yes 1.230 1.228 04030178 03/02104 131.10 Yes 1.216 1.221 04021740 02/24/04 131.35 Yes 1.218 1.218 04021295 02/17/04 131.32 Yes 1.221 1.227 04020785 02/10/04 131.57 Yes 1.219 1.222 04020282 02/03/04 131.39 Yes 1.220 1.227 04011819 01/27/04 131.38 Yes 1.219 1.224 04011327 01/20/04 131.63 Yes 1.219 1.226 04010936 01/13/04 131.50 Yes 1.220 1.226 04010242 01/06/04 131.42 Yes 1.219 1.227 03121961 File: Degradationof DC connection D201 25 of 34

(.

7/17/2008 - 2:19:33 PM J EDRI(SON SSOUII4FB CAIIF}CWNIA An EISO)N I%rTNN4rr'n%4, C ,nywn)

ATTACHMENT 3 Calculation of joint resistance of degraded connections during the normal charger test of 3/17-21/08 1). Calculation of contact resistance while the swing charger is working:

BATTERY - SWING I I213008 -CHARGER NORMAL -22 2Bi CHARGER LI-2B2 2B002 >tIC S.................*

C* ...........

SH UN T ......................

DEGRADED CONNECTION BREAKER IL to 2D201 *:¢I 125 VDC bus DCload5 On 3/17/08, when the normal charger (2B002) was removed from service and the swing charger (2B022) was placed in service, the swing charger voltmeter read 131.52 (+/- 0.15)

VDC and 112 ADC. Based on plant computer data, the 2D2 bus voltage under these conditions was between 130.7 and 131.8 VDC. Based on this information, the 2D201 connection resistance at this time would be 0 to 8 mf2.

IL = 112 ADC (from MO #08011467)

Vsc = 131.52 +/- 0.15 VDC = 131.67-131.37 VDC V2D1) 130.7-131.8 VDC (RTime reading of 132 VDC and calibration data from Notification #200055175).

Rmax = (Vscmax - V2Dmin)/IL = (131.67 - 130.7)/112 = 8 mfl Rmin = (Vscmin - -V2Dmax)/IL File: Degradationof DC connection D201 26 of 34

7/17/2008 - 2:19:33 PM Ij A0 t.

SOUI11FRN CM IICRNIA EDISON M % )%TfX,4t 10%41.-* ý However, V2D can only be as high as the swing-charger's voltage, otherwise current would flow in the "wrong" direction, thus:

Rmin = (131.37 - 131.37)/112 =0 m(Q

2) Calculation of contact resistance when the normal charger resumed operation:

BATTERY - SWING 2B008 CHARGER NORMAL S2B022 CHARGER 2B002 S...........

t Ic DEGRADED CONNECTION I= IL + IC 125 VDC bus IL DC load On 3/21/08, when the normal charger (2B002) was returned to service, the 2D2 bus voltage was 130.7-131.8 VDC. The battery terminal voltage under these conditions was 131.38 (+/- 0.15) VDC. Assuming a typical float current of 0.1 amps into the battery, the corresponding connection resistance at this time would be equal to 0-5.7 Q. Statistically, this resistance is orders of magnitude higher than the connection resistance on 3/17/08.

IC= 0.1 ADC Vb= 131.38 +/- 0.15 VDC = 131.53-131.23 VDC V2D= 130.7-131.8 VDC Rmax = (V2Dr-ax Vbmin)/IC = (131.8 - 131.23)/0.1 = 5.7 fl Rmin = (V2Dmin - Vbmax)/IC File: Degradationof DC connection D201 27 of 34

JSOU)I)F N CAlIORNI*lA 7/17/2008 - 2:19:33 PM A~ftHIM^ I%TAft41IC)\AL - Compan)

However, V2D can only be as high as the -charger's voltage, otherwise current would flow in the "wrong" direction, thus:

Rmin = (131.53 - 131.53)/0.1 = O*l

3) Calculation of VD2 based on measured value As shown on the schematic below, the value of VD2 is measured on a computer, after it is affected by the error of the resistance divider, analog-digital converter, and truncated at the computer. Thus, if the value displayed in the computer for VD2 is 132 VDC, then the true value at the bus is as shown below.

VD2displayed Digital truncation In order to measure the relation between the true value of VD2 and the value displayed on the computer, a careful "as-found" test performed on 7/15/08 (Ref. 11) yielded the following.

VD2displayed = 132 VDC 4 true value of VD2 = 130.8-131.7 VDC File: Degradationof DC connection D201 28 of 34

7/17/2008 - 2:19:33 PM I SOI)IHFR% CAiFORNIA AnlEI.7SO%'J%r,'Rft~OV4L - C.ompan 7 ATTACHMENT 4 Calculation of voltage values during analyzed accidents and maximum acceptable resistance in degraded connection t3 EU Fdk &1w rwrt Fwnmt Io* Qata Wndo H* _-n -9 x a tjUtn eprJINqCASE, ,-- ý. .. . " . .

M" fir 8OUNDING CASE .4 .- . ,

F a 1 LI1

-N 5 i -. T u V ...

CONMXTO'3 F W4IYTETME PEFW Nu~fl21 TVWPEF TO LaT4j-srzE- SI alwooo li 1-30 .mm2_ &1I 1 1-20 1 20.99 0.90 IPANR~fAOId 0.1 1 .3 m6 2 1202 M04.11 I IJmm to." cm.

00 lb.

lb.

110.

106 U., U t0-71j 22 24,;Po4wlaJln 25 P497(Z3nl7 2?_.

211 332 2 , .0

... .14 "C.9C - -~llO 54 DO v? I Ivp g C-O ' 6000 GA I 12.0 4q 41 0 0090"n oo91~ .il.l 7.44DE-44 a DNCAE 2020 1-RrK. Coa...lr0GF~t!195tnd. 0.0054 Q2 46 (6000206 W-4.1C7030 W____ __

471 ____________________

t_-______-_ 7 49 . - I 0~~Ad'-o1. I-Ol*. ~.l so _ _ ___ I____________

S4 --iýM -TabIC 0.3. 36~: ~ .. 1~ ~-.0, ~h8 8~T-H TseS8-tJMZVSIQI-1 FTaJIC .4.18-VLI. z~~

ReadV

~RatI~

~~ . Ic~~nccrri.. AZZ1F169-LOSIJM.105II~iCcwocftElce-LOYV- S;CMP fPM 5:2 File: Degradationof DC connection D201 29of 34

7/17/2008 - 2:19:33 PM SSOtUIHFR*

rEDISON t

AAtl~.9lS0.'V.%rF9.l%AflO*4L.

CMIIONNIA CUM PSM.

EDj zile Edt kAev. jnoet Prin.t lad Daa AIM. Hep VTIK 1",-t q,  ;,-,-h-t - - 9 X 9.. ; - '0, _..

a2 tge "% 43j & .. o. . Lr -- ý!ý Arial-50

• Bit U6-0 ffum ~ twi ~

~ ~ * ~ '- II a e)375% *U D5 -

. A H I J K L M _ N 0 P A S . T I .U V" 17 AV 64 6 0458 6.322 15.8115.61 15.81 ISS 157 5.10 5.10 50 50 .5 2-1125 108205 10401 1M 408 12AVG r .572 0oD46 2.64 244 244 264 2.64 2-12 2.12 2.12 22 2.12 21.42i 105.54 M102 107.73 is0m 23 AVO r 572 90,575 076 0.76 0.76 0.76 876 035 (135 0.35 025 6.35 2L-345 M722 T26 10.5 20 574-p 12 AV r.r__U-_ 200 2 200 208 I107.65 200 226 226 2.26 226 22 10540 23 1 4l__ 124 ba 26.80 0.00 0. 00 0.004 104 620 0.00 0.0 0.00 P4'12L3731 65M 10w 1.36 24 15

____ 1.57 0.297 31.10 6.30 6.30 6.30 6.30 5--.32 165 1.89 1059 165 P49qa 5 76.29 106.72 103.41 25 340 12AVG3 1.572 0.6705 26.530 6.30 1 6.30- 6.30 6.30 161.64 422 1 4.22 422 1 4.22 P14521373 69.58 104.35 1014 I 26 i Be . 0t0371 0.0052 128646 1286 2846 504" 50.66 0.67 647 0.67 0.47 0.47 2Y1002 108.93 112.91 W09.66 27 1152 .02C 3~7 0.0297 41071 60 0.5 1.61 1.01 124 O023 0.03 O023 0.03 2MS4706 10830 173.55 110.30 28 44 2AVG 0-203 0.00m 1.39 3 3 0.74 0.74 0.07 001 0.01 a01 0.01 27007 10959 113.6 (10.31 25 BIG 22AVG 1.572 1.2029 131 13 0.6 0.66 0.66 1.08 t.58 0.75 0.75 0.75 2L-71 108.01 11158 109.53 30 24 Q2AVG 1T972 0-6305 0.08 0 0.08

.08 0.0502 0.08 05 0.08 040S 0.08 12L-345 105D.54 113.51 15026 31 32 1peel vo~ag associated withmotor stariNg.

6 37 Mwit "1

VPC tomis r

[,1105.414

'. "Feeders

.113 W.,e0.0 CortoWro a3* e4 OUT Conne*tor

etio~r.:*I.

3-..mpet Cables C-rtoeoor

'-. '. ,e200E 5600-05 1.,e

.00 G.(J 12,11" 38 ToWalCc-trorz 62.00 W9 40; 41 Total Connectons Resistance 7.440E-04 D

.4 20201 Areakei Connecto'st... Resistance

, .T*

0.0054 3 8n3 0 4 3-Jumperoral,,

Cables Resisrance 3.830E-04 p 47 7 46 g ~~(1008 =201t 11CIt3101 ____ ___

49. .- .

4T tA,Jdad t(r CC il. Ft. '.t.

-1Minimum Inet.e Yolltare 05.414i

. 2I

~HTtd.3.0-8008 T g1.8S~a~8 A'-0osz~io ATlS8.3-61TVN-1Tablee8AS42VGd /1T4_8-3R1 Iii _- _ý.4 Ready Aiuaotjl 4 4 ~ 1 adat-o'DC-r...j (D>A Fies-Lctus Otsfl~tcrosoftEntced-Seu W/MM0NetAMpIt 5: 30PM File: Degradationof DCconnection D201 30 of 34

%(lMll I RCA1FORtNJA 7/17/2008 - 2:19:33 PM .J Mn 1.10 EDISON I% tf4110".4L-I ATTACHMENT 5 Breaker 2D201 and Battery 2B008 Sequence of Events 05/02/2002 MO 01061449 completed 5/27/02 connect monitoring equipment (2A04, Train B ESF test) 02/11/2004 MO 03031098 completed 9/30/04 Support various Operations procedures.

Completed the DG tests "Train B" 2A06. (ESF) 02/20/2004 MO 03031098 Support various Operations procedures. Completed the DG tests "Train B" 2A06. (LOVS) 02/27/2004 MO 04021613 retested new breaker for long time trip on 2/20/04 per ECP 031200986-9. On 2/10/04 obtained new breaker and trip tested per SO123 4.7.

03/17/2004 MO 03100406 completed changing the Short Time Delay Settings and installing new breaker 2D201 per Calc E4C-109. (ECP 001000280-05, ECN A20785) 01/06/2006 MO 05081829 completed 3/28/06, the DG tests "Train B" 2A06. (ESF) 01/21/2006 MO 06011546 support Operation's procedure for 2G003 DG. (LOVS) 02/25/2006 MO 05011306 completed 5-year Battery Performance Test (21)201 breaker operated per WAR 2-R41)2BAT) 07/15/2006 MO 06041451 completed quarterly battery surveillance on B008 battery.

10/03/2006 MO 06070800 completed quarterly battery surveillance on B008 battery.

12/28/2006 MO 06100206 completed quarterly battery surveillance on B008 battery.

01/30/2007 MO 06070902 Annual Battery Performance test after reaching 85% service life in October 2006 and performance of Thermography (2D201 breaker operated per WAR 2-0601303)

File: Degradationof DC connection D201 31 of 34 n-

7/17/2008 - 2:19:33 PM

..n f MiEDISO r.rif%4 10%4 j %~ A -. C wap.

03/27/2007 MO 07010094 completed quarterly battery surveillance on B008 battery.

06/13/2007 MO 07033365 completed quarterly battery surveillance on B008 battery.

09/17/2007 MO 07060901 completed quarterly battery surveillance on B008 battery.

11/29/2007 MO 06122943 completed the ESF tests "Train B" 2A06.

12/08/2007 MO 07091048 completed quarterly battery surveillance on B008 battery.

12/27/2007 MO 07020582 Perform pre-weekly quarterly battery performance test.

12 ?.8 '2:,

0 '0 ,i( ) 7 l:(T, l [) It.

D lIi-SlY

.'_ ") LW1'. .. 2,!,. C dLa 2c... 2H!102.

in (-3A5 7 6lv ". .l) ( j C  ! ' -d l, 01/09/2008 MO 08010611 support for ESF tests (LOVS) 01/302008 MO 07020585 Weekly lE battery inspection completed - sat.

02/19/2008 MO 08020977 Weekly I E battery inspection completed - sat.

MO 08011261 Perform pre-weekly quarterly battery performance test.

02/22/2008 MO 07120476 Connect 2/3B00X to 2D206 (2B008). This temp ECP to connect battery 2/3BOOX to bus 2D2 to support 2B008 battery bank performance testing. (Temp ECP 070900092-3)

(2D201 operated per MO with In-test)

\1() U ),LLI i2.l t'olIlIl2 t.iCI v I(.I t; .oC lwct . i n 2( h,.i! gýr (o" k w ';,k 02/25/2008 MO 08011292 3 1-day post test weekly/quarterly inspection 02/27/2008 MO 07020581 Annual Battery Performance test after reaching 85% service life in October 2006 and performance of Thermography.

(2D201 operated per WAR 2-0702901 via an In-test)

MO 07020584 Perform Physical inspection of battery 03/03/2008 MO 08010255 completed quarterly battery surveillance on B008 battery.

03/11/2008 MO 08030184 Weekly I E battery inspection completed - sat File: Degradationof DC connection D201 32 of 34

7/17/2008 -2:19:33 PM EDISON An F01SON IN T M%4710 N4 1.

• Umps.

03/17/2008 MO 08030095 31-day post test weekly/quarterly inspection 03/21/2008 MO 06100929 B002 Battery Charger performance test and thermography inspection.

(21)201I not operated per WAR 2-0702504)

MO 08011467 2B002, Bus D2 Normal Charger, placed in service and 2B022, Swing charger removed from service by soft transfer operation. (Operations)

MO 07042009 Charger Performance Test, perform Operability readings on 2B008 while on the Swing Charger 2B022.

03/25/2008 MO 08031473 completed weekly battery surveillance which discovered low voltage on battery 2D2.

Operations declared 2D2 battery inoperable.

Electricians discovered eight loose bolts on the battery side of breaker 2D201.

ACE 080301117 generated to determine cause of loose bolts on bus bars of 2D201.

MO 08031721 Tighten and torque the loose bus bar bolts.

MO 08031738 Perform weekly/quarterly inspection for declaring batteries operable 03/28/2008 MO 07101740 perform annual energization testing on swing charger 2B022.

(2D201 not operated per WAR 2-0702931 )

04/01/2008 MO 08031922 Weekly 1E battery inspection completed - sat.

04/02/2008 MO 07101741 Swing Battery Charger 2B022 Perform 12-hour Full Load Test

- sat 04/10/2008 OE 26600 Operating Experience generated on loose Bus Bars bolts causing low battery voltage.

05/19/2008 MO 08030494 completed quarterly battery surveillance on B008 battery.

NOTE: all dates are MO completed dates unless noted rev.08 File: Degradationof DC connection D201 33 of 34

7/17/2008 - 2:19:33 PM A M EDISON IHA*J* W% z (A IFORMA SM/'Ir*,."o Aft Iff~hO i%lr4fli4.K

%A1U U- .pl ATTACHMENT 6 2B008 battery DEGRADED -)

} loads Other 2D2P 1 CONNECTIONS

'N

'N

'N I) ')

21B002 Charger I) I)

-~ I-I A IA 2D2 bus Other 2D2 I I loads 2A06 switchgear Voltage monitor File. Degradationof DC connection D201 34 of 34