ML093100257: Difference between revisions

Joseph Kowalewski, Vice President, Operations

Entergy Operations, Inc.

Waterford Steam Electric Station, Unit 3

17265 River Road

Killona, LA 70057-3093

SUBJECT:

WATERFORD STEAM ELECTRIC STATION, UNIT 3 NRC INSPECTION

REPORT 05000382/2009008 PRELIMINARY WHITE FINDING

Dear Mr. Kowalewski:

On September 24, 2009, the U.S. Nuclear Regulatory Commission (NRC) completed an

inspection at your Waterford Steam Electric Station, Unit 3. The enclosed inspection report

documents the inspection finding, which was discussed on September 24, with you and other

members of your staff. The report documents baseline inspection activities related to the

Train B 125 Vdc battery surveillance failure on September 2, 2008. The inspection examined

activities conducted under your license as they related to safety and compliance with the

Commissions rules and regulations and with the conditions of your license. The inspectors

reviewed selected procedures and records, observed activities, and interviewed personnel.

The enclosed inspection report discusses a finding that appears to have low to moderate safety

significance (White). As described in Section 1R15 of the report, the Train B 125 Vdc battery

was rendered inoperable because electricians failed to properly assemble and test a battery

intercell connection following corrective maintenance in May, 2008. This finding was assessed

based on the best available information, using the applicable Significance Determination

Process (SDP). The preliminary significance was based on the battery being incapable of

performing its safety function for between 50 and 100 days, depending on the failure mode

assumptions. The primary assumptions associated with the preliminary SDP are documented in

Attachment 2 to this report. The finding is also an apparent violation of NRC requirements and

is being considered for escalated enforcement action in accordance with the NRC Enforcement

Policy, which can be found on the NRCs Web site at http://www.nrc.gov/reading-rm/doc-

collections/enforcement.

Before we make a final decision on this matter, we are providing you with an opportunity to

(1) attend a Regulatory Conference where you can present to the NRC your perspective on the

facts and assumptions the NRC used to arrive at the finding and assess its significance, or

(2) submit your position on the finding to the NRC in writing. If you request a Regulatory

Conference, it should be held within 30 days of the receipt of this letter and we encourage you

to submit supporting documentation at least one week prior to the conference in an effort to

make the conference more efficient and effective. If a Regulatory Conference is held, it will be

open for public observation. If you decide to submit only a written response, such submittal

should be sent to the NRC within 30 days of your receipt of this letter. If you decline to request

UNITED STATES

NUCLEAR REGULATORY COMMISSION

R E GI ON I V

612 EAST LAMAR BLVD, SUITE 400

ARLINGTON, TEXAS 76011-4125

Entergy Operations, Inc.

- 2 -

a Regulatory Conference or submit a written response, you relinquish your right to appeal the

final SDP determination, in that by not doing either, you fail to meet the appeal requirements

stated in the Prerequisite and Limitation sections of Attachment 2 of IMC 0609.

Please contact Jeff Clark by phone at (817) 860-8147 and in writing within 10 days from the

issue date of this letter to notify the NRC of your intentions. If we have not heard from you

within 10 days, we will continue with our significance determination and enforcement decision.

The final resolution of this matter will be conveyed in separate correspondence.

Because the NRC has not made a final determination in this matter, no Notice of Violation is

being issued for these inspection findings at this time. In addition, please be advised that the

number and characterization of the apparent violation(s) described in the enclosed inspection

report may change as a result of further NRC review.

In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and its

enclosure will be made available electronically for public inspection in the NRC Public

Document Room or from the NRCs document system (ADAMS), accessible from the NRC Web

site at http://www.nrc.gov/reading-rm/adams.html

Sincerely,

/RA/

Dwight D. Chamberlain, Director

Division of Reactor Projects

Docket: 50-382

License: NPF-38

Enclosures:

NRC Inspection Report 05000382/2009008

w/Attachments:

1. Supplemental Information

2. Significance Determination

Entergy Operations, Inc.

- 3 -

cc w/Enclosure:

Senior Vice President

Entergy Nuclear Operations

P. O. Box 31995

Jackson, MS 39286-1995

Senior Vice President and

Chief Operating Officer

Entergy Operations, Inc.

P. O. Box 31995

Jackson, MS 39286-1995

Vice President, Operations Support

Entergy Services, Inc.

P. O. Box 31995

Jackson, MS 39286-1995

Senior Manager, Nuclear Safety

and Licensing

Entergy Services, Inc.

P. O. Box 31995

Jackson, MS 39286-1995

Site Vice President

Waterford Steam Electric Station, Unit 3

Entergy Operations, Inc.

17265 River Road

Killona, LA 70057-0751

Director

Nuclear Safety Assurance

Entergy Operations, Inc.

17265 River Road

Killona, LA 70057-0751

General Manager, Plant Operations

Waterford 3 SES

Entergy Operations, Inc.

17265 River Road

Killona, LA 70057-0751

Manager, Licensing

Entergy Operations, Inc.

17265 River Road

Killona, LA 70057-3093

Chairman

Louisiana Public Service Commission

P. O. Box 91154

Baton Rouge, LA 70821-9154

Parish President Council

St. Charles Parish

P. O. Box 302

Hahnville, LA 70057

Director, Nuclear Safety & Licensing

Entergy, Operations, Inc.

440 Hamilton Avenue

White Plains, NY 10601

Louisiana Department of Environmental

Quality, Radiological Emergency Planning

and Response Division

P. O. Box 4312

Baton Rouge, LA 70821-4312

Chief, Technological Hazards

Branch

FEMA Region VI

800 North Loop 288

Federal Regional Center

Denton, TX 76209

Entergy Operations, Inc.

- 4 -

Emergency Diesel Generator

Electronic distribution by RIV:

Regional Administrator (Elmo.Collins@nrc.gov)

Deputy Regional Administrator (Chuck.Casto@nrc.gov)

DRP Director (Dwight.Chamberlain@nrc.gov)

DRP Deputy Director (Anton.Vegel@nrc.gov)

DRS Director (Roy.Caniano@nrc.gov)

DRS Deputy Director (Troy.Pruett@nrc.gov)

Senior Resident Inspector (Mark.Haire@nrc.gov)

Resident Inspector (Dean.Overland@nrc.gov)

Branch Chief, DRP/E (Jeff.Clark@nrc.gov)

Senior Project Engineer, DRP/E (Ray.Azua@nrc.gov)

WAT Site Secretary (Linda.Dufrene@nrc.gov)

Public Affairs Officer (Victor.Dricks@nrc.gov)

Branch Chief, DRS/TSB (Michael.Hay@nrc.gov)

RITS Coordinator (Marisa.Herrera@nrc.gov)

Regional Counsel (Karla.Fuller@nrc.gov)

Congressional Affairs Officer (Jenny.Weil@nrc.gov)

ACES (Rick.Deese@nrc.gov)

OE (Cynthia.Carpenter@nrc.gov)

RIDSOeMailCenter

OEMail Resource

ROPreports

DRS STA (Dale.Powers@nrc.gov)

OEDO RIV Coordinator (Leigh.Trocine@nrc.gov)

File located: R\\_REACTORS\\_WAT\\2009\\WAT 2009-008.doc

ADAMS ML093100257

SUNSI Rev Compl.

7Yes No

ADAMS

7Yes No

Reviewer Initials

RA

Publicly Avail

7Yes No

Sensitive

Yes 7 No

Sens. Type Initials

RA

Acting SRI:DRP/E

RI:DRP/E

SPE:DRP/E

C:DRP/E

SRA:DRS

M. Haire

D. Overland

R. Azua

J. Clark

M. Runyan

/RA - E//

/RA - E/

/RA/

/RA RAzua for/

/RA Caniano/

11/05/09

ES/ACES

C:OE

D:NRR/ADES

D:DRS

D:DRP

RDeese

GBowman

MCunningham

RCaniano

DChamberlain

/RA -E/

/RA/

11/05/09

11/02/09

11/05/2009

11/06/2009

OFFICIAL RECORD COPY

T=Telephone E=E-mail F=Fax

- 1 -

Enclosure

U.S. NUCLEAR REGULATORY COMMISSION

REGION IV

Docket:

50-352

License:

NPF-38

Report:

05000285/2007011

Licensee:

Entergy Operations, Inc

Facility:

Waterford Steam Electric Station, Unit 3

Location:

17265 River Road

Killona, LA 70057-3093

Dates:

December 15, 2008 through September 24, 2009

Inspector:

D. Overland, Resident Inspector

Reactor Analyst:

M. Runyan, Senior Reactor Analyst

Branch Chief

Jeff Clark, Chief, Project Branch E

Division of Reactor Projects

Approved By:

Dwight Chamberlain, Director

Division of Rector Projects

- 2 -

Enclosure

SUMMARY OF FINDINGS

IR 05000382/2009008; 12/15/08 - 09/24/09; Waterford Steam Electric Station, Unit 3;

Operability Evaluation.

The report covered a 40 week period of inspection by a resident inspector. One preliminary

White violation was identified. The significance of most findings is indicated by their color

(Green, White, Yellow, or Red) using Inspection Manual Chapter 0609, Significance

Determination Process. Findings for which the significance determination process does not

apply may be Green or be assigned a severity level after NRC management review. The NRC's

program for overseeing the safe operation of commercial nuclear power reactors is described in

NUREG-1649, Reactor Oversight Process, Revision 3, dated July 2000.

A.

NRC-Identified Findings and Self-Revealing Findings

Cornerstone: Mitigating Systems

TBD. Following a September 2, 2008 train B 125 Vdc battery failure, the licensee

identified an apparent violation of Technical Specification 6.8.1.a for the failure to follow

plant procedures during corrective maintenance on the safety-related battery. Following

the replacement of the entire battery bank during a 2008 refueling outage, craftsmen

identified a faulty battery cell. When replacing the faulty cell, plant workers did not follow

all of the specified procedural steps in the work package. The additional work resulted in

a loose battery connection that rendered the entire battery bank inoperable. The

licensee also failed to address an indicator of the loose connection during the battery

discharge test. The condition then went undetected for several months. The licensee

entered this finding in their corrective action program as Condition Report

CR-WF3-2008-4179.

This finding was greater than minor because it was similar to non-minor example 4.a in

NRC Inspection Manual Chapter 0612, Appendix E, Examples of Minor Issues, in that

the failure to follow site procedures adversely affected safety related equipment. Using

the Inspection Manual Chapter 0609, Significance Determination Process, Phase 1

screening worksheet, the finding required a Phase 2 significance determination

because it resulted in the loss of a single train of safety related equipment for greater

than the technical specification allowed outage time. Using a T/2 exposure time of

50 days, the inspectors used the Risk-Informed Inspection Notebook for Waterford

Nuclear Power Plant Unit 3, Revision 2.01 and its associated Phase 2 Pre-Solved

Table, and determined that a Phase 3 significance determination was necessary. A

Region IV senior reactor analyst performed a preliminary Phase 3 significance

determination and found that the finding was White. This preliminary Phase 3

significance determination is included as Attachment 2 to this report. This finding had a

cross cutting aspect in area of Human Performance (work practices component)

because maintenance personnel failed to use appropriate human error prevention

techniques, such as peer checking (quality control hold points) and tracking battery

components that were loosened (H.4.a). (Section 1R15).

- 3 -

Enclosure

REPORT DETAILS

1.

REACTOR SAFETY

Cornerstones: Initiating Events, Mitigating Systems, and Barrier Integrity

1R15 Operability Evaluations (71111.15)

a.

Inspection Scope

The inspectors reviewed the operability evaluation for the safety-related Train B 125 Vdc

station battery. The inspectors selected this potential operability issue based on the risk-

significance of the associated component. The inspectors compared the operability and

design criteria in the appropriate sections of the Technical Specifications and Updated

Safety Analysis Report to the licensees evaluations, to determine whether the

components or systems were operable and to ensure the licensee is operating and

maintaining the battery in accordance with specified requirements. The inspectors

developed a full chronology (time-line) that included significant event elements of the

September 2, 2008 Train B battery failure. This included a review of work orders and

actions associated with the May 2008 battery replacement. The inspectors determined

that sufficient information was communicated to operators and station management to

make informed decisions regarding the operability of the battery. The inspectors

reviewed the licensees DC load and battery design calculations to determine if proper

consideration was given to the effect of the loose battery connection and how it affected

the battery operability. Specific documents reviewed during this inspection are listed in

the attachment.

This activity constitutes completion of one (1) operability evaluations inspection sample

as defined in Inspection Procedure 71111.15-05

b.

Findings

Introduction. Following a September 2, 2008 Train B 125 Vdc battery failure, the

licensee identified a preliminary white violation of Technical Specification 6.8.1.a for the

failure to follow plant procedures during corrective maintenance on the safety-related

battery. Following the replacement of the entire battery bank during a 2008 refueling

outage, the licensee identified a faulty battery cell. When replacing the faulty cell, plant

workers did not follow all of the specified procedural steps in the work package. The

additional work resulted in a loose battery connection that rendered the entire battery

bank inoperable. The licensee also failed to address an indicator of the loose

connection during the battery discharge test. The condition then went undetected for

several months.

Description. In May 2008, during refuel outage 15, the Train B 125, Vdc battery was

replaced under Work Order 152819. The battery bank was composed of 60 individual

cells that were connected in series via bolted bus bars. Each individual cell had four

posts, two positive and two negative. The two negative posts of one cell were

connected to the two positive posts of the next cell via an intercell connector. Each

- 4 -

Enclosure

intercell connector consists of four bus bars and four bolts (one bolt for each post

connection). Electricians were required to torque the bolts on each battery post to

160 inch-pounds.

On May 24, 2008, as part of the postmaintenance testing for the battery bank

replacement, intercell connection resistance checks were performed on all of the battery

connections in accordance with Procedure ME-004-213, Battery Intercell Connections,

Revision 12. The intercell resistance checks involved resistance measurements across

the bolted connections. Technical Specification Surveillance Requirement 4.8.2.1.c.3

delineated a maximum acceptable intercell resistance of 150 micro-Ohms (a very small

resistance value). The inspectors noted that because battery discharge currents can be

very high (more than 700 Amperes), even relatively low values of intercell resistance can

have adverse consequences. The large current across a high resistance connection

dissipates a relatively large amount of energy at the connection point.

During additional postmaintenance testing on May 24, electricians determined that cell

56 would not charge. Electrical maintenance and engineering personnel decided to

replace cell 56 with a spare battery cell. Work Order 152819 did not contain specific

work instructions to replace cell 56 but the licensee believed that the replacement of cell

56 could be accomplished under the general guidance in the existing work package.

While station procedures recommended that the package be returned to the planning

department for the inclusion of specific maintenance steps and postmaintenance testing,

this was not required for minor scope changes. Procedure EN-WM-105, Planning,

Revision 3 stated, in part:

When the scope of work changes from that originally planned, determine if new

instruction or postmaintenance testing are necessary and if the work document

classification is still adequate. Scope changes should [emphasis added] be

subject to the same level of reviews as the original planning of the task.

Since the original work package was utilized to replace cell 56, the scope change was

not subject to the same level of reviews as the original planning of the task.

After cell 56 was replaced, the licensee tightened the connections and performed

intercell resistance checks on the battery posts that they believed were disturbed by the

maintenance. However, one additional battery post (between cells 57 and 58) was

loosened but not retightened.

The licensee identified that critical steps of Work Order 152819 were not completed. In

summary, the plant personnel did not: (1)torque all of the affected intercell connections

to 160 in-pounds; (2) obtain the required quality control inspector verification that all

affected connections were torqued appropriately; (3) ensure that all of the necessary

intercell resistance checks were performed; and (4) obtain a quality control verification

that the intercell resistance checks met technical specification limits.

On May 27, the licensee conducted Procedure ME-003-230, Battery Service Test,

Revision 301. During the test, the battery was discharged at a rate of over 700

Amperes. Since the battery passed the test, the licensee concluded that the defective

- 5 -

Enclosure

connection was made up reasonably well at the time. It was possible to pass this

particular test with a battery intercell resistance that exceeded the technical specification

limit of 150 micro-Ohms. The battery appeared capable of performing its safety function

during this test, however, it may not have been able to perform this same function during

a seismic event.

The licensee also noted that plant personnel had failed to follow the corrective action

program in response to an unexpected test result. Specifically, plant workers noted an

indicator of a loose connection during the ME-003-230 service test. During the test,

voltage across cell 57 dipped to an unusually low level (about 1.76 Vdc, while all the

other cells maintained voltage above 1.84 Vdc). The test apparatus alarmed on this

condition. Plant personnel failed to follow Procedure EN-LI-102, Corrective Action

Program, Revision 12. Attachment 9.2 required that a condition report be initiated for

events or conditions that could negatively impact reliability or availability. It also required

a condition report for conditions affecting a safety related system or component that

rendered the quality of an item indeterminate.

During the next several months, the licensee performed routine checks of the battery in

accordance with technical specifications. Those surveillances were limited to pilot cell

checks, total battery voltage checks, and visual inspections. None of these checks were

intended to identify a high resistance battery connection. The pilot cell check verified

that the battery cell voltage (for the selected pilot cell) was greater than 2.13 Vdc. The

total battery voltage check verified that the overall battery voltage was greater

than 125 Vdc.

On September 2, 2008, both pilot cells for the train B 125 Vdc battery were found at less

than 2.07 Vdc. Subsequent troubleshooting identified the loose connection between

cells 57 and 58. While the connection appeared tight during a visual inspection, the

licensee found the intercell resistance at more than 5 Ohms (more than 33,000 times the

limit). Two bolts on the connection were loose. The bolts should have been torqued to

160 inch-pounds but one was found 1 full turn loose while the second was about three

full turns loose.

The licensee postulated that the battery connections were in sufficient contact to pass

the discharge test on May 27. However, because of the loose connection, at some point

between May 27 and September 2, some slight movement occurred which increased the

intercell resistance. At the time of discovery, September 2, 2008, the battery was

inoperable.

Analysis. The failure to follow work order instructions was a performance deficiency.

This finding was greater than minor because it was similar to non-minor example 4.a in

NRC Inspection Manual Chapter 0612, Appendix E, Examples of Minor Issues, in that

the failure to follow site procedures adversely affected safety related equipment. Using

the Inspection Manual Chapter 0609, Significance Determination Process, Phase 1

screening worksheet, the finding required a Phase 2 significance determination

because it resulted in the loss of a single train of safety related equipment for greater

than the technical specification allowed outage time. Using a T/2 exposure time of

50 days, the inspectors used the Risk-Informed Inspection Notebook for Waterford

- 6 -

Enclosure

Nuclear Power Plant Unit 3, Revision 2.01 and its associated Phase 2 Pre-Solved

Table, and determined that a Phase 3 significance determination was necessary. A

Region IV senior reactor analyst performed a preliminary Phase 3 significance

determination and found that the finding was potentially White. This preliminary Phase

3 significance determination is included as Attachment 2 to this report. This finding had

a cross cutting aspect in area of Human Performance (work practices component)

because maintenance personnel failed to use appropriate human error prevention

techniques, such as peer checking (quality control hold points) and tracking battery

components that were loosened (H.4.a).

Enforcement. Technical Specification 6.8.1.a states that written procedures shall be

established, implemented, and maintained covering a. The applicable procedures

recommended in Appendix A of Regulatory Guide 1.33, Revision 2, February 1978.

Regulatory Guide 1.33, Appendix A, Typical Procedures for Pressurized Water

Reactors and Boiling Water Reactors, Section 9, Procedures for Performing

Maintenance, recommends procedures for maintenance that can affect the performance

of safety-related equipment. Work Order 152819 was a procedure that could affect the

performance of the safety-related Train B 125 Vdc battery. The work order stated, in

part:

The following work instructions can be worked out-of-sequence OR omitted at

the discretion of the cognizant supervisor, as long as the work scope is fully met

[emphasis added]

4.12

Torque in accordance with Vendor Technical Manual RS-1476 intercell

connections to 160 in-pounds (+10/-0).

Inspector Note: Step 4.12 included a quality control hold point which required

that an independent quality control inspector verify that the appropriate torque

was applied to each connection.

4.13

Perform ME-004-213, Station Battery 3A OR 3B OR 3AB Intercell

Resistance (18-Month) Surveillance, Revision 301, Sections 9.3, 9.4 and

9.5 in conjunction with, Vendor Technical Manual RS-1476 for interior and

interaisle connections [intercell resistance checks].

Inspector Note: Step 4.13 also included a quality control hold point which

required that an independent quality control inspector verify that the intercell

resistance values for each connection were less than the technical specification

limits.

Contrary to the above, on May 24, 2008, the licensee performed Work Order 152819

steps out of sequence, when battery cell 56 was replaced with a new cell, but failed to

ensure that the work scope was fully met. Specifically, the electricians did not:

(1) torque all of the affected intercell connections to 160 in-pounds (+10/-0); (2) obtain

the required quality control inspector verification that all affected connections were

torqued appropriately; (3) ensure that all of the necessary intercell resistance checks

were performed; and (4) obtain a quality control verification that the intercell resistance

- 7 -

Enclosure

checks met technical specification limits. The licensee entered this finding in their

corrective action program as Condition Report CR-WF3-2008-4179. This is a

preliminary White apparent violation pending completion of a final significance

determination. White 05000382/2009008-01: Inoperable 125 Vdc battery because

electricians failed to follow work instructions (EA-09-018).

4OA6 Meetings

Exit Meeting Summary

On September 24, the inspector presented the preliminary results of the inspection to

Mr. J. Kowalewski, Vice President, Operation, and other members of the licensee staff

who acknowledged the findings. The inspector verified that no proprietary information

was retained.

ATTACHMENTS:

1. SUPPLEMENTAL INFORMATION

2. PHASE 3 SIGNIFICANCE DETERMINATION

A-1

Attachment 1

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

M. Adams, Supervisor, System Engineering

S. Anders, Manager, Plant Security

B. Briner, Technical Specialist IV, Componet Engineering

K. Christian, Director, Nuclear Safety Assurance

K. Cook, Manager, Operations

C. Fugate, Assistant Manager, Operations

D. Gallodoro, Senior Engineer, Design Engineering

J. Kowalewski, Site Vice President, Operations

B. Lanka, Manager, Design Engineering

J. Lewis, Manager, Emergency Preparedness

B. Lindsey, Manager, Maintenance

M. Mason, Senior Licensing Specialist, Licensing

W. McDonald, Senior Engineer, System Engineering

W. McKinney, Manager, Corrective Action and Assessments

R. Murillo, Manager, Licensing

K. Nicholas, Director, Engineering

O. Pipkins, Senior Licensing Specialist, Licensing

R. Putnam, Manager, Programs and Components

G. Scot, Engineer, Licensing

R. Williams, Senior Licensing Specialist, Licensing

LIST OF ITEMS OPENED

Opened 05000382/2009008-01

AV

Inoperable 125 Vdc battery because electricians failed to

follow work instructions

A-2

Attachment 1

LIST OF DOCUMENTS REVIEWED

Section 1R15: Operability Evaluations

CONDITION REPORTS

WORK ORDERS

108092

152819

51655765

148345

51639921

51641394

51642811

51645301

51646600

51647737

51655919

51648845

51654686

51655765

163830

51670476

164047

160936

154656

51653558

51649933

51651031

51652069

PROCEDURES/DOCUMENTS

NUMBER

TITLE

REVISION /

DATE

EN-LI-118

Root Cause Analysis Process

8

EN-HU-103

Human Performance Error Reviews

1

EN-WM-102

Work Implementation and Closeout

2

EN-WM-105

Planning

4

EN-MA-101

Conduct of Maintenance

6

MG-33

Configuration and Control Guidelines & Completing Lifted

Lead & Switch Manipulation Forms

1

White Paper

Evaluation of Potential Tampering or Sabotage to Station

Battery 3B-S

12/22/08

White Paper

Recovery Action Evaluation for Battery 3B-S Loose Cell

57 Connection

A-3

Attachment 1

NUMBER

TITLE

REVISION /

DATE

White Paper

Engineering Evaluation for Potential to Damage Battery

3B-S Loose Cell #57 Connection

White Paper

Core Damage Risk Associated with Waterford 3

DC-EBAT-B Unavailable

2

ME-004-213

Battery Intercell Connections

13

ME-003-220

Station Battery Bank and Charger (18 month)

301

ME-003-230

Battery Service Test

301

ME-003-200

Station Battery Bank and Charger (Weekly)

301

ME-003-210

Station Battery Bank and Charger (Quarterly)

12

OP-901-313

Loss of a 125V DC Bus

300

OI-037-000

Operations Risk Assessment Guideline

2

OP-006-003

125 VDC Electrical Distribution

301

OP-902-005

Station Blackout Recovery

13

OP-009-002

308

08-0540

EOS Checklist for Battery 3B-S Inoperable

9/3/08

A-1

Attachment 2

Phase 3 Analysis

Waterford 3

Battery Loose Inter-cell Connection

Performance Deficiency:

Inadequate maintenance following replacement of a cell on Station Battery 3B-S on May 24,

2008, resulted in a loose connection between cells 57 and 58. The battery was determined to be

non-functional on September 2, 2008, based on a measurement of connector resistance and

tests of individual cell voltage.

Assumptions:

1. Battery 3B-S was potentially capable of performing its safety function immediately following

its replacement on May 24, 2008, based on a satisfactory service test. The battery became

non-functional sometime after May 24 and sometime before September 2 (100 days later).

The weekly individual cell voltage measurements were not true tests of the battery's ability

to perform its safety function because they did not simulate the initial load condition that

would exist following a loss of offsite power. Therefore, the point in time that the battery

became non-functional is unknown, but is assumed as being half way between the two

known points (t/2). Repair time was approximately 2 days. Therefore, the exposure time of

the condition is estimated as 100 days/2 + 2 days = 52 days.

2. During the exposure period, it is assumed that the battery would fail to provide any service

function, including the start of the Train B emergency diesel generator. Following a loss of

offsite power event, recovery of the battery would be possible depending on the extent of

damage from the current surge across the loose connection. For the purpose of this

analysis, it is assumed, based on a qualitative estimate, that there is a 15 percent probability

that damage of an irreparable nature would occur, and an 85 percent chance that the

battery would remain intact and could be recovered by tightening the loose connection,

jumpering out the damaged cell, or by installing a spare.

The core damage sequences that contribute to the delta-CDF are of durations of 1 or

6 hours6.944444e-5 days 0.00167 hours 9.920635e-6 weeks 2.283e-6 months . It is assumed that a one-hour recovery of the battery would not be possible and

therefore, only the 6-hour sequences are considered available for recovery.

Using the SPAR-H Human Reliability Analysis Method, NUREG/CR-6883, the following

assumptions were made for the diagnosis and action performance shaping factors:

DIAGNOSIS (0.01 NOMINAL)

Performance

Shaping Factor

Level

Factor

Available Time

Expansive Time

0.01

Stress

High

2

Complexity

Moderate

2

Experience/Training

Low

10

Procedures

Not Available

50

Ergonomics

Nominal

1

Nominal

1

Work Processes

Nominal

1

A-2

Attachment 2

Diagnostic Result = (0.01)(20)/[(0.01)(20 - 1) +1] = 0.168

Available Time: It is estimated that the nominal time to diagnose the condition would be

one hour. Considering the short time needed to correct the problem, approximately 3 hours3.472222e-5 days 8.333333e-4 hours 4.960317e-6 weeks 1.1415e-6 months

of time would be available to diagnose the condition and leave enough time to either tighten

the connection, jumper the cell, or replace it. Therefore, the time available is greater than

2 times nominal and > 30 minutes, meeting the criteria for expansive.

Stress: The condition of an SBO would be high stress for the operators, but not extreme,

because immediate threats to health and life would be absent.

Complexity: There could be conditions under which the source of the battery problem would

not be readily apparent. This could lead to a need to check all of the cells individually, or a

decision to abandon recovery of the battery and focus on recovering the alternate EDG.

Experience/Training: Operators do not have experience in diagnosing this type of failure

(low).

Procedures: Procedures were not available directing the diagnosis of the battery condition.

Ergonomics: There are no ergonomic impediments.

Fitness for Duty and Work Processes: These factors were considered nominal.

ACTION (0.001 NOMINAL)

Performance

Shaping Factor

Level

Factor

Available Time

>5 times nominal

0.1

Stress

High

2

Complexity

Nominal

1

Experience/Training

Low

3

Procedures

Nominal

1

Ergonomics

Nominal

1

Nominal

1

Work Processes

Nominal

1

Action result = 6E-4

Available Time: It is estimated that the nominal time to perform the actions would be one-

half hour. Given diagnosis within 3 hours3.472222e-5 days 8.333333e-4 hours 4.960317e-6 weeks 1.1415e-6 months , an additional 3 hours3.472222e-5 days 8.333333e-4 hours 4.960317e-6 weeks 1.1415e-6 months would be available before

battery depletion. This meets the criteria for being > 5 times nominal.

Stress: The condition of an SBO would be high stress for the operators, but not extreme,

because immediate threats to health and life would be absent.

Complexity: The steps needed to perform the recovery are not complex.

Experience/Training: Operators do not have experience in performing this recovery.

A-3

Attachment 2

Procedures: Procedures are available and are of a quality commensurate with standard

plant procedures.

Ergonomics: There are no ergonomic impediments.

Fitness for Duty and Work Processes: These factors were considered nominal.

Total HRA result = 0.168 + 0.0006 = 0.169

3. In the event that the battery is heavily damaged and cannot be recovered, it would be

possible to recover the Train B EDG (and dc bus through the battery charger) by connecting

an alternate dc source and starting the Train B EDG. Because loss of the dc bus would be

obvious, the diagnosis portion of the recovery was considered to be the operator decision to

attempt the special recovery. Although a procedure (using a special rigging of automobile

batteries) existed previously to perform this recovery, a subsequent revision removed it prior

to the beginning of the exposure period for this condition. Using the SPAR-H Human

Reliability Analysis Method, NUREG/CR-6883, the following assumptions were made for the

diagnosis and action performance shaping factors:

[Note: the CDF sequences that lead to core damage within one hour were considered to be

too short in time to accomplish a recovery. Therefore, the following assessment applies only

to sequences with a time to core damage of greater than one hour, which, in this case, are

exclusively the 6-hour sequences.]

DIAGNOSIS (0.01 NOMINAL)

Performance

Shaping Factor

Level

Factor

Available Time

Extra Time

0.1

Stress

High

2

Complexity

Nominal

1

Experience/Training

Low

10

Procedures

Not Available

50

Ergonomics

Nominal

1

Nominal

1

Work Processes

Nominal

1

Diagnostic Result = (0.01)(100)/[(0.01)(100 - 1) +1] = 0.502 (1 in 2 chance that the operators

will attempt the alternate recovery procedure)

Available Time: It is estimated that the nominal time to diagnose the condition and decide to

proceed with the alternate dc procedure would be approximately two hours. Therefore, for 6

hour or greater sequences, the amount of time available to decide to use the procedure, but

still have enough remaining time to perform the actions, would between 1X and 2X nominal

and greater than 30 minutes.

Stress: The condition of an SBO would be high stress for the operators, but not extreme,

because immediate threats to health and life would be absent.

Complexity: Nominal

A-4

Attachment 2

Experience/Training: Operators do not have experience in diagnosing this type of failure

(low).

Procedures: Procedures were not available directing the use of the alternate dc source.

Ergonomics: There are no ergonomic impediments

Fitness for Duty and Work Processes: These factors were considered nominal.

ACTION (0.001 NOMINAL)

Performance

Shaping Factor

Level

Factor

Available Time

Nominal

1

Stress

High

2

Complexity

Moderately Complex

2

Experience/Training

Low

3

Procedures

Not Available

50

Ergonomics

Poor

10

Nominal

1

Work Processes

Nominal

1

Action Result = (0.001)(6000)/[(0.001)(6000 - 1) +1] = 0.857

Available Time: It is estimated that the nominal time to perform the actions necessary to

start the Train B EDG with an alternate dc source would be approximately two hours.

Therefore, for 6 hour6.944444e-5 days 0.00167 hours 9.920635e-6 weeks 2.283e-6 months or greater sequences, the amount of time available would be

considered nominal.

Stress: The condition of an SBO would be high stress for the operators, but not extreme,

because immediate threats to health and life would be absent.

Complexity: The steps needed to perform the recovery are moderately complex.

Experience/Training: Operators do not have experience in performing this recovery.

Procedures: Procedures are available but are not of a quality commensurate with standard

plant procedures.

Ergonomics: There are some difficulties associated with ergonomic impediments.

Fitness for Duty and Work Processes: These factors were considered nominal.

The total failure probability is the inverse of the probability that both diagnosis and action are

successful. Total HEP = 1 - (1 - 0.502)(1 - 0.857) = 0.93.

4. A common cause failure of the other vital 125 volt batteries (3A-S and 3AB-S) was not

considered to be applicable to this failure. The replacement and maintenance performed on

Battery 3B-S was not performed contemporaneously on the other batteries. Also, the

condition, if it had previously existed on the other batteries, would most likely have been

discovered through testing. All of the connections on the other two batteries were verified to

A-5

Attachment 2

be tight. The probability of the basic event for the common cause loss of all vital 125-volt dc

batteries is 1.551E-7 in the base case. When the failure of battery 3B-S is assigned a value

of 1.0 in SAPHIRE (indicating an independent failure), the common cause probability is

recalculated to reflect a two-battery system (instead of three). The revised common cause

failure probability is 4.789E-7. Because the independent failure of the batteries is 4.8E-5,

the change in the common cause probability had a negligible effect on the analysis. For

reference, if the condition had been determined to be a common cause situation, and the

Battery 3B-S basic event was assigned a value of TRUE instead of 1.0, the common cause

failure probability would have been adjusted to 3.231E-3. This would have significantly

increased the estimated significance of the finding.

5. An error was discovered in the Waterford 3 SPAR model concerning power supplies to the

EFW flow control valves. A revised model was provided by INL for use by the analyst. The

impact of the change was to decrease the significance of the finding by approximately

20 percent.

6. An error was found in the Waterford 3 SPAR model concerning excluded test and

maintenance basic events. The events ACW-CTF-TM-A/B (ACCW wet cooling tower test

and maintenance) were miscoded as ACW-CTW-TM-A/B. Because of this problem, test

and maintenance situations prohibited by technical specifications were being inappropriately

included in the tabulation. This error was corrected.

7. The Waterford 3 SPAR model credits a 4-hour battery capacity for station blackout

sequences. The licensee PRA model credits a battery capacity of 6 hours6.944444e-5 days 0.00167 hours 9.920635e-6 weeks 2.283e-6 months following a station

blackout. This value is contingent on operators implementing a dc load shed procedure that

is part of their training program. The Waterford SPAR model credits a 4-hour battery

capacity. The analyst revised the SPAR model to credit a 6-hour battery. Although operator

action is required to extend the battery capacity, the probability that operators will fail to

shed loads according to the procedure is very small (~E-3), such that the contribution to the

significance of the finding that would result in modeling this operator failure would be

negligible.

8. Hurricane Gustav, which passed several hundred miles west of the plant during the

exposure period, increased the probability of a loss of offsite power. However, for SDP

analyses, average conditions are assumed for external events as well as test and

maintenance activities, reflecting the philosophy that the performance deficiency could have

occurred at any time. Also, the plant shut down when projected local wind speeds were

within the range of hurricane force. Therefore, no adjustments were made for the hurricane.

Analysis:

The analysis was performed with the Waterford 3 SPAR model, Revision 3.45, dated July 13,

2008, and revised by INL and corrected as discussed above. Average test and maintenance

was used and truncation was set at 1.0E-13. The basic event DCP-BAT-LP-3BS, Failure of

Division 3B 125 VDC Battery 3B-S, was set to a value of 1.0.

[for reference purposes, the first analysis was performed without recovery of the Train B EDG]

A-6

Attachment 2

The result using SAPHIRE 7.27 was a Delta-CDF of 7.914E-5/yr. The following were the top

8 sequences contributing to the change in CDF (99.8% of the total):

SEQUENCE

INITIATING EVENT AND

SYSTEMS THAT FAIL

DELTA-

PERCENTAGE

OF TOTAL

LOOP 15-21

(LOOP)(EPS)(CBO)(RSUB)(OPR-

06H)(DGR-06H)

6.149E-5

77.7

LOOP 15-30

(LOOP)(EPS)(EFW)(OPR-

01H)(DGR-01H)

1.239E-5

15.7

LOOP 14

(LOOP)(EFW)

4.007E-6

5.06

LOOP 15-27

(LOOP)(EPS)(SRV)(OPR-

01H)(DGR-01H)

5.169E-7

0.653

LOOP 15-24

(LOOP)(CBO)(RSUB)(RCPSI)(

OPR-01H)(DGR-01H)

3.549E-7

0.448

LDCAB 12

(LDCAB)(FW)(COND)

7.651E-8

0.097

LOMFW 12

(LOMFW)(FW)(COND)

5.749E-8

0.073

LOCHS 12

(LOCHS)(FW)(COND)

4.598E-8

0.058

LOOP: Loss of offsite power

EPS: Emergency AC power (diesel generators)

CBO: Controlled bleedoff isolated

RSUB: Reactor coolant subcooling maintained

OPR-01H: recovery of offsite power in 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months

DGR-01H: recovery of an emergency diesel generator in 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months

OPR-06H: recovery of offsite power in 4 hours4.62963e-5 days 0.00111 hours 6.613757e-6 weeks 1.522e-6 months

DGR-06H: recovery of an emergency diesel generator in 4 hours4.62963e-5 days 0.00111 hours 6.613757e-6 weeks 1.522e-6 months

EFW: Emergency feedwater system

FW: EFW and main feedwater systems

LOMFW: Loss of main feedwater

COND: Secondary cooling using condensate system

SRV: Safety relief valves are closed

LOCHS: Loss of condenser heat sink

RCPSI: RCP seal integrity maintained

LDCAB: Loss of DC Bus 3AB-DC-S

The non-LOOP sequences, contributing slightly over 0.3% to the result, included failures of a

fast-bus transfer to the vital 4160 vac bus following a reactor trip, followed by battery failure and

a failure to start the Train B EDG. This scenario would challenge the battery in a manner

equivalent to a LOOP event and therefore the associated sequences were considered

applicable to this analysis.

Assuming an exposure period of 52 days, the estimated no-recoverydelta-CDF of the finding is

7.914E-5/yr (52 days/yr/365 days/yr) = 1.13E-5/yr.

Application of Train B Battery and EDG B recoveries:

In the SAPHIRE result above, 99.99 percent of the delta-CDF was developed through base

case cut sets that contained the independent failure of Battery 3B-S (base failure probability =

4.8E-5) that were increased in value by assigning a failure probability of 1.0. The common

A-7

Attachment 2

cause basic event (which was increased from 1.551E-7 in the base to 4.789E-7 in the case)

was virtually not represented in the tabulation because it was very small and was almost entirely

truncated out (almost all cutsets containing the common cause term had values less than the

truncation limit of 1.0E-13).

Sequence LOOP 15-21 is a six-hour sequence and was considered applicable to both

recoveries. The other listed sequences (LOOP 15-30, LOOP 14, LOOP 15-27, LOOP 15-24,

LDCAB 12, LOMFW 12, and LOCHS 12) are short sequences and were not credited with a

recovery.

According to Assumption #2, there is 85 percent probability that the battery will not be damaged

beyond a state that allows for its recovery. The HRA estimate for this recovery is 0.169.

For this situation, the basic event DCP-BAT-LP-3BS was set to a failure probability of 0.169 (the

non-recovery probability) and the common cause basic event DCP-BAT-CF-ALL was reset to its

original 2-battery group value of 4.789E-7. Sequence LOOP 15-21 was re-quantified. The

change in Delta-CDF for this sequence is shown below:

SEQUENCE

DELTA-CDF

VALUE W/O

RECOVERY

DELTA-CDF

VALUE W/

RECOVERY

DECREASE

IN DELTA-

LOOP 15-21

6.149E-5

1.040E-5

5.009E-5

According to Assumption #2, there is 15 percent probability that the battery will be damaged

beyond a state that allows for its recovery. The HRA estimate for recovery (Assumption 3) of

the EDG is 0.93.

The basic event DCP-BAT-LP-3BS was set to a failure probability of 0.93 (this acceptably

simulates an EDG recovery for modeling purposes), and the common cause basic event

DCP-BAT-CF-ALL was reset to the 2-battery group value of 4.789E-7. Sequence LOOP 15-21

was re-quantified. The change in Delta-CDF for this sequence is shown below:

SEQUENCE

DELTA-CDF

VALUE W/O

RECOVERY

DELTA-CDF

VALUE W/

RECOVERY

DECREASE

IN DELTA-