ML071430289

IR 05000298-07-007; 01/28/07 - 04/24/07; Cooper Nuclear Station. Special Inspection in Response to Emergency Diesel Generator 2 Failure on January 18, 2007

ML071430289
Person / Time
Site:	Cooper
Issue date:	05/22/2007
From:	Howell A NRC/RGN-IV/DRP
To:	Minahan S Nebraska Public Power District (NPPD)
References
EA-07-090 IR-07-007
Download: ML071430289 (39)

See also: IR 05000298/2007007

Text

May 22, 2007

EA 07-090

Stewart B. Minahan, Vice

President-Nuclear and CNO

Nebraska Public Power District

P.O. Box 98

Brownville, NE 68321

SUBJECT: COOPER NUCLEAR STATION - NRC SPECIAL INSPECTION

REPORT 05000298/2007007

Dear Mr. Minahan:

On April 24, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed a special

inspection at your Cooper Nuclear Station. This inspection examined activities associated with

the Emergency Diesel Generator 2 failure that occurred on January 18, 2007. During this

event the emergency diesel generator automatically isolated from the electrical bus following an

over-current condition as a result of a voltage regulator failure. The NRCs initial evaluation

satisfied the criteria in NRC Management Directive 8.3, NRC Incident Investigation Program,

for conducting a special inspection. The basis for initiating the special inspection is further

discussed in the Charter, which is included as Attachment 2 to the enclosed report. The

determination that the inspection would be conducted was made by the NRC on January 26,

2007, and the inspection started on January 29, 2007.

The enclosed special inspection report documents the inspection findings which were discussed

on April 24, 2007, with Mr. Mike Colomb, General Manager of Plant Operations, and other

members of your staff. The inspection examined activities conducted under your license as

they relate to safety and compliance with the Commissions rules and regulations and with the

conditions of your license. The team reviewed selected procedures and records, observed

activities, and interviewed personnel.

The enclosed report discusses one finding that appears to have low to moderate safety

significance (White). As described in Section 6.0 of this report, the NRC concluded that the

failure to establish appropriate procedural controls for evaluating the use of parts of

indeterminate quality prior to their installation in safety-related systems resulted in Emergency

Diesel Generator 2 failure on January 18, 2007. The safety significance of this finding was

assessed on the basis of the best available information, including influential assumptions, using

the applicable Significance Determination Process and was preliminarily determined to be a

White (i.e., low to moderate safety significance) finding. Attachment 3 of this report provides a

detailed description of the preliminary risk assessment. In accordance with NRC Inspection

Nebraska Public Power District -2-

Manual Chapter (IMC) 0609, Significance Determination Process, we intend to complete our

evaluation using the best available information and issue our final determination of safety

significance within 90 days of this letter.

This finding does not represent an immediate safety concern because of the corrective actions

you have taken. These actions included the installation of properly dedicated voltage regulator

components for Emergency Diesel Generator 2 following the failure that occurred on

January 18, 2007.

Also, this finding constitutes an apparent violation of NRC requirements and is being

considered for escalated enforcement action in accordance with the NRC Enforcement Policy.

The current Enforcement Policy is included on the NRCs Web site at www.nrc.gov; select

About NRC, How We Regulate, Enforcement, then Enforcement Policy.

Before we make a final decision on this matter, we are providing you an opportunity (1) to

present to the NRC your perspectives on the facts and assumptions, used by the NRC to arrive

at the finding and its significance, at a Regulatory Conference or (2) submit your position on the

finding to the NRC in writing. If you request a Regulatory Conference, it should be held within

30 days of the receipt of this letter and we encourage you to submit documentation at least one

week prior to the conference in an effort to make the conference more efficient and effective. If

a Regulatory Conference is held, it will be open for public observation. If you decide to submit

only a written response, such submittal should be sent to the NRC within 30 days of the receipt

of this letter.

Please contact Michael Hay at (817) 860-8144 within 10 business days of the date of this letter

to notify the NRC of your intentions.

If you choose to provide a written response, it should be clearly marked as a Response to An

Apparent Violation in Inspection Report No. 05000298/2007007: EA-07-090 and should

include: (1) the reason for the apparent violation, or, if contested, the basis for disputing the

apparent violation; (2) the corrective steps that have been taken and the results achieved; (3)

the corrective steps that will be taken to avoid further violations; and (4) the date when full

compliance will be achieved. Your response may reference or include previously docketed

correspondence, if the correspondence adequately addresses the required response. If an

adequate response is not received within the time specified or an extension of time has not

been granted by the NRC, the NRC will proceed with its enforcement decision or schedule a

Regulatory Conference.

Since the NRC has not made a final determination in this matter, no Notice of Violation is being

issued for the inspection finding at this time. In addition, please be advised that the number

and characterization of the apparent violation described in the enclosed inspection report may

change as a result of further NRC review.

The report also documents one NRC-identified finding, which was evaluated under the risk

significance determination process as having very low safety significance (Green). This finding

was determined to involve a violation of NRC requirements. However, because of the very low

safety significance and because it is entered into your corrective action program, the NRC is

Nebraska Public Power District -3-

treating this finding as a noncited violation consistent with Section VI.A.1 of the NRC

Enforcement Policy. If you contest the noncited violation in this report, you should provide a

response within 30 days of the date of this inspection report, with the basis for your denial, to

the U.S. Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington,

DC 20555-0001, with copies to the Regional Administrator, U.S. Nuclear Regulatory

Commission, Region IV, 611 Ryan Plaza Drive, Suite 400, Arlington, Texas 76011-4005; the

Director, Office of Enforcement, U.S. Nuclear Regulatory Commission, Washington,

DC 20555-0001; and the NRC Resident Inspectors at the Cooper Nuclear Station.

In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and its

enclosures will be made available electronically for public inspection in the NRC Public

Document Room or from the Publicly Available Records (PARS) component of NRCs

document system (ADAMS). ADAMS is accessible from the NRC Web site at

http://www.nrc.gov/reading-rm/adams.html.

Sincerely,

/RA/

Arthur T, Howell III, Director

Division of Reactor Projects

Docket: 50-298

License: DPR-46

Enclosure: NRC Inspection Report 05000298/2007007

Attachment 1: Supplemental Information

Attachment 2: Special Inspection Charter

Attachment 3: Significance Determination Evaluation

cc w/Enclosure:

Gene Mace P. V. Fleming, Licensing Manager

Nuclear Asset Manager Nebraska Public Power District

Nebraska Public Power District P.O. Box 98

P.O. Box 98 Brownville, NE 68321

Brownville, NE 68321

Michael J. Linder, Director

John C. McClure, Vice President Nebraska Department of

and General Counsel Environmental Quality

Nebraska Public Power District P.O. Box 98922

P.O. Box 499 Lincoln, NE 68509-8922

Columbus, NE 68602-0499

Nebraska Public Power District -4-

Melanie Rasmussen, Radiation Control

Chairman Program Director

Nemaha County Board of Commissioners Bureau of Radiological Health

Nemaha County Courthouse Iowa Department of Public Health

1824 N Street Lucas State Office Building, 5th Floor

Auburn, NE 68305 321 East 12th Street

Des Moines, IA 50319

Julia Schmitt, Manager

Radiation Control Program Ronald D. Asche, President

Nebraska Health & Human Services and Chief Executive Officer

Dept. of Regulation & Licensing Nebraska Public Power District

Division of Public Health Assurance 1414 15th Street

301 Centennial Mall, South Columbus, NE 68601

P.O. Box 95007

Lincoln, NE 68509-5007 Kevin V. Chambliss, Director of

Nuclear Safety Assurance

H. Floyd Gilzow Nebraska Public Power District

Deputy Director for Policy P.O. Box 98

Missouri Department of Natural Resources Brownville, NE 68321

P. O. Box 176

Jefferson City, MO 65102-0176 John F. McCann, Director, Licensing

Entergy Nuclear Northeast

Director, Missouri State Emergency Entergy Nuclear Operations, Inc.

Management Agency 440 Hamilton Avenue

P.O. Box 116 White Plains, NY 10601-1813

Jefferson City, MO 65102-0116

Keith G. Henke, Planner

Chief, Radiation and Asbestos Division of Community and Public Health

Control Section Office of Emergency Coordination

Kansas Department of Health 930 Wildwood, P.O. Box 570

and Environment Jefferson City, MO 65102

Bureau of Air and Radiation

1000 SW Jackson, Suite 310 Chief, Radiological Emergency

Topeka, KS 66612-1366 Preparedness Section

Kansas City Field Office

Daniel K. McGhee, State Liaison Officer Chemical and Nuclear Preparedness

Bureau of Radiological Health and Protection Division

Iowa Department of Public Health Dept. of Homeland Security

Lucas State Office Building, 5th Floor 9221 Ward Parkway

321 East 12th Street Suite 300

Des Moines, IA 50319 Kansas City, MO 64114-3372

Nebraska Public Power District -5-

Electronic distribution by RIV:

Regional Administrator (BSM1)

DRP Director (ATH)

DRS Director (DDC)

DRS Deputy Director (RJC1)

Senior Resident Inspector (NHT)

Branch Chief, DRP/C (MCH2)

Senior Project Engineer, DRP/C (WCW)

Team Leader, DRP/TSS (CJP)

RITS Coordinator (MSH3)

Only inspection reports to the following:

DRS STA (DAP)

L. Trocine, OEDO RIV Coordinator (LXT)

ROPreports

CNS Site Secretary (SEF1)

D. Starkey, OE (DRS)

OE Mail

M. Ashley, NRR (MAB)

M. Vasquez (GMV)

V. Dricks (VLD)

W. Maier (WAM)

SUNSI Review Completed: _WCW ADAMS: : Yes G No Initials: __WCW

Publicly Available G Non-Publicly Available G Sensitive  : Non-Sensitive

R:\_REACTORS\CNS\2007\CNS2007-07 RP Special.wpd_

RIV:DRS RI:DRP/C SRA:DRS C:DRP/C ACES

SPRutenkroger NHTaylor MFRunyan MCHay MHaire

E-WCWalker T-Walker /RA/ /RA/ /RA/

5/14/07 5/17/07 5/9/07 5/17/07 5/10/07

D:DRP

ATHowell III

/RA/

5/22/07

OFFICIAL RECORD COPY T=Telephone E=E-mail F=Fax

U.S. NUCLEAR REGULATORY COMMISSION

REGION IV

Docket.: 50-298

License: DPR-46

Report: 05000298/2007007

Licensee: Nebraska Public Power District

Facility: Cooper Nuclear Station

Location: P.O. Box 98

Brownville, Nebraska

Dates: January 28 through April 24, 2007

Inspectors: N. Taylor, Resident Inspector

S. Rutenkroger, PHD, Reactor Inspector

Approved By: Arthur T. Howell III, Director

Division of Reactor Projects

-1- Enclosure

SUMMARY OF FINDINGS

IR 05000298/2007007; 01/28/07 - 04/24/07; Cooper Nuclear Station. Special Inspection in

response to Emergency Diesel Generator 2 failure on January 18, 2007.

This report documents the special inspection activities conducted by one resident inspector and

one reactor inspector. One apparent violation and one non-cited violation were identified. The

significance of the issues is indicated by their color (Green, White, Yellow, or Red) and was

determined by the significance determination process in Inspection Manual Chapter 0609.

Findings for which the significance determination process does not apply are indicated by the

severity level of the applicable violation. The NRC's program for overseeing the safe operation

of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight

Process, Revision 3, dated July 2000.

A. NRC-Identified and Self-Revealing Findings

Cornerstone: Mitigating Systems

• TBD. The team identified an apparent violation of 10 CFR Part 50, Appendix B,

Criterion V, "Instructions Procedures, and Drawings," for the failure to establish

procedural controls for evaluating the use of parts of indeterminate quality prior

to their installation in safety-related applications. This procedural deficiency

resulted in the installation of a voltage regulator circuit board of indeterminate

quality that adversely affected the function of Emergency Diesel Generator 2.

Specifically, following installation of the part on November 11, 2006, failure of the

part occurred following 35 hours4.050926e-4 days <br />0.00972 hours <br />5.787037e-5 weeks <br />1.33175e-5 months <br /> of operation resulting in an over-voltage trip of

Emergency Diesel Generator 2 on January 18, 2007.

The finding is greater than minor because it is associated with the equipment

performance cornerstone objective to ensure the availability, reliability, and

capability of systems that respond to initiating events to prevent undesirable

consequences. Using NRC Inspection Manual Chapter 0609, Significance

Determination Process, Phase 1 Worksheet, a Phase 2 evaluation was required

because the finding resulted in the loss of the safety function of Emergency

Diesel Generator 2 for greater than the Technical Specification completion time.

The Phase 2 evaluation concluded that the finding was of low to moderate safety

significance. A Phase 3 preliminary significance determination analysis also

determined the finding was of low to moderate safety significance.

• Green. The team identified three examples of a noncited violation of Technical

Specification 5.4.1.a involving the failure to establish adequate maintenance

procedures for work performed on Emergency Diesel Generator 2. These

inadequate procedures failed to identify a degraded condition in the voltage

regulator off-manual-auto switch and contributed to an over-voltage trip of

Emergency Diesel Generator 2 that occurred on November 13, 2006.

-2- Enclosure

The finding is more than minor because it is associated with the Mitigating

Systems cornerstone attribute of procedure quality and affects the associated

cornerstone objective to ensure the availability, reliability, and capability of

systems that respond to initiating events. Specifically, the performance

deficiency resulted in: (1) the failure to discover a degraded condition in the

Emergency Diesel Generator 2 voltage regulator, and (2) an over-voltage trip

during the tuning of Emergency Diesel Generator 2 on November 13, 2006.

Using the Manual Chapter 0609 Appendix G, "Shutdown Operations Significance

Determination Process," Phase 1 Checklist, the finding is determined to have

very low safety significance because one operable diesel generator was still

capable of supplying power to the class 1E electrical power distribution

subsystems. This finding has a cross-cutting aspect in the area of human

performance in that the licensees procedures were not complete and provided

inadequate instructions for conducting maintenance associated with Emergency

Diesel Generator 2.

-3- Enclosure

REPORT DETAILS

1.0 Special Inspection Report

The NRC conducted this special inspection to better understand the circumstances

surrounding the failure of Emergency Diesel Generator (EDG) 2 that occurred on

January 18, 2007. The failure of EDG 2 occurred during a routine monthly surveillance

test following approximately four hours of operation. The cause of the event was the

result of a failure of a voltage regulator card that had been installed in November 2006.

In accordance with NRC Management Directive 8.3, NRC Incident Investigation

Program, it was determined that this event met the deterministic criteria and had

sufficient risk significance to warrant a special inspection.

The team used NRC Inspection Procedure 93812, Special Inspection Procedure, to

conduct the inspection. The special inspection team reviewed procedures, corrective

action documents, as well as design and maintenance records for the equipment of

concern. The team interviewed key station personnel regarding the event, reviewed the

root cause analysis, and assessed the adequacy of corrective actions. A list of specific

documents reviewed is provided in Attachment 1. The charter for the special inspection

is provided as Attachment 2.

2.0 Review of EDG Voltage Regulator Problems

The team conducted a review of all work orders, surveillance test records, and condition

reports associated with the electrical components of EDG 2 written from January 2004

through January 2007. The following discusses several discrete periods of degraded

performance of the EDG 2 voltage regulator.

EDG 2 Voltage Regulator Performance in 2005

On three occasions in 2005, operators initiated condition reports (CRs) documenting

difficulties maintaining stable reactive load on EDG 2. In all reported occurrences,

kilovolt-ampere-reactive (KVAR) step changes, ranging from 100-300 KVAR, and spikes

up to 2000 KVAR were received during manipulation of the lower-raise voltage regulator

switch in the control room. This remote switch sends demand signals to the motor

operated potentiometer (MOP) providing a reference voltage signal to the voltage

regulator. On the basis of these observations, the licensee replaced the EDG 2 MOP in

December 2005. In addition, the licensee initiated a preventive maintenance (PM) plan

in June 2006 implementing MOP cleaning and inspection activities every operating cycle

and MOP replacement every third operating cycle.

The team noted there were five similar condition reports pertaining to EDG 1 in the

2000-2007 time period. The KVAR changes documented in these condition reports

were consistent with MOP issues affecting EDG 2. The team noted that corrective

actions consisting of the previously discussed PM activities appeared adequate to

address the problem.

-4- Enclosure

The licensee evaluated the impact of these KVAR step changes on the operability of the

EDGs after each occurrence. The KVAR rating of the EDGs is 5000 KVAR, and the

maximum reactive load seen during these transients was approximately 2000 KVAR.

During these KVAR step changes, the licensee did not observe any frequency or real

load changes, and all acceptance criteria of the surveillance test procedures were

satisfied. As such, the licensee determined the operability of the EDGs was not affected

by the KVAR swings, but that the KVAR changes were indicative of a degraded

condition in the voltage regulator system. The team reviewed the licensees evaluation

for this degraded condition and determined it adequately provided reasonable

assurance of operability.

EDG 2 Voltage Regulator Performance in 2006

Despite the installation of a new MOP in December 2005, unpredictable KVAR changes

began occurring again early in 2006. In addition to KVAR swings during manipulation of

the lower-raise voltage regulator switch, the licensee noted KVAR swings during steady

state operation of EDG 2. This new trend suggested that a failure mechanism existed

that was unrelated to the MOP. In response to this trend, the licensee initiated CR-

CNS-2006-2729 and developed actions to review the overall health of the voltage

regulating system and utilized vendor support to develop actions to resolve the problem.

One of the actions coming from the CR was to perform extenistive troubleshooting

during Refueling Outage 23 (RE23), specifically directed at cleaning the MOP and

checking circuit continuity throughout the voltage regulator system. Work Order (WO)

4514076 was created to perform this task on EDG 2.

EDG 2 Over-Voltage Event on November 11, 2006

One of the troubleshooting steps included in WO 4514076 was to wipe the R13

feedback adjust potentiometer on the voltage regulator printed circuit board. This

involved turning a small screw on the potentiometer and exercising the slide wire

resistor through its full range of travel. During performance of this step on November 8,

2006, the resistor failed due to an open circuit condition. The licensee had anticipated

problems with the printed circuit board and initiated a contingency in the WO to replace

the board with a spare.

Replacement of the printed circuit board required a tuning process to establish the

correct voltage regulator response. WO 4514076 contained the procedural guidance to

complete this task. This tuning process involved, in part, running EDG 2 unloaded,

initiating small voltage changes, measuring the time response of the ensuing voltage

transient, and making incremental adjustments in the position of the R13 potentiometer.

During this process the licensee had several pieces of test equipment installed to

monitor the EDG output voltage response.

During the unloaded run of EDG 2 for this activity, operations personnel shut the

machine down because of unexpected fuel rack movement (the team reviewed this

condition and determined that this was normal fuel rack behavior that did not affect EDG

2 operability). While the EDG was shutdown, electrical technicians investigated a

potential problem with an installed oscilloscope being used as test equipment. The

-5- Enclosure

technicians connected a variac to the oscilloscope and introduced a 60 volt test signal.

The technicians realized immediately that they had failed to disconnect the oscilloscope

from the voltage regulator system and disconnected it. Unknown at the time to the

technicians, this erroneous act resulted in blowing a fuse on one of three phases of the

voltage regulator potential transformer, adversely affecting the voltage regulator

summing network that senses generator output voltage. As a result, during the

subsequent restart of EDG 2, the voltage regulator sensed an artificially low output

voltage and drove voltage up until the EDG 2 over-voltage trip occurred at 6125 volts.

The performance aspects of this event resulted in a non-cited violation of NRC

requirements, which is described in NRC Integrated Inspection Report 05000298/2006005 (ML070360639).

EDG 2 Over-voltage Event on November 13, 2006

On November 13, 2006, the licensee replaced the blown fuse and continued with the

voltage regulator tuning process. The process required operators to make small

changes in voltage around rated voltage, after which engineers reviewed the ensuing

voltage transient as recorded on a strip chart. This was an iterative process that

involved several voltage changes. Approximately ten seconds after one voltage change

EDG 2 experienced a pair of voltage spikes; the first to approximately 5500 volts and

the second to greater than 5900 volts, resulting in an over-voltage EDG trip.

In response to this event, the licensee initiated CR-CNS-2006-9096. The licensee

conducted significant troubleshooting efforts to ascertain the cause of the over-voltage

condition. These troubleshooting efforts included continuity checks, testing of voltage

regulator connectors, and five monitored test runs of EDG 2. Unable to find a definitive

cause for the trip, the licensee contacted a nuclear field service contractor experienced

in voltage regulator maintenance. In the apparent cause report attached to CR-CNS-

2006-9096, the licensee reported that on the basis of the troubleshooting conducted, the

acceptable operation of EDG 2 in the five test runs, and the vendors experience, the

apparent cause of the trip was the erratic behavior of one or both of the potentiometers

of the voltage regulator card. The team reviewed the design of the voltage regulator,

vendor instructions, industry maintenance guidelines, and industry operating experience,

and was unable to find any information that corroborated this apparent cause. The

apparent cause report suggested corrective actions to improve the training of those

involved in voltage regulator maintenance and to develop a voltage regulator testing

procedure to allow onsite personnel to tune the voltage regulator.

Following the completion of troubleshooting, EDG 2 was run without incident in a

number of surveillance tests, including a sequential load test, at the end of the RE23

outage. In addition, a routine four-hour surveillance test was completed successfully on

December 12, 2006.

EDG 2 Over-voltage Event on January 18, 2007

On January 18, 2007, the licensee conducted Surveillance Procedure 6.2DG.101,

Diesel Generator 31 Day Operability Test. After approximately 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> and 13

minutes at full load, the EDG 2 output breaker tripped open on an over-current

-6- Enclosure

condition, followed immediately by an over-voltage trip of EDG 2. During the transient,

the KVAR loading of the machine reached a peak of 10,667 KVAR and the output

current reached a peak of approximately 1700 amps when the EDG output circuit

breaker and the bus tie breaker to the safety related bus tripped open on over-current.

With EDG 2 disconnected from the bus, output voltage rose rapidly until the machine

tripped on over-voltage at 6009 volts.

Following this third over-voltage trip of EDG 2, the licensee formed a root cause analysis

team and completed a significant troubleshooting effort with vendor support. Unable to

locate a failed component, the licensee replaced the suspect voltage regulator card and

performed a six hour loaded run of EDG 2. Based on acceptable performance during

this test and the belief that the degraded condition that caused the January 18th trip no

longer existed, the licensee declared EDG 2 operable on January 22, 2007.

3.0 Root Cause and Immediate Corrective Actions

Root Cause for Voltage Regulator Failures

As documented in CR-CNS-2007-00480, the licensee determined that the root cause of

the January 18, 2007 trip of EDG 2 was that the original procurement process did not

provide technical requirements to reduce the probability of infant mortality failure in the

voltage regulator board in the EDG voltage regulating subsystem. The team reviewed

the licensees root cause methodology and agreed with this conclusion.

The procurement error associated with the voltage regulator card occurred in the Fall of

1973 and involved the inadequate procurement of a commercial grade component for

use in safety related applications. The team noted that in the late 1980's, the NRC

conducted a series of inspections to understand licensees procurement and

commercial-grade dedication programs. These inspections resulted in the NRC

providing additional guidance related to acceptable licensee commercial grade

procurement and dedication programs. Specifically, the NRC issued Generic Letter 91-

05, Licensee Commercial-Grade Procurement and Dedication Programs that provided

guidance for complying with the requirements of 10 CFR Part 50, Appendix B, Quality

Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants, specifically

related to assuring the quality of items purchased and installed in safety related

applications.

The team reviewed Engineering Procedure 3-CNS-DC-138.2, Dedication, Revision 1,

and determined that it had been adequately revised to define expectations for the

dedication of commercial-grade components based on the new guidance. The team

noted that during the years in which substandard procurement practices were in place

that components of indeterminate quality had been accepted into the warehouse as

spare parts acceptable for use in safety- related applications. The team determined that

the licensee had failed to establish adequate procedural guidance to evaluate the use

of these parts of indeterminate quality prior to their installation in safety-related

applications.

The enforcement aspects of this issue are fully described in Section 6 of this report.

-7- Enclosure

Immediate Corrective Actions

In response to the failure of EDG 2 on January 18, 2007, the licensee formed an

integrated response team and staffed the Outage Control Center to track the status of

troubleshooting efforts.

A prompt assessment was performed to determine whether the cause of the failure of

EDG 2 was applicable to EDG 1. The licensee demonstrated that during RE23, EDG 2

had undergone a significant amount of invasive voltage regulator maintenance. In

addition, EDG 2 twice experienced over-voltage trips during the outage. EDG 1 had not

demonstrated this behavior and did not undergo any invasive voltage regulator

maintenance. On the basis of these facts, the licensee determined that EDG 1

operability was not affected by the failure of EDG 2.

The licensee began a rigorous failure modes and effects analysis (FMEA), focusing on

the condition of any electrical component that could affect the control circuit in the

voltage regulator assembly. The FMEA included continuity checks, verification of relay

settings, and an in-situ test of the suspect voltage regulator printed circuit board. The

board passed the in-situ test with no anomalies, and was then replaced with a spare part

obtained from another nuclear utility. After tuning the new circuit board, the licensee

conducted a six hour surveillance run on EDG 2 followed by a full load reject test. No

anomalies were noted during the surveillance testing.

On the basis of the FMEA results and the successful completion of the surveillance test,

the licensee declared EDG 2 operable on January 22, 2007.

4.0 Review of Licensee Troubleshooting Efforts

The team examined vendor manuals, industry maintenance guidelines, and circuit

drawings for the Basler series boost static regulator (SBSR) voltage regulator and

compiled a list of all components, which could fail in a manner as to potentially replicate

the January 18 event. The team reviewed relevant operating experience to identify

additional failure mechanisms experienced at other facilities. The team then compared

the independently generated list to the licensees FMEA to determine whether any

failure modes had not been considered or eliminated inappropriately.

In addition, the team observed a regularly scheduled surveillance test of EDG 1 on

January 29, 2007, conducted in accordance with Procedure 6.1DG.101, Diesel

Generator 31 Day Operability Test. The team observed control room and local

indications during EDG startup and steady state operations, reviewed the data collected

and the completed surveillance package. The team noted no anomalies in the

performance of EDG 1.

The team reviewed the licensees analysis concerning the potential effects that the high

voltage and current conditions could have on affected electrical equipment during the

three EDG 2 over-voltage transients described above. The team reviewed vendor

documentation, procurement records, drawings, relay setpoint calculations and

-8- Enclosure

conducted interviews with design and system engineering staff. The team determined

that the protective devices installed, such as the over-voltage and over current relays,

provided adequate protection for EDG 2 and connected equipment.

During troubleshooting following the failure, a maintenance technician discovered two

loose terminal screws on the back of the EDG 2 OFF-MANUAL-AUTO switch (OMAS).

The licensee had concluded that this was a potential cause of the January 18 failure.

The team reviewed the design of the switch, its function in the circuit, and conducted

several interviews with the technician who discovered the condition. On the basis of the

intermittent nature of the OMAS discontinuity and the prolonged nature of the transient

on January 18 (at least 30 seconds in duration), the team determined that the OMAS

discontinuity was a possible but implausible mode of failure. The team shared this view

with licensee management during a debrief on February 1, 2007.

The team identified several examples of failure modes that were not considered or

eliminated without a technical basis, each of which was brought to the attention of

licensee management during a debrief on February 1, 2007. Following these

discussions the licensee performed additional EDG 2 inspections on February 8, 2007,

and performed eight hour loaded runs on February 8 and February 12, 2007. These

additional inspections and loaded runs did not identify any deficient conditions.

5.0 Offsite Testing of Suspect Voltage Regulator Card

Following the over-voltage event on January 18, 2007, the licensee conducted an in-situ

test of the voltage regulator printed circuit board to identify potential failure mechanisms.

This test was similar to a factory acceptance test in that it powered up the board to

identify hard component failures. No such failures were detected during the in-situ

testing.

The licensee sent the board to a commercial test laboratory where, under vendor

supervision and with input from CNS engineering staff, a series of visual and electrical

tests were conducted to identify failure modes. The laboratory identified that a zener

diode on the printed circuit was in a failed state. CNS demonstrated that this failed

zener diode could have caused the over-voltage event that occurred on January 18,

2007.

The failed circuit board was one of two circuit boards commercially purchased in 1973

that were evaluated by the CNS staff as acceptable for use in safety related applications

without obtaining reasonable assurance that the parts were of sufficient quality to

support safety-related diesel generator functions. After the failure mode was identified,

the team challenged the licensee regarding the treatment of the spare circuit board still

in the warehouse. The licensee subsequently placed the spare circuit board in a

blocked status pending further testing or evaluation of its level of quality.

-9- Enclosure

6.0 Procurement of Spare Voltage Regulator Card

a. Inspection Scope

The team reviewed the quality assurance controls associated with the voltage regulator

printed circuit board that contained the failed zener diode. Specifically, the team

reviewed pertinent procurement documents and quality assurance program guidance to

determine whether the requirements of 10 CFR 50 Appendix B, Quality Assurance

Criteria for Nuclear Power Plants and Fuel Reprocessing Plants, were satisfied.

b. Findings

Introduction. The team identified an apparent violation of 10 CFR Part 50, Appendix B,

Criterion V, Instructions, Procedures, and Drawings, involving the failure to establish

appropriate procedural controls for evaluating the use of parts of indeterminate quality

prior to their installation in safety-related applications. This deficiency resulted in the

installation of a defective voltage regulator circuit board in EDG 2 that failed after 35

hours of operation.

Description. On January 18, 2007, approximately three hours and thirteen minutes into

a loaded surveillance run, EDG 2 experienced an automatic trip because of a voltage

regulator printed circuit board failure. The licensees root cause report, documented in

CR-CNS-2007-00480, determined that the cause of the failure was that the original

procurement process did not provide technical requirements to reduce the probability of

infant mortality failure in the voltage regulator board. The board had been installed in

EDG 2 on November 11, 2006, as corrective maintenance to repair a failure of a

potentiometer on the previously installed circuit board.

The licensee determined that the failed circuit board had been purchased from the

Basler Electric Company in 1973, but that the procurement of the part had not specified

any technical requirements from the vendor. In effect, the part was purchased as a

commercial grade item from a non-Appendix B source and placed into storage as an

essential component, ready for use in safety-related applications, without any

documentation of its suitability for that purpose. The licensee determined that the

specification of proper technical requirements, such as inspections and/or testing, would

have provided an opportunity to discover the latent defect prior to installing the card in

an essential application.

The team noted that in February 1992, the NRC had conducted an inspection at CNS to

review the implementation of Nebraska Public Power Districts (NPPD) programs for the

procurement and dedication of commercial grade items used in safety-related

applications. The results, documented in NRC Inspection Report 05000298/1992-201,

consisted of several identified deficiencies of the CNS quality assurance program.

These deficiencies described that NPPD had been purchasing commercial grade items

and dedicating them based solely on a part number verification, and warehousing these

parts as ready for essential use. The report stated this practice was insufficient to

-10- Enclosure

provide reasonable assurance that the parts were suitable for essential applications, and

as a result, parts of indeterminate quality had been inappropriately labeled as essential.

In response to these deficiencies, CNS made improvements to the existing quality

assurance program and performed a review of all commercial grade procurements

made during the period of the inspection (January 1990 through February 1992). The

team noted these process improvements failed to address that substandard qualification

of essential components potentially occurred since new construction, and that any

essential parts commercially procured prior to 1990, were potentially of indeterminate

quality. This resulted in the licensees failure to establish procedural controls to evaluate

the adequacy of these previously procured commercial grade parts that were deemed

acceptable for use in safety related applications.

The team reviewed the current requirements of Administrative Procedure 0.40.4,

Planning, Revision 2 and Site Services Procedure 1-CNS-MP-115, Material Issues

and Staging, Revision 6. Procedure 0.40.4 establishes the steps taken by work

planners to select the correct quality class part for a given work order, and provides

instructions for planners if the required part safety classification does not match the

available asset. Procedure 1-CNS-MP-115 defines the activities taken by the

warehouse personnel to verify that parts obtained from the warehouse satisfy the

requirements identified by the work planners. These program requirements included

checks for shelf life, post work testing, and other appropriate barriers. However, as

previously discussed, neither procedure evaluated the use of parts that were

inadequately procured for use in safety related applications prior to 1990. On the basis

of the deficiencies in the CNS procurement program previously identified by the NRC,

and a review of site quality procedures, the team determined that the licensee had failed

to establish appropriate procedural controls for evaluating the use of parts of

indeterminate quality prior to their installation in safety-related applications.

The team noted that if the licensee had implemented administrative controls to review

the suitability of pre-1990 procurement items prior to their use in safety-related

applications, then the licensee could have had the opportunity to discover the

inadequate measures taken to qualify the voltage regulator circuit boards for essential

applications. Given this information, the licensee could have either procured a new

board from an essential source, or dedicated the old part using Engineering Procedure

3-CNS-DC-138.2, Dedication. Procedure 3-CNS-138.2 directs the user to Engineering

Procedure 3-CNS-DC-138, Technical Evaluation Process, for the identification of

critical characteristics important to provide reasonable assurance that a part is ready for

essential use. Attachment 3 to Procedure 3-CNS-DC-138 provides examples of

appropriate critical characteristics, one of which is a burn-in endurance test. The team

reviewed industry standards for such tests, including MIL-STD-750D, Department of

Defense Test Method Standard for Semiconductor Devices, which recommends a

96 hour0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> burn in test. Given that the voltage regulator card failed after approximately 35

hours of service, the team concluded that a burn-in or other equivalent test would have

given CNS the opportunity to discover the latent defect. To better understand current

CNS standards for acceptance of safety-related electrical components, the inspectors

reviewed Change Evaluation Document (CED) 6017822, that is being implemented to

install new circuit cards in the Average Power Range Monitor system. Purchase

-11- Enclosure

Order 4500055181 procured the new circuit cards and imposed a burn-in test prior to

shipment. The team also noted that the non-safety related electrical components for

CEDs 6010820 and 6016542, being implemented to install new reactor water level and

feedwater control systems, have been energized on site for testing for over one year in

an attempt to complete logic verification and identify latent defects.

The team noted that the failure to evaluate these parts of indeterminate quality resulted

in the failure to identify the latent defect in the voltage regulator circuit board in EDG 2.

Specifically, this deficiency resulted in installing a voltage regulator circuit board of

indeterminate quality in EDG 2 on November 11, 2006, that experienced an infant

mortality failure after 35 hours4.050926e-4 days <br />0.00972 hours <br />5.787037e-5 weeks <br />1.33175e-5 months <br /> of operation on January 18, 2007.

Analysis: This finding is considered to be a performance deficiency because the

licensee failed to establish appropriate procedural guidance to ensure that commercially

procured components are of sufficient quality prior to their installation in safety related

applications. This finding is more than minor because it is associated with the

equipment performance attribute of the Mitigating Systems cornerstone and adversely

affects the cornerstone objective of ensuring the availability, reliability, and capability of

systems that respond to initiating events.

The team evaluated the issue using the Significance Determination Process (SDP)

Phase 1 Screening Worksheet provided in Manual Chapter 0609, Appendix A,

"Significance Determination of Reactor Inspection Findings for At-Power Situations."

The screening indicated that a Phase 2 analysis was required because the finding

represents a loss of safety function for EDG 2 for greater than its Technical

Specification allowed completion time. The Phase 2 and 3 evaluations preliminarily

concluded that the finding was of low to moderate safety significance (See Attachment 3

for details).

Enforcement: 10 CFR Part 50, Appendix B, Criterion V, Instructions, Procedures, and

Drawings, requires, in part, that activities affecting quality shall be prescribed by

procedures of a type appropriate to the circumstances and shall include appropriate

acceptance criteria for determining that important activities have been satisfactorily

completed. Contrary to this, the licensee failed to establish appropriate procedural

controls for evaluating the use of parts of indeterminate quality prior to their installation

in safety-related applications. This deficiency resulted in the installation of a voltage

regulator circuit board of indeterminate quality in EDG 2 that prematurely failed after 35

hours of operation. This issue was entered into the licensee's corrective action program

as CR-CNS-2007-00480. Pending determination of the findings final safety

significance, this finding is identified as Apparent Violation (AV) 05000298/2007007-

001, "Inadequate Procedures Result in Failure of Emergency Diesel Generator Voltage

Regulator."

-12- Enclosure

7.0 Review of EDG 2 Maintenance Procedures

a. Inspection Scope

The team assessed the maintenance procedures and work orders used to perform

preventive and corrective maintenance activities on EDG 2 between 2005 and the failure

on January 18, 2007. In addition to reviewing the written procedures, the team

interviewed procedure writers and maintenance technicians to determine if the work

instructions provided were adequate to achieve their intended objectives.

b. Findings

Introduction: The team identified three examples of a Green noncited violation (NCV) of

TS 5.4.1.a regarding inadequate maintenance procedures associated with EDG 2.

Description: Maintenance Procedure (MP) 7.3.8.2, Diesel Generator Electrical

Examination and Maintenance, is performed by the licensee approximately every

eighteen months and is intended to perform inspections and preventive maintenance on

engine and generator electrical components. This procedure was conducted on EDG 2

on March 14, 2006, and November 8, 2006. MP 7.3.8.2, Revision 19, contained a

specific section on voltage regulator maintenance that included instructions to check

lugs and screw terminals for tightness and integrity. The team noted that the scope of

the voltage regulator checks was limited to only those components within the voltage

regulator cabinet. This deficiency prevented maintenance technicians from identifying

that loose terminal screws existed on the voltage regulator OFF-MANUAL-AUTO switch

(OMAS) because it is physically located on the EDG 2 metering panel. These loose

terminal connections were discovered during troubleshooting efforts following the

January 18, 2007, failure of EDG 2. The licensee demonstrated that these loose

terminations represented a degraded condition that could cause an over-voltage event

similar in nature to that experienced on January 18, 2007. MP 7.3.8.2 was inadequate

in that it did not contain sufficient procedural guidance to identify the degraded

conditions in voltage regulator system components outside the voltage regulator

cabinet. The licensee documented this performance deficiency in CR-CNS-2007-00794.

In the second example, WO 4514076 was planned for the RE23 refueling outage as an

assigned corrective action from CR-CNS-2006-2729. The intent of the work order was

to perform a thorough inspection of EDG 2 voltage regulator system components to

identify the cause of continued voltage and reactive load perturbations experienced

during surveillance testing. WO 4514076 contained specific instructions for checking

the integrity of electrical connections in the EDG 2 voltage regulator cabinet and

metering panel. On the basis of interviews conducted with the system engineer who

wrote the work instructions, the team learned the engineer had intended for the

maintenance personnel to specifically check the terminal screws on the OMAS switch

for tightness. The guidance to perform this check was contained in a note at the front of

the procedure and not in the individual work order step for the switch. As a result, the

OMAS switch connections were not checked for tightness, resulting in another missed

opportunity to discover the loose terminal connections on the switch. The licensee

documented this performance deficiency in CR-CNS-2007-01021.

-13- Enclosure

In the third example, WO 4514076 contained inadequate instructions for tuning the EDG

2 voltage regulator during the RE23 refueling outage. On November 11, 2006, the

printed circuit board in the EDG 2 voltage regulator was replaced to correct a degraded

potentiometer. WO 4514076 contained instructions for tuning the new voltage regulator

card following installation, but was inadequate in that it contained acceptance criteria

that were inappropriate for the voltage regulators installed at CNS. The instructions

directed maintenance personnel to adjust the R13 potentiometer to obtain quarter wave

dampening in the EDG output voltage response. The technicians noted that adjusting

R13 did not change the amplitude of the sinusoidal response, and determined that the

acceptance criteria in WO 4514076 could not be satisfied. Technicians then set the

R13 potentiometer resistance on the new card to the same value as found on the old

card. During subsequent measurements of the voltage regulator response, the

technicians noted that the time required for the output voltage to oscillate through one

complete cycle had increased from 3.1 to 3.8 seconds. The technicians accepted this

new response characteristic without any engineering evaluation, procedural guidance, or

vendor technical reference demonstrating its acceptability. A subsequent evaluation

performed by engineering demonstrated that this change did not interfere with the safety

function of the EDG. Additionally, EDG 2 experienced an over-voltage trip on November

13, 2006, during the tuning process. In condition report CR-CNS-2006-9096, the

licensee documented the apparent cause of the EDG trip as the erratic behavior of one

or both of the potentiometers on the voltage regulator card and went on to explain that

industry operating experience and vendors both recognize that this over-voltage trip

could have been caused by the tuning process. The corrective actions proposed in the

apparent cause report included improvements to the EDG voltage regulator tuning

process and additional training for maintenance personnel performing the activity. The

licensee documented this procedural inadequacy in CR-CNS-2007-1307.

Analysis: The performance deficiency associated with this finding involved the

licensees failure to provide adequate instructions for performing maintenance on

EDG 2. The finding is more than minor because it is associated with the Mitigating

Systems cornerstone attribute of procedure quality and affects the associated

cornerstone objective to ensure the availability, reliability, and capability of systems that

respond to initiating events. Specifically, the performance deficiency resulted in: (1) the

failure to discover a degraded condition in the EDG 2 voltage regulator, and (2) an over-

voltage trip during the tuning of EDG 2 on November 13, 2006. Using the Manual

Chapter 0609 Appendix G, "Shutdown Operations Significance Determination Process,"

Phase 1 Checklist, the finding is determined to have very low safety significance

because one operable diesel generator was still capable of supplying power to the class

1E electrical power distribution subsystems.

This finding has a cross-cutting aspect in the area of human performance in that the

licensees procedures were not complete and provided inadequate instructions for

persons conducting maintenance on safety related equipment.

Enforcement. Technical Specification 5.4.1.a requires that written procedures be

established, implemented, and maintained, covering the activities specified in

Regulatory Guide 1.33, Revision 2, Appendix A, dated February 1978. Regulatory

Guide 1.33, Appendix A, Section 9 (a), requires that maintenance affecting the

-14- Enclosure

performance of safety-related equipment should be performed in accordance with

written procedures. Contrary to this, Maintenance Procedure 7.3.8.2 and Work Order 4514076 did not contain adequate instructions to identify the degraded condition of the

EDG 2 OFF-MANUAL-AUTO switch. In addition, Work Order 4514076 did not contain

adequate instructions for the tuning of EDG 2 following the replacement of the voltage

regulator printed circuit board, resulting in an over-voltage trip of EDG 2 on November

13, 2006. Because the finding is of very low safety significance and has been entered

into the licensees corrective action program in Condition Reports CR-CNS-2007-00794,

CR-CNS-2007-01021, and CR-CNS-2007-01307, this violation is being treated as an

NCV consistent with Section VI.A of the Enforcement Policy: NCV 05000298/2007007-

002, "Inadequate Procedures for Conducting Maintenance on Emergency Diesel

Generator 2.

8.0 Long-Term Corrective Actions

The team interviewed system and design engineering personnel to ascertain the long

term plans the licensee intends to implement to improve EDG electrical performance.

The licensee has already implemented measures to improve their ability to monitor EDG

performance during surveillance testing through the installation of special test

equipment. Future plans include installation of permanent external test connections to

minimize the unavailability time required to hook up the test equipment for each

scheduled surveillance.

The licensee has initiated actions to establish technical requirements for burn-in or other

equivalent testing for safety-related DG system circuit boards. Additionally, the licensee

plans to define other safety related systems with circuit boards that need similar

treatment.

The licensee also plans to implement several modifications to improve EDG reliability in

upcoming refueling outages. The changes include installation of a digital MOP during

the next refueling outage and replacement of the entire voltage regulator system with a

digital system in the subsequent outage.

9.0 Potential Generic Issues

The team noted that CNS submitted an operating experience report to alert the industry

of the potential failure of zener diodes in Basler SBSR voltage regulators. The team did

not identify any potentially generic issues during the inspection.

-15- Enclosure

4OA6 Meetings, Including Exit

On February 1, 2007, the preliminary results of this inspection were presented to

Mr. M. Colomb and other members of his staff who acknowledged the findings.

Following additional in-office reviews, the final results of the inspection were presented

to Mr. Colomb and his staff on April 24, 2007. The team confirmed that the supporting

details in this report contained no proprietary information.

ATTACHMENT 1: SUPPLEMENTAL INFORMATION

ATTACHMENT 2: SPECIAL INSPECTION CHARTER

ATTACHMENT 3: SIGNIFICANCE DETERMINATION EVALUATION

-16- Enclosure

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee Personnel

M. Bergmeier, Operations Support Group Supervisor

D. Buman, System Engineering Manager

K. Cohn, Engineering Support

M. Dickerson, Design Engineering

J. Flaherty, Licensing Engineer

P. Fleming, Licensing Manager

C. Gaedeke, Maintenance

T. Hottovy, Equipment Reliability Manager

J. Larson, Quality Assurance Supplier Supervisor

M. McCormack, Electrical Systems/I&C Engineering Supervisor

E. McCutchen, Regulatory Affairs Senior Licensing Engineer

M. Metzger, System Engineer

B. Morris, Maintenance Support Superintendent

R. Noon, Root Cause Team Leader, Corrective Action & Assessments

S. Norris, Assistant Operations Manager

R. Rexroad, System Engineering

K. Sutton, Risk Management Supervisor

D. Willis, Operations Manager

NRC Personnel

S. Graves, Reactor Inspector

M. Haire, Enforcement Specialist

R. McIntyre, Quality & Vendor Branch A

R. Pettis, Quality & Vendor Branch A

P. Prescott, Quality & Vendor Branch A

S. Rutenkroger, PHD, Reactor Inspector

S. Schwind, Senior Resident Inspector

N. Taylor, Resident Inspector

D. Thatcher, Chief, Quality & Vendor Branch A

A1-1 Attachment 1

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

05000298/2007007-01 AV Inadequate Procedures Result in Failure of Emergency

Diesel Generator Voltage Regulator

Opened and Closed

05000298/2007007-02 NCV Inadequate Procedures for Conducting Maintenance on

Emergency Diesel Generator 2

LIST OF ACRONYMS

AV Apparent Violation

CFR Code of Federal Regulations

CNS Cooper Nuclear Station

CR condition report

EDG emergency diesel generator

FMEA failure modes and effects analysis

KVAR kilovolt-ampere-reactive

MOP motor operated potentiometer

MP maintenance procedure

NCV noncited violation

NPPD Nebraska Public Power District

NRC U.S. Nuclear Regulatory Commission

OMAS off-manual-auto switch

PM preventive maintenance

RE23 refueling outage 23

SBSR series boost static regulator

SDP significance determination process

WO work order

LIST OF DOCUMENTS REVIEWED

Procedures Reviewed:

Administrative Procedure 0.40.4, Planning, Revision 2

Site Service Procedure 1-CNS-MP-15, Material Issues and Staging, Revision 6

Engineering Procedure 3-CNS-DC-138, Technical Evaluation Process, Revision 1

Engineering Procedure 3-CNS-DC-138.2, Dedication, Revision 1

A1-2 Attachment 1

Quality Assurance Instruction QAI-9, Guidelines for Establishing Quality Classifications of

Components and Materials, Draft 10/3/73

Condition Reports:

CR-CNS-2005-00938, CR-CNS-2005-07806, CR-CNS-2005-08336, CR-CNS-2006-02091, CR-

CNS-2006-02140, CR-CNS-2006-00279, CR-CNS-2006-02963, CR-CNS-2006-05149, CR-

CNS-2006-08798, CR-CNS-2006-08999,CR-CNS-2006-09096, CR-CNS-2006-09301, CR-

CNS-2007-00480

Work Orders:

4338439, 4424754, 4458094, 4472125, 4499755, 4514076, 4536182, 4535878, 4536371,

4548656, 4548698, 4548841, 4548860, 4551090

Controlled Drawings:

Basler Electric Company drawing 9032100910, Revision N01

Basler Electric Company drawing 9072400910, Revision N03

14EK-0144, Rev. N17

BR 3012, Sheet #3, Rev. N17

BR 3012, Sheet #6, Rev. N15

0223R0558, Sheet #33, Rev. N22

14DK0921, Rev. N01

BR 3257, Sheet 48H, Rev. N01

BR 3251, Sheet 11, Rev. N17

G5-262-743, Sheet 10A, Rev. N03

Miscellaneous Documents:

Purchase Order 73440, September 21, 1973

Nonconformance Report 002, October 27, 1973

Purchase Order 75149, November 15, 1973

Certificate of Compliance for Purchase Order 75149, December 3, 1973

CNS Vendor Manual VM-0246 [Basler Type SBSR HV Series Boost Exciter-Regulator]

A1-3 Attachment 1

Instruction Manual SM-100, Synchronous Motors, Generators, D.C. Exciters & Brushless

Equipment, Ideal Electric (no date)

Memo from Ideal Electric and Manufacturing Company to Cooper Bessemer Company, dated

8-28-70 (provided specific ratings for CNS generators)

CNS Design Criteria Document, DCD-1, Diesel Generators

EPRI Technical Report 1011110, Basler SBSR Voltage Regulators for Emergency Diesel

Generators, Final Report, November 2004

NUREG/CR-6819, Vol. 1, Common-Cause Failure Event Insights, Emergency Diesel

Generators

NRC Information Notice 96-23, Fires in Emergency Diesel Generator Exciters During

Operation Following Undetected Fuse Blowing, April 22, 1996

IEEE Standard 336-1971, Installation, Inspection and Testing Requirements for

Instrumentation and Electric Equipment During The Construction of Nuclear Power Generating

Stations

A1-4 Attachment 1

SPECIAL INSPECTION CHARTER

January 25, 2007

MEMORANDUM TO: Nicholas H. Taylor, Resident Inspector, Cooper Nuclear Station

Project Branch C, Division of Reactor Projects

Dr. Scott P. Rutenkroger, Reactor Inspector

Engineering Branch 1, Division of Reactor Safety

FROM: Arthur T. Howell III, Director, Division of Reactor Projects /RA/ AVegel for

SUBJECT: SPECIAL INSPECTION CHARTER TO EVALUATE THE COOPER

NUCLEAR STATION EMERGENCY DIESEL GENERATOR FAILURE

A Special Inspection Team is being chartered in response to the Cooper Nuclear Station

emergency diesel generator (EDG) failure. The EDG failed during surveillance testing on

January 18, 2007. You are hereby designated as the Special Inspection Team members.

Mr. Taylor is designated as the team leader. The assigned senior reactor analyst (SRA) to

support the team is Mike Runyan.

A. Basis

On January 18, 2007, during performance of a monthly surveillance test, Emergency

Diesel Generator 2 automatically isolated from the electrical bus following an over-

current condition. The licensee determined this condition resulted from a high voltage

condition. The licensee has preliminarily identified the cause of the failure to be either:

(1) a loose electrical connection affecting the voltage regulator circuit, or (2) a latent

failure of the voltage regulator printed circuit board. The licensee has experienced

previous voltage regulator problems, resulting in replacement of EDG 2 voltage

regulator components during the last refueling outage (RE23). The most recent failure

of EDG 2, and previous licensee efforts to identify and correct EDG 2 voltage regulator

problems, draws into question the effectiveness of the licensees corrective actions.

Additionally, prior to the failure on January 18, 2007, EDG voltage regulator

troubleshooting and postmaintenance activities have resulted in additional automatic trips due to high voltage conditions.

This Special Inspection Team is chartered to review the circumstances related to

historical and present EDG 2 voltage regulator problems and assess the effectiveness

of the licensees actions for resolving these problems. The team will also assess the

effectiveness of the immediate actions taken by the licensee in response to the EDG 2

failure that occurred on January, 18, 2007.

A2-1 Attachment 2

B. Scope

The team is expected to address the following:

1. Develop an understanding of the EDG degraded conditions and failures related

to voltage regulator problems.

2. Assess licensee effectiveness in identifying previous EDG voltage regulator

problems, evaluating the cause of these problems, and implementation of

corrective actions to resolve identified problems.

3. Identify and assess additional actions planned by the licensee in response to the

declining performance of the EDG 2 voltage regulator, including the timeline for

completion of these actions.

4. Assess the licensees root cause evaluation, the extent of condition, and the

licensees common mode evaluation.

5. Evaluate pertinent industry operating experience and potential precursors to the

January 18 event, including the effectiveness of licensee actions taken in

response to the operating experience.

6. Determine if there are any potential generic issues related to the failure of the

EDG 2 voltage regulator. Promptly communicate any potential generic issues to

Region IV management.

7. Determine if the Technical Specifications were met when the EDG failed.

8. Collect data as necessary to support a risk analysis.

C. Guidance

Inspection Procedure 93812, Special Inspection, provides additional guidance to be

used by the Special Inspection Team. Your duties will be as described in Inspection

Procedure 93812. The inspection should emphasize fact-finding in its review of the

circumstances surrounding the event. It is not the responsibility of the team to examine

the regulatory process. Safety concerns identified that are not directly related to the

event should be reported to the Region IV office for appropriate action.

The Team will report to the site, conduct an entrance, and begin inspection no later than

January 28, 2007. While on site, you will provide daily status briefings to Region IV

management, who will coordinate with the Office of Nuclear Reactor Regulation, to

ensure that all other parties are kept informed. A report documenting the results of the

inspection should be issued within 30 days of the completion of the inspection.

This Charter may be modified should the team develop significant new information that

warrants review. Should you have any questions concerning this Charter, contact me at

(817) 860-8144.

A2-2 Attachment 2

SIGNIFICANCE DETERMINATION EVALUATION

Significance determination process Phase 1:

In accordance with NRC Inspection Manual Chapter 0609, Appendix A, Significance

Determination of Reactor Inspection Findings for At-Power Situations, the inspectors

conducted a significance determination Phase 1 screening and determined that the

finding resulted in loss of the safety function of Emergency Diesel Generator 2 for

greater than the Technical Specification allowed completion time. Therefore, a

Significance Determination Process Phase 2 evaluation was required.

Significance determination process Phase 2:

In accordance with Manual Chapter 0609, Appendix A, Attachment 1, User Guidance

for Determining the Significance of Reactor Inspection Findings for At-Power

Situations, the inspectors evaluated the EDG 2 failure using the Risk-Informed

Inspection Notebook for Cooper Nuclear Station, Revision 2.

Assumptions

1. The Phase 2 analysis assumed that the EDG 2 was unable to perform its

function beginning on November 16, 2006. This date assumes that the failure of

the voltage regulator was a run-time degradation (consistent with the licensees

root cause) and recognizes that in the licensees risk model EDG 2 must be

capable of providing its safety function for 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> following an accident. This

resulted in an applied exposure time of 64 days.

2. Recovery credit was not appropriate due to the lack of a useable procedure or

operator training in operating the EDG 2 voltage regulator in the manual mode

during a loss of offsite power.

3. The loss of EDG 2 affects the full mitigation credit for other safety functions on

the Loss of Offsite Power worksheet (reference special useage rule 1.6).

4. Phase 2 Analysis Results: The Phase 2 analysis indicated that the significance

of the finding was potentially Greater than Green. The dominant accident

sequence involved a loss of offsite power with recovery within the 8 hour9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> life of

the CNS batteries, and a loss of offsite power without high pressure injection and

recovery within one hour. Completion of all applicable sequences in the Loss of

Offsite Power worksheet resulted in one sequence with a score of 8, three

sequences with a score of 7 and one sequence with a score of 6. Based on this

result the issue screened as White in the SDP Phase II analysis.

A3-1 Attachment 3

Significance determination process Phase 3:

The analyst estimated the risk increase resulting from the EDG 2 voltage regulator

failure. The diesel was run at the following times with durations reported as the period

of time that the voltage regulator was energized (all of these operational runs were

conducted after the performance deficiency occurred):

Date of EDG Run Duration of EDG Run

November 13, 2006 1 hr 30 min

November 14, 2006 6 hrs 46 min

November 15, 2006 1 hr 35 min

November 16, 2006 9 hrs 23 min

November 17, 2006 5 hrs 3 min

November 18, 2006 2 hrs 28 min

December 12, 2006 5 hrs 41 min

January 18, 2007 4 hrs 16 min (point of failure)

Assumptions:

1. It is assumed that the voltage regulator degraded only during times that it was

energized, which is closely correlated to engine runtime. This implies that no

degradation occurred while the EDG was secured and in a standby status. It is

further assumed that the failure was a deterministic outcome set to occur after a

specific number of operating hours. Therefore, it is assumed that EDG 2 would

have failed to run at four hours following a loss of offsite power (LOOP) demand

at any time during the 37 day period from its last successful surveillance test on

December 12, 2006, until the test failure that occurred on January 18, 2007.

Prior to this date, EDG 2 would have run and failed at 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> for the 24 day

period from November 18, 2006, to December 12, 2006. The EDG was run for

approximately 17 hours1.967593e-4 days <br />0.00472 hours <br />2.810847e-5 weeks <br />6.4685e-6 months <br /> in the two days preceding November 18, 2006. Because

very little exposure existed prior to November 18, 2006, this date is chosen as

the cutoff for this analysis.

2. The voltage regulator could not have been repaired in place or replaced with a

new unit in time to affect the outcome of any of the core damage sequences.

Also, procedures did not exist and training was not conducted to operate the

EDG in a manual voltage regulation mode. Therefore, it is assumed that the

EDG 2 voltage regulator failure would not have been recoverable for any

accident sequence.

A3-2 Attachment 3

3. For the purpose of this analysis, it is assumed that EDG 2 would not be

unavailable or fail to operate for the first four hours or 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> of its hypothetical

start demand during the 37 day and 24 day exposure periods, respectively. This

introduces a slight inconsistency to the risk estimate, but because it would

similarly affect both the base and current case, it does not significantly influence

the result of this analysis.

4. Common cause vulnerabilities for EDG 1 did not exist, that is, the failure mode is

assumed to be independent in nature. This is because the root cause

investigation determined that the failure was the result of an infant mortality of a

voltage regulator conponent, which had been installed for only two months. The

same component in EDG1 had been installed for several years and had

operated reliably beyond the "burn-in" period without experiencing failure from

manufacturing defects.

The CNS SPAR model, Revision 3.31, dated October 10, 2006, was used in the

analysis. A cutset truncation of 1.0E-12 was used. Average test and maintenance was

assumed.

To represent the assumed failure of EDG 2, the basic event EPS-DGN-FR-DG1B

(diesel generator 1B fails to run) was set to one. A flag set house event for DG1B out

of service was added to the EDG1B fault tree and set to FALSE in the base case, and

TRUE in the current case in order to remove non-minimal cutsets. Also, the common

cause probability for fail-to-run events was restored to its nominal value.

Internal Events Analysis:

A. Risk Estimate for the 37-day period between December 12, 2006 and

January 18, 2007:

During this exposure period, EDG 2 is assumed to have been capable of running for

four hours. The LOOP frequency used in the analysis was adjusted to reflect the

situation that only LOOPs with durations greater than four hours would result in a risk

increase attributable to the voltage regulator failure. The base LOOP frequency is

3.59E-2/yr. The 4-hour non-recovery of offsite power is 0.1566. The 4-hour non-

recovery of diesel generators is 0.4835. To account for having only one EDG to recover

during the first four hours (since recovery of EDG 2 is assumed to be running during the

first four hours of the event), the EDG non-recovery factor was adjusted to the square

root of the base non-recovery factor. This adjusts the recovery from a one out of two

EDG recovery to a one out of one recovery. This factor is (0.4835)1/2 = 0.695. Therefore

the adjusted LOOP frequency, representing the frequency of LOOPs that are not

recovered in four hours by either restoring offsite power or recovering a failure of EDG 1

is 3.59E-2(0.1566)(0.695) = 3.91E-3/yr. For the base case, the adjusted LOOP

frequency considers that both EDGs are hypothetically recoverable. Therefore the base

case LOOP frequency is 3.59E-2(0.1566)(0.4835) = 2.72E-3/yr.

A3-3 Attachment 3

Resetting event time t=0 to four hours following the LOOP event requires that the

recovery factors for offsite power and the EDGs be adjusted. For example, in two hour

sequences in SPAR, the basic event for non-recovery of offsite power should be

adjusted to the non-recovery at 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />, given that recovery has failed at four hours.

An adjustment to account for the diminishment of decay heat must be considered. This

is because the magnitude of decay heat at four hours following shutdown is less than in

the early moments following a reactor trip, and the timing of core damage sequences is

affected by this fact. In the SPAR model, recovery times for either offsite power, EDGs,

or both are set at the intervals of 30 minutes, two hours, four hours, eight hours, and ten

hours. The analyst determined that the average decay heat level in the first 30 minutes

is approximately two times the average level that exists between four and five hours

following shutdown. Therefore, baseline 30-minute SPAR model sequences, that

essentially account for boil-off to fuel uncovery were adjusted to one hour sequences.

The two hour sequences model safety relief valve failures to close, and are based more

on inventory control than core heat production. Therefore, no adjustment was made for

these sequences. The analyst determined that decay heat rates leveled out quickly

following shutdown and could find no basis for adjusting the times associated with the

four, eight, and ten hour sequences.

The following table presents the adjusted offsite power non-recovery factors for the

event times that are relevant in the SPAR core damage cutsets:

SPAR SPAR base SPAR base SPAR base Modified

recovery offsite power offsite power offsite power SPAR

time non-recovery non-recovery at non-recovery at recovery

4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> + SPAR (Column 4

recovery time in divided by

Column 1 Column 3)

30 min. 0.7314 0.1566 0.12051 0.769

2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> 0.3181 0.1566 0.09637 0.615

4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> 0.1566 0.1566 0.06718 0.429

8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> 0.06718 0.1566 0.04040 0.258

10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> 0.05070 0.1566 0.03346 0.214

1. A SPAR recovery time of 1.0 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> is used, as discussed above, to account for the lessening of decay heat.

The following table presents the analogous non-recovery factor adjustments for EDG 1

recovery times for the current case (it is assumed that EDG 2 is not recoverable):

A3-4 Attachment 3

SPAR SPAR base non- SPAR base SPAR base Modified

recovery recovery for two EDG non- EDG non- SPAR

time EDGs recovery at 4 recovery at 4 recovery

hours for 1 EDG hours + SPAR (Column 4

(square root of recovery time in divided by

0.4835) Column 1 Column 3)

(square root)

30 min. 0.8570 0.695 0.6511 0.937

2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> 0.6482 0.695 0.612 0.881

4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> 0.4835 0.695 0.544 0.783

8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> 0.2959 0.695 0.439 0.632

10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> 0.2374 0.695 0.397 0.571

1. A SPAR recovery time of 1.0 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> is used, as discussed above, to account for the lessening of decay heat.

The following table presents the EDG non-recoveries used for the base case (both

EDGs are assumed available for recovery in the base case):

SPAR SPAR base non- SPAR base SPAR base Modified

recovery recovery for two EDG non- EDG non- SPAR

time EDGs recovery at 4 recovery at 4 recovery

hours hours + SPAR (Column 4

recovery time in divided by

Column 1 Column 3)

30 min. 0.8570 .4835 0.42401 0.877

2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> 0.6482 .4835 0.3742 0.774

4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> 0.4835 .4835 0.2959 0.612

8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> 0.2959 .4835 0.1926 0.398

10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> 0.2374 .4835 0.1576 0.326

1. A SPAR recovery time of 1.0 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> is used, as discussed above, to account for the lessening of decay heat.

The SPAR base case was updated to reflect the new LOOP frequency and non-

recovery times for offsite power and EDGs (column 5 figures).

The SPAR base case update result, after applying the applicable revised LOOP

frequency and offsite power and EDG recovery figures, was 1.039E-5/yr. The current

case result, with the EDG 2 fail-to-run set to one and the flag set house event set to

TRUE, and the changed recoveries inserted for offsite power and the EDGs was

5.373E-5/yr.

A3-5 Attachment 3

Therefore, the estimated ICCDP of the 37-day period during which EDG 2 was assumed

to be in a condition that guaranteed its failure at four hours is (5.373E-5/yr. - 1.039E-

5/yr.) (37 days/365 days/yr.) = 4.4E-6/yr.

B. Risk Estimate for the 24-day period between November 18, 2006 and

December 12, 2006:

During this exposure period, EDG 2 is assumed to have been capable of running for 10

hours. The LOOP frequency used in the analysis was adjusted to reflect the situation

that only LOOPs with durations greater than 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> would result in a risk increase

attributable to the voltage regulator failure. The base LOOP frequency is 3.59E-2/yr.

The 10-hour non-recovery of offsite power is 5.070E-2. The 10 hour1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> non-recovery of

diesel generators is 0.2374. To account for having only one EDG to recover during the

first 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> (since recovery in this analysis only applies to the postulated failure of

EDG 1), the EDG non-recovery factor was adjusted to the square root of the base non-

recovery factor. This adjusts the recovery from a one out of two EDG recovery to a one

out of one recovery. This factor is (0.2374)1/2 = 0.487. Therefore, the adjusted LOOP

frequency, representing the frequency of LOOPs that are not recovered in ten hours by

either restoring offsite power or recovering a failure of EDG 1, is 3.59E-2(5.070E-

2)(0.487) = 8.86E-4/yr. For the base case, the adjusted LOOP frequency considers that

both EDGs are hypothetically recoverable. Therefore, the base case LOOP frequency

is 3.59E-2(5.070E-2)(0.2374) = 4.32E-4/yr.

The analyst considered an adjustment to account for the diminishment of decay heat as

in the four hour case above. The analyst determined that the average decay heat level

in the first 30 minutes is approximately three times the average level that exists between

10 and 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> following shutdown. Therefore, the baseline 30 minute SPAR models,

that essentially account for boil-off to fuel uncovery were adjusted to 1.5 hour5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />

sequences. The two hour sequences model safety relief valve failures to close, and are

based more on inventory control than core heat production. Therefore, no adjustment

was made for these sequences. Sequences of four and eight hours were increased by

30 minutes each, but no change was made to the 10 hour1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> sequences.

The following table presents the adjusted offsite power non-recovery factors for the

event times that are relevant in the SPAR core damage cutsets:

A3-6 Attachment 3

SPAR SPAR base SPAR base SPAR base Modified

recovery offsite power offsite power offsite power SPAR

time non-recovery non-recovery at non-recovery at recovery

10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> + (Column 4

SPAR recovery divided by

time in Column Column 3)

1

30 min. 0.7314 0.0507 0.04271 0.842

2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> 0.3181 0.0507 0.0404 0.797

4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> 0.1566 0.0507 0.03212 0.633

8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> 0.06718 0.0507 0.02412 0.475

10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> 0.05070 0.0507 0.0220 0.434

1. A SPAR recovery time of 1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> is added to 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br />, as discussed above, to account for the falloff of decay heat.

2. The SPAR recovery time is increased by 30 minutes, as discussed above.

The following table presents the analogous non-recovery factor adjustments for EDG 1

recovery times:

SPAR SPAR base non- SPAR base SPAR base Modified

recovery recovery for two EDG non- EDG non- SPAR

time EDGs recovery at 10 recovery at 10 recovery

hours for 1 EDG hours + SPAR (Column 4

(square root of recovery time in divided by

0.2374) Column 1 Column 3)

(square root)

30 min. 0.8570 0.4872 0.4511 0.926

2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> 0.6482 0.4872 0.439 0.901

4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> 0.4835 0.4872 0.3882 0.796

8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> 0.2959 0.4872 0.3212 0.659

10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> 0.2374 0.4872 0.300 0.616

1. A SPAR recovery time of 1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> is added to 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br />, as discussed above, to account for the falloff of decay heat.

2. The SPAR recovery time is increased by 30 minutes, as discussed above.

The following table presents the EDG recoveries used for the base case (both EDGs are

assumed available for recovery in the base case):

A3-7 Attachment 3

SPAR SPAR base non- SPAR base SPAR base Modified

recovery recovery for two EDG non- EDG non- SPAR

time EDGs recovery at 10 recovery at 10 recovery

hours for 1 EDG hours + SPAR (Column 4

recovery time in divided by

Column 1 Column 3)

30 min. 0.8570 0.2374 0.20301 0.855

2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> 0.6482 0.2374 0.1926 0.811

4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> 0.4835 0.2374 0.15022 0.633

8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> 0.2959 0.2374 0.10302 0.434

10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> 0.2374 0.2374 0.0898 0.378

1. A SPAR recovery time of 1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> is added to 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br />, as discussed above, to account for the falloff of decay heat.

2. The SPAR recovery time is increased by 30 minutes, as discussed above.

The SPAR base case was updated to reflect the new LOOP frequency and non-

recovery times for offsite power and EDGs (column 5 figures).

The SPAR base case update result, after applying the applicable revised LOOP

frequency and offsite power and EDG recovery figures, was 1.008E-5/yr. The current

case result, with the EDG 2 fail-to-run set to one and the flag set house event set to

TRUE and the changed recoveries inserted for offsite power and the EDGs, was

2.830E-5/yr.

Therefore, the estimated ICCDP of the 24-day period during which EDG 2 was assumed

to be in a condition that guaranteed its failure at 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br />, is (2.830E-5/yr. - 1.008E-5/yr.)

(24 days/365 days/yr.) = 1.2E-6/yr.

Total Internal Events Result:

Exposure Period

37-Day (12/12/06 - 01/18/07) 4.4E-6/yr.

24-Day (11/18/06 - 12/12/06) 1.2E-6/yr.

Total Internal Events Result 5.6E-6/yr.

Sensitivity of EDG 2 Recovery:

In the analysis presented above, it was assumed that EDG 2 could not be recovered in

time to lower the risk of the relevant core damage sequences. This was because the

failed voltage regulator could not be repaired or replaced quickly and operation of the

emergency diesel generator in a manual voltage regulation mode was not a subject of

operator training and not explicitly expressed in plant procedures. As a sensitivity to this

assumption, a low bounding (highest allowance for recovery) case for operating the

A3-8 Attachment 3

EDG in manual voltage regulation mode was considered using the SPAR-H

methodology. The results of this analysis are presented in the table below.

Performance Diagnosis (0.01) Action (0.001)

Shaping Factor

Available Time Extra Time (0.1) >5 Times Required (0.1)

Stress High (2) High (2)

Procedures Incomplete (20) Incomplete (20)

Experience/Training Low (10) Low (3)

Total1 0.288 0.012

Overall Total HRA 0.3

1. This reflects the result using the formula for cases where 3 or more negative PSFs are present.

The nominal time for performing the actions was small compared to the minimum time

of four hours available to restore power following failure of EDG 2 at four hours into the

event. The Class 1E batteries are capable of supplying eight hours of power. For the

core damage sequences that comprise most of the risk relative to this finding, it is

assumed that EDG 1 fails initially and the Division 1 battery begins to deplete. Division

1 dc is necessary for control of the RCIC system. EDG 2 is assumed to fail after four

hours of run time, and therefore the Division 1 batteries have four hours of remaining

capacity at this time. Therefore, extra credit for time available was applied for both

diagnosis and action. High stress was assumed because the station would be in a

blackout condition. Procedures for manual operation were not available, but credit for

incomplete procedures was applied as a bounding assumption. Low training and

experience was assumed because the plant staff had not performed this mode of

operation and had not received training.

The result of the SPAR-H analysis was a failure probability of 0.3. Although there are

some short-term sequences in the SPAR results, corresponding to the failure of dc-

powered high pressure injection sources, their contribution to core damage was less

than two percent of the total risk. Therefore, for the purposes of this sensitivity

assessment, as an adequate first-order approximation, the non-recovery probability of

0.3 was applied to every core damage sequence. The result is presented in the

following table:

Exposure Period

37-Day (12/12/06 - 01/18/07) 1.3E-6/yr.

24-Day (11/18/06 - 12/12/06) 3.6E-7/yr.

Total Internal Events Result 1.7E-6/yr.

A3-9 Attachment 3

External Events:

The risk increase from fire initiating events was reviewed and determined to have a

small impact on the risk of the finding. Only two fire scenarios were identified where

equipment damage could cause a LOOP to occur. One was a control room fire that

affected either Vertical Board F or Board C. The second was a fire in the Division 2

critical switchgear. For the control room fires, the scenario probabilities are remote

because of the confined specificity of their locations and the fact that a combination of

hot shorts of a specific polarity are needed to cause a LOOP. In addition, recovery from

a LOOP induced in this manner would be likely to succeed because a minimum of four

hours would be available (based on an 8-hour battery capacity and a four hour depletion

of the Division 1 battery that provides power to the reactor core isolation cooling system)

prior to the EDG 2 failure, power would presumably be available in the switchyard, and

the breaker manipulations needed to complete this task would be possible and within

the capability of an augmented plant staff that would respond to the event.

The other class of fires that would result in a LOOP were those that require an

evacuation of the control room. In this case, plant procedures require offsite power to

be isolated from the vital buses and the preferred source of power, the Division 2 EDG,

is used to power the plant. With the assumption that the Division 2 EDG will fail four

hours into the event, a station blackout would occur at this time. The sequences that

could lead to core damage would include a failure of the Division 1 EDG, such that

ultimate success in averting core damage would rely on recovery of either EDG or of

offsite power. A review of the onsite electrical distribution system did not reveal any

particular difficulties in restoring switchyard power to the vital buses in this scenario,

especially given that at least four hours are available to accomplish this task.

In general, the fire risk importance for this finding is small compared to that associated

with internal events because onsite fires do not remove the availability of offsite power in

the switchyard, whereas, in the internal events scenarios, long-term unavailability of

offsite power is presumed to occur as a consequence of such events as severe weather

or significant electrical grid failures.

The CNS IPEEE Internal Fire Analysis screened the fire zones that had a significant

impact on overall plant risk. When adjusted for the exposure period of this finding, the

cumulative baseline core damage frequency for the zones that had the potential for a

control room evacuation (and a procedure-induced LOOP) or an induced plant centered

LOOP was approximately 3.6E-7/yr. The methods used to screen these areas were not

rigorous and used several bounding assumptions. The analyst qualitatively assumed

that the increase in risk from having EDG 2 in a status where it is assumed to fail at four

hours would likely be somewhat less than one order of magnitude above the baseline, or

3.6E-6/yr. This is easily demonstrated by an assumption that failure to re-connect

offsite power within a period of at least four hours is well less than 10 percent. Based

on these considerations, the analyst concluded that the risk related to fires would not be

sufficiently large to change the risk characterization of this finding.

The seismicity at CNS is low and would likely have a small impact on risk for an EDG

issue.

A3-10 Attachment 3

As a sensitivity, data from the RASP External Events Handbook was used to estimate

the scope of the seismic risk particular to this finding. The generic median earthquake

acceleration assumed to cause a loss of offsite power is 0.3g. The estimated frequency

of earthquakes at CNS of this magnitude or greater is 9.828E-5/yr. The generic median

earthquake frequency assumed to cause a loss of the diesel generators is 3.1g, though

essential equipment powered by the EDGs would likely fail at approximately 2.0g. The

seismic information for CNS is capped at a magnitude of 1.0g with a frequency of

8.187E-6. This would suggest that an earthquake could be expected to occur with an

approximate frequency of 9.0E-5/yr that would remove offsite power but not damage

other equipment important to safe shutdown. In the internal events discussion above, it

was estimated that LOOPs that exceeded four hours duration would occur with a

frequency of 3.91E-3/yr. Most LOOP events that exceed the four hour duration would

likely have recovery characteristics closely matching that from an earthquake. The ratio

between these two frequencies is 43. Based on this, the analyst qualitatively concluded

that the risk associated with seismic events would be small compared to the internal

result.

Flooding could be a concern because of the proximity to the Missouri River. However,

floods that would remove offsite power would also likely flood the EDG compartments

and therefore not result in a significant change to the risk associated with the finding.

The switchyard elevation is below that of the power block by several feet, but it is not

likely that a slight inundation of the switchyard would cause a loss of offsite power. The

low frequency of floods within the thin slice of water elevations that would remove offsite

power for at least four hours, but not render the diesel generators inoperable, indicates

that external flooding would not add appreciably to the risk of this finding.

Based on the above, the analyst determined that external events did not add

significantly to the risk of the finding.

Large Early Release Frequency:

In accordance with Manual Chapter 0609, Appendix A, Attachment 1, Step 2.6,

"Screening for the Potential Risk Contribution Due to LERF," the analyst reviewed the

core damage sequences to determine an estimate of the change in large early release

frequency caused by the finding.

The LERF consequences of this performance deficiency were similar to those

documented in a previous SDP Phase 3 evaluation regarding a misalignment of gland

seal water to the service water pumps. The final determination letter was issued on

March 31, 2005 and is located in ADAMS, Accession No. ML050910127. The following

excerpt from this document addressed the LERF issue:

The NRC reevaluated the portions of the preliminary significance determination related

to the change in LERF. In the regulatory conference, the licensee argued that the

dominant sequences were not contributors to the LERF. Therefore, there was no

change in LERF resulting from the subject performance deficiency. Their argument was

A3-11 Attachment 3

based on the longer than usual core damage sequences, providing for additional time to

core damage, and the relatively short time estimated to evacuate the close in population

surrounding Cooper Nuclear Station.

LERF is defined in NRC Inspection Manual Chapter 0609, Appendix H, Containment

Integrity Significance Determination Process as: the frequency of those accidents

leading to significant, unmitigated release from containment in a time frame prior to the

effective evacuation of the close-in population such that there is a potential for early

health effect. The NRC noted that the dominant core damage sequences documented

in the preliminary significance determination were long sequences that took greater than

12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> to proceed to reactor pressure vessel breach. The shortest calculated interval

from the time reactor conditions would have met the requirements for entry into a

general emergency (requiring the evacuation) until the time of postulated containment

rupture was 3.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />. The licensee stated that the average evacuation time for CNS,

from the declaration of a General Emergency was 62 minutes.

The NRC determined that, based on a 62-minute average evacuation time, effective

evacuation of the close-in population could be achieved within 3.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />. Therefore, the

dominant core damage sequences affected by the subject performance deficiency were

not LERF contributors. As such, the NRCs best estimate determination of the change

in LERF resulting from the performance deficiency was zero.

In the current analysis, the total contribution of the 30-minute sequences to the current

case CDF is only 0.17% of the total. For two hour sequences, the contribution is only

0.04 percent. That is, almost all of the risk associated with this performance deficiency

involves sequences of duration four hours or longer following the loss of all ac power.

Based on the average 62 minute evacuation time as documented above, the analyst

determined that large early release did not contribute to the significance of the current

finding.

References:

GE-NE-E1200141-04R2, Table 5-1, Shutdown Power at Cooper Nuclear Station

(proprietary)

Green Screen Source Data, External Events PRA model, Nine Mile Point, Unit 1

NUREG/CR-6890, Reevaluation of Station Blackout Risk at Nuclear Power Plants,

Analysis of Loss of Offsite Power Events: 1986-2004"

A3-12 Attachment 3