IR 05000266/2008007

IR 05000266-08-007, 05000301-08-007, on 01/18/2008 - 03/07/2008, Point Beach, Special Inspection to Review Circumstances Surrounding the Loss of the Unit 1, 1X 04 Transformer and Safety Bus 1B 04

ML081130194
Person / Time
Site:	Point Beach
Issue date:	04/21/2008
From:	Stephanie West NRC/RGN-III
To:	Mccarthy J Florida Power & Light Energy Point Beach
References
IR-08-007
Download: ML081130194 (37)

Text

ril 21, 2008

SUBJECT:

POINT BEACH NUCLEAR PLANT NRC SPECIAL INSPECTION REPORT 05000266/2008007(DRS) AND 05000301/2008007(DRS)

Dear Mr. McCarthy:

On March 7, 2008, the U.S. Nuclear Regulatory Commission (NRC) completed a special inspection at your Point Beach Nuclear Plant Units 1 and 2. The special inspection evaluated the facts and circumstances surrounding the declaration of a Notice of Unusual Event (NOUE)

on January 15, 2008, due to a loss of offsite power following a Unit 1 low voltage station auxiliary transformer lockout.

Based on the risk and deterministic criteria specified in Management Directive 8.3, "NRC Incident Investigation Program," Inspection Procedure 71153, "Followup of Events and Notices of Enforcement Discretion," and the equipment performance problems which occurred, a special inspection was initiated in accordance with Inspection Procedure 93812, "Special Inspection."

The special inspection evaluated the NOUE, as well as the actions taken by your staff in response to the unexpected equipment condition. The inspection focus areas are detailed in the special inspection charter (Attachment 3).

The enclosed inspection report documents the inspection results, which were discussed at the March 7, 2008, exit meeting with you and other members of your staff. The determination that the inspection would be conducted was made by the NRC and the inspection started on January 18, 2008.

The inspection examined activities conducted under your license as they relate to safety and compliance with the Commission's rules and regulations and with the conditions of your license.

The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.

The report documents one NRC-identified finding and two self-revealing findings of very low safety significance (Green). The three findings were determined to involve violations of NRC requirements. However, because of the very low safety significance and because they are entered into your corrective action program, the NRC is treating these findings as Non-Cited Violations (NCVs) consistent with Section VI.A.1 of the NRC Enforcement Policy. If you contest any NCV in this report, you should provide a response within 30 days of the date of this inspection report, with the basis for your denial, to the Nuclear Regulatory Commission, ATTN.: Document Control Desk, Washington, DC 20555-0001; with copies to the Regional Administrator, Region III; the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC 20555-0001; and the NRC Resident Inspector at the Point Beach Nuclear Plant.

In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRC's document system (ADAMS). ADAMS is accessible from the NRC Website at http://www.nrc.gov/reading rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA by A. Boland Acting For/

Steven West, Director Division of Reactor Safety Docket Nos. 50-266; 50-301 License Nos. DPR-24; DPR-27 Enclosure: Inspection Report 05000266/2008007(DRS); 05000301/2008007(DRS)

w/Attachments: 1. Supplemental Information 2. Timeline of Events Unit 1 3. Special Inspection Charter cc w/encl: M. Nazar, Senior Vice President and Nuclear Chief Operating Officer J. Stall, Senior Vice President and Chief Nuclear Officer R. Kundalkar, Vice President, Nuclear Technical Services Licensing Manager, Point Beach Nuclear Plant M. Ross, Managing Attorney A. Fernandez, Senior Attorney K. Duveneck, Town Chairman Town of Two Creeks Chairperson Public Service Commission of Wisconsin J. Kitsembel, Electric Division Public Service Commission of Wisconsin State Liaison Officer

SUMMARY OF FINDINGS

IR 05000266/2008007(DRS); 05000301/2008007(DRS); 01/18/2008 - 03/07/2008; Point

Beach Nuclear Plant; Special Inspection to Review Circumstances Surrounding the Loss of the Unit 1, 1X-04 Transformer and Loss of Unit 1 Safety Bus 1B-04.

This report covers a 49-day period of special inspection by two NRC Region III inspectors and one resident inspector. The inspection identified three Green findings. The significance of most findings is indicated by their color (Green, White, Yellow, Red) using Inspection Manual Chapter (IMC) 0609, "Significance Determination Process" (SDP). Findings for which the SDP does not apply may be "Green" or be assigned a severity level after NRC management review. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG 1649, "Reactor Oversight Process," Revision 3, dated July 2000.

A. Inspector-Identified and Self-Revealed Findings

Cornerstone: Initiating Events

Green.

The inspectors identified a finding of very low safety significance and an Non-Cited Violation (NCV) of 10 CFR Part 50, Appendix B, Criterion XI, Test Control, for the licensees failure to establish a test program that adequately demonstrated that medium voltage cables subjected to submersion would perform satisfactorily in-service. Specifically, the on-line, energized partial discharge testing methodology that Point Beach adopted through the 2003 Excellence Plan, to periodically assess the condition of power cables that had been submerged, failed to provide any indication of declining cable performance or indication of an imminent failure of the 1X-04 transformer cables before the actual failure on January 15, 2008. All previous test results for the 1X-04 transformer cables showed only low levels of deterioration.

This finding was determined to be more than minor in accordance with IMC 0612,

Power Reactor Inspection Reports, Appendix B, Issue Screening, because if left uncorrected the finding would become a more significant safety concern. In addition, it affected the Initiating Events cornerstone attribute of equipment performance reliability as well as the Initiating Events cornerstone objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The finding did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions would not be available. Therefore, the finding screened as having very low safety significance. Additionally, the inspectors determined that the finding has a cross-cutting aspect in the area of problem identification and resolution. Specifically, the licensee failed to use operating experience information, including internally generated lessons learned, to support plant safety by collecting and evaluation relevant internal and external operation experience (P.2(a)). (Section 4OA3.8b.1).

Green.

A self-revealing finding of very low safety significance and an NCV was identified for the licensees failure to comply with 10 CFR Part 50, Appendix B,

Criterion XVI, Corrective Actions, for the licensees failure develop effective corrective actions to maintain the design environment for the underground cables at Point Beach. Specifically, since 1997, numerous corrective action documents were generated to capture concerns associated with cable submergence and water ingress through underground cableways and manholes. However, adequate corrective actions to address the groundwater issue were not implemented for all the manholes and cableways with a known history of flooding. The failure to implement timely corrective actions to address a long term solution to the site-submerged cable issues, identified since 1997, led to the January 15, 2008, failure of the 1X-04 transformer cables due to prolonged exposure to water.

This finding was determined to be more than minor in accordance with IMC 0612,

Power Reactor Inspection Reports, Appendix B, Issue Screening, because the finding could reasonably be viewed as a precursor to a significant event and if left uncorrected, the finding could become a more significant safety concern. In addition, it affected the Initiating Events cornerstone objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Specifically, the failure to correct the submerged cable issue in a timely minor could potentially lead to other cable failures as a result of continued degradation of submerged cables. The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The 1X-04 cable failure that occurred did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions would not be available. Therefore, the finding screened as having very low safety significance. The inspectors also determined that the primary cause for this finding is related to the cross-cutting area of problem identification and resolution. Specifically, under the component of corrective action program, the licensee failed to take appropriate corrective actions to address safety issues and adverse trends in a timely manner, commensurate with their safety significance and complexity (P.1(d)).

(Section 4OA3.8b.2)

Cornerstone: Mitigating Systems

Green.

A self-revealing finding of very low safety significance and an NCV was identified for the licensees failure to comply with 10 CFR Part 50, Appendix B,

Criterion III, Design Control, for the licensees failure to assure that the design basis, associated with the ABB-GKT 50G relays, was correctly translated into specifications for the relays setpoints. As a result, the high frequency transients caused by the repeated grounding of the non-safety-related 1X-04 cables on January 15, 2008, caused the unintended actuation of the 50G/A52-84 Relay and the isolation of power to safety-related bus 1B-04.

This finding was determined to be more than minor in accordance with IMC 0612,

Power Reactor Inspection Reports, Appendix B, Issue Screening, because, if left uncorrected, the issue would have become a more significant safety concern. In addition, the finding affected the Mitigating Systems attributes of design control of plant modifications and equipment performance availability and reliability. This finding also affected the Mitigating Systems cornerstone objective of ensuring the availability, reliability, and capability of systems. The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The finding was considered to be of very low safety significance (Green)

because all of the questions in IMC 0609.04 Table 4a - Characterization Worksheet for the Mitigating Systems Cornerstone were answered No. Additionally, there was no cross cutting aspect associated with this finding because the performance deficiency was not indicative of current performance. (Section 4OA3.4b.1)

Licensee-Identified Violations

No findings of significance were identified.

REPORT DETAILS

SUMMARY OF PLANT EVENT On January 15, 2008, at 2:04 pm, Point Beach Unit 1 experienced a loss of the safety-related safeguards bus 1B-04 and a lockout of the low voltage station auxiliary transformer 1X-04, ultimately resulting in a Unit 1 shutdown and cool down to cold shutdown conditions. The loss of transformer 1X-04 resulted in a Notice of Unusual Event (NOUE) declaration.

Before the NOUE, both units were operating at one hundred percent power. At 1:46 am, on the day of the event, plant personnel heard multiple loud reports from manhole number 3 (Z-065C) located on the east side of the G-05 Building. The noise lasted approximately 5 minutes and no arcing was observed in the manhole. At 2:30 am, the plant manager, operations manager, and the duty and call supervisor were informed. At 5:05 am, the shift manger informed the NRC resident inspector of the manhole issue. The licensee implemented hourly inspections of the manhole and subsequently completed walkdowns of all plant 4160 Volt alternating current (Vac) and 13.8k Vac busses. All observations and indications were normal. At 2:04 pm, the Unit 1 480 Vac safeguards bus 1B-04 experienced a loss of power, followed shortly thereafter by a lockout and loss of Unit 1s low voltage station auxiliary transformer 1X-04. Transformer 1X-04 was supplying offsite power to busses 1A-03 and 1A-04 at the time of the event. Busses 1A-03 and 1A-04 were, in turn, supplying power to the 4160 Vac safeguards busses 1A-05 and 1A-06. This perturbation of the electrical distribution system resulted in multiple Technical Specification Action Conditions (TSACs). At 2:15 pm, classification of an unusual event was made due to the loss of offsite power. At 8:31 pm, power was restored to the 480 Vac safeguards bus 1B-04. On January 16, 2008, at 3:49 pm, Unit 1 commenced a Technical Specification required shutdown. At 6:07 pm, offsite power was restored to the 4160 Vac safeguards busses 1A-05 and 1A-06. At 7:48 pm, the Unit 1 reactor was tripped and Unit 1 entered Mode 3. At 8:35 pm, the NOUE was terminated due to restoration of offsite power to both safety-related 4160 Vac busses.

Based on the probabilistic risk and deterministic criteria specified in Management Directive 8.3, "NRC Incident Investigation Program," and Inspection Procedure 71153, "Event Followup," and due to the equipment performance problems which occurred, a Special Inspection was initiated in accordance with Inspection Procedure 93812, "Special Inspection."

The inspection focus areas included the following charter items:

• Establish the sequence of events related to the loss of bus 1B-04 and the 1X-04 transformer, including a historical timeline. When creating the timeline, include related abnormal conditions such as electrical grounds, faults and unusual indications that have occurred in the recent past (since 2002);

• Assess the licensee response to the event from an operations perspective. This assessment should include decision-making for the plant shutdown (e.g., timing, training, etc.), Emergency Action Level classifications, and actions following the isolation of a letdown valve;

• Assess the licensees extent of condition evaluation associated with the loss of the 1X-04 transformer and/or underground cabling, including the licensees evaluation of the condition of cables in the same cable bus duct, cables of similar age, or cables of similar environment;

• Monitor and assess the licensees determination of the cause for the tripping of the bus 1B-04 supply breaker 1A52-84;

• Verify that procedures used to cross-connect the Unit 1 busses through the Unit 2, 2X-04 Transformer were applicable to the plant configuration, including a review of revisions in place when the cross-connect was performed, revisions in place before the cross-connect and plant status assumed for implementation of the procedures. Also verify change screenings and 10 CFR 50.59 reviews were performed for recent procedure revisions;

• Monitor and assess the licensees corrective actions taken in response to this event to restore normal offsite power to the Unit 1 busses and address the extent of cause;

• Evaluate the licensee calculations used to justify the cross-connect of the Unit 1 busses through the Unit 2, 2X-04 transformer, including voltage drop calculations, short circuit calculations and coordination calculations. This effort should focus on verifying the system calculations and models bounded the plant configuration;

• Evaluate the adequacy of the licensees corrective actions taken in response to the known manhole and cable vault flooding issues originally identified in 2003, and discussed in the licensees response to Generic Letter 2007-01;

• Monitor and assess the licensees root cause activities and evaluations regarding this event; and

• Identify potential design deficiencies or vulnerabilities regarding the onsite electrical distribution system.

4OA3 Special Inspection

.1 Establish the Sequence of Events Related to the Loss of Bus 1B-04 and the 1X-04

Transformer, Including a Historical Timeline

a. Inspection Scope

The inspectors reviewed operator logs, plant parameter recordings, and computer trending information, and conducted interviews with licensee operating crew members in developing the sequence of events. In addition, the inspectors sequence of events was reviewed against the licensee generated sequence of events to ensure completeness and accuracy.

b. Findings and Observations

No findings of significance were identified. The inspectors generated sequence of events is included with this report as Attachment 1 and an event narrative summary was presented in this reports Summary of Plant Event, discussed above.

.2 Assess the Licensee Response to the Event from an Operations Perspective

a. Inspection Scope

The inspectors used direct observation of the event and subsequent licensee activities in conjunction with reviews of logs and the sequence of events, and personnel interviews to assess the quality and effectiveness of the licensees response to the event.

b. Findings and Observations

Members of the inspection team observed the licensees response to the event from inside the control room as well as the outage control center. Overall, control operators and senior reactor operators were observed to have responded appropriately utilizing station procedures. All applicable Technical Specification (TS) action statements were entered and the unit was safely shutdown within the TS required time. The notice of unusual event (NOUE) classification was declared in a timely manner and was appropriately classified in-accordance with the stations emergency plan. Operating procedures were appropriately revised and just-in-time operator training was completed before the satisfactory implementation of the Unit 1 to Unit 2 cross-tie evolution, as well as the backfeed evolution, which successfully aligned offsite power through the Unit 1, 1X-02 transformer eliminating the need for the unit cross-tie.

Based on these inspection activities and observations by several inspectors during the event, the inspectors concluded that plant operators maintained the unit in a safe condition. No findings of significance were identified; however, the following observation was made.

b.1 Untimely Response to Letdown Isolation On January 15, 2008, at 9:11pm, the licensee attempted to reestablish normal letdown after bus 1B-04 was re-energized; however, the attempt failed due to the inability to open letdown isolation valve 1CV-371A. Because normal letdown could not be established, the operators were required to maintain the unit on excess letdown, which operated at a significantly lower flow rate and limited the rate at which Unit 1 could optimally shutdown. The inspectors observed that following initial discovery at 9:11pm, troubleshooting activities did not begin until approximately 5:30am, the next day, in the control room and 8:30am in containment. At 3:03pm, the letdown isolation valve 1CV-371A was returned to service and normal letdown was established at 3:27pm. As a result, the plant shutdown was delayed by almost 7 hours8.101852e-5 days <br />0.00194 hours <br />1.157407e-5 weeks <br />2.6635e-6 months <br />. The inspectors concluded that the activities associated with the licensees troubleshooting efforts and return to service of the letdown isolation valve were untimely and due to inadequate prioritization. This issue was considered minor in nature since the shutdown still commenced in time to meet the TS required shutdown deadline. The inspectors considered this issue to be minor in nature because the TS required shutdown time limit was met and the licensee captured the issue in CRs 01119997 and 01120194.

.3 Assess the Licensees Extent of Condition Evaluation Associated with the Loss of the

1X-04 Transformer and/or Underground Cabling

a. Inspection Scope

The inspectors observed the licensees root cause team activities, interviewed personnel, independently reviewed manhole and cableway design documents and drawings, observed in-field manhole inspections, reviewed CRs and work orders, and reviewed subsequent cable testing results to assess the licensees extent of condition evaluation.

b. Findings and Observations

Overall, the inspectors concluded that the licensees extent of condition evaluation was conducted in a thorough and methodical manner. The licensees extent of condition analysis placed its focus on underground inaccessible cables because the cause of the cable failures on the 1X-04 transformer was in the underground duct bank. Cables at 600 Vac or less were excluded from the scope because the failure modes for cables of low voltage were different in general than those for cables of 1000 Vac and above.

All cables within the scope of the maintenance rule were considered; however, only those with all or part of their length running underground were further analyzed. Most of the plants power feeds did not fall into this category as they were not underground or inaccessible. Below is a summary of the licensees evaluation results for those cable groups determined to be in-scope:

• Transformer 2X-04 to busses 2A-03 and 2A-04. The cables had the same insulation (butyl rubber) and the same age but were in a dissimilar environment (less instances of observed submergence and conduits never plugged).

• Emergency diesel generators (EDGs) G-01 and G-02. The cables had the same insulation (butyl rubber) and the same age but were in a dissimilar environment (cables mostly above ground) and dissimilar operating conditions (not constantly energized).

• Circulating water pump motor power cables. EDGs G-03 and G-04, and busses 1A - 06 and 2A-06. The cables were newer and had ethylene propylene rubber (EPR) insulation and a dissimilar environment (conduits were never plugged).

• G-05 gas turbine generator and other 13.8 kVac power cables. The cables were newer and had EPR insulation and a dissimilar environment (conduits never plugged).

• Medium voltage cables in manholes 1, 2, 5, 14, 15, 16, 18, 19, and 20. The cables were inspected to identify any anomalies including the presence of plugged conduits; none were found except for seals on two low voltage conduits that contained EPR type cable and which were required for Appendix R fire separation purposes.

No findings of significance were identified; however, the following observation was made.

b.1 Inadequate Manhole Inspection Frequency On February 2, 2008, the inspectors and NRC regional management discussed the licensees extent of condition evaluation results with the licensee before restart.

Based on NRC questions, the licensee subsequently opened and inspected the Z-067 (A - D) manholes since they had not been inspected since June 2006. As a result, water levels were discovered which required pump down in three of the four manholes.

The inspectors noted that the licensee made the decision in 2003 not to include the Z-067A-D manholes in the periodic call-up program, which had been established to prevent cable submergence, despite a documented history of water intrusion.

Manholes Z-067A-D contained 4160 Vac power cables that were connected to the safety-related Unit 1 and Unit 2 B train safety injection pumps. In 2003, during initial screening for manhole inclusion in the call-up program, the licensee stated that the Z-067 series manholes were not included because the cables were only in-service for approximately 15 years, the cable insulation was EPR insulation which is less susceptible to water induced degradation, the cables were not constantly energized, and the cable testing results were satisfactory.

The inspectors questioned the licensees basis for not including the Z-067 series manholes in the call up program and concluded that the frequency of inspection was insufficient to prevent the long term submersion of the safety-related cables in the Z-067 series manholes. Specifically, the inspectors noted that, although the Z-067 manholes were periodically inspected, they were not inspected at the same weekly or monthly frequency as other manholes with similar groundwater intrusion issues. The inspectors also noted that even though EPR insulation is less susceptible to water treeing, it is not immune to that failure mechanism and that long term water submersion is undesirable. As a result, the licensee issued CR 01121086 to reevaluate the adequacy of the frequency of inspections and pump downs of the Z-067 series manholes.

.4 Monitor and Assess the Licensees Determination of the Cause for the Tripping of the

Bus 1B-04 Supply Breaker 1A52-84

a. Inspection Scope

The inspectors observed the licensees root-cause team activities; including reviewing the root-cause teams refute matrix and fault-tree analysis. The inspectors also observed the in-field troubleshooting of the 1A52-84 breaker and its associated 50G relay. In addition, the inspectors independently reviewed condition reports, vendor manuals and system design-basis documents.

b. Findings and Observations

b.1 Inappropriate Relay Setpoint Selection

Introduction:

A Green, self-revealing NCV of 10 CFR Part 50, Appendix B, Criterion III, Design Control, was identified for the licensees failure to assure that the design basis associated with the ABB-GKT 50G relays were correctly translated into relay setpoint specifications

Description:

On January 15, 2008, a ground fault event occurred on the 4160 Vac medium voltage cables running from the secondary side of Unit 1 transformer 1X-04 to busses 1A-03 and 1A-04. Approximately 24 seconds before the 1X-04 transformer lockout, the 50G ground fault protection relay on breaker 1A52-84 spuriously actuated, which caused the isolation of power to safety-related bus 1B-04 for approximately 6.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />. The licensee later determined that the most probable cause of this unintended relay actuation was high frequency transients caused by the repeated grounding of the non-safety-related 1X-04 cables.

The licensees root cause team also concluded that the length of the cables associated with the 1A52-84 breaker (5,010 ft) was a significant factor in the 50G relays spurious actuation. Specifically, the potential for high frequency currents to adversely affect the 50G relay increases as the length of the conductors increase. Based on the licensees analysis of cable lengths, numbers of conductors per phase, and pickup setpoint, the 50G relay of breaker 1A52-84 was found to be particularly susceptible to this potential actuation mechanism.

In 1995, Point Beach completed a significant modification to the 4160 Vac electrical distribution system under modification 91-116. There are currently six applications of this ABB-GKT 50G, Model 202L2118 relay in-service at Point Beach, and all were put in place as part of this modification. The other 4160 Vac breaker ground fault relays used throughout the rest of the plant are Westinghouse ITH 50G relays with an average of 364 ft of supply cables, versus 5,010 ft of cable for the 1A52-84 50G relay.

As such, the licensee concluded that the likelihood of developing similar undesired high frequency currents in the Westinghouse relays was very low. The Westinghouse relays are also considered immune to high frequency signal interference due to basic internal design differences.

Through the licensees failure investigation process, the 1A52-84 50G relay was removed and sent to an FPL laboratory for additional testing. This testing confirmed that the relay was particularly responsive to high frequency primary currents. The relay was found to actuate on a primary current spike of duration less than the time delay setting. Therefore, if the high frequency current pulse was of sufficient magnitude, the relay would spuriously actuate before the time delay setpoint.

Through this review, the inspectors concluded that the licensee did not adequately evaluate the potential adverse effects posed by the cable lengths used in Modification 91-116 on the ABB-GKT 50G relays. The licensee implemented corrective actions to increase the primary current pickup setpoint of the 1A52-84 breaker 50G relay from 10 Amps to 30 Amps in order to provide additional margin to prevent premature tripping the 50G relays. Additionally, engineering personnel were also considering changing the time delay setpoint from 0.1 seconds to 0.3 seconds to also decrease the probability of spurious relay actuation. The licensee planned to change the setpoint of all six ABB-GKT type 50G relays.

Analysis:

The inspectors determined that the failure to establish appropriate relay setpoints specific to the design application, which led to the loss of a safety-related bus, was a performance deficiency warranting a significance determination. This finding was determined to be more than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Screening, because, if left uncorrected, the issue would have become a more significant safety concern. In addition, the finding affected the Mitigating Systems attributes of design control of plant modifications and equipment performance availability and reliability. This finding also affected the Mitigating Systems cornerstone objective of ensuring the availability, reliability, and capability of systems that respond to initiating events to prevent undesirable consequences. The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The finding was considered to be of very low safety significance (Green) because all of the questions in IMC 0609.04 Table 4a - Characterization Worksheet for the Mitigating Systems Cornerstone were answered No. Additionally, there was no cross cutting aspect associated with this finding because the performance deficiency was not indicative of current performance.

Enforcement:

Title 10 of the CFR Part 50, Appendix B, Criterion III, Design Control, requires, in part, that measures shall be established to assure that applicable regulatory requirements and the design basis are correctly translated into specifications, drawings, procedures, and instructions.

Contrary to the above, from initial in-service installation (1995) to March 7, 2008, the licensee failed to establish appropriate septoints to ensure that the design basis was correctly translated into specifications, drawings, procedures, and instructions.

Because of the very low safety significance of this finding and because the finding was captured in the licensees corrective action program as CR 01119996-01, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy. Subsequently, the licensee initiated corrective actions to decrease the probability of spurious relay actuation by modifying the 50G relay setpoints. In addition, the licensees root cause team recommended that an engineering study be performed to model the components involved in the event and determine the settings required for the relays. (NCV 05000266/2008007-01(DRS);05000301/2008007-01(DRS))

.5 Verify That Procedures Used to Cross-connect the Unit 1 Busses Through the Unit 2,

2X-04 Transformer Were Applicable to the Plant Configuration

a. Inspection Scope

The inspectors reviewed Procedure OP 3A, Power Operation to Hot Standby, Revision 73, interviewed plant personnel, monitored control room evolutions, and reviewed corrective action program documents to assess the applicability of cross-feeding the Unit 1 busses from Unit 2 using OP 3A.

b. Findings and Observations

No findings of significance were identified.

.6 Monitor and Assess the Licensees Corrective Actions Taken in Response to this

Event to Restore Normal Offsite Power to the Unit 1 Busses and Address the Extent of Cause

a. Inspection Scope

The inspectors attended licensee meetings, interviewed plant personnel, observed maintenance activities, reviewed applicable procedures and corrective action program documents to monitor and assess the licensees corrective action and extent of cause activities.

b. Findings and Observations

No findings of significance were identified.

The licensee replaced the underground cable runs between the low-side of the 1X-04 transformer and the façade building. The inspectors observed no safety concerns with the offsite power restoration activities. Testing was completed satisfactorily on all the indoor above ground cables between the façade building splice points to busses 1A-03 and 1A-04. The licensees root cause evaluation recommended a corrective action to prevent recurrence by completing a permanent modification to replace all cables from 1X-04 and busses 1A-03 and 1A-04. In addition, the cables associated with the 2X-04 transformer low side were scheduled to be replaced and tested (i.e., similar to the 1X-04 changes) during the U2R29 Refueling Outage scheduled for April 2008. The inspectors concluded that the licensees corrective action plans and extent of cause activities were appropriate.

.7 Evaluate the Licensee Calculations Used to Justify the Cross-connect of the Unit 1

Busses Through the Unit 2, 2X-04 Transformer

a. Inspection Scope

The inspectors attended licensee meetings, reviewed 10 CFR 50.59 screenings, and reviewed Calculation 2004-0002-002-C, AC Electrical System Analysis, Revision 2-C, to evaluate the licensees cross-connect operations to shut the unit down.

b. Findings and Observations

No findings of significance were identified.

.8 Evaluate the Adequacy of the Licensees Corrective Actions Taken in Response to the

Known Manhole and Cable Vault Flooding Issues Originally Identified in 2003, and Discussed in the Licensees Response to Generic Letter 2007-01

a. Inspection Scope

The inspectors reviewed corrective action documents, work orders, the licensees responses to the NRCs GL 2007-01, the licensees response to a 2002 corrective action finding related to manhole flooding and submerged cables, the 2003 Point Beach Excellence Plan, vendor testing and inspection reports, and industry standards documents relating to the testing of medium voltage cables.

b. Findings and Observations

b.1 Inadequate Cable Test Program

Introduction:

The inspectors identified an NCV of 10 CFR Part 50, Appendix B, Criterion XI, Test Control, for the licensees failure to establish a test program that adequately demonstrated that medium voltage cables subjected to submersion would perform satisfactorily in-service. Specifically, the on-line energized partial discharge (PD) testing methodology failed to provide any indication of declining cable performance or indication of an imminent failure of the 1X-04 cables before the actual failure on January 15, 2008.

Description:

The failure of the licensees predictive cable testing program to identify deterioration of the 1X-04 transformer cables was determined to be a contributing factor to the January 15, 2008 event. The inspectors reviewed the 2003 Excellence Plan and the specific action plan EQ-15-016, Determine Condition of Underground Electrical Cables Which Have Been Submerged. The objective of EQ-15-016 was to identify underground medium voltage cables that may have degraded by being frequently submerged in water, to define a method of assessing these cables, and to establish periodic cable assessment call-ups. In May 2003, PD testing was chosen as the method of assessing and monitoring the affected 4160 Vac and 13.8 kVac cables at Point Beach.

The licensees PD test utilized a proprietary vendor process in which the data was analyzed by the vendor to determine the condition of the cable system, and then results were provided to the licensee in a test report. The inspectors reviewed the vendor test reports performed at Point Beach and noted that in all instances, the results of the 1X-04 transformer cables to busses 1A-03 and 1A-04 consistently exhibited a very low level of deterioration.

The inspectors also searched and reviewed various sources of information that were available to the site during the time the decision was made to adopt PD testing in 2003. The inspectors concluded that sufficient information was available to the licensee regarding the ability of PD testing to identify water induced cable degradation.

Specifically, technical information was available on the IEEE searchable database, dating back to September 2000, which concluded that growing water trees do not produce PD signals. Additionally, when the inspectors reviewed plant health committee meeting minutes from March 2003, it was evident that at that time, Point Beach technical staff also questioned the adequacy of PD testing when they stated that the testing could be inconclusive due to uncertainties with the test results. More recently, IEEE Standard 400.3-2006, IEEE Guide for Partial Discharge Testing of Shielded Power Cable Systems in a Field Environment, discussed issues with the accuracy of PD test data. Specifically, the accuracy in interpreting test data correctly was found to be diminished when testing cable systems in the range between very good and very bad conditions. It is the remaining life of the cables between these two extremes that cannot be predicted with great accuracy. In addition, PD testing cannot determine with certainty that a specific cable is in very good condition with essentially no probability of failure in the near future, as failure can be caused by phenomena that do not generate PD, like water trees.

The inspectors also noted that in 2005, it was evident that the site staff did not have complete confidence in the PD tests ability to detect degradation before failure because the methodology was proprietary and could not be validated by other testing methods. This concern was documented in a 2005 ODMI evaluation issued in association with CR 031655. This ODMI identified the risk of solely relying on PD testing to assess cable health.

Overall, the inspectors concluded that the cable testing program that Point Beach adopted was inappropriate for their specific application given that the objective of EQ-15-016 was to periodically assess the condition of power cables that have been submerged. The inspectors concluded that PD test acceptance criteria would not provide reasonable assurance that cable degradation would be identified before a cable failure, and, as such, were inadequate to demonstrate that structures, systems, and components would perform satisfactorily in-service.

Analysis:

The inspectors determined that the licensees failure to establish an appropriate testing program that adequately demonstrated that those medium voltage cables subjected to submergence would perform satisfactorily in-service was a performance deficiency warranting a significance determination. The finding was determined to be more than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Screening, because if left uncorrected the finding would become a more significant safety concern. In addition, it affected the Initiating Events cornerstone attribute of equipment performance reliability as well as the Initiating Events cornerstone objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The finding did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions would not be available. Therefore, the finding screened as having very low safety significance (Green). Additionally, the inspectors determined that the finding has a cross-cutting aspect in the area of problem identification and resolution. Specifically, the licensee failed to use operating experience information, including internally generated lessons learned, to support plant safety by collecting and evaluation relevant internal and external operation experience (P.2(a)).

Enforcement:

Title 10 of the CFR Part 50, Appendix B, Criterion XI, Test Control, requires, in part, that a test program shall be established to assure that all testing required to demonstrate that structures, systems, and components will perform satisfactorily in-service is identified and performed in accordance with written test procedures which incorporate the requirements and acceptance limits contained in applicable design documents.

Contrary to the above, from May 15, 2003, to March 7, 2008, the licensee failed to establish a test program that adequately demonstrated that medium voltage cables subjected to submersion would perform satisfactorily in-service. Because of the very low safety significance of this finding and because the finding was captured in the licensees corrective action program as CR 01120426-02, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy (NCV 05000266/2008007-02(DRS);05000301/2008007-02(DRS)).

The licensee has implemented corrective actions to remove PD testing from its cable monitoring program and plans to assess and select a suitable testing methodology for future cable testing.

b.2 Inadequate and Untimely Corrective Actions to Address Cable Submergence

Introduction:

A self-revealing finding an NCV of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Actions, was identified for the licensees failure to implement timely corrective actions to address the longstanding issue of submerged cables at Point Beach. The failure to implement timely corrective actions to address the site-wide submerged cable issues identified since 1997 led to the January 15, 2008, failure of the 1X-04 transformer cables due to prolonged cable exposure to water.

Description:

The inspectors noted that the concern of water-induced cable degradation was first documented on May 5, 1997, in condition report (CR) 97-1497/00005638, North Circulating Water Pump House Manhole Does Not Have a Means to Be Pumped. This CR concluded that affected manholes should have permanent drainage installed to avoid immersion of cables that could lead to operational problems. The inspectors noted that the corrective action due date was extended nine times from its original due date in 1997, and was finally closed out in July 2003 to actions taken by the Excellence Plan.

Also, during that same time period, in October 1997 the issue of groundwater in-leakage into the façade building emerged in the corrective action program as CR 97-3541/ 00149997. This CR documented that groundwater was flowing into the façade building through the cable conduits from manholes 3 and 4, which contained the cables from the Unit 1 and Unit 2, X-04 transformers. The condition evaluation performed for this CR, which was not completed until July 1999, determined that the issue was due to a deficiency with the subsoil drainage system; later modified in 2001.

This CR also documented that as a result of related evaluations, the licensee determined that water intrusion into the manholes would be significantly reduced if the sites overall groundwater level was reduced. The 2001 modification was intended to lower the groundwater level, but was subsequently found in 2003 to be ineffective at eliminating the manhole flooding or façade building in-leakage issues. This condition report was closed upon completion of the modification.

In December 2002 the NRC issued a finding for the licensees failure to establish timely and adequate corrective actions to address the flooding of manholes which contain safety and non-safety-related systems, structures, and components (FIN 50-266/301/2002013-03). The inspectors identified that the licensee had not implemented effective corrective actions to address long-standing problems with flooding in manholes and had deferred the implementation of corrective actions with insufficient basis. To address this issue, the licensee initiated actions to evaluate the frequency of pumping out the manholes and to evaluate the need to inspect other manholes for similar conditions. Shortly thereafter, the Excellence Plan was issued.

The 2003 Excellence Plan included two action plans to address the manhole and water induced cable degradation issues: EQ-15-012, Manhole and Cable Vault Flooding, and EQ-15-016, Determine Condition of Underground Electrical Cables Which Have Been Submerged. Action plan EQ-15-012 was created to implement a solution to keep the cables in the Maintenance Rule scoped manholes from becoming submerged. Action plan EQ-15-016 was developed to assess the condition of the underground 480V, 4160V, and 13.8kV cables that are safety-related or important to safety and ensure that their condition is understood and monitored.

As a subtask of EQ-15-016, a consulting firm was contracted to inspect the manholes from across the site that have exhibited a history of flooding and provide solutions.

The contractor inspected 14 manholes on June 10 and 11, 2003, and identified that groundwater was infiltrating through cracks in manhole walls and ceilings as well as around and through the conduits themselves. The contractor recommended a number of corrective actions based on their findings and engineering judgment. These actions included the rehabilitation of the affected manholes by means of resealing walls, grouting cracks, and waterproofing the hatches, as well as the rehabilitation of the conduits through applying a liner. These corrective actions were not intended as a permanent solution but were intended as a means to allow the site additional time to implement a permanent solution for the removal of infiltration. The licensee did not implement these actions.

The Excellence Plan relied upon periodic inspection and pumping of manholes to keep the cables from being continuously submerged. The inspectors noted that from January 1, 2000, to January 15, 2008, there have been 342 instances in which cables were found to be submerged despite the periodic pumping call-ups. In that same time frame, 246 condition reports relevant to submerged cables have been created.

Additionally, the Excellence Plan relied on the cable testing program to monitor the health of those cables subjected to submergence. As discussed in this inspection report, the testing methodology employed by the licensee was found to be inadequate for predictive monitoring of cable health for the sites specific application.

In summary, since 1997, the licensee generated numerous corrective action documents to address concerns at Point Beach associated with cable submergence and water ingress through underground cableways and manholes. However, timely and adequate corrective actions to address the groundwater issue were not implemented for all the manholes and cableways with a known history of flooding.

The corrective action documents deferred to actions taken as a result of the 2003 Excellence Plan. Aside from those interim corrective actions established by the Excellence Plan (i.e., to perform periodic inspections and pump-downs of select manholes and to perform periodic cable testing), the plan included a very narrow scope of long term corrective actions that only corrected the groundwater issue with two manholes to date (1 and 2). No other similarly affected manholes or cableways have had their water ingress condition permanently corrected.

Analysis:

The inspectors determined that the licensees failure to maintain the environment, for which the underground cables had been designed, was a performance deficiency warranting a significance determination. Specifically, an effective long term solution to the site-wide submerged cable problem, identified in 1997, had not been developed.

This finding was determined to be more than minor in accordance with IMC 0612, Power Reactor Inspection Reports, Appendix B, Issue Screening, because the finding could reasonably be viewed as a precursor to a significant event and if left uncorrected, the finding could become a more significant safety concern. In addition, it affected the Initiating Events cornerstone objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Specifically, the failure to correct the submerged cable issue in a timely manor could potentially lead to other cable failures as a result of continued degradation of submerged cables. The inspectors evaluated the finding in accordance with IMC 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The 1X-04 cable failure that occurred did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions would not be available. Therefore, the finding screened as having very low safety significance. The inspectors also determined that the primary cause for this finding is related to the cross-cutting area of problem identification and resolution. Specifically, under the component of corrective action program, the licensee failed to take appropriate corrective actions to address safety issues and adverse trends in a timely manner, commensurate with their safety significance (P.1(d)).

Enforcement:

The cables associated with the affected portion of the onsite 4160 Vac electrical distribution system are classified as augmented quality. In accordance with Point Beach Quality Assurance Program Nuclear Policy NP-811, the requirements of 10 CFR Part 50, Appendix B, Criterion XVI, Corrective Action, is applicable to these augmented quality components. Title 10 of CFR Part 50, Appendix B, Criterion XVI, requires, in part, that measures shall be established to assure that conditions adverse to quality, such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and non-conformances are promptly identified and corrected.

Contrary to the above, from 1997 to March 7, 2008, the licensee failed to implement timely and adequate corrective actions to resolve a condition adverse to quality concerning an appropriate method to eliminate the long term submergence and subsequent intrusion of water into underground electrical power cables. Because of the very low safety significance of this finding and because the finding was captured in the licensees corrective action program as RCE01119996-02, this violation is being treated as an NCV consistent with Section VI.A.1 of the NRC Enforcement Policy (NCV 05000266/2008007-03(DRS);05000301/2008007-03(DRS)).

The licensee has implemented or planned a number of corrective actions to address the issues identified during this event. These included the installation of a temporary modification to replace the 1X-04 cables to restore transformer 1X-04 to an operable status (EC11877) and the planned completion of a permanent modification (EC11861)to replace cables from transformer 1X-04 to busses 1A-03 and 1A-04. The design process for this modification will address all environmental concerns. Additional corrective actions have been planned, or were underway at the end of this inspection period, such as: the replacement of the 2X-04 cables during the next refueling outage in April 2008 (U2R29); testing of the 13.8 kV cables for the low-voltage side of 2X-03 during the U2R29 refueling outage; and testing of the 13.8 kVac cables for the high-voltage side of 2X-04 during the U2R29 refueling outage. The licensee also planned to assign a higher priority to the project to implement a permanent solution to the groundwater intrusion issue. These actions were documented in Root Cause Evaluation (RCE) 01119996-02.

.9 Monitor and Assess the Licensees Root Cause Activities and Evaluations Regarding

this Event

a. Inspection Scope

The inspectors conducted personnel interviews, monitored the licensees root cause team activities and the root cause evaluation progress.

b. Findings and Observations

The inspectors noted that the licensees root cause team used industry accepted root cause evaluation tools (i.e., Event and Causal Factor Charting, Why Staircase, Fault Tree Analysis). The inspectors also noted that the licensees root cause team was comprised of multi-disciplined individuals from engineering, maintenance, and operations.

The inspectors concurred with the licensees conclusion that the cause of the cable failure from the 1X-04 transformer to busses 1A-03 and 1A-04 was a direct fault to ground of the B5 cable for the feed to bus 1A-03 and that this fault was caused by long-term water induced degradation of the cables outer jacket, shield, and insulation.

The licensees root cause team concluded that the organization failed to assign the appropriate prioritization to address the issue of submerged power cables.

The inspectors concurred with the licensees assessment that the cause of the spurious actuation of the 50G relay of the 1A52-84 breaker was likely due to high frequency transients caused by the repeated grounding of cabling associated with the low side of the 1X-04 transformer. As a result, the licensees root cause team concluded that the settings for this relay were too conservative and based solely on generic IEEE guidance. In addition, laboratory testing determined that the relay would actuate on a primary current spike duration less than the existing time delay setting.

Overall, the inspectors had no findings of significance or observations.

.10 Identify Potential Design Deficiencies or Vulnerabilities Regarding the Onsite Electrical

Distribution System

a. Inspection Scope

The inspectors reviewed the licensees preliminary safety analysis report, final safety analysis report, safety evaluation reports and other associated licensing basis documents and drawings.

b. Findings and Observations

No findings of significance were identified.

The results of the inspectors review indicated that the licensees onsite electrical distribution system was within the design basis documents. However, the licensee recognized that there were some problems with the existing design and has implemented CAP01121685, Potential Vulnerability of X04 Transformer, dated February 12, 2008, to review and analyze the stations electrical distribution vulnerabilities. In addition, the licensee was in the process of designing and implementing a modification to allow back-feeding through the main transformer to supply power to the non-essential loads with the station shutdown.

4OA6 Management Meetings

.1 Exit Meeting Summary

On March 7, 2008, the inspectors presented the inspection results to Mr. J. McCarthy and members of his staff, who acknowledged the findings. The licensee acknowledged the information presented. The inspectors asked the licensee whether any materials examined during the inspection should be considered proprietary. No proprietary information was identified.

.2 Interim Exit Meetings

No interim exits were conducted.

ATTACHMENTS: 1.

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee

S. Aerts, Finance Manager

R. Anderson, Site Vice-President (DAEC)

J. Bjorseth, Plant Manager

S. Bowe, I&C Manager

S. Cassidy, Communications Manager

F. Flentje, Licensing Supervisor

F. Hennessy, Engineering Supervisor

K. Locke, Regulatory Affairs Analyst

J. McCarthy, Site Vice-President (PBNP)

P. Olson, Manager of Operations Support

M. Ray, Regulatory Affairs Manager

L. Schofield, Employees Concern Manager

J. Schweitzer, Manager of Projects

W. Smith, Production Planning Manager

T. Staskal, Regulatory Affairs Engineer

D. Tomaszewski, Engineering Director

G. Young, Recovery Plan Manager

Nuclear Regulatory Commission

J. Lara, Chief, Engineering Branch 3

R. Krsek, Senior Resident Inspector

Attachment 1

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened

05000266/2008007-01(DRS) NCV Inappropriate Relay Setpoint Selection (Section 4OA3.4b.1)

05000266/2008007-02(DRS) NCV Inadequate Cable Test Program (Section 4OA3.8b.1)

05000266/2008007-03(DRS) NCV Inadequate and Untimely Corrective Actions to Address Cable Submergence (Section 4OA3.8b.2)

Closed

05000266/2008007-01(DRS) NCV Inappropriate Relay Setpoint Selection (Section 4OA3.4b.1)

05000266/2008007-02(DRS) NCV Inadequate Cable Test Program (Section 4OA3.8b.1)

05000266/2008007-03(DRS) NCV Inadequate and Untimely Corrective Actions to Address Cable Submergence (Section 4OA3.8b.2)

Discussed

None.

Attachment 1