IR 05000271/2008009

IR 05000271-08-009, Vermont Yankee Nuclear Power Station, NRC Special Inspection

ML082840367
Person / Time
Site:	Vermont Yankee File:NorthStar Vermont Yankee icon.png
Issue date:	10/10/2008
From:	Clifford J Division Reactor Projects I
To:	Ted Sullivan Entergy Nuclear Operations
powell, rj
References
IR-08-009
Download: ML082840367 (29)

Text

October 10, 2008

SUBJECT:

VERMONT YANKEE NUCLEAR POWER STATION - NRC SPECIAL INSPECTION REPORT 05000271/2008009

Dear Mr. Sullivan:

On October 8, 2008, the U.S. Nuclear Regulatory Commission (NRC) completed a special inspection at your Vermont Yankee Nuclear Power Station. This inspection examined the circumstances associated with the identification of a leak in the east, non-safety-related cooling tower on July 11, 2008, and the potential applicability to the operability of the safety-related cell and adjacent seismically designed cell in the west cooling tower. This special inspection was initiated in accordance with NRC Management Directive (MD) 8.3, NRC Incident Investigation Program, and Inspection Manual Chapter (IMC) 0309, Reactive Inspection Decision Basis for Reactors. While the conditional risk assessment indicated that the overall risk from the identified cooling tower condition was low, the decision to conduct a special inspection was based on deterministic criteria in MD 8.3 and IMC 0309 for a repetitive event, or an event with potential generic implications for the safety-related cooling tower cell 2-1 or the seismically designed cell 2-2, in the west cooling tower, coupled with the consideration of factors such as openness and public interest. Although the July 11, 2008, leak occurred in a non-safety-related cooling tower, the NRC determined it was necessary to conduct a special inspection to ensure the continued operability of the safety-related cell and the seismically designed buffer cell, and for assurance that the safety-related cell is adequately inspected and maintained. The determination that the inspection would be conducted was made by the NRC on July 13, 2008, and the inspection started on July 14, 2008.

The enclosed inspection report documents the inspection results, which were discussed on October 8, 2008, with you and members of your staff. The inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations, and with the conditions of your license. The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel. This report documents one self-revealing finding of very low safety significance (Green).

The NRC conducted additional inspection activities following an additional circulating water system leak that occurred in the east cooling tower on September 16, 2008. This inspection was conducted because, while the NRCs primary focus is on safety-related and risk significant systems, the NRC maintains awareness of non-safety-related and low risk systems as they may have causal implications on safety-related or risk significant systems. The NRC screens issues associated with non-safety-related and low risk systems to ensure that there are no significant ramifications on safety-related and risk significant systems. The team evaluated the cooling tower leaks for these considerations, as well as for programmatic implications that may apply to safety-related systems, risk significant systems, or balance of plant systems. This is part of the overall NRC awareness of non-safety-related, low risk system issues which result in unplanned initiating events. With respect to the cooling tower issues discussed in this report, these issues have resulted in unplanned rapid downpowers. The NRC evaluates these downpowers for challenges imposed on control room operators. The NRC monitors the number of unplanned downpowers within the Reactor Oversight Process utilizing an Initiating Events Cornerstone performance indicator, specifically Unplanned Power Changes Per 7000 Critical Hours. If performance indicators reach a threshold, the NRC would engage the licensee more formally, including additional inspections to identify root and contributing causes.

We will be conducting a public outreach meeting in Brattleboro, Vermont, on October 14, 2008, to discuss our inspection activities with interested public stakeholders. We understand that you will participate in that meeting to discuss your inspection and maintenance activities for the non-safety-related cells in both the east and west cooling towers, as well as your actions to ensure the continued operability of the safety-related and seismic cells in the west cooling tower.

In accordance with 10 CFR 2.390 of the NRCs Rules of Practice, a copy of this letter, its enclosure, and your response (if any) will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of the NRCs document system (ADAMS). ADAMS is accessible from the NRC Web Site at http://www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

James W. Clifford Deputy Division Director Division of Reactor Projects

Docket No. 50-271 License Nos. DPR-28

Enclosure: Inspection Report No. 05000271/2008009 w/ Attachments: (1) Supplemental Information and (2) Special Team Inspection Charter, Revision 2, Vermont Yankee Nuclear Power Station

cc w/encl:

Vice President, Operations, Entergy Nuclear Operations Senior Vice President, Entergy Nuclear Operations Vice President, Oversight, Entergy Nuclear Operations Senior Manager, Nuclear Safety & Licensing, Entergy Nuclear Operations Senior Vice President and COO, Entergy Nuclear Operations Assistant General Counsel, Entergy Nuclear Operations

SUMMARY OF FINDINGS

IR 05000271/2008009; 07/14/2008 - 10/08/2008; Vermont Yankee Nuclear Power Station;

Special Inspection.

This report covered a 4-day period (July 14 through July 17, 2008) of onsite inspection, with in office review through October 8, 2008, by a special inspection team consisting of one branch chief, one senior resident inspector, one resident inspector, and three subject matter experts.

The team was also on site September 30, 2008, and October 1, 2008. One Green self-revealing finding was identified. The significance of most findings is indicated by their color (Green, White, Yellow, Red) using Inspection Manual Chapter (IMC) 0609, Significance Determination Process (SDP). Findings for which the SDP does not apply may be Green or be assigned a severity level after NRC management review. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649,

"Reactor Oversight Process," Rev. 4, dated December 2006.

Cornerstone: Initiating Events

Green.

A self-revealing finding of very low safety significance was identified because Entergy did not verify the technical adequacy of a design change prior to placing the circulating water system piping in east cooling tower cell 1-1 in service. As a result, four horizontal circulating water pipe support beams failed. Upon identification of the failure,

Entergy decreased reactor power to 46 percent and removed both the east and west cooling towers from service for investigation and repair. Entergys corrective actions included immediate replacement or repair of damaged and degraded structures, verification of design change acceptability, and implementation of several procedure and policy changes.

The performance deficiency was that Entergy did not perform an adequate design review as described in Entergy procedure EN-DC-115, Engineering Change Development. The finding was more than minor because it was associated with the Design Control attribute of the Initiating Events Cornerstone and affects the cornerstone objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Specifically, the inadequate design change analysis resulted in the failure of horizontal pipe supports in cooling tower cell 1-1 which damaged the circulating water system piping and resulted in a significant power reduction. The finding was determined to be of very low safety significance because it did not contribute to both the likelihood of a reactor scram and the likelihood that mitigating equipment or functions would not be available. The finding had a cross-cutting aspect related to resources in the area of Human Performance.

Entergy did not ensure that complete, accurate and up-to-date design documentation was available to adequately construct portions of non-safety-related cooling tower cells.

Specifically, Entergy did not provide detailed drawings or instructions supported by engineering calculations to implement a design change affecting the circulating water pipe horizontal support design. H.2(c) (Section 4OA3.9)

Licensee-Identified Violations

None.

REPORT DETAILS

4OA3 Special Inspection

.1 Background - Cooling Tower Design

Vermont Yankee (VY) has two cooling towers: Tower 1 (East) and Tower 2 (West). The towers provide supplemental or alternate cooling for two distinct purposes - (1)supplemental cooling for compliance with the State of Vermont thermal discharge limit while operating at full power during warm weather, and

(2) one safety-related cell to provide alternate means for removal of residual heat from the reactor in the event the service water pump house becomes unavailable. The two cooling towers are cross-flow, mechanical draft design. Each structure consists of a wooden frame supporting two water distribution headers (one along the east side of the tower and one along the west side) that direct warm water along the top of the tower. Water flows from distribution valves in each header along the tower length to a distribution deck. Water then flows through holes in the deck, and is evenly distributed through plastic channels called fill. Water cascades through the fill and is cooled by air drawn through the tower by fans at the top centerline of the tower. The cool water is collected in a basin at the bottom of the tower and circulated back to the Connecticut River or back to the plant condenser. Each cooling tower is divided into eleven cells, separated from each other by a dividing wall. There is one fan for each cell. Cooling tower 1 is a non-safety-related structure and is not of seismic design. Cooling tower 2 has two cells (designated as cells 2-1 and 2-2) which are both of seismic design. Cell 2-1 is of safety-related design and is an alternate cooling system (ACS) for removal of residual heat from the reactor in the event the service water pump house becomes unavailable. The ACS piping in cell 2-1 is independent of the circulating water distribution header. Cell 2-2 is designed with breakaway ties at the interface of cell 2-3 to facilitate a controlled separation of the seismic portions (cells 2-1 and 2-2) of the structure from the non-safety-related non-seismic portion. A similar breakaway design is used at the cell 2-3 and cell 2-4 interface to provide additional assurance that a failure of non-safety-related cells will not affect either cell 2-1 or cell 2-2.

.2 Background - 2007 Partial Collapse of Cooling Tower Cell 2-4

On August 21, 2007, a portion of cooling tower cell 2-4 collapsed while the plant was operating at full power. To maintain Vermont State thermal discharge requirements, Entergy rapidly reduced power from 100 percent to approximately 35 percent.

As documented in NRC Inspection Report 05000271/2007004, Entergy completed a root cause analysis to identify the causes associated with the partial cell collapse.

Entergy determined that root causes included both mechanistic and programmatic aspects:

• Mechanistic Aspect - Entergy concluded that a number of the wooden 4 x 4 vertical columns located inside the fill area of cell 2-4 had failed prior to the cell collapsing. The columns were already heavily loaded from the weight of the water on the distribution deck and in the circulating water header. The columns were further weakened due to stresses from a chemical iron-salt attack related to iron bolting used to connect the wooden columns, a biological fungal attack in areas that the wooden columns had been affected by the iron-salt, and from over-tightened bolts at spliced locations. The column failures caused the distribution deck to sag and the water header to separate; this resulted in additional water on the distribution deck, which increased the loading on the support columns. The added weight on columns that were already stressed caused additional columns to buckle, resulting in the collapse of a portion of the cell.

• Programmatic Aspect - The cooling tower inspection program did not require inspections of the vertical columns in the normally inaccessible fill area.

Specifically, Entergy had routinely performed remote boroscopic and/or visual inspections, but had not recognized the importance of utilizing hands-on inspection techniques to detect degraded structural conditions, such as iron-salt and fungal attack or over-tightened bolts at spliced locations.

The inspection report also documented a non-cited violation of Technical Specification 6.4, Procedures, for Entergys failure to effectively incorporate readily available industry operating experience into the cooling tower inspection program and processes (NCV 05000271/2007004-01, Inadequate Inspection Program Resulted in the Partial Collapse of a Non-Safety-Related Cooling Tower Cell).

While near term corrective actions for the partial cell collapse included inspection program procedure changes, long term actions included replacement of wooden columns with fiberglass reinforced plastic (FRP) components. The first phase of the FRP replacement was completed in May, 2008, with numerous columns replaced in the non-safety-related cooling tower cells.

.3 Event Summary

On July 11, 2008, an auxiliary operator observed leakage from a slip joint in the circulating water distribution piping in cooling tower cell 1-1. The distribution piping appeared to be sagging and significant leakage was observed from the slip joint adjacent to where the distribution piping enters the cell from the inlet riser pipe. A total of four circulating water pipe horizontal support beams (specifically, locations (also known as bents) 2, 3, 4, and 5) supporting the 60 inch diameter fiberglass pipe were observed to have failed in cell 1-1. Upon identification of the leak, Entergy decreased reactor power to 46 percent and removed both the east and west cooling towers from service for investigation and repair. The power decrease was necessitated by Vermont Yankees State of Vermont river thermal discharge permit, which limits the maximum water temperature that the plant can discharge into the Connecticut River. Upon securing the towers, Entergy performed extent-of-condition walkdowns of cooling towers 1 and 2. The licensee did not identify any significant degradation in safety-related cell 2-1 or the seismically designed buffer cell 2-2. Therefore, the licensee concluded that safety-related cell 2-1 and the seismically designed cell 2-2 were structurally sound and fully operable.

Additionally, on September 16, 2008, plant operators identified abnormal leakage from four slip joints in the circulating water distribution piping in cells 1-3 and 1-4 in the non-safety-related east cooling tower. The most severe leak was estimated to be approximately 60 gallons per minute. Upon identification of the leak, Entergy decreased reactor power to approximately 55 percent and removed both the east and west cooling towers from service for investigation and repair. The power decrease was necessitated by Vermont Yankees State of Vermont river thermal discharge permit, which limits the maximum water temperature that the plant can discharge into the Connecticut River.

On September 17, 2008, Entergy performed additional inspections of the structural materials below the water deck on cell 1-3 as this was the cell where the most leakage was identified. Entergy identified two significantly degraded columns and a number of deficiencies in other columns and horizontal members that were replaced, scheduled for repair, or marked for future inspection in accordance with their cooling tower inspection procedure.

.4 Inspect the Structural Pipe Supports for the Leaking Pipe in Cell 1-1 and the Additional

Issues Identified in Cells 2-3 and 2-4; Apply Insights From These Inspections to a Subsequent Inspection in Cells 2-1 and 2-2.

a. Inspection Scope

The team performed inspections of cell 1-1 to evaluate the failure of horizontal pipe supports, also known as saddle supports, which caused a circulating water system pipe slip joint to open on July 11, 2008. The team observed the configuration and design of the cooling towers structures, including the pipe support joints, members, and braces.

The inspectors also reviewed deficiencies identified by Entergy on cells 2-3 and 2-4 that included missing hardware and cracks in the horizontal pipe support wooden members.

Insights gleaned from these inspections were applied to the subsequent inspections described in Section 4OA3.5.

b. Findings and Observations

No findings of significance were identified. The team observed that the failure of the horizontal piping supports in cell 1-1 was indicative of an overstressed condition. It was evident that the overstressed condition was due to missing support hardware that transferred a significant portion of load from the horizontal supports to the support columns. The team verified that the licensee repaired all of the deficiencies identified in cells 2-3 and 2-4 prior to returning the west cooling tower to service.

.5 Inspect All Appropriate Portions of Cells 2-1 and 2-2.

a. Inspection Scope

The team performed detailed inspections of all accessible horizontal pipe supports, including their structural members, joints, and fasteners, as well as structural columns and cross members at the water deck level of cooling tower cells 2-1 and 2-2. The team also inspected a sample of other accessible structural members, joints, braces, and fasteners in the plenum area of cells 2-1 and 2-2. The purpose of these inspections was to verify that the deficiencies that were associated with the failure of cell 1-1 were not present in cells 2-1 and 2-2. Furthermore, the inspectors evaluated whether the pipe supports and joints conformed to current design drawings and that structural members, joints, bolts, and other fasteners were in good material condition.

b. Findings and Observations

No findings of significance were identified. The team identified several minor deficiencies in cells 2-1 and 2-2 including splits in wood members used in pipe supports and minor corrosion of steel braces, fasteners, bolts, and brackets. Entergy placed these items in their corrective action program (condition reports (CRs) 2008-2961, 2008-2962, 2008-2974) for evaluation. A support beam which did not conform to current design drawings is discussed in Section 4OA3.8. The operability of safety-related cell 2-1 is discussed in Section 4OA3.7.

.6 Review All Deficiencies in Cells 2-1 and 2-2 Identified by the Licensee and NRC

Inspectors and Assess Licensee Actions to Address These Deficiencies.

a. Inspection Scope

Prior to the licensee returning the west cooling tower to service, the inspectors reviewed the licensees disposition of NRC and licensee-identified deficiencies in cells 2-1 and 2-

2. This assessment included a review of the licensees tracking list of deficiencies

identified and completion status and condition reports written for the deficiencies.

During the walkdowns documented in Section 4OA3.5 of this report, the inspectors observed portions of the cells where repairs had been made.

b. Findings and Observations

No findings of significance were identified. The inspectors verified that licensee and NRC identified deficiencies in cells 2-1 and 2-2 were appropriately evaluated. Items warranting repair were either completed prior to returning the west cooling tower to service or, if minor, scheduled for repair during future maintenance periods.

.7 Review and Reach a Conclusion on the Current Operability Determination for Cells 2-1

and 2-2.

a. Inspection Scope

The inspectors reviewed the licensees operability determination for cells 2-1 and 2-2.

Primary factors in the inspectors review included the applicability of the preliminary mechanistic failure mode to cells 2-1 and 2-2 and the inspectors direct observation of the material condition of accessible portions of cells 2-1 and 2-2.

b. Findings and Observations

No findings of significance were identified. Although the team identified several minor deficiencies (such as minor corrosion of fasteners, loose fasteners, and splits in the wood members), the team concluded that Entergys assessment confirming the Technical Specification operability of cooling tower cells 2-1 and 2-2 was reasonable.

This was based on:

• The design changes (FRP installation) that led to only two clips supporting the horizontal pipe support beams in cell 1-1 were not implemented in cells 2-1 and 2-2.

• All hardware documented in design drawings was located in cells 2-1 and 2-2 and was found to be in acceptable condition, with the exception of a support beam which did not conform to current design drawings which is discussed in Section 4OA3.8.

• The material condition of the pipe supports and structural members were assessed to be adequate by team members with structural engineering expertise.

.8 Evaluate the Design Adequacy of Apparent Non-Standard Horizontal Pipe Supports in

Cells 2-1 and 2-2.

a. Inspection Scope

As previously discussed, the inspectors conducted a walkdown of cells 2-1 and 2-2 to visually examine the structural members and the header beam pipe supports on the water deck. The inspectors reviewed applicable design specifications, fabrication drawings, material lists, and repair and modification history. Also, the inspectors interviewed system engineers and other personnel who were knowledgeable in the component function, maintenance and operational history. The inspectors reviewed CR VTY-2008-02915 which documented identified nonconforming issues within cell 2-1 and cell 2-2. Specifically, one horizontal support member was made from different materials than the remainder of the horizontal supports in the cooling tower, which was not in accordance with design drawings.

b. Findings and Observations

No findings of significance were identified. However, the inspectors noted that the horizontal support at bent 6 in cell 2-1 was not constructed in accordance with the design specification.

The design of the circulating water pipe support header beams specifies a bolted assembly consisting of one 4 x 10 center member sandwiched between two 2 x 10 members. The materials of construction are specified to be Douglas Fir #1 grade.

The inspectors noted, however, that the horizontal support in bent 6 (not a failed support) in cell 2-1 had been fabricated using plywood in substitution for the design-specified solid center 4 x 10 member. Although the materials of construction were not in accordance with the specification requirements, the inspectors noted that the horizontal support did not exhibit evidence of distress that was indicative of an active failure mechanism. The licensee replaced the horizontal support with a support fabricated in accordance with the design specification prior to returning the west cooling tower to service.

The inspectors reviewed calculation VYC-3075, Rev. 0, Evaluation of Header Beams; Cooling Tower CW [Circulating Water] Pipe, and found no reference within the calculation that recognized the plywood composite fabrication in the analysis as an acceptable configuration. In addition, the disposition to CR VTY-2008-02915 did not provide a quantitative basis for the past operability of the nonconforming design. The inspectors, therefore, continued to question past operability. In response, the licensee provided additional analysis on August 7, 2008, which analytically supported the past operability of the as-found condition.

Although the inspectors determined the licensees past operability determination was adequate, the installation of an unanalyzed modification in safety-related cell 2-1 constituted a violation of 10 CFR 50, Appendix B, Criterion III, Design Control.

However, because subsequent analysis determined the modification was acceptable and the as-found condition of the non-conforming condition was adequate, the inspectors concluded the issue to be a minor violation, not subject to enforcement, in accordance with the NRCs Enforcement Policy.

.9 Evaluate Entergys Root Cause Determination and Any Planned Interim and Permanent

Corrective Actions.

a. Inspection Scope

The team reviewed Entergys root cause analysis of the pipe support failures to assess the analysis as well as the adequacy of corrective action plans. In addition, the team reviewed Entergys interim corrective actions to address deficiencies noted during inspections of the cooling tower by Entergy staff and NRC inspectors. The review and verification of the interim actions are documented in sections 4OA3.4,

.5, and.6 of this

report.

b. Findings and Observations

The team determined that Entergys root cause analysis was adequate in that it thoroughly examined causal factors and provided for corrective actions to adequately address the root cause and contributing causes. The analysis identified the mechanistic root cause of the pipe support failures to be an overload of the FRP-to-wooden horizontal support connection due to an inadequately performed design analysis.

Specifically, the overload condition was due to missing support clips on the FRP support columns that transferred load from the pipe support beam to the vertical FRP column.

This conclusion was supported by subsequent engineering analysis by a vendor that determined that the two-clip configuration was inadequate and could lead to failure of the horizontal support beam. The analysis also identified organizational and programmatic weaknesses that contributed to the failure including: a lack of commitment to design control program implementation, inadequate impact assessment of schedule changes, inadequate interface among organizations, and a less than adequate configuration change document. The inspectors concluded that the information reviewed did not suggest a generic breakdown in Entergys engineering or work control processes related to non-safety related work activities, but represented a specific example of discrete human performance errors within those processes.

Additional details are described in the finding description below.

Introduction.

A self-revealing finding of very low safety significance was identified because Entergy did not verify the technical adequacy of a design change prior to placing the circulating water system piping in cooling tower cell 1-1 in service. As a result, four circulating water pipe support header beams failed resulting in circulating water pipe joint leakage. Upon identification of the failures, Entergy decreased reactor power to 46 percent and removed both the east and west cooling towers from service for investigation and repair. The power decrease was necessitated by Vermont Yankees State of Vermont river thermal discharge permit, which limits the maximum water temperature that the plant can discharge into the Connecticut River.

Description.

On July 11, 2008, an auxiliary operator observed leakage from a slip joint in the circulating water distribution piping in cooling tower cell 1-1. The distribution piping appeared to be sagging and significant leakage was observed from the slip joint adjacent to where the distribution piping enters the cell from the inlet riser pipe. A total of four circulating water pipe support header beams (bents 2, 3, 4, and 5) supporting the 60 inch diameter fiberglass pipe in this area were observed to have failed in cell 1-1.

The horizontal pipe supports were a saddle-support design constructed of a composite beam that consists of two 2 x 10 wooden members bolted to either side of a 4 x 10 central wooden member. Historically, this composite member was fastened at each end to a 4 x 4 structural column with three 90-degree angle brackets, or clips. There was one clip for each piece of the three-piece composite member. The weight of the circulating water pipe was supported by the composite horizontal support. The horizontal supports that failed were observed to have both of the 2 x 10 members broken and the 4 x 10 beam intact lying on the cooling tower cell distribution deck.

Entergy performed a root cause analysis to identify the causal factors of the failures and to develop corrective actions to restore the cooling tower to operation and prevent recurrence of the pipe support failures. Entergy identified the mechanistic root cause of the pipe support failures in cell 1-1 to be an overload of the FRP-to-wooden horizontal support connection due to an inadequate design of the hardware connecting the wood horizontal member to FRP load bearing columns. Specifically, Entergy changed the design of the connection hardware to include two clips instead of the original three clip design without properly verifying the design.

Following a partial collapse of cooling tower cell 2-4 in August 2007 (NRC Integrated Inspection Report 05000271/2007004), Entergy hired a cooling tower vendor to complete needed repairs to the cooling towers and developed a cooling tower task team to determine the scope of cooling tower repairs, modifications, and procedures that were needed. On February 20, 2008, Entergy concluded that they were going to replace several wood columns in the cooling towers with FRP columns. Entergy contracted with the same vendor to install the FRP columns, but did not contract for any engineering or design work from the vendor. The horizontal saddle supports were to be repaired using wood replacements as necessary. FRP structural replacements were expected to be essentially a direct substitution for the existing wood components and the engineering change package did not require changes to connecting hardware. Therefore, Entergy did not specifically analyze the connections between the FRP columns and the wooden horizontal supports in the engineering design package.

Entergy elected to commence work on the cooling tower repairs prior to completion of engineering change (EC) package 4721 utilizing a work-at-risk process per procedure EN-DC-115, Engineering Change Development. The work-at-risk process allows implementation of an activity requiring engineering authorization prior to the approval of related engineering changes when faced with competing schedule requirements. The work-at-risk process can be implemented if certain criteria are met including: the complexity of activities is such that normal work control processes will provide for proper control of implementation, changes must be recoverable, and final configuration can be verified, tested, and as-built documents can be created. Entergys decision to utilize this process was based primarily on the perceived expertise of the vendor and the belief that experienced workers and supervisors needed little design information to repair the towers properly. Work orders used to perform the repairs did not contain details regarding the connections between the FRP columns and the wooden horizontal supports. When questions arose in the field, vendor engineers or Entergy engineers used engineering judgment to answer them. The workers made several connections between wood and FRP members during the course of repair work. The work was performed by vendor workers under the direction of vendor supervision as skill-of-the-craft. Because there were no detailed work instructions or design drawings for these connections, Entergy field oversight and engineering did not have details needed to verify or question the actual construction. Entergys root cause analysis documents that the work-at-risk process was misapplied or inadequately implemented for a project of this complexity. Corrective actions to revise procedure EN-DC-115 to strengthen controls when using the work-at-risk process for significant projects were included in the analysis.

Following the receipt of design details from Entergys vendor in April and May 2008, Entergy questioned the capacities of certain FRP joint designs and their installation in the field. Entergy performed field inspections that identified some joint configurations were inconsistent with that expected by the vendors engineering staff, specifically, they were not using bushings in all of the joints. The resulting investigation identified that workers were assembling the structure without any design drawings and that the field supervisor did not have much experience with wood to FRP connections. Entergys root cause analysis identifies this as a missed opportunity to identify other configuration issues.

On May 20, 2008, prior to returning the towers to service, Entergy identified a configuration discrepancy on several connections between the FRP columns and the wooden horizontal supports. Specifically, Entergy noticed that the original three clip design to connect the horizontal support beam to the support column was replaced with a two clip design, with the clip that supported the 4 x 10 center beam removed.

Entergy communicated the observation to their vendor. The vendor stated that several alternative configurations were acceptable for the support connection including adding a third clip or strengthening the other two clips with bushings or a larger bolt. Entergy elected to install additional bushings instead of installing the third clip due, in part, to the difficulty of adding a third clip during the late stage of construction. Entergy engineers focused on the capability of the joint hardware to carry load and did not recognize that the 3 member composite support beam would not act as a composite if the third clip was not present. Without the third clip, loads would be transferred from the pipe to the two 2 x 10 members only. Because neither Entergy nor the vendor possessed original design documentation describing the configuration of the connections, and Entergy relied on the expertise of the vendor, engineers did not question the change from three clips to two clips. Furthermore, the vendor was not aware that the horizontal member supporting the pipe was a three piece composite; the vendor traditionally uses solid horizontal members in this design. Entergys root cause analysis identifies these organizational interface and configuration change documentation issues as contributing causes to the event. Corrective actions include developing current design drawings, investigating improved structural connections for the joints, and to review procedures and conduct training related to contractor training and qualifications, vendor communications, and vendor use of the corrective action program.

As a part of Entergys root cause, Entergy hired an additional vendor to determine the adequacy of the two clip design. The vendors calculation concluded that the two-clip design was inadequate and could lead to failure.

The inspectors identified a performance deficiency in that Entergy did not verify the adequacy of the changes to the design of the cooling tower structural connections between the FRP columns and wooden horizontal supports. As per EN-DC-115, Engineering Change Development, and EN-DC-116, Engineering Change Installation, Entergy was responsible for ensuring that the appropriate drawings and documents are provided to adequately implement the design change. Furthermore, engineers were responsible to adequately resolve technical issues that were identified during the design change.

Analysis.

The finding was more than minor because it was associated with the Design Control attribute of the Initiating Events Cornerstone and affected the Cornerstones objective of limiting the likelihood of those events that upset plant stability and challenge critical safety functions during shutdown as well as power operations. Specifically, the inadequate design change analysis resulted in the subsequent failure of pipe supports in cooling tower cell 1-1, damage to the cooling tower, and damage to the circulating water system piping. The damage caused operators to reduce plant power to approximately 46 percent power. The significance of the finding was determined using Inspection Manual Chapter Attachment 0609.04, Phase 1 - Initial Screening and Characterization of Findings. The inspectors determined that the finding was of very low safety significance (Green) because the finding did not contribute to both the likelihood of a reactor trip and the likelihood that mitigation equipment or functions will not be available.

The finding had a cross-cutting aspect related to resources in the area of Human Performance. Entergy did not ensure that complete, accurate and up-to-date design documentation was available to adequately construct portions of the non-safety-related cooling tower cells. Specifically, Entergy did not provide detailed drawings or instructions supported by engineering calculations to implement a design change affecting the circulating water pipe horizontal support design. H.2(c)

Enforcement.

No violation of regulatory requirements occurred because the event involved non-safety-related systems. Entergy entered this condition into their corrective action program (CR-2008-2904). Corrective actions identified in the root cause report included inspection of the cooling tower piping and structural components for deficiencies and repairing or replacing components as necessary; revising engineering procedures to strengthen controls when using the work-at-risk clause for significant projects; revising the operations inspection procedure to include verification of proper saddle support configuration including the three clip design; and developing measures to ensure contractor workers are aware of the expectation to write condition reports for any configuration issues. Since this issue is of very low safety significance and has been entered into Entergys corrective action program, it is classified as a finding. (FIN 05000271/2008009-001, Inadequate Design Change Review Causes Failure of Circulating Water System Pipe Supports)

.10 Review and Understand the Failure Mode for the Pipe Support in Cell 1-1.

a. Inspection Scope

The inspectors reviewed the licensees final mechanistic root cause determination to determine whether the failure mode was appropriately identified.

b. Findings and Observations

No findings of significance were identified. The licensees analysis identified the mechanistic root cause of the pipe support failures to be an overload of the FRP to wooden horizontal support connection due to an inadequately performed design analysis. Specifically, the overload condition was due to missing support clips on the FRP support columns that transferred load from the pipe support beam to the vertical FRP column. This conclusion was supported by engineering analysis by a vendor that determined that the two-clip configuration was inadequate and could lead to failure.

.11 Evaluate the Thoroughness of Entergys Extent-of-Condition Review.

a. Inspection Scope

The team reviewed Entergys plan for examining the extent of condition of the apparent cause of the pipe support failures. Entergy determined, following initial examination of the pipe support failures, that the probable cause of the failure was missing support clips on the FRP columns. As such, Entergy planned detailed examinations of all support columns in both cooling towers to verify that proper support hardware was installed. In addition, Entergy examined the accessible portions of the horizontal supports, including their members, joints, and fasteners, as well as structural columns and cross members for other signs of degradation. Entergy compared in field configurations with design drawings to verify proper configuration.

b. Findings and Observations

No findings of significance were identified. Although the inspection team identified several minor deficiencies that Entergy did not identify during their inspections, the team determined that Entergys extent-of-condition review was effective and of appropriate scope and depth. Entergys extent-of-condition review identified several minor material and hardware deficiencies and a number of deficiencies (cracking in wood members) requiring replacement of several wood members. The team verified that Entergy immediately addressed the issues identified in cells 2-3 and 2-4 and determined that the actions were adequate and appropriate. In addition, the team verified that all deficiencies identified in cells 2-1 and 2-2 were addressed as described in Section 4OA3.6.

.12 Evaluate Entergys Cooling Tower Inspection Program to Determine Whether the

Deficient Conditions Associated With the July 11, 2008, Event Should Have Been Identified Earlier Through the Licensees Implementation of the Cooling Tower Inspection Program.

a. Inspection Scope

The team reviewed Entergys procedures that describe periodic inspections of the cooling towers to determine if Entergy staff could have identified the deficiencies that caused the failure of the pipe supports in cell 1-1 prior to the actual failure.

b. Findings and Observations

No findings of significance were identified. The team observed that Entergys cooling tower inspection procedures were designed to evaluate the as-found material condition of the cooling towers and implement corrective actions as appropriate. The inspection procedures were not designed to review or verify any design modifications that are implemented. As such, the team concluded that it was not reasonable to expect the inspection procedure to have identified the design deficiency that caused the pipe support failures. Entergys design change program implements procedures designed to require reviews of design changes prior to implementation to verify the adequacy of the new designs and identify any potential problems. Entergys root cause analysis determined that inadequate implementation of the design change process, not poor inspections or inspection procedures, ultimately resulted in the failure of the pipe supports.

.13 Evaluate the Licensees Analysis of the Current Condition of the Horizontal Pipe

Supports in Cells 2-1 and 2-2.

a. Inspection Scope

The inspectors reviewed the design inputs, assumptions, loading considerations, loading combinations and acceptance criteria in used in Calculation YVC-3075 to determine whether the design calculation supported the as left condition of cells 2-1 and 2-2. In addition, the inspectors reviewed calculations supporting the use of steel channel instead of wood as the horizontal pipe support member in bent 1 of cell 2-1.

b. Findings and Observations

No findings of significance were identified. The current condition of the horizontal pipe support header beams in cells 2-1 and 2-2 were verified as meeting the original design specifications by Calculation VYC-3075. The calculation evaluation was based on the use of materials (timber, bolting, clip supports) and structural configurations and fabrications which meet or exceed the original specification. The inspectors reviewed the design inputs, assumptions, loading considerations, loading combinations and acceptance criteria. The inspectors concluded that the design of the pipe supports in cells 2-1 and 2-2 were adequate to provide support to the circulating water pipe under all postulated design conditions.

.14 Identify Any Potential Generic Issues That May Require Follow-up Action.

No potential generic issues were identified. Entergy did not verify the technical adequacy of design changes prior to placing the circulating water system piping in cooling tower cell 1-1 in service.

.15 Evaluate Entergy's Information Associated With Their Plans to Modify Cells 2-1 and 2-2

During the Upcoming Refueling Outage, Including Schedule, 50.59 Reviews Completed, and Available Design Documents. Provide a Proposal For a Follow-up Inspection to Verify the Acceptability of the Licensee's Modification to Cells 2-1 and 2-2 to be Completed Prior to Entergys Implementation of the Modifications.

Entergy informed the inspectors that an FRP modification to cells 2-1 and 2-2 will not be implemented during the Fall 2008, refueling outage. The NRC will implement the appropriate baseline inspection activity if and when Entergy prepares an FRP modification package.

.16 Independently Review the Leak That Occurred From a Slip-joint on the Circulating

Water Header in Cooling Tower Cells 1-3 and 1-4 on September 16, 2008, to Determine if the Leak was Related to Structural Defects or Failures.

a. Inspection Scope

The inspectors interviewed Entergy staff, inspected the condition of the circulating water pipe and cooling tower structures in cells 1-3 and 1-4, and reviewed Entergys evaluation of inspection activities in cells 1-3 and 1-4 to determine if structural defects caused the leaks observed on September 16, 2008.

b. Findings and Observations

Entergy determined that the leaks in cells 1-3 and 1-4 were not related to structural defects in the cooling tower. Entergys review identified that packing material used to mitigate small leaks in the circulating water pipe joints dislodged, allowing existing leaks to recur. Entergy inspected the circulating water piping, cooling tower water deck and cooling tower fan deck in cells 1-3 and 1-4. Entergy determined there was no sagging in the water deck or fan decks and no deflection in the circulating water piping that would indicate failure of the cooling towers structural members. Entergy performed complete inspections of the structural members in cell 1-3. Entergy identified a number of wood members that were degraded including one column piece that was split and failed and others with lesser defects such as splits at the ends of members, localized crushing at or around knots, and at scarf joints. Entergy replaced, scheduled repairs for, or marked for future inspection all identified deficiencies in accordance with their cooling tower inspection procedure. Entergy concluded that these degradations were typical of what was found in previous cooling tower inspections and did not cause the leakage observed in the circulating water piping.

The inspectors concluded that the packing extrusion was likely not related to a structural problem in cell 1-3. The inspectors observed that the circulating water pipe and surrounding structural materials did not appear deflected or otherwise damaged.

The inspectors independently observed the failed and degraded members identified by Entergy during follow-up inspections and concluded that, although there were two members with significantly degraded capability to carry load, the cooling tower structure remained capable of performing its function. The inspectors also noted that the wood members that were degraded exhibited typical degradation modes for wood and no new failure mechanisms were identified. The magnitude of the leakage observed did not affect the structural integrity of the cooling tower and did not affect the ability of the cooling tower to perform its cooling function. The circulating water piping joints in the safety-related and seismically designed cells (cells 2-1 and 2-2) were not impacted by loss of packing material because there was no leakage identified from those joints and therefore they did not require packing material. Furthermore, the ACS piping in cell 2-1 is carbon steel pipe with welded joints. The ACS pipe does not have slip joints that could leak and require packing material to be inserted for leak mitigation.

.17 Determine the Applicability of Identified Failed Structural Components to the Previous

Conclusion on the Operability of Safety-Related and Seismically Designed Cells 2-1 and 2-2.

a. Inspection Scope

The inspectors reviewed Entergys inspection findings and cooling tower functionality assessment related to the circulating water piping leak and resultant cooling tower structural inspections that occurred on September 16, 2008. The inspectors also interviewed plant personnel, walked down portions of cooling tower cell 1-3, observed degraded structural members that were removed by Entergy, and reviewed documents related to the structural design, and inspection of the cooling towers.

b. Findings and Observations

Entergy performed a 100 percent inspection of cell 1-3 structural members above the waterline to determine if structural damage may have caused the circulating water system leaks observed on September 16. Entergy used procedure OP 52114, Cooling Tower Structure and Repair, to inspect members and joints, classify degraded components, and repair or replace damaged members. Entergy identified eight items for immediate repair or replacement, one of which was a column that was split and failed and another that was split such that its individual load carrying capability was reduced significantly. The remaining six items were replaced not because imminent failure of the individual items was likely but for opportunity to repair a degraded member while the cooling tower was out of service. Five additional items were identified as having minor degradations such that they could be repaired during the next cooling tower seasonal outage (fall 2008 through spring 2009). These minor degradations were characterized as having very localized degradation (e.g., crushing or splits) and they would remain load bearing and capable of transferring loads. One item was identified as requiring additional monitoring during future inspection, that is, the defect identified was minor and did not require replacement but warranted future attention to detect further degradation.

The inspectors observed the degraded members requiring replacement and two of the members requiring replacement during the next maintenance opportunity. The inspectors observed that the wood members exhibiting the most severe degradation appeared to be old. The inspectors also observed that the members that were replaced exhibited degradation modes common to aging wood, that is, splits and localized crushing near knots. The inspectors concluded that the degradation and failures observed were typical of aging wood members.

The inspectors reviewed Entergys cooling tower inspection program, maintenance history of the cooling towers, and cooling tower design to determine if the operability of safety-related and seismically designed cooling tower cells, 2-1 and 2-2 respectively, was adversely affected. The design of cells 2-1 and 2-2 included, in some areas, approximately twice the amount of diagonal bracing members in longitudinal sections of the tower cells as the non-safety-related portions of the cooling tower as well as more robust transverse bracing between columns B and C which carries much of the load directly underneath the circulating water piping. Additionally, much of the bracing in cells 2-1 and 2-2 is more robust than in the non-safety-related cells, that is, they utilize 4 x 4 and 4 x 6 members instead of 2 x 4 members for transverse and longitudinal bracing.

Following the partial collapse of cooling tower cell 2-4 in August 2007, Entergy implemented a long term project to systematically upgrade both cooling towers by replacing key structural members in the non-safety-related portions of the tower and increased inspections and member replacements in cells 2-1 and 2-2. The project includes replacement of portions of the non-safety-related cell structure with FRP columns and pipe supports to improve strength and eliminate failure mechanisms associated with aging wood. It also includes weekly, annual, and opportunity-based inspections of the cooling tower structural material. To date, Entergy inspected approximately 64 percent of the columns within the fill area in cooling tower 1 and replaced several of those columns (approximately 36 percent of total columns).

Likewise, in cooling tower 2, Entergy inspected 70 percent of the columns within the fill area and replaced 41 percent of the total number of columns. The columns within the fill area are considered inaccessible with respect to inspections. Inspections in this area are possible only when the cooling towers are out of service and significant work is done to remove obstacles to allow personnel access. All other structural members can be inspected without significant preparation and are done so on at least an annual basis. During the Spring 2008 maintenance period, Entergy inspected 100 percent of the columns in the normally inaccessible fill area of cell 2-1 and 50 percent of the columns in cell 2-2. All observed degradations were repaired or replaced during the maintenance outage. In addition to repairs made during the Spring 2008 maintenance outage and following the July 11 pipe support failures, Entergy plans to replace 100 percent of the structural members in cell 2-1, excluding the relatively new partition walls, over the next three refueling outages. Entergy plans to perform inspections of 100 percent of the normally inaccessible columns during each of the next three refueling outages. Entergy also plans to replace the wood members in cell 2-2 as a part of the long-term project but has not developed specific project schedules to do so. The inspectors concluded that, because Entergy did not have a robust cooling tower inspection program as documented in the root cause report for the August 2007 partial collapse of cell 2-4 (CR-VTY-2007-3243), and that portions of the non-safety-related cooling tower cells have yet to be inspected, it was not unexpected that Entergy found some degraded wood members in the inspections performed on September 16 and 17.

Furthermore, it is reasonable to conclude that Entergy will find additional degraded and failed members in future inspections. The inspectors concluded that Entergys scope of efforts to replace key load bearing members with FRP, improve inspection procedures and processes, and align cooling tower inspections with industry standards and best practices were adequate to improve the structural reliability of the non-safety-related portions of the cooling towers.

The inspectors concluded that the degraded structural members identified during inspections on September 17 did not represent information that rendered cells 2-1 and 2-2, or the ACS, inoperable. The inspectors came to this conclusion based on the following: the identification of degraded members in non-safety-related cells was not unexpected because all members have not yet been inspected in those cells and historical inspection programs at the site were identified as being weak; the failed members appearance indicated that they were old and did not recently fail; new inspection programs have been implemented at the site that adequately identify and repair degraded members at a frequency compliant with industry standards; 100 percent of the structural members in safety-related cell 2-1 were inspected recently and several of those members were repaired or replaced; Entergy will perform inspections of 100 percent of the normally inaccessible structural members in cell 2-1 in each of the next three refueling outages; one-third of the structural members in cell 2-1 will be replaced with new wood members in each of the next three refueling outages; the design of cells 2-1 and 2-2 is more robust than the non-safety-related structure and has more capacity to distribute load to intact members if other members should fail.

4OA6 Meetings, including Exit

Exit Meeting Summary

On October 8, 2008, the inspectors presented the inspection results to Mr. Theodore Sullivan, Site Vice President, and other members of the VY staff. The inspectors confirmed that no proprietary information was provided or examined during the inspection.

ATTACHMENTS:

(1) Supplemental Information

(2) Special Team Inspection Charter, Revision 2, Vermont Yankee Nuclear Power Station

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Entergy Personnel

S. Buckley

Design Engineer

S. Goodwin

Design Engineering Supervisor

D. Grimes

Design Engineer

D. Mannai

Licensing Manager

N. Rademacher

Director of Engineering

J. Rogers

Design Engineering Manager

LIST OF ITEMS OPENED, CLOSED, AND DISCUSSED

Opened and Closed

05000271/2008009-001 FIN Inadequate Design Review Causes Failure of Circulating Water System Pipe Supports (Section 4OA3.9)