IR 05000369/2007009

IR 05000369-07-009 and 05000370-07-009, on 09/11/07-09/18/07; McGuire Nuclear Station, Units 1 and 2; Special Inspection

ML072820165
Person / Time
Site:	Mcguire, McGuire
Issue date:	10/09/2007
From:	Moorman J NRC/RGN-II/DRP/RPB1
To:	Gordon Peterson Duke Energy Carolinas, Duke Power Co
References
IR-07-009
Download: ML072820165 (20)

Text

ber 9, 2007

SUBJECT:

MCGUIRE NUCLEAR STATION - SPECIAL INSPECTION REPORT 05000369/2007009 AND 05000370/2007009

Dear Mr. Peterson:

On September 18, 2007, the U.S. Nuclear Regulatory Commission (NRC) completed a Special Inspection at your McGuire Nuclear Station Units 1 and 2. The enclosed inspection report documents the inspection results, which were discussed at the exit meeting on September 18, 2007, with you and members of your staff.

The determination that the Special Inspection would be conducted was made by the NRC on September 4, 2007. This determination was based on the risk and deterministic criteria specified in NRC Management Directive 8.3, NRC Incident Investigation Program. The Special Inspection Team (SIT) was dispatched to the site on September 11, 2007. The inspection was conducted in accordance with Inspection Procedure 93812, Special Inspection.

The purpose of this inspection was to inspect and assess the circumstances associated with the discovery of misaligned oil cooler endbells on the Unit 1 Train A, Unit 1 Train B, and Unit 2 Train B centrifugal charging/high head safety injection pumps. The inspection focus areas are detailed in the Special Inspection Team Charter (Attachment).

The inspection examined activities conducted under your license as they relate to safety and compliance with the Commissions rules and regulations and with the conditions of your license.

The inspectors reviewed selected procedures and records, observed activities, and interviewed personnel.

Based on the results of this inspection, no findings of significance were identified. A licensee-identified violation, which was determined to be of very low safety significance (Green), is listed in this report. The NRC is treating this violation as a non-cited violation (NCV) consistent with Section VI.A.1 of the NRC Enforcement Policy. If you contest the NCV in this report, you should provide a written response within 30 days of the date of this inspection report, with the basis for your denial, to the Nuclear Regulatory Commission, ATTN: Document Control Desk, Washington, DC, 20555-0001; with copies to the Regional Administrator Region II; the Director, Office of Enforcement, United States Nuclear Regulatory Commission, Washington, DC, 20555-0001; and the NRC Senior Resident Inspector at the McGuire Nuclear Station.

DEC 2 In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter and its enclosure will be available electronically for public inspection in the NRC Public Document Room or from the Publicly Available Records (PARS) component of NRCs document system (ADAMS). ADAMS is accessible from the NRC Web site at www.nrc.gov/reading-rm/adams.html (the Public Electronic Reading Room).

Sincerely,

/RA/

James H. Moorman, III, Chief Reactor Projects Branch 1 Division of Reactor Projects Docket Nos.: 50-369, 50-370 License Nos.: NPF-9, NPF-17

Enclosure:

Special Inspection Report 05000369/2007009 and 05000370/2007009 w/Attachment: Supplemental Information

REGION II==

Docket Nos: 50-369, 50-370 License Nos: NPF-9, NPF-17 Report No: 05000369/2007009, 05000370/2007009 Licensee: Duke Power Company LLC Facility: McGuire Nuclear Station, Units 1 and 2 Location: Huntersville, NC 28078 Dates: September 11-18, 2007 Inspectors: A. Hutto, Resident Inspector (Lead Inspector)

C. Peabody, Reactor Inspector (Region II, DRS)

Approved by: James H. Moorman, III, Chief Reactor Projects Branch 1 Division of Reactor Projects Enclosure

SUMMARY OF FINDINGS

IR 05000369/2007-009, 05000370/2007-009; 09/11/07-09/18/07; McGuire Nuclear

Station, Units 1 and 2; Special Inspection.

This inspection was conducted by a team consisting of an inspector from the NRCs Region II office and a resident inspector from the Oconee Nuclear Station. The NRC's program for overseeing the safe operation of commercial nuclear power reactors is described in NUREG-1649, Reactor Oversight Process, Revision 4, dated December 2006. A Special Inspection Team was established in accordance with NRC Management Directive 8.3, "NRC Incident Investigation Program" and implemented using Inspection Procedure 93812, Special Inspection.

NRC-Identified and Self-Revealing Findings

None.

Licensee-Identified Violations

One violation of very low safety significance, which was identified by the licensee, has been reviewed by the inspectors. Corrective actions taken or planned by the licensee have been entered into the licensees corrective action program. This violation and corrective action tracking number is listed in Section 4OA7 of this report.

REPORT DETAILS

Summary of Plant Events On September 4, 2007, the licensee discovered during routine maintenance on the Unit 2 Train B centrifugal charging/high head safety injection (NV) pump that the bearing oil cooler reversing endbell was incorrectly rotated 45 degrees from its design configuration. The cooler in question was an American Standard Type SSCF four-pass heat exchanger and was supplied by the pump vendor as part of an integrated skid-mounted pump package. Proper endbell orientation is necessary to ensure the four-pass flow path is utilized. Misalignment results in bypassing two of the flow paths and therefore reduces the cooling capability of the heat exchanger. The licensee inspected other pumps with similar oil coolers and found that both Unit 1 NV pumps had oil coolers with reversing endbells rotated 90 degrees from their design configuration.

There are two coolers on each pump, a bearing oil cooler and a speed reducer oil cooler. The Unit 1 Train A NV pump had only the speed reducer oil cooler reversing endbell rotated. The Unit 1 Train B NV pump had both the speed reducer and bearing oil cooler reversing endbells rotated. The licensee declared both trains of the Emergency Core Cooling System (ECCS) inoperable based on the reduced capacity of the coolers and entered Technical Specification (TS) 3.0.3. The licensee was able to the correct the misaligned endbells, restore the pumps to operable status, and exit the TS prior initiating a shutdown of Unit 1.

Inspection Scope Based on the probabilistic risk and deterministic criteria specified in Management Directive 8.3, NRC Incident Investigation Program, Inspection Procedure 71153, Event Follow-up, and the significance of the operational events which occurred, a Special Inspection was initiated in accordance with Inspection Procedure 93812, Special Inspection. The inspection focus areas included the following charter items:

• Develop a time line of events, including applicable management decision points from original installation of the coolers through restoration from the event.

• Review the licensees corrective action documents (PIPs) on this issue and assess whether the licensee knew about this problem prior to the event. Assess any corrective action the licensee took as a result of any previous PIPs to determine whether the actions were adequate and timely.

• Assess the ability of the centrifugal charging pump oil coolers to provide adequate cooling to the pump bearings and speed changer during accident conditions, for the various positions the endbells were found in, and for the various Lake Norman and nuclear service water pond water temperatures during the period of time that the problem has existed.

• Assess the extent of the maintenance and engineering organization contribution to the performance deficiency. Assess the extent of the condition as it relates to the oil bearing coolers on Safety Injection pumps.

• Assess the extent of condition at the Catawba Nuclear Station. Determine any differences in system operation between the two sites and provide an assessment.

• Assess the previous performance of the coolers by reviewing historical documents such as bearing temperature and oil sample data, inspection and maintenance procedure results, and maintenance rule applications.

• Collect data necessary to develop and assess the safety significance of any findings in accordance with Inspection Manual Chapter 0609, Significance Determination Process.

• Identify any potential generic safety issues and make recommendations for appropriate follow-up actions (e.g., Information Notices, Generic Letters, Bulletins).

4OA5 Special Inspection

.1 Develop a time line of events, including applicable management decision points from

original installation of the coolers through restoration from the event.

a. Inspection Scope

The team identified the events that have occurred at McGuire and documented the specific events and the corrective actions that were implemented as a result in chronological order. In order to develop this sequence of events, the inspection team reviewed corrective action documents, control room logs, the NV pump maintenance history, receipt inspection documents, and available pre-operational test reports. The inspection team also interviewed several licensee staff members in the Engineering and Maintenance departments in order to validate and further establish the sequence of events documented in this report.

b. Findings and Observations

• 07-08/74, Unit 1 and 2 NV pumps were received on site. The coolers were included as part of the skid mounted pump package. No evidence from receipt inspection documentation of end bell removal.

• 07-12/78, Unit 1 NV System pre-operational functional testing. No issues identified with cooler performance.

• 01/83, Unit 2 NV System pre-operational flow balance functional testing. No issues identified with cooler performance.

• 12/81, Unit 1 begins commercial operation.

• 03/84, Unit 2 begins commercial operation.

• 12/86, First documentation of routine NV pump cooler cleaning preventive maintenance (PM) being implemented (MP/0/A/7150/058, American Standard Oil Coolers Corrective Maintenance)

• 12/86 - 04/04, Multiple work orders to perform routine cleaning and corrective maintenance on the NV pump oil coolers where the endbells are removed. There are no specific instructions in the maintenance procedure for assuring correct endbell orientation.

• 12/12/03, North Anna operating experience (OE) event date

• 02/18/04, McGuire PIP (M-04-0806) is written to evaluate the North Anna OE. No verification of correct endbell orientation was performed for existing equipment configuration.

• 03/08/04, Docutracks initiated for procedure change to MP/0/A/7150/058 to add steps to matchmark endbells as a measure to ensure correct reassembly.

• 04/12/04, Procedure change implemented.

• 04/04 - 9/07, Cooler cleaning PMs continue utilizing the changed procedure.

• 09/04/07, The licensee misalignment (45 degrees) of the endbell discovered on the Unit 2 Train B NV pump bearing oil cooler by maintenance personnel during routine PMs. Extent of condition reveals that the Unit 1 Train A NV pump speed reducer cooler, the Unit 1 Train B NV pump bearing oil cooler, and the Unit 1 Train B NV pump speed reducer endbells are misaligned by 90 degrees. TS 3.0.3 entered at 1705.

• 09/04/07, The Unit 1 Train B NV bearing oil cooler and speed reducer oil cooler endbells are properly installed and TS 3.0.3 exited.

.2 Review the licensees corrective action documents (PIPs) on this issue and assess

whether the licensee knew about this problem prior to the event. Assess any corrective action the licensee took as a result of any previous PIPs to determine whether the actions were adequate and timely.

a. Inspection Scope

The inspection team performed corrective action analysis searches of the licensees PIP database targeting PIPs associated with NV and Safety Injection (NI) pump oil coolers and heat exchangers, reviewed the licensees extent of condition review for this issue, and held discussions with engineering and maintenance personnel to determine if there were prior occurrences of this issue and whether the licensee took prior appropriate corrective actions in response to the issues.

b. Findings and Observations

No findings of significance were identified. The team did not identify any additional PIPs in the licensees corrective action database that documented misaligned oil cooler endbells. The team identified two PIPs which documented increased operating temperatures associated with the Unit 1 Train B NV and Unit 1 Train A NV pumps respectively. PIP M-02-5457 documented an increase in the Unit 1 Train B NV pump bearing and the speed reducer gearbox oil temperatures during October 2002. This pump had both oil coolers misaligned at 90 degrees and this condition would have contributed to the observed temperature increase. PIP M-06-4643, documented an increase in the Unit 1 Train A pump NV speed reducer gearbox oil temperatures during October 2006. Again, this cooler was one of the affected coolers and likely contributed to the temperature increase. However, the McGuire Nuclear Station experiences a phenomenon of seasonal service water fouling due to changes in iron valences with anoxic conditions that occurs during the fall months. The licensee concluded that the observed temperature increases, which were still within acceptable limits, were attributed to seasonal fouling which was consistent with previous trends. As a result, the inspection team concluded that these conditions did not present a reasonable opportunity for the licensee to identify and correct the misaligned endbells.

.3 Assess the ability of the centrifugal charging pump oil coolers to provide adequate

cooling to the pump bearings and speed changer during accident conditions, for the various positions the endbells were found in, and for the various Lake Norman and nuclear service water (RN) pond water temperatures during the period of time that the problem has existed.

a. Inspection Scope

The team reviewed the past operability analysis for the gearbox and bearing lube oil heat exchanger operation with the as-found improper positioning of the reversing endbells to verify that accident condition heat removal capability had been maintained.

The team compared the past operability heat removal calculations to the required accident heat load as well as manufacturers specifications to verify that the computer code used in the calculation modeled the system accurately and conservatively.

b. Findings and Observations

No findings of significance were identified. The team determined that the licensee's operabiltiy calculation adequately demonstrated that the pumps were capable of performing their safety function under worst case accident conditions with the degraded oil cooling capability. Additional margin was available as the worst case RN temperature used in the calculation of 95 degrees was conservative, as the actual historical high temperatures for the lake and service water impoundment, which are the RN sources, were significantly lower (69 and 82 degrees respectively).

.4 Assess the extent of the maintenance and engineering organization contribution to the

performance deficiency. Assess the extent of the condition as it relates to the oil bearing coolers on Safety Injection pumps.

a. Inspection Scope

The inspection team reviewed corrective action documents, reviewed the maintenance history of the NV pumps, reviewed the licensees extent of condition, interviewed engineering and maintenance personnel, reviewed maintenance heat exchanger team interview notes of the event, and performed walkdowns of the safety injection pumps to assess the extent to which the maintenance and engineering organizations contributed to the performance deficiency, and to assess the extent of condition as it relates to the oil bearing coolers on the NI pumps, as these pumps utilize the same make and type oil coolers as the NV pumps.

b. Findings and Observations

No findings of significance were identified. The inspection team found that the misalignment of the endbells probably occurred as a result of previous maintenance of these coolers that had been performed since the early operating history of the plant.

The licensee has performed PM procedure MP/0/A/7150/058, NV Pump American Standard Oil Coolers Corrective Maintenance multiple times for each cooler since initial plant startup to clean and inspect the coolers. This procedure, which requires disassembly of the endbells, did not provide instructions for correct reassembly during alignment, and instead relied on the skill-of-the-craft until April 2004.

In February 2004, PIP M-04-0806 was written for engineering to evaluate a North Anna OE item that documented misaligned endbells associated with their charging pump speed increaser oil coolers, as it was determined to have applicability to the McGuire NI and NV pump oil coolers. Engineering did not perform a walkdown of the applicable pumps or evaluate applicable pertinent data to determine if the cooler endbells were assembled correctly. The assumption was that they were assembled in the correct configuration at the time of the OE. The inspectors determined that this represented a missed opportunity to identify and correct the misaligned endbells. The only corrective action specified by engineering was to add steps to the maintenance procedure to ensure that the endbells were correctly installed; however, no specific guidance was given to the procedure writers for implementation. The procedure change that was ultimately made was to matchmark the head to the cooler prior to disassembly and use the matchmark to align the head for reassembly. This corrective action only preserved the misaligned endbell configuration during subsequent maintenance as it did not contain guidance or instructions to ensure the proper endbell assembly. Additionally, the heat exchanger team had been using this practice as a skill-of-the-craft technique during prior maintenance, and the North Anna OE stated that matchmarking was used at the site and was not effective in ensuring that the endbells are correctly installed.

There was no check by engineering of the implemented procedure change to assess the adequacy of the change. The inspectors determined that engineerings assessment of the North Anna OE and the associated corrective actions were inadequate. This performance deficiency is documented in Section 4OA7 as a licensee-identified violation, as the condition was ultimately identified by a maintenance technician on September 4, 2007, during routine PMs of the Unit 2 Train B pump bearing oil cooler, when the technician noticed that the head gasket did not appear to have correct fitup with the tube sheet. Had the technician not made this observation and contacted engineering, the misaligned endbells would not have been identified and the condition would still exist.

.5 Assess the extent of condition at the Catawba Nuclear Station. Determine any

differences in system operation between the two sites and provide an assessment

a. Inspection Scope

As a result of the endbell misalignment issue at McGuire, the Catawba Nuclear Station (CNS) initiated PIP C-07-4662 to evaluate the extent of condition of this problem at Catawba as they have the same oil coolers installed for their charging and safety injection pumps. The inspection team reviewed PIP C-07-4662, CNSs operability calculation for the Unit 1 Train A NI pump bearing oil cooler, CNS maintenance history for these pumps and held discussions with the CNS Senior Resident Inspector to assess the extent of condition at CNS and to determine any differences in system operation between the two sites that may be significant.

b. Findings and Observations

No findings of significance were identified. CNS identified that the Unit 1 Train A NI pump bearing oil cooler endbell was misaligned by 45 degrees. This was the only oil cooler endbell of the 12 at CNS that was misaligned. A significant design difference between CNS and McGuire was that component cooling water, which was a treated closed system, was used as the cooling medium verses RN which is used at McGuire.

As such, fouling was not a problem at CNS. Therefore PMs to clean and inspect the coolers were not required. The Unit 1 Train A NI bearing oil cooler had never had its endbell removed during its operational history and it was concluded that the condition existed from the time of receipt of the pump. McGuires use of service water provided more cooling margin, as the lake water which is the source, was cooler than component cooling water. The inspection team compared CNSs operability calculation with McGuires and found the calculations to be consistent with respect to methodology and key assumptions. Both operability evaluations ultimately determined that the affected pumps remained operable under worst case design conditions.

.6 Assess the previous performance of the coolers by reviewing historical documents such

as bearing temperature and oil sample data, inspection and maintenance procedure results, and maintenance rule applications

a. Inspection Scope

The inspection team reviewed various NV pump bearing temperature trend data, cooler performance data and testing, oil analysis reports, and machinery history to determine if any indications of improper orientation of the reversing endbells existed and to determine if the licensee had overlooked indication of the problem prior to discovery on September 4, 2007. The team also reviewed corrective and preventative maintenance records that involved disassembly of the endbells for cooler cleaning as well as work orders and corrective actions to identify any previous opportunities to identify the offset in the endbells.

b. Findings and Observations

No findings of significance were identified. The inspection team found that due to the oil cooler over capacity, oil and bearing temperatures associated with the misaligned endbells stayed within operational limits and, in general, were not out of line with the other correctly configured oil coolers. Oil analysis reports also did not indicate any appreciable reduction in oil viscosity which would indicate an over heating condition.

Corrosion and wear products were also reported to be in the normal range. Additionally, nuclear service water flow and cooler differential pressure for the coolers with the misaligned endbells were consistent with coolers that had correct endbell orientations.

As part of the licensees investigation of the event, the system engineer performed a review of historical trends for data captured by the plant operational aid computer (OAC)for the NV pump thrust, and inboard and outboard radial bearing temperatures and identified a maintenance cooler cleaning that likely resulted in an endbell misalignment for the Unit 1 Train B NV pump bearing oil cooler. This maintenance occurred in May 1993, when the oil cooler was disassembled, cleaned and reassembled. Following the cleaning, it would be expected that the bearing temperatures would be lower as no other maintenance was performed on the pump. However, all three bearing temperatures increased from 11.5 to 15.8 degrees, despite service water increasing only 6.4 degrees.

There was no evidence that this condition was captured in the licensees corrective action program and evaluated. This example supports the conclusion that the endbell misalignments occurred during the extensive maintenance history of these coolers.

4OA6 Management Meetings

On September 18, 2007, the inspectors presented the inspection results to Mr. G.

Peterson, Site Vice President, and other members of his staff, who acknowledged the findings. The inspectors confirmed that any proprietary information that was provided or examined during the inspection was returned to the licensee.

4OA7 Licensee-Identified Violations

The following violation of very low safety significance (Green) was identified by the licensee and is a violation of NRC requirements which meets the criteria of Section VI of the NRC Enforcement Policy, NUREG-1600, for being dispositioned as an NCV.

• 10 CFR 50 Appendix B, Criterion 16 requires that measures shall be established to assure that conditions adverse to quality are promptly identified and corrected.

Contrary to the above, the licensee failed to identify and correct misaligned charging pump bearing and speed reducer gearbox oil coolers following the evaluation of industry operating experience of the same issue in 2004. Additionally, the corrective action that was implemented from the OE to make a maintenance procedure change to ensure that the cooler endbells were properly installed following removal was not adequate as subsequent performance of the procedure resulted in the continued misaligned condition. The condition was eventually discovered by maintenance personnel on September 4, 2007 during routine maintenance of the Unit 1 Train B NV pump speed reducer oil cooler. The risk was determined to be of very low safety significance as the licensee demonstrated through their operability calculation that the charging pumps would have been able to perform their safety function under worst case accident conditions.

ATTACHMENT:

SUPPLEMENTAL INFORMATION

KEY POINTS OF CONTACT

Licensee personnel

Ashe, K., Manager, Regulatory Compliance

Ballard, S., Primary systems Engineering Supervisor

Bare, B., Balance of Plant Engineer

Boyle, J., Manager, Modification Engineering

Bradshaw, S., Superintendent, Plant Operations

Crane, K., Licensing Specialist

Evans, K., Superintendent, Maintenance

Hashim, K., Maintenance

Jenkins, J., Safety Review Group Manager

Kammer, J., Manager, Safety Assurance

Meyer, B., Primary System Engineer

Parker, R., Superintendent, Work Control

Pattison, G., Heat Exchanger Engineer

Peterson, G., Site Vice President, McGuire Nuclear Station

Repko, R., Station Manager

Rosenau, S., Rotating Equipment Pump Engineer

Schuerger, P., Acting Manager, MCE Engineering

Smith, J., Rotating Equipment Pump Engineer

NRC personnel

Moorman, III, J., Chief, Reactor Projects Branch 1

Brady, J., Senior Resident Inspector

LIST OF ITEMS

OPENED AND CLOSED None