05000461/LER-2001-001

LER-2001-001,

Event date:
Report date:
Reporting criterion:	10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications
4612001001R00 - NRC Website

FACILITY NAME (1) PAGE (3) LER NUMBER (6) DOCKET (2) Clinton Power Station 05000461 2001 - 001 00

DESCRIPTION OF EVENT

On January 12, 2001, the plant was in Mode 1 (POWER OPERATION) at about 100 percent reactor power. At 1930 hours0.0223 days <br />0.536 hours <br />0.00319 weeks <br />7.34365e-4 months <br />, the Division 2 emergency diesel generator [DG] was started and loaded for the purpose of performing the 24-hour endurance testing required by Technical Specification (TS) Surveillance Requirement (SR) 3.8.1.14.

The Division 2 DG configuration is tandem (in-line) diesel engines, a 12-cylinder engine [ENG] and a 16-cylinder engine, with the electric generator [GEN] unit situated between the engines. Prior to the 24-hour run, the generator bearing on the side adjacent to the 12- cylinder diesel engine had been indicating increasing wear during recent operation to fulfill periodic TS surveillance requirements. The onset of generator bearing degradation was observed as indicated by bearing vibration data.

On November 22, 2000, the Divison 2 DG was being run for the monthly operability surveillance test. While the DG was running at idle speed of about 400 revolutions per minute (rpm), an operator noticed an abnorMal audible noise at the 12-cylinder engine end of the generator. The DG was shut down. The DG was restarted at idle speed and vibration measurements were taken. Vibration levels were acceptable at running speeds of 400 and 900 rpm. The DG was loaded and the surveillance was completed satisfactorily. Vibration readings taken during the run were 0.40 to 0.45 inches per second based on a standard low frequency range. These readings were above the alert threshold of 0.325 inches per second, but still acceptable. An operability evaluation was performed, and based on extensive vibration analysis, lubrication analysis, and thermography, engineering concluded that the bearing was degraded but capable of supporting a design basis DG run.

On December 27, 2000, during another monthly Division 2 DG run, the vibration levels took a step increase. The vibration level at the 12-cyinder engine end increased from 0.568 inches per second to 0.684 inches per second, an increase of 0.116 inches per second in 45 minutes. This increase was attributed to expanding the upper frequency range to include frequencies up to 5000 hertz. Engineering concluded that this new information was still bounded by the operability evaluation.

During the 24-hour run of the DG on January 12, 2001, it was confirmed that generator bearing degradation had progressed to the point that pre-established criteria based on industry standards were exceeded. Therefore, at 0228 hours0.00264 days <br />0.0633 hours <br />3.769841e-4 weeks <br />8.6754e-5 months <br /> on January 13, 2001, the DG was secured to preclude further degradation and to facilitate proactive repairs. Action Request (AR) F11288 was initiated to replace the generator bearing.

The criteria for securing the DG from the 24-hour run established prior to commencing the run were: bearing vibration greater than 0.99 inches per second analyzed for 0 to 5000 hertz; bearing vibration greater than 0.60 inches per second analyzed for 0 to 1000 hertz; and bearing temperature greater than 300 degrees Fahrenheit (F). The horizontal bearing vibration levels recorded during the performance of the 24-hour run were determined to be in excess of both shutdown action levels for vibration. The maximum recorded vibration level in the 0 to 1000 hertz range was 1.01 inches per second in the horizontal direction.

The maximum recorded vibration level in the 0 to 5000 hertz range was 1.69 inches per second in the horizontal direction. The observed average bearing temperatures during the run were 115 degrees F for the bearing on the 12-cylinder engine end of the generator and 131 degrees F for the bearing on the 16-cylinder engine end of the generator. Engineering FACILITY NAME (1) PAGE (3) LER NUMBER (6) DOCKET (2) Clinton Power Station 05000461 2001 - 001 00 determined that degradation of the roller bearing on the 12-cylinder engine end of the generator had caused the vibration levels to exceed the specified predictive maintenance limits. Further, Engineering determined that the vibration data indicated that the bearing was generating frequencies consistent with damage to the roller elements and the outer race.

TS Limiting Condition for Operation (LCO) 3.8.1, "AC Sources - Operating," requires the Division 1, Division 2 and Division 3 diesel generators (DGs) to be operable during Modes 1, 2 (STARTUP), and 3 (HOT SHUTDOWN). With one DG inoperable, Condition B applies and Required Action B.4 requires restoring the inoperable DG to an operable status within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> and 6 days from the discovery of failure to meet the LCO if contiguous failures involving other required AC sources are incurred. Otherwise, Condition F is entered such that Mode 3 is required to be entered in 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> per Required Action F.1, and Mode 4 (COLD SHUTDOWN) is required to be entered in 36 hours4.166667e-4 days <br />0.01 hours <br />5.952381e-5 weeks <br />1.3698e-5 months <br /> per Required Action F.2.

Condition B of TS LCO 3.8.1 was entered for one required DG inoperable upon taking the Division 2 DG out of service. Hence, in accordance with Action B.4, the DG was required to be restored to operable status within 72-hours, 0228 hours0.00264 days <br />0.0633 hours <br />3.769841e-4 weeks <br />8.6754e-5 months <br /> on January 16, 2001. The failure to meet Action B.4 would require a shutdown of CPS.

CPS had been preparing contingency plans for replacing the generator bearing for several weeks prior to this event in the event that emergency repairs would become necessary. Due to the extensive nature of the maintenance required to replace the bearing, CPS estimated that an additional four days, approximately, beyond the current Completion Time (72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />) would be necessary to complete the repairs. The scope of the work included dismantling and de-terminating the generator, uncoupling the generator from the diesel unit, employing special rigging and lifting the generator out of its current position, replacing the bearing, reassembling and re-coupling the diesel to the generator, and performing post- maintenance testing, including a 24-hour endurance run.

Additionally, prior to this event, CPS prepared and submitted an application for amendment of the CPS Operating License to permanently extend the allowed outage time (AOT) from the current 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> to 14 days for the Divisions 1 and 2 DGs (AmerGen Energy Company LLC (AmerGen) Letter U-603416 dated December 29, 2000).

When it was discovered that the Division 2 DG should not be returned to service with the bearing in its degraded condition, CPS Management contacted the NRC to request enforcement discretion for allowing the DG to remain inoperable beyond the current TS limit. At about 1400 hours0.0162 days <br />0.389 hours <br />0.00231 weeks <br />5.327e-4 months <br /> on January 13, 2001, via a telephone conference, AmerGen requested enforcement discretion to extend the required Completion Time of LCO 3.8.1, Action B.4, from the current limit of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> to 14 days. An extension of the AOT from 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> to 14 days for Required Action B.4 of LCO 3.8.1 and associated Required Action A.2 would allow the required repairs and post-maintenance testing for the Division 2 DG to be completed.

Further, granting the extension would avoid undesirable transients as a result of forcing compliance with the Actions for the LCO in subjecting the plant to an unnecessary shutdown, thus minimizing potential adverse safety consequences and operational risks.

FACILITY NAME (I) DOCKET (2) LER NUMBER (6) PAGE (3) Clinton Power Station 05000461 2001 - 001 00 The NRC verbally granted the request for enforcement discretion for the Division 2 DG on a one-time basis at 1605 hours0.0186 days <br />0.446 hours <br />0.00265 weeks <br />6.107025e-4 months <br /> on January 13, 2001. The enforcement discretion was to be effective until the Division 2 DG was returned to an operable status or until the requested 14-day AOT expired, whichever occurred first. A follow-up letter was submitted pursuant to the verbal request (AmerGen Letter U-603451 dated January 14, 2001).

The repairs of the Division 2 DG and the 24-hour run were completed with satisfactory results, and the DG was declared operable on January 21, 2001, at about 1055 hours0.0122 days <br />0.293 hours <br />0.00174 weeks <br />4.014275e-4 months <br />.

The degraded generator bearings were sent to Exelon Power Labs for failure analysis. The analysis of the bearing issue performed by Exelon concluded that the DG would have remained capable of performing its safety function despite the bearing degradation.

Condition Report (CR) 2-01-01-076 was initiated to track a cause and corrective action determination for this event.

No automatic or manually initiated safety system responses were necessary to place the plant in a safe and stable condition. Other inoperable equipment or components did not directly affect this event.

CAUSE OF EVENT

The cause of this event is attributed to inadequate standards for design review of a design change in 1993 that installed a generator shaft guard on the south end of the generator.

The General Design Review Standards did not adequately identify that attachments to large motors or generators required further analysis, including analysis of possible impacts to breaching insulated bearings. The use of the inadequate design review standards led to an error in Engineering Change Notice (ECN) 28036 that defeated the insulation of the bearing, causing an electrical short circuit around the insulated bearing. The electrical short circuit caused fluting (electrical pitting) of both generator bearings by electrical ground current through the bearing rollers, cup and cones. The designer for ECN 28036 did not recognize and understand the need for an electrical insulation barrier in the south end bearing. A causal factor identified was the designer's tacit acceptance that the shaft guards were safe to install because the coupling and shaft guards were designed and supplied by the original DG supplier, Stewart and Stevenson. Consequently, the design change was not rigorously evaluated prior to implementation. The design change did not contain a design input review and was not routed to other engineering disciplines, including Electrical Design, for their review of the shaft guard design. The failure to complete a multi-discipline review bypassed a process barrier that may have identified the potential impact of breaching the bearing insulation.

A contributing cause was failure of the vibration monitoring to detect the bearing degradation earlier. Predictive maintenance practices monitor vibration at frequency ranges below 1200 hertz. Since most bearing defects generate frequencies above 2000 hertz during initial stages of degradation, early detection of bearing degradation was not possible.

FACILITY NAME (1) PAGE (3) LER NUMBER (6) DOCKET (2) Clinton Power Station 05000461 2001 - 001 00 A second contributing cause was the failure to periodically monitor generator shaft voltage to detect breakdown of insulation. The existing procedure for monitoring generator shaft voltage was incorrectly understood to be for bearing rework post-maintenance testing only, therefore, this testing had not been performed before this event.

The cause evaluation for this event identified a common cause between the Divisions 1 and 2 DGs. The Division 1 DG is showing vibration signatures similar to the Division 2 DG, but at much lower peak velocities that remained steady during past DG runs and thus remains operable. No common cause was identified for the Division 3 DG.

CORRECTIVE ACTION

The generator bearings in the Division 2 DG were replaced.

General Design Review Standard GD (RS)-08, "Electrical Design Standard," will be revised to require analysis of attachments to large motors or generators, specifically for impact on bearing insulation.

The generator shaft guards have been removed from the Divisions 1 and 2 DGs; the shaft guards will be either eliminated or insulated prior to installation.

Maintenance procedure CPS 8507.01, "Division I/II Diesel Generator Maintenance," has been revised to improve guidance on shaft current monitoring and require periodic monitoring of generator shaft currents for trending purposes.

Generator shaft current testing was completed for the Divisions 1 and 2 DGs. Preventive maintenance tasks will be revised or initiated to measure generator shaft potential during certain runs of the Divisions 1 and 2 DGs.

The preventive maintenance practices for vibration monitoring have been changed to monitor vibration on the generators of the three emergency diesel generators at a frequency range of 10,000 hertz.

ANALYSIS OF EVENT

This event is reportable under the provisions of 10CFR50.73(a)(2)(i)(B) as an operation or condition prohibited by the plant's Technical Specifications. Although enforcement discretion was granted to temporarily allow an exception to TS LCO 3.8.1, the requirements of the LCO in the existing CPS TS do not allow continued operation of the plant in Mode 1 with the Division 2 DG inoperable for more than 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> as described in this event.

During this event, no design basis accident occurred requiring the emergency diesel generators to perform their designed safety functions and off-site power remained available to provide power to the engineered safety features buses. Further, the remaining DGs, Division 1 and Division 3, were operable during this event and are sufficient for performing the required safety functions during a design basis accident.

FACILITY NAME (1) DOCKET (2) LER NUMBER (6) PAGE (3) Clinton Power Station 05000461 6 � OF � 6 An extensive risk evaluation and technical basis were provided to the NRC in the request for a license amendment to permanently extend the DG AOT from 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> to 14 days for the Divisions 1 and 2 DGs (submitted via AmerGen Letter U-603416 dated December 29, 2000).

This evaluation concluded that the proposed DG AOT change would have only a small quantitative impact on plant risk.

Based on the above, extending the AOT for an inoperable DG for up to 14 days had minimal effect on plant risk.

ADDITIONAL INFORMATION

The Division 2 generator is a model SA6-44, manufactured by Ideal Electric Company.

In January 1999, a bearing failure of Shutdown Service Water System [BI] pump [P] 1SX01PA due to shaft electrical currents was identified in CR 1-99-01-062. This issue was reported in LER 1999-001. The corrective action for this CR included a review of maintenance to identify changes needed for other equipment susceptible to shaft currents. The DG maintenance procedure was revised to include shaft current testing following bearing rework. However, the event described in this LER was the first time the bearings were removed from the diesel generator.

For further information regarding this event, contact S. J. Kowalski, System Manager, at (217) 937-3902.