05000325/LER-2006-007
| Brunswick Steam Electric Plant (Bsep) | |
| Event date: | |
|---|---|
| Report date: | |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications |
| 3252006007R01 - NRC Website | |
Energy Industry Identification System (EIIS) codes are identified in the text as [XX].
Introduction Initial Conditions At the time of this event, Unit 1 was in Mode 1, at approximately 100 percent of rated thermal power. Unit 2 was in Mode 3. The Unit 2 Startup Auxiliary Transformer (SAT) [EA] was inoperable; having previously been lost at approximately 1823 EST on November 1, 2006. The four Emergency Diesel Generators (EDGs) [EK] appropriately started and ran until the time of the event. EDGs 3 and 4 were supplying the Unit 2 emergency buses. EDGs 1 and 2 started and were running unloaded, consistent with plant design.
EDGs 2, 3, and 4, as well as the remaining offsite power, were not adversely affected by the trip of EDG 1.
Reportability Criteria EDG 1 is being considered inoperable from the time it was removed from service, for routine maintenance, on October 23, 2006, at 0208 EDT until it was returned to service on November 7, 2006, at approximately 2125 EST. This constitutes approximately 15 days, 20 hours2.314815e-4 days <br />0.00556 hours <br />3.306878e-5 weeks <br />7.61e-6 months <br /> and 17 minutes of inoperability, applicable to Unit 1. Unit 2 entered Mode 4 at approximately 0624 EST on November 4, 2006, exiting the Mode of Applicability for Limiting Condition for Operation (LCO) 3.8.1, "AC Sources - Operating," for Unit 2 (i.e., approximately 12 days, 5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> and 16 minutes of inoperability, applicable to Unit 2). The Technical Specification (TS) 3.8.1 allowed out of service time for an inoperable EDG is 7 days. This condition is being reported in accordance with 10 CFR 50.73(a)(2)(0(3) as operation prohibited by the plants' Technical Specifications. Though EDG 1 is being considered inoperable from the start of the maintenance period on October 23, 2006, it was functional and available from 0005 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> EDT on October 30, 2006 until it was placed under clearance on November 3, 2006, at approximately 0400 hours0.00463 days <br />0.111 hours <br />6.613757e-4 weeks <br />1.522e-4 months <br /> EST. Although EDG 1 tripped at approximately 0400 EST on November 2, 2006, it could have been restarted, had the need arisen.
Event Description
Sequence of Events At approximately 1823 EST on November 1, 2006, the Unit 2 SAT was lost. As a result, the four EDGs appropriately started and ran. EDGs 3 and 4 were supplying the Unit 2 emergency buses. EDGs 1 and 2 continued to run unloaded, per design, until, at approximately 0400 EST on November 2, 2006 (i.e., after approximately 9 hours1.041667e-4 days <br />0.0025 hours <br />1.488095e-5 weeks <br />3.4245e-6 months <br /> and 37 minutes of run time), EDG 1 tripped.
Previously, at approximately 0200 on November 2, 2006, an EDG 1 lube oil duplex strainer high differential pressure (dP) alarm was received. The strainer dP was locally observed to be 9 psid compared with an alarm setpoint of 10 psid. Operations personnel swapped the duplex strainer from the west basket to the east basket. At approximately 0300, a second high dP alarm was received with differential pressure across the east basket at 18 psid. The strainer was swapped back to the west basket and the east basket was Event Description (continued) removed for cleaning. Following cleaning and reassembly, maintenance personnel commenced the process of filling the east basket by partially diverting flow from the in-service basket in accordance with plant procedure OPM-FLT515, "Diesel Generator Lube Oil Strainer Cleaning and Servicing." During this activity, the outlet pressure of the strainer dropped below the low pressure trip setpoint of 28.2 psig.
Operators, who were present at the time, observed pre-trip pressures of approximately 58 to 60 psig at the pump outlet and 38 to 40 psig downstream of the duplex strainers.
EDG 1 tripped during the lube oil duplex strainer swapping evolution. The current duplex swapping instructions require the empty basket to be filled slowly with the process fluid by opening the vent valve on the empty basket lid and then slowly adjusting the valve handle to allow a portion of the process flow to fill the empty basket; while maintaining flow through the in-service strainer to the lube oil header. The diverting of the oil flow to the path of least resistance (i.e. the empty strainer basket versus the high dP strainer) results in a momentary drop in lube oil header pressure. This pressure drop was sufficient to result in a low lube oil header pressure trip. To address the latent procedural weakness, which led to the momentary drop in lube oil header pressure, procedure OPM-FLT515, will be revised to require filling a cleaned duplex strainer basket with non-process oil if the associated EDG is running.
Subsequently, it was determined that the cause of the elevated duplex strainer dPs experienced prior to the EDG 1 trip was a result of: (1) fibrous material accumulating on the strainers, combined with (2) bearing babbit material released as a result of failure of EDG 1 bearing #9.
Fibrous Material The source of the fibrous material found on the EDG 1 duplex strainer was a cleaning towel which had been left in the EDG 1 lube oil sump during the maintenance activity performed from October 23 through 27, 2006. This towel was found at the EDG 1 engine driven lube oil pump suction strainer.
A 72-month preventive maintenance (PM) activity was performed on EDG 1 from October 23 through October 27, 2006. This PM included draining of the EDG lube oil with the subsequent cleaning and inspection of the lube oil sump. This was followed by a closeout inspection prior to filling of the lube oil system with new oil. The PM also replaced a number of lube oil hoses, performed bearing lift checks, and inspected visible sections (i.e., no disassembly or removal was performed) of the bearings. A three hour loaded run was conducted at 3500 kw at the completion of the PM.
Subsequent to the PM, problems not affecting the lube oil system required corrective maintenance to be performed on October 27 and 28, and again on October 28 and 29. These activities required runs of EDG 1 on October 28 and 29. Including the post-PM activities, five runs were required to satisfy maintenance testing and operability requirements. This provided a cumulative run time of approximately 15.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> prior to declaring EDG 1 operable on October 30, 2006 at approximately 0005 EDT. As discussed above, EDG 1 then ran for approximately 9.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> in response to the loss of the Unit 2 SAT. Thus, EDG 1 Event Description (continued) accumulated approximately 25 hours2.893519e-4 days <br />0.00694 hours <br />4.133598e-5 weeks <br />9.5125e-6 months <br /> of total run time with the cleaning towel at the engine driven lube oil pump suction strainer with no indications of degradation from the pump and only marginally high dPs, remaining at or below the alarm setpoint of 10 pisd, at the lube oil duplex strainer.
Bearing #9 The source of the bearing material found on the EDG 1 duplex strainer was the failure of EDG 1 bearing #9.
The exact cause of the bearing #9 failure could not be determined with certainty. The select cause of the bearing #9 failure is the presence of increased main crankshaft bearing clearances which results in lower initial bearing crush and greater operational stresses on the bearings. Analysis of the bearing supports the conclusion that the bearing experienced a loss of crush over time (i.e., since 1992 when last replaced). The loss of crush led to rocking of the bearing in the bearing bore, which in turn led to further mechanical deformation and further loss of crush. Ultimately, the bearing lost effective lubrication and the surface of the bearing was wiped, thereby resulting in the bearing material found on the EDG 1 duplex strainer.
The presence of the cleaning towel, in itself, did not cause the failure of EDG 1 bearing #9. As stated above, the failure was the result of long term degradation, which ultimately resulted in bearing wipe on November 2, 2006. As stated earlier, EDG 1 accumulated approximately 25 hours2.893519e-4 days <br />0.00694 hours <br />4.133598e-5 weeks <br />9.5125e-6 months <br /> of total run time with the cleaning towel at the engine driven lube oil pump suction strainer with no indications of degradation from the pump and only marginally high dPs at the lube oil duplex strainer. This indicates that despite any flow reductions from direct blockage by the towel or clogging of the duplex strainer, oil flow was maintained within normal operational limits and bearing damage should not have occurred unless other factors were involved as discussed above.
Conclusion Although the direct cause of the November 2, 2006, trip of EDG 1 was a momentary drop in lube oil header pressure during duplex strainer manipulation, the longer term impact of the presence of the cleaning towel has not been fully evaluated with respect to the ability of EDG 1 to operate for an extended period of time (i.e., at minimum for greater than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />, the Mitigating System Performance Index (MSPI) mission time) without operator intervention. As such, EDG 1 is being considered inoperable due to the presence of the cleaning towel in the lube oil sump.
Event Cause The root cause of the EDG 1 trip is inadequate foreign material exclusion (FME) controls. As a result, a cleaning towel was left in the lube oil system. The presence of the towel, combined with bearing material, caused higher than normal lube oil duplex strainer dPs. This in turn led to the duplex strainer swapping that ultimately resulted in tripping of the EDG.
Event Cause (continued) The failure to identify the towel during the system closeout was a barrier failure with regards to FME controls. The lube oil sump had been determined to require moderate FME controls because it was determined that any foreign material left in the sump would be readily detectable. Under moderate FME controls, the requirement and expectation was that all foreign material be removed prior to system closeout and subsequent oil refill. A cleanliness inspection was performed, in accordance with plant procedures, which required verification of system cleanliness by the responsible lead personnel and a quality control inspector. These inspections were performed but were not adequate to detect the towel. Had the lube oil sump work required high FME controls, a log of material brought into the control boundary would have been maintained; thereby ensuring that no material was inadvertently left behind.
Safety Assessment The safety significance of this condition is considered minimal.
The safety significance of this event was minimal since, upon the trip of EDG 1, the associated emergency bus remained energized via offsite power. Failure of one DG is within the design basis of the plant and is not anticipated to have jeopardized plant safety. The diesel generator was available until it was placed under clearance on November 3, 2006, at approximately 0400 hours0.00463 days <br />0.111 hours <br />6.613757e-4 weeks <br />1.522e-4 months <br /> EST.
Corrective Actions
The following corrective action to prevent recurrence will be completed as a result of this event.
- A high FME job classification for the EDG lube oil sumps was established through revision to the plant procedures which control maintenance activities associated with the sumps.
Although not attributed as the root cause of this event, the following corrective action to prevent recurrence of the EDG 1 bearing 9 failure will be completed.
- An engineering change will be completed to install oversized bearings in EDG 1 to address the presence of increased main crankshaft bearing clearances. The oversized bearings are currently scheduled to be installed in June 2009.
Although not attributed as the root cause of this event, the following corrective action to address the latent procedural weakness, which was the direct cause of the EDG 1 trip, will be completed.
- Procedure OPM-FLT515, will be revised to require filling a cleaned duplex strainer basket with non process oil if the associated EDG is running. This revision is currently scheduled to be completed by February 28, 2007.
Previous Similar Events
A review of LERs for the past three years identified no similar events of TS operability being impacted by foreign material left in a system. LERs 1-2004-001, dated March 4, 2004, and 1-2004-003, dated October 13, 2004, dealt with operation prohibited by TSs due to EDG inoperability, however, the root causes and corrective actions associated with these events could not have been expected to prevent the event reported in this LER (i.e., LER 1-2006-007).
Commitments No regulatory commitments are contained in this report.