05000247/LER-2011-002
| Indian Point 2 | |
| Event date: | 10-03-2011 |
|---|---|
| Report date: | 12-01-2011 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications 10 CFR 50.73(a)(2)(v), Loss of Safety Function |
| 2472011002R00 - NRC Website | |
Note: The. Energy Industry Identification System Codes are identified within the brackets {}.
DESCRIPTION OF EVENT
On October 3, 2011, at approximately 04:00 hours, while at 100% steady state reactor power with the 1/2/3 Service Water (SW) {BI} header as the essential header, an attempt was made to start the 21 Service Water Pump (SWP) {P} in response to low unexpected flows to the containment fan cooler units during testing, but the 21 SWP failed to start as required. When the attempt was made to start the 21 SWP, the red indicating light, which is designed to illuminate upon closure of the load breaker did not illuminate as expected. The green indicating light, which implies that control power is available to close the supply breaker extinguished after the breaker close attempt.
Technical Specification (TS) 3.7.8 Condition A was entered for one of the required SWP's on the essential header inoperable. Personnel were sent to investigate the 21 SWP breaker {BKR} and at approximately 04:15 hours operators discovered the control power fuses {FU} had blown. The fuses were replaced and a second attempt made to close the breaker. The breaker inertia latch was identified as stiff and binding throughout its movement. With the breaker inertia latch toggled and not reset, the breaker will be mechanically blocked from closing and will result in control fuse actuation. At approximately 05:23 hours, TS 3.7.8 condition A was exited due to swapping to the 4/5/6 SW header as the essential header. The 21 SWP breaker was repaired, tested and declared operable on October 4, 2011, at 00:05 hours. The condition was recorded in the Indian Point Energy Center (IPEC) Corrective Action Program (CAP) as Condition Report CR-IP2-2011-04893.
Following the failure of the 21 SWP breaker to close, an investigation discovered the breaker inertia latch was stiff and binding throughout its full movement. The latch exhibited resistance and was difficult to remove from the pivot pin. The 21 SWP breaker is a Westinghouse DB-50 breaker with the new style inertia latch which provides a reset spring pressure and added weight for seismic requirements. The breaker inertia latch is designed to prevent the breaker from re-closing following a trip operation.
The force of a breaker opening rotates the inertia latch until it temporarily latches onto a pin on the closing lever, preventing the contact arms from moving. Gravity and spring pressure then rotates the latch free of this anti-bounce position to allow the next close operation. The inertia latch is made of carbon steel with a stainless steel (SS) bushing insert. The inertia latch is mounted on the operating mechanism mounting pin, which is made of stainless steel (SS). The operating mechanism and inertia latch were found plated with zinc dichromate. The zinc was applied to protect the carbon steel from rust. In the past, during the plating process by the original equipment manufacturer (OEM) (Westinghouse), the SS parts were inadvertently plated with the rust protection coating. In the 1990's the OEM introduced zinc dichromate plating when it was determined that the plating on the SS parts did not adhere well and could flake off resulting in foreign material in the clearance between the inertia latch bushing and mounting pin. The zinc dichromate flaking results in an abrasive material that tends to reduce the clearance between the pin and latch bushing resulting in increased friction/binding. The OEM applied the gold/yellowish coating (dichromate) as a marker to note that the zinc coating was applied. The OEM issued Tech Bulletin TB-07-04 and a Nuclear Safety Advisory Letter (NSAL-98-009) to alert the industry to the potential breaker issue with recommendations for removal of the plating from the SS parts. For the 21 SWP event, the inertia latch had not rotated free therefore the breaker would not close and the closing coil remained energized actuating the control fuse. Upon removal, the mounting pin showed signs of material buildup and the gold/yellowish dichromate. The latch was sent for failure evaluation where it was found that the inner bushing surface had material buildup which was determined to be zinc and gold/yellowish tinted dichromate. The 21 SWP breaker {BKR} is a 480 volt supply breaker (52/SW1) manufactured by Westinghouse {W120) Model DB-50.
Cause of Event
The direct cause for the breaker failure to close was the breaker inertia latch was not reset and prevented the breaker from closing on demand. The failure to reset was due to binding caused by flaking of zinc dichromate plating whose unintended application at the inertia latch SS bushing and pin location lead to localized material deposited in the clearance. The root cause (RC) was a failure of workers to perform the procedural steps required to remove and clean the zinc dichromate plating as required by the PM procedure. Review of completed Preventive Maintenance (PM) procedures for previous breaker PMs had statements that no gold zinc dichromate plating was present on latch or pin surfaces. An EOC inspection of the breakers found that surfaces in most cases had gold/yellow plating. Insufficient worker practices resulted in failure to properly execute PM procedure steps. There was a failure of workers to identify the yellow/gold plating on breaker pin and latch surfaces therefore adequate cleaning was not performed. Workers failed to properly utilize human performance tools such as self checking to correctly execute the PM steps.
A contributing cause was the OEM notifications (NSAL-98-009 and Tech Bulletin TB-07-04) did not disclose the two coatings involved with zinc dichromate plating or the complexity of removal of the coatings. The NSAL-98-009 stated the bushing in the inertia latch and pivot pin had been plated with zinc dichromate. The NSAL recommended inspecting the inertia latch bushing and pivot pin during the next maintenance interval and removal of the plating using Scotch-Brite or a fine emery cloth. In March 2002, the OEM issued MPM-DB-Breaker, the maintenance Program Manual for Safety Related Type DB low voltage Metal Enclosed switchgear. The guidance included inspection for evidence of zinc dichromate and removal recommendations (Scotch-Brite or a fine emery cloth). The guidance provided by the OEM was vague and no distinction was made between the zinc plating and the dichromate finish. The OEM notices did not discuss the level of effort that was necessary and required to remove not only the gold/yellow tinted dichromate but the zinc olatina as well.
Corrective Actions
The following corrective actions have been or will be performed under Entergy's Corrective Action Program to address the cause and prevent recurrence:
- Corrective maintenance was performed on the 21 SWP breaker (52/SW1), and breaker returned to service following satisfactory testing.
- A review was performed by engineering of previous DB breaker PM work packages to identify which latches and pins were not adequately cleaned.
- The OEM provided in EOC phase 3 a detailed step list for onsite cleaning of the inertia latch bushing and pivot pin which will also be included in revised breaker PM procedures.
- Maintenance completed a human performance error review in accordance with EN-HU- 103 (Human Performance Error Reviews) and re-enforced expectations of worker practices in department communications and prejob briefs regarding the RC of this event.
- An apparent cause evaluation was completed per CR-IP3-2011-00645 to address procedure use and adherence issues. A corrective action included performing a Snapshot assessment on procedure use and adherence and implementing the MARC process for placekeeping/procedure use events that fell short of procedure requirements and site expectations. Any necessary additional actions will be performed to address any identified AFI's or negative observations in the area of procedure use and adherence or worker practices.
- Maintenance procedures 2-BRK-022-ELC, 0-BRK-410-ELC, and 0-BRK-401-ELC will be revised to include the new OEM detailed step list for onsite cleaning of the inertia latch bushing and pivot pin and will include independent verification sign offs for the latch and pin cleaning steps.
- A standing order (SO) was issued by Operations to inspect inertia latches for correct position on DB breakers following an open operation.
- The OEM will review the guidance for cleaning DB-50 breaker inertia latch and pin plating material to identify any missed opportunities in previously issued industry notices.
- Engineering will review the OEM letter from the October 2011 site visit that supported the breaker inertia latch inspections and any additional required actions will be implemented.
Event Analysis
The event is reportable under 10CFR50.73(a)(2)(i)(B). The licensee shall report any operation or condition which was prohibited by the plants TS. On October 3, 2011, during quarterly surveillance test (2-PT-Q16) for SW flow, with the 1/2/3 SW header as the essential header, all five Fan Cooler Units (FCUs) indicated inadequate SW flow.
TS 3.6.6 Condition F was entered for three trains of FCUs inoperable and as required TS 3.0.3 entered. On October 3, 2011, at 04:00 hours, Operations initiated start of the 21 SWP as a result of the low SW flow but it failed to start. TS 3.7.8 Condition A was entered with actions to return to operable within 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. At 05:19 hours, the essential SW header was swapped to the 4/5/6 header. At 05:23 hours, TS 3.7.8 condition A was exited due to swapping to the operable 4/5/6 SW header. An engineering review of the breaker for past operation determined the breaker was last operated and left in the open position on September 30, 2011, at 01:13 hours. The total time the SW function was impacted due to the 21 SWP breaker inoperability was 76 hours8.796296e-4 days <br />0.0211 hours <br />1.256614e-4 weeks <br />2.8918e-5 months <br />, 10 minutes.
TS 3.7.8 has an allowed outage time of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> for one SWP inoperable.
This event meets the reporting criteria because the inoperable condition during past operation exceeded the 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> allowed completion time for TS 3.7.8 and the required actions were not performed. During this time frame, the SWPs on the essential header had degraded performance due to silt accumulation in the SWP bay. This condition was recorded in CR-IP2-2011-04894 and reported by LER-2011-003.
There was no safety system functional failure reportable under 10CFR50.73(a)(2)(v) for the 21 SWP failure as the minimum required components were available to perform the function during the time the 21 SWP was inoperable (at least two SWPs operable on the essential SW header). In accordance with reporting guidance in NUREG-1022, an additional random single failure need not be assumed in that system during the condition. The 22 and 23 SWPs were available during the time the 21 SWP was considered inoperable (See LER-2011-003 for a discussion of the flow issue associated with the containment FCUs).
Past Similar Events
A review was performed of the past three years of Licensee Event Reports (LERs) for events reporting a TS violation due to a breaker failure to close. LER-2009-003 reported the loss of a single train 21 Pressurizer Backup Heater for remote shutdown due to failure of its breaker to close. The breaker was inoperable due to its anti pump lever being misaligned. The misalignment was likely due to maintenance activities during previous breaker rack-in. The CAs of the event reported in LER-2009-003 would not have prevented this event as the causes were different.
Safety Significance
This event had no significant effect on the health and safety of the public. There were no actual safety consequences for the event because there were no accidents or transients during the time the 21 SWP was inoperable. The 21 SWP was designated as one of the three SWPs (1/2/3) on the essential header. In accordance with design (TS Basis 3.7.8) the essential SW system heat loads can be cooled by any two of the three SWPs on the essential header. During this event the two remaining SWPs on the essential header (22 and 23 SWPs) were operable and all three SWPs on the non-essential header were operable (See LER-2011-003 for a discussion of the flow issue associated with the containment FCUs). There were no multiple emergency core cooling system (ECCS) trains inoperable during the time the 21 SWP was inoperable.
The risk significance with the 21 SWP considered inoperable while on the essential header (assuming no other out of service equipment) raises the annual Core Damage Frequency (CDF) in EOOS from the baseline of 1.24E-5/year to 1.33E-5/year and represents a delta CDF of 9E-7/year. Assuming an exposure time of 76.2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> (time of past inoperability of the 21 SWP) would give an incremental conditional core damage probability (ICCDP) of 8E-9 [9E-7/year x (76.2 hour2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />s/8760 hours/year)]. A ICCDP below 5E-7 would typically not be considered risk significant.