05000247/LER-2005-001
| Docket Number | |
| Event date: | 1-19-2005 |
|---|---|
| Report date: | 8-3-2005 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(B), Prohibited by Technical Specifications |
| 2472005001R01 - NRC Website | |
Note: The Energy Industry Identification System Codes are identified within brackets { }
DESCRIPTION OF EVENT
On January 19, 2005, at approximately 1438 hours0.0166 days <br />0.399 hours <br />0.00238 weeks <br />5.47159e-4 months <br />, while at 100% steady state reactor power, the 21 Auxiliary Component Cooling Water (ACCW) {CC) Pump {P) failed to develop the proper differential pressure across the pump during performance of quarterly surveillance test PT-Q31A, 21 Auxiliary Component Cooling Pump. Quarterly surveillance test PT-Q31A was halted, Operations notified and the 21 ACCW Pump was declared inoperable. Two ACCW Pumps provide cooling water flow to the motor coolers for the two Containment Recirculation Pumps {BP} of the emergency core cooling system (ECCS) for required cooling during the recirculation phase of a design basis accident. For Technical Specification (TS) purposes, the inoperability of one ACCW Pump is assumed to render one Containment Recirculation pump of the ECCS inoperable. The Action Statement for the ECCS TS Limiting Condition for Operation (LCO), TS 3.5.2 requires three trains of ECCS to be operable. The required action A.1 for TS 3.5.2 Condition A, One or more trains inoperable, is to restore the train(s) to operable within a completion time of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. An investigation and troubleshooting indicated the discharge check valve for the 22 ACCW Pump, valve 755A, was not fully closed causing short circuiting of the 21 ACCW Pump discharge flow. To verify that the 22 ACCW Pump discharge check was not fully closed, the 22 ACCW Pump discharge isolation valve (753E) was closed. A retest was then performed on the 21 ACCW Pump per PT-Q31A and the test results were satisfactory. The test confirmed that the 22 ACCW Pump discharge check valve (755A) was not fully closed and Operations returned the 21 ACCW Pump and its associated containment recirculation pump to operable status. The TS 3.5.2 Allowed Outage Time (AOT) for one or more trains of ECCS inoperable is 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. Maintenance opened valve 755A and discovered that the valve was off its seat with the valve disc hinge bound up due to problems with the clearances between the hinge pin and the hinge bushings. Upon valve disassembly, Maintenance discovered one of the bushings had a raised burr where the bushing interfaces with the outside surface of the hinge pin. Maintenance performed cleaning and repairs including removal of the burr and returned the valve to service. Operations declared the 22 ACCW Pump and its associated Containment Recirculation Pump operable and exited TS LCO Action A.1 at approximately 1230 hours0.0142 days <br />0.342 hours <br />0.00203 weeks <br />4.68015e-4 months <br />, on January 20, 2005. Engineering evaluated the condition for past operability and determined on February 9, 2005, that operability of valve 755A could not be demonstrated after the valve's maintenance on November 3, 2004.
The valve did test satisfactorily in the open direction during the 22 ACCW pump surveillance test per PT-Q31B on December 23, 2004. In the November 2004 inspection, the valve was also found to have a slight binding problem.
Component engineering assessment of the as-found condition concluded the valve was acceptable as per procedure. The bind-up was attributed to rust on the hinge pin, which after cleaning provided for the disc to swing free. The last successful performance of the quarterly surveillance functional test for the 21 ACCW pump was on October 14, 2004. The successful test of the 21 ACCW pump during the October 14, 2004 test also confirmed that the 22 ACCW pump discharge check valve 755A was closed. The duration of containment recirculation pump assumed inoperability due to ACCW pump inoperability exceeded the TS allowed outage time of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. However, both ACCW pumps are designed to start to support recirculation pumps motor cooling and with both pumps operating the not fully closed 22 ACCW pump discharge check valve would not have affected supply of ACCW to the recirculation pump motors.
Post work testing (PWT) was performed on check valve 755A on November 5, 2004, during the refueling outage that confirmed the proper discharge flow rate for the 22 ACCW pump and that valve 755A satisfactorily opened. The PWT also checked that discharge check valve 755A was closed by operating the redundant 21 ACCW pump with the 22 ACCW pump secured and observing that the 22 ACCW pump shaft remained stationary.
Engineering review of valve maintenance identified that valve 755A had not been intrusively inspected since April 1989, because after that time the ACCW discharge check valves were removed from the PM program. Subsequently, a review in 2003 concluded they should be included in the 10 year PM program and they were re-instated and scheduled for the next outage (2004). The event was recorded in the IPEC corrective action program (CAP) as condition report CR-IP2- 2005-00252. The ACCW Pump discharge check valves (755A&B) are three (3) inch, Velan {V085), Model 0114B, ANSI 150 psig check valves. An extent of condition review was performed. Two other valves of the same make and model were identified at Unit 2; ACCW Pump discharge valve 755B, and excess letdown heat exchanger inlet check valve 790. Neither valve has any history of failing to seat. Valve 755B was inspected intrusively (full disassembly and inspection) in May, 1995, with no binding noted. Valve 790 has never been inspected intrusively, although it tested satisfactorily prior to refueling outage cycle 16 in October 2004. Both valves are scheduled to be inspected intrusively in the next refueling outage in the spring of 2006.
CAUSE OF EVENT
The direct cause of the 21 ACCW Pump failure to meet surveillance test differential pressure criteria was the short cycling of pump discharge flow through the unseated discharge check valve 755A for the 22 ACCW Pump. The cause of 22 ACCW Pump check valve (755A) failure to seat was an interference in the valve hinge pin and hinge bushing interface which prevented the valve disc from fully closing. A burr was removed from one of the valve bushings during corrective maintenance (CM) in January 2005. The apparent cause of the event was inadequate maintenance due to either damage to the valve internals during valve reassembly in November 2004, or the relocation of an undetected pre-existing flaw (burr) to a more problematic location during or following the November 2004 reassembly. During the November 2004 inspection, the hinge was discovered to be slightly bound up. Maintenance concluded the hang up was due to rust on the hinge pin and cleaned the pin. Free swing of the valve disc (freedom of movement check) was confirmed as part of the maintenance and Maintenance personnel believed the hang up to be corrected. No burr was identified during the November 2004 valve maintenance however, a burr on one of the bushings that interfered with the valve hinge pin was identified in January 2005 following disassembly as a result of the quarterly surveillance test problem during PT-Q31A.
Contributing causes (CC) were: CC-1: insufficient valve maintenance procedure/vendor instructions. The PM procedure does not require any measurements of clearances between valve hinge pin and bushing and the vendor manual contains no requirements for checking tolerances at the critical hinge pin and bushing interface, CC-2: Incorrect part. The wrong plug gasket was used. During the CM in January 2005, spiral wound gaskets were found but the vendor drawing calls for soft iron gaskets. An incorrect gasket could shift the bushing inward affecting clearance between the bushing and the lever, CC-3: The PWT did not reveal a failure of the valve to close. The ACCW pump discharge check valves have safety functions in both the open and closed positions and are components included in the In-service Test (IST) program. The check valve open position is verified by testing for proper flow. The closed check valve position is verified by observing that the respective ACCW pump does not rotate in reverse when the ACCW pump in the parallel flow path is operating. Had the differential head of the parallel pump been measured during the PWT then the problem with the 22 ACCW pump discharge check valve 755A would likely have been revealed assuming the condition existed at that time. Only observing pump shaft rotation may not be adequate due to the tight operating band of the ACCW pumps and because the Component Cooling Water System (CCWS) discharge pressure on the suction of a non operating ACCW pump may retard pump reverse rotation for an unseated ACCW pump discharge check valve when the opposite ACCW pump is operating.
CORRECTIVE ACTIONS
The following corrective actions have been or will be performed under the CAP to address the causes of this event and prevent recurrence.
- The 22 ACCW Pump discharge check valve 755A was disassembled, cleaned, deburred and reassembled [Work Order (WO) IP2-05-10855].
- The 22 ACCW Pump was retested successfully, with the discharge check valve 755A performing its function in the open (SC-0) and closed positions (SC-C) (WO IP2-05-10873).
- Maintenance personnel were coached on management's expectation on attention to detail. Staff meetings with Component Engineering discussed the event and lessons learned. Similar meetings were held with Maintenance personnel.
- Notes were added to the next scheduled inspections of valves 755B and 790 with the requirement for full disassembly.
- The proper clearances for these valves were obtained from the manufacturer.
- Disassemble and inspect 21 ACCW Pump discharge check valve 755B (WO IP2-03- 28995) and excess letdown heat exchanger inlet check valve 790 (WO IP2-05- 11999) to check for clearances and correct as required (full disassembly and inspection of components for defects and proper clearances). Scheduled completion is by next refueling outage in spring 2006 (2R17).
- Procedure DC-907, "ASME Code Section XI-Repair/Replacement Program," Section XI Traveler, which has IST Engineer directions for particular retest activities for Code class components (such as the ACCW check valves) was appropriately coded in the IST Database to ensure IST Engineers provide the requirement to verify that the head of the opposite parallel pump remains in the normal expected acceptable range as part of the PWT requirement.
- A review was performed of applicable Preventive Maintenance (PM) procedures to determine if any additional information should be included to check on potential clearance problems. Various Velan swing check valve procedures were combined into one site PM procedure. Included in the site PM was a step requiring the valve to be cycled several times to ensure that no binding or sticking occurs.
- The response to the condition report for this event was added to the applicable PM procedures for operating experience (OE) (procedures 2-VLV-012-VCK and 3-VLV-017-GEN) and their successor document, 0-VLV-423-VCK.
EVENT ANALYSIS
The event is reportable under 10CFR50.73(a)(2)(i)(B). The licensee shall report any operation or condition which was prohibited by the plant TS. This event meets the reporting criteria because one or more trains of ECCS were declared inoperable for greater than the TS allowed completion time of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br /> for TS LCO 3.5.2. The 22 ACCW pump discharge check valve failure to seat could result in less than design cooling flow to the containment recirculation pump motor coolers for a single failure of 22 ACCW pump to start. The 21 ACCW pump was demonstrated operable by quarterly surveillance test PT-Q31A on October 14, 2004, and a 10 year PM was performed on ACCW pump 22 check valve 755A in November 2004, during refueling outage cycle 16. After the inspection, post work testing was performed with a test run of the 22 ACCW pump (verified check valve 755A opened), and run of the redundant 21 ACCW pump with a check of the 22 ACCW pump for shaft rotation. However, during the 21 ACCW pump test run, the discharge head was not measured to verify normal acceptable operating parameters. Therefore, the condition of the valve could have existed since the completion of the refueling outage inspection/maintenance and its PWT on November 5, 2004. Operability of the recirculation pumps and therefore the cooling support from the ACCW pumps, was not required until the plant returned from refueling to Mode 3 (hot shutdown), which was on November 19, 2004, at 12;30 hours3.472222e-4 days <br />0.00833 hours <br />4.960317e-5 weeks <br />1.1415e-5 months <br />. The possible time of inoperability would be from November 19, 2004, through January 19, 2005, a duration which exceeds the TS allowed completion time of 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />. During the period of potential 22 ACCW discharge check valve inoperability, neither the 22 ACCW pump or its power supply (Emergency Diesel Generator 23) were unavailable.
Therefore both ACCW pumps would have started on demand and provided cooling water flow to the recirculation pumps motor coolers.
PAST SIMILAR EVENTS
A review of the past two years of Licensee Event Reports (LERs) for events that involved inoperable components that exceeded the TS allowed completion time identified LER 2003-002 which reported a TS prohibited condition on May 20, 2003, due to the unavailability of Boric Acid Storage for greater than the TS allowed outage time. The event was an inability to verify a boric acid flow path to the core when using the 22 boric acid transfer pump aligned to the blender.
The cause was a diaphragm valve that had failed due to incorrect installation when assembling the valve following maintenance. The event is similar in that improper maintenance resulted in an inoperable TS required component whose inoperability exceeded the TS allowed outage time. The corrective actions for the event reported in LER 2003-002 would not have prevented this event because the valve types and system were different.
SAFETY SIGNIFICANCE
This event had no effect on the health and safety of the public.
There were no actual safety consequences for the event because there were no accidents or transients requiring the ACCW or ECCS.
There were no significant potential safety consequences of this event under reasonable and credible alternative conditions. Had a LOCA occurred during the time the 22 ACCW pump discharge check valve was inoperable, adequate cooling of the containment recirculation pumps motor coolers may not have been available assuming a failure or unavailability of the 22 ACCW pump. However, either one of the two RHR pumps can also be used to provide backup for recirculation. The containment recirculation pump's motor coolers are not required in the injection phase and only needed for long term cooling in the recirculation phase. For this event, the plant retained adequate ECCS component availability to meet minimum safeguards requirements. A risk assessment of the condition was performed assuming the inoperable 22 ACCW pump discharge check valve would have resulted in an inability of 21 ACCW pump to provide adequate flow assuming a failure of 22 ACCW pump. The valve was considered operable prior to the PM on November 3, 2004, and assumed to have been inoperable after completion of the PWT on November 5, 2004. The valve was returned to service during a refueling outage in a mode which the TS do not require the recirculation trains to be operable. In accordance with TS 3.5.2, ECCS, three trains of ECCS are required to be operable in Modes 1, 2 and 3. The plant transitioned to mode 3 at approximately 12:30 hours on November 19, 2004. The 22 ACCW pump discharge check valve was considered to be potentially inoperable from November 19, 2004, to January 19, 2005, a total exposure time of 61 days.
The Incremental Core Damage Probability (ICDP) was determined to be 4.58E-7 using the annual average model with a change in core damage frequency (CCDF) of 2.74E-6 per year. Therefore, risk for this event is considered low.
l'itil.t-V1-11V1 JOOAI I ZUL111