05000321/LER-2004-001
| Edwin I. Hatch Nuclear Plant - Unit 1 | |
| Event date: | 2-3-2004 |
|---|---|
| Report date: | 03-30-2004 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(v), Loss of Safety Function 10 CFR 50.73(a)(2)(i) |
| 3212004001R00 - NRC Website | |
FACILITY NAME (1)
DOCKET
05000-321 LER NUMBER (6) 2004 -- 001 -- 0
PLANT AND SYSTEM IDENTIFICATION
General Electric - Boiling Water Reactor Energy Industry Identification System codes appear in the text as (EIIS Code XX).
DESCRIPTION OF EVENT
On 2/03/2004 at 1400 ET, Unit 1 was in the Run mode at a power level of approximately 2511 CMWT (91 percent rated thermal power). At that time, an evaluation revealed that the Net Positive Suction Head requirements (NPSHr) for the 1C Residual Heat Removal Service Water (RHRSW) pump that had been in service from 12/20/2002 until 11/17/2003 were not met from 9/14/2003 until Operations removed the 1C RHRSW pump for maintenance on 11/17/2003 at 0325 ET. During this period of time the river level went below 65.9 feet. Corporate Engineering was requested to evaluate the 1C RHRSW pump because of the problems that had been encountered with it. The Corporate Engineering evaluation concluded that the 1C RHRSW pump could not meet its NPSHr with the river level below approximately 65.9 feet. A "composite pump" was used to replace the 1C RHRSW pump. This pump was installed and the operability test completed on 12/09/2003. The duration of this condition exceeded the 30 days allowed by the plant's Technical Specifications 3.7.1 Condition A for a single RHRSW pump to be inoperable. The Residual Heat Removal Service Water pumps are used in the long-term containment heat removal mode.
A "cutter pump" (Johnston Job Number 1507) design was installed for the 1C RHRSW pump on 12/20/2002 as part of the "cutter pump" modifications being implemented as corrective action for a long-standing equipment reliability problem. The pump consisted of a new first stage impeller and suction bell with succeeding stages rebuilt by the vendor. The suction bell modifications were to be typical of cutter pumps. That is, the outer dimensions and inlet cross-sectional area of the bell were supposed to be the same as a conventional pump. The sole enhancement was supposed to be the height of the cruciform struts which support the central shaft bearing. The strut was heightened in order to reduce the clearance between the struts and the rotating impeller, which provides the "cutter" action.
The sole enhancement in the first stage impeller was supposed to be a sharpened leading edge of each impeller blade which improves the cutting action. During the development of the cutter modification the vendor was asked about the potential for changes in hydraulic performance due to the cutter modification. The vendor provided documentation stating that the cutter pump modification had no adverse impact on hydraulic performance. The vendor did not perform full-speed testing of the cutter modified pumps to ensure that hydraulic performance, including NPSHr, remained unaffected by either modification. It was not clear from the documentation provided by the vendor that full speed testing had not been performed. Full-speed testing should have been performed since NPSHr is a critical component of hydraulic performance and is determined in part by fluid velocity at the pump suction.
The statement that the cutter pump modification had no adverse impact on hydraulic performance is now known to have been incorrect.
FACILITY NAME (1) DOCKET LER NUMBER (6) PAGE (3) When this pump was installed, the initial in-service testing (IST) performance showed that the pump - produced 379 psid of total dynamic head (TDH). The acceptance criterion per 34SV-E1 1-004-0 is 366.75 psid (847 ft.-head.) at 4000 gpm, which was the minimum acceptance criterion prior to the advent of standardized Technical Specifications. The FSAR design point is 955 ft.-head. at 4000 gpm, which is the same as the vendor's design point. Although the performance of this pump met the acceptance criteria, it was lower than the design point and lower than other, newly installed pumps.
Given the normal, expected wear of raw-water system pumps due to abrasives in the water, the longevity of the pump would have been only three or four years before output fell below the minimum acceptable.
Operations removed the 1C RHRSW pump for maintenance on 11/17/2003 at 0325 ET. The system engineer witnessed two operability runs on this pump and heard sounds which indicated cavitation. In particular, during a run performed on the evening of 11/20/2003, the pump was unable to exceed 4000 gpm regardless of the position of the system flow control valve, 1E11-F068A. Based on the decline in performance and the sounds the pump was making, it was postulated that the pump had ingested foreign material. However, when divers entered the suction bay, they found that no foreign material was present in the pump suction area. Consequently, the decision was made to remove the pump and look for foreign material when it was disassembled.
Pump #1507 was removed and replaced with "cutter pump" #1508 from warehouse stock. Upon testing, pump #1508 performed worse than #1507. On 11/26/2003, pump #1508 was able to produce 65 psig at a flow rate of 3800 gpm. Due to audible indications of pump distress (a "rushing" sound plus indications that the stuffing box bushing was wiped), the engineer requested that the pump be shut down.
Subsequently, pump #1508 was returned to the vendor where it was disassembled and inspected side by side with pump #1507. When this was done, it was evident that the impellers in stage 1 and stage 4 of pump #1508 were different from the impellers in pump #1507. It was then decided to replace these two impellers with two from pump #1507. In addition, the "cutter" style suction end bell was replaced with a conventional end bell. This "composite" pump was installed and tested satisfactorily on 12/09/2003.
The RHRSW System is designed to provide cooling water for the Residual Heat Removal (RHR) System heat exchangers, required for a safe reactor shutdown following a Design Basis Accident (DBA) or transient. The RHRSW System is operated whenever the RHR heat exchangers are required to operate in the shutdown cooling mode or in the suppression pool cooling or spray mode of the RHR System. The RHRSW System consists of two independent and redundant subsystems. Each subsystem is made up of a header, two 4000 gpm pumps, a suction source, valves, piping, heat exchanger, and associated instrumentation. Either of the two subsystems is capable of providing the required cooling capacity with two pumps operating to maintain safe shutdown conditions. A review of the Required Action Sheet (RAS) entries and clearances for Unit 1 during this current operating FACILITY NAME (1) DOCKET LER NUMBER (6) PAGE (3) cycle (since completion of the Unit 1 Spring 2002 refueling outage) was performed. This review determined that there were always at least two RHRSW pumps that were operable (not counting the 1C RHRSW pump) since the last refueling outage with the exception of approximately an 11 hour1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> period that began at 1744 EST on 11/3/2003 and ended at 0426 EST on 11/4/2003. During this 11 hour1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> period of time, both the 1B and 1D RHRSW pumps were tagged out for an RHR system outage. A review of the clearance removing these pumps from service determined that it would have taken approximately 20 to 30 minutes for these pumps to have been restored. During the rest of the operating cycle, at least 2 pumps were always operable. If one pump in the "B" RHRSW subsystem was not operable, it was confirmed that the lA RHRSW pump was operable. The lA RHRSW pump could be used in conjunction with one of the two "B" RHRSW subsystem pumps by opening the RHRSW system crosstie valves 1E11-F119A and 1E11-F119B to connect the two RHRSW divisions.
CAUSE OF EVENT
The pump manufacturer did not ensure that hydraulic performance remained unaffected by design changes including changes in material (i.e., from bronze to steel impellers) and construction (i.e., modifications to impellers and suction bells arising from the weed cutter modification). The pump manufacturer did not ensure that component dimensions were consistent with design and with each other. SNC purchase, acceptance, and verification processes did not result in the supplier performing all the steps desired to ensure expected pump performance.
REPORTABILITY ANALYSIS AND SAFETY ASSESSMENT
This report is required by 10 CFR 50.73(a)(2)(v) because a condition existed that alone could have prevented the fulfillment of the safety function of a system needed to remove residual heat and to mitigate the consequences of an accident. A Corporate Engineering evaluation concluded that the 1C RHRSW pump installed 12/20/2002 could not meet its NPSHr with the river level below approximately 65.9 feet. Consequently, with the 1C Residual Heat Removal Service Water (RHRSW) unable to meet its NPSHr requirements, in conjunction with the 1B and 1D RHRSW pumps being removed from service to support a system outage, the ability of the RHRSW system to fulfill its safety function of removing containment heat following a Design Basis Loss-of-Coolant Accident could have been impaired. This report is also required by 10 CFR 50.73(a)(2)(i) because a condition existed that was prohibited by the plant's Technical Specifications. The time the river level was below 65.9 feet and the Net Positive Suction Head requirements (NPSHr) for the 1C Residual Heat Removal Service Water (RHRSW) pump was not met exceeded the allowable out-of-service times given in the TS 3.7.1 Condition A for an inoperable RHRSW pump.
The RHRSW system is designed to provide cooling water to the RHR system heat exchangers which are required for a safe reactor shutdown following a design basis accident or transient. The RHRSW system consists of two independent and redundant subsystems. Each subsystem consists of a header, two 4000 gallon-per-minute pumps, a suction source, valves, piping, heat exchanger, and associated FACILITY NAME (1) DOCKET LER NUMBER (6) 05000-321 2004 -- 001 -- 0 instrumentation. Either of the two subsystems is capable of providing the required cooling capacity with two pumps operating. The two subsystems are separated from each other by two normally closed, motor operated crosstie valves in series.
The RHRSW system is initiated manually and removes heat from the suppression pool to limit the suppression pool temperature and primary containment pressure following a loss-of-coolant accident.
This ensures the primary containment can perform its function of limiting the release of radioactive materials to the environment and the emergency core cooling system pumps, whose suction source is the suppression pool, have adequate net positive suction head. As discussed in the Unit 1 Final Safety Analysis Report, manual initiation of an RHRSW subsystem and the associated RHR subsystem are assumed to occur ten minutes after the onset of the accident. The RHRSW flow rate assumed in the analyses is 4000 gallons per minute per pump with two pumps providing cooling water to one RHR heat exchanger. In this case, the maximum suppression pool water temperature is approximately 210°F which is well below the design temperature of 281°F.
In this event, there were always at least two RHRSW pumps operable (not counting the 1C RHRSW pump) since the last refueling outage with the exception of approximately an 11 hour1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> period that began at 1744 EST on 11/3/2003 and ended at 0426 EST on 11/4/2003. During this 11 hour1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br /> period of time, both the 1B and 1D RHRSW pumps were tagged out for an RHR system outage. A review of the clearance removing these pumps from service determined that it would have taken approximately 20 to 30 minutes to return the pumps to an operable status. The plant's TS 3.7.1 Condition D does govern this condition and requires that if both RHRSW subsystems are inoperable, then one subsystem must be restored to an operable status within 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />. The 8 hour9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> completion time for restoring one subsystem is based on the completion times provided for the RHR suppression pool cooling and spray functions which state that it is acceptable due to the low probability of a DBA and because alternative methods to remove heat from primary containment are available. If at least one subsystem is not restored within 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />, then TS 3.7.1 Condition E is entered. This condition requires that the plant must be placed in Mode 3 within 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br />. This condition would have been exited 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> after entry because at that time the 1B and 1D pumps were restored. During the rest of the operating cycle, at least 2 pumps were always operable. If one pump in the "B" RHRSW subsystem was not operable, it was confirmed that the 1A RHRSW pump was operable. The 1A RHRSW pump could be used in conjunction with one of the two "B" RHRSW subsystem pumps by opening the RHRSW system crosstie valves 1E11-F119A and 1E11-F119B to connect the two RHRSW divisions. With two RHRSW pumps operable the RHRSW system is capable of fulfilling its safety function of long-term containment heat removal during the Design Basis Loss-of-Coolant Accident. Therefore, it may be concluded that this event did not adversely impact nuclear safety or the health and safety of the public and did not represent a substantial safety hazard., This analysis applies to all operating conditions and power levels where a Design Basis Loss-of-Coolant Accident may occur.
FACILITY NAME (1) DOCKET LER NUMBER (6) PAGE (3)
CORRECTIVE ACTIONS
On 12/09/2003 the 1C RHRSW pump was replaced with a "composite pump" and successfully tested.
Generic vendor data indicates that all Unit 1 and Unit 2 RHRSW pumps are acceptable for operation down to a river level of 60.7 feet, which is the Unit 1 and 2 Technical Specifications surveillance limit for the Plant Service Water/Ultimate Heat Sink specification 3.7.2. Additionally, flow testing was performed at the manufacturer's facility using various combinations of modified and non-modified equipment. This testing demonstrated improved performance when compared with the performance of the original #1508 pump. However, because of the previously noted discrepancies between design and installed equipment, additional actions are being implemented as follows: Operating guidance has been established to perform additional RHRSW pump testing if the river level decreases below 61.8 feet for all RHRSW pumps, except for 1C RHRSW. (Actual plant data exists for all RHRSW pumps to a river level of 61.8 feet, except 1C RHRSW). Operating guidance has been established on the 1C pump to test it at river levels of 65, 64, 63, and 62 feet.
The purchase specification for RHRSW pumps has been revised by Component Engineering to require that on the next RHRSW pump purchased, the vendor shall test the fully assembled pump at conditions which will ensure design requirements are met. This action will ensure that manufacturing tolerances or dimensional problems, if any exist, have no adverse impact on pump performance.
To prevent recurrence of this event with this vendor, a detailed repair specification shall be written that establishes repair guidelines and performance test requirements for the RHRSW pumps. This repair specification contains testing requirements and shall be used to manage changes such as new purchase orders, repairs by alternative vendors, and design modifications. This repair specification will be completed by September 2004.
To prevent recurrence of this event should a major design change take place, the Supplemental Engineering Guidance Document will be revised to require that Component Engineering Group be consulted for pump testing requirements whenever a safety related pump undergoes a design change.
This revision will be completed by July 2004.
ADDITIONAL INFORMATION
An LER 1-03-003 was written on the 1C RHRSW pump being inoperable during this same time frame for a problem with the seismic supporting capability of this pump. That LER identified that on 11/17/2003 personnel replacing corroded anchor bolt nuts securing the 1C Residual Heat Removal Service Water (RHRSW) pump determined corrosion on two of the nuts was so extensive that the adequacy of the seismic supporting capability of the nuts for that pump was questionable for the design loads. This postulated seismic failure of the 1C RHRSW pump could have prevented this pump from being able to perform its design function during a seismic event. Because the corrosion of the nuts FACILITY NAME (1)
DOCKET
05000-321 LER NUMBER (6) 2004 -- 001 -- 0 occurred over time, and based upon the extent of the corrosion found on the nuts securing the pump, it was concluded that this condition existed for a period of time longer than the 30 days allowed by the plant's Technical Specifications 3.7.1 Condition A.
No systems other than those previously described in this report were affected by this event.
This LER does not contain any permanent licensing commitments.
There were no previous similar events reported in the past two years in which the plant entered a condition prohibited by Technical Specifications as the result of a vendor furnishing equipment that failed to meet all of its procurement specification requirements.
Failed Component Information:
Master Parts List: 1E11-0001C Manufacturer: Johnston Manufacturer Code: J105 Model Number: 18DC Type: RHR Service Water Pump EIIS System Code: BS EIIS Component Code: P Root Cause Code: B Reportable to EPIX: Yes