ML11336A369
| ML11336A369 | |
| Person / Time | |
|---|---|
| Site: | Oconee, Nine Mile Point, Palo Verde, Byron, Braidwood, Susquehanna, Waterford, South Texas  |
| Issue date: | 12/01/2011 |
| From: | Stepantschenk P Engine Systems |
| To: | Office of Nuclear Reactor Regulation |
| References | |
| 45459 | |
| Download: ML11336A369 (11) | |
Text
1210112011 U.S. NuclearRegulatory Commission OperationsCenter Event Report Page I General Information (PAR) Event# 45459 Rep Org: ENGINE SYSTEMS, INC Notification Date / Time: 10/23/2009 16:32 (EST)
Supplier: ENGINE SYSTEMS, INC Event Date / Time: 08/24/2009 (EDT)
Last Modification: 12/01/2011 Region: 1 Docket #:
City: ROCKY MOUNT Agreement State: Yes County: License #:
State: NC NRC Notified by: PAUL STEPANTSCHENK Notifications: NEIL OKEEFE R4DO HQ Ops Officer: BILL HUFFMAN J.THORP (e-mail) NRR Emergency Class: NON EMERGENCY O.TABATABAI (e-mail) NRO 10 CFR Section:
21.21 UNSPECIFIED PARAGRAPH INTERIM REPORT ON THERMOSTATIC VALVE FAILURE ON PALO VERDE EDG "This interim report is being issued because Engine Systems, Inc. (ESI) is not able to complete an evaluation of an identified deviation within the 60 day requirement of 10CFR21.21. The evaluation is expected to be completed no later than November 30, 2009.
"ESI began an evaluation of a thermostatic valve element failure on August 24, 2009. Palo Verde Nuclear Plant notified ESI of the failure as a result of a failure analysis they were performing on a thermostatic valve that had been removed from the lube oil system of their 2A-EDG. The element was in service since April 2008 and Palo Verde verified operation of the element prior to installation.
"The Palo Verde failure analysis determined that one of two elements within the valve was defective. The element failure was attributed to wax leakage past the diaphragm seal on one of two power pills within the element.
Evidence of mechanical binding of the piston is believed to have caused the wax leakage. If the piston was jammed, the expanding wax could have over pressurized the diaphragm seal leading to wax leakage. The failure analysis noted the following to support piston binding:
- The piston was initially difficult to remove from its guide tube.
- A gouge was observed on the piston surface.
- The rubber plug within the power pill exhibited brass machining chip debris.
"ESI has been coordinating with Palo Verde and the manufacturer (AMOT) to complete our evaluation and to determine if this is a generic issue or if it is an isolated incident.
"To date, no other similar failures with AMOT thermostatic valves have been reported to ESI."
(~J/e7
12/01/2011 U.S. Nuclear Regulatory Commission OperationsCenter Event Report Pare 2 General Information (PAR) Event# 45459 Palo Verde has Cooper Bessemer KSV-20T diesel engines that use one 6" thermostatic valve in the engine jacket water system and one in the engine lube oil system to regulate system temperatures during engine operation. The thermostatic valve is an AMOT model 6HAS. The AMOT thermostatic valve element (P/N 9760X) is the defective part. ESI did not provide any information on other nuclear power plants that have EDGs that utilize this model thermostatic valve.
- *
- UPDATE RECEIVED VIA EMAIL FROM PAUL STEPANTSCHENKO TO DONG PARK AT 1642 EST ON 12/01/09 * * *
"This report is a follow-up to an interim report (10CFR21-0098-INT) issued by Engine Systems, Inc. (ESI) on 10/23/09 which identified a deviation with an Amot thermostatic valve element. The interim report was issued because ESI was not able to complete the evaluation within the 60 day requirement of 10CFR21. The evaluation was completed on 11/30/09 and the deviation was determined be a reportable defect as by defined by 10CFR21.
"The Exelon analysis also reports that similar brass machining debris was observed on the plugs from the other three power pills to varying degrees. None of the stems of these pills displayed evidence of gouging or binding.
"To date, no other similar failures with Amot thermostatic valves have been reported to ESI.
"ESI has contacted the valve manufacturer (Amot) to discuss these findings. A copy of the Palo Verde failure analysis and eleven (11) element assemblies from ESI inventory were sent to Amot for evaluation. The following elements were sent to Amot for evaluation:
"Qty. 8: PIN 9760 X-170' (CES PIN 2-05V-419-107)
"Qty. 3: PIN 9760 X-160' (CES PIN 2-05V-419-109)
"Both part number elements are the same except for the temperature setting ('-170' indicates 170°F nominal and 160' indicates 160'F nominal).
"Upon completion of their evaluation, Amot has reported the following:
"Fine shavings/powder of brass was observed in some of the element pills.
"None of the pill stems had any evidence of gouging.
"The pills used in the 9760X elements are made by converting another part number pill. This conversion consists of removing the stem from the pill and performing some machining. Amot believes the brass debris may have entered the pill as a result of this conversion process.
"Amot has not made any changes to this conversion process in recent history and has not had reports of similar problems with these elements.
"Machining debris, while undesirable, was evident in other pills which did not exhibit any operability issues; therefore this is not believed to be the cause of the pill failure.
"The primary cause of the failure is believed to be the gouge found in the pill stem. The gouge could have occurred during the conversion process as the stem is removed and handled at that time.
"As a precaution, Amot has made changes to their conversion process for this pill. The drilling fixture was modified to eliminate the possibility of chips entering the pill during the machining operation. This change was made effective 10/22/09.
1210112011 U.S. Nuclear Regulatory Commission OperationsCenter Event Report Paea 3 General Information (PAR) Event# 45459 "A listing of users with the thermostatic valves that contain the Amot 9760X element is provided in the table below.
"Site - Thermostatic Valve - System:
"Braidwood - 6HAS - Lube "Byron - 6HAS - Lube "Nine Mile Point - 6HAS- Lube "Oconee - 4HAS & 6HAS - Water "Palo Verde - 6HAS - Lube & Water "South Texas Project - 6HAS - Lube & Water "Susquehanna - 6HAS - Lube &Water "Waterford - 5HAS & 6HAS - Lube & Water "Corrective Action: The element failure at Palo Verde is considered to be an isolated incident related to a gouge in the pill stem. Thus, there is no recommended corrective action for users of the Amot 9760X element. The evaluation also indicated a weakness in Amot's manufacturing process for the element pill which introduced machining debris. While not believed to be the cause of the Palo Verde element failure, machining debris within the element pill is undesirable and increases the potential for failure in the future. Users with thermostatic valves containing Amot PIN 9760X elements should be aware of this issue so that they can monitor their systems for any indications of thermostat element problems.
Notified R1 DO (Holody), R2DO (Guthrie), R3DO (Riemer), R4DO (Deese), NRR (Thorp) via e-mail, NRO (Tabatabai) via email.
- *
- UPDATE RECEIVED VIA FAX FROM TOM HORNER TO DONG PARK AT 1613 EST ON 04/01/11 ***
Two sentences were added to address the safety hazard which is created or could be created by this defect.
"This defect could affect operability of the thermostatic valve within the diesel engine cooling water and/or lube oil system, resulting in elevated fluid system temperatures during engine operation. Engine performance and/or load carrying capability could be impacted with the possibility of eventual engine failure, thereby preventing the emergency diesel generator from performing its safety related function."
Notified R1 DO (Powell), R2DO (Sykes), R3DO (Peterson), R4DO (Lantz), PART 21 GROUP via e-mail.
- *
- UPDATE RECEIVED VIA FAX FROM TOM HORNER TO VINCE KLCO AT 1611 EST ON 12/01/11**
The report was updated to revise part numbers for Oconee, Waterford and Laguna Verde (Mexico). Affected users added include the following: Susquehanna 5th EDG and Ergytech/lberdroia (Spain).
Notified R1 DO (Schmidt), R2DO (Desai), R3DO (Riemer), R4DO (Farnholtz), PART 21 GROUP via e-mail.
ESI) ENGINE SYSTEMS, INC.
175 Freight Road, Rocky Mount, NC 27804 Telephone', 252/977-2720 Fax: 252/446-1134 TELEFAX Date: December 1, 2011 Company: NRC Operations Center Fax Number: 301/816-5151 Verification No.: 301/816-5100
Reference:
Report No. 1DCFR21-0098, Rev. 3 From: Tom Homer Page: 1 of 8
Dear Sir:
Following this cover is a copy of our report 100CFR21-0098, Rev. 3, for a 10CFR21 reportable notification about an Amot thermostatic valve element, P/N 9760X. Revision 3 of this report is being issued to update affected users and part numbers.
A copy of the revised report will be mailed to the NRC Document Control Desk and to our affected nuclear customers.
Should you have questions, please let us know.
Sincerely, ENGINE SYSTEMS, INC.
Tom Homer Quality Assurance Manager Quality Performance With Pride
a ENGINE SYSTEMS, INC.
175 Freight Road. Rocky Mount, NC 27804 Telephone: 252/977-2720 Fax: 252/446-1134 Report No. 1 OCFR21 -009a Rev. 0: 12/01/09 Rev. 1: 03/30/11 Rev. 2; 07/19/11 Rev. 3: 12/01/11 10CFR21 REPORTING OF DEFECTS AND NON-COMPLIANCE COMPONENT: Amot thermostatic valve element P/N: 9760X SYSTEM: Emergency Diesel Generator - lube oil & jacket water system CONCLUSION: Reportable in accordance with IOCFR21 Prepared By: Date: /2// -/
-' Engineering Manager Reviewed By: Date: d--'
Quality Assurance Manager Quality Performance With Pride
Report No. 10CFR21-0098 Record of Revisions Page: 1 of 1 REV DATE PAGEL DESCRIPTION 0 12/01/09 Initial issue.
1 03/30/11 1 Under Purpose, added the last 2 sentences to address the safety hazard which is created or could be created by this defect.
2 07/19/11 Updated report in its entirety to address a 2 nd similar failure at STP.
3 12/01/11 Report revised to update affected users and part numbers as follows, balance of report remains unchanged:
1 Under Component, removed P/N 9760X-180 from table.
Under Purpose, changed "is being issued" to "was issued" in the first sentence.
3 In third paragraph, changed "in March 2011" to "on March 16, 2011:.
5 In Listing of Users with 9760X Elements:
- Oconee: changed P/N from "9760X-1 60" to "9760X-170"
- Susquehanna: added "5th EDG" to site name
- Waterford: added P/N 2-05V-419-111
- Laguna Verde: changed P/N from "n/a" to "9760X-170"
- Ergytech/lberdrola: added listing to table with corresponding note.
Under Corrective Action, changed "March 2011" to "March 16, 2011".
Report No. 10CFR21-0098 Revision: a Date: 12/01/11 Page: 1 of 5 COMPONENT:
Amot thermostatic valve element Amot P/N: 9760X ESI has provided the following 9760X elements under the Amot or Cooper-Bessemer part numbers:
Amot PIN C-B PIN Nominal Setting 9760X-160 2-05V-419-109 1600 F 9760X-1 65 2-05V-419-108 165aF 9760X-170 2-05V-419-107 1700 F 9760X-175 2-05V-419-111 175 0F PURPOSE:
Revision 2 of this report was issued to address another similar failure of an Amot 9760X series thermostatic valve element. The first failure was reported by Palo Verde and the second failure was reported by South Texas Project (STP). Both failures pertain to an Amot P/N 9760X-160 thermostatic element within the lube oil temperature regulating valve of their Cooper-Bessemer KSV standby diesel generators. This element carries Cooper-Bessemer P/N 2-05V-419-109. The evaluation of the Palo Verde failure acknowledged machining debris was found within the power pills; however, the failure was determined to be an isolated incident caused by a gouge on the power pill piston stem. The failed STP element did not have signs of gouging on the piston stem and machining debris was again observed. As a result of this second incident, ESI is revising the root cause to be machining debris and to notify affected users accordingly.
IMPACT:
These elements are used in some Cooper-Bessemer KSV diesel engine jacket water and/or lube oil systems to maintain proper system temperatures during engine operation. A very limited number of EMD diesel engines also use this element in the jacket water system. Failure of a thermostatic element would impact operability of the temperature regulating valve within the diesel engine cooling water and/or lube oil system, resulting in elevated fluid system temperatures during engine operation. Engine performance and/or load carrying capability could be impacted with the possibility of eventual engine failure, thereby preventing the emergency diesel generator from performing its safety related function.
SUMMARY
Palo Verde Nuclear Plant notified ESI of a failure of a thermostatic valve element (P/N 2-05V-219-109) as a result of a failure analysis they were performing on two (2) thermostatic elements that had been removed from the lube oil system of their 2A-EDG. Exelon Power Labs performed the failure analysis for Palo Verde (Exelon Power Labs report APS-56396 dated 9/15/09). The element was in service since April 2008 and Palo Verde verified operation of the element prior to installation. ESI began a deviation evaluation of this failure on 8/24/09, issued an interim report (10CFR21-0098-INT) on 10/23/09 and completed the evaluation on 11/30/09.
Report No. 10CFR21-0098 Revision: 3 Date: 12/01/11 Page: 2 of 5 STP provided two (2) thermostatic elements (P/N 2-05V-419-109) to ESI for failure analysis after observing increasing temperature trends within their diesel generator lube oil. Both elements were removed from the lube oil temperature regulating valve of their standby diesel generator SBDG 13.
STP reported they had installed these elements as part of a normal 5 year replacement during the week of 11/30/2009 and they were removed from service on 08/11/2010. The elements were later sent to ESI for a failure analysis. A deviation evaluation began on 5/26/11 and was completed on 7/15/11.
The failure analyses for the Palo Verde and STP elements yielded very similar findings:
- Only one of the two elements within the valve did not function properly.
" Only the lower power pill (one of the two power pills within the element) had failed,
- Both of the failed power pills were from date code 07213.
- The piston of the failed pill was initially difficult to remove from its guide tube.
- A gouge was observed on the piston surface of the failed pill (Palo Verde element only).
- Brass machining debris was found within in the piston guide and on the rubber plug of the failed pill.
" Brass machining debris was also observed within the other three power pills to varying degrees; all of these pills continued to function properly. None of the stems of these pills displayed evidence of gouging or binding.
- The element failure was attributed to wax leakage past the diaphragm seal on one of two power pills within the element. Evidence of mechanical binding of the piston is believed to have caused the wax leakage. If the piston was jammed, the expanding wax could have over-pressurized the diaphragm seal leading to wax leakage.
During evaluation of the Palo Verde element, ESI sent a copy of the Exelon Power Labs failure analysis report and eleven (11) 9760X element assemblies from ESI inventory to Amot for evaluation. Eight of the elements were P/N 9760X-170 and three (3) were P/N 9760X-160. Both part numbers are the same except for the temperature setting (the last 3 digits represent the nominal temperature setting in degrees F) which is controlled by the wax mixture. Upon completion of their evaluation, Amot reported the following;
- Fine shavings/powder of brass was observed in some of the element pills.
- None of the pill stems had any evidence of gouging.
- The pills used in the 9760X elements are made by converting another part number pill.
This conversion involves machining the piston and the piston guide tube to the correct length for the 9760X series elements. The machining debris is believed to have entered the guide tube during this conversion process.
" Prior to the Palo Verde failure, Amot had not made any changes to this conversion process in recent history and had no reports of similar problems with these elements.
- Machining debris, while undesirable, was evident in other pills which did not exhibit any operability issues; therefore this was not believed to be the cause of the pill failure.
- The primary cause of the failure was believed to be the gouge found in the pill stem. The gouge could have occurred during the conversion process as the stem is removed and handled at that time.
Report No, 100FR21-0098 Revision: 3 Date: 12/01/11 Page: 3 of 5 During the 2009 evaluation of the Palo Verde element failure, ESI consulted with Amot to develop corrective actions to address this issue. Amot implemented changes to their machining process to eliminate entrance of debris during the machining process of the guide tube. Amot implemented this process change on 10/22/2009.
As a result of this (STP) failure, ESI met with Amot to further address this issue. Amot has initiated additional training with their personnel to ensure the revised machining process is implemented to prevent the entrance of machining debris.
In addition to the Amot process change, ESI developed an inspection procedure for all 9760X elements (including P/N 2-05V-419-109). This procedure was implemented within ESI's commercial grade dedication process on March 16, 2011 and includes removal of the upper and lower power pill pistons to facilitate inspection of the piston guide tube for machining debris.
During this process, the pistons are also inspected for damage (such as gouges). The elements are reassembled and then functionally tested in a temperature bath to verify proper operability. All 9760X elements dedicated by ESI are subjected to this inspection and test process.
DISCUSSION:
The Amot type 9760X elements are used in the Amot type "H"thermostatic valve. These valves are used in some Cooper-Bessemer KSV diesel engine jacket water and/or lube oil systems to maintain proper system temperatures during engine operation. A very limited number of EMO diesel engines also use the type "H"valve in the jacket water system.
Each valve uses 2 thermostatic elements, Amot type 9760X (Figure 1). The elements investigated for the Palo Verde and STP applications have a nominal temperature setting of 160 0 F and therefore the element part number is 9760X-1 60.
The 9760X element contains two power pills (Figure 2). The power pills contain a temperature sensitive wax that expands with increasing temperature and provides the motive force to lift the element off of its seat.
Report No. 10CFR21-0098 Revision: 3 Date: 12/01/11 Page: 4 of 5 Element 9760X (qty. 2)
(nearside & farside)
FIGURE 1: AMOT THERMOSTATIC VALVE FIGURE 2: VALVE ELEMENT 9760X
Report No, 10CFR21 -0098 Revision: 3 Date: 12/01/11 Page: 5 of 5 AFFECTED USERS:
A listing of users with the thermostatic valves that contain the Amot 9760X element is provided in the table below. The column on the far right side indicates the part number(s) shipped by ESI.
LISTING OF USERS WITH 9760X ELEMENTS Diesel Engine Thermostatic VleSystems Element P/N5 sipdb Site Valve shipped by ES!
Mfg Model Braidwood/Byron C-B KSV 6HAS Lube 2-05V-419-108 Nine Mile Point C-B KSV 6HAS Lube 2-OSV-419-109 Oconee EMD 645 4HAS & 6HAS Water 9760X-i70
.... ,=2-05V-41 9-109 Palo Verde C-B KSV 6HAS Lube & Water 2-05V-419-107 2-05V-419-107 South Texas Project C-B KSV 6HAS Lube & Water 2-OSV-419-107 2-05V-419-107 Susquehanna 5 th EDG G-B KSV 6HAS Lube & Water n/a 2-OSV-41 9-109 Waterford C-B KSV SHAS & 6HAS Lube & Water 2-OSV-419-109 2-05V-419-1 11 Laguna Verde - EMD 645 6HAS Water 9760X-170 Mexico Ergytech / lberdrola - * ,
- 9760X-170 Spain
- The Ergytech engine applicationis unknown.
CORRECTIVE ACTION:
Users should return any 9760X elements in their inventory to ESI for inspection of machining debris. Note: Elements shipped after March 16, 2011 by ESI are not suspect because additional inspectionprocedureswere implemented at that time.
Users with installed 9760X elements should continue to trend their EDG lube oil and/or cooling water system temperatures and monitor for any indication of thermostatic element problems. The thermostatic elements should be replaced at the first sign of increasing temperature trends.