05000269/LER-2010-001
| Docket Number | |
| Event date: | 02-20-2010 |
|---|---|
| Report date: | 12-13-2010 |
| Reporting criterion: | 10 CFR 50.73(a)(2)(i)(b) |
| 2692010001R01 - NRC Website | |
EVALUATION:
BACKGROUND
This event is reportable per 10 CFR 50.73(a)(2)(i)(b) as operation prohibited by Technical Specifications and as a defect reportable under 10CFR 21.21.
The Standby Shutdown Facility (SSF) [EllS:NB] at Oconee Nuclear Station (Oconee) provides an alternate and independent means to achieve and maintain Mode 3 for all three of the Oconee Units following sabotage, turbine building flooding, or a design basis (10 CFR 50, Appendix R) fire. The SSF is credited as the coping source of alternate AC power and decay heat removal during a station blackout event and provides defense in depth for a tornado event.
The SSF System includes a Reactor Coolant Make-Up (RCMU) Pump, which is located in the lower level of each Unit's Reactor Building and which is powered and controlled from the SSF. During SSF scenarios, the RCMU system uses the Spent Fuel [EllS:DA] Pool associated with that unit as the source of inventory to supply Reactor Coolant Pump seal injection flow and make-up flow to compensate for normal Reactor Coolant (RCS) [El IS:AB] leakage and shrinkage which results from going from power operation to hot standby (Mode 3). After RCS volume has been restored, the RCMU Pump continues to supply seal injection flow and a letdown flow path is established to return water from the RCS to the Spent Fuel Pool to control pressurizer level.
The SSF RC Letdown Line flow path on each unit includes valve HP-426 in a one inch diameter, schedule 160 line. A flanged pressure-reducing orifice is located on each unit just downstream of valve HP-426. These orifices were initially selected in 1982 during the design of the SSF and purchased as part of a "steam trap drain orifice" assembly. The assembly included a one inch, 60 Mesh (9 mil or 228 micrometer (micron)) dome shaped wire strainer, with an integral flange gasket, mounted just upstream of the orifice plate. The orifices on the three units were resized to 0.240 - inch diameter in 1994/1995, but equivalent strainers were retained in the design.
TS 3.10.1 addresses the SSF which is required to be operable in Modes 1, 2, and 3. Condition C allows the RCMU system to be inoperable with a seven day completion time.
EVENT DESCRIPTION
Unit 1 Event Details On 10/11/2009, Unit 1 was in Mode 5 for 1E0C25, a scheduled refueling outage. Operations attempted to perform a flow verification test of the SSF RC Letdown Line. Operations was unable to meet the acceptance criterion. At this point, Operations considered the test failed. A problem investigation was initiated.
On 10/19/2009, a visual inspection was performed as part of troubleshooting the failed test.
Maintenance personnel found foreign material (FM) in the SSF RCMU letdown flow path between an orifice screen associated with the pressure-reducing orifice and valve 1HP-426. Some of the material was shipped to the Duke Energy Metallurgy and Welding Services Lab (MetLab) for material analysis. After removal of the FM, a visual inspection was performed using a Boroscope (performed by entering the pipe at the orifice flange); inspecting the pipe in both directions. There was no visual evidence of any further FM.
The formal MetLab report was provided on 10/20/2009. A primary component of the debris was graphite, consistent with graphite material used in valve packing and gaskets. Additional "legacy" FM included austenitic stainless steel shavings, and particles of epoxy.
On 10/22/2009, the problem investigation was formally upgraded to a root cause investigation.
On 10/23/2009, 1HP-426 was disassembled for inspection of the valve internals. The inspection revealed that a Grafoil gasket was missing from a location under the valve backseat. GRAFOIL® is a registered trademark of GrafTech International Holdings Inc.
On 10/25/2009, additional debris from the ONS Unit 1 SSF letdown line had been collected in two (2) sock filters, which were also submitted to the MetLab. The majority of the larger visible particles included irregular-shaped chunks of epoxy, shreds of Grafoil, and one paint chip.
Although a new backseat gasket was installed in 1HP-426, an Engineering Change was processed to remove it and allow operation without a backseat gasket in place. Another Engineering Change was implemented to remove (delete) the orifice strainer from Unit 1. Both were implemented prior to returning the Unit 1 SSF letdown line to service.
The investigation team discussed the possible cause of the missing gasket with Flowserve.
Flowserve indicated that they had received no reports of a missing or damaged backseat gasket with this type of valve.
NUREG-1022, Rev. 2, page 34 states; An LER is required if a condition existed for a time longer than permitted by the technical specifications [i.e., greater than the allowed completion time] even if the condition was not discovered until after the allowable time had elapsed and the 'condition was rectified immediately upon discovery. This guidance is consistent with that previously given. (For the purpose of this discussion, it is assumed that there was firm evidence that a condition prohibited by technical specifications existed before discovery, for a time longer than permitted by technical specifications.)" Because there was no "firm evidence" of when the line became blocked and the problem was designated to be a random occurrence, the Unit 1 event was initially determined to be not reportable. Duke Energy did inform the industry about this issue via Nuclear Network as Operating Experience item 0E30114 on 12/01/2009.
At the conclusion of the refueling outage Unit 1 entered Mode 1 (12/02/2009), and subsequently resumed normal power operations at 100% power.
Unit 2 and 3 Event Details As part of the Unit 1 root cause evaluation, an extent of condition review was performed for the Units 2 and 3 SSF Letdown systems. Engineering reviewed data from prior SSF RCMU letdown line flow tests (Unit 2, tested 10/26/2008; Unit 3, tested 4/26/2009). The initial extent of condition review indicated that flow rates in the Unit 2 and 3 SSF letdown lines did not appear to be substantially affected. In December 2009, Engineering reviewed the. original test data and began a re-analysis. As the methodology evolved, the review indicated that the flow on Unit 2, was lower than expected but still adequate to support the requirements of the SSF. On 1/25/2010, a Problem Investigation Process (PIP) report was initiated and the formal Operability process entered. On 2/17/2010, a Prompt Determination of Operability concluded that Unit 2 was Operable.
On 02/18/2010, the NRC Resident Inspectors at Oconee raised an additional concern that the blockage could result in high differential pressure (dP) across the strainer during a SSF event such that the strainer might collapse and further degrade letdown flow.
Since the impact of strainer deformation/collapse on the SSF letdown line flow rate could not be quantified, the conservative decision was made that a reasonable basis for operability did NOT power was reduced to approximately 20% to reduce dose for a containment entry to access the strainer. On 2/20/2010, the Unit 2 SSF letdown line strainer was removed. FM, including Grafoil, was found on the strainer.
On 2/23/2010, Unit 3 power was reduced to remove the SSF letdown line strainer. FM, including Grafoil, was found on the strainer.
Operating Experience item 0E30789, dated 3/22/2010, provided an update to reflect additional information from Unit 2 and 3.
(Note: The following paragraph is based on information known at the time Revision 0 of this report Was submitted.) Based on the confirmation that at least some Grafoil material was released on all three units, Duke Energy reassessed the initial reportability determination. The existence of similar discrepancies in multiple lines indicates that it is inappropriate to assume the condition occurred at the time of discovery. The "event date" is considered to be 2/20/2010 based on the discovery of Grafoil FM on the Unit 2 strainer, confirming that this issue went beyond a single random failure. The exact mechanism for release and transport of the Grafoil material is not fully understood at this time, and there remains no "firm evidence" as to when the Grafoil material was actually released from the valves into the system on each unit. The current conclusion, based on available data, is that the material could have been released any time following installation of this model valve (Unit 1, .November 1995; Unit 2, April 1998; Unit 3, November 1998). Therefore, Duke Energy concludes that Unit 1 must be considered to have operated prior to discovery of this issue for a period longer than allowed by TS 3.10.1. The actual impact of the FM on Unit 2 and 3 past operability is still under evaluation. Until this evaluation is complete, Duke Energy considers that Units 2 and 3 were also affected, based on the existence of Grafoil on those units' strainers. Additional details related to Units 2 and 3 will be included in a supplement to this report.
A contributing cause of valve manufacturing deficiency was identified. Therefore, Duke Energy considers this issue to be a reportable defect as defined by 10 CFR 21. See Additional Information section below for information required by 10 CFR 21.21(d)(4)(i) - (viii).
Additional information for Revision 1:
Additional laboratory mock-up testing at Alden Research Laboratory, Inc. has been completed for conditions found on both Units 2 and 3. The root cause was reopened in part to incorporate the findings of this testing. This revised root cause report was finalized on 11/5/10. The mock-up testing confirmed that the quantity of Grafoil in a single gasket was sufficient to clog the strainer even without the additional "legacy" FM (metal shavings, particles of epoxy, etc). This testing also demonstrated that larger pieces of Grafoil on the strainer would initially pass acceptable flow, confirming the conclusions from the earlier in-place flow testing. However, as the duration of the mock-up tests continued, the pieces would degrade during flow, until a size was reached which clogged the strainer mesh. This resulted in increased differential pressure across the strainer, strainer deformation, and reduced flow (below that required for design function). The mock-up testing performed at Alden also provided insight that as-found legacy FM alone was not sufficient to substantially reduce flow. Therefore the valve issue was upgraded from a contributing cause to a root cause as described in the Causal Factors section below.
A separate cause evaluation was performed to address the failure to identify and correct Unit 2 and Unit 3 SSF RCMU letdown line degradation in a timely manner after degradation was identified on Unit 1. The causes and corrective actions from that evaluation were entered into the Duke Energy corrective action program.
CAUSAL FACTORS
The original scope of the root cause investigation was to:
- Determine the cause of the blocked flow path in the SSF letdown line.
- Evaluate timeliness of corrective actions from the root cause for LER 269/2006-03 concerning FM in the Reactor Building Emergency Sump (because the SSF letdown flow testing was initiated as a result of the corrective actions from that event).
- Review extent of condition for use of an orifice strainer device that could have a negative consequence on performance of a system safety function.
However, as additional information became available, including the fact that Units 2 and 3 were also affected, the scope of the root cause was expanded (Revisions 1 and 2) to include:
- Flow degradation on Unit 2.
- .. Performance of a detailed review of the Organizational and Programmatic (O&P) aspects leading to this event,
- A safety culture analysis of the event.
- The addition of a PRA plant risk significance section.
The content and conclusions of the root cause evolved according to the additional information and expansion of scope.
Revision 2 of the root cause investigation concluded that the root causes of the flow degradation event were 1) an inadequately selected strainer, upstream of the SSF letdown orifice, which was susceptible to clogging, and 2) a vendor design deficiency which allowed the valve backseat gasket to degrade and release Grafoil material as foreign material in sufficient quantity to cause the blockage and flow degradation on the Unit 1, 2 and 3 SSF letdown strainers.
With respect to the second cause, a significant portion of the FM on Unit 1 was determined to be Grafoil backseat gasket debris from upstream valve 1HP-426. Smaller quantities of the same material were found on Units 2 and 3, indicating that valves 2HP-426 and 3HP-426, which are the same model as 1HP-426, also released this material. No maintenance activities have occurred at Oconee that could have damaged the backseat gaskets. Grafoil was not the only type of foreign material found on the SSF letdown strainers. However, testing conducted at Alden Research Labs for this specific issue provides evidence that the Grafoil alone was sufficient to lead to significant blockage and flow degradation. Potential causes of Grafoil release related to valve vendor issues were identified.
In a letter dated 5/25/2010, Flowserve acknowledged that the valve design provided limited support for the gasket, the applied torque compressed the gasket, and the relatively large clearance between the backseat and the body bore is sufficient to allow the gasket material to extrude into the valve when the gasket expands radially inward due to that gasket compression.
CORRECTIVE ACTIONS
Immediate:
1. On Unit 1, the portions of the SSF RCMU letdown line near the strainer/orifice assembly were inspected. FM was found and removed. The strainer associated with the orifice was removed.
Subsequent:
1. The Extent of Condition focused on locating other strainer/orifice assemblies installed in the plant. The initial review consisted of a systematic electronic search on the key phrases "orifice" and "strainer," as well as searching for vendor documents in selected series which are associated with strainers. Other than the SSF letdown line application on all three units, this review did not identify any additional fluid system strainer/orifice combinations similar to the design described in this event.
2. Three spare valves, of the same type as 1, 2, 3HP-426, from the Oconee warehouse were inspected to determine the condition of the backseat gaskets and to evaluate the vendor practices for installing the gasket. All three valves' backseat gaskets were found intact but deformed.
3. Engineering implemented an Engineering Change to remove the backseat gasket from valve 1HP-426.
4. On Units 2 and 3, the portions of the SSF RCMU letdown line near the strainer/orifice assembly were inspected. FM was found and removed. The strainer associated with the orifice was removed.
5. The SSF RCMU Letdown flow test method for Unit 1 was revised to allow improved monitoring and trending capabilities.
Planned:
1. Engineering will create and implement calculations needed to support test acceptance criteria for the SSF letdown line test. Once calculations are developed, the test acceptance criteria (TAC), Design Basis Documents, and test procedures will be revised accordingly. (Rev 1: Now complete) 2. Engineering will change the frequency for the SSF RCMU Letdown flow test to every refueling outage.
(Rev 1: Now complete) 3. The SSF RCMU Letdown flow test method for Units 2 and 3 will be revised to allow improved monitoring and trending capabilities. (Unit 1 is complete. See subsequent action 5 above.)( Rev 1:
Now complete for Units 2 and 3 also) 4. Engineering, will take actions necessary to ensure the SSF RCMU Letdown flow tests are performed during unit startup rather than shutdown to ensure proper function before unit startup. (Rev 1: Now complete) 5. Engineering will implement Engineering Changes to remove the backseat gaskets from valves 2HP 426 and 3HP-426. (Rev 1: Now complete) Immediate Corrective Action 1 and Subsequent Corrective Action 4 (Rev 1: also Subsequent Corrective Action 3 and Planned Corrective Action 5) were identified by the root cause report as Corrective Actions to Prevent Recurrence (CAPRs) of the root cause and are complete. Planned Corrective Actions 1, 2, 3, and 4 listed above were identified by the root cause report as CAPRs for contributing causes of this event. (Rev 1: Listed Planned Corrective actions are now complete.) Additional corrective actions were identified within the Duke Energy corrective action program.
None of the listed planned corrective actions are considered NRC Commitment items. There were no NRC Commitment items contained in this LER other than the commitment to submit a Supplement. That commitment has been met by submittal of this revision.
SAFETY ANALYSIS
With a blocked SSF RC letdown orifice strainer, the Unit 1 SSF RC letdown line was not capable of providing an adequate letdown flow path during an initiating event that requires operation of the SSF. On Units 2 and 3, Duke Energy has concluded that the strainers would have clogged prior to operating long enough to complete the mission time for an SSF event.
Therefore, this problem resulted in Maintenance Rule Functional Failures of the SSF System for all three Oconee units.
Because strainers had been installed in these lines since original construction of the system, and because the Grafoil gaskets were introduced as integral parts of the valves as supplied by the vendor, Duke Energy concludes that the units were vulnerable to this failure mode from installation of this model valve, (Unit 1, November 1995; Unit 2, April 1998; Unit 3, November 1998) until removal of the strainers as described in this report.
The NRC significance determination process concluded that the failure to maintain the SSF RCMU Letdown subsystem for all three units operable as required by Technical Specifications should be characterized as having substantial safety significance. The results of Duke Energy's own significance evaluation were similar to the NRC conclusions in many respects except that this issue was characterized as low to moderate significance. The difference in these conclusions is attributed to different assumptions regarding certain fire scenarios. However, no event occurred during the period of vulnerability that required use of the SSF, therefore there was no actual impact to the health and safety of the public as a result of this issue..
ADDITIONAL INFORMATION
three Oconee units. That issue/event resulted in a 2007 "White Finding" which resulted in additional corrective actions related to FM. This current SSF letdown line event involves FM found as a result of testing which arose from that corrective action.
A recurring event determination was performed not only on the root cause for this event, but for the contributing causes as well. Per the criteria specified in the applicable Duke Energy administrative directive, the Oconee Problem Report (PIP) database was searched using cause codes for the root and contributing causes for prior occurrences within the last five years. It was concluded that this event is not similar/recurring. That is, the, corrective actions from the items found in the PIP database were not ineffective in preventing this event.
There were no releases of radioactive materials, radiation exposures or personnel injuries associated with this event.
This issue is considered reportable under the Equipment Performance and Information Exchange (EPIX) program. While the valves themselves continued to function, the FM resulted in actual or potential flow blockages affecting the system function on each Oconee Unit. The EPIX reports treated this as failure of the valves.
The following information is provided at this time to meet the requirements of 10 CFR 21.21(d)(4)(i) - (viii).
(i) Name and address of the individual or individuals informing the Commission.
Dave Baxter Vice President Oconee Nuclear Station 7800 Rochester Highway Seneca, SC, 29672 (ii) IdentifiCation of the facility, the activity, or the basic component supplied for such facility or such activity within the United States which fails to comply or contains a defect.
Facility:
Oconee Nuclear Station Basic component which fails to comply or contains a defect:
Size 1 inch Class 1878 socket welded globe valve with a threaded backseat (Flowserve Drawing No. W9524446, Rev. E). The Flowserve drawing identified part 101, the backseat gasket, as being made of Grafoil. GRAFOIL® is a registered trademark of GrafTech International Holdings Inc. The gasket is identified as non-QA in the Duke Energy spare parts system.
(iii) Identification of the firm constructing the facility or supplying the basic component which fails to comply or contains a defect.
Manufactured by:
Anchor Darling (now Flowserve Corp., Flow Control Division) (iv) Nature of the defect or failure to comply and the safety hazard which is created or could be created by such defect or failure to comply.
Nature of the defect:
Grafoil gasket material was released from the valve into the flow path as foreign material (FM). Upon inspection the Grafoil backseat gasket on valve 1HP-426 was found to be missing.
The applicable Duke Energy purchase specification required these valves to be ASME Section III Class 1 as well as clearly stating that the component is important to nuclear safety. That is, the valve was purchased under the Duke Energy QA program, and is therefore subject to 10 CFR 21. The specification included the following:
"Valves shall be capable of safe, proper and continuous operation under the full range of pressure and temperature conditions without undue strain, corrosions, deterioration, leakage, vibration, or other adverse effects on function or structural integrity.
Based on discussions with the valve vendor, it was believed that the 1HP-426 gasket could have been damaged during backseat installation in manufacturing or that the 1HP-426 backseat did not have the correct amount of torque applied in manufacturing. Other mechanisms could exist for gasket damage or gasket deterioration with the valve in service.
Any of these mechanisms could have caused Grafoil to be released from the valve. Since Grafoil release was subsequently found to have occurred on Units 2 and 3, valves 2HP-426 and 3 refueling outages. (Rev 1: ,Confirmed.) Safety hazard which could be created by such defect:
The valve body did not deteriorate or lose function, but the backseat gaskets clearly deteriorated to the point that the gasket material was released into a system, becoming FM.
In certain system configurations (e.g. fine mesh strainers), this FM could block flow paths.
reduced system flow below the acceptable minimum. The actual impact on Units 2 and 3 is still under evaluation. (Rev 1: Duke Energy confirmed that Unit 2 and 3 SSF Letdown lines were also affected.) (v) The date on which the information of such defect or failure to comply was obtained.
The original discovery of Grafoil material on the Unit 1 strainer occurred on 10/19/2009.
Based on initial information this was considered to be a random failure. Indications of a generic problem were confirmed upon inspection of the Unit 2 and 3 strainers on 2/20/2010 and 2/23/2010, respectively.
(vi) In the case of a basic component which contains a defect or fails to comply, the number and location of all such components in use at, supplied for, or being supplied for one or more facilities or activities subject to the regulations in this part.
The three valves in this application (SSF RCMU letdown) are the only known instances of this defect. Duke Energy has identified a total of 55 valves of the same overall valve vendor model (but of various sizes) installed in various applications at Ocdnee. Each of the valves is being evaluated on a case by case basis to determine if removal of the backseat gasket is warranted. (Rev 1. During additional review, an additional application was discovered, raising the total number of installed valves of this model to 59. The 59 valves include 1, 2, and 3 HP-426, for which the gaskets have been removed. Each of the additional 56 valves was evaluated as to whether the backseat gasket could pose a risk to system operation.
None challenged Operability.) Flowserve stated that they have supplied thousands of these valves with no reports of similar failures of this gasket. Duke Energy has no information on those valves/customers.
(vii) The corrective action which has been, is being, or will be taken; the name of the individual or organization responsible for the action; and the length of time that has been or will be taken to complete the action.
Corrective actions taken or planned:
Duke Energy has removed, the downstream strainers in the three specific affected locations at Oconee.
Duke Energy has removed the backseat gasket from valve 1HP-426, and will remove it from 2,3HP-426 during the next refueling outage on Unit 2 and 3 respectively. As stated in (vi) above, each of the 55 similar valves at Oconee is being evaluated on a case by case basis to determine if removal of the backseat gasket is warranted. (Rev 1: Gasket removal from 1,2,3HP-426 and evaluation of the additional 56 valves is complete) Duke Energy informed the industry about this issue via Nuclear Network as Operating Experience item 0E30114 on 12/01/2009 and updated that information via 0E30789, dated 3/22/2010.
The vendor, Flowserve, was notified, performed a Part 21 evaluation, and concluded that this issue was not reportable. Duke Energy determined to report this issue under Part 21, since the issue would be addressed as an equipment failure in this LER.
Individual or organization responsible for the action:
The majority of corrective actions will be the responsibility of the Oconee Engineering Section. See page one of this LER for licensee contact information.
Length of time to complete the action:
Rev 1: Duke Energy corrective actions listed in vii are now complete.
(viii) Any advice related to the defect or failure to comply about the facility, activity, or basic component that has been, is being, or will be given to purchasers or licensees.
As stated above, Duke Energy has provided information on this event to the industry via Nuclear Network. In addition, this LER, and planned supplement, will be submitted to the NRC. No other advice or notification is planned at this time.