ML093420812

Root Cause Evaluation Report: Tritium Identified in Emergency Service Water (ESW) Vault

ML093420812
Person / Time
Site:	Oyster Creek
Issue date:	06/29/2009
From:	Jason White NRC/RGN-I/DRS/PSB2
To:	Collins S NRC Region 1
References
FOIA/PA-2009-0214
Download: ML093420812 (65)

Text

WI' Request for MRC Approval of CR Investigation, CAPR, or Collective EFR Assignment Due Date Extension Requesto (b)(6) partment: Engineering CR & Assignment Number: 907846 CR

Title:

Root Cause Evaluation for Tritium Leak Present Due Date: 5126109 Desired Due Date: 615109 Reason for Extension Request: Request 10 day extension due to 1F20 support work requiring additional resources, Risk Associated with Extension (discuss Interim corrective actions): Little risk due to tritium Issues are being tracked on the emergent issues list and through the OCC when required.

Impact Associated with Extension: None , /

Department information in this record was deleted do of Information I

in accordance with the FOIA- L

V Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault

Title:

Tritium Identified In Emergency Service Water (ESW) Vault Unit(s): Oyster Creek, Unit I Event Date: April 15, 2009 Event Time: 15:38 Action Tracking Item Number: 00907846 Report Date: June 5, 2009 Sponsoring Managerý (j(6) _/

Invatlnntnmm.

(b)(6)

Executive Summary:

On April 15, 2009, in preparation for work insld9 the Emergency Service Water (ESW) vault, water was found inside the vault. As part of stdndard practices for water removal, the water was pumped Into drums and sampled for gamma emitters, tritium, and pH. Sample analysis Identified titium levels at 102,000 pCI/I, Based on a verbal agreement, the station provides a report to the New Jersey Department of Environmental Protection when groundwater tritium levels are greater than or equal to 2,000 picocuries per liter (pCI/I), which is the state's lower limit for detectable tritium activity. The total activity was below the Environmental Protection Agency's reportable limit of 100 Ci for tritlum as specified In 40 CFR Part 302, "Designation, Reportable Quantities, and Notification."

Investigation for the issue determined that the release of tritiated water was caused by leaks In the 8-Inch and 10-Inch carbon steel Condensate System lines, SS-4 and CS-24 respectively.

This investigation aimed to determine separate root causes for the material condition failure of the pipes and for the programmatic aspects related to these failures. The root cause investigation determined that the piping leaks developed due to a corrosion mechanism known as anodic dissolution. Poor application of pipe coating left the buried pipes susceptible to this corrosion. The station's Buried Pipe Program was reviewed as part of the evaluation of programmatic and organizational aspects related to the pipe failure. The Investigation determined that.the program basis document was flawed due to inadequate configuration management and design controls, which resulted in invalid assumptions being used In the 2

4 Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault development of the program. Inadequate configuration management and design controls resulted InInvalid assumptions that were used as the basis for development of the program, the associated assessment of risks and consequences, and, consequently, the station's strategic approach to tritium leak mitigation. The station missed opportunities to properly characterize risks and consequences associated with the degraded Condensate System piping. These risks and consequences were developed as part of the program basis document The program basis document is flawed inthat the 8-inch Condensate line was incorrectly identified as stainless steel as opposed to CS piping, which is the currently installed material.

Scope of the Investioation The purpose of this investigation is to determine the Root Cause and contributors to pipe failures at Oyster Creek Generating Station that resulted in the leakage of tritlated condensate transfer system water into the station's groundwater. The investigation will review sources of tritlum onsite, the site's procedures for mitigating the risk of tritium releases to the environment, station efforts to mdtigate this risk, and organizational and programmatic effectiveness associated with tritium risk and Impact mitigation. Additionally, the Investigation will determine the appropriate corrective actions for the causes Identified.

Summary! f Events A team was formed to identify potential sources of tritiated water leaks and actions were taken to sample onsite groundwater wells and the plant discharge. A sample of well MW-K15-1A, located southwest of the Condensate Storage Tank (CST), measured 4.5 x 106 pCi/. This result was determined to be similar to tritium levels seen inthe CST. Based on this Information, the leakage source was narrowed to the Condensate System. Using Oyster Creek Topical Report 116, "Oyster Creek Underground Piping Program Description and Status," a list of high probability locations for the leakage were selected.

Root Cause and Corrective Actions to Prevent Recurrence (CAPR§)

The Root Cause of the degraded 8-inch and 10-inch Condensate System piping Is "anodlc dissolution" resulting from disbondment of the coating and susceptible material (Root Cause 1). The Corrective Action to Prevent Recurrence (CAPR) Is to Implement a strategic plan that includes moving direct buried Condensate System piping either above ground or in monitored trenches.

The station missed opportunites to properly characterize risks and consequences associated with the degraded Condensate System piping. These risks and consequences were developed as part of the program basis document. The Root Cause of these deficiencies is that the program basis document is flawed due to inadequate configuration management and design controls that resulted in Invalid assumptions being used as the basis for development of the program, the associated assessment of risks and consequences, and, thereby, the 3

9 Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault station's strategic approach to tritium leek mitigation (Root Cause 2). The CAPR for this Root Cause is to revise the program basis document (i.e., TR-1 16) to correct plant design details, risks, consequences, recommend inspection frequencies, and inspection methods following a thorough program assessment.

Contributina Causes Improperly applied coatings and lack of coatings insome areas of the pipe contributed to coating dlsbondment and the resultant localized corrosion (Contributing Cause #1). A review of work order closure identified that repairs were limited to the relatively small areas Identified as requiring either coating or pipe repairs. InSeptember 1991, an Engineering evaluation was performed and documented in P.E. 125-1 File No. 1001-91, "Coating Repair on I-Inch, 8-inch and 10-inch CS Underground Condensate Lines." This evaluation supported the visual inspection of the pipe coating, with actions to remove coatings, inspect, repair, and recoat In select areas. It is noteworthy that the associated maintenance activities did not remove all coatings for a visual and/or UT inspections on the entirety of the exposed pipe. Such inspections would have allowed a more rigorous examination, particularly given that several pipes were Identified as having some degree of degradation and since the 10-inch CS line had multiple holes, indicating the potential for extensive corrosion of the pipe. Consistent with the point-by-point examination, the repairs of the piping and coating were performed ina "patchwork" manner.

Inaddition, "spark testing" was performed on additional areas of coated piping on the 8-inch and 10-inch lines. The associated work order did not identify a required voltage for this testing, and did not document consideration of environmental factors as recommended in industry guidance document ASTM D5162 - 08, "Standard Practice for Discontinuity (Holiday) Testing of Nonconductive Protective Coating on Metallic Substrates." Without such considerations, spark testing can cause coating damage. The coating inconsistencies and defects were the result of Inadequate guidance Inwork Instructions. Additionally, failure to properly and accurately document some completed maintenance and repair activities resulted in quality records not being established and retained for the completed work. Some work order closing remarks did not Include adequate descriptions for completed work. Examples exist where references made in work orders could not be followed to a retrievable document. Some underground piping program drawings were not updated and property maintained as part of modifications. The incamplete/inaccurate documentation led to Invalid assumptions around pipe configuration, configuration changes, soil and material condition, and abandoned pipe in the program basis document The change management processes prior to implementing the Exelon Buried Pipe Program did not support effectively managing design changes and related projects during site ownership and management changes (Contributing Cause #2). The station liad several changes of ownership and management between 1991 and 2009. Also, several modifications aimed at mitigating the station's risk to underground tritium leaks were planned. including design changes to move lines above ground, move piping into concrete trenches, and replace 4

41, Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault lines using more corrosion-resistant materials. Inmost instances, these planned modifications were in response to identified leaks. However, most were not implemented. Inaddition, management decisions were made in the md-1990s to allow the station's operating license to expire. Modifications not implemented, as well as cancelled maintenance and repair activities, should have been re-evaluated as vulnerabilities for long-term piping integrity.

The combined information from non-lntruslve and direct inspections is used to Identify the need for long-term repairs and replacements. The inspections rely on available technologies, each of which has specific limitations. This Includes Visual Inspection, Ultrasonic Testing (UT), and Guided Wave inspections. Any of these methods alone would not allow for 100% assurance of pipe integrity. Instead, they are used in combination to find localized areas of concern. Since 100% verification of pipe integrity is not practical, even these extensive measures leave the site vulnerable to localized corrosion because the methodologies used by the buried pipe program do not, in all instances, locate defects, and cannot assess entire continuous full lengths of pipe (Contributing Cause #3).

The Industry considers Guided WaveTM technology to be an acceptable means to perform quick non-Intrusive inspections of long sections of pipe to identify degradation. However, Industry experience has shown that at the current lerel of development, even the most sophisticated indirect Inspection methods are not as accurate at sizing the axial and circumferential extents of a corroded area as UT and, as a result, a follow-up direct examination such as UT would be required to characterized the area of corrosion. As a result, the current risk to the organization Is that other buried piping may have similar localized corrosion that is not being effectively detected and managed.

This event was reviewed for Safety Culture Components in accordance with step 4.2.4 of procedure LS-AA-125-1001 by Regulatory Assurance and no additional investigations are required.

Significance of Event:

The tritium level in the water that was leaking from the 8-inch and 10-inch Condensate lines was above the reporting limits for the State of New Jersey (2,000 pCI/L) based on a voluntary verbal agreement with the New Jersey Department of Environmental Protection. The Environmental Protection Agency's reportable quantity Is 100 Ci per 40 CFR Part 302. There is a small radiological risk associated with the underground pipe leaks. The leaking water spread radioactive contamination to the environment. The water contained tritium that produces a low energy beta dose. The urgency to identify the source of the leak led to excavation of several piping lines during a forced outage that lasted roughly 8 days.

Significant financial and personnel resources were required to restore the integrity of the piping. In the event the soil needs to be remedlated, a significant cost will be incurred by the company.

Event

Description:

5

! p Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault The following event description written in a sequential narrative format is also detailed in the Event and Causal Factor Chart In Attachment C.

1980s: Four-inch and six-inch buried aluminum lines around the CST and the Demineralized Tanks were confirmed leaking.

Early 1990s: The station Initiates the Oyster Creek Underground Piping Program after several buried piping leaks were discovered, Including the aluminum Condensate Transfer lines. The leaking pipes were repaired or replaced.

b)(4) 1991: The CST developed a leak requiring replacement of the tank bottom.

1991: GPU Nuclear Document OC-MM-323643-001, Rev. 0, "Mini-Mod for Replacement of Condenser Letdown Line from the Turbine Building to the Condensate Transfer Pump Building" was Initiated, but not implemented. The goals of the modification were as follows.

" Reroute piping from beneath the Condensate Transfer Building to outside

" Flange SS piping to existing CS piping in the Condenser Bay

" Pre-assemble, coat, and wrap below-ground piping

" Spray-coat coated pipe with a PolykenTM type primer followed by non-conductive PolykenTm tape July 22, 1991: Work Order C0032859 repaired a hole In the 8-inch carbon steel piping wth a plug installed Into a seal welded coupling. The CREM Identified the cause of failure as pipe corrosion.

July 25,1991: Job Order 032927 completed a weld repair on a hole In the 8-inch Condensate pipe via instructions written in 125-1 Evaluation 0924-91. Job Order 032927 also completed Ultrasonic Testing (UT) at pipe locations marked with duct tape.

October 8, 1991 through January 8, 1992: Work order C0033031 recoats the 8-inch and 10-inch Condensate lines per 125-1 Evaluation 1001-91. The work order and the evaluation are consistent in that no activity was designated to remove all coatings on these lines and inspect the entire pipe for degradation. Only those areas where coatings were removed were inspected, and the coating was reapplied only to the Inspected areas and to areas where degraded piping was Identified.

6

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault

" October 9, 1991: Spark testing revealed five small (quarter-sized) defects in the 10-inch line and two areas of defects in the B-inch line (approximately two square feet)

" November 22, 1991: 8-inch and 10-inch coating failure locations were ready to recoat

" November 25, 1991: The transfer line repairs were canceled with no mention of which lines were cancelled Inthe CREM

" November 26, 1991: The 8-inch and 10-inch pipes were recoated in the failed areas

• December 2, 1991: The B-inch and 10-inch carbon steel piping was recoated with Devoe 167 primer and Devoe 235 coating

• December 7, 1991: "Wet sponge" holiday testing of all carbon steel pipeline coating was completed successfully. Three layers of Polyken Tm tape were applied to the 8-Inch carbon steel pipe section near the furthest west location around the new pier/rebar/concrete pad. Wet sponge holiday testing on Devoe-coated sections of the 8-inch and 10-inch pipe was reviewed by Quality Assurance and passed. The coating on the 6-inch aluminum line exhibited significant degradation and the recommendation was made to replace the coating on the entire length of this 6-inch line.

1996: A modification was completed to bring the majority of Condensate and Condensate Transfer piping above ground, The modification was tracked through OC-CCD-328376-001 and left four condensate lines buried, including SS-4 and CS-24.

b)(4) 1997: TDR-1218, "Evaluation of Oyster Creek Underground Piping Which Contain Contaminated Fluids," was created.

1997: Oyster Creek owner, GPU Nuclear, decides to decommission or the sell the plant.

2000: AmerGen purchases Oyster Creek.

2001: Work orders completed before 2001 were transferred into PIMS, 2004: Exelon announced the decision to pursue relicensing of Oyster Creek to 2029.

2005: The Buried Piping Program was revised to include an assessment of pipe service life and to include considerations for license renewal. Work orders were reviewed for developing the program basis document, including C0033031. Although this work order indicates that the 7

ID Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vauft coating repairs were only applied in specific sections of pipe, the program basis document considered the coating to be completely new for the entire length of the Condensate piping as of 1991. Another assumption used Inthe risk ranklng of the program basis document Is that the 8-Inch line was replaced with stainless steel per OC-MM-323643-001, based on drawing 3D-421-22-1000, Revision 0. This drawing was associated with a modification to replace the 8-Inch CS piping with stainless steel. The modification was not implemented.

August 2006: CST-2, the lysimeter on the southwest comer of the CST, was sampled and had a tritium concentration of less than 200 pCi/L.

(b)(4)

October 16, 2007 through October 18, 2007: Inaccordance with the Buried Piping Program, lines A-4 and CS-24 were Inspected by Structural Integrity Associates, Inc. (SIA), using Guided WaveTM Technology per Work Order C2015637. PLR-07-441, Revision 1, describes the results on CS-24 as follows: "This section of pipe appears to be a minor concern. Possible coating failure or thickness change." The Guided WaveTM inspection was attempted on SS-4, but discovered to be impossible due to configuration. IR00686803 was written to document the Inability to complete Guided Wave Inspection on twelve lines, including SS-4. IR 00696852 was written to document the need to inspect the lines during Fall 2008 refueling outage I R22.

2008: A partial visual inspection was completed on line A-4 per the Buried Piping Program.

(b)(4)

L-Z008: In-Service Testing (IST) surveillance 644.4.002, for the Condensate Storage System, was updated to indicate that an underground leak might cause an IST failure.

January 2009: Buried Piping Program Owner makes the recommendation to modify CS-26 and CS-38 to an above ground configuration.

8

I Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault March 2009: Oyster Creek Engineering performed Technical Evaluation Passport ATI 891862 Assignment 4 to establish an inspection schedule for the CST. The first scheduled inspection is December 2009.

March 10, 2009 through March 12, 2009: The following wells were monitored with tritium levels less than 200 pCi/L: W-3, W-4, W-5, W-6 and MW-15K-1A.

April 15, 2009, at 15:38: Tritlum was identified in the ESW vault. Inpreparation for work Inside the vault, water found inside the vault was pumped Into drums and sampled for gamma emitters, tritium, and pH. There were no gamma emitters identified, pH was 7.62, and the tritiuum was 102,000 pCi/I. The reporting threshold to the New Jersey DEP for tritlum Is 2,000 pCi/I based on a verbal agreement between the station and the NJ DEP.

April 15, 2009: IR00907846 was initiated to document the tritdated water, April 16, 2009: A team was formed to Identify potential sources of tritiated water leaks. Based on measurements of tritlum found Inwell MW-K15-1A, it was determined that the source of leakage was from the Condensate/Condensate Transfer System to the ground. This was postulated because the well sample tritium activity, 4.5 x 106 pCI/I, was similar to the expected activity of CST water. Using Oyster Creek Topical Report 116, "Oyster Creek Underground Piping Program Description and Status." a list of high probability piping locations was created for pinpointed troubleshooting.

April 18, 2009: Guided Wave TM inspections were performed on buried Condensate lines.

Guided WaveTlM was intended to help find the leak location. Itwas also used to determine the extent of condition for the lines Inthe area potentially contributing to the tritium found in the sampling wells. UT and visual Inspections were also used. Guided WaveTM inspections were combined with UT Inspections to verify the results. The following lines were Inspected. The 6-inch Condensate Transfer discharge (aluminum), 10-inch diameter HotweUl Level Control to Hotwell (carbon steel), and 8-inch diameter Hotwell Level Control to Hotwell (carbon steel) were inspected using Guided WaveT". Guided WaveTM results Indicated severe corrosion on the 8-inch line; however, follow-up UT performed on April 28, 2009, found only minor corrosion.

April 20, 2009: Perform a PINV and obtain management approval. After approval, submit the PINV to the NDO Mailbox and the NRC Resident. The PINV included the following plan:

Evaluate all inputs into and from the ESW vault for potential sources of tritium

1. Inspect other vaults in the vicinity and sampling for tritium as warranted

2. Collect water samples from:

a. Monitoring wells in the general vicinity of the cable vault - W-3, W-4, W-5, W-6 and MW-15K-1A, These wells were sampled during the March 10-12, 2009 period. The results of tritlum analyses on those samples were all < 200 pCl/L.

b. Lysimeter in the vicinity of the cable vault - CST-9. This lysimeter was sampled on March 11, 2009 and the tritium concentration was < 200 pCi/L.

9

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault

c. Lysimeter on the southwest comer of the Condensate Storage Tank - CST-2. This lysimeter was last sampled in August of 2006 when the tritium concentration was found to be < 200 pCi/L.

d. Surface water sample from the main condenser discharge. This is not a routine sampling point.

The samples were shipped to Teledyne Brown Engineering to be analyzed for tritium, strontium-90 and gamma emitting radionuclides utilizing the detectable limits (LLDs) specified for the Radiological Groundwater Protection Program.

April 26, 2009: Underground Condensate lines SS-4 and CS-24 were identified as the sources of the leak of the tritiated water Into the ground water table using the described methods. A leak on the bottom of the 8-inch carbon steel line, SS-4, was identified during excavation. A pipe clamp was Installed to stop leakage. Non-destructive examination (NDE) of the area around the hole Identified a T-Inch by 7-Inch square area of involved corrosion and degraded piping. Based on UT results, the station Initially replaced a ten-foot section of the 8-inch pipe, which included the degraded areas. Subsequently, the Condensate System was restored to operation. While transferring water from the Hotwell to the CST, a leak was Identified on the 10-inch carbon steel CST to Hotwell makeup line, C8-24, in an area that had not been fully excavated. The pipe was pressurized to no more than 40 psig during the transfer evolution and the leak was Isolated shortly after discovery. Guided WaveTU analysis was performed on the removed section of the 8-inch line to determine the efficacy of this technology in assessing pipe integrity. Evaluation of the data did not identify the known flaw In the 8-inch line. Based on an inability to detect the degraded section of the 8-inch line, the station conservatively excavated and replaced approximately 30 feet of the 8-inch and 10-inch Condensate System lines.

10

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault The ultrasonic testing results from 1F20 for excavated lines between the Condensate Transfer and Turbine Buildings are documented in the table below.

Sz St 87.5% Code Actual Pipe Size System Material Design Schedule I of Min (lowest) UT Comments

1. 1.2. Press. Thick Wall Wall Results 1" 421 CH- CS- 75 psi 80 / 0.179" 0.157 0.004" 0.152" Judged OK 1c A106 Gr. " until 1R23 AorB 6" 424 CH- Alum - 200 psi 40 10.280" 0.245 0.109" > 87.5% OK 5c 6061 Gr.

T4 or T6 8" 421 CH- CS - 370 psi 40 0.322" 0.282 0.105" a) Through Replaced all 2a A106 " wall hole but 3" at wall Gr. B near Cond Transfer house b) 0.265" @

- 4 ft from TB wall c) 0.117" @

5' from T1B wall 11

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault 12

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault (b)(4)

April 29, 2009; A 'window" containing the through- wall hole was sectioned from the two-foot segment of the 8-inch CS pipe and transported to PowerLabs for analysis. PowerLabs performed an assessment of the overall 00 and IDcondition prior to receiving the sample and in the laboratory. The laboratory analysis assessed the coating condition, Identified the leak mechanism, determined the IDor OD initiation, and contributing factors.

April 29, 2009: The majority of the 8-Inch and 10-inch buried piping was replaced (reference AR A2222268) and three remaining portions were evaluated via a technical evaluation (AR A2222268-13). An Operational Technical Decision-Making document was also prepared (ref IR 00907846-11) prior to startup from forced outage I F20. Piping inspections performed during and following forced outage 1F20 indicated that the coatings on the 8-inch and 10-inch lines were inconsistently applied. The coatings were applied in a "patchwork" manner.

Additionally, spark testing performed on these lines likely induced damage to the coatings on these lines. This indicated that the maintenance activities surrounding the repair and replacement activities were both 1) inadequate to mitigate corrosion on the affected piping, and 2) contributed to impairing coatings such that increased corrosion rates may have resulted.

May 5, 2009: An OTDM was prepared and included the following:

1-inch Condensate Transfer to Hotwell - CS (operating pressure: 22 psig)

Pressure testing of this line confirmed its integrity. This line is presently leak tight, but will be isolated when not In service to minimize any risk for future leakage. This line will be re-routed or replaced with material not susceptible to corrosion. The line can be isolated while the unit Is on-line to support maintenance. This line will be used once for the start-up of the Condensate system and will be isolated following system start-up until a basis for future use is determined or an alternate routing is established.

1-Inch Control Rod Drive (CRD) system Minimum Flow Bypass pipe to Condensate Storage Tank (CST) - stainless steel (operating pressure: 100 pslg)

There Is no evidence that this line Is leaking. The line Is stainless steel, which is less susceptible to corrosion. In 1993, Oyster Creek inspected two buried stainless steel lines.

Records indicate that no degradation was found. This line was not inspected during 1F20 due to excavation restrictions caused by the excavation adjacent to the Condensate Transfer Pump Building.

4-inch Condensate Transfer Building Drain pipe - CS A temporary plug was installed prior to startup to prevent leakage from the building. With the plug Installed, the potential for leakage from this path is eliminated. The Condensate Transfer 13

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Building Drain Line .runs below grade and leas into the CST and Demineralized Water Tank Overflow Line. This is a 12-inch line that runs approximately seven feet below grade, entering the Turbine Building on the west wall and terminating (open ended) just above the Turbine Building floor about 20 feet below grade (ref. BR 2193 and BR 2180). For water to back up Into the 4-Inch line from the 12-inch line, the line would require significant flow (over 2000 gpm) to create enough of a backpressure in the 12-Inch line to overcome the approximate 20 foot elevation difference between the Turbine Building floor and the Condensate Transfer Building floor. These flow rates are not likely to occur. Removal of this drain does increase the probability of a potential spill from a leak In the pump building. If a leak of sufficient magnitude develops, It could overflow the existing sump and mote area. The sump is equipped with a high level alarm. Increased attention to this potential is warranted. Line replacement will be performed, in the near term, as determined by the Buried Pipe program. A project has been initiated to Improve the overall containment and management of the system within the Condensate Transfer Building (IR 914427) 6.inch Condensate Transfer Discharge aluminum piping (operating pressure 150 psig)

This piping was replaced in 1994. The coating inspection results Indicate that the coating has not aged significantly since installation In 1994 and remains in good condition, No evidence of outside diameter (OD) corrosion was identified. Ultrasonic Testing (UT) inspection found minor inside diameter (ID)wall loss of approximately 8%. With the exception of four feet of pipe east of the Condensate Transfer Pump Building, the length of pipe from the Turbine Building to the Condensate Transfer Pump Building is visible. The four feet of pipe is covered by about two inches of loosely packed dirt. There was no observed leakage from this pipe section. The pipe has remained pressurized through the inspection period and leakage would be visible Ifpresent. Excavation and inspection efforts have been In progress since Saturday, April 25, 2009, and no signs of leakage have been identified. Coatings and wrappings shall be restored to design condition. The initial failure analysis on the 8-Inch line concluded the failure was due to a coating breakdown causing OD to ID corrosion. There Is no significant active damage mechanism occurring In this pipe.

8-Inch diameter Hotwell Level Control to Hotwell (Carbon Steel) pipe (operating pressure 200-250 psig)

During Inspection, a leak was Identified in this pipe. The buried pipe will be replaced from near the Turbine Building wall to the above ground area in the Condensate Transfer Pump Building.

The pipe between the turbine building wall and the new pipe will be inspected to ensure that no degradation exists in this short section. Coatings and wrap will be restored to design conditions.

10-inch diameter Hotwell Level Control to Hotwell carbon steel pipe (operating pressure 40 pslg)

During restoration of the Condensate system, this line was pressurized and a leak developed Ina portion of pipe, The buried pipe will be replaced from near the Turbine Building Wall to the above ground area in the Condensate Transfer Pump Building. The pipe between the Turbine Building wall and the new pipe will be inspected to ensure that no degradation exists in this short section of pipe. Coatings and wrap will be restored to design conditions.

14

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Condensate Storage Tank (CST) (Aluminum)

A walkdown of the area surrounding the tank does not indicate any leakage from the tank wall.

Tank water level is being evaluated to determine if water level changes are proportional to operational condition requirements. CST Inspections are scheduled to determine the condition of the floor of the tank for long-term operation. The divers are scheduled for May 4, 2009, Divers will perform full visual internal inspection of the tank bottom. Sixty random spots on the floor of the tank will be ultrasonically inspected. An ACMP is in place to monitor leakage to ensure that additional sources of active leakage do not exist.

The bottom of the CST was replaced In 1991. As part of this repair, the configuration of the bottom was improved to mitigate corrosion. This included caulking the interface between the tank bottom and the concrete base and improved drainage capability of the sand at the bottom of the tank.

May 7, 2009: IR 00916938 was written to capture an NRC recommendation that an additional monitoring well in the Kirkwood Aquifer (W-4K) be included in the monitoring program. The NRC recommended installing the well to provide assurance to the public that the on-site groundwater contamination is not migrating to off-site wells.

May 2009: PowerLabs analyzed the pipe sample per project number OYS-44923 with the following result: "The pipe leak was OD-initlated cause by localized galvanic corrosion initiated by a breach in the corrosion barrier coating."

May 2009: The Condensate Storage Tank was Inspected during May 2009 per work order R2119514 In accordance with specification SP-1 302-52-108. The work order closing remarks describe the results as satisfactory.

May 5, 2009: In an e-mail from PowerLabs to Exelon Engineering, additional details were described. "The pipe leak was OD-initiated and was associated with localized OD galvanic corrosion." The galvanic corrosion was caused by coating disbondment.and progressive corrosion, as illustrated by a "halo" effect surrounding the through-wall hole, "Additional areas of localized OD pitting corrosion were observed and were associated with the primary leak location. The primary leak and associated damage were located within one quadrant on the pipe. The pattern is consistent with progressive coating dlsbondment. The extent of damage suggests that the corrosion has been occurring for an extended period.

No mechanical damage was observed on the pipe OD at or near the leak location. The ID pipe surface was pristine. The drawing marks from the original manufacture of the pipe were still easily visible, indicating no significant wall loss due to ID corrosion, The majority of the mastic coating had been removed, However, one piece remained at the end opposite the leak.

The remnant piece was well-adhered with a thickness measuring 0.170". The remainder of pipe 00 was in good conditlon, suggesting the general condition of the coating was adequate, For example, less than 1t2" from the "halo" associated with the leak original pipe stamping was 15

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault clearly visible. The nominal wall thickness measured 0.328", consistent with 8-inch schedule 40 piping."

Inconjunction with the above description and the Event and Causal Factor Chart (Attachment C of this report), the following causal factors were Identified: 1) anodic dissolution resulting from disbondment of the coating and susceptible material; 2) a flawed program basis document; 3) Inadequate work Instructions, documentation, and work quality; 4) the change management processes prior to Implementing the Exelon Burled Pipe Program did not support effectively managing design changes and related projects during site ownership and management changes; and 5) limitations in available technologies used to assess pipe material condition.

16

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Extent of Condition:

Condition being Extent of Condition addressed /

Two buried piping lines The extent of condition is related to piping in the station's buried in the Condensate pipe program, as outlined in Attachment B of this report. The System developed current risk to the organization Is that other buried piping may leaks, resulting in a have similar localized corrosion that is not being effectively release of tritium to the managed because the methodologies used by the burled pipe ground water. program, which are consistent with Industry best practices, are not locating the defects. There are three primary methods to identify degraded areas of piping that may develop leaks: Guided WaveTM, UT, or excavation combined with direct inspection. The large number of components in the buried pipe program allows prioritized non-intrusive inspections to be used to collect information on the burled pipe population as a whole. Information gathered through internal plant experience dictates the need for direct examinations, The combined information from non-intrusive and direct inspections is used to implement long-term repairs and replacements.

Examples of ways the station is addressing this condition are as follows:

In 2008, Oyster Creek completed replacement of all underground safety-related ESW piping (approximately 600 linear feet). Risk ranking for these lines has been revised; the lines are no longer considered "High Risk." In 2004 and 2008, Oyster Creek replaced approximately 20% of all underground Service Water System piping (approximately 150 linear feet).

In 2008, the program performed coating inspections on six lines in the area of the Reactor Building southeast vault. The inspection found that the coating on five lines was degraded and required repair. The deficiencies were entered into the station's Corrective Action Program (Issue Report (IR) 00813967). The IR was accompanied by an evaluation, which concluded that the coating conditions did not pose an immediate operability concern. Repairs are planned per work order R2130898. This inspection Is now required as a repetitive task (PM57304M). This PM also opens and drains water out of the vault.

In 2008, the program excavated and inspected a 6" Condensate Transfer line located west of the Turbine Building. The Inspection found that the coating was in good condition.

17

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Condition being Extent of Condition addressed Upon entering the period of extended operation, focused Inspections of burled piping and components will be performed on a ten-year frequency and as opportunities occur within this ten-year period. The inspections will include at least one carbon steel, one aluminum, and one cast iron pipe or component. In addition, for each of these materials, the locations selected for inspection will Include at least one location where the pipe or component has not been previously replaced or recoated, if any such locations remain. The stainless steel piping in the vault will continue to be periodically inspected, and the bronze material is addressed by the buried carbon steel pipe coating Inspections.

18

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault

.1j Through the methods listed above, the team identified two root causes and three contributing causes. Leaks that developed in the Condensate Transfer lines during 1991 were not addressed adequately to ensure an expected service life was achieved.

Work quality was identified as a failed barrier using the event and casual factor and barrier analysis root cause methodologies. The work quality issues led to anodic dissolution of the pipe. In addition, the program Initially placed an emphasis on Service Water, ESW, and Fuel Oil System buried lines due to these lines being Initially ranked as having the highest degree of safety and environmental impacts. Interviews with the previous program owner indicated that the high risk ranking of the three systems drove the station to replace most of the buried lines of these systems with above ground or trenched lines. The remaining buried pipes were risk ranked and inspected as required by the buried piping program. The program basis document was developed with an erroneous assumption that a modification to the 8-inch carbon steel line was &ompleted; the modification would have replaced the 8-inch carbon steel line with stainless steel.

Based on this assumption, a lower risk ranking was assigned to this line with the consideration for stainless steel's resistance to corrosion. Associated design changes and documentation were not adequately controlled during the various change management processes that occurred over the course of the program's life, Discussions were held with Exelon Corporate and site Engineers with experience in corrosion and buried piping to help Investigate the corrosion mechanism. Initial discussions focused on galvanic corrosion. In general, galvanic corrosion is an 19

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault electrochemical process that can occur when two dissimilar metals are in close proximity and are immersed in a substance that serves as an electrolyte or conductor.

The electrolyte provides a means for ion migration whereby metallic Ions can move from the anode to the cathode. This leads to the anodic metal corroding more quickly than it otherwise would; the corrosion of the cathodic metal is retarded, possibly even to the point of stopping any corrosion.

Given that some dissimilar metals were found in the area of the leak, general galvanic corrosion was considered as part of evaluating the primary failure mechanism due to known susceptibility to this phenomenon. For instance, some dissimilar metals found In the area of the leak include the following: 1) 8-Inch carbon steel line CH-2a; 2) 10-Inch carbon steel line CH-ic; 3) stainless steel clamps were attached to the 8-Inch CH-2a carbon steel line in sections without pipe wrap; 4) abandoned aluminum piping approximately two feet above the 8-inch line; 5) copper grounding grid at the northwest corner of the turbine building.

Carbon steel and copper will undergo galvanic interaction when the two metals are both Immersed in the same aqueous solution. The overall affect will be that the carbon steel will corrode.' The corrosion rate is affected by temperature; pH of the fluid, the amount and surface area of copper, the amount and surface area of carbon steel, and the distance between the metals. Anodic carbon steel (primary element is iron) and copper, which is noble, have an electrode potential of 0.777 volts. This is considered a galvanic couple in which the Iron Inthe carbon steel will corrode. A portion of the station's grounding grid Is located on the west side of the Turbine Building. The grounding wires for this grid are mainly four feet below grade and are all copper. These wires are intended to electrically ground the Turbine Building and the station transformers. The copper wires are attached to grounding rods that are driven deep into the ground. Most of this grid is located to the south of the leaking lines (on the south side of the Condensate Transfer Building). However. two copper wires are located dose to the 8-Inch line on the north side of the Condensate Transfer Building. These two wires are on top of duct banks that traverse from the Condensate Transfer Building to the CST and to the Demineralized Water Storage Tank. Comparing plant design drawings BR 3179 and BR 2193, the 8-inch and 10-inch lines are approximately 12 feet from these copper wires in the area where the leaks occurred. In addition, the Turbine Building groundingI system has copper wires exiting the west wall of the Turbine Building near these lines.

After review, it was determined that the spatial arrangement and electrical potential of these materials did not support development of a significant galvanic cell that would Corrosion andi Corrosion Control. Third Edluon Uhlig and Revis, Wiley and Sons, 1984; Com-osion pfgfltg Fontana, Third Edition, McGraw MIfN, 1986; Corrosion Handbook. Uhllg, 1948, Wiley and

Sons, 2 1984; NACE Corrosion Engineer's Reference Bock, Ubrary of Congress 79-67175 Corosion Enalneerina, Fontana, Third Edition, McGraw Hill, I986; NACE Corroslon Engineer's Reernc-Iok, Library of Congress 79-67175 Plant drawings BR 3179, Rev. 9, and BR 2193%Rev. 10 4 Plant drawings BR 3100, Rev. 3, BR 3180, Rev. 12, and BR E0805, Rev. 0 20

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault induce the degree of piping degradation Identified on the carbon steel piping. The copper grounding grid was approximately 12 feet from the area of the degraded pipe.

At this distance, a strong electrolyte would be needed to produce a significant coupling between the grid and the CS piping. The stainless steel (SS) damps were electrically isolated from the CS by rubber matting applied between the damp and the CS pipe. In addition, the bolts on the clamp appear to be CS and are not corroded. The SS galvanic potential is cathodic when compared to CS, but there is only a small difference in this potential. A CS-SS couple would be considered weak and the corrosion rate associated with such a couple would be considered too weak to develop the coupling required to induce the degree of piping degradation identified. Additionally, aluminum has a galvanic potential of -1.1 volts and the CS galvanic potential is -0.5 volts. This means that the aluminum, having a significantly higher galvanic potential, is more anodic and will selectively corrode over carbon steel. Abandoned aluminum piping in the area of the leak did not exhibit significant degradation as would be expected had galvanic coupling occurred.

With respect to Condensate System piping, anodic dissolution, a localized corrosion mechanism, occurred. Anodic dissolution resembles general galvanic corrosion In that the precursors include a coating defect and a material susceptible to electrochemical reduction.

Anodic dissolution is a corrosion phenomenon that is characterized by a localized wall loss producing shear-walled, smooth-bottomed defects. This mechanism is typical in buried pipe systems where a defect in the coating of a pipe establishes a small anode that is driven by a large cathode. The adequately coated pipe acts as the cathode and, due to its size, produces a substantial driving force for corrosion. The relatively small anode is corroded at a rate that is proportional to the ratio of anode to cathode. Chemically, iron reacts with the hydroxide Ion in water to form iron-hydroxide (Fe(OH)2), leaving an excess of H*Ions In the pit area. The suppression of pH results In the formation of H2 gas through the combination of two H"ions in the reaction.

If anodic dissolution Is occurring over a large area of dislodged coating, the corrosion footprint is different. The large area produces a wastage that Is manifested as thinning over the entire area. The thinning Is generally most severe at the center of the region of coating loss. The cross sectional profile slopes from the Intact coating to the center defect or through-wall defect.

21

I Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault

--- ý- b)(4) nformation collected through the indirect Inspections are incorporated Into the direct inspection process, step three of the buried piping program. Information collected during the indirect inspections Is used to locate areas of Unes that should be Inspected 6 Closure remarks for work order C0033031 Corrosion Handbook.Uhlg, 1948, Wiley and Sons, 1984 22

i Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault using direct methods such as ultrasonic testing or a visual Inspection with manual measurements. The following table from ER-AA-5400 details how the two inspection methodologies are related.

,)w-These lines were assessed as a medium risk factor since they are direct buried and could result in an unmonitored radiological leak and possibly lead to a plant shutdown.

Degradation of these lines will most likely be due to degradation of the coating and external corrosion of the carbon steel pipe wall. Based on plant operating experience the coating has the potential service life of 15 to 40 years and the pipe wall has the potential life of 4 to 15 years. Therefore, these lines should have a minimum service life of 19 years. Since these lines were inspected in 1993, they should be re-inspected, pressure tested, or replaced by 2012.

In 2007, these lines were considered for Guide Wave Inspection. Unfortunately the technology cannot inspect lines that are 1 Inch in diameter or smaller. In addition, these 23

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault lines are too short for "CScan" Technology. IR00861645 was submitted to pursue modification of these lines.

As indicated by the timeline, the above example, and the program basis document, Oyster Creek was Implementing the Buded Piping Program as required per ER-AA-5400.

24

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Evaluation:

Cause (Describe the cause Problem Statement and identify whether it Basis for Cause Determination is a root cause or contributing cause)

Two buried piping Root Cause 1 - Anodic PowerLabs Report OYS-44923, lines in the dissolution resulting from "Evaluation of a Leak In Buried Condensate System disbondment of the Condensate Piping for Oyster Creek Unit developed leaks, coating and susceptible 1" identified the following: Analysis was resulting In a release material performed on a section of the 8-inch line.

of tritlum to the The analysis concluded that the pipe leak ground water. was OD-Inltlated caused by locatlized galvanic corrosion initiated by a breach in the corrosion barrier coating. The galvanic corrosion caused additional coating disbondment and progressive corrosion, as illustrated by a "halo" effect surrounding the through-wall hole.

Additional areas of localized OD pitting corrosion were observed and linked with the primary leak location. The primary leak and linked damage were located within one quadrant of the pipe. The pattern is consistent with progressive coating disbondment. Chlorine was detected at concentrations less than 1 wt, % in the external corrosion product on the pipe. The presence of chlorides can increase the corrosion rate. The extent of damage suggests this corrosion has been occurring for an extended period of time.

No mechanical damage was observed on the pipe OD at or near the leak location.

However, one piece remained at the end opposite the leak. The remnant piece was well-adhered with a thickness measuring 0,170". The remainder of pipe OD was in good condition, suggesting the general condition of the coating was adequate. For example, original pipe

________________________ L -

stamping was clearly visible less than 1/2" 25

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and Identify whether it Basis for Cause Determination is a root cause or contributing cause) from the leak ehalo'. The ID pipe surface was pristine. The drawing marks from the original manufacture of the pipe were still easily visible, Indicating no significant wall loss due to IDcorrosion. The nominal wall thickness measured 0.328",

consistent with National Pipe Standard (NPS) 8-inch Schedule 40 piping."

A 10-inch carbon steel line, CS-24, is direct buried between the Condensate Building and the Turbine Building. This line supplies flow from the Hot Level Control System in the Condensate Building to the Hotwell. The 10-Inch line (CS-24) returns condensate from the CST via flow control valve V-2-16 to the Hotwell. Since this line Is directly connected to all three Hotwells, it is under vacuum when the plant is at power. If a through wall hole develops in the pipe wall, leakage Is drawn into the pipe and not out. Minimal condensate would be released to the surrounding soil. This line was assessed as a Medium Risk factor since it Is a direct buried carbon steel line.

This line was assessed as Medium Radiological Consequence factor since a leak could lead to an unmonitored release and a Medium Consequence factor since a significant leak could lead to a plant shutdown. Degradation of this line will most likely be due to degradation of the coating and external corrosion of the carbon steal pipe wall. Based on plant operating experience the coating has the potential service life of 15 to 40 years and the pipe wall has the potential life of 12 to 50 years. Therefore, these lines should 26

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and Identify whether it Basis for Cause Determination is a root cause or contributing cause) have a minimum service life of 27 years.

Since this line was inspected in 1993, it should be inspected or pressure tested by 2020.

The 8-inch CS line (SS-4ICS-25) is the Hotwell Level Control supply line from the Condensate Pump discharge header to flow control valve V-2-17, This line Is pressurized to between 200 pslg and 250 psig during normal operation. Upon developing a through-wall leek in the pipe, leakage would Immediately exit the pipe and enter the surrounding soil. This line was assessed as having a Low Risk factor since it was assumed to be a direct burled SS line. This line was assessed a Medium Radiological Consequence factor since a leak could lead to an unmonitored release and a Medium Consequence factor since a significant leak could lead to a plant shutdown.

The fact that the two lines developed leaks at the same time can be explained as follows:

After plant shutdown on-Apri 25, the Condensate system was secured, vacuum was broken on the Main Condenser, and the 10-inch line became positively pressurized from the head of water in the CST. At this time, it is.

believed that the line began to leak water from the Condensate system to the surrounding soil.

The 10-inch line probably developed a through wall hole before the 8-inch line; however, because of the vacuum In the 27

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and identify whether It Basis for Cause Determination is a root cause or contributing cause) line during normal operation, it likely did not release tritiated water to the surrounding soil until the Condensate System was secured.

28

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and Identify whether It Basis for Cause Determination is a root cause or contributing cause)

Two buried piping Root Cause #2 - The Inadequate configuration management lines Inthe program basis document and design controls resulted in Invalid Condensate System Is flawed assumptions that were used as the basis developed leaks, for development of the burled pipe resulting in a release program and the assessment of risk and of tritium to the consequences used to Implement a ground water. strategic approach to tritium leak mitigation, Following the July 1991 leak on the 8-inch CS line, a modification was initiated to install flanged SS piping to replace CS piping from the Condensate Transfer Pump Building into the Condenser Bay and to reroute the piping from beneath the Condensate Transfer Building to outside. The modification was not implemented. However, GPU drawing 3D-421-22-1000, "Condensate System 8-Inch CH-2 Partial Replacement," Revision 0, was issued documenting the design change. The bill of materials in this drawing, GU-3BM-421-50-1001, specifies that SS be used for the replacement pipe.

The modification process at the time did not provide for requiring that the design documentation and the in-field installation of the modification be inextricably linked.

In this instance, approval and Issuance of the related design documents occurred separate from, and without coordination with, actual in-plant Installation of the modification Indrawings labeled as "INTERIM." This configuration management issue was not discovered until the discovery of tritiated water leak in 2009. Issue Report 00919332 was initiated to document this discrepancy.

29

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and Identify whether It Basis for Cause Determination is a root cause or contributing cause)

Based on design drawing 3D-421 1000, it was thought that the 8-inch line was SS, This assumption was included inthe Buried Pipe program basis document as ilrustrated in TR-1 16, Appendix 4, "Inventory of Risk and Consequence Significant Lines -

Stainless Steel Pipe." Infact, a work order to perform coatings inspections on Condensate piping in 2008 describes an 8-inch stainless steel line to be excavated as part of the work activity. Also included inthe program basis document was an assumption that coatings were replaced on the CS piping and that a 15-year life expectancy was valid for these coatings.

This assumption was based on work order C0033031. In addition, a degradation rate was determined for the piping based on service life and expected corrosion rates. Life expectancy for these lines was determined by adding the coating life expectancy to the service life expectancy for the piping. For example, the coating on the 8-Inch CS line had a postulated service life of 15 years. Based on corrosion rates, the piping service life was expected to be 15 years. Based on these assumptions, the service life of this line was determined to be 30 years.

Two buried piping Contributing Cause #1 - In September 1991, an Engineering lines in the Inadequate work evaluation was performed and Condensate System instructions, documented in P.E. 125-1 File No. 1001-developed leaks, documentation, and 91, "Coating Repair on 1-inch, 8-inch and resulting in a release work quality I0-In Snderground Condensate of tritium to the ground water. (b)(4) 30

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and Identify whether It Basis for Cause Determination is a root cause or contributing cause)

(b)(4)

'Z71 Further, the closure comments do not ocument that pipe preparations were conducted In accordance with SSPC-SP6; to the contrary, the work order notes that the pipe coating was removed and replaced In accordance with Engineering direction, and references Maintenance procedure 1000-ADM-1 100.07, "Abrasive Blasting Safe Work Procedure, but not SSPC-SP6.

The work order specified that spark testing be performed on the 8-inch and 10-inch lines. The work order does not specify a voltage for this testing and does not cite a reference for conducting the testing. This is significant in that ASTM D5162 - 08, "Standard Practice for 31

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault r

I V Cause (Describe the cause Problem Statement and Identify whether It Basis for Cause Determination Is a root cause or contributing cause) ~~1 IU.>U A'~

I- IUI~t -+II I

, 6Y -/-/

From this testing, five small areas on the 10-inch line were Identified as requiring repair. On the 8-inch line, two areas, totaling approximately two square feet, required repairs. A review of work order closure Identified that repairs were limited to the relatively small areas identified as requiring either coating or pipe repairs. It Is noteworthy that these maintenance activities did not remove all coatings for visual and/or UT inspections on the entirety of the exposed pipe. Such inspections would have allowed a more rigorous examination, particularly given that several pipes were Identified as having some degree of degradation and since the 10-inch CS line had multiple holes, Indicating the potential for 32

I Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and Identify whether it Basis for Cause Determination Is a root cause or contributing cause) extensive corrosion of the pipe.

Consistent with the point-by-point examination, the repairs of the piping and coating were performed in a *patchwork" manner.

Formal documentation for some changes was not used to establish quality records for retention, resulting in a lack of/Inaccurate documentation around pipe configuration, configuration changes, soil and material condition, and abandoned pipe. Work Order closing remarks reviewed as part of the investigation did not Include adequate descriptions of work completed. Notations in work order CREM are vague and interpretation could have altered the meaning of the notes.

Work Orders had Incomplete references to vendor manuals and procedures.

Examples exist where references made in work orders could not be followed to a retrievable document Underground piping program drawings may not have been updated and properly maintained as part of modifications. The 6-inch line was replaced In 1994, but BR-2193 was not updated to reflect the modification, The 8-inch line, SS-4, was not replaced and has no postings, Work Order C0032859 specifies that Engineering is to be involved in determining the proper method for repair of the line based on inspection results.

There is no documentation that Engineering evaluated the repair of this line. A non-conformina condition was

________________________________________I 33

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault

. Cause (Describe the cause Problem Statement and Identify whether it Basis for Cause Determination Is a root cause or contributing cause) identified as MNCF 91-0242; however, the work order closure remarks do not document the specific non-conformance.

The documented cause of failure is listed simply as upipe corrosion;" no additional investigation of cause or extent of condition is documented.

Two buried piping Contributing Cause #2 - The station had several changes of lines in the The change ownership and management between the Condensate System management processes time the leaks were Identified In 1991 and developed leaks, prior to Implementing the the 2009 leaks. During this dynamic era, resulting in a release Exelon Buried Pipe the Buried Pipe program began and of tritium to the Program did not support evolved with changes In procedures and ground water. effectively managing processes that governed its design changes and implementation. A station-specific related projects during program was Implemented in 1991. The site ownership and initial focus of the program was on management changes Service Water and Emergency Service Water systems due to their implications for plant safety. Additionally, fuel oil piping was a primary focus due to environmental concerns. Several modifications aimed at mitigating the station's risk to underground tritium leaks were planned, including design changes to move lines above ground, move piping into concrete trenches, and to replace lines using more corrosion-resistant materials. In most Instances, these planned modifications were In response to Identified leaks.

Inadequate configuration management and design controls resulted in invalid assumptions that were used as the basis for development of the program, the associated assessment of risks and consequences, and, consequently, the station's strateaic approach to tritium leak 34

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and Identify whether it Basis for Cause Determination is a root cause or contributing cause) mitigation. The station missed opportunities to properly characterize risks and consequences associated with the degraded Condensate System piping.

These risks and consequences were developed as part of the program basis document.

The program basis document is flawed in that the 8-Inch Condensate line was incorrectly identified as stainless steel as opposed to the installed CS piping.

(b)(4)

Two buried piping Contributing Cause #3 - The combined Information from non-lines in the Limitations in available intrusive and direct Inspections is used to Condensate System technologies used to identify the need for long-term repairs and developed leaks, assess pipe material replacements. The Inspections rely on resulting in a release condition. available technologies, each of which has of tritium to the specific limitations. This includes Visual ground water, inspection, Ultrasonic Testing (UT),

Guided WaveTM, and C-ScanTM inspections.

Any of these methods alone would not allow for 100% assurance of pipe integrity. Instead, they are used in combination to find localized areas of concern. Since 100% verification of pipe integrity is not practical, even these extensive measures leave vulnerable to localized corrosion because the methodologies used by the buried pipe program do not, in all instances, locate existing defects and cannot assess entire continuous full lengths of pipe.

35

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause (Describe the cause Problem Statement and identify whether it Basis for Cause Determination Is a root cause or contributing cause)

For example, a UT examination typically covering just a few linear feet of large diameter pipe can take several hours, and requires an additional shift or more to property prepare the pipe surface prior to the inspection. This makes UT inefficient in assessing the condition of a piping system; consequently, this method is not efficient with respect to finding wall thinning before It leaks. Inthe case of buried pipe, costly and time consuming excavation would be required for complete OD access to the piping, or equally challenging operational and configuration changes would be required to allow complete ID access.

The industry considers Guided Wave TM technology to be an acceptable means to perform quick non-intrusive Inspections of long sections of pipe to identify degradation. However, industry experience has shown that at the current level of development, even the most sophisticated indirect Inspection methods are not as accurate at sizing the axial and circumferential extents of a corroded area as UT and, as a result, a follow-up direct examination such as UT would be required to characterized the area of corrosion.

36

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Extent of Cause:

Cause being Extent of Cause addressed (b)(4)

Root Cause #1 - Anodic dissolution resulting from disbondment of the coating and susceptible material Root Cause #2 - The program basis document *",7 Is flawed.

Contributing Cause #1 -

Inadequate work instructions, document, and work'quality Contributing Cause #2 -

The change management processes prior to Implementing the Exelon Buried Pipe Program did not support effectively managing design changes and related projects during site ownership and management changes 37

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause being addressed Extent of Cause Contributing Cause #3 - (b)(4)

Limitations in available technologies used to assess pipe material condition.

38

w Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Risk Assessment:

PLANT-SPECIFIC RISK BASIS FOR DETERMINATION CONSEQUENCE (bX4) 39

w Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Equipment Checklist:

Step I Run To Failure (RTF) Classification Check Is the component incorrectly classified as Critical, Non-Critical or Run-to-Failure per MA-AA-716-210? No Step 2 PMIPDM Review Has the past PMIPDM not been performed In accordance with the PCM template? No PCM Template was found for Burled Piping, only Cathodic Protection.

Step 3 Maintenance Performance Assessment Is there a deficiency with the performance of the most recently performed maintenance? Yes, compared to today's requirements.

Step 4 Performance Monitodng Assessment Has system/component monitoring been deficient in Identifying normal or abnormal equipment degradation? Yes, see Timeline for details.

Step 5 Operating Experience Review Is there a deficiency in how past operating experience (OPEX) applicable to this component has been addressed? No, risk ranking included a review of applicable OPEX.

Step 6 PCM Template Review Is there a deficiency in any PCM template applicable to this component?

Preventative Maintenance is part of the program and not covered through the PCM Template. Cathodic Protection Is not utilized at the site, so the PCM Template for it does not apply.

Step 7 Operational Performance Review Are the operating procedures or practices for this component inappropriate or unacceptable? They are acceptable per the Buried Piping Program.

Step 8 Maintenance Practice Review Are there problems with the maintenance practices, behaviors or training for this component? There is no training for coating applications. This Is addressed in the corrective actions section of the report.

Step 9 Design Review Is the design configuration for this component Incorrect? This is addressed in the cause and corrective action sections of the report, Step 10 ManufactureNendor Quality Check Is there a concern with the quality of parts, shipping or handling? No Step 11 Problemlissue Management Review Have previous issues not been adequately addressed including but not limited to aging, obsolescence, chronic problem, scheduling, or business planning? The inspections were based on risk assessments made and effectively managed based on the assumption that the risk assessments were correct.

Step 12 Latent Weaknesses Document in the investigation report whether the event should have been 40

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault reasonably prevented by improved work preparation, effective troubleshooting, or management oversight. See Attachment 7 for guidance. This Is documented throughout the report.

Step 13 Unknown or Different Cause Did the equipment fail due to an unknown cause or other cause than listed in steps 1 through 11 above? No Previous Events:

9ý'/7//

41

V Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault 42

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault 43

NNNNNNNNNMXý Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Immediate Actions:

Immediate Actions Taken or Planned Owner Due Date (Include AT Assignment #)

IR written to document IR 00907846 (b)(6) !Complete the discovery of tritium in the Emergency Service Water Vault.

Created PIMS AR from PIMS AR A2222268 Complete IR 00907846 to track all work orders Initiated to locate and repair the source of the tritium leak _

Perform a Prompt IR 00907846 Assignment 2 Complete Investigation and obtain management approval.

After approval, submit the PINV to the NDO Mailbox and the NRC Resident.

Evaluate all inputs into IR 00907846 Assignment 2 Complete and from the ESW vault for potential sources of tritium and inspect other vaults in the vicinity and sampling for tritium as warranted Collect water samples IR 00907846 Assignment 2 Complete from:

Monitoring wells in the general vicinity of the cable vault: W-3, W-4, W-5, W-6 and MW-15K-IA along with Lysimeter in the vicinity of the cable vault (CST-9), and the Lysimeter on the southwest comer of the

-Condensate Storage Tank (CST-2). Monitor surface water sample from the Main Condenser discharae.

44

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Immediate Actions Taken or Planned (lIclude AT Assignment #)

Ship the samples to IR 00907846 A Teledyne Brown Engineering to be analyzed for tritium, strontium-90 and gamma emitting radionuclides utilizing the detectable limits (LLDs) specified for the Radiological Groundwater Protection Proaram.

Install temporary clamp Work Order C2021073 repair on the Identified leaks Drill sample wells per Work Order C2021076 digging permit _

CS-26 and CTB drain Work Order C2021083 line inspection Inspect and repair if Work Order C2021104 necessary the 1-inch condensate pipe Repair/Replace the 8- Work Order C2021105 inch condensate pipe ......

Repair/Replace the 10- Work Order C2021108 inch condensate pipe I Prepare and Issue OTDM IR 00907846 Assignment 11 for the leak in the condensate transfer svstem 45

t I Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Corrective Actions to Prevent Recurrence (CAPRs):

Root Cause Being Corrective Action to Prevent Addressed (Include Recurrence (CAPR)

TapRoot Codes) (Include AT Assignment #)

Anodic dissolution CAPR Passport ATI 907846 resulting from Assignment 14: Implement a disbondrnent of the strategic plan that includes coating and susceptible moving direct buried material. (Corrective Condensate and Condensate Action Needs Transfer System piping either Improvement 5CL) above ground or In monitored trenches (Root Cause 1)

The Program Basis CAPR Passport ATI 907846 September Document is Flawed Assignment 15: Perform a 25, 2009 (SPAC has a Technical thorough program assessment Error5TE) and revise the program basis document (i.e., TR-1 16) to correct errors In plant design details, risks, consequences, and recommend Inspection frequencies and methods following the assessment (Root Cause 2) 46

. F Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Corrective Actions:

Cause Being Corrective Action (CA) or Addressed (Include Action Item (ACIT) Owner Due Date TapRoot Codes) (Include AT Assignment #)

(b)(6)

Root Cause #1: Anodic CA Passport AT) 907846 December 11, dissolution resulting from Assignment 16: Site Design 2009 disbondment of the Engineering develop engineered coating and susceptible solutions for bringing buried material. (Corrective piping above ground or Action Needs entrenching the pipes.

Improvement 5CL) ACIT Passport ATI 907846 August 28, Assignment 17: Perform an 2009 extent of condition review for maintenance performed on direct buried piping prior to 0,.

implementation of the Exelon Buried Pioe Proaram Root Cause #1: Anodic ACIT Passport ATI 907846 October 30, dissolution resulting from Assignment 18: Evaluate 2009 disbondment of the installing sacrificial coating and susceptible anodes/impressed current to material. (Preventative mitigate galvanic corrosion and Maintenance Needs present to PHC.

Improvement 4PL) ACIT Passport ATI 907846 October 30, Assignment 19: Clearly 2009 document As-Left conditions of buried piping in the program basis document.

Root Cause #2: The program basis document ACIT Passport ATI 907646 Assignment 20: Review work July 31, 2009 I is flawed. (SPAC has a packages associated with the technicalerror 5TE) 2009 leak for adequate closure documentation, documentation of as-left conditions, and documentation of compliance with guidance documents for the work performed 47

I I Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Cause Being Corrective Action (CA) or Addressed (Include Action Item (ACIT)

TapRoot Codes) (include AT Assignment U I

ACIT Passport ATI 907846 September 25, Assignment 21: Maintenance 2009 Curriculum Review Committee evaluate the need for work group training on buried pipe coating application.

ACIT Passport ATI 907846 September 25, Assignment 22: Engineering 2009 Curriculum Review Committee evaluate the need for work group training on buried pipe coating aDDUlcation.

August 28, Contributing Cause #1: CA Passport ATI 907846 Inadequate work Assignment 23: Institutionalize 2009 instructions, guidance to document As-Left documentation, and work conditions in work order closure quality (Work documentation following Package/PermitNeeds excavations Improvement 5WP)

Contributing Cause #1: ACIT Passport ATI 907846 September 25, Inadequate work Assignment 24: Maintenance 2009 Instructions, Curriculum Review Committee documentation, and work evaluate the need for work group quality (Standards, training on work order closure Policies, or quality.

Administrative Controls ACIT Passport ATI 907846 September 25, are Not Strict Enough AssIgnment 25: Engineering 2009 5NS) Curriculum Review Committee evaluate the need for work group training on work order closure quality.

48

6 Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause Being Corrective Action (CA) or Addressed (Include Action Item (ACIT) Owner Due Date TapRoot Codes) (Include AT Assignment #)

Contributing Cause #2: CA Passport ATI 907846 (b)(6)

August 28, The change Assignment 26: Incorporate 2009 management processes signatures for Engineering prior to Implementing the inspection/approvals for pipe Exelon Buried Pipe repair and replacement in work Program did not support orders to ensure the desired effectively managing results through maintenance are design changes and being achieved.

related projects during site ownership and management changes.

(Standards,Policies, or Administrative Controls are Confusing or Incomplete 5C1J Contributing Cause #2: CA Passport ATI 907846 December 18, The change Assignment 27: Update buried 2009 management processes piping design documents to as-prior to implementing the built conditions Exelon Buried Pipe Program did not support effectively managing design changes and related projects during site ownership and management changes.

(Standards,Policies, or Administrative Controls has a Technical Error 5TEJ

£ _______________________ L 49

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Cause Being Corrective Action (CA) or Addressed (Include Action Item (ACIT) Owner Due Date TapRoot Codes) (Include AT Assignment #)

Contributing Cause #2: ACIT Passport ATI 907846 (b)(6) September 28, The change Assignment 28: Review the 2009 management processes Oyster Creek buried piping prior to implementing the database for inaccuracies Exelon Buded Pipe Program did not support effectively managing design changes and related projects during site ownership and management changes.

(Employee CommunicationsNeeds Improvement 5EQ)

Contributing Cause #2- CA Passport ATI 907846 September 28, The change Assignment 29: Update the 2009 management processes Oyster Creek buried pipe prior to implementing the database following reviews Exelon Buried Pipe completed per Passport ATI Program did not support 907846 Assignment 28 effectively managing design changes and related projects during site ownership and management changes.

(No Standard Turnover/Transition Process Exists 5TP)

Contributing Cause #3 - ACIT Passport ATI 907846 October 16, Umitatlons in available Assignment 30: Review Guided 2009 technologies used to Wave and Ultrasonic results from assess pipe material previous inspections for condition (Technological Indications of non-conservative Limitations IZ) results. ] ....

50

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Effectiveness Reviews (EFRs):

Effectiveness Review Action CAPR I CA Being (Single/Collective) Owner Due Date Addressed (Include AT Assignment #)

(b)(6)

CAPR Passport ATI 907846 EFR Passport All 907846 December Assignment 14: Implement a Assignment 31: Determine the 4, 2009 strategic plan that Includes acceptance criteria for the amount moving direct buried of Condensate and Condensate Condensate and Transfer System piping that Condensate Transfer remains direct buried without System piping either above inclusion in the strategic plan ground or in monitored following 1R23; measure trenches effectiveness against this criteria.

CAPR Passport AT] 907846 EFR Passport ATI 907846 (b)(6) July 157 Assignment 15: Revise the Assignment 32: Monitor the 2010 program basis document health of the program against (I.e., TR-116) to correct plant current performance indicators for design details, risks, improvements after the program consequences, and basis document has been recommend inspection corrected.

frequencies and methods following a thorough program assessment.

Programmatic/Organizational Issues:

Programmatic and Corrective Action (CA) or Organizational Weaknesses Action Item (ACIT)

(Causal Factor) (Include AT Assignment #)

The Program basis document ACIT Passport ATI 907846 is flawed. Assignment 33: Develop a schedule for periodic focused area assessments of the site's buried pipe program on a set frecuencv.

51

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Other Issues:

Other Issues Corrective Action (CA) or identified during Action Item (ACIT) Owner Due Date investigation (Include AT Assignment #)

All issues adverse to No additional corrective N/A N/A quality were Identified actions are required.

Inthe body of the report. I_------

Communications Plan:

Lessons Learned to be Communication Plan Action Owner Due Date Communicated (Include AT Assignment # _)_E :I Root and Contributing NNOE Passport ATI 907846 b)(6) Complete Causes of this event. Assignment 13: Communicate a summary of this root cause report to the appropriate parties via NNOE.

Attachments:

Title A Root Cause Report Quality Checklist B Oyster Creek Buried Piping Program Description and Status C Event and Causal Chart D Why Staircase E Condensate Transfer Lines As-Found Condition F Three Mile Island Anodic Dissolution Photographs 52

Root Cause Evaluation Report Tritium Identified In Emergency Service Water (ESW) Vault Attachment A Root Cause Report Quality Checklist Page 1 of 2 (b)(4) 53

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Attachment A Root Cause Report Quality Checklist (b)(4) 54

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Attachment B: Oyster Creek Buried Piping Program Description and Status F

4-I J 55

Root Cause Evaluation Report Tritium Identified in Emergency Service WaterIESM Vauff

/

Root Cause Evaluation Report Tritium Identified in Emeraency Service Water (ESW) Vault 57

Root Cause Evaluation Report Tritium Identified In Emeraencv Service Wate 58

Root Cause Evaluation Report Tr~hitm Identified in Fmernancv Service Water IESW) Vat It 59

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault 60

Root Cause Evaluation Report Tritium Identified in Emergencv Service Water (ESW) Vault 61

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault 62

0 Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Attachment C: Event and Causal Chart "Trlt*kr 20M9 Event and Causal Chart 2.p 63

P Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Attachment D: Tritium Leak 2009 Why Staircase Condensate Lines Leaked Anodic Dissolution Anodic Dissolution Coating Failure Coating Failure Poor Application Poor Application Coating Repaired in Small Areas only Coating Repaired in Technical Document Small Areas Only 125-1 Evaluation 1001-91 Directions Technical Document No Fleet Standard 125-1 Evaluation Exists for Pipe 1001-91 Directions Coating 64

Root Cause Evaluation Report Tritium Identified in Emergency Service Water (ESW) Vault Condensate Lines Leaked Thinning of Pipe Not Identified by the Program Thinning of Pipe Not Identified SS-4 Assumed to by the Program be Stainless Steel and thought to have a lower susceptibility than Actual SS-4 Assumed to be Stainless Steel OC-MM-323643-001 And thought to have a'lower documented mod Susceptibility than Actual indicated change to Stainless Steel assumed complete OC-MM-323643-001 documented Poor Documentation Mod indicated change to Stainless Steel assumed complete Condensate Lines Leaked

• Thinning of Pipe Not Identified by the Program Thinning of Pipe Not Identified , Program Basis Document contained Incorrvct assumptions Program Basis Document Lack of independent Contained incorrect assumptions review of the document Lack of independent review of Insufficient/

the document nonexistent program reviews 65