ML022750125

Confirmatory Action Letter - Revision 1 to Root Cause Analysis Report Regarding Reactor Pressure Vessel Head Degradation at the Davis-Besse Nuclear Power Station

ML022750125
Person / Time
Site:	Davis Besse
Issue date:	09/23/2002
From:	Myers L FirstEnergy Nuclear Operating Co
To:	Dyer J NRC/RGN-III
References
1-1289
Download: ML022750125 (186)

Text

FENOC F

O5501 North State Route 2 FirstEnergy Nuclear Operating Company Oak Harbor, Ohio 43449 Lew W Myers 419-321-7599 Chief Operating Officer Fax: 419-321-7582 Docket Number 50-346 License Number NPF-3 Serial Number 1-1289 September 23, 2002 Mr. J. E. Dyer, Administrator United States Nuclear Regulatory Commission Region III 801 Warrenville Road Lisle, IL 60532-4351

Subject:

Confirmatory Action Letter Response - Revision 1 to Root Cause Analysis Report Regarding Reactor Pressure Vessel Head Degradation Ladies and Gentlemen:

On April 18, 2002, the FirstEnergy Nuclear Operating Company submitted the Root Cause Analysis Report regarding the Reactor Pressure Vessel (RPV) head degradation at the Davis-Besse Nuclear Power Station, Unit 1 (DBNPS) pursuant to the Nuclear Regulatory Commission's (NRC) March 13, 2002, Confirmatory Action Letter (CAL).

In concert with the Root Cause Analysis Report, which evaluated the technical issues of the RPV head condition, a Root Cause Analysis Report on Failure to Identify RPV Head Degradation was prepared to address the non-technical aspects of the condition. The non-technical Root Cause Analysis Report was submitted to NRC by FENOC letter Serial Number 1-1286, dated August 21, 2002.

The technical Root Cause Analysis Report regarding the RPV head degradation also preliminarily addressed several non-technical issues that have been more fully investigated and documented in the non-technical Root Cause Analysis Report.

Therefore, the technical Root Cause Analysis Report has been revised to delete! those conclusions that are more completely addressed in the non-technical Root Cause Analysis Report. The revised technical Root Cause Analysis Report regarding the RPV head degradation is enclosed. The revisions are discussed in summary in the "Purpose and Scope of the Root Cause Analysis Report" section of the enclosure.

Please be advised that this revised technical Root Cause Analysis Report does not address findings of inspections of the degraded RPV head section at Framatome ANP's facilities that have been recently reported; i.e., discovery of a previously unidentified SEP 6 ?007

Docket Number 50-346 License Number NPF-3 Serial Number 1-1289 Page 2 of 2 circumferential crack in Nozzle 3 and discovery of previously unidentified cracks in the cladding area. Evaluation of these issues is continuing. It is not anticipated that the evaluations will have any effect on the conclusions of the enclosed technical Root Cause Analysis Report; therefore, no further revision of the technical Root Cause Analysis Report is contemplated.

If you have any questions or require additional information, please contact Mr. Patrick J.

McCloskey, Manager - Regulatory Affairs, at (419) 321-8450.

Very truly yours, R

Enclosure and Attachment cc:

USNRC Document Control Desk J.B. Hopkins, DB-1 NRC/NRR Senior Project Manager S.P. Sands, DB-1 NRC/NRR Backup Project Manager C.S. Thomas, DB-1 Senior Resident Inspector Utility Radiological Safety Board

Docket Number 50-346 License Number NPF-3 Serial Number 1-1289 Enclosure Page 1 of 1 Root Cause Analysis Report (182 Pages Follow)

Docket Number 50-346 License Number NPF-3 Serial Number 1-1289 Attachment Page 1 of 1 COMMITMENT LIST The following list identifies those actions committed to by the Davis-Besse Nuclear Power Station (DBNPS) in this document. Any other actions discussed in the submittal represent intended or planned actions the DBNPS. They are described only for information and are not regulatory commitments. Please notify the Manager - Regulatory Affairs (419-321-8450) at the DBNPS of any questions regarding this document or associated regulatory commitments.

COMMITMENTS DUE DATE None

FirstEnergy, DAVIS-BESSE NUCLEAR POWER STATION Root Cause Analysis Report Significant Degradation of the Reactor Pressure Vessel Head CR 2002-0891, Dated 3-8-2002 REVISION 1, DATE: 8-27-2002 Prepared by:

S. A. Loehlein Reviewed by:

J. J. Powers Root Cause Team Lead Director Approved by<ýý L. W. Myers iief~perating Officer

PURPOSE AND SCOPE OF THE ROOT CAUSE ANALYSIS REPORT Purpose Determine the root and contributing causes for Reactor Pressure Vessel closure head (RPV head) damage experienced at nozzle 3 and minor corrosion at nozzle 2, to support the operability determination for the station's as-found condition and the future repair plan.

Scope Very early in the development of the response to this condition, it became clear that the technical causes behind the cracking of the Control Rod Drive Mechanism (CRDM) nozzles and the ensuing corrosion of the head material would draw much attention and comparison to the previously developed body of knowledge on related topics and conditions. In fact, the possibility of nozzle cracks existing at Davis-Besse was well recognized prior to the condition, but the identified significant damage to the RPV head had not been anticipated.

The unexpected finding of the significant damage at Davis-Besse immediately became a concern, both for the possible extent of condition implications at Davis-Besse and the potential impact to the industry. Therefore, the objective of the Initial Investigative Team was a prompt investigation into the primary cause(s) of the damage. The findings within this report are expected to invite input from industry experts and scientists resulting in additional study of the evidence, and further research into the topics of CRDM nozzle cracking and boric acid corrosion.

In order to provide timely insight for the plant and the industry, and to comply with the NRC's Confirmatory Action Letter on the head degradation, revision 0 of this report was prepared with the full knowledge that a number of confirmatory activities were still continuing. Revision 1 of this report is being issued as an update of the relevant technical issues associated with the RPV head damage. No changes of note have occurred in any technical conclusions from revision 0.

No additional future revisions of this report are anticipated.

Management issues identified in revision 0 have been largely removed from this report, having been separately investigated in detail. The results of the management and human performance investigation appear in the Root Cause Analysis Report entitled "Failure to Identify Significant Degradation of the Reactor Pressure Vessel Head", dated 8-13-2002.

Investigative Team Membership Steve Loehlein, FENOC (Beaver Valley), Team Lead Chuck Ackerman, FENOC (Davis-Besse)

Ted Lang, FENOC (Davis-Besse)

Todd Pleune, FENOC (Davis-Besse)

Neil Morrison, FENOC (Beaver Valley)

William Mugge, FENOC (Davis-Besse)

Joseph Rogers, FENOC (Davis-Besse)

Technical expertise provided by:

Dr. Mark Bridavsky, FirstEnergy, Beta Labs - Failure Analysis Expert Stephen Hunt, Dominion Engineering - Corrosion Expert Steve Fyfitch, Framatome ANP, Metallurgical Expert Christine King, EPRI, Material Reliability Program Manager

Assessment of management aspects/decision making:

John B. Martin, Corporate Nuclear Review Board E. J. Galbraith, Senior Representative, Assistance, Institute of Nuclear Power Operations

Table of Contents Title Page No.

1.0 Problem Statement 1

1.1 Reason for Investigation 1

1.2 Consequences of Event/Condition Investigated 1

1.3 Immediate Actions Taken 1

2.0 Event Narrative 2

2.1 Background

2 2.2 Sequence of Events 3

3.0 Data Analysis 4

3.1 Non-Destructive Examinations of RPV Head and Nozzles 4

3.1.1 Potential Evidentiary Request List 5

3.1.2 Locations of Cracks and Corrosion on RPV Head 5

3.1.3 NDE Examinations of CRDM Nozzles 5

3.1.4 Visual Examinations of RPV Top Head and Penetrations 7

3.1.5 Boric Acid Sample Results 7

3.2 Cracks, Leaks and Corrosion 10 3.2.1 CRDM Nozzle Cracks and Propagation to Leakage 10 3.2.2 Leakage Rate From CRDM Nozzle Cracks 18 3.2.3 Source of Boric Acid Deposits on RPV Head 22 3.2.4 Corrosion of RPV Top Head Surface 23 3.3 Investigation of Lead Indicators 29 3.3.1 Timeline 29 3.3.2 Sequence of Relevant Events 30 3.3.3 CRDM Flange and RPV Head Inspections during Refueling Outages 30 3.3.4 Containment Air Cooler Cleaning 35 3.3.5 Containment Radiation Monitor RE4597 Observations & Filter Plugging 37 3.3.6 Containment Recirculation Fan/Fan Failures 40 3.4 Programs Important to Preventing Problems 41 3.4.1 B&W Owners Group and Industry CRDM Nozzle Related Initiatives 41 3.4.2 Davis-Besse Boric Acid Corrosion Control Program 46 3.4.3 Davis-Besse Inservice Inspection Program 48 3.4.4 Evaluation of Condition Report Responses 49 3.5 Related Issues 50 3.5.1 RPV Head Inspections 50 3.5.2 Restart Readiness 51 3.6 Causal Factors/Conclusions 51 Root Cause Analysis Report Table of Contents

• i I

4.0 Experience Review 54 4.1 Davis-Besse Experience 54 4.2 Nuclear Industry Experience 54 4.3 Conclusions 54 5.0 Root Cause Determination 55 5.1 Probable/Root Causes 55 5.2 Contributing Causes 55 6.0 Extent of Condition 56 6.1 Degradation Mechanism Issues 56 7.0 Recommended Corrective Actions 58 7.1 Probable/Root Causes Corrective Actions 58 7.2 Contributing Causes Corrective Actions 59 7.3 Additional Actions 59 8.0 References 61 8.1 Davis-Besse References 61 8.2 Vendor References 63 8.3 NRC References 64 8.4 INPO References 65 8.5 Industry References 66 8.6 Other References 66 9.0 Personnel Interviews 67 9.1 Personnel Interviewed 67 9.2 Personnel Consulted 68 10.0 Methodologies Employed 69 Root Cause Analysis Report Table of Contents

• ii I Root Cause Analysis Report Table of Contents

• ii I

Tables Title Page No.

1.

Nozzle 1 NDE Examination Results 70

2.

Nozzle 2 NDE Examination Results 71

3.

Nozzle 3 NDE Examination Results 72

4.

Nozzle 5 NDE Examination Results 73

5.

Nozzle 47 NDE Examination Results 74

6.

Comparison of Davis-Besse to Other B&W Design Plants 75

7.

Nuclear Industry Experience Review Results 76 Root Cause Analysis Report Tables

• iii I Root Cause Analysis Report Tables 9 iiiI

Figures Title Page No.

1.

Davis-Besse RPV Top of Head Section View 88

2.

Davis-Besse RPV Top of Head Plan View 89

3.

Davis-Besse CRDM Nozzle General Arrangement 90

4.

Boric Acid and Iron Oxide on Vessel Flange at 12RFO 91 5

Nozzle 2 Corrosion Area Location, Size, and Profile 92

6.

Cavity in Reactor Vessel Head between Nozzle 3 and 11 93

7.

Locations of Cracks and Corrosion on Davis-Besse RPV Head at 13RFO 94

8.

Nozzle 1 Crack Locations and Sizing 95

9.

Nozzle 2 Crack Locations and Sizing 96

10.

Nozzle 3 Crack Locations and Sizing 97

11.

Nozzle 5 Crack Locations and Sizing 98

12.

Nozzle 47 Crack Locations and Sizing 99

13.

Corrosion and Possible Impingement at Nozzle N-3 100

14.

Nozzle 3 Clad Thickness Measurements 101

15.

Hoop Stresses and Operating Condition Deflections in CRDM Nozzles 2-5 102

16.

Location of Leaking Nozzles in B&W Design Plants 103

17.

Distribution of Leaking Nozzles in B&W Design Plant 104

18.

CRDM Nozzle Leakage Observed at Oconee 3 105

19.

Unidentified Leak Rate at Davis-Besse (Cycle 13) 106

20.

As Found Locations of Boric Acid Deposits on Davis-Besse Vessel Head 107

( 1ORFO to 13RFO)

21.

Nozzle Crack Leakage Rate Calculation Results 108

22.

Finite Element Model Boundary Conditions to Simulate Axial Crack 109

23.

Crack Opening Displacement with the Crack Surface Nodes Released 110

24.

Boric Acid Deposits on Top of Head at Start of 13 RFO 111

25.

Corrosion Rate for EPRI Experiments (Proprietary) 112

26.

Timeline of Key Events Related to Reactor Vessel Head Boric Acid Wastage 113

27.

Events & Casual Factor Chart 1 14a-e Root Cause Analysis Report Figures

• iv I

[

Root Cause Analysis Report Figures a iv I

Title Page No.

28.

Leaking Flanges Found and Repaired During Each Outage 115

29.

Flange Leakage with Stalactite Formation from Insulation and Stalagmite 116 Formation on top of Reactor Vessel Head (8RFO)

30. Flange Leakage Crusted On Side of Nozzles and Stalactites from Gaps in 117 Insulation (8RFO)

31.

Reddish Brown Boron Deposits Crusted on Side of Nozzle (8RFO) 118

32.

Boron Deposits - Source Unclear (8RFO) 119

33.

North Side of Reactor Vessel Head (10RFO) 120

34.

Boron Deposits Near Top of Reactor Vessel Head (1 ORFO) 121

35.

Typical Deposits for Periphery (1ORFO) 122

36.

Red Rusty Boric Acid Deposits on Vessel Flange (I2RFO) 123

37.

Boron Piled Under the Insulation (1 IRFO) 124

38.

Boric Acid Deposits with Heavy Iron Concentration on Underside of Nozzle 3 125 (13RFO)

39.

2000 Interferences with CRDM Flange Inspection 126

40. RE4597 Sample Location 127

41.

CTMT Radiation Monitors RE4597AA/BA (Combined Iodine Channels) 128

42.

CTMT Radiation Monitors RE4597AA & BA (Both Noble Gas Channels) 129

43.

Potential Effects of Boric Acid Deposits on Vessel Top Head Surface 130

44.

Crack Profile for Nozzle 3, Flaw #1 131 Root Cause Analysis Report Figures

• v

Attachments Title Page No.

1.

Potential Evidentiary Request List (Rev. 4) 132

2.

Sequence of Relevant Events 135 Root Cause Analysis Report Attachments

• vi Root Cause Analysis Report Attachments

• viI

1.0 Problem Statement 1.1 Reason for Investigation Significant degradation of the reactor pressure vessel top head base metal was discovered at nozzle 3 (toward nozzle 11) and minor corrosion at nozzle 2 during the thirteenth refueling outage (13RFO) in March, 2002.

This root cause report addresses the cause of the loss of RPV head base metal in the region of nozzles 3 and 2.

1.2 Consequences of Event/Condition Investigated The RPV head is an integral part of the reactor coolant pressure boundary, and its integrity is vital to the safe operation of the plant. Degradation of the RPV head or other portions of the reactor coolant pressure boundary can pose a significant safety risk if permitted to progress to the point where there is risk of a loss of coolant accident. Analysis indicates that the as-found condition of the affected nozzles would not have been expected to result in failure of the pressure integrity of the reactor coolant system. However, the degraded condition had been progressing over a period of time, without knowledge of the condition.

1.3 Immediate Actions Taken

1. At the time of discovery, the plant was already in a safe, shutdown condition. Ongoing outage activities related to the repair of the CRDM nozzle on the RPV head were suspended.

2. A root cause evaluation team was convened to perform the initial investigation.

3.

A plan was created to preserve and collect, evidence necessary for the investigation.

Root Cause Analysis Report 1.0 Problem Statement

• 11

2.0 Event Narrative

2.1 Background

Davis-Besse is a raised loop pressurized water reactor (PWR) manufactured by Babcock and Wilcox (B&W). The reactor licensed thermal power output is 2772 megawatts. The plant achieved initial criticality on August 12, 1977. The RPV has an operating pressure of 2155 psig and a design pressure of 2500 psig. Davis-Besse has accumulated 15.78 effective full power years (EFPY) of operation when the plant was shut down for 13RFO.

The RPV head has 69 CRDM nozzles welded to the RPV head of which 61 are used for CRDMs, seven are spare, and one is used for the RPV head vent piping. Each CRDM nozzle is constructed of Alloy 600 and is attached to the RPV head by an Alloy 182 J-groove weld. The RPV head is constructed of low-alloy steel and is internally clad with stainless steel. Figures 1, 2 and 3 show the arrangement of the Davis-Besse RPV head. Figure 1 is a section view through the RPV centerline, Figure 2 is a plan view from the top of the RPV closure head, and Figure 3 shows how the CRDM nozzles are welded into the RPV head.

Throughout this report the CRDM nozzles will be addressed as nozzles 1, 2,....69 and not the associated nozzle core grid location. Given that many of the sources referenced during the root cause analysis utilized the nozzle core grid location, the list below provides a correlation between the CRDM nozzle and core grid location.

CRDM NOZZLE #

1 2

3 4

5 6

7 8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 CORE GRID LOCATION H8 G7 G9 K9 K7 F8 H10 L8 H6 F6 FI0 LIO L6 E7**

E9*

GII*

KII*

H9*

M7*

K5*

G5*

D8 H12 CRDM NOZZLE #

24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 CORE GRID LOCATION N8 H4 E5 Ell Mil M5 D6 D10 F12 L12 NI0 N6 L4 F4 C7 C9 G13 K13 09 07 K3 G3 D4 CRDM NOZZLE #

47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 CORE GRID LOCATION D12 N12 N4 C5 CII E13 M13 O11 O5 M3 E3 B8 H14 P8 H2 1B6 B10 F14 L14 P1O P6 L2 F2

• Spare nozzles

"**Head vent connection Root Cause Analysis Report2.EvnNarte 2I 2.0 Event Narrative o 2 1

On August 12, 2001, Davis-Besse received Nuclear Regulatory Commission (NRC)Bulletin 2001-01 Circumferential Cracking of Reactor Pressure Vessel Head Penetration Nozzles (reference 8.3.5). In discussion held with the NRC on November 28, 2001, in response to this bulletin, Davis-Besse committed to a 100% qualified visual inspection, non-destructive examination (NDE) of 100% of the CRDM nozzles and characterization of flaws through destructive examination should cracks be detected. During performance of these inspections during 13RFO significant degradation of the RPV top head base metal was discovered between nozzles 3 and 11 and some minor corrosion at nozzle 2 in March 2002.

2.2 Sequence of Events Because the sequence of events is in part developed based upon inferred information rather than conclusive validated facts, a sequence of events will not be discussed here. Instead the data analysis section will develop the bases for the sequence of events in determining the associated causes. Attachment 2 provides a sequence of relevant events from source documents reviewed during the root cause analysis process. Figures 26 and 27 provide a timeline of key events related to RPV head boric acid corrosion and the event and causal factors chart that provide a summary level sequence of events information developed as a result of the data analysis.

Root Cause Analysis Report 2.0 Event Narrative 9 3 1

3,0 Data Analysis The data analysis section provides a summary of the data gathered during the root cause investigation. The data gathered for this analysis is from related potential condition adverse to quality reports (PCAQRs) and condition reports (CRs), the System Engineer's System Performance Book, pictures taken during inspections, personnel interviews, procedures, and other documents identified in the references section.

3.1 Non-Destructive Examination of RPV Head and Nozzles The following is a summary of the non-destructive examination effort on the cracked nozzles and degraded RPV head. After removal of insulation from the RPV flange early in 13RFO, boric acid crystal deposits and iron oxide were found to have flowed out from several of the openings (mouse holes) in the lower service structure support skirt (Figure 20). Figure 4 shows deposits on the flange during the inspection in the twelfth refueling outage (1 2RFO).

Blade probe ultrasonic (UT) examination of the CRDM nozzles from below the RPV head for circumferential cracks and large axial cracks, identified axial cracks in nozzles 1, 2, 3, 5, 47, and

58. Supplemental top-down UT examination of these nozzles confirmed through-wall axial cracks extending above the J-groove weld elevation in nozzles 1, 2 and 3. Axial cracks were confirmed in nozzles 5 and 47, but they did not extend above the top of the J-groove weld and would not have caused a leak. Axial cracking was not confirmed in nozzle 58. A small (340 arc length and 0.34" deep) circumferential crack was discovered above the J-groove weld on the outside of nozzle 2 on the downhill side. It was determined that all five nozzles would be repaired by boring out the lower part of the nozzle containing the cracks, and rewelding the end of the nozzle to the opening in the RPV head using the Framatome ANP repair method.

After removing the lower part of nozzle 3, a cavity was discovered in the low-alloy steel RPV head material above the J-groove weld on the downhill side. Additionally, after removing the lower part of nozzle 2, a smaller area of corrosion of the low-alloy steel RPV head material was discovered between the bottom of the machined nozzle and the top of the J-groove weld (Figure 5). This area of corrosion was found to extend under the portion of the nozzle left in place after machining.

After pulling nozzle 3 and cleaning by hydrolasing, the top of the RPV head was inspected using a video camera on a long pole through the vacated nozzle 3 penetration. This inspection showed a large cavity in the low-alloy steel RPV head material between nozzles 3 and 11 (Figure 6). The area with missing material was reported as being about 6.6" long and approximately 4-5" at the widest point. Ultrasonic thickness measurements from the underside of the RPV head showed the thickness of the remaining material (cladding) to be an average of approximately 0.3", which is greater than the 3/16" nominal or 1/8" minimum specified clad thickness. The videotape inspections also showed a small area of corrosion where nozzle 2 penetrates the RPV top head surface. The small area of corrosion at the top of nozzle 2 was found to lie directly over the area of corrosion at the bottom of nozzle 2 as seen in Figure 5. The videotape inspection also showed evidence of a small leak path where nozzle I penetrates the RPV top head surface.

Root Cause Analysis Report 3.0 Data Analysis

• 4 I Root Cause Analysis Report 3.0 Data Analysis

• 4 1

3.1.1 Potential Evidentiary Request List One of the first priorities of the root cause evaluation team was to collect and preserve the evidence necessary to facilitate the root cause evaluation. A Potential Evidentiary Request List was created specifying the evidence to be collected and preserved and the reason for collecting the evidence. The list was revised several times throughout the evaluation (see Attachment 1, which is current to early April, 2002). This list was used to create integrated examination and inspection plans for field implementation.

3.1.2 Locations of Cracks and Corrosion on RPV Head Figure 7 shows locations of cracks and corrosion on the Davis-Besse RPV top head surface. This figure is included to serve as a reference for the following descriptions of the degraded areas.

3.1.3 NDE Examinations of CRDM Nozzles Automated ultrasonic examinations of all 69 CRDM nozzles were performed from beneath the RPV head using the ARAMIS inspection tool and a "Circ." blade probe. The techniques utilized for this examination are intended for the detection and through-wall (depth) sizing of circumferential inside diameter (ID) and outside diameter (OD) initiating flaws in the nozzle base metal only. Forward scatter time of flight detector (TOFD), longitudinal-wave techniques are used. The examinations were conducted from the bore of the CRDM nozzles in the J-groove weld region of the nozzle.

The examinations performed with the blade probe consisted of scanning for circumferential and significant axial flaws within the nozzle wall. The tooling consisted of a blade containing a nominal 5 MHz, 50 degree TOFD transducer set. The circ. blade probe provides flaw detection (axial and circumferential flaws) and sizing (non-axial flaws) information. For the forward scatter transducers, flaw detection is identified by loss of signal response either from the lateral wave or backwall responses as well as the presence of crack tip diffracted responses.

Prior to the examinations, demonstrations were performed using the Electric Power Research Institute (EPRI)/Materials Reliability Program (MRP) samples removed from Oconee. This demonstration showed that circumferential and axial flaws can be detected with the circ. blade probe.

During the initial examination, some nozzles had areas where the gap between the nozzle and the leadscrew mechanism was too narrow to insert the probe. All of these areas were rescanned after the initial inspection by moving the leadscrew support tube to open the gap for examination.

The examination with the blade probe identified potential flaw indications in nozzles 1, 2, 3, 5, 47 and 58. Because almost all of the flaws detected on these nozzles were characterized as axial, only limited information was available with the circ. blade probe. These axial flaws could have been characterized using axial blade probes. However, because there was a high probability that these nozzles would require repair, the CRDMs for these nozzles were removed to perform additional UT examination using the top-down tool.

The top-down tool contains 10 transducers and provides the ability to detect and characterize axial and circumferential flaws and also provides additional information required for the repair activity. Images of the nozzles identified for repair are included in the reference 8.1.1 report and show the various features required to implement the repair. These generally include the location of the RPV head OD, the elevation of the proposed cut line, and the location of the top of the J groove weld.

Root Cause Analysis Report 3.0 Data Analysis 9 5

Automated ultrasonic examinations of CRDM nozzles 1, 2, 3, 5, 47 and 58 were performed using the top-down inspection tool. The techniques utilized for the examination are intended for the detection and through-wall (depth) sizing of axial and circumferential ID and OD initiating flaws in the nozzle base metal only. Forward scatter, longitudinal-wave and backward scatter shear wave techniques were used. The examinations were conducted from the bore of the CRDM nozzles in the J-groove weld region. The potential flaw indication on nozzle 58 was determined to be a false indication using the top-down tool, potentially due to nozzle ovality.

The inspections consisted of scanning for axial and circumferential flaws within the nozzle. The tooling consisted of a transducer head that holds 10 individual search units. These search units were divided into two sets, one for the axial beam direction and one for the circumferential beam direction. The axial beam search units consisted of 5.0 MHz, longitudinal wave forward scatter time of flight search units with angles of 300 and 450; backward scatter pulse echo, 2.25 MHz 600 shear wave search units; and a 5.0 MHz 00 search unit. The circumferential beam search units consisted of 5.0 MHz, longitudinal wave forward scatter time of flight search units with angles of 450, 550, and 650; backward scatter pulse echo, 2.25 MHz 600 shear wave search units; and a 5.0 MHz 0' search unit.

The detection of flaw indications is based upon the expected responses for each search unit and technique. The 00 transducer provides weld position information and also provides positional information regarding any lack of backwall response in the region of the flaw. The forward scatter time of flight techniques provide flaw detection and sizing information. For the forward scatter transducers, flaw detection is identified by loss of signal response either from the lateral wave or backwall responses as well as detection of crack tip diffracted responses. The 600 shear wave transducer provides detection by means of corner trap responses between the flaw and nozzle surface and sizing with tip diffracted signals.

Reference 8.1.1 contains the data sheets from ultrasonic examination of the six CRDM nozzles that were identified as having flaws with the blade probe. Included in this report are the data sheets for the blade UT and the rotating UT using the top-down tool. Images of the UT data are also included to show the features identifying detected flaws.

The data was also reviewed for evidence of a leak path in the penetration bore with the blade and rotating UT techniques. Leak paths were detected in nozzles 1, 2, and 3 with blade and rotating UT. Images of the leak paths are included in the reference 8.1.1 report. Subsequent review by EPRI of all UT results indicated that although nozzle 46 had no detected cracks, some evidence of a leakage flow path was identified, due to the characteristics of the backwall reflection.

The examination results are summarized in the following table:

Nozzle #

Summary of NDE Results 1

9 Axial Flaws, 2 through-wall (TW) with a leak path 2

9 Axial Flaws, 1 Circ. Flaw, 6 TW with a leak path 3

4 Axial Flaws, 2 TW with a leak path 5

1 Axial Flaw 46 No Flaw Indication, potential leak path 47 1 Axial Flaw 58 No Recordable Indications Root Cause Analysis Report 3.0 Data Analysis

• 6 I Root Cause Analysis Report 3.0 Data Analysis 9 61

A pictorial layout of the identified flaws in nozzles 1, 2, 3, 5 and 47 is provided in Figures 7-12.

Detailed NDE results are provided in Tables 1-5.

3.1.4 Visual Examinations of RPV Top Head and Penetrations Visual examinations were made of the RPV top head surface both before and after removing a section of insulation over the nozzles of concern. The RPV head penetrations were also examined following removal of nozzles 2 and 3. Results of these examinations were as follows:

Degradation at Nozzle 3 Degradation observed at nozzle 3 is pictorially shown in Figure 13. The 1800 (uphill toward nozzle 1) location is essentially intact, with little to no degradation. The 00 (downhill toward nozzle 11) location exhibits the worst degradation, with the low-alloy steel material corroded away, down to the stainless steel cladding, for approximately 6.6 inches in length and 4 to 5 inches at the widest part. Figure 14 shows cladding thickness measurements made by UT from the underside of the RPV head. (Note: There are several low readings outside the designated area of damage. These are attributed to inclusions or bad readings. These readings do not correspond with visual observations, and will be further verified following excavation of the damaged area.)

From the 2700 location to the 0' location (counterclockwise looking down from the top of the RPV head), there is a large undercut area. From the 00 location to the 900 location (counterclockwise looking down from the top of the RPV head), the corrosion is less.

Degradation at Nozzle 2 Degradation was observed at nozzle 2 following the initiation of repair efforts. Figure 5 shows the observed area of corrosion. The overall corroded area, based on the video examination and approximate measurements from the impression, is 3-1/2 to 4 inches in length starting from the top of the RPV head, about 3/8 inch deep (at the deepest location approximately 1-3/4 inches from the top of the RPV head), and between 1-1/4 to 2 inches at it's widest location. The depth of corrosion decreased as the annulus opening was approached. This type of corrosion profile is similar to testing that has been performed by EPRI (reference 8.5.3) regarding location of the deepest corrosion and the fact that it could have been identified on the top of the RPV head.

Degradation at Nozzle 1 The observed degradation at the nozzle I location is minimal. A small, crevice (<1/16 inch wide and about 3/4 inch circumferentially), located at the 2700 location (looking down from top of RPV head clockwise with 00 in the West direction), was identified at the surface of the RPV head. The observed degradation at nozzle I was within the boundary of the pre-established repair plan.

3.1.5 Boric Acid Sample results Boric Acid Samples were taken from the reactor head prior to cleaning as well as from nozzle 3 and from the nozzle 2 cavity. These sample results are reported in reference 8.2.14 and 8.2.15.

As with all sample collection, the purity of the sample and the accuracy of its collection are critical. The samples taken from the reactor head were collected by scooping the material with a long handled tool. Due to the difficulty of collecting the field samples, the probability of cross contamination is fairly high and could lead to false or compromised results. In addition, transport of boric acid and corrosion products in aerosol or liquid form almost certainly caused some mixing of old deposits with newer deposits. However, a selection of the sample results as RotCas A

nlssRpr3.DaaAays 7I 3.0 Data Analysis

• 71 Root Cause Analysis Report

determined by inductively coupled plasma mass spectroscopy (ICP-MS)) is reproduced in the following table from reference 8.2.14:

Location of sample:

Element N2-N1 N1-N3 N7 N2 N3 flange Boron (ppm) 163000 86300 99500-130000 61700 Iron (ppm) 100000 269000 157000 220000 359000 Lithium (ppm) 17000 10800 9900 13000 11300 Chrome (ppm)7790 54100 7100 3900 5800 Nickle (ppm) 3120 6000 1600 2100 1800 B/Li ratio 9.6 8.0 10.1 10.0 5.5 Fe/B ratio 0.61 3.12 1.58 1.69 5.82 Cr/B ratio 0.048 0.63 0.071 0.030 0.094 Ni/B ratio 0.019 0.070 0.016 0.016 0.029 Cr/Ni ratio 2.49 9.01 4.43 1.85 3.22 Cs 134/137 0.14 1.11 0.61 0.41 NA Age (date) 6/1999 8/2001 10/2000 3/2000 NA The entries titled N2 -N1 and NI-N3 signify that the sample was collected between those nozzles.

The most compelling conclusion that can be taken from the above data is that the age of the sample in the proximity of nozzle 3 was the most recent. This supports that the material originated predominantly from nozzle 3.

The nozzle 3 area sample (N1-N3) also had a considerably higher iron content than the rest of the material collected from the reactor head. Additionally, the chromium/boron ratio and the chromium to nickel ratio are relatively high. This could be evidence that this recent material had been in more intimate contact with the 309 Stainless Steel cladding material than the other samples.

309 Stainless Steel has a higher chromium content than the nickel-based alloy 600 nozzle material.

Therefore, some evidence is provided that the earlier samples did not effectively contact the liner, i.e. that the corrosion proceeded predominantly from the top of head downward. As the corrosion front reached the liner, the most recent deposits acquired more chromium.

Boron to Lithium ratios indicate that leakage occurred during startup or power operation when significant Lithium is present. However, the boron to lithium ratio is much lower than reactor coolant, perhaps indicating that volatile loss of lithium is different from that of boron.

The age of deposits from the bottom of the N3 flange may support that the material was ejected upward to this location at an earlier time, when corrosion of iron was proceeding, but the liner had not yet been reached. The underlying assumption is that the annulus was significantly narrower when the nozzle 3 flange acquired the deposits. A narrow, intact annulus would produce a focused upward jet of leakage and entrained corrosion products to deposit on the bottom of the nozzle 3 flange. By the time that the stainless steel cladding was reached, the annulus would have been consumed by the corrosion front. At that point, the upward velocity Root Cause Analysis Report 3.0 Data Analysis e 81

would be reduced, despite a potentially greater leak rate. Cesium dating of the nozzle flange deposits could either dispel or help to confirm this theory. However, confirmation of this effect is beyond the necessary scope of this report. The EPRI MRP may clarify this point through continuing corrosion modeling efforts.

References 8.2.15 and 8.2.14 (respectively) also provide X-ray diffraction analysis of the chemical compounds found in the samples of nozzle 2 corrosion products (removed after head cleaning) and of the samples boric acid on the head discussed above. Essentially, the principle compounds found were Iron Borate (Fe3BO 6), Maghemite (or "red rust" - Fe20 3), Sassolite (or Boric Acid - H3B0 3), Goethite (FeOOH), and Metaborite (HBO 2). Traces (<5wt%) of Iron Carbide and Lithium compounds were also noted at some locations. The nozzle 1 area, very near nozzle 3, was highest in Maghemite. The nozzle 3 area was mostly Iron Borate, with some Sassolite and Metaborite. This was similar to the deposits under the flange of nozzle 3, which were nearly exclusively Iron Borate. This provides additional evidence that the deposits under the nozzle 3 flange originated from the nozzle 3 cracks. Interestingly, Sassolite was found at greater than 25 weight percent only in the nozzle 2 and 7 area samples.

Inside the nozzle 2 cavity (it should be noted that these samples would have been taken after head cleaning with water), major constituents (> 25wt%) were Magnetite (Fe30 4) and Hematite (Fe 20 3). Medium constituents (I 0-25wt%) were Maghemite, Iron Borate, and Lithium Iron Oxide.

Minor and Trace constituents included Nickel and Chromium compounds. The presence of the black iron oxide inside the cavity, Magnetite, may be noteworthy. This compound tends to form in a restricted oxygen environment and was not noted in the samples on top of the head. Conversely, Maghemite (Fe20 3) was found at high levels at selected locations both on top of the head and inside the nozzle 2 cavity.

In summary, although sample integrity is not assured, the sample results are consistent with other evidence that the material on the reactor head originated primarily at nozzle 3, and flowed or extruded away from that location. This is supported both by the aging and by the elemental analysis. The material on top of the head appeared to be well oxygenated, with more restricted oxygenation in the nozzle 2 cavity.

Root Cause Analysis Report 3.0 Data Analysis e 9 1

3.2 Cracks, Leaks and Corrosion This data analysis section is a technical review of the causes of cracks and leaks throughout the industry, and the resultant corrosion of the RPV top head surface. This information provides key inputs to the probable cause determination.

3.2.1 CRDM Nozzle Cracks and Propagation to Leakage The following is a review of cracking experience in Alloy 600 RPV head CRDM nozzles, and identification of the possible causes of cracks in Davis-Besse nozzles 1, 2, 3, 5 and 47.

Primary Water Stress Corrosion Cracking of Alloy 600 and Alloy 82/182 Materials There have been numerous incidents of cracked Alloy 600 nozzles, and Alloy 82/182 welds, in domestic non-steam generator related PWR plant primary system applications since a pressurizer instrument nozzle leak at San Onofre 3 in 1986. These applications include pressurizer instrument nozzles, pressurizer heater sleeves, hot leg piping instrument nozzles, CRDM nozzles, steam generator drain nozzles, RPV outlet nozzle butt welds, and a pressurizer spray line safe end. In all cases, the leakage has been discovered before failure of the components.

In all but a few cases, cracking in nozzle applications has been attributed to primary water stress corrosion cracking (PWSCC). The mechanism of PWSCC is not completely understood, and prediction of crack initiation time has proven to be difficult, if not impossible. It is known, however, that PWSCC of Alloy 600 occurs as a result of the following three factors:

"* A susceptible material

"* A high tensile stress (including both operating and residual stress) at J-groove welds, roll expansions, and expansion transitions

"* An aggressive environment (PWR primary water at high temperature)

The few exceptions are related to weld defects (Ringhals J-groove welds) and resin intrusions (Zorita). These incidents are documented in EPRI TR-103696, PWSCC of Alloy 600 Materials in PWR Primary System Penetrations (reference 8.5.4).

The susceptibility of Alloy 600 material depends on several factors including the chemical composition, heat treatment during metal production, heat treatment during fabrication of the component, and operating parameters. Alloy 600 is known to be susceptible to PWSCC with some heats of material being more susceptible than others principally due to a poorer microstructure.

High stresses are induced into the nozzle by the J-groove weld. Since the RPV head is stiff relative to the nozzle wall, shrinkage of the J-groove weld during cooling pulls the nozzle wall radially outward causing high tensile hoop stresses as shown inFigure 15 (the deflections in Figure 15 are exaggerated to illustrate the welding induced distortion). If the nozzle is machined prior to welding, higher stresses can be induced in the cold worked machined surface.

Questions arose from readers of revision 0 of this report asking if PWSCC cracks could initiate due to welding or fabrication stresses at conditions of reduced temperature and pressure.

PWSCC has been the subject of much research and analysis in recent years as a result of the many leaks that have been attributed to PWSCC. It is generally accepted that the initiation time for a given crack can be estimated by a standard Arrhenius activation energy approach. For example, in reference 8.5.11, a temperature adjusted degradation time was developed to rank plant susceptibility as compared to a reference temperature. The equation presented (simplified for a single operating temperature) is Root Cause Analysis Report 3.0 Data Analysis 9 10

EDY60oF = AEFPY exp [-Qi/R(l/Thead- /Tref)]

Where:

EDY 60oF

= Effective Degradation Years, normalized to 600'F AEFPY

= Effective Full Power Years Qi

= Activation Energy for Crack initiation (50 kcal/mol)

R

= universal gas constant (1.103x 10-3 kcal/mol-°R)

Thead

= Full power head temperature (OR)

Tref

= Arbitrary reference temperature ( 600 'F = 1059.67°R)

Using this equation, with a EFPY = 14.7 years through February 2001, and a head temperature of 605 °F, this equation placed Davis-Besse at 17.97 EDY, or 7th in calculated susceptibility as compared to all US PWRs.

As implied above, the actual initiation time of a crack is also related to both the microstructure of the alloy, and the state of stress of the component. Typically, the stresses of importance are those that are induced by fabrication (e.g. welding), which can be large compared to operational stresses. Equations have been proposed to take these factors into account. However, the parameters of interest are rarely known in the field and the sensitivity is lower than that of temperature. For time spent at less than full temperature, for example at 200 OF, the above equation indicates that the EDY accumulates at only 5x10-12 of the rate at 600'F. Therefore, the practice tends to simply utilize an equation similar to the above, and compare the predicted EDY to the EDY at onset of failures in similar components. Thus, a comprehensive and accurate crack initiation model has yet to be developed. Nonetheless, the current understanding and models in use support the conclusion that exposure to long periods at low temperature and pressure does not measurably affect the predicted crack initiation time.

Effect of Alloy 600 Heat Treatments Chemical composition and heat treatment are interrelated in several ways. For example, one reason for annealing Alloy 600 is to solutionize the carbon in the alloy. As the material cools, the available carbon and chromium will precipitate (in the form of chromium carbides) from solution at both intragranular and intergranular locations. If the cooldown from the anneal is sufficiently slow, a greater number of carbides will precipitate at the grain boundaries (i.e.,

intergranularly), and the resistance to PWSCC will be improved.

Well decorated grain boundaries are an indication that an Alloy 600 material has received a proper heat treatment and that sufficient carbon was available in solution to combine with chromium. If adequate amounts of carbon and chromium exist, but the anneal was not at a high enough temperature or sufficient time was not allowed to solutionize the carbon, an adequate amount of carbon will not be available to precipitate intergranularly as chromium carbides, leading to minimal grain boundary decoration. For example, a temperature of 1850'F is necessary to solutionize material with a carbon content of 0.04%.

The actual annealing temperatures for the Davis-Besse CRDM nozzle Alloy 600 materials could not be located. However, the minimum range of annealing temperatures used by the Root Cause Analysis Report 3.0 Data Analysis o 11 1

manufacturer (B&W Tubular Products) at the time was 1600-1700'F. Therefore, it can be assumed that the microstructure of the heats of material utilized on the Davis-Besse RPV head is likely to be less than optimum relative to resistance to PWSCC. Additional information on the microstructure will be obtained during destructive examinations of nozzles 3 and 2, but this is not expected to affect the contents of this report (see section 7.3.3).

RPV Head Nozzle and Weld Leakage Experience The first leak from an RPV head CRDM nozzle occurred at the EdF Bugey 3 plant in France in 1991. A small amount of leakage [<1 liter/hr (0.004 gpm)] was discovered on the outside surface of the RPV head during a primary system hydrostatic test. Investigation showed the leak was from a through-wall crack in an outer row CRDM nozzle that had initiated from the inside surface. Failure analysis confirmed that the crack was PWSCC and that contributing factors included susceptible material microstructure, stress concentration at a counterbore on the nozzle inside surface, high hardness of the cold worked machined surface, and high residual tensile stresses induced in the nozzle during welding.

Subsequent to the Bugey 3 experience, PWSCC of Alloy 600 base metal and welds has been discovered in other PWR RPV heads worldwide. In 1994 a partial through-wall crack was found at DC Cook 2. Like the Bugey 3 crack, the crack at DC Cook 2 initiated on the ID surface of the nozzle at the elevation of the J-groove weld. CRDM nozzle inspections have also found shallow craze cracking on the nozzle ID near the weld in a few nozzles at Oconee 2 and Millstone 2.

As of February 2000, about 6.5% of all EdF nozzles inspected had been found to contain cracks and about 1.25% of inspected nozzles in other plants worldwide had been found to contain cracks greater than the minimum measurable depth of about 2 mm (0.08 inches). Further details regarding the extent of condition are provided in MRP-44, Part 2, PWR Materials Reliability Program - Interim Alloy 600 Safety Assessments for US PWR Plants, Part 2: Reactor Vessel Top Head Penetrations (reference 8.5.5).

In November 2000 a through-weld leak at a CRDM nozzle weld at Oconee I was attributed to PWSCC. Laboratory analysis of a boat sample removed from this weldment confirmed that the crack was PWSCC.

In February 2001, PWSCC was detected in nine nozzles at Oconee 3 that were from one material heat (M3935) that had a yield strength of 48.5 ksi. Most of these cracks were axial and initiated on the nozzle OD surface. However, some axial cracks had initiated on the ID and propagated partially through the wall. In addition, most of the cracks that were found in Oconee 3 nozzles initiated below the weld, similar to those found at Davis-Besse. However, three Oconee 3 nozzles contained OD circumferential cracks above the weld.

In April 2001, Oconee 2 performed a visual inspection of the RPV head during a refueling outage at the end-of-cycle 18 (approximately 21 EFPY). Boric acid crystals were observed at four CRDM nozzles. The inspections performed at Oconee-2 in 2001 identified OD crack-like axial indications below the weld on all four nozzles. Ultrasonic examinations showed that these indications were OD-initiated and that none of the indications were through-wall. An OD initiated circumferential indication, 0.1 inch (2.5 mm) in depth and 1.26 inch (32 mm) in length, was noted above the weld on one of the nozzles. Eddy current examinations of the ID of the nozzles revealed shallow craze-type flaw clusters in all four nozzles that were distributed around the entire ID circumference (3600 and above the weld). Based on these results, the leak path was through the interface between the nozzle and the J-groove weld.

Root Cause Analysis Report 3.0 Data Analysis

• 12

In November 2001, a visual inspection of the top surface of the Oconee 3 reactor RPV head showed evidence of primary water leakage on the RPV head surface. This inspection was performed in accordance with Duke Energy's response to NRC Bulletin 2001-01 as a "Qualified Visual" inspection. Boric acid deposits with a wet appearance were identified around four CRDM nozzles and determined to be probable leak locations. Three additional CRDM nozzles were identified as being masked by boric acid crystal deposits from an indeterminate leakage flow path and were therefore classified as possible leaking nozzles. This is the same visual inspection performed during the previous outages except that a VT-2 qualified inspector participated. UT examinations showed that five nozzles had indications that extended from below the weld to above the weld indicating a leak path in addition to various other ID and OD indications. One nozzle had a circumferential indication in the nozzle above the weld. Seven CRDM Nozzles were repaired during this outage using the automated Framatome-ANP "ID Ambient Temper Bead Repair" technique that is being employed at Davis-Besse.

At least five other PWRs have identified similar PWSCC in the last year.

In summary, since November 2000, leaks have been discovered from at least 30 CRDM nozzles at PWRs in the United States. Most of the leaks have been through axial cracks in the nozzle base material, but some have also been through axial/radial oriented cracks in the J-groove welds. Investigation of these leaks has led to the discovery of circumferential cracks above the J groove weld at some plants, including 1650 through-wall circumferential cracks in two nozzles at Oconee 3.

None of these plants reported loss of material due to general corrosion that was similar to Davis Besse nozzle 3.

RPV Head Corrosion Associated with Boric Acid There have been several earlier cases of significant corrosion of low alloy steel on the reactor head. For example, in 1970, Beznau unit I (Switzerland) developed a canopy seal weld leak that was sufficient to cause fouling of the containment air coolers. Left in service for 2 months after being noted due to the fouling, the leak succeeded in promoting corrosion of the low alloy steel head over an area 50mm wide and 40mm deep. However, the head was analyzed and returned to service without repair. Leakage for the Beznau event was estimated at 0.02 gpm. In 1986, a conoseal leak at Turkey Point was reportedly a small contributor to a total unidentified leakage rate of 0.45 gpm for an operating period of 43 days. This leakage, dripping onto the top of the head deposited an estimated 40-60 pounds of boric acid and resulted in a loss of material over a 8.5 x 1.5 inch area, approximately 0.25 inches deep. Although not due to nozzle leakage, these cases demonstrate that significant corrosion of the head can occur at high temperatures whenever deposits remain moist.

PWSCC Cracks in B&W Design PWR Plants The most directly related experience has been cracking and leakage of CRDM nozzles at the other B&W design plants: Arkansas Nuclear One Unit I (ANO 1), Crystal River 3, Oconee 1-3, and Three Mile Island Unit I (TMI 1). All of these units have experienced cracks and leaks from the nozzles near the J-groove weld elevation. The cracks have been predominately axial and have tended to initiate on the outside surface of the nozzle at the weld toe or in the weld. In some cases, there have been circumferential cracks in the nozzle wall above and below the J groove weld.

Root Cause Analysis Report 3.0 Data Analysis o 131

Laboratory examination of specimens removed from Oconee 1 and Oconee 3 confirmed that the axial cracks (Oconee 1) and circumferential cracks above the J-groove weld (Oconee 3) were PWSCC.

Figure 16 shows the locations of the leaking nozzles in the B&W design plants. It should be noted that the leaking nozzles are distributed across the RPV head. Figure 17 provides further information regarding the distribution of leaking nozzles. This figure shows the fraction of all of the nozzles at each row that have leaked and whether the cracks are purely axial or axial with circumferential cracks above the J-groove weld elevation.

Lack of Fusion in J-Groove Welds During a CRDM nozzle inspection at Ringhals Unit 2 in 1992, an indication was detected in the J-groove weld at one of the penetrations. The indication was not indicative of PWSCC; rather, the indication was attributed to a weld defect that occurred during fabrication of the CRDM nozzle to the RPV head. The B&WOG took action to address this concern by acquiring additional data from several sources. First, the data from Ringhals Units 2 and 4 and data from a cancelled Westinghouse reactor, Shearon Harris, were acquired from the Westinghouse Owners Group (WOG). Second, the B&W Owners Group (B&WOG) performed an inspection of the RPV head from Midland Unit 1, which was a cancelled nuclear station fabricated by B&W.

An addendum to the B&WOG safety evaluation was prepared to analyze these data (reference 8.2.9). This evaluation included a statistical review and analysis of the J-groove weld inspection data and a stress analysis of the CRDM J-groove weld to determine the minimum weld area that is required to meet the American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel (B&PV) Code primary shear stress limits. It was shown in this report that the maximum areas of weld lack of fusion detected for the Midland Unit 1, Shearon Harris, and Ringhals Unit 2 RPV heads are well below the ASME Code allowable limits for weld structural integrity. It was concluded that a large margin exists between the statistical bound of the total lack of weld fusion areas in the Midland Unit I RPV head and the ASME Code allowable limits. Therefore, although some areas of lack of fusion are expected to be observed, they do not give rise to a safety concern.

Comparison of Davis-Besse Nozzles to Other B&W Design Plants Table 6 is a comparison of key features of the Davis-Besse RPV head to the other B&W design plants from the standpoint of cracks and leaks. This table shows several potentially significant design and fabrication differences.

Operating Temperature The Davis-Besse operating condition RPV head temperature is reported to be 605'F relative to the 601-602'F for the other six B&W design plants. Thi§ small temperature difference has some effect on the predicted time to leakage, and this fact is reflected in the row which reports the EFPYs adjusted to a common 600'F operating RPV head temperature. The effect of the higher Davis-Besse RPV head temperature is offset by the shorter operating time, leading to a temperature adjusted time that is less than Oconee 1, 2 and 3, and ANO 1.

Counterbores at the Top and Bottom of CRDM Nozzles All of the B&W design plants except for Davis-Besse were designed with a counterbore machined into the penetration hole in the RPV head before installing the CRDM nozzles.

Elastic-plastic finite element stress analyses show little difference in welding residual and operating stresses for the two designs, especially for nozzles near the center of the RPV head Root Cause Analysis Report 3.0 Data Analysis e 14

where the counterbore is only a short distance above the top of the J-groove weld. Therefore, the lack of a counterbore in the Davis-Besse's design is not considered to be a factor in the condition.

RPV Head to Hot Leg Vent Line at CRDM Nozzle 14 Davis-Besse has a vent line that runs from nozzle 14 to the steam generator 2 upper primary hand hole. This line is unique to Davis-Besse. The purpose of the line is to vent non condensable gases from the head during a loss of coolant accident. This vent line could have a minor effect on head temperature. However, since this nozzle is displaced from the cracked nozzles, its effect on other nozzles is considered to be very small. There is no evidence of thermal fatigue on this penetration.

Susceptibility of Davis-Besse CRDM Nozzles to PWSCC The 69 CRDM nozzles in the Davis-Besse RPV head were manufactured from four different heats of material as shown in the following table (reference 8.2.1). Three of the heats (C2649-1, M3935, and M4437) of material were manufactured by B&W Tubular Products (B&W-TPD), and the fourth heat was manufactured by the International Nickel Corporation (INCO). According to an assessment performed by Framatome ANP, these CRDM nozzles have a relatively high susceptibility to PWSCC, mainly because of the residual stress distribution calculated in the vicinity of the J-groove weld and the Davis-Besse RPV head operating temperature of 605'F (reference 8.2.2).

CRDM Nozzle Heats at Davis-Besse Heat YS UTS Carbon Anneal Number No. of Nozzles (ksi)

(ksi)

(%)

Temp (F)

Nozzle Nos.

7, 12, 16, 20, C2649-1 32 44.9 92.6 0.042 1600-1700 22-25, 27-29, 38-44, 47-55, 57, 64, 65, 68, 69 M3935 5

48.5 85.6 0.028 1600-1700 1-5 8-10, 13-15, M4437 23 35.9 92.2 0.059 1600-1700 17-19, 21, 26, 30-37, 61-63, 67 NX5940 9

39.0 83.0 0.030 1600 min.

6, 11,45,46, 56, 58-60, 66 Experience to date has shown that:

"* There are more leaks (15) from nozzles fabricated from heat M3935 than any, other single heat (4 max) in B&W design plants, and

"* A larger fraction of nozzles (20.3%) from heat M3935 have developed leaks than any other single heat (13.3% max) in B&W design plants.

Nozzles 1 through 5 at Davis-Besse are from heat M3935. Nozzles 1, 2 and 3 have through wall axial cracks and leaks with nozzle 2 having notable RPV head corrosion and nozzle 3 having extensive corrosion. In addition, nozzle 5 had an axial crack requiring repair. Nozzle 47 also had an axial crack requiring repair, but the nozzle was from heat C2649-1. In Root Cause Analysis Report 3.0 Data Analysis a 15

summary, the leaks at Davis-Besse are all from the heat of material that has previously resulted in more leaks than any other heat in the industry.

Range of Interference Fits Davis-Besse is similar in design to Oconee, Crystal River-3, TMI 1 and ANO 1, which have demonstrated an ability to identify leaking CRDM nozzles through an interference fit by visual inspection for boric acid crystal deposits. During fabrication, CRDM bores were inspected for final top and bottom bore diameter and verticality. After custom grinding individual CRDM nozzle shaft to approximately 0.001" greater in diameter than the final CRDM bore diameter, the shafts were measured at both the top and the bottom of the custom ground length. CRDM nozzle shafts are longer than CRDM bores are deep. Thus, CRDM nozzle shaft diameter measurements do not directly line up with CRDM bore diameter measurements, although in the case of Davis-Besse these locations should be fairly close because of the lack of counterbores. Therefore, the resulting top and bottom dimensional fits are considered approximate. The values for the Davis-Besse RPV head are calculated to range from a gap of 0.0010" to a maximum interference fit of 0.0021 High PWSCC Susceptibility in Heat M3935 The reason for the higher susceptibility of heat M3935 has not been determined although it may be related to a lower than optimum annealing temperature or through-thickness hardness gradient created in the material by a forming operation after annealing. Additional data will be acquired during examinations of nozzle 3 (see section 7.3.3). However, this data is not needed to support the basis of this root cause document.

While heat M3935 appears to have higher PWSCC susceptibility than some other heats, several other heats of material have also experienced multiple leaks in B&W design plants. Heats of material that have not experienced leaks to date may experience cracks and leaks in the future.

The same is true for J-groove welds.

Other Possible Causes of Cracks Several other potential mechanisms for crack initiation and propagation were considered for the observed flaws in the CRDM nozzles. These are:

Fabrication and Inspection Anomalies All the CRDM nozzle Alloy 600 materials used by B&W during the Davis-Besse RPV head manufacturing were either supplied by the B&W-TPD or by INCO. The materials were ordered to ASME Boiler and Pressure Vessel (B&PV)Section II Specification SB-167 and Section I1I requirements. Any fabrication or inspection anomalies would have been identified since dye penetrate testing (PT and UT was performed).

Thermal Fatigue CRDM nozzles may be subject to thermal fatigue induced by thermal fluctuations, which result from particular operating transients. Several permutations of stratified fluid conditions have been observed to result in fatigue cracking and component failures in PWRs. However, no CRDM nozzles have experienced this type of failure, and there is no historical evidence to support thermal fatigue cracking. Given the past experience, it seems unlikely that thermal fatigue degradation would result in the formation of discrete axial cracks located at high stress locations (uphill and downhill sides) within the bore of the CRDM nozzles.

Root Cause Analysis Report 3.0 Data Analysis e 16 1

Intergranular Stress Corrosion Cracking (IGSCC)

Cracking can possibly occur due to additional factors not normally associated with PWSCC.

Contaminant species such as sulfur, chloride, or fluoride compounds could result in IGSCC.

Oxygen at levels found in boiling water reactors also can cause IGSCC. The presence of these contaminants in combination with high stress and less than ideal material microstructure could lead to IGSCC. However, there is no evidence that this occurred at Davis-Besse (see following section). In PWRs, there is insufficient oxygen to cause IGSCC.

Davis-Besse has not experienced any incidents of resin ingress, which is the most common source of sulfur (Reference 8.2.1). Also, chlorides and fluorides are controlled in the primary water. Thus IGSCC is discounted as a failure mechanism since oxygen, chlorides, fluorides or sulfur were not present in sufficient quantities in the reactor coolant system (RCS).

RCS Chemistry Control Chemical transients in the primary water were considered to determine if nozzle cracking was influenced by conditions other than those causing PWSCC. In response to Generic Letter 97 01, Framatome report BAW-2301 (reference 8.2.1) summarized abnormal chemistry time periods at each of the B&WOG plants. The primary water chemistry analysis results at each of the B&WOG plants were reviewed for excursions during power operation, hot shutdowns, and cold shutdowns. At the time of this report, no events have occurred at Davis-Besse since December 10, 1983 when a resin specification problem led to a short transient in chlorides (up to 0.26 ppm) and lithium. This event is not considered significant.

The amount of hydrogen in the primary coolant during the last three cycles was analyzed to confirm that excess oxygen Was not available to promote corrosion within the primary side.

Boric acid quality was researched as a possible issue, and potential for impurities to contribute to nozzle cracking. It was determined that the boric acid used at Davis-Besse is common to the industry and that the quality control program in place for the boric acid is appropriate. Additionally, pure boric acid with no impurities has been shown in the Boric Acid Corrosion Guidebook (reference 8.5.2 or 8.5.3) to be capable of the corrosion rates seen in this condition.

Other Failure Mechanisms Other failure mechanisms were considered briefly and discarded since either they would already be encompassed by the environmentally assisted mechanisms noted above or the review of the evidence did not support them. These include environmentally assisted fatigue, mechanically induced fatigue, and hydrogen damage.

Conclusions Regarding Source of Cracks The similarity of the flaws in the Davis-Besse CRDM nozzles 1, 2, 3, 5, and 47 to the PWSCC cracks found at the other B&W designed nuclear power plants supports the evidence for concluding that the flaws are PWSCC. The flaws are similar in length and orientation to confirmed PWSCC at other plants and there is no other credible mechanism for these types of flaws. Four of the five cracked nozzles at Davis-Besse, and all three of the leaking nozzles at Davis-Besse, are from heat M3935 that has exhibited the highest percentage of leakage of any heat of material in domestic PWR plants. Therefore, the probable cause of cracks in the Davis Besse nozzles is PWSCC.

Root Cause Analysis Report 3.0 Data Analysis

• 171

Crack Propagation to Leak PWSCC of Alloy 600 components in RCS can lead to through-wall cracking, and, thus, leakage of primary water. Based on the visual inspections of the Davis-Besse RPV head, containment air cooler cleaning frequency, interviews, etc., a reasonable time-frame for the appearance of leakage on the RPV head at Davis-Besse is approximately 1994-1996. Utilizing an average PWSCC crack growth rate of approximately 4 mm/year (reference 8.5.9) through the 16 mm (0.62 inch) thick CRDM nozzle material, the time-frame at which crack initiation occurred would correspond to approximately 1990 +/- 3 years. This is a reasonable approximation to the more detailed type of calculations performed by the B&WOG in the safety assessment (reference 8.5.5), which assumes approximately 4-6 years for a through-wall flaw to develop in the area near the J-groove weld.

3.2.2 Leakage Rate From CRDM Nozzle Cracks Nozzles with through-wall PWSCC cracks in either the nozzle wall or J-groove weld can develop leaks into the annulus between the nozzle and hole in the RPV head. The following is a discussion of Davis-Besse and industry experience regarding leak rates.

Industry Experience Prior to Davis-Besse, reported industry experience had been that PWSCC cracks at RPV head nozzle penetrations only result in a small ring of boric acid crystal deposits as shown in Figure

18. Estimates from Oconee are that the volume of deposits from these leaks is less than I in3.

Using Figure 6-3 of the Boric Acid Corrosion Guidebook, Revision I (reference 8.5.3), and an assumed average boron concentration of 750 ppm over an operating cycle, one cubic inch of boric acid corresponds to leakage of about 1 gallon of water. This corresponds to an average leak rate of about 1 x 10-6 gpm over an operating cycle (two year period).

Low leak rates have been reported for most other nozzles attached by J-groove welds including pressurizer instrument nozzles, pressurizer heater sleeves, and hot leg piping instrument nozzles.

However, there have been some cases where larger leakage has been reported. These cases include a pressurizer heater sleeve containing a failed heater (ANO 2), and several piping instrument nozzles (ANO 1 & Palo Verde 2. In summary, while most through-wall cracks at Alloy 600 nozzle attachment welds result in very small leaks, there are exceptions where greater leakage has occurred.

There are several main theories that explain why leak rates are typically low.

"* The cracks only extend a short length in the high tensile residual stress zone above the J groove weld.

"* PWSCC cracks are tight and may become plugged by small amounts of impurities in the primary coolant.

"* The leaking fluid flashes within the crack, leaving boric acid deposits that block further flow through the crack.

The most likely explanation is that low leakage results from tight PWSCC cracks that extend a short distance above the J-groove weld. The basis is that low leakage is also observed from most smaller diameter instrument nozzles that are also installed in pressure boundary parts by J-groove welds without an interference fit.

While the exact cause of the previously observed low leak rates has not been conclusively established, it has been determined that the leak rates are typically low, and qualified visual Root Cause Analysis Report 3.0 Data Analysis

• 181

inspection programs are required to identify the leaks. A "qualified visual inspection" requires 1) a clean enough RPV head to identify small rings of boric acid crystal deposits, 2) visual access to locations where nozzles penetrate the RPV head, and 3) confirmation that there will be a flow path through the annulus between the RPV head and nozzle under operating conditions. Because an interference fit was predicted, Davis-Besse could not "qualify" nozzles 1-5 for a visual exam, whether or not they were clean. However, because of the very small percentage of actual metal to-metal interference (i.e., high points only) leakage is anticipated despite the predicted interference.

Unidentified Primary System Leaka2e Rates at Davis-Besse Normal operational leakage in the RCS is recorded and analyzed in MODES 1-4, looking for adverse trends and to verify compliance with Technical Specification 4.4.6.2. This Technical Specification requires that there is no pressure boundary leakage, and that unidentified leakage is maintained at less than 1 gpm. Although the surveillance is required once per 72 hours8.333333e-4 days <br />0.02 hours <br />1.190476e-4 weeks <br />2.7396e-5 months <br />, RCS leakage is generally trended once per day. DB-SP-03357, RCS Water Inventory Balance, provides the methodology to determine the RCS leakage rate. The test calculates total RCS leakage by resolving changes in initial and final values of Pressurizer and RCS Makeup Tank levels over a 1 to 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> period, and providing corrections based on RCS temperatures and pressures. Identified sources of leakage that are apparent through changes in Pressurizer Quench Tank level, normal (measured) Reactor Coolant Pump seal leakage, or quantified primary to secondary tube leakage, are subtracted from the calculated total RCS leakage to obtain the unidentified leakage value. This method of determining unidentified leakage has a significant daily variation (in the range of 0.05 to 0.1 gpm) that depends on accuracy of identified leakage measurements, stability of the plant during data collection, and duration of data collection.

Therefore, monthly or running averages are most useful to determine leakage trends, similar to that presented in the figure below.

Monthly Average Unidentified Leakage, Cycles 10 through 13 (gpm) 0.9 0.8

0.7

0.

0)}

0.6 0')

-I "0

.I 0.5 0.4 0.3 0.2 0.1 U)~~~~~

M)

M)(

0

(

.N 0

(

0

(

0 0

0 0

0 (0

(

0 0

(Z(

0

(

0 M

(

(0 0

(0 0 M (0

(0 0

>0 0W 0-0 z

z N

z z Mo z

Year z

z Month I Year Root Cause Analysis Report 3.0 Data Analysis e 191

During operating cycles 10 through 13 (November 1994 through February 2002), measured unidentified leakage has ranged from slightly negative to as much as approximately 0.8 gpm in April 1999. From review of data over that time span, it would appear that average unidentified leakage tends to be in the range of 0.0 to 0.03 gpm when the plant is "tight" with no pending maintenance concerns. If unidentified leakage begins to trend upward, efforts are expended to determine the cause, and if necessary, effort is made to repair the source of the leakage. For example, the high leakage in April 1999 was identified as from the Pressurizer Code Safety Valves and the plant was shutdown to service the valves. Following that shutdown, unidentified leakage remained in the range of 0.15 to 0.25 gpm, some of which was attributable to CRDM flange leakage and some of which may have been attributed to CRDM nozzle leakage.

When a CRDM nozzle begins to leak, experience has shown that one of the first signs is the appearance of a small boric acid deposit crust at the base of the CRDM flanges where it emerges from the RPV head. To put nozzle leakage in perspective, a leak rate of 0.00001 gpm could deposit 10 cubic inches of boric acid over the course of a fuel cycle, which would be quite visible on a clean RPV head. Compared with unidentified leakage of a "tight" RCS at 0.03 gpm, it is apparent that unidentified leakage measurements cannot be used to detect early through-wall leakage at a CRDM nozzle. Figure 19 shows the unidentified leakage rate over cycle 13. It is possible that the approximately 0.10-0.15 gpm increase in unidentified leakage starting in October 2001 is related to changing conditions at the crack in nozzle 3. Subtracting a base leakage rate of 0.05 gpm from the total unidentified leak rate in Figure 19, the maximum leakage rate from the CRDM nozzles did not exceed 0.15- 0.20 gpm at any point in time.

In summary, RCS leakage is at best a late-term indicator of leaking CRDM nozzles. By the time RCS leakage can be used directly, there is a potential for advanced corrosion of the low-alloy steel RPV head.

Predicted Leak Rates from PWSCC Cracks Dominion Engineering, Inc. Calculation No. C-5509-00-6 (reference 8.2.4) provides predicted leak rates from nozzles with PWSCC cracks. Key results are plotted in Figure 21. The basis for these leak rates are as follows:

Length of Cracks in Davis-Besse Nozzles 2 and 3 The longest crack lengths above the top of the J-groove weld determined by UT measurements are 1.1" for nozzle 2 and 1.2" for nozzle 3. The longest cracks above the J groove weld previously discovered in other plants with low observed leakage are <1.0 inch.

Since the Davis-Besse cracks are longer than in other B&W design plants, higher leak rates would be expected.

Leakage From an Axial Crack in a Pipe Cracks above the J-groove weld can be modeled as through-wall axial cracks in a straight length of pipe subjected to internal pressure. The model used for the results plotted in Figure 21 is based on crack opening areas from the EPRI Ductile Fracture Handbook (reference 8.5.6). Leak rates are computed from the crack opening area using models developed by EPRI [Steam Generator Integrity Assessment Guidelines: Revision I (reference 8.5.7), and PWR Steam Generator Tube Repair Limits: Technical Support Document for Expansion Zone PWSCC in Roll Transitions (reference 8.5.8)].

The predicted leak rates are 0.025 gpm for the 1.2" crack at Davis-Besse nozzle 3, 0.018 gpm for the 1.1" crack at nozzle 2, and about 0.012 gpm for the longest previously encountered crack. These results show higher predicted leak rates for the longer Davis-Besse cracks and Root Cause Analysis Report 3.0 Data Analysis 9 20

that the total predicted leak rate is significantly less than the 0.1-0.2 gpm inferred from the measured unidentified leak rate.

Effect of J-Groove Weld on Crack Opening Area The finite element model in Figure 15 shows the hoop stresses in the nozzle including the effects of welding residual stresses and operating pressure and temperature. The tensile hoop stress shown in the area of the J-groove weld will tend to open a crack at this location.

The finite element model shown in Figure 15 was modified by releasing the nodes on the plane of symmetry in the area of the axial crack as shown in Figure 22 and described in Dominion Engineering, Inc. Calculation No. C-5509-00-7 (reference 8.2.5). The resultant crack opening displacements shown in Figure 23 result in a predicted leak rate of over I gpm for the 1.2" long crack at nozzle 3.

An additional load step was added to the model to simulate the loss of low-alloy steel from behind the J-groove weld. Removal of the constraint provided by this material resulted in the crack closing up somewhat and a predicted leak rate of about 0.8 gpm for the 1.2" long crack in nozzle 3.

The above analyses illustrate the potential for leakage rates ranging from 0.025 gpm for the case of a 1.2" long axial crack in a straight run of nozzle material remote from the weld to 1 gpm or more for the case where weld shrinkage forces act on a long crack that extends 1.2" above the top of the J-groove weld.

Estimated Leak Rate Based on Boric Acid Deposits on RPV Head at 13RFO An alternate means to estimate leak rates for this condition would be from boric acid accumulations. Because of uncertainties in how much boric acid left the head region, this method is only useful as a comparison. Figure 20 shows boric acid deposits on the RPV head prior to cleaning at 13RFO. The volume and weight of these deposits are estimated to be 11.5 ft3 and 900 lb at an assumed density of 1.25 g/cc (reference 8.2.13), which is about midway between the density of boric acid crystals and powdered boric acid. Assuming that the average boron concentration in the primary coolant is 750 ppm, Figure 6-3 of Boric Acid Corrosion Guidebook, Revision 1 (reference 8.5.3) shows that 900 lb of boric acid deposits are the result of about 20,000 gallons of water. Assuming a linearly increasing leak rate over a two-year period of time, the maximum leak rate at the end of three years would be about 0.04 gpm. However, a substantial (but unquantified) amount of boric acid appears to have been drawn out of the CRDM service structure by the CRDM ventilation system and deposited in containment (CTMT). Thus, the leakage estimate may be low. More specifically, 10 ft3 of wet boric acid were removed from the containment air cooler plenum during 13RFO. While this is almost as much material as was found on the RPV head, it is important to realize that the total unidentified primary leakage during cycle 13 was more than 0.1 gpm. It is not likely that all of this leakage was from nozzle cracks on the RPV head. Therefore, the estimate, 0.04 gpm, based on 900 lbs. of boric acid is a minimum predicted leakage rate from the nozzle cracks.

Conclusions Regarding Leak Rate From PWSCC Cracks The unidentified leakage during late 2001 attributed to CRDM nozzle leaks (0.1-0.2 gpm) is bracketed by the predictions based on leakage from an axial crack in a pipe (0.025 gpm) and the finite element analysis of crack opening area at the J-groove weld elevation after corrosion of the low-alloy steel material (0.8 gpm). Further refinement of the predicted leak rate is not possible due to the significant uncertainty regarding the exact shape of the crack in the nozzle wall and in Root Cause Analysis Report 3.0 Data Analysis e 21

the J-groove weld. However, the analyses clearly demonstrate the potential for significant increases in flow rate as the crack grows in length.

3.2.3 Source of Boric Acid Deposits on RPV Head As shown in Figure 24 there were extensive boric acid deposits on the RPV top head surface at the start of 13RFO. These deposits are considered to have come from two main sources, leakage from CRDM nozzle flange joints (uncleaned from previous cycles) and leakage from PWSCC cracks at nozzles 2 and 3.

Leakage From CRDM Nozzle Flange Gaskets Figure 3 shows a typical CRDM flanged joint in a B&W-design plant. The joint consists of an Alloy 600 nozzle welded to the underside of the RPV head by a J-groove weld, a stainless steel flange welded to the Alloy 600 nozzle, a flange on the CRDM, two spiral wound gaskets, two 180' split nut ring segments below the flange and eight bolts.

Leakage from the CRDM flange gaskets was experienced early in life at B&W designed plants.

Leakage from the flanged joints sometimes resulted in formation of concentrated boric acid on the flange with resultant corrosion of the originally installed low-alloy steel nut ring segments.

One such condition at ANO I in 1989 is described in the Boric Acid Corrosion Guidebook (reference 8.5.2). During the 1980's and 1990's, the gaskets were changed to graphite/stainless steel (SST) spiral wound gaskets and the split nut ring was changed to a corrosion resistant SST material.

Leakage from Davis-Besse CRDM Nozzle Flange Gaskets Prior to 13RFO It is reported that graphite/SST gaskets and corrosion resistant nut rings were installed at Davis Besse over several outages.

"* 6RFO Replaced 23 gaskets

"* 7RFO Replaced 15 gaskets

"* 8RFO Replaced 14 gaskets

"* 9RFO Replaced 8 gaskets

"* 10RFO Replaced 9 gaskets It has been reported by Framatome that Davis-Besse is the only plant to have experienced leaks with the new gaskets and bolting materials. Specifically, 0 8RFO Replaced gasket on nozzle 66 (a minor leaker) 11 RFO Small leak detected at nozzle 31 (was not repaired) 12RFO Nozzle 31 identified as leaker and repaired. Nozzles 3, 6, 11, and 51 identified as possible leakers and gaskets replaced 0

13RFO No flange leaks identified The largest of these leaks was from nozzle 31 at 12RFO. It is reported that steam cutting had occurred and that flange repairs were required in addition to just replacing the gasket.

Source of Boric Acid Deposits on Davis-Besse RPV Head It is considered that most of the boric acid deposits found on the Davis-Besse RPV head at 13RFO have come from leaking nozzle 3 with potential contributions from nozzle 2. The basis is that the vessel head was reported to be clean at 9RFO, significant boric acid deposits had appeared on the vessel head by 1 IRFO, there were no significant gasket leaks prior to I IRFO, experience in the industry does not suggest that leakage from the nozzle 31 flange gasket would Root Cause Analysis Report 3.0 Data Analysis

• 22 1

have resulted in extensive deposits on the vessel head at 12RFO, and additional deposits appeared during cycle 13 when there were no reported flange leaks.

The source of the deposits is further supported by the reactor head boric acid sample results reported in reference 8.2.14 and described previously in section 3.1.5 of this report.

Volume of Boric Acid Deposits on Davis-Besse RPV head at 13RFO The volume of boric acid deposits on the RPV head at 13RFO is estimated in a Dominion Engineering, Inc. calculation (reference 8.2.13). The approach used was to divide the RPV head into sixteen areas, estimate the depth of deposits in each area by reviewing inspection videotapes, and then calculate the weight of deposits in each area using the area, depth of coverage for each sector, plus an assumed density midway between that of solid boric acid and loose boric acid crystals. The worksheet calculations show an estimated volume of 11.5 ft3 and a weight of 900 pounds.

In summary, while the case is not conclusive, it is probable that the approximately 900 pounds of boric acid deposits that accumulated on the RPV head are the result of leakage from the PWSCC crack at nozzles 2 and 3.

3.2.4 Corrosion of RPV Top Head Surface As shown in Figure 6, the RPV top head surface was corroded. During this investigation, attention was focused on boric acid corrosion as the source of the large volume of material loss downhill from nozzle 3. The potential for boric acid corrosion of low-alloy steel RPV heads has been known since the mid-1980's and there is no other plausible explanation for loss of this much material.

Historical Perspective on Boric Acid Corrosion of PWR Primary System Components The potential for boric acid corrosion of PWR primary loop components has been recognized since the plants were designed. Several incidents between the late 1970's and the mid 1980's led to the NRC issuing Generic Letter 88-05, Boric Acid Corrosion of Carbon Steel Reactor Pressurizer Boundary Components in PWR Plants (reference 8.3.1). EPRI issued the original Boric Acid Corrosion Guidebook in 1995 (reference 8.5.2), and the guidebook was revised in 2001 (reference 8.5.3). The later document includes summary descriptions of more than 100 incidents including corrosion of RPV heads, high pressure injection nozzles, reactor coolant pump studs, etc.

Previous Boric Acid Corrosion of RPV Top Head Surfaces Prior to the current condition at Davis-Besse, the greatest reported quantity of boric acid deposits on a RPV head was over 500 pounds at Turkey Point 4. These deposits were kept wet from a leak rate of less than 0.45 gpm (from a Conoseal leak). Corrosion on the RPV head was relatively minor, (approximately 0.25 inches depth).

There was corrosion of the low-alloy steel bottom head of a Combustion Engineering pressurizer at ANO 2 in 1987. In this case, a leak of about 0.002 gpm over less than six months time resulted in a corroded area about 1.5 inches in diameter and 0.75 inches deep. This leak resulted from a crack in an Alloy 600 sleeve associated with swelling of a failed Alloy 600 heater inside the sleeve.

Root Cause Analysis Report 3.0 Data Analysis - 23 1

Estimated Corrosion Rates at Davis-Besse Nozzle 3 The volume of material lost at the cavity between nozzles 3 and 11 was estimated to be about 125 in3 giving a weight loss of approximately 35 pounds.

Review of the sequence of relevant events in Attachment 2 suggests that the corrosion rate began to increase significantly starting at about 1 IRFO and acted for a four year period of time. With the maximum corrosion length of about 8 inches between nozzles 3 and 11, the average corrosion rate would be about 2.0 inches/year. As a bounding assumption, if the rate increased linearly with time, the maximum corrosion rate near the end of Cycle 13 would be about 4.0 inches/year.

The rates growing laterally from the main axis of the cavity would be about half of the rates growing axially, or 1.0 to 2.0 inches/year.

Figure 25 from the Boric Acid Corrosion Guidebook, Revision 1 (reference 8.5.3) summarizes the available test data regarding boric acid corrosion. These data show that most of the data points for borated water dripping onto hot metal surfaces, impinging onto hot metal surfaces, or leaking into a heated annulus, are in the range of 1.0 to 5.0 inches/year. This is consistent with the observed conditions.

Further effort is ongoing to better define the corrosion rates based on the final measured size of the cavity and thermal-hydraulic modeling being performed by the MRP. Technical insights gained from that effort may provide improved understanding, but are not expected to conflict with the evidential basis for the projections made here.

Progression from Initial Small Leak to High Corrosion Rates An important issue is why some of the leaking CRDM nozzles (especially nozzle 3) at Davis Besse progressed to a high leak rate and corrosion while leaks at the six other B&W design plants have remained small. Several possibilities were explored.

Crack Grows Longer With Time One possibility is that the axial PWSCC crack simply grows longer with time and this increases the leakage rate. Prior to Davis-Besse, the greatest crack extension above the J groove weld was just under 1 inch. The longest cracks at Davis-Besse extend 1.1" above the top of the J-groove weld at nozzle 2 and 1.2" above the top of the J-groove weld at nozzle 3.

Corrosion Begins Deep in Annulus and Increases With Time It is likely that corrosion initiates deep within the crevice and progresses to the surface as indicated by the corrosion at nozzle 2. This would be consistent with a test conducted by Southwest Research Institute for EPRI and described in the Boric Acid Corrosion Guidebook, Revision 1 (reference 8.5.3). However, for there to be significant boric acid corrosion below the surface, there would have to be evidence of boric acid crystal deposits at the annulus outlet. Since other plants have not reported significant boric acid crystal deposits around the annulus this model does not explain the difference.

Boric Acid on Top of RPV Head Acts as Incubator or Insulator Laboratory test experience with bolted flanges has demonstrated that corrosion rates can increase for conditions where leaking borated water is retained in a bolted flanged joint by insulation or a loose fitting band. Boric acid deposits on the RPV head from other sources, such as leakage from CRDM flange joints, could possibly provide the same type of "incubator" as insulation on flanged joint. However, this is only expected to be a short term "head start" since leakage of borated water from a PWSCC crack will eventually create its Root Cause Analysis Report 3.0 Data Analysis

• 24 I Root Cause Analysis Report 3.0 Data Analysis e 241

own boric acid deposits at the annulus which would behave the same as boric acid from flanged joints.

Morphology of the Affected Area as Damage Progresses Based on the investigations of the root cause team, it is clear that leakage from PWSCC cracks was a necessary precursor to the material loss adjacent to nozzles 2 and 3. These leaks led to local environmental conditions that produced modest material loss around nozzle 2 and much more extensive degradation around nozzle 3. The main effect of the leakage was to provide a boric acid solution that concentrated through boiling heat transfer along the leak path. Provided that sufficient levels of oxygen are available-either directly or remotely through a crevice corrosion mechanism-the concentrated liquid boric acid solution may cause relatively high corrosion rates up to on the order of four inches per year. A possible secondary effect of the leakage is to enhance the material loss of the low alloy steel through flow-related mechanisms.

These mechanisms are flow accelerated corrosion (FAC), droplet and particle impingement erosion, and potentially steam cutting.

Given the current limited experimental data applicable to the observed degradation and the lack of existing detailed analytical calculations of the thermal-hydraulic and thermochemical environment along the nozzle leak path, it is not possible to definitely state the exact progression of mechanisms that led to the observed material loss. The environment along the leak path from the primary system pressure inside the CRDM nozzle, through the axial PWSCC crack extending above the top of the J-groove weld, up through the annulus or cavity on the periphery of the nozzle, and then out to the ambient pressure above the top head surface-is the result of complex processes such as critical two-phase flow, two-phase frictional and acceleration pressure drops, boiling heat transfer, boiling point elevation due to boric acid solution concentration, oxygen and hydrogen transport, various electrochemical processes, convective heat transfer on the surfaces of the head, and conduction heat transfer within the head materials. Therefore, a detailed description of the damage progression including the precise physical mechanisms with a quantitative breakdown of the relative importance of each mechanism would be speculative.

However, the degradation modes on the two extremes of the overall progression are known with reasonable confidence, and some conclusions can be made regarding the possible modes of degradation in between these two extremes. The first extreme is associated with the lack of material loss and extremely small leak rates observed for most of the leaking CRDM nozzles in the industry. For these extremely low leakage rates (on the order of 10-6 to 10-5 gpm) the leaking flow completely vaporizes to steam immediately downstream of the principal flashing location, most likely at the exit of the PWSCC crack. The result is to keep the gap between the nozzle and head dry, precluding high rates of low alloy steel material loss. In addition, the small velocities associated with the extremely small leakage preclude the flow mechanisms from being active.

The other extreme of the degradation progression is associated with the large cavity located adjacent to nozzle 3. For this cavity, it is clear that the degradation proceeded by the classic boric acid corrosion mechanism associated with liquid boric acid solution concentrated through boiling and oxygen directly available for corrosion from the ambient atmosphere. The magnitude of the boiling heat transfer associated with the relatively high leak rate of nozzle 3 is sufficient to cool the head enough to allow liquid solution to cover the walls of the cavity.

In between these two extremes, increase in the extent of the axial PWSCC crack above the top of the J-groove weld resulted in increasing rates of leakage for nozzle 3. It is likely that the degradation proceeded at relatively low rates by an erosion mechanism, enhanced to some degree Root Cause Analysis Report 3.0 Data Analysis

• 25

by galvanic corrosion associated with the dissimilar metal couple of the Alloy 600 nozzle and low alloy steel head material.

While it is not possible within current knowledge to definitively identify a progression of corrosion mechanisms, the overall effect and cumulative timeframe is apparent. Linking the corrosion mechanisms that were described above (with some supplemental understanding), it is possible to construct a "viable" progression of events.

Stage 1 - Crack initiation and progression to through wall First, based on the body of knowledge available, a crack initiated in nozzle 3 at around 1990

(+/-3years) due to PWSCC. The crack grew at a rate consistent with industry data, progressing to a through-wall crack that penetrated above the J-groove weld in the 1994 to 1996 time frame. At this point the RCS leakage would have been miniscule, and in no way detectable by any currently installed leakage monitoring system.

Stage 2 - Minor Weepage / Latency Period Leakage would have entered the annular region between the Alloy 600 nozzle and the low alloy steel base material of the RPV head. However, the interference fit that was initially expected in nozzle 3 is composed of only about 5 percent of actual metal-to-metal high point contact. At its tightest, the rest of the interface is essentially an annular "capillary" flow path.

Even if the flow path could not actively leak, it would still be permeated with moisture from the newly developed crack. With addition of moist boric acid in this bi-metallic annulus, several forms of boric acid corrosion are possible, in addition to galvanic attack. These corrosion mechanisms would open an annular gap, if it did not previously exist, and allow leakage flow to the surface. If the RPV head had been initially clean, and if a timely 100%

bare head visual inspection had been completed, the leakage would most probably have expressed itself within a short time as the classical "popcorn" crust of boric acid deposits.

This would have been apparent within one or two fuel cycles from the time the crack progressed through the nozzle wall and would not have been accompanied by large scale corrosion of the low-alloy steel. However, at Davis-Besse, the "popcorn" manifestation was not yet observed, and its detection could have been obscured by previous flange leakage deposits.

Given time, the crack continued to grow, leakage increased, and the annular gap increased in width. With an ever widening gap, oxygen may begin to enter the annulus, thus accelerating boric acid corrosion in the gap and diminishing the relative importance of galvanic corrosion.

However, due to restriction of oxygen and moisture, corrosion mechanisms have not been fully accelerated. As observed at other facilities (and also in Davis-Besse nozzle 1, the least advanced of the leaking nozzles), there was widening of the annular gap and development of flow "channels" in the annulus leads to the near certainty that the principle flow resistance would have been due to the dimensions of the crack, and not due to any restriction offered by the annulus. This is also supported by the relatively low crack growth rate (i.e., &0.2 inches/year with microscopic opening width) compared to documented boric acid corrosion rates on the order of 0.02 to 0.08 inches per year for similar geometry as cited in the Boric Acid Corrosion Guidebook (reference 8.5.3). Since the growth in annulus width tends to occur over a broad length around the annulus, the annulus flow area increases faster than the crack flow area. Thus, the crack dimensions dominate the flow resistance, and the majority of the pressure drop occurs as effluent traverses the crack. Based on the reactor coolant enthalpy at the RPV head, approximately 45% of the reactor coolant that escapes from the crack flashes upon discharge, the rest is immediately vaporized by heat transfer from the Root Cause Analysis Report 3.0 Data Analysis 9 26

metal surfaces at this stage. Thus, boric acid is both atomized with the steam and deposited as molten boric acid on the surrounding surfaces, with moisture escaping as steam.

Stage 3 - Deep Annulus Corrosive Attack Toward the end of the latency period, the crack leakage increases. While the annulus is tight, single phase erosion is possible. However, oxygen penetration may be ever more pervasive if the flow area in the annulus hasoffset increases in leakage due to crack growth. This would cause annulus velocity and differential pressure to decrease, allowing greater inward penetration of oxygen. With a decreased annulus velocity, single phase erosion would decrease, and forms of flow accelerated corrosion, droplet impingement, and flashing induced corrosion could become dominant. It is probable that a small amount of material would be preferentially corroded in the vicinity of the crack, as evidenced by test EPRI-6, modeling of leakage into annular gaps original Boric Acid Corrosion Guidebook (reference 8.5.2). This test was characterized as having excessive oxygen in the supply water, which would be similar in effect to having oxygen supplied by alternative means (i.e., from the top downward). The net effect is that the corrosion rate can be substantially greater in areas of greater velocity. The velocity increase does not need to be sufficient to cause scrubbing of beneficial oxide layers (i.e., erosion-corrosion), rather, it simply needs to maintain a fresh supply of new reactive oxidizing ions in the boundary layer near the corroding metallic surface. The expected pattern was found at nozzle 2, and would be accompanied by copious amounts of boric acid deposits.

The appearance of the corroded low alloy steel in the Davis Besse nozzle #2 bore bears a striking resemblance to that of the post test EPRI-6 test specimens. As pointed out in Reference 8.6.2, the maximum corrosion rates attained in this test program were as much as 2.5 inches/year. However, these tests were not conducted for a long enough duration to determine if the high rates would be sustained after the annulus dimensions substantially increased.

Stage 4 - General Boric Acid Corrosion Progression to this stage is dependent on crack leakage rate. With high leakage rates, the annulus is flooded with an ever increasing amount of moist steam, partially flashing as it exits. Due to the fact that the annulus still exists, basically in the same geometry, any effluent is directed vertically upward. A large amount of discharged boric acid would have already accumulated in the area around the nozzle. With increased leakage, heat transfer from the surrounding metal is no longer sufficient to immediately vaporize the portion of leakage that does not flash (due to its own initial enthalpy and pressure reduction) as it exits the crack.

Recent Finite Element heat transfer calculations completed by Dominion Engineering (reported in reference 8.6.2) indicate that a substantial cooling effect is in progress by the time a leakage rate of approximately 0.05 gpm exists. By the time leakage has increased to approximately 0.1 gpm, the metal surface temperature on top of the head will be'suppressed to the boiling point due to the large heat flux required to vaporize the leaking coolant. Thus, the principle characteristic of this stage is that the annulus begins to overflow or expel unflashed liquid, causing an area to be wetted underneath the accumulations of boric acid.

General boric acid corrosion on the wet, oxygenated surface of the low-alloy steel RPV head progresses rapidly. Even reductions in reactor coolant system boric acid concentration toward the end of the operating cycle may not diminish the corrosion rate because the concentration at the metal surface is continuously re-supplied by the boric acid that was previously stored. The wetted surface area is dictated by the leakage rate as determined by Root Cause Analysis Report 3.0 Data Analysis

• 27

crack size and system pressure, the ability of the RPV head to vaporize the liquid via conduction of heat from the interior of the RPV (i.e., it would be vaporized as it runs), and is also affected by the character of surrounding deposits. Calculations at full temperature and pressure indicate that the affected area would be consistent with the amount of leakage that appears to have occurred. Further, since the wetted area would be the result of liquid overflow, it would be expected to be predominantly downhill from the nozzle, leaving the uphill side much less affected, and affecting an oblong area. This is the pattern observed at nozzle 3.

As general corrosion progresses, it would tend to carve out a "bowl" of corroded (or, oxidized) material. Initially, this bowl would gradually increase in surface area as the leak rate from the crack increases. The area in the middle, having been wetted longer, would be slightly deeper. With a sufficient flow rate, the bowl could begin to fill with a saturated boric acid solution. The saturation temperature and consistency in the bowl could be anywhere between that of dilute boric acid (-watery at 212F), to that of moist, molten orthoboric acid (H3BO 3) (viscous at <365F). As the bowl deepens, thermal effects would limit the widening of the bowl, even as leakage incrementally increases. As the bowl deepens, there would be a lesser need for as much projected surface area to transfer the heat. This is because the thermal resistance to heat transfer would continually decrease as the corrosion depth approaches the stainless steel cladding. Further, as the bowl attains a liquid level, lateral heat transfer from the sides would increase the steaming rate, and tend to govern level. A third self-governing effect would be that if leakage increased, decreasing the boric acid concentration in the bowl, the boiling temperature would decrease. This would increase heat transfer (and vaporization rate) by increasing the temperature difference. Heat transfer would also increase due to decreases in viscosity. Thus, with a relatively constant level, the corrosion surface slope might well be expected to be very steep. Finally, when the liquid at the corrosion front reaches the depth of the stainless steel, downward progression ends. At this point the wetted surface would stop its vertical travel and begin to cause undercutting.

The height of liquid boric acid would tend to increase with further increases in leakage, unless the increases in diameter due to outward corrosion were sufficient to offset the increases in leakage. This represents the as-found condition of nozzle 3, with steep walls and an undercut nose on the downhill side.

Throughout the majority of this process, being predominantly top-down, the annulus could remain somewhat intact until the approaching general corrosion front overcomes it. This is because the annular region would remain somewhat protected by the upward flow of de oxygenated water and steam. Thus, flashing effluent from the crack would be directed upward and out of the annulus while the annulus is in place. However, as soon as the low alloy steel corrosion front is below the elevation of the crack, the effluent would be directed laterally. This would undoubtedly change the degree of atomization of boric acid and affect the particle size of the boric acid carryover late in the process. If this sequence is accurate, the point at which the corrosion depth reached the crack location might have been around May 2001. At that time, the cleaning frequency of containment air coolers (CACs) due to boric acid fouling decreased.

In addition to this construction, the chromium and nickel content of the sample deposits is consistent with a top-down progression of corrosion (see section 3.1.5).

Root Cause Analysis Report 3.0 Data Analysis

• 28 Root Cause Analysis Report 3.0 Data Analysis 9 28

Boric Acid Formations on the RPV Head The following is a general description of phase changes that boric acid is known to undergo as temperature is increased. This information is being used as part of the MRP modeling effort to develop a consistent model, including boric acid morphology as the corrosion progresses.

When boric acid is left behind by boiling water, it is first deposited as orthoboric acid (H 3BO3).

Although solubility of this material is limited at cooler temperatures, near the melting point of 365°F, the solubility in water is infinite. Thus, as boric acid is deposited on the RPV head, it would tend to increase in temperature from that of saturated water (212'F) to 365°F, at which point it is a viscous liquid. In this form it will tend to flow, causing the formations that have been noted. However, even before all the free water is driven out, at around 340'F the H3BO3 begins to dehydrate to metaboric acid (HBO 2). This is a white, cubic crystalline solid, and is only slightly soluble in cool water. Metaboric acid has a melting point of 457°F, and may tend to form a "crust" on the deposits and formations of orthoboric acid. With further application of heat, the HBO 2 will further dehydrate at approximately 572°F to tetraboric acid (H 4B 4 0 7 ).

Tetraboric acid is a vitreous solid or white powder, and is water soluble. At the temperatures encountered on the RPV head, all of the above forms can be found, depending on age, contact with the RPV head, and local temperature.

When boric acid accumulates at a leaking nozzle, some flowing of the orthoboric acid would be expected. Boric acid in the cavity formed at nozzle 3 is most likely highly hydrated H 3BO 3,

since moisture is continually supplied. As it was expelled or extruded from the cavity, it would flow, and undergo the above transformations. These transformations would drive off some moisture that could conceivably contribute to corrosion, but this is expected to be a trivial effect.

However, experimental data to predict the extent of motion or the degree of corrosion has not been located.

When nozzle 3 was removed, it was reported from the field that the column of boric acid surrounding the nozzle was porous, with winding tube-like channels (at the time still believed to be carbon steel due to the rust color). A small cavity was below the material, where liquid boric acid of lesser concentration presumably drained or washed out during machining for the original aborted repair attempt. The boric acid remaining would have solidified during cooldown, but would be expected to be full of voids and steam tubes to allow venting of the flashing leakage.

The appearance of other formations is consistent with expectations of the transformations and crusty appearance.

3.3 Investigation of Lead Indicators This data analysis section provides a discussion of plant operational and equipment issues that provide possible lead indicators for the subject condition.

3.3.1 Timeline An early step in the root cause evaluation was to establish a timeline of key events. The timeline was revised as the data analysis proceeded and the current evolution is shown in Figure 26.

While the timeline was created based on the information that follows, it is presented first so that it can serve as a useful guide to help focus subsequent discussions.

The timeline summarizes the following information:

"* Years from 1995 to present

"* Operation from Cycle 10 to Cycle 13 Root Cause Analysis Report 3.0 Data Analysis

• 29 1

"* Refueling outages lORFO through 13RFO including mid-cycle outage during Cycle 12

"* Condition of the CRDM flanges, RPV head flange and RPV top head surface at each outage

"* RCS unidentified leakage (discussed previously)

"* Containment air cooler cleaning operations

"* Containment radiation monitor performance and filter cleaning

"* The estimated weight of boric acid deposits on RPV head

"* Dates of key industry findings and initiatives relative to RPV head condition 3.3.2 Sequence of Relevant Events is a table of events relevant to the subject condition. This table was used as input to creating the timeline, the logic chart of key decision points and the other potential lead indicators.

Figure 27 is an events and causal factors chart outlining key decision points and other potential lead indicators.

3.3.3 CRDM Flange and RPV Head Inspections during Refueling Outages In the early 1990's, several B&W design plants began cutting openings in the service structure surrounding the RPV head to afford better access to the center top of the RPV head for inspection and cleaning. Framatome ANP (Framatome Technologies, Inc. at the time) provided proposals to Davis-Besse over a period of several years to perform this work. However, Davis-Besse has not installed these openings. Without these openings, the head visual inspection through the mouse holes is hampered in that the pole-mounted camera can only be inserted a finite distance. The curvature of the RPV head impedes seeing the top of the RPV head with this inspection arrangement. Based on review of video by the root cause team in the presence of the inspector during 1 IRFO, the optical illusion created by the short focal length of the camera, the curvature of the RPV head and the close proximity of the insulation (nominally 2") at the top of the RPV head appears to have potentially led inspectors to believe that the top of the RPV head had been inspected; however, the inspection may have been approximately 1-2 nozzles away from the center of the RPV head.

Framatome provides a tool to inspect CRDM flanges for leakage with two cameras that is lowered down between adjacent flanges. The lower camera is angled up to look under the flanges for boric acid deposits. The upper camera is a straight ahead view of the flange interface.

The housing for the cameras is designed to rest on top of the insulation. At this height, the lower and upper cameras are properly positioned relative to the flange.

Prior to 1996 During this time frame, B&W had recommended replacing the original CRDM flange gasket with an improved graphite/SST spiral wound gasket to fix leakage problems that all the B&W design plants had experienced. The plant replaced all of the CRDM flange gaskets by 1996.

Davis-Besse developed a priority ranking system and replaced a number of leaking flange gaskets each outage based on outage duration rather than 100% repair. The ranking system was developed by the RCS engineer and is as follows:

Root Cause Analysis Report 3.0 Data Analysis

• 30 Root Cause Analysis Report 3.0 Data Analysis

• 30

Ranking System Developed by the RCS System Engineer Category Description No.

Weepage visible above nozzle at motor tube (MT) 1 interface and/or below the nozzle at the nut ring (N.R.) joint Minimal leakage at M.T. and/or N.R. to nozzle interfaces (with one or more runs)

Moderate leakage at M.T. and/or N.R. to nozzle 3

interfaces (with appreciable boron deposits adherent to the flange) 4 Heavy leakage with boron bridging adjacent flange surfaces Excessive boron accumulations on the insulation below the nozzle In 1990 (6RFO), gaskets were replaced in 23 CRDM flanges. Figure 28 shows the leaking flanges. There are no specific records indicating an inspection of the RPV head.

In 1991 (7RFO), the RCS engineer reported an excessive amount of boron on the RPV head. The boron flowed through the mouse holes and stopped on the RPV head flange by the closure bolts.

The CRDM flanges were inspected and 21 were identified as leaking and 15 were repaired.

Figure 28 shows all the leaking flanges identified in 1991 and the flanges that were justified for use-as-is.

In 1993 (8RFO), an inspection of the RPV head was performed, shown in Figures 29-32. In Figure 29, the boron deposits are dripping through the gaps in the insulation forming stalactites.

The boron deposits started forming stalagmites on the RPV head. Figures 30 and 31 show more boron deposits coming through gaps in the insulation and clinging to the side of the CRDM nozzles. The boron deposits in Figure 31 were reddish brown in color. The boron deposits on the RPV head in Figure 32 do not exhibit a clear picture of the source of leakage (i.e., CRDM flange or nozzle leakage).

Based on the results of the head inspection, the RPV head and flange was cleaned with deionized water. The effectiveness of the cleaning could not be verified in that the RPV head had already been returned to the RPV. A cleaning effectiveness inspection was recommended as a follow-up activity for the next outage. The CRDM flange inspection revealed 15 leaking flanges as shown in Figure 28. Framatome generated a non-conformance report (NCR) that noted degradation to the flange sealing surface found during the repair of CRDM nozzle 3 1. The corrective action taken was to perform flange surface polishing and gasket replacement. The 1993 NCR also recommended that the flange surface be machined if further leakage occurs.

In 1994 (9RFO), the CRDM flanges were inspected; however, no records have been identified indicating a visual inspection of the RPV head was completed. Performing a video inspection of weep holes was an activity in the outage schedule. There were no boric acid deposits interference problems with inspection equipment reported. Eight CRDM flanges were identified as leaking and repaired during this outage (Figure 28).

Root Cause Analysis Report 3.0 Data Analysis

• 31

1ORFO (1996)

Figure 20 provides an overview of the boric acid deposits on the RPV head from 1ORFO to 13RFO.

In 1ORFO, the remaining ten flanges without the new gasket material were upgraded. As discussed in PCAQR 98-0649, and as confirmed in interviews with the engineer responsible for performing inspections of the CRDM flanges during 1ORFO, one CRDM exhibited signs of minor leakage in 1ORFO. The 1ORFO's head visual inspection under the insulation, the majority of the RPV head was inspected except for the top center. A couple of nozzles are shown in a couple of background frames (Figures 33 and 34). These frames are approximately two to three nozzles away from the top center of the RPV head. The most conservative assumption that can be made from these figures is that boric acid extended from behind nozzles 2, 3, 4, and 5 to the bottom of the insulation. The assumed footprint of the boric acid is shown in Figure 20.

Comparing Figure 33 and Figures 29 and 30, the underside of the insulation in 1996 does not show crusted boron deposits or stalactites hanging.

The boric acid was powdery and white. Boric acid seemed to be flowing toward the mouse holes. The boric acid was very thin at the front edge with powder and small clumps of boric acid on top. Because the mouse hole locations were not periodically noted during the visual inspection, the location of this flow path is uncertain. However, based on future evidence, it is assumed to be the southeast quadrant of the RPV head. The remaining area of the RPV head was clean with speckles of white boric acid deposit. Figure 35 show a typical photo of the condition of the RPV head during this inspection. Additionally, PCAQR 96-0551 reported that there was some rust or brown-stained boron in the area around nozzle 67.

llRFO (1998)

Nozzle 31 was identified as having a minor flange leak using the following criteria: 1) there were no stalactites hanging from the flange, 2) there was no boric acid bridging to adjacent flanges, and 3) there was no rust present on either the flange or the split nut rings. Initial and follow-up review of the leaking flange by Davis-Besse Plant Engineering indicated that no immediate repair was required, and that this drive should be inspected during 12RFO and repairs made as required.

Framatome reiterated the recommendation from 8RFO to machine the surface of the nozzle 31 flange if further leakage occurred. Unidentified leakage data was reviewed for the past several cycles. With the numerous flange leaks present in both 7RFO and 8RFO, the highest unidentified leakage was approximately 0.3 gpm in cycle 7 and 0.4 gpm in cycle 8. The unidentified leakage in cycle 11 averaged 0.05 gpm. No Technical Specifications were exceeded even when the highest flange leakage was present. During the visual inspection of the control rod drive flanges, no interferences from boric acid accumulation on top of the insulation were identified.

During I IRFO, boric acid deposits were identified flowing out of the mouse holes in the southeast quadrant of the RPV head flange. The boric acid was a reddish rusty color. The RPV flange was decontaminated prior to the inspection of the RPV head.

The Service Water System Engineer conducted the RPV head visual inspection during 1 RFO.

The engineer worked with a Framatome crew using a pole-mounted camera to inspect the RPV head for "cracks in nozzles and degradation adjacent to the nozzle".

During the head visual inspection, the center nozzles were again very difficult to inspect through the mouse holes using available techniques. The engineer noted white streaks on the nozzles; Root Cause Analysis Report 3.0 Data Analysis

• 32 1

however, there was no boron hanging from the insulation. The engineer noted in a recent interview that some of the nozzles had indications of upward travel of the droplets as opposed to what would be expected (downward travel). The upward travel of the droplets was noted on several nozzles and attributed to ventilation flow. Boric acid was present in fist-sized clumps behind nozzles 9 and 13. Boric acid was collecting on the RPV head as shown in Figure 37.

Boric acid seemed to be falling from the top of the RPV head and collecting behind peripheral nozzles especially in the northwest and southeast quadrants. During this outage inspection, the boric acid was noted to be a mix of white and red deposits. Upon identification of red, rusty boric acid mixed in with white boric acid on the RPV head, the engineer worked with the Framatome crew to vacuum the RPV head and remove as much boron as possible. The equipment available to do the work and the limited access to the very top of the RPV head limited the removal process. During the removal of boric acid from the RPV head, the boric acid was noted to be brittle and porous. Other than these areas of accumulated boric acid, the RPV head was basically clean. Due to the limited inspection capability, the video evidence suggests that the most conservative estimate of the boric acid present would be to assume that behind nozzles 6, 7, 8, and 9 the boric acid extends to the bottom of the insulation and tapers off to the back of the next nozzle location. The approximate footprint of boric acid on the RPV head is shown in Figure 20.

12RFO (2000)

During the CRDM flange inspection, the upper camera was not positioned properly at four locations. The interference was attributed to a pile of boron on top of the insulation. The boron was a red, rusty color and hard. Normally, boron found on top of the insulation is a loose powder and in the color range from white to yellow depending upon age (based on video and interviews).

The boron pile encountered during this inspection was hard and could not easily be pushed to the side with the Framatome inspection tool. The underside of nozzle 3 was caked with red boric acid deposits. The inspection of the flange interface was accomplished by lifting the lower camera to see the upper flange interface. The interference locations, as shown in Figure 39, were identified in the center of the following nozzle blocks:

"* Nozzles 6, 15, 11, and 3

"* Nozzles 11, 27, 32, and 16

"* Nozzles 15, 31, 27, and 11

"* Nozzles 1, 3, 7, and4 Based on the CRDM flange inspection, nozzles 3, 6, 11, 31 and 51 flange leaks were repaired.

The CRDM flange on nozzle 31 was machined to remove a steam cut from the seating surface.

The Service Water System Engineer that conducted the RPV head visual inspection in the previous refueling outage requested to inspect the RPV head during this outage. To prepare for the inspection, he interviewed design and mechanical engineers familiar with this component and the industry issues associated, with it. Another contributing factor for the Service Water engineer to request to assist in the inspection of the RPV head was the fact that the RCS engineer was new to the Davis-Besse power plant. By assisting in the inspection, any changing conditions of the boric acid on the RPV head could be easily identified based on his experience in the 1998 inspection.

As shown in Figure 36, boric acid had accumulated on the RPV head flange behind the studs flowing out of the mouse holes in the southeast quadrant. The boric acid still had a red, rusty appearance. The mouse holes in this quadrant were significantly blocked with boric acid Root Cause Analysis Report 3.0 Data Analysis 9 33 1

deposits. With the studs in place on the RPV head flange and the accumulation of boric acid, the inspection through the mouse holes was significantly hampered. The engineer requested that the RPV flange be decontaminated and the studs removed to afford a better inspection. This work was completed. Boric acid on the RPV head was identified as an outage issue.

The RCS engineer supervised the cleaning effort, which entailed the following:

"* Pressurized (approximately 200 psi), demineralized water heated to 175 0F.

"* Water was sprayed on the boron deposits through the mouse holes and ventilation duct openings.

"* Estimated volume of water 100 to 600 gallons.

"* An inspection video was required post cleaning.

"* If the video revealed boric acid remaining on the RPV head, the cleaning steps were expected to be repeated.

The RCS engineer acknowledges that the cleaning was not 100% successful and some boric acid deposits were left behind on the RPV head. The engineer stated that he was running out of time to continue cleaning the RPV head (the RPV head was scheduled to return to the RPV during the next shift). Outage management concurred that no additional time and dose should be spent because further attempts would not produce successful results and the results were believed to be acceptable. Radiation Work Permit (RWP) 2000-5132 package was written as a tool to control radiological exposure for cleaning boric acid from the RPV head on April 6, 2000. The RWP identified 30 man-hours and a 100 mRem dose was estimated for the work. There were 282.31 man-hours and 1611 mRem expended for cleaning the RPV head.

No written evaluation was performed to allow the boric acid to remain on the RPV head. At this point in time, the modification to cut the openings in the service structure was scheduled for the next outage. With these openings and a more aggressive cleaning technique, the RPV head could be completely cleaned of the boric acid deposits and inspected. The amount of boric acid deposits left on the RPV head can not be estimated.

13RFO (2002)

During the CRDM flange inspection, the camera again encountered a boron pile in the vicinity of nozzle 3 making the inspection of the underside of the flange difficult. No flange leakage was identified during this outage indicating that previous repairs were successful.

The engineers responsible for inspecting the CRDM flanges reported boric acid deposits flowing out of the mouse holes and piled up to 4 inches high in the southeast quadrant on the RPV head flange and extending 3600 around theRPV head flange. The boric acid deposits in the southeast quadrant were hard-baked, whereas the deposits around the remainder of the RPV head flange were loose. During the inspection of the RPV head under the infsulation, significant boric acid was encountered in the southeast quadrant. In the remaining quadrants, significant piles of boric acid were encountered two to three nozzles in towards the center of the RPV head as shown in Figure 24. The deposits were hard, porous deposits and were a mixture of reddish brown and white deposits. The deposits were removed by hydrolasing, which operates at approximately 2,000 psi.

Root Cause Analysis Report 3.0 Data Analysis

• 34 Root Cause Analysis Report 3.0 Data Analysis

• 34

Documentation Available for Review RPV Head Flange RPV Head Accumulation Under Insulation Above Insulation Prior to 1996 PCAQR 91-0353 Video lORFO (1996)

PCAQR 96-0551 Video PCAQR 98-0767 &

11 RFO (1998)

Pictures 98-0649 Video 12RFO(2000)

CR 2000-0782 CR 2000-1037 CRDM Flange Pictures Video Inspection Video 13RFO (2002)

CR 2002-00685 CR 2002-00846 CRDM Flange Video Video Inspection Video PCAQR: Potential Condition Adverse to Quality Report CR: Condition Report 3.3.4 Containment Air Cooler Cleaning The CAC system is an engineered safety feature and is provided, in conjunction with the Containment Spray System, to meet the requirements of 10 CFR 50, Appendix A, General Design Criteria (GDC) 38, Containment Heat Removal. It consists of three separate tube/fin fan coolers, one associated with each of two trains. Two of the three coolers are associated with each of two safety related trains. The third cooler is a swing cooler (spare) and can be aligned mechanically and electrically to take the place of either of the other two coolers. Service water, ultimately supplied from Lake Erie, is supplied directly through the cooling coils to remove heat from Containment under both normal operating or accident conditions. The system has safety functions to cool CTMT during postulated accident conditions such as Loss of Coolant Accidents and Steam Line Breaks. During postulated accidents, operating in slow speed, each CAC is designed to move 58,000 cfm. During normal operation, the CACs are operated in high-speed and are available to remove normal process heat in CTMT, maintaining a maximum air temperature of 120'F at the inlet of the CACs. The three CACs are located in a row, next to each other, on the 585' elevation in CTMT. All CAC inlet air is drawn in at this location through the sides of the tube banks by the fan. The cooled discharge air supplies a distribution network inside the secondary shield structures, the reactor incore instrument tank, and RPV regions. The outside surfaces of the tube banks are readily visible from outside the coolers.

During operation, the service water supplied to the CACs is typically between approximately 40'F and 75°F. Being substantially cooler than CTMT, depending on CTMT humidity, the CACs remove water from the CTMT by condensation on the fin surfaces. This action would be expected to vary throughout the year. Both the dampness of the fin surfaces and high volumetric air flow rate cause the CACs to readily acquire a loading of boric acid particulate material, if it is present. In addition to collecting on the CAC cooling fins, boric acid accumulations have been observed and removed from CAC ductwork. For example, approximately 75 gallons (10 cubic feet) of wet boric acid were removed from the CAC plenum during 13RFO. Fouling of the CACs can be trended remotely by indication of plenum pressure. At Davis-Besse a fouling Root Cause Analysis Report 3.0 Data Analysis

• 351

condition occurred in 1992 (PCAQR 92-00072) due to a leaking flange on the primary side of a steam generator. Inspection of the CACs at that time revealed that the CACs were evenly fouled with white boric acid, which was readily cleaned with either steam or hot water sprays. After repair of the flange leak, the fouling of the CACs ended and no further cleanings (for rapid boric acid fouling) were needed for several years.

In October of 1998, there was a concern over the configuration of the pressurizer code safety valve discharge piping configuration. In brief, the safety valves discharged to a piping tee, with a rupture disc on each branch. Any weeping from the safety valves would be contained by the rupture discs and conducted to the pressurizer quench tank through a small drain line, quantified, and not counted as unidentified leakage. The tee was used with the design assumption that both rupture discs would simultaneously relieve if the code safety valves actuated. This would produce equal and opposite piping reactions, canceling each other to produce a zero net bending moment. After it was postulated that one or the other, but not both discs might relieve, it was realized that the design could result in a very large moment. Short term remedial action to resolve that concern involved deliberately failing the rupture disks. In November of 1998, PCAQR 98-1980 identified that fouling of the CACs appeared to be resuming, based on plenum pressure trends, coinciding with increased leakage from the pressurizer safety valves. Cleaning of the CACs continued, with 17 cleanings being needed between November 1998 and May 1999.

During the May 1999 mid-cycle outage, a pressurizer code safety valve piping modification resolved that issue. However, two subsequent CAC cleanings were still required, one in June 1999 and another in July 1999. Although the boric acid was generally reported to be white, a written post-job critique was located from the July 1999 cleaning that indicated a "rust color" was noticed "on and in the boron being cleaned away" from CAC 1.

After 12RFO, in June 2000, CAC plenum pressure again began to decrease (CR 2000-1547),

requiring resumption of cleaning. This was followed by five total cleanings in June, August, October and December of 2000. Cleanings continued in 2001, with four more (total) in January, February, March, and May. Following May 2001, the need to clean the CACs ended for the balance of the operating cycle.

During 12RFO, some CRDM flange leakage was repaired. Following 12RFO, but before 13RFO, it was not known whether those repairs had been fully successful. Therefore, the CAC cleaning could potentially have been attributed to CRDM flange leakage. However, 13RFO inspections revealed that the CRDM flange repairs in 12RFO were apparently successful.

Further, earlier experience with leaking flanges (pre-1992, and 1993 - 1998) did not result in the need to clean CACs. Therefore, CRDM flange leakage can now reasonably be ruled out as the cause of the cleaning of the CACs after 12RFO.

Attributing the need for CAC cleaning to leaking CRDM nozzles is plausible, but has several inconsistencies that would need to be explained. The most prominent is that if nozzle leakage continued on an increasing trend from May 2001 until February 2002, why did the need to clean CACs end in May 2001? The answer to this question can only be postulated and will not be known unless a different source of leakage is later identified. However, there are several potential explanations. These are related to CTMT humidity vs. SW temperature over the period, to reduction in RCS boron concentration at the end of the fuel cycle, but also possibly to changes in the morphology of the nozzle leak. For example, if the corrosion cavity at CRDM nozzle 3 enlarged substantially during the last half of the fuel cycle (affecting exit velocity), or the boric acid cap contained the leakage differently, the nature and amount of particulate matter might have changed. Larger particles might settle and not be subject to ingestion by the CACs. (The I

Root Cause Analysis Report 3.0 Data Analysis o 36 1

later theory has some anecdotal support based on observations that the boric acid dust on horizontal CTMT surfaces was more granular in 13RFO, as opposed to fine powder in earlier outages). However, this conjecture is subject to the similar pitfalls of the earlier (disproved) hypothesis that CRDM flange leakage (circa 1999) was different from CRDM flange leakage (pre-1992) and was therefore able to cause CAC fouling.

In summary, there was circumstantial evidence that CAC fouling was related to nozzle leakage prior to 13RFO. Because of variations in plant conditions, CAC fouling, by itself, could not be directly correlated with CRDM nozzle leakage.

3.3.5 Containment Radiation Monitor RE4597 Observations & Filter Plugging Radiation monitors RE 4597AA and RE 4597BA are two identical CTMT air sample monitoring systems, each with three detection channels and two sample locations. The monitors provide two of the three RCS leakage detection methods described by Reg. Guide 1.45 and required by TS 3.3.3.1 and 3.4.6.1, namely CTMT particulate and noble gas activity detection. These parameters are monitored because of their sensitivity and rapid response to leaks in the Reactor Coolant Pressure Boundary. Detection of radioactive iodine is also provided. A continuous sample drawn from CTMT passes through a fixed particulate filter, an iodine cartridge, and a pump. The sample then passes through a noble gas chamber and is discharged back to containment atmosphere. The containment radioactive gas monitor is less sensitive than the containment air particulate monitor and would function in the event that significant RC gaseous activity existed from fuel cladding defects. The normal sample location of RE4597AA is approximately 4 feet above the top elevation of the South wall of the West secondary shield structure in CTMT (see Figure 40). The alternate sample location is below the polar crane, at approximately 270 degrees azimuth (due West) in CTMT. The normal sample location of RE4597BA is approximately 4 feet above the top elevation of the East secondary shield structure in CTMT, but against the CTMT wall at approximately 90 degrees azimuth (due East). The alternate sample location is by the stairway to the incore instrument tank platform on the 603' elevation (i.e., near the personnel lock).

The areas of interest pertaining to the Containment Radiation Monitors revolves around two issues: 1) their capability to detect a leaking CRDM nozzle directly by their output indication, or

2) other incidental maintenance observations. For the case in point, the maintenance observations centered on unusual collection of boric acid and iron deposits on the filter elements of the monitors, necessitating frequent replacement. These points are discussed in the following paragraphs.

Particulate Monitor (Channel 2)

The containment airborne particulate monitor measures the buildup of particulates on a fixed filter and compares this to the integrated sample flow that produced the particulate buildup. In five minutes the airborne particulate radioactivity monitor can detect the increase in particulate radioactivity concentrations from a 0.1 gpm reactor coolant leak into the containment vessel postulated to occur when reactor coolant fission product activity concentrations result from 0.1 %

failed fuel at the beginning of core life (4 EFPD). Once in the equilibrium cycle with 0.1% failed fuel, a 1 gpm leak can also be detected in five minutes. The particulate monitor consists of a fixed particulate filter in a 3 inch 4 pi lead shield. A beta detector is inserted into the lead shield to detect the activity deposited on the filter paper. The filter paper is 99 percent efficient for 0.3 micron and larger particles. The output from the detector is fed to the microprocessor where the counts per minute are converted to pCi/cc. Although this detector is effective in identification of Root Cause Analysis Report 3.0 Data Analysis o 37

a rapid change in leakage, some of the predominant isotopes that provide the indication include long lived Cesium 137 and Cobalt 60. These isotopes have a half life in the range of 5 to 30 years. Based on this, even with a constant RCS leak rate and coolant activity, they could tend to constantly accumulate in containment over the course of a fuel cycle, giving a continuously increasing detector response that could be difficult to distinguish from subtle changes in leakage.

The output would also be expected to fluctuate with filter changes. Therefore, the particulate detector was not further considered for possible long term detection of CRDM nozzle leakage.

Iodine Monitor (Channel 3)

After passing through the particulate filter, the sample is drawn through an iodine collector. The iodine monitor is a 3 inch 4 pi lead shield containing the iodine collection cartridge and a gamma scintillation detector. The iodine collector efficiency is greater than 95 percent. The output from the detector is fed to a microprocessor. The microprocessor looks at two windows from this detector. The upper window is a background (5 percent above the iodine peak) and the lower window is centered on the iodine peak. The upper value is subtracted from the lower value giving a true iodine reading with output converted to pCi/cc. The iodine detector is capable of detecting iodine radioactivity on concentrations as low as 7x 10-7 pCi/cc of containment air. The predominant Iodine isotopes released from the reactor coolant are Iodine 131 and 133 with half lives of 8 days and 21 hours2.430556e-4 days <br />0.00583 hours <br />3.472222e-5 weeks <br />7.9905e-6 months <br /> respectively. For a constant RCS leakage rate and coolant activity, these isotopes will reach a stable equilibrium value in containment and would thus theoretically provide a direct and valid indication of a slowly evolving RCS leakage trend. These isotopes have the added advantage of being actively trended in the reactor coolant for tracking of fuel defects, so that changes in coolant activity could theoretically be accounted for in so that output could be used to determine RCS leakage rates.

Output data from the detectors was manually recorded on a monthly basis from late 1992 through the present. However, due to high cycle 13 coolant activity and known increases in RCS leakage, the detectors frequently saturated during the fall of 2001. This resulted in a loss of alarm function for the remaining channels. Therefore, the carbon filters were removed from the detectors in November 2001, effectively removing the Iodine Channels from service. Data prior to this time is presented on Figure 41. Although the output indicates a clearly increasing trend, the output readings from this monitor suffer from a significant amount of scatter. The cause of the scatter is not definitively known, however, it might be related to readings being taken near the time of change-out of the carbon elements (response not at equilibrium) or it might be related to actual changes in CTMT atmosphere conditions (e.g. scrubbing of the iodine by condensate on the containment air coolers, or retention by condensate in the sample lines.) Further, a large portion of the trend is undoubtedly due to increasing RCS activity due to fuel defects. An attempt was made to separate the effects of the coolant activity by taking a ratio of detector output with coolant activity. This also resulted in an increasing trend, but it suffered doubly from the scatter in both the monitor data and RCS activity data. In the end, although increased leakage was clearly detectable, there is no means to distinguish CRDM nozzle leakage from any other RCS leakage, and so this indication was not particularly valuable.

Noble Gas Monitor (Channel 1)

After passing through the particulate and Iodine monitors, the gas sample is finally drawn through the noble gas monitor. The monitor housing is a 4 inch 4 pi lead container. The detector is a beta detector with an internal light emitting diode (LED) check source. The output of the detector is fed to a microprocessor where the counts per minute are converted to pCi/cc. Some of the predominant isotopes that remain to be counted by this detector are Xenon 133 and 135, Root Cause Analysis Report 3.0 Data Analysis

• 38 1 I

with half lives of 9.2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> to 5.2 days. For a constant RCS leakage rate and coolant activity, these isotopes will also reach a stable equilibrium value in containment and would thus theoretically provide a direct and valid indication of a slowly evolving RCS leakage trend. Trend data from these monitors is presented in Figure 42.

The output from these detectors is much more consistent than the output of the iodine channels.

These detectors are sensitive and reflective of RCS leakage trends and changes in RCS activity.

Detector output is particularly sensitive when RCS activity is high, as during cycle 13. Under this condition, noble gas activity could provide indication of very small RCS leakage prior to positive identification through the RCS inventory balance. To determine a leak rate, a representative combination isotopes in the RCS would need to be found to achieve an appropriate scaling factor to screen out the effects of RCS activity. Assuming this was accomplished, other RCS leakages could still mask the relatively small leakage expected from a CRDM nozzle leak.

Therefore, these detectors are also of limited value for diagnosis of CRDM nozzle leakage.

RE4597 BA and RE4597AA Filter Changes RE4597AA and RE4597BA have been the subject of numerous Condition Reports due to moisture in the lines, and clogging of the filter elements with boric acid. Moisture in the lines has been associated with restarting from outages and is attributable to high CTMT humidity. The humidity arises from the initial humidity when CTMT is closed and long term accumulation from a variety of primary and secondary leak sources. Temperature changes along the sample piping as the sample is continuously drawn from CTMT to the monitors can cause condensation of water in the piping before the monitors and can interfere with monitor operation. Due to the variety of sources of moisture, and the fact that this condition has occurred for a prolonged period of time (reference PCAQR 92-0346), it is not particularly associated with nozzle leakage.

Relatively large RCS leakage sources have the demonstrated potential to produce an aerosol mist due to flashing and evaporization of the jet of liquid as it exits from the leak. An example of this type of leakage occurred when the "head to hot leg vent" line developed a flange leak in 1992.

When the leakage source contains borated water, the boric acid is dispersed with the aerosol as a fine particulate material. This material remains suspended in the CTMT atmosphere for a prolonged period, before eventually settling out on CTMT surfaces and appearing as a fine powder. When ingested by the CTMT radiation monitors, the boric acid will prematurely clog the monitor filters and require frequent filter changes. This condition occurred during the RPV head to hot leg vent line flange leak, but returned to normal following that repair.

Normally, the change frequency for the RE4597AA and RE4597BA filters is approximately 30 days, and is dictated by schedule rather than low flow. However, in March of 1999 fouling of the monitor filters began to recur (CR1999-0372, CR1999-0861, CR1999-0882). Initially, this was attributed to the disabling of the pressurizer code safety valve rfipture discs in late 1998 (discussed in the CAC section). It was noted that the service life of the filters had depreased, particularly for RE4597BA. However, by May 19, 1999, the boric acid deposits on the filters had developed a "yellow" or "brown" appearance. Under CR1999-1300, sample filters were sent to Southwest Research Institute (SRI) for analysis. The SRI report (Project 18-2321-190) indicated that the samples contained predominantly ferric oxide from corrosion of iron components. An adjunct report from Sargent and Lundy (Project 10294-033) indicated that the fineness of the particles suggested that it was attributable to a steam leak. From May of 1999 until April 2001, filter changes on RE4597BA were required on an irregular I to 3 week interval.

It was noted in CR01-1110 that the filter life had reduced to around 3 days. The sample point was changed to the alternate location near the personnel lock, and service life improved slightly.

3.0 Data Analysis e 39 Root Cause Analysis Report

However, by November of 2001, filter replacements were again required approximately every other day. On November 2, a blank (no carbon) cartridge was installed in the iodine channels of both monitors to eliminate a frequent alarm condition. Throughout the period of 1999 through 2001, RE4597AA exhibited similar, but slightly less severe symptoms.

The reactor service structure, which encloses the CRDMs, CRDM flanges, and CRDM nozzles, is ventilated by one of two fans that take suction on the area immediately surrounding the CRDMs and CRDM flanges. It takes an indirect suction on the area surrounding the CRDM nozzles, drawing through the mirror insulation that separates the CRDM nozzles from the CRDMs and CRDM flanges. The fans discharge on the 603 elevation, in the North-East quadrant of the reactor building. Airborne flange leakage or nozzle leakage would be exhausted by the ventilation fans to this area. The fan exhaust is closer to the normal suction of RE4597BA than to the normal suction of RE4597AA. This would tend to explain why boric acid fouling was more severe for RE4597BA, and why the symptoms were reduced when switching to the alternate sample location, which is diametrically opposite the CRDM ventilation fans.

Accumulation of boric acid on the radiation monitor filters was recognized to be symptomatic of an RCS leak as soon as it occurred. Significant efforts were made, especially during the cycle 12 mid-cycle outage in 1999 and 12RFO in 2000 to locate the source of leakage. During that outage, the only significant leakage potentially capable of producing the amount of boric acid necessary to exhibit the necessary symptoms was found to be leaking CRDM flanges, particularly at the nozzle 31 location. However, the presence of iron oxide in the boric acid on the filter elements was not explained.

In August 1999, four high efficiency particulate filters were placed in CTMT near the elevator on the 603' elevation. These 500 cfm filtration units were intended to help clean up the CTMT atmosphere on the theory that airborne material was left over from the cycle 12 mid-cycle outage.

The filters were removed in October 1999. The filters had no notable effect on the CTMT atmosphere.

Based on the observations that there was a high boric acid accumulation near the CRDM exhaust fans and no leaking CRDM flanges found in 13 RFO, it can now be inferred that the boric acid found in the RE4597 filters (and in the CACs) originated at the CRDM nozzles and was dispersed by the CRDM exhaust fan.

3.3.6 Containment Recirculation Fan/Fan Failures The Containment Recirculation System (CRS) is composed of two non-safety related fans and associated ductwork. The CRS circulates the air in the Containment Dome during all plant modes of operation to eliminate the temperature stratification. The CRS is normally operated continuously. However, in February of 1999, CRS fan I failed and remained out of service. In March of 2001, CRS fan 2 failed and remained out of service. The failure mode involved failure of the motor bearings, and significant destructive rubbing of the fan blades on the housings.

Failure of fan 1 significantly preceded discovery of brown deposits on RE4597 filters in May of 1999. Failure of fan 2 did not occur until well after iron appeared on the RE4597 filters. Iron continued to appear on the filters well after both fans were out of service. The out of service dates also do not coincide with other events. The particulate iron that would be expected from the fan blades is not similar in particle size that was found on the RE4597 filters. Therefore, the failure of the CRS fans does not appear to be the cause of the iron deposits on the RE4597 filters.

Root Cause Analysis Report 3.0 Data Analysis e 40

3.4 Programs Important to Preventing Problems This section of the data analysis provides a discussion of programs the root cause team viewed as important to preventing this type problem. Industry programs are intended to provide advance warning and to recommend approaches to avoiding significant problems. The Boric Acid Corrosion Control and the Inservice Inspection (ISI) programs are intended to provide a level of defense by ensuring the integrity of the RCS and supporting systems used to mitigate plant transients. The review included interviews with the program owners. Both programs were reviewed as part of the root cause investigation.

3.4.1 B&W Owners Group and Industry CRDM Nozzle Related Initiatives In November 1990, the B&WOG Materials Committee issued Report 51-1201160-00, Alloy 600 SCC Susceptibility: Scoping Study of Components at Crystal River 3 (reference 8.2.10). Very little attention had been given to inspection for PWSCC in Alloy 600 applications other than that associated with the steam generator tubing. As a result of the reported instances of PWSCC in the pressurizer heater sleeves and instrument nozzles in several domestic and foreign PWRs, the NRC felt that it may be prudent for licensees of all PWRs to review their Alloy 600 applications in the primary coolant pressure boundary, and, when necessary, implement an augmented inspection program (reference IN 90-10). The Materials Committee initiated a scoping study to investigate potential problems associated with PWSCC of Alloy 600 material used in B&W designed RCS components. The report summarized the completed research regarding Alloy 600 components used at a target B&WOG plant Crystal River 3. Based on this information, a susceptibility rating was given, along with recommendations for ensuring RCS integrity through inspections of appropriate components. The applications of Alloy 600 at other B&W operating plants were identified and the applicability of the target plant evaluation to these other operating plants was confirmed. This summary was to be used by the B&WOG Materials Committee in assessing the potential for future PWSCC occurrences with Alloy 600 components at B&W operating plants. The report notes that it is expected that the locations having the highest temperatures in the RCS would be the most susceptible to PWSCC. The RPV upper head is identified as one area where attention should be given. The report recommends the control rod housing bodies be inspected, if possible, at an opportune time. The report includes a table of Alloy 600 locations at Davis-Besse, which includes the 69 CRDM nozzles. The report also includes a summary of PWSCC occurrences of in-service RCS Alloy 600 components.

In December 1990, EPRI issued EPRI NP-7094, Literature Survey of Cracking of Alloy 600 Penetrations (EPRI Project 2006-18) (reference 8.5.10) to document the problem of stress corrosion cracking of Alloy 600 penetrations in PWR pressurizers and to identify corrective actions that utilities can take to address this issue. The document lists the CRDM nozzles as an Alloy 600 component.

In October 1991, the first EPRI workshop on PWSCC of non-steam generator Alloy 500 materials in PWR plants was held, with representatives from the U.S. and French nuclear facilities, all U.S. Owners Groups (Westinghouse, Combustion Engineering, and B&W), EPRI, the U.S. Navy, and various vendors/consultants. This workshop provided extensive coverage of PWSCC in pressurizer instrument nozzles, pressurizer heater sleeves, steam generator drain lines, and hot leg instrument nozzles. The B&WOG provided an update on B&W activities, including the Materials Committee scoping study of Crystal River 3 and the areas of concern, including the Control Rod Housing Bodies. Later, it was learned that during a 10-year hydrostatic test in September 1991, the French Bugey 3 plant discovered a leak in a CRDM nozzle, via a through-wall crack. The crack was caused by PWSCC in an area of high residual Root Cause Analysis Report 3.0 Data Analysis *, 411

stresses caused by the J-groove weld joining the nozzle to the RPV head. Additional cracks were subsequently found in other plants in France, Sweden, and Belgium.

On May 12, 1992, the B&WOG Materials Committee met with the NRC staff and provided a presentation on "Work on PWSCC of Alloy 600 Nozzles and Components" which included information on the Bugey 3 CRD nozzle leakage. NRC concurred with the B&WOG that, based on the available information on the French CRDM nozzle inspection, there is no immediate safety concern due to the fact that the identified cracks are axial in nature. The NRC suggested another meeting during 1 st quarter 1993 to cover the following on the CRDM nozzle cracking vis-a-vis B&WOG plants:

1. 50.59 Safety Evaluation to provide sufficient assurance that the issue is not a safety concern

2. CRDM nozzle inspection strategy/criteria

3. Evaluation of leak detection/monitoring system.

On 8/10/92 - 8/11/92, there was an EPRI Alloy 600 Coordinating Group Meeting Concerning CRDM Nozzle Cracking attended by representatives from each of the NSS vendors, several utilities, and Dominion Engineering. Work on CRDM nozzle cracking in the Owners Groups was presented and discussed. One item discussed was that no one was expected to inspect CRDM nozzles during the 1992 fall outage schedule unless required by the NRC. The NRC position was expected to be finalized at a Westinghouse Owners Group (WOG) meeting on 8/18/92.

On August 18, 1992, the NRC met with members of WOG to discuss the safety significance of CRDM penetration cracking and update the status of WOG's Alloy 600 program. The meeting was attended by representatives from each of the Nuclear Steam Supply (NSS) vendors, each of the owners groups, several utilities, and consultants. The NRC provided an overview of Alloy 600 PWSCC and their view on CRDM nozzle inspections. The staff viewed the CRDM nozzle cracking as a minimal safety impact, but that prudence suggested an orderly inspection program.

The NRC was concerned that the potential for cracking exists in a large number of nozzles and that there is concern with boric acid corrosion of the RPV head. The staff presentation slides indicated the following inspection, evaluation, and repair guidance: (1) For PWR plants refueling before spring 1993, visual inspection during leakage test, with special attention to CRDM penetrations at periphery locations and visual inspections (VT-2 quality) remote or direct to inspect the inside surface of the spare CRDM penetrations; (2) For PWR plants refueling after Spring 1993, PT and eddy current (EC) inspections of the inside surface of all spare CRDM penetrations; (3) EC inspection of CRD sleeved penetrations if cracks are found; (4) Provide flaw acceptance criteria; and (5) Develop corrective actions for CRDM penetrations. Recent work on CRDM nozzle cracking in the WOG was then presented and discussed. It was stated that inspection of CRDM nozzles during the 1992 fall outage schedule was not planned by any of the owners groups unless ongoing safety evaluations indicate that there is a safety concern. The NRC appeared to agree with this, but wanted to review the WOG safety evaluation (scheduled for completion 10/31/92) and requested another meeting with the WOG in November. The NRC also stated that they would entertain a submittal without an NDE (ECT or UT) inspection plan but the basis for this decision must be very convincing. Coordination of the activities of the Owners Groups on Alloy 600 CRDM penetration cracking was planned to be done by Nuclear Utility Management and Resource Council (NUMARC). The NRC staff believed the reported cracking in CRDM penetrations was not an immediate safety issue requiring regulatory action.

There was time for a thorough, disciplined analysis of the safety significance, the approach to RPV head inspection, criteria for taking repair actions, and possible regulatory guidance.

Root Cause Analysis Report 3.0 Data Analysis e 42

On October 2, 1992, the B&WOG issued a proprietary Alloy 600 PWSCC Time-To-Failure Models report (reference 8.2.11), presenting a PWSCC susceptibility ranking model and six susceptibility models that had been proposed within the nuclear industry to model time-to-failure of Alloy 600 components as a result of PWSCC. The PWSCC susceptibility ranking model for Alloy 600 RCS components was based on carbon content of the material, anneal temperature and duration, operating temperature, and operating and residual stresses. A ranking of 4, 4-5, or 5 indicates a high (50%) probability of failure within 20 years; a ranking of 3 or 3-4 indicates a medium (50%) probability of failure within 40 years; and a ranking of 2-3 or below indicates a low probability of failure within 40 years. All failures at the time had been ranked between 4 and 5 with this ranking model. The report provided the susceptibility ranking of the Alloy 600 components on B&W designed plants. The Davis-Besse CRDM nozzles were of four different heat numbers: heat M3935 was ranked as 2-3; heat NX5940 was ranked 3-4; heat C2649 was ranked 5; and heat M4437 was ranked 4-5. Based on one of the models, the time-to-failure calculation for the worst case (heat C2649) predicted 123 EFPY for 50% of the population to initiate cracks. The report concluded that, although none of the models addressed in this document accurately predicts any of the existing industry failures of Alloy 600 components, it contained a good base of ideas to improve the time-to-failure model.

In December 1992, the second EPRI workshop on PWSCC of Alloy 600 in PWRs was held, with representatives from U.S., French, Swedish, and Japanese nuclear facilities, all U.S. Owners Groups, the U.S. Navy, and various vendors. Workshop sessions focused on concerns about PWSCC of alloy 600 penetrations in the RPV head (CRDM nozzles) in several plants, including the Bugey 3 plant in France. A stress analysis summary concluded the stresses are highest in the outermost nozzles for Westinghouse plants, while the stresses are essentially the same for central and outer row nozzles for B&W plants. Another report indicated field experience to date shows that cracks have occurred predominantly in peripheral row nozzles, consistent with the results of finite element stress analyses.

Later that month, B&W issued a proprietary CRDM Nozzle Characterization report (reference 8.2.12), regarding PWSCC of CRDM nozzles. The fabrication and manufacturing processes for B&W-design CRDM nozzles and French-design CRDM nozzles were discussed. A comparison of this information was made, and the similarities and differences were noted. It was determined that B&W-design nozzles are not significantly different than the French-design nozzles, and, thus, are not immune to PWSCC. In the report, Davis-Besse is noted as having all 24 of its peripheral nozzles rated as "very high susceptibility" for PWSCC, as are 40 of its 45 non peripheral nozzles. This report differs from the previous report (10/2/92) in that heat NX5940 was now ranked as 5 (instead of the previous 3-4). The report also lists the heat number for each CRDM nozzles and notes that nozzles 1-5 are all of heat number M3935, the lowest susceptibility ranking (2-3) for Davis-Besse nozzles.

An Ad Hoc Advisory Committee (AHAC) headed by NUMARC with members from all three Owners Groups and EPRI was formed to formulate the CRDM nozzle inspection criteria and coordinate the relevant industry activities. On March 3, 1993, the AHAC met with the NRC and discussed the WOG Safety Evaluation. The B&WOG committed to perform an evaluation of the safety significance of potential nozzle cracking.

On May 26, 1993, the B&WOG issued BAW-10190P, Safety Evaluation For B&W Design Reactor Vessel Head Control Rod Drive Mechanism Nozzle Cracking (reference 8.2.7) summarizing the stress analysis, crack growth analysis, leakage assessment, and wastage assessment for flaws initiating on the inner surface of the B&W designed CRDM nozzles. The Root Cause Analysis Report 3.0 Data Analysis

• 43

overall conclusion reached in this evaluation was that the potential for cracking in the CRDM nozzles does not present a near-term safety concern. Crack growth analysis predicted that once a crack initiates, it will take a minimum of six years for the flow to propagate through-wall. If a crack propagates through-wall above the nozzle-to-head weld, leakage was anticipated and a large amount of boric acid deposition was expected. Once boric acid deposition occurs from leakage, wastage of the RPV head can initiate. It was predicted that wastage of the RPV head can continue for six years before ASME code limits are exceeded. The B&WOG utilities developed plans to visually inspect the CRDM nozzle area to determine if through-wall cracking had occurred and if boric acid deposition was occurring as result of a through-wall crack. The report identifies that at each of the B&WOG utilities' plants, a walkdown inspection of the RPV head was implemented as part of the response to NRC Generic Letter 88-05. Enhanced visual inspection of the CRDM nozzle areas was also incorporated. If any leaks or boric acid crystal deposits are located during the inspection of the RPV head area, an evaluation of the source of the leak and the extent of any wastage was required to be completed. A conservative wastage volume of 1.07 cubic inches per year was believed to be possible from a leaking CRDM nozzle.

The postulated corrosion wastage within and in the vicinity of the RPV head penetration from a leaking CRDM nozzle would not affect safe operation of the plant for at least six years. The boric acid deposition was expected to be detectable by the current GL 88-05 inspections. Since inspections of the RPV head area (for leakage and boric acid deposits) are performed during each outage, it was thought to be unlikely that a leak would go undetected for a period of six years.

The evaluation concludes excessive wastage of the RPV head will not occur before leakage is detected either by visual observations in accordance with utility responses to GL 88-05 or the plant leakage detection system. The B&WOG also stated it was evaluating the potential for crack initiation and propagation on the nozzle outer surface, although preliminary evaluation of the through-wall stress distribution indicates that, even if a circumferential crack initiates on the outer surface, the crack will be self-relieving and will not cause separation of the nozzle. The B&WOG was continuing its involvement in the NUMARC-sponsored AHAC for PWSCC of CRDM nozzles, including the industry-sponsored crack growth testing of CRDM penetration materials. Duke Power Company scheduled an inspection of one B&W designed reactor in the fourth quarter of 1994.

On November 19, 1993, the NRC issued its Safety Evaluation for Potential Reactor Vessel Head Adaptor Tube Cracking to NUMARC (reference 8.3.20). The staff concluded that there was no immediate safety concern for cracking of the CRDM penetrations. The bases for this conclusion (reference 8.3.2) were that if PWSCC occurred at RPV head closure penetrations: the cracks would be predominately axial in orientation, the cracks would result in detectable leakage before catastrophic failure, and the leakage would be detected during visual examinations performed as part of surveillance walkdown inspections before significant damage to the RPV head would occur. This finding was predicated on the performance of the Visual inspection activities requested in Generic Letter (GL) 88-05. Also, special nondestructive examinations were scheduled to commence in the spring of 1994 to confirm the safety analyses for each PWR owners group.

On December 14, 1993, the B&WOG Materials Committee issued BAW-101 90P Addendum 1, External Circumferential Crack Growth Analysis for B&W Design Reactor Vessel Head CRDM Nozzles (reference 8.2.8) providing an evaluation of external circumferential crack growth, gross leak-before-break mechanism, and the stress affects of CRDM nozzle straightening. The report concludes that there was no possibility for an external circumferential flaw indication to grow circumferentially to the point of becoming a safety concern. The overall conclusions presented in Root Cause Analysis Report 3.0 Data Analysis e 44

B&W-10190P remained unchanged with this addendum. It was concluded the GL 88-05 walkdown visual inspections of the RPV head areas provide adequate leak detection capability.

In March 13, 1994, the RCS System Engineer initiated PCAQR 94-0295 regarding a commitment in the commitment management system that was closed (complete) and not converted to an ongoing commitment. The commitment required a visual inspection of the RVP head every refueling to determine the potential for CRDM nozzle cracking in support of a B&W safety evaluation to the NRC. The PCAQR evaluation identifies the inspection is covered in the existing program as outlined in NG-EN-00324, Boric Acid Corrosion Control. The commitment was closed based on the NG-EN-00324 inspections and the fact that the NRC saw enhanced inspections as being "prudent" but not necessary were to be put in the next outage contract.

In November 1994, the 1994 EPRI Workshop on PWSCC of Alloy 600 in PWRs was held. The workshop summarized the field experience associated with PWSCC of Alloy 600 CRDM nozzles, reviewed the current status of inspection, repair, and remedial methods as well as strategic planning models, and discussed stress analysis results as well as PWSCC initiation and growth in Alloy 600. The workshop was attended by domestic and overseas utilities, PWR vendors, research laboratories, and consulting organizations. Three U.S. plants had inspected CRDM nozzles; no cracks were found in one plant and only minor cracking was observed on one nozzle in each of the other two plants. Results of inspections in France, Sweden, Spain, Belgium, Japan, and Brazil revealed a trend toward earlier axial cracking in plants with forged nozzles as opposed to those made from rolled bars or extrusions. It was also thought that other factors such as surface finishing could play a role (see reference 8.5.4).

On April 7, 1997, Davis-Besse received GL 97-01 Degradation of CRDM/CEDM Nozzle and other Vessel Closure Head Penetrations (reference 8.3.2). The letter requested plants describe their program for ensuring the timely inspection of PWR CRDM and other RPV head penetrations (VHP). In July1997, the B&WOG Materials Committee issued BAW-2301, B&WOG Integrated Response to Generic Letter 97-01: "Degradation of Control Rod Drive Mechanism Nozzle and other Vessel Closure Head Penetrations" (reference 8.2.1). On July 28, 1997, Davis-Besse responded to the GL 97-01 endorsing BAW-2301. The BAW topical report provides the justification and schedule for an integrated VHP inspection program.

The BAW-2301 introduction reiterates conclusions discussed in references 2.7 and 3.20. The introduction furthermore states PWSCC for CRDM nozzles and other VHPs will not become a long-term safety issue provided the enhanced boric acid visual inspections, performed in accordance with GL 88-05, are continued. An axial crack would lead to a leak on one or more nozzles and result in a significant deposition of boron crystals. It is very unlikely that this type of accumulation would continue undetected with regular walkdown inspections of the RPV head area. If the crystals remain hidden by the RPV insulation, the insulation would begin to bulge as a result of this accumulation of crystals. This deposition would easily be detected prior to significant damage to the RPV head. Therefore, the RPV head's structural integrity Would not be jeopardized, thereby eliminating any safety concerns with PWSCC of these nozzles. In order to assure the assumptions of the original safety evaluation remain valid, an integrated inspection program had been developed to address this issue for the B&WOG plants.

The BAW-2301 report presents the integrated B&WOG inspection program. Oconee 2 and Crystal River 3 are identified as two of the B&WOG plants most susceptible to PWSCC, as currently ranked. These two plants either have or will perform inspections of the RPV head nozzles from beneath the RPV head. Oconee I and 3, Davis-Besse, ANO 1, and TMI I do not have CRDM nozzle inspections planned in the near term (1998-2000).

Root Cause Analysis Report 3.0 Data Analysis

• 451

In May 1998, the Davis-Besse Materials Committee representative initiated a procedure change request to NG-EN-00324, Boric Acid Corrosion Control. The change requested the B&WOG Materials Committee Report 51-1229638 Boric Acid Corrosion Data Summary and Evaluation be add to a note that identifies material that contain helpful reference material for determining boric acid corrosion rates. The information was incorporated into the procedure as requested in April 1999.

On April 30, 2001, the NRC issued Information Notice (IN) 2001-05 to alert plants to the recent detection of through-wall circumferential cracks in two CRDMs nozzles and weldments at Oconee 3. On May 2, 2001, CR 01-1191 initiated identifying the need for a project plan with team members developed to prepare Davis-Besse for a cracked CRDM J-groove weld. The CR identifies all three units at Oconee and one unit at ANO have inspected for and found cracked J groove welds around their CRDM nozzles.

On August 3, 2001, NRC issues NRC Bulletin 2001-01 Circumferential Cracking of Reactor Pressure Vessel Head Penetration Nozzles. The discussion section identifies recent identification of circumferential cracking in CRDM nozzles and axial cracking in the J-groove weld has resulted in the NRC staff reassessing its conclusion in GL 97-01 that cracking of VHP nozzles is not an immediate safety concern. Circumferential cracking in CRDM nozzles were identified at Oconee 2 and 3, and axial cracking in the J-groove weld in CRDM nozzles were identified at Oconee 1 and ANO 1 (i.e., B&W plants). The findings at Oconee 2 and 3 highlight the possible existence of a more aggressive environment in the CRDM housing annulus following through wall leakage; potentially highly concentrated borated primary water could become oxygenated in this annulus and possibly cause increased propensity for the initiation of cracking and higher crack growth rates. Regulatory Affairs initiated CR 01-2012 in response to the bulletin.

Between September 4 and November 30, 2001, Davis-Besse met with and docketed responses to the NRC regarding NRC Bulletin 2001-01. In discussion held with the NRC on November 28, 2001, Davis-Besse committed to a 100% qualified visual inspection, non-destructive examination (NDE) of 100% of the CRDM nozzles and characterization of flaws through destructive examination should cracks be detected. Several other commitments were also made at that time including moving forward the start of the scheduled refueling outage from April 1 to no later than February 16, 2001.

3.4.2 Davis-Besse Boric Acid Corrosion Control Program As discussed above, and as part of the root cause evaluation of the Davis-Besse RPV head degradation, the Boric Acid Corrosion Control program was reviewed. The intent of this review was to compare various aspects of the program to GL 88-05 and how the program is currently being implemented as it relates to the RPV head.

Generic Letter 88-05 was issued March 17, 1988 to address the effects of boric acid leaks on carbon steel components. All license holders for PWRs were required to address the'issues identified in the generic letter. GL 88-05 identifies four areas that must be addressed in the plant specific boric acid program. The areas include:

"* determination of principal leak locations where leaks may occur that are smaller than technical specifications allow

"* procedures for locating small leaks

"* methods of conducting examinations and performing engineering evaluations to address the impact of the leak

"* corrective actions to prevent recurrence of this type of corrosion Root Cause Analysis Report 3.0 Data Analysis

• 46 I

The Boric Acid Corrosion Control program procedure (NG-EN-00324) was reviewed against these same points.

Principal Leak Locations The procedure identifies the following areas as principal locations for possible leaks:

"* Steam Generator and Pressurizer manways and handholes

"* Seal Welds

"* Thermowells

"* Reactor Coolant Pump and other pump seals and casing flanges

"* Control Rod Drive Flanges

"* Piping Flanges and Bolted Connects

"* Valve Bonnets and Packing Glands

"* Reactor Vessel Head O-rings The procedure does not address industry known leakage areas such as the CRDM nozzle issue or potential leakage areas such as the lower RPV head area. Currently the procedure is limited to inspections of systems and components inside Containment.

As an example, PCAQR 94-0295 discussed the need for enhanced inspections of the RPV head in addressing the CRDM nozzle cracking issue. The later text in PCAQR 94-0295 states that B&W amended its safety evaluation following feed back from the NRC that stated enhanced inspections were not required. B&W's amended safety evaluation took credit for the GL 88-05 inspections. Discussion with Framatome indicate the safety evaluation dated May 1993 was never changed to eliminate the need for enhanced inspections. See PCAQR 94-0295 discussion in the condition report section for additional details concerning this specific issue.

Procedures for Locating Small Leaks Step 2.1.1 of the Boric Acid Corrosion Control procedure identifies a number of station procedures that support inspection and identification of leakage. Several of the procedures were reviewed to generally determine the kinds of inspections that are required. DB-OP-06900 Plant Heatup requires an inspection at operating temperature and pressure. DB-OP-01200 Reactor Coolant System Leakage Management provides guidance and trigger points with regard to Technical Specification leakage values. The DB-OP-01200 includes trigger points for "buildup of boric acid on equipment requires frequent containment entries to clean and/or inspect". Both procedures prompt action to identify and characterize leakage however, the values of unidentified RCS leakage causing this type of a condition are relatively small.

DB-PF-03065 Pressure and Augmented Leakage Test performs a leakage inspection at temperature and pressure in support of the ISI program. This procedure will be addressed in the ISI section.

Conducting Examinations and Engineering Evaluations The procedure outlines various activities that "should" be performed to ensure the component is serviceable and meets code requirements. These activities include assessing for corrosion, wastage, and performing engineering evaluations. The procedure provides reasonable guidance in this area; however, there are many places in the procedure that use the word "should" instead of "shall." The use of "should" allows a choice to be made in an area that involves technical insight. The use of "should" may be used if the technical staff involved with the decision making is highly experienced or reviewed by a highly experienced peer or supervisor.

Root Cause Analysis Report 3.0 Data Analysis

• 47

The RCS System Engineer was interviewed concerning activities related to the spring 2000 refueling outage (12RFO). The Boric Acid Corrosion Control Inspection Checklist (BACCIC) provides a method to both characterize and disposition a Boric Acid leak. The RCS System Engineer was questioned concerning the dispositioning of the BACCIC that was issued to document the Boric Acid on the RPV head. The BACCIC closure process is not well defined.

The RCS System Engineer could not recall how the BACCIC was closed out in 12RFO.

Additionally, there was no evaluation to address the Boric Acid remaining on the RPV head during cycle 13. See CR 2000-0782 in the Condition Report section for additional details concerning this specific issue.

Corrective Actions to Prevent Recurrence This portion of the procedure describes various types of modifications to prevent leaks and/or mitigate the outcome of the leak. The section does not discuss, however, reviewing the maintenance history of the leaking component, reviewing maintenance procedures and work practices, reviewing industry documents such as "EPRI Good Bolting Practices," and reviewing industry issues (Operating Experience) for possible improvements to recurring issues.

The program owner was interviewed on the subject of the Boric Acid Corrosion Control program. The program owner characterized his role in the Boric Acid Corrosion Control program as a caretaker. The program owner coordinates the walk down of Containment and then provides the System Engineer with a copy of the Boric Acid Corrosion Control Inspection Checklist resolution. The program owner also is responsible for maintaining and updating the administrative procedure that controls the program.

Step 6.7.4 of the Boric Acid Corrosion Control procedure (NG-EN-00324) identifies increased responsibilities for the program owner during outages. The procedure describes the program owner functions as: coordinates decontamination and insulation removal for detailed inspection of components, develop plans to resolve leaks (identify and prioritize), coordinate repairs with the pressure test engineer, and provide a status of the repair activities to Outage Management.

The Boric Acid Corrosion Control program does not require the retention of any Boric Acid Corrosion Control Inspection Checklist. The BACCIC contains both the initial assessment of the leak (including corrosion) and the results of any subsequent evaluations. The BACCIC contains signatures for the resolution of the leak, but does not require review or supervisory acceptance.

Neither the Boric Acid Corrosion Control program owner or the RCS System Engineer could produce copies of the completed BACCIC sheets from the 12RFO, therefore an evaluation of the effectiveness of the actual reviews could not be performed.

3.4.3 Davis-Besse Inservice Inspection Program The focus in the ISI program is related to the pressure test performed at the end of a refueling outage (Mode 3 walk down) and performing certain cold (Mode 5) inspections.

In support of this review, the following documents were reviewed:

"* DB-PF-03065 Pressure Test dated 5/20/98 (11 RFO)

"* WO 99-000320-000 (RX VESSEL) Reactor Vessel Bolting VT-2 Examination at the start of 12 RFO

"* DB-PF-03065 Pressure Test dated 5/13/00 (12RFO)

"* DB-PF-03010 RCS Leak and Hrydrostatic Test dated 6/2/00 (12RFO)

"* Inservice Test Plan (IST Plan) Volume II Second Ten Year Interval Pressure Test Program dated 10/27/99 Root Cause Analysis Report 3.0 Data Analysis

• 48 1 I

DB-PF-03065 Pressure Test dated 5/20/98 (11 RFO) was reviewed for program compliance and understanding of the objects being inspected. The person involved with the 1 IRFO Pressure Test that was performed at the conclusion of the outage was interviewed. The person was level 3 certified in NDE. The person described his entering the Reactor Cavity and walking around the RPV head looking for evidence of leakage from the CRDM nozzles. There was no requirement for a hold time at pressure. The test was performed per the requirements of the ISI program and the person demonstrated cognizance of the task.

Work order (WO) 99-000320-000 (RX VESSEL) for 12RFO included performing a VT-2 examination of the installed RPV studs, nuts, and washers. The VT-2 examination could not be performed because of the presence of Boric Acid on the RPV head flange. Condition Report 2000-0781 was written to document the condition of the RPV fasteners. See the Condition Report section for the additional details concerning this specific issue. Final inspection of the fasteners occurred after removal for refueling activities; however, no evidence was presented to document a follow up examination of the RPV head following boric acid removal.

DB-PF-03065 Pressure Test dated 5/13/00 and DB-PF-03010 RCS Leak and Hrydrostatic Test dated 6/2/00 (both from 12RFO) were reviewed. The two inspection reports include CRDM nozzle inspections with the plant at operating temperature and pressure (Mode 3). The examination report dated 5/13/00 indicates that the CRDM nozzles were included in the examination by inspection on top of the service structure looking downward; however, the CRDM nozzle to CRDM flange weld view is obstructed by the CRDM mechanisms and the CRDM flange. It is not clear what is being inspected by this line item. The 6/2/00 dated examination report identifies the CRDM nozzle to RPV head welds were inspected for leakage looking for indications of leakage under the RPV. The report identifies that the inspection for leakage below vessel meets code requirements. No leakage was identified during these inspections.

The ISI Pressure Test Engineer was asked how post boric acid removal inspections are handled specifically related to the RPV head. The person processing the BACCIC would contact the ISI Pressure Test Engineer to have the affected area inspected. If he is not contacted, there is no follow up. The RPV head has not been specifically included in the Boric Acid Corrosion Control program. In addition, the ISI program inspection techniques did not identify CRDM nozzle weld leakage when leakage is now believed to be present.

3.4.4 Evaluation of Condition Report Responses The Root Cause team as part of the investigation, reviewed the completion of a number of Condition Reports. The results of this review are discussed with each Condition Report and then at the end of this section.

PCAQR 94-0295 - Addresses the need to convert commitment A16892 to an Ongoing Commitment. The PCAQR was closed by Regulatory Affairs after they determined that B&W had changed the safety evaluation related to the CRDM nozzle issue to eliminate the need for enhanced inspections. The statement goes on to take credit for the GL 88-05 program to address leaking CRDM nozzles. Discussions with Framatome (purchased B&W) indicate the safety evaluation dated May 1993 was never changed to eliminate the need for enhanced inspections.

PCAQR 96-0551 - Addresses boric acid in several areas on the RPV head and that all of the steps for the Boric Acid Corrosion Control program procedure in effect at that time (NG-EN 00324 rev 1) may not have been followed. The text within the document response takes Root Cause Analysis Report 3.0 Data Analysis 9 49 1

credit for the GL 88-05 walk downs (GL 88-05 walk downs related to the RPV head at DB only address CRDM flange leakage). The response to the PCAQR endorses implementing MOD 94-0025 (inspection holes for Service Structure). The modification is outstanding.

CR 1999-1300 - Addressed the iron oxide deposits on RE4597 AA/BA. The response discusses the results of the radiation monitor action plan. The lab analysis report evaluating the iron oxide states that the iron oxide was not from magna flux powder but appears to be from an iron based component in containment. There does not appear to have been an aggressive effort to locate the actual source of iron oxide after the original proposed source was disproven. The source of the iron oxide was not conclusively determined under this CR.

CR 2000-0782 - (Categorized as "Routine") This CR was written to address the buildup of boric acid on the RPV head. The CR describes the areas affected by the boric acid. A BACCIC sheet 1 was attached to the CR. The BACCIC characterized the leak as "heavy",

red/brown deposits, new leakage not seen during 11 RFO, and recommended a detailed inspection. The CR response does not address the concerns discussed in the CR text or the BACCIC. The responder to the CR believed the leakage was from CRDM flange leakage. It discusses whether there is a need for issuing OE. There is no inspection or evaluation for Boric Acid corrosion and no discussion about other possible sources of boric acid.

CR 2000-1037 - (Categorized as "Routine") This CR was written to document the boric acid on the RPV head and on top of the mirror insulation. Operations Review block contained the following note "This CR should be sent to SYME for resolution. This CR will address the effects of the boron on the head. CR 2000-0782 will address the hardware issue of leaking flanges." The response to the CR states that "Accumulated boron deposits between the RPV head and the thermal insulation was removed during cleaning process performed under WO 00-001846-000. No boric acid induced damage to the head surface was noted during the subsequent inspection." The boric acid being left on the RPV head for cycle 13 was not discussed in the CR and not evaluated.

Condition Reports 2000-0782 and 1037 were characterized as "Routine". Both Condition Reports discuss accumulated boric acid on the RPV head. The Condition Reports were not elevated to the appropriate significance level. In addition reference section 4.1, Davis-Besse Experience Review regarding a discussion of the RC-2, Pressurizer Spray Valve incident in 1998.

3.5 Related Issues After issuance of this Report, a new team was convened to investigate the management and human performance aspects of the damage to the RPV head. That team's investigation results are reported in the Root Cause Analysis Report entitled "Failure to Identify Significant Degradation of the Reactor Pressure Vessel Head" dated 8/13/02. That Report replaces Section 3.5 of Revision 0 of this report, with the exception of the subsections presented below.

3.5.1 RPV Head Inspections Prior to 13RFO, head inspections were not a scheduled activity. The Framatome inspection contract only generated a video tape for retention and use by First Energy. The procedure for conducting the analysis by Davis-Besse personnel was not followed. The lack of analysis was a missed barrier to identification of the leak evaluating the structural integrity of the RPV head.

Root Cause Analysis Report 3.0 Data Analysis

• 50 Root Cause Analysis Report 3.0 Data Analysis

• 50

3.5.2 Restart Readiness There is no standard structure to the Restart Readiness Review done at plant startup. The Restart Readiness Review for 12RFO did not identify the fact that boric acid remained on the head of the RPV. Topics were selected for review that senior management considered significant to the restart of the plant and since as previously discussed, management involvement and awareness of the significance of the boron left on the head was inadequate, it was not a requested topic for review. A structured Restart Readiness Review program that included a review of boric acid issues might have increased management awareness of the issues associated with boric acid residue left on the RPV. Information supplied for the Reactor Coolant System in regards to Restart Readiness Review did not discuss the presence of boric acid deposits that remained on the RPV head following cleaning.

3.6 CAUSAL FACTORS/CONCLUSIONS The Events and Causal Factors Chart (Figure 27) identifies the following undesired events that if prevented would not have resulted in the degradation of RPV head base metal.

"* CRDM nozzle crack initiated

"* CRDM nozzle crack propagation to through wall leak

"* Plant not identifying the through wall crack/leak during outages

"* Plant returned to power with boron on the RPV head after outages

"* Plant not identifying degradation of RPV head base metal during 12RFO The following are the conclusions from the causal factors review and cause determination as identified during the root cause team's data analysis: The physical factors that caused corrosion of the RPV head in the regions of nozzles 2 and 3 are the CRDM nozzle leakage associated with through-wall cracking, followed by boric acid corrosion of the RPV low-alloy steel. In order to be defined as a ROOT CAUSE, the identified cause must be something that can be validated.

Since it is unlikely that sufficient physical evidence is still retrievable to provide this validation, this ROOT CAUSE must be categorized as a PROBABLE CAUSE. Although it is unlikely that the physical evidence will be retrieved to prove what caused the crack(s), the report provides details why PWSCC is concluded to be the damage mechanism.

Since PWSCC of CRDM nozzles is a known degradation mechanism of Alloy 600 materials, and similar corrosion as experienced near nozzle 3 has not been reported from this cause at other nuclear plants, this PROBABLE CAUSE does not provide the explanation for the extent of damage that occurred in the evolution of this condition.

Corrosion damage of the severity experienced at nozzle 3 could only have occurred with an adequate supply of the corrosive element, in combination with environmental conditions conducive to high corrosion rates. The major question surrounding the boric acid's contribution to the extent of damage, and its rate of progression, is when, and from what source boric acid accumulated on the RPV head. In determining this, the team considered the fact that dried boric acid crystals at normal RPV head temperatures do not result in any significant attack of low-alloy steel surfaces. However, there are examples of 'wet' boric acid leaks causing damage to a RPV head.

Other plants are known to have experienced accumulation of boric acid on RPV heads, due primarily to CRDM flange leaks, or conoseal leaks, without damage similar to that of Davis Besse nozzle 3. What made Davis-Besse's situation different were the lengths of the cracks (and associated leaks) and the length of time the leaks went undetected. Ultimately, since the leakage Root Cause Analysis Report 3.0 Data Analysis

• 511

appears to have continued for at least 3 to 4 years, boric acid would have accumulated sufficiently during this period to have provided the necessary environment to begin significant RPV head corrosion. The pre-existence of substantial accumulation of boric acid from other sources, like flange leaks, may have accelerated the corrosion and increased its severity. The defense against damage from leaking boric acid is provided by the station's boric acid corrosion control program. For this condition, an additional ROOT CAUSE was the Less Than Adequate Program/Process, which allowed accumulation of boric acid to remain on the RPV head, and thereby allowed the nozzle leaks to go undetected and uncorrected, in time to prevent damage to the head. (Note - The Root Cause Analysis Report on the "Failure to Identify Significant Degradation of the Reactor Pressure Vessel Head", dated 8/13/02, concluded that failure to comply with specific requirements of the Boric Acid Corrosion Control Program was a ROOT CAUSE.)

The design of the RPV head/service structure makes access to the top of the RPV head difficult for cleaning and inspection. In the original design, only approximately 2 inches of clearance existed between the top of the RPV head and the bottom surface of the permanently installed reflective insulation. It also provided very limited access for maintenance, consisting only of small drainage openings near the bottom of the RPV head, along the periphery, referred to as mouseholes. Deferral of the modification to the service structure for improved access when the modification was first considered resulted in the continued limited ability to prevent significant boric acid accumulations and allow for better visual determination of leakage sources. Since the severity of the damage that occurred to the RPV head is judged to have required years to develop after the initiation of a CRDM nozzle leak, the deferral is considered a CONTRIBUTING CAUSE to the condition. This is also supported by the less than fully successful attempts to clean and inspect the head using alternate methods from the mduseholes, in refueling outages prior to 13RFO.

Environmental factors, such as temperature conditions and radiation dose, also impeded efforts to inspect and clean the RPV head, in that they affected the methods to be used, and the amount of time allocated to perform the tasks.

Boric acid that accumulated on the top of the RPV head over a period of years inhibited the station's ability to confirm visually that neither nozzle leakage nor RPV corrosion was occurring.

Evidence now available shows that leakage from the nozzles began 2 to 4 operating cycles ago.

Acceptance of the condition of boric acid accumulation on the RPV head was a CAUSAL FACTOR. The investigation concluded that some of the early boric acid accumulation was likely due to CRDM flange leakage, rather than nozzle leakage, but the effect of its accumulation on the RPV head would have been the same regardless of its origin. The main effect was to inhibit inspection of the top of the RPV head and associated nozzles. While this preexisting boric acid may have accelerated the initial corrosion, this effect'is considered minor since borated water leaking from the cracks in the CRDM nozzle would have soon produced its own deposits.

Historically, there have been problems with CRDM flange leakage both at Davis-Besse and in the industry. This appears to have obscured the recognition that boric acid accumulation on the RPV head might also be due to nozzle leakage.

Davis-Besse's boric acid corrosion control program specifically includes the CRDM flanges as an area of concern for the RPV. Potential leakage from CRDM nozzles was not a specific consideration of the program.

Root Cause Analysis Report 3.0 Data Analysis 9 521

The potential for significant corrosion of the RPV head as a result of accumulating boric acid and local leakage was not recognized as a safety significant issue by the staff and management of the plant. The lack of understanding of this was a CAUSAL FACTOR, and ultimately resulted in its own root cause investigation.

Containment building related conditions like iron oxide, boric acid and moisture found in radiation monitor filters, boric acid accumulations on the air coolers and boric acid accumulations on the RPV flange were all recognized, but no collective significance was recognized. However, it is not clear if these could have led to the discovery of the problem on the RPV head in time to prevent significant damage.

All three CRDM nozzles that were found to have leaks were located in the center top region of the RPV head. The team was not able to determine how important this location would be to the potential for development of corrosion as a result of an unattended leak, compared to that of a leak that might exist on the steeper sloped regions of the RPV head. It is probable that the close proximity of the RPV head to the overhead insulation layer allowed for boric acid to concentrate and remain in this region. This in turn could have provided a means for accelerated corrosion rates earlier in the process, in that large accumulations of boric acid may have been available to mix with a continuous moisture supply, once it developed from below.

The Industry continues to study why the corrosion at Davis-Besse was more severe than at other B&W design plants such as Oconee 1-3, ANO 1, TMI 1 and Crystal River 3. The team has identified two possible reasons for this:

" First, the cracks in nozzles 2 and 3 at Davis-Besse extend farther above the top of the J groove weld (1.1"- 1.2") than cracks measured at other B&W design plants (<1.0").

Analyses in Section 5 demonstrate that the leak rate is sensitive to the length that the crack extends above the J-groove weld. However, the analyses also show that changes in support provided by the low-alloy steel RPV head material can affect the crack opening displacement and area.

Second, presence of pre-existing boric acid deposits on top of the RPV head may have increased the initial corrosion rates at the exit of the annulus. This theory is supported by test data, which shows that placing insulation around a bolted flange tends to capture the escaping steam and increase the corrosion rate on the heaviest corroded stud, and increase the corrosion rate at other studs around the flange.

In any event, the large-scale corrosion occurred as a result of not detecting and arresting the leakage until advanced symptoms had occurred.

Hoot Cause Analysis Report 3.0 Data Analysis

• 53 I Root Cause Analysis Report 3.0 Data Analysis

• 53 1 I

4.0 Experience Review An experience review was performed and the Davis-Besse and nuclear industry searches identified the following related issues.

4.1 Davis-Besse Experience In 1998, two body-to-bonnet flange nuts on RC-2, Pressurizer Spray Valve, were identified as missing. The CR 1998-0020 root cause analysis report identifies the nuts were missing as a result of boric acid corrosion. Boric acid corrosion resulted due to a packing leak and the nuts being carbon steel versus stainless steel. The root and contributing causes are similar to the conditions described in this root cause report. The investigation into the "Failure to Identify Significant Degradation of the Reactor Pressure Vessel Head", dated 8/13/02, included examination of why corrective actions from that event were not effective in successfully detecting the conditions on the RPV head.

4.2 Nuclear Industry Experience Reference discussions provided throughout the Data Analysis section and Table 7 Nuclear Industry Experience Review Results for a summary of Davis-Besse response to NRC and Institute Of Nuclear Operations (INPO) related documents.

4.3 Conclusions Previous Davis-Besse and nuclear industry experience were not effectively used to prevent the current condition and therefore is considered a casual factor. (Note - The Root Cause Analysis Report on the "Failure to Identify Significant Degradation of the Reactor Pressure Vessel Head",

dated 8/13/02, concluded that this was a ROOT CAUSE.

Root Cause Analysis Report 4.0 Experience Review *, 54 1

5.0 Root Cause Determination This summary presents the collective judgment of the Root Cause Investigative Team based on the data and evidence that has been characterized at this time in the investigation (current to 08/05/02. The data that supports these causes is summarized in Section 3.6. Additional Root Causes are identified in the Root Cause Analysis Report on the "Failure to Identify Significant Degradation of the Reactor Pressure Vessel Head," dated 8/13/02.

5.1 Probable/Root Causes

1. Probable Cause - Less than Adequate Material Selection. PWSCC cracking in the CRDM nozzle interface at the J-groove weld due to material susceptibility in the presence of a suitable environment resulted in:

"* CRDM nozzle crack initiated

"* CRDM nozzle crack propagation to through wall leak

"* Boric acid corrosion of the low-alloy steel RPV head material

2. Root Cause - Less than Adequate Boric Acid Corrosion Control and ISI programs and program implementation regarding the RPV head resulted in:

"* Plant not identifying the through wall crack/leak during outages

"* Plant returned to power with boron on the RPV head after outages

"* Plant not identifying degradation of RPV head base metal during 12RFO

"* Boric acid corrosion of the low-alloy steel RPV head material 5.2 Contributing Causes

1. Less than Adequate Environmental Conditions. Cramped conditions due to the design and high radiation at the RPV head (that remained uncorrected through deferral of proposed modifications) resulted in:

"* Plant not identifying the through wall crack/leak during outages

"* Plant returned to power with boron on the RPV head after outages

"* Plant not identifying degradation of RPV head base metal during 12RFO

"* Boric acid corrosion of the low-alloy steel RPV head material

2. Less than Adequate Maintenance and Testing. Corrective Maintenance did not promptly correct the problem with equipment condition (CRDM flange leakage, especially at nozzle 31 due to its close proximity to nozzle 3). This resulted in:

"* Plant not identifying the through wall crack/leak during outages

"* Plant not identifying degradation of RPV head base metal during 12RFO

"* Boric acid corrosion of the low-alloy steel RPV head material Root Cause Analysis Report 5.0 Root Cause Determination

• 55 I Root Cause Analysis Report 5.0 Root Cause Determination

• 551

6.0 Extent of Condition 6.1 Degradation Mechanism Issues There are two specific degradation mechanisms observed on the RPV head that will be addressed in this extent of condition evaluation. The mechanisms are PWSCC and boric acid corrosion.

The 69 CRDM RPV head penetrations will be evaluated by the RPV head repair team and Engineering ensuring all necessary inspections and examinations are performed on the CRDM nozzles to address extent of condition.

The extent of condition within the containment will be evaluated via walkdowns of structures, systems and components (SCC) within containment. In defining the scope of the walkdowns, three separate criteria were developed to ensure that a bounding evaluation is performed. These three separate criteria are:

(1) Sources: As used in this evaluation, sources are components containing borated water that are considered likely leak locations. The sources are further divided into three groups:

Valves, Threaded/Bolted joints (e.g. thermowells, manways, handholes, reactor coolant pumps), and Alloy 600 components/welds. The Alloy 600 components/welds are susceptible to PWSCC. The intent is to (1) verify there is no additional RCS pressure boundary leakage at Davis-Besse (from Alloy 600 components/welds) and (2) verify that evidence of RCS leakage from any source is properly evaluated (including the potential impact on susceptible materials of the RCS pressure boundary).

(2) Targets: As used in this evaluation, targets are components within the RCS pressure boundary that utilize materials susceptible to boric acid corrosion (carbon and low-alloy steels) as part of the pressure boundary. The targets include the following RCS components: RPV, steam generators, pressurizer, RCPs and individual piping sections.

The intent is to verify that boric acid corrosion has not degraded the RCS pressure boundary. Additionally, although technically not within the RCS pressure boundary, the core flood tanks will be evaluated as targets. It should be noted that certain valves within the RCS pressure boundary may contain susceptible materials but for convenience the valves are listed as sources.

(3) Safety-related (non RCS pressure boundary) SSCs: This criteria refers to safety related SSCs that utilize materials susceptible to boric acid corrosion but are not part of the RCS pressure boundary. The intent is to verify that boric acid corrosion has not adversely impacted the function of safety related SSCs.

Methodology:

(1) Plant Engineering will develop a list of inspection points to address the sources and targets. A table of valves and threaded/bolted connections previously developed for the boric acid corrosion control program mode 5 walkdowns will be used to identify these sources (most of these walkdowns are complete at this date). A list of Alloy 600 components/welds within the RCS pressure boundary has been provided by Design Basis Engineering. A series of inspection points will be needed to adequately address each target. Each target will receive a visual inspection of the external surfaces of installed insulation for evidence of leakage (boric acid residue or bulging of the insulation).

Root Cause Analysis Report 6.0 Extent of Condition

• 56 1

Additionally, each connection point between a target and non-susceptible piping will be inspected to verify that no boric acid has migrated undetected under the insulation to reach a susceptible component. This will require removal of insulation to permit a visual inspection. These inspections (external inspection of the insulation and visual inspection of connection points) will provide adequate assurance that there is no undetected degradation of the RCS pressure boundary. It should be noted that many of the "connection points" are Alloy 600 components/welds that also require inspection as potential sources. These inspections will be performed by VT-2 qualified personnel.

Representative photographs will be made to document the "as found" condition of each inspection point.

(2) The use of visual inspection of Alloy 600 components/welds to detect evidence of throughwall PWSCC requires adequate access to perform a visual inspection.

Additionally the design of component/weld must provide assurance that leakage will be detectable at the surface. This may require additional evaluation of certain nozzles (such as incore nozzles) to verify that a visual inspection is adequate.

(3) In any case where evidence of boric acid deposits exists, the source of the deposits and the leak path must be traced to ensure that there is no wastage of the RCS pressure boundary. It is known that there are boric acid deposits on the insulation on the bottom of the RPV. There are boric acid deposits on the seam between pieces of insulation suggesting that the boric acid came from inside the insulation. It is therefore necessary to perform an inspection under the insulation to determine whether or not there is wastage on the RPV and to determine the source of the boric acid. Due to the difficulty of this task and ALARA considerations, a specific plan is being developed to perform this inspection.

(4) The third category, safety-related (non RCS pressure boundary) SSCs, will be addressed by general area walkdowns of the containment building. These walkdowns will be primarily conducted by Design Engineering Mechanical/Structural (DEMS) and Design Engineering Electrical/Controls (DEEC). The DEMS personnel will focus on safety related SSCs such as structural steel, concrete, pipe supports, control rod guide tube supports, susceptible non RCS piping and coatings. DEECS will focus on cabling, conduit, junction boxes, etc. Plant Engineering will perform inspections of ventilation systems within containment (such as CACs and ductwork). Photographs will be made to document any boric acid deposits/corrosion discovered during these walkdowns.

(5) It is expected that (after proper documentation) existing boric acid deposits will be cleaned up. This will prevent future degradation of susceptible materials due to re wetting of dry boric acid deposits. It will additionally ensure a proper baseline condition for future inspections.

(6) It is also expected that any SSC that has experienced degradation due to boric acid corrosion will be evaluated then reworked or preserved as needed to ensure high standards of material condition and housekeeping.

Note: The investigation into the management and human performance aspects of the head damage determined that the potential exists for additional extent of condition considerations for other systems and programs. For details, refer to Section 7 of the report entitled "Failure to Identify Significant Degradation of the Reactor Pressure Vessel Head",

dated 8/13/02.

Root Cause Analysis Report 6.0 Extent of Condition

• 57 I Root Cause Analysis Report 6.0 Extent of Condition

• 57 1 I

7.0 Recommended Corrective Actions 7.1 Probable/Root Causes Corrective Actions Less than Adequate Material Selection. PWSCC cracking in the CRDM nozzle interface at the J-groove weld due to material susceptibility in the presence of a suitable environment.

1. (Remedial) Develop a plan to monitor for CRDM nozzle leakage. The plan must include steps to repair once leakage is detected. (Plant Engineering)

2. (Remedial) Review Davis-Besse results for CRDM nozzle crack initiation/propagation against the susceptibility model. (Design Basis Engineering, Completion prior to restart)

3. (Remedial) Replace the RPV Closure Head

4. (Preventive) Obtain and install a new RPV Closure Head that does not use alloy 600 for CRDM nozzles.

Less than Adequate Boric Acid Corrosion Control and ISI programs and program implementation regarding the RPV head.

5. (Remedial) An extent of condition review for boric acid damage will be performed to ensure that there are no latent unidentified issues related to boric acid corrosion. The results will be reviewed by the senior management team prior to startup. (Plant Engineering, Completion prior to restart)

6. (Preventive) Perform Self-Assessments of the boric acid corrosion control and ISI programs.

(Plant Engineering Completion prior to restart) The purpose of these Self-Assessments is to evaluate the deficiencies documented in this report. Items to be considered should include:

Boric Acid Corrosion Control Program

"* Incorporating as areas for inspection, industry issues such as CRDM nozzle leakage

"* Incorporating into the inspection plan systems that carry borated water and provide mitigating type functions that help to preserve the Reactor Coolant Pressure Boundary during plant transients and/or accidents

"* Incorporate Boric Acid Corrosion Control Inspection Checklist document retention requirements (retention should be at least several fuel cycles)

"* Incorporating a signature block for the Boric Acid Corrosion Control Program Owner to document his review and concurrence with the disposition activities

"* Review the use of "should" versus "shall" throughout the procedure.

"* Incorporating requirement that boric acid "shall" be removed from affected areas and the affected area inspected to identify any signs of potential corrosion.

"* Incorporating a signature block for the System Engineers supervisor to document his review and concurrence with the disposition activities

"* Review station commitments to determine if other areas or equipment must be included in the Boric Acid Corrosion Control Program

"* Establish a hard link between the Boric Acid Corrosion Control Program and the ISI Program that requires both groups to approve the close out of a Boric Acid Corrosion Control Inspection Checklist.

7.0 Recommended Corrective Actions e 58 1 Root Cause Analysis Report

ISI Program

"* Improve the text descriptions of the areas to be inspected, include sketches of the area and provide a pre-job brief prior to inspecting for bolted connections and Mode 3 leakage during plant heat up

"* Eliminate the conflicting text descriptions that are contained in some of the inspection plans

"* Evaluate the techniques employed for monitoring CRDM nozzle welds for leakage.

"* Reinforce the obligation the ISI program has to protect and preserve the RCS pressure boundary including addressing Boric Acid deposits on the RCS pressure boundary when that specific area was not included in the original inspection plan

"* Establish a hard link between the ISI Program and the Boric Acid Corrosion Control Program that requires both groups to approve the close out of a Boric Acid Corrosion Control Inspection Checklist 7.2 Contributing Causes Corrective Actions Less than Adequate Environmental Conditions. Cramped conditions due to the design and high radiation at the RPV head (that remained uncorrected through deferral of proposed modifications).

1. Provide improved access for inspection and cleaning of the RPV head. (Design Basis Engineering; Completion prior to restart)

Less than Adequate Maintenance and Testing. Corrective Maintenance did not promptly correct the problem with equipment condition (CRDM flange leakage, especially at nozzle 31 due to its close proximity to nozzle 3).

2. No corrective action is required. No CRDM flange leakage was noted during 13RFO. This contributing cause has been resolved. The monitoring for leakage will continue through Preventive Maintenance number 1629 and assessment for corrosion through the Boric Acid Corrosion Control Program.

7.3 Additional Actions

1. It is recommended that a historical Alloy 600 review be conducted. The review should include documents associated with the CRDM nozzles and summarize the results in a FENOC-level program document. Potential items for consideration include:

0 The 1994 EPRI Workshop report 0 The EPRI TR-103696 report referenced in the 1994 Workshop Report.

• EPRI NP-6719-M-SD (Feb 8-10, 1989) 0 March 5, 1996 NEI white paper entitled Alloy 600 RPV Head Penetration PWSCC 1997 EPRI Workshop on PWSCC of Alloy 600 in PWRs Parts 1 & 2 (TR-109138-P2).

• EPRI Workshop on PWSCC Alloy 600 in PWRs, 2/14-16/2000, St. Pete Beach.

• EPRI MRP Alloy 600 Industry Workshop, 6/13-6/14/2001, Atlanta, Report 1006278.

• B&WOG Materials Committee Report 51-1229638

• Automated Ultrasonic Inside Surface Examinations of Reactor Coolant System Alloy 82/182 Nozzle Welds Performed in Spring 2001: PWR Materials Reliability Project Alloy 600 Issue Task Group, 82/182 Weld Integrity Inspection Committee, EPRI Report 1006225 Root Cause Analysis Report 7.0 Recommended Corrective Actions

• 59 1

Note - Revision 0 of this report had recommended that the results of this review be included in the report. It will be more useful as a separate document to aid in managing issues with alloy 600 components.

2. Revision 0 had recommended that a more extensive CRDM flange leak and repair historical review be conducted and summarized in the Root Cause Analysis Report. With the decision to replace the RPV head, this additional historical review provides no benefit to DBNPS.

The report already contains sufficient information to understand the role of flange leaks in the issue.

3. Revision 0 recommended that the analysis of samples collected from the RPV head wastage root cause investigation be collected and tracked. The following samples had been collected:

"* Four samples of rusty boric acid from initial head investigation following insulation removal

"* Nozzle 3

"* Four samples of deposits including corrosion products from Nozzle Two Removal

"* Nozzle 2 Additionally, the wastage area adjacent to nozzle three was removed from the head and is to be investigated destructively. The sample results are included in Section 3.1.5, Section 3.2.4, Morphology, Stage 3. References 8.2.14 and 8.2.15 are the analysis reports. The destructive testing will be tracked under corrective action CA 02-00891-114.

4. Extensive effort is currently in progress by the MRP to develop a model for how small leaks from PWSCC cracks progress to modest amounts of corrosion such as seen at nozzle 2 and much larger amounts of corrosion as seen at nozzle 3. While the corrosion is obviously due to the boric acid, the exact stages of progression are being assessed. Mechanisms being evaluated include boric acid corrosion, crevice corrosion, impingement, flow accelerated corrosion, low oxygen corrosion, steam cutting, molten boric acid corrosion, etc. This work includes finite element thermal-hydraulic modeling to determine the effect of steam leakage on locally suppressing the metal temperature in the annulus. Revision 0 recognized the possibility that this work could conceivably have affected conclusions in this report. DBNPS has remained in contact with the EPRI work, and has reviewed the draft results, which continue to support all conclusions made in this report. Sections 3.2.1 and 3.2.4 have been updated providing additional clarity on the subject of morphology.

5. Revision 0 recommended a review of the stresses of the CRDM nozzles at both operating conditions and cold conditions. This was to determine based upon the stress review if extended time periods at mode 5 conditions increase the likelihood of PWSCC crack initiation. The results of this review have been included inothe revision to section 3.2.1.

Root Cause Analysis Report 7.0 Recommended Corrective Actions 9 60 1

8.0 References 8.1 Davis-Besse References

1. Davis-Besse 13RFO CRDM Nozzle Examination Report, Revision 1, Framatome ANP UT Report, March 11, 2002.

2. Potential Condition Adverse to Quality Reports 90-0120 Boron Leakage and CRDM Stator Cooling 90-0221 CRDM Flange F-2 Slight Erosion of Outer Gasket Groove 91-0353 Boron on Reactor Vessel Head from Leaking CRDM Flanges 92-0072 CAC Cooler Degraded Below Acceptable Performance 92-0248 Boron Found in Filter RE4597AA 93-0098 Reactor Head Vent Flange Leakage 93-0132 Reactor Coolant Found Leaking from CRD Flanges 93-0175 Service Water Piping to CAC's Have Accumulated Boric Acid 94-0295 TERMS A 16892 Requires Visual Exam of Reactor Vessel Head each Outage 94-0912 Documents CRDM Leakage 94-0974 Documents Scratches and Gouges on Seating Surface Location G-5 94-0975 Document 1/2 Moon Gouge CRDM Flange M-3 94-1338 Westinghouse CRDM part 21 96-0551 Video of CRDM Flanges Shows Evidence of Leakage 96-0650 VT-2 Exam of RCP Stud Shows Evidence of Boric Acid Leakage 96-1018 Info Notice 96-032 Received Concerning Augmented Inspection of Rx Vessel 1998-0649 Inspection Results of Reactor Vessel Head 1998-0650 Video Inspection Results CRDM Nozzle/Head Interface 1998-0824 CAC's 2 and 3 Have Accumulated Boric Acid 1998-1164 Water Collecting in Sample Line for RE4597AA 1998-1885 RC-2 Carbon Steel Nuts 1998-1895 Containment Normal Sump Leakage > 1 GPM 1998-1980 Containment Cooler Plenum Pressure Decreasing

3. Condition Reports 1998-0020 Multiple Problems with RC-2 1999-0372 Containment Rad RE4597AA/AB High Root Cause Analysis Report

• 61

1999-0510 RE4597AA OOS Low Flow 1999-0845 Boric Acid Clumps Room 181 1999-0861 RE4597AA Sample Line Full of Water 1999-0928 Document Increased RE Filter Change Frequency 1999-1300 RE Filter Analysis Results from Southwest Research Institute 1999-1614 LER 1998-009 1999-1098 Issues with DB-OP-01200 RCS Leakage Management 2000-0781 Boric Acid on RV Studs 2000-0782 RV Flange Boric Acid from Weep Holes 2000-0903 Two CRDM Flange Fasteners Fail Preservice Exam.

2000-0994 CRDM Flange F-10 Pitted 2000-0995 CRDM Flange D-10 Pitted 2000-1037 Reactor Head Inspection Indicates Boric Acid Accumulation 2000-1210 CRDM D-10 Out of Plum 2000-1547 Containment Cooler Plenum Pressure Dropped 00-4138 Increased Frequency of Containment Air Cooler Cleaning 01-0039 Step Drop in Containment Air Cooler Plenum Pressure 01-0487 Higher Containment Temperatures 01-0890 RCS Leakage Calculation Data Scatter 01-1110 RE4597BA Filter Change Occurring More Frequently 01-1822 Increasing Frequency of RE4597BA Filter Changeout 01-1857 RCS Leakage Anomalies 01-2012 Circumferential Cracking of Reactor Pressure Vessel Head Penetration Nozzles 01-2769 Containment Wide Range Radiation Element (RE2387) Spiking 01-2795 RE4597BA Alarm 01-2862 Potential Adverse Trend in Unidentified RCS Leakage 01-2936 Unable to Perform RE4597BA/BB Functional by the Technical Specification 01-3025 RCS Leakage 01-3411 Equipment Failure on Detector Saturation during RE4597BA Testing 02-00685 Boron Build Up on Reactor Vessel Head 02-00846 More Boron on Head than Expected 02-00891 Control Rod Drive Nozzle Crack Indication 02-00932 CRDM Nozzle Crack Indications 02-01053 Unexpected Tool Movement Root Cause Analysis Report 8.0 References

• 621

4. Procedures DB-OP-0 1200 Reactor Coolant System Leakage Management (rev 0 thru 3)

DB-OP-06900 Plant Heatup DB-PF-00204 ASME XI Pressure Testing DB-PF-03010 RCS Leakage and RCS Hydrostatic Test DB-PF-03065 Pressure and Augmented Leakage Test NG-EN-00324 Boric Acid Corrosion Control (rev 2)

5. Other Station Documents Davis Besse System Health Report, 4th Quarter 2001 Request For Modification 94-0025 Install Service Structure Inspection Opening Inservice Inspection Plan (ISI Plan) Volume HI Third Ten-Year Interval Pressure Test Program Inservice Inspection Plan (ISI Plan) Volume II Second Ten-Year Interval Pressure Test Program Relief Request RR-A3 Insulated ASME Class 1 and 2 Pressure Retaining Bolted Connections Relief Request RR-A10 ASME Class 1 and 2 Pressure Retaining Bolted Connections System

Description:

• SD-022B Containment Air Cooling System and Recirculation System 0 SD-39A Reactor Coolant System Technical Specifications:

3/4.4.6.1 Reactor Coolant Leakage Detection Systems

• 3/4.4.6.2 Reactor Coolant System Operational Leakage

• 3/4.4.10 Structural Integrity ASME Code Class 1, 2, and 3 Components Updated Safety Analysis Report Sections:

5.1 Reactor Coolant System summary Description

• 5.2 Integrity of Reactor Coolant Pressure Boundary (RCPB) 11.4.4.4.5 Containment Vessel Monitor

• Fig. 5.1-2 Functional Drawing Reactor Coolant System 0 Fig. 5.1-3 Reactor Coolant System and Supporting Structures - Plan 0 Fig. 5.1-4 Reactor Coolant System and Supporting Structures - Plan RWP 2000-5132 Clean Boric Acid from Rx Head 8.2 Vendor References

1. B&WOG Integrated Response to NRC Generic Letter 97-01 Degradation of Control Rod Drive Mechanism Nozzle and Other Vessel Closure Head Penetrations, BAW-2301, Framatome ANP Report, July 1997 Root Cause Analysis Report 8.0 References

• 63 I

2. Framatome ANP Report 51-5001951-01, Alloy 600 PWSCC Susceptibility Model, December 9, 1998 (Proprietary)

3. Oconee I RPV Head Nozzle Leaks presented by Dave Whitaker at EPRI Alloy ITG meeting January 19, 2001

4. Dominion Engineering, Inc. Calculation No. C-5509-00-6 Davis Besse CRDM Leak Rates using ANSYS Crack Opening Area (non-safety related), Revision 0 3/19/2002 (Proprietary)

5. Dominion Engineering, Inc. Calculation No. C-5509-00-7 Davis Besse CRDM Nozzle Crack Opening Displacement Analysis, Revision 0 3/19/2002 (Proprietary)

6. Dominion Engineering, Inc. Calculation No. C-5509-00-5 Leak Rate through Axial Crack in Davis Besse CRDMs (non-safety related), Revision 1 3/19/2002 (Proprietary)

7. BAW-10190P Safety Evaluation for B&W-Design Reactor Vessel Head Control Rod Drive Mechanism Nozzle Cracking (Proprietary)

8. BAW-1019P Addendum 1 External Circumference Crack Growth Analysis for B&W Design Reactor Vessel head CRDM Nozzles (Proprietary)

9. BAW-1019P Addendum 2 Safety Evaluation for Control Rod Drive Mechanism Nozzle J Groove Weld (Proprietary)

10. B&WOG Materials Committee Report 51-1201160-00 Alloy 600 SCC Susceptibility:

Scoping Study of Components at Crystal River 3

11. B&W Report 51-1218440-00 Alloy PWSCC Time-To-Failure Models (Proprietary)

12. B&W Report 51-1219143-00 CRDM Nozzle Characterization (Proprietary)

13. Dominion Engineering, Inc. Calculation No. C-5509-00-7 Volume and Weight of Boric Acid Deposits on Vessel Head.

14. Framatome-ANP report #51-5018613-00, Davis-Besse Reactor Vessel Head Deposit Characterization results, June 2002

15. Framatome-ANP report #51-5018965-00, Davis-Besse Reactor Head Deposit Sample Characterization (Second Batch, Nozzle #2 Removal), July 2002

16. Framatome-ANP report #51-5018376-00, Davis-Besse CRDM Crack Profiles, May 13, 2002.

8.3 NRC References

1. GL 88-05 Boric Acid Corrosion of Carbon Steel Reactor Pressure Boundary Components in PWR Plants

2. GL 97-01 Degradation of CRDM/CEDM Nozzle and Other Vessel Closure Head Penetrations

3. Regulatory Guide 1.45 Reactor Coolant Pressure Boundary Leakage Detection Systems

4. Bulletin 82-2 Degradation of Threaded Fasteners in the Reactor Coolant Pressure Boundary of PWR Plants

5.Bulletin 2001-01 Circumferential Cracking of Reactor Pressure Vessel Head Penetration Nozzles

6. Bulleting 2002-01 Reactor pressure Vessel head Degradation and Reactor Coolant Pressure Boundary Integrity Root Cause Analysis Report 8.0 References o 64

7. IN 80-27 Degradation of Reactor Coolant Pump Studs

8. IN 82-6 Failure of Steam Generator Primary Side Manway Closure Studs

9. IN 86-108 Degradation of RCS Pressure Boundary Resulting From Boric Acid Corrosion

10. IN 86-108 Supplements 1 & 2 Degradation of RCS Pressure Boundary Resulting From Boric Acid Corrosion

11. IN 86-108 Supplement 3 Degradation of RCS Pressure Boundary Resulting From Boric Acid Corrosion

12. IN 90-10 Primary Water Stress Corrosion Cracking (PWSCC) of Inconel 600

13. IN 94-63 Boric Acid Corrosion of Charging Pump Casing Caused by Cladding Cracks

14. IN 96-11 Ingress of Demineralizer Resins Increases Potential for Stress Corrosion Cracking of Control Rod Drive Mechanism Penetrations

15. IN 2001-5 Through-Wall Circumferential Cracking of Reactor Pressure Vessel Head Control Rod Drive Mechanism Penetration Nozzles at Oconee Nuclear Station, Unit 3

16. IN 2000-17 Crack in Weld Area of Reactor Coolant System Hot Leg Piping at V.C. Summer

17. IN 2000-17 Supplement 1 Crack in Weld Area of Reactor Coolant System Hot Leg Piping at V.C. Summer

18. IN 2000-17 Supplement 2 Crack in Weld Area of Reactor Coolant System Hot Leg Piping at V.C. Summer

19. IN 2002-11 Recent Experience with Degradation of Reactor Pressure Vessel Head

20. Safety Evaluation for Potential Reactor Vessel Head Adaptor Tube Cracking, November 19, 1993 8.4 INPO References

1. SOER 81-12 Reactor Coolant Pump Closure Stud Corrosion

2. SOER 84-5 Bolt Degradation or Failure in Nuclear Power Plants

3. SER 46-80 Reactor Coolant Pump Closure Stud Corrosion

4. SER 35-81 Corrosion of Reactor Coolant System Piping

5. SER 11-82 Reactor Coolant Pump Closure Flange Stud Corrosion

6. SER 57-83 Cracking in Stagnant Boric Acid Piping

7. SER 72-83 Damage to Carbon Steel Bolts and Studs on Valves in Small Diameter Piping Caused by Leakage of Borated Water

8. SER 32-84 Contamination of Reactor Coolant System by Magnetite and Sulfates

9. SER 41-85 Containment Spraying Events

10. SER 13-87 Reactor Vessel Stud Corrosion from Primary Coolant Leak

11. SER 31-87 Pressurizer Vessel Corrosion due to Pressurizer Heater Rupture

12. SER 35-87 Non-Isolable Reactor Coolant System Leak Root Cause Analysis Report 8.0 References

• 65 1 I

13. SER 10-89 Reactor Coolant Pump Flange Leak from Loss of Bolt Preload.

Bolts should be checked for preload

14. SER 90-2 Pressurizer Heater Sleeve Cracking

15. SER 20-93 Intergranular Stress Corrosion Cracking in Control Rod Drive Mechanism Penetrations

16. SER 4-01 Recent Events Involving Reactor Coolant System Leakage at Pressurized Water Reactors

17. SEN 6 Boric Acid Corrosion

18. SEN 18 Reactor Vessel Head Corrosion

19. SEN 190 Pressurizer Spray Valve Bonnet Nuts Dissolved by Boric Acid

20. SEN 216 Leakage from Reactor Vessel Nozzle-to-Hot Leg Weld

21. SEN 220 Pressure Boundary Leakage at Palisades. Palisades had a through-wall crack in a CRDM housing

22. O&MR 348 Failure of a Limitorque Operator Stem Nut 8.5 Industry References I. PWSCC of Alloy 600 Materials in PWR Primary System Penetrations, EPRI TR-103696.

(Proprietary)

2. EPRI Technical Report -104748 Boric Acid Corrosion Guidebook (Proprietary)

3. EPRI Technical Report -1000975 Boric Acid Corrosion Guidebook, Revision I (Proprietary)

4. EPRI Technical Report -103696 PWSCC of Alloy 600 Materials in PWR Primary System Penetrations (Proprietary)

5. MRP-44, Part 2, PWR Materials Reliability Program - Interim Alloy 600 Safety Assessments for US PWR Plants, Part 2: Reactor Vessel Top Head Penetrations (Proprietary)

6. EPRI NP-6301 -D, Ductile Fracture Handbook

7. EPRI Technical Report -107621-R 1, Steam Generator Integrity Assessment Guidelines:

Revision 1 (Proprietary)

8. EPRI draft report NP-6864-L, PWR Steam Generator Tube Repair Limits: Technical Support Document for Expansion Zone PWSCC in Roll Transitions

9. MRP crack growth rate report (Proprietary)

10. EPRI NP-7094, Literature Survey of Cracking of Alloy 600 Penetrations

11. EPRI technical report 1006284, Report MRP-48, PWR Materials Reliability Program Response to NRC Bulletin 2001-01, August 2001 (Proprietary) 8.6 Other References

1. V.C. Summer Nuclear Station Root Cause Investigation "A" Hot Let Nozzle Weld Cracks

2. Dominion Engineering Meeting Presentation to NRC technical Staff, May 22, 2002 Root Cause Analysis Report 8.0 References o 661

9.0 Personnel Interviews 9.1 Personnel Interviewed Andrew Siemaszko, current Davis-Besse RCS System Engineer Ed Chimahusky, former Davis-Besse RCS System Engineer Dan Haley, former Davis-Besse RCS System Engineer George Chung, current Davis-Besse Radiation Monitor System Engineer Bob Hovland, former Davis-Besse Radiation Monitor System Engineer Walt Molpus, current Davis-Besse Boric Acid Corrosion Control Program owner Peter Mainhardt, performed Davis-Besse Reactor Vessel Head inspections Jerry Lee, Davis-Besse Leak Program owner Glenn McIntrye, former Davis-Besse Mechanical Systems Supervisor Jim Marley, Davis-Besse System Engineering Pete Seniuk, Davis-Besse ISI Pressure Test Engineer Chuck Daft, Davis-Besse ISI Engineer Mike Shepherd, Davis-Besse ISI Engineer Prasoon Goyal, Davis-Besse B&WOG Material Committee representative Ken Byrd, Davis-Besse Nuclear Engineering (PSA Engineer) Supervisor Rich Edwards, Davis-Besse Chemistry Technologist Bruce Geddes, Davis-Besse Containment Deconing Mark Mclaughlin, Davis-Besse CRDM Project Manager Charles (Steve) Steagall, Davis-Besse VT-2 Inspector Richard Cockrell, Davis-Besse VT-2 Inspector Chuck Ackerman, FENOC Quality Assurance Engineering Supervisor Henry Stevens, FENOC Manager Quality Assurance Dave Lockwood, Davis-Besse Manager Learning Organization and Regulatory Programs Dave Geisen, Davis-Besse Design Basis Engineering Manager Dave Eshelman, former Davis-Besse Plant Engineering Manager Joe Rogers, Davis-Besse Outage Director Scott Coakley, Davis-Besse Outage Director Steve Moffitt, Davis-Besse Director Technical Services John Messina, Davis-Besse Director Work Management John Wood, FENOC Vice President Engineering Services Jim Harris, Framatome 13R Reactor Services Lead Fred Currence, Framatome 13R Reactor Services Lead Mike Hacker, Framatome UT expert Rich Garrison, Framatome CRDM Nozzle Inspection/Repair Manager Ron Pillow, Framatome CRDM Component Engineer Steve Fyfitch, Framatome Metallurgist Cary Bowles. Framatome Rod Emery, Oconee CRDM Engineer Root Cause Analysis Report 9.0 Personnel Interviews

• 67 1

9.2 Personnel Consulted Steve Moffitt, Davis-Besse Director Technical Services John Hickling, EPRI Materials expert Kim Kietzman, EPRI UT expert Chuck Welty, EPRI Director Jeff Gorman, Dominion Engineering PhD Materials expert Chuck Marks, Dominion Engineering PhD Chemistry expert Matt Brown, Radiation Protection Servicemen 9.0 Personnel Interviews e 68 1 Root Cause Analysis Report

10.0 Methodologies Employed Event & Causal Factors Charting Procedure Review/Analysis Difference Analysis Barrier Analysis Possible Cause Analysis Structured Interviewing Root Cause Analysis Report 10.0 Methodologies Employed *, 69 1

A FRAMATOME ANP Table 1. Nozzle 1 NDE Examination Results CRDM Nozzle Ultrasonic Examination Data Sheet Customer:

FENOC Plant:

Davis Besse Unit:

n/a Nozzle:

1 Procedure:

54-151-100-08 CA. FRA-02-002, DB-02-012 Nozzle Dimensions: (in.)

ID: 2.765 OD: 4.06 Thickness:

0.649 Downhill Side of Nozzle (deg.):

183 End of Noz. (in 29.6 3robe Serial No's: Ch 1 2078-01002-OL Ch 6 21GB-01002-45L Axial Scan Start: -6, 15.06 Stop: 360, 29.63 Setup:

1 Ch 2 21GF-01004-30L Ch 7 21GC-01001-55L Files:

T2061_12.36.51 Ch 3 21GA-01004-45L Ch 8 22CD-01001-65L Circ. Scan Start: -5, 19.23 Stop: 360, 29.63 Setup:

2 Ch 4 2623-01002-60S Ch 9 2624-01005-60S Files:

T2061_11.11.08 Oh 5 2623-01002-605 Ch 10 2624-01005-60S Flaw Surface Depth End Point 1 End Point 2 Axial Adjusted Circ. Extent Flaw Flaw Flaw Flaw Flaw Weld Location No.

(ID/OD) to Min Min Max Max Total Min Max Total Length Angle TWD Aspect Orientation

@ Flaw Flaw Tip (in.)

(deg.)

(in.)

(deg.)

(in.)

(deg.)

(deg.)

(in.)

(in.)

(deg.)

(in.)

Ratio Min Max 1

OD 0.29 26.97 133 28.31 128 1.34 50.0 55.0 0.18 1.35 8

0.36 0.27 AXIAL In Weld Region 2

OD 0.24 26.63 115 28.29 113 1.66 68.0 70.0 0.07 1.66 2

0.41 0.24 AXIAL In Weld Region 3

OD 0.63 27.71 51 28.11 53 0.40 132.0 130.0 0.07 0.41 10 0.02 0.05 AXIAL In Weld Region 4

OD TW 26.9 31 28.67 29 1.77 152.0 154.0 0.07 1.77 2

0.65 0.37 AXIAL In Weld Region 5

IZL 6

OD 0.04 27.1 334 28.8 334 1.70 209.0 209.0 0.00 1.70 0

0.61 0.36 AXIAL In Weld Region 7

OD TW 25.95 285 29.43 291 3.48 258.0 252.0 0.21 3.49 3

0.65 0.19 AXIAL In Weld Region 8

OD 0.32 27.58 233 28.45 233 0.87 310.0 310.0 0.00 0.87 0

0.33 0.38 AXIAL In Weld Region 9

OD 0.28 27.6 202 28.35 202 0.75 341.0 341.0 0.00 0.75 0

0.37 0.49 AXIAL In Weld Region 10 OD 0.24 27.64 181 28.86 181 1.22 2.0 2.0 0.00 1.22 0

0.41 0.34 AXIAL In Weld Region 11 12 13 14 15 16 17 Data Loc.

183 213 243 273 303 333 3

33 63 93 123 153 183 Degrees WELD Noz. Loc.

0 30 60 90 120 150 180 210 240 270 300 330 360 Degrees Noz. End 29.60 29.60 29.60 29.60 29.60 29.60 29.60 29.60 29.60 29.60 29.60 29.60 29.60 Inches PROFILE MAX 27.85 27.82 27.89 27.89 27.89 27.89 27.97 27.97 27.93 27.85 27.89 27.82 27.85 Inches MIN.

26.55 26.55 26.67 26.71 26.59 26.40 26.40 26.40 26.44 26.59 26.59 26.59 26.55 Inches Notes:

Adjusted Circ. Extent is relative to downhill side of nozzle;clockwise looking down. TWD is Through-Wall Dimension Comments:

Data was encoded with positive Theta going counterclockwise. Adjusted circ. positions have corrected the position to read clockwise looking down.

Flaw # 5 was identified as an axial flaw using the circ. blade probe but is not confirmed with the rotating UT. Therefore, flaw #5 is not relevant.

Analyzed by:

K.C.Gebetsberger Date:

3/5/02 Analyzed by:

M.G. Hacker Date:

3/5/02 Root Cause Analysis Report Tables

• 70 Root Cause Analysis Report Tables 9 70

Table 2. Nozzle 2 NDE Examination Results CRDM Nozzle Ultrasonic Examination Data Sheet Customer:

FENOC Plant:

Davis Besse Unit:

N/A Nozzle:

2 Procedure:

54-ISI-100-08 CA: FRA-02-002, DB-02-012 Nozzle Dimensions: (in.)

ID: 2.765 OD: 4.06 Thicknes 0.649 Downhill Side of Nozzle (deg.):

315 End of Noz. (in.):

30.78 Probe Serial No.'s: Ch 1 2078-01002-OL Ch 6 21GB-01002-45L Axial Scan Start: -5, 16.1 Stop: 360, 30.77 Setup:

1 Ch 2 21GF-01004-30L Ch 7 21GC-01001-55L Files:

T2061 09.12.19 Ch 3 21GA-01004-45L Ch 8 22CD-01001-65L Circ. Scan Start: 0, 18.95 Stop: 360, 29.52 Setup:

2 Ch 4 2623-01002-60S Ch 9 2624-01005-60S Files:

T2061_07.25.10 Ch 5 2623-01002-60S Ch 10 2624-01005-60S Flaw Surface Depth End Point I End Point 2 Axial Adjusted Circ. Extent Flaw Flaw Flaw Flaw Flaw Weld Location No.

(ID/OD) to Min Min Max Max Total Min Max Total Length Angle TWD Aspect Drientatiot 6 Flaw Flaw Tip (in.)

(deg.)

(in.)

(deg.)

(in.)

(deg.)

(deg.)

(in.)

(in.)

(deg.)

(in.)

Ratio Min I Max 1

OD 0.236 27.46 291.0 29.51 275.0 2.05 24.0 40.0

-0.57 2.13 165 0.41 0.19 AXIAL In Weld Region 2

OD TW 26.59 262.0 30.37 240.0 3.78 53.0 75.0

-0.78 3.86 168 0.65 0.17 AXIAL In Weld Region 3

0.65 1

4 OD TW 26.69 148.0 29.39 141.0 2.70 167.0 174.0

-0.25 2.71 175 0.65 0.24 AXIAL In Weld Region 5

OD 0.33 27.87 130.0 28.7 127.0 0.83 185.0 188.0

-0.11 0.84 173 0.32 0.38 AXIAL In Weld Region 6

OD TW 26.8 67 29.36 78 2.56 248.0 237.0 0.39 2.59 9

0.65 0.25 AXIAL In Weld Region 7

8 OD TW 26.35 32 30.16 61 3.81 283.0 254.0 1.03 3.95 15 0.65 0.16 AXIAL in Weld ReIion 9

10 OD TW 27.39 7

30.35 26 2.96 308.0 289.0 0.67 3.04 13 0.65 0.21 AXIAL In Weld Region 11 OD 0.344 27.9 314 27.75 347 0.15 361.0 328.0 1.17 1.18 83 0.31 0.26 CIRC.

0.1 1

0.1 12 OD 0.572 29.02

.320 29.6 327 0.58 5.0 12.0 0.25 0.63 23 0.08 0.12 AXIAL In Weld Region 13 OD TW 26.6 259.0 29.81 258.1 3.21 101.9 101.0

-0.03 3.21 179 0.65 0.20 AXIAL In Weld Region 14 15 Revision 2 5/4/02 16 17 Data Loc.

315 345 15 45 75 105 135 165 195 225 255 285 315 Degrees WELD Noz. Loc.

0 30 60 90 120 150 180 210 240 270 300 330 360 Degrees Noz, End 30.78 30.78 30.78 30.78 30.78 30.78 30.78 30.78 30.78 30.78 30.78 30.78 30.78 Inches PROFILE MAX.

29.17 29.09 29.02 28.84 28.61 28.49 28.46 28.49 28.76 28.92 29.04 29.14 29.17 Inches MIN.

28.06 27.79 27.36 27.39 27.31 27.16 27.16 27.24 27.36 27.39 27.84 27.89 28.06 Inches Notes:

Adjusted Circ. Extent is relative to downhill side of nozzle;clockwise looking down. TWD is Through-Wall Dimension Comments:

Data was encoded with positive Theta going counterclockwise. Adjusted circ. positions have corrected the position to read clockwise looking down.

Flaws #3, 7,and 9 were identified as axial flaws using the circ. blade probe but are not confirmed with the rotating UT. Therefore, flaws #3, 7, and 9 are not relevant.

Analyzed by:

K.C.Gebetsberger Date:

3/5/2002 Analyzed by:

M.G. Hacker Date:

3/5/2002 Root Cause Analysis Report Tables e 71

A FRAMATOM E ANP Table 3. Nozzle 3 NDE Examination Results CRDM Nozzle Ultrasonic Examination Data Sheet Customer:

FENOC Plant:

Davis Besse Unit:

n/a Nozzle:

3 Procedure:

54-11-100-08 CA FRA-02-002, DB-02-012 Nozzle Dimensions: (in.)

ID: 2.765 OD: 4.06 Thickness:

0.649 Downhill Side of Nozzle (deg.):

150 End of Noz. (in 30.75 Drobe Serial No.'s: Ch 1 2078-01002-OL Ch 6 21GB-01002-45L Axial Scan Start: -5, 16 Stop: 360, 30.81 Setup:

1 Ch 2 21GF-01004-30L Ch 7 21GC-01001-55L Files:

T2061_15.39.37 Ch 3 21GA-01004-45L Ch 8 22CD-01001-65L Circ. Scan Start: 6, 20.3 Stop: 360, 30.88 Setup:

2 Ch 4 2623-01002-60S Ch 9 2624-01005-60S Files:

T2061_14.09.39 Oh 5 2623-01002-60S Ch 10 2624-01005-60S Flaw Surface Depth End Point I End Point 2 Axial Adjusted Circ. Extent Flaw Flaw Flaw Flaw Flaw Weld Location No.

(IDIOD) to Min Min Max Max Total Min Max Total Length Angle TWD Aspect Orientation

@ Flaw Flaw Tip (in.)

(deg.)

(in.)

(deg.)

(in.)

(deg.)

(deg.)

(in.)

(in.)

(deg.)

(in.)

Ratio Mi Max 1

OD TW 26.6 151.0 30.68 156.0 4.08 1.0 6.0 0.18 4.08 2

0.65 0.16 AXIAL In Weld Region 2

3 OD 0.234 28.07 275.0 29.19 280.0 1.12 125.0 130.0 0.18 1.13 9

0.42 0.37 AXIAL In Weld Region 4

OD TW 26.07 319.0 29.89 330.0 3.82 169.0 180.0 0.39 3.84 6

0.65 0.17 AXIAL In Weld Region 5

OD 0.212 28.4 136.0 29.46 143.0 1.06 346.0 353.0 0.25 1.09 13 0.44 0.40 AXIAL In Weld Region 6

7 8

9 10 11 12 13 14 15 16 17 Data Loc.

150 180 210 240 270 300 330 360 30 60 90 120 150 Degrees WELD Noz. Loc.

0 30 60 90 120 150 180 210 240 270 300 330 360 Degrees Noz. End 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 Inches PROFILE MAX 29.08 29.08 29.02 28.70 28.54 28.38 28.35 28.41 28.63 28.80 28.96 29.02 29.08 Inches MIN.

27.83 27.77 27.51 27.23 27.07 26.94 26.95 27.00 27.19 27.42 27.67 27.80 27.83 Inches Notes:

Adjusted Circ. Extent is relative to downhill side of nozzle;clockwise looking down. TWD is Through-Wall Dimension Comments:

These are axial flaws that extend from below the weld region into the weld region.

They were also detected with the circ. blade probe.

Flaw # 2 was identified as an axial flaw using the circ. blade probe but is not confirmed with the rotating UT. Therefore, flaw #2 is not relevant.

Analyzed by:

K.C. Gebetsberger Date:

3/5/02 Analyzed by:

M.G. Hacker Date:

3/5/02 Root Cause Analysis Report Tables

• 72 Root Cause Analysis Report Tables e 72

A F RAMATOM E ANIP Table 4. Nozzle 5 NDE Examination Results CRDM NazzlA tlltrn~onir. Fyxnminntinn flntz RhA_*t Customer:

FENOC Plant:

Davis Besse Unit:

N/A Nozzle:

5 Procedure:

54-1S1-100-08 CA FRA-02-002, DB-02-012 Nozzle Dimensions: (in.)

ID: 2.765 OD: 4.06 Thickness:

0.649 Downhill Side of Nozzle (deg.):

320 End of Noz. (in 30.75

robe Serial No.'s: Ch 1 2078-01002-OL Ch 6 21GB-01002-45L Axial Scan Start: -4, 16.11 Stop: 360, 30.78 Setup:

1 Ch 2 21GF-01004-30L Ch 7 21GC-01001-55L Files:

T2061 18.30.12 Ch 3 21GA-01004-45L Ch 8 22CD-01001-65L Circ. Scan Start: -6, 19 Stop: 360, 29.41 Setup:

2 Ch 4 2623-01002-60S Ch 9 2624-01005-60S Files:

T2061_16.53.38 Ch 5 2623-01002-60S Ch 10 2624-01005-60S Flaw Surface Depth End Point I End Point 2 Axial Adjusted Circ. Extent Flaw Flaw Flaw Flaw Flaw Weld Location No.

(IDIOD) to Min Min Max Max Total Min Max Total Length Angle TWD Aspect Orientation

@ Flaw Flaw Tip (in.)

(deg.)

(in.)

(deg.)

(in.)

(deg.)

(deg.)

(in.)

(in.)

(deg.)

(in.)

Ratio I Min Max 1

OD 0.2 28.44 274.0 29.69 271.0 1.25 274.0 271.0

-0.11 1.25 5

0.45 0.36 AXIAL In Weld Region 2

3 4

5 6

7 8

9 10 12 13 14 15 16 17 Data Loc.

320 350 20 50 80 110 140 170 200 230 260 290 320 Degrees WELD Noz. Loc.

0 30 60 90 120 150 180 210 240 270 300 330 360 Degrees Noz. End 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 30.75 Inches PROFILE MAX.

29.10 29.07 29.07 28.91 28.73 28.60 28.52 28.40 28.46 28.67 28.91 28.96 29.10 Inches M IN.

27.90 27.89 27.89 27.68 27.39 27.21 27.13 27.10 27.10 27.21 27.68 127.91 27.90 Inches Notes:

Adjusted Circ. Extent is relative to downhill side of nozzle;clockwise looking down. TWD is Through-Wall Dimension Comments:

This is an axial flaw that extends into the weld region. This flaw was also detected sing the circ. blade probe.

.Analyzed by:

K. C. Gebetsberger Date:

3/5/02 Analyzed by:

M.G.Hacker Date:

3/5/02 Root Cause Analysis Report Tables

• 73 Root Cause Analysis Report Tables e 73

A FRAMATOME ANP Table 5. Nozzle 47 NDE Examination Results CRDM Nozzle Ultrasonic Examination Data Sheet Customer:

FENOC Plant:

Davis Besse Unit:

N/A Nozzle:

47 Procedure:

54-ISI-100-08 CA FRA-02-002, DB-02-012 Nozzle Dimensions: (in.)

ID: 2.765 OD: 4.06 Thickness:

0.649 Downhill Side of Nozzle (deg.):

143 End of Noz. (in 45.9

robe Serial No.'s: Ch 1 2078-01002-OL Ch 6 21GB-01002-45L Axial Scan Start: -6, 29.9 Stop: 360, 45.9 Setup:

1 Ch 2 21GF-01004-30L Ch 7 21GC-01001-55L Files:

T2062_01.40.41 Ch 3 21GA-01004-45L Ch 8 22CD-01001-65L Circ. Scan Start: -6, 34 Stop: 360, 46 Setup:

2 Ch 4 2623-01002-60S Ch 9 2624-01005-60S Files:

T2062_23.53.48 Ch 5 2623-01002-60S Ch 10 2624-01005-60S Flaw Surface Depth End Point 1 End Point 2 Axial Adjusted Circ. Extent Flaw Flaw Flaw Flaw Flaw Weld Location No.

(ID/OD) to Min Min Max Max Total Min Max Total Length Angle TWD Aspect Orientation

@ Flaw Flaw Tip (in.)

(deg.)

(in.)

(deg.)

(in.)

(deg.)

(deg.)

(in.)

(in.)

(deg.)

(in.)

Ratio Min Max 1

2 3

OD 0.06 43.23 181.0 45 202.0 1.77 38.0 59.0 0.74 1.92 23 0.59 0.31 AXIAL In Weld Region 4

5 6

7 8

9 10 11 12 13 14 15 16 17 Data Loc.

143 173 203 233 263 293 323 353 23 53 83 113 143 Degrees WELD Noz. Loc.

0 30 60 90 120 150 180 210 240 270 300 330 360 Degrees Noz. End 45.90 45.90 45.90 45.90 45.90 45.90 45.90 45.90 45.90 45.90 45.90 45.90 45.90 Inches PROFILE MAX 44.48 44.58 44.10 43.40 42.67 42.10 41.75 41.94 42.58 43.31 44.01 44.42 44.48 Inches MIN.

43.10 42.96 42.42 41.49 40.54 39.62 39.39 39.49 40.19 41.40 42.38 42.96 43.10 Inches Notes:

Adjusted Circ. Extent is relative to downhill side of nozzle;clockwise looking down. TWD is Through-Wall Dimension Comments:

Flaw #3 is an axtal flaw that extends into the weld region. This flaw was also detected with the circ. blade probe.

Flaws #1 and #2 were identified with the circ. blade probe but were determined not to be valid detections with the rotating UT. Nozzle ovality in the location of these indications is the source of these false indications. Flaw #4 was detected with the rotating UT but it is located in the J-groove weld fillet and outside the nozzle wall and is therefore outside the scope of this procedure.

Analyzed by:

K. C. Gebetsberger Date:

3/4/02 Analyzed by:

M. G. Hacker Date:

3/4/02 Root Cause Analysis Report Tables

• 74 Root Cause Analysis Report Tables

• 74

Table 6. Comparison of Davis-Besse to Other B&W Design Plants Parameter Oconee 1 Oconee 2 Oconee 3 ANO-1 Davis-Besse TMI-1 Crystal River 3 NSSS*

B&W B&W B&W B&W B&W B&W B&W Material Supplier*

BWTP BWTP BWTP BWTP BWTP BWTP BWTP Head Fabricator*

B&W B&W B&W B&W B&W B&W B&W Design Nozzle Fit (mils)*

0.5 -1.5 0.5-1.5 0.5 -1.5 0.5-1.5 0.5-1.5 0.5-1.5 0.5 -1.5 EFPYs Through Feb 2001*

20.4 20.3 20.1 8.0 14.7 16.8 14.9 Head Temp (OF)*

602 602 602 602 605 601 601 EFPYs Normalized to 600°F*

22.1 22.0 21.7 19.5 17.9 17.5 15.6 EFPYs to Reach Oconee 3*

-0.3

-0.2 0.0 2.1 3.1 4.1 5.9 Access Ports in Lower Shroud Yes Yes Yes No No Yes Yes Boric Acid on Head Small Amount Small Amount Large Amount Some Large Amount Some Some Prior to 2000 Number of CRDM Nozzles 69 69 69 69 69 69 69

- With Leaks 1

4 14 1

3 5

1

-Leaks & Circ Cracks 0

1 4

0 1

0 1

S......................

I........................

I.........................

e......

4................

-With Heat M3935 0

0 68 1

5 0

0 Number of T/C Nozzles 8

0 0

0 0

8 0

- With Leaks 5 confirmed N/A N/A N/A N/A 8

N/A Counterbore at Bottom of Yes Yes Yes Yes No CRDM Nozzles Yes Yes As-Built Fit Range for Clarnce Clearance to Clearance to 0.4-0.7 0.-2.0 Leaking Nozzles (mils) ea 1.4 Interference 1.0 Interference Wastage at Leaks No No No No Yes No No

• Data from MRP-48, PWR Materials Reliability Program-Response to NRC Bulletin 2001-01 (EFPY data as of February 2001).

Root Cause Analysis Report Tables e 75

Table 7. Nuclear Industry Experience Review Results NRC Documents Document Davis-Besse Response/Actions Comments Bulletin 82-2, Degradation of Threaded Maintenance procedures for threaded In 1987, an NRC inspection of the Bulletin Fasteners in the Reactor Coolant Pressure fasteners were written.

concluded there were no violations or Boundary of PWR Plants.

0 Inspection and cleaning of fasteners was deviations.

"* Implement maintenance procedures for added to the maintenance procedure.

threaded fasteners.

9 Ten CRDM flanges and OTSG lower

"* Inspect and clean fasteners when primary hand holes have leaked.

removed.

CRD reactor vessel nozzle bolts and

• List RCS closures that have leaked.

OTSG manway & hold down bolts are

• List where thread lubricants and lubricated.

Furminite was used on RCS fasteners.

0 One of the RCS cold leg thermowells was Furminited.

GL 88-5, Boric Acid Corrosion of Carbon The Davis-Besse program consists of several Although CRDM flanges are inspected, Steel Reactor Pressure Boundary programs and procedures.

CRDM nozzles are not specifically listed.

Components in PWR Plants. The document 9 Leakage Management Program, which requested assurances that Davis-Besse have a identifies and the location of the leakage During an audit of the boric acid corrosion program to ensure that boric acid corrosion and evaluates the boric acid concern.

prevention program, the NRC found the does not lead to degradation of the RCS 9 Shutdown procedure, which requires a program met the intent of the Generic Letter.

boundary. The program should include:

walkdown of containment valves and a Implementing procedures still need to be

• Listing where small leaks could cause general containment walkdown.

made effective. Engineers should be trained.

degradation,

• ASME Section XI Inservice Pressure Inspections should be documented.

• Procedures for finding small leaks, Test, which performs a visual inspection

• Evaluating the impact of leaks, &

to look for discoloration. If boric acid Preventive actions for corrosion, residue is identified, find the source, determine the extent, and repair.

• CRD Flanges are inspected each refueling. Gaskets are replaced on leaking joints. This will be incorporated into the PM program.

Root Cause Analysis Report Tables 9 76

Document Davis-Besse Response/Actions Comments

"* Periodic fastener inspection as a result of the IE Bulletin 82-2, Degradation of Threaded Fasteners in the RC Pressure Boundary of PWRs.

"* Limited Thermographic Inspections in containment to detect steam leaks as part of the current outage.

"* Live Load Packing of Valves to reduce stem leakage may be used if it proves a viable method.

Davis-Besse will implement a Boric Acid Corrosion Program to include all the requirements of GL 88-5 in 1989.

Generic Letter 97-1, Degradation of Control The response is in B&WOG Topical Report, Responses to requests for additional Rod Drive Mechanism Nozzle and Other "B&WOG Integrated Response to Generic information were answered by NEI for the Vessel Closure Head Penetrations. An Letter 97-01: Degradation of Control Rod industry. The response emphasized that the integrated, long-term program, which Drive Mechanism Nozzle and Other Vessel integrated program is an ongoing program includes periodic inspections and monitoring, Closure Head Penetrations," BAW -2301.

that will be implemented in conjunction with is necessary. The following is requested:

EPRI, the PWR Owners Groups, the

"* Results of CRDM nozzle inspections.

Inspections for B&W plants will be participating utilities, and the Material

"* Schedule for subsequent CRDM nozzle preformed based on susceptibility.

Reliability Project's Subcommittee on Alloy inspections.

600.

"* The scope of subsequent inspections.

There have been no resin bed intrusions at Or justify why no inspection is needed.

B&W plants.

• A description of resin bed intrusions.

NEI proposed an integrated inspection program based on susceptibility.

IN 80-27, Degradation of Reactor Coolant An inspection of the Davis-Besse studs in Also described in SOER 81-12 and SER 46 Pump Studs. Several reactor coolant pump 1980 revealed no corrosion in the studs for 3

80.

studs incurred boric acid wastage as a result of 4 RCPs. A small amount of rust and boric of leaks in the pump flanges. If undetected, acid around the studs for 1 RCP was from an Root Cause Analysis Report Tables

• 77

Document Davis-Besse Response/Actions Comments corrosion of RCP studs could cause the loss overhead valve leak, which was fixed of the RCS pressure boundary. To detect, previously. A work order was issued to clean supplemental visual examinations and the area.

instrumented leak detection are needed.

Undetected wastage could occur in other There is a drain between the inner and outer components.

gaskets which goes to the containment sump, but there is no monitoring of the leakage and the drain valve is normally closed.

IN 82-6, Failure of Steam Generator Primary Response was deferred to the response to Side Manway Closure Studs. There have NRC Bulletin 82-2 Degradation of Threaded been a significant number of failed or Fasteners in the RC Pressure Boundary of degraded bolts and studs due to stress PWR plants.

corrosion cracking and corrosion wastage that are difficult to detect.

IN 86-108, Degradation of RCS Pressure The Davis-Besse HPI line geometry is The response is limited and fails to recognize Boundary Resulting From Boric Acid different.

the larger issue of boric acid corrosion.

Corrosion. Boric acid from a leaking valve caused wastage of a carbon steel HPI line.

Provisions regarding iron oxide stains on The primary defense is to minimize leaks, RCS piping insulation will be included in the detect and stop leaks soon after they start, ASME Section XI Inservice Pressure Tests and promptly clean up any boric acid residue.

procedure.

Detection of leaks will be enhanced by an evaluation of any iron oxide stains on insulation.

IN 86-108 Supplements 1 & 2, Degradation During shutdowns, a mode 3 containment The mode 3 walkdowns cannot inspect the of RCS Pressure Boundary Resulting From walkdown will look for any buildup of boron reactor head.

Boric Acid Corrosion. Supplement 1: Boric on piping or valves and to notify engineering acid corrosion/wastage on the head of the of any of any potential problem areas.

Turkey Point 4 reactor and boric acid crystals in the CRDM cooling ducts. Small RCS An RCS leakage management policy leaks can concentrate the boric acid and maintains RCS leakage as low as possible Root Cause Analysis Report Tables

• 78

Document Davis-Besse Response/Actions Comments rapidly corrode carbon steel. Supplement 2:

and identifies and evaluates corrosion Boric acid corrosion/wastage on the head of concerns.

the Salem 2 reactor and failure of a shutdown cooling valve bolts due to boric acid corrosion. The INs recommended that inspection programs be reviewed to ensure adequate monitoring.

IN 90-10, Primary Water Stress Corrosion BWOG studied the problem in B&W This was evaluated along with SER 2-90 by Cracking (PWSCC) of Inconel 600. Plants Document 51-1201160-00. We expected the RFA 90-83 1. However, the NRC made the should review their Inconel 600 applications BWOG to recommend additional inspections, issue much broader than INPO.

and implement an augmented inspection The study demonstrates that the issue of program.

Inconel 600 applications is adequately We deferred our evaluation to the BWOG, reviewed and inspections are being which is summarized in the "Other formulated. Therefore, the intent of the IN is Documents" below.

met.

IN 86-108 Supplement 3, Degradation of The Boric Acid Corrosion Control program The response just make the statement that the RCS Pressure Boundary Resulting From addresses the issue.

Boric Acid Corrosion Control program Boric Acid Corrosion. Issued in 1995.

covers the concern but provides no basis for Corrosion problems at Calvert Cliffs and the conclusion.

TMI had earlier indication of leakage and in both cases, boric acid leakage was not immediately cleaned and stopped. The primary defense is minimize leakage, detect and stop leaks, & promptly clean the residue.

IN 94-63, Boric Acid Corrosion of Charging This is not applicable to Davis-Besse since The Davis-Besse evaluation was narrowly Pump Casing Caused by Cladding Cracks.

the Make-up Pumps and HPI pumps are solid focused on the charging pump and not on Although boric acid wastage occurs slowly, stainless steel.

boric acid corrosion in general.

an attack can eventually lead to significant thinning of carbon steel cladding and possibly leakage. Corrosion of the base metal is easy to find though visual inspection.

Root Cause Analysis Report Tables e 79

Document Davis-Besse Response/Actions Comments IN 96-11, Ingress of Demineralizer Resins The response deals with intrusion of The Davis-Besse evaluation was narrowly Increases Potential for Stress Corrosion demineralizer resins in the RCS. Davis-focused on the resin intrusion and did not Cracking of Control Rod Drive Mechanism Besse has had no resin intrusion. PWSCC address PWSCC.

Penetrations. EPRI is researching ways to probability is low because of water chemistry mitigate PWSCC and developed a and actions would be taken on high sulfate demonstration program to ensure that levels.

inspections performed on CRDM penetrations are highly reliable in detecting and determining the size of flaws. Resin intrusion into the RCS will cause circumferential Intergranular Stress Corrosion Cracking. There is a high probability that CRDM penetrations contain cracks caused by PWSCC.

IN 2000-17, Crack in Weld Area of Reactor This is preliminary information and no action Although the IN only contained information Coolant System Hot Leg Piping at V.C.

can be taken at this time. The information and gave no recommendation on what could Summer. A crack was found on a weld on a was adequately distributed for current needs.

be done, it may have been more appropriate hot leg pipe. Elevated leakage and radiation This information will be added to the final to have the system experts make that call.

was not seen. It was found by discovering OE evaluation.

boric acid. When the root cause is See the V.C. Summer Root Cause in the determined, a supplement will be issued.

"Other Documents" section.

IN 2000-17 Supplement 1, Crack in Weld This is preliminary information and no action Although the IN only contained information Area of Reactor Coolant System Hot Leg can be taken at this time. The information and gave no recommendation on what could Piping at V.C. Summer. A multi-disciplined was adequately distributed for current needs.

be done, it may have been more appropriate team will conduct a root cause. A foreign This information will be added to the final to have the system experts make that call.

plant also had crack indications in the hot OE evaluation.

leg. When the root cause is determined, See the V.C. Summer Root Cause in the another supplement will be issued.

"Other Documents" section.

IN 2000-17 Supplement 2, Crack in Weld The issue is still under evaluation and we The OE program incorrectly assumed that Area of Reactor Coolant System Hot Leg expect further information to be released by more information would be issued.

Piping at V.C. Summer. The crack was the NRC. The only action needed at this time However, the V.C. Summer Root Cause Root Cause Analysis Report Tables e 80

Document Davis-Besse Response/Actions Comments caused by PWSCC. Extensive weld repairs is information distribution. When the final Evaluation was complete. Yet it wasn't were a contributing cause. The V.C. Summer document is evaluated, this information will obvious to the review committee that this root cause was thorough and concluded it be attached.

supplement listed the generic causes. It may was PWSCC. Welding met code have been more appropriate to have the requirements. Leak detection enhancements system experts review the information.

will be made. The following generic issues need to be addressed.

There are several references to additional

"* NDE failed to detect the cracks.

problems, but there was no effort to seek out

"* ASME code allows multiple weld repairs.

the additional information.

• Weaknesses in leak detection systems.

• Applicability of "Leak before break" See the V.C. Summer Root Cause in the analysis.

"Other Documents" section.

IN 2001-5, Through-Wall Circumferential Response was deferred to the response to The response to the Information Notice failed Cracking of Reactor Pressure Vessel Head NRC Bulletin 2001-1.

to follow the OE program. See CR 2001 Control Rod Drive Mechanism Penetration 2997.

Nozzles at Oconee Nuclear Station, Unit 3.

INPO SEE-IN Documents Document Davis-Besse Response/Actions Comments SOER 81-12, Reactor Coolant Pump Closure The DB response said that the RCP studs This SOER was last reviewed in March Stud Corrosion. The SOER noted that were inspected in 1980 and no damage was 2001.

insulation reduces the likelihood of found. Boric acid was found and cleaned.

discovering leakage/boric acid deposits and The SOER and evaluation is very focused on the insulation may have caused retention of We have a procedure and PM to inspect the RCP studs. However, it brings out the facts borated water and increased the possibility of studs. Both perform a visual examination that boric acid corrosion can be rapid and corrosion. The SOER noted that the rate of and generate a Material Deficiency if insulation needs to be removed to find boric corrosion increased when boric acid deposits anything relevant is found.

acid deposits.

are wetted and present inspection frequencies Root Cause Analysis Report Tables

• 81

Document Davis-Besse Response/Actions Comments are not adequate for timely detection.

The response says that if boric acid deposits Also described in IN 80-27 and SER 46-80.

Recommended a visual inspection of the are found, areas will be inspected & deposits RCP closure studs. Recommended removal removed according to NG-EN-324.

of residual leakage and boron deposits from the closure flange area.

SOER 84-5, Bolt Degradation or Failure in Practices are in place to identify and fix A Green SOER that is no on the INPO 97-10 Nuclear Power Plants. The SOER noted that leaks.

list. This SOER was last reviewed in late fastener failures are occurring due to boric 1987.

acid corrosion and stress corrosion cracking.

We perform walkdowns in. containment to The SOER recommended that we ensure find and fix leaks (if possible) to minimize The response many times cited routine prompt repair of leaking joints with boric boric acid damage.

inspections or walkdowns that we perform, acid deposits.

but those can't identify leaks in containment.

Work requests for boric acid leaks receive higher priority due to radiation and The response still didn't seem to recognize contamination corrosion concerns.

the importance of boric acid corrosion. The response says boric acid leaks are repaired because of radiation and contamination concerns, not because of corrosion concerns.

Based on the lack of action to fix RC2, we did not promptly repair the leaking joint with boric acid deposits.

SER 46-80, Reactor Coolant Pump Closure No specific DB response was found.

This issue was subsequently described in Stud Corrosion. The SER noted that leaking SOER 81-12. Also described in IN 80-27.

gasketed joints (e.g., Control rod drives &

reactor vessel head) might be affected by boric acid attack. Although closure studs are subject to inservice inspections, corrosion damage was not detected.

SER 35-81, Corrosion of Reactor Coolant No DB response was found.

System Piping. The SER says corrosive Root Cause Analysis Report Tables e 82

Document Davis-Besse Response/Actions Comments attack could reduce primary boundary integrity. INPO will continue to evaluate this event.

SER 11-82, Reactor Coolant Pump Closure No DB response was found.

Flange Stud Corrosion. The repeat of stud corrosion and the amount of corrosion re inforces the importance of frequent visual inspections and removal of boric acid deposits - as described in SOER 81-12.

SER 57-83, Cracking in Stagnant Boric Acid Seven line handwritten response saying boric Piping. Many cracking incidents have acid piping is inspected in the ISI program occurred.

and this hasn't happened here. The SER was distributed for information.

SER 72-83, Damage to Carbon Steel Bolts The evaluation was deferred to SOER 84-5.

In previous responses, we've claimed that and Studs on Valves in Small Diameter The SER was distributed for information, boric acid piping is inspected during by the Piping Caused by Leakage of Borated Water.

ISI program, yet this has warned us that the When scheduling maintenance, take boric ISI is not adequate to detect these problems.

acid corrosion rates into account. Ten year ISI may not be frequent enough.

SER 32-84, Contamination of Reactor No DB response was found.

Although this discusses RCS leakage, this Coolant System by Magnetite and Sulfates.

doesn't appear to provide any insight to this issue.

SER 41-85, Containment Spraying Events.

DB recognizes that prompt clean up is The evaluation failed to address the problems Prompt clean up of boric acid reduces essential to ensuring the integrity of carbon with insulation.

corrosion. Boric Acid solutions in insulation steel. The ability to detect and clean up each are hard to remove, boric acid spill will depend on the The erosion/corrosion program response has circumstances.

no bearing on the concern.

An Erosion/corrosion program will find degradation.

SER 13-87, Reactor Vessel Stud Corrosion We inspect reactor head area by operations The body of the SER was focused on Root Cause Analysis Report Tables e 83

Document Davis-Besse Response/Actions Comments from Primary Coolant Leak. Inspect reactor walkdown during shutdowns.

fasteners and said that no structural integrity head for boron during all planned and was effected. This may have influenced the unplanned outages. The 1 GPM T.S. won't During startups, we inspect containment, evaluators against concerns about what is detect small leaks.

happening in the service structure.

Operations walkdowns would not be able to detect boric acid on the head. At best, this evaluation may have assumed that operations could see any boric acid draining down onto the reactor head studs.

The evaluation failed to understand that a detailed internal inspection was needed.

During the times cited in the evaluation, this could not have been done.

SER 31-87, Pressurizer Vessel Corrosion Evaluation of boric acid damage was The evaluation missed the point that the due to Pressurizer Heater Rupture. The SER deferred to the evaluation of SER 13-87.

insulation needs to be removed to find the noted that Boric Acid corroded a 1/2 inch Evaluation of inspection for boric acid was damage. There was no effort made to try to diameter, 3/4 inch deep hole in the lower deferred to the evaluation of SER 13-87.

highlight this concern.

pressurizer head and could only be seen with the insulation removed. Boric acid corrosion Since maintenance will walk down and causes damage and extends outages. Rates determine repairs, boric acid damage will be can be up to 1.65 inches per year. Small found and fixed.

leaks can cause severe damage. Periodic inspections are needed to identify leaks.

Sources of leaks need to be repaired.

SER 35-87, Non-Isolable Reactor Coolant Spec M-452Q considers component The response was superficial and missed the System Leak. Make sure that resistant specifications.

point, but has little bearing on this issue.

material is used for valves. If a valve in the boric acid system fails, consider possible Maintenance reports as found conditions to boric acid causes.

the plant engineers. They would recommend corrective actions.

Root Cause Analysis Report Tables

• 84

Document Davis-Besse Response/Actions Comments The SER was distributed for information.

SER 10-89, Reactor Coolant Pump Flange Preload was checked due to other reasons The focus and recommendations are on RCP Leak from Loss of Bolt Preload.

Bolts earlier.

stud tightness and not boric acid corrosion, should be checked for preload.

which is referenced back to SOER 81-12 &

SER 13-87.

SER 90-2, Pressurizer Heater Sleeve The overall evaluation was deferred to the We were given the right answers, it's Cracking. Inspect Inconel 600 pressurizer BWOG Material Committee "to monitor this unknown if we recognized it and used it.

heater sleeves for leakage.

issue to conclusion."

This is a very interesting issue. NRC IN 90 10 was also issued on Inconel 600 Stress The SER was distributed for information.

Corrosion Cracking and made much broader recommendations. The industry conducted studies on the problem. Based on the detail in related documentation, we seem to recognize the concern and we expended much effort in studying the problem. In memorandum NED 91-20038, we recognized that only a visual inspection can find a through wall crack. Boric acid is an indicator of a potential problem. It recommended that we inspect the CRDM tubes.

Based on damage DB incurred in 6RFO, we understood the consequences of boric acid corrosion.

See the BWOG safety evaluation, which is summarized in the "Other Documents" below.

SER 20-93, Intergranular Stress Corrosion Response deferred to BWOG.

The response documentation includes a Cracking in Control Rod Drive Mechanism BWOG Project Authorization Request for the Penetrations. The affected plants (in Europe)

The conclusion said, "Based on the Material Committee. Task 5.4 is for planned on inspected all head penetrations completed safety evaluation and the ongoing developing top-of-head inspection tooling for Root Cause Analysis Report Tables

• 85

Document Davis-Besse Response/Actions Comments and installing new insulation to allow leak industry effort, no further action with respect CRDM nozzles. The task was planned for detection testing. The cracks are not to this SER is deemed necessary."

1996.

significant to safety. Plants with similar head penetrations should review their testing and There seems to be a gap of SEE-IN inspection programs.

documents addressing boric acid corrosion and stress corrosion cracking between 1990 and 2000 - as if both issues fell off the nuclear radar screen. This was the only SEE IN document found in that time frame.

See the BWOG safety evaluation, which is summarized in the "Other Documents" below.

SER 4-01, Recent Events Involving Reactor NG-EN-00324, Boric Acid Corrosion The response gave the impression that the Coolant System Leakage at Pressurized Control, provides the required actions to program was comprehensive. There was one Water Reactors. Detailed reactor inspections identify, evaluate, and resolve boric acid OERC member who did feel the response are important to identify boric acid. Of leakage and corrosion. Any identified was not adequate, but backed off. The particular concern are areas covered by leakage is evaluated to determine corrective response did not raise the issues that are insulation or otherwise inaccessible, actions. For leakage that is not repaired, coming to light now that we were unable to Undetected or uncorrected RCS leakage can monitoring is specified. The specific inspect the center part of the head and there result in reactor coolant system locations include Control Rod Drive Flanges.

was boric acid there and that we had decided pressure-retaining component degradation Inservice inspection program will perform not to fix or clean those areas. The response from corrosion and wastage. RCS leakage leakage inspections beneath the reactor vessel did not give any hints that there were can result in extended outages or substantial head insulation, weaknesses.

increases in personnel radiation exposure.

Small leaks often are not detected by installed leak detection systems or RCS inventory balance calculations, emphasizing the need for thorough visual and other nondestructive examinations. Oconee modified the service structure and cleaned Root Cause Analysis Report Tables

• 86

Root Cause Analysis Report Tables

• 87 Document Davis-Besse Response/Actions Comments the head to allow easier detection. Although still in study, VC Summer is doing Noble Gas sampling.

SEN 6, Boric Acid Corrosion.

Evaluation deferred to SER 13-87.

SEN 18, Reactor Vessel Head Corrosion Evaluation deferred to SOER 81-12.

SEN 190, Pressurizer Spray Valve Bonnet No evaluation found. Distributed for A Davis-Besse event.

Nuts Dissolved by Boric Acid.

information.

SEN 216, Leakage from Reactor Vessel OERC determined that the document only Although the SEN only contained Nozzle-to-Hot Leg Weld.

contained preliminary information and no information and gave no recommendation on action can be taken at this time. Distributed what could be done, it may have been more for information, appropriate to have the system experts make that call.

SEN 220, Pressure Boundary Leakage at Deferred to SEN 4-01.

Palisades. Palisades had a through-wall crack in a CRDM housing.

O&MR 348, Failure of a Limitorque DB is in compliance with recommendations.

This does not seem to provide any value to Operator Stem Nut this issue.

Root Cause Analysis Report Tables

• 87

Figures RV HEAD INSULATION SERVICE STRUCTURE SUPPORT STEEL 18 ACCESS OPENINGS "MOUSE-HOLES" AT DAVIS BESSE CRDM NOZZLES 2" MIN GAP BETWEEN INSULATION AND TOP OF RV HEAD Figure 1. Davis-Besse RPV Top of Head Section View Root Cause Analysis Report Figures.88 Root Cause Analysis Report Figures

• 88

Source: EPRI/DEI N_

Location of existing drain Q0 0 QQ0 holes used for CRDM cleaning and Nozzles and Q

inspection designators Q

0 0 00 0

o0 o=

c7 do4 o=

o0 d, d,

d, °0 0

8 d9 d00 Eý c

o-33 o1o 400 0

o 0

01d d o0 Closure studs S

Figure 2. Davis-Besse RPV Top of Head Plan View Root Cause Analysis Report Figures.89 Root Cause Analysis Report Figures 9 89

Bolts Low-Alloy Steel Reactor Vessel Head I

I Control Rod Drive Mechanism Flexitallic Type Gaskets "Mirror" Type Insulation SStainless Steel Cladding Figure 3. Davis-Besse CRDM Nozzle General Arrangement Root Cause Analysis Report Figures.90 Root Cause Analysis Report Figures e 90

Figure 4. Boric Acid and Iron Oxide on Vessel Flange at 12RFO Root Cause Analysis Report Figures e 91

Nozzle 2 Corrosion Profile (DOWNHILL SIDE)

A-A L'1 A

Figure 5 Nozzle 2 Corrosion Area Location, Size, and Profile.

Root Cause Analysis Report Figures *92 FENOC 12 Root Cause Analysis Report Figures e 92

Figure 6. Cavity in Reactor Vessel Head Between Nozzle 3 and 11 Root Cause Analysis Report Figures.93 Root Cause Analysis Report Figures e 93

0 Ci2 Ci2 C

0 cj) 0 0

U C#)

U 0

U2 0

0 0

0 t

0 CL 4)

Tq>D~nD*

0 90 160 20 S......................................................................................................................................

---

-)i

--t- - *i

- i i i...

I..

..i i i i i..

-- 1m2 -- 3 m4 *5~~--------

m~

1-1 m13 4

1 m1 m1

---

----------- ------- ------- -----------------------S i z i n g -------------

Root Cause Analysis Report Figures @95 Figures e 95 Root Cause Analysis Report

hrie2 Rdle TbpQwbsia 0

m~

I-1-2-3-4-5-6--7-8-9-10 12-13 14-15--

16-Nz. Erd Figure 9. Nozzle 2 Crack Locations and Sizing Root Cause Analysis Report Figures @96 Root Cause Analysis Report Figures 9 96

Tq0nma 0

D 180 ZO 360 220 22 5 ----------------------------------------------------------------.

2 3 00 --------------------------------.-.--

23 5 ---------------------------------------------------------------. ---.--------------------------------------------------------

Z- -

3-4---6-----8----10-11 12-13-14-15---------

XFigure 10.----

Nozzle-3 Crack-- Locatios a-S 270 -- ---------

5-2 7 5 -..------------

7 -

0 1 - 1

• 1 4 - 1 6

~. r Figure 10. Nozzle 3 Crack Locations and Sizing Root Cause Analysis Report Figures @97 Figures 9 97 Root Cause Analysis Report

N395 RdiIle TTCWMa 0

255 227

• Z5

-mo 2R5 mt

- M 5

3.0 3t5-18I 1-2-3-4-5 7--8-9-10-11 13 14-15-16-Nyz ErBdi Figure 11. Nozzle 5 Crack Locations and Sizing Root Cause Analysis Report Figures.98 so Root Cause Analysis Report Figures

• 98

Nm~ele4tfdilee Tq>Dn~da 0

355 NO

,..5" 380 385 320 "3 425 E

041.0 4041.5 425 400-440 445 450 45 18D 2-3-4 6--7-9-10-11 13 14-15--16-Nz Erd Figure 12. Nozzle 47 Crack Locations and Sizing Root Cause Analysis Report Figures.99 Z Z ---

I ------

\\ -----------

Root Cause Analysis Report Figures e 99

Figure 13. Corrosion and Possible Impingement at Nozzle N-3 Root Cause Analysis Report Figures e 100

DAVIS-BESSE RFO-13 A

V I

B C

,J.-T 900 IW.,ý~

1800 Nozzle 3 K

1 I

2700 3.9 6.2 6.2 NR 0.34, 6 1.2 6

-5.9--

-NR~-

NR-

-07309- -0.260- -0.365- -0.305- -0.3

-NR 6.0

-3.6-

-3.6-

-0.303- -0.255- -0.73

-0.260- -0.302- -0.99- -5.-

-R

-6.6-

-3.0-

-0.30- -0.240- -0.294- -0.30-

-0.303- -0.02-0.310

-0.3-5

-3.6-

-6.2-

-0.299- -0.300- -0.344- -0.298-

-0.304- -3.600-

-3.600- -6.4-6.2-0.301- -0.300- -07300- -0.300- -0.300-3.400-3.400- -5.8

-6.8-

-6.2-0.7-9

-0.280- -0.340- -0.350- -0.360- -0.370-

-0.370

-0203 0 7 3 0-----_

NR N 0.2-9 0276

.38 W,,

0.r 0 o-0 A

.7 R

A B

C j

E F

G H

3*.',

Nozzle 11 Approximate Location Figure 14. Nozzle 3 Clad Thickness Measurements Root Cause Analysis Report Figures *1O1

£,..*

• _,-j Root Cause Analysis Report Figures e 101

DB CRDM(8d.48.5k,4/2.765,0 milm'.

no MON.

ks WE it

-00 19Is Imik IV Rw gft Z

,GNP, ANSYS 5.7 MAR 19 2002 14:46:34 PLOT NO.

1 NODAL SOLUTION TIME=4004 SY (AVG)

RSYS=11 DMX =.442069 SMN =-30956 SMX =76101

-30956

-10000

~0 10000 20000 30000 40000 50000 100000 Figure 15. Hoop Stresses and Operating Condition Deflections in CRDM Nozzles 2-5 Root Cause Analysis Report Figures @102 Root Cause Analysis Report Figures 9 102

Source: EPRI/DEI Oconee 3 Uphill 165' 100% Thru Also location of ANO-1 leaking nozzle without circ crack above weld Leaking CRDM Nozzle (with circ cracks)

Leaking CRDM Nozzle (no circ cracks)

Leaking TIC Nozzle Oconee 3 Uphill 165 Doo 100% Thru 0

0 Oconee 3 X

0Downhill 480 o

29% Thin Davis-Besse O4 W/

Downhill 330 55% Thru Oconee 3 Downhill 66° 35% Thru Figure 16. Location of Leaking Nozzles in B&W Design Plants Root Cause Analysis Report Figures @103 Root Cause Analysis Report Figures

• 103

M Leaks Without Circ Cracks Above Weld 0 Leaks With Circ Cracks Above Weld Figure 17. Distribution of Leaking Nozzles in B&W Design Plants Root Cause Analysis Report Figures @104 0.25 Circ Cracks on Downhill Side 0.20 Fraction of 0.15 Nozzles With Leaks 0.10 0.05 0.00 0.0 8.0 11.4 16.2 18.1 23.2 24.7 26.1 30.1 33.9 Angle (deg) 35.0 36.2 38.5 Root Cause Analysis Report Figures 9 104 Circ Cracks on Uphill Side

lFigure 18. CI{DM Nozzle Leakage Observed at Oconee 3 Root Cause Analysis Report Figures.105 Root Cause Analysis Report Figures 9 105

0.4 daily readings 30 day average 0.3 CIS 0.2 V

6/1/00 8/1/00 10/1/00 12/1/00 1/31/01 4/2/01 6/2/01 8/2/01 10/2/01 12/2/01 Date Figure 19. Unidentified Leak Rate at Davis-Besse (Cycle 13) 2/1/02 Root Cause Analysis Report Figures e 106

Source: EPRIIDEI N

L.eaking Nozzles N-1, N-2, N-3 0

0 0

0 Possible Leaking Flangeas N-3, N-6, N-1 1, N-51 0 /

0 0

@0 W

E Deposits Reported 0

t Knon eaingFlng Around Flange at RF-13 K)..

N-31 6

(No Photos Available)

E C 0 q S

IIAffected Area at RF-10 0Additional Affted Area at RF-11 SAdditional Affected Area at RF-12 s

Additional Affected Area at RF-13 Figure 20. As-Found Locations of Boric Acid Deposits on Davis-Besse Vessel Head (a 0RFO to 13RFO)

Root Cause Analysis Report Figures.107 Root Cause Analysis Report Figures 9 107

Figure 21. Nozzle Crack Leakage Rate Calculation Results Root Cause Analysis Report

---

ANSYS Model -Head Material Intact ANSYS Model - Head Material Corroded

• .-.--- Zahoor Analytical Model D

Davis-Besse Nozzle N-3 10_

E 0.1 0.0 0) 0.001 Ir

° 0.0001 0.00 0.20 0.40 0.60 0.80 1.00 1.20 1.40 1.60 Crack Length Above Weld (inches)

Figures 9 108

Head Material Removed 90° Around Nozzle From Symmetry Plane 2401 1.26" Nodes Spaced --

Axially at 0.125" 1401 Weld Top Crack Bottom 101 15 Downhill Plane Nodes are O's Series Uphill Plane Nodes are 80,000's Series Tube Node Series: l's at Nozzle ID, 5's at Nozzle OD Shell Node Series: 5's at Shell ID (merged w/tube OD) in weld region 6's at Shell ID above weld region 15's at edge of shell section Node Numbers Increase by 100 up the length of the tube and shell Node Numbers Increase by 1 along the tube and shell radius Figure 22. Finite Element Model Boundary Conditions to Simulate Axial Crack Root Cause Analysis Report Figures.109 5

Root Cause Analysis Report Figures 9 109

"s)

Di Nozzle Midplane (90" From Downhill)

S--Top of A AxiaW HalfWi DB CRDM(8d,48.5k,4/2.765,)-

Ax. Crack to 1.25 in.

Above Weld

• mimetry splacement estaints Ield trck idth Figure 23. Crack Opening Displacement with the Crack Surface Nodes Released Root Cause Analysis Report Figures 9 110

"\\ \\

0./

2400 0

48 33

@I 0

0 0

0 0

0 0

0 Figure 24. Boric Acid Deposits on Top of Head at Start of 13 RFO Root Cause Analysis Report Figures @111 I

34 ~

~

l 3I 6l I

/2 1,

Root Cause Analysis Report Figures e 111

Source: EPRI/DEI Boric Acid Crystals In humid air 0 701F (EPRI-3)

Deearated Water 2,500 pp. 0 70)7 (A) 2,500 ppm 0 100t] (A) 2,500 ppm 0 1401F (A) 1,000 ppm 0 3921F (B) 3,000 ppm 0 59077 (B)

Low Oxxyen Water 723 pp.f 3501F (D) ppma350:7 (C) 20,000 PP F

0 1807 (0EPRI-4)

Aerated Water -

30 to 10017 2,500 ppm@

701F (A) eoo0ppme a

0)7 (0) 2,000 ppma 100:p (07R1-2) 2,00pm 100)7 (A) 2,000Opyp a104:7 (B)

Aerated Water -

140 to 180a F 2,500 ppm a 140)F (A) 22,800 pp.

m 1401F (E) 20,000 ppm a 1801F (EPRI-1)

Aerated Water -

212 to 2201F 4,000 ppm a 21237 (M) 4.00 ppa a 212:7 (P*)

- 22,000 ppm a 220 7 (H)

- 26,000 ppm a 220:7 (H)

- 44,000 ppm a 20017 (0)

- 79,000 ppm a 2201:

(H)

Aerated meter -

3507F

-1 4,000 pp 5:

7 Aerated Water -- 600:7 4,000 ppm a 600:7 (7)

Dripping. onto Beated Surface 2,000 ppm 5

1807F (EPP0-5)

- 26,100 ppM 0 210)F (G)

- 14,000 ppma 3001F (2)

- 13,000 ppm 2

300iF (J)

- 13,500 ppm 500:F (J) 13,500 ppm'0 575) (j) 2,000 ppm 600IF (EP0.-5)

Ieoniemant oj meatged Surfaeo

,1000 ppm 0 170:p (0) 2,000 ppm a160O: (BPBI-7)Plemga 1,000 ppem 35007 (r) 1,000 ppm 0 600:1F L) 2,000 ppm 0 60017 (07R0-7)Plange 20ea0ape Snt 0

Annulue 1,000 ppm 0 600:7 (0)

-2,000 ppm a 600:7 (0701-6)

S S 2

2 F

0 8)

S 02 :1 013

.5 Rev. 3 (4/17/01 tOPt1t 1 AnnukI pp P

LItiTtIfl1zttU 0.0001 0.001

.01

.1 Corrosion Rate (inches/year)

Figure 25. Corrosion Rate for EPRI Experiments (Proprietary)

Root Cause Analysis Report Figures.112 Rit InZAnlu 10 Root Cause Analysis Report Figures e 112

)

1 S~~Law Conodon Mike, for Low...

• Oxygan AdddlDudnqUeTIM Ted In LMw (60-00 ppb} O~ygen

=_=77ý YT 7

---F--ý

.T ----

I I-- ----

I4 Tha airoslnov id 4 ---

---

T p la nM ft i

dn Iiwn BolingP=l an

= ry dtI ne oTyial Low~V Lorwlqo Rof Bowedcu fo

-ge I Tem eatr

_+ ____*_

• ~

~

L-0-da

1.0 0.9 "0.8 Zo 0.7

> 0.6

,z 0.5 S0.4 0.6 05 0ý2 00 Containment Monitor Filter Replacement S[AcidGGuideboo IN 96-11: resin ingress I

inceeases cracking II I

L S....

...i...i L........

.....1 --- -----

ji............. ---

I I 

I -

I -

Lm inu

BAC workshop I *.e RDM nozzle leak t

I I

I I

I I

I I

I

-d

I I

BA AA Number of Air Cooler Cleanings Calendar Operating Cycles

[

Significant Information Prior to IORFO Monthly replacement for preventive maintenance Monthly replacement for preventive maintenance I

I P-11-Relie V., Lkg

...--.....- -- 4--

F...----- -

L Bulletin 2001-01: circ cracks I

BAC Guidebook, Rev. I 4.....

S........

i.....

-i fI j.....

r.

..1 F-.

I.

1

50 40 S30 N' 10 r p n

[FPo ortable OtPA lifters Replactenon rae due to.w.

insalled Aug to Sept.

flow SwRI tests sboo iron 1999, uro toS oxide c, o ce ntea t ton L re to e iTo o oxide Iweek t-2dy 2wks t wk 2wksý E2]

IORFO IIRFO 12RFO 13RFO CRDM Flange Conditions RPV Flange Conditions RPV Head Conditions Figure 26. Timeline of Key Events Related to twor iause Analysis Keport Figures.113 Rev. 7 April 5, 2002 t) Nozzle 66 gasket replaced at 7RFO and 8 RFO t) Nozzle 31 gasket leaks but not changed

1) Replaced gaskets on flanges 3, 5, 6, 11, 31

1) No leakage from flange gaskets

1) One flange with indications ofaminor

2) BA is powder and white

2) "Red" deposits under nozzle 3 flange

2) "Red" deposits under nozzle 3 flange leakage

3) Bottom of nozzle 3 flange not inspected due

3) Difficult to get camera in place due to hard, to hard BA brown, boric acid

1) Head flange clean in 1994

1) BA flowing towards mouseholes in S-E First indication of red colored boric acid deposits Significant flow of red colored boric acid t) Significant flow ofboric acid deposits from quadrant from mouseholes in quadrant of S-E quadrant deposits from mouseholes, piling to 6" deep mouseholes in S-E quadrant behind studs in S-E quadrant

2) BA deposits extend around entire flange P" deep max i) BA deposits dripping through insulation at 8RFO

1) Visual inspection 65/69 nozzles

1) Vsual inspect 50/69 nozzles I) Visual inspect 45/69 nozzles l) Visual inspect 347/69 nozzles

2) Head "cleaned" at 8RFO

2) Minor BA deposits around outer nozzles of RV

2) Loose BA piling op behind outer nozzles

2) BA definitely red and hard to break up N-I: leak

[Nozzles 5 & 47 ceacked)

3) Head not inspected during 9RFO Head

3) BA brittle, breaks easily (needed crowbar) "lava-like" N-2: leak, ci"e & minor wastage 1

3) BA deposits at enter of head N-3: leak & major wastage K~oot C~ause Analysis Report Figures e 113 Q

I I

I I

I I a

a I

Reactor Vessel Head Boric Acid Corrosion outurc,:

Ex l r*

Figure 27. Events and Causal Factors Chart on Following 5 pages:

Root Cause Analysis Report Figures.114 Root Cause Analysis Report Figures e 114

~~?.~~~

2

&",~~~~A

2.

'~--

.'2Z 2 2

41 0

A.

0 0

0 0

D 0

0 0

0 0

0 0

0 a

0 0

0 1994 Repaired CRDM Flanges (8) 1996 Repaired CRDM Flanges (10) 2000 Repaired CRDM Flanges (5) 1990 Repaired CRDM Flanges (23) 1991 Repaired CRDM Flanges (15) 1991 Leaking CRDM Flanges (7) 1993 Repaired CRDM Flanges (15) 0 0

Figure 28. Leaking Flanges Found and Repaired During Each Outage Root Cause Analysis Report Figures @115 0

0 0

0 O

Root Cause Analysis Report Figures 9 115

Figure 29. Flange Leakage with Stalactite Formation from Insulation and Stalagmite Formation on top of Reactor Vessel Head (8RFO)

Root Cause Analysis Report Figures e 116

Figure 30. Flange Leakage Crusted On Side of Nozzles and Stalactites from Gaps in Insulation (8RFO)

Root Cause Analysis Report Figures @ 117

Figure 31. Reddish Brown Boron Deposits Crusted on Side of Nozzle (8RFO)

Root Cause Analysis Report Figures e 118

Figure 32. Boron Deposits - Source Unclear (8RFO)

Root Cause Analysis Report Figures @ 119

Figure 33. North Side of Reactor Vessel Head (1ORFO)

Root Cause Analysis Report Figures

• 120 Root Cause Analysis Report Figures e 120

9 I

$

I SI*

/

,.

I I

I I

I



I 1

A

-'-1 Figure 34. Boron Deposits Near Top of Reactor Vessel Head (1ORFO)

Root Cause Analysis Report Figures 9 121 i*"

i*

P",

Figure 3 5: Typical Deposits for Periphery (1ORFO)

Root Cause Analysis Report Figures.122 Root Cause Analysis Report Figures e 122

Figure 36. Red Rusty Boric Acid Deposits on Vessel Flange at 12RFO Root Cause Analysis Report Figures

• 123

Figure 37. Boron Piled Under the Insulation (1 IRFO)

I HAVE NO PICTURES FROM 2000 OUTAGE.

Figures.124 Iron rich deposits on Root Cause Analysis Report

Figure 38. Boric Acid Deposits with Heavy Iron Concentration on Underside of Nozzle 3 (13RFO)

Root Cause Analysis Report Figures

• 125

0 a

a 0) 0 0

0 0

0 0

Q 0

0 0

a 0

0 0

0 0

0 0

0 0

0 0

a

© 0

0 0

0 Figure 39. 2000 Interferences with CRDM Flange Inspection Root Cause Analysis Report Figures @126 0

0 0

0 0

0 0

a 0

Root Cause Analysis Report Figures e 126

CTMT Air Coolers:

Lower level - 585' o

CRDM Vent. Fans (2)

Refueling floor -608' (not to scale)

Figure 40. RE4597 Sample Location Root Cause Analysis Report Figures @127 Root Cause Analysis Report Figures 9 127

CTMT Radiation Monitors RE4597AA/BA (Combined Iodine Channels)

(0 (0 (o (o (0 l

0) 0)0)0)0)0)

I I &

L I*

I t ro ww w

MM ZPqq q)Z Figure 41. CTMT Radiation Monitors RE4597AA/BA (Combined Iodine Channels)

Root Cause Analysis Report Figures

• 128 E

c 0

"-5 C

0~

1.E-07 1.E-07 1.E-07 8.E-08 6.E-08 4.E-08 2.E-08 O.E+00

0)

0) cý ED

0) 0) 0)

0) 0)0 Iý I,1

0) 0)

00) a) z 000 000 C) 9U Month/Year 00 U) z 9

c:

-T) 0009 LL >,

CoUco a.>

(D 0 (n z (N

0 c

cu)

Root Cause Analysis Report Figures e 128

CTMT Radiation Monitors RE4597AA & BA (Noble Gas Channels) 8.00E-04

-=3 Noble BA (chl)

PRZR Code Safety Mid-cycle

~ Nbl AA(ci)

Rupture Discs Cut Outage 2:

A--

Noble AA1

/

2,chlo)

> 6.002-04

~12 RFO 11RFOA 4.00E /,A AAA 2.002-0 AA

/ IAAA A AAA AAA*A, AA,*

0.00E+00 A

-5

>3 a>

.>m 5 c

c

>, 5 a > =

75 0- >

0 a>

CUn.

)z) 5)0 Month/Year Figure 42. CTMT Radiation Monitors RE4597AA & BA (Both Noble Gas Channels)

Root Cause Analysis Report Figures @ 129

Source: EPRI/DEI Molten Boric Acid 0

Boric Acid on Head Acts to Retain Moisture at Annulus and Possibly Provide Some Thermal Insulation Effect At High Leak Rates, Boric Acid May Channel Flow Up Through Insulation (see nozzle 3 flange)

N 5 5

S N

N NN NN 5

5 e

NNNN NNSN N

NN NN%

.f tt i

.ff..

%NN N

N N

N S

N

.t..

t

.N.N.N.N.

N.N.

N

N N

N

'N N

N N

N%

ft

%/

fft~

tf NNNNN NN~N "N I'll If There Were No Pre-Existing Boric Acid Deposits on Head, Leakage From PWSCC Crack Would Soon Create Deposits on the Head at the Annulus Exit

\\ Possible Crervice Corrosion in Annulus at an Intermediate Stage of Degradation Figure 43. Potential Effects of Boric Acid Deposits on Vessel Top Head Surface.

Root Cause Analysis Report Figures.130 Root Cause Analysis Report Figures e 130

Crack Profile for Nozzle 3, Flaw #1 Figure 44. Crack Profile for Nozzle 3, Flaw #1 Root Cause Analysis Report Figures

• 131 o

0.6 E

0.5 0.4

0.

0.3 S0.2 o

0.1 o

0~

cN U, OR "I

CJ C C !

cp q,-

C.

J, C.J "t

qo

)

Crack Elevation Root Cause Analysis Report Figures

• 131

. Potential Evidentiary Request List I

Metallurgical Samples From Nozzle 3 It is desirable to obtain the remaining section of Nozzle 3 from the elevation of the cut to the bimetallic weld to the CRDM nozzle flange. Note: The specimen should not be cleaned prior to input from the Root Cause Team. This specimen may be used for the following examinations and tests:

a.

Examination of external surface of nozzle looking for evidence of flow up through the annulus.

Include high quality photographs

b.

Metallurgical examinations including chemistry, microstructure, etc.

c.

Hardness traverse through the wall thickness (similar to Oconee 3)

d.

Tensile properties at three locations through thickness (similar to Oconee 3)

e.

Others as identified

2.

Non-destructive Inspections of Top Head Surface at Nozzle 3 Location It is desirable to perform several non-destructive inspections of the top head surface:

a.

Priority 1: High quality photographs of the corroded areas adjacent to Nozzle 3. The purpose of the photographs is to show:

General extent of corrosion Evidence of flow across clad and base metal surface Evidence of possible impingement of steam jet on surfaces

b.

Priority 2: Casting impression of cavity. The purpose of the impression would be to further aid in identifying the boric acid corrosion mechanisms such as Volume loss Location of volume loss relative to leak Undercutting of low-alloy steel at cladding interface (potential)

3.

Specimens From Remaining Material at Nozzle 3 Location It is desirable to remove specimens of the unsupported J-groove weld and adjacent areas of the unsupported clad. The priorities for these examinations are as follows:

a.

Priority 1: The section of the J-groove containing the downhill (;0°) crack should be removed. This specimen would be used to:

Determine the crack geometry (single crack, branches, etc.)

Determine the crack width Assess flow induced erosion on the crack faces

- Assess the potential for the crack to have started at the J-groove weld surface Assess the potential for weld defects

- Assess the clad thickness and integrity Root Cause Analysis Report Attachments

• 132

b.

Priority 2: The exposed surface at the location of the uphill crack located by UT examination

(ý1800) should be evaluated.

The first step should be to perform a PT examination of the surface to determine if the crack remains at the machined surface of the weld metal and weld buttering, and if there is any wastage of the low alloy steel that may have occurred as a result of the leakage.

If there is evidence of the crack, or of wastage that extends deeper than the machined surface, a casting impression should be made of the surface to record the crack and wastage.

c.

Priority 3: A section from the J-groove weld and small amounts of adjacent low-alloy steel base metal and cladding at the triple point between the weld, low-alloy steel and unsupported cladding.

This specimen will be used to assess the surface of the corroded low-alloy steel and the potential for galvanic corrosion between the Alloy 182 weld/clad and low-alloy steel material.

d.

Priority 4: If the unsupported section of clad and J-groove weld are to be removed as part of the repair, it is desirable to remove this entire piece intact including a small amount of the low-alloy steel base material at the ends of the unsupported section of the J-groove weld. This larger specimen would be used for:

Further assessment of flow and impingement on the clad surface Thickness and structural integrity of the complete unsupported clad Corrosion of the low-alloy steel material adjacent to the cladding

4.

Examinations and Potential Specimens From Nozzle 2 The wastage uncovered when the lower part of the nozzle was removed needs to be further characterized since it maybe a lead indicator of the type of wastage discovered at Nozzle 3:

a.

A casting impression should be taken of the wastage below the remaining section of the nozzle.

b.

Specimens of boric acid deposits from the cavity behind the remaining nozzle wall should be removed and should be removed and collected in a clean specimen container.

c.

The cavity behind the remaining nozzle should be further probed to establish height above the bottom edge of the remaining nozzle, width, and depth. This information will supplement the already performed boroscope examination.

d.

After access is provided to the top surface of the vessel head the location where the nozzle penetrates the vessel head should be photographed 360' around the nozzle in its current condition. The surface should then be cleaned of any remaining boric acid deposits and the area photographed again.

Finally, any crevice between the nozzle and penetration should be characterized by feeler gauge measurements to establish the width and depth of the cavity.

If the above examinations show that the areas of wastage on the top and bottom of the vessel head are not vertically aligned, the Root Cause Evaluation Team should be notified immediately to determine if further examination is required.

Root Cause Analysis Report Attachments

• 133

If the nozzle is removed as part of the repair, a casting impression should be made of the inside surface of the bore in the vessel head that contains the wastage.

-180 deg Note: Angular positions are the same as for the UT examinations and are as viewed fronm the top of the head Rev. 4 Root Cause Analysis Report Attachments.134 Root Cause Analysis Report 1 - I-Attachments 9 134

Date Time Source Description 5/30/1980 M80-1188 DB responds to IN 80-27. Inspection showed no corrosion of the studs at DB.

6/17/1980 IN 80-27 DB receives IN 80-27 Degradation of Reactor Coolant Pumps (Fort Calhoun 1 reactor coolant pump casing flange studs).

3/16/1982 IN 82-06 DB receives IN 82-06 Failure of Steam Generator Primary Side Manway Closure Studs.

6/10/1982 1

IEB 82-02 DB receives IEB 82-02 Degradation of Threaded Fasteners in the Reactor Coolant Pressure Boundary of PWR Plants (Fort i

Calhoun RCP closure studs and Maine Yankee steam generator primay manway closure studs).....

8/4/1982 Serial 1-284 DB responds to IEB82-02.

10/22/1982 Log A82-1651 DB rson-ds to IN82-0.Closed to IEB 82-02.

1/9/1987 IN 86-108

'DB receives IN 86-108 Degradation of Reactor Coolant System Pressure Boundary Resulting from Boric Acid Corrosion 4/8D825 (ANO-1 HPI nozzle thermal sleeve) 4e3n1987tN8 NED-87-20156 6-108.D-13 4/24/1987 7 IN 86-108 Sup1 DB receives IN 86-108 Supplement I Degradation of Reactor Coolant System Pressure Boundary Resulting from Boric Acid

__________°_

Corrosion (Turkey Point 4 reactor vessel head)_

11/30/1987,,

IN 86-108 Sup2 DB receives IN 86-108 Supplement 2 Degradation of Reactor Coolant System Pressure Boundary Resulting from Boric Acid

,_I Corrosion (Salem 2 reactor vessel head and San Onofre 2 valve packing) 12/22/1987 NES-87-10423 1DB responds to IN 86-108, Supplement 1 and 2. RCS leak management policy incorporates the need to identify, if possible,

ýwhere leakage is and evaluate any boric acid corrosion concerns.

3/10/1988 Cycle History-

_Begin 5RFO 3/30/1988 Log 2532 DB receives GL 88-05 Boric Acid Corrosion of Carbon Steel Reactor Pressure Boundary Components in PWR Plants.

5/27/1988 Serial 1527

,DB provides response to GL 88-05. No commitment to inspect and remove boric acid from the head.

12/15/1988 Cycle History

!End 5RFO 6/26/1989 Serial 1-885 DB provides revised response to GL 88-05. No commitment to inspect and remove boric acid from the head.

1/26/1990 Cycle History i Begin 6RFO 2/8/1990 Log 3166 NRC audit of DB boric acid corrosion prevention program has resulted in an acceptable finding and considered the issue

_iclosed.

2/21/1990 PCAQR 90-0120

!During an inspection of the CRDM to nozzle flange interface (RV Head) a chunk of boron was noticed laying on the floor of the CRDM stator cooling plenum (ductwork) in front of the "I" air flow hole in the RV head service structure shroud. This chunk was cone shaped, approximately 5 inches from the tip to base of the cone, and approximately 8 inches in diameter. It was loose on the inside floor of the plenum and was left as is (there were smaller chunks which may have fallen off). Flange leakers were noticed during this inspection.

3/5/1990 IN 90-10 DB receives IN 90-10 Primary Water Stress Corrosion Cracking (PWSCC) of Inconel 600.

3/9/1990 PCAQR 90-0120 A video inspection of the CRD flanges was performed by B&W and reviewed by System Engineering to determine which CRD flanges show evidence of leakage and therefore should be re-worked during 6RFO. Based on the inspection, the following locations identify which CRD flanges should be reworked: F2, C5, L2, D8, C9, F8, L6, H8, 07, 09, L12, H14, E3, D4, F4, G7, N8, Kll, H12, G13, F14, and N1.

Proposed remedial action for PCAQR 90-0120 is to disassemble, clean, and reassemble each of the leaking CRD flanges using new gaskets. Additionally, a PM is already scheduled to inspect the service structure vent fan internals to ensure there is no damage/potential damage from any boric acid that may have reached the fans. Also, a video inspection of the reactor vessel head (below the insulation) will be done during 6R to ensure

_there is no leakage onto the head itself.

Sequence of Relevant Events Page 135

3/19/1990 3/20/1990 3/21/1990 4/10/1990 7/3/1990 9/9/1990 Dec-90 12/28/1990 RFA 90-0510 PCAQR 90-0221 MOD 90-0012 i MOD 90-0012 initiated to install multiple access ports with closure plates in the closure head to permit cleaning and inspection of the reactor head. Boric acid has leaked from the CRD flanges and has accumulated on the reactor head. The reactor head is carbon steel and therefore is susceptible to degradation.

PCAQR 90-0120 Inspection of fan internals found no boron deposits in either fan. Based on additional inspections of CRD flanges during re work of the originally identified flanges, KI 1 was not re-worked because it was not leaking and G3 was added to the ones to be re-worked because it appeared to be leaking. Inspection of the reactor vessel closure head below the insulation found three areas with boron deposits. The areas were located near reactor vessel stud holes 3, 34, and 45. These areas were 1accessible through the service structure mounting flange drain holes. The three areas were cleaned by RC personnel using wire brushes and a vacuum cleaner. After cleaning, these areas were visually re-inspected by Systems Engineering personnel to be sure the deposits were removed and there were no surface irregularities from the deposits. The deposits 1were removed and no surface irregularities were found. Root cause was determined to be inadequate CRDM flange gasket performance (a known problem). In future outages, when leaking CRDM flanges are found, replace the gaskets with the new Cycle History

!End 6RFO RFM 90-0012 Y Telcon between DB and Crystal River to find out what Crystal River's experience was during their recent refueling outage when they modified their service structure. Nine 12" diameter holes were installed equally spaced around the service structure. Took two 10 hour1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> shifts to machine the access holes and bolt holes. Takes -30 minutes to install covers. No

!problems encountered with installation. Boron was found on the head. Removed boron with scrapers and vacuum cleaner.

Half a wheelbarrow of boron removed. No degradation of the reactor vessel head or insulation support steel was found.

Crystal River has done many visual and video inspections of the reactor vessel head through the mouse holes. In 1981 or 1982, they tried to clean the head through the mouse holes using long handled tools. The cleaning was unsuccessful due to

_the oor access and the inability to see the entire head. Overall, the modification was worthwhile.

EPRI NP-7094 EPRI issued EPRI NP-7094, Literature Survey of Cracking of Alloy 600 Penetrations in PWRs (EPRI Project 2006-18) to document the problem of stress corrosion cracking of alloy 600 penetrations in PWR pressurizers and to identify corrective actions that utilities can take to address this issue. Lists CRDM Nozzles as an Alloy 600 component.

PCAQR 90-0120 Maintenance Procedure DB-MM-09023, Routine CRDM Maintenance, revised to reflect the use of the new gasket parts and require the use of the ultrasonic measurement techniques.

Sequence of Relevant Events Page 136 RFA noted an inspection of the reactor vessel head revealed several areas where boric acid has leaked down from the CRD flanges and accumulated on the head (PCAQR 90-0120). The head is carbon steel and is therefore susceptible to degradation from the boric acid. The RFA requests Design prepare a modification package to install access holes in the service structure to allow cleaning and subsequent inspection. Sketches from B&W were included, as B&W was currently doing the analysis to do this work for Crystal River.

CRDM F2 vessel flange has slight erosion in outer gasket groove. CRDM F4 vessel flange has 2 small irregularities on face.

1/9/1991 EXT-91-00088 DB received B&WOG Materials Committee Report 51-1201160-00, "Alloy 600 SCC Susceptibility: Scoping Study of B&WOG Materials Components at Crystal River 3" dated November 1990. This document summarizes the completed research regarding Alloy Committee Report 51 600 components used at a target B&WOG plant (Crystal River 3). Based on this information, a susceptibility rating is given, 1201160-00 along with recommendations for ensuring RCS integrity through inspections of appropriate components. The applications of Alloy 600 at other B&W operating plants were identified and the applicability of the target plant evaluation to these other operating plants is confirmed. This summary is to be used by the B&WOG Materials Committee in assessing the probable potential for future SCC occurrences with Alloy 600 components at B&W operating plants. The report notes that it is expected that the locations having the highest temperatures in the RCS would be the most susceptible to SCC. The reactor

!vessel upper head is identified as one area where attention should be given. The report recommends the control rod housing

!bodies be inspected, if possible, at an opportune time. The report includes a table of Alloy 600 locations at Davis-Besse, whic 1/21/1991 NED-91-20038

'Memo summarizes the evaluation of PWSCC of Inconel 600 material, reviews industry information available on PWSCC of Inconel 600 (IN 90-10, SER 2-90), and provides recommended actions related to Davis-Besse. The B&W Owners Group Materials Committee sponsored a task to identify all Inconel 600 locations and assess the relative potential of those locations

!for PWSCC. The 69 CRDM tubes are included in this list. B&W further recommended that those items marked with an E asterisk be scheduled for visual inspection (the CRDM tubes were marked with an asterisk). This recommendation was i made with the assumption that all materials are essentially equivalent in microstructure, therefore the priority should be on 1components in elevated temperature service. However, until a complete accounting of the specific materials is made, it is not

'known if a more sensitive material heat is in a lower temperature service condition. Recommendations: (1) Visually inspect

.those components in 7RFO. Visual inspection can only determine if a through-wall crack is present. The incipient crack will

!not be identified. Additionally, the ANO-1 experience showed that as the plant was cooling down from Mode 3, the nozzle sto 1/24/1991 NEO-91-00067

'DB responds to IN 90-10.

8/31/1991

-Cycle Histor

Begin 7RFO 9/12/1991 PCAQR 91-0353 iAn inspection of the reactor vessel head flange noted an excessive amount of boron on the reactor vessel head. One boron 2flow location ran along the curvature of the head and stopped on the head flange by the closure bolts. Identified leakage on several CRDM flanges and reworked several flanges.

9/23/1991 EPRI TR-103345 IAt Bugey III (France), during the mandatory 10 years hydrotest required by French regulations, a leak was detected at CRDM

penetration situated on the peripher of the vessel head.

10/8/1991 EPRI TR-1 00852 1991 EPRI Workshop on PWSCC of non-steam generator Alloy 600 materials in PWR plants was held. Provided extensive coverage of PWSCC in Pressurizer Instrument nozzles, Pressurizer Heater Sleeves, Steam Generator Drain Lines, and Hot Leg Instrument Nozzles. The B&WOG provided an update on B&W activities, including the Materials Committee scoping Istudy of Crystal River 3 and the areas of concern, including the Control Rod Housing Bodies. Davis-Besse did not send a

.irepresentative.

11/7/1991 Cycle Histoy iEnd 7RFO 2/24/1992 PCAQR 92-0072 Visual inspection of the CAC coil face revealed that a white (assumed to be boric acid) build up exists all around it. Cooler I

]performance over the last two weeks had decreased.

3/25/1992

-i PCAQR 92-0139-During filter changeout of RE 4597AA boron was found on the old filter. Boron has been found in the radiation monitors

_____tbefore due to a pressurizer vent valve leak.

Sequence of Relevant Events Page 137

5/14/1992 NED-92-20101 DB engineer issued trip report summary of B&WOG Materials Committee meeting presentation (Work on PWSCC of Alloy 600 Nozzles and Components) with NRC staff held on 5/12/92. Presentation included information on Bugey III CRD nozzle leakage. The NRC seemed to be satisfied with the actions being taken by the B&WOG on the PWSCC of Alloy 600 nozzles and components issue. Regarding the emerging CRDM cracking issue, NRC concurred with the B&WOG that, based on the available information on the French CRDM nozzle inspection, there is no immediate safety concern due to the fact that the identified cracks are axial in nature. The following were suggested by NRC during the above meeting: To meet with NRC during 1st quarter 1993 to cover the following on the CRDM nozzle cracking vis-a-vis B&WOG plants:

11. 50.59 Safety Evaluation to provide sufficient assurance that the issue is not a safety concern.

2. CRDM nozzle inspection strategy/criteria S13.

Evaluation of leak detection/monitoring system

!The decision was made to track these B&WOG items on TERMS to track the B&WOG response to these questions, so TERMS Commitment Al 6892 was created.

6/19/1992 MOD 90-0012

-MOD 90-0012 Void Request submitted. Modification no longer required. This modification was initiated to allow easier access for inspection of CRDM flanges and for cleaning of the reactor vessel head. Current inspection techniques using high

!powered cameras preclude the need for inspection ports. Additionally, cleaning of the reactor vessel head during last 2

!outages was completed successfully without requir. cess ports._

7/7/1992 MOD 90-0012 MOD 90-0012 Void Request rejected by PRG meeting. Mod has been removed from the void process and placed in unbudgeted 9R MODs until after 8R and will be re-evaluated.

8/10/1992 B&W Trip Report i Trip Report 92-020 documents the results of the EPRI Alloy 600 Coordinating Group Meeting Concerning CRDM Nozzle Alloy 600 Program Cracking on Behalf of the B&WOG. The meeting was attended by representatives from each of the NSS vendors, several 1992 Deliverables utilities, and Dominion Engineering. Recent work on CRDM nozzle cracking in the Owners Groups was presented and discussed. One important item discussed was that no one is expected to inspect CRDM nozzles during the 1992 fall outage schedule unless required by the NRC. The NRC position is expected to be finalized at a WOG meeting on 8/18/92.

8/17/1992 B&W Trip Report Trip Report 92-022 documents the results of the Westinghouse Owners Group. NRC Meeting Concerning PWSCC of Alloy Alloy 600 Program 600 CRDM Nozzle Cracking. The meeting was attended by representatives from each of the NSS vendors, each of the 1992 Deliverables Owners Groups, several utilities, and consultants. The NRC provided an overview of Alloy 600 PWSCC and their view on CRDM nozzle inspections. The staff views the CRDM nozzle cracking as a minimal safety impact, but that prudence suggests an orderly inspection program. The NRC is concerned that the potential for cracking exists in a large number of nozzles and that there is concern with boric acid corrosion of the reactor vessel head. The staff presentation slides indicated the following inspection, evaluation, and repair guidance: (1) For PWR plants refueling before Spring 1993, visual inspection Iduring leakage test, with special attention to CRD penetrations at periphery locations and visual inspections (VT-2 quality)

.remote or direct to inspect the inside surface of the spare CRD penetrations; (2) For PWR plants refueling after Spring 1993, 9/10/1992 MOD 90-0012 MOD 90-0012 Void Request submitted. Modification no longer required. This modification was initiated to allow easier access for inspection of CRDM flanges and for cleaning of the reactor vessel head. Current inspection techniques using high powered cameras preclude the need for inspection ports. Additionally, cleaning of the reactor vessel head during last 3 outages was completed successfully without requiring access ports.

Sequence of Relevant Events Page 138

B&W 51-1218440-00 10/2/1992 12/1/1992

-7EPTRI TR-103345 1992 EPRI Workshop on PWSCC of Alloy 600 in PWRs is held. See Proceedings in EPRI TR-103345. Workshop sessions focused on current concerns about PWSCC of alloy 600 penetrations in the reactor pressure vessel head in several plants, including Bugey 3 plant in France. Framatome presented a summary of stress analysis, concluding the stresses are highest in the outermost nozzles for Westinghouse plants. B&W presented a summary of stress analysis, concluding the stresses are essentially the same for central and outer row nozzles. Another report indicated filed experience shows cracks have ioccurred predominantly in peripheral row nozzles, consistent with the results of finite element stress analyses.

B&W 51-1219143-00 B&W issued CRDM Nozzle Characterization, proprietary document 51-1219143-00, regarding PWSCC of CRDM nozzles.

The fabrication and manufacturing processes for B&W-design CRDM nozzles and French-design CRDM nozzles are discussed. A comparison of this information is made, and the similarities and differences are noted. It is determined that B&W-design nozzles are not significantly different than the French-design nozzles, and, thus, are not immune to PWSCC.

i Cycle History egin 8RFO PCAQR 93-0098

'Head vent flange on SG 1-2 has evidence of boric acid corrosion S

PCAQR-3---0132 -Reactor coolant found leaking from CRDM flanges. Several CRDM flanges identified and reworked.

PCAQR 93-0175 -'Boric acid residue on service water p iping-connections to the CACs.

TERMS A16892 TERMS update memo from V. Kumar: An "Ad Hoc Advisory Committee (AHAC)" headed by NUMARC with members from B&WOG, WOG, CEOG, and EPRI has been formed and working to formulate the needed CRDM nozzle inspection criteria

' and coordinate the relevant industry activities. AHAC met with NRC on 3/3/93 during which WOG Safety Evaluation was discussed. WOG has decided to include an evaluation of the OD initiated cracking, seen by the French, in the Safety Evaluation. NRC will not review the WOG Safety Evaluation (nor any other OG'S) until the form of payment has been

'determined. The following actions for NUMARC resulted: (a) Notify NRC ASAP a schedule for Safety Evaluation submittals land the basis for waiting for a leak before break scenario; and (b) Determination of acceptance criteria for issuance to NRC.

i Contingent upon inspection/repair/and mitigation technique availability three US utilities are likely to perform CRDM nozzle

!inspection in 1994.

Video

ýCRDM Inspection (8RFO)

Cycle History

[End 8RFO Sequence of Relevant Events Page 139 B&W issued Alloy 600 PWSCC Time-To-Failure Models, proprietary document 51-1218440-00, presenting a PWSCC susceptibility ranking model and six failure models that have been proposed within the nuclear industry to model time-to failure of Alloy 600 components as a result of PWSCC. A ranking of 4, 4-5, or 5 indicates a high (50%) probability of failure within 20 years; a ranking of 3 or 3-4 indicates a medium (50%) probability of failure within 40 years; and a ranking of 2-3 or below indicates a low probability of failure within 40 years. All failures to date have been ranked between 4 and 5 with this ranking model. The report concluded that, although none of the models addressed in this document accurately predicts any of the existing industry failures of Alloy 600 components, there is a good base of ideas to improve the time-to-failure model. It is recommended that this model be further refined based on industry and research data that may become available.

12/18/1992 3/1/1993 318/1993 3/19/1993 3/30/1993 3/31/1993 4/30/1993 I

5/26/1993 EXT-93-02137 B&WOG Materials Committee issues Letter OG-1214 to NRC (NRR). At the 3/3/93 meeting between NRC and NUMARC AHAC for Alloy 600 CRDM Nozzle Cracking, the B&WOG committed to perform an evaluation of the safety significance of potential nozzle cracking. Safety Evaluation attached which summarizes the stress analysis, crack growth analysis, leakage assessment, and wastage assessment for flaws initiating on the inner surface of the CRDM nozzles. The overall conclusion reached in this evaluation is that the potential for cracking in the CRDM nozzles does no present a near-term safety concern.

Crack growth analysis predicts that once a crack initiates, it will take a minimum of six years for the flow to propagate through wall. If a crack propagates through-wall above the nozzle-to-head weld, leakage is anticipated and a large amount of boric lacid deposition is expected. Once boric acid deposition occurs from leakage, wastage of the reactor vessel head can initiate.

It is predicted that wastage of the reactor vessel head can continue for six years before ASME code limits are exceeded.

5/26/1993 BAW-10190P B&WOG Materials Committee issues BAW-1 01 90P, "Safety Evaluation for B&W Design Reactor Vessel Head CRDM Nozzle EXT-93-02136 Cracking" via letter OG-1217. The B&WOG utilities have developed plans to visually inspect the CRDM nozzle area to Idetermine if through-wall cracking has occurred. At each of the B&WOG utilities' plants, a walkdown inspection of the RV head has been implemented in response to NRC GL 88-05. Enhanced visual inspection of the CRDM nozzle areas has also been incorporated. If any leaks or boric acid crystal deposits are located during the inspection of the RV head area, an evaluation of the source of the leak and the extent of any wastage will be completed. A conservative wastage volume of 1.07 cubic inches per year is believed to be possible from a leaking CRDM nozzle. The postulated corrosion wastage within and in the vicinity of the RV head penetration from a leaking CRDM nozzle would not affect safe operation of the plant for at least six years. Since inspections of the head area (for leakage and boric acid deposits) are performed during each outage, it is

!unlikely that a leak will go undetected for a period of six years.

5/28/19 EXT-93-02156 iB&WOG issued Letter ESC-407 to Davis-Besse (V. Kumar) forwarding copy of BAW-10190P Safety Evaluation.

7/7/1993 EXT-93-02596 i B&WOG Materials Committee issues the non-proprietary B&WOG Report BAW-1 0190, "Safety Evaluation for B&W Design

'Reactor Vessel Head CRDM Nozzle Cracking" dated June 1993 via letter OG-1236. Report includes a stress analysis of I

_B&W Design CRDM nozzles, crack growth analysis, leakage assessment, and wastage assessment..

7/19/1993 SER 20-93 Intergranular Stress Corrosion Cracking in Control Rod Drive Mechanism Penetrations 9/27/1993 MOD 90-0012 MOD 90-0012 Void Request approved. Current inspection techniques using high powered cameras preclude the need for inspection ports, additionally, cleaning of the reactor vessel head during last 3 outages was completed successfully without requiring access ports.

11/19/1993 NRC Letter

!NRC letter dated 11/19/93 to NUMARC attaches safety evaluation on NUMARC's submittal of 6/16/93 addressing Alloy 600 PCAQR 94-0295 CRDM PWR vessel head penetration cracking issue. The staff concluded there is no immediate safety concern for cracking 1of the CRDM penetrations. This finding is predicated on the performance of the visual inspection activities requested in GL 88-05. The NRC stated in its evaluation that "the staff believes it is prudent for NUMARC to consider the implementation of jan enhanced leakage detection method for detecting small leaks during plant operation. Since there is no commitment made Ito the NRC by DB or by the B&WOG to perform any other inspections than those already being performed to satisfy the requirements of GL88-05, TERMS Commitment Al 6892 is CLOSED.

Dec-93 EPRI TR-103104 EPRI issued EPRI TR-103104 (Project 3223-02), "Residual Stress Measurements on Alloy 600 Pressurizer Nozzle and

-I-Heater Sleeve Weld Mockups," to quantify residual stresses in prototypical instrument nozzles and heater sleeves of Alloy 600 before and after welding.

Sequence of Relevant Events Page 140

12/14/1993 BAW-10190P B&WOG Materials Committee issues BAW-10190P Addendum 1, "External Circumferential Crack Growth Analysis for B&W EXT-93-04330 Design Reactor Vessel Head CRDM Nozzles" via letter OG-1322. Report provides an evaluation of external circumferential crack growth, gross leak-before-break, and CRDM nozzle straightening. Potential for circumferential cracking presents no immediate safety concern to the operation of B&W designed vessels. The overall conclusions presented in B&W-10190P remain unchanged with this addendum. The current GL88-05 walkdown visual inspections or the reactor vessel head areas 94-0295.provide adequate leak detection capability_

3/17/1994 PCAQR 94-0295 TERMS commitment Al 6892 requires a visual inspection of the reactor vessel head every refueling to determine the potential for CRDM nozzle cracking in support of B&W safety evaluation to the NRC discussing CRDM nozzle cracking. This safety evaluation requires a visual inspection be performed to either no cracking exists or to confirm its presence. Regulatory Affairs land Design Engineering believe that although the enhanced visual inspection is not a commitment made to the NRC, it is Srecommended that it be done.

4/29/1994 PCAQR 94-0295 Since the enhanced visual inspection of the reactor vessel head is not a commitment to the NRC and due to the fact that no cases of head cracks have been identified in the U.S. and boric acid leakage through the CRDM nozzle flanges is low, Plant

'Engineering doesn't think there is significant risk of a crack being present. In addition, the inspection methods currently available to us are not highly reliable. Therefore, he does not believe that it is necessary to perform the inspection at this t___

time.

5/27/1994 MOD 94-0025 Initiated MOD 94-0025 to install service structure inspection openings. Reasons for the modification include ongoing industry i concern involving corrosion of the Inconel 600 CRDM reactor vessel nozzles. There is no access to the reactor vessel head or the CRDM reactor vessel nozzles without the installation of the modification. Inspections of the reactor vessel head for boric acid corrosion following an operating cycle is difficult and not always adequate. Video inspections of the head for the CRDM nozzle issue and as a follow-up to the CRDM flange inspection do not encompass a 100% inspection of the vessel head. Cleaning of excessive boric acid residue from the reactor vessel head also does not encompass 100%. Installation of these inspection openings would allow a thorough inspection and cleaning of the head. All B&W plants with the exception of I_

i Davis-Besse and ANO-1 have installed this modification.

7/18/1994 MOD 94-0025 i MOD 94-0025 approved for budget and design approval.

9/12/1994

1.

IN 94-63 1DB receives IN 94-63 Boric Acid Corrosion of Charging Pump Casing Caused by Cladding Cracks (North Anna 1 high head I

I

__safety injection pump_ casing_)

10/1/1994 Cycle History

!Begin 9RFO 10/10/1994 PCAQR 94-0912 jCRDM leakage video inspection identified the following CRDM flanges as leaking M3, K3, G5, M1 1, 011, E13, K5, and M9.

10/17/1994 PCAQR 94-0974 Scratches present on and across seating surface of CRDM nozzle flange at core location G-5.

10/17/1994.

_PCAQR 94-0975 Half moon gouge found on CRDM nozzle flange at core location M-3.

11/14/1994 j

Cycle History End 9RFO 11/15/1994 EPRI TR-105406 11994 EPRI Workshop on PWSCC of Alloy 600 in PWRs is held. See Proceedings in EPRI TR-105406 Parts 1 and 2.

Workshop summarized the field experience associated with PWSCC of alloy 600 CRDM nozzles, reviewed the current status lof inspection, repair, and remedial methods as well as strategic planning models, and discussed stress analysis results as iwell as PWSCC initiation and growth in Alloy 600. Workshop was attended by domestic and overseas utilities, PWR vendors, research laboratories, and consulting organizations. Three U.S. plants have inspected CRDM nozzles; no cracks were found in one plant and only minor cracking was observed on one nozzle in each of the other two plants. Results of inspections in France, Sweden, Spain, Belgium, Japan, and Brazil revealed a trend toward earlier axial cracking in plants with forged inozzles as opposed to those made from rolled bars or extrusions. Other factors such as surface finishing could also play a

_role.

See also EPRI Report TR-103696. Davis-Besse did not send a representative.

Sequence of Relevant Events Page 141

12/20/1994 PCAQR 94-1338 10CFR21 report on sensitized alloy 600 material that may be susceptible to an increased rate of intergranular attack (IGA) due to increased sulfur levels in the RCS.

1/5/1995 IN 86-108 Sup3 DB receives IN 86-108 Supplement 3 Degradation of Reactor Coolant System Pressure Boundary Resulting from Boric Acid

____Corrosion (Calvert Cliff I incore instrumentation flange and TMI 1 pressurizer spray valve body-to-body gasket) 1/18/1995 QAD-95-70017 DB responds to IN 94-63 (MU and HPI pumps have solid stainless steel casings).

3/7/1995 DBPRC Meeting MOD 94-0025 (cycle 11 R) tabled at the request of plant engineering manager at PRG. Twenty five percent of B&W plants do History not have additional inspection openings at this time. Plant engineering manager is waiting for additional information prior to concluding that the $250K cost is worth the increased degree of assurance.

3/8/1995....

QAD-95-70078 DbB responds to IN 86-108, Supplement 3. NG-EN-00324 Boric Acid Corrosion Controldiscusses boric corrosion, actions to take if identified, and methods to minimize or prevent corrosion.

4/4/1995 1 6/15/1995 2/29/1996 3/12/1996 4/8/1996 4/19/1996 4/21/1996 4/30/1996 5/1/1996 5/8/1996 6/11996 7/16/1996 7/16/1996 1/7/1997 2/20/1997 4/7/1997 4/23/1997 7/25/1997 DBPRC Meeting MOD 94-0025 (cycle 11 R) decision tabled at PRG. The cycle 11 R MOD was presented for inclusion in the scope of 1 ORFO.

History_

DBPRC Meeting MOD 94-0025 discussion at WSC. Open PRC issue being held open pending further industry information/investigation History jconceming actual benefit.

QAD-96-70113 SERI DB responds to SER 20-93. Efforts via the B&WOG BAW-10190P Safety Evaluation for B&W Design Reactor Vessel Head 93

-]Control Rod Drive Mechanism Nozzle Cracking credited.

IN 96-11 D1B receives IN 96-11 Ingress of Demineralizer Resins Increases Potential for Stress Corrosion Cracking of Control Rod Drive tMechanism Penetrations Cycle History_.

Begin IORFO.....

Video

,Weep Hole Video Inspection PCAQR 96-0551 Video tape of CRDM nozzle inspection shows several patches of boric acid accumulation on the RV head. CRDM nozzle 67 (core location P-6) shows rust or brown stained boron at the bottom of the nozzle at the head. The head area in the vicinity lalso has rust or brown stained boron accumulation. The inspection of the CRDM nozzle flange did not show any sign of

_leakage which indicates leakage is from aevious rating ccle.

PCAQR 96-0650 RCP 1-1pump casing stud leaka e Video jDavis-Besse Weej_ Hole Cleaning Nozzle 67 NPE-96-00260 lWhite paper that deals with control rod drive nozzle cracking with distribution to the Senior Management Team. Focus on crack aspects (doesn't address wastage issue).

I Cycle History_

IEnd 10RFO NEN-96-10179 DB responds to IN 96-11. RCS water chemistry sampled every day of the week for sulfate intrusion and action will be taken Limmediately if RCS sulfate concentration exceeds allowable limits.

PCAQR 96-1018

_IN 96-032 Augmented Examination of Reactor Vessel.

DBPRC Meeting I MOD 94-0025 approved schedule change to 12RFO at PRG. No further industry information was available since it was last History reviewed. Comments made include no work done to allow an opportunity to obtain indications of boron leaks, PCAQ last outage on nozzle boron leakage, and PCAQ not answered as there was a problem in quantifying the amount of boron.

DBPRC Meeting

!MOD 94-0025 approved schedule change to 12RFO at WSC due to no further industry information available since last History reviewed by WSC.

GL 97-01 DB receives GL 97-01 Degradation of CRDM/CEDM Nozzle and other Vessel Closure Head Penetrations.

Serial 2439a

,B provides initial response to GL 97-01. DB plans to submit the requested information by July 29, 1997.

.B&WOG submitted its intearated roararn and Topical BAW-2301 regardin GIL 97-01.

Sequence of Relevant Events Page 142

and Evaluation" as a Reference in NG-EN-00324, Boric Acid Corrosion Control, for determining boric acid corrosion rates.

5/4/1998 Video Reactor Head Cleaning 5/19/1998 1 DB-PF-03065

'Test RCOiL and RC02 (completed test date 5/26/98 1200) identified no leakage for CRD nozzles.

5/23/1998 i Cycle History_

End 11 RFO 6/24/1998 iDB tornado event 9/1/1998 [

LCR 1998-0020 1 RC-2 body-to-bonnet nut #2 found missing (boric acid corrosion because nut not stainless steel).

9/1/1998 DBPRC Meeting

'MOD 94-0025 recommended for approval to 13RFO at PRG. There is less than 50% accessibility to the reactor vessel head, History which does not allow for complete inspection or cleaning of potential boric acid deposits. The MOD resolves PCAQ 96-0551, jone of ten oldest open PCAQs. The MOD also addresses plant life extension issues. It is desired to Implement the MOD in

!12RFO to establish a baseline of potential past boric acid corrosion on the reactor head. On-going industry concern of acid

!leakage from CRDM reactor vessel head nozzles could be better assessed. The committee concurred that the MOD should

!be approved but discussed various issues related to scheduling the modification in 12RFO.

9/9/1998 ICR 1998-002011 RC-2 boq-t0-bonnet nut #4 found missing(boric acid corrosion because nut not stainless steel).

9/17/1998 DBATS MOD 94-0025 budget approval.

9/17/1 BPRC Meeting

MOD 94-0025 recommended for approval to 13RFO at WSC. There is less than 50% accessibility to the reactor vessel head, History which does not allow for complete inspection or cleaning. The MOD resolves PCAQ 96-0551, one of ten oldest open PCAQs.

The MOD will address ongoing industry concern of boric acid leakage from CRDM reactor vessel head nozzles. Plant manager (confirm) asked what was the basis for the 13RFO schedule. Response included issue has been around since 1994, there are no failures in the industry, Engineers voice they were comfortable with the 13RFO schedule, RCS leakage source is known and it is not on the head, we have inspected any boric acid sitting on the head, boric acid has been in a dry condition and corrosion attack is not an issue, delay in schedule to 13RFO does not add risk, however aging is a factor and the MOD should be addressed.

9/17/1998 Log 5339 NRC request additional information (RAI) to GL 97-01.

10/17/1998 CR 1998-0020 Remains of a carbon steel nut found when the RC body-to-bonnet location 4 removed.

10/17/1998 TM 98-0036 TM 98-0036 installed to functionally remove the pressurizer code safety valve rupture disks and severed the drain line to the

______lquench tank.

Sequence of Relevant Events Page 143 A

'A \\

7/28/1997 Serial 2472 DB provides response to GL 97-01. Topical Report BAW-2301 provides the justification and schedule for an integrated vessel head penetrations inspection program representative of the B&WOG plants. Inspections will be performed based on the B&WOG plants determined to be most susceptible to CRDM nozzle cracking. The Topical Report concludes that there have been no conductivity excursions indicative of resin intrusions at any of the B&WOG plants.

9/3/1997 DBPRC Meeting MOD 94-0025 re-classified from capital to O&M at WSC. Design Basis Engineering Manager explained that section of the SHistory

~

reactor vessel head cannot be inspected and or cleaned. This poses a risk tosystem maintenance efforts.

Dec-97 BAW-10190P.

B&WOG Materials Committee issues BAW-10190P Addendum 2 4/10/1998 Cycle History

!Begin 11 RFO 4/17/1998 Video CRDM Inspection 4/18/1998 i PCAQR 98-0649 Inspection of the reactor vessel head identified existence of boric acid residue. There were indications that CRDM D-1 0 had 1past leakage.

4/25/1998 PCAQR 98-0767 Video inspection where the CRDM nozzles enter the reactor vessel head indicate several "fist" size clumps of boric acid.

5/2/1998 PCR 98-1124 Recommends adding B&WOG Materials Committee Report Number 51-1229638-00, "Boric Acid Corrosion Data Summary

10/19/1998

[

CR 1998-1895 Performance of DB-OP-03006 showed a CTMT normal sump leakage in excess of 1 gpm. A portion of the leakage is suspected to be originating from the pressurizer code safety valve leakage was originally channeled to the pressurizer quench tank and classified as identified leakage. Implementation of a TM that severed the discharge rupture disks and disconnected the drain lines, allowed the leakage to escape into the CTMT atmosphere.

11/12/1998 PCAQR 98-1980 CAC plenum pressure decreasing for 3.0"H20 in early September to 2.0"H20.

11/19/1998 I

CAC SPB CAC #2 & #3 cleaning 11/19/1998 Serial 2569 DB provides RAI response to GL 97-01. Draft responses to the RAI questions are being developed by the Owners Groups,

'EPRI, NSSS vendors, and contractors and integrated into a single response by NEI.

11/30/1998 CAC SPB CAC #2 & #3 cleaning 12/10/1998, CAC SP.B CAC #2 & #3 cleaning_

12/21/1998 CAC SPB GCAC #2 & #3 cleaning 12/29/1998 CAC SPB ImCAC #2 & #3 cleaning 1/8/1999 CAC SPB CAC #2 & #3 clearing 1/14/1999 Serial 2581 1DB provides RAI response to GL 97-01. NEI submitted response on 12/11/98. Enclosure 3 to the NEI provides the NRC RAI

.items applicable to the B&WOG members.

1/18/1999 GAG SPB GCAC #2 & #3 cleaning 1/27/1999 CAC SPB CAC #2 & #3 cleaning 2/5/1999 CAC SPB GCAC #2 & #3 cleaning 2117/1999 CAC SPB

'CAC #2 & #3 cleaning 2/25/1999 CAC SPB I CAC #2 & #3 cleaning 3/4/1999 CAC GAGSPB CAC #2 & #3 cleaning 3/6/1999 PCAQR 99-0372 Receiving computer point R297 CTMT Rad RE4597ANAB high.

3/15/1999 CGA SPB CAG #2- & #3 cleaning_

3/25/1999 CGAC SP3 CAC #2 &-#3-cleaning 3/30/1999 CR 1998-0020 1 Final RC2 packing leak manageement root cause report issued.

4/1/1999 CAC SPB CAC #2 & #3 cleaning-,-

4/10/1999 CAC SPB CAC #2 & #3 cleaning 4/21/1999 CAC SPB CAC #2 & #3 cleaning 4/24/1999 I

Mid-Cycle Log Begin C

ycle 12 mid-cycle outage 4/27/1999 PCR 98-1124 Incorporated PCR 98-1124 (see 5/2/98) to include B&WOG Materials Committee Report Number 51-1229638-00, "Boric Acid Corrosion Data Summary and Evaluation" as a Reference in NG-EN-00324, Boric Acid Corrosion Control, for determining boric acid corrosion rates.

_Video CRDM flange inspection (cycle 12 mid-cycle) 5/6/1999 TM 98-0036 TM 98-0036 removed.

5/8/1999 DBATS MOD 97-0085 modified the pressurizer code safety valve nozzle implemented 5/10/1999 CR 1999-0861 RE4597AA sample lines full of water. This is a reoccurring condition when starting up after an outage.

5/10/1999 Mid-Cycle Log End Cycle 12 mid-cycle outage 5/13/1999 RE SPB RE4597BA low flow 5/15/1999 RE SPB RE4597AA low flow 5/15/1999 RE SPB RE4597BA low flow 5/17/1999 RE SPB RE4597AA low flow 5/17/1999 RE SPB RE4597BA low flow Sequence of Relevant Events Page 144

RE SPB

___RE SPB RE SPB RE SPB RE SPB CR 1999-0928 RE4597AA Filter Brown, Boron Crystals RE4597AA Filter Brown, some Boron RE4597BA Filter Brown, Significant Boron Crystals RE4597AA Filter Brown, Significant Boron RE4597BA Filter Brown, Boron Crystals Increased frequency that the particulate and charcoal filters for RE4597BA are being changed. The particulate filter had a 5/19/1999 5/20/1999 5/20/1999 5/21/1999 5/22/1999 5/23/1999 5/23/1999 5/23/1999 5/24/1999 5/25/1999 5/26/1999 5/26/1999 5/27/1999 5/28/1999 5/30/1999 5/30/1999 5/30/1999 6/1/1999 6/2/1999 6/2/1999 6/3/11999 6/3/1999 6/5/1999 6/6/1999 6/7/1999 6!8/1999 CAC SPB CAC #1, 2, and 3 cleaning RE SPB RE4597BA Filter Brown, minimal boron crystals RE SPB RE4597BA Filter Brown, some Boron Crystals RE SPB RE4597AA Filter Brown, no Boron, low flow REsPB RE4597AA Filter Yellow, no Boron, ChemistrySample RE SPB RE4597BA Filter Brown, some Boron Crystals, low flow RE SPB RE4597AA Filter Brown, no Boron RE SPB RE4597AA Filter Yellow, no Boron RE SPB RE4597BA Filter Brown, some BoronCrystals RE SPB RE4597BA Filter Brown RE SPB RE4597AA Filter Brown, low flow RE SPB RE4597BA Filter Yellow, no Boron RE SPB RE4597AA low flow RE SPB RE4597BA Filter brownboroncystals, low flow RE SPB RE4597BA Filter brown, boron crystals, low flow Sequence of Relevant Events Page 145 Lsignificant amount of boron crystals while the charcoal filter had very little.

RE SPB RE4597AA Filter Yellow, Boron Crystals RE SPB i RE4597BA Filter Brown, Significant Boron Crystals, Low flow RE SPB,

RE4597BA Filter Brown, Boron Crystals RE SPB RE4597AA Filter Brown, Boron Crystals I

RE SPB RE4597AA Filter Yellow RE SPB RE4597BA Filter Brown, Some Boron Crystals RE SPB RE4597BA Filter Yellow RE SPB1 RE4597AA Filter Brown, little Boron Crystals CR 1999-0510 RE4597BA low flow alarm caused by boron buildup on the particulate filter.

........ RE SPB

_RE4597AA Filter Brown, Boron Crystals,_low flow RE SPB 2RE4597BA Filter Brown, no Boron Crystals RE SPB RE,597BA Filter Brown, no Boron Crystals RE SPB RE4597AA Filter Brown, no Boron RE SPB 1 RE4597AA Filter White, No Boron(rplacement for containment samplee)

RE SPB 1RE4597AA Filter Brown, Boron Crystals on Filter, low flow A

RE SPB

[RE4597BA Filter Brown, minimal boron Cryals RE SPB

_RE_4597BA Filter Brown, Boron Crystals, low flow RE SPB I RE4597AA Filter Brown, minimal Boron Crystals RE SPB _

RE4597BA Filter Brown, Boron Crystals, low flow RE SPB RE4597AA Filter Brown, Boron Crystals, low flow 6/9/1999 6/9/1999 6/10/1999 6/12/1999 6/12/1999 6/12/1999 6/13/1999 6/14/1999 6/15/1999 6/22/1999 6/23/1999 6/23/1999 6/28/1999 6/28/19991 6/29/1999 1

l RE SPB RE4597AA Filter Brown, low flow RE SPB RE4597BA Filter brown, boron cystals, low flow CAC SPB CAC #1, 2, and 3 cleaning RE SPB RE4597BA Filter brown, no boron, low flow RE SPB RE4597BA Filter brown, no boron, low flow RE SPB RE4597BA Filter Brown I

RE SPB RE4597AA Filter Brown RE SPB RE4597AA Filter Brown RE SPB RE4597BA Filter Brown RE SPB 1RE4597BA Filter Brown RE SPB RE4597BA Filter Brown RE SPB RE4597AA Filter Brown RE SPB RE4597BA Filter Brown RE SPB RE4597AA Filter Brown RE SPB RE4597AA Filter Brown RE SPB

__RE4597BA Filter Brown RE SPB

_RE4597BA Filter Brown, Black Particulate RE SPB RE4597BA Filter Brown RE SPB jE4597AA Filter ight brown RE SPB1 RE4-597BA Filter Brown, Boron Crystals, Maintenance replacement RE SPB RE4597BA Boron RE-R-

F RE SPB RE4597BA Filter Brown

__RE SPB RE4597AA Filter brown RE SPB 1 RE4597AA Filter brown, Maintenance replacement RE SPB__

RE4597BA Filter Brown RE SPB RE4597AA Filter brown RE SPB

_ RE4597BA Filter Brown RE SPB RE4597AA Filter Orange, erratic flow RE SPB RE4597BA Filter Tan RE SPB FRE4597AA Filter Brown, Incorrect Orientation I

RE SPB RE4597AA Filter Yellow, Maintenance replacement RE SPB RE459*7BA Filter Brown, Correct Orientation RE SPB RE4597AA Filter Brown, Correct Orientation RE SPB 1 RE4597AA Filter Orange, erratic flow RE SPB 1 RE4597BA Filter Brown, Correct Orientation CR 1999-1300 Several filters from the CTMT radiation monitors and a sample from the White Bird used for CTMT pressure releases were sent to Southwest Research Institute for analysis. The RE4597BA filter from 7/3/99 contained primarily iron oxide (10-100 microns with some smaller particles down to 1 micron). There was also some measurable chlorine. The iron oxide particles had a granular appearance indicating the source is from corrosion. The RE4597BA filter from 7/9/99 also had three darker spots on it which were analyzed to contain potassium and chlorine. A sample from the White Bird also contained iron oxide.

No boron was detected, however, there would have to be a large quantity to detect it.

RE SPB RE4597AA Filter Brown Sequence of Relevant Events Page 146

7/31/1999 RE SPB RE4597BA Filter Brown 8/1/1999 RE SPB RE4597AA Filter Brown 8/1/1999...

RE SPB RE4597AA Filter Yellow, Replaced prior to calibration 8/1/1999 RE SPB RE4597BA Filter Brown 8/10/1999 CR 1999-1300 TM 99-0022 installed four portable HEPA filtration units in containment (WO 99-005029-000) to reduce the particulate concentration.

10/1/1999 NG-EN-00324 NG-EN-00324 Boric Acid Corrosion Control revision 2 became effective. Revision 2 implements corrective actions from the CATS RC2 event.

10/8/1999 WO 99-005029-001 TM 99-0022 removed.

11/5/1999 Project #10294-033 Memorandum on analysis by Sargent and Lundy regarding RE4597 filters (CR 99-1300). The fineness of the iron oxide particulate, would indicate it probably was formed from a very small steam leak. The particulate was likely originally ferrous

'hydroxide in small condensed droplets of steam and was oxidized to ferric oxide in the air before it settled on the filters. The steam leak is likely at a high elevation in containment as it is reported there is a uniform settlement of iron oxide particulate on horizontal surfaces. The presence of concentrated chemicals contained in the containment sump indicates the particulate came from a steam source. The presence of copper on the radiation monitor sample filters may indicate there is a water chemistry imbalance problem. The iron oxide does not appear to be coming from general corrosion of a bare metal surface

_I in containment or from steam impingement on a metal surface.

12/6/1999 Log 5585 NRC staffs assessment identifies since the additional volumetric inspections performed to date have confirmed that PWSCC is not an immediate safety concern with respect to the structural integrity of vessel head penetrations in domestic PWRs, and since we have approved the integrated program for implementation, we concluded that the integrated program provides an 4/1/2000...............Log

.a~cceptable basis for evaluating your vessel head penetrations.

4/1/2000 i 12R Log IBegin 12RFO 4/6/2000 RWP 2000-5132 RWP written as a tool to control radiological exposure for cleaning boric acid from Rx head. Estimate 30 man hours and 100 i_

ImRem.

4/6/2000 CR 2000-0782 Inspection of the reactor flange indicated boric acid leakage from the weep holes. The leakage is re/brown in color. The

ýleakage is worst on the east side weep holes. Five leaking CRDs were identified at locations F10, DIO, C11, F8, and G9.

SCRDM FIO (Nozzle 11) and D10 (Nozzle 31) a believed to be the major source of leakage. Boric acid corrosion control inspection checklist completed. Detailed inspection recommended because new leakage from head which was not evident i during 11 RFO.

4/6/2000 Video Davis-Besse 12RFO CRDM Leak Inspection (flanges and/or head?)

4/7/2000 RCS SPB There are no boron deposits on the vertical faces of the flange of G9 (nozzle 3) drive. The bottom of the flange of G9 drive is inaccessible for inspection due to the boron buildup on the head insulation, not allowing full camera insertion. Since the boron

!is evident only under the flange and not on the vertical surfaces, a high probability exists that G9 is a leaking CRD.

4/9/2000 12R Log Rx vessel head removed.

4/12/2000 12R Log Video inspection of reactor head 4/12/2000 12R Log Boric acid on reactor head is an Outage Issue 4/12/2000 RCS SPB Today should be called "Boron removal day". Decon people broke to the inside of the Rx head with crowbars and reported

'solid rock hard deposits of boron on the head. Recommendation at this time continue to remove as much boron as possible,

!evaluate head condition, contact B&WOG to justify not removing all the deposits, DO NOT recommend use of water or steam

____i better to justify leaving boron on head.

Sequence of Relevant Events Page 147

4/16/2000 4/16/2000 4/17/2000 4/17/2000 4/18/2000 4/20/2000 4/25/2000 4/30/2000 5/13/2000 5/18/2000 6/2/2000 6/30/2000 8/4/2000 9/7/2000 10/30/2000 12/21/2000 12/29/2000 1/5/2001 CR 2000-0994 CR 2000-0995 CR 2000-1037 Video Davis-Besse 12RFO 1-2R Log Last time boric acid on reactor head is an Outage Issue 12R Log

__Head decon is complete RWP 2000-5132

!Total dose is 224 mRem. Total estimated dose was changed to 600 mRem.

12R Log Reactor vessel head is on the reactor vessel DB-PF-03065

'Test RC001 H (completed test date 6/5/00 1550), test type identified as code case N-498-1, inspect on top of service structure looking downward, identifies no leakage for CRD nozzles, flanges, and assemblies.

12R Log

!End 12RFO S..... !CR 2000-1547 CAC plenum pressure decreasing following 12RFO.

1 CAC SPB CAC #1, 2, and 3 cleani CAC SPB

__CAC

1. 1, 2, and 3 cleaning DBPRC Meeting

'MOD 94-0025 recommended for deferral to 14RFO at PRG.

History 1/31/2001-1 4-2/2/2001 2/14/2001 2/20/2001 3/29/2001w*

3/31/20011 I CAC SPB

CAC #1, 2, and 3 cleaning_

CAC SPB CAC #1,2, and 3 cleanin~g_

CR 2000-4138 The frequency for cleaning boron from the Containment Air Cooler (CAC's) fins has increased to an interval of approximately i8 weeks. If the rate continues to remain steady we will clean the CAC's approximately 6 times for 2001, this will expend 1.2 Person Rem in Dose for 2001. An evaluation or assessment team is recommended in reviewing the following items: Station Dose Impact, Potential Plant shut down conditions due CAC's, Potential sources of boron suspension in containment, CAC cleaning (more effective methods), CAC monitoring frequency, 13 RFO Impact, and Boron Depletion.

CR 2001-0039 CAC plenum pressure experienced a step drop from 1.75"wg to 1.50"wg. The drop occurred from 0900 - 2000 on 1/4/01.

Plenumpressure has been decreasing at a rate of 0.02"wg/day since the coils were cleaned on 12/21/00.

CAC SPB CAC #1, 2, and 3 cleaning DBPRC Meeting MOD 94-0025 RCS system engineer assigned as project manager.

History CAC SPB

_ CAC #1, 2, and 3 cleaning CR 2001-0487 i Temperatures inside the CTMT (SG 1 area) for the year 2000 are seeing higher temperatures (10 to 40F) than the previous L worst case years.

CR 2001-0890 Unidentified RCS leak rate varies daily by a much as 100% of the value. The data is not consistent and averaging method is p*es__entNY used to determine the "true" value of the leak.

CAC SPB GCAC #1, 2, and 3 cleaning Sequence of Relevant Events Page 148 The RV head CRDM nozzle at location F1 0 has a large pit in the outer gasket groove with 2 small pits on the inner gasket.

The RV head CRDM nozzle flange at location D10 has extensive pitting across the outer gasket groove. The inner gasket

_groove also has pitting.

Inspection of the reactor head indicated accumulation of boron in the area of the CRDM nozzle penetrations through the head. Boron accumulation was also discovered on the top of the thermal insulation under the flanges. There are no boron deposits on the vertical faces of the flange of G9 drive (nozzle 3). The bottom of the flange of G9 drive is inaccessible for Iinspection due to the boron buildup on the reactor head insulation, not allowing full camera insertion. Since the boron is evident only under the flange and not on the vertical surfaces, there is a high probability-that G9 is a leaking CRD.

3/31/2001 1

Apr-01 51-5011603-01 B&WOG Materials Committee issue RV Head Nozzle and Weld Safety Assessment 4123/2001 CR 2001-1110 Chemistry chaning the filters on RE4597BA more frequently due to low flow. All filters contained boron crystals.

4/27/2001 0240 CR 2001-1110 Sample point for RE4597BA swapped from top of the east D-ring to personnel hatch area. Filter frequency reduced from once per 3 days to once per 14 days.

4/30/2001 IN 2001-05 NRC issues IN 2001-05 Through-wall Circumferential Cracking of Reactor Pressure Vessel Head Control Rod Mechanism Penetration Nozzles at Oconee Nuclear Station, Unit 3 5/2/2001 CR 2001-1191 A project plan with team members needs developed to prepare DB for a cracked CRDM J-groove weld. All three units at Oconee and one unit at ANO have inspected for and found cracked J-groove welds around their CRDM nozzles.

5/30/2001 CAC SPB

_CAC #1, 2, and 3cleanin9 5/30/2001 CR 2001-1191 Individual assigned by Outage Management Team as 13RFO Project Manager responsible for activities associated with the inspection and repair of CRD nozzles.

7/11/2001 RCS SPB i MRP Plant-Specific Data Verification Form updated at MRP request to QA data. Update included identifying previous i inspections were partial and detected boric acid accumulation which was attributed to a CRDM flange leak.

7/23/2001 CR 2001-1822 Frequency at which the RE4597BA filters are being changed out is increasing (frequency between 2 to 7 days). There were boric acid crystals on the particulate filter.

7/25/2001

[

CR 2001-1857 RCS unidentified leakage has been about 0.125 to 0.145 gpm over the past few weeks. About every 7 to 10 days the

unidentified leakage uumpso about 0.25 for a day or two and then returns to the average value.

8/3/2001 Bulletin 2001-01 NRC issuesBulletin 2001-01 Circumferential Cracking of Reactor Pressure Vessel Head Penetration Nozzles.

8/7/2001 CR 2001-2012 Regulatory Affairs initiates for NRC Bulletin 2001-01 Circumferential Cracking of Reactor Pressure Vessel Head Penetration

_Nozzles.

8/13/2001 Bulletin 2001-01 1DB receives NRC Bulletin 2001-01 Circumferential Cracking of Reactor Pressure Vessel Head Penetration Nozzles.

9/4/2001 Serial 2731 _

DB responds to NRC Bulletin 2001-01.

10/17/2001 Serial 2735 DB provided supplemental information response to NRC Bulletin 2001-01.

10/18/2001 CR 2001-2769

'CTMT wide range radiation element RE2387 spiked above the ALERT and high setpoints for approximately three days.

There were no indications of this condition at the radiation monitor panel. Probable cause unknown.

10/19/2001 0541 Unit Log

'Generator output breakers open 10/20/2001 1435 ChemLog RE4597BA filter has abnormally dark brown discoloration.

10/20/2001 0039 1 Unit Log Generator output breakers closed 10/22/2001 CR 2001-2795 1RE4597BA alarming on saturation on high activity. The filter was change less than 19 hours2.199074e-4 days <br />0.00528 hours <br />3.141534e-5 weeks <br />7.2295e-6 months <br /> previous to receiving the alarm.

040L5i The frequency of filter changeout has been increasing for several months.

10/24/2001 Log 5881 Drop-in visit with NRC regarding NRC Bulletin 2001-01.

10/25/2001 1----CR 2001-2862 Calculated unidentified leakage for the RCS has indicated an increasing trend following the scheduled October 20 Sdownpower.

10/27/2001 1935 Chem Log RE4597AA and RE4597BA filters had some boric acid crystals and it was rust color.

10/30/2001

_Serial 2741 DB provided responses to RAI concerning NRC Bulletin_2001-01.

10/30/2001 Serial 2744 DB provided transmittal of results of RPV CRDM nozzle penetration examinations.

11/1/2001 Serial 2745 DB provided transmittal of risk assessment of CRDM-nozzle cracks.

11/2/2001 CR 2001-2795 TM 01-0018 and 01o-0019 installed removing the iodine filter cartridge from RE4597AA and BA and replacing it with a

_cartridge housing with its internal charcoal removed. The higher iodine level in CTMT atmosphere is a known condition.

Sequence of Relevant Events Page 149

11/3/2001 CR 2001-2936 RE4597BNBB monthly functional test could not be performed due to the inability to clear the particulate channel 2 alert and high alarms. The airborne activity in containment had increased as identified on the DAAS monitor following the containment down power on Oct 19 and Nov. 17. The unidentified leakage and normal sump had also been identified as an increase following the containment down powers. The reduction in power twice within 30 days and plant configuration had created an airborne transient in containment. The monitors in question functioned as designed and calibrated, alerting operations and RP to the increasing airborne activity in containment. As plant conditions have stabilized, the transient has abated and containment activity has equilibrated at a level below the set points.

11/8/2001 Log 5885 Meeting with NRC to discuss NRC Bulletin 2001-01.

11/9/2001 Log 5883 Mee-t ing with NRC to discuss NRC Bulletin 2001-01.

11/10/2001 CR 2001-3025 Moderator Temperature Coefficient test performed.

11/12/2001 CR 2001-3025 Increase in RCS unidentified leakage that occurred over the weekend.

11/14/2001

-Log 5880 Meeting with NRC to discuss NRC Bulletin 2001-01.

11/15/2001 Log 5879 conference call with NRC to discuss NRC Bulleting 2001-01.

11/16/2001 2038_

Unit Log Begindown power to 55%

11/17/2001 CR 2001-2862

-ialkdownnCTMT "targets" to determine potential sources of unidentified RCS leakage failed to reveal a solid contributor.

11/19/2001 1109 Unit Log I Return to 100% power 11/27/2001 Log 5902 iMe-eting with NRC to discuss NRC Bulletin 2001-01.

11/28/2001 Serial 2747

!Meeting with NRC to discuss NRC Bulletin 2001-01.

11/30/20011 Serial 2747 IDB rovided supplemental information in response to November 28 meeting regarding NRC Bulletin 2001-01.

12/1 3/2001 2025 Unit Log iCommenced Tave reduction from 582F to 574F.

12/1512001 1245 Unit Log....

Completed Tave reduction to 574F.

12/18/2001 CR 2001-3411

'Received equipment fail alarm the detector saturation whileperorming check source on RE4597BA channel 2.

Feb-02 CD Davis-Besse Bare Head Video Inspection 13RFO 2/16/2002 13R Log

!Begin 13RFO 2/21/2002 CR 2002-00685

!As part of FTI's reactor vessel head work it was identified that there was loose boron 1-2" deep 75% around the I circumference of the flange. On the other 25% from stud 16 to 30 (clockwise), the boron was hard baked 3-4" thick on southeast quadrant (x-y axis). The large boron accumulation is in the same region as seen in 12RFO, but not as deep.

2/25/2002 Video Davis-Besse RFO13 Nozzle Visual Inspection Tape 1 2/25/2002 Video Davis-Besse RFO1 3 Nozzle Visual Inspection Tape 2 2/25/2002 Video Davis-Besse RFO1 3 Nozzle Visual Inspection Tape 3 2/25/2002 P

Video Davis-Besse RFO13 Nozzle Visual Inspection Tape 4 2/25/2002 Video Davis-Besse RFO13 Nozzle Visual.nsp~ecti0n Tape 5 2/26/2002 CR 2002-00846 During performance of the video inspection of the reactor vessel head, more boron than expected was found on the top of the L

head.

2/27/2002 CR 2002-00891 Ultrasonic testing (UT) performed on the #3 Control Rod Drive Mechanism (CRDM) nozzle (location G9) revealed indications

!of through wall axial flaws in the weld region. (See report for nozzle #3 per procedure 54-ISI-100-08, M.G. Hacker, dated 2/27/02) These indications represent potential leakage paths. Further characterization will be performed per the Reactor head nozzle action plan using the "top-down" UT tooling.

Sequence of Relevant Events Page 150

2/28/2002 CR 2002-00932 There are indications of cracks on 5 nozzles: NOZZLE #1 (location H8): Axial cracks, some with pressure boundary leakage.

NOZZLE #2 (location G7): Axial cracks, some with pressure boundary leakage, and a partial depth circumferential crack of approx. 30 degrees. (Note: this crack is sufficiently small that there was no risk of nozzle failure - stresses had substantial margin before reaching ASME code allowable values.) NOZZLE #3 (location G9): Axial cracks, some with pressure boundary leakage (CR 02-00891)

NOZZLE #5 (location K7): Small axial cracks, predominantly below the weld, no leakage but requiring repair NOZZLE #47 (location D12): Small axial cracks, predominantly below the weld, no leakage but requiring repair Nozzles #1, 2, and 3 have

!leakage paths apparent on UT, which is corroborated by boric acid deposits on the reactor head. UT results with the "top down" tool also provide some evidence of carbon steel base metal corrosion at nozzles72 and 3. Nozzle 2 also exhibits channeling_ of the alloy 600 material to a maximum depth of approximately- 0.050 inches to form part of the leakage flow path.

3/5/2002 CR 2002-01053 While machining reactor vessel head nozzle number 3 the nozzle machining tool moved approximately 15 degrees. This is an unexpected equipment movement.

3/8/2002 CR 2002-01128

',Evaluation of bottom up ultrasonic test data in the area of reactor pressure vessel head nozzle number 3 shows significant

_________degradation of the reactor vessel head pressure boundary.

3/8/2002 Video

! Post Inspection of Nozzles 1, 2, & 3 3/10/2002 CD iDavis-Besse CRDM Nozzles 3/10/2002 R 2002-01159 iDuring a video tape review by the Technical Services Director and the Design Engineering Manager, an indication was found on the newly machined face on the mid-span of the CRDM nozzle. The indication appears to be throughwall in the immediate vicinity of the base metal indications. Further review and potentially additional NDE is required. This CR will document that review.

3/14/2002 Video

' Root Cause Video of Nozzle #3 and Adjacent Nozzles i__

CD 2 Davis-Besse Reactor Head Video snspection 11 RFO and 12RFO Video (Nozzle #2 Crevice Inspection Tape #10 Video

.12RFO Reactor Head Inspection Sequence of Relevant Events Page 151

-p x

X XI T --X m

-ii xx x

ixI x

x x

x X

-4 x

x x

x X_

Xb Sequence of Relevant Events Page 152

x x

X X

xx I

S o

I I

~~1~1 Sequence of Relevant Events Page 153 x

x.

_ x x

x

-1.

-[

-TA F

-L x

x x

x x

Sequence of Relevant Events Page 154 i

x x

--A HX-J Sequence of Relevant Events Page 155

Sequence of Relevant Events Page 156

-i E cm cgc

0) 0 Uf)

i

-I I

x x

x x

x X

4 x

x

-v X

xX ix Sequence of Relevant Events Page 158

x X

i xi x

I 1-x X

X x

x

-4 X

X X x IL

-4 Sequence of Relevant Events Page 159 xx X IX X

x TX X

I x

x x

x I

I I

-T f

p i

x x

x x

x x

Ix x

X x

x x

x X

X xi x

x X

F x

x7 x

x x

x X

x FTX V

1 x

x x

X x

F

.1.

L S

Sequence of Relevant Events Page 160 x

x x

x I

x

-I 1 -

-II I

x X

x X

x x

x x

x x

x x

x x

i x

x x

X x

x x

x x

x x

x x

-I i

X Xi 4-I 4-4x xi

_x a-ix, X

Sx

-I-

_X_

Sequence of Relevant Events Page 161 x xl h

I I

P

1

1

x x

x x

x

-i x

x x

x-x x

x IX_

x F

x x-T_

x

-pX

-H x

x xx Z9 L 068d Z

uW4e~

x x

x x

x x

x x

x x

x i

x x

x X

x x

x x

X X-11, tL x

x x

x x

x x

x x

x x

x x

x x

x x

x X

x x

x x

x x

X x

x X--

x x

x x

x x

X x

x x

x x

x t

x i

M GBed Z 4uewqoe:4v SJUBA3 JUBAGION 10 aouenbeS x x

x xV x

x

-L xxx x

i i

i

[

x x

x x

x Sequence of Relevant Events Page 164 x

x x

i

x x

x xx x

x x

• x x

x x

xx I x 4-x i

x x

X-I x

x x

-t x

If

-I 4-F Sequence of Relevant Events Page 165

I XE 4

>1 X

X XI

~x X

x XIX X

CD

-0 CD 0,

CD 0

(DCD<

>CD

______ -t-4*-F-t- 1

j**hi'

I

______ 

______ I

______ Lt I

1 1 I



I f

'---I

1

'I t77



I I. -+  -v

XXX

XX

I I 

-) I II X

ýxi X

><X X:

xi I

X xi x

I f

-p x

XA x x I x F i-r

! X 4-X i

i-x x

Li X xx s

x_._

X X

Sequence of Relevant Events Page 167

.4-I

--q i

Sequence of Relevant Events Page 168