ML15118A154
| ML15118A154 | |
| Person / Time | |
|---|---|
| Site: | Oconee  |
| Issue date: | 11/04/1996 |
| From: | Jaudon J, Peebles T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | |
| Shared Package | |
| ML15118A153 | List: |
| References | |
| 50-269-96-15, 50-270-96-15, 50-287-96-15, NUDOCS 9612030116 | |
| Download: ML15118A154 (50) | |
See also: IR 05000269/1996015
Text
U. S. NUCLEAR REGULATORY COMMISSION
REGION II
AUGMENTED INSPECTION TEAM (AIT) INSPECTION
Docket Nos:
50-269, 50-270, 50-287
License Nos:
Report No:
50-269/96-15, 50-270/96-15, 50-287/96-15
Licensee:
Duke Power Company
Facility:
Oconee Nuclear Station Units 1, 2 and 3
Location:
7812B Rochester Highway
Seneca, SC 29672
Dates:
September 24 - October 8, 1996
Team Leader:
Thomas A Peebles, Chief
Date Signed
Operator Licensing and Human Performance Branch
Division of Reactor Safety
Inspectors:
P. Harmon, Senior Reactor Inspector, DRS
N. Economos, Reactor Inspector, DRS
D. Forbes, Inspector, DRS
E. Brown, AEOD
G. Hornseth, NRR
W. Scott, Inspector, DRCH/HQMB
R. Correia, DRCH/HQMB
G. Humphrey, Oconee Re 50ent Ins ector
Approved by:
.LZLAL
J ns P. Jaudon
Date Signed
puty DirectorI
ivision of Reactor Safety
Enclosure
9612030116 961104
PDR ADOCK 05000269
G
EXECUTIVE SUMMARY
Oconee Nuclear Station Units 1, 2, and 3
NRC Inspection Report 50-269/96-15, 50-270/96-15, 50-287/96-15
The objectives of the inspection established for the NRC Augmented Inspection Team were
to: (1) determine the facts surrounding the specific event, (2) assess licensee response to
the event, (3) assess generic aspects of operations/inspections that may have had broad
applicability to other facilities, (4) oversee licensee activity during their event review, and
(5) interface with other on-site entities such as Occupational Safety and Health
Administration.
On September 24, 1996, at 4:45 p.m., during the restart of Oconee Unit 2, facility personnel
were manually realigning the moisture separator reheater drains at approximately 50 percent
power, when an 18 inch heater drain line ruptured. The water and steam, at approximately
400 degrees F., 250 psig, severely burned seven plant workers in the Turbine Building. The
personnel burned were involved in the manual realignment to feed-forward the heater drains.
Control room operators immediately tripped the Unit 2 reactor and turbine. The 18 inch pipe
between the 2A and 2B second stage reheaters and the 2A1 and 2A2 feedwater heaters had
ruptured at the point where a 45 degree 10 inch stub pipe was attached.
The NRC team reviewed the procedures for the quarantine of the failed piping and
determined that they were appropriate. The team also reviewed the proposal for the
laboratory analysis of the Unit 2 failed piping and found that it was appropriate. The team
monitored the on-going laboratory analyses and agreed with the preliminary finding that the
rupture was due to a one time over-pressure event that was large enough to have ruptured a
new piece of pipe. The 12 inch branch similarly failed due to internal overpressure.
The
piping configuration near the rupture included the 18 to 12 inch reducer, the 45 degree dead
leg, and the 12 inch tee, all of which reflected and intensified the impact force. Thus, the
pipe location with the geometric stress concentrators contributing was the point of the pipe
rupture.
The NRC team monitored both the licensee's Event Investigation Team and their Failure
Investigation Process Team's progress. The NRC team's conclusions agree with those of the
Failure Investigation Process Team. The Event Investigation Team's report was not available
for review in this report.
The team monitored the licensee's progress on precursor event reviews and reached
conclusions on: the root cause of the event; and the adequacy of corrective actions for prior
water hammer events.
The team's review of the most significant event precursors found that:
A water nammer event in the Unit 2 second stage reheater drain system occurred in
May of 1996 which caused a support to fail in the area of the current pipe rupture.
The cause of this water hammer, on the same pipe that later ruptured, was due to the
opening of the high level divert valve, 2HD-26, and resultant reverse flow. The design
modification process evaluated the damaged pipe support but did not rigorously
pursue the root cause of the water hammer event.
During the July 1996 startup of Unit 2, the facility system engineers spent
considerable time working with operations to find a better way of realigning flow from
the second stage reheater drain tank from the condenser to the feedwater heaters.
The general procedure for feed-forward of the drains was used, and it was decided to
close two manual valves prior to startup in an attempt to eliminate possible water
hammers.
The criteria for opening the valves included: a) assuring the pressures in
the system would cause the flow to be in the forward direction, to the feedwater
heaters; and b) that the valves were to be opened very slowly, to allow time for the
lines to warmup. This startup of the second stage reheater drain system was
accomplished with no problems noted.
During the September 24, 1996, startup of Unit 2, the same procedure was used;
however, the procedure had not been modified to include the specific guidance about
system pressures and valve opening timing. The operators began to open the valves
earlier in the startup than in July. Also, they opened the valves over a period of
minutes instead of the one and one half hour evolution that was used during the July
startup.
The team concluded that beginning the realignment evolution, when one of the second stage
reheater drain tanks was at a lower pressure than the A feedwater heaters along with the
relatively quick opening of the isolation valves, caused a backflow and subsequent water
hammer. The operators' actions were conducted with an inadequate procedure, which had
not been changed or put "on-hold" per the site's administrative procedures.
The team's root cause analysis found that the root cause was that the Oconee staff and
management, both operations and engineering, had not appreciated the potential of past
water hammer events. This led to each water hammer event's root cause analysis being less
than complete. During the past several years, the site staff proposed several design
modifications to minimize water hammer events in the second stage reheater drain system.
None of these modifications had been installed at the time of the event.
The team found that neither safety equipment availability nor operability was directly affected,
but that access to the area, if needed, to get to the safety-related equipment or support
equipment in the basement was severely hampered, during and immediately after the event.
The team found that numerous water hammer events had previously occurred on the second
stage reheater drain systems of all three Oconee units.
The team assessed the Erosion/Corrosion program as it pertained to this piping system and
found that it was covered and that thickness measurements were periodically taken on
portions of this piping system. As a result of past inspections, some piping components had
been replaced. The team did not find any evidence that this piping failure was the result of
erosion or corrosion.
The team found that the Maintenance Rule applied to this line and that the licensee staff
agreed it applied, and they have been implementing the Maintenance Rule appropriately with
respect to this line and components.
The team found the startup procedures were general and did not benefit from lessons
learned from previous events. The scheduled revision date for the procedure in question was
October 14, 1996; and the administrative procedure, which required that this procedure be
put "on hold," was not done. Corrective action modifications completed prior to the event
were the repair of specific components damaged during past water hammer events. Planned
future corrective actions included investigation of better operating techniques and preparing
modification packages to minimize water hammers.
The team found that there was no detectable radioactive contamination released as a result
of this event.
The team found that the performance related to emergency response was appropriate. The
notification calls were made to the NRC and courtesy calls to the state and county. Their
corporate staff informed the State Occupational Safety and Health Administration.
TABLE OF CONTENTS
0
Paq e
1.0
INTRODUCTION - AUGUMENTED INSPECTION TEAM FORMATION AND
IN IT IA T IO N . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1
B ackg ro und . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2
Augmented Inspection Team Formation ............................
1
2.0
EVENT DESCRIPTION
............................................
1
2.1
System Description ...........................................
1
2.2
Description of Event .........................................
2
3.0
EQUIPMENT FAILURES/PERFORMANCE ..............................
3
3.1
Determination of the operational history of this piping system
and whether or not it had been subject to water hammer events
in the past ................................................
3
3.1.1
Augmented Inspection Team Findings and Conclusions . . . . 4
3.2
Assessment of past inspections applicable to the piping systems
in question and the applicability and effectiveness of the
Erosion/Corrosion program at the Oconee Nuclear Station ..............
4
3.2.1
Augmented Inspection Team Findings and Conclusions . . . . 8
3.3
Determination of the status of the components of this system
in relation to the new maintenance rule 10 CFR 50.65 ................
8
3.3.1
Scope of Structures and Components Included within the Rule
. . . . 8
3.3.1.1
Augmented Inspection Team Findings and
Conclusions .....................
9
3.3.2 Goals, Monitoring, and Effective Preventative Maintenance ........ 9
3.3.2.1
Augmented Inspection Team Findings and
Conclusions ...............................
9
3.3.3
Engineering Knowledge of the Maintenance Rule .............
10
3.3.3.1
Augmented Inspection Team Findings and
Conclusions ..............................
10
3.4
Review of the corrective actions taken previously to minimize
water hammer or other operational events related to the system ........
10
3.4.1
Augmented Inspection Team Findings and Conclusions ...
10
4.0
HUMAN FACTOR/PROCEDURAL DEFICIENCIES
.......................
11
4.1
Assessment of the startup (i.e., warm-up and heat-up) and
power ascension procedures and techniques with regard to the
secondary systems in question ................................
11
4.1.1
Augmented Inspection Team Findings and Conclusions ......... 12
5.0
RADIOLOGICAL CONSEQUENCES ..................................
13
5.1
Independent verification of the status of radioactive
contamination associated with this pipe and determine if there
was a release of contaminated material ..........................
13
5.1.1
Augmented Inspection Team Findings and Conclusions .........
13
6.0
EMERGENCY RESPONSE ........................................
14
6.1
Assessment of the licensee's performance related to emergency
response, i.e., classifying this event, offsite notifications,
onsite response and interface with offsite emergency agencies ......... .14
6.1.1
Augmented Inspection Team Findings and Conclusions ......... 14
7.0
ASSESSMENT OF THE LICENSEE'S INVESTIGATION OF THESE EVENTS ...
15
7.1
Monitoring and review of the licensee activities related to
event investigation such as quarantine procedures, laboratory
analyses of failed piping materials, root cause analysis and
precursor event reviews .....................................
15
7.1.1
Quarantine Procedures .............
..
15
7.1.2
Laboratory analyses of failed piping materials ................
15
7.1.2.1
Augmented Inspection Team Findings and
Conclusions ..............................
18
7.1.3 Augumented Inspection Team root cause analysis and review
of precursor events ...................................
18
7.2
Assessment of the licensee's overall technical response and
activities to this event .......................................
19
7.2.1
Augmented Inspection Team review of the licensee's
Failure Investigation Process Team Assessment .............. 19
7.2.1.1
Augmented Inspection Team Findings and
Conclusions ..............................
21
7.2.2
Augmented Inspection Team Review of the Licensee's
Event Investigation Team Assessment .....................
21
7.2.2.1
Augmented Inspection Team Findings and
Conclusions ..............................
22
7.3
Team leader shall interface with onsite regulatory entities/
authorities, such as Occupational Safety and Health
Administration .............................................
22
8.0
EXIT MEETING .................................................
22
APPENDIX A - Augmented Inspection Team Charter
APPENDIX B - SEQUENCE OF EVENTS
The sequence of events associated with the Moisture Separator/Reheater pipe rupture
of September 24, 1996, at the Oconee Nuclear Station.
APPENDIX C SPECIFIC ACTIVITY SEQUENCE IMMEDIATELY BEFORE AND AFTER
UNIT 2 HEATER DRAIN LINE RUPTURE
APPENDIX D WATER HAMMER -PROCEDURE AND SYSTEM HISTORY -MOISTURE
SEPARATER REHEATER DRAIN LINES
APPENDIX E - THE LICENSEE REVIEW TEAMS
APPENDIX F - EXIT ATTENDANCE
FIGURE 1 -Unit 2 Heater Drain Piping Elevational Drawing
FIGURE 2 -Unit 2 Piping Layout in Area of Pipe Rupture
FIGURE 3 -Unit 2 Heater Drain Line Rupture Mechanism in Progress
REPORT DETAILS
1.0
INTRODUCTION
1.1
Background
On September 24, 1996, at approximately 4:45 p.m., during the restart of Oconee Unit 2,
non-licensed operators and instrument technicians were manually realigning the Moisture
Separator Reheater Drains at approximately 50 percent power on Unit 2, when an 18 inch
heater drain line ruptured. The contents of the line (hot water and steam at approximately
400 degrees F., 250 psig) severely burned seven plant workers in the Turbine Building.
Control room operators immediately tripped the Unit 2 reactor and turbine.
1.2
Augmented Inspection Team Formation
On the evening of September 24, 1996, a special NRC inspection team arrived on the site to
review the pipe rupture event. On September 26, 1996, senior NRC managers concluded
that events surrounding the pipe rupture warranted further independent evaluation; an
Augmented Inspection Team was formed. A detailed charter was developed to guide the
team (the Augmented Inspection Team Charter is Appendix A).
2.0
EVENT DESCRIPTION
2.1
System Description
The four Moisture Separator/Reheaters for each Oconee unit had two separate stages of
reheat bundles for reheating the High Pressure turbine exhaust steam prior to the exhaust
steam entering the Low Pressure Turbines. The Moisture Separator/Reheaters first stage
reheater was supplied by the bleed or extraction steam from the High Pressure turbine and
was drained through a Moisture Separator/Reheaters drain tank. The Moisture
Separator/Reheaters second stage reheater heating steam was supplied directly from the
main steam system via 8 inch, main steam control Valves (2MS 112 and 2MS 173). Valve
2MS 112 supplied reheating steam to the second stage reheaters of Moisture
Separator/Reheaters 2A1 and 2A2, while 2MS 173 supplied steam to Moisture
Separator/Reheaters 2B1 and 2B2.
After the High Pressure turbine exhaust steam was reheated, the second stage reheater
steam (actually hot water at approximately 460 degrees F., 530 psig.) was returned to the
condensate/feedwater system via two second stage reheater drain tanks. The second stage
reheater drain tanks collected the pressurized hot water and directed it to either the main
condenser (at power less than 60 percent), or to the High Pressure feedwater heaters (at
power greater than 60 percent). The valves to the main condenser were four inch high level
divert Valves (2HD 26 for second stage reheater Drain Tank B and 2HD 25 for second stage
reheater drain tank A). The level range for these level control valves to operate was 31 inch
(Shut) to 45 inch (Full Open).
The second stage reheater drain tanks are approximately 66
inches high. The high level divert was used until approximately 60 percent power to control
tank level. At that time, the system was realigned to the "Feed Forward" mode. The drain
tank's contents were then routed through three inch level control valves, 2HD 95 for drain
tank 2B and 2HD 92 for drain tank 2A. The level control range for these valves to operate
2
was 12 inch (Shut) to 36 inch (Full Open). The outlet of the control valves was directed into
an 18 inch common header. From that point the drain tank contents were sent to the A
(highest pressure) feedwater heaters, 2A1 and 2A2, to provide additional feedwater heating.
The changeover to the feed forward mode was accomplished by opening manual isolation
Valves (2HD 91 and 2HD 94), upstream of the level control valves. The outlet of the isolation
valves then combined at an 18 inch common header, before splitting into two parallel 12 inch
headers to the A feedwater heaters. Since there were no check valves in the lines, the two
second stage reheater drain tanks were cross-connected at the 18 inch common header. A
potential existed for "sluicing" to occur between the two tanks, due to uneven heating from
the two separate Moisture Separator/Reheaters control systems.
2.2
Description of Event
This event involved piping between the 2A and 2B second stage reheaters and the 2A1 and
2A2 feedwater heaters (see Figure 1). The physical configuration had two second stage
reheater drain tanks, each with a 12 inch drain line which went to a 12 inch gate isolation
Valve (2HD 91 for the A tank and 2HD 94 for the B tank), then to a 3 inch level control valve
(2HD 92 and 2HD 95) and finally through a 12 inch butterfly valve (2HD 93 and 2HD 96).
These were parallel lines that eventually joined in a 18 inch common header. Each tank also
had a high level divert line with a 4 inch level control Valve, (2HD 25 and 2 HD 26) that
returned flow to the condenser. This was used to maintain tank level when the unit was at
less than approximately 60 percent power. The 18 inch common line divided into individual
12 inch pipes, with each running to an A Feedwater heater.
The second stage reheater drain lines exit from the drain tanks and generally stay at an
elevation within five feet of the bottom of the tank. The 12 inch line from the 2A second
stage reheater drain tank runs approximately 40 feet horizontally, with a 90 degree elbow,
before an expander to an 18 inch common line where the 2B second stage reheater drain
line joins via a tee. The rupture occurred approximately one foot from where the 2B second
stage reheater drain line joined the 18 inch line at 90 degrees, where a capped, 16 inch long,
10 inch diameter pipe joined at 45 degrees. The 45 degree pipe was an abandoned
connection from a previous modification (see Figure 2). The 18 inch pipe continued at the
same elevation for approximately 41 feet, with a 90 degree horizontal elbow, and then a 90
degree vertical elbow. The pipe rises 26.5 feet vertically, and all of this upstream piping was
sloped to drain back toward the second stage reheater drain tanks. The 18 inch pipe then
turns horizontal via a 90 degree elbow for approximately 56 feet and begins to slope toward
the feedwater heaters. The pipe then vertically drops about seven feet via a 90 degree
elbowwhich is followed by a 90 degree elbow that turns the pipe horizontal for 42 feet, then
upward for two feet, and finally runs 43 feet before dropping vertically about nine feet to the
elevation of the 2A1 and 2A2 feedwater heaters, where it splits into the two 12 inch pipes.
This configuration introduces high points in the 18 inch diameter pipe that were 56 feet long
and 43 feet long and a low point volume, loop seal, that was 42 feet long. This portion of
the piping was sloped to drain toward the feedwater heaters. There were no check valves to
either prevent flow between the second stage reheater drain tanks or back flow from the
3
Steam pressurizing and heating the 56 foot long high point (Figure 3) was the most likely
mechanism to intensify the water hammer, resulting in pipe rupture. The 2HD-91 and 2HD
94 valves were closed while the unit was shutdown, allowing an accumulation of low
temperature water in the entire section of piping that drained toward the second stage
reheater drain tanks. Figure 3 illustrates this situation with the piping being heated and
pressurized with steam from the A feedwater heaters. The steam percolated through the low
point, 42 foot, water filled leg, (refer to Figure 3(a)), and the 56 foot long high point was
pressurized and heated with steam. At this stage in the event sequence, the 2B second
stage reheater drain tank pressure was lower than that in the down stream piping, while the
2A second stage reheater drain tank pressure was approximately equal in pressure to the
down stream piping and the A feedwater heaters.
When the two isolation valves were manually opened, reverse flow in the piping initiated
through the 2HD-94 valve toward the 2B drain tank. The fluid in the vertical pipe and all
piping upstream from there was filled with sub-cooled condensate, which began flow toward
the drain tank. Figure 3(b) illustrates motion of the 56 foot long steam void, -99 cubic feet,
followed by the 42 foot water slug toward the drain tank. The 42 foot water slug only partially
filled the 56 foot long high point pipe such that pockets of steam bubbles formed along the
length. These bubbles then collapsed, due to the liquid acting like a surface condenser.
This flow continued, as depicted in Figure 3(c), until the steam void was in the 26 foot vertical
section and the water slug started to cascade downward in the vertical pipe section. This
caused the steam void to rapidly collapse, to -1 cubic foot, while the water slug accelerated
to fill the void and collided with the water in the upstream piping.
An impact pressure wave was caused by the acceleration impact of the two water slugs
colliding which rapidly propagated through the fluid in the pipe, toward the second stage
reheater drain tanks. This pressure wave was additive to the pipe internal pressure and
effectively increased the pipe hoop stress in the immediately affected area. The piping
configuration near the rupture included the 18 to 12 inch reducer, the 45 degree dead leg,
and the 12 inch tee, all of which reflected and intensified the impact force. Thus, the pipe
location with geometric stress concentrators contributing was the point of the pipe rupture.
3.0
EQUIPMENT FAILURES/PERFORMANCE
3.1
Determination of the operational history of this piping system and whether or not it
had been subject to water hammer events in the past.
Oconee had experienced severe and frequent water hammers in the second stage reheater
drain system since initial operation. Design studies driven by the Corrective Action Program
and the problem reports from those water hammer events revealed several design and
operating flaws. A proposed Nuclear Station Modification 2941, developed in 1991 and 1993
described the Moisture Separator/Reheaters controls as inadequate and recommended
substantial changes to the valves, piping and controls of the two related systems. Another
modification, Nuclear Station Modification 2901, focused on the design of the second stage
reheater drain system and components. Principal recommendations of Nuclear Station
Modification 2901 included installing check valves to prevent backflow from the A feedwater
heaters to the second stage reheater drain tanks and to prevent tank-to-tank sluicing.
4
Control valve design and control changes were also recommended. Both Nuclear Station
Modification scoping documents refer to problems with water hammer and inadequate system
controls that existed since the early days of plant operation. Several Station Problem
Reports and Problem Identification Reports documented water hammers, rapid, uncontrolled
heatups of the systems and support and hanger damage.
3.1.1, Augmented Inspection Team Findings and Conclusions
The Moisture Separator/Reheaters, first stage reheater and second stage reheater drain
systems all had separate steam sources and controls for heating and separate cooling or
condensing points for water cooling/steam condensing on both the Moisture
Separator/Reheaters and feedwater heaters. Each train met at critical points that included
loop seals, drain tanks, drain tank level control valves, and the common 18 inch header
between the drain tanks and the feedwater heaters. Lack of adequate controls and system
temperature excursions in combination with the system layout had contributed to the past
problems.
3.2
Assessment of past inspection applicable to the piping systems in question and the
applicability and effectiveness of the Erosion/Corrosion program at the Oconee
Nuclear Station.
The team inspected the pipe's exterior weld surface and the interior surface through the
fracture opening. Areas of specific interest included the weld's root surface and crack
surfaces.
The team observed no apparent evidence of erosion/corrosion in the pipe sections where the
failure occurred. Also, the balance of the weld that joined the 10 inch angled dead leg
branch to the 18 inch drain line had no evidence of cracking. There was no evidence of
rejectable fabrication defects in this area. A total of three fractures or cracks were observed
during the initial field inspection. One of the three fractures was at the edge of the weld
joining the 12 inch branch that connected valve 2HD-96 to the 18 inch heater drain line, just
upstream from the dead leg branch. This crack was approximately six inches long. The
fracture surface of this crack could not be observed due the tightness of the crack. A second
fracture was observed in the 18 inch drain line. This fracture could be described as a "fish
mouthed" tear which ran parallel with and was connected to the short 10 inch diameter dead
leg branch connection that was attached to the drain line at a 45 degree angle. This pipe
section (18 inch diameter) appeared to have experienced some bulging over a distance of
several feet. The fracture surface had a relatively smooth matte appearance with evidence of
shear lipping and appeared to have the characteristics of a single fast crack type failure.
Evidence of a chevron pattern was discernable on the fracture surface with a direction that
suggested the fracture initiated at the intersection between the 18 inch pipe and the 10 inch
dead leg branch connection. The fracture progressed down the branch connection for a
distance of approximately 18 inches in length. Chevrons were also observed on the fracture
surface of this crack, and these were oriented toward the direction of the weld joint between
the drain line and the branch connection.
6
tank discharge areas, where the inspections were concentrated. Five of the components with
single inspection data were replaced with stainless steel piping, and four of the areas on this
line were scheduled for reinspection during the upcoming EOC-17 refueling outage.
The expanders, elbows, and piping immediately downstream of the control valves on both
trains were replaced with stainless steel during outage EOC-9. Also, at EOC-9, the reducers
and pipe downstream of Valve 1 HD-91 were replaced with stainless steel piping. The elbow
downstream of butterfly Valve 1 HD-93 (B-Train) had been inspected during four previous
outages and was scheduled to be replaced during EOC-17 with stainless steel piping. The
discharge elbow for Valve 1HD-96 (A-train) was replaced with stainless steel during EOC-13.
Unit 3
There were 20 Erosion/Corrosion program inspection locations included in this run of piping.
Multiple data sets were available for 11 of the 20 test areas (one inspection on nine
components, two inspections on two components, three or more inspections on nine
components). The only areas that showed signs of appreciable wear were the valve and
tank discharge areas, where the inspections had been concentrated. Four of the
components with single inspection data were replaced with stainless steel and five of the
areas in this line were scheduled for reinspection during the upcoming EOC-16 refueling
outage.
The first elbow off the B drain tank was replaced with stainless steel piping during EOC-1 1.
The A drain tank discharge elbow had been inspected three times previously and was
scheduled for reinspection at the next outage. The expanders, elbows, and piping
immediately downstream of the control valves on both trains were replaced with stainless
steel piping during EOC-9. The elbows immediately downstream of butterfly valves 3HD-93
and 3HD-96 have been inspected two and three times respectively and were scheduled for.
reinspection during upcoming outages EOC-16 and 17.
These lines were originally installed per USAS B31.1.0-1967. The following table includes
pertinent information for the material in the area of the failure:
Description
Material Size Schedule
Nom. Wall
HoopStressMin
Main Header
A106B 18"
Standard
.375"
.293"
Capped Lateral Branch
A106B 10"
Standard
.365"
.175"
Branch from B Tank
A106B 12"
Standard
.375"
.210"
Visual and Maqnetic Particle Examinations
As a followup to the inspection of the fracture surfaces and pipe condition in the immediate
vicinity of the pipe failure, the licensee performed visual and magnetic particle examinations
of pipe welds and hanger lug attachment welds on either side of the effected components to
determine weld integrity. Results of these inspections were as follows:
5
'
The pipe saddle welded for reinforcement around the 10 inch branch connection, was also
fractured but its fracture surface was not evaluated at this time. The dead leg branch to the
pipe saddle weld had a circumferential fracture, about eight inches long. This crack ran
along the toe of the saddle weld on the dead leg branch side of the joints. The fracture
surface of this crack had a rusty rough woody appearance without discernable fracture
characteristics. Therefore, judging from its appearance, it was difficult to make an
assessment on the evolution of the crack, although this could have been the result of a single
event, e.g., water hammer.
Erosion/Corrosion Test Proqram
Through discussions with cognizant personnel and by document review, the team ascertained
that the subject line was in the Erosion/Corrosion program and in the CHECKWORKS
computer model. As such, several upstream and downstream components were inspected
during previous outages. The area associated with the failure had not been included in the
program as a result of the relatively low velocity and flow rates, the ranking in the computer
model and the lack of flow in the 10 inch dead leg branch pipe. In general, it appeared that
the Erosion/Corrosion inspections had focused on the heater drain tank discharge piping,
isolation and control valve arrangements, expansion loops and heater inlet areas.
Within this area, the team noted that there were 24 Erosion/Corrosion program inspection
locations in this run of piping. Test results showed that the only areas showing signs of
appreciable wear were the valve and tank discharge areas where inspections had been
O
concentrated. The most recent inspection data had been included in the Checkworks model
for Pass 2 Analysis.
Pipe Repairs and Replacement Unit 2
The first fittings off the A and B drain tanks were replaced with stainless steel during outages
EOC-10 and EOC-11 respectively. The expanders, elbows, and piping immediately
downstream of the control valves on both trains were replaced with stainless steel during
outage EOC-7. The elbow downstream of butterfly Valve 2HD-93 (B-train) was replaced
during outage EOC-9 with stainless steel. The discharge elbow for Valve 2HD-96 (A-train)
was inspected during outages EOC-8, 11, 13, and was scheduled for reinspection during the
upcoming outage EOC-16.
In Unit 2, a limited number of preliminary ultrasonic thickness measurements indicated the
material thickness around the affected area was within the manufacturing tolerance.
Additional ultrasonic thickness measurements were planned during the continuing failure
investigation process.
Unit 1
There are 22 Erosion/Corrosion program inspection locations included in this run of piping.
Multiple data sets were available for 14 of the 22 test areas (one inspection on seven
components, two inspections on five components, three or more inspections on nine
components). The only areas which showed signs of appreciable wear were the valve and
7
'9
Unit 3
Weld No.
Size
Description
3-05B-58-2
10"x0.365"
Lateral dead leg to 18" diameter drain line. Rejectable
undercut indication.
3-05B-60-1,2,3
12"xO.375"
Elbow welds from valve 3HD-92 to 18" diameter drain
line. No rejectable indications.
3-05B-64-1
12"xO.375"
Pipe weld between valve 3HD-95 and 18" drain line. No
rejectable indication observed.
Unit 2
2-05B-16-11,12,13
18"xO.375"
18" drain line welds. Linear indications on either side of
weld #12.
2-05B-16-15
10"xO.365"
Lateral dead leg to 18" diameter drain line. No rejectable
indications.
2-05B-16-15A
10"xO.365"
Dead leg cap weld. No rejectable indications.
2-05B-19-1thru5
12"xO.375"
A:2" linear indication on elbow extrados between welds
- 1 and
B:Several indications on elbow, between welds #2 and #3
- some removed by light buffing.
2-05B-19-8,9,10
3"xO.216"
Stainless steel pipe installed under erosion/corrosion
program. No rejectable indications observed.
2-05B-18-5thrul0
3"xO.216"
Same as above.
Unit 1
1-05B-4-109CA
10"x0.365"
Welded cap no rejectable indications.
1-05B-4-147C
10"xO.365"
Lateral dead leg to 18" drain line weld. Rejectable
undercut approximately 1/4" long, 3/32" deep.
1-05B-07-63,64,65
10"xO.375"
Linear indication from toe of weld #65 into pipe, axial.
Linear indication from toe of weld #64 into elbow, axial.
Arc gouge in elbow grinding gouge near weld #63.
The team witnessed visual and magnetic particle examinations on all carbon steel welds
listed above; i.e., 10, 12, and 18 inch diameter welds. The team concurred with the above
described findings. The inspections were performed using a code acceptable, magnetic
8
particle procedure. The technique used followed code prescribed practices. Inspectors who
performed these examinations were adequately trained and qualified to perform this assign,ed
task.
Inspection of as Built Piping Configuration
Following the removal of insulation, the team, along with members of the licensee's Quality
Control staff, inspected as built piping for visible anomalies. Piping sections inspected
included those around the 10 inch diameter, capped lateral, the 12 inch diameter T
connections from drain tanks A and B and the 18 inch drain line header.
Within these areas, the combined inspection team noted that not all of the 10 and 12 inch
branch connections to the 18 inch header had a reinforcing ring welded on the header
around the branch connection. The controlling code for these lines, as referenced in Section
3.2.2.2 of the Oconee Final Safety Analysis Report is the Power Piping Code USAS B31.1,
1967 edition, (code). Paragraph 104.3 of the code states in part, that when a pipe is
penetrated by a branch connection, the size of which weakens the pipe, additional
reinforcement must be provided. The amount of reinforcement provided must meet
Paragraph 104.3.1, D and E requirements.
At the close of this inspection, the licensee was making arrangement to inspect all high
energy, balance of plant lines to verify that as built conditions met code requirements. Their
inspection will include associated supports and hangers in these systems.
3.2.1
Augmented Inspection Team Findings and Conclusions
The team concluded that the pipe rupture was the result of a single water hammer
event. The area in the vicinity of the rupture showed no evidence of significant
erosion/corrosion, and the team did not believe that erosion/corrosion was associated
with the pipe rupture.
The licensee's erosion/corrosion program was consistent with industry guidelines and
was implemented in a conservative manner.
As-built heater drain lines did not appear to be consistent with the piping code of
record, B31.1, requirements as many of the locations where branch connections
penetrate main headers lacked reinforcement collars.
3.3
Determine status of the components of this system in relation to the new maintenance
rule 10 CFR 50.65.
3.3.1
Scope of Structures, Systems, and Components Included Within the Rule
During the onsite inspection, the inspectors reviewed certain of the licensee's maintenance
rule records and program documents to determine if the Heater Drain System (which had
experienced the pipe rupture) was in the scope of the Maintenance Rule. The licensee had
determined that the system is in the scope of the Maintenance Rule as required by
9
10 CFR 50.65(b)(iii), nonsafety-related structures, systems, and components whose failure
could cause a reactor scram or an actuation of a safety-related system. The licensee's
decision was based on system analysis and actual system performance.
3.3.1.1
Augmented Inspection Team Findings and Conclusions
The inspectors concluded that the licensee had correctly scoped the Heater Drain System as
being covered by the maintenance rule.
3.3.2
Goals, Monitoring and Effective Preventive Maintenance
The licensee's Maintenance Rule program generally followed the guidance of NUMARC 93
01 (May 1993), "Industry Guideline for Monitoring the Effectiveness of Maintenance at
Nuclear Power Plants." NUMARC 93-01 was endorsed through Regulatory Guide 1.160
(June 1993), "Monitoring the Effectiveness of Maintenance at Nuclear Power Plants."
In accordance with NUMARC 93-01, the licensee categorized the Heater Drain System as
normally operating and non-risk (low-safety) significant. Plant level performance criteria of <2
reactor scrams/fuel cycle and <8 percent forced outage rate were established for monitoring
Heater Drain System performance and demonstrating effective preventive maintenance, as
required by 10 CFR 50.65(a)(2).
Part of the licensee's initial implementation phase of the rule included a historical review of
.
each unit's Heater Drain System performance against the performance criteria. This was
done to place structures, systems, and components into the 10 CFR 50.65(a)(1) and (a)(2)
categories. The licensee determined that a Unit 3 system failure that resulted in reactor
scram would require the system to be placed in the (a)(1) category. The Heater Drain
Systems for Units 1 and 2 had met their performance criteria and were placed in the (a)(2)
category.
The licensee's goals, monitoring, and corrective actions under Section (a)(1) of the
Maintenance Rule included the addition of two moisture/separator reheater level detectors
and appropriate new controls. Even though the Heater Drain Systems for Units 1 and 2 had
not experienced a failure that resulted in a reactor scram, the licensee decided to make the
same system changes as for Unit 3. This change was made to Unit 1, with Units 2 and 3
scheduled to receive the change during future refueling outages.
3.3.2.1
Augmented Inspection Team Findings and Conclusions
Based on a review of system and plant performance data and discussions with the licensee
site Maintenance Rule Coordinator, the inspectors determined that the licensee's
maintenance rule categorizations, performance criteria, monitoring and corrective actions for
the Heater Drain System for all three units satisfied the requirements of 10 CFR 50.65 (a)(1)
and (a)(2).
10
3.3.3 Engineering Knowledge of the Maintenance Rule
The inspectors interviewed the licensee's site Maintenance Rule Coordinator to assess
understanding of the maintenance rule and associated responsibilities.
3.3.3.1
Augmented Inspection Team Findings and Conclusions
The licensee's site Maintenance Rule Coordinator was very knowledgeable of the
requirements of the maintenance rule, industry guidelines, and plant systems. The
information established by the coordinator that was provided to the inspectors reflected a
clear understanding of site specific systems' performance as they related to the rule. The
inspectors also noted that the site coordinator understood the responsibilities of the position
in overseeing that the requirements were correctly, consistently and timely implemented.
3.4
Review corrective actions taken previously to minimize water hammer or other
operational events related to the system.
The corrective actions in the past have been primarily limited to upgrading the hangers and
supports that have been damaged during the water hammer events. In most cases, the
hanger designers had been unable to predict accurately the loading of the water hammers.
The descriptions of the system limitations and the proposed corrective actions described
below were contained in the two Nuclear Station Modifications referenced in Paragraph 2.1
above.
Second stage reheater tube bundle, and Moisture Separator/Reheater tube bundle
heatup rates were exceeded when Valves MS-112 and 173 were allowed to control
tube bundle heatup in AUTO. (The 30 degree F /30 minute heatup limit was
exceeded, it actually was approximately 260 degrees F in 30 minutes). SOLUTION:
(a) Replace MS-1 12 and MS-1 73 valve internals with Class IV valves, having better
control characteristics, and (b) Replace MS-1 12 and MS-1 73 control system with
digital valve controllers.
Frequent and severe water hammers in the second stage reheater drain systems
occur due to uncontrolled heating of the system as a result of Moisture
Separator/Reheaters controls, backflow between drain tanks and from the feedwater
heaters, interactions between the drain tank level controllers. SOLUTION: (a) add
check valves between the tanks and between the tanks and feedwater heaters, and
(b) replace level control circuit and level control valves to reduce valve failures and
leakage.
3.4.1
Augmented Inspection Team Findings and Conclusions
None of the proposed solutions had been incorporated, although the general
recommendations had been approved for implementation prior to the steam break event.
The schedule for implementing the Nuclear Station Modifications was set for beginning with
the Unit 2 outage beginning October 1997, and continuing with the other units in outage
11
sequence. However, a modification meeting September 23, two days before the steam break
event, reset the implementation schedule to start with the Unit 3 outage beginning March
1998. The licensee has decided to implement the modifications during the current outages.
4.0
HUMAN FACTOR/PROCEDURAL DEFICIENCIES
4.1
Assess the startup (i.e., warm-up and heat-up) and power ascension procedures and
techniques with regard to the secondary systems in question.
Water hammer was noted as being a long standing problem on this system in that problems
were referenced in Nuclear Station Modification-1,2,3 2901 scoping document, which
identified operational procedure changes that were made in 1981 to OP/1,2,3/A/1106/14,
Moisture Separator Reheater, to lessen the severity of water hammers experienced when
valving in and out the Moisture Separator/Reheaters and Drain Tanks. The procedure
changes were reported to have greatly alleviated the pipe wear due to corrosion, but water
hammers continued which were severe enough to cause valve seat damage and breakage
of tank and pipe supports. These problems were documented in, but not limited to Problem
Identification Reports 1-094-1001, 1-095-0115, 1-95-0513, 2-96-0984, 3-96-1860, and earlier
Station Problem Reports.
In order for the operation of the system to have better control, design changes and system
modifications were planned. The team evaluated the Design Scoping Documents for Nuclear
Station Modification, Nuclear Station Modification-1,2,3 2901, and Nuclear Station
Modification-1,2,3 2941 which were initiated in 1991 and 1993 respectively. These proposed
modifications were for valve replacements associated with the Moisture Separator Reheaters,
first and second stage heater drains and for piping configuration changes to eliminate water
or steam perturbations that have been experienced in that system during previous years.
Nuclear Station Modification scoping document, Nuclear Station Modification-1,2,3 2941,
further documented problems associated with automatic control of the startup and operating
relationship with the reheat steam system. This inadequate control had resulted in heatup
rate limits that were exceeded and also suggested that these problems had been
experienced since the beginning of plant operation. This scoping document recommended
that the pipe layout and supports be evaluated and that valves be added to better control the
startup of the system.
An earlier design modification in the time frame of June 1980, Scoping Document Nuclear
Station Modification 844, Revision 0, Moisture Separator/Reheater First Stage Piping
Revision, involved the removal of two 10" lines to separate the first and second stage
Moisture Separator/Reheaters drains and to reroute the first stage drains to the "B" feedwater
heaters and the second stage to the "A" feedwater heater. This was done to increase
Moisture Separator/Reheaters scavaging steam flow from 0.5 percent to 10 percent in order
to improve plant efficiency and Moisture Separator/Reheaters stability operation by reducing
condensate flooding of reheater tubes. In lieu of cutting the 18 inch line for the branch
removal, the method of removal was to cut and cap the 10" lines and leaving a short length
of piping connected at a 45 degree angle to the bottom of the 18 inch header. The 18 inch
header at the 10 inch intersection was the area that ruptured during the steam break event.
12
This configuration provided a trap for water collection during plant startup and shutdowns.
An acceptable design could have been to cut the 18 inch line on both sides of the stub/s and
replace the existing section with an 18 inch straight piece of piping, eliminating the high
stress areas and collection point for water.
During recent and previous water hammer events, extensive pipe hanger damage had been
noted. The team reviewed Problem Identification Report 1-095-0513 which documented that
the second stage reheater Drain Tank and associated piping and supports were not designed
for steam/water hammer transients such as those experienced during plant startups.
A review of the licensee's Evaluation of Pipe Support 2-05B-2-0-1410N-H98 revealed it was
reworked in May 1996 after one of the two support legs pulled out of the concrete during an
apparent water hammer event. The arrangement supported the 18 inch pipe that ruptured
during the most recent water hammer event. The modification included moving the West leg
that was attached to the floor by a baseplate with anchors installed in the concrete to the
East side of the main support. During the event in May 1996, the anchors pulled out of the
concrete on the support leg on the East side of the main support member. It was noted that
concrete in the area where the anchors pulled out did not contain rebar. The licensee
advised that the floor of the Turbine Building contained rebar, but after the original floor was
completed an additional 12 inches of concrete without rebar was placed on top to cover drain
lines and other piping routed on the floor. It is in this top 12 inches of concrete cover that the
anchor bolts were installed.
O
Although the design engineering effort for Design Scoping Documents, Nuclear Station
Modification-1, 2, 3 2901, and Nuclear Station Modification-1,2,3 2941, was in progress at
the time of this inspection, it had not progressed sufficiently for the team to evaluate it's final
adequacy. The team's review included field inspections of the existing piping configuration,
review of relevant Problem Identification Reports, drawings, and various discussions with
engineering personnel. The team discussed several of the as-found installation
configurations with the licensee such as loads on embed plates attached to the ceiling,
anchor bolt installation in concrete in conjunction with baseplate to embed plate welding,
vacant holes adjacent to installed anchor bolts and the practice of supporting one support
from another support using an L beam. The team concluded, based on these discussions,
that the licensee was performing field inspections of these various type installations and the
engineers were performing evaluations to determine the installations' adequacy.
4.1.1
Augmented Inspection Team Findings and Conclusions
Onsite documentation showed that water hammers occurred since initial plant startup. The
licensee's design review did not identify that water hammers were a problem in the 18 inch
lines when the 10 inch lines were removed and capped. This earlier engineering design
modification did not eliminate these high stress areas and collection points for condensed
water. The team concluded that leaving the stub pieces protruding from the bottom of the 18
inch header did not cause the water hammer event. However, due to the stress risers
created by the piping geometry in the area and to the trapped water, it did cause the water
hammer to concentrate more in this area. Combining this with inherently high stresses that
occur at pipe junctures increased the potential for pipe rupture due to water hammer events.
13
S
When water hammer occurred in the 18 inch piping with the stub piece that projects from the
bottom of the 18 inch pipe full of water, excessive reaction forces occurred. The water in the
stub piece reacted like a piston with downward and outward forces occurring. Stresses at the
stub piece juncture were significantly higher than stresses in a straight piece of pipe. The
piping configuration in the area, including the 12 inch branch connection and the 12 inch to
18 inch expander, were also stress concentrator contributors.
5.0
RADIOLOGICAL CONSEQUENCES
5.1
Independently verify the status of radioactive contamination associated with this pipe
and determine if there was a release of contaminated material.
Following the event, the inspectors interviewed Radiation Protection personnel involved with
event recovery and discussed boundary controls, surveys performed, instruments used, and
survey documentation.
Records reviewed for the Unit 2 Turbine Building sump monitor and unit ventilation monitors
determined the event did not result in any discharge of radioactivity to the environment above
background levels. Also, surveys- reviewed independently verified no radioactive
contamination above background was detected in the Turbine Building, change area,
O
canteen, in asbestos insulation debris, on injured personnel, ambulances, and other
miscellaneous equipment associated with the event. Some discrepancies in survey
documentation were noted by the team. These discrepancies did not alter survey results.
However, this issue was discussed with Radiation Protection supervision, and survey
corrections were subsequently performed.
During tours of the facility, the team observed radiological monitoring equipment in use to
include: portal monitors, contamination instruments, radiation instruments, The Turbine
Building sump monitors and unit ventilation monitors. Calibration records reviewed verified
equipment in use at the time of the event was currently calibrated.
5.1.1
Augmented Inspection Team Findings and Conclusions
The team found that radioactive contamination associated with this event was not detectable
and that no release of radioactive material occurred.
--. o
14
6.0
EMERGENCY RESPONSE
6.1
Assessment of the licensee's performance related to emergency response, i.e.,
classifying this event, offsite notifications, on-site response and interface with offsite
emergency agencies.
The team reviewed the declaration and termination actions taken by the licensee for the
licensee's Notice of an Unusual Event associated with this event to verify the licensee
complied with their Emergency Coordinator procedures.
The inspectors verified the event was classified in accordance with licensee Procedure
RP/O/B/1000/01, Emergency Classification, Change 3, dated July 16, 1996. The licensee
classified this event as an Unusual Event based on emergency action levels identified in the
procedure.
The Event Notification form was reviewed and verified that the event was declared and
terminated at 2040 hours0.0236 days <br />0.567 hours <br />0.00337 weeks <br />7.7622e-4 months <br /> on September 24, 1996, from the Technical Support Center.
During the event debrief, the licensee identified that the Emergency Coordinator procedure
did not contain adequate guidance for event declarations and termination. Specifically, an
event checklist used in the Control Room and Emergency Operating Facility for terminating
an event was not available in the Technical Support Center Emergency Coordinator
procedure. The licensee initiated a Problem Identification Report to evaluate the problem
and completed a draft procedural revision for the Emergency Coordinator Technical Support
Center procedure prior to the end of the onsite inspection.
A review of the Event Notification form, control room logs, and interviews with personnel
determined that the offsite notifications to the required agencies were accomplished in a
timely manner and met licensee requirements for conducting the notifications. Throughout
the event, the NRC, State of South Carolina, Oconee County, and Pickens County
emergency agencies were continuously updated on plant status and conditions resulting from
the event. The inspectors noted during event debriefs that site assembly to account for
personnel in the Turbine Building was also conducted in a timely manner. In addition, a
debrief for personnel involved in the medical evacuation of injured personnel to hospitals was
conducted. The Medical Emergency Response Team personnel commented during the
debrief that coordination and communications between site personnel and medical
responders was good.
The licensee notified the State of South Carolina office of Occupational Safety and Health
Administration of this event which resulted in seven personnel injuries and Occupational
Safety and Health Administration responded to the site with an inspection team.
6.1.1
Augmented Inspection Team Findings and Conclusions
The team found that the performance related to emergency response was appropriate. They
made the notification call to the NRC and courtesy calls to the state and county. Their
corporate staff informed the State Occupational Safety and Health Administration.
15
7.0
ASSESSMENT OF THE LICENSEE'S INVESTIGATION OF THESE EVENTS
7.1
Monitor and review licensee activities related to event investigation such as quarantine
procedures, laboratory analyses of failed piping materials, root cause analysis and
precursor event reviews.
7.1.1
Quarantine Procedures
The team reviewed the procedures for the quarantine of the failed piping and determined that
these were appropriate. The team reviewed the proposal for the laboratory analyses of the
Unit 2 failed piping and found that it was appropriate.
7.1.2
Laboratory analyses of failed piping materials
On October 2 and 3, 1996, an inspection was conducted at the Duke Power Company
corporate metallurgy lab located at the McGuire plant site. A metallurgist from the Materials
and Chemical Engineering Branch of the Office of Nuclear Reactor Regulation, conducted the
inspection. The purpose of the visit was to monitor and evaluate the licensee's metallurgical
investigation of a catastrophic rupture of an 18 inch diameter heater drain line from Oconee
Unit 2. A licensee metallurgist performed the metallurgical investigations and was
interviewed during this inspection.
The licensee's proposed investigation plan was reviewed prior to the inspector's arrival at the
.
metallurgical lab. The plan of the investigation was detailed in a two page outline of the
documentation, metallurgical examinations and mechanical tests that would be performed to
determine the root cause of the pipe failure. Listed steps of the outline included:
documenting the "as received" condition of the pipe, a plan showing locations for removing
sections from various portions of the pipe, metallurgical evaluations, and chemistry and
mechanical tests to determine the material properties. The plan was found to be
comprehensive and complete. No significant revisions were necessary. The investigation
plan was fully adequate to support the root cause determination.
At the time of the inspection, sample removal and preparation were under way. Most of the
"as received" documentation was complete. Since the inspection was conducted prior to the
end of the metallurgical examinations and mechanical property tests, the results noted herein
are preliminary.
The sample consisted of a section of 18 inch nominal pipe size, standard schedule (0.375
inch wall), carbon steel pipe with two branch connections. One branch connection was 10
inch nominal pipe size and the other 12 inch nominal pipe size. The 10 inch branch was a
45 degree lateral with an external reinforcing saddle, constructed in accordance with usual
practice for moderate energy lines. This branch was capped off roughly 1 foot down the line.
The 12 inch branch was a 90 degree "tee" and lacked the normally expected external
reinforcing saddle or ring. The absence of the external reinforcing saddle was determined to
be a construction deficiency.
16
Typically, construction material used for this class of service was seamless carbon steel pipe,
ASTM A-106 grade B. Laboratory analysis results reviewed by the team verified that a
typical carbon steel, such as A-106 grade B, was the material of construction.
The primary failure was at the 10 inch branch connection. It had the attributes of a classic
hoop stress driven rupture. (Hoop stress is the principal pipe stress that arises from internal
pressure. When the hoop stress exceeds the structural capabilities of a pipe, a longitudinal
rupture occurs.) The rupture consisted of a throuigh-wall longitudinal split originating at the
inside corner of the 45 degree 10 inch lateral. A fracture would be expected to originate in
the 45 degree corner at the branch connection (because of the stress concentration that
arises from the geometry) and then propagate as observed. The rupture ran roughly 1 foot
along each line. It ran through both the reinforcing saddle, connecting welds and underlying
pipe of the 18 inch line. It continued some distance past the edge of the reinforcing saddle
before arresting in the pipe base material. In the 10 inch pipe, the fracture ran from the
same origin, along the entire length of the branch, and arrested in the pipe cap. A secondary
fracture ran about 900 around the edge of the reinforcing saddle of the 18 inch line. As a
result of the size of the rupture opening displacement, considerable bending distortion existed
in the 18 inch line all around the 10 inch lateral. There was a sufficient fracture opening
displacement to allow visual observation of the fracture faces over nearly its entire length.
The 12 inch tee showed much less damage. Two cracks were evident. They were located at
the tee to main branch weld, in the 18 inch line, and ran four to six inches circumferentially
around the connecting weld for the 12 inch line. The crack opening displacement was
O
negligible. These cracks were situated such that both were primarily longitudinal to the 18
inch line but on opposite sides of the 12 inch connection from each other. Because of the
lesser degree of damage, it was concluded that the internal pressure that caused these
cracks was substantially less at the location of the 12 inch branch compared to the 10 inch
branch. Additionally, the location of the cracks at the 12 inch branch suggested other
differences in the failure mechanism.
All the fracture faces exhibited classic ductile tensile overload features: 45 degree shear lips
were readily apparent on all the fracture surfaces through the base and weld materials. This
type of failure mode is normal for a ductile material such as carbon steel. After these initial
observations, the investigation concentrated upon examining the details of the fractures,
looking for pre-existing damage, if any, or other contributing conditions.
A number of sections were removed from the welds and base material along the fracture
path. Micro hardness traverses of these different weld, heat affected zone and base material
sections were performed. The hardness of the base material was normal for carbon steel
pipe. The welds showed normal hardness values for welds in the as-welded condition.
Carbon steel of this thickness (nominal wall 0.375 inch) did not require heat treatment after
welding. These welds exhibited this normal construction code practice. Additional testing
was planned to characterize more fully the mechanical properties of the base and weld
materials. However, this work had not been started at the time of the inspection (it was to be
performed by a commercial laboratory with additional testing facilities). It is expected that,
based upon the general observations of the fractures and the hardness test results, that no
material deficiencies will be found.
17
Microstructural examination of the various sections removed for fracture surface
characterization and micro hardness tests revealed a normal grain structure. Several fracture
faces were examined with a scanning electron microscope to verify the failure mode. Classic
ductile failure artifacts were noted. This again demonstrated the failure was the result of a
tensile overload of the material rather than a material deficiency or progressive failure mode
such as fatigue.
During examination of the primary fracture origin, it was noted that several small (1/4 inch
deep), heavily oxidized cracks were on the edge of the cut opening in the 18 inch pipe at the
10 inch connection. Several such cracks were adjacent to either side of the main fracture.
One such crack served as the initiation site for the main fracture. Several of these cracks
were opened for surface fractography. All showed heavy oxidation with consequent
broadening of the crack opening and blunting of the crack tip. From this crack morphology it
was obvious that these small cracks were quite old. Additionally, since there was no sharp
crack front and no indication of recent propagation, these cracks were inactive. The one that
served as the origin for the main fracture was similarly well oxidized. Due to its small size
and lack of ongoing propagation evidence, it was judged that it had not been a significant
contributing cause to the main fracture. Instead, it simply served as a local stress
concentrator at the point in the pipe where the highest tensile overload stress had occurred.
A larger, similarly well oxidized crack also existed roughly 1/2 inch away (also in the edge of
the 10 inch hole) but it did not propagate or play any role in the formation of the main
fracture. Consequently, it was concluded that these small flaws were insignificant to the
structural integrity, were not precursors to a progressive failure, and were not a cause of the
O
failure.
At the time of the inspection, the bulk of the investigative effort was concentrated upon the
large rupture at the 10 inch branch, since it was the location of the greatest damage. Late in
the inspection it was noted that the 2 cracks at the 12 inch branch may have had precursors.
Some part-through-wall cracks, showing oxidized surfaces, were noted at portions of the
fracture surfaces. The oxides did not appear, upon preliminary examination, to be as heavy
as in the case of the small heavily oxidized cracks associated with the 10 inch branch
connection. This suggests these small cracks were more recent and possibly active. Since
this branch lacked a code required reinforcement, the existence of possibly active precursor
cracks is plausible. It was noted that the fresh fracture surfaces resulting from the failure
event exhibited ductile features, supporting the previous conclusion that the material
properties were nominal. At the end of the inspection, the licensee's metallurgical evaluation
of this component was ongoing.
An in-depth review was conducted to determine the possible role of erosion/corrosion wall
thinning as a contributing factor or causative factor in the failure. Some minor wall thinning
was noted in a limited area of the 18 inch line. However, the slightly thinned area was
remote from the fractures and was of no consequence to the failure.
S
18
7.1.2.1
Augmented Inspection Team Findings and Conclusions
The team concluded that the licensee's investigation plan was comprehensive. The
licensee's metallurgist demonstrated excellent knowledge and capabilities. The ongoing
examinations were thorough, and the preliminary conclusions were congruent with the
available evidence. The preliminary results indicated the 10 inch branch failure resulted from
a one time internal pressure overload that exceeded the design and ultimate strength
capabilities of the pipe. No material property deficiencies, significant prior degradation,
progressive failure mechanisms, or construction deficiencies contributed to the failure that
occurred at the 10 inch branch.
The 12 inch branch similarly failed due to internal overpressure. However, preliminary
observations suggest there may have been a construction deficiency (lack of branch
reinforcement) at the 12 inch branch. Some small oxidized cracks were noted along parts of
the 12 inch branch fracture faces. These precursor cracks may have been active and may
have contributed to the through-wall cracking of the 12 inch branch connection.
7.1.3 Augmented Inspection Team root cause analysis and review of precursor events.
The team's review of the most significant event precursors found that a water hammer event
in the Unit 2 second stage reheater drain system occurred in May of 1996 that caused a
support to fail in the area of the current pipe rupture. The design modification process
evaluated the damaged pipe support. However, the cause of this water hammer, on the
same pipe that later ruptured, was due to the opening of the high level divert valve, 2HD-26,
and resultant reverse flow. The cause of these contributors was not rigorously pursued.
During the July 1996 startup of Unit 2, the facility system engineers spent considerable time
working with operations to find a better way of realigning flow from the second stage reheater
drain tank from the condenser to the feedwater heaters. The general procedure for feed
forward of the drains was used, and it was decided to close two manual valves prior to
startup in an attempt to eliminate possible water hammers.
The criteria for opening the
valves included: (a) assuring the pressures in the system would cause the flow to be in the
forward direction, to the feedwater heaters; and (b) that the valves were opened very slowly,
to allow time for the lines to warmup. This startup of the second stage reheater drain system
was accomplished with no problems noted.
During the September 24, 1996, startup of Unit 2, the same procedure was used; however,
the procedure had not been modified to include the guidance about system pressures and
valve opening timing. The operators began to open the valves earlier in the startup than in
July. Also, they opened the valves over a period of minutes instead of the one and one half
hour evolution that was used during the July startup.
The team concluded that beginning the realignment evolution when one of the second stage
reheater drain tanks was at a lower pressure than the A feedwater heaters, along with the
relatively quick opening of the isolation valves, caused a backflow and subsequent water
hammer. The operators' actions were conducted with an inadequate procedure, which had
not been changed or put "on-hold" per the site's administrative procedures until the revisions
reflecting the July experience were made.
19
The team's root cause analysis found that the root cause was that the Oconee staff and
management, both operations and engineering, had not appreciated the potential of past
water hammer events. This led to each water hammer event's root cause analysis being less
than complete. During recent years, the site staff began several design modifications to
minimize water hammer events in the second stage reheater drain system.
None of these
modifications had been installed at the time of the event.
7.2
Assess the licensee's overall technical response and activities to this event.
The team developed their independent evaluation of the events and root causes and
reviewed their results against the licensee's teams findings. The Augmented Inspection
Team concluded that the licensee completed a thorough job of review and that their root
cause determinations were reasonable. The licensee's level of management involvement in
their investigations and in their internal critiques of their investigations was in-depth and
involved the highest levels of their respective organizations. The Augmented Inspection
Team findings basically agree with that of the licensee as noted below and in specific places
in the report.
7.2.1
Augmented Inspection Team Review of the licensee's Failure Investigation Process
Team Assessment
The Failure Investigation Process Team determined that the failure mechanism was believed
to be the cumulative effect of water slug induced water hammer and steam void collapse
O
induced (pressure pulse) water hammer. The initiation point of the rupture was at a pre
existing, minor defect noted at the junction of the branched "Y" connection.
The Failure Investigation Process Team found it important to differentiate this event from
water hammers which have occurred in the Heater Drain System in the past. The differences
included:
-
The approach to operating the Heater Drain System.
-
The condition of system components, particularly the second stage reheater drain tank
dump valves.
-
The Heater Drain System ambient conditions, including pressures and temperatures,
prior to alignment for feed forward operation.
-
The Feedwater heater steam admission rates.
Contributing Factors:
1)
System Design Inadequacies:
-
Mechanical - The historical review revealed that the Heater Drain Systems on
all units have a noteworthy history of maintenance and operational concerns.
Review of site documentation revealed that these concerns resulted in several
design studies and proposed modifications.
20
-
Control - The control of the second stage reheater steam admission valves,
S
2MS- 112 and 173 as well as the second stage reheater drain tank dump
valves, 2HD-25 and 26, appeared to have made system control difficult.
These valves are critical to controlling system pressures and temperatures
such that back flow was prevented.
2)
Communications Concerns:
Interviews with personnel involved revealed that system concerns were well
acknowledged; however, personnel were focusing on unit power as the criteria to use
in establishing drain tank forward flow. The appropriate plant data to consider to
lessen the potential for backflow was the differential pressures between the second
stage reheater drain tanks and the A Feedwater Heaters.
Recommended Corrective Actions:
-
Provide a method to drain stagnant fluid from the high point pocket (loop seal)
in the 18 inch drain line and a method to drain the stagnant water from the
piping between the new check valves and the vertical riser, upstream of the
high points.
-
Decrease the Heater Drain System piping stresses by: eliminating unnecessary
discontinuities; and minimizing multiple discontinuities in close proximity to one
another.
-
Modify the Heater Drain System operation to avoid sudden changes in the fluid
flow direction and/or thermodynamic state.
-
Improve the Heater Drain System piping strength by incorporating forged
fittings.
-
Improve the plant control systems relating to Feedwater steam admission and
Heater Drain tank level control.
-
Improve the Heater Drain System instrumentation by providing actual pressure
indication in the first and second stage heater drain tanks; providing
temperature indication in the Heater Drain System riser and high point; and
providing a dynamic pressure transmitter at the base of the Heater Drain
System riser.
3)
Insufficient Procedural Guidance:
The Operations Procedure used provided insufficient guidance for initiating second
stage reheater drain flow without introducing the potential for Heater Drain System
backflow.
21
SUMMARY
The Failure Investigation Process Team concluded that the root cause of the event was the
cumulative effect of water slug water hammer and steam void collapse (pressure pulse) water
hammer. Significant contributing factors were poor design, poor procedural guidance and
ineffective communication.
7.2.1.1
Augmented Inspection Team Findings and Conclusions
The Augmented Inspection Team independently evaluated the failure mechanism, root cause
of the event, the contributing factors and potential corrective actions and, as delineated in the
first paragraphs of this report, agreed with the Failure Investigation Process Team findings
with the following exception:
-
An additional Contributing Factor was the speed of valve opening. During the July
1996 startup, the Heater Drain manual valves were cracked open and then left for the
piping to warm. The team believes that this technique vwuld minimize the magnitude
of any potential water hammer.
7.2.2
Augmented Inspection Team Review of the licensee's Event Investigation Team
Assessment
The licensee's Event Investigation Team Charter included:
O Identification of any strengths or good practices worthy of sharing with other sites.
Development and validation of a sequence of events associated with this event.
Evaluation of the significance of the event with regard to radiological consequences,
safety system performance and proximity to safety limits as defined in the technical
specifications.
Identification of procedures in use during and leading up to the event and used to
recover from the event.
Evaluation of the adequacy of administrative controls and implementation of those
controls.
Identification of any human factors, training or procedural deficiencies.
Evaluation of any necessary event classification and other pertinent emergency
planning issues.
Evaluate as necessary, the impact of plant material condition, maintenance,
modification/implementation group and engineering on the event.
22
For any pertinent equipment malfunctions, the following determinations will be made:
Root Cause
Any known deficiencies prior to the event
Equipment history
Pre-event status of surveillances, testing, preventative maintenance,
modifications
Any previous corrective actions associated with the equipment
Similar Duke and industry events shall be identified and the application of corrective
actions from those events shall be evaluated.
The root cause and any contributing factors for the event.
An evaluation of applicable management actions prior to, during and following the
event
An evaluation of the response to the medical attention of injured personnel.
7.2.2.1
Augmented Inspection Team Findings and Conclusions
The team followed the licensee's Event Investigation Team Assessment while in progress at
the site. The team listened to their on-site debrief on October 3, 1996. The licensee's team
covered the response to the injured by the Medical Emergency Response Team, the
operation of the heater drain system by the facility and a summary of their sequence of
events. The NRC team's conclusions agreed with the licensee's team's comments and
conclusions.
The licensee's Event Investigation Team Assessment report was released to be reviewed and
evaluated when this inspection was concluded.
7.3
Team leader shall interface with onsite regulatory entities/authorities, such as
Occupational Safety and Health Administration.
The team leader attended discussions concerning the event between the State of South
Carolina Occupational Safety and Health Administration and the facility.
8.0
EXIT MEETING
A public exit meeting was held on October 8, 1996, at 11:00 a.m. on the Oconee site.
Several members of the public attended.
No dissenting comments were received from the
licensee.
Unit 2 Heater Drain Piping
Elevational Drawing
2A
SSRH
Drain
Tank
@ ---
784'-3"
52785'-9
To Condenser
787'-3"
--
784'-3-
784*-3"
2B
SSRH
2HD-92 2HD-93
Drain
Tank
785-9"
7859
To Condenser
-
784*-3"
787'-3"
-56
@ 813'-9"
784'-3"
782'-3"7
-43 @ 808'
~42' @ 806-6
2A1
FDWV
799'-6'
HTR
z
E
2A2
FOW
/787'-3"
HTR
FIGURE 1
Ie ,.12!.
MCK
gEoucck
lo"~ S-1-b 13.W
CA-P
N rT
C
FIGURE 2
UN~IT 2 PIPING IN AREA OF PIPE RUPTURE
UNIT 2 HEATER DRAIN LINE RUPTURE
r
MECHANISM IN PROGRESS
Ag'060
4026
S INITI AL
e
1
CONDI TIONSc
s
00INCRASEDCONDNSIN
94
~~ONDRENSE
OwNDNS
8 STEAM
AF TEi
/VOID7
DR
IN
0c
BEO INS
Hof
ep
1795
,
P067
SLUG ACCELERATING RAPIDLY
CONDENSATE
BEING SWEPT
INTO RISER
RAPID CONDENSATION OCCURRING
STEAM CAVITY COLLAPSE
IN PROGRESS
NOTE= PIPE LENGTHE' ARE APPROY1t-1-TE
9A
FIGURE. 3
UNITED STATES
00
NUCLEAR REGULATORY COMMISSION
REGION II
o!P
101 MARIETTA STREET, N.W., SUITE 2900
ATLANTA, GEORGIA 30323-0199
September 26, 1996
MEMORANDUM TO:
Thomas A. Peebles
Team Leader
Augmented Inspection Team
FROM:
Stewart D. Ebneter
-2
Regional Administrator
SUBJECT:
AUGMENTED INSPECTION TEAM CHARTER
An Augmented Inspection Team (AIT) has been established to inspect and assess
the moisture Separator/Reheater pipe rupture of September 24, 1996 at the
Oconee Nuclear Station. The team composition is as follows:
Team Leader:
T. Peebles
P. Harmon (Operator)
D. Forbes (Emergency Preparedness and
Radiological aspects)
N. Economos (Materials Application)
E. Brown (Water Hammer Events)
Others (To be determined on an as needed basis.)
The objectives of the inspectioh are to (1) determine the facts surrounding
the specific event, (2) assess licensee response to the event, (3)
assess
generic aspects of operations/inspections that may have broad applicability to
other facilities, (4) oversee licensee activity during their event review and,
(5) interface with other on-site entities such as Occupational Safety and
Health Administration (OSHA).
For the period during which you are leading this inspection and documenting
the results, you shall report directly to me. The guidance of Inspection
Manual Chapters 0325 and 0610 apply to your inspection and the report.
If you
have any questions regarding the objectives or the attached charter, contact
me.
Attachment: AIT Charter
cc w/att:
J. Milhoan. EDO
F. Miraglia, NRR
E. Jordan, AEOD
A. Thadani, NRR
S. Varga. NRR
H.
Berkow, NRR
APPENDIX A
AUGMENTED INSPECTION TEAM CHARTER
OCONEE NUCLEAR STATION
STEAM LINE RUPTURE EVENT
The objectives of the inspection are to (1) determine the facts surrounding
the specific event (2)
assess licensee response to the event (3)
assess
generic aspects of operations/inspections that may have broad applicability to
other facilities (4) oversee licensee activity during their event review and
(5) interface with other on-site entities such as Occupational Safety and
Health Administration (OSHA).
Monitor and review licensee activities related to event investigation
such as quarantine procedures, laboratory analyses of failed piping
materials, root cause analysis and precursor event reviews.
Develop a sequence of events associated with the Moisture
Separator/Reheater pipe rupture of September 24, 1996, at the Oconee
Nuclear Station.
Determine whether the pipe rupture event adversely affected safety
equipment availability or operability.
Determine the operational history of this piping system and whether or
not it had been subject to water hammer events in the past.
Assess past inspection applicable to the piping systems in question and
the applicability and effectiveness of the Erosion/Corrosion program at
the Oconee Nuclear Station. Determine status of the components of this
system in relation to the new maintenance rule 10 CFR 50.65.
Assess the startup (i.e., warm-up and heat-up) and power ascension
procedures and techniques with regard to the secondary systems in
question. Review corrective actions taken previously to minimize water
hammer or other operational events related to the system.
-Independently verify the status of radioactive contamination associated
with this pipe and determine of there was a release of contaminated
material.
Assess the licensee's performance related to emergency response, i.e.,
classifying this event. offsite notifications. on-site response and
interface with offsite emergency agencies.
Assess the licensee's overall technical response and activities to this
event.
Team leader shall interface with on-site regulatory entities/
authorities, such as Occupational Safety and Health Administration.
Offsite activities with regard to these entities shall be referred to
Region II.
Document the inspection findings and conclusion in an inspection report
within 30 days of the inspection completion.
ATTACHMENT
OVERALL
SEQUENCE OF EVENTS SEPTEMBER 24, 1996
UNIT 2 HEATER DRAIN LINE RUPTURE
September 23, 1996
Procedure OP/2/A/1106/14 (Moisture Separator/Reheaters), Enclosure 3.6 (Abnormal
Operating Conditions (1st Stage and 2nd Stage Reheater Drain Operation With 1st and 2nd
Stage Reheaters In Service)) has 2HD-91 and 2Hd-94 closed as an initial condition.
Use of this procedure enclosure was the result of an agreement between Operations and
Systems Engineering to prevent water hammers from occurring. This system evolution was
first performed July 22, 1996 on Unit 2 under Systems Engineering direction.
September 24, 1996
1130
Plant conditions prior to placinq Unit 2 Generator On-Line
Generator Load = 0 Mwe
2A1/2A2 Feedwater Heater Shell pressure = 0 psig
2A1/2A2 second stage reheater Tube Supply pressure = 0 psig
2B1/2B2 second stage reheater Tube Supply pressure = 0 psig
2A second stage reheater Drain Tank temp. = 136 degrees F
2B second stage reheater Drain Tank temp. = 155 degrees F
second stage reheater Feed Forward temperature to 2A Feedwater Heaters =
112 degrees F
1135
Unit 2 Generator placed On-Line, automatically picks up 5 percent load.
1140
Plant conditions after Unit 2 Generator On-Line
Generator Load = 42 Mwe
2A1/2A2 Feedwater Heater Shell pressure = 0 psig
2A1/2A2 second stage reheater Tube Supply pressure = 0 psig
2B1/2B2 second stage reheater Tube Supply pressure = 0 psig
2A second stage reheater Drain Tank temperature = 136 degrees F
2B second stage reheater Drain Tank temperature = 156 degrees F
second stage reheater Feed Forward temperature to 2A Feedwater Heaters =
114 degrees F
1147 Second stage reheater Drain Tank 2B high level alarms/clears.
1216 Second stage reheater Drain Tank 2A and 2B low level alarms.
1248
2A1 Feedwater Heater Inlet, (2HPE-6) open. Begins heating, pressurizing the 2A1
APPENDIX B
2
1310
Second stage reheater Drain Tank 2A level low level alarm clears.
1317 2HPE-10 (2A2 Feedwater Heater Inlet) open. Begins pressurizing the 2A2 Feedwater
Heater. Second stage reheater Drain Tank 2B low level alarm clears.
1336
Feedwater Heater 2A1 and 2A2 low level alarms cleared (level established in the 2A1
and 2A2 Feedwater Heaters).
1411
2MS-76 (TO 2A second stage reheater) and 2MS-79 (TO 2B second stage reheater)
not fully closed. (Throttled open by operators because 2MS-112 and 2MS-173 not
operating properly.)
1422
2HD-25 cycles not closed to closed.
1423
2HD-25 not closed.
1435
2HD-25 closed.
1441
Second stage reheater 2A Drain Tank low level alarm clears.
1448
Power escalation stopped to investigate 2B second stage reheater drain tank level
problems. This was due to high tank level alarm and 2HD-26 indication closed.
1500 Power escalation resumed. Work order written for 2HD-26.
1509:59
2HD-25 not closed.
1510:30
2HD-25 closed.
1512:46
2HD-25 not closed. NOTE: With 2A second stage reheater Drain Tank
operating off High Level Control, 2HD-92 will be open, but with the manual
isolation valve 2HD-91 still shut, feed-forward level control has no effect on
tank level.
1513:17
2HD-25 closed.
1610:53
2HD-26 not closed. NOTE: With 2B second stage reheater Drain Tank
operating off High Level Control, 2HD-95 will be full open.
1611:46
2HD-26 closed.
1612:02
2B second stage reheater Drain Tank level not high. NOTE: probable
adjustment of 2HD-26 setpoint occurred around this time. I&E called the
Unit 2 control room and reported they found 2HD-91 and 2HD-94 closed. I&E
suggested that this was the cause of the high level. The Unit 2 Shift
APPENDIX B
3
Supervisor and the Units 1 and 2 CR SRO decided to open 2HD-91 and 2HD
94 now at 500 Mwe rather than waiting for 600 Mwe as specified on the
Septemver 23, 1996, 1800 Worklist (night orders). There was no day shift
Worklist.
(NOTE: .Engineers had previously specified 600 paig in second stage reheater
to ensure no backflow from Feed Water heaters.)
1625:19
2HD-26 not closed.
1625:21
2HD-26 closed..
1630:00
After a pre-job briefing, Operators were dispatched to open 2HD-91 and 2HD
94 per OP/2/A/1 106/14 (MSRH), Enclosure 3.6 (Abnormal Operating
Conditions (1st Stage & 2nd Stage Reheater Drain Operation With 1st and 2nd
Stage Reheaters In Service)). It was stressed to the Operators to slowly open
these valves.
Current plant conditions:
Generator Load = 500 Mwe
2A1/2A2 Feedwater Heater Shell pressure = 251 psig
2A1/2A2 second stage reheater Tube Supply pressure = 254 psig/266 psig
2B1/2B2 second stage reheater Tube Supply pressure = 250 psig/240 psig
2A second stage reheater Drain Tank temperature = 400 degrees F
2B second stage reheater Drain Tank temperature = 357 degrees F
second stage reheater Feed Forward temperature to 2A Feedwater Heaters =
237 degrees F
1640:30
Probable opening time of 2HD-91 and 2HD-94. Back flow from 2A Feedwater
Heaters can be observed from Operator Aid Computer, alarm typewriter data.
Interviews indicated that both 2HD-91 and 2HD-94 were throttled 1/4 open
simultaneously. Post accident, this was found to be 12-13 turns open. 1/4
open on this gate valve would allow for essentially full flow, depending on d/p
across the valve. Interviews indicate it took 3-4 minutes to open these valves
to 1/4 open. At that time then the line lunged to the East, a steam leak was
heard and then the line ruptured.
NOTE: during the 3-4 minute period required to open the valves to the 1/4
open point, no line movement or noise was heard.
1640:34
2HD-26 not closed. Level in 2B second stage reheater Drain Tank is
increasing, probably from backflow.
APPENDIX B
4
Current plant conditions:
Generator Load = 498 Mwe
2A1/2A2 Feedwater Heater Shell pressure = 250 psig
2A1/2A2 second stage reheater Tube Supply pressure = 254/265 psig
2B1/2B2 second stage reheater Tube Supply pressure = 250 psig/240 psig
2A second stage reheater Drain Tank temperature = 403 degrees F
2B second stage reheater Drain Tank temperature = 372 degrees F (dec)
second stage reheater Feed Forward temperature to 2A Feedwater Heaters =
237 degrees F (inc)
1640:42
2HD-26 closed (Increased frequency of 2B second stage reheater Drain Tank
dump valve cycling indicates increased backflow into Tank from 2A Feedwater
Heaters.
1640 to 1641 2HD-26 cycled seven times. This further substantiates that additional water
volume was coming into the 2B second stage reheater Drain Tank.
1642
2A2 Feedwater Heater Level Low.
Current plant conditions:
Generator Load = 495 Mwe
2A1/2A2 Feedwater Heater Shell Pressure = 238 psig
2A1/2A2 second stage reheater Tube Supply Pressure = 252 psig/263 psig
2B1/2B2 second stage reheater Tube Supply Pressure = 247 psig/237 psig
2A second stage reheater Drain Tank temperature = 403 degrees F
2B second stage reheater Drain Tank temperature = 338 degrees F (dec)
second stage reheater Feed Forward Temperature to 2A Feedwater Heaters =
290 degrees F (inc)
Simultaneous high and low level alarms on both the 2A and 2B second stage reheater
Drain Tank. The alarms cleared and the 2A second stage reheater Drain Tank high
and low level alarms returned within approximately 20 seconds. The alarms cleared
and the 2A second stage reheater Drain Tank high and low level alarms returned with
approximately 20 seconds. (utility typer)
1642:16
Lube Oil Purifier 2A tripped. NOTE: Steam cloud is theorized as actuating
sprinkler which trips the 2A and 2B lube oil purifiers. 2B second stage
reheater Drain Tank temperature reached its minimum and started increasing.
APPENDIX B
5
Current plant conditions:
Generator Load = 495 Mwe
2A1/2A2 Feedwater Heater Shell pressure = 234 psig
2A1/2A2 second stage reheater Tube Supply pressure = 251 psig/262 psig
2B1/2B2 second stage reheater Tube Supply pressure = 246 psig/236 psig
2A second stage reheater Drain Tank temperature = 403 degrees F
2B second stage reheater Drain Tank temperature = 333 degrees F (inc)
second stage reheater Feed Forward temperature to 2A Feedwater Heaters =
297 degrees F (inc)
1642:53
2A second stage reheater Drain Tank Level Low. 2B second stage reheater
Drain Tank Level High.
Current plant conditions:
Generator Load = 486 Mwe
2A1/2A2 Feedwater Heater Shell pressure = 224 psig
2A1/2A2 second stage reheater Tube Supply pressure = 243 psig/254 psig
2B1/2B2 second stage reheater Tube Supply pressure = 238 psig/228 psig
2A second stage reheater Drain Tank temperature = 403 degrees F
2B second stage reheater Drain Tank temperature = 355 degrees F (inc)
second stage reheater Feed Forward temperature to 2A Feedwater Heaters =
320 degrees F (inc)
1642:55
Second stage reheater Drain Tank Level alarms clear.
1643
The Unit 2 Shift Supervisor exited the Units 1 and 2 control room. When he
(heard/saw) the steam in the Turbine Building 5th floor, he returned to the
control room. He then directed the Unit 2 Reactor Operators to trip the
Reactor manually and to isolate steam to the 1st and 2nd Stage Moisture
Separator/Reheaters.
1643:18
2MS-76 and 2MS-79 closed. This isolates steam to the Moisture
Separator/Reheaters, second stage reheaters.
1643:25
Reactor manually tripped.
1643:35
2B second stage reheater Drain Tank level low. 2HD-95 failed open due to
loss of its instrument air lines. This allowed the 2B second stage reheater
Drain Tank to blow down.
1704
Attempted to isolate break from 2A Feedwater Heaters. 2HD-106 closed but
2HD-105 did not.
APPENDIX B
6
1705
Exited Emergency Operating Procedure.
2040
Notification of Unusual Event declared, terminated, based on "explosion within plant
resulting in visible damage to permanent structures/equipment." Terminated after
deciding that the damaged equipment had no effect on ability to reach/maintain safe
shutdown.
0
9
APPENDIX B
SPECIFIC ACTIVITY SEQUENCE
IMMEDIATELY BEFORE AND AFTER
UNIT 2 HEATER DRAIN LINE RUPTURE
1.
Non-licensed Operators OPEN 2HD-91
Nothing happens because the 2A second stage reheater Drain Tank
temperature was 403 degrees F (Psat approximately 242 psig).
2A Feedwater Heater was 250 psig which was close enough to 2A second
stage reheater Drain Tank Psat so there was no D/P across 2HD-91.
(Interviews stated 2HD-91 was easy to operate).
There were no water hammers because there was not enough D/P to force
flow through the lines in either direction.
2.
Non-licensed Operators start opening 2HD-94
They had to "tug" on the valve chain to operate the valve which indicates a
D/P across 2HD-94.
2B second stage reheater Drain Tank temperatures was 372 degrees F (Psat
163 psig).
2A Feedwater Heaters 250 psig resulting in a 90 psig reverse D/P.
With 2B second stage reheater Drain Tank pressure less than 2A feedwater
heaters' shell pressure and adequate D/P to force flow, there was reverse flow
from the 2A Feedwater Heaters to the 2B second stage reheater Drain Tank.
This corresponds to a decrease in 2B second stage reheater Drain Tank
temperature.
After approximately 2 minutes of back flow into 2B second stage reheater
Drain Tank, the line ruptured.
3.
When the line ruptured:
2HD-95 failed open on a loss of control air pressure (Instrument air line blown
off).
With 2HD-95 failed open, there was a blow down path from both the 2A1 and
2A2 Feedwater Heaters, the 2A and 2B second stage reheater Drain Tanks,
and the second stage reheater Tube bundles for all four Moisture
Separator/Reheaters.
APPENDIX C
2
4.
When 2MS-76 and 2MS-79 were closed at 1643:18, this isolated steam to the second
stage reheater Tube bundles. As 2A second stage reheater Drain Tank level
decreased, 2HD-92 closed isolating the 2A second stage reheater Drain Tank from
the break. 2B second stage reheater Drain Tank continued to blow down until it was
completely empty at 1643:35.
5.
When the reactor was tripped, it manually isolated extraction steam to the 2A
feedwater heaters and they continued to blow down.
APPENDIX C
PROCEDURE AND SYSTEM HISTORY
MOISTURE SEPARATER REHEATER DRAIN LINES
NOTE: Water hammer history and events were documented in-depth for only one year.
Other numerous water hammers were routine from beginning of plant operation.
Moisture Separator/Reheaters Procedure and System History
1.
October 2, 1973 - Original Procedure OP/2/A/1106/14 issued. No specific guidance
for feeding forward second stage reheater and first stage reheater drains.
2.
August 21, 1975 - Change #1 to procedure. General Electric recommendation.
Heater drains (first stage reheater and second stage reheater) routed forward to the
'A' (Highest pressure) feedwater heaters to provide part of final heating of Main
3.
December 28, 1977 - Reissued procedure requirement added for turbine to be above
300 MWe prior to aligning second stage reheater drains to 'A' Feedwater heaters
(feed forward).
4.
= 1980- First stage reheater drain, 10 inch pipe, into 18" header cut off and rerouted
to flash tanks.
5.
November 3, 1992 - Revision 23, Guidance in procedure (Enclosure 3.8 Abnormal
Operating Conditions, Startup of first stage reheaters and second stage reheater at
Power) to require Moisture Separator/Reheaters Tube Supply Steam Pressures above
the A/B feedwater heater extraction steam pressure. (Prior to this, the entire system
was brought on-line with the drain aligned to the "feed forward" mode. Guidance also
included to slowly open HD-91, 94 valves when aligning to "feed forward". This same
guidance is not in Enclosure 3.6, Reheater Drain Operation, used to align first stage
reheater, second stage reheater drains to feed forward/dump back to condenser.
6.
July 24, 1994 - Problem Identification Report 94-1001 - Evidence of water hammer on
Unit 1 second stage reheater Drain Tank during transition of feed forward mode.
(Lines shook, lagging shaken loose) Dump back to condenser periodically opened,
causing control interaction with feed forward level controls. "B" side lines continued to
swing two feet side to side.
7.
May 9, 1995 - Problem Identification Report-1 -95-513 - Water Hammer on 1B second
stage reheater drain system broke drain tank supports. Problem Identification Report
evaluation revealed excessive heatup rates, thermal transients on all three units'
second stage reheaters, first stage reheaters, and Moisture Separator/Reheaters.
(=260 degrees F/30 minutes versus 50 degrees F/30 minute limit) Problem
Identification Report corrective actions included reevaluation of second stage reheater
Tank and Line Supports, replacing level control/dump valves and control systems,
APPENDIX D
2
6
development of Nuclear Station Modification 2941 for replacing MS-112, 173 valves
and control systems, changes to Procedure OP/1106/14 to implement restrictive
heatup rates (30 degrees/30 minutes) per vendor.
8.
April 27, 1995 - Problem Identification Report-95-0457 - During Unit 1 Startup, wrong
Enclosure to Procedure OP/A/1106/14 used for securing "feed forward" mode,
resulted in Moisture Separator/Reheaters relief actuation.
9.
September 21, 1995 - Change #24 changed ambiguous enclosure titles to prevent
use of wrong enclosure addressed in Problem Identification Report-95-0457.
10.
October 1995 - Nuclear Station Modification 2941 Scoping document issued.
11.
December 1995 - Unit 1 Startup using revised OP/A/1106/14 using Manual loading of
MS-112, 173. No water hammer, but heatup rates close to 100 degrees F/hour.
Operator burden (local air-loading MS-112, 173 while Control room operator in contact
with info on temperature change) deemed too great. Recommended Nuclear Station
Modification implementation to revise valve/control design change.
12.
May 1996 - Temporary modification TT/2/B/0271/009 setup and control of Moore
Controller for 2MS-1 73 developed to test capability of manual loading of MS-1 12, 173
using new, digital controller. Attempt to see if controller change out alone could
provide acceptable steam admission rates to Moisture Separator/Reheaters.
13.
May 4, 1996 - Revision #26 to OP/2/A/1106/14, Changes to:
a.
Allow manual loading of 2MS-1 12, 173 up to a point (350 degree F second
stage reheater Tube Temperature) at which time the Auto Controller will
control adequately by its control program to prevent exceeding second stage
reheater Heatup Rate Limits.
b.
Remove Loading Curve from procedure because of conflict with second stage
reheater Heatup Limits for normal power escalation rates and initially valve-in
Main Steam to second stage reheater with second stage reheater Tube
Temperature 50 degrees F> LP Turbine Inlet Temperature as discussed and
recommended by Engineering.
c.
Initiate MS to second stage reheater later (after T-G on-line at approximately
200 Mwe) to reduce Operator burden of placing second stage reheater in
service in Manual during T-G Startup and at advice of Engineering (Main
Steam to second stage reheater's not needed at low loads and there is an
increased potential for introducing unnecessary thermal stress to the second
stage reheater tube bundles).
APPENDIX D
3
14.
May 7, 1996 - OP/2/A/1106/14 Moisture Separator/Reheaters Operation, in
conjunction with TT/2/B/0271/009, performed for Unit 2 Post Refueling outage Startup
of second stage reheater's. Results were acceptable for controlling second stage
reheater Heatup Rate and showed that replacement of the MS-112, 173 Control
System with minor valve trim replacement for improved seat leakage was acceptable.
However, a severe water hammer was introduced in the second stage reheater Heater
Drain System during the second stage reheater startup sequence that broke a support
at the junction of the two second stage reheater Drain Tank feed forward pipes. This
water hammer was later found to be caused by 2HD-26 failing open while the normal
level control valve 2HD-95 to FEED WATER Heater 2A also OPEN. (Reverse flow
from heaters to condenser.) (Problem Identification Report-2-96-0984)
15.
July 1996 - The data analysis for the Unit 2 5-96 second stage reheater Startup was
complete and it was concluded that the water hammer was initiated by backflow from
the 2A High Pressure Feedwater Heaters through the failed-open 2A second stage
reheater Drain Tank Dump Valve. The second stage reheater Drain Tank Pressure
could not be maintained greater than the 2A High Pressure Feedwater Heater. This
confirmed the argument for placing Check Valves in these Drain Lines, and the
Nuclear Station Modification-2941 scope was changed accordingly.
16.
July 16, 1996 - A meeting was held between Engineering and Operations
representatives to discuss the water hammer phenomenon in the second stage
reheater Drain Pipes and to reach consensus on recommendations for short-term and
long-term Corrective Action (later addressed in Problem Identification Report-2-96
0984). This meeting was prompted by the Unit 2 upcoming Startup and the fact that
procedural changes (not yet identified in the incomplete Problem Identification Report)
may be required in support of the Unit 2 Startup. The following is a summary of the
discussions and outcome resulting from that meeting:
a.
Described the May 7, 1996, Unit 2 Water Hammer event that was caused by
backflow in the second stage reheater Heater Drain feed forward pipe when
second stage reheater Drain Pressure was less than High Pressure Feedwater
Heater Shell Pressure.
b.
Described how second stage reheater Drain Pressure may be less than High
Pressure Feedwater Heater Shell Pressure; dependent on MS-1 12, 173
operation and second stage reheater Dump Valve operation (it was stated that
a second stage reheater Dump Valve open failure can lead to depressurization
of second stage reheater Drain and initiation of backflow from the High
Pressure Feedwater Heaters).
c.
Described that the most significant water hammers can be eliminated by
preventing backflow in the second stage reheater Heater Drain System by
maintaining second stage reheater Pressure greater than "A" High Pressure
Feedwater Shell Pressure (with additional consideration of static head in the
second stage reheater Drain Pipe).
APPENDIX D
4
d.
Described further that initial slow admission of Main Steam to second stage
reheater's via MS-1 12, 173 should help to minimize water hammer as
evidenced in the previous Unit 1 Startup.
e.
Described the Long-Term Solutions proposed for Nuclear Station Modification
2941: Proposed addition of Check Valves in the second stage reheater Drain
Pipes to prevent backflow conditions. Change-out of MS-1 12, 173 trim for
positive seating and finer control. Change-out of MS-112, 173 Control System
for allowing finer control and with possible new control logic for minimizing
excessive heat-up rates.
f.
Opened discussions of a Short-Term Solution for second stage reheater
Startup to prevent backflow. The following options were discussed and
evaluated:
(1)
Manually Close HD-91 and HD-94 prior to any second stage reheater
Startup (following Shutdown). Then Manually and slowly throttle HD-91
and HD-94 open during the Startup Sequence while maintaining second
stage reheater Pressure greater than 'A' Heater Shell Pressure (with
margin) with MS-112, 173.
(2)
Manually Close isolation valves downstream of level control valves HD
92, HD-95, HD-105 and HD-106, prior to any second stage reheater
Startup (following Shutdown). Then Manually and slowly throttle HD
105 and HD-106 open during the Startup Sequence while maintaining
second stage reheater Pressure greater than 'A' Heater Shell Pressure
(with margin) with MS-112, 173.
(3)
Maintain second stage reheater Pressure greater than 'A' Heater Shell
Pressure (with margin) by slowly throttling MS-112, 173 open and
communicating with the Control Room prior to and during Main Turbine
roll and throughout the startup sequence using a similar procedure to
the current Unit 1 OP/1106/14 Moisture Separator/Reheaters.
g.
Option 3 was ruled out as not having enough positive control to prevent
backflow (i.e., not having the closed manual isolation valves in the second
stage reheater Heater Drain System and due to the Operator burden and the
extreme attention required while using a Main Steam Admission and Drain
Tank level control system known for inaccuracy and potential failures).
h.
Option 1 was modified when it was concluded that there would be optimum
control if the throttling of HD-91, 94 began at a second stage reheater
Pressure greater than the Design Pressure of the 'A' Heater Shell Pressure
APPENDIX D
5
which was shown to be 500 psig. A second stage reheater Pressure of
600 psiq was chosen for conservatism as the starting point for throttling open
HD-91, 94 to ensure a positive second stage reheater drain flow from any
operating condition.
Option 2 was discussed with personnel safety, compared to Option 1, initially
considered as one of its benefits. It was later ruled out based upon the
thought that these valves would be difficult (or impossible) to open manually
with a differential pressure across the valves and because of their potentially
confining location that might have been less safe than that presented in
Option 1.
j.
Consensus was reached in utilizing the Modified Option 1, requiring 600 psiq
second stage reheater Pressure, prior to slowly throttling HD-91, 94 open to
valve in second stage reheater Drains to the 'A' High Pressure Feedwater
Heaters.
k.
A discussion followed on how best to initiate this change. It was discussed
that a procedure change to the existing Procedure OP/2/A/1106/14 would be
necessary to include the requirements of item (j). It was brought to the table
that there already existed an Enclosure in the procedure that directed manual
operation of the second stage reheater feed forward drains and that it could be
utilized with additional instruction to perform the desired steps for manually
valving in second stage reheater drains to 'A' Feedwater Heaters. It was also
stated that the OP/1106/14 Procedures for all three units were undergoing re
write. Because of these circumstances, consensus was reached to utilize the
Unit Operations Daily Worklist until such time that the procedure rewrites were
completed for the additional directions necessary to ensure that HD-91, 94
were isolated prior to startup and that after reaching 600 psig second stage
reheater Pressure, HD-91, 94 would be slowly throttled open per the existing
OP/1106/14 Enclosure and procedure to introduce second stage reheater
Drain feed to 'A' Feedwater Heaters.
NOTE: Administrative Procedure OMP 1-9 prescribes the method to be used
to place a procedure on Administrative Hold to prevent its use prior to
incorporating the necessary changes. The decision to rely on non-controlled
processes (verbal precautions, Daily Worklist) resulted in use of a procedure
that was not adequate. Those items were covered adequately during the
July 22, 1996 system startup when system engineers accompanied the
operators, but not for the September 24, 1996 event.
I.
It was agreed that Engineering would be present to observe the Unit 2 second
stage reheater Startup sequence, including the manual feed forward of second
stage reheater drains to the 'A' Feedwater Heaters.
9
APPENDIX D
6
17.
July 22, 1996 - Unit 2 startup accomplished with no problems during transition to feed
forward mode. Engineering accompanied operators, drain tank d/t, d/p maintained
closely. Also, operators cracked open 2HD-91, 94 much less than 1/4 open, then
waited approximately 40 minutes before opening further. (Operators left area for
"coffee break" to allow slow, thorough heatup and equalization.)
18.
Operators charged with revising OP/A/1106/14 to incorporate precautions used during
July 21, 1996 startup and July 5, 1996 meeting between OPERATIONS/Engineering.
The revision due date was October 14, 1996. No effective communication to shift
crew on September 24, 1996 startup. Only reference is "slow" opening of 2HD-91,
94, erroneous reference to "600 Mwe" vs agreed upon value of 600 psig.
5
APPENDIX D
THE LICENSEE REVIEW TEAMS
Event Investigation Team
G. Gilbert - NSRB staff - Team Leader
C. Curry - Operational Assessment, NGO - Assistant Team Leader
M. Pyne - Mechanical Equipment Engineering, NGO
A. Buzhardt - Safety and Industrial Hygiene, NGO
M. Langel - Mechanical/Civil Equipment Engineering, MNS
S. Shillinglaw - Security (MERT), CNS
J. Bryant - Safety Review Group, ONS
C. Goslow - INPO, Event Analysis
F. Krauss - Operating Experience Assessment, NGO
K. Wilmer - Corporate Safety and Industrial Hygiene
Failure Investiqation Process Team
D. Coyle - Investigation and Recovery Coordinator
B. Millsaps - Management Oversight
B. Dobson - Management Oversight
T. Royal - Supervisory Direction
B. Heineck - Supervisory Direction
P. Fisk - Investigation Lead
N. Watson - Investigation Lead
G. Lareau - MSE
.
A.
Park - MSE
H. Harling - SME
V. Bowman - Pressure Transient
M. Haynes - Operations
D. Smith - Operations
R. Bowman - Operations
B. Jones - Operations
M. Miller - I&C
D. Phelps - I&C
B. Davis - Documentation
K. Alter - Documentation
C. Arnold - Outage Team Liaison
D. Kelley - Recovery Team Engineering Supervisor
S. Anderson - Metal Analysis
T. Brown - Stress Analysis
M. Kelly - Vibration
APPENDIX E
5
EXIT ATTENDANCE
Licensee
J. Hampton, Vice President, Oconee Site
B. Peele, Station Manager
D. Coyle, Systems Engineering Manager
J. Davis, Engineering Manager
T. Coutu, Operations Support Manager
W. Foster, Safety Assurance Manager
B. Millsaps, Engineering Manager
D. Hubbard, Maintenance Superintendent
E. Burchfield, Regulatory Compliance
C. Little, Electrical Systems/Equipment Manager
J. Smith, Regulatory Compliance
G. Ridgeway, Acting Operations Superintendent
NRC
J. Jaudon, Deputy Director, DRS
M. Scott, Senior Resident Inspector, Oconee
O
N.
Economos, Reactor Inspector, DRS
T. Peebles, Augmented Inspection Team leader
APPENDIX F