IR 05000272/1988002
| ML18093A699 | |
| Person / Time | |
|---|---|
| Site: | Salem  |
| Issue date: | 02/11/1988 |
| From: | Eugene Kelly, Swetland P NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML18093A698 | List: |
| References | |
| 50-272-88-02, 50-272-88-2, 50-311-88-02, 50-311-88-2, NUDOCS 8803040097 | |
| Download: ML18093A699 (30) | |
Text
I
.
-
U. S. NUCLEAR REGULATORY COMMISSION
REGION I
Docket and Report No /88-02 50-311/88-02 Licensee Nos.:
DPR-70 DPR-75 Licensee:
Public Service Electric and Gas Company 80 Park Plaza Newark, New Jersey 07101 Facility:
Salem Nuclear Generating Station, Units 1 and 2 Location:
Hancocks Bridge, New Jersey Dates:
January 4-7, 1988 Team Members:
Daniele Oudinot, NRR Events Analysis Branch Harold Gregg, Senior Reactor Engineer, Region I Glenn Meyer, Project Engineer, Region I Approved Section 2B Inspection Summary:
7/;( f?Jz Date A special team inspection (130 hours0.0015 days <br />0.0361 hours <br />2.149471e-4 weeks <br />4.9465e-5 months <br />) was conducted by a group consisting of two inspectors, a representative from the NRR Events Analysis Branch, and a team leade The team performed detailed inspections in the areas of operations, maintenance, surveillance, and engineering suppor The team independently confirmed the sequence of events of a service water flooding of Unit 1 Bay Number 1 on December 22, 198 The team assessed the root cause and safety significance of the event, the restoration and maintenance of affected equipment, and the design adequacy of service water isolation valve No violations were identified. A summary of the inspection team's conclusions is contained in Detail Specfic additional corrective action items are outlined in Detail The licensee committed to a special report to be submitted to NRC Region I when the results of analyses on the failed 13SW20 valve are availabl PDf~
ADDCK 05000272 G
'.
TABLE OF CONTENTS Page 1. 0 Summary...............................................................
2.0 Introduction.......................................................... 5 2.1 Scope............................................................
2.2 Team Composition................................................. 5 2.3 Summary of Conclusions and Additional Actions.................... 5 3.0 Event Description (Figure I-System Configuration at Time of Event).... 7 4.0 Service Water Design.................................................. 9 System Description (Figure 2-Normal Flow Schematic).............. 9 4.2 Isolation Valve Application (Figures 3 and 4-Valve Details)...... 10 Flow Requirements and Leakage Tolerance.......................... 12 4.4 Conclusions...................................................... 13 5.0 Service Water Surveillance............................................ 13 Inservice Testing................................................ 13 5.2 Effect of Hydrostatic Testing.................................... 15 Pump Time Response Testing....................................... 15 5.4 Conclusions...................................................... 16 6.0 Service Water Maintenance............................................. 16 Valve Maintenance History (Figure 5-Pictures).................... 16 Bay Number 1 Restoration......................................... 17 Post-Maintenance Testing......................................... 18 6.4 Conclusions...................................................... 18 7.0 Service Water System Interactions..................................... 18 8.0 Management Activities................................................. 18 Recovery Team and Safety Committees.............................. 18 Exit Meeting..................................................... 20 Followup of Findings............................................. 20 ATTACHMENTS: Detailed Chronology Personnel Interviewed Management Meeting Attendance
\\ '
-3-1.0 Summary Salem Unit l experienced a flood in the Number l Service Water Bay on December 22, 1987, which resulted in a spilling of an estimated 144,000 gallons of water and submersion of service water equipment for approximately three to four hour Unit l was in Mode 5, Cold Shutdown, at the time of the flood event and had been in a refueling outage for approximately two months prior to the floo Reactor coolant temperature was 114 degrees F, and the primary system was vented to the containment, which was pressurized to 47 psig for a Type A leak rate tes Two independent service water headers are normally available for Unit l, but only one header is required during refueling operations for shutdown coolin The Number 11 nuclear service water header was isolated at the time for repair work being conducted in Number 1 Service Water Ba Service water was also isolated to the turbine building using closed and tagged 30-inch butterfly isolation valves (13SW20 and 1SW26) which are designed to isolate safety related cooling loads from non-safety load Blank flanges near these isolation valves, which were installed to perform a ten year in-service inspection hydrostatic test of the service water system, were left on but loosened at two location The flanges were loosened to provide leak paths for small amounts of service water known to be leaking past the isolation valv One valve, 13SW20, was providing isolation from the open flange In addition, a three-inch connection was open downstream of the isolation valves to perform a weld repair neces-sitated by a leak found during the hydrostatic test on that lin The apparent root cause of this event was found to be the partial disen-gagement of the rubber seat ring on service water isolation valve 13SW2 The degraded seat ring, which provides a seating surface for the valve disc, was found to be torn from its normal position over approximately 18% of its circumferenc The licensee plans to perform an analysis of the failed valve seat ring and surface, and suspects that a similar failure found in August 1984 on the same valve (attributed to erosion and corrosion within the service water system) aggravated the failure on December 22, 198 The failure of the valve seat ring also occurred approximately two to three hours after time response testing had been performed for the service water pump However, the testing of the pumps was concluded to not be a direct cause for the failure of the 13SW20 valve seat rin Service water leaked past the 13SW20 valve and flooded through an open flange immediately downstream of that valv Flooding also occurred through the other isolation valve (1SW26) which was approximately one-half inch to one inch off of its seat, although indicating full close The leakage past the 1SW26 valve also allowed leakage out of the three-inch open connection being weld repaired, as well as into the turbine building by way of another loose blank flang The apparent cause of the partially open 1SW26 valve was an incorrectly set limit switch/mechanical stop setting, which incorrectly indicated full closure.
,,
-4-The inspection team concluded that the configuration of the service water system at the time of the flooding was an unusual lineup that would be only expected during maintenance or repair activities, but not during reactor operatio As a result of the flooding, the permanently installed sump pump was unable to keep up with leakage, and the bay flooded to within one foot of the ceilin Larger capacity portable sump pumps were installed in Bay Number 1 approximately one hour after the flood began and ultimately removed flood water from the ba Approximately eight hours after the beginning of the flood, service water inleakage was essentially stopped by the installation of a mechanical plug in the three-inch connection open for previous weld repai Unit 1 had been in the second month of a planned refueling outage at the time, and because of the low decay heat removal loads, reactor safety was never a factor during the even The relatively low amount of decay heat present was easily handled with the Number 12 nuclear service water header using one service water pump (Number 13).
Under normal operational conditions, or with higher decay heat loads, the maintenance repair configuration that the system was in (i.e. flanges installed and loosened)
would not be expecte Service water bays were found to be appropriately separated such that flooding in one bay would not cause flooding in adjacent bays or affect any other safety related equipmen Plant operators and station staff responded appropriately to the event and adequately diagnosed and corrected equipment failures and malfunction Therefore, no adverse safety effects on the station were realize Recovery efforts to drain down and replace or refurbish Bay Number 1 equipment were well planned and completed within approximately ten day Because of previously identified problems with corrosion and erosion in the service water system, Salem Nuclear Station has embarked on an extensive materials replacement program for piping and valves in that syste A presentation was made to the NRC in the Region I office on September 29, 1987, describing the licensee's project to replace service water materials with higher corrosion resistent molybdenum stainless steel pipin Therefore, this team inspection did not address previous problems with corrosion and the associated programs to address those concern Valve 1SW26 was properly reset for limit switch and stop setting Valve 13SW20 was replaced with a new valve of the original desig Both valves were then successfully leak-teste The team noted that all large butterfly valves used in the service water bays will be replaced by the next cycle of operation for both unit This will involve 14 ~alves in both unit The newer valves are of a different type with better corrosion resistance, better seating capabilities, and are therefore less likely to leak or corrod Because of concerns for the service water system, its importance to overall plant risk, and the possibility of a common mode failure of
.
'
-5-existing valve materiais, this special inspection team review of the flooding event was performed during the period January 4 through 7, 198 No violations of regulatory requirements were identifie The team noted several areas where the licensee had planned further action or committed to review the need for additional corrective action These actions (described in 2.3, below) included in part to setting of limit switches and mechanical stops, and leak rate criteria and testing for the service water system valves in questio.0 Introduction 2.1 Scope of Inspection In response to the flooding of Service Water Bay 1 at Salem Station on December 22, 1987, the NRC formed a special team to confirm the sequence of events, determine the root cause, and assess the safety significance of the event The team established the chronology of the event, reviewed service water equipment performance both prior to and during the event, reviewed maintenance to restore equipment in Bay 1 including post-maintenance testing and assessed the adequacy of the service water system designed basis, particularly the function of the isolation valves which failed during the flood even The NRC team leader and a team member visited the Salem site on December 30, 1987, to obtain information pertinent to the flood event and to observe the condition of Bay The team then held an initial. planning meeting on the morning on January 4, 1988 and arrived on site later that day to begin the inspection which lasted through January 7, 198 Attendees at the entrance and exit interviews are listed in Attachment Individuals formally interviewed by the NRC team during the course of the inspection are listed in Attachment.2 Team Composition The team was composed of a team leader, two NRC Region I specialists, and a representative from the NRC Office of Nuclear Reactor Regulation (NRR) Events Analysis Branc The team had expertise.in plant main-tenance, valve design, and systems operatio.3. Summary of Conclusions and Additional Actions The consensus of the team with respect to cause, safety significance and licensee response to the December 22, 1987 flooding event was:
The cause of the event was an apparent random failure of the seat of isolation valve (13SW20) coincident with an off-normal alignment of the service water syste This line up was necessary to support ongoing maintenance activities, in which
-6-valve 13SW20 was relied upon as the only isolation protection between the operating and inactive (breached) portions of the syste The team concluded that the service water system configuration at the time of the event was unusu~l and would not occur during normal power operation The event itself did not result in a significant loss of equipment necessary for operations in the plant conditions at the tim The design of the facility precludes a total lnss of service water for this type of event. Accordingly, throughout the event and recovery, sufficient service water from the redundant bay existed to supply the safety related needs of the plan The safety significance of similar valve failures (e.g. seat failure of one isolation valve coincident with a partially open redundant valve due to incorrect limit settings) during normal plant operations was also judged to be lo The failure could have resulted in loss of only a small portion of the safety-related service water flow to the non-safety related part of the syste Ample service water flow was verified to support all safety-related components needed to respond to design basis event The operating staff quickly recognized the flood situation and took appropriate mitigating action Reactor plant safety was always maintained and never in jeopard Recovery actions to reestablish equipment operability in the bay were appropriate and well organize The team also confirmed at the exit meeting on January 7, that the licensee would address the following additional items, identified during the course of the inspectio The licensee committed to include these items in a special report to NRC Region I when the results of the analysis of the failed valve (13SW20) internals are complete Evaluate the feasibility of leak rate testing the essential to non-essential service water isolation valve Develop a limit for allowable leakage for the essential to non-essential service water isolation valve Determine more appropriate guidance for maintenance procedure M3L for setting the limit switches and stops on the large service water isolation valves and similar valve Determine the actual failure mode of the failed seat ring on valve 13SW20 and assess whether the failure is related to a prior maintenance activity, due to excessive corrosion, conducted in August 198 \\.
-7-3.0 Event Description Unit 1 had been in a refueling outage since November 2, 1987, and was scheduled to startup the last week of Decembe The unit was in Mode 5, Cold Shutdown, and reactor coolant temperature was 114 degrees One nuclear header of the nuclear service water system was in service with one pump feeding a component cooling heating exchange Decay heat levels were very lo The Number 11 nuclear service water header was isolated for repair work on three valve In addition, service water to the turbine building was also isolated using valves 11SW20, 1SW26, and 13SW2 Another isolation valve, ST-1, located in the turbine building had also been removed and a loose blank flange was installed. The blank flanges had originally been installed for a 10 year IS! hydrostatic test which had been completed during the prior wee The flanges were left on but loosened to provide a leak path to facilitate draining of the piping for subsequent welding on a three inch line downstream of the isolation valves (and in Bay 1) but upstream of the ST-1 valve in the Turbine Buildin A controlled leakage path was required because there was noticeable leakage (estimated to be on the order of 1-2 gpm) past isolation valve 13SW20 and the area in the vicinity of the weld repair had to remain dr The weld repair was on a three inch connection found to be leaking as part of the ISI hydrostatic tes Just prior to the flood event, which was estimated to occur at approximately 7:37 p.m. on December 22, testing was in progress on five of the six Unit 1 service water pump The testing (described in Detail 5.3.1) involved a brief start and time response test of each pum The testing raised header pressure by approximately 20 pounds for a few second The testing was performed between 4:00 and 6:30 p.m. on December 22 after which all but the Number 13 service water pump were taken out of servic The con-figuration of the service water system at a time just prior to the flood event is depicted in Figure An operator had been dispatched to Bay 1 to check on leakage every hour since a sump alarm had been intermittently annunciating because of controlled leakage during the previous 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> Approximately 15 minutes before the flooding began, an operator had been in Bay 1 and reported no overflow condition The first indication that larger leakage had occurred was when a turbine area sump alarm was annunciated at 7:37 p.m, the time that the licensee considered that the flooding bega Subsequent turbine sump alarms came in 12 minutes later at which time control room operators dispatched a secondary operator to investigate the second sump alar Upon arriving in the turbine building area where the sump was located, the operator found water leaking by the loose blank flange immediately upstream of a nonsafety related isolation valve (ST-1) in the Turbine Buildin The operator reported the leakage to the control roo In the next fifteen minutes, additional turbine area sump alarms were annunciate At 8:14 p.m., a fire watch coincidentally on a routine patrol near the service water structure entered Bay 1 for approximately
-8-15 seconds, noted the flooding condition, and reported the flood to control room operator An operator responded in approximately eight minutes to the initial report of flooding, briefly entered Bay 1, and subsequently reported the conditio The source of the flooding was not immediately known by control room personne However, operators secured the Number 13 service water pump physically located in Bay 1 to protect it from floodin The Number 15 service water pump located in redundant Service Water Bay 3 was started to continue service water supply to the number 12 nuclear heade Operators then shut the cross-connect valves between the 11 and 12 nuclear service water headers in attempt to isolate the lea However, these actions were unsuccessful in terminating the source of the flooding, which was later found to be from two sources:
the blank flange in the cross-over piping between Bay Numbers 1 and 3 and located in Bay l*(downstream of valve 13SW20); as well as from the three-inch open connection that had experienced previous weld repair By 8:15 p.m., a 600 gallon per minute portable sump pump was put in service at Bay 1 in an attempt to pump the water in the bay back into the rive However, the pump was unsuccessful in keeping up with the leak, later estimated by the licensee to be 800 gallons per minut By 9:30 p.m., operators had determined that the 13SW20 valve (known to be leaking slightly prior to the flood event) was the apparent source of the leakage and unsuccessfully attempted to manually seat the valv At this point, water level in Bay 1 had reached the top of the pump motor By 10:30 p.m., a larger capacity 2,000 gallon per minute sump pump was used in an attempt to pump out water which had reached to within one foot from the top of the ceilin Based upon the size of the bays, an estimated 144,000 gallons of water had submerged all equipment in the ba Upon making the larger capacity sump pump operational, water level began to decrease at the rate of about one foot every 10 minute By 3:37 a.m. the next morning, the level of water in the Service Water Bay 1 was reduced to about 10 inches above the floo Equipment was submerged for approximately four hours, and equipment located at higher elevations, such as service water pump motors, for an estimated two to three hour Leakage was essentially terminated by 4:00 a.m., on December 23, with the installation of a mechanical plug in the three-inch open ended pipe connection which had been welded prior to the even Plant management met on the morning of December 23 to discuss the cause of the flood, the equipment which had been affected, and to develop a recovery pla A project team was assembled and began immediate recovery action A special investigative team was headed by the QA organization to establish root cause and confirm the sequence of event By December 31, 1987, the valve internals of the 1SW26 isolation valve had been inspected, and the disc of the valve was found to be rotated past its seat in a partial open position, approximatley one inch off the sea The valve was then repositioned to the shut positio Later that day valve 13SW20 was removed and inspected, and its rubber seat ring was found to be disengaged from the groove in the valve body for approximately 18% of its internal
-9-circumferenc At 6:00 p.m. on December 31, the licensee successfully pressure tested valve 1SW26 which had been closed earlier in the day, and the valve held at 230 psig for 20 minutes with no leakage observe On January 1, 1988, the licensee replaced the existing valve 13SW20 with a new valve that was similar in kin Post-maintenance testing was satisfactorily completed on January 2, and the valve was successfully stroke tested and exercise Unit 1 startup was begun on January 4 and at the end of the inspection period Unit 1 was in Mode 3, Hot Standb The above sequence of events is listed chronologically in Attachment.0 Service Water Design System Description The service water system supplies cooling water to safety related thermal loads as well as nonessential auxiliary equipment located in the Turbine Buildin Two portions of cooling flow are supplied, nuclear area and turbine generator are Isolation valves 1SW26, 12SW20 and 13SW20 function in the design basis to isolate the turbine generator area loads from the nuclear area load Nuclear area loads include reactor containment fan cooler units, component cooling water heat exchangers, diesel generator auxiliaries and various ECCS equipment room coolers and lube oil cooler Unit l and 2 have separate service water system For each unit, the service water systems are divided into two bay Unit l utilizes Service Water Bay Nos. 1 and Each bay has three vertical turbine type pumps sized to deliver 10,875 gallons per minute eac Typical service water header pressure is 125 psi Bay Nos. l and 3 can be crosstied to supply either the No. 11 nuclear service header er the No. 12 service header for Unit Each nuclear service header supplies approximately one half of the total nuclear area requirement Service water design is described in FSAR Section 9. In an accident condition, a minimum of two service water pumps are required for safety injection, and three service water pumps for later long-term recirculation coolin Typically, four of the total six Unit 1 service water pumps are running supplying service water cooling loads at power operatio During the event which occurred on December 22, 1987, Unit 1 was in a refueling outage in Mode 5 with reactor coolant temperature at approximately 114 degree A containment integrated leak rate test was underway at the time, and cooling loads were a minimum; essentially no decay heat loads were present, and various portions of the service water header were disassembled or unconnected for maintenanc J
I.'
- - - - - - - - - -
--
- - -
-10-Each service water bay is physically separate from other service water bay Service water pumps are mounted in dewaterable cells or bays, equipped with permanent sump pump Because the service water system is designed for seismic considerations, the downstream turbine area piping (outside of the intake structure and bays) must be isolated in a design even The isolation valves described above isolate the service water nuclear headers from the turbine area piping in the event of a safety injection signal or a blackout conditio Normal service water system configuration is depicted in Figure Four hundred gallon sumps located in each service water bay are serviced by a permanent 100 gpm sump pump, which automatically starts on high leve A high level sump alarm is also annunciated in the main control room to indicate a pipe rupture or flooding in the cubicl.2 Isolation Valve Application The inspector reviewed information pertaining to service water isolation valve design and made observations of the failed 13SW20 valve, installed valves, drawings, catalog information and other records revie Additional information relating to valve operability, problems, testing, maintenance and plans for valve upgrading were obtained through vendor catalog and drawing information and discussions with licensee personne. Valve Seating Surface The service water isolation valves which failed in this event are Pratt Company 30-inch, 150 pound Triton Model XR?O, wafer type butterfly valves with rubber seat rings and a separate rubber linin The valves have H4BC Limitorque operators with SMB motor There are seven valves of this type installed in the Unit 1 service water bays and 7 in Unit 2 bays (including several 24 11 valves with H3BC operators).
The valves are fabricated to ASME Class 3 requirement The.valve body is carbon steel machined with a dovetail groove in the seating are A specially shaped Buna-N seat ring is housed in the body groov The back of the seat ring also contains a dovetail recess into which epoxy is injected during assembl The epoxy positions the seat against the disc, and also forms a reverse taper which retains the seat ring in the bod The epoxy hardens and, while there is no bonding to the seat ring or body, the reverse taper mechanically locks the seat ring in the body groov The body also h2s a separate rubber lining that is applied to the body-wetted surface The lining extends into the *
4. *
4. *
-11-body seat groove so that the seat ring will slightly overlap the linin Figure 3 depicts the existing Pratt valve seating detail Inspection of Removed Valve The inspector observed the failed 13SW20 valve that was removed after the even This valve was replaced with a new Pratt valve of the same internal desig Observation of the removed valve was made from the downstream side with the valve disc approximately 5 degrees ope The seat ring was out of the body seat groove between the one and four o'clock position The seat ring was not broken circumferentially, and the seating surface area was not damage The dislodged portion of the ring, however, was ragged on the rear surfaces that retained the ring in the bod The seat material was noted to be very hard, and there was little evidence of any epoxy on the back of the rin The body groove that retains the seat ring was significantly eroded on both the upstream and downstream sides where the ring was ou The rubber body liner was completely gone in over 50% of the wetted body surface, and the body was badly corrode Corrosion and missing liner material were also evident at the seat-to-liner joint locations around the viewable seat ring circumferenc The seat liner material that still remained was resilien The valve disc, a stainless steel material, was in excellent conditio Past Valve History The rubber-lined Pratt valves have required extensive repair during past outages due to body corrosion when the liner is torn or due to some other type of openin Several of the valves are the original installed valves that were fabricated in 197 Approximately 50% of these valves had been replaced with valves fabricated in 1983-198 The failed 13SW20 valve had been reworked in August 1984 due to excessive corrosio The seat retaining groove and body were weld repaired, remachined, and a new seat ring was installed. The body was rubber lined at that tim Observations made during January 4-7, 1988 of the failed 13SW20 valve were that severe corrosion and removal of the seat groove had occurred agai The corrosion problems have been attributed to brackish Delaware River Bay water conditions, which erode and corrode the carbon steel body at locations where the liner is torn or has a hol....
- *
-**
4. Valve Upgrade Program The licensee has recognized service water system problems and is committed to an extensive upgrade of system materials including isolation valve replacement The seven safety-related Pratt butterfly valves in each unit's service water system will be replaced with soft seat Jamesbury Type 815L Model A valves of aluminum-bronze body and dis Figure 4 depicts the details of the Jamesbury valv The new valves do not need liner A Tefzel soft seat is mechanically held in place with a socket head cap screw arrangemen The licensee has performed seat leakage testing with a valve of this type and has observed good result All 14 of the new Jamesbury valves for Units 1 and 2 service water bays have been ordere Two of the valves, Nos. 12SW17 and 14SW20, were installed in Unit 1 during the recent refueling outag The remaining five Unit 1 valves are planned to be installed during the eighth refueling outage scheduled for March 198 The seven valves for Unit 2 are scheduled to be installed during the fourth refueling outage planned for September 1988.
The inspector reviewed Design Change Requests (OCR)
IEC-2202 for the Unit 1 valves and IEC-2203 for Unit 2 valve replacement The new valves with their corrosion resistant bodies that do not r~quire liners should eliminate a common problem sourc Also, the more positive means of mechanically fastening the seat ring to the body is another advantage for leak tightness of the new valve Flow Requirements and Leakage Tolerance The inspector reviewed the design basis for the service water system in the Final Safety Analysis Report (FSAR) and discussed various topics with the system engineer responsible for service wate Analyses of service water flow requirements under accident conditions do not assume leakage past the turbine header isolation valves (viz. 1SW26, 11SW20 and 13SW20 Unit 1 valves).
Accordingly, FSAR Table 9.2-1 lists zero as the flow to the turbine services under all assumed configurations during accident condition Further, the table lists required flows of 17,653 gpm during a station blackout (with no accident) and 14,170 gpm during the injection phase of a loss-of-coolant accident (LOCA).
Based on a review of inservice testing baseline data, the inspector found that the minimun flow from the two lowest output pumps would be 20,300 gp Therefore, the leakage flow of approximately 800 to 1000 gpm experienced during the flooding event would have been well within
- 1..,
- the excess capacity of the service water syste Similar excess system capacity exists for the LOCA recirculation phas In addition, the PSE&G system engineer for service water presented the results of an analysis dated September 24, 1986, which reviewed different operating conditions for the containment fan cooler unit The analysis concluded that, even with service water temperatures five degrees above the current design assumption of 85 degrees, the system flow capability could meet thermal loads for three containment fan cooler units with flow reduced by a total of 2550 gp Conclusions The inspection team concluded that adequate separation between service water bays existed so that a flood in one bay would not affect another ba Considering flow rate requirements for the service water system, the team concluded that the isolation valves were properly designed to isolate secondary load Based on the above review the inspector concluded that sufficient heat removal capacity exists in the service water system for it to have performed its design functions concurrent with a leakage flow to the turbine building header of a magnitude seen in the flooding event, i.e.,
approximately 1000 gpm.
The Pratt butterfly design currently employed to isolate service water nuclear headers from the Turbine Building nonessential loads appear capable of being cycled through full stroke but would not be tightly close Valves of this type are susceptible to body corrosion resulting from tears or holes in the rubber-liner/seat ring joint This type of corrosion was seen with the failures experienced with valve 13SW2 Rep 1 acement with the Jamesbury design by the next cycle of operation should greatly improve the leak-tightness and corrosion resistance of these isolation valve.0 Service Water Surveillance The inspector reviewed the document file records of the Pratt Company original factory seat leakage tests dated 7/30/73 for valves 1SW26, 13SW26, 11SW17, 12SW17, 2SW26, 21SW1, and 22SW1 Each of the valves had zero leakage at a test pressure of 200 psig and a test time of five minute.1 Inservice Testing A review of the inservice testing (IST) requirement was made for valves 11SW20, 12SW20, 13SW20, 14SW20 and 1SW2 !ST requires
... ~
"'
-14-exercising and stroke testing of these valve Valves 1SW26 (Unit 1) and 2SW26 (Unit 2) are stroke tested during outages because they cannot be tested during power operatio The other valves are stroke tested quarterl The SW valves are normally open valves and seat leakage testing is not required in the IST progra This is similar to the IST requirements at other plant The inspector reviewed the latest IST surveillance procedure 4. test results and some earlier test results of valves 1SW26, 11SW20 and 13SW2 The test results of January 2, 1988 showed the stroke times to be acceptabl Prior results of testing for 11SW20 and 13SW20 dated August 13, 1987 and May 15, 1987 also showed stroke times to be acceptabl The Unit 2 test results of December 12, 1987 for valves 21SW20 and 23SW20 and of December 3, 1987 for valve 2SW26 were reviewed, and these valves also had acceptable stroke time..1. 2 Valve Inspections The licensee performs inspections of different portions of the service water system during each outag The licensee issues a Field Directive (FD) that describes each of the areas and items to be inspecte The valves to be inspected are also included in the F The inspector examined FD S-1-M600-MFD-497-Rl dated August 17, 1987 which identifies the inspection areas for Unit 1 during the seventh refueling outag Inspection of valve 1SW26 was required in the FD and was performe The FD S-2-M600-MFD-0408-RO for Unit 2 dated August 16, 1986 to be performed during the third refueling outage was aiso reviewe This FD required the inspection of valves 2SW26 and 21SW2 The inspector noted that the valves are not removed for these inspection While the piping is large (30 11 and
11 ), a worker climbing into a line may not be close enough to see small problem area Open and Close Settings of the Valves The inspectors reviewed procedure M3L-1 titled 11 Limitorque Maintenance Surveillance and MOVATS Testing 11, and also held discussions with maintenance personnel regarding the open and close settings of the valve The procedure does not describe in step-by-step sequence how to manipulate the valve to its position to set the limit switches, the gear box mechanical stops and the torque switch setting Currently, the valves are set for limit switch operation
- - - - - - --- ----
-
---
...
-15-but there did not exist precise procedural records to verify details of setting For example, as-found settings are not documented and the mechanical stops are not addresse The team concluded that Engineering and Maintenance should further develop the procedure to describe how the valves are to be set, what position the disc is in, and how all settings are performe In their investigative report SQA-88-001 dated January 2, 1988, the licensee recognized a need to revise these procedure.2 Effect of Hydrostatic Testing The licensee had performed a ten year ISI hydrostatic test of service water piping prior to the flooding even Blind flanges were installed as boundries for the hydrostatic testin Valves 1SW26 and 13SW20 had been open during this test; therefore, subseque~t leakage through these valves when placed in the closed position was not due to the ISI testin.3 Pump Testing 5. Time Response Tests The licensee had performed service water pump response time testing several hours prior to the bay flooding even The pumps (SW 11, 13, 14, 15, and 16) were tested satisfactorily in accordance with surveillance procedure SP-(0)-4.3.2.1.3 (Rev. 7).
The time response testing is performed once per 18 months to measure the time for a pump to reach operating pressur Testing is conducted in the following manner:
with one pump in operation, the next pump to be tested is given a start signa When the discharge pressure of the pump being tested reaches the minimum pressure required, the time is recorded (actual times recorded on December 22 ranged between one and three seconds).
The pump initially operating is then turned off, and the next pump to be tested is given a start signa The inspector reviewed the test procedure to determine if the pump tests may have contributed to the seat failure of the 13SW20 valve or the partial opening of the 1SW26 valv The inspector could not find any evidence that the pump response time tests had any effect on the valve problem Pressure pulses experienced in the service water headers were approximately 20-30 psig above normal system pressures, and the seal displacement later observed on the disassembled valve would have necessitated pressures of greater magnitud J
. '.
5. Pump Leakage The service water pumps are Layne and Bowler Verti-Line pump The inspector observed these pumps during numerous visits to the SW bays during the inspectio The running pumps had substantial packing leakage spraying out of the packing bo The team identified this condition to the licensee, but determined that the pump leakage was neither a contributive factor to or a result of the flooding even.4 Conclusions The team concluded that adequate surveillance test procedures existed and satisfactory results were obtained, with the exception that valve position and limit switch setting procedures were not sufficiently detailed to assure correct closure of valve 1SW2 This finding was identified in the licensee 1s event investigation, and will be addressed in the special report to be submitted to NR The team also recommended that leakage testing of the isolation valves which received safety injection close signals be considered, even though ISi ASME Code Section XI requirements are not applicable to these valve The valves were capable of being cycled from open to closed position but would not be tightly close Service water valves of this type are vulnerable to body corrosion due to tears and holes in the rubber lining and at the seat ring-to-lining joint Similar type problems have been encountered at other plants with lined valve Although the time response testing of the service water pumps preceded the flooding event by hours, the team could not conclude that the testing had any causal effect upon the failure of valve 13SW2.0 Service Water Maintenance Valve Maintenance History The inspector reviewed the maintenance history of the large butterfly valves in the service water bay Each unit has a total of seven motor-operated butterfly valves (24 inches and 30 inches).
The previously discussed corrosion problems experienced have also affected these large butterfly valve Although the valves are fully rubber lined, once the rubber liner is no longer intact, corrosion of the steel valve body eventually undercuts the groove which retains the rubber seat, and the loose seat permits leakage through the valv The maintenance of these valves has consisted of repair or replacement of the valves when the above failure mechanism has caused valve leakage.
' '
-17-The inspector reviewed the maintenance of the butterfly valves with a Senior Staff Engineer responsible for valve maintenance, and who is a member of the PSE&G Service Water System Task Forc The engineer stated that, while replacement of leaking valves with factory-repaired valves was normally done, the 13SW20 valve had been weld repaired on-site in August 1984 utilizing the aid of factory representative Further, the 13SW20 had been repaired with Belzona R-metal in May 1984 as a short term repair to last a few months until the later weld repair during the refueling outag In both repairs Belzona Super Rubber was applied to those areas where the rubber liner was gone or had been removed to permit the repai The inspector reviewed Work Orders 922465 and 8270460 covering the May 1984 and August 1984 repairs, respectively, and found them to be acceptabl In addition, the inspector reviewed previous work order records over the previous two years for valves 13SW20 and 1SW26 and found no problem The inspector had no further concern.2 Bay No. 1 Restoration The inspector reviewed the restoration of Service Water Bay No. The restoration approach included the following:
Refurbishment of the four Limitorque motor operators Overhaul of the pump motors and strainer motors Inspection and megger testing of cables Overhaul of all air-operated valves Replacement of all solenoid valves and pressure gauges Inspection and testing of controllers General cleanup including checks of other equipment The inspector visually inspected Bay 1 on January 6-7, 1988, following the restoration and return-to-service of the ba The inspector found the condition of the bay and its equipment to be acceptable and confirmed that the condition of the equipment was consistent with the restoration plan described abov The inspector reviewed a sample of the work orders under which the restoration was performe The inspector noted that much of the equipment was in operation during the inspection and that equipment problems, if any, would have been found during the equipment's startu.3 Post-Maintenance Testing The inspector reviewed the surveillance tests performed prior to the bay's return to service, including the pump testing, stroke and time testing of the motor-operated valves, and confirmation of valve position indicatio '.
-18-6.4 Conclusions The inspector concluded that the restoration and testing of service water bay 1 was acceptabl.0 Service Water System Interactions The team walked down the service water bay and pump intake areas, reviewed plan and section detail drawings on bay construction, and discussed with licensee engineers any possible interactions of the bays with other safety related equipmen The team concluded that a flood in any one bay would not have any probable interaction with any of the other bays for either unit, nor would any other safety related equipment be affecte The team based its conclusions by considering ventilation duct openings to motor control centers at the intake structure and the possibilities of inner connections of sumps and piping with other plant area.0 Management Activities PSE&G Recovery Team and Committee Involvement In the days immediately following the flooding incident that occurred on December 22, the station m5nagement met in daily mee~1ngs to discuss the extent of the problem and immediate corrective action An investigation team was formed and headed up by the Manager of Quality Assuranc The team was tasked with developing a sequence of events for the flood, a root cause analysis for the valve failures and recommendations to be proposed to prevent future flooding incident The investigative findings were documented in a report (SQA-88-001)
to the General Manager of Nuclear Quality Assurance dated January 2, 198 The investigative report was published during the week that the inspection team was on site and, as of the end of the inspection, was under review with licensee management for further actio The team considered the report to be complete and well detailed, and left essentially no unanswered safety issue One exception was the details associated with the Service Water Bay No. 1 sump alarm which had been annunciated on the operating shifts immediately preceding the flooding even However, the licensee committed to addressing the role that that alarm may have played in the event in the special report to be submitted to the NR The NRC inspection team concluded that appropriate maintenance histories and surveillance test records had been reviewed by the investigative team and that adequate personnel interviews had been conducted upon which to base the conclusion The licensee determined the root cause of the flooding by use of fault tree and event charts
'.
-19-attached to the investigation repor The licensee concluded that the 1SW26 valve had been partially open for an undetermined period of time, and that the rubber-lined seal on the 13SW20 valve failed due to degradation of the seat ring surfac The licensee concluded that the cycling of service water pumps immediately preceding the flood apparently caused the seat ring to be displace However, the NRC inspection team could find no direct correlation between that testing and the valve failure based on the acceptable condition of the bay 45 minutes after completion of all pump testin The inspection team concluded that a more likely precipitating cause of the 13SW20 valve failure was a sudden failure of the valve body seat groove as a result of accumulated corrosion similar to that experienced in 1984 for the same valv The NRC inspection team generally agreed with the licensee's recommendations in their investigative repor These included benchmarking valve open and closed positions on the shaft and operator bodies of the valve Another recommendation proposed by the licensee involved revision of test procedures associated with setting the limits and stops for the butterfly isolation valves, so as to verify actual valve position in accordance with the benchmark positions mentioned abov The licensee's recovery team was headed by a senior engineer f~om the Engineering and Plant Betterment Organization who was also the service water project manage The recovery team developed schedules and work plans and essentially recovered Bay 1 equipment within ten day The NRC inspection team confirmed that appropriate maintenance had been performed and repairs and/or replacements were implemented including adequate post-maintenance testin Regarding the licensee's safety review groups and safety operating review committees, no formal convening of those committees occurred prior to the inspection team's onsite followup of the flooding even However, the service water flood incident and the published investi-gative report were discussed in Safety Operating Review Committee (SORC) Meeting No. 88-02 conducted on January 7, 198 In discussions with the Assistant Vice President for Nuclear Operations, the team leader confirmed that licensee management would be acting upon the findings and recommendations of their independent investigatio The inspection team also contacted the Manager of the Offsite Safety Review (OSR) group formed in 1983 to conduct safety reviews and full time safety studie The group consists of eight engineers, four of whom are dedicated to the Salem sit The manager of the OSR group stated that, because of the extensive programs and attention to service water system corrosion problems, the service water system would be the subject of a detailed review by OSR personnel during calendar year 198 *<
- *
-20-Exit Meeting The team leader discussed the preliminary findings of the team inspection with the General Manager of Salem Operations on January 6, 1988 and again with the Vice President of Nuclear Operations on January 7, 198 An exit meeting was held on January 7 and the attendees are noted in Attachment C to this repor A summary of the team's findings was presented on a daily basis during the inspection period with the Manager of Quality Assuranc During those daily meetings, questions or concerns were addressed and appropriate personnel to be interviewed by the inspection team were made available by the license *
At the exit meeting on January 7, the team leader presented the findings of the report, which are summarized in Detail 1 of this report including a reiteration of the additional corrective action items found in Detail At that point, the team leader confirmed with the licensee that a special report would be submitted to the NRC within a one to two-month time frame that would address the concerns raised by the !~RC team, as well as the results of PSE&G analyses being performed on the failed 13SW20 valv The licensee's representatives indicated that the items discussed in this inspection report did not involve proprietary informatio No written inspection material was provided to licensee representatives during the inspection.
'.
ATTACHMENT A DETAILED CHRONOLOGY The following sequence of events leading up to and following the December 22, 1987 Service Water Bay No. 1 flood was assembled from plant logs and personnel interviews, as well as from the independent investigation (SQA-88-001)
conducted by PSE&G and issued on January 2, 198 No significant variatiors in the times recorded from different sources for the same event were foun Where log entries were not clear or only brief statements provided, they have been supplemented with commentar Initial conditions Salem Unit 1 was shutdown in Mode 5 at 114 degrees F reactor coolant temperature with primary system vented to containment for Type A integrated leak testin Unit 1 was in the second month of a refueling outage and was planning a restart in late Decembe December 20-21, 1987 Isolation valves 11SW20, 1SW26 and 13SW20 which separate the safety related portion of the service water system from the nonsafety related portion of the system were blocked and tagged for repair wor Isolation valve 13SW20 was leaking b An intermittent sump high level alarm in Bay 1 was ~xperienced during previous shift A permanent sump pump was capable of periodically removing accumulated water in the 400 gallon capacity sum In order to allow welding on an open three-inch line downstream of 1SW26, blank flanges in the turbine building and upstream of 1SW26 were loosened to drain the pipin Both flanges had been originally installed for the 10 year ISI hydrostatic tes The No. 12 Nuclear Service Water header was in service and a single SW pump, No. 13 in Bay 1, was in operation as required during refuelin The No. 11 Nuclear Service Water header was tagge December 21, 1987 Time 2nd & 3rd shifts Event Bay 1 sump high level alarm annunciated consistently, but regular hourly checks in the Bay 1 by an equipment operator indicated no overflow conditions.
..
ATTACHMENT A December 22, 1987 Time 1600-1830 1830 1910-1922 1937 1949 1958 2001 2014-2-Event Number 11, 13, 14, 15 and 16 SW pumps were each briefly operated satisfactorily in accordance with time-response test procedures (see Detail 5.3.1).
Note:
SW12 pump motor was uncoupled for maintenanc All service water pumps were taken out of service except Number 13 which supplied water to Number 12 Nuclear Service Water Heade Operator checks service water bay No flooding condition note An equipment operator had been regularly observing Bay 1 conditions on an hourly basis since the beginning of shift (1530 hours0.0177 days <br />0.425 hours <br />0.00253 weeks <br />5.82165e-4 months <br />).
Number 15 11Turbine Area Sump Pump Alarm 11 *
Note:
Number 15 is the smallest sump in the turbine are Overflow from sump No. 15 cascades into the next larger size sump until it eventually reaches the largest sump which discharges to the rive Number 14 11Turbine Area Sump Pump Alarm 11 *
Note:
Control room operators dispatched Unit #1 secondary operator to investigate second turbine area sump pump alar Operator observed the loose blank flange upstream of valve ST-1 in the Turbine Building leaking and returned to control room report the condition at about the time Bay 1 was reported flooding by the fire watch perso Number 13 11Turbine Area Sump Pump Alarm 11 *
Note:
Secondary operator was already in the Turbine Building in area, as noted abov Number 11 11Turbine Area Sump Pump Alarm 11 *
Routine fire watch enters Service Water Bay 1 (for approximately 17 seconds) and reports flooding to control roo '
..
ATTACHMENT A 2022 2025 2110 2115 2130 2147-2221 2230 December 23, 1987 0337 0400-3-Operator enters Bay 1 for approximately 11 seconds and reports to control room that Bay 1 is floodin Control room operators started SW15 pump and stopped the SW13 pum Closed the 11SW17 and 12SW17 cross-connect valves between Number 11 and 12 nuclear service water headers, in accordance with Emergency Instruction EI-14, Service Water System Malfunctio Water level in Bay 1 was approximately one foot below SW pump motor base Moved portable 600 gpm sump pump from B~y Number 3 to Bay Number 1 (Water level one foot up on the motors).
Portable sump pump was operating, but could not keep up with the leakag Control room shift supervisor inspects Bay Attempts to manually seat valve 13SW20 (already electrically isolated) faile Water level in Bay 1 was up to the top of the SW pump motor Turbine Area Sump Pump Alarms Installed another, larger portable 2000 gpm sump pump in Bay Water level was one foot from the top of the 18 foot ceilin Estimated leakage was approximately 144,000 gallon Service water system pressure was reduced from 100 to 70 psig, concurrent with the portable sump pump operatio Water level in Bay 1 decreases at about one foot every 10 minutes from thereoni Level of the water in Number 1 Service Water Bay approximately 10 inches dee Bay 1 flooding inleakage controlled after installation of mechanical plug in the three-inch open connection.
.~.
ATTACHMENT A
-4-Inspected 1SW26 valve internals following removal of expansion join Found that the valve disc had rotated past its seat and was partially ope Position indicator showed valve close Valve 1SW26 repositioned to shut positio Valve 13SW20 was remove The rubber seat ring was found to be torn from its normal position over about 18% of the pipe's circumferenc Successfully hydrostatic test on valve 1SW26 to 230 psig for 20 minutes with no observed leakag Replaced valve 13SW2 Limitorque testing on valve 13SW20 conducted satisfactoril Stroked valve 13SW20 satisfactorily.
- --
--------- -------------
A
'
ATTACHMENT B PERSONNEL INTERVIEWED The below listed PSE&G individuals were formally interviewed by the NRC team during the course of the inspectio Management S. Miltenberger, Vice President, Nuclear Operations f. liden, Manager, Offsite Safety Review J. Zupko, Salem General Manager Operations J. Gueller, Operations MAnager J. Musemeci, Operations Engineer N. Neiheiser, Equipment Operator R. Watson, Shift Supervisor Maintenance P. White, Manager R. Sambuca, Electrical Maintenance J. Webster, Senior Staff Engineer Engineering and Site Services I. Owen, System Engineer K. King, Engineer W. Straubmuller, Project Manager W. Treston, Senior ISI Supervisor J. Zimmatore, Senior Valve Engineer J. Rowey, Project Engineer F. Sullivan, Principal Engineer QA 0--:- Perkins, Manager, Salem QA R. Dulee, Principal Engineer Licensing M. Gray, Engineer G. Roggio, Station Licensing Engineer 8. Preston, Manager of Licensing and Regulation
- '
ATTACHMENT C MANAGEMENT MEETING ATTENDANCE The following personnel were present for either the entrance interview on January 4, 1988 or the exit interview on January 7, 198 The findings of this inspection were also verbally discussed by phone with Mr. S. Miltenberger, Vice President-Nuclear Operations, on January 7, 198 PDblic Service Electric and Gas (PSE&G)
D. Budzik, Manager of Projects for Salem 1&2 R. Burricelli, General Manager, Engineering & Plant Betterment M. Gray, Licensing Engineer J. Gueller, Salem Operations Manager D. Perkins, Salem Manager, Station QA G. Roggio, Station Licensing Engineer B. Preston, Manager, Licensing & Regulation J. Musumeci, Operations, Salem W. Straubmuller, Projects Mar.ager P. White, Maintenance Manager J. Zupko, General Manager, Salem New Jersey Department of Environmental Protection W. Cristali, Nuclear Engineer N. Di Nucci, Nuclear Engineer K. Tosch, Nuclear Engineer U. S. Nuclear Regulatory Commission Team Members T. Kenny, Senior Resident Inspector
,.
...
- .
FIGURE 1 SERVICE WATER SYSTEM Configuration on 12/22/87 UNIT 1 in MODE 5,COLD SHUTDOWN RCS Temperature 114degrees Decay Heat removal via Nuclear Header #12 with One Service Water Pump (#13) running @ 11,000gpm to Turbine Bldg. Loads
.Turbine Bld Service Water Intake Strulu-re * *.. *
S-Q)
-0
- ~
ctl Q)
,......
u
- s z
l 13 ~ws-I (closed &tagg~d)
l~ Sw "lO
. seat ring I fai ed t-=---...
,,t--,---'----.----L--_.JLJI l
J I
J J
- (closed &tagged)
l ~w 1.-b
~Open slightly off seat t l $W? (removed ( c'l osed &tagged)
SERVICE WATER PUMPS BAY #1
)
..-i
'
..-i
=1'!:::
s....
Q)
-0 ctl Q)
- c s....
ctl Q)
r-0
- l z
,.
...
N
.-I
=ti::
.s....
Q)
"O
....
rt:S Q)
.s....
rt:S Q)
..--u
- .:i z
-
Se.fyice Water Pumps B.l\\V #3 FIGURE 2 SERVICE WATER SYSTEM SALEM UNIT 1 Normal Flow Schematic Turbine Bld Service Water Intake StruJture * * -- *
I 13 !>W~ I J
l~~w io (isolates onfsIAS)
I crosst l J
I I
J l
I 1'
I I
- I I
J *.
I.....
-- ~.... -. f. -----~-----t--- :
~
~,
Turbine Bldg. Loads (non safety-related)
~
(iS..olates on SIAsT l 'Sw 1.-6 tl 5W?
(isolates on SIAS)
Service Water Pumps BAY #1 r-u
- s z:
..
....
... ~:
...
FIGURE 3 PRATT VALVE DETAILS CA~SotJ STEE SODY E*LOP SEATING SYST!M I
METAL TO METAL Cl.EARANCE PTO Ve"
- ....
~;..... -.** ~
.-------
I..
STP.ttJU.:SS STEEL l>I S FIGURE 4 JAMESBURY VALVE DETAIL
,;..*
/\\Ll.JMl~"M Seot.lZE SopY ~ DIS _j SEAT
' ' \\
\\
I
F'
,
,.'
I