ML20245D085

Generator Reactor Coolant Spills of 870128 & 0201

ML20245D085
Person / Time
Site:	05000000, Sequoyah
Issue date:	02/01/1987
From:	TENNESSEE VALLEY AUTHORITY
To:
Shared Package
ML082490787	List: ML20245D056 ML20245D059 ML20245D066 ML20245D071 ML20245D085
References
I-87-01-SQN, I-87-1-SQN, NUDOCS 8711040352
Download: ML20245D085 (38)

Text

{{#Wiki_filter:_ -., } TENNESSEE VALLEY AUTHORITY NUCLEAR MANAGER'S REVIEW GROUP REPORT h'O I-87-01-SQN 1. 4 b Sequoyah Nuclear Plant Steam Generator Reactor Coolant Spills of 1 January 28 and February 1, 1987 l 4 S 8711040352 871002 PDR ADOCK 05000327 P pop

~ 1 1 TABLE OF CONTENTS Page I. Introduction and Scope. 1 2 II. Management Summary. III. Sequence of Events..... 5 13 IV. Analysis. 28 I V. Findings. APPENDIX 1 - List of Aeronyans Used in This Report. 33 wo { PN hs CQ ?@ if F F 5 3

I. INTRODUCTION AND SCOPE On January 28 and February 1.1987, act.!vities at Sequoyah Nuclear Plant (SQN) unit I resulted in reactor coolant spills to the containment. The Manager of Nuclear Power requested the Nuclear Manager's Review Group i' (NMRG) to conduct a review of the two spills to determine the root cause(s) of each, j i A review team was assemu ed consisting of NMRG persottnel and two licensed senior reacto: o,perators (SR0s) with Westinghouse pressurized water reactor (PWR) operations esperience, one from Watts Bar Nuclear Plant, and one from Duke Powar Company. Tht-team conducted its onsite activities between February 11 and March 4. 1987. Information was gathered through interviews, observations of activities, examination of the spill area and equipment involved, review of documents and drawings. consultations. and photographs. The methodology used tr.cluded Management Oversight and Risk Tree Analysis t.nd bsale accident investigation techniques. The results were presented in an oral briefing to SQN management on March 6. 1987. The following is a report of the review team's findings. N Section II Management Summary, contains a brief description of each spill and the major contributing factors for each. Section III contains N the reconstructed sequence of events. Section IV contains an analysis of the events and their causes. Where appropriate, speelfic errors are N ldentitled. and alternate actions which could have prevented the spflis are discussed. Section V summarizes the findings and groups them into g functional areas of interest. M T c' O { O i.. .. t 'g. ~. ,..... -, m,

~. II. MAllA0RMRWT

SUMMARY

For both events, the reactor was la mode 5, depressurized, and partially drained down. The residual heat removal (RNR) system was in operation. for decay. heat removal. Reactor core decay heat was very low, since the reactor had'been shut down for several months. The reactor coolant temperature was about 95'F. The primary aianways were removed from all steam generatars, and Combustion Engineering work crews were performing-stress relief on the stems generator tubes. A 75 GPM feed and bleed was in progress through the r.houtcal volume control system (CVCS). Reactor coolant system (RCS) level was being maintained about six inches above the center 11ne of the RCS hot les piping. In the first event, a plugged RCS level indicator resulted in a slow RCS level change going undetected for several days. When the plus cleared, a rapid eleven-inch rise in indicated level resulted. Over the next several hours, as operators attempted to determine the true RCS level and establish an acceptable level, the actual level was lowered enough that RHR suction was lost and was raised high enough to overflow to the containment through the open steam generator primary side aanways. A few hundred gallons were spflied to the containment. o' In the second event, a series of valve cycling surve111snee tests were A being conducted. The surveillance test procedure for cycling 1-FCV-63-1, a 14-inch gate valve.for RHR suction from the refueling N water storage tank (RVST), was not adequate for the existing plant conditions, and the shift operators decided to conduct the test without g, developing an adequate procedure. The flow path between the RWST and the depressurized RCS had not been adequately blocked by shutting the normal RHR suction valves before 1-FCV-63-1 was cycled open. When it M was cycled open, the RVST was connected to the RCS through 14-inch piping. The gravity head of about 44 feet from the RWST forced enough Y water into the RCS to fill it and overflow out the stesa generator manways before the flow could be stopped. Between 2,000 and 3,000 q. gallone of water overflowed into the containment. C' Some personnel contamination resulted from the second event, and ares O contamination resulted from both. Fortunately, there was no personnel injury because no personnel were working in the steam generators when the spill occurred. Had personnel been working in the steam generators, the rush of water could easily have caused injuries. No equipment damage resulted from either event. A small amount of insulating material may have entered the RCS during the second event. The significance and possible corrective actions for the Insulation material entry was being investigated. Though many of the direct causes and contributing factors associated with these events were errors by shift operations personnel, weaknesses in management controls for shift operations and in tralning and o ) l .....-.. pe== e w*m==r s.,, w 7,,, ;. ;--

directions provided to operators are considered the more significant and fundamental causes. :The nr.st significant improvements needed, based on i analysis of'these events, are as follows: 1...; Involvement of operations shift supervisory personnel in' plant .1 operating acitivities needs to be substantially strengthened to provide the appropriate level of support, oversight, and direction f .i for unit operators, Supervisory involvement results more from requests for assistr.nce from subordinates rather than from actlve. jl oversight and monitoring efforts. Supervisors do not require' operators to keep them well informed of abnormal onditions. i' \\. 2. Adherence to procedures by plant operators, and use of procedures to respond to abnormal conditions, need to be improved. Though SQN has l' requirements to conduct operations of critical systems, structures and components in accordance with approved procedures, operators often operate without'such procedures and espressed an inappropriate willingness to deviate from proceduces. Adequacy of operating procedures-is a contributing factor to adherence problems. 3. Operations personnel need to be better trained in analyzing and 1 responding to abnormal indications. Operators intentionally j O . proceeded with actions to change reactor coolant level without an f urgent need and without knowing the actual level. Excessive l g reliance was placed-on the single level indicator after it had shown j N erratic operation. j 1 'N -4 Reliability of the RCS level indicating system needs to be improved. Only a single means of level indication is available, and that system has plugged on at least two occasions. Corrective ] action for the first instance was extensive but not effective g. because the likely cause, an orifice prone to plugging, was not v addressed. T 5. Measures to stab!11:e RCS level during partially drained conditions need to be strengthened. A feed and bleed process that may not have U been necessary contributed to an undetected level change during the o first event. C Some findings identified in this report were already known to plant management and corrective action was in progress. For example, SQN has an ongoing commitment to an extensive operations procedures review and f upgrade. That process is expected to yleid substantially improved operational procedures. During the course of the review, several noteworthy highlights emerged. I l The general attitude of SQN personnel contacted during interviews and while gathering information was very supportive. Personnel were candid with information and willing to give whatever time was necessary. This positive attitude reflects a willingness to learn from experience and l avoid repeating past mistakes.

II Good-leadership ability and concern for the safety of his personnel was shown by the steam generator maintenance supervisor, a TVA employee. Sensing hesitancy on the part of craft personnel assigned to personally performed the first entry into each steam generator toHis action was consi ascertain that. conditions were safe for work. highly praiseworthy by his subordinates. ~C F N N M IT C O IO b G % 8 .' e a. i

III. SgQUgNCg 0F IVgNTS The sequence.of events was developed from recorded logbook and f surveillance instruction (SI) entries, written statements, and interviews with involved individuals. l Times in parentheses are approximate times. A. January 28, 1987 Epill Inittel conditions Unit 1 was in mode 5, depressurized, partially drained down, and utilizing the RHR system for temperature control of the reactor core. Reactor core decay heat was very low, since the reactor had been shut down for several months, and the reactor coolant temperature was about 95*F. The primary manways were removed from all steam generators, and work crews were performing stress -relief-on the steam generator tubes. The CVCS was supplying water continuously to the RCS and reactor coolant pump seals from the volume control tank (VCT) at about 75 GPM. RCS level was maintained at the approximate elevation of 695 feet 6 inches, by an offsetting 75 GPM letdown rate from the RCS ~~ to the VCT. This level was about six inches above the centerline of cn the RCS hot les piping. hs Makeup, for normal RCS coolant usage, was supplied from the VCT. 04 VCT level was being maintained between approximately 28 and 50 percent full by periodically kdding water from the EWST. The only means of determining the RCS level was a permanently p) installed sight glass in the number one fan room of the reactbr ~7 containment. The sight glass was connected by a removable spool piece to the number 1 RCS loop crossunder piping low point drain. 97 The sight glass was vented to the containment atmosphere. lC' The sight glass was continuously monitored by a remote-controlled television (TV) camera that fed two TV monitors in the main control

C3 room. One monitor and the camera controls were located on the lC) electrical control panel approximately 60 feet from the unit 1 operator control area. The other monitor was located on top of the unit 1 computer console within the unit 1 operator control area.

RCS level was being logged every 30 minutes per SI-673. The RCS level at the beginning of this event was 695 feet 7 inches. Wednesday. January 28. 1987 0033 The VCT level had decreased to approximately 28 percent and re. quired makeup. The reactor operator (RO) opened 1-FCV-62-135 and -136 auction valves to the centrifugal

) I I charging pumps (CCP) from the NWST and closed 1-FC7-62-132 and -133 sur. tion valves to the CCPs from the FCT to add . water to the RCS and increase YCT level. (0045) The'TCT level had increased to approximately $6 percent t (approximately 600 gallons added). The RO opened the CCP l suction valves from the VCT and closed the CCP suction l valves from the RVST to terminate makeup. l 0100 RCS level logged at 695 feet 7 inches. 0130 The RO checked the RCE sight glass via the TV monitor and was unable to determine the level. He immediately notified the assistant shift engineer (ASE). i The ASE said that he had seen this problem previously. He said the 1.shting in the area of the sight glass was . critical for level visibility on the TV monitor and probably had been moved, resulting in insufficient lighting. 3 The RO and others in the control room scanned the camera up and down the sight glass for approximately six to ett,ht inches, but could not determine ~the level. The RO verified W that CCP flow, VCT level, and RHR flow were as expected. "#^ (0145) .The RO told an assistant unit operator (AUO) to stand by g for a containment entry to verify RCS level and adjust sight glass lighting. Actual entry was delayed by the RO CQ. for about one hour to combine those activities with another a unrelated activity. g (0245) AUO entered containment. iy 0330 The AUO reported back to the RO that the actual sight glass level was 696 feet 6 inches (about one foot higher than v espected). The ADO determined that the sight glass scale had not been moved and verified no apparent blockage of the C sight glass vent line. Normally, the sight glass level fluctuates approximately o one-eighth inch, and the indicated level was fluctuating that amount. Therefore, the sight glass level was considered to be correct. Yalve lineups on the cold leg accurnulator, upper head injection, and primary water paths were checked by the RO. All appeared normal. The operator concluded that there was no unknown source of makeup to the b RCS. l' l' o i -

l ,1

• \\

(0400) RCS level logged at 496 feet 6 inches. The ASE and RO decided to drain the RCS level to 695 feet. 7 inches. The R0 increased letdown flow from approzinately 75 to 130 GPM.' When high letdown flow alarm was received '(120 GPM) the R0 reduced letdown flow to approzinately 100 CPM to clear the alarm. j j Sight glass levels had not been recorded, as required, since 0130, when the level could not be located. The ASE l then directed the RO to backlog sight glass level readings j to 0130. The it0 did so for the four missed readings, between 0130 and 0300, entering 696 feet 6 inches for each. 0430 RCS level logged at 696 feet 4 inches. 0500 RCS level'1ogged at 696 feet 4 inches. 0530 RCS level logged at 696 feet 4 inches. 0600 RCS level logged at 696 feet 3-1/2 inches. ) x 0620 A low RHR flow alarm annunciated and the low flow recirculation valve for "A" RHR purnp started open. The RO i F . verified low flow conditions and noted that the pump motor amperes were fluctuating. f ~ 4 g The "A" RHR pump was insnediately stopped and locked out, thus terminating letdown and RHR flow. Based on the loss of RHR suction, the RO concluded that the actual RCS level was lower than the 696 feet 4 inches, Indicated by the sight glass and began adding borated water to the RCS by opening the CCP suction valves from the RVST q. and closing the CCP suction valves from the VCT. O The 20 decided to return the RCS level to an indicated sight glass level of 696 feet 6 inches, since the RRE pumps

O had been operating satisfactorily at that indicated level.

l The RO also increased charging flow from 75 to 120 GPM. The ASE contacted the people working in and around the steam generators to inform them of the level indication problems and directed them to secure work and vacate the insnediate area around the steam generators. No one was y actually inside the steam generators at that time. The shift engineer (SE) was informed that RER had been secured. This was the first notification to the SE of any unusual conditions involving RCS level (five hours after the initial. loss of level indication). 4 .s 7 M, - -. w-. w m

. v, n. _. \\ I f t } The 30 seat sa AU0 to "A" RHR pump.to vent it so that it-l could be return ^d to service. Another ADO was sent to "S" RME pump to align it for letdown to the CfCS.~ RCS level logged at 696 feet 4 inches. 0630: RCS level logged et 696 feet 6 inches. 0700 i NOTE: The RO later stated that he continued adding water 'l at 120 GPM untli the sight glads level reached 696 feet 8 inches, when CCP flow was reduced to I 70 GPM. The RO stated that the addition of water to l the RCS was not stopped at 696 feet 6 inches. j because the RO believed additional level was desirable. A person working.on the steam ganerator contacted the RO (0714) and stated that the steam generator was about five mlnutes away from overflowing. The R0 inunediately stopped the CCP. 1 closed CCP suction valves from the RWST, and closed the charging header isolation valves. The RO then checked the sight glass level. It had increased to 697 feet 0 inches. y' This level corresponds to the bottom of the steam generator manut.y. e The steam generator personnel called the control room and i fN (0728) notified the RO that water was overflowing from the primary N manways on number two and three steam generators. The R0 started "B" RHC pump, verified flow equal to or .(0729) greater than 2,500 GPM, and observed a momentary decrease f*) in sight glass level to 696 feet 10 inches, and a return to T 697 feet 0 inches. Y NOTE: Normal charging and letdown was established at a inter undetermined time (not logged). p B; 0745 Shift turnover occurred. Dayshift initiated a work request (WR) to install a tygon hose as an alternate method of 59 level indication. The hose was to be connected to the same spool piece used for the sight glass, the only available connection point. 0800 RCS level logged at 697 feet 0 inches. to 0900 NOTE: The RCS level decreased from 697 feet 0 inches. et 0900 to 696 feet 6 inches, at 1030 where it remalned until 1730 hours. i ! I

s 1

ii 1740 The tygon hose installation was completed and placed in service. The level in:the tygon hose was 696 feet 0 inches.

1756 Sight glass level was'696 feet 6 inches. Calibration of the level indicators was reguested. f 1910 Instrument mechanics (IM) completed calibration of the tygon hose snd sight glass level indicators. Basically, this calibration consisted of measuring the distance from the indicator scales to the floor (which was assumed to be 679.78 feet as shown on plant drawing 47W200-18). The sight glass scale was correct and the tygon hose scale was low by one inch. An entry was made in the. operator's daily journals requiring one inch to be added to,the' indicated level of the tygon hose, but the seals wai not repositioned .O to give accurate readings. 1950 IMs and an AUO connected an' air line to the sight glass drain, isolated the supply valve and blew air through the drain line into the RCC to clear any blockege in the piping between the RCS and ?,he sight glass. After' the blowdown, sight glass level was 696 feet 6 inches. The corrseted E9' tygon hose level was 694 feet 11-1/2 inches, The IMs and ADO then drained the s'f ght glanhi ar.d blew air 2025 through the sight gis.ss via its drain line. One of the 43 IMs stated that he saw some tiny particles of black SS material or " crud" in the sight glass during this operation. No atteupt was made to sample this material. The sight glass and tygon hose were then placed back in service. A sight glass level.of 695 feet 6 inches, and a q) tygon hose level of 695 feet 1 inch, were recorded in the s' RO daily journal. 5" NOTE: Those levels remained' approximately the same until 2230, two days later, p 9 Fridar January 30, 1987 S 2231 The tygon hose was found to have a significant amount of entrained air in localized high spots between the RCS tap and the level scale. After removal of air,'the tygon hose level increased to 695 feet S inches. 2330 Sight glass level logged at 695 feet 9 inches. Tyson hose level logged at 695 feet 9 inches. } J -9 { s {

i 't b o r / . S. February 1. 1987 spill + { {3ngry 9.' 1987 On this date, a night order was issued emphasizing the need to-perform sis on time.' Attached to the night order was License Event toport JLER) SQN 86-051 of November 6, 1986, concernir;g some valves that were not being maintained in their surveillance frequency. ~ because.of'an inadeqvste procedure. Alsoattachedtotkhalght order was a memorandum from thi rochanical tect section supervisor-acknowledging procedur( probicas.and emphasizing the importance of completing tests in a cisely unner. The night ordei emphasized the: potential of a $50,000 fine and/or escalated enforcement action if r c survelliance: tests'were not performed within prescribed. time limits-. February 1. 1987 ( 0700 Unit I was being maintein d/in a par ally drained, depressurized condition 'Nd,11 sing the RHR system fer. temperature control ol' th6'rdsetdr coolant system.: Reactor coolant level was being asihta!.ned at about ele'<ation -F i 695 feet 9 inches.(conted !jne of the hot les pi' ping is i / approximately 695 feet 0 irebes). Makeup was being added .o to the RCS by one centrifugal charging pump, and letdown B was.in progress to control RCS level. 'The primary side I manway covers were removed from all steam generators, and i work was in progress on steam generator tubes. N SI 166.8 had been sent to the control room on or about January 31, 1987, for performance of valve stroke tests on 4 -five valves. The tests were to be completed by February 2, f 1987. 1 0700 Four of the five valves were stroke tested satisfactorily. The one remaining valve was 1-FCV-63-1, RHR system suction to y 1254 valve from the RWST. This is a 14-!nch motor-operated gate af I y / R valve. ) E The RO and trainee discussed the stroke test of this valve, including several aspects of the, test such as valve g alignment, necessary equipment 7pnfigurati,on changes, and potential problems. Because the steam generator primary sides were open, a caution tag was attached to 1-YCV-63-1 to prevent inadvertent operation that might ejd water rapidly to the The 'aut pn tag simply required reactor coolant system. c the SE's permission before operating'l-FCV-63-1. As required by the caution tag She 13 gated the SE for its i 1 4 l l i u-i I ...---.n

+=

e

<m '4 /, t!, .o y ,,1 o a i. a w ./ j o, .e i 3 L > .. permission to operate 1-FCf-63-l'and, based solely on the h ss's. confidence in the 10's knowledge and esportence, f}- permissioe was. granted.. -,,Y, ~ 1254 The 20 stopped the "B" RHR pump'and ch sed the pump j discharge valves la preparation for 4tethfris 1-FC7-63-1.. The'RO failsd to close the 14-inch 1 ste, tion v417es from the loop 4 RCS hot leg -(1-FC7-74-3. as:t.2), / i When 'the "P.' IIHR discharge valb wer e edesed,i the RO opened 1-FCV-61-1 using the controbsiitch in the control .As soon ts 1-FCV-63-1 began _to open 'the RO realized room. I ca direct flow path had been set up't? ore the,RWST to the Ireactor coolegt, mystem through the:It-inch RHR saction Spiping. However, there was no quick way to stop or reverse tb. opening motion of, the meCor operator. The vtive takes approximately 20 seconds to open 'sig soon as the valve:_ ' ndh ated full open, the 20 placed h e co prol switch to i the closed position and timed the valve'ti the closed i H, I position (21.3 seconds) tovaatlefy thatSIlrequirements, e RCS' sight glass'1evel ediately went high, out of' range of the control rooch!0er (it w6s determined later R-tobeatlesst15incheiadtenormal)\\ 'A' I (go ' n y N When water began spraying out 4 the steam ger.erator N 'annways. Radeon technicians-1[tfie teks lastructed all personnel in the area of the steam generators to leave-N /- i?" fu '/' containment. Some people;were wet by the spill but no one 9 a/ waa'Jajured or contaminated. No one use in a steam 'genetitipg at the time. d-4 h h' f, e,d i i / 9 / .(1U56) The RO reopened the "B" RHR pump discharge valves, restarted "B" RHR pump / and estabMne4'uazimum letdown e flow.. An AUO was dirgtche6' tn endure that 1-FCV-63-1 went p fully closed.- (The AVO s @ equently tightened the valve-S,, approximately 10 txfna' af tt6 handwheel' This valve, which E g @. f. '/N has a tota?. travel tfWe 450 turns, was in fact shut before the oporr. tor drived. A The. addMionel 10 turns-3 merely seated tid dise'more fienly.)' g-t j The control roc @ was.cottfled by savoraV individuals that waterwasgushing]omtt,asteamgenerat9e'manways. ( W! 1335 RCS level was reestabli/jed at elevation bM feet 9 inches by using RHR letdown to'the CVCS and into the the holdup j i tank. a s The ASE estab11she6"conriunications with the engineer in 3 charge 6f the otear generator work and was inferraed that ,^ 8 water had rallied on four or,five people inside containment. p i? l N. i$ 4 ) .. u - i s .g ,{ g. ...... ~ &v, i / DL-w

l-An AUC was dispatched to containment to investigate current conditions. Th AUC did not go inside the polar crane wall, but reported approximately 1/2-to 3/4-inch of water on the floor in the area of loop 2 and 3 steam generators, approximately one inch of water in the area of loop 1 steam generator and approximately 1-1/2 inches of water on the lay down area under loops 2 and 3 steam generators. Follow-up activities were centered upon reestablishing RCS level and riecentaminating the steam generator work, areas. J C' N W P'? E E? E9 D "4.

IV, ANA1.YSIS In this section, an analysis of the information gathered is presented to develop and illustrate the contributing factors for each spill. Where assumptions were made to logically develop the analysis, they are clearly indicated. Additional information relevant to the analysis is contained in Section III, Sequence of Rvents,.and Section V. Findings. January 28, 1987 RCS Spl*d, Unit I was being maintained in a partially drained down, e depressurized condition. utilizing the RHR system for decay heat removal. The reactor vessel head was installed. RCS charging and letdown rates were being balanced to maintain the RCS level at approximately 695 feet 6 inches (6 inches above the centerline of the reactor coolant horizontal legs). Makeup to the RCS for normal system losses came from the CVCS VCT. Its level was maintained 'between approximately 28 and 50 percent full through periodle addition of water frora the RVST. RCS level was indicated by a single sight glass connected to the RCS loop 1 cross-under piping low point drain. fb The RCS was partially drained down to permit steam generator primary side work, and the aanway covers were removed. Two RCS levels were O important: (1) RHR suction could be lost at about the 695-foot level because the RHR suction piping taps-off the side of the hot les at about that elevation, and (2) water would be spilled from.the N steam generator aanway at about the 697-foot level. The procedure used to establish the partially drained condition initially established the level between 695 feet 6 inches, and 696 feet. However, once the level was initially established, there was no . M g written procedure for maintaining it..and no level control band was speelfled. Operators were aware of the levels associated with loss _, 9 of RHR suction and steam generator overflow. T .h At 0130, the RO atterated to read the RCS sight glass level on the 'C' TV monitor, but could not see the water level in the sight gissa. The ASE was informed. The ASg stated this had happened before and ,., O had been caused by poor lighting. The sight glass was scanned with the TV camera and monitor from the control room, but the level was o not located because too small a segment was scanned. The actual sight glass level was 11 inches above the initial level, but the camera was only scanned up about six to eight inches. The TV camera was espable of scanning the entire sight glass. It is not clear why the operators did not scan a wider segment, except they did not g expect the level to move as far as it actually had, and they believed lack of lighting was the reason the level could not be h located. A broader scan would have been easy to accomplish and might have located the level much more quickly.

• r.
• f 6

'E I d I q [

At that time, the only thing known about RC8_ level was that it was-high enough to maintain RNR suction and low enough to prevent everflow through the steam generators. Nonetheless, efforts to have an AUO enter containment and read the sight glass level locally were less than aggressive. ADO entry into containment was deleyed approximately one hour so that the AU0 could be assigned another-task, associated with the steam generator work, to be performed during the same entry. The RCS level of 696 feet 6 inches, was ' finally reported to the control room approximately 45 minutes.after the AUO entered conta!nment. A total of two hours had lapsed-between loss of leral Indication in' the control room and determination of level locally. Sight. glass readings were required by SI-673 to be recorded every

half hour. During the two hours when the level'was' indeterminate.-

four readings were not: recorded. Contrary to good logkeeping practices, the four missing readings were entered late (after the level had been determined locally). All four entries showed the level to be 696-feet 6 inches. A better logkeeping practice would have been to record these as alssed readings, since that was the case. Both the RO and ASE believed the sight glass level was correct. O They based this beller on the level meniscus oscillating about 1/8-inch, which was normal, and the lack of any previous problem with the sight glass. This assumption was not appropriate because is the sight gisse had just shown a level change that was considerably faster than any previous level change, and it had changed at a time W when no operations were in progress that could cause a rapid level. e change. Sources of water that could have caused the level change were checked and found to be lined up normally, so there was no to identitled cause for the level change. There was, however, enough information for the operators to conclude that the RCS level T indicator might be inaccurate and that the actual RCS level was 'ot n known. Since RHR was operating satisfactorily and the RCS level was not near overflowing, there was no urgent need to change the actual e RCS level. The proper course of action would have been to stop any attempts to change level until the cause of the level anomaly was O known and accurate level indication was again established. - O' Nonetheless, in an attempt to reduce the RCS level and provide more margin to overflow, the RO and ASE decided to lower RCS level by 11 inches from 696 feet 6 inches to 695 feet 7 inches, using the level indicator they believed to be accurate. Normally, in a partially drained condition, the RCS level was maintained by adjusting the CVCS makeup and letdown rates a few GPM e l to produce slow level changes. The operators, therefore, were not familiar with the rate of level change to be expected with the .j 14 l 4 . e e, em-

n o 25 GPM escess letdown rate that they established. No data was readily available in the control room that would have given them this information. During the first 30 minutes of draining, the sight glass reading dropped two inches; then it stabilized for 1-1/2 hours, even though draining continued. Here. again, was clear indication that something unexplained was happening and that the RCS level indicator was not accurate. 0.7e and a half hours at a 25 GPM excess letdown rate equates to ove,r.4 2.000 gallon decrease in volume, a decrease that clearly should have produced a change in level. This failure of the system to responi as expected should have prompted the operators to seek assistance and to stop the draining evolution, but neither action was taken. Draining continued until the low RHR flow alarm was received, reflecting loss of RHR pump suction. The act of securing the RHR pump, because the ampere meter was fluctuating i 10 amps and the RHR pump bypass valve opened, was not immediately recognized as a reporta61e incident. After information developed during the review was presented to the plant manager, an LER was prepared on February 27. 1987. The SE was informed when RHR cooling was stopped. That was the ~~ first notification to the SE that something was wrong, five hours c5 after the level indication was initially lost. The SE should have been notified when level indication was first lost. bs Suction to the RHR pump is lost at an RCS level of approximately 695 ?4 feet; the sight glass, however, showed a level of 696 feet 4 inches. Clearly the sight glass level indication was in error and unreliable. Nonetheless, water had to be added to the RCS promptly rg to allow restoration of RHR cooling, so the RO began charging from the RWST at an accelerated rate to restore level in the RCS. EP Despite the evidence indicating the sight glass was inaccurate, the - ' ~~ EP RO decided to add water to bring the RCS level up to 696 feet 6 inches, using the sight glass for indication. The RCS and RHR systems had previously been stable and operating satisfactorily at 2" that level. No attempt was made to establish an alternate means of 3 level indication until much later, when the operatins shift was relieved. 3 At this point, the oporators could have taken advantage of another independent means to determine when enough water had been added to the RCS to restore an adequate level. At about 0620. the RO began adding water to the RCS at a rate of 120 GPM from the RWST via the CVCS charging pumps. With the RHR system secured, there was no letdown; therefore, water was being added to the RCS about five times the rate at which it was being drained just before. An estimate of the time needed to bring the level to 696 feet 6 inches. l l

sould have beoo made based on the relative charging and desining rates, which wer. taown. Draining the RC3 at 25 GPN for approsisately 2 hours and 20 minutes produced a loss of section to the RNE pump. Adding water at 120 OPM for about 28 minutes, therefore,.should have returned the RCS level to near the original 696-foot 6-inch level. Actually, less time than that would be needed to reach the desired 695-foot 6-inch level. e However, no such estimate was attempted. At 0700,' after about 40 minutes of charging, P.he sight glass level had risen two laches to 696 feet 6 inches, the level desired by the RO. Being partially plugged, the sight glast level really had no validity, but it showed the level the RO wanted to achieve. Based upon charging rate and time, the actual level was probably'near the indicated level. It certainly was at a level where RHR cooling could be restarted. However,'the RO decided that.a still higher level was desirable, and continued charging without letdown. In making this decision, the R0 again placed total reliance on the RCS level indicator, even though it was clearly. inaccurate when RHR suction was lost. Based on . Interview results, the decision to rely on the level. Indicator and to continue charging without letdown was made without advice or p*.. , direction from control room supervision. There was no clear criteria for determining when and if sufficient water had been .D added, except by eventually reaching the overflow point. By'0714, the RCS level had reached 697 feet (the approximate elevation of the .fN bottom of the steam generator manways), and water was spilling from the manways. The decision to secure work in the steam ganerators when RER suction' was lost at 0620 was prudent, since no one was in the steam f9 generator at the time of the spill. g T' Throughout these events it appears that ASE and SE involvement was minimal. The RO performed activities and made decisions with .) minimal guidance, support, or overs!sht from supervisors. c In order to provide water to the reactor coolant pump seals, a CCP Q was operated continually while the RCS was in the partially drained condition. Because this amounted to continuous makeup of 75 GPM to O the RCS, an offsetting 75 GPM letdown flow was required to maintain RCS level approximately constant. This feed and bleed operation may not have been necessary, compilcated RCS level control, and was a contributing factor to the eventual spill. One of the review team members was an experienced operations manager 4 from Duke Power Company on plants similar to SQN. He stated that since the RCS water level was below the RCP seals, no seal flow was needed. This was separately confirmed by a Westinghouse representative. Since primary chemistry was stable, use of the 1 4

purification ion eschangers was apparently not needed. Therefore, the CCPs and letdown flow could likely have been secured leaving the RCs level basica111 static with ne significant gain er less of water. This conditten would have been significantly more stable than the. actual; condition established. Three measurable parameters indicating RCS system stability were RCS water consumption (as-measured by the makeup to the VCT), makeup and letdown rates (which could be read to'about 12 GPM of actual flow), and RCS level.. The following discussion. illustrates the difficulties experienced in using these parametere and indicates the need for TVA to reconsider using the feed and bleed process when RCS level is below the RCP seals. The CVCS charging pumps took suction from the VCT and discharged to the RCS.and the RCP seals. RHR letdown discharged to the VCT. Consequently,:if' RCS level was maintained constant, normal water losses from the CVCS and RCS resulted in a gradual decrease of VCT level. About 600 gallons of water was added to the VCT from the RWST daily prior to the. spill. Based upon a review of the VCT volume stripchart during' stable operation af ter the spill, the normal p, addition rate with a stable RCS level was roughly half' that. 600 gallons every two days. Had additions of water to the VCT been Ch recorded In the operator logs, those entires may have been useful in identifying the excessive makeup rate. However, these additions Ps-were not recorded. Additions of water to the VCT were reflected on the VCT volume strip chart, but reviewing the strip chart is CY cumbersome and not routinely performed. Addition of water to the VCT st twice the normal rate Indicated that to the CYCS makeup and letdown rates were not balanced. The imbalance resulted in a not coolant volume incre'ase in the RCS of about MT 300 gallons per day. Assuming the additional 300 gallons of water was added at a constant rate over a 1-day period, the makeup / letdown I 'I imbalance would be lest than 1/2 GPN. That imbalance could not be C3 detected on the makeup and letdown flow meters. Consequently, that measurable parameter was incapable of showing the imbalance. C The sight glass was, therefore, the only available Indication of RCS C3 makeup / letdown balance. It was indicating a constant 695-foot 6-inch level that erroneously indicated the system was balanced. The sight glass system was plugged. Af ter the VCT level was increased on January 28, the sight glass reading suddenly incressed by about 11 inches to 696 feet 6 inches. The action of filling the VCT and the sudden increase in RCS level l are probably coincidental. In all probability the indicated level increase was due to a temporary release of the sight glass system I ' l \\

y l r 8 blockage because of lacreased level in the RCS and resultant higher differential pressure across the blockage. The resultant level more escurately reflected the true RCS level. l The conclusion that the resultant level was more accurate is bered ) upon calculatfor.s of RCS volume using data from drawings. t information obtained from Westinghouse, and letdown and charging rates provided by the operators. During the initial RCS level reduction..the RO letdown from the RCS at a rate of about l 20 to 30 GPM for 140 minutes for a total of about 4.200 gallons. The volume of water le the RCS between elevation 696 feet 6 laches, and elevation 695 feet (the level where RHR suction would be lost) is approximately 4.700 gallons. After RHR suction was lost, the RO stated that charging was begun at 120 GPM and later reduced to 70 CPM. No accurate time is known for the reduction in charging rate to 70 CPM but based upon discussion with the RO It is assumed to be about 0700 or 40 minutes after charging began. At the assumed cha'rging rate and time for each rate, approstmately 5.800 gallons had been added to the RCS when the spill occurred. The calculated volume of the RCS between elevation 695 feet and 697 feet (bottom of the manway) is approximately y 5.200 gallons. cd As previously stated, the sight glass blockage was only temporarily released. Once the sight glass level initially increased to N 696 feet 6 inches. Its level did not change by more than four inches during the entire subsequent letdown and makeup evolution. The N actual RCS level change was about 24 inches between loss of RHR I suction and spillage from the panway, to One of the reasons that the RO and ASE assumed the sight glass level T was accurate was that the watee level in the sight glass was i oscillating about 1/8-inch (a normal condition). Using only the oscillation of the meniscus as a reliable indication of normalcy was Y not well advised, because oscillation could have been produced by other means, such as. trapped gas in the sensing lines vlbrations, C-or minute changes In containment atmospheric pressure. O C In all probability, the slow response of the sight glass during initial letdown was considered normal. With RCS level normally at about the 695-foot 6-Inch level (approximately six inches above the centerline of reactor coolant piping), more water would have to be added or removed to change RCS level a given amount than at any l l higher level. The amount of water required to change RCS level one inch at this normal level was not known at the time of the spill. but according to personnel interviewed, was later determined to be I between 400 and $60 gallons per inch. The 400 gallon per inch number has been posted on the TV monitor, and is now known by the f operators. 1 9 { l

Actions by the R0 and ASR indicate that a constant RCS volume per inch ratio was assumed for all elevations. In reality, 400 gallons per lach may be accurate at some level, but that ratio decreases as the reactor coolant piping is filled above the 695-foot elevation. With the horisontal portions of reactor coolant piping full at 696 feet 2-1/2 inches, Westinghouse approximates the volume per inch retto of the reactor vessel with laternals at 89 gallonsiper inch therefore, the level change per gallon added would be about five times that with the horizontal RCS piping half full. Further analysis adds perspective to the significance of the decrease in volume per inch ratio. Raising the RCS level from 695 feet to 696 feet 2-1/2 inches (a total of 14-1/2 inches), requires approximately 4,300 gallons of water and takes about 43 minutes at the 100 CPM rate used. Raising the RCS level from 696 feet 2-1/2 inches, to 697 feet (a total of 9-1/2 inches), requires only about 800 gallons. In this event, at the 70 CPM charging rate, when the RO had charged water to the 696-foot 6-inch level, the RCS was only eight minutes away from s' pilling water from the aanways, and the RO did not know it. While these calculations are only rough approximations, they point in out the value of operators having a sense of RCS volume per inch ratio information, which they did not have. ch RCS level control had been troublesome in the past primarily due to bs problems associated with level monitoring using a tygon hose. 04 Typical problems reported included entrained air in the tube, loss of level visibility due to movement of the remote TV camera, and poor visibility on the control room TV monitors due to poor lighting l of the tygon hose scale. A loss of suction to the RER pump in s9 August 1983 was attributed to inadequate level indication resulting from a blockage in the tygon hose system. That loss of RHR prompted IT a Field Change Request in December 1983 to fabricate and install a If permanent sight glass for RCS level indication. The sight glass was installed and first used on unit 1 in September 1985. =a According to site personnel, the design and installation of the 3 sight glass had minimal Division of Nuclear Engineering support. ) Much of the design work was performed by site personnel. The 3 Installation of a sight glass was to be the permanent solution to j the problems encountered with the tygon hose system; but'that was l not the result. A key factor. apparently not considered'in the design, was that the sight glass was to be viewed remotely via a TV camera. The sight I j glass scale was installed about 2-1/2 inches behind the sight l glass. The physical location of the camera in relation to the sight glass was not speelfied, and instrument mechanics were allowed to l position it where they wanted. For this event, the TV camera was i 4 _

I located about 7 feet above the 695-foot 4-lach region of interest. ] This configuration produced a 3-inch parallas errors the observed SCS level was three inches below.the actual level. Though parallas can be detected and eliminated easily when reading the sight gisse locally, it is not easy to detect and correct on the remote TV monitor. The parallas error was corrected by a temporary scale affixed by operators close to the sight glass. As of March 6, 1987, no permanent correction of this parallas problem had been developed. Visibility of'the altht glass level was also a problem. The sight 1 glass segments have a red stripe for visibility enhancement. The red stripe becomes grootly magnified when viewed through water, so i that the sight glass appears completely red in the portion l containing water. The active segment, however, was rotated sufficiently to render that aid ineffective. To compensate, a card with diagonal lines was added behind the sight glass. The diagonal lines, when covered by a water-filled sight glass, appear to change direction. However, the lines on the card were too narrow to be fully effective. In addition, sight glass lighting was provided by a portable extension light that could be easily moved out of place. The sight glass system had a number of other problems that could

• O have contributed to blockage or level errors. The tubing connecting the sight glass to the'RCS had numerous local high points, resulting (b

from heavy objects being placed on unsupported sections of tubing, bending it downward. Several hoses and cables were observed lying N across or hanging from the tubing. Those high points trap gas in pockets as it comes out of solution, producing level errors. In a similar configuration, several trapped gas pockets were observed in the tygon hose level indicator high points after it had been connected for a time. In addition, the sight glass tubing was not P? Installed with a continuous rise throughout its run to ease removal of trapped gases. 9 Both the sight glass and tygon hose were attached by a common u oolpiece to an ICS low point drain. Foreign material and C corrosion products naturally collect at these low point drains. This, combined with the very low flow rates to and from the level O' indicator, and the presence of a 3/8-inch orifice on the ICS side of the spoolplace, results in an installation prone to plugging. O Should plugging occur at the orifice, the most likely place, both level indication systems are affected. At the time of this spill, the sight glass and connecting spoolplace were not routinely flushed to ensure the sensing path did not become blocked. SQN personnel described a flushing procedure that was used following this incident, but it did not flush through the orifice and out of the system. The procedure described actually blew air through the orifice and into the RCS. carrying foreign material into the RCS. Since the foreign material is only being relocated within the system, it remains available to plus the system again. 4 5;

~ The review team member from Duke Power Company provided information to the team and to 8QN personne1'on a superior reactor coolant level indicating system used by Duke Power. This system employs two independent level sensing loops, each tapped into a separate RCS loop and each containing a level transmitter with remote indication in.the control room. One of the loops also contains a local sight ? glass siellar to that used at SQN. For this system, then, two independent level indications are available in the control room, neither of which dapend on temporary installation of TV and lighting. In sunenary, the following factors were significant causes.or contributors to the severity of this event. ,4 1. There was only one method utilized to measure RCS level, the sight glass..That sight glass was connected to the RCS low point drain containing a 3/8-inch orifice that was susceptible l to plugging. An alternate method of measuring RCS level, a 1 tygon hose, was available but not used. Even if it had been l. used, both the sight glass and tygon hose would have been affected if.the 3/8-inch restrictor became plugged. Even though the sight glass showed indications of unreliability (i.e., rapid rise of about 11 inches for no apparent reason and lack of decrease in level during RCS draining), operations personnel N continued to use the sight glass as though it were reliable. CN 2. There was no periodle flushing of the sight glass system and its N connection to the RCS low point drain through a 3/8-inch orifice. The low point 3/8-inch restriction is susceptible to N plugging, and periodic flushing is needed to promote good fluid conssunication between the RCS and the sight glass, especially since this is the only means of level indication. 3. A feed and bleed operation was in progress on the RCS. The CVCS T was operating to provide water to the RCP seals. Therefore, askeup and letdown rates had to be balanced to maintain the RCS V level constant. The makeup and letdown rates could not be i adequately balanced using available instrumentation other than I the unreliable RCS level sight glass. Supplying water to the o RCP seals when the RCS is drained below the seals may be unnecessary. .O i I 4 The deelslon to intentionally change RCS level after the unexpected approximate 11-inch increase and before the cause of l the increase was known, was inappropriate. The level was l abnormally high, but RHR cooling was operating normally, there l was no water in the steata generators, and no water was being )' added to the RCS. Consequently, conditions were relatively stable. Time was available to study the situation and plan a j l methodical approach for determining the cause(s) and returning the system to normal. . l

5. Available technical resources within.the control room were not sufficiently directed toward analysing ano solving the problem. The-20 appeared to be both the primary inesstigator and controller of the event and had insufficient support from supervision. The SE was not notified ustil-the situation degraded to the. point that RHR cooling was lost. 5 hours after l the level.became indeterminate. 6. There were no procedures available for maintaining level in RCS. Operators uere allowed to maintain and adjust levels as they thought best. I 7. When available procedures were considered not appilcable, Insignificant, or incorrect, there was a willingness to operate without procedures. There was no emergency requiring immediate operator action without the aid of procedures.. In fact. there-was ample tlke to study and plan a. response to the. initial level increase. Plant administrative instruction, (AIs) require critical systems structures and' components (CSSC) operations to be performed by procedure. However.-based on discussions with operators, it is normal practice to perform routine control room operations on CSSC without procedures. In this incident, undue ca importance uas placed on promptly adjusting RCS level when operating conditions did not reflect an urgent noeu. Ch 8. The volume per inch retto in the RCS at different elevations was D* ' not known, and operators could not readily compare expected level changes with actual level cha,nges. eq February 1. 1987 Solli P7 The RCS was being maintained in the same condition as described for the January 28, 1987 spill. Clearance bounderles had been 97 established for maintenance on the primary side of the steam 37 generators. C7 Several factors contributed to this spill, and as is often the case, correction of any one of several errors could have prevented the C3 incident. The causes and contributing factors are discussed below. The January 9 night order emphastning the importance of timely completion of surveillance tests and the possibility of a 350,000 fine and/or escalated enforcement action for failure to complete tests on time were strong factors in the minds of the operators responsible for performing the test. As a result of this and other factors, the operators did not consider delaying any of these costs in order to obtain better instructions, even though they recognl ed that the available written instructions were inadequate. 4

• l l

sis are often performed on weekends, when the level of activity is lower than on week 4ays. On or about Saturday, January 31, 1987, l I several sis were seat to the control room for performance.. One, SI 166.0, was for stroke testing five Amerlean society of Mechanical Engineers (ASMI) Section II valves, including 1-FCT-43-1. l Available written procedures for stroke testing 1-FCT-63-1 were not adequate for the existing conditions, and operators took no steps to obtain adequate, foraally approved procedures. 1-FCT-63-1 is j normally tested in cold shutdown with a pressure in the RCS. Under a these conditions, when 1-FCV-63-1 is stroked, flow through 1-FCV-63-1 is prevented by a check valve held shut by RCS pressure. When the SI was first issued, it included, as appropriate, an initial condition that RCS pressure be 30 pounds per square inch i gage or more. This condition is necessary to ensure that RCS pressure is greater than the gravity head pressure from the RWST so that the check valve remains seated and no flow passes through 1-FCV-63-1. That regulroment was deleted in a later revision for reasons that are not now apparent.' As a result, the current initial conditions for.this procedure were no longer adequate. Under the current conditions, with the reactor vessel partially drained and at atmospheric pressure, gravity head from the RWST g would force water into the RNR and RCS systems unless its path was blocked by shut valves. N The RO knew that testing 1-FCV-63-1 would be more complicated than when normally performed with the RCS closed and pressurized. A N valve realignment would be required. Nevertheless, no written Instruction for this stroke test of 1-FCV-63-1 was prepared. Instead, the 10 devised a valve lineup from memory without referring t9 to flow diagrams. This action was apparently consistent with typical plant operating practices, but not with plant AIs.

• f Plant AIs require all CSSC operations to b'e performed in accordance Y

with written, approved instructions. The SQN procedural system provides a simple method for on-shift preparation and approval of e needed procedures. Even though that provision was available, normal 'O practice was reportedly to manipulate CSSC valves and pumps for routine operations without written procedures. Had a written O procedure been prepared, the increased attention to its preparation and review would likely have been enough to ensure that an adequate valve lineup was made before cycling 1-FCV-63-1. The ASE granted permission to perform the test by signature on SI 166.8. However, the ASE stated that approval signified permission to perform the SI, but not a review or approval of the test method to be used. The R0 had discussed the impact of the test, and the valve lineup to prevent flooding the RCS, with a trainee. The ASE and the SE were aware that the RO and trainee had.. _ _ ___ _ _ _ _J

l discussed the conduct of the test, were comfortable with the RO's knowledge and capabilities, and chose not to get further involved. As a result, no qualified person backed up the R0's decision or preparations by verifying that the valve lineup was adequate. Nad the supervisors become more involved, they might have prevented the - 20 free conducting the test-before a satisfactory valve lineup had been established. e Having obtained poemission to perform the SI, the 20 and trainee stopped the operating RHR pump and shut valves isolating the RWST from the RCS, but thsy neglected to shut the two 14-inch IHR suction valves from the RCS. sellering that the plant'was properly lined up, the RO now felt ready to cycle 1 FCV-63-2. Valve 1-FCV-63-1 was one of the boundaries' utilized in the clearance instruction to provide personnel safety during steam generator primary side maintenance. The protective boundary was established in accordance with an approved maintenance procedure, which improperly allowed the use of a cahtion. tag on 1-FCV-63-1. A caution tag was used instead of a more secure hold tag because of the'possible rapid need for a source of borated water in the event. of a loss-of-coolant accident (r..OCA). With the RCS partially drained, depressurized and with the steam generator manways open, C3 the likelihood of a LOCA is very small and the need for 1-FCV-63-1 to be immediately operable is not evident. The hazard to personnel C2 working in steam generators should have been of more immediate concern. rc eq Caution tags are designed to identify abnormal equipment operating conditions and to provide special instructions for the operation of the equipment. Plant equipment tagged with caution tags, including valves, are routinely operated based on the operator's knowledge and PO determination of need, considering the. instructions.on the tag. As required by the tag, the RO received permission to open 1-FCV-63-1 'I from the ASE. Notification of po'esonnel working on the steam 37 generators was neither required nor given before the valve was opened. The ASg had not taken.the appropriate steps to ensure that Cp the valve could be operated safely before giving his permission. Therefere, in effect, the intent of the tag instructions, to ensure C) careful attention before operating the valve, was not met because of g) the lack of attention from the ASE. In most instances, caution tag instructions should state the specific actions needed to permit safe operation. This can help protect against oversights such as occurred in this case. Hold tags are required by SQN AI-3 when personnel safety is involved, and a hold tag would have been appropriate for 1-FCV-63-1 in this condition. Hold tags are assigned by name to an individual responsible for the safety of the personnel protected by the tags. No equipment can be operated with a hold tag attached, and hold tags n,- a. c

sannot be removed without documented approval of the perses for whom they are issued. The approved malatenance lastructies was la violation of the AI fsr tagging and did not provide the appropriate level of personnel safety for the steam generator workers. Had a hold tag.been on the valve, the' additional formal steps required to establish a substitute barrior and remove the hold tag would likely have prevented the spill. Though AI-3 clearly requires use of hold orders (tags) to permit work to be safely performed, it could be improved by more clearly stating that caution tage cannot be used -for personnel safety. When 1-FCV-63-1 was opaned, the RCS sight glass level quickly rose past the steam generator overflow point. Water flowed by gravity from the RWST. backwards through the 14-!nch RHR suction valves, and into the RCS number 4 hot leg. The result was the spill of several thousand gallons of water from the open steam generator manways without warning. Actions taken after each spill by operations to analyze events. determine causes, and pass on information did not reflect a high level of concern regarding the significance of each spill. Written statements were prepared by personnel directly involved immediately following the spills. preparat!on of written statements immediately.after an event ensures O some information is captured while fresh in the minds of those involved. Those statements were provided to'the SQN spill Investigation team that was conducting a thorough evaluation of each eq spill. Further evaluation of the statements and gathering of additional information was not performed by operations, For example, no operations personnel at the ASE level or above entered containment for the purpose of examining and evaluting the spill M areas and RCS level indication systems for contributing causes of each spill. 7 v Though there was certainly considerable discussion among operations personnel directly involved in the spills, there was no structured C feedback to other operations shifts or the other T7A sites about either spill. Collecting and disseminating information soon after o en event is beneficial in capturing focts, actions, impressions, and p* causes as they actually occurred. Fu fewing both these spl11s, however, the principal mechanisms used lor the transfer of information were word of south and night order. Even though a formal investigation was in progress, a more aggressive information gathering, analysis, and feedback system is needed to provide timely, effective feedback to operations personnel on the causes and results of operational events. l ......r.

After the first spill, two actions were takan to improve RC8 level' indication reliability. First, the sight glass system was' flushed to remove the blockago.. That task was not as effective es it could have been, because one step blew the blockage material into the RCS with untiltered air. Second, the tygon hose was installed for a redundant indication of level. Although the tyson hose was capable of providing as accurate a level indication as the sight slass, it-i was used with a known one-inch error in its scale, and level differences of several inches betwen the sight glass and tygon hose were allowed to exiwt with no explanation or corrective action. Establishing a periodic sight glass system blowdown and removing personnel from the steam generators before any planned changes in RCS level would have br.en appropriate additional corrective actions after the first spill. Following the second spill, several steps were taken to provide a short-term solution to problems encountered and to allow steam generator work to. continue. On February 1, 1987, a night order was issued that, for the first time, established a level band for the RCS, 695 feet 6 inches to 697 feet. The lower level was si inches above where RHR suction would be lost, and the upper level was at the steam generator aanway overflow point. The band was revised on p. February 9 to between 695 feet 6 inches and 696 feet 8 inches, giving a four-inch buffer between the upper allowable level and the C overflow point. O Additional safety precautions associated with steam generator work g and actions to improve RCS level reliability were initiated by a night order dated February 1, 1987. That order required consaunications between the control room and steam generator work area.to be verified operable each shift and required all operations M that could cause,a change in RCS level to be conusunicated to the steam generator work crews. In addition, whenever personnel were in T a steam generator, the motor operator for 1-FCV-63-1 was deenergized y to prevent inadvertent opening of that valve. In order to provide a source of water to flood the reactor core, an ADO was to be C-. stationed at 1-FCV-63-1 for manual operation when its motor operator was deenergized. O O In order to increase reliability of RCS level indications the February 6 night order required the sight glass to be flushed periodically. A flushing procedure was being developed during the review and was not available for evaluation. The night order also required the tygon hose to be placed in service once per day and its level compared with the sight glass. These actions were sufficient to allow steam generator work to continue, but did not address long term corrective actions. 1

There were, however, some problems with the corrective measures taken by plant staff. One of these had to do with level comparisons between the sight glass and the tygon hose. Comparison checks were intended to be a check on the accuracy of the level indicators. No criteria were established for the agreement between the two level indicators. Level differences of up to six inches existed for days without explanation. They were a result of gas accumulation in the tygon hose and incorrect placement of the tygon hose scale. A level agreement criteria ci three inches was estabitehed for a one time l performance of SI-166.3. An additional criterla was established l that the level indicators agree within one-half inch before allowing steam generator work to resume. Though the Indicators were made to j agree within three inches, steam generator work was resumed with more than one-half inch disagreement. These two level indicators, properly adjusted and maintained, were clearly capable of agreement l within one half to one inch. This sort of an agreement criteria would be appropriate to provide assurance of accuracy, but operators were satisfied with less accuracy b.ased on the readings that had been experienced and the range of level to the overflow and loss of RHR points. Operations personnel reported tha the plant manager emphasized on February 9, the importance of following procedures. He stated that, I. where adequate procedures did not exist, they were to be developed C3 before proceeding. That action produced confusion among the operators. They were used to performing some routine operations on co CSSC equipment without procedures. That practice had been allowed for some time and they felt that procedures were not available for CM many of their routine evaluations. This effort to improve procedural adherence by operatcrs will likely not be successful without more guidance and direction to operators and substantial p) follow-up attention by managers and supervisors. %T l C? C3 C) j

.m ) 7. FINDINGS The findings contained in this section were developed free information gathered during the. investigation of both spills. For the'most part thess findings repeat what has been. described in greater detall in Section IV Analysis. They are presented in this section to provide a sunusary statement of the findings for each spill, a grouping of the findings into areas for potential corrective action, and to provide a 4 few findings not disc'essed in the analysis section. A. Sight Class Deslan. Inste11stion and Maintenance Concerns 1. Current methods for monitoring RCS level (tygon-hose and sight glass) are not sufficiently_ reliable. Errors in indicated level have resulted in a loss of RHR cooling on two occasions (August 6. 1983, using the tygon hose and January 28, 1987, using the sight glass). Both installations were connected to an-RCS low point through a common." connection containing a 3/8-inch ortflee. The installation appeared to plug on both occaslons. Specific problems noted with the installation include the following: i v a. Tubing connecting the unit 1 sight glass to the RCS does not have a continuous upward slope to promote purging C the tubing of gas. b. The sight glass remote monitoring TV camera was not N installed in an optimum position. The comblaation of camera position and sight glass scale construction produced a remote monitoring parallax error of about three inches that had to be corrected with a toeporary M. scale. c. Sight glass lighting provided only marginally acceptable y remote visibility, and could be easily moved out of place by persons in the area. C d. The 3/8-inch restrictor in the permanent piping of the O sight glass increases the potential for blockage of g-level sensing piping, e. Prior to the January 28, 1987 spill there were no plant procedures for maintenance or flushing of the sight glass system, the only method available for RCS level monitoring. Following the spill, a special flush of the sight glass system blew potential blockage material back into the RCS with unfiltered service air. That practice could introduce impurities contained in the unfiltered ~28-

i air. Subsequent flushing did not include verifying flew through the 3/8-inch orifice, the most likely location of pluggirs. f. Red striping on the sight glass segments was rotated l tros its proper position such that it was ineffective in enhancing level visibility. .e g. The tygon hose scale was not installed accurately. -Rough m9asurements taken in reference to en elevation benchr. ark indicated that the tygon hose scale was set approximately three inches higher than the sight glass scale. Since the January 28, 1987 spill, the tygon hose and sight glass levels have been recorded daily for comperison, but no criteria for_ acceptable differences had been given. Records reviewed revealed variations in the two levels from aero to several inches with no corrective measures specified. 8. Procedure Adeouser and Adherence-1. A significant majority of operations personnel interviewed-openly expressed willingness'to disregard portions of 1.n procedures considered by the user as either incorrect, inelsnificant, or not applicable. Management has emphasized C the importance of procedural adherence. Operators, however, are used to performing routine operations without procedures U where they are either not available or are not adequate. This g practice has not been corrected by management. Clearer guidance and management follow-up are needed to achieve an appropriate level of procedural edherence for operators. M 2. The survalliance instruction governing the stroke testing of T 1-FCV-63-1, SI-166.3, did not provide adequate prerequisites or instructions to safely perform the test under existing plant y conditions. C-3. There are no written procedures covering some incidental, routine operations of CSSC equipment such as pump and valve O operations and maintaining RCS level. Operations personnel had become accustomed to operating'CSSC equipment for these incidental evolutions without using procedures. These habits may have fostered the lack of formality and supervisory involvement preceding both spills. 4 Temporary procedures were not prepared and used as required despite existing SQN provisions for them. The development and approval processes for temporary procedures, had they been followed, would have forced a degroo of supervisory oversight which could have resulted in better plant control and possibly g -.. -.. -.......

prevented the' spills. Operations personnel were not familiar with precedures for preparing and approving temporary procedures, and did not understand how temporary precedures should be used in circumstances siallar to these spill' events. The valve lineup and procedure used by the reactor operator to stroke test 1-FCV-63-1 on February 1. 1987, were neither written er approved as required nor were they reviewed by any other qualified individual. 5. Established cl u rsace boundaries for steam generator work were not adequate., Clearance procedures require the use of hold tags for personnsi safety, but only caution tags were used to prevent operation of three valves (1-FCV-63-1, 1-FCV-63-135, and 1-FCV-63-136). 1-FCV-63-1 was subsequently: operated without adequate attention or notification of affected personnel working on the steam generators. Careful adherence to confined space entry instructions in Nazard Control Instruction G8 might have prevented this error, because that instruction requires a job saf.ety analysis before entry into confined spaces such as the steam generator primary plenum. 6. Operations logkeeping practices'need substantial improvement to' ensure that relevant information reflecting plant conditions 4 and operations is. recorded. Significant evolutions and events such as CCP and RHR pump starts and stops, VCy makeups, and-o level abnormalities were not recorded in operating logs and journals. Sight glass level entries not recorded when the RCS CPS level was unknown were later filled in using estimated data. Missed entries should not be backlogged. In cases such as N this, when levels to be recorded are not known,_ missed log entries should be noted as such with an explanation of why they were missed. Such practices compilcate event reconstruction pi) and do not provide a good record of plant events for review by relieving shifts and operations supervision. W 7. Agreement criteria between RCS level indicators were not I appropriate and, in cases. were not observed. The temporary p change to SI-166.3 dated February 4, 1987, established agreement criteria of three laches between the sight glass and si) tygon hose. This tolerence was based on the level difference 5S experienced, and not on the capabilities of the installation. With the scales properly aligned, these direct reading indicators are easily capable of much greater accuracy. A separate instruction to operators required the level indicators to agree within 1/2-inch prior to allowing steam generator work to resume. Steam generator work was, nevertheless, resumed and continued with greater than 1/2-inch disagreement between the two indications. (

i 8. A temporary change to WI-144.3 dated February 25, 1987,. resulted is a mar ud-up precedure that contained incomplete requirements. partially revised steps, and confusing sequencing. Operations personnel reported that the document, in its approved form, was very difficult to follow. Though considerable effort was put into it, the purpose of the change was not clear since it expired before the nest scheduled test of any component (valve) affected. 9. Procedures foe pertially draining the RCS did not specify a control band for 3CS level in the partially drained condition. A level control band was first established by a night order of February 1. 1987. following the second spill. C. Practices 1. Operation of CYCS charging pumps with the RCS drained below the RCP seals, may not be necessary, and complicates RCS level control by substantially increasing the potential for level changes. It also promotes wear on CCP discharge throttle. valves due to the high' differential pressure across the valves with the RCS depressurized. D*' 2. Follow-up actions immediately following abnormal events need C3 to be substantially strengthened. Although a formal investigation was initiated for long-term corrective action, ca the immediate collection, analysis, and dissemination of information and development of corrective actions within the C\\' operations section was not~ timely. No formal critique or feedback sessions were cor. ducted to learn or transfer information on causes.' lessons learned..or proposed corrective pg action to all shift operations personnel and other sites. Information regarding the events of both spflis was distributed ~. 97 within the operations section primarily by word of mouth. 'T 3. Decontamination and cleanup efforts following the second spill CF were not thorough. Two personnel contamination events resulted from residual spill water in and around the steam C3 generator lay down areas following area decontamination and cleanup. I C? D. Steam Generator Foreign Material Erelusion (FME) 1. FME measures did not block one path for entry of materials into the RCS loop piping. Though cleanliness covers were installed in the RCS loop nozzles to the steam generators, the 0.6a-inch dealn ports between the steam generator lower cavities and the loop risers were not blocked as they should have been. l . ~...

- e 2. Af ter the February 1.1987 spill, six ceraale beads and a small amoust of heat treataeat probe insulatten asterial may have entered the RCs whos the flew of water from the loops displaced the clearilisees severs. The beads apparently de not present a problem since they dissolve under elevated pressures and toeperatures In borated water. Determination of appropriate ccrrective action for the insulation material was in progress at the time of this investigation. 3. Nessures to ensure steam generator workers were familiar with FHg measures needed strengthening. Craftsmen working on the steam generators vere required to be briefed on FME practices before beginning work, but no records were made to ensure each worker had, in fact received the briefing. d .c co-(Y P? %7 C? C3 C3 (

I -{ o LIST OF ACIONfMNS USED IN THIS REPORT i AI Administrative Instruction i ASE Assistant Shift Engineer ASME Amerler.n Society of Mechanical Engineers i AUO Assistant Unit Operator CCP Centrifugal Charging Pump CVCS Chemical Volume Control System o CSSC Critical Systeins, Structures, and Components l FME Foreign Material Exclusion GPM Gallons per Minuto Ch IM Instrument Mechanic ES LER Licensee Event R,eport 9 LOCA Loss of Coolant Accident NMRG Nuclear Manager's Review Group RCP Reactor Coolant Pump g p RCS Reactor Coolant System F RHR Residual Heat Removal ? 20 Reactor Operator RVST Refueling Vater Storage Tank 9 SE Shift Engineer SI Surysillance Instruct!on SQN Sequoyah Nuclear Plant SRO Senior Reactor Operator TV Television VCT' Volume Control Tank VR Work Request 4

ATTACHMENT 1 TO ENCLOSURE 2 1 l I f t i l 1 l l 1 1 1 l l

i iJ y. bo m eo lC b s s s s s s s is o o o o o o e e e e o o e e e ss N N N N N N Y Y Y Y N N Y Y Y se ec co cr AP m s s a o o o o e e o o o o o o o o o l N N N N Y Y N N N N N N N N N A r e d s s s s s s s r o o o o o o e e e e o o e e e o N N N N N N Y Y Y Y N N Y Y Y F c O e L R N E O V I E 2 T L A E C D R I N n U D A yo l l l l l l S N ai a a a a a a O I E l t c c c C R R R R R R c c R R R L R pa o o o O C C C C C C o o C C C C G U sc L L L L H M N H M M L L M H M N N T io E I A DL D R F I E O V P O M 1 R E P T TN N E O E M I R p p p H T U p C A S p p m m m m A T S m m e e e e T N E e e t t t t T E R t t A M P t t t t U e e e e t t e e e e e R S r r r r e e l l l l e r e T C e e u u u u l l t t t t r u r S R r r s s s s n n u u u u u t u N u u s s s s i i o o o o t a t I s s e e e e a r a s s r r r r r r r r r r r e r e e p p p p e e e e e e e p e r r g g g g g g p m p p p e e e e n n n n n n m e m g g g g a a a a a a e t e n n r r r r h h h h h h t t n o o a a a a c c c c c c g o i i h h h h x x x x x x g e g i t t c c c c e e e e e e e l e t c c s s s s l l o u u i i i i t t t t t t d i s S d d d d a a a a a a t l t r e e e e e e o o o c A B A B A B H H H H H H h c h s e A B A B A B A B A B A B I 1 2 D p p p p p p p p p p p p p p p m m m m m m o o o o o o o o o u u u u u u o o o o o o o o o p p p p p p l l l l l l l l l R R R R R R R R R R R R S S S H H H H H H H H H H H H C C C R R R R R R R R R R R R R R R t ne 2 8 3 6 4 5 9 9 8 0 8 4 m 4 2 6 1 1 2 1 2 2 3 3 4 1 1 2 ur re 4 4 4 4 4 4 4 4 4 4 4 4 8 8 8 tb 7 7 7 7 7 7 7 7 7 7 7 7 6 6 6 sm nu I I I I T T E E E E I I E E E IN P P P P P P T T T T T T T T T }ll l

x e n r o 'y c t o i bn n a e i c ea i lC e d b s s s s s s s s s s s s h n is e e e e e e e e e o e e e t i ss Y Y Y Y Y Y Y Y Y N Y Y Y se y t ec b a co h cr r t AP o ~ t s a e r t e o p n o e _m' d e 'o s s s s s h l o o o e e o 'o o e e e t a A o 1 N N N N N Y Y N N N Y Y Y c A o o t L d e d m i o r v o e 's o r d s s s s s s s r r e e e e e o o e e o e o C p l o Y Y Y Y Y N N Y Y N Y N N o c s r i t e R n e o r c u t n a 'n r ia e m ai ~ p yo m e l t R R R R R R A R R R R R R e h pa C C C C C C / C C C C C C t t sc N H M M H H N M H M M H H io e n DL r i o c l 2 e l n e a s p se t v ne r n o am. tc r a et e pn e e e r a r e r e r al u r u r u f p t a t u t l o n t a t a t a e e e e e oe r a r a r r r r r v n h e r e r e u u u u e o dt p e p e p s s s s l i e

n p

m p m s s s s t dn e m e m e e e e e e a ii t e t e t r r r r g c v t t p p p p n r r r i on n g g g a e e e d ro o e g e g e g g s g r z z z n pi i l e l e l e e e e i i i i t t l l l l l l e r r r sa o d d d d u u u t ic i l t l t l t t t t i s s s c o r o o o o o o o o o w s s s e nl c c h c h c h h h h e e e r o s r r r r i ir e 2 3 3 4 4 4 4 1 3 e p p p d t e D z ah p p p p p p p p i n n n a ct o o o c o o o o o r o e o io o o o c o o o o o u l l l I l l l l l s 1 2 3 e, dnem s s i S S S S S S S S S e p p p e o C C C C C C C C C r a a a h ts R R R R R R R R R P T T T t a o. ht ta t se sd nl ee t op td n 1 0 4 3 i u oi e 1 3 0 5 3 6 8 3 4 2 4 3 2 t o nv m 4 4 6 6 8 6 6 6 6 3 3 3 3 i c eo ur d o d r re 8 8 8 8 8 8 8 8 8 8 8 8 8 d m p tb 6 6 6 6 6 6 6 6 6 6 6 6 6 ar R sm e Cs nu E E E E E I T I I I T T T nh Hi IN T T T T T P P P P L P P P I t 7 i; l}}