IR 05000361/1986011

Insp Repts 50-361/86-11 & 50-362/86-11 on 860328-0512. Violation Noted:Water Level Indicator Installed W/O Appropriate Procedure & Failure to Comply W/Station Procedure for Reactor Startup

ML20206S011
Person / Time
Site:	San Onofre
Issue date:	06/20/1986
From:	Dangelo A, Huey F, Johnson P, Stewart J, Tang R, Tatum J NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION V)
To:
Shared Package
ML20206R930	List: IR 05000361/1986011 ML20206R926 ML20206R942
References
50-361-86-11, 50-362-86-11, NUDOCS 8607070275
Download: ML20206S011 (23)

Text

_ _ _

.

U.S. NUCLEAR REGULATORY COMMISSION

REGION V

Report No /86-11, 50-362/86-11 Docket No , 50-362 License No NPF-10, NPF-15 Licensee: Southern California Edison Company P. O. Box 800, 2244 Walnut Grove Avenue Rosemead, California 92770 Facility Name: San Onofre Units 2 and 3 Inspection at: San Onofre, San Clemente, California Inspection conducted: ar h 28 through May 12, 1986 Inspectors:hFT M () Mb ey, Senior Resident Da'te Signed Inspe or U its 1, 2 and 3 h n war , Resident Inspector Lizda Date Signed

@

,A.D'An

. . , - c/-/n lo, Resident Inspector Date Signed Nf2 Ev kJ. un, esident Inspector Date Signed w  % In N . C. T g, Resident Inspector Date Signed Approved By: [ & 2.[f t, P.H.Joh@n, Chief Date Signed Reactor PMjects Section 3 Inspection Summary Inspection on March 28 through May 12, 1986 (Report Nos. 50-361/85-11 and 50-362/85-11)

Areas Inspected: Routine resident inspection of Units 2 and 3 Operations Program including the following areas: operational safety verification, evaluation of plant trips and events, monthly surveillance activities, monthly maintenance activities, refueling activities, independent inspection, licensee events report review, and follow-up of previously identified item Inspection Procedures 37701, 71707, 73051, 62703, 73052, 73753, 93702, 62700, 72701, 62703, 60710, 92700, 92701, 61726, 60705 and 92702 were covered.

(

.

O

l

-

g

\

'

-2-Results: Of the areas examined, two apparent violations were identified: (1)

installation of a quality affecting water level indicator without an appropriate procedure (paragraph 8.c) and (2) failure to comply with station procedure for reactor start-up (paragraph 8.a).

J

- - - - - - - .. --- - - - , , ,- . . . , , - --, . , , . . - - -- , , , -

i l

. i

,

l DETAILS

.

1. Persons Contacted Southern California Edison Company H. Ray, Vice President, Site Manager

• G. Morgan, Station Manager

• M. Wharton, Deputy Station Manager

• D. Schone, Quality Assurance Manager D. Stonecipher, Quality Control Manager

• R. Krieger, Operations Manager

• D. Shull, Maintenance Manager J. Reilly, Technical Manager P. Knapp, Health Physics Manager

• B. Zintl, Compliance Manager J. Wambold, Training Manager D. Peacor, Emergency Preparedness Manager P. Eller, Security Manager

• W. Marsh, Operations Superintendent, Units 2/3

• V. Fisher, Assistant Operations Superintendent, Units 2/3

• B. Joyce, Maintenance Manager, Units 2/3

• R. Santosuosso, Instrument and Control Supervisor

• T. Mackey, Compliance Supervisor G Gibson, Compliance Supervisor .

• C. Kergis, Compliance Engineer

• P. King, Quality Assurance Supervisor

• R. Waldo, Plant Computer Supervisor San Diego Gas & Electric Company

• R. Erickson, San Diego Gas and Electric

• Denotes those attending the exit meetin The inspectors also contacted other licensee employees during the course of the inspection, including operations shift superintendents, control room supervisors, control room operators, QA and QC engineers, compliance engineers, maintenance craftsmen, and health physics engineers and technician . Operational Safety Verification The inspectors performed several plant tours and verified the operability of selected emergency systems, reviewed the Tag Out log and verified proper return to service of affected components. Particular attention was given to housekeeping, examination for potential fire hazards, fluid leaks, excessive vibration, and verification that maintenance requests had been initiated for equipment in need of maintenanc . Evaluation of Plant Trips and Events

_ - _ _ _ _

_ _

2 Reactor Trip on April 12, 1986 (Unit 3)

On April 12, 1986, at 1630 the reactor tripped from 100% power due to loss of load when the turbine tripped. The turbine trip was caused by a voltage transient in the non-1E uninterruptible power supply (UPS). A similar reactor trip had occurred on Unit 2 on August 1, 1985, and a change to the turbine control system was implemented to prevent future spurious reactor trips. This change I to the turbine control system was not implemented on Unit 3 at that time, but the licensee has now implemented the change to eliminate future spurious reactor trips on Unit Reactor Trip on April 13, 1986 (Unit 3)

On April 13, 1986, at 1116 while the licensee was conducting a reactor startup to return the unit to service, the reactor tripped when criticality occurred earlier than predicted. The reactor went critical with regulating group 4 control rods withdrawn l approximately 80 inches, and group 4 control rod withdrawal continued to approximately 114 inches before the control operator realized that the reactor was critical. At this time, regulating group 4 control rods were inserted to approximately 98 inches. The

reactor exceeded 10 % power with regulating groups 1, 2 and 3 -

control rods slightly misaligned. With these groups slightly d, group 4 rods at 98 inches, and reactor power greater j

misaligng%,

than 10 the core protection calculators (CPCs) tripped the reactor on departure from nucleate boiling ratio (DNBR) and high local power density (LPD). Although an unsafe DNBR or LPD condition did not actually exist, the control rod configuration caused the CPCs to generate penalty factors sufficient to trip the reacto The estimated critical position (ECP) for criticality was 60 inches j on regulating group 6 rods. However, the licensee subsequently I determined that the ECP was in error because the xenon tables used to determine the xenon reactivity worths were incorrect. The licensee corrected the xenon tables.and the unit was returned to service on April 14, 1986. Additional follow-up investigation of the circumstances surrounding this reactor trip is included in paragraph 8.a of this repor . Monthly Surveillance Activities Unit 2 During the current refueling outage, the inspector observed operation of diesel generator 2G002 during the 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> operational test which was conducted to satisfy the 18 month surveillance requirement. The diesel appeared to operate satisfactorily during the test. The inspector observed that inside the local control cubicle, the following conditions existed:

A red circuit and a green circuit were found to be terminated. No tag was found to indicate the status of these circuit . _ - - . _ - - -- - - -

_ . . - .

.

9

A red jacketed cable was found routed through the control l cubicle, which appeared to be a temporary installatio The cable was not labeled to indicate its statu ]

Some small diameter rope was found under the cubicle which appeared to be routed through a penetratio In response to the inspector's observations, the licensee documented these items on Field Surveillance Reports E-164-86, E-177-86 and MN-186-8 In addition, the licensee issued Problem Review Reports S0-164-86, S0-175-86 and Corrective Action Request S0-F-1157 to ensure corrective actions are properly pursued. This is an open item (50-361/86-11-01). Unit 3 The inspector observed several shutdown margin calculations associated with the Unit 3 reactor trip that occurred on April 13, 1986. The calculations were conducted in accordance with the procedure, and were satisfactory in all but one instance. The licensee discovered an error in the calculated xenon reactivity worths which are used by the operators, and this had a significant effect on the shutdown margin which was calculated during the reactor start-up on April 13, 1986. Additional discussion regarding this issue is included in paragraph 8.a of this repor No deviations or violations were identifie . Monthly Maintenance Activities Unit 2 The inspector observed maintenance activities associated with In-Service Inspection (ISI) of the hot leg nozzle welds. While conducting the hot leg ISI, a fixture which supports four of the transducers-broke. The ISI equipment was removed from the reactor vessel, and the broken fixture was replaced with a spare. The inspector observed that proper QC and FME requirements were implemented for this activit Unit 3 The inspector observed portions of the repair activity to replace a power supply associated with CPC channel The power supply was starting to show signs of degradation as evidenced by spurious alarm indications. The power supply was replaced, and a satisfactory functional test was complete No violations or deviations were identifie . Engineered Safety Feature Walkdown During the inspection period, the inspector walked down portions of the component cooling water and saltwater cooling systems for Unit 3. The

_ _ _ - _ - _ - - _ _ _ _

'

inspector noted that all the valves examined were in the proper position required for system operabilit No violations or deviations were identifie . Refueling Activities The inspector observed several fuel movements during the Unit 2 refuelin No violations or deviations were identifie . Independent Inspection Follow-Up Inspection on April 13, 1986 Unit 3 Reactor Trip Due to the unusual circumstances surrounding the Unit 3 reactor trip discussed in paragraph 3.b of this report, additional inspection effort was necessary to assess the significance of the event and licensee performance in dealing with the event. The inspector reviewed operating logs, interviewed operations personnel and core physics engineers and reviewed data from printouts associated with the Critical Functions Monitoring System (CFMS) and neutron flux level instrument (1) Sequence of Events On April 13, 1986, the licensee conducted a reactor startup following a spurious reactor trip that had occurred on April 12, 1986. The estimated critical position (ECP) was 60 inches

, on group 6 control rods. The regulating group control element i

assemblies (CEAs) were being withdrawn by a trainee who was being supervised by the Assistant Control Operator (ACO). The i Control Operator (CO) was coordinating the startup activities under the direct supervision of the Control Room Supervisor (CRS), and the Shift Superintendent (SS) was overseeing the reactor startup on Unit 3 as well as outage activities associated with Unit 2. The SS and CRS were both licensed senior reactor operators, and the CO and ACO were both licensed reactor operators. The shutdown bank CEAs and the part length CEAs had been withdrawn before the trainee had arrived in the control roo Prior to withdrawing the regulating group CEAs, the trainee reviewed the startup procedure with the ACO. The CRS directed the trainee to start withdrawing the' regulating group CEAs and, as group 1 CEAs reached the upper group stop (UGS), the CRS directed the trainee to continue pulling group 2 CEAs instead of stopping to make the final minor adjustments to group 1 CEAs 1 (referred to as " dressing the rods"). The procedure cautioned that if the CEAs in groups 1, 2, and 3 are not properly r (CPC) reactor trip could dressed, a Core Protectioneeded Calculatg%

10 (the CPCs automatically occur if reactor power come out of bypass at 10exg% reactor power). The CRS wanted to

.. - - - . - - . _ - -. ._ . .- .. __ _

.

make the reactor critical before xenon decay could significantly affect the ECP calculation and had intended to after the reactor reached criticality but before dress the 10 exceeding CEAg% reactor power.Initially, as the trainee withdrew the regulating group CEAs, he stopped periodically to allow the startup rate (SUR) and neutron flux level indications j to stabilize. The CRS did not allow much time for hesitation,

'

however, and continuously directed the trainee to withdraw the CEAs. As pointed out previously, the CRS wanted to expedite the reactor startup so that xenon decay would not have a significant impact on the ECP calculation. After the trainee had withdrawn groups 1, 2, and 3 CEAs to the UGS, he asked again if the CEAs should be dressed. The CRS directed the trainee to continue pulling CEAs to make the reactor critical, r

As group 4 CEAs were starting to come up off the bottom of the core, the trainee noticed an increasing SUR and flux leve Since the reactor wasn't supposed to reach criticality until '

group 6 CEAs were withdrawn 60 inches (based on ECP calculation), the CRS assumed that substantial CEA withdrawal was still required before the reactor would be critical. The trainee continued to withdraw the control rods and, with group tely 80 inches, the reactor went

4 CEAs withdrawn critical at about I approximg% X 10 reactor powe The power dependent insertion limit (PDIL) for criticality - group 5 CEAs withdrawn more than 60 inches -- was not satisfie The trainee continued to withdraw the control rods. As group 4 i ange CEAs neutronwere detectors approaching de-energized 100 inches withdrawn, at approximately 5X the 10source g%

!

, reactor power. The absence of the source range audible indication was recognized by the CRS, CO and ACO, but none of

the operators made the correlation that the reactor was already I critica to withdraw group 4 CEAs,

the CPCs came Asthetraineecontinueg%reactorpowerandtheHigh out of bypass at 10 Log Power Level Trip Bypass Permissive annunciator subsequently came in. This annunciator alerts the operator to bypass the high log power trip so reactor power can be increased above

,

0.89%. The CO bypassed channels A and B of the High Log Power i

Level Trip, but before channels C and D of the High Log Power

, High Pre-Trip

'

LevelTripwerebypassed,theLogPowerLeve}%reactorpower annunciator came in at approximately 9 X 10 .

The trainee had withdrawn group 4 CEAs to approximately 114

inches, and finally started inserting CEAs on his own initiative. The ACO directed the trainee to stop inserting the control rods and the trainee stopped momentarily. The CRS,

,

'

after observing the positive SUR with no CEA movement, directed the trainee to insert the CEAs. The SS concurred with the CRS and the trainee inserted group 4 CEAs to approximately 98

'

'

,

inches, where the reactor tripped. The operators initiated boration of the reactor coolant system and commenced taking post trip actions in accordance with applicable procedures.

l t

. _ - _ _ _ _ . _ . , _ _ _ __,_ _ _ . . _ . _ . . . . , . _ . _ _ , _ _ _ . - - - - _ . _ _ _ _ _ ._.. . - - - ~

_ _w .a 4, m .#. ..2 ,._ 12 4 4 ..as

.

he reactor trip, reactor power During the 6 minutes had increased from 10pg% to 10 eceding,g%andSURhadachieveda maximum value of approximately 1.6 decades per minute (dpm).

(2) Evaluation of Operator Actions The procedure which was used to perform the reactor start-up directs the operator to anticipate criticality at any tim Clearly, during this reactor start-up, criticality was not anticipated. The following factors contributed to this event:

, (a) Inadequate Supervision: The CRS was holding over from the previous shift and was not accustomed to working with the

shift that was on duty. The operators were relatively

inexperienced and the ACO had previously performed only one reactor start-up. The reactor start-up was rushed and

'

did not provide the trainee with proper emphasis on how a reactor start-up should be conducte t (b) Operator Inattentiveness: During the reactor start-up, i

the CRS was focusing his primary attention on CEA j position, based on the calculation, instead of properly

'

monitoring all available indications. The AC0 and the trainee were primarily monitoring the SUR and control rod

, position indications, and not paying adequate attention to the neutron count rate and log power level indications.

i The recorders for the neutron count rate and log power i

level channels were also not properly monitored. These

recorders provided indication that the reactor was

approaching criticality several minutes prior to the l reactor trip. When the source range instruments i deenergized and the audible count rate indication stopped,

, none of the operators made the connection that the reactor 4 was already critical. Additionally, the CRS was filling out his turnover sheets and preparing for turnover during

{ the reactor start-u (This improper activity was

,

observed by the SS prior to the trip and CRS attention was redirected to the reactor start-up).

(c) Failure to Follow Procedures: The start-up procedure cautioned that criticality should be anticipated any time

., the CEAs are being withdrawn. The procedure also required q that the operators closely monitor all channels of nuclear

!

instrumentation while starting up the reactor, and

periodically stop CEA withdrawal and wait until the nuclear instrumentation trends can be determine In conducting the reactor start-up, the operators.did not properly adhere to these start-up procedure requirements

.

and precautions, j (d) ECP Error: The ECP which was calculated for the reactor

, start-up was in error. The ECP error resulted because the table used by the operators to determine xenon reactivity worths was incorrec Initially, during cycle 1 operation i

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ . _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ . _ _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ____

_. - - _ . . - - - - __ . __ - .- - _ . - - - . . .

l l

l l'

of Unit 2, the licensee determined that the predicted values provided by CE were inaccurate. The licensee used a correction factor to adjust the values such that the xenon reactivity worths more closely represented the actual plant conditions that existed following a reactor shutdown. The licensee notified CE of this chang Administratively, this change to the xenon reactivity worths was made to both Units 2 and 3 as follows:

• The Nuclear Design Data Book (NDDB) was changed for each unit. The NDDB is the licensee's master compilation of reactor physics data, and changes to the NDDB receive independent verification that the

data are correc ' The data used by the-operators was copied from the

] NDDB for each unit and entered into the Operations

'

Physics Summary Book (OPSB). This is an

] administrative function, and only receives one level of review. The operators use the OPSB to determine values for xenoa reactivity worths for ECP

calculation During cycle 2 operation on Unit 2, the licensee had originally applied the previously used correction

factor to the CE data to arrive at the predicted l cycle 2 xenon reactivity worths. The licensee's

~

predicted values turned out to be incorrect, and the licensee subsequently determined that the predicted

'

values that had been supplied by CE for Cycle 2 were correct without applying a correction factor. The

operators were instructed to use the xenon reactivity

'

values that had been predicted by CE for cycle 2 operation of Unit 2 via a memo. Contrary to existing

'

administrative procedures, the operators then changed the OPSB for Unit 2 to incorporate the xenon i reactivity values as specified by the memo. Because the administrative procedures had not been followed, the change was not made to the NDDB for Unit 2. When

the NDDB was compiled for cycle 2 operation of Unit 3, the data was gathered essentially by obtaining it from the Unit 2 NDDB which did not have the correct data for xenon reactivity worths. Even though the Unit 3 NDDB received two levels of review, this error went undetected and was subsequently included in the Unit 3 OPSB for cycle 2 operation.

i

Findings The licensee's actions to correct the weaknesses in the areas of

shift supervision and operator attentiveness during reactor plant operation and training evolutions will be examined, and is an open item applicable to Units 2 and 3 (50-362/86-11-01). 1 J

_ _ . _ _ _ _ _________ _ ,__ _ _ _ , _ _ _ _ . _ _ _ , _ . , , , _ . . - . _ _ -_ _ . _ _ _ _ _ _ _ . _. _ ,_ -. , , , _

-- . _ -. . . . _ - -_ _-

The failure of operations personnel to follow procedures during a reactor plant start-up which resulted in a reactor criticality before the PDIL was satisfied and subsequent reactor trip is a violation applicable to Unit 3 (50-362/86-11-02).

The licensee's failure to follow administrative procedures when implementing changes to the NDDB and OPSB was recognized to be an apparent violation of NRC requirements identified by the licensee for which appropriate licensee actions were taken or initiate Consistent with Section IV.A of the NRC Enforcement Policy, enforcement action was not initiated by Region V. However, the licensee was requested to review their verification policy as it applies to engineering calculations which are used in conducting plant operations. This request was made when the inspector noted that ECP calculations, when performed by station engineering, do not receive documented verification. This is an open item (50-362/86-11-03).

4 Evaluation of Post Trip Review Following the reactor trip discussed in 8.a above, a post trip

! review package was completed. The operations personnel involved with the event were individually debriefed by the Operations

! Division Duty Manager, and the STA completed the event identification record. A post trip review meeting was also held to ensure that the reactor trip was well understood and that all corrective actions had been identified. Among those who attended the meeting were the Station Manager, Operations Manager, Station l Technical Manager, Operations Division Duty Manager, Shift Technical Advisor, and the Shift Superviso '

The inspector's review of the post-trip review package and discussions with persons who were involved indicated that the post-trip review had been conducted in an effective manne However, the following observations were made:

Although the cause of the reactor trip was appropriately discussed in the post-trip review meeting, the review package did not describe it completely; i.e., t the reactor had

,

achieved criticality early, exceeded 10thg% power, and encountered off-normal control rod positions which resulted in

, the trip. These matters were, however, discussed at the post-trip review meeting and responsibilities for correctiv actions were assigned.

Although the SS and CRS participated in the post-trip review meeting, they were not individually interviewed before the meetin Licensee actions to improve the post-trip review process will be

examined during a future inspection. (50-362/86-11-04)

i

,

No violations or deviations were identified, i

l I

k f

,..,,.-..-n,,, , , - - - . , - , , . . _ .

.-r - - - - - -

- . - _ _ ___

-- -

c. Follow Up Inspection on March 26, 1986, Loss of Shutdown Cooling (Unit 2)

(1) Summary of Event:

On March 26, 1986, at 2150 PST Unit 2 lost shutdown cooling due to the loss of pump suction and pump cavitation. At the time of this event Unit 2 was in mode 5 with reactor coolant system (RCS) temperature at approximately 115 F and the RCS loops

drained to below midloop in the hot leg (approximately 10" of water in the hot leg, which is 64" above the tope of the core).

The event was initiated by inadvertent draining of too much water from the RCS due to malfuntioning reactor vessel level instrumentation. The bulk RCS temperature rose from 115 F to approximately 190 F between 2208 and 2255, when forced flow through the reactor core was reestablished. Core exit temperature exceeded 200 F for a period of approximately 15 minutes and boiling within the reactor core existed for approximately seven minutes. As a result of the boiling, a small gaseous airborne release of 2 curies (primarily Xenon-133) occurred between 2255 and 013 It was determined that no fuel cladding damage occurred as a result of the boiling and that the gaseous release was the result of previous fuel cladding failure (2) Personnel Interviewed The inspector's interviewed the following Units 2 and 3 personnel concerning the March 26, 1986 shutdown cooling even *

Assistant Operaticas Superintendent Operations Shift Superintendents

i

Control Room Supervisors (Six CRSs)

Control Operators (Two R0s)

Assistant Control Operators (Two R0s)

Nuclear Plant Equipment Operators (Three R0s)

Steam Generator (SG) Nozzle Dam Maintenance Foreman

SG Nozzle Dam Maintenance General Foreman

Station Technical Engineers

Refueling Group Engineers Health Physics Technicians and Engineers (3) Documentation Reviewed The inspectors reviewed the following documents to determine the sequence of events and to assess underlying causes:

,

Unit 2 Reactor Operator Log, March 15-27, 1986 Licensee Event Report 86-07, April 25, 1986

'

Harold Ray Letter to Dr. Papay, (April 7, 1986)

'

Engineering Evaluation, (March 31, 1986)

Loss of Shutdown Cooling Sequence of Events, (March 27, 1986 Debrief Following the Event)

'

S023-VIII-1, Recognition and Classification of Emergencies

Maintenance Orders 85123242, 86013697,86032305, 86033112

S023-0-36, Attachment 2, Abnormal Alignment and

,

Evolutions, Completed on March 26, 1986

S023-0-36, Control of System Alignments

S023-3-1.8, Draining the Reactor Coolant System

S02-SPE-1, Refueling Water Level Indication (RWLI)

S023-II-9.14, Foxboro Differential Pressure Transmitter Series E13 Calibration (For RWLI)

(4) Sequence of Events Leading Up to Incident The specific sequence of events leading up to the incident was as follows (Primarily from operator logs):

March 19, 1986

-

0740 Commenced draining RCS to 50 percent pressurizer leve Technicians recalibrated RCS water level indicators 1520A and 1520B due to errors noted in base design data by a ,

reviewing enginee (1520A is the narrow range, 0-42 inches of water in the RCS hot leg piping).

2326 Commenced draining RCS to radwaste per S023-3- March 20, 1986 1305 Secured draining RCS, water level at 20" on narrow range (1520A).

1410 Placed RCS eductor in service for hydrogen gas and radioactive gas removal prior to opening RC Shift supervisor accelerated maintenance (SSAM) initiated on refueling level indicators 1520A and 1520B due to inaccurate readings (1520A read zero inches with eductor in service). Shutdown cooling pump motor current and pump flow stabl March 21, 1986 0107 Secured RCS eductor for investigation of refueling level indicato I&C technicians troubleshooting level indicators under SSA Level indicator inaccuracy resolved with eductor secured. Level dropped 6.5" after draining water from reference leg. Narrow range now reads 13.5" with eductor secure Placed RCS eductor in service. Refueling level indicator inaccuracy recurs.

,

,-

- -.

March 22, 1986 0235 Removed refueling indicator from service to install tygon tube indicator in parallel with RCS level indicators 1520A and 1520 Tygon refueling level indicator placed in service and reads -78" below reactor vessel flange (RVF) (mid hot leg loop). Shift superintendent observes that both of the remote RCS level indicators are failed low (they were taken out of service at 0235; no time was noted when they were returned to service. Discussion with involved operations and maintenance personnel indicate that the level instruments were placed back in service about 1300).

0945 RCS level - 80" (Tygon) below RV Secured RCS eductor, RCS level -83" (Tygon) below RV RCS eductor returned to service after removal of steam generator (SG) primary manway cover March 24, 1986 0000 RCS water level -83" (Tygon) below RVF. Level indicators 1520A and 1520B, and eductor in service. SG nozzle dams being installe March 25, 1986 1332 RCS water level estimated at -83.5" (Tygon) below RV Secured RCS educto March 26, 1986 0000 Mode RCS water level -83" (Tygon) below RVF. SG nozzle dams being installed. Eductor secure Started high pressure safety injection (HPSI) pump P019 to fill RCS in preparation for detensioning and removal of reactor vessel hea Completed filling RCS. RCS water level is -5" (-0.4 feet)

per level indicator 1520B (wide range) and -22" per Tygon tube. Mismatch of 17 inche SG nozzle dam on cold leg to RCP P003 leaking.

'

1450 Initiated draining RCS to 6" below middle of RCS hot leg to support repairs of SG nozzle dam. This evolution was being performed in accordance with the abnormal alignment and evolution operating instruction S023-0-36, which was approved by the Units 2 and 3 operations supervisor. The

_ _ _ _ _ _ _ , _ , _ _ _

- -

.

procedure as written by the CRS required that the RCS water level be lowered to approximately -83 (Tygon) or approximately 13" (1520A, narrow range), which was the RCS water level prior to the RCS fill evolution from 1015-1120 (three and one half hours earlier). The precaution to monitor LPSI pump amps and pump flow was also included in the procedure. The narrow range indicator, if correct, should read 15" (6" below 21", which is midloop), and the Tygon tube, if correct should read -85.5" when six inches below midloop. In actuality the Tygon tube indicator being used was a scribe marking (magic marker) on a lamp pole in containment, which was previously used for reactor coolant pump seal replacement several years earlier. The scale, which was later determined to be in error by 2",

also did not go below -82", and therefore, any level below this level was estimated by the operator. Discussion with involved operations personnel identified that the rationale for draining to -83" on the Tygon tube was that the Tygon tube read approximately 2 inches higher than a.ctual level. And the rationale for draining to 13" on the narrow range 1520A indicator, instead of 15", was that it read 2" lower than the actual RCS water leve Therefore, a mismatch of 4 inches existed between the tygon and narrow range indicator The Unit 2 control operator's log entry states " completed draining RCS to hot leg midloop tygon tubing reads -77".

Based on interviewing the three Unit 2 control room operators on shift at 0630 - 1830, the inspector noted that several events occurred during the period before 1715 which were not

~

logged or passed on to the oncoming shift at 1815 - 1830 during turnover. The occurrence of the following events were also not communicated to the Units 2 and 3 shift superintendent:

During the RCS drain down, the operators noted that the tygon tubing was not tracking with the level indicators 1520A and 1520B. (1520B and the tygon were in disagreement by 17" when the drain down started at 1450.)

Operators secured draining several times when RCS water level was between 25" and 13" on the 1520A indicator. The operators secured draining to see if the tygon tube indication would decrease to the same level of -83" as it had in the morning when the 1520A indicator was 13".

With RCS water level indicating -77" (tygon) below the RVF and the indicator 1520A now indicating 13", the four inch mismatch (-83" on tygon and 13" on 1520A) had now increased to a ten inch mismatc *

The operators were not certain which indicator was correct, therefore, they decided to drain below 13" on the 1520A indicator. At this point, after draining for

. - - - - - - - -_.- . .

approximately 3 minutes, the operators noticed the shutdown cooling pump amps (normal 50 amps) start to oscill te between 45 and 55 amps (one operator indicated 40 to 60 amps). The tygon level decreased from -77 to-78".

The operators recognized the onset of vortexing due to pump cavitation and secured draining. The operators immediately filled the RCS by adding water for 2 minutes using HPSI pump P019 until evidence of pump cavitation ceased. RCS water level was -77" (tygon) and 13" (1520A indicator).

The operators did not feel that the increase in level indicator mismatch or pump cavitation was significant enough to pass on to the oncoming shift. The reasons appear to be that these events were not significant in that t'ea RCS was at the same level on the narrow range 1520A indicator as in the morning, the RCS water level would remain at this level for only a few more hours until the SG nozzle dam was repaired, and the tygon tube would be removed in a few days. Also, they stated that if maintenance would have requested additional draining, they would have investigated the level mismatch, because of the observed pump cavitatio (5) Event Sequence The specific sequence of events during the shift when the incident occurred was as follows:

March 26, 1986 1800 Oncoming shift, which had been off for four days and had not worked on Unit 2 in a week, is briefed at pre-shift briefin Turnover to oncoming shift complete Maintenance general foreman requested that the shift superintendent have the RCS water level lowered 12 inches due to high water level in the cold leg loo The general foreman based this request on a request by the foreman repairing the SG dam (at 2000) that

'

the water level be lowered because he had splashed water on himsel Unit 2 control room supervisor (CRS)

completed S023-3-1.8 attachment 3 " Draining the RCS to the Refueling Water Storage Tank".

2130 Closed 2 BRA 16 (electrical breaker to 2HV8163) and opened 2HV8163, LPSI P016 mini flow to lower RCS

level approximately 6", to 83" below RVF, per maintenance reques Throttled shutdown cooling (SDC) flow (SDC) to 2000 GPM to facilitate drain down and prevent vortexin Secured RCS draining by shutting 2HV8163 valve due to amperage swings on LPSI P016. Tygon tubing indicates level at mid-loop (-78").

2150 Stopped LPSI P016 due to amperage swings of 20 amps and low suction pressur Restarted LPSI P016. Amp swings steadied for a few minutes and started swinging agai Stopped LPSI P01 Started LPSI P015; ran steady for 3 minutes, then flow and suction pressure went to zero.

l 2208 Stopped LPSI P015. Reviewed loss of SDC abnormal operating instruction and dispatched operators to vent SDC suction piping outside penetration and at LPSI P01 Notified health physics (HP) that SDC venting was planne Shift superintendent notified. SS directed SDC suction piping and LPSI P016 be vented. Quite a bit of air vented by 223 Shift technical adviser (STA) notifie Unit 3 CRS evaluated emergency plan implementing procedures: notified Unit 2/3 operations superintendent at hom Completed venting and restarted LPSI P016. RCS hot

-

leg loop post accident monitor indicator (PAMI)

increased to 210 Received both trains FHIS. Gas channels indicated 300 CPM. Motor current steady on P01 Started HPSI P019 to make up to RCS. Lowest indicated level was 9.5" on 1520 Requested HP to evacuate containmen Stopped make up to RCS, hot leg temperature PAMI is less than 200 F and decreasin Requested chemistry samples of RC __ -- - -. . ._ - _ _ _ _ _ , - -

- - - . . _---_

--

2315 Started make up to RC Level at 13" on 1520A at 233 March 27, 1986

-

0000 RCS cold leg at 170 F (discharge of SDC heat exchanger), RCS water level 13" on 1520 Operations superintendent in control roo Stopped HPSI P019. Requested chemistry sampl Commenced cooldown to 120 Red phone report to NR Received RCS chemistry sample result NRC resident inspector notified of even (6) Safety Significance of Event (a) Radioactive Gaseous Effluent As a result of the boiling in the RCS, a radioactive gaseous effluent release of 2 curies (primarily xenon 133)

occurred during a two and one half hour period. The maximum release rate from any one hour period was 1 css than one percent of the limits of the Emergency Plan Implementing Procedures for the declaration of an Unusual Event. The total exposure of an individual at the Exclusion Area Boundary would have been less than 0.002, mrem for the releas (b) No Fuel Damage Occurred The licensee's analysis determined that the source of the radioactive gaseous release was previous fuel cladding failures. The licensee's analysis also demonstrated that no additional fuel cladding failures occurred during the loss of SDC event based on RCS chemistry samples; estimated maximum fuel temperature, conservatively assuming no heat transfer to coolant; and the total amount of radioactive noble gases release . (c) RCS Water Inventory Maintained RCS water addition could have been made at any time during

~

the loss of SDC event. HPSI P019 was used to maintain RCS water vessel after RCS flow was reestablished. Additional flow paths of RCS makeup water were also available including gravity drain from the refueling water storage tank and safety injection tanks (which were still filled).

- . _ . __-

_

.

The charging system was not available for water addition due to maintenance activities during the even (d) Boiling in Reactor Core for Seven Minutes The total loss of SDC flow lasted from 2208 to 2257. The estimated decay heat load at the time of the event was 6

, MW (thermal) or 0.18 percent full power. The heat rate was estimated at 0.14 percent of the critical heat flux valu The core " delta T" (the difference between the hot and cold leg temperatures) was approximately 19 F with shutdown cooling flow at 3000 gpm at 2130. Reduced flow through the core existed during several occasions from 2137 to 2208 due to the stopping or cavitation of the LPSI pumps or throttling of the SDC flow by the operators. At 2208, when the total loss of SDC flow was initiated, the hot leg core exit temperature was 116 An analysis performed by the licensee stated that, assuming no ambient heat losses, the decay heat input would have caused the average temperature of the entire RCS inventory to rise by 1.4 F per minute. Since there was no forced flow through the core, not all the RCS

inventory was available for circulation. The licensee analysis estimated that the mass of the " equivalent volume" (of water and fuel) available to circulate and/or remove decay heat during the loss of SDC was approximately 150,000 lbm. With this reduced " equivalent volume" the estimated time to the onset of boiling was 41 minutes (2249), close to the actual observation of the onset of boiling after 40 minutes (2248), indicating a heatup rate of 2.5 F/ minute for the " equivalent volume". Actual boiling continued from 2248 to 2255, when SDC flow was reestablishe (e) Technical Specification (TS) - Limiting Conditions for Operation (LCO)

(1) RCS Cold Shutdown - Loops Not Filled During this event T.S. 3.4.1.4.2 was apparently exceeded as the core outlet temperature was not maintained at least 10 F below saturation temperatur The inspector determined that during the event no core exit temperature indication was available available to assist the operators in complying with this LCO. The licensee had disabled the core exit thermocouples prior to the event, in preparation for RV head removal. The only temperature indications available were in stagnant leg ___ _

._ _ _ _ _ _ _ _ __ _

'

!

(2) Reactor Coolant System Pressure / Temperature Limits During this event T.S. 3.4.8.1 was exceeded in that a maximum heatup rate of 30 F in a one hour period with RCS cold leg temperature less than 280*F occured as follows:

(1) RCS hot leg temperature increased from 112 F at 2148 to 212 F at 2248, an increase of 100 F in 60 minute (2) RC5coldlegtemperatureincreasedfrom115 Fat 2259 to 160 F at 2304, or 45'F in a 5-minute perio (The RCS cold leg temperature had previously increased from 104 F at 2208 to 132 F at 2255 before restarting of shutdown cooling flow at 2255).

An engineering evaluation performed by Combustion Engineering.at the licensee's request concluded that these heatup rates were acceptabl (7) Contributing Causes of the Loss of SDC (a) Installation of the Tygon Tubing Without an Appropriate Procedure A specific written procedure was not used to install the tygon tubing which was used to monitor reactor vessel water level. A quality class 3 " Blanket" maintenance order was used to document effort performed by maintenance personnel installing the tygon tubing; however, the maintenance order contained no written requirement covering the installation of the tygon tubing level indicator. Due to the lack of a proper written procedure, the tygon tube was installed in the wrong. location and with a loop seal, which allowed an air bubble to become entrapped and resulted in a significant nonconservative error in level indication. The improperly installed tygon tube was used by plant operators to verify proper reactor vessel water leve Improper water level indication was the primary contributing cause to the loss of shutdown cooling. Failure to use an appropriate procedure for the installation of the tygon tubing, an activity affecting quality, is a violation (50-361/86-11-02).

(b) Inadequate Turnover Between Shifts at 1800 The offgoing shift at 1800 did not turn over to the oncoming shift the observed SDC pump cavitation and water addition to the RCS using the HPSI pump, which then eliminated the pump cavitation. This onset of vortexing was not passed on because it was determined not to be

- . . . - -=-.. . - - - -. . - . . - .-

.

4 significant in relation to plant conditions at the time of shift turnove I

'

(c) Inadequate Training of Personnel for MidLoop Operations

and Decay Heat Removal

4

'

Based on interviews of ten operators and two maintenance foreman the inspector determined the following:

!

Operators' general acceptance of known water level indication deficiencies indicated a lack of proper training. The tygon tubing was attached to an

,

indicator marked on a lamppost with a magic marker.

i Also, the magic marker indicator used only went to-l 2.5" below midloop, requiring operators to visually i estimate RCS water level below -82" on the tygon tub *

Operators lacked understanding of expected RCS water level decrease during Jraining of the RCS to the RWST. Although the RCS was drained for about 10 I

minutes at a rate of 100 gpm, the CRS continued to

! drain the RCS because the tygon tube water level did l not decrease. The narrow range indicator showed a j water level decrease of 3.5" during RCS draining from

! 2130 to 214 t

Operators lacked a thorough understanding of

vortexing, including indications of the cause of pump

'

amperage swings at low RCS water leve *

The control room supervisor (CRS), although having

difficulties with maintaining SDC flow from 2147 to 2225, did not contact the Si until 2225. Also, the CRS did not contact the SS when he reduced the SDC f

flow from 3000 gpm to 2000 gpm at 2135. Based on interviews of reactor operators, the inspector determined that operators in general were not aware of the amount of decay heat in the core, nor the expected time to the onset of coolant boiling, if the loss of SDC flow occurred. The operators indicated

'

, that there should not be any boiling for at least one l hour since the technical specifications indicated that SDC flow could be stopped for one hour.

i (8) Licensee Corrective Action The licensee's planned corrective actions to prevent future loss of SDC events are discussed in LER 86-07. The inspector

,

reviewed the proposed corrective actions and they appear to be adequate. The inspector will perform follow up inspection of

'

the implementation of the planned corrective action (50-361/86-11-03)

i

,

-7 , - --. ,. -- .--- ,,- -, - ,- , , , . - - , ~ . , , - - , - - , - , . . . _,,a .-r,. - - - - , - - . - - - - - . , .

- - -

,

9. Review of Licensee Event Report _s Through direct observations, discussions with licensee personnel, or review of the records, the following Licensee Event Reports (LERs) were closed:

Unit 2 82-01 CREACUS Fire Protection system Modification to Manual System (Rev. 1)

82-02 Loss of Shutdown Cooling and Reactor Coolant System Dilution Event 82-06 Stratified Boron Concentration in Radwaste Storage Tank (RWST) (Rev. 2)

82-12 Control Room (CR) Emergency Air Cleanup System Found Inoperable During Surveillance Test

82-15 Overspeed Trip of Diesel Generator During Monthly Operability Test 82-23 Uncontrolled Actuation of Cable Tunnel Auto Deluge Valve

,

'

82-29 Failed Limit Switch on Auxiliary Feedwater (AFW) Pump 82-37 Remote Shutdown Boronometer Inoperable 82-43 CREACUS Emergency Chilled Water Pump Would Not Start Due to Wrong Fuse Size 82-56 Failure of One Level Instrument on Each Steam Generator 82-78 DNBR Monitor Channel "A" Failed Channel Check Test 82-125 Level Control Switch Out of Calibration (Rev. 1)82-177 Surveillance Test for Fire Doors Not Being Done 83-106 Surveillance Requirements for the 125V Battery Banks83-112 Improper Number of Heat Sensors Found in Zones 12/42 83-130 Spurious Trip of Train "B" Emergency Chiller 83-144 0xygen Inleakage into Waste Gas System Due to Compression Fittings (Rev. 1)83-146 Nitrogen Pressure in Safety Injection Tanks Exceeded 84-66 Polar Crane Malfunction 84-70 Horizontal Refueling Cavity Inflatable Seal Failed In Service 84-79 Snubbers /18 Month Surveillance Results 84-79 Summarizes Results of Engineering Evaluation and Actions (Rev. 1)

85-15 Technical Specification Snubber Tables Inaccurate 85-22 Inadvertent Actuation of Safety Injection System 85-50 Reactor Trip Due to Failed Nipple and False Hi Level Signal from MSR Drain Tank 85-51 Reactor Trip on ASI 85-60 Reactor Trip ON ASI Unit 2 Special Reports

.

06-07-82 Shutdown Cooling System Relief Valve Used to Mitigate Reactor Coolant System Pressure Transient 08-20-83 Inoperable Fish Return System 08-26-83 Inoperable Fire Rated Assemblies

_ _ _ . _ ___

-

.

09-09-83 Followup on Missing Fire Penetration Seals 01-13-X4 Fire Barriers Not in Conformance with Fire Protection Plan 01-26-X4 Followup Report for Special Report on January 13, 1984 Unit 3 83-38 Cable Tunnel 3 Spray System Iaoperable l 83-54 Containment Spray Chemical Storage Tank Low Level Alarm Inoperable 83-55 Technical Specification Resistance Temperature Detector (RTD) Failed 83-56 Fuel Tank for Diesel Inoperable Level Indication 83-57 Inoperable Steam Generator Level Indicator due to Air in Instrument Lines 83-58 Containment Isolation Valve Stuck in Mid-Position 83-59 Main Steam Line (MSL) Radiation Monitor Inoperable 83-60 Control Element Assembly Calculator (CEAC) Inoperable due to Failed Input Amplifier Card 83-61 Pressurizer Pressure Indicator Inoperable During Mode 3 84-12 Main Steam Isolation Valves Inoperable 84-25 System Status Following Testing 10. Follow-Up of Previously Identified Items (Closed) Open Item (50-362/83-27-01) Environmental Qualification. Maintenance Program Status The inspector verified that the licensee's maintenance program does incorporate the vendor's recommendations, and a formal review process is used such that environmental qualification is taken into consideration. This item is closed, (Closed) Open Item (50-362/03-27-02) Administrative Controls Associated with AFW Lube Oil Cooler Shroud The shroud was installed and CAR S023-P-517 was issued to address the root cause such that disassembly and assembly of components is administratively controlled. This item is close (Closed) Open Item (50-362/83-27-03) Uncontrolled P& ids in the Shif t Supervisor's Of fice The uncontrolled P& ids were removed, and controlled F& ids were issued. This item is close (Closed) Open Item (50-362/83-42-01) Reactor Coolant Activity High During the first refueling outage on Unit 3, the fuel was examined ,

and 105 failed fuel pins were found which affected a total of 23 l fuel assemblies. The activity levels and dose equivalent iodine in i the reactor coolant has subsequently decreased to normal values upon I returning the unit to service. This item is close . _ __

.

.

21 (Closed) Open Item (50-362/84-11-02) Methodology for Specifying the Meaning of Signature Blocks The station policy regarding the meaning of signature blocks was promulgated by the Plant Manager in a memo to all station managers dated June 4, 1984. This policy was also included in Administrative Procedure S0123-VI- This item is close (Closed) Open Item (50-362/84-11-03) ICG Review of Maintenance and Surveillance Procedures The licensee initiated a program to review other procedures in the maintenance and surveillance area to_ ensure that independent verification is performed where appropriate to establish or maintain equipment operability. This item is close (Closed) Open Item (50-362/84-11-04) NRC Staff Review of Signature Policy The inspector has reviewed the licensee's signature policy which essentially states an individual signs for an action based upon firsthand knowledge. Additional discussion of this item.is included I in paragraph 10.e of this report. This item is close . Exit Meeting On May 1, 1986, an exit meeting was conducted with the licensee representatives identified in Paragraph The inspectors summarized the inspection scope and findings as described in this report.

.

.

i I

. _ _ . . . . . , - , - - - , ,, _ - - , . , - , - . . -