ML20045D589
| ML20045D589 | |
| Person / Time | |
|---|---|
| Site: | North Anna  |
| Issue date: | 06/30/1993 |
| From: | Kauffman J, Spence R, Steinke W EG&G IDAHO, INC. |
| To: | NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| Shared Package | |
| ML20045D357 | List: |
| References | |
| NUDOCS 9306290189 | |
| Download: ML20045D589 (28) | |
Text
-
' h '.
7 TRIP REPORT:
ONSITE ANALYSIS OF THE HUMAN FACTORS OF AN EVENT AT NORTH ANNA 2 ON APRIL 16, 1993 DISABLING AUXILIARY FEEDWATER SYSTEM DURING REACTOR TRIP RECOVERY John Kauffman Robert Spence William Steinke Published June 1993 Idaho National Engineering Laboratory EG&G Idaho, Inc.
P. O. Box I625 Idaho Falls, ID 834I5 Prepared for the Office for Analysis and Evaluation of Operational Data U.S. Nuclear Regulatory Commission Washington, D.C. 20555 Under DOE Contract No. DE-AC07-76fDOI570 I
l 9306290189 930623- !
EXECUTIVE
SUMMARY
t The Office for- Analysis and Evaluation of Operational Data of the U.S.
Nuclear Regulatory Commission has a program to study human performance during ,
operating events at nuclear reactors. As part of this program, a team conducted an onsite analysis of the event that occurred at the North Anna Power Station during the day shift on April 16, 1993. Unit 2 had been operating at 100% power, when a control problem developed in the main generator voltage regulator. Increased excitation on the main generator led to a field forcing (overexcited) condition, which initiated a control room annunciator. The unit operator attempted to manually lower the output of the i voltage regulator. Seventeen seconds after the field forcing annunciator, a generator differential lockout protection occurred and caused a main generator, turbine, and reactor trip. '
Control room operators entered the emergency operating procedures for plant recovery. While the crew implemented Procedure 2-ES-0.I, Reactor Trip Recove y, the reactor coolant system dropped below 547'F because of. full auxiliary feedwater flow to the steam generators. The procedure reader was at Step 6.and attempted to go back to Step 1 for instructions to throttle or control auxiliary feedwater flow for the purposes of limiting reactor coolant system cooldown, although Step 1 was not identified as'a continuous action step. At this point, the Unit 2 supervisor stopped the procedure reader and ;
communicated directly with the secondary operator. The secondary operator, ;.
after receiving permission from the Unit 2 supervisor to " secure auxiliary feedwater," established main feedwater flow to the steam generator and disabled the auxiliary feedwater pumps to stop auxiliary feedwater flow.
Reactor coolant temperature began to increase from 540 F. Recovery of the plant continued until the procedure reader reached Step 12, " Stopping AFW ;
. Pumps." Realizing the procedure still called for the pumps to be running and that the switches were in a pull-to-lock position (locked in the off position), the procedure reader informed the shift supervisor. -The auxiliary feedwater system was immediately returned to an operable configuration by the crew. AFW had been in'an inoperable configuration for approximately 18 .
minutes. The required heat sink conditions for the reactor, which calls for iii' -
one steam generator with a narrow range level >11%, were maintained while the auxiliary feedwater system was inoperable. !
The human factors analysis focused on the factors that influenced the '
performance of operations staff and technical support staff throughout this
- event. The analysis was based on data derived from interviews with operations ,
and technical staff, review of plant logs and recordings, and review of procedures and training material.
The following is a summary of the results of the analysis of the human factors in this event.
Operatino Procedures ,
Step 1 of Procedure 2-ES-0.1, Reactor Trip Response, requires adjustment of auxiliary feedwater flow if the reactor coolant system average temperature is less than the no-load setpoint (547 F), but was not a continuous action '
step. When the operators performed this step, reactor coolant system average
- temperature was trending down towards 547 F and this step did not require auxiliary feedwater flow adjustment. When reactor coolant system temperature decreased below 547*F several minutes later, the procedure did not provide i clear direction.
Step 2 of the same procedure required the operator to verify flow from the auxiliary feedwater system "or" the main feedwater system. No detailed ,
instructions were included in that step to support any system alignments.
Operator actions were taken based on this step to stop flow from the auxiliary feedwater system and initiate flow from the main feedwater system. As a result, the steps relating to auxiliary feedwater control were not used and ,
" skill-of-the-craft" was used to manipulate the feedwater systems, which rer,ulted in disabling a required engineered safety feature.
Feedwater heater relief valves lifted because of the number of feedwater ,
and condensate pumps running after the trip with only one recirculation path to the condenser. The emergency operating procedures do not direct the i
iv !
i
operator to remove extra equipment until late in the recovery procedure.
Earlier shutdown of this equipment may have corrected the over pressure condition in the feedwater system allowing the feedwater relief valves to reseat.
D Command. Control and Communications Management oversight of system configuration was not effectively; maintained during the plant recovery. Several contributing factors led to this situation. Closed-loop communications were not always used by the crew to ensure information was received. The terminology _used was interpreted differently by sender and receiver, and a questioning attitude was not evident during this transfer. The procedure reader was not -included in the-command -
path to ensure procedure requirements were addressed when feedwater system directions were issued and carried out. Multiple supervisors were giving guidance to the operators during the event: the shift supervisor, the unit supervisor, and procedure reader. However, none of these three verified-the actions the operators took to secure auxiliary feedwater or the annunciators it caused during the system alignment.
Teamwork Effective teamwork could have prevented this event by ensuring that all ;
actions taken were understood and agreed upon by all the control room crew.
Teamwork was not reinforced by crew briefings from supervision to keep all crew members apprised of conditions. As part of their training, the crew had attended a one day teamwork building session, but not all as members of this Crew.
L Trainino The secondary operator did not control the auxiliary feedwater system ficw by throttling the discharge valves in accordance with his training. The .
lack of clear guidance in the Reactor Trip Procedure regarding appropriate v i i
1
, - - + -- -y
.- 1
.l l
l l
.1 methods to control auxiliary and main feedwater and cool downs were not l identified and resolved in the training / procedure writing process.
l Simulator sessions offered many opportunities to identify and correct !
weaknesses in use of standardized terminology, repeat backs, and supervisory verification of operator actions that contributed to this event. Based on j interviews of members of different crews who were involved in the event, these l problems do not appear to be isolated to the event crew. l l
l The good performance of North Anna has caused fewer trips, power changes, )
and equipment problems and has resulted in fewer opportunities for operator performance of various routine control manipulations. In their interviews, j the operators expressed a desire for more simulator training in-these areas, ;
as their confidence in their ability to perform some seemingly routine activities (such as, starting up or shutting down a feedwater. pump) has decreased. Each licensed operator received about 80 hours9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> of simulator training each year. A majority of this time was spent on mandatory major event training with the emergency operating procedures.
Shift Staffina The composition of the shift staff was a contributor because of new personnel. One reactor operator had been on the shift for only 2 months, and i one supervisor had just returned to the shift after absences of several months j for outage duties. The last simulator session had concentrated on startup -
activities and, as a result, this crew had not trained on a major event as a i group. Previously mentioned areas of communications and teamwork, as well' as l
stress, were affected.
Stress Crew members' unfamiliarity with each other may have led to increased stress during the recovery. A new crew member perceived that'less communication feedback was provided during the event, then he had experienced vi
)
i I
. _ . . . _ _ ~ _ .
on another crew. In addition, the operator broke a glass cover on a control board indicator as he was attempting to verify the reading on othat indicator. .
In turn, the crew member's stress level may have contributed to the feeling of urgency the secondary operator felt with regard to reducing auxiliary feedwater flow to stop the reactor coolant system cooldown. A large number of people had accumulated in the back of the control room following the reactor trip. It was not reported as a problem, but the operators were aware of their presence.
Human Machine Interface 1
The condition of the auxiliary feedwater system with pumps running and valves closed, as allowed by the procedures, requires operator action to reinitiate the engineered safety function of the system.
There were multiple indications available in the control room, had they been used,-to alert other operators to the fact that the auxiliary feedwater pumps had been disabled. There were two red annunciators and valve position indication associated with the turbine-driven auxiliary feedwater pump valves, two pull-to-lock handles on the motor-driven auxiliary feedwater pumps, and motor-driven auxiliary feedwater discharge valve position indications. Some plants with the pull-to-lock capability on engineered safety feature equipment have additional electrical section annunciators to provide the status of ;
control switches.
The shift technical advisor was unaware that the auxiliary feedwater system was inoperable during the event. No detailed system status indication of auxiliary feedwater pumps and valves were provided on the safety parameter I display system critical safety function pages he was monitoring. .
The physical location of recorders on the vertical section of the control board for important parameters may require an operator to leave the area of controls on the console section. The crew has incorporated the use of a spotter (additional person stationed at the recorders) if staff is available.
The additional person adds another communication link in the control process vii
)
i i
~
t i
and is subject to availability of personnel. In one previous instance, communication between the spotter and the control operator resulted in a flow adjustment on one steam generator in response to a level value being reported
-from another steam generator.
I s
h
?
viii .- - -
-+,
ACKNOWLEDGMENTS We appreciate the North Anna staff's cooperation in freely providing the necessary information and scheduling interviews to analyze the human factors '
of the operating event. We thank the Unit 2_ operators and technical staff _
who were on duty during the event for their cooperation during the interviews.
We thank the North Anna resident staff, Mark Lesser in particular, for their assistance during the site visit. Also, we wish to thank Dr. Susan Hill, who participated in the Human Factors review process.
f-iX
. 1 e
CONTENTS EXECUTIVE
SUMMARY
. . . . . . . . . . . . . . . . . . . . . . . . . . . . iii ACKNOWLEDGMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix-ACRONYMS ................................ xi
- l. INTRODUCTION ............................ I 1.1 Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . I 1.2 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I 1.3 Onsite Analysis Team ..................... 2
- 2. DESCRIPTION OF THE EVENT ANALYSIS . . . . . . . . . . . . . . . . . . 3 2.1 Background .......................... 3 .
2.2 Time Line of the Event .................... 7 2.3 Analysis ........................... 9 2.3.1 Operations Procedures . . . . . . . . . . . . ._. . . . 9 2.3.2 Command, Control, and Communications . . . . . . . . . . 11 2.3.3 Teamwork . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3.4 Training ........................ 12 2.3.5 Stress . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.3.6 Shift Staffing ..................... 15 2.3.7 Human Machine Interface . . . . . . . . . . . . . . . . 15 Figure 1. North Anna control room staffing (normal conditions). . . . . 17-Figure 2. North Anna control room organization (Unit 2 event) . . . . . 18 1
X s
i ACRONYMS AE00 Analysis and Evaluation of Operational Data AFW Auxiliary Feedwater AMSAC Accident Mitigation System Actuation Circuit ,
A0 Auxiliary Operator DSP Division of Safety Programs ESF Engineered Safety Function E0P Emergency Operating Procedure INEL Idaho National Engineering Laboratory MFW Main Feedwater NRR Nuclear Reactor Regulation NRC Nuclear Regulatory Commission RCS Reactor Coolant System RMS Radiation Monitoring System R0 Reactor Operator ROAB Reactor Operations Analysis Branch S/G Steam Generator SS Shift Supervisor STA Shift Technical Advisor TSC Technical Support Center VAR Volt Ampere Reactive 4
e xi
3
- 1. INTRODUCTION i
1.1 Purpose P
The Office for Analysis and Evaluation of Operational Data (AE00) of the U.S. Nuclear Regulatory Commission (NRC) has a program to study human performance during operating events. As part of this program, AEOD formed a team to conduct an onsite analysis of the event that occurred at the North Anna Power Station during the day shift on April 16, 1993. This report documents the human factors analysis performed as part of the study.. The Idaho National Engineering Laboratory (INEL) provided program assistance.
Unit 2 had been operating at 100% power, when a control problem developed '
in the main generator voltage regulator. Increased excitation on the main generator led to a field forcing (overexcited) condition initiating a control room annunciator. The unit operator attempted to manually lower the output of' the voltage regulator. Seventeen seconds after the field forcing annunciator, ,
a generator differential lockout protection occurred, causing a main generator, turbine, and reactor trip. Control room staff entered the emergency operating procedures for plant recovery. While the crew implemented Procedure 2-ES-0.1, Reactor Trip Recovery, the auxiliary feedwater (AFW) system was inoperable for approximately 18 minutes. The system misalignment was discovered by the procedure reader when he reached a step in the procedure for placing the AFW system in standby. Shift supervision immediately' directed the AFW system returned to service. The required heat sink conditions for the reactor were maintained while the AFW was inoperable.
1.2 Scope The human factors analysis focused on the factors that influenced the performance of operations and technical support staff throughout this event.
The analysis was based on data. derived from interviews with operations and technical staff, review of plant logs and recordings, and review of procedures and training material.
I
- l 1
1.3 Onsite Analysis Team i
The onsite analysis team visited the North Anna Power Station during the-period of April 20-22, 1993, and was composed of the following members:
John Kauffman, NRC/AE00/DSP/ROAB (team leader)
Robert Spence, NRC/AE0D/bSP/R0AB William Steinke, INEL/EG&G Idaho, Inc.
1 i
2
- 2. DESCRIPTION OF THE EVENT ANALYSIS 2.1 Backaround The North Anna Power Station, located in central Virginia, is owned and operated by Virginia Electric and Power Company. The two nearly identical-pressurized water reactors are rated at 2893 MW-thermal each and have Westinghouse nuclear steam supply systems and subatmospheric containment buildings. The units are operated from a common control room. Units 1 and 2 have been in operation since 1978 and 1980, respectively.
On April 16, 1993, Unit I had reached 75% power returning from a refueling outage and Unit 2 was operating at 100% power. Crew A was on day two of their 12-hour shift schedule (7:00 am to 7:00 pm) following 6 days off. .
Shift manning included three senior licensed operators and three-reactor operators. These personnel were assigned as shift supervisor (SS), Unit I supervisor, Unit 2 supervisor, Unit I reactor operator, Unit 2 reactor operator, and backboard operator (see Figure 1). A shift technical advisor (STA) was also assigned to the shift.
At 7:16 a.m., the SS was briefing crew A in the Technical Support Center adjacent to the control room when a field forcing annunciator was received on Unit 2. The Unit 2 reactor operator observed volt amperes reactive (VARS) indication, for the main electrical generator, was reading full scale in the VARS out direction as a result of a failed voltage regulator. While the Unit 2 operator was attempting to take manual control and lower excitation, a differential lockout was received, initiating a main generator trip and concurrent turbine trip, followed immediately by a reactor trip.
The Unit 2 supervisor.was on the phone at the time of the reactor _ trip with the system dispatcher' discussing a severe weather alert that had just been issued. He immediately terminated the call and located Procedure 2-E-0, Reactor Trip and Safety injection, while directing the operators to perform their immediate actions. The Unit I supervisor announced the reactor trip over the plant paging system and proceeded to the Unit 2 side of the control 3
-l l
room to assume procedure reader responsibilities (see Figure 2). The j backboard operator was on the Unit I side of the control room in anticipation of assisting with feedwater control because of problems experienced with flow oscillations. He immediately went to the Unit 2 side to assume secondary plant responsibilities following the reactor trip. No safety injection had occurred, and the crew exited Procedure 2-E-0 at Step 4 and transitioned to !
Procedure 2-ES-0.1, Reactor Trip Response. It was at this point the SS, other crew members, and some dayshift personnel entered the control room to assist.
I l
At the time Procedure 2-ES-0.1 was entered, the average temperature l (Tavg) of the reactor coolant system (RCS) was approaching the control (no !
load) setpoint of 547 F with maximum auxiliary feedwater (AFW) flow to the ;
steam generators (S/G). Two motor-driven and one turbine-driven AFW pumps were running with all associated motor-operated discharge valves fully open.
Levels of all three S/Gs had gone below the narrow range indication, as expected shortly after the reactor trip. Two condensate pumps and two main feedwater pumps were still running with recirculation flow through one line back to the condenser.
An unidentified person in the plant paged the control room on the plant paging system to inform them that an abnormal amount of steam was present in the turbine building. The SS dispatched an auxiliary operator (AO) to investigate. Several minutes later the A0 returned to the control room and reported that all of the feedwater heat exchanger relief valves were lifting.
At this time, the SS dispatched the dayshift operations maintenance coordinator present in the control room and two A0s to the turbine building to isolate the feedwater heaters. In addition, the SS directed the turbine building A0 to locally unisolate another main feed pump recirculation path in an effort to lower feedwater system pressure and to help reseat the lifting relief valves.
4
The primary operator encountered several abnormal indications during this time period:
. Following the reactor trip, one control rod bottom position indicator (analog) indicated the rod was n'ot inserted as required,
. An air ejector high radiation alarm had been received, and
. A reactor coolant pump vibration alarm had occurred.
The SS responded to the vibration alarm and reset it, informing the primary operator that the alarm had cleared. The primary operator.
mechanically agitated the analog rod position indicator and inadvertantly broke its glass cover. The indicator moved to the rod-in-position. The primary operator concluded the instrument had been stuck and the control rod was actually in the proper position.
As the crew progressed through Procedure 2-ES-0.1, the primary operator announced several times that RCS Tavg was decreasing without receiving sufficient communication to alleviate his over concern about the RCS cooldown rate. The primary operator increased the charging flow in response to the -
cooldown to maintain pressurizer level and closely monitored the pressurizer pressure, which was slow in recovering. To assist in monitoring primary system parameters, a trainee was used as a " spotter" to read recorders located on the vertical section of the control board and relay this information to the primary operator at the console section.
Hearing the primary operator voice his concerns several times about the cooldown of the RCS, the secondary operator informed the Unit 2 supervisor that S/G B narrow range level was >11% (a condition where AFW flow can be decreased below 400 gpm per Step 6 of Procedure 2-ES-0.1). The secondary operator then requested permission from the Unit 2 supervisor to " secure AFW" and go on main feedwater. The secondary operator also requested that the accident mitigation system actuation circuit (AMSAC) be reset. The Unit 2 supervisor then had the primary operator reset AMSAC. At this point, the procedure reader was at Step 6 and attempted to go back to Step 1 for instructions to throttle or control AFW flow for the purposes of limiting RCS cooldown, although Step 1 was not identified as a continuous action step. The 5 ,
1 i
i Unit 2 supervisor stopped the procedure reader at that point. Another SRO, ,
from the previous shift, expressed his opinion to the SS that the cooldown was not that severe (i.e., 2'F in 5 minutes). ~After the Unit 2 supervisor and the SS conferred briefly, the Unit 2 supervisor gave permission directly to the secondary operator to " secure AFW." The secondary operator, without direction from the procedure reader, opened two of the main feedwater bypass valves to establish flow to the S/Gs and then stopped the AFW pumps by placing the ,
motor-driven pumps in pull-to-lock position-(switch is locked in the off ,
position) and closing the two steam supply valves to the turbine-driven pump.
After authorizing the secondary operator to secure AFW, the Unit 2 supervisor went over to the nuclear instrument section of the control boards, leaving the secondary operator to carry out the direction to secure AFW. The primary operator had previously reported to the Unit 2 Supervisor that the N-31 source range channel had energized and the meter was not indicating any -
source level counts. Channel N-32 source range was responding as expected.
The Unit 2 supervisor directed the primary operator to enter Abnormal Procedure 2-AP-4.1, Malfunction of Source Range Instrumentation.
The STA arrived in the control room about one minute after the trip. He
- had been attending the crew briefing when the trip occurred. His first action was to obtain a copy of Procedure 2-F-0, Critical Safety function Status Trees. Next, he called up the status trees on the Safety Parameter Display System (SPDS). After completing his assessment of the critical safety function status trees, the STA reported to the SS and Unit 2 supervisor that ,
the only yellow path condition was heat sink (S/G levels <11% with feedwater flow >400 gpm) . A yellow path condition does not require immediate operator attention and can be addressed in parallel with other activities. All other safety functions were satisfied (green path).
i When the procedure reader reached Step 12 of 2-ES-0.1, " Stopping AFW pumps," he noted that the pumps were in pull-to-lock (locked in the off I
position). He informed the SS of this discrepancy, who immediately directed the secondary operator to place the AFW pumps back in an auto start configuration (steam supply valves open and motor-driven switches in auto).
6
. - . _ _ ~- . .
Hain feedwater flow had raised S/G levels above 20% by this time, and the AFW ,
pump auto start signal set at 18% had cleared. As a result, the A"8 pumps remained stopped after they were placed in automatic. The AFW system had been inoperable for about 18 minutes.
The crew continued plant recovery by completing Procedu"e 2-ES-0.1 and transitioning to Procedure 2-0P-3.2, Unit Shutdown frow Mode 3 to Mode 4. The SS informed plant management of the AFW system configurat.'< n and initiated post trip review actions in accordance with Procedure VPAP-1404, Reactor Control. At 10:55 a.m., after management reviewed the events during the recovery, the NRC was notified of the reactor trip and AFW pump lockout (4-hour notification) in accordance with 10 CFR 50.72(b)(2)(ii) and 10 CFR 50.72(b)(2)(iii)(A).
2.2 Time Line of the Event The following time line sequence of the event was developed from interviews with the on-duty shift personnel and technical staff, copies of the control room logs, and plant computer printouts.
TIME EVENTS 7:00 a.m. Secondary operator assumed feedwater control responsibilities on Unit 1 due to problems experienced on previous shifts with flow oscillations caused by system flow control problems.
7:16:28 a.m. Exciter Field Forcing annunciator alarmed. Unit 2 primary reactor operator responded and attempted to lower excitation.
7:16:45 a.m. Reactor trip due'to turbine trip (crew entered Procedure 2-E-0, Reactor Trip and Safety Injection, and performed the immediate actions).
7:16:51 a.m. All three AFW pumps auto started on S/G Lo lo level (18%).
7:17:19 a.m. AMSAC initiated (S/G levels <13% in two of three S/Gs).
S/G levels all shrank below narrow range indication.
7 I
- l 7:20 a.m. Crew completed required actions of Procedure 2-E-0 and transitioned to Procedure 2-ES-0.1, Reactor Trip Response. ]
7:21 a.m. (time approx.) Tavg was trending down to 547*F when j checked in Step 1 of Procedure 2-ES-0.1.
7:22 a.m. (time approx.) Unidentified person paged the control room j i
and reported an abnormal amount of steam in the turbine building. The SS dispatched personnel to investigate.
7:22-7:24 a.m. Primary operator announced several times that RCS temperature was decreasing.
7:25:14 a.m. Primary operator reset AMSAC upon request from secondary j operator. l 7:26 a.m. . Primary operator reported Tavg had decreased to 540*F.
. Secondary operator verified main feedwater bypass flow capability and requested authorization from the Unit 2 ;
1 supervisor to " secure AFW." (Both main feedwater. pumps j were running and bypass valve control was available.) !
. Procedure reader started to go back to Step 1 for j directions to adjust AFW flow. Unit 2 Supervisor stopped crew and conferred with SS. l 7:27 a.m. S/G B narrow range level was >11% with maximum AFW flow.
Secondary operator received permission from the Unit 2 supervisor to " secure AFW."
. Secondary operator opened two main feedwater bypass valves were opened to establish flow.
. Secondary operator placed both motor-driven AFW pumps in pull-to-lock and isolated steam to the turbine-driven pump. !
. Primary operator reported Source Range N-31 energized and not indicating properly. Unit 2 supervisor left I the secondary control panel area and went to the nuclear instrument section.
7:30:38 a.m. S/G B lo lo level reactor trip signal cleared (input to auto start of AFW pump at 18% level). !
7:32 a.m. Abnormal Procedure AP-4.1, Malfunction of Source Range ]
Instrumentation, was entered.
8 ;
7:40:45 a.m. S/G C lo lo level reactor trip signal cleared (input to auto start of AFW pump at 18% level).
7:43:55 a.m. S/G A lo lo level reactor trip signal cleared (input to auto start of AFW pump at 18% level).
7:45 a.m. Step 12 of Procedure 2-ES-0.1 directing the shutdown of AFW was reached. Procedure reader noted AFW pumps were in pull-to-lock and informed the SS, who directed the secondary operator to return all AFW pumps to auto. No pumps started due to S/G levels >20% a+ this time.
8:30 a.m. Transitioned to Procedure 2-0P-3.2, Unit Shutdown from Mode 3 to Mode 4.
9:30 a.m. Post trip review was initiated.
10:55 a.m. NRC was notified of the reactor trip and disabled AFW pumps during the trip recovery.
2.3 Analysis 2.3.1 Operations Procedures Controlling cooldown of the RCS following a reactor trip is essential to minimize the pressure / temperature transient and usually requires operator action to adjust auxiliary feedwater flow. The importance is illustrated by the fact that Procedure 2-ES-0.1 addressed it first in the sequence (i.e.,
Step 1) as follows:
i ACTION / EXPECTED RESPONSE RESPONSE NOT OBTAINED l
- 1. CHECK RCS TEMPERATURES: lE temperatures are less than 547'F i
. Check RCS average temperature if AND decreasing, THEN stop dumping any RCP is running - STABLE AT OR steam.
TRENDING TO 547'F. IE cooldown continues, THEN:
OR ;
. Check RCS cold leg temperatures a) Adjust total AFW flow to if no RCPs are running - STABLE 400 gpm (340 gpm with RCPs off)
AT OR TRENDING TO 547 F. until at least one S/G narrow ;
range level is >11%. ;
1 9 i i
If the response to the Step 1 left column was yes, the procedure reader moved to Step 2 and RCS temperature was not addressed again until Step 13 where the same control parameters are restated prior to exiting the procedure. Actions to throttle or adjust AFW flow by the operator in Step 1 right column were completed only if conditions required a negative response to the question posed in the left column. Without a reduction of AFW flow, a cooldown of the RCS would result under most conditions and could lead to an inadvertent low-pressure safety injection. As the procedure was written, the operator was left to his own judgement on how to mitigate a cooldown if it occurred after Step 1 in the procedure, as it did in this event.
It could be argued that the required manipulation of the AFW system in this event was " skill-of-the-craft" and does not need to be specified in a written procedure. Procedure OPAP-0002, Operations Department Procedures, , specifically identifies adjustment of main feedwater flow as
" skill-of-the-craft," but does not include AFW flow which leads to the conclusion that AFW flow adjustment must always be under the administrative control of a procedure.
l Step 2 of Procedure ES-0.1, directed the operator to check feedwater status and to verify adequate feedwater (auxiliary or main) flow. If adequate flow was not available, the operator was directed to establish AFW or MFW. No procedural direction was given for shutting down the AFW system and placing i main feedwater in service. When the decision was made to " secure AFW" and use main feedwater, the task was accomplished without procedure guidance. The use of " skill-of-the-craft" for main feedwater adjustments by the operator led directly to the misalignment of the AFW system without direction from the procedure reader.
In this event, the operator disabled the AFW pumps from an auto start signal on S/G lo lo level by using pull-to-lock (locked in the off position).
As written in the E0Ps, AFW flow can be controlled (i.e., throttled to any value) if one S/G level is >11% by closing the motor-operated valves. Both system alignments would have required operator action to reestablish the required AFW flow to the S/Gs. The potential for bypassing a safety system 10
~.- _ .
9 .
J
-due to closed discharge valves was identified as an inspector follow up item (IFI 50-339/93-17-03) in NRC Inspection Report Nos. 50-339/93-17 and 50-338/93-17.
2.3.2 Command. Control, and Communications Command and control was not effective.in maintaining management oversight of system configuration during the plant recovery. Several contributing.
factors led to this situation: procedure reader not included in command path to ensure procedure requirements were addressed, misunderstood terminology, ,
multiple supervisors giving guidance to the operators during the event, and lack of closed-loop communications.
The control room crew had two recent additions. The Unit 2 operator had joined the crew just 2 months prior to the event and the Unit I supervisor had just returned from outage-related duties. The new group had trained together on startup activities before this event. Looking at Figure 2, the chain of j command appears relatively clear. However, complications developed during- ,
the reactor trip recovery when the SS became involved in directing actions on the unit in the turbine building and the unit supervisor communicated directions directly to the primary and secondary operators. The position of procedure reader was bypassed during these instances, specifically when directions were given for AFW and main feedwater, providing no backup or check ,
against written procedure requirements.
A common understanding of the word " secure" was not evident in the communications among the secondary operator, Unit 2 supervisor, and SS ;
concerning the AFW system. Company procedure VPAP-0213 Abbreviations, Acronyms, and Action Verbs, provides the following definition for secure: "To- t remove systems or lineups from service and take appropriate action to prevent return." The actions taken by the secondary operator were covered-within this definition. The secondary operator communicated his intentions via the term
" secure." Interviews with the Unit 2 supervisor and SS revealed that they interpreted this as the usual request to close the AFW valves to reduce flow 11
and did not question the terminology used by an experienced operator.
Consequently the authorization was given to " secure AFW," which did not clearly identify the individual actions expected.
Multiple supervisors were giving guidance to the operators during the event: the shift supervisor, the unit supervisor, and procedure reader.
However, none of these three verified the actions the operators took to secure auxiliary feedwater or the annunciators it caused during the system alignment. ,
Verbatim repeat back of instructions was not always required. Even closed-loop communicating, if it had been used, by informing supervision that "AFW is secured" would not have conveyed enough information to identify a system misalignment. Communications from the primary operator to supervisory personnel were sometimes addressed to no specific individual and the primary operator received insufficient acknowledgement to allay his concerns.
2.3.3 Teamwork i Effective teamwork could have prevented this event by ensuring that all actions taken were understood and agreed upon by all members of the control room crew. The crew had attended a teamwork building session, but not all as members of this crew. The responsibilities of unit supervisor and procedure )
reader were divided between two senior licensed individuals. This is different than the initial training provided to the supervisors for their senior license where one person has both responsibilities. It would appear this was an enhancement to limit the task workload on the supervisor and add another member to the team for backup. However, the procedure reader was not involved in the feedwater decision and did not provide successful backup. The ,
error in system alignment was not detected until the procedure directed ;
attention to it in Step 12.
i 2.3.4 Trainina The topic of overriding or disabling ESF equipment or signals has been specifically addressed by plant management within the last year and is an
]
12 l
L integral part of the licensed operator training program. Even with this level of effort, ESF equipment was turned off. The following training' issues were ;
considered as contributors to the event: AFW system (ESF status), procedure usage, system performance, and teamwork, including communications which has been discussed previously.
Given the circumstance of an uncomplicated reactor trip (i.e., no major event such as a pipe break), the secondary operator's perception at that time was the AFW system was a non-ESF system unless a safety injection had actuated. Once heat-sink requirements were met, he took what he thought to be -
appropriate action for the conditions. It appears the design basis of the AFW system with S/G levels between 11% and 18% became unclear to the operator at that time. Procedurally, the AFW system had accomplished-its ESF purpose by reestablishing a heat sink, but was still required by technical specifications to be in an operable condition. The AFW pumps could not be shutdown and placed in automatic because of existing start signals from low-low S/G' levels.
The operators previous training with these plant conditions did not prevent his rendering the AFW system inoperable (i.e., not capable of automatically producing flow), when permission was received to use main feedwater.
The team concluded from the interviews that previous training had not established consistent procedure usage for stopping a cooldown under the event conditions. When several operators were presented with Procedure ES-0.1 and the plant conditions that existed at the beginning of the event, no consistent answer could be given on how to use the procedure to stop the cooldown. -This created the opportunity for the secondary operator to take action based on his Concerns.
The secondary operator did not choose to stop the cooldown by closing the AFW motor-operated discharge valves for two reasons, as expressed in the interview. In the past, operating the AFW pumps on recirculation flow through a 1-inch line had led to degradation of the pumps. Following the procedure and leavirg the pumps running with the valves closed created that . condition. ,
A 6-inch line had been installed for surveillance testing, which was isolated by valves during normal operation. Engineering had analyzed this problem and 13
found that using the 1-inch line did not create a problem when used for short ,
~
periods of time. This conclusion was apparently not conveyed to all operators. The second system characteristic that influenced his decision was stroke time of the valves' versus the time to start a pump for reestablishing flow. The valves require about 25 seconds to stroke, and the pump requires about 10 to 15 seconds to produce flow. A misconception by the secondary operator about valve performance (i.e., full stroke needed to produce required flow) resulted in the opinion that a pump start could produce flow sooner and more reliably if needed. A recent failure of a motor-operated-valve on the AFW system following an outage may have influenced the operator's decision to secure the pumps rather than closing the motor-operated-valves. He thought the pumps were more likely to start than the valves were to reopen if needed to restore auxiliary feedwater flow. Skill-of-the-craft action based on these concerns to secure the AFW pumps was thought to be appropriate by the secondary operator.
The good performance of North Anna has caused fewer trips, power changes, and equipment problems and has resulted in . fewer opportunities for operator performance of various reutine control manipulations. In their interviews, the operators expressed a desire for more simulator training in these areas, as their confidence in their ability-to perform some seemingly routine I activities (such as, starting up or shutting down a feedwater pump) has decreased.
Each licensed operator received about 80 hours9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> of simulator training each year. A majority of this time was spent on mandatory major event training with the emergency operating procedures. Controlling AFW to maintain normal operating temperature is often overridden by the need to cooldown when a major event occurs in training scenarios.
)
i 2.3.5 Stress Unfamiliarity of the crew members with each other may have led to increased stress during the recovery. A new crew member perceived ;
-1 14
communications to be different, offering less feedback than he previously experienced on another crew, which increased his stress level. In turn, his apparent stress level may have contributed to the feeling of urgency the secondary operator felt with regard to reducing AFW flow to stop the RCS cooldown.
r 2.3.6 Shift Staffina t
Shift composition was a contributor to this event with the addition of new personnel. The primary operator had joined the crew two months before the event and Unit 2 supervisor had returned a week earlier at the completion of the Unit 1 outage. The crew had trained together on startup activities on the simulator just prior to the event. More complicated major event training with emergency operating procedures had not been covered in that session.
Previously mentioned areas of communications, teamwork, and stress were ;
affected as a result of staffing.
The STA provided the intended backup functions by performing status checks of critical safety functions. However, the procedure he was using only addressed feedwater flow requirements for the S/Gs. The inputs to the Safety -
Parameter Display System (SPDS) for heat sink did not include AFW pump status.
2.3.7 Human Machine Interface When the secondary operator disabled the AFW pumps, the indications available to alert the other crew members to this condition were:
. two annunciators and valve position indication associated with the turbine-driven auxiliary feedwater pump valves,
. two pull-to-lock handles on the motor-driven auxiliary feedwater ,
pumps, and
. motor-driven auxiliary feedwater discharge valve position indications. ,
These annunciator windows illuminated in red, are located among the other secondary alarms that were already lit from existing plant conditions. Some plants with the pull-to-lock capability on ESF equipment have additional 15
- electrical section annunciators to provide status of ESF control switches.
This indication is not available at North Anna.
The .TPD5 :i~cical safety function status tree display for heat sink does *
~
not include the status of the AFW pumps and valves. In'this event, it would have provided the STA with a diagnostic tool capable of verifying required system configuration.
1 The physical location of recorders on the vertical section of the control board for trending important parameters (e.g., pressurizer- pressure or level) may require leaving the area of the controls on the console section. The crew-has incorporated the use of a spotter (additional person stationed at the recorders) if staff is available. In one previous instance, communication between the spotter and the control operator resulted in a flow adjustment on one S/G in response to a level value being reported from another S/G. The-additional person adds another communication link in the control process and is subject to availability of personnel.
5 b
16 -
I l
- 1 1
Shift Supervisor j SR0 - 10 yr. l Unit 1 Supervisor Shift Technical Unit 2 Supervisor :
Advisor :
SR0 - 3 yr. Non-Licensed SR0 - 1 yr.
" nit i Operator Unit 2 Operator R0 - 3 yr. RO - 2.5 yr.
7 Backboard Operator R0 - 4 yr.
Note : The SR0 and R0 license periods are for North-Anna and do not include any previous licensing or operating experience. ,
Figure 1. North Anna control room staffing (normal conditions).
[
i 17 :
P 1
y ,y- , -
Shift Supervisor i
Shift Technical Unit 2 Supervisor Advisor Unit 1 Supervisor (Procedure Reader)
Unit 1 Operator Unit 2.0perator (Primary)
Backboard Operator (Secondary) :
tiote : Each crew has an additional qualified reactor operator assigned as a procedure writer and is available to the control room if needed. 1 Figure 2. liorth Anna control room organization (Unit 2 event).
18 l
1 s