ML20070J582
| ML20070J582 | |
| Person / Time | |
|---|---|
| Issue date: | 07/31/1994 |
| From: | Lewis P, Mumaw R, Roth E NRC OFFICE OF NUCLEAR REGULATORY RESEARCH (RES), WESTINGHOUSE ELECTRIC COMPANY, DIV OF CBS CORP. |
| To: | NRC OFFICE OF NUCLEAR REGULATORY RESEARCH (RES) |
| References | |
| CON-FIN-L-1505 NUREG-CR-6208, NUDOCS 9407250208 | |
| Download: ML20070J582 (143) | |
Text
NUREG/CR-6208 An Empirical Investigation of Cgerator Performance in t,ognr:1ve y Demanc.mg Simu a~:ec Emergencies i
i
.....,w I
Prepared by E. M. Roth R. J. Mumaw, L' STC P. M. Lewis, USNRC l
i a
i Westinghouse Science and Technology Center e
Prepared for U.S. Nuclear Regulatory Commission f
CR-6208 R PDR
AVAILABILITY NOTICE Availabihty of Reterence Matenals Cited in NRC Pubhcations Most documents cited in NRC publications will be avaliable from one of the following sources:
1 The NRC Public Document Room. 2120 L Street, NW., Lower Level, Washington, DC 20555-0001
[
2.
The Superintendent of Documents. U.S. Government Printing Office, Mail Stop SSOP, Washington, DC 20402-9328 3
The National Technical information Service. Springfield. VA 22161 Althougrt the listing that follows represents the majority of documents cited in NRC publications, it is not in.
tended to be exhaustive.
Referenced documents avai'able for inspection and copying for a fee from the NRC Public Document Room include NRC correspondence and internat NRC memoranda NRC bulletins, circulars, information notices in-I l
spection and Investigation notices; licensee event reports; vendor reports and correspondence: Commission papers; and applicant and licensee documents and correspondence.
The following documents in the NUREG series are available for purchase from the GPO Sales Program: formal NRC staff and contractor reports. NRC-sponsored conference proceedings, international agreement reports, grant publications. and NRC booklets and brochures. Also available are regulatory guides NRC regulations 6n tne Code of reacrat Regatahons. and Nuclear Regulatory Commission issuances, Documents available from the National Technicalinformation Service include NUREG-series reports and tech-nical reports prepared by other Federal agenes and reports prepared by the Atomic Energy Commission.
forerunner agency to the Nuclear Regulatory Commission.
Documents avahable from pubhc and special technicallibraries include all open hterature items, such as books,
journal articles. and transactions. Federal Register notices, Federal and State legislation, and congressional reports can usually be htained from these libraries.
Documents such as theses, dissertations, foreign reports and translations, and non-NRC conference pro-ceedings are avanab'e for purchase from the organl2ation sponsoring the publication cited.
Single copies of NRC draf t reports are available free, to the extent of supply, upon written request to the Office of Administration. Distribution and Mail Services Section. U.S. Nuclear Regulatory Commission, Washington.
DC 20555-0001,
)
Copies of industry codes and standards used in a substantive manner in the NRC regulatory process are main-tained at the NRC Library = 7920 Norfolk Avenue. Bethesda, Maryland, for use by the public Codes and stan-dards are usuany copyrighted and may be purchased from the originating organization or, if they are American l
National Standards, from the American National Standards institute,1430 Broadway. New York, NY 10018.
l l
t l
l DISCLAIMER NOTICE i
l This report was prepared as an account of work sponsored by an agency of the United States Government.
Nerther the United States Government nor any agency thereof, or any of their employees, rnakes any warranty, expressed or implied, or assumes any legal liability of responsibility for any third party's use, or the results of l
Such use, of any information, apparatus, product or process disclosed in this report, or represents that its use by such third party would not infringe privately owned rights.
NUREG/CR-6208 An Empirical Investigation of Operator Performance in Cognitively Demanding Simulated Emergencies Manuscript Completed: April 1994 Date Published: July 1994 Prepared by E. M. Roth, R. J. Mumaw, E STC P. M. Lewis, USNRC Westinghouse Science and Technology Center 1310 Beulah Road Pittsburgh, PA 15235 Prepared for Division of Systems Research Office of Nuclear Regulatory Research U.S. Nuclear Regulatory Commission Washington, DC 20555-0001 NRC FIN L1505
1 l
l.
Abstract This report documents the results of an empirical study of nuclear power plant operator performance in cognitively demanding simulated emergencies. During emergencies operators follow highly prescriptive written procedures. The objectives of the study were to understand and document what role higher-level cognitive activities such as diagnosis, or more generally ' situation assessment / play in guiding operator performance, given l
that operators utilize procedures in responding to the events. The study examined crew performance in variants of two simulated emergencies: (1) an Interfacing System Loss of Coolant Accident and (2) a Loss of Heat Sink scenario. Data on operator performance were collected using training simulators at two plant sites. Up to 11 crews from each plant participated in each of two simulated emergencies for a total of 38 cases analyzed. Crew performance was videotaped and partial transcripts were produced and analyzed. The results revealed a number of instances where higher-level cognitive activities such as situation assessment and response planning enabled operators to handle aspects of the situation that were not fully addressed by the procedures This report documents these cases and discusses their implications for the development and evaluation of training and control room aids, as well as for human reliability analyses.
1 iii NUREG/CR-6208
1 3
Contents Abstract..
iii Figures..
. vii Tables.
..viii Executive Summary.
ix Acknowledgments.
..xiii Acronyms..
. xv 1 Introduction..
1 1.1 Overview of Study..
2 1.2 Background.
3 1.3 Examining Crew Interaction Skills in Cognitively Demanding Scenarios.,
5 1.4 Structure of the Report.
5 2 Study Methodology.
7 2.1 Approach..
7 2.2 A Model cf Cognitive Activities involved in Operator Performance in Emergencies...
7 2.2.1 Situation Assessment.
8 2.2.2 Response Planning..
10 2.3 Overview of Emergency Scenarios..
. 10 2.3.1 ISLOCA Scenarios...
11 2.3.2 Loss of Heat Sink Scenarios.
. 13 2.4 Data Collection and Analysis..
15 2.4.1 Participants.
15 2.4.2 Procedure..
16 2.4.3 Data Analysis..
17 2.4.3.1 Information Recorded in the Protocol..
17 2.4.3.2 Behaviorally Anchored Rating Scales (BARS) of Team Interaction Skills.
19 3 Cognitive Performance in the Simulated Emergencies
. 21 3.1 ISLOCA 1: ISLOCA into RHR Leading to Pipe Rupture in Auxiliary Building.
21 3.1.1 Summary of Simulated Scenario ;
. 21 3.1.2 Characteristics of Participating Crews 21 3.1.3 A Case Where a Step in the EOP Explicitly Requests the Crews to Identify and Isolate a Leak.
. 22 3.1.4 A Case Where Operators Needed to Determine Whether Plant Behavior was the Result of Known Manual Actions or a Plant Fault -
28 31.5 Illustrative Protocol of Crew Performance in ISLOCA 1 ;
30 3.1.6 Variability in Crew Performance..
34 3.2 ISLOCA 2: ISLOCA In' a RHR Leading to a Break in the RHR Heat Exchanger to the CCW.
35 3.2.1 Characteristiu of Participating Crews.
. 35 3.2.2 A Case Where the Procedure Containing Relevant Guidance Could not be Reached Within the EOP Transioon Network.
35 3.2.3 A Case Where Operators Needed to Determine Whether Plant Behavior was the I
Result of Known Manual Actions or a Plant Fault -
. 41 3.2.4 Cases Where Operators Evaluated and Redirected the Procedure Path..
. 41 3.2.5 Variability in Crew Performance..
. 45 3.3 Loss of Heat Sink 1: Total Loss of Secondary Heat Sink (Feedwater Never Recovered)-
46 3.3.1 Characteristics of Participating Crews.
46 3.3.2 A Case Where Operators Needed to Determine Whether Plant Behavior was the Result of Known Manual and/or Automatic Actions or the Result of a Plant Fault..
46 v
Contents 3.3.3 A Case Where Operators Had to Decide Whether to Manually Initiate a Safety System Based on Consideration and Balancing of Multiple Goals.
51 3.3.4 Cases Where Evaluating the Procedure Path Enabled Operators to Catch Their Own Errors..
54 3.3.5 Variability in Crew Perrormance=
55 3.4 Loss of Heat Sink 2: Total Loss of Secondary Heat Sink (Feedwater Recovered).
56 3.4.1 Characteristics of Participating Crews.
57 3.4.2 A Case Where Operators Needed to Determine Whether Plant Behavior was the Result of Known Manual and/or Automatic Actions or the Result of a Plant Fault --
57 3.4.3 A Case Where Operators Were Required to Evaluate the Appropriateness of Procedure Steps Given the Specifics of the Situation -
60 3.4.4 Variability in Crew Performance..
65 4 Crew Interaction in the Simulated Emergencies.
.. 67 4.1 Cognitively Demanding Situations Where Good Crew Interaction was Important..
. 67 4.1.1 Cases Where Crews Needed to Pursue Multiple Objectives..
67 4.1.2 Cases Where Situation Assessment Required Integration ofInformation Across Multiple Crew Members..
68 4.1.3 Cases Where Crews had to Evaluate Whether to Take Actions Outside the Procedures.
. 69 4.1.4 Summary..
69 4.2 BARS Ratings of Crew Interaction Skills.
70 4.2.1 Variability among Crews on BARS Dimensions..
70 4.2.2 Evidence of a Link between Crew Interaction Skills and Technical Performance..
71 4.3 General Discussion of Team Interaction Skills Results.
72 5 Discussion of Results and Their Implications..
. 75 5.1 General Discussion.
. 75 5.2 Summary of Results..
. 76 5.3 The Role of Situation Assessment and Response Planrung in Cognitively Demanding Emergencies.
78 5.3.1 Situation Assessment.
78 5.3.2 Response Planning..
78 5.4 Alternative Views of the Role of Procedures and Implications of Results 79 5.4.1 View 1: Procedures Should Provide Detailed Guidance for Every Contingency...
79 5.4.2 View 2: Procedures Are Not Intended to be Optimal..
80 5.4.3 View 3: Situation Assessment and Response Planning Enable Operators to l
Handle Unanticipated Situations...
. 80 l
5.5 Implications of Results.
80 5.5.1 Implications for Training..
80 i
5.5.2 Implications for Control Room Aids..
82 l
5.5.3 Implications for HRA..
83 5.6 Conclusions..
83 6 References..
. 85 7 Glossary.
89 APPENDICES.
91
)
Appendix A: Detailed Descriptions of Scenarios -
93 I
i ISLOCA 1..
93 ISLOCA 2.
96 Loss of Heat Sink 1 (LHS 1)..
. 99 Loss of Heat Sink 2 (LHS 2)..
=103 Appendix B: Instructions to Study Participants and Sample Summary Sheets..
- 107 Appendix C
- Behaviorally Anchored Rating Scales (BARS)..
x113 Appendix D: A Cognitive Demands Checklist
.119 NUREG/CR-6208 vi i
l
Figures Figure 2.1 'lhe cognitive activities encompassed under situation assessment and response planning.
8 Figure 2.2 Types of observable behaviors that result from situation assessment and response planning.
9 Figure 2.3 Simplified diagram of the systems relevant to the ISLOCA scenarios..
12 Figure 2.4 Simplified diagram of the systems relevant to the Loss of Heat Sink scenarios.
14 Figure 2.5 Illustration of the logic employed to infer situation assessment and response planning..
18 Figure 3.1 EOP transitions between LOCA procedure (E-1) and $1 Termination Procedure.
29 Figure 5.1 Operator knowledge required to support situation assessment and response planning.
81 Figure A.1 More detailed diagram of the Residual Heat Removal System..
94 Figure A.2 EOP transitions relevant to ISLOCA event at Plant 1.
95 Figure A.3 EOP transitions relevant to ISLOCA event at Plant 2..
97 Figure A.4 Structure of Off Normal Procedure (OEN) for CCW System Malfunction..
.100 Figure A.5 EOP transitions for Loss of Heat Sink event..
.105 l
I t
i f
vii NUREG/CR-6208
l Tables Table 1.1 A listing of crew behaviors that provided evidence of situation assessment and response planning.. 3 l
Table 3.1 ISLOCA 1. Hypothesized explanations for plant symptoms.
25 Table 3.2 ISLOCA 1. Consideration of RHR train isolation.
27 l
Table 3.3 ISLOCA 2. Crew recognition that event was not a simple LOCA..
. 37 I
Table 3.4 ISLOCA 2. Hypothesized explanation for RHR problem.
. 38 Table 3.5 ISLOCA 2. First hypothesis generated to explain the CCW problem.
. 39 Table 3.6 LHS I. Crew identification of a steam space leak.
. 48 Table 3.7 LHS 1. Decisions to close the PORV block valve.
. 51 Table 3.8 LHS I. Crews that considered initiating SIor going to a bleed and feed..
52 Table 3.9 LHS 1. Crew decision regarding manualinitiation of St.
53 Table 3.10 LHS 2. Whether crews closed the pressurizer PORV block valve..
58 Table 3.11 LHS 2. First point at which crews considered manual SL 60 Table 3.12 LHS 2. Crew response to Reactor Trip procedure steps.
. 62 Table 3.13 LHS 2. Foldout page SI criteria met and crew response.
62 Table 4.1 Mean ratings of crews on the BARS dimensions.
70 l
Table 4.2 Mean BARS ratings for crews that differed in technical performance...
72 l
l l
i l
l 1
\\
l 1
l i
viii NUREG/CR-6208
Executive Summary l
An empirical study was conducted to examine Overview of Methodology operator performance in cognitively demanding simulated emergencies. During emergencies operator The study examined crew performance in variants of crews are required to follow highly prescriptive two cognitively demanding simulated emergencies:
Emergency Operating Procedures (EOPs). The (1) an Interfacing System Loss of Coolant Accident objective of the study was to understand and (ISLOCA) and (2) a Loss of Heat Sink (LHS) scenario document what role higher-level cognitive activities complicated by a leaking pressurizer power operated
)
such as situation assessment and response planning relief valve (PORV).
play in guiding operator performance during ' omplex c
emergencies, given that operators utilize EOPs in Data on operator performance were collected using responding to the events.1 training simulators at two plant sites. Two utilities were asked if they would voluntarily participate in an One view is that in emergencies the operators' empirical study of operator perfonnance in primary role is to rotely follow the EOPs. According cognitively complex simulated emergencies. Both to this view all that is needed for successful agreed to run an ISLOCA and a Loss of Heat Sink performance is that operators be able to read and event as part of the regularly scheduled follow the individual steps in the EOPs.
requalification training exercises at one of their nuclear power plant sites. Up to 11 crews from each Another view is that higher-level cognitive activities plant, including both actual operator crews currently such as situation assessment and response planning on shift and staff crews, participated in each of two continue to play an important role, even when EOPs simulated emergencies for a total of 38 cases are employed. According to this view the role of analyzed.
situation assessment and response planning is to enable crews to identify and deal with situations that Crew performance was videotaped and partial are not fully addressed by the procedures.
transcripts of the crew performance were produced.
These transcripts were then analyzed to:
These altemative views have very different implications for the kinds of training, procedures, Identify situations where higher-level cognitive displays, and decision-aids that need to be provided activities enabled operators to deal with aspects to control room operators. They also have different of the situation that were not fully handled by the implications for the kinds of analyses that are procedure; required to assess human reliability.
Document behaviors the operators engaged in to The study we conducted was designed to shed light handle these situations.
on the role of higher-level cognitive activities in guiding operator performance in cognitively demanding emergencies.
Overview of Results The results of the study supported the view that crew 1Situation assessment is defined as constructing an explanation to situation assessment and response planning continue account for observed plant behavior. It is similar to ' diagnosis' but to play an important role, even when EOPs are broader in scope. Diagnosis typically refers to the process of employed. We found a number of situations where searching for the cause(s) for abnormal plant behavior. Situation assessment encompasses explanations that are generated to situation assessment and response P annin8 enabled l
account for plant behavior during all plant conditions (i.e., normal the crews to handle aspects of the situation that were as well as abnormal plant states). Response planning refers to deciding on a course of action given a particular situation assessment. These concepts are described more fully in Section j
2.2.
1 Executive Summary j
i not fully covered by the procedures. These included:
of the leaks. Without active situation assessment, and response planning, they would not have been able to An EO? step tl at explicitly requested that crews identify and isolate the leaks.
identify and >> late a leak on their own; At the same time most of the crews recognized the A case whne the procedure containing relevant importance of continuing to proceed through the guidance could not be reached within the EOP EOPs. They perceived getting to the Cooldown and transition network; Depressurization procedure as a high priority activity.
Balancing the dual requirements to pursue the leak Cases where operators needed to determine into the RHR with the need to proceed expeditiously whether plant behavior was the result of known through the EOPs provided one of the most manual and/or automatic actions (e.g., a challenging aspects of the ISLOCA scenarios.
controlled cooldown) or the result of a plant fault; The ISLOCA scenarios also provided evidence of A case where operators were required to evaluate crews actively engaging in reasoning about the the appropriateness of procedure steps given the procedure logic. Clear inctances were found or crews specifics of the situation; reasoning at two levels. The crews were engaging in situation assessment and goalidentification. At the Cases where operators had to evaluate the same time they were reasoning about the strategies procedure path and take action to redirect the underlying the EOPs, and the EOP transition procedure path; network logic in order to assess whether the procedure they were following would enable them to A case where operators had to decide whether to achieve plant goals in a timely manner.
manually initiate a safety system based on consideration and balancing of multiple goals We found instances where monitoring the related to safety.
appropriateness of the procedure path enabled crews to identify when they were in an unproductive loop, and to identify another procedure path that would In each of the simulated scenarios situations arose allow them to take necessary actions more where operators needed to engage in situation expeditiously.
assessment and response planning in order to handle aspects of the situation that were not fully covered by The Loss of Heat Sink scenarios provided further the EOPs.
evidence that complex multiple fault conditions can arise where operators need to actively engage in In one variant of the ISLOCA scenario (ISLOCA 1) the situation assessment and response planning. In the crews were required to identify and isolate a leak into Loss of Heat Sink scenarios the procedures provided the Residual Heat Removal System (RHR) without no guidance in identifying and responding to the explicit procedural guidance. In the second variant of leaking pressurizer PORV. The majority of crews the scenario (ISLOCA 2), while there was a procedure were successfully able to detect the symptoms on the transition available to an ISLOCA procedure, it could primary system and integrate them to identify the not always be reached. Even in the cases where the leak. This was a difficult cognitive task that required ISLOCA procedure was reached, the procedure did recognizing that the primary side behavior could not not cover all aspects of the situation, i.e., a leak from be entirely accounted for by the ongoing cooldown the RHR into the Component Cooling Water System caused by efforts to recover the heat sink.
In one variant of the Loss of Heat Sink scenario Most crews actively sought information to help (LHS 1), the crews were faced with a decision identify the sources of leaks into the RHR and CCW, regarding manual initiation of a safety system. The and identified and took actions in an attempt to only EOP guidance available to them was in a caution isolate the leaks. They actively utilized resources that indicated that they had discretion to tum on the beyond the EOPs, such as schematics and alarm safety system if conditions in the plant " degraded."
printouts, to support their identification and isolation The decision of whether to tum on the safety system NUREG/CR-6208 x
Executive Summary 5
required balancing multiple goals. Manualinitiation Crews differed in the extent to which they detected of the safety system would respond effectively to the plant symptoms, actively sought an explanation for degrading conditions in the primary system caused unexpected findings, and attempted to come up with by the leaking PORV, but could potentially delay a coherent explanation that accounted for all the recovery of heat sink. The crews had some difficulty observed symptoms. In each scenario there was at with this aspect of the scenario. Most of the crews did least one crew that had difficulty identifying the not recognize that they had the discretion to decide source of the problem and taking appropriate action whether to turn on the safety system. Further, few of to mitigate it (i.e., approximately 10% of crews run in the crews showed evidence of considering the the event). The fact that not all crews in the scenarios tradeoffs involved. The majority of crews chose to let formed the correct situation assessment suggests that conditions continue to degrade until a criterion was there is room for improvement.
reached for which more explicit procedural guidance was available.
The results also clarified the role of group interaction in situation assessment and response evaluation, and The second variant of the Loss of IIcat Sink scenario provided suggestive evidence of the conditions under (LHS 2) provided additional opportunity to examine which crew interaction skills may be expected to the role of situation assessment in guiding crew affect technical performance of crews.
performance. In this scenario a case arose where operators had to decide the appropriateness of specific procedure steps based on their en situation Overview of Conclusions assessment. In LHS 2 the crews recovered feedwater on the secondary side, thus restoring the heat sink.
The results of this study provide support for the As required by the EOPs they then retumed to the position that situation assessment and response procedure that had been in effect prior to the loss of planning continue to be important for successful heat sink, which was the reactor trip procedure. This operator performance, even when EOPs are procedure contained some steps that required them to employed.
undo actions they had just taken to recover feedwater.
If they followed those steps it would result in a loss of in our scenarios a number of cognitively demanding heat sink again. The EOP background document situations arose where situation assessment enabled explicitly recognized that this type of situation could operators to handle aspects of the situation that were arise and indicated that in those cases operator not covered by the procedures. While these cases j
judgment would be required in determining were drawn from variants of two specific emergency appropriate action.
scenarios, we believe that the types of situations we identified are generic classes that are likely to arise in l
Most of the crews correctly recognized that some of other emergency scenarios. It is reasonable to assume the steps in the reactor trip procedure were that similar situations may arise in actual events. In inappropriate to the situation and should not be such situations the ability of operators to form followed. This included steps that called for initiation accurate situation assessments and to generate of a safety system. The decision that initiation of the response plans to cover aspects of the situation that safety system was not needed was based on situation are not fully addressed by the procedures will be assessment. 'Ihe crews had to determine whether the important.
conditions in the primary system were due to cooldown or a plant fault. This was not a simple The results of the study, taken in combination with determination, as attested by the fact that, in the case evidence from actual incidents, and experiences in of two of the crews who faced that decision, there was related domains support the position that situation a leak present (leaking pressurizer PORV), but the assessment and response planning enable operators to crews nevertheless initially attributed the primary handle unanticipated situations that are not fully side symptoms to cooldown, and decided against addressed by procedures.
manualinitiation of the safety system.
The conclusion that situations may arise where crews While most of the crews performed well, variability need to engage in situation assessment and response in performance was observed in all the scenarios.
Executive Summary planning has implications for the development and support HRA is included in Appendix D.
evaluation of training and control room aids, as well as for human reliability analyses (HRA) With respect A Unal conclusion of the study regards the value of to training, the results suggest that explicit attention empirical studies of operator performance in may need to be paid to the development of these simulated emergencies for addressing human i
cognitive skills. The results also have potential performance issues of concern to the NRC. Well implications for the development and evaluation of designed empirical studies can provide specific, clear control room aids. In particular, they suggest conclusions for practical decision making. The potential value for displays and decision-aids that are present study illustrates how empirical studies of i
explicitly intended to support situation assessment operator performance in simulated emergencies can and response planning.
be used to investigate a human performance issue -
in this case the role of higher-level cognitive activity The results also have potential implications for HRA.
in operator response to cognitively demanding They suggest that human reliability assessments are emergencies. The study provided (1) evidence that likely to be more accurate if the dynamics of the situations can arise where higher-level cognitive event, and the factors that are likely to complicate activity on the part of operators is needed and (2) situation assessment and response planning, are objective data on how different operator crews explicitly considered in performing the analyses. A responded to these situations.
Cognitive Demands Checklist that can be used to NUREG/CR-6208 xii
~ _ - - -
f Acknowledgments We wish to gratefully acknowledge the enthusiastic operations and emergency operating procedures. We support provided by the management and staff at the also wish to thank Robert. J. Lutz and Debra Ohkawa two utilities that participated in the simulator study.
of the Engineering Technology Department at NTD j
Without their voluntary participation and support, for generously giving of their time and their patience this type of study would not have been possible. The in answering questions about plant behavior and utilities generously provided access to their training operating practice. We would also like to simulator and operating crews, 4.nd allowed us to run acknowledge Roger A. Mundy, of the Nuclear and and videotape specially tailored scenarios as part of Technology Department at NTD for processing the their requalification training.
plant parameter data tapes recorded from the high fidelity training simulators. We are also grateful to Everyone at the two utilities was exceedingly Donna L. Rutt, of the Human Sciences group at the supportive and worked hard to accommodate the Westinghouse Science & Technology Center (STC), for needs of the study. We particularly wish to her support in transcribing the videotapes of crew acknowledge the training simulator personnel who performance during the simulated emergencies and in modified the simulator models to accommodate the preparation of the final report. Finally, we are most requirements of our scenarios.
grateful to Doris A. Pollitt, of the Business Operations Department at STC, who served as technical editor, We wish to especially thank the training instructors at for her patience and excellent advice.
the two utilities who contributed substantively to the design of the simulator scenarios and were We also wish to acknowledge the role of Dr. Harry E.
exceedingly conscientious in ensuring that the Pople, Jr. of Seer Systems. Dr. Pople actively scenarios were run in a controlled manner, and that contributed to the design of the empirical studies and the videotapes, data tapes, and summary data sheets participated in the plant data collection visits. His were properly recorded. In a very real sense they insights regarding the factors that influence operator deserve credit as co-investigators in this study.
cognitive performance and ideas about cognitive modeling were extremely stimulating and influenced We also wish to acknowledge the contributions of a many of the ideas presented here, number of people at Westinghouse. We wish to thank Robert G. Orendi, of the Plant Operations and Finally, we wish to thank the operator and staff Evaluations department at the Westinghouse Nuclear crews that participated as subjects in the study for Technology Division (NTD), for his role in providing allowing us to observe and report on their expert consultation on Nuclear Power Plant performance.
l xiii NUREG/CR-6208
a
.s wm.=
a 4.wz.
-ea-awe-=
e,.-
w
-u m
e r
.es e....
m
,A.4m.3 m:4 - ta s.a..;.4 a. u a._
es
_.. a j ca s. -. a u-.
A 49s Aw, u k.,e.
4_+,#w m
,.,.h.
4.-A, e
.4.,m-A>4e 4
1 i
j i-
^
i 5
f
(
i i
l2 i
f i
e i
l s
A a
?
I r
T I
r f
8 i
r 7
i b
+
3 I
?
Y i
e i
r I
l I
a--
+,- -
Acronyms Al ArtificialIntelligence BARS Behaviorally Anchored Rating Scale BIT Boron Injection Tank BOP Balance of Plant Operator CT Cannot Tell i
CES Cognitive Environment Simulation CCP Centrifugal Charging Pump CCW Component Cooling Water CREATE Cognitive ReliaM"ty Assessment Technique EDO Emergency Duty Officer EOP Emergency Operating Procedures ERG Emergency Response Guidelines FRG Functional Restoration Guidelines HRA Human Reliability Analyses ISLOCA Interfacing System Loss of Coolant Accident LHS Loss of Heat Sink LOCA Loss of Coolant Accident N/A not applicable LUPD Level Trending Up with Pressure Trending Down NATD Nuclear and Advanced Technology Division NPP Nuclear Power Plant NRC U. S. Nuclear Regulatory Commission OFN Off Normal Procedure PORV Power Operated Relief Valve PRA Probabilistic Risk Assessment PRT Pressurizer Relief Tank PRZR Pressurizer PWR Pressurized Water Reactor RCS Reactor Coolant System RHR Residual Heat Removal RNO Response Not Obtained RO Reactor Operator RVLIS Reactor Vessel Level Indication System RWST Refueling Water Storage Tank SG Steam Generator SI Safety Injection SO Supervising Operator SRO Senior Reactor Operator SS Shift Supervisor STA Shift Technical Advisor STC Science & Technology Center xv NUREG/CR-6208
l l
l i
j 1 Introduction J
l Human performance is a significant contributor to EOPs provide predefined strategies for accomplishing nuclear power plant (NPP) safety (e.g., Trager,1985; these functions. When an emergency arises that Kauffman, Lanik, Trager, and Spence,1992). During causes the reactor to trip, the operators are required to emergency situations operator action can have a take out the EOPs and follow the procedures step by substantial impact on the ability to return the plant to step. The EOPs provide detailed guidance on what safe operation. Operators may take recovery actions plant parameters to check, how to interpret the that mitigate the emergency situation. Alternatively, symptoms observed, and what control actions to take, errore in performance can delay or hinder plant recovery.
Given that operators utilize highly prescriptive procedures in responding to emergencies, a question Examination of actual incidents both inside and arises regarding the nature and extent of cognitive outside the NPP industry indicates that incidents activity required of operators to adequately handle often involve complicating factors (e.g., failed sensors; emergencies.
multiple faults) that impose difficult cognitive demands on operators (Perrow,1984; Wagenaar and One view is that all that is needed of operators is that Groeneweg,1987; Reason,1990; Woods,Johannesen, they understand and follow the steps in the EOP.
Cook, and Sarter,1993). Complications include Under this view what is needed for successful sensor failures that make situation assessment performance is that operators be able to read and difficult, cases where available procedures do not understand the individual steps in the procedure, that I
map well to the specifics of the situation, and they be able to locate and read the plant parameter situations where balancing of multiple goals related to values specified in the procedure steps, and that they safety is required (e.g., NRC, NUREG-1154; NRC, be able to locate the controls and take the actions NUREG-1455; Kauffman et al.,1992).
indicated in the procedure steps.
As part of a U. S. Nuclear Regulatory Commission Another view is that higher-level cognitive activities (NRC) project to model the cognitive activities that such as situation assessment and response planning underlie NPP operator performance in emergencies, continue to be important for successful operator an empirical study was conducted to examine performance, even when EOPs are employed. Under operator performance in cognitively demanding this view the role of situation assessment and simulated emergencies. This report presents the response planning is to enable crews to identify and results of the empirical study.
deal with situations that are not fully addressed by the procedures.
During emergencies operator crews are required to follow highly prescriptive Emergency Operating These alternative views have very different Procedures (EOPs). The objective of the study was to implications for the kinds of training, procedures, understand and document what role higher-level displays and decision-aids that need to be provided to cognitive activities such as situation assessment and operators. They also have very different implications response planning play in guiding operator for the kinds of analyses that are required to assess performance during complex emergencies, given that human reliability.
operators utilize EOPs in responding to the events.
The study we conducted was designed to shed light In an emergency the role of the operator is to ensure on the role of higher-level cognitive activities in plant safety. The operator monitors automatic plant guiding operator performance in emergencies. The safeguard systems, initiates recovery actions to design and analysis of the study was guided by a minimize radiation release and equipment damage model of higher-level cognitive activities involved in and return the plant to a stable condition, and ensures operator performance in emergencies. The model that critical safety functions are maintained.
described how situation assessment and response 1
Introduction planning was expected to affect operator performance The analysis identified six kinds of situations where i
in emergencies. The model helped to specify operators needed to engage in situation assessment cognitively demanding scenarios where the influence and response planning in order to adequately deal of situation assessment and response planning on with the situation. These situations are listed below operator behavior would be most readily observed, with the simulated scenarios in which they arose and specified the kinds o-
-rator behaviors to look presented in parentheses.
for as evidence of these L.cr-level cognitive t
An EOP step that explicitly requested that crews activities.
identify and isolate a leak (ISLOCA 1);
1.1 Overview of Study A case where the procedure containing relevant guidance could not be reachea within the EOP i
The study examined crew performance in two transition network (ISLOCA 2);
cognitively demanding simulated emergencies: (1) an Interfacing System Loss of Coolant Accident Cases where operators needed to determine (ISLOCA) and (2) a Loss of Heat Sink (LHS) scenario whether plant behavior was the result of known complicated by a leaking pressurizer power operated manual and/or automatic actions (e.g., a relief valve (PORV).
controlled cooldown) or the result of a plant fault (all four simulated events);
Data on operator performance were collected using training simulators at two plant sites. Two utilities A case where operators were required to evaluate were asked if they would voluntarily participate in aa the appropriateness of procedure steps given the empirical study of operator performance in specifics of the situation (LHS 2);
cogni' vely complex simulated emergencies. Both J
agreed to run an ISLOCA and a Loss of Heat Sink Cases where operators had to evaluate the event as part of the regularly scheduled appropriateness of a procedure path and take requalification training exercises at one of their action to redirect the procedure path (ISLWA 2:
nur bar power plant sites. In this report the plants are LHS 1; LHS 2);
-eferred to as Plant 1 and Plant 2. A different variant of the events was run at each plant.
A case where operators had to decide whether to l
manually initiate a safety system based on l
Up to 11 crews from each plant, including both actual consideration and balancing of multiple goals operator crews currently on shift and staff crews, related to safety (LHS 1).
participated in each of two simulated emergencies for a total of 38 cases analyzed.
Examination of crew performance in these situations Crew performance was videotaped and partial revealed clear instances of crews:
transcripts of the crew performance were produced.
These transcripts were then analyzed to:
Actively engaging in situation assessment and goalidentification; Identify situations that arose where operators needed to engage in higher-level cognitive Actively monitoring the appropriateness of steps activities in order to deal with the situation; in the EOPs for achieving progress toward these identified goals; and, Document behaviors the operators engaged in to handle those situations that were not explicitly Adapting the procedures to the situation.
directed by a specific EOP step (henceforth referred to as extra-procedural activities).
l Table 1.1 presents the types of extra-procedural l
The extra-procedural activities provided evidence of behaviors that were observed in the study and the situation assessment and response planning.
scenarios in which the clearest examples of each type 1
l m
a r
P l
Table 1.1 A lisimg of crew behaviors that provided evidence of situation assessment and response planning, and the scenarios in which clear examples of those behaviors were observed.
.j Crew Behaviors Observed ISLOCA 1 ISLOCA 2 LHS1 LHS2 I
Situation Assessment:
Checking for evidence to confirrr. hypothesis V
Explaining observed symptoms V
V V
Ider tifying unexpected plant behaviors V
V Detecting abnormal plant behaviors 4
V V
Ideritifying problems (e.g., plant malfunctions)
V V
V V
Detecting alarms / symptoms that had been missed V
Response Plann;ng:
Identifying goals V
Evaluating appropriateness of EOP procedure path V
V V
Evaluating consequences of actions V
Adapting procedures to the situation V
V 4
Catching errors V
V V
of behavior were found. These specific cases strategies embodied in the procedures in dealing with demonstrated the role that situation assessment and the specifics of the situation. These conclusions have response planning played in enabling the crews to potentialimplications for advanced man-machine handle these cognitively demanding scenarios.
interfaces, computerized procedures, operator While the evidence for the role of higher-level cognitive activities in directing operator performance is drawn from particular situations that arose in the two specific emergency scenarios that we ran, the
1.2 Background
types of situations we identified represent generic classes that are likely to arise in other emergency The empirical study reported here was part of a larger scenarios that involve different EOP procedures. The Project that was initiated by the U. S. NRC to study conclusions regarding the role that higher-level and model the cognitive activities that underlic cognitive activities play in guiding operator Operator performance during NPP emergencies. The performance in emergencies can reasonably be Project included two inter-related activities: (1) generalized beyond the particular scenarios and analyses of human crew performance during crews that we observed.
simulated emergencies and (2) development of an artificial intelligence (AI) computer simulation, called A main conclusion of the study is that, while EOPs Cognitive Environment Simulation (CES), that have greatly reduced the need for operators to simulates some of the cognitive activities involved in develop diagnostic and response strategies on their responding to a NPP emergency situation own in real time, they have not climinated the need (NUREG/CR-4862; NUREG/CR-5213; Roth, Woods, for operators to form their own situation assessment and Pople,1992).
and evaluate the effectiveness of the response 3
_ _ _ - - ~
Introduction As part of the CES development process, small-scale result, it was decided to perform a more extensive
. studies of human performance in simulated empirical study of operator performance in
)
emergencies were conducted using crews composed cognitively complex emergencies in order to confirm of training instmetors. Results of these small-scale and expand the results obtained in the ISLOCA studies suggested that higher-level cognitive activities exercises. 2 play a substantial role in guiding operator performance in complex emergencies, even when The empirical study reported here was designed to EOPs are used. A primary motivation for conducting address some of the limitctions of the earlier small-l the larger-scale study of operator performance in scale ISLOCA study. 'Ihe current study includes a simulated emergencies described in this report was to larger set of scenarios. The two ISLOCA variants that establish the generality of these preliminary results were used in the CES exercises are replicated. In and provide a clearer characterization of the role that addition, two new scenarios - two variants of a less cognitive activities play in guiding operator response of Heat Sink event - are used. The study also uses a in cognitively demanding emergencies.
larger, and more representative, set of operator crews as participants. Up to eleven crews were run in each in one case CES was exercised on two variants of an event. These crews included all operator crews ISLOCA into the residual heat removal system (Roth, currently on shift and all staff crews undergoing Woods and Pople,1992). Protocols recording the requalification training at each of two plants. The responses of CES to the ISLOCAs were produced and crews were composed of four to five operators, compared to the performance of human crews on the including a shift technical advisor and a shift same events. Both parallels and differences in the supervisor, which is more representative of the crew performance of CES and the human crews were size and structure that would normally participate in found. CES and human crews both attended to the an emergency event.
same evidence, accessed the same types of NPP knowledge, and followed the same line of reasoning The fact that the current study includes a larger set of in identifying the ISLOCA. However, CES was able emergency scenarios, and a larger, more to detect disturbances sooner, and follow implications representative, sample of operator crews, makes it of disturbances more thoroughly than the human possible to generalize to other crews and types of crews did.
emergency scenarios with more confidence. Because the study participants included all the operator crews One of the striking differences in the performance of currently on shift as well as staff crews undergoing CES and the human crews was that human crew retraining at each of two plants, the range of crew activity was strongly govemed by the EOPs whereas performance observed on the events should be the diagnostic performance of CES was not restricted reasonably representative of the range of performance in the same way. While human crews displayed of operator crews as a whole.
active situation assessment and response planning, these activities occurred primarily during periods where the demands of following the procedures were low.
The results of the ISLOCA study suggested that higher-level cognitive activities were needed to handle complex scenarios such as the ISLOCA events 2 A second activity in:tiated under the NRC project was an
- even when EOPs are used. However, the extent to attempt to enhance the capabilities of CES to model operators' use t
which the results could be generalized was not clear f EOPs in handling emergencies. This activitv depended on because of severallimitations of the data on human upgrades to the EAGOL software (on top of which the CES model was t uilt). While some progress was made in incorporating performance. In particular: (1) only one type of knowledge of procedures in EAGOL, complete CES runs event, an ISLOCA, was used; (2) the crews were demonstrating enhanced procedure-following capabilities could composed of training instructors rather than actual not be produced within the time frame of this project. Upon the operators; (3) only one crew was run on cach event; rec mmend tion of NRC senior managers. emphasis of the project (4) the crews were made up of two individuals rather
[hifted frem further development of CES to the empmcal than the standard three-to five-person crew. As a NUREG/CR-6208 4
Introduction 1.3 Examining Crew Interaction 1.4 Structure of the Report Skills in Cognitively Demanding Scenarlos This report documents the empirical study that was conducted and the conclusions drawn. The report is In addition to examining the role of higher-level cognitive activity in guiding operator performance, Section 1 provides an overview of the study and we also examined the role of crew interaction in describes the generalbackground and motivation for handling the cognitively demanding scenarios.
conducting the study.
Under a separate program sponsored by the U. S.
NRC, Montgomery et al. (1992) identified six Section 2 describes the methodology that was used in dimensions of team interaction skill, and developed conducting and analyzing the study. Included is a Behaviorally Anchored Rating Scales (BARS) for discussion of a simplified model of higher-level measuring crew performance on those dimensions. In cognitive activities involved in operator performance this study we examined crew performance in the in emergencies that provided the conceptual scenarios to identify cognitively demanding framework for the analysis of crew performance m situations that arose where good crew interaction the simulated events.
skills appeared to be important for successful performance from a technical perspective (i.e., for Section 3 presents the main results on operator correctly identifying plant malfunctions and taking cognitive performance in the study. The results are appropriate action). We identified particular crew organized around the six types of situations that were behaviors that characterized good performance on identified where higher-level cognitive activity was BARS dimensions, and appeared to be important for needed to handle the situation.
successful technical performance on the scenarios.
Section 4 presents results on team performance. It The analysis particularly focused on how crews includes a discussion of how multiple crew members organized themselves to rnanage the dual contributed to situation assessment and response requirements of (1) following through the steps in the identification and evaluation activities. It also EOPs and (2) engaging in extra-procedural activities discusses how crews organized themselves to deal in order to handle aspects of the situation that were with the dual needs to (1) engage in extra-procedural not covered by the EOPs. We focused on examining activities in order handle aspects of the situation that how different crews divided up these dual are not covered by the EOPs and (2) proceed through responsibihties, and whether differences in technical the EOP steps in order to ensure that all major safety performance resulted.
functions are maintained, and that actions required to return the plant to a safe state are performed in a A second aspect of the analysis focused on the timely manner. Section 4 includes presentation and usefulaess of the BARS rating scales per se in discussion of ratings of team interaction skills that evalu sting team interactions skills in these scenarios.
were made using the BARS scales.
- - examined crew ratings on the BARS scales to assess (1) whether there was variability in crew scores Section 5 summarizes the results of the study and on the BARS dimensions, and (2) whether there was a discusses conclusions and implications.
relationship between BARS ratings of team skill and crew performance on the scenario; from a technical Appendices A through C provide more details on the perspective.
study methodology. Appendix A provides detailed descriptions of the scenarios. Appendix B presents the instructions and sample summary sheets that were used in conducting the study. Appendix C presents the BARS team interaction skills rating scales.
Introduction -
' Appendix D presents a Cognitive Demands Checklist capture some of the findings of the project in a form that provides a structured list of factors (e.g.,
that can be used directly by the NRC staff to assess characteristics of the event, the procedures, the man-characteristics of an accident sequence or situation i
machine interface) that can result in cognitive errors.
that make errors of intention more likely.
The Cognitive Demands Checklist was developed to i
r t
k 6
l NUREG/CR-6208 6
2 Study Methodology 2.1 Approach The protocols were then analyzed to:
The design and analysis of the study was guided by a Identify situations that arose where operators model of higher-level cognitive activities involved in needed to engage in higher-level cognitive operator performance in NPP emergencies. The activities in order to deal with the case; model guided the design of cognitively demanding emergency scenarios that challenged operator Document extra-procedural activities that situation assessment and response planning. It also operators engaged in that provided evidence of specified the kinds of operator behaviors to look for these higher-level cognitive activities.
as evidence of situation assessment and response planning. This model is presented in Section 2.2.
The model of cognitive activity provided the framework for linking the specific extra-procedural Two types of simulated emergencies were included in activities observed to the higher-level cognitive the study: two variants of the ISLOCA into the activities.
Residual Heat Removal system (RHR) scenario that was run as part of the cognitive environment Descriptions of the data collection and analysis simulation (CES) exercises, and two variants of a loss methods are provided in Section 2.4.
of heat sink (LHS) event complicated by a stuck open pressurizer power operated relief valve (PORV). The ISLOCA scenarios were designed to be challenging from the point of view of situation assessment. The 2.2 A Model of Cognitive Activities loss of heat sink scenarios were designed to be Involved in Operator challenging both with respect to situation assessment and response planning. An overview of these Performance in Emergencies-scenarios is provided in Section 2.3. More detailed descriptions are provided in Appendix A.
The design and analysis of the study was guided by a model of higher-level cognitive activities involved in Two utilities were asked if they would voluntarily Operator performance in NPP emergencies. The participate in an empirical study of operator m del we used draws on concepts that underlie the performance in cognitively complex simulated CES model. These concepts are consistent with the emergencies. Both agreed to run an ISLOCA and a c re body of knowledge and theory in cognitive Loss of Heat Sink event as part of the regularly psychology and are supported by a large empirical scheduled requalification training exercises at one of base on human decision-making in real world their nuclear power plant sites. In this report the settings. 3 plants are referred to as Plant 1 and Plant 2. A different variant of the events was run at each plant.
The model includes two components: situation assessrnent and responseplanning. Describing The events were run on a high fidelity training cognitive processes in terms of these two higher-level simulator at the plant site, Crew performance in the cognitive activities is consistent with standard ways simulated emergencies was videotaped. Partial of modeling decision-making (e.g., Orasanu, transcripts tracing crew performance in each event Dismukes and Fischer,1993). Figure 2.1 provides a were generated from the videotapes.
3More detailed reviews of concepts from cognitive psychology and their application to NPP crew performance can be found in Mumaw Swatzler, Roth and Thomas 1994 and Woods and Roth.
1986.
Study Methodology Higher-Level Cognitive Situation Assessment Response Planning 1
Activity Constructing an explanation to accountfor Deciding on a course ofactwn observed plant behavior: a mental representation giten a particular situation of factors knoten or hypothesized to be affectina assessment.
plant state.
This mental representation generates:
Inteltes:
- expeciations about other plant parameters
. establishinggoals expectations aboutfuture consequences
. identifying / generating a response e
e explanationforobsertutions e evaluating / monitoring effectiveness
= identification of unexpectedplant behavior of responseplan and searchfor explanalion e anticipation ofpotentialfuture problems e adapting responseplan Figure 2.1 The cognitive activities encompassed under situation assessment and response planning.
summary of the cognitive activities encompassed the context of NPP operations, situation assessment under these two labels. Figure 2.2 shows the types of involves developing and updating a mental observable behaviors that result from these cognitive representation of the factors known or hypothesized activities.
to be affecting plant state at a given point in time.4 Situation assessment refers to both the process of Below, we describe in more detail the processes building the mental representation and the resulting involved in situation assessment and response mental representation.
planning and the resulting observable behaviors.
Section 3 contains specific examples of cases where Situation assessment is similar in meaning to these types of behaviors were observed in the
' diagnosis' but is broader in scope. Diagnosis simulated emergency scenarios.
typicaD, refers to searching for the cause(s) of abnornal symptoms (e.g., a disease, a malfunctioning 2.2.1 Situation Assessment piece of equipment). Situation assessment encompasses explanations that are generated to l
A fundamental finding in the field of cognitive account for normal as well as abnormal conditions. It I
psychology is that people actively try to construct a is similar to the concept of ' situation awareness' used coherent explanation to account for their observations in the aviation literature (Endsley,1993; Sarter and l
(Bartlett,1932; Bransford,1979). The process of Woods,1991).
l constructing an explanation to account for observations is referred to as situation assessment. In 4 A situation assessment need not be complete or fully accurate.
l
l l
1 Study Methodology liigher-Level Cognitive Situation Assessment Response Planning Activity
\\
/
Check for Anticipate Catch errors identify Future Problems Evidence to Goals Fillin gaps Confirm Search for in procedures Adapt procedures Hypothesis Explanation Evaluate l
identify to situation Unexpected Ppmpriateness Evaluate Plant Behavio of EOP procedural path Consequences of Actions Explain Observed Detect alarms / symptoms that Symptoms had been missed Detect abnormal Identify problems plant behavior (e.g., sensor failures; plant malfunctions)
Figure 2.2 Types of observable behaviors that result from situation assessment and response planning.
Evidence of these operator behaviors can be obtained through observation of operator actions and utterances Situation assessment is a complex activity that may current situation assessment. His search is a form of entail using mental models of physical systems and knowledge-driven monitoring.5 how they work. Examples from a nuclear power plant application include considering the physical They also use expectations they have generated to interconnections among systems (e.g., considering explain observed symptoms. If a new symptom is piping and valve interconnections to figure out how observed that is consistent with their expectations, I
w'.ter from one system could get into another), and they have a ready explanation for the finding. This considering the effects of mass and energy changes in gives them greater confidence in their situation one system on the behavior of a second system (e.g.,
assessment.
the effect of a cooldown in the primary system on secondary side steam generator level behavior).
When a new symptom is inconsistent with their expectation, it is recognized as an unexpected plant People form expectations based on their current behavior that suggests a need to revise the situation situation assessment. These expectations include the assessment. His symptom triggers a search for a events that should be happening at the same time, better explanation of the situation, which may how events should evolve over time, and effects that may occur in the future (i.e., expectations about future 1
" 7 edge-driven monitoring refers to monitoring that is driven consequences).
by an internally generated perceived need for a piece of information. This is contrasted with data-driven monitoring that is They use these expectations in several ways. They triggered by salient external stimuli such as alarms, and use them to search for evidence to confirm their procedure-driven monitoring that is determined by procedures that include explicit directives to monitor a parameter.
Study Methodology involve developing a hypothesis for what might be for achieving identified goals. As we will show, occurring, and then searching for evidence in the response pian monitoring enables operators to catch environment to confirm that hypothesis (i.e.,
errors, including errors made by the operators knowledge-driven monitoring). It can result in themselves and errors that may be present in the detecting abnormal plant behavior that might not procedures.
ctherwise have been observed, detecting plant symptoms and alarms that may have otherwise been Another cognitive activity included under response missed, and identifying problems such as sensor planning is response plan adaptation. His includes failures or plant malfunctions.
filling in gaps in a procedure, adapting a procedure to the specific situation, and redirecting the procedure People also use these expectations to project into the path. Adapting procedures includes taking actions future and anticipate potential problems. These that are not stated in the procedure, not taking projections are used in generating and evaluating actions that are stated in the procedure, and taking response plans (Klein and Calderwood,1991).
actions specified in the procedure but changing them in some way (e.g., changing a plant parameter value Situation assessment allows operators to detect mentioned in the procedure). In the analysis that abnormal plant behavior early and anticipate follows we will provide several examples of operators potential future problems.
adapting procedures to handle cases that could not be fully addressed by following the procedures verbatim.
2.2.2 Response Planning 2.3 Overview of Emergency Response planning refers to deciding on a course of action, given a particular situation assessment. In SCenarlOS general, response planning involves identifying goals, generating one or more alternative response The model described above guided design of plans, evaluating the response plans, and selecting emergency scenarios that were expected to be the response plan that best meets the goals identified.
cognitively demanding. Variants of two base scenarios were run: an ISLOCA into the RHR system While these are the classic activities associated with and a Loss of Heat Sink (LHS) event complicated by a response planning, one or more of these steps may be leaking pressurizer PORV. Rese emergency skipped or modified under certain circumstances. In scenarios were designed to create situations where the case of NPP emergencies where there are EOPs active situation assessment and response plan that provide predefined response plans, the need to evaluation and adaptation were needed on the part of generate a response plan in real-time is largely the operating crew to handle the events.
eliminated. However, as we will show through illustration, some elements of response planning The core characteristics of the scenarios were defined remain to be accomplished. Operators still need to by the project team based on pilot studies previously identify appropriate goals based on their own conducted on the Westinghouse training simulator situction assessment, evaluate whether the actions using training instructors as crew members. Using they are taking based on the procedures they are these base scenarios as a starting point, the project following are suf ficient to achieve those goals, and team worked with training instructors at each of the adapt the procedure to the situation if they decide it is two plants to tailor the events to the individual plants.
necessary.
Differences existed between the plants in procedures, simulator characteristics, and operating and training One cognitive activity included under response philosophy. Thus the events run at the two plants, planning is monitoring the effectiveness of the while similar in many sespects, differed in several response plan embodied in the EOPs. Response plan important ways.
monitoring includes evaluating the consequences of particular actions specified in the procedure steps, Sections 2.3.1 and 23.2 provide a brief description of and evaluating the appropriateness of the EOP path the main features of each scenario. More detailed NUREG/CR-6208 10
)
Study Methodology descriptions of the scenarios are provided in eventually reached a step in the LOCA procedure that Appendix A.6 asked them to "try and identify and isolate the leakage." Thus we were able to observe crew performance in a situation where the EOP explicitly 2.3.1 ISLOCA Scenarios required the crews to identify and isolate the leak without more detailed procedural guidance.
The ISLOCA scenarios involved a leak from the high pressure Reactor Coolant System (RCS) to the low At Plant 2 in the second variant of the event,ISLOCA pressure Residual Heat Removal (RHR) System. In 2, while there was an explicit transition to the one variant of the event (ISLOCA 1) the RCS leak into ISLOCA procedure from the LOCA procedure, either the RHR ever'ually led to an RHR pipe rupture in the transition step could not be reached, or the criteria the Auxiliary Building causing reactor coolant fluid to f r transitioning to the ISLOCA procedure were not spill onto the floor of the Auxiliary Building. In the met when the transition step was reached. Thus we second variant (ISLOCA 2) the event started in the were able to observe crew performance in a situation same way; however, the buildup of pressure in the where the procedure containing relevant guidance RHR led to a break in the heat exchanger between the c uld not be reached within the EOP transition network.
RHR system and the Component Cooling Water J
(CCW) system causing RCS fluid to get into the CCW In both variants of the scenario the crews had to system. (See Figure 2.3 for a simplified diagram of the systems m.volved in the ISLOCA scenarios.7) identify the ISLOCA into the RHR in attempting to isolate the leak. This situation assessment was i
man ng ause initial symptoms The ISLOCA scenarios were designed to be difficult from the point of view of situation assment. The Wer typical of a LOCA inside containment. Correct objective was to create a situation where the crews situation assessment required integrating multiple had to identif and isolate the leak into the RHR Symptoms across different systems. The first alarms Y
tadicate pressure and level decreases in the without explicit procedural guidance.
pressurizer. These are soon followed by alarms indicating radiation inside containment. Radiation in While the EOPs contain procedures for identifying and isolating an ISLOCA, it was possible io aeate a containment strongly point > to an RCS leak directly into containment (i.e., a LOCA). In fact, the radiation situation where the crews could not reach the in containment is caused by the leak into the RHR. A ISLOCA procedure within the EOP network. This is relief valve in the RHR system vents to the because the plant symptoms generated early in the Pressurizer Relief Tank (PRT) inside containment.
event are similar to the pattern of symptoms that The PRT eventually ruptures, resulting in radiation in would be produced by a Loss of Coolant Accident (LOCA)inside containment. By timing the dynamics containment. The crews needed to recognize these physical system interconnections in order to link the of the event carefully we were able to create a situation where the EOPs directed the operators to a symptoms in containment with a potential problem in the RHR' procedure for a LOCA inside containment.
In ISLOCA 1 a correct situation assessment required At Plant 1 in one variant of the event, ISLOCA 1, once in the LOCA procedure there was no explicit the ces to connect the symptoms in containment with the symptoms m, the Auxiliary Building.
transition to the ISLOCA procedure. The crews ISLOCA 2 was cognitively more demanding because it required the crews to integrate evidence across The overview of the scenarios presented in Sections 23.1 and m re systems and postulate a more complex causal 6
23.2 is simplined. Because the simplifications are necessarily chain of events to account for all the symptoms abstract, readers who are familiar with NPP design and observed. In particular, the crews needed to terminology might prefer to read Appendix A at this time.
recognize that the radiation in the CCW was due to RCS fluid that leaked into the RHR and entered the 7Figure 23 is a highly simplified diagram. A more complete and CCW via a heat exchanger between the RHR and the accurate diagram of the RilR system is provided in Figure A.1 in Appendix A.
'A Z
E C
c.
W s
O if
~
e.
b S-Reactor Coolant System 5
in Containment Steam Generator Pressurizer Rehet Tank Water O
Storage Reactor Pressurizer Coolant Tank p
O my (J
Surge ll h
Tank I
{
/
\\
W l
Reactor
\\
D FIH RHR Heat Relief k
Exchanger Valve F
RHR, Isola, tion Vatves CCW Pump RHR Pump g
Corrponent Cooling Residual Heat Water System Removal System
\\
O O
Figure 2.3 Simplified diagram of the systems relevant to the ISLOCA scenarios. (1) Two RHR isolation valves began to leak. (2) The RHR relief valve opened, directing radioactive fluid to the Pressurizer Relief Tank. (3) The Pressurizer Relief Tank broke, resulting in radiation symptoms inside containment. (4) Eventually a break occurred in the RHR system. In ISLOCA 1 the break was in an RHR pipe in the Auxiliary Building. In ISLOCA 2 the break was in the RHR heat exchanger with the CCW system.
Study Methodology Once the operators identified a leak into the RHR they pressure. In the event we ran the pressurizer PORV needed to take action to attempt to isolate the leak.
never fully closes (although it read closed), resulting The appropriate action to take depended on the in a leak on the primary side. The analysis focused on postulated source of the leak. In the event we ran how the operators discovered and dealt with the there were two hypotheses for the source of the leak leaking PORV, given that the EOPs provided no that were equally plausible in that they could fully explicit guidance. (See Figure 2.4 for a simplified explain the available evidence. One was diagram of the systems involved in the Loss of Heat a failure of the two isolation valves between the hot Sink scenarios.)
leg loop of the RCS system and the RHR on the suction side of the RHR pump. This is the event that
%e scenario was demanding from the perspective of we postulated. Given this hypothesis the actions situation assessment because it created a situation required to isolate the leak are to call the Auxiliary where operator judgment was needed to discriminate Building to request that the valves be re-energized, to plant behavior that was the result of known factors verify that they are closed, and to close them if they (i.e., an operator induced cooldown) from plant are not. The alternative hypothesis was that there behavior that signaled an additional plant fault.
was a leak back from the RCS through a series of Many of the early symptoms of theleaking failed check valves. Given this hypothesis, the leak pressurizer PORV (i.e., decreasing pressurizer level could be isolated by closing an isolation valve on the and pressure) could be attributed to a cooldown discharge side of the RHR pump that is normally kept caused by the control actions that the operators were open.8 taking to recover the secondary side heat sink. As the event progressed the symptoms on the primary side In ISLOCA 2 the crews also needed to take action to became more severe (i.e., reactor vessel level isolate the leak from the RHR into the CCW. This step decreased; a bubble formed in the venel; the required that they identify the RHR heat exchanger as f essurizer became solid). Those symptoms could not I
the source of the leak and take action toisolate it.
be explained by a cooldown caused by activities on the secondary side.
2.3.2 Loss of Heat Sink Scenarios The Loss of Heat Sink scenario was also designed to be challenging from the perspective of response The Loss of Heat Sink event involved a total loss of planning. In ne variant of the scenario, LHS 1, at feedwater flow complicated by a leaking pressurizer Plant 1, secondary side feedwater is never recovered.
power operated relief valve (PORV). The objective As a result the crews remain in the Loss of Heat Sink was to create a situation where the EOPs focused procedure. This variant was designed to place crews operator attention on one high priority problem - a in situation where they had to decide whether to loss of heat sink - and then examine how the crews manually initiate a safety system under conditions discovered and dealt with a second potentially where procedural guidance was minimal, and serious fault that arose: a leaking pressurizer PORV.
multiple goals needed to be considered and balanced.
The Loss of Heat Sink event was designed to be Specifically, the crews had to decide whether to cognitively demanding from the perspective of both manually initiate safety injection (SI). There was a situation assessment and response planning. In this Step early in the Loss of Heat Sink procedure that had 1
scenario feedwater to the steam generators is lost and the crews block SI.9This action has potentially serious the EOPs direct the operators to a Loss of Heat Sink safety consequences bwause it means that a major procedure that specifies actions the operators should automatic safety actuation system is no longer in take in attempting to recover feedwater. While operation and must be manually initiated if needed.
following the Loss of Heat Sink procedure, the Be only procedural guidance available to the operators are directed to open and then close the operators regarding manual initiation of SI was in a pressurizer PORV in order to reduce pressurizer 951 is blocked to avoid spurious activation of safety injection hhe location of the RilR discharge-side check valves and when the stearn generators are depressurized below an Si isolation valves are shown in Figure A.1 in Appendix A.
actuation set point later in the procedure.
(n CL e
x E
O E
n N
b if O
'I 09 x
C Pressurizer Block Relief Tank Valve Main Feedwater Isolation Valves Steam Generator Condenser T
Pressurizer Main Feedwater Condensate (j
Pumps 5
Reactor Condensate M
pegetor Vessel ant ank Auxiliary Pump Feedwater l
Isolation Vaives I
i L
Pumps Figure 2.4 Simplified diagram of the systems relevant to the Loss of Heat Sink (LHS) sces.arios. (1) In the LHS event there was a loss of both main and auxiliary feedwater. (2) As part of a step in the EOP the pressurizer PORV is opened. It is then closed but begins to leak. (3) Pressure and level in the pressurizer begin to fall. (4) The Pressurizer Relief Tank begins to fill and eventually breaks. (5) Eventually, a bubble forms in the reactor vessel. (6) The leak through the PORV can be terminated by closing the block valve.
Study Methodology l
caution that stated: "Following block of automatic Si not perceived to be appropriate to the specific actuation, manual Si actuation may be required if situation.10 conditions degrade."
The LHS scenario was designed to place the crews in 2.4 Data Collection and Analysis a situation where they had to decide whether to initiate SI under conditions where there were multiple 2.4.1 Participants goals that needed to be considered. The leaking pressurizer PORV created a situation where RCS Crews from two NPP sites participated in the study.
conditions became progressively more abnormal.
The two plants were pressurized water reactor -
Eventually, RCS pressure decreased to the point (PWR) plants. These will be referred to as Plant 1 and where a bubble formed in the reactor vessel. Level in Plant 2. The ISLOCA and Loss of Heat Sink scenarios the reactor vessel continued downward, while level in were run as part of the regularly scheduled l
the pressurizer started to go up. In some cases the requalification training at these plants. As a result all pressurizer became full. 'lhe degrading RCS the crews undergoing requalification training at the conditions could be mitigated by manually initiating two plants during that period participated in the Sl; however, the decision of whether to manually study. This included both actual operator crews initiate SI is made complex because it affects heat currently on shift as well as crews composed of sink recovery efforts. Initiating S1 would impede administrative staff undergoing requalification efforts to recover feedwater flow on the secondary training to maintain their Reactor Operator (RO) or side, and increase the probability that the crews Senior Reactor Operator (SRO) licenses.
would have to resort to a less desirable means of achieving a heat sink (i.e., bleed and feed). The A total of 11 crews from Plant 1 (five shift crews and objective of this aspect of the scenario was to examine six staff crewsil) and 11 crews from Plant 2 (six shift how crews responded to the degrading conditions in crews and five staff crews) were included in the the RCS, given that the only relevant procedural guidance available to them was in a caution.
study.
. Specifically, the analysis focused on whether the A full complement crew included five members:
crews chose to initiate Si and the rationale for their decision.
The second variant of the Loss of Heat Sink event, 10 The developers of the EOPs recognized that the type of
' LHS 2, at Plant 2, was also demanding from the situation created in LilS 2, w here crews retum to a procedure that perspective of response planning. In this scenario the includes sicPs that are not appropriate to the situation, could arise.
crews eventually Eot feedwater back. As a result the
- """ U"'####' '#"#"8 """ U""'" 6'#"##" "#'"
h Loss of Heat Sink procedure transitioned them back Response Guidelines and Background Document explicitly addresses the type of situation created in LilS 2. It states "After to the procedure they had been in when feedwater restoration of any Critical Safety Function from a RED or was lost, which was the Reactor Trip Response ORANGE condition, recovery actions may continue when the procedure. This transition introduced new cognitive FRG is complete... Upon continuation of recovery actions, some challen8es because some of the steps in the Reactor judgment is required by the operator to avoid inadvertent remstatement of a RED or ORANGE condition by undoing some -
Trip Response procedure were no longer appropriate.
critical step in a Function Restoration Guideline." (Westinghouse
. The crews were now feeding through the condensate Owners Group Emergency Raponse Guidelines, Users Guidefor ~
. system which involves a different plant configuration Emergency Response Guidelines and Background Documents, than is assumed by the Reactor Trip Response september 1,1983, pg.17.) The use of the phrase "some Judgment is required by the operator" suggests that the developers Procedure. Some of the steps in the Reactor Trip of the EOPs recognize that in these circumstances operators need Response procedure,if followed verbatim, would to evaluate the appropriateness of certain procedure steps based on l
undo actions that had been performed to recover their own situation assessment.
feedwater, causing a loss of heat sink.- This variant of the Loss of Heat Sink scenario provided the 11 videotapes were collected for two additional crews at Plant 1; oEportunity to observe how operators respond in however there were technical problems with the quality of the videotapes which prevented the performance of these crews to be cases where actions specified m. procedure steps are included in the study.
15 NUREG/CR-6208 e
n e--
a
Study Methodology The Supervising Operator (50) responsible for purpose of the research is to understand the decision-reading the procedures and directing and making process involved in diagnosing and coordinating operator activity; responding to challenging emergency scenarios. They were asked to " Handle these events as you would if The Reactor Operator (RO) responsible for they were actually happening in the plant. Use all of monitoring and control actions on the reactor the resources available to you - anything you would side; use in a real situation to mitigate the event.'" The i
crews were informed that the session would be The Balance of Plant Operator (BOP) responsible videotaped to facilitate analysis but they would only for monitoring and control actions on the balance be viewed by the research team. It was emphasized of plant side; that this was strictly a research project, that they were i
not being evaluated, and that every reasonable effort The Shift Supervisor (SS), the most senior would be made to preserve the anonymity of the crew management level staff person on the scene, participants in reporting the results.
responsible for acclaring site emergencies, and approving acuons; Occasionally, one or more of the crew members had prior knowledge of the event to be run. This The Shift Technical Advisor (STA) responsible for primarily happened in cases where the training monitoriug the status trees and providing instructor staffs were used to fill out a crew. In those technical advice.
cases the individual was directed to play a passive role, providing information and taking control In several cases four-member crews were run with actions as requested, but not volunteering one person simultaneously taking on the SS and STA interpretation or advice, or actively participating in roles.
situation assessment discussions. The crews where one or more members were cognizant of the event are in order to preserve the anonymity of the crews, each explicitly noted in the analysis.
crew was assigned an arbitrary letter (at Plant 1) or number (at Plant 2) to serve as identifier. These were After each event was run a debriefing was conducted.
used as crew identifiers in the transcribed protocols, The instructor began by asking whether any crew and in the presentation of results in Section 3.
member had prior knowledge of the event. If so, that was noted. He then reminded them that the event they had participated in was part of a research project 2.4.2 Procedure and requested that they do not discuss the event with anyone who has not yet participated in the events.
Two scenarios (c ne variant of an ISLOCA and one The instructor then led a relatively unstructured variant of a Loss of Heat Sink) were run by the debriefing similarin format to the type of debriefing training instructors at each plant as part of the conducted during regular simulator training exercises. The instructor had the crew summarize the regularly scheduled requalification training P ant faults they thought were present, actions they l
i conducted on a high fidelity training simulator. Crew members did not have prior knowledge of the event decided to take, and the reason for their actions.
to be run.
The simulator sessions, including the debriefing, Instructions were prepared to be read by the training were videotaped. The videotapes included a date and instructor to the crews at the beginning of the session.
time stamp that enabled identification of the time at Appendix B contains a copy of these instructions.
which key activities took place.
At the end of the session the instructors filled out Briefly, the instructions informed the crews that the two events they would be participating in were part summary data sheets for each crew on each event run.
of a research project being conducted by the The summary sheets included backgrotmd Westinghouse Science & Technology Center for the information on the crew members (whether it was a NRC's Research Office. They were told that the crew on shift or a staff crew, the licenses each crew member held, and their education); indication as to NUREG/CR-6208 16
Study Methodology which crew members, if any, had prior knowledge of In Figure 2.5 procedure-driven activities appear l
the event; yes/no questions regarding whether key without a border and extra-procedural activities situation 2ssessments and actions were made; and an appear inside a rectangular border.
area where the instructor could write in brief remarks.
Examples of the sununary sheets are included in Extra-procedural activities served as the primary A pendix B.
focus of analysis. Situations where extra-procedural P
activities arose were examined to:
In addition, data tapes were made recording key plant parameters and alarms off of the high fidelity training (1) Identify the kinds of situations that arose in the simulator while the event was running. These data scenarios where extra-procedural activities were tapes were used as a backup resource to check on required to deal with the situation; and plant parameter behavior in cases where questions (2) Obtain evidence that the crews were engaging in arose.
situation assessment and response planning in these situations.
2.4.3 Data Analysis The model of cognitive activities presented in Section The crew performance videotapes constituted the 2.2 provided the framework for inferring higher level primary source of data in the study. Protocols of the cognitive activities from specific extra-Trocedural performance of each crew on each event run were activities. The model specified types r f observable produced from the videotapes. The protocols behavior that result from situation as.essment and consisted of partial transcripts of crew dialogue that response planning (see Figure 2.2). '1he specific documented crew observations, hypotheses, extra-procedural activities documented in the discussions, and actions related to the key faults in the Protocols were examined to determine if they events.
represented instances of these types of behavior.
Examples include cases where operators monitored a l
P ant parameter to obtain evidence to confirm a The analysis primarily focused on providing evidence of the role of higher-level cognitive activity in guiding hypothesis, or to search for an explanation for an operator performance in the scenarios. The logic unaccounted plant behavior. These documented employed in the analysis is described below, and cases were then used as evidence of the higher-level captured schematically in Figure 2.5.
cognitive activity. Figure 2.5 illustrates the link between specific observed instances of extra-The protocols document observable behavior - either Procedural activities, the type of behavior they actions or utterances. Some examples are provided in exemplified, and the higher-level cognitive t.crivity Figure 2.5. These include monitoring plant for which they provided evidence. For example, cases parameters, interpreting plant state, taking a control where operators monitored a plant parameter to action, reviewing steps in the EOPs, and checking confirm a hypothesis were used as evidence of schematics. Two main types of activities were situation assessment.
observed:
(1) Behaviors that were directly the result of 2.43.1 Information Recorded in the Protocol following steps in the EOPs. These are refermf to as procedure-driven activities, and were not The objective of the protocol was to document the analyzed further, observations made, hypotheses considered, situation assessments made, and actions taken related to (2) Other behaviors not directed by the specific step identifying and responding to the faults in the in the EOP that the crew was following at that scenario. Particular attention was' paid to point in time. These behaviors are referred to as documenting instances where operators summarized extra-procedural activities..
their situation assessment; engaged in extra-procedural activity; reflected on the goals or objectives of the procedures; made judgment calls; or modified a procedure step, 17 NUREG/CR-6208
u Study Methodology Higher Level Cognitive Situation Assessment Response Planning Activity N
I I
l I
Check for Search for Fillin Gaps Evidence t Explanation li m Procedures Confirm Hypothes.is Evaluate Appropriateness identify Problems (e.g., sensor failures; Adapt Procedures to Situation plant malfunctions)
II If If 3 7 37 Monitor Monitor Interpret Monitor Take Take Observable Plant Plant Plant Plant Control Review Check Control Behavior:
Parameter Paramcter State Parameter Action EOP Schematics Action Figure 2.5 Illustration of the logic employed to infer situation assessment and response planning from the observable behaviors documented in the protocols. Specific observable behaviors - either actions or utterances - were classified as procedure-driven (no border) or extra-procedural (a rectangular border). The extra-procedural activities were examined to determine if they represented instances of observable behaviors that result from situation assessment and response planning (see Figure 2.2). In this way a link was established between specific observed instances of extra-procedural activities, the type of behavior they exemplified, and the higher-level cognitive activity for which they provided evidence.
Key procedure steps (e.g., steps that resulted in Activities recorded in the protocolincluded:
procedure transitions; steps that resulted in loop Observation of alarms, plant parameter valuer, back to earlier steps; steps that required judgment
=
and automatic system actuation (e.g., reactor trip, calls; steps that required operators to engage in SI; pressurizer pressure and level; PRT symptoms; diagnosis and response planning; key cautions);
RHR symptoms; containment symptoms);
NUREG/CR-6208 18 f
Study Methodology Hypotheses considered; dialogue that the behavior occurred, and assumed e
I that the behavior did not occur. In this sense, the Actions taken or considered related to dealing protocols and analysis are based on the externally e
with the faults (e.g., closing isolation valves; observable distributed group cognition (Hutchins, closing block valves; manual SI);
1990;1991).
Information that is volunteered or requested that is not strictly called for by the procedure; 2.4.3.2 Dehaviorally Anchored Rating Scales (BARS) of Team Interaction Skills Cases where operators are performing activities not called out in procedures (e.g., reviewing Based on the videotapes of crew performance, ratings schematics);
were made of team interaction skills using the BARS scales developed by Montgomery et al. (1992). This Discussions among staff regarding the rating scale was developed under the sponsorship of a
interpretation of the situation or appropriate the U S. Nuclear Regulatory Commission specifically actions to be taken; to support evaluation of NPP crew team interaction skills.
Instances where someone " recapitulates" the situation as they understand it; In the BARS methodology crew interaction skills are evaluated on each of six dimensions:
Instances where someone requests opinions or consensus about the situation assessment or
- Communications actions to be taken; Openness e
Task coordination Discussions regarding the EOPs and their Team spirit appropriateness to the situation (e.g., whether the Maintaining task focus in transitions procedures will eventually get them to where Adaptability they want to be; whether other procedures should be consulted).
Montgomery et al. (1992) define each of these 1
dimensions as follows:
For each of these cases the protocol documented the The communications dimension consists of the time it occurred, who was involved (50, RO, BOP, transmission of factual information in a clear and STA or SS), and a verbatim transcription or very close concise manner. In terms of crew behaviors this paraphrase of what was said. Statements that are not includes listening skills, nonverbal behavior, and verbatim transcriptions are presented in parentheses articulation of plant status or instructions for or between stars (e.g., '** The SO asked the STA to future activities. Communication does not check the status trees '"). To preserve the anonymity include emotional aspects of information of the crew members, in cases where crew members transmission.
were referred to by name, their crew position was substituted for the name. In cases where the Openness consists of crew members' tendency to individual making the statement could not be ask for, give, and receive suggestions. It includes identified, question marks were used in place of crew questioning decisions and discussing alternatives position.
to arrive at the best possible decision. Openness also hwolves the reactions of crew members to Note that all conclusions about plant symptoms feedback, which should focus on the task rather observed, conclusions drawn, and actions taken are than on the person when reviewing actions, derived from analysis of crew dialogue during the videotaped scenarios and the debriefing. If a crew Task coordination refers to the crew members' member noticed a parameter, or took an action, but ability to match the available resources, such as did not mention it either during the event or as part of people and procedures, to the task in order to the debriefing, then we had no evidence from the achieve the optimal workload distribution.
Study Methodology Team spirit consists of mutual support, situation, to be flexible in responding to the cohesiveness, group identity, and the extra effort environment, and to recognize the need for that crew members exhibit to accomplish a change.
j common goal.
Performance on each of these dimensions is rated on a Maintaining taskfocus in transitions deals with seven point scale where a 1 is low (poor) and 7 is high e
crew members' responses to changes from normal (good). Copies of the BARS rating scales are presented to non-normal plant conditions (e.g., loss of in Appendix C.
pressure in feedwater pump). These responses include focusing on the task, avoiding emotional The ratings were performed by a single individual 1
overreaction or panic, and maintaining poise.
(the first author) who also prepared the protocols from the videotapes. The ratings for each crew on Adaptability reflects crew members' ability to each scenario are presented in Section 4 on team adjust or modify their behavior based on the performance.
l l
3 Cognitive Performance in the Simulated Emergencies This section presents the main results for operator LOCA procedure that they were following.12 The cognitive performance in the four simulated step asks whether pressure in all steam generators is emergency scenarios. The results for each scenario stable or increasing. The purpose of this step is to are organized around cases where extra-procedural check for the possibility of a faulted steam generator activity was needed to handle the situation. In each (SG), which would cause SG pressures to decrease.13 case we examined the extra-procedural activities In fact, the SG was not faulted, but there was a observed for supporting evidence of situation cooldown in progress, which also results in steam assessment and response planning as described.
generator pressures decreasing. Based on training and EOP background documents, crews are The data presented include tables that summarize instructed that if they are taking actions that are performance across crews on particular aspects of the producing a cooldown they should consider a scenario. These tables indicate the consistency and decrease in SG pressure to be " stable or increasing."
variability in performance among crews.
Thus, this is a case where response to a procedure step is based on situation assessment rather than In addition, protocol segments for particular crews literalinterpretation of the procedure step. In are presented that cover a more extensive portion of deciding how to respond to the procedure step they the scenario. These protocol segments illustrate the need to assess whether the plant behavior they are complexity of the situations the crews confronted and observing is the result of known manual and/or the role that situation assessment and response automatic actions that are producing a cooldown, or planning played in guiding crew performance.
the result of a plant fault. The analysis focused on how crews responded in cases where SG pressures were decreasing when they got to this step.
3.1 ISLOCA 1: ISLOCA into RHR Leading to Pipe Rupture in 3.1.2 Characteristics of Participating Auxiliary Building Crews 3.1.1 Summary of Simulated Scenario Eleven crews from Plant 1 participated in the event.
Of these, five were crews currently on shift and six Analysis of crew performance in ISLOCA 1 identified were staff crews made up of administration two cases where operators had to engage in situation personnel. Crew size ranged from four to five people.
assessment and response planning to deal with the in four cases one or more of the crew members had situation. The first case is the situation around which prior knowledge of the event. Those individuals did the ISLOCA 1 scenario was designed: a case where a not actively participate in situation assessment and step in the EOP explicitly requests the crews to response plannmg.14 identify and isolate a leak, without providing more l
detailed procedural guidance. Analysis of the extra-procedural activities the crews engaged in to identify 12 m M m omm Wures %ss d and isolate the ISLOCA into the F11R. provided Reactor or Secondary Coolant Procedure," which is also referred specific, concrete examples of the types of situation to as E-1. In this report, LOCA procedure, loss of Reactor or assessment and response planmng behaviors which Secondary Coolant procedure, and E-1 are used synonomously.
are listed in Figure 2.2.
13A faulted steam generator is defined as having a discontinuity in the pressure boundary allowing either steam or feedwater to The second case is a specific example of a situation leak out. Examples are steam line breaks and feed line breaks.
14 where operators need to engage in situation The crews that included individuals with prior knowledge of the event were: Crew B (the STA). Crew D (the SS), Crew L (the p
assessment.in order to determine how to respond to a SS and STA), and Crew G (the STA).
procedure step. The specific case arose in a step in the 21 NUREG/CR-6208
Cognitive Performance 3.1.3 A Case Where a Step in the EOP the same underlying fault, and if so on what Explicitly Requests the Crews to basis; i
Identify and Isolate a Leak Actions that the crews took to identify and isolate the ISLOCA, given that they had no explicit EOP The ISLOCA scenario was ttmed so that containment guidance in performing this task.
symptoms occurred early in the event.15 Given containment symptoms, the EOP directed the This section describes the extra-procedural activities operators to the LOCA procedure. Once in the the crews engaged in to identify and isolate the LOCA procedure there was no explicit procedure ISLOCA into the RHR. These activities are specific, transition to the ISLOCA procedure.16 A step in the concrete examples of the types of situation assessment LOCA procedure checked for radiation in the and response planning behaviors which are listed in Auxiliary Building. If there was radiation, which is Figure 2.2.
an indicator of an ISLOCA, the step said to:" Identify and isolate the leakage."
Situation Assessment: Detectingabnormal RHR By the time the operators reached that step in the symptoms LOCA procedure, the RHR piping had ruptured, resulting in radiation in the Auxiliary Building. This lhe analysis examined whether crews detected enabled us to observe crew performance in a situation symptoms of a problem in the RHR system. Since the where the EOP explicitly required the crews to EOPs did not explicitly direct the crews to check for identify and isolate a leak without providing more RHR symptoms, identification of RHR symptoms was detailed procedural guidance.
based on situation assessment activities.
The data analysis focused on the activities the crews The first symptom of an RHR problem was an RHR engaged in to identify and isolate the ISLOCA into the discharge pressure high alarm that came in prior to RHR. Specifically:
the reactor trip. Meters were also available on the control board that, if the crews checked, provided Whether crews detected symptoms of a problem symptoms of a problem in the RHR system (i.e., high a
in the RHR system; RHR discharge pressure and high RHR discharge temperature).
Whether crews identified a problem in the RHR system, and if so on what basis; Not all the crews cietected the RHR high discharge pressure alarm when it came in.17 Six of the 11 crews How crews explained plant symptoms they were not aware that an RHR alarm came on.181he observed; fact that six of the crews did not know about the RHR alarm pr vided the opportunity to look at the Whether crews recognized that the symptoms inside and outside containment all resulted from 17Among reasons for the difference in detection is that, because of simulator characteristics, the event did not always run in exactly the same way. Sometimes the RilR alarm sounded up to 15 There were symptoms inside containment because the RilR 30 seconds prior to the reactor trip, u hereas other times it came system includes a relief valve that vents to the PRT, which is within 10 seconds of the reactor trip.
inside containment. When pressure in the RilR increased, the 18here are two points to note with respect to detection of the relief valve opened and radioactive fluid was directed to the PRT.
RiiR alarm. First, that alarms will not necessarily be noticed if De PRT ruptured causing radiation alarms within containment.
they are embedded in a large set of alarms as occurred here. Five of the 11 crews did not mention outloud that there was an RilR 161t should be noted that, while at the panicular time that we ran alarm at the time it came on. A related point is that, even if an this event, there was no explicit transition from the LOCA alarm is called out, it may be forgotten during the event. In one procedure to the ISLOCA procedure in the EOPs used at that case (Crew G), although someone on the crew called out the alarm plant, the latest version of the Emergency Response Guidelines when it came on, the cause of the alarm was not pursued In the (ERGS) does include an explicit transition from the LOCA debriefing everyone on the crew claimed to have been completely pmcedure to the ISLOCA procedure.
unaware of the alarm.
l l
Cognitive Performance j
difference in performance between crews that noticed The contrast in performance between these two the RHR alarm early, and crews that did not. As will groups provided a concrete example of the value of be seen the performance of these two groups varied in knowledge-driven monitoring and situation significant ways.
assessment in identifying problems carly.
One of the first differences to be noted is whether the A difference existed between the crews that had crews checked for RHR symptoms on the control initially observed the RHR alarm and those that had board (i.e., RHR discharge pressure and RHR not. Four of the five crews that had initially observed discharge temperature). All five crews who detected the RHR alarm identified a problem in the RHR carly the IU IR alarm checked RHR control board based on the RHR alarm or the RHR discharge parameters and noticed abnormal RHR parameter pressure control board readings. In contrast, of the behavior. In contrast, five of the six crews who did six crews that were not aware of the RHR alarm, four not know about the RHR alarm, failed to detect the did not identify the RHR problem until they received RHR symptoms. The search for RHR symptoms a call from an instructor, playing the role of an provides a clear instance of knowledge-driven Auxiliary Building operator, telling them that there monitoring that leads to the identification of was an RHR problem (in the case of Crew E), or abnormal plant behavior. In this case, the crews had providing a strong clue to that effect (i.e., telling them no procedural guidance to check RHR symptoms.
that there was radioactive fluid outside the RHR They checked them because the RHR alarm led them pump room).
to suspect an RHR problem The remaining two crews that had not noticed the RHR alarm (CrewsJ and L) were never told about Situation Assessment: De'ayed detection of the radioactive fluid outside the RHR pump room.
alarms / symptoms For these crews the only evidence pointing to an RHR problem was the increase in PRT level and One crew (Crew E) provided a concrete example of subsequent break in the PRT. Of the two crews in that how a situation assessment can aid in detecting situation, one crew (Crew L) never localized the RHR alarms and symptoms that were missed earlier. This problem. The other crew (Crew J) localized the crew had not noticed the RIIR symptoms early in the problem by pursuing sources of input into the PRT, event. The crew decided to check for RHR symptoms and then checking the alarm printout. This latter late in the event, after they were informed by a call crew provided an excellent example of how a crew from the Auxiliary Building operator that there was a utilizes knowledge-driven monitoring and extemal problem in the RHR. By that point the primary resources,in this case the alarm print out, to form a indicator in the control room of an RHR problem, situation assessment, localize a fault, and determine a RHR discharge pressure high, was no longer present course of action. The performance of this crew will be because RHR discharge pressure decreased when the examined in more detail below.
RHR pipe in the Auxiliary Building broke. The crew decided to examine the alarm printout. By reviewing The contrast in performance between the crews that the alarm printout they were able to identify that an detected the RHR alarm early and those that did not RHR discharge pressure high alarm had occurred illustrates the value of knowledge-driven monitoring prior to the reactor trip. This case provides a concrete and situation assessment. The crews that detected the example of how searching for an explanation for RHR alarm identified a problem in the RHR earlier in unexplained plant behavior can result in the detection the event than the crews that required symptoms of a of alarms that were previously missed.
burst RHR pipe in the Auxiliary Building before identifying the RHR problem. In an actual ISLOCA incident, early detection of a problem in the RHR Situation Assessment: Identifying a problem in the RHR would be important, because it would provide the potential for isolating the leak into the RHR before the The analysis examined whether crews identified a RHR piping burst. Once the RHR pipe burst the problem in the RHR system, and if so on what basis.
ISLOCA became unisolatable.
Crews that had detected the RHR alarm identified the problem in the RHR earlier than crews that had not.
Cognitive Performance Situation Assessment: Explaining radiation in Ten of the 11 crews identified the Auxiliary Building containment Problem based on the Auxiliary Building alarms which came in early in the event, before they reached The analysis examined how crews explained observed the LOCA procedure.20 Of those ten crews, five symptoms, and how situation assessment affected called the Auxiliary Builoing operator to ask them to explanation of symptoms.
search for possible sources of RCS leak into the Auxiliary Building. This action was not directed by Early plant symptoms included high temperature and the procedure the operators were following. The call pressure in the PRT, the PRT rupturing, and radiation to the Auxiliary Building operator illustrates extra-in containment. The containment radiation was procedural activity in search of an explanation for an caused by the rupture of the PRT, which resulted in unexplained plant symptom. In this case they were i
release of radioactive fluid into containment. We trying to obtain information to aid them in localizing examined whether crews observed the PRT and the source of the leak into the Auxiliary Building.
containment symptoms, and whether they linked the There is an EOP procedure step that explicitly asks containment symptoms to the rupturing of the PRT.
crews to check for Auxiliary Building symptoms, but that step occurs later in the EOPs. Calling the Ten of the 11 crews noticed the high temperature and Auxiliary Building when the alarm is received, while pressure PRT alarms prior to the reactor trip. Later in not based on an explicit procedure step, is considered the event the PRT ruptured. Of the ten crews that good practice based on training and standard noticed the PRT alarms, nine mentioned the PRT practice. 'Ihis extra-procedural action provides rupturing at some point in the event. Only one crew another illustration of the role of situation assessment (Crew M) showed no indication of being aware of the activities in enabling crews to detect and pursue plant PRT symptoms and rupture.
symptoms earlier in the event than would otherwise When containment symptoms arose, six of the 11 crews attributed the containment symptoms to the fact that the PRT had ruptured.19The remaining Situation Assessment: Scarchingforan explanation of crews gave no verbalindication of how they symptoms both inside and outside containment explained the containment symptoms. Some of these may have also attributed the containment symptoms To isolate the leak into the RHR, the crews had to to the PRT rupture without verbalizing it.
identify the source of the leak. This required active situation assessment. We examined how crews The fact that the majority of crews recognized the explained the symptoms in the RHR and whether source of radiation in containment illustrates the role they recognized that the symptoms inside and outside of situation assessment in explaining observed containment all resulted from the same underlying symptoms.
fault. The case provided a concrete example of an active search for an explanation that linked multiple Situation Assessment: Requesting a search and explanationfor symptoms in the Auxiliary Building Table 3.1 presents the bypotheses that were considered by the crews to explain the symptoms Containment radiation provided a concrete example observed incide and outside containment. In these of a symptom that could be explained by the tables 'LOCA' refers to a loss of coolant accident
(
operators' situation assessment. In contrast, radiation inside containment; ' check valve' refers to the in the Auxiliary Building provided an example of hypothesis that the RHR symptoms were caused by a l
where an unexplained symptom triggered extra-leak back through a series of check valves.
l procedural activity in search of an explanation.
19 One of the crews (Crew B) correctly attributed the 20 I
containment symptoms to the rupturing of the PRT, but attributed The Auxiliary Building alarms came in while the crews were in the PRT symptoms to a PRZR steam space leak.
the entry EOP procedure, which is the Reactor Trip or Safety Injection procedure, also called E-0.
l Cognitive Performance Table 3.1 ISLOCA 1. liypothesized explanations for plant symptoms.
Crew No.
First Hypothesis Revised Hypothesis Final Hypothesis A-check valves no; leak terminated check valves B
steam space leak and SI ISLOCA RHR ISLOCA check valves C
RHR ISLOCA check valves check valves D
RHR ISLOCA isol. or check valves isol. or check valves E
LOCA and letdown line break check valves check valves F
RHR ISLOCA check valves isol. or check valves G
RWST to RHR to aux, floor no RHR problem H
RHR ISLOCA check valves check valves J
RCP sealleak offs check valves isol. or check valves L
LOCA and ISLOCA no two leaks - LOCA and ISLOCA M
RHR che:k valves Unisol. check valve two leaks - check valve and small LOCA Initial hypotheses were varied and included several Of the remaining three crews, one crew, who had cases where crews postulated two separate originally considered the hypothesis that the hypotheses (Crews E, L and B) to explain the radioactive water was coming from the RWST into the symptoms inside and outside containment. By the RHR, had an unspecific situation assessment of some end of the event most crews correctly identified the problem in the RHR by the end of the scenario. The RIIR problem, and either attributed it to back flow remaining two crews believed that there were two through check valves (five crews) or specifically leaks: a LOCA inside containment and an ISLOCA by indicated that it could be either back flow through the the end of the scenario.
check valves or leaking isolation valves between the Several important points should be noted from these RCS and the RHR (three crews). These eight crews results. First, the majority of crews (eight of 11),
correctly determined that the RHR problem explained ended with a highly plausible and specific situation the symptoms in containment as well. The final assessment to account for the observed symptoms in situation assessment reached by these eight crews can the Auxiliary Building, in the PRT, and in.
be considered to be as complete, specific, and accurate containment. Second, not all crews were able to as would be possible to reach given the evidence recognize that there was a single explanation that presented.21 could connect all the symptoms. Two of the crews postulated two independent faults, a LOCA inside containment and an ISLOCA, as their final hypotheses. This explanation is less plausible because 2I While the check valve hypothesis was not the fault we it assumes two independent faults; further,it does originally postulated for this scenario, the training instructors felt not link the symptoms in the PRT with the symptoms it was at least as plausible an explanation for the observed in containment.22 symptoms as two leaking isolation valves between the RCS and the RilR systems. In several cases when the crews took action to isolate the check valves (i.e., they isolated the RIIR train), the Another point to be noted regards the process of instructors terminated the leak into the RilR; thus in those cases hypothesis revision and the role that multiple crew the scenario was run as if the check valves were the source of the leak. Because the scenario was sometimes run in this way, and members play in revising hypotheses. Five of the 11 because in cases where the crews did ask the Auxiliary Building crews (Crews E, J, L, G, and B) began with hypotheses i
oprator to check the two isolation valves between the RCS system and the R11R, they were told the valves read closed, it was decided to consider both hypotheses to be equally valid. A more 22The crews that postulated two independent faults did not
{
strict criterion would require crews to have explicitly considered mention that the PRT had ruptured at any point in the event, both hypotheses.
suggesting that they may have failed to detect that the PRT had ruptured.
=
Cognitive Performance that were telatively imMausible. Of these, three evaluating potential mitigating actions based on (Crews E, J, and b) eventually revised their situation assessment.
hypotheses to a more plausible explanation. In all three cases the crew member who suggested the more Two plausible hypotheses could explain the flow of plausible hypothesis was different from the crew RCS water into the RHR system.23 One hypothbsis is member who suggested the original hypothesis. This that the two isolation valves between the RCS hot leg result provides suggestive evidence of the positive loop and the suction side of the RHR system are open.
contribution of multiple crew members in situation Given this hypothesis, the leak into the RHR could be assessment and problem-solving. It is consistent with terminated by closing these valves, Seven of the 11 the argument that has been made by several crews mentioned the possibility of these valves being researchers (e.g., Reason,1990) that an incorrect open, but only two crews (Crews D and H) checked situation assessment, is more likely to be corrected by that the valves were open. One of these crews (Crew someone new on the scene than by the person who H) checked the valves as part of following the generated the incorrect hypothesis in the first place.
ISLOCA procedure. In most cases, while the crews considered the possibility that RCS fluid could be coming in through these valves, they rejected it as Response Planning: Accessing additional resources implausible because the valves were supposed to be closed and de-energized.
Since the LOCA procedure provided little direct guidance in localizing and isolating the ISLOCA leak, Given the hypothesis of back leakage via a series of this case provided a concrete example of a situation check valves, a second plausible action for where response planning on the part of crew terminating the ISLOCA is to isolate the RHR train members was required to localize and isolate the leak.
from the cold leg side of the RCS by closing a valve on We examined what actions crews decided to take the discharge side of the RHR pump that isolates the based on their situation assessment, and what RHR system from the cold leg loops.24This valve is resources they drew on to support identification and nonnally open. One of the actions called out in the evaluation of candidate actions.
ISLOCA procedure is to close this valve and check if that terminates the leak. Table 3.2a presents data on One potential resource was the ISLOCA procedure, whether crews considered closing this valve which provided step-by-step instruction for opening
(. consider Isolate RHR train'), which crew member and closing valves to locate and isolate the ISLOCA.
suggested it, and the basis for the suggestion. As can Four of the 11 crews decided to consu!t the ISLOCA be seen ten of the 11 crews considered closing this procedure for guidance. Note that these crews had t valve. A point to note is that in all cases the actively decide to consult the ISLOCA procedure.
suggestion to close the valve was based on a situation There was no explicit EOP transition to the ISLOCA procedure.
assessment rather than a particular procedure step.
Other resources accessed included schematic prints and alarm printouts. All11 crews consulted schematic prints to identify potential flow paths from the RCS system into the Auxiliary Building and the RHR system. In addition three of the 11 crews reviewed alarm printouts.
Response Planning: Identifying and evahtating extra-procedural response actions Since the LOCA procedure provided no specific e location of the isolat;on valves between the RCS hot leg
" side of the RilR system is shown m Figure guidance on the actions that should be taken, this n Ap nd A.
case provided a concrete example of identifying and 2%e location of the valve on the discharge side of the RilR pump is shown in Figure A-l. in Appendix A.
NUREG/CR4208 26
Cognitive Performance f
Table 3.2a ISLOCA 1. Consideration of RHR train isolation.25,26 i
Crew No.
Consider Isolating RHR Train Crew Member Reason Given i
A yes BOP check valve hypothesis B
yes BOP check valve hypothesis D
yes RO check valve hypothesis F
yes STA,SS check valve hypothesis J
yes RO check valve hypothesis L
no n/a n/a M
yes SS RHR problem Table 3.2b ISLOCA 1. Decision to isolate RHR train.
Crew No.
Decision Reason Given A
delay no procedural guidance B
yes n/a C
yes n/a D
delay no procedural guidance E
yes n/a F
delay check train B operable G
delay use ISLOCA procedure. as guidance H
yes SO says OK since RHR pumps off J
yes SO checks with SS and Emergency Duty Officer L
n/a
.n/a M
yes can reopen valve if needed D SO= Supervising Operator; RO= Reactor Operator; BOP = Balance of Plant Operator; SS= Shift Supenisor; STA= Senior Technical Advisor; n/a = not applicable.
26"RilR problem" indicates that the crews recognized that there was a leak into the RHR but were not more specific with respect to the source of the leak; n/a = not applicable.
Cognitive Performance Response Planning: Etuluating consequences of extra-c) Whether alternative ways to achieve the functions proceduralactions the system is intended to support are available; Table 3.2b shows whether the crews decided to close d) Whether there are procedures available to the valve that isolates the RHR train, given that it was support taking the action they believe is suggested, and the basis for the decision. As can be appropriate -if so they may choose to use these seen, six crews decided to close the valve right away, as guidance.
while four crews decided to wait. The basis for the decisions are illuminating in that they reveal the cautiousness of the operators in taking an action not 3.1.4 A Case Where Operators Needed to explicitly called out in the EOPs. Of the six crews Determine Whether Plant Behavior 1
who decided to close the valve, three articulated the was the Result of Known Manual basis of their decision. Of those, the reasons provided Act. ions or a Plant Fault by two showed that they checked for potential negative consequences before taking the action. One crew said that the RHR pumps were off, so closing the in ISLOCA 1 a case arose that provided a specific valves would have no effect. The other crew said that example of a situation where operators need to they would be able to reopen the valve if needed. In engage in situation assessment in order to determine the case of the third crew, they checked with the SS how to respond to a particular procedure step. The and the Emergency Duty Officer (EDO)27 for c se ccurred at a step in the LOCA procedure that approval before taking the action. Similarly, of the asks whether pressure m all steam generators is four crews who delayed taking the action,in three
" stable or increasing." If SG pressures are not all cases, the reason given was that they had no explicit stable or increasing, the step directs the crews to procedural guidance to take the action. Of those one return to Step 1 of the procedure (see Figure 3.1).
crew waited until they got to the appropriate step in the ISLOCA procedure before taking the action. The The purpose of this step is to check for the possibility other two eventually decided to close the valve. The f a faulted SG, which would cause SG pressures to fourth crew (Crew F) decided to wait until they decrease. In fact, there was no faulted SG, but there checked that RHR train B was operable before they was a cooldown in progress, which also results in isolated train A.
steam generator pressures decreasing.
De decision faced by the crews with respect to Based on training and EOP background documents,
(
whether to isolate the RHR train, provides a concrete crews are instructed that if they are taking actions that I
example of crews identifying and evaluating a
""' pr ducing a cooldown they should consider a response action. These results show that the majority decrease in SG pressure to be " stable or increasing.
of the crews were cautious in taking actions that are D"S, this is a case where response to a procedure not explicitly called out in the procedures. While Step is based on situation assessment rather than many of the crews did decide to take the action if they literalinterpretation of the procedure step. In felt it could mitigate the problem, they considered determmmg how to respond to this procedure step several factors before taking the action:
the operators need to understand the intent behind j
the procedure step (i.e., its purpose is to check for the l
a) Whether the functions performed by the system Possibility of a faulted steam generator), determine l
were currently needed or a need could be whetlyer the plant behavior they are observing can be exP amed by known mfluences on the plant (i.e., their l
foreseen; own actions or those of automatic systems) or b) The reversibility of the action; whether there may be a faulted SG, and decide how to respond to the procedure step based on that situation assessment.
27 The EDO or Emergency Duty Officer is a manager at the plant who wears a beeper w hile on call. lie is notified and consulted in 1
cases of abnormal plant conditions.
Cognitive Performance SI Termination Loss of Reactor Procedure or Secondary Coolant Procedure (E-1)
+ Step 1 Stop allbut one CCP RCS pressure stable or increasing: NO Si should be reduced: YES Establish 60gptn Chargingpow isolate the BIT
- RCS pressure stable or increasing Pressurizer level continues to decrease: YES ~
- Pressurizer levelgreater than 4%
Steam generator pressure stableorincreasing: No (return to step 1)
Post LOCA Cooldown Checkif RCS cooldown and y
and Depressurization g
depressurization is Required: Yes Procedute Figure 3.1 EOP transitions between LOCA procedure (E-1) and SI Termination Procedure.
Response Planning: Identifying goals and intent behind literal reading of the procedure step.
procedure steps The interaction of one of the eraws (Crew G) when i
Six of the 11 crews were in the situation where steam they came to the steam generator pressure " stable or generator pressures were decreasing when they got to increasing" step provides a clear illurtration of the this step. Five of the six crews decided to consider the reasoning involved in determining how to respond to steam generator pressure behavior " stable or this procedure step. Below we provide an excerpt increasing" and go forward in the procedure rather from their protocol.
than return to step 1 as would be required from a 29 NUREG/CR-6208
Cognitive Performance (e.g., RCS temperature due to GI; the fact that the steam generators are being fed at the maximum rate)
Example of Crew Protocol Showing Role of and determine whether these are sufficient to account Situation Assessment for the observed steam generator pressure behavior.
1 In effect the step requires that they discriminate Crew G between a faulted steam generator and a decrease in steam generator pressure due to a cooldown.
7:48:20 step 10 A second point illustrated by this protocol is that the Pressure in all SG - BOP: Right now on a decreasing operators need to understand the intent of the trend all of them.
procedure step to know what evidence to seek and consider in deciding how to respond. The fact that 7:49:38 SO (to SS): OK, We've got a decision to make the crew understood the intent of the procedure step here on this step here - pressure in all generators is illustrated by the statement made by the SO that stable or increasing, if it is no, we are going to be in "we have no indication of a faulted steam generator at this do loop;if we can say yes to this we can go on, this point." This suggests that they knew the intent of stay in E-1 and we'll cooldown and depressurize, and the procedure step, and evaluated the possibility of a that's what I think we need to be doing.
faulted SG before responding.28 '
7:49:55 BOP: Right now they are decreasing.
Finally, the crew dialogue provides an example of a crew identifying a response goal based on the crew's SS: The definition of stable, are we controlling the situation assessment, and evaluating the procedure decrease, you are feeding three of them at maximum path with respect to achievement of this goal. This is rate.
illustrated by the early statement made by the SO to the SS: "If it is no we are going to be in this do loop; if 7:50:09 SO: Not only that but RCS temperature is also we can say yes to this we can go on, stay in E-1 and decreasing from the SI flow.
we'll cooldown and depressurize, and that's what I think we need to be doing." By this statement the SO SO: So I think the decrease in the steam generator reveals that he thinks they need to be moving toward pressure at this point is due to our feeding the a cooldown and depressurization, and that answering generators, and the cooldown of the RCS.
"yes" to the steam generator pressure " stable or increasing" question will allow them to get to that 7:50:32 SS: It's pretty much controlled, so it's stable.
point in the procedure more directly.
SO: We have no indication of a faulted steam generator, that is the point.
3.1.5 Illustrative Protocol of Crew
""C" 7:50:45 SS: I would call that stable based on the parameters, since you are injecting 200,000 into it.
While the analysis above provides an overview of the perf rmance f all the crews, and quantitative 7:50:56 SO: How about RO and BOP, do you guys evidence of crews engaging in situation assessment agree with that?
and response planning, the analysis does not fully Capture the extent of cognitive activity and crew Both say OK.
dynamics observed in the scenarios. In this section we trace the performance of a crew that exhibited good situation assessment and response planning, to illustrate the types of situations that can arise in i
l This protocol excerpt illustrates several points. First, emergency events that require cognitive activity, and to determme how to respond to this step, the operators must actively consider the various known factors that are influencing steam generator pressure 28A faulted steam generator is equivalent to a steam line break.
I NUREG/CR-6208 30
Cognitive Performance provide an example of good crew responses to those 11:15:55 BOP: Yea, I m thinking it might be this relief situations.
valve right here on the seal water return l
Below we present select portions of the protocol for 11:16:00 SO: OK, let me go through the immediate Crew J. This crew did not see the RHR discharge action steps, we'll let SS make his announcement and pressure high alarm at the beginning of the event, then we will pursue the leakage.
Further, they were not provided the clue of water spilling outside the RHR pump room. As a
"* When they get radiation sympto'ns in containtrwnt and consequence, they had to identify the RHR problem in the Auxiliary Building the SO shws evidence of with only the PRT symptoms as a clue. This crew considering the possibuity of a link between the two illustrates the use of schematic prints and alarm symptoms.'"
printouts to identify the leak into the RHR and attempt to isolate it. It also demonstrates the role of 11:20:55 SO: STA where do we have area rad high multiple crew members in generating the situation alcrm? Can you determine thati Now we have a high assessment. All crew members participated in the high alarm.
situation assessment and response planning. The SO explicitly solicits crew opinion and seeks consensus 11:21:00 STA: Sure, we have them all over before taking actions not explicitly called out in the containment, we have them all over the Auxiliary procedures.
Building - so it's in both places.
Text in italics shows our annotations of the protocol 11:21:14 SO: Two pieces of informationfantiounces to that provide interpretation and comments on crew group). We've got area rad hi and area m.i high high, performance. The symbol * ' indicates that large and we've got high area rad monitors all over segments of the crew dialogue have been omitted.
containment, and all over the Auxiliary Building, so when we go loc, king for this leak that is something to keep in mind.
Illustration of Crew Performance in ISLOCA 1 11:28:21 step 24 containment radiation is abnormal-Crew J so transitioning to E-1 loss of reactor and secondary coolant.
"* In this case the crew missed the RHR discharge pressure high alarm initially so they hate no direct
"* They had to actively attempt to identify Ihe source of evidence that the leakfrom the RCS is going into the RHR.
the leak into the Auxiliary Building when they got to step The instructor never provided them the clue Ihat the leak 12 in Ed - at Ihat point they show evidence ofIrying to in the Auxiliary Building was outside the RHR pump pursue the source of the leak into the PRT. *"
room. As a result the only clue they had of a problem in the RHR was the problem in the PRT."*
11:40:41 step 12 B - try to identify and isolate the
- leakagr, 11:12 Reactor Trip.
11:41:12 SO: Let's have a conference here for just a
"" While stillin the entry procedure (E-0), the crew minute. Look at where we are at and where we are showed evidence of beginning to think about possible going. We are down to the point where we need to sources of the leak into the PRT, although they took no evaluate plant status.. We've got abnormal action.*"
containment and Auxiliary Building radiation, which tells us that we need to verify and isolate the leakage 11:15:30 BOP: It looks to me like our source of leakage until we isolate the leakage. We need to try to is not the safeties or PORVs; however, it does look determine where we need to go to isolate the leakage.
like it is going to the PRT, cause the PRT is over 200 degrees, so I think it is one of the other sources.
11:41:55 SO: The one thing, let me finish two points, the one thing that came in, that is weighing heavy on 11:15:50 Some other relief valve?
my mind, is that the first alarm that came in is PRT 31 NUREG/CR-6208
Cognitive Performance temp. high, level high, something like that, and then SS: Where does that come in on the print?
we started losing RCS levels, so what I'm thinking,like RO pointed out initially, whatever Comes right in through those four valves here.
this leak is it is going into the PRT. We need to identify that source of leakage. Comments?
SS: I think that s a good idea. It's a suspected source; just go ahead and isolate it and see what happens.
11:43:15 *" At this point the crew brings out prints and eteryone is looking at them and participating in the
- " The next portion illustrates that the crew carefully discussion. They consider all the possible sources ofleak considers the potentialfar negatite consequences before into the Auxiliary INilding. Atfirst they dismiss the taking a control action that is not explicitly calledfor in I
RHR relief tulte possibility as implausible and consider the the procedure. It also illustrates the prac; ice ofsoliciting possibility of a reactor coolant pump seal leak off. When the opinion and attempting to get consensu< before taking an leak continues after they isolate the reactor coo l ant pump action that is not explicitly calledfor in the procedure. "*
seal leak off, they go to the alarm printsfor a possible clue.
It is at that point that they discorer the RHR discharge 11:46:25 SO: One question, what's the consequence of pressure high alarm, which leads them to identify the closing it?
problem in the RHR vystem. "*
BOP: We don't have reactor coolant pumps runnmg.
BOP: We know we're pumping RCS through the Auxiliary Building right?
So then where is the seal injection going?
11:43:18 BOP: The only thing that we haven't isolated It will go directly into the RCS.
that could be coming from the RCS, we didn't get a temperature rise on the PORVs and all that stuff it should not hurt anything initially. At the same time we got the temp. rise here, I think, and I've thought all along,is that we've got SO: Yes, close the leak off isolation valves.
leakoff here from reactor coolant pump sealleakoffs.
OK,I don't have a good indication of that back there
- " When the leak continues uffer they hate isolated the that I can tell, but ii we isolated reactor coolant pump pump sealleakoffs, they reconsider the possibility of a leak rcal leakoffs we would know in a matter of a few into the RHR."
seconds.
11:47:42 SO: Why did we rule out the RHR suction Why? Do you think that RCS pressure would start reliefs?
turning? Yea.
BOP: I'm not saying that should be ruled out. I'm just Letdown is isolated, RHR pump suctions those were saying its not as likely as something that is energized.
de-energized and shut so it shouldn't be that.
11:48:11 BOP: What is the (RCS) prcssure right now?
Excess letdown was isolated when we started. We had no indication that it opened.
805. Still decreasing.
The only thing left is the pump seat leakoffs.
11:48:24 So that wasn't it apparently.
"* The portion below illustrates taking an action
"* When theirfirst hypothesis protes incorrect they then specifically intended to test a hypothesis as to the source of searchfor another possible explanation. This leads them to the ISLOCA.*"
consider bringing out the alarm printout."*
This one is not isolated still. Why don't we go ahead RO and BOP are in back discussing the source of the leak; and do that?
SS and SO look at the prints.'"
SO: SS Do you agree?
11:49:59 BOP: There are other inputs to the PRT. Is there anything else we can isolate?
Cognitive Performance 11:53:40 SO: I have some information for you. The
relicf valve is right here (to RO and BOP who are at the board).
11:50:32 STA: So 3 ou are in the PRT prints, looking for the inputs.
11:53:45 RO: Here is a postulated solution. Here, the 9
check valves are leaking; RCS leaks back through the 11:50:35 SS: Basically, we have hit everything that RHR system pressurizes it, lifts the relief right here.
makes sense;it doesn't mean that there is not The only way you are going to stop that is to take the something else here, but excess letdown has been RHR system out of service by isolating the discharge.
isolated; RHR suctions de-energized, there is nothing we can do with those; this was just shut; normal That's not something you normally do. You are letdown is isolated.
disabling an ECCS system with a LOCA here.
11:51:05 SO: STA, what's the alarm printer show that We don't need the RHR pump right now because we came in first, before the trip?
are above the shutoff head.
STA: The alarm printer shows, PRT temp., PRT Now is the time to do it.
pressure, then it had pressurizer heaters on.
SO: Ixt me get concurrence.
11:51:35 SO: There were two enunciators that came in at the same time and then there was a third one that 11:54:29 SO (to SS): Alright we have one option came in that was over here somewhere.
available to us that we need to evaluate.
STA: The one that came off of the printer, RHR alpha SS: This is the relief that is lifting on us? These are discharge pressure high.
normally closed down. What kind of pressure do you have in the RHR system?
11:52:16 SO: Centlemen, let's re-group here for just a minute. I've got some information off the Alarm 11:55:50 SO (to SS): We can isolate this pump, tum it printer. The alarm printer says that the first alarm that off, shut these other - 8809 alpha, we are on the alpha came in prior to the event was RHR alpha discharge Pump, isolate these check valves, and try to see if that pressure high. That tells me, that would explain why is our source of leakage coming back this way and we had the rad problems in the Auxiliary Building, going out that relief valve.
and the problems in containment. It is obviously because we blew the rupture disk. I'm not sure 1 11:56:14 BOP: As far as the suction valves,if the understand how we did that, how we ended up with suction valves are leaking by, we are just screwed.
PRT problems.
STA: So you are going to shut 8809 alpha off the
- " At this point the BOP suggests the hypothesis of alpha RHR pump and shut the 8701 alpha going to backflow through leaking check tulves. '"
the loop hot leg, that's what we are trying to do.
11:52:50 BOP: Irakmg back through the RHR pump it should be shut.
and then blew down to the PRT through the suction relief.
STA: Yea it should be.
- " They consult prints.*"
"* Here again is an illustration of the SO attempting to RO: I think now is the time to do it beiore we are i
keep Ihe crew auure of his thinking and to seek consensus below the shutofihead oi the RHR pump. Because we for proposed actions. This segment also illustrates again the don't need it right now, we can shut it and see what fact that this crew makes sure there are no negative happens.
consequences ofcontemplated actions bcfore the actions are taken. "*
Cognitive Performance 11:56:53 SS: Let me get a hold of the EDO, tell him 3.1.6 Variability in Crew Perforrnance what we propose to do.
While the majority of the crews successfully 11:59:32 '"* Terminate the event."'
identified the leak into the RHR and took action in an attempt to isolate the leak, variability in perfofmance was observed. Variability was observed in identification of plant symptoms (e.g., the RHR alarm, The protocol illustrates severalimportant points the PRT rupture); identification of a problem in the about crew performance on the ISLOCA. First, it RHR; and decisions with regard to what actions to provides a clear case where identifying and isolating take in an attempt to isolate the leak.
the ISLOCA required active situation assessment and response planning on the part of the crew. Initially, The crews varied in the extent to which they pursued the only clue pointing to the RHR available to the symptoms and attempted to formulate a coherent crew was the PRT behavior. To correctly identify the explanation to account for all the symptoms observed.
source of the ISLOCA the crew had tc actively The contrast in performance of Crews J and L provide consider the sources of mput into the PRT. This led a case in point. For both crews the only symptoms them to consider the possibility of an RHR problem.
they had pointing to the RHR problem were the Similarly, active cognitive activity on the part of the abnormal symptoms in the PRT. One of the crews crew was required to identify actions that could (Crew J) was able to identify the problem in the RHR potentially isolate the leak.
by pursuing sources of input into the PRT, and then checking the alarm printout. The other crew never Second, the protocol provides clear evidence of the identified the problem in the RHR. As a result, they use of additional resources to support situation failed to take action that might have terminated the assessment and response planning. The crew used leak.
the schematic prints to identify inputs into the PRT, and to identify which valves to close in an attempt to The ability to identify the RHR problem was isolate the leak. In addition, the protocol illustrates important because it led the crews to identify actions the use of the alarm printout to identify symptoms that could potentially terminate the leak. The that were previously missed. By the time the crew majority of crews correctly recognized that the leak began to suspect an RHR problem, RHR discharge into the RHR could be through the RHR isolation pressure was no longer high. The alarm printout valves or through the check valves. The crews varied provided the only remaining record of symptoms in with respect to the extent to which they pursued those the RHR.
possibilities. While seven of the 11 crews considered the possibility of a leak through the RHR isolation Third, the protocol illustrates that the crew carefully valves only two crews called to have the valves re-checked for potential negative consequences before energized. In the simulated event we ran, re-taking an action that was not explicitly called for in energizing the valves made no difference because the the EOP.
valves were closed. Had one of the valves been open, re-energizing them would have enabled the crew to Fourth, the protocol illustraus the contribution of detect the open valve and terminate the leak by multiple crew members to situation assessment and closing it.
response planning. The SO in this crew provided a particularly good example of " openness" in crew interaction. He solicited opinion, and sought consensus before taking actions not explicitly called for in the EOP.
l Cognitive Performance l
3.2.1 Characteristics of Participating 3.2 ISLOCA 2: ISLOCA Into RHR Crews Leading to a Break in the RHR Heat Exchanger to the CCW Eleven Crews from Plant 2 participated in ISLOCA 2.
]
Two crews were eliminated from the data analysis in ISLOCA 2 we identified three cases where because three or more crew members were aware of operators had to engage in situation assessment and the event. Of the remaining nine crews, six were j
response planning in order to deal with the situation.
currently on shift, and three were composed of 1
administrative staff. Crew size ranged from four to j
The first case is the situation around which the five people. In one case (Crew 1), two of the crew ISLOCA 2 scenario was designed; a case where the members (SS and BOP) had prior knowledge of the procedure containing relevant guidance could not be event. Those individuals did not actively participate reached within the EOP transition network. The in situation assessment and response planning.
analysis focused on the extra-procedural activities the crews engaged in in order to identify and isolate the ISLOCA into the RHR and the leak into the CCW.
3.2.2 A Case Where the Procedure Containing Relevant Guidance The second case where a need for situation Could not be Reached Within the assessment was identified was a case that also arose EOP Transition Network in ISLOCA 1. There was a step in the LOCA procedure that asked whether pressure in all the In ISLOCA 2 there were two leaks that the operators steam generators was stable or mereasing. Operators needed to identify and attempt to isolate: the leak into are instructed that if they are in a controlled the RHR from RCS, and the leak from the RHR heat cooldown they should consider a decrease in steam exchanger into the CCW system.
generator pressure to be ' stable or increasing. This is a case where operators need to determine whether the In contrast to the EOPs of Plant 1, the EOPs of Plant 2 plant behavmr is the result of known manual and/or did contain a transition from the LOCA procedure to automatic actions or the result of a plant fault. As in the ISLOCA procedure. A transition can be made ISLOCA 1 the analysis focused on how crews based on radiation in the Auxiliary Building. So the responded in cases where steam generator pressures scenario was timed so that radiation in the Auxiliary were decreasing when they got to this step.
Building would not appear until after the crew had passed the transition step. Therefore, the crews in The third case was a situation where operators had to Plant 2 also had no direct access to procedural evaluate the appropriateness of a procedure path and guidance for identifying and isolating the ISLOCA.
take actwn to redirect the procedure path. In the case into the RHR and had to resort to extra-procedural of two of the crews that ran in ISLOCA 2, a situation acsivities to identify and isolate the leak.
arose where the EOPs had crews repeatedly loop between the LOCA procedure and the SI termination Because of the dynamics of the event, only one of the Procedure. 'the crews recognized that they needed t nine crews observed in this event (Crew 4) met the -
get out of this loop and get on to the Post-LOCA procedural criteria to transition to the ISLOCA Cooldown procedure, but there was no procedure-procedure from the LOCA procedure. The other eight
- driven way to do so. The analysis focused on how the crews had no procedural means of reaching the crews evaluated the appropriateness of the procedure ISLOCA procedure.
path, and what actions they took to redirect the Procedure path.
In the case of six of the crews the criteria to transition to the ISLOCA were not met when they reached the transition step in the LOCA procedure. Specifically, in the case of these six crews, the CCW surge tank had not yet overfilled by the time the crews got to the step in the LOCA procedure asking about radiation in 35' NUREG/CR-6208
Cognitive Performance the Auxiliary Building. As a result there was no into the RHR and the leak into the CCW. Specifically, l
radiation in the Auxiliary Building when the crews we examined:
got to that step; so the literal criteria for transitioning to the ISLOCA procedure were not met.29 Later, the Whether crews identified a problem in the RHR CCW Surge Tank did overfill, spilling radioactive system, and if so on what basis; fluid onto the floor of the Auxiliary Building. At that point the criteria for transitioning to the ISLOCA Whether crews recognized that the symptoms procedure were met but by then the crews had across systems (i.e., symptoms in containment, passed the relevant step in the LOCA procedure. The symptoms in the RHR, and symptoms in the EOP rules of usage provide no basis for retuming to CCW) all resulted from the same underlying that step.
fault, and if so on what basis; j
In the case of the remaining two crews they had no The actions that crews took to identify and isolate procedurally directed means of reaching the ISLOCA the ISLOCA, given that they could not reach the procedure because they never reached the transition relevant procedure within the EOP transition l
step. These crews transferred to the SI Termination network; procedure from the LOCA procedure before they got j
to the step that asked about radiation in the Auxiliary The actions that crews took to identify and isolate Building. From the SI Termination procedure they the source of the leak into the CCW, given that j
transferred directly to the Post-LOCA Cooldown and the only procedure advice available was in an Depressurization procedure. As a result these two OFN.
crews never reached a step that enabled a transition to the ISLOCA procedure. (See Figure 3.1 for an overview oi the procedure transitions.)
Situation Assessment: Recognizing that it tvas not a simple LOCA by the absence ofexpected symptoms or The performance of these eight crews provided the r>resence of unexpected symptoms.
opportunity to examine the role of situation assessment and response planning in guiding in the case of ISLOCA 2 the RHR discharge pressure operator performance in a case where the procedure high alarm was suppressed. As a result the primary containing relevant guidance could not be reached indicator of a problem in the RHR was missing. The within the EOP transition network.
remaining indicators of a problem in the RHR were indirect.
While the ISLOCA procedure contained guidance on isolating the leak into the RHR, there was no The first alarms that came in, low pressurizer procedure in the EOPs that explicitly addressed the Pressure and level, suggested a LOCA insib leak from the RHR into the CCW. In order to identify containment. The primary indicators that this was and isolate the leak into the CCW the operators had to not a simple LOCA inside containment were (1) either diagnose the source of the leak on their own, or abnormal activity in the PRT and its eventual rupture access the Off-Normal Procedure (OFN) for CCW and (2) the fact that containment symptoms that System Malfunction as guidance. This provided an would be expected in the case of a LOCA inside additional opportunity to examine how operators containment (i.e., increases in humidity and radiation identify and isolate a leak in a case where no explicit inside containment) were not present. We examined
(
procedural guidance was available, whether the crews noticed thet,e unexpected p; ant behaviors and whether that led them to identify a i
The analysis focused on the extra-procedural activities Problem in the RHR.
the crews engaged in to identify and isolate the leak Seven of the nine crews noticed the PRT rupture early in the event. One crew (Crew 7) did not mention
,A The simulated scenario was intended to be timed so that noticing the PRT rupture until 40 minutes after the Auxiliary Building radiation symptoms did not appear until after reactor tri. One crew (Crew 3) never verbally P
I the crews passed the step in E-! that checks for Auxiliary i
Building symptoms. This was the case for all but one of the crews.
communicated the PRT rupture. In the case of this latter crew, the SS indicated that he had noticed the 1
NUREG/CR-6208 36 i
Cognitive Performance PRT rupturing but he never mentioned it to the rest of The remaining four crews recognized that the event the crew. As a result at no point in the event did the was not a simple LOCA based on observation of plant l
SO realize that the PRT ruptured.
symptoms that were unexpected given a LOCA hypothesis. In three cases this was based on Five of the seven crews who noticed the PRT rupture observation of symptoms in the PRT. In the last case early checked for possible sources of input into the (Crew 10) it was based on observation of RHR PRT. This is an example of knowledge-driven symptoms, which in turn were found in the process of monitoring in search of an explanation for an searching for an explanation of the symptoms in the unexpected plant behavior. As will be shown, the PRT.
search for sources of input into the TRT led these crews to identify RHR symptoms that they otherwise Identification of unexpected plant behavior led the might not have noticed.
crews to search for an explanation. One crew searched for a potential steam generator problem.
Table 3.3 presents the point at which crews One crew called the Auxiliary Building operator to recognized that the event was not a simple LOCA search for a potentialleak. One crew specifically inside containment, who mentioned it, the reason suspected a leak from the RCS to the RHR via the given for that conclusion, and the action taken as a isolation valves on the suction side of the RHR and result. Five of the crews recognized early that the called to have them re-energized. These actions event was not a simple LOCA because of the absence anticipated later EOP steps.
. of symptoms they expected based on their situation assessment. These crews indicated that given the rate These results provide specific instances where of level and pressure drop in the pressurizer, they expectations guided operator performance.
expected to observe more symptoms in containment Recognizing that the event was not a simple LOCA (increasing pressure and humidity) than they saw, allowed the operators to realize that there was an All five of these crews concluded that the event was addit:onal problem that needed to be identified and not a simple LOCA before they got any positive solved.
evidence of 2. problem outside containment.
Table 3.3 ISLOCA 2. Crew recognition that event was not a simple LOCA.30 Crew No.
Crew Member Keason Given Action 1
ct not enough containment symptoms none 3
RO not enough containment symptoms none 4
SS not enough containment symptoms none 6
SO not enough containment symptoms watch for SG problem 7
SO not enough containment symptoms calls to search Aux. Building 8
SS PRT symptoms check sources 9
SS PRT symptoms re-energize isolation valves 10 BOP RHR symptoms none 11 ct PRT symptoms check sources 30.ct' ="cannot telL" It means that the information could not be determined from the videotape.
Cognitive Performance Situation Assessment: Hypothesizing the cause of the alarm they first thought to check the RCP thermal barriers. They thought of the RHR in the context of RHR symptoms what could be getting into the PRT. That led them to Since the RHR discharge pressure high alarm was detect that RHR discharge pressure was high and suppressed there were no salient cues of a problem in temperature was very high. This led them to se. arch the RHR. There were symptoms of an RHR problem for ways that fluid in the RHR could get to the CCW.
available in the control room; specifically, there were They identified the RHR heat exchanger as a RHR discharge pressure and temperature meters that possibility because it is the biggest interface between read abnormally high and vacillated in value as the the two systems. During this period they referred to RHR relief valve opened and closed. However, the schematic prints for inputs to the PRT and interfaces operators had no alarms or procedure steps to direct between the RHR and the CCW systems.
them to check those meters. Nevertheless, all of the crews eventually detected the RHR symptoms, and so Not all crews noticed the RHR symptoms in the identified a problem in the RHR.
search of an explanation for the symptoms in the PRT.
Crew 3, which had not detected the PRT symptoms, One of the most striking characteristics of noticed the RHR symptoms incidentally as they were performance in ISLOCA 2 is the degree to which the performing a later step in the EOP. One of the steps crews pursued potential sources of leaks into the PRT.
in the Post-LOCA Cooldown add Depressurization This search allowed them to identify abnormal RHR procedure had the crew stop the Bravo S1 pump. In symptoms that they would otherwise not have taking this action the BOP noticed the Bravo RHR observed. Seven of the crews detected the RHR discharge pressure was not behaving the way he symptoms early, as a result of checking for potential expected. He expected to see Bravo RHR pump sources of input into the PRT.
discharge pressure to go down. Instead it stayed at around 400 lb. This violated expectation led him to One of the crews during the debriefing provided an suspect a problem in the RHR. This aspect provides articulate description of the situation assessment and another illustration of the power of failed expectations knowledge-driven monitoring activities they engaged in guiding situation assessment.
in during the scenario. The crew indicated that they were aware of all the sources of input to the PRT and Table 3.4 shows what explanations the crews considered each in turn. One crew member said generated for the RHR symptoms. As can be seen, "Given the amount we were losing it just appeared to the failed isolation valves between the RCS hot leg be the RHR, it's the largest relief valve We didn't and the suction side of the RHR system were have monitoring for it, we had monitoring for the considered at least as often as the failed check valve others and they were fine." When they got the CCW hypothesis.
Tame 3.4 ISLOCA 2. Hypothesized explanation for RHR problem.
Crew No.
First Hypothesis Crew Member Revised Hypothesis 1
isolation valves et n/a 3
check valves SO n/a 4
check valves SS isolation valves 6
isolation valves BOP isolation or check valves 7
check valves SS isolation valves 8
ct n/a et 9
isolation valves SS isolation or check valves 10 check valves BOP isolation or check valves 11 isolation valves ct isolation or check valves NUREG/CR-6208 38
Cognitive Performance Situation Assessment: Recognizing that symptoms in the Situation Assessment: Explaining symptoms in the CCW CCW, PRT and Containment had a common source system One of the questions we examined was whether crews Table 3.5a presents the first hypothesis the crews recognized that the symptoms across systems all generated to explain the symptoms in the CCW resulted from the same underlying fault.
system, who generated it, and the reason given for that explanation. The first column indicates whether One aspect of this question is whether the crews the crews identified a problem in the RHR system recognized that the containment symptoms were before the CCW alarm came in. As can be seen there caused by the rupture of the PRT. Five of the seven was a difference in the hypotheses generated between crews who noticed the PRT rupture early explicitly the crews who had detected a problem in the RHR connected the containment symptoms to the rupture before the CCW alarm came in, and those who of the PRT, thus illustrating the role of situation detected the CCW alarm first. The three crews who assessment in explaining observed symptoms.
were not aware of a problem in the RHR first suspected a leak into the CCW from the service loop.
Another aspect is whether crews recognized that the In contrast, three of the six crews who already knew PRT symptoms were caused by the problem in the there was a problem in the RER, suspected a leak RHR. All eight crews who had observed the PRT into the CCW from the F.HR.
rupture correctly attributed the PRT symptoms to the release of fluid from the RHR via the relief valve to Table 3.5b shows whether that initial hypothesis was the PRT.
revised later, and if so, what the revised hypothesis was. Of the three crews who detected the CCW alarm A third aspect is whether crews recognized that the first, two later revised their explanation of the cause symptoms in the CCW were caused by the problem in of the problem in the CCW when they identified a the RHR. All the crews identified a problem in the problem in the RHR system. By the end of the event CCW when alarms indicated high radiation in the five of the nine crews correctly identified that the leak CCW. Of the nine crews, seven eventually into the CCW came from the RHR heat exchanger.
recognized that the problem in the CCW was due to a leak from the RHR system.
Table 3.5a ISLOCA 2. First hypothesis generated to explain the CCW problem.31 Crew No.
Noticed RHR Problem First Hypothesis Crew Member Reason Given Before CCW Problem 1
yes service loop ct et 3
no RCP thermal barriers SO CCW alarm 4
yes service loop SO symptom after swap CCW train
]
6 no service loop SO CCW alarm j
7 no service loop SS symptom after swap CCW train 8
yes et n/a n/a 9
yes RHR pump seal cooler STA RHR problem 10 yes RHR pump seal cooler et RHR problem 11 yes RHR pump sealcooler STA RHR problem 31 lhe OFN procedure has the crews trarsfer the service loop to the other CCW train. When the cr?ws swap the CCW train it causes radiation alarms in the second train. This led some crews to incorrectly conclude that the leak was rom the service loop.
Cognitive Performance Table 3.5b ISLOCA 2. Revised hypothesis to explain the CCW problem.
Crew No.
Revised Hypothesis Crew Member Reason Given 1
l 1
RHR heat exchanger.
ct ct 3
ct n/a n/a 4
no n/a n/a 6
RHR heat exchanger RO RHR symptoms 7
RHR pump seal cooler SS RHR symptoms 8
RHR heat exchanger SO OFN logic 9
RHR heat exchanger SS told by instructor i
10 RHR heat exchanger et et 11 service loop SO symptom after swap CCW train Response Planning: Employing additional resources to action, given that relevant guidance on isolating an idenhfy the source of theleaks ISLOCA could not be accessed via the EOP transition network.
The previous analysis focused on situation assessment i
activities. We also examined the actions that the crews All nine crews considered the possibility of a leak took in attempting to isolate the leaks into the RHR through the isolation valves between the hot leg of the and the CCW system. The analysis focused on the RCS and the suction side of the RHR pump and called additional resources they accessed to support them in the Auxiliary Building to have them re-energized. In identifying and isolating the leaks, the actions they all cases the decision to re-energize the isolation took, and the reasons for selecting those actions, valves was based on their situation assessment rather than an explicit procedure step. In one case (Crew 4)
At least three of the crews used the CCW OFN for the crew decided to wait for an explicit step in the guidance. In addition, four of the crews either ISLOCA procedure before performing this action.
transitioned to the ISLOCA procedure or used it as guidance. As mentioned earlier, only one crew (Crew Seven of the nine crews decided to isolate the RHR l
- 4) had radiation symptoms in the Auxiliary Building train. In two cases (Crews 4 and 6) the action was when they reached the step in the LOCA procedure performed as part of the ISLOCA procedure. In the that checked for Auxiliary Building radiation, which case of the other five crews the action was identified is the literal criterion for transferring to the ISLOCA based on their situation assessment.
l procedure. The other three crews (Crews 3,6, and 11) accessed the ISLOCA procedure for guidance based The second question was what actions did the crews on their own assessment of the situation. Finally, at decide to take in attempting to isolate the leak into the least six of the crews accessed schematic prints in CCW and what guided the identification of that I
attempting to identify and isolate the leaks in the response action, given that the only procedural CCW and the RHR systems.
guidance available was in an OFN procedure.
Specifically, we examined whether the crews considered isolating the RHR heat exchanger, which Response Planning: Attempting toiwlateleaksinto the was the source of the leak into the CCW and the basis RHR and the CCW for identifying that response action.
]
One question was what actions did the crews decide All of the crews considered the RHR heat exchanger 1
to take in attempting to isolate the leak into the RHR as a possible source of the leak. In three cases (Crews and what guided the identification of that response NUREG/CR-6208 40
l Cognihve Performance 3,7 and 8) the possibility of isolating the RHR heat In several instances the operators explicitly discussed exchanger was based on the CCW OFN.
whether the procedure path they were on would enable them to achieve important goals in a timely At least seven of the nine crews took action to isolate manner or whether they needed to take action to the RHR heat exchanger. One crew (Crew 3) decided redirect themselves in the procedure network.
against isolating the heat exchanger because the SO did not believe it could be the source of the leak.
Below we present segments of two protocols where the crews provided evidence that they were monitoring the procedure path they were on. Both 3.2.3 A Case Where Operators Needed to cases involve crews that transitioned from the LOCA Determine Whether Plant Behavior procedure (E-1) to the SI Termination procedure. In was the Result of Known Manual both cases the crews recognized that given the size and nature of the leak, a high priority goal was to Actions or a Plant Fault begin cooldown and depressurization. This step entails going to the Post-LOCA Cooldown and As in the case of ISLOCA 1, we examined how the Depressurization Procedure. In both cases crew crews responded to the step in the LOCA procedure members raised the question of whether the that asked whether steam generator pressures were procedure path would get them to the Post-LOCA stable or increasing. In the case of six of the nine Cooldown and Depressurization procedure in a crews steam generator pressure was decreasing when timely manner. They actively engaged in problem-they got to that step. Of those six crews, four decided solving to identify a procedure path that would get to consider the steam generator pressure behavior them from where they were to the Post-LOCA
' stable.' This is similar to the behavior observed in Cooldown procedure.
ISLOCA 1.
Figure 3.1 (see page 29) provides a diagram of the This case exemplifies a situation where operators EOP transitions among these three procedures. The based their decision on their situation as s :sment in Post-LOCA Cooldown and Depressurization order to move expeditiously through the procedures.
procedure can be reached from either E-1 or the SI In particular, in this situation crews needed t Termination procedure. As shown in Figure 3.1, in E-discriminate energy effects (e.g., cooldown caused by 1 there is a step that checks whether SI should be known influenced from mass effects (e.g., a faulted reduced. If the answer is yes the crews are steam generator)in order to know how to respond t transitioned to the Si Termination procedure. From the procedure step.
there,if pressurizer level continues to decrease, the crews are transitioned to the Post-LOCA Cooldown nd Depressurization procedure.
3.2.4 Cases Where Operators Evaluated and Redirected the Procedure Path In one of the protocol segments we present the crew was transitioned to the SI Termination procedure.
They recognized that they had a large enough leak Response Planning: Identifying Goals and Etuluating the that they needed to get to the Post-LOCA Cooldown l
Pmcedure Path procedure, but when they got to the step in the SI Termination procedure that checks for transition to Several of the protocols provided evidence that the the Post-LOCA Cooldown procedure, the literal crews in question were reasoning at two levels. They criteria for transition were not met. The question they l
were engaging in situation assessment and goal faced was how to proceed to the Post-LOCA identification. Simultaneously they were monitoring Cooldown procedure, given that they did not meet the procedures they were following to ensure that the the literal criteria for transition.
actions specified in the steps were appropriate to the situation as they perceived it, and that the procedure It is also possible to transition from the SI Termination path would result in achievement of plant goals.
procedure back to the LOCA procedure. In the SI Termination procedure there is a step that checks that RCS pressure is stable or increasing. If the answer is 41 NUREG/CR-6208 t
Cognitive Performance no the crews are transitioned back to step 1 in E-1.
BIT is just going to get us back to re-establish the The second protocol segment we present is a case BIT.32 where the crew was repeatedly transitioned between E-1 and the SI Termination procedure. The question The 50 acknowledges the RO's point, but argues that they they Iaced was how to get out oi this loop and move are on a correct procedure path. He indicates that where on to the Post-LOCA procedure.
they want to be is in the Post-LOCA Cooldown and Depressurization Procedure, and there is a procedure path that willget thern to that procedurefroin where they are.
Case 1: Redirecting the procedure path 10:20:50 SO: OK, right, We were increasing after that in the first case we present the crew transitioned td point.
Termination from the LOCA procedure (E-1) because RCS pressure was increasing and pressurizer level BOP: Just for a little while.
was greater than 4%. Once in the SI Termination procedure, however, they wondered whether they SO: Until we established the 60 gpm charging. Now were on an appropriate procedure path. They we'll get off of the BIT. We'll come over to see if we recognized that they needed to get to the Cooldown can stop SI. We're not going to be able to. We'll get and Depressurization procedure and discussed into ES-11 POST-LOCA COOLDOWN AND among themselves which procedure path would get DEPRESSURIZATION, and that's where we need to them there. 'Ihey decided to stay in the SI head right now, so that's what we are going to do.
Termination procedure with the intention of transitioning to the Post-LOCA procedure from there.
10:21:22 BOP: I understand.
However, when they got to the step in the SI Termination procedure that enabled a transition to the SO: Need to Stop the CCP then close the BIT inlet and Post-LOCA procedure, they found that they did not BIT outlet valves.
meet the literal criteria for the transition. The SO decided to make a judgment call and transition to the RO: Pressurizer level is indeterminate right now.
Post-LOCA Cooldown procedure nevertheless.
They reach the procedure stm in the Si terrnination procedure that allows thern to transition to the Post-LOCA Evaluating and Redirecting the Procedure Path Cooldown and Depressurization procedure; howeier, Crew 10 pressurizer level is increasing slightly, so they do not rneet the literal criteriafor transitioning to the Post-LOCA This segrnent begins in the Si terrnination procedure. When Cooldown and Depressurization procedure, which is a they got to the step in the Si Terrnination procedure that decrease in pressurizer level. At this point the SO asks about RCS pressure, RCS pressure was increasing so announces that he is rnaking a judginent call and they did not transition back to E-1. A little later RCS transitioning to the Post-LOCA procedure (ES-11) in spite press are turned back and started to decrease. At this point of thefact that they do not ineet the literal transition the RO asks the crew whether rnaybe they should have criteria.
transitioned back to E-1 since they still have a LOCA in progress.
10:23:00 SO: Take a minute here to evaluate everything gentlemen.
i 10:20:25 RO: One question gentlemen,I'd like to bring up maybe as a point. We just stepped by the step a Pressurizer level is slowly indicating an increase.
little bit ago. Maybe we didn't wait long enough here for stable and increasing. We are in a loss of reactor Very slow decrease on RCS pressure.
coolant right now and would E-1 not be a good place to be? I mean if we are stabilizing right here with BIT 1 have a trend here on the computer of increasing flow going and everything else, we can pretty much levelin the pressurizer.
just figure that we're losing a lot of volume still somewhere, probably to containment, so isolating the 32BIT refers to Boron Injection Tank.
l Cognitive Performance l
is decreasing. As a result the crew is transitioned 10:24:05 SO: I'm going to have to make a judgment back to step 1 of the LOCA procedure. Because I
call, we're going to ES-11.. Doing it on max. charging pressurizer level is less than 4% they turn the CCPs isn't the way to be going. Re-align through the BIT back on. When they again get to the SI termination and go to ES-11.
criteria step in the LOCA procedure, pressurizer pressure is again increasing slightly, and the EOP has them transition to the Si termination procedure for the second time. Because the flow in from the CCPs
'Ihis protocol segment illustrates several points. First, just keeps up with the flow out from the leak, the it illustrates openness in interaction among crew crew could be kept in this loop between the LOCA members. In this case the RO raised a concern with and the SI termination procedure indefinitely. The regard to appropriateness of procedure path. Second, STA recognizes this problem, and searches for a way it illustrates evaluation of the current procedure path to get out of the procedure loop and get to the in light of the crew's perception of appropriate goals, Cooldown and Depressurization procedure.33 and redirection of the procedure path is judged appropriate. When the RO raised his wncern, the SO affirmed that there was a procedure path that would allow them to get from the S1 Termination procedure Understanding the Transition Logic Among EOPs to the Post-LOCA Cooldown procedure. Later, when Crew 9 it looked like they might miss the transition to the Post-LOCA Cooldown procedure because they did 9:57 '" This segment starts in the SI Termination not explicitly meet the transition criteria, the SO Procedure to which they have been transitionedfrom th 1
decided to make a judgment call and transition to the LOCA procedure (E-1)for the second time. '"
Post-LOCA Cooldown procedure in any case.
9:58 Stop alpha centrifuge charging pump.
Case 2: Understanding the transition logic among EOPs SO: If we get less than 4% on the pressurizer, we'll The next protocol segment illustrates the importance of understanding the transition logic among EOP procedures. This crew gets in a procedure loop that keeps them from getting to the Cooldown and 33 Depressurization procedure. The STA recognizes that 1n this case the STA suggestion for how to get out of the loop a high priority goal is to get to the Cooldown and between the LOCA procedure (E-1) and the F; terminauoa P***d " ' ""'"
- C # d "' # "'P'*""
DePressurization Procedure; he activel en8a8es in Y
pressure will not be increasing when th y Fet to the step in E.1 problem solving to identify a way to get cut of the that directs a transition to the SI termination procedure based en procedure loop.
increasing pressuriier pressure. An expert in the use of the Westinghouse ERG has suggested that a preferable resolution The step in the LOCA procedure intended to w' uld have been to turn on the CCPs as directed by E-1, but then etma the trans nt the SI te naq n pmcedure n the determine whether 51 should be reduced asks second pass even though the hteral critena for transition were met.
whether pressurizer pressure is stable or increasing-The argument made is that once the crew transitions back to E-1 In the case of this crew, pressurizer pressure was from the Si termination procedure, it has been determined and increasing slightly when they reached that step. This should be understood that the reason for the transition back to E-1 meets the criteria for SI termination so the crew was a small LOCA that does not meet the Si termination criteria
- I'*
'*"""*""P"**""'
" "N "'
transitioned to the Si Termination Procedure. That entered. The crew would then eventually get to the step m E-1 that procedure has them turn off all but one Centrifugal would transition them to the Post-LOCA Cooldown and Charging Pump (CCP). Turning off the CCPs results Depressurization procedure. Note that this alternative resolution-in pressurizer pressure and level going down. As a also depends on a) an accurate situation assessment; b) a deep result, when the crew gets to the step in the SI understanding of the rationale behind procedure steps and Pmcedum transin n1 Sic; and c) a deviation from the literal Termination Procedure that asks whether Pressurizer procedure step criteria for transition :o the SI termmation f
pressure is stable or increasing, pressurizer pressure procedure.
l l
43 NUREG/CR-6208 i
Cognitive Performance STA: Well, lets think about it. We could leave it off with the strategy that will allow them to get past a and then transition back to E-1 and get farther in E-1 procedure step that is putting them in a loop.
this time. We're going to have to get cooldown Specifically, he suggests that they not turn on the anyway so we're never going to get there this way.
CCPs so that pressurizer pressure would be There's plenty of subcooling. I'd leave it off and decreasing when they get to the procedure step that transition back to E-1. That's a recommendation.
checks for Si termination criteria.
You'll get farther in E-1 next time.
This protocol segment points out that cases can arise
'" STA and SO look at EOPs together ""
where crews need to engage in reasoning about the STA to SO: You're not going to go back to step 1 procedure logic, and how best to respond to (referring to a step in E-1 that would loop back to step I if procedure steps to get to the point in the procedure RCS pressure is increasing or SG pressure is decreasing).
where they need to be. It is reasonable to assume that You're going to go on through here. This will get you this case is net unique and that similar situations to POST-LOCA COOLDOWN AND requiring active monitoring of the procedure path will DEPIESSURIZATION. That's where we want to be, arise in emergency events. This analysis suggests that it is important that crews reason at two levels. ney STA: I'd say leave the CCP off and let E-1 get you to need to engage in situation assessment and goal ES-11 (the Post-LOCA procedure).
identification and they need to reason about the strategies underlying the EOPs, and the EOP
"* SO and SS review the LOCA and SI Terrnination transition network logic in order to assess whether the procedures tofigure out how to get out of the continuous current procedure path they are on will enable them loop - they eventually agree to do what the STA suggests.
to achieve plant goals in a timely manner, or whether l
they need to take action to redirect themselves within the EOP network. The implications of this conclusion for training are discussed in Section 5.
l This protocol segment provides a concrete example of a case where a crew understood the logic of the EOP Other cases where crews evaluated and redirected the transition network, actively monitored whether Procedure path adequate progress was being made toward high priority goals, and when it was determined that the Other examples exist of crews monitoring procedure current procedure path was unproductive, actively steps for appropriateness to the situation. In some engaged in problem-solving to identify a way to get cases a given procedure step was judged to be on a more appropriate procedure path, while still inappropriate given the particular state of the plant.
staying within the EOP framework.
For example, in one case a crew (Crew 6) reached a step in the LOCA procedure that said " Establish CCW ne discussion between the STA, the 50, and the SS flow to RCP thermal barriers." At the point the crew reveals that they have an accurate situation got to this step they knew they had RCS fluid leaking assessment. They understand that the leak is just into the CCW but had not yet identified the source of l
barely being compensated by the CCPs. They also the leak. The SO decided that given the problem in understand that a primary goal given the situation is the CCW it would not be appropriate to establish to get to cooldown and depressurization CCW flow to the RCP thermal barriers and does not expeditiously. Finally, they understand the structure take this action. SO: 'Tm trying to think here.They of procedure transitions among the LOCA procedure, want us to restore CCW to the reactor coolant pump the SI Termination procedure, and the Post-LOCA thermal barriers as part of this step. I'm not sure it Cooldown and Depressurization procedure. When would be advisable at this time."
they realize that they are in a procedure loop that is keeping them from getting to the Post-LOCA In another case the crew identified a situation where Cooldown and Depressurization procedure, they the actions they had taken to deal with one problem actively engage in problem-solving to identify a way prevented them from accomplishing a step in the to get out of the loop. In this case, the STA comes up EOPs. In this case the crew (Crew 7) had divided into two subgroups with the SO and RO continuing with NUREG/CR-6208 44
)
l Cognitive Performance the EOP to get to cooldown and depressurization, and dialogue they engaged in is presented in the protocol l
the SS and DOP using the CCW OFN to try and segment below.
identify and isolate the leak into the CCW. At some point the 50 reached a step in the Post-LOCA Cooldown and Depressurization procedure that asked to establish CCW flow to RCS at some target Catching Errors value. This was not possible, however, because the Crew 4 SS had isolated the CCW service loop as part of his attempts to identify and isolate the leak in the CCW.
13:30:31 SO: SS,I'm going to deviate slightly here At this point the SS who had been closely from the approved method af using these procedures coordinating with the SO says "We isolated that (the because if I answer this question correctly, I go to the CCW service loop) when we started to encounter our RNO column which sends me to tube rupture.
CCW problem. I think we pretty well determined it is on the Bravo Safety loop. Why don't we go ahead and 13:30:45 SO: This step right here says high radiation restore service loop alpha and get CCW back?"
from any SG steamline radiation monitor (Step 24 d).
The answer is no. That puts me over here. I don't l
This case provides an example of a situation where a want to go there,I want to go on.
crew needed to understand the effect of plant state on the ability to perform procedure steps, the goals to be SS: I understand, and approve.
accomplished by the procedure step, and how to achieve these goals given the current plant state. In this particular case, the crew had to determine that it The protocol segment above provides an example was possible to place the CCW in service in spite of where the crew monitored the appropriateness of the leak in the CCW system, and to reconfigure the procedure steps based on their own situation CCW system to allow this (i.e., switch to the A train assessment and goalidentification. In this particular CCW).
case the SO knew that there was no evidence of a tube rupture present and that it was inappropriate to transition to the steam generator tube rupture Catching errors procedure. This allowed him to detect a small error in the procedure, and obtain concurrence from the SS to A final example of crews monitoring and evaluating deviate from the literal statement of the procedure the appropriateness of procedure steps, illustrates the 34 step 1
role of situation assessment and response planning in catching errors. In this case a crew (Crew 4) caught a small error in the logic of a procedure step in the 3.2.5 Variability in Crew Performance Reactor Trip or Si procedure. As part of a step to check for a steam generator rupture, one of the While most crews succeeded in identifying the source substeps read "High radiatjon from any Steam Generator steamline radiation momtor. According t of leaks into the IUIR and the CCW and in identifying the EOP two column format,if that criten,on is nol the correct response while attempting to isolate the met, the operator is directed to the Response Not source of the leak, there was variability in performance across crews. Some crews identified the Obtamed (RNO) column which says to go to the RHR problem sooner in the event than others. This Steam Generator Tube Rupture procedure.
enabled them to quickly recognize that the leak into the CCW was from the RHR, to localize the leak to the i
In this case the operator knew that having no steam RHR heat exchanger, and to take action to isolate the generator steam line radiation moni or alarms was not an indicator of a steam geneaur tube ru; *ure RHR heat exchanger.
and that it would not be appropriate to transition to the Steam Generator Tube Rupture procedure. He consulted with the SS who concurred, and they 34 The error in the wording of the EOP step has since been decided not to transition to that procedure. The corrected.
Cognitive Performance One crew (Crew 3) had particular difficulty in in the procedure they were following. In all three identifying and isolating the leaks. This crew cases the crews recognized their error within two exhibited poor communication, and an inability to steps. The process by which the crews recognized the engage in systematic diagnostic activity to identify mistake they made and took action to get back on the and isolate the leaks. In particular, while at least one right procedure path provided an example of how l
member of the crew noticed the PRT rupturing, that response plan monitoring allows operators to catch information was not communicated to the SO. As a and correct their own errors.
result the crew did not identify a problem in the RHR l
until late in the event, and then only because the RO l
incidentally noticed abnormal RHR behavior while 3.3.1 Characteristics of Participating performing a procedure step. The crew accessed the Crews l
CCW OFN procedure but failed to follow the l
procedure systematically in order to identify and Ten crews performed the LHS 1 scenario. Of these, l
isolate the leak. One reason was that the SO seemed two were eliminated from the analysis because an to incorrectly equate the fact that they had been told inadvertent SI occurred during the RCS that the Bravo RHR train was out of service with it depressurization early in the event. Of the remaining being already isolated. As a result, the SO believed eight crews four were staff crews and four were crews the RHR heat exchanger was already isolated. This currently on shift. Crew size ranged from four to five confusion was never corrected by any other member individuals. Two of the staff crews (Crews B and D) of the group. As a result, the crew never attempted t ncluded one individual who had prior knowledge of isolate the Bravo RHR heat exchanger in spite of the the event. These were training instructors, who filled fact that the CCW OFN explicitly includes a step t the role of STA or SS. These individuals did not offer isolate the RHR heat exchanger.
op nions or participate in situation assessments or response evaluations.
3.3 Loss of Heat Sink 1: Total Loss 3.3.2 A Case Where Operators Needed to of Seconda Heat Sink Determine Whether Plant Behavior (Feedwater Never Recovered) was the Result of Known Manual and/or Automatic Actions or the In the Loss of Heat Sink 1 scenario we identified three g pg cases where operators had to engage m situation assessment and response planning to deal with the One of the questions we examined was the ability of situation.
the crews to identify the presence of a leak on the Two of these cases had been explicitly designed into Primary side (i.e., the leaking pressurizer PORV),
the scenario. These were: (1) a case where operators given that the focus of the procedures and operator attention was on the loss of heat sink event on the needed to discriminate plant behavior that was the result of known factors (i.e., an operator induced secondary side of the plant.
cooldown) from plant behavior that signaled an additional plant fault and (2) a case where operators To identify the leak the crews had to recognize that had to decide whether to manually initiate a safety the behavior on the primary side could not be system based on consideration and balancing of explained by the known factors influencing plant l
multiple goals related to safety.
behavior. The early symptoms of the leaking pressurizer PORV, a decrease in pressurizer pressure The third case was an example of a situation where and level, could be attributed to a cooldown resulting operators had to evaluate the appropriateness of a from the actions the operators were taking on the procedure path. We observed three cases where sec ndary side of the plant. Later symptoms crews got to a step in the EOP that called for a (Pressurizer level going up while pressure continued transition to a different procedure, but failed to make to go down, a bubble forming in the reactor vessel, the transition. Instead, they continued with the steps and activity in the PRT) could not be accounted for NUREG/CR-6208 46
i 1
Cognitive Performance by a cooldown. These symptoms,in combination, observation of RVLIS was based on knowledge-l pointed to a steam spaceleak in the pressurizer. A driven monitoring. Similarly, there were no direct steam space leak refers to steam leaking from the indications of a bubble in the head of the reactor pressurizer. Examples of steam space leaks are vessel. This situation assessment required an leaking pressurizer PORVs and leaking pressurizer inference based on loss of subcooling and/or the safety valves.
observation of pressurizer level going up, while RVLIS was going down. Crew H provided no To identify the steam space leak, the crews had to evidence of having detected the loss of subcooling, recognize that the symptoms on the primary side, checked RVLIS, or deduced a bubble in the reactor specifically pressurizer level going up while pressure vessel.
continued to go down and the activity in the PRT, could not be explained as the result of known manual in at least four cases RVLIS level went below 90%. In or automatic actions. The procedures did not provide the case of Crew H it went as low as 76%. In no case any guidance in identifying the steam space leak.
was the criterion for the core cooling safety function
" red path" reached (40% RVLIS).
The analysis examined:
All the crews identified a problem in the PRT either Whether crews detected the symptoms of the based on early symptoms (PRT pressure and leaking PORV; temperature), or when the PRT ruptured.
How they explained the early symptoms that could be accounted ior by the occurrences on the lltuation Assessment: Explaining observed symptoms secondary side; As described above, the majority of the crews noticed Whether the crews identified the steam space leak the symptoms providing evidence to a steam space on the primary side, leak. We next examined how the crews explained these symptoms, and whether they correctly identified a steam space leak on the primary side.
Situation Assessment: Detecting abnormalplant behavior We examined the point in the event when a crew We examined whether crews detected the RCS member first mentioned primary side plant behavior symptoms that provided evidence of a problem on the and the explanation given for the observation. In the primary side.
case of five of the eight crews the first comment was made by the RO and occurred early in the event when All the crews observed the pressurizer level going up.
the pressurizer level and pressure were decreasing.
In most cases (five of eight) comments on the At that point the crews were depressurizing the steam pressurizer level going up were first made by the RO, generators and a decrease in pressurizer pressure and who has the responsibility of monitoring and level was expected due to cooldown. In three of the controlling the primary side of the plant. With the five cases the crews explicitly attributed the observed exception of Crew D, that had closed the pressurizer decrease to a cooldown. This is an example of a PORV when the pressurizer level started to go up, situation where expectations derived from a situation the pressurizer either became full or approached it, assessment ( a decrease in pressurizer level and All but one of the crews (Crew H) commented on pressure due to cooldown) are used to explain
- this, observed symptoms.
Six of the eight crews explicitly commented on loss of In three cases (Crews C, D, and H) the crews did not subcooling. Six of the crews gave a verbalindication comment on the primary side plant behavior until the of checking reactor vessellevel by looking at the level in the pressurizer started to increase and/or the reactor vessellevelindication system (RVLIS). Seven subcooling limit was reached.
of the crews explicitly concluded that a bubble had formed in the reactor vessel. Since there were no alarms or procedural directives to check RVLIS, 47 NUREG/CR-6208
Cognitive Perfonnance Table 3.6. LHS I. Crew identification of a steam space leak.
Crew No.
Identified stearn spaceleak Crew Mernber Reason Given I
1 A
yes SO Level up, pressure down i
B no n/a n/a C
yes SO Level up, pressure down D
'E no n/a n/a i
no n/a n/a M
yes et Pressurizerlevel up Situation Assessinent: Identifying a problem-stearn observed to identily the steam space leak relatively spaceleak quickly. The second crew never identified the steam space leak.
Identifying a steam space leak required recognizing that some of the observed symptoms could not be explained by the iactors the operators knew to be Case 1: Seeking a single explanation to accountfor all the influencing plant behavior, and searching for an observed syrnptorns explanation that would account for these symptoms.
i The first crew actively sought a single explanation Table 3.6 shows which of the crews identified a steam that would account for all of the observed symptoms.
i space leak and what symptoms led them to that This led them to consider the hypothesis of a leaking j
assessment (the ' Reason Given' column). Some of the PORV and to decide to close the PORV block valve in crews explicitly used the tenn " steam space leak."
attempting to terminate the leak. As a result they
[
Others hypothesized a PORV leak in particular.
were able to terminate the leak before the pressurizer F
Those cases are indicated with the label "PORV Leak."
became full, and before the PRT ruptured, reducing Five of the eight crews were able to correctly diagnose the severity of the incident (i.e., no radioactive fluid a steam space leak from the symptoms observed. In spilled into containment).
all cases the identification of the steam space leak was based on identification of primary side symptoms that could not be explained in terms of the known factors influencing plant behavior. In three cases it was when Seeking a Single Explanation they observed pressurizer level going up while Crew M pressure was going down. In two cases it was when they observed PRT symptoms.
The protocol seginent starts at the point where the RO identsfies an unexpected behavior in the pressurizer. This While all the crews identified the main symptoms that leads the SS to identify a stearn space leak, pointed to a steam space leak (i.e., pressurizer level going up; a bubble in the reactor vessel; PRT 12:05:40 RO: My pressurizer level is screaming. We symptoms), not all the crews were able to integrate either just voided something, or something has just the evidence to identify the steam space leak.
happened.
We present protocol segments from two crews to SS: It's coming down?
illustrate the complexity of the situation assessment involved, and the variability in crew performance RO: No,it is screaming up.
observed. Both crews observed the same symptoms.
The first crew was able to integrate the evidence NUREG/CR-6208 48
Cognitive Performance 12:05:50 RO:Just lost subcnoling. That's why I was 12:10:49 SS: This PORV must have been leaking by.
concerned.
Looks like our pressure in the PRT stopped coming up.
SS: Maybe we sprung a leak.
- " The event tvas terminated shortly after. *"
It's possible.
It would be a leak in the precsurizer, right?
The protocol segment above provides evidence that 12:06:10 That's right; it would be steam right?
the crew used expectations to guide their monitoring of plant behavior, integrated evidence to identify a Yea.
steam space leak, and identified a response action that would terminate the leak, closing the block valve.
12:08:35 SS: For a leak anywhere it's got to be in the The protocol segment also illustrates that multiple steam space of the pressurizer.
crew members contributed to the situation assessment and response identification. In this case the SS, SO, SO: Or we are just swapping the bubble?
and RO were all participating.
We got pressure coming right on down.
Case 2: Recognizing an unexplained problem in the RCS 12:09:38 Pressure is decreasing and a bubble is forming.
The next protocol segment is of a crew that recognized there was an unexplained problem in the RO: That's why subcooling went away.
RCS, but did not identify the steam space leak. This protocol segment is presented to illustrate the
"* At this point the SS explicitly considers the possibility difficulty involved in making the situation of a leak through the PORV. The SO suggests closing the assessment. The crew noticed all the symptoms POR V block talte to test that hypothesis.*"
pointing to a steam space leak, but were unable to generate a single explanation that would connect all 12:09:50 SS: For pressurizer level to do that it's got to the symptoms. They postulated multiple faults to be in the pressurizer. Have we lifted a safety? How account for the set of symptoms, and never about a PORV7 considered the possibility of closing the PORV block valve to terminate the leak. As a result, plant RO: We haven't had a PORV lift, right? We may have conditions became more degraded in the case of this some leak by, but it is not significant.
crew as compared to the first crew.
12:10:05 SO: You have one of them armed, right? We can go back and block this and see what leaked by.
This valve can really be open for some reason.
Recognizing an Unexplained Problem in the RCS Crew E What's the PRT?
The protocol segment begins toith the RO alerting the SO 12:10:37 SO: It looks like a leak on the pressurizer that pressurizer level is up to 7S%. This quickly leads them with pressure decreasing and level coming up.
to detect that RCS pressure is decreasing, that they are losing subcooling, and that there is likely a bubble in the SS: It's a pressurizer leak. It's not going to head of the reactor tessel.
containment.
14:15:30 RO: Well, SO, we are about 75% on the RO: It's going into the PRT.
pressurizer now.
I 14:15:33 SO: Better get some letdown going.
Cognitive Performance 14:2450 SO: What's the vessel level doing here, STA, 14:15:38 RO: Probably voiding somewhere. We you were watching it?
couldn't have gone up to that high without voiding.
14:24:55 RO: We had 97 in the lowest one. Now we are down to 80.
SO: We are right at saturation.
14:15:54 RO:I got 23 on pumps on and 95 pumps off SO: The vessellevelis dropping?
(subcooling).
SO: Wide range pressure is below a 1000 on the RCS.
"* At this point the crew considers the possibility of a seal leak to explain containment symptoms. They show no 14:16:07 SO to STA: You are still monitoring that core evidence of trying to come up with a single connected cooling?
explanation to accountfor all the symptoms observed."*
14:16:17 STA: Yea,.
14:25:22 SO: Seal injection filter delta P, RO, they're bouncing.
"* At this point pressurizer becomesfull and pressurizer pressure is around 800 psig. The RO alerts the SO to this, 14:25:40 SO: You got a containment sump level high.
who suggests actions they can take to attempt to recover Have we blown a seal or something?
RCS pressure. *"
"* The event continued but the crew never identified the 14:17:24 RO: Pressurizer level is at 100% or greater.
steam spaceleak."*
14:17:32 RO: Pressure is a little above 800.
- " At this point the RO alerts the crew that the PRTis The contrast in performance of the two crews points about to rupture. While they notice the PRT symptoms, out the complexity of the situation assessment they do not integrate them with the symptoms in the involved, and illustrates the variability in crew pressuri:cr to conclude a steam space leak. This contrasts periormance observed. The first crew was able to with the performance ofCrew M. ""
identify the steam space leak and terminate it before the PRT ruptured. As a result they were able to l
14:18:28 RO: PRT pressure is screaming up.
reduce the severity of the incident. In contrast, the second crew, while observing all the relevant l
SO: From what source?
symptoms, was unable to connect the symptoms into l
a single coherent explanation. They did not identify
[
14:18:35 RO: I don't know but the pressure is now the steam space leak or attempt to take action to I
screaming back down. We obviously just blew the terminate leak. As a result conditions in the RCS rupture disk on the PRT.
continued to degrade. The pressurb.er became full, and the PRT ruptured, releasing radiation into 14:1951 SO: Any idea of what the source of water into containment.
the PRTis?
The fact that the first crew was able to terminate the 14:1959 RO: No, I'm not really missing any yet.
leak and reduce the severity of the incident provides an example of the positive role correct situation
- " The crew monitors R VLIS level and notes that it has assessment can play in mitigating incidents, decreased sigmficantly.*"
14:22:30 SO: What.. is wrong with the PRT? Relief valve some place?
Cognitive Performance 3.3.3 A Case Where Operators Had to The way LHS 1 was run the PORV leak was not Decide Whether to Manually Initiate necessarily terminated when the crew closed the PORV block valve. The leak was terminated in the a Safety System Based on case of Crew D. In the case of the other crews the leak Consideration and Balancing of was continued in order to examine whether the crew Multiple Goals would manually initiate St All the crews detected symptoms of RCS degradation, The only procedural guidance available to the.
and the majority identified the steam space leak on operators regarding manual initiation of SI was in a the primary side. The Loss of Heat Sink procedure caution that stated: " Following block of automatic SI provided no explicit guidance for dealing with the actuation, manual SI actuation may be required if -
leak. We examined the options the crews considered conditions degrade." This caution allows crews to for dealing with the leak on the primary side, and manually initiate Si at their discretion.
their decisions.
Manual initiation of SI would recover conditions in For the crews that considered the possibility of a the RCS but could result in a delay in recovery of the leaking PORV, a viable option for terminating the leak secondary heat sink or a need to resort to bleed and was to close the block valve. Table 3.7 shows which feed, which is a less desirable way to achieve a heat crews explicitly considered the possibility of a leaking sink.
pressurizer PORV, the reason given for this hypothesis, and whether they considered closing the if SI is not manually initiated, conditions in the RCS PORV block valve to isolate the leak. Five crews will continue to degrade. Eventually, reactor vessel hypothesized the possibility of a leaking PORV and level (RVLIS) would decrease to less than 40%. At considered the option of closing the PORV block that point, based on a core cooling critical safety valve. Four of the five crews decided to take this function criterion, the EOPs would direct the action. This decision turned out to be relatively simple operators to transition to a procedure designed to because closing the block valve could potentially respond to loss of core cooling. However, by that terminate the leak, and had minimal negative side point conditions in the RCS would be significantly effects. Only one of the crews (Crew A) decided degraded with increased risk of core damage.
against closing the block valve. The SO for this crew worried that if he closed the block valve he might not Crew performance was examined for evidence that be able to open it later if he needed it. None of the they recognized that they could manually initiate SI if other crews raised that concern.
they determined it was necessary, and for evidence that they considered the multiple goals that needed to be balanced in deciding whether to initiate SI.
Table 3.7 LHS 1. Decisions to close the PORV block valve.
Crew No.
Consider Leaking PORV Crew Member Reason Given Action A
yes SO PRT rupture Do not close block valve B-no n/a n/a n/a C
yes SO PRT rupture Close block valve
]
D yes BOP PRT symptoms Close block valve E
no n/a n/a n/a F
yes SS PRT rupture Close block valve H
no n/a n/a n/a
]
M yes SS Level up, pressure down Close block valve 51 NUREG/CR-6208
i j
- Cognitive Performance i
i Table 3.8 LilS 1. Crews that consiaeced initiating SIor going to a bleed and feed.
Crew No.
Consider Sior Bleed & Feed Crew Member Reason Given A
yes RO Containment symptoms; lost subcooling B
yes SO Containment symptoms C
yes SS Containment symptoms D
yes et Level up; bubble in head E
yes SO Cannot tell; checks procedure options F
no n/a n/a M
in the debriefing n/a In the debriefing: if PRT kept going up r
Table 3.8 shows whether crews explicitly considered manually initiate SL The last column in Table 3.9 the option of manually initiating Sl or going directly indicates whether the caution was read aloud at the to a bleed and feed (which would include a manual point when SI was blocked. As can be seen, only four SI). Six of the eight crews explicitly discussed the of the eight crews read the caution aloud. None of the possibility of initiating SI or going to a bleed and feed.
crews mentioned the caution in their discussions of A seventh crew (Crew M) indicated during the whether to manually safety inject or not. In addition, debriefing that they would have considered a manual when the caution was explicitly brought up by the SI, if RCS conditions continued to degrade after they instructor during the debriefing, the crews did not closed the PORV block valve.35 believe that it applied in this situation. In general, they did not interpret the caution as permission to Table 3.9 shows what decision the crews came to and initiate SI based on their own judgment.
the reason they gave for their decision. Of the seven crews who considered the possibility, five decided During the debriefing six of the eight crews indicated against it. Only one of the crews (Crew B) decided to that they believed they could not take any action to initiate SL A second crew (Crew M) indicated during deal with the degraded RCS conditions until they met the debriefing that they would have initiated S1 if an explicit procedure criterion. They interpreted the conditions continued to degrade, phrase " conditions degrade" in the caution to mean meeting some explicit procedure criterion that would Examination of the reasons given for deciding not to direct them to tum on SI; specifically, they believed initiate SI indicates that in the majority of cases the they had to wait until they either explicitly met the crews did not recognize that they were procedurally bleed and feed criteria, or met the criterion to allowed to initiate SL Four of the six crews which transition to the core cooling critical safety function explicitly considered the possibility of manually procedure (40% RVLIS). Since the event was initiating SI during the event decided against it terminated well before RVLIS reached 40% it is not i'
because they could not find any procedural guidance possible to know whether the crews would actually directing them to initiate 51.
have waited for reactor vessel level to decrease below 40% before manually initiating SL
'The caution that appeared in the Loss of Secondary Heat Sink procedure just before the SI signal was blocked was not considered in deciding whether to 35f n the case of Crew M the event was terminated shortly after they closed the PORV bkxL vahe. As a result they had no chance to observe funher degradation of conditions in the RCS.
.NUREG/CR-6208 52 i
Cognitive Performance 1
Table 3.9 LIIS 1. Crew decision regarding manualinitiation of SI.
Crew No.
Decision Reason Given Caution Read Aloud A
no No procedural guidance yes B
delay STA says wait yes C
no SO says wait no D
no No procedural guidance no No procedural guidance no E
no No procedural guidance F
no no H
n/a n/a yes M
n/a n/a yes i
The decision of whether to manually initiate SI is a reactor vessel and the pressurizer has gone solid. The crew complex decision that requires consideration oi monitors RVLIS level and notes that it has decreased multiple factors. It requires consideration of the sigmficantly."'
multiple goals to be achieved (the goal of terminating the LOCA, the goal of maintaining core cooling, and 14:24:50 SO: What's the vessel level doing here STA?
the goal of recovering a heat sink); alternative means You were watching it.
available for achieving those goals and their relative desirability (continuing to try to re-establish 14:24:55 RO: We had 97 in the lowest one, Now we feedwater vs. going to a bleed and feed); and the are down to 80.
potential costs in increased risk of delays in taking action (the costs of delaying mitigating the leak on the SO: The vessellevelis dropping? (yes).
primary side vs. the cost of delay in recovery of feedwater). The crews participating in this event provided little evidence of reasoning about goals in this way in deciding whether to initiate SI.
14:26:48 BOP: Where are we at? At RCS pressure?
While the crews expressed concem regarding the RO: 650 Narrow range and it's fluctuating.
degrading conditions in the primary system the main response was to search for an explicit procedure 14:26:57 SO to STA: You've still been monitoring that directive to initiate SI. Below, we present a protocol core cooling?
segment of a crew that showed the clearest evidence of considering the severity of conditions in the RCS in STA: Yea.
deciding whether to initiate SI. This crew, as the others did, searched for an explicit procedure
"* At this point the SO checks whether they meet the EOP criterion ior initiating SI. Fanling to find one, they criteriafor bleed andfeed, which is three steam generators concluded that their best option was to continue less than 24% They do not meet this criterion. The SO eiforts to recover icedwater.
checks the EOP, and concludes that he is procedurally
\\
bound to remain in a loop until the bleed andfeed criterion is met.*"
Determining Whether to Manually Initiate SI 14:27:15 SO: Still have wide range level in two of your Crew E generators?
"* This protocol segment begins late in the event, after the 14:27:22 BOP: I've got 20% in A and B, and 50% in PRT has ruptured, a bubble has formed in the head of the Charlie and Delta.
Cognitive Performance 14:27:30 SO: As far as I can tell, there is nothing that This protocol segment illustrates several points. First, jumps me beyond this step, wide range level less than the protocol segment provides evidence that the SO j
24% in three of the steam generators. It jumps you to was considering multiple goals in deciding how to step 10 (bleed andfeed step).
respond to the situation. The crew monitored core cooling and considered the state of conditions in the
- " At this point the RO retninds the SO that containtnent RCS as well as the heat sink problem in deciding how radiation is increasing.
to respond to the situation. Second, the protocol illustrates that the crew searched for explicit 14:29:17 RO: Containment radiation is going up. We procedural guidance with respect to initiating St. In have 1000 mr in the person hatch.
this case the SO concluded that he was procedurally bound to stay in the Loss of Heat Sink procedure until
- " At this point the SO provides evidence that he has feedwater was recovered, he met the bleed and feed considered optionsfor dealing with the une.xylained criteria, or he met the red path criteria for the core probiern in the RCS, and has decided that his best course of cooling safety function. Discussions during the action is to continue af ternpts to recoverfeedwater and debriefing support this interpretation.
postpone action with respect to the RCS problern. While he recognizes that conditions in the RCS have degraded considerably (RVLIS levelis less than 80%, and they have 3.3.4 Cases Where Evaluating the a " yellow" path on core cooling), he decides that conditions Procedute Path Enabled Operators to are not severe enough to take action. '"
Catch Their Own Errors 14:31:10 SO: It looks to me we are not having any In the Loss of Heat Sink event we observed three problem with the core right now. 'Ihere's something cases where crews got to a step in the EOP that called we can't account for over here but it is not causing for a transition to a different procedure, but failed to any possibility of a core melt or excess temp or make the transition. Instead, they continued to go on anything. We still have core cooling. We have a with the steps in the procedure. In all three cases the problem but it is not yet critical. The next most.
crews recognized their error within two steps. These important thing is get a heat sink. I'd rather stay in cases provided a concrete example of a situation l'ere and get that if there is any way of doing it.
where evaluating the procedure path enabled crews
~
"* At this point the 50 exhibits the snark of an irnportant crew interaction skill " openness." He polls the crewfor
.lhe three cases arose in the step in the E-0 procedure alternative opinions and seeks consensus before tabng that transitioned to the Reactor Trip Response "CN "
procedure if SI was not required. The procedure step first checked if SI was actuated. In the LHS event SI was not actuated. If SI was not actuated according to 14:31:40 SO: Are you in agreement with me? I'm open the EOP rules of usage, the crews were required to go to discussion on this. (They all agrec.)
to the RNO column in the procedure. The RNO column had the crews check if S1 was required. In the 14:31:46 STA: Still yellow on core cooling.
event we ran, SI was not required. In this case the EOP step directed the crews to transition to the 14:31:55 SO: OK, we are yellow on core cooling; we're Reactor Trip Response procedure.
green on subcriticality; we're still red on heat sink.
Since the same transition occurred in both LHS 1 and 14:32:10 RO: And we are at 1300 mr at the personnel LHS 2 we examined the performance of crews in both hatch' events. Of the ten crews that participated in LHS 1,36 14:32:15 SO: I still need you to keep looking;if you have a leak where it is at.
3hhe ten crews in LilS 1 included two crews that were dropped from the main analysis because an inadvertent S1 occurred during RCS depressurization.
Cognitive Performance eight correctly went to the RNO column and irom
"* When they get to step 6 that tenfies Containment there transitioned to the Reactor Trip Response Isolation Phase A (CISA), the RO recognizes that procedure. Two crews failed to go to the RNO Containment isolation Phase A is not appropriate to the column and continued to the next step in the E-0 situation and tells the SO who then realizes he is on the procedure. Of the ten crews that participated in LHS wrong procedure path. "*
2, one continued to the next step in E-0 instead of going to the RNO column.
10:12:40 step 6 -- Ensure CISA.
In all three cases the crews caught their error two RO: Wait a minute we don't have a CISA.
steps later when they got to a step that asked them to take an action that they knew was not appropriate to SO: Oh, I'm sorry, you are right, let me back up here.
the situation. This led them to recognize they were on the wrong procedure path, retrace their steps, and
"* At that point the SO backs up to step 4, goes to the find the point where they had missed the transition to RNO column, checks that Siis not required, and correctly the Reactor Trip Response procedure.
transitions to the Reactor Trip Response procedure.*"
Specifically, the crews realized they were on the wrong procedure path when they got to a step that asked them to verify Containment Isolation Phase A.
The three crews that missed the transition to the Containment Isolation Phase A is something that Reactor Trip Response procedure and then caught occurs automatically upon actuation of SI. The crews their error, provide examples of the role of situation knew that Si had not actuated, and they knew that assessment and response plan monitoring in catching Containment Isolation Phase A was not needed. This and correcting errors. In all three cases the crews led the crews to recognize that they were on the caught their error because they realized that the wrong procedure path. At that point the crews actions specified in the procedures were not moved back in the EOP and reread the Si actuation appropriate to the situation as they understood it, step. They recognized that they had failed to go to the This led them to realize they were on the wrong RNO column, corrected their error, and transitioned procedure path and to retrace their steps in search of to the Reactor Trip Response procedure.
the missed transition that would get them on the correct procedure path.
A protocol segment from one of the crews that missed the transition to the Reactor Trip response procedure and then reversed themselves (Crew J) is 3.3.5 Variability in Crew Performance provided below.
While most of the crews performed well in the LHS 1 event, there was variability in performance. All the crews detected the main plant symptoms that Catching an Error by Evaluating the Procedure Path indicated a steam space leak, but not all were able to Crew J integrate the evidence correctly. Only five of the eight crews c trectly identilied the steam space leak.
"* When theyget to step 4 that checksfor Siactuation, they correctly indicate that Si did not actuate, but instead One of the crews (Crew H) never realized there was a ofgoing to the RNO column they more on to step S that leak in the RCS. While this crew observed degrading venfiesfeedwater isolation."*
conditions in the RCS (e.g., a bubble forming in the core; pressurizer levelincreasing), they attributed 10:12:00 step 4 - SI is not actuated yet.
the behavior in the RCS to the fact that letdown was n t in service.37 In spite of the fact that conditions in 10:12:20 step 5 - Feedwater isolation.
37 While the fact that letdown was not in service might explaia an increase in pressurizer level, it could not explain the bubble forming in the core.
Cognitive Performance the RCS degraded considerably (RVLIS less than 80%;
of cognitive complexity because it included steps that RCS pressure down to 500; 50 degrees superheated),
were not appropriate to the situation.
they didn't realize that in addition to the fundamental, serious problem in the secondary side, The analysis identified two cases where operators had they also had a fundamental serious problem in the to engage in situation assessment and response primary side, i.e., the leaking pressurizer PORV.
planning in order to deal with the situation. The first case involved identifying and explaining the The variability in crew performance indicates that degrading conditions in the RCS. As in LHS 1 this identifying the leaking pressurizer PORV was entailed discriminating plant behavior that was the cognitively difficult. It required recognizing that the result of known factors (i.e., an operator induced primary side behavior could not be explained by cooldown) from plant behavior that signaled an known factors influencing the plant (e.g., net additional plant fault (i.e., the leaking pressurizer charging; the cooldown resulting from depressurizing PORV).
the steam generators), and searching for a coherent explanatien that would account for all the symptoms A difference between LHS 1 and LHS 2 is that in LHS observed. The crews were not all equally successful 2 if a crew decided to close the PORV block valve the in these activities.
leak was terminated. Eight of the 10 crews run in the LHS 2 scenario terminated the leak by closing the Differentiating expected from unexpected primary PORV block valve relatively early in the event. This side behavior requires having an accurate mental provided us the opportunity to examine crew model of the factors that influence primary side response to degrading RCS conditions in cases where behavior and the size and direction of effect of each of there was no leak as well as in cases where there was these factors. It also requires qualitative reasoning to a leak.
determine expected primary side behavior based on the net effect of the known influences affecting Even when there was no leak on the primary side, primary side behavior at the time. The fact that not RCS conditions degraded. RCS pressure went down all crews recognized that the degrading conditions in to less than 1830 psig, and pressurizer level went to the RCS were unexpected given the known factors less than 4% These decreases were due to the influencing the RCS suggests that there is room for cooldown caused first by the depressurization of the improving operator knowledge and skills in these steam generators, and later by the feeding of the areas.
steam generators through the condensate system. The fact that pressurizer level and pressure decreased to this extent just due to the cooldown made clear the cognitive difficulty faced by crews in both LHS 1 and 3.4 LOSS of Heat Sink 2: Total LOSS LHS 2 in distinguishing expected pressurizer of Secondary Heat Sink behavior explained by a cooldown from abnormal behavior that indicates a leak.
(Feedwater Recovered)
. case pmvided a concrete example of
- C "
As in the case of the LHS 1 scenario, LHS 2 involved where actions specified in a procedure are not a totalloss of feedwater flow complicated by a leaking aPPmPriate to the situation. The Reactor Tnp pressurizer PORV. There were two main differences Response procedure included several steps that were between the two scenarios. One difference is that if a p tentially inappropriate to follow literally given that crew decided to close the PORV block valve the the crews had just recovered feedwater using the pressurizer leak was terminated. This aspect c ndensate system. This mcluded steps,both in the provided the opportunity to examine crew response body of the procedure and on the foldout page, that to degrading RCS conditions in cases where there was specifi d that the operators should initiate SI. The no leak as well as in cases where there was a leak. A EOP background documents anticipated the j
second difference between the two scenarios is that in pm y f *ps being inappropriate and explicitly l
LHS 2 the crews eventually recovered feedwater. As ir
,ed that operator judgment may be needed 1
a result they transitioned to the Reactor Trip Response procedure. This procedure introduced a new source NUREG/CR-6208 56
Cognitive Performance under these circumstances.38 This situation allowed SO. In this case the focus of study was the situation us to examine operator performance in a case where assessment and decision-making activities of a single operators were required to evaluate the individual rather than a group.
appropriateness of procedure steps given the specifics of the situation and to modify the steps if judged 3.4.2 A Case Where Operators Needed to necessary.
Determine Whether Plant Behavior The analysis focused on whether crews chose to was the Result of Known Manual 1
deviate from the procedure steps that were judged t and/or Automatic Actions or the be inappropriate and the basis for thetr decision. We h M a PM M had the opportumty to examine crew response to procedure steps that called for manually initiating SI in both cases where there was no leak present in the Situation Assessment: Identifying a problem -leaking RCS and cases where there was a leak.
pressurizer PORV i
The analysis examined whether crews identified the 3.4.1 Characteristics of Participating leaking pressurizer PORV. Table 3.10 lists the crews that closed the block valve, the reasons they gave for i.
Crews closing it, and whether they explicitly suspected a leaking PORV when they closed the block valve. Of -
Ten crews participated in the Loss of Heat Sink 2 the ten crews who participated in the event, three scenario. Four of the crews were staff and six were crews closed the PORV block valve at the time that currently on shift. In the case of two of the staff crews they closed the pressurizer PORV when RCS was (Crews 1 and 5) one or more crew members were depressurized to less than 1920 psig.39 As a result, aware of the event. In Crew 1 the SS had prior these three crews never experienced a leaking PORV.
knowledge of the event.
Of the remaining seven crews, five crews suspected a In the case of Crew 5 three of the four crew members leaking pressurizer PORV and closed the block valve were " confederates" in that they were fully aware of as a n'sult. Three of these crews suspected a leaking the event. Only the SO did not know the event. This Pressunzer PORV because of a rapid decrease in crew was not dropped from the study because the pressunzer pressure. The other two crews suspected three confederates did an exceptionally good job of a leakmg pressurizer PORV when they got PRT providing the SO with the information on plant state
"""8' he would need to identify the leaking PORV, without integrating the information for him. As a result it Two of the crews (Crew 5 and Crew 11) never provided the opportumty to observe the situation suspected a leak in the PORV. As a result, the leak assessment and response evaluation activities of the was still present when they transitioned to the Reactor 38 The Users Guidefor the Westmghouse Owners Group Emergency Response Guidelines and Background Document explicitly addresses the type of situation created in IJIS 2. It states "After restoration of any Critical Safety Function from a RED or ORANGE condition, recovery actions may continue w hen the FRG is complete... Upon continuation of recovery actions, some judgment is required by the operator to avoid inadvertent reinstatement of a RED or ORANGE condition by undoing some critical step in a Function Restoration Guideline." (Westinghouse l
Owners Group Emergency Response Guidelines, Users Guidefor -
39 Emergency Response Guidelines and Background Documents, One possible reason that three oithe ten crews closed the block September 1,1983, pg.17.) The use of the phrase "some valve at the time the PORV was closed is that at this plant the judgment is required by the operator" suggests that the developers block valve is normally closed below 2185 psig. The operators i
of the EOPs recognire that in these circumstances operators need opened the block valve as part of the RCS depressurization, but to evaluate the appropriateness of c:rtain procedure steps based on then returned it back to its original configuration after the RCS their own situation assessment.
target pressure was reached.
57 NUREG/CR-6208 (c
Cognitive Performance Table 3.10 LHS 2. Whether crews closed the pressurizer PORV block valve and their reason for closing it.
Crew No.
Close Block Vahe Crew Member Reason Given Suspect PORV Leak 1
yes RO Pressure down yes 2
no n/a n/a no 6
yes RO Pressure down yes j
7 yes RO When close PORV n/a I
8 yes RO Pressure down yes j
9 yes BOP When close PORV n/a 10 yes BOP When close PORV n/a 11 no BOP n/a no Situation Assessment: Idenhfying unexpected plant 10:07:35 RO Co SO): RCS pressure.. we are getting behavior closer to the unacceptable region on the subcooling curve but with him dumping as much steam we The variability in performance across crews indicates should come back.. pretty quickly.
that identifying the leaking pressurizer PORV was cognitively challenging. It required discriminating 10:08:03 RO: Due to the cooldown my pressurizer the effects of a leak from effects of other known levelis decreasing.
influences on RCS behavior, such as the cooldown that resulted from activities on the secondary side.
SO: I understand.
The following protocol segment illustrates the 10:08:11 RO: 20% on pressurizer level, I don't think
' difficulty of discriminating expected from unexpected that's an uncontrolled decrease.
plant behavior in this event in this pmtocol segment a crew at first attributes the RCS behavior to a 10:08:47 RO: Just got less than 17%.
cooldown, when in fact there is also a leak present.
Eventually, they get alarms indicating a problem in 10:08:58 SS: It's shrinking you down.
the PRT from which they infer that the pressurizer PORV must be leaking. At that point they reassess the
"* At this point, while they continue to believe the RCS situation and conclude that the RCS symptoms they symptoms are due to a cooldown they begin to take actions have been observing were at least in part due to the to try and recover level. In this case they start a centrifugal leak.
charging pump. This has the effect ofintroducing yet anotherfactor influencing RCS behavior, making it more difficult to sort out all of the known influences affecting RCS and distinguish eapectedfrom unexpected RCS behavior. *"
Difficulty of Discriminating Effects of a Leak from Effects of a Cooldown 10:09:00 RO: Do you want me to start the CCP?
Crew 2 10:09:10 SO: Agrees. No sense taking pressurizer level The crew observes that pressurizer level is decreasing, RCS down and out of sight.
pressure is decreasing and subcooling is decreasing. Atfirst they attribute it to a cooldown.
i i
Cognitive Performance 10:09.59 RO: Taking manual control of PDP speed margin - was not very different from the RCS controller and the CCP flow control valve and I will symptoms exhibited in cases where there was no leak.
l secure the PDP.
The difficulty of the required discrimination was 10:10:00 SO: I understand..
compounded by the fact that the crews were constantly taking action that changed the pattern of l
- " At this point they get a PRTlevel high alarm tehich influences on the RCS. For example, they turned on leads the RO to suspect a steam space leak. and close the heaters, started pumps, and isolated letdown,in an pressurizer PORV block vahr. "'
effort to bring pressurizer level and pressure back up.
These actions increased the difficulty of predicting 10:12:10 RO: PRT level high (acknowledges alarm).
what the RCS behavior should be and detecting discrepancies.
10:12:16 RO: I'm going to close the pressurizer seal iso block valve for possible leakage pathway; that PRT Detecting the leak in the RCS required qualitative level has continued to increase.
reasoning comparing the expected decrease in RCS parameter values due to known ongoing influences
"* The SO reassesses the situation and concludes that the with the observed decrease. Since crews do not get pressurizer level decrease must have been in part much experience with attempts to provide feedwater attributable to the leak in the pressurizer. The RO agrees.
through the condensate system, they did not have much basis with which to predict the size of shrink to expect due to the cooldown caused by the rapid 10:12:27 SO: You mean that may be part of your level depressurization of the SGs and subsequent start of decrease?
feedwater via the condensate system.
RO: Yes.
The need to discriminate cooldown effects from effects due to a leak on the primary side is reinforced 10:14:39 RO: Pressurizer level has tumed and it is by the behavior of the crews once the leak was recovering. rm still putting in approx. 200 gpm terminated. Table 3.11 shows the first point at which charging.
crews considered manual initiation of SI in the scenario, the procedure they were in at the time, and the basis for their concem, whether they decided to initiate SI, and the basis for their decision. As can be his protocol segment illustrates that the crew was seen, seven of the crews were sufficiently concemed engaging in situation assessment. 'Ihey showed with primary side behavior that they explicitly i
evidence of maintaining a model of the factors considered manual Si at least once in the scenario. In influencing RCS behavior and using that to explain the case of six of these seven crews there was no leak observed RCS behavior. At first they explained RCS present.40 The degraded RCS conditions were due to behavior by the cooldown caused by activities on the a cooldown. After checking for additional evidence of secondary side. Only after they identified an a leak and discussions among the crew members, independent symptom that pointed to a pressurizer these crews eventually (correctly) decided that the leak (PRT level high) did they update their model of RCS behavior was explained by a cooldown and the factors influencing RCS behavior.
decided not to initiate SI.
The protocol also illustrates the difficulty of detecting The difficulty of the discrimination is highlighted by the influence of a small leak in RCS when there are a the fact that Crew 11 also decided against manually number of other influences on the RCS at the same initiating SI the first time the possibility was raised.
time. The behavior of the RCS exhibited at this point In their case there was a leak present, and initiating SI
- decrease in pressurizer level less than 17%, decrease would have been appropriate.
in pressurizer pressure, decrease in subcooling Dere was a leak in the case of Crew 11, 59 NUREG/CR-6208
Cognitive Performance Table 3.11 LHS 2. First point at which crews considered manual SI.
Crew No.
Procedure Crew Member Reason Given Decision Reason Given 1
RO Przr level <4%
no Levelinc.; SS says not to 2
n/a n/a n/a n/a n/
3 LHS RO Przr level down no SS says controlled cooldown S
n/a n/a n/a n/a n/a 6
n/a n/a n/a n/a n/a 7
Reactor Trip RO Przr level <4%
no Ope-ator induced 8
LHS RO Przrlevel down no Try to stabilize 9
Reactor Trip RO Przr pressure no Due to cooldown 10 Reactor Trip RO Przr pressure no Operator induced 11 Reactor Trip RO Przr pressure SS says not due to a leak no Response Planning: Reading and interpreting caution in 3.4.3 A Case Where Operators Were the Loss of Hect Sinkprocedure Required to Evaluate the We also examined whether crews read aloud the PPropriateness of Procedure Steps caution regarding manual activation of SI that Given the Specifics of the Situation appeared prior to the step in the Loss of Heat Sink procedure where SI signals were blocked. Nine of the Because feedwater was recovered in LHS 2, we were ten crews read the caution aloud. Two of these crews able to observe the crew's actions when they (Crew 6 and Crew 8) explicitly mentioned the caution transitioned to the Reactor Trip Response procedure l
when they had concerns about RCS conditions and and reached steps that seemed inappropriate to the l
were considering courses of action.
situation. Of the ten crews that participated in the event, one crew (Crew 2) had the scenario terminated The results regarding reading and interpreting the before they got to the reactor trip procedure, and a caution contrast with the results on LHS 1. In that second crew (Crew 8) manually initiated SI based on case operators indicated that they interpreted the foldout page criterion of pressurizer level less
" conditions degrade" as degrading severely enough to than 4% as soon as they entered the procedure.
meet explicit EOP transition criteria (i.e., bleed and feed criteria or RVLIS less than 40% ). In the case of This left eight crews that went through the Reactor LHS 2, which was run at a different plant, some of Trip procedure. Of those, six of the crews had the crews considered pressurizer level less than 17%
terminated the leak through the pressurizer PORV to be a degraded condition warranting consideration before they entered the Reactor Trip procedure and i
l of manual St The wide variability in interpretation of two crews still had a leak. This difference allowed us l
the phrase " conditions degrade" suggests that to examine how crews responded to steps in the j
clarification of the intent of that statement may be Reactor Trip procedure that called for an SI, both in i
required.
cases where there was no leak and an SI would be inappropriate, and in cases where a leak still existed l
and an S1 was required.
The decision of whether to initiate SI was important because either course of action potentially had negative <;onsequences if the crew's situation assessment turned out to be wrong. If they decided to NUREG/CR-6208 60
Cognitive Performance safety inject when it was not needed, it could place crews to SI if pressurizer pressure was lower than the crews in a loss of feedwater event again or at the 1830. A 'yes' in that column indicates that the crews minimum delay recovery. If they decided to not decided to omit the procedure step and not initiate SI.
initiate SI, and there was a leak in progress, The next column labeled 'who decides' indicates conditions would continue to degrade, with which crew member made the decision. A "SO w/SS" i
increased risk.
indicates that the decision was made by the SO and approved by the SS. 42 Response Planning: Omitting a procedure step considered As can be seen all seven oi the crews that reached that inappropriate to the situation step decided against manualinitiation of SI. In most cases the decision was not made unilaterally by the Table 3.12 shows how crews responded to the steps in SO, but involved input from other crew members and the Reactor Trip procedure that were not appropriate agreement by the SS.
given that feedwater was being provided through the condensate system. One step required that the While all crews came to the same decision regarding operators close the feedwater isolation valves. Crew whether to initiate SI based on pressurizer pressure response to that step are presented in the column less than 1830, the decision was not necessarily labeled " Omit Feed Isol. Valve Step" in Table 3.12. A correct in all cases. In the case of two of the crews
'yes'in that column indicates that the crew decided to (Crews 5 and 11), there was an ongoing leak in the omit the step and not close the isolation valve. Since PORV. The decrease in pressurizer pressure was not the crews had intentionally opened the feed isolation entirely due to a cooldown as they assumed. This valve as part of the Loss of Heat Sink procedure, and result re-emphasizes the need for crews to be able to closing that valve would result in a loss of feedwater distinguish effects of leaks from effects of cooldown, again, the crews had no difficulty deciding not to and the difficulty of making this discrimination.
close that valve. All eight crews decided to leave the feedwater isolation valve open. In most cases the In addition to explicit procedure steps, a foldout page decision was made by the SO with no discussion in the Reactor Trip procedure specifies criteria for required. Examples of explanations given by the SO manually initiating SI. Steps on the foldout page are for the decision are "We opened them intentionally, I intended to apply at all times and the actions i
am not going to close them again"(Crew 6); and "We specified are supposed to be taken as soon as the are not going to do that because that is how we are criteria are met. Table 3.13 shows which foldout page feeding the generators" (Crew 10). 41 SI criterion was met, whether there was a leaking PORV at the time, whether the crew decided to The second step that had to be adapted had operators initiate SI, who made the decision, and the reason check that pressurizer pressure was greater than 1830, given.
and initiate manual SI if pressurizer pressure was lower. Pressurizer pressure was less than 1830 in the
'Ihere was one SI criterion specified on the foldout case of all seven crews that reached that step. In the page that was met even in cases where there was no case of five of the crews this was primarily due to leak. This was the criterion of pressurizer levelless cooldown. However,in the case of two of the crews than 4%. In several cases pressurizer level reached there was a leak in the RCS that contributed to the zero strictly because of the on-going cooldown. Five low pressurizer pressure.
crews met the less than 4% level Si criterion. In none of these cases was there a leak. One crew (Crew 8)
Column 4 in Table 3.12 labeled " Omit Przr Press. step" immediately safety injected based on the foldout page shows crew performance on the step that required criterion. In the case of the other four crews, three concluded the level decrease was due to an operator induced cooldown and decided against SI. The fourth 41In this scenario none of the crews made the mistake of crew (Crew 9) never considered this SI criterion. The following a procedure step that undid an action that was taken earlier to restore a critical plant function. There is anecdotal evidence of an instance where a crew didjust that during a 42.S0 w/SS. STA' indicates that the decision was made by the 50 training exercise at a plant.
and approved by the SS and STA.
Cognitive Performance Table 3.12 LHS 2. Crew response to Reactor Trip procedure steps that required modification.
Crew No.
Omit Feed Isolation Valve Step Crew Member Omit Przr Press. Step Crew Member SO n/a n/a 1
yes 2
yes n/a yes SO w/SS 3
n/a n/a n/a n/a 5
n/a n/a n/a n/a 9
yes SO yes SO 10 yes SO yes SO w/SS,STA 11 yes SO yes SO w/SS Table 3.13 LHS 2. Foldout page Si criteria met and crew response.
Crew No.
51 Criterion Met I.rak initiate SI Crew Member Reason Given 1
level <4%
no no SO w/SS Levelinc., SS says no 2
n/a no n/a n/a n/a 3
n/a no n/a n/a n/a 5
subcooling yes no no decision 6
n/a no n/a n/a n/a 7
level <4%
no no SO w/SS, STA Operator induced 8
level <4%
no yes SO w/SS Foldout page criteric,n 9
level <4%
no no SI not considered n/a 10 level <4%
no no SO w/SS,STA Operator induced 11 subcooling yes yes SO Foldout page criterion variability in response of the crews to this foldout confederates who were aware of tl ent. These page criterion re-emphasizes the difficulty of the crew members informed the 50 tk c pressurizer level discriminations and decisions the crews confronted.
was going up, that pressure was continuing to go down, and that subcooling was less than 30 degrees In the case of the two crews where there was an (the Si criterion), without integrating the evidence for ongoing pressurizer PORV leak (Crews 5 and 11), the the SO, or pointing out that the foldout page SI low pressurizer level criterion was not met in the criterion was met. In this case the 50 did not identify Reactor Trip response procedure because by that the leak nor realize that a foldout page Si criterion point a bubble had formed in the reactor head and was met by the time the simulation was terminated.
pressurizer level was increasing. These crews both it is likely that had the simulation continued he met the loss of subcooling Si criteria however. In the would have recognized the need for a manual SI.
case of one of the crews (Crew 11) the loss of subcooling was identified by the STA and the crew The contrast in performance between Crew 11 that manually initiated SI. In the case of the other crew recognized that the S1 foldout page criteria were met, (Crew 5), four of the five members of the crew were and Crew 5 that did not, may be due to the difference NUREG/CR-6208 62 uueur
Crew Interaction in number oicrew members actively participating in operator induced cooldoum and that consequently there situation assessment and response evaluation. It was no need to safety inject. He argues that an SI would provides some evidence for the positive contribution not help the situation. He suggests that they check other of multiple crew members to situation assessment and parameters in assessing whether 91is needed. The SO response evaluation.
agrees and tells the RO to monitor pressurizer leteland if it looks like it cannot be maintained they willinitiate a manual SI. "*
Situation Assessment: Checkingfor evidence to confirm hypothesis; Siis not needed 10:29:17 STA: That's operator induced... if level is increasing, I think we could safely stay where we are Many of the crews that correctly decided against a t.
manually initiating SI as required by the procedure provided evidence that they considered the intent Subcooling is greater than 70 deg.
behind the procedure step, engaged in knowledge-driven monitoring to ensure that the problem 10:30:10 STA: I don't think that doing a SI right now situation was not present, and considered the would do us any good.
consequences of initiating SI before deviating from the procedure step. In addition, in most cases, they SO: I agree, but by the letter of the law I don't want to sought consensus among crew members before taking get in troub!c here.
the action.
STA: I say we all talk about it and know what is Protocol segments from two crews illustrate the happening, and look at all the other indications.
cognitive difficulty of the decision involved and the types of extra-procedural activities the crews engaged 10:30:34 SO (to RO): If it looks like we can't maintain in before deciding whether to deviate from the level let me know and we will safety inject.
procedure. In both cases there was no leak present in the RCS.
- " Eventually pressurizer level comes back up. confirming the crew's assessment of the situation *"
Case 1: Deciding whether to initiate Si based on afoldout page criterion 10:33:49 RO: Levelin the pressurizer is increasing. It is about 6%.
In the first protocol segment the crew had to decide whether to manually Si based on the foldout page criterion of pressurizer level less than 4%
Case 2: Deciding whether to initiate Si based on pressurizer pressure less than 1830.
Decision on Initiating SI The next protocol segment is of a crew that has to Based on a Specific Criterion decide whether or not to manually SI based on a step Crew 7 in the Reactor Trip Response procedure regarding pressurizer pressure less than 1830.
'" The SO observes that pressurizer level is below the foldout page criterion for initiating Si and calls the SS over for consultation. "*
Decision on Initiating SI l
10:28:20 SO to SS: I have a question regarding foldout Based on Low Pressurizer Level page. If pressurizer level cannot be maintained Crew 10 greater than 4% then SI. Pressurizer level reads 0 on one meter and 2% on two ucters.
- " At this point in the scenario the crews observe that they have lost pressurizer level and attributed it to cooldown. "*
"* The STA comes over to join SS and SO and suggests that the pressurizer level behavior can be e.tplained by an 63 NUREG/CR-6208
Crew Interaction 10:40:12 SO: We lost pressurizer level. We cooled off 10:45:50 STA: RCP seals indicating proper, way fast.
containment pressure maintaining zero, area rad monitors, ril check those. Area rad, monitors in We are recovering real slow, containment stable.
'" At this point the SO gets to the step that checks that 10:46:54 STA: Natural cire. RVLIS indicates 100%
pressurizer pressure is greater than 1830 psig. It is not, so he goes to the RNO column which indicates that Si should 10:49:10 We have subcooling of 60 on one and 20 on be initiated. The SO indicates that he does not uunt to $1.
the other.
The RO concurs, pointing out that RCS pressure is stable.
"* At this point someone points out that if they did actitute Si it would have negative consequences since it 10:41:45 Step 5 - Check PRZR pressure greater than might cause them to lose thefeeduuter they had just 1830 - no.
recovered. '"
10:42:30 Goes to RNO - Verify Si actuation. -
10:49:22 If we activate SI we willlose all feed.
SO: I don't think we waut to do that.
10:49:31 Now it is coming back.
10:42:40 BOP: RCS pressure is stable right now.
10.49:43 RCS pressure is hanging around 1000 lb.
RO: No, we don't want to SI. Where do you see that?
10:42:56 "* SO, RO and BOP alllook at the procedure."*
These two protocol segments illustrate that in deciding against manual initiation of SI the crews
"* At this point the SO consults with SS who concurs engaged in several extra-procedural activities. First, that SIis not needed. He indicates that the low pressurizer they sought to explain the observed RCS behavior. In pressure is ' operator induced' meaning that it is due to a both cases they determined that the RCS behavior cooldown brought on by the activities of the operators on could be explained by known iniluences. Both crews the secondary side. The STA agrees. "*
mentioned that the RCS behavior was " operator induced"; that is, that it was due to the cooldown that 10:43:06 SO to SS: I need your opinion. I'm at this step they initiated. Second, they explicitly checked to it says,. Verify SI actuation... We don't want to do make sure there were no other symptoms of a fault
- that, present in the RCS. Based on these observations and situation assessment they determined that a manual SS: This is an operator induced thing... we will go on SI was not needed. Third, they judged that taking the l
with this procedure in doing the steps we can, action (i.e., initiating a manual SI) would have l
negative consequences. Finally, the crews engaged in l
10:44:29 SS confers with STA: I don't think we want these situation assessment and response evaluation toSI.
activities as a group and sought consensus in the decision.
10.44:40 STA: We are here because we put ourselves here, so it is a controlled action.
The results highlight the role of group interaction in I
situation assessment and response evaluation. The
- " The RO asks the crew to check that there are no other protocol segments presented above illustrate that indicatioas ofan RCS leak. The STA checks a number of parameters and reports back that they all read normal.*"
10.45:43 RO: We don't have any indications of an RCS leak, right gentlemen? (Others confirm that no, they did not see any indications of a leak.)
Crew Interaction multiple crew members contributed evidence and the simulation was terminated. Crew 11 believed the opinion in formulating situation assessments and RCS symptoms were due to a cooldown for a long evaluating response options. This was particularly period of time. In particular, when they first raised true when the crews reached points in the EOP where the possibility of manually initiating SI due to low their assessment of the correct action to take diverged pressurizer pressure, they decided against it, from the actions specified in the procedure. At those explicitly indicating that they did not believe they had points the sos generally sought input from all crew a leak on the primary side. This crew did decide to members and approval from the SS before deciding to manually SI at a later point in the event based on diverge from a procedure step.
meeting the loss of subcooling Si criteria.
I l
The fact that two crews had trouble identifying the 3.4.4 Variability in Crew Performance leaking pressurizer PORV suggests that discriminating between RCS behavior due to a The majority of crews correctly identified the leaking cooldown and RCS behavior due to a LOCA was a 1 ORV and closed the block valve. When they difficult cognitive task. Sources of complexity returned to the Reactor Trip Response procedure they included (1) the fact that manual and automatic correctly judged that SI was not needed and modified actions were constantly occurring that changed the the Reactor Trip Response procedure accordingly.
pattern of influences on the RCS, making it difficult to Nevertheless, variability in performance was predict RCS behavior and discriminate expected from observed.
unexpected behavior, and (2) the fact that the crews had little experience with feeding on the condensate Two of the ten crews that participated in the scenario system, so they had little basis to judge what the had trouble identifying the leaking pressurizer PORV.
expected effect of the cooldown on RCS behavior Crew 5, where four of the crew members were would be.
confederates, had not identified the leak by the time l
4 Crew Interaction in the Simulated Emergencies Section 3 focused on the role of higher-level cognitive 4.1 Cognitively Demanding activity in guiding operator performance in the Situations Where Good Crew simulated emergencies. We also examined crew Interaction was Important interaction in handling these cognitively demanding 8""^'I S'
We identified three types of cognitively demanding Team interaction skills play an important role in crew situations where specific types of crew interaction performance in complex dynamic situations (cf.,
appeared to contribute positively to successful crew P'
n mf n tuhmcalperspective. These Swezey and Salas,1992). Under a separate program e'
sponsored by the U. S. NRC, Montgomery et al.
(1992) identified six dimensions of team interaction skill and developed BARS scales for measuring crew Cases where operators needed to pursue multiple performance on those dimensions. The dimensions objectives. Specifically, casds where they had to identified were: (1) communication, (2) openness, (3) manage dual requirements to (1) proceed through coordination, (4) team spint, (5) task focus, and (6) the EOPs to cool down the plant and bring it to a adaptability. As part of the present study we more stable state in a timely manner and (2) examined crew performance on these dimensions.
engage in extra-procedural activities to handle aspects of the situation that were not coverH by the EOPs'-
The objectives of the analysis were: (1) to clarify the conditions under which crew interaction skills might be expected to affect techmcal performance of crews Cases where situation assessment required and (2) to begin the process of describing specibe integration of information that was distributed across crew members; and crew behaviors that potentially contribute to better technical performance.
Cases where crews had to evaluate the The second aspect of the analysis focused on the aPPmpriateness of a procedure path and/or usefulness of the BARS rating scales per se in decide whether to take actions not explicitly evaluating team interactions skills. We examined SPecified in the procedures, crew ratings on the BARS scales to assess (1) whether there was variability in crew scores on the BARS In each case we examined characten. t.s ics of crew dimensions and (2) whether there was a relationship between BARS ratings of team skill and crew mteraction that appeared to contribute positively to technical performance on the scenarios.
crew perf rmance fr m a techmcal perspective.
Researchers have had limited success to date in finding a positive link between crew interaction skills 4.1.1 Cases Where Crews Needed to and technical performance. While the results we Pursue Multiple Objectives present below are by no means definitive, they provide evidence suggestive of a positive link In the two ISLOCA scenarios crews needed to engage between crew interaction skills and technical in extra-procedural activity to identify and isolate the performance, and point to the kinds of studies and leak into the RHR. They also needed to proceed with analyses that could provide more definitive results.
the cooldown as rapidly as possible to reduce the effect of the leak and stabilize the plant. We examined how crews organized themselves to deal 67 NUREG/CR-6208 e
-~
Crew Interaction with these multiple objectives, and whether some to the cooldown procedure. Collapsing across the two crew styles of organization led to better performance scenarios the mean time to cooldown for " divide and than others. These behaviors fall under the BARS conquer" crews was 33 minutes (n=6), while the
" adaptability" dimension of crew interaction skills.
mean time to cooldown for " alternate" crews was 52 minutes (n=10). This difference is statistically Two crew styles of organization were identified. Some significant using a two-tailed t-test (p < 0.05).
crews appeared to alternate between following the steps in the EOP and situation assessment and Since proceeding expeditiously to the Post-LOCA response planning activities. For example, when these Cooldown and Depressurization procedure is a high crews got to the step in the LOCA procedure priority goal, the results suggest that a " divide and requiring them to identify and isolate the leak, they conquer" crew organization style may have certain tended to stay a long time on that step. This crew benefits over an " alternate" crew style because it is style was labeled "altemate." A second crew style we likely to allow the crews to proceed through the EOPs identified was characterized by a tendency for the more rapidly. These benefits only hold if the two crew to divide into two subgroups, with one subgroups maintain close communication and subgroup concentrating on trying to identify and coordination to ensure that they are not taking actions kolate the ISLOCA and the second subgroup that interfere with one another. The groups that used-condr&g on moving through the procedures in a " divide and conquer" strategy tended to use the SO order to get to tha cooldown more quickly. For as a focal point and alerted him of all major actions example, in the case of one crew (Crew F) the SO before taking them. Crew 7 in ISLOCA 2 provides an explicitly requested that the SS and RO use the example of a case where the actions taken by the ISLOCA procedure to try and identify and isolate the subgroup that was pursuing the source of the leak leak into the RHR, while he and the BOP continued into the RHR (isolating the CCW service loop) with the LOCA procedure. We labeled this crew style affected activities of the subgroup that was working
" divide and conquer."
through the Post-LOCA Cooldown and Depressurization procedure (procedure steps that We examined whether one crew style of organization assumed the CCW service loop was available).
enabled the crews to reach a cooldown state more Because the two subgroups communicated their quickly than the other. We computed the time in actions, a potential impasse was identified and minutes from reactor trip to the time the crews started resolved.
the Post-LOCA Cooldown and Depressurization procedure.43 In both the case ofISLOCA 1 and These results point to the importance of the team ISLOCA 2 the crews that were identified as " divide skills of communication, coordination, and and conquer" reached the cooldown procedure faster adaptability to changing plant conditions in dealing than the crews that were identified as " alternate."
with situations that require simultaneous pursuit of multiple objectives. More specifically, the results in ISLOCA 1 seven crews reached the cooldown suggest particular crew behaviors that may lead to procedure. Of these, four crews were classified as improved technical performance (i.e., crews breaking
" divide and conquer" and had a mean time of 34 up into subgroups with the SO as the point of focus minutes to get to the cooldown procedure. Three were for communication and coordination).
classified as " alternate" and had a mean time of 42 minutes to get to the cooldown procedure, In the case of ISLOCA 2 two crews were classified as " divide and 4.1.2 Cases Where Situation Assessment conquer" and had a mean time of 32 minutes to reach Required Integration of Information the cooldown procedure. Seven were classified as Across Multiple Crew Members
" alternate" and had a mean time of 56 minutes to get A second case where crew interaction skills appeared In the case of the two crews (Crew 6 and Crew 4) that to be important to technical performance was in 43 transitioned to the ISLOCA procedure, the time to cooldown was forming correct situation assessment in cases where computed as the time from reactor trip to the Loss of Emergency the pieces of evidence that had to be identified and Coolant Recirculation procedure.
integrated were distributed across crew members.
Crew Interaction Two of the BARS dimensions of crew interaction skills 4.1.3 Cases Where Crews Had to Evaluate appeared to be important to technical crew Whether toTake Actions Outside performance in these cases. One was communica' ion.
the Procedures in the simulated scenarios cases arose where a piece of evidence that was needed to identify the plant fault was only seen by a single crew member, and there A third type of situation where a positive role of crew was no EOP step that specifically requested that mteraction on techmcal performance was identified piece of information. In those cases correct situation was when crews had to evaluate the appropriateness assessment depended on the crew member f a Procedure path and/or decide whether to take recognizing the value of the information and actions not explicitly specified in the procedures.,
communicating it to the rest of the crew. A specific A"*'.ysis mdicated that ' openness m crew mteraction case in point was the rupture of the PRT in ISLOCAs was imp rtant both from the perspective of I and 2. The crew member who noticed the Sener tin 8 Proposed actions to take, and from the symptoms in the PRT needed to communicate that perspective f evaluating those proposed actions. A clear examP e occurred in ISLOCA 1 where crews l
information to the other crew members in order for c nsidered whether to isolate the affected RHR tra_.
m the leak into the RHR to be identified. In one case (Crew 3,ISLOCA 2) one of the crew members knew hanunation of aew performance in that case the PRT had ruptured but failed to communicate it to revealed that the initial suggestion to isolate the RHR the SO and the rest of the crew. This crew did not was made by crew members in a variety of positions identify the problem in the RHR untillate in the (i.e., RO, STA, SS, BOP, SO). In all cases the crews did decide to isolate the RHR train but only after examination of the possible consequences of the A second dimension of crew interaction skill that acti n by the crew as a whole. The final decision was appeared to be important for correct situation made by the 50 after soliciting input from other crew assessment was cpenneas. The results showed that members and approval from the SS. Sumlar results were bserved in the LHS 2 scenario where crews had crew members in all positions contributed positively to decide whether to deviate from the literal to hypothesis generation and revision. This was shown most clearly in the case of ISLOCA 1. While requirements of procedure steps m the Reactor Tn,p the first hypothesis generated to explain the plant Response procedure.
symptoms was most often generated by the SO (five 4.1.4 Summary out of 11 cases), there were also cases where it was the SS or the BOP that generated the first hypothesis.
Further, when we looked at cases where the initial The analysis provided above revealed cognitively hypothesis was revised, and examined which crew demanding situations where contributions of j
member suggested the revised hypothesis, we found multiple crew members appeared to play a role in that crew members in all positions were represented successful crew technical performance. It also (i.e., RO, STA, SS, BOP). In cases where the first suggested some specific crew behaviors (e.g.,
hypothesis that was generated was relatively dividing into subteams; communicating indications of implausible, and it was revised to a more plaustble bnormal plant behavior; volunteering hypotheses; explanation, the crew member who suggested the critiquing hypotheses; proposing response actions; revised hypothesis was different from the crew evaluating proposed actions) that fell under the BARS member who suggested the original hypothesis.
dimensions of crew interaction skills that appeared to These results suggest that having multiple crew c ntribute positively to the technical performance of the crews.
members participate in the generation and revision of hypotheses contributes positively to correct situation assessment, in turn, this suggests that " openness" of Section 4.2 contins the results of the crew ratings on crew members with respect to suggesting and the BARS scales, providing a further description of critiquing hypotheses contributes positively to correct specific crew behaviors that characterized good crew situation assessment.
Performance on the BARS scales, and appeared to contribute positively to technical performance of the crews.
Crew Interaction Implications of these results for the training and were about to be started that would strongly affect testing of crew interaction skills are discussed below plant state (e.g., depressurizing a steam generator following presentation of the BARS ratings results.
that would result in a cooldown). Cases where crews failed to communicate critical plant state information (e.g., that the PRT ruptured) or operator actions (e.g.,
closing the PORV block valve) resulted in lower 4.2 BARS Ratings of Crew scores on the communication dimension.
Interaction Skills Crews varied in the ' openness' dimensions. Crews The BARS scales were used to rate crew performance with a high openness score tended to include crew in each of the scenarios. The resulting BARS ratings members who volunteered situation assessments or were examined to assess (1) whether there was suggestions for actions, and sos who explicitly variability in crew scores on the BARS dimensions solicited the opinion of crew members and sought and (2) whether there was a relationship between consensus for all major situation assessments and BARS ratings of team skill and crew technical decisions, performance on the scenarios.
Crews also varied on the dimension of Task 4.2.1 Variability among Crews on BARS Coordination. There were several opportunities to bserve the role of crew coordination. In the ISLOCA Dimensions scenarios crews differed in how they organized themselves to deal with both the need to identify and Table 4.1 presents the mean ratings of the crews on is late the leak outside containment and the need to each of the BARS dimensions for each scenario. The pr ceed expeditiously to the Post-LOCA Cooldown.
standard deviations appear in parentheses. 'Ihere was variability in crew ratings on four of the six In the Loss of Heat Sink scenarios crew coordination dimensions. Little or no variability was observed in was required to depressurire the RCS and block the SI the ratings of Team Spmt and Task Focus.
signal without inadvertently safety injecting. Crews that scored high on the coordination dimension Crews varied extensively in degree of tended to have sos that provided the operators an communication. Specific behaviors that contributed veniew f the steps about to be taken. These sos to a high score on the communication dimension ne t g ve th cmw an oveniew of tk whok included making sure that allimportant plant maneuver before trutiatm, g the RCS depressurization changes and crew actions were known to all crew and to explicitly assign specific roles for the different members, providing periodic summaries of current P ' '*'
situation assessment, and announcing activities that Table 4.1 Mean ratings of crews on the BARS dimensions.
(Standard deviations appear in parentheses.)
Scenario Communic.
Openness Coordination Team Spirit Task Adapt.
Focus ISLOCA 1 5.1 (1.0) 5.5. (0.8) 5.0 _ (1.4) 4.2 (0.4) 4.0 (0) 4.6 (1.7)
ISLOCA 2 4.5 (1.5) 5.2 (1.5) 4.8 (1.4) 3.7 (1.0) 4.0 (0) 5.3 (1.2)
LHS 1 4.6 (1.1) 5.0 (0.9) 4.3 (1.5) 3.9 (0.4) 4.0 (0) 4.8 (1.6)
LHS 2 4.9 (0.9) 5.3 (0.5) 5.1(12) 4.0 (0) 4.0 (0) 5.1 (1.2)
Crew Interaction The following protocol segment provides an example dimensions. All the crews showed positive team of an SO who looked ahead in the procedures and spirit. Expressions of anger or frustration at each provided the operators an overview of maneuvers to other were extremely rare.
come.
The fact that variability in ratings occurred across crews on four of the six dimensions suggests that these dimensions may be useful in evaluating crew Looking Ahead in the Procedures interaction performance. Previous attempts to use t', -
Crew 1 BARS scales had found limited variability in crew ratings on the events examined. It is possible that This protocol segment starts just at the point where the there was more variability in crew interaction crew reaches the RCS depressurization step.
performance in this study because of the greater cognitive demands of the scenarios. As discussed in 14:48:29 SO reads caution prior to step 7.
Section 3, a number of cognitively demanding situations arose in those scenarios where good 14:48:47 Step 7.
technical performance depended on the contributions and coordination of multiple crew members. It is 14:49:05 SO to BOP: De alert for monitoring pressure possible that these scenarios placed greater demands for RO, so that when we get to less than 1920, RO can on team interaction skills and thus provided the block steam line pressure SI, low pressurizer pressure.
opportunity to observe variability in performance.
SI.
14:50:11 SO: Alright, two things here guys. One, we 4.2.2 Evidence of a Link between Crew know that we can do Charlie feed reg. valve if we Interaction Skills and Technical need to. Don't worry about that. Don't forget that Performance we have the ability of opening it. Two, we want to go to arm so that we don't lose the ability to have the We also examined whether a link could be established block valve open on the pressurizer PORV that you between crew performance on the BARS ratings of are using. Team work here. (To RO:) You open the crew interaction and crew technical performance. In I
PORV; when pressure gets less than 1920 you close general, crew technical performance on the scenarios the PORV. (To BOP:) At that time you're over here was very good. The large majority of crews correctly blockmg low pressure SI and low steam line pressure identified the leaks and took appropriate action in SI. Both of you ready. Proceed.
attempting to isolate the leaks. Nevertheless, in each scenario there was one crew whose technical performance was clearly less good than that of the other crews (Crew L in ISLOCA 1, Crew 3 in ISLOCA Crews also varied on the dimension of ' adaptability.'
2, Crew H in LHS 1, and Crew 11 in LHS 2.) These The ' adaptability' dimension was used to rate crews four crews failed to reach a correct situation on how quickly they detected and responded t assessment and as a result failed to take actions changing plant circumstances. High ratings on this needed to isolate the leaks.
dimension tended to be given to crews that detected and pursued the primary symptoms in each event BARS ratings for these four crews on the events in while continuing to proceed through the EOPs. In the question were compared to the BARS ratings for the ISLOCA these were the symptoms of a leak outside remaining cases (33 cases). The mean ratings on the containment. In the Loss of Heat Sink scenario the four BARS scales for which variability across crews primary symptoms were those of a leaking was observed are presented in Table 4.2. Crews that pressurizer PORV.
The dimensions of ' team spirit' and ' maintaining task focus in transitions'seemed less useful in that there seemed to be less variance across crews on these 71 NUREG/CR-6208
Crew Interaction Table 4.2 Mean BARS ratings for crews that differed in technical performance.
(Standard deviations appear in parentheses.)
Crew Technical Number of Communic.
Openness Coordination Adapt.
Performance Crews Good 33 4.9 (0.9) 5.4 (0.8) 5.0 (1.3) 5.2 (1.3)
Less Good 4
3.5 (2.1) 4.5 (1.9) 3.5 (1.3) 3.0 (0.8) were classified as ' good' from a technical perspective a link between crew interaction skills and technical had higher mean BARS ratings on all four BARS performance. While the results are not definitive, dimensions than the crews that were classified as 'less they point to the kinds of studies and analyses that good' from a technical perspective Analyses of could provide more definitive results.
variance indicated that the mean differences in BARS ratings were statistically significant (p < 0.05) in the There was more variability in BARS ratings of crew case of three of the four BARS dimensions:
interaction skills in this study than in previous studies communication, coordination, and adaptation. In the (Montgomery et al.,1992). One possible explanation case of the dimension of " openness" the mean is that the scenarios used in the present study were difference in ratings was not statistically significant.
more cognitively demanding. A number of cognitively demanding situations arose in these The statistically significant difference that was scenarios where better technical performance obtained on some of the BARS dimensions between depended on the contributions and coordination of crews that performed technically well on the multiple crew members. These scenarios may have scenarios and crews that performed less well is an pla ed greater demands on team interaction skills and important finding. Researchers have generally had thus provided the opportunity to observe variability difficulty establishing a link between team interaction in performance.
skills and technical performance. If the finding is reliable it would support the position that team This argument suggests that future studies that interaction skills contribute to better let hnical atternpt to establish a link between team interaction performance.
skills and technical performance should employ scenarios that are specifically designed to be However, because only a single rater (the first demanding from the perspective of team interaction.
author) was used, the reliability of the BARS ratings
'1he scenarios should be designed so that technical obtained, and therefore the robustness of the evidence performance depends on the contributions and connecting BARS ratings to technical performance, is coordination of multiple crew members. The results not clear. Because of the potential importance of the presented in Section 4.1 begin to point to the kinds of result it may be worthwhile to attempt to replicate cognitively demanding situations where crew the result using a larger group of raters.
interaction skills would be expected to affect crew technical performance. Specific crew behaviors that are indicators of gozi crew interaction should be 4.3 General Discussion of Team identified a priori. The analysis should focus on whether crews exhibit these behaviors, and in turn Interaction Skills Results whether these behaviors are linked to good technical performance. Researchers examining aircrew team The results served to clarify conditions under which interaction skills have proposed similar crew interaction skills may t e expected to affect methodological approaches (Fowlkes, Lane, Salas, technical performance of crews. They also revealed Oser and Pince,1992).
specific crew behaviors that may characterize good crew interaction and contribute to technical The results also pointed to specific crew behaviors performance. In addition, they provided evidence of that appeared to characterize better performance on NUREG/CR-6208 72
Crew Interaction the BARS dimensions of crew interaction skills and to Understanding the specific behaviors that contribute positively to technical performance of the characterize team skills is important for guiding crews. Examples include splitting into subteams, development of team skills training programs. While having all crew members participate in situation the present results are suggestive, more research is assessment and response planning activities, ensuring needed to establish a definitive link between specific that all crew members are cognizant of key plant state crew interaction behaviors and crew technical information and control actions that are taken, and performance.
providing periodic recaps of current situation assessment and upcoming activities.
73 NUREG/CR-6208 e
. _... _ _ = _ _ =.
5 Discussion of Results and TheirImplications 5.1 General Discussion A case where operators had to decide whether to manually initiate a safety system based on in Section 1 we contrasted two alternative views of consideration and balancing of multiple goals the nature and extent of cognitive activity required of related to safety (LHS 1).
operators to adequately handle emergencies. One view was that in emergencies the operator's primary in all these cases we found evidence of operators role is to follow the EOPs by rote. According to this actively engaging in situation assessment and view all that is needed of operators is that they be able response planning in handling the situation.
to understand and follow the individual steps in the EOPs.
There are three alternative interpretations of these results, each with distinct implications. If one starts This position was contrasted with the view that from the premise that procedures should provide situation assessment and response planning continue detailed guidance for every contingency, then one to be important for successful operator performance, interpretation of the results is that they demonstrated even when EOPs are employed. According to this deficiencies in the particular procedures that were view situation assessment and response planning included in the study. According to this view if enable crews to identify and deal with situations that situations are identified that are not covered by the are not fully addressed by the procedures.
procedures, then the procedures should be rewritten to handle those situations. Given this view, the The results of this study provide support for the results have primary implications for the specific second position. We found a number of situations procedures employed in the study, that were not fully addressed by the EOPs. These included:
A second view is that the EOPs are not intended to i
diagnose and respond to particular faults optimally.
j An EOP step that explicitly requested that crews They are intended to provide a systematic approach identify and isolate a leak on their own (ISLOCA to emergency response that minimizes the possibility j
1);
of core damage. According to this view, while the l
operators may have engaged in situation assessment A case where the procedure containing relevant and response planning in these scenarios, these guidance could not be reached within the EOP cognitive activities were not necessary, and were transition network (ISLOCA 2);
possibly not even desirable. Had the operators followed the procedures implicitly they would have Cases where operators needed to determine eventually been directed to take actions that would whether plant behavior was the result of known have mitigated the consequences of the leaks and manual and/or automatic actions (e.g., a Prevented core damage. Given this view, the primary controlled cooldown) or the result of a plant fault contribution of the study is that it demonstrates that (all four simulated events);
Operators take a more active role in diagnosing and responding to events than might have been believed; A case where operators were required to evaluate however, the results have minimal implications for the appropriateness of procedure steps given the training and procedures.
specifics of the situation (LHS 2);
A third view is that the types of situations that were Cases where operators had to evaluate the identified in the study are generic classes that are appropriateness of a procedure path and take likely to arise in other emergency scenarios.
l action to redirect the procedure path (ISLOCA 2; According to this view, the complexity of NPPs make LliS 1; LHS 2);
it difficult to anticipate and develop EOPs that cover 75 NUREG/CR-6208
Discussion every possible contingency in detail; therefore it is In ISLOCA 1 the crews were required to identify and reasonable to assume that situations may arise that isolate the leak into the RHR without explicit are not fully addressed by the procedures. It will be procedural guidance. In ISLOCA 2, while there was a important in such situations for the operators to have procedure transition available to an ISLOCA the ability to form accurate situation assessments and procedure,in the case of several of the crews it could to generate response plans to cover aspects of the not be reached. Even in the cases where the ISLOCA situation that are not fully addressed by the procedure was reached, it did not cover all aspects of procedures. Examination of recent actualincidents the situation (i.e., the leak into the CCW).
support this position (NRC, NUREG-1455; Kauffman et al.,1992).
Most crews actively sought information to help identify the sources of leaks into the RHR and CCW, A logical consequence of this third view is that in the and identified and took actions in attempting to development and evaluation of training and control isolate the leaks. They actively utilized resoe ca room aids (e.g., procedures, displays, decision-aids),
beyond the EOPs to support their identification and explicit attention should be paid to supporting isolation of theleaks. Without active situation operator situation assessment and response planning.
assessment, knowledge-driven monitoring, and response planning, they would not have been able to The results of the study by themselves do not identify and isolate the leaks.
definitively support one view over the others.
However, we present evidence from actualincidents, At the same time, most of the crews recognized the experiences in other domains, and logical arguments, importance of continuing to proceed through the in support of the third view.
EOPs. They perceived getting to the Cooldown and Depressurization procedure as a high priority activity.
In Section 5.2 we summari7e some of the main results llalancing the dual requirements to pursue the leak of the study that provided evidence of a need for into the RHR with the need to proceed expeditiously situation assessment and response planning on the through the EOPs provided one of the most part of the crews. In Section 53 we describe the roles challenging aspects of the ISLOCA scenarios.
that situation assessment and response planning played in guiding operator performance in these The ISLOCA scenarios also provided evidence of cognitively complex emergencies. In Section 5.4 we crews actively engaged in reasoning about the examine the three attemative ways of interpreting the procedure logic. Protocol segments showed crews results and their implications. We provide arguments reasoning at two levels. They were engaging in in support of the third view that operators need to situation assessment and goalidentification. At the engage in situation assessment and response planning same time they were reasoning about the strategies in handling emergencies. In Section 5.5 we discuss underlying the EOPs, and the EOP transition the implications of this view for the development and network logic to assess whether the procedure path evaluation of training and control room aids, as well they were on would enable them to achieve plant as for Human Reliability Analyses.
goals in a timely manner.
We conclude with some specific comments on the We found instances where monitoring the value of simulator-based empirical studies of operator appropriateness of the procedure path enabled crews performance.
to identify when they were in an unproductive loop, and to identify another procedure path that would allow them to take necessary actions more "P"d
"*'Y' 5.2 Summary of Results The Loss of Heat Sink scenarios provided further Situations arose in each of the scenarios where evidence that complex multiple fault conditions can operators needed to engage in situation assessment arise that require operators to actively engage in and response planning.
situation assessment and response evaluation. In the Loss of Ileat Sink scenarios the procedures provided no guidance in identifying and responding to the NUREG/CR-6208 76
Ir.ussion leaking pressurizer PORV. The majority of crews primary side symptoms to cooldown and decided were successfully able to detect the symptoms on the against manual initiation of SI.
i
(
primary system and integrate them to identify the l
steam space leak. This difficult cognitive task In general, across scenarios, the majority of crews required recognizing that the primary side behavior performed well. They identified the faults and took could not be entirely explained by the ongoing appropriate action in response. The behavior of these cooldown. This task required qualitative reasoning crews clearly indicated that they were actively about the size and direction of effects on the primary engaged in situation assessment and response system that could be expected from the rapid planning.
depressurization of the steam generators.
While most of the crews performed well, variability In the Loss of Heat Sink 1 scenario, the crews were in performance was observed. Crews differed in the faced with a decision regarding manualinitiation of a extent to which they detected plant symptoms, safety system. The only EOP guidance available to actively sought an explanation for unexpected them was in a caution that indicated that they had findings, and attempted to come up with a coherent discretion to turn on the safety system if conditions in explanation that accounted for all the observed the plant " degraded." The decision of whether to turn symptoms. In each scenario there was at least one on the safety system required balancing multiple crew that failed to identify the source of the problem goals. Manualinitiation of the safety system would correctly and consequently failed to take appropriate respond effectively to the degrading conditions in the action to mitigate it. Given that the number of crews primary system due to the leaking PORV, but could that participated in each scenario ranged from eight to potentially delay recovery of heat sink. The crews eleven, this means that approximately 10% of the had difficulty with this aspect of the scenario. Most of crews experienced difficulty.44 The fact that not all the crews did not recognize that they had the crews in the scenarios formed the correct situation discretion to decide whether to turn on the safety assessment suggests that there is room for system. Further, few of the crews showed evidence improvement. Section 5.5 discusses potential ways to of considering the tradeoffs involved. The majority of enhance crew situation assessment and response crews chose to let conditions continue to degrade planning.
until n criterion was reached for which more explicit procedural guidance was available.
Finally, the results clarified the role of group interaction in situation assessment and response Loss of Heat Sink 2 provided additional opportunity evaluation. The results indicated that multiple crew to examine the role of situation assessment and members contributed evidence and opinion in response planning in guiding crew performance. In formulating situation assessments and evaluating this event crews returned to the Reactor Trip response options. This was particularly true when the procedure after recovering feedwater using the crews reached points in the EOP where their condensate system. In this case the crews were assessment of the correct action to take diverged from explicitly required, based on training and EOP the actions specified in the procedure. At those points background documents, to use their judgment in the sos generally sought input from all crew deciding the appropriateness of particular procedure members and approval from the SS before deciding to steps. Most of the crews correctly recognized that diverge from a procedure step.
some of the steps were inappropriate to the situation and should not be followed. This included steps that called for initiation of a safety system. The decision that initiation of the safety system was not needed was based on situation assessment. The crews had to determine that the conditions in the primary system 44While approximately 10% of the crews failed to reach the were due to cooldown and not a plant fault. 'Ihis was correct situation avessment,it should he pointed out that plant not a simple decision, as attested by the fact that,in safety was alwave.naintained. Even if these crews did not come j
t the correct situation assessment the EOPs would have the case of two of the crews who faced that decision, eventually led them to take action that would have prevented any there was a leak resent (leakinS Pressurizer PORV),
P serious consequences to core integrity.
but the crews nevertheless initially attributed the 77 NUREG/CR-6208
~
--- ~_
1 Discussion The importance of situation assessment is 5.3 The Role of Situation underscored by the frequency of recent actual Assessment and Response incidents where crews were required to discriminate (t"^l "^lf""'ti "S " '^il'd **"S
'S ' '^lS*
PlanninE,in Cognitively alarms (Kauffman et al.,1992). The results of the 1
Demandmg Emergencies present study as well as analyses of actualincidents suggest that it is important for operators to develop Situation assessment and response planning enabled and maintain an accurate situation assessment in the crews to handle aspects of the scenarios that were order to handle aspects of incidents that are not fully not fully covered by the EOPs. This section describes addressed by the procedures. Important elements of the role that situation assessment and response situation assessment include (1) an awareness of planning played in guiding operator response in the abnormal plant symptoms, (2) an assessment of the scenarios. It is reasonable to assume that the results likely malfunctions that could produce those observed can be generalized beyond these particular symptoms, and (3) an awareness of manual and crews and scenarios, and that situation assessment automatic system actions that are being taken, and 7
and response planning would play a similar role in their effect on plant state.
guiding operator performance in other cognitively demanding emergencies.
5.3.2 Response Planning 5.3.1 Situation Assessment The scenarios were designed to produce situations where operators were required to engage in response The scenarios provided extensive evidence of crews planning. In some cases this involved identifying and trying to develop an understanding of plant state. We evaluating response actions on their own. In other observed operators engaging in knowledge-driven cases, it involved monitoring the appropriateness of -
monitoring to confirm their understanding of a response actions specified in the procedures, and situation and seeking explanation for unexpected adapting the procedures to the situation if judged plant behavior. We also observed operators actively necessary.
trying to form a coherent explanation to account for multiple symptoms across diverse systems. These We found evidence of crews reasoning at two levels.
activities enabled the crews to identify and respond to They engaged in situation assessment and goal problems that were not fully addressed by the EOPs.
identification. At the same time they monitored the procedure path they were following to evaluate Situation assessment enabled the crews to:
progress toward high priority goals.
Detect abnormal plant behavior earlier in the Response planning enabled the crews to:
event than would be possible if they waited for an alarm or a step in the procedure to check those Move through the procedures efficiently; parameters; Catch and recover from errors -- both operator Detect symptoms or alarms that they had missed (rrors and errors in the procedures; earlier; Assess whether the procedure path they were on l
Identify and deal with additional problems that was appropriate to the situation; were not addressed by the procedures.
Fill in gaps and adapt procedures to the situation; it is reasonable to assume that situation assessment and would play a similar role in enabling crews to identify and deal with problems in other cognitively Deal with unanticipated situations that went demanding situations.
beyond the available procedural guidance.
m.
Discussion it is reasonable to assume that the role of response While this position is viable in principle, in practice it planning in enabling crews to deal with these is likely to be difficult to anticipate and provide situations would generalize to other cognitively detailed guidance for every possible contingency.
demanding emergencies.
His argument is supported by experience in attempting to develop detailed procedural guidance ne results provide evidence that it is important for in other domains (Roth, llennett, and Woods,1987; operators to be able to develop and evaluate response Suchman,1987). It is also supported by analyses of
- plans. It is also important for them to understand the actual incidents that often involve multiple faults and assumptions and logic behind the EOPs. This complications whose possibility had not been understanding includes the intent behind specific foreseen (Kauffman, Lanik, Trager, and Spence,1992; procedure steps, the overall response strategies NRC, NUREG-1455; Perrow,1984; Wagenaar and inherent in the procedures, and the transition logic Groeneweg,1987).
I among particular procedures in the EOPs.
Some of the cases identified in the scenarios could be 5.4 Alternative Views of the Role of handled by rewriting the particular procedure to explicitly deal with the case. An example is the Procedures and Implications of situation that arose in ISLOCA 1 where the EOPs Results "Sk d the Perators to identify and isolate the leak without providing further guidance. His procedure The results provide clear evidence that situations
- uld be rewritten to provide more detailed guidance arose in the simulated scenarios where operators with respect to identifymg and isolating the leak.
needed to engage in situation assessment and There were other cases, however, that could not be response planning to deal with aspects of the event that were not fully addressed by the EOPs.
easily handled by rewriting the procedures.
Examples include the case that arose in ISLOCA 2, 1
At the beginning of Section 5 three alternative where detailed guidance for identifying and isolating the ISLOCA was available but could not be reached mterpretations of these results were outh.ned that through the EOP transition network. The reason the have different implications for trammg, procedures, and decision aids. In this section these three views ISLOCA procedure could not be reached had to do are examined m more detail. While the results of the with the detailed dynamics of the event that study do not definitively support one view over the determined when symptoms came m. relative to when others, arguments are presented in favor of the third pmcedure steps were reached. Developing procedures that anticipate and provide for the variety view: operators need to engage m. situation assessment and response planning to handle of possible event trajectories that could arise would be a difficult task.
unanticipated situations that are not fully covered by the EOPs. This view has implications for training, procedures, and decision aids.
Procedure writers recognize limits in their ability to foresee all possible situations. - In some circumstances 5.4.1 View 1: Procedures Should Provide operators are explicitly directed by the EOPs to take action based on their own situation assessment. There Detailed Guidance for Every were three cases in the simulated scenarios where the l
. Contingency procedures or related background documents i
explicitly directed operators to determine appropriate One view starts from the premise that procedures action based on their own situation assessment:
should provide detailed guidance for every contingency. Given this premise, the results could be 1.
The case in the ISLOCA scenarios where viewed as providing evidence of deficiencies in the operators were asked whether pressure in all particular procedures that were included in the study.
steam generators is " stable or increasing;"
According to this view if situations are identified that are not covered by the procedures then the 2.
The caution that appeared in the loss of heat sinic procedures should be rewritten to provide detailed procedure that provided the operators discretion guidance for those situations.
in initiating a safety system; 79 NUREG/CR-6208
Discussion 3.
The case that arose in LHS 2 where operators to form accurate situation assessments and to were expected to determine whether particular generate response plans to cover aspects of the procedure steps in the Reactor Trip procedure situation that are not fully addressed by the were appropriate to the situation and should be procedures will be important.
followed.
Severallines of evidence support this position including, experience in developing detailed 5.4.2 View 2: Procedures Are Not procedural guidance in other domains (Roth, Bennett, and Woods,1987; Suchman,1987); experience in Intended to be Optimal.
introducing automation (Norman,1986); and analyses of actualincidents that involved multiple A second view is that the EOPs are not intended to f uits and complications that had not been foreseen diagnose and respond to particular faults optimally.
(Kauffman, Lanik, Trager, and Spence,1992; NRC, They are intended to provide a systematic approach NUREG-1455; Perrow,1984; Wagenaar and to emergency response that minimizes the possibility Groeneweg,1987.)
of core damage. Had the operators followed the procedures by rote they would have eventually been The results of the study, taken in combination with directed to take action that would have mitigated the evidence from actual incidents, and experiences in consequences of the leaks and prevented core related domains support the position that situation damage. According to this view, while the operators assessment and response plarming enable operators to may have engaged in situation assessment and handle unanticipated situations that are not fully response planning in these scenarios, these cognitive addressed by procedures. In Section 5.5 we discuss activities were not necessary.
the implications of this view for the development and evaluation of training and control room aids, as well -
This position underlies the development of the EOPs as for human reliability analyses.
and provides the rationale for requiring operators to follow procedures by rote. The results of this study do not contradict this position. In both the ISLOCA and the LHS scenarios, had the operators followed the 5.5 Implicalions of Results procedures by rote they would have eventually been directed to take action that would have prevented The view that unanticipated situations may arise in severe core damage; however, conditions would have actual incidents where operators need to engage in degraded sigmficantly before the procedures directed
,'Mtion assessment and response planning to deal the operators to take action to address the problem.45 This raises a concern because when conditions are addressed by the procedures has potential allowed to degrade the potential for nsk is increased.
implications for:
Training of operators; 5.4.3 V.iew 3: Situation Assessment and Development of displays and decision-aids to Response Planning Enable Operators to Handle Unanticipated Situations support operator cognitive performance; and Human reliability analysis.
A third view is that the complexity of NPPs make it difficult to anticipate and develop EOPs that cover every possible contingency in detail. According to this view it is reasonable to assume that situations 5.5.1 Implications for Training may arise that are not fully addressed by the procedures. In such situations the ability of operators The view that situations may arise where crews need to engage in situation assessment and response plann ng suggests that in developing and evaluating 451n the case of the ISLOCAs a large amount of primary coolant operator training programs attention may need to be would have been depleted. In the case of LilS 1. reactor vessel Paid to the development of these cognitive skills.
level would have decreced below 40%.
Discussion Kinds of Operator Knowledge r
Mental Representations Knowledge of EOPs of Plant Systems Goal-Means e goal prioritization
(" mental models")
Knowledge e response plan
- transition logic Y
Higher-Level Cognitive Situation Assessment Activity Response Planning V
Monitor Monitor Interpret Take Take Review Check Observable Plant Plant Plant Control Control ema ics Behavior:
Parameter Parameter State Action Action J L Emergency Operating Procedures Figure 5.1 Operator knowledge required to support situation assessment and response planning.
While most of the crews in the study were able to identify the leaks correctly and take appropriate Figure 5.1 shows three kmds of operator knowledge action, not all the crews formed an accurate situation required to support situation assessment and assessment. Crew performance might be improved response planning:
i by providing explicit training in situation assessment 1.
Operators need accurate mental models of plant and response planning.
systems. In our study we found evidence of 81 NUREG/CR-6208
Discussion situations where crews needed to utilize mental 5.5.2 Implications for Control Room Aids models of physical plant systems and to reason qualitatively about expected effects of different The view that unanticipated situations can arise factors influencing plant state in order to localize where operators need to engage in situation plant faults and identify actions to mitigate them.
assessment and response planning also has implications for the development and evaluation of 2.
Another type of knowledge needed is knowledge control room aids. In particular,it suggests potential of important plant goals and means to achieve value for displays and decision-aids that are explicitly them. Our study found evidence that operators intended to support situation assessment and needed to reason about plant goals, and evaluate response planning.
alternative means to achieving them, particularly in the Loss of Heat Sink 1 event.
The results of the study showed that operators sometimes had to engage in situation assessment 3 Finally, operators need knowledge of the EOPs, activity that required tracking multiple influences on l
which includes not only knowledge of how to plant state and distinguishing plant behavior due to follow the individual EOP steps, but also known influences (e.g., a cooldown) from unexpected
)
knowledge of the logic that underlies the EOPs.
plant behavior due to an unidentified fault. These l
This includes knowledge of the goal prioritization judgments often required integrating evidence across inherent in the EOPs, knowledge of the response space and time. Displays and decision-aids could be plans embodied in the EOPs and their rationale, developed to support these situation assessment and knowledge of the EOP transition network. It activities.
may be beneficial to explicitly address these types of know' edge in training programs.
Similarly, situations arose where crews had to evaluate responses for potential negative Mumaw, Swatzler, Roth and Thomas (1994) provide a consequences. This evaluation step occurred in the detailed review of training techniques for developing ISLOCA incident where crews needed to consider the these types of knowledge and cognitive skills.
implications of isolating systems for future recovery activities. It also occurred in the Loss of Heat Sink One way to foster situation assessment and response event where crews had to consider the positive and planning skills is to develop cognitively demanding negative consequences of initiating SI. Displays and training scenarios that provide the opportunity to decision-aids that facilitate identification of side practice specific cognitive skills (Roth, Mumaw &
cffects and consequences of contemplated actions Pople,1992). For example, training scenarios can be could be developed to support response evaluation.
developed that specifically focus on the ability to form accurate situation assessments. An example is a The results also have implications for procedures.
scenario that requires crews to discriminate effects Two findings in the study have potential implications due to cooldown from effects due to actual for design of procedures, particularly computerized malfunctions. Other scenarios can be developed that procedures. One finding is that it was important for focus on response evaluation. For example, scenarios operators to understand the logic and rationale l
can be developed that require operators to evaluate behind the procedures. This has implications for the the appropriateness of particular procedure steps to a content and organization of procedures. Another given situation and to take discretionary action as finding is that operators did not necessarily move appropriate.
linearly through a single procedure path. Crews i
looked ahead in the procedures, they moved back to l
The objective of the cognitive training would be to earlier steps, and they looked at other procedures in build operator okill in handling cognitively parallel as guidance. This finding has implications for j
demanding events. Since actualincidents typically the design of computerized procedures. It suggests i
involve multiple factors that make them unique, that ease of navigation through the procedure l
cognitive training may better equip operators to network is likely to be important for facilitating handle these unique features resulting in improved performance in complex emergencies.
safety.
1 I
Discussion i
5.5.3 Implications for HRA The view that operator performance is partly guided by situation assessment and response planning has Among the main conclusions of the study is that, potential implications for human reliability analyses while symptom-based EOPs have greatly reduced the (HRA). The results indicated that operators are need for operators to develop diagnostic and response engaged in a number of activities in addition to strategies on their own in real time, they have not following the steps in the EOP. Moreover, the results entirely eliminated the need for operators to engage in showed that following the EOP steps was not always situation assessment and response planning. In our straightforward. In some cases determination of how scenarios a number of cognitively demanding to respond to a procedure step depended on situation situations arose where operators were required to assessment These results suggest that analyses that exercisejudgment and take action based on their own focus on the ability of crews to follow individual steps ssessment of the situation.
in the EOPs may be insufficient.
The types of situations we identified are generic The results highlighted the importance of the classes that are likely to arise in other emergency dynamics of the event in determining what evidence scenarios. The ability of operators to form accurate is likely to be available at different points in the event, situation assessments and to generate response plans and what procedure transitions are likely to be made that adequately address the situation were shown to as a consequence. These results suggest that the be important for these situations.
dynamics of an event play an important role in determining human reliability. An implication is that The results are consistent with the view that situation human reliability assessments are likely to be more assessment and response planning enable operators to accurate if the dynamics of the event are explicitly handle unanticipated situations that are not fully considered in performing; them. This can best be addressed by procedures. This view has implications accomplished by running several crews through the for the development and evaluation of training, and specific events using a high fidelity dynamic control room aids (e.g.. grocedures, displays, simulator.
decision-aids); speca.cally it suggests that attention should be paid to the need to support and augment A decond implication of the results is that more operator situation assessment and response planning accurate human reliability assessments are likely to be etivities.
obtained if analysts take explicit consideration of factors in the events that may complicate situation The results also have potential implications for assessment or response planning. We have human reliability analyses. They suggest that developed a ' cognitive demands checklist' that lists analyses that focus only on the ability of crews to many of these factors that can be used to support f 11 w individual steps in the EOPs may be human reliability assessment. Appendix D contains insufficient. Human reliability assessments are likely the ' cognitive demands checklist..
t be more accurate if the dynamics of the event, and the factors that are likely to complicate situation A third potentid implication of the results relates to aussment and response planning, are explicitly the estimation of human error probabilities. An HRA c ns dered.
analyst who needs to estimate the human error probability for failure to diagnose a rare, complex A finale nelusi n f the study regards the value of event, that is not practiced in training, and whose empitical studies of operator performance in solution is not prescribed in a straightforward way by simulated emergencies for addressing human EOPs, might consider that these types of events were Performance issues of concern to the NRC Well simulated in this study, and that approximately 10%
designed empirical studies can provide specific, clear of the crews did not reach a fully adequate situation
"*88*#"
The EOPs provided recovery paths that did not depend on accurate situation assesstnent. As a result the probability of 46 should be noted that in these scenarios a correct situation failing to take a recovery action would be significantly lower than 1
assessment was not necessary to take appropriate rccovery action.
10%.
83
m Discussion conclusions for practical decision making. The present study provided: (1) evidence that situations can arise study illustrates how empirical studies of operator where higher-level cognitive activity on the part of performance in simulated emergencies can be used to operators is needed and (2) objective data on how investigate a human performance issue - in this case different operator crews responded to these the role of higher-level cognitive activity in operator situations.
response to cognitively demanding emergencies. The NUREG/CR-6208 84
6 References Barsalou, L. W., Cognitive Psychology: An Oivrviewfor Collaborative Work, Lawrence Erlbaum,Inc.,
Cognitity Scienlists, Lawrence Erlbaum Assoc.,
Hillsdale, NJ,1990.
Hillsdale, N. J.,1992.
Uutchins, E., How a Cockpit Remembers its Speed, Bartlett, F. C., Remembering: A Study in Experimental Technical Report, Distributed Cognition and Social Psychology, Cambridge University Laboratory, University of California at San Press, London,1932.
Diego,1991.
Beare, A. N., C. D. Gaddy, G. W. Parry, and A. Singh, Kahneman, D., P. Slovic, and A. Tversky, eds.,
"An Approach for Assessment of the Reliability Judgment under Uncertainty: Heuristics and Biases, of Cognitive Response for Nuclear Power Plant New York: Cambridge University Press,1982.
Operating Crews," in Probabilistic Safety Assessment and Management, G. Apostolakis Kauffman, J. V., G. F. Lanik, E. A. Trager, and R. A.
(Ed.), Elsevier Science Publishing Co., Inc.,1991.
Spence, Operating Experience Feedback Report -
Human Performance in Operating Events, Bransford, J. D., Human Cognition: Learning, NUREG-1275, Office for Analysis and Understanding and Remembering, Wadsworth Evaluation of Operational Data, U. S. Nuclear Publishing Company, Belmont, Ca.,1979.
Regulatory Commission, Washington, D. C.,
December,1992.
Corker, K. M., "An Architecture and Models for Cognitive Engineering Analysis of Advanced Klein, G. A. and R. Calderwood, " Decision Models:
Automation Control Environment," Proceedings Some Lessons from the Field," IEEE Transactions of the American Nuclear Society Topical Meeting on On Systems, Man, and Cybernetics,21,1018-1026, Nuclear Plant instrumentation, Control, and Man-1991.
Machine Interface Technologies,1993.
Laird, J. E., A. Newell, and P. S. Rosenbloom, " Soar:
(
Endsley, M. R: " Situation Awareness in Dynamic An Architecture for GeneralIntelligence,"
Human Decision Making: Measurement",
A rnficial in telligence, 33,1-64,1987.
Proceedings of the 1st International Conference on Situation Awareness in Complex Systems,1993.
Montgomery, J. C., C. D. Gaddy, R. C. Lewis-i l
Clapper, S. T. Hunt, C. W. Holmes, A. J.
Fowlkes, J. E., N. E. Lane, R. Salas, R. Oser and C.
Spurgin, J. L Toquam, and A. Bramwell " Team Prince, " TARGETS for Aircrew Coordination Skills Evaluation Criteria for Nuclear Power Training," Proceedings of the 14th Plant Control Room Crews," working draft, Interservicellndustry Training Systems Conference, 1992. (Available in NRC Public Document 1992.
Room.)
Fujita, Y., L Yanagisawa, K. Nakata, J. Itoh, N.
Mumaw, R. J., D. Swatzler, E. M. Roth, and Wm. A.
Yamane, and R. Kubota, "Modeling Operator Thomas, Cognitite Skill Trainingfor Decision with Task Analysis in Mind," Proceedings of the Making. NUREG/CR-6126, U. S. Nuclear
' American Nuclear Society Topical Meeting on Regulatory Commission, Washington, D. C.,
Nuclear Plant instrumentation, Control, and Man-June,1994.
Machine Interface Technologies,1993.
Norman, D. A. " Categorization of Action Slips".
Hutchins, E.,"The Technology of Team Navigation,"
Psychological Review, 88,1 15,1981.
in J. Galegher, R. Kraut, and C. Egido, eds.,
Intellectual Teamwork: Social and Technical Bases of 85 NUREG/CR-6208
1 l
References Norman, D. A. "The Problem with ' Automation':
Sarter, N. B. and D.D. Woods, " Situation Awareness:
Inappropriate Feedback and Interaction, Not A Critical but lil-defined Phenomenon".
1
'Over-Automation'," Philosophical Transactions of international Journal of Atdation Psychology,1(1),
the Royal Society of London, B327,1990.
43-55,1991.
NRC, NUREG-1154, Loss of Main and Auxiliary Spurgin, A. J., P. Moieni, C. D. Gaddy, G. Parry, D.
Feedwater at the Datis-Besse Plant on June 9,198S.
D. Orivs, J. P. Spurgin, V. Joksimovich, D. P.
U. S. Nuclear Regulatory Commission, Gaver, and G. W. Hannaman, Operator Washington, DC 200555,1985.
Reliability Experiments Using Pouer Plant Simulators. EPR1 NP-6937. Palo Alto, Calif.:
NRC, NUREG-1455, Transformer failure and Common.
Electric Power Research Institute, December, Mode Loss ofinstrument Power at Nine Mile Point 1989.
Unit 2 on August 13,1991. U. S. Nuclear Regulatory Commission, Washington, DC Suchman, L. A. Plans and Situated Action: The Problem 20555,0ctober,1991.
of Human-Machine Communication. Cambridge:
Cambridge University Press,1987.
Orasanu, J., R. K. Dismukes and U. Fischer " Decision Errors in the Cockpit," Proceedings of the Human Swain, A. D. and H. E. Guttmann, Handbook of Factors and Ergonomics Society 37th Annual Human Reliability Analysis with Emphasis on Meeting. Seattle, Washington, October 1993.
Nuclear Power Plant Applications. NUREG/CR 1278. Albuquerque, N. M.: Sandia National Perrow, C., Normal Accidents. Living with High-Risk Laboratories,1983.
Technologies, Basic Books,1984.
Swezey, R. W. and E. Salas, eds., Teams: Their Training Rasmussen,J., Information Processing and Human and Performance, New Jersey: Ablex Publishing MachineIntcraction: An Approach Io Cognitire Corp.,1992.
Engineering. North Holland, New York,1986.
Trager, Jr., E. A., Case Study Report on Loss of Safety Reason,J., Human Error. Cam ~oridge, England:
System Function Events. Technical Report Cambridge University Press,1990.
AEOD/C5(M, Office of Analysis and Evaluation of Operational Data, U. S. Nuclear Regulatory Roth, E. M., K. B. Bennett and D. D. Woods, " Human Commission,1985.
Interaction with an ' Intelligent' Machine" International Journal of Man-Machine Studies, 27, Wagenaar, W., and J. Groeneweg, " Accidents at Sea:
479-525,1987.
Multiple Causes and impossible Consequences," In terna tional Journal of Man-Roth, E. M., R. J. Mumaw and H. E. Pople, Machine Studies,27,587-598,1987.
" Enhancing the Training of Cognitive Skills for improved Human Reliability: Lessons Learned Woods, D. D., L. J. Johannesen, R. I. Cook and N. B.
from the Cognitive Environment Simulation Sarter, Behind Human Error: Cognitite Systems, Project," in Proceedings of the 1992 IEEE Fifth Computers and Hindsight, CSERIAC State-of-the-Conference on Human Factors and Power Plants, Art Report, November,1993.
1992.
Woods, D. D., H. E. Pople, and E. M. Roth, The Roth, E. M., D. D. Woods, and H. E. Pople, " Cognitive Cognitive Environment Simulation as a Toolfor Simulation as a Tool for Cognitive Tasks Modeling Human Performance and Reliability, Analysis". Ergonomics, Special Issue on NUREG/CR-5213, U. S. Nuclear Regulatory Cognitive Engineering,3S(10),1163-1198,1992.
Commission, Washington, D. C.,1990.
References Woods, D. D. and E. M. Roth, Models of Cognitive NUREG/CR-4862, U. S. Nuclear Regulatory Behavior in Nuclear Power Plant Personnel, Commission, Washington D. C.,1987.
NUREG/CR-4532, U. S. Nuclear Regulatory Commission, Washirgton, D. C.,1986.
Woods, D. D., J. A. Wise, and L. F. Hanes. Er aluation ofSafety Parameter Display Concepts. Electric Woods, D. D., E. M. Roth, and 11. E. Pople, Cognitive Power Research Institute, Palo Alto, CA,1982.
Environment Simulation: An ArttficialIntelligence NP-2239.
Systemfor Human Performance Assessment, I
l l
7 Glossary i
Abnormal plant behavior Plant behavior that is not the desired behavior for that mode of operation.
1 Adapting procedures Deviating from the literal statement of procedure steps. It includes taking actions that are not stated in the procedures, not taking actions that are stated in the procedures, and taking actions specified in the procedure but changing them in some way (e.g., changing a plant parameter value mentioned in the procedure.)
Data-driven monitoring Operator monitoring that is triggered by a salient external stimuli such as an alarm.
Diagnosis The process of identifying the cause(s) of abnormal plant behavior.
Extra-procedural activity Operator behavior that is not explicitly directed by a specific procedure step.
Knowledge-driven monitoring Operator monitoring that is driven by an internally generated perceived need for a piece of information.
Procedure-driven monitoring Operator monitoring that is determined by procedures that include an explicit directive to monitor a particular plant parameter.
Response planning The process of deciding on a course of action given a particular situation assessmeat.
(
Response plan monitoring Monitoring the effectiveness of the response plan embodied in the EOPs.
Response plan monitoring includes evaluating the consequences of particular actions specified in procedure steps, and evaluating the appropriateness of the EOP procedure path for achieving identified goals.
Situation assessment The process of constructind an explanation to account for observed plant behavior. In the context of NPP operations situation assessment involves developing and updating a mental representation of the factors known or hypothesized to be affecting plant state at a given point in time.
Situation assessment refers to both the process of building the mental representation and the resulting mental representation. It is broader than diagnosis in that it encompasses explanations that are generated to account for plant behavior during all plant conditions, including normal as well as abnormal or emergency conditions.
Steam space leak A leak that results in steam escaping from the pressurizer. Examples of steam space leaks are leaking pressurizer PORVs and leaking pressurizer safety valves.
Glossary Unexpected plant behavior Plant behavior that is unaccounted for by the current situation amessment (i.e., by the known or hypothesized factors influencing the I
plant). Unexpected plant behavior is not necessarily the same as abnormal plant behavior. Plant behavior can be abnormal but not l
unexpected. For example, in a LOCA, the decrease in pressurizer pressure, would be abnormal but not unexpected.
l T
i I
APPENDICES i
l 91 NUREG/CR-6208
Appendix A: Detailed Descriptions of Scenarios 1
I ISLOCA 1 operators to the LOCA inside containment procedure before they got to the point in the EOP procedure that This scenario, which was run at Plant 1, is an checks for ISLOCA symptoms. Once the operators are ISLOCA from the high pressure Reactor Coolant in the LOCA procedure there is no explicit procedure System to the low pressure Residual Heat Removal transition that allows them to get to the ISLOCA (RHR) System. Figure 2.3 provides a simple diagram procedure.49 A diagram of the relevant EOP of the systems involved in the scenario. Figure A.1 procedures and transitions for the plant at which the j
provides a more detailed diagram of the RHR scenario was run is provided in Figure A.2.
system.47 The first alarms that come in are an RHR discharge Two isolation valves between the hot leg loop of the pressure high alarm and pressurizer pressure and RCS system and the RHR system that are normally level low alarms. This results in a reactor trip that kept closed and de-energized begin to leak.48 occurs approximately 30 seconds later. At that point Specifically these were two isolation valves in series the crew is required to turn to the Reactor Trip and SI on the suction side of train A of the RHR system. The Procedure (E-0) in the EOPs (see Figure A.2). They leak into the RHR produces an increase in pressure in reach a step in the procedure that at,ks if the RCS is
)
the RHR, which in this scenario resulted in a break in intact. By that point the PRT has ruptured, resulting the RHR piping in the Auxiliary Building in radiation alarms inside containment. Therefore, approximately five minutes into the event (a 2000 the answer is no, and the EOPs direct a transition to i
gpm leak). This piping break caused reactor coolant the Loss of Reactor or Secondary Coolant Procedure fluid to fall to the floor of the Auxiliary Building (E-1). There is a step later in the E-0 procedure that resulting in Auxiliary Building " misc. sump level checks for Auxiliary Building radiation symptoms high" and radiation alarms.
and if the answer is yes directs them to an ISLOCA procedure, but the operators transition to E-1 before A key element that makes this ISLOCA event they get to that step. Once in E-1 there is no explicit complex is that containment symptoms suggestive of transition to the ISLOCA procedure. There is a step a LOCA inside containment appear early in the event.
that checks for Auxiliary Building radiation. By the This complexity occurs because the RHR system time the operators reach that step they do have includes a relief valve that vents to the PRT which is Auxiliary inside containment. When pressure in the RHR increases, the relief valve opens and fluid is directed to the PRT, which eventually ruptures (approximately three minutes into the event). This action produces radiation alarms within containment suggesting the possibility of a LOCA inside containment. By timing the dynamics of the event carefully it was possible to create a situation where the EOP directed the 47Figure A.1 is a partial schematic of the RHR system. For simplification some components have been omitted.
O ln the postulated event the valve seats on the suction valves fail, O
causing leakage into the RHR system. To increase the credibility lt should be noted that, when we ran this event, there was no of the event we postulate that the valve seat on one of the suction explicit transition from the LOCA procedure to the ISLOCA valves had failed earlier without being detected and that the valve procedure in the EOPs used at that plant, the latest version of the seat on the second suction valve failed at the initiation of the Emergency Response Guidelines (ERGS) includes an explicit event.
transition from the LOCA procedure to the ISLOCA procedure.
REACTOR N,
D=, 5449ee7-E SYSTEM W COMPONENT PRESSURIZER COMPONENT W@EM REUEF Twe(
j COOUNG Q C000NG g
WATER WATER M' H I
e
['}-- +
2 WATER A
(
W t
ACCUuutATOR W
'g j
,__w ECTION SY$TEW h.
REACTOR f
HCV t
COLD LEC L W t COOLANT SYSTEM i e t
1
!l s
(g RHR A
L N
i s
i AcCUuutAtoR l
P T KAT EXCP#4GER D
tWE@ SYSTEu 1
4.CouPONEM
- A*
s COOUNG 3
[
j j
j WATER f
-+
+-- I
- w
! *d O
i
>..I l
i i
REFUELDeG WATER s
j STORAGE TANK
{
t ACCUuutATOR 23 N@ N W
OOP 3
'MM Y###
~'*
p COMPONCNT RELEF Tme(
COOLDeG l
3N CNM 4-ER wAT TE WATER WATER A
j s
ACCuuutAfoe Pv w
- -+
- 2
_r,
,__w l
EoS E".
'E O ' 0, 7
} t' Qy O3 O e
REACT 0*
P
[
cootwT sYsrEu. *, l l
1 y
%;i^-"
le, l g s
HOT LEG LOOP 8
N e
1 l
l 3
NEAT EYbR l
I "S*E ! "
e e
v t
i l
l
+ _ _.s Run
-t Pe ComAmwENT v
i ZC Figure A.1 More detailed diagram of the Residual Heat Removal System. (1) The two isolation valves between the Reactor Coolant 2
System Hot Leg Loop and the RHR that leaked. (2) The RHR heat exchanger with the Component Cooling Water system Q
that broke in ISLOCA 2. (3) The series of check valves between the Reactor Coolant System Cold Leg Loop and the RHR g
system. Back leakage through those check valves provided an alternative hypothesis to explain the inflow of RCS fluid into g
the RHR system.
Reactor Trip or Sa fety Injection Procedure (E-0)
Step 1 Loss of Reactor RCS Intact? NO y
r Sec ndary Coolant "rocedure (E-1)
Check 1l Aux. Building Rad. Normal: NO 4 Step 1 e
SI should be reduced: YES Y
Procedure V
Steam Generator Pressure Stable or Increasing: No (return to step 1)
LOCA Outside Containment Procedure Check Aux. Building Rad.
y Try to ident:fg Normal: NO and isolate y
the leakage Check if RCS Cocidown and DepresNuri~.ation is Required: Yes Post LOCA Cooldown
[
and Depressurization E
Procedure 5
EF S
a Figure A.2 EOP transitions relevant to ISLOCA event at Plant 1.
~
Building symptoms in that case the procedure directs Auxiliary Building to request that the valves be re-them to " identify and isolate the leakage." Thus at energized, to verify that they are closed, and to close that point they are explicitly required to diagnose the them if they are not. In the scenario as we constructed source of the leak and take action to terminate it on it, the valves read closed but were leaking. If the their own. The next steps in the procedure specify a operators called to check on the status of these transition to a Post-LOCA Cooldown and isolation valves they were told that the valves read Depressurization Procedure in order to depressurize closed. In this scenario, the leak is unisolatable and and cooldown the RCS and reduce the leak rate.
the best course of action is to proceed with the cooldown and depressurization, while conserving By the time radiation symptoms appear in the Refueling Water Storage Tank (RWST) level.
Auxiliary Building, the primary symptoms of an RHR problem (i.e., RHR discharge pressure and An alternative plausible hypothesis that would temperature high) are gone because the break into the account for all of the evidence available is that there is Auxiliary Building relieves the pressure buildup in leak back from the RCS through a series of failed the RHR. Therefore,if the crews did not detect the check valves (See Figure A.1). Given this hypothesis, RHR discharge pressure alarm early in the event or the leak could be isolated by closing an isolation valve observe the RHR discharge pressure or temperature on the discharge side of the RHR pump that is symptoms, they had few clues as to the source of the normally kept open. 'Ihis valve isolates the RHR RCS leak into the Auxiliary Building.
system from the Accumulator Injection System Cold Leg Loops. Since the failed check valve hypothesis The only other piece of evidence that the source of the was determined by the instructors at Plant 1 to be leak is the RHR system is the pressure buildup in the equally plausible, in some cases, when crews closed PRT and its subsequent break. If the crew pursues the isolation valve on the discharge side of the RHR potential sources of input into the PRT, it would lead pump, the instructors terminated the leak.
them to suspect the RHR system, since other potential sources of input to the PRT read closed and had no reason to open (e.g., pressurizer PORV and safeties),
ISLOCA 2 and/or are too small to generate the pressure buildup in the PRT that was observed (e.g., CVCS letdown ISLOCA 2 differed from ISLOCA 1 in three main relief; seal water return).
respects. First, the RHR discharge pressure high alarm, which provided the primary indicator of a The combination of evidence of buildup of pressure in problem in the RHR in ISLOCA 1, was suppressed in the PRT and radiation in the Auxiliary Building can ISLOCA 2. This change removed the primary be simply explained by an RHR problem. The indicator of a leak into the RHR system. Second, the alternative is to postulate two independent problems leak into the RHR system led to a break in the RHR to explain the PRT symptoms and the Auxiliary heat exchanger to the CCW system. This break caused Building symptoms, which is less plausible.
abnormal radiation symptoms in the CCW system.
These two factors combined meant that more active Later in the event the crews are given a stronger clue search and integration of evidence was required to that the problem is in the RHR system.
identify the problem in the RHR and conect the Approximately 15-20 minutes into the event, they problem in the RHR system with the problen in the receive a call from the Auxiliary Building indicating CCW system.
that the sumps outside the RHR pump room are flooded.
The third difference between ISLOCA 1 and ISLOCA 2 was in the procedures available. In the case of Once the operators identify a leak into the RHR there ISLOCA 2, the LOCA procedure included a step to are two piausible hypotheses for the source of the transition to the ISLOCA procedure if there was high leak. One is a failure of the two isolation valves radiation in the Auxiliary Building (see Figure A.3).
between the hot leg loop of the RCS system and the However, because of the dynamics of the event,in RHR on the suction side of the RHR pump. This is the some cases the transition step was skipped. In other event that we postulated. Given this hypothesis the cases the criteria for transitioning to the ISLOCA actions required to isolate the leak are to call to the procedure were not met when that step was reached.
Reactor Trip or CCW System Safety Injection Loss of Reactor Malfunction Procedure (E-0) or Secondary Coolant Step 1 Procedure (E-1)
(Off Normal Procedure)
RCS Intact? NO
+ Step 1 Check if Aux. Building SI should be reduced: YES SI Termination Rad. Nc.rmal: NO Procedure Steam Generator Pressure StableorIncreasing: No (return to step D f
Check Aux. Building Radiatton LOCA LOCA Outside Normal: NO U
Conta,nment Procedure Outside i
Containment Procedure V
Checkif RCS Cooldown and Post LOCA Cooldown Depressurization is Requrred: Yes and Depressurization
[
Procedure mo N
lC h
o*
Figure A 3 EOP transitions relevant to ISLOCA event at Plant 2.
This plan allowed us to examine crew performance in the flow path by which contaminated reactor a
cases where the procedure containing relevant coolant fluid reached the CCW system (i.e.,
guidance could not be reached within the EOP contaminated water from the reactor coolant transition network, system reaching the CCW system via the RHR system);
As in the case of ISLOCA 1, two isolation valves between the hot leg loop of the RCS system and the the source of pressure buildup in the PRT (i.e.,
=
RHR system that are normally kept closed and de-via the RHR relief valve);
energized were failed open.50Specifically, these were the source of radiation in containment (i.e., the two isolation valves in series on the suction side of train B of the RHR system. Pressure in the "B" train break in the PRT);
RHR system increases to the point where the suction relief valve begins to relieve to the PRT. Eventually, the source of radiation in the Auxiliary Building the PRT ruptures (within approximately five minutes (i.e., the overflowing CCW surge tanks).
of the reactor trip) causing high radiation levels inside containment.
It also required that they recognize that all these symptoms are due to a single underlying fault (i.e., an The pressure buildup in the RHR eventually causes a RCS leak into the RHR).
tube in the RHR heat exchanger to the CCW to break, causing a high CCW radiation alarm to come in, The first alarms that came in were the pressurizer followed by indications that the CCW Surge Tank level and pressure low alarms, and the PRT level and levelis rapidly increasing. Rupture of the RHR heat pressure high alarms. These alarms were followed exchanger tube reduces RHR pressure to below the within approximately 20 seconds by a Reactor Trip.
lift setpoint of the RHR suction relief valve, thereby causing the LOCA to be redirected to the CCW The primary clue pointing to an RHR problem was System. Eventually the CCW Surge Tanks ovenlow the increase in PRT pressure and subsequent break of causing reactor coolant fluid to spill onto the floor of the PRT. If crews searched for possible sources of the Auxiliary Building, resulting in Auxiliary input into the PRT, it could lead them to identify the Building alarms.51 abnormal RHR discharge pressure. There is an RHR discharge pressure meter in the control room. If the l
Several factors made ISLOCA 2 diagnostically more crew had looked at the meter before the break in the challenging than ISLOCA 1. First, the RHR high RHR heat exchanger occurred, they would have seen discharge pressure alarms were suppressed, so that that it read abnormally high. However, there were no l
crews did not get an early indication of an RHR alanns or procedure steps that would direct the problem.52 Second, greater understanding of the perator to look at that meter.
system interconnections and potential flowpaths Another difference between ISLOCA 1 and ISLOCA 2 among the systems is required to connect the symptoms observed in the different systems. A is in the procedures available to support the perators. While overall the EOPs at Plant 1 and complete situation assessment required that the crews
- identify, Plant 2 were similar m structure and content, there were some differences. Figure A.3 provides a graphic representation of the relevant procedures at Plant 2.
SOAs in the case of Plant 1, although the two valves read closed There are two main differences to note. One they were stuck open and leaked.
difference is in the step in the Loss of Reactor or
[
The simulated scenario was timed so that Auxiliary Building Secondary Coolant Procedure (E-1) that checks for SI radiation symptoms did not appear until af ter the crews passed the Auxiliary Building radiation symptoms. At Plant 2, if j
step in E-1 that checks for Auxiliary Building symptoms. This the Auxiliary Building radiation is detected, the was the case for all but one of the crews-operators are directed to the LOCA outside containment procedure, whereas at Plant 1, the
$2The fact that the RilR discharge pressure high alarm did not operators were directed to identify and isolate the come on. is an important dif ference between how the event was run at Plant I and at Plant 2.
leak on their own. In principle, this means that the crews at Plant 2 had a procedure path by which to get 98 NUREG/CR-6208
I Scenario Descriptions to the ISLOCA procedure from the Loss of Reactor or being serviced by the CCW) is aligned to the affected Secondary Coolant Procedure (E-1). However, due to CCW train. Ifit is, as was the case in this scenario, the the dynamics of the event, that transition was not procedure checks for possible sources of leakage into always accessible. One reason is that it was possible to the CCW from the service loop (e.g., excess letdown transition out of the E-1 procedure to the Si heat exchanger, RCP Thermal Barriers). It then has Termination procedure before reaching the step in E-1 the operators transfer the service loop to the that checks for Auxiliary Building radiation. As is unaffected CCW train. The procedure then checks shown in Figure A.3, it was possible to transition whether level continues to go up in the CCW train from E-1 to the S1 Termination procedure before the that was previously aligned to the service loop. If the Auxiliary Building radiation step was reached, and CCW surge tank level continues to go up, it means then to transition directly from the S1 Termination that the in leakage is not from the service loop. It can procedure to the Post-LOCA Cooldown and then be concluded that the in-leakage is from the Depressurization Procedure. In this way the step Safety Train - specifically the RHR heat exchanger, checking for Auxiliary Building radiation was never The procedure then directs the operators to isolate reached.
the RHR heat exchanger but does not indicate which valves to close. In fact, there are two valves that can it was also possible that when the step checking for be closed to isolate the RHR heat exchanger. One Auxiliary Building radiation in E-1 was reached there valve is on the CCW side of the heat exchanger. A was no radiation in the Auxiliary Building because second valve is on the RHR side of the heat exchanger the CCW surge tank had not yet overflowed. As a (between the discharge side of the RHR pump and the result the criteria for transitioning to the ISLOCA heat exchanger).
procedure were not met.
Loss of Heat Sink 1 (LHS 1)
Thus, as in the ISLOCA 1 case, the dynamics of the event created a situation for many of the crews where The Loss of Heat Sink event involved a totalloss of there was no procedurally directed way to reach the feedwater flow complicated by a leaking pressurizer ISLOCA procedure.
PORV. Figure 2.4 provides a simple diagram of the systems involved in the scenario.
A second difference in terms of procedures available, is that at Plant 2 there was an abnormal operating In the simulated event both the auxiliary and main I
procedure available to support identifying and feedwater systems are made unavailable so that isolating the leak into the CCW: CCW System operators are forced to use the condensate system to Malfunction procedure. While this procedure is not supply feedwater. This change requires that they part of the EOPs,it could be consulted. This depressurize the RCS and block SI signals. The proced ure provides a clearly laid out logic for auxiliary spray valve is shut so that the operators identifying and isolating the source of an outside leak have to use the pressurizer PORV to depressurize the into the CCW. If the crews chose to consult that RCS. After they open and close the pressurizer procedure, and followed it correctly, it would enable PORV, it starts to leak (though the pressurizer PORV them to localize the problem to the RHR heat indicator reads closed). One of the key features of this exchanger. We were interested in whether crew situation is that Si is blocked and must be started members would consult the off normal procedure for manually to deal with the PORV leak. However, the guidance, and if not, whether they would use a main focus of operator attention and the procedures is similar line of reasoning in identifying and isolating on recovering a secondary heat sink. This creates a the leak.
situation that allows us to examine how operators discover and handle an unexpected second fault.
Figure A.4 provides a graphic rep,resentation of the
}
main logic of the CCW System Malfunction procedure. The procedure begins by checking i
whether the level in the CCW surge tanks is going up.
If, as in this case, the CCW surge tank level is going up, it indicates a leak into the CCW. The procedure then checks whether the service loop (i.e., the systems 99 NUREG/CR-6208
CCW System Malfunction Step 1 Determine if the in-leakage isfrom the service loop and attempt to isolate:
- excess letdown heat exchanger
- letdown heat exchanger
- RCP thermalbarriers Transfer the Service Loop to Ihe unaffected train Check CCW surge tank levelin both trains iflevel continues to increase uncontrollably on safety train prwiously on semice then isolateleak on Safety train:
In this version of the event the crews are never given resulting in a reactor trip. This automatically causes feedwater back. As a result they remain in the Loss of the main feedwater pumps to trip and the auxiliary Secondary Heat Sink procedure where the only feedwater system to come on. At this point the crew guidance available for dealing with the leaking PORV is required to go to the Reactor Trip and SI procedure I
is in a caution that states that manual initiation of SI in the EOP (E-0). Four minutes later a turbine-driven may be required if plant conditions degrade.
auxiliary feedwater pump high temperature alarm comes on. Three minutes after that the turbine-driven At the start of the scenario the plant is at 50% power.
auxiliary feedwater pump trips. At this point the The crews are told that the B motor-driven auxiliary crew is in a total loss of feedwater event.
feedpump and the motor-driven main feed startup pump are tagged as being out of service. The crews According to the rules of usage of the EOPs, the are asked to increase load at 10% per hour. Five crews are required to go through the E-0 until they -
minutes into the event,.m earthquake occurs are transitioned to an emergency guideline procedure.
100 NUREG/CR-6208
Scenario Descriptions Because the reactor trip was due to an earthquake, procedure, initiating plant parameters were tuned so the transition from E-0 is to a Reactor Trip response that SG levels never reached bleed and feed criteria.
procedure. Once they are transitioned to an emergency guideline procedure, the STA is required The procedure first has the operators try to re-to begin monitoring five prioritized critical safety establish auxiliary feedwater flow. These attempts functions: subcriticality; core cooling; heat sink; fail. (The crew is told that the turbine-driven auxiliary integrity; and containment. Since all feedflow is lost, feedpump has a bearing problem and that the motor-the heat sink safety function is violated, and the crews driven auxiliary feedpump has a breaker problem and are directed to the loss of Secondary Heat Sink that they will take an extended period to fix.) The function restoration guideline. Figure A.5 provides a procedure then directs the crew to attempt to start the diagram of the relevant procedures and procedure main feedwater pumps. This attempt fails as well.
transitions for this event.
The pumps trip, and the crew is told that they cannot be restarted.
Function Restoration Guidelines (FRG) are intended to restore critical safety functions and thus they have At this point the procedure directs the operators to try a special status. Crews are required to stay within the to establish feedwater via the Condensate System. A FRG procedure until the critical safety function is number of steps are required to establish feedflow recovered or violation of a higher priority safety through this system. Because the condensate pumps function is identified.
operate at a lower pressure, the SGs need to be depressurized.54 Before this can be done, the RCS The Loss of Secondary Heat Sink procedure guides system needs to be depressurized to less than 1920 the crew to attempt to restore feedwater through a psig. This is accomplished using either the auxiliary number of alternative means. These attempts require spray or the pressurizer PORV. In the event as we ran extensive interaction via phone communication with it, the auxiliary spray failed to come on so that the personnel outside the control room. In the training crew was forced to use the pressurizer PORV. In this simulator, these interactions are simulated by having event when the PORV is closed, it never completely training instructors act as auxiliary crew.
rescats. As a result, although it reads closed, there is a small leak from the pressurizer PORV from this point The procedure directs the operamrs to first attempt t on in the event.55 re-establish auxiliary feedwater flow. If this attempt fails the operators are directed to try to establish main As part of this step in the procedure the crews are also feedwater flow. If this fails they are directed to try to required to block signals for automatic actuation of SI.
establish feedwater flow through the condensate This is done to avoid spurious SI when the steam system. As a means oflast resort the crews are generators are depressurized.56 This action has directed to use bleed and feed to provide cooling. This method involves tnitiating SI and then opening the potentially serious consequences since it means that a pressurizer PORV. Since this method involves major automatic safety actuation rystem is no longer mtentionally creating a break in the RCS system it is in operation and must be manually initiated if the least preferred alternative. The procedure needed. To emphasize this point, a caution appears specifies that if at any point in the event wide range prior to the step directing the crew to block S1 that level in any three steam generators (SGs) is less than a states, " Following block of automatic SI actuation, specified value, or pressurizer pressure is greater than or equal to a criterion value due to the loss of heat sink, then a bleed and feed must be initiated 54De exact SG pressure value varies frc n plant to plant but is in immediately.53 the range of 550 psig.
55 In this event feedwater is never re-established. Since A 5% PORV leak with a 500 second ramp was used at Plant 1.
we did not want the crews to go to a bleed and feed ne leak was insened at the point when the crew started to depressurize a steam generator.
56 SI Signals for low steamline pressure and low pressurizer 53 ne values vary from plant to plant.
pressure are blocked.
101 NUREG/CR-6208
Scenario Descriptions manual Si actuation may be required if conditions pressurizer behavior can no longer be attributed to degrade."
cooldown due to secondary side activity.
The event was specifically designed to place the crew At some point both the pressurizer level and the in a situation where a leak through the pressurizer pressure decrease sufficiently that subcooling is lost.
PORV would cause RCS conditions to degrade. A At that point, RCS begins to become superheated and goal was to determine whether crews would choose a bubble forms in the reactor vessel. As a result the to manually safety inject.
level in the pressurizer starts to go up, while the level in the reactor vessel starts to go down. The reactor Once the RCS is depressurized, and the SI signals are vessel level indicator (RVLIS) in the control room blocked, the crews are directed to depressurize at provides an indication of reactor vessellevel. A least one SG and then establish condensate flow to RVLIS value of 100% or less with pressurizer level that SG. Establishing a condensate flow path involves going up indicates a bubble in the reactor vessel. The a number of steps that require actions to be taken by pressurizer level going up while the pressurizer auxiliary operators. First, jumpers must be used to pressure continues to decrease provides a similar bypass the fast close action on the feedwater isolation indication. This combination of symptoms cannot be valves. This bypass requires that Instrumentation and explained by a cooldown. They can only be explained l
Control technicians come in and physically jumper by a leak out of the pressurizer: Possibilities include a the valves in the control room. An equipment leak out of the pressurizer PORVs or a leak out of the operator must then be dispatched to manually jack pressurizer safety valves. These are collectively open the feedwater regulating valve for the desired referred to as a steam space leak. Since the pressurizer steam generator. In addition, a discharge valve on at PORVs read closed, there is no direct evidence to least one of the main feed pumps must be opened.
discriminate between a leaking PORV hypothesis and a leaking safety valve hypothesis.
1 In this event we did not want feedwater to be re-
/
established. This situation was accomplished by If the leak is not terminated, pressurizer level introducing delays in getting the feedwater regulating continues to go up and the pressurizer eventually valves open. Examples include auxiliary operators becomes completely filled with water (i.e., goes solid).
going to the wrong valve (the feedwater isolation valve instead of the feedwater regulating valve);
Another indication of a steam space leak is activity in auxiliary operators being detained by health physics the PRT. Since the pressurizer PORV is opened to because of having been exposed to radiation; auxiliary depressurize the RCS, some activity in the PRT is operators being unable to manually jack open the expected. However, since the PORV continues to leak valves; and auxiliary operators breaking the into the PRT, symptoms continue to increase even i
feedwater regulator valve actuators. As a result of after the PORV is closed. Eventually, the PRT i
i these delay tactics a great deal of operator activity ruptures resulting in radiation symptoms in f
involved calling the operators in the Auxiliary containment. The rupture of the PRT cannot be Building for status reports and receiving calls.
explained by the cooldown hypothesis.
The Loss of Heat Sink procedure focuses operator One set of questions in this scenario concerned attention on the secondary side and recovery of identification by the crews of a problem in the RCS. A feedwater. In the meantime there is a secondary fault:
second set of questions concemed what actions,if l
a leak on the primary side through the pressurizer any, crews decide to take to deal with the leak in the PORV that manifests itself in a number of ways.
RCS. One option available is to close the PORV block valve. This would terminate the leak.57 Another The first symptoms are a decrease in pressurizer level and pressure. Since the crews are depressurizing the 57 SG around this time, at first the pressurizer level and 1n this scenario, closing the block valve did not terminate the pressure behavior can be attributed to a cooldown leak for all crews. In some cases the leak was continued caused by depressurizing the SG. Once the SG (p stulating a leak in the block valve as well)in order to allow us depressurization is completed, however, the P.obec whether the crews would choose to manually safety inject as RCS conditions contmued to degrade.
NUREG/CR-6208 102
Scenario Descriptions option is to initiate a manual SI. While the Loss of feedwater flow complicated by a stuck open Heat Sink procedure did not include an explicit pressurizer PORV. As in LHS 1, the crews were criterion for a manual SI,it did include the caution unable to recover either auxiliary feedflow or main that manual SI actuation may be required if feedflow. As a result they had to use the condensate conditions degrade. This caution was intended to system. However, LHS 2 differed in two essential l
allow crews to manually initiate Si at their discretion.
respects from LHS 1. First, if the crews decided to l
close the PORV block valve the leak was terminated.
)
Given an undetected or uncontrollable steam space Second, eventually the crews were allowed to recover l
leak, there are several arguments in favor of manual feedwater and return to the procedure which they initiation of SI. First, the leak through the pressurizer had been following, which in this case, was the PORV is a LOCA. Unless the leak is isolated (e.g., by Reactor Trip Response procedure.
closing the pressurizer PORV block valve) an SI will eventually be required. If the crew successfully The fact that the crews transitioned to the Reactor recovered feedwater they would be transitioned back Trip Response procedure provided the opportunity to to the procedure they were following. Once in that observe how crews responded when they reached a procedure, they would immediately meet the criteria step in the procedure that appeared inappropriate for for manualinitiation of SI. If the crew did not recover the current situation. Several steps in the Reactor feedwater, they would eventually meet the criteria for Trip Response procedure were inappropriate to a bleed and feed, which itself involves initiating SI; follow literally given that they had just transitioned thus whether feedwater is recovered or not, unless the from a Loss of Heat Sink procedure and that they leak is terminated, the crew would eventually have to were using the condensate system to feed the steam initiate SI.
generators. These steps required the operators to reverse actions that they had intentionally taken as There are also arguments against manual initiation of part of the LHS procedure. EOP background SI. One is that there is no specific procedure directive documents explicitly recognize that some of the steps to initiate SI. Another argument is that initiating SI may be inappropriate when returning from a critical could result in delay of recovery of feedwater, function restoration procedure, such as the Loss of possibly causing the crew to have to go to a bleed and Heat Sink procedure, and state that operator feed which is undesirable.
judgment may be needed under these circumstances.
We wished to evaluate how the crews responded to If the operators take no action on their own, these steps.
conditions in the RCS will continue to degrade.
Eventually, reactor vessel level would decrease to less The initiating conditions for LHS 2 also varied slightly than 40%. At that point, based on the " red path" core from LHS I. As in LHS 1, at the start of the scenario cooling critical safety function criteria, the EOPs the plant was at SO% power. The crews were told that would direct the operators to a core cooling function the "B" essential service water, the "B" auxiliary restoration procedure designed to respond to loss of feedwater pump, and the "B" diesel generator were core cooling. However, by that point conditions in out of service.
the RCS would be significantly degraded with increased risk of core damage.
The main steam isolation valves inadvertently close I
causing a reactor trip. This action plays the role that Once the crew identified a problem on the primary the earthquake played in LHS I. The crews go to the side, and determined a course of action,if any, the Reactor Trip and SI procedure in the EOP (E-0). When simulation was terminated. As a result the simulation the turbine driven auxiliary feedwater pump starts, was terminated before the loss of core cooling safety the coupling between the turbine and the pump fails, function " red path" criterion was reached.
preventing feedflow. The "A" motor-driven auxiliary feedwater pump also fails due to a seized bearing.
This action results in a total loss of feedwater.
Loss of Heat Sink 2 (LHS 2) l The crews continue through E-0 until they are i
Loss of Heat Sink 2 was similar in most respects to the transitioned to the Reactor Trip Response procedure.
Loss of Heat Sink 1 scenario. There was a total loss of At that point the STA begins monitoring critical safety 103 NUREG/CR-6208
4 l
l Scenario Descriptions l
functions. Since all feedflow is lost, the heat sink Once feedwater is restored the Loss of Heat Sink safety function is violated, and the crews are directed procedure directs the crews to return to the procedure t
to the Loss of Secondary Heat Sink function that was in effect when feedwater was lost. In this restoration guideline. Figure A.S provides a diagram case it was the Reactor Trip Response procedure. The of the relevant procedures and procedure transitions Reactor Trip Response procedure includes a number for this event, of steps that are not applicable given that the steam
[
generators are being fed via the condensate system.
'Ihe procedure first has the operators try to re-Figure A.5 specifies these steps.
establish auxiliary feedwater flow. 'Ihese attempts fail. The procedure then directs the crew to attempt to One step asks the crew to verify that the feedwater start the main feedwater pumps. This attempt fails as isolation valves are closed. If they are not the EOP well. The main feedwater pumps are tripped and specifies that they should be closed. In fact, the flow cannot be reset.
of feedwater through the condensate system requires that the valves be open.
At this point the procedure directs the operators to try to establish feedwater via the Condensate System.
Another step has the crew check that pressurizer This step requires the RCS system to be depressurized pressure is greater than 1830 psig. If it is not, the step to less than 1920 psig. As in LHS 1, auxiliary spray directs the crew to manually actuate SI. The fails to come on so the crew will have to use the pressurizer had been intentionally depressurized to pressurizer PORV. When the PORV is closed, it never less than 1920 as part of the Loss of Heat Sink completely reseats. As a result, although it reads procedure. Pressurizer pressure tended to be below closed there is a small leak from the pressurizer PORV 1830 psig, partly due to cooldown, when that step in from this point on in the event. However, if the crew the Reactor Trip Response procedure was reached.
closes the pressurizer PORV block valve the leak is However, a manual initiation of SI would have been terminated.
inappropriate in these circumstances.
As in the LHS 1 event the EOPs direct the crews to Finally, the foldout page for the Reactor Trip i
block SI signals. A caution appears immediately prior Response procedure specified criteria at which SI to this step indicating that manual initiation of SI may should be immediately actuated. These criteria were:
j be required as conditions degrade.
(1) if pressurizer level cannot be maintained greater i
than 4% or (2) if RCS subcooling was less than 30 Once the RCS is depressurized, and the SI signals are degrees F. When the crews returned to the Reactor blocked, the crews are directed to depressurize at Trip Response procedure from the Loss of Heat Sink least one SG and then establish condensate flow to procedure, in many cases pressurizer level was less that SG. This sequence involved the same activities as than 4% due to cooldown. The feedflow into the in LHS I. The one difference is that the feedwater steam generator from the condensate system resulted regulating valves are eventually opened and in a large cooldown on the primary side which caused feedwater is restored, a shrink in pressurizer level. While the pressurizer level met the SI criteria, the behavior of the pressurizer was not abnormal under those conditions.
NUREG/CR-6208 104
Response to Reactor Trip y
L ss of Secondary Reactor Tn.p or
> Response Eleat Sink Procedure Safety Injection Procedure Procedure (E-0) ry to establish aux.feedflow Red path on Heat ink: Yes Step 1 Try to establish mainfeedflow Check ifSiis required: No Venfyfeedwater A
Caution: Following block ofautomatic isolation tulves SIactuation, manual SIactuation may be CLOSED required ifconditions degrade.
y Try to establishfeedJlowfrom Pressurizerpressure Condensate System greater than 1830:
depressurize RCS to less than NO-Venfy Siactuation 1920psig (use one PRZR PORV if aux. spray Foldout page not atuilable)
SI Criteria:
block SI si:nals
- Pressurizer level depressurize at least one SG to cannot be
=
maintained greater less than Sxx psig.
than 4%
2
x less than 30 deg.
return to procedure and step in effect Nx Figure A.5 EOP transitions for Loss of Heat Sink event. The Reactor Trip procedure is from Plant 2 and was used in the LHS 2 scenario.
Appendix B: Instructions to Study Participants and 5 ample Summary Sheets 107 NUREG/CR-6208
l Instructions Instructions to Study Participants The simulator exercises you are about to participate in will be used as part of a research project being conducted by the Westinghouse Science & Technology Center for the NRC's Research Office.
The purpose of the research is to understand the decision making process involved in diagnosing and responding to challenging postulated accident scenarios. Many of the actual nuclear power plant incidents, for example, Three Mile Island, have had twists or complications that made the events challenging to handle..As a result, the research team is interested in how operators use their knowledge, training, procedures and any other resources available to them in handling similar situations. They are trying to develop a computer simulation that responds to accident scenarios much as operating crews would. The long term goal is to use this computer simulation to predict situations that are likely to be challenging, and to help define aids - training, procedures, or control room displays - that could help operators in these situations.
The members of the research team are:
Dr. Emilie M. Roth, a human factors psychologist at the Westinghouse Science & Technology Center and principal investigator on this project.
Dr. Harry Pople, Jr., a computer scientist from the University of Pittsburgh and the developer of the computer simulation.
Dr. Roth and Dr. Pople are contractors to the NRC for this research project. Dr. Paul M. Lewis, of the NRC Office of Nuclear Regulatory Research, is the contract monitor for the project. His role is to ensure that the activities of the research contractors remain within the scope of the contract.
Today you will be participating in two simulated accident scenarios that have been designed to be challenging.
flandle these events as you would if they were actually happening in the plant. Use all of the resources available to you - anything you would use in a real situation to mitigate the event.
{
1 It is important to emphasize that this is strictly a research project and you are not being evaluated. The events are expected to be challenging and performance may not always be successful. Any problems that may occur are expected. There are no pass / fail criteria, merely an observation of your decision making process. The results of these exercises are not to be used as a means for evaluating individual operators, operating crews, and the plant.
j This is understood and agreed to by Westinghouse, the NRC, and the utility management.
i As part of this research program, data will be collected on how actual operating crews respond to challenging scenarios. The idea is to understand how operators handle these events and to use that as input in developing and testing the simulation program that is being built.
The simulator scenarios will be videotaped so that they can be reviewed later to understand what happened in more detail. The videotapes will belong to the plant and are being made solely for this research project. They will be borrowed by Westinghouse for analysis and then returned directly to the plant. According to the agreement with the plant, the only people who will see the tapes are the Westinghouse researchers and Dr. Paul Lewis. No one else will be provided access to the tapes without explicit permission from the plant.
The research team will largely rely on the communication among the crew to keep track of the event. They are especially interested in your thoughts about what is happening to the plant, what you are concerned about, and i
1 109 NUREG/CR-6208 l
Instructions what you are trying to do to respond to the situation as the event progresses. It would be helpful if you verbalize your thoughts and actions as you address these situations.
Following each scenario there will be a short debriefing to recap what happened during the event. You may be asked questions about what you thought was happening, your concerns, and why you took the actions you did.
Based on the analysis of the recorded data, written descriptions of the events run and how they were haadled will be generated. Rese descriptions will be used as input in developing and testing the simulation program. ney may also be included in articles for professional society journals. Every reasonable effort will be made to preserve the anonymity of the crew participants. The reports will not mention the plant at which the exercises were run.
Are there any questions? (Answer questions)
If there are no more questions, before we start,I would like to ask whether any of you have heard anything about the accident scenarios that we are using in the study. Without telling me what you think the event is, can each of you answer "Yes, I have," or "No, I have not hear i what the accident scenario is, or any other information that might help an operator respond to the event?"
[Ask each operator individually to say whether they have heard what the events are. If anyone says yes they have heard pertinent information, take them aside and have them tell one of the instructors what they have heard.
Without naming particular individuals who might have told them about the experiment, ask them to describe in general how they found out about it. If the accident scenarios described by the operator are the correct ones, then we might have to cancel the session. If his/her description of the scenarios is incorrect or so vague as to encompass many potential scenarios, we can let him/her know that these are not the events we will be running, and ask if he/she is still interested in participating.]
[After this ask: "Is there anything else about the experiment you have heard that you want to mention?"]
At the end of the two scenarios the participating operators will be told:
Thank you for participating in the research. The exercises you have just participated in will be very useful to us in developing a computer model of how crews handle challenging scenarios. Hopefully this computer model will help us provide improved means to support you in doing your job.
We will be running these same accident scenarios with other crews of operators at this plant and possibly also at other plants. We want to make sure that operators who might participate in the research do not find out ahead of time any information about the scenarios or how they might be handled. As a result we ask you to please not discuss your experience in the research, and especially the scenarios we ran, with others who have not yet participated in the research. Your cooperation in keeping these scenarios confidentialis critical to ensure that the results of the research are meaningful and helpful.
Thank you again for your cooperation.
1 NUREC/CR-6208 110
l 1
Sample Summary Sheet l
l-Sample Summary Sheet:
ISLOCA Case CREW DATE I
L I
Years Activities Other I
Li d In Last Year Exn/Educ.
Did any crew member have prior knowledge of the event? Y/N l
Which position?
2.
Did event sequence go as planned? Y/N 3.
Did the operators:
Identify RHRdisturbance? Y/N a.
- b. Connect RHR/PRT symptoms? Y/N c.
Connect RHR/ AUX BLDG symptoms? Y/N
- d. Connect RHR/CFMTsymptoms? Y/N c.
Attempt to close RHR valves? Y/N f.
Stay in LOCA procedure? Y/N g.
Transition /Use LOCA Outside Containment procedure? Y/N
- h. Transition /Use Loss of Emergency Coolant Recirculation procedure? Y/N REMARKS:
I t
111 NUREG/CR-6208 s
=
Sample Summary Sheet Sample Summary Sheet:
Loss of IIeat Sink Case CREW DATE Years Activities Other Licensed In Last Year Em/Educ.
SS SO RO BOP STA LOSS OF HEAT SINK 1.
Did any crew member have prior knowledge of the event? Y/N Which Position?
2.
Did event sequence go as planned? Y/N 3.
Did the operators:
Identify abnormalPZR Pressure? Y/N a.
b.
Identify leaking PZR PORV7 Y/N c.
Activate Sl? Y/N
- d. Correctly manage Loss of Heat Sink? Y/N e.
Bleed and Feed? Y/N REMARKS:
NUREG/CR-6208 112
Behaviorally Anchored Rating Scales Appendix C: Behaviorally Anchored Rating Scales (BARS)
Introduction and Source of BARS Attached are copies of the draft Behaviorally Anchored Rating Scales (BARS) of team interaction skills developed by Montgomery et al. (1992) under the sponsorship of the U. S. Nuclear Regulatory Commission. These rating scales are taken directly from a draft report by Montgomery et al. (1992). The BARS scales were used in this study to rate crew interaction skills in the simulated scenarios.
ll' l
l 113' NUREG/CR-6208
a
..s..
n-n....- - -
a-ty,..
Behaviorally Anchored Rating Scales GUIDELINES FOR MAKING BEHAV10 RALLY ANCHORED RATINGS j
-I General Guidelines Consider how crew members interact.
Focus on the crew as a team.
Avoid making ratings based on a single crew member.
a Facts about the Behavioral Frequency Ratings Scales i
A 7-point scale is used.
l For these ratings you will consider how c
beha s on a given dimension.
Guidelines for Making Ratings Carefully read the anchors for ach a in eractica skills dimension.
If the crew always behaved in the m er scribed in the statement, then circle the number above that ratt If the crew did not beh e in act man described in the anchor, circle the mos ap riate nu between the two anchors that best describe the ew' be
- ior, member, for each dimension a number on the scale mus be i c1 Example of the 7-Point Behutiorally cho ed Rating Scale Low g
.High v(m 2
4 S
6 7
Low Anchor erage Anchor High Anchor I
l NUREG/CR-6208 114 i
l
i Behaviorally Anchored Rating Scales l
i BARS -- COMMUNICATIONS LOW AVERAGE HIGH 1
2 3
4 5
6 7
LOW AVERAGE HIGH Provide insufficient Provide plant status aintain constant information about updates to one anoth-awareness of plant,
plant status and plans er; generally appear tatus and plans for for stabilizing the aware of plans for st ilizing the plant.
pl ant. Crew members stabilizing the pia rew. mbers transmit are difficult to Crew members tra s. t fa ual 'nformation in understand when factual inform io a ci d concise transmitting factual that is mostl cl ar manner.
Communica-information.
They and concise bu occ tions including fac-seldom acknowledge the sionally i di u
tual information are receipt of factual to understan toi-always verbally or information.
Communi-munications inv vin nonverbally acknowl-cations include a high factual informatio edged by recipients proportion of non-are lly or non-(e.g., "I understand,"
task-relevant informa-ver ally ac
- edged or waving a hand or
- tion, b rec' nts abou making the "OK" sign).
hal of he
'me l
BARS -.0_penness LOW A
E HIGH wv 1
2 3
4 5
6 7
LO AVERAGE HIGH Crew.em rs sug-Task-focused sugges-Crew members request gest' ns o feedback l tions or feedback are suggestions and feed-inclu ers 1
provided, but seldom back.
Responses are reference Cr request such informa-task-focused.
Crew members rec e sug-tion. Crew members' members receive sug-gestions or fe a
reactions to feedback gestions and feedback by regularly inte -
are mostly positive, in a positive manner rupting or replying but occasionally may (e.g., thank the sarcastically.
be negative (e.g.,
sender).
"Get off my case").
115 NUREG/CR-6208
Behaviorally Anchored Rating Scales BARS -- TASK C00RDTHATION LOW AVERAGE HIGH 1
2 3
4 5
6 7
LOW AVERAGE HIGH
~
Resources within the Staff and resources rew members use control room are within the control available and appro-allocated without room are used effec-iate resources both considering the task.
tivelpmostofthe wit n and outside the Consult procedures, time.
Future needs ntr room.
Crew me rs nsult pro-but do not rely on are neither anti them to guide pated nor consi re.
cedur en neces-responses.
sary.
rrectly anticipate future needs and activities.
BARS -- TEAM SPIRIT LOW AV Q HIGH V"
1 2
3 4
5 6
7 LOW AVE 1G HIGH Crew members seem Cre niembe hesitate Actively and willingly unable to recogn i he in each other, cooperate in crew when another crew erbal and nonverbal activities.
Verbal member needs assis-port for crew mem-and nonverbal support tance. Verb on-ber present only for team members is verbal sup rt for ur' g normal oper-provided during normal crew mem rs dom at ng conditions.
operating and emer-expres. d der any gency conoition's condi on. Disagre (e.g., "That's okay, ment re t unre-we can take care of solved are ' nor this," or " Good work, I needed that.").
I NUREG/CR-6208 116
Behaviorally Anchored Rating Scales BARS -- MAINTAINING TASK FOCUS IN TRANSITIONS LOW AVERAGE HIGH 1
2 3
4 5
6 7
LOW AVERAGE HIGH Crew members express Crew members tend to hen a novel or un-anger or frustration wait and adapt slowly usual event occurs, to each other when to pl. ant conditions.
c w members discuss novel nr unusual Options are discus d.
opti s calmly, thor-y conditions occur.
Crew members expr s hly, and rapidly.
some frustratio, i Voi s re ain the same i
is not directe a as whe ormal condi-another crew iem r.
tions occur (calmly).
Anger,. frustration, or tension cannot be detected.
BARS -- ADAPTABILITY LOW EP HIGH
\\
l 2
3 4
5 6
7 LOW AV.GE HIGH After a change in fter a change in After a change in plant conditions, crew p nt conditions, crew plant conditions, crew.
members occ mem rs may recognize members immediately recogniz he need t need to change, rec.ognize the need for
- change, ri itie ay often change prior-change and rapidly or may no change, a d ities slowly, and shifts priorities to some or ssignment; change work assign-reflect changing and i
event lly ange, b t
ments only after a rapid adjustments in others not.
significant period of work assignments.
time elapses.
117 NUREG/CR-6208
I l
Appendix D: A Cognitive Demands Checklist This project has provided evidence for the role of cognitive activities in guiding operator performance in complex accident scenarios. We developed a Cognitive Demands Checklist that is intended to capem some of the findings of the project in a form that can be used directly by the NRC staff to assess characteristics of an accident sequence or situation (e.g., characteristics of the event, the procedures, or the man-machine interface) that make errors of intention more likely.
The checklist provides a structured list of factors (e.g., characteristics of the event, the procedures, the man-machine interface) that can result in errors ofintention (deciding to take a wrong action). It also includes factors that can contribute to errors ofexecution (intending to take the correct action but executing it incorrectly). The structured list is guided by the model of cognitive performance that underlies the CES simulation, and the results of the empirical analysis of crew performance in simulated emergencies. The results of both the CES simulation efforts and the empirical study of crew performance emphasize the importance of situation assessment and the expectations derived from this situation assessment in the formation of operator intention. The checklist also draws on the Rasmussen model of operator performance (Rasmussen,1986) as well as other cognitive psychology literature on decision processes and decision biases (Kahneman, Slovic and Tversky,1982) and error classification schemes (Norman,1981; Reason,1990).
The list is primarily targeted at operator performance during emergency situations where performance is guided by Emergency Operating Procedures. It contains a list of factors that are likely to help performance as well as factors that are likely to hinder performance.
The Cognitive Demands Checklist can be used by NRC staff members as a " check list" to identify situations that can lead to cognitive errors. The checklist can be used:
(1) to establish a protocol for use by an onsite incident investigation team; (2) to identify common psychological root causes across different incidents; (3) to investigate potential cognitive sources of error as part of a human reliability analysis; (4) to design / classify / calibrate accident scenarios used in simulator training and testing with respect to cognitive (i.e., " thinking") skills being exercised and level of difficulty; (5) to evaluate the potential impact of proposed changes in M-MI, training, or procedures on cognitive performance.
The Cognitive Demands Checklist can be incorporated as part of HRA analyses that use more traditional approaches for quantification of probability estimates such as THERP (Swain and Guttmann,1983 ) or the Human Cognitive Reliability (HCR) model (Spurgin, Moieni, Gaddy, Parry, Orvis, Spurgin, Joksimovich, Gaver, and Hannaman,1990). The checklist can be used as a screening tool to identify situations that may lead to cognitive error. These can then be analyzed in more depth, for example, by running crews through the events using training simulators as recommended by the HCR model, or by using expert judgment techniques such as SLIM-MAUD. A similar approach has been recommended by Beare, Gaddy, Parry, and Singh (1991) as an adjunct to the HCR model.
119 NUREG/CR-6208
Behaviorally Anchored Rating Scales Cognitive Demands Checklist L. Detection / Observation Will the operator detect abnormal plant indications?
Help: Symptoms salient / alarmed
.._ indicators are alarmed
_ parameter is highly salient (i.e., position; size; discriminability)
__ target and upper and lower bound values for parameter are displayed l
__ other indicators are quiet (no other alarms) when indications occur
__. operator workload is low when indications occur Help: Operator has reason to check parameter
__ parameter is routinely monitored l
_._ procedures direct operator to monitor this parameter
__ hypotheses currently entertained suggest relevance of monitoring parameter Hinder: Symptoms not salient
__ indicators are not alarmed
__ indicators are not located near likely operator positions (e.g., located on a back panel or outside the control room)
_._ indicators are difficult to read out
_ other alarms occur at the same time
_ operator workload is high when indications occur Hinder: Operator has no reason to check parameter
_ parameter is not routinely monitored
__ procedures do not direct operator to monitor this parameter
__ parameter not relevant to hypotheses currently entertained l
Hinder: Symptoms / indications are masked or obscured
_ misleading indications exist (e.g., sensor failure; M-MI displays demand position rather than actual position)
__ other malfunctions occur to obscure or mask primary event
_ other manual or automatic system action occurs to obscure or mask primary indications (e.g., shrink and I
swell)
_ symptoms may not yet be present or may have dissipated at point in procedure where request to monitor parameter is made 121
Behaviorally Anchored Rating Scales Hinder: Identifying indicator tulue as abnormal requires mental effort (e.g., memory recall, mental calculations)
_ _ target and upper and lower bound values for parameter are not displayed
__ mental calculation required (e.g., comparison of severalindicators; calculation of rate)
_ knowledge of special context required (e.g., setpoint shift)
- 2. Situation Assessment-Explanation of Obsgrysd Plant Behavior Will the operator develop the correct interpretation of plant state?
Help: Explanation will be called to mind as afunction offamiliarity (fregaency); recency; perceived likelihood; and representativeness ofsymptoms
_ symptoms / indications are very clear and lead to single conclusion
___ event is highly familiar to operators (e.g., frequently practiced on simulator; occurs with high frequency)
_ similar event has occurred recently or has otherwise been brought to the attention of the operators
_ event is perceived by operators to be a high-likelihood event
_ multiple symptoms / indications point to conclusion (e.g., valve position; flow rate; discharge temperature)
__ procedural guidance for correct situation assessment is available
__ the procedure has " catch" steps to detect errors in interpretation flinder: Other highly salient explanation is available that can accountfor much of the symptoms
__ symptoms can be (at least partially) explained by other known or hypothesized influences:
_ a manual or automatic control system action (e.g., shrink and swell resulting from cooldown)
__ another malfunction known to be present
___ symptoms can be (at least partly) explained by a more familiar hypothesis (e.g., events that are routinely practiced during training)
__ symptoms can be (at least partly) explained by an event that has recently occurred or has otherwise been brought to the operators' attention.
l
_ symptoms / indications can be " explained away" as " noise" or a false alarm.
_ symptoms appear in multiple diverse systems and require knowledge of system inter-connections to integrate into a coherent explanation
_ _ some critical indicator is available only to a single operator and is unlikely to be picked up by other control l
room personnel
_ event is perceived by operators to be a very low-likelihood event
__ cues are not reliable (given the event)
{
NUREG/CR-6208 122
Behaviorally Anchored Rating Scales
- 3. Intention to Act - Procedure Selection Will the operator identify and transfer to correct procedure?
- Help: Indicalions and procedure criteria are clearfor transition to correct procedure
__ criterion for transition to correct procedure is explicit step in current procedure or part of standard operating procedure
___ criterion for transition to correct procedure requires simple reading of indicatioru and requires no judgment or interpretation l
l l
Hinder: Indications may not be clear or criteriafor transition may be ambiguous l
cnterion for transition to correct procedure is not explicit in current procedure
.__ criterion for transition to correct procedure requires judgment or interpretation criterion for transition to correct procedure requires sustained monitoring to judge (e.g., trends over time)
_ primary indications for transition may not be manifest when transition step is reached
___ primary indications for transition may dissipate or disappear before transition step is reached 58 j
,__ other indications may result in transition to another procedure before " desired" transition step is reached l
_ there are strong indications to transfer to another procedure
- 4. Intention to Act - Schedulinn/Prioritizing the Actinn Are there factors that would cause an operator to postpone an action due to workload / scheduling constraints or cause him/her to forget to take the action (i.e., a memory lapse)?
Help: Action takes precedence over other actions and can be executed immediately
_ _ the action is very high in priority
_ the action can be executed immediately; it does not depend on completion of some other action or event
_ _ the action is needed to allow other operators to continue working Hinder: Other actions competefar resources or there is delay before action can occur
_._._ there are other actions of greater importance or greater urgency
_ the procedure is written to allow significant flexibility for sequencing of actions (e.g., words such as'"as time permits...")
__ the action cannot be executed immediately because there is a need for another criterion to be satisfied first (e.g., wait till a parameter reaches value x)
_ the action requires several operators to coordinate activities 58 This can arise in cases where there are multiple faults and'or where the initial fault produces secondary failures as a side effect (e.g., an interfacing system loss of coolant accident leading to rupturing of the PRT and radiation in containment).
123 s
l Behaviorally Anchored Rating Scales l
- 5. Intention to Act - Contributors to Intentional Deviation from Procedure Are there factors that would cause the operator to delay or avoid taking an action explicitly indicated in a procedure or to take an action outside procedures (i.e., commission errors)?
Help: At tion will be taken in accordance with procedure: Action is compatible with all goals
__ the criteria for taking the action are clear and unambiguous
_ the action's effect is clearly understood and fits well with the goals of the current procedure
_ taking the action has no perceived negative consequences (i.e., no goal trade-offs)
_ In cases where there are multiple conflicting goals, the procedure provides clear guidance on goal prioritization (e.g., goal prioritization via status trees)
__ training and organizational climate (i.e., safety culture) instill and reinforce appropriate goal prioritization Hinder: Other goals conflict with action providing motivation to signi6cantly delay or totally avoid action
_ taking action may violate standard operating practice (e.g., take operator out of usual operating band)59
__ taking action may lead to reduced availability of safety systems, equipment, or instruments
__ taking action may have a potential negative effect on some other safety function (e.g., lead to overfill of pressurizer)
_ there is significant uncertainty or unknown risk associated with taking the action (e.g., PORV after being opened may stick open)
_ taking the action will adversely affect areas within plant and further burden recovery (e.g., contaminate Auxiliary Building which will increase effort needed to do maintenance)
_ taking the action will have severe consequences associated with cost (e.g., plant will be shut down for major cleanup after bleed and feed)
_ taking the action will release radiation to environment Hinder: Consequences of delay (or omission) of action are perceived to be smal!
_ perception that action is not relevant or constitutes " overkill" under the particular circumstances
_ perception that undesirable action can be delayed without negative consequences (i.e., with negligible probability of negative consequences)
_ criterion for taking action is perceived to be overly conservative
__ process can be monitored and action taken if situation degrades
_ delaying action would buy needed time to rectify situation by alternative means
__ action violated routinely without negative safety consequences (probability of negative safety consequences from failure to take action is extremely small)
Hinder: Criteriafor taking action are ambiguous
_ criteria for taking action are ambiguous, difficult to determine, or require a judgment call requirement for action is presented in a caution 59 Conversely, an action that is outside procedures may be taken (e.g., blocking a safety system) if it is permitted or routinely performed under other circumstances without incurring negative consequences.
NUREG/CR-6208 124
Behaviorally Anchored Rating Scales
- 6. Execution Will the operator omit a step or execute it incorrectly?
Help: Context, procedures, etc. lead to specific actions
__ procedure is highly practiced or memorized
__ action is logically required to proceed in procedure (e.g., interlock or permissive)
__ controls are labeled or grouped to make them easily identified
_ execution uses controls with only two settings; controls are clearly marked Hinder: Procedures incomplete, complex, or poorlyformatted
_ procedure steps are not arranged in logical units (i.e., no higher order grouping)
__._ procedure step contains complex logic that can be misinterpreted procedure step includes negatives (e.g., "not")
___ procedure step includes complex conjunctions (e.g., "and" and "or")
___ action is presented in a caution or note (not in procedure step)
__ procedure is incomplete or underspecified (i.e., some necessary actions are not explicitly stated)
__ specific information (e.g., valve control number) is not specified in procedure
__ execution requires a long list of substeps
__ order of actions specified in procedure is inefficient (e.g., requires mcving back and forth across control board) so that execution is likely to be done in order different from the order specified in the procedure.
__ execution requires the use of more than one operating procedure Hinder: Displays or controls lead to confusion (i.e., slip or " mode" error)
__ controls are not placed near important indicators that determine execution l
__ controls are likely to be confused with other similar controls
__ controls go against standard operational stereotype (e.g., flip a toggle up to turn off)
__ control system has more than one setting, so that the same control action has different consequences depending on the setting (i.e., a mode error; this occurs most commonly with soft-controls on computer display systems) i
,__._. execution requires a control action to be taken outside the control room
_ a major component or set of actions is strongly associated with another context and may, therefore, lead to inappropriate actions (capture-type slip)
Hinder; Difficult timing or coordination requirements
_ execution requires some type of continuous control (e.g., tuning) where feedback is difficult to judge (e.g., delayed in time)
,_, execution requires maintaining a parameter within a tight operating band (e.g., to avoid inadvertent trip or safety system activation)
_ execution requires rapid response (e.g., a rapid rate of change that requires a quick response)
__ execution requires a difficult coordination between operators l
i 125 l
Behaviorally Anchored Rating Scales
- 7. Execution - Detection of Errors Will the operator recognize that an error has been made?
Help: Formal checks to identify errors
.__ procedure has explicit catch steps or verifications
__ other operators are likely to do careful checking of performance
_._ there is a salient indication when error is made or when action was successful (e.g., alarm, interlock)
Hinder: Little or nofeedback/ indication error was inade
__ other operators are all occupied in some other activity and will not check performance
_ there is poor feedback on effect of control action
- 8. Execution - Recovery from Error Will the operator be able to recover from error?
Help: Formal procedure to recover
___. there is procedure written for recovery from error Hinder: Little or no indication of how error has changed situation; recovery actions unclear
_ incorrect execution cannot be recovered due to damage done
_ recovery requires a set of actions different from the set of actions done incorrectly
___. there are severe time constraints for executing recovery actions 126 NUREG/CR-6208 l
l l
conu 235 U.S. NUCLE AR REGULATORY COMMISSION 1
Po T N ER untu i m' w A6 - - a-u e.,
noi. aro2 BIBLIOGRAPHIC DATA SHEET ise,/au = r.o.onra,,,. m es NUFIG/CR-6208
- 2. TITLE AND SU8 TITLE An Pe irical Investigation of Operator Performance 3.
DATE REPORT PUBLISH (D in if dnitively Demanding Simulated Emergencies l
va July 1994
- 4. FIN oR GRANT NUMBE R L1505
- 5. AUTHOR (S)
- 6. TYPE oF REPORT E. M. Roth, R. J. Mumaw, W_
STC P. M. Lewis, NRC
- 1. PE R toO Cove R E O DocM6,e onsees S. P F ANf 2ATioN - NAME AND ADORESS ist mac e ide oudon, Oer.or er neysmi, us m=*ar a,yudereny'c
. dauausqy.meeseciteene ceer,p dser Westinghouse Science and Technology Center 1310 Beulah Road Pittsburgh, PA 15235 9.
G ANtZATioN - NAME AND ADDRESS #f mac eroe
- sea, as see.e ;dr erseeer p, Jde sac p,4 ien, Orno, er seyjen. ut mcm, n.
e m -- i i..
Division of Systems Research Office of Nuclear Regulatory Research U. S. Nuclear Regulatory Commission Washington, DC 20555 -0001
- 10. SUPPLEMENTARY NOTES
- 11. ABSTRACT #200 weeuh er se=1 This report documents the results of an empirical study of nuclear power plant operator performance in cognitively demanding simulated emergencies. During emergencie:
operators follow highly prescriptive written procedures. The objectives of the study were to understand and document what role higher-level cognitive activities such as diagnosis, or more generally ' situation assessment', play in guiding operator performance, given that operators utilize procedures in responding to the events. The-study examined crew performance in variants of two emergencies: (1) an Interfacing System Loss of Coolant Accident and (2) a Loss of Heat Sink scenario. Data on operator performance were collected using training simulators at two plant sites. Up to 11 crews from each plant participated in each of two simulated emergencies for a total of 38 cases.
Crew performance was videotaped and partial transcripts were produced and analyzed.
The results revealed a number of instances where higher-level cognitive activities such as situation assessment and response planning enabled crews to handle aspects of the situation that were not fully addressed by the procedures. This report l
documents these cases and discusses their implications for the development and evaluation of training and control room aids, as well as for human reliability analyses.
- 12. KEY WoRDS/DESCRIPToRS tuse we,we arpar.sre enet m a shr eeeeam*ses m asceeme rne espore.J t3. AVAeLAsiuYY &T A1(MLNT Unlimited human reliability decision-making cognitive model crew performance human error operator performance Ifnci m:1 fi ed human factors nuclear power plant Unclassified cognitive emergencies simulator IS, NUM8E R oF PAGES simulation team it price NMC f ORM 3M (249) p
Printed on recycled paper Federal Recycling Program
MUIEGTGVarrw e
DEMANDING SIMULATED EMERGENCIES
.l UNITED STATES
,,,c,c3,oy,,cta,,gn,,
NUCLEAR REGULATORY COMMISSION POSTAGE AND FEES PAID WASHINGTON, D.C. 20555-0001 usuac PERMf7 NO, G-67 OFFICIAL BUSINESS PENALTY FOR PRIVATE USE. $300 1 1AN1PK 120555139531 hy"h0i PUBLICATIONS SVCS.
TPS-POR-NUREG 2WFN-6E7 DC 70555 WASHINGTON
=6
- ' -...