ML20217A471
| ML20217A471 | |
| Person / Time | |
|---|---|
| Issue date: | 03/16/1998 |
| From: | Callan L NRC OFFICE OF THE EXECUTIVE DIRECTOR FOR OPERATIONS (EDO) |
| To: | Diaz N, Dicus G, Shirley Ann Jackson, Mcgaffigan E, The Chairman NRC COMMISSION (OCM) |
| References | |
| NUDOCS 9803250080 | |
| Download: ML20217A471 (38) | |
Text
- 1 MEMORANDUM TO: Chairm:n Jackson March 16, 1998 l 1
Commissioner Dieus o Commtsioner Diaz Commissioner McGaffigan Originni Signed by FROM: L. Joseph Callan Executive Director for Operatio,ns W @hn
SUBJECT:
TRANSMITTAL OF PAPER TO BE PRESENTED AT THE ENLARGED HALDEN PROGRAM GROUP MEETING, MARCH 16-20,1998 in accorcance with Management Directive 3.9, attached for your information are copies i
of two papers to be presented by Dr. J.J. Persensky of the Office of Nuclear Regulatory l l
Research at the subject international conference in Lillehammer, Norway. The first paper is "An j Experimental Evaluation of Alarm Processing and Display Characteristics." It contains general information for those who may be interested in NRC's activities in this area. The presentation slides for this paper are also attached. He will be presenting the second paper " Outline of the HAMMLAB 2000 Research Agenda"in his role as Chairman of Halden's HAMMLAB 2000 Task Force. This paper was prepared by Halden staff and is being reviewed by the Task Force members; it is not a direct result of NRC activities.
l
Attachment:
As stated cc w/att: SECY OCA ;
Contact:
J.J. Persensky (301) 415-6759 CS3 C //I/
DISTRIBUTION ccm n r- r: " " - ? ^ 7 ~'
Central File f1 L -- ~ ~ ' " "
PUBLIC CIHFB RF JWachtel .
4 '. - ~3 Q(ifb DOCUMENT NAME: G:\ClHFB\HALDEN.MEM *SEE PREVIOUS CONCURRENCE T2 receive a copy of this dwument. Indicate in the box: "C" = Copy without enclosures *E" = Copy with enclosures "N" - No copy OFFICE DST:RES l DST:RES l D: DST l D:RES l EDO l NAME JPersensky* TKing* MWHodges* MKnapp* AThadani DATE 03/ 06/98 03/09/98 03/09/98 03/12/98 03/ /98 OFFICE $D$ , l NAME LTdffNii DATE 03b/98 9803250080 080316 OFFICIAL RECORD COPY ,
It~
t UNITED STATES f-y j
NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20066 4 001 l
%,****+ March 16, 1998 MEMORANDUM TO: Chairman Jackson Commissioner Dieus Commissioner Diaz Commissioner McGal'figan FROM: L. Joseph Callan Executive Director o per ions
SUBJECT:
TRANSMITTAL OF PAPER TO BE PRESENTED AT THE ENLARGED HALDEN PROGRAM GROUP MEETING, MARCH 16-20,1998 in accordance with Management Directive 3.9, attached for your information are copies of two papers to be presented by Dr. J.J. Persensky of the Office of Nuclear Regulatory Research at the subject intemational conference in Lillehammer, Norway.. The first paper is "An Experimental Evaluation of Alarm Processing and Display Characteristics." It contains general information for those who may be interested in NRC's activities in this area. The presentation slides for this paper are also attached. He will be presenting the second paper " Outline of the HAMMLAB 2000 Research Agenda"in his role as Chairman of Halden's HAMMLAB 2000 Task Force. This paper was prepared by Halden staff and is being reviewed by the Task Force members;it is not a direct result of NRC activities.
Attachment:
As stated
. cc w/att: SECY ,
i
Contact:
J.J. Persensky, RES j
(301) 415-6759
{
f i
l 1
i
I* l
( An Experimental Evaluation of Alarm Processing and Display Characteristics l John O'Hara and William Brown Bruce Hallbert and Gyrd Skraaning Department of Advanced Technology Halden Reactor Project {
l Brookhaven National Laboratory Halden, Norway l Upton,NY l1973 l J Persensky and Jerry Wachtel Office of Netlear Regulatory Research U.S. Nuclear Regulatory Commission Washington,DC 20555 This paper describes a research program sponsored by the U.S. Nuclear Regulatory Commission to address the human factors engineering (HFE) aspects of nuclear power plant alarm systems. The overall .
objective of the progam is to develop HFE review guidr.nce for advenced alarm systems. As part of this l program, guidar,ce has been developed based on a broad base of technical and research literature. In the j course of guidence development, aspects of alarm system design for which the tect nical basis was l insufficient to support complete guidance development were identified. The primary purpose of the research reported in this paper was to evaluate the effects of three of these alarm system design characteristics on operator perfonnance in order to contribute to the understanding of potential safety issues and to provide data to support the development of design review guidance in these areas. Three alarm system design characteristics studied were (1) alarm processing (degree of alarm reduction), (2) alarm availability (dynamic prioritization and suppression), and (3) alarm display (a dedicated tile format, a mixed tile and message list format, and a format in which alarm infonnation is integrated into the process displays). A secondary purpose was to provide confirmatory evidence of selected alann system guidance developed in an earlier phase of the project. The alarm characteristics were combined into eight separate experimental conditions. Six, two-person crews of professional nuclear power plant operators participated in the study. Following training, each crew completed 16 test trials which consisted of two trials in each of the eight experimental conditions (one with a low-complexity scenario and one with a high-complexity scenario). Measures of process performance, operator task performance, situation awareness, and workload were obtained. In addition, operator opinions and evaluations of the alarm processing and display conditions were collected. Numerous strengths and weaknesses associated with individual alarm design characteristics.
1 Intreduction l The need to improve the human factors engineering (HFE) of alarm systems has led to the development of advanced systems in which alarm data are processed beyond the traditional "one sensor - one alarm" framework. While this technology promises to provide a means of correcting many known alann system deficiencies, there is also the j potential to negatively impact operator performance [1]. A research program, sponsored by the U.S. Nuclear Regulatory Commission (NRC), is underway to address the HFE aspects of nuclear power plant alarm systems. The objective of the study is to develop HFE review guidance for advanced, computer-based alarm systems. As part of the development effort, aspects of alarm design for which the tcchnical basis was insufficient to support guidance l development were identified and research to address the most significant issues was initiated. !
A general methodology was established to develop HFE guidance to support the NRC's review of NPP HSIs [2].
The methodology has been applied to several areas of new HSI technology and the guidance has been integrated into NUREG-0700, Revision 1 [3]. Guidance development proceeds as shown in Fig.1. The methodology seeks to l establish valid guidelines in a cost-effective manner. Validity is defined along two dimensions. " Internal" validity is
}' the degree to which the individual guidelines are based upon an auditable research trail. " External" validity is the l
degree to which the guidelines are subjected to independent peer review. The peer review process is considered a good method of screening guidelines for conformance to accepted human engineering practices. Validity can be inherited from the source materials that are used to develop the guidelines. Thus, for example, for a specific topic there are sometimes existing documents, such as industry guidance documents and standards, that have an auditable 1
I
- e. ,
l research trail and have been the subject of extensive peer review. We esfer to these as primary source documents.
Where source materials lack validity, it must be establish for the new guidance as part of the guidance development process itself.
Since they already possess internal and external validity, primary source documents are sought rust in our approach t
to guidance development. Even when such a documents are available, their guidance must still be adapted to an NPP HS! application. When prunary source documents alone do not provide a sufficient basis on which to develop guidelines, additional sources ofinformation ars ==aary. Secondary source documents are those with either '
internal or external validity (not both). Many indusay guidance documents fit into this category. Dey are good l from the standpoint abat their information is already expressed in guideline format. However, they either provide a l good trail to their technical basis or have been peer reviewed, so the missing aspect of validity needs to be established as part of the design review guidance development. Tertimy documents, such as HFE handbooks, generally do not provide information in guidance form and they do not possess either form of validity. Thus considerable effort may be involved in guideline preparation and valuinian using these sources.
De three final sources ofinformation for guidance development (see Fig.1) require the most effort. Basic literature and industry experience are used where guidelines cannot be obtained from the other sources. Results are evaluated from basic literature including articles frorn refereed technicaljournals, reports from research l organizations, and papers from technical conferences. Industry expmience can be obtained from surveys and interviews. Industry experience is a valuable information scurce ofinformation for identifying performance issues associated with aetual systems and tested design solutions to problems that have been resolved.
, ,, Yass*m*a's"."
.r-- ac::,u::::e .
~
{dah 3 ".I E *
- 4.,
h"" .~
W.'."*"*
4., ,f e um .g
- n.masamm.
La,**g
,, 2"".*" !N .
o U amn 4,, ..!::lll=2 o <==- > < >
g.g -
4-e-a ,,
M" Fig.1. Guidance development methodology.
Original research is the last category and refers to the systematic manipulation of the HSI design features ofinterest in order to determine their effects on performance under controlled conditions. De research should generally be performed in a dynamic, real-time context; e.g., a full-scope simulator or high-fidelity engineering simulator. His type of research plays two important roles in guidance development: technical basis development and guidance confirmation. First, when the technical basis does not exist in the other source materials the results of experiments l can be used to fill the knowledge gap, i.e., to provide the information upon which design review guidance can be 2
De:
4
~ developed. For example, such studies can identify what aspects of system design that are significant to human performance.
De second important role of experimental research is guidance confirmation. When guidance has been developed based on the other sources ofinformation listed in Fig.1, testing may be necessary to provide confirmatory evidence that (1) the guidance is an acceptable extraction, synthesis, or interpretation of the data, and (2) that the guidance is appropriate to an NPP application.
De great advantage of original research is the ability to focus on the specific design characteristics and human l performance issues ofinterest. It has the disadvantage of being the most costly method of technical basis l development relative to the range ofissues that can be addressed Further, such research can be limited in
! generalizability because any single experiment uses a relatively small sample of operators, a small sample of -
testbeds (plant types), and may be constrained by the specific way in which HSis are designed for the study (additional generalizability considerations are discussed in [5]).
l l Using this guidance development method, draft alarm review guidance was developed using each type of information source listed in Fig. I except for origiral research. Each individual guideline included the technical sources ofinformation that formed its technical basis. This information provides the basis for evaluating the internal validity of guidelines. De technical bases vary for each guideline. Some guidelines are based on technical conclusions from a preponderance of empirical evidence, some on a consensus of existing standards, and others on l
~ judgement that a guideline represents good practices based upon the infonnation reviewed De draft guidelines were then evaluated by independent peer-reviewers who assessed: (1) the internal validity of the guidance, (2) the !'
, relevance of the guideline to the nuclear plant setting, and (3) the appropriateness of the guideline for NRC safety i! reviews. This peer review constitutes the extemal validation of the guidelines. A revision to the draft guidance based on the reviews was accomplished. He guidance development and technical basis is documented in NUREG/CR-6105 [4] and the guidance itselfis integrated into NUREG-0700 [3].
However, there were aspects of advanced alarm system design for which the available information did not fully support guidance development. A program of original research was developed to address these characteristics. The program is discussed in the next section.
This paper will summarize the current status of the program. Section 2 will provide a summary of relevant alarm )
system human performance issues. In Section 3, the experimental methodology will be described to illustrate how l the alarm system design features are being studied. The results are presented in Section 4 and conclusions are presented Section 5. A detailed report of the alarm experiments is in preparation [6). l 2 Human Performance issues Associated with Alarm System Design During guidance development, several human performance issues associated with advanced alarm systems were j identified. Those issues associated with alarm processing, availability, and display were considered to have the j highest priority. Each is briefly discussed below. j Ilarm Praemim:
l One of the most important objectives in the design of advanced alarm systems is to reduce the number of alarms that occur during plant disturbances. Alarm processing is intended to accomplish this objective. These techniques were i developed to identify which alarms are significant and to reduce the crew's need to infer plant conditions. Alarm processing refers to the rules or algorithms that are used to determine the operational importance of alarm ;
. conditions. Many of the techniques can be classified into two categories based upon how the information that operators receive is affected. Nulsance Alarm Processing techniques essentially climinate alarms that are inelevant to the current mode of the plant, e.g., a low temperature alann on a line that is out of service for maintenance.
Redundant Alarm Processing techniques analyze alarms to determine which are less important because they provide information that is redundant with other alarms. For example, in causal relationship processing only causes are alarmed end consequences are considered redundant. In addition to reducing the actual number of alarms, however, these redundant alarm processing techniques may adversely affect the information used by the operator for situation 3
l e assessment, decision-making, or confirmation that the situation represented by the "true" alarm has occurred.
%c various processing methods and the degree of alarm reduction should be evaluated for their relative effects on operator performance. However, research that has addressed the effects of alarm processing on performance has been equivocal. Some studies have found an effect of alarm processing on performance while others have not. This could be due to many factors such as type of processing used, degree of alarm reduction achieved, and user familiarization with the system. The effects could also be transient dependent, e.g., dependent on the specific scenario, on the operators ability to recognize familiar pattems, or on pl ant type. System complexity should also be considered, The operator, as the system supervisor, should easily cotaprehend alarm information, how it was processed, and the bounds and limitations of the system. An alarm system combining multiple processing methods may be so complex that it cannot be readily interpreted by operators in time-critical situations. An understanding of this relationship is essential to the development of alarm sy: tem improvements and review guidance.
Alarm Availability This refers to the method by which the results of alarm processing are made available to the operating crew (rather than how they are presented, which is alarm display). Two of the techniques that have been used include suppression and dynamic prioritization. Suppression is when less important are suppressed and not presented to the operators, but can be accessed by operator request or by the alarm system based upon changing plant conditions.
Dynamic prioritization is when less important alarms are presented to operators but somehow distinguished from those that are more important, such as presenting them in a different color or in a different location than other alarms.
Suppression also removes potentially distracting alarms; however, since they are accessible on auxiliary displays, additional workload may be imposed by requiring operator action to retrieve them. Dynamic prioritization does not conceal any information from operators.1lowever, the operator must perceptually " filter" alarms (e.g., scan for red alarms) and a potential, therefore, exists for distraction from less important alarms. Thus, there are tradeoffs between these approaches and an issue remains conceming when the various options should be employed.
Alarm Disnig Alarm displays can be considered along three dimensions: spatial dedication (whether an alarm is always displayed in the same physical location or in variable locations), display permanence (whether an alarmed is always visible or visible only when in an alarmed state), and integration (whether that alarms are presented as a separ te system or integrated with other process information. These three dimensions distinguish three main types of alarm displays.
Spatially-dedicated continuously-visible (SDCV) alarm displays provide a display ofinformation in a permanent location. Lighted tile alarms are an example. Temporary alarm displays, such as a VDU message list, display alarm messages only when the alarm is in a valid state. Specific alarms usually not appear in spatially dedicated location although they may always be presented on the same VDU. Integrated alarms present alarm information as an integral part of other displays, such as process displays. For example, if alarms are built into a system mimic display, trouble with a component such as a pump can be depicted by a change in color or flashing of the pump icon. These displays may be in a fixed or variable location and are typically not permanent displays. While alarms have traditionally been separate information systems from other indicators, it is thought that the operators information processing is supported by integration ofinformation into a single displays. The benefits of these types of displays are thought to include: (1) enhancement of parallel processing (lowering cognitive workload), (2) enhancement of the operators ability to better understand the relationships between display elements, and (3) enhancement of the operators ability to develop a more rapid and accurate awareness of the situation.
SDCV displays are often preferred by operators and have been shown to have performance advantages under high-l alarm conditions. But, placing all alarms on such displays (potentially many thousands of alarms in advanced plants) leads to the alarm overload problem for operators. VDU message lists have not been completely successful alternatives, however. Message lists have been demonstrated to be problematic in high-alarm conditions. Further, although the research is limited, integrated graphic displays have not been shown to improve performance. To serve the different functions of the alarm system, multiple display formats may be required. Thus the display format and the degree to which alarm information is integrated with other process information are important safety 4
.e
. j I-U-
considerations. De role, relative benefits, and design of each type of alann display format in the presentation of alarm infonnation is an issue.
l -3 - Esperissestal MetMi.,4 ,
In order to help address these issues, an experiment was performed to evaluate the impact of alarm processing, .
availability, and display characteristics on plant and operator performance. De extent to which alarm numbers are reduced is a ffunction of the alarm processing tw:hniques that are applied. In this study, a variety of alarm processing methods were employed that are representative of near-term applications, and therefore, near-term segulatory review considerations. Dree levels of alarm reduction were used. De first processed nuisance alarms to achieve modeste alarm reduction (called Tier I processing). He second processed redundant alarms, which in combination with nuisance alarm processing, achieved maximum reduction (called Tier 2 processing). A third '
condition of no alarm processing was used to provide a baseline for comparison (called Tier 0 processing).
De differential effect of two types of alarm availability was evaluated: suppression and dynamic prioritization. In
- the suppression condition, less important alarms were not presented in the primary alarm displays but were available .
to operators on a suppressed alarm list. In the dynamic prioritization condition, less important alarms were color coded to indicate their status.
Dree types of VDU-based primary alarm displays weir compared: a dedicated " tile-like" format, a mixed tile and
' message list format, and a mixed integrated graphic and message list format, ne graphic provides alann. j l -
information integrated into process display formats. Dese display formats enabled the examination of two aspects -l of alann display design: spatial dedication and degree ofintegration with process information. A secondary alarm display consisting of a chronological event list was also available to operators in each condition.
' He various types of processing, availability, and display were combined to form eight experimental conditions, i.e.,
unique alarm system configurations (see Table 1). In addition to varying alarm characteristics, two types of scenarios were used: complex and simple. Eight exemplars of each were developed for the study.
Table 1 ExperimentalConditions Pr - ==i>=
PI' P2 P3 Availability NA Al A2 Al A2 Display type i D1 1, 7 D2 2 3 4 5 6 !
D3 8
, i
!~ Notes:
D = Displays (D1: tile format; D2: tile + message list; D3: integrated + message list)
P = Processing (PI: Tier 0-none; P2: Tier 1-nuisance; P3: Tier 2-redundant)
A = Availability (A1: prioritization; A2: suppression)
Each experimental condition included both levels of complexity (not shown in the table for simplicity).
, De. tests were conducted using the Human-Machine Laboratory (HAMMLAB) at the Halden Reactor Project in Norway. De plant model simulates a pressurized water reactor power plant with two parallel feedwater trains, turbines and generators. It is l i
~
closely related to the plant model used in the large scale training simulator at the Loviisa nuclear power stction in Finland. The '
participants were professional nuclear power plant operators from the Loviisa plant. Six crews of operators participated each
,- made up of a reactor operator and turbine operator. Each crew made 16 experimental trials, two in each of the eight alarm I
conditions (one with a low complexity scenario and one in a high complexity scenario). Dere were a total of 16 scenarios so that no scenario was used more than once for each crew. The order of presentation of scenarios was balanced, as was the
- relationship between individual scenarios and experimental conditions.
5
The measurement of performance in the study included process measures, operator task performance, and operator cognitive processes (e.g., situation awareness and workload). The subjective opinions of the operators were also obtained.
4 Results General Findinn ne alarm systems employed in this study all represented designs that had been developed using a human engineering design process that included operator input and prototype testing as part of the design process. The development process used NUREG-0700, Rev I guidance. Dus the designs provided the opportunity to provide a field trial of the guidance and thereby provide confirmatory evidence for their technical basis.
Overall, the analyses revealed only slight differences in objective performance measures between alarm features and none of the characteristics were significantly and consistently associated with poor performance. Bus, confirmatory support for the alarm guidance used was obtained.
J Alarm Disnlavs Modest differences were observed between the display types in plant, operator, and cognitive performance. Operator comments provided significant insights into differences between the three. The spatial dedication of alarm displays was strongly supported. De benefits of such displays included the fact that important alanns were easy to find and interpret and no j important alarms were " hidden" from their view. The operators also found the tiles to be much better when there are many
]
alarms. One operator commented that he used both the list and the tile display, but in difficult scenarios with many alarms he l
only looked at the tile display.
However, the benefits of the spatial dedication was reduced when the number of tiles significantly increased. This was reflected in operator preference for the mixed display condition where the number of tiles was relatively small. In the tile condition, where all alarms were presented in tiles, operators indicated that it was sometime hard to find new alarms and that it was difficult to get an overview of the plant situation when many alarms were coming in. Due to these considerations, the operators stated that the key alarms should be on alarm tiles.
De problems associated with the tiles were that they did not provide information operators indicated was necessary to understand a disturbance; i.e., time, alarm sequence infonnation, alarm setpoints, and parameter values.
The alarm message lists were most useful for obtaining such detailed alarm information. Many operators indicated that the sequence of alarms was important to understand what initiated an event and how it progressed. The main problem with the alarm lists, however, was that they were time consuming to read and difficult to use when there were many alarms. As a result, operators could not effectively use the lists when the number of alarms was high. A significant problem identified was when i the alarms exceeded one page (one VDU display). In such circumstances, the operators did not like the fact that there were alarms on pages they could not see. Further, operators were reluctant to scroll to unseen alarm pages (older alarms) and sometimes abandoned scrolling the alarm lists when workload became high.
~
The integration of alarms into the process overview displays and process mimics was efTective and had many advantages similar to the tiles: good for a rapid assessment of a disturbance and when the number of alarms was high these displays were l preferred over the message lists. The integration of alarms into the process displays made it easier to understand the relationship between individual alarms and between the alarms and the process disturbance causing them. He problem with the integrated display was the fact that some alarms were hidden (e.g., in lower level process formats). In addition, given the way the alarms were implemented in this study, they could not determine if an alarm parameter was high or low and in which direction it was going. However, this limitation could easily be corrected, ne results of the study suggest that the most effective alarm display may be an alarm system including three elements: tiles (SDCV display), message lists, and alarms integrated into process monitoring displays. An important issue to address in such a system would be coordination of alarms across all three types of alarm displays to support easy and rapid transition between them and to avoid overwhelming operators with alarm information. I l
6 I
1.'
Alarm Processing
. De alarm processing techniques provided a considerable degree of alarm reduction. Tier 1 processing reduced the number of
~alarms to approximately 50 percent of the Tier 0 baseline ofno alarm processing. Tier 2 processing reduced the number of l alarms to approximately 25 percent of Tier 0. Operators expressed a clear preference for the maximum alarm reduction
! ' because it made the identification and understanding ofimportant alarms easier. Based on their assessments of the alarms that were " eliminated" by the processing rules, the tech iiques were acceptable, and across all sixteen scenarios, the operator did not .
l Identify important information that was eliminated. nus based on the degree of Alarm reduction achieved, operator preference for maximum alarm reduction, and operator verification that the processing did not remove any important alarms, the processing was apparently successful.~
However, the processing had little effect on performance measures, including plant, task, and cognitive variables. What is most surprising is that based on operator comments, it was expected that situation assessment and workload would have been affected. De comments clearly indicated difficulty in finding new altrms and seeing where they fit into the overall picture
. when the number of alarms was high. Derefore, one would have expected situation assessment to have been better and workload lessened under maximum processing conditions. Except for several modest effects, it was not. While this finding is consistent with several other studies, the question remains as to why performance, especially on cognitive measures, was not !
- significantly affected. A possible explanation for this is that some form of compensatory behavior or strategy shift may have I taken place when the number of alarms became high. Operators may have been able to shift their strategies for coping with !
high alarm situations because of the advanced nature of the HAMMLAB control room. NPP control rooms are information rich with diverse sources ofinformation. His is especially true of advanced control room designs which reflect efforts to improve deficiencies in conventional control room designs.
Alarm Availmhility The results of this study provided support for the suppression of alarms. Nearly all of the operators preferred suppression over dynamic prioritization. Operators indicated that, although prioritization had the advantage of making all information immediately available, there was often little useful information in the low priority list, and they were concerned that an !
operator could become distracted by the list or might mistakenly read the wrong list. I On the other hand, operators did not want alarm completely removed (i.e., filtering) for several reasons. First, there was
! concem over the fact that processing logic may not be 100 percent correct and may under some circumstanc:s remove important alarms. Second, operators sometimes will use such alarms for other purposes, such as status infortnation to check that events occurred as expected.
Interactions One of the most significant findings of this study is the importance ofinteractions. Dat is, ills difficult to make general statements about alarm system characteristics without making some clarifying remark about other factors. Alarm system characteristics frequently interacted with scenario complexity in the analyscs of performance mt:asures. The operator comments about specific alarm system characteristics frequently reflected interactions with other alarm system characteristics and with other factors, such as,the type of process disturbance.
The operator comments regarding alarm displays frequently reflected interactions with alarm processing and other factors, e.g.,
- it would not be good to integrate alarms into process formats if there was no alarm processing, but would be good if there was l a lot of alarm processing. Similarly, the alarm message lists were judged most useful when a high degree of alarm processing j i was in use. Dus alarm processing and display type are not independent considerations. Operators also indicated that the best j
' display depends on the type of disturbance and whether they were in the early or late phases of a disturbance. He effects of i scenario complexity were also expressed mainly through interactions with alarm system characteristics. l Dese interactions r ' lect the fact that alarm information is used by operators in many different ways: As an alerting system,
)- for situation assessment (e.g., to see the relationship between alarms, components, systems, and functions), for response
- planning (e.g., as a check on component and system availability), and for post-disturbance analysis. Different combinations of processing and display att needed to support these various activities. !
1 i
- 7 1
c 5 Comelmsloos The nuclear and human factors communities have developed a significant database upon which HFE review guidance for advanced alarm systems was developed. Information supporting guidance development was available not only from alarm guidance documents, but also from published reports of research and operational experience. Further, advanced alarm j
systems, particularly those utilizing computer based interfaces, share many HSI characteristics with other control room '
resources. nus HFE principles associated with VDUs, graphics displays, dialog structures (such as menus and command language) and computer input devices (such as touch screens, keyboards, and trackballs) are applicable to alarm systems. nis ,
information was used to develop HFE guidance for the review of alarm systems.
]
k was also found that there remain notable human performance issues related to alarm processing, availability, and display.
The primary purpose of this project is the development ofdesign review guidance for these characteristics of alarm systems.
The current research plays two important roles in guidance development: technical basis development and guidance . .
confirmation. Dese results have many implications for both the development of new guidance, as well as, potential revisions l or clarifications to existing guidance. De use of these findings for guidance development will be the subject of a related !
project task.
1 6- Acknowledgments j
Dis research is being sponsored by the U.S. Nuclear Regulatory Commission. The views presented in this paper represem those of the authors alone, and not necessarily those of the NRC. The authors wish to extend our gratitude to our colleagues at ,
the NRC, BNL, and Halden who contnbuted to this project and to the Imatran Voima Oy - Loviisa NPP personnel who shared I their knowledge and expertise with the project staff. j j
7 References !
1
[1] O'Hara, J., & Brown, W. (1996). AdvancedAlarm Systems and Human Performance (BNL Report A3967-1-6/96). I Washington, D.C.: U.S. Nuclear Regulatory Commission. .
[2] O'Hara, J. (1994). Advanced Human System Interface Design Review Guideline (NUREG/CR-5908, Volume 1: General Evaluation Model, Technical Development, and Guideline Description). Washington, D.C.: U.S. Nuclear Regulatory
_ Commission.
[3] NRC (1996). Human-System Interface Design Review Guideline (NUREG-0700, Rev.1). Washington, D.C.: U.S.
Nuclear Regulatory Commission.
[4] O'Hara,J., Brown, W. HigginsJ, & Stubler, W. (1994). Human factors Engineering Guidelinesfor the Review of AdvancedAlarm Systems (NUREG/CR-6105). U.S. Nuclear Regulatory Commission Washington, D.C.
[5] O'Hara, J., Stubler, W., Brown, W., & Higgins,J. (1997). IntegratedSystem Validation: Methodology andReview Criteria (NUREG/CR-6393). Washington, D.C.: U.S. Nuclear Regulatory Commission.
.[6] O'Hara, J., Brown, W., Hallbert, B., Skraning, G., Wachtel. J., & Persensky, J. (in preparation). The Efects ofAlarm processing and Display on Operator Performance (Draft NUREG/CR). Washington, D.C.: U.S. Nuclear Regulatory Commission.
8
F ',
t I
t .
An Experimental Evaluation of Alarm Processing and Display Characteristics l
John O'Hara & William Brown J Persensky & Jerry Wachtel
, Brookhaven National Laboratory Nuclear Regulatory Commission Bruce Hallbert & Gyrd Skraaning Halden Reactor Project Enlarged Halden Programme Group Meeting Lillehammer, Norway l
March 1998 Outline
. General Guidance Development Methodology
. Background to Alarm Research
- Alarm Experiment i
t
. Conclusions
l l
Guidance Development NUREG-0700
- ' Reviewprocedures Human-Systesilaterface Desism Review Guideline HFE guidelines tJA Mustest seedsomry ComudmAs come ssene.or m.s.eswy s a DRG software Guidance Document Objective !
Establish a guidance development process that:
- is generalizable and can be applied to any aspect of HSt technology
- Wli produce technically defensible review criteria e intomal validity
- Extemal validity
- Best utilizes available resources identify gaps in the guidance, e.g.,
- Advanced alarm system characteristics
- Hybrid systems (that integrate analog and digital technology Both studies reflect the application of the methodology l
L-_
Ie, 4
Guidance Development Guideline Validity Considerations
+
Intemal validity is the degree to which the individual guidelines are based on the documented technical basis
- Information upon which the guideline is establisted End justified
- Technical bases vary for individual guidelines
- Audittrailpurposes
- The technical frerit of the gukielme to be evolueted by othecs
- The more inforrned opplcotson of the guklehne
- Devletons of exceptions to the guideline to be evolueted Extemal validity is the degree to which the guidelines are subjected to independent peer review
- Conformance to accepted human engineering practices
- Comparison of guidelines to practical operational experience in actual systams Both are examined in source material and addressed as part of the guidance development process i Guidance Development Approach
, _ , *'i:l'."Jl:'.'." "
=~- w - s, ,
'.s s
4
'm-- -
~
.n h4,. 8ep ' 7 '
~* *
,
- fe = men,
- 3
. o:==m -- > <
m >
nuseaw.ese e h*
m -
I I
Ie j
h I
Guidance Development DocumentIntegration
~:.^
=.~. ,
pipagg49gg AddfeOS M NUdtEG#10p
=-- ..<... + _--
gg
==y g
GoWeene N measprseus menerassa y
,,,,,,,, m.
em m MBEL
~
r Background Historical Perspective Conventional alarms systems issues (circa TMI)
- Operators do not find alarm systems useful during plant upsets
- Alarm density and lack of priority
- False and nuisance alarms Research on upgrades to alarm improvements
~
.- Color code priority
- First-outpanel
- Reflash
- Eliminate grouping of multiple alarms
- Blackboard
- Relocation and organization of tiles Results
- Situation improved but relies on operator to perceptually screen alarms j - High alarm handling workload with upgrades
[e 1 i
l
Background
Applications of Advanced Technology i
1
- Computer-processing of alarms
- Alarm filtering
- Dynamicprioritization
- Alarmgeneration
- Alarm hierarchies and integrated displays
- CRTpresentationof alarminformation
+ Impact on crew performance not well known
- Fault detection and situation assessment
- Disturbance mitigation and response planning j
- Cognitiveworkload j
+ Design review guidance is needed
Background
Objectives and Approach
- Objective
- To determine the effects of alarm system design characteristics on performance
- To develop guidance for alerrn system evaluation
+ General approach p -> Aserm Design NuREOCR Leamed Revwe GuWence s10s O
4 Rowlewof Almam
.. w, w a. + ,
Identfy Human Priormae opies
(
I I
Alarm Experiment Research Questions
+ What is the impact of alarm processing, availtbility, and display characteristics on performance, e.g.:
- What is the efftd of spatialdedication of displayed alarms and dos this effect depend on level of alarm processing?
- What is the effect of the integration of alarm inintmation in displays?
- What is the effect of alarrn reductson?
- What is the effect of alarm availability methods and does this effect depend on the levelof alarm reduction?
- To what extend do the above effects depend on the complexity of the ev.nt?
Alarm Experiment General Approach
- Emphasis on extemal validity (generalization of results):
- Experiencedoperators (Six operating crews from Loviisa NPP)
- Fu!Wnission, dynamic simulation under a wide range of scenarios (Halden Reactor Project, Norway)
- Fully-equippedcontrolroom (Without advanced information processing support systems)
- Realisticalarmsystemdesigns
- Decouple alarm system characteristics to test their independent effects
- Many studies evaluate whole systems so characteristics are confounded
. Effects tested with a broad spectrum of performance measures
?' .
t t :
l i' 4 l
Alarm Experiment Test Variables
- Independent variables I
- Alarm processing type (3 levels)
- Alarm availability method (2 levels) I
- Alarm displaytype(3 levels)
- Scenarlocomplexity(2 levels)
+
Performance measures:
- Systemperformance(Scenariodependent)
- Operatorperformance(Scenanodependent)
- Cognitive processes (Situation awareness and workload)
- Operatorsubjectiveevaluations l
l l
Alarm Experiment I Composite Within-Subjects Experimental Design Processing Ter 0: Test 1: Ter 2:
None Nuh ance Nuisance + Redunfg Availability NA Priority Suppress Priortty Suppress Display Type
,- Tiles 1 7 TileWessage Ust 2 3 4 5 6 IntegrmtedMessage Ust d Notes:
- The number in each cell identifes a unique experimental condition Each experirnental condition included two levels of scenario complexity Each crew participated in 16 experimental trials
- r. ,
o y j
- 1 Alarm Experiment Procedunes l 1
]
I
- Training
- In-planttraining
- Two days of simulator training
+ Experimental trials
- Balancing of scenarios presentations
- 16 trials over approximately three days i
- Data collected via computer and data ecliection forms
- Debrief'mg I
Alarm Experiment l Statistical Comparisons i
+
What is the effect of spatial dedication of displayed alarms?
Does this effect depend on level of elenn processing?
Compensons: 1v2 7v4v8 iv2v7t4
+ What is the effect of the integration of alarm information in displays?
Compensons 7&4 v 8
+
What is the effect of alarm reduction?
Compensons. 1v7 2y3&4v5&6
+ What is the effect of the method of makirig processed alarm information available?
Does this effect depend on the level of eterri reduchon?
Comparisons: 3v4v5v6 l
+
To what extent do the above effects depend on the complexity of the event?
!~ Comparisons interectaons in the enetyees above l
l
c.
l Alarm Experiment Summary of Results e
in general, operators performance was acceptable with all alarm systems
- Modest ddferences or trends were observed for alarm characteristics
- Operators made extensive use of process displays and trends !
- Display Results
- Operators preferred the mixed display but identifed positive aspects to each dsplay format (would like all three)
- Spatial dedication preferred in high workload conditions 1
- Spatial dedication es the most important feature to operators
- Development of rapid situation assessment
- Usts were good for details but not very usable in high alarm conditions
- Integrated display results similar to alarm tiles but some alarms were hidden
- Owrators found interface management demands unacceptable in high workload conditions i
Alarm Experiment l Summary of Results (Continued) ;
- Alarm Processing
- Considerable alarm reduction was achieved, but little impact on actual task performance was observed
- Operators preferred maximum reduction
- Operators did not think important information was lost due to processing
~~
a Alarm Availability
- Operators preferred alarm suppression l
- Concerned about dynamic prioritization l
- Operators thought all alarms should be available (no filtering)
Scenario Complexity
- Complexity had a strong main effect on performance
- Performance effects are often reflected in interactions l - Operator comments reflect considerations of interactions l l
\ l 1
l j
a l Le l
i l
l Conclusions Operators use alarm systems in many different ways which require different alarm characteristics
- As an alerting system
- For situation assessment (ksd: f-;Mriven monitoring)
- Forpost4sturbanceanalysis Sest alarm system may have characteristics of all three approaches Scenario differences are very important
- Strong sflect of complexity
- Much of the variance due to scenarios was unexplained .
- Application of results
- Confirmatory evidence for alarm guidance
- Implications for new alarm system guidance is currently being assessed
- Results of new literature is being included as well
,e
y Outline of the HAMMLAB 2000 Research Agenda by J.J. Persensky,1, David Beraha,2, Jean-Luc Doutre,3, Bj5rn Wahlstr5m,4, Erik Holinagel,5, Jon Kvalem,5, Fridtjov Owre,5 Abstract A major activity within the man-machine systems research prograrn at the Halden Project during the 1997-1999 program period is to extend the functionalityand capability of the Halden Man-Machine Laboratory (HAMMLAB). This is achieved through a project called HAMMLAB 2000.
The purpose of this paper is to give an outline of a the research program planned for the HAMMLAB 2000 experimental facility.The final program will be based on a deeper understanding of the actual needs of the member organisations of the Halden Project.
A brief history of the HAMMLAB operation up till now is included, as well as information about activities in similar facilities elsewhere. The paper continues with a discussion on what the future of process control might be, leading to some basic assumptions for the research program. Finally, main topics proposed to be included in the program are discussed.
I: Utilted states Nuclear Regulatory Commission. usa 2: GesellschaA flir Anlagen- und Reaktarsicherheit. Germany 3: Electricite de France, France i:hrT Automation. Finland *
$:llalden Reactor Project. Norway
l Outline of the HAMMLAB 2000 Research Agenda Presented at The Enlarged Halden Programme Group Meeting Lillehammer, Norway,15 -20 March,1998.
- 1. HAMMLAB Operational History The Halden Man-Machine Laboratory,HAMMLAB,was established in 1983 in order to serve as the main environment for performing realistic experiments within the Man-Machine Systems usearch of the OECD Halden Reactor Project. Since its establishment, HAMMLAB has been the focal point l of the experimental meearch for Human Facton, as well as the main test bed for computerised j operatorsupport systems being developed both at the Halden Project and at member organisations. !
The NORS full-scope simulator has been the laboratory's simulator basis since the establishment of l HAMMLAB. NORS is based on the Loviisa nuclear power plant in Finland (x).
l HAMMLAB has undergone major upgrades and improvements since 1982, the last one performed in )
1996 with the introduction of a new unified human-machine interface and a new control room setup (x, x) and in 1998 with the introduction of a Large Overview Display. The upgrades have partly been made to support the Halden Project's research programmes, partly due to specific requinments set forth in bilateral funded experiments and studies. The Halden Project has experienced an increased demand for a facilityable to support advanced human factor's related experiments,both throughjoint research programmes,as well as through requests for doing specific studies for certain organisations on a bilateral basis.
The dominant type of experiments during the first years of experimentation in HAMMLAB were evaluation of computerised operator support systems. Today, large-scale human factors experiments are dominating.Such experiments are both time consuming and costly, because they require extensiw laboratory time and significant manpower for planning and analysis. Most of these large-scale ;
experiments require perticipation of commercial operators from the Loviisa plant knowing the NORS j process, while in the first HAMMLAB experiments Halden operators were largely used as experimental subjects. Appendix 1 presents a summary of experiments being performed in HAMMLAB in the period from 1982 to 1997.
The HAMMLAB 2000 Project The HAMMLAB2000 project was initiated by means of a pre-project early in 1996. The conclusions
~of tbe pm-project were presented at tbe Enlarged Halden Programme G roup meeting at Loen, Norway in May 1996 (x). The overall conclusion was that today's HAMMLAB would not fully meet the demands of the coming decades. Following a thorough discussion within the Halden Programme l Group, support was given for starting the HAMMLAB 2000 project with the aim of building an experimentalfacility to meet the research demands of the futun. The focus of the HAMMLAB 2000 project is summarised below:
Introduce new simulators, a western PWR and a BWR, with extended simulation capabilities to cover operational modes from low power to accidents (x,x),
e introducea softwareand hardwareinfrastructurepossessingthe necessary flexibility to allow re-structuring and easy lategration of new systems (x,x), and a introducing an experimental environment, including laboratories, experimental equipment, data collection facilities, and analysis facilities, arranged to suit future experiment demands (x).
The member organisations of the Halden Project have exprer, sed a clear interest in following the development of HAMMLAB 2000 through establishing two advisory groups, one having created an I
laternet-based discussion forum, the other operating in closer co-operation with the HAMMLAB 2000 project and reporting directly to the Halden Programme Group.
2 2.1 The HAMMLAB 2000 Task Force The Halden Programme Group proposed to establish an advisory group for the HAMMLAB 2000 project, directing the Halden staff. Based on this proposal, the HAMMLAB 2000 Task Force was established early in 1997.
The main tasks of the HAMMLAB 2000 Task Force were identified as follows:
a follow the progress of the project a give advice to the Halden Project staff
- report to the Halden Programme Group
= visit trievant facilities in order to provide ideas to the Halden Project, and, maybe the most important:
e assist in defining a research agenda for the new facility.
The Task Force has regular meetings, three in 1997, and so far the meetings have combined visiting relevant facilities to provide input to HAMMLAB 2000, and discussingthe HAMMLAB 2000 research agenda.
~
- 3. Work Related to HAMMLAB 2000 Performed Elsewhere When building an experimental facility like HAMMLAB 2000 it is worthwhile also to look into similar work being performed elsewhere. It is important to establish contact with organisations, having comparable facilities.or a slightly different focus, such that relevant experience can be collected. Even though the Halden Project itself, with its 15 years of experience of running the HAMMLAB czperimental facility,has a considerable history oflessons learned, collection oflessons learned from external sources would be very valuable as input to HAMMLAB 2000.
There are not many com parable facilitieswithin the nuclearcom m unity.bowever,a few are mentiones below. Halden stafiregularlyvisit these facilities within the nuclearindustry. Within other industries l l
there are more research laboratories with focus on human factors' research, e.g within the aviation l' indestry. The HAMMLAB 2000 Task Force visited several facilities within the aviation field on the
[ East Coast of the United States in September 1997. A short summary of these visits are also given
! below.
Tokyo Electric Power Company (TEPCO), Japan TEPCO at Kawasaki, Tokyo, Japan has established a Research Simulator siming to study human factors issues and performing experiments (x). The simulator was installed in a technology developenent centre in 1995, and is similar to HAMMLAB in that it consists of a fully screen-based control room. No conventional panels are used, all operator actions are performed through the computer-basedinterfaces.Several studies have been performed since the establishment in 1995, e.g.
sandy of possible effects of the change to Safety Integration units on operators' response, e improvement of annunciation system,
= lastallation oflarge display panel.
3.2 Human Factors Laboratory (HFL), NUPEC, Tokyo, Japan NUPEC is a governmentalorganisation under MITI with the main responsibilityto assess the integrity of NPP's including checking of Human Factors issues. In 1996 NUPEC established a small scale experimental facility in central Tokyo to study HRA methods, workload measures etc. (x). This facility has two operatorstations,a supervisors desk and an experimenterconsole. it has a generic CRT based MMI, that is,it is not a copy of an existing plan. The simulator modelis a copy of one of the newest Japanese 1100 MW PWR's. The model goes into beyond design basis accidents.
NUPEC is now in the process of developing a long term research plant for the facility, as the basis for the experimental studies. NUPEC has defined a series of safety issues and are now mapping a range of measures that will have an impact on these safety issues.
Up till 1997, NUPEC has focused on the following 3 lines of research:
. Human behaviour and modelling
= Development of Human reliability analysis method
~ ^
= Establishment of Human error event database In the coming short term period NUPEC will focus on the following three research areas:
= Advanced research on man-machine interfaces
. Development of more concrete methods to reduce Human Error
. Establishment of human error event database.
In the HFL laboratory NUPEC has facilities for taking a full range of physiological measures, including eye-mark reconting and movement recording. The equipment has been used in Workload assessment studies. Finally, the HFL facility contains several utility rooms where stilities can come in and use for own experimentation.
~
3.3 Japan Atomic Energy Research institute (JAERI), Japan Jaert's facility for human factors studies includes a reactor simulator, the interface, and the experimen tal control and monitoring equipment (x). The reactor simulator provides a high-fidelity simulation of a nuclear ship reactor system. This is a two-loop PWR with a rated core power of 36 MW, thermal. The simulator provides capabilities for modelling 47 malfunctions.
The interface consists of two large display units (i.e.,100-in. monitors) and three control terminals with two CRTs each. Operators can access 64 display pages and 260 control panels. 2400. variables are displayed,700 analogue and 1700 binary. The exper! mental control and monitoring equipment consists of an engineering workstation for system control, AV recording equipment, and eyemark reconiers.
Other features of the experimentalfacility include controllability, rapid implementation capabilities, and a replay function. The system is updated every second, the system response time is less than a second,and display pages are updated every 0.25 seconds. A display format can be drawn in 2-3 days, on average,providing an easily-es pandableinterface. A replay function,used mainly for data analysis, allows experimenters to replay cperator actions such as choosing display pages and implementing control actions.
A detailed analysis of verbal protocol data has been performed to identify operator strategies. The time line data, sequential data consisting of simulator logs and operator dialogues, was analysed to determine the operator's navigation among sub-systems and to associate dialogue with functional systems.
3.4 Human Factors Research Centre (HFC), CRIEPI, Japan Ten electric utilities fund the private research organisation CRIEPI.The HFC was established in 1987 to improve safety and reliability of NPP's by reducing human errors during operation and maintenance.
Today HFC has two main lines of activities:
= Learning lessons from the past
. Investigating human characteristics 1
- 1
'A program called "Siinalatingcognitiveand behaviouralmeasures"has developed a team behavioural model called SYBORG in close collaboration with expert NPP operators operating a BWR training simulator.From thesein-depth studies and interviews an operator model and team model have been established in order to emulate operator's thinking patterns and to identify the aims behind their actions and utterances. The system is operational in the HFC lab where this " electronics operator team"is directly coupled to a compact simulator.
In the HFC lab CRIEPI also has the capability to take a full range of Telemetry physiological j measures, including eye-mark recording and movement recordings.
I i
3.5 Korean Atomic Energy Research Institute (KAERl), Korea KAERI's lategrated test Facility (ITF) is a simulator-based human-machine systems research laboratory. It is based upon a full-scope simulation of a two loop Korean PWR. The experimental control room is of the advanced type, with two groups of five screens, a large overview display, a supervisor position and an expedmenters' gallery (x). The ITF is operational from May 1997, and slace then one experiment has been run to evaluate their ADIOS (Alarm and Diagnosis Integrated Operator Support) system.
The background for establishing ITF in 1997 was to improve NPP safety through human error reduction, to respond to enhancement of Regulatory requirements on Human factors and to study ecnerging issues related to operators role and information processing due to the increased use of computer technology.
ITF consists of a human machine simulator including a full scope PWR simulator, instructor and engineeringstations,large-scale display and touch screen CRTs, window alarm tiles, data stomge and analysis systems,and a full range of measurement capabilities, including telemetri and physiological techniques, eye movement tracking, audio / video reconting and a motion analysis system.
ITF will be a test-bed for the development of HMI's, experimental evaluations of HMI designs, collection of human error data, analysis of operator cognitive mechanisms, research on automation level, development and evaluation of operating procedures.
3.6 Visits to Research Labs within the Aviation Industry In September 1997, the Task Force visited three laboratories within the aviation industry. All were located at the east coast of the United States, and they were:
- Federal Aviation Administration's(FAA)TechnicalCentrein Atlantic City
. The MITRE Corporation in Vienna
- The NASA Langley in Hampton 3.6.1 The FAA Research, Development and Human Factors Laboratory The Research, Development and Human Factors Laboratory (RDHFL) provides a state-of-the-art facility where aviation related human-factors are studied in a controlled scientific environment. This research environment is specifically designed to measure and assess buman performance and workload. RDHFL also investigates how new technologies should be lategrated into Air Traffic Contml and Airways Facilities systems.
De benefits to the HAMMLAB 2000 project were first of all found in the flexible and adjustable physical infrastructure of the RDHFL. Many of the chosen technical solutions should be carefully evaluated in relation to the HAMMLAB 2000 implementation.In addition, similarities were found in the experimetal set-up, use of experimental equipment for performece measures, and the human factors experimental programme, as such.
i
3.6.2 The MITRE Corporation The MITRE Corporation has an Integration and Interaction laboratory (I-lab) where they provide state-of-the-art real-time simultaion capabilities to conduct valid, credible research programs. FAA is the maain partnerand established the I-lab to develop dynamic simulation models of the Air Traffic Control for evaluating research products.The main experimatal focus in I-lab is more like usability stadies, than directly human factors research.
The benefits to HAMMLAB 2000 were mostly experience related to a basis hardware / software infrastructure, and not so much their experimental programme.
3.6.3 The NASA Langley ne NASA Langley has many unique facilities,among them a large aumber of flight simulators.Many of them can easily be reconfigured to qualify as other aircrafts.The partners of NASA Langley are the majorairline constructors,like Boeing and Lockheed Martin, the militaryand the FAA. Only partners are able to use the NASA facilities.
The benefits to HAMMLAB 2000 were numerous, NASA Langley's experience in using the latest in computer technology, their long-time experience in building advanced and effective infrastructures, their experiencein human assessment measururements,and a human factors research programme of great similarity with the types of studies addressed within the Halden Project.
The Future of Process Control It is an extremely difficult task to try to look into what process control might be in the 21st century.
The problems are easy to understand if we look back and consider what the available technology was e.g. in 1970 or 1980.Who would,at that time, have been able to predict correctly the present situation?
Similarly,who can confidently predict what the situation will be like ten, or even five, years from now?
Although the technological development may be unpredictable, the human operator serves as a stabilising factor. People are known to be very flexible and adaptive,but this adaptation occurs within rather narrowly defined limits. The human capacity for perceiving, thinking, and acting develops so slowly that it for all practical purposes may be considered constant. This makes the prediction of future research needs for controlling complex processes a little bit easier, because it must concentrate
'on developmentand adaptation of new technology startingout from the characteristics of the human being.
The development of process control has been driven to a large extent by the technology. This is true both for I&C and the information presentation and control - better known as the Man-Machine Interaction (MMI). The imagination of designers is stimulated by the available technology, and this often leads to solutior.s that look promising, but in the long term may be less advantageous than originally thought (x).
As an example, consider the problems of large, centralised control rooms. The centralised control room was necessary to improve efficiency;it meant that a few persons could monitorand control many different processes (e.g. a paper mill or a power plant). But centralisation also led to an increasein the
1 l'
, amonat ofinformation (measurements and control) that was provided,and went along with increased automation and increased complexity. This created some basic human factors problems which have been with as ever since. The advent of computing technology has not solved these problems. On the contran,it has addel new problems- some due to bad solutions of the old problems (such as simple-aminded computerisationofconventionalinstrumentationand alarms),and some due to the lure of new technological possibilities (such as over-reliance on expert systems and graphical interfaces).
In enter to make any pudictions for the future of process control, it is necessary to make some basic assumptions, and the following assumptions are considered pertinent:
l . People (humans) will remain an essential part of the control of complex dynamic processes. In otherwords, there will not be full automation.Sub-systems may become fully automated, but they will always be embedded in a larger system. Experience seems to indicate that also in the foreseeable future any system can fail, hence there is always the need for people to keep the system running. ,
. The scope of control will increase in terms of the system boundaries and the time horizon. The j system boundaries will grow and technology will enable processes to be coupled over significant I distances. The control will be of the coupled process rather than of the local process which may be highly automated and control need no longer be confined to a central control room.
. The time horizon must be extended to ensure a sufficient economy.It is necessary to plan not only for the current situation, but also for the future,e.g. after a change of process parameters. Down-time must be reduced and availability increased, involving aspects of preventive and state-based maintenance. Safety will, however,still be in focus both on the short-term and the long-term scale.
. There will be a greater need to predict as part of control, hence a need for modelling and simulation.This follows partly from the extension of the system boundaries and time horizon, but also from the increased speed and complexity of processes. Prediction is an essential function both !
for the process control operator and for managers, and for operation as well as e.g. maintenance (out.4ges), procurement, decommissioning, design, safety assessment, etc.
. Organisationalmemory will still be very short. It is a common trait that organisations only seem i to be able to learn from recent events,indicatingan organisationalmemory span of 2-5 years.This obviously has consequences for how a process is controlled, since learning from the past is just as )
Important as predicting the future. If the memory span is too short, the organisation is doomed
~ to repeat previous mistakes and adaptation will be insufficient.
. The design of MMI has on the whole been driven by technologicalinnovation rather than by sound !
human factors principles or user needs. Despite a growing awareness of the importance of human factors,it is probably safe to assume that the development of MMI will to a laqge extent continue to be driven by technology. Since people will be the same with their recognised strengths and weaknesses, this will perpetuate the known problems and probably even create some new ones.
( These basic assumptions creates the basis for preparing a research programme for man-machine systems research in general, as they point to issues for further research. Parts of identified man-machine systems research will require the existence of advanced experimental facilities. This means that the future experimental programmes will set strict requirements to the experimental facilities,
Incindingexperimentalequipment methodologiesand measures,availabilityofexperimentalsubjects and knowledgable experimenter staff.
l l
Building a Research Programme - Reviewing User Needs i
When developinga research programme for HAMMLAB 2000 it is important that the items are not ;
only extrapolationsof current developments,but also more visionarystatements,albeit educated ones, . '
about the likely state of science 10-20 years from now. It is necessary to develop a comprehensive noenrch programme for the following resons:
To identify important research topics. HAMMLAB 2000 must address the problems of today as well as the needs of tomorrow, the year 2000 and beyond. The research agenda must therefore l combine short and long term needs, and bridge between the two. ')
The research agenda must consolidate future research topics with the HRP joint research programme, and ensure interaction with and feedback from member organisations. !
. The research agenda mest also help to specify requirements for success in research. This includes the required human skills, the methods and techniques to be applied,the models and theories that constitute the conceptual foundation, and the staff. Other aspects are the technological facilities, l
the physical and functional infrastructure, the organisational capabilities and services, and the aspects oflaternal and external co-operation. l There is,in particular, three nuclear organisations that have a strong and positive international reputation of taking the problems related to human factors seriously in terms of research programs
]
- and development of guidelines.The aim, of conne,is that nuclear control rooms are designed to good human factom engineering principles and that operator performance and reliabilityare appropriately ;
supported ensuring public health and safety. The three organisations are the United States Nuclear Regulatory Commission (USNRC), the International Atomic Energy Agency (IAEA) and the OECD Nuclear Energy Agency (NEA). In the following, quotations from important documents from these organisationsare used to demonstrate the importancethey place on the Human Machine Interaction issues.
5.i The international Atomic Energy Agency -IAEA The International Nuclear Safety Advisory Group (INSAG,1988) of IAEA, in their basic safety principles, indicated that " .one of the most important lessons of abnormal events, ranging from minor incidents to serious accidents is that they have so often been the result ofincorrect human action".
Further," continued knowledge and understanding of the status of the plant on the part of the operating staff is a vital component of defences in depth."
In the IAEA Safety report: " Safety Issues for Advanced Protection, Control and Human-Machine Interface Systems in Operating Nuclear Power Plants" (IAEA,1997) one chapter deals with safety approaches for human-machine interfaces. In the introduction it is stated: "The human-machine lateraction problems are complex. In many applications, the role of the human operators is often
W f* i l neglectedin design and the human functions are defined by default, governed by the limitations and I l gaps of hardware and software. It is questioned whether the role defined by implication for the human L operator can be effectively and reliably performed. For example: l l Is information presented at a sufficiently high level that it supports human decision making? l l
. Does information integration cause additional cognitive burdens? '
= An displays easily readable and understandable?
= Is information readily accessible?
Operators are often conservativeand reluctant to accept technology changes.To avoid problems with user acceptance, verification and validation program has to be prepared to ensure adequate testing of the new human-machineinterface. Dynamic testing and validation utilising training sim ulators may reveal possible problems long before plant installatloc. The installation of new systems and ways to work has influence on the whole organisation and should be carefully evaluated." j 5.2 OECD Nuclear Energy Agency - NEA OECD NEA has established a Senior Group of Experts on Safety Research (SESAR) with members having wide responsibilitiesand experience in OECD Member countries' nuclear power programmes.
The task of the SESAR group is to review current situation in Member countries with regards to safety research,to reflect on a rational for safety research in the yean to come, to identify future needs, and to establish a priority list (NEA,1993). j In the 1993 report it is stated on human factors (page 23): .." Humans have a vital role to play in all aspects of normal operation and their performancein the role can determine the safety of the activity. l Additionally, the flexibility and ability of humans can contribute significantly to the successful I management of accident situations which have developed. In all these aspects the human factor has become ofincreased importance.The topic embraces man / machine interfaces and communication in the control room and other plant areas.; the use of computers and the reliability of the associated software; the role of simulators in training and simulation exercises; efficient and effective maintenance and its quality assurance; total safety management; system effectiveness; characterising the performance of individuals and groups in modelling the total plant safety system; and the optimisation of the balance between human activity and engineered, automatic response. It is a vast and difficult ama involving laterdisciplinary research from sociological, physiological, and technical areas where the use of cognitive science techniques is becoming essential in addressing human performance."In the Priorities chapter (page31)it is stated: "It is the opinion of SESAR that it is impossible to exaggerate the importance of establishing an improved understanding of the performance of humans in the vast range of activities that are related to reactor safety."
l In the 1995 SESAR follow-up report (NEA d raft, November 1995): " Nuclear Safety Research in OECD l
countdes, Areas of Agreement, Areas for Further Action, Increasing Need for Collaboration" some of the major themes for further research in the areas of Human Factors are:
Characterising and assessing the performance ofindividuals, teams and organisations
- Man-machine laterfaces and communications in the control room and other plant areas l
e e
= Selection and training of staff,'
and in the area of Plant Control and Monitoring are:
. . Signal validation methodologies for severe accident situations j
=
Developseent of operator support systems using advanced data processing and human-machine laterfaces
= Condition monitoring methods.
5.3 United States Nuclear Regulatory Comission - USNRC The United States National Researth Council did, by the end of 1994, appoint a committee to examine the use of digitalinstrumentationand control systems (I&C)in nuclear power plants. In phase 1 of the work the committeedefined six important safety and reliabilityissues that arise from the use of digital I&C(x):
= software quality assurance
=
common mode software failure potential
= safety and reliability assessment methods
= human factors and human machine interfaces a
dedication of commercial off-the-shelf hardware and software.
In phase 2, the committee identified criteria for review and acceptance of digital I&C technology in both retrofitted and new reactors of advanced design.
The former Director of the Office of Nuclear Regulatory Research at USNRC, Dr. Eric S. Beckjord
- wrote a 10-year vision on NRC Research in 1995. Here he underlines that "internationalcollaboratie is vital to success and safety of nuclear installations, including operation, regulation and safety research. In the case of safety research the benefits ofinternationalcollaboration are cost sharing and avoiding unnecessary duplication of effort in experimental projects, bringing the minds of best qualified people together on a world scale, and rapidly dissemination of results."
- 6. Building a Research Programme - Defining the Framework Before entering the task of trying to identify topics for a future research program,it is important to establish an overall basis, or framework, for this research program. Below is listed some items of importance to consider for such an overall framework.
Approachesfor use ofthe Laboratory A discussion of basic approaches for use of the facility should be initiated.It is no reason to believe that
]
a drastic change of use will take place in HAMMLAB 2000, as opposed to today's activities, i.e.
I l
F6 i
- t. 1
- Human factors experiments
- Studies related to control room design
- Studies related to system design.
It is difncult to predict the division between these three diffetent types of studies or experiments.
However, each should carefully be taken into consideration when developing the laboratory, as well as in the research programme preparation.
- Type andsize ofExperiments Another aspect which should be carefully considered in the process is the division between large-scale and small-scale experiements and studies. The trend in today's HAMMLAB has been to perform a j small number oflarge-scale experiments, while a judicious mixture of a few large and more smaller
- i. experiments per ynrare foreseen in HAMMLAB 2000. The laboratory should be developed to meet such requirements.
- Expandedoperationalmodes l
The introduction of new simulators with extended capabilities into HAMMLAB 2000, improves the I l
possibilities for performing studies in a broader operational domain. Low power operation and accident states can not be adressed in today's HAMMLAB, but the availablity of such simulation capabilities in the future, will enable studies of such operational states.
- Team co-operation, distributed work It is a truism that a person never works alone, but despite that most experiments have looked at the
- single operator. It is time to relinquish that restriction. Furthermore, the technology encourages
- decentralisationand distributed work. Research topics are collaboration,com munication,information sharing,co-ordination between people and technology,regardless of physical and temporal locations.
- More realistic work settings There have been two different schools in human factors research: the well-controlled laboratory j experiment and the field study. Both have provided useful results, but they have sometimes been in !
apparent conflictin terms of the concepts and theories they have used.The revision of the information )
! processing basis means that there is less emphasis on strict experimentalcontrol,and it may therefore ]
be an appropriate task for an experimental facility to demonstrate that the two can be combined.This
.will lead to experiments - or " simulated field studies" - with a higher degree of realism (longer scenarios, more realistic tasks, more naturalistic and complex work settings, work in teams), to j supplement the mon conventionalhypothesis testing experiments. This will require the development !
of a different set of methods, e.g. for analysing complexity, understanding how work becomes organised, modelling the lateraction between people (communication and control), etc.
l
- Increasing thepoolofoperators !
l The availabilityof few commercialoperaton has been the main limiting factor of today's HAMMLAB.
With the introduction of new simulators of common western type of nuclear power plants, the availability of a broad pool of commercial operaton will ease the situation of getting access to 4 experimental subjects.
s
- Human measurements, data collection and anelysis In order to perform studies and experiments with success, collection and analysis of data is maybe the l most important issue. Measuring the response of human beings can be made in many ways, e.g. by objective measurements, performance measurements, physiological measurements, or simply by making subjective assessments. Developing new, and better methods for assessing the performance of human beings will be an important research issue in the coming years, closely coupled to the j experimental facility.
Experimental equipment, e.g. audio / video, eye movement tracking, physiological measuring equipment, plays an important role in data collection. Such data collection facilities improve all the time,and its use will be graduallyextended in experimentsas equipment gets better and requirments to extensive data collection is expanding.
- Deliverables There will be different types of output, or deliverables,from the research activities in Hammlab 2000 such as:
. experimental results, from e.g. testing of systems, operator performance in dynamic scenarios, development and use of human factors methods and measures a technical basis for regulatory guidance e lessons learned reports,i.e. data collected and analysed from a series of experiments a support systems,i.e. various prototype systems developed a methods, used in the development of systems a tools, developed in the course of system development The output from the activities will come either as research reports or in terms of computer programs.
- 7. An Outline of a Research Agenda Based upon the assumptions stated above,the current research being performed, and the items of the overall framework, a prediction of possible themes for future research can be made. Some of the themes are continuation of current research,some are deduced from today's research. The following sections give a first proposal of possible topics for a future HAMMLAB 2000 research programme.
7.1 Control room engineering The transistion of control rooms from the conventional panel-based control rooms, th rough the hybrid control rooms, introducing computer-based information presentation, to the modern control rooms with complete CRT-based supervision and control, will become a major issue within the nuclear industry in the years to come. Such a transition rises many important issues to be studies more carefully.
i 1
l
o l .-
[*
i 7.1.1 Advanced control room layout and development Issues Development of basic rules for design of computerised control rooms
. How many screens should be installed acconting to various criteria?
- To what degree a re Operator Support Systems needed ( the whole functionalspectrum; pre-alarm, ;
alarm, diagnosis, prognosis, procedure, optimisation of normal operation, planning, incident and !
accident management, etc.), as separate or integrated solutions Long Term Issues Advice on transition from conventional to screen based control rooms (I&C backfits)
. Quantify the impact in terms of efficiency and risk of a computerised control' room versus a l conventional one 7.1.2 Design and Evaluation of Human Machine interfaces Issues
= Design and Evaluateememingdisplay proposals such as overview display, task oriented and task adaptive displays, compact displays, smart displays
=
Human-computerinteraction devices (mouse, trackerball, light-pen, touch-screen,hard versus soft keyboard, voice I/o, etc.)
- How to use " soft" buttons and icons to obtain optimum operational safety?
Long Term Issues
- Advanced Display Design for Process Control
. Information integration and adaptive information display 7.1.3 Information Density and Navigation in Computerised Information Spaces Issues .
3 Develop the concept ofinformation density
. Refine various navigation principles and techniques
. What is the best way to navigate in computerised workspaces?
Long Term Issues l
= What is the adequate level of details to present to the operator?
l l
7.1.4 Response Times in Computerised Control Rooms Issues Evaluate a comfortable time response of computer driven interfaces as conceived by operators
- Evaluate time response of hardwaaw/soAware infrastructures and communication networks Long Term Issues
. Real-time issues for coupled / dependent processes 7.2 Human Factors and Cognitive Engineering issues Understanding the impact of new technology on the role and performance of operating personell is crucial la decision making concerning safety of nuclear power plants. The Halden Project aims to obtain such an understanding through a combination of activities addressing operator cognition and information processing in various control room situations, function and task allocation methods, and tests and evaluation of support systems. The program will also provide feedback on the usefulness of new methods and measures for studying operator performance.
l 7.2.1 Automation Issues Define and determine the specific short-term and long-term effects ofincreased automation a Develop methods to define the optimal degree of automatio t
. Task sharing vs. task allocation vs. task trading Long Term Issues What is the impact of automation on safety and efficiency Quantify the impact of automation on operator efficiency, operator workload, and team organisation
- Improving operator ability to intervene when automation breaks down
.* ' Development of trust / distrust in automation
. Model-based prediction of automation effects !
= De-skilling, segregation of functions," clumsy" automation
. Balanced work, the impact of automation on people and organisations 7.2.2 Human Performance Failures and Human Reliability .
Issues Why does human erroneous actions occur, can we predict human erroneous actions, can operators
o s
e.
rectify erroneous actions?
9
= Complexity factors
. Measures of human erroneous actions Long Term Issues
. Modelling and prediction of human erroneous actions a
Cognitive failums as a function of personality or technology
= How can HRA be improved based on models and experiments?
. How can this be lategrated with PSA?
7.2.3 Human Performance Measurements and Modelling Issues ModcIling and Measunments of human performance
. Individuals and crews Long Term Issues Generation oflarger sets of human performance data from limited-subject de/.a
. Valid'ty of specific measurements 7.2.4 Computer Based Training Issues Use of VR and Equipment emulation a
Computer based training versus Intelligent training systems
. On the job trainirig Training needs analysis (systematic)
Evaluation of training programs Long Term Issues
=
Intelligent training systems
= User (student) modelling
. Adaptive on-line training systems 7.3 Plant Monitoring and Control This program aims at improving the support given to operatom in computerised control rooms for a variety of tasks in different operating modes ranging from normal operation to accident
fo l.
management. The development program relies on Hammlab 2000 to evaluate methods and prototype systems through realistic experiments where operators take place as test subjects. The systems to be developed will include general methods and tools which can be readily adapted to specific plants.
l 7.3.1 Plant Surveillance Issues Support in low power cad shutdown operation
=
Willalarm processingand event detection based upon neural nets and fuzzy logic improve alarm
- handling? Can it reduce the amount of effort involved in parametrization?
l Long Term Issues Distributed, collaborative process control: Same/different time, same/different place The " electronic" operator rather than support systems 7.3.2 Plant Performance and Optimisation j Issues How can planning, optimisation and maintenance activities benefit from advances in computer technology?
= Plant performance support systems
- Maintenance support systems
- Operational planning i
. Optimisation support systems l Long Term Issues
. Impact of Prediction Simulation models on Operation
_ Investigate various means to support the operator in normal and fault operation conditions 7.3.3 Accident and Emergency Management Issues Accident Management: How plant personnelcan be trained to understand unfamiliarand rare events and act in the best way? Will they benefit from introduction of advanced simulator tools including severe accident simulations? How to best co-ordinate activities and communicate among teams in accident conditions?
- Crises team test and evaluation l
1
e' b.
~-
c.
. Connections to national drills Evaluation of accident and ememency management procedures Long Term Issues Analyselaformation needs, communicationand lateraction smong personnelin accident conditions.
- Develop prototypes with information from accident simulators.
. Modelling of accident management Experiments with several teams with different roles during accident management.
- Integration of tools and procedures 7.3.4 Computerised Procedures Issues How can teams work with computerised procedures and how should procedures be presented in different kinds of control rooms?
. How to improve V&V of procedures?
. Good precedure develop;nent and maintenance practise Determine the cognitivedifferences between using a state based procedum and an event based one
. Long Term Issues
- Hybrid Procedures
- Integration between procedures, alarms and general human machine interfaces
- 8. Next steps This paper r,. em arises ib background material and the present ideas that the Task Force and the Halden Projdt have for the research programme for the new laboratory. The work of developing a )
research programme will continue la parallel with the development of the laboratory, and it will be carried out in close co-operation with the Halden Project's member organisations.
'I5e Halden Project rely on their membem to come up with more input to this important work. In addition, the Halden Project will utilise the identified HAMMLAB 2000 Task Force, and involve them in the further preparation of the research agenda.
<