ML20132E263

Forwards Comments on AP600 human-sys Interface Design Implementation Plan Re Element 7 of Hfrprm

ML20132E263
Person / Time
Site:	05200003
Issue date:	12/19/1996
From:	Huffman W NRC (Affiliation Not Assigned)
To:	Liparulo N WESTINGHOUSE ELECTRIC COMPANY, DIV OF CBS CORP.
References
NUDOCS 9612230187
Download: ML20132E263 (20)

Text

r g

/perg%a UNITED STATES g

,j

~t NUCLEAR REGULATORY COMMISSION WASH!NGTON, D.C. 20666 0001 l

k 8 December 19, 1996

%. , * ,o l

Mr. N'cholas J. Liparulo, Manager Nuclet.r Safety and Regulatory Analysis '

l Nuclear and Advanced Technology Division Westinghouse Electric Corporation P.O. Box 355

Pittsburgh, Pennsylvania 15230

SUBJECT:

ComENTS ON AP600 RELATED OPEN ITEMS ASSOCIATED WITH ELEMENT 7 0F THE HUMAN FACTORS ENGINEERING PROGRAM REVIEW MODEL (HFEPRM)

Dear Mr. Liparulo:

In a letter to Westinghouse dated July 25, 1995, the Nuclear Regulatory d Commission staff provided comments on the AP600 human-system interface design I implementation plan related to Element 7 of the HFEPRM. Subsequently, Westinghouse revised the human factors discussions in Section 18.8 of the AP600 standard safety analysis report (SSAR, Revision 9) and submitted i WCAP-14396, " Man-in-the-loop Test Plan Description" to address the staff's I comments. The staff has reviewed this material and provided an update on the

Element 7 open item status as an enclosure to this letter.

i In a letter to Westinghouse dated March 22, 1996, and during a meeting with '

Westinghouse on May 21 and 22, 1996 (see meeting summary issued June 19, i

1996), the staff noted that Westinghouse has supported its AP600 desigt; certification with material that has never been docketed with the NRC.

Spec'Tically, information contained in the Westinghouse AP600 Program Operating Procedures document (WCAP-12601) and the Design Review Manual (WCAP-9817) is relied upon to support the staff's safety evaluation of the AP600 human factors engineering program. Until the pertinent information in these references is docketed with the NRC, the staff will not be able to reach a final safety conclusion related to the AP600 human factors engineering program.

J i umEniCaqs COPY

$9 O

i 9612230187 961219 PDR ADOCK 05200003 A PDR

December 19, 1996 If you have any questions regarding this matter, you can contact me at

.(301) 415-1141.

Sincerely, original signed by:

William C. Huffman, Project Manager Standardization Project Directorate Division of Reactor Program Management Office of Nuclear Reactor Regulation Docket No.52-003

Enclosure:

AP600 DSER Open Item ,

Resolution of Element 7 Human-System Interface Design-cc w/ enclosure:

See next page DISTRIBUTION:

iDocket File- PDST R/F TMartin ,

PUBLIC DMatthews TRQuay TKenyon BHuffman JSebrosky DJackson JMoore, 0-15 B18 WDean, 0-17 G21 ACRS (11) BBoger, 0-10 H5 CThomas, 0-10 D24 ,

JBongarra, 0-10 D24 l

j i

DOCUMENT NAME: A:EL-7REV1.LTR T2 smeelve e copy of tids document,indleste in the box: *C" = Copy without ettschment/ enclosure *E" = Copy with attechment/ enclosure *N" = No copy 0FFICE PM:PDST:DRPM BCHNFB:DSCH-lN/ D:PDST:DRPM l l l l NAME WCHuffmashWA (Tbppiff TRQuay 41M l DATE 12//V/96 12/ g 96 12/(f/96 0FFICIAL RECORD COPY I

i

Mr. Nicholas J. Liparulo Docket No.52-003 Westinghouse Electric Corporation AP600 cc: Mr. B. A. McIntyre Mr. Ronald Simard, Director Advanced Plant Safety & Licensing Advanced Reactor Programs Westinghouse Electric Corporation Nuclear Energy Institute Energy Systems Business Unit 1776 Eye Street, N.W.

P.O. Box 355 Suite 300 Pittsburgh, PA 15230 Washington, DC 20006-3706 Mr. John C. Butler Ms. Lynn Connor Advanced Plant Safety & Licensing Doc-Search Associates Westinghouse Electric Corporation Post Office Box 34 Energy Systems Business Unit Cabin John, MD 20818 Box 355 Pittsburgh, PA 15230 Mr. James E. Quinn, Projects Manager LMR and SBWR Programs Mr. M. D. Beaumont GE Nuclear Energy Nuclear and Advanced Technology Division 175 Curtner Avenue, M/C 165 Westinghouse Electric Corporation San Jose, CA 95125 One Montrose Metro 11921 Rockville Pike Mr. Robert H. Buchholz Suite 350 GE Nuclear Energy Rockville, MD 20852 175 Curtner Avenue, MC-781 San Jose, CA 95125 Mr. Sterling Franks U.S. Department of Energy Barton Z. Cowan, Esq. I NE-50 Eckert Seamans Cherin & Mellott '

19901 Germantown Road 600 Grant Street 42nd Floor l Germantown, MD 20874 Pittsburgh, PA 15219  !

I Mr. S. M. Mcdro Mr. Ed Rodwell, Manager Nuclear Systems Analysis Technologies PWR Design Certification Lockheed Idaho Technologies Company Electric Power Research Institute Post Office Box 1625 3412 Hillview Avenue Idaho Falls, ID 83415 Palo Alto, CA 94303 Mr. Frank A. Ross Mr. Charles Thompson, Nuclear Engineer U.S. Department of Energy, NE-42 AP600 Certification Office of LWR Safety and Technology NE-50 19901 Germantown Road 19901 Germantown Road Germantown, MD 20874 Germantown, MD 20874

]

l l

)

l l

Element 7: Human-System Interface Design Element 7 is being reviewed at an Implementation Plan Review level. There-fore, Westinghouse's submittals should describe the proposed methodology in i sufficient detail for the staff to determine whether the methodology will lead to products that meet the HFE PRM acceptance criteria for the element. The ]

actual completion of the plan will then take place after design certification. j While some implementation plans can be reviewed on their own merits, the staff )

may request a sample analysis which demonstrates the application of the  :

methodology and its results. ITAAC/DAC are needed for completing the imple- l mentation plan and providing the results to the staff for review.

A meeting was held in Pittsburgh on March 8 through 10, 1995, to discuss Element 7 open items. As part of the discussions, Westinghouse agreed to make design process documentation and sample design process products, such as HFE guidelines documents, available for staff review. A review of this documenta-tior, was conducted on April 5 and 6, 1995, at the Westinghouse office in Rockville. On the basis of this review and using information obtained in the meeting in Pittsburgh, the status of the open items was reviewed. Insights and clarifications based on the review and meeting also led to a reevaluation of specific material contained in the SSAR. All three sources of information contributed to the review of the Element 7 open items. As a result of the I review several open items were resolved. The results of the review were sent I to Westinghouse in a letter dated 25 July 1995.

To address the remaining open items, Westinghouse submitted SSAR (Revision 9)

Section 18.8, Human-System Interface Design. In addition WCAP-14396 (Revision 1), Man-in-the-loop Test Plan Description was submitted to address the AP600 HFE test program. A number of open items also remained concerning SPDS. SSAR (Revision 9) Section 18.8.2, Safety Parameter Display System, was submitted to address these open items.

The following is an overview of the status of the results of the review:

Open Item (0!TS f. DSER f) Current Status (1354) 18.8.1.3-1: HSI Design Process Guidance Action W (1355) 18.8.1.3-2: Task-Related HSI Requirements Resolved (1356) 18.8.1.3-3: HSI Characteristics Action W ,

(1357) 18.8.1.3-4: Design Feature Selection Resolved  !

(1358) 18.8.1.3-5: Detailed Guidelines Resolved (1359) 18.8.1.3-6: Detailed Design Analysis Resolved (1360) 18.8.1.3-7: HSI Evaluation Action W (1361) 18.8.1.3-8: HSI Design Documentation Action W Enclosure

- - +

, (1354) Onen Item 18.8.1.3-1: HSI Desian Process Guidance

1. HSI Desian Process Guidance Criterfon: The HSI design process should be organized and documented to support its standardized and consistent use by members of the design team and their contractors. Guidance should be provided to the team for accomplishing 3 the following (each is defined in the criteria to follow):

l

• Task-related HSI requirements

• General HSI design 4

• Detailed HSI design l

!

• HSI evaluation 2

• Final HS! design documentation i DSER Evaluatfon: The M-MIS design implementation process is described in SSAR l

, Section 28.8.2.1.3. According to the SSAR, " specific implementation guidance i is provided to the M-MIS subsystem designers so that each designer implements

the function-based task analysis outputs consistently and according to human

, engineering principles established for the design." A subsystems integration 4

document is also provided "since each of these subsystems provides only a j portion of the support required from the complete interface." The process by which the design will be evaluated is described in SSAR Section 18.8.2.3.2.

According to SSAR Section 18.8.2.3.4, the results of these evaluations will allow conclusions to be drawn regarding "the effectiveness of particular M-MIS features in supporting human performance; the factors that contribute to human performance difficulty; and enhancement to the M-MIS required to improve human

performance." The specific means are not discussed by which the conclusions will provide feedback to the design process (e.g., the process by which the

conclusions are communicated to the designers and the method for establishing i that any design changes address the conclusions). The process is not de-l scribed for reflecting the results of the evaluations in the design guidance and incorporating changes into the final design documentation.

The Westinghouse response to RAI 620.40 states that implementation guidelines documents, subsystems integration documents and design basis guidelines 1

documents will not be completed until after design certification. Similarly,

) the Westinghouse response to RAI 620.34 states that the documentation that will guide the COL holder in making changes to the M-MIS will be available at the time of combined license application. The process is not described by which these documents will be developed.

1 f,

Proposed Resolutfon: SSAR (Revision 9) Section 18.8, HSI Design, addresses I the design of the HSI based on task analysis and other design inputs. It l provides a general description of the translation of task requirements to HSI resource requirements, the procedures for development and documentation of the detailed design, and design tests and evaluations. To support a more in depth I i examination of the design process, on April 5 and 6,1995 at the Westinghouse office in Rockville, the staff reviewed the following Westinghouse proprietary 1 documents describing the AP600 design process:

i N

I j *

. WCAP-12601, AP600 Program Operating Procedures (Revision 15, dated I April 1, 1995)

~

1 j 1

WCAP-9817, Design Review Manual (Revision 2, dated June, 1991) l i '

AWARE Advanced Alarm Management System Intermediate Design Review, IDR-90-15, Final Report, dated September 24, 1991.

i

{ A general description of these detailed design documents is provided in the j Element I review. In addition to supporting the review of Element I criteria, they provided information related to the Element 7 design review procedures as well. Generally, the Element I review focuses on high-level programmatic i issues while the Element 7 review addresses more detailed guidance to be used j[ by designers. Thus, additional documents which reflect more detailed design guidance to designers were reviewed as well. Based on this review the

Westinghouse HSI design process guidance was found to be acceptable for all

but the criterion 3 issue (OI #1356 which follows). Thus, until the DSER

! issues raised by the staff related to criterion 3 is resolved this item l (criterion 1) remains open.

STATUS OF QPEN ITEM: Action W 1

j (1355) Onen Item 18.8.1.3-2: Task-Related HSI Reeuir-- ats l 2. Task-Related HSI Reauirements a

l Criterfon: This criterion addresses the identification of the HSI require-j ments to support human functions and tasks using the results of earlier HFE j) . PRM elements as a basis. The requirements should address alarms, displays, controls, and operator aids. For example, the range and accuracy of displayed

! information should be consistent with operator information requirements for i making decisions regarding the plant state. Precision requirements for the

!- display of plant information (e.g., number of demarcations on a scale) should be defined to a level that is consistent with task requirements without i '

j burdening the operator with unnecessary detail (e.g., excessive number of f

decimal places). Units of measurement should be defined to be consistent across related operator tasks (e.g., operators should not have to convert

, values from one measurement system to another). The technical basis for task-l related HSI requirements should be documented.

i DSER Evaluation: The SSAR describes the function-based task analysis as a method for identifying control and displays needed for operator tasks.

!. Westinghouse's response to RAI 620.81 indicates that design reviews will identify omissions. Additional opportunities for verifying the completeness j of the design (e.g., cross-checks against emergency procedure guidelines) should be identified. The process should be described by which the correction

of omissions is assured in the final design. l 1 The function-based task analysi

: presented in SSAR Sections 18.8.2.1.2 and

] 18.9.1.3 provides a structured approach for identifying information and j

l b

1 i

. l t

l

! controls that are required for performing specific functions. While the j example provided in SSAR Section 18.9.1.3 describes how parameters and j specific values are defined, it is not clear how the range, accuracy, preci- t

. sion, and measurement units for individual displays and controls will be  !

1 defined. The means by which these items are defined in the initial stages of i

! the design process should be described.

Proposed Aesolution: SSAR (Revision 9) Section 18.8.1.7, Task-Related HSI l j Requirements, addresses the derivation of HSI requirements from task require-  ;

j ments. The task analyses to be performed in support of AP600 HSI design  ;

include " traditional" task analyses using an operational sequence analysis  :

(0SA) methodology in addition to the FBTAs as discussed above. The methodolo- i gy for OSA was reviewed and found acceptable. Included in the information obtained from these task analyses is the identification of operational i

information requirements, e.g., the alarm, parameters, and controls needed to perform the task sequences.

This information is used to develop descriptions of the HSIs. For example, a i description may include detailed information of what the display needs to provide the operator to complete a task. The description includes the necessary calculated values and supporting algorithms to support the operators  ;

task requirements.

This information acceptably addresses the staff concerns. The open item is considered resolved and the HFE PRM criterion is satisfied. i l

STATUS OF OPEN ITEM: Resolved 1 (1356) Open Item 18.8.1.3-3: MSI Characteristics

3. HSI Characteristics Criterion: The HSI should provide the task-required alarms, displays, controls, and operator aids (as defined in Criterion 3) for process monitor-ing, decision-making, and control. The HSI design should support human performance and usability through the following characteristics:

• Compatibility with the cognitive and physiological capabilities of plant personnel.

• Minimization of the demands of secondary tasks. Secondary tasks are activities performed when interfacing with the system but that are not directed to the primary task of process monitoring, deci-sion-making, and control.

• Examples include efforts operators must expend managing the inter-face, such as navigation through displays, managing windows, and accessing data. Although necessary, performance of secondary tasks detracts from the crew's performance of primary tasks.

l l

I

} '

i l  ;

i

,

• Support for the use of the HSI, such as providing (1) flexibility i l (e.g., multiple means to carry out actions or verify automatic '

, actions), (2) guidance on HSI use, and (3) error tolerance and mitigation.

. l

!

• Accommodation of human performance under the range of conditions, '

e.g., normal as well as credible extreue conditions. The design

! process should take into account the use of the HSI over the 1 duration of a shift and in plausible scenarios that may result in i

reduced visibility and ventilation or CR evacuation. The design of l non-CR HSIs, such as LCSs, should address constraints imposed by i the environment (e.g., noise, temperature, contamination) and by

}

protective clothing.

! DSER Evaluation: It is evident from examining the description of evaluation issues (SSAR Section 18.8.2.3.5) that provisions have been made to assess the effects of interface management on operator performance. For each of the major. classes of operator activity there are evaluation issues in which the dependent measures include indicators of the accuracy-and efficiency of the use of displays, controls or procedures. The workload associated with secondary tasks is not discussed in the context of the evaltation issues.

Westinghouse's response to RAI 620.84 states that " measures of workload 1 (including mental workload) will play a role in the integrated validation study" since these measures are most meaningful "when realistic and complete operator tasks are being studied" and that either the subjective workload assessment technique (SWAT) technique and NASA task load index (TLX) technique will be used to assess workload in the integrated validation study. However, high workload may also be imposed in the course of "part-task" evaluations, and provisions are not described for the detection of workload-related problems early in the design process. ,

The SSAR does not describe specific features of the HSI designed to enhance usability. The guidance to be provided to designers for correcting usability  ;

problems identified in the course of HSI evaluations should be described. l l

The description of the control room in SSAR Section 18.9.1 and SSAR Fig-ure 18.9.1-1 indicates that the operators will be sitting at individual workstations for extended periods of time. This contrasts with conventional control rooms in which operators often stand or walk about the control room to access information and perform control actions. Possible negative effects of such an arrangement (e.g., postural or visual fatigue, loss of alertness) should be considered as compared with other design alternatives. Evaluations of similar workstations in other work environments should be consulted or performed. Design rationales should be documented and features of the design intended to mitigate negative aspects should be described.

SSAR Section 18.8.2.1.3.4 states that guidance documents will direct the layout of workstations, the arrangement of the control room, and the area environmental requirements. (These documents are not among those identified as available for review). The description of the documents indicates that they will provide guidance in the context of activities and requirements of

1 i

3 1 the operating crew as determined by the operations tasks model and will

! contain references to source material. The contents description does not t i mention degraded control room conditions or environments outside the control '

room. The design basis environmental conditions in which the plant would

! still be operated from the control room should be specified, and the likely l effects on operator performance should be considered. It should be

! demonstrated that the design will support the required performance under such

! conditions. In the event of evacuation of the control room, monitoring and

! control is performed from the remote shutdown room (SSAR Sec-l tion 18.8.2.1.1.2.4). SSAR Section 18.11 indicates that the environmental j conditions of the remote shutdown room are specified such that human and

machine performance will not be degraded. Design information and criteria for some aspects of the environment (illumination, HVAC, and shielding) are I addressed elsewhere in the SSAR. This section also states that proper j acoustic criteria will be used but, it does not cite any specific standards.

References to appropriate standards should be provided.  ;

I Local control stations are described in SSAR Section 18.8.2.1.1.2.8, which

! states that the use of local control stations during normal and emergency j operations "is consistent with the overall operator staffing and performance 2

considerations developed from the task analysis." According to Westinghouse's j response to RAI 620.82, critical local actions will be identified during the j design process; these actions will be included in the verification and j validation plan.- Local control stations are described as " habitable areas;"

)- the same term is used to describe the MCR. There is no further discussion of j

environmental conditions at local control stations of how their design will take these conditions into account. A process should be established whereby i

the worst credible conditions at each local control station are identified and

! the effects on operator performance considered. The means should be specified l by which the negative effects of environmental factors (e.g., noise, heat, and I

radiation sources) and protective clothing (e.g., noise protectors, respira-

! tors, and gloves) are addressed in the design of these local control stations.

4 i

Proposed Resolutfon: On June 7, 1995, Westinghouse provided a draft response 1 to this open item. Westinghouse described how situations of high workload

would be identified early in the design process through the use of analytic

techniques and part task simulations as referenced in a revision to the SSAR on Task Analysis and in the AP600 document No: OCS-T5-001, Man-in-the Loop i Test Plan. The test plan will specifically addresses the impact on operator l performance of secondary tasks associated with display navigation and manage-ment. Westinghouse committed to provide design guidance for correcting

usability problems encountered in the course of HSI evaluations and referenced J accepted industry guidance documents and a Westinghouse-specific document j (OCS-J7C-001) to direct the layout of workstations, the control room, remote j- shutdown room, local control stations, and the areas' environmental require-j ments.

4 i The response to RAI 620.84 was incorporated into SSAR (Revision 9) Sec-

] tion 18.8.1.9, HSI Characteristics: Identification of High Workload Situa-tions. The Man-in-the-loop test plan AP600 document No: OCS-T5-001 was

.l submitted in final form as WCAP-14396 (Revision 1), Man-in-the-loop Test Plan f

l l

i

Description. Two problems were noted. First, the information in the RAI response was not included in the SSAR in detail but was summarized. In the summary, the description of approaches to subjective workload measurement were not included. Thus the revised description does not suggest an approach to workload assessment beyond indicating the subjective techniques will be used.

Second, the SSAR indicates that the concept tests will include assessments of workload for the impact of secondary tasks such as display system navigation.

The staff considers this important due to concerns over the potential for such tasks to impose high workload and to be distracting from operators primary tasks of monitoring and controlling the plant. However, the associated test described in WCAP-14396 (Revision 1), Man-in-the-loop Test Plan Description do not include workload as performance measure. Section 4.2 of the WCAP address-es the tests to be performed for workstation displays. Concept Test 4:

" Ability to navigate displays, finding information" addresses the staff's concern, but workload is not identified as a performance measure. In fact, workload is only mentioned in conjunction with one of the concept tests (Test 3) defined in WCAP-14396 (Revision 1).

Westinghouse should clarify the measurement of workload and its use in the concept tests in order to resolve this open item.

STATUS OF OPEN ITEM: Action W f1357) Open Item 18.8.1.3-4: Desian Feature Selection

4. General HSI Desian Feature Selection Criterion: This criterion addresses the selection of general HSI design features, such as the selection of a large screen MCR display panel (compared to workstation displays only) or to utilize touch screen controls (compared to hard controls or trackballs). The selection of general features should be based upon a consideration of alternative approaches for addressing the HSI design characteristics (as identified in criterion 4 above). OSER Evaluation methods can include operating experience and literature analyses, trade-off studies, engineering evaluations and experiments, and benchmark evaluations.

Such evaluations should consider the strengths and limitations of design options. The process should be documented for evaluating alternatives and the justification for their final selection.

DSER Evaluation: SSAR Section 18.8.2.3.2.1 describes the following M-MIS features as " central" to the AP600 design: wall panel information station, ]

(

functionally organized alarm system, compact workstations, functionally and physically organized workstation displays, computer-based procedures, and plant communication system. These features "are used as a starting point to define how the M-MIS is intended to support operator performance..."

Westinghouse's response to RAI 620.41 indicates that the central elements of  !

the HSI design were established based on a " comprehensive model of operator performance," which incorporates information from a variety of sources, e.g., i l

1 P

reports of problems with current control technology, studies of human perfor- i mance, Westinghouse expertise, and industry experience as reflected in the  !

EPRI ALWR Utility Requirements Document (URD) requirements.

SSAR Section 18.8.2.3.2.4 reviews the " rationale for each M-MIS feature,"

i.e., tha wall panel information station, functionally organized alarm system, compact workstations, functionally and physically organized workstation displays, computer-based procedures, and plant communication system. For each operator activity identified by Westinghouse (detection and monitoring,  ;

.l interpretation and planning, and controlling plant state), the ways in which '

the relevant features support the activity are described. However, there is i

i no explicit consideration of possible limitations of the design features, and .

i the reason (s) for choosing these features over other potential alternatives is l not specified.

i

' Additional information is needed on the process Westinghouse will use to evaluate design alternatives (e.g., documentation of decisions based on studies of human engineering trade-offs, tests of alternatives, and evalua-tions of previous applications). ,

1 Proposed Resolution: SSAR (Revision 9) Section 18.8.1.8 addressed this item.  !

! The SSAR stated that the HSI resources identified were selected as a starting 1 i point for meeting the information and control needs for general human activi- l

! ties (such as detection, planning, and control) identified in the operator

! decision-making model (described in WCAP-14695). The relationship between the )

i 3 j

human sion 9) activities and the control room resources are described in SSAR (Revi-Figure 18.8-3. For example, detection and monitoring are supported by 1 the alarm system, the wall panel information system, the QDPS and the plant j information system. The principal source for the initial selection was utility requirements and operating experience review. The acceptability of each resource and the evaluation of design alternatives for the detailed i implementation of each resource is accomplished through the test and evalua-i tions that are performed during concept testing and final V&V. The results of 1

testing will be used to refine the design. The basis of all resource design

, decisions will be documented in the functional design documentation.

i Based on this information, the DSER item is resolved and the criterion is j satisfied.

4 i STATUS OF QPEN ITEM: Resolved i

! (1358) Ooen Item 18.8.1.3-5: Detailed Guidelines 1

5. Guidelines for Detailed HSI Desian l

} Criterion: The applicant should utilize HFE guidelines for the detailed j design of the selected general HSI features, layout, and environment. This j will facilitate the standard and consistent application of HFE principles to i the detailed design. Generic HFE guidance documents should be tailored to the j applicant's specific HSI design and documented in a guidance or specification i

! I i

i 9-f document. HFE guidance documents should contain statements of their intended 4 scope, references to source materials, instructions for their proper use, and

procedures to be followed when discrepancies are found.

j DSER Evaluation: SSAR Section 18.8.2.1.3 states that guidance documents are i provided to designers of the alarm system, the information display system, the 4

controls interface, and the workstation and control room layout, arrangement, and environment. In SSAR Figure 18.8.2-1 (see also Westinghouse's response to !

RAI 620.59), a set of six guideline documents is identified

alarm guide- ,

i lines, display guidelines, controls guidelines, training guidelines, anthropo- '

! metric guidelines, and guidelines for integration of subsystems.

! Westinghouse's response to RAI 620.59 states that the guidance will be

! developed from existing guidelines documents, supplemented as necessary "to

! address issues that are not covered sufficiently." SSAR Sec-j tion 18.8.2.3.5.4.1 and SSAR Table 18.8.2-2 (sheet 25) cite NUREG-0700, MIL-STD-1472, ASHRAE 55-1981, and ANSI /HFS-100, and EPRI NP-3659 as sources.

I Limited applicability of NUREG-0700-is noted. Westinghouse's response to l

RAI 620.20 indicates that supplementary material can be drawn from a variety of sources, e.g., research on the psychology of graphic displays and ecologi-i cal interfaces, lessons learned from the experience of the aerospace industry with automation, research on navigation of computer displays, experience in 2

the design of expert systems, and techniques for cognitive modeling of i operator performance. The response indicates that Westinghouse has developed i a display design handbook and alarm design guidelines (see also Westinghouse's i response to RAI 620.49) based on such sources. Westinghouse's response to I RAI 620.59 states that guidance will be " tailored to the AP600 interface" and 1 i "may include guidance and principles developed from Westinghouse human factors i research." (Westinghouse's response to RAI 620.83 suggests that the results i of early concept test may also contribute to the tailored guidance.)

t i Westinghouse's response to RAI 620.90 states that a plant labeling guideline i will be developed which will be based on EPRI NP-6209.

I l Although Westinghouse's responses to RAI 620.43 and RAI 620.76 indicate that some AP600 human factors design documentation is currently complete, the documents referenced by Westinghouse were not available to the staff in time j for review and integration into the DSER. A copy of the display design l handbook was requested in RAI 620.59, but was not made available.

{ Proposed Resolution: To address this open item, Westinghouse made examples of

their design guidance available for staff review. They were

)

• Techniques and Principles for Computer-Based Display of Data (STC l Report 90-ISJ4-HUNIN-RI), September 1991, 1

!

• NOK ANIS Display Project, Display Standards Document, September 10,

1993; and i

]

• Alarm System Design Guidelines, Revision 0, December 1993.

i I

i J

l i

I.

yy '

e - --

i g .

i l L i i- .

i  ! c These detailed guideline products were reviewed as samples of the products of-I the Westinghouse design process. Since they were not AP600 specific docu- i i ments, the detailed contents, e.g., the actual guidelines themselves, were not i i reviewed. Such a review will take place after design certification when the +

AP600 design specific guidelines are developed.

} These documents were reviewed in terms of statements of their intended scope,

references to source materials, instructions for their proper use, and j procedures to be followed.

j The Techniques and Principles for Computer-Based Display of Data (STC i

! Report 90-ISJ4-HUMIN-R1, September 1991) identifies an approach to display '

design that goes beyond a presentation of guidelines. The guidance is fairly general and does not represent an AP600 specific application. A plant- 1 i specific document (although, not AP600) was examined which provided an example j of how the general display features are implemented (discussed in the next

paragraph below). The general principles document provides a clear statement 1 of its application and identifies many of the inadequacies of other guidance i documents. It addresses the general aspects of display design and provides

! comprehensive treatment, for example: general principles; display " atoms" l (such as font size and coding); display elements (such as labels, icons,

units,); formats (such as text, tables, trend plots, and mimics); and the 4

integration of formats into higher-level displays. The organization of the j total set of displays is addressed as well. The document provides a clear l rationale as to the basis for the guidance. This is a positive feature which

! should facilitate its use by designers in evaluating tradeoffs. The document i also contains numerous graphics and illustrations providing examples of the design principles which will further support its use by the design team. j

References to numerous appropriate source documents are included such as

the '

l Boff Human Engineering Compendium; Smith and Mosier; Tufte, 1983; Helendar,

, 1988; and NUREG-0700.

l A more detailed design-specific display standard was also examined (NOK ANIS i Display Project, Display Standards Document, September 10,1993). Display

' types were identified and presented in a hierarchal manner. The goal of each display was identified along with what information was presented (e.g.,

l status, values, reliability), and pokefields (fields on the displays which

! access additional displays). The way the information is to be displayed was also specified. Numerous displays designed in accordance with the design standard were provided.

! Alarm System Design Guidelines, Revision 0, December 1993, were also reviewed.

1 This is a very comprehensive document that addresses alarms from the perspec- ,

j tive of their role in plant operations and not simply the end-point design. I

! For example, the document addresses the historical problems with alarm system j design, e.g., the identification of alares in bottom-up fashion by the

designers of individual components and systems. This method is a different

~j perspective of the plant than when it is an integrated whole or of the integration of alarms. Further, individual system designers are inclined to j create alarms without thinking about the operator actions with which the alarm I l 1  ;

i d l 4 _ _ __

L. . l

} .

i i '

l t .

should be associated. To address this problem, a combination of top-down and

~

j e bottom-up provides a merger. Top-down refers to a definition of alarms to  !

j support operator functional and tasks.

l' The alarm system document contains guidelines on alarm identification for use I by MIS designers (top-down alarms) and plant system designers (bottom-up I

. alarms). The information that should be included in each proposed alarm is ,

i identified, including: description, setpoint, source, plant condition when l J valid, conditions which may make the alarm condition worse, plant functions  !

are related to the alarm (identified by MIS designer not system designer), l rational reasons for the alarm, operator actions, time criticality, and i operator interface locations. The technical basis for the alare guidance I

included references to numerous appropriate sources such as EPRI 3448, ALWR URD (1989), Van Cott and Kinkade, IEEE 1023-1988; and NUREGs 0737, 0696, 0800, 1342, and Regulatory Guide 1.97.

i

! In conclusion, the Westinghouse design process provides for the development of

{ comprehensive detailed design guidance and provides sufficient information _to

! support its standard and consistent application. The application of the process to AP600 guidance is addressed in SSAR (Revision 9) Section 18.8.1.2,

Design Guidelines. The specific commitment to develop HSI d% ign guidance for i

each HSI resource is identified. A general description of the content of the guidance documents is provided and includes: intended scope, references to sources, instructions for use, design conventions and guidelines, and provi-

! sions for guideline deviations based on a documented rationale.

i Based upon this information, this DSER issue is considered resolved and the l HFE PRM criterion is satisfied.

l STATUS OF OPEN ITEM: Reso1ved i

! (1359) Open Item 18.8.1.3-6: Detailed Desien Analysis

6. Analysis for Detailed HSI Desian Criterion: Design details, problems, issues that are not well defined by guidelines or where guidelines conflict should be analyzed. Analysis methods can include operating experience and literature analyses, trade-off studies, engineering evaluations and experiments, and benchmark evaluations. For example,

• Mockups and models may be used to resolve access, workspace and related HFE problems and incorporate these solutions into system design.

• Dynamic simulation and HSI prototypes should be considered for use to evaluate design details of equipment requiring critical human performance or equipment not adequately addressed by guidelines.

1 l

l DSER Evaluatfon: Westinghouse's response to RAI 620.20 acknowledges that "no j formally documented guidance exists to address many of the advanced control l room design issues" owing in part to the fact that most guidance documents  ;

maintain a conservative standard with respect to the basis for the guidance.

Westinghouse's response to RAI 620.59 states that elements of the design may 1

, be based on " guidance and principles developed from Westinghouse human factors i

! research." As indicated in Westinghouse's response to RAI 620.83, the results

of evaluations (especially early concept tests) will be important in resolving

! issues not well defined by available guidance.  !

Proposed Resolutfon: Based upon discussions held Westinghouse clarified that l the evaluation issues discussed in SSAR (Revision 0) Section 18.8.2.3.5,  !

Evaluation Issues and Descriptions, represented design details, problems, and 4 issues that are not well defined by guidelines and which are being addressed 1 l through the evaluation test program. A total of 17 issues were identified. l The last two of these are part of verification and validation and, therefore, '

are addressed in the staff's V&V review (see the Element 10 review). The l remaining 15 issues address significant HFE topics. They are organized into t

three groups based on the type of operator activity being analyzed

detection i and monitoring, interpretation and planning, and controlling the plant state.

l Issues such as use of wall panel and workstation displays to support situation )

i j assessment and use of alarm information during multi-fault events will be  !

i evaluated. Based upon the staff's understanding of the human performance l issues and guideline limitations as discussed in NUREG/CR-5908, this list appears to be comprehensive in scope.

)

{

Each issue was discussed with resect to conceptual and performance testing j phases. For each, the following information is generally provided: the

hypotheses, experimental manipulations, subject characteristics, minimum i tested requirements, measurements and performance criteria, timing (when in ,

} the design process the test should be conducted), and use of the results. The l comprehensive approach to analyzing human performance issues not addressed by l guidance should appropriately address these issues.

]

I The feedback provided to the design process for each of the evaluations was j described. For example, the results from Evaluation I will be used to contribute to the development of functional requirements for the design of overview displays for the wall panel information station and workstation.

The material used to address the DSER issue has been incorporated into SSAR (Revision 9) Section 18.11 HSI Design Test Program and in WCAP-14396 (Revi-sion 1), Man-in-the-Loop Test Plan Description. The presentation is slightly changed from the earlier material reviewed. For example, the test are not described in terms of conceptual and performance phases and the information provided for each is slightly changed. However, the changes do not negatively impact the quality of the material. (Note the issue raised in connection with Open Item 18.8.1.3-3: HSI Characteristics related to workload assessment is not addressed here).

Thus, based upon the information reviewed, this DSER issue is considered resolved and this HFE PRM criterion satisfied.

1 i

1 l

STATUS OF OPEN ITEM: Reso1ved '

(1360) Open Item 18.8.1.3-7: HSI Evaluation

7. HSI Evaluation Criterion: HSI should be evaluated in an ongoing fashion to assure its acceptability for task performance and conformance to HFE, criteria, stan- i dards, and guidelines. Special attention should be given to those HSIs that are unique or safety related. This should be done to assure that poor design solutions do not remain undetected until Element 10 V&V, at which time design changes become more difficult.

Aspects of the HSI that are at variance with design guidance or for which HFE guidance is lacking should be analyzed. The applicant may use many means to resolve these issues including operating experience and literature analyses, trade-off studies, engineering evaluations and experiments, and benchmark evaluations.

Evaluations should be conducted to ensure that the HSI includes all informa-tion and controls required to perform operator tasks and that extraneous controls and displays not required for the accomplishment of any tasks are excluded.

The outcomes of these evaluations and rationale for resulting design decisions should be documented and available for review.

DSER Evaluatfon: According to the SSAR, evaluation of conformance to stan-dards and guidelines is conducted "throughout the functional requirements phase of the M-MIS design process." The SSAR does not mention evaluations against tailored " guidelines" (see Criterion 6) provided to the designers of each subsystem (see SSAR Section 18.8.2.1.3). RAI 620.59 questioned whether the general design guidel m s cited by Westinghouse, taken together, were sufficiently comprehensive and recommended the use of tailored guidance in the evaluations. Westinghouse's response to RAI 620.59 indicates that tailored guidance wi1l be used in the evaluations. The adequacy of such guidance for these evaluations will depend on the degree to which their development meets i

Criterion 6.

SSAR Section 18.8.2.3.5.4 indicates that the experimental evaluations dis- I cussed above will be performed at two stages - concept testing and acceptance testing. The evaluation of M-MIS concepts against human engineering guide-lines is said to occur at "various stages" in the development process.

Westinghouse's response to RAI 620.20 refers to design reviews for each of the  !

interfaces "at major miiestones in their development." Criteria should be estab11shed for identifying unique or safety-related HSIs and for planning the stages at which M-MIS design elements are evaluated against human engineering guidelines.

l J

-- - - - -- - . - - _ ~ . - . - -

I

~

\

j t ,

i i l Westinghouse's response to RAI 620.59 further states that the design guidance l 1 will specify some design decisions, but will be " written at a fairly high 4 level" to allow a knowledgeable designer to consider trade-offs when neces-sary. No formal process for identifying or documenting the resolution of ,

3 design issues is mentioned. '

1

) Westinghouse's response to RAI 620.81 states that the availability of controls and displays defined by the task analysis is assured by the design review of l the displays and controls, which will " identify if any information determined i necessary by the task analysis has been left out." The response also indi-

! cates that proposed indications and controls that might be recommended by the i

system designers for any giver. location are " filtered through the task j analysis and, if found unnecessary to support specific tasks identified for j that given location, they are deleted."

The approach to defining major issues on which to evaluate the M-MIS design is

described in SSAR Section 18.8.2.3.2.5. The issues are organized according to i

the three major classes of operator activity and are centered on the aspects

of the M-MIS designed to support the activity. Within each activity group, the issues consider either single or multiple features in either straightfor-ward or complex situations. Due to the scarcity of guidance for the design of

advanced control rooms, these evaluations are an important part of the design

, process. This is reflected in the detailed specification of the test plans j (e.g., hypotheses, test bed and subject requirements, manipulations, dependent ,

4 measures) for each issue. To the extent the evaluations are not exhaustive i

! (i.e., every display, procedure, or control is not exercised under all conditions), the rationale for selecting those that are included in the DSER i l Evaluation plans should be discussed, and a plan for taking account of the  !

implications of the evaluations in the overall design should be described. '

i Proposed Resolution: As indicated in the review of criterion 6 above, l Westinghouse made examples of their design guidance available for staff

review. It was concluded that the Westinghouse design process provides for j the development of comprehensive detailed design guidance and provides sufficient information to support its standard and consistent application. As i was identified in SSAR (Revision 0) Section 18.8.2.3.5.4.1, Evaluation i Issue 16, this guidance will be used to evaluate the design against HFE i guidelines at various stages of design development. Thus, the aspect of the criterion addressing use of HFE guidelines in the evaluation was acceptably i addressed.

l A second part of the criterion is the use of analysis for aspects of the HSI

that are at variance with design guidance or for which HFE guidance was

lacking. As was discussed with respect to Criterion 7 above, and based upon i discussions held with Westinghouse, the role of the evaluation issues dis-

! cussed in SSAR (Revision 0) Section 18.8.2.3.5, Evaluation Issues and Descrip-

} tions, was clarified. These evaluations will address aspects of the design

that cannot be resolved using available HFE guidance. These evelvations will 1 also take place at various points in the design process. Thus, the part of

, the criterion addressing use of analyses in the evaluation is acceptably l addressed.

e

3

l i

1 The third part of the criterion is to evaluate the design to ensure that the i i

HSI includes all information and controls required to perform operator tasks 1 and that extraneous controls and displays not required for the accomplishment l of any tasks are excluded. This type of evaluation did not appear to be  !

discussed by Westinghouse as part of the HSI design process. A methodology to  !

perform this analysis was included in WCAP-14401 (Revision 1), Programmatic  !

Level Description of the AP600 Human Factors Verification and Validation Plan.

While this is good practice and acceptably meets the criterion, the staff 1

recommends that such analyses also be conducted at various points in the l design process as are the HFE guidelines evaluations.

The fourth part of the criterion is to document the results of these evalua- l tions. As per the Westinghouse design process described in WCAP 12601, the  ;

results of design evaluations are documented as part of the design files. j Thus, the part of the criterion addressing documentation of analyses was acceptably addressed. l The information from earlier SSAR revisions was acceptably included in SSAR Revision 9. SSAR (Revision 0) Section 18.8.2.3.5.4.1, Evaluation Issue 16, is discussed in SSAR (Revision 9), HSI Design Test Program and a more detailed description is include in WCAP-14401 (Revision 1), Programmatic Level Descrip-tion of the AP600 Human Factors Verification and Validation Plan. This SSAR (Revision 9) section also included the discussion above of SSAR (Revision 0)

Section 18.8.2.3.5, Evaluation Issues and Descriptions.

In conclusion, the Westinghouse design process provides for the accepcable evaluation of HSIs. However, until the pertinent information in WCAP-12601 is placed in the SSAR or a docketed secondary reference, this item can not be resol ved.

STATUS OF OPEN ITEM: Action W (1361) Open Item 18.8.1.3-8: HSI Desian Documentation l

8. HSI Desian Documentation Criterion: The HSI design should be documented to include:

{

• The detailed HSI description including the format and performance  !

characteristics J

• The basis for the HSI design characteristics with respect to l operating experience and literature analyses, trade-off studies, engineering evaluations and experiments, and benchmark evaluations i DSER Evaluation: The results of the design process for the main control area are described in SSAR Section 18.9. General descriptions of major equipment (wall panel information station, operator and supervisor workstations, and safety panel) are provided in SSAR Section 18.9.1. The alarm system and

4

! computer based procedures are described in greater detail in SSAR Sec-tions 18.9.2 and 18.9.8, respectively. Design process results for other areas within the control room and for control centers outside the control room are described in SSAR Sections 18.9 and 18.10, respectively.

As indicated in the DSER Evaluation of Criterion 5 above, the basis for the central elements of the control room design are not specifically described in the SSAR.

1 To address this criterion, Westinghouse must, taking into account the concerns

identified by the staff in their DSER Evaluation of this criterion, describe 4

how the final HSI design will be documented, incorporating the bases given in

! the criterion. The item is open pending staff review of the acceptability of the information provided by Westinghouse. Alternatively, Westinghouse must

provide justification / alternatives for not meeting this criterion.

Proposed Resolutfon: A full documentation of the AP600 M-MIS is not currently available because the design is'not yet completed. SSAR (Revision 9) Sec-4 tion 18.8, Human System Interface Design, and 18.12, Displays, Alarms, and Controls documents the current status of the main control room resources which includes HSI requirements, description, and technical basis.

The complete documentation process for the final design is described and controlled under WCAP 12601, AP600 Program Operating Procedures (Revision 15, dated April 1, 1995), which provides a description of the M-MIS documentation process. Procedure AP-3.1, AP600 System Specification Documents (SSDs),

Revision 1, dated February 28, 1991, establishes requirements for SSDs. SSDs identify specific system design requirements and show how the design satisfies the requirements. They provide a vehicle for documenting the design and its l basis. General Step C states that the SSDs provide for the control room MMI design. Step E and Appendix C provide a list of the AP600 systems for which

. SSDs are required, which includes the Operation and Control Centers (OCS).

Appendix A provides a top level Table of Contents by section for each SSD and i Appendix B provides a summary description of what should go into sections of the SSD.

WCAP-12601, Procedure AP-3.2, Design Configuratioa Change Control, Revision 3, March 11, 1994, provides the required process and actions in order to imple-ment a design change in a document that is under configuration control. The scope of the procedure includes SSDs, drawings, etc. It has considerable e

information on responsibilities, procedures, documentation, and approvals.

, WCAP-12601, Procedure AP-3.6, AP600 Design Criteria Documents, Revision 2, March 11, 1994, specifies requirements for the preparation, review, approval and revision of Design Criteria Documents, which define the requirements for specific aspects of the AP600 design, typically in a single discipline or subdiscipline.

In conclusion, the Westinghouse design process defined in WCAP-12601 and illustrated in the SSAR for the current state of the AP600 M-MIS design i

i

completion will provide an acceptable documentation of the detailed HSI

. design. However, until the pertinent information in WCAP-12601 is placed in the SSAR or a docketed secondary reference, this item can not be resolved.

STATUS OF DPEN ITEM: Action W 4

4 i

i i

1 i

i A

! i a i 5

5 3

i i

i 1

i u

1 1