ML20035F921

Draft Ter,Review of ABB-CE Sys 80+ Human Factors Program Plan

ML20035F921
Person / Time
Site:	05200002
Issue date:	03/22/1993
From:	Higgins J, Ohara J BROOKHAVEN NATIONAL LABORATORY
To:	Polk H Office of Nuclear Reactor Regulation
Shared Package
ML20035F916	List: ML20035F915 ML20035F921 ML20035F926 ML20035F929
References
CON-FIN-E-2090 E2090-T2-1-3-93, NUDOCS 9304230063
Download: ML20035F921 (9)

Text

'

r.

e Advanced Reactor Human Factors (FIN E-2090)

Task Order No. 2: ABB-CE System 80+ Review BNL Technical Report E2090-T2-1-3/93 Drafi TechnicalEvaluationReport i

Review of the ABB-CE System 80+

Human Factors Program Plan ed 9

Prepared for:

U.S. Nuclear Regulatory Commission l

Omce of Nuclear Reactor Regulation Washington, D.C. 20555 i

Prepared by:

John O'Hara & James Higgins Department of Nuclear Energy Brookhav.en National Laboratory Upton,NY 11973 i

March 22,1993 j

l 9304230063 930416 PDR ADOCK 05200002 A

PDR

I l

i PREFACE i

This draft technical repon (DTR) has been prepared by Brookhaven National Laboratory for the i

Human Factors Assessment _ Branch of the U.S. Nuclear Regulatory Commission's (NRC's)

J Office of Nuclear Reactor Regulation. This report is submitted under the AdvancedReactor Human Factors Review Project (FIN E-2090) as pan of Task 2 " Review of the ABB-CE System l

80+ Advanced Reactor Human Factors Program." "Ihe DTR addresses Subtask 2 " Review and i

Evaluate Responses from ABB-CE" by providing a draft TER evaluating ABB-CE's human i

factors program plan. The NRC Project Manager is Harold Polk, the Project Engineer is Clare j

Goodman, and the Technical Monitor for Task 2 is Garmon West. The BNL Principal Invesugator is John O'Hara.

l l

E 1

l 1

t l

l l

Draft TERfor Element 1 Program Plan (March 22.1993)

Pagei l

P

?

1.

INTRODUCTION ne NRC Human Factors Engincedng Program Review Model (PRM) for advanced evolutionary reactors specified that a formal Human Factors Engineering (HFE) Program (Element 1) should be established to guide HFE activities. The staffs Draft Safety Evaluation Report (DSER) review of the CESSAR has identified several open issues related to PRM Element 1 (i.e., DSER Issues 18.3.1-1,18.3.2-1, and 18.3.5-1).

He PRM was developed assuming a HFE program plan would be developed at the beginning of the HFE effort. However, ABB-CE has already completed significant HFE analysis and design activities (before the PRM was developed). It was therefore, considered appropriate to modify some of the details of the PRM Element I criteria to accommodate completed activities so lonps the substantive contributions of HFE activities to plant safety am not compromised (e.g., that the intent of the HFE program elements are accomplished even though some differences between specific PRM criteda and ABB-CE activities may exist).

2.

OBJECTIVES 8

he objective of this preliminary review is to provide an evaluation of the ABB-CE efforts related to PRM Element 1 - Human Factors Engineering Program Management.

3. METHODOLOGY i

3.1 Material Reviewed De following ABB-CE documents were used in this myiew:

j 1.

CESSAR Section.c 18.2 " Design Team Organization and Responsibilities" and 18.4.2

" Human Factors Program Plan", Draft 3/2/93.

2.

Human Factors Program Plan for the System 80+ Standard Plant Design (NPX80-lC-DP790-01, Rev 01), 12/15/92, hereafter referred to as the " Plan."

I 3.2 Review Scope The review focused on (1) resolution of DSER issues, and (2) evaluation of the ABB-CE -

documents with respect to the topics and general criteria of the PRM. Table 1 provides a

" compliance matrix" providing a cross-reference between review items and the pertinent sections of the Plan. As indicated in the introduction, absolute adherence to the PRM was not considered to be i

mandatory. Differences in approach would be considered acceptable provided (1) the program can still meet the HFE commitment and goals, (2) the difference between the proposed cdteria and those contained in the PRM are adequatelyjustified, and (3) there is no adverse impact on other program elements.

Due to the fact that the CE System 80+ plant and control room design is quite far along, there is a considerable amount of actual design matedal available. CE has included such design infomtation -

along with the HFE Program description. Hence, the scope of the documents reviewed go beyond the " submittal requirements" of the PRM. Since the review was model driven, those portions of the documents that were within scope of the PRM were reviewed. Therefore, portions of the Plan which address design feature jusdfication (see Plan Section 3, for example) were not reviewed.

Draft 7ERfor Element 1 Program Plan (March 22,1993)

Pagei

9 Table 1 Comprison of PRM and the Plan NRC REVIEW MODEL TER REVIEW ABB-CE PLAN COMPONENT SECTION SECTION Pu rpose, Scope, & Organir.atbn 4.2.1 1.1/ Appendix A Goals and Objectives 4.2.2 1.2/3/ Appendix A Management & Organization 4.23 Design Team & Organization 4.23.1 13.1 l

Integration into Process 4.23.2 13 Program Milestones 4.233 13.1.2/7 Documentation 4,23.4 1.3.2/7 Subcontractor Efforts 4.23.5 1.3.1.4 Literature / Practices Review 4.23.6 Appendix A Issue Tracking System 4.23.7 Appendix A Tedmical Program 4.2.4 2-6/ Appendix A 3.3 Review Procedure The review began following the identi5 cation of DSER issues. A draft of the Plan was developed to address DSER issues related to PRM Element I and was submitted to the review team in October 1992. A draft evaluation of the plan was prepared in November 1992 providing preliminary questions and raising points of clari5 cation. A meeting was held with CE in November 1992 to discuss the review comments which was followed up by a telephone coaference in early December to clarify the reviewers comments. The Plan was revised following these discussions and the review of the revised plan is the subject of this TER.

The following materials were consultbd a's part of the evaluation:

1.

NUREG-1492 Draft Safety Evaluation Report, September,1992. (DSER) 2.

Public Meeting minutes from September 10-11,1992, hereafter referred to as,the

" September meeting."

3.

NRC HFE Program Review Model for Evolutionary Reactors (PRM).

Draft TERfor Element 1 Program Plan (March 22.1993)

Page2

4 4.

RESULTS 4.1 DSER Review i

4.1.1 DSER Issue In the staff's initial review of this element reponed in the DSER, the following Open Issues wem identified:

• 18.3.1-1 Human Factors Engineering Program

• 18.3.2-1 HFE Program Milestones and Task Schedules

• 183.5-1 Design Goals In the September meedng, CE agreed to address the open issues by developing a HFE Program Plan which included:

(a)

Addressing the human-centered design goals of criterion 1 of the NRC's PRM and specifying how the goals will be evaluated throughout the design process, including the V&V effort.

(b)

Providing a schedule for tests and evaluadon in the plan that (a) shows the relationship between the HF activities and the overall nlant design process; and (2) identifies the HFE products associated with the milestones.

l 4.1.2 Issue Resolution Item at Addressing the human-centered design goals of criterion 1 of the NRC's PRM and specifying how the goals will be evaluated throughout the design process, including the V&V i

effort.

Evaluation: This item is discussed in Secdon 4.2.2 - Overall HFE Program Goals and Objectives l

below.

l Status: Acceptable (msolved) s i

~

Item b; Providing a schedule for tests and evaluation in the plan that (a) shows the relationship l

between the HF activities and the overall plant design process; and (2) identifies the HFE products associated with the milestones.

Evaluation-This item is discussed in S5ction 4.2.3.3 - HFE Program Milestones below.

Status: Acceptable (resolved)

{

1 4.2 PRM Criteria-Based Evaluation 4.2.1 General Purpose, Scope, and Organization Criterion: The PRM specifies that the plan should address the overall purpose and organization of the plan.

Drap TERfor Elemera 1 Program Phn (March 22,1993)

Page3

)

i

s Evaluation:

1. Purnose - The pumose of the plan generally encompasms the topics identified in the PRM and goes beyond the PRM by providing overviews of analyses conducted to date and design justification and bases for design features proposed. This expanded purpose is acceptable given (1) the evolutionary approach to HFE being employed for the System 80+ HFE program, and (2) the considerable work completed to date. As noted above, however, the details encompassed by the expanded purpose were not reviewed in this TER (but will be reviewed in conjunction with the appropriate PRM element reviews).

2. Scope - Here does not appear to be a one-to-one relationship between the main body of the plan description and the program element descriptions in Appendix A. (He main body of the plan contains a description of the HFE activities completed to date and plans for HFE activities that are and will be performed as the design proceeds while Appendix A provides more detailed goals, requirements, and criteria for these HFE activities.) Here am scope differences between the Plan (pp. 5-6) and the design process equirements in Appendix A (pp. A-10-11). The stated scope of the Plan is as follows:

a main control room - ful! plan

• TSC - full plan

• remote shutdown panel-full plan

• local control panels "related HSI considerations" In Appendix A:

• main control room - full plan

• remote shutdown panel-full plan

• local control panels - apply *HSI guidance"

• Procedure Guideline " critical tasks" - Elements 5-8 The Appendix does not reference the TSC and the status oflocal panels and the TSC with respect to HFE involvement needs to be clarified. Also the OSC and EOF are not addressed and should receive the same treatment as the TSC.

A formal Procedure Element is explicitly excluded from the plan. CE states that detailed procedure development is a licensee activity. CE will prepare procedure guidelines as technical input to the

., licensee. Since this guidance will focus on content and not format,it is not covered in the HFE program. Thus there is no Element in the CE requirements document that corresponds to PRM Element ~1.

Element 7 of the PRM was included with the following objective: " Plant and Emergency Operating Procedures shall be developed to support and guide human interaction with plant systems and to control plant-related events and activities. Human engineering principles and criteria shall be applied along with all other design requirements to develop procedures that are technically accurate, comprehensive, explicit, easy to utilize, and validated." While detailed procedure development was considered a licensee activity already addressed by NRC license reviews, this element was retained to address three issues: First, to en'sure tight coupling of the task analysis and detailed HSI design activities with procedure requirements and development. Second, to ensure the proper human engineering of procedures themselves which should be the responsibility of the human factors activity. Hird, operating and emergency procedures are necessary in order to evaluate the HSI. It appears that CE may have addressed the first i' tem in the Plan and deferred the second to the COL applicant. However, the third item is not addressed and the reviewers are unsure how the EPGs will be utilized to perform validation testing (since they are not referenced to the HSI).

Drap TERfor Element i Program Plan (March 22.1993)

Page 4

~

Rationale for the exclusion of a procedure development element from the CE requimments needs i

funher discussion and clarification to ensum that all imponant aspects of the procedum element will be addmssed at appropriate times.

I

3. Organi7ation -The Plan provides a description of the HFE design team and the management structure for the HFE effort. The plan also provides a clear description of the overall technical program and its relationship to completed activities and planned analyses. No issues conceming Plan organization wemidentified.

l Status: Open. The purpose and organization of the plan are acceptable. Several issues remain to j

be clarified conceming scope, most notably the issue of procedure development.

4.2.2 Overall HFE Program Goals and Objectives Criterion: The primary goal of the HFE program shall be to develop an HSI which makes possible safe, efficient, and reliable operator performance and which satisfies all regulatory requirements as stated in 10 CFR. 'Ihe general objectives of this program shall be stated in " human-centemd" terms which, as the HFE program develops, shall be objectively definea and shall serve as criteria for test and evaluation activities. Generic " human-centered" HFE design goals are listed in General Criterion 1 of the PRM.

Evaluation: Funher discussion of the general goals was a provision of the September 10-11 meeting on DSER Issue resolution (see item "a" under Section 4.1.2 DSER Issue Resolution, above).

1. The design philosophy and subsidiary " philosophies" appear to mflect a reasonable and acceptable set of high-level design goals which are generally consistent with the PRM.

2. The reviewers do not agree with the discussion of the Plan's human-centered goals. The purpose ofidentifying a set of generic human centered goals for a HFE program was to assure that the focus of the HFE program remained on the crew and that imponant factors affecting crew performance are addressed. While the PRM expressed these goals in relatively general terms,it is expected that designers will translate them into explicit design goals, design solutions to achieve the goals, and " measurable" evaluation criteria to assess their achievement. For example, one goal is to " minimize operator memory load." If such as design goal is explicitly stated, designers can look for opportunities to avoid memory burden. Note the comment made by System 80 licerised t

operators:

Operators have to remember (memorize) their inidal post trip actions and are expected to accomplish them prior to procedural checks. Aids should be available to the operators for this purpose. (BNL Technical Report E2090-T2-4-3/93)

A design process focusing on memo [y lo'ad reduction will address such concems.

Like all hypothetical constmets or "high-inference" vadables, multiple operational definitions of these concepts are possible. Unlike the description in the Plan, concepts such as workload and situation awareness go well beyond " aviation" and have a useful place in NPP V&V. Just because there may be several different ways to assess workload (as with any hypothetical construct), for example, does not mean the construct is not a useful one to evaluate. As an example, performing validation testing of an advanced control room and measuring tim'e and errors alone without asking operators about HSI imposed workload or their ability to maintain situation awareness, would deny the designers imponant information about the design and may fail to identify potential _

cogmuve errors (e.g., performing the correct action for the wrong reason). Funher, in highly automated systems where monitoring and situation assessment is a primary task, assessment of the Draft TERfor Element 1 Program Plan iMarch 22,199.0 Page5

cognitive aspects of tasks is more critical. Finally, skilled operators can perform well in simulated situations despite high workload and poor information presentation. Thus, task time and errors may not be sensitive measures of operator performance or HSI acceptability.

l The discussion ofindividual human-centered goals reflects a somewhat casual treatment of these r

concepts. For example, for situatina awareness the Plan states that " validation exercises will i

ensure the acceptability of crew sauma awan ness in tenns of system-defined time and performance criteria" (p. I1). System. fined time and performance criteria are not adequate -

f measures of situatien awareness. They are valuable and necessary criteria in their own right, however, they do not necessatily imply good situation awareness - hence the distinct concept.

Successful perfonnance can be achieved even when poor situation awareness exists, especially in highly automated systems or in facilities with highly redundant safety systems. He Plan also includes the use of subjective measures as part of SA evaluations.

i ne reviewers think that these concepts should not be summarily dismissed because their evaluation is not as straight forward as time measurement. CE has generally dismissed the human-centered goals as "high inference" variables prone to unreliable results" and whose

• correspondence to objective evaluation criteria is limited both on theoretical and pragmade grounds " The plan then goes on to address them anyway. The reviewers are thus concemed, based upon the discussion above, that the designers have no real commitment to the human-centered goals.

1 Status: Acceptable - While the reviewers find the Plan's treatment of the human-centered goals is weak,it does establish the commitment to evaluate human-centered goals and it is not a requirement of the PRM to define how these factors will be assessed (that is part of later elements).

P 4.2.3 Program Management and Organization 4.2.3.1 HSI Design Team ReviewedMaterial: Section 1.3 andAppendixA.

The PRM requires an HSI Design Team shall have the responsibility, authority, placement within the organization, and composition to ensure that the design commitment to HFE is achieved.

The review centered upon the plans discussion of the following topics as defined by the PRM: (1)

Responsibility (2) Scope,(3) Organizational Placement and Authority, and (4) Composition.

1

1. Responsibility i

Criterion: The PRM specified that the team shall be responsible for (1) the development of all HFE plans and procedures; (2) the oversight and review of all HFE design, development, test, and evaluation activities; (3) the initiation, recommendation, and provision of solutions through designated channels for problems identified in the implementation of the HFE activities:(4) verification ofimplementation of team recommendations,(5) assurance that all HFE actisities comply to the HFE plans and procedures, and (7) scheduling of activities and milestones.

l Evaluation: The HSI design team review was closed based on NRC review (see DSER Section i

18.3.3).

l Status: Closed by NRC

2. Stone of Team's Resnonsibility Criterion: The PRM specified that the scope of the Team's responsibility shall include: (1) control i

j and instrumentadon equipment; (2) operations, maintenance, test, and inspection ofir'terfaces and Draft TERfor Element i Program Plan (March 22,1993)

Page 6

.]

=w w

w r

u -

i facilities both within and outside the control room; (3) procedures; and (4) training mquirements developmt Evaluation: The HSI design team review was closed based on NRC review (see DSER Section l 8.3.3).

Status: Closed by NRC

3. Orennizational Placement anr' Authority Criterion: The PRM specified that the Team shall have the authority and organizational placement and freedom to ensure that allits amas of responsibility are accomplished and to identify problems in the implementation of the HSI design. The team shall have the authority to determine where its input is required, access work areas, design documentation. The Team shall have the authority to control further processing, delivery, installation or use of HFE/HSI products until the disposition of a non-conformance, deficiency or unsatisfactory condition has been achieved.

Evaluation: 'Ihe HSI design team review was closed based on NRC review (see DSER Section I8.3.3).

Status: Closed by hTC

4. Composition Criterion: The PRM specified that the HSI Design Team be composed of specific expenise (see i

PRM).

(

Evaluation: The HSI design team review was closed based on NRC review (see DSER Section 18.3.3).

Status: Closed by NRC 4.2.3.2 Integration of HFE and Other Plant Design Activities Criterion: According to the PRM, the Plan should identify integrated design activities.

Evaluation: Based upon the Plan description, HFE is well integrated into the design process.

There are three mechanisms for this integration. First, the design process shows HFE analyses and evaluations throughout the HS1 design cycle. Figures 1.3-1 through 1.3-6 of the Plan j

illustrate the relationship between HFE structemd analyses and the system design. Second, there.

has becn active involvement of HFE specialists in the multi-disciplinary design team. Third, HFE activities are part of design review meetings.

Status: Acceptable 4.2.3.3 HFE Program Milestones Criterion: According to the PRM, the Plan should identify HFE milestones and provide a program schedule.

Evaluation: Further discussion of the milestones and schedule was also a provision of the September 10-11 meeting on DSER Issue resolution (see item "b" under Section 4.1.2 DSER i

Issue Resolution, above).

Generic HFE activities are described in Figure (s) 1.3-2 through -6 of the Plan. The figures depict the flow of HFE efforts in terms of parallel and serial activities and interdependencies. Milestones Draft TERfor Elemer.t 1 Program Plan (March 22,1993)

Page 7

and general documentation outputs are illustrated on the Figures. These figures provide a clear overview of the overall program and its products.

Status: Acceptable 4.2.3.4 IIFE Documentation Criterion-According to the PRM, the following items were expected for each element:

• Implementation Plan

= Analysis Report

• Design Team Review Repon Evaluation:

Section 1.3.2 provides a list of HFE products. Several PRM-specified items are not on the list:

1. Implementation plans for Elements 1-4: - This is acceptable because CE's effons in these areas is well underway (or in some cases essentially complete). Further, CE has agreed to include a description of methodology for those elements in their submittals for staff review.

2. Procedure development repons (this will not be considered as pan of this item since it has i

already been addressed in the discussion of scope).

3. All repons of the design review evaluations. The reviewers are unsure of the function of the Design Review Meetings (discussed in Section 1.3.1.3) with respect to the PRM. Clearly, DRMs are an important aspect to any design effon and they are commonly applied by most, if not all, vendors of complex systems. In CE's Plan, the DRM also appears to be intended to serve as the approach to satisfying design team reviews of technical HFE program products (as identified in the i

PRM). If so, the reviewers do not think that the approach meets the PRM requirement of fortr e HFE product reviews and cenainly not the review repons. At the November 19 meeting between CE and the staff, CE indicated that a document review process is utilized but is not formally documented. It was expected that the description of the review process would be expanded in the current version of the plan to address how documents are reviewed, but no such discussion was found. Therefore, additionalinformation about the HFE products review process and the documentation of such reviews should be provided to assure the interdisciplinary review of all HFE efforts.

Status: Open, pending receipt of a description of how documents are reviewed by the HFE design team how comments are addressed, and how this is reported.

4.2.3.5 HFE in Subcontractor; Efforts Criterion: The PRM specifies that HFE in subcontractor efforts should be discussed with respect to the following:

Protide a copy of the HFE requirements proposed for inclusion in each subcu a t.

Describe the manner in which the designer proposes to monitor the subcontractor's

=

compliance with HFE requirements.

i Evaluation: The subcontractor requirements issue was closed based on NRC review (see DSER i

Section 18.3.3).

Drap TERfor Element i Pwgrarn Plan (March 22,1993)

Page8 l

1 l

Status: Closed by NRC 4.2.3.6 Literature and Current Practices Review Criterion: PRM General Criterien 2 identines acceptable references upon which an HFE program can be developed.

Evaluation Plan Appendix A indicates that the plan and m1ated criteria were based upon a review of 15 source documents. These include many of the same sources as used as technical bases of the i

PRM.

Status: Acceptable i

4.2.3.7 HFE Issue Tracking System Criterion: He PRM identifies four criteria for the issues tracking system (see PRM).

Evaluation: he HSI design team review was closed based on NRC review (see DSER Seedon 18.3.4).

Status: Closed by NRC r

I 4.2.4 Technical Program Criterion: Identify and desedbe the development of implementation plans, analyses, and evalua-tion / verification of:

Operating Experience Review System Functional Requirements Development Allocation of Function Task Analysis Interface Design Plant and Emergency Operating Procedure Development HF Vedfication and Validation l

Evaluation:

CE's HFE program is organized into eight elements:

~

Element 1 - Human F5ctors Engineering Program Management Element 2 - Incorporation of Industry Experience Element 3 - Evaluation and Allocation of System Functions Element 4 -Task Analysis Element 5 - Man-Machine Interface Design Element 6 - Availability Verification Element 7 - Suitability Vedfication i

Element 8 - Validation of Ensemble.

2 he general relationship between the NRC PRM and CE model elements is presented below:

Draft TERfor Element 1 Program Plan (March 22,1993)

Page 9 l

NRC PRM CE PL A N NOTES 1

1 2

2 3

3 G Bement combines function requirements and allocation 4

3 G Bement combines function requirements and allocation 5

4 i

6 5

1 7

G model defines pma:dures as out of scope 8

6 G model breaks PRM V&V into three separate elements (6. 7, & 8) 7 8

With the exception of a procedures element, the CE HFE technical plan acceptably contains all main components of the PRM (although the organization is slightly different).

While an general evaluation of CE's Plan with respect to PRM elements is provided below, the i

staff has agmed to resolve detailed diffemaces between element descriptions during the review of the respective element products. Thus it is not expected that the Plan will be revised to resolve the issues identified below. Since the first four PRM element reviews are ongoing, their details are not t

addressed here. Dus the comments below address Element 5 through Element 8 Validation. '

PRM Element 5 - Task Analysis:

1. The overall purpose / goal of the task analysis is significantly restricted in the CE requirements compared with the NRC criteria. The commitment in the PRM is to identify the behavioral requirements of tasks that plant personnel are to perform. This includes " input and output requirements" as required in the CE document, but also includes decision making requirements, I

detailed response requirements, task support requirements, communications, etc. It is unclear whether the output of the analysis required by CE would provide for the full compliment of behavioral performance requirements addressed in the PRM.

2. De scope of the CE task analysis does not include as broad a range of operational conditions as i

is defined in the PRM. For example, Low power and shutdown operations are not identified in the CE model.

~. 3' PRM Element 5 - General Criterion 2 requires detailed analysis ofimportant human actions defined in the PRA. CE Requirement A-3.4.2.7 specifically excludes human error from analysis with the justification that it is the responsibility of the PRA program and, therefom, not governed by the HFE plan. The reviewers find this rationale weak and unacceptable. The analysis of tasks identified as imponant to risk should be conducted by the HSI designers to assure that these tasks are properly supported by the HSI design and that they can reliably be performed by the operating i

crew.

l

5. PRM Element 5 - General Criterion 4 requires that the analysis be performed iteratively and that the output should " enable specification of detailed requirements for alarms, displays, data processing, and controls for human task accomplishment." He CE output is identified as limited i

to " inputs and outputs" which are not further specified; thus it is difficult to ascertain what the specific products of the analysis will be.

i

6. PRM Element 5 - General Criterion 5 requires a commitment to provide task analysis fmdings to i

training program development. No such requirement exists in the CE Plan.

1 Drop TERfor Element 1 Program Plan (March 22.1993)

Page10 l

i PRM Element 6 - Human-System interface Desien

1. While the goal of CE Element 5 seems clear, the requirements am a bit unclear. Requirement A-3.5.2.1.1 states that HFE Principles "shall be assembled by the design team to be applied to the HS1 design as HFE design Guidance." Funher, Requirement A-3.5.2.1.5 states that the guidance "shall be promulgated by the Responsible Management Structure to the design team for implementation in the design." This mquirement may not be sufficient. It is well known that l

i generic HFE guidance is of little use to designers. To be useful, HFE specialists must tailor generic guidance to the specific system being designed. The tailored guidelines become a " design specification" to guide the H3! development, assum that HFE pdnciples are employed, and to control the design for consistency, etc. (e.g., configuradon management). Rather than " assemble" i

guidance, Element 6 of the PRM mquires an HFE design team to develop a design specific guideline. For example, rather than providing guidance indicating that "The window must be visually distinct for the rest of the screen by a distinct border", the window framing design should l

be specified.

General Criterion 7 states that " Human engineering guidance regarding the design paniculars shall be developed by the HSI designer to (1) insure that the human-system interfaces are designed to currently accepted HFE guidelines and (2) insure proper consideration of human l

capabilities and limitations in the developing system. This guidance shall be derived from sources such as expen judgement, design guidelines and standards, and quantitative (e.g., anthropometric) and qualitative (e.g., relative effectiveness of differing types of displays for different conditions) data. Procedures shall be employed to ensure HS1 adherence with standards." As stated, the scope of the HS1 design element proposed by CE will not adequately meet this requirement.

2. CE Element 5 Requirement A-3.5.2.1.3 provides the acceptable basis for HFE Design Guidance. The results of design trade-offs, analyses, tests, and evaluations may also provide acceptable bases for guidance.

o

3. CE Element 5 Requirement A-3.5.2.1.4 provides a list of required topics. While these topics l

should be addressed, it is not clear that the listing includes other imponant topics such as operator l

aids, graphic displays, and large panel displays.

4. CE Element 5 Requirement A-3.5.2.2.2 should state words to the effect of: the HFE "shall ensure" the ability of the design to sansfy the requirements of 10 CFR for minimum staffing rather i

than " shall not preclude."

PRM Element 7 - Plant and Emercenev Oneratine Procedure Develonment Not included in CE's HFE program (see discussion in Section 4.2.1 of this report).

PRM Element 8 - Human Factors Verification and Validation i

Taken together, the three CE elements (6,7, & 8) do not fully cover the scope of the V&V effon mquired by the PRM. Elements 6 and 7 r,oughly correspond to General Requirement 2 (except that

'3 procedures are excluded from the CE model) and General Requirement 3. CE Element 8 corresponds to the PRM Element 8-General Criteria 4 and 5. In addition:

1. Are there missing subsections to Requimment A-3.6.2.3? (On page A-31 reference is made to a l

section A-3.6.2.3.2).

2. NRC General Requirement 2 includes procedure evaluation in ihe " suitability evaluation" but it is not identified in the CE model.

3. Suitability Verification Requimment A-3.7.2.2 is not clear. It states that "Because of the necessarily generic and context-free nature of HFE Design Guidance, and the context-dependent Draft TERfor Element i Program Plan (March 22,1993)

Page11

l i

J nature of design trade-offs, conformance to HFE Design Guidance is not itself a requirement." Yet the next sentence states that "However, HFE Design Guidance shall provide the primary reference against which Suitability is evaluated" (p. A-41). If "confonnance to HFE Design Guidance is not itself a requirement", why include such a review as part of the verification process. Requirement A-3.7.2.2 raises severalissues:

a. It may be more appropriate to indicate that conformance to any specific individual guideline is not a requirement because guidelines are insensitive to the trade-offs between design features and functions that typically occur in final designs. These trade-offs may result in discrepancies between an acceptable final design and a specific guideline. Further,it would be correct to indicate that strict adherence to generic guidelines is not a requirement. Instead a verification against generic guidelines is meant to identify potential concerns which should be addressed by the reviewer, but which may be perfectly acceptable, given design studies, tests, and trade-off analyses as justified by the designer.

b. According to the Suitability Verification Requirement A-3.7.2.1, the formal analysis i

requirement, the basis for the evaluation is the HFE Design Guidance developed in A-3.5.

Would not this guidance be design specific rather than generic? The reviewers would have expected the designers preparing the HFE Design Guidance to have reviewed the HFE literature as required by that element and developed a guidance document to serve as a detailed l

HSI design style guide or specification tailored to the design under review. At that point the guidance is no longer generic, trade-offs are already identified, and violations could be potentially more serious than would be the case with generic guidelines.

4. Related to the procedure questions above, the goal of Validation is identified as "to ensure that the sum of the various HSI features afforded by the main control room, the remote shutdown area, and the local control stations required for executing emergency procedures, provide usable operating ensembles that support the successful accomplishment of the operator's functional role (i.e., as specified by training and procedures) under dynamic, real-time conditions." The reviewers agree with this goal, but have reservations about its attainment without procedures.

5. De PRM Element 8 - General Criterion 6 indicated several classes of scenarios to be evaluated, including: normal plant evolutions, instrument failures, HSI equipment and processing failure, transients, and accidents. CE validation requirement may not address this range of scenarios. CE Requirement A-3.8.2.2 identifies events only as SRDBEs.

6. While Validation of Ensemble Requirement A-2.8.2.2 identifies " applicable operating, tech spec, and safety function limits" as the acceptance criteria, no specific requirements for performance measurement is provided, nor are operator-centered acceptance criteria prosided.

The reviewers consider it important to collect operator opinions and perfonnance measures in order to provide an assessment of the usability of the HSI and its capability to meet high-level HFE and system goals. Task loading, for example, is required in CE's Task Analysis Element, where methods to make such evaluation are cmde at best. The Validation element should verify that task analysis estimations of workload are reasonably accurate.

7. PRM Element 8 - General Criterion 8 requires "A verification shall be made that all critical human actions as defined by the task analysis and PRA/HRA have be adequately supported in the design. The design of tests and evaluations to be performed as part of HFE V&V activities shall specifically examine these actions." No such verification is required in the CE plan. Further definition of SRDBEs would partially address this concern.

Status: Open, pending resolution of Procedure element exclusion. The specific details of element discrepancies will be addressed when the details of the elements are reviewed.

Draft TERfor Element 1 Program Plan (March 22.1993)

Page12

5.

CONCLUSIONS The following DSER Issues are resolved:

• 18.3.1-1 Human Factors Engineering Program

+ 18.3.2-1 HFE Program Milestones and Task Schedules

= 18.3.5-1 Design Goals

'Ihe results of the PRM Element 1 evaluation are summarized in Table 2.

Table 2 Summary of Element 1 Evaluation Status f

PRM TER REVIEW STATUS COMPONENT SECTION Purpose, Scope, & Organization 4.2.1 Open Goals and Objectives 4.2.2 Acceptable Management & Organization 4.23 Design Team & Organization 4.23.1 Closed by NRC Integration into Pmcess 4.23.2 Acceptable Program Milestones 4.233 Acceptable Documentation 4.23.4 Open Subcontractor Effons 4.23.5 Closed by NRC Literature / Practices Review 4.23.6 Acceptable Issue Trrking System 4.23.7 Closed by NRC Tedmical Program 4.2.4 Open 5

Draft TERfor Element i Program Plan (March 22,1993)

Page13 i

-