ML18351A453

US-APWR, Chapter 18, Human Factors Safety Evaluation

ML18351A453
Person / Time
Site:	05200021
Issue date:	07/29/2019
From:	George Wunder NRC/NRO/DLSE
To:
Wunder G
References
Download: ML18351A453 (126)

Text

18 HUMAN FACTORS ENGINEERING 18.0 Review Considerations This chapter of the safety evaluation report (SER) provides the U.S. Nuclear Regulatory Commission (NRC) staffs (the staff) review of the human factors engineering (HFE) portion of the Mitsubishi Heavy Industries, Ltd. (MHI) (the applicant) United States - Advanced Pressurized Water Reactor (US-APWR) Design Certification Document (DCD).

18.0.1 Purpose of Review The overall purpose of the HFE review is to verify:

• The applicant has integrated HFE into plant development, design, and evaluation.

• The applicant has provided HFE products (e.g., human system interfaces (HSIs), procedures, and training) that allow safe, efficient, and reliable performance of operation, maintenance, test, inspection, and surveillance tasks.

• The HFE program and its products reflect state-of-the-art human factors principles and satisfy all specific regulatory requirements.

18.0.2 Areas of Review Chapter 18, Human Factors Engineering, of NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition (the SRP) and DCD Chapter 18 identify 12 areas of HFE review that are needed for successful integration of human characteristics and capabilities into nuclear power plant design. Corresponding to the 12 elements of an HFE program identified in NUREG-0711, Human Factors Engineering Program Review Model, Revision 21, these areas of review are:

• HFE Program Management

• Operating Experience Review (OER)

• Functional Requirements Analysis (FRA) and Function Allocation (FA)

• Task Analysis

• Staffing and Qualifications

• Human Reliability Analysis (HRA)

• Procedure Development

• Training Program Development

• HSI Design

• Human Factors Verification and Validation (V&V)

• Design Implementation (DI)

• Human Performance Monitoring (HPM) 1 In accordance with Title 10 Code of Federal Regulations 52.47(a)(9), the applicant is using NUREG-0711, Revision 2 vice Revision 3.

18-1

HFE activities related to procedure design and training design are addressed by programs discussed in Chapter 13, Conduct of Operations. Combined License (COL) applicants address the HFE requirements associated with HPM.

18.0.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

The following NRC requirements and guidance apply to all areas of review that are referred to in Section 18.0.2, Areas of Review, of this report:

• Title 10 of the Code of Federal Regulations (10 CFR) Part 52.47, Contents of applications; technical information, requires that applications for design certification of new reactor designs meet the technically relevant portions of the Three Mile island (TMI) requirements contained in 10 CFR 50.34(f), Additional TMI-Related Requirements, (except for 10 CFR 50.34(f)(1)(xii), (f)(2)(ix), and (f)(3)(v)). The staff bases its HFE review on current regulatory requirements established post-TMI in 10 CFR 50.34(f). The staff reviews HFE aspects of new control rooms to verify that they reflect state-of-the-art human factors principles as required by 10 CFR 50.34(f)(2)(iii), and that personnel performance is appropriately supported. 10 CFR 50.34, Contents of applications, technical information, also requires a safety parameter display system (SPDS), automatic indication of bypassed and operable status of safety systems, and monitoring capability in the control room for a variety of system parameters.

• For plants licensed under 10 CFR Part 52, the requirements of 10 CFR 50.34(f) are incorporated via 10 CFR 52.47 and 10 CFR 52.79, Contents of applications; technical information in final safety analysis report. Meeting these requirements provides evidence that plant design, staffing, and operating responsibilities are acceptable and that there is reasonable assurance that plant safety will not be compromised by human error or by deficiencies in HSIs, considering both hardware and software.

• NUREG 0711 contains review objectives and acceptance criteria for all review areas and provides the staff guidance to conduct and document the HFE evaluations that follow in Sections 18.1 through 18.12 of this report. For a limited number of specific topics, the staff used criteria from other review guidance documents. These other criteria are identified in the specific sections where they apply.

18.0.4 Technical Evaluation - Levels of Review The staff may perform three different levels of review depending on the type of information provided: Complete element level, implementation plan (IP) level, and programmatic level. For the US-APWR, the applicant provided information commensurate with the IP or complete element level reviews. The programmatic level review was not used.

18-2

A complete element level of review is performed when the applicant has completed the HFE activity by addressing all criteria associated with the activity and submitted a description of it, with products, for staff review. If the staff determines that the applicants description and product(s) have met all of the NUREG-0711 criteria, then the activity is acceptable.

An IP level of review is performed when the applicant has not completed an HFE activity (i.e.,

provided a description and product). NUREG-0711, Page 2, states:

An implementation plan gives the applicants proposed methodology for meeting the acceptance criteria of the element. An implementation plan review gives the applicant the opportunity to obtain staff review of and concurrence in the applicants approach before conducting the activities associated with the element.

Such a review is desirable from the staff's perspective because it provides the opportunity to resolve methodological issues and provide input early in the analysis or design process when staff concerns can more easily be addressed than when the effort is completed.

Table 18.0-1, Levels of Staff HFE Review, summarizes the level of review that the staff performed for each of the 12 HFE areas of review related to the US-APWR design certification (DC).

Table 18.0-1 Levels of Staff HFE Review HFE Area Level of Review HFE Program Management Complete Element Operating Experience Review Implementation Plan Functional Requirements Analysis and Function Allocation Implementation Plan Task Analysis Implementation Plan Staffing and Qualifications Implementation Plan Human Reliability Analysis Complete Element Human-System Interface Design Implementation Plan Procedure Development See Chapter 13 Training Program Development See Chapter 13 Human Factors Verification and Validation Implementation Plan Design Implementation Implementation Plan Human Performance Monitoring COL action Item 18.0.5 Inspection, Tests, Analyses, and Acceptance Criteria (ITAAC)

In conformance with the ITAAC identified as part of the generic ITAAC initiative, the applicant identified an ITAAC for the integrated system validation (ISV) and another to verify the as-built control room HFE design configuration conforms to the validated design. These ITAAC are contained in DCD Tier 1, Section 2.9, Human Factors Engineering. The staffs evaluation of these ITAAC is ongoing and will be discussed in Section 14.3.9, ITAAC for Human Factors Engineering, of this report. Because the acceptance criteria for the ISV ITAAC are contained in the V&V IP, specific sections of that IP are designated as Tier 2*. This designation prohibits changes to these sections without prior NRC approval. The expiration date of the Tier 2*

18-3

designation is after completion of the IP and related ITAAC, typically after a Commission finding on ITAAC in accordance with 10 CFR 52.103(g), Operation under a combined license.

18.0.6 Use of Design Acceptance Criteria for HFE The NRC accepts the use of design acceptance criteria (DAC), as described in SECY-92-053, Use of Design Acceptance Criteria during 10 CFR Part 52 Design Certification Reviews, issued February 19, 2002. DAC are a special ITAAC, and they are used in lieu of detailed design information in the HFE area. The NRC allows the use of the DAC process because providing detailed design information is not practicable for applicants using technologies that change so rapidly that the design may have become obsolete between the time the NRC certifies the design and the time a plant is eventually built. For this section and the remaining sections of this report, the use of the acronym ITAAC refers to all ITAAC, including DAC.

18.0.7 Introduction to US-APWR HFE Design Certification Method The applicant used a multi-phase process, described in multiple documents, to design the HFE aspects of the US-APWR control room. Two of the principal documents are, Topical Report (TR) MUAP-07007, Human System Interface System Description, and TR MUAP-09019, US-APWR Human Factors Engineering Program Management Plan. The applicant used three phases to develop the US-APWR HFE design, where the starting point is an HSI design derived from Japanese control rooms. The first phase has two parts, Phase 1a and Phase 1b.

Phase 1a converted the Japanese language, engineering units, and cultural differences to an Americanized HSI starting point, called the US-Basic HSI System. Phase 1a also implemented improvements identified from operating experience with U.S. nuclear plants and additional, generic digital HSI technology experience. Phase 1b resolved deficiencies identified during Phase 1a, and validated design changes that resulted from Phase 1a.

Phase 2 consists of the design and V&V of the US-APWR HFE design. Phase 2 also has two parts (Phase 2a and Phase 2b). Phase 2a uses the NUREG-0711 HFE design process to generate the US-APWR-specific HSI inventory of alarms, displays, procedures, and controls which are added to the US-Basic HSI System design to become the complete US-APWR HFE design. In Phase 2b, the US-APWR HFE design is verified and validated to ensure design specifications were met and that the integrated system provides for an effective operator interface. During the third and last phase (Phase 3), the design and V&V of the HSI inventory for a US-APWR site-specific application will be done, along with training the operators for a site using the US-APWR.

The US-APWR DCD references TR MUAP-07007, Human System Interface System Description, which describes the generic US-Basic HSI System design. This generic design is the basis for the US-APWR HFE design and was approved by the staff for application to the US-APWR HFE design in an SER (ADAMS Accession No. ML15202A337)). In general, TR MUAP-07007 describes the HFE design of the HSIs that are used as the basis for more specific applications, such as the US-APWR. When the US-Basic HSI System design is referenced within specific applications (e.g., US-APWR), it must be supplemented with design-specific content derived using NUREG-0711.

18-4

18.1 Human Factors Engineering Program Management 18.1.1 Introduction The HFE program management section describes the program for applying human factors principles to the design and engineering of the US-APWR. The objective of the staffs review is to confirm that the applicant has adequately considered the role of HFE and the means by which HFE activities will be accomplished.

18.1.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9, Human Factors Engineering.

DCD Tier 2: The applicant provided a Tier 2 description in Section 18.1, HFE Program Management, (HFEPM) identifying the HFE program goals, assumptions, and constraints. A description is provided of the applicants HFE program management, which includes the HFE design team and organization; the HFE process and procedures; HFE issues tracking; the HFE technical program; and COL information.

ITAAC: There are no ITAAC associated with this element.

Technical Specifications (TS): There are no TS associated with this element.

Topical Reports: Topical Reports associated with this element are:

• MUAP-07007, Human System Interface System Description, Revision 6, issued May, 2014 (ADAMS Accession No. ML14164A603).

• PQD-HD-19005, Quality Assurance Program (QAP) Description For Design Certification of the US-APWR, Revision 6, issued October 2013 (ADAMS Accession No. ML13350A511).

Technical Reports: Technical Reports associated with this element are as follows:

• MUAP-09019, US-APWR Human Factors Engineering Program Management Plan, Revision 5, issued August 2014 (ADAMS Accession No. ML14245A183).

• MUAP-10012, US-APWR Human Factors Verification and Validation Implementation Plan, Revision 4, issued May 2014 (ADAMS Accession No. ML14164A601).

• MUAP-10009, US-APWR Human-System Design Implementation Plan, Revision 4, issued May 2014 (ADAMS Accession No. ML14164A599).

• MUAP-13009, US-APWR Task Analysis Implementation Plan, Revision 1, issued May 2014 (ADAMS Accession No. ML14164A607).

18-5

18.1.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

• 10 CFR 50.34(f)(1)(i)

• 10 CFR 50.34(f)(2)

• 10 CFR 50.34(f)(3)(i)

• 10 CFR 50.34(f)(3)(vii)

• 10 CFR 50.54 (i) to (m)

• 10 CFR 50.120 Regulatory guidance is found in:

• NUREG-0711, Revision 2, Human Factors Engineering Program Review Model, Chapter 2, HFE Program Management, Section 2.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering.

• NUREG-0696, issued February 1981, Functional Criteria for Emergency Response Facilities.

18.1.4 Technical Evaluation The staff performed a complete element level review as described in NUREG-0711, and Section 18.0.4 of this report.

This section presents the applicable review criteria from NUREG-0711 (reproduced below) followed by an evaluation of each criterion. HFEPM review topics include the following:

• General HFE program goals and scope

• HFE team and organization

• HFE process and procedures

• HFE issues tracking

• Technical program 18.1.4.1 General HFE Program Goals and Scope NUREG-0711 includes six criteria for this topic. The sixth criterion addresses plant modifications and is not applicable to new reactors; thus the staff evaluated the first five criteria as discussed below.

18-6

Criterion 1 HFE Program Goals - The general objectives of the program should be stated in human-centered terms, which, as the HFE program develops, should be defined and used as a basis for HFE test and evaluation activities. Generic human-centered HFE design goals include the following:

• personnel tasks can be accomplished within time and performance criteria,

• the HSIs, procedures, staffing/qualifications, training and management and organizational support will support a high degree of operating crew situation awareness,

• the plant design and allocation of functions will maintain operation vigilance and provide acceptable workload levels i.e., to minimize periods of operator underload and overload, and

• the operator interfaces will minimize operator error and will provide for error detection and recovery capability.

The Staffs Evaluation of Criterion 1 MUAP-09019, Section 2.1, Human Factors Engineering Program Goals, identifies general program goals identical to those in the NUREG criterion. Specific details on how these goals are achieved is described in subsequent sections of Chapter 18 and in the IPs referenced within the DCD. For example, key human factors principles such as workload, situational awareness, and error reduction are measured, evaluated, and managed throughout the HFE design process to minimize operator error. Incorporation of these principles within the program goals and program IPs, ensures the HFE design is centered on maximizing the operators effectiveness.

Accordingly, the staff finds that the applicants treatment of program goals conforms to this NUREG-0711 criterion.

Criterion 2 Assumptions and Constraints - An assumption or constraint is an aspect of the design, such as a specific staffing plan or the use of specific HSI technology that is an input to the HFE program rather than the result of HFE analyses and evaluations. The design assumptions and constraints should be clearly identified.

The Staffs Evaluation of Criterion 2 DCD, Section 18.1.1.1, Assumptions and Constraints Identification, identifies the assumptions and constraints of the US-APWR HFE design. These include:

18-7

• The plant is designed to be operated with one reactor operator (RO) and one senior reactor operator (SRO) in the main control room (MCR). MUAP-09019 adds that the plant is designed to be operated by two operators in modes 1 and 2, including stabilization after an abnormal event, including events within and outside the design basis. While this is a design constraint, the application clearly states that the MCR staffing will meet the regulatory requirements of Title 10 of the Code of Federal Regulations (10 CFR), Part 50.54(m)(2)(iii)

• The functional requirement specifications for the Japanese APWR HSI design serves as the initial source of input to the US-APWR HFE design effort.

MUAP-09019, Section 2.2.1, Background, and Section 2.2.2, HSIS starting point, describe the relationship between the US-Basic HSI System design, as described in the TR, and the US-APWR HFE design. The US-Basic HSI System is the starting point for the US-APWR HFE design and is clearly identified as a constraint. In general, the US-Basic HSI System design specifies the general design, arrangement, and integration of the HSI components. These aspects of the US-Basic HSI System design will not change for the US-APWR HFE design unless some unique situation is identified. The US-APWR HFE design work is focused on identifying the specific controls, displays and alarms that will be added to the physical structure of the US-Basic HSI System.

The staff concludes that the assumptions and constraints discussed above completely describe the initial limitations placed on the HFE program. Accordingly, the staff finds that the applicants treatment of assumptions and constraints conforms to this NUREG-0711 criterion.

Criterion 3 Applicable Facilities - The HFE program should address the main control room, remote shutdown facility, technical support center (TSC), emergency operations facility (EOF), and local control stations (LCSs).

The Staffs Evaluation of Criterion 3 DCD, Section 18.1.1.2, Applicable Plant Facilities, states that the US-APWR HFE program addresses all the facilities listed in this criterion. MUAP-09019, Section 2.3, Applicable Plant Facilities, states that HFE analysis and HSI design activities are limited to those LCSs used by licensed or non-licensed operators. For other LCSs (e.g., LCSs specific to support chemistry, radiological control, maintenance, testing), the V&V program element encompasses communication between the operators in the MCR or Remote Shutdown Room (RSR) and personnel using these local stations.

EOF HFE design responsibilities are divided between the DC applicant and COL applicant. The DCD scope is limited to defining the plant safety information requirements (i.e., safety parameter display system) and requirements for voice communication with plant operators in the MCR, RSR, and TSC. MUAP-09019, Section 2.3.1.2, Emergency Operations Facility, states that the COL applicants define other communication and HSI inventory needs, and all human factors and HD considerations in accordance with NUREG-0696, Functional Criteria for Emergency Response Facilities.

18-8

NUREG-0696 addresses HFE design as outlined below:

• NUREG-0696, Section 4.7, Instrumentation, Data System Equipment, and Power Supplies, states, The design of the EOF data system equipment shall incorporate human-factors engineering with consideration for both operating and maintenance personnel.

• NUREG-0696, Section 4.8, Technical Data and Data System, states, Human-factors engineering shall be incorporated in the design of the EOF.

• The guidance in NUREG-0696 addresses HSI usability in the following paragraph from Section 4.8.

Trend-information display and time-history display capability is required in the EOF to give EOF personnel a dynamic view of plant systems, radiological status, and environmental status during an emergency. The EOF displays shall be designed so that call-up, manipulation, and presentation of data can be easily performed. The displays shall be partitioned to facilitate the retrieval of information by the different functional groups in the EOF. This may be accomplished with either separate display units or by logically separated information display pages available on a call-up basis at each data display unit. The EOF data display formats shall present information so that it can be easily understood by the EOF personnel operating the system. If display capabilities for news media briefings are provided in the EOF, these displays shall be separated physically from the EOF functional displays.

• NUREG-0696 contains guidance on V&V. Section 9, Verification and Validation Criteria, states, The design, development, qualification, and installation of the SPDS [safety parameter display system], TSC, EOF, and NDL [nuclear data link] facilities and systems shall be independently verified and validated by qualified personnel other than the original designers and developers.

Unlike the specific regulatory guidance for the Control Room HFE design, regulatory guidance for the EOF and TSC is general. The Control Room guidance addresses the HFE design process and final design attributes while the EOF/TSC guidance addresses final design attributes. Since NUREG-0696 provides accepted regulatory guidance for EOF functional standards and the staff identified no safety case for making the EOF HFE design more prescriptive, the staff finds the commitment to complete the EOF HFE design in accordance with NUREG-0696 to be acceptable. In addition, the staff notes that similar to the control room HFE design, the EOF HFE design is subject to an integrated system validation test. This integrated system validation test for the EOF is a full-participation emergency plan exercise which is included in the Emergency Planning ITAAC. This exercise demonstrates EOF functionality which in turn demonstrates that the HSIs are acceptable.

In Request for Additional Information (RAI) 780-5888, Question18-129 (ADAMS Accession No. ML112060570), the staff requested additional information regarding how the HFE elements 18-9

described in NUREG-0711 would be applied to the EOF communication and information requirements specifically addressed in the DCD scope. In its response to RAI 780-5888, Question 18-129, dated August 19, 2011 (ADAMS Accession No. ML11234A265), the applicant stated that the US-APWR task analysis for the EOF information and communication requirements will be conducted in conjunction with the development of the Severe Accident Management Guidelines. This process is similar to the task analysis conducted in conjunction with the development of the Emergency Response Guidelines and Emergency Operating Procedures. The applicant revised MUAP-13009 Task Analysis Implementation Plan, Section 2.0, Scope, to include this information.

Section 2.9.1.1, General HFE Program and Scope, of Tier 1 (Rev. 4) indicates that the HFE program will be applied to the facilities listed in NUREG-0711 including the MRC, RSR, EOF, LCS, and TSC.

The staff finds that the applicants HFE design program includes the appropriate facilities, and clearly identifies the standards that will be addressed within the design process for each. The standards conform to the regulatory guidance in NUREG-0711 and NUREG-0696. Accordingly, the staff finds that the applicants treatment of applicable facilities conforms to this NUREG-0711 criterion.

Criterion 4 Applicable HSIs, Procedures and Training - The applicable HSIs, procedures, and training included in the HFE program should include all operations, accident management, maintenance, test, inspection and surveillance interfaces (including procedures).

The Staffs Evaluation of Criterion 4 MUAP-09019, Section 2.4, Applicable Human-System Interfaces, Procedures, and Training, defines the HSIs that are included in the scope of the HFE program as it is applied to each facility. HSIs supporting operations, accident management, maintenance, test, inspection and surveillance interfaces are included within the scope description.

The application of HFE principles to procedures and training are addressed in Chapter 13, Conduct of Operations. The HFE design process identifies inputs to procedures and training as identified in MUAP--3009 (Task Analysis Implementation Plan), Section 3.1, Interfaces with Other HFE Program Elements and Section 1.0, Scope. Accordingly, the staff finds that the applicants treatment of applicable HSIs, procedures, and training conforms to this NUREG-0711 criterion.

Criterion 5 Applicable Plant Personnel - Plant personnel who should be addressed by the HFE program include licensed control room operators, as defined in 10 CFR Part 55 and the following categories of personnel defined by 10 CFR 50.120:

non-licensed operators, shift supervisor, shift technical advisor, instrument and control technician, electrical maintenance personnel, mechanical maintenance personnel, radiological protection technician, chemistry technician, and engineering support personnel. In addition, any other plant personnel who perform tasks that are directly related to plant safety should be addressed.

18-10

The Staffs Evaluation of Criterion 5 In MUAP-09019, Section 2.5, Applicable Plant Personnel, the applicant identifies the personnel addressed in the HFE program. The list includes the personnel identified in this acceptance criterion. Accordingly, the staff finds that the treatment of applicable personnel conforms to this NUREG-0711 criterion.

18.1.4.2 Human Factors Engineering Team and Organization The staff reviewed the responsibility, organizational placement and authority, composition, and staffing of the HFE design team described in US-APWR DCD, Tier 2, to determine whether it acceptably addresses these topics, as defined by NUREG-0711.

Criterion 1 Responsibility - The team should be responsible (with respect to the scope of the HFE program) for: (a) the development of all HFE plans and procedures; (b) the oversight and review of all HFE design, development, test, and evaluation activities; (c) the initiation, recommendation, and provision of solutions through designated channels for problems identified in the implementation of the HFE activities; (d) verification of implementation of team recommendations; (e) assurance that all HFE activities comply with the HFE plans and procedures; and (f) scheduling of activities and milestones.

The Staffs Evaluation of Criterion 1 The applicant uses an HFE team comprised of an HSI System design team, an HSI System V&V team, and an Expert Panel. The HFE team conducts all hardware and software design activities. The V&V team responsibilities are limited to conducting the V&V IP and are evaluated within that element of the HFE design process. The Expert Panel is generally used to provide independent reviews of decisions made by the other two groups. In the MUAP-09019, Section 3.1, HFE Responsibility, the applicant provides a list of HFE team responsibilities that encompass the HFE program responsibilities listed in the criterion. This criterion is limited to a statement of HFE team responsibilities; therefore the list provided by the applicant describing the HFE team responsibilities is sufficient. Subsequent sections of the DCD explain how these responsibilities are implemented within the HSI design process and are evaluated by the staff in subsequent sections of this safety evaluation (SE) chapter. Accordingly, the staff finds that the applicants treatment of the HFE teams responsibility conforms to this NUREG-0711 criterion.

Criterion 2 Organizational Placement and Authority - The primary HFE organization(s) or function(s) within the organization of the total program should be identified, described, and illustrated (e.g., charts to show organizational and functional relationships, reporting relationships, and lines of communication). When more than one organization is responsible for HFE, the lead organizational unit responsible for the HFE program plan should be identified. The team should have the authority and organizational placement to ensure that all its areas of responsibility are accomplished and to identify problems in the implementation of the overall plant design. The team should have the authority to control further 18-11

processing, delivery, installation, or use of HFE products until the disposition of a nonconformance, deficiency, or unsatisfactory condition has been achieved.

The Staffs Evaluation of Criterion 2 MUAP-09019, Section 3.2, HFE Organizational Placement and Authority, describes the project organization and the responsibilities of key positions in the organization. The DC applicant (MHI) is the lead organization for the US-APWR HFE design project, HSI system design, and HSI system V&V activities. Mitsubishi Electric Corporation (MELCO) is the lead organization for the conversion of the HSI functional design, into software and hardware for the HSI system test facilities and the actual plants. Figure 3-1, HFE Team Organization, in Section 3.3, HFE Organizational Composition, provides an organization chart which illustrates the organization from the Head Officer of Nuclear Division through the Engineering Management Director to the HFE Manager and all the HFE related organizations reporting to him. The HFE Manager is responsible for organizing the HFE team, oversight of the HFE processes, and controlling HFE resources including those outside of his direct line organization. As stated in Section 3.2, the HFE Manager has the authority to limit further processing, delivery, installation, or use of HFE products until the disposition of a nonconformance, deficiency, or unsatisfactory condition has been achieved.

MUAP-09019, Section 3.3, HFE Organizational Composition, states that the HFE design team has the responsibility to identify HFE issues and to oversee their correction and implementation in the overall plant design. Four specific steps describe how this responsibility is accomplished and include:

• Organizing meetings to identify and resolve HFE issues and discuss solutions,

• Defining organization responsibility for issue resolution,

• Tracking issue resolution, and

• Verifying the HFE issues have been resolved effectively.

The staff concludes that the DCD clearly describes the management organization associated with HFE activities and delineates management and HFE team responsibilities. The responsibilities are explicitly associated with management positions which, in the staffs judgment, have the authority to ensure that the responsibilities are accomplished. This authority includes control over any nonconformance or deficiency within its areas of responsibility to ensure an acceptable solution. Accordingly, the staff finds that the applicants treatment of the HFE teams organizational placement and authority conforms to this NUREG-0711 criterion.

Criterion 3 Composition - The HFE design team should include the expertise described in the Appendix [to NUREG-0711].

The Staffs Evaluation of Criterion 3 In MUAP-09019, Section 3.3, HFE Organizational Composition, the applicant addresses HFE team composition. All expertise areas identified in NUREG-0711, Appendix, HFE Design Team Composition, are included. Some areas of expertise are represented by matrixed engineers that report organizationally through other technical groups.

18-12

The combination of matrixed and dedicated staff personnel is an acceptable organizational structure. It is consistent with the NUREG-0711 guidance because the professional experience is satisfied by the HFE design team as a collective whole rather than on an individual basis.

MUAP-09019, Table 3-1, HFE Team General Qualifications states the minimum qualifications by degree and/or experience, which conform to NUREG-0711 guidance.

Accordingly, the staff finds that the applicants treatment of the HFE teams composition conforms to this NUREG-0711 criterion.

Criterion 4 Team Staffing - Team staffing should be described in terms of job descriptions and assignments of team personnel.

The Staffs Evaluation of Criterion 4 MUAP-09019, Section 3.2, HFE Organizational Placement and Authority, describes supervisory positions, their responsibility for HFE related activities, and teams that report to these supervisors. While specific personnel assignments are not provided, the description of responsibilities for the supervisors and teams provides sufficient explanation of how personnel are being used and is thus an acceptable substitute for job descriptions and personnel assignments. Accordingly, the staff finds that the applicants treatment of HFE team staffing conforms to this NUREG-0711 criterion.

18.1.4.3 Human Factors Engineering Process and Procedures Criterion 1 General Process Procedures - The process through which the team will execute their responsibilities should be identified. The process should include procedures for:

• assigning HFE activities to individual team members

• governing the internal management of the team

• making management decisions regarding HFE

• making HFE design decisions

• governing equipment design changes

• design team review of HFE products The Staffs Evaluation of Criterion 1 MUAP-09019, Section 4.1, General Process Procedures, states that the HFE review team will execute its responsibilities through procedures that include each of the areas identified in the acceptance criterion and as stated in the DCD, which are executed under the applicants quality assurance program (QAP) for the US-APWR (PQD-HD-19005, Quality Assurance Program (QAP) Description For Design Certification of the US-APWR).

18-13

The staff concludes that working level procedures are available to communicate work control responsibilities. Accordingly, the staff finds that the applicants treatment related to general HFE process procedures conforms to this NUREG-0711 criterion.

Criterion 2 Process Management Tools - Tools and techniques (e.g., review forms) to be utilized by the team to verify they fulfill their responsibilities should be identified.

The Staffs Evaluation of Criterion 2 General design process management tools are described in MUAP-09019, Section 4.2, Process Management Tools. These tools and techniques implement the QAP and include typical engineering controls such as independent reviews; the forms needed to track and document these reviews; procedures for the review, approval, release, distribution, and revision of design interface documents; and methods for documenting and tracking engineering change requests. For HFE design, more specific design commitments are contained in the IPs submitted as part of the DCD. These plans also contain more specific process management tools. Several examples are provided in the following list:

• MUAP-10012, Section 4.1.1, Sampling Dimensions, provides an example of a table used to compile the results of V&V activities for each HSI. This facilitates the determination that the HSI testing sample has been completed.

• MUAP-10012 provides for logs and forms to record data from the integrated system validation.

• A database is used to track all Human Error Deficiencies (HEDs). The HEDs themselves are part of a corrective action process that ensures problems are identified and fixed.

• MUAP-13009, Section 4.2.2, Basic Task Analysis - HSI Inventory, provides a table to capture task analysis results used as input for downstream design activities.

The staff concludes that these tools and techniques provide the administrative support needed by the team to verify that their responsibilities are met. Accordingly, the staff finds that the applicants treatment of the criterion for process management tools conforms to this NUREG-0711 criterion.

Criterion 3 Integration of HFE and Other Plant Design Activities - The integration of design activities should be identified, that is, the inputs from other plant design activities to the HFE program and the outputs from the HFE program to other plant design activities. The iterative nature of the HFE design process should be addressed.

The Staffs Evaluation of Criterion 3 18-14

In MUAP-09019, Section 4.3, Integration of HFE and Other Plant Design Activities, the applicant illustrates the work flow process in Figures 4.1, HFE Work Flow, and 4.2, Engineering Work Process and Integration between HFE Team and Plant Design Organization, showing interfaces with general plant design and feedback loops within the HSI design process that illustrate its iterative nature. Inputs and outputs for each element of the HFE design process are identified. Accordingly, the staff finds that the applicants treatment of HFE integration conforms to this NUREG-0711 criterion.

Criterion 4 HFE Program Milestones - HFE milestones should be identified so that evaluations of the effectiveness of the HFE effort can be made at critical check points and the relationship to the integrated plant sequence of events is shown.

A relative program schedule of HFE tasks showing relationships between HFE elements and activities, products, and reviews should be available for review.

The Staffs Evaluation of Criterion 4 MUAP-09019, Section 4.4, HFE Program Milestones, identifies specific points in the HFE design process where program effectiveness is evaluated. These checkpoints are incorporated within a relative program schedule of HFE tasks (Figure 4-3, HFE Program Milestones Embedded in the Plant Design, Procurement, Construction, and Operation,) so it is clear when they would be performed. Accordingly, the staff finds that the applicants treatment of HFE program milestones conforms to this NUREG-0711 criterion.

Criterion 5 HFE Documentation - HFE documentation items should be identified and briefly described along with the procedures for retention and access.

The Staffs Evaluation of Criterion 5 Specific documents to be generated for each HFE program element are described in MUAP-09019, Section 4.5, HFE Documentation, where it specifically notes that IPs and Results Summary Reports are design basis documents falling with the scope of the QAP. The documents are explained and the standard formats used to ensure proper content documentation are stated. Accordingly, the staff finds that the applicants HFE documentation conforms to this NUREG-0711 criterion.

Criterion 6 Subcontractor HFE Efforts - HFE requirements should be included in each subcontract and the subcontractor's compliance with HFE requirements should be periodically verified.

The Staffs Evaluation of Criterion 6 MUAP-09019, Section 4.6, Subcontractor HFE Efforts, states that the HFE team verifies the subcontractor is properly trained and complies with the US-APWR HFE IPs and MHI's internal work procedures. The MHI QA organization verifies subcontractors conduct their work in accordance with the MHI QAP or the subcontractors QAP as contracted. Accordingly, the staff 18-15

finds that the applicants control of subcontractor HFE efforts conforms to this NUREG-0711 criterion.

18.1.4.4 Human Factors Engineering Issues Tracking Criterion 1 Availability - A tracking system should be available to address human factors issues that are (a) known to the industry (defined in the Operating Experience Review [OER] element, see Section 18.3 of this report) and (b) identified throughout the life cycle of the HFE aspects of design, development, and evaluation. Issues are those items that need to be addressed at some later date and thus need to be tracked to provide reasonable assurance that they are not overlooked. It is not necessary to establish a new system to track HFE issues that is independent from the rest of the design effort. An existing tracking system may be adapted to serve this purpose (such as a plant's corrective action program, CAP).

The Staffs Evaluation of Criterion 1 US-APWR DCD Section 18.1.4, HFE Issues Tracking, and MUAP-09019, Section 5.1, Human Engineering Discrepancy Process, describes the HFE issues tracking system. This system ensures that HFE problems, issues and HEDs identified throughout the development and evaluation of the HFE design are addressed. The tracking system includes the known industry issues and human factors issues identified throughout the execution of the US-APWR HFE program elements. MUAP-09019 specifies that the HFE design team members are responsible for issue logging, tracking, resolution, resolution acceptance, and ensuring the quality standards associated with this work, as described in the QAP, will be followed. Accordingly, the staff finds the applicants treatment of the HFE issue tracking system conforms to this NUREG-0711 criterion.

Criterion 2 Method - The method should document and track HFE issues from identification until the potential for negative effects on human performance has been reduced to an acceptable level.

The Staffs Evaluation of Criterion 2 US-APWR DCD, Section 18.1.4, HFE Issues Tracking, states that the HFE design team is responsible for issue logging, tracking, resolution, and resolution acceptance. MUAP-09019, Section 5.1.2, Human Engineering Discrepancy (HED) Evaluation, states that outstanding HEDs are evaluated at least every six months and prior to completing each HFE program phase. HFE IPs define the HEDs that must be closed to initiate or complete a specific program element. For example, all HEDs must be closed prior to initiating the V&V program element and any HEDs generated during or after V&V must be closed prior to completing the Design Implementation program element. Accordingly, the staff finds that the applicants treatment of HFE issue tracking methodology conforms to this NUREG-0711 criterion.

Criterion 3 18-16

Documentation - Each issue or concern that meets or exceeds the threshold established by the design team should be entered into the system when first identified, and each action taken to eliminate or reduce the issue or concern should be thoroughly documented. The final resolution of the issue should be documented in detail, along with information regarding design team acceptance.

The Staffs Evaluation of Criterion 3 US-APWR DCD, Section 18.1.4, HFE Issues Tracking, states that all HFE issues and concerns that are not immediately resolved are entered in the HFE issues tracking system.

When these issues are resolved, the resolution is also documented along with any test results that validate the resolution. MUAP-09019, Section 5.1.3.2, Human Engineering Discrepancy Processing, provides additional detail regarding how HEDs are processed. Tables are used to illustrate the documentation associated with each process step.

The staff concludes that the information recorded in the HED database provides sufficient documentation of the problem, actions taken to resolve the problem, and the final resolution.

Accordingly, the staff finds that the applicants treatment of the HFE issues tracking system documentation conforms to this NUREG-0711criterion.

Criterion 4 Responsibility - When an issue is identified, the tracking procedures should describe individual responsibilities for issue logging, tracking and resolution, and resolution acceptance.

The Staffs Evaluation of Criterion 4 MUAP-09019, Section 5.0, HFE Issues Tracking, describes four steps in the HED process:

discrepancy identification and problem statement, discrepancy evaluation, discrepancy resolution, and discrepancy closure. Subsections describe the responsibilities. In general, the person who identifies the problem, documents it. This includes the HFE design team members, the operators participating in validation work, and those who evaluate the questionnaires, surveys, and performance data. Discrepancy evaluation and resolution is the responsibility of the HFE design team. Resolution closeout is the responsibility of an expert panel assisted by the HFE design team. A controlled database is used to ensure only those authorized can change the status of an HED. This HFE Issues Tracking System has fields for documenting information, including fields to clearly identify issue significance, assignment of actions to responsible organizations and individual action owners, due dates, and resolution status.

Accordingly, the staff finds that the applicants treatment of HED responsibilities conforms to this NUREG-0711 criterion.

18.1.4.5 Technical Program The evaluation of the HFE technical program, as part of Element 1 of NUREG-0711, addresses scoping, resources, and management details. Actual technical details are addressed in the respective element reviews. NUREG-0711 includes five criteria for this topic. The fourth and fifth criteria address plant modifications and are not applicable to new reactors, thus only the first three criteria are evaluated below.

18-17

Criterion 1 The general development of implementation plans, analyses, and evaluation of the following should be identified and described:

• OER

• FRA and FA

• Task analysis

• Staffing and qualifications

• HRA

• HSI design

• Procedure design

• Training design

• Human factors V&V

• Design implementation

• Human performance monitoring The Staffs Evaluation of Criterion 1 The applicants technical program, as presented in US-APWR DCD Tier 2, Chapter 18, incorporates all of the identified NUREG-0711 elements. MUAP-09019, Section 4.3, Integration of HFE and Other Plant Design Activities, describes the element interfaces. For example, Figure 4-1, HFE Work Flow, provides a functional block diagram illustrating the work flow between elements. Table 4-1, INPUT and OUTPUT between HFE Activities and Other Plant Design Organizations, describes the inputs and outputs for each activity. MUAP-09019, Section 6.1, Implementation Plans, Analyses, and Evaluations, describes the evaluations to be completed for each of the HFE program elements listed in this criterion. Accordingly, the staff finds that the applicants treatment of HFE IPs, analyses, and evaluations conforms to this NUREG-0711 criterion.

Criterion 2 The HFE requirements imposed on the design process should be identified and described. The standards and specifications that are sources of HFE requirements should be listed.

The Staffs Evaluation of Criterion 2 MUAP-09019, Section 6.2, Human Factors Engineering Requirements, contains an extensive listing of nuclear industry documents that encompass codes and standards, NRC documents, and other industry documents, such as those from the Institute of Electrical and Electronics Engineers (IEEE). These documents encompass those commonly used as sources for HFE design. Accordingly, the staff finds the applicants treatment of HFE requirements conforms to this NUREG-0711 criterion.

Criterion 3 HFE facilities, equipment, tools, and techniques (such as laboratories, simulators, rapid prototyping software) to be utilized in the HFE program should be specified.

18-18

The Staffs Evaluation of Criterion 3 MUAP-09019, Section 6.3, HFE Facilities, Equipment, Tools, and Techniques, describes an iterative HSI design process. Initially a part-task simulator is used to develop a set of plant control parameters and graphical interfaces. As the HSI design proceeds, the part task simulator proceeds through a series of iterative evaluations, resulting in the development of a full-scope control room simulator. The simulator facility is used as the focal point for HFE development, engineering design verification, and operator evaluation/validations throughout the HSI design process. Accordingly, the staff finds that the applicants treatment of HFE facilities conforms to this NUREG-0711 criterion.

18.1.5 Combined License Information Items There are no COL information items related to this area of review. The staff determined that no COL information items need 2.4 to be included in DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for HFE program management consideration.

18.1.6 Conclusions The staff evaluated HFE Program Management at a complete element level using the review criteria in NUREG-0711, Section 2.4. Section 18.0.4 of this report provides a discussion of review levels. For the reasons set forth above, the staff finds that the applicants DCD and Technical Report MUAP-09019, Human Factors Engineering Program Management Plan, have identified general HFE program goals and scope, specified an acceptable HFE team and organization, implemented appropriate HFE processes and procedures, developed an HFE issues tracking system, and established an acceptable HFE technical program. Therefore, the staff concludes that HFE considerations with respect to program management have been adequately addressed, and that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47 related to this technical area are satisfied.

18.2 Operating Experience Review 18.2.1 Introduction The objective of this review is to verify that the applicant has identified and analyzed HFE related problems and issues in previous designs so that these problems and issues may be avoided in the development of the new design. This review should also verify that the applicant has retained positive features of previous designs.

18.2.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9, Human Factors Engineering, and Table 2.9-1, Human Factors Engineering Inspections, Tests, Analyses, and Acceptance Criteria.

DCD Tier 2: The applicant provided a Tier 2 description in DCD, Section 18.2, Operating Experience Review, that describes the method used to evaluate operating experience for lessons learned that would improve the HFE design. This review includes licensee event reports; Institute of Nuclear Power Operations (INPO) significant event reports and significant 18-19

operating experience reports; plant corrective action systems; operational and maintenance logs and records; and data from interviews with experienced plant personnel.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

Topical Reports: There are no Topical Reports associated with this element.

Technical Reports: Technical Reports associated with this element are:

• MUAP-13005, US-APWR Operating Experience Review Implementation Plan, Revision 1, issued May 2014 (ADAMS Accession No. ML14164A605).

• MUAP-09019, US-APWR Human Factors Engineering Program Management Plan, Revision 5, issued August 2014.

18.2.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

• 10 CFR 50.34(f)(3)(i)

• 10 CFR 50.34(f)(2)

Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 3, Operating Experience Review, Section 3.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition - Human Factors Engineering.

18.2.4 Technical Evaluation The staff performed a complete element level of review as described in NUREG-0711 and Section 18.0.4 of this report. This section presents the applicable review criteria from NUREG-0711, followed by an evaluation of each. OER topics include the following:

• Scope

• Issue analysis, tracking, and review 18.2.4.1 Scope Criterion 1 Predecessor/Related Plants and Systems the review should include information pertaining to the human factors issues related to the predecessor plant(s) or highly 18-20

similar plants and plant systems. For a review of plant modifications, the scope of the OER should be focused to provide information relevant to the plants systems, HSIs, procedures, or training that are being modified. It should address the operating experience of the plant that will be modified, including experiences with the systems that will be modified, and with technologies that are similar to those under consideration for it.

Some useful information may be found in the plants CAP. Also, when personnel are unfamiliar with the proposed technology, attention should be paid to the operating experience of other plants that already have the technology.

The Staffs Evaluation of Criterion 1 MUAP-13005, Section 1.0, Purpose, states that the objective of the OER is to identify and analyze HFE-related problems and issues encountered in previous nuclear plant designs that are similar to the US-APWR, so that the negative features are not repeated and the positive features are retained. The OER identifies past performance information from earlier designs.

Performance information from predecessor designs is identified at the start of the design process and used to improve the plant design.

The applicants US-APWR OER includes:

• Japanese conventional 3-loop pressurized-water reactor (PWR) with full digital instrumentation and control (I&C) and HSI, which is an operating plant,

• Japanese 4-loop APWR with full digital I&C and HSI which is a plant under licensing,

• Japanese conventional 2-loop PWR with full digital I&C and HSI modernization which is an operating plant, and

• Currently operating U.S. PWRs.

The Japanese plants are the predecessor design and were used as the starting point for development of the US-Basic HSI System design. The US-Basic HSI System design is the foundation of the US-APWR HSI System design and is described in MUAP-07007 and evaluated in the SER for that topical report.

The MUAP-13005, Section 2.0, Scope, provides additional detail on the specific sources of information for the plants listed above. These sources include:

• NUREG/CR-6400, HFE Insights For Advanced Reactors Based Upon Operating Experienc.

• INPO database

• Nuclear Information Archives (NUCIA) database - Japan Nuclear Technologies Institute (JANTI) databases are called NUCIA. NUCIA includes data from thousands of events, including failures, operational errors etc., from Japanese nuclear power stations.

18-21

The applicant has identified operating experience from the predecessor plant and operating U.S. plants as input to the US-APWR design and designated the sources for this material and the types of reports to be evaluated. Accordingly, the staff finds that the applicants treatment of operating experience from predecessor/related plants and systems conforms to this NUREG-0711 criterion.

Criterion 2 Recognized Industry HFE Issues - NUREG/CR-6400, Human Factors Engineering (HFE) Insights for Advanced Reactors Based Upon [sic] Operating Experience (Higgins and Nasta, 1996) issues should be addressed. The issues are organized into the following categories:

• unresolved safety issues/generic safety issues

• TMI issues

• NRC generic letters and information notices

• reports of the former NRC Office for Analysis and Evaluation of Operational Data

• low power and shutdown operations

• operating plant event reports The Staffs Evaluation of Criterion 2 MUAP-13005, Section 4.1.2, Recognized Industry Human Factors Engineering Issues from NUREGs, restates the acceptance criterion and notes that this source document only provides a brief summary of each issue. The IP augments the operating experience source list with other selected NUREGs and searches of the NRCs Agencywide Documents Access and Management System (ADAMS) for generic letters and information notices. Accordingly, the staff finds that the applicants treatment of recognized industry HFE issues conforms to this NUREG-0711 criterion.

Criterion 3 Related HFE Technology - The OER should address related HFE technology.

For example, if touch screen interfaces or computerized procedures are planned, HFE issues associated with their use should be reviewed.

The Staffs Evaluation of Criterion 3 MUAP-13005, Section 4.1.3, Related Human Factors Engineering Technology, states that nuclear and non-nuclear industry experience related to HFE technologies such as touch screen interfaces, large screen displays, and computerized procedures used in the US-APWR HSI System design will be evaluated. The non-nuclear industry scope includes the chemical, transportation, and electrical transmission industries. This additional scope is added due to the limited nuclear industry experience with the technologies used in the US-APWR design.

The applicant outlined a detailed plan for assessing the operating experience for new HFE technologies which includes relevant experience outside the nuclear industry. By including non-nuclear experience the applicant has maximized the potential for applying lessons learned on new technology applications within the US-APWR design. Accordingly, the staff finds that the applicants treatment of related HFE technology conforms to this NUREG-0711 criterion.

18-22

Criterion 4 Issues Identified By Plant Personnel - Personnel interviews should be conducted to determine operating experience related to predecessor plants or systems.

The following topics should be included in the interview:

• Normal plant evolutions

• Instrument failures

• HSI equipment and processing failure

• Transients

• Accidents

• Reactor shutdown and cooldown using remote shutdown system HFE Design Topics:

• Alarm and annunciation

• Display

• Control and automation

• Information processing job aids

• Real-time communications with plant personnel and other organizations

• Procedures, training, staffing/qualifications and job design The Staffs Evaluation of Criterion 4 MUAP-13005, Section 4.1.4, Issues identified by plant personnel, states that personnel interviews were conducted to collect information on all the topics listed in the acceptance criterion.

The applicant has consistently used operating crews in a full scope simulator as part of their design development process. During the US-Basic HSI System Phase 1 V&V program described in MUAP-07007 and evaluated in the associated SER, U.S. plant operators (13 crews, 32 operators in all) evaluated the Japanese-Basic HSI System (the starting point for the US-Basic HSI System) by testing the system in selected simulator scenarios. After these tests, the operators were interviewed by the US-APWR HFE team to compare the Japanese-Basic HSI System performance with the operators experience with similar scenarios at their own plants.

Similarly, operating crews were used to support design testing during the US-APWR HSI design development described in Section 18.7 of this report. Operators provided feedback regarding the functionality of all the design topics listed in the acceptance criteria.

The staff concludes that the applicant is using operator interviews as well as operator involvement in design testing as in input to the US-APWR HSI design. Accordingly, the staff finds that the applicants treatment of issues identified by plant personnel conforms to this NUREG-0711 criterion.

18-23

Criterion 5 Risk-Important Human Actions - The OER should identify risk-important human actions that have been identified as different or where errors have occurred. The human actions should be identified as requiring special attention during the design process to lessen their probability.

The Staffs Evaluation of Criterion 5 In MUAP-13005, Section 4.2.3, Important Human Actions, the applicant states that the OER includes risk-important human actions (RIHAs) from the US-APWR probabilistic risk assessment (PRA) and those important human actions deterministically derived from the US-APWR transient accident analysis and diversity, and defense-in-depth (D3) coping analysis.

For RIHAs, the subject matter expert (SME) assessment confirms the historical HFE issues from operating experience (OE) are accurately reflected in the assessment of human error probability and the HFE design addresses these issues. For the deterministically derived important manual actions, SME assessment confirms the historical HFE issues from OE are accurately reflected in the time required to execute the credited manual action, or that this time accurately reflects adjustments facilitated by the US-APWR plant design or HSI features that are different from those in the historical HFE issue.

The staff concludes that the applicant is appropriately using operating experience to 1) validate PRA assumptions relative to the important human actions and to 2) confirm the US-APWR HFE design addresses the historical HFE issues including the time required to implement the human action. Accordingly, the staff finds that the applicants treatment of important human actions conforms to this NUREG-0711 criterion.

18.2.4.2 Issue Analysis, Tracking, and Review Criterion 1 Analysis Content: The issues should be analyzed with regard to the identification of:

• Human performance issues, problems, and sources of human error

• Design elements that support and enhance human performance The Staffs Evaluation of Criterion 1 MUAP-13005, Section 4.2.1, Extraction and Analysis Process, details the evaluation process used by the subject matter experts and HSI Design Team to select and analyze the issues. The process specifically states that the evaluation objective is to identify issues associated with the two areas described in the acceptance criterion bullets. If the issue is not addressed in the current design, an HED is generated to resolve it. Accordingly, the staff finds that the applicants treatment of OER analysis content conforms to this NUREG-0711 criterion.

Criterion 2 Documentation - The analysis of operating experience should be documented in an evaluation report.

18-24

The Staffs Evaluation of Criterion 2 MUAP-13005, Section 6.0, Results Summary Report Content, describes the documentation the applicant will provide containing the analysis of the OER activity. The report includes the OER execution results containing details that demonstrate compliance to the Methodology section of MUAP-13005. The report will include:

• A table detailing the OE references reviewed.

• The OER database which details all HFE issues extracted from the data source review and the OER evaluation.

• The HEDs identified during the operating experience review.

• The list of important human actions and their evaluation results.

• Evidence that the OER review was performed in accordance with the implementation plan.

The staff concludes that all essential elements of the OER are being documented. Accordingly, the staff finds the applicants treatment of OER documentation conforms to this NUREG-0711 criterion.

Criterion 3 Incorporation into the Tracking System - Each operating experience issue determined to be appropriate for incorporation in the design (but not already addressed in the design) should be documented in the issue tracking system.

The Staffs Evaluation of Criterion 3 MUAP-13005, Section 4.2.1, Extraction and Analysis Process, states that issues identified from the OER that are not addressed in the current US-APWR HFE design are included in the HED database tracking system. MUAP-13005, Section 4.2.2, Extraction and Analysis Documentation, detail the use and content of two documents for tracking OER identified issues:

the OER Issues and Resolutions List and the HED list generated from the OER process.

The staff concludes that potential HFE issues identified during the OER are being documented in an issue tracking system. Accordingly, the staff finds that the applicants treatment of OER HED tracking conforms to this NUREG-0711 criterion.

18.2.5 Combined License Information Items There are no COL information items related to this area of review. The staff determined that no COL information items need to be included in DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for the HFE OER.

18-25

18.2.6 Conclusions The staff evaluated the OER with respect to HFE, at an IP level using the review criteria in NUREG-0711, Section 3.4, Review Criteria. Section 18.0.4 of this report provides a discussion of review levels. The staff determined that the OER information provided in MUAP-13005, MUAP-09019, and the DCD, for the US-APWR demonstrates that plant operating experience is adequately considered and addressed, thereby minimizing the effect of HFE-related problems and issues that have occurred in the past. Therefore, the staff concludes that HFE considerations with respect to OER have been adequately addressed, and that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47, related to this technical area, are satisfied.

18.3 Functional Requirements Analysis and Function Allocation 18.3.1 Introduction Functional Requirements Analysis is the identification of functions that must be performed to satisfy plant safety objectives, that is to prevent or mitigate the consequences of postulated accidents that could cause undue risk to the health and safety of the public. Function Allocation is the analysis of requirements for plant control and the assignment of control functions to:

(1) personnel (e.g., manual control); (2) system elements (e.g., automatic control and passive, self-controlling phenomena); and (3) combinations of personnel and system elements (e.g.,

shared control, automatic systems with manual backup).

The objective of the staffs review is to verify that: (1) the plant's functions that must be performed to satisfy plant safety objectives have been defined, and (2) the allocation of those functions to human and system resources has resulted in a role for personnel that takes advantage of human strengths and avoids human limitations.

18.3.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in DCD, Section 2.9, Human Factors Engineering, and Table 2.9-1, Human Factors Engineering Inspections, Tests, Analyses, and Acceptance Criteria.

DCD Tier 2: The applicant has provided a Tier 2 description in Section 18.3, Functional Requirements Analysis and Function Allocation, which describes the method for performing the FRA and FA.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

Topical Reports: The topical report associated with this element is:

• PQD-HD-19005, Quality Assurance Program (QAP) Description For Design Certification of the US-APWR, Revision 6, issued October 2013.

Technical Reports: The technical reports associated with this element are:

18-26

• MUAP-13007, US-APWR Functional Requirements Analysis and Function Allocation Implementation Plan, Revision 1, issued May 2014 (ADAMS Accession No. ML14164A606).

• MUAP-13009, US-APWR Task Analysis Implementation Plan, Revision 1, issued May 2014.

18.3.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

• 10 CFR 50.34(f)(3)(i)

• 10 CFR 50.34(f)(2)

Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 4, Functional Requirements Analysis and Function Allocation, Section 4.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18.II.A.1, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition -

Human Factors Engineering.

18.3.4 Technical Evaluation The staff performed an IP level review, as described in NUREG-0711 and Section 18.0.4 of this report.

This section presents the applicable review criteria from NUREG-0711 followed by an evaluation of each criteria. The eleventh criterion of this NUREG-0711 element is related to plant modifications and is not applicable to new reactor DCs. Thus, this criterion is not included in the staffs evaluation.

With respect to FRA and FA methodology, MUAP-13007, provides the detailed process for conducting the FRA and FA.

Criterion 1 Functional requirements analysis and function allocation should be performed using a structured, documented methodology reflecting HFE principles. An example function allocation process and considerations are shown in Figure 4.1

[Allocation of functions to human and machine resources] of NUREG-0711.

The functional requirements analysis and function allocation may be graded based on:

• the degree to which the functions of the new design differ from those of the predecessor, and 18-27

• the extent to which difficulties related to plant functions were identified in the plants operating experience and will be addressed in the new design.

The Staffs Evaluation of Criterion 1 The staff reviewed the applicants overall FRA/FA methodology using this criterion. A graded approach limited to changes from a predecessor plant is not used; rather, the MHI methodology follows the traditional approach of defining two high level goals, safety and power production, then the high level functions needed to obtain the goals, then the success paths needed to obtain the high level functions. The success paths actions are then allocated to human and system resources resulting in a role for personnel that takes advantage of human strengths and avoids human limitations. A success path is defined as, the aggregate of sub-functions, plant systems, key components, and the actions to be performed to maintain or restore a high-level function.

MUAP-13007, Section 3.1, Functional Requirements Analysis and Function Allocation, provides three figures (Figure 3-1, FRA/FA Analytical Data Flow; Figure 3-1, FRA/FA Process Overview; and Figure 3-3, FRA Hierarchical Structure,) which together, graphically present a structured methodology for performing FRA/FA. The methodology is based, in part, on NRC guidance such as NUREG/CR-3331, A Methodology for Allocation of Nuclear Power Plant Control Functions to Human and Automated Control, and on accepted industry sources such as the International Electro technical Commission (IEC) standard 60964 (Design for Control Rooms of Nuclear Power Plants, Edition 2) and IEC 61839 (Nuclear Power Plants - Design of Control Rooms - Functional Analysis and Assignment, Edition 1).

MUAP-13007, Section 4.2, FRA/FA Configuration and Control, states that the FRA/FA follows preliminary plant design development and is sufficiently detailed to enable specification of detailed plant design and HSI design requirements. HFE principles such as time available, action frequency, workload, and task complexity are considered in the FA analysis. The processes that support the applicants FRA/FA methodology are evaluated in more detail in subsequent review criteria.

The staff concludes that the FRA/FA process described in the MUAP-13007 provides a structured, documented methodology reflecting appropriate HFE principles. Accordingly, the staff finds that the applicants treatment of the FRA/FA methodology conforms to this NUREG-0711 criterion.

Criterion 2 The functional requirements analysis and function allocation should be kept current over the life cycle of design development and held until decommissioning so that it can be used as a design basis when modifications are considered.

Control functions should be re-allocated in an iterative manner, in response to developing design specifics, operating experience, and the outcomes of ongoing analyses and trade studies.

The Staffs Evaluation of Criterion 2 MUAP-13007, Section 4.2, FRA/FA Configuration Control, and Section 4.11, Modification Requiring Additional FRA/FA Consideration, indicate that FRA/FA activities are included in the overall MHI QAP, which includes design changes and revisions that affect the designs 18-28

functional requirements and function allocation assignments. Engineering changes, which might affect FRA/FA, are part of this quality program and subject to the QAP design change process.

The staff concludes that the FRA/FA is kept current over the life cycle of the plant by virtue of being incorporated into the applicants QAP. Accordingly, the staff finds that the applicants treatment of FRA/FA records conforms to this NUREG-0711 criterion.

Criterion 3 A description of the functions and systems should be provided along with a comparison to the reference plants/systems, i.e., the previous plants or plant systems on which the new system is based. This description should identify differences that exist between the proposed and reference plants/systems.

Safety functions (e.g., reactivity control) include functions needed to prevent or mitigate the consequences of postulated accidents that could cause undue risk to the health and safety of the public. For each safety function, the set of plant system configurations or success paths that are responsible for or capable of carrying out the function should be clearly defined. Function decomposition should start at top-level functions where a very general picture of major functions is described, and continue to lower levels until a specific critical end-item requirement emerges (e.g., a piece of equipment, software, or human action (HA)). The functional decomposition should address the following levels:

• Hhgh-level functions (e.g., maintain reactor coolant system (RCS) integrity) and critical safety functions (e.g., maintain RCS pressure control).

• Specific plant systems and components.

The Staffs Evaluation of Criterion 3 MUAP 13007, Section 4.3.2, High-Level Functions, defines the safety functions that must be maintained to achieve the plant goals (safety and power production). The response to industry operating experience Fuel Storage Control, is included. For each high level function, success paths are developed which include the following components:

• sub-functions (e.g., reactor coolant system (RCS) level control),

• plant systems (e.g., chemical and volume control system),

• key components (e.g., letdown control valves, charging pumps), and

• action required of those key components (e.g., modulate).

Sub functions and systems are not described (other than by example) but it is clear from the direction and example forms provided that this information is collected as part of the analysis.

Differences between the proposed and predecessor plant/systems are listed in Section 4.3.4, Comparison to Reference Plant Systems, even though the FRA/FA analyses do not assume the predecessor plant configuration as a starting point.

Plant operations SMEs perform the success path identification, reviewing postulated accidents (PAs) and anticipated operational occurrences (AOOs) against the FRA success paths to 18-29

ensure that adequate success paths exist to protect high-level functions during these events.

The full range of plant operating modes (full power, low power, shutdown (including refueling)),

conditions (e.g., normal or abnormal), and parameters indicating the functions needed, are addressed in the FRA/FA analyses.

The staff concludes that the high level functions needed to meet plant goals are clearly identified and are complete. The methodology and the documentation of analysis results provide for complete descriptions of the systems, components, and actions that constitute the success paths needed to control the high level functions. Accordingly, the staff finds that the applicants treatment of FRA success paths conforms to this NUREG-0711 criterion.

Criterion 4 A description should be provided for each high-level function which includes:

• purpose of the high-level function

• conditions that indicate that the high-level function is needed

• parameters that indicate that the high-level function is available

• parameters that indicate the high-level function is operating (e.g., flow indication)

• parameters that indicate the high-level function is achieving its purpose (e.g.,

reactor vessel level returning to normal)

• parameters that indicate that operation of the high-level function can or should be terminated Note that parameters may be described qualitatively (e.g., high or low). Specific data values or set points are not necessary at this stage.

The Staffs Evaluation of Criterion 4 MUAP-13007, Section 4.3, FRA Methodology, provides the detailed process to accomplish the FRA. Plant operations SMEs perform the FRA by completing data sheets which contain a minimum set of attributes. These attributes are presented in a table (MUAP-13007, Figure 4-1, FRA/FA Combined Data Sheet Example is an example of the table) which describes and evaluates (1) plant goals and high-level functions; (2) decomposition of the functions into success paths (sub-functions, systems, components, and actions); and (3) includes several columns that describe the high-level function in terms of the characteristics listed in the acceptance criterion. The table has additional columns to identify the time available for when a success path must be completed to protect or maintain the power production or safety goal and to identify how often the success path must be taken to respond to the condition that requires the activity (i.e., return the high-level condition to its normal condition).

MUAP-13007, Section 4.4, High-Level Function Purpose and Description, includes all the characteristics specified by this NUREG-0711 criterion. The purpose of each characteristic is explained along with directions for documenting data associated with the characteristic.

18-30

The staff concludes that the characteristics needed to define each high level function are clearly identified and are complete. The methodology provides for complete documentation of each characteristic. Accordingly, the staff finds that the applicants treatment of high level function characteristics conforms to this NUREG-0711 criterion.

Criterion 5 The technical basis for modifications to high-level functions in the new design (compared to the predecessor design) should be documented.

The Staffs Evaluation of Criterion 5 MUAP-13007, Section 4.5, Predecessor Design, states that the US-APWR FRA/FA is not based on a predecessor design. However, as information, Section 4.3.4, Comparison to Reference Plant Systems, lists design changes from the conventional PWR plant that impact high-level functions in the US-APWR plant system design. Accordingly, the staff finds that the applicants treatment of changes from the predecessor design conforms to this NUREG-0711 criterion.

Criterion 6 The technical basis for all function allocations should be documented; including the allocation criteria, rationale, and analyses method. The technical basis for function allocation can be any one or combination of the evaluation factors (see NUREG-0711, Fig 4.1). For example, the performance demands to successfully achieve the function, such as degree of sensitivity needed, precision, time, or frequency of response, may be so stringent that it would be difficult or error prone for personnel to accomplish. This would establish a basis for automation (assuming acceptability of other factors, such as technical feasibility or cost).

The Staffs Evaluation of Criterion 6 MUAP-13007, Section 4.6, Function Allocation, provides a detailed explanation of the process used to allocate personnel functions for the US-APWR design. The technical basis for the US-APWR function allocation is derived from five evaluation characteristics that are described in IEC 6094 (Design for Control Rooms of Nuclear Power Plants, IEC 60964, Edition 2, International Electro-technical Commission, 2009) and IEC 61839 (Nuclear Power Plants -

Design of Control Rooms - Functional Analysis and Assignment, IEC 61839, Edition 1, International Electro-technical Commission, 2000). These five evaluation characteristics are:

load, time available, rate, action logic complexity, and decision type complexity. These evaluation characteristics are accepted FA metrics. They are used to establish the level of automation and/or operator manual action for performing the success paths.

Plant operations and system SMEs, assess the functions identified by the FRA for each success path to determine the role of personnel, automation, or a combination, using the five evaluation characteristics. Each of the characteristics is rated and scores are assigned to each characteristic. The scores reflect the level of influence each characteristic has on whether the success path is best suited for completion by humans, automation, or a combination. MUAP-13007 defines each characteristic and provides guidance for how to assess each characteristic to derive a rated value. The rated value is used to determine the function allocation (the 18-31

concept is explained in more detail in the staffs evaluation of Criterion 9 below). The guidance for calculating rated values was validated through a pilot study.

The technical basis and rationale for function allocations are captured in data tables of which examples are provided in MUAP-13007. Completed data entries from these tables are incorporated into a summary table showing the relationship of the FAs to the success paths derived from the FRA.

The staff concludes that the criteria and process for determining function allocations is clearly and completely described and incorporates accepted HFE practices. The basis for allocations is well documented. Accordingly, the staff finds that the applicants treatment of function allocation conforms to this NUREG-0711 criterion.

Criterion 7 The OER should be used to identify modifications to function allocations, if necessary. If problematic OER issues are identified, then an analysis should be performed to (a) justify the original analysis of the function, (b) justify the original human-machine allocation, and (c) identify solutions such as training, personnel selection, and procedure design that will be implemented to address the OER issues.

The Staffs Evaluation of Criterion 7 MUAP-13007, Section 4.7, Operating Experience Review, discusses the role of operating experience in the applicants function allocation process. Plant operations SMEs are responsible for addressing this activity with input from systems engineering SMEs, as needed.

The SMEs use input from the OER to identify issues that might affect the US-APWR function allocations causing the allocations to change. OER issues related to US-APWR high-level functions or success paths are reviewed. For issues that are identified from the review, a data sheet is prepared which records a cross-reference to the item in the OER, a brief description or title of the OER issue, the high-level function success path associated with the OER issue, and a description of the allocation issue and resolution (e.g., FA changed, HED initiated).

If the OER allocation differs from the FA results, the FA is re-evaluated. If the re-evaluation does not change the FA, this is noted in the data sheet. When an FA is affected, an HED is generated.

The issue and resolution are documented in the data sheet. Any changes made to the FA are re-checked against the US-APWR design. New HEDs are generated, if necessary.

The staff concludes that operating experience is being used as an input into function allocation decisions, challenges and decisions are being appropriately documented, and HEDs are being used to track problem resolution. Accordingly, the staff finds that the applicants treatment of operating experience input to function allocation conforms to this NUREG-0711 criterion.

Criterion 8 The allocation analysis should consider not only the primary allocations to personnel, but also their responsibilities to monitor automatic functions and to assume manual control in the event of an automatic system failure.

The Staffs Evaluation of Criterion 8 18-32

MUAP-13007, Section 4.6.2, Load, describes how operator workload is assessed as part of the function allocation decision. The load analysis determines mental and physical workload for humans supervising (including monitoring automatic functions) and/or manually controlling success paths to maintain a specific high-level function, while concurrently supervising or controlling other success paths to concurrently maintain all other high-level functions. The load analysis evaluates the integrated workload across all applicable high-level functions for the specific plant mode/condition without considering the potential distribution of that workload among multiple personnel.

The assumption of manual control following an automatic system failure is not addressed in the FA load analysis. Section 7.2, Justification of Deviations from NUREG-0711, states that actions to take manual control following automation failure are addressed in the task analysis.

MUAP-13009, Task Analysis Implementation Plan, explains that failures of automatic actions identified from the FRA/FA are analyzed to identify tasks related to monitoring and backing up automation and a separate analysis is also performed for manual actions to accommodate automation failure. The Task Analysis (TA) also confirms the FRA/FA allocations.

The staff concludes that the load analysis provides for an assessment of the operators responsibility for monitoring and/or controlling multiple success paths as an input to FA decisions. The deviation taken on manual control following automation failure is acceptable because the task analysis provides a more detailed evaluation of failure impacts and the task analysis process introduces a design iteration that verifies the initial function allocation is acceptable after the workload analysis results are obtained. In addition, both the FA and TA processes input to the HSI design element. Insights from either process should help designers accomadate highworkload conditions, regardless of which process is used to conduct the load analysis. Accordingly, the staff finds that the applicants treatment of operator automation interfaces conforms to this NUREG-0711 criterion.

Criterion 9 A description of the integrated personnel role across functions and systems should be provided in terms of personnel responsibility and level of automation.

The Staffs Evaluation of Criterion 9 The integration of personnel action and automation is achieved by the FA process outlined in MUAP-13007, Section 4.6, Function allocation. The process uses selected FA characteristics in the following manner:

• Load, time available, and rate are the characteristics used to control staff overload and underload by directing toward automation during a time of heavy workload and toward humans (manual) during low workload.

• Time available, rate, action logic complexity, and decision type complexity differentiate human and machine strength and capabilities.

Each characteristic is discussed within Section 4.6 and instructions are provided on who is responsible for assessing each characteristic, what the assessment is to consider, how the characteristic is rated, and how the ratings are processed to reach a conclusion.

18-33

In summary, the rated values for each characteristic are computed for all success paths for both power production and safety high-level functions and for all plant modes and conditions. As each success path is evaluated an aggregate analysis is completed that identifies the impact on the characteristic from other high-level functions related to the target success path. That is, each characteristic is determined for individual success paths and across success paths. This provides the integration of personnel actions within the assessment. Individual characteristic ratings and aggregate ratings are then compared to procedural guidelines to determine the final FA disposition and the final FA disposition is confirmed using expert judgement.

The staff concludes that the FA process provides both necessary and sufficient direction to ensure the integration of human action with automation in a manner that provides reasonable assurance that safety functions are appropriately controlled. The subsequent Task Analysis and Integrated System Validation provides additional assurance that the function allocations are appropriate. Accordingly, the staff finds that the applicants treatment of the integration of operator action and automation conforms to this NUREG-0711 criterion.

Criterion 10 The functional requirements analysis and function allocation should be verified:

• all the high-level functions necessary for the achievement of safe operation are identified,

• all requirements of each high-level function are identified, and

• the allocations of functions result in a coherent role for plant personnel.

The Staffs Evaluation of Criterion 10 As stated in MUAP-13007, Section 4.10, Verification, verification of the FRA/FA is performed by independent plant operations SME reviews and by completing the activities described in MUAP-13009, Task Analysis Implementation Plan, and MUAP-10012, Human Factors Verification and Validation Implementation Plan. Discrepancies identified by the SMEs are resolved through discussions between those who performed the FRA/FA and the independent reviewer. In addition, the FRA/FA team, which is composed of plant operations, HFE, I&C engineering, systems engineering, PRA, safety system engineering members, further review those cases where discrepancies between the performer and the reviewer need resolution.

The staff concludes that the FRA/FRA results are independently verified and any discrepancies are resolved. Accordingly, the staff finds that the applicants treatment of FRA/FA verification conforms to this NUREG-0711 criterion.

18.3.5 Combined License Information Items There are no COL information items related to this area of review. The staff determined that no COL information items need to be included in DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for HFE functional requirements analysis and function allocation.

18-34

18.3.6 Conclusions The staff evaluated FRA and FA with respect to HFE, at an IP level using the review criteria in NUREG-0711, Section 4.4, Review Criteria. Section 18.0.4 of this report provides a discussion of review levels. The staff determined that the FRA/FA methodology that is described in the US-APWR documentation above sufficiently defines the plants functions that must be performed to satisfy plant safety objectives, and sufficiently describes how the allocation of those functions to human and system resources has resulted in a role for personnel that takes advantage of human strengths and avoids human limitations.

Therefore, the staff concludes that HFE considerations, with respect to FRA and FA, have been adequately addressed, and that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47 related to this technical area are satisfied.

18.4 Task Analysis 18.4.1 Introduction Task analysis is the analysis of human actions resulting from the function allocation and the identification of HSI design characteristics needed to support personnel task accomplishment.

The objective of the staffs review is to ensure that the applicant's task analysis identifies the specific tasks that are needed for function accomplishment and their associated information, control, and task support requirements.

18.4.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Tier 1, Section 2.9 and Table 2.9-1.

DCD Tier 2: The applicant provided a Tier 2 description in Section 18.4 that describes the method used to conduct a task analysis that identifies the control room inventory and evaluates the workload.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

Topical Reports: There are no topical reports associated with this element.

Technical Reports: Technical Reports associated with DCD Tier 2, Section 18.4 are as follows:

• MUAP-13009, US-APWR Task Analysis Implementation Plan, Revision 1, issued May 2014.

• MUAP-09019, US-APWR Human Factors Engineering (HFE) Program Management Plan Revision 5, issued August 2014.

• MUAP-10008, US-APWR Staffing and Qualifications Implementation Plan, Revision 4, issued May 2014 (ML14164A609).

18-35

• MUAP-13007, US-APWR Functional Requirements Analysis and Function Allocation Implementation Plan, Revision 1, issued May 2014.

18.4.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

• 10 CFR 50.34(f)(2)

• 10 CFR 52.47 Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 5, Task Analysis, Section 5.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering.

18.4.4 Technical Evaluation Criterion 1 The scope of the task analysis should include:

• selected representative and important tasks from the areas of operations, maintenance, test, inspection, and surveillance,

• full range of plant operating modes, including startup, normal operations, abnormal and emergency operations, transient conditions, and low-power and shutdown conditions,

• human actions that have been found to affect plant risk by means of PRA importance and sensitivity analyses should also be considered risk-important.

Internal and external initiating events and actions affecting the PRA Level I and II analyses should be considered when identifying risk-important actions, and

• where critical functions are automated, the analyses should consider all human tasks including monitoring of the automated system and execution of backup actions if the system fails.

The Staffs Evaluation of Criterion 1 MUAP-13009, Section 2.0, Scope, incorporates each of the four bullets of Criterion 1 above.

Therefore the task analysis scope is consistent with the guidance. Accordingly, the staff finds that the applicants treatment of task analysis scope, conforms to this NUREG-0711 criterion.

18-36

Criterion 2 Tasks should be linked using a technique such as operational sequence diagrams (OSD). Task analyses should begin on a gross level and involve the development of detailed narrative descriptions of what personnel have to do.

The analyses should define the nature of the input, process, and output needed by and of personnel. Detailed task descriptions should address (as appropriate) the topics listed in Table 5.1 [Task Considerations] of NUREG-0711.

The Staffs Evaluation of Criterion 2 MUAP-13009, Section 4.0, Methodology, describes a three step task analysis process. The task analysis process begins at a general level with a basic task analysis which develops a task narrative, HSI inventory table, and task evaluation for each task included in the task analysis scope.

Section 4.1, Task Selection, describes the selection of tasks to be used as inputs to the process. It includes all procedurally defined operational tasks as well as a selection of other reasonable and necessary tasks (such as inspections, tests, maintenance tasks, and important human actions).

MUAP-13009, Section 4.2.1, Basic Task Analysis - Task Narrative, provides a list of proprietary topics that are addressed in the narrative. The list includes elements that address the objectives of NUREG-0711 including inputs from the operating experience, function allocation, and Important Human Action elements. As a group, the topics address how, when, where, and why tasks are performed and identify the controls, alarms and indications needed to support task performance.

MUAP-13009, Section 4.2.3, Basic Task Analysis - Task Evaluation, provides a set of proprietary review criteria which assess whether or not performance is likely to be affected by other factors such as type of action, timing constraints, staffing, and operating experience.

Those tasks meeting the criteria receive a detailed Task Analysis (TA).

MUAP-13009, Section 4.3, Detailed Task Analysis, describes the detailed task analysis process. If a detailed TA is needed, the operator workload and the margin between time required and time available for each subtask are determined based on quantitative analytical methods. Workload is evaluated by a comparison of the time engaged and the time available for a task. The time engaged includes an assessment of the physical and cognitive action times the operator expends in performing the task. It is based on operational sequence diagram results for the primary task, adjusted for expected secondary tasks and other factors including decision-making, communications, workplace factors and hazards, task support requirements, and situational and performance-shaping factors. The detailed TA ensures a workload that leaves the operator with adequate mental resources to maintain overall plant-level situational awareness. If not, an HED is generated. The elements from Table 5.1, Task Considerations, of this acceptance criterion (each is discussed in detail below) are incorporated within this TA.

Task Considerations from NUREG-0711, Table 5.1 The considerations listed on NUREG-0711, Table 5.1 are typically addressed as part of the Detailed Task Analysis process (exceptions are noted below). MUAP-13009, Section 4.2.3, Basic Task Analysis - Task Evaluation, describes 12 selection criteria to be used to initiate 18-37

this process. The process requires a detailed task analysis for any task that fails to satisfy any of the 12 selection criteria. The selection criteria provide an appropriate method to prompt further analysis of the task considerations listed on NUREG-0711, Table 5.1.

The treatment of each of the items in Table 5.1 is described below.

1. Information Requirements - The Basic TA process identifies the controls, alarms and indications needed to accomplish all tasks within the scope of the TA process.

2. Decision-making Requirements - Section 4.2.1, Basic Task Analysis - Task Narrative, documents decision-making associated with a task in support of the detailed task analysis if it is needed. Section 4.3.5, Detailed Task Analysis Output, provides direction on determining alarms, indications or other information needed to support the decision-making.

3. Response Requirements - Section 4.3.2.1(1), Operational Sequence Diagram

[OSD] Time, uses the OSD to provide information regarding types of tasks, time available, and temporal constraints. OSD times are based initially upon analytic methods and then validated and revised, if necessary, based upon simulator data. Section 4.3.2.1(2), Task Characterization Times, expands upon the OSD times by considering the impact of task related factors such as decision-making, communications, and performance shaping factors. SME judgment is used to add additional time to the estimates where appropriate.

4. Communication Requirements - Section 4.2.1, Basic Task Analysis - Task Narrative, specifies that typically three-way communications will be used. Any deviations from this will be described in the task narratives which are part of the Basic Task Analysis. Additionally, Section 4.3.5 Detailed Task Analysis Output, describes how the OSD will be used to estimate communication time. This section classifies different types of communications between various operators.

5. Workload - The proposed method uses a common method of workload estimation by comparing the estimates to an appropriate standard. Workload conditions that exceed the standard will generate HEDs. Low workload conditions are addressed during the staffing and qualifications assessment which also generates HEDs if the workload is below a threshold and meets particular selection criteria.

Specifically, MUAP-13009, Section 4.3.3, Workload Assessment, defines workload by comparing the time needed to execute a task to the time available for the operator to complete the task. This workload estimate is then compared to standards in MIL-HDBK-46855A Department of Defense Handbook, Human Engineering Program Process and Procedures, which provides guidance regarding acceptable workloads. The equations presented include weighting factors to account for complexity added by concurrent tasks. Section 4.3.2.2(1),

Concurrent Administrative Workload, describes a weighting factor used to estimate tasks such as answering phones that are not necessarily related to primary task completion. Additionally, Section 4.3.2.2(2), Concurrent Critical Function Workload, describes a second weighting factor used to consider additional plant related control and monitoring tasks. These weighting factors are 18-38

based, in part on the FRA/FA results. The results were validated in a task analysis pilot study; however the results of the pilot study are not included in the IP. In RAI 7311, Question 18-263 ADAMS Accession No. ML14030A426), the staff questioned the validity of the process and the results of the pilot study.

In its response to RAI 7311, Question 18-263, dated February 28, 2014 (ADAMS Accession No. ML14069A022), the applicant provides details regarding the pilot testing including information about the processes used, the test subjects, and the simulation used. The response also contains an explanation of the weighting factors used. These weights were not intended to produce precise time values; rather they are used to add additional time to conservative baseline measures.

This added time produces additional conservatism in the time estimates (time estimates can only increase from the baseline and cannot be reduced).

Moreover, any tasks that are greater than or equal to the limits described in MIL-HDBK-46855A, will trigger additional analysis. The V&V process is used to provide additional confirmation of these estimates.

6. Task Support Requirements - Section 4.2.1, Basic Task Analysis - Task Narrative, states that the task support requirements are described as part of the Basic TA process.

7. Workplace Factors - Section 4.3.5, Detailed Task Analysis Output, assumes a 15 minute transit time between two points in the plant as a default value. This value will be revised as the design develops and exceptions to the 15 minute time can be made as necessary. The basis for the 15 minute worst case transit time is not explained in the IP. RAI 7405, Question 18-265 (ADAMS Accession No. ML14035A511), was issued to clarify the basis for this assumption.

In its response to RAI 7405, Question 18-265, dated March 6, 2014 (ADAMS Accession No. ML14070A211), the applicant states that the 15 minute worst case time is based upon the average walking speed of humans and describes how the process adds time to the baseline estimate as necessary to refine the initial estimate. This was confirmed as a reasonable estimation by SMEs.

However, this baseline estimate only accounts for the walking time from location to location. Factors that may affect the transit time are then considered using the methods described in Subsection 4.3.5, Detailed Task Analysis Output. These factors are used to adjust the time estimates as necessary.

In addition to transit time, Section 4.3.5 contains a commitment to evaluate ingress and egress paths, workspace envelope, as well as many other reasonable workplace factors not specifically indicated in the criterion.

8. Situational and Performance Shaping Factors - Section 4.3.2.1(2), describes how situational and performance shaping factors are used as a weighting factor within the workload analysis. Use of the weights reduces the possibility that estimates will systematically underestimate the task workloads that are subject to situation and performance shaping factors.

Additionally, Section 4.3.5, Detailed Task Analysis Output, addresses many considerations related to physical and physiological stresses which may 18-39

influence human reliability. The examples considered include the examples in Table 5-1, TA Implementation Summary, as well as several others.

9. Hazard Identification - Section 4.2.1 Basic Task Analysis - Task Narratives, indicates that hazards will be identified as part of the narratives developed in the Basic TA process. These hazards will then be considered within the scope of the time estimations considered in the workload analysis (as described in Section 4.3.2.1(2), Task Characterization Times, paragraph 1). This is further elaborated upon in Section 4.3.5, Detailed Task Analysis Output, which addresses hazard identification within the detailed TA.

The staff concludes that the operator tasks are described in detail and there is a systematic, clearly described process for determining those tasks deserving additional evaluation. The criteria to be addressed within the detailed TA are complete and the scope of the analysis is clearly defined in quantitative terms. Accordingly, the staff finds that the applicants treatment of task analysis conforms to this NUREG-0711 criterion.

Criterion 3 The task analysis should be iterative and become progressively more detailed over the design cycle. It should be detailed enough to identify information and control requirements to enable specification of detailed requirements for alarms, displays, data processing, and controls for human task accomplishment.

The Staffs Evaluation of Criterion 3 MUAP-13009, Section 4.0, Methodology, describes the TA as beginning with preliminary design documents and continuing through various stages of the design cycle up to and including the use of a dynamic training simulator (which would only be available later in the design process). Also, the methodology described in Section 3.0, Methodology Overview, includes a movement from the basic TA to a detailed TA for some tasks. Together these two points describe a process that is consistent with the wording in the criterion which says become progressively more detailed over the design cycle.

Section 4.2.2, Basic Task Analysis - HSI Inventory, describes the expected details that should come out of the basic TA process. These details are necessary for the specifications for alarms, display, data processing and controls referred to in the criterion.

The staff finds that the TA process is an iterative process that supports the increasing level of detail needed in the design cycle. Accordingly, the staff finds that the applicants treatment of the iterative TA conforms to this NUREG-0711 criterion.

18-40

Criterion 4 The task analysis should address issues such as:

• the number of crew members,

• crew member skills, and

• allocation of monitoring and control tasks to the (a) formation of a meaningful job and (b) management of crew members physical and cognitive workload.

The Staffs Evaluation of Criterion 4 MUAP-13009, Section 4.2.1, Basic Task Analysis - Task Narrative, identifies the number of crew members and their qualifications as part of the basic TA. Operator skills are an inherent part of qualification and are defined in [American National Standards Institute/American Nuclear Society] ANSI/ANS 3.1, Selection, Qualification, and Training of Personnel for Nuclear Power Plants, as endorsed by Regulatory Guide (RG) 1.8, Qualification and Training of Personnel for Nuclear Power Plants. Specialized performance requirements such as time constraints, decision making, communications, and situation specific factors are identified in other elements of the basic TA. This information is an input to the Staffing and Qualifications Assessment which further validates the staffing levels and the knowledge, skills, and abilities for each qualification level.

MUAP-13009, Section 4.1, Task Selection, states that tasks originate from operating procedures and surveillance, test, inspection, and maintenance activities. By virtue of being extracted from a procedure, the task being analyzed constitutes a meaningful job. The TA identifies the sequence of necessary subtasks such as monitoring and control actions that must be accomplished to complete the task. For new design elements, the functional requirements analysis and the function allocation process assigns functions to humans or machines. In the TA, human actions are decomposed to identify all control tasks and related monitoring tasks.

These tasks represent a meaningful job because they have been derived from a common function.

Section 4.3, Detailed Task Analysis, addresses the physical and mental workload of the operators. This analysis determines the operator action times which includes times required for decision making, communications, workplace factors and hazards, task support requirements, and situational and performance-shaping factors. These factors are used to determine the timeline for operators to perform the task. Task difficulty, complexity, frequency, and accuracy are used to adjust the time line for stress induced mental workload. Detailed directions are provided within the text regarding how to integrate these areas into a workload conclusion.

The staff concludes that the TA method addresses the variables that affect the operators ability to implement a task. Accordingly, the staff finds the applicants treatment of the TA conforms to this NUREG-0711 criterion.

Criterion 5 The task analysis results should be used to define a minimum inventory of alarms, displays, and controls necessary to perform crew tasks based on both task and instrumentation and control requirements.

18-41

The Staffs Evaluation of Criterion 5 MUAP-13009, Section 4.2.2, Basic Task Analysis - HSI Inventory, lists the information provided for each task. It includes the equipment, controls, alarms, indications, and interlocks, blocks, and overrides needed to perform the task. For each indication, the range, units, resolution, refresh and update rate and display characteristics needed to support the task performance are identified. Tasks are selected so that this inventory represents the complete control room inventory. Historically, minimum inventory has been used to refer to the RG 1.97 parameters and the inventory needed to implement the Emergency Operating Procedures. This minimum inventory is a subset of the control room inventory identified by the TA. RG 1.97 parameters are also specifically evaluated by an interdisciplinary team as part of Chapter 7, Instrumentation and Controls, of this report.

The staff concludes that the TA develops a complete control room inventory along with the instrumentation specifications needed to support the operator tasks. Accordingly, the staff finds that the applicants treatment of equipment inventory conforms to this NUREG-0711 criterion.

Criterion 6 The task analysis results should provide input to the design of HSIs, procedures, and personnel training programs.

The Staffs Evaluation of Criterion 6 MUAP-13009, Section 3.1, Interfaces with Other HFE Program Elements, states that the TA results are inputs to the HSI design, procedure development and training program development (as well as other elements of the HFE program) and describes how the input is used.

Accordingly, the staff finds that the applicants treatment of TA interfaces conforms to this NUREG-0711 criterion.

18.4.5 Combined License Information Items There are no COL information items related to this area of review. The staff determined that no COL information items need to be included in DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for HFE program management consideration.

18.4.6 Conclusions The staff evaluated the HFE TA process at an IP level using the review criteria in NUREG-0711, Section 5.4, Review Criteria. The staff concludes that the TA process proposed for the US-APWR provides an analysis method that is consistent with all of the relevant review criteria.

The method provides for identifying the control room inventory and for determining that there is reasonable assurance that the operator tasks can be implemented effectively. Therefore, the staff concludes that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47, related to this technical area, are satisfied.

18-42

18.5 Staffing and Qualifications 18.5.1 Introduction The objective of the staffs review is to verify that the applicant has analyzed the requirements for the number and qualifications of personnel in a systematic manner that includes a thorough understanding of task requirements and applicable regulatory requirements. The applicants analysis described in this section serves as input to aid in the development of an adequate staffing plan for the operating crew, as well as for the organization and management supporting the operation and maintenance of the plant.

18.5.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 and Table 2.9-1.

DCD Tier 2: The applicant provided a Tier 2 description in Section 18.5, Staffing and Qualifications, [S&Q] which describes the methodology for conducting a staffing and qualifications analysis. The applicants analysis determines the number and background of personnel for the full range of plant conditions and tasks including operational tasks (normal, abnormal, and emergency), plant maintenance, and plant surveillance and testing.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

Topical Reports: There are no topical reports associated with this element.

Technical Reports: The technical reports associated with this element are:

• MUAP-10008, US-APWR Staffing and Qualifications Implementation Plan, Revision 4, issued May 2014.

• MUAP-10012, US-APWR Human Factors Verification and Validation Implementation Plan, Revision 4, issued May 2014.

18.5.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections including Chapter 13, Section 13.1.1, Management and Technical Support Organization, through Sections 13.1.2-13.1.3, Operating Organization, which specifically deal with staffing and qualifications.

• 10 CFR 50.54(i) through (m)

• 10 CFR 50.55

• 10 CFR 120 18-43

Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 6, Staffing and Qualifications, Section 6.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering.

• SECY-05-197, Review of Operational Programs in a Combined License Application and Generic Emergency Planning Inspections, Tests, Analyses, and Acceptance Criteria, issued February 2006.

• RG 1.206, Combined License Applications for Nuclear Power Plants

[Light-Water Reactor] LWR Edition), Section C.IV.4, Operational Programs.

18.5.4 Technical Evaluation The staff performed an IP level review as described in NUREG-0711 and Section 18.0.4 of this report.

This section presents the applicable review criteria from NUREG-0711 followed by an evaluation of each.

The plant staff and their qualifications are important considerations throughout the design process. Initial staffing levels may be established based on the experience with previous plants, staffing goals (such as for staffing reductions), initial analyses, and government regulations.

Many plant staff actions require teamwork and communication among control room staff, auxiliary operators, and other plant staff. The NRC staff reviewed the applicant's analysis using the review criteria in this element to determine the staffing requirements for accomplishing these actions.

Criterion 1 Staffing and qualifications should address applicable guidance in NUREG-0800, Section 13.1 and 10 CFR 50.54.

The Staffs Evaluation of Criterion 1:

See Section 13.1 of this SER for an evaluation of the US-APWR using the guidance in SRP Section 13.1.

The 10 CFR 50.54 (m)(2)(i) specifies minimum control room staffing levels. The US-APWR staffing levels are specified in MUAP-10008, US-APWR Staffing and Qualifications Implementation Plan, as:

• One SRO located at the plant fulfilling the role of Shift Manager.

18-44

• One SRO located within the MCR fulfilling the role of MCR Supervisor and (STA)

[Shift Technical Advisor].

• One additional SRO or STA at the plant.

• One RO located at the controls of the plant in the MCR.

• One RO located at the plant.

The applicant also provided a description of the roles for these personnel.

The staff finds that this minimum staffing list for the US-APWR complies with the criteria in 10 CFR 50.54(m)(2)(i). The addition of a dual role STA in the plant allows the MCR Supervisor to focus on his assigned responsibilities during emergencies. Accordingly, the ataff finds that the applicants treatment of staffing levels conforms to this NUREG-0711 criterion.

Criterion 2 The staffing analysis should determine the number and background of personnel for the full range of plant conditions and tasks including operational tasks (normal, abnormal, and emergency), plant maintenance, and plant surveillance and testing. The scope of personnel that should be considered is identified in the HFE Program Management element (see Section 2.4.1, Criterion 5).

The Staffs Evaluation of Criterion 2 MUAP-10008, Section 2.0, Scope, states that the staffing analysis will determine the number and background of personnel for the full range of plant conditions and tasks including operational tasks (normal, abnormal, and emergency), plant maintenance, and plant surveillance and testing.

MUAP-10008, Section 3.0, Methodology Overview, establishes a qualification and staffing baseline level for both operating and plant personnel.

The operating personnel baseline is derived from previous HFE program element inputs. To establish the final US-APWR staffing and qualifications, the baseline is evaluated using a broad sampling of scenarios. The ability of the baseline operating crew to manage each scenario is compared to a crew from a predecessor plant for the same scenario, with consideration of US-APWR plant and HSI design differences. The comparison is conducted for multiple scenarios during normal, abnormal and emergency plant conditions within each plant mode. The scenarios encompass the staffing related issues identified by the OER, all actions identified by the HRA, and a broad sampling of specific actions and tasks selected from the FRA/FA and TA using a selection method that considers both high and low-workload conditions. This evaluation provides a diverse method to validate the inputs from the previous HFE program elements.

The plant personnel baseline is derived from a predecessor plant. The staffing baseline includes I&C technicians, electrical maintenance personnel, mechanical maintenance personnel, radiological protection technicians, chemistry technicians, and engineering support personnel. This is the same group of people identified in the HFE Program Management element, Section 2.4.1, General HFE Program Goals and Scope, Criterion 5. The qualification 18-45

baseline is established by ANSI/ANS 3.1, Selection, Qualification, and Training of Personnel for Nuclear Power Plants and ANSI N18.71976/ANS-3.2, Administrative Controls and Quality Assurance for the Operational Phase of Nuclear Power Plants. SMEs evaluate staffing related issues for non-operations positions that were previously identified in the OER program element.

This includes examination of OER issues that were assumed to be resolved by the US-APWR plant design or US-APWR local control station designs, to confirm that the assumptions remain correct. The S&Q also ensures that any OER HEDs pertaining to non-operations positions are resolved. The SMEs adjust the baseline considering differences in the US-APWR plant design compared to predecessor U.S. four-loop PWR plants.

The staff concludes that the methods used for establishing staffing and qualification levels addresses the full range of plant conditions and activities. It appropriately uses the input from the previous HFE program elements. Accordingly, the ataff finds that the applicants treatment of staffing levels conforms to this NUREG-0711 criterion.

Criterion 3 The staffing analysis should be iterative; that is, initial staffing goals should be reviewed and modified as the analyses associated with other elements are completed.

The Staffs Evaluation of Criterion 3 MUAP-10008, Section 4.2, Operating Staff Baseline Input from other HFE Program Elements, describes the process used to provide input to the S&Q process from other HFE elements.

Minimum staffing is used as a design constraint for the HFE elements. Where an issue arises regarding the minimum staffing constraint, a HED is documented and then resolved prior to further S&Q evaluations. MUAP-10008, Section 4.3, Staffing and Qualifications Evaluation, describes the process that is used to further evaluate the staffing baseline for both operations and non-operations staff, and modify them as appropriate. During this phase, the staffing level and qualifications goals are compared to predecessor plants. The evaluation includes a full range of plant conditions, all plant operating modes, secondary tasks and interruptions.

The staff concludes that, even though the staffing level is used as a design constraint, the S&Q methodology provides for the modification of the staffing goals via the HED resolution process.

Accordingly, the staff finds that the applicants treatment of staffing levels conforms to this NUREG-0711 criterion.

Criterion 4 The basis for staffing and qualifications should be modified to address these issues:

• Operating Experience Review

- operational problems and strengths that resulted from staffing levels in predecessor systems

- initial staffing goals and their bases including staffing levels of predecessor systems and a description of significant similarities and differences between predecessor and current systems 18-46

- staffing considerations described in NRC Information Notice 95-48, "Results of Shift Staffing Study"

- staffing considerations described in NRC Information Notice 97-78, "Crediting of Operator Actions in Place of Automatic Actions and Modifications of Operator Actions, Including Response Times"

• Functional Requirements Analysis and Function Allocation

- mismatches between functions allocated to personnel and their qualifications

- changes the roles of personnel due to plant system and HFE modifications

• Task Analysis

- the knowledge, skills, and abilities needed for personnel tasks addressed by the task analysis

- personnel response time and workload

- personnel communication and coordination, including interactions between them for diagnosis, planning, and control activities, and interactions between personnel for administrative, communications, and reporting activities

- the job requirements that result from the sum of all tasks allocated to each individual both inside and outside the control room

- decreases in the ability of personnel to coordinate their work due to plant and HFE modifications

- availability of personnel considering other activities that may be ongoing and for which operators may take on responsibilities outside the control room (e.g., fire brigade)

- actions identified in 10 CFR 50.47, NUREG-0654, and procedures to meet an initial accident response in key functional areas as identified in the emergency plan

- staffing considerations described by the application of ANSI/ANS 58.8-1994, "Time Response Design Criteria for Safety-Related Operator Actions"

• Human Reliability Analysis

- the effect of overall staffing levels on plant safety and reliability 18-47

- the effect of overall staffing levels and crew coordination for risk-important activities

- the effect of overall staffing levels and the coordination of personnel on human errors associated with the use of advanced technology

• HSI Design

- staffing demands resulting from the locations and use (especially concurrent use) of controls and displays

- coordinated actions between individuals

- decreases the availability or accessibility of information needed by personnel due to plant system and HFE modifications

- the physical configuration of the control room and control consoles

- the availability of plant information from individual workstations and group-view interfaces

• Procedure Development

- staffing demands resulting from requirements for concurrent use of multiple procedures

- personnel skills, knowledge, abilities, and authority identified in procedures

• Training Program Development

- crew coordination concerns that are identified during the development of training The Staffs Evaluation of Criterion 4 MUAP-10008, Section 4.2, Operating Staff Baseline Input from other HFE Program Elements, describes how the analysis elements of the HFE program (OER, FRA/FA, TA, and HRA) act as inputs to the S&Q analysis. The S&Q analysis, in turn, contributes to the HSI design, procedure development and training development. MUAP-10008, Sections 4.2, and 4.3, Staffing and Qualifications Evaluation, provide the details of how this is accomplished. Sections 4.2 and 4.3 also address the specific issues for each program element noted in the acceptance criterion.

Any identified issues will be evaluated and resolved per the HED evaluation process. This HED resolution process may include modification to the staffing level or staff qualifications requirements.

The staff concludes that the applicant has provided a basis for modifying the staffing assumptions to address inputs from the OER, FRA/FA, TA, HRA, HSI design, procedures development, and training development. Accordingly, the staff finds that the applicants treatment of S&Q inputs conforms to this NUREG-0711 criterion.

18-48

18.5.5 Combined License Information Items There are no COL information items related to this area of review. The staff determined that no COL information items need to be included in DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for HFE staffing and qualifications consideration.

18.5.6 Conclusions The staff evaluated S&Q at an IP level using the review criteria in NUREG-0711, Section 6.4, Review Criteria. Section 18.0.4 of this report provides a discussion of review levels. The staff concludes that the staffing levels and qualifications established as a baseline conform to regulation and regulatory guidance. The analysis used to verify the baselines is conservative and addresses the full range of plant conditions and activities. Therefore, the staff concludes the S&Q considerations, with respect to HFE, have been adequately addressed, and that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47 related to this technical area are satisfied.

18.6 Human Reliability Analysis 18.6.1 Introduction Human Reliability Analysis identifies risk significant human actions important to plant safety.

The objective of the staffs review is to verify that the applicant has considered these actions in designing the HFE aspects of the plant to minimize the likelihood of personnel error, and to help ensure that personnel can detect and recover from any errors that occur.

18.6.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9.

DCD Tier 2: The applicant provided a Tier 2 system description in Section 18.6, which describes an iterative process for incorporating the results of the HRA into the HFE design process, with the objective of minimizing personnel errors, allowing detection of human errors, and providing capability for recovery following human errors.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

Topical Reports: There are no topical reports associated with this element.

Technical Reports: The technical reports associated with DCD Tier 2, Section 18.6, Human Reliability Analysis, are as follows:

• MUAP-07030, US-APWR Probabilistic Risk Assessment, Revision 3, issued June 2011.

• MUAP-07014, Defense-in-Depth and Diversity Coping Analysis, Revision 5, issued September 2011 (ADAMS Accession No. ML11286A341).

18-49

• MUAP-09019, US-APWR Human Factors Engineering Program Management Plan, Revision 5, issued August 2014.

• MUAP-10009, US-APWR Human-System Interface Design Implementation Plan, Revision 4, issued May 2014.

• MUAP-10012, US-APWR Human Factors Verification and Validation Implementation Plan, Revision 4, issued May 2014.

• MUAP-13007, US-APWR Functional Requirements Analysis and Function Allocation Implementation Plan, Revision 1, issued May 2014.

• MUAP-13009, US-APWR Task Analysis Implementation Plan, Revision 1, issued May 2014.

• MUAP-13014, US-APWR Human Reliability Analysis Implementation Plan, Revision 1, issued May 2014 (ADAMS Accession No. ML14164A597).

18.6.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria, which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

• 10 CFR 50.34(f)(1)(i)

• 10 CFR 52.47(b)(1)

• 10 CFR 50.54(i) through (m)

Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 7, Human Reliability Analysis, Section 7.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18.II.A.1, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering.

18.6.4 Technical Evaluation The staff performed a complete element level review as described in NUREG-0711 and Section 18.0.4 of this report. The risk important human actions have been identified and are incorporated into the other elements of the HFE design process.

This section presents the applicable review criteria from NUREG-0711 followed by an evaluation of each.

18-50

Criterion 1 Risk-important human actions should be identified from the PRA/HRA and used as input to the HFE design effort.

• These actions should be developed from the Level 1 (core damage) PRA and Level 2 (release from containment) PRA including both internal and external events. They should be developed using selected (more than one) importance measures and HRA sensitivity analyses to provide reasonable assurance that an important action is not overlooked because of the selection of the measure or the use of a particular assumption in the analysis.

The Staffs Evaluation of Criterion 1 US-APWR DCD, Section 18.6, Human Reliability Analysis, states that the incorporation of HRA/PRA results into the HSI design process involves identifying risk-important human actions (HAs), which are extracted from the Level 1 PRA and Level 2 PRA (including both internal and external events). US-APWR DCD, Section 19.1.4.1.1, Description of the Level 1 PRA for Operations at Power, describes the HRA process used within the PRA, and Section 19.1.4.1.2, Results from the Level 1 PRA for Operations at Power, describes the results of the PRA/HRA in Tables 19.1-34 Human Error FV Importance; 35 Human Error RAW; 48 Human Error FV importance for LRF; 49 Human Error RAW for LRF [large release frequency]; 66 Human Error FV Importance for Fire; 67 Human Error RAW for Fire; 75 Human Error FV Importance for Flood; 76 Human Error RAW for Flood; 97 Human Error FV Importance of POS 8-1 for LPSD

[low power and shutdown] PRA; and 98 Human Error RAW of POS 8-1 for LPSD PRA.

These tables list HAs with the highest Fussell-Vesely (FV) and Risk Achievement Worth (RAW) importance measures for the various PRA analyses: Level 1 internal events core damage frequency, Level 2 internal events large release frequency, external events fire and flood, and low power/shutdown PRA. The seismic PRA for the US-APWR uses a seismic margins analysis which takes no credit for any HA, thus no actions are added to the list of risk-important HAs from this analysis. The list contains 43 risk-important HAs, including HAs with different levels of dependency, HAs from the Level 1 and Level 2 PRA, HAs from the internal and external events PRA, and HAs from the low power and shutdown PRA.

MUAP-07030, Section 9, Human Reliability Analysis, of the US-APWR PRA contains the HRA.

Tables 9.2.1-1, The List of Type A Human Failure Events, and 9.2.3-1, The List of Type C Human Failure Events, contain a listing of all human failure events, together with their human error probabilities. MUAP-07030, Section 9.4, Assessment of Dependency between Human Failure Events, provides the dependency analysis. For each risk-important human action (RIHA), the HFE characteristics assumed in the PRA are identified and associated with the RIHA in a MHI internal document so they are readily available to subsequent HFE program elements.

Based on the application submittal date, the applicant is committed to NUREG-0711, Revision 2, which addresses risk important HAs. Revision 3 adds deterministically important Has, which the applicant has elected to incorporate into the HFE design method. MUAP-13014, Section 2.0, Scope, describes the methodology for extracting these actions from the transient and accident analysis in DCD, Chapter 15, Transient and Accident Analysis, and the diversity and defense-in-depth coping analysis described in MUAP-07014, Defense-in-Depth and Diversity Coping Analysis, which is referenced in DCD, Chapter 7. The risk-important HAs may overlap some of the deterministically important HAs.

18-51

The staff concludes that the important HAs have been appropriately identified from both the Level 1 and 2 PRAs and the transient and accident analyses. The ataff finds that the applicants use of the two proposed importance measures, Fussell-Vesely and Risk Achievement Worth, acceptable. These measures have a sufficiently low threshold that risk important HAs important to safety will be identified. Accordingly, the ataff finds that the applicants treatment of important HAs conforms to this NUREG-0711 criterion.

Criterion 2 Risk-important HAs and their associated tasks and scenarios should be specifically addressed during function allocation analyses, task analyses, HSI design, procedure development, and training. This will help verify that these tasks are well supported by the design and within acceptable human performance capabilities (e.g., within time and workload requirements).

The Staffs Evaluation of Criterion 2 MUAP-13014, Section 3.0, Methodology Overview, provides an overview of the full HFE program as it relates to HRA. This section describes the integrating role of the HRA in the HFE program. It also illustrates the use of important HAs in the HFE design process for every element in NUREG-0711. The HFE program is part of the ongoing design process, and changes in the design occur in an iterative fashion. This helps ensure that the important HAs can be addressed in each of the design stages. The PRA and HRA are not considered complete until late in the design process, which allows incorporation of design changes made using the iterative design process into the PRA.

The staff concludes that the applicant has defined how important HAs are addressed in each NUREG-0711 element. Accordingly, the ataff finds that the applicants treatment of HRA interfaces conforms to this NUREG-0711 criterion.

Criterion 3 The use of PRA/HRA results by the HFE design team should be specifically addressed; that is, how are risk-important HAs addressed (through HSI design, procedural development, and training) under the HFE program to minimize the likelihood of operator error and provide for error detection and recovery capability.

The Staffs Evaluation of Criterion 3 The staffs evaluation of important HAs, in each NUREG-0711 element other than FRA/FA, can be found in the respective sections of this report. The staffs evaluation of important HAs within the FRA/FA element is documented below.

MUAP-13014, Section 4.4, Treatment of IHAs during FRA/FA, provides an overview of the process to incorporate important HAs into the FRA/FA. The detailed process is described in MUAP-13007, Section 4.7.1, Important Human Actions (IHAs). In general, the important HAs (deterministic and risk-important HAs) are verified to be part of the safety function success paths developed as part of the functional requirements analysis. By inclusion in the success paths, allocations to the human or machine are made with full knowledge of the importance of 18-52

the actions within the success paths. This is acceptable, as one of the objectives for feedback of the PRA into the FRA/FA is to minimize the potential for error in the conduct of the IHAs.

The staff concludes that the important HAs are integrated into all of the HFE elements. In summary, the collective staff evaluations for each element finds that the important HAs receive specific reviews in each element that provides reasonable assurance that operator error is minimized and errors are detected. Accordingly, the staff finds that the applicants treatment of important HAs conforms to this NUREG-0711 criterion.

Criterion 4 HRA assumptions such as decision making and diagnosis strategies for dominant sequences should be validated by walkthrough analyses with personnel with operational experience using a plant-specific control room mockup or simulator. Reviews should be conducted before the final quantification stage of the PRA.

The Staffs Evaluation of Criterion 4 MUAP-13014, Section 3.3.8, Addressing IHAs during V&V, states that risk-important HAs are confirmed in the integrated system validation, and that the HRA serves as an input to the operational conditions sampling process and scenario definition.

The staff concludes that the V&V provides an acceptable validation of HRA assumptions.

Specifically, the integrated system validation provides for the testing of all important HAs in a full scope simulator environment. Factors including workload, situational factors, decision-making, and teamwork are tested under a variety of scenario conditions that provide reasonable assurance that the important HAs can be reliably implemented. This will be done prior to the final quantification stage of the PRA. Accordingly, the ataff finds that the applicants treatment of important HAs validation conforms to this NUREG-0711 criterion.

18.6.5 Combined License Information Items There are no COL information items related to this area of review. The staff determined that no COL information items need to be included in DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for HFE HRA consideration.

18.6.6 Conclusions The staff evaluated the HRA element, at a complete element level using the review criteria in NUREG- 0711, Section 7.4, Review Criteria. Section 18.0.4 of this report provides a discussion of the review levels.

The staff concludes that the important HAs are appropriately identified and integrated into the HFE design process. Human error mechanisms are adequately addressed in the HFE design.

The HFE design provides reasonable assurance that the likelihood of personnel error is minimized and that errors are detected and corrective actions are taken. The staff concludes that the HRA considerations with respect to HFE have been adequately addressed, and that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47, related to this technical area, are satisfied.

18-53

18.7 Human System Interface Design 18.7.1 Introduction The HSI design element represents the translation of function and task requirements into HSI design specifications. The objective of this review is to evaluate how HSI designs are identified and refined. The review verifies that the applicant has appropriately translated functional and task requirements to the detailed design of alarms, displays, controls, and other aspects of the HSI through the systematic application of HFE principles and criteria.

18.7.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this section is found in Tier 1, Section 2.9 and Table 2.9-1.

DCD Tier 2: The applicant provided a Tier 2 system description in Section 18.7, Human-System Interface Design, that describes the HSI design process including the translation of function and task requirements into the design of alarms, displays, and controls.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

Topical Reports: The topical reports associated with this element are:

• MUAP-07007, Human System Interface System Description, Revision 6, issued May 2014.

• PQD-HD-19005, Quality Assurance Program (QAP) Description for Design Certification of the US-APWR, Revision 6, issued October 2013.

Technical Reports: The technical reports associated with this element are:

• MUAP-09019, US-APWR Human Factors Engineering Program Management Plan, Revision 5, issued August 2014.

• MUAP-10009, US-APWR HSI Design Implementation Plan, Revision 4, issued May 2014.

• MUAP-13009, US-APWR Task Analysis Implementation Plan, Revision 1, issued May 2014.

• MUAP-07030, US-APWR Probabilistic Risk Assessment, Revision 3, issued June 2011.

• MUAP-07004, Safety I&C System Description and Design Process, Revision 8, issued November, 2013 (ADAMS Accession No. ML13226A841).

• MUAP-07005, Safety System Digital Platform - MELTAC, Revision 9, issued December 2013 (ADAMS Accession No. ML13350A498).

18-54

18.7.3 Regulatory Basis The SRP identifies the relevant Commission regulations for the HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

• 10 CFR 50.34(f)(2)

Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 8, Human-System Interface Design, Section 8.4, Review Criteria.

• NUREG-0700, Human-System Interface Design Review Guidelines.

• Digital Instrumentation and Controls Interim Staff Guidance 05 (DI&C-ISG-05),

Task Working Group No. 5: Highly Integrated Control Rooms - Human Factors Issues, Revision 1, issued November 3, 2008, Chapter 2, Minimum Inventory.

• Safety parameter display system requirements, as described in 10 CFR 50.34(f)(2)(iv), NUREG-0835, NUREG-1342, and Supplement 1 of NUREG-0737.

• Bypassed and inoperable status indication for nuclear power plant (NPP) safety systems, as described in RG 1.47.

• Instrumentation for light-water-cooled nuclear power plants to access plant and environmental conditions during and following an accident, as described in RG 1.97.

• Functional criteria for emergency response facilities, as described in NUREG-0696.

18.7.4 Technical Evaluation The staff performed an IP level review as described in NUREG-0711 and Section 18.0.4 of this report. This section presents the applicable review criteria from NUREG-0711 (reproduced below) followed by an evaluation of each criterion.

HSI design review topics include the following:

• HSI design inputs

• concept of operations

• functional requirement specification

• HSI concept design

• HSI detailed design and integration 18-55

• HSI tests and evaluations

- Tradeoff evaluations

- Performance-based tests

• HSI design documentation The staff reviewed the proposed SPDS using the requirements of 10 CFR 50.34(f)(2)(iv) and the criteria set forth in NUREG-0711 and NUREG-0700, Section 5, Human-System Interface Design Review Guidelines, issued May 2002. The SPDS design is evaluated in Section 18.7.4.7 of this report.

Within this element, Topical Report MUAP-07007, Human System Interface System Description, which describes the US-Basic HSI System design is frequently referenced. The US-Basic HSI System design is the foundation for the US-APWR HFE design and was approved for application to the US-APWR HFE design in a staff SER. In general, MUAP-07007 describes the HFE design of the HSIs that are used as the foundation for more specific applications, such as the US-APWR. When the US-Basic HSI System design is referenced within specific applications (e.g., US-APWR), it is supplemented with design-specific content derived from NUREG-0711. More specifically, MUAP-07007 provides the functional design for the HSIs, a HFE style guide, and control room layout. The NUREG-0711 analyses identify the control room inventory (controls, displays and alarms) that that are displayed on the HSIs as well as identifying and evaluating important HAs, assessing workload, and providing a final V&V to ensure integrated control room functionality.

In the evaluation below, many acceptance criteria are addressed by MUAP-07007 and have been evaluated in the SER associated with that topical report. For clarity, the staffs evaluation of MUAP-07007 is summarized within this report.

18.7.4.1 Human-System Interface Design Inputs Criterion 1 Analysis of Personnel Task Requirements - The analyses performed in earlier stages of the design process should be used to identify requirements for the HSIs. These analyses include:

• Operational experience review - Lessons learned from other complex human-machine systems, especially predecessor designs and designs involving similar HSI technology should be used as an input to HSI design.

• Functional requirement analysis and function allocation - The HSIs should support the operators role in the plant, e.g., appropriate levels of automation and manual control.

• Task analysis - The set of requirements to support the role of personnel is provided by task analysis. The task analysis should identify:

- Tasks that are necessary to control the plant in a range of operating conditions for normal through accident conditions; 18-56

- Detailed information and control requirements (e.g., requirements for display range, precision, accuracy, and units of measurement);

- Task support requirements (e.g., special lighting and ventilation requirements); and

- Risk-important HAs and their associated performance shaping factors, as identified through HRA should be given special attention in the HSI design process.

• Staffing/qualifications and job analyses - The results of staffing/qualifications analyses should provide input for the layout of the overall control room and the allocation of controls and displays to individual consoles, panels, and workstations. They establish the basis for the minimum and maximum number of personnel to be accommodated and requirements for coordinating activities between personnel.

The Staffs Evaluation of Criterion 1 MUAP-10009, Section 3.3, HSI Design Input, lists the HSI design inputs identified in the acceptance criterion and provides additional detail describing the expected design inputs from each area and how they are integrated into the control room design. Accordingly, the ataff finds that the applicants treatment of task requirements conforms to this NUREG-0711 criterion.

Criterion 2 System Requirements - Constraints imposed by the overall instrumentation and control (I&C) system should be considered throughout the HSI design process.

The Staffs Evaluation of Criterion 2 System constraints are addressed in the US-Basic HSI System described in MUAP-07007. In summary, the staff concluded that the constraints imposed by the I&C system, such as redundancy, equipment qualification, and coping with common mode failures are inputs for the HSI design and are considered throughout the HSI design process. In addition, MUAP-10009, Section 3.3.2, Instrumentation and Control System Designs, reiterates that the constraints continue to be considered through the US-APWR HSI design process. This section reinforces the consideration of I&C system changes needed for HED resolutions.

The integration of I&C interfaces into the HSI design process provides reasonable assurance that the HSI design will reflect the constraints imposed by the I&C system without causing undue restrictions on HED resolution. Accordingly, the staff finds that the applicants treatment of I&C system conforms to this NUREG-0711 criterion.

Criterion 3 Regulatory Requirements - Applicable regulatory requirements should be identified as inputs to the HSI design process.

18-57

The Staffs Evaluation of Criterion 3 Regulatory requirements are addressed in the US-Basic HSI System described in MUAP-07007. In summary, the MUAP-07007 list includes the applicable Code of Federal Regulations requirements as well as regulatory guidance, such as NUREG-0800, NUREG-0711, and NUREG-0700. NUREG-0696 provide guidance for HSIs in the emergency response facilities. Applicable Staff Requirement Memoranda, Branch Technical Positions (BTP), and regulatory guidance documents are also listed.

The applicant has provided a complete list of documents addressing regulatory requirements and guidance associated with the HSI design. Accordingly, the staff finds that the applicants treatment of regulatory requirements conforms to this NUREG-0711 criterion.

Criterion 4 Other Requirements - The applicant should identify other requirements that are inputs to the HSI design.

The Staffs Evaluation of Criterion 4 MUAP-10009, Section 3.3, HSI Design Inputs, identifies the US-Basic HSI System described in MUAP-07007 as the initial design input for the US-APWR HFE design. Accordingly, the staff finds that the applicants treatment of other requirements conforms to this NUREG-0711 criterion.

18.7.4.2 Concept of Operations Criterion 1 A concept of operations should be developed indicating crew composition and the roles and responsibilities of individual crew members based on anticipated staffing levels. The concept of operations should:

• Identify the relationship between personnel and plant automation by specifying the responsibilities of the crew for monitoring, interacting [with], and overriding automatic systems and for interacting with computerized procedures systems and other computerized operator support systems.

• Provide a high-level description of how personnel will work with HSI resources.

Examples of the types of information that should be identified [are] the allocation of tasks to the main control room or local control stations, whether personnel will work at a single large workstation or individual workstations, what types of information each crew member will have access to, and what types of information should be displayed to the entire crew.

• Address the coordination of crew member activities, such as the interaction with auxiliary operators, and coordination of maintenance and operations.

18-58

The Staffs Evaluation of Criterion 1 The concept of operation is addressed in the US-Basic HSI System described in MUAP-07007.

In summary, the US-Basic HSI System addresses the following subjects:

• Crew Composition.

• Roles and responsibilities of individual crewmembers.

• Personnel interaction with plant automation.

• Use of control room resources by crewmembers.

• Methods used to ensure good coordination of crewmember activities, including non-licensed operators, technicians, and maintenance personnel.

MUAP-07007, Section 4.1, Design Summary, describes the design basis for the HSI system which includes the concept of operations, as described in this criterion. The computer-based HSI system provides operational visual display units (VDUs) as the fundamental interface. The operator monitors plant status and initiates actions from the VDU by using a mouse and touching or clicking on the appropriate sections of the screen. The operators workload is significantly reduced by providing relevant process control information in integrated displays on the VDUs and utilizing a compact console that minimizes required operator movement. The HSI system also provides operational support functions that utilize the computer to consolidate large amounts of data into meaningful displays. The HSI system has the specific objective of improving overall operator performance and reducing the potential for human error.

Section 4.1 identifies the following interfaces and responsibilities:

• Using this design, the operating crew is responsible for checking the standby condition of equipment before operation, monitoring the plant parameters and identifying plant behavior during operation.

• The operating staff is responsible for alarm diagnosis, control actions, procedure execution, and monitoring auxiliary functions. These activities are supported by inter-linked screens for related tasks where the functional and/or operational relationships are inter-linked.

• The HSI design facilitates a continuous awareness of critical safety functions while immediate focus may be plant maneuvering and power production.

• A single operator can execute procedures that historically involve multiple operators to coordinate multiple safety divisions and nonsafety systems. This simplifies task coordination for maintaining critical safety functions.

• Operators can execute computer based procedures with integrated information and manual controls.

• The HSI design minimizes operator transitions between safety and nonsafety VDUs, thereby reducing the operator workload during critical plant situations.

18-59

• The HSI design provides for automatic verification of the component status which limits the operator workload and stress during plant startups, shutdowns and emergency conditions.

• The Large Display Panel (LDP) provides Spatially Dedicated Continuously Visible (SDCV) information to the operation personnel to enhance situation awareness. The LDP helps operators maintain continuous awareness of overall plant status and critical status changes, while they are engaged in operational details on a VDU display for a specific plant system or function. The secondary purpose of the LDP is to help the operations staff coordination and communication by providing a common visualization of plant information.

Control Room crew coordination is achieved through the LDP. In MUAP-07007, Section 4.9, Large Display Panel, the applicant provides a detailed description of the information provided by the LDP including fixed display area information and how it is managed based on operating conditions, the variable display area and how automatic display verses manual requests are managed, and alarm display conventions.

Coordination between operations and maintenance activities is supported by a tagging feature on the operations VDU screens (described in MUAP-07007, Section 4.5.3, Switch features) that enables operators to attach and remove tags and call up detailed tag information on the VDU screen. The tagging process provides soft electronic tags for the HSI system and physical tags for plant components. The electronic tag identification (ID) for the HSI system and the physical tag ID for plant components are identical for each component. Electronic tags are implemented within the HSI system and physical tags are attached at each component in local areas of the plant. This section of MUAP-07007 also provides additional direction for the tagging sequence.

Additional coordination support is provided by the voice communications systems described in DCD, Section 9.5.2, Communication Systems.

Crew composition is addressed in MUAP-07007, Section 4.1.6, Main Control Room Staff, and consists of one RO and one SRO in the MCR. The normal MCR staff is supplemented by one additional SRO and one additional RO that will be at the plant to accommodate unexpected conditions. While the HSI system is designed to support the minimum MCR staffing described above, the space and layout of the MCR are designed to accommodate the foreseen maximum number of operating and temporary staff. Section 18.6 of this report provides a detailed assessment of staffing and qualifications, and adjusts the crew composition if its determined to be necessary.

MUAP-10009, Section 4.1, Concept of Operation, explains that the concept of operation used for the US-Basic HSI System is carried forward into the US-APWR design. The concept of operations for the US-Basic HSI System is supplemented with a description of communication systems used to support operations.

The staff concludes that the applicant provides a detailed description of the concept of operations that explains the operators interface with HSIs. Accordingly, the ataff finds that the applicants treatment of the concept of operations conforms to this NUREG-0711 criterion.

18-60

18.7.4.3 Functional Requirement Specification Criterion 1 Functional requirements for the HSIs should be developed to address:

• The concept of operations.

• Personnel functions and tasks that support their role in the plant as derived from function, task, and staffing/qualifications analyses.

• Personnel requirements for a safe, comfortable working environment.

The Staffs Evaluation of Criterion 1 Functional requirement specifications are addressed in the US-Basic HSI System described in MUAP-07007. In summary, functional specifications for the concept of operations are described in the previous acceptance criteria. A safe, comfortable work environment is accomplished by application of anthropometric design guidance which, in staff reviews, were found to conform to regulatory guidance provided in NUREG-0700, Part III, Workstation and Workplace Design.

Phase 2 of the applicants HSI design plan develops US-APWR specific specifications derived for the OER, FRA/FA, TA, HRA, and Staffing and Qualification elements of the HSI design process. Each of these areas identifies functional specifications, as discussed in the respective sections of the DCD, Chapter 18. The most significant new functional specifications are derived from the TA, which replaces generic US-Basic HSI System controls, alarms and displays with US-APWR specific ones. MUAP-10009, Section 4.3, Functional Requirement Specification, states that the functional specifications incorporated into the US-Basic HSI System are confirmed and, where needed, are supplemented by the OER, FRA/FA, TA, HRA, and Staffing and Qualification evaluations. The HFE design of local HSIs is also developed.

During the US-APWR HFE design process, all functional HSI specifications are added to the US-Basic HSI System configuration to become the complete US-APWR HFE design.

The staff concludes that this plan provides for a complete set of functional specifications for the US-APWR HFE design. Accordingly, the staff finds that the applicants treatment of functional requirements conform to this NUREG-0711 criterion.

Criterion 2 Requirements should be established for various types of HSIs, e.g., alarms, displays, and controls.

The Staffs Evaluation of Criterion 2 This subject is addressed in the previous criterion.

18-61

18.7.4.4 Human-System Interface Concept Design The development of an HSI concept design is one of three key elements of the HSI design plan along with style guide development and detailed HSI design integration.

Criterion 1 The functional requirement specification should serve as the initial source of input to the HSI design effort. If the design is a direct evolution from a predecessor, rather than a new design concept, the criteria in this section should be considered relative to operating experience of the predecessor and the design features (e.g., aspects of the process, equipment, or operations) of the new design that may be different from the predecessor. Human performance issues identified from operating experience with the predecessor design should be resolved.

The Staffs Evaluation of Criterion 1 The US-APWR functional specifications are derived from two sources. First, MUAP-07007 establishes specifications derived from predecessor plant experience. In summary, these specifications provide for the functional HSI operation, display conventions via a HFE style guide, and control room layout. The second part of the functional specifications are derived from the OER (human performance issues identified from operating experience with the predecessor design are specifically addressed within this element.), FRA/FA, TA, HRA, and Staffing and Qualification analyses described in the other sections of US-APWR DCD, Chapter 18. These specifications are specific to the US-APWR HFE design and are incorporated within the US-Basic HSI System to produce the complete US-APWR HFE design.

The combination of these specification sources is an acceptable method for identifying functional specifications as it combines predecessor information with HFE evaluations specific to the US-APWR application. In addition, these specification sources clearly identify the predecessor design and apply the same HFE analyses as described in NUREG-0711, including consideration of operating experience with the predecessor design Accordingly, the ataff finds that the applicants treatment of the HSI design conform to this NUREG-0711 criterion.

Criterion 2 Alternative approaches for addressing HSI functional requirements should be considered. A survey of the state-of-the-art in HSI technologies should be conducted to:

• Support the development of concept designs that incorporate advanced HSI technologies.

• Provide assurance that proposed designs are technically feasible.

• Support the identification of human performance concerns and tradeoffs associated with various HSI technologies.

The Staffs Evaluation of Criterion 2 18-62

Alternative approaches are addressed in the US-Basic HSI System described in MUAP-07007.

In summary, state-of-the-art surveys, alternative approaches, trade-off studies, and integrated system tests by nuclear plant operators, resulted in the selection of the employed HSI technologies such as those used for the LDP, touch screens, and the VDU-based methods of alarm, indication, and control.

The staff concludes that the identification of alternate approaches completed for the US-Basic HSI system is sufficient to provide reasonable assurance that the state-of-the-art HFE design is factored into the US-APWR control room since the US-Basic HSI design is the foundation for the US-APWR design. The subsequent analyses completed in accordance with NUREG-0711 are focused on identifying a complete inventory, verifying workload, satisfactory performance of important HAs and validating integrated system performance. These activities do not require alternate approaches to be reevaluated unless a HED is identified and then the HED resolution process specifies that alternate approaches be identified. Accordingly, the ataff finds that the applicants treatment of alternative approaches conforms to this NUREG-0711 criterion.

Criterion 3 Alternative approaches for addressing HSI functional requirements should be considered. Evaluation methods can include operating experience and literature analyses, tradeoff studies, engineering evaluations and experiments.

The Staffs Evaluation of Criterion 3 This subject is addressed in the previous criterion.

Criterion 4 Alternative concept designs should be evaluated so that one can be selected for further development. The evaluation should provide reasonable assurance that the selection process is based on a thorough review of design characteristics and a systematic application of selection criteria. Tradeoff analyses, based on the selection criteria, should provide a rational basis for the selection of concept designs.

The Staffs Evaluation of Criterion 4 Alternative concept designs were addressed in the development of the US-Basic HSI System described in MUAP-07007. In summary, operating experience and integrated system tests by nuclear plant operators were used in the selection and confirmation of the concept designs included in the US-Basic HSI system. Similar to the identification of alternate approaches, the remaining US-APWR design activities do not require alternate concept designs to be reevaluated unless a HED is identified. When identified the HED resolution process specifies that alternate approaches be re-evaluated. Accordingly, the staff finds that the applicants treatment of alternative concept designs conforms to this NUREG-0711 criterion.

Criterion 5 HSI design performance requirements should be identified for components of the selected HSI concept design. These requirements should be based on the functional requirement specifications but should be refined to reflect HSI 18-63

technology considerations identified in the survey of the state of the art in HSI technologies and human performance considerations identified in the human performance research.

The Staffs Evaluation of Criterion 5 Component performance specifications are addressed in the US-Basic HSI System described in MUAP-07007. In summary, performance-related requirements pertaining to screen content, screen controls, screen navigation, response times, and plant component controls were identified. The applicants style guide provides specific and standardized direction for component performance specifications. The style guide was verified to conform to NUREG-0700 guidance as part of the MUAP-07007 review.

US-APWR design specific evaluations, completed in accordance with NUREG-0711, focus on identifying a complete inventory, verifying workload, satisfactory performance of important HAs, and validating integrated system performance. These analyses confirm, correct, or augment the component performance specifications identified in the US-Basic HSI design. Accordingly, the staff finds that the applicants treatment of component performance specifications conforms to this NUREG-0711 criterion.

18.7.4.5 Human System Interface Detailed Design and Integration Criterion 1 Design-specific HFE design guidance (style guide) should be developed. HFE Guidelines should be utilized in the design of the HSI features, layout, and environment.

• The content of the Style Guide should be derived from (1) the application of generic HFE guidance to the specific application, and (2) the development of the applicants own guidelines based upon design-related analyses and experience.

Guidelines that are not derived from generic HFE guidelines may be justified by the applicant based on an analysis of recent literature, analysis of current industry practices and operational experience, tradeoff studies and analyses, and the results of design engineering experiments and evaluations. The guidance should be tailored to reflect design decisions by the applicant to address specific goals and needs of the HSI design.

• The topics in the Style Guide should address the scope of HSIs included in the design and address the form, function, and operation of the HSIs as well as environmental characteristics relevant to human performance.

• The individual guidelines should be expressed in concrete, easily observable terms. In general, generic HFE guidelines should not be used in their abstract form. Such generic guidance should be translated into more specific design guidelines that can, as much as possible, provide unambiguous guidance to designers and evaluators. They should be detailed enough to permit their use by design personnel to achieve a consistent and verifiable design that meets the applicant's guideline.

18-64

• The Style Guide should provide procedures for determining where and how HFE guidance is to be used in the overall design process. The Style Guide should be written so it can be readily understood by designers. The Style Guide should support the interpretation and comprehension of design guidance by supplementing text with graphical examples, figures, and tables.

• The guidance should be maintained in a form that is readily accessible and usable by designers and that facilitates modification when the contents require updating as the design matures. Each guideline included in the guidance documentation should include a reference to the source upon which it is based.

• The Style Guide should address HSI modifications. This guidance should specifically address consistency in design across the HSIs.

The Staffs Evaluation of Criterion 1 The Style Guide is addressed in the US-Basic HSI System described in MUAP-07007. In summary, the Style Guide is a fully developed engineering document (JEJC-1763-1001) providing guidance for general display format, display element format, and display design policy.

It is updated as technology changes by virtue of being a controlled document subject to the applicants QAP. The staffs reviews concluded that it acceptably implemented HFE design guidance from NUREG-0700 as well as good practices derived from the operating experience.

The Style Guide continues to be applicable to the US-APWR design as stated in MUAP-10009, Section 4.4.1, Style Guide Development, and is updated, when necessary, to reflect resolution of HEDs generated by the NUREG-0711 element evaluations.

The previously approved Style Guide continues to be used in the appropriate applications.

Accordingly, the staff finds that the applicants treatment of the style guide conforms to this NUREG-0711 criterion.

Criterion 2 The HSI detailed design should support personnel in their primary role of monitoring and controlling the plant while minimizing personnel demands associated with use of the HSIs (e.g., window manipulation, display selection, display system navigation). NUREG-0700 describes high-level HSI design review principles that the detailed design should reflect.

The Staffs Evaluation of Criterion 2 Detailed HSI design and its support of operating personnel is addressed in the US-Basic HSI System, described in MUAP-07007. In summary, the topical report addresses display design consistency, understandability of information, grouping of information, readability of information, distinctive coding, and uncluttered displays. The description of each concept clearly relates the concept back to facilitating personnel performance. The direction provided was found to be consistent with NUREG-0700.

US-APWR design specific evaluations completed in accordance with NUREG-0711 focus on identifying a complete inventory, verifying workload, satisfactory performance of important HAs, and validating integrated system performance. These analyses confirm, correct, or augment the US-Basic HSI system and ensure the operators primary role of monitoring and controlling the 18-65

plant is fully supported. For example, a detailed TA, which includes a workload analysis, is completed for each important HA to confirm the controls, displays, alarms and work environment to facilitate implementation of the actions. Accordingly, the staff finds that the applicants treatment of HSIs in their role of supporting personnel during operating activities conforms to this NUREG-0711 criterion.

Criterion 3 For risk-important HAs, the design should seek to minimize the probability that errors will occur and maximize the probability that an error will be detected if one should be made.

The Staffs Evaluation of Criterion 3 This criterion is addressed generically within the US-Basic HSI System described in MUAP-07007. In summary, errors are minimized using:

• Two touch operation to activate a control. The first action enables the soft control popup window. The second action activates the desired control.

• For the operational VDU, the soft control popup window is selected by touching an icon that represents the component to be controlled. The icon is presented in a graphical display that depicts the component within a system mimic thereby, promoting correct component selection.

• Control components are clearly and uniquely labeled.

• Soft control pop-up windows show component status feedback in real time, allowing operators to immediately detect control errors.

• If an operator action erroneously disables a safety function or erroneously creates a condition that threatens a critical safety function, dedicated alarms are provided on the LDP.

This criterion is addressed specifically in DCD Chapter 18. Risk-important HAs for the US-APWR design are identified by the US-APWR PRAs and documented in MUAP-07030. These actions along with credited manual actions in the Design Basis Event analyses described in Chapter 15 and the Diverse Actuation System described in Chapter 7 are inputs to the HFE design process. Each of the important HAs receives specific attention within the design process elements to ensure the final design minimizes the probability that errors will occur and maximizes the probability that an error will be detected if one should be made. For example, the TA develops a detailed narrative description for each action which includes the following topics:

• Information requirements

• Decision-making requirements

• Response requirements

• Communication requirements

• Workload

• Task support requirements 18-66

• Workplace factors

• Situational and performance shaping factors (PSFs)

• Hazard identification This information is then used to identify the controls, alarms and displays needed to support the action as well as a workload analysis that verifies the action can be reliably implemented within the control room environment. The resulting information is used as an input to the HSI design.

The staff concludes that the consideration given to each important HA within the NUREG-0711 design process elements, is sufficiently detailed to ensure that the variables associated with important HAs are managed effectively. When added to the error reduction designs included in the US-Basic HSI System, the resulting US-APWR HFE design minimizes the probability that errors will occur and maximizes the probability that an error will be detected if it should occur.

Accordingly, the staff finds that the applicants treatment of important HAs conforms to this NUREG-0711 criterion.

Criterion 4 When developing functional requirements for monitoring and control capabilities that may be provided either in the control room or locally in the plant, the following factors should be considered:

• Communication, coordination, and workload

• Feedback

• Local environment

• Inspection, test, and maintenance

• Importance to safety The Staffs Evaluation of Criterion 4 This criterion is addressed generically within the US-Basic HSI System described in MUAP-07007. In summary, the US-Basic HSI System incorporated functional specifications derived from the International Electrotechnical Commission standard 964, Design for Control Rooms of Nuclear Power Plants, issued 1989, which were found to include the functions listed in this criterion. The general design principle applied is that all control functions supporting power and safety success paths are accessible in the MCR and remote shutdown room (RSR) for normal and emergency plant conditions that are within the design basis and under normal HSI conditions. For loss of all nonsafety HSI, LCSs are credited to maintain the plant in a stable, powered condition. For common-cause failure (CCF) of all digital systems, including digital HSI, LCSs are credited to maintain the plant after achieving stable shutdown conditions from the MCR.

For the specific US-APWR HFE design, MUAP-10009, Section 4.4.4, Requirements for Allocation to MCR or LCS, states that the factors listed in this criterion will continue to be applied as part of the TA when developing functional specifications for monitoring and control capabilities in the control room or locally in the plant. Accordingly, the staff finds that the applicants treatment of monitoring and control capabilities conforms to this NUREG-0711 criterion.

18-67

Criterion 5 The layout of HSIs within consoles, panels, and workstations should be based upon (1) analyses of operator roles (job analysis) and (2) systematic strategies for organization such as arrangement by importance, frequency of use, and sequence of use.

The Staffs Evaluation of Criterion 5 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the US-Basic HSI System describes a control room configuration where accepted anthropometric practices have been applied to the location of consoles, panels and workstations. The layout and design of HSIs within the panels and workstations is a major subject of the topical report. In general, the layout for panels with conventional HSI devices (e.g., alarms, indicators, controls) follows historical practice which arranges alarms at the top of the panel, indicators in the middle and controls in the lower section. This historical practice typically supports importance, frequency of use, and sequence of use and is subjected to a phased validation process to demonstrate effectiveness. The US-Basic HSI System also uses system mimics to organize display information.

The US-APWR HFE design is not expected to alter this part of the design. However, the HSI layout defined by the US-Basic HSI System will be updated to reflect resolution of HEDs from previous program elements, as necessary. Any changes introduced are evaluated as part of the integrated system validation activity.

The staff concludes that the iterative review of the HSI layout, first using a predecessor design then followed by confirmation from detailed input from the NUREG-0711 elements, provides an acceptable method for organizing control room HSIs. Accordingly, the staff finds that the applicants treatment of the HSI layout conforms to this NUREG-0711 criterion.

Criterion 6 Personnel and task performance should be supported during minimal, nominal, and high-level staffing.

The Staffs Evaluation of Criterion 6 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the US-Basic HSI System uses a minimum staffing of one RO and one SRO to perform control room activities, an additional SRO acting as shift supervisor (SS) and STA, and an additional RO. Typical staffing adds one additional person acting as a STA (the SS and STA functions are independent). This is defined as maximum continuous manning. Facilities for shift changes are located in the control room along with a dedicated communication board, working places for temporary personnel, and a working area for reading paper based documentation. Computer-based HSI workstations for the additional personnel expected during outages and commissioning are located in the computer room or the switching and tagging room.

MUAP-10009, Section 4.4.6. Support for Staffing Range, states that there are no additional HSI design activities specifically required to address minimum staffing but the staff levels will be further confirmed through the US-APWR HRA, TA, S&Q, and V&V program elements. The US-18-68

Basic HSI System will be updated to reflect resolution of HEDs from any of these previous program elements, as necessary.

MUAP-10009, Section 4.4.6, states that the HSI design supports shutdown mode staffing levels and space and crew meeting facilities to accommodate shift turnover. In its response to RAI 725-5408, Question18-98, dated April 27, 2011 (ADAMS Accession No. ML11119A208), the applicant indicated that the following positions are also considered in the control room layout and facility design:

• One shift crew assistant, responsible for assisting the shift supervisor and handling communications.

• One additional RO, responsible for assisting the two ROs and interacting with other members of the plant staff.

• One NRC observer.

• One Plant management observer.

• Two equipment operators.

This response satisfactorily addresses the HSI design supporting manning levels for various modes and activities. Accordingly, the staff finds that the applicants treatment of the HSI design for various staffing levels conforms to this NUREG-0711 criterion.

Criterion 7 The design process should take into account the use of the HSIs over the duration of a shift where decrements in performance due to fatigue may be a concern.

The Staffs Evaluation of Criterion 7 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the US-Basic HSI System uses lighting, ergonomics, and layout design to mitigate excessive fatigue. The applicant also conducts a US-APWR specific workload analysis to identify potential decrements in performance over the duration of a shift and demonstrates during integrated system validation that fatigue is not affecting operator performance.

Accordingly, the staff finds that the applicants treatment of fatigue over the shift duration conforms to this NUREG-0711 criterion.

Criterion 8 HSI characteristics should support human performance under the full range of environmental conditions, e.g., normal as well as credible extreme conditions.

For the main control room requirements should address conditions such as loss of lighting, loss of ventilation, and main control room evacuation. For the remote shutdown facility and local control stations, requirements should address constraints imposed by the ambient environment (e.g., noise, temperature, contamination) and by protective clothing (if necessary).

18-69

The Staffs Evaluation of Criterion 8 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the applicant controls the MCR environment, so there is a specific design window in which the HSI functions. The HSI style guide addresses the scope of HSIs included in the design and addresses the form, function, and operation of the HSIs, as well as environmental characteristics relevant to human performance. The MCR is a highly controlled environment without a significant fluctuation of environmental conditions and includes emergency lighting, ventilation, and control room habitability systems for plant accident conditions. Emergency lighting is addressed in Section 9.5.3, Lighting Systems, ventilation in Section 9.4, Air Conditioning, Heating, Cooling, and Ventilation Systems, and control room habitability in Section 6.4, Habitability Systems.

MUAP-10009, Section 4.4.8, Environmental Conditions, states that the RSR has the same redundant heating, ventilation, and air conditioning (HVAC) and emergency lighting system design as in the MCR.

Specific to the US-APWR HFE design, MUAP-13009, Section 4.2, Basic Task Analysis, provides for identifying workplace factors that affect specific tasks. Factors unique to the US-APWR HFE design would be identified by this plan and then factored into the US-APWR HSI design.

The staff concludes that this approach addresses the HSI designs support for human performance under a range of environmental conditions for both the control room and local control stations. Accordingly, the staff finds that the applicants treatment of environmental conditions conforms to this NUREG-0711 criterion.

Criterion 9 The HSIs should be designed to support inspection, maintenance, test, and repair of (1) plant equipment and (2) the HSIs. The HSIs should be designed so that inspection, maintenance, test, and repair of the HSIs do not interfere with other plant control activities (e.g., maintenance tags should not block the operators views of plant indications).

The Staffs Evaluation of Criterion 9 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the US-Basic HSI System provides a tagging system integrated within the operational VDU. Detailed tagging information on a component is available in a popup window and tagging status information is provided on the soft operation switch popup window and on the switch selection button on the operation VDU screen. Tagging is an administrative status function that has no effect on the operability of the component and tagging information is positioned so it is coordinated with other component information.

The HSIs are part of the integrated Digital I&C platform and are continuously checked by the platform self-diagnostic features, which are described in detail in MUAP-07005, Section 4.1.5,

[Controller] Self Diagnosis, and evaluated by the NRC staff in Chapter 7 of this SE. In summary, the platform self-diagnostic features continuously check the integrity of processing and communication components, as well as the range of process inputs. These self-diagnostic 18-70

features allow early detection of failures, and allow easy and quick repair that improves system availability. Information regarding detected failures is gathered through system communication networks and provided to the maintenance staff in a comprehensive manner. Alarms are generated in the MCR for any failures that effect system functionality. The platform self-diagnostic features control the redundant configuration to maintain all system functions for most single failures.

The staff concludes that the design features described above appropriately support inspection, maintenance, testing, and repair activities. The equipment status is clearly communicated to the operator and does not interfere with operational activities. Technology has been applied to minimize operator performance of routine inspection and testing activities. The automatic performance of these activities further supports the operators ability to manage other operational activities. Accordingly, the staff finds that the applicants treatment of test, inspection and maintenance activities conforms to this NUREG-0711 criterion.

18.7.4.6 Human System Interface Tests and Evaluations This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the US-Basic HSI System used iterative validation activities on a full scope simulator.

Qualified operators were used to respond to scenarios designed to verify the design, and was found to be acceptable. HEDs were written for the improvements identified during testing. This phase (Phase 1), established the basic physical design for the HSIs including translation to the English language and American engineering units, anthropometric changes to the consoles for American body types, adoption of the US-style step-by-step operating procedures, automated auxiliary feedwater control, LDP improvements, and computer based procedures. Modifications to the HSI design were tested and evaluated using a full scope simulator and qualified operators. While not as detailed as the integrated system validation described in NUREG-0711, Section 11.4.3, Integrated System Validation, the testing and evaluation contained all the elements of the integrated system validation. This iterative validation methodology was determined by the NRC staff to be equal to or better than the design testing and evaluation methods described in NUREG 0711, Section 8.4.6, HSI Tests and Evaluations. The staffs findings, as they pertain to this iterative validation method performed in Phase 1, are summarized in the following sections so that the basis for accepting the HSI physical design developed in Phase 1 is readily available.

Phase 2 defines the US-APWR HFE design using the US-Basic HSI System as its basis.

Following the HFE design process described in NUREG-0711, the applicant develops the HSI inventory (controls, displays and alarms) specific to the US-APWR application. HEDs from Phase 1 continue to be resolved, which may identify additional modifications but generally the physical HSI design is established and design testing and evaluation is more limited. Testing and evaluation that is performed follows the same methods used in Phase 1. A static, portable HSI system analysis tool is also used to evaluate the consistency of the HSI inventory displays, and to supplement the simulator displays for design verification against the style guide and procedures.

18.7.4.6.1 Trade-Off Evaluations Criterion 1 Aspects of human performance that are important to task performance should be carefully selected and defined so that the differential effects of design options on 18-71

human performance can be adequately considered in the selection of design approaches. The following factors should be considered when developing selection criteria:

• Personnel task requirements.

• Human performance capabilities and limitations.

• HSI system performance requirements.

• Inspection and testing requirements.

• Maintenance requirements.

• Use of proven technology and the operating experience of predecessor designs.

Criterion 2 The selection process should make explicit the relative benefits of design alternatives and the basis for their selection.

The Staffs Evaluation of Criteria 1-2 Additional testing of the HSI physical design is not expected in Phase 2. If additional testing is needed, MUAP-10009, Section 4.5.2, Performance-Based Tests, states that the applicant will use the same Phase 1 analysis methods which the staff has already accepted. In summary, the staff accepted the testing methodology because the use of qualified U.S. operators in a full scope simulator to test and evaluate HSI design decisions is more effective than using engineers to make the initial decisions and then waiting until the integrated system validation to verify the effectiveness of those decisions. Specific to this criterion, qualified operators, because of the training and experience required for their qualification, directly apply all elements described in this criterion within the validation test. MUAP-07007 documents the design options chosen and their bases. Accordingly, the staff finds that the applicants treatment of design options conforms to these NUREG-0711 criteria.

18.7.4.6.2 Performance-Based Tests Criterion 1 Performance-based tests can have many different purposes; therefore, the hypotheses should be structured to address the specific questions being addressed.

Criterion 2 The general approach to testing should be based on the test objective. The design of performance-based tests should be driven by the purpose of the evaluation and the maturity of the design.

18-72

Criterion 3 The specific design features or characteristics of design features should be carefully defined. If the characteristics are to be manipulated in the test, i.e.

systematically varied, the differences between test conditions should be specified in detail.

Criterion 4 The selection of testbeds for the conduct of performance-based tests should be based upon the requirements imposed by the test hypotheses and the maturity of the design.

The Staffs Evaluation of Criteria 1-4 These criteria are addressed within the US-Basic HSI System described in MUAP-07007. As described in the previous section, performance-based testing used qualified operators in a full scope simulator reflecting the HSI design approved in MUAP-07007, which provides the closest modeling possible to actual operating conditions. The testing purpose, scope, and objectives are documented within the scenarios used to conduct the tests.

Specific to this criterion, MUAP-10009, Section 4.5.2, Performance-Based Tests, states that, as the final design matures, additional performance-based testing is used to evaluate important HED resolution choices prior to entering the ISV. Other performance-based tests are conducted for complex elements of the US-APWR HSI inventory. These performance-based tests are primarily conducted for task-based VDU screens that include controls for the following complex control system functions:

• Control rod drive mechanism - control system.

• Turbine control system.

• Turbine bypass valve control system.

• Makeup control system.

• RCS temperature and pressure control systems (including heat-up and cooldown).

• Steam generator level control system (including normal and emergency feedwater).

The staff concludes that the performance testing is appropriately controlled by the test scenarios used to implement the tests. The HEDs and design complexity are being appropriately used to determine any additional performance testing as the design matures. A full scope simulator is an acceptable test bed providing for optimum performance testing. Accordingly, the staff finds that the applicants treatment of performance testing conforms to these NUREG-0711 criteria.

18-73

Criterion 5 The selection of performance measures should be based on a consideration of:

• Measurement characteristics

• Identification and selection of variables to represent measures of the aspects of performance under investigation

• Development of performance criteria The Staffs Evaluation of Criterion 5 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the objective and subjective performance measures addressed in the anthropometric design, plant response, situational awareness, teamwork, supervisory oversight, and workload were evaluated against the acceptance criteria. The measures and criteria were specific to each scenario, which in turn is specific to the aspects of performance being investigated. The same practices are used for Phase 2 performance testing. Accordingly, the staff finds that the applicants treatment of performance measures conforms to this NUREG-0711 criterion.

Criterion 6 The selection of participants for HSI design tests should be based on the nature of the questions being addressed in test objectives and the level of design maturity.

The Staffs Evaluation of Criterion 6 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the qualified operators in a full scope simulator were used to test and evaluate HSI design decisions. The qualified operators continue to be used in Phase 2 performance tests.

The observers include Operations and HFE experts. This combination provides the expertise required to maximize the effectiveness of design testing and ensure that the test objectives are addressed. Accordingly, the staff finds that the applicants treatment of testing participants conforms to this NUREG-0711 criterion.

Criterion 7 The test design should permit the observation of performance in a manner that avoids or minimizes bias, confounds, and error variance (noise).

The Staffs Evaluation of Criterion 7 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the testing used multiple operating crews, independent observers, and test procedures to minimize bias, confounds and error variance. Phase 2 testing is limited to focusing on specific HSI elements using prototypes with a dynamic part-task simulator. Bias, confounds, and error variance are addressed as part of the integrated system testing.

18-74

Based on the robust testing of the US-Basic HSI system and the limited need for testing in Phase 2, the staff concludes that addressing bias, confounds, and error variance, as part of the integrated system testing, is acceptable. Accordingly, the staff finds that the applicants treatment of testing reliability conforms to this NUREG-0711 criterion.

Criterion 8 Test data should be analyzed using established analysis techniques.

The Staffs Evaluation of Criterion 8 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the applicant used the Converging Perspectives methodology for analyzing data. In this methodology, multiple performance measures are used to identify performance challenges and to confirm positive performance. Subjective performance measures use a Likert rating scale, from one to five, to quantify the input from the testing participants. These are standard data analysis techniques. These techniques continue to be used in Phase 2 testing.

Accordingly, the staff finds that the applicants treatment of data analysis conforms to this NUREG-0711 criterion.

Criterion 9 Design solutions, such as modifications of the HSIs or user training requirements, should be developed to address problems that are identified during the testing and evaluation of the HSI detailed design.

The Staffs Evaluation of Criterion 9 This criterion is addressed within the US-Basic HSI System described in MUAP-07007. In summary, the HSI modifications were generally used to transform the Japanese Basic HSI system to the US-Basic HSI system. This was acceptable as the training and procedure programs for the US-APWR had not been developed.

MUAP-09019, Section 5.1.3.2, Human Engineering Discrepancy Processing, outlines various options for resolving the HEDs. This list includes HSI modifications, and changes to the procedures and training programs.

The staff concludes that the IPs provide a clear emphasis on modifying HSIs to provide a state-of-the-art control room HFE design. Procedure and training program changes are used to correct inaccuracies rather than substitute for an improved design. Accordingly, the staff finds that the applicants treatment of design solutions conforms to this NUREG-0711 criterion.

18-75

18.7.4.6.3 Human System Interface Design Documentation Criterion 1 The HSI design should be documented to include:

• The detailed HSI description including its form, function and performance characteristics,

• The basis for the HSI requirements and design characteristics with respect to operating experience and literature analyses, tradeoff studies, engineering evaluations and experiments, and benchmark evaluations, and

• Records of the basis of the design changes.

Criterion 2 The outcomes of tests and evaluations performed in support of HSI design should be documented.

The Staffs Evaluation of Criterion 1 and 2 The US-Basic HSI System is documented in MUAP-07007 and the controlled design documents including the following:

• The HSI Design Style Guide.

• The US-Basic HSI Nomenclature which defines the standard acronyms and abbreviations and equipment description guidelines used in the HSI design.

• The US-Basic HSI Component Control Design Guide that describes generic control logic and information processing logic to support operator control face plate operation, including associated indications and alarms.

• US-Basic HSI System Detailed Design Description.

• Graphic display and panel layout drawings.

• The HSI database, which defines characteristics (e.g., Instrumentation ranges, alarm prioritization) and links the VDU display icons, parameters, trends, alarms, soft controls, etc. and panel hardware devices to the database of the control and protection systems.

• Logic and algorithm diagrams for the HSI function processing, such as OK status monitoring (indications of systems operating properly), bypassed and inoperable status indication, and critical safety function monitoring.

• Detailed room and console configuration diagrams (layout drawings).

18-76

MUAP-10009, Section 4.6, HSI Design Documentation, provides for a comparable documentation list but specific to each US-APWR facility (e.g., MCR, RSR) and each HSI system (e.g., VDUs, LDP). For each facility and HSI system, the following documentation is captured:

• Starting point from the US-Basic HSI System.

• Inputs from previous US-APWR HFE analyses conducted in accordance with NUREG-0711 guidance.

• Inputs from US-APWR plant design (including site-specific assumptions for a complete plant).

• Inputs from predecessor plants.

• HSI design outputs (including documentation).

• Key design aspects.

• Tests/Analysis.

• Designer SMEs.

• Reviewer SMEs.

• Review criteria.

These inputs support key design decisions which are listed for each facility and HSI system component. From these decisions, design outputs are derived and these outputs are listed.

They are similar to the US-Basic HSI System design documentation sited above, except the documentation is now US-APWR specific.

The staff finds that the applicant has outlined a thorough approach to documenting the design results and their basis. Accordingly, the staff finds that the applicants treatment of HSI design documentation conforms to these NUREG-0711 criteria.

18.7.4.7 Safety Parameter Display System Design A SPDS is required by 10 CFR 50.34(f)(2)(iv), which applies to DC applications by virtue of 10 CFR 52.47(a)(8). The NRC previously used NUREG-0737, Clarification of TMI Action Plan Requirements, Supplement 1, and NUREG-1342, A Status Report Regarding Industry Implementation of Safety Parameter Display System, issued April 1989, for review guidance, but this guidance has been integrated into Section 5, Safety Function and Parameter Monitoring System, of NUREG-0700.

Accordingly, an evaluation of the conformance to the requirements of 10 CFR 50.34(f)(2)(iv) follows:

18-77

Criterion 1 10 CFR 50.34(f)(2)(iv) - General SPDS Requirements Title 10, Subsection 50.34(f)(2)(iv) of the Code of Federal Regulations requires that the design provide a plant safety parameter display console that will (1) display to operators a minimum set of parameters defining the safety status of the plant, (2) be capable of displaying a full range of important plant parameters and data trends on demand, and (3) be capable of indicating when process limits are being approached or exceeded.

The Staffs Evaluation of Criterion 1 In the US-APWR design, the SPDS is an integral part of the Control Room HSI design rather than a stand-alone, add-on system as is used at most operating plants. Parameters, data trends, and alarms needed to address the regulation are available on the Operational VDUs, the Safety VDUs, and the LDP. The LDP is the HSI specifically reviewed for meeting the regulation because of the following design functions:

• providing SDCV information to the operation personnel to enhance situation awareness,

• helping operators maintain continuous awareness of overall plant status and critical status changes, and

• helping the operations staffs coordination and communication by providing a common visualization of plant information.

These design functions are expanded upon in MUAP-10009, Section 4.6.3,Large Display Panel, where the following key LDP design concepts are identified:

• LDP HSI inventory is defined by the US-Basic HSI System and confirmed or supplemented by the specific HSI inventory required for US-APWR. This inventory includes:

- Critical safety and power production functions, including key parameters.

- Normal and emergency success paths, including key components and key parameters.

- Post-Accident Monitoring (PAM) Type A and B variables.

- Indications and alarms to prompt IHAs.

• Logical alarm and display grouping.

• Algorithms that are easily understood by plant operators.

18-78

• Readability from all operator workstations provides continuous display for the status of all critical safety functions and the plant systems used to control those safety functions. DCD, Section 18.7.3.2, Safety Aspects of the HSI, identifies the following functions that define the displayed parameters:

- safety function monitoring

- periodic testing of protection system actuation functions

- bypased and inoperable status indication for plant safety systems

- manual initiation of protective actions

- instrumentation required to assess plant and environmental conditions during and following an accident

- instrumentation required to assess plant and environmental conditions during and following an accident

- setpoints for safety-related instrumentation

- HSIs for the TSC.

MUAP-07007 provides a detailed description of the LDP. Section 4.9.2.1, Fixed Display Area, and Section 4.9.2.2, Variable Display Area, explain how parameters and data trends are organized and displayed. Section 4.9.3, Alarm Display on the Large Display Panel, explains alarms, alarm processing, and alarm displays. In summary, the staff found that the LDP configuration conformed to NUREG-0700, Section 5, Safety Function and Parameter Monitoring System.

As part of the SE for MUAP-07007, the staff noted that the MHI design provides for a seamless display of the information (even though three screens are being used) and the plant mimic used coordinates and organizes the information so that the operators can efficiency and reliably locate needed information. The LDP presents a significant amount of information but always within a construct (such as safety function, engineered safety features (ESF) status) that associates the information with higher level safety objectives. The OK Monitor that provides a status of automatic checks on actuation results for RPS and ESF alarm system (ESFAS) and the Critical Safety Function Monitor which provides a status of automatic checks of the critical safety function status tree logic are examples of how the information is managed to minimize the time that the operators spend completing verification checks, while at the same time providing summarized critical safety information. The LDP maintains the same label, symbol, and color conventions used in the VDU display design thus minimizing the operators confusion in moving between HSIs.

MUAP-10009, Section 4.6.3, Large Display Panel, provides for the confirmation or supplementation of parameters, data trends, and alarms provided on a LDP. Specific US-APWR Design information defined by the procedure includes:

18-79

• Plant mimic.

• Input ID tags and algorithms for information processing related to all icons (e.g.,

OK status monitor, critical safety functions), signal selection algorithms, trend arrows).

• Alarm grouping into icons and tiles.

The staff concludes that the US-APWR integrated SPDS design achieves the purpose of enhancing the operators ability to comprehend plant conditions and interact in situations that call for human intervention. This is accomplished by providing a concise display of critical plant variables and safety function status, along with its trends and alarm conditions that aid the operators in rapidly and reliably determining the safety status of the plant. Therefore, the staff also concludes that 10 CFR 50.34(f)(2)(iv) is met.

18.7.4.8 Minimum Inventory 18.7.4.8.1 Regulatory Criteria The concept of minimum inventory originated from the agency policy described in SECY-92-053. PAM guidance contained in RG 1.97, Revision 4, which endorses the standard, Institute of Electrical and Electronic Engineers (IEEE) 497-2002, significantly overlaps the following minimum inventory guidance:

• SRP Chapter 14.0, Initial Test Program and ITAAC-Design Certification, Section 14.3.9, Human Factors Engineering - Inspections, Tests, Analyses, and Acceptance Criteria.

• Interim Staff Guidance (ISG) DI&C-ISG-05, Revision 1, Section 2, Minimum Inventory, issued November 3, 2008.

• BTP 18-1, Guidance for Evaluating Minimum Inventory of Alarms, Controls, and Displays for New Light Water Reactor Plant Designs, issued September 11, 2009.

IEEE 497-2002 has the most inclusive criteria for accident monitoring and is used in this SE.

18.7.4.8.2 Technical Review Results IEEE 497-2002 guidance was evaluated by an interdisciplinary team that included staff from I&C, Human Factors, Primary Systems, PRA, and TS Branches. The results of this evaluation are described in Chapter 7 of this safety evaluation. In summary, the staff concludes that the applicant identified a complete set of accident monitoring instrumentation that conforms to IEEE 407-2002, as endorsed by RG 1.97.

18.7.4.9 Computerized Procedures The MUAP-07007 describes the US-Basic HSI System, which serves as the foundation for the US-APWR HFE design. In the TR SER, the staff concluded that the design of the computer based procedure (CBP) interface satisfactorily conforms to the regulatory guidance for display 18-80

design and the system-user interface as stated in DI&C ISG 05, Chapter 1, Computer Based Procedures. The design provides reasonable assurance that the operating crew will be able to efficiently use the CBPs. Backup procedures are maintained for use when the CBPs are not available.

In MUAP-10009, Section 4.6.12, Computer-Based Procedure Screen, the applicant describes the US-APWR specific design inputs and outputs for the CBP display screen. In summary, hyperlinks, ID tags, and monitoring algorithms are added to the US-Basic CBP platform.

The staff concludes that the US-APWR CBP design provides a reliable platform for implementing operating procedures. The CBP HSI enhances the operators ability to manage plant conditions by providing information and controls within the immediate context of the procedure step being implemented. Backup paper based procedures remain available if there is degradation or failure of the CBPs. Accordingly, the staff concludes that the US-APWR CBPs conform to the applicable review criteria.

18.7.5 Combined License Information Items There are no COL information items related to this area of review. The staff determined that no COL information items need to be included in DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for the HFE HSI design consideration.

18.7.6 Conclusions The staff evaluated the HSI design, at an IP level, using the review criteria in NUREG-0711, Section 8.4, Review Criteria. Section 18.0.4, of this report, provides a discussion of review levels. The staff finds that the HSI design provides a satisfactory process by which US-APWR functional and task specifications will be appropriately translated into the detailed design of the alarms, controls, and displays, and other aspects of the HSI through the systematic application of HFE principles and criteria. The designs developed through this process, when incorporated within the US-Basic HSI System described in Topical Report MUAP-07007, which was previously approved by the staff, will provide a complete US-APWR HFE design. This includes the minimum inventory of HSIs for the MCR and the remote shutdown station. Accordingly, the staff concludes that the HSI design considerations, with respect to HFE, have been adequately addressed, and that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47, related to this technical area, are satisfied.

18.8 Procedure Development Procedure development is included in NUREG-0711, because there are HFE attributes associated with the procedures. However, as an operating program, procedures are reviewed in Section 13.5, Plant Procedures, of this report. The staffs conclusions are documented in that section.

18.9 Training Program Development The training program development is included in NUREG-0711 because of the interfaces between the HFE design, procedures and training. However, as an operating program, training is reviewed in Section 13.2, Training, of this report. The staffs conclusions are documented in that section.

18-81

18.10 Human Factors Verification and Validation 18.10.1 Introduction The objective of the staffs review is to assure that the applicants human factors V&V demonstrate that the design conforms to the HFE design principles and enables plant personnel to successfully perform operational tasks. The overall scope for V&V should include the MCR, the remote shutdown panel, and local control stations associated with the risk important HAs.

18.10.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 and Table 2.9-1.

DCD Tier 2: The applicant provided a Tier 2 description in Section 18.10, Verification and Validation, which describes the methods for selecting operational scenarios, completing the task and design verification and conducting the integrated system validation.

ITAAC: The ITAAC associated with this element are listed in Tier 1, Section 2.9, Table 2.9-1.

TS: There are no TS associated with this element.

Topical Reports: The topical reports associated with DCD Tier 2, Section 18.10 are as follows:

• MUAP-07007, Human System Interface System Description, Revision 6, issued May 2014.

• PQD-HD-19005, Quality Assurance Program (QAP) Description for Design Certification of the US-APWR, Revision 6, issued October 2013.

Technical reports: The technical reports associated with this element are:

• MUAP-10012, US-APWR Human Factors Verification and Validation Implementation Plan, Revision 4, issued May 2014.

• MUAP-09019, US-APWR Human Factors Engineering Program Management Plan, Revision 5, issued August 2014.

• MUAP-13009, US-APWR Task Analysis Implementation Plan, Revision 1, issued May 2014.

• MUAP-07005, Safety System Digital Platform - MELTAC, Revision 9, issued December 2013.

18.10.3 Regulatory Basis The SRP identifies the relevant Commission regulations for the HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

18-82

• 10 CFR 50.34(f)(2)

Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 11, Human Factors Verification and Validation, Section 11.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18.II.A.1, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering.

18.10.4 Technical Evaluation The staff performed an implementation level review, as described in NUREG-0711 and Section 18.0.4 of this report.

NUREG-0711 criteria were used to evaluate the detailed methodology. The V&V review criteria used were from the following sections of NUREG-0711:

• Section 11.4.1 - Operation Condition Sampling.

- Sampling Dimensions (three review criteria).

- Identification of Scenarios (two review criteria).

• Section 11.4.2 - Design Verification.

- Inventory and Characterization (three review criteria).

- HSI Task Support Verification (five review criteria).

- HFE Design Verification (three review criteria).

• Section 11.4.3 - Integrated System Validation.

- Test Objectives (one review criteria).

- Validation Testbeds (nine review criteria).

- Plant Personnel (four review criteria).

- Scenario Definition (three review criteria).

- Performance Measurement (five review criteria).

- Test Design (nine review criteria).

- Data Analysis and Interpretation (five review criteria).

- Validation Conclusions (two review criteria).

In this report, the NUREG-0711 criteria are used to assess the completeness of the IP and its acceptability as an IP. The results of the staffs evaluation of the V&V IP, with respect to the NUREG-0711 criteria, are provided below.

18.10.4.1 Operational Conditions Sampling NUREG-0711, Section 11.4.1, Operational Conditions Sampling, states:

18-83

The sampling methodology will identify a range of operational conditions to guide V&V activities. The review of operational conditions sampling considers the dimensions to be used to identify and select conditions and their integration into scenarios.

The objective of reviewing operational condition sampling (OCS) is to verify that the applicant has identified a sample of operational conditions that: (1) includes conditions that are representative of the range of events that could be encountered during operation of the plant, (2) reflects the characteristics that are expected to contribute to system performance variation, and (3) considers the safety significance of HSI components. These sample characteristics are best identified through the use of a multidimensional sampling strategy to provide reasonable assurance that variation along important dimensions is included in the V&V evaluations.

18.10.4.1.1 Sampling Dimensions The sampling dimensions addressed in NUREG-0711, Section 11.4.1.2, Operational Conditions Sampling Review Criteria, include plant conditions, personnel tasks, and situational factors known to challenge personnel performance.

Criterion 1 The following plant conditions should be included:

• normal operational events including plant startup, plant shutdown or refueling, and significant changes in operating power

• Failure events, e.g.,

- instrument failures [e.g., safety-related system logic and control unit, fault tolerant controller, local field unit for multiplexer (MUX) system, MUX controller, and break in MUX line] including I&C failures that exceed the design basis, such as a common mode I&C failure during an accident

- HSI failures (e.g., loss of processing and/or display capabilities for alarms, displays, controls, and computer-based procedures)

• transients and accidents, e.g.,

- transients (e.g., turbine trip, loss of off-site power, station blackout, loss of all feedwater, loss of service water, loss of power to selected buses or main control room (MCR) power supplies, and safety and relief valve transients)

- accidents (e.g., main steam line break, positive reactivity addition, control rod insertion at power, anticipated transient without scram, and various-sized loss-of-coolant accidents)

- reactor shutdown and cooldown using the remote shutdown system 18-84

• reasonable, risk-significant, beyond-design-basis events, which should be determined from the plant specific PRA

• consideration of the role of the equipment in achieving plant safety functions [as described in the plant safety analysis report (SAR)] and the degree of interconnection with other plant systems. A system that is interconnected with other systems could cause the failure of other systems because the initial failure could propagate over the connections. This consideration is especially important when assessing non-class 1E electrical systems.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.1.1, Sampling Dimensions, contains a list of operating conditions included in the operational conditions sample. The list addresses all elements contained in the acceptance criteria. Additional detail and examples are provided for general categories which clearly communicate that plant conditions recognized as being challenging for the control room staff are included in the sample. There are also areas included in the sample that provide good additions to the scope established by the acceptance criterion. Accordingly, the staff concludes that the applicants treatment of plant conditions included in the operational conditions sample conforms to this NUREG-0711 criterion.

Criterion 2 The following types of personnel tasks should be included:

• Risk-significant HAs, systems, and accident sequences - All risk-important HAs should be included in the sample. These include [those] identified in the PRA and those identified as risk-important in the SAR and NRCs SER. Situations where human monitoring of an automatic system is risk-important should be considered. Additional factors should be sampled that contribute highly to risk, as defined by the PRA, including:

- dominant human actions (selected via sensitivity analyses)

- dominant accident sequences

- dominant systems (selected via PRA importance measures such as Risk Achievement Worth or Risk Reduction Worth)

• OER-identified difficult tasksThe sample should include all personnel tasks identified as problematic during the applicants review of operating experience.

• Range of procedure guided tasksThese are tasks that are well defined by normal, abnormal, emergency, alarm response, and test procedures. The operator should be able to, as part of rule-based decision-making, understand and execute the specified steps. RG 1.33, Appendix A, contains several categories of typical safety-related activities that should be covered by written procedures. The sample should include appropriate procedures in each relevant category:

18-85

- administrative procedures

- general plant operating procedures

- procedures for startup, operation, and shutdown of safety-related systems

- procedures for abnormal, off normal, and alarm conditions

- procedures for combating emergencies and other significant events

- procedures for control of radioactivity

- procedures for control of measuring and test equipment and for surveillance tests, procedures, and calibration

- procedures for performing maintenance

- chemistry and radiochemical control procedures

• Range of knowledge-based tasks - these are tasks that are not as well defined by detailed procedures. Knowledge-based decision-making involves greater reasoning about safety and operating goals and the various means of achieving them. A situation may call for knowledge-based decision-making if the rules do not fully address the problem, or the selection of an appropriate rule is not clear.

An example in a pressurized water reactor plant may be the difficulty in diagnosing a steam generator tube rupture (SGTR) with a failure of radiation monitors on the secondary side of the plant because (1) there is no main indication of the rupture (the presence of radiation in secondary side), and (2) the other effects of the rupture (i.e., slight changes in pressures and levels on the primary and secondary sides) may be attributed to other causes. While the operators may use procedures to treat the symptoms of the event, the determination that the cause is an SGTR may warrant situational assessment based on an understanding of the plants design and the possible combinations of failures that could result in the observed symptoms. Errors in rule-based decision-making result from selecting the wrong rule or incorrectly applying a rule. Errors in knowledge-based decision-making result from mistakes in higher-level cognitive functions such as judgment, planning, and analysis. The latter are more likely to occur in complex failure events where the symptoms do not resemble the typical case, and thus, are not amenable to pre-established rules.

• Range of human cognitive activitiesThe sample should include the range of cognitive activities performed by personnel, including:

- detection and monitoring (e.g., of critical safety-function threats)

- situation assessment (e.g., interpretation of alarms and displays for diagnosis of faults in plant processes and automated control and safety systems)

- response planning (e.g., evaluating alternatives for recovery from plant failures) 18-86

- response implementation (e.g., in-the-loop control of plant systems, assuming manual control from automatic control systems, and carrying out complicated control actions)

- obtaining feedback (e.g., of the success of actions taken)

• Range of human interactionsThe sample should reflect the range of interactions among plant personnel, including tasks that are performed independently by individual crew members and tasks that are performed by crew members acting as a team. These interactions among plant personnel should include interactions between:

- main control room operators (e.g., operations, shift turnover walk downs)

- main control room operators and auxiliary operators

- main control room operators and support centers (e.g., the technical support center and the emergency offsite facility)

- main control room operators with plant management, NRC, and other outside organizations

• Tasks that are performed with high frequency.

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.1.1, Sampling Dimensions, contains a list of personnel tasks included in the operational conditions sample. The list addresses all elements contained in the acceptance criteria. Additional detail and examples are provided for general categories which clearly communicates that tasks recognized as being challenging for the control room staff are included in the sample. There are also areas included in the sample that provide good additions to the scope established by the acceptance criterion. Notably, the applicant has included deterministically important HAs extracted from the transient and accident analyses and diversity and D3 coping analysis. Also an excellent (and proprietary) list of knowledge based tasks has been provided that thoroughly scopes this task source. Accordingly, the staff concludes that the applicants treatment of operator tasks included in the operational conditions sample conforms to this acceptance criterion.

Criterion 3 The sample should reflect a range of situational factors that are known to challenge human performance, such as:

• Operationally difficult tasksThe sample should address tasks that have been found to be problematic in the operation of NPPs, e.g., procedure versus situation assessment conflicts. The specific tasks selected should reflect the operating history of the type of plant being validated (or the plants predecessor).

18-87

• Error-forcing contextsSituations specifically designed to create human errors should be included to assess the error tolerance of the system and the capability of operators to recover from errors should they occur.

• High-workload conditionsThe sample should include situations where human performance variation due to high workload and multitasking situations can be assessed.

• Varying-workload situationsThe sample should include situations where human performance variation due to workload transitions can be assessed. These include conditions that exhibit (1) a sudden increase in the number of signals that must be detected and processed following a period in which signals were infrequent and (2) a rapid reduction in signal detection and processing demands following a period of sustained high task demand.

• Fatigue and circadian factorsThe sample should include situations where human performance variation due to personnel fatigue and circadian factors can be assessed.

• Environmental factorsThe sample should include situations where human performance variation due to environmental conditions such as poor lighting, extreme temperatures, high noise, and simulated radiological contamination can be assessed.

The Staffs Evaluation of Criterion 3 The V&V IP, Section 4.1.1, Sampling Dimensions, contains a list of situational factors included in the operational conditions sample. The list addresses all elements contained in the acceptance criteria with the exception of Operationally difficult tasks, and Fatigue and circadian factors. Operationally difficult tasks are fully addressed within the previous section listing personal tasks. Fatigue is satisfactorily addressed by a specific line item addressing high frequency tasks. The staff has found that circadian factors are impractical to consider in the ISV and notes that it is better addressed by 10 CFR Part 26, Fitness for duty programs.

Accordingly, the staff concludes that the applicants treatment of plant conditions included in the situational factors sample conforms to this acceptance criterion.

18.10.4.1.2 Identification of Scenarios Criterion 1 The results of the sampling should be combined to identify a set of scenarios to guide subsequent analyses. A given scenario may combine many of the characteristics identified by the operational event sampling.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.1.2, Identification of Scenarios, explains that scenario developers, using expert judgment, establish the goals and conditions to be included for each scenario selected based on the OCS. During the identification of the scenarios, a comparative table is developed that compares the OCS criteria and each scenario. The scenario developers use 18-88

this table to assure that all OCS criteria are addressed at least once by the composite set of scenarios. Accordingly, the staff concludes that the applicants treatment of scenario identification conforms to this acceptance criterion.

Criterion 2 The scenarios should not be biased in the direction of over representation of the following:

• scenarios for which only positive outcomes can be expected

• scenarios that for integrated system validation are relatively easy to conduct administratively (scenarios that place high demands, data collection or analysis are avoided)

• scenarios that for integrated system validation are familiar and well structured (e.g., which address familiar systems and failure modes that are highly compatible with plant procedures such as textbook design-basis accidents)

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.3.4, Scenario Definition, provides specific direction to the scenario developers that scenarios should not be biased in the areas listed in the acceptance criterion.

This is reinforced by the detailed descriptions of conditions that are to be addressed in the operational conditions sample; many challenging conditions are described. Also, an independent expert panel reviews and approves the completed scenarios to ensure that the V&V IP guidance has been followed. Accordingly, the staff concludes that the applicants treatment of scenario bias conforms to this acceptance criterion.

18.10.4.2 Design Verification 18.10.4.2.1 Inventory and Characterization Criterion 1 Scope - The applicant should develop an inventory of all HSI components associated with the personnel tasks based on the identified operational conditions. The inventory should include aspects of the HSI that are used for interface management such as navigation and display retrieval in addition to those that control the plant.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.2.1, HSI Inventory and Characterization, states that the HSI inventory includes all of the HSI components associated with the personnel tasks identified in the operational conditions sample. This matches the scope identified in the acceptance criterion.

The navigation and display retrieval are addressed as a HSI characteristic which is addressed in more detail under the next criterion. Accordingly, the staff concludes that the applicants treatment of the HSI inventory scope conforms to this acceptance criterion.

18-89

Criterion 2 HSI Characterization - The inventory should describe the characteristics of each HSI component within the scope of the review. The following is a minimal set of information for the characterization:

• a unique identification code number or name

• associated plant system and subsystem

• associated personnel functions/subfunction

• type of HSI component

- computer-based control (e.g., touch screen or cursor-operated button and keyboard input)

- hardwired control (e.g., J-handle controller, button, and automatic controller)

- computer-based display (e.g., digital value and analog representation)

- hardwired display (e.g., dial, gauge, and strip chart recorder)

• display characteristics and functionality (e.g., plant variables/parameters, units of measure, accuracy of variable/parameter, precision of display, dynamic response, and display format (bar chart, and trend plot))

• control characteristics and functionality (e.g., continuous versus discrete settings, number and type of control modes, accuracy, precision, dynamic response, and control format (method of input))

• user-system interaction and dialog types (e.g., navigation aids and menus)

• location in data management system (e.g., identification code for information display screen)

• physical location in the HSI (e.g., control panel section), if applicable.

Photographs, copies of video display unit screens, and similar samples of HSI components should be included in the HSI inventory and characterization.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.2.1, HSI Inventory and Characterization, lists the information needed to characterize each component in the HSI inventory. The list includes all the characterization information contained in the acceptance criterion. Accordingly, the staff concludes that the applicants treatment of the HSI inventory characterization conforms to this acceptance criterion.

18-90

Criterion 3 Information Sources - The inventory should be based on the best available information sources. Equipment lists, design specifications, and drawings describe HSI components. These descriptions should be compared by directly observing the components, both hardwired and computer-generated, to verify that the inventory accurately reflects their current state.

The Staffs Evaluation of Criterion 3 The MUAP-10012, Section 4.2.1, HSI Inventory and Characterization, identifies component control guides, equipment lists, design specifications, and software databases as information sources for the HSI characterization. The characterization descriptions are represented in a PC-based tool. The PC-based tool also provides displays that represent control panels, including the diverse HSI panel and local HSIs.

Before the verification program uses the PC-based tool, the V&V team verifies the accuracy of the tool by comparing a sample, based on the identified scenarios, of the tool-generated displays to the same displays generated by the actual plant control and monitoring system (PCMS) and protection and safety monitoring system (PSMS) software. If the team finds discrepancies, the PC-based tool is modified before the start of the verification and re-verified by taking a new 10 percent sample of those displays not included in the scenarios. If additional discrepancies are found during the re-verification, the tool is modified to fix the discrepancies and 100 percent of the remaining displays are verified.

The staff concludes that the sampling technique used to verify that the design characteristics has been accurately transposed into the PC-based tool is acceptable. There are a large number of design characteristics, so checking a 10 percent sample provides a reasonable verification that the design characteristics are consistent with the current HSI configurations. Accordingly, the staff finds that the applicants treatment of inventory information sources conforms to this acceptance criterion.

18.10.4.2.2 Human-System Interface Task Support Verification Criterion 1 Criteria Identification - The criteria for Task Support Verification come from task analyses of HSI requirements for performance of personnel tasks that are selected operational conditions should be defined. [That is, the criteria for Task Support Verification are the HSI requirements identified by task analysis.]

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.2.2, HSI Task Support Verification, states that the verification criteria comes from the most recent TA and reflect both manual actions and tasks where the operators supervise and back up automation. Accordingly, the staff finds that the applicants treatment of task support verification criteria conforms to this acceptance criterion.

18-91

Criterion 2 General Methodology - The HSIs and their characteristics (as defined in the HSI inventory and characterization) should be compared to the personnel task requirements identified in the task analysis.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.2.2, HSI Task Support Verification, states that a three-member multidisciplinary task support verification team, following a procedure containing the task verification criteria, conducts a detailed comparison of the personnel task requirements identified by the TA with the available alarms, displays, information sources, and control capabilities in the HSI inventory. Section 4.2.2 provides a list of specific, proprietary items addressed within the task support verification. Results from the teams decision are independently reviewed by an expert panel. These process elements provide reasonable assurance that the task verification will be effective.

The staff concludes that the task support verification methodology provides for a detailed verification that personnel task requirements are properly incorporated within the HSI design and additionally provides for:

• Minimizing bias through the use of a team review and an independent check of the teams results.

• Appropriate qualifications of the subject matter experts. The qualifications of the team members are listed in MUAP-10012, Table 5-1, V&V Implementation Summary, and in MUAP-09019, Table 3, HFE Team General Qualifications.

These qualifications are consistent with the guidance in NUREG-0700, Section 2, HFE Program Management.

Accordingly, the staff finds that the applicants treatment of task verification methodology conforms to this acceptance criterion.

Criterion 3 Task Requirements Deficiencies - HEDs should be identified when:

• an HSI needed for task performance (e.g., a [needed] control or display) is not available

• HSI characteristics do not match the personnel task requirements, e.g., a display shows the necessary plant parameter but not the range or precision needed for the task.

The Staffs Evaluation of Criterion 3 MUAP-10012, Section 4.2.2, HSI Task Support Verification, states that HEDs are to be written when either condition stated in the acceptance criteria is identified. The HEDs are also written when the verification team cannot reach a consensus that the HSI design supplies the needed 18-92

interface identified by the TA. Accordingly, the staff finds that the applicants treatment of the HEDs conforms to this acceptance criterion.

Criterion 4 Unnecessary HSI Component - An HED should be identified for HSIs that are available in the HSI but are not needed for any task. Unnecessary HSIs introduce clutter and can distract personnel for the selection of appropriate HSIs.

It is important to verify that the HSI is actually unnecessary. Appropriate HSI components may not appear to be associated with personnel tasks for the following reasons:

• The HSI component is needed for a task that was not addressed by the task analysis (e.g., it was not within the scope of the design review).

• The task analysis was incomplete, and thus overlooked the need for the HSI component.

• The HSI component only partially meets the personnel task requirements that were established.

If an HSI component has no associated personnel tasks because the function and task analysis was incomplete, then the applicant should identify and resolve any shortcomings in that analysis.

The Staffs Evaluation of Criterion 4 MUAP-10012, Section 4.2.2, HSI Task Support Verification, states that a HED will be written for HSIs that are available but not needed. It also includes direction that the HED resolution process will confirm that the HSI is actually not needed and provides a list of things to check as part of the resolution. The list includes the bulleted items from the acceptance criteria. The direction clearly states that if it is determined that the TA was incomplete the shortcomings will be addressed through the HED process. Accordingly, the staff finds that the applicants treatment of unnecessary HSI components conforms to this acceptance criterion.

Criterion 5 HED Documentation - HEDs should be documented to identify the HSI, the relevant task criterion, and basis for the deficiency (what aspect of the HSI has been identified as not meeting task requirements).

The Staffs Evaluation of Criterion 5 MUAP-10012, Section 4.2.2, HSI Task Support Verification, states that the HED documentation includes the relevant task criteria and bases for the deficiency. This is consistent with MUAP-09019, the HFE Program Plan, Table 5.1, HED Creation Data Fields, which lists the information documented in a HED. This information includes a description of the HSI and associated deficiency, the basis for the deficiency (Guidance), and a reference to information related to the HED (Design Reference). These elements completely address the areas identified in the acceptance criteria. Accordingly, the staff finds that the applicants treatment of the HED documentation conforms to this acceptance criterion.

18-93

18.10.4.2.3 Human Factors Engineering Design Verification Criterion 1 Criteria Identification - The criteria for this verification are the HFE guidelines.

The selection of guidelines used in the review depends upon the characteristics of the HSI components included in the scope of the review, as defined in the HSI characterization. It also depends upon whether the applicant has developed a style guide (design-specific HFE guideline document). When a style guide is used by the applicant, its acceptability should be reviewed by the staff. The procedures involved are described in [NUREG 0711] Section 8.4.5. The HFE guidelines contained in NUREG-0700 may be used to support the staff's review of the guidance contained in an applicant's style guide. When an NRC reviewed Style Guide has been used, it can provide the criteria for HFE design verification.

When no style guide is available, the guidelines in NUREG-0700 can be used for the HFE design verification. However, since not all of these guidelines will be applicable to each review, the selection of guidelines should be based on the characteristics of the HSI components being evaluated. A subset of guidelines appropriate to the specific design implementation should be identified based on the HSI characterization.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.2.3, HFE Design Verification, states that the APWR HSI Design Style Guide is the principle design specification against which the detailed HSI design is compared.

The APWR HSI Design Style Guide is an acceptable source of criteria. It was previously reviewed and found to conform to the guidance in NUREG-0700 (see Section 18.7 of this report). NUREG-0700 and industry good practices are used as secondary sources for aspects of the design that are not addressed by the style guide. This approach provides a comprehensive set of criteria to verify the design. Accordingly, the staff finds that the applicants treatment of the design verification criteria conforms to this acceptance criterion.

Criterion 2 General Methodology - The characteristics of the HSI components should be compared with HFE guidelines. These guidelines are applicable to different aspects of the design: task-independent features (e.g., font size), task-specific features (e.g., scale units), and task-integration features (e.g., proximity of control-display).

A single guideline may apply to many identical HSI components, especially in the case of significant HSI modifications and HSIs for new plants. In addition, some environmental considerations (e.g., lighting) may be applicable. To simplify the application of guidelines and reduce redundancy when reporting findings, the guidelines may be applied to features of the HSI as follows:

• Global featuresglobal HSI features are those relating to the configurational and environmental aspects of the HSI, such as MCR layout, general workstation 18-94

configuration, lighting, noise, heating, and ventilation. These aspects of the review, e.g., MCR lighting, tend to be evaluated only once.

• Standardized featuresstandardized features are those that were designed using HFE guidelines applied across individual controls and displays (e.g.,

display screen organization, display format conventions, and coding conventions). Therefore, their implementation should be more consistent across the interface than features that were not designed with guidelines. Thus, for example, if display labeling is standardized by the applicants HFE guidelines (style guide), which have been accepted by the NRC, then display labels can be spot-checked rather than being verified individually.

• Detailed featuresdetailed features are the aspects of individual HSIs that are not addressed by general HFE guidelines. The latter can be expected to be more variable than the standardized design features.

For each guideline, it should be determined whether the HSI is "acceptable" or "discrepant" from the guideline (therefore, potentially unacceptable), i.e., an HED. Acceptable should be indicated only if there is total compliance, i.e., only if every instance of the item is fully consistent with the criteria established by the HFE guidelines. If there is any instance of noncompliance, full or partial, then an evaluation of discrepant conditions should be given, and a notation made as to where noncompliance occurs.

Discrepancies should be evaluated as potential indicators of additional issues.

For example, identifying an inappropriate format for presenting data on an individual display should be considered a potential sign that other display formats could be incorrectly used or that the observed format is inappropriately used elsewhere. As a result, the sampling strategy could be modified to encompass other display formats. In some cases, discovering these discrepancies could warrant further review in the identified areas of concern.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.2.3, HFE Design Verification, states that the sample of the US-APWR HSIs identified from the OCS is reviewed to confirm application of the HFE design principles across the entire HSI. To simplify the verification, design specifications are applied to the HSI based on the level of its features as defined in the acceptance criterion above. A team of three HFE experts perform the verification and reach a consensus regarding the status of the HSIs verified. Dissenting opinions of the experts are documented in the final results. The HEDs document any HSI configuration that does not conform to the HFE design principles. HEDs are evaluated by the HFE design verification team to identify the extent of the discrepancy and potential indicators of additional issues across the HSI. As a result, the sampling based on the OCS is expanded to encompass other display and control formats of the HSI, where appropriate. Final verification results are reviewed by an independent expert panel.

The staff finds that this methodology addresses all elements in the acceptance criteria and additionally provides for:

• Minimizing bias through the use of a team review and an independent check of the teams results.

18-95

• Appropriate qualifications of the subject matter experts. The qualifications of the team members are listed in MUAP-10012, Table 5-1, V&V Implementation Summary, and in MUAP-09019, Table 3, HFE Team General Qualifications.

These qualifications are consistent with the guidance in NUREG-0700, Section 2, HFE Program Management.

Accordingly, the staff finds that the applicants treatment of the design verification methodology conforms to this acceptance criterion.

Criterion 3 HED Documentation - HEDs should be documented by the applicant in terms of the HSI component involved and how its characteristics depart from a particular guideline.

The Staffs Evaluation of Criterion 3 MUAP-10012, Section 4.2.3, HFE Design Verification, states that the HED documentation includes the HSI component and its deficiency. This is consistent with MUAP-09019, the HFE Program Plan, Table 5.1, HED Creation Data Fields, which lists the information documented in a HED. This information includes a description of the HSI and associated deficiency, the basis for the deficiency (Guidance), and a reference to information related to the HED (Design Reference). These elements completely address the areas identified in the acceptance criteria.

Accordingly, the staff finds that the applicants treatment of HED documentation conforms to this acceptance criterion.

18.10.4.3 Integrated System Validation The objective of reviewing integrated system validation methodology is to verify that the applicants methodology will validate the integrated system design (i.e., hardware, software, and personnel elements) using performance-based tests that will determine whether it acceptably supports safe operation of the plant.

18.10.4.3.1 Test Objectives Criterion 1 Detailed objectives should be developed to provide evidence that the integrated system adequately supports plant personnel in the safe operation of the plant.

The test objectives and scenarios should be developed to address aspects of performance that are affected by the modification [of the] design, including personnel functions and tasks affected by the modification. The objectives should be to:

• Validate the role of plant personnel.

• Validate that the shift staffing, assignment of tasks to crew members, and crew coordination (both within the control room as well as between the control room and local control stations and support centers) is acceptable. This should include validation of the nominal shift levels, minimal shift levels, and shift turnover.

18-96

• Validate that for each human function, the design provides adequate alerting, information, control, and feedback capability for human functions to be performed under normal plant evolutions, transients, design-basis accidents, and selected, risk-significant events that are beyond-design basis.

• Validate that specific personnel tasks can be accomplished within time and performance criteria, with a high degree of operating crew situation awareness, and with acceptable workload levels that provide a balance between a minimum level of vigilance and operator burden. Validate that the operator interfaces minimize operator error and provide for error detection and recovery capability when errors occur.

• Validate that the crew can make effective transitions between the HSIs and procedures in the accomplishment of their tasks and that interface management tasks such as display configuration and navigation are not a distraction or undue burden.

• Validate that the integrated system performance is tolerant of failures of individual HSI features.

• Identify aspects of the integrated system that may negatively affect integrated system performance.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.1, Test Objectives, provides a list of objectives that includes those identified in the acceptance criteria. Additional objectives are added to address important human actions credited in the Design Basis Event analyses and the Diverse Actuation System (DAS) analysis. Accordingly, the staff finds that the applicants treatment of ISV test objectives conforms to this acceptance criterion.

18.10.4.3.2 Validation Testbeds Review Criteria (1) through (7) in NUREG-0711, Section 11.4.3.2.2, Validation Testbeds, provide for the review of simulation testbed fidelity. The NUREG states that one approach to identifying a testbed that meets the staffs fidelity criteria is to ensure its compatibility with ANSI/ANS 3.5. The applicants ISV testbed is described in the MUAP-10012, Section 4.3.2, Validation Test bed, which states that a full-scope simulator meeting the guidance of ANS 3.5-2009 will be used for ISV. Accordingly, the staff finds that the applicants treatment of simulator fidelity, as described in Criterion (1) through (7), conforms to these acceptance criterion. For completeness, the individual criteria are listed below. Testbed review Criteria 8 and 9 are reviewed below.

Criterion 1 Interface Completeness - The testbed should completely represent the integrated system. This should include HSIs and procedures not specifically required in the test scenarios. For example, adjacent controls and displays may affect the ways 18-97

in which personnel use those that are addressed by a particular validation scenario.

The Staffs Evaluation of Criterion 1 The applicants testbed meets this criterion by reference to ANS 3.5.

Criterion 2 Interface Physical Fidelity - A high degree of physical fidelity in the HSIs and procedures should be represented, including presentation of alarms, displays, controls, job aids, procedures, communications, interface management tools, layout and spatial relationships.

The Staffs Evaluation of Criterion 2 The applicants testbed meets this criterion by reference to ANS 3.5.

Criterion 3 Interface Functional FidelityA high degree of functional fidelity in the HSIs and procedures should be represented. All HSI functions should be available. High functional fidelity includes HSI component modes of operation, i.e., the changes in functionality that can be invoked on the basis of personnel selection and/or plant states.

The Staffs Evaluation of Criterion 3 The applicants testbed meets this criterion by reference to ANS 3.5.

Criterion 4 Environment Fidelity - A high degree of environment fidelity should be represented. The lighting, noise, temperature, and humidity characteristics should reasonably reflect that expected. Thus, noise contributed by equipment, such as air handling units and computers should be represented in validation tests.

The Staffs Evaluation of Criterion 4 The applicants testbed meets this criterion by reference to ANS 3.5.

Criterion 5 Data Completeness Fidelity - Information and data provided to personnel should completely represent the plant systems monitored and controlled from that facility.

The Staffs Evaluation of Criterion 5 The applicants testbed meets this criterion by reference to ANS 3.5.

18-98

Criterion 6 Data Content Fidelity - A high degree of data content fidelity should be represented. The information and controls presented should be based on an underlying model that accurately reflects the reference plant. The model should provide input to the HSI in a manner such that information accurately matches that which will actually be presented.

The Staffs Evaluation of Criterion 6 The applicants testbed meets this criterion by reference to ANS 3.5.

Criterion 7 Data Dynamics Fidelity - A high degree of data dynamics fidelity should be represented. The process model should be capable of providing input to the HSI in a manner such that information flow and control responses occur accurately and in a correct response time; e.g., information should be provided to personnel with the same delays as would occur in the plant.

The Staffs Evaluation of Criterion 7 The applicants testbed meets this criterion by reference to ANS 3.5.

Criterion 8 For important actions at complex HSIs remote from the main control room, where timely and precise human actions are required, the use of a simulation or mockup should be considered to verify that human performance requirements can be achieved. (For less risk-important HAs or where the HSIs are not complex, human performance may be assessed based on analysis such as task analysis rather than simulation.)

The Staffs Evaluation of Criterion 8 MUAP-10012, Section 4.3.2, Validation Test bed, [sic] states that HSIs remote from the MCR will be validated in accordance with MUAP-10013, Design Implementation Plan. MUAP-10013, Section 4.5.1, Local HSI Design, states that SMEs walk through the operating procedures that invoke the use of a particular local HSI to confirm that the HSI meets the plant operational needs. The SMEs document the procedure and the steps within that procedure that are used to confirm each Local HSI. Problems are documented on the HEDs.

The staff concludes that using the as-built HSI to demonstrate human performance objectives can be achieved, while increasing the potential of HSI modifications, provides a direct measure of the HSIs effectiveness in supporting human performance. Accordingly, the staff finds that the applicants treatment of remote HSI validation conforms to this acceptance criterion.

18-99

Criterion 9 The testbeds should be verified for conformance to the testbed characteristics identified above before validations are conducted.

The Staffs Evaluation of Criterion 9 MUAP-10012, Section 4.3.1, Test Objectives, states that the testbed simulator is verified and accepted based on the simulator design specification for conformance to ANSI/ANS 3.5-2009 before the ISV starts. Accordingly, the staff finds that the applicants treatment of testbed verification conforms to this acceptance criterion.

18.10.4.3.3 Plant Personnel Criterion 1 Participants in the validation tests should be representative of actual plant personnel who will interact with the HSI, e.g., licensed operators rather than training or engineering personnel.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.3, Plant Personnel, states that the ISV operating crew personnel have or have had the same qualifications and licenses for the positions held in the ISV as those required by the NRC for currently operating plants. Accordingly, the staff finds that the applicants treatment of the ISV operating crew participants conforms to this acceptance criterion.

Criterion 2 To properly account for human variability, a sample of participants should be used. The sample should reflect the characteristics of the population from which the sample is drawn. Those characteristics that are expected to contribute to system performance variation should be specifically identified and the sampling process should provide reasonable assurance that variation along that dimension is included in the validation. Several factors that should be considered in determining representativeness include: license and qualifications, skill/experience, age, and general demographics.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.3, Plant Personnel, states that the ISV crews will be selected from a pool of experienced U.S. plant staff that represent standard industry distributions for age, gender, education level, and experience. A minimum of three crews are randomly selected from the pool.

The staff concludes that the crew selection from an operator pool established using these factors and conducting tests with a minimum of three crews, reasonably accounts for human variability. Accordingly, the staff finds that the applicants treatment of the ISV operating crew selection conforms to this acceptance criterion.18-100

Criterion 3 In selection of personnel, consideration should be given to the assembly of minimum and normal crew configurations, including shift supervisors, reactor operators, shift technical advisors, etc., that will participate in the tests.

The Staffs Evaluation of Criterion 3 MUAP-10012, Section 4.3.3, Plant Personnel, states that the crew size for the validation tests includes a range of expected sizes to assure that the HSI supports operations and event management. The minimum staffing level is one RO and one SRO in accordance with design assumptions for Mode 1 and 2 operations. Maximum manning is 11 people in the control room to simulate larger numbers of secondary personnel, such as technicians and accident response personnel in the control room. Modes 3 - 6 staffing requirements as identified in the staffing analysis are also addressed. The crew positions considered, include shift supervisors, ROs, SROs, and shift technical advisors.

The staff concludes that the ISV staffing plans address the full range of crew configurations.

Accordingly, the staff finds that the applicants treatment of the ISV operating crew size conforms to this acceptance criterion.

Criterion 4 To prevent bias in the sample, the following participant characteristics and selection practices should be avoided:

• participants who are part of the design organization,

• participants in prior evaluations. and

• participants who are selected for some specific characteristic, such as using crews that are identified as good or experienced.

The Staffs Evaluation of Criterion 4 MUAP-10012, Section 4.3.3, Plant Personnel, provides a list of practices that will be avoided in the crew selection. This list includes the first two bullets. The last bullet is accomplished by the random selection process used to select the ISV crews from the operator pool of qualified candidates. Accordingly, the staff finds that the applicants treatment of the ISV operating crew selection conforms to this acceptance criterion.

18.10.4.3.4 Scenario Definition Criterion 1 The operational conditions selected for inclusion in the validation tests should be developed in detail so they can be performed on a simulator. The following information should be defined to provide reasonable assurance that important performance dimensions are addressed and to allow scenarios to be accurately and consistently presented for repeated trials:

18-101

• description of the scenario and any pertinent "prior history" necessary for personnel to understand the state of the plant upon scenario start-up,

• specific initial conditions (precise definition provided for plant functions, processes, systems, component conditions and performance parameters, e.g.,

similar to plant shift turnover),

• events (e.g., failures) to occur and their initiating conditions, e.g., time, parameter values, or events,

• precise definition of workplace factors, such as environmental conditions,

• task support needs (e.g., procedures and technical specifications),

• staffing objectives,

• communication requirements with remote personnel (e.g., load dispatcher via telephone),

• the precise specification of what, when and how data are to be collected and stored (including videotaping requirements, questionnaire and rating scale administrations), and

• specific criteria for terminating the scenario.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.4, Scenario Definition, provides general directions for scenario development and includes a statement that the operational conditions selected for inclusion in the validation tests should be developed in detail so that they can be performed on a simulator and to allow scenarios to be accurately and consistently presented for repeated trails. Test procedures are used to provide specific direction and in Section 4.3.6.2, Test Procedures, a list of scenario inputs provides detailed directions regarding what is to be included. This list includes all the elements from this acceptance criterion along with additional guidance on data collection and communication practices. This list becomes the frame work for Attachment A, the scenario format template, which is used by the scenario developers to develop specific scenarios. The template incorporates all the elements described in the acceptance criteria along with additional direction and standardized wording that helps ensure that repeated trials will be accurate and consistent. Accordingly, the staff finds that the applicants treatment of the scenario development conforms to this acceptance criterion.

Criterion 2 Scenarios should have appropriate task fidelity so that realistic task performance will be observed in the tests and so that test results can be generalized to actual operation of the real plant.18-102

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.4, Scenario Definition, restates the criterion as one of the objectives of scenario development. The example scenarios provided in Appendix A demonstrates effective implementation of this guidance by their use of detailed, realistic task descriptions.

The staff also notes that simulator exercises used in the Phase 1 testing, used realistic tasks that allowed generalization of the test results to actual operation. The use of a full scope simulator facilitates implementation of this acceptance criterion. Given the procedural direction contained in MUAP-10012, previous demonstrated performance, and the use of a full scope simulator, the staff concludes that there is reasonable assurance that the test results can be generalized to actual plant operation. Accordingly, the staff finds that the applicants treatment of the scenario development conforms to this acceptance criterion.

Criterion 3 When evaluating performance associated with operations remote from the main control room, the effects on crew performance due to potentially harsh environments (i.e., high radiation) should be realistically simulated (i.e., additional time to don protective clothing and access radiologically controlled areas).

The Staffs Evaluation of Criterion 3 MUAP-10012, Section 4.3.4, Scenario Definition, incorporates this acceptance criterion within the direction for developing scenarios that incorporate local actions. The applicant has a well-defined method accounting for travel times and impacts on travel time including the environmental impact identified in the acceptance criterion. This method is explained in the TA IP (MUAP-13009), and evaluated in more detail in Section 18.4 of this report. Section 4.3.6.2, Test Procedures, and Appendix A, the scenario format template, translate this direction into the working tools used by the scenario developers. Accordingly, the staff finds that the applicants treatment of the scenario development conforms to this acceptance criterion.

18.10.4.3.5 Performance Measurement The review of performance measurement covers measurement characteristics, performance measure selection, and performance criteria.

18.10.4.3.5.1 Measurement Characteristics Criterion 1 Performance Measurement Characteristics - Performance measures should acceptably exhibit the following measurement characteristics to provide reasonable assurance that the measures are of good quality (it should be noted that some of the characteristics identified below may not apply to every performance measure):

• Construct ValidityA measure should accurately represent the aspect of performance to be measured.

• DiagnosticityA measure should provide information that can be used to identify the cause of acceptable or unacceptable performance.18-103

• ImpartialityA measure should be equally capable of reflecting good as well as bad performance.

• ObjectivityA measure should be based on phenomena that are easily observed.

• ReliabilityA measure should be repeatable; i.e., if the same behavior is measured in exactly the same way under identical circumstances, the same measurement result should be obtained.

• ResolutionA measure should reflect the performance at an appropriate level of resolution, i.e., with sufficient detail to permit a meaningful analysis.

• SensitivityA measure's range (scale) and the frequency of measurement (how often data are collected) should be appropriate to the aspect of performance being assessed.

• SimplicityA measure should be simple both from the standpoint of executing the tests and from the standpoint of communicating and comprehending the meaning of the measures.

• UnintrusivenessA measure should not significantly alter the psychological or physical processes that are being investigated.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.5, Performance Measures Characteristics, lists the characteristics which are applicable to the performance measures used to judge the ISV performance. The list incorporates all the characteristics listed in the acceptance criterion. MUAP-10012, Table C-1 Performance Measurement Characteristics, of Appendix C, Performance Measures, lists the performance measures applied to the ISV. For each performance measure, Table C-1 also documents the relevancy of each measurement characteristic listed above. While not all characteristics are relevant to each measure, the applicant has provided a thorough analysis of why each measurement was chosen relative to the strengths and weaknesses of the measurement. The results provide reasonable assurance that the measures chosen are of good quality. Accordingly, the staff finds that the applicants treatment of measurement characteristics conforms to this acceptance criterion.

18.10.4.3.5.2 Performance Measure Selection Criterion 1 A hierarchal set of performance measures should be used which includes measures of the performance of the plant and personnel (i.e., personnel tasks, situation awareness, cognitive workload, and anthropometric/physiological factors). Some of these measures could be used as "pass/fail" criteria for validation and the others to better understand personnel performance and to 18-104

facilitate the analysis of performance errors. The applicant should identify which are in each category.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.5.2, Performance Measure Selection, states that there are two hierarchies, those performance measures selected to validate the design (pass/fail) and those that identify design improvements by providing information to better understand performance of the design (performance improvement measures). The category each measure falls into is clearly identified in Table C-1 of Appendix C.

Section 4.3.5.2 identifies the following measures to be used to assess the ISV performance.

Personnel task measurement:

• Situational awareness.

• Cognitive workload.

• Anthropometric and physiological factors.

The staff concludes that the performance measures used address the appropriate elements of plant and personnel performance. More critical measures have been identified as pass/fail while other measures are available to capture potential improvements. Accordingly, the staff finds that the applicants treatment of performance measurement selection conforms to this acceptance criterion.

Criterion 2 Plant Performance Measurement - Plant performance measures representing functions, systems, components, and HSI use should be obtained.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.5.3, Plant Performance Measurement, states that the plant performance resulting from operator action or in-action, includes plant process data (i.e.,

pressures, temperatures, flows, levels, radiation level) and component states (i.e., off/on; open/closed) as a function of time. The data covers the entire plant from the reactor to switchyard. The simulator collects the full range of possible values for a given parameter every second providing enough sensitivity to indicate whether the plant personnel are able to respond in a timely manner to the scenario demands. The data collected is used to identify variations from expected values and as a way to compare performance across crews. Specific measures include parameter limits derived from analyses and procedures, initiation of operator actions, and confirmation of response times.

The staff concludes that plant performance data is appropriately collected and subjected to a set of measures that identify performance challenges. Accordingly, the staff finds that the applicants treatment of performance measurement selection conforms to this acceptance criterion.18-105

Criterion 3 Personnel Task Measurement - For each specific scenario, the tasks that personnel are [needed] to perform should be identified and assessed. Two types of personnel tasks should be measured: primary (e.g., start a pump), and secondary (e.g., access the pump status display). Primary tasks are those involved in performing the functional role of the operator to supervise the plant; i.e., monitoring, detection, situation assessment, response planning, and response implementation. Secondary tasks are those personnel must perform when interfacing with the plant, but which are not directed to the primary task, such as navigation and HSI configuration. This analysis should be used for the identification of potential errors of omission.

• Primary tasks should be assessed at a level of detail appropriate to the task demands. For example, for some simple scenarios, measuring the time to complete a task may be sufficient. For more complicated tasks, especially those that may be described as knowledge-based, it may be appropriate to perform a more fine-grained analysis such as identifying task components: seeking specific data, making decisions, taking actions, and obtaining feedback. Tasks that are important to successful integrated system performance and are knowledge-based should be measured in a more fine-grained approach.

• The measurement of secondary tasks should reflect the demands of the detailed HSI implementation, e.g., time to configure a workstation, navigate between displays, and manipulate displays (e.g., changing display type and setting scale).

• The tasks that are actually performed by personnel during simulated scenarios should be identified and quantified. (Note that the actual tasks may be somewhat different from those that should be performed). Analysis of tasks performed should be used for the identification of errors of commission.

• The measures used to quantify tasks should be chosen to reflect the important aspects of the task with respect to system performance, such as:

- time

- accuracy

- frequency

- errors (omission and commission)

- amount achieved or accomplished

- consumption or quantity used

- subjective reports of participants

- behavior categorization by observers The Staffs Evaluation of Criterion 3 MUAP-10012, Section 4.3.5.4, Personnel Task Measurement, provides a definition, examples and a discussion of how primary and secondary tasks are identified and measured.

Measurements include the list contained in the acceptance criterion and are sufficiently detailed to allow for an appropriate evaluation of the task. Knowledge based tasks, for example, are 18-106

measured using a proprietary list of measures that go beyond the regulatory guidance and provide for a thorough analysis of the complexity of these tasks.

The applicant also provides measures for parallel and potentially distracting tasks, such as implementation of the site emergency plan. Data capture from cameras, screen views, and the simulator, documents actual actions and allows a comparison to what was expected. The comparison between actual and expected is an important method to identify errors of omission and commission. Collectively, these elements provide for the effective evaluation of personnel task performance. Accordingly, the staff finds that the applicants treatment of personnel task measurement conforms to this acceptance criterion.

Criterion 4 Situation Awareness - Personnel situation awareness should be assessed. The approach to situation awareness measurement should reflect the current state-of-the-art.

The Staffs Evaluation of Criterion 4 MUAP-10012, Section 4.3.5.5, Situation Awareness, provides a detailed discussion of the data collection tools and measures for assessing situation awareness. In summary, a 3-part model is used that addresses:

• Perception of items in the environment.

• Comprehension of the items (why the items are important at that time and place).

• Projection of the items values into the future.

To measure situation awareness, the ISV applies a combination of objective measures along with subjective post-scenario questionnaire methods. Both intrusive and nonintrusive measures are used to collect data, and intrusive measures are managed so that they have minimum impact on the operators performance.

Collectively, these elements are consistent with the industry best practices and reflect the current state-of-the-art for assessing situational awareness. Accordingly, the staff finds that the applicants treatment of situational awareness conforms to this acceptance criterion.

Criterion 5 Cognitive Workload - Personnel workload should be assessed. The approach to workload measurement should reflect the current state-of-the-art.

The Staffs Evaluation of Criterion 5 MUAP-10012, Section 4.3.5.6, Cognitive Workload, describes a four part method for evaluating cognitive workload. The four parts are:

• A post-scenario questionnaire is administered that includes Likert-scale rating questions soliciting the operators subjective self-assessment of mental workload.18-107

• Scripted queries are introduced that ask the operator to gather specific plant information. The operators verbal response and the time it takes to gather the information and feed it correctly back while managing the scenario is collected and evaluated as an indirect measure of the level of cognitive workload and how much reserve capacity the operator has at a given time.

• Qualitative information from the observation of the crew performance during the scenarios and verbal debriefs that are conducted at the conclusion of the scenarios provide important background information for interpreting the results of the data.

• The National Aeronautics and Space Administration Task Load Index is used as another subjective measurement of workload.

Collectively, these elements are consistent with the industry best practices and reflect the current state-of-the-art for assessing cognitive workload. Accordingly, the staff finds that the applicants treatment of cognitive workload conforms to this acceptance criterion.

Criterion 6 Anthropometric and Physiological Factors - Anthropometric and physiological factors include such concerns as visibility of indications, accessibility of control devices, and ease of control device manipulation that should be measured where appropriate. Attention should be focused on those aspects of the design that can only be addressed during testing of the integrated system, e.g., the ability of personnel to effectively use the various controls, displays, workstations, or consoles in an integrated manner.

The Staffs Evaluation of Criterion 6 MUAP-10012, Section 4.3.5.7, Anthropometric and Physiological Factors, states that these factors are considered in evaluating the ability of the plant personnel to use the various HSIs individually and as a team during a scenario. The test facility will simulate and the scenario definitions will specify expected plant conditions such as work station layout, background noise, lighting, and display characterization with emphasis on those attributes that are not addressed during the design verification. Anthropometric challenges are collected by test personnel observations during the scenarios or during review of video recordings.

The staff concludes that this approach provides reasonable assurance that the ability of personnel to effectively use the various controls, displays, workstations, or consoles in an integrated manner will be verified. Accordingly, the staff finds that the applicants treatment of anthropometric and physiological factors conforms to this acceptance criterion.

18.10.4.3.5.3 Performance Criteria Criterion 1 Criteria should be established for the performance measures used in the evaluations. The specific criteria that are used for decisions as to whether the design is validated or not should be specified and distinguished from those being used to better understand the results.18-108

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.5.9, Criteria Used, discusses the applicants general approach to using acceptance criteria. MUAP-10012, Table C-1 of Appendix C lists the performance measures applied to the ISV. For each performance measure, Table C-1 specifies the acceptance criteria applicable to the measure and important contextual information about the criterion used. The added context may be a description of the criterion, a range of the value of the performance measure, and/or units of the performance measure. There is a separate column in the table that identifies whether the measurement is to be used as a pass/fail criterion or a more limiting measure designed to identify improvements in the HSI design. Appendix A contains a scenario format template which is used by scenario developers as the template for specific scenarios. It includes a section that identifies the scenario specific measures and performance criteria to be used.

These elements provide the specific criteria needed to validate the design and/or identify improvements in the design. Accordingly, the staff finds that the applicants treatment of the performance criteria selection conforms to this acceptance criterion.

Criterion 2 The basis for criteria should be defined, e.g., requirement-referenced, benchmark referenced, normative referenced, and expert-judgment referenced.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.5.10, Criteria Basis, lists the same basis definitions as listed in the acceptance criterion and states that the basis for each criterion is documented in MUAP-10012, Table C-1 of Appendix C. Accordingly, the staff finds that the applicants treatment of performance criteria bases conforms to this acceptance criterion.

18.10.4.3.6 Test Design 18.10.4.3.6.1 Coupling Crews and Scenarios Criterion 1 Scenario Assignment - Important characteristics of scenarios should be balanced across crews. Random assignment of scenarios to crews is not recommended.

The value of using random assignment to control bias is only effective when the number of crews is quite large. Instead, the validation team should attempt to provide each crew with a similar and representative range of scenarios.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3, Integrated System Validation, states that a minimum of three crews will participate in the ISV and that each crew must successfully achieve the pass/fail criteria for the HFE design to be considered acceptable for that scenario. This acceptance criterion is achieved because all scenarios are performed by all three crews rather than assigning each crew a subset of the scenarios. Accordingly, the staff finds that the applicants treatment of scenario assignment conforms to this acceptance criterion.18-109

Criterion 2 Scenario Sequencing - The order of presentation of scenario types to crews should be carefully balanced to provide reasonable assurance that the same types of scenarios are not always being presented in the same linear position, e.g., the easy scenarios are not always presented first.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.6.1, Coupling Crews and Scenarios, states that the scenario developers, using expert judgment, will vary the scenario sequences to ensure that the testing minimizes training bias and crew expectations of upcoming activities. The sequencing is independently reviewed by the expert panel to verify bias from scenario sequencing has not been introduced. The staff concludes that the combination of expert judgment and independent verification provides reasonable assurance that potential bias form scenario sequencing will be minimized. Accordingly, the staff finds that the applicants treatment of scenario sequencing conforms to this acceptance criterion.

18.10.4.3.6.2 Test Procedures Criterion 1 Detailed, clear, and objective procedures should be available to govern the conduct of the tests. These procedures should include:

• The identification of which crews receive which scenarios and the order that the scenarios should be presented.

• Detailed and standardized instructions for briefing the participants. The type of instructions given to participants can affect their performance on a task. This source of bias can be minimized by developing standard instructions.

• Specific criteria for the conduct of specific scenarios, such as when to start and stop scenarios, when events such as faults are introduced, and other information discussed in Section 11.4.3.2.4, Scenario Definition.

• Scripted responses for test personnel who will be acting as plant personnel during test scenarios. To the greatest extent possible, responses to communications from operator participants to test personnel (serving as surrogate for personnel outside the control room personnel) should be prepared. There are limits to the ability to preplan communications since personnel may ask questions or make requests that were not anticipated. However, efforts should be made to detail what information personnel outside the control room can provide, and script the responses to likely questions.

• Guidance on when and how to interact with participants when simulator or testing difficulties occur. Even when a high-fidelity simulator is used, the 18-110

participants may encounter artifacts of the test environment that detract from the performance for tasks that are the focus of the evaluation.

Guidance should be available to the test conductors to help resolve such conditions.

• Instructions regarding when and how to collect and store data. These instructions should identify which data are to be recorded by:

- simulation computers

- special purpose data collection devices (such as situation awareness data collection, workload measurement, or physiological measures)

- video recorders (locations and views)

- test personnel (such as observation checklists)

- subjective rating scales and questionnaires.

• Procedures for documentation, i.e., identifying and maintaining test record files including crew and scenario details, data collected, and test conductor logs. These instructions should detail the types of information that should be logged (e.g., when tests were performed, deviations from test procedures, and any unusual events that may be of importance to understanding how a test was run or interpreting test results) and when it should be recorded.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.6.2, Test Procedures, provides a list of subjects included in the test procedure. This list includes all the elements from this acceptance criterion. This list becomes the frame work for Attachment A, the scenario format template, which is used by the scenario developers to develop specific scenarios. Accordingly, the staff finds that the applicants treatment of test procedures conforms to this acceptance criterion.

Criterion 2 Where possible, test procedures should minimize the opportunity of tester expectancy bias or participant response bias.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.6.2, Test Procedures, provides specific direction on both ISV performances in general and each specific scenario. By providing a pre-established, disciplined approach to the work being performed, the individual bias and variation is minimized. Examples include:

• direction on scenario ordering so there is no predictable pattern,18-111

• scripted responses are provided for interfaces between the control room and contacts outside the control room,

• standardized crew briefing before each scenario,

• standard guidance on when and how to interact with the operating crew when the simulator encounters difficulties,

• specific criteria for starting and terminating the scenarios, and

• direction to the each crew to refrain from discussing the scenarios with other crews to minimize bias.

Crew selection, training, testing, a formal consensus process, and independent review of the results also work in conjunction with the test procedures to control bias. Collectively, these features of the test program assure consistency, control test bias, support repeatable results, and provide focused attention on each scenario as it is performed. Accordingly, the staff finds that the applicants treatment of testing bias conforms to this acceptance criterion.

18.10.4.3.6.3 Test Personnel Training Criterion 1 Test administration personnel should receive training on:

• the use and importance of test procedures,

• experimenter bias and the types of errors that may be introduced into test data through the failure of test conductors to accurately follow test procedures or interact properly with participants, and

• the importance of accurately documenting problems that arise in the course of testing, even if due to test conductor oversight or error.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.6.3, Test Personnel Training, describes a detailed set of training objectives for the test administration personnel that includes the three items in the acceptance criterion. Significant emphasis is placed on communications between testing personnel, observation documentation, data collection, understanding of the scenarios, and administering the post scenario questionnaires and interviews. Accordingly, the staff finds that the applicants treatment of test administration personnel training conforms to this acceptance criterion.

18.10.4.3.6.4 Participant Training Criterion 1 Participant training should be of high fidelity; i.e., highly similar to that which plant personnel will receive in an actual plant. The participants should be trained to provide reasonable assurance that their knowledge of plant design, plant 18-112

operations, and use of the HSIs and procedures is representative of experienced plant personnel. Participants should not be trained specifically to perform the validation scenarios.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.6.4, Operating Crew, Test Participant Training, states that each operating crew test participant has qualified as either an RO or SRO on a conventional PWR.

Additionally, participants undergo a high-fidelity, hands-on US-APWR training program for the ISV through classroom instruction and operation of the US-APWR simulator taught by training experts who are experienced in licensed operator training. The training goal is to have the participants at a level of proficiency that is representative of the plant personnel who will operate the US-APWR plant. Classroom training includes US-APWR plant systems, operating procedures, and HSIs. Operating crew test participants are trained on numerous plant events, using the simulator, that encompass the knowledge needed to perform the validation scenarios, but they are not trained on the specific validation scenarios.

The staff concludes that the ISV operator training provides for a level of knowledge similar to what the operations staff at operating plants receive and that the training will not introduce specific information about the ISV scenarios. Accordingly, the staff finds that the applicants treatment of operating crew training conforms to this acceptance criterion.

Criterion 2 Participants should be trained to near asymptotic performance (i.e., stable, not significantly changing from trial to trial) and tested prior to conducting actual validation trials. Performance criteria should be similar to that which will be applied to actual plant personnel.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.6.4, Operating crew, Test Participant Training, states that a written test followed by a practical test on the US-APWR simulator is administered to assure near-asymptotic performance and a consistent level of proficiency between individuals making up the operating crews. Training instructors also verify that a consistent level of performance for the individuals and crews has been reached. Accordingly, the staff finds that the applicants treatment of operating crew training conforms to this acceptance criterion.

18.10.4.3.6.5 Pilot Testing Criterion 1 A pilot study should be conducted prior to conducting the integrated validation tests to provide an opportunity to assess the adequacy of the test design, performance measures, and data collection methods.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.6.5, Pilot Testing, discusses pilot testing and contains a list of objectives that includes assessing the adequacy of the test design, performance measures and data collection methods. It is also used to verify that the observer training programs have been 18-113

effective and to identify and address any remaining controllable bias. Accordingly, the staff finds that the applicants treatment of pilot testing conforms to this acceptance criterion.

Criterion 2 If possible, participants who will operate the integrated system in the validation tests should not be used in the pilot study. If the pilot study must be conducted using the validation test participants, then:

• the scenarios used for the pilot study should be different from those used in the validation tests, and

• care should be given to provide reasonable assurance that the participants do not become so familiar with the data collection process that it may result in response bias.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.6.5, Pilot Testing, states that the pilot test will use a crew that is not part of the ISV. Accordingly, the staff finds that the applicants treatment of pilot testing conforms to this acceptance criterion.

18.10.4.3.7 Data Analysis and Interpretation Criterion 1 Validation test data should be analyzed through a combination of quantitative and qualitative methods. The relationship between observed performance data and the established performance criteria should be clearly established and justified based upon the analyses performed.

The Staffs Evaluation of Criterion 1 MUAP-10012, 4.3.7.1, Data Analysis, discusses the applicants general approach to data analysis. The plan uses a converging validity method of data analysis where multiple data inputs are used to identify and validate challenges. This method utilizes an analysis of both objective (pass/fail criteria) and subjective (performance improvement measures) data collected during ISV scenarios. Cause analysis is subsequently used to further understand the significance and scope of any challenge identified. Section 4.3.7.3, Use of Convergent Validity, provides details regarding data sources that are compared as well as methods to compare performance between the crew in areas such as situational awareness, error tolerance, workload and teamwork. Accordingly, the staff finds that the applicants treatment of data analysis conforms to this acceptance criterion.

Criterion 2 For performance measures used as pass/fail indicators, failed indicators must be resolved before the design can be validated. Where performance does not meet criteria for the other performance measures, the results should be evaluated using the HED evaluation process.18-114

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.7.2, Interpretation of Results, states that a failure to meet a pass/fail performance criterion by any one crew that is determined through the analysis to have been caused by the HFE design, operating procedures, or operator training are documented and tracked in the HED process and are considered failures that must be resolved before the HFE design is considered validated. Performance improvement measures that do not meet their acceptance criterion result in the initiation of a HED, which will be evaluated through the HED process. This guidance ensures that the ISV failures, as well as potential improvements, are documented and resolved. Accordingly, the staff finds that the applicants treatment of data analysis results conforms to this acceptance criterion.

Criterion 3 The degree of convergent validity should be evaluated, i.e., the convergence or consistency of the measures of performance.

The Staffs Evaluation of Criterion 3 See the staffs evaluation of Criterion 1 above.

Criterion 4 The data analyses should be independently verified for correctness of analysis.

The Staffs Evaluation of Criterion 4 MUAP-10012, Section 4.3.7.4, Independent Review, states that the ISV data analysis is independently reviewed by the expert panel. Specific direction is given regarding the records in which the panel reviews. Accordingly, the staff finds that the applicants treatment of independent data analysis review conforms to this acceptance criterion.

Criterion 5 The inference from observed performance to estimated real-world performance should allow for margin of error; i.e., some allowance should be made to reflect the fact that actual performance may be slightly more variable than observed validation test performance.

The Staffs Evaluation of Criterion 5 MUAP-10012, Section 4.3.7.5, Margin of Error Estimation, states that expert judgment based on a consensus, simple-majority opinion of the V&V team determines whether there is a sufficient margin of error between the ISV test and real-world performance. The V&V team contains a range of experience and expertise that is consistent with NUREG-0711 acceptance criterion 2.4.2 (3). The staff finds that these qualifications provide reasonable assurance that the V&V team can provide suitable judgments for the margin of error provided. Accordingly, the staff finds that the applicants treatment of margin of error conforms to this acceptance criterion.18-115

18.10.4.3.8 Validation Conclusions Criterion 1 The statistical and logical bases for determining that performance of the integrated system is and will be acceptable should be clearly documented.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.3.8, Validation Conclusions, provides the following list as its basis for concluding the performance of the integrated system and will be acceptable.

• A comprehensive testing program performed in compliance with this IP and its supporting test procedures by an independent multidisciplinary ISV team.

• The ISV test platform is a high fidelity and representative of the actual system, model, and HSI in all aspects that are important to the integrated systems performance. Variable aspects of the system are adequately sampled.

• Measures for acceptance criteria are logical and reflect good measurement practices and are representative of important aspects of performance.

• Test design is logical such that bias or confounding effects are understood and minimized so as not to affect the validity of the results of the ISV.

• Statistical conclusions are logical and based on convergence of multiple measures.

• The specific pass/fail HEDs as well as the extent of the identified issue.

• The consensus opinion of the observers/administrators.

The staff finds this list to be complete and acceptably supported by the guidance provided for each bullet in the IP. Accordingly, the staff finds that the applicants treatment of validation conclusions conforms to this acceptance criterion.

Criterion 2 Validation limitations should be considered in terms of identifying their possible effects on validation conclusions and impact on design implementation. These include:

• aspects of the tests that were not well controlled,

• potential differences between the test situation and actual operations, such as absence of productivity-safety conflicts,

• potential differences between the validated design and plant as built (if validation is directed to an actual plant under construction where such 18-116

information is available or a new design using validation results of a predecessor).

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.3.8, Validation Conclusions, states that the ISV limitations are considered in terms of their possible effects on validation conclusions and that the impact on design implementation is considered. A list of specific areas addressed is provided that includes the three bulleted items in the acceptance criterion as well as the effects of uncontrolled bias and unexpected events. Accordingly, the staff finds that the applicants treatment of validation limitations conforms to this acceptance criterion.

18.10.4.4 Human Engineering Discrepancy Resolution The objectives of this section of the review are to verify that:

• The applicants HED evaluation acceptably prioritizes HEDs in terms of their need for improvement. (An HED evaluation is required only if the applicant does not plan to correct all HEDs. If all HEDs are to be corrected, design improvements should be identified, see Review Criteria 4 through 6 below).

• The applicant develops design solutions and a realistic schedule for implementation to address those HEDs selected for correction.

MUAP-10012, Section 4.4, Human Engineering Discrepancy Resolution, states that the ISV is performed after design verification HEDs are resolved and any resulting HFE design changes have been implemented in the test facility prior to the start of the ISV.

Criterion 1 HED Justification - Discrepancies could be acceptable within the context of the fully integrated design. If sufficient justification exists, a deviation from the guidelines may not constitute an HED. The technical basis for such a determination could include an analysis of recent literature or current practices, tradeoff studies, or design engineering evaluations and data. Unjustified discrepancies should be identified as HEDs to be addressed by the HED resolution.

The Staffs Evaluation of Criterion 1 MUAP-10012, Section 4.4, Human Engineering Discrepancy Resolution, describes the general HED resolution process. Within this process, the HEDs can be found acceptable in the context of the integrated design, after an evaluation by the HFE team and independent confirmation by the expert panel. The decision for accepting an HED without change in the integrated design is based on accepted HFE practices, current published HFE literature, trade-off studies, tests, or engineering evaluations. All decisions for such an HED determination are documented.

The staff concludes that sufficient justification is being provided for HEDs that are found to be acceptable within the context of the fully integrated design. Accordingly, the staff finds that the applicants treatment of HED justification conforms to this acceptance criterion.18-117

Criterion 2 HED Analysis - The following should be included in the HED evaluations:

• Plant systemthe potential effects of all HEDs relevant to a single plant system should be evaluated. The potential effects of these HEDs on plant safety and personnel performance should be determined, in part, by the safety significance of the plant system(s), their effect on SAR accident analyses, and their relationship to risk significant sequences in the plant PRA.

• HED scope

- Global features HEDsthese are HEDs that relate to configurational and environmental aspects of the design such as lighting, ventilation, and traffic flow. They relate to general human performance issues.

- Standardized features HEDsthese are HEDs that relate to design features that are governed by the applicants design guidelines used across various controls and displays of the HSI (e.g., display screen organization and conventions for format, coding, and labeling). Because a single guideline may be used across many aspects of the design, a single HED could be applicable to many personnel tasks and plant systems.

- Detailed features HEDsthese are HEDs that relate to design features that are not standardized, thus [their] generality has to be assessed.

- Otherthis subcategory specifically pertains to HEDs identified from integrated system validation that cannot be easily assigned to any of the three preceding categories.

• Individual HSI or procedureHEDs should be analyzed with respect to individual HSIs and procedures. The potential effects of these HEDs on plant safety and personnel performance are determined, in part, by the safety significance of the plant system(s) that are related to the particular component.

• Personnel functionHEDs should be analyzed with respect to individual personnel functions. The potential effects of these HEDs is determined, in part, by the importance of the personnel function to plant safety (e.g.,

consequences of failure) and their cumulative effect on personnel performance (e.g., degree of impairment and types of potential errors).

HEDs should also be analyzed with respect to the cumulative effects of multiple HEDs on plant safety and personnel performance. While an individual HED might not be considered sufficiently severe to require correction, the combined effect of several HEDs upon the single aspect of the design could have 18-118

significant consequences to plant safety and, therefore, necessitate corrective action. Likewise, when a single plant system is associated with multiple HEDs that affect a number of HSI components, then their possible combined effect on the operation of that plant system should be considered.

In addition to addressing the specific HEDs, the analysis should treat the HEDs as indications of potentially broader problems. For example, identifying multiple HEDs associated with one particular aspect of the HSI design, such as the remote shutdown panel, could also indicate that there are other problems with that aspect of the design, such as inconsistent use of procedures and standards.

In some cases, the evaluation of HEDs could warrant further review in the identified areas of concern.

The Staffs Evaluation of Criterion 2 MUAP-10012, Section 4.4, Human Engineering Discrepancy Resolution, states that the HEDs resulting from the V&V program follow the process applied to all HEDs that result from the HFE program, as described in the HFE Program Management Plan. The staffs evaluation of this plan is contained in Section 18.1.4.4, Human Factors Engineering Issues Tracking, of this report. Additionally, the ISV related HEDs are characterized by their impact. The list of impacts in the MUAP-10012 include all the areas described in the first four bullets of the acceptance criterion. Within each characteristic, the HEDs extent and relationship across the HFE design is evaluated for broader issues, and the interrelationships between the HEDs are assessed and documented. This assessment includes the elements identified in the last two bullets. This analysis is accomplished at three levels:

• First, the test participants is asked explicitly in both the questionnaires and structured verbal debriefing if they believe the issue is representative of a larger or underlying problem.

• Second, at the end of each day of testing, the test observers are instructed to review each HED and reach consensus on each HEDs relationship to other HEDs and the possible extent of the HED.

• Third, the expert panel reviews each HED; part of its review explores this issue; each of these reviews is documented.

This approach is dependent on expert judgment. The staff finds this to be acceptable because test participants, observers, and expert panel members have training and experience that meets regulatory guidance as described in previous sections of this report. Also the analyses are being conducted independently, and the expert panel has the benefit of the HEDs being binned by impact and priority. Accordingly, the staff finds that the applicants treatment of HED analysis conforms to this acceptance criterion.

Criterion 3 HED Prioritization - Identification of HEDs for correction should be based upon a systematic evaluation, such as that illustrated in Figure 11.2. Priority 1 HEDs should be those with direct safety consequences and those with indirect or potential safety consequences. HEDs with significant safety consequences are those that affect personnel performance where the consequences of error could 18-119

reduce the margin of plant safety below an acceptable level, as indicated by such conditions as violations of operating limits, or Technical Specification safety limits or limiting conditions for operations. They include deviations from personnel information requirements or HFE guidelines for personnel tasks that are related to plant safety. These could include the following:

• are required by personnel tasks but are not provided by these

• do not satisfy all personnel information needs (e.g., information not presented with the proper range or precision), and

• contain deviations from HFE guidelines that are likely to lead to errors that would prevent personnel from performing the task.

HEDs with indirect safety consequences include deviations from HFE guidelines that would seriously affect the ability of personnel to perform the task. The severity of an HFE guideline deviation should be assessed in terms of the degree to which it contributes to human performance problems, such as workload and information overload.

Priority 2 HEDs should be those that do not have significant safety consequences, but do have potential consequences to plant performance/operability, nonsafety-related personnel performance/efficiency, or other factors affecting overall plant operability. These include deviations from personnel information requirements and HFE guidelines for tasks associated with plant productivity, availability, and protection of investment. These HEDs should be considered for correction.

The remaining HEDs are those that do not satisfy the criteria associated with the first and second priorities. Resolution of these HEDs is not an NRC safety concern but may be resolved at the discretion of the applicant.

The Staffs Evaluation of Criterion 3 MUAP-10012, Section 4.4, Human Engineering Discrepancy Resolution, states that the HEDs will be assigned one of three priorities. Priority 1 HEDs are those that have a direct or indirect, potential impact on plant safety. This includes HEDs that document failure of pass/fail measures and crosscutting issues. Priority 2 and 3 HEDs replicate the direction provided in the acceptance criterion. Accordingly, the staff finds that the applicants treatment of HED prioritization conforms to this acceptance criterion.

Criterion 4 HED Evaluation Documentation - Each HED should be fully documented including assessment category (priority for correction), associated plant system, associated personnel function, and associated HSI or procedure. The documentation should clearly show whether the HED was dismissed or identified as needing design modification, and the basis for this determination in terms of consequence to plant safety or operation should be clearly described.18-120

The Staffs Evaluation of Criterion 4 MUAP-09019, Section 5.2.2, Human Engineering Discrepancy Database Description, lists the documentation needed for each HED. The documentation entered includes: significance of an HED, HSI area (with description of location or equipment with which the HED is associated),

and basis for the HED including applicable design documents, significance, resolution description, and documentation and testing needed to close the HED. The database tracks the HED through creation, evaluation, resolution, and closure. Accordingly, the staff finds that the applicants treatment of HED documentation conforms to this acceptance criterion.

Criterion 5 Development of Design SolutionsDesign solutions to correct HEDs should be identified. The design solutions should be consistent with system and personnel requirements identified in the Preparatory Analysis (i.e., Operating Experience Review, Function and Task Analysis, and HSI Characterization).

Inter-relationships of individual HEDs should be evaluated. For example, if a single HSI component is associated with multiple HEDs, then design solutions should be considered to address these HEDs together. If a single plant system is associated with multiple HSI components that are associated with HEDs, then the design of the individual solutions should be coordinated so that their combined effect enhances rather than detracts from that systems operation.

The Staffs Evaluation of Criterion 5 MUAP-09019, Section 5.1, Human Engineering Discrepancy Process, states that both process (training, procedures) and design changes are used to resolve the HEDs and the procedure applies the following actions to both process and design changes. All the HEDs are reviewed at least every six months for status, design decisions, and progress of design changes. For any change, the HED documentation direction states that the basis for the change must be recorded. Where HEDs are grouped together for closure, the expert panel ensures that the resolution is sufficient for each HED in the group. Accordingly, the staff finds that the applicants treatment of design solutions conforms to this acceptance criterion.

Criterion 6 Design Solution Evaluation - Designs should be evaluated by repeating the appropriate analyses of the V&V. For example, the HSI Task Support Verification should be conducted to provide reasonable assurance that the design satisfies personnel task requirements. Portions of the HFE design verification analysis should be conducted to provide reasonable assurance that the design is consistent with HFE guidelines, and integrated system validation could be conducted to evaluate its usability. When the problems identified by an HED cannot be fully corrected, justification should be given.

The Staffs Evaluation of Criterion 6 MUAP-09019, Section 5.1.4, Human Engineering Discrepancy Closure, states that some HED closure requirements require only updated documentation, others require a documented plan for testing, and others require actual test completion. This determination is made by the HFE 18-121

design team and the expert panel based on considering the extent of the change and the degree of confidence in the resolution. The staff finds that this method provides that reasonable assurance appropriate retests will be identified. The HFE design team has the expertise to identify needed retests, and the expert panel has the expertise and independence to ensure that the tests are appropriate and complete. The members of both teams have training and experience that meets regulatory guidance as described in previous sections of this report.

Accordingly, the staff finds that the applicants treatment of design solution evaluations conforms to this acceptance criterion.

18.10.5 Combined License Information Items There are no combined license information items related to this area of review. The staff determined that no COL information items need to be included in DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for HFE V&V.

18.10.6 Conclusions The staff reviewed the applicants HFE V&V, at an IP level using the review criteria in NUREG-0711, Section 11.4, Design Verification. Section 18.0.4 of this report provides a discussion of review levels. For the reasons set forth above, the staff concludes that the V&V Program, as described in MUAP-10012, provides an acceptable methodology for the following:

• Identifying a sample of operational conditions that: (1) includes conditions that are representative of the range of events that could be encountered during operation of the plant, (2) reflects the characteristics that are expected to contribute to system performance variation, and (3) considers the safety significance of HSI components.

• Developing a HSI inventory and characterization that accurately describes all the HSI displays, controls, and related equipment that are within the defined scope of the HSI design review.

• Verifying that the HSI provides all alarms, information, and control capabilities needed for personnel tasks.

• Verifying that the characteristics of the HSI and the environment in which it is used conform to HFE guidelines.

• Validating the integrated system design (i.e., hardware, software, and personnel elements) using performance-based tests to determine whether it acceptably supports safe operation of the plant.

• Developing an HED evaluation process that acceptably prioritizes the HEDs in terms of their need for improvement and developing design solutions and a realistic schedule for implementation to address those HEDs selected for correction.

Therefore, the staff concludes that V&V considerations with respect to HFE have been adequately addressed, and that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47, related to this technical area, are satisfied.18-122

18.11 Design Implementation 18.11.1 Introduction The objective of the staffs review is to ensure that the applicants as-built design will conform to the verified and validated design that resulted from the HFE design process.

18.11.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 and Table 2.9-1.

DCD Tier 2: The applicant provided a Tier 2 system description in Section 18.11, Design Implementation, which states that aspects of the design not addressed in the design V&V will be evaluated using an appropriate V&V method. This may include design characteristics, such as new or modified displays for plant-specific design features, and features that cannot be evaluated in a simulator, such as control room lighting.

ITAAC: The ITAAC associated with this element is listed in Tier 1, Section 2.9, Table 2.9-1.

TS: There are no TS associated with this element.

Topical Reports: There are no topical reports associated with this element.

Technical Reports: The technical reports associated with this element are:

• MUAP-10013, US-APWR Design Implementation Implementation Plan, Revision 4, issued May 2014.

• MUAP-09019, US-APWR Human Factors Engineering Program Management Plan, Revision 5, issued August 2014.

18.11.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

• 10 CFR 50.34(f)(2)

Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 12, Design Implementation, Section 12.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18.II.A.1, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering.18-123

18.11.4 Technical Evaluation The staff performed an IP level of review as described in NUREG-0711 and Section 18.0.4 of this report. This section presents the applicable review criteria from NUREG-0711 (reproduced below) followed by an evaluation of each criterion.

Criterion 1 Aspects of the design that were not addressed in V&V should be evaluated using an appropriate V&V method. Aspects of the design addressed by this criterion may include design characteristics such as new or modified displays for plant-specific design features and features that cannot be evaluated in a simulator such as CR lighting and noise.

The Staffs Evaluation of Criterion 1 MUAP-10013, Section 4.1.3, Excluded HSI Features, states that other aspects that were not simulated in V&V but that are pertinent to HFE are evaluated using appropriate V&V methods such as walkdowns. Section 4.1.3 also provides a list of the aspects of the design which were not addressed during ISV but will be evaluated during design implementation. The list includes control room lighting and noise, storage for the paper based procedures, control room temperature and humidity, and accommodations for visitor interaction with the SRO.

The staff concludes that aspects of the HSI design, not addressed in the ISV, will be appropriately verified and validated. Accordingly, the staff finds that the applicants treatment of V&V conforms to this acceptance criterion.

Criterion 2 The final (as-built in the plant) HSIs, procedures, and training should be compared with the detailed design description to verify that they conform to the design that resulted from the HFE design process and V&V activities. Any identified discrepancies should be corrected or justified.

The Staffs Evaluation of Criterion 2 The staff evaluated the applicants process for comparing the final as-built HSIs to the design that is a result of the HFE design process and V&V activities. An evaluation of the applicants process for comparing the final procedures and training to the V&Vd design was not done because the verification of procedures and training is conducted during the inspection of operational programs.

US-APWR DCD, Section 18.11.1, Objectives and Scope, describes the high-level objectives and scope for the as-built HSI verification. The DCD states that the objective of the US-APWR design implementation is to demonstrate that the as-built HSI configuration accurately reflects the V&Vd design.

MUAP-10013, Scope, includes guidance to check that the as-built design matches the V&Vd hardware configuration, software configuration, and facility configuration for the MCR, RSR, TSC, and LCSs. Where there are acceptable differences, the SME documents the basis for keeping the as-built configuration. This is done through a design change analysis conducted by 18-124

a HFE SME which verifies that the change has no impact on human performance. Any unacceptable difference is documented as a HED and addressed in the HED resolution process. After the design verification process is complete, the results are entered into a Final Summary Report.

The staff finds that the applicant has provided a clear methodology for comparing the as-built HSI configuration to the V&Vd design and ensuring any deviations are reconciled. Accordingly, the staff finds that the applicants treatment of the as-built design verification conforms to this acceptance criterion.

Criterion 3 All HFE-related issues documented in the issue tracking system should be verified as adequately addressed.

The Staffs Evaluation of Criterion 3 MUAP-10013, Section 4.8, Human Engineering Discrepancies, states that all HEDs are closed prior to the completion of design implementation. Accordingly, the staff finds that the applicants treatment of the HED documentation conforms to this acceptance criterion.

18.11.5 Combined License Information Items There are no COL information items related to this area of review. The staff determined that no COL information items need to be included in US-APWR DCD Tier 2, Table 1.8-2, Compilation of All Combined License Applicant Items for Chapters 1-19, for HFE design implementation consideration.

18.11.6 Conclusions The staff evaluated the HFE design implementation at an IP level using the review criteria in NUREG-0711, Section 12.4. Section 18.0.4 of this report provides a discussion of the review levels. The staff concludes that the applicant has an acceptable process to verify that the as-built HSI configuration conforms to the verified and validated design that resulted from the HFE design process. Therefore, the staff concludes that design implementation considerations with respect to HFE have been adequately addressed, and that the requirements in 10 CFR 50.34(f) and 10 CFR 52.47, related to this technical area, are satisfied.

18.12 Human Performance Monitoring 18.12.1 Introduction The objective of the staffs review is to assure that the applicant has prepared a human performance monitoring strategy for ensuring that no significant safety degradation occurs because of any changes that are made in the plant and to verify that the conclusions that have been drawn from the human performance evaluation remain valid over the life of the plant.

18.12.2 Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 and Table 2.9-1.18-125

DCD Tier 2: The applicant identified a COL action item that will address this element.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

Topical Reports: There are no topical reports associated with this element.

Technical Reports: There are no technical reports associated with this element.

18.12.3 Regulatory Basis The SRP identifies the relevant Commission regulations for HFE and the associated acceptance criteria which are summarized below. The SRP also identifies the review interfaces with other SRP sections.

• 10 CFR 50.34(f)(2)

Regulatory guidance is found in:

• NUREG-0711, Revision 2, Chapter 13, Human Performance Monitoring, Section 13.4, Review Criteria.

• NUREG-0800, Revision 2, Chapter 18.II.A.1, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering.

18.12.4 Technical Evaluation The DCD does not address this element. A COL action item is identified to ensure that the subject is addressed in the COLA.

18.12.5 Combined License Information Items There is one COL information item listed in Table 1.8-2 of the DCD for this area of review.

Table 18.12-1 US-APWR Combined License Information Items Identified in DCD Item Description Section No.

18.12(1) The COL Applicant is to develop the Human Performance 18.12 Monitoring Program 18.12.6 Conclusions A COL action item has been identified for this element. The staff has determined that the COL action item is sufficient to ensure that the subject will be addressed during the review of any COL application referencing this design.18-126