ML18269A174

APR1400 Chapter 18, Human Factors Engineering, Final Safety Evaluation Report

ML18269A174
Person / Time
Site:	05200046
Issue date:	09/28/2018
From:	Cayetano Santos NRC/NRO/DLSE/LB2
To:
Ward W /NRO/ 415-7038
Shared Package
ML18087A364	List: ML18043A887 ML18044A185 ML18067A292 ML18078A016 ML18081A505 ML18082A298 ML18173A101 ML18176A051 ML18180A327 ML18197A219 ML18199A332 ML18199A333 ML18211A569 ML18212A092 ML18215A239 ML18215A240 ML18215A241 ML18215A242 ML18227A061 ML18261A161 ML18264A141 ML18269A174 ML18269A241 ML18269A242 ML18270A072 ML18270A394 ML18270A452 ML18271A121 ML18271A132
References
Download: ML18269A174 (175)

Text

18 HUMAN FACTORS ENGINEERING 18.0 Overview This chapter describes the results of the U.S. Nuclear Regulatory Commission (NRC) staffs review of the human factors engineering (HFE) portion of the Advanced Power Reactor 1400 (APR1400) design certification (DC) application as described in Chapter 18 of the APR1400 Design Control Document (DCD). The staffs review of HFE is being conducted in accordance with NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants, (SRP) Chapter 18, Human Factors Engineering. Consistent with SRP Chapter 18, the review used the detailed review criteria in NUREG-0711, Revision 3, Human Factors Engineering Program Review Model. SRP Chapter 18 identifies 12 areas of review for successful integration of human characteristics and capabilities into nuclear power plant (NPP) design. These areas of review correspond to the 12 elements of an HFE program identified in NUREG-0711.

HFE Program Management Operating Experience Review (OER)

Functional Requirements Analysis (FRA) and Function Allocation (FA)

Task Analysis (TA)

Staffing and Qualifications Treatment of Important Human Actions (TIHA)

Human-System Interface (HSI) Design Procedure Development Training Program Development Human Factors Verification and Validation (V&V)

Design Implementation (DI)

Human Performance Monitoring As described in Regulatory Guide (RG) 1.206, Combined License Applications for Nuclear Power Plants, Section C.III.5, Design Acceptance Criteria, the NRC accepts the use of design acceptance criteria (DAC) in lieu of detailed design information in a limited number of design areas on a case-by-case basis, as requested by the DC applicants. Korea Hydro and Nuclear Power Co., Ltd. (KHNP) requested to use DAC for the APR1400 HFE design. Thus, the applicant submitted implementation plans (IPs) that describe the proposed methodology a combined license (COL) applicant or holder will follow to complete the elements in NUREG-0711. In order to gain reasonable assurance that the applicants methodology will generate acceptable results that satisfy the relevant regulations, the staff compared the application to all of the relevant1 review criteria provided in NUREG-0711. The staff has organized Chapter 18 of this safety evaluation report (SER) to align with the 12 elements, and the relevant acceptance criteria are listed2 in each of the sections below.

1 Not all of the criteria in NUREG-0711 are relevant to a design certification application. For example, some criteria are relevant only to licensees who are performing modifications to a control room design at an operating reactor. Those criteria are identified in NUREG-0711, and they are not included in this safety evaluation because they are not in the scope of the review for a design certification application.

2 Some of the criteria in NUREG-0711 contain additional Information, which explains the basis for the criteria or gives examples to support the staff reviewers understanding of the criteria. Although it is not copied here in this SER, the staff did consider the additional information identified in NUREG-0711.

1

18.1 Human Factors Engineering Program Management Introduction The HFE program management section describes the program for applying human factors principles to the design and engineering of the APR1400. The objective of the staffs review is to confirm that the applicant has adequately considered the role of HFE and the means by which HFE activities will be accomplished.

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: The applicant provided a Tier 2 description in Section 18.1, identifying the HFE program goals, assumptions, and constraints. A description is provided of the applicants HFE program management, which includes the HFE design team and organization; the HFE process and procedures; HFE issues tracking; the HFE technical program; and COL information. See APR1400 Design Control Document Tier 2, Chapter 18 Human Factors Engineering, APR1400-K-X-FS-14002.

Changes to Tier 2 information are governed by the change control processes in the DC rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs.

Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC): There are no ITAAC associated with this element.

Technical Specifications (TS): There are no TS associated with this element.

Topical Reports (TRs): There are no TRs associated with this element.

Technical Report (TeRs): TeRs associated with this element are as follows:

APR1400-E-I-NR-14001, Human Factors Engineering Program Management Plan (HFE PP), (Agencywide Documents Access and Management System (ADAMS) Accession No. ML18212A336).

APR1400-K-Q-TR-11005, KHNP Quality Assurance Program Description (QAPD) for the APR1400 Design Certification, (ML18085B039).

APR1400-E-I-NR-14007, Human-System Interface Design Implementation Plan (HD IP), (ML18178A202). The HD IP describes the method that will be used by the COL applicant to develop the APR1400 HSI Design.

Where the additional information is needed to understand the staffs application of a criterion, the staff has added a footnote. The additional information can be found in NUREG-0711.

2

APR1400-E-I-NR-14011, Basic Human-System Interface (Basic HSI TeR),

(ML18178A202). The Basic HSI TeR describes the APR1400 Basic HSI conceptual design3. It is an input to the method described in the HD IP.

APR1400-E-I-NR-14008, Human Factors Verification and Validation Implementation Plan (V&V IP), (ML18178A202). The V&V IP contains the methodology that will be used by the COL applicant to perform the V&V activities, including the integrated system validation (ISV).

APR1400-K-I-NR-14005, Staffing and Qualifications Implementation Plan (S&Q IP), (ML17094A129). The S&Q IP describes the method that will be used by the COL applicant to perform the staffing analyses.

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

Title 10 of the Code of Federal Regulations (10 CFR) 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) - Provide, for Commission review, a control room design that reflects state-of-the-art human factor principles prior to committing to the fabrication or revision of fabricated control room panels and layouts Other regulatory guidance documents are as follows:

NUREG-0711, Revision 3, Human Factors Engineering Program Review Model, Chapter 2, HFE Program Management, Section 2.4, Review Criteria NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.1, HFE Program Management NUREG-0696, Functional Criteria for Emergency Response Facilities NUREG-0700, Human-System Interface Design Review Guidelines, Revision 2 3 Although the Basic HSI TeR describes the conceptual design of the APR1400 main control room, the Basic HSI TeR is not conceptual design information (CDI). DCD Tier 2, Section 1.8, explains that CDI is indicated by double brackets in the text and tables and cloud marks in the figures, and CDI is the portion of the plant for which the application does not seek certification. The Basic HSI TeR and DCD Tier 2, Chapter 18, do not contain double brackets or cloud marks. DCD Tier 2, Table 1.6-2, shows that the Basic HSI TeR is incorporated by reference into DCD Tier 2, and therefore, is Tier 2 information.

Additionally, the Basic HSI TeR contains information that is consistent with information contained in DCD Tier 2, Chapter 7.

3

Technical Evaluation 18.1.4.1 General HFE Program Goals and Scope NUREG-0711, Section 2.4.1, General HFE Program Goals and Scope, includes seven criteria for this topic. The seventh criterion addresses plant modifications and is not applicable to new reactors; the staff therefore evaluated the first six criteria as discussed below.

Criterion 1 HFE Program Goals - The applicant should state the general objectives of the program in human-centered terms. As the HFE program develops, they should be further defined and used as a basis for HFE tests and evaluations.

The Staffs Evaluation of Criterion 1 The HFE PP, Section 1, Purpose, identifies general program goals equivalent to those in this criterion. Section 1 identifies additional goals of the program that go beyond what is necessary to fulfill this criterion. Specific details on how these goals are achieved are described in subsequent sections of the Chapter 18 submittal and in the IPs referenced within the DCD. The HFE PP, Section 4.7.2, Element Structure, provides a template for all of the IPs to be used in the HFE process. In addition, there is a Purpose section in each of the subsequent IPs. For example, key human factors principles such as workload, situational awareness, and error reduction are measured, evaluated, and managed throughout the HFE design process to minimize operator error. Incorporation of these principles within the program goals and program IPs ensures the HFE design is centered on maximizing operator effectiveness. Additionally, DCD Tier 2, Section 18.1.1, General Human Factors Engineering Program Goals and Scope, identifies several human-centered design goals which will guide the HFE process.

Accordingly, the staff finds that the application conforms to this criterion.

Criterion 2 Assumptions and Constraints - The applicant should identify the design assumptions and constraints.

The Staffs Evaluation of Criterion 2 The DCD Tier 2, Section 18.1.1.1, Assumptions and Constraints Identification, identifies the assumptions and constraints of the KHNP HFE design. These include:

The plant is assumed to be operated by the crew, the detailed make-up of which is described.

The plant is designed to be operated with the minimum staffing level required by 10 CFR 50.54(m)(2)(iii).

The human-system interface (HSI) in the main control room (MCR):

o will accommodate the operating crew, 4

o is designed based on the APR1400 Basic HSI conceptual design, which the applicant described in the Basic HSI TeR, and o is designed in accordance with the associated HSI Design IP.

Additional constraints are identified in the HFE PP, Section 4.7.1, Design Process Elements, which identifies the predecessor plant used as the starting point for the HSI design process.

This description is expanded upon in Section 8, Definitions, which identifies the previously approved DC for the Combustion Engineering System 80+ design as the predecessor design.

Section 8 also identifies two specific predecessor plants: Palo Verde Nuclear Generating Station and the KHNP Optimized Power Reactor (OPR) plants. The reference plant, the plant that best represents the APR1400 design, is identified as Shin-Kori (SKN) 3&4 in Korea.

DCD Tier 2, Section 18.1.1.1 and the HFE PP, Section 4.7.3.6, Human-System Interface Design Interfaces, describe how the predecessors listed above were used to create the APR1400 Basic HSI conceptual design. This conceptual design is the starting point that will be analyzed and modified as necessary by a COL applicant via the HFE process resulting in a final design, the APR1400 HSI Design, which includes the HSIs as well as the HSI facilities.4 The APR1400 HSI Design will be subject to HFE validation and verification in accordance with the V&V IP.

The HFE PP, Section 8 reiterates the specific predecessor design, predecessor plant, and reference plants. While the application consistently identifies predecessor and reference plants, the staff requested more specific information on how these designs were related to and supported the APR1400 design. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-4, to address this issue (ML15177A387).

On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-4 (ML15254A492). In this letter, the applicant clarified how specific elements and components of several existing designs (some of which are operating plants) were used to develop the conceptual design of the APR1400. The APR1400 Basic HSI conceptual design relies heavily on the SKN 3&4 design as a starting point for the APR1400 HSI design, and on several other plants/designs to varying extents. The applicant also described the human factors integrated system validation completed for SKN 3&4, which is similar to, although not the same as, the current standard in NUREG-0711. This provides evidence supporting the assumption that SKN 3&4 is a reasonable starting point for the conceptual design. The HFE design process described in the HD IP will then produce improved design iterations on that conceptual design, which are ultimately based on the guidance in NUREG-0711. The RAI response sufficiently clarified the use of multiple predecessor designs and provided sufficient information regarding the use of an iterative human factors process that will ultimately improve on the conceptual design and validate that it can be safely operated. Therefore, RAI 54-7963, Question 18-4, was resolved and closed.

The staff concludes that the assumptions and constraints discussed above adequately describe the initial limitations placed on the HFE program. Although the predecessor plants were not validated according to current standards, there is no regulatory or safety basis for them to be.

4 The HD IP, Abstract, states, The integration of the APR1400 HSIS [APR1400 HSI System] and APR1400 HSI Facilities is referred to as the APR1400 HSI Design. The APR1400 HSIS includes the soft and conventional indications, alarms, controls, and operating procedures that encompass the HSI inventory in the task analysis and the APR1400 plant system designs. The APR1400 HSI Facilities include the main control room, remote shutdown room, and the technical support center.

5

The final APR1400 HSI design, if designed according to the presumed approved IPs, will ultimately meet current standards. Accordingly, the staff finds that the application conforms to this criterion.

Criterion 3 HFE Program Duration - The applicants HFE program should be in effect at least from the start of the design cycle through completion of initial plant startup test program.

The Staffs Evaluation of Criterion 3 The HFE PP, Section 4.2, Program Duration, indicates that [

]

The staff finds that the application conforms to this criterion.

Criterion 4 Facilities - The applicants HFE program should cover the MCR, remote shutdown facility (RSF), technical support center (TSC), emergency operations facility (EOF), and local control stations (LCS). The 12 HFE elements should be applied to each of them, unless otherwise noted for a specific HFE element. However, applicants may apply the elements of the HFE program in a graded fashion to facilities other than the MCR and RSF, providing justification in the HFE program plan.

The Staffs Evaluation of Criterion 4 The HFE PP, Section 2, Scope, indicates that the KHNP HFE program addresses all the facilities listed in this criterion. All twelve HFE elements are included within the scope for the MCR and remote shutdown facility (called the remote shutdown room or RSR). Additionally, the staff found a description of a remote control center (RCC) in DCD Tier 2, Section 7.7.1.1.o.3.c, Remote control center. The RCC contains the minimum equipment needed to achieve hot standby plant condition and is provided to meet the requirements in 10 CFR 50.150. The application described the RCC as an alternative to the MCR, and thus it is akin to the RSR.

Therefore, the design of the RCC should conform to the design method described in NUREG-0711 in the same way that the RSR conforms to this method. However, the staff did not find any description of the RCC in DCD Tier 2, Chapter 18. Therefore, on July 1, 2016, the staff issued RAI 502-8647, Question 07.07-17 to address this issue (ML16188A169).

On September 28, 2016, the applicant provided a response to RAI 502-8647, Question 07.07-17 (ML16272A480). As part of the response, the applicant stated the design of the RCC conforms to the design method described in NUREG-0711 in the same way that the RSR conforms to this method. Additionally, the applicant provided revisions to the DCD to include the RCC in the scope of the HFE program. As part of the response, the applicant also included information in DCD Tier 2, Section 7.7.1.1.o.3.c, Remote Control Center, that will be revised to address an issue identified by staff reviewing DCD Tier 2, Section 19.5, Aircraft Impact Assessment. On January 12, 2017, the staff and the applicant discussed revising the response to RAI 502-8647, Question 07.07-17, to align with the information that will be revised in DCD Tier 2, Section 7.7.1.1.o.3.c. On February 10, 2017, the applicant submitted a revised response to RAI 502-8647, Question 07.07-17 (ML17041A197), and proposed changes to DCD Tier 2, Section 7.7.1.1.o.3 such that the RAI response aligns with the information in the DCD.

6

Based on the review of the DCD and HD IP, the staff has confirmed incorporation of the changes described above; therefore RAI 502-8647, Question 07.07-17 was resolved and closed.

A graded approach is used is used for the remaining areas: TSC, EOF, and LCS. The HFE design of the EOF applies to the communication and information requirements only. DCD Tier 2, Section 18.1.1.2, Applicable Plant Facilities, clarifies that the LCS will use the HFE process to design those aspects of the design associated with important human actions. The staff did not find justification for the use of a graded HFE program as suggested by the acceptance criterion. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-5 to address this issue (ML15177A387).

On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-5 (ML15254A492). The applicant added additional detail clarifying how the review is graded in areas other than the MCR and remote shutdown room. The applicant will use NUREG-0696 and NUREG-0700 in the design of the EOF instead of implementing the entire NUREG-0711 process to these areas. Section 18.7.4.4, HSI Detailed Design and Integration, Subsection titled Emergency Operations Facility, of this SER describes in additional detail that NUREG-0696 includes general HFE criteria for emergency response facilities such as the EOF, and these criteria are an acceptable an alternative to the NUREG-0711 criteria. The applicant also listed particular HSIs that will be the same in both the MCR and the EOF; therefore, these particular systems will be designed using the complete NUREG-0711 process.

The complete NUREG-0711 process will be utilized for LCSs that are used for important human actions. However, other LCSs that are not used for completion of important human actions, and therefore have little potential safety impact, will be designed using NUREG-0700 instead of the full NUREG-0711 process. This treatment applies the same rigorous standard as the MCR to those LCSs with potential for safety impacts. Therefore, RAI 54-7963, Question 18-5 was resolved and closed.

Additionally, the HFE PP, Section 2.1, Applicable Human-System Interfaces, Procedures, and Training, indicates that the HFE program applies to the design of HSIs and provides input to procedure and training development. This applies to operations, emergency response, maintenance, test, inspection surveillance interfaces and procedures.

The applicant described how the human factors process will be applied to the MCR, RSR designs and to other areas around the plant. The applicant described a systematic approach that appropriately grades the design work in a way that focuses design effort on those areas that are most likely to have a safety impact. The applicant described appropriate design processes for those areas that have less of an impact on safety using appropriate alternate standards. Therefore, the staff finds that the application conforms to this criterion.

Criterion 5 HSIs, Procedures and Training - The applicants HFE program should address the design of HSIs and identify inputs to the development of procedures and training for all operations, accident management, maintenance, test, inspections, and surveillance tasks that operational personnel will perform or supervise. In addition, the HFE design process should identify training program input for the following personnel identified in 10 CFR 50.120: instrument and control technician, electrical maintenance personnel, mechanical maintenance personnel, radiological protection technician, chemistry technician, and engineering support personnel. In addition, any other personnel who perform tasks directly related to plant safety should be included, such as 7

information technology technicians who troubleshoot and maintain support systems and their HSIs.

The Staffs Evaluation of Criterion 5 The HFE PP, Section 2.1 provides a high-level summary that closely resembles the criteria description above. This section addresses the design of HSI and provides inputs to training and procedure development consistent with the criterion.

The DCD Tier 2, Section 18.1.1.3, Applicable HSIs, Procedures, and Training, indicates that the training and procedure development processes are supported by the HFE program but are described in Chapter 13, instead of in Chapter 18. The HFE PP, Section 3, Methodology Overview, describes how these processes are supported by the HFE program. Figure 4-3, Human Factors Engineering Design Process, illustrates the interface between the Procedure Development and Training Programs relative to the HFE program elements described in Chapter 18.

The applicant used the HSI design process to inform the development of procedures and training programs. The training input is applied to the specific categories of plant staff identified in 10 CFR 50.120. Accordingly, the staff finds the application conforms to this criterion.

Criterion 6 Personnel - The applicants HFE program should consider operations staffing and qualifications, including licensed control-room operators as defined in 10 CFR Part 55, and the following categories of personnel: non-licensed operators, shift supervisor, and shift technical advisor.

The Staffs Evaluation of Criterion 6 The DCD Tier 2, Section 18.1.1.4, Applicable Plant Personnel, addresses those plant personnel addressed in the criterion as well as non-licensed operators. It specifies that other plant personnel performing tasks that are directly related to plant safety will also be included in the HFE program staffing considerations. The S&Q IP contains specific details about how staffing and qualifications will be planned and executed.

The applicant includes the personnel described in this criterion in its HFE program thus providing assurance that the final design will ultimately support the varied needs of different users; therefore, the staff finds that the application conforms to this criterion.

18.1.4.2 Human Factors Engineering Team and Organization NUREG-0711, Section 2.4.2, HFE Team and Organization, includes four criteria for this topic.

Criterion 1 Responsibility - The applicants team should be responsible for:

developing all HFE plans and procedures; overseeing and reviewing all activities in HFE design, development, test, and evaluation, including the initiation, recommendation, and provision of solutions 8

through designated channels for problems identified in implementing the HFE work; verifying that the teams recommendations are implemented; assuring that all HFE activities comply with the HFE plans and procedures; and scheduling work and milestones.

The Staffs Evaluation of Criterion 1 DCD Tier 2, Section 18.1.2.1, Responsibility, describes how the applicant uses a multidisciplinary HFE design team that consists of three sub-groups: the Architect Engineering Group (A/E), the nuclear steam supply system (NSSS) group, and the operating group. The specific responsibilities of each group are described in the HFE PP, Section 4.3.1, Human Factors Engineering Design Team Responsibilities. The responsibilities listed are consistent with those listed in the criterion. Additional supporting information is found in Table 4-1, Human Factors Engineering Design Team Organization. Accordingly, the staff finds that the application conforms to this criterion.

Criterion 2 Organizational Placement and Authority - The applicant should describe the primary HFE organization(s) or function(s) within the engineering organization designing the plant or modification. The organization should be illustrated to show organizational and functional relationships, reporting relationships, and lines of communication. The applicant also should address the following:

When more than one organization is responsible for HFE [

], the lead organizational unit answerable for the HFE program plan should be identified. If organization changes are expected over time (e.g., from design through construction to startup) necessary transitions between responsible organizations should be described.

The team should have the authority and organizational placement to reasonably assure that all its areas of responsibility are completed, and to identify problems in establishing the overall plan or modifying its design.

The team should have the authority to control further processing, delivery, installation, or use of HFE products until the disposition of a nonconformance, deficiency, or unsatisfactory condition is resolved.

The Staffs Evaluation of Criterion 2 DCD Tier 2, Section 18.1.2.2, Organizational Placement and Authority, describes the key roles in the HFE design team. Included are the project manager, the quality assurance organization, the HFE design team leader, the HFE coordinator, the A/E, the operating group, the NSSS design group, and the HFE engineers. Figure 18.1-1, APR1400 HFE Design Team Organization, illustrates the relationship between these roles and others involved in the design and review process.

9

The HFE Design Team leader is responsible for oversight of HFE related activities and reports to the Project Manager. The HFE design team has design authority to manage HFE related decisions and is placed in the organization in a position that is equal to other design groups.

The engineering groups report through the HFE design team leader ensuring that HFE issues can be addressed appropriately. Additional proprietary information regarding specific details of this process can be found in the HFE PP, Section 4.3.2, HFE Design Team Organizational Placement and Authority.

The HFE PP, Section 4.2 indicates that the applicant will use a [

] This transition assures that information and issues identified during the design process are accessible to the COL applicant who will ultimately be responsible for following up and resolving them.

The HFE design team has the authority to identify issues, approve resolutions, and approve the final HSI design according to Section 4.3.2, Human Factors Engineering Design Team Organizational Placement and Authority. This authority extends through the end of the human factors validation and verification process. Table 4-1 identifies the engineering groups that perform HFE Verification and Validation (V&V) and Design Implementation activities. Therefore, the design team has the authority to stop work if a nonconformance, deficiency, or unsatisfactory condition is identified during the delivery and installation phases of construction in accordance with the final bullet of this criterion.

The staff concludes that the IP describes the management organization associated with HFE activities and delineates management and HFE team responsibilities. These responsibilities are associated with a management structure which, in the staffs judgement, have the authority to ensure that the responsibilities are accomplished. This includes the authority to identify non-conformances and deficiencies as well as providing a structure to ensure that various parts of the team are responsible for identifying acceptable resolutions. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 3 Composition - The applicants HFE design team should include the expertise described in the Appendix to NUREG-0711.

The Staffs Evaluation of Criterion 3 The HFE PP, Section 5, Implementation Team, describes the qualifications that will be included on the HFE design team. [

]

This combination of matrixed and dedicated staff personnel is a reasonable organizational structure because individuals that possess the specific skills may not be available. It is consistent with the NUREG-0711 guidance because the professional experience is satisfied by the HFE design team as a collective whole rather than on an individual basis. This is consistent with interdisciplinary teams, which are typically preferred in an HFE design process.

Section 5 also describes how [

] This assures that the results of each HFE process are auditable as described in Appendix A of NUREG-0711.

10

The qualifications, as described in Section 5, are equivalent to Appendix A of NUREG-0711 with some additional positions/qualifications added. The added roles are not likely to interfere with the roles prescribed by the appendix. Accordingly, the staff finds that the application conforms to this criterion.

Criterion 4 Team Staffing - The applicant should describe team staffing in terms of job descriptions and assignments of team personnel.

The Staffs Evaluation of Criterion 4 The HFE PP, Section 4.3.1, Human Factors Engineering Design Team Responsibilities, describes the responsibilities of the HFE design team leader as well as the various engineering groups. Table 4-1 provides information regarding the responsibilities of the three specific design teams that provide input to the HFE design team leader.

Section 4.3.2, Human Factors Engineering Design Team Organizational Placement and Authority, describes how the HFE design team, as defined in Section 4.3.1, is integrated into the larger design process. This section also explains how personnel are used to implement the HFE design process. Section 4.3.3, Human Factors Engineering Organizational Composition, provides additional supplemental information.

Section 5.0, Implementation Team, primarily describes the qualifications associated with various roles on the HFE design team and provides additional information about how these staff, with various specific qualifications, will be assigned to various teams.

While specific personnel assignments are not provided, the descriptions of responsibilities for the supervisors and various design teams provide sufficient explanation of how personnel will be used and is therefore an acceptable substitute for job descriptions and personnel assignments. Specific personnel assignments are not expected nor required by the criterion.

Accordingly, the staff finds that the application conforms to this criterion.

18.1.4.3 Human Factors Engineering Process and Procedures NUREG-0711, Section 2.4.3, HFE Processes and Procedures, includes six criteria for this topic.

Criterion 1 General Process Procedures - The applicant should identify the process through which the team will execute its responsibilities. It should include procedures for the following:

assigning HFE activities to individual team members governing the internal management of the team making decisions on managing the HFE program making HFE design decisions controlling changes in design of equipment 11

reviewing HFE products The Staffs Evaluation of Criterion 1 The HFE PP, Section 4.4.2.1, General Process Procedures, provides descriptions about how the HFE design team will carry out its responsibilities. These descriptions include how to assign activities, processes for making HFE design decisions, and controlling design changes. The descriptions are consistent with this criterion.

The description of how the applicant plans to manage the HFE team and processes for reviewing HFE program results incorporates by reference the Project Procedure Manual (PPM).

The applicant changed the name of the PPM to the Engineering Procedures Manual (EPM) after the initial license application was submitted. This caused inconsistencies in the terminology used in several submitted documents. Therefore, on September 2, 2015, the staff issued RAI 196-8164, Question 18-25 to address this issue (ML15254A494).

On November 25, 2015, the applicant submitted clarifying information in response to RAI 196-8164, Question 18-25 (ML15329A383). The response indicated that all references to the PPM in the HFE PP and in the DCD Tier 2 will be removed and replaced with the correct references to the EPM. These corrections, when completed, should make the docketed information internally consistent and support future audits/inspections. The proposed changes will not negatively affect the acceptability of the submitted material with respect to this criterion.

Based on the review of the DCD and HFE PP, the staff has confirmed incorporation of the changes described above; therefore, RAI 196-8164, Question 18-25 was resolved and closed.

The EPM is managed under the Quality Assurance Program. Design decisions, design changes, and review of HFE program results will be conducted in accordance with the Quality Assurance Program as described in APR1400-K-Q-TR-11005.

The staff concludes that working level procedures are available to communicate work control responsibilities that address the activities listed in this criterion. Accordingly, the staff finds that the application conforms to this criterion.

Criterion 2 Process Management Tools - The applicant should identify the tools and techniques (e.g.,

review forms) the team uses to verify that they fulfill their responsibilities.

The Staffs Evaluation of Criterion 2 The HFE PP, Section 4.4.2.3, Process Management Tools, describes the process management tools designed to support communication and efficiency in the design process.

The Review and Comment system is one tool which allows designers and independent reviewers to document opinions and issues regarding design documents. Another tool is the Issue Tracking System (ITS) which is used to formally document, track, resolve, and close human engineering discrepancies (HED). The staff did not understand how the Review and Comment system was used by the HFE team to fulfill their responsibilities. Therefore, on September 2, 2015, the staff issued RAI 196-8164, Questions 18-24 and 26 to address this issue (ML15254A494). Additionally, the staff did not understand how and when each of these tools are used. Therefore, on February 1, 2016, the staff issued RAI 360-8452, Question 18-90 to address this issue (ML16011A248).

12

On November 25, 2015, the applicant submitted clarifying information in response to RAI 196-8164, Questions No. 18-24 and 26 (ML15329A386). The response explained how the applicant intended to use the Review and Comment System by describing how comments are entered, reviewed, resolved, and stored within the system. The applicant indicated that it will update the HFE PP to better reflect this information and provided a suitable marked-up draft for review.

The November 25, 2015, RAI response also clarified that the Review and Comment System, ITS, and Corrective Action Program are independent systems. On May 19, 2016, the applicant provided additional clarification about how these systems remain independent in response to RAI 360-8452, Question 18-90 (ML16141A768). This submittal provides detailed comparisons of the Review and Comment System, ITS, and Corrective Action Program. The separation of scope, timelines, and responsible parties provides reasonable assurance that issues will be entered into the appropriate tracking system, thus ensuring the independence of each system.

Therefore, RAI 360-8452, Question 18-90 was resolved and closed.

The applicant submitted a marked-up revision to the HFE PP to include a description of a process for ensuring that workers are properly qualified for various design jobs in the November 25, 2015, response to RAI 196-8164, Questions 18-24 and 26. This additional information provides assurance that the processes will be implemented as intended by ensuring that only qualified personnel are involved. The staff confirmed that HFE PP was revised as committed in the responses to RAI 196-8164, Questions 18-24 and 18-26. Therefore, RAI 196-8164, Questions 18-24 and 18-26 were resolved and closed.

Section 4.3.3 indicates that [

].

This section also indicates that the HFE design team leader is responsible for [

]

The staff has reviewed the tools used to identify, document, and resolve issues during various stages of design and implementation. The applicant has identified the tools used and clarified the processes that will be used to apply these tools. Each of these systems helps track issues to make sure they are resolved appropriately. In addition, the applicant used technological safeguards in addition to an administrative process that helps ensure that only qualified personnel conduct resolution activities. Therefore, the staff finds that the application conforms to this criterion.

Criterion 3 Integration of HFE and Other Plant or Modification Design Activities - The applicant should describe the process for integrating the design activities (i.e., the inputs from other design work to the HFE program, and the outputs from the HFE program to other plant design activities).

The applicant should also discuss the iterative aspects of the HFE design process.

The Staffs Evaluation of Criterion 3 The HFE PP, Section 4.4.3, Integration of the Human Factors Engineering Program with other Plant Design, describes an iterative process that integrates the HFE program into the plant design. Integration is controlled by the Quality Assurance Program as described in APR1400-K-Q-TR-11005 which contains high-level descriptions of how the applicant intends to 13

perform design control (Section 3) and design verification (Section 3.1), including for the HFE design.

The HFE process receives inputs from other plant design areas. Results Summary Reports (ReSR) generated as part of the HFE process are provided to other design areas upon completion. The HFE design team works with the other design areas to assess and implement design changes. A formalized report describing design changes is sent to the HFE design team for subsequent approval. The HFE design team leader is responsible for documenting integration activities through a series of processes and reports. Summaries of the specific inputs and outputs from other HFE program elements can be found in the HFE PP, Section 4.7.3, Element Input and Output, as well as in the corresponding IPs.

DCD Tier 2, Section 18.1.3.3, Integration of the HFE Design with Other Plant Design Activities, describes the use of various process management tools to further integrate the HFE process.

The process lists several integration related activities that are the responsibility of the HFE design team leader to implement. Section 18.1.2.2 describes the role of the HFE coordinator. It includes responsibilities of coordinating designers in the A/E, operations group and NSSS groups for resolution of HED.

DCD Tier 2, Figure 18.1-3, HFE Design Process, illustrates the iteration in the HFE process.

Results of HFE elements that occur later in the design process are fed back into iterations on the earlier HFE element processes. These issues are ultimately addressed via HED resolution and/or HFE V&V process.

The applicant described an iterative process that is well integrated both within the HFE program as well as within the more inclusive plant design program. The HFE program inputs and outputs are summarized in the HFE PP and additional detail is described in subsequent IPs. The staff finds that the application conforms to this criterion.

Criterion 4 HFE Program Milestones - The applicant should identify HFE milestones that show the relationship of the elements of the HFE program to the integrated plant design, development, and licensing schedule. A relative program schedule of HFE tasks should be available for the Nuclear Regulatory Commission (NRC) staffs review showing relationships between the HFE elements and the activities, products, and reviews.

The Staffs Evaluation of Criterion 4 The HFE PP, Section 4.4.1, Human Factors Engineering Program Milestones, Schedule, and Duration, contains Figure 4-2, APR1400 HFE Program Milestones, which shows the relative expected milestones for the HFE elements. This diagram anchors the HFE work within appropriate licensing and operating schedules. Accordingly, the staff finds that the application conforms to this criterion.

Criterion 5 HFE Documentation - The applicant should identify the HFE documentation items, such as ReSRs and their supporting materials, and briefly describe them, along with the procedures for their retention and for making them available to the staff for review.

14

The Staffs Evaluation of Criterion 5 The HFE PP, Section 4.4.4, Human Factors Engineering Documentation, lists the specific written deliverables used as part of the HFE process including the ReSRs, HFE program IPs, and several other HFE documents. Figure 4-2 anchors various documents to important project milestones, as described in Criterion 4 above.

The Review and Comment System, also described in Section 4.4.4, is used to retain all of the documents described and to control access to these documents. These documents will be retained for the life of the program. The quality assurance program described in APR1400-K-Q-TR-11005, Section 3.2 provides supplemental information regarding the creation and retention of design records.

The HFE PP, Section 6, Results Summary Report, indicates that all ReSRs are available for NRC review. The applicant described the expected contents of each ReSR in Section 6 of the corresponding IPs. However, the plan for submitting the various ReSRs to the NRC is unclear.

Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-6 to address this issue (ML15177A387).

On August 18, 2015, the applicant provided a response to RAI 54-7963, Question 18-6 (ML15230A528). This letter indicated that no ReSR will be complete during the DC process.

These reports will be the responsibility of the COL applicant. A rough expected schedule for completion is provided, however, it is acknowledged that this schedule is likely to change because KHNP has no control over the potential COL applicant schedules. Audits of ReSRs and inspections of the simulator and HFE V&V process can be scheduled with the COL applicant as needed. Therefore, RAI 54-7963, Question 18-6 was resolved and closed.

The applicant adequately described a method for documenting the HFE process. Therefore, the staff finds that the application conforms to this criterion.

Criterion 6 Subcontractor HFE Efforts - The applicant should include HFE requirements in each subcontract contributing to the HFE program. The applicant should periodically verify the subcontractor's compliance with HFE requirements. The HFE plan should describe milestones and the methods used for this verification.

The Staffs Evaluation of Criterion 6 DCD Tier 2, Section 18.1.3.6, Subcontractor HFE Efforts, and the HFE PP, Section 4.5, Subcontractor Human Factors Engineering Efforts, refer to programs used to review and verify subcontractor efforts. Procurement specifications that contain the necessary HFE requirements as well as the HFE Style Guide are sent to subcontractors.

The applicant indicates that HFE requirements will be included in subcontractor procurement documents and describes how the Quality Assurance Program will be used to verify compliance. Therefore, the staff finds that the application conforms to this criterion.

18.1.4.4 Human Factors Engineering Issues Tracking NUREG-0711, Section 2.4.4, Tracking HFE Issues, includes four criteria for this topic.

15

Criterion 1 Availability - The applicant should have a tracking system to address human factors issues that are:

known to the industry; identified throughout the life cycle of the HFE aspects of design, development, and evaluation; and deemed by the HFE program as HEDs.

The Staffs Evaluation of Criterion 1 The HFE PP, Section 4.6, Tracking Human Factors Engineering Issues, describes the ITS.

The tracking system will be used to document known industry issues, human factors issues, and HEDs identified throughout the execution of the KHNP HFE program elements.

The HFE PP, Section 4.6 specifies the roles associated with the HFE design team leader with respect to the system. DCD Tier 2, Section 18.1.4, Tracking of HFE Issues, indicates that the HFE team is responsible for implementing the logging, tracking, and resolution processes.

Additional information about the use of ITS by other HFE design team members is addressed throughout the subsections of Section 4.6.

The applicants plan sufficiently described the tracking system that will be used to track known industry issues as well as issues and HEDs identified during the design process. Therefore, the staff finds that the application conforms to this criterion.

Criterion 2 Method - The applicants method should:

establish criteria for when issues are entered into the system and track issues until the potential for negative effects on human performance is reduced to an acceptable level.

The Staffs Evaluation of Criterion 2 The HFE PP, Section 4.6.1.2, Human Engineering Discrepancies Entry, describes high-level criteria for entry of HEDs into the ITS. It explains that HEDs can be identified during any human factors process subject to the conditions indicated in the associated IP. Entries will be made with sufficient identifying information so that they can be easily tracked. A diagram of the ITS is shown in Figure 4-4.

Section 4.6.1.1, Access to Issue Tracking System, describes safeguards used to ensure that the ITS is accessed by authorized personnel only and is not modified without authorization from the HFE design team leader.

Section 4.3.1 specifies the responsibilities of the design team with respect to HED entry and tracking.

16

Section 4.6.1.3, Human Engineering Discrepancies Resolution, and Section 4.6.1.4, Human Engineering Discrepancies Closeout, explain the process used to assess potential negative effects on performance, and closure of HEDs when it is determined resolution of HEDs is acceptable. Section 4.6.1.4 suggests that HEDs that do not require any actions do not require further tracking. This statement is inconsistent with the second bullet of this NUREG-0711 criterion in that the synergistic effect of multiple low priority HEDs should be accounted for. Therefore, on November 9, 2015, the staff issued RAI 298-8356, Question 18-41 to address this issue (ML15314A023).

On April 22, 2016, the applicant provided supplemental information in response to RAI 298-8356 Question 18-41 (ML16113A441). The applicant clarified that the cumulative effects of HEDs are considered during the HFE V&V process. The staff confirmed that HFE PP and V&V IP were revised as committed in the response to RAI 298-8356, Question 18-41. Therefore, RAI 298-8356, Question 18-41 was resolved and closed.

The applicant described the criteria used for entering issues into the ITS and described how issues will be tracked, assessed, and subsequently closed out. Therefore, the staff finds that the application conforms to this criterion.

Criterion 3 Documentation - The applicant should document the actions taken to address each issue in the system; if no action is required, this should be justified.

The Staffs Evaluation of Criterion 3 The HFE PP, Section 4.6.1.5, Documentation, explains that HED resolutions that require design changes will be documented in the ITS and will be summarized in the respective HFE element as well as in the appropriate design documents/change orders. HEDs will also be documented in the ReSR for the respective element of the HFE program in which it was identified. Resolutions to Priority 1 HEDs and summaries of Priority 2 HEDs will be included in the ReSR for the human factors V&V element.

The applicant also addressed closure of HEDs not requiring future tracking or action in Section 4.6.1.4, Human Engineering Discrepancies Closeout. These HEDs require proper documentation as well as approval from the HFE design team leader. They are maintained in the ITS. However, the description provided in this section does not explain how Priority 3 HEDs are justified and how this justification is documented. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-8 to address this issue (ML15177A387).

On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-8 (ML15254A492), and on April 22, 2016, provided supplemental information in RAI 298-8356, Question 18-41. These submittals revised the definition of Priority 3 HEDs to be consistent with the criterion and with the intent of NUREG-0711 Section 11.4.4, Subsection (1). The applicant clarified that the cumulative effects of HEDs is considered during the human factors V&V process. The staff confirmed that HFE PP was revised as committed in the response to RAI 54-7963, Question 18-8. Therefore, RAI 54-7963, Question 18-8 was resolved and closed.

The applicant described a process that documents and tracks HEDs throughout the design process through resolution. The process also included justification for any HEDs that do not require additional actions. Therefore, the staff finds that the application conforms to this criterion.

17

Criterion 4 Responsibility - After identifying an issue, the applicants tracking procedures should describe individual responsibilities for logging, tracking, and resolving it, along with the acceptance of the outcome.

The Staffs Evaluation of Criterion 4 DCD Tier 2, Section 18.1.4 identifies the HFE design team as the responsible party for logging, tracking, and resolving of issues. According to the HFE PP, Section 4.6.1.3, Human Engineering Discrepancies Resolutions, the HFE design leader is responsible for assigning HEDs to a cognizant-engineer for oversight of resolutions. Section 4.6.1.6, Responsibility, indicates that the HFE design team leader is responsible for approving HED entries, resolutions, and closeout of ITS entries. Section 4.6.1.4 indicates that HEDs can be closed when appropriate changes to HSI documents are complete and have been approved by the HFE design team leader. Alternatively, HEDs that do not cause changes to the design can be closed when a justification is documented in the ITS.

DCD Tier 2, Section 18.1.4, indicates that thresholds for HED tracking are identified in individual IPs. The staff reviewed a sample of the IPs and HED entry criteria were confirmed to be found in the IPs as described by the applicant.

The applicant clearly defined the responsibilities related to logging, tracking, resolving, and accepting HEDs. Therefore, the staff finds that the application conforms to this criterion.

18.1.4.5 Technical Program NUREG-0711, Section 2.4.5, Technical Program, includes four criteria for this topic. The fifth criterion addresses plant modifications and is not applicable to new reactors; therefore, the staff evaluated the first four criteria as discussed below Criterion 1 The applicant should describe the applicability and status of each of the following HFE elements:

Operating Experience Review (OER)

Functional Requirements Analysis (FRA) and Function Allocation Task Analysis Staffing and Qualifications Treatment of Important Human Actions HSI Design Procedure Development (Described in SRP, Chapter 13 submittal)

Training Development (Described in SRP, Chapter 13 submittal)

Human Factors V&V 18

Design Implementation Human Performance Monitoring The Staffs Evaluation of Criterion 1 The applicants technical program, as presented in the HFE PP, Section 3.0, Methodology Overview, incorporates all of the eleven identified NUREG-0711 elements listed above.

Human Performance Monitoring is identified as the responsibility of the COL applicant, and Training and Procedure Development are addressed in the Chapter 13 submittal.

Section 4.7, Technical Program, and the associated subsections describe the element interfaces and the inputs and outputs to each program element. It also includes a description of the final work products. Figure 4-3 provides a functional block diagram illustrating the work flow between elements.

The applicant appropriately applied all of the listed human factors program elements and described how they are functionally related. Therefore, the staff finds that the application conforms to this criterion.

Criterion 2 The applicant should identify the approximate schedule for completing any HFE activities that are unfinished at the time of the application.

The Staffs Evaluation of Criterion 2 Section 4.4.1, Figure 4-2 provides a schedule that benchmarks various HFE submittals between the DC application and operation. The elements are relative to each other and do not include approximate dates for submittals (specific dates are not required by the criterion as they are difficult to identify and under the control of COL applicants).

The relative schedule shown in Figure 4-2 shows the best estimate for a COL schedule possible at this time without a COL application pending. Therefore, the staff finds that the application conforms to this criterion.

Criterion 3 The applicants plan should identify and describe the standards and specifications that are sources of the HFE requirements.

The Staffs Evaluation of Criterion 3 The HFE PP, Section 2.3, Applicable Codes, Standards, Guides and Regulations, contains an extensive listing of nuclear industry documents that encompass codes and standards, NRC documents, and other industry documents such as those from the Institute of Electrical and Electronics Engineers (IEEE), and American National Standards Institute/American Nuclear Society (ANSI/ANS). These documents are consistent with those commonly used as sources for HFE design. Therefore, the staff finds that the application conforms to this criterion.

19

Criterion 4 The applicants plan should specify HFE facilities, equipment, tools, and techniques (such as laboratories, simulators, rapid prototyping software) that the HFE program will employ.

The Staffs Evaluation of Criterion 4 The HFE PP, Section 4.7.1 indicates that a full-scope MCR simulator, part-task simulator, mock-ups, as well as special tools and equipment will be used to support HFE design work. DCD Tier 2, Section 18.1.3.2, Process Management Tools, also identifies regularly scheduled HFE design team meetings as a process management tool to facilitate coordination between design groups. The staff did not understand how the process or processes would use these tools and others described in the IP to create the final design. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-10 to address this issue (ML15177A387).

On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-10 (ML15254A492). This response clarified that the APR1400 basic HSI (based on SKN 3&4, and other precedent designs) will be used to develop the full-scope simulator prototype. The detailed functional design specifications are developed in accordance with the HD IP and incorporated into the resulting APR1400 full-scope dynamic simulator developed to meet ANSI/ANS-3.5-2009, Nuclear Power Plant Simulators for Use in Operator Training and Examination, that will be used for the human factors validation and verification. The applicant response sufficiently clarified ambiguities in the application; therefore, RAI 54-7963, Question 18-10 was resolved and closed.

The applicant provided a logical and systematic process that utilized a simulator based on the precedent design as a basis to develop a high-fidelity simulator that will ultimately be used to validate the MCR design. Therefore, the staff finds that the application conforms to this criterion.

Combined License Items There are no COL items associated with Section 18.1 of the APR1400 DCD.

Conclusion The staff evaluated the applicants method for HFE program management and finds that it conforms to the criteria in NUREG-0711, Section 2.4. Therefore, the staff concludes that the applicants HFE program addresses goals and scope of the HFE program, identifies the HFE team and member qualifications, identifies HFE processes and procedures, describes methods for tracking HFE issues, and provides an overview of how each of the HFE program elements will be addressed for the APR1400. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element.

18.2 Operating Experience Review Introduction The objective of this review is to verify that the applicant has identified and analyzed HFE related problems and issues encountered in previous designs so that these problems and issues may be avoided in the development of the new design. This review should also verify that the applicant has retained positive features of previous designs. This is done through an 20

evaluation of licensee event reports; Institute of Nuclear Power Operations significant event reports and significant operating experience reports; plant corrective action systems; operational and maintenance logs and records; and data from interviews with experienced plant personnel.

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: The applicant provided a Tier 2 description in Section 18.2 of the DCD which summarizes the method described in APR1400-E-I-NR-14002, Operating Experience Review Implementation Plan, that will be used by the COL applicant to complete the operating experience review program element.

Changes to Tier 2 information are governed by the change control processes in the design certification rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE implementation plans.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: TeRs associated with this element are:

APR1400-E-I-NR-14002, Operating Experience Review Implementation Plan (OER IP), (ML18081A107).

APR1400-E-I-NR-14001, Human Factors Engineering Program Management Plan (HFE PP), (ML18212A336).

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

10 CFR 50.34(f)(3)(i) 10 CFR 50.34(f)(2)(iii) 10 CFR 52.47(a)(8)

Other regulatory guidance documents are as follows:

NUREG-0711, Revision 3, Chapter 3, Operating Experience Review, Section 3.4, Review Criteria.

21

NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.2, Operating Experience Review.

Technical Evaluation 18.2.4.1 Scope NUREG-0711, Section 3.4.1, Scope, includes five criteria for this topic.

Criterion 1 Predecessor/Related Plants and Systems - The applicants operating experience review (OER) should include information about human factors issues in the predecessor plant(s) or highly similar plants, systems, and HSIs, including the following:

The OER should identify previous or predecessor design(s)/plant(s) used as part of the design basis of the plant being reviewed.

The OER should define the relevance of each predecessor plant/design to the new design, when there is more than one predecessor.

The OER should detail how the applicant identified and analyzed any HFE-related problems in the previous plants/designs, and how these issues are avoided in the new design.

The OER should address how the applicant identified, evaluated, and incorporated or retained any positive features of previous plants/designs.

The OER should describe the predecessor plant(s) and systems, explaining the relationship of each to the new design.

For applicants proposing to use new technology or systems that were not used in the predecessor plants, the OER should review and describe the operating experience of any other facilities that already use that technology.

The Staffs Evaluation of Criterion 1 The OER IP, Section 1 states that the objective of the OER is to identify, evaluate, track, and incorporate HFE-related operating experience (OE) encountered in previous nuclear plant designs and OE sources so that the negative features are not repeated, and the positive features are retained. Performance information from predecessor designs is identified at the start of the design process and used to improve the plant design.

The applicants APR1400 OER includes:

Currently operating U.S. pressurized water reactors (PWRs)

All Korean plants that are operating, or have operated, which includes SKN (SKN) 1, 2, 3 and 4 U.S. OE databases, NUREGs, and vendor groups 22

SKN 3&4 operator and instructor interviews Non-nuclear applications - Chemical, Transportation, Electrical As discussed in the staffs evaluation of Criterion 2 in Section 18.1.4.1, General HFE Program Goals and Scope, of this document, the applicant defined predecessor design, predecessor plant, and reference plant in the HFE PP, Section 8, and also described the relationship of these plants to the APR1400 HSI Design. The starting point for the APR1400 HSI Design is the APR1400 Basic HSI conceptual design, which is based on SKN 3&4.

Section 4, Implementation, of the OER IP describes the OER process. Section 4.1, Internationally Sourced Operating Experience Data Collection; Section 4.2, U.S. Sourced Operating Experience Data Collection; Section 4.3, Operator Interviews as a Source of Operating Experience; and Section 4.4, Non-nuclear Sources of Operating Experience, describe the sources of OE.

Section 4.5, The Process of Screening Operating Experience for Applicability, describes the screening process. A number of questions are provided that the OER reviewer answers to determine if the OE identifies any HFE-related problems to be avoided in the design or positive features to be retained.

The OEs are then grouped, classified, and analyzed for lessons learned. Interviews with operators from SKN 3&4 are used to identify and retain positives features. Any lesson learned that affects another HFE program element is entered into the Issue Tracking System (ITS) as a human engineering discrepancy (HED) where they can be analyzed by other program element reviewers and be tracked to completion. All OEs are entered into the OE Database.

The applicant has provided an OER process which identified operating experience from the predecessor plants and operating U.S. plants as input to the APR1400 design, designates the sources for this material and the types of reports to be evaluated, and provides a means of tracking cross-cutting OEs. Therefore, the staff finds that the application conforms to this criterion.

Criterion 2 Recognized Industry HFE Issues - The applicant should address the HFE issues identified in NUREG/CR-6400. The issues are organized into the following categories:

unresolved safety issues/generic safety issues (see 10 CFR 52.47(a)(21) and NUREG-0933)

Three Mile Island (TMI) issues NRC generic letters and information notices operating experience reports in the NUREG-1275 series, Volume 1 through 14 low power and shut down operations operating plant event reports 23

Additionally, the applicant should review and discuss all operating experience in the preceding categories that was published since NUREG/CR-6400, Human Factors Engineering (HFE)

Insights for Advanced Reactors Based upon Operating Experience, was published in 1996.

The Staffs Evaluation of Criterion 2 The OER IP, Section 4.2, U.S. Sourced Operating Experience Data Collection, provides NUREG/CR-6400 as one of the sources from which operating experience is collected. Section 4.6, Grouping Operating Experience, states that the issues described in the criterion are used as categories to group operating experience. During the review of applicants initial submittals, the staff noted that the sources for recognized industry HFE issues, other than NUREG/CR-6400, listed in the OER IP were not detailed enough to enable the staff to verify the depth of the OER review. Therefore, on June 26, 2015, in RAI 53-7982, Question 18-2, the staff asked the applicant to provide greater specificity and more details in this area (ML15177A386). Similarly, in Question 18-3, the staff asked the applicant to be specific about the SKN 3&4 OEs that were used in APR1400 design and whether or not, the SKN 3&4 OER followed the guidelines in NUREG-0711.

As stated in the applicants September 4, 2015, response to RAI 53-7982, Questions 18-2 and 18-3 (ML15247A164), the OER IP augments the operating experience list with other sources, such as an extensive list of NUREGs (e.g., 0696, 0737, and 0933), Westinghouse and Combustion Engineering vendor groups, and accident and failure records from the Institute for Nuclear Power Operations. Additionally, the applicant stated that OE that was published after 1996 is encompassed in the SKN 3&4 OER, which is the basis for the APR1400 OER. The SKN 3&4 OER is supplemented with a review of everything after that review was done with the process described in the IP. However, the staff noted that although the applicant, in its RAI response, had provided details that were satisfactory, it did not commit to updating the OER IP to include the additional details provided in the response. Therefore, on August 3, 2016, the staff asked the applicant in an email (ML17116A260) to revise the OER IP to incorporate the additional details provided in the RAI response.

To address this issue, the applicant agreed to revise the OER IP to include the details provided in the RAI 53-7982 response dated September 4, 2015. Accordingly, on August 25, 2016, the applicant provided a revised response (ML16238A384) to the RAI which also included a revision to the OER IP. The staff reviewed the proposed changes to the OER IP and found the revision incorporates the additional information in the initial response to RAI 53-7982, Questions 18-2 and 18-3.

Based on the review of the OER IP, the staff has confirmed incorporation of the changes described above; therefore RAI 53-7982, Questions 18-2 and 18-3 was resolved and closed.

During the APR1400 Subcommittee Meeting of the Advisory Committee on Reactor Safeguards (ACRS) held on June 21, 2017 (ML17200A091), an ACRS member questioned whether the staff audited the results of the SKN 3&4 OER to confirm that relevant OE that occurred before the close date of the SKN 3&4 OER had indeed been adequately addressed in the SKN 3&4 OER. The ACRS member expressed concern that relevant OE, such as OE from operating plant event reports from U.S. plants, could be excluded from the OER performed by the COL applicant.

Following the ACRS meeting, the staff re-evaluated the OER IP. The OER IP, Section 4.5, The Process of Screening Operating Experience for Applicability, states, OE is first screened to determine whether it transpired before or after the close date of the SKN 3&4 OER. OEs with 24

dates before the SKN 3&4 close date are assumed to be included in the SKN 3&4 OER and are not be screened again. Because the staff did not audit the results of the SKN 3&4 OER, the staff could not confirm that all relevant operating experience, including events related to the categories listed in this review criterion, were adequately addressed in the SKN 3&4 OER. As such, the staff determined it was possible that relevant OE could be excluded from the OER conducted by the COL applicant. Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-136 to address this issue (ML17249A979).

On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-136 (ML17271A188) and stated that the OER IP will be revised to state that OEs that occurred before the SKN 3&4 close date will first be evaluated to determine whether they were included in the SKN 3&4 OER. If they were included in the SKN 3&4 OER, then they may be screened out only if the lessons learned were identified and determined to be adequately addressed using the guidance in NUREG-0711, Revision 3. The applicant also provided revisions to the OER IP that aligned with the applicants RAI response.

Because the results of the SKN 3&4 OER will first be reviewed to determine whether a lesson learned from a particular operating experience event has been addressed, and if not, will be included in the OER, the staff concludes that the proposed revisions to the OER IP ensure that relevant OE are not excluded from the OER.

Additionally, during the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member questioned why the OER IP states that OEs will be grouped according to the categories in NUREG/CR-6400. The categories listed in NUREG/CR-6400 identify various types of events and lessons learned that, at a minimum, should be included in the scope of the applicants OER. The ACRS member questioned how grouping OEs by these categories provided insights during the OER process because grouping OEs by other characteristics, such as common causes of operating events, would provide insight into the reasons why OE events occurred and could help identify adequate design solutions. Following the ACRS meeting, the staff requested the applicant address this question as part of RAI 553-9084, Question 18-136. The applicants response to RAI 553-9084, Question 18-136 stated that NUREG/CR-6400 provides expanded HFE design issue categories and proposed resolutions. OE grouping following this issue category helps designer to clarify his/her OE-related design issues and to decide the resolutions. Grouping OE into the categories identified in NUREG/CR-6400 may not provide as much insight as to the human errors that contributed to plant events as grouping the OE into other categories (e.g., related causes), however, the concept of grouping OE is a good practice. Further, there is no requirement that applicants group OE as part of their analysis. Therefore, the applicants proposed method is acceptable.

Based on the review of the OER IP, the staff has confirmed incorporation of the changes described above; therefore RAI 553-9084, Question 18-136 was resolved and closed.

Therefore, the staff finds that the application conforms to this criterion.

Criterion 3 Related HSI Technology - The applicants OER should cover operating experience with the proposed HSI technology in the applicants design.

The Staffs Evaluation of Criterion 3 The OER IP, Section 4.4, Non-nuclear Sources of Operating Experience, states that non-nuclear industry experience related to digital screen-based HSI technology will be evaluated.

25

The non-nuclear industry scope includes the chemical, transportation, and electrical transmission industries. This additional scope is added due to the limited nuclear industry experience with the technologies used in the KHNP design.

The applicant outlined a detailed plan for assessing the operating experience for digital screen-based technologies which includes relevant experience outside the nuclear industry. By including non-nuclear experience, the applicant has maximized the potential for applying lessons learned on new technology applications within the KHNP design. Therefore, the staff finds that the application conforms to this criterion.

Criterion 4 Issues Identified by Plant Personnel - The applicants OER should discuss issues identified through interviews with plant personnel based on its operating experience with plants or systems applicable to the new design. As a minimum, the interviews should include the following topics:

Plant Operations o normal plant evolutions (e.g., startup, full power, and shutdown) o failure modes and degraded conditions of the I&C systems, including, but not limited to, the sensor, monitoring, automation and control, and communications subsystems. These include, for example, the safety-related system logic and control unit, fault tolerant controller (NSSS), the local "field unit" for the multiplexer (MUX) system, the MUX controller (balance-of-plant), and a break in the MUX line failure modes o degraded conditions of the HSI resources (e.g., losses of video display units, of data processing, and of large overview display) o transients (e.g., turbine trip, loss of offsite power, station blackout, loss of all feedwater, loss of service water, loss of power to selected buses or MCR power supplies, and safety/relief valve transients) o accidents (e.g., main steam line break, positive reactivity addition, control rod insertion at power, control rod ejection, anticipated transients without scram, and various-sized loss-of-coolant accidents) o reactor shutdown and cooldown using the remote shutdown system HFE Design Topics o alarms and annunciation o displays o controls and automation o information processing and job aids o real-time communications with plant personnel and other organizations 26

o procedures, training, staffing/qualifications, and job design The Staffs Evaluation of Criterion 4 OER IP, Section 4.3, Operator Interviews as a Source of Operating Experience, states that personnel interviews are conducted to collect information on all the topics listed in the acceptance criterion. In addition, the interview question structure is developed to match the topics in this criterion.

The staff concludes that the applicants interview process addresses the aspects of this criterion, and the applicant is using operator interviews as well as operator involvement in design testing as an input to the KHNP HSI design. Therefore, the staff finds that the application conforms to this criterion.

Criterion 5 Important Human Actions (IHAs) - The applicants OER should identify important HAs in the predecessor plants or systems, and determine whether they remain important in the applicants design. Additional considerations cover the following:

For the important HAs, the OER should identify the scenarios wherein actions are needed, and state whether they were needed and successfully completed.

Those aspects of the design that helped ensure success should be identified.

If errors occurred in the execution of the HAs, the applicant should identify insights to the needed improvements in human performance.

When important HAs for the new plant are determined to differ from those of the predecessor plant, the OER should specify whether there is any operational experience with these different HAs.

The Staffs Evaluation of Criterion 5 OER IP, Section 4.10, Important Human Actions, states that for IHAs, the OE reviewer will use the list of important human actions from the treatment of important human actions (TIHA) element to verify it against the predecessor and document differences as HEDs. The OE reviewers assessment also identifies HFE issues that include human actions credited for accident mitigation. These actions include backup manual actions. Lastly, an assessment of the aspects that deter or ensure success is done. Where an OE aspect caused a deterrence or error in a human action, they are documented as a lesson learned and an HED, then tracked so that they can be adequately considered in the HFE design.

Section 4.10 also states that the OE reviewer reviews the list of IHAs developed in the TIHA element and determines where IHAs for the new plant design differ from those of the predecessor plant. The implementation team then identifies whether there is an OE with the different human actions and writes an HED for inclusion in the ITS and tracking purposes.

The staff concludes that the applicant is appropriately using operating experience to ensure that the APR1400 design facilitates the successful performance of important human actions.

Therefore, the staff finds that the application conforms to this criterion.

27

18.2.4.2 Issue Analysis, Tracking and Review NUREG-0711, Section 3.4.2, Issue Analysis, Tracking and Review, includes four criteria for this topic.

Criterion 1 OER Process - The applicant should discuss the administrative procedures for evaluating the operating, design, and construction experience, and for ensuring that applicable important industry experiences will be provided in a timely manner to those designing and constructing the plant.

The Staffs Evaluation of Criterion 1 At a high level, the applicant has provided in the OER IP the method for meeting this criterion.

The sources of OE the applicant has discussed include the items mentioned in this criterion.

The overall placement of the HFE process, and the HFE design personnel, to the overall APR1400 design process are discussed in the HFE PP. The HFE PP, from a high level, provides assurance that applicable important industry experiences found during OER via the HFE process will be provided in a timely manner to other disciplines during the design and construction of the APR1400. Therefore, the staff finds that the application conforms to this criterion.

Criterion 2 Analysis Content - The applicant should analyze issues to identify:

human performance issues and sources of human error design elements supporting and enhancing human performance The Staffs Evaluation of Criterion 2 OER IP, Section 4.5, The Process of Screening Operating Experience for Applicability, details the evaluation process used by the OE reviewer to select and analyze human performance issues. Questions used in the selection process are provided in Section 4.5. Within the questions, the two areas described in the acceptance criterion bullets are addressed. During the applicants lessons learned analysis phase, an HED is generated to resolve items that are assigned to other HFE program elements. Therefore, the staff finds that the application conforms to this criterion.

Criterion 3 Documentation - The applicant should document the analysis of operating experience.

The Staffs Evaluation of Criterion 3 OER IP, Section 6 describes the documentation the applicant will provide containing the analysis of the OER activity. The report includes the OER execution results containing details that demonstrate compliance to the Methodology section of the OER IP. The report will include:

Predecessor/related plants and systems 28

Review Methodology OE sources/documents reviewed Names and qualifications of all members of the team that performed the OER Conduct of the OER and results of reviewing relevant HSI technology Findings from interviews with plant personnel and other users List of the OEs that are incorporated into the design Number and status of the open HEDs that are still being tracked in the ITS The staff concludes that that all essential elements of the OER are being documented.

Therefore, the staff finds that the application conforms to this criterion.

Criterion 4 Incorporation into the Tracking System - The applicant should document each issue determined to be relevant to the design, but yet to be addressed, in the issue-tracking system The Staffs Evaluation of Criterion 4 OER IP, Section 4.8, Operating Experience Lessons Learned Analysis, describes a process whereby the negative OEs are assessed. After the assessment, the cause determined for the negative OE is documented as a lesson learned. Section 4.11, Tracking and Verification with Respect to Operating Experience Lessons Learned, states that the lessons learned are entered into the OE database for tracking. All lessons learned that affect other HFE program elements are entered into the ITS and tracked to completion.

The staff concludes that potential HFE issues identified during the OER are being documented in an ITS. Therefore, the staff finds that the application conforms to this criterion.

Combined License Items There are no COL items associated with Section 18.2 of the APR1400 DCD.

Conclusion The staff evaluated the applicants method for conducting the OER and finds that it conforms to the criteria in NUREG-0711, Section 3.4. Therefore, the staff concludes that the applicants OER sufficiently identifies, analyzes, and addresses HFE-related problems in previous designs similar to the APR1400. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii), 10 CFR 50.34(f)(3)(i), and 10 CFR 52.47(a)(8) related to this element.

18.3 Functional Requirements Analysis and Function Allocation Introduction FRA is the identification of functions that must be performed to satisfy plant safety objectives; that is to prevent or mitigate the consequences of postulated accidents that could cause undue risk to the health and safety of the public. Function Allocation (FA) is the analysis of 29

requirements for plant control and the assignment of control functions to: (1) personnel (e.g.,

manual control); (2) system elements (e.g., automatic control and passive, self-controlling phenomena); and (3) combinations of personnel and system elements (e.g., shared control, automatic systems with manual backup).

The objective of the staffs review is to verify that: (1) the plant's functions that must be performed to satisfy plant safety objectives have been defined, and (2) the allocation of those functions to human and system resources has resulted in a role for personnel that takes advantage of human strengths and avoids human limitations.

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: The applicant has provided a Tier 2 description in Section 18.3 of the DCD, which summarizes the method described in APR1400-E-I-NR-14003, Functional Requirements Analysis and Function Allocation Implementation Plan, that will be used by the COL applicant to complete the functional requirements analysis and function allocation program element.

Changes to Tier 2 information are governed by the change control processes in the design certification rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: TeRs associated with this element are:

APR1400-E-I-NR-14003, Functional Requirements Analysis and Function Allocation Implementation Plan (FRA/FA IP), (ML18081A107)

APR1400-E-I-NR-14004, Task Analysis Implementation Plan (TA IP),

(ML18178A202)

APR1400-E-I-NR-14008, Human Factors Verification and Validation Implementation Plan (V&V IP), (ML18178A202)

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

10 CFR 52.47(a)(8) 30

10 CFR 50.34(f)(2)(iii)

Other regulatory guidance documents are as follows:

NUREG/CR-3331, A Methodology for Allocation of Nuclear Power Plant Control Functions to Human and Automated Control NUREG-0711, Revision 3, Chapter 4, Functional Requirements Analysis and Function Allocation, Section 4.4, Review Criteria NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.3, Functional Requirements Analysis and Function Allocation Technical Evaluation NUREG-0711, Section 4.4, Review Criteria, includes nine criteria for this topic. The ninth criterion addresses plant modifications and is not applicable to new reactors; therefore, the staff evaluated the first eight criteria as discussed below.

Criterion 1 The applicant should use a structured, documented methodology reflecting HFE principles to perform FRA and FA.

The Staffs Evaluation of Criterion 1 The staff reviewed the applicants overall FRA/FA methodology in the FRA/FA IP using this criterion.

The KHNP methodology follows the traditional approach of defining two high-level goals, safety and power production, then the high-level functions needed to obtain the goals, and then the success paths needed to obtain the high-level functions. The success path actions are then allocated to human and system resources resulting in a role for personnel that takes advantage of human strengths and avoids human limitations. A success path is defined as an individual branch of processes, systems, components, and control actions capable of maintaining each critical function or restoring a critical function when required.

FRA/FA IP provides three figures: Figure 3-1, FRA/FA Process Flow Diagram; Figure 4-1, Upper Functional Hierarchy; and Figure 4-2, Lower Functional Hierarchy, which together, graphically present a structured methodology for performing FRA/FA. The methodology is based, in part, on NRC guidance such as NUREG/CR-3331 and on the staff approved System 80+ methodology documented in NPX80-IC-RR970-02, Human Factors Evaluation and Allocation of System 80+ Functions.

FRA/FA IP, Section 4.5, FRA/FA Configuration and Control, states that the FRA/FA follows preliminary plant design development and is sufficiently detailed to enable specification of detailed plant design and HSI design requirements. HFE principles such as time available, workload, and task complexity are considered in the FA analysis. The processes that support the applicants FRA/FA methodology are evaluated in more detail in subsequent review criteria.

31

The staff concludes that the FRA/FA process described in the FRA/FA IP provides a structured, documented methodology reflecting appropriate HFE principles. Therefore, the staff finds that the application conforms to this criterion.

Criterion 2 The applicants FRA and FA should be performed iteratively to keep it current during design development and operation up to decommissioning, so that it can be used as a design basis when modifications are considered.

The Staffs Evaluation of Criterion 2 The FRA/FA IP, Section 4.3.3, Specification of Functional Hierarchy, Success Paths, and Requirements, explains that during FRA, the success paths will be identified for each critical safety function (CSF) and power production function. Success paths are sets of plant system configurations that are able to maintain or restore either a critical safety function (CSF) or a power production function. The process of identifying success paths for each function is referred to as decomposition. Section 4.3.3 states, [

]

During the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member questioned why the FRA/FA IP states that generic assumptions about site-specific plant systems will be needed given the fact that the COL applicant will be performing the activities in the FRA/FA IP, and the COL applicant will know site-specific information. Following the ACRS meeting, the staff re-evaluated Section 4.3.3 of the FRA/FA IP. Because site-specific information will be available when a COL applicant performs FRA and FA, the site-specific information will need to be used to perform FRA and FA in order to ensure the FRA and FA results are kept current with the plant design. As such, the staff determined the application did not fully conform to this criterion. Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-137 to address this issue (ML17249A979).

On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-137 (ML17271A188) and stated, The generic assumptions support the preliminary results of the FRA/FA and TA, and those results provide the basis for the HSI design. As site-specific information is known, the generic assumptions are modified as necessary. When the COL applicant performs the HFE activities, the site-specific information is applicable to develop the APR1400 HSI design at the site, and the preliminary results of the FRA/FA and TA are updated accordingly. The updated information leads to the complete HSI design which is verified and validated during the HF V&V program element (PE). DI PE confirms the as-built design with the application of the site-specific information. Where the site-specific information is not reflected, the DI PE conducts a regression analysis to define the necessary HFE rework.

The applicant also provided revisions to the FRA/FA IP, Section 4.3.3, to clarify that the COL applicant is to use the site-specific information to complete the activities in the FRA/FA IP. The 32

staff concludes that the proposed revisions to the FRA/FA IP ensure that the results of the FRA and FA will accurately reflect the plant design that exists when the activities in the FRA/FA IP are performed.

Based on the review of the FRA/FA IP and the TA IP, the staff has confirmed incorporation of the changes described above; therefore, RAI 553-9084, Question 18-137 was resolved and closed.

Additionally, the applicant described how the results of the FRA and FA will remain current during the life of the plant in the FRA/FA IP, Section 4.2, Analysis Updates (Iterations), which states, [

]

Section 4.2 describes three mechanisms that maintain the FRA and FA results current:

[

] These are discussed in more detail below:

DCD Tier 2, Section 18.3.2.3, FRA/FA Implementation, states, The HED resolution process described in the HFE PP provides a mechanism to track HFE feedback as the plant design progresses. As the design information becomes more detailed and complete or modified, issues relative to the HFE design are identified. To the extent those issues impact the FRA/FA and require changes to functional requirements or allocations, the FRA/FA is updated.

The FRA/FA IP, Section 4.2, states, [

]

The staff concludes that the FRA/FA is kept current over the life cycle of the plant by being incorporated into the applicants overall quality assurance program required by 10 CFR 52.79, Contents of applications; technical information in final safety analysis report, and the design project configuration management and design change procedures. Accordingly, the staff finds that the application conforms to this criterion.

Criterion 3 The applicant should describe the plants functional hierarchy including, as appropriate, goals, functions, processes, and systems. The description should include:

comparing them with the predecessor or reference plants and systems, i.e., the previous ones on which the new plant is based identifying the differences between the proposed and reference plants and systems documenting the technical basis for modifications to high-level functions in the new design compared to the predecessor design 33

defining, for each safety function and other plant function (e.g., electrical power generation), the set of system configurations or success paths that are responsible for, or able to carry out the function decomposing the functions, starting at high-level functions where a very general picture of major functions is described, and continuing to lower levels, until a specific critical end-item requirement emerges (e.g., a piece of equipment, software, or an HA). The functional decomposition should address the following levels o high-level functions (e.g., maintain reactor coolant system integrity) o the processes, as appropriate, that enable achievement of these functions o specific plant systems and components o HAs, as appropriate The Staffs Evaluation of Criterion 3 FRA/FA IP, Section 4.3, Functional Hierarchy, defines the safety functions that must be maintained to achieve the plant goals of safety and power production. For each high-level function, success paths are developed which include the following components:

sub-functions (e.g., reactivity control) process (e.g., boration, or rod insertion) plant systems (e.g., chemical and volume control system, control rod drive, etc.)

key components (e.g., shutdown rods) action required of those key components (e.g., drop rods, or insert rods manually)

Sub functions and systems are not described (other than by example) but it is clear from the direction and example forms provided that this information is collected as part of the analysis.

In addition, differences between the proposed and predecessor plant/systems are listed in Section 1.3, Comparison with Other Facilities.

Plant operations SMEs perform the success path identification. NPX80-IC-RR790-02 and CEN-152, Combustion Engineering Emergency Procedure Guidelines, documents are used as a starting point to establish the high-level critical safety functions. The System 80+ is then compared to the APR1400 to identify identical, modified, and new success paths. This process is done to ensure that adequate success paths exist to protect high-level functions during the full range of plant operating modes (full power, low power, shutdown) and conditions (e.g.,

normal or abnormal).

In Section 3.2.1, Identification of Critical Safety Functions, the applicant stated that the Containment Environment CSF differs from the predecessor design in that it was split into two items: 1) Containment Combustible Gas Control and 2) Containment Temperature and 34

Pressure Control. In general, the CSFs for the APR1400 are the same as the predecessor design, and the applicant stated that no changes are expected for the APR1400 design.

However, if any modification should occur in the CSFs, they would be documented in the ReSR.

The staff concludes that the high-level functions needed to meet plant goals are clearly identified and are complete. The methodology and the documentation of analysis results provide for complete descriptions of the systems, components, and actions that constitute the success paths needed to control the high-level functions. Therefore, the staff finds that the application conforms to this criterion.

Criterion 4 For each high-level function, the applicant should identify requirements related to:

purpose of the high-level function conditions indicating that the high-level function is needed parameters indicating that the high-level function is available parameters indicating that the high-level function is operating (e.g., flow indication) parameters indicating that the high-level function is achieving its purpose (e.g.,

reactor vessel level returning to normal) parameters indicating that the operation of the high-level function can or should be terminated The Staffs Evaluation of Criterion 4 FRA/FA IP, Section 4.4, Functional Requirements, includes all the characteristics specified by this NUREG-0711 criterion. Overall, FRA/FA IP, Section 4 provides the detailed process to accomplish the FRA. Plant operations SMEs perform the FRA by completing functional definition tables (FDT) which contain the lower level decomposition information of the higher level critical safety functions. The decomposition information is presented in Table 4-1, Example of a Function Definition Table for a Critical Function, as an example table which describes: 1) the plant goals and high-level functions; 2) the decomposition of the functions into success paths (e.g., sub-functions, systems, components, and actions) and; 3) includes cells that describe the high-level function in terms of the characteristics listed in the acceptance criterion.

The staff concludes that the characteristics needed to define each high-level function are clearly identified. The methodology provides for complete documentation of each characteristic.

Therefore, the staff finds that the application conforms to this criterion.

Criterion 5 The applicant should allocate functions to a level of automation (e.g., from manual to fully automatic) and identify the technical bases for the allocations.

35

The Staffs Evaluation of Criterion 5 FRA/FA IP, Section 4.5, Allocation of Functions, provides a detailed explanation of the process used to allocate functions to a level of automation ranging from manual to fully automatic for the KHNP design.

Plant operations and system SMEs assess the functions identified by the FRA for each success path to determine the role of personnel, automation, or a combination, using the evaluation process described in NUREG/CR-3331 and NPX80-IC-RR790-02, Human Factors Evaluation and Allocation of System 80+ Functions. NUREG/CR-3331 describes an acceptable method for allocating functions. The workload, time available, action complexity, and decision complexity for each function are evaluated and compared to guidelines in NUREG/CR-3331 to establish the level of automation. Then the technical basis and rationale for the function allocations are captured in an allocation table, of which an example is provided in the FRA/FA IP. Completed allocation entries from the table show the relationship of the FAs to the success paths derived from the FRA.

The staff concludes that the criteria and process for determining functional allocations is clearly and completely described and incorporates accepted HFE practices. The basis for allocations is well documented in NUREG/CR-3331. Therefore, the staff finds that the application conforms to this criterion.

Criterion 6 The applicants FA should consider not only the primary allocations to personnel, those functions for which personnel have the primary responsibility, but also its responsibilities to monitor automatic functions, detect degradations and failures, and to assume manual control when necessary.

The Staffs Evaluation of Criterion 6 FRA/FA IP, Section 4.6, Additional Considerations, states that the FA includes the operators responsibilities listed in the criterion. In addition, the responsibilities listed in the criteria are captured in NUREG/CR-3331, Section 4.1, Step 11: Hypothesize Allocation of Functions, which the applicant uses in Appendix C of the FRA/FA IP as part of its FA process.

The staff concludes that the function allocation provides for an assessment of the operators responsibility for monitoring automatic functions, detecting degradations and failures, and assuming manual control when necessary. Therefore, the staff finds that the application conforms to this criterion.

Criterion 7 The applicant should describe the overall role of personnel by considering all functions allocated to them.

The Staffs Evaluation of Criterion 7 FRA/FA IP, Section 4.7, Overall Personnel Roles, states [

] through the Task Analysis and Staffing & Qualifications (S&Q) elements. Task analysis provides a more detailed 36

evaluation of the tasks allocated to operators. The task analysis process also verifies that the initial functional allocation is acceptable after the workload analysis results are obtained. The S&Q analyzes the requirements for the number of personnel and their qualifications using the task and regulatory requirements.

The staff concludes that the applicant has considered all of the functions allocated to personnel and has described the overall role of personnel. Therefore, the staff finds that the application conforms to this criterion.

Criterion 8 The applicant should verify that the FRA and FA accomplish the following:

All the high-level functions needed to achieve safe operation are identified.

All requirements of each high-level function are identified.

The allocation of functions to humans and automatic systems assures a role for personnel that takes advantage of human strengths and avoids human limitations.

The Staffs Evaluation of Criterion 8 FRA/FA IP, Section 4.8, Independent Review, states that confirmation of the FRA/FA is performed by an independent plant operations SME that did not take part in the FRA/FA. Also, completing the activities described in the TA IP and V&V IP verifies that the allocation of functions to humans and automatic systems takes advantage of human strengths and avoids human limitations. The FRA/FA team leader ensures the discrepancies identified by the SMEs are resolved during the review.

The staff concludes that the FRA/FA results are independently verified, and any discrepancies are resolved. Therefore, the staff finds that the application conforms to this criterion.

Combined License Items There are no COL items associated with Section 18.3 of the APR1400 DCD.

Conclusion The staff evaluated the applicants FRA and FA method and finds that it conforms to the criteria in NUREG-0711, Section 4.4. Therefore, the staff concludes that the FRA/FA methodology sufficiently defines the plants functions that must be performed to satisfy plant safety objectives and sufficiently allocates those functions to human and system resources. Accordingly, the staff concludes the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element are satisfied.

18.4 Task Analysis Introduction Task analysis identifies the tasks that plant personnel must perform to accomplish the functions that are allocated to human actions (HAs). Task analysis also identifies the alarms, information, controls, and task support that must be available for plant personnel to successfully perform 37

these tasks. Task analysis generates input to several program elements: staffing and qualifications, HSI design, procedure development, training program development, and verification & validation.

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: Section 18.4, Task Analysis, summarizes the method described in APR1400-E-I-NR-14004, Task Analysis Implementation Plan, that will be used by the COL applicant to complete the task analysis program element.

Changes to Tier 2 information are governed by the change control processes in the design certification rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: TeRs associated with this element are:

APR1400-E-I-NR-14004, Task Analysis Implementation Plan (TA IP)

(ML18178A202). The TA IP contains the method that will be used by the COL applicant to perform task analysis.

APR1400-E-I-NR-14001, Human Factors Engineering Program Management Plan (HFE PP) (ML18212A336). The HFE PP describes the relationship between the task analysis program element and other HFE program elements.

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii)

Other regulatory guidance documents are as follows:

NUREG-0711, Revision 3, Chapter 5, Task Analysis, Section 5.4, Review Criteria 38

NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.4, Task Analysis Technical Evaluation NUREG-0711, Section 5.4, Review Criteria, includes 10 criteria for this topic. The tenth criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first nine criteria as discussed below.

Criterion 1 The scope of the applicants task analysis should include:

All IHAs as determined by probabilistic and deterministic means.

The applicant should select tasks for analysis that represent the full range of plant operating modes, including startup, normal operations, low power and shutdown conditions, transient conditions, abnormal conditions, emergency conditions, and severe accident conditions. The chosen tasks should cover:

o tasks that were not identified as important HAs but have negative consequences if performed incorrectly; o tasks that are new compared to those in predecessor plants, such as ones related to new systems or procedures; o tasks that, while not new, are performed significantly differently from predecessor plants; o tasks related to monitoring of automated systems that are important to plant safety, and the use of automated support aids for personnel, such as computer- based procedures; o tasks related to identifying the failure or degradation of automation, and implementing backup responses; o tasks anticipated to impose high demands on personnel, e.g., little time or high workload (such as administrative tasks that contribute to work load and challenge ability to monitor the plant);

o tasks important to plant safety that are undertaken during maintenance, tests, inspections, and surveillances; and o tasks with potential concerns for personnel safety (such as maintenance tasks performed in the containment).

The Staffs Evaluation of Criterion 1 The staff reviewed DCD Tier 2, Section 18.4, and the TA IP. Three sections in these documents describe the scope of task analysis: DCD Tier 2, Section 18.4.1, Objectives and Scope; the TA IP, Section 2; and the TA IP, Section 4.1, Task Selection. DCD Tier 2, Section 18.4.1 and the TA IP, Section 2 list the types of tasks that will be included in the scope 39

of the task analysis. In these sections, the staff found most, but not all, of the types of tasks listed in Criterion 1. The TA IP, Section 4.1 states that tasks selected for task analysis include all the types of tasks listed in this criterion, with one exception. The application describes the scope of the task analysis in three different sections, and the information is not consistent in these three sections, nor does it conform to Criterion 1 in any of these sections. Therefore, on October 19, 2015, the staff issued RAI 260-8283, Question 18-30 and Question 18-31 to address this issue (ML15293A581).

On December 31, 2015, the applicant provided a response to RAI 260-8283, Question 18-31 (ML15365A562) and on January 21, 2016, the applicant provided a response to RAI 260-8283, Question 18-30 (ML16021A516). In the responses, the applicant submitted a revision to DCD Tier 2, Section 18.4.1; the TA IP, Section 2, Scope; and the TA IP, Section 4.1, Task Selection. The revised sections contain information about the scope of the task analysis that is consistent in each of the sections that discuss the scope of the task analysis and also state that the scope of the task analysis includes all of the types of tasks listed in Criterion 1.

Based on the review of the DCD and TA IP, the staff has confirmed incorporation of the changes described above; therefore RAI 260-8283, Questions 18-30 and 18-31 were resolved and closed.

Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should describe the screening methodology used to select the tasks for analysis, based on criteria specifically established to determine whether analyzing a particular task is necessary.

The Staffs Evaluation of Criterion 2 The TA IP, Section 4.1 contains a proprietary method for identifying tasks that are within the scope of task analysis and lists the sources that will be used to identify these tasks. The HFE PP and DCD Tier 2, Section 18.4.1 describe the sources used for task selection as follows.

The HFE PP, Section 4.7.3.3, Task Analysis Interfaces, describes that three HFE program elements provide inputs to task selection: OER, function requirements analysis and function allocation (FRA/FA), and TIHA. OER identifies human actions in the APR1400 design that are associated with known human performance issues at operating plants. FRA/FA identifies actions allocated to humans to meet the plants safety and power production goals during all modes and for normal, abnormal, and emergency conditions. TIHA identifies risk-significant (i.e., risk-important) human actions and human actions that must be performed to mitigate the consequences of plant transients and accidents.

DCD Tier 2, Section 18.4.1 describes that tasks identified in the normal, abnormal, emergency, and alarm response procedures that are available at the time the TA is conducted or are from the predecessor plants or predecessor designs are selected.

The three other HFE program elements identify human actions that are performed during all modes of operation and under normal, abnormal, and emergency conditions, including the IHAs and actions with known human performance issues. The APR1400 is an evolutionary design, 40

and thus the predecessor plants operating procedures identify a significant portion of the actions that plant personnel must take to operate the APR1400 plant during normal, abnormal, and emergency conditions. Thus, the staff concludes that the applicants sources that will be used to identify tasks that personnel must perform to operate the plant, including tasks that with relatively high risk and safety significance, are adequate.

DCD Tier 2, Section 18.4.2, Methodology, summarizes the method in the TA IP for selecting tasks for analysis. Two levels of task analysis will be performed: a basic task analysis (BTA) and a task timing analysis (TTA). A BTA will be conducted for all of the tasks that are identified from the OER, FRA/FA, TIHA, and plant operating procedures. As part of the BTA, all tasks will be screened using selection criteria to determine if a TTA must be performed.

DCD Tier 2, Section 18.4.2.2, Task Timing Analysis, states that the TTA evaluates task workload and time margin to identify tasks that may produce an unacceptably high workload or have an inadequate time margin. This allows for early5 identification of changes to the HFE design or plant design that may be necessary to ensure the safe operation of the plant.

Because not all tasks are likely to have high workloads or be subject to time constraints, the staff concludes that it is appropriate to perform a TTA only for tasks that are likely to have a high workload or inadequate time margin during task performance.

The TA IP, Section 4.2.3, Task Evaluation, contains a proprietary list of criteria that will be used to select tasks for a TTA. The staff reviewed the list and concluded that the screening criteria include tasks that must be performed within a time limit or have certain characteristics that may cause the task to involve a relatively high workload.

Thus, the staff finds that the application describes the method for selecting tasks for both a BTA and a TTA. Also, the staff finds that the application contains appropriate criteria specifically established to determine whether performing a TTA is necessary. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 3 The applicant should begin task analysis with detailed narratives of what personnel must do.

The analysis should be sufficiently detailed to define the alarms, information, controls, and task support needed to accomplish the task. The detailed task descriptions should address (as applicable to the task) the topics listed in NUREG-0711, Table 5-1, Task Considerations.

The Staffs Evaluation of Criterion 3 DCD Tier 2, Section 18.4.2.1, Basic Task Analysis, describes the three products that result from the BTA, which is performed for each task: the task narrative, the HSI inventory, and the task evaluation. These are described below.

5 Workload and task performance time are assessed during the ISV, which is the final validation of the HFE design. If the ISV demonstrates that workload for certain tasks is not acceptable or that operators cannot perform tasks with time constraints in the time available, then changes to the integrated system (i.e., the HSI, procedures, or training program) may be necessary to reduce workload or the time it takes operators to perform a given task. All or portions of the ISV may need to be repeated to demonstrate that the changes are effective. Estimating workload and time margin during task analysis provides a means of identifying tasks that may have unacceptable workload levels or insufficient time margins relatively early in the HFE design process when the components of the integrated system are still being developed, which should help reduce the likelihood that significant design changes will have to be made late in the design process.

41

BTA begins with developing a task narrative, which describes what plant personnel need to do to perform the task. The TA IP, Section 4.2.1, Task Narrative, provides a list of proprietary topics that must be documented in the task narrative. The staff reviewed the list of topics and concluded that as a group, the topics address what personnel must do to perform the task, the HSI inventory (alarms, controls, and indications) that personnel need to perform the task, task support requirements, situational and performance-shaping factors, and time constraints. Therefore, the staff finds that the task analysis begins with developing detailed task narratives that describe what personnel must do and the alarms, information, controls, and task support needed to accomplish the task.

Once the task narrative is complete, the BTA continues with the HSI inventory.

The HSI inventory expands on the task narrative by identifying the characteristics of the HSI identified in the task narrative. The TA IP, Sections 4.2.2.1, Process Monitoring, and 4.2.2.2, Component Control, provide a list of proprietary topics that must be addressed in the HSI inventory. The staff reviewed the list of topics and concluded that as a group, the topics address, in detail, the characteristics and features of the alarms, information, controls, and task support that personnel need to perform the task (e.g., the units of measurement that should be displayed for a particular parameter).

The BTA concludes with the task evaluation, which determines whether a TTA is necessary. As described in the staffs evaluation of Criterion 2, a TTA will be performed for tasks that meet the screening criteria for a TTA. The TTA results in an: (1) estimation of workload and time margin for a particular task and (2) an evaluation of whether the workload and time margin are acceptable.

Taken together, the topics addressed in the task narrative and the HSI inventory produced for each task during BTA encompass all of the topics listed in NUREG-0711, Table 5-1 with the exception of workload. This is acceptable because workload will be assessed only for the tasks that meet the screening criteria for a TTA. Thus, workload is not a topic that is applicable for every task.

Therefore, the staff finds that the task narrative and the HSI inventory for each task and the workload assessment, if applicable to the task, address all of the topics in NUREG-0711, Table 5-1 and will therefore ensure that the alarms, controls, information, and task support that personnel need to accomplish the task will be defined. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 4 The applicant should identify the relationships among tasks.

The Staffs Evaluation of Criterion 4 The TA IP, Section 4.2.1 provides a list of proprietary topics that must be documented in the task narrative for each task. The staff reviewed the list of topics and concluded that it includes the relationship of a task to other tasks and whether the task can be performed in parallel with related tasks or must be performed in a specific sequence. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

42

Criterion 5 The applicant should estimate the time required to perform each task.

The Staffs Evaluation of Criterion 5 DCD Tier 2, Section 18.4.2.2 states that time required (TmRq) is estimated by adding the process delay time (e.g., if a task is to open a valve, then the process delay time is the time it takes a valve to stroke from the closed to open position) and the time an operator is engaged in the task (i.e., time engaged or TmEn). Process delay times can be determined from documents that describe the operating characteristics of plant components or estimated using information contained in those documents. The TA IP, Section 4.3.1.2, Time Engaged, contains a proprietary method for estimating the time engaged, which is discussed in the staffs evaluation of Criterion 9. TmRq is an input to time margin, which is also discussed further in the staffs evaluation of Criterion 9.

The TA IP, Section 4.3, Task Timing Analysis, states that TmRq is estimated using this method for tasks that meet the criteria for a TTA. As discussed in the staffs evaluation of Criterion 2 of this section, tasks selected for a TTA includes tasks that are likely to produce high workloads or that are subject to time constraints. Therefore, time required does not need to be estimated during task analysis for tasks that are not subject to time constraints. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 6 The applicant should identify the number of people required to perform each task.

The Staffs Evaluation of Criterion 6 The TA IP, Section 4.2.1 provides a list of proprietary topics that must be documented in the task narrative for each task. The staff reviewed the list of topics and concluded that it includes the number of people required to perform the task. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 7 The applicant should identify the knowledge and abilities required to perform each task.

The Staffs Evaluation of Criterion 7 The TA IP, Section 4.2.1 provides a list of proprietary topics that must be documented in the task narrative for each task. The staff reviewed the list of topics and concluded that it did not adequately address knowledge and abilities required to perform tasks. Knowledge and abilities must be defined during task analysis because this information is an input to the staffing and qualifications program element. Therefore, on October 19, 2015, the staff issued RAI 260-8283, Question 18-39 to address this issue (ML15293A581). On January 21, 2016, the applicant provided a response to RAI 260-8283, Question 18-39 (ML16021A516). The response reiterated the information previously submitted, and thus it did not adequately address RAI 260-8283, Question 18-39. Therefore, on July 25, 2016, the NRC held a public meeting with the applicant to discuss the response. During the meeting, the staff stated that NUREG-1122, Knowledge and Abilities Catalog for Nuclear Power Plant Operators: Pressurized Water Reactors, contains knowledge and abilities (K/As) for tasks performed by operators at PWRs.

43

Because the APR1400 plant is an evolutionary PWR, NUREG-1122 contains a significant portion of the K/As applicable to tasks operators will perform at the APR1400 plant. The applicant stated that it would revise the TA IP to refer to NUREG-1122 and to include any K/As that are not documented in the K/A catalog in the task narrative.

On August 9, 2016, the applicant provided a revised response to RAI 260-8283, Question 18-39 (ML16222A916). The response incorporated the revision discussed above. The staff confirmed that TA IP was revised as committed in the response to RAI 260-8283, Question 18-39.

Therefore, RAI 260-8283, Question 18-39 was resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 8 The applicants task analysis should be iterative and updated as the design is better defined.

The Staffs Evaluation of Criterion 8 The TA IP, Section 6 states that performance of the task analysis program element is a one-time event. The task analysis program element is performed in accordance with the method defined in the TA IP, Section 4, Methodology. The results of the BTA and the TTA for individual tasks (i.e., the task analyses), including the task narrative and HSI inventory, are documented in a database.

DCD Tier 2, Section 18.4.1, Objectives and Scope, states:

For tasks related to plant systems that are site specific, such as the switchyard and ultimate heat sink, the TA [task analysis] is based on generic assumptions that are made to establish a complete plant design that is ultimately reflected in the complete APR1400 HSI design for V&V. These generic assumptions are modified as necessary for each plant-specific application of the APR1400 during the design implementation (DI) program element.

APR1400-E-I-NR-14004, Task Analysis Implementation Plan (TA IP), Section 2, Scope, contains similar statements. During the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member questioned why the application states that generic assumptions about site-specific plant systems will be needed given the fact that the COL applicant will be performing the activities in the TA IP, and the COL applicant will know site-specific information. Following the ACRS meeting, the staff re-evaluated the TA IP and the DCD. Because site-specific information will be available when a COL applicant performs TA, the design will be better defined than the design that is described in the DCD. The staff agrees that the COL applicant will need to use site-specific information in order to ensure the TA is performed using current information about the plant design. As such, the staff determined the application did not fully conform to this criterion. Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-137 to address this issue (ML17249A979).

On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-137 (ML17271A188) and stated:

The generic assumptions support the preliminary results of the FRA/FA and TA, and those results provide the basis for the HSI design. As site-specific information is known, the generic assumptions are modified as necessary. When the COL applicant performs the HFE activities, the site-specific information is 44

applicable to develop the APR1400 HSI design at the site, and the preliminary results of the FRA/FA and TA are updated accordingly. The updated information leads to the complete HSI design which is verified and validated during the HF V&V program element (PE). DI PE confirms the as-built design with the application of the site-specific information. Where the site-specific information is not reflected, the DI PE conducts a regression analysis to define the necessary HFE rework.

The applicant also provided revisions to DCD Tier 2, Section 18.4.1, and the TA IP, Section 2, to clarify that the COL applicant will use the site-specific information to complete the activities in the TA IP. The staff concludes that the proposed revisions to the DCD and TA IP ensure that the task analyses will accurately reflect the plant design that exists when the activities in the TA IP are performed.

Based on the review of the DCD and TA IP, the staff has confirmed incorporation of the changes described above; therefore RAI 553-9084, Question 18-137 was resolved and closed.

Following completion of the activities in the TA IP, the task analyses may need to be updated as the HFE design and plant design evolve. HEDs identified during the implementation of other HFE program elements will be evaluated to determine if the task analyses need to be revised.

Also, as described in the TA IP, Section 6 plant design changes are evaluated for their impact on all of the HFE program elements, including task analysis. Changes to task analyses are documented and evaluated using the HED resolution process.

Because the task analysis will be updated as necessary as the HFE design and plant design evolve, the staff finds the application conforms to this criterion.

Criterion 9 The applicant should provide analyses of the feasibility and reliability for IHAs that address the following:

(1) The analysis establishes the time available using an analysis method and acceptance criteria consistent with the regulatory guidance associated with the actions. The basis for the time available is documented.

(2) The analysis of the time required is based on a documented sequence of operator actions (based on task analysis, vendor-provided generic technical guidelines for emergency operating procedure development, or plant-specific EOPs, depending on the maturity of the design).

(3) The analysis of the action sequence is conducted at a level of detail sufficient to identify individual task components, including cognitive elements such as diagnosis and selection of appropriate response.

(4) The sequence of actions uses only alarms, controls, and displays that would be available and operable during the assumed scenario(s).

(5) The estimated time for operators to complete the credited action is sufficient to allow successful execution of applicable steps in the EOPs.

45

(6) Techniques to minimize bias are used when estimates of time required are derived using methods that are dependent on expert judgment. Uncertainties in the analysis of time required are identified and assessed.

(7) Staffing for analysis is justified, and if credited manual actions require additional operators beyond the assumed staffing, the justification for timely availability of the additional staffing is provided and the estimate of time required includes any time needed for calling in additional personnel.

(8) The analysis identifies a time margin to be added to the time required and the basis for the adequacy of the margin.

The Staffs Evaluation of Criterion 9 The TA IP, Section 4.3 describes the applicants method of performing a TTA, which assesses workload and time margin. Time margin is the difference between time available6 (TmAv) to perform the task and the time required7 (TmRq) for the operators to perform the task. A TTA will be performed for all IHAs to analyze, in part8, the feasibility and reliability of tasks that are IHAs.

Bullet 1 The TA IP, Section 4.2.1 states that the TmAv for each IHA [

] Because the TmAv from the applicable plant analysis will be documented in the task narrative for each IHA, and because the source of the information will also be documented, the staff finds that the application conforms to this NUREG-0711 criterion.

Bullet 2 As discussed previously in the staffs evaluation of Criterion 5 above, DCD Tier 2, Section 18.4.2.2 states that TmRq is estimated by adding the time an operator is engaged in the task (time engaged or TmEn) and the process delay time (PdTm). PdTm is the amount of time it takes the plant component(s) to respond to the action(s) taken by the operator during task 6 The TA IP, Section 8 defines time available as the time period from the presentation of a cue for human action or equipment response to the time of adverse consequences if no action is taken.

7 The TA IP, Section 8 defines time required as the time it takes an operator to complete the action that prevents adverse consequences.

8 The TTA assesses workload level, which can impact human performance, as well as time margin. This NUREG-0711 criterion specifically addresses the assessment of time margin as a means of analyzing the feasibility and reliability of IHAs.

9 Refer to Section 7.8, Chapter 19, and Chapter 15 for the staffs evaluation of the applicants analysis method(s) and determination that the applicable acceptance criteria have been satisfied.

46

performance. PdTm contributes to the total time required to perform a task, and therefore it is added to the TmEn to determine the total time required to perform a task.

TmEn is an estimate of the amount of time the operator is engaged in task performance. [

]

[

]

Determination of subtask time is described in more detail under Bullet (3) below.

Thus, TmEn is based on a sequence of operator actions, [

] Because TmEn is an input to TmRq, TmRq is also based in part on a documented sequence of operator actions. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Bullet 3 As described under Bullet 2 above, [

] The scenario descriptions do not provide a sufficient amount of detail about the individual task components that operators must perform to accomplish the task that is an IHA. For example, DCD Tier 2, Section 15.1.4, Inadvertent Opening of a Steam Generator Relief or Safety Valve, identifies the following task that is an IHA: operator manually closes the inadvertently opened atmospheric dump valve (ADV) on the affected steam generator. To accomplish this task, the operator must perform a series of subtasks in a specific order. This information can be obtained from procedures of the predecessor plant or reference plant procedures. However, the detail in the procedures may also be limited mainly to physical elements of the task, such as, identify the affected steam generator. To identify the affected steam generator, the operator will need to monitor specific plant parameters using the displays and instrumentation in the control room. This may require navigating between screens on the operator consoles or walking from the operator console to another panel in the control room.

These activities add to the TmEn, but the plant procedures typically do not include this level of detail. Therefore, additional detail will need to be added to the action sequence in order to accurately estimate the TmEn and thus the TmRq.

[

] Therefore, on November 16, 2015, the staff issued RAI 315-8091, Question 18-50 to address this issue (ML15320A406).

47

On April 6, 2016, the applicant provided a response to RAI 315-8091, Question 18-50 (ML16097A633). [

] The staff confirmed that TA IP was revised as committed in the response to RAI 315-8091, Question 18-50. Therefore, RAI 315-8091, Question 18-50 was resolved and closed.

Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Bullet 4 The TA IP, Section 4.3 states that [

] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Bullet 5 As discussed under Bullet (2) above, [

]

[

10 This is important because for some scenarios, such as those analyzed in the D3CA, the set of alarms, controls, and indications that operators normally use may be degraded or inoperable, and therefore the operator may need to use alarms, controls and indications that he or she normally does not use.

Operators may not be as familiar with using the alternative indications and controls, and this can result in an increase to the TmEn and thus the TmRq.

11 DCD Tier 2, Section 18.8.2, Methodology, states that task analysis is an input to the APR1400 procedure development, which is a COL item (see DCD Tier 2, Section 13.5).

48

] Therefore, the estimate of TmRq will include the time it takes the operators to perform procedurally-guided actions.

Additionally, the V&V IP, Section 3.1, V&V Interfaces in the HFE Program, states that all important HAs are tested during the integrated systems validation (ISV). The V&V IP states that each scenario in the ISV is developed to simulate actual plant operator tasks, which are performed using plant procedures such as emergency operating procedures. Thus, the ISV provides the opportunity to demonstrate that operators can successfully perform credited actions in accordance with and as directed by the emergency operating procedures in the time available. If the operators cannot successfully complete IHAs during the ISV in accordance with plant procedures and in the time available, then the V&V IP, Section 4.5.5.2, Performance Measure Information and Validation Criteria, states that this will result in a failure of a scenario, and an HED will be documented and resolved in accordance with the HFE PP.

The inclusion of all tasks associated with important HAs in the ISV provides the opportunity to demonstrate that the operator(s) can use procedures to perform their tasks. Accordingly, the staff finds that the ISV will provide an opportunity to demonstrate that the operators can complete credited actions (i.e., IHAs) in accordance with EOPs, and the application conforms to this NUREG-0711 criterion.

Bullet 6 The TA IP, Section 4.3.1.2.2 describes the method for determining [

] These are subtasks, or individual task components that operators need to do to perform tasks that are IHAs. [

]

The TA IP, Section 5 [

] Thus, the staff has reasonable assurance that the plant operations SMEs will have the training and operating experience that will allow them to make valid estimates of the time it takes to perform unique subtasks when it is necessary to use SME judgment to derive TmEn and thus TmRq.

[

49

] Therefore, the staff finds that these additional time estimates provide a means of accounting for uncertainties in the estimate of TmRq.

Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Bullet 7 The TA IP, Item 18, Staff, in Section 4.2.1 [

] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Bullet 8 The TA IP, Section 4.3.2, Time Margin, states that the time required to perform a task [

] NUREG-0800, Appendix A, states, The basis for the specific time margin used in the analysis should be justified and documented. Insights from the HFE program, especially the OER and human reliability analysis, should be used. The application does not conform to this guidance. Therefore, on November 16, 2015, the staff issued RAI 315-8091, Question 18-53 to address this issue (ML15320A406).

On March 14, 2016, the applicant provided a response to RAI 315-8091, Question 18-53 (ML16074A301.) In the response, the applicant stated that the TA IP would be revised to state that the [

] The staff confirmed that TA IP was revised as committed in the response to RAI 315-8091, Question 18-

53. Therefore, RAI 315-8091, Question 18-53 was resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Combined License Items There are no COL items associated with Section 18.4 of the APR1400 DCD.

50

Conclusion The staff evaluated the applicants task analysis method and finds that it conforms to the criteria in NUREG-0711, Section 5.4. Therefore, the staff concludes that the applicants task analysis method provides for identifying the control room inventory and for determining there is reasonable assurance the operator tasks can be implemented effectively. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element.

18.5 Staffing and Qualifications Introduction The objective of the staffs review is to verify that the applicant has systematically analyzed the number and necessary qualifications of personnel in concert with task demands and regulatory requirements. The analysis addresses the full range of plant conditions and tasks including operational tasks (normal, abnormal, and emergency), plant maintenance, and plant surveillance and testing.

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: The applicant provided a Tier 2 description in Section 18.5, Staffing and Qualifications, of the DC which summarizes the method described in APR1400-K-I-NR-14005, Staffing and Qualifications Implementation Plan, that will be used by the COL applicant to complete the staffing and qualifications program element.

Changes to Tier 2 information are governed by the change control processes in the design certification rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: TeRs associated with this element are:

APR1400-K-I-NR-14005, Staffing and Qualifications Implementation Plan (S&Q IP) (ML17094A129)

APR1400-E-I-NR-14011, Basic Human-System Interface (Basic HSI TeR)

(ML18178A202) 51

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii)

Other regulatory guidance documents are as follows:

NUREG-0711, Revision 3, Chapter 6, Staffing and Qualifications, Section 6.4, Review Criteria NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.5, Staffing and Qualifications SECY-05-0197, Review of Operational Programs in a Combined License Application and Generic Emergency Planning Inspections, Tests, Analyses, and Acceptance Criteria RG 1.206, Combined License Applications for Nuclear Power Plants [Light Water Reactor] LWR Edition), Section C.IV.4, Operational Programs Technical Evaluation NUREG-0711, Section 6.4, Review Criteria, includes six criteria for this topic.

Criterion 1 The applicant should address the applicable staffing and qualifications guidance in NUREG-0800, Section 13.1.

The Staffs Evaluation of Criterion 1:

This criterion is evaluated in Section 13.1 of this SER.

Criterion 2 The applicant should address the applicable staffing and qualifications guidance in 10 CFR 50.54, Conditions of Licenses.

The Staffs Evaluation of Criterion 2:

The requirements contained in 10 CFR 50.54(k) and 50.54(m) identify requirements for licensed operators. The requirements are conditions in every nuclear power reactor operating license issued under 10 CFR Part 50, Domestic Licensing of Production and Utilization Facilities.

Also, the requirements are conditions in every combined license issued under 10 CFR Part 52, Licenses, Certifications, and Approvals for Nuclear Power Plants; however, they are only applicable after the Commission makes the finding under 10 CFR 52.103(g) that the acceptance criteria in the combined license are met. Although the requirements in 10 CFR 50.54 apply to 52

license holders and not DC applicants, the staff evaluates a DC applicants proposed control room design to determine whether a license applicant will be able to comply with the staffing requirements in 10 CFR 50.54. If the license applicant will not be able to comply, then the license applicant will need to request an exemption. The license applicant would need to provide sufficient technical justification to support the exemption request.

Alternatively, the DC applicant may provide sufficient technical justification with the DC application to support a design-specific staffing rule that would be used in lieu of 10 CFR 50.54(m). This option provides the greatest degree of issue finality, which may be preferable to DC applicants. Thus, the staff evaluates whether an applicants proposed control room design would enable a licensee to comply with the staffing requirements in 10 CFR 50.54(m) at the DC stage to determine whether additional documentation must be submitted to develop a design-specific staffing rule.

The staff determined that the applicants proposed control room design provides for a licensees compliance with the staffing requirements of 10 CFR 50.54(m) and 10 CFR 50.54(k) based on the following information included in the application:

The Basic HSI TeR, Section 3.5.1, Crew Composition, lists the number of licensed operators and senior operators that the applicant initially assumes will staff the MCR. The number of licensed operators and senior operators satisfies the minimum required by 10 CFR 50.54(m)(1), 10 CFR 50.54(m)(2)(ii), and 10 CFR 50.54(m)(2)(iii) for control room staff and on-site staff.

Requirements in 10 CFR 50.54(m)(2)(i) state that for a single unit operated from a single control room, two senior operators and two operators must be on-site when the unit is operating. DCD Tier 2, Section 18.1.1.1 describes the initial staffing level assumed for the main control room. It states that there will be two senior operators, which will be the shift supervisor and the STA, and there will be three operators, which will be the reactor operator, the turbine operator, and the electrical operator. Therefore, the initial staffing assumption will help to ensure a licensee can comply with the requirements in 10 CFR 50.54(m)(2)(i).

However, other portions of the DCD that described staffing did not fully describe the qualification requirements of the STA. Specifically, those portions indicated that the STA would have an engineering degree, but they did not indicate that the STA would be a senior operator. The staff considered this to be an inconsistency in the application. Therefore, on August 21, 2015, the staff issued RAI 107-8039, Question 18-11 to address this issue (ML15234A008). On September 15, 2015, the applicant provided a response to RAI 107-8039, Question 18-11 (ML15258A646). The applicant confirmed that the STA would also be a senior operator and submitted a proposed revision to the HFE PP, Section 4.1 to align it with DCD Tier 2, Section 18.1.1.1. The staff confirmed that HFE PP was revised as committed in the response to RAI 107-8039, Question 18-11. Therefore, RAI 107-8039, Question 18-11 was resolved and closed.

The HFE PP, Section 4.1, Assumptions and Constraints, identifies the requirement in 10 CFR 50.54(m)(2)(iii) as a constraint for MCR staff level. The final staffing level for the control room results from the S&Q analyses; however, because of the constraint, the number of licensed operators in the control room cannot be lower than that required by 50.54(m)(2)(iii). The requirements of 10 53

CFR 50.54(k) will also be satisfied because compliance with 10 CFR 50.54(m)(2)(iii) ensures compliance with 10 CFR 50.54(k).

The Basic HSI TeR, Section 3.5.1 describes the staffing concept for the MCR.

Some members of the MCR staff may leave the control room at times to perform tasks outside the control room. However, the staffing concept acknowledges the staffing constraint, and therefore ensures that the requirements of 10 CFR 50.54(m)(2)(iii) and 10 CFR 50.54(k) can be satisfied.

Requirements in 10 CFR 50.54(m)(2)(iv) state that a senior operator must directly supervise core alternations (including fuel loading or transfer). This is an administrative requirement that is not dependent on the design, and therefore the staff concludes that the design does not preclude a licensee from meeting the requirement.

Thus, the staff finds the applicant has sufficiently addressed the staffing requirements in 10 CFR 50.54(m) such that an applicant for an operating license or combined license will be able to conform to the conditions in the license. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 3 The applicant should use the results of the task analysis as an input to the staffing and qualification analyses. Personnel tasks, addressed in task analysis, should be assigned to staffing positions to ensure that jobs are defined considering:

the task characteristics, such as the knowledge and abilities required, relationships among tasks, time required to perform the task, and estimated workload; the persons ability to maintain situation awareness within the area of assigned responsibility; and teamwork and team processes, such as peer checking.

The Staffs Evaluation of Criterion 3:

S&Q IP, Section 4.2, Task Analysis Input to S&Q, describes two steps in the S&Q analysis. In the first step, a subject matter expert collects all the task analysis data, including the elements listed in the first bullet of the acceptance criterion, and completes an examination of the related tasks making judgments regarding staffing numbers and qualifications. The second step links tasks into sequences which are assigned to specific staff positions. The staff position is then assessed for the ability to maintain situation awareness and teamwork.

The staff concludes that the task analysis data is being appropriately collected and input into the S&Q analysis. The S&Q analysis appropriately uses the task analysis information to assess staffing level and qualification. Therefore, the staff finds that the application conforms to this criterion.

Criterion 4 The applicants staffing analysis should determine the number and qualifications of operations personnel for the full range of plant conditions and tasks, including operational tasks (under 54

normal, abnormal, and emergency conditions), plant maintenance, plant surveillance, and testing.

The Staffs Evaluation of Criterion 4 S&Q IP, Section 4.3.2, S&Q Analysis, states that within the general S&Q analysis method, a broad spectrum of evolutions representative of all plant operating modes (Modes 1 through 6) under normal, abnormal, and emergency conditions are identified and analyzed for their impact on S&Q. The IP provides a proprietary list of criteria that define how these evolutions are identified. Generalized, these criteria place a priority on analyzing:

HEDs affecting S&Q identified in the previous HFE design elements including the OER and the task analysis, evolutions presenting the greatest challenge to the operations crew, and evolutions containing important human actions.

Plant maintenance, plant surveillance, and testing activities are included in the criteria used to determine evolutions presenting the greatest challenge to the operations crew.

The staff concludes that the analysis method used for establishing staffing and qualification levels addresses the full range of plant conditions and activities and has appropriate prioritization criteria to identify those most challenging to staffing and qualifications. The method appropriately uses the input from the previous HFE program elements. Therefore, the staff finds that the application conforms to this criterion.

Criterion 5 The applicants staffing analysis should be iterative; that is, the initial staffing goals should be modified as information from the HFE analyses from other elements becomes available.

The Staffs Evaluation of Criterion 5:

S&Q IP, Section 4.4, Process Iterations, states that the S&Q analysis is iterative in that HEDs generated by other HFE program elements are evaluated for any potential changes needed in these analyses. Similarly, plant design changes are evaluated for their impact on the output of all HFE program elements, including the output of the S&Q.

The staff concludes that the S&Q methodology provides for modification of the staffing goals via the HED resolution process. Therefore, the staff finds that the application conforms to this criterion.

Criterion 6 The applicant should address the basis for staffing and qualification levels considering the specific staffing-related issues noted below. These considerations may be identified in other HFE elements or in related source documents as follows:

Operating Experience Review o operational problems and strengths resulting from staffing levels in predecessor designs 55

o initial staffing goals and their bases, including staffing levels of predecessor designs and a description of significant similarities and differences between predecessor and current designs o staffing considerations described in NRC Information Notice 95-48, Results of Shift Staffing Study o possible impact on staffing of requirements of limits to work hours, required break times, and required days off, as specified in 10 CFR 26.205, Work Hours, as part of the Fitness for Duty Rule o Regulatory Issue Summary (RIS) 2009-10, Communications Between the NRC and Reactor Licensees During Emergencies and Significant Events Functional Requirements Analysis and Function Allocation o potential mismatches between functions allocated to personnel and their qualifications o changes to the roles of personnel due to modifying the plants systems and HFE aspects Task Analysis o time needed to perform a task, and the workload involved o personnel communication and coordination, including interactions between individuals for diagnosing, planning, and controlling the plant, and interactions between personnel for administrative, communications, and reporting activities o the job requirements resulting from the sum of all tasks allocated to each individual inside and outside the control room o potential decreases in the ability of personnel to coordinate their work due to changes to the plant o availability of personnel considering other work that may be ongoing, and for which operators may be responsible outside the control room (e.g., fire brigade) o actions identified in 10 CFR 50.47, Emergency Plans; NUREG-0654; and procedures to implement an initial accident response in key functional areas, as denoted in the emergency plan o staffing considerations described by the application of ANSI/ANS 58.8-1994, Time Response Design Criteria for Safety-Related Operator Actions (ANS, 1994), if used by the applicant Treatment of Important Human Actions o the effect of staffing levels on the performance of the identified important HAs 56

o the effect of staffing levels on personnel coordination for important HAs o NUREG/CR-6753, Review of Findings for Human Performance Contribution to Risk in Operating Events Procedure Development o staffing demands resulting from requirements to concurrently use multiple procedures o personnel knowledge, abilities, and authorities identified in the procedures Training Program Development o concerns about coordinating personnel that are identified during the development of training The Staffs Evaluation of Criterion 6 S&Q IP, Section 4.5, Considerations for Staffing and Qualification Levels, describes how HFE program elements (OER, FRA/FA, TA, and human reliability analysis [HRA]) act as inputs to the S&Q analyses and provides for the consideration of the manning level considerations identified in the acceptance criterion. The initial staffing and qualification levels are derived from the Shin Kori 3&4 reference plant, the Palo Verde Nuclear Generating Station predecessor plant, and the System 80+ predecessor design. The HFE program elements identify weaknesses in these predecessor plant configurations and document these as HEDs and the HEDs are evaluated within the S&Q analysis. The HED resolution process may modify the staffing level or staff qualification requirements. Criteria within the OER, task analysis, and TIHA elements were not addressed within the IP. Therefore, on August 21, 2015, the staff initiated RAI 107-8039, Question 18-12 to address this issue (ML15234A008).

On October 07, 2015, the applicant provided a response to RAI 107-8039, Question 18-12 (ML15280A151) which included a markup of the IP showing how the omitted criteria would be addressed. The added information is consistent with this criterion; therefore, the response is satisfactory. The staff confirmed that S&Q IP was revised as committed in the response to RAI 107-8039, Question 18-12. Therefore, RAI 107-8039, Question 18-12 was resolved and closed.

The staff concludes that the applicants S&Q analysis method provides a basis for staffing levels and qualifications that address inputs from the OER, FRA/FA, TA, HRA, HSI design, procedures development, and training development. Therefore, the staff finds that the application conforms to this criterion.

Combined License Items There are no COL items associated with Section 18.5 of the APR1400 DCD.

Conclusion The staff evaluated the applicants staffing and qualifications analysis method and finds that it conforms to the criteria in NUREG-0711, Section 6.4. Therefore, the staff concludes that the applicants staffing and qualifications analysis method provides for systematically analyzing the 57

required number and necessary qualifications of personnel, in concert with regulatory requirements and task requirements. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element.

18.6 Treatment of Important Human Actions Introduction The TIHA program element identifies the human actions (HAs) that are most important to safety and considers those HAs in the HFE design of the plant. The design should minimize the likelihood of personnel error and help ensure that personnel can detect and recover from any errors that occur.

Probabilistic and deterministic analyses are used to identify IHAs. The probabilistic risk assessment (PRA), which includes HRA, identifies risk-important human actions (RIHAs).

Deterministic engineering analyses identify deterministically important human actions (DIHAs) that are credited with the prevention or mitigation of accidents and transients.

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: Section 18.6, Treatment of Important Human Actions, summarizes the methodology described in APR1400-E-I-NR-14006, Treatment of Important Human Actions Implementation Plan, that will be used by the COL applicant to identify IHAs and integrate them into the HFE program.

Changes to Tier 2 information are governed by the change control processes in the design certification rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: TeRs associated with this element are:

APR1400-E-I-NR-14006, Treatment of Important Human Actions Implementation Plan (TIHA IP) (ML18178A202)

APR1400-E-I-NR-14001, Human Factors Engineering Program Plan (HFE PP)

(ML18212A336)

APR1400-E-I-NR-14003, Function Requirements Analysis/Function Allocation Implementation Plan (FRA/FA IP) (ML18081A107) 58

APR1400-E-I-NR-14004, Task Analysis Implementation Plan (TA IP)

(ML18178A202)

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii)

Other regulatory guidance documents are as follows:

NUREG-0711, Revision 3, Chapter 7, Treatment of Important Human Actions, Section 7.4, Review Criteria NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.6, Human Reliability Analysis Technical Evaluation NUREG-0711, Section 7.4, Review Criteria, includes four criteria for this topic. The fourth criterion addresses plant modifications and is not applicable to new reactors; therefore, the staff evaluated the first three criteria as discussed below.

Criterion 1 The applicant should identify risk-important HAs from the PRA/HRA.

The Staffs Evaluation of Criterion 1 The TIHA IP, Section 4.1, Risk-Important Human Actions, explains the method for ensuring the RIHAs and their associated HFE characteristics, as identified in the PRA documentation, are included in the HFE design program. Section 4.1 states that [

] Although this NUREG-0711 criterion is limited to identifying the RIHAs, the staff concludes that documenting the assumptions associated with each RIHA and confirming them during other HFE program elements (e.g., HSI design and integrated system validation) is a good practice that provides an opportunity to validate assumptions in the PRA.

The TIHA IP, Section 4.1 states, [

59

] The staff reviewed DCD, Revision 0, Chapter 1912 and found that it discusses risk-significant operator actions, which are labeled as key operator actions in Tables 19.1-24, -25, -38, -39, -55, -56, -61, -62, -72, -73, -78, -79, -104, -105, -117, -118, -130, -

131, -145, -146, -159, and -160. The risk-significant operator actions have been identified using two different importance measures: risk achievement worth (RAW) and Fussell-Vesely (FV) importance. Because the risk-significant human actions (i.e., the RIHAs) have already been identified in DCD Tier 2, Chapter 19, the staff did not understand why it was necessary to defer identification of the RIHAs to the COL applicant when the information was provided as part of the design certification application. Additionally, the staff did not find any information about HFE characteristics for the RIHAs in DCD Chapter 19, and the staff determined it was necessary to ask the applicant to clarify in the TIHA IP where the information is located. Therefore, on August 5, 2015, the staff issued RAI 128-7980, Questions 18-13 and 18-14, to address this issue (ML15227A008).

On December 18, 2015, the applicant provided a response to RAI 128-7980, Questions 18-13 and 18-14 (ML15352A246). The applicant proposed a revision to the TIHA IP to include a list of the RIHAs and its associated HFE characteristics from the PRA documentation as well as references for the information containing the HFE characteristics in Appendix C in the next revision of the TIHA IP. The staff reviewed the list of RIHAs provided in response to RAI 128-7980, Question 18-13, and compared it to the tables of RIHAs in DCD Tier 2, Chapter 19. The staff found that the list of RIHAs provided in response to RAI 128-7980, Question 18-13 contained the same RIHAs as the tables of key operator actions in DCD Tier 2, Chapter 19, but a few of the RIHAs in DCD Tier 2, Chapter 19, were not included.

The staff reconsidered whether it was prudent to have two different parts of the application identify RIHAs because if the information in DCD Tier 2, Chapter 19 is revised during later stages of the review, then even if the information in the TIHA IP is initially the same as that in the DCD, that may not be true in the future. Differences between the list of RIHAs in the TIHA IP, Appendix C and in the DCD Tier 2, Chapter 19 could cause confusion for the COL applicant when performing the activities in the TIHA IP, which could result in some RIHAs not being included in the HFE design program.

As, such, the staff requested the applicant to instead add more detail to the PRA documentation to ensure that the RIHAs and its HFE characteristics would indeed be clearly labeled such that an SME with no PRA knowledge would be able to look at the right documents and identify RIHAs reliably and accurately. Therefore, on August 1, 2016, the staff issued RAI 510-8650, Question 18-127 to address this issue (ML16214A307).

On August 26, 2016, the applicant provided a response to RAI 510-8650, Question 18-127 (ML16239A436), which contained a proposed revision to the TIHA IP, Section 4.1, to say that the RIHAs would be identified from the PRA documentation by [

] The proposed revision also added additional sources of the HFE characteristics to the list of references in the TIHA IP. These sources include the HRA documentation prepared for the design certification. Additionally, the applicant retained the proposed Appendix C to the TIHA IP, but the applicant clarified that the information in the appendix is preliminary information only.

The staff concluded the proposed revisions were acceptable to ensure the RIHAs and their associated HFE characteristics will be included in the HFE design program for two reasons.

12 Section 19.1 of this SER contains the staffs evaluation of the applicants method for identifying the RIHAs and performing the HRA.

60

First, SMEs that do have [ ] are necessary to ensure the RIHAs and their associated HFE characteristics are correctly identified from the PRA documentation (i.e., DCD Tier 2, Chapter 19 and the PRA-related documents listed in the TIHA IP). Second, the TIHA IP states that the source of the information duplicated in Table 4-1, which is the input to the rest of the HFE design program, is the PRA documentation (i.e., DCD Tier 2, Chapter 19 and the PRA-related documents listed in the TIHA IP) and not the preliminary list in Appendix C of the TIHA IP. The staff confirmed that the TIHA IP was revised as committed in the response to RAI 510-8650, Question 18-127. Therefore, RAI 510-8650, Question 18-127 was resolved and closed.

During the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member questioned why the PRA developed for the design certification will be used to perform the activities described in the TIHA IP given that the COL applicant will perform these activities, and the COL applicant will develop a site-specific PRA and HRA. Also, an ACRS member questioned whether the level of expertise of the SMEs who will complete Table 4-1 is sufficient to ensure that the RIHAs and their associated HFE characteristics will be included the other HFE program elements.

Additionally, an ACRS member discussed that some aspects of the site-specific PRA (i.e., the quantification of seismic risk) may not be determined until fuel load, which occurs after the control room has been constructed. The ACRS member questioned how any RIHAs identified as a result of quantifying the seismic PRA will be addressed in the HFE design program.

Following the ACRS meeting, the staff reevaluated the TIHA IP and determined it did not fully conform to guidance in NUREG-0711 in the following three ways:

NUREG-0711, Section 7.1, Background, states, The analyses should be updated iteratively as the design progresses (including the final PRA/HRA) to ensure the actual important HAs are captured and considered. As such the staff determined it was necessary to request the applicant specify that the COL applicant use the site-specific PRA rather than the PRA developed for the generic design to perform the activities described in the TIHA IP because the site-specific PRA will contain more information than the generic PRA.

Also, given that the COL applicant should use the site-specific PRA to complete Table 4-1, the staff requested the applicant remove Appendix C, which was based on the design certification application PRA results.

NUREG-0711, Section 7.1 states, HRA is an integral part of a completed PRA. Applicants submit PRAs in accordance with the NRCs current requirements. An HRA evaluates the potential for, and mechanisms of human error that might affect plant safety.

Thus, it is an essential feature in assuring the HFE program goal of generating a design to minimize personnel errors, support their detection, and ensure recovery capability. The HRA is an integrated activity supporting both the HFE design and PRA activities. The robustness and quality of the HRA largely depends on the analyst's understanding of the causes, modes and probabilities of human error, the personnel tasks to be performed, information about those tasks, and any task-specific factors that may influence the human performance of them.

Analysts should employ the descriptions and analyses of personnel functions and tasks, along with the operational characteristics of the HSIs. The HRA provides 61

valuable insights into the desirable characteristics of the HSI design.

Consequently, the HFE design should pay special attention to those plant scenarios, risk-important HAs, and HSIs that the PRA/HRA highlights as vital to plant safety and reliability.

As explained in NUREG-0711, the results of the HRA provide input to both the PRA and HFE design activities. Specific expertise is necessary to develop the HRA. The staff questioned whether SMEs with [PRA knowledge], which does not specifically include HRA expertise, will be able to fully understand and apply the HRA results to the HFE design program. As such, the staff determined it was necessary to ask the applicant to specify that SMEs with HRA knowledge will also be necessary to accurately complete Table 4-1.

Additionally, the staff could not find where the applicant explained whether or how any RIHAs identified as a result of quantifying the seismic PRA will be addressed in the HFE design program.

Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-133 to address these three issues (ML17249A979).

On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-133 (ML17271A188). As part of the response, the applicant proposed the following changes to the application:

Section 4.1 of the TIHA IP will be revised to specify that the site-specific PRA will be used to complete Table 4-1 of the TIHA IP.

DCD Tier 2, Section 18.6.2 and Section 4.1 of the TIHA IP will be revised to explain that I&C engineering subject matter experts (SMEs) will coordinate with personnel who have PRA/HRA knowledge to extract the RIHAs from the PRA documentation and complete Table 4-1. The TIHA IP will also be revised to provide context for the preliminary output in Appendix C. Specifically, the TIHA IP will be revised to state the following:

[

]

The staff concludes that these changes to the application ensure that Table 4-1, which provides input to the other HFE elements in the HFE design program, is completed by SMEs with adequate expertise to reliably and correctly identify the RIHAs and their associated HFE characteristics. Further, because site-specific PRA documentation will be used, the most current PRA information available when the TIHA activities are completed will be used.

Therefore, the staff finds that the applicants method to ensure RIHAs will identified from the PRA documentation and included in the HFE design program is acceptable.

62

Additionally, as part of the response to RAI 553-9084, Question 18-133, the applicant addressed treatment of additional RIHAs that may be identified from quantification of the seismic PRA. The applicant stated:

As stated in Section 3.5.8 of the TIHA IP, Revision 1, the design implementation (DI) PE demonstrates that the as-built HSI design, including the HSI for IHAs, for each site specific APR1400 reflects the HSI design output from the human factors verification and validation (HF V&V) PE. Therefore, if additional IHAs are identified for a specific plant, and those IHAs are not encompassed by the HF V&V PE, HEDs are generated. Those HEDs are resolved as part of the DI PE, using HED resolution process of the HFE PP. The design changes after DI PE will be resolved using COLAs corrective action program.

The staff finds the applicants response acceptable for the following reasons. The activities in the HFE IPs, including identification of RIHAs for inclusion in the HFE design process, must be completed by the COL applicant in order to close the HFE-related ITAAC identified in DCD Tier 2, Section 2.9, Human Factors Engineering. These ITAAC must be complete before fuel load.

Section 50.71(h)(1) of 10 CFR, requires a COL applicant to quantify seismic risk by fuel load, and therefore, it is possible that potentially important contributions to risk that affect hardware design and human factors engineering would not be identified until the plant has been constructed. As such, changes to the plant design, plant operating procedures, or the plant personnel training programs that may be needed following identification of any new RIHAs from the quantification of the seismic risk will need to be addressed by implementation of operational programs, such as the corrective action program, plant procedure development program, and plant personnel training program, which are subject to appropriate change control processes and NRC inspection.

Based on the review of the DCD and TIHA IP, the staff has confirmed incorporation of the changes described above; therefore RAI 553-9084, Question 18-133 was resolved and closed.

Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should identify deterministically-important HAs from the following licensing analyses:

Operator actions credited in the DCD Chapter 15 accident and transient analyses Operator actions identified in the D3 coping analyses (D3CA) performed for DCD Chapter 7, as specified in Section 1 and 2 of Interim Staff Guidance DI&C-ISG-02, Diversity and Defense in Depth (D3) Issues (NRC, 2009)

The Staffs Evaluation of Criterion 2 Unlike DCD Tier 2, Chapter 19, which contains tables that label and list the operator actions that are risk-significant (i.e., the RIHAs), the operator actions that are required to mitigate accidents described in the D3CA and DCD Tier 2, Chapter 15, are described in the event evaluation subsections of these documents. The TIHA IP, Section 4.2, Deterministically Important Human Actions, states that a plant operations or systems safety engineering SME will review the transient and accident analyses (TAA) in DCD Tier 2, Chapter 15 and the diversity and defense-in-depth coping analysis (D3CA) of DCD Tier 2, Chapter 7, to identify the DIHAs to be included in the HFE design process. The SMEs will [

63

]

The HFE PP, Section 5 lists the qualifications of the plant operations and systems safety engineering SMEs. Plant operations SMEs are required to have previous experience operating a nuclear power reactor and to have completed an extensive amount of training on the operation of nuclear power plants during transients and accidents. Systems safety engineering SMEs are required to have previous experience in safety system engineering. The staff concludes that SMEs with these qualifications will have knowledge of how safety systems operate and the actions that operators need to take to operate the safety systems, and therefore SMEs with these qualifications will be able to identify the DIHAs in the D3CA and DCD Tier 2, Chapter 15, and include them in the HFE design process. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 3 The applicant should specify how important HAs are addressed by the HFE program, in Function Allocation, Task Analysis, HSI design, Procedural Development, and Training Program Development, in order to minimize the likelihood of human error and facilitate error-detection and recovery capability.

The Staffs Evaluation of Criterion 3 DCD Tier 2, Section 18.6.1, Objectives and Scope, states that all IHAs are addressed in the functional requirements analysis/function allocation, task analysis, HSI design, procedural development, and training program development program elements to minimize the likelihood of human error and facilitate error-detection and recovery capability as described below.

Functional Requirements Analysis/Function Allocation: The HFE PP, Section 4.7.3.2, Functional Requirements Analysis and Function Allocation Interfaces, states that the function allocation confirms that the actions the PRA, TAA, and D3CA analyses assume will be performed by human actions (i.e., the IHAs) have been appropriately assigned to humans by applying the allocation criteria contained in the FRA/FA IP, Appendix C, Function Allocation Selection Analysis. If IHAs have been inappropriately assigned to humans, then the likelihood of human error during performance of the IHA increases. If the function allocation process concludes that the actions should be automated either partially or fully in order to ensure the safe operation of the plant, then an HED will be documented and resolved in accordance with the HFE PP, Section 4.6. Because the IHAs are reviewed during the function allocation process to confirm that IHAs have been appropriately allocated to human actions, the staff finds that the applicants function allocation method adequately addresses IHAs.

Task Analysis: DCD Tier 2, Section 18.4.2.1 states that a BTA and a TTA will be performed for each task that is an IHA. The BTA identifies the HSI that must be available for operators to perform the IHA, which is an input to the HSI design program element. The TTA provides an estimate of the workload and time margin associated with performing each IHA. The workload and time margin are then compared to acceptance criteria defined in the TA IP, Section 4.3.1, Workload, and Section 4.3.2, Time Margin. When workload and time margin do not meet the acceptance criteria, an HED will be documented and resolved in accordance with the HED resolution process described in the HFE PP, Section 64

4.6. HEDs may result in changes to the HSI to help minimize the likelihood of human error or facilitate error-detection and recovery capability, as necessary.

Because IHAs are assessed during the task analysis process to ensure that they are within acceptable human performance capabilities, the staff finds that the applicants task analysis method adequately addresses IHAs.

HSI Design: DCD Tier 2, Section 18.6.3, Results, states that the TIHA ReSR includes the list of IHAs and their associated HFE characteristics. These characteristics include the alarms, controls, and indications (i.e., the HSI) that the analyses in the PRA, TAA, and DC3A assume will be available to operators to perform actions that are required to mitigate the consequences of transients, design basis accidents, and beyond-design basis accidents. The HFE PP, Section 4.7.3.5, Treatment of Important Human Action Interfaces, states that the TIHA ReSR is an input to the HSI design. This ensures that the HSI credited in the plant analyses is included in the HSI inventory in the plant. Because the information about the HSI required to support the performance of IHAs is included in the HSI design process to ensure that the HSI that operators need is available, the staff finds that the applicants HSI design process adequately addresses IHAs.

Procedure Development: Procedures provide step-by-step directions for performing tasks in the plant, including IHAs. Detailed procedures and use of procedure adherence during task performance help to minimize human errors.

DCD Tier 2, Section 18.4.2.1 states that tasks that are IHAs are included in the task analysis process, and a BTA and TTA will be performed for each IHA. Task narratives are developed during the BTA, and they specify what personnel have to do and relationships among tasks. The HFE PP, Section 4.7.3.3 states that the task narratives are inputs to procedure development to identify the actions in the procedures and the sequence of those actions in the procedures. Thus, procedures will be developed that direct how to perform IHAs in the plant.

Therefore, the staff finds that the applicants procedure development process adequately addresses IHAs.

Training Program Development: Because all IHAs are included in the task analysis process, a BTA will be developed for each task that is an IHA. The TA IP, Section 4.2.1, Basic Task Analysis, states that each task narrative developed during the BTA identifies the knowledge and abilities that plant personnel must have to perform the task. The HFE PP, Section 4.7.3.3 describes that the learning objectives included in the training program will be derived in part from the task narratives produced during the task analysis.

Systematic analysis of tasks, which occurs during task analysis, and the development of learning objectives from that analysis are two of the five elements of a systems approach to training13. In DCD Tier 2, Section 13.2.1, Plant Staff Training Program, the applicant included a COL item for the COL applicant to develop a site-specific training program by using NEI 06-13A, Template for an Industry Training Program Description, which describes a systems approach to training method. Including IHAs in the operator training program helps to ensure that operators are trained to correctly perform those 13Section 55.4 of 10 CFR, Definitions, defines the five elements of a systems approach to training.

65

actions, which helps to minimize human error. Therefore, the staff finds that the applicants training development process adequately addresses IHAs.

The applicant also described that the IHAs are addressed in other HFE program elements, including V&V. NUREG-0711, Section 11.4.3.2, Test Objectives, states that one objective of the applicants ISV, which is part of the V&V, should be to validate that the HSIs minimize personnel error and assure error detection and recovery capability when errors occur. V&V IP, Section 4.5.2, Test Objectives, includes this objective, and Section 3.1 states that all IHAs are included in the ISV scenarios.

Therefore, the staff concludes that the applicant addresses and incorporates IHAs in the function allocation, task analysis, and HSI design program elements, as well as in the V&V Program Element, in order to minimize the likelihood of human error and facilitate error-detection and recovery capability. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Combined License Items There are no COL items associated with Section 18.6 of the APR1400 DCD.

Conclusion The staff evaluated the applicants treatment of important human actions and finds that it conforms to the criteria in NUREG-0711, Section 7.4. Therefore, the staff finds there is reasonable assurance that the applicants method identifies IHAs and considers them in the HFE design. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element.

18.7 Human System Interface Design Introduction The HSI design element represents the translation of function and task requirements into HSI design specifications. The objective of this review is to evaluate how HSI designs are identified and refined. The review verifies that the applicant has a process to translate functional and task requirements to the detailed design of alarms, displays, controls, and other aspects of the HSI through the systematic application of HFE principles and criteria.

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: The applicant provided a description in Section 18.7, Human-System Interface Design, that summarizes the HSI design process described in APR1400-E-I-NR-14007, Human-System Interface Design Implementation Plan, that will be used by the COL applicant to develop the HSI design.

66

Changes to Tier 2 information are governed by the change control processes in the design certification rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: TeRs associated with this element are:

APR1400-E-I-NR-14007, Human-System Interface Design Implementation Plan (HD IP) (ML18178A202). The HD IP describes the method that will be used to develop the APR1400 HSI Design.

APR1400-E-I-NR-14011, Basic Human-System Interface (Basic HSI TeR)

(ML18178A202). The Basic HSI TeR describes the APR1400 Basic HSI conceptual design. It is an input to the method described in the HD IP to develop the APR1400 HSI Design.

APR1400-E-I-NR-14012, Style Guide (Style Guide) (ML18081A107). The Style Guide contains HFE guidance and design-specific conventions applicable to all HSI resources (e.g., displays and controls used to operate the plant) and facilities, including the MCR, RSR, and TSC. It is applicable to the APR1400 HSI Design.

APR1400-E-I-NR-14001, Human Factors Engineering Program Plan (HFE PP)

(ML18212A336).

APR1400-E-I-NR-14002, Operating Experience Review Implementation Plan (OER IP) (ML18081A107).

APR1400-E-I-NR-14003, Functional Requirements Analysis and Function Allocation Implementation Plan (FRA/FA IP) (ML18081A107).

APR1400-E-I-NR-14004, Task Analysis Implementation Plan (TA IP)

(ML18178A202).

APR1400-K-I-NR-14005, Staffing and Qualifications Implementation Plan (S&Q IP) (ML17094A129).

APR1400-E-I-NR-14006, Treatment of Important Human Actions Implementation Plan (TIHA IP) (ML18178A202).

APR1400-E-I-NR-14008, Human Factors Verification and Validation Implementation Plan (V&V IP) (ML18178A202).

APR1400-Z-J-NR-14012, Control System CCF Analysis, (ML18212A336).

APR1400-Z-A-NR-14019, CCF Coping Analysis, (ML18086B746).

67

APR1400-Z-J-NR-14001, Safety I&C System, (ML18212A336).

Regulatory Basis The relevant requirements of the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

10 CFR Part 50, Appendix A, General Design Criteria (GDC) 19 - control room 10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) 10 CFR 50.34(f)(2)(iv) - safety parameter display system (SPDS) 10 CFR 50.34(f)(2)(v) - automatic indication of the bypassed and operable status of safety systems 10 CFR 50.34(f)(2)(xi) - relief and safety valve indication 10 CFR 50.34(f)(2)(xii) - auxiliary feedwater system flow indication 10 CFR 50.34(f)(2)(xvii) - containment related indications 10 CFR 50.34(f)(2)(xviii) - core cooling indications 10 CFR 50.34(f)(2)(xix) - instrumentation for monitoring post-accident conditions that includes core damage 10 CFR 50.34(f)(2)(xxvi) - leakage control 10 CFR 50.34(f)(2)(xxvii) - radiation monitoring Other regulatory guidance documents are as follows:

NUREG-0711, Revision 3, Chapter 8, Human-System Interface Design, Section 8.4, Review Criteria NUREG-0700, Human-System Interface Design Review Guidelines, Revision 2 NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.7, Human-System Interface Design NUREG-0800, Appendix 18-A, Guidance for Crediting Manual Operator Actions in Diversity and Defense-in-Depth (D3) Analyses, Revision 0 NUREG-0835, NUREG-1342, and Supplement 1 of NUREG-0737 relating to safety parameter display system requirements, as described in 10 CFR 50.34(f)(2)(iv) 68

Bypassed and inoperable status indication for nuclear power plant (NPP) safety systems, as described in RG 1.47, Bypassed and Inoperable Status Indication for Nuclear Power Plant Safety Systems Instrumentation for light water cooled nuclear power plants to access plant and environmental conditions during and following an accident, as described in RG 1.97, Criteria for Accident Monitoring Instrumentation for Nuclear Power Plants Functional criteria for emergency response facilities, as described in NUREG-0696 Technical Evaluation 18.7.4.1 HSI Design Inputs NUREG-0711, Section 8.4.1, HSI Design Inputs, includes four criteria for this topic.

Criterion 1 Analysis of Personnel Task Requirements - The applicant should use the following analyses, performed in earlier stages of the design process, to identify requirements for the HSIs:

Operational Experience Review - An input to the HSI design should encompass lessons learned from other complex human-machine systems, especially predecessor designs and those involving similar HSI technology.

Functional Requirements Analysis and Function Allocation - The HSIs should support the roles of personnel in the plant, e.g., appropriate levels of automation.

Task Analysis - The set of requirements to support the role of personnel is provided by task analyses that should identify:

o tasks needed to control the plant during a range of operating conditions from normal through accident conditions o detailed information and control requirements (e.g., requirements for display range, precision, accuracy, and units of measurement) o task support requirements (e.g., special lighting and ventilation requirements) o Important Human Actions (IHAs), as defined in Section 7 of NUREG-0711, that should be given special attention in the HSI design process Staffing and Qualifications - The findings from analyses of staffing/qualifications should provide input for deciding upon the layout of the overall control room and allocating controls and displays to individual consoles, panels, and workstations.

The staffing/qualifications analyses establish the basis for the minimum and maximum number of personnel to be accommodated, and requirements for coordinating activities between them.

69

The Staffs Evaluation of Criterion 1 The HD IP describes the process that will be used by a COL applicant to develop the APR1400 HSI Design, which will be verified and validated in accordance with the V&V IP. The starting point for the development of the APR1400 HSI Design is the APR1400 Basic HSI conceptual design, which the applicant described in the Basic HSI TeR14. As discussed in more detail in the staffs evaluation of Criterion 2 in Section 18.1.4.1 of this document and in DCD Tier 2, Section 18.7.2.4, HSI Concept Design, the APR1400 Basic HSI conceptual design is based on SKN 3&4, which are the plants that best represent the APR1400 design.

The process described in the HD IP that a COL applicant will use to develop the APR1400 HSI Design from the APR1400 Basic HSI conceptual design includes using the results of the analyses listed in this criterion as follows.

Operational Experience Review (OER): The HD IP, Section 3.5.1, Operating Experience Review, states that issues identified during the OER, which is conducted in accordance with the OER IP, may require changes to the APR1400 Basic HSI conceptual design. As described in the OER IP, Section 4.8, Operating Experience Lessons Learned Analysis, lessons learned from reviewing relevant operating experience will be documented and evaluated to determine whether any changes must be made to the APR1400 HSI Design.

The OER IP, Section 4.7, Classifying Operating Experience, states that design changes will be made when the operating experience indicates it is necessary to protect the health and safety of the plant staff. Therefore, the results of the OER will identify inputs to the APR1400 HSI Design by identifying changes that may need to occur to address lessons learned identified by the OER.

Functional Requirements Analysis (FRA) and Function Allocation (FA): The HD IP, Section 3.5.2, Functional Requirements Analysis and Function Allocation, describes that the results of the FRA, which includes the identification of the controls and displays operators need to monitor and control the critical safety functions, is used to develop the HSI inventory, which includes controls and displays, for the APR1400 HSI Design.

As described in the FRA/FA IP, Section 4.1, Methodology Structure and Documentation, [

] Therefore, the results of the FRA and FA will identify inputs to the APR1400 HSI Design.

14 The APR1400 Basic HSI conceptual design is referred to as the APR1400 Basic HSI in the Basic HSI TeR.

70

Task analysis (TA): DCD Tier 2, Section 18.7.2.1, HSI Design Input, describes how the results of the TA are used to develop the APR1400 HSI Design. TA encompasses tasks that are necessary to control the plant for the full range of operating conditions, from normal through accident conditions, during normal and degraded HSI conditions. This includes the tasks necessary to execute the important human actions (IHAs). TA generates detailed information and control requirements (ICRs) (e.g., requirements for display range, precision, accuracy, units of measurement) that are implemented during HD in the APR1400 HSI. TA also generates task support requirements (e.g., special lighting, ventilation requirements) that are incorporated into the APR1400 HSI Design. Therefore, the results of the TA will identify inputs to the APR1400 HSI Design.

Staffing and Qualifications (S&Q): DCD Tier 2, Table 18.5-1, Staffing and Qualification Assumptions for the APR1400 MCR, lists the initial staffing level and qualifications assumed for the control room operations staff. The Basic HSI TeR, Figure 3-1, Main Control Room, shows that the initial staffing level is incorporated into the control room layout for the APR1400 Basic HSI conceptual design. The S&Q IP, Section 1, Purpose, states the final staffing number and qualifications will be determined by the COL applicant using the process described in the S&Q IP. The results of the S&Q analysis will either confirm the initial staffing level and qualifications are sufficient or identify modifications to the number and qualifications of the APR1400 operations staff. If the S&Q analysis identifies a modification to the initial staffing level or qualifications are necessary, then the issue will be tracked as a human engineering discrepancy (HED). The S&Q IP, Section 4.3.3, Resolution of S&Q-Related HEDs, describes that HEDs resulting from the S&Q analysis may result in changes to the APR1400 HSI Design. Therefore, the S&Q analysis is an input to the APR1400 HSI Design.

Therefore, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 2 System Requirements - The applicant should identify any constraints on the HSI design imposed by the overall I&C system (e.g., constraints on the information that can be presented due to sensor data availability).

The Staffs Evaluation of Criterion 2 DCD Tier 2, Section 18.7.2.1, HSI Design Input, states I&C requirements and constraints imposed by the I&C system are significant inputs for the HSI design. The applicant provided additional detail in the HD IP, Section 3.1, APR1400 Basic HSI, which states:

[

71

]

Thus, the APR1400 Basic HSI conceptual design described in the Basic HSI TeR encompasses I&C system constraints the applicant identified during the development of the APR1400 Basic HSI conceptual design. Because the APR1400 Basic HSI conceptual design is the starting point for the development of the APR1400 HSI Design that will be developed in accordance with the HD IP, the APR1400 HSI Design also encompasses the I&C system constraints that were identified during the development of the APR1400 Basic HSI conceptual design.

Therefore, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 3 Regulatory Requirements - The applicant should identify the applicable regulatory requirements as inputs to the HSI design process.

The Staffs Evaluation of Criterion 3 The HD IP, Section 4.1.4, Basic HSI Conformance with Regulatory Guidance, identifies the regulatory requirements that are inputs to the HSI design process. The staff compared this list to the list of regulatory requirements in the NUREG-0800, Chapter 18,Section II.A.7, Human-System Interface, and concluded that the applicant has identified the applicable regulatory requirements and also applicable regulatory guidance as inputs to the HSI design process.

Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 4 Other Requirements - The applicant should identify any other requirements, such as customer requirements, that are inputs to the HSI design.

The Staffs Evaluation of Criterion 4 The HD IP, Section 4.1.7, Basic HSI Tests and Evaluations, identifies a requirement imposed by the applicant for a COL applicant to conduct a test of the APR1400 Basic HSI conceptual design during the HSI design process using operators licensed in the US and a simulator. [

] Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

72

18.7.4.2 Concept of Use and HSI Design Overview NUREG-0711, Section 8.4.2, Concept of Use and HSI Design Overview, includes two criteria for this topic.

Criterion 1 The applicant should develop a concept of use stating the roles and responsibilities of operations personnel based upon anticipated staffing levels. The concept of use should:

provide a high-level description of how personnel will work with HSI resources, and address the coordination of personnel activities, such as interactions with auxiliary operators and the coordination of maintenance and operations.

The Staffs Evaluation of Criterion 1 The Basic HSI TeR, Section 3.5.1 describes the roles and responsibilities of operations personnel based on an anticipated control room staffing level of five licensed operators (Section 18.6, Staffing & Qualifications, of this report provides a detailed assessment of the control room staffing) and provides an overview of how control room personnel will work with the HSI resources. The main HSI available in the MCR include the operator consoles (OCs), the safety console (SC), and the large display panel (LDP). These are described below.

The [

73

]

Accordingly, information about plant status is readily available and accessible to each control room operator at his or her workstation.

The SC in the MCR is available to allow the operators [

]

The LDP is [

]

The MCR is also equipped with communication systems to allow for coordination of plant activities with personnel outside the MCR. The MCR includes a [

]

The staff concludes that the applicant has provided a description of the operator roles and responsibilities and how the HSIs support these roles and responsibilities. Also, the staff concludes that the applicant has addressed how plant activities will be coordinated and how the HSIs facilitate this coordination. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should provide an overview of the HSI, covering the technical bases demonstrating that they constitute a state-of-the-art HSI design supporting personnel performance. These bases may include analyses of operating experience and the literature, tradeoff studies simultaneously considering multiple alternatives, and engineering tests and evaluations. The overview should include a description of:

facility layouts, including workstations, large screen displays, and the nominal staff working positions; 74

key HSI resources and their functionality, such as alarms, displays, controls, CBPs, and other support and job aids; technologies to support teamwork and communication within the MCR and between the MCR, the remote shutdown facility, the TSC, EOF, and LCSs; and the responsibilities of the crew for monitoring, interacting, and overriding automatic systems and for interacting with computerized procedures systems and other computerized operator support systems.

The Staffs Evaluation of Criterion 2 The Basic HSI TeR provides an overview of the APR1400 Basic HSI conceptual design, and it includes the following topics.

The facility layouts and technologies to support teamwork and communication:

This includes viewing angles, physical configuration, and presentation formats.

The concept includes the LDP, OCs, and SC. The Basic HSI TeR, Section 3.5.4, Crew Coordination, describes the features of the design that the applicant selected to support teamwork and coordination among personnel from within and outside of the MCR. For example, within the MCR, [

]. Outside of the MCR, the SPDS provides the same information to staff in the RSR, the TSC and EOF, which means that emergency response personnel will be able to monitor changes in plant status and alert the rest of the members of the emergency response organization, including the operators in the control room.

Additionally, the Basic HSI TeR, Section 3.5.4.2, Communication Systems, states that [

]

Therefore, the overview of the HSI includes a description of the facility layouts and the technologies that support teamwork and communication.

The key HSI resources, including the alarms, displays, controls, and CBPs, and their functionality; and the responsibilities of the crew for interacting with automatic systems: The Basic HSI TeR, by virtue of the design descriptions, reinforces that the primary role of the operator is to monitor automatic actuations to ensure that they function properly and to take manual action as directed by procedures if a malfunction occurs. Alarm logic, LDP indications, and the CBPs 75

support the operator in this role. The Basic HSI TeR, Section 4.11.4, Overriding Automatic ESF Actuation, and Section 4.11.5, Interacting with Other Automation, provide detail on how manual control is established, automatic actions are initiated, and automatic actions are reset. The CBPs assist the operator in determining when to take these actions. Therefore, the overview of the HSI includes a description of the key HSI resources and the responsibilities of the crew for interacting with automatic systems.

The HFE guidelines and standards that will be applied to the APR1400 HSI Design: The Basic HSI TeR, Section 3.6, Basic HSI Style Guide, states that the Style Guide applies to all HSI and facilities in the scope of the HSI design process, and DCD Tier 2, Section 18.7.2.5, HSI Detailed Design and Integration, states that the Style Guide includes the HFE guidelines contained in NUREG-0700. NUREG-0700 contains acceptable industry standards for HFE design. The staffs evaluation of how the Style Guide conforms to the HFE guidelines in NUREG-0700 are discussed in Section 18.7.4.4 of this report. The conformance of the APR1400 HSI Design to the HFE standards and guidelines in the Style Guide will be verified during verification and validation (V&V).

Therefore, the overview of the HSI includes a description of the HFE guidelines that apply to the design.

The Basic HSI TeR, Section 5, Development Process, describes the technical bases of the APR1400 Basic HSI conceptual design:

[

]

76

APR1400 Basic HSI conceptual design is the starting point and basis for the APR1400 HSI Design. The HD IP describes the process that will be used to develop the APR1400 HSI Design from the APR1400 Basic HSI conceptual design. The HD IP, Section 4.1.7 describes that this process includes [

. The Basic HSI TeR, Section 5, Development Process, states:

[

]

Thus, the staff concludes that the applicant has provided an overview of the HSI and described the technical bases. The overview includes a description of the facility layouts, key HSI resources, technologies to support teamwork and communication, and responsibilities of the operators for interacting with automatic systems. The staff also concludes that the APR1400 Basic HSI conceptual design is an acceptable starting point for a state-of- the-art HFE design for two reasons. First, the APR1400 Basic HSI concept evolved from a predecessor design previously approved by the staff, and it also incorporated operating experience from other designs as well as the results of HSI tests and evaluations. Second, the applicant developed a Style Guide applicable to the APR1400 Basic HSI conceptual design and the APR1400 HSI Design based on NUREG-0700, which contains acceptable industry standards for HFE design.

The staff also finds that the HSI evaluations that will be performed in accordance with the HD IP provide additional assurance that US operating practices will be incorporated in the APR1400 HSI Design and provide an opportunity to implement technology improvements. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

18.7.4.3 HFE Design Guidance for HSIs NUREG-0711, Section 8.4.3, HFE Design Guidance for HSIs, includes five criteria for this topic.

Criterion 1 The topics in the applicants style guide(s) should address the scope of HSIs included in the design, and address their form, function, and operation, as well as the environmental conditions in which they will be used that are relevant to human performance.

The Staffs Evaluation of Criterion 1 The staff reviewed the Style Guide and found that it addresses the scope of the HSIs included in the design in Section 1.1, Scope (i.e., HSIs in the MCR, RSR, TSC, EOF, and the LCSs that are used for performing IHAs). The Style Guide contains generic HFE guidance derived from NUREG-0700 as well as design-specific HFE standards and guidance that address the form, function, and operation of the HSI resources. The Style Guide addresses the environmental conditions for MCR HSIs, but not for other HSIs that are listed in the Style Guide, Section 1.1, 77

Scope. Therefore, on January 22, 2016, the staff issued RAI 374-8481, Question 18-96 to address this issue (ML16022A233).

On April 16, 2016, the applicant provided a response to RAI 374-8481, Question 18-96 (ADAMS Accession No.ML16107A016). In the response, the applicant stated that the Style Guide would be revised to specify that the same guidance for environmental conditions for the MCR will also apply to the RSR. This is acceptable because the RSR is provided as an alternative to the MCR when the MCR is uninhabitable. Also, the Style Guide will be revised to include the guidance from NUREG-0700 for the environmental conditions for the LCSs.

The applicant also stated that the environmental conditions in the Style Guide for the MCR are not applicable to the TSC and EOF. This is acceptable because, as discussed in the staffs evaluation of Criterion 4 in Section 18.1.4.1, General HFE Program Goals and Scope, of this document, the EOF and TSC will be designed in accordance with NUREG-0696, which is an acceptable alternative to NUREG-0700 for emergency response facilities. The staff confirmed that the Style Guide was revised as committed in the response to RAI 374-8481, Question 18-

96. Therefore, RAI 374-8481, Question 18-96 was resolved and closed.

The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 2 The guidance in the applicants style guide(s) should be developed from generic HFE guidance and HSI design-related analyses. It should be tailored to reflect the applicants design decisions in addressing specific goals of the HSI design.

The Staffs Evaluation of Criterion 2 The Basic HSI TeR, Section 2 states that the APR1400 Basic HSI conceptual design includes the Style Guide, which defines the design standards for the APR1400 Basic HSI. The staff reviewed the Style Guide and determined that the information contained most of the HFE guidance identified in NUREG-0700, which contains generic HFE guidance. Some criteria were not addressed, and it was not clear to the staff why they were omitted. Therefore, on December 22, 2015, the staff issued RAI 353-8372, Questions 18-83 through 18-88 to address this issue (ML15356A486).

On March 21, 2016, the applicant provided a response to RAI 353-8372, Questions 18-85 and 18-88, (ML16081A345). On April 12, 2016, the applicant provided a response to RAI 353-8372, Question 18-86 (ML16133A562). On April 22, 2016, the applicant provided a response to RAI 353-8372, Questions 18-83, 18-84, and 18-87 (ML16113A407). Collectively, the applicants responses state the omitted criteria were either not applicable to the design or were overlooked in the Style Guide. The applicant provided revisions to the Style Guide to include the criteria that are applicable to the design. The staff finds the response acceptable as it provides for a complete set of applicable HFE design criteria consistent with NUREG-0700 guidance. With respect to the Style Guide additions, change pages were provided with the RAI response that acceptably address the omissions. The staff confirmed that the Style Guide and Basic HSI TeR were revised as committed in the response to RAI 353-8372, Questions 18-83 through 18-88.

Therefore, RAI 353-8372, Questions 18-83 through 18-88 were resolved and closed.

The Style Guide also includes four appendices that show how the generic HFE guidelines in the Style Guide have been tailored to the design of the APR1400 Basic HSI conceptual design. For example, Appendix D, Labeling Guidelines, contains the design-specific labeling guidelines for 78

the APR1400 Basic HSI conceptual design that are based on generic labeling guidelines in the Style Guide, Section 7.5, Labeling and Marking.

Accordingly, the staff concludes that the Style Guide incorporates generic HFE guidance in NUREG-0700, and the generic guidance has been tailored to reflect the applicants design decisions (i.e., the APR1400 Basic HSI conceptual design). The staff finds the application conforms to this criterion.

Criterion 3 The individual guidelines in the applicants style guide(s) should be expressed precisely and describe easily observable HSI characteristics, such as Priority 1 alarms are shown in red.

The guidelines in the style guide(s) should be sufficiently detailed so that design personnel can deliver a consistent, verifiable design meeting the applicant's guidelines.

The Staffs Evaluation of Criterion 3 The staff reviewed the Style Guide and found that it contains individual guidelines that are written precisely and describe easily observable HSI characteristics. For example, the Style Guide, Appendix C, The Standard Symbols for Video Display Unit, and Appendix D, Labeling Guidelines, contain detailed drawings that demonstrate how plant components and instruments will be depicted on displays. Accordingly, the staff concludes that the guidelines in the Style Guide are sufficiently detailed so that design personnel can deliver a consistent, verifiable design meeting the applicant's guidelines. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 4 The applicants style guide(s) should contain procedures for determining where and how HFE guidance will be used in the overall design process. They should be written so designers can readily understand them; the text should be supplemented with graphical examples, figures, and tables to facilitate comprehension.

The Staffs Evaluation of Criterion 4 The Style Guide, Section 1.1, Scope, states that the Style Guide contains the design guidance for the APR1400 HSIS. The HD IP, Section 2.2, APR1400 HSIS, lists the HSI resources that are included in the APR1400 HSIS. This section states that the HSI design method described in the HD IP ensures that the HSI resources incorporate the HSI inventory identified in the task analysis process and from plant system design documentation in accordance with the guidance in the Style Guide. The HD IP, Sections 4.2.1, Critical Safety Function Displays, to 4.2.9, Local Control Stations and Facilities, identify the method used to create the detailed design documentation. The method includes verification that the HSI design products (e.g., a drawing of a system display for the reactor coolant system) will conform to the guidance in the Style Guide as well as to the guidance in the nomenclature and labeling guide and the component control and instrumentation design guide.

Additionally, the Style Guide, Appendix C, The Standard Symbols for Video Display Unit, and Appendix D, Labeling Guidelines, contain detailed drawings with graphical examples and figures. Guidance in the main body of the Style Guide directs the user to refer to the specific guidance contained in these appendices.

79

Because the design process instructs design personnel on where and how the HFE guidance in the style guides will be used, and because the Style Guide contains detailed figures and examples, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 5 The applicant should maintain the style guide(s) in a form that is readily accessible and usable by designers, and is easily modified and updated as the design matures. The guidance should include a reference(s) to the source upon which it is based.

The Staffs Evaluation of Criterion 5 The Style Guide, Section 8, References, lists the sources upon which it is based. This includes NUREG-0700. The HFE PP, Section 4.4.4 states that all HFE design program documents, including the Style Guide, will be retained in the Review and Comments System for the life of the program. The Review and Comments System allows access to all HFE documents, including the Style Guide, to personnel involved in the HFE design program.

Human engineering discrepancies (HEDs) are used to document and resolve issues identified during the development of the HFE design. HEDs are tracked using the ITS. The HFE PP, Section 4.6 states that [

] The staff concludes that as the design matures, design personnel can use HEDs to document any changes that need to be incorporated into the HFE documents, including the Style Guide.

Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

18.7.4.4 HSI Detailed Design and Integration The criteria in this section are divided into several subsections.

General NUREG-0711, Section 8.4.4.1, General, includes eight criteria for this topic. The eighth criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first seven criteria as discussed below Criterion 1 For IHAs (see Element 7), the applicants design should minimize the probability that errors will occur, and maximize the probability that any error made will be detected.

The Staffs Evaluation of Criterion 1 The TIHA IP, Section 3.5.4, Human-System Interface Design, describes three ways that the design minimizes the probability that errors will occur when personnel perform IHAs: the use of

[

] These are described in detail below.

The Basic HSI TeR, Section 4.7.1.2, Task Displays, describes task displays as follows:

80

[

]

The Basic HSI TeR, Section 3.5.2, Situation Awareness, states that important alarms and indications will be displayed with SDCV HSI features. SDCV HSI features are dedicated, fixed displays on the LPD and the SC in the MCR and on the LPD in the TSC. SDCV HSI features display [

] The use of dedicated screens on the LDP to alert personnel of plant conditions that require IHAs to stabilize the plant promotes situational awareness of the plant condition among all of the control room operators. Ensuring that the prompting alarms for IHAs are visible to all operators at all times maximizes the probability that the operator(s) will see the alarms and use appropriate alarm response procedures to begin taking actions to stabilize the plant.

The HD IP, Section 3.2.9, Local Control Stations, states that [

] Some IHAs may need to be performed from LCSs outside of the MCR. The HFE standards and conventions identified in the Style Guide, the Nomenclature and Labeling Guide, and the Component Control and Instrumentation Design Guide are applied to the design of the MCR as well as to the design of LCSs that will be used by operators to perform IHAs to maintain consistency in the HFE design of the HSI in these locations where operators may have to perform IHAs.

Therefore, the applicant described features of the HSI design that will minimize the probability that errors will occur and maximize the probability that any error made will be detected when operators perform tasks that are IHAs. Therefore, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should base the layout of HSIs within consoles, panels, and workstations on: (1) analyses of personnel roles (job analysis), and (2) systematic strategies for organization, such as arrangement by importance, and frequency and sequence of use.

The Staffs Evaluation of Criterion 2 The Basic HSI TeR, Section 4.7.1, Contents and Organization, describes how the indications and controls are organized within the IFPD information display hierarchy. Operators normally access the displays at the IFPDs that form the OCs, and the displays that can be viewed on the 81

IFPDs can also be displayed on the variable portion of the LDP. The IFPD display inventory includes [

]

Thus, the applicants organization of indications and controls on the displays is based on systematic strategies for organization.

Additionally, the Basic HSI TeR, Section 3.1.2, Operator Console, describes how the [

] Thus, each operator may arrange the displays at his or her workstation based on the importance of the HSI to the task(s) that must be performed and also on the frequency that he or she needs to use the HSI.

The Basic HSI TeR, Section 3.1.1, Large Display Panel, lists the indications on the SDCV portion of the LDP. The SDCV indications include the key parameters that allow operators to

[

] The variable portion of the LDP allows the operators to select the displays that the crew determines are important for facilitating crew coordination when performing different tasks. Thus, the applicant has arranged the LDP into a fixed portion that contains the indications that are most important to assessing plant safety and plant power production and a variable portion that the operators can customize to support performance of a given task.

The Basic HSI TeR, Section 4.3, Safety Console, describes the basis for the controls and displays allocated to the SC. The SC controls and displays include those HSIs that allow operations to continue, including plant shutdown and accident mitigation, under degraded HSI 82

conditions that result in a loss of all of the OCs. Therefore, the applicant based the layout of HSIs on the SC to ensure that plant operations can continue during degraded HSI conditions.

Thus, the applicant based the layout of HSIs within consoles, panels, and workstations on personnel roles. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 3 The applicant should design the HSIs to support inspection, maintenance, test, and repair of:

(1) plant equipment, and (2) the HSIs. The applicant should design the latter so that inspection, maintenance, test, and repair of the HSIs do not interfere with other plant-control activities (e.g.,

maintenance tags should not block the operators views of plant indications).

The Staffs Evaluation of Criterion 3 The Style Guide, Appendix C, Display Symbol and Legend General Notes - SHT 1, shows an example of the indication that will be appear on the IFPDs to inform operators about plant equipment that is unavailable for maintenance, testing, or repair. The staff concluded that the method of indication is located next to the applicable plant equipment in a way that does not obscure view of other plant indications. Additionally, the APR1400 Basic HSI TeR, Section 4.3.3, Operator Modules, describes HSIs in the MCR that are provided to allow for the conduct of maintenance and testing on the systems that provide signals to safety-related equipment to operate when needed under abnormal and emergency conditions. Also, the TA IP, Section 4.1, states that tasks that are important to plant safety that are undertaken during maintenance, tests, inspections, and surveillances are included in the TA process. The TA results, in addition to the piping and instrumentation diagrams (P&IDs) for the plant systems, which show instrumentation that is used for testing and inspection, are inputs to the HSI design process. Accordingly, the staff finds that the HSI design has considered the inspection, maintenance and repair, and testing requirements for the plant equipment in a way that does not interfere with plant-control activities.

The HD IP, Section 4.1.3.1, Failure of Individual HSI Components, describes how the redundancy incorporated into the MCR design helps to ensure that the repair of HSIs will not interfere with plant control. There are multiple, identical OCs available in the MCR that can be used to operate plant equipment, and there are redundant control methods. Therefore, having redundant displays and controls available ensures that if one method of display or control fails, then the operators can use another method in the control room Therefore, the staff concludes that the maintenance and repair of the HSIs will not interfere with plant control activities. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 4 The applicants design should support personnel task performance under conditions of minimum-, typical-, and high-level or maximum staffing.

83

The Staffs Evaluation of Criterion 4 NUREG-0711 provides additional information for this criterion as follows:

typical staffing is that used for routine plant operations, minimum staffing is that defined by the plant technical specifications, and maximum staffing includes augmented staff for accident situations (e.g., plant personnel who staff the EOF and TSC).

As discussed in more detail in the staffs evaluation of Criterion 2 in Section 18.5.4, Technical Evaluation, of this document, DCD Tier 2, Section 18.1.1.1, Assumptions and Constraints Identification, identifies the APR1400 initial staffing level for all conditions and operating modes as follows:

A fundamental assumption of the APR1400 HFE design is that it is possible to operate the plant during postulated plant operating modes (modes 1 through 6) for normal, abnormal, and emergency conditions, with the following personnel in the main control room (MCR): one reactor operator (RO) with a reactor operator license, one turbine operator (TO) with a reactor operator license, one electric operator (EO) with a reactor operator license, one shift supervisor (SS) with a senior reactor operator (SRO) license, and one shift technical advisor (STA) with a senior reactor operator license.

Thus, two senior operators and three operators comprise the expected or typical number of personnel assigned tasks to perform in the control room. The HD IP, Section 4.1.2, HSI Concept of Use, describes that the MCR design provides [

]

Therefore, the design supports task performance for the typical staffing level because each operator has his or her own workstation, which can be used to perform any task he or she may be assigned to perform.

DCD Tier 2, Chapter 16, Technical Specifications, includes Technical Specification 5.2.2(b),

which states that the number of operators onsite may be less than the minimum requirement of 10 CFR 50.54(m)(2)(i) for a period of time not to exceed 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> in order to accommodate unexpected absence of on-duty shift crew members provided immediate action is taken to restore the shift crew composition to within the minimum requirements (i.e., two senior operators and two operators onsite). Also, HFE PP, Section 4.1, Assumptions and Constraints, identifies the minimum control room staffing requirement in 10 CFR 50.54(m)(2)(iii) as an additional APR1400 staffing constraint (i.e., one senior operator in the control room and either one other senior operator or an operator at the controls when the plant is in an operational mode other than cold shutdown or refueling as defined by the plant technical specifications). Further, as discussed in additional detail in the staffs evaluation of Criterion 3 in Section 18.10.4.3, Integrated System Validation, of this document, the concept of operations described in the Basic HSI TeR, Section 3.5, MCR Concept of Operations, describes administrative controls that help to ensure the typical staffing level is maintained while the plant is operating. When the plant is operating, there are more tasks to perform compared to when the plant is not operating, and therefore keeping the staffing at the typical level helps to ensure that the workload is not excessive for the control room personnel. If only the minimum number of operators is on shift when the plant is operating, it is possible that the workload for the staff in the control room could be higher than when the control room is staffed with the typical number of operators. As discussed in additional detail in the staffs evaluation of Criterion 3 in Section 18.10.4.3, Integrated System Validation, of this document, the integrated system validation (ISV) 84

confirms that the workload is acceptable under a wide variety of plant conditions for both the minimum and typical staffing level. If the workload is not acceptable, or if the design does not support task performance, then HEDs will be documented and resolved in accordance with the HED resolution process. It may be necessary to change an aspect of the HSI design to resolve such HEDs. Therefore, the performance of the ISV and resolution of such HEDs in accordance with the V&V IP provides assurance that the design supports task performance during conditions of minimum and typical staffing.

For maximum staffing, the HD IP, Section 4.1.4.18, Technical Support Center (TSC) and Section 4.1.4.19, Emergency Offsite Facility, describe the facilities that will be available to allow emergency response personnel to perform their tasks when required. Because these facilities are external to the MCR and will have voice communications with the operators in the MCR, the staff finds that these aspects of the design account for maximum staffing levels while still enabling the control room team to have the space and resources they will need to operate the plant. Accordingly, the staff finds that the application conforms to this criterion.

Criterion 5 The applicants design process should account for using the HSIs over the duration of a shift where decrements in human performance due to fatigue may be a concern.

The Staffs Evaluation of Criterion 5 Decrements in human performance due to fatigue are a concern in nuclear power plants because the operators typically work 8-12 hours shifts, and the operators typically rotate between day and night shifts. Fatigue can result in a reduction in an individuals situational awareness, which can make an operator less effective at monitoring indications of plant status.

To compensate for this, DCD Tier 2, Section 18.7.2.5, HSI Detailed Design and Integration the HSI, describes that the LDP includes alarms that indicate the performance of the critical functions as well as plant system-level alarms and component-level alarms of high priority to ensure the operators are informed when there are changes in the plant status that require attention. As described in the Style Guide, Section 5.3, Alarm System, alarms provide visual and audible indication to the operators when there is a condition that requires operator action(s).

By providing audible and visual indications in the MCR, the alarm system helps to ensure that operators maintain awareness of abnormal conditions. Accordingly, the staff finds that the application conforms to this criterion.

Criterion 6 The characteristics of the applicants HSIs should support human performance under the full range of environmental conditions, ranging from normal to credible extreme conditions, such as loss of lighting and of ventilation. For the RSF and LCSs, the applicants HFE design should consider the ambient environment (e.g., noise, temperature, contamination) and the need for and type of protective clothing.

The Staffs Evaluation of Criterion 6 NUREG-0700, Section 12, Workplace Design, states, environmental factors that can have important effects on operators' performance include thermal comfort, illumination, the auditory environment, and facility layout. The staffs evaluation of how the applicant addressed facility layout is included in Criterion 2 in this section of this document. The Basic HSI TeR, Section 4.1.3.1, Humidity, Temperature, and Ventilation, Section 4.1.3.2, Illumination, 85

Section 4.1.3.3, Auditory Environment, and the Style Guide, Section 6.2, Environment Design, identify design-specific guidelines that conform to the guidelines in NUREG-0700 that help to ensure that the MCR design supports human performance under the full range of environmental conditions by ensuring that adequate lighting and ventilation are available during normal and credible extreme conditions, such as loss of normal lighting and normal ventilation.

Additionally, the Style Guide, Section 6.2, Environment Design, includes guidance for the ambient environment of the RSR. However, guidance for the ambient environment for LCSs was missing in the Style Guide. Therefore, on January 22, 2016, the staff issued RAI 374-8481 Question 18-96 to address this issue (ML16022A233). On April 16, 2016, the applicant provided a response to RAI 374-8481, Question 18-96 (ML16107A016). The applicant stated that it will revise the Style Guide to specify the guidance for the ambient environment at LCSs.

Specifically, the revision to the Style Guide will include the guidance for environmental conditions at LCSs contained in Section 12.2, Local Control Stations of NUREG-0700. Thus, the applicant considered the ambient environment for the LCSs and the RSR. The staff confirmed that the Style Guide was revised as committed in the response to RAI 374-8481, Question 18-96. Therefore, RAI 374-8481, Question 18-96 was resolved and closed.

As discussed in the TA IP, Section 4.2.1, Task Narrative, the TA identifies [

] Thus, the applicant provided guidelines in the Style Guide that will be implemented to support human performance under a range of environmental conditions and task support needs. Therefore, the staff finds that the application conforms to this criterion.

Criterion 7 The applicant should identify how in an operating plant:

the HSIs are modified and updated; temporary HSI changes are made, such as modifying the set points; and personnel-defined HSIs are created, such as temporary displays that personnel define for monitoring a specific situation.

The Staffs Evaluation of Criterion 7 The staff did not find in the application how the applicant addressed Criterion 7. Therefore, on January 22, 2016, the staff issued RAI 374-8481 Question 18-101 to address the issue (ML16022A233).

On May 20, 2016, the applicant provided a response to RAI 374-8481, Question 18-101 (ML16142A038). In the response, the applicant identified how, in an operating APR1400 plant, modifications to the HSIs, will be [

] This is consistent with other operating reactors in the United States.

In the RAI response, the applicant also described how temporary changes can be made to HSIs when the plant is operating. The applicant stated that operators can [

86

]

The applicant stated in the RAI response that it will revise: (1) the HFE PP to clarify that the HD IP covers HSI design changes for operating APR1400 plants, (2) the HD IP to clarify the distinction between features of the HSI design that permit plant operators to create new HSI inventory (i.e., custom displays and alarms), without changing existing HSI inventory, and actual changes to existing HSI inventory, and (3) the Basic HSI TeR to describe the custom display and alarm features. Thus, the applicants RAI response describes the process that governs changes to HSIs, how temporary changes to the HSIs can be made, and how personnel-defined HSIs can be created. On July 23, 2018, the applicant provided a revised response to RAI 374-8481, Question 18-101 (ML18204A406). In the revised response the applicant removed a statement in the HFE PP that contradicted some of the information provided in the initial RAI response. Based on the review of the HFE PP, HD IP, and Basic HSI TeR, the staff has confirmed incorporation of the changes described above; therefore RAI 374-8481, Question 18-101 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Main Control Room NUREG-0711, Section 8.4.4.2, Main Control Room, includes 15 criteria for this topic.

Criterion 1 Safety Parameter Display System - The applicant should describe the safety parameter display system (SPDS), addressing the following:

Identification of Critical Safety Functions (CSFs) - The CSFs needed to meet the requirement for an SPDS should be identified. NUREG-1342 Section III.F, Minimum Parameters for Display, lists the five CSFs that personnel monitor using an SPDS for boiling water reactor (BWRs) and pressurized water reactor (PWRs). For new designs, applicants should verify that these CSFs are suitable for their design, identifying any changes needed based on their designs detailed characteristics. CSFs may differ for non-light water reactor designs, such as high-temperature gas-cooled reactors and liquid-metal reactors.

Identification of the Parameters Personnel will use to Monitor Each CSF - The applicant should identify the plant parameters personnel need to monitor each CSF and describe the means by which plant data are synthesized, combined, or otherwise evaluated to provide the information presented in the SPDS display.

Section III.F of NUREG-1342 has guidance on acceptable parameters for the 87

current fleet of PWRs and BWRs. The applicants identification of parameters should consider the unique characteristics of the plants design.

Evaluation of SPDS HSIs - The applicant should verify that the SPDS HSIs conform to acceptable HFE practices using NUREG-0700, Section 5 and other SPDS HFE guidance.

The Staffs Evaluation of Criterion 1 The applicant described the SPDS as follows.

Identification of Critical Safety Functions (CSFs): The FRA/FA IP, Section 3.2.1, Identification of Critical Safety Functions, states:

[

]

The applicant also identified the APR1400 CSFs listed in the FRA/FA IP, Table 3-1, in DCD Tier 2, Section 7.7.1.4 (d)(1), SPADES+. The staff compared the System 80+

CSFs in NUREG-1462, Final Safety Evaluation Report Related to the Certification of the System 80+ Design, Section 7.7.1.21, Data Processing System, which identifies eight CSFs, including the five listed in NUREG-1342,Section III.F. The staff observed that the APR1400 CSFs are the same as the CSFs for the System 80+ design, with one exception. The exception is that the System 80+ design includes a CSF for containment environment, and the APR1400 design has split this CSF into two individual CSFs: one CSF for containment temperature and pressure control and a separate CSF for containment combustible gas control (i.e., radioactive emissions control). This is appropriate because as described in NUREG-1462, Section 1.6, Index of Applicable Regulations and Exemptions, the System 80+ design used RG 1.97, Instrumentation for Light-Water-Cooled Nuclear Power Plants to Assess Plant and Environs Conditions During and Following an Accident, to select the variables operators need to monitor to maintain the CSFs. Containment pressure is used to monitor the containment environment. Separating the CSF for containment environment into the two CSFs in the APR1400 specifies that containment pressure as well as containment combustible gas and temperature will be used to determine whether the environment in containment is acceptable to ensure its integrity.

Therefore, the applicant identified the APR1400 CSFs that are the starting point for the FRA to be performed by the COL applicant in accordance with the FRA/FA IP; however, the applicant expects these CSFs will be confirmed because no changes are expected to be made to the CSFs during the FRA. The applicant also identified changes from the 88

System 80+ that are applicable to the APR1400 design. Accordingly, the staff finds that the application conforms to the first bullet of this criterion.

Identification of the Parameters Personnel will use to Monitor Each CSF: The Basic HSI TeR, Section 3.5.2, Situation Awareness, states that the [

] IEEE Standard 497-2002, IEEE Standard Criteria for Accident Monitoring Instrumentation for Nuclear Power Generating Stations, which is endorsed by RG 1.97, Criteria for Accident Monitoring Instrumentation for Nuclear Power Plants, Revision 4, defines Type B variables as those variables that provide primary information to the control room operators to assess the critical safety functions. IEEE Standard 497-2002 also describes a method for identifying Type B variables. In DCD Tier 2, Section 7.5.1.1, Accident Monitoring Instrumentation, the applicant stated that its accident monitoring instrumentation is designed to meet the guidance of RG 1.97, Revision 4. In DCD Tier 2, Table 7.5-1, Accident Monitoring Instrumentation Variables, the applicant identified its Type B variables. The staffs evaluation of the applicants selection of Type B variables in accordance with RG 1.97 is in Chapter 7.5 of this document. Thus, the parameters used to monitor each CSF are, at a minimum, the Type B variables in DCD Tier 2, Table 7.5-1.

The Basic HSI TeR, Section 4.9, Safety Parameter Display System, describes the SPADES+ application program, which synthesizes and evaluates plant data (i.e.,

parameters) that operators need to monitor the CSFs and sends this information to the CSF displays that are viewable in the MCR, RSR, EOF, and TSC. The SPADES+

application program is executed by the non-safety related Information Processing System (IPS), which is described in detail in DCD Tier 2, Section 7.7.1.4, Information Processing System. Also, DCD Tier 2, Section 7.5.1.1, Accident Monitoring Instrumentation, explains that the safety related Qualified Indication and Alarm System-P (QIAS-P) receives Type B variables from the plant protection system (PPS),

engineered safety features - component control system (ESF-CCS), and the core protection calculator system (CPCS) via a safety system data network (SDN) and auxiliary process cabinet - safety (APC-S) and process instrumentation via a hardwired connection. The staffs evaluation of the IPS and the QIAS-P is in Chapter 7 of this document. Because the parameters have been identified and the applicant has described the means by which plant data are synthesized and evaluated, the staff finds that the application conforms to the second bullet of this criterion.

Evaluation of SPDS HSIs: The Basic HSI TeR, Section 4.9 and the HD IP, Section 3.2.1, Critical Safety Function Displays, describe the SPDS HSIs provided by the IPS, [

]

89

The staffs evaluation of the how the SPDS HSIs conform to the HFE guidance listed in NUREG-0700, Section 5, Safety Function and Parameter Monitoring, follows15.

Criterion 5.1-1: Plant parameters and variables important to safety should be displayed in a way that is convenient and readily accessible. [

] Accordingly, the staff finds that plant parameters important to safety are displayed in a way that is convenient and readily accessible.

Criterion 5.1-2: Critical safety function displays should be readable from the workstations of users needing access to these displays. [

] Accordingly, the staff finds that the CSF displays are readable from the workstations of the users needing access to these displays.

Criterion 5.1-3: Critical plant variables and parameters should be displayed to help users evaluate the plant's safety status. The HD IP, Section 3.2.1 state that the [

] Accordingly, the staff finds that the critical plant parameters that help users evaluate safety status are displayed.

Criterion 5.1-4: The display system should display information about severe accident symptoms associated with the plant safety parameters and functions.

The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the SPDS displays. One of these inputs is DCD Tier 2, Section 7, Instrumentation and Control, which provides the design basis for the SPDS HSIs. DCD Tier 2, Section 7.5.1.2, Inadequate Core Cooling Monitoring Information, lists the plant parameters that provide unambiguous indication of inadequate core cooling (ICC) and warning of the approach of ICC. These parameters are referred to as ICC variables. The SPADES+ program displays information about each of the ICC variables on the 15 NUREG-0711 provides additional information for this criterion, which states, SPDS requirements are described in 10 CFR 50.34(f)(2)(iv), and related guidance in NUREG-0835, NUREG-1342, Supplement 1 of NUREG-0737, and NUREG-0700, Section 5. These NUREGs discuss the NRCs review guidance for SPDS, with NUREG-0700 being the primary one; the others encompass supplemental guidance, examples, and technical bases. Therefore, the staff evaluated the application using the SPDS guidance in NUREG-0700, Section 5, and referred to the other guidance documents if necessary to clarify the guidance in NUREG-0700, Section 5.

90

CSF displays in the MCR and the TSC. Data generated by SPADES+ is also transmitted to the EOF. Because the SPADES+ program displays information about the ICC variables, which provide indication of severe accident conditions, the staff finds that the display system displays information about severe accident symptoms.

Criterion 5.1-5: Critical plant variables should be displayed in a concise format.

The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the CSF displays. [

] Accordingly, the staff finds that the critical plant variables (i.e., the CSFs), are displayed in a concise format.

Criterion 5.1-6: The display's response to transient and accident sequences should keep the user informed of the current plant status. The SPADES+

program monitors plant variables indicative of CSF status and uses algorithms to determine the status of each CSF. [

Accordingly, the staff finds that the SPDS HSIs keep users informed of the current plant status.

Criterion 5.1-7: Critical safety function displays should allow users to comprehend a change in safety status in a matter of seconds. The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the SPDS displays. [

] However, it was not clear to the staff how this will be accomplished. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-116 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-116 (ML16133A607). In the response, the applicant stated that the [

91

] The staff confirmed that the Style Guide was revised as committed in the response to RAI 400-8425, Question 18-116. Therefore, RAI 400-8425, Question 18-116 was resolved and closed. Accordingly, the staff finds that the SPDS HSIs keep users informed of the current plant status.

Criterion 5.1-8: The sampling rate for each critical plant variable should be consistent with the users' needs for performing tasks. The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the SPDS displays.

However, it was not clear to the staff how this will be accomplished. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-116 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-116 (ML16133A607). [

] The staff confirmed that the Style Guide was revised as committed in the response to RAI 400-8425, Question 18-116. Therefore, RAI 400-8425, Question 18-116 was resolved and closed. Accordingly, the staff finds that the sampling rate is consistent with users needs for performing tasks.

Criterion 5.1-9: Each critical variable should be displayed with sufficient accuracy for the user to discriminate between normal conditions and those affecting plant safety status. The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the SPDS displays. [

] However, it was not clear to the staff how this will be accomplished.

Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-116 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-116 (ML16133A607). In the response, the applicant stated that [

] The Style Guide will be revised to include this guidance. The staff finds it acceptable that the applicant provided a design-specific guideline for accuracy that will be included in the design that is tested during V&V. Whether the accuracy is sufficient for user to discriminate between normal conditions and abnormal conditions will be determined in the ISV. The staffs review of the ISV is discussed in Section 18.10.4.3 of this document. The staff confirmed that the Style Guide was revised as committed in the response to RAI 400-8425, Question 18-116. Therefore, RAI 400-8425, Question 18-116 was resolved and closed.

92

Criterion 5.1-10: The display should provide magnitudes and trends for critical plant variables or derived variables. The HD IP, Section 4.2.1 describes the design basis (i.e., design characteristics) for the CSF displays that will be used to create the detailed design documents that will be used to build the SPDS displays. [

] Accordingly, the staff finds that the CSF displays provide trends for CSF variables.

Criterion 5.1-11: Displays for monitoring safety parameters and functions should continuously display this information. The Basic HSI TeR, Figure 4-21, CFM/BISI Section of the LDP, shows the portion of the [

] Accordingly, the staff finds that the CFM/BISI provide trends for CSF variables.

Criterion 5.1-12: Where plant operating modes impose different demands, separate display pages should be provided for each mode. The HD IP, Section 4.2.1 describes the design requirements that must be included in the detailed design documentation for the SPDS displays.

DCD Tier 2, Chapter 16, Technical Specifications, Table 1.1-1, Modes, shows there are six modes for the APR1400.

Accordingly, the staff finds that separate display pages will be provided for each of the different modes of operation.

Criterion 5.2-1: The system should assist the user in monitoring critical parameters, especially parameters that change very rapidly or very slowly, by alerting the user when values are out of range. DCD Tier 2, Section 7.7.1.4(d)(1), SPADES+, states that the SPADES+ application program monitors the status of the CSFs during normal, abnormal, and emergency operating conditions and provides alarms when any of the CSFs are not being maintained within range. Accordingly, the staff finds that the SPDS program alerts users when values are out of range.

Criterion 5.2-2: Where feasible, the system should provide perceptual (audible or visual) cues to alert personnel to abnormal operation conditions that potentially warrant corrective action. [

]

Accordingly, the staff finds that the SPDS will provide perceptual cues to alert personnel to abnormal conditions that warrant corrective actions.

Criterion 5.2-3: While viewing secondary (lower-level) displays, a perceptual (audible or visual) cue should be provided by the safety parameter or function monitoring system to alert the user to return to the primary (higher level) display format if significant information in that display requires user attention. The 93

SPADES+ program [

]

Accordingly, the staff finds that cues are available to alert the user to changes that require attention even when the operators is viewing lower level displays.

Criterion 5.2-4: User interactions with the display system should be within the skill capability of the control room crew and should not significantly increase personnel workload. [

] This is well within the skill capability of the control room crew. Accordingly, the staff finds that the user interactions with the display system are within the skill capability of the crew and will not increase workload.

Criterion 5.3-1: The display should not give false indications of plant status. The processing of SPDS information is performed by the SPADES+ application program. The Information Processing System (IPS) executes all application programs, including SPADES+. DCD Tier 2, Section 7.7.1.4 describes the characteristics of the IPS. The IPS application functions are executed via redundant IPS servers. One server is the primary (active) unit and the other is a dedicated backup. If the primary IPS server experiences a failure, its dedicated backup server assumes all processing tasks of the failed unit.

Additionally, the status of the CSFs will normally be viewed on the LDP and on the OCs; however, if the OCs or the LDP are not available, then the status of the CSFs can be viewed on the SC, which is the backup to these normal displays.

Furthermore, as discussed below under Criterion 5.3-2, the IPS provides operators with indication of the quality of the data it is processing to indicate the reliability of the information. Accordingly, the staff finds that because there are redundant processors and redundant displays, and because the IPS provides indication of data quality that can be used to determine the reliability of the data, the SPDS HSIs provide sufficiently reliable indications of plant status if there are failures of a single processor or display or failures that affect data processing.

Criterion 5.3-2: Critical plant variables should be reliable and should be validated in real time. The staff could not find information about validation of critical parameters. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-117 to address this issue (ML16035A020). On May 13, 2016, the applicant provided a response to RAI 400-8425, Question 18-117 (ML16134A573). The applicant stated data validation of the critical plant variables is performed by the IPS, which is described in DCD Tier 2, Section 7.7.1.4. DCD Tier 2, Section 7.7.1.4 states, The IPS makes the information available to the plant operating staff both on a real-time and historical basis. Additionally, the applicant provided additional information about how the IPS monitors the quality of data provided to operators and how the quality of that data is communicated. The staff confirmed that DCD Tier 2 was revised as 94

committed in the response to RAI 400-8425, Question 18-117. Therefore, RAI 400-8425, Question 18-117 was resolved and closed.

Criterion 5.3-3: The status of the data should be displayed to the operator with an appropriate data quality indicator. The staff could not find information about how data quality is indicated. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-117 to address this issue (ML16035A020). On May 13, 2016, the applicant provided a response to RAI 400-8425, Question 18-117 (ML16134A573). The applicant described how indications of data quality (i.e., good, fair, poor, and bad) will be displayed to the operator. The staff confirmed that DCD Tier 2 was revised as committed in the response to RAI 400-8425, Question 18-117. Therefore, RAI 400-8425, Question 18-117 was resolved and closed.

Criterion 5.4-1: The location of displays for monitoring safety parameters and functions should not interfere with the normal movement of the control room crew. [

] the staff finds that the location of the SPDS displays will not interfere with normal movement of the crew.

Criterion 5.4-2: The display system should not interfere with visual access to other control room operating systems or with displays that are important to safe operation of the plant. [

] the staff finds that the display system does not interfere with visual access to other important displays.

Criterion 5.4-3: Display devices for monitoring safety parameters and functions should be labeled and readily distinguished from other devices. The Basic TeR, Figure 4-21, CFM/BISI Section of the LDP, [

] Accordingly, the staff finds that the SPDS displays are readily distinguished from other displays and devices.

The applicant has described the SPDS, including the SPADES+ application program, which processes plant data and sends information about the status of CSFs to displays, [

]

The applicant has identified a minimum set of CSFs and parameters used to indicate the status of the CSFs, and the operators will be able to monitor trends of these parameters.

] Accordingly, the staff finds the SPDS HSI resources conform to the SPDS guidance in NUREG-0711 and NUREG-0700, and the application complies with 10 CFR 50.34(f)(2)(iv).

95

Criterion 2 Bypassed and Inoperable Status Indication - The applicant should describe how the HSI assures the automatic indication of the bypassed and inoperable status of a safety function, and the systems actuated or controlled by the safety function. [10 CFR 50.34(f)(2)(v) - I.D.3]

RG 1.47 includes the following guidance related to the display of bypassed and inoperable status of safety systems:

The status indication should be in the MCR.

Administrative procedures should be supplemented by an automatic indication system that shows, for each affected safety system or subsystem, the bypass or deliberately induced inoperability of a safety function, and the systems it actuates or controls.

Provisions should be made allowing the operations staff to confirm that a bypassed safety function was properly returned to service.

Annunciating functions for system failure and automatic actions based on the self-test or self-diagnostic capabilities of digital computer-based I&C safety systems should be consistent with the above bullets.

The indication system for bypass and inoperable status should include the ability to ensure its operable status during normal plant operation to the extent to which the indicating and annunciating functions can be verified.

Bypass and inoperable status indicators should be arranged such that personnel can determine whether it is permissible to continue operating the reactor.

The control room of all affected units should receive an indication of the bypass for their shared system safety functions.

The Staffs Evaluation of Criterion 2 The applicant described the Bypassed and Inoperable Status Indication (BISI) as follows.

The HD IP, Section 4.1.4.2, Bypassed and Inoperable Status Indication, states that the critical function monitoring/bypassed and inoperable status indication (CFM/BISI) displays on the SDCV portion of the LDP fulfill the requirements for the BISI. As shown in the Basic HSI TeR, Figure 4-21, CFM/BISI Section of LDP, [

]

the staff finds that the application conforms to the first bullet.

DCD Tier 2, Section 7.5.1.3, Bypassed and Inoperable Status Indication, describes how the BISI provides automatic indication of bypassed or inoperable safety systems:

96

The bypassed or inoperable condition of ESF [engineered safety features]

components is communicated to the IPS [information processing system], which indicates a system-level bypassed or inoperable conditionSystem-level alarms are actuated when a component actuated by a protection system is bypassed or deliberately rendered inoperable.

Because the status of safety components is displayed automatically at the system-level on the CFM/BISI display, the staff finds the application conforms to the second bullet.

The staff could not find a description of the provisions for allowing the operations staff to confirm that a bypassed safety function was properly returned to service.

Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-118 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-118 (ML16133A607). [

] The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 400-8425, Question 18-118. Therefore, RAI 400-8425, Question 18-118 was resolved and closed.

DCD Tier 2, Section 7.5.1.3 describes that BISI is an indication system that provides indications and alarms. It does not perform automatic actions other than providing indications and alarms. The Basic HSI TeR, Section 4.7.2, Display Page/Information Access, describes that [

] This feature allows operators to confirm the display is updating and operating. There are no self-test or automatic actions associated with the BISI. Therefore, this criterion is not applicable.

The staff could not find a description of how the operators can verify the indication and annunciating functions of the BISI. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-118 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-118 (ML16133A607). [

]

The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 400-8425, Question 18-118. Therefore, RAI 400-8425, Question 18-118 was resolved and closed.

The staff could not find how the application addresses the following: bypass and inoperable status indicators should be arranged such that personnel can determine whether it is permissible to continue operating the reactor. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-118 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-118 (ML16133A607). [

97

] The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 400-8425, Question 18-118.

Therefore, RAI 400-8425, Question 18-118 was resolved and closed.

The staff could not find how the application addresses the following: the control room of all affected units should receive an indication of the bypass for their shared system safety functions. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-118 to address this issue (ML16035A020).

On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-118 (ML16133A607).

The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 400-8425, Question 18-118. Therefore, RAI 400-8425, Question 18-118 was resolved and closed.

The staff finds that the application describes how the HSI assures automatic indication of the bypassed and inoperable status of a safety function and the systems associated with the safety function. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(v).

Criterion 3 Relief and Safety Valve Position Monitoring - The applicant should describe how the HSI indicates the position of the relief and safety valves (open or closed) in the control room

[10 CFR 50.34(f)(2)(xi) - II.D.3]

The Staffs Evaluation of Criterion 3 NUREG-0737,Section II.D.3, Direct Indication of Relief and Safety Valve Indication, clarifies that this requirement applies to safety and relief valves in the RCS. DCD Tier 2, Section 5.4.14, Safety and Relief Valves, states that pilot-operated safety relief valves provide overpressure protection for the RCS. It also states that these valves are part of an emergency success path for one of the APR1400 CSFs. The HD IP, Section 4.1.4.3, Relief and Safety Valve Position Monitoring, states that [

] Thus, the position of the relief and safety valves will be indicated in the LDP in the MCR. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xi).

Criterion 4 Manual Feedwater Control - The applicant should describe how the HSI provides automatic and manual initiation of the auxiliary feedwater system, and indicates auxiliary feedwater system flow in the control room. [Applicable to PWRs only, 10 CFR 50.34(f)(2)(xii) - II.E.1.2]

The Staffs Evaluation of Criterion 4 The HD IP, Section 4.1.4.4, Manual Feedwater Control, states that [

98

] Thus, the position of the relief and safety valves will be indicated in the LDP in the MCR. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xii).

Criterion 5 Containment Monitoring - The applicant should describe how the control rooms HSIs (alarms and displays) inform personnel about: (A) containment pressure; (B) containment water level; (C) containment hydrogen concentration; (D) containment radiation intensity (high level); and (E) noble gas effluents for all potential, accident release points. [10 CFR 50.34(f)(2)(xvii) -

II.F.1]

The Staffs Evaluation of Criterion 5 The HD IP, Section 4.1.4.5, Containment Monitoring, states that [

] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xvii).

Criterion 6 Core Cooling - The applicant should describe how the HSI provides unambiguous indication of inadequate core cooling, such as with primary coolant saturation meters in PWRs, and a suitable combination of signals from indicators of coolant level in the reactor vessel and in-core thermocouples in PWRs and BWRs. [10 CFR 50.34(f)(2)(xviii) - II.F.2]

The Staffs Evaluation of Criterion 6 As described in DCD Tier 2, Section 4.4.6.3, Other Monitoring Systems, the following instruments are used to monitor inadequate core cooling (ICC): hot and cold leg reactor coolant system resistance temperature detectors, pressurizer pressure instruments, and core-exit thermocouples (CETs) for indication of primary coolant saturation margin and heated junction thermocouples (HJTCs) for indication of reactor vessel water level. The HD IP, Section 4.1.4.6, Core Cooling, states that [

].

DCD Tier 2, Section 7.5.1.2, Backup ICC Displays, states that the QIAS-P flat panel displays (PFPDs) on the SC provide Class 1E backup indication in case the primary display is unavailable. The PFPDs provide unambiguous indication of ICC by displaying saturation 99

margin, reactor vessel level, and core exit temperature. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xviii).

Criterion 7 Post-accident Monitoring - The applicant should describe how the HSI assures monitoring of plant and environmental conditions following an accident including core damage. [10 CFR 50.34(f)(2)(xix) - II.F.3, and RG1.97]

The Staffs Evaluation of Criterion 7 DCD Tier 2, Table 7.5-1, Accident Monitoring Instrumentation Variables, identifies the variables, by type, that allow plant personnel to monitor the plant and environmental conditions following design basis events16.

The HD IP, Section 4.1.4.7, Post-Accident Monitoring Instrumentation, describes how these variables will be displayed to plant personnel. It states that [

] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xix).

Criterion 8 Auxiliary Heat Removal -- The applicant should describe how that necessary automatic and manual actions can be taken to ensure proper functioning of auxiliary heat removal systems when the main feedwater system is not operable. [Applicable to BWRs only, 10 CFR 50.34(f)(2)(xxi) - II.K.1.22]

The Staffs Evaluation of Criterion 8 The APR1400 is a PWR, not a BWR. Therefore, this criterion is not applicable.

Criterion 9 Reactor Level Monitoring - The applicant should describe how the HSI gives a record of the reactor vessels water level in one location on displays that meet normal post-accident recording requirements. [Applicable to BWRs only, 10 CFR 50.34(f)(2)(xxiv) - II.K.3.23]

The Staffs Evaluation of Criterion 9 The APR1400 is a PWR, not a BWR. Therefore, this criterion is not applicable.

16 Refer to Section 7.5 of this SER for the staffs evaluation of the applicants method of identifying Type A-E variables.

100

Criteria 10 Leakage Control - The applicant should describe how the HSI provides for leakage control and detection in the design of systems outside containment that contain (or might contain) accident-source-term radioactive materials after an accident. [10 CFR 50.34(f)(2)(xxvi) - III.D.1.1]

The Staffs Evaluation of Criterion 10 The HD IP, Section 4.1.4.10, Leakage Control, states that [

DCD Tier 2, Section 5.2.5.4, Intersystem Leakage, lists several systems connected to the reactor coolant system that could potentially contain radioactivity following an accident. This section of the DCD specifically identifies alarms and indications the operators will need to detect and control leakage outside containment. However, the HD IP did not identify this information as an input to the HSI design process. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-122 to address this issue (ML16035A020). On May 23, 2016, the applicant provided a response to RAI 400-8425, Question 18-122 (ML16144A662). The applicant stated that the Therefore, the staff concludes that the applicant has described how the HSIs provide for leakage control and detection in systems outside containment. The staff confirmed that the HD IP Sections 3.2.2, 3.6.2, and 4.2.2, were revised as committed in the response to RAI 400-8425, Question 18-122.

Therefore, RAI 400-8425, Question 18-122 was resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xxvi).

Criterion 11 Radiation Monitoring - The applicant should describe how the HSI provides appropriate monitoring of in-plant radiation and airborne radioactivity under a broad range of routine and accident conditions. [10 CFR 50.34(f)(2)(xxvii) - III.D.3.3]

The Staffs Evaluation of Criterion 11 The HD IP, Section 4.1.4.11, Radiation Monitoring, states that [

]

Additionally, DCD Tier 2, Table 7.5-1, Accident Monitoring Instrumentation, identifies some radiation monitors as Type E variables. DCD Tier 2, Section 7.5.2.1, Accident Monitoring Instrumentation, also states that all of the AMI variables, including Type E, will be displayed by the IPS, which provides indication on the IFPDs at the OCs, and by the QIAS-N at the SC.

The HD IP states that other radiation indications and alarms are provided on the [

] The staff did not find direction in the HD IP for the SMEs to include these indications as inputs to any of the HSIs. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-124 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-124 (ML16133A607). In the 101

response, the applicant clarified [

] DCD Tier 2, Table 11.5-1, Gaseous Process and Effluent Radiation Monitors, lists APR1400 radiation monitors. These radiation monitors are part of the APR1400 plant systems, and Table 11.5-1 shows that alarms will be provided in the MCR for some radiation monitors listed in Table 11.5-1. These radiation monitors that will have alarms include alarms for radiation monitors that can be used for monitoring in-plant radiation and airborne radioactivity under a broad range of routine and accident conditions. The staff confirmed that the HD IP was revised as committed in response to RAI 400-8425, Question 18-124. Therefore, RAI 400-8425, Question 18-124 was resolved and closed. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xxvii).

Criterion 12 Manual Initiation of Protective Actions - The applicant should describe how the HSI supports the manual initiation of protective actions at the system level for safety systems otherwise initiated automatically. [RG 1.62]

The Staffs Evaluation of Criterion 12 The HD IP, Section 4.1.4.12, Manual Initiation of Protective Actions, states that [

] the staff has reasonable assurance that the process described in the HD IP will result in a design that satisfies this requirement. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 13 Diversity and Defense-in-depth - The applicant should describe how the HSI provides displays and controls in the MCR for manual, system-level actuation of critical safety functions, and for monitoring those parameters that support them. These displays and controls are independent of, and different from, the normal I&C. [I&C BTP7-19, Point 4]

The Staffs Evaluation of Criterion 13 The HD IP, Section 4.1.4.13, Diversity and Defense in Depth, states that the [

]

DCD Tier 2, Table 7.8-3, Diverse Actuation Signals, and Table 7.8-4, Display and Control Parameters for the DIS, list the diverse controls that provide for system-level actuation of 102

equipment needed to manage the CSFs and indications that provide for monitoring of the CSFs.

DCD Tier 2, Section 7.8.2.2, Diverse Manual Engineered Safety Features Actuation Switches, and Section 7.8.2.3, Diverse Indication System, describe how the DMA switches and DIS are independent and different from the normal I&C. The staffs evaluation of the independence and diversity of this system is documented in Section 7 of this SER.

Because the HD IP, Section 4.2.7 states that the [

], the staff has reasonable assurance that the process described in the HD IP will result in a design that conforms to this criterion.

Accordingly, the applicant has described how the HSI provides controls in the MCR for manual, system-level actuation of the equipment required to manage the CSFs, and displays for monitoring the CSFs that are independent and diverse from the normal I&C. Therefore, the staff finds the application conforms to this criterion.

Criterion 14 Important HAs - The applicant should describe how the HSI provides the controls, displays, and alarms that ensure the reliable performance of identified IHAs. Section 7 of NUREG-0711 discusses important HAs.

The Staffs Evaluation of Criterion 14 IHAs are identified using the process described in the TIHA IP. This process results in a list of IHAs and the controls, displays (indications), and alarms that the operators will need to perform the IHAs reliably. This information is derived from DCD Tier 2, Chapters 7, 15, and 19. The list of IHAs and the controls, indications, and alarms will be documented and is an input to the task analysis process to develop the HSI inventory for tasks associated with IHAs. The HSI inventory developed during the task analysis is an input to the HSI design process. Using the HSI inventory, the HD IP identifies inputs to the design of the alarm system, task displays, controls in the MCR and at LCSs, and the CBPs, as follows:

[

]

Reliable performance must be demonstrated for IHAs during the ISV conducted in accordance with the V&V IP. As described in the V&V IP, if IHAs are not performed successfully, then 103

HEDs will be documented and resolved in accordance with the HFE PP. This may result in design changes to the HSI to facilitate reliable performance of the IHAs. Accordingly, the applicant has described how the HSI provides the controls, displays, and alarms that ensure the operators can perform IHAs. Therefore, the staff finds the application conforms to this criterion.

Criterion 15 Computer-Based procedure platform - The applicants computer-based procedures should be consistent with the design review guidance in NUREG-0700, Section 8, Computer-Based Procedure System, and in Section 1 of DI&C-ISG-5 (NRC, 2008).

The Staffs Evaluation of Criterion 15 The HD IP, Section 4.2.6, Computer-Based Procedures, states that the design of the APR1400 computer-based procedure system will conform to design standards in the Style Guide. The Style Guide, Section 5.2, Computer-Based Procedures, lists design standards for CBP systems that are consistent with the guidance in NUREG-0700, Section 8, with two exceptions. The Style Guide, Items 5.2.1.2(e) and (f), state that words from standard Korean should be used, and punctuation should conform to standard Korean usage. Therefore, on February 1, 2016, the staff issued RAI 383-8458, Question 18-104 to address this issue (ML16032A104). On March 25, 2016, the applicant provided a response to RAI 383-8458 and stated that the application will be revised to change the standard language from Korean to English. The staff confirmed that the Style Guide was revised as committed in response to RAI 383-8485, Question 18-104. Therefore, RAI 383-8485, Question 18-104 was resolved and closed.

The staff reviewed the design of the computer-based procedure system using the review guidance in DI&C-ISG-5, Section 1. The Basic HSI TeR, Section 4.8, Computer-based Procedures, and the HD IP, Section 4.2.6 describe how the design of the CBPs are consistent with the guidance in DI&C-ISG-5, Section 1. The staff found that the design of the CBPs is consistent with the guidance in DI&C-ISG-5, Section 1, except for the following review criteria:

1. 3, #4, #7, #15, #16, #17, #19, #22, and #24. Therefore, on February 1, 2016, the staff issued RAI 383-8458, Question 18-105 to address this issue (ML16032A104). On May 20, 2016, the applicant provided a response to RAI 383-8458, Question 18-105 (ML16142A045). The applicant described how the remaining criteria were addressed and stated that the Basic HSI TeR, Section 4.8 would be revised to include additional information to address the remaining criteria. The staff found that the additional information addressed the remaining criteria with the exception of #22. Therefore, on August 24, 2016, the staff issued RAI 518-8654, Question 18-128 to address the issue (ML16237A410). On September 19, 2016, the applicant provided a response to RAI 518-8654, Question 18-128 (ML16263A430), and described why the CBP system is not able to change the approved procedure. Thus, the CBP will not be able to make changes to procedures that have been approved, which is consistent with Criterion #22 in DI&C-ISG-5, Section 1. RAI 518-8654, Question 18-128 was resolved and closed. The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 383-8485, Question 18-105. Therefore, RAI 383-8485, Question 18-105 was resolved and closed.

The staff concludes that the applicants CBP system conforms to the guidance in NUREG-0700, Section 8 and DI&C-ISG-5, Section 1. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

104

Technical Support Center NUREG-0711, Section 8.4.4.3, Technical Support Center, includes seven criteria for this topic.

Criterion 1 The applicant should describe how the HSIs give personnel the information needed to:

analyze the plants steady-state and dynamic behavior before and throughout an accident so TSC personnel can guide the MCR operators in managing the abnormal conditions and mitigating the accident without interfering with the MCR activities; undertake the needed environmental- and radiological-monitoring functions of the EOF when it is not operational; offer technical support to personnel during recovery operations after an emergency; and provide reliable voice-communications facilities to the control room, the operations support center, the EOF, the NRC, and with state and local operations centers.

The Staffs Evaluation of Criterion 1 The HD IP, Section 4.1.4.18, Technical Support Center, and the Basic HSI TeR, Section 3.3, Technical Support Center, state that the [

] The staff concludes that because the information available to control room personnel and TSC personnel will be the same, and because the design of the displays will be the same in both locations and therefore information will be presented identically in both locations, personnel in the TSC will have the information required to analyze plant behavior, perform the functions of the EOF if necessary, and offer technical support to the operators. However, the HD IP did not address how the HSIs provide reliable voice communications. Therefore, on January 22, 2016, the staff issued RAI 373-8480 Question 18-93 to address this issue (ML16022A232). On April 5, 2016, the applicant provided a response to RAI 373-8480 Question 18-93 (ML16096A193). The applicant described the voice communications available in the TSC, which includes [

], and stated that it would revise the HD IP to include the information. The staff confirmed that the HD IP was revised as committed in response to RAI 373-8480, Question 18-93. Therefore, RAI 373-8480, Question 18-93 was resolved and closed. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should describe how the HSIs give personnel the information needed for:

determining the plants steady-state operating conditions before the accident, 105

ascertaining the transient conditions producing the initiating event, gauging plant systems dynamic behavior throughout the accident, reviewing the accident sequence, deciding upon appropriate mitigating actions, evaluating the extent of any damage, and assessing the plants status during recovery operations.

The Staffs Evaluation of Criterion 2 As discussed in the staffs evaluation of Criterion 1 above, [

] By having this information available, the TSC personnel can determine status before the event, ascertain transient conditions producing the event, gauge dynamic behavior during the event, review of the accident sequence by evaluating how plant parameters have changed since the accident began, and assess status during recovery.

The Basic HSI TeR, Section 3.3, Technical Support Center, also explains that the TSC includes [

] Thus, TSC personnel can use the indications of plant status and the procedures to decide the mitigating actions the MCR operators should perform. Also, phones can be used to communicate with emergency response personnel who perform tasks in the plant and can provide updates on the extent of damage in the plant. Thus, having access to the full inventory of HSIs that the MCR staff has and also providing a means of communicating with personnel outside of the TSC ensures that that emergency response personnel in the TSC have the information needed to perform the functions described in this criterion. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 3 The applicant should describe how the HSIs provide an SPDS that replicates the SPDS in the MCR (to improve the exchange of information between personnel in the MCR and the EOF). If the SPDS in the MCR is composed of multiple displays, then multiple displays also should be provided in the TSC.

The Staffs Evaluation of Criterion 3 The HD IP, Section 4.1.4.18, Technical Support Center, states that [

] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 4 The applicant should describe how the HSIs provide as a minimum, the set of variables specified in RG 1.97, Revision 4, plus all sensor data and calculated variables not specified in 106

RG 1.97 but included in the data sets for the SPDS, for the EOF, or for transmission to offsite locations.

The Staffs Evaluation of Criterion 4 As discussed in the staffs evaluation of Criterion 1 above, TSC personnel will have access to the [

] DCD Tier 2, Section 7.7.5.1, Accident Monitoring Instrumentation, states that the IPS provides displays for all AMI variables, which are the variables specified in RG 1.97. Also, the HD IP, Section 4.1.4.18, Technical Support Center, state that the [

] See the staffs evaluation of Criterion 1 under Main Control Room in this section of this document for more detail on the AMI and SPDS. Because the TSC personnel have access to the same information as the control room personnel and the EOF personnel, the staff concludes that the TSC HSI provide the set of variables specified in RG 1.97, Revision 4, and the SPDS. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 5 The applicant should describe how the HSIs allow all TSC personnel to complete its assigned tasks with unhindered access to alphanumeric and/or graphical representations of:

plant systems variables, in-plant radiological variables, meteorological information, and offsite radiological information.

The Staffs Evaluation of Criterion 5 The HD IP, Section 4.1.4.18, Technical Support Center, states that the types of information listed in this criterion are included in the HSI inventory available to personnel in the TSC. The HD IP, Section 4.2.8, Central Facilities, lists the method used to design the TSC and states that [

] Because the workplace design in NUREG-0700 applies specifically to either the MCR or LCSs, the staff requested the applicant clarify whether these standards would also be applied to the workplace design of the TSC.

Therefore, on January 22, 2016, the staff issued RAI 373-8480, Question 18-94 to address this issue (ML16022A232). On April 22, 2016, the applicant provided a response to RAI 373-8480, Question 18-94 (ML16113A419) and specified the criteria in NUREG-0700 that is applicable to the TSC, including criteria related to access to workstations that contain the information listed in this criterion. The staff confirmed that the HD IP was revised as committed in response to RAI 373-8480, Question 18-94. Therefore, RAI 373-8480, Question 18-94 was resolved and closed.

The staff finds that the application conforms to this NUREG-0711 criterion.

107

Criterion 6 The applicant should describe how the HSIs provide the trend-information displays and time-history displays that give the TSC personnel a dynamic view of the plants status during abnormal operating conditions.

The Staffs Evaluation of Criterion 6 The HD IP, Section 4.1.4.18, Technical Support Center, states that the [

] DCD Tier 2, Section 7.5.1, System Description, also states that the IPS includes historical data storage, retrieval, and trending capability. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 7 The applicant should describe how HFE was incorporated into the TSC design to ensure that personnel easily understand and use the HSIs.

The Staffs Evaluation of Criterion 7 The HD IP, Section 4.1.4.18, Technical Support Facility, states that the TSC will be designed to the same HFE standards as the MCR. Thus, the HFE guidelines in the Style Guide are applied to the HSIs in the TSC. The purpose of a Style Guide based on accepted HFE principles, such as those in NUREG-0700, is to provide design guidelines to ensure that accepted HFE principles are applied to the design of HSIs. Applying accepted HFE principles to the design of HSIs helps to personnel understand and use the HSIs. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Emergency Operations Facility The HFE PP, Section 2 states that a graded HFE approach will be applied to the EOF.

NUREG-0711, Section 2.4.1, "General HFE Program Goals and Scope, allows the use of a graded HFE approach applied to facilities other than the MCR and RSF, providing justification in the HFE program plan. The applicant did not provide justification for using a graded approach to these other facilities. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-5 to address the issue (ML15177A387). On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-5 (ML15254A492). The applicant stated, the guidance provided in NUREG-0696, Functional Criteria for Emergency Response Facilities, and NUREG-0700, Human-System Interface Design Review Guidelines are applied to the design of the TSC and EOF instead of the full-scope described in NUREG-0711, since NUREG-0696 and NUREG-0700 provide guidance specific to the design of emergency response facilities and human system interfaces, respectively.

NUREG-0800, Chapter 18, Draft Revision 3, states: NUREG-0696, Functional Criteria for Emergency Response Facilities, also includes general HFE criteria for these facilities and the staff has accepted a commitment to implement these criteria as an alternative to the NUREG-0711 criteria. This is an acceptable alternative, and RAI 54-7963, Question 18-5 was resolved and closed. However, the application did not include this information. Therefore, on January 22, 2016, the staff issued RAI 373-8480, Question 18-91 to address this issue (ML16022A232).

108

On May 13, 2016, the applicant provided a response to RAI 373-8480, Question 18-91 (ML16134A593). The applicant stated that DCD Tier 2, Section 18.1.1.2, Applicable Plant Facilities and the HD IP, Sections 4.1.4.1 and 4.1.4.19 would be revised to state that the EOF will be designed in accordance with the guidance in NUREG-0696 and NUREG-0700. SRP Chapter 18, Revision 3,Section I.6, states, The emergency operating facility (EOF) and technical support center (TSC) are also included within the scope of HFE reviews. Again, a graded approach is used. NUREG-0711 provides specific direction on which review criteria are applied to these facilities.

NUREG 0696, Functional Criteria for Emergency Response Facilities, also includes general HFE criteria for these facilities and the staff has accepted a commitment to implement these criteria as an alternative to the NUREG-0711 criteria.

Thus, the applicant has revised the application to identify that commitment for the COL applicant to design the EOF in accordance with NUREG-0696. The staff confirmed that DCD Tier 2 and the HD IP were revised as committed in response to RAI 373-8480, Question 18-91. Therefore, RAI 373-8480, Question 18-91 was resolved and closed. Accordingly, the staff finds the application conforms to this NUREG-0711 criteria.

Remote Shutdown Facility NUREG-0711, Section 8.4.4.5, Remote Shutdown Facility, includes two criteria for this topic.

Criterion 1 The applicant should describe how the HSI provides a design capability for remote shutdown of the reactor outside the MCR. [10 CFR Part 50, Appendix A, GDC 19]

The Staffs Evaluation of Criterion 1 In 10 CFR Part 50, Appendix A; GDC 19 states in part, Equipment at appropriate locations outside the control room shall be provided: (1) with a design capability for prompt hot shutdown of the reactor, including necessary instrumentation and controls to maintain the unit in a safe condition during hot shutdown, and (2) with a potential capability for subsequent cold shutdown of the reactor through the use of suitable procedures. DCD Tier 2, Table 7.4-1, Remote Shutdown Console Instrumentation and Controls for Hot Shutdown, and DCD Tier 2, Table 7.4-2, Remote Shutdown Console Instrumentation and Controls for Cold Shutdown, list the necessary indications and controls outside the control room to permit remote shutdown of the reactor outside the MRC. The staffs evaluation of the adequacy of these indications and controls to achieve safe shutdown is in Section 7.4 of this SER. DCD Tier 2, Section 7.4.1.l.3, Emergency shutdown from outside the MCR, describes that these controls and indications are available through the soft controls and displays on the four IFPDs and ESCMs in the Remote Shutdown Console (RSC) in the RSR. The IFPDs are the same as those in the MCR, and therefore the displays and controls on the RSC are the same type as those on the consoles in the MCR. DCD Tier 2, Figure 7.4-4, Layout of Remote Shutdown Room, shows the layout of the RSC in the RSR. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should describe how the HSIs at the RSF are consistent with those in the MCR.

109

The Staffs Evaluation of Criterion 2 The Basic HSI TeR, Section 4.16, Remote Shutdown Room, describes that the RSR will include one RSC [

] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion.

Local Control Stations NUREG-0711, Section 8.4.4.6, Local Control Stations, includes two criteria for this topic.

Criterion 1 The applicant should describe the basis for deciding which HSIs will be included in the MCR design, and which will be provided locally.

The Staffs Evaluation of Criterion 1 As described in the HD IP, Section 4.2, APR1400 HSIS and Facilities, the MCR HSI inventory of indications and displays is provided to the operators by the [

]

As described in Section 4.2, APR1400 HSIS and Facilities, the information and controls (i.e.,

the HSI inventory) included in the MCR design are determined by the following sources: (1) the results of the function requirements analysis, task analysis, and TIHA program elements (e.g.,

the HSI inventory developed as part of the basic task analysis (BTA) for each task and the HSI that are assumed to be available in the transient and accident analyses); (2) regulatory requirements (e.g., the requirement in 10 CFR 50.34(f)(2)(v) for bypassed and inoperable status indication in the MCR); (3) plant design documentation (e.g., the plant system P&IDs are the basis for the system displays); and (4) the inventory of the predecessor plant (e.g., the system displays from the predecessor plant are the starting point for designing the system displays for the APR1400). The staff concludes that these sources provide a comprehensive and thorough basis for the HSI inventory that operators will need to perform their tasks in the MCR.

The HD IP, Section 4.2.9, Local Control Stations and Facilities, describes that the HSI inventory that will be available at LCSs are determined by the following sources: (1) the results of the task analysis and the TIHA program elements, which identify HSI inventory requirements for tasks that require local control, including tasks that are performed as part of IHAs; (2) plant design documentation; and (3) the LCSs of the predecessor plant. The staff concludes that these sources also provide a comprehensive and thorough basis for selecting the HSI inventory that is appropriately assigned to LCSs. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

110

Criterion 2 The applicant should describe how HFE was incorporated into the HSIs for LCSs to ensure they are consistent with those in the MCR, and that personnel easily understand and use the HSIs.

The Staffs Evaluation of Criterion 2 Licensed operators in the MCR do not typically perform actions outside of the MCR at LCSs.

Rather, they typically direct non-licensed operators to perform actions outside of the MCR.

However, it is possible that a licensed operator would perform an IHA at an LCS. The HD IP, Section 3.2.9 states that [

]

Applying the same HFE standards and guidelines used in the design of the HSIs in the MCR to the HSIs for LCSs used to perform IHAs for beyond-design basis events helps to ensure that when these actions are performed by licensed operators, they will be able to use them with ease. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

18.7.4.5 Degraded I&C and HSI Conditions NUREG-0711, Section 8.4.5, Degraded I&C and HSI Conditions, includes four criteria for this topic.

Criterion 1 The applicant should identify:

the effects of automation failures and degraded conditions on personnel and plant performance HFE-significant I&C degradations; i.e., the failure modes and degraded conditions of the I&C system that might adversely affect the HSIs personnel use to accomplish important HAs The Staffs Evaluation of Criterion 1 The applicant identified effects of degraded conditions and automation failures of the non-safety control systems on plant performance in DCD Tier 2, Table 7.7-1, Control Groups for the NSSS Control Functions, and the Control System CCF (common cause failure) Analysis. The applicant identified effects of degraded conditions and automation failures of the safety-related systems in DCD Tier 2, Section 7.2.3.1, Failure Modes and Effects Analysis (for the Plant Protection System), and DCD Tier 2, Section 7.3.3.1, Failure Modes and Effects Analysis (for the Engineered Safety Features Actuation System). Additionally, the applicant identified the effects of a CCF of the digital safety I&C system during the postulated accidents (PAs) and abnormal operating occurrences (AOOs) described in DCD Tier 2, Chapter 15 on plant performance in the CCF Coping Analysis. A CCF of the digital I&C safety system during a PA or AOO is an HFE-significant I&C degradation because operators must use backup and diverse HSI to perform IHAs. The staffs evaluation of these analyses is documented in Chapter 7 of this report.

111

The V&V IP, Section 4.1.1.1, Plant Conditions, lists the types of operational conditions that the ISV will sample, and it includes [

] Additionally, the V&V IP, Section 4.1.1.2, Personnel Tasks, states that all IHAs are included, including those identified in DCD Tier 2, Chapter 7, that must be performed by operators following a CCF of the digital safety I&C system that occurs during a PA or AOO. By including these types of degraded I&C conditions and automation failures in the ISV scenarios and collecting performance measurements described in the V&V IP, Section 4.5.5, Performance Measurement, the applicant will identify its effects on personnel performance. If satisfactory personnel performance, as described in the V&V IP, Section 4.5.7, Data Analysis and HED Identification, cannot be demonstrated during the ISV scenarios, the applicant will document the issue as an HED and resolve the HED in accordance with the HFE PP, Section 4.6.1, Method.

Therefore, the staff finds that the applicant has identified the effects of automation failures and degraded conditions, including HFE-significant I&C degradations, on plant performance. Also, the applicant will identify the effects of automation and degraded I&C and HSI conditions on personnel performance by including a sample of these conditions in the ISV. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should specify the alarms and other information personnel need to detect degraded I&C and HSI conditions in a timely manner, and to identify its extent and significance.

The Staffs Evaluation of Criterion 2 The HD IP, Section 4.2.5, Alarms, describes the method for developing the APR1400 alarm inventory. [

] The applicant described the kinds of alarms that will alert operators following various modes of IFPD failures in its response to RAI 323-8281, Question 07.03-19, dated March 15, 2016 (ML16075A425). In the response, the applicant also described that the IFPDs inform operators of the status of the safety I&C system via system status displays which are top level health displays on the IFPDs. The applicant also stated that it will revise the Safety I&C System report, Section 4.1.2.7, Information Processing System, and Section 4.1.4, Human System Interfaces, to include the information. The staff confirmed that Safety I&C System report, Sections 4.1.2.7 and 4.1.4 were revised as committed in the response to RAI 323-8281, Questions 07.03-19. Therefore, RAI 323-8281, Question 07.03-19 was resolved and closed.

[

]

Finally, the Basic HSI TeR, Section 4.7.2, Display Page/Information Access, describes that

[

] These indications allow the operators to verify that the HSI at their workstations is capable of communicating information. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

112

Criterion 3 The applicant should determine any needed back-up systems to ensure that important personnel tasks can be completed under degraded I&C and HSI conditions.

The Staffs Evaluation of Criterion 3 The applicant identified the following as backup methods available to operators under a range of degraded HSI conditions:

The Basic HSI TeR, Section 4.2, Operator Consoles, describes that if any IFPD at an OC is degraded, then the operator can use [

]

The Basic HSI TeR, Section 4.3.1, Fixed Position Indications and Alarms, states that the [

] The mini-LDP receives information from the non-safety qualified indication and alarm system (QIAS-N), which is a non-safety system that is diverse from the IPS.

As described in the Basic HSI TeR, Section 4.3.2, Fixed Position Controls, minimum inventory controls as well as [

]

As described in DCD Tier 2, Section 7.5.1.2(b), Backup ICC Displays, the post-accident qualified indication and alarm system (QIAS-P) provides backup indication on the SC of indications of inadequate core cooling, which can be used if the IFPDs are not available.

If there is a failure of the computer-based procedure system, then operators can use the [ ] as described in the Basic HSI TeR, Section 4.1.1, Configuration.

In the event of a common-cause failure (CCF) of digital safety I&C systems including the plant protection system (PPS) and engineered safety features-component control system (ESF-CCS), the normal HSI will not be available to the operators to perform the IHAs required to mitigate an AOO or PA. Therefore, the applicant has included a diverse protection system (DPS), which includes a DAS, diverse manual engineered safety features (ESF) actuation (DMA) switches, and a diverse indication system (DIS). The DMA switches and DIS are available at the SC.

113

Incorporating redundancy into the design of the MCR HSI ensures that the operators have reliable sources of information available during degraded HSI conditions. The applicant also identified the following backup systems that operators will be able to use to monitor and control the plant under a range of degraded I&C conditions:

As described in DCD Tier 2, Section 7.7.1.4 the IFPDs on the OCs receive information from the IPS. The IPS is a computer-based system that provides operational means for monitoring and control of the plant. The information is derived from other I&C systems and self-contained algorithms called application programs. The IPS makes the information available to the plant operating staff both on a real-time and historical basis. DCD Tier 2, Section 7.7.1.4 (b), IPS Configuration, states that there are two redundant IPS servers. One server is the primary (active) unit and the other is a dedicated backup. If the primary IPS server experiences a failure, its dedicated backup server assumes all processing tasks of the failed unit.

As described in DCD Tier 2, Section 7.1.1.5, Information Systems Important to Safety, there are two redundant and separate alarm systems. The alarm systems are redundantly implemented by the IPS and QIAS-N. The IPS and QIAS-N are independent and diverse from each other. Therefore, any single alarm system failure will not cause a total loss of the plants alarm system.

Incorporating redundancy into the design of the I&C systems that interface with the HSI resources in the MCR ensures that the operators have reliable sources of information available during degraded I&C conditions. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 4 The applicant should determine the necessary compensatory actions and supporting procedures to ensure that personnel effectively manage degraded I&C and HSI conditions, and the transition to back-up systems.

The Staffs Evaluation of Criterion 4 The TA IP, Section 2 states that [

] The TA IP states that a BTA will be performed for all tasks. The BTA produces a task narrative, which lists the actions operators must perform to accomplish the task. The task narrative is an input that the COL applicant uses to develop operating procedures. The task narrative is also an input to the HSI design process.

Additionally, the applicant identified actions that operators need to take and the diverse HSI resources that will be available during a CCF of the digital I&C safety system that occurs during a PA or AOO, which is an HFE-significant I&C degradation that results in loss of the preferred HSI resources in the MCR, in the CCF Coping Analysis (the staffs evaluation of the applicants CCF Coping Analysis is in Section 7.8 of this report). The actions that operators must take during such an event are IHAs. IHAs and the diverse HSI described in the CCF Coping Analysis are inputs to the HSI design process. The HD IP, Section 3.5.4 states that [

] will be 114

developed to help ensure that personnel effectively manage this type of degraded I&C and HSI condition. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

18.7.4.6 HSI Tests and Evaluations The criteria in this section are divided into two subsections.

Trade-off evaluations are comparisons between design options, based on aspects of human performance that are important to successful task performance, and to other design considerations.

Performance-based tests involve assessing personnel performance, including subjective opinions, to evaluate design options and design acceptability.

Trade-Off Evaluations NUREG-0711, Section 8.4.6.1, Trade-off Evaluations, includes two criteria for this topic.

Criterion 1 In comparing design approaches, the applicant should consider those aspects of human performance important to performing tasks. The applicant should take into account the following factors when developing criteria to apply in selecting one design approach over another:

personnel-task requirements human-performance capabilities and limitations HSI-system performance requirements inspection and testing needs maintenance demands use of proven technology and the operating experience of predecessor designs The Staffs Evaluation of Criterion 1 The Basic HSI TeR, Section 5.0, Development Process, describes the basis for the APR1400 Basic HSI Concept. The APR1400 is an evolutionary plant design, and the Section 5 states,

[

115

]

[

]

Furthermore, the HD IP, Section 4.1.5.1, Design Evolution, describes how [

]

Thus, the staff finds that the technology and operating experience of predecessor designs have been taken into account, as well as HSI-system performance requirements (e.g., those related to seismic durability), and human performance capabilities and limitations.

The HD IP, Section 3.5.3, Task Analysis, describes how personnel task requirements are considered in the design of the HSI resources. The task narratives developed in accordance with the TA IP will be used to establish alarm priorities, alarm logic, displays, and operator aids during the HSI design process. For example, the HD IP, Section 4.2.3 states that [

] Therefore, the staff concludes that the applicants HSI design process incorporates the results of the task analysis (i.e., personnel task requirements) process.

Like the predecessor designs, the APR1400 Basic HSI concept accounts for testing needs and maintenance demands. For example, as described in the Basic HSI TeR, Section 4.3.3, Operator Module, the [

] The HFE guidelines and standards defined in the Basic HSI TeR and the Style Guide apply to the design of the OMs. Additionally, the TA IP, Section 4.1, Task Selection, states that are included in task analysis. The results of the task analysis, specifically the HSI inventory developed for each task that is analyzed, is an input to the HSI design process. Additionally, the applicant will perform task support verification (TSV) as part of V&V. The V&V IP, Section 4.3 states [

] Therefore, the staff finds that the applicants design approach accounts for inspection 116

and testing needs and maintenance demands. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should state explicitly the relative benefits of design alternatives and the basis for the design approach selected.

The Staffs Evaluation of Criterion 2 As discussed in the staffs evaluation of Criterion 1, the HD IP, Section 4.1.5.1, Design Evolution, states that [

] that differ from those found in conventional, analog, predecessor plants. The design features, their relative benefits, and their bases for selection, are described below.

[

]

Control panels in legacy control rooms have a wide variety of switches, pushbuttons, controllers, indications, and displays. These control panels are typically distributed throughout the control room and require operators to walk to and from panels to operate equipment. Therefore, the staff concludes that the IFPDs and soft controls, which are designed in accordance with design-specific style guides and are accessible to the operators directly from the operator consoles where the operators are seated, do increase the standardization of the plant-personnel interface, and provide for easier access to plant controls and indications compared to legacy plants.

117

The applicant identified the relative benefits of the CBP system compared to paper procedures in the Basic HSI TeR, Section 4.8. This section states that

[

] The staff concludes that this reduces mental workload and improves the likelihood that continuous action steps will be performed in a timely manner.

The applicant identified the relative benefits of the digital LDP, which has fixed and selectable displays, as compared to legacy plants in the Basic HSI TeR, Section 4.6, Large Display Panel. [

] The Basic HSI TeR, Section 4.6, Large Display Panel, states,

[

]

Thus, the staff finds that compared to conventional, analog control rooms, where operators must locate individual indications and controls located on multiple panels in the control room to understand the overall plant status, the information presented on the LDP allows plant personnel to more easily locate and integrate information and more readily assess plant status. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Performance-Based Tests NUREG-0711, Section 8.4.6.2, Performance-Based Tests, includes three criteria for this topic.

Criterion 1 The applicant should identify the specific objectives of the tests.

The Staffs Evaluation of Criterion 1 The HD IP, Section 4.1.7 states that the [

118

]

Additionally, the HD IP, Section 3.2.7, Performance-Based Tests, states that performance-based tests will be conducted for

] The applicant identified the following specific objectives of these performance-based tests:

]

Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicant should base the general approach to testing on the tests objective(s). The following aspects of the tests should be described (note that not all items are applicable to every type of test):

participants testbed design features or characteristics of the HSI being tested tasks or scenarios used performance measures test procedures data analyses The Staffs Evaluation of Criterion 2 The HD IP, Section 4.1.7 describes participants, testbed, design features to be tested, and scenarios to be used for the performance-based test of the APR1400 Basic HSI Concept. [

119

]

The HD IP did not describe specific test procedures, performance measures, or data analyses.

This is acceptable for two reasons. First, the objective of this test is to obtain feedback from operators licensed in the U.S. about how aspects of the APR1400 Basic HSI Concept may be improved. The staff finds that the test method described in the HD IP is sufficient for allowing operators licensed in the U.S. to experience interacting with the APR1400 Basic HSI Concept and provide feedback about the experience. Second, the APR1400 HSI Design that results from the HSI design process described in the HD IP will be tested during the ISV using the test method described in the V&V IP. The V&V IP specifies requirements for test procedures, performance measures, and data analyses in addition to requirements for test participants, an adequate testbed, and scenarios. Thus, the staff concludes that the performance-based test of the APR1400 Basic HSI Concept that the applicant will conduct prior to V&V should help to ensure that successful test results can be obtained during the ISV when the final HFE design is tested.

The HD IP, Section 4.2.3, Task Displays, and Section 4.2.4, Application Displays, specify the testbed, participants, and design features to be tested, and performance measures for performance-based tests of the task displays and application displays, respectively. [

]

[

]

The HD IP did not describe specific test procedures, performance measures, or data analyses for these performance-based tests. The staff finds this is acceptable for two reasons. First, the objective of testing the task displays, the application displays, and the alarms and CBPs that are used in conjunction with task displays that operators use to perform IHAs is to confirm that 120

these HSIs are correctly interfaced to the applicable plant systems. For alarms, the tests also confirm that the alarms are correctly prioritized and that the alarm logic is correct. By using a part-task simulator, the SMEs will be able to determine whether the CBPs, alarms, and plant system indications and controls on the task and application displays are integrated prior to the ISV. Thus, the staff finds that these tests will confirm the functionality of these displays prior to the ISV, which should help to ensure that successful test results will be obtained during the ISV.

Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Criterion 3 The conclusions from the tests and their impact on design decisions should be described.

The Staffs Evaluation of Criterion 3 The HD IP, Section 3.1, APR1400 Basic HSI, states that [

]

Additionally, the HD IP, Section 3.2.7, Performance-Based Tests, states that if a

[

]

Accordingly, the staff finds the application conforms to this NUREG-0711 criterion.

Combined License Items There are no COL items associated with Section 18.6 of the APR1400 DCD.

Conclusion The staff evaluated the applicants HSI design method and finds that it conforms to the criteria in NUREG-0711, Section 8.4. Therefore, the staff concludes that the applicants HSI design method is an acceptable process for translating functional and task requirements to HSI design requirements and for applying HFE design principles. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii), 10 CFR 50.34(f)(2)(iv), 10 CFR 50.34(f)(2)(v), 10 CFR 50.34(f)(2)(xi), 10 CFR 50.34(f)(2)(xii), 10 CFR 50.34(f)(2)(xviii), 10 CFR 50.34(f)(2)(xix), 10 CFR 50.34(f)(2)(xxvi), 10 CFR 50.34(f)(2)(xxvii), and 10 CFR 52.47(a)(8) related to this element.

18.8 Procedure Development Procedure development is included in NUREG-0711 because there are HFE attributes associated with the procedures. However, as an operating program, procedure development is reviewed in Section 13.5 of this report. The staffs conclusions are documented in Section 13.5.

121

18.9 Training Program Development The training program development is included in NUREG-0711 because of the interfaces between the HFE design, procedures, and training. However, as an operating program, training program development is reviewed in Section 13.2 of this report. The staffs conclusions are documented in this section.

18.10 Human Factors Verification and Validation Introduction The review of the verification and validation (V&V) element was conducted at the IP level, as described in NUREG-0711, Section 1.2.2, Review Elements. Thus, the review focuses on the methodology the applicant will use to conduct the V&V. The objectives of the review were to verify that:

The applicant identified a sample of operational conditions that: (1) includes conditions representative of the range of events that could be encountered during the plants operation, (2) reflects the characteristics expected to contribute to variations in the systems performance, and (3) considers the safety significance of human system interfaces (HSIs). These sample characteristics are best identified by using a multidimensional sampling strategy to reasonably assure that V&V evaluations include variation along important dimensions. (See Section 18.10.4.1 below for the staffs evaluation).

The applicant's HSI inventory and characterization accurately describes all HSI displays, controls, and related equipment lying within the scope defined by the sampling of operational conditions. (See Section 18.10.4.2 below for the staffs evaluation).

The applicant verified that the HSI provides the needed alarms, information, controls, and task support defined by task analysis for personnel to perform its tasks. (See Section 18.10.4.2 below for the staffs evaluation).

The applicant verified that the design of the HSIs conforms to HFE guidelines (such as the applicants style guide). (See Section 18.10.4.2 below for the staffs evaluation).

The applicant validated, using performance-based tests, that the integrated system design (i.e., hardware, software, procedures, and personnel elements) supports the safe operation of the plant. (See Section 18.10.4.3 below for the staffs evaluation).

The applicant has: (1) evaluated HEDs to determine if they require corrections, (2) identified design solutions to address HEDs that must to be corrected, and (3) verified the completed implementation of these HED design solutions. (See Section 18.10.4.6 below for the staffs evaluation).

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

122

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: Section 18.10 summarizes the methodology described in APR1400-E-I-NR-14008, Human Factors Verification and Validation Implementation Plan, that will be used by the COL applicant to select operational scenarios, complete task support verification and HFE design verification, and conduct the integrated system validation (ISV).

Changes to Tier 2 information are governed by the change control processes in the design certification rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs.

ITAAC: The ITAAC associated with this element are listed in DCD Tier 1, Section 2.9, Table 2.9-1. Section 14.3.9 of this SER contain the staffs evaluation of the HFE ITAAC.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: TeRs associated with this element are:

APR1400-E-I-NR-14001, Human Factors Engineering Program Plan (HFE PP)

(ML18212A336)

APR1400-E-I-NR-14008, Human Factors Verification and Validation Implementation Plan (V&V IP) (ML18178A202)

APR1400-E-I-NR-14010, Human Factors Verification and Validation Scenarios (V&V Scenarios Document) (ML18081A107)

APR1400-E-I-NR-14012, Style Guide, (ML18081A107)

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii)

Other regulatory guidance documents are as follows:

NUREG-0711, Revision 3, Chapter 11, Human Factors Verification and Validation, Section 11.4, Review Criteria NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.10, Verification and Validation 123

Technical Evaluation 18.10.4.1 Sampling of Operational Conditions The criteria in this section are divided into several subsections.

Sampling Dimensions NUREG-0711, Section 11.1, Background, states, Sampling of Operational Conditions to support V&V tests is important because reviews of new plants and significant HSI modifications can involve hundreds or thousands of individual HSIs, and it is impractical and unnecessary to review all of them. Therefore, the applicant can employ a sampling strategy to guide the selection of HSIs to review. NUREG-0711, Section 11.4.1.1, Sampling Dimensions, identifies three criteria for determining the conditions that will be included in the sample of HSIs that will be reviewed during the V&V activities. These criteria are listed below.

Criterion 1 The applicant should include the following plant conditions:

normal operational events including plant startup, shutdown or refueling, and significant changes in operating power I&C and HSI failures and degraded conditions that encompass:

o the I&C system, including the sensor, monitoring, automation and control, and communications subsystems; [e.g., safety-related system logic and control unit, fault tolerant controller, local field unit for multiplexer (MUX) system, MUX controller, and a break in MUX line]

o common cause failure of the I&C system during a design basis accident (as defined by Branch Technical Position BTP 7-19) o HSIs including, loss of processing or display capabilities for alarms, displays, controls, and CBPs transients and accidents, such as:

o transients (e.g., turbine trip, loss of off-site power, station blackout, loss of all feedwater, loss of service water, loss of power to selected buses or MCR power supplies, and safety and relief valve transients) o accidents (e.g., main-steam-line break, positive reactivity addition, control rod insertion at power, anticipated transient without scram, and various-sized loss-of-coolant accidents) o reactor shutdown and cooldown using the remote shutdown system o reasonable, risk-significant, beyond-design-basis events that should be determined from the plant-specific PRA 124

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.1.1, Sampling Dimensions, states that a list of conditions and a checklist is prepared that SMEs use to construct the scenarios. All of the necessary dimensions are included in at least one scenario. Section 4.1.1.1, Plant Conditions, provides the listing of plant conditions that will be included in the scenarios. The staff compared these to the above criteria and found the process and the list acceptable.

Section 5 of the V&V Scenarios Document provides the seven specific scenarios that have been constructed by the applicant. Each scenario is described in summary fashion in Section 5 and in more detail in seven appendices to the report. The V&V Scenarios Document in Section 3, Sampling of Operational Conditions for Integrated System Verification, contains a matrix of plant conditions and scenarios that illustrates which plant conditions are addressed by each scenario. The staff reviewed the scenario descriptions and Table/matrix of Section 3.1 and verified that all required plant conditions were addressed by at least one scenario. The staff finds that the application conforms to this review criterion.

Criterion 2 The applicant should include the following types of personnel tasks:

Important human actions, Systems, and Accident Sequences - The sample should include all important HAs, as determined in Section 7 of NUREG-0711.

Additional factors that contribute highly to risk, as defined by the PRA, also should be sampled:

o dominant accident sequences o dominant systems (selected through PRA importance measures, such as Risk Achievement Worth or Risk Reduction Worth)

Manual Initiation of Protective Actions - The sample should include manual system-level actuation of critical safety functions.

Automatic System Monitoring - The sample should include situations in which humans must monitor a risk-important automatic system.

OER - Identified Problematic Tasks - The sample should include all personnel tasks identified as problematic during the applicant's review of operating experience.

Range of Procedure Guided Tasks -The sample should include tasks that are well defined by procedures. Personnel should be able to understand and execute the specified steps as part of their rule-based decision-making. RG 1.33, Appendix A, contains several categories of typical safety-related activities that should be covered by written procedures. The sample should include appropriate procedures in each category:

o administrative procedures o general plant operating procedures 125

o procedures for startup, operation, and shutdown of safety-related systems o procedures for abnormal, off-normal, and alarm conditions o procedures for combating emergencies and other significant events (e.g.,

reactor accidents, and declaration of emergency-action levels) o procedures for controlling radioactivity o procedures for controlling measuring and test equipment and for surveillance tests, procedures, and calibration o procedures for performing maintenance o chemistry and radiochemical control procedures Range of Knowledge-Based Tasks - The sample should include tasks that are not well defined by detailed procedures.

Range of Human Cognitive Activities - The sample should include the range of cognitive activities that personnel perform, including:

o detecting and monitoring (e.g., of critical safety-function threats) o situation assessment (e.g., interpreting alarms and displays to diagnose faults in plant processes and in automated control and safety systems) o planning responses (e.g., evaluating alternatives to recover from plant failures) o response implementation (e.g., in-the-loop control of plant systems, assuming manual control from automatic control systems, and carrying out complicated control actions) o obtaining feedback (e.g., feedback of the success of actions taken)

Range of Human Interactions - The sample should include the range of interactions among plant personnel, including tasks performed independently by individual crew members, and those undertaken by a team of crew members.

These interactions among plant personnel should include interactions between:

o MCR operators (e.g., operations, shift turnover walk downs) o MCR operators with auxiliary operators and other plant personnel performing tasks locally (e.g., maintenance or I&C technicians, chemistry technicians) o MCR operators and the TSC and the emergency offsite facility (EOF) o MCR operators with plant management, the NRC, and other outside organizations 126

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.1.1 states that a list of conditions and a checklist is prepared that SMEs use to construct the scenarios. All of the necessary dimensions are included in at least one scenario. Section 4.1.1.2 lists the types of personnel tasks that will be included in the scenarios and states that all of them will be included in the sample. Also, Table 4-2 SOC Dimension Matrix Example (Types of Personnel Tasks) contains the same list of types of personnel tasks in a checklist format that will be used by the SMEs to assign all the listed types of tasks to the scenarios. The staff compared the types of tasks listed in Section 4.1.1.2 and Table 4-2 to the above criteria and found Section 4.1.1.2 and Table 4-2 include all of the types of personnel tasks listed in the criterion.

The APR1400 V&V Scenarios Document, Section 5, provides seven specific scenarios that have been constructed by the applicant. Each scenario is summarized in Section 5 and described in more detail in seven appendices to the report. The V&V Scenarios Document, Section 3.2, Types of Personnel Tasks, contains the same checklist provided in Table 4-2 of the V&V IP that illustrates which task types are addressed by each scenario. The staff reviewed the scenario descriptions and Table/matrix of Section 3.2 of the V&V Scenarios Document and verified that all of the types of personnel tasks listed in the V&V IP, Section 4.1.1.2 were addressed by at least one scenario (with the below-noted exceptions).

In Table/matrix of Section 3.2 of the V&V Scenarios document, there are five of these personnel task types that are blank (e.g., the types of tasks have not yet been assigned to a scenario).

The five personnel task types are: IHAs, dominant systems from the PRA, OER-identified problematic tasks, procedures for controlling radioactivity, and maintenance procedures. There is table note 1 on IHAs in Table 4-2 of the V&V IP that explains that the TIHA ReSR needs to be completed in order to complete Table 4-2, but the same note is not included in the V&V Scenarios Document. Related to this issue, Section 4.1 of the V&V Scenario Document is titled Major Operator Errors and lists two errors. However, there is no explanation as to what major means or why the two errors are listed here, particularly since elsewhere in the document the important human action information is blank. The listing of two major errors seems to contradict other statements elsewhere that IHAs cannot yet be identified. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-61 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-61 (ML16074A294). In its response, the applicant stated that there currently is insufficient information to complete the identification of those five types of tasks and assignment of those tasks to specific scenarios. When the TIHA, OER, and procedure development human factors program elements are completed by the COL, then the information needed to finish assigning personnel tasks to scenarios will be available.

The applicant stated that the personnel tasks that have yet to be identified in the scenarios document will be identified and assigned to scenarios using the process outlined in Section 4.1.4 of the V&V IP. The assignment process for scenario definition was reviewed and found acceptable in the section below titled Scenario Definition. As part of the process of scenario definition, the V&V IP, Section 4.1.4 states that SMEs will use Table 4-2 to help ensure that all of the different types of personnel tasks are included in a scenario. The applicant also stated that the criteria for identifying the specific tasks are determined by the subject matter experts during their evaluation. The staff finds this acceptable because the V&V IP contains a process for updating the scenarios to include the types of personnel tasks listed in the criterion 127

when the information is available, and selection of the specific tasks is within the expected capabilities of the SMEs.

The applicant also committed to revise the V&V IP and V&V Scenario Document to address inconsistencies in the table notes between the V&V IP and the V&V Scenarios Document.

Change pages are provided with the RAI response that the staff finds acceptably clarify the inconsistencies. With respect to Major Operator Errors, the applicant stated that the information in Section 4.1 of V&V Scenario Document is not necessarily operator errors, but rather failures that might result from various operator errors or equipment failures. As such, the applicant said it will delete Section 4.1 of the V&V Scenarios Document. The staff noted that the information being deleted was characterized in the document as a summary of APR1400 PRA information, which is also summarized in DCD Chapter 19. Because the PRA information provided in DCD Chapter 19 is being reviewed by the staff as part of the design certification process, and because the PRA information will be further refined by the COL applicant, the PRA information could change. It would not be desirable for PRA information to be listed in more than one place in the application because it is possible the information may be updated in one part of the DCD, but not in the V&V Scenarios Document, which could cause confusion for the SMEs who will be completing the activities in the V&V IP. Thus, the staff finds it acceptable to remove this information from the V&V Scenarios Document because the V&V IP contains direction for the SMEs to refer to the TIHA ReSR, which extracts the PRA information that should be included in the HFE program from the most current source.

The staff confirmed that the V&V IP and V&V Scenarios document were revised as committed in response to RAI 352-8205, Question 18-61. Therefore, RAI 352-8205, Question 18-61 was resolved and closed. The staff finds that the application conforms to this review criterion.

Criterion 3 The applicant should include the following situational factors or error-forcing contexts known to challenge human performance. It also should include situations specifically designed to create human errors to assess the systems error tolerance, and the ability of personnel to recover from any errors, should these occur, for example:

High-Workload Situations - The sample should include situations where variations in human performance due to high workload and multitasking situations can be assessed.

Varying-Workload Situations - The sample should include situations wherein variations in human performance due to workload transitions can be determined.

These include conditions where there is: (1) a sudden increase in the number of signals that must be detected and processed after a period in which signals were infrequent, and (2) a rapid reduction in the need for detecting signals and processing demands following a time of high sustained task-demand.

Fatigue Situations - To the extent possible, the sample should include situations that may be associated with fatigue, such as work on backshifts and tasks performed frequently with repetitive actions, such as repeated inputs to a touch screen during plant operations or pulling rods.

Environmental Factors - To the extent possible, the sample should include environmental conditions that may cause human performance to vary, e.g., poor 128

lighting, extreme temperatures, high noise, and simulated radiological contamination.

The Staffs Evaluation of Criterion 3 The V&V IP, Section 4.1.1, Sampling Dimensions, states that a list of conditions and a checklist is prepared that SMEs use to construct the scenarios. All of the necessary dimensions of this criterion will be included in at least one scenario. Section 4.1.1.3, Situational Factors that are known to Challenge Human Performance, provides the listing of factors that will be included in the scenarios. The staff compared these to the above criteria and found the list acceptable. Also, the V&V IP, Section 4.5.2, Bullet 4, states that a test objective is to validate that the HSIs minimize personnel error and provide reasonable assurance of error detection and recovery capability when errors occur. This is accomplished through selecting error-forcing contexts outlined in the tables discussed in the following paragraph.

The V&V scenarios in Section 5, APR1400 V&V Scenarios, provides the seven specific scenarios that have been constructed by the applicant. Each scenario is described in summary fashion in Section 5 and in more detail in seven appendices to the report. The V&V Scenarios Document in Section 3 contains two tables that list personnel tasks and situational factors to be included in the V&V scenarios. Table 3.2 addresses 25 different personnel tasks and is reviewed in the previous criterion. Table 3.3 contains Situational Factors or Error-Forcing Contexts vs. scenarios that illustrate which ones are addressed by each scenario. The staff reviewed the scenario descriptions and the Tables of Section 3.3 and all four bullets of this criterion are addressed.

Regarding the 4th bullet, Environmental Factors, the applicant stated, in Section 4.4.2, General Methodology, of the V&V IP, that:

Environmental factors such as noise, lighting, temperature, and humidity of the simulator room can be made similar to the control room, but it is not certain that the environment can be duplicated so that it is identical to the actual control room.

This difficulty is to be expected. To address this, the applicant has included one scenario where only emergency lights are available and another scenario where the operators in the MCR must wear protective clothing due to simulated contamination in the MCR. The applicant further includes in the V&V IP a verification of environmental conditions in the actual completed MCR as part of the HFE Design Implementation element. The applicant also notes in the V&V IP, Section 4.4.3, HED Identification, that issues with environmental factors will be entered into the HED process and the issues tracking system.

However, during the initial review the staff was not able to determine if error tolerance and the ability of personnel to recover from any errors was included in the V&V IP or the scenarios document. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Questions 18-62 and 18-63 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-62 and 18-63 (ML16074A294). In its response to RAI 352-8205 Question 18-62, the applicant stated that error tolerance and the ability of personnel to recover from any errors are included in the V&V IP and the scenarios document but agreed to enhance its description. A description of the inclusion into V&V of error tolerance of the system and the ability of operators to recover from errors will be added to Section 4.1.1.3 of the human factors V&V IP. Also, it will add to the V&V 129

IP, Section 4.5.2 an objective to validate that the system has the ability to tolerate errors. The applicant provided acceptable drafts of both changes.

In its response to RAI 352-8205 Question 18-63, the applicant noted that V&V IP Section 4.4.1(4), Environmental Factors, states in part that:

Environmental factors such as noise, lighting, temperature, and humidity of the simulator room can be made similar to the control room, but it is not certain that the environment can be duplicated so that it is identical to the actual control room

. Therefore, the verification of the control room environment is conducted in the actual plant main control room or other locations within the scope of the V&V during the DI (Design Implementation).

The applicant also notes that there are short discussions of environmental factors in the V&V IP Sections: 4.1.1.3, Table 4-3, 4.1.5, 4.4.2, 4.4.3, 4.5.3, 4.5.5.1(5) and in the V&V scenarios document, ISV 4, Appendix D, page D9, Item 4 on protective clothing. The applicant agreed to correct some related minor errors to Table 4-3 on page 13 of the V&V IP. The changes were supplied as part of the response to Question 18-61 and are acceptable. The RAI 352-8205 Question 18-63 response is acceptable.

The staff confirmed that the V&V IP was revised as committed in responses to RAI 352-8205, Questions 18-62 and 18-63. Therefore, RAI 352-8205, Questions 18-62 and 18-63 were resolved and closed. The staff finds that the application conforms to this review criterion.

Identification of Scenarios NUREG-0711, Section 11.4.1.2, Identification of Scenarios, includes two criteria for this topic.

Criterion 1 The applicant should combine the results of the sampling to identify a set of V&V scenarios to guide subsequent analyses.

The Staffs Evaluation of Criterion 1 In the V&V IP, Section 4.1.2, Identification of Scenarios, the applicant stated that The identification of V&V scenarios is based on the sample of operational conditions that are selected in the [sampling of operational conditions SOC], as described in Subsection 4.1.1.

Further the document human factors V&V Scenarios provides 7 scenarios and demonstrates in the three tables of Section 3 how the results of sampling were used to identify the scenarios.

The staff finds that the application conforms to this review criterion.

Criterion 2 The applicant should not bias the scenarios by overly representing the following:

scenarios for which only positive outcomes are expected scenarios that, for ISV, are relatively easy to conduct (i.e., scenarios should not be avoided simply because they are demanding to set up and run on a simulator) 130

scenarios that, for ISV, are familiar and well structured (e.g., which address familiar systems and failure modes that are highly compatible with plant procedures, such as textbook design-basis accidents)

The Staffs Evaluation of Criterion 2 In the V&V IP, Section 4.1.2, the applicant stated that scenarios are balanced by avoiding an over reliance on scenarios with the following attributes and then lists the attributes of this criterion. Further the staff reviewed the 7 scenarios in the human factors V&V scenarios document and did not identify any of the above practices that could potentially bias the V&V such that only positive results could be obtained from the scenario. The staff finds that the application conforms to this review criterion.

Scenario Definition NUREG-0711, Section 11.4.1.3, Scenario Definition, includes three criteria for this topic.

Criterion 1 The applicant should identify operational conditions and scenarios to be used for HSI Task Support Verification, Design Verification, and ISV. The applicant should develop detailed scenarios suitable for use on a full-scope simulator. The level of detail should be comparable to what one would include in a test plan. For each one, the following information should be defined to reasonably assure that important dimensions of performance are addressed, and to allow the scenarios to be accurately and consistently presented for repeated trials:

a description of the scenario and any pertinent prior history necessary for personnel to understand the state of the plant at the start-up of the scenario specific initial conditions (a precise definition of the plants functions, processes, systems, component conditions, and performance parameters, e.g., similar to that at shift turnover) events (e.g., failures) that will occur during the scenario and their initiating conditions, e.g., based on time, or a value of a specific parameter precise definition of workplace factors, (e.g., environmental conditions, such as low levels of illumination) needs for task support (e.g., procedures and technical specifications) staffing level details of communication content between control room personnel and remote personnel (e.g., load dispatcher via telephone) scripted responses for test personnel who will act as plant personnel in the test scenarios the precise specification of what, when, and how data are to be collected and stored (including videotaping, questionnaires, and rating-scale administrations) 131

precise specifications on simulator set up specific criteria for terminating the scenario The Staffs Evaluation of Criterion 1 In the V&V IP, Section 4.1.3, Scenario Design Requirements, the applicant provides an overall philosophy as to how the scenarios are constructed and generally what activities are required of operators as the scenarios unfold. In Section 4.1.4 the applicant stated that the scenarios, based on the SOC, are to be used for HSI Task Support Verification, HFE Design Verification, and ISV. Section 4.1.4 further provides some background on how scenarios are actually constructed by SMEs using an iterative process as plant design and procedures are finalized.

In the human factors V&V IP, Section 4.1.5, Scenario Components, provides a list of items included in each scenario to ensure that the scenarios are understandable, include all important dimensions are addressed, and can be accurately and consistently performed. The list of items includes all of the above criteria. In the human factors V&V scenarios document, the staff also reviewed a selected scenario, namely a small-break, loss of coolant accident with computer-based procedure failure and HSI display failure. Each of the criteria in: (1) above was addressed in the scenario. The staff finds that the application conforms to this review criterion.

Criterion 2 The applicants scenarios should realistically replicate operator tasks in the tests; then, the findings from the test can be generalized to the plants actual operations.

The Staffs Evaluation of Criterion 2 The human factors V&V IP, Sections 4.1.3, 4.1.4 and 4.1.5 taken as whole explain in a number of detailed steps how the scenarios realistically replicate operator tasks, thus allowing the findings from the ISV to be generalized to the plants actual operations. For example, in Section 4.1.3 the V&V IP states:

[

]

In Section 4.1.4, the V&V IP states In Section 4.1.5, the V&V IP states that each scenario includes the items listed below to provide reasonable assurance that all important dimensions of performance are addressed.

[

132

]

Additionally, the staff reviewed the summaries of all scenarios in the human factors V&V Scenarios document and further concludes that these realistically replicate operator tasks. The staff finds that the application conforms to this review criterion.

Criterion 3 When the applicants scenarios include work associated with operations remote from the MCR, the effects on personnel performance due to potentially harsh environments (e.g., high radiation) should be realistically simulated (e.g., additional time to don protective clothing, and access radiologically controlled areas).

The Staffs Evaluation of Criterion 3 As described under Sampling Dimensions, Criterion (3), above, the applicant has specified the inclusion of environmental conditions that may cause human performance to vary, e.g., poor lighting, extreme temperatures, high noise, and simulated radiological contamination into the V&V program. Specifically, the V&V IP, Section 4.1.1.3, Situational Factors that are known to Challenge Human Performance, states situational factors are reflected in the selection of the operational conditions. Also, the V&V IP, Section 4.1.5, Scenario Components, states each scenario includes precise definition of workplace factors (e.g., environmental conditions) and the needed task support, which could include protective clothing. Items needed for task support, 133

are identified during task analysis, which is an input V&V (refer to the TA IP, Section 4.2.1, Task Narrative, Item 19, Support, which states any protective clothing needed to accomplish the task should be identified). As such, the staff finds the applicant has provided direction in the V&V IP for environmental conditions and the items needed to support completion of tasks that occur in those environmental conditions to be identified so they may be included or simulated in the scenarios. Therefore, the staff finds that the application conforms to this review criterion.

18.10.4.2 Design Verification The criteria in this section are divided into several subsections.

HSI Inventory and Characterization NUREG-0711, Section 11.4.2.1, HSI Inventory and Characterization, includes three criteria for this topic.

Criterion 1 Scope - The applicant should develop an inventory of all HSIs that personnel require to complete the tasks covered in the validation scenarios that were identified by the applicants Sampling of Operational Conditions (SOC). The inventory should include aspects of the HSI used for managing the interface, such as navigation and retrieving displays, as well as those that control the plant.

The Staffs Evaluation of Criterion 1 In the V&V IP, Section 4.2.1.1, Scope, the applicant described the scope of the inventory. The scope includes all HSIs needed to perform scenarios identified in the SOC. The inventory is maintained in an HSI database. The staff finds that the application conforms to this review criterion.

Criterion 2 HSI Characterization - The applicants inventory should describe the characteristics of each HSI within the scope of the verification. The following is a minimal set of information for this characterization:

a unique identification code number or name associated plant system and subsystem associated personnel functions and tasks type of HSI, e.g.,

o computer-based control (e.g., touch screen or cursor-operated button and keyboard input) o hardwired control (e.g., J-handle controller, button, and automatic controller) o computer-based display (e.g., digital value and analog representation) 134

o hardwired display (e.g., dial, gauge, and strip-chart recorder) display characteristics and functionality [e.g., plant variables/parameters, units of measure, accuracy of variable/parameter, precision of display, dynamic response, and display format (e.g., bar chart or trend plot)]

control characteristics and functionality [e.g., continuous vs. discrete settings, number and type of control modes, accuracy, precision, dynamic response, and control format (method of input)]

user-system interaction and dialog types (e.g., navigation aids and menus) location in data-management system (e.g., identification code for information display screen) physical location in the HSI (e.g., control panel section), if applicable The applicant should include photographs, copies of display screens, or similar samples of HSIs in the HSI inventory and characterization.

The Staffs Evaluation of Criterion 2 In the V&V IP, Section 4.2.1.2, HSI Characterization, the applicant described the HSI information contained in the inventory. The description of the characterization is consistent with the staffs review criterion and includes additional information that will be useful in the verification process, e.g., environment where the HSI is located. The staff finds that the application conforms to this review criterion.

Criterion 3 Inventory Verification - The applicant should verify the inventory description of HSIs to ensure that it accurately reflects its current state.

The Staffs Evaluation of Criterion 3 In the V&V IP, Section 4.2.1.3, Inventory Verification, the applicant described the verification of the inventory. The contents of the HSI database are verified by an SME to ensure all appropriate HSIs are represented and accurately depict the final design. The staff finds that the application conforms to this review criterion.

HSI Task Support Verification NUREG-0711, Section 11.4.2.2, HSI Task Support Verification, includes five criteria for this topic. The fifth criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first four criteria as discussed below Criterion 1 Verification Criteria - The applicant should base the HSI task support criteria on the alarms, controls, displays, and task support needed by personnel to complete their tasks as identified by the applicants task analysis.

135

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.3.1, Verification Criteria, states that the HSI task support criteria are based on the alarms, controls, displays, and task support needs that will be identified by the applicants task analysis. The staff finds that the application conforms to this review criterion.

Criterion 2 General Methodology - The applicant should compare the HSIs and their characteristics (as defined in the HSI inventory and characterization) to the needs of personnel identified in the task analysis for the defined sampling of operational conditions, noted in Section 11.4.1 of NUREG-0711.

The Staffs Evaluation of Criterion 2 The applicants HSI Task Support Verification methodology is described in Section 4.3.2 of the V&V IP. The V&V IP also contains Figure 4-1 depicting the process. A five-step process is used: Prepare an HSI checklist using the HSI Database, prepare task support items identified during the task analysis, collect HSI design data within selected operational conditions, compare HSI design documents against HSI and task support checklist, and prepare documentation.

The final step provides input to the human factors V&V ReSR. This method will effectively accomplish task support verification. The staff finds that the application conforms to this review criterion.

Criterion 3 Human engineering discrepancies (HED) Identification - The applicant should identify and document an HED when:

An HSI needed for task performance (e.g., a necessary control or display) is unavailable.

HSI characteristics do not match the requirements of the personnel task (e.g., a display may show the needed plant parameter but not within the range or precision needed for the task).

HSIs are available that are not needed for any task.

The Staffs Evaluation of Criterion 3 The applicants HSI Task Support Verification HED identification is described in Section 4.3.3 of the V&V IP. The V&V IP states that HEDs will be identified for each condition identified in the staffs review criterion. The staff finds that the application conforms to this review criterion.

Criterion 4 HED Documentation - The applicant should document HEDs to identify the HSI, the tasks affected, and the basis for the deficiency (what aspect of the HSI was identified as not meeting task requirements).

136

The Staffs Evaluation of Criterion 4 The applicants HED documentation is described in Section 4.3.3 and Section 4.5.9.5 of the V&V IP. Section 4.3.3 of the V&V IP states that: HEDs will be entered into the ITS for evaluation and resolution. However, no information is provided as to what information is entered into the system for each HED. Section 4.5.9.5 of the V&V IP provides generic information on HED documentation but does not provide specific information on Task Support Verification HEDs, e.g., the specific HSI and the basis for the HED such as the aspect of the HSI that was identified as not meeting task requirements. The ITS is also described in Section 4.6 of the HFE PP, but that does not provide this information either. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-64 to address this issue (ML15356A485).

On April 2, 2016, the applicant provided a response to RAI 352-8205, Question 18-64 (ML16093A024). The applicant clarified the information to be included in the issues tracking database for HEDs arising from TSV. This information includes a description of the current conditions and the issue creating the HED (i.e., the basis for the HED). It also includes identification of the specific HSI by its related system code, as well as the corrective actions taken to resolve the HED. The staff finds the proposed changes to the V&V IP ensure the documentation of HEDs contains the information listed in the criterion.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-64. Therefore, RAI 352-8205, Question 18-64 was resolved and closed. The staff finds that the application conforms to this review criterion.

Human Factors Engineering Design Verification NUREG-0711, Section 11.4.2.3, HFE Design Verification, includes five criteria for this topic.

The fifth criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first four criteria as discussed below.

Criterion 1 Verification Criteria - The applicant should base the criteria used for HFE Design Verification on HFE guidelines.

The Staffs Evaluation of Criterion 1 The applicants HFE Design Verification criteria are described in Section 4.4.1 of the V&V IP.

The criteria to be used for this verification are HFE guidelines in the applicants Style Guide (APR1400-E-I-NR-14012). The staff finds that the application conforms to this review criterion.

Criterion 2 General Methodology - The applicants HFE Design Verification methodology should include the following:

Procedures for comparing the characteristics of the HSIs with HFE guidelines for:

(1) the defined sampling of operational conditions, as noted in Section 11.4.1, and (2) the general environment in which HSIs are sited, including workstations, control rooms, and environmental characteristics (e.g., lighting and noise).

137

Procedures for determining for each guideline whether the HSI is acceptable or discrepant. If discrepant, it should be designated as an HED, tracked, and evaluated (see Sections 2.4.4 and 11.4.4).

Procedures for evaluating whether an HED is a potential indicator of additional issues.

The Staffs Evaluation of Criterion 2 Sections 4.4.1 and 4.4.2 of the V&V IP describe the HFE design verification methodology. The methodology specifies five steps. However, it is not clear from the methodology description how the characteristics of the HSIs will be compared to the HFE guidelines in the Style Guide.

Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-65 to address this issue (ML15356A485).

On May 12, 2016, the applicant provided a response to RAI 352-8205, Question 18-65 (ML16133A590). The applicant clarified the Style Guide is used as the criteria against which the HSI is verified. Specifically, the guidance in the Style Guide is developed into a checklist, which is then used to compare with the detailed design of the HSI. This acceptably addresses the staffs concern. The applicant has proposed changes to the V&V IP to clarify these points and the staff finds that these changes acceptably capture the information.

Based on the review of the V&V IP, the staff has confirmed incorporation of the changes described above; therefore RAI 352-8205, Question 18-65 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 3 HED Identification - The applicant should identify an HED when a characteristic of the HSI is "discrepant" from a guideline.

The Staffs Evaluation of Criterion 3 The applicants HED identification is described in Section 4.4.1 and 4.4.3 of the V&V IP. If the design does not conform to the criteria in the Style Guide, an HED is identified. Also, the verification team checks related HSIs to determine whether the HED is associated with only the HSI originally identified or more pervasively appears for other HSIs. If so, a new HED is created to capture the cross-cutting presence of the HED. The staff finds that the application conforms to this review criterion.

Criterion 4 HED Documentation - The applicant should document HEDs in terms of the HSI involved, and how its characteristics depart from a particular guideline.

The Staffs Evaluation of Criterion 4 The applicants HED documentation is described in Section 4.4.1 and 4.4.3 of the V&V IP.

Deviations from criteria are identified for each HED entered into the ITS. The staff finds that the application conforms to this review criterion.

138

18.10.4.3 Integrated Systems Validation The objective of reviewing integrated system validation methodology is to verify that the applicants methodology will validate the integrated system design (i.e., hardware, software, and personnel elements) using performance-based tests that will determine whether it acceptably supports safe operation of the plant.

The criteria in this section are divided into several subsections.

Validation Team NUREG-0711, Section 11.4.3.1, Validation Team, includes one criterion for this topic.

Criterion 1 The applicant should describe how the team performing the validation has independence from the personnel responsible for the actual design.

The Staffs Evaluation of Criterion 1 V&V IP, Section 4.5.1, Validation Team, describes the applicants validation team. The V&V IP states that the team is independent of the design organization. While the functions of the team are described, the relationship and interfaces between the validation team and the design team are not explained. This information is needed to verify the teams independence.

Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-66 to address this issue (ADAMS Accession No ML15356A485).

On May 13, 2016, the applicant provided a response to RAI 352-8205, Question 18-66 (ML16134A569). The applicant provided an explanation of the organizational and administrative controls that ensure independence of the validation team from the design team.

These controls include organizational placement of the validation team, its training, and the review by independent SMEs to evaluate independence. This acceptably addresses the staffs concern. The applicant has proposed changes to the V&V IP to clarify these points and the staff finds that these changes acceptably capture the information.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-66. Therefore, RAI 352-8205, Question 18-66 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Test Objectives NUREG-0711, Section 11.4.3.2, Test Objectives, includes two criteria for this topic. The second criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first criteria as discussed below Criterion 1 The applicant should develop detailed test objectives to provide evidence that the integrated system adequately supports plant personnel in safely operating the plant, to include the following considerations:

Validate the acceptability of the shift staffing level(s), the assignment of tasks to crew members, and crew coordination within the control room, between the 139

control room and local control stations and support centers, and with individuals performing tasks locally. This should encompass validating minimum shift staffing levels, nominal levels, maximum levels, and shift turnover.

Validate that the design has adequate capability for alerting, informing controlling, and feedback such that personnel tasks are successfully completed during normal plant evolutions, transients, design-basis accidents, and also under selected, risk-significant events beyond-design basis, as defined by sampling operational conditions.

Validate that specific personnel tasks can be accomplished within the time and performance criteria, with effective situational awareness, and acceptable workload levels that balance vigilance and personnel burden.

Validate that the HSIs minimize personnel error and assure error detection and recovery capability when errors occur.

Validate the assumptions about performance on important HAs.

Validate that the personnel can effectively transition between the HSIs and procedures in accomplishing their tasks, and that interface management tasks, such as display configuration and navigation, are not a distraction or an undue burden.

The Staffs Evaluation of Criterion 1 The applicants test objectives are described in Section 4.5.2 of the V&V IP. The objectives described correspond to those identified in the review criterion. The applicant has included an additional objective to validate assumptions made regarding the performance impacts of new technology. The staff finds that the application conforms to this review criterion.

Validation Testbeds NUREG-0711, Section 11.4.3.3, Validation Testbeds, includes nine criteria for this topic.

Criterion 1 Interface Completeness - The applicants testbed should represent completely the integrated system. It should include HSIs and procedures not specifically required in the test scenarios.

The Staffs Evaluation of Criterion 1 NUREG-0711 states that One approach an applicant can use to acceptably meet criteria 1 through 7 in this section is to use a testbed that is compliant with the American Nuclear Societys Nuclear Power Plant Simulators for Use in Operator Training (ANS 3.5, 2009).

ANS 3.5, 2009, Section 3, General Requirements, states:

The simulator shall be referenced to a specific unit. The scope of simulation shall be such that the operator is required to take the same action on the simulator to conduct an evolution as on the reference unit, using the reference 140

unit operating procedures. The scope of simulation shall permit conduct of all of the evolutions required in this section until a stable condition is obtained.

The evolutions listed in ANS 3.5, 2009, Section 3 are evolutions that operators perform during normal plant operations, such as plant startup, as well as evolutions operators perform to respond to the abnormal and emergency plant conditions that have been analyzed and described in the final safety analysis report (e.g., in Chapter 15, Transient and Accident Analyses). As such, all of the reference unit operating procedures that operators must use to perform normal and emergency evolutions from the control room are available in simulators that conform to ANS 3.5, 2009.

The applicants validation testbed is described in Section 4.5.3 of the V&V IP. The V&V IP states that A full-scope dynamic simulator developed to meet ANSI/ANS 3.5-2009 (Reference

13) is used as the validation testbed. Because the applicant stated that the simulator that will be used as the validation testbed will meet ANS 3.5, 2009, the staff concluded that the simulator will include all of the operating procedures that operators must use to perform all the evolutions listed in ANS 3.5, 2009, Section 3, even though the test scenarios only include a sample of those evolutions. Therefore, the staff initially concluded that the application conformed to this review criterion as well as to Criteria 2-7 listed below.

However, during the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member expressed concern that statements in the HFE PP and the HD IP indicated that only the procedures that will be used during the conduct of the test scenarios will be available in the simulator used for validation. Following the ACRS meeting, the staff reviewed the HD IP and the HFE PP and agreed that some statements in these documents did indicate that only the procedures that will be used for the ISV scenarios will be available in the simulator used for validation. If only the procedures that will be used during the conduct of the test scenarios will be available in the simulator, then opportunities to identify: (1) human performance or system errors associated with selecting and using procedures and (2) possible ways to reduce such errors will be minimized. Also, the staff determined that these statements contradicted the statement in Section 4.5.3 of the V&V IP that a full-scope dynamic simulator developed to meet ANSI/ANS 3.5-2009 (Reference 13) is used as the validation testbed. As such, the staff concluded the application did not fully conform to Criterion 1.

Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-135 to address this issue (ML17249A979).

On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-135 (ML17271A188) and proposed edits to the HD IP and the HFE PP to clarify that the simulator used for validation will also include procedures that are not needed for the ISV scenarios.

Specifically, the applicant revised the HFE PP and HD IP to explain that the HD program element converts the operating procedures executed from the control room, which are produced during the Procedure Development (PD) program element, into computer-based procedures prior to ISV. The inventory of CBP for the ISV scenarios will include additional procedures that are related to the ISV scenarios, but not actually needed to execute the ISV scenarios, to ensure the operator decisions are not influenced by the CBP inventory.

Based on the review of the HFE PP, the staff has confirmed incorporation of the changes described above; therefore RAI 553-9084, Question 18-135 was resolved and closed.

Because the applicant stated in Section 4.5.3 of the V&V IP that the testbed used for validation will be developed to meet ANS 3.5, 2009, and because the applicants proposed revisions to the HFE PP and HD IP to clarify that procedures not specifically used in the test scenarios will be 141

included in the simulator used for validation, the staff finds that the application conforms to this review criterion.

Criterion 2 Interface Physical Fidelity - The testbeds HSIs and procedures should be represented with high physical fidelity to the reference design, including the presentation of alarms, displays, controls, job aids, procedures, communications equipment, interface management tools, layout, and spatial relationships.

The Staffs Evaluation of Criterion 2 NUREG-0711 states that One approach an applicant can use to acceptably meet criteria 1 through 7 in this section is to use a testbed that is compliant with the American Nuclear Societys Nuclear Power Plant Simulators for Use in Operator Training (ANS 3.5, 2009). The applicants validation testbeds are described in Section 4.5.3 of the V&V IP. The V&V IP states that A full-scope dynamic simulator developed to meet ANSI/ANS 3.5-2009 (Reference 13) is used as the validation testbed. The staff finds that the application conforms to this review criterion.

Criterion 3 Interface Functional Fidelity - The testbeds HSI and procedure functionality should be represented with high fidelity to the reference design. All HSI functions should be available.

The Staffs Evaluation of Criterion 3 See Evaluation of Criterion 2.

Criterion 4 Environmental Fidelity - The testbeds environmental fidelity should be represented with high physical fidelity to the reference design, including the expected levels of lighting, noise, temperature, and humidity. Thus, for example, the noise contributed by equipment, such as air-handling units, computers, and communications equipment should be represented in validation tests.

The Staffs Evaluation of Criterion 4 See Evaluation of Criterion 2.

Criterion 5 Data Completeness Fidelity - Information and data provided to personnel should completely represent the plants systems they monitor and control.

The Staffs Evaluation of Criterion 5 See Evaluation of Criterion 2.

142

Criterion 6 Data Content Fidelity - The testbeds data content fidelity should be represented with high physical fidelity to the reference design. The presentation of information and controls should rest on an underlying model accurately mirroring the reference plant. The model should provide input to the HSI such that the information accurately matches that which is presented during operations.

The Staffs Evaluation of Criterion 6 See Evaluation of Criterion 2.

Criterion 7 Data Dynamics Fidelity - The testbeds data dynamics fidelity should be represented with high fidelity to the reference design. The process model should be able to provide input to the HSI so that information flow and control responses occur accurately and within the correct response time; e.g., information should be sent to personnel with the same delays as occur in the plant.

The Staffs Evaluation of Criterion 7 See Evaluation of Criterion 2.

Criterion 8 For important HAs at complex HSIs remote from the main control room (e. g., a remote shutdown facility), where timely, precise actions are essential, the use of a simulator or mockup should be considered to verify that the requirements for human performance can be met. (For less important HAs, or for non-complex HSIs, human performance may be assessed on analysis, such as task analysis, rather than on simulations.)

The Staffs Evaluation of Criterion 8 The applicants treatment of important HAs at complex HSIs remote from the MCR is described in Section 4.5.3 of the V&V IP. The V&V IP states that, For IHAs at complex HSIs remote from the MCR (e.g., Remote Shutdown Room) where precise actions are essential, a simulator or mockup is used to verify that the requirements for human performance can be met. For less non-complex HSIs, human performance is assessed based on analysis rather than on simulations. The staff finds that the application conforms to this review criterion.

Criterion 9 The applicant should verify the conformance of the testbed to the testbed-required characteristics before validation tests are conducted.

The Staffs Evaluation of Criterion 9 The applicants verification of the conformance of the testbed to the testbed-required characteristics is described in Section 4.5.3 of the V&V IP. The V&V IP states that

[

143

] The staff finds that the application conforms to this review criterion.

Plant Personnel NUREG-0711, Section 11.4.3.4, Plant Personnel, includes four criteria for this topic.

Criterion 1 Participants in the applicants validation tests should be representative of plant personnel who will interact with the HSI (e.g., licensed operators, rather than training personnel or engineers).

The Staffs Evaluation of Criterion 1 The V&V IP, Sections 4.5.4, Plant Personnel (Participants), and 4.5.6.4, Participant Training, describes the ISV participants. The V&V IP states that the participants will be operators who are license holders, familiar with PWR designs and who have held previous positions similar to the role they will have in the ISV tests. The staff finds that the application conforms to this review criterion.

Criterion 2 To properly account for human variability, the applicant should use a sample of participants that reflects the characteristics of the population from which it is drawn. Those characteristics expected to contribute to variations in system performance should be specifically identified; the sampling process should reasonably assure that the validation encompasses variation along that dimension. Determining representativeness should include considering the participants license type and qualifications, skill/experience, age, and general demographics.

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.5.4 states that the ISV participants will be selected from a pool of candidates who are representative of the characteristics of expected APR1400 operators. The V&V IP lists license type, training, experience, skill level, age, and demographics among the factors considered. The staff finds that the application conforms to this review criterion.

Criterion 3 In selecting personnel for participating in the tests, the applicant should consider the minimum shift staffing levels, nominal levels, and maximum levels, including shift supervisors, reactor operators, shift technical advisors, etc.

The Staffs Evaluation of Criterion 3 DCD Tier 2, Section 18.1.1.1 identifies HFE design constraints imposed by regulations in 10 CFR 50.54(m) and 10 CFR 50.54(k) on the MCR staffing levels. It also describes the initial staffing and qualifications (S&Q) assumptions for the APR1400:

A fundamental assumption of the APR1400 HFE design is that it is possible to operate the plant during postulated plant operating modes (modes 1 through 6) for normal, abnormal, and emergency conditions, with the following personnel in the main control room (MCR): one reactor operator (RO) with a reactor operator 144

license, one turbine operator (TO) with a reactor operator license, one electric operator (EO) with a reactor operator license, one shift supervisor (SS) with a senior reactor operator (SRO) license, and one shift technical advisor (STA) with a senior reactor operator license.

The Basic HSI TeR, Section 3.5.1 describes the initial MCR staffing concept and provides an overview of the roles of each of these personnel in the MCR. It states the [

] The concept reflects the initial staffing assumption as well as the staffing constraint. The V&V IP, Section 4.5.2 states, [

] Thus, it is possible that the S&Q analyses will result in a final staffing level that may change the initial staffing assumption; however, the final staffing level will be subject to the minimum control room staffing constraint imposed by staffing requirements in 10 CFR 50.54(k) and (m), which will ensure the MCR staffing consists of at least one operator (the RO) and one senior operator (the SS).

The staff reviewed the V&V IP and the V&V scenarios to evaluate how the applicant considered the staffing level in the selection of test participants. The V&V IP, Section 4.5.4 states that the ISV participants will consist of [

] Additionally, the staff found that one of the scenarios models the situation where some control room operators are outside of the control room when an event occurs. This scenario [

] For the rest of the scenarios, the staff found that the five test participants remain in the control room for the duration of the scenarios, which allows the nominal staffing level to be validated. However, no scenario simulates the case where the total possible number of operators that could be outside of the control room at the start of the event are outside of the control room at the start of an event.

The staff did not understand how the scenarios could validate the possible minimum control room staffing level that could exist. Therefore, on January 23, 2017, the staff issued RAI 534-8723, Question 18-132 to address this issue (ML17023A031).

On April 12, 2017, the applicant provided a response to RAI 534-8723, Question 18-132 (ML17102A979) and stated that [

] The applicant also provided revisions to the Basic Human-System Interface Technical Report to include this information. Thus, the V&V scenarios provide the opportunity to validate the applicants staffing concept.

Based on the review of the Basic HSI TeR, the staff has confirmed incorporation of the changes described above; therefore RAI 534-8723, Question 18-132 was resolved and closed. The staff finds that the application conforms to this review criterion.

Criterion 4 The applicant should prevent bias in the sample of participants by avoiding the use of participants who:

are members of the design organization 145

participated in prior evaluations were selected for some specific characteristic, such as crews identified as good performers or more experienced The Staffs Evaluation of Criterion 4 The V&V Section 4.5.4 describes the applicants treatment of sample bias. The V&V IP states that the participants will not be selected on any predetermined characteristic and that any operators that have participated in the design or any aspect of V&V will be excluded as potential participants. The staff finds that the application conforms to this review criterion.

Performance Measurement Types of Performance Measures NUREG-0711, Section 11.4.3.5.1, Types of Performance Measures, includes six criteria for this topic.

Criterion 1 The applicant should identify the specific plant performance measures applicable to each ISV scenario.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.5 explains the applicants general approach to plant performance measurement. The specific measures applicable to each scenario are contained in the detailed scenario descriptions in the human factors V&V scenarios document. In the latter, the measures are identified in the pass/fail criteria. For example, in the scenario, Anticipated Transient without Trip with Distributed Control System Failures, the reactor startup task has a pass criterion of The crew starts up and increases power toward 25 percent at a predetermined and proceduralized rate per GOP [general operating procedure] ... The staff reviewed a sample of the measures used for other events in this scenario and other scenarios and found them to be appropriate to the event performance being addressed. The staff finds that the application conforms to this review criterion.

Criterion 2 The applicant should identify the primary task measures applicable to each ISV scenario.

For each scenario, the applicant should identify the primary tasks operators must perform to accomplish scenario goals, so that such measures can be developed.

The measures chosen to evaluate personnel task performance should reflect those aspects of the task that are important to system performance, such as:

o time, o accuracy, o frequency, 146

o amount achieved or accomplished, o consumption or quantity used, o subjective reports of participants, and o behavior categorization by observers.

The analysis of primary tasks will support the identification of errors of omission (primary tasks not performed). Also, any actions and tasks that operators actually perform that deviate from the primary tasks should be identified and noted. These actions should be used to identify errors of commission.

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.5.5 explains the applicants general approach to primary task measurement. The specific measures applicable to each scenario are contained in the detailed scenario descriptions in the human factors V&V scenarios document. In the latter, the scenarios are subdivided into events and the events subdivided into primary tasks. For each primary task, measures are identified in the evaluation criteria. For example, in the Anticipated Transient without Trip with Distributed Control System Failures, one primary task is The crew performs an RCS [reactor coolant system] cooldown from the Safety Console with a minimum inventory of HSI alarms, displays, controls and procedures. The acceptance criterion for this task is The crew performs RCS cooldown to SDC [shutdown cooling] entry conditions per AOP

[abnormal operating procedure] without any unplanned transients, exceeding any procedural or regulatory limits, or reaching any unexpected protective actuation setpoints. The measurement of primary tasks is generally appropriate; however, the following was not clear to the staff:

At the end of each scenario in the instructions, observers are asked to evaluate whether all primary tasks were completed in a safe and timely manner and whether they were performed error free both individually and as a team. It is not clear how this evaluation is different from the specific criteria provided for each primary task during the course of the scenario.

In Appendix A, the Small Break Loss-of-Coolant Accident with Computer-Based Procedure Failure and Human-System Interface Display Failure scenario, the pass criterion for the primary task of Determine that SG Downcomer Radiation Monitor is malfunctioning and respond per alarm procedure Alarm-3761-01, Radiation Monitoring System is that The crew determines the alarm is the result of an instrument failure and responds per Alarm-3761-01, Radiation Monitoring System. The failure criterion is The crew misdiagnoses the alarm as a Steam Generator Tube Leak (SGTL) and responds per specified SGTL procedures. It is not clear how will the pass/fail criterion will be applied if the crew initially misdiagnoses the event but then recovers. If this is considered a Pass, it is not clear how the initial misdiagnosis with subsequent recovery will be addressed in the V&V findings or how the generic aspects of this question be applied to other scenarios.

Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-69 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-69 (ML16074A294). Regarding the first point in the RAI, the applicant clarified that the evaluations in question are an overall assessment of crew 147

performance that is distinct from the assessment of performance during the scenarios. The evaluation will be made at the end of the scenarios, which is appropriate for such a summative evaluation. The applicant has proposed changes to the V&V IP to clarify these points that acceptably capture the information.

Regarding the second point in the RAI, the applicant clarified that the evaluation of crew response to changes in plant performance are based on its initial actions. Thus, in the staffs example, the initial response was incorrect and a fail would be recorded and an HED created.

The staff finds this is a conservative approach to the assessment of crew responses and is therefore acceptable. In addition, it provides the opportunity to identify design changes as part of the HED assessment process. The applicant has proposed changes to the V&V IP to clarify these points and the staff finds that these changes acceptably capture the information.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-69. Therefore, RAI 352-8205, Question 18-69 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 3 The applicant should identify the secondary task measures applicable to each scenario.

The Staffs Evaluation of Criterion 3 The V&V IP, Section 4.5.5 explains the applicants general approach to secondary task measurement. The V&V IP states that secondary tasks are measured based on observations by operations experts and participant debriefings. The applicant identifies a list of secondary tasks that are measured, including navigation between displays, searching for controls, and formatting displays. Section 4.5.5.1, Types of Performance Measures Used, states that each is assessed by looking at [

] The V&V IP states that a seven-point Likert scale will be used to capture participant feedback at the end of each scenario. The staff finds that the application conforms to this review criterion.

Criterion 4 The applicant should identify the measures of situation awareness applicable to each scenario.

The Staffs Evaluation of Criterion 4 The V&V IP, Section 4.5.5 explains the applicants measurement of situation awareness (SA).

The primary way SA will be measured is the situation awareness rating technique (SART).

SART is administered at the end of the scenario. The SART is a widely-used SA measure that has been successfully used in multiple industries, including the commercial nuclear industry.

Information on operator SA will also be obtained from questionnaires and debriefings. The staff finds that the application conforms to this review criterion.

Criterion 5 The applicant should identify the workload measures obtained for each scenario.

148

The Staffs Evaluation of Criterion 5 The applicants measurement of workload is described in Section 4.5.5 of the V&V IP. The primary way workload will be measured is the National Aeronautics and Space Administration (NASA) Task Load Index (TLX). The TLX is a widely-used workload assessment tool with applications in many industries, including the commercial nuclear industry. It uses a rating scale method to assess six dimensions of workload, including mental demand, physical demand, temporal demand, performance, effort, and frustration level. The TLX is administered immediately following a scenario. The staff finds that the application conforms to this review criterion.

Criterion 6 The applicant should identify the anthropometric and physiological measures obtained for each scenario.

The Staffs Evaluation of Criterion 6 The applicants measurement of anthropometric and physiological factors is described in Section 4.5.5 of the V&V IP. The applicants primary way assessing these considerations is during HSI Design Verification. This is appropriate since the design style guide defines most of the human performance considerations for these factors. During the ISV tests, observers will assess the dynamic aspects of anthropometric and physiological factors such as movement around the control room. Information will also be obtained from operators using post scenario questionnaires. These are reasonable approaches in general, however, the observer and operator questionnaires are not provided, thus the acceptability of the implementation of the approaches cannot be determined. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-71 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-71 (ML16074A294). The applicant described how these assessments will be made using questionnaires to be completed by operators and observers. The questionnaires will be based on HFE and test design best practices, HSI design, and test objectives. The applicant described how these assessments will be made using 7-point rating scales. The applicant has proposed changes to the V&V IP and the staff finds that these changes acceptably capture the information.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-71. Therefore, RAI 352-8205, Question 18-71 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Performance Measure Information and Validation Criteria NUREG-0711, Section 11.4.3.5.2, Performance Measure Information and Validation Criteria, includes five criteria for this topic.

Criterion 1 The applicant should describe the methods by which these measures are obtained, e.g., by simulator data recording, participant questionnaires, or observation by subject-matter experts.

149

The Staffs Evaluation of Criterion 1 The applicants methods for collecting data are described in Section 4.5.5 of the V&V IP. For each measure described, the means of collecting the information is provided. In the case of the behaviorally anchored rating scale (BARS) and TLX, the rating scales are provided in appendices of the V&V IP. The staff finds that the application conforms to this review criterion.

Criterion 2 The applicant should specify when each measure is obtained (recorded), such as continuously, at specific points during the scenario, or after the scenario ends.

The Staffs Evaluation of Criterion 2 The applicants description of when measures are collected is contained in Section 4.5.5 of the V&V IP and in the detailed scenario description contained in the human factors V&V scenarios document. For SA measurement, the detailed scenario instructions for instructors, identify at what points the scenario should be stopped and the situational awareness global assessment technique (SAGAT) given to operators. The instructions also list the post-test evaluations to be made, such as the BARS and debriefings. The staff identified two concerns about when measures are taken.

1. It appears that the SAGAT measure of SA is taken twice for each scenario; once during a scenario stoppage and the second time at the end of the scenario. One of the values of the SAGAT method is that the changes in SA over the course of a scenario can be measured since the assessment is usually made at several points. The staff is concerned that a onetime measurement of SA during the scenario may not provide an accurate assessment. The staff is also concerned that use of the SAGAT measure following scenario termination may be of limited value. SAGAT is designed to capture the operators SA as events unfold. Some of the questions typically assess the operators awareness of where events are leading. At the end of the scenario, the plant has been stabilized. Since SAGAT is not a retrospective measure, all it may show is that the operators are aware they have achieved the stable condition.

2. The V&V IP states that the TLX will be collected at the end of each scenario.

However, the detailed scenario descriptions in the V&V scenarios document do not include the TLX in the list of Post Exercise Evaluation items.

Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-72 to address this issue (ML15356A485). On April 5, 2016, the applicant provided a response to RAI 352-8205, Question 18-72 (ML16096A208). The applicants response to the first set of the staffs questions provided a change in its approach to SA measurement. Rather than using the SAGAT method, SART will be used. SART is administered at the end of the scenario, thus the staffs concerns contained in question 1 are no longer applicable. Regarding the staffs second question, the applicant clarified that the TLX will be obtained at the end for each scenario and it will be added to the post-scenario descriptions. These responses acceptably address the staffs concern. The applicant has proposed changes to the HF V&V Scenarios IP to clarify these points and the staff finds that these changes acceptably capture the information.

150

Based on the review of the HF V&V Scenarios IP, the staff has confirmed incorporation of the changes described above; therefore RAI 352-8205, Question 18-72 was resolved and closed.

The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 3 The applicant should describe the characteristics (see Table 11-1 of NUREG-0711) of the performance measures.

Table 11-1 Characteristics of Performance Measures Characteristic Meaning Construct A measure should represent accurately the aspect of performance it is Validity intended to measure.

Reliability A measure should be repeatable; i.e., same behavior measured in exactly the same way under identical circumstances should yield the same results.

Sensitivity A measure's range (scale) and its frequency (how often data are collected) should be appropriate to that aspect of performance being assessed.

Unobtrusiveness A measure should minimally alter the psychological or physical processes that are being investigated.

Objectivity A measure should be based on easily observed phenomena.

The Staff's Evaluation of Criterion 3 The applicant's description of measurement characteristics is contained in Section 4.5.5 of the V&V IP; however, they are given only cursory, and often incomplete, treatment in the description of the specific measures. For example, for the use of Halden's behaviorally anchored rating scale (BARS) questionnaire, the V&V IP states that: 'T he behavior categorization by observers provides reliability and unobtrusiveness." Observer ratings may be reliable or unreliable, depending on the characteristics of the scales being used and the instructions given to observers on what the scales mean and how they should be used. The same can be said for unobtrusiveness. The V&V IP does not mention the other characteristics that are listed in Table 11-1 of NUREG-0711. Another example, in the discussion of the SAGAT measure of SA, the V&V IP simply has "(constructvalidity, reliability)" at the end of one sentence with no explanation. Additional information is needed that shows how the measurement characteristics, for the measures selected, achieve the attributes described in Table 11-1 of this NUREG-0711 criterion, and that are defined on Page 34 of the V&V IP. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-73 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-73 (ML16074A294). The applicant proposed V&V IP changes which involve adding a paragraph to each measurement description that describes the characteristics addressed in NUREG-0711, Table 11-1. Each type of measure is evaluated below.

151

Plant Performance Measures The applicant has stated that these measures are obtained from the simulator log and the measures are objective and are collected unobtrusively from the operators standpoint. The proposed changes to the V&V IP include a description of each of the characteristics in NUREG-0711, Table 11-1 of plant performance measures.

Primary Task Measures Time - This measure is obtained from the simulator log and has the same characteristics as plant parameters.

Subjective Reports - The applicant stated that such reports are collected post scenario, thus the reports are unobtrusive. The response further states that the characteristics of construct validity, reliability, sensitivity, and objectivity are not applicable. Since the subjective reports are debriefing sessions, the staff agrees.

BARS - These are rating scales filled out by observers. The applicants response indicates that BARS are an accepted behavioral assessment tool that is unobtrusively used. While this is generally true, the design of the specific BARS to be used by the applicant should be addressed. As rating scales, they can be well-designed for their purpose or poorly designed.

The applicants response further states that the characteristics of construct validity, sensitivity, and objectivity are not applicable. The staff does not agree with this position. This is not consistent with the fact that specific scale being used by the applicant was carefully designed to provide construct validity (see NUREG/IA-0137), a reliability study was conducted, its sensitivity has been demonstrated, and it has been used successful in commercial nuclear industry. The applicant should clarify the response and explain why construct validity, sensitivity, and objectivity are not applicable to its BARS.

A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. To support the meeting, KHNP provided the following information:

The response to [RAI 352-8205,] Question 18-73 will be revised to add to Behavior categorization by observers (page 36) construct validity and sensitivity. The characteristic of objectivity is not applicable to the application of BARS since they require subjective opinions and therefore are not easily observed. Objectivity will remain identified as not applicable. In addition, a reference to Appendix B will be added.

On August 9, 2016, the applicant provided a revised response to RAI 352-8205 Question 18-73 (ML16222A948), which clarified that the characteristics of construct validity, sensitivity, and objectivity for BARS is applicable Based on this information, the staff reviewed Appendix B, BARS Questionnaire, and determined that it addressed operator behavior through observation of five itemscommunication, team spirit, openness, coordination as crew, and task focus/decision making. The staff determined that this application of the BARS questionnaire is evaluating activities beyond what is directed by guidance documents. While it is important to ensure the ISV team is functional the regulatory guidance establishes this functionality by providing standards for operator experience, ISV participant training, and an ISV pilot. RAI 352-8205, Question 18-73 regarding the BARS questionnaire was resolved and no further regulatory reviews are being conducted of this activity since it is beyond regulatory guidance and has no negative impact on the portion of the ISV for which regulatory guidance does apply.

152

Errors - The applicant provided information on error measurement. Potential errors are identified for each scenario and its occurrence identified by operations experts, supporting its construct validity and sensitivity. Operations experts can unobtrusively and reliably identify them.

Thus, the staff finds the applicant has described the characteristics of the primary task performance measures, and the characteristics are reasonable.

Secondary Tasks The applicant did not provide information on secondary task measurement characteristics.

A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. To support the meeting, KHNP provided the following information:

A discussion of the characteristics for secondary task measures will be added on the top of page 36 of the V&V IP, as was done for primary tasks in the response, to the revised response to the Question 18-73.

In the response to RAI 352-8205 Question 18-73 (ML16222A948), the applicant provided the characteristics for the secondary task measures. The applicant said ((the characteristics of the means used to assess secondary task performance (e.g., time, subjective report of participants, behavior categorization by observers, and errors), which are also means used to assess primary task performance, have the same characteristics of the means used to assess primary task performance.)) As explained above under Primary Tasks, the staff finds the applicant has described the performance characteristics of the means used to evaluate primary tasks. As such, the applicant has described the characteristics of the secondary task measures.

Situational Awareness In RAI 352-8205, Question 18-72 KHNP stated that SA would be assessed using the SART rather than the SAGAT. The staffs questions about performance measure characteristics about SAGAT are no longer applicable.

Workload The NASA TLX will be used. As the applicant stated, this is an industry accepted method.

However, the applicant has stated that the TLXs reliability and objectivity are not applicable.

The staff does not agree with this position. The applicant should address these characteristics.

A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. To support the meeting, KHNP provided the following information:

NASA-TLX can be considered a reliable measure, based on its industry performance. However, since it is based on opinion, it does not represent an objective measure. The response to 18-73 on page 37 of the human factors V&V IP for Workload will be revised to identify the characteristic of reliability.

The staff finds this conclusion to be consistent with industry experience.

153

Video and audio recording measures The applicant provides measurement characteristics for video and audio recordings. The staff does not understand what this provides. Measurement characteristics have to do with the specific aspects of performance measures being assessed. Thus, they are applicable to specific measures being used to assess aspects of performance such as operator task performance, situation assessment, and workload. Video and audio recordings are not aspects of performance, per se; although they provide the raw data used by analysts.

A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. To support the meeting, KHNP provided the following information:

The response to Question 18-73, page 38 of the V&V IP will be revised by deleting the proposed revision to (6).

The staff finds this acceptable as it treats video and audio recordings as a data collection tool rather than a performance measure.

Overall the applicants response at the public meeting addresses the staffs request. On August 9, 2016, the applicant provided a revised response to RAI 352-8205, Question 18-73.

The applicant stated it will revise the V&V IP, Section 4.5.5.1 to clarify the use of video and audio recordings for data collection. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-73. Therefore, RAI 352-8205, Question 18-73 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 4 The applicant should identify the specific criterion for each measure used to judge the acceptability of performance and describe its basis.

The Staffs Evaluation of Criterion 4 The applicants description of measurement characteristics is contained in Section 4.5.5 of the V&V IP. The applicants description of the specific criterion to be used for each measurement and its basis is contained in Section 4.5.5 of the V&V IP and in the V&V scenarios document.

The staff had the following questions related to the specific criterion for each measure:

1. The specific criteria to be used for many of the measures are in the detailed instructions for the validation scenarios. However, specific criteria are not provided for all measures, e.g., for the BARS, SA, and workload measures.

2. The bases for the criteria are summarized in Table 4-6. The V&V IP states that the criteria basis for SA and TLX are benchmarked to a predecessor or reference plant. The staff was unable to determine which plant SAGAT and TLX were benchmarked.

Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-74 to address this issue (ML15356A485). On April 5, 2016, the applicant provided a response to RAI 352-154

8205, Question 18-74 (ML16096A208). The applicant proposed changes to the V&V IP to clarify the following:

Plant operations SMEs will determine the pass/fail criteria for BARS, which is a measure of teamwork. The V&V IP, Appendix B, contains the BARS questionnaires and a 7-point ranking scale that will be evaluated by the plant operations SMEs when they observe the operators during the ISV. These plant operations SMEs will determine the pass/fail threshold for BARS. This is acceptable because the plant operations SMEs will have experience operating in a control room team and will be knowledgeable about behaviors that contribute to effective teamwork and collaboration, and therefore, these SMEs have the appropriate qualification to: (1) make evaluations about whether the integrated system provides the environment for operators to exercise behaviors that promote effective teamwork, and (2) identify the criteria for the BARS rating that will be indicate that the integrated system promotes effective teamwork and therefore is acceptable.

For SA and workload, the applicant revised the V&V IP to identify the plants that will be used to benchmark the SART and NASA TLX values. These predecessor plants are the APR1400 design as constructed in Korea and the United Arab Emirates. The applicant stated in the RAI response that the ISV for these plants will be completed before the ISV for the APR1400 in the US. The SART and TLX values must at least meet the benchmarked values at the predecessor plants to be acceptable. Because the predecessors are similar in design and operations to the APR1400, this is a reasonable method for evaluating whether the values of SA and workload are acceptable.

Based on the review of the V&V IP, the staff has confirmed incorporation of the changes described above; therefore RAI 352-8205, Question 18-74 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 5 The applicant should identify whether each measure is a pass/fail one or a diagnostic one.

The Staffs Evaluation of Criterion 5 The applicants identification of pass/fail vs. diagnostic measures is contained in Section 4.5.5 of the V&V IP. The applicant defines each category as follows: Pass/fail (P/F) measures are identified in each scenario before the start of the ISV and are used to determine whether the design is validated. Diagnostic measures are collected to assess personnel performance during each scenario and to analyze errors and its root causes as they relate to the HSI design.

However, the staff notes that the relative proportion of P/F vs. diagnostic measures is heavily weighted to P/F measures, thus the rational for selecting P/F vs. diagnostic measures is needed. Also, based on the applicants definition of P/F measures, the staff would not expect measures such as the behaviorally anchored rating scale (BARS) to be included. Additional information describing the application of the applicants definition to individual measures is needed. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-75 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-75 (ML16074A294). The applicant indicated that only objective measures will be used as P/F 155

criteria. The applicants response proposed changes to the methodology to reflect this position, e.g., BARS evaluations were changed to diagnostic rather than a P/F criterion. The staff agrees with these changes. The applicant has proposed changes to the IP to clarify these points and the staff finds that these changes acceptably capture the information.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-75. Therefore, RAI 352-8205, Question 18-75 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Test Design Scenario Sequencing NUREG-0711, Section 11.4.3.6.1, Scenario Sequencing, includes two criteria for this topic.

Criterion 1 The applicant should balance scenarios across crews to provide each crew with a similar, representative range of scenarios.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.6.1, Scenario Sequencing, describes the applicants sequencing of scenarios. The ISV will use three crews and seven scenarios. Each crew will perform each scenario. This will provide a similar and representative range of scenarios to all crews. The staff finds that the application conforms to this review criterion.

Criterion 2 The applicant should balance the order of presentation of scenarios to crews to provide reasonable assurance that the scenarios are not always presented in the same sequence (e.g.,

the easy scenario is not always used first).

The Staffs Evaluation of Criterion 2 The applicants sequencing of scenarios is described in Section 4.5.6.1 of the V&V IP. The ISV will use three crews and seven scenarios. Each crew will perform each scenario. Table 4-7 of the V&V IP illustrates the order of scenario presentation across crews. The order is balanced so each crew is presented with scenarios in a different sequence. The staff finds that the application conforms to this review criterion.

Test Procedures NUREG-0711, Section 11.4.3.6.2, Test Procedures, includes two criteria for this topic.

Criterion 1 The applicant should use detailed, unambiguous procedures to govern the conduct of the tests.

These procedures should include the following:

the identification of which crews receive which scenarios, and the order in which they should be presented 156

detailed and standardized instructions for briefing the participants specific directions for the testing personnel on conducting the test scenarios, as elaborated in Scenario Definition (NUREG-0711, Section 11.4.1.3) guidance on when and how to interact with participants when difficulties occur in simulation or testing instructions on when and how to collect and store data. These instructions should stipulate which data are to be recorded by:

o simulator computers o special-purpose instruments and devices for collecting data such as situation awareness- and workload-questionnaires, or physiological measures o video recorders (locations and views) o test personnel and subject-matter experts (such as via observational checklists) procedures for documentation:

o identifying and maintaining files of test records including details of the crew and scenarios o data collected o logs created by those who conducted the tests The procedures should detail the types of information that should be logged (e.g., when the tests were performed, deviations from the test procedures and why they occurred, and any unusual events that may be important to understanding how a test was run or for interpreting the findings from it). The procedure also should state when the types of information should be recorded.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.6.2, Test Procedures, describes the applicants test procedures. The human factors V&V scenarios document contains many aspects of the procedures that are specific to each of the seven scenarios. Table 4-7 of the V&V IP identifies the type of sequences that will be used for presenting scenarios for each crew. The human factors V&V scenarios document contains the scenarios descriptions and directions for testing personnel, instruction on data collection procedures and documentation. However, there are two aspects of the test procedures that were not addressed: (1) the detailed and standardized instructions used to brief participants, and (2) the guidance to test conductors for when and how to interact with participants when difficulties occur in simulation or testing. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-76 to address this issue (ML15356A485).

157

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-76 (ML16074A294). On August 9, 2016, the applicant provided a revised response to RAI 352-8205, Question 18-76 (ML16222A948), which included minor corrections and clarifications to the earlier response. The applicant provided the detailed instructions to brief participants for each scenario. These instructions are added to the scenario descriptions. In addition, the applicant developed a new Attachment 5 for each scenario providing explicit instructions for interacting with operators when test issues arise. This acceptably addresses the staffs request.

As noted above, the applicant has proposed changes to the scenario descriptions and the staff finds that these changes acceptably capture the information.

The staff confirmed that the V&V IP was revised as committed in the response to RAI 352-8205, Question 18-76. Therefore, RAI 352-8205, Question 18-76 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 2 The applicants test procedures should minimize the opportunity for bias in the test personnels expectations and in the participants responses.

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.5.6.2, Test Procedures, describes the applicants use of test procedures to minimize bias. The V&V IP states that the procedures will minimize bias; however, little information is provided on how they will do so. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-77 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-77 (ML16074A294). The applicant provided the detailed instructions for minimizing the opportunity for bias in the test personnel's expectations and in the participants responses by ensuring that all scenarios are managed in a consistent and controlled manor. This acceptably addresses the staffs request. The applicant has proposed changes to the scenario descriptions and the staff finds that these changes acceptably capture the information.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-77. Therefore, RAI 352-8205, Question 18-77 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Training Test Personnel NUREG-0711, Section 11.4.3.6.3, Training Test Personnel, includes one criterion for this topic.

Criterion 1 The applicant should train test personnel (those who conduct or administer the validation tests) on the following:

the use and importance of test procedures bias and errors that test personnel may introduce into the data through failures to follow test procedures accurately or to interact with participants properly 158

the importance of accurately documenting problems arising during testing, even if they were due to an oversight or error of those conducting the test The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.6.3, Test Personnel Training, describes the applicants training of test personnel. Personnel will receive five days of training prior to pilot testing. The scope of the training will address use of procedures, potential sources of bias, and the importance of documentation. Several additional topics will be addressed as well, including use of pass/fail and diagnostic measures, APR1400 HSI and system design, and the roles of validation team members. The staff finds that the application acceptably conforms to this review criterion.

Training Participants NUREG-0711, Section 11.4.3.6.4, Training Participants, includes two criteria for this topic.

Criterion 1 The applicants training of participants should be very similar to the training plant personnel receive. It should reasonably assure that the participants knowledge of the plants design, and operations, and the use of the HSIs and procedures represents that of experienced plant personnel. Participants should not be trained specifically to carry out the selected validation scenarios.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.6.4, Participant Training, describes the applicants training of test participants. The V&V IP indicates that all participants are licensed and experienced with PWR operations. All have held positions similar to those they will assume in ISV testing. They will then receive training prior to selection to participate in ISV trials. The training will involve plant design, operations, operating procedures, and HSIs. Participants will not be trained on the specific scenarios used for ISV. The staff finds that the application conforms to this review criterion.

Criterion 2 To assure that the participants performance is representative of plant personnel, the applicants training of participants should result in near asymptotic performance (i.e., stable, not significantly changing from trial to trial) and should be tested for such before conducting the validation.

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.5.6.4, Participant Training, describes the applicants training of test participants. Participants will receive training on plant design, operations, operating procedures, and HSIs. Participants will have to obtain an 80 percent passing grade to demonstrate proficiency and eligibility to be included in ISV tests. The staff finds that the application acceptably conforms to this review criterion.

Pilot Testing NUREG-0711, Section 11.4.3.6.5, Pilot Testing, includes two criteria for this topic.

159

Criterion 1 The applicant should conduct a pilot study before the validation tests begin to offer an opportunity for the applicant to assess the adequacy of the test design, performance measures, and data-collection methods.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.6.5, Pilot Testing, indicates that a pilot test will be performed approximately six months prior to ISV trials to allow for resolutions of problems. The pilot test will be conducted by running all seven scenarios under actual test conditions. The testing will identify potential issues with the simulator, executing scenarios, test procedures, data collection, and training of testers and participants. The staff finds that the application conforms to this review criterion.

Criterion 2 The applicant should not use participants in the pilot testing who will then be participants in the validation tests.

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.5.6.5, Pilot Testing, states that the plant personnel involved with pilot testing will not be included in ISV testing crews. The staff finds that the application conforms to this review criterion.

Data Analysis and HED Identification NUREG-0711, Section 11.4.3.7, Data Analysis and HED Identification, includes seven criteria for this topic.

Criterion 1 The applicant should use a combination of quantitative and qualitative methods to analyze data.

The analysis should reveal the relationship between the observed performance and the established performance criteria.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.7 describes the applicants data analysis. The V&V IP indicates that the analysis will use both qualitative and quantitative methods to assess the acceptability of the results. The staff finds that the application conforms to this review criterion.

Criterion 2 The applicant should discuss the method by which data is analyzed across trials, and include the criteria used to determine successful performance for a given scenario.

The Staffs Evaluation of Criterion 2 The V&V IP, Sections 4.5.7.1, Individual Performance Measures, and 4.5.7.2, Individual Scenarios and Assumptions of IHAs, describe the applicants data analysis. For many evaluations, the applicant plans to conduct T-tests using a significance level of p .05.

160

However, the staff has two questions about how T-test will be applied in the analyses. The first question concerns the use of the p .05 significance level. This is the typical significance level used to test hypotheses in behavioral science research, and the minimum set for claiming two means are significantly different. However, its application to the integrated system validation analyses is unclear. Using workload as an example, if the average workload of the APR1400 design is found to be 3.5, and the average for the reference plant is 4, but the difference is not statistically significant, then it is not apparent that a conclusion can be drawn regarding the acceptability of the workload. One cannot conclude the workload levels are statistically the same; that would be confirming the null hypothesis, which is not statistically justified. Greater specificity in the V&V IP is needed on the use of the p .05 significance level and its application to the integrated system validation analyses.

The second question concerns the fact that for the statistical tests conducted separately for each scenario, the means are based on three data points (three crews). Such a situation will lead to very few degrees of freedom for the statistical test and a great chance of failing to find statistical significance. Additional information is needed to address the issue of low degrees of freedom for scenario evaluations. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-78 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-78 (ML16074A294). The applicant provided additional clarification that the statistical analyses only provide input to the assessment of results which also include qualitative assessments and the convergence of results on a common result.

Since the statistical tests only provide input to the analysis and are not the basis of deriving conclusion about the results, this response satisfies the staffs question about the low number of degrees of freedom in the statistical tests. The applicant proposed changes to the V&V IP to implement its response. While the proposed changes are reasonable, they do not include key information from the applicants response. The staff thought that the following parts of the applicants response to the RAI were important considerations for the data analysis and should be included in appropriate locations in the V&V IP:

Although simple quantitative statistical tools are applied to support the understanding of the meaning of the ISV data, the APR1400 ISV data analysis must rely on qualitative methods. These include the application of SME analysis and the assessment of the degree of convergence of multiple measures of performance, convergent validity.

The conclusions drawn from the ISV use the resulting P values to only inform the qualitative analysis.

A public meeting with KHNP was held on July 25, 2016, to communicate this concern. KHNP stated the response to Question 18-78 will be revised by adding this material to the V&V IP. On August 9, 2016, the applicant provided a revised response to RAI 352-8205, Question 18-78 (ML16222A948). In the response, the applicant stated that it will revise the V&V IP, Sections 4.5.7 and 4.5.8 to clarify the use of the resulting P values. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-78. Therefore, RAI 352-8205, Question 18-78 was resolved and closed.

In all cases, the applicant defines the criteria for successful performance. For example, for the BARS measure, an average 4 is considered a success. For SA, a score the comparison reference plant is considered a success. For plant performance measures, the V&V IP states 161

that the [

]

However, the method used to complete this comparison is not clearly described. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-79 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-79 (ML16074A294). The applicant provided an explanation of how these analyses are performed.

They are conducted by the implementation team using the scenario descriptions, simulator data, and video/audio tapes to compare expected performance (scenario descriptions) to the tasks actually performed by the crews. The methodology provides a means to assess the acceptability of deviations from one crew to another. This acceptably addresses the staffs request. The applicant has proposed changes to the V&V IP that acceptably capture this information.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-79. Therefore, RAI 352-8205, Question 18-79 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 3 The applicant should evaluate the degree of convergence between related measures (i.e.,

consistency between measures expected to assess the same aspect of performance).

The Staffs Evaluation of Criterion 3 In the fifth paragraph of Section 4.5.5 of the V&V IP, the applicant stated that [

] However, how this evaluation is performed is not discussed. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-80 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-80 (ML16074A294). The applicant provided an explanation of how the analysis of convergence is performed. The analysis is performed following the collection of test data. Multiple analyses are performed including a review by test personnel of the different types of data collected to determine the consistency across measures. This acceptably addresses the staffs request.

The applicant has proposed changes to the V&V IP and the staff finds that these changes acceptably capture this information.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-80. Therefore, RAI 352-8205, Question 18-80 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 4 When interpreting test results, the applicant should allow a margin of error to reflect the fact that actual performance may be slightly more variable than observed validation-test performance.

162

The Staffs Evaluation of Criterion 4 The V&V IP, Section 4.5.7 describes the applicants data analysis. However, the discussion does not specifically address margin of error in the interpretation of performance. Margin is discussed in Section 4.5.9, Human Engineering Discrepancy Resolution Review Criteria, e.g.,

[

] However, how margin is used to make decisions is not discussed.

Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-81 to address this issue (ML15356A485).

On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-81 (ML16074A294). The applicant provided explanation of how the margin of error is evaluated.

Three methods are discussed including: (1) a comparison by SME of the observed results and what would be expected in actual operating plants, (2) an assessment by SMEs of the differences between scenario-simulated plant parameters and those done for the safety analysis, and (3) an evaluation by SMEs of the differences between observed human performance and that expected. The staff identified some questions about these analyses:

As of the first analysis, the staff is not sure what a comparison of results to that expected in actual operating plants reveals. Since the APR1400 is a new design, difference between such results may related to plant design and operational differences. The staff is not sure what this tells the analyst about margin of error.

As of the second analysis, the staff thinks an assessment of the differences between scenario simulated plant parameters and those done for the safety analysis relates to the suitability of the simulator modeling for representing the behavior of the plant being modeled (or the validity of the safety analysis). The staff thinks such an analysis would be performed as part of simulator qualification before the simulator is used for ISV.

As of the third analysis, the staff is not sure what the SMEs are evaluating.

Clarification is needed as to what these analyses consist of and how they are used to consider margin of error when interpreting the results.

A public meeting with KHNP was held on July 25, 2016, to communicate these concerns.

KHNP explained that the general approach for applying margins during the interpretation of results was to apply subject matter expert knowledge of where actual operations diverged from the ISV test setting. But since KHNP has applicable operating experience from predecessor plants, it also wanted to utilize that experience to help identify conditions where actual experience has or could diverge from the ISV test setting. This approach was also developed to provide more quantifiable actions in support of its implementation level submittal. The applicant also stated it will revise the V&V IP to include the additional information provided in its revised response to RAI 352-8205, Question 18-81. The staff found this approach to be satisfactory as it complemented the usual practice of just using SME judgement to establish margins.

On August 9, 2016, the applicant provided a revised response to RAI 352-8205, Question 18-81 (ML16222A948), which included the changes made in response to the staffs concerns discussed during the public meeting on July 25, 2016. Based on the review of the V&V IP, the staff has confirmed incorporation of the changes described above; therefore RAI 352-8205, 163

Question 18-81 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion.

Criterion 5 The applicant should verify the correctness of the analyses of the data. This verification should be done by individuals or groups other than those who performed the original analysis, but may be from the same organization.

The Staffs Evaluation of Criterion 5 The V&V IP, Section 4.5, Integrated System Validation, describes verification of results. The V&V IP states [

] The staff finds that the application conforms to this review criterion.

Criterion 6 The applicant should identify HEDs when the observed performance does not meet the performance criteria.

The Staffs Evaluation of Criterion 6 The V&V IP, Section 4.5.7 describes the applicants data analysis. Individual measures are described in Section 4.5.7.1, Individual Performance Measures. For each measure, the applicant has identified when the results will be classified as an HED. (Note that the staff requested clarification on how plant performance measures are assessed, but that is addressed in RAI 352-8205, Question 18-81, related to NUREG-0711 Criterion 11.4.3.7-2 reviewed above.)

In addition, Section 4.5.7.3, Extent of the Issue Determination, indicates that HEDs will be evaluated to determine the extent to which they may represent cross-cutting issues. If such a determination is made, the cross-cutting issue will be identified as an HED. The staff finds that the application conforms to this review criterion.

Criterion 7 The applicant should resolve HEDs identified by pass/fail measures before the design is accepted.

The Staffs Evaluation of Criterion 7 The V&V IP, Section 4.5.8, Validation Conclusions, indicates all HEDs associated with pass/fail measures will be closed before the design is considered validated. The staff finds that the application conforms to this review criterion.

Validation Conclusions NUREG-0711, Section 11.4.8, Validation Conclusions, includes two criteria for this topic.

Criterion 1 The applicant should document the statistical and logical bases for determining that performance of the integrated system is and will be acceptable.

164

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.8 describes the applicants formulation and documentation of validation conclusions. The V&V IP presents the criteria to be used in determining that the design is validated, including the resolution of HEDs. An overview of the criteria including all measures is presented as an example in Table 4-15, Example of Acceptance Criteria for ISV. The V&V IP further states that all analysis results will be documented in a ReSR. The staff finds that the application conforms to this review criterion.

Criterion 2 The applicant should document the limitations in the validation tests, its possible effects on the conclusions of the validation, and its impact on implementing the design.

The Staffs Evaluation of Criterion 2 The V&V IP, Section 4.5.8 describes the applicants formulation and documentation of validation conclusions. The V&V IP indicates that limitations of testing will be addressed and documented in a ReSR. The staff finds that the application conforms to this review criterion.

18.10.4.4 Human Engineering Discrepancy Resolution Review NUREG-0711, Section 11.4.4, Human Engineering Discrepancy Resolution Review Criteria, includes five criteria for this topic.

Criterion 1 HED Analysis - The applicants HED analyses should include the following:

Personnel Tasks and Functions - The impact of HEDs on personnel tasks and the functions supported by those tasks.

Plant Systems - The impact of HEDs on plant systems, considering the safety significance of that system(s), their effect on accident analyses, and their relationship to risk-significant sequences in the plants PRA.

Cumulative Effects of HEDs - The analysis of HEDs should identify the cumulative effects that multiple HEDs may have on plant safety and personnel performance.

HEDs as Indications of Broader Issues - As well as addressing specific HEDs, the applicants analysis should determine whether the HEDs point to potentially broader problems.

The Staffs Evaluation of Criterion 1 The V&V IP, Section 4.5.9.1, Human Engineering Discrepancy Analysis, states that the analysis of HEDs evaluates the impact of the HED on the following categories:

Personnel tasks and functions - Determination of the importance of the personnel function to safety and the cumulative effect on personnel performance 165

Plant systems - Analysis of the impact of HEDs on plant systems, considering the safety significance of the systems, the effect on accident analysis, and the relationship to risk-significant sequences in the PRA Cumulative effects of HEDs - Analysis of HEDs to identify the cumulative effects that multiple HEDs may have on plant safety and personnel performance.

Section 4.5.9.1 also states that they will evaluate HEDs as indications of broader issues, including a determination of whether an HED indicates a potentially broader problem or an extent of the issue across the HSI system. Further, Section 4.5.7.3 notes that HEDs may cut across scenarios and thus represent cross-cutting issues. This section explains how these issues will be evaluated and gives examples of areas to be considered in the data analysis step (e.g., procedures, training, displays, alarms, and controls). Section 4.5.9.3, HED Analysis and Development of Design Solutions, describes in more detail how the analysis of the HEDs is done by the team of SMEs. The V&V IP, Section 5 provides information on the various SME teams and specifies that the HED Analysis Team will contain experts in: plant operations, HFE, systems engineering, I&C engineering.

The staff finds that the application conforms to this review criterion.

Criterion 2 Selection of HEDs to Correct - The applicant should conduct an evaluation to identify which HEDs to correct. The evaluation should identify those HEDs that are acceptable as is. The remaining discrepancies should be denoted as HEDs to be addressed by the HED-resolution process.

HEDs the applicant should correct are those with direct safety consequences, namely, those that could adversely impact personnel performance such that the margin of plant safety may be reduced below an acceptable level. Unacceptability is indicated by such conditions as violations of Technical Specification safety limits, operating limits, or limiting conditions for operations, or failing an ISV pass/fail criterion.

HEDs with potential safety impact, not as severe as those described above, also should be corrected unless the applicant justifies leaving the condition as is.

The applicant should correct HEDs that may adversely impact personnel performance in a way that has potential consequences to plant performance or operability of structures, systems, or components, and personnel performance or efficiency. This may include failing to meet personnel information needs or violating HFE guidelines for tasks associated with plant productivity, availability, and protecting investment.

The Staffs Evaluation of Criterion 2 The HFE PP, Section 4.6.1.4 and the V&V IP, Section 4.5.9.2, Selection of HEDs to Correct, both describe a prioritization scheme for HEDs that is used in the evaluation and determination of which HEDs to correct and which can remain as-is. In this scheme, the applicant sorts HEDs into Priority 1, 2, and 3 as follows. Priority 1 HEDs are identified as must be corrected. Priority 2 HEDs are corrected unless the analysis justifies the HED can remain as is. Priority 3 HEDs are acceptable as-is.

166

The HFE PP states that Priority 2 HEDs are defined as those with no direct safety significant consequences but potential safety consequences to plant performance/operability, non-safety personnel performance/efficiency, or other factors affecting overall plant operability. The HFE PP states that Priority 3 HEDs are all those remaining HEDs that are not Priority 1 or 2.

Section 4.6.1.4 of the PP defines Priority 1 HEDs as ...the margin of plant safety may be reduced below an acceptable level.... Section 4.5.9.2 of the V&V IP defines them as ...the margin of plant safety is reduced to below an acceptable level. These two instances are somewhat contradictory. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-82 to address this issue (ML15356A485).

On April 16, 2016, the applicant provided a response to RAI 352-8205, Question 18-82, (ML16107A079). In the response to Question 18-82, the applicant has clarified that The definition of HED priorities in Section 4.5.9.2 of the human factors V&V IP will be revised to match those in Section 4.6.1.4 of the HFE PP. This is consistent and is acceptable. The staff finds that the application conforms to this review criterion.

The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-82. Therefore, RAI 352-8205, Question 18-82 was resolved and closed.

Criterion 3 Development of Design Solutions - The applicant should identify design solutions to correct HEDs. As part of the design solution, the application should evaluate the interrelationships of individual HEDs.

The Staffs Evaluation of Criterion 3 The V&V IP, Section 4.5.9.3 states that the SME team is responsible for the analysis of V&V HEDs. As part of this analysis, the SMEs consult design experts for input on design options.

The SME team as a whole determines the final design option and forwards the recommendation to management per the HFE PP.

The V&V IP, Section 4.5.7.3 states HEDs are evaluated to see if they cut across scenarios and thus represent a cross-cutting issue. The HEDs are also reviewed for commonality by considering the characteristics of the HSIs (e.g., display format, display content, panel layout).

Any issue found to represent a cross-cutting problem is identified as such and added as a separate HED. Per Section 4.5.9.1 the HEDs are analyzed to identify cumulative effects of multiple HEDs and to determine whether an HED indicates a broader problem with the overall HSI system.

The staff finds that the application conforms to this review criterion.

Criterion 4 Design Solution Evaluation - The applicant should evaluate design solutions to demonstrate the resolution of that HED and to ensure that new HEDs are not introduced. Generally, the evaluation should use the V&V method that originally detected the HED.

The Staffs Evaluation of Criterion 4 The V&V IP, Section 4.5.9.4, Design Solution Evaluation, states that HED design solutions are evaluated to demonstrate the resolution of that HED, and to ensure that new HEDs are not 167

introduced. This evaluation will also determine any effects on earlier HFE program elements. If necessary, the specific earlier HFE analysis will be redone. If additional testing is needed, e. g.,

part of the ISV, then that will be repeated. The selected resolution is also assessed for its potential impact on operating procedures and the operator training program. The staff finds that the application conforms to this review criterion.

Criterion 5 HED Evaluation Documentation - The applicant should document each HED, including:

the basis for not correcting an HED related personnel tasks and functions related plant systems cumulative effects of HEDs HEDs as indications of broader issues The Staffs Evaluation of Criterion 5 The V&V IP, Section 4.5.9.5, HED Evaluation Documentation, states that resolution of each HED is documented in the V&V ReSR. The bulleted aspects listed in this criterion are included in the documentation requirements.

The HFE PP, Section 4.6.1.3 states that the resolution process for HEDs includes describing the resolution and related documents/drawings. The criteria for the acceptance of the resolution are included in the ITS.

The HFE PP, Section 4.6.1.4 states that HEDs are closed if they are properly incorporated into the related HSI documents and drawings. An HED may also be closed when the issue has been determined not to require any actions and therefore does not require further tracking.

HEDs that do not require action are also documented in the ITS.

The HFE PP, Section 4.6.1.5 states that HED resolutions requiring design changes are documented in the ITS, and are summarized in both the respective program element of the HFE program and in design documents. Further, the ReSRs of the HFE program include summaries of the identified HEDs and their status. All Priority 1 HEDs and a summary of Priority 2 HEDs are included in the ReSR for the V&V element.

The staff finds that the application conforms to this review criterion.

Combined License Items There are no COL items associated with Section 18.10 of the APR1400 DCD.

Conclusion The staff reviewed the application for human factors V&V using the criteria in NUREG-0711, Revision 3), Section 11, Human Factors Verification and Validation. The review was conducted at the IP level; thus, the focus was on the methodology the applicant will use to conduct the V&V.

168

The results of the V&V program will be documented in a ReSR. The contents of the ReSR are described in DCD Section 18.10.3 and the V&V IP, Section 6. These contents, as described by the applicant, are consistent with the ReSR contents described in NUREG-0711, Section 11.3.

The staff evaluated the applicants V&V method and finds that it conforms to the criteria in NUREG-0711, Section 11.4. Therefore, the staff concludes that the applicants V&V method provides for identifying the control room inventory and for determining there is reasonable assurance the operator tasks can be implemented effectively. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element.

18.11 Design Implementation Introduction The objective of the staffs review is to ensure that the applicants as built design conforms to the verified and validated design that resulted from the HFE design process.

Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400 Design Control Document Tier 1 APR1400-K-X-IT-14001.

Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design.

DCD Tier 2: The applicant provided a Tier 2 system description in Section 18.11, Design Implementation, that summarizes the design implementation process described in APR1400-K-I-NR-14009, Design Implementation Plan, that will be used by the COL applicant.

Changes to Tier 2 information are governed by the change control processes in the design certification rule for the ARP1400 design. Section 14.3.9 of this SER contains the staffs evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs.

ITAAC: The ITAAC associated with this element is listed in DCD Tier 1, Section 2.9, Table 2.9-1. Section 14.3.9 of this SER contain the staffs evaluation of the HFE ITAAC.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: TeRs associated with this element are:

APR1400-K-I-NR-14009, Design Implementation Plan (DI IP) (ML17094A129)

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter 18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows:

10 CFR 52.47(a)(8) 169

10 CFR 50.34(f)(2)(iii)

Other regulatory guidance are as follows:

NUREG-0711, Revision 3, Chapter 12, Design Implementation, Section 12.4, Review Criteria NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.11, Design Implementation Technical Evaluation NUREG-0711, Section 12.4.1, Final HFE Design Verification for New Plants and Control Room Modifications, includes four criteria for this topic.

Criterion 1 The applicant should evaluate aspects of the design that were not addressed in V&V by an appropriate V&V method.

The Staffs Evaluation of Criterion 1 The DI IP, Section 3 lists design aspects that were not verified and validated during the V&V process as one of the items that is evaluated during the design implementation (DI) process.

The overall scope of DI includes the MCR, RSR, TSC, EOF, and LCSs associated with IHAs.

DI IP Section 4 specifically calls out aspects of the design which are not addressed during ISV but that will be evaluated during design implementation. The list includes control room lighting and noise, control room temperature and humidity, and site-specific configurations that were not modeled during ISV. In regard to control room lighting, noise, and temperature and humidity, Section 4.2.3, Condition measurements, provides a process for measurement and verification of these characteristics. In terms of the site-specific configuration, Section 4.2.2, Walkdowns, provides the method used to evaluate the as-built design, whereby the SME compares the as-built configuration or site-specific configuration to the approved final design documentation. The DI IP also describes other V&V methods such as acceptance testing and performance testing to ensure the design criteria have been satisfied.

In Section 4.2.3, the IP has the SME check to ensure the mean value for each condition is within the boundary recommended by the HFE guidelines and approved HSI design documentation. The specific documents to be used were not identified. Therefore, on December 15, 2015, the staff issued RAI 337-8388, Question 18-58 to address this issue (ML15349A920).

On January 12, 2016, the applicant provided a response to RAI 337-8388, Question 18-58 (ML16012A543). The response stated that APR1400-E-I-NR-14012, Style Guide, Section 6.2 describes the boundary of environmental conditions such as temperature, humidity, intensity of illumination, and noise. APR1400-K-I-NR-14009, Design Implementation Plan, will be revised to reference the style guide. This provides for specific acceptance criteria and is therefore acceptable. The staff confirmed that the Design IP was revised as committed in response to RAI 352-8205, Question 18-58. Therefore, RAI 352-8205, Question 18-58 was resolved and closed.

170

The staff concludes that aspects of the design not addressed in the ISV will be appropriately verified and validated. Therefore, the staff finds that the application conforms to this criterion.

Criterion 2 The applicant should compare the final HSIs, procedures, and training with the detailed description of the design to verify that they conform to the planned design resulting from the HFE design process and V&V activities. This verification should compare the actual HSI, procedures, and training materials to design descriptions and documents. Any identified discrepancies should be corrected or justified.

The Staffs Evaluation of Criterion 2 The staff evaluated the applicants process for comparing the final as-built HSIs to the design that is a result of the HFE design process and V&V activities. An evaluation of the applicants process for comparing the final procedures and training to the validated and verified design was not done because the verification of procedures and training is conducted during the inspection of operational programs.

KHNP DCD, Section 18.11.1, Objectives and Scope, describes the high-level objectives and scope for the as-built HSI verification. The DCD states that the objective of the KHNP design implementation is to demonstrate that the as-built HSI configuration is the same as the validated and verified design.

DI IP, Section 4 includes guidance to check that the as-built design matches the validated and verified hardware configuration, software configuration, and facility configuration for the MCR, RSR, TSC, and LCSs. Where there are acceptable differences to the validated and verified design, the SME documents the basis for keeping the as-built configuration. This is done through a design change analysis conducted by an HFE SME which verifies that the change has no impact on human performance. Any unacceptable difference is documented as a HED and addressed in the HED resolution process. After the design verification process is complete, the results are entered into a Final Summary Report.

The staff finds that the applicant has provided a clear methodology for comparing the as-built HSI configuration to the validated and verified design and ensuring any deviations are reconciled. Therefore, the staff finds that the application conforms to this criterion.

Criterion 3 The applicant should verify that all HFE-related issues in the issue-tracking system are adequately addressed.

The Staffs Evaluation of Criterion 3 DI IP, Section 4.3.6, HEDs and HFE-related issues, states that all HEDs are closed, or evaluated, justified, and documented to ensure the final V&V results are unaffected prior to the completion of Design Implementation. Therefore, the staff finds that the application conforms to this criterion.

171

Criterion 4 The applicant should provide a description of how the HFE program addressed each important HA.

The Staffs Evaluation of Criterion 4 APR1400-E-I-NR-14006 describes in detail how IHAs are addressed within the HFE program; specifically, the IP describes how IHAs are selected and validated. Section 18.6 of this SER describes the staffs review of that IP.

Regarding the treatment of IHAs solely in the DI IP, Section 3.6, Important Human Actions, states that all IHAs associated with the MCR and RSR are verified and validated during the ISV.

The IHAs related to the LCSs are verified and evaluated during DI. Section 6 states that the description of how the HFE program addressed delineation of each IHA is included in the ReSR.

Therefore, the staff finds that the application conforms to this criterion.

Combined License Items There are no COL items associated with Section 18.11 of the APR1400 DCD.

Conclusion The staff evaluated the applicants design implementation method and finds that it conforms to the criteria in NUREG-0711, Section 12.4. Therefore, the staff concludes that the applicants design implementation method provides for reasonable assurance that the as-built design conforms to the design resulting from the V&V, and the implementation of plant changes consider the impact on personnel performance and safe operation. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element.

18.12 Human Performance Monitoring Introduction The objective of the staffs review is to assure that the applicant has prepared a human performance monitoring strategy for ensuring that no significant safety degradation occurs because of any changes that are made in the plant and to verify that the conclusions that have been drawn from the human performance evaluation remain valid over the life of the plant.

Summary of Application DCD Tier 1: There is no Tier 1 information associated with this element.

DCD Tier 2: The applicant identified a COL item that will address this element.

ITAAC: There are no ITAAC associated with this element.

TS: There are no TS associated with this element.

TRs: There are no TRs associated with this element.

TeRs: There are no TeRs associated with this element.

172

Regulatory Basis The relevant requirements for the Commissions regulations for this element are described in Section II, Acceptance Criteria, of Chapter18.0, Human Factors Engineering, of NUREG-0800. The applicable regulatory requirements are as follows 10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii)

Other regulatory guidance are as follows:

NUREG-0711, Revision 3, Chapter 13, Human Performance Monitoring, Section 13.4, Review Criteria NUREG-0800, Revision 2, Chapter 18, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering,Section II.A.12, Human Performance Monitoring Technical Evaluation The DCD Tier 2, Section 18.12 contains one COL item pertaining to human performance monitoring. The acceptability of the COL item is evaluated below in this SER section. The staff concluded that no additional COL items were needed.

Combined License Items The DCD Tier 2, Section 18.12 contains one COL item pertaining to human performance monitoring. The acceptability of the COL item is discussed below. The staff concluded that no additional COL items were needed.

Table 18.12-1 Combined License Items Identified in the DCD Item Description Section No.

18.12(1) The COL Applicant is to develop the Human Performance 18.12 Monitoring Program.

Conclusion A COL item has been identified for this element because a human performance monitoring IP was not provided for this HFE element. This is acceptable as the monitoring of human performance, which includes maintaining personnel skills and ensuring no safety degradation from modifications to the design, starts after the plant becomes operational and is therefore a COL activity. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element.

173

References

1. APR1400-E-I-NR-14003, Functional Requirements Analysis and Function Allocation Implementation Plan, (ML18081A107).

2. APR1400-E-I-NR-14001, Human Factors Engineering Program Plan, (ML18212A336).

3. APR1400-E-I-NR-14002, Operating Experience Review Implementation Plan, (ML18081A107).

4. APR1400-E-I-NR-14012, Style Guide, (ML18081A107).

5. APR1400-E-I-NR-14004, Task Analysis Implementation Plan, (ML18178A202).

6. APR1400-E-I-NR-14007, Human-System Interface Design Implementation Plan, (ML18178A202).

7. APR1400-E-I-NR-14011, Basic Human-System Interface, (ML18178A202).

8. APR1400-E-I-NR-14008, Human Factors Verification and Validation Implementation Plan, (ML18178A202).

9. APR1400-E-I-NR-14006, Treatment of Important Human Actions Implementation Plan, (ML18178A202).

10. APR1400-E-I-NR-14010, Human Factors Verification and Validation Scenarios, (ML18081A107).

11. APR1400-K-I-NR-14005, Staffing and Qualifications Implementation Plan, (ML17094A129).

12. APR1400-K-I-NR-14009, Design Implementation Plan, (ML17094A153).

13. APR1400-K-Q-TR-11005, KHNP Quality Assurance Program Description (QAPD) for the APR1400 Design Certification, (ML16123A404).

14. NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition.

15. NUREG-0711, Human Factors Engineering Program Review Model, Revision 3, November 2012 (ML12324A013).

16. RG 1.206, Combined License Applications for Nuclear Power Plants, June 2007 (ML070720184).

17. NUREG-0696, Functional Criteria for Emergency Response Facilities, February 1981 (ML051390358).

174

18. NUREG-0700, Human-System Interface Design Review Guidelines, May 2002 (ML021700373).

19. NUREG/CR-6400, Human Factors Engineering (HFE) Insights for Advanced Reactors Based upon Operating Experience, January 1997 (ML063480112).

20. NUREG/CR-3331, A Methodology for Allocation of Nuclear Power Plant Control Functions to Human and Automated Control, August 1983.

21. NUREG-1122, Knowledge and Abilities Catalog for Nuclear Power Plant Operators:

Pressurized Water Reactors, Revision 2, Supplement 1, October 2007 (ML071580631).

22. SECY-05-0197, Review of Operational Programs in a Combined License Application and Generic Emergency Planning Inspections, Tests, Analyses, and Acceptance Criteria, October 28, 2005 (ML052770225).

23. NUREG-0835, Human Factors Acceptance Criteria for the Safety Parameter Display System, October 1981 (ML102520360).

24. NUREG-1342, A Status Report Regarding Industry Implementation of Safety Parameter Display Systems, April 1989 (ML090060858).

25. NUREG-0737, Clarification of TMI Action Plan Requirements, Supplement 1, January 1983 (ML102560009).

26. RG 1.47, Bypassed and Inoperable Status Indication for Nuclear Power Plant Safety Systems, February 2010 (ML092330064).

27. RG 1.97, Criteria for Accident Monitoring Instrumentation for Nuclear Power Plants, June 2006 (ML061580448).

28. APR1400-Z-J-NR-14012, "Control System CCF Analysis," (ML18212A336).

29. APR1400-Z-A-NR-14019, CCF Coping Analysis, (ML18086B746).

30. APR1400-Z-J-NR-14001, Safety I&C System, (ML18212A336).

175