ML18269A174

APR1400 Chapter 18, Human Factors Engineering, Final Safety Evaluation Report

ML18269A174
Person / Time
Site:	05200046
Issue date:	09/28/2018
From:	Cayetano Santos NRC/NRO/DLSE/LB2
To:
Ward W /NRO/ 415-7038
Shared Package
ML18087A364	List: ML18043A887 ML18044A185 ML18067A292 ML18078A016 ML18081A505 ML18082A298 ML18173A101 ML18176A051 ML18180A327 ML18197A219 ML18199A332 ML18199A333 ML18211A569 ML18212A092 ML18215A239 ML18215A240 ML18215A241 ML18215A242 ML18227A061 ML18261A161 ML18264A141 ML18269A174 ML18269A241 ML18269A242 ML18270A072 ML18270A394 ML18270A452 ML18271A121 ML18271A132
References
Download: ML18269A174 (175)

Text

{{#Wiki_filter:1 18 HUMAN FACTORS ENGINEERING 18.0 Overview review of the human factors engineering (HFE) portion of the Advanced Power Reactor 1400 (APR1400) design certification (DC) application as described in Chapter 18 of the APR1400 with NUREG-the Review of Safety Analysis Reports for Chapter 18, the review used the detailed review criteria in NUREG-Factors Engineering Program Reviesuccessful integration of human characteristics and capabilities into nuclear power plant (NPP) design. These areas of review correspond to the 12 elements of an HFE program identified in NUREG-0711. HFE Program Management Operating Experience Review (OER) Functional Requirements Analysis (FRA) and Function Allocation (FA) Task Analysis (TA) Staffing and Qualifications Treatment of Important Human Actions (TIHA) Human-System Interface (HSI) Design Procedure Development Training Program Development Human Factors Verification and Validation (V&V) Design Implementation (DI) Human Performance Monitoring Power acceptance criteria (DAC) in lieu of detailed design information in a limited number of design areas on a case-by-case basis, as requested by the DC applicants. Korea Hydro and Nuclear Power Co., Ltd. (KHNP) requested to use DAC for the APR1400 HFE design. Thus, the applicant submitted implementation plans (IPs) that describe the proposed methodology a combined license (COL) applicant or holder will follow to complete the elements in NUREG- acceptable results that satisfy the relevant regulations, the staff compared the application to all of the relevant1 review criteria provided in NUREG-0711. The staff has organized Chapter 18 of this safety evaluation report (SER) to align with the 12 elements, and the relevant acceptance criteria are listed2 in each of the sections below. 1 Not all of the criteria in NUREG-0711 are relevant to a design certification application. For example, some criteria are relevant only to licensees who are performing modifications to a control room design at an operating reactor. Those criteria are identified in NUREG-0711, and they are not included in this safety evaluation because they are not in the scope of the review for a design certification application. 2 Some of the criteria in NUREG-criteria or gives examples to support the s-0711. 2 18.1 Human Factors Engineering Program Management Introduction The HFE program management section describes the program for applying human factors to confirm that the applicant has adequately considered the role of HFE and the means by which HFE activities will be accomplished. Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of -K-X-IT-14001. Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2: The applicant provided a Tier 2 description in Section 18.1, identifying the HFE program goals, assumprogram management, which includes the HFE design team and organization; the HFE process and procedures; HFE issues tracking; the HFE technical program; and COL information. See 1400 Design Control Document Tier 2, Chapter 18 Human Factors Engineering, APR1400-K-X-FS- Changes to Tier 2 information are governed by the change control processes in the DC rule for the ARP1400 design. Section 14.3.9 of this SER contains the stainformation in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs. Inspections, Tests, Analyses, and Acceptance Criteria (ITAAC): There are no ITAAC associated with this element. Technical Specifications (TS): There are no TS associated with this element. Topical Reports (TRs): There are no TRs associated with this element. Technical Report (TeRs): TeRs associated with this element are as follows: APR1400-E-I-NR-Engineering Program Management (ADAMS) Accession No. ML18212A336). APR1400-K-Q-TR- (ML18085B039). APR1400-E-I-NR--(HD IP), (ML18178A202). The HD IP describes the method that will be used by the COL applicant to develop the APR1400 HSI Design. -0711. 3 APR1400-E-I-NR--(ML18178A202). The Basic HSI TeR describes the APR1400 Basic HSI conceptual design3. It is an input to the method described in the HD IP. APR1400-E-I-NR-methodology that will be used by the COL applicant to perform the V&V activities, including the integrated system validation (ISV). APR1400-K-I-NR-IP), (ML17094A129). The S&Q IP describes the method that will be used by the COL applicant to perform the staffing analyses. Regulatory Basis NUREG-0800. The applicable regulatory requirements are as follows: Title 10 of the Code of Federal Regulations (10 CFR) 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) Provide, for Commission review, a control room design that reflects state-of-the-art human factor principles prior to committing to the fabrication or revision of fabricated control room panels and layouts Other regulatory guidance documents are as follows: NUREG-Program Review NUREG- of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engineering NUREG- NUREG-- 2 3 gn of the APR1400 main control room, the indicated by double brackets in the text and tables and cloud marks in the figures, and CDI is the portion of the plant for which the application does not seek certification. The Basic HSI TeR and DCD Tier 2, Chapter 18, do not contain double brackets or cloud marks. DCD Tier 2, Table 1.6-2, shows that the Basic HSI TeR is incorporated by reference into DCD Tier 2, and therefore, is Tier 2 information. Additionally, the Basic HSI TeR contains information that is consistent with information contained in DCD Tier 2, Chapter 7. 4 Technical Evaluation 18.1.4.1 General HFE Program Goals and Scope NUREG-for this topic. The seventh criterion addresses plant modifications and is not applicable to new reactors; the staff therefore evaluated the first six criteria as discussed below. Criterion 1 HFE Program Goals The applicant should state the general objectives of the program in -used as a basis for HFE tests and evaluations. The criterion. Section 1 identifies additional goals of the program that go beyond what is necessary to fulfill this criterion. Specific details on how these goals are achieved are described in subsequent sections of the Chapter 18 submittal and in the IPs referenced within the DCD. The example, key human factors principles such as workload, situational awareness, and error reduction are measured, evaluated, and managed throughout the HFE design process to minimize operator error. Incorporation of these principles within the program goals and program IPs ensures the HFE design is centered on maximizing operator effectiveness. Additionally, - Accordingly, the staff finds that the application conforms to this criterion. Criterion 2 Assumptions and Constraints The applicant should identify the design assumptions and constraints. assumptions and constraints of the KHNP HFE design. These include: The plant is assumed to be operated by the crew, the detailed make-up of which is described. The plant is designed to be operated with the minimum staffing level required by 10 CFR 50.54(m)(2)(iii). The human-system interface (HSI) in the main control room (MCR): o will accommodate the operating crew, 5 o is designed based on the APR1400 Basic HSI conceptual design, which the applicant described in the Basic HSI TeR, and o is designed in accordance with the associated HSI Design IP. Additional which identifies the predecessor plant used as the starting point for the HSI design process. s the previously approved DC for the Combustion Engineering System 80+ design as the predecessor design. Section 8 also identifies two specific predecessor plants: Palo Verde Nuclear Generating Station and the KHNP Optimized Power Reactor (OPR) plants. The reference plant, the plant that best represents the APR1400 design, is identified as Shin-Kori (SKN) 3&4 in Korea. -System Interface ted above were used to create the APR1400 Basic HSI conceptual design. This conceptual design is the starting point that will be analyzed and modified as necessary by a COL applicant via the HFE process resulting in a final design, the APR1400 HSI Design, which includes the HSIs as well as the HSI facilities.4 The APR1400 HSI Design will be subject to HFE validation and verification in accordance with the V&V IP. The HFE PP, Section 8 reiterates the specific predecessor design, predecessor plant, and reference plants. While the application consistently identifies predecessor and reference plants, the staff requested more specific information on how these designs were related to and supported the APR1400 design. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-4, to address this issue (ML15177A387). On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-4 (ML15254A492). In this letter, the applicant clarified how specific elements and components of several existing designs (some of which are operating plants) were used to develop the conceptual design of the APR1400. The APR1400 Basic HSI conceptual design relies heavily on the SKN 3&4 design as a starting point for the APR1400 HSI design, and on several other plants/designs to varying extents. The applicant also described the human factors integrated system validation completed for SKN 3&4, which is similar to, although not the same as, the current standard in NUREG-0711. This provides evidence supporting the assumption that SKN 3&4 is a reasonable starting point for the conceptual design. The HFE design process described in the HD IP will then produce improved design iterations on that conceptual design, which are ultimately based on the guidance in NUREG-0711. The RAI response sufficiently clarified the use of multiple predecessor designs and provided sufficient information regarding the use of an iterative human factors process that will ultimately improve on the conceptual design and validate that it can be safely operated. Therefore, RAI 54-7963, Question 18-4, was resolved and closed. The staff concludes that the assumptions and constraints discussed above adequately describe the initial limitations placed on the HFE program. Although the predecessor plants were not validated according to current standards, there is no regulatory or safety basis for them to be. 4 APR1400 HSI Facilities is referred to as the APRand conventional indications, alarms, controls, and operating procedures that encompass the HSI inventory in the task analysis and the APR1400 plant system designs. The APR1400 HSI Facilities include the main control room, remote shutdown room, and the technical support center. 6 The final APR1400 HSI design, if designed according to the presumed approved IPs, will ultimately meet current standards. Accordingly, the staff finds that the application conforms to this criterion. Criterion 3 HFE Program Duration of the design cycle through completion of initial plant startup test program. Criterion 3 The staff finds that the application conforms to this criterion. Criterion 4 Facilities (RSF), technical support center (TSC), emergency operations facility (EOF), and local control stations (LCS). The 12 HFE elements should be applied to each of them, unless otherwise noted for a specific HFE element. However, applicants may apply the elements of the HFE program in a graded fashion to facilities other than the MCR and RSF, providing justification in the HFE program plan. he facilities listed in this criterion. All twelve HFE elements are included within the scope for the MCR and remote shutdown facility (called the remote shutdown room or RSR). Additionally, the staff found a description of a remote control center (RCC) in DCD Tier 2, Section 7.7.1.1.o.3.c, standby plant condition and is provided to meet the requirements in 10 CFR 50.150. The application described the RCC as an alternative to the MCR, and thus it is akin to the RSR. Therefore, the design of the RCC should conform to the design method described in NUREG-0711 in the same way that the RSR conforms to this method. However, the staff did not find any description of the RCC in DCD Tier 2, Chapter 18. Therefore, on July 1, 2016, the staff issued RAI 502-8647, Question 07.07-17 to address this issue (ML16188A169). On September 28, 2016, the applicant provided a response to RAI 502-8647, Question 07.07- 17 (ML16272A480). As part of the response, the applicant stated the design of the RCC conforms to the design method described in NUREG-0711 in the same way that the RSR conforms to this method. Additionally, the applicant provided revisions to the DCD to include the RCC in the scope of the HFE program. As part of the response, the applicant also included address an issue identified by staff reviewing DCD Tier 2, Section 19.5, Aircraft Impact to RAI 502-8647, Question 07.07-17, to align with the information that will be revised in DCD Tier 2, Section 7.7.1.1.o.3.c. On February 10, 2017, the applicant submitted a revised response to RAI 502-8647, Question 07.07-17 (ML17041A197), and proposed changes to DCD Tier 2, Section 7.7.1.1.o.3 such that the RAI response aligns with the information in the DCD. [ ] 7 Based on the review of the DCD and HD IP, the staff has confirmed incorporation of the changes described above; therefore RAI 502-8647, Question 07.07-17 was resolved and closed. A graded approach is used is used for the remaining areas: TSC, EOF, and LCS. The HFE design of the EOF applies to the communication and information requirements only. DCD process to design those aspects of the design associated with important human actions. The staff did not find justification for the use of a graded HFE program as suggested by the acceptance criterion. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-5 to address this issue (ML15177A387). On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-5 (ML15254A492). The applicant added additional detail clarifying how the review is graded in areas other than the MCR and remote shutdown room. The applicant will use NUREG-0696 and NUREG-0700 in the design of the EOF instead of implementing the entire NUREG-0711 NUREG-0696 includes general HFE criteria for emergency response facilities such as the EOF, and these criteria are an acceptable an alternative to the NUREG-0711 criteria. The applicant also listed particular HSIs that will be the same in both the MCR and the EOF; therefore, these particular systems will be designed using the complete NUREG-0711 process. The complete NUREG-0711 process will be utilized for LCSs that are used for important human actions. However, other LCSs that are not used for completion of important human actions, and therefore have little potential safety impact, will be designed using NUREG-0700 instead of the full NUREG-0711 process. This treatment applies the same rigorous standard as the MCR to those LCSs with potential for safety impacts. Therefore, RAI 54-7963, Question 18-5 was resolved and closed. -System Interfaces, Procedures, and procedure and training development. This applies to operations, emergency response, maintenance, test, inspection surveillance interfaces and procedures. The applicant described how the human factors process will be applied to the MCR, RSR designs and to other areas around the plant. The applicant described a systematic approach that appropriately grades the design work in a way that focuses design effort on those areas that are most likely to have a safety impact. The applicant described appropriate design processes for those areas that have less of an impact on safety using appropriate alternate standards. Therefore, the staff finds that the application conforms to this criterion. Criterion 5 HSIs, Procedures and Training design of HSIs and identify inputs to the development of procedures and training for all operations, accident management, maintenance, test, inspections, and surveillance tasks that operational personnel will perform or supervise. In addition, the HFE design process should identify training program input for the following personnel identified in 10 CFR 50.120: instrument and control technician, electrical maintenance personnel, mechanical maintenance personnel, radiological protection technician, chemistry technician, and engineering support personnel. In addition, any other personnel who perform tasks directly related to plant safety should be included, such as 8 information technology technicians who troubleshoot and maintain support systems and their HSIs. The HFE PP, Section 2.1 provides a high-level summary that closely resembles the criteria description above. This section addresses the design of HSI and provides inputs to training and procedure development consistent with the criterion. the training and procedure development processes are supported by the HFE program but are describ-3, re Development and Training Programs relative to the HFE program elements described in Chapter 18. The applicant used the HSI design process to inform the development of procedures and training programs. The training input is applied to the specific categories of plant staff identified in 10 CFR 50.120. Accordingly, the staff finds the application conforms to this criterion. Criterion 6 Personnel qualifications, including licensed control-room operators as defined in 10 CFR Part 55, and the following categories of personnel: non-licensed operators, shift supervisor, and shift technical advisor. personnel addressed in the criterion as well as non-licensed operators. It specifies that other plant personnel performing tasks that are directly related to plant safety will also be included in the HFE program staffing considerations. The S&Q IP contains specific details about how staffing and qualifications will be planned and executed. The applicant includes the personnel described in this criterion in its HFE program thus providing assurance that the final design will ultimately support the varied needs of different users; therefore, the staff finds that the application conforms to this criterion. 18.1.4.2 Human Factors Engineering Team and Organization NUREG-iteria for this topic. Criterion 1 Responsibility developing all HFE plans and procedures; overseeing and reviewing all activities in HFE design, development, test, and evaluation, including the initiation, recommendation, and provision of solutions 9 through designated channels for problems identified in implementing the HFE work; implemented; assuring that all HFE activities comply with the HFE plans and procedures; and scheduling work and milestones. 1 multidisciplinary HFE design team that consists of three sub-groups: the Architect Engineering Group (A/E), the nuclear steam supply system (NSSS) group, and the operating group. The Factors Engineering Design Team Responsibilities.with those listed in the criterion. Additional supporting information is found in Table 4-application conforms to this criterion. Criterion 2 Organizational Placement and Authority The applicant should describe the primary HFE organization(s) or function(s) within the engineering organization designing the plant or modification. The organization should be illustrated to show organizational and functional relationships, reporting relationships, and lines of communication. The applicant also should address the following: When more than one organization is responsible for HFE [ ], the lead organizational unit answerable for the HFE program plan should be identified. If organization changes are expected over time (e.g., from design through construction to startup) necessary transitions between responsible organizations should be described. The team should have the authority and organizational placement to reasonably assure that all its areas of responsibility are completed, and to identify problems in establishing the overall plan or modifying its design. The team should have the authority to control further processing, delivery, installation, or use of HFE products until the disposition of a nonconformance, deficiency, or unsatisfactory condition is resolved. key roles in the HFE design team. Included are the project manager, the quality assurance organization, the HFE design team leader, the HFE coordinator, the A/E, the operating group, the NSSS design group, and the HFE engineers. Figure 18.1-1, roles and others involved in the design and review process. 10 The HFE Design Team leader is responsible for oversight of HFE related activities and reports to the Project Manager. The HFE design team has design authority to manage HFE related decisions and is placed in the organization in a position that is equal to other design groups. The engineering groups report through the HFE design team leader ensuring that HFE issues can be addressed appropriately. Additional proprietary information regarding specific details of The HFE PP, Section 4.2 indicates that the applicant will use a [ ] This transition assures that information and issues identified during the design process are accessible to the COL applicant who will ultimately be responsible for following up and resolving them. The HFE design team has the authority to identify issues, approve resolutions, and approve the final HSI rough the end of the human factors validation and verification process. Table 4-1 identifies the engineering groups that perform HFE Verification and Validation (V&V) and Design Implementation activities. Therefore, the design team has the authority to stop work if a nonconformance, deficiency, or unsatisfactory condition is identified during the delivery and installation phases of construction in accordance with the final bullet of this criterion. The staff concludes that the IP describes the management organization associated with HFE activities and delineates management and HFE team responsibilities. These responsibilities are ensure that the responsibilities are accomplished. This includes the authority to identify non- conformances and deficiencies as well as providing a structure to ensure that various parts of the team are responsible for identifying acceptable resolutions. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 3 Composition - Appendix to NUREG-0711. included on the HFE design team. [ ] This combination of matrixed and dedicated staff personnel is a reasonable organizational structure because individuals that possess the specific skills may not be available. It is consistent with the NUREG-0711 guidance because the professional experience is satisfied by the HFE design team as a collective whole rather than on an individual basis. This is consistent with interdisciplinary teams, which are typically preferred in an HFE design process. Section 5 also describes how [ ] This assures that the results of each HFE process are auditable as described in Appendix A of NUREG-0711. 11 The qualifications, as described in Section 5, are equivalent to Appendix A of NUREG-0711 with some additional positions/qualifications added. The added roles are not likely to interfere with the roles prescribed by the appendix. Accordingly, the staff finds that the application conforms to this criterion. Criterion 4 Team Staffing The applicant should describe team staffing in terms of job descriptions and assignments of team personnel. describes the responsibilities of the HFE design team leader as well as the various engineering groups. Table 4-1 provides information regarding the responsibilities of the three specific design teams that provide input to the HFE design team leader. the larger design process. This section also explains how personnel are used to implement the provides additional supplemental information. s the qualifications associated with various roles on the HFE design team and provides additional information about how these staff, with various specific qualifications, will be assigned to various teams. While specific personnel assignments are not provided, the descriptions of responsibilities for the supervisors and various design teams provide sufficient explanation of how personnel will be used and is therefore an acceptable substitute for job descriptions and personnel assignments. Specific personnel assignments are not expected nor required by the criterion. Accordingly, the staff finds that the application conforms to this criterion. 18.1.4.3 Human Factors Engineering Process and Procedures NUREG- six criteria for this topic. Criterion 1 General Process Procedures - The applicant should identify the process through which the team will execute its responsibilities. It should include procedures for the following: assigning HFE activities to individual team members governing the internal management of the team making decisions on managing the HFE program making HFE design decisions controlling changes in design of equipment 12 reviewing HFE products The 1 the HFE design team will carry out its responsibilities. These descriptions include how to assign activities, processes for making HFE design decisions, and controlling design changes. The descriptions are consistent with this criterion. The description of how the applicant plans to manage the HFE team and processes for reviewing HFE program results incorporates by reference the Project Procedure Manual (PPM). The applicant changed the name of the PPM to the Engineering Procedures Manual (EPM) after the initial license application was submitted. This caused inconsistencies in the terminology used in several submitted documents. Therefore, on September 2, 2015, the staff issued RAI 196-8164, Question 18-25 to address this issue (ML15254A494). On November 25, 2015, the applicant submitted clarifying information in response to RAI 196-8164, Question 18-25 (ML15329A383). The response indicated that all references to the PPM in the HFE PP and in the DCD Tier 2 will be removed and replaced with the correct references to the EPM. These corrections, when completed, should make the docketed information internally consistent and support future audits/inspections. The proposed changes will not negatively affect the acceptability of the submitted material with respect to this criterion. Based on the review of the DCD and HFE PP, the staff has confirmed incorporation of the changes described above; therefore, RAI 196-8164, Question 18-25 was resolved and closed. The EPM is managed under the Quality Assurance Program. Design decisions, design changes, and review of HFE program results will be conducted in accordance with the Quality Assurance Program as described in APR1400-K-Q-TR-11005. The staff concludes that working level procedures are available to communicate work control responsibilities that address the activities listed in this criterion. Accordingly, the staff finds that the application conforms to this criterion. Criterion 2 Process Management Tools - The applicant should identify the tools and techniques (e.g., review forms) the team uses to verify that they fulfill their responsibilities. The management tools designed to support communication and efficiency in the design process. The Review and Comment system is one tool which allows designers and independent reviewers to document opinions and issues regarding design documents. Another tool is the Issue Tracking System (ITS) which is used to formally document, track, resolve, and close human engineering discrepancies (HED). The staff did not understand how the Review and Comment system was used by the HFE team to fulfill their responsibilities. Therefore, on September 2, 2015, the staff issued RAI 196-8164, Questions 18-24 and 26 to address this issue (ML15254A494). Additionally, the staff did not understand how and when each of these tools are used. Therefore, on February 1, 2016, the staff issued RAI 360-8452, Question 18-90 to address this issue (ML16011A248). 13 On November 25, 2015, the applicant submitted clarifying information in response to RAI 196-8164, Questions No. 18-24 and 26 (ML15329A386). The response explained how the applicant intended to use the Review and Comment System by describing how comments are entered, reviewed, resolved, and stored within the system. The applicant indicated that it will update the HFE PP to better reflect this information and provided a suitable marked-up draft for review. The November 25, 2015, RAI response also clarified that the Review and Comment System, ITS, and Corrective Action Program are independent systems. On May 19, 2016, the applicant provided additional clarification about how these systems remain independent in response to RAI 360-8452, Question 18-90 (ML16141A768). This submittal provides detailed comparisons of the Review and Comment System, ITS, and Corrective Action Program. The separation of scope, timelines, and responsible parties provides reasonable assurance that issues will be entered into the appropriate tracking system, thus ensuring the independence of each system. Therefore, RAI 360-8452, Question 18-90 was resolved and closed. The applicant submitted a marked-up revision to the HFE PP to include a description of a process for ensuring that workers are properly qualified for various design jobs in the November 25, 2015, response to RAI 196-8164, Questions 18-24 and 26. This additional information provides assurance that the processes will be implemented as intended by ensuring that only qualified personnel are involved. The staff confirmed that HFE PP was revised as committed in the responses to RAI 196-8164, Questions 18-24 and 18-26. Therefore, RAI 196- 8164, Questions 18-24 and 18-26 were resolved and closed. Section 4.3.3 indicates that [ The staff has reviewed the tools used to identify, document, and resolve issues during various stages of design and implementation. The applicant has identified the tools used and clarified the processes that will be used to apply these tools. Each of these systems helps track issues to make sure they are resolved appropriately. In addition, the applicant used technological safeguards in addition to an administrative process that helps ensure that only qualified personnel conduct resolution activities. Therefore, the staff finds that the application conforms to this criterion. Criterion 3 Integration of HFE and Other Plant or Modification Design Activities The applicant should describe the process for integrating the design activities (i.e., the inputs from other design work to the HFE program, and the outputs from the HFE program to other plant design activities). The applicant should also discuss the iterative aspects of the HFE design process. The HFE PP, Section 4.4.3design. Integration is controlled by the Quality Assurance Program as described in APR1400-K-Q-TR-11005 which contains high-level descriptions of how the applicant intends to ]. This section also indicates that the HFE design team leader is responsible for [ ] 14 perform design control (Section 3) and design verification (Section 3.1), including for the HFE design. The HFE process receives inputs from other plant design areas. Results Summary Reports (ReSR) generated as part of the HFE process are provided to other design areas upon completion. The HFE design team works with the other design areas to assess and implement design changes. A formalized report describing design changes is sent to the HFE design team for subsequent approval. The HFE design team leader is responsible for documenting integration activities through a series of processes and reports. Summaries of the specific inputs and outputs from other HFE program elements can be found in the HFE PP, Section describes the use of various process management tools to further integrate the HFE process. The process lists several integration related activities that are the responsibility of the HFE design team leader to implement. Section 18.1.2.2 describes the role of the HFE coordinator. It includes responsibilities of coordinating designers in the A/E, operations group and NSSS groups for resolution of HED. DCD Tier 2, Figure 18.1-Results of HFE elements that occur later in the design process are fed back into iterations on the earlier HFE element processes. These issues are ultimately addressed via HED resolution and/or HFE V&V process. The applicant described an iterative process that is well integrated both within the HFE program as well as within the more inclusive plant design program. The HFE program inputs and outputs are summarized in the HFE PP and additional detail is described in subsequent IPs. The staff finds that the application conforms to this criterion. Criterion 4 HFE Program Milestones The applicant should identify HFE milestones that show the relationship of the elements of the HFE program to the integrated plant design, development, and licensing schedule. A relative program schedule of HFE tasks should be available for the elements and the activities, products, and reviews. n Factors Engineering Program Milestones, Schedule, and -expected milestones for the HFE elements. This diagram anchors the HFE work within appropriate licensing and operating schedules. Accordingly, the staff finds that the application conforms to this criterion. Criterion 5 HFE Documentation The applicant should identify the HFE documentation items, such as ReSRs and their supporting materials, and briefly describe them, along with the procedures for their retention and for making them available to the staff for review. 15 c written deliverables used as part of the HFE process including the ReSRs, HFE program IPs, and several other HFE documents. Figure 4-2 anchors various documents to important project milestones, as described in Criterion 4 above. The Review and Comment System, also described in Section 4.4.4, is used to retain all of the documents described and to control access to these documents. These documents will be retained for the life of the program. The quality assurance program described in APR1400-K-Q- TR-11005, Section 3.2 provides supplemental information regarding the creation and retention of design records. NRC review. The applicant described the expected contents of each ReSR in Section 6 of the corresponding IPs. However, the plan for submitting the various ReSRs to the NRC is unclear. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-6 to address this issue (ML15177A387). On August 18, 2015, the applicant provided a response to RAI 54-7963, Question 18-6 (ML15230A528). This letter indicated that no ReSR will be complete during the DC process. These reports will be the responsibility of the COL applicant. A rough expected schedule for completion is provided, however, it is acknowledged that this schedule is likely to change because KHNP has no control over the potential COL applicant schedules. Audits of ReSRs and inspections of the simulator and HFE V&V process can be scheduled with the COL applicant as needed. Therefore, RAI 54-7963, Question 18-6 was resolved and closed. The applicant adequately described a method for documenting the HFE process. Therefore, the staff finds that the application conforms to this criterion. Criterion 6 Subcontractor HFE Efforts The applicant should include HFE requirements in each subcontract contributing to the HFE program. The applicant should periodically verify the subcontractor's compliance with HFE requirements. The HFE plan should describe milestones and the methods used for this verification. and verify subcontractor efforts. Procurement specifications that contain the necessary HFE requirements as well as the HFE Style Guide are sent to subcontractors. The applicant indicates that HFE requirements will be included in subcontractor procurement documents and describes how the Quality Assurance Program will be used to verify compliance. Therefore, the staff finds that the application conforms to this criterion. 18.1.4.4 Human Factors Engineering Issues Tracking NUREG-16 Criterion 1 Availability The applicant should have a tracking system to address human factors issues that are: known to the industry; identified throughout the life cycle of the HFE aspects of design, development, and evaluation; and deemed by the HFE program as HEDs. 1 The tracking system will be used to document known industry issues, human factors issues, and HEDs identified throughout the execution of the KHNP HFE program elements. The HFE PP, Section 4.6 specifies the roles associated with the HFE design team leader with respect to the system. DCD Tier 2, Section 1HFE team is responsible for implementing the logging, tracking, and resolution processes. Additional information about the use of ITS by other HFE design team members is addressed throughout the subsections of Section 4.6. industry issues as well as issues and HEDs identified during the design process. Therefore, the staff finds that the application conforms to this criterion. Criterion 2 Method establish criteria for when issues are entered into the system and track issues until the potential for negative effects on human performance is reduced to an acceptable level. The -level criteria for entry of HEDs into the ITS. It explains that HEDs can be identified during any human factors process subject to the conditions indicated in the associated IP. Entries will be made with sufficient identifying information so that they can be easily tracked. A diagram of the ITS is shown in Figure 4-4. ds used to ensure that the ITS is accessed by authorized personnel only and is not modified without authorization from the HFE design team leader. Section 4.3.1 specifies the responsibilities of the design team with respect to HED entry and tracking. 17 effects on performance, and closure of HEDs when it is determined resolution of HEDs is NUREG-0711 criterion in that the synergistic effect of multiple low priority HEDs should be accounted for. Therefore, on November 9, 2015, the staff issued RAI 298-8356, Question 18-41 to address this issue (ML15314A023). On April 22, 2016, the applicant provided supplemental information in response to RAI 298- 8356 Question 18-41 (ML16113A441). The applicant clarified that the cumulative effects of HEDs are considered during the HFE V&V process. The staff confirmed that HFE PP and V&V IP were revised as committed in the response to RAI 298-8356, Question 18-41. Therefore, RAI 298-8356, Question 18-41 was resolved and closed. The applicant described the criteria used for entering issues into the ITS and described how issues will be tracked, assessed, and subsequently closed out. Therefore, the staff finds that the application conforms to this criterion. Criterion 3 Documentation The applicant should document the actions taken to address each issue in the system; if no action is required, this should be justified. on 3 design changes will be documented in the ITS and will be summarized in the respective HFE element as well as in the appropriate design documents/change orders. HEDs will also be documented in the ReSR for the respective element of the HFE program in which it was identified. Resolutions to Priority 1 HEDs and summaries of Priority 2 HEDs will be included in the ReSR for the human factors V&V element. The applicant also addressed closure of HEDs not requiring future tracking or action in documentation as well as approval from the HFE design team leader. They are maintained in the ITS. However, the description provided in this section does not explain how Priority 3 HEDs are justified and how this justification is documented. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-8 to address this issue (ML15177A387). On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-8 (ML15254A492), and on April 22, 2016, provided supplemental information in RAI 298-8356, Question 18-41. These submittals revised the definition of Priority 3 HEDs to be consistent with the criterion and with the intent of NUREG-0711 Section 11.4.4, Subsection (1). The applicant clarified that the cumulative effects of HEDs is considered during the human factors V&V process. The staff confirmed that HFE PP was revised as committed in the response to RAI 54- 7963, Question 18-8. Therefore, RAI 54-7963, Question 18-8 was resolved and closed. The applicant described a process that documents and tracks HEDs throughout the design process through resolution. The process also included justification for any HEDs that do not require additional actions. Therefore, the staff finds that the application conforms to this criterion. 18 Criterion 4 Responsibility procedures should describe individual responsibilities for logging, tracking, and resolving it, along with the acceptance of the outcome. DCD Tier 2, Section 18.1.4 identifies the HFE design team as the responsible party for logging, -indicates that the HFE design team leader is responsible for approving HED entries, resolutions, and closeout of ITS entries. Section 4.6.1.4 indicates that HEDs can be closed when appropriate changes to HSI documents are complete and have been approved by the HFE design team leader. Alternatively, HEDs that do not cause changes to the design can be closed when a justification is documented in the ITS. DCD Tier 2, Section 18.1.4, indicates that thresholds for HED tracking are identified in individual IPs. The staff reviewed a sample of the IPs and HED entry criteria were confirmed to be found in the IPs as described by the applicant. The applicant clearly defined the responsibilities related to logging, tracking, resolving, and accepting HEDs. Therefore, the staff finds that the application conforms to this criterion. 18.1.4.5 Technical Program NUREG-criterion addresses plant modifications and is not applicable to new reactors; therefore, the staff evaluated the first four criteria as discussed below Criterion 1 The applicant should describe the applicability and status of each of the following HFE elements: Operating Experience Review (OER) Functional Requirements Analysis (FRA) and Function Allocation Task Analysis Staffing and Qualifications Treatment of Important Human Actions HSI Design Procedure Development (Described in SRP, Chapter 13 submittal) Training Development (Described in SRP, Chapter 13 submittal) Human Factors V&V 19 Design Implementation Human Performance Monitoring 1 incorporates all of the eleven identified NUREG-0711 elements listed above. Human Performance Monitoring is identified as the responsibility of the COL applicant, and Training and Procedure Development are addressed in the Chapter 13 submittal. Section 4.interfaces and the inputs and outputs to each program element. It also includes a description of the final work products. Figure 4-3 provides a functional block diagram illustrating the work flow between elements. The applicant appropriately applied all of the listed human factors program elements and described how they are functionally related. Therefore, the staff finds that the application conforms to this criterion. Criterion 2 The applicant should identify the approximate schedule for completing any HFE activities that are unfinished at the time of the application. Section 4.4.1, Figure 4-2 provides a schedule that benchmarks various HFE submittals between the DC application and operation. The elements are relative to each other and do not include approximate dates for submittals (specific dates are not required by the criterion as they are difficult to identify and under the control of COL applicants). The relative schedule shown in Figure 4-2 shows the best estimate for a COL schedule possible at this time without a COL application pending. Therefore, the staff finds that the application conforms to this criterion. Criterion 3 The sources of the HFE requirements. extensive listing of nuclear industry documents that encompass codes and standards, NRC documents, and other industry documents such as those from the Institute of Electrical and Electronics Engineers (IEEE), and American National Standards Institute/American Nuclear Society (ANSI/ANS). These documents are consistent with those commonly used as sources for HFE design. Therefore, the staff finds that the application conforms to this criterion. 20 Criterion 4 facilities, equipment, tools, and techniques (such as laboratories, simulators, rapid prototyping software) that the HFE program will employ. The HFE PP, Section 4.7.1 indicates that a full-scope MCR simulator, part-task simulator, mock- ups, as well as special tools and equipment will be used to support HFE design work. DCD Tier design team meetings as a process management tool to facilitate coordination between design groups. The staff did not understand how the process or processes would use these tools and others described in the IP to create the final design. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-10 to address this issue (ML15177A387). On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-10 (ML15254A492). This response clarified that the APR1400 basic HSI (based on SKN 3&4, and other precedent designs) will be used to develop the full-scope simulator prototype. The detailed functional design specifications are developed in accordance with the HD IP and incorporated into the resulting APR1400 full-scope dynamic simulator developed to meet ANSI/ANS-3.5-ear Power Plant Simulators for Use in Operator Training and response sufficiently clarified ambiguities in the application; therefore, RAI 54-7963, Question 18-10 was resolved and closed. The applicant provided a logical and systematic process that utilized a simulator based on the precedent design as a basis to develop a high-fidelity simulator that will ultimately be used to validate the MCR design. Therefore, the staff finds that the application conforms to this criterion. Combined License Items There are no COL items associated with Section 18.1 of the APR1400 DCD. Conclusion management and finds that it conforms to the criteria in NUREG-0711, Section 2.4. Therefore, the staff concludes that the team and member qualifications, identifies HFE processes and procedures, describes methods for tracking HFE issues, and provides an overview of how each of the HFE program elements will be addressed for the APR1400. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element. 18.2 Operating Experience Review Introduction The objective of this review is to verify that the applicant has identified and analyzed HFE related problems and issues encountered in previous designs so that these problems and issues may be avoided in the development of the new design. This review should also verify that the applicant has retained positive features of previous designs. This is done through an 21 evaluation of licensee event reports; Institute of Nuclear Power Operations significant event reports and significant operating experience reports; plant corrective action systems; operational and maintenance logs and records; and data from interviews with experienced plant personnel. Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of -K-X-IT-14001. Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2: The applicant provided a Tier 2 description in Section 18.2 of the DCD which summarizes the method described in APR1400-E-I-NR-perience Review experience review program element. Changes to Tier 2 information are governed by the change control processes in the design certification rule for the Aevaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE implementation plans. ITAAC: There are no ITAAC associated with this element. TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: TeRs associated with this element are: APR1400-E-I-NR- (OER IP), (ML18081A107). APR1400-E-I-NR- (ML18212A336). Regulatory Basis - 0800. The applicable regulatory requirements are as follows: 10 CFR 50.34(f)(3)(i) 10 CFR 50.34(f)(2)(iii) 10 CFR 52.47(a)(8) Other regulatory guidance documents are as follows: NUREG-0711, 22 NUREG-Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Technical Evaluation 18.2.4.1 Scope NUREG-Criterion 1 Predecessor/Related Plants and Systems The applishould include information about human factors issues in the predecessor plant(s) or highly similar plants, systems, and HSIs, including the following: The OER should identify previous or predecessor design(s)/plant(s) used as part of the design basis of the plant being reviewed. The OER should define the relevance of each predecessor plant/design to the new design, when there is more than one predecessor. The OER should detail how the applicant identified and analyzed any HFE- related problems in the previous plants/designs, and how these issues are avoided in the new design. The OER should address how the applicant identified, evaluated, and incorporated or retained any positive features of previous plants/designs. The OER should describe the predecessor plant(s) and systems, explaining the relationship of each to the new design. For applicants proposing to use new technology or systems that were not used in the predecessor plants, the OER should review and describe the operating experience of any other facilities that already use that technology. The OER IP, Section 1 states that the objective of the OER is to identify, evaluate, track, and incorporate HFE-related operating experience (OE) encountered in previous nuclear plant designs and OE sources so that the negative features are not repeated, and the positive features are retained. Performance information from predecessor designs is identified at the start of the design process and used to improve the plant design. Currently operating U.S. pressurized water reactors (PWRs) All Korean plants that are operating, or have operated, which includes SKN (SKN) 1, 2, 3 and 4 U.S. OE databases, NUREGs, and vendor groups 23 SKN 3&4 operator and instructor interviews Non-nuclear applications Chemical, Transportation, Electrical Goplant, and reference plant in the HFE PP, Section 8, and also described the relationship of these plants to the APR1400 HSI Design. The starting point for the APR1400 HSI Design is the APR1400 Basic HSI conceptual design, which is based on SKN 3&4. -describe the sources of OE. screening process. A number of questions are provided that the OER reviewer answers to determine if the OE identifies any HFE-related problems to be avoided in the design or positive features to be retained. The OEs are then grouped, classified, and analyzed for lessons learned. Interviews with operators from SKN 3&4 are used to identify and retain positives features. Any lesson learned that affects another HFE program element is entered into the Issue Tracking System (ITS) as a human engineering discrepancy (HED) where they can be analyzed by other program element reviewers and be tracked to completion. All OEs are entered into the OE Database. The applicant has provided an OER process which identified operating experience from the predecessor plants and operating U.S. plants as input to the APR1400 design, designates the sources for this material and the types of reports to be evaluated, and provides a means of tracking cross-cutting OEs. Therefore, the staff finds that the application conforms to this criterion. Criterion 2 Recognized Industry HFE Issues The applicant should address the HFE issues identified in NUREG/CR-6400. The issues are organized into the following categories: unresolved safety issues/generic safety issues (see 10 CFR 52.47(a)(21) and NUREG-0933) Three Mile Island (TMI) issues NRC generic letters and information notices operating experience reports in the NUREG-1275 series, Volume 1 through 14 low power and shut down operations operating plant event reports 24 Additionally, the applicant should review and discuss all operating experience in the preceding categories that was published since NUREG/CR-Insights for Advanced Reactors B NUREG/CR-6400 as one of the sources from which operating experience is collected. Section the staff noted that the sources for recognized industry HFE issues, other than NUREG/CR- 6400, listed in the OER IP were not detailed enough to enable the staff to verify the depth of the OER review. Therefore, on June 26, 2015, in RAI 53-7982, Question 18-2, the staff asked the applicant to provide greater specificity and more details in this area (ML15177A386). Similarly, in Question 18-3, the staff asked the applicant to be specific about the SKN 3&4 OEs that were used in APR1400 design and whether or not, the SKN 3&4 OER followed the guidelines in NUREG-0711. -7982, Questions 18-2 and 18-3 (ML15247A164), the OER IP augments the operating experience list with other sources, such as an extensive list of NUREGs (e.g., 0696, 0737, and 0933), Westinghouse and Combustion Engineering vendor groups, and accident and failure records from the Institute for Nuclear Power Operations. Additionally, the applicant stated that OE that was published after 1996 is encompassed in the SKN 3&4 OER, which is the basis for the APR1400 OER. The SKN 3&4 OER is supplemented with a review of everything after that review was done with the process described in the IP. However, the staff noted that although the applicant, in its RAI response, had provided details that were satisfactory, it did not commit to updating the OER IP to include the additional details provided in the response. Therefore, on August 3, 2016, the staff asked the applicant in an email (ML17116A260) to revise the OER IP to incorporate the additional details provided in the RAI response. To address this issue, the applicant agreed to revise the OER IP to include the details provided in the RAI 53-7982 response dated September 4, 2015. Accordingly, on August 25, 2016, the applicant provided a revised response (ML16238A384) to the RAI which also included a revision to the OER IP. The staff reviewed the proposed changes to the OER IP and found the revision incorporates the additional information in the initial response to RAI 53-7982, Questions 18-2 and 18-3. Based on the review of the OER IP, the staff has confirmed incorporation of the changes described above; therefore RAI 53-7982, Questions 18-2 and 18-3 was resolved and closed. During the APR1400 Subcommittee Meeting of the Advisory Committee on Reactor Safeguards (ACRS) held on June 21, 2017 (ML17200A091), an ACRS member questioned whether the staff audited the results of the SKN 3&4 OER to confirm that relevant OE that occurred before the close date of the SKN 3&4 OER had indeed been adequately addressed in the SKN 3&4 OER. The ACRS member expressed concern that relevant OE, such as OE from operating plant event reports from U.S. plants, could be excluded from the OER performed by the COL applicant. Following the ACRS meeting, the staff re-determine whether it transpired before or after the close date of the SKN 3&4 OER. OEs with 25 dates before the SKN 3&4 close date are assumed to be included in the SKN 3&4 OER and are staff could not confirm that all relevant operating experience, including events related to the categories listed in this review criterion, were adequately addressed in the SKN 3&4 OER. As such, the staff determined it was possible that relevant OE could be excluded from the OER conducted by the COL applicant. Therefore, on September 6, 2017, the staff issued RAI 553- 9084, Question 18-136 to address this issue (ML17249A979). On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-136 te that OEs that occurred before the SKN 3&4 close date will first be evaluated to determine whether they were included in the SKN 3&4 OER. If they were included in the SKN 3&4 OER, then they may be screened out only if the lessons learned were identified and determined to be adequately addressed using the guidance in NUREG- response. Because the results of the SKN 3&4 OER will first be reviewed to determine whether a lesson learned from a particular operating experience event has been addressed, and if not, will be included in the OER, the staff concludes that the proposed revisions to the OER IP ensure that relevant OE are not excluded from the OER. Additionally, during the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member questioned why the OER IP states that OEs will be grouped according to the categories in NUREG/CR-6400. The categories listed in NUREG/CR-6400 identify various types of events and lessons learned that, at a minimum, should be included in categories provided insights during the OER process because grouping OEs by other characteristics, such as common causes of operating events, would provide insight into the reasons why OE events occurred and could help identify adequate design solutions. Following the ACRS meeting, the staff requested the applicant address this question as part of RAI 553- 9084, Question 18--9084, Question 18-136 stated -6400 provides expanded HFE design issue categories and proposed resolutions. OE grouping following this issue category helps designer to clarify his/her OE- in NUREG/CR-6400 may not provide as much insight as to the human errors that contributed to plant events as grouping the OE into other categories (e.g., related causes), however, the concept of grouping OE is a good practice. Further, there is no requirement that applicants acceptable. Based on the review of the OER IP, the staff has confirmed incorporation of the changes described above; therefore RAI 553-9084, Question 18-136 was resolved and closed. Therefore, the staff finds that the application conforms to this criterion. Criterion 3 Related HSI Technology -states that non- nuclear industry experience related to digital screen-based HSI technology will be evaluated. 26 The non-nuclear industry scope includes the chemical, transportation, and electrical transmission industries. This additional scope is added due to the limited nuclear industry experience with the technologies used in the KHNP design. The applicant outlined a detailed plan for assessing the operating experience for digital screen- based technologies which includes relevant experience outside the nuclear industry. By including non-nuclear experience, the applicant has maximized the potential for applying lessons learned on new technology applications within the KHNP design. Therefore, the staff finds that the application conforms to this criterion. Criterion 4 Issues Identified by Plant Personnel through interviews with plant personnel based on its operating experience with plants or systems applicable to the new design. As a minimum, the interviews should include the following topics: Plant Operations o normal plant evolutions (e.g., startup, full power, and shutdown) o failure modes and degraded conditions of the I&C systems, including, but not limited to, the sensor, monitoring, automation and control, and communications subsystems. These include, for example, the safety- related system logic and control unit, fault tolerant controller (NSSS), the local "field unit" for the multiplexer (MUX) system, the MUX controller (balance-of-plant), and a break in the MUX line failure modes o degraded conditions of the HSI resources (e.g., losses of video display units, of data processing, and of large overview display) o transients (e.g., turbine trip, loss of offsite power, station blackout, loss of all feedwater, loss of service water, loss of power to selected buses or MCR power supplies, and safety/relief valve transients) o accidents (e.g., main steam line break, positive reactivity addition, control rod insertion at power, control rod ejection, anticipated transients without scram, and various-sized loss-of-coolant accidents) o reactor shutdown and cooldown using the remote shutdown system HFE Design Topics o alarms and annunciation o displays o controls and automation o information processing and job aids o real-time communications with plant personnel and other organizations 27 o procedures, training, staffing/qualifications, and job design iterion 4 personnel interviews are conducted to collect information on all the topics listed in the acceptance criterion. In addition, the interview question structure is developed to match the topics in this criterion. criterion, and the applicant is using operator interviews as well as operator involvement in design testing as an input to the KHNP HSI design. Therefore, the staff finds that the application conforms to this criterion. Criterion 5 Important Human Actions (IHAs) predecessor plants or systems, and determidesign. Additional considerations cover the following: For the important HAs, the OER should identify the scenarios wherein actions are needed, and state whether they were needed and successfully completed. Those aspects of the design that helped ensure success should be identified. If errors occurred in the execution of the HAs, the applicant should identify insights to the needed improvements in human performance. When important HAs for the new plant are determined to differ from those of the predecessor plant, the OER should specify whether there is any operational experience with these different HAs. that for IHAs, the OE reviewer will use the list of important human actions from the treatment of important human actions (TIHA) element to verify it against the predecessor and document differences as HEDs. The OE E issues that include human actions credited for accident mitigation. These actions include backup manual actions. Lastly, an assessment of the aspects that deter or ensure success is done. Where an OE aspect caused a deterrence or error in a human action, they are documented as a lesson learned and an HED, then tracked so that they can be adequately considered in the HFE design. Section 4.10 also states that the OE reviewer reviews the list of IHAs developed in the TIHA element and determines where IHAs for the new plant design differ from those of the predecessor plant. The implementation team then identifies whether there is an OE with the different human actions and writes an HED for inclusion in the ITS and tracking purposes. The staff concludes that the applicant is appropriately using operating experience to ensure that the APR1400 design facilitates the successful performance of important human actions. Therefore, the staff finds that the application conforms to this criterion. 28 18.2.4.2 Issue Analysis, Tracking and Review NUREG-this topic. Criterion 1 OER Process The applicant should discuss the administrative procedures for evaluating the operating, design, and construction experience, and for ensuring that applicable important industry experiences will be provided in a timely manner to those designing and constructing the plant. At a high level, the applicant has provided in the OER IP the method for meeting this criterion. The sources of OE the applicant has discussed include the items mentioned in this criterion. The overall placement of the HFE process, and the HFE design personnel, to the overall APR1400 design process are discussed in the HFE PP. The HFE PP, from a high level, provides assurance that applicable important industry experiences found during OER via the HFE process will be provided in a timely manner to other disciplines during the design and construction of the APR1400. Therefore, the staff finds that the application conforms to this criterion. Criterion 2 Analysis Content The applicant should analyze issues to identify: human performance issues and sources of human error design elements supporting and enhancing human performance 2 the evaluation process used by the OE reviewer to select and analyze human performance issues. Questions used in the selection process are provided in Section 4.5. Within the questions, the two areas described in the acceptance criterion bullets are addressed. During d to resolve items that are assigned to other HFE program elements. Therefore, the staff finds that the application conforms to this criterion. Criterion 3 Documentation The applicant should document the analysis of operating experience. aluation of Criterion 3 OER IP, Section 6 describes the documentation the applicant will provide containing the analysis of the OER activity. The report includes the OER execution results containing details that demonstrate compliance to the Methodology section of the OER IP. The report will include: Predecessor/related plants and systems 29 Review Methodology OE sources/documents reviewed Names and qualifications of all members of the team that performed the OER Conduct of the OER and results of reviewing relevant HSI technology Findings from interviews with plant personnel and other users List of the OEs that are incorporated into the design Number and status of the open HEDs that are still being tracked in the ITS The staff concludes that that all essential elements of the OER are being documented. Therefore, the staff finds that the application conforms to this criterion. Criterion 4 Incorporation into the Tracking System The applicant should document each issue determined to be relevant to the design, but yet to be addressed, in the issue-tracking system whereby the negative OEs are assessed. After the assessment, the cause determined for the into the OE database for tracking. All lessons learned that affect other HFE program elements are entered into the ITS and tracked to completion. The staff concludes that potential HFE issues identified during the OER are being documented in an ITS. Therefore, the staff finds that the application conforms to this criterion. Combined License Items There are no COL items associated with Section 18.2 of the APR1400 DCD. Conclusion conforms to the criteria in NUREG-OER sufficiently identifies, analyzes, and addresses HFE-related problems in previous designs similar to the APR1400. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii), 10 CFR 50.34(f)(3)(i), and 10 CFR 52.47(a)(8) related to this element. 18.3 Functional Requirements Analysis and Function Allocation Introduction FRA is the identification of functions that must be performed to satisfy plant safety objectives; that is to prevent or mitigate the consequences of postulated accidents that could cause undue risk to the health and safety of the public. Function Allocation (FA) is the analysis of 30 requirements for plant control and the assignment of control functions to: (1) personnel (e.g., manual control); (2) system elements (e.g., automatic control and passive, self-controlling phenomena); and (3) combinations of personnel and system elements (e.g., shared control, automatic systems with manual backup). performed to satisfy plant safety objectives have been defined, and (2) the allocation of those functions to human and system resources has resulted in a role for personnel that takes advantage of human strengths and avoids human limitations. Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of -K-X-IT-14001. Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2: The applicant has provided a Tier 2 description in Section 18.3 of the DCD, which summarizes the method described in APR1400-E-I-NR-to complete the functional requirements analysis and function allocation program element. Changes to Tier 2 information are governed by the change control processes in the design evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs. ITAAC: There are no ITAAC associated with this element. TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: TeRs associated with this element are: APR1400-E-I-NR- (ML18081A107) APR1400-E-I-NR-(ML18178A202) APR1400-E-I-NR- (ML18178A202) Regulatory Basis described in - 0800. The applicable regulatory requirements are as follows: 10 CFR 52.47(a)(8) 31 10 CFR 50.34(f)(2)(iii) Other regulatory guidance documents are as follows: NUREG/CR-Functions to Human and Automated NUREG- NUREG-Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Function Technical Evaluation NUREG-criterion addresses plant modifications and is not applicable to new reactors; therefore, the staff evaluated the first eight criteria as discussed below. Criterion 1 The applicant should use a structured, documented methodology reflecting HFE principles to perform FRA and FA. methodology in the FRA/FA IP using this criterion. The KHNP methodology follows the traditional approach of defining two high-level goals, safety and power production, then the high-level functions needed to obtain the goals, and then the success paths needed to obtain the high-level functions. The success path actions are then allocated to human and system resources resulting in a role for personnel that takes advantage of human strengths and avoids human limitations. A success path is defined as an individual FRA/FA IP provides three figures: Figure 3--1, -graphically present a structured methodology for performing FRA/FA. The methodology is based, in part, on NRC guidance such as NUREG/CR-3331 and on the staff approved System 80+ methodology documented in NPX80-IC-RR970- preliminary plant design development and is sufficiently detailed to enable specification of detailed plant design and HSI design requirements. HFE principles such as time available, workload, and task complexity are considered in the FA analysis. The processes that support A methodology are evaluated in more detail in subsequent review criteria. 32 The staff concludes that the FRA/FA process described in the FRA/FA IP provides a structured, documented methodology reflecting appropriate HFE principles. Therefore, the staff finds that the application conforms to this criterion. Criterion 2 development and operation up to decommissioning, so that it can be used as a design basis when modifications are considered. l be identified for each critical safety function (CSF) and power production function. Success paths are sets of plant system configurations that are able to maintain or restore either a critical safety function (CSF) or a power production function. The process of identifying success paths for each function is [ ] During the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member questioned why the FRA/FA IP states that generic assumptions about site-specific plant systems will be needed given the fact that the COL applicant will be performing the activities in the FRA/FA IP, and the COL applicant will know site-specific information. Following the ACRS meeting, the staff re-evaluated Section 4.3.3 of the FRA/FA IP. Because site-specific information will be available when a COL applicant performs FRA and FA, the site-specific information will need to be used to perform FRA and FA in order to ensure the FRA and FA results are kept current with the plant design. As such, the staff determined the application did not fully conform to this criterion. Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-137 to address this issue (ML17249A979). On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-137 (ML17271A188) and stated, The generic assumptions support the preliminary results of the FRA/FA and TA, and those results provide the basis for the HSI design. As site-specific information is known, the generic assumptions are modified as necessary. When the COL applicant performs the HFE activities, the site-specific information is applicable to develop the APR1400 HSI design at the site, and the preliminary results of the FRA/FA and TA are updated accordingly. The updated information leads to the complete HSI design which is verified and validated during the HF V&V program element (PE). DI PE confirms the as-built design with the application of the site-specific information. Where the site-specific information is not reflected, the DI PE conducts a regression analysis to define the necessary HFE rework. The applicant also provided revisions to the FRA/FA IP, Section 4.3.3, to clarify that the COL applicant is to use the site-specific information to complete the activities in the FRA/FA IP. The 33 staff concludes that the proposed revisions to the FRA/FA IP ensure that the results of the FRA and FA will accurately reflect the plant design that exists when the activities in the FRA/FA IP are performed. Based on the review of the FRA/FA IP and the TA IP, the staff has confirmed incorporation of the changes described above; therefore, RAI 553-9084, Question 18-137 was resolved and closed. Additionally, the applicant described how the results of the FRA and FA will remain current states, [ ] Section 4.2 describes three mechanisms that maintain the FRA and FA results current: resolution process described in the HFE PP provides a mechanism to track HFE feedback as the plant design progresses. As the design information becomes more detailed and complete or modified, issues relative to the HFE design are identified. To the extent those issues impact the FRA/FA and require changes to functional requirements or allocations, the FRA/FA is The FRA/FA IP, Section 4.2, states, [ ] The staff concludes that the FRA/FA is kept current over the life cycle of the plant by being project configuration management and design change procedures. Accordingly, the staff finds that the application conforms to this criterion. Criterion 3 The applicanfunctions, processes, and systems. The description should include: comparing them with the predecessor or reference plants and systems, i.e., the previous ones on which the new plant is based identifying the differences between the proposed and reference plants and systems documenting the technical basis for modifications to high-level functions in the new design compared to the predecessor design[ ] These are discussed in more detail below: 34 defining, for each safety function and other plant function (e.g., electrical power generation), the set of system configurations or success paths that are responsible for, or able to carry out the function -ions where a very general picture of major functions is described, and continuing to lower levels, until a specific critical end-item requirement emerges (e.g., a piece of equipment, software, or an HA). The functional decomposition should address the following levels o high-level functions (e.g., maintain reactor coolant system integrity) o the processes, as appropriate, that enable achievement of these functions o specific plant systems and components o HAs, as appropriate 3 maintained to achieve the plant goals of safety and power production. For each high-level function, success paths are developed which include the following components: sub-functions (e.g., reactivity control) process (e.g., boration, or rod insertion) plant systems (e.g., chemical and volume control system, control rod drive, etc.) key components (e.g., shutdown rods) action required of those key components (e.g., drop rods, or insert rods manually)Sub functions and systems are not described (other than by example) but it is clear from the direction and example forms provided that this information is collected as part of the analysis. In addition, differences between the proposed and predecessor plant/systems are listed in Plant operations SMEs perform the success path identification. NPX80-IC-RR790-02 and CEN-a starting point to establish the high-level critical safety functions. The System 80+ is then compared to the APR1400 to identify identical, modified, and new success paths. This process is done to ensure that adequate success paths exist to protect high-level functions during the full range of plant operating modes (full power, low power, shutdown) and conditions (e.g., normal or abnormal). stated that the Containment Environment CSF differs from the predecessor design in that it was split into two items: 1) Containment Combustible Gas Control and 2) Containment Temperature and 35 Pressure Control. In general, the CSFs for the APR1400 are the same as the predecessor design, and the applicant stated that no changes are expected for the APR1400 design. However, if any modification should occur in the CSFs, they would be documented in the ReSR. The staff concludes that the high-level functions needed to meet plant goals are clearly identified and are complete. The methodology and the documentation of analysis results provide for complete descriptions of the systems, components, and actions that constitute the success paths needed to control the high-level functions. Therefore, the staff finds that the application conforms to this criterion. Criterion 4 For each high-level function, the applicant should identify requirements related to: purpose of the high-level function conditions indicating that the high-level function is needed parameters indicating that the high-level function is available parameters indicating that the high-level function is operating (e.g., flow indication) parameters indicating that the high-level function is achieving its purpose (e.g., reactor vessel level returning to normal) parameters indicating that the operation of the high-level function can or should be terminated this NUREG-0711 criterion. Overall, FRA/FA IP, Section 4 provides the detailed process to ontain the lower level decomposition information of the higher level critical safety functions. The decomposition information is presented in Table 4-1, describes: 1) the plant goals and high-level functions; 2) the decomposition of the functions into success paths (e.g., sub-functions, systems, components, and actions) and; 3) includes cells that describe the high-level function in terms of the characteristics listed in the acceptance criterion. The staff concludes that the characteristics needed to define each high-level function are clearly identified. The methodology provides for complete documentation of each characteristic. Therefore, the staff finds that the application conforms to this criterion. Criterion 5 The applicant should allocate functions to a level of automation (e.g., from manual to fully automatic) and identify the technical bases for the allocations. 36 rion 5 used to allocate functions to a level of automation ranging from manual to fully automatic for the KHNP design. Plant operations and system SMEs assess the functions identified by the FRA for each success path to determine the role of personnel, automation, or a combination, using the evaluation process described in NUREG/CR-3331 and NPX80-IC-RR790-and Allocation of Sy-3331 describes an acceptable method for allocating functions. The workload, time available, action complexity, and decision complexity for each function are evaluated and compared to guidelines in NUREG/CR-3331 to establish the level of automation. Then the technical basis and rationale for the function allocations are captured in an allocation table, of which an example is provided in the FRA/FA IP. Completed allocation entries from the table show the relationship of the FAs to the success paths derived from the FRA. The staff concludes that the criteria and process for determining functional allocations is clearly and completely described and incorporates accepted HFE practices. The basis for allocations is well documented in NUREG/CR-3331. Therefore, the staff finds that the application conforms to this criterion. Criterion 6 functions for which personnel have the primary responsibility, but also its responsibilities to monitor automatic functions, detect degradations and failures, and to assume manual control when necessary. responsibilities listed in the criterion. In addition, the responsibilities listed in the criteria are captured in NUREG/CR-which the applicant uses in Appendix C of the FRA/FA IP as part of its FA process. responsibility for monitoring automatic functions, detecting degradations and failures, and assuming manual control when necessary. Therefore, the staff finds that the application conforms to this criterion. Criterion 7 The applicant should describe the overall role of personnel by considering all functions allocated to them. [ ] through the Task Analysis and Staffing & Qualifications (S&Q) elements. Task analysis provides a more detailed 37 evaluation of the tasks allocated to operators. The task analysis process also verifies that the initial functional allocation is acceptable after the workload analysis results are obtained. The S&Q analyzes the requirements for the number of personnel and their qualifications using the task and regulatory requirements. The staff concludes that the applicant has considered all of the functions allocated to personnel and has described the overall role of personnel. Therefore, the staff finds that the application conforms to this criterion. Criterion 8 The applicant should verify that the FRA and FA accomplish the following: All the high-level functions needed to achieve safe operation are identified. All requirements of each high-level function are identified. The allocation of functions to humans and automatic systems assures a role for personnel that takes advantage of human strengths and avoids human limitations. performed by an independent plant operations SME that did not take part in the FRA/FA. Also, completing the activities described in the TA IP and V&V IP verifies that the allocation of functions to humans and automatic systems takes advantage of human strengths and avoids human limitations. The FRA/FA team leader ensures the discrepancies identified by the SMEs are resolved during the review. The staff concludes that the FRA/FA results are independently verified, and any discrepancies are resolved. Therefore, the staff finds that the application conforms to this criterion. Combined License Items There are no COL items associated with Section 18.3 of the APR1400 DCD. Conclusion The staff evaluated the applicants FRA and FA method and finds that it conforms to the criteria in NUREG-0711, Section 4.4. Therefore, the staff concludes that the FRA/FA methodology and sufficiently allocates those functions to human and system resources. Accordingly, the staff concludes the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element are satisfied. 18.4 Task Analysis Introduction Task analysis identifies the tasks that plant personnel must perform to accomplish the functions that are allocated to human actions (HAs). Task analysis also identifies the alarms, information, controls, and task support that must be available for plant personnel to successfully perform 38 these tasks. Task analysis generates input to several program elements: staffing and qualifications, HSI design, procedure development, training program development, and verification & validation. Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of -K-X-IT-14001. Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2-E-I- NR-complete the task analysis program element. Changes to Tier 2 information are governed by the change control processes in the design evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs. ITAAC: There are no ITAAC associated with this element. TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: TeRs associated with this element are: APR1400-E-I-NR-IP) (ML18178A202). The TA IP contains the method that will be used by the COL applicant to perform task analysis. APR1400-E-I-NR-PP) (ML18212A336). The HFE PP describes the relationship between the task analysis program element and other HFE program elements. Regulatory Basis Sec NUREG-0800. The applicable regulatory requirements are as follows: 10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) Other regulatory guidance documents are as follows: NUREG-39 NUREG-Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Factors Engin Technical Evaluation NUREG-criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first nine criteria as discussed below. Criterion 1 All IHAs as determined by probabilistic and deterministic means. The applicant should select tasks for analysis that represent the full range of plant operating modes, including startup, normal operations, low power and shutdown conditions, transient conditions, abnormal conditions, emergency conditions, and severe accident conditions. The chosen tasks should cover: o taconsequences if performed incorrectly; o tasks that are new compared to those in predecessor plants, such as ones related to new systems or procedures; o tasks that, while not new, are performed significantly differently from predecessor plants; o tasks related to monitoring of automated systems that are important to plant safety, and the use of automated support aids for personnel, such as computer- based procedures; o tasks related to identifying the failure or degradation of automation, and implementing backup responses; o tasks anticipated to impose high demands on personnel, e.g., little time or high workload (such as administrative tasks that contribute to work load and challenge ability to monitor the plant); o tasks important to plant safety that are undertaken during maintenance, tests, inspections, and surveillances; and o tasks with potential concerns for personnel safety (such as maintenance tasks performed in the containment). The staff reviewed DCD Tier 2, Section 18.4, and the TA IP. Three sections in these Section 18.4.1 and the TA IP, Section 2 list the types of tasks that will be included in the scope 40 of the task analysis. In these sections, the staff found most, but not all, of the types of tasks listed in Criterion 1. The TA IP, Section 4.1 states that tasks selected for task analysis include all the types of tasks listed in this criterion, with one exception. The application describes the scope of the task analysis in three different sections, and the information is not consistent in these three sections, nor does it conform to Criterion 1 in any of these sections. Therefore, on October 19, 2015, the staff issued RAI 260-8283, Question 18-30 and Question 18-31 to address this issue (ML15293A581). On December 31, 2015, the applicant provided a response to RAI 260-8283, Question 18-31 (ML15365A562) and on January 21, 2016, the applicant provided a response to RAI 260-8283, Question 18-30 (ML16021A516). In the responses, the applicant submitted a revision to DCD consistent in each of the sections that discuss the scope of the task analysis and also state that the scope of the task analysis includes all of the types of tasks listed in Criterion 1. Based on the review of the DCD and TA IP, the staff has confirmed incorporation of the changes described above; therefore RAI 260-8283, Questions 18-30 and 18-31 were resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 2 The applicant should describe the screening methodology used to select the tasks for analysis, based on criteria specifically established to determine whether analyzing a particular task is necessary. The TA IP, Section 4.1 contains a proprietary method for identifying tasks that are within the scope of task analysis and lists the sources that will be used to identify these tasks. The HFE PP and DCD Tier 2, Section 18.4.1 describe the sources used for task selection as follows. that three HFE program elements provide inputs to task selection: OER, function requirements analysis and function allocation (FRA/FA), and TIHA. OER identifies human actions in the APR1400 design that are associated with known human performance issues at operating plants. FRA/FA identifies actions all modes and for normal, abnormal, and emergency conditions. TIHA identifies risk-significant (i.e., risk-important) human actions and human actions that must be performed to mitigate the consequences of plant transients and accidents. DCD Tier 2, Section 18.4.1 describes that tasks identified in the normal, abnormal, emergency, and alarm response procedures that are available at the time the TA is conducted or are from the predecessor plants or predecessor designs are selected. The three other HFE program elements identify human actions that are performed during all modes of operation and under normal, abnormal, and emergency conditions, including the IHAs and actions with known human performance issues. The APR1400 is an evolutionary design, 41 actions that plant personnel must take to operate the APR1400 plant during normal, abnormal, used to identify tasks that personnel must perform to operate the plant, including tasks that with relatively high risk and safety significance, are adequate. tasks for analysis. Two levels of task analysis will be performed: a basic task analysis (BTA) and a task timing analysis (TTA). A BTA will be conducted for all of the tasks that are identified from the OER, FRA/FA, TIHA, and plant operating procedures. As part of the BTA, all tasks will be screened using selection criteria to determine if a TTA must be performed. workload and time margin to identify tasks that may produce an unacceptably high workload or have an inadequate time margin. This allows for early5 identification of changes to the HFE design or plant design that may be necessary to ensure the safe operation of the plant. Because not all tasks are likely to have high workloads or be subject to time constraints, the staff concludes that it is appropriate to perform a TTA only for tasks that are likely to have a high workload or inadequate time margin during task performance. used to select tasks for a TTA. The staff reviewed the list and concluded that the screening criteria include tasks that must be performed within a time limit or have certain characteristics that may cause the task to involve a relatively high workload. Thus, the staff finds that the application describes the method for selecting tasks for both a BTA and a TTA. Also, the staff finds that the application contains appropriate criteria specifically established to determine whether performing a TTA is necessary. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 3 The applicant should begin task analysis with detailed narratives of what personnel must do. The analysis should be sufficiently detailed to define the alarms, information, controls, and task support needed to accomplish the task. The detailed task descriptions should address (as applicable to the task) the topics listed in NUREG-0711, Table 5- DCD from the BTA, which is performed for each task: the task narrative, the HSI inventory, and the task evaluation. These are described below. 5 Workload and task performance time are assessed during the ISV, which is the final validation of the HFE design. If the ISV demonstrates that workload for certain tasks is not acceptable or that operators cannot perform tasks with time constraints in the time available, then changes to the integrated system (i.e., the HSI, procedures, or training program) may be necessary to reduce workload or the time it takes operators to perform a given task. All or portions of the ISV may need to be repeated to demonstrate that the changes are effective. Estimating workload and time margin during task analysis provides a means of identifying tasks that may have unacceptable workload levels or insufficient time margins relatively early in the HFE design process when the components of the integrated system are still being developed, which should help reduce the likelihood that significant design changes will have to be made late in the design process. 42 BTA begins with developing a task narrative, which describes what plant personnel task narrative. The staff reviewed the list of topics and concluded that as a group, the topics address what personnel must do to perform the task, the HSI inventory (alarms, controls, and indications) that personnel need to perform the task, task support requirements, situational and performance-shaping factors, and time constraints. Therefore, the staff finds that the task analysis begins with developing detailed task narratives that describe what personnel must do and the alarms, information, controls, and task support needed to accomplish the task. Once the task narrative is complete, the BTA continues with the HSI inventory. The HSI inventory expands on the task narrative by identifying the characteristics proprietary topics that must be addressed in the HSI inventory. The staff reviewed the list of topics and concluded that as a group, the topics address, in detail, the characteristics and features of the alarms, information, controls, and task support that personnel need to perform the task (e.g., the units of measurement that should be displayed for a particular parameter). The BTA concludes with the task evaluation, which determines whether a TTA is iterion 2, a TTA will be performed for tasks that meet the screening criteria for a TTA. The TTA results in an: (1) estimation of workload and time margin for a particular task and (2) an evaluation of whether the workload and time margin are acceptable. Taken together, the topics addressed in the task narrative and the HSI inventory produced for each task during BTA encompass all of the topics listed in NUREG-0711, Table 5-1 with the exception of workload. This is acceptable because workload will be assessed only for the tasks that meet the screening criteria for a TTA. Thus, workload is not a topic that is applicable for every task. Therefore, the staff finds that the task narrative and the HSI inventory for each task and the workload assessment, if applicable to the task, address all of the topics in NUREG-0711, Table 5-1 and will therefore ensure that the alarms, controls, information, and task support that personnel need to accomplish the task will be defined. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 4 The applicant should identify the relationships among tasks. The TA IP, Section 4.2.1 provides a list of proprietary topics that must be documented in the task narrative for each task. The staff reviewed the list of topics and concluded that it includes the relationship of a task to other tasks and whether the task can be performed in parallel with related tasks or must be performed in a specific sequence. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. 43 Criterion 5 The applicant should estimate the time required to perform each task. DCD Tier 2, Section 18.4.2.2 states that time required (TmRq) is estimated by adding the process delay time (e.g., if a task is to open a valve, then the process delay time is the time it takes a valve to stroke from the closed to open position) and the time an operator is engaged in the task (i.e., time engaged or TmEn). Process delay times can be determined from documents that describe the operating characteristics of plant components or estimated using information ontains a evaluation of Criterion 9. Criterion 2 of this section, tasks selected for a TTA includes tasks that are likely to produce high workloads or that are subject to time constraints. Therefore, time required does not need to be estimated during task analysis for tasks that are not subject to time constraints. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 6 The applicant should identify the number of people required to perform each task. The TA IP, Section 4.2.1 provides a list of proprietary topics that must be documented in the task narrative for each task. The staff reviewed the list of topics and concluded that it includes the number of people required to perform the task. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 7 The applicant should identify the knowledge and abilities required to perform each task. The TA IP, Section 4.2.1 provides a list of proprietary topics that must be documented in the task narrative for each task. The staff reviewed the list of topics and concluded that it did not adequately address knowledge and abilities required to perform tasks. Knowledge and abilities must be defined during task analysis because this information is an input to the staffing and qualifications program element. Therefore, on October 19, 2015, the staff issued RAI 260-8283, Question 18-39 to address this issue (ML15293A581). On January 21, 2016, the applicant provided a response to RAI 260-8283, Question 18-39 (ML16021A516). The response reiterated the information previously submitted, and thus it did not adequately address RAI 260- 8283, Question 18-39. Therefore, on July 25, 2016, the NRC held a public meeting with the applicant to discuss the response. During the meeting, the staff stated that NUREG-1122, PWRs. 44 Because the APR1400 plant is an evolutionary PWR, NUREG-1122 contains a significant portion of the K/As applicable to tasks operators will perform at the APR1400 plant. The applicant stated that it would revise the TA IP to refer to NUREG-1122 and to include any K/As that are not documented in the K/A catalog in the task narrative. On August 9, 2016, the applicant provided a revised response to RAI 260-8283, Question 18-39 (ML16222A916). The response incorporated the revision discussed above. The staff confirmed that TA IP was revised as committed in the response to RAI 260-8283, Question 18-39. Therefore, RAI 260-8283, Question 18-39 was resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 8 iterative and updated as the design is better defined. The TA IP, Section 6 states that performance of the task analysis program element is a one- time event. The task analysis program element is performed in accordance with the method individual tasks (i.e., the task analyses), including the task narrative and HSI inventory, are documented in a database. DCD Tier 2, Section 18.4.1, For tasks related to plant systems that are site specific, such as the switchyard and ultimate heat sink, the TA [task analysis] is based on generic assumptions that are made to establish a complete plant design that is ultimately reflected in the complete APR1400 HSI design for V&V. These generic assumptions are modified as necessary for each plant-specific application of the APR1400 during the design implementation (DI) program element. APR1400-E-I-NR-contains similar statements. During the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member questioned why the application states that generic assumptions about site-specific plant systems will be needed given the fact that the COL applicant will be performing the activities in the TA IP, and the COL applicant will know site-specific information. Following the ACRS meeting, the staff re-evaluated the TA IP and the DCD. Because site-specific information will be available when a COL applicant performs TA, the design will be better defined than the design that is described in the DCD. The staff agrees that the COL applicant will need to use site-specific information in order to ensure the TA is performed using current information about the plant design. As such, the staff determined the application did not fully conform to this criterion. Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-137 to address this issue (ML17249A979). On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-137 (ML17271A188) and stated: The generic assumptions support the preliminary results of the FRA/FA and TA, and those results provide the basis for the HSI design. As site-specific information is known, the generic assumptions are modified as necessary. When the COL applicant performs the HFE activities, the site-specific information is 45 applicable to develop the APR1400 HSI design at the site, and the preliminary results of the FRA/FA and TA are updated accordingly. The updated information leads to the complete HSI design which is verified and validated during the HF V&V program element (PE). DI PE confirms the as-built design with the application of the site-specific information. Where the site-specific information is not reflected, the DI PE conducts a regression analysis to define the necessary HFE rework. The applicant also provided revisions to DCD Tier 2, Section 18.4.1, and the TA IP, Section 2, to clarify that the COL applicant will use the site-specific information to complete the activities in the TA IP. The staff concludes that the proposed revisions to the DCD and TA IP ensure that the task analyses will accurately reflect the plant design that exists when the activities in the TA IP are performed. Based on the review of the DCD and TA IP, the staff has confirmed incorporation of the changes described above; therefore RAI 553-9084, Question 18-137 was resolved and closed. Following completion of the activities in the TA IP, the task analyses may need to be updated as the HFE design and plant design evolve. HEDs identified during the implementation of other HFE program elements will be evaluated to determine if the task analyses need to be revised. Also, as described in the TA IP, Section 6 plant design changes are evaluated for their impact on all of the HFE program elements, including task analysis. Changes to task analyses are documented and evaluated using the HED resolution process. Because the task analysis will be updated as necessary as the HFE design and plant design evolve, the staff finds the application conforms to this criterion. Criterion 9 The applicant should provide analyses of the feasibility and reliability for IHAs that address the following: (1) The analysis establishes the time available using an analysis method and acceptance criteria consistent with the regulatory guidance associated with the actions. The basis for the time available is documented. (2) The analysis of the time required is based on a documented sequence of operator actions (based on task analysis, vendor-provided generic technical guidelines for emergency operating procedure development, or plant-specific EOPs, depending on the maturity of the design). (3) The analysis of the action sequence is conducted at a level of detail sufficient to identify individual task components, including cognitive elements such as diagnosis and selection of appropriate response. (4) The sequence of actions uses only alarms, controls, and displays that would be available and operable during the assumed scenario(s). (5) The estimated time for operators to complete the credited action is sufficient to allow successful execution of applicable steps in the EOPs. 46 (6) Techniques to minimize bias are used when estimates of time required are derived using methods that are dependent on expert judgment. Uncertainties in the analysis of time required are identified and assessed. (7) Staffing for analysis is justified, and if credited manual actions require additional operators beyond the assumed staffing, the justification for timely availability of the additional staffing is provided and the estimate of time required includes any time needed for calling in additional personnel. (8) The analysis identifies a time margin to be added to the time required and the basis for the adequacy of the margin. method of performing a TTA, which assesses workload and time margin. Time margin is the difference between time available6 (TmAv) to perform the task and the time required7 (TmRq) for the operators to perform the task. A TTA will be performed for all IHAs to analyze, in part8, the feasibility and reliability of tasks that are IHAs. Bullet 1 ] Because the TmAv from the applicable plant analysis will be documented in the task narrative for each IHA, and because the source of the information will also be documented, the staff finds that the application conforms to this NUREG-0711 criterion. Bullet 2 Section 18.4.2.2 states that TmRq is estimated by adding the time an operator is engaged in the task (time engaged or TmEn) and the process delay time (PdTm). PdTm is the amount of time it takes the plant component(s) to respond to the action(s) taken by the operator during task 6 7 The TA IP, Section 8 defines 8 The TTA assesses workload level, which can impact human performance, as well as time margin. This NUREG0711 criterion specifically addresses the assessment of time margin as a means of analyzing the feasibility and reliability of IHAs. 9 method(s) and determination that the applicable acceptance criteria have been satisfied. The TA IP, Section 4.2.1 states that the TmAv for each IHA [ 47 performance. PdTm contributes to the total time required to perform a task, and therefore it is added to the TmEn to determine the total time required to perform a task. TmEn is an estimate of the amount of time the operator is engaged in task performance. [ ] [ ] Determination of subtask time is described in more detail under Bullet (3) below. Thus, TmEn is based on a sequence of operator actions, [ ] Because TmEn is an input to TmRq, TmRq is also based in part on a documented sequence of operator actions. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Bullet 3 As described under Bullet 2 above, [ ] The scenario descriptions do not provide a sufficient amount of detail about the individual task components that operators must perform to accomplish the entifies the following task that is an IHA: operator manually closes the inadvertently opened atmospheric dump valve (ADV) on the affected steam generator. To accomplish this task, the operator must perform a series of subtasks in a specific order. This information can be obtained from procedures of the predecessor plant or reference plant procedures. However, the detail in the procedures may also be limited mainly to physical fy the affected steam generator, the operator will need to monitor specific plant parameters using the displays and instrumentation in the control room. This may require navigating between screens on the operator consoles or walking from the operator console to another panel in the control room. These activities add to the TmEn, but the plant procedures typically do not include this level of detail. Therefore, additional detail will need to be added to the action sequence in order to accurately estimate the TmEn and thus the TmRq. ] Therefore, on November 16, 2015, the staff issued RAI 315-8091, Question 18-50 to address this issue (ML15320A406). [ 48 On April 6, 2016, the applicant provided a response to RAI 315-8091, Question 18-50 (ML16097A633). [ ] The staff confirmed that TA IP was revised as committed in the response to RAI 315- 8091, Question 18-50. Therefore, RAI 315-8091, Question 18-50 was resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Bullet 4 The TA IP, Section 4.3 states that [ ] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Bullet 5 ] [ 10 This is important because for some scenarios, such as those analyzed in the D3CA, the set of alarms, controls, and indications that operators normally use may be degraded or inoperable, and therefore the operator may need to use alarms, controls and indications that he or she normally does not use. Operators may not be as familiar with using the alternative indications and controls, and this can result in an increase to the TmEn and thus the TmRq. 11 DCD Tier 2, procedure development, which is a COL item (see DCD Tier 2, Section 13.5). As discussed under Bullet (2) above, [ 49 ] Therefore, the estimate of TmRq will include the time it takes the operators to perform procedurally-guided actions. important HAs are tested during the integrated systems validation (ISV). The V&V IP states that each scenario in the ISV is developed to simulate actual plant operator tasks, which are performed using plant procedures such as emergency operating procedures. Thus, the ISV provides the opportunity to demonstrate that operators can successfully perform credited actions in accordance with and as directed by the emergency operating procedures in the time available. If the operators cannot successfully complete IHAs during the ISV in accordance with and an HED will be documented and resolved in accordance with the HFE PP. The inclusion of all tasks associated with important HAs in the ISV provides the opportunity to demonstrate that the operator(s) can use procedures to perform their tasks. Accordingly, the staff finds that the ISV will provide an opportunity to demonstrate that the operators can complete credited actions (i.e., IHAs) in accordance with EOPs, and the application conforms to this NUREG-0711 criterion. Bullet 6 The TA IP, Section 4.3.1.2.2 describes the method for determining [ The TA IP, Section 5 [ ] Thus, the staff has reasonable assurance that the plant operations SMEs will have the training and operating experience that will allow them to make valid estimates of the time it takes to perform unique subtasks when it is necessary to use SME judgment to derive TmEn and thus TmRq. ] These are subtasks, or individual task components that operators need to do to perform tasks that are IHAs. [ ] [ 50 ] Therefore, the staff finds that these additional time estimates provide a means of accounting for uncertainties in the estimate of TmRq. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Bullet 7 [ ] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Bullet 8 [ ] NUREG-for the specific time margin used in the analysis should be justified and documented. Insights application does not conform to this guidance. Therefore, on November 16, 2015, the staff issued RAI 315-8091, Question 18-53 to address this issue (ML15320A406). On March 14, 2016, the applicant provided a response to RAI 315-8091, Question 18-53 (ML16074A301.) In the response, the applicant stated that the TA IP would be revised to state that the [ ] The staff confirmed that TA IP was revised as committed in the response to RAI 315-8091, Question 18- 53. Therefore, RAI 315-8091, Question 18-53 was resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Combined License Items There are no COL items associated with Section 18.4 of the APR1400 DCD. 51 Conclusion in NUREG-method provides for identifying the control room inventory and for determining there is reasonable assurance the operator tasks can be implemented effectively. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element. 18.5 Staffing and Qualifications Introduction plicant has systematically analyzed the number and necessary qualifications of personnel in concert with task demands and regulatory requirements. The analysis addresses the full range of plant conditions and tasks including operational tasks (normal, abnormal, and emergency), plant maintenance, and plant surveillance and testing. Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400-K-X-IT-14001. Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2Qualification-K-I-NR-14005, complete the staffing and qualifications program element. Changes to Tier 2 information are governed by the change control processes in the design evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs. ITAAC: There are no ITAAC associated with this element. TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: TeRs associated with this element are: APR1400-K-I-NR-1IP) (ML17094A129) APR1400-E-I-NR--(ML18178A202) 52 Regulatory Basis The relevant requirements for - 0800. The applicable regulatory requirements are as follows: 10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) Other regulatory guidance documents are as follows: NUREG- NUREG-Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human SECY-05-Application and Generic Emergency Planning Inspections, Tests, Analyses, and Acceptance Technical Evaluation NUREG-includes six criteria for this topic. Criterion 1 The applicant should address the applicable staffing and qualifications guidance in NUREG-0800, Section 13.1. This criterion is evaluated in Section 13.1 of this SER. Criterion 2 The applicant should address the applicable staffing and qualifications guidance in 10 CFR The requirements contained in 10 CFR 50.54(k) and 50.54(m) identify requirements for licensed operators. The requirements are conditions in every nuclear power reactor operating license Also, the requirements are conditions in every combined license issued under 10 CFR Part 52, applicable after the Commission makes the finding under 10 CFR 52.103(g) that the acceptance criteria in the combined license are met. Although the requirements in 10 CFR 50.54 apply to 53 room design to determine whether a license applicant will be able to comply with the staffing requirements in 10 CFR 50.54. If the license applicant will not be able to comply, then the license applicant will need to request an exemption. The license applicant would need to provide sufficient technical justification to support the exemption request. Alternatively, the DC applicant may provide sufficient technical justification with the DC application to support a design-specific staffing rule that would be used in lieu of 10 CFR 50.54(m). This option provides the greatest degree of issue finality, which may be preferable to would enable a licensee to comply with the staffing requirements in 10 CFR 50.54(m) at the DC stage to determine whether additional documentation must be submitted to develop a design- specific staffing rule. compliance with the staffing requirements of 10 CFR 50.54(m) and 10 CFR 50.54(k) based on the following information included in the application: licensed operators and senior operators that the applicant initially assumes will staff the MCR. The number of licensed operators and senior operators satisfies the minimum required by 10 CFR 50.54(m)(1), 10 CFR 50.54(m)(2)(ii), and 10 CFR 50.54(m)(2)(iii) for control room staff and on-site staff. Requirements in 10 CFR 50.54(m)(2)(i) state that for a single unit operated from a single control room, two senior operators and two operators must be on-site when the unit is operating. DCD Tier 2, Section 18.1.1.1 describes the initial staffing level assumed for the main control room. It states that there will be two senior operators, which will be the shift supervisor and the STA, and there will be three operators, which will be the reactor operator, the turbine operator, and the electrical operator. Therefore, the initial staffing assumption will help to ensure a licensee can comply with the requirements in 10 CFR 50.54(m)(2)(i). However, other portions of the DCD that described staffing did not fully describe the qualification requirements of the STA. Specifically, those portions indicated that the STA would have an engineering degree, but they did not indicate that the STA would be a senior operator. The staff considered this to be an inconsistency in the application. Therefore, on August 21, 2015, the staff issued RAI 107-8039, Question 18-11 to address this issue (ML15234A008). On September 15, 2015, the applicant provided a response to RAI 107-8039, Question 18-11 (ML15258A646). The applicant confirmed that the STA would also be a senior operator and submitted a proposed revision to the HFE PP, Section 4.1 to align it with DCD Tier 2, Section 18.1.1.1. The staff confirmed that HFE PP was revised as committed in the response to RAI 107-8039, Question 18-11. Therefore, RAI 107-8039, Question 18-11 was resolved and closed. requirement in 10 CFR 50.54(m)(2)(iii) as a constraint for MCR staff level. The final staffing level for the control room results from the S&Q analyses; however, because of the constraint, the number of licensed operators in the control room cannot be lower than that required by 50.54(m)(2)(iii). The requirements of 10 54 CFR 50.54(k) will also be satisfied because compliance with 10 CFR 50.54(m)(2)(iii) ensures compliance with 10 CFR 50.54(k). The Basic HSI TeR, Section 3.5.1 describes the staffing concept for the MCR. Some members of the MCR staff may leave the control room at times to perform tasks outside the control room. However, the staffing concept acknowledges the staffing constraint, and therefore ensures that the requirements of 10 CFR 50.54(m)(2)(iii) and 10 CFR 50.54(k) can be satisfied. Requirements in 10 CFR 50.54(m)(2)(iv) state that a senior operator must directly supervise core alternations (including fuel loading or transfer). This is an administrative requirement that is not dependent on the design, and therefore the staff concludes that the design does not preclude a licensee from meeting the requirement. Thus, the staff finds the applicant has sufficiently addressed the staffing requirements in 10 CFR 50.54(m) such that an applicant for an operating license or combined license will be able to conform to the conditions in the license. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 3 The applicant should use the results of the task analysis as an input to the staffing and qualification analyses. Personnel tasks, addressed in task analysis, should be assigned to staffing positions to ensure that jobs are defined considering: the task characteristics, such as the knowledge and abilities required, relationships among tasks, time required to perform the task, and estimated workload; assigned responsibility; and teamwork and team processes, such as peer checking. 3: the first step, a subject matter expert collects all the task analysis data, including the elements listed in the first bullet of the acceptance criterion, and completes an examination of the related tasks making judgments regarding staffing numbers and qualifications. The second step links tasks into sequences which are assigned to specific staff positions. The staff position is then assessed for the ability to maintain situation awareness and teamwork. The staff concludes that the task analysis data is being appropriately collected and input into the S&Q analysis. The S&Q analysis appropriately uses the task analysis information to assess staffing level and qualification. Therefore, the staff finds that the application conforms to this criterion. Criterion 4 f operations personnel for the full range of plant conditions and tasks, including operational tasks (under 55 normal, abnormal, and emergency conditions), plant maintenance, plant surveillance, and testing. 4 broad spectrum of evolutions representative of all plant operating modes (Modes 1 through 6) under normal, abnormal, and emergency conditions are identified and analyzed for their impact on S&Q. The IP provides a proprietary list of criteria that define how these evolutions are identified. Generalized, these criteria place a priority on analyzing: HEDs affecting S&Q identified in the previous HFE design elements including the OER and the task analysis, evolutions presenting the greatest challenge to the operations crew, and evolutions containing important human actions. Plant maintenance, plant surveillance, and testing activities are included in the criteria used to determine evolutions presenting the greatest challenge to the operations crew. The staff concludes that the analysis method used for establishing staffing and qualification levels addresses the full range of plant conditions and activities and has appropriate prioritization criteria to identify those most challenging to staffing and qualifications. The method appropriately uses the input from the previous HFE program elements. Therefore, the staff finds that the application conforms to this criterion. Criterion 5 modified as information from the HFE analyses from other elements becomes available. S&Q generated by other HFE program elements are evaluated for any potential changes needed in these analyses. Similarly, plant design changes are evaluated for their impact on the output of all HFE program elements, including the output of the S&Q. The staff concludes that the S&Q methodology provides for modification of the staffing goals via the HED resolution process. Therefore, the staff finds that the application conforms to this criterion. Criterion 6 The applicant should address the basis for staffing and qualification levels considering the specific staffing-related issues noted below. These considerations may be identified in other HFE elements or in related source documents as follows: Operating Experience Review o operational problems and strengths resulting from staffing levels in predecessor designs 56 o initial staffing goals and their bases, including staffing levels of predecessor designs and a description of significant similarities and differences between predecessor and current designs o staffing considerations described in NRC Information Notice 95-48, o possible impact on staffing of requirements of limits to work hours, required break times, and required days off, as specified in 10 CFR Rule o Regulatory Issue Summary (RIS) 2009-10, Communications Between the NRC and Reactor Licensees During Emergencies and Significant Events Functional Requirements Analysis and Function Allocation o potential mismatches between functions allocated to personnel and their qualifications o and HFE aspects Task Analysis o time needed to perform a task, and the workload involved o personnel communication and coordination, including interactions between individuals for diagnosing, planning, and controlling the plant, and interactions between personnel for administrative, communications, and reporting activities o the job requirements resulting from the sum of all tasks allocated to each individual inside and outside the control room o potential decreases in the ability of personnel to coordinate their work due to changes to the plant o availability of personnel considering other work that may be ongoing, and for which operators may be responsible outside the control room (e.g., fire brigade) o -0654; and procedures to implement an initial accident response in key functional areas, as denoted in the emergency plan o staffing considerations described by the application of ANSI/ANS 58.8- -Related Operator Acti applicant Treatment of Important Human Actions o the effect of staffing levels on the performance of the identified important HAs 57 o the effect of staffing levels on personnel coordination for important HAs o NUREG/CR-Contribution to Risk in Operating Procedure Development o staffing demands resulting from requirements to concurrently use multiple procedures o personnel knowledge, abilities, and authorities identified in the procedures Training Program Development o concerns about coordinating personnel that are identified during the development of training program elements (OER, FRA/FA, TA, and human reliability analysis [HRA]) act as inputs to the S&Q analyses and provides for the consideration of the manning level considerations identified in the acceptance criterion. The initial staffing and qualification levels are derived from the Shin Kori 3&4 reference plant, the Palo Verde Nuclear Generating Station predecessor plant, and the System 80+ predecessor design. The HFE program elements identify weaknesses in these predecessor plant configurations and document these as HEDs and the HEDs are evaluated within the S&Q analysis. The HED resolution process may modify the staffing level or staff qualification requirements. Criteria within the OER, task analysis, and TIHA elements were not addressed within the IP. Therefore, on August 21, 2015, the staff initiated RAI 107-8039, Question 18-12 to address this issue (ML15234A008). On October 07, 2015, the applicant provided a response to RAI 107-8039, Question 18-12 (ML15280A151) which included a markup of the IP showing how the omitted criteria would be addressed. The added information is consistent with this criterion; therefore, the response is satisfactory. The staff confirmed that S&Q IP was revised as committed in the response to RAI 107-8039, Question 18-12. Therefore, RAI 107-8039, Question 18-12 was resolved and closed. and qualifications that address inputs from the OER, FRA/FA, TA, HRA, HSI design, procedures development, and training development. Therefore, the staff finds that the application conforms to this criterion. Combined License Items There are no COL items associated with Section 18.5 of the APR1400 DCD. Conclusion conforms to the criteria in NUREG-0711, Section 6.4. Therefore, the staff concludes that the ing and qualifications analysis method provides for systematically analyzing the 58 required number and necessary qualifications of personnel, in concert with regulatory requirements and task requirements. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element. 18.6 Treatment of Important Human Actions Introduction The TIHA program element identifies the human actions (HAs) that are most important to safety and considers those HAs in the HFE design of the plant. The design should minimize the likelihood of personnel error and help ensure that personnel can detect and recover from any errors that occur. Probabilistic and deterministic analyses are used to identify IHAs. The probabilistic risk assessment (PRA), which includes HRA, identifies risk-important human actions (RIHAs). Deterministic engineering analyses identify deterministically important human actions (DIHAs) that are credited with the prevention or mitigation of accidents and transients. Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of -K-X-IT-14001. Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2methodology described in APR1400-E-I-NR-ment of Important Human Actions into the HFE program. Changes to Tier 2 information are governed by the change control processes in the design certification revaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs. ITAAC: There are no ITAAC associated with this element. TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: TeRs associated with this element are: APR1400-E-I-NR- (ML18178A202) APR1400-E-I-NR-(ML18212A336) APR1400-E-I-NR- (ML18081A107) 59 APR1400-E-I-NR-IP) (ML18178A202) Regulatory Basis NUREG-0800. The applicable regulatory requirements are as follows: 10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) Other regulatory guidance documents are as follows: NUREG- NUREG- of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human lity Technical Evaluation NUREG-criterion addresses plant modifications and is not applicable to new reactors; therefore, the staff evaluated the first three criteria as discussed below. Criterion 1 The applicant should identify risk-important HAs from the PRA/HRA. -the RIHAs and their associated HFE characteristics, as identified in the PRA documentation, are included in the HFE design program. Section 4.1 states that [ ] Although this NUREG-0711 criterion is limited to identifying the RIHAs, the staff concludes that documenting the assumptions associated with each RIHA and confirming them during other HFE program elements (e.g., HSI design and integrated system validation) is a good practice that provides an opportunity to validate assumptions in the PRA. The TIHA IP, Section 4.1 states, [ 60 ] The staff reviewed DCD, Revision 0, Chapter 1912 and found that it -Tables 19.1-24, -25, -38, -39, -55, -56, -61, -62, -72, -73, -78, -79, -104, -105, -117, -118, -130, - 131, -145, -146, -159, and -160. The risk-significant operator actions have been identified using two different importance measures: risk achievement worth (RAW) and Fussell-Vesely (FV) importance. Because the risk-significant human actions (i.e., the RIHAs) have already been identified in DCD Tier 2, Chapter 19, the staff did not understand why it was necessary to defer identification of the RIHAs to the COL applicant when the information was provided as part of the design certification application. Additionally, the staff did not find any information about HFE characteristics for the RIHAs in DCD Chapter 19, and the staff determined it was necessary to ask the applicant to clarify in the TIHA IP where the information is located. Therefore, on August 5, 2015, the staff issued RAI 128-7980, Questions 18-13 and 18-14, to address this issue (ML15227A008). On December 18, 2015, the applicant provided a response to RAI 128-7980, Questions 18-13 and 18-14 (ML15352A246). The applicant proposed a revision to the TIHA IP to include a list of the RIHAs and its associated HFE characteristics from the PRA documentation as well as references for the information containing the HFE characteristics in Appendix C in the next revision of the TIHA IP. The staff reviewed the list of RIHAs provided in response to RAI 128-7980, Question 18-13, and compared it to the tables of RIHAs in DCD Tier 2, Chapter 19. The staff found that the list of RIHAs provided in response to RAI 128-7980, Question 18-Tier 2, Chapter 19, but a few of the RIHAs in DCD Tier 2, Chapter 19, were not included. The staff reconsidered whether it was prudent to have two different parts of the application identify RIHAs because if the information in DCD Tier 2, Chapter 19 is revised during later stages of the review, then even if the information in the TIHA IP is initially the same as that in the DCD, that may not be true in the future. Differences between the list of RIHAs in the TIHA IP, Appendix C and in the DCD Tier 2, Chapter 19 could cause confusion for the COL applicant when performing the activities in the TIHA IP, which could result in some RIHAs not being included in the HFE design program. As, such, the staff requested the applicant to instead add more detail to the PRA documentation an SME with no PRA knowledge would be able to look at the right documents and identify RIHAs reliably and accurately. Therefore, on August 1, 2016, the staff issued RAI 510-8650, Question 18-127 to address this issue (ML16214A307). On August 26, 2016, the applicant provided a response to RAI 510-8650, Question 18-127 (ML16239A436), which contained a proposed revision to the TIHA IP, Section 4.1, to say that the RIHAs would be identified from the PRA documentation by [ ] The proposed revision also added additional sources of the HFE characteristics to the list of references in the TIHA IP. These sources include the HRA documentation prepared for the design certification. Additionally, the applicant retained the proposed Appendix C to the TIHA IP, but the applicant clarified that the information in the appendix is preliminary information only. The staff concluded the proposed revisions were acceptable to ensure the RIHAs and their associated HFE characteristics will be included in the HFE design program for two reasons. 12 Section 19.1 of this SER RIHAs and performing the HRA. 61 First, SMEs that do have [ ] are necessary to ensure the RIHAs and their associated HFE characteristics are correctly identified from the PRA documentation (i.e., DCD Tier 2, Chapter 19 and the PRA-related documents listed in the TIHA IP). Second, the TIHA IP states that the source of the information duplicated in Table 4-1, which is the input to the rest of the HFE design program, is the PRA documentation (i.e., DCD Tier 2, Chapter 19 and the PRA- IP. The staff confirmed that the TIHA IP was revised as committed in the response to RAI 510- 8650, Question 18-127. Therefore, RAI 510-8650, Question 18-127 was resolved and closed. During the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member questioned why the PRA developed for the design certification will be used to perform the activities described in the TIHA IP given that the COL applicant will perform these activities, and the COL applicant will develop a site-specific PRA and HRA. Also, an ACRS member questioned whether the level of expertise of the SMEs who will complete Table 4-1 is sufficient to ensure that the RIHAs and their associated HFE characteristics will be included the other HFE program elements. Additionally, an ACRS member discussed that some aspects of the site-specific PRA (i.e., the quantification of seismic risk) may not be determined until fuel load, which occurs after the control room has been constructed. The ACRS member questioned how any RIHAs identified as a result of quantifying the seismic PRA will be addressed in the HFE design program. Following the ACRS meeting, the staff reevaluated the TIHA IP and determined it did not fully conform to guidance in NUREG-0711 in the following three ways: NUREG-updated iteratively as the design progresses (including the final PRA/HRA) to determined it was necessary to request the applicant specify that the COL applicant use the site-specific PRA rather than the PRA developed for the generic design to perform the activities described in the TIHA IP because the site-specific PRA will contain more information than the generic PRA. Also, given that the COL applicant should use the site-specific PRA to complete Table 4-1, the staff requested the applicant remove Appendix C, which was based on the design certification application PRA results. NUREG-0711, Section 7.1 states, HRA is an integral part of a completed PRA. Applicants submit PRAs in potential for, and mechanisms of human error that might affect plant safety. Thus, it is an essential feature in assuring the HFE program goal of generating a design to minimize personnel errors, support their detection, and ensure recovery capability. The HRA is an integrated activity supporting both the HFE design and PRA activities. The robustness and quality of the HRA largely depends on the analyst's understanding of the causes, modes and probabilities of human error, the personnel tasks to be performed, information about those tasks, and any task-specific factors that may influence the human performance of them. Analysts should employ the descriptions and analyses of personnel functions and tasks, along with the operational characteristics of the HSIs. The HRA provides 62 valuable insights into the desirable characteristics of the HSI design. Consequently, the HFE design should pay special attention to those plant scenarios, risk-important HAs, and HSIs that the PRA/HRA highlights as vital to plant safety and reliability. As explained in NUREG-0711, the results of the HRA provide input to both the PRA and HFE design activities. Specific expertise is necessary to develop the not specifically include HRA expertise, will be able to fully understand and apply the HRA results to the HFE design program. As such, the staff determined it was necessary to ask the applicant to specify that SMEs with HRA knowledge will also be necessary to accurately complete Table 4-1. Additionally, the staff could not find where the applicant explained whether or how any RIHAs identified as a result of quantifying the seismic PRA will be addressed in the HFE design program. Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-133 to address these three issues (ML17249A979). On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-133 (ML17271A188). As part of the response, the applicant proposed the following changes to the application: Section 4.1 of the TIHA IP will be revised to specify that the site-specific PRA will be used to complete Table 4-1 of the TIHA IP. DCD Tier 2, Section 18.6.2 and Section 4.1 of the TIHA IP will be revised to explain that I&C engineering subject matter experts (SMEs) will coordinate with personnel who have PRA/HRA knowledge to extract the RIHAs from the PRA documentation and complete Table 4-1. The TIHA IP will also be revised to provide context for the preliminary output in Appendix C. Specifically, the TIHA IP will be revised to state the following: ] The staff concludes that these changes to the application ensure that Table 4-1, which provides input to the other HFE elements in the HFE design program, is completed by SMEs with adequate expertise to reliably and correctly identify the RIHAs and their associated HFE characteristics. Further, because site-specific PRA documentation will be used, the most current PRA information available when the TIHA activities are completed will be used. PRA documentation and included in the HFE design program is acceptable. [ 63 Additionally, as part of the response to RAI 553-9084, Question 18-133, the applicant addressed treatment of additional RIHAs that may be identified from quantification of the seismic PRA. The applicant stated: As stated in Section 3.5.8 of the TIHA IP, Revision 1, the design implementation (DI) PE demonstrates that the as-built HSI design, including the HSI for IHAs, for each site specific APR1400 reflects the HSI design output from the human factors verification and validation (HF V&V) PE. Therefore, if additional IHAs are identified for a specific plant, and those IHAs are not encompassed by the HF V&V PE, HEDs are generated. Those HEDs are resolved as part of the DI PE, using HED resolution process of the HFE PP. The design changes after DI PE The staff findthe HFE IPs, including identification of RIHAs for inclusion in the HFE design process, must be completed by the COL applicant in order to close the HFE-related ITAAC identified in DCD Tier Section 50.71(h)(1) of 10 CFR, requires a COL applicant to quantify seismic risk by fuel load, and therefore, it is possible that potentially important contributions to risk that affect hardware design and human factors engineering would not be identified until the plant has been constructed. As such, changes to the plant design, plant operating procedures, or the plant personnel training programs that may be needed following identification of any new RIHAs from the quantification of the seismic risk will need to be addressed by implementation of operational programs, such as the corrective action program, plant procedure development program, and plant personnel training program, which are subject to appropriate change control processes and NRC inspection. Based on the review of the DCD and TIHA IP, the staff has confirmed incorporation of the changes described above; therefore RAI 553-9084, Question 18-133 was resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 2 The applicant should identify deterministically-important HAs from the following licensing analyses: Operator actions credited in the DCD Chapter 15 accident and transient analyses Operator actions identified in the D3 coping analyses (D3CA) performed for DCD Chapter 7, as specified in Section 1 and 2 of Interim Staff Guidance DI&C-ISG-02, Diversity and Defense in Depth (D3) Issues (NRC, 2009) Unlike DCD Tier 2, Chapter 19, which contains tables that label and list the operator actions that are risk-significant (i.e., the RIHAs), the operator actions that are required to mitigate accidents described in the D3CA and DCD Tier 2, Chapter 15, are described in the event evaluation ing SME will review the transient and accident analyses (TAA) in DCD Tier 2, Chapter 15 and the diversity and defense- in-depth coping analysis (D3CA) of DCD Tier 2, Chapter 7, to identify the DIHAs to be included in the HFE design process. The SMEs will [ 64 ] The HFE PP, Section 5 lists the qualifications of the plant operations and systems safety engineering SMEs. Plant operations SMEs are required to have previous experience operating a nuclear power reactor and to have completed an extensive amount of training on the operation of nuclear power plants during transients and accidents. Systems safety engineering SMEs are required to have previous experience in safety system engineering. The staff concludes that SMEs with these qualifications will have knowledge of how safety systems operate and the actions that operators need to take to operate the safety systems, and therefore SMEs with these qualifications will be able to identify the DIHAs in the D3CA and DCD Tier 2, Chapter 15, and include them in the HFE design process. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 3 The applicant should specify how important HAs are addressed by the HFE program, in Function Allocation, Task Analysis, HSI design, Procedural Development, and Training Program Development, in order to minimize the likelihood of human error and facilitate error-detection and recovery capability. functional requirements analysis/function allocation, task analysis, HSI design, procedural development, and training program development program elements to minimize the likelihood of human error and facilitate error-detection and recovery capability as described below. Functional Requirements Analysis/Function Allocation: The HFE PP, Section states that the function allocation confirms that the actions the PRA, TAA, and D3CA analyses assume will be performed by human actions (i.e., the IHAs) have been appropriately assigned to humans by applying the allocation criteria ction Allocation Selection likelihood of human error during performance of the IHA increases. If the function allocation process concludes that the actions should be automated either partially or fully in order to ensure the safe operation of the plant, then an HED will be documented and resolved in accordance with the HFE PP, Section 4.6. Because the IHAs are reviewed during the function allocation process to confirm that IHAs have been appropriately allocated to human actions, the staff Task Analysis: DCD Tier 2, Section 18.4.2.1 states that a BTA and a TTA will be performed for each task that is an IHA. The BTA identifies the HSI that must be available for operators to perform the IHA, which is an input to the HSI design program element. The TTA provides an estimate of the workload and time margin associated with performing each IHA. The workload and time margin are then compared to acceptance criteria defined in the TA IP, Section 4.3.1, do not meet the acceptance criteria, an HED will be documented and resolved in accordance with the HED resolution process described in the HFE PP, Section 65 4.6. HEDs may result in changes to the HSI to help minimize the likelihood of human error or facilitate error-detection and recovery capability, as necessary. Because IHAs are assessed during the task analysis process to ensure that they are within acceptable human performance capabilities, the staff finds that the includes the list of IHAs and their associated HFE characteristics. These characteristics include the alarms, controls, and indications (i.e., the HSI) that the analyses in the PRA, TAA, and DC3A assume will be available to operators to perform actions that are required to mitigate the consequences of transients, design basis accidents, and beyond-design basis accidents. The HFE PP, the TIHA ReSR is an input to the HSI design. This ensures that the HSI credited in the plant analyses is included in the HSI inventory in the plant. Because the information about the HSI required to support the performance of IHAs is included in the HSI design process to ensure that the HSI that operators need is addresses IHAs. Procedure Development: Procedures provide step-by-step directions for performing tasks in the plant, including IHAs. Detailed procedures and use of procedure adherence during task performance help to minimize human errors. DCD Tier 2, Section 18.4.2.1 states that tasks that are IHAs are included in the task analysis process, and a BTA and TTA will be performed for each IHA. Task narratives are developed during the BTA, and they specify what personnel have to do and relationships among tasks. The HFE PP, Section 4.7.3.3 states that the task narratives are inputs to procedure development to identify the actions in the procedures and the sequence of those actions in the procedures. Thus, procedures will be developed that direct how to perform IHAs in the plant. process adequately addresses IHAs. Training Program Development: Because all IHAs are included in the task analysis process, a BTA will be developed for each task that is an IHA. The TA tive developed during the BTA identifies the knowledge and abilities that plant personnel must have to perform the task. The HFE PP, Section 4.7.3.3 describes that the learning objectives included in the training program will be derived in part from the task narratives produced during the task analysis. Systematic analysis of tasks, which occurs during task analysis, and the development of learning objectives from that analysis are two of the five elements of a systems approach to training13. In DCD Tier 2, Section 13.2.1, applicant to develop a site-specific training program by using NEI 06-13A, systems approach to training method. Including IHAs in the operator training program helps to ensure that operators are trained to correctly perform those 13 66 actions, which helps to minimize human error. Therefore, the staff finds that the The applicant also described that the IHAs are addressed in other HFE program elements, including V&V. NUREG-personnel error and assure error detection and recovery capability when errors occur. V&V IP, included in the ISV scenarios. Therefore, the staff concludes that the applicant addresses and incorporates IHAs in the function allocation, task analysis, and HSI design program elements, as well as in the V&V Program Element, in order to minimize the likelihood of human error and facilitate error- detection and recovery capability. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Combined License Items There are no COL items associated with Section 18.6 of the APR1400 DCD. Conclusion conforms to the criteria in NUREG-0711, Section 7.4. Therefore, the staff finds there is HFE design. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element. 18.7 Human System Interface Design Introduction The HSI design element represents the translation of function and task requirements into HSI design specifications. The objective of this review is to evaluate how HSI designs are identified and refined. The review verifies that the applicant has a process to translate functional and task requirements to the detailed design of alarms, displays, controls, and other aspects of the HSI through the systematic application of HFE principles and criteria. Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of APR1400-K-X-IT-14001. Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2-System Interface that summarizes the HSI design process described in APR1400-E-I-NR-14007, -to develop the HSI design. 67 Changes to Tier 2 information are governed by the change control processes in the design evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs. ITAAC: There are no ITAAC associated with this element. TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: TeRs associated with this element are: APR1400-E-I-NR--System Interface D(HD IP) (ML18178A202). The HD IP describes the method that will be used to develop the APR1400 HSI Design. APR1400-E-I-NR--(ML18178A202). The Basic HSI TeR describes the APR1400 Basic HSI conceptual design. It is an input to the method described in the HD IP to develop the APR1400 HSI Design. APR1400-E-I-NR-Guide contains HFE guidance and design-specific conventions applicable to all HSI resources (e.g., displays and controls used to operate the plant) and facilities, including the MCR, RSR, and TSC. It is applicable to the APR1400 HSI Design. APR1400-E-I-NR-(ML18212A336). APR1400-E-I-NR- (OER IP) (ML18081A107). APR1400-E-I-NR- (ML18081A107). APR1400-E-I-NR-IP) (ML18178A202). APR1400-K-I-NR-IP) (ML17094A129). APR1400-E-I-NR-Implementation Pla (ML18178A202). APR1400-E-I-NR- (ML18178A202). APR1400-Z-J-NR- (ML18212A336). APR1400-Z-A-NR- (ML18086B746). 68 APR1400-Z-J-NR- (ML18212A336). Regulatory Basis NUREG-0800. The applicable regulatory requirements are as follows: 10 CFR Part 50, Appendix A, General Design Criteria (GDC) 19 control room 10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) 10 CFR 50.34(f)(2)(iv) safety parameter display system (SPDS) 10 CFR 50.34(f)(2)(v) automatic indication of the bypassed and operable status of safety systems 10 CFR 50.34(f)(2)(xi) relief and safety valve indication 10 CFR 50.34(f)(2)(xii) auxiliary feedwater system flow indication 10 CFR 50.34(f)(2)(xvii) containment related indications 10 CFR 50.34(f)(2)(xviii) core cooling indications 10 CFR 50.34(f)(2)(xix) instrumentation for monitoring post-accident conditions that includes core damage 10 CFR 50.34(f)(2)(xxvi) leakage control 10 CFR 50.34(f)(2)(xxvii) radiation monitoring Other regulatory guidance documents are as follows: NUREG-- NUREG-- 2 NUREG-Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human -System Interface NUREG-0800, Appendix 18-in Diversity and Defense-in- 0 NUREG-0835, NUREG-1342, and Supplement 1 of NUREG-0737 relating to safety parameter display system requirements, as described in 10 CFR 50.34(f)(2)(iv) 69 Bypassed and inoperable status indication for nuclear power plant (NPP) safety for Nuclear Power Plant Safety Instrumentation for light water cooled nuclear power plants to access plant and environmental conditions during and following an accident, as described in Functional criteria for emergency response facilities, as described in NUREG-0696 Technical Evaluation 18.7.4.1 HSI Design Inputs NUREG-Criterion 1 Analysis of Personnel Task Requirements The applicant should use the following analyses, performed in earlier stages of the design process, to identify requirements for the HSIs: Operational Experience Review An input to the HSI design should encompass lessons learned from other complex human-machine systems, especially predecessor designs and those involving similar HSI technology. Functional Requirements Analysis and Function Allocation The HSIs should support the roles of personnel in the plant, e.g., appropriate levels of automation. Task Analysis The set of requirements to support the role of personnel is provided by task analyses that should identify: o tasks needed to control the plant during a range of operating conditions from normal through accident conditions o detailed information and control requirements (e.g., requirements for display range, precision, accuracy, and units of measurement) o task support requirements (e.g., special lighting and ventilation requirements) o Important Human Actions (IHAs), as defined in Section 7 of NUREG-0711, that should be given special attention in the HSI design process Staffing and Qualifications The findings from analyses of staffing/qualifications should provide input for deciding upon the layout of the overall control room and allocating controls and displays to individual consoles, panels, and workstations. The staffing/qualifications analyses establish the basis for the minimum and maximum number of personnel to be accommodated, and requirements for coordinating activities between them. 70 The HD IP describes the process that will be used by a COL applicant to develop the APR1400 HSI Design, which will be verified and validated in accordance with the V&V IP. The starting point for the development of the APR1400 HSI Design is the APR1400 Basic HSI conceptual design, which the applicant described in the Basic HSI TeR14. As discussed in more detail in in DCD Tier 2, SKN 3&4, which are the plants that best represent the APR1400 design. The process described in the HD IP that a COL applicant will use to develop the APR1400 HSI Design from the APR1400 Basic HSI conceptual design includes using the results of the analyses listed in this criterion as follows. ues identified during the OER, which is conducted in accordance with the OER IP, may require changes to the APR1400 Basic HSI conceptual design. As described in the OER IP, Section 4.8, reviewing relevant operating experience will be documented and evaluated to determine whether any changes must be made to the APR1400 HSI Design. changes will be made when the operating experience indicates it is necessary to protect the health and safety of the plant staff. Therefore, the results of the OER will identify inputs to the APR1400 HSI Design by identifying changes that may need to occur to address lessons learned identified by the OER. Functional Requirements Analysis (FRA) and Function Allocation (FA): The HD describes that the results of the FRA, which includes the identification of the controls and displays operators need to monitor and control the critical safety functions, is used to develop the HSI inventory, which includes controls and displays, for the APR1400 HSI Design. [ ] Therefore, the results of the FRA and FA will identify inputs to the APR1400 HSI Design. 14 The APR1400 Basic HSI conceptual design is referred to as TeR. 71 how the results of the TA are used to develop the APR1400 HSI Design. TA encompasses tasks that are necessary to control the plant for the full range of operating conditions, from normal through accident conditions, during normal and degraded HSI conditions. This includes the tasks necessary to execute the important human actions (IHAs). TA generates detailed information and control requirements (ICRs) (e.g., requirements for display range, precision, accuracy, units of measurement) that are implemented during HD in the APR1400 HSI. TA also generates task support requirements (e.g., special lighting, ventilation requirements) that are incorporated into the APR1400 HSI Design. Therefore, the results of the TA will identify inputs to the APR1400 HSI Design. Staffing and Qualifications (S&Q): DCD Tier 2, Table 18.5-Qualification Assumptions for the and qualifications assumed for the control room operations staff. The Basic HSI TeR, Figure 3-incorporated into the control room layout for the APR1400 Basic HSI conceptual qualifications will be determined by the COL applicant using the process described in the S&Q IP. The results of the S&Q analysis will either confirm the initial staffing level and qualifications are sufficient or identify modifications to the number and qualifications of the APR1400 operations staff. If the S&Q analysis identifies a modification to the initial staffing level or qualifications are necessary, then the issue will be tracked as a human engineering discrepancy (HED). The -resulting from the S&Q analysis may result in changes to the APR1400 HSI Design. Therefore, the S&Q analysis is an input to the APR1400 HSI Design. Therefore, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 2 System Requirements The applicant should identify any constraints on the HSI design imposed by the overall I&C system (e.g., constraints on the information that can be presented due to sensor data availability). imposed by the I&C system are significant inputs for the HSI design. The applicant provided [ 72 Thus, the APR1400 Basic HSI conceptual design described in the Basic HSI TeR encompasses I&C system constraints the applicant identified during the development of the APR1400 Basic HSI conceptual design. Because the APR1400 Basic HSI conceptual design is the starting point for the development of the APR1400 HSI Design that will be developed in accordance with the HD IP, the APR1400 HSI Design also encompasses the I&C system constraints that were identified during the development of the APR1400 Basic HSI conceptual design. Therefore, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 3 Regulatory Requirements The applicant should identify the applicable regulatory requirements as inputs to the HSI design process. Evaluation of Criterion 3 regulatory requirements that are inputs to the HSI design process. The staff compared this list to the list of regulatory requirements in the NUREG-- requirements and also applicable regulatory guidance as inputs to the HSI design process. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 4 Other Requirements The applicant should identify any other requirements, such as customer requirements, that are inputs to the HSI design. by the applicant for a COL applicant to conduct a test of the APR1400 Basic HSI conceptual design during the HSI design process using operators licensed in the US and a simulator. [ ] ] Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. 73 18.7.4.2 Concept of Use and HSI Design Overview NUREG-criteria for this topic. Criterion 1 The applicant should develop a concept of use stating the roles and responsibilities of operations personnel based upon anticipated staffing levels. The concept of use should: provide a high-level description of how personnel will work with HSI resources, and address the coordination of personnel activities, such as interactions with auxiliary operators and the coordination of maintenance and operations. The Basic HSI TeR, Section 3.5.1 describes the roles and responsibilities of operations personnel based on an anticipated control room staffing level of five licensed operators (Section ol room staffing) and provides an overview of how control room personnel will work with the HSI resources. The main HSI available in the MCR include the operator consoles (OCs), the safety console (SC), and the large display panel (LDP). These are described below. The [ 74 Accordingly, information about plant status is readily available and accessible to each control room operator at his or her workstation. The SC in the MCR is available to allow the operators [ The MCR is also equipped with communication systems to allow for coordination of plant activities with personnel outside the MCR. The MCR includes a [ ] The staff concludes that the applicant has provided a description of the operator roles and responsibilities and how the HSIs support these roles and responsibilities. Also, the staff concludes that the applicant has addressed how plant activities will be coordinated and how the HSIs facilitate this coordination. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 2 The applicant should provide an overview of the HSI, covering the technical bases demonstrating that they constitute a state-of-the-art HSI design supporting personnel performance. These bases may include analyses of operating experience and the literature, tradeoff studies simultaneously considering multiple alternatives, and engineering tests and evaluations. The overview should include a description of: facility layouts, including workstations, large screen displays, and the nominal staff working positions; ] ] ] The LDP is [ 75 key HSI resources and their functionality, such as alarms, displays, controls, CBPs, and other support and job aids; technologies to support teamwork and communication within the MCR and between the MCR, the remote shutdown facility, the TSC, EOF, and LCSs; and the responsibilities of the crew for monitoring, interacting, and overriding automatic systems and for interacting with computerized procedures systems and other computerized operator support systems. The Basic HSI TeR provides an overview of the APR1400 Basic HSI conceptual design, and it includes the following topics. The facility layouts and technologies to support teamwork and communication: This includes viewing angles, physical configuration, and presentation formats. The concept includes the LDP, OCs, and SC. The Basic HSI TeR, Section 3.5.4, at the applicant selected to support teamwork and coordination among personnel from within and outside of the MCR. For example, within the MCR, [ ]. Outside of the MCR, the SPDS provides the same information to staff in the RSR, the TSC and EOF, which means that emergency response personnel will be able to monitor changes in plant status and alert the rest of the members of the emergency response organization, including the operators in the control room. Additionally, the Basic HSI TeR, Section 3.5.4.states that [ Therefore, the overview of the HSI includes a description of the facility layouts and the technologies that support teamwork and communication. The key HSI resources, including the alarms, displays, controls, and CBPs, and their functionality; and the responsibilities of the crew for interacting with automatic systems: The Basic HSI TeR, by virtue of the design descriptions, reinforces that the primary role of the operator is to monitor automatic actuations to ensure that they function properly and to take manual action as directed by procedures if a malfunction occurs. Alarm logic, LDP indications, and the CBPs ] 76 support the operator in this role. The Basic HSI TeR, Section 4.11.4actions are initiated, and automatic actions are reset. The CBPs assist the operator in determining when to take these actions. Therefore, the overview of the HSI includes a description of the key HSI resources and the responsibilities of the crew for interacting with automatic systems. The HFE guidelines and standards that will be applied to the APR1400 HSI Design: Style Guide applies to all HSI and facilities in the scope of the HSI design process, and DCD Tier 2, NUREG-0700. NUREG-0700 contains acceptable industry standards for HFE ms to the HFE guidelines in NUREG-0700 are discussed in Section 18.7.4.4 of this report. The conformance of the APR1400 HSI Design to the HFE standards and guidelines in the Style Guide will be verified during verification and validation (V&V). Therefore, the overview of the HSI includes a description of the HFE guidelines that apply to the design. APR1400 Basic HSI conceptual design: [ ] 77 APR1400 Basic HSI conceptual design is the starting point and basis for the APR1400 HSI Design. The HD IP describes the process that will be used to develop the APR1400 HSI Design from the APR1400 Basic HSI conceptual design. The HD IP, Section 4.1.7 describes that this process includes [ . states: ] Thus, the staff concludes that the applicant has provided an overview of the HSI and described the technical bases. The overview includes a description of the facility layouts, key HSI resources, technologies to support teamwork and communication, and responsibilities of the operators for interacting with automatic systems. The staff also concludes that the APR1400 Basic HSI conceptual design is an acceptable starting point for a state-of- the-art HFE design for two reasons. First, the APR1400 Basic HSI concept evolved from a predecessor design previously approved by the staff, and it also incorporated operating experience from other designs as well as the results of HSI tests and evaluations. Second, the applicant developed a Style Guide applicable to the APR1400 Basic HSI conceptual design and the APR1400 HSI Design based on NUREG-0700, which contains acceptable industry standards for HFE design. The staff also finds that the HSI evaluations that will be performed in accordance with the HD IP provide additional assurance that US operating practices will be incorporated in the APR1400 HSI Design and provide an opportunity to implement technology improvements. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. 18.7.4.3 HFE Design Guidance for HSIs NUREG-topic. Criterion 1 The design, and address their form, function, and operation, as well as the environmental conditions in which they will be used that are relevant to human performance. T The staff reviewed the Style Guide and found that it addresses the scope of the HSIs included in are used for performing IHAs). The Style Guide contains generic HFE guidance derived from NUREG-0700 as well as design-specific HFE standards and guidance that address the form, function, and operation of the HSI resources. The Style Guide addresses the environmental conditions for MCR HSIs, but not for other HSIs that are listed in the Style Guide, Section 1.1, [ 78 -8481, Question 18-96 to address this issue (ML16022A233). On April 16, 2016, the applicant provided a response to RAI 374-8481, Question 18-96 (ADAMS Accession No.ML16107A016). In the response, the applicant stated that the Style Guide would be revised to specify that the same guidance for environmental conditions for the MCR will also apply to the RSR. This is acceptable because the RSR is provided as an alternative to the MCR when the MCR is uninhabitable. Also, the Style Guide will be revised to include the guidance from NUREG-0700 for the environmental conditions for the LCSs. The applicant also stated that the environmental conditions in the Style Guide for the MCR are f this document, the EOF and TSC will be designed in accordance with NUREG-0696, which is an acceptable alternative to NUREG-0700 for emergency response facilities. The staff confirmed that the Style Guide was revised as committed in the response to RAI 374-8481, Question 18- 96. Therefore, RAI 374-8481, Question 18-96 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 2 c HFE guidance and HSI design-in addressing specific goals of the HSI design. The Basic HSI TeR, Section 2 states that the APR1400 Basic HSI conceptual design includes the Style Guide, which defines the design standards for the APR1400 Basic HSI. The staff reviewed the Style Guide and determined that the information contained most of the HFE guidance identified in NUREG-0700, which contains generic HFE guidance. Some criteria were not addressed, and it was not clear to the staff why they were omitted. Therefore, on December 22, 2015, the staff issued RAI 353-8372, Questions 18-83 through 18-88 to address this issue (ML15356A486). On March 21, 2016, the applicant provided a response to RAI 353-8372, Questions 18-85 and 18-88, (ML16081A345). On April 12, 2016, the applicant provided a response to RAI 353-8372, Question 18-86 (ML16133A562). On April 22, 2016, the applicant provided a response to RAI 353-8372, Questions 18-83, 18-84, and 18-responses state the omitted criteria were either not applicable to the design or were overlooked in the Style Guide. The applicant provided revisions to the Style Guide to include the criteria that are applicable to the design. The staff finds the response acceptable as it provides for a complete set of applicable HFE design criteria consistent with NUREG-0700 guidance. With respect to the Style Guide additions, change pages were provided with the RAI response that acceptably address the omissions. The staff confirmed that the Style Guide and Basic HSI TeR were revised as committed in the response to RAI 353-8372, Questions 18-83 through 18-88. Therefore, RAI 353-8372, Questions 18-83 through 18-88 were resolved and closed. The Style Guide also includes four appendices that show how the generic HFE guidelines in the Style Guide have been tailored to the design of the APR1400 Basic HSI conceptual design. For -specific labeling guidelines for 79 the APR1400 Basic HSI conceptual design that are based on generic labeling guidelines in the Style Guide, Section 7.5, Accordingly, the staff concludes that the Style Guide incorporates generic HFE guidance in NUREG-decisions (i.e., the APR1400 Basic HSI conceptual design). The staff finds the application conforms to this criterion. Criterion 3 The guidelines in the style guide(s) should be sufficiently detailed so that design personnel can deliver a consistent, verifiable design meeting the applicant's guidelines. The staff reviewed the Style Guide and found that it contains individual guidelines that are written precisely and describe easily observable HSI characteristics. For example, the Style Guidewill be depicted on displays. Accordingly, the staff concludes that the guidelines in the Style Guide are sufficiently detailed so that design personnel can deliver a consistent, verifiable design meeting the applicant's guidelines. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 4 guidance will be used in the overall design process. They should be written so designers can readily understand them; the text should be supplemented with graphical examples, figures, and tables to facilitate comprehension. Thare included in the APR1400 HSIS. This section states that the HSI design method described in the HD IP ensures that the HSI resources incorporate the HSI inventory identified in the task analysis process and from plant system design documentation in accordance with the guidance in the Style Guide. The HD IP, Sections 4.2.1, documentation. The method includes verification that the HSI design products (e.g., a drawing of a system display for the reactor coolant system) will conform to the guidance in the Style Guide as well as to the guidance in the nomenclature and labeling guide and the component control and instrumentation design guide. figures. Guidance in the main body of the Style Guide directs the user to refer to the specific guidance contained in these appendices. 80 Because the design process instructs design personnel on where and how the HFE guidance in the style guides will be used, and because the Style Guide contains detailed figures and examples, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 5 The applicant should maintain the style guide(s) in a form that is readily accessible and usable by designers, and is easily modified and updated as the design matures. The guidance should include a reference(s) to the source upon which it is based. includes NUREG-0700. The HFE PP, Section 4.4.4 states that all HFE design program documents, including the Style Guide, will be retained in the Review and Comments System for the life of the program. The Review and Comments System allows access to all HFE documents, including the Style Guide, to personnel involved in the HFE design program. Human engineering discrepancies (HEDs) are used to document and resolve issues identified during the development of the HFE design. HEDs are tracked using the ITS. The HFE PP, Section 4.6 states that [ ] The staff concludes that as the design matures, design personnel can use HEDs to document any changes that need to be incorporated into the HFE documents, including the Style Guide. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. 18.7.4.4 HSI Detailed Design and Integration The criteria in this section are divided into several subsections. General NUREG-criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first seven criteria as discussed below Criterion 1 occur, and maximize the probability that any error made will be detected. -design minimizes the probability that errors will occur when personnel perform IHAs: the use of [ ] These are described in detail below. as follows: 81 alarms and indications will be displayed with SDCV HSI features. SDCV HSI features are dedicated, fixed displays on the LPD and the SC in the MCR and on the LPD in the TSC. SDCV HSI features display [ ] The use of dedicated screens on the LDP to alert personnel of plant conditions that require IHAs to stabilize the plant promotes situational awareness of the plant condition among all of the control room operators. Ensuring that the prompting alarms for IHAs are visible to all operators at all times maximizes the probability that the operator(s) will see the alarms and use appropriate alarm response procedures to begin taking actions to stabilize the plant. The HD IP, Secti that [] Some IHAs may need to be performed from LCSs outside of the MCR. The HFE standards and conventions identified in the Style Guide, the Nomenclature and Labeling Guide, and the Component Control and Instrumentation Design Guide are applied to the design of the MCR as well as to the design of LCSs that will be used by operators to perform IHAs to maintain consistency in the HFE design of the HSI in these locations where operators may have to perform IHAs. Therefore, the applicant described features of the HSI design that will minimize the probability that errors will occur and maximize the probability that any error made will be detected when operators perform tasks that are IHAs. Therefore, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 2 The applicant should base the layout of HSIs within consoles, panels, and workstations on: (1) analyses of personnel roles (job analysis), and (2) systematic strategies for organization, such as arrangement by importance, and frequency and sequence of use. and controls are organized within the IFPD information display hierarchy. Operators normally access the displays at the IFPDs that form the OCs, and the displays that can be viewed on the [ ] 82 IFPDs can also be displayed on the variable portion of the LDP. The IFPD display inventory includes [ systematic strategies for organization. [ ] Thus, each operator may arrange the displays at his or her workstation based on the importance of the HSI to the task(s) that must be performed and also on the frequency that he or she needs to use the HSI. portion of the LDP. The SDCV indications include the key parameters that allow operators to [ ] The variable portion of the LDP allows the operators to select the displays that the crew determines are important for facilitating crew coordination when performing different tasks. Thus, the applicant has arranged the LDP into a fixed portion that contains the indications that are most important to assessing plant safety and plant power production and a variable portion that the operators can customize to support performance of a given task. for the controls and displays allocated to the SC. The SC controls and displays include those HSIs that allow operations to continue, including plant shutdown and accident mitigation, under degraded HSI ] 83 conditions that result in a loss of all of the OCs. Therefore, the applicant based the layout of HSIs on the SC to ensure that plant operations can continue during degraded HSI conditions. Thus, the applicant based the layout of HSIs within consoles, panels, and workstations on personnel roles. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 3 The applicant should design the HSIs to support inspection, maintenance, test, and repair of: (1) plant equipment, and (2) the HSIs. The applicant should design the latter so that inspection, maintenance, test, and repair of the HSIs do not interfere with other plant-control activities (e.g., indications). example of the indication that will be appear on the IFPDs to inform operators about plant equipment that is unavailable for maintenance, testing, or repair. The staff concluded that the method of indication is located next to the applicable plant equipment in a way that does not obscure view of other plant indications. Additionally, the APR1400 Basic HSI TeR, CR that are provided to allow for the conduct of maintenance and testing on the systems that provide signals to safety-related equipment to operate when needed under abnormal and emergency conditions. Also, the TA IP, Section 4.1, states that tasks that are important to plant safety that are undertaken during maintenance, tests, inspections, and surveillances are included in the TA process. The TA results, in addition to the piping and instrumentation diagrams (P&IDs) for the plant systems, which show instrumentation that is used for testing and inspection, are inputs to the HSI design process. Accordingly, the staff finds that the HSI design has considered the inspection, maintenance and repair, and testing requirements for the plant equipment in a way that does not interfere with plant-control activities. redundancy incorporated into the MCR design helps to ensure that the repair of HSIs will not interfere with plant control. There are multiple, identical OCs available in the MCR that can be used to operate plant equipment, and there are redundant control methods. Therefore, having redundant displays and controls available ensures that if one method of display or control fails, then the operators can use another method in the control room Therefore, the staff concludes that the maintenance and repair of the HSIs will not interfere with plant control activities. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 4 minimum-, typical-, and high-level or maximum staffing. 84 NUREG-0711 provides additional information for this criterion as follows: typical staffing is that used for routine plant operations, minimum staffing is that defined by the plant technical specifications, and maximum staffing includes augmented staff for accident situations (e.g., plant personnel who staff the EOF and TSC). Identificatioas follows: A fundamental assumption of the APR1400 HFE design is that it is possible to operate the plant during postulated plant operating modes (modes 1 through 6) for normal, abnormal, and emergency conditions, with the following personnel in the main control room (MCR): one reactor operator (RO) with a reactor operator license, one turbine operator (TO) with a reactor operator license, one electric operator (EO) with a reactor operator license, one shift supervisor (SS) with a senior reactor operator (SRO) license, and one shift technical advisor (STA) with a senior reactor operator license. Thus, two senior operators and three operators comprise the expected or typical number of [ ] Therefore, the design supports task performance for the typical staffing level because each operator has his or her own workstation, which can be used to perform any task he or she may be assigned to perform. which states that the number of operators onsite may be less than the minimum requirement of 10 CFR 50.54(m)(2)(i) for a period of time not to exceed 2 hours in order to accommodate unexpected absence of on-duty shift crew members provided immediate action is taken to restore the shift crew composition to within the minimum requirements (i.e., two senior operators the minimum control room staffing requirement in 10 CFR 50.54(m)(2)(iii) as an additional APR1400 staffing constraint (i.e., one senior operator in the control room and either one other senior operator or an operator at the controls when the plant is in an operational mode other than cold shutdown or refueling as defined by the plant technical specifications). Further, as that help to ensure the typical staffing level is maintained while the plant is operating. When the plant is operating, there are more tasks to perform compared to when the plant is not operating, and therefore keeping the staffing at the typical level helps to ensure that the workload is not excessive for the control room personnel. If only the minimum number of operators is on shift when the plant is operating, it is possible that the workload for the staff in the control room could be higher than when the control room is staffed with the typical number of operators. As ment, the integrated system validation (ISV) 85 confirms that the workload is acceptable under a wide variety of plant conditions for both the minimum and typical staffing level. If the workload is not acceptable, or if the design does not support task performance, then HEDs will be documented and resolved in accordance with the HED resolution process. It may be necessary to change an aspect of the HSI design to resolve such HEDs. Therefore, the performance of the ISV and resolution of such HEDs in accordance with the V&V IP provides assurance that the design supports task performance during conditions of minimum and typical staffing. allow emergency response personnel to perform their tasks when required. Because these facilities are external to the MCR and will have voice communications with the operators in the MCR, the staff finds that these aspects of the design account for maximum staffing levels while still enabling the control room team to have the space and resources they will need to operate the plant. Accordingly, the staff finds that the application conforms to this criterion. Criterion 5 where decrements in human performance due to fatigue may be a concern. Decrements in human performance due to fatigue are a concern in nuclear power plants because the operators typically work 8-12 hours shifts, and the operators typically rotate nal awareness, which can make an operator less effective at monitoring indications of plant status. e performance of the critical functions as well as plant system-level alarms and component-level alarms of high priority to ensure the operators are informed when there are changes in the plant status that require attention. As described in the Style Guideand audible indication to the operators when there is a condition that requires operator action(s). By providing audible and visual indications in the MCR, the alarm system helps to ensure that operators maintain awareness of abnormal conditions. Accordingly, the staff finds that the application conforms to this criterion. Criterion 6 range of environmental conditions, ranging from normal to credible extreme conditions, such as consider the ambient environment (e.g., noise, temperature, contamination) and the need for and type of protective clothing. NUREG-important effects on operators' performance include thermal comfort, illumination, the auditory enlayout is included in Criterion 2 in this section of this document. The Basic HSI TeR, 86 -specific guidelines that conform to the guidelines in NUREG-0700 that help to ensure that the MCR design supports human performance under the full range of environmental conditions by ensuring that adequate lighting and ventilation are available during normal and credible extreme conditions, such as loss of normal lighting and normal ventilation. Additionally, tambient environment of the RSR. However, guidance for the ambient environment for LCSs was missing in the Style Guide. Therefore, on January 22, 2016, the staff issued RAI 374-8481 Question 18-96 to address this issue (ML16022A233). On April 16, 2016, the applicant provided a response to RAI 374-8481, Question 18-96 (ML16107A016). The applicant stated that it will revise the Style Guide to specify the guidance for the ambient environment at LCSs. Specifically, the revision to the Style Guide will include the guidance for environmental -0700. Thus, the applicant considered the ambient environment for the LCSs and the RSR. The staff confirmed that the Style Guide was revised as committed in the response to RAI 374-8481, Question 18-96. Therefore, RAI 374-8481, Question 18-96 was resolved and closed. [ ] Thus, the applicant provided guidelines in the Style Guide that will be implemented to support human performance under a range of environmental conditions and task support needs. Therefore, the staff finds that the application conforms to this criterion. Criterion 7 The applicant should identify how in an operating plant: the HSIs are modified and updated; temporary HSI changes are made, such as modifying the set points; and personnel-defined HSIs are created, such as temporary displays that personnel define for monitoring a specific situation. The staff did not find in the application how the applicant addressed Criterion 7. Therefore, on January 22, 2016, the staff issued RAI 374-8481 Question 18-101 to address the issue (ML16022A233). On May 20, 2016, the applicant provided a response to RAI 374-8481, Question 18-101 (ML16142A038). In the response, the applicant identified how, in an operating APR1400 plant, modifications to the HSIs, will be [ ] This is consistent with other operating reactors in the United States. In the RAI response, the applicant also described how temporary changes can be made to HSIs when the plant is operating. The applicant stated that operators can [ 87 The applicant stated in the RAI response that it will revise: (1) the HFE PP to clarify that the HD IP covers HSI design changes for operating APR1400 plants, (2) the HD IP to clarify the distinction between features of the HSI design that permit plant operators to create new HSI inventory (i.e., custom displays and alarms), without changing existing HSI inventory, and actual changes to existing HSI inventory, and (3) the Basic HSI TeR to describe the custom display changes to HSIs, how temporary changes to the HSIs can be made, and how personnel-defined HSIs can be created. On July 23, 2018, the applicant provided a revised response to RAI 374- 8481, Question 18-101 (ML18204A406). In the revised response the applicant removed a statement in the HFE PP that contradicted some of the information provided in the initial RAI response. Based on the review of the HFE PP, HD IP, and Basic HSI TeR, the staff has confirmed incorporation of the changes described above; therefore RAI 374-8481, Question 18- 101 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Main Control Room NUREG-Criterion 1 Safety Parameter Display System The applicant should describe the safety parameter display system (SPDS), addressing the following: Identification of Critical Safety Functions (CSFs) The CSFs needed to meet the requirement for an SPDS should be identified. NUREG-1342 Section III.F, Minimum Parameters for Display, lists the five CSFs that personnel monitor using an SPDS for boiling water reactor (BWRs) and pressurized water reactor (PWRs). For new designs, applicants should verify that these CSFs are suitable for their design, identifying any changes needed based on theicharacteristics. CSFs may differ for non-light water reactor designs, such as high-temperature gas-cooled reactors and liquid-metal reactors. Identification of the Parameters Personnel will use to Monitor Each CSF The applicant should identify the plant parameters personnel need to monitor each CSF and describe the means by which plant data are synthesized, combined, or otherwise evaluated to provide the information presented in the SPDS display. Section III.F of NUREG-1342 has guidance on acceptable parameters for the ] 88 Evaluation of SPDS HSIs The applicant should verify that the SPDS HSIs conform to acceptable HFE practices using NUREG-0700, Section 5 and other SPDS HFE guidance. The applicant described the SPDS as follows. Identification of Critical Safety Functions (CSFs): The FRA/FA IP, Section 3.2.1, states: The applicant also identified the APR1400 CSFs listed in the FRA/FA IP, Table 3-1, in CSFs in NUREG-SCSFs, including the five listed in NUREG-1342, Section III.F. The staff observed that the APR1400 CSFs are the same as the CSFs for the System 80+ design, with one exception. The exception is that the System 80+ design includes a CSF for individual CSFs: one CSF for containment temperature and pressure control and a separate CSF for containment combustible gas control (i.e., radioactive emissions control). This is appropriate because as described in NUREG--Water-Cooled Nuclear Power Plants to Assess Plant and need to monitor to maintain the CSFs. Containment pressure is used to monitor the containment environment. Separating the CSF for containment environment into the two CSFs in the APR1400 specifies that containment pressure as well as containment combustible gas and temperature will be used to determine whether the environment in containment is acceptable to ensure its integrity. Therefore, the applicant identified the APR1400 CSFs that are the starting point for the FRA to be performed by the COL applicant in accordance with the FRA/FA IP; however, the applicant expects these CSFs will be confirmed because no changes are expected to be made to the CSFs during the FRA. The applicant also identified changes from the [ ] 89 System 80+ that are applicable to the APR1400 design. Accordingly, the staff finds that the application conforms to the first bullet of this criterion. Identification of the Parameters Personnel will use to Monitor Each CSF: The Basic HSI [ ] IEEE Standard 497-Criteria for Accident Monitoring Instrumentation for Nuclear Pprovide primary information to the control room operators to assess the critical safety functions. IEEE Standard 497-2002 also describes a method for identifying Type B applicant stated that its accident monitoring instrumentation is designed to meet the guidance of RG 1.97, Revision 4. In DCD Tier 2, Table 7.5-accordance with RG 1.97 is in Chapter 7.5 of this document. Thus, the parameters used to monitor each CSF are, at a minimum, the Type B variables in DCD Tier 2, Table 7.5-1. SPADES+ application program, which synthesizes and evaluates plant data (i.e., parameters) that operators need to monitor the CSFs and sends this information to the CSF displays that are viewable in the MCR, RSR, EOF, and TSC. The SPADES+ application program is executed by the non-safety related Information Processing hat the safety related Qualified Indication and Alarm System- P (QIAS-P) receives Type B variables from the plant protection system (PPS), engineered safety features - component control system (ESF-CCS), and the core protection calculator system (CPCS) via a safety system data network (SDN) and auxiliary process cabinet - safety (APC-S) and process instrumentation via a hardwired -P is in Chapter 7 of this document. Because the parameters have been identified and the applicant has described the means by which plant data are synthesized and evaluated, the staff finds that the application conforms to the second bullet of this criterion. Evaluation of SPDS HSIs: The Basic HSI TeR, Section 4.9 and the HD IP, the IPS, [ ] 90 NUREG-15. Criterion 5.1-1: Plant parameters and variables important to safety should be displayed in a way that is convenient and readily accessible. [ ] Accordingly, the staff finds that plant parameters important to safety are displayed in a way that is convenient and readily accessible. Criterion 5.1-2: Critical safety function displays should be readable from the workstations of users needing access to these displays. [ ] Accordingly, the staff finds that the CSF displays are readable from the workstations of the users needing access to these displays. Criterion 5.1-3: Critical plant variables and parameters should be displayed to help users evaluate the plant's safety status. The HD IP, Section 3.2.1 state that the [ ] Accordingly, the staff finds that the critical plant parameters that help users evaluate safety status are displayed. Criterion 5.1-4: The display system should display information about severe accident symptoms associated with the plant safety parameters and functions. The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the SPDS displays. One of provides the design basis for the SPDS HSIs. DCD Tier 2, Section 7.5.1.2, ameters that provide unambiguous indication of inadequate core cooling (ICC) and warning of SPADES+ program displays information about each of the ICC variables on the 15 NUREG-described in 10 CFR 50.34(f)(2)(iv), and related guidance in NUREG-0835, NUREG-1342, Supplement 1 of NUREG-0737, and NUREG-0700, Section 5. These NUREGs discuss SPDS, with NUREG-0700 being the primary one; the others encompass supplemental guidance, in NUREG-0700, Section 5, and referred to the other guidance documents if necessary to clarify the guidance in NUREG-0700, Section 5. 91 CSF displays in the MCR and the TSC. Data generated by SPADES+ is also transmitted to the EOF. Because the SPADES+ program displays information about the ICC variables, which provide indication of severe accident conditions, the staff finds that the display system displays information about severe accident symptoms. Criterion 5.1-5: Critical plant variables should be displayed in a concise format. The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the CSF displays. [ ] Accordingly, the staff finds that the critical plant variables (i.e., the CSFs), are displayed in a concise format. Criterion 5.1-6: The display's response to transient and accident sequences should keep the user informed of the current plant status. The SPADES+ program monitors plant variables indicative of CSF status and uses algorithms to determine the status of each CSF. [ Accordingly, the staff finds that the SPDS HSIs keep users informed of the current plant status. Criterion 5.1-7: Critical safety function displays should allow users to comprehend a change in safety status in a matter of seconds. The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the SPDS displays. [ ] However, it was not clear to the staff how this will be accomplished. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-116 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-116 (ML16133A607). In the response, the applicant stated that the [ 92 ] The staff confirmed that the Style Guide was revised as committed in the response to RAI 400-8425, Question 18-116. Therefore, RAI 400-8425, Question 18-116 was resolved and closed. Accordingly, the staff finds that the SPDS HSIs keep users informed of the current plant status. Criterion 5.1-8: The sampling rate for each critical plant variable should be consistent with the users' needs for performing tasks. The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the SPDS displays. However, it was not clear to the staff how this will be accomplished. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-116 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-116 (ML16133A607). [ ] The staff confirmed that the Style Guide was revised as committed in the response to RAI 400-8425, Question 18-116. Therefore, RAI 400-8425, Question 18-116 was resolved and closed. Accordingly, the staff finds that the sampling rate is consistent with u Criterion 5.1-9: Each critical variable should be displayed with sufficient accuracy for the user to discriminate between normal conditions and those affecting plant safety status. The HD IP, Section 4.2.1 describes the inputs that will be used to create the detailed design documents that will be used to build the SPDS displays. [ ] However, it was not clear to the staff how this will be accomplished. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-116 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-116 (ML16133A607). In the response, the applicant stated that [ ] The Style Guide will be revised to include this guidance. The staff finds it acceptable that the applicant provided a design-specific guideline for accuracy that will be included in the design that is tested during V&V. Whether the accuracy is sufficient for user to discriminate between normal conditions and abnormal conditions will be determined in the ISV. is discussed in Section 18.10.4.3 of this document. The staff confirmed that the Style Guide was revised as committed in the response to RAI 400-8425, Question 18-116. Therefore, RAI 400-8425, Question 18-116 was resolved and closed. 93 Criterion 5.1-10: The display should provide magnitudes and trends for critical plant variables or derived variables. The HD IP, Section 4.2.1 describes the design basis (i.e., design characteristics) for the CSF displays that will be used to create the detailed design documents that will be used to build the SPDS displays. [ ] Accordingly, the staff finds that the CSF displays provide trends for CSF variables. Criterion 5.1-11: Displays for monitoring safety parameters and functions should continuously display this information. The Basic HSI TeR, Figure 4-21, the [ ] Accordingly, the staff finds that the CFM/BISI provide trends for CSF variables. Criterion 5.1-12: Where plant operating modes impose different demands, separate display pages should be provided for each mode. The HD IP, Section 4.2.1 describes the design requirements that must be included in the detailed design documentation for the SPDS displays. Table 1.1- Accordingly, the staff finds that separate display pages will be provided for each of the different modes of operation. Criterion 5.2-1: The system should assist the user in monitoring critical parameters, especially parameters that change very rapidly or very slowly, by alerting the user when values are out of range. DCD Tier 2, monitors the status of the CSFs during normal, abnormal, and emergency operating conditions and provides alarms when any of the CSFs are not being maintained within range. Accordingly, the staff finds that the SPDS program alerts users when values are out of range. Criterion 5.2-2: Where feasible, the system should provide perceptual (audible or visual) cues to alert personnel to abnormal operation conditions that potentially warrant corrective action. [ ] Accordingly, the staff finds that the SPDS will provide perceptual cues to alert personnel to abnormal conditions that warrant corrective actions. Criterion 5.2-3: While viewing secondary (lower-level) displays, a perceptual (audible or visual) cue should be provided by the safety parameter or function monitoring system to alert the user to return to the primary (higher level) display format if significant information in that display requires user attention. The 94 ] SPADES+ program [ Accordingly, the staff finds that cues are available to alert the user to changes that require attention even when the operators is viewing lower level displays. Criterion 5.2-4: User interactions with the display system should be within the skill capability of the control room crew and should not significantly increase personnel workload. [ ] This is well within the skill capability of the control room crew. Accordingly, the staff finds that the user interactions with the display system are within the skill capability of the crew and will not increase workload. Criterion 5.3-1: The display should not give false indications of plant status. The processing of SPDS information is performed by the SPADES+ application program. The Information Processing System (IPS) executes all application programs, including SPADES+. DCD Tier 2, Section 7.7.1.4 describes the characteristics of the IPS. The IPS application functions are executed via redundant IPS servers. One server is the primary (active) unit and the other is a dedicated backup. If the primary IPS server experiences a failure, its dedicated backup server assumes all processing tasks of the failed unit. Additionally, the status of the CSFs will normally be viewed on the LDP and on the OCs; however, if the OCs or the LDP are not available, then the status of the CSFs can be viewed on the SC, which is the backup to these normal displays. Furthermore, as discussed below under Criterion 5.3-2, the IPS provides operators with indication of the quality of the data it is processing to indicate the reliability of the information. Accordingly, the staff finds that because there are redundant processors and redundant displays, and because the IPS provides indication of data quality that can be used to determine the reliability of the data, the SPDS HSIs provide sufficiently reliable indications of plant status if there are failures of a single processor or display or failures that affect data processing. Criterion 5.3-2: Critical plant variables should be reliable and should be validated in real time. The staff could not find information about validation of critical parameters. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-117 to address this issue (ML16035A020). On May 13, 2016, the applicant provided a response to RAI 400-8425, Question 18-117 (ML16134A573). The applicant stated data validation of the critical plant variables is performed by the IPS, which is described in DCD Tier 2, information available to the plant operating staff both on a real-time and historical Additionally, the applicant provided additional information about how the IPS monitors the quality of data provided to operators and how the quality of that data is communicated. The staff confirmed that DCD Tier 2 was revised as 95 committed in the response to RAI 400-8425, Question 18-117. Therefore, RAI 400-8425, Question 18-117 was resolved and closed. Criterion 5.3-3: The status of the data should be displayed to the operator with an appropriate data quality indicator. The staff could not find information about how data quality is indicated. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-117 to address this issue (ML16035A020). On May 13, 2016, the applicant provided a response to RAI 400-8425, Question 18-117 (ML16134A573). The applicant described how indications of data quality (i.e., good, fair, poor, and bad) will be displayed to the operator. The staff confirmed that DCD Tier 2 was revised as committed in the response to RAI 400-8425, Question 18-117. Therefore, RAI 400-8425, Question 18-117 was resolved and closed. Criterion 5.4-1: The location of displays for monitoring safety parameters and functions should not interfere with the normal movement of the control room crew. [ ] the staff finds that the location of the SPDS displays will not interfere with normal movement of the crew. Criterion 5.4-2: The display system should not interfere with visual access to other control room operating systems or with displays that are important to safe operation of the plant. [ ] the staff finds that the display system does not interfere with visual access to other important displays. Criterion 5.4-3: Display devices for monitoring safety parameters and functions should be labeled and readily distinguished from other devices. The Basic TeR, ] Accordingly, the staff finds that the SPDS displays are readily distinguished from other displays and devices. The applicant has described the SPDS, including the SPADES+ application program, which processes plant data and sends information about the status of CSFs to displays, [ ] The applicant has identified a minimum set of CSFs and parameters used to indicate the status of the CSFs, and the operators will be able to monitor trends of these parameters. ] Accordingly, the staff finds the SPDS HSI resources conform to the SPDS guidance in NUREG-0711 and NUREG-0700, and the application complies with 10 CFR 50.34(f)(2)(iv). Figure 4-[ 96 ] Criterion 2 Bypassed and Inoperable Status Indication The applicant should describe how the HSI assures the automatic indication of the bypassed and inoperable status of a safety function, and the systems actuated or controlled by the safety function. [10 CFR 50.34(f)(2)(v) - I.D.3] RG 1.47 includes the following guidance related to the display of bypassed and inoperable status of safety systems: The status indication should be in the MCR. Administrative procedures should be supplemented by an automatic indication system that shows, for each affected safety system or subsystem, the bypass or deliberately induced inoperability of a safety function, and the systems it actuates or controls. Provisions should be made allowing the operations staff to confirm that a bypassed safety function was properly returned to service. Annunciating functions for system failure and automatic actions based on the self-test or self-diagnostic capabilities of digital computer-based I&C safety systems should be consistent with the above bullets. The indication system for bypass and inoperable status should include the ability to ensure its operable status during normal plant operation to the extent to which the indicating and annunciating functions can be verified. Bypass and inoperable status indicators should be arranged such that personnel can determine whether it is permissible to continue operating the reactor. The control room of all affected units should receive an indication of the bypass for their shared system safety functions. The applicant described the Bypassed and Inoperable Status Indication (BISI) as follows. that the critical function monitoring/bypassed and inoperable status indication (CFM/BISI) displays on the SDCV portion of the LDP fulfill the requirements for the BISI. As shown in the Basic HSI TeR, Figure 4- [ the staff finds that the application conforms to the first bullet. describes how the BISI provides automatic indication of bypassed or inoperable safety systems: 97 The bypassed or inoperable condition of ESF [engineered safety features] components is communicated to the IPS [information processing system], which indicates a system-level bypa-level alarms are actuated when a component actuated by a protection system is bypassed or deliberately rendered inoperable. Because the status of safety components is displayed automatically at the system-level on the CFM/BISI display, the staff finds the application conforms to the second bullet. The staff could not find a description of the provisions for allowing the operations staff to confirm that a bypassed safety function was properly returned to service. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-118 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-118 (ML16133A607). [ ] The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 400-8425, Question 18-118. Therefore, RAI 400-8425, Question 18-118 was resolved and closed. DCD Tier 2, Section 7.5.1.3 describes that BISI is an indication system that provides indications and alarms. It does not perform automatic actions other than providing indications and alarms. The Basic HSI TeR, Section 4.7.2, that [ ] This feature allows operators to confirm the display is updating and operating. There are no self-test or automatic actions associated with the BISI. Therefore, this criterion is not applicable. The staff could not find a description of how the operators can verify the indication and annunciating functions of the BISI. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-118 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-118 (ML16133A607). [ ] The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 400-8425, Question 18-118. Therefore, RAI 400-8425, Question 18-118 was resolved and closed. inoperable status indicators should be arranged such that personnel can Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-118 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-118 (ML16133A607). [ 98 ] The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 400-8425, Question 18-118. Therefore, RAI 400-8425, Question 18-118 was resolved and closed. room of all affected units should receive an indication of the bypass for their Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-118 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-118 (ML16133A607). The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 400-8425, Question 18-118. Therefore, RAI 400-8425, Question 18-118 was resolved and closed. The staff finds that the application describes how the HSI assures automatic indication of the bypassed and inoperable status of a safety function and the systems associated with the safety function. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(v). Criterion 3 Relief and Safety Valve Position Monitoring The applicant should describe how the HSI indicates the position of the relief and safety valves (open or closed) in the control room [10 CFR 50.34(f)(2)(xi) - II.D.3] NUREG-0737, Section II.D.3, that this requirement applies to safety and relief valves in the RCS. DCD Tier 2, Section 5.4.14, -operated safety relief valves provide overpressure protection for the RCS. It also states that these valves are part of an emergency success path [ ] Thus, the position of the relief and safety valves will be indicated in the LDP in the MCR. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xi). Criterion 4 Manual Feedwater Control The applicant should describe how the HSI provides automatic and manual initiation of the auxiliary feedwater system, and indicates auxiliary feedwater system flow in the control room. [Applicable to PWRs only, 10 CFR 50.34(f)(2)(xii) - II.E.1.2] iterion 4 [ 99 ] Thus, the position of the relief and safety valves will be indicated in the LDP in the MCR. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xii). Criterion 5 Containment Monitoring and displays) inform personnel about: (A) containment pressure; (B) containment water level; (C) containment hydrogen concentration; (D) containment radiation intensity (high level); and (E) noble gas effluents for all potential, accident release points. [10 CFR 50.34(f)(2)(xvii) - II.F.1] [ ] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xvii). Criterion 6 Core Cooling The applicant should describe how the HSI provides unambiguous indication of inadequate core cooling, such as with primary coolant saturation meters in PWRs, and a suitable combination of signals from indicators of coolant level in the reactor vessel and in-core thermocouples in PWRs and BWRs. [10 CFR 50.34(f)(2)(xviii) - II.F.2] instruments are used to monitor inadequate core cooling (ICC): hot and cold leg reactor coolant system resistance temperature detectors, pressurizer pressure instruments, and core-exit thermocouples (CETs) for indication of primary coolant saturation margin and heated junction thermocouples (HJTCs) for indication of reactor vessel water level. The HD IP, Section 4.1.4.6, [ -P flat panel displays (PFPDs) on the SC provide Class 1E backup indication in case the primary display is unavailable. The PFPDs provide unambiguous indication of ICC by displaying saturation ]. 100 margin, reactor vessel level, and core exit temperature. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xviii). Criterion 7 Post-accident Monitoring The applicant should describe how the HSI assures monitoring of plant and environmental conditions following an accident including core damage. [10 CFR 50.34(f)(2)(xix) - II.F.3, and RG1.97] DCD Tier 2, Table 7.5-variables, by type, that allow plant personnel to monitor the plant and environmental conditions following design basis events16. -variables will be displayed to plant personnel. It states that [ ] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xix). Criterion 8 Auxiliary Heat Removal -- The applicant should describe how that necessary automatic and manual actions can be taken to ensure proper functioning of auxiliary heat removal systems when the main feedwater system is not operable. [Applicable to BWRs only, 10 CFR 50.34(f)(2)(xxi) - II.K.1.22] The APR1400 is a PWR, not a BWR. Therefore, this criterion is not applicable. Criterion 9 Reactor Level Monitoring The applicant should describe how the HSI gives a record of the -accident recording requirements. [Applicable to BWRs only, 10 CFR 50.34(f)(2)(xxiv) - II.K.3.23] The APR1400 is a PWR, not a BWR. Therefore, this criterion is not applicable. 16 A-E variables. 101 Criteria 10 Leakage Control The applicant should describe how the HSI provides for leakage control and detection in the design of systems outside containment that contain (or might contain) accident- source-term radioactive materials after an accident. [10 CFR 50.34(f)(2)(xxvi) - III.D.1.1] on of Criterion 10 [ lists several systems connected to the reactor coolant system that could potentially contain radioactivity following an accident. This section of the DCD specifically identifies alarms and indications the operators will need to detect and control leakage outside containment. However, the HD IP did not identify this information as an input to the HSI design process. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-122 to address this issue (ML16035A020). On May 23, 2016, the applicant provided a response to RAI 400-8425, Question 18-122 (ML16144A662). The applicant stated that the Therefore, the staff concludes that the applicant has described how the HSIs provide for leakage control and detection in systems outside containment. The staff confirmed that the HD IP Sections 3.2.2, 3.6.2, and 4.2.2, were revised as committed in the response to RAI 400-8425, Question 18-122. Therefore, RAI 400-8425, Question 18-122 was resolved and closed. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xxvi). Criterion 11 Radiation Monitoring The applicant should describe how the HSI provides appropriate monitoring of in-plant radiation and airborne radioactivity under a broad range of routine and accident conditions. [10 CFR 50.34(f)(2)(xxvii) - III.D.3.3] of Criterion 11 [ Additionally, DCD Tier 2, Table 7.5-radiation monitors as Type E variables. DCD Tier 2, Section the IPS, which provides indication on the IFPDs at the OCs, and by the QIAS-N at the SC. The HD IP states that other radiation indications and alarms are provided on the [ ] The staff did not find direction in the HD IP for the SMEs to include these indications as inputs to any of the HSIs. Therefore, on February 4, 2016, the staff issued RAI 400-8425, Question 18-124 to address this issue (ML16035A020). On May 12, 2016, the applicant provided a response to RAI 400-8425, Question 18-124 (ML16133A607). In the ] response, the applicant clarified [ ] DCD Tier 2, Table 11.5-and Effluent are part of the APR1400 plant systems, and Table 11.5-1 shows that alarms will be provided in the MCR for some radiation monitors listed in Table 11.5-1. These radiation monitors that will have alarms include alarms for radiation monitors that can be used for monitoring in-plant radiation and airborne radioactivity under a broad range of routine and accident conditions. The staff confirmed that the HD IP was revised as committed in response to RAI 400-8425, Question 18-124. Therefore, RAI 400-8425, Question 18-124 was resolved and closed. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion and complies with 10 CFR 50.34(f)(2)(xxvii). Criterion 12 Manual Initiation of Protective Actions The applicant should describe how the HSI supports the manual initiation of protective actions at the system level for safety systems otherwise initiated automatically. [RG 1.62] ion 12 [ ] the staff has reasonable assurance that the process described in the HD IP will result in a design that satisfies this requirement. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 13 Diversity and Defense-in-depth The applicant should describe how the HSI provides displays and controls in the MCR for manual, system-level actuation of critical safety functions, and for monitoring those parameters that support them. These displays and controls are independent of, and different from, the normal I&C. [I&C BTP7-19, Point 4] [ ] DCD Tier 2, Table 7.8---level actuation of 102 103 equipment needed to manage the CSFs and indications that provide for monitoring of the CSFs. describe how the DMA switches and DIS are diversity of this system is documented in Section 7 of this SER. Because the HD IP, Section 4.2.7 states that the [ ], the staff has reasonable assurance that the process described in the HD IP will result in a design that conforms to this criterion. Accordingly, the applicant has described how the HSI provides controls in the MCR for manual, system-level actuation of the equipment required to manage the CSFs, and displays for monitoring the CSFs that are independent and diverse from the normal I&C. Therefore, the staff finds the application conforms to this criterion. Criterion 14 Important HAs The applicant should describe how the HSI provides the controls, displays, and alarms that ensure the reliable performance of identified IHAs. Section 7 of NUREG-0711 discusses important HAs. IHAs are identified using the process described in the TIHA IP. This process results in a list of IHAs and the controls, displays (indications), and alarms that the operators will need to perform the IHAs reliably. This information is derived from DCD Tier 2, Chapters 7, 15, and 19. The list of IHAs and the controls, indications, and alarms will be documented and is an input to the task analysis process to develop the HSI inventory for tasks associated with IHAs. The HSI inventory developed during the task analysis is an input to the HSI design process. Using the HSI inventory, the HD IP identifies inputs to the design of the alarm system, task displays, controls in the MCR and at LCSs, and the CBPs, as follows: ] Reliable performance must be demonstrated for IHAs during the ISV conducted in accordance with the V&V IP. As described in the V&V IP, if IHAs are not performed successfully, then [ 104 HEDs will be documented and resolved in accordance with the HFE PP. This may result in design changes to the HSI to facilitate reliable performance of the IHAs. Accordingly, the applicant has described how the HSI provides the controls, displays, and alarms that ensure the operators can perform IHAs. Therefore, the staff finds the application conforms to this criterion. Criterion 15 Computer-Based procedure platform - -based procedures should be consistent with the design review guidance in NUREG--Based -ISG-5 (NRC, 2008). -APR1400 computer-based procedure system will conform to design standards in the Style Guide. The Style Guide, S-CBP systems that are consistent with the guidance in NUREG-0700, Section 8, with two exceptions. The Style Guide, Items 5.2.1.2(e) and (f), state that words from standard Korean should be used, and punctuation should conform to standard Korean usage. Therefore, on February 1, 2016, the staff issued RAI 383-8458, Question 18-104 to address this issue (ML16032A104). On March 25, 2016, the applicant provided a response to RAI 383-8458 and stated that the application will be revised to change the standard language from Korean to English. The staff confirmed that the Style Guide was revised as committed in response to RAI 383-8485, Question 18-104. Therefore, RAI 383-8485, Question 18-104 was resolved and closed. The staff reviewed the design of the computer-based procedure system using the review guidance in DI&C-ISG--based the CBPs are consistent with the guidance in DI&C-ISG-5, Section 1. The staff found that the design of the CBPs is consistent with the guidance in DI&C-ISG-5, Section 1, except for the following review criteria: #3, #4, #7, #15, #16, #17, #19, #22, and #24. Therefore, on February 1, 2016, the staff issued RAI 383-8458, Question 18-105 to address this issue (ML16032A104). On May 20, 2016, the applicant provided a response to RAI 383-8458, Question 18-105 (ML16142A045). The applicant described how the remaining criteria were addressed and stated that the Basic HSI TeR, Section 4.8 would be revised to include additional information to address the remaining criteria. The staff found that the additional information addressed the remaining criteria with the exception of #22. Therefore, on August 24, 2016, the staff issued RAI 518-8654, Question 18- 128 to address the issue (ML16237A410). On September 19, 2016, the applicant provided a response to RAI 518-8654, Question 18-128 (ML16263A430), and described why the CBP system is not able to change the approved procedure. Thus, the CBP will not be able to make changes to procedures that have been approved, which is consistent with Criterion #22 in DI&C- ISG-5, Section 1. RAI 518-8654, Question 18-128 was resolved and closed. The staff confirmed that the Basic HSI TeR was revised as committed in response to RAI 383-8485, Question 18-105. Therefore, RAI 383-8485, Question 18-105 was resolved and closed. the guidance in NUREG-0700, Section 8 and DI&C-ISG-5, Section 1. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. 105 Technical Support Center NUREG-seven criteria for this topic. Criterion 1 The applicant should describe how the HSIs give personnel the information needed to: -state and dynamic behavior before and throughout an accident so TSC personnel can guide the MCR operators in managing the abnormal conditions and mitigating the accident without interfering with the MCR activities; undertake the needed environmental- and radiological-monitoring functions of the EOF when it is not operational; offer technical support to personnel during recovery operations after an emergency; and provide reliable voice-communications facilities to the control room, the operations support center, the EOF, the NRC, and with state and local operations centers. Criterion 1 [ ] The staff concludes that because the information available to control room personnel and TSC personnel will be the same, and because the design of the displays will be the same in both locations and therefore information will be presented identically in both locations, personnel in the TSC will have the information required to analyze plant behavior, perform the functions of the EOF if necessary, and offer technical support to the operators. However, the HD IP did not address how the HSIs provide reliable voice communications. Therefore, on January 22, 2016, the staff issued RAI 373-8480 Question 18-93 to address this issue (ML16022A232). On April 5, 2016, the applicant provided a response to RAI 373-8480 Question 18-93 (ML16096A193). The applicant described the voice communications available in the TSC, which includes [ ], and stated that it would revise the HD IP to include the information. The staff confirmed that the HD IP was revised as committed in response to RAI 373-8480, Question 18-93. Therefore, RAI 373-8480, Question 18-93 was resolved and closed. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 2 The applicant should describe how the HSIs give personnel the information needed for: determining the p-state operating conditions before the accident, 106 ascertaining the transient conditions producing the initiating event, accident, reviewing the accident sequence, deciding upon appropriate mitigating actions, evaluating the extent of any damage, and 2 [ ] By having this information available, the TSC personnel can determine status before the event, ascertain transient conditions producing the event, gauge dynamic behavior during the event, review of the accident sequence by evaluating how plant parameters have changed since the accident began, and assess status during recovery. includes [ ] Thus, TSC personnel can use the indications of plant status and the procedures to decide the mitigating actions the MCR operators should perform. Also, phones can be used to communicate with emergency response personnel who perform tasks in the plant and can provide updates on the extent of damage in the plant. Thus, having access to the full inventory of HSIs that the MCR staff has and also providing a means of communicating with personnel outside of the TSC ensures that that emergency response personnel in the TSC have the information needed to perform the functions described in this criterion. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 3 The applicant should describe how the HSIs provide an SPDS that replicates the SPDS in the MCR (to improve the exchange of information between personnel in the MCR and the EOF). If the SPDS in the MCR is composed of multiple displays, then multiple displays also should be provided in the TSC. ] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 4 The applicant should describe how the HSIs provide as a minimum, the set of variables specified in RG 1.97, Revision 4, plus all sensor data and calculated variables not specified in [ 107 RG 1.97 but included in the data sets for the SPDS, for the EOF, or for transmission to offsite locations. ersonnel will have access to the [ ] provides displays for all AMI variables, which are the variables specified in RG 1.97. Also, the [ ] the TSC personnel have access to the same information as the control room personnel and the EOF personnel, the staff concludes that the TSC HSI provide the set of variables specified in RG 1.97, Revision 4, and the SPDS. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 5 The applicant should describe how the HSIs allow all TSC personnel to complete its assigned tasks with unhindered access to alphanumeric and/or graphical representations of: plant systems variables, in-plant radiological variables, meteorological information, and offsite radiological information. 5 listed in this criterion are included in the HSI inventory available to personnel in the TSC. The HD IP, Section 4.2.8that [ ] Because the workplace design in NUREG-0700 applies specifically to either the MCR or LCSs, the staff requested the applicant clarify whether these standards would also be applied to the workplace design of the TSC. Therefore, on January 22, 2016, the staff issued RAI 373-8480, Question 18-94 to address this issue (ML16022A232). On April 22, 2016, the applicant provided a response to RAI 373-8480, Question 18-94 (ML16113A419) and specified the criteria in NUREG-0700 that is applicable to the TSC, including criteria related to access to workstations that contain the information listed in this criterion. The staff confirmed that the HD IP was revised as committed in response to RAI 373-8480, Question 18-94. Therefore, RAI 373-8480, Question 18-94 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. 108 Criterion 6 The applicant should describe how the HSIs provide the trend-information displays and time- abnormal operating conditions. tates that the [ ] the IPS includes historical data storage, retrieval, and trending capability. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 7 The applicant should describe how HFE was incorporated into the TSC design to ensure that personnel easily understand and use the HSIs. hat the TSC will be designed to the same HFE standards as the MCR. Thus, the HFE guidelines in the Style Guide are applied to the HSIs in the TSC. The purpose of a Style Guide based on accepted HFE principles, such as those in NUREG-0700, is to provide design guidelines to ensure that accepted HFE principles are applied to the design of HSIs. Applying accepted HFE principles to the design of HSIs helps to personnel understand and use the HSIs. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Emergency Operations Facility The HFE PP, Section 2 states that a graded HFE approach will be applied to the EOF. NUREG-graded HFE approach applied approach to these other facilities. Therefore, on June 26, 2015, the staff issued RAI 54-7963, Question 18-5 to address the issue (ML15177A387). On September 11, 2015, the applicant provided a response to RAI 54-7963, Question 18-guidance provided in NUREG-and NUREG--design of the TSC and EOF instead of the full-scope described in NUREG-0711, since NUREG- 0696 and NUREG-0700 provide guidance specific to the design of emergency response facilities and human system interfaces, NUREG--staff has accepted a commitment to implement these criteria as an alternative to the NUREG- -7963, Question 18-5 was resolved and closed. However, the application did not include this information. Therefore, on January 22, 2016, the staff issued RAI 373-8480, Question 18-91 to address this issue (ML16022A232). 109 On May 13, 2016, the applicant provided a response to RAI 373-8480, Question 18-91 (ML16134A593). The applicant stated that Dwill be designed in accordance with the guidance in NUREG-0696 and NUREG-0700. SRP Chapter 18, Revision 3, Section I.6, states, The emergency operating facility (EOF) and technical support center (TSC) are also included within the scope of HFE reviews. Again, a graded approach is used. NUREG- 0711 provides specific direction on which review criteria are applied to these facilities. general HFE criteria for these facilities and the staff has accepted a commitment to implement these criteria as an alternative to the NUREG-0711 criteria. Thus, the applicant has revised the application to identify that commitment for the COL applicant to design the EOF in accordance with NUREG-0696. The staff confirmed that DCD Tier 2 and the HD IP were revised as committed in response to RAI 373-8480, Question 18-91. Therefore, RAI 373-8480, Question 18-91 was resolved and closed. Accordingly, the staff finds the application conforms to this NUREG-0711 criteria. Remote Shutdown Facility NUREG-o criteria for this topic. Criterion 1 The applicant should describe how the HSI provides a design capability for remote shutdown of the reactor outside the MCR. [10 CFR Part 50, Appendix A, GDC 19] In 10 CFR Part outside the control room shall be provided: (1) with a design capability for prompt hot shutdown of the reactor, including necessary instrumentation and controls to maintain the unit in a safe condition during hot shutdown, and (2) with a potential capability for subsequent cold shutdown -Shutdown Console Instrumentation and Controls for Hot Shutd- necessary indications and controls outside the control room to permit remote shutdown of the tion of the adequacy of these indications and controls to achieve safe shutdown is in Section 7.4 of this SER. DCD Tier 2, Section 7.4.1.l.3, available through the soft controls and displays on the four IFPDs and ESCMs in the Remote Shutdown Console (RSC) in the RSR. The IFPDs are the same as those in the MCR, and therefore the displays and controls on the RSC are the same type as those on the consoles in the MCR. DCD Tier 2, Figure 7.4-the RSC in the RSR. Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Criterion 2 The applicant should describe how the HSIs at the RSF are consistent with those in the MCR. 110 include one RSC [ ] Accordingly, the staff finds that the application conforms to this NUREG-0711 criterion. Local Control Stations NUREG-Criterion 1 The applicant should describe the basis for deciding which HSIs will be included in the MCR design, and which will be provided locally. of indications and displays is provided to the operators by the [ ] the HSI inventory) included in the MCR design are determined by the following sources: (1) the results of the function requirements analysis, task analysis, and TIHA program elements (e.g., the HSI inventory developed as part of the basic task analysis (BTA) for each task and the HSI that are assumed to be available in the transient and accident analyses); (2) regulatory requirements (e.g., the requirement in 10 CFR 50.34(f)(2)(v) for bypassed and inoperable status indication in the MCR); (3) plant design documentation (e.g., the plant system P&IDs are the basis for the system displays); and (4) the inventory of the predecessor plant (e.g., the system displays from the predecessor plant are the starting point for designing the system displays for the APR1400). The staff concludes that these sources provide a comprehensive and thorough basis for the HSI inventory that operators will need to perform their tasks in the MCR. inventory that will be available at LCSs are determined by the following sources: (1) the results of the task analysis and the TIHA program elements, which identify HSI inventory requirements for tasks that require local control, including tasks that are performed as part of IHAs; (2) plant design documentation; and (3) the LCSs of the predecessor plant. The staff concludes that these sources also provide a comprehensive and thorough basis for selecting the HSI inventory that is appropriately assigned to LCSs. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. 111 ] Criterion 2 The applicant should describe how HFE was incorporated into the HSIs for LCSs to ensure they are consistent with those in the MCR, and that personnel easily understand and use the HSIs. Licensed operators in the MCR do not typically perform actions outside of the MCR at LCSs. Rather, they typically direct non-licensed operators to perform actions outside of the MCR. However, it is possible that a licensed operator would perform an IHA at an LCS. The HD IP, Section 3.2.9 states that [ Applying the same HFE standards and guidelines used in the design of the HSIs in the MCR to the HSIs for LCSs used to perform IHAs for beyond-design basis events helps to ensure that when these actions are performed by licensed operators, they will be able to use them with ease. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. 18.7.4.5 Degraded I&C and HSI Conditions NUREG-topic. Criterion 1 The applicant should identify: the effects of automation failures and degraded conditions on personnel and plant performance HFE-significant I&C degradations; i.e., the failure modes and degraded conditions of the I&C system that might adversely affect the HSIs personnel use to accomplish important HAs The applicant identified effects of degraded conditions and automation failures of the non-safety control systems on plant performance in DCD Tier 2, Table 7.7-ntrol Groups for the NSSS applicant identified effects of degraded conditions and automation failures of the safety-related the Engineered Safety Features Actuation System). Additionally, the applicant identified the effects of a CCF of the digital safety I&C system during the postulated accidents (PAs) and abnormal operating occurrences (AOOs) described in DCD Tier 2, Chapter 15 on plant performance in the CCF Coping Analysis. A CCF of the digital I&C safety system during a PA or AOO is an HFE-significant I&C degradation because operators must use backup and diverse of this report. 112 The V&ISV will sample, and it includes [ ] Additionally, the V&V IP, Section 4.1.1.2, d in DCD Tier 2, Chapter 7, that must be performed by operators following a CCF of the digital safety I&C system that occurs during a PA or AOO. By including these types of degraded I&C conditions and automation failures in the ISV scenarios and collecting performance measurements described in personnel performance. If satisfactory personnel performance, as described in the V&V IP, scenarios, the applicant will document the issue as an HED and resolve the HED in accordance Therefore, the staff finds that the applicant has identified the effects of automation failures and degraded conditions, including HFE-significant I&C degradations, on plant performance. Also, the applicant will identify the effects of automation and degraded I&C and HSI conditions on personnel performance by including a sample of these conditions in the ISV. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 2 The applicant should specify the alarms and other information personnel need to detect degraded I&C and HSI conditions in a timely manner, and to identify its extent and significance. inventory. [ ] The applicant described the kinds of alarms that will alert operators following various modes of IFPD failures in its response to RAI 323- 8281, Question 07.03-19, dated March 15, 2016 (ML16075A425). In the response, the applicant also described that the IFPDs inform operators of the status of the safety I&C system via system status displays which are top level health displays on the IFPDs. The applicant also confirmed that Safety I&C System report, Sections 4.1.2.7 and 4.1.4 were revised as committed in the response to RAI 323-8281, Questions 07.03-19. Therefore, RAI 323-8281, Question 07.03-19 was resolved and closed. ] [ ] These indications allow the operators to verify that the HSI at their workstations is capable of communicating information. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. [ 113 Criterion 3 The applicant should determine any needed back-up systems to ensure that important personnel tasks can be completed under degraded I&C and HSI conditions. The applicant identified the following as backup methods available to operators under a range of degraded HSI conditions: if any IFPD at an OC is degraded, then the operator can use [ ] states that the [ ] The mini-LDP receives information from the non-safety qualified indication and alarm system (QIAS-N), which is a non-safety system that is diverse from the IPS. minimum inventory controls as well as [ - accident qualified indication and alarm system (QIAS-P) provides backup indication on the SC of indications of inadequate core cooling, which can be used if the IFPDs are not available. If there is a failure of the computer-based procedure system, then operators can use the [ ] as described in the In the event of a common-cause failure (CCF) of digital safety I&C systems including the plant protection system (PPS) and engineered safety features- component control system (ESF-CCS), the normal HSI will not be available to the operators to perform the IHAs required to mitigate an AOO or PA. Therefore, the applicant has included a diverse protection system (DPS), which includes a DAS, diverse manual engineered safety features (ESF) actuation (DMA) switches, and a diverse indication system (DIS). The DMA switches and DIS are available at the SC.] 114 Incorporating redundancy into the design of the MCR HSI ensures that the operators have reliable sources of information available during degraded HSI conditions. The applicant also identified the following backup systems that operators will be able to use to monitor and control the plant under a range of degraded I&C conditions: As described in DCD Tier 2, Section 7.7.1.4 the IFPDs on the OCs receive information from the IPS. The IPS is a computer-based system that provides operational means for monitoring and control of the plant. The information is derived from other I&C systems and self-contained algorithms called application programs. The IPS makes the information available to the plant operating staff both on a real-time and historical basis. DCD Tier 2, Sectithe primary (active) unit and the other is a dedicated backup. If the primary IPS server experiences a failure, its dedicated backup server assumes all processing tasks of the failed unit. systems are redundantly implemented by the IPS and QIAS-N. The IPS and QIAS-N are independent and diverse from each other. Therefore, any single system. Incorporating redundancy into the design of the I&C systems that interface with the HSI resources in the MCR ensures that the operators have reliable sources of information available during degraded I&C conditions. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 4 The applicant should determine the necessary compensatory actions and supporting procedures to ensure that personnel effectively manage degraded I&C and HSI conditions, and the transition to back-up systems. The TA IP, Section 2 states that [ ] The TA IP states that a BTA will be performed for all tasks. The BTA produces a task narrative, which lists the actions operators must perform to accomplish the task. The task narrative is an input that the COL applicant uses to develop operating procedures. The task narrative is also an input to the HSI design process. Additionally, the applicant identified actions that operators need to take and the diverse HSI resources that will be available during a CCF of the digital I&C safety system that occurs during a PA or AOO, which is an HFE-significant I&C degradation that results in loss of the preferred CCF Coping Analysis is in Section 7.8 of this report). The actions that operators must take during such an event are IHAs. IHAs and the diverse HSI described in the CCF Coping Analysis are inputs to the HSI design process. The HD IP, Section 3.5.4 states that [ ] will be 115 developed to help ensure that personnel effectively manage this type of degraded I&C and HSI condition. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. 18.7.4.6 HSI Tests and Evaluations The criteria in this section are divided into two subsections. Trade-off evaluations are comparisons between design options, based on aspects of human performance that are important to successful task performance, and to other design considerations. Performance-based tests involve assessing personnel performance, including subjective opinions, to evaluate design options and design acceptability. Trade-Off Evaluations NUREG--Criterion 1 In comparing design approaches, the applicant should consider those aspects of human performance important to performing tasks. The applicant should take into account the following factors when developing criteria to apply in selecting one design approach over another: personnel-task requirements human-performance capabilities and limitations HSI-system performance requirements inspection and testing needs maintenance demands use of proven technology and the operating experience of predecessor designs 1The Basic HSI TeR, Basic HSI Concept. The APR1400 is an evolutionary plant design, and the Section 5 states, [ 116 ] [ ] [ ] Thus, the staff finds that the technology and operating experience of predecessor designs have been taken into account, as well as HSI-system performance requirements (e.g., those related to seismic durability), and human performance capabilities and limitations. considered in the design of the HSI resources. The task narratives developed in accordance with the TA IP will be used to establish alarm priorities, alarm logic, displays, and operator aids during the HSI design process. For example, the HD IP, Section 4.2.3 states that [ ] Therefore, the staff concludes that the appliprocess incorporates the results of the task analysis (i.e., personnel task requirements) process. Like the predecessor designs, the APR1400 Basic HSI concept accounts for testing needs and maintenance demands. For example, as described in the Basic HSI TeR, Section 4.3.3, [ ] The HFE guidelines and standards defined in the Basic HSI TeR and the Style states that are included in task analysis. The results of the task analysis, specifically the HSI inventory developed for each task that is analyzed, is an input to the HSI design process. Additionally, the applicant will perform task support verification (TSV) as part of V&V. The V&V IP, Section 4.3 states [ ] 117 and testing needs and maintenance demands. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 2 The applicant should state explicitly the relative benefits of design alternatives and the basis for the design approach selected. [ ] that differ from those found in conventional, analog, predecessor plants. The design features, their relative benefits, and their bases for selection, are described below. Control panels in legacy control rooms have a wide variety of switches, pushbuttons, controllers, indications, and displays. These control panels are typically distributed throughout the control room and require operators to walk to and from panels to operate equipment. Therefore, the staff concludes that the IFPDs and soft controls, which are designed in accordance with design-specific style guides and are accessible to the operators directly from the operator consoles where the operators are seated, do increase the standardization of the plant-personnel interface, and provide for easier access to plant controls and indications compared to legacy plants.[ ] 118 The applicant identified the relative benefits of the CBP system compared to paper procedures in the Basic HSI TeR, Section 4.8. This section states that ] The staff concludes that this reduces mental workload and improves the likelihood that continuous action steps will be performed in a timely manner. The applicant identified the relative benefits of the digital LDP, which has fixed and selectable displays, as compared to legacy plants in the Basic HSI TeR, [ ] states, ] Thus, the staff finds that compared to conventional, analog control rooms, where operators must locate individual indications and controls located on multiple panels in the control room to understand the overall plant status, the information presented on the LDP allows plant personnel to more easily locate and integrate information and more readily assess plant status. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Performance-Based Tests NUREG--Criterion 1 The applicant should identify the specific objectives of the tests. The HD IP, Section 4.1.7 states that the [ [ [ 119 ] -- based tests will be conducted for ] The applicant identified the following specific objectives of these performance-based tests: Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 2 following aspects of the tests should be described (note that not all items are applicable to every type of test): participants testbed design features or characteristics of the HSI being tested tasks or scenarios used performance measures test procedures data analyses2 The HD IP, Section 4.1.7 describes participants, testbed, design features to be tested, and scenarios to be used for the performance-based test of the APR1400 Basic HSI Concept. [ ] 120 The HD IP did not describe specific test procedures, performance measures, or data analyses. This is acceptable for two reasons. First, the objective of this test is to obtain feedback from operators licensed in the U.S. about how aspects of the APR1400 Basic HSI Concept may be improved. The staff finds that the test method described in the HD IP is sufficient for allowing operators licensed in the U.S. to experience interacting with the APR1400 Basic HSI Concept and provide feedback about the experience. Second, the APR1400 HSI Design that results from the HSI design process described in the HD IP will be tested during the ISV using the test method described in the V&V IP. The V&V IP specifies requirements for test procedures, performance measures, and data analyses in addition to requirements for test participants, an adequate testbed, and scenarios. Thus, the staff concludes that the performance-based test of the APR1400 Basic HSI Concept that the applicant will conduct prior to V&V should help to ensure that successful test results can be obtained during the ISV when the final HFE design is tested. testbed, participants, and design features to be tested, and performance measures for performance-based tests of the task displays and application displays, respectively. [ ] [ ] The HD IP did not describe specific test procedures, performance measures, or data analyses for these performance-based tests. The staff finds this is acceptable for two reasons. First, the objective of testing the task displays, the application displays, and the alarms and CBPs that are used in conjunction with task displays that operators use to perform IHAs is to confirm that ] 121 these HSIs are correctly interfaced to the applicable plant systems. For alarms, the tests also confirm that the alarms are correctly prioritized and that the alarm logic is correct. By using a part-task simulator, the SMEs will be able to determine whether the CBPs, alarms, and plant system indications and controls on the task and application displays are integrated prior to the ISV. Thus, the staff finds that these tests will confirm the functionality of these displays prior to the ISV, which should help to ensure that successful test results will be obtained during the ISV. Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Criterion 3 The conclusions from the tests and their impact on design decisions should be described. [ - ] Accordingly, the staff finds the application conforms to this NUREG-0711 criterion. Combined License Items There are no COL items associated with Section 18.6 of the APR1400 DCD. Conclusion conforms to the criteria in NUREG-method is an acceptable process for translating functional and task requirements to HSI design requirements and for applying HFE design principles. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii), 10 CFR 50.34(f)(2)(iv), 10 CFR 50.34(f)(2)(v), 10 CFR 50.34(f)(2)(xi), 10 CFR 50.34(f)(2)(xii), 10 CFR 50.34(f)(2)(xviii), 10 CFR 50.34(f)(2)(xix), 10 CFR 50.34(f)(2)(xxvi), 10 CFR 50.34(f)(2)(xxvii), and 10 CFR 52.47(a)(8) related to this element. 18.8 Procedure Development Procedure development is included in NUREG-0711 because there are HFE attributes associated with the procedures. However, as an operating program, procedure development is [ ] 122 18.9 Training Program Development The training program development is included in NUREG-0711 because of the interfaces between the HFE design, procedures, and training. However, as an operating program, training documented in this section. 18.10 Human Factors Verification and Validation Introduction The review of the verification and validation (V&V) element was conducted at the IP level, as described in NUREG-methodology the applicant will use to conduct the V&V. The objectives of the review were to verify that: The applicant identified a sample of operational conditions that: (1) includes conditions representative of the range of events that could be encountered during tof human system interfaces (HSIs). These sample characteristics are best identified by using a multidimensional sampling strategy to reasonably assure that V&V evaluations include variation along important dimensions. (See Section The applicant's HSI inventory and characterization accurately describes all HSI displays, controls, and related equipment lying within the scope defined by the evaluation). The applicant verified that the HSI provides the needed alarms, information, controls, and task support defined by task analysis for personnel to perform its evaluation). The applicant verified that the design of the HSIs conforms to HFE guidelines evaluation). The applicant validated, using performance-based tests, that the integrated system design (i.e., hardware, software, procedures, and personnel elements) supports the safe operation of the plant. (See Section 18.10.4.3 below for the evaluation). The applicant has: (1) evaluated HEDs to determine if they require corrections, (2) identified design solutions to address HEDs that must to be corrected, and (3) verified the completed implementation of these HED design solutions. (See evaluation). Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of -K-X-IT-14001. 123 Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2: Section 18.10 summarizes the methodology described in APR1400-E-I-NR-14008, applicant to select operational scenarios, complete task support verification and HFE design verification, and conduct the integrated system validation (ISV). Changes to Tier 2 information are governed by the change control processes in the design evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs. ITAAC: The ITAAC associated with this element are listed in DCD Tier 1, Section 2.9, Table 2.9- TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: TeRs associated with this element are: APR1400-E-I-NR-(ML18212A336) APR1400-E-I-NR-actors Verification and Validation (ML18178A202) APR1400-E-I-NR-(V&V Scenarios Document) (ML18081A107) APR1400-E-I-NR- (ML18081A107) Regulatory Basis NUREG-0800. The applicable regulatory requirements are as follows: 10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) Other regulatory guidance documents are as follows: NUREG- NUREG-0800, Revision 2, Chapter 1Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human 124 Technical Evaluation 18.10.4.1 Sampling of Operational Conditions The criteria in this section are divided into several subsections. Sampling Dimensions NUREG-support V&V tests is important because reviews of new plants and significant HSI modifications can involve hundreds or thousands of individual HSIs, and it is impractical and unnecessary to review all of them. Therefore, the applicant can employ a sampling strategy to guide the -0711, Sectthree criteria for determining the conditions that will be included in the sample of HSIs that will be reviewed during the V&V activities. These criteria are listed below. Criterion 1 The applicant should include the following plant conditions: normal operational events including plant startup, shutdown or refueling, and significant changes in operating power I&C and HSI failures and degraded conditions that encompass: o the I&C system, including the sensor, monitoring, automation and control, and communications subsystems; [e.g., safety-related system logic and system, MUX controller, and a break in MUX line] o common cause failure of the I&C system during a design basis accident (as defined by Branch Technical Position BTP 7-19) o HSIs including, loss of processing or display capabilities for alarms, displays, controls, and CBPs transients and accidents, such as: o transients (e.g., turbine trip, loss of off-site power, station blackout, loss of all feedwater, loss of service water, loss of power to selected buses or MCR power supplies, and safety and relief valve transients) o accidents (e.g., main-steam-line break, positive reactivity addition, control rod insertion at power, anticipated transient without scram, and various- sized loss-of-coolant accidents) o reactor shutdown and cooldown using the remote shutdown system o reasonable, risk-significant, beyond-design-basis events that should be determined from the plant-specific PRA 125 checklist is prepared that SMEs use to construct the scenarios. All of the necessary dimensions plant conditions that will be included in the scenarios. The staff compared these to the above criteria and found the process and the list acceptable. Section 5 of the V&V Scenarios Document provides the seven specific scenarios that have been constructed by the applicant. Each scenario is described in summary fashion in Section 5 and in more detail in seven appendices to the report. The V&V Scenarios Document in Section 3, essed by each scenario. The staff reviewed the scenario descriptions and Table/matrix of Section 3.1 and verified that all required plant conditions were addressed by at least one scenario. The staff finds that the application conforms to this review criterion. Criterion 2 The applicant should include the following types of personnel tasks: Important human actions, Systems, and Accident Sequences The sample should include all important HAs, as determined in Section 7 of NUREG-0711. Additional factors that contribute highly to risk, as defined by the PRA, also should be sampled: o dominant accident sequences o dominant systems (selected through PRA importance measures, such as Risk Achievement Worth or Risk Reduction Worth) Manual Initiation of Protective Actions The sample should include manual system-level actuation of critical safety functions. Automatic System Monitoring The sample should include situations in which humans must monitor a risk-important automatic system. OER - Identified Problematic Tasks The sample should include all personnel tasks identified as problematic during the applicant's review of operating experience. Range of Procedure Guided Tasks The sample should include tasks that are well defined by procedures. Personnel should be able to understand and execute the specified steps as part of their rule-based decision-making. RG -related activities ude appropriate procedures in each category: o administrative procedures o general plant operating procedures 126 o procedures for startup, operation, and shutdown of safety-related systems o procedures for abnormal, off-normal, and alarm conditions o procedures for combating emergencies and other significant events (e.g., reactor accidents, and declaration of emergency-action levels) o procedures for controlling radioactivity o procedures for controlling measuring and test equipment and for surveillance tests, procedures, and calibration o procedures for performing maintenance o chemistry and radiochemical control procedures Range of Knowledge-Based Tasks The sample should include tasks that are not well defined by detailed procedures. Range of Human Cognitive Activities The sample should include the range of cognitive activities that personnel perform, including: o detecting and monitoring (e.g., of critical safety-function threats) o situation assessment (e.g., interpreting alarms and displays to diagnose faults in plant processes and in automated control and safety systems) o planning responses (e.g., evaluating alternatives to recover from plant failures) o response implementation (e.g., in-the-loop control of plant systems, assuming manual control from automatic control systems, and carrying out complicated control actions) o obtaining feedback (e.g., feedback of the success of actions taken) Range of Human Interactions The sample should include the range of interactions among plant personnel, including tasks performed independently by individual crew members, and those undertaken by a team of crew members. These interactions among plant personnel should include interactions between: o MCR operators (e.g., operations, shift turnover walk downs) o MCR operators with auxiliary operators and other plant personnel performing tasks locally (e.g., maintenance or I&C technicians, chemistry technicians) o MCR operators and the TSC and the emergency offsite facility (EOF) o MCR operators with plant management, the NRC, and other outside organizations 127 The V&V IP, Section 4.1.1 states that a list of conditions and a checklist is prepared that SMEs use to construct the scenarios. All of the necessary dimensions are included in at least one scenario. Section 4.1.1.2 lists the types of personnel tasks that will be included in the scenarios and states that all of them will be included in the sample. Also, Table 4-Matrix Example (Types of Personnel Tasksin a checklist format that will be used by the SMEs to assign all the listed types of tasks to the scenarios. The staff compared the types of tasks listed in Section 4.1.1.2 and Table 4-2 to the above criteria and found Section 4.1.1.2 and Table 4-2 include all of the types of personnel tasks listed in the criterion. The APR1400 V&V Scenarios Document, Section 5, provides seven specific scenarios that have been constructed by the applicant. Each scenario is summarized in Section 5 and described in more detail in seven appendices to the report. The V&V Scenarios Document, -2 of the V&V IP that illustrates which task types are addressed by each scenario. The staff reviewed the scenario descriptions and Table/matrix of Section 3.2 of the V&V Scenarios Document and verified that all of the types of personnel tasks listed in the V&V IP, Section 4.1.1.2 were addressed by at least one scenario (with the below-noted exceptions). In Table/matrix of Section 3.2 of the V&V Scenarios document, there are five of these personnel task types that are blank (e.g., the types of tasks have not yet been assigned to a scenario). The five personnel task types are: IHAs, dominant systems from the PRA, OER-identified problematic tasks, procedures for controlling radioactivity, and maintenance procedures. There is table note 1 on IHAs in Table 4-2 of the V&V IP that explains that the TIHA ReSR needs to be completed in order to complete Table 4-2, but the same note is not included in the V&V Scenarios Document. Related to this issue, Section 4.1 of the V&V Scenario Document is titled is no explanation as to what document the important human action information is blank. The listing of two major errors seems to contradict other statements elsewhere that IHAs cannot yet be identified. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-61 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-61 (ML16074A294). In its response, the applicant stated that there currently is insufficient information to complete the identification of those five types of tasks and assignment of those tasks to specific scenarios. When the TIHA, OER, and procedure development human factors program elements are completed by the COL, then the information needed to finish assigning personnel tasks to scenarios will be available. The applicant stated that the personnel tasks that have yet to be identified in the scenarios document will be identified and assigned to scenarios using the process outlined in Section 4.1.4 of the V&V IP. The assignment process for scenario definition was reviewed and scenario definition, the V&V IP, Section 4.1.4 states that SMEs will use Table 4-2 to help ensure that all of the different types of personnel tasks are included in a scenario. The applicant also stated that the criteria for identifying the specific tasks are determined by the subject matter experts during their evaluation. The staff finds this acceptable because the V&V IP contains a process for updating the scenarios to include the types of personnel tasks listed in the criterion 128 when the information is available, and selection of the specific tasks is within the expected capabilities of the SMEs. The applicant also committed to revise the V&V IP and V&V Scenario Document to address inconsistencies in the table notes between the V&V IP and the V&V Scenarios Document. Change pages are provided with the RAI response that the staff finds acceptably clarify the information in Section 4.1 of V&V Scenario Document rather failures that might result from various operator errors or equipment failures. As such, the applicant said it will delete Section 4.1 of the V&V Scenarios Document. The staff noted that the information being deleted was characterized in the document as a summary of APR1400 PRA information, which is also summarized in DCD Chapter 19. Because the PRA information provided in DCD Chapter 19 is being reviewed by the staff as part of the design certification process, and because the PRA information will be further refined by the COL applicant, the PRA information could change. It would not be desirable for PRA information to be listed in more than one place in the application because it is possible the information may be updated in one part of the DCD, but not in the V&V Scenarios Document, which could cause confusion for the SMEs who will be completing the activities in the V&V IP. Thus, the staff finds it acceptable to remove this information from the V&V Scenarios Document because the V&V IP contains direction for the SMEs to refer to the TIHA ReSR, which extracts the PRA information that should be included in the HFE program from the most current source. The staff confirmed that the V&V IP and V&V Scenarios document were revised as committed in response to RAI 352-8205, Question 18-61. Therefore, RAI 352-8205, Question 18-61 was resolved and closed. The staff finds that the application conforms to this review criterion. Criterion 3 The applicant should include the following situational factors or error-forcing contexts known to challenge human performance. It also should include situations specifically designed to create er from any errors, should these occur, for example: High-Workload Situations The sample should include situations where variations in human performance due to high workload and multitasking situations can be assessed. Varying-Workload Situations The sample should include situations wherein variations in human performance due to workload transitions can be determined. These include conditions where there is: (1) a sudden increase in the number of signals that must be detected and processed after a period in which signals were infrequent, and (2) a rapid reduction in the need for detecting signals and processing demands following a time of high sustained task-demand. Fatigue Situations To the extent possible, the sample should include situations that may be associated with fatigue, such as work on backshifts and tasks performed frequently with repetitive actions, such as repeated inputs to a touch screen during plant operations or pulling rods. Environmental Factors To the extent possible, the sample should include environmental conditions that may cause human performance to vary, e.g., poor 129 lighting, extreme temperatures, high noise, and simulated radiological contamination. The V&V IP, Sectchecklist is prepared that SMEs use to construct the scenarios. All of the necessary dimensions onal Factors included in the scenarios. The staff compared these to the above criteria and found the list acceptable. Also, the V&V IP, Section 4.5.2, Bullet 4, states that a test objective is to validate that the HSIs minimize personnel error and provide reasonable assurance of error detection and recovery capability when errors occur. This is accomplished through selecting error-forcing contexts outlined in the tables discussed in the following paragraph. scenarios that have been constructed by the applicant. Each scenario is described in summary fashion in Section 5 and in more detail in seven appendices to the report. The V&V Scenarios Document in Section 3 contains two tables that list personnel tasks and situational factors to be included in the V&V scenarios. Table 3.2 addresses 25 different personnel tasks and is reviewed in -Forcing reviewed the scenario descriptions and the Tables of Section 3.3 and all four bullets of this criterion are addressed. Environmental factors such as noise, lighting, temperature, and humidity of the simulator room can be made similar to the control room, but it is not certain that the environment can be duplicated so that it is identical to the actual control room. This difficulty is to be expected. To address this, the applicant has included one scenario where only emergency lights are available and another scenario where the operators in the MCR must wear protective clothing due to simulated contamination in the MCR. The applicant further includes in the V&V IP a verification of environmental conditions in the actual completed MCR as part of the HFE Design Implementation element. The applicant also notes in the V&V IP, the HED process and the issues tracking system. However, during the initial review the staff was not able to determine if error tolerance and the ability of personnel to recover from any errors was included in the V&V IP or the scenarios document. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Questions 18-62 and 18-63 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-62 and 18-63 (ML16074A294). In its response to RAI 352-8205 Question 18-62, the applicant stated that error tolerance and the ability of personnel to recover from any errors are included in the V&V IP and the scenarios document but agreed to enhance its description. A description of the inclusion into V&V of error tolerance of the system and the ability of operators to recover from errors will be added to Section 4.1.1.3 of the human factors V&V IP. Also, it will add to the V&V 130 IP, Section 4.5.2 an objective to validate that the system has the ability to tolerate errors. The applicant provided acceptable drafts of both changes. In its response to RAI 352-8205 Question 18-63, the applicant noted that V&V IP Environmental factors such as noise, lighting, temperature, and humidity of the simulator room can be made similar to the control room, but it is not certain that the environment can be duplicated so that it is identical to the actual control room conducted in the actual plant main control room or other locations within the scope of the V&V during the DI (Design Implementation). The applicant also notes that there are short discussions of environmental factors in the V&V IP Sections: 4.1.1.3, Table 4-3, 4.1.5, 4.4.2, 4.4.3, 4.5.3, 4.5.5.1(5) and in the V&V scenarios document, ISV 4, Appendix D, page D9, Item 4 on protective clothing. The applicant agreed to correct some related minor errors to Table 4-3 on page 13 of the V&V IP. The changes were supplied as part of the response to Question 18-61 and are acceptable. The RAI 352-8205 Question 18-63 response is acceptable. The staff confirmed that the V&V IP was revised as committed in responses to RAI 352-8205, Questions 18-62 and 18-63. Therefore, RAI 352-8205, Questions 18-62 and 18-63 were resolved and closed. The staff finds that the application conforms to this review criterion. Identification of Scenarios NUREG-is topic. Criterion 1 The applicant should combine the results of the sampling to identify a set of V&V scenarios to guide subsequent analyses. identification of V&V scenarios is based on the sample of operational conditions that are Further the document human factors V&V Scenarios provides 7 scenarios and demonstrates in the three tables of Section 3 how the results of sampling were used to identify the scenarios. The staff finds that the application conforms to this review criterion. Criterion 2 The applicant should not bias the scenarios by overly representing the following: scenarios for which only positive outcomes are expected scenarios that, for ISV, are relatively easy to conduct (i.e., scenarios should not be avoided simply because they are demanding to set up and run on a simulator) 131 scenarios that, for ISV, are familiar and well structured (e.g., which address familiar systems and failure modes that are highly compatible with plant -basis accidents) aluation of Criterion 2 criterion. Further the staff reviewed the 7 scenarios in the human factors V&V scenarios document and did not identify any of the above practices that could potentially bias the V&V such that only positive results could be obtained from the scenario. The staff finds that the application conforms to this review criterion. Scenario Definition NUREG-Criterion 1 The applicant should identify operational conditions and scenarios to be used for HSI Task Support Verification, Design Verification, and ISV. The applicant should develop detailed scenarios suitable for use on a full-scope simulator. The level of detail should be comparable to what one would include in a test plan. For each one, the following information should be defined to reasonably assure that important dimensions of performance are addressed, and to allow the scenarios to be accurately and consistently presented for repeated trials: a description of the scenario and any pertinent prior history necessary for personnel to understand the state of the plant at the start-up of the scenario systems, component conditions, and performance parameters, e.g., similar to that at shift turnover) events (e.g., failures) that will occur during the scenario and their initiating conditions, e.g., based on time, or a value of a specific parameter precise definition of workplace factors, (e.g., environmental conditions, such as low levels of illumination) needs for task support (e.g., procedures and technical specifications) staffing level details of communication content between control room personnel and remote personnel (e.g., load dispatcher via telephone) scripted responses for test personnel who will act as plant personnel in the test scenarios the precise specification of what, when, and how data are to be collected and stored (including videotaping, questionnaires, and rating-scale administrations) 132 precise specifications on simulator set up specific criteria for terminating the scenario 1 philosophy as to how the scenarios are constructed and generally what activities are required of operators as the scenarios unfold. In Section 4.1.4 the applicant stated that the scenarios, based on the SOC, are to be used for HSI Task Support Verification, HFE Design Verification, and ISV. Section 4.1.4 further provides some background on how scenarios are actually constructed by SMEs using an iterative process as plant design and procedures are finalized. included in each scenario to ensure that the scenarios are understandable, include all important dimensions are addressed, and can be accurately and consistently performed. The list of items includes all of the above criteria. In the human factors V&V scenarios document, the staff also reviewed a selected scenario, namely a small-break, loss of coolant accident with computer- based procedure failure and HSI display failure. Each of the criteria in: (1) above was addressed in the scenario. The staff finds that the application conforms to this review criterion. Criterion 2 actual operations. The human factors V&V IP, Sections 4.1.3, 4.1.4 and 4.1.5 taken as whole explain in a number of detailed steps how the scenarios realistically replicate operator tasks, thus allowing the findings Section 4.1.3 the V&V IP states: In Section 4.1.4, the V&V IP states In Section 4.1.5, the V&V IP states that each scenario includes the items listed below to provide reasonable assurance that all important dimensions of performance are addressed. [ ] [ 133 ] Additionally, the staff reviewed the summaries of all scenarios in the human factors V&V Scenarios document and further concludes that these realistically replicate operator tasks. The staff finds that the application conforms to this review criterion. Criterion 3 the effects on personnel performance due to potentially harsh environments (e.g., high radiation) should be realistically simulated (e.g., additional time to don protective clothing, and access radiologically controlled areas). Criterion 3 inclusion of environmental conditions that may cause human performance to vary, e.g., poor lighting, extreme temperatures, high noise, and simulated radiological contamination into the operational conditions. Alsoscenario includes precise definition of workplace factors (e.g., environmental conditions) and the needed task support, which could include protective clothing. Items needed for task support, 134 are identified during task analysis, which is an input V&V (refer to the TA IP, Section 4.2.1, the task should be identified). As such, the staff finds the applicant has provided direction in the V&V IP for environmental conditions and the items needed to support completion of tasks that occur in those environmental conditions to be identified so they may be included or simulated in the scenarios. Therefore, the staff finds that the application conforms to this review criterion. 18.10.4.2 Design Verification The criteria in this section are divided into several subsections. HSI Inventory and Characterization NUREG- this topic. Criterion 1 Scope The applicant should develop an inventory of all HSIs that personnel require to Sampling of Operational Conditions (SOC). The inventory should include aspects of the HSI used for managing the interface, such as navigation and retrieving displays, as well as those that control the plant. In the scope includes all HSIs needed to perform scenarios identified in the SOC. The inventory is maintained in an HSI database. The staff finds that the application conforms to this review criterion. Criterion 2 HSI Characterization within the scope of the verification. The following is a minimal set of information for this characterization: a unique identification code number or name associated plant system and subsystem associated personnel functions and tasks type of HSI, e.g., o computer-based control (e.g., touch screen or cursor-operated button and keyboard input) o hardwired control (e.g., J-handle controller, button, and automatic controller) o computer-based display (e.g., digital value and analog representation) 135 o hardwired display (e.g., dial, gauge, and strip-chart recorder) display characteristics and functionality [e.g., plant variables/parameters, units of measure, accuracy of variable/parameter, precision of display, dynamic response, and display format (e.g., bar chart or trend plot)] control characteristics and functionality [e.g., continuous vs. discrete settings, number and type of control modes, accuracy, precision, dynamic response, and control format (method of input)] user-system interaction and dialog types (e.g., navigation aids and menus) location in data-management system (e.g., identification code for information display screen) physical location in the HSI (e.g., control panel section), if applicable The applicant should include photographs, copies of display screens, or similar samples of HSIs in the HSI inventory and characterization. Evaluation of Criterion 2 information contained in the inventory. The description of the characterization is consistent with additional information that will be useful in the verification process, e.g., environment where the HSI is located. The staff finds that the application conforms to this review criterion. Criterion 3 Inventory Verification The applicant should verify the inventory description of HSIs to ensure that it accurately reflects its current state. the inventory. The contents of the HSI database are verified by an SME to ensure all appropriate HSIs are represented and accurately depict the final design. The staff finds that the application conforms to this review criterion. HSI Task Support Verification NUREG-0711, topic. The fifth criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first four criteria as discussed below Criterion 1 Verification Criteria The applicant should base the HSI task support criteria on the alarms, controls, displays, and task support needed by personnel to complete their tasks as identified by 136 Evaluation of Criterion 1 based on the alarms, controls, displays, and task support needs that will be identified by the f finds that the application conforms to this review criterion. Criterion 2 General Methodology The applicant should compare the HSIs and their characteristics (as defined in the HSI inventory and characterization) to the needs of personnel identified in the task analysis for the defined sampling of operational conditions, noted in Section 11.4.1 of NUREG-0711. V&V IP. The V&V IP also contains Figure 4-1 depicting the process. A five-step process is used: Prepare an HSI checklist using the HSI Database, prepare task support items identified during the task analysis, collect HSI design data within selected operational conditions, compare HSI design documents against HSI and task support checklist, and prepare documentation. The final step provides input to the human factors V&V ReSR. This method will effectively accomplish task support verification. The staff finds that the application conforms to this review criterion. Criterion 3 Human engineering discrepancies (HED) Identification The applicant should identify and document an HED when: An HSI needed for task performance (e.g., a necessary control or display) is unavailable. HSI characteristics do not match the requirements of the personnel task (e.g., a display may show the needed plant parameter but not within the range or precision needed for the task). HSIs are available that are not needed for any task. The 3 the V&V IP. The V&V IP states that HEDs will be identified for each condition identified in the The staff finds that the application conforms to this review criterion. Criterion 4 HED Documentation The applicant should document HEDs to identify the HSI, the tasks affected, and the basis for the deficiency (what aspect of the HSI was identified as not meeting task requirements). 137 V&V IP. Section 4.3.3 of the V&V IP states that: HEDs will be entered into the ITS for evaluation and resolution. However, no information is provided as to what information is entered into the system for each HED. Section 4.5.9.5 of the V&V IP provides generic information on HED documentation but does not provide specific information on Task Support Verification HEDs, e.g., the specific HSI and the basis for the HED such as the aspect of the HSI that was identified as not meeting task requirements. The ITS is also described in Section 4.6 of the HFE PP, but that does not provide this information either. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-64 to address this issue (ML15356A485). On April 2, 2016, the applicant provided a response to RAI 352-8205, Question 18-64 (ML16093A024). The applicant clarified the information to be included in the issues tracking database for HEDs arising from TSV. This information includes a description of the current conditions and the issue creating the HED (i.e., the basis for the HED). It also includes identification of the specific HSI by its related system code, as well as the corrective actions taken to resolve the HED. The staff finds the proposed changes to the V&V IP ensure the documentation of HEDs contains the information listed in the criterion. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-64. Therefore, RAI 352-8205, Question 18-64 was resolved and closed. The staff finds that the application conforms to this review criterion. Human Factors Engineering Design Verification NUREG-The fifth criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first four criteria as discussed below. Criterion 1 Verification Criteria The applicant should base the criteria used for HFE Design Verification on HFE guidelines. Section 4.4.1 of the V&V IP. (APR1400-E-I-NR-14012). The staff finds that the application conforms to this review criterion. Criterion 2 General Methodology the following: Procedures for comparing the characteristics of the HSIs with HFE guidelines for: (1) the defined sampling of operational conditions, as noted in Section 11.4.1, and (2) the general environment in which HSIs are sited, including workstations, control rooms, and environmental characteristics (e.g., lighting and noise). 138 evaluated (see Sections 2.4.4 and 11.4.4). Procedures for evaluating whether an HED is a potential indicator of additional issues. Sections 4.4.1 and 4.4.2 of the V&V IP describe the HFE design verification methodology. The methodology specifies five steps. However, it is not clear from the methodology description how the characteristics of the HSIs will be compared to the HFE guidelines in the Style Guide. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-65 to address this issue (ML15356A485). On May 12, 2016, the applicant provided a response to RAI 352-8205, Question 18-65 (ML16133A590). The applicant clarified the Style Guide is used as the criteria against which the HSI is verified. Specifically, the guidance in the Style Guide is developed into a checklist, which is then used to compare with the detailed design of the HSI. This acceptably addresses e applicant has proposed changes to the V&V IP to clarify these points and the staff finds that these changes acceptably capture the information. Based on the review of the V&V IP, the staff has confirmed incorporation of the changes described above; therefore RAI 352-8205, Question 18-65 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 3 HED Identification The applicant should identify an HED when a characteristic of the HSI is "discrepant" from a guideline. design does not conform to the criteria in the Style Guide, an HED is identified. Also, the verification team checks related HSIs to determine whether the HED is associated with only the HSI originally identified or more pervasively appears for other HSIs. If so, a new HED is created -t the application conforms to this review criterion. Criterion 4 HED Documentation The applicant should document HEDs in terms of the HSI involved, and how its characteristics depart from a particular guideline. Deviations from criteria are identified for each HED entered into the ITS. The staff finds that the application conforms to this review criterion. 139 18.10.4.3 Integrated Systems Validation The objective of reviewing integrated system validation methodology is to verify that the personnel elements) using performance-based tests that will determine whether it acceptably supports safe operation of the plant. The criteria in this section are divided into several subsections. Validation Team NUREG-topic. Criterion 1 The applicant should describe how the team performing the validation has independence from the personnel responsible for the actual design. IP states that the team is independent of the design organization. While the functions of the team are described, the relationship and interfaces between the validation team and the design team are not explained. This Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-66 to address this issue (ADAMS Accession No ML15356A485). On May 13, 2016, the applicant provided a response to RAI 352-8205, Question 18-66 (ML16134A569). The applicant provided an explanation of the organizational and administrative controls that ensure independence of the validation team from the design team. These controls include organizational placement of the validation team, its training, and the concern. The applicant has proposed changes to the V&V IP to clarify these points and the staff finds that these changes acceptably capture the information. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-66. Therefore, RAI 352-8205, Question 18-66 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Test Objectives NUREG-second criterion addresses plant modifications and is not applicable to new reactors, thus the staff evaluated the first criteria as discussed below Criterion 1 The applicant should develop detailed test objectives to provide evidence that the integrated system adequately supports plant personnel in safely operating the plant, to include the following considerations: Validate the acceptability of the shift staffing level(s), the assignment of tasks to crew members, and crew coordination within the control room, between the 140 control room and local control stations and support centers, and with individuals performing tasks locally. This should encompass validating minimum shift staffing levels, nominal levels, maximum levels, and shift turnover. Validate that the design has adequate capability for alerting, informing controlling, and feedback such that personnel tasks are successfully completed during normal plant evolutions, transients, design-basis accidents, and also under selected, risk-significant events beyond-design basis, as defined by sampling operational conditions. Validate that specific personnel tasks can be accomplished within the time and performance criteria, with effective situational awareness, and acceptable workload levels that balance vigilance and personnel burden. Validate that the HSIs minimize personnel error and assure error detection and recovery capability when errors occur. Validate the assumptions about performance on important HAs. Validate that the personnel can effectively transition between the HSIs and procedures in accomplishing their tasks, and that interface management tasks, such as display configuration and navigation, are not a distraction or an undue burden. described correspond to those identified in the review criterion. The applicant has included an additional objective to validate assumptions made regarding the performance impacts of new technology. The staff finds that the application conforms to this review criterion. Validation Testbeds NUREG-Criterion 1 Interface Completeness system. It should include HSIs and procedures not specifically required in the test scenarios. NUREG-through 7 in this section is to use a testbed that is compliant with the American Nuclear The simulator shall be referenced to a specific unit. The scope of simulation shall be such that the operator is required to take the same action on the simulator to conduct an evolution as on the reference unit, using the reference 141 unit operating procedures. The scope of simulation shall permit conduct of all of the evolutions required in this section until a stable condition is obtained. The evolutions listed in ANS 3.5, 2009, Section 3 are evolutions that operators perform during normal plant operations, such as plant startup, as well as evolutions operators perform to respond to the abnormal and emergency plant conditions that have been analyzed and use to perform normal and emergency evolutions from the control room are available in simulators that conform to ANS 3.5, 2009. -scope dynamic simulator developed to meet ANSI/ANS 3.5-2009 (Reference 13) will be used as the validation testbed will meet ANS 3.5, 2009, the staff concluded that the simulator will include all of the operating procedures that operators must use to perform all the evolutions listed in ANS 3.5, 2009, Section 3, even though the test scenarios only include a sample of those evolutions. Therefore, the staff initially concluded that the application conformed to this review criterion as well as to Criteria 2-7 listed below. However, during the APR1400 Subcommittee Meeting of the ACRS held on June 21, 2017 (ML17200A091), an ACRS member expressed concern that statements in the HFE PP and the HD IP indicated that only the procedures that will be used during the conduct of the test scenarios will be available in the simulator used for validation. Following the ACRS meeting, the staff reviewed the HD IP and the HFE PP and agreed that some statements in these documents did indicate that only the procedures that will be used for the ISV scenarios will be available in the simulator used for validation. If only the procedures that will be used during the conduct of the test scenarios will be available in the simulator, then opportunities to identify: (1) human performance or system errors associated with selecting and using procedures and (2) possible ways to reduce such errors will be minimized. Also, the staff determined that these statements contradi-scope dynamic simulator developed to meet ANSI/ANS 3.5-2009 (Reference 13) is used as the validation . Therefore, on September 6, 2017, the staff issued RAI 553-9084, Question 18-135 to address this issue (ML17249A979). On September 28, 2017, the applicant provided a response to RAI 553-9084, Question 18-135 (ML17271A188) and proposed edits to the HD IP and the HFE PP to clarify that the simulator used for validation will also include procedures that are not needed for the ISV scenarios. Specifically, the applicant revised the HFE PP and HD IP to explain that the HD program element converts the operating procedures executed from the control room, which are produced during the Procedure Development (PD) program element, into computer-based procedures prior to ISV. The inventory of CBP for the ISV scenarios will include additional procedures that are related to the ISV scenarios, but not actually needed to execute the ISV scenarios, to ensure the operator decisions are not influenced by the CBP inventory. Based on the review of the HFE PP, the staff has confirmed incorporation of the changes described above; therefore RAI 553-9084, Question 18-135 was resolved and closed. Because the applicant stated in Section 4.5.3 of the V&V IP that the testbed used for validation the HFE PP and HD IP to clarify that procedures not specifically used in the test scenarios will be 142 included in the simulator used for validation, the staff finds that the application conforms to this review criterion. Criterion 2 Interface Physical Fidelity high physical fidelity to the reference design, including the presentation of alarms, displays, controls, job aids, procedures, communications equipment, interface management tools, layout, and spatial relationships. NUREG-through 7 in this section is to use a testbed that is compliant with the American Nuclear -scope dynamic simulator developed to meet ANSI/ANS 3.5-2009 (Reference 13) is criterion. Criterion 3 Interface Functional Fidelity d be represented with high fidelity to the reference design. All HSI functions should be available. See Evaluation of Criterion 2. Criterion 4 Environmental Fidelity represented with high physical fidelity to the reference design, including the expected levels of lighting, noise, temperature, and humidity. Thus, for example, the noise contributed by equipment, such as air- handling units, computers, and communications equipment should be represented in validation tests. See Evaluation of Criterion 2. Criterion 5 Data Completeness Fidelity Information and data provided to personnel should completely they monitor and control. See Evaluation of Criterion 2. 143 Criterion 6 Data Content Fidelity physical fidelity to the reference design. The presentation of information and controls should rest on an underlying model accurately mirroring the reference plant. The model should provide input to the HSI such that the information accurately matches that which is presented during operations. See Evaluation of Criterion 2. Criterion 7 Data Dynamics Fidelity fidelity to the reference design. The process model should be able to provide input to the HSI so that information flow and control responses occur accurately and within the correct response time; e.g., information should be sent to personnel with the same delays as occur in the plant. See Evaluation of Criterion 2. Criterion 8 For important HAs at complex HSIs remote from the main control room (e. g., a remote shutdown facility), where timely, precise actions are essential, the use of a simulator or mockup should be considered to verify that the requirements for human performance can be met. (For less important HAs, or for non-complex HSIs, human performance may be assessed on analysis, such as task analysis, rather than on simulations.) reatment of important HAs at complex HSIs remote from the MCR is described the MCR (e.g., Remote Shutdown Room) where precise actions are essential, a simulator or mockup is used to verify that the requirements for human performance can be met. For less non-complex HSIs, human performance is assessed based on analysis rather than on on. Criterion 9 The applicant should verify the conformance of the testbed to the testbed-required characteristics before validation tests are conducted. ed to the testbed-required characteristics is described in Section 4.5.3 of the V&V IP. The V&V IP states that [ 144 ] The staff finds that the application conforms to this review criterion. Plant Personnel NUREG-0711, Section 11.4.3.4, Criterion 1 will interact with the HSI (e.g., licensed operators, rather than training personnel or engineers). describes the ISV participants. The V&V IP states that the participants will be operators who are license holders, familiar with PWR designs and who have held previous positions similar to the role they will have in the ISV tests. The staff finds that the application conforms to this review criterion. Criterion 2 To properly account for human variability, the applicant should use a sample of participants that reflects the characteristics of the population from which it is drawn. Those characteristics expected to contribute to variations in system performance should be specifically identified; the sampling process should reasonably assure that the validation encompasses variation along license type and qualifications, skill/experience, age, and general demographics. T The V&V IP, Section 4.5.4 states that the ISV participants will be selected from a pool of candidates who are representative of the characteristics of expected APR1400 operators. The V&V IP lists license type, training, experience, skill level, age, and demographics among the factors considered. The staff finds that the application conforms to this review criterion. Criterion 3 In selecting personnel for participating in the tests, the applicant should consider the minimum shift staffing levels, nominal levels, and maximum levels, including shift supervisors, reactor operators, shift technical advisors, etc. DCD Tier 2, Section 18.1.1.1 identifies HFE design constraints imposed by regulations in 10 CFR 50.54(m) and 10 CFR 50.54(k) on the MCR staffing levels. It also describes the initial staffing and qualifications (S&Q) assumptions for the APR1400: A fundamental assumption of the APR1400 HFE design is that it is possible to operate the plant during postulated plant operating modes (modes 1 through 6) for normal, abnormal, and emergency conditions, with the following personnel in the main control room (MCR): one reactor operator (RO) with a reactor operator 145 license, one turbine operator (TO) with a reactor operator license, one electric operator (EO) with a reactor operator license, one shift supervisor (SS) with a senior reactor operator (SRO) license, and one shift technical advisor (STA) with a senior reactor operator license. The Basic HSI TeR, Section 3.5.1 describes the initial MCR staffing concept and provides an overview of the roles of each of these personnel in the MCR. It states the [ ] The concept reflects the initial staffing assumption as well as the staffing constraint. The V&V IP, Section 4.5.2 states, [ ] Thus, it is possible that the S&Q analyses will result in a final staffing level that may change the initial staffing assumption; however, the final staffing level will be subject to the minimum control room staffing constraint imposed by staffing requirements in 10 CFR 50.54(k) and (m), which will ensure the MCR staffing consists of at least one operator (the RO) and one senior operator (the SS). The staff reviewed the V&V IP and the V&V scenarios to evaluate how the applicant considered the staffing level in the selection of test participants. The V&V IP, Section 4.5.4 states that the ISV participants will consist of [ ] Additionally, the staff found that one of the scenarios models the situation where some control room operators are outside of the control room when an event occurs. This scenario [ ] For the rest of the scenarios, the staff found that the five test participants remain in the control room for the duration of the scenarios, which allows the nominal staffing level to be validated. However, no scenario simulates the case where the total possible number of operators that could be outside of the control room at the start of the event are outside of the control room at the start of an event. The staff did not understand how the scenarios could validate the possible minimum control room staffing level that could exist. Therefore, on January 23, 2017, the staff issued RAI 534-8723, Question 18-132 to address this issue (ML17023A031). On April 12, 2017, the applicant provided a response to RAI 534-8723, Question 18-132 (ML17102A979) and stated that [ ] The applicant also provided revisions to the Basic Human-System Interface Technical Report to include this concept. Based on the review of the Basic HSI TeR, the staff has confirmed incorporation of the changes described above; therefore RAI 534-8723, Question 18-132 was resolved and closed. The staff finds that the application conforms to this review criterion. Criterion 4 The applicant should prevent bias in the sample of participants by avoiding the use of participants who: are members of the design organization 146 participated in prior evaluations were selected for some specific characteristic, such as crews identified as good performers or more experienced Evaluation of Criterion 4 that the participants will not be selected on any predetermined characteristic and that any operators that have participated in the design or any aspect of V&V will be excluded as potential participants. The staff finds that the application conforms to this review criterion. Performance Measurement Types of Performance Measures NUREG-this topic. Criterion 1 The applicant should identify the specific plant performance measures applicable to each ISV scenario. eral approach to plant performance measurement. The specific measures applicable to each scenario are contained in the detailed scenario descriptions in the human factors V&V scenarios document. In the latter, the measures are identified in the pass/fail criteria. For example, in the scenario, Anticipated Transient without Trip with Distributed Control System Failures, the reactor startup task has a and procedurasample of the measures used for other events in this scenario and other scenarios and found them to be appropriate to the event performance being addressed. The staff finds that the application conforms to this review criterion. Criterion 2 The applicant should identify the primary task measures applicable to each ISV scenario. For each scenario, the applicant should identify the primary tasks operators must perform to accomplish scenario goals, so that such measures can be developed. The measures chosen to evaluate personnel task performance should reflect those aspects of the task that are important to system performance, such as: o time, o accuracy, o frequency, 147 o amount achieved or accomplished, o consumption or quantity used, o subjective reports of participants, and o behavior categorization by observers. The analysis of primary tasks will support the identification of errors of omission (primary tasks not performed). Also, any actions and tasks that operators actually perform that deviate from the primary tasks should be identified and noted. These actions should be used to identify errors of commission. The V&V IP, Section 4.5.5 expmeasurement. The specific measures applicable to each scenario are contained in the detailed scenario descriptions in the human factors V&V scenarios document. In the latter, the scenarios are subdiviprimary task, measures are identified in the evaluation criteria. For example, in the Anticipated Transient without Trip with Distributed Control System Failures, one primary task is performs an RCS [reactor coolant system] cooldown from the Safety Console with a minimum ng] entry conditions per AOP [abnormal operating procedure] without any unplanned transients, exceeding any procedural or of primary tasks is generally appropriate; however, the following was not clear to the staff: At the end of each scenario in the instructions, observers are asked to evaluate not clear how this evaluation is different from the specific criteria provided for each primary task during the course of the scenario. -of-Coolant Accident with Computer-Based Procedure Failure and Human-Monitor is malfunctioning and respond per alarm procedure Alarm-3761-01, of an instrument failure and responds per Alarm-3761-01, Radiation Monitoring Generator Tube Leak (SGTL) and responds per specified is not clear how will the pass/fail criterion will be applied if the crew initially clear how the initial misdiagnosis with subsequent recovery will be addressed in the V&V findings or how the generic aspects of this question be applied to other scenarios. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-69 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-69 (ML16074A294). Regarding the first point in the RAI, the applicant clarified that the evaluations in question are an overall assessment of crew 148 performance that is distinct from the assessment of performance during the scenarios. The evaluation will be made at the end of the scenarios, which is appropriate for such a summative evaluation. The applicant has proposed changes to the V&V IP to clarify these points that acceptably capture the information. Regarding the second point in the RAI, the applicant clarified that the evaluation of crew The staff finds this is a conservative approach to the assessment of crew responses and is therefore acceptable. In addition, it provides the opportunity to identify design changes as part of the HED assessment process. The applicant has proposed changes to the V&V IP to clarify these points and the staff finds that these changes acceptably capture the information. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-69. Therefore, RAI 352-8205, Question 18-69 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 3 The applicant should identify the secondary task measures applicable to each scenario. The Sta measurement. The V&V IP states that secondary tasks are measured based on observations by operations experts and participant debriefings. The applicant identifies a list of secondary tasks that are measured, including navigation between displays, searching for controls, and is assessed by looking at [ ] The V&V IP states that a seven-point Likert scale will be used to capture participant feedback at the end of each scenario. The staff finds that the application conforms to this review criterion. Criterion 4 The applicant should identify the measures of situation awareness applicable to each scenario. The primary way SA will be measured is the situation awareness rating technique (SART). SART is administered at the end of the scenario. The SART is a widely-used SA measure that has been successfully used in multiple industries, including the commercial nuclear industry. Information on operator SA will also be obtained from questionnaires and debriefings. The staff finds that the application conforms to this review criterion. Criterion 5 The applicant should identify the workload measures obtained for each scenario. 149 5 primary way workload will be measured is the National Aeronautics and Space Administration (NASA) Task Load Index (TLX). The TLX is a widely-used workload assessment tool with applications in many industries, including the commercial nuclear industry. It uses a rating scale method to assess six dimensions of workload, including mental demand, physical demand, temporal demand, performance, effort, and frustration level. The TLX is administered immediately following a scenario. The staff finds that the application conforms to this review criterion. Criterion 6 The applicant should identify the anthropometric and physiological measures obtained for each scenario. The during HSI Design Verification. This is appropriate since the design style guide defines most of the human performance considerations for these factors. During the ISV tests, observers will assess the dynamic aspects of anthropometric and physiological factors such as movement around the control room. Information will also be obtained from operators using post scenario questionnaires. These are reasonable approaches in general, however, the observer and operator questionnaires are not provided, thus the acceptability of the implementation of the approaches cannot be determined. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-71 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-71 (ML16074A294). The applicant described how these assessments will be made using questionnaires to be completed by operators and observers. The questionnaires will be based on HFE and test design best practices, HSI design, and test objectives. The applicant described how these assessments will be made using 7-point rating scales. The applicant has proposed changes to the V&V IP and the staff finds that these changes acceptably capture the information. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-71. Therefore, RAI 352-8205, Question 18-71 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Performance Measure Information and Validation Criteria NUREG-0711, Section 11.4.3includes five criteria for this topic. Criterion 1 The applicant should describe the methods by which these measures are obtained, e.g., by simulator data recording, participant questionnaires, or observation by subject-matter experts. 150 each measure described, the means of collecting the information is provided. In the case of the behaviorally anchored rating scale (BARS) and TLX, the rating scales are provided in appendices of the V&V IP. The staff finds that the application conforms to this review criterion. Criterion 2 The applicant should specify when each measure is obtained (recorded), such as continuously, at specific points during the scenario, or after the scenario ends. ection 4.5.5 of the V&V IP and in the detailed scenario description contained in the human factors V&V scenarios document. For SA measurement, the detailed scenario instructions for instructors, identify at what points the scenario should be stopped and the situational awareness global assessment technique (SAGAT) given to operators. The instructions also list the post-test evaluations to be made, such as the BARS and debriefings. The staff identified two concerns about when measures are taken. 1. It appears that the SAGAT measure of SA is taken twice for each scenario; once during a scenario stoppage and the second time at the end of the scenario. One of the values of the SAGAT method is that the changes in SA over the course of a scenario can be measured since the assessment is usually made at several points. The staff is concerned that a onetime measurement of SA during the scenario may not provide an accurate assessment. The staff is also concerned that use of the SAGAT measure following scenario termination may be of limited leading. At the end of the scenario, the plant has been stabilized. Since SAGAT is not a retrospective measure, all it may show is that the operators are aware they have achieved the stable condition. 2. The V&V IP states that the TLX will be collected at the end of each scenario. However, the detailed scenario descriptions in the V&V scenarios document do items. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-72 to address this issue (ML15356A485). On April 5, 2016, the applicant provided a response to RAI 352-8205, Question 18-SAGAT method, SART will be used. SART is administered at the end of the scenario, thus the question, the applicant clarified that the TLX will be obtained at the end for each scenario and it will be added to the post-scenarconcern. The applicant has proposed changes to the HF V&V Scenarios IP to clarify these points and the staff finds that these changes acceptably capture the information. 151 Based on the review of the HF V&V Scenarios IP, the staff has confirmed incorporation of the changes described above; therefore RAI 352-8205, Question 18-72 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 3 The applicant should describe the characteristics (see Table 11-1 of NUREG-0711) of the performance measures. Table 11-1 Characteristics of Performance Measures Characteristic Meaning Construct Validity A measure should represent accurately the aspect of performance it is intended to measure. Reliability A measure should be repeatable; i.e., same behavior measured in exactly the same way under identical circumstances should yield the same results. Sensitivity A measure's range (scale) and its frequency (how often data are collected) should be appropriate to that aspect of performance being assessed. Unobtrusiveness A measure should minimally alter the psychological or physical processes that are being investigated. Objectivity A measure should be based on easily observed phenomena. The Staff's Evaluation of Criterion 3 The applicant's description of measurement characteristics is contained in Section 4.5.5 of the V&V IP; however, they are given only cursory, and often incomplete, treatment in the description of the specific measures. For example, for the use of Halden's behaviorally anchored rating scale (BARS) questionnaire, the V&V IP states that: 'T he behavior categorization by observers provides reliability and unobtrusiveness." Observer ratings may be reliable or unreliable, depending on the characteristics of the scales being used and the instructions given to observers on what the scales mean and how they should be used. The same can be said for unobtrusiveness. The V&V IP does not mention the other characteristics that are listed in Table 11-1 of NUREG-0711. Another example, in the discussion of the SAGAT measure of SA, the V&V IP simply has "(constructvalidity, reliability)" at the end of one sentence with no explanation. Additional information is needed that shows how the measurement characteristics, for the measures selected, achieve the attributes described in Table 11-1 of this NUREG-0711 criterion, and that are defined on Page 34 of the V&V IP. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-73 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-73 (ML16074A294). The applicant proposed V&V IP changes which involve adding a paragraph to each measurement description that describes the characteristics addressed in NUREG-0711, Table 11-1. Each type of measure is evaluated below. 152 Plant Performance Measures The applicant has stated that these measures are obtained from the simulator log and the proposed changes to the V&V IP include a description of each of the characteristics in NUREG- 0711, Table 11-1 of plant performance measures. Primary Task Measures Time - This measure is obtained from the simulator log and has the same characteristics as plant parameters. Subjective Reports The applicant stated that such reports are collected post scenario, thus the reports are unobtrusive. The response further states that the characteristics of construct validity, reliability, sensitivity, and objectivity are not applicable. Since the subjective reports are debriefing sessions, the staff agrees. BARS BARS are an accepted behavioral assessment tool that is unobtrusively used. While this is generally true, the design of the specific BARS to be used by the applicant should be addressed. As rating scales, they can be well-designed for their purpose or poorly designed. sitivity, and objectivity are not applicable. The staff does not agree with this position. This is not consistent with the fact that specific scale being used by the applicant was carefully designed to provide construct validity (see NUREG/IA-0137), a reliability study was conducted, its sensitivity has been demonstrated, and it has been used successful in commercial nuclear industry. The applicant should clarify the response and explain why construct validity, sensitivity, and objectivity are not applicable to its BARS. A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. To support the meeting, KHNP provided the following information: The response to [RAI 352-8205,] Question 18-73 will be revised to add to sensitivity. The characteristic of objectivity is not applicable to the application of BARS since they require subjective opinions and therefore are not easily observed. Objectivity will remain identified as not applicable. In addition, a reference to Appendix B will be added. On August 9, 2016, the applicant provided a revised response to RAI 352-8205 Question 18-73 (ML16222A948), which clarified that the characteristics of construct validity, sensitivity, and objectivity for BARS is applicable Based on this information, the staff reviewed Appendix B, of five itemscommunication, team spirit, openness, coordination as crew, and task focus/decision making. The staff determined that this application of the BARS questionnaire is evaluating activities beyond what is directed by guidance documents. While it is important to ensure the ISV team is functional the regulatory guidance establishes this functionality by providing standards for operator experience, ISV participant training, and an ISV pilot. RAI 352- 8205, Question 18-73 regarding the BARS questionnaire was resolved and no further regulatory reviews are being conducted of this activity since it is beyond regulatory guidance and has no negative impact on the portion of the ISV for which regulatory guidance does apply. 153 Errors The applicant provided information on error measurement. Potential errors are identified for each scenario and its occurrence identified by operations experts, supporting its construct validity and sensitivity. Operations experts can unobtrusively and reliably identify them. Thus, the staff finds the applicant has described the characteristics of the primary task performance measures, and the characteristics are reasonable. Secondary Tasks The applicant did not provide information on secondary task measurement characteristics. A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. To support the meeting, KHNP provided the following information: A discussion of the characteristics for secondary task measures will be added on the top of page 36 of the V&V IP, as was done for primary tasks in the response, to the revised response to the Question 18-73. In the response to RAI 352-8205 Question 18-73 (ML16222A948), the applicant provided the characteristics for the secondary task measures. The applicant said [[the characteristics of the means used to assess secondary task performance (e.g., time, subjective report of participants, behavior categorization by observers, and errors), which are also means used to assess primary task performance, have the same characteristics of the means used to assess primary described the performance characteristics of the means used to evaluate primary tasks. As such, the applicant has described the characteristics of the secondary task measures. Situational Awareness In RAI 352-8205, Question 18-72 KHNP stated that SA would be assessed using the SART SAGAT are no longer applicable. Workload The NASA TLX will be used. As the applicant stated, this is an industry accepted method. The staff does not agree with this position. The applicant should address these characteristics. A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. To support the meeting, KHNP provided the following information: NASA-TLX can be considered a reliable measure, based on its industry performance. However, since it is based on opinion, it does not represent an objective measure. The response to 18-73 on page 37 of the human factors V&V IP for Workload will be revised to identify the characteristic of reliability. The staff finds this conclusion to be consistent with industry experience. 154 Video and audio recording measures The applicant provides measurement characteristics for video and audio recordings. The staff does not understand what this provides. Measurement characteristics have to do with the specific aspects of performance measures being assessed. Thus, they are applicable to specific measures being used to assess aspects of performance such as operator task performance, situation assessment, and workload. Video and audio recordings are not aspects of performance, per se; although they provide the raw data used by analysts. A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. To support the meeting, KHNP provided the following information: The response to Question 18-73, page 38 of the V&V IP will be revised by deleting the proposed revision to (6). The staff finds this acceptable as it treats video and audio recordings as a data collection tool rather than a performance measure. August 9, 2016, the applicant provided a revised response to RAI 352-8205, Question 18-73. The applicant stated it will revise the V&V IP, Section 4.5.5.1 to clarify the use of video and audio recordings for data collection. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-73. Therefore, RAI 352-8205, Question 18-73 was resolved and closed. The staff finds that the application conforms to this NUREG- 0711 criterion. Criterion 4 The applicant should identify the specific criterion for each measure used to judge the acceptability of performance and describe its basis. The and its basis is contained in Section 4.5.5 of the V&V IP and in the V&V scenarios document. The staff had the following questions related to the specific criterion for each measure: 1. The specific criteria to be used for many of the measures are in the detailed instructions for the validation scenarios. However, specific criteria are not provided for all measures, e.g., for the BARS, SA, and workload measures. 2. The bases for the criteria are summarized in Table 4-6. The V&V IP states that reference plant.were benchmarked. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-74 to address this issue (ML15356A485). On April 5, 2016, the applicant provided a response to RAI 352-155 8205, Question 18-74 (ML16096A208). The applicant proposed changes to the V&V IP to clarify the following: Plant operations SMEs will determine the pass/fail criteria for BARS, which is a measure of teamwork. The V&V IP, Appendix B, contains the BARS questionnaires and a 7-point ranking scale that will be evaluated by the plant operations SMEs when they observe the operators during the ISV. These plant operations SMEs will determine the pass/fail threshold for BARS. This is acceptable because the plant operations SMEs will have experience operating in a control room team and will be knowledgeable about behaviors that contribute to effective teamwork and collaboration, and therefore, these SMEs have the appropriate qualification to: (1) make evaluations about whether the integrated system provides the environment for operators to exercise behaviors that promote effective teamwork, and (2) identify the criteria for the BARS rating that will be indicate that the integrated system promotes effective teamwork and therefore is acceptable. For SA and workload, the applicant revised the V&V IP to identify the plants that will be used to benchmark the SART and NASA TLX values. These predecessor plants are the APR1400 design as constructed in Korea and the United Arab Emirates. The applicant stated in the RAI response that the ISV for these plants will be completed before the ISV for the APR1400 in the US. The SART and TLX values must at least meet the benchmarked values at the predecessor plants to be acceptable. Because the predecessors are similar in design and operations to the APR1400, this is a reasonable method for evaluating whether the values of SA and workload are acceptable. Based on the review of the V&V IP, the staff has confirmed incorporation of the changes described above; therefore RAI 352-8205, Question 18-74 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 5 The applicant should identify whether each measure is a pass/fail one or a diagnostic one. the V&V IP. The applicant defines each category as follows: Pass/fail (P/F) measures are identified in each scenario before the start of the ISV and are used to determine whether the design is validated. Diagnostic measures are collected to assess personnel performance during each scenario and to analyze errors and its root causes as they relate to the HSI design. However, the staff notes that the relative proportion of P/F vs. diagnostic measures is heavily weighted to P/F measures, thus the rational for selecting P/F vs. diagnostic measures is ion of P/F measures, the staff would not expect measures such as the behaviorally anchored rating scale (BARS) to be included. Additional needed. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-75 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-75 (ML16074A294). The applicant indicated that only objective measures will be used as P/F 156 e.g., BARS evaluations were changed to diagnostic rather than a P/F criterion. The staff agrees with these changes. The applicant has proposed changes to the IP to clarify these points and the staff finds that these changes acceptably capture the information. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-75. Therefore, RAI 352-8205, Question 18-75 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Test Design Scenario Sequencing NUREG-Criterion 1 The applicant should balance scenarios across crews to provide each crew with a similar, representative range of scenarios. quencing of scenarios. The ISV will use three crews and seven scenarios. Each crew will perform each scenario. This will provide a similar and representative range of scenarios to all crews. The staff finds that the application conforms to this review criterion. Criterion 2 The applicant should balance the order of presentation of scenarios to crews to provide reasonable assurance that the scenarios are not always presented in the same sequence (e.g., the easy scenario is not always used first). The will use three crews and seven scenarios. Each crew will perform each scenario. Table 4-7 of the V&V IP illustrates the order of scenario presentation across crews. The order is balanced so each crew is presented with scenarios in a different sequence. The staff finds that the application conforms to this review criterion. Test Procedures NUREG-Criterion 1 The applicant should use detailed, unambiguous procedures to govern the conduct of the tests. These procedures should include the following: the identification of which crews receive which scenarios, and the order in which they should be presented 157 detailed and standardized instructions for briefing the participants specific directions for the testing personnel on conducting the test scenarios, as elaborated in Scenario Definition (NUREG-0711, Section 11.4.1.3) guidance on when and how to interact with participants when difficulties occur in simulation or testing instructions on when and how to collect and store data. These instructions should stipulate which data are to be recorded by: o simulator computers o special-purpose instruments and devices for collecting data such as situation awareness- and workload-questionnaires, or physiological measures o video recorders (locations and views) o test personnel and subject-matter experts (such as via observational checklists) procedures for documentation: o identifying and maintaining files of test records including details of the crew and scenarios o data collected o logs created by those who conducted the tests The procedures should detail the types of information that should be logged (e.g., when the tests were performed, deviations from the test procedures and why they occurred, and any unusual events that may be important to understanding how a test was run or for interpreting the findings from it). The procedure also should state when the types of information should be recorded. human factors V&V scenarios document contains many aspects of the procedures that are specific to each of the seven scenarios. Table 4-7 of the V&V IP identifies the type of sequences that will be used for presenting scenarios for each crew. The human factors V&V scenarios document contains the scenarios descriptions and directions for testing personnel, instruction on data collection procedures and documentation. However, there are two aspects of the test procedures that were not addressed: (1) the detailed and standardized instructions used to brief participants, and (2) the guidance to test conductors for when and how to interact with participants when difficulties occur in simulation or testing. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-76 to address this issue (ML15356A485). 158 On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-76 (ML16074A294). On August 9, 2016, the applicant provided a revised response to RAI 352- 8205, Question 18-76 (ML16222A948), which included minor corrections and clarifications to the earlier response. The applicant provided the detailed instructions to brief participants for each scenario. These instructions are added to the scenario descriptions. In addition, the applicant developed a new Attachment 5 for each scenario providing explicit instructions for As noted above, the applicant has proposed changes to the scenario descriptions and the staff finds that these changes acceptably capture the information. The staff confirmed that the V&V IP was revised as committed in the response to RAI 352-8205, Question 18-76. Therefore, RAI 352-8205, Question 18-76 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 2 Criterion 2 procedures to minimize bias. The V&V IP states that the procedures will minimize bias; however, little information is provided on how they will do so. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-77 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-77 (ML16074A294). The applicant provided the detailed instructions for minimizing the opportunity all scenarios are managed in a consistent and controlled manor. This acceptably addresses the nt has proposed changes to the scenario descriptions and the staff finds that these changes acceptably capture the information. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-77. Therefore, RAI 352-8205, Question 18-77 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Training Test Personnel NUREG-topic. Criterion 1 The applicant should train test personnel (those who conduct or administer the validation tests) on the following: the use and importance of test procedures bias and errors that test personnel may introduce into the data through failures to follow test procedures accurately or to interact with participants properly 159 the importance of accurately documenting problems arising during testing, even if they were due to an oversight or error of those conducting the test Criterion 1 personnel. Personnel will receive five days of training prior to pilot testing. The scope of the training will address use of procedures, potential sources of bias, and the importance of documentation. Several additional topics will be addressed as well, including use of pass/fail and diagnostic measures, APR1400 HSI and system design, and the roles of validation team members. The staff finds that the application acceptably conforms to this review criterion. Training Participants NUREG-Criterion 1 lar to the training plant personnel operations, and the use of the HSIs and procedures represents that of experienced plant personnel. Participants should not be trained specifically to carry out the selected validation scenarios. participants. The V&V IP indicates that all participants are licensed and experienced with PWR operations. All have held positions similar to those they will assume in ISV testing. They will then receive training prior to selection to participate in ISV trials. The training will involve plant design, operations, operating procedures, and HSIs. Participants will not be trained on the specific scenarios used for ISV. The staff finds that the application conforms to this review criterion. Criterion 2 training of participants should result in near asymptotic performance (i.e., stable, not significantly changing from trial to trial) and should be tested for such before conducting the validation. The Sta participants. Participants will receive training on plant design, operations, operating procedures, and HSIs. Participants will have to obtain an 80 percent passing grade to demonstrate proficiency and eligibility to be included in ISV tests. The staff finds that the application acceptably conforms to this review criterion. Pilot Testing NUREG-160 Criterion 1 The applicant should conduct a pilot study before the validation tests begin to offer an opportunity for the applicant to assess the adequacy of the test design, performance measures, and data-collection methods. approximately six months prior to ISV trials to allow for resolutions of problems. The pilot test will be conducted by running all seven scenarios under actual test conditions. The testing will identify potential issues with the simulator, executing scenarios, test procedures, data collection, and training of testers and participants. The staff finds that the application conforms to this review criterion. Criterion 2 The applicant should not use participants in the pilot testing who will then be participants in the validation tests. The V&V IP, Section 4.5.testing will not be included in ISV testing crews. The staff finds that the application conforms to this review criterion. Data Analysis and HED Identification NUREG-0711, Section for this topic. Criterion 1 The applicant should use a combination of quantitative and qualitative methods to analyze data. The analysis should reveal the relationship between the observed performance and the established performance criteria. the analysis will use both qualitative and quantitative methods to assess the acceptability of the results. The staff finds that the application conforms to this review criterion. Criterion 2 The applicant should discuss the method by which data is analyzed across trials, and include the criteria used to determine successful performance for a given scenario. sis. For many evaluations, the applicant plans to conduct T-161 However, the staff has two questions about how T-test will be applied in the analyses. The first question concerns the use of the used to test hypotheses in behavioral science research, and the minimum set for claiming two means are significantly different. However, its application to the integrated system validation analyses is unclear. Using workload as an example, if the average workload of the APR1400 design is found to be 3.5, and the average for the reference plant is 4, but the difference is not statistically significant, then it is not apparent that a conclusion can be drawn regarding the acceptability of the workload. One cannot conclude the workload levels are statistically the same; that would be confirming the null hypothesis, which is not statistically justified. Greater specificity in the V&V IP is needed to the integrated system validation analyses. The second question concerns the fact that for the statistical tests conducted separately for each scenario, the means are based on three data points (three crews). Such a situation will lead to very few degrees of freedom for the statistical test and a great chance of failing to find statistical significance. Additional information is needed to address the issue of low degrees of freedom for scenario evaluations. Therefore, on December 22, 2015, the staff issued RAI 352- 8205, Question 18-78 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-78 (ML16074A294). The applicant provided additional clarification that the statistical analyses only provide input to the assessment of results which also include qualitative assessments and the convergence of results on a common result. Since the statistical tests only provide input to the analysis and are not the basis of deriving degrees of freedom in the statistical tests. The applicant proposed changes to the V&V IP to implement its response. While the proposed changes are reasonable, they do not include key d be included in appropriate locations in the V&V IP: Although simple quantitative statistical tools are applied to support the understanding of the meaning of the ISV data, the APR1400 ISV data analysis must rely on qualitative methods. These include the application of SME analysis and the assessment of the degree of convergence of multiple measures of performance, convergent validity. The conclusions drawn from the ISV use the resulting P values to only inform the qualitative analysis. A public meeting with KHNP was held on July 25, 2016, to communicate this concern. KHNP stated the response to Question 18-78 will be revised by adding this material to the V&V IP. On August 9, 2016, the applicant provided a revised response to RAI 352-8205, Question 18-78 (ML16222A948). In the response, the applicant stated that it will revise the V&V IP, Sections 4.5.7 and 4.5.8 to clarify the use of the resulting P values. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-78. Therefore, RAI 352- 8205, Question 18-78 was resolved and closed. In all cases, the applicant defines the criteria for successful performance. For example, for the reference plant is considered a success. For plant performance measures, the V&V IP states 162 that the [ ] However, the method used to complete this comparison is not clearly described. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-79 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-79 (ML16074A294). The applicant provided an explanation of how these analyses are performed. They are conducted by the implementation team using the scenario descriptions, simulator data, and video/audio tapes to compare expected performance (scenario descriptions) to the tasks actually performed by the crews. The methodology provides a means to assess the request. The applicant has proposed changes to the V&V IP that acceptably capture this information. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-79. Therefore, RAI 352-8205, Question 18-79 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 3 The applicant should evaluate the degree of convergence between related measures (i.e., consistency between measures expected to assess the same aspect of performance). In the fifth paragraph of Section 4.5.5 of the V&V IP, the applicant stated that [ ] However, how this evaluation is performed is not discussed. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-80 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-80 (ML16074A294). The applicant provided an explanation of how the analysis of convergence is performed. The analysis is performed following the collection of test data. Multiple analyses are performed including a review by test personnel of the different types of data collected to The applicant has proposed changes to the V&V IP and the staff finds that these changes acceptably capture this information. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-80. Therefore, RAI 352-8205, Question 18-80 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 4 When interpreting test results, the applicant should allow a margin of error to reflect the fact that actual performance may be slightly more variable than observed validation-test performance. 163 The V&V IP, does not specifically address margin of error in the interpretation of performance. Margin is [ ] However, how margin is used to make decisions is not discussed. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-81 to address this issue (ML15356A485). On March 14, 2016, the applicant provided a response to RAI 352-8205, Question 18-81 (ML16074A294). The applicant provided explanation of how the margin of error is evaluated. Three methods are discussed including: (1) a comparison by SME of the observed results and what would be expected in actual operating plants, (2) an assessment by SMEs of the differences between scenario-simulated plant parameters and those done for the safety analysis, and (3) an evaluation by SMEs of the differences between observed human performance and that expected. The staff identified some questions about these analyses: As of the first analysis, the staff is not sure what a comparison of results to that expected in actual operating plants reveals. Since the APR1400 is a new design, difference between such results may related to plant design and operational differences. The staff is not sure what this tells the analyst about margin of error. between scenario simulated plant parameters and those done for the safety behavior of the plant being modeled (or the validity of the safety analysis). The staff thinks such an analysis would be performed as part of simulator qualification before the simulator is used for ISV. As of the third analysis, the staff is not sure what the SMEs are evaluating. Clarification is needed as to what these analyses consist of and how they are used to consider margin of error when interpreting the results. A public meeting with KHNP was held on July 25, 2016, to communicate these concerns. KHNP explained that the general approach for applying margins during the interpretation of results was to apply subject matter expert knowledge of where actual operations diverged from the ISV test setting. But since KHNP has applicable operating experience from predecessor plants, it also wanted to utilize that experience to help identify conditions where actual experience has or could diverge from the ISV test setting. This approach was also developed to provide more quantifiable actions in support of its implementation level submittal. The applicant also stated it will revise the V&V IP to include the additional information provided in its revised response to RAI 352-8205, Question 18-81. The staff found this approach to be satisfactory as it complemented the usual practice of just using SME judgement to establish margins. On August 9, 2016, the applicant provided a revised response to RAI 352-8205, Question 18-81 discussed during the public meeting on July 25, 2016. Based on the review of the V&V IP, the staff has confirmed incorporation of the changes described above; therefore RAI 352-8205, 164 Question 18-81 was resolved and closed. The staff finds that the application conforms to this NUREG-0711 criterion. Criterion 5 The applicant should verify the correctness of the analyses of the data. This verification should be done by individuals or groups other than those who performed the original analysis, but may be from the same organization. s. The V&V IP states [ ] The staff finds that the application conforms to this review criterion. Criterion 6 The applicant should identify HEDs when the observed performance does not meet the performance criteria. Tapplicant has identified when the results will be classified as an HED. (Note that the staff requested clarification on how plant performance measures are assessed, but that is addressed in RAI 352-8205, Question 18-81, related to NUREG-0711 Criterion 11.4.3.7-2 reviewed above.) evaluated to determine the extent to which they may represent cross-cutting issues. If such a determination is made, the cross-cutting issue will be identified as an HED. The staff finds that the application conforms to this review criterion. Criterion 7 The applicant should resolve HEDs identified by pass/fail measures before the design is accepted. associated with pass/fail measures will be closed before the design is considered validated. The staff finds that the application conforms to this review criterion. Validation Conclusions NUREG- criteria for this topic. Criterion 1 The applicant should document the statistical and logical bases for determining that performance of the integrated system is and will be acceptable. 165 The V&V IP, Sectconclusions. The V&V IP presents the criteria to be used in determining that the design is validated, including the resolution of HEDs. An overview of the criteria including all measures is presented as an example in Table 4-further states that all analysis results will be documented in a ReSR. The staff finds that the application conforms to this review criterion. Criterion 2 The applicant should document the limitations in the validation tests, its possible effects on the conclusions of the validation, and its impact on implementing the design. The V&V IP, Section 4.5.8 describes conclusions. The V&V IP indicates that limitations of testing will be addressed and documented in a ReSR. The staff finds that the application conforms to this review criterion. 18.10.4.4 Human Engineering Discrepancy Resolution Review NUREG-includes five criteria for this topic. Criterion 1 HED Analysis Personnel Tasks and Functions The impact of HEDs on personnel tasks and the functions supported by those tasks. Plant Systems The impact of HEDs on plant systems, considering the safety significance of that system(s), their effect on accident analyses, and their relationship to risk- PRA. Cumulative Effects of HEDs The analysis of HEDs should identify the cumulative effects that multiple HEDs may have on plant safety and personnel performance. HEDs as Indications of Broader Issues As well as addressing specific HEDs, potentially broader problems. analysis of HEDs evaluates the impact of the HED on the following categories: Personnel tasks and functions Determination of the importance of the personnel function to safety and the cumulative effect on personnel performance 166 Plant systems Analysis of the impact of HEDs on plant systems, considering the safety significance of the systems, the effect on accident analysis, and the relationship to risk-significant sequences in the PRA Cumulative effects of HEDs Analysis of HEDs to identify the cumulative effects that multiple HEDs may have on plant safety and personnel performance. Section 4.5.9.1 also states that they will evaluate HEDs as indications of broader issues, including a determination of whether an HED indicates a potentially broader problem or an extent of the issue across the HSI system. Further, Section 4.5.7.3 notes that HEDs may cut across scenarios and thus represent cross-cutting issues. This section explains how these issues will be evaluated and gives examples of areas to be considered in the data analysis step is done by the team of SMEs. The V&V IP, Section 5 provides information on the various SME teams and specifies that the HED Analysis Team will contain experts in: plant operations, HFE, systems engineering, I&C engineering. The staff finds that the application conforms to this review criterion. Criterion 2 Selection of HEDs to Correct The applicant should conduct an evaluation to identify which HEDs to correct. The evaluation should identify those HEDs that are acceptable as is. The remaining discrepancies should be denoted as HEDs to be addressed by the HED-resolution process. HEDs the applicant should correct are those with direct safety consequences, namely, those that could adversely impact personnel performance such that the margin of plant safety may be reduced below an acceptable level. Unacceptability is indicated by such conditions as violations of Technical Specification safety limits, operating limits, or limiting conditions for operations, or failing an ISV pass/fail criterion. HEDs with potential safety impact, not as severe as those described above, also should be corrected unless the applicant justifies leaving the condition as is. The applicant should correct HEDs that may adversely impact personnel performance in a way that has potential consequences to plant performance or operability of structures, systems, or components, and personnel performance or efficiency. This may include failing to meet personnel information needs or violating HFE guidelines for tasks associated with plant productivity, availability, and protecting investment. both describe a prioritization scheme for HEDs that is used in the evaluation and determination of which HEDs to correct and which can remain as-is. In this scheme, the applicant sorts HEDs into Priority 1, 2, and 3 as follows. Priority 1 HEDs are identified as must be corrected. Priority 2 HEDs are corrected unless the analysis justifies the HED can remain as is. Priority 3 HEDs are acceptable as-is. 167 consequences but potential safety consequences to plant performance/operability, non-safety PP states that Priority 3 HEDs are all those remaining HEDs that are not Priority 1 or 2. may be reduced below margin of plant safety is reduced to below wo instances are somewhat contradictory. Therefore, on December 22, 2015, the staff issued RAI 352-8205, Question 18-82 to address this issue (ML15356A485). On April 16, 2016, the applicant provided a response to RAI 352-8205, Question 18-82, (ML16107A079). In the response to Question 18-definition of HED priorities in Section 4.5.9.2 of the human factors V&V IP will be revised to acceptable. The staff finds that the application conforms to this review criterion. The staff confirmed that the V&V IP was revised as committed in response to RAI 352-8205, Question 18-82. Therefore, RAI 352-8205, Question 18-82 was resolved and closed. Criterion 3 Development of Design Solutions The applicant should identify design solutions to correct HEDs. As part of the design solution, the application should evaluate the interrelationships of individual HEDs. 3 The V&V IP, Section 4.5.9.3 states that the SME team is responsible for the analysis of V&V HEDs. As part of this analysis, the SMEs consult design experts for input on design options. The SME team as a whole determines the final design option and forwards the recommendation to management per the HFE PP. The V&V IP, Section 4.5.7.3 states HEDs are evaluated to see if they cut across scenarios and thus represent a cross-cutting issue. The HEDs are also reviewed for commonality by considering the characteristics of the HSIs (e.g., display format, display content, panel layout). Any issue found to represent a cross-cutting problem is identified as such and added as a separate HED. Per Section 4.5.9.1 the HEDs are analyzed to identify cumulative effects of multiple HEDs and to determine whether an HED indicates a broader problem with the overall HSI system. The staff finds that the application conforms to this review criterion. Criterion 4 Design Solution Evaluation The applicant should evaluate design solutions to demonstrate the resolution of that HED and to ensure that new HEDs are not introduced. Generally, the evaluation should use the V&V method that originally detected the HED. evaluated to demonstrate the resolution of that HED, and to ensure that new HEDs are not 168 introduced. This evaluation will also determine any effects on earlier HFE program elements. If necessary, the specific earlier HFE analysis will be redone. If additional testing is needed, e. g., part of the ISV, then that will be repeated. The selected resolution is also assessed for its potential impact on operating procedures and the operator training program. The staff finds that the application conforms to this review criterion. Criterion 5 HED Evaluation Documentation The applicant should document each HED, including: the basis for not correcting an HED related personnel tasks and functions related plant systems cumulative effects of HEDs HEDs as indications of broader issues 5 HED is documented in the V&V ReSR. The bulleted aspects listed in this criterion are included in the documentation requirements. The HFE PP, Section 4.6.1.3 states that the resolution process for HEDs includes describing the resolution and related documents/drawings. The criteria for the acceptance of the resolution are included in the ITS. The HFE PP, Section 4.6.1.4 states that HEDs are closed if they are properly incorporated into the related HSI documents and drawings. An HED may also be closed when the issue has been determined not to require any actions and therefore does not require further tracking. HEDs that do not require action are also documented in the ITS. The HFE PP, Section 4.6.1.5 states that HED resolutions requiring design changes are documented in the ITS, and are summarized in both the respective program element of the HFE program and in design documents. Further, the ReSRs of the HFE program include summaries of the identified HEDs and their status. All Priority 1 HEDs and a summary of Priority 2 HEDs are included in the ReSR for the V&V element. The staff finds that the application conforms to this review criterion. Combined License Items There are no COL items associated with Section 18.10 of the APR1400 DCD. Conclusion The staff reviewed the application for human factors V&V using the criteria in NUREG-0711, conducted at the IP level; thus, the focus was on the methodology the applicant will use to conduct the V&V. 169 The results of the V&V program will be documented in a ReSR. The contents of the ReSR are described in DCD Section 18.10.3 and the V&V IP, Section 6. These contents, as described by the applicant, are consistent with the ReSR contents described in NUREG-0711, Section 11.3. NUREG-provides for identifying the control room inventory and for determining there is reasonable assurance the operator tasks can be implemented effectively. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element. 18.11 Design Implementation Introduction the verified and validated design that resulted from the HFE design process. Summary of Application DCD Tier 1: The Tier 1 information associated with this element is found in Section 2.9 of -K-X-IT-14001. Changes to Tier 1 information are governed by the change control processes in the design certification rule for the ARP1400 design. DCD Tier 2-K- I-NR- Changes to Tier 2 information are governed by the change control processes in the design evaluation of how the information in DCD Tier 1, Section 2.9, constrains changes to Tier 2 information, including the HFE IPs. ITAAC: The ITAAC associated with this element is listed in DCD Tier 1, Section 2.9, Table 2.9-1. Section 14.3.9 of this SER contain the staf TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: TeRs associated with this element are: APR1400-K-I-NR- (ML17094A129) Regulatory Basis NUREG-0800. The applicable regulatory requirements are as follows: 10 CFR 52.47(a)(8) 170 10 CFR 50.34(f)(2)(iii) Other regulatory guidance are as follows: NUREG- NUREG-0800, Revision 2, Chapter 18, of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Technical Evaluation NUREG-cation for New Plants and Control Room Criterion 1 The applicant should evaluate aspects of the design that were not addressed in V&V by an appropriate V&V method. Criterion 1 The DI IP, Section 3 lists design aspects that were not verified and validated during the V&V process as one of the items that is evaluated during the design implementation (DI) process. The overall scope of DI includes the MCR, RSR, TSC, EOF, and LCSs associated with IHAs. DI IP Section 4 specifically calls out aspects of the design which are not addressed during ISV but that will be evaluated during design implementation. The list includes control room lighting and noise, control room temperature and humidity, and site-specific configurations that were not modeled during ISV. In regard to control room lighting, noise, and temperature and humidity, of these characteristics. In terms of the site-provides the method used to evaluate the as-built design, whereby the SME compares the as- built configuration or site-specific configuration to the approved final design documentation. The DI IP also describes other V&V methods such as acceptance testing and performance testing to ensure the design criteria have been satisfied. In Section 4.2.3, the IP has the SME check to ensure the mean value for each condition is within the boundary recommended by the HFE guidelines and approved HSI design documentation. The specific documents to be used were not identified. Therefore, on December 15, 2015, the staff issued RAI 337-8388, Question 18-58 to address this issue (ML15349A920). On January 12, 2016, the applicant provided a response to RAI 337-8388, Question 18-58 (ML16012A543). The response stated that APR1400-E-I-NR-describes the boundary of environmental conditions such as temperature, humidity, intensity of illumination, and noise. APR1400-K-I-NR-to reference the style guide. This provides for specific acceptance criteria and is therefore acceptable. The staff confirmed that the Design IP was revised as committed in response to RAI 352-8205, Question 18-58. Therefore, RAI 352-8205, Question 18-58 was resolved and closed. 171 The staff concludes that aspects of the design not addressed in the ISV will be appropriately verified and validated. Therefore, the staff finds that the application conforms to this criterion. Criterion 2 The applicant should compare the final HSIs, procedures, and training with the detailed description of the design to verify that they conform to the planned design resulting from the HFE design process and V&V activities. This verification should compare the actual HSI, procedures, and training materials to design descriptions and documents. Any identified discrepancies should be corrected or justified. -built HSIs to the design process for comparing the final procedures and training to the validated and verified design was not done because the verification of procedures and training is conducted during the inspection of operational programs. -level objectives and scope for the as-built HSI verification. The DCD states that the objective of the KHNP design implementation is to demonstrate that the as-built HSI configuration is the same as the validated and verified design. DI IP, Section 4 includes guidance to check that the as-built design matches the validated and verified hardware configuration, software configuration, and facility configuration for the MCR, RSR, TSC, and LCSs. Where there are acceptable differences to the validated and verified design, the SME documents the basis for keeping the as-built configuration. This is done through a design change analysis conducted by an HFE SME which verifies that the change has no impact on human performance. Any unacceptable difference is documented as a HED and addressed in the HED resolution process. After the design verification process is complete, the results are entered into a Final Summary Report. The staff finds that the applicant has provided a clear methodology for comparing the as-built HSI configuration to the validated and verified design and ensuring any deviations are reconciled. Therefore, the staff finds that the application conforms to this criterion. Criterion 3 The applicant should verify that all HFE-related issues in the issue-tracking system are adequately addressed. -evaluated, justified, and documented to ensure the final V&V results are unaffected prior to the completion of Design Implementation. Therefore, the staff finds that the application conforms to this criterion. 172 Criterion 4 The applicant should provide a description of how the HFE program addressed each important HA. APR1400-E-I-NR-14006 describes in detail how IHAs are addressed within the HFE program; specifically, the IP describes how IHAs are selected and validated. Section 18.6 of this SER states that all IHAs associated with the MCR and RSR are verified and validated during the ISV. The IHAs related to the LCSs are verified and evaluated during DI. Section 6 states that the description of how the HFE program addressed delineation of each IHA is included in the ReSR. Therefore, the staff finds that the application conforms to this criterion. Combined License Items There are no COL items associated with Section 18.11 of the APR1400 DCD. Conclusion the criteria in NUREG-0711, Section design implementation method provides for reasonable assurance that the as-built design conforms to the design resulting from the V&V, and the implementation of plant changes consider the impact on personnel performance and safe operation. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element. 18.12 Human Performance Monitoring Introduction The objective performance monitoring strategy for ensuring that no significant safety degradation occurs because of any changes that are made in the plant and to verify that the conclusions that have been drawn from the human performance evaluation remain valid over the life of the plant. Summary of Application DCD Tier 1: There is no Tier 1 information associated with this element. DCD Tier 2: The applicant identified a COL item that will address this element. ITAAC: There are no ITAAC associated with this element. TS: There are no TS associated with this element. TRs: There are no TRs associated with this element. TeRs: There are no TeRs associated with this element. 173 Regulatory Basis NUREG-0800. The applicable regulatory requirements are as follows 10 CFR 52.47(a)(8) 10 CFR 50.34(f)(2)(iii) Other regulatory guidance are as follows: NUREG- NUREG-0800, Revision 2, Chapter 18, of Safety Analysis Reports for Nuclear Power Plants: LWR Edition Human Technical Evaluation The DCD Tier 2, Section 18.12 contains one COL item pertaining to human performance monitoring. The acceptability of the COL item is evaluated below in this SER section. The staff concluded that no additional COL items were needed. Combined License Items The DCD Tier 2, Section 18.12 contains one COL item pertaining to human performance monitoring. The acceptability of the COL item is discussed below. The staff concluded that no additional COL items were needed. Table 18.12-1 Combined License Items Identified in the DCD Item No. Description Section 18.12(1) The COL Applicant is to develop the Human Performance Monitoring Program. 18.12 Conclusion A COL item has been identified for this element because a human performance monitoring IP was not provided for this HFE element. This is acceptable as the monitoring of human performance, which includes maintaining personnel skills and ensuring no safety degradation from modifications to the design, starts after the plant becomes operational and is therefore a COL activity. Accordingly, the staff finds the application satisfies the requirements in 10 CFR 50.34(f)(2)(iii) and 10 CFR 52.47(a)(8) related to this element. 174 References 1. APR1400-E-I-NR- Allocation Implementati (ML18081A107). 2. APR1400-E-I-NR- (ML18212A336). 3. APR1400-E-I-NR- (ML18081A107). 4. APR1400-E-I-NR- (ML18081A107). 5. APR1400-E-I-NR- (ML18178A202). 6. APR1400-E-I-NR--(ML18178A202). 7. APR1400-E-I-NR-- (ML18178A202). 8. APR1400-E-I-NR- (ML18178A202). 9. APR1400-E-I-NR-(ML18178A202). 10. APR1400-E-I-NR-(ML18081A107). 11. APR1400-K-I-NR-(ML17094A129). 12. APR1400-K-I-NR- (ML17094A153). 13. APR1400-K-Q-TR-ption (QAPD) for (ML16123A404). 14. NUREG-Nuclear Power Plants: LWR 15. NUREG- 3, November 2012 (ML12324A013). 16. (ML070720184). 17. NUREG-(ML051390358). 175 18. NUREG-an-(ML021700373). 19. NUREG/CR- (ML063480112). 20. NUREG/CR-Allocation of Nuclear Power Plant Control 1983. 21. NUREG- (ML071580631). 22. SECY-05-and Generic Emergency Planning Inspections, Tests, Analyses, and Acceptance (ML052770225). 23. NUREG-r the Safety Parameter Display (ML102520360). 24. NUREG- (ML090060858). 25. NUREG-1983 (ML102560009). 26. (ML092330064). 27. June 2006 (ML061580448). 28. APR1400-Z-J-NR-14012, "Control System CCF Analysis," (ML18212A336). 29. APR1400-Z-A-NR- (ML18086B746). 30. APR1400-Z-J-NR- (ML18212A336).}}