Text

LLC Submittal of Human Factors Engineering Verification and Validation Results Summary Report, RP-1018-61289, Revision 1
	ML19212A773
Person / Time
Site:	NuScale
Issue date:	07/31/2019
From:	Rad Z; NuScale
To:	; Document Control Desk, Office of New Reactors
Shared Package
ML19212A772	List: ML19212A773;
References
	LO-0719-66476 RP-1018-61289-NP, Rev 1
	Download: ML19212A773 (122)
	v • d • e

LO-0719-66476 July 31, 2019 Docket No.52-048 U.S. Nuclear Regulatory Commission ATTN: Document Control Desk One White Flint North 11555 Rockville Pike Rockville, MD 20852-2738

SUBJECT:

NuScale Power, LLC Submittal of Human Factors Engineering Verification and Validation Results Summary Report, RP-1018-61289, Revision 1

REFERENCES:

Letter from NuScale Power, LLC to U.S. Nuclear Regulatory Commission Submittal of Human Factors Engineering Verification and Validation Results Summary Report, RP-1018-61289, Revision 0, dated March 18, 2019 (ML19077A331)

NuScale Power, LLC (NuScale) hereby submits Revision 1 of the Human Factors Engineering Verification and Validation Results Summary Report, (RP-1018-61289). contains the proprietary version of the report titled Human Factors Engineering Verification and Validation Results Summary Report, Revision 1. NuScale requests that the proprietary version be withheld from public disclosure in accordance with the requirements of 10 CFR § 2.390. The enclosed affidavit (Enclosure 3) supports this request. Enclosure 2 contains the nonproprietary version of the report titled Human Factors Engineering Verification and Validation Results Summary Report, Revision 1.

This letter makes no regulatory commitments and no revisions to any existing regulatory commitments.

If you have any questions, please contact Nadja Joergensen at 541-452-7338 or at njoergensen@nuscalepower.com.

Sincerely, Zackary W. Rad Director, Regulatory Affairs NuScale Power, LLC Distribution: Gregory Cranston, NRC, OWFN-8H12 Samuel Lee, NRC, OWFN-8H12 Prosanta Chowdhury, NRC, OWFN-8H12 : Human Factors Engineering Verification and Validation Results Summary Report, RP-1018-61289-P, Revision 1, proprietary version : Human Factors Engineering Verification and Validation Results Summary Report, RP-1018-61289-NP, Revision 1, nonproprietary version : Affidavit of Zackary W. Rad, AF-0719-66477 NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 Office 541.360-0500 Fax 541.207.3928 www.nuscalepower.com

LO-0719-66476 :

Human Factors Engineering Verification and Validation Results Summary Report, RP-1018-61289-P, Revision 1, proprietary version NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 Office 541.360-0500 Fax 541.207.3928 www.nuscalepower.com

LO-0719-66476 :

Human Factors Engineering Verification and Validation Results Summary Report, RP-1018-61289-NP, Revision 1, nonproprietary version NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 Office 541.360-0500 Fax 541.207.3928 www.nuscalepower.com

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Human Factors Engineering Verification and Validation Results Summary Report July 2019 Revision 1 Docket: 52-048 NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 www.nuscalepower.com

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 COPYRIGHT NOTICE This report has been prepared by NuScale Power, LLC and bears a NuScale Power, LLC, copyright notice. No right to disclose, use, or copy any of the information in this report, other than by the U.S. Nuclear Regulatory Commission (NRC), is authorized without the express, written permission of NuScale Power, LLC.

The NRC is permitted to make the number of copies of the information contained in this report that is necessary for its internal use in connection with generic and plant-specific reviews and approvals, as well as the issuance, denial, amendment, transfer, renewal, modification, suspension, revocation, or violation of a license, permit, order, or regulation subject to the requirements of 10 CFR 2.390 regarding restrictions on public disclosure to the extent such information has been identified as proprietary by NuScale Power, LLC, copyright protection notwithstanding. Regarding nonproprietary versions of these reports, the NRC is permitted to make the number of copies necessary for public viewing in appropriate docket files in public document rooms in Washington, DC, and elsewhere as may be required by NRC regulations.

Copies made by the NRC must include this copyright notice and contain the proprietary marking if the original was identified as proprietary.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Department of Energy Acknowledgement and Disclaimer This material is based upon work supported by the Department of Energy under Award Number DE-NE0008820.

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 CONTENTS Abstract ....................................................................................................................................... 1 Executive Summary .................................................................................................................... 2 1.0 Introduction ..................................................................................................................... 4 1.1 Purpose ................................................................................................................. 4 1.2 Scope .................................................................................................................... 4 1.3 Abbreviations and Definitions ................................................................................ 4 2.0 Verification and Validation Results Overview............................................................... 8 2.1 General .................................................................................................................. 8 2.2 Acceptance Criteria ............................................................................................... 9 2.3 Diagnostic Measures ............................................................................................. 9 3.0 Simulator Overview ....................................................................................................... 11 3.1 Detailed Simulator Description ............................................................................ 12 3.1.1 Simulator Hardware ............................................................................................. 12 3.1.2 Simulator Models ................................................................................................. 14 3.1.3 NuScale Simulator Interface Development Environment .................................... 14 3.1.4 Simulated Plant Systems .................................................................................... 15 3.2 Simulator Certification Testing ............................................................................. 23 3.3 Simulator Deficiencies ......................................................................................... 24 3.4 Simulator Issues During Integrated System Validation........................................ 28 4.0 Design Verification ........................................................................................................ 29 4.1 Human-System Interface Inventory and Characterization ................................... 29 4.1.1 Inventory and Characterization Results .............................................................. 30 4.2 Human Factors Engineering Design Verification ................................................. 30 4.2.1 Human-System Interface Display Page Icon Consistency .................................. 31 4.2.2 Human-System Interface Display Page Icon Consistency Results ..................... 32 4.2.3 Human-System Interface Display Page Theme .................................................. 32 4.2.4 Human-System Interface Display Page Theme Results ..................................... 33 4.2.5 Overall Main Control Room Design Results ........................................................ 35 4.3 Human-System Interface Task Support Verification ............................................ 36 4.3.1 Embedded Procedures ........................................................................................ 37 4.3.2 Automations ........................................................................................................ 37 4.3.3 Human-System Interface Task Support Results .................................................. 37

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.0 Integrated System Validation Test Implementation ................................................... 39 5.1 Scenario Development ........................................................................................ 40 5.1.1 Scenario Construction ......................................................................................... 40 5.1.2 Scenario Descriptions ......................................................................................... 42 5.1.3 Scenario Assumptions ......................................................................................... 54 5.1.4 Scenario-Based Testing....................................................................................... 55 5.2 Operating Crew Overview ................................................................................... 56 5.2.1 Operating Crew Participant Training Overview.................................................... 58 5.2.2 Integrated System Validation Scenario Security Overview.................................. 61 5.3 Validation Team ................................................................................................... 62 5.4 Observation Team Overview ............................................................................... 63 5.4.1 Observation Team Training.................................................................................. 63 5.5 Pilot Testing ......................................................................................................... 64 5.6 Data Collection Methods ..................................................................................... 66 5.6.1 Electronic Data Collection ................................................................................... 66 5.6.2 Manual Data Collection ....................................................................................... 67 5.7 Data Analysis ....................................................................................................... 67 5.8 Performance Measures ....................................................................................... 71 5.8.1 Acceptance Criteria Overview ............................................................................. 71 5.8.2 Diagnostic Measures Overview ........................................................................... 80 5.9 Test Administration Issues during Integrated System Validation ......................... 91 6.0 Human Engineering Deficiencies Overview ............................................................... 93 6.1 Human Engineering Discrepancy Description ..................................................... 93 6.2 Human Engineering Discrepancies Closed Prior to Results Summary Report Submittal .................................................................................................. 98 7.0 References ................................................................................................................... 102 8.0 Appendices .................................................................................................................. 103 Appendix A. Integrated System Validation Testing Schedule ........................................... 104 Appendix B. Test Performance Matrix ................................................................................. 108 Appendix C. Training Inputs to COL Holder Approved Training Program ....................... 109

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 TABLES Table 1-1 Abbreviations ............................................................................................................ 4 Table 1-2 Definitions ................................................................................................................. 6 Table 3-1 List of Plant Systems Simulated ............................................................................. 15 Table 3-2 List of Plant Systems Not Simulated ....................................................................... 18 Table 3-3 List of Simulator Deficiencies for Integrated System Validation .............................. 25 Table 3-4 Simulator Performance Issues ................................................................................ 28 Table 5-1 Test Plan to Scenario Cross Reference .................................................................. 40 Table 5-2 Sample of Operating Conditions to Scenario Cross Reference .............................. 41 Table 6-1 The Following Human Engineering Discrepancies were Identified During Design Verification .............................................................................................................. 93 Table 6-2 The Following Human Engineering Discrepancies were Identified During Integrated System Validation Testing ...................................................................... 96 Table 6-3 Design Verification Human Engineering Discrepancies Closed Prior to Results Summary Report Submittal ..................................................................................... 98 Table 6-4 Integrated System Validation Human Engineering Discrepancies Closed Prior to Results Summary Report Submittal .................................................................. 100 Table B-1 ((2(a),(c) ....................................................................... 108 FIGURES Figure 1-1 NUREG-0711, Figure 11-1, Overview of verification and validation activities ........... 2 Figure 3-1 Simulator Layout ..................................................................................................... 13 Figure 3-2 NuScale Simulator .................................................................................................. 14 Figure 4-1 NuScale Human-System Interface Display Page Theme ....................................... 33 Figure 5-1 Discrepancy Categories .......................................................................................... 69 Figure 5-2 Ratios of Actual Performance Compared to the Time Allowed for Primary Tasks ... 77 © Copyright 2019 by NuScale Power, LLC vi

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Abstract The Verification and Validation (V&V) Human Factors Engineering (HFE) element consists of four major activities: Sampling of Operational Conditions (SOC), Design Verification, Integrated System Validation (ISV), and Human Engineering Discrepancy (HED) Resolution. NuScale employed a sampling strategy to guide the selection of human-system interface (HSI) to evaluate. This is important because it is impractical to review all HSI due to the large number of tasks and combinations of HSI that are possible. Scenarios were created that sampled tasks involving normal, abnormal, and emergency conditions. Design verification involves HFE design verification, HSI inventory and characterization, and HSI task support verification. HFE design verification verifies that the HSIs are designed in accordance with HFE design guidance. HSI inventory and characterization describes all HSI displays, controls, and related equipment within the scope defined by the SOC. HSI task support verification verifies that the HSI supports task performance as defined by the task analysis. Integrated System Validation is an evaluation, using performance-based tests, to determine whether an integrated system design (i.e., hardware, software, and personnel elements) meets performance requirements and supports the plants safe operation. Human Engineering Discrepancies are identified if HSI displays are discrepant when compared against design guidance or ISV performance criteria are not met. Verification and Validation evaluations comprehensively determined that the HFE design conforms to HFE design principles and that they enable plant personnel to successfully perform tasks to ensure plant safety and operational goals. Verification and Validation evaluations were conducted in accordance with the methodology described in Human Factors Verification and Validation Implementation Plan. This report documents the V&V evaluation results and conclusions. © Copyright 2019 by NuScale Power, LLC 1

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Executive Summary NUREG-0711, Chapter 11, Human Factors Verification and Validation, provides guidance for performing verification and validation. This guidance is addressed in the Human Factors Verification and Validation Implementation Plan, (Reference 7.1.12), which provides specific direction for completing the design verification and the ISV of the NuScale human factors engineering design. Design verification and ISV are both subsets of the verification and validation process. Figure 1-1 from NUREG-0711 illustrates how both the design verification and ISV work together to ensure that completed main control room design supports safe operation. Figure 1-1 NUREG-0711, Figure 11-1, Overview of verification and validation activities Design verification verifies that the HFE design conforms to HFE design principles and that the control room HSI implements requirements from the design and the systematic task analysis. This enables plant personnel to successfully and reliably perform their tasks to ensure plant safety and operational goals. Design verification activities were performed by human factors engineering subject matter experts independently checking work performed by the interface design team. Design verification activities were mostly completed prior to ISV to determine how effectively the control room HSI supported the operators during dynamic testing. Integrated system validation is the process by which an integrated system design (i.e., hardware, software, and personnel elements) is evaluated using performance-based tests to determine whether it acceptably supports safe operation of the plant. The validation process is a check of a sampling of conditions that demonstrate that normal operation, abnormal operation, emergency operation, and certain beyond-design-basis events can be performed. The validation process does not check every possible combination of conditions, but is a representative sample to provide confidence in a safe control room design. ISV comprises dynamic exercises involving trained operating crews, a full scope simulator, and evaluation by a qualified team of observers. © Copyright 2019 by NuScale Power, LLC 2

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 The NuScale ISV was performed from July 23 through September 6, 2018. Prior to the performance of ISV, individuals were selected to participate in a training program to qualify them as ISV-certified operators. This training primarily focused on technical design knowledge, but also stressed the importance of providing feedback during the ISV testing period. (( }}2(a),(c) scenarios were developed based on the sampling of operational conditions listed in NUREG-0711. Taken together, these scenarios covered all of the sampling conditions listed. Specific focus was given to ensure that all credited operator actions performed in the main control room were assessed. All of the credited operator actions are used to mitigate beyond-design-basis events and there are no actions credited within the scope of design-basis events. ((

                                                                             }}2(a),(c) The crews were rotated so that they each participated equally. ((
                                                                   }}2(a),(c)

An observation team of HFE and Operations subject matter experts observed and commented on each scenario. The ISV crew members provided feedback through various questionnaires and structured critique sessions. Analysis of the feedback, observations, and test data were used to identify human engineering discrepancies (HEDs). Human Engineering Discrepancies that resulted from the validation testing process were documented for resolution. Human Engineering Discrepancies with nuclear safety significance are required to be resolved and retested prior to approval of the NuScale design certification application; however, no HEDs were identified that have nuclear safety significance. Lower level HEDs were prioritized and categorized in accordance with the Human Factors Verification and Validation Implementation Plan (V&V IP). HEDs were entered into the Human Factors Engineering Issue Tracking System (HFEITS) to track resolution. © Copyright 2019 by NuScale Power, LLC 3

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 1.0 Introduction 1.1 Purpose This report describes the results of the verification and validation activities and describes how these results demonstrate the HFE design provides for safe operation of the plant and reflects state-of-the-art human factor principles in accordance with 10 CFR 50.34(f)(2)(iii). 1.2 Scope Design verification reviews and integrated system validation are described by this document. 1.3 Abbreviations and Definitions Table 1-1 Abbreviations Term Definition CFD containment flood and drain system CI containment isolation CNT containment CNV containment vessel COL combined operating license CRH control room habitability system CRS Control Room Supervisor CRV control room ventilation system CVCS chemical and volume control system CWS circulating water system DHRS decay heat removal system EAL Emergency Action Level ECCS emergency core cooling system EDS highly reliable DC power system ESFAS engineered safety features actuation system GVD group view display HED human engineering discrepancy HFE human factors engineering © Copyright 2019 by NuScale Power, LLC 4

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Term Definition HFEITS human factors engineering issues tracking system HSI human-system interface I&C instrumentation and control IHA important human action ISV integrated system validation LTOP low temperature over pressure MCS module control system MPS module protection system MSIV main steam isolation valve NRC Nuclear Regulatory Commission NSIDE NuScale simulator interface development environment P&ID piping and instrumentation diagram PCS plant control system PCU pool cleanup PRA probabilistic risk assessment PZR pressurizer RCCWS reactor component cooling water system RCS reactor coolant system RO Reactor Operator RSR results summary report RTS reactor trip system RX reactor SA situational awareness SBT scenario-based testing SCWS site cooling water system SDI safety display and indication SFP spent fuel pool SM Shift Manager SOC sample of operating conditions © Copyright 2019 by NuScale Power, LLC 5

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Term Definition STA Shift Technical Advisor TG turbine generator TLX Task Load Index (NASA) TRM Technical Requirements Manual V&V verification and validation VDU visual display unit Table 1-2 Definitions Term Definition A person that has completed the NuScale HFE/Operations initial Independent observer company training program and has had no involvement in NuScale control room design decisions. A person that has experience in critical observations that has completed ISV-specific observation training. Observers as a Observer group applied their HFE and Operations expertise to the assessment of ISV activities. A person that has completed the NuScale HFE/Operations initial company training program, has previous licensed operating Operations subject matter nuclear plant experience, and has performed task analysis or expert NuScale system reviews so they are familiar with the NuScale plant design. A person that has been selected as a control room operator to perform ISV exams. This person has not been involved in the Participant plant design and has no knowledge of the ISV scenarios prior to test performance. A document that describes the initial conditions, sequence of Scenario guide events, and evaluation criteria that is used in simulator testing of participants. Facility constructed to model (as close as practical) the actual NuScale design control room. NUREG-0711 uses the term Simulator validation testbed to describe the area in which the HSI is displayed for performance evaluations. Within this document the testbed is referred to as the simulator. A person assigned to operate the simulator or provide outside Simulator operator (Booth support for the performance of ISV. Simulator operators may act operator) as plant personnel to role play interactions or inject and control the scenario inputs as directed by the scenario guides. © Copyright 2019 by NuScale Power, LLC 6

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Term Definition A database used by the simulator development group to TestTrack prioritize and track simulator changes. All of the personnel involved with performing ISV-related Validation Team activities including observers, simulator operators, test developers, and test participants. An information management system used to build an integrated learning system foundation. This program is used to document VISION functional requirements analysis/function allocation (FRA/FA) and task analysis from which procedures, training, and qualification instruments are created. © Copyright 2019 by NuScale Power, LLC 7

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 2.0 Verification and Validation Results Overview 2.1 General Design verification activities were conducted between August 23, 2017 and July 23, 2018, in parallel with operator training and pilot testing activities in order to capture dynamic interaction with the HSI interfaces and embedded operating procedures and controls. Most design verification activities were completed prior to validation testing commencing on July 23, 2018. The conclusion of design verification testing is that the main control room design and HSIs support monitoring and controlling the plant. A total of (( }}2(a),(c) HEDs were identified, but none had a direct or indirect adverse impact on plant safety or resulted in a cross cutting issue that adversely affected plant safety. The ISV testing was conducted between July 23 and September 6, 2018. The overall conclusion of the testing is that the NuScale control room design and staffing plan support safe operation of the NuScale plant. Performance measures with a low threshold were used to ensure low plant safety risk issues were identified and addressed prior to becoming potential nuclear safety issues. Individual performance measures were analyzed and categorized into HEDs. A total of (( }}2(a),(c) HEDs were identified, but none had a direct or indirect adverse impact on plant safety or resulted in a cross cutting issue that adversely affected plant safety. Shift staffing levels are appropriate and demonstrate that all tasks observed in a comprehensive sample of operational conditions can be performed. ((

        }}2(a),(c)

The HSI design is adequate in alerting, informing, and controlling evolutions as demonstrated by a comprehensive sample of operational conditions. The primary method for operators to monitor and perform actions for safe plant operation is through the use of safety functions. The safety functions are monitored automatically and operators confirm indications and perform actions based on safety function indications. The ability to confirm safety functions and perform actions was consistently rated by ISV crew members as very easy to perform. Personnel tasks that are performed for plant safety, personnel protection, or equipment protection were completed in all cases within 30 percent of the time allowed through analysis. This provides a large margin (over 70 percent) of success. Overall average workload was very low (( }}2(a),(c) and overall average situational awareness was very high at 92 percent. Occasionally, human performance errors were observed during ISV testing, but not one occurrence was observed of an incorrect component actually being manipulated. This © Copyright 2019 by NuScale Power, LLC 8

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 demonstrates that the HSI helps to prevents errors, and when they occur, it is easy to detect and recover. Important human actions were performed in all trials within 28 percent of the allowed time. Operating procedures are incorporated within the HSI, allowing operators to transition between procedures and controls without interruption. Improvement opportunities were identified and captured as HED-proposed resolutions. Additional overview information of ISV testing includes: ((

                                                                                      }}2(a),(c) 2.2      Acceptance Criteria

((

                                                         }}2(a),(c)

Important human actions were completed within 28 percent of the time allowed.

2.3 Diagnostic Measures

Overall workload (( }}2(a),(c) was reported to be low in every scenario. Reactor Operator 1 (RO1) was shown to consistently have the lowest reported workload.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                  }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 3.0 Simulator Overview The NuScale simulator is located in Corvallis, Oregon and was used for the NuScale integrated systems validation testing. The purpose-built facilitys design, models, and software are based upon the NuScale 12-unit small modular reactor plant design as described in the design certification application (DCA). The simulation facility is modeled using core neutronics, thermal-hydraulics, electrical simulation, and mechanical simulation software such that indications described within the design are available to plant operators. The fidelity of the parameters is such that operators are reliably and in real-time provided a best-estimate response that represents the expected actual plant response. The simulation facility is referred to as-designed because a reference plant does not exist so the ability to benchmark simulator plant performance against actual plant performance is not possible. The simulator must be able to perform to a standard that is high enough to provide credible results. Nuclear Power Plant Simulators for Use in Operator Training and Examination, ANSI/ANS-3.5-2017 (Reference 7.1.15) was used as a basis for the scope, fidelity, and functionality of the simulator, and the development of simulator performance testing. The 2017 version of the standard was used because it contains a new appendix that provides methodology and criteria for certifying a limited scope simulator. ((

                                                                                                       }}2(a),(c)

The simulator modeling and interface was acceptable to support the necessary tasks sampled per the operating conditions for ISV as follows:

Interface completeness - Interfaces supported all of the tasks associated with the broad range of operational conditions tested. Design verification testing performed prior to the start of ISV provided confidence of completeness. Improvements were identified in some of the interfaces, but in general, operators could easily navigate to the correct controls and indications.
Interface physical fidelity - The physical layout of the control room is the same as the plant design with the exception of ceiling height, movable furniture, and two large columns that are part of the space used for the simulator. The presentation of alarms, displays, controls, embedded procedures, layout, and spatial relationships are modeled per the design of the NuScale control room.
Interface functional fidelity - HSI functions, automation, and procedures were developed directly from task analysis to support plant operation. Procedures were developed to support expected responses as well as alternate paths not intended by the ISV test design team.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

Environmental fidelity - The simulator is representative of the main control design with regards to layout of screens, workstations, and controls. The environment reflects expected levels for noise, lighting, temperature, and humidity. Where necessary, approximations were provided based on best available information such as control room habitability ventilation noise level.
Data completeness fidelity - Information and data provided to personnel represented the plant system information necessary to support the sampling of operational conditions and demonstrate safe plant operation. Other systems are developed to a level of fidelity similar to existing commercial nuclear plant training simulators.
Data content fidelity - Twelve units are simultaneously and independently modeled.

Systems that interface with multiple units are integrated into the plant model such that impacts from one unit through common systems are accurately conveyed to other units. This underlying model information provides input to the HSI on workstation clients such that the information presented matches what would be expected during actual operation.

Data dynamics fidelity - Each reactor is a combination of high fidelity models working together to provide the most accurate results without sacrificing the ability to run in real-time. ((

                                            }}2(a),(c)

Test bed conformance - The NuScale simulator is certified to perform ISV testing.

((

                       }}2(a),(c) 3.1      Detailed Simulator Description The NuScale simulator is constructed of three core components: the hardware including the physical room, desks, chairs, monitors, etc.; the models ((
                                                        }}2(a),(c) and the digital HSI hosted by NuScale Simulator Interface Development Environment (NSIDE), which is described in the sections below.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

The simulator ceiling is lower than the plant design.
The simulator has drop flooring and easily reconfigurable desks so different evolutions of room layout could be evaluated.
There are two 2 ft by 2 ft support columns in the simulator that are not included in the control room design. These columns are located near the Shift Technical Advisor (STA) desk and conference table locations and can limit the Control Room Supervisor (CRS) and STA views of the safety display and indication (SDI) displays when viewing from their workstations.

The computers, monitors, and hardware switches supporting operator actions are carefully placed to provide the operators with a high degree of situational awareness and a functional work space. Figure 3-1 and Figure 3-2 show the simulator layout. Figure 3-1 Simulator Layout Each individual unit control station consists of a 55 in. overview screen that provides high level display information for that unit. Beneath the overview is a second 55 in. display that shows four individual 24 in. HSI displays that can be used to operate equipment on that unit. The common systems panel consists of an 80 in. overview screen providing common system information and critical trending parameters of all the units, and six individual 24 in. displays used to operate common system equipment. The five operator desk stations each contain four 24 in. screen displays. © Copyright 2019 by NuScale Power, LLC 13

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 The SDI displays consist of two 24 in. visual display units (VDUs) for each unit and two for common systems for a total of 26 VDUs with specific preselected important parameters displayed. Figure 3-2 NuScale Simulator 3.1.2 Simulator Models The simulator is equipped with a data historian allowing observers to capture precise timing of events or to validate an observation. Multiple cameras and a microphone system allow capture of all conversations and video coverage of actions within the control room with the exception of behind-the-individual unit stations, behind-the-SDI displays, and at the conference table. There is a computer that hosts the data historian and camera controls. All testing data is recorded on an external drive or memory device. The communication system utilized by operating crew participants to communicate with personnel outside the control room is standard office phone system equipment connected directly and only to the simulator booth. Twelve units are simultaneously and independently modeled. Systems that interface with multiple units are integrated into the plant model such that impacts from one unit are accurately conveyed to the other units. ((

                                                                                     }}2(a),(c) 3.1.3    NuScale Simulator Interface Development Environment NuScale Simulator Interface Development Environment is a NuScale-developed software package consisting of two parts: server and clients. The server receives data from the plant model discussed above and provides an interface for the simulator support personnel, hosts any processes running, provides notices and notifications, and outputs all relevant data to the clients. The clients are the collection of displays shown in

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 the simulator control room. The NSIDE hosts all of the HSI, displays data, provides access to procedures and automations, and equips the operator with a way to start automations and manipulate simulated plant components. The NSIDE was developed using commercial technologies ((

                                               }}2(a),(c) and undergoes rigorous testing, version control, and issue tracking.

((

                                               }}2(a),(c)

The NSIDE workstation clients are simple interfaces for relaying information to the operator in a method defined by the human factor design team. Charts, system-mimicked controls, numeric values, trends, colors, and other HSI products are all used to keep the operators informed and engaged. The clients features such as navigation, font sizes, and colors, are developed in accordance with the Human-System Interface Style Guide, (Reference 7.1.13), and tested to provide usefulness, ease of understanding, and allow for the different operator preferences. Procedures and automations are available to the operator through the clients HSI so hard-copy binders are generally not needed. Operating procedures are derived from the task analysis and the HSI design process in order to keep them aligned and accurate. 3.1.4 Simulated Plant Systems The systems listed in Table 3-1 are modeled by the simulator (i.e., simulated). A detailed description of each of these systems can be found in the NuScale DCA. The systems listed in Table 3-2 are systems listed in the DCA, but are not modeled for the reasons stated in the table. Table 3-1 List of Plant Systems Simulated System System Title Code BAS Boric Acid System BPSS Backup Power Supply System CAR Condenser Air Removal System CE Containment Evacuation System CFD Containment Flooding and Drain System CHW Chilled Water CPS Condensate Polisher Resin Regeneration System © Copyright 2019 by NuScale Power, LLC 15

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 System System Title Code CRDS Control Rod Drive System CRH Control Room Habitability System CRV Normal Control Room HVAC System CNT Containment System CVCS Chemical and Volume Control System CW Circulating Water System DHRS Decay Heat Removal System DW Demineralized Water System ECCS Emergency Core Cooling System EDNS Normal DC Power System EDS Highly Reliable DC Power System EHV 13.8 KV and Switchyard ELV Low Voltage AC Electrical Distribution System EMV Medium Voltage AC Electrical Distribution System FD Fire Detection System FP Fire Protection System FW Condensate and Feedwater System GRW Gaseous Radioactive Waste Management System HVD Heater Vents and Drain System IA Instrument and Control Air System ICI In-core Instrumentation System LRW Liquid Radioactive Waste Management System MCS Module Control System MEM Meteorological and Environmental Monitoring System MHS Module Heatup System MPS Module Protection System MS Main Steam System NDS Nitrogen Distribution System NMS Neutron Monitoring System PCS Plant Control System © Copyright 2019 by NuScale Power, LLC 16

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 System System Title Code PCU Pool Cleanup System PLD Pool Leakage Detection Systems PPS Plant Protection System PSC Pool Surge Control System PZR Pressurizer RBV Reactor Building HVAC System RCS Reactor Coolant System RCCW Reactor Component Cooling Water System RM Fixed Area Radiation Monitoring System RPC Reactor Pool Cooling System RPV Reactor Pressure Vessel RXC Reactor Core System RXF Reactor Fuel Assembly RXM Reactor Module RXS Reactor System RWBV Radioactive Waste Building HVAC SA Service Air System SCW Site Cooling Water SDI Safety Display and Indication SGS Steam Generator System SM Seismic Monitoring System SFPC Spent Fuel Pool Cooling System TBV Turbine Building HVAC System TG Turbine Generator UHS Ultimate Heat Sink © Copyright 2019 by NuScale Power, LLC 17

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Table 3-2 List of Plant Systems Not Simulated System System Title Reason Not Simulated Potential impact to ISV Code ABV Annex Building ABV system control design is not There is no impact to the ISV. HVAC developed at this stage of plant The ABV system is a minor design. Control of this system is ancillary support system, with no expected to be performed using a plant operational impact. This local control station, with no control system is not normally modeled room interface. in existing fleet simulators. AB Auxiliary Boiler AB system control design is not There is no impact to the ISV. System developed at this stage of plant Auxiliary steam heat sources are design. The simulator models the normally modeled in existing heat input of the auxiliary steam fleet simulators as an ON/OFF supply to the Module Heatup input which is provided from a system (MHS) as an ON/OFF node. local field controller, and is No control room interface to control controlled by communication to the auxiliary boilers or the auxiliary the simulator booth operator. steam flowpath are modeled. This method was successfully employed during ISV performance. BPD BOP Drains The BPD system consists of There is no impact to the ISV. System building sumps, sump pumps, and The BPD system is a minor connected piping in the plant ancillary support system, with no building outside of the RCA. The plant operational impact. This details of the BPD system controls system is not normally modeled are not developed at this stage of in existing fleet simulators. plant design. It is anticipated that this system will be controlled using local control stations, with no control room interface. COM Communication The simulator uses a commercial There is no impact to the ISV. Systems Private Branch Exchange (PBX) ISV exercised the phone system to mimic expected communications of the control communication tasking. The actual room staff to external work communication system will be groups. Formal development of selected by a combined operating the communication equipment license applicant. was not required in order to evaluate this activity. CP Cathodic The CP system protects the CW There is no impact to the ISV. Protection system and condenser waterboxes. The CP system is a minor System The details of the CP system ancillary support system, with no controls are not developed at this short term plant operational stage of plant design. It is impact. This system is not anticipated that this system will be normally modeled in existing controlled using local control fleet simulators. stations, with no control room interface. © Copyright 2019 by NuScale Power, LLC 18

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 System System Title Reason Not Simulated Potential impact to ISV Code CRA Control Rod The Control Rod Assembly is a There is no impact to the ISV. Assembly physical component which is The movement of control rods in withdrawn and inserted into the and out of the core was core during normal operation. The simulated throughout ISV, and Control Rod Drive (CRD) system functioned as designed. The controls movement of the control Control Rod Assembly will have rod in and out of the core. an operator interface for latching Operation of the CRD system to and unlatching the control rod perform the withdrawal and from the drive shaft which has insertion are modeled, and has a not been designed yet. Use of control room interface. this feature was not exercised in any ISV scenarios. DGBV Diesel Generator DGBV system control design is not There is no impact to the ISV. Building HVAC developed at this stage of plant Operation of the DGBV system design. Control of this system is was not exercised in any ISV expected to be performed using a scenarios, so there was no local control station, so there will be impact to the ISV. no control room interface. GLP Grounding and The GLP system is a passive There is no impact to the ISV. Lightning system. The details of the GLP The GLP system is a minor Protection system controls are not developed ancillary support system, with no System at this stage of plant design. It is plant operational impact. This anticipated that this system will be system is not normally modeled controlled using local control in existing fleet simulators. stations, with no control room interface. FHE Fuel Handling The FHE system consists of the There is no impact to the ISV. Equipment Fuel Handling Machine Performance of various refueling (Bridge/Crane), the New Fuel activities was included in ISV, Elevator, and New Fuel Jib Crane. and assumed to be conducted The details of these FHE by dedicated fuel handling component controls are developed, personnel operating locally in but not finalized at this stage of the field. The movement of fuel plant design. It is anticipated that assemblies in and out of the these components will be controlled core or the spent fuel racks was using local control stations, with no not exercised in any ISV control room interface. scenario. ISV was successfully performed using this methodology. FWT Feedwater The FWT system is used to provide There is no impact to the ISV. Treatment chemical treatment of the The FWT system is a minor condensate and feedwater of each ancillary support system, with no unit. The details of the FWT system plant operational impact. This controls are not developed at this system is not normally modeled stage of plant design. It is in existing fleet simulators. anticipated that this system will be controlled using local control stations, with no control room interface. © Copyright 2019 by NuScale Power, LLC 19

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 System System Title Reason Not Simulated Potential impact to ISV Code HPN Health Physics The HPN network is used by the There is no impact to the ISV. Network System HP department to control access to The HPN system is a minor areas within the RCA. The system ancillary support system, with no is designated but currently plant operational impact. This undefined. It is anticipated that this system is not normally modeled system will be controlled using local in existing fleet simulators. control stations, with no control room interface. MAE Module Assembly MAE consists of the containment There is no impact to the ISV. Equipment and reactor vessel support stands used during module disassembly and re-assembly during refueling operation. These components do not have any operator or control room interface. MAEB Module Assembly MAEB consists of the remote bolt There is no impact to the ISV. Equipment - fastening and unfastening Bolting equipment used in the containment and reactor vessel support stands during module disassembly and re-assembly during refueling operation. This equipment has no control room interface. NFS New Fuel The NFS system consists of the There is no impact to the ISV. Storage System new fuel storage racks, which are a physical component with no controls. There is no control room interface with this component. NSA Neutron Source The Neutron Source Assembly is a There is no impact to the ISV. Assembly physical component within the core Full core modeling is included in that is used during reactor startups the ISV simulator. when no source neutrons are available. There is no unique control room interface for NSA, beyond typical core instrumentation. PL Plant Lighting The PL system is a passive system. There is no impact to the ISV. System The details of the PL system The PL system is modeled to a controls are not developed at this limited degree with the lighting in stage of plant design. It is the simulator. anticipated that this system will be controlled using local control stations. © Copyright 2019 by NuScale Power, LLC 20

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 System System Title Reason Not Simulated Potential impact to ISV Code PSS Process The PSS is used by chemistry There is no impact to the ISV. Sampling System department personnel to sample The PSS system is a minor plant systems. This system will be ancillary support system, with no controlled using local control plant operational impact. The stations, with no control room system was modeled on a interface. limited basis with simulator operator controlled manual annunciators - this is similar to the modeling in current fleet simulators. PVM Plant-wide Video The PVM system is designated but There is no significant impact to Monitoring currently undefined. The use of the ISV. SME judgement System realistic input of real-time plant indicates a potential slight video feed for a simulated control increase in workload with the room is beyond the scope of use of the PVM system, but the existing fleet simulators, and not information provided is modeled in the ISV simulator. anticipated to enhance overall performance during ISV scenarios. PW Potable Water PW system control is not designed There is no impact to the ISV. System at this stage of plant development. The PW system is a minor This typical plant system is ancillary support system, with no normally controlled using local plant operational impact. This control stations, and no control system is not normally modeled room interface is provided. in existing fleet simulators. RBSS Reactor Building The RBSS is a locally controlled There is no impact to the ISV. Spray System system which aligns a temporary pump to a sprinkler header. There is no control room interface for this system. RVI Reactor Vessel RVI consists of the physical There is no impact to the ISV. Internals components inside the reactor vessel. Full modeling of the reactor vessel, the core, the control rods and the reactor coolant system is modeled in the ISV simulator. There is no stand-alone, unique HSI control room interface for the reactor vessel internals. RWD Radioactive The RWD system is operated from There is no impact to the ISV. Waste Drain the Rad Waste control room. There System is no control room interface. RWM Radioactive The RWM system is operated from There is no impact to the ISV. Waste the Rad Waste control room. There Management is no control room interface. System © Copyright 2019 by NuScale Power, LLC 21

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 System System Title Reason Not Simulated Potential impact to ISV Code SBV Security Building SBV system control design is not There is no impact to the ISV. HVAC developed at this stage of plant The SBV system is a minor design. Control of this system is ancillary support system, with no expected to be performed using a plant operational impact. This local control station, with no control system is not normally modeled room interface. in existing fleet simulators. SDS Site Drainage The SDS consists of storm drains, There is no impact to the ISV. System required lift station, and connected The SDS is a minor ancillary piping outside of site buildings. The support system, with no plant details of the SDS controls are not operational impact. This system developed at this stage of plant is not normally modeled in design. It is anticipated that this existing fleet simulators. system will be controlled using local control stations, with no control room interface. SEC Plant Security Details of the SEC system are not There is no impact to the ISV. System available to be modeled in the This system is not normally simulator. No control room interface modeled in existing fleet is provided or planned. simulators. SFS Spent Fuel The SFS system is comprised of There is no impact to the ISV. Storage System physical fuel storage racks, with no control room interface. The Spent Fuel Pool Cooling system, Reactor Pool Cooling System and Ultimate Heat Sink are modeled in the ISV simulator. SRW Solid Radioactive The SRW system is operated from There is no impact to the ISV. Waste the Rad Waste control room. There Management is no control room interface. System SP Security Power Details of the SP system are not There is no impact to the ISV. System available to be modeled in the Operation and control of the SP simulator. No control room interface System is performed by security is provided or planned. personnel. This system is not normally modeled in existing fleet simulators. TLOS Turbine Lube Oil Operation of the TLOS system is There is no impact to the ISV. Storage System not required to support routine The TLOS system is a minor power operation. The system is ancillary support system, which used on an as needed basis to has no short term plant makeup to a turbine generator operational impact. This system sump. TLOS system control design is not normally modeled in is not developed at this stage of existing fleet simulators. plant design. Control of this system is expected to be performed using a local control station, with no control room interface. © Copyright 2019 by NuScale Power, LLC 22

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 System System Title Reason Not Simulated Potential impact to ISV Code UW Utility Water The UW system provides raw, There is no impact to the ISV. System untreated water for the CW system Operation of this system is not and SCW system. It also provides required to support routine clarified water for the fire water operations during ISV scenarios tanks and the PW system water because the supported systems treatment facility. UW system have sufficient inventory to control design is not developed at complete the scope of ISV this stage of plant design. No without requiring any makeup. control room interface has been provided for the ISV simulator. 3.2 Simulator Certification Testing The purpose of ISV simulator certification testing is to ensure simulator performance is sufficiently complete and accurate to perform a representative validation test. Certification testing referenced guidance in NUREG-0711 as it pertains to simulators used during the ISV activities described in Section 11, Human Factors Verification and Validation. The ISV simulator performance testing is more comprehensive than SBT that was used to validate the individual ISV scenarios. SBT is described in Section 5.1.4. The following criteria were used to evaluate simulator performance:

Real time and repeatability - the simulator operates in real time and the same events be repeated with consistent responses
Limits of simulation - the simulator maintains parameters within the limits described within the applicable engineering documents describing plant limits
Normal evolutions - the simulator models normal evolutions such as start-ups and shutdowns
Malfunction testing - the simulator models a large array of malfunctions as described in the Final Safety Analysis Report (FSAR)
Steady state - the simulator maintains all units in a steady state condition for a period of time without deviations

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Performance testing was competed prior to the start of ISV testing. All performance criteria were met. One discrepancy was noted and entered into the HFEITS: HFEITS# HED Description Priority ((

                                                                                                                }}2(a),(c)

((

                           }}2(a),(c) The deficiency was determined to have no impact on the ISV testing schedule.

The NuScale simulator was certified to perform ISV testing upon satisfactory review and approval of the simulator certification testing results. 3.3 Simulator Deficiencies Simulator deficiencies that were known and not corrected were identified prior to the start of ISV testing. This list, shown in Table 3-3, List of simulator deficiencies for ISV, was generated based on differences between the design and the behavior of the simulator. The simulator development group tracks deficiencies using TestTrack, a relational database and work tracking program. Items that remain within the simulator development work scope associated with detailed design but would not impact the represented task performance in the control room are not listed. ((

                                                                                            }}2(a),(c) Each issue is described below and the disposition section explains why it is not considered detrimental to conducting a valid ISV test. The TestTrack identification number is included with the item title.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Table 3-3 List of Simulator Deficiencies for Integrated System Validation

  #    Item                     Comments                                            Disposition

((

                                                                                                                                                       }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 25

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

  #    Item                     Comments        Disposition

((

                                                                                                               }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 26

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

  #    Item                     Comments        Disposition

((

                                                                                                              }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 27

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 During the conduct of ISV, an additional item was identified as a simulator deficiency as it became apparent through the first few weeks of testing. ((

                                                                           }}2(a),(c) 3.4      Simulator Issues During Integrated System Validation The following simulator performance issues (Table 3-4) were observed during the conduct of ISV testing. None of the issues resulted in the need for a retest or invalidated collected test data.

Table 3-4 Simulator Performance Issues

  #    Comments                                           Disposition

((

                                                                                                               }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 28

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 4.0 Design Verification This section describes the process and results of performing design verification on the NuScale HFE design in accordance with the Section 3 of the V&V IP. The design verification verifies the HFE design conforms to HFE design principles and enables plant personnel to successfully and reliably perform their tasks to assure plant safety and operational goals. This is done by executing the three main verification processes: HSI Inventory and Characterization, HFE Design Verification, and HSI Task Support Verification. Traditional HFE V&V activities verify that the HSI design adheres to a set of design requirements (i.e., HSI Style Guide) and supports operator tasks through a series of mostly static verification tests. To achieve a compressed design, training, and testing schedule, the NuScale HFE design team developed a state-of-the art approach to the design verification processes that included: ((

                         }}2(a),(c)

As a result, design verification activities were efficient and confirmed a consistent HSI product that followed the design requirements documented in the HSI Style Guide (Reference 7.1.13) for all HSI displays defined by the sampling of operational conditions. The overall conclusion of design verification testing is that the current main control room design and HSIs supported the operating personnel's primary task of monitoring and controlling the plant. Usability of the HSI did not impose an excessive workload (e.g., window manipulation, display selection, and navigation). The HSI also supported the recognition, tolerance, and recovery from human errors in support of the safe operation of the NuScale plant. Details for each activity performed under HFE design verification is provided below with a summary of the results. 4.1 Human-System Interface Inventory and Characterization Characterization defines the functionality of each HSI discussed in design documents such as equipment lists, design specifications, piping and instrumentation diagram (P&ID) and input/output lists that are produced during HSI design. Inventory and characterization verification was performed on all display pages within the SOC as described Section 3.1.2 of the V&V IP (Reference 7.1.12) using an inventory and characterization form, the results of HFE design verification, applicable P&IDs, and © Copyright 2019 by NuScale Power, LLC 29

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 the HSI Style Guide (Reference 7.1.13). The form captured aspects of the HSI used to manage the interface; such as display navigation and plant controls. 4.1.1 Inventory and Characterization Results ((

                                                              }}2(a),(c)

This includes common aspects of each display page such as navigation controls and retrieving displays. 4.2 Human Factors Engineering Design Verification HFE Design Verification addresses the suitability of the HSI with regard to human capabilities and limitations. The design-specific HSI Style Guide developed by NuScale was used for verifying the NuScale HFE design. Design verification confirms the visual aspects (alarms, controls, indications, embedded procedures and the means of navigation between elements) of the HSI, including conformance to the NuScale HSI Style Guide. During HFE design verification, the HFE design team evaluated three elements of the NuScale HFE/HSI design:

1. HSI display page icon consistency

2. HSI display page theme

3. Overall main control room design The Human-System Interface Style Guide requirements were used as the acceptance criteria during testing and the element was deemed acceptable if compliance was complete (i.e., only if every instance of the item is fully consistent with the criteria established by the HSI guidelines and requirements). If there was any noncompliance, full or partial, the discrepant condition was described in an HED.

In addition to verifying the required information listed in Section 3.1.2 of the V&V IP (Reference 7.1.12), the HFE design team performed verifications on elements of the page not covered under icon and/or page theme testing (e.g., static labels, roll-over labels, component and text spacing, workbench label accuracy, navigation links, piping size, colors, font sizes) as well as performing dynamic testing as follows: ((

                                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                        }}2(a),(c)

These additional HFE design verifications were performed during characterization to provide additional validation of the HSI design and a more comprehensive test form of each display page. 4.2.1 Human-System Interface Display Page Icon Consistency ((

                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 4.2.2 Human-System Interface Display Page Icon Consistency Results ((

                                           }}2(a),(c) 4.2.3    Human-System Interface Display Page Theme The NuScale HSI has a general display theme that frames an area in which the plant operators work in (i.e., Display Page Area) as shown in Figure 4-1. The theme was designed to provide the common elements required by the HSI Style Guide (e.g.,

heartbeat and page title) while only requiring a single verification. This approach eliminated the redundant verification of each element during HSI task support verification of each individual display page. Specialty page display theme exceptions designated by the HFE design team such as overview displays, the procedure interface display, and limited other pages were noted on the test forms used to verify those particular pages. © Copyright 2019 by NuScale Power, LLC 32

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Figure 4-1 NuScale Human-System Interface Display Page Theme 4.2.4 Human-System Interface Display Page Theme Results Seventeen elements were verified and no discrepancies were identified. The majority of the 17 elements where utilized during other verification activities providing them hundreds of additional dynamic testing opportunities throughout design verification. A summary of the 17 verified items is as follows:

1. Heartbeat icon - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior was consistent everywhere the icon was used. The list below is the locations where the heartbeat icon is presented.

a. Upper Navigation Bar

b. Unit GVD

c. Plant Overview GVD

d. SDI pages

e. Safety Function Monitoring page

2. Last page button icon - Verified icon was compliant to the HSI Style Guide and dynamically tested to verify that it allowed the operator to step back through a history of viewed pages.

3. Next page button icon - Verified icon was compliant to the HSI Style Guide and dynamically tested to verify that it allowed the operator to step forward through a history of viewed pages.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

4. The plant area dropdown selection box - Verified icon was compliant to the HSI Style Guide and that the selection box provided the options to select Units 01-12, Common plant 00, 0A, 0B and selections applicable to the entire plant.

Note: While performing the Quick Key verification, it was observed that the Plant Area did not update correctly when plant area pages were selected. This issue was documented in HFEITS item H-03428. It was entered as an Upper Navigation Bar issue because the quick key did provide the correct page, but the plant area portion of the upper navigation bar did not change as expected. This item was corrected, retested, and the HFEITS item was closed.

5. Breadcrumb drop-down selection boxes - Verified icon was compliant to the HSI Style Guide and that the selection box provided the options to select all of the HSI displays developed within the scope defined by the sampling of operational conditions.

Note: While performing the Breadcrumb navigation verification, inconsistent dithering behavior was recognized and reported in HFEITS item H-03450. This item was corrected, retested, and the HFEITS item was closed.

6. Alarm icon - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (status indication) was correct based on the location where the icon was used. The list below is the locations where the Alarm icon is presented.

a. Upper Navigation Bar

b. Unit GVD

c. Plant Overview GVD

d. 12 Module Overview

7. Caution icon - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (status indication) was correct based on the location where the icon was used. The list below is the locations where the Caution icon is presented.

a. Upper Navigation Bar

b. Unit GVD

c. Plant Overview GVD

d. 12 Module Overview

8. Notice icon - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (status indication) was correct based on the location where the icon was used. The list below is the locations where the Notice icon is presented.

a. Upper Navigation Bar

b. 12 Module Overview

9. Tile View quick key - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (provided the Tile Navigation Page to the operator) was correct.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

10. Module Overview quick key - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (provided the Module Overview Page to the operator) was correct.

11. 12-Unit Overview quick key - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (provided the 12-Unit Overview Page to the operator) was correct.

12. Plant Notification quick key - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (provided the Plant Notification Page to the operator) was correct.

13. Plant Event Log quick key - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (provided the Plant Event Log Page to the operator) was correct.

14. Process Library quick key - Verified icon was compliant to the HSI Style Guide and that the dynamic behavior (provided the Process Library Page to the operator) was correct.

15. The server connection information area - Verified text was compliant to the HSI Style Guide and was verified that the correct server information was presented.

16. The Current date and time - Verified text was compliant to the HSI Style Guide and date and time was verified.

17. The Active/Passive icon - Verified icon(s) were compliant to the HSI Style Guide and dynamically tested each icon to ensure that each possible entry was displayed on the lower navigation bar and the Unit and Plant Overview GVD.

4.2.5 Overall Main Control Room Design Results The simulator HFE design verification testing evaluated many HSI Style Guide requirements. Upon completion of the verification, all requirements were found to be acceptable. A summary of the results is shown below:

1. The lighting did not produce glare on any of the (( }}2(a),(c) VDUs and allowed for safe working conditions in the simulator.

2. No pinch points, sharp edges, knee obstructions, or tripping hazards were found.

3. Hardwired switches, phones, keyboards and mouse interfaces were available and easily accessible.

4. Noise levels in the room did not interfere with normal communications.

5. Accessibility and convenience to bound documents was verified.

6. Accessibility to adequate supplies (e.g., printer, paper, and pens) was readily available.

The design verification activities resulted in (( }}2(a),(c) HEDs. ((

                                                }}2(a),(c) No HEDs were defined as Priority 1. Many of the HEDs involved font size violations, mislabeling or other minor consistency concerns.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 HEDs that involved incorrect component links or affected display page navigation where promptly fixed and tested. Many of the HEDs were able to be closed. The remaining (( }}2(a),(c) HEDs were of low consequence and selected to be resolved following ISV performance. A listing of all design verification HEDs are shown in Section 6. 4.3 Human-System Interface Task Support Verification Task support verification verified that the HSI provided the alarms, controls, and indications for personnel to perform tasks described by the task analysis within the scope of the sample of operational conditions. Scenario-based testing (Section 5.1.4) was performed on each ISV scenario and included observing plant performance and operator response to ensure the plant behavior is consistent with design assumptions. SBT packages were assembled for each of the (( }}2(a),(c) ISV scenarios and contain a list all notifications (alarms and cautions) received during the scenario test as well as plant event logs that verified the appropriate controls and indications were provide to the operators such that the scenarios can be successfully completed. ((

                                                                                      }}2(a),(c)

For HSI task support verification related to performance (e.g., accuracy and dynamic response), the full scope simulator was used to perform specific tasks selected through application of the SOC. Task support verification was performed on two of the HSI elements created to aid the operators in performing their task:

1. Embedded Procedures

2. Automations These elements were dynamically tested to verify that the information and aids developed to allow operators to perform their primary task of monitoring and controlling the plant met criteria found in the HSI Style Guide and could be achieved without increasing workload associated with using the HSI.

Task support verification concluded the selected embedded procedures and automations used in conjunction with other HSI were successful in delivering the controls, display pages, and needed information to the operators to perform those tasks. The SBT packages were assembled for each of the (( }}2(a),(c) ISV scenarios. Each package contains a list all alarms and cautions received during the scenario test. The actuated alarm lists were verified as appropriate for each scenario ((

                      }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 4.3.1 Embedded Procedures A sampling of (( }}2(a),(c) procedures was selected from the catalog of approximately 2000 procedures developed for ISV. Each sampled procedure evaluation verified embedded components were correct, and that the procedure interface was able to be opened, place-kept, and completed. ((

                                          }}2(a),(c) The verifier observed operators during training using the process library interface and selected procedure(s). Usability testing was performed by answering a set of predetermined questions listed on the testing form.

The procedure usability testing was used to understand points of user frustration and measures the usability, or ease of use, of a specific object or set of objects. During a usability test, the evaluation

determined if participants were able to complete specified tasks successfully based on the procedure or automation being completed and closed.
observed participants reactions with the HSI design product by monitoring facial expressions, eye squinting, or verbal frustration.
identified changes required to improve user performance and satisfaction.

4.3.2 Automations Automation display pages are considered HSI specialty pages designed to perform specific tasks that would have otherwise been performed manually by operators following a procedure. Automation displays are designed separately from system display pages. For this reason, the design verification was performed on automation displays by using a combination of the inventory and characterization form and the usability questionnaire used for the embedded procedures. The verifier observed operators during training using the interface and selected automation(s). Usability testing was performed by answering a set of predetermined questions listed on the testing form. 4.3.3 Human-System Interface Task Support Results ((

                                                                                                               }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                       }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.0 Integrated System Validation Test Implementation Integrated system validation testing was performed from July 23 through September 6, 2018. The ISV test schedule as implemented is contained in Appendix A, ISV Testing Schedule. The ISV testing consisted of (( }}2(a),(c) scenarios containing a variety of normal and abnormal events. The sequence of events was developed such that they were not formulaic (e.g., events did not always proceed from easy to hard). (( }}2(a),(c) independent ISV participant crews of five were tested using the (( }}2(a),(c) scenarios. Each scenario was performed by (( }}2(a),(c) crews. Each crew was rotated to equalize the testing among all participants. Appendix B, Test Performance Matrix, shows which crews performed which scenarios. All probabilistic risk assessment (PRA) credited operator actions were tested in more than one scenario and were performed by all (( }}2(a),(c) crews. Section 5.8.1.1 describes how the rotation of crews ensured each crew demonstrated each PRA credited action. The following staff and qualifications are assumed to be available as part of the on-shift operating crew:

Five licensed operators in the main control room consisting of the following:

one control room supervisor maintaining an active senior reactor operator license one shift technical advisor maintaining an active senior reactor operator license and having a degree in a science or applied science field three reactor operators maintaining active reactor operator licenses The personnel used for ISV testing did not have active operating licenses but were trained to perform the duties and tasks expected of licensed operators in an actual plant. ((

                                               }}2(a),(c)

Each scenario was evaluated by a (( }}2(a),(c) observation team. There were no instances where test results were inconclusive ((

                 }}2(a),(c) Human performance variances existed between trials, but did not result in inconsistent outcome of task performance.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Data was collected (( }}2(a),(c) through observations and critiques. Some minor test issues were documented throughout the ISV testing and are listed in Section 0. They did not invalidate any of the collected test data or skew test results. All collected data was valid. 5.1 Scenario Development The V&V IP, Section 2.0, outlines the process used for scenario development. This section provides a description of the scenarios used in the ISV and how they met the V&V objectives. 5.1.1 Scenario Construction The purpose of the ISV scenarios is to validate HSI design, plant design, staffing levels, procedures, automation, and training results in control room personnel safely operating all 12 units of the NuScale plant while achieving anticipated operational goals by successfully performing selected tasks. (( }}2(a),(c) scenarios were developed to meet the stated ISV test goals and to have contained an event that tests each NUREG-0711 sample condition. Table 5-1 illustrates how the (( }}2(a),(c) ISV scenarios meet or exceed the ISV test plan development goals. Table 5-1 Test Plan to Scenario Cross Reference ((

                                                                                                                    }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 40

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Table 5-2 illustrates how each of the twelve ISV test scenarios contain an event that correlates to operating conditions listed in NUREG-0711. Table 5-2 Sample of Operating Conditions to Scenario Cross Reference ((

                                                                                                        }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 41

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.1.2 Scenario Descriptions Each scenario is a collection of discrete events that tests various operator tasks. The tasks are selected to meet the NUREG-0711 sample criteria. During the conduct of ISV, the scenarios were implemented through detailed scenario guides. These guides not only describe the scenario but also provide the following information used by the test lead and simulator operators:

pass/fail acceptance criteria
simulator initial conditions
turnover information to the ISV crew members
a list of cues that are provided during the scenario
expected plant response
specific individual event evaluation criteria
event continuation criteria In some instances, events occur concurrently and in others events are in sequence. The event continuation criteria lists what conditions must occur for the next event to proceed.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                  }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 43

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                             }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                               }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                               }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                              }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                              }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                    }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 51

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                               }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.1.3 Scenario Assumptions The following major assumptions were incorporated into the ISV testing:

ISV crew members performed the roles of control room supervisor, shift technical advisor, and three reactor operators. The description of the roles and responsibilities of these positions is described in the NuScale Concept of Operations (Reference 7.1.2).

        *    ((              }}2(a),(c) nonlicensed operators are assigned to an operating shift.

Nonlicensed operators and plant support staff tasks were not evaluated. The interaction of nonlicensed operators with the control room crew was simulated by test staff that role played pre-scripted responses. Nonlicensed operator assignments were tracked with best estimate times used to complete field activities and report back results to the control room.

Fire brigade duties are not performed by the operations assigned nonlicensed personnel. ISV crew members were trained to dispatch one nonlicensed operator to act as a communicator between fire response and the main control room. Some existing nuclear facilities use nonlicensed operators as fire brigade members while others do not. This assumption is consistent with current industry practice.
Refueling operations are not directed from the control room. Refueling is a planned activity and has a dedicated staff assigned for specific performance and oversight.

Because the NuScale module is electrically and mechanically disconnected during refueling, the control room operators have no direct interaction with the refueling team other than operating common system components (e.g., align reactor pool cooling) or to update the Shift Manager on refueling status.

Most day-to-day work interactions and authorization of work occurring between the maintenance crafts and operations department take place in a work control center.

The work control center is assumed to be outside the control room but is staffed by a member of the operations department that has the authority to authorize plant work. This reduces the distractions to the control room crew and is common practice among existing nuclear plants. Testing staff role played as the work control center when appropriate and did not lead the crew in decision making or provide additional task support for tasks that are completed in the main control room. © Copyright 2019 by NuScale Power, LLC 54

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.1.4 Scenario-Based Testing This testing is done to confirm the simulator performs acceptably prior to administering a performance-based simulator scenario test (i.e., ISV testing). Scenario-based testing is an integrated test methodology used to collect and analyze data to ensure the simulator adequately models the NuScale power plant design. Individual SBT packages were assembled for each scenario following many iterative rounds of testing. Each scenario has been pilot tested in real time with staff crews that had no knowledge of the scenario. Feedback from pilot testing was incorporated prior to performing the final documented SBT. The SBT was performed using the final version of procedures, the final version of the scenario, and the simulator configuration that was used to conduct ISV testing. Scenario-based testing was performed on each ISV scenario and included observing plant performance and operator response to ensure the plant behavior is consistent with design assumptions. For specific scenario events, additional observations by engineering, safety analysis, risk assessment, and I&C were performed as needed to validate correct simulator response. The assembled SBT test results package includes the following sections: ((

                                                                  }}2(a),(c)

NuScale Document Control Records Management maintains the SBT test results package. The results of the scenario-based testing determined that each scenario was acceptable for ISV testing based on passing the following criteria:

The simulator was able to perform all actions required to demonstrate task performance as described in the scenario guide.
The simulator displayed the correct initial conditions as described in the scenario guide and conformed to plant design with respect to reactor status, plant configuration, and system operation.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

Plant maneuvers and automation sequences were performed in real time.

Administrative tasks and communications were performed in real time during pilot testing, but condensed during SBT data collection when they did not result in an observable change to a key parameter (e.g., no time allotted for prejob briefs or discussion that would be expected during ISV test performance).

The simulator demonstrated expected plant responses to operator input and to normal, transient, and accident conditions.
The simulator permitted use of the plant procedures so that the scenario was completed without procedural exceptions, simulator performance exceptions, or deviation from the scenario sequence.
The simulator did not fail to actuate an expected alarm or automatic action and did not cause an unexpected alarm or automatic action.
Observable change in simulated parameters corresponded in trend and direction to those expected from best estimate response of the reference plant.
Plant design limitations were not exceeded.
The simulator modeled the expected plant response for each scenario malfunction.
The SBT conducted in a manner sufficient to ensure that simulator fidelity has been demonstrated for this scenario.
Modeling and hardware discrepancies identified during the conduct of SBT are documented.

The SBT testing was performed after pilot tests and ISV crew training was completed. During pilot testing and ISV crew training, many updates to procedures and refinements to automations and HSI displays were identified and corrected. There were no discrepancies identified during the performance of SBT that required mitigation prior to the start of ISV testing. 5.2 Operating Crew Overview The individual operating crews participating in the ISV were selected to approximate the expected crew characteristics that would operate a NuScale facility. The expected crew would have individuals with previous commercial nuclear operating experience, previous Navy nuclear power operating experience, and direct input operators that have a technical or engineering degree but no nuclear operating experience. Each crew was comprised of five crew members. The crew positions used during ISV were:

Control Room Supervisor
Shift Technical Advisor
Reactor Operator 1
Reactor Operator 2

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

Reactor Operator 3 The operating crew participants were not involved in any aspect of HFE design review elements as described by NUREG-0711. All selected crew participants performed no activities associated with ISV test development or ISV pilot testing. ((

                              }}2(a),(c)

As determined by supervisory and observer observations, the crew members demonstrated that they were appropriately critical of the interface during all methods of feedback. None of the crew participants demonstrated a bias that would impact the ISV either positively or negatively. ((

                                                     }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.2.1 Operating Crew Participant Training Overview Operating crew participant training was required to ensure crew members had sufficient knowledge of the NuScale plant design, plant controls, and conduct of operations to interact with the HFE design in the same manner as experienced plant personnel. The training program establishes expected operator responses and ensures that observations and diagnostic measures are assessing actual performance or interface deficiencies and not inexperienced operator performance. The operating crew participant training consisted of the following:

        *    ((          }}2(a),(c) hours of classroom training. This training was separated into the following modules:

Introductory Courses NuScale Power Module Primary Systems Balance of Plant Systems Instrumentation & Control and Electrical Systems Miscellaneous Systems Admin Procedures Integrated Operation Abnormal and Emergency Operation (( }}2(a),(c) quizzes were administered during the classroom training to assess individual comprehension.

        *    ((        }}2(a),(c) hours of simulator familiarization. This training was performed with the participants assigned to a specific crew. The simulator training lessons were both task and scenario based. The training included most of the tasks to be performed in ISV but without training to the actual ISV scenarios. For example, tagging of components was practiced but the actual components being tagged in ISV were not specifically performed. This was done so as to not train to the test.

(( }}2(a),(c) simulator scenario tests were performed to assess crew performance and to practice collecting data, assessing ISV test performance measures, and analyzing results. (( }}2(a),(c) simulator audit was performed exactly as ISV test performance was designed. The audit was performed as a final assessment of crew performance, including providing meaningful and critical feedback. Operations and HFE observers were assessed to ensure the observation team was familiar with their role in providing meaningful and critical observations. © Copyright 2019 by NuScale Power, LLC 58

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.2.1.1 Simulator Assessments (( }}2(a),(c) assessments were administered during the simulator portion of the training. Each assessment consisted of a dynamic simulator exercise. Each assessment provided an opportunity to assess ISV crew member performance and also to have observation team members practice observation methods. Crew performance assessments were focused on improving crew human performance. The goal of the simulator assessments was to verify crew members were retaining technical knowledge and also exhibiting expected behaviors as described in the NuScale Concept of Operations (Reference 7.1.2). Observation team members were assigned to observe specific ISV crew members during the evaluation. The ISV crew members performed a debrief of their experience with a member of the observation team in order to train ISV crew members on providing specific feedback and to avoid generalizations and feelings. The ISV crew members completed situational awareness questions and workload measures exactly as administered during actual ISV testing. ((

                                                                                   }}2(a),(c)

All individual ISV crew members were graded on the following categories of performance:

Interpretation and diagnosis of events
Procedure use and adherence
Identification of technical specifications
Locating and manipulating controls on the HSI
Performing peer checks when required
Providing clear and concise communications
Using three-way communication, crew updates, and crew briefs Crew members selected to act as a control room supervisor or a shift technical advisor were additionally graded on the following categories of performance:

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

Providing oversight of evolutions in progress
Providing direction and soliciting crew feedback The crew as a whole was graded on the following categories of performance:
Crew communication - how effective was team communication?
Unit ownership - how do the crew members use active/passive control?
Task performance as a group - how well did the team complete tasks?
Crew debrief effectiveness - was the crew feedback critical, specific, and meaningful?

5.2.1.2 Training Audit Each ISV crew performed an audit that was intended to be a final check and practice for both the ISV crew members and the ISV observation team to ensure readiness prior to the start of ISV testing. The audit schedule replicated the ISV testing schedule. No modification was identified during the audit that resulted in a change to the anticipated ISV test schedule. The simulator audit was performed by a different crew each day from June 18 through June 20, 2018. Six simulator audit objectives were identified prior to performance of the audit. Each objective was considered met at the end of the evaluation, although a list of additional training items was generated to raise crew proficiency in specific areas. The objectives were:

1. ISV crew proficiency - completed training is sufficient for ISV performance (e.g.,

knowledge of plant systems, HSI, conduct of operations)

2. ISV crew proficiency - crew members are proficient at providing critical feedback

3. Staff proficiency - observation team is proficient at performing critical observations

4. Staff proficiency - successfully administer an ISV-style exam

5. Staff proficiency - obtain, record, and analyze test data

6. Simulator proficiency - simulator quality/fidelity adequately supports an ISV-style exam 5.2.1.3 Training Records An ISV participant training curriculum was created within the NuScale Learning Management System. Each ISV crew member was assigned the curriculum, which included the classroom and simulator training as described in Section 5.2.1. Attendance sheets were collected for each training session and used to ensure full attendance.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 NuScale proficiency verification was used to document completion of all requirements for ISV participation certification. The proficiency requirements included: ((

                                                                       }}2(a),(c)

The NuScale training department maintains:

lesson plans and associated training material used during ISV classroom training
lesson plans used for ISV simulator training
each ISV crew members training attendance record
each ISV crew members proficiency verification documentation 5.2.2 Integrated System Validation Scenario Security Overview The ISV test plan provides requirements for storing and transporting ISV test material prior to and during the performance of ISV. This ensures the ISV testing is not compromised by the ISV crew members having knowledge of the scenarios prior to ISV testing. The following actions were taken to prevent compromising exam security:
Access to ISV scenario information was strictly limited to personnel on a need-to-know basis including observers, test developers, and selected Licensing and management personnel.
ISV exam security training was conducted with all observers and test developers.
Electronic files associated with ISV testing and training assessments were stored on the NuScale corporate shared drive but with strict file access controlled by the IT department. Only authorized personnel could access these files and only if they had logged onto a corporate password-protected terminal.
Printed copies of scenario information were only printed on an as-needed basis.

When the need for the printed copy was no longer required, the copy was placed in locked disposal bins.

Printed copies of ISV scenario information were marked with a red watermark ISV EXAM - RESTRICTED and a red border. Additionally, red folders or binders with red covers were used to contain printed material when not actively being used for a test.
The simulator and simulator control booth have electronically locked doors. The access to either of these two areas was restricted and not allowed to ISV crew members.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

Physical proximity was also used as a method of protecting exam security. Only the simulator, simulator control booth, and conference room McKenzie (located adjacent to the simulator) were used to view or allow printed ISV material to be left. The doors to these areas were closed with posted signs stating Testing in Progress -

AUTHORIZED PERSONNEL ONLY in red with red border.

Individual notes taken during the conduct of an ISV exam and simulator critique notes were strictly controlled within the restricted areas mentioned above and were controlled or placed in a locked disposal bin.
The McKenzie conference room has one-way mirrored glass to view the simulator.

The shades to block this view were fully lowered at all times.

Initial and follow-up training was provided to ISV crew members on how to spot ISV exam material and what actions to take if material was found unattended.

((

                                         }}2(a),(c) 5.3      Validation Team The ISV validation team consisted of:

((

                                                 }}2(a),(c)

The test lead is responsible for ensuring all aspects of the ISV test are administered in accordance with the ISV test plan. This includes ensuring test prerequisites are complete, the test is administered in accordance with the scenario guide, and performance data is correctly collected (such as TLX and situational awareness questionnaires) at appropriate times. The ISV test plan implementation checklist (Appendix A of the test plan) was completed prior to the start of every scenario. The simulator operator lead is responsible for establishing the simulator initial conditions, initiate and terminate the scenario, and initiate the individual malfunctions as prescribed in the scenario guide at the direction of the test lead. Each scenario guide contained a simulator functionality checklist that was completed prior to the start of every scenario. © Copyright 2019 by NuScale Power, LLC 62

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 The observation team consists of HFE and operations personnel and is described in more detail in Section 5.4. The test support staff role-played as nonlicensed operators and other plant staff as needed. Additionally, the support staff helped to maintain control of ISV crew members when they were outside of the simulator test environment. 5.4 Observation Team Overview ((

                                                                                                }}2(a),(c)

NUREG-0711, Section 11.4.3.1 states, The applicant should describe how the team performing the validation has independence from the personnel responsible for the actual design. Independence is met by not having been involved in the design, development, or testing of the HFE program, HSI, or concept of operations prior to the start of ISV testing. ((

                                                                                        }}2(a),(c) 5.4.1    Observation Team Training Observer training was performed to ensure each observer specifically understood the goals of ISV and the role each plays in providing insightful observations. Observers completed training covering the following topics:

ISV design, scope, and goals

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

familiarization with the forms and worksheets used to collect and analyze data
simulator familiarization including control navigation, notifications, conduct of operations, plant notification system message recognition, computer-based procedure use, and automation interfaces
observers roles and interactions with the participants
importance of and manner in which to make objective and unbiased observations
specific roles of the two independent observers and their importance to mitigate team bias and the importance of independent observer input Observer training was performed through a specific classroom presentation, electronic reading material, and through practical application.

The classroom training was conducted on July 13, 2018, to reinforce training topics prior to the actual start of ISV testing. Each HFE observer completed electronic reading of HFE-related documents through a NuScale core competencies course for HFE, which is documented within the NuScale training program within the Operations core curriculum. Practical observation experience was obtained by having the observers watch and critique the ISV crew simulator training evaluations and audit. An internal assessment was completed during the conduct of the ISV to review the effectiveness of the independent observers and verify compliance with the test plan. The assessment concluded, NuScale has adequately ensured that all independent observer observations are adequately recorded, and all observations have been satisfactorily addressed (( }}2(a),(c) 5.5 Pilot Testing Pilot testing is described in NUREG-0711 as a study performed prior to actual ISV testing to verify the adequacy of the test design, performance measures, and data-collection methods. In addition to checking the methods and process of testing, the pilot tests were used to gather input on the procedures, training, and in some cases changes to the interface in order to improve these products prior to the ISV test. At no time was an ISV crew member involved in pilot test activities where an ISV test scenario was used. ((

                 }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                               }}2(a),(c)

Many action items were identified during the phases of pilot testing, which included: ((

                                          }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.6 Data Collection Methods Test data was collected both electronically and manually throughout the ISV testing. 5.6.1 Electronic Data Collection ((

                                             }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.6.2 Manual Data Collection Various performance measures described in the V&V IP were implemented by multiple manual data collection inputs. The test lead captured operator timing data and general notes on the in-use scenario guides. During scenario performance, the simulator operator lead maintained a log of all communications made with the ISV crew that were outside the guidance provided within the scenario guide. The simulator operator lead also maintained accountability of the four assumed nonlicensed operators such that the crew was not able to use more resources in the field than available on a tracking form. The ISV crew members participated in a critique of their performance and experience using the HSI. This critique was recorded on a critique worksheet included within the ISV test plan. Only one critique worksheet was completed for each crew. Comments from this worksheet were then entered into a comment tracking spreadsheet for analysis. The observation team conducted a debrief following each ISV scenario test. Each observer maintained a set of personal comments written on an observation comment form included within the ISV test plan. Observers were able to take free-form notes on the comment sheets and were used as input to discussions held during the debrief. ((

                     }}2(a),(c)

All manually completed sheets used during the ISV test ((

                                                                                                     }}2(a),(c) were collected.

5.7 Data Analysis During the ISV testing, data was collected as described in Section 5.6. Each day of testing, a preliminary review of data was performed to ensure:

electronic survey results were collected and valid
simulator data was valid
performance measure data that exceeded thresholds was documented Once validated, data was exported to a saved file.

((

                                        }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Data analysis was primarily focused on identifying problem statements for HED generation. Additional comments were generated when a performance measure exceeded a predetermined threshold. Workload, situational awareness, and HFE questionnaires contained threshold criteria that are described in Section 5.8.2 of this report. Upon completion of ISV testing, a review of all collected data and comments was performed to ensure data entry and calculated values were correct. Any missed acceptance criteria or performance measure that exceeded a predefined threshold resulted in a comment. Additionally, all crew critique and observation comments were directly accepted as comments for evaluation. This process resulted in ((

                                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                              }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Binned comments were then further reviewed and grouped based on the following:

HED priority (e.g., no single problem statement would include items considered two separate HED priority levels)
further grouping of similar comments within a bin
potential workgroup assignment The final sets of grouped comments were then issued a title, problem statement, and proposed resolution actions. ((

                                  }}2(a),(c) The detailed HED descriptions including the comments associated with each HED is listed in Section 6.1.

((

                                                                                                            }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Additional methods of data analysis are described separately within each diagnostic measure description in Section 5.8. 5.8 Performance Measures Performance measures used during the ISV test included both deterministic and subjective measurements. Acceptance criteria are pass/fail measures that verify important tasks can be performed within the times assumed by the design. Diagnostic measures include workload and situational awareness questionnaires, critiques, and observations. These measures are generally subjective and oriented towards identifying issues at low levels. Diagnostic measures are not pass/fail and are used together to detect trends and support problem identification. A description of each performance measure characteristics is described in Appendix A of the Human Factors Verification and Validation Implementation Plan. 5.8.1 Acceptance Criteria Overview Acceptance criteria were developed for each scenario based on the following criteria: ((

                           }}2(a),(c)

The criteria above were then applied to the specific scenarios to provide direct and easy-to-evaluate scenario acceptance criteria which were listed on each scenario guide. The scenario guide acceptance criteria would list the specific action and associated completion time. ((

                                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                          }}2(a),(c)

Credited important human actions were completed within one-third of the time allowed in all the scenarios requiring the actions. ((

                                                                                                                   }}2(a),(c) 5.8.1.1 Credited Operator Actions There are a limited number of operator actions that are credited by PRA analysis within the NuScale design. All of the actions credited are used to mitigate beyond-design-basis events. There are no operator actions credited to mitigate design-basis events.

The PRA analysis investigates fault combinations of sequences that can lead to core damage. There are generally three sequences of events that can lead to core damage with each having a very low probability of occurrence. ((

                       }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                         }}2(a),(c)

There are seven credited operator actions identified through analysis of PRA accident sequences. Two of these actions are considered important human actions and described separately in Section 5.8.1.2. Through analysis of PRA accident sequences, there are seven credited operator actions. ((

                                                                                                                       }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                           }}2(a),(c) 5.8.1.2 Important Human Actions Important human actions are a subset of the actions identified as credited operator actions by the PRA analysis. Although the core damage frequency and large release frequency in the NuScale PRA are extremely low, two operator actions were identified as important human actions. Different PRA importance measures are used to identify candidates for risk significance; these two operator actions met the criteria based on their contribution to risk.

The two important actions identified in the NuScale design are shown below. ((

                                                                                                               }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                           }}2(a),(c) 5.8.1.3 Primary Tasks A primary task is one in which the operator must perform the task to directly complete a test goal. All of the acceptance criteria are associated with primary tasks. This section includes primary tasks that are not described by PRA and are not considered only a notification.

Primary tasks are those that have one of the following characteristics:

required to be complete within a specific time, typically determined by technical specifications
operator action to mitigate either equipment damage or a personnel hazard

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 A primary task acceptance criterion was included in every scenario. The allowed time varied depending on the event but was generally based on either a technical specification action time, subject matter expert experience, or an industry norm. The ratios of actual performance compared to the time allowed for primary tasks are illustrated below in Figure 5-2. © Copyright 2019 by NuScale Power, LLC 76

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                                                }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.8.1.4 Notifications to Off-Site Stakeholders There are no operator actions credited in the FSAR Chapter 15 design-basis events and very few actions credited in Chapter 19 beyond-design-basis events. During accident conditions, operators will typically be tasked with verifying passive cooling systems function properly and reporting plant status as required. While not having a direct impact on plant safety, notification tasks were included as acceptance criteria due to their association with the health and safety of the public. The notification acceptance criteria were developed based on timing requirements that are currently being used by the nuclear industry. This is a conservative assumption that no changes occur in those timing requirements for a NuScale plant. For instance, emergency plan entry conditions must be detected and declared by the crew within 15 minutes from when conditions are present under current guidelines. This is done to support notification of off-site agencies for plants with an emergency planning zone that is outside of the owner controlled property. It is probable that a NuScale plant will either have less restrictive times or no required times at all based on a smaller planning zone. The notification acceptance criteria consist of both emergency and nonemergency notifications. In addition to declaring emergency action level entry conditions, the crews were tested on completing required 4- or 8-hour nonemergency reporting documentation. A requirement for reactor operators to quickly announce plant trips was also included as it could be a precursor to either emergency or nonemergency notifications. ((

                                                                                                              }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                      }}2(a),(c) 5.8.1.5 Probabilistic Risk Assessment Update

((

                          }}2(a),(c) 5.8.2    Diagnostic Measures Overview Diagnostic measures were used to aid in identifying issues related to HSI design, procedures, and staffing roles. Each measure has a defined method of data collection and a means to evaluate against a threshold. The purpose of thresholds was to identify issues at a low level for further analysis. The following measures were used:

((

                         }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

Overall workload was reported to be very low when viewed over the wide range of operational conditions sampled. The RO1 position consistently demonstrated the lowest workload ratings among the crew members. Individual crew members experienced discrete elevations in workload, which provided opportunity to understand issues. Typically, workload comments reinforced comments received through other testing methods.

((

                                                                                                }}2(a),(c)

Overall situational awareness was high with a 94 percent success rate. ((

                                                                                                            }}2(a),(c)

A standard set of (( }}2(a),(c) HFE-related questions were administered at the end of each scenario. Each question contained a rating and allowed crew members to add clarifying comments to the rating. ((

                                                                                                           }}2(a),(c)

Overall, the HSI usability was considered easy ((

                                                                         }}2(a),(c) The questionnaire was a valuable tool for providing crew feedback.

The primary method for operators to monitor and perform actions for safe plant operation is through the use of safety functions. The safety functions are monitored automatically and operators confirm indications and perform actions based on safety function indications. The ability to detect, prioritize, and confirm safety functions was rated the easiest characteristic with an overall rating of (( }}2(a),(c).

At the conclusion of each ISV scenario test, the ISV crew members assembled to discuss crew performance and capture additional comments. The ISV crew members focused on how plant design, HSI design, procedures, automation, and training impacted the performed tasks. All critique comments were captured and directly considered for HED identification analysis.

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

At the conclusion of each ISV scenario test, the observation team met to discuss observations collected for that day and track observed cross cutting issues.

((

                                                                                                           }}2(a),(c) 5.8.2.1 Workload NASA TLX was used to provide a measure of workload that can be quantitatively evaluated. This is a measurement tool used in other industries and studies with a track record of effectively measuring workload for individuals performing tasks.

TLX was used for a control room staffing plan validation test conducted in 2016 to assess workload conditions experienced by a crew of five control room operators.

The following lessons learned from the staffing validation test were applied to how TLX was administered for ISV. ((

                                                   }}2(a),(c)

TLX methodology used for ISV testing was benchmarked at two operating nuclear plants during their licensed operator requalification training. The following insights were observed from the data collected during this benchmark:

((

                                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                            }}2(a),(c)

TLX was used during ISV crew training, crew training evaluations, and crew training audit. This provided familiarization with how the specific ISV crew individuals rate workload. Some individuals experience the same event with different ratings. By having experience in observing TLX data with the same group prior to administering ISV testing, the following insights were obtained:

((

                                                                                                             }}2(a),(c)

ISV crew members completed electronic TLX rating forms during specific points within and at the conclusion of each scenario. The form rates the attributes of: mental demand (MD), physical demand (PD), temporal demand (TD), performance (P), effort (E), and frustration (F) on a scale of 1-100. ((

                                            }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                           }}2(a),(c)

Overall workload was shown to be very low when rated holistically over the entirety of the ISV testing. ((

                                                             }}2(a),(c) therefore, the composite average demonstrated the individuals felt workload at a NuScale plant under the sampling of operational conditions was very low.

((

                                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                  }}2(a),(c) 5.8.2.2 Situational Awareness ISV crew members completed situational awareness questions ((
                                                                                                 }}2(a),(c)

Situational awareness questions were developed to assess operators ability to maintain awareness during the various tested sampling conditions. The questions were designed to check the crews awareness of the following items: ((

                                                                 }}2(a),(c)

Overall, the crew situational awareness was demonstrated to be high with an overall average score of 92 percent. The following illustrates the average situational awareness by position in comparison to the overall individual average: © Copyright 2019 by NuScale Power, LLC 85

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                        }}2(a),(c)

Additionally, situational awareness is determined through the use of observations and crew feedback. 5.8.2.3 Human Factors Engineering Questionnaire At the completion of each ISV scenario test, each ISV crew member completed a standard (( }}2(a),(c) question survey to evaluate the usefulness of the HSI. ((

                                                                                                         }}2(a),(c)

The HSI as designed is easy to use ((

                                                                              }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 The average ratings of each characteristic over the entire sampled operating conditions: ((

                                                                                              }}2(a),(c)

These questions are arguably the most pertinent to safe operation of the plant. The overall rating for these questions was considered very easy (( }}2(a),(c). 5.8.2.4 Integrated System Validation Crew Critique At the conclusion of each ISV scenario test, the ISV crew members assembled in a quiet conference room to discuss the performance of the crew and capture additional comments. The ISV crew members were instructed to focus on how the plant design, HSI design, procedures, automation, and training impacted the performed tasks. A post-scenario critique check sheet was completed with the following categories intended to provide structure to the critique session: ((

                                                                                      }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                     }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                     }}2(a),(c) 5.8.2.5 Observation Team Critique During ISV testing, each member of the observation team documented their observations on an observation form.

((

                                                                                             }}2(a),(c)

At the conclusion of each ISV scenario test, the observation team would meet to discuss observations collected for that day and track observed cross cutting issues. ((

                                                                        }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                     }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 5.9 Test Administration Issues during Integrated System Validation The following testing issues were observed during the conduct of ISV testing. None of the issues resulted in the need for a retest or invalidated collected test data.

  #    Comments                                         Disposition

((

                                                                                                                }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 91

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1

  #    Comments                                         Disposition

((

                                                                                                                }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 92

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 6.0 Human Engineering Deficiencies Overview Comments obtained through acceptance criteria and performance measures were analyzed to identify specific problem statements. These problem statements resulted in the set of identified HEDs. Each HED was assigned a priority defined as: Priority 1: potential direct or indirect impact on plant safety and are resolved prior to submittal of the V&V Results Summary Report. HEDs initiated as a result of a performance measure not being met (pass or fail performance measures) are Priority 1 HEDs. Cross-cutting issues determined through HED or performance measure analysis are also Priority 1 HEDs due to their global impact on HSE design performance. Priority 2: direct or indirect impact on plant performance and operability and are resolved before turning over HFE program responsibilities to a licensee. Priority 3: does not fall into Priority 1 or Priority 2. Priority 3 HEDs are resolved in accordance with QA policy-related programs and processes. ((

                              }}2(a),(c)

Each HED lists a title, description, and priority, and is traceable to the original comments or testing forms used to define the HED. HEDs have been entered into the HFEITS database with additional detail to track work assignments and resolution. 6.1 Human Engineering Discrepancy Description Table 6-1 The Following Human Engineering Discrepancies were Identified During Design Verification ((

                                                                                                                 }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 93

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                 }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 94

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                 }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 95

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                        }}2(a),(c)

Table 6-2 The Following Human Engineering Discrepancies were Identified During Integrated System Validation Testing ((

                                                                                                                 }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 96

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                               }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 97

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                          }}2(a),(c) 6.2      Human Engineering Discrepancies Closed Prior to Results Summary Report Submittal The following HEDs have been resolved prior to approval and submittal to the NRC of the verification and validation result summary report.

Table 6-3 Design Verification Human Engineering Discrepancies Closed Prior to Results Summary Report Submittal ((

                                                                                                                  }}2(a),(c)
   © Copyright 2019 by NuScale Power, LLC 98

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                     }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 99

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                          }}2(a),(c)

Table 6-4 Integrated System Validation Human Engineering Discrepancies Closed Prior to Results Summary Report Submittal ((

                                                                                                                 }}2(a),(c)
     © Copyright 2019 by NuScale Power, LLC 100

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                               }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 101

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 7.0 References 7.1.1 U.S. Nuclear Regulatory Commission, Human Factors Engineering Program Review Model, NUREG-0711, Rev. 3, November 2012. 7.1.2 Concept of Operations, RP-0215-10815, Rev. 2. 7.1.3 National Aeronautics and Space Administration, NASA Task Load Index (TLX), Version 1.0, Human Performance Research Group, Moffett Field, CA. 7.1.4 U.S. Nuclear Regulatory Commission, Technical Basis for Regulatory Guidance for Assessing Exemption Requests from the Nuclear Power Plant Licensed Operator Staffing Requirements Specified in 10 CFR 50.54(m), NUREG/CR-6838. 7.1.5 U.S. Nuclear Regulatory Commission, Workload, Situation Awareness, and Teamwork, NUREG/CR-7190. 7.1.6 U.S. Nuclear Regulatory Commission, A Study of Control Room Staffing Levels for Advanced Reactors, NUREG/IA-0137. 7.1.7 Eitrheim, M. H., et al., Staffing Strategies in highly automated future plants: Results from the 2009 HAMMLAB Experiment, OECD Halden Reactor Project, HWR-938, Halden, Norway, 2010. 7.1.8 U.S. Nuclear Regulatory Commission, Event Report Guidelines:10 CFR 50.72 and 50.73, NUREG-1022, Rev. 2, October 2000. 7.1.9 NuScale Power Standard Plant Design Certification Application, Rev. 2. 7.1.10 Control Room Staffing Plan Validation Results, RP-0516-49116, Rev. 1. 7.1.11 Human Factors Engineering Staffing and Qualifications Results Summary Report, RP-0316-17617, Rev. 0. 7.1.12 Human Factors Verification and Validation Implementation Plan, RP-0914-8543, Rev. 4. 7.1.13 Human-System Interface Style Guide, ES-0304-1381, Rev. 2. 7.1.14 U.S. Nuclear Regulatory Commission, Contents of Applications; technical information 10 CFR 50.34. 7.1.15 American National Standards Institute/American Nuclear Society, Nuclear Power Plant Simulators for Use in Operator Training and Examination, ANSI/ANS-3.5-2017, Draft version, LaGrange Park, IL. © Copyright 2019 by NuScale Power, LLC 102

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 8.0 Appendices The following appendices are: Appendix A. Integrated System Validation Testing Schedule Appendix B. Test Performance Matrix Appendix C. Training Inputs to COL Holder Approved Training Program © Copyright 2019 by NuScale Power, LLC 103

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Appendix A. Integrated System Validation Testing Schedule ((

                                                                                                                    }}2(a),(c)
© Copyright 2019 by NuScale Power, LLC 104

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                  }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 105

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                               }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 106

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                                                                                                                }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 107

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Appendix B. Test Performance Matrix ((

               }}2(a),(c)

Table B-1 (( }}2(a),(c) ((

                                                                                                           }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 Appendix C. Training Inputs to COL Holder Approved Training Program ((

                                                                                                       }}2(a),(c)

Human Factors Engineering Verification and Validation Results Summary Report RP-1018-61289-NP Rev. 1 ((

                     }}2(a),(c)

LO-0719-66476 : Affidavit of Zackary W. Rad, AF-0719-66477 NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 Office 541.360-0500 Fax 541.207.3928 www.nuscalepower.com

NuScale Power, LLC AFFIDAVIT of Zackary W. Rad I, Zackary W. Rad, state as follows: (1) I am the Director of Regulatory Affairs of NuScale Power, LLC (NuScale), and as such, I have been specifically delegated the function of reviewing the information described in this Affidavit that NuScale seeks to have withheld from public disclosure, and am authorized to apply for its withholding on behalf of NuScale. (2) I am knowledgeable of the criteria and procedures used by NuScale in designating information as a trade secret, privileged, or as confidential commercial or financial information. This request to withhold information from public disclosure is driven by one or more of the following: (a) The information requested to be withheld reveals distinguishing aspects of a process (or component, structure, tool, method, etc.) whose use by NuScale competitors, without a license from NuScale, would constitute a competitive economic disadvantage to NuScale. (b) The information requested to be withheld consists of supporting data, including test data, relative to a process (or component, structure, tool, method, etc.), and the application of the data secures a competitive economic advantage, as described more fully in paragraph 3 of this Affidavit. (c) Use by a competitor of the information requested to be withheld would reduce the competitors expenditure of resources, or improve its competitive position, in the design, manufacture, shipment, installation, assurance of quality, or licensing of a similar product. (d) The information requested to be withheld reveals cost or price information, production capabilities, budget levels, or commercial strategies of NuScale. (e) The information requested to be withheld consists of patentable ideas. (3) Public disclosure of the information sought to be withheld is likely to cause substantial harm to NuScales competitive position and foreclose or reduce the availability of profit-making opportunities. The accompanying report reveals distinguishing aspects about the process by which NuScale develops its human factors verification and validation. NuScale has performed significant research and evaluation to develop a basis for this process and has invested significant resources, including the expenditure of a considerable sum of money. The precise financial value of the information is difficult to quantify, but it is a key element of the design basis for a NuScale plant and, therefore, has substantial value to NuScale. If the information were disclosed to the public, NuScale's competitors would have access to the information without purchasing the right to use it or having been required to undertake a similar expenditure of resources. Such disclosure would constitute a misappropriation of NuScale's intellectual property, and would deprive NuScale of the opportunity to exercise its competitive advantage to seek an adequate return on its investment. (4) The information sought to be withheld is in the enclosed report entitled Human Factors Engineering Verification and Validation Results Summary Report, RP-1018-61289-P, Revision 1. The enclosure contains the designation Proprietary" at the top of each page containing proprietary information. The information considered by NuScale to be proprietary is identified within double braces, "(( }}" in the document. (5) The basis for proposing that the information be withheld is that NuScale treats the information as a trade secret, privileged, or as confidential commercial or financial information. NuScale relies upon AF-0719-66477 Page 1 of 2

the exemption from disclosure set forth in the Freedom of Information Act ("FOIA"), 5 USC § 552(b)(4), as well as exemptions applicable to the NRC under 10 CFR § 2.390(a)(4) and 9.17(a)(4). (6) Pursuant to the provisions set forth in 10 CFR § 2.390(b)(4), the following is provided for consideration by the Commission in determining whether the information sought to be withheld from public disclosure should be withheld: (a) The information sought to be withheld is owned and has been held in confidence by NuScale. (b) The information is of a sort customarily held in confidence by NuScale and, to the best of my knowledge and belief, consistently has been held in confidence by NuScale. The procedure for approval of external release of such information typically requires review by the staff manager, project manager, chief technology officer or other equivalent authority, or the manager of the cognizant marketing function (or his delegate), for technical content, competitive effect, and determination of the accuracy of the proprietary designation. Disclosures outside NuScale are limited to regulatory bodies, customers and potential customers and their agents, suppliers, licensees, and others with a legitimate need for the information, and then only in accordance with appropriate regulatory provisions or contractual agreements to maintain confidentiality. (c) The information is being transmitted to and received by the NRC in confidence. (d) No public disclosure of the information has been made, and it is not available in public sources. All disclosures to third parties, including any required transmittals to NRC, have been made, or must be made, pursuant to regulatory provisions or contractual agreements that provide for maintenance of the information in confidence. (e) Public disclosure of the information is likely to cause substantial harm to the competitive position of NuScale, taking into account the value of the information to NuScale, the amount of effort and money expended by NuScale in developing the information, and the difficulty others would have in acquiring or duplicating the information. The information sought to be withheld is part of NuScale's technology that provides NuScale with a competitive advantage over other firms in the industry. NuScale has invested significant human and financial capital in developing this technology and NuScale believes it would be difficult for others to duplicate the technology without access to the information sought to be withheld. I declare under penalty of perjury that the foregoing is true and correct. Executed on July 31, 2019. Zackary W. Rad AF-0719-66477 Page 2 of 2}}

ML19212A773

Text

SUBJECT:

REFERENCES:

Navigation menu

Search