Text

LLC Submittal of Human Factors Engineering Verification and Validation Implementation Plan, RP-0914-8543, Revision 5
	ML19119A372
Person / Time
Site:	NuScale
Issue date:	04/29/2019
From:	Bergman T; NuScale
To:	; Document Control Desk, Office of New Reactors
Shared Package
ML19119A371	List: ML19119A372;
References
	LO-0219-64513
	Download: ML19119A372 (80)
	v • d • e

L 0-0219-64513 April 29, 2019 Docket No.52-048 U.S. Nuclear Regulatory Commission ATTN: Document Control Desk One White Flint North 11555 Rockville Pike Rockville, MD 20852-2738

SUBJECT:

NuScale Power, LLC Submittal of "Human Factors Engineering Verification and Validation Implementation Plan," RP-0914-8543, Revision 5

REFERENCE:

Letter from NuScale Power, LLC to U.S. Nuclear Regulatory Commission, "NuScale Power, LLC Submittal of Human Factors Verification and Validation,"

Revision 4, Dated January 4, 2018 (ML18004B974)

NuScale Power, LLC (NuScale) hereby submits Revision 5 of the "Human Factors Engineering Verificiation and Validation Implementation Plan," TR-0914-8543. contains the proprietary version of the report entitled "Human Factors Engineering Verification and Validation Implementation Plan," RP-0914-8543, Revision 5. NuScale requests that the proprietary version be withheld from public disclosure in accordance with the requirements of 10 CFR § 2.390. The enclosed affidavit (Enclosure 3) supports this request. Enclosure 2 contains the non proprietary version of the report entitled "Human Factors Engineering Verification and Validation Implementation Plan," RP-0914-8543, Revision 5.

This letter makes no regulatory commitments and no revisions to any existing regulatory commitments.

If you have any questions, please contact Carrie Fosaaen at 541-452-7126 or at cfosaaen@nuscalepower.com.

Sincerely, Distribution: Gregory Cranston, NRC, OWFN-8H12 Samuel Lee, NRC, OWFN-8H12 Prosanta Chowdhury, NRC, OWFN-8H12 : "Human Factors Engineering Verification and Validation Implementation Plan,"

RP-0914-8543-P, Revision 5, proprietary version : "Human Factors Engineering Verification and Validation Implementation Plan,"

RP-0914-8543-NP, Revision 5 nonproprietary version : Affidavit of Thomas A. Bergman, AF-0219-54514 NuScale Power, LLC 1100 NE Circle Blvd , Suite 200 Corvallis, Oregon 97330 Office 541.360-0500 Fax 541.207.3928 www.nuscalepower.com

LO-0219-64513 :

Human Factors Engineering Verification and Validation Implementation Plan, RP-0914-8543-P, Revision 5, proprietary version NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 Office 541.360-0500 Fax 541.207.3928 www.nuscalepower.com

LO-0219-64513 :

Human Factors Engineering Verification and Validation Implementation Plan, RP-0914-8543-NP, Revision 5 nonproprietary version NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 Office 541.360-0500 Fax 541.207.3928 www.nuscalepower.com

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 Human Factors Verification and Validation Implementation Plan April 2019 Revision 5 Docket: 52-048 NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 www.nuscalepower.com

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 COPYRIGHT NOTICE This report has been prepared by NuScale Power, LLC and bears a NuScale Power, LLC, copyright notice. No right to disclose, use, or copy any of the information in this report, other than by the U.S. Nuclear Regulatory Commission (NRC), is authorized without the express, written permission of NuScale Power, LLC.

The NRC is permitted to make the number of copies of the information contained in this report that is necessary for its internal use in connection with generic and plant-specific reviews and approvals, as well as the issuance, denial, amendment, transfer, renewal, modification, suspension, revocation, or violation of a license, permit, order, or regulation subject to the requirements of 10 CFR 2.390 regarding restrictions on public disclosure to the extent such information has been identified as proprietary by NuScale Power, LLC, copyright protection notwithstanding. Regarding nonproprietary versions of these reports, the NRC is permitted to make the number of copies necessary for public viewing in appropriate docket files in public document rooms in Washington, DC, and elsewhere as may be required by NRC regulations.

Copies made by the NRC must include this copyright notice and contain the proprietary marking if the original was identified as proprietary.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 Department of Energy Acknowledgement and Disclaimer This material is based upon work supported by the Department of Energy under Award Number DE-NE0008820.

This report was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor any agency thereof, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or any agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or any agency thereof.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 CONTENTS 1.0 Introduction ..................................................................................................................... 3 1.1 Purpose ................................................................................................................. 3 1.2 Scope .................................................................................................................... 3 1.3 Abbreviations and Definitions ................................................................................ 4 2.0 Sampling of Operational Conditions and Scenario Development .............................. 7 2.1 Sampling Dimensions ............................................................................................ 7 2.2 Identification of Scenarios ..................................................................................... 7 2.3 Scenario Definition ................................................................................................ 8 3.0 Design Verification Methodology ................................................................................ 10 3.1 Human-System Interface Inventory and Characterization ................................... 11 3.2 Human-System Interface Task Support Verification ............................................ 12 3.3 Human Factors Engineering Design Verification ................................................. 14 4.0 Integrated System Validation ....................................................................................... 16 4.1 Validation Team ................................................................................................... 16 4.2 Test Objectives .................................................................................................... 17 4.3 Validation Test Beds ............................................................................................ 18 4.4 Plant Personnel ................................................................................................... 21 4.5 Performance Measurement ................................................................................. 22 4.6 Test Design .......................................................................................................... 28 4.7 Data Analysis ....................................................................................................... 31 4.8 Validation Conclusions ........................................................................................ 32 5.0 Human Engineering Discrepancy Resolution ............................................................ 34 5.1 HED Design Solution Implementation ................................................................. 34 5.2 Human Engineering Discrepancy Analysis .......................................................... 35 5.3 Design Solution Testing ....................................................................................... 37 6.0 Verification & Validation Results Summary Report ................................................... 38 7.0 NUREG-0711 Conformance Evaluation ....................................................................... 39 8.0 References ..................................................................................................................... 63 8.1 Source Documents .............................................................................................. 63 8.2 Referenced Documents ....................................................................................... 63 Appendix A. Performance Measure Characteristics ......................................................... 64

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 FIGURES Figure 3-1. Phase I: Human-system interface inventory and characterization flow chart ...... 10 Figure 3-2. Phase II: HSI task support verification & HFE design verification flow chart ....... 11 Figure 5-1. Human engineering discrepancy resolution process ........................................... 35 TABLES Table 1-1. Abbreviations ......................................................................................................... 4 Table 1-2. Definitions .............................................................................................................. 5 Table 4-1. Characteristics of performance measures ........................................................... 27 Table 4-2. Basis for performance criteria.............................................................................. 28 Table 7-1. Conformance with NUREG-0711......................................................................... 39

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 Abstract Human factors verification and validation is a critical element of the human factors engineering (HFE) program that performs evaluations to verify that the HFE design conforms to HFE design principles and that it enables plant personnel to successfully and reliably perform their tasks to assure plant safety and operational goals. Human engineering discrepancies are identified and resolved during the verification and validation process.

This implementation plan describes the methodology for conducting the evaluations and identifying and resolving human engineering discrepancies. The methodology described is consistent with the applicable provisions of Section 11 of NUREG-0711, Revision 3.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 Executive Summary The human factors verification and validation (V&V) element of the human factors engineering (HFE) program consists of the following four major activities:

sampling of operational conditions

design verification

integrated system validation

identifying and resolving human engineering discrepancies Sampling of operational conditions identifies the conditions that are representative of the events that may be encountered during plant operation, conditions that reflect the characteristics that may contribute to variations in system performance, and conditions that consider the safety significance of the human-system interfaces (HSIs). These identified operational conditions are used in HSI inventory and characterization, HSI task support verification, HFE design verification, and integrated system validation.

The HSI inventory and characterization accurately describes all HSI displays, controls, and related equipment lying within the scope defined by the sampling of operational conditions. The HSI task support verification confirms that the HSIs provide the alarms, information, controls, and support needed for personnel to perform their tasks as defined by the task analysis. HFE design verification confirms that the design of the HSIs conform to HFE guidelines. Integrated system validation verifies, using performance-based tests, that the integrated system design (i.e., hardware, software, procedures, and personnel elements) supports the safe operation of the plant.

Human engineering discrepancies (HEDs) are identified during the V&V process. HED resolution may be performed iteratively. That is, the identified HEDs are evaluated and resolved appropriately during one V&V activity before conducting other V&V activities. The preferred order of the process is HSI inventory and characterization, HSI task support verification, HFE design verification, and integrated system validation, although iteration may be needed.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 1.0 Introduction 1.1 Purpose This document provides the human factors verification and validation (V&V) implementation plan (IP) for the NuScale Power, LLC (NuScale) plant human-system interface (HSI) design. The HSI design includes the hardware, software, and personnel elements used to operate a NuScale plant.

The NuScale human factors V&V program confirms that the HSI design

conforms to the specified design.

conforms to appropriate design criteria.

performs within acceptable limits under analyzed operating modes and conditions.

provides the complete set of alarms, controls, indications, and procedures needed to support the personnel tasks as identified in the task analysis (TA).

adequately supports plant personnel in the safe and reliable operation of the plant.

1.2 Scope This IP describes the methodology for conducting the four major activities of the human factors V&V element (sampling of operational conditions, design verification, integrated system validation (ISV), and human engineering discrepancy (HED) resolution),

including:

identification of sampling dimensions and scenarios used for validation of the HSI

human-system interface inventory and characterization

the criteria used for task support verification and human factors engineering (HFE) design verification

selection and training of the Validation Team

determination of validation test objectives

use of the main control room (MCR) test bed for validation

selection and training of personnel used as operating crews (i.e. plant personnel)

scenario selection and definition for the validation

performance measures to be used in the validation

design of testing

data analysis methods applied to validation data

validation of procedures

guidance for initiation and evaluation of HEDs

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 This IP provides a description of the methodology for the identification of scenarios for the ISV. The V&V results summary report (RSR) will provide the information as discussed in Section 6.0. A detailed ISV test report will be developed which supports the findings documented in the V&V RSR; both documents will be submitted to the NRC.

The V&V RSR will also confirm and document that the human factors ISV scope includes the alarms, controls, indications, and procedures for the HFE program.

The HFE program scope is described in the Human Factors Engineering Program Management Plan (Reference 8.2.1). Sampling dimensions with regard to locations, HSIs, conditions, types of tasks, and situational factors are described in Section 2.1.

1.3 Abbreviations and Definitions Table 1-1. Abbreviations Abbreviation Definition ADDIE analysis, design, development, implementation, and evaluation HED human engineering discrepancy HFE human factors engineering HFEITS human factors engineering issue tracking system HSI human-system interface I&C instrumentation & control IHA important human action IP implementation plan ISV integrated system validation MCR main control room PRA probabilistic risk assessment RSR results summary report SA situation awareness SME subject matter expert SOC sampling of operational conditions TA task analysis V&V verification & validation

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 Table 1-2. Definitions Term Definition Embedded procedure ((2(a),(c) Human Factors Engineering Generic term for the Plant Operations organization which consists of Design Team Operators, Human Factor Engineers, and Simulator Developers. The HFE Design Team does not include Plant Personnel. The HFE Design Team is responsible for the human factors engineering associated with the NuScale design. Also referred to as the design team. Human System Interface The human-system interface (HSI) is that part of the system through which personnel interact to perform their functions and tasks. In this document, "system" refers to a nuclear power plant. Major HSIs include alarms, information displays, controls, and procedures. Use of HSIs can be influenced directly by factors such as, (1) the organization of HSIs into workstations (e.g., consoles and panels) (2) the arrangement of workstations and supporting equipment into facilities such as a main control room, remote shutdown station, local control station, technical support center, and emergency operations facility and (3) the environmental conditions in which the HSIs are used, including temperature, humidity, ventilation, illumination, and noise. HSI use can also be affected indirectly by other aspects of plant design and operation such as crew training, shift schedules, work practices, and management and organizational factors. Plant Personnel Operating crew members participating in the ISV. Plant personnel are not part of the HFE Design Team or Validation Team. Simulator Operator Person responsible for running the simulator during design, training, and testing. During training and testing, simulator operators should keep track of directions given to nonlicensed operators (NLOs) and other personnel simulated outside the control room. Simulator operators role play as personnel outside the control room and may only provide data that is allowed per the applicable scenario or training guide. Simulator operators may answer questions asked by the crew but should not lead them to the correct answer or diagnosis. Simulator operators are also referred to as booth operators. Simulator Review Board The Simulator Review Board reviews the results of simulator testing and compares them to analysis and engineering calculations to certify that the simulator reflects the plant design. This board consists of representatives from Safety Analysis, probabilistic risk assessment (PRA), engineering, and operations. Their review is focused on realism to the operator and model validity. (( Validation Team

                                                    }}2(a),(c)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 Term Definition The VISION Developer application is a relational database that is used to store the FRA/FA, task analysis, staffing and qualifications VISION analysis, development of human-system interfaces (HSI), procedures, and training data. In this document it may be referred to as the FRA/FA & TA database or database. © Copyright 2019 by NuScale Power, LLC 6

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 2.0 Sampling of Operational Conditions and Scenario Development The purpose of sampling of operational conditions (SOC) is to identify a broad and representative range of operating conditions to be sampled during the HSI inventory and characterization (see Section 3.1), task support verification (see Section 3.2), HFE design verification (see Section 3.3), and ISV testing (see Section 4.0). The sample is deemed representative if the samples safety significance, risk significance, and challenges to the operating crew are considered to be within the range of events that the operators could encounter during the plant life cycle. ((

                                                                }}2(a),(c) 2.1      Sampling Dimensions A range of plant conditions, personnel tasks, and situational factors is considered within the sampling dimensions included in Section 11.4.1 of Human Factors Engineering Program Review Model, NUREG-0711, Rev. 3 (Reference 8.1.1) as applicable to the NuScale design.

NuScale operates up to 12 reactors from a single control room and utilizes a digital control system and relies heavily on automation and computer-based procedures. The sampling dimensions include normal operational events, transients and accidents. Due to the increased use of digital technology in the NuScale control room, scenarios will specifically provide an emphasis on instrumentation and control (I&C) and HSI failures as well as degraded conditions. Scenario development goals are written to ensure the scenarios are comprehensive, and when taken together, cover aspects of all sampling dimensions relevant to the NuScale design. 2.2 Identification of Scenarios Members of the NuScale HFE Team develop the ISV scenarios using multiple sampling dimensions to accomplish the goals and set the conditions to be included in each scenario based on the SOC. Biases for individual dimensions are possible, but collectively, the scenarios avoid bias by representing scenarios that

have both positive and negative outcomes
require varying degrees of administrative burden to run (test bed set-up, instructor input)
minimize the use of well-known and well-structured sequences (i.e., textbook design-basis accident mitigation)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 During identification of scenarios for ISV, the HFE Design Team develops a table to compare the SOC criteria in each scenario; the comparison table helps assure that representative SOC criteria are addressed by the composite set of scenarios. This comparison table is used to document the bases for assurance that the selected scenarios are representative of expected operational conditions as discussed in Section 6.0. The ISV scenarios are then reviewed by the appropriate SMEs and approved by operations management. Upon approval, the ISV scenarios and test plan will be available for review or audit by the NRC sufficiently before the conduct of ISV so that comments or concerns can be adequately addressed prior to commencing ISV. 2.2.1 Scenario Security The following scenario security steps are maintained throughout the ISV entire development and testing process.

The scenario descriptions and collection of tasks are stored in VISION in a separate work area with access only granted to the scenario and testing developers.
The selected operating crew member participants (Plant Personnel) are not allowed to review documents associated with the completed scenarios (i.e., scenario guides).
Printed copies of scenario information are destroyed or placed in a secure location when not in use.

2.3 Scenario Definition The scenarios used for design verification and ISV testing are selected during the SOC and scenario development process. Scenarios are run in the test bed to validate performance of the integrated system (i.e., hardware, software, and personnel elements) and ensure the design is consistent with the objective. The defined scenarios are designed to involve major plant evolutions or transients, reinforce team concepts, and identify the role each individual plays within the team. Tasks performed by operators remote from the MCR are modeled in the ISV scenario to realistically simulate effects on personnel performance due to potentially harsh environments. Effects such as additional time to don protective clothing, set up of radiological access control areas, and employment of damage control, emergency, or temporary equipment are described in scenarios by use of time constraints/additions. The NuScale ISV scenarios are developed in a systematic manner and include all applicable test attributes:

a synopsis
objectives
initial condition of the entire plant
specific initial conditions pertinent to commencement of the scenario

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

a timeline of events to be run including initiating conditions where appropriate
critical tasks to be conducted
workplace factors (e.g., environmental conditions)
any material or knowledge-based needs to support the task to be tested
staffing level
where specific types of communications are necessary (e.g., an event notification to regulators via dedicated telephone line) details of that expected communication content
scripted responses for test personnel (both in and out of the MCR)
data to be collected by observers/instructors (rating scales for administrators are included where appropriate)
pass/fail criteria for any part of the scenario
initial test bed set up
specific criteria for terminating the scenario The ISV scenarios are developed to be representative of the range of events that could be encountered during the plants operation, determined by SOC as described in Section 2.1. Scenarios developed by the HFE Design Team that lead to only positive outcomes, scenarios that are easy to conduct, and scenarios that are well-structured and often practiced are not selected. Scenarios are selected to confront the operating crew with challenging normal conditions and abnormal events containing multiple and unanticipated failures.

Test objectives are discussed in Section 4.2. An individual scenario cannot address all test objectives, but the aggregate ISV includes testing of all objectives. Each scenario tests some portion of the HSI for primary actions (control and verification via the plant response) and secondary actions (navigating the HSI for monitoring of other plant parameters); communication equipment is also verified during scenarios. © Copyright 2019 by NuScale Power, LLC 9

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 3.0 Design Verification Methodology The design verification activity is accomplished during two phases of the V&V process; Phase I (Figure 3-1) is HSI inventory and characterization, and Phase II (Figure 3-2) is HSI task support verification and HFE design verification. The flow charts for each activity are shown below followed by a discussion of each phase. ISV testing can involve hundreds or thousands of individual HSIs, and it is impractical and unnecessary to review all of them. Therefore, NuScale employs a sampling strategy to guide the selection of HSIs to review. ((

                                                                                                          }}2(a),(c)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                                                                                                       }}2(a),(c)

Figure 3-2. Phase II: HSI task support verification & HFE design verification flow chart NOTE: HED identification and resolution details are discussed in Section 5.0. 3.1 Human-System Interface Inventory and Characterization The objective of the HSI inventory and characterization is to accurately describe the set of selected HSI displays, controls, and related equipment within the scope defined by the SOC. Automation and the associated embedded procedures are also included in the scope of HSI inventory and characterization. The HFE Design Team follows a process that includes verifying all HSI elements against the TA and provides a feedback loop back to the HSI input block as shown in Figure 3-1. 3.1.1 Human-System Interface Inventory The list of HSI inventory is generated during TA and developed during Phase I of the HSI design process. The TA defines the inventory and characterization for the alarms, controls, indications, and procedures needed to execute operator tasks for normal and abnormal plant conditions including manual tasks, automation support tasks, and automation monitoring tasks. In preparation for characterization, the output of TA and HSI design is compared to the HSIs that personnel will need for the tasks in the scenarios developed for SOC. Characterization defines the functionality of each HSI. 3.1.2 Human-System Interface Characterization Characterization defines the functionality of each HSI selected for verification. HSI design documents such as equipment lists, design specifications, and input/output lists are produced during HSI design. Characteristics of each HSI component are included in the associated design document which includes the minimum set of information: © Copyright 2019 by NuScale Power, LLC 11

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

a unique equipment identification code that links the HSI component to the associated plant system or subsystem
associated personnel functions/sub-functions
type of HSI (indication, control, alarm, procedure, hard-wired, screen-based, etc.)
HSI characteristics and functionality (unit of measure, accuracy of variable/parameter, format, continuous or discrete (if a control), system response time, etc.)
HSI control characteristics and functionality (modes, accuracy, precision, format)
method of use and associated user-aids
physical or virtual (i.e., on a screen) location of HSI 3.1.3 Inventory Verification Inventory verification confirms the visual aspects (alarms, controls, indications, embedded procedures and the means of navigation between elements) of the HSI, including conformance to the NuScale Human System Interface Style Guide (Reference 8.2.2) during HFE design verification. This also includes verification of other HSI characteristics such as tag number, location, piping, and instrument diagram or logic diagram implementation.

NuScale HSI navigation and notifications are part of the spatially dedicated continuously visible main navigation bar. These elements do not need to be verified for every system HSI developed. These global elements are verified once during this verification phase for all selected HSI following the process used during staffing plan validation. 3.2 Human-System Interface Task Support Verification The purpose of HSI task support verification is to verify the HSIs support the task requirements on the selected HSI. The assessment verifies that HSIs provide the alarms, controls, indications, and task support for personnel to perform their tasks as defined by the TA. For HSI task support verification related to performance (e.g., accuracy and dynamic response), the validation test bed is used. The HSI task support verification process is shown in Figure 3-2. 3.2.1 HSI Task Support Verification Criteria The task support verification is based on the TA results that define the inventory and characterization for the alarms, controls, indications, procedures, automation, and task support needed to successfully execute operator tasks. 3.2.2 HSI Task Support Evaluation Methodology The HFE Design Team conducts HSI task support verification using a verification process to control bias and improve consistency. The task support verification process entails a detailed comparison of the personnel task requirements identified by the TA © Copyright 2019 by NuScale Power, LLC 12

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 (i.e., the planned attributes) with the alarms, controls, indications, procedures, automation, and task support in the HSI inventory and characterization (i.e., the actual attributes). The HFE Design Team follows a process that provides a Retest step if needed as shown in Figure 3-2. Results of the task support verification are based on the criterion that the information, control, and functional characteristics to support the task requirements identified during TA are present in the HSI that is being verified for the task. Results are documented for each task in the V&V RSR (see Section 6.0) once the V&V activities are complete. 3.2.3 Task Support Verification The verification process is conducted using internal procedures and is based on

the most recently completed TA.
the personnel task requirements identified by the TA with the available alarms, controls, indications, and procedures in the HSI inventory.
guidelines for determining whether the HSI is "acceptable" or "discrepant" based on the associated HFE design criteria.
completed Inventory and Characterization forms used to verify that the elements on the pages have the appropriate design characteristics, including dynamic behavior.
completed HFE Design Verification forms used to verify that the elements on the pages were consistent both on the page being tested and with other pages in the inventory.
the most recent version of HSI display pages.
an active simulator.

3.2.4 Human-System Interface Acceptability Criteria Internal procedures are used for determining whether the HSI is "acceptable" or "discrepant." This procedural guidance includes:

1. A judgment that an HSI is acceptable should be made only if compliance is total (i.e., only if every instance of the item is fully consistent with the criteria established by the HFE guidelines).

2. If there is any noncompliance, full or partial, then an evaluation of discrepant should be given, and a notation made as to where it occurs.

3. If discrepant, it should be designated as an HED, tracked, and evaluated.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

a documented list of each test team members findings used to develop a team consensus.
a completed task support verification form used to verify the tasks can be performed using the display pages.
a description of the means of comparing HFE design criteria to HSI components in the context of the various environmental conditions or locations of those HSIs (e.g.,

noise, lighting). This piece of the task support verification will be performed in the simulator during operator training or testing. 3.3 Human Factors Engineering Design Verification The HFE design verification is conducted to confirm that HSI characteristics conform to HFE guidelines as represented in the style guide. The style guide consists of procedures for use, general considerations, and system-specific guidance for screen-based HSIs (the term system-specific applies to plant systems as well as HSI systems). The HFE design verification process is shown in Figure 3-2. 3.3.1 Verification Criteria The criteria for HFE design verification is provided by the HSI style guide. The style guide includes procedural guidance for determining appropriate design criteria when the style guide does not apply to the characteristics of the HSI component being designed. ((

        }}2(a),(c) 3.3.2    Design Verification Evaluation Methodology HFE design verification is conducted in accordance with a written process to assure consistency of results and to control bias. The design verification phase for all selected HSI follows a process that provides a Retest step if needed as shown in Figure 3-2.

Procedures describing HFE design verification include

checklists and guidelines for comparison of the HFE design criteria (style guide) to HSI components (e.g., alarms, controls, indications, procedures, navigation aids)
a description of the means of comparing HFE design criteria to HSI components in the context of the various environmental conditions or locations of those HSIs (e.g.,

noise, lighting, ambient temperature and humidity)

guidelines for determining whether the HSI is acceptable or discrepant based on the associated HFE design criteria

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

methods for preparation and review of the HFE design verification as well as course of action when reviewers do not agree on the results
design verification HEDs are generated for HSIs that do not meet the HFE design criteria completely

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 4.0 Integrated System Validation The ISV is the process by which an integrated system design (i.e., hardware, software, and personnel elements) is evaluated using performance-based tests to determine whether it acceptably supports safe operation of the plant. The ISV is undertaken only after HEDs that were identified in the upstream process, including design verification, have been resolved and the resulting changes implemented. Scenarios are developed using the guidance described in the implementing procedures. Performance measures used for assessing the results of an ISV are summarized in Section 4.5 and further described in implementing procedures. 4.1 Validation Team Some validation team members can be selected from the HFE design team but at least two of the observers must have independence from ISV test design. Objective performance measures and success criteria are developed as part of the methodology and listed within the scenario guides used for the conduct of ISV tests. Objective performance measures are designed to trigger evaluation of the condition regardless of observation comments, and are purposely set at a low threshold. The methodology, scenarios, ISV test plan, and ISV test performance are available for NuScale management assessment or NRC audit well in advance of or during the conduct of the ISV in order to allow for an outside perspective to detect and influence potential bias concerns. The validation team members are trained and qualified to conduct the ISV in an objective and unbiased manner. A detailed ISV test report is developed which supports the results documented in the V&V RSR; both documents will be submitted to the NRC. The HFE Design Team developing and conducting the ISV is analogous to a commercial nuclear plants Training Department developing and conducting an NRC license exam or annual requalification exam. ((

                                              }}2(a),(c) The observers are trained and qualified using the NuScale training program. At least two of the selected observers in each ISV test performance must have independence from HFE Design Team (i.e. has not been involved in the design, development, or testing of the NuScale HFE program, HSI, or concept of operations). Observations made by these observers are relied upon to provide independent, unbiased, and objective observations of the test performance.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 All observer comments will be assessed by consensus to determine which will result in HEDs and for priority assignment. The independent observers are equal participants during scenario debriefs. If consensus agreement cannot be met on resultant HEDs or priority, the conflict will be presented to management for resolution. The administrators (test lead, test bed engineer, and test bed support staff) manage the ISV, control each scenario in accordance with the test procedure, maintain and set up the test bed, and collect the test bed archived data following each scenario. The Validation Team personnel may act as simulated plant personnel as necessary within each scenario. The administrators are trained and qualified using the NuScale training program. Bias is further reduced by the training program applicable to each validation team member, and the fact that results are obtained by consensus of the team rather than individual observations. Observer training will consist of practice observations to ensure understanding of the measurement techniques and ensure understanding of the test objectives and acceptance criteria. ((

                                                          }}2(a),(c) 4.2      Test Objectives The objectives of the ISV are to validate

the acceptability of the shift staffing, the assignment of tasks to operating crew members, and crew coordination within the control room, between the control room and local control stations and support centers, and with individuals performing tasks locally. This should encompass validating minimum shift staffing levels, nominal levels, higher levels, and shift turnover.
that the design has adequate capability for alerting, informing, controlling, and feedback such that personnel tasks are successfully completed during normal plant evolutions, transients, design-basis accidents, and also under selected risk significant events beyond-design basis, as defined by the SOC.
that specific personnel tasks can be accomplished within the time and performance criteria, with effective situational awareness, and acceptable workload levels that balance vigilance and personnel burden.
that the HSIs minimize personnel error and ensure error detection and recovery capability when errors occur.
the assumptions about performance on important human actions (IHAs).
personnel can effectively transition between the HSIs and procedures in accomplishing their tasks, such as display configuration and navigation, are not a distraction or an undue burden.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 4.3 Validation Test Beds The principal validation test bed for the ISV is the control room simulator. The fidelity of the validation test beds models and HSI are verified to represent the current, as-designed NuScale plant prior to use for the validation. The test bed model is made up of four modeling software packages, all working from current NuScale designs. Together, they provide a high level of fluid and reactivity modeling. Precisely modeling the predicted behavior of the reactor core, thermodynamic performance, balance of plant, and electrical system design is desired as NuScale does not have a comparison reference plant. All 12 units are simultaneously and independently modeled, but they all correctly share systems that provide input for multiple units. The test bed is validated against the seven criteria described in Section 11.4.3.3 of Reference 8.1.1: interface completeness, interface physical fidelity, interface functional fidelity, environment fidelity, data completeness fidelity, data content fidelity, and data dynamics fidelity. These criteria are further discussed in sections 4.3.1 thru 4.3.7 below. The validation test bed attempts to accurately simulate a NuScale plant MCR environment. Where this is not achievable by the test bed (e.g. room temperature and lighting during a loss of all AC power), an exception is taken and documented in the V&V RSR as discussed in Section 6.0. If necessary, changes are also made to the ISV test procedure to reflect the alternate test bed configuration. In some limited cases, the V&V team may consider the test bed discrepancies to affect specific aspects of the validation results. If so, an HED is generated to document the discrepancy and the concern. The HED is resolved in accordance with the HED resolution process described in Section 5.0. 4.3.1 Interface Completeness The test bed represents a complete and integrated system with HSI and procedures not specifically required in the test scenarios. (e.g., alternate procedures). The test bed further represents interfaces with the RSS and local control stations (i.e., communications) to provide an integrated system. 4.3.2 Interface Physical Fidelity High physical fidelity in the HSI and procedures is represented, including presentation of alarms, displays, controls, procedures, automation, job aids, communications, interface management tools, layout, and spatial relationships. The test bed is a replica in form, appearance, and layout of the NuScale MCR design. 4.3.3 Interface Functional Fidelity High functional fidelity in the HSI, procedures, and automation is represented so that the HSI functions are available and the HSI component modes of operation, types of feedback, and dynamic response characteristics operate in the same way as the actual plant. © Copyright 2019 by NuScale Power, LLC 18

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 4.3.4 Environmental Fidelity The test bed is representative of the actual NuScale plant with regard to environmental features such as lighting, noise, temperature, humidity, and ventilation characteristics. In cases where the test bed cannot accurately simulate the environment, the ISV captures human factors engineering issue tracking system (HFEITS) entries for evaluation and resolution. 4.3.5 Data Completeness Fidelity In the test bed, information and data provided to personnel represent the complete set of plant systems monitored and controlled from that facility. 4.3.6 Data Content Fidelity The test bed represents a high degree of data content fidelity. The alarms, controls, indications, procedures, and automation presented are based on an underlying plant model that accurately reflects the engineering design of the NuScale plant. The model also accurately provides input to the HSI, such that the information matches what is presented during operations. 4.3.7 Data Dynamics Fidelity The test bed represents a high degree of data dynamic fidelity. The plant model provides input to the HSI in a manner such that information flow and control responses occur accurately and in a correct response time. Information is provided to personnel with the same anticipated delays as would occur in the plant. 4.3.8 Remote Human-System Interfaces Containing Important Human Actions NuScale has no IHAs that are conducted outside of the MCR. In the event that a remote IHA is determined in a later design stage, the test bed uses mockups to verify human performance requirements for IHAs conducted at HSIs remote from the MCR. The simulation or mockup considers, for example, transit times, use of personal protective equipment, and delays associated with the need for operator precision (self-checking). 4.3.9 Test Bed Conformance The test bed is verified by performance testing to conform to required characteristics before validation tests are conducted. 4.3.10 ISV Simulator Performance Testing The purpose of ISV Simulator performance testing is to ensure simulator performance is sufficiently complete and accurate to meet the requirements recommended in NUREG-0711, Human Factors Engineering Program Review Model as it pertains to simulators used during ISV activities. NUREG-0711 recommends that the simulator used for ISV should have fidelity and functionality compliant with industry standard ANSI/ANS-3.5-2009, Nuclear Power Plant Simulators for Use in Operator Training and © Copyright 2019 by NuScale Power, LLC 19

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 Examination. ANSI/ANS-3.5 is intended to provide standards used to train licensed operators at an operating facility and not to conduct ISV testing. Therefore, the ISV Simulator Performance Testing uses selected criteria in a similar manner to the concept already used within the ANSI/ANS-3.5 Appendix D standard to establish criteria for part-task and limited-scope simulators. The following criteria are used to evaluate ISV Simulator performance:

real time and repeatability testing
limits of simulation testing
normal evolution testing
malfunction testing
steady state testing Prior to the start of ISV, the ISV simulator will have completed ISV Simulator performance testing to validate overall performance. The ISV Simulator performance testing provides a comprehensive evaluation of overall simulator performance, while Scenario-Based Testing provides a detailed review of the simulator response to the individual ISV scenarios.

4.3.11 Scenario-Based Testing The testing is conducted by determining a set of key parameters to be evaluated and ensuring those parameters behave as expected for the developed ISV scenarios. ANSI/ANS-3.5-2009 was referenced for a draft list to select steady state and transient parameters. The scenarios are then conducted in real time, to ensure the completion of the objectives and termination point is reached. The procedures are executed as described in the current task analysis. The freeze feature may only be used during testing to obtain additional data and shall have no effect on the simulator parameters or resuming of the scenario for the test to be considered valid. The following criteria are used to evaluate the simulator performance while running the ISV scenarios:

        *    ((
                                                                               }}2(a),(c)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

        *    ((
                           }}2(a),(c) 4.4      Plant Personnel Individual operating crews participating in the ISV may be previously licensed commercial reactor or senior reactor operators, operators with Navy nuclear experience, or design engineering staff members familiar with the NuScale Power plant design. The personnel participating in ISV are trained, qualified, and are assigned to roles commensurate with their experience, skill, and knowledge level.

Personnel who constitute the ISV operating crews are not part of the HFE V&V team or HFE design team. Operating crew makeup is not varied from scenario to scenario and remains consistent throughout the validation (i.e., crew members are not rotated between operating crews). To control crew bias, individual crew members are distributed across crews with consideration for:

age distribution
gender distribution
education level distribution

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

experience distribution; generally industry operators have a minimum of one year of experience, while engineers have a minimum of two years experience in addition to NuScale plant systems training Operating crew size for the validation tests includes a range of expected sizes to ensure that the HSI supports operations and event management. This range includes the minimum operating crew, nominal levels, and higher levels as defined during the staffing and qualifications program element NuScale Human Factors Engineering Staffing and Qualifications Results Summary Report (Reference 8.2.3) for a range of plant operating modes. The crew size for each scenario is identified in the ISV test procedure, and scenarios are not repeated with different crew sizes.

The ISV includes at least one scenario with more than minimum crew staffing defined in Reference 8.2.3 (e.g., additional licensed operators to complete a complex evolution) to simulate times of high control-room traffic and distractions and high environmental loading. The roles of the additional personnel and their interaction with the operating crew are determined by the scenario developers based on meeting all the test objectives and goals and by applying the SOC criteria. 4.5 Performance Measurement Performance measures for ISV are hierarchical and include measures of plant performance, personnel task performance, situation awareness (SA), cognitive and physical workload, and anthropometric or physiological factors. Both pass or fail and diagnostic measures are applied. 4.5.1 Types of Performance Measures 4.5.1.1 Plant Performance Measures Plant performance resulting from operator action or inaction includes plant process data (e.g., temperature, pressure) and component status (e.g., on/off; open/closed) as a function of time at as many locations in the plant simulation as is possible. These data are obtained from the entire plant: nuclear, fluid, structural, and electrical components. Any component that provides plant process data or component status in the plant is simulated with appropriate fidelity. The test bed has the ability to record all plant process data and component status (including state changes) for the full length of any ISV scenario. 4.5.1.2 Personnel Task Performance Measures For each scenario, tasks that personnel are required to perform are identified and assessed. Primary and secondary personnel tasks are evaluated. Primary tasks are those involved with function and task completion including detection, assessment, planning, and response. The level of detail to which primary tasks are measured and performance measures selected are assessed based on the complexity of the task. It may only be necessary to measure time and accuracy for a lower level © Copyright 2019 by NuScale Power, LLC 22

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 rule-based task to recognize and respond, while tasks that are knowledge-based (e.g., detection, seeking additional data, making decisions, or taking actions) may entail the use of more detailed performance measures. Secondary task performance measures reflect the workload associated with HSI manipulations associated with maintaining the overall plant. Test personnel evaluate secondary tasks in conjunction with primary tasks to observe effects on overall performance and workload both at individual and operating crew level. Personnel task performance measurements are selected to reflect those aspects of the task that are important to system performance and used depending on the particular scenario such as

time
accuracy
frequency
amount achieved or accomplished
consumption or quantity used
subjective report of participants
behavior categorization by observers

((

                                      }}2(a),(c)

Objective measures of individual or crew and system performance are also collected during validation scenarios and are used for documenting the performance and future use. They include

video recordings of operator performance
alarm history log

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

operator control interactions
plant variable control interactions (resulting from operator controls)
component status change
HSI use log (display screen request history and operational history)

The capturing of data using cameras enables NuScale to document the operators actions as they are performed. With the information archived, it is then available for the life of the design for tracking purposes. The comparison between actual and expected actions is an important test criterion when trying to identify errors of omission and commission. NuScale performs this comparison during the V&V testing process and will maintain a retrievable video library, as a contingency, for instances where observations conflict or actions come into question. 4.5.1.3 Situational Awareness Performance Measures To measure SA, ISV applies a combination of objective measures along with subjective questionnaire methods. ((

             }}2(a),(c)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                          }}2(a),(c) 4.5.1.4 Cognitive and Physical Workload Performance Measures To measure cognitive workload, the ISV employs the NASA Task Load Index using the following methods:
        *    ((
                                             }}2(a),(c)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 4.5.1.5 Anthropometric and Physiological Factor Performance Measures The primary purpose of anthropometric and physiological performance measures during ISV is to assess those aspects of the design that cannot be evaluated during design verification. Anthropometric and physiological performance measures evaluate how well the HSI supports plant personnel in monitoring and control of the plant. Many of these design aspects are assessed as part of verifying the HFE design. Therefore, the focus is on those areas of the design that only can be addressed by testing the integrated system, e.g., the ability of personnel to effectively use the various controls, displays, workstations, or consoles while performing their tasks. ((

                                                    }}2(a),(c) 4.5.2    Performance Measure Information and Validation Criteria 4.5.2.1 Collection Methods Subjective assessments of the HSI and its impact on performance, including self-ratings of workload, SA, and teamwork, are conducted by test personnel operating crews.

Operator feedback on the HSI is collected via scenario debriefs and questionnaires. Both types of operator feedback include scale rating questions and open feedback (long answer) questions. Objective data (e.g., video recording, administrator observations) collected during test scenarios are analyzed to assess impacts of operator actions on plant processes and equipment states. The analysis compares the performance derived from parameters and times collected by the test bed to the evaluation criteria for operator actions and for overall plant process behavior developed for each scenario. Test observers and administrators document individual assessments of crew performance on a post-scenario observer form immediately after the scenario. ((

                           }}2(a),(c)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

        *    ((
                                 }}2(a),(c)

In addition to HSI performance problems, observers and administrators rate technical and teamwork performance on the post-scenario observer form. Crew size sufficiency is rated, and any potential or noticeable HEDs are identified. Test subjects also document their feedback on a post-scenario test subject form immediately after the scenario. The test subject form is similar to that of the observer and administrator with observations of HSI performance problems, technical and teamwork performance observations, crew size sufficiency ratings, and potential or noticeable HEDs. The data collected from subjective and objective sources are analyzed by the HFE team to determine the sufficiency of the HFE design. 4.5.2.2 Performance Measure Characteristics and Bases Performance measures to be observed during ISV contain the characteristics described in Table 4-1. Table 4-1. Characteristics of performance measures Characteristic Meaning A measure should represent accurately the aspect of performance it is intended to Construct Validity measure. A measure should be repeatable; i.e., same behavior measured in exactly the Reliability same way under identical circumstances should yield the same results. A measure's range (scale) and its frequency (how often data are collected) should Sensitivity be appropriate to that aspect of performance being assessed. A measure should minimally alter the psychological or physical processes that are Unobtrusiveness being investigated. Objectivity A measure should be based on easily observed phenomena. The basis for inclusion of a performance criterion in the ISV (or a particular scenario within ISV) used to judge acceptability of that criterion is determined during the development of the scenario. Bases for performance criteria are described in Table 4-2. © Copyright 2019 by NuScale Power, LLC 27

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 Table 4-2. Basis for performance criteria Criteria Basis Meaning The observed performance of the integrated system is compared with a quantified Requirement performance requirement; i.e., the requirements for the performance of systems, subsystems, and personnel are defined through engineering analyses. The observed performance of the integrated system is compared with a criterion Benchmark established using a benchmark system, e.g., a current system is predefined as acceptable. The observed performance of the integrated system is compared with a criterion Norm using many predecessor systems (rather than a single benchmark system). The observed performance of the integrated system is compared with a criterion Expert Judgment established by subject-matter experts. Performance measures are designated as pass, fail or diagnostic. Diagnostic is measureable and the criteria include both range and unit of measure. Appendix A provides specific attributes for the measures being used. With respect to each measure, the associated characteristics are rated as Hi, Med, or Low to provide a qualitative rating of how well the characteristic is achieved. An explanation for the rating is also provided. 4.6 Test Design Test design refers to the process of developing scenarios, test plans, and conducting ISV based on the integrated HSI as described in the preceding sections. The goal of test design is to permit the observation of integrated system performance while minimizing bias. Once the ISV test plan and scenarios are developed they will be reviewed by the appropriate SMEs and approved by operations management. Upon approval, the ISV scenarios and test plan will be available for review or audit by the NRC sufficiently before the conduct of ISV so that comments or concerns can be adequately addressed prior to commencing ISV. This section describes characteristics of the test design important to supporting ISV validity. 4.6.1 Scenario Sequencing Integrated System Validation: Methodology and Review Criteria, NUREG/CR-6393 (Reference 8.1.2), is employed as the standard for selection of crew and scenario order. ISV scenarios contain variations of normal operation and abnormal events and are sequenced to ensure that operating crews are not expecting events and actions at the same time during each scenario. The scenario performance sequence is developed using the following guidance: © Copyright 2019 by NuScale Power, LLC 28

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

Equalize the opportunity for testing among all participants.
Vary the types of scenarios within the sequence; such that all are not easy at first and then progress to hard.
A minimum of two operating crews perform each scenario.

At least three participant crews will be assembled but only two of those three will be scheduled to perform any one scenario. ((

                                                               }}2(a),(c) 4.6.2    Test Procedures Prior to ISV, detailed test procedures are prepared to manage tests, assure consistency, control test bias, support repeatable results, and focus the test on the specific scenario objectives. The test observers/administrators use the test procedures to set up each scenario, manage the scenario, and analyze the test results. Scenario developers use test procedures to build the scenario set.

ISV test procedures are designed to minimize the introduction of bias by both observer/administrators and operating crews. A standardized scenario template is part of the test procedure. Test procedures include © Copyright 2019 by NuScale Power, LLC 29

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

scenario order for each crew and order of crews when running a single scenario multiple times
detailed and standardized instructions for briefing the test participants before each scenario
specific instructions and criteria for observer/administrators on conduct of scenarios
scripted questions and responses for observer and administrators acting as plant staff during the scenario
guidance on when and how to interact with the operating crew when the test bed encounters difficulties
specification of unique data to be collected and stored (including what, when, and how) (Section 4.5)
guidance for documenting o operating crews and scenario details o deviations from the test procedure, test difficulties, significant unusual events o collected plant raw data o observer and administrator notes o post-scenario and final debriefing notes o crew questionnaires o observer and administrator questionnaires o observer and administrator consensus notes o video and audio recordings o human engineering discrepancies
post-testing instructions for each operating crew that instruct them not to discuss the scenarios and HSI with others 4.6.3 Training Test Personnel Prior to starting ISV, observer and administrators are trained and qualified on NuScale plant systems, the HSI, and ISV test procedures. Training consists of both classroom and test bed time. Training goals include
assuring familiarity with test procedures and scenarios
reduction of bias and errors that may be introduced by the observers and administrators due to test-based learning, failure to follow the test procedure, or incorrect interaction with the operating crew
use of the test procedure

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

documentation needs for each test, including o where the test did not follow the scenario o problems that occur during testing, even if they were due to an oversight or error of those conducting the test
the necessity of limiting observer and administrator interaction with test personnel to that which is in the scenario description
how to conduct post-scenario debriefings
familiarity with HFE data collection tools and techniques
familiarity with observation techniques, goals, and responsibilities specific to each observers role 4.6.4 Training Participants Test participants undergo training similar to that which plant operators receive including conduct of operations, plant systems, HSI, plant events, and operating procedures. Test participants are not trained specifically on the scenarios in which they will participate.

To assure near-asymptotic performance and a consistent level of proficiency between individuals making up the operating crews, only participants who have successfully completed the training program and have reached an acceptable level of proficiency are considered to be qualified for operating crew assignment. 4.6.5 Pilot Testing A test operating crew, which does not participate in ISV, conducts a pilot test (a pre-validation test) to

assess the adequacy of test design, performance measures, and data-collection methods
give the observers and administrators experience in running the test
ensure that the ISV runs smoothly and correctly 4.7 Data Analysis Test data is analyzed using both quantitative and qualitative methods. The analysis identifies the relationship between the observed and measured performance and the established acceptance criteria described in Section 4.5.2.

Data is collected from multiple sources including crew debriefs, observer debriefs, NASA TLX questionnaires, Situational Awareness questionnaires, and management observations. The data is collected and added to a database where an HFE Subject Matter Expert (SME) and an Operations SME bin and code the performance data and then independently identify significant issues and trends within the data. This analysis compares and contrasts data sources, data across crews, data across trials, and data © Copyright 2019 by NuScale Power, LLC 31

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 across scenarios. The HFE and Operations SMEs then collaborate on trending results and Human Engineering Discrepancy (HED) identification. ((

                                                                                    }}2(a),(c) 4.8      Validation Conclusions ISV conclusions are based on

a comprehensive testing program performed by an independent ISV team using test procedures covering the scope described above
a high-fidelity test platform representative of the actual system, model, and HSI in aspects important to the integrated systems performance; variable aspects of the integrated system are adequately sampled
acceptance criteria are measurable, reflect good operational practices, and are representative of important aspects of performance

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

test design minimizes bias or confounding effects so as not to affect the validity of the results
statistical conclusions, where possible, are based on convergence of multiple measures
specific pass and fail performance criteria documented as HEDs also identify the extent of the issue ISV conclusions documented in the V&V results summary report include
the statistical and logical bases for determining that performance of the integrated system is acceptable
the limitations in identifying possible effects on validation conclusions and that the impact on the design integration HFE program element is considered, including o aspects of the tests not well controlled o potential differences between the test situation and actual operations such as the absence of productivity-safety conflicts o differences between test platform design and the as-built NuScale plant

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 5.0 Human Engineering Discrepancy Resolution Human engineering discrepancies (HEDs) are identified, documented, and resolved throughout the verification and validation process. NuScale begins to record HEDs after the completion of staffing plan validation. HEDs may not always be resolved; HEDs may be found acceptable after an evaluation in the context of the integrated design. The basis for a decision for accepting an HED without change in the integrated design is documented. It may be based on accepted HFE practices, current published HFE literature, trade-off studies, tests, or engineering evaluations. HEDs are identified in the V&V process during

task support verification (Section 3.2)
HFE design verification (Section 3.3)
ISV (Section 4.0)

HFE issues and HEDs are identified and tracked in the HFEITS database. The HFEITS database is available to any member of the HFE team and identification of issues is part of the NuScale corporate culture. The HFEITS database is maintained until fuel load. A sampling of HEDs found during the V&V process will be discussed in the V&V RSR HED evaluation documentation section and include information on the potential cumulative effects of HEDs observed and samples of HEDs which may have shown an indication of broader issues seen during testing. 5.1 HED Design Solution Implementation During ISV testing, HEDs are analyzed by the HFEITS team for priority selection and design category placement (e.g., HSI or simulator). Once the HED has been received, a discrepancy entry is created in the HFEITS database and the HED is prioritized as Priority 1, Priority 2, or Priority 3 HEDs according to their importance as follows:

Priority 1 HEDs have a potential direct or indirect impact on plant safety and are resolved prior to submittal of the V&V Results Summary Report. HEDs initiated as a result of a performance measure not being met (pass or fail performance measures) are Priority 1 HEDs. Cross-cutting issues determined through HED or performance measure analyses are also Priority 1 HEDs due to their global impact on the HSI design performance.
Priority 2 HEDs have a direct or indirect impact on plant performance and operability and are resolved before turning over HFE program responsibilities to a licensee.
Priority 3 HEDs are those that do not fall into Priority 1 or Priority 2. Priority 3 HEDs are resolved in accordance with QA policy-related programs and processes.

The HED is then routed to the appropriate group for resolution. HEDs related to the HSI are sent to the HFE design team, and HEDs related to simulator modeling are sent to the simulator review board. It is possible for HEDs to be routed to both groups. © Copyright 2019 by NuScale Power, LLC 34

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 The HED is then resolved, and the discrepancy entry closed. The HED resolution is reviewed for final closure in the HFEITS database by an HFE Review committee. The HED resolution process is depicted in Figure 5-1. Figure 5-1. Human engineering discrepancy resolution process 5.2 Human Engineering Discrepancy Analysis HFE V&V HEDs are categorized based on their principal impact on

personnel tasks and functions
plant systems
human-system interface feature
individual HSI component
operating procedure The categorization is documented in the HFEITS database.

Extent of condition and causal effect across the various HSI design features and functions are assessed as part of the HED process and documented in the HFEITS database. Extent of condition determination considers © Copyright 2019 by NuScale Power, LLC 35

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5

cumulative or combined effects of multiple HEDs
human engineering discrepancies that may represent a broader issue

((

                                            }}2(a),(c)

The broad-reaching testing and number of performance measures to be evaluated limit the ability to perform statistical analyses. Testing of multiple scenarios with multiple crews (generally, each crew will develop a different strategy) makes it impractical to make conclusions based on performance of the population or deviations from a norm. Therefore, observer and administrators, test participants, and the Validation Team evaluate any instance where a performance measure is not met to determine causal factors.

Design-related deficiencies determined for alarms, controls, indications, and procedures are documented in an HED. Any previous HFE program element may need to be evaluated to resolve the deficiency. The HSI design is not considered validated until an HED initiated by pass/fail measures as a result of ISV is resolved.
Test-related deficiencies are documented in the HFEITS and may result in changes to the test procedure or scenario definition.

((

                                 }}2(a),(c)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                                                                                         }}2(a),(c)

Data and data-analysis tools (e.g., equations, measures, spreadsheets, expert opinions, resulting HEDs) are documented for subsequent audit and application during design integration and/or human performance monitoring HFE program elements. Individual HFEITS items are maintained as auditable records in the HFEITS database. 5.3 Design Solution Testing Generally, design solutions will be verified to be acceptable using the same V&V method that originally detected the issue. For example, if an HED-1 is identified during performance of an ISV scenario, a similar scenario would be run to verify the solution was acceptable. Because the impact of design solutions vary widely this general practice may be adjusted using engineering judgment to ensure a thorough and appropriate test is conducted. The following elements are considered when making this judgment:

number of procedures affected
number of HSIs affected
complexity of the condition under which the design solution is used
uniqueness of the design solution

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 6.0 Verification & Validation Results Summary Report Following completion of all V&V activities, the results will be compiled in an RSR. The RSR will contain

a matrix of HFE V&V team participants and roles
human factors engineering V&V results overview and principal findings from design verification
a sampling of priority 1 HEDs generated from the V&V, the analyses associated with these HEDs, and their resolutions
human factors engineering V&V execution results o verification a description of the application of the verification program verification results based on TA verification results based on the HSI design style guide discussion of HEDs that resulted from the verification, extent of condition, resolution, and any subsequent HSI design changes made prior to validation verification test procedures verification procedure and analysis tools used to draw conclusions and provide assurance that selected scenarios are representative of expected operational conditions (tools may include tables or checklists) o validation a description of the application of the validation program validation test procedures integrated system validation procedure, including scenarios a detailed description of the specific scenario sets used in testing including:

test instructions, data collection instruments, SOC versus scenario comparison table, and scenario identification summary table data analysis results and validation conclusions, as compared to the minimum set of test objectives a discussion of pass and fail HEDs that resulted from the validation, extent of condition, resolution, and any subsequent HSI design changes, analyses, or retest a discussion of performance improvement measures a discussion of validation results and conclusions that pass/fail criteria have been met © Copyright 2019 by NuScale Power, LLC 38

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 7.0 NUREG-0711 Conformance Evaluation Table 7-1 indicates where each NUREG-0711, Revision 3 criterion is addressed in this IP. Table 7-1. Conformance with NUREG-0711 HFE V&V IP Section Review Criteria No. and paragraph 11.4.1.1 Sampling Dimensions Section 2.1, bullet 1 The following sampling dimensions are addressed below: Plant conditions, personnel tasks, and situational factors known to challenge personnel performance. (1) The applicant should include the following plant conditions:

normal operational events including plant startup, shutdown or refueling, and significant changes in operating power
I&C and HSI failures and degraded conditions that encompass:

         - The I&C system, including the sensor, monitoring, automation and control, and communications subsystems; [e.g., safety-related system logic and control unit, fault tolerant controller, local "field unit" for multiplexer (MUX) system, MUX controller, and a break in MUX line]
         - common cause failure of the I&C system during a design basis accident (as defined by BTP 7-19)
         - HSIs including, loss of processing or display capabilities for alarms, displays, controls, and computer-based procedures

transients and accidents, such as:

         - transients (e.g., turbine trip, loss of off-site power, station blackout, loss of all feedwater, loss of service water, loss of power to selected buses or MCR power supplies, and safety and relief valve transients)
         - accidents (e.g., main-steam-line break, positive reactivity addition, control rod insertion at power, anticipated transient without scram, and various-sized loss-of coolant accidents)
         - reactor shutdown and cooldown using the remote shutdown system
         - reasonable, risk-significant, beyond-design-basis events that should be determined from the plant-specific PRA

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (2) The applicant should include the following types of personnel tasks: Section 2.1, bullet 2

Important HAs, Systems, and Accident Sequences - The sample should include all important HAs, as determined in Section 7.

Additional factors that contribute highly to risk, as defined by the PRA, also should be sampled:

         - dominant accident sequences
         - dominant systems (selected through PRA importance measures, such as Risk Achievement Worth or Risk Reduction Worth)

Manual Initiation of Protective Actions - The sample should include manual system level actuation of critical safety functions.
Automatic System Monitoring - The sample should include situations in which humans must monitor a risk-important automatic system.
OER-Identified Problematic Tasks - The sample should include all personnel tasks identified as problematic during the applicant's review of operating experience.
Range of Procedure Guided Tasks -The sample should include tasks that are well defined by procedures. Personnel should be able to understand and execute the specified steps as part of their rule-based decision-making. Regulatory Guide 1.33, Appendix A, contains several categories of "typical safety-related activities that should be covered by written procedures." The sample should include appropriate procedures in each category:

         - administrative procedures
         - general plant operating procedures
         - procedures for startup, operation, and shutdown of safety-related systems
         - procedures for abnormal, off-normal, and alarm conditions
         - procedures for combating emergencies and other significant events (e.g., reactor accidents, and declaration of emergency-action levels)
         - procedures for controlling radioactivity
         - procedures for controlling measuring and test equipment and for surveillance tests, procedures, and calibration
         - procedures for performing maintenance
         - chemistry and radiochemical control procedures

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph

Range of Knowledge-Based Tasks - The sample should include Section 2.1, bullet 2 tasks that are not well defined by detailed procedures.

Additional Information: A situation may demand knowledge-based decision-making if the procedural rules do not fully address the problem, or when the selection of an appropriate rule is unclear. An example in a pressurized water reactor plant may be the difficulty in diagnosing a steam generator tube rupture (SGTR) with a failure of radiation monitors on the plants secondary side. This happens because (1) there is no main indication of the rupture (the presence of radiation in secondary side), and (2) the other effects of the rupture (i.e., slight changes in pressures and levels on the primary and secondary sides) may be attributed to other causes. While the operators may use procedures to treat the symptoms of the event, the determination that the cause is a SGTR may call for a situational assessment based on an understanding of the plant's design and the possible combinations of failures that entail the observed symptoms. Errors in rule-based decision-making result from selecting the wrong rule, or incorrectly applying a rule. Errors in knowledge-based decision-making result from mistakes in higher-level cognitive functions, such as judgment, planning, and analysis. The latter are more likely to occur in complex failure events wherein the symptoms do not resemble the typical case, and thus, are not amenable to pre-established rules.

Range of Human Cognitive Activities - The sample should include the range of cognitive activities that personnel perform, including:

         - detecting and monitoring (e.g., of critical safety-function threats)
         - situation assessment (e.g., interpreting alarms and displays to diagnose faults in plant processes and in automated control and safety systems)
         - planning responses (e.g., evaluating alternatives to recover from plant failures) response implementation (e.g., in-the-loop control of plant systems, assuming manual control from automatic control systems, and carrying out complicated control actions)
         - obtaining feedback (e.g., feedback of the success of actions taken)

Range of Human Interactions - The sample should include the range of interactions among plant personnel, including tasks performed independently by individual crew members, and those undertaken by a team of crew members. These interactions among plant personnel should include interactions between:

         - main control room operators (e.g., operations, shift turnover walkdowns)
         - main control room operators with auxiliary operators and other plant personnel performing tasks locally (e.g. , maintenance or I&C technicians, chemistry technicians)
         - main control room operators and the TSC and the EOF
         - main control room operators with plant management, the NRC, and other outside organizations

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (3) The applicant should include the following situational factors or error- Section 2.1, bullet 3 forcing contexts known to challenge human performance. It also should include situations specifically designed to create human errors to assess the systems error tolerance, and the ability of personnel to recover from any errors, should these occur, for example:

High-Workload Situations - The sample should include situations where variations in human performance due to high workload and multitasking situations can be assessed.
Varying-Workload Situations - The sample should include situations wherein variations in human performance due to workload transitions can be determined. These include conditions where there is (1) a sudden increase in the number of signals that must be detected and processed after a period in which signals were infrequent, and (2) a rapid reduction in the need for detecting signals and processing demands following a time of high sustained task-demand.
Fatigue Situations - To the extent possible, the sample should include situations that may be associated with fatigue, such as work on backshifts and tasks performed frequently with repetitive actions, such as repeated inputs to a touch screen during plant operations or pulling rods.
Environmental Factors - To the extent possible, the sample should include environmental conditions that may cause human performance to vary, e.g., poor lighting, extreme temperatures, high noise, and simulated radiological contamination.

11.4.1.2 Identification of Scenarios Section 2.2, all (1) The applicant should combine the results of the sampling to identify a set of V&V scenarios to guide subsequent analyses. Additional Information: A given scenario may combine many of the characteristics identified by sampling of operational conditions. (2) The applicant should not bias the scenarios by overly representing Section 2.2, all the following:

scenarios for which only positive outcomes are expected
scenarios that, for ISV, are relatively easy to conduct (i.e., scenarios should not be avoided simply because they are demanding to set up and run on a simulator)
scenarios that, for ISV, are familiar and well structured (e.g., which address familiar systems and failure modes that are highly compatible with plant procedures, such as textbook design-basis accidents)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph 11.4.1.3 Scenario Definition Section 2.3, all (1) The applicant should identify operational conditions and scenarios to be used for HSI Task Support Verification, Design Verification, and ISV. The applicant should develop detailed scenarios suitable for use on a full-scope simulator. The level of detail should be comparable to what one would include in a test plan. For each one, the following information should be defined to reasonably assure that important dimensions of performance are addressed, and to allow the scenarios to be accurately and consistently presented for repeated trials:

a description of the scenario and any pertinent prior history necessary for personnel to understand the state of the plant at the start-up of the scenario
specific initial conditions (a precise definition of the plants functions, processes, systems, component conditions, and performance parameters, e.g., similar to that at shift turnover)
events (e.g., failures) that will occur during the scenario and their initiating conditions, e.g., based on time, or a value of a specific parameter
precise definition of workplace factors, (e.g., environmental conditions, such as low levels of illumination)
needs for task support (e.g., procedures and technical specifications)
staffing level
details of communication content between control room personnel and remote personnel (e.g., load dispatcher via telephone)
scripted responses for test personnel who will act as plant personnel in the test scenarios Additional Information: Test personnel act as surrogates for personnel outside the control room. To the greatest extent possible, prepare responses to questions that may be asked by operators communicating with the personnel outside the control room. There are limits to the ability to preplan communications because personnel may ask unanticipated questions or make unforeseen requests. However, efforts should be made to detail what information personnel outside the control room can provide, and script the responses to likely questions.
the precise specification of what, when, and how data are to be collected and stored (including videotaping, questionnaires, and rating-scale administrations)
precise specifications on simulator set up
specific criteria for terminating the scenario

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (2) The applicants scenarios should realistically replicate operator Section 2.3, all tasks in the tests; then, the findings from the test can be generalized to the plants actual operations. (3) When the applicants scenarios include work associated with operations remote from the main control room, the effects on personnel performance due to potentially harsh environments (e.g., high radiation) should be realistically simulated (e.g., additional time to don protective clothing, and access radiologically controlled areas). 11.4.1.4 Additional Considerations for Reviewing the HFE Aspects of N/A Plant Modifications In addition to any of the criteria above that relate to the modification being reviewed, the applicant should address the following considerations. (1) The applicants operational conditions should reflect tasks that involve a modification, rather than the entire range of topics discussed in Section 11.4.1. (2) For ISV, the applicants operational conditions should encompass the transfer of learning effects on personnel performance when modifying an old HSI or procedure. Additional Information: Negative transfer of learning may occur when the new and old components are different and impose different demands on personnel. (3) For ISV, when both old and new versions of the same HSIs are permanently present in the HSI but with different means of presentation and methods of operation, then the applicants evaluations should reasonably assure that personnel can alternate their use of these HSIs without degrading performance. (4) Where old HSIs are to be deactivated but left in place in the HSI, the applicant should identify conditions for an ISV that would test the potential for their interfering with tasks. Additional Information: For example, the presence of deactivated HSIs may cause visual clutter that interferes with the ability of personnel to locate and use other HSIs. 11.4.2 Design Verification Review Criteria Section 3.1.1, all (1) 11.4.2.1 HSI Inventory and Characterization (1) Scope - The applicant should develop an inventory of all HSIs that personnel require to complete the tasks covered in the validation scenarios that were identified by the applicants Sampling of Operational Conditions. The inventory should include aspects of the HSI used for managing the interface, such as navigation and retrieving displays, as well as those that control the plant. © Copyright 2019 by NuScale Power, LLC 44

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (2) HSI Characterization - The applicants inventory should describe the Section 3.1.2, all characteristics of each HSI within the scope of the verification. The following is a minimal set of information for this characterization:

a unique identification code number or name
associated plant system and subsystem
associated personnel functions and tasks
type of HSI, e.g.,

         - computer-based control (e.g., touch screen or cursor-operated button and keyboard input)
         - hardwired control (e.g., J-handle controller, button, and automatic controller)
         - computer-based display (e.g., digital value and analog representation)
         - hardwired display (e.g., dial, gauge, and strip-chart recorder)

display characteristics and functionality [e.g., plant variables/parameters, units of measure, accuracy of variable/parameter, precision of display, dynamic response, and display format (e.g., bar chart or trend plot)]
control characteristics and functionality [e.g., continuous versus discrete settings, number and type of control modes, accuracy, precision, dynamic response, and control format (method of input)]
user-system interaction and dialog types (e.g., navigation aids and menus)
location in data-management system (e.g., identification code for information display screen)
physical location in the HSI (e.g., control panel section), if applicable The applicant should include photographs, copies of display screens, or similar samples of HSIs in the HSI inventory and characterization.

(3) Inventory Verification - The applicant should verify the inventory Section 3.1.3, all description of HSIs to ensure that it accurately reflects their current state. 11.4.2.2 HSI Task Support Verification Section 3.2 Section 3.2.1, all HSI Task Support Verification addresses the availability of items needed to support task requirements. As stated in Section 11.2, the objective of the HSI Task Support Verification review is to ensure that the applicant verified that the HSI provides the needed alarms, information, controls, and task support for personnel to perform their tasks, defined by the task analysis. (1) Verification Criteria - The applicant should base the HSI task support criteria on the alarms, controls, displays, and task support needed by personnel to complete their tasks as identified by the applicants task analysis. (2) General Methodology - The applicant should compare the HSIs and Section 3.2.2, all their characteristics (as defined in the HSI inventory and characterization) to the needs of personnel identified in the task analysis for the defined sampling of operational conditions, noted in Section 11.4.1. (3) HED Identification - The applicant should identify and document an Section 5.1 all HED when: © Copyright 2019 by NuScale Power, LLC 45

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph

An HSI needed for task performance (e.g., a necessary control or display) is unavailable.
HSI characteristics do not match the requirements of the personnel task (e.g., a display may show the needed plant parameter but not within the range or precision needed for the task).
HSIs are available that are not needed for any task.

Additional Information: Unnecessary HSIs introduce clutter, and can distract personnel from selecting the appropriate ones. It is important to verify that the HSI is unnecessary. Appropriate ones may not appear to be needed with personnel tasks for the following reasons:

The HSI is essential for a task that the task analysis did not address (i.e., it was not within the scope of the design review).
The task analysis was incomplete, overlooking the need for the HSI.
The HSI only partially meets the established requirements for the personnel task.

(4) HED Documentation - The applicant should document HEDs to Section 5.2, all identify the HSI, the tasks affected, and the basis for the deficiency (what aspect of the HSI was identified as not meeting task requirements). Additional Information: The analysis and correction of HEDs is detailed in Section 11.4.4, Human Engineering Discrepancy Resolution Review Criteria. © Copyright 2019 by NuScale Power, LLC 46

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (5) Additional Methodology Considerations for Plant Modifications - In N/A. addition to any of the criteria above that relate to the modification being reviewed, the applicant should address the following considerations:

HSI Task Support Verification should address all aspects of HSIs described above related to the modification. For modifications to plant systems that do not include modifications of the HSIs, verification of task support should highlight any new demands for monitoring and control, and assess whether the existing HSI design adequately addresses them.
HSI Task Support Verification should cover configurations in the modification in which old HSIs are deactivated permanently, but not removed (e.g., abandoned in place). Criterion 4 in this subsection states that the HSIs should not contain any information, displays, or controls that do not support personnel tasks. This verification should identify deactivated HSIs that might negatively affect personnel performance, such as obstructing the view of important information or adding visual clutter that could interfere with monitoring. The applicant should identify deactivated HSIs requiring further evaluation through HFE design verification or ISV.
HSI Task Support Verification should address the temporary configurations of the HSIs and plant systems that may be created when establishing the modification, and so used by operations and maintenance personnel when the plant is not shutdown. These configurations may include:

         - the use of HSIs that differ from the intended final design
         - combinations of HSIs and system configurations that differ from both the original design and the intended final one For each temporary HSI configuration, the task requirements of personnel should be identified and compared to the information and control capabilities available.

Additional Information: For example, if a temporary configuration of plant systems introduces special monitoring requirements, the HSIs should provide the necessary information. © Copyright 2019 by NuScale Power, LLC 47

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph 11.4.2.3 HFE Design Verification Section 3.3 HFE Design Verification addresses the suitability of the HSI with regard to Section 3.3.1, all human capabilities and limitations. As stated in Section 11.2, the objective of the HFE Design Verification review is to evaluate the applicants verification that the design of the HSIs conforms to HFE guidelines. (1) Verification Criteria - The applicant should base the criteria used for HFE Design Verification on HFE guidelines. Additional Information: The choice of guidelines used in this verification depends upon whether the applicant developed a design-specific style guide. The acceptability of the style guide used by the applicant should be reviewed by the NRC staff using the review guidance in Section 8.4.3, HFE Design Guidance for HSIs. Using an NRC-reviewed style guide affords the criteria for verifying the HFE design. When no style guide is available, the guidelines in NUREG-0700 can be used by the applicant for this purpose. However, because not all of the guidelines therein will be applicable to each review, the applicant should select those based on the characteristics of the HSIs being evaluated. Applicants should identify a subset of guidelines appropriate to a specific design based on the HSI characterization. (2) General Methodology - The applicants HFE Design Verification Section 3.3.2, all methodology should include the following:

Procedures for comparing the characteristics of the HSIs with HFE guidelines for (1) the defined sampling of operational conditions, as noted in Section 11.4.1, and (2) the general environment in which HSIs are sited, including workstations, control rooms, and environmental characteristics (e.g., lighting and noise).

Additional Information: A single guideline may apply to many HSIs. By verifying all HSIs within the scenarios defined in Section 11.4.1, the consistency of applying a guideline across multiple HSIs can be assessed.

Procedures for determining for each guideline whether the HSI is "acceptable" or "discrepant." If discrepant, it should be designated as an HED, tracked, and evaluated (see Sections 2.4.4 and 11.4.4).

Additional Information: A judgment that an HSI is acceptable should be made only if compliance is total, i.e., only if every instance of the item is fully consistent with the criteria established by the HFE guidelines. If there is any noncompliance, full or partial, then an evaluation of discrepant should be given, and a notation made as to where it occurs.

Procedures for evaluating whether an HED is a potential indicator of additional issues.

Additional Information: For example, identifying an inappropriate format for presenting data on an individual display should be considered a potential sign that other display formats might be used incorrectly, or that the observed format is employed inappropriately elsewhere. Then, the sampling strategy should be modified to encompass other display formats. In some cases, discovering these discrepancies will warrant further review in the identified areas of concern. © Copyright 2019 by NuScale Power, LLC 48

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (3) HED Identification - The applicant should identify an HED when a Section 5.0 characteristic of the HSI is "discrepant" from a guideline. (4) HED Documentation - The applicant should document HEDs in terms of the HSI involved, and how its characteristics depart from a particular guideline. Additional Information: The analysis and correction of HEDs is addressed in Section 11.4.4, Human Engineering Discrepancy Resolution Review Criteria. (5) Additional Considerations for Reviewing the HFE Aspects of Plant N/A Modifications - In addition to any of the criteria above that relate to the modification being reviewed, the applicant should address the following considerations:

The scope of HFE design verification may be restricted to the modified HSIs and their interactions with the rest of the HSIs.
When both old and new versions of similar HSIs are available, this verification should offer reasonable assurance that their means of presentation and methods of operation are compatible, such that personnel performance will not be impaired when alternating the use of each one.
HEDs should be identified for the following:

         - failure to meet "personnel-identified" functionality in addition to that specified by system designers. When a digital system replaces an existing system, it is important to ensure that all operational uses of the former system were addressed, even those that were not intended in the original design. The replacement system's design should consider the ways in which personnel actually used the former system
         - poor integration with the rest of the HSI
         - poor integration with procedures and training

Temporary configurations of the HSIs and plant systems that operations and maintenance personnel may use when the plant is not shutdown, should be reviewed to verify that their design is consistent with the principles of good HFE design, including consistency with the rest of the HSIs.

11.4.3 Integrated System Validation Section 4.0 11.4.3.1 Validation Team Section 4.1, all (1) The applicant should describe how the team performing the validation has independence from the personnel responsible for the actual design. Additional Information: The members of the Validation Team should have no responsibility for the design; i.e., they should never have been part of the design team. While they may work for the same organization, their responsibilities must not include contributions to the design, other than validating it. 11.4.3.2 Test Objectives Section 4.2, all (1) The applicant should develop detailed test objectives to provide evidence that the integrated system adequately supports plant personnel in safely operating the plant, to include the following considerations: © Copyright 2019 by NuScale Power, LLC 49

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph

Validate the acceptability of the shift staffing level(s), the assignment of tasks to crew members, and crew coordination within the control room, between the control room and local control stations and support centers, and with individuals performing tasks locally. This should encompass validating minimum shift staffing levels, nominal levels, maximum levels, and shift turnover (see Section 6 for definitions).
Validate that the design has adequate capability for alerting, informing controlling, and feedback such that personnel tasks are successfully completed during normal plant evolutions, transients, design-basis accidents, and also under selected, risk significant events beyond-design basis, as defined by sampling operational conditions.
Validate that specific personnel tasks can be accomplished within the time and performance criteria, with effective situational awareness, and acceptable workload levels that balance vigilance and personnel burden.
Validate that the HSIs minimize personnel error and assure error detection and recovery capability when errors occur.
Validate the assumptions about performance on important HAs.

Additional Information: For example, the HRA within the plant PRA contains several assumptions regarding the performance of risk-important HAs. These assumptions should be validated for dominant sequences, such as decision-making and diagnosis strategies, and also for the human actions. This process should be completed before the final quantification stage of the PRA.

Validate that the personnel can effectively transition between the HSIs and procedures in accomplishing their tasks, and that interface management tasks, such as display configuration and navigation, are not a distraction or an undue burden.

(2) Additional Considerations for Reviewing the HFE Aspects of Plant N/A Modifications - In addition to any of the criteria above that relate to the modification being reviewed, the tests objectives and scenarios should be developed to encompass aspects of performance affected by the modified design (even when the HSIs are not modified), including personnel tasks. 11.4.3.3 Validation Test beds Section 4.3 A test bed is the HSI representation used to perform validation evaluations. Section 4.3.1, all One approach an applicant can use to acceptably meet criteria 1 through 7 in this section is to use a test bed that is compliant with "Nuclear Power Plant Simulators for Use in Operator Training" (ANS, 2009). (1) Interface Completeness - The applicants test bed should represent completely the integrated system. It should include HSIs and procedures not specifically required in the test scenarios. Additional Information: Adjacent controls and displays may affect the ways in which personnel use those addressed by a particular validation scenario. (2) Interface Physical Fidelity - The test beds HSIs and procedures Section 4.3.2, all should be represented with high physical fidelity to the reference design, including the presentation of alarms, displays, controls, job © Copyright 2019 by NuScale Power, LLC 50

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph aids, procedures, communications equipment, interface management tools, layout, and spatial relationships. (3) Interface Functional Fidelity - The test beds HSI and procedure Section 4.3.3, all functionality should be represented with high fidelity to the reference design. All HSI functions should be available. Additional Information: High fidelity covers the HSI modes of operation (i.e., the changes in functionality that can be invoked by personnel selecting them), or changes in plant states. (4) Environmental Fidelity - The test beds environmental fidelity should Section 4.3.4, all be represented with high physical fidelity to the reference design, including the expected levels of lighting, noise, temperature, and humidity. Thus, for example, the noise contributed by equipment, such as air-handling units, computers, and communications equipment should be represented in validation tests. (5) Data Completeness Fidelity - Information and data provided to Section 4.3.5, all personnel should completely represent the plants systems they monitor and control. (6) Data Content Fidelity - The test beds data content fidelity should be Section 4.3.6, all represented with high physical fidelity to the reference design. The presentation of information and controls should rest on an underlying model accurately mirroring the reference plant. The model should provide input to the HSI such that the information accurately matches that which is presented during operations. (7) Data Dynamics Fidelity - The test beds data dynamics fidelity Section 4.3.7, all should be represented with high fidelity to the reference design. The process model should be able to provide input to the HSI so that information flow and control responses occur accurately and within the correct response time; e.g., information should be sent to personnel with the same delays as occur in the plant. (8) For important HAs at complex HSIs remote from the main control Section 4.3.8, all room (e. g., a remote shutdown facility), where timely, precise actions are essential, the use of a simulator or mockup should be considered to verify that the requirements for human performance can be met. (For less important HAs, or for non-complex HSIs, human performance may be assessed on analysis, such as task analysis, rather than on simulations.) (9) The applicant should verify the conformance of the test bed to the Section 4.3.9, all test bed-required characteristics before validation tests are conducted. 11.4.3.4 Plant Personnel Section 4.4, all (1) Participants in the applicants validation tests should be representative of plant personnel who will interact with the HSI (e.g., licensed operators, rather than training personnel or engineers). (2) To properly account for human variability, the applicant should use a sample of participants that reflects the characteristics of the population from which it is drawn. Those characteristics expected to contribute to variations in system performance should be specifically identified; the sampling process should reasonably assure that the validation encompasses variation along that dimension. Determining representativeness should include considering the participants © Copyright 2019 by NuScale Power, LLC 51

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph license type and qualifications, skill/experience, age, and general demographics. (3) In selecting personnel for participating in the tests, the applicant should consider the minimum shift staffing levels, nominal levels, and maximum levels, including shift supervisors, reactor operators, shift technical advisors, etc. (4) The applicant should prevent bias in the sample of participants by avoiding the use of participants who:

are members of the design organization
participated in prior evaluations
were selected for some specific characteristic, such as crews identified as good performers or more experienced 11.4.3.5 Performance Measurement Section 4.5, all ISV employs a hierarchal set of performance measures including measures of plant performance, personnel task performance, situation awareness, cognitive workload, and anthropometric/physiological factors. Errors of omission and commission also are identified. A hierarchal set of measures provides sufficient information to validate the integrated system design and affords a basis to evaluate deficiencies in performance and thereby identify needed improvements. Pass/fail measures are those used to determine whether the design is or is not validated. Diagnostic measures are used to better understand personnel performance and to facilitate the analyses of errors and HEDs.

11.4.3.5.1 Types of Performance Measures Section 4.5.1.1, all; Appendix A (1) The applicant should identify the specific plant performance measures applicable to each ISV scenario.

Additional Information: They may address the performance of functions, systems, or component.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (2) The applicant should identify the primary task measures applicable Section 4.5.1.2, all; to each ISV scenario. Appendix A

For each scenario, the applicant should identify the primary tasks operators must perform to accomplish scenario goals, so that such measures can be developed.

Additional Information: The primary tasks are those involved in carrying out the functional role of the operator in supervising the plant; i.e., monitoring, detection, situation assessment, response planning, and response implementation. Primary tasks should be assessed at a level of detail appropriate to the tasks demands. For example, for some simple scenarios, measuring the time to complete a task may suffice. For complicated tasks, especially those described as knowledge-based, it may be appropriate to undertake a fine-grained analysis, such as identifying the tasks components, viz., seeking specific data, making decisions, taking actions, and obtaining feedback.

The measures chosen to evaluate personnel task performance should reflect those aspects of the task that are important to system performance, such as:

         - time
         - accuracy
         - frequency
         - amount achieved or accomplished
         - consumption or quantity used
         - subjective reports of participants
         - behavior categorization by observers

The analysis of primary tasks will support the identification of errors of omission (primary tasks not performed). Also, any actions and tasks that operators actually perform that deviate from the primary tasks should be identified and noted. These actions should be used to identify errors of commission.

(3) The applicant should identify the secondary task measures Section 4.5.1.2, applicable to each scenario. paragraph 3; Additional Information: Secondary tasks are those personnel must Appendix A perform when interfacing with the HSI, such as navigating through computer screens to find a needed display and to configure HSIs. The measurement of secondary task performance should reflect the demands of the detailed HSI implementation, e.g., time to configure a workstation, navigate between displays, and manipulate them (e.g., changing display type and scale settings). (4) The applicant should identify the measures of situation awareness Section 4.5.1.3, all; applicable to each scenario. Appendix A Additional Information: Situation awareness is the degree to which personnels perception of plant parameters and understanding of the plant's condition corresponds to its actual condition at any given time and influences predictions about future states. (5) The applicant should identify the workload measures obtained for Section 4.5.1, all; each scenario. Section 4.5.1.4; Additional Information: Workload is comprised of the physical, Appendix A cognitive, and other demands that tasks place on plant personnel. © Copyright 2019 by NuScale Power, LLC 53

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph The impact of one or many of these aspects of workload should be considered in the performance measures. (6) The applicant should identify the anthropometric and physiological Section 4.5.1.5, all; measures obtained for each scenario. Appendix A Additional Information: Anthropometric and physiological factors include such concerns as visibility of displays, accessibility of control devices, and ease of manipulating the control device. Many of these design aspects are assessed as part of verifying the HFEs design. Therefore, attention should focus on those areas of the design that only can be addressed by testing the integrated system, e.g., the ability of personnel effectively to use the various controls, displays, workstations, or consoles while performing their tasks. 11.4.3.5.2 Performance Measure Information and Validation Section 4.5.2, all; Criteria Section 4.5.2.1; (1) The applicant should describe the methods by which these Appendix A measures are obtained, e.g., by simulator data recording, participant questionnaires, or observation by subject-matter experts. (2) The applicant should specify when each measure is obtained Section 4.5.2.2, (recorded), such as continuously, at specific points during the paragraph 3 scenario, or after the scenario ends. (3) The applicant should describe the characteristics (see Table 11-1) of Section 4.5.2.2, Table the performance measures. 4-1; Appendix A Table 11-1 Characteristics of Performance Measures Characteristic Meaning Construct Validity A measure should represent accurately the aspect of performance it is intended to measure. Reliability A measure should be repeatable; i.e., same behavior measured in exactly the same way under identical circumstances should yield the same results. Sensitivity A measure's range (scale) and its frequency (how often data are collected) should be appropriate to that aspect of performance being assessed. Unobtrusiveness A measure should minimally alter the psychological or physical processes that are being investigated. Objectivity A measure should be based on easily observed phenomena. (4) The applicant should identify the specific criterion for each measure Section 4.5.2.2, Table used to judge the acceptability of performance and describe its 4-2; Appendix A basis. Additional Information: Table 11-2 describes the different bases for performance criteria. Table 11-2 Basis for Performance Criteria Criteria Basis Meaning Requirement The observed performance of the integrated system is compared with a quantified performance requirement; i.e., the requirements for the performance of systems, subsystems, and personnel are defined through engineering analyses. © Copyright 2019 by NuScale Power, LLC 54

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph Benchmark The observed performance of the integrated system is compared with a criterion established using a benchmark system, e.g., a current system is predefined as acceptable. Norm The observed performance of the integrated system is compared with a criterion using many predecessor systems (rather than a single benchmark system). Expert The observed performance of the integrated system is Judgment compared with a criterion established by subject-matter experts. (5) The applicant should identify whether each measure is a pass/fail Section 4.5.2.2, final one or a diagnostic one. paragraph; Appendix A 11.4.3.6 Test Design Section 4.6 11.4.3.6.1 Scenario Sequencing Section 4.6.1, all (1) The applicant should balance scenarios across crews to provide each crew with a similar, representative range of scenarios. Additional Information: Random assignment of scenarios to crews for ISV is undesirable. The value of using random assignment to control bias is effective only when the number of crews is quite large. (2) The applicant should balance the order of presentation of scenarios Section 4.6.2, bullet 1 to crews to provide reasonable assurance that the scenarios are not always presented in the same sequence (e.g., the easy scenario is not always used first). © Copyright 2019 by NuScale Power, LLC 55

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph 11.4.3.6.2 Test Procedures Section 4.6.2, all (1) The applicant should use detailed, unambiguous procedures to govern the conduct of the tests. These procedures should include the following:

the identification of which crews receive which scenarios, and the order in which they should be presented
detailed and standardized instructions for briefing the participants Additional Information: The type of instructions given to participants can affect their performance on a task. This source of bias is minimized by developing standard instructions.
specific directions for the testing personnel on conducting the test scenarios, as elaborated in Scenario Definition (Section 11.4.1.3)
guidance on when and how to interact with participants when difficulties occur in simulation or testing Additional Information: Even when a high-fidelity simulator is used, the participants may encounter artifacts of the test environment that detract from their performance of the tasks that are the focus of the evaluation. Guidance should be available to the test conductors to help resolve such conditions.
instructions on when and how to collect and store data. These instructions should stipulate which data are to be recorded by:

         - simulator computers
         - special-purpose instruments and devices for collecting data (such as situation awareness- and workload-questionnaires, or physiological measures)
         - video recorders (locations and views)
         - test personnel and subject-matter experts (such as via observational checklists)

procedures for documentation:

         - identifying and maintaining files of test records including details of the crew and scenarios
         - data collected
         - logs created by those who conducted the tests

The procedures should detail the types of information that should be logged (e.g., when the tests were performed, deviations from the test procedures and why they occurred, and any unusual events that may be important to understanding how a test was run or for interpreting the findings from it). The procedure also should state when the types of information should be recorded.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (2) The applicants test procedures should minimize the opportunity for Section 4.6.2, final bias in the test personnels expectations and in the participants paragraph responses. Additional Information: The expectancies of test personnel may introduce a bias if the expectations of the testers systematically influence the collection of data. Expectancies can influence performance in many ways (e.g., test personnel may, by giving subtle cues or communications, provide direction to participants, or they may tend to evaluate the performance of participants in ways that reflect more favorably upon the design than would an objective observer). Participant response bias means that the design of the test itself affects the data obtained from participants. It is not necessarily implied that a response bias represents any deliberate attempt by the participants to be untruthful. The test environment can influence participants in ways that have little to do with the tests objectives. Response bias can occur in four ways. First, participants may wish to influence outcomes and so be biased toward producing data consistent with their desired result. Second, participants may want to provide data that they think the test personnel want to obtain. Third, participants may try to figure out how performance should vary under different conditions, and then influence data to be consistent with such differences. Fourth, participants may want to excel because they know that they are being observed. See NUREG/CR 6393 (O'Hara et al., 1997) for additional information. 11.4.3.6.3 Training Test Personnel Section 4.6.3, all (1) The applicant should train test personnel (those who conduct or administer the validation tests) on the following:

the use and importance of test procedures
bias and errors that test personnel may introduce into the data through failures to follow test procedures accurately or to interact with participants properly
the importance of accurately documenting problems arising during testing, even if they were due to an oversight or error of those conducting the test 11.4.3.6.4 Training Participants Section 4.6.4, paragraph 1 (1) The applicants training of participants should be very similar to the training plant personnel receive. It should reasonably assure that the participants knowledge of the plants design, and operations, and the use of the HSIs and procedures represent that of experienced plant personnel. Participants should not be trained specifically to carry out the selected validation scenarios.

(2) To assure that the participants performance is representative of Section 4.6.4, plant personnel, the applicants training of participants should result paragraph 2 in near asymptotic performance (i.e., stable, not significantly changing from trial to trial) and should be tested for such before conducting the validation. © Copyright 2019 by NuScale Power, LLC 57

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph 11.4.3.6.5 Pilot Testing Section 4.6.5, all (1) The applicant should conduct a pilot study before the validation tests begin to offer an opportunity for the applicant to assess the adequacy of the test design, performance measures, and data-collection methods. (2) The applicant should not use participants in the pilot testing who will then be participants in the validation tests. 11.4.3.7 Data Analysis and HED Identification Section 5.0 (1) The applicant should use a combination of quantitative and qualitative methods to analyze data. The analysis should reveal the relationship between the observed performance and the established performance criteria. (2) The applicant should discuss the method by which data is analyzed Section 5.0 across trials, and include the criteria used to determine successful performance for a given scenario. (3) The applicant should evaluate the degree of convergence between Section 5.0 related measures (i.e., consistency between measures expected to assess the same aspect of performance). Additional Information: For example, if situation assessment is measured by both a participant questionnaire, and an observer rating scale, the results should be consistent with each other. If they do not converge, the reason for this should be identified. (4) When interpreting test results, the applicant should allow a margin Section 5.0 of error to reflect the fact that actual performance may be slightly more variable than observed validation-test performance. (5) The applicant should verify the correctness of the analyses of the Section 5.0 data. This verification should be done by individuals or groups other than those who performed the original analysis, but may be from the same organization. (6) The applicant should identify HEDs when the observed performance Section 5.0 does not meet the performance criteria. Additional Information: The analysis and correction of HEDs is addressed in Section 11.4.4, Human Engineering Discrepancy Resolution Review Criteria. (7) The applicant should resolve HEDs identified by pass/fail measures Section 5.0 before the design is accepted. © Copyright 2019 by NuScale Power, LLC 58

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph 11.4.3.8 Validation Conclusions Section 5.0 (1) The applicant should document the statistical and logical bases for determining that performance of the integrated system is, and will be acceptable. (2) The applicant should document the limitations in the validation tests, their possible effects on the conclusions of the validation, and their impact on implementing the design. Additional Information: Examples of possible limitations include:

aspects of the tests that were not well controlled
potential differences between the test situation and actual operations, such as the absence of productivity-safety conflicts
potential differences between the validated design and the as-built plant or system (if validation is directed to a plant under construction where such information is available, or to a new design using the validation findings from a predecessor)

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph 11.4.4 Human Engineering Discrepancy Resolution Review Criteria Section 5.0 (1) HED Analysis The applicants HED analyses should include the following:

Personnel Tasks and Functions - The impact of HEDs on personnel tasks and the functions supported by those tasks.

Additional Information: The potential effects of HEDs is determined, in part, by the importance of the personnel function to plant safety (e.g., consequences of failure), and their cumulative effect on personnel performance (e.g., degree of impairment and types of potential errors).

Plant Systems - The impact of HEDs on plant systems, considering the safety significance of that system(s), their effect on accident analyses, and their relationship to risk-significant sequences in the plants PRA.

Additional Information: The potential effects of these HEDs on the plants safety and personnel performance are determined, in part, by the safety significance of the plant system(s) related to the particular component.

Cumulative Effects of HEDs - The analysis of HEDs should identify the cumulative effects that multiple HEDs may have on plant safety and personnel performance.

Additional Information: Although an individual HED might not be considered sufficiently severe to warrant correction, the combined effect of several of them on a single aspect of the design could significantly degrade plant safety, and therefore, necessitate corrective action. Likewise, when a single plant system with multiple associated HEDs affects several HSIs, then their possible combined effect on the operation of that plant system should be considered.

HEDs as Indications of Broader Issues - As well as addressing specific HEDs, the applicants analysis should determine whether the HEDs point to potentially broader problems.

Additional Information: For example, identifying multiple HEDs associated with one particular aspect of the HSI design, such as the remote shutdown panel, also might suggest other problems with that aspect of the design, such as inconsistent use of design procedures and style guides. In some cases, findings from evaluating HEDs could warrant further review in the identified areas of concern, e.g., when multiple cases of mislabeling are found, the reviewers may wish to do a more complete examination of labeling. © Copyright 2019 by NuScale Power, LLC 60

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (2) Selection of HEDs to Correct Section 5.0, final paragraph The applicant should conduct an evaluation to identify which HEDs to correct. The evaluation should identify those HEDs that are acceptable as is (The Additional Information below provides examples). The remaining discrepancies should be denoted as HEDs to be addressed by the HED-resolution process. HEDs the applicant should correct are those with direct safety consequences, namely, those that could adversely impact personnel performance such that the margin of plant safety may be reduced below an acceptable level. Unacceptability is indicated by such conditions as violations of Technical Specification safety limits, operating limits, or limiting conditions for operations, or failing an ISV pass/fail criterion. HEDs with potential safety impact, not as severe as those described above, also should be corrected unless the applicant justifies leaving the condition as is. The applicant should correct HEDs that may adversely impact personnel performance in a way that has potential consequences to plant performance or SSC operability, and personnel performance or efficiency. This may include failing to meet personnel information needs or violating HFE guidelines for tasks associated with plant productivity, availability, and protecting investment. Additional Information: HEDs could be acceptable within the context of the fully integrated design. The technical basis for such a determination could include an analysis of recent research literature, current practices, tradeoff studies, or design engineering evaluations. (3) Development of Design Solutions Section 5.2, all Also described in The applicant should identify design solutions to correct HEDs. As Reference 8.2.1. part of the design solution, the application should evaluate the interrelationships of individual HEDs. Additional Information: HEDs should not be considered in isolation and to the extent possible, their potential interactions should be considered when developing and implementing solutions. For example, if the HSI for a single plant system is associated with many HEDs, then the set of design solutions should be coordinated to enhance overall performance and avoid incompatibilities between individual solutions. Similarly, if a single plant system is associated with multiple HSIs associated with HEDs, then the design of individual solutions should be harmonized so that the outcome enhances rather than detracts from that system's operation. Approaches that develop design solutions to some HEDs before all are identified in a particular V&V activity are acceptable provided that the potential interactions between HEDs are specifically considered before implementing the design solutions. © Copyright 2019 by NuScale Power, LLC 61

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 HFE V&V IP Section Review Criteria No. and paragraph (4) Design Solution Evaluation Section 5.2, all The applicant should evaluate design solutions to demonstrate the Also described in resolution of that HED and to ensure that new HEDs are not Reference 8.2.1. introduced. Generally, the evaluation should use the V&V method that originally detected the HED. Additional Information: For example, if the HED was identified using HFE Design Verification, then that verification should be employed to evaluate the solution. However, there may be reasons for documenting a satisfactory resolution using other methods. For example, if an aspect of the HSI was significantly changed from the resolution of multiple HEDs, the final HSI design may be validated to ensure that the net effect of all the changes is acceptable. (5) HED Evaluation Documentation Section 5.2 all The applicant should document each HED, including:

the basis for not correcting an HED
related personnel tasks and functions
related plant systems
cumulative effects of HEDs
HEDs as indications of broader issues Additional Information: Some, or all, of this documentation may be included in the issues tracking system (Section 2.4.4). Other information, such as cumulative effects or indications of broader issues, may be documented separately.

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 8.0 References 8.1 Source Documents 8.1.1 U.S. Nuclear Regulatory Commission, Human Factors Engineering Program Review Model, NUREG-0711, Rev. 3, November 2012. 8.1.2 U.S. Nuclear Regulatory Commission, Integrated System Validation: Methodology and Review Criteria, NUREG/CR-6393, January 1997. 8.2 Referenced Documents 8.2.1 Human Factors Engineering Program Management Plan, RP-0914-8534. 8.2.2 NuScale Power Small Modular Reactor Human System Interface Style Guide, ES-0304-1381. 8.2.3 Human Factors Engineering Staffing and Qualifications Results Summary Report, RP-0316-17617. © Copyright 2019 by NuScale Power, LLC 63

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                                                                                                }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 64

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                                                                                              }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 65

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                                                                                             }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 66

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                                                                                              }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 67

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                                                                                             }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 68

Human Factors Verification and Validation Implementation Plan RP-0914-8543-NP Rev. 5 ((

                                                                                            }}2(a),(c)
  © Copyright 2019 by NuScale Power, LLC 69

LO-0219-64513 : Affidavit of Thomas A. Bergman, AF-0219-64514 NuScale Power, LLC 1100 NE Circle Blvd., Suite 200 Corvallis, Oregon 97330 Office 541.360-0500 Fax 541.207.3928 www.nuscalepower.com

NuScale Power, LLC AFFIDAVIT of Thomas A. Bergman I, Thomas A. Bergman, state as follows: (1) I am the Vice President of Regulatory Affairs of NuScale Power, LLC (NuScale), and as such, I have been specifically delegated the function of reviewing the information described in this Affidavit that NuScale seeks to have withheld from public disclosure, and am authorized to apply for its withholding on behalf of NuScale. (2) I am knowledgeable of the criteria and procedures used by NuScale in designating information as a trade secret, privileged, or as confidential commercial or financial information. This request to withhold information from public disclosure is driven by one or more of the following: (a) The information requested to be withheld reveals distinguishing aspects of a process (or component, structure, tool, method, etc.) whose use by NuScale competitors, without a license from NuScale, would constitute a competitive economic disadvantage to NuScale. (b) The information requested to be withheld consists of supporting data, including test data, relative to a process (or component, structure, tool, method, etc.), and the application of the data secures a competitive economic advantage, as described more fully in paragraph 3 of this Affidavit. (c) Use by a competitor of the information requested to be withheld would reduce the competitors expenditure of resources, or improve its competitive position, in the design, manufacture, shipment, installation, assurance of quality, or licensing of a similar product. (d) The information requested to be withheld reveals cost or price information, production capabilities, budget levels, or commercial strategies of NuScale. (e) The information requested to be withheld consists of patentable ideas. (3) Public disclosure of the information sought to be withheld is likely to cause substantial harm to NuScales competitive position and foreclose or reduce the availability of profit-making opportunities. The accompanying report reveals distinguishing aspects about the method by which NuScale develops its human factors engineering verification and validation implementation plan. NuScale has performed significant research and evaluation to develop a basis for this method and has invested significant resources, including the expenditure of a considerable sum of money. The precise financial value of the information is difficult to quantify, but it is a key element of the design basis for a NuScale plant and, therefore, has substantial value to NuScale. If the information were disclosed to the public, NuScale's competitors would have access to the information without purchasing the right to use it or having been required to undertake a similar expenditure of resources. Such disclosure would constitute a misappropriation of NuScale's intellectual property, and would deprive NuScale of the opportunity to exercise its competitive advantage to seek an adequate return on its investment. (4) The information sought to be withheld is in the enclosed report entitled "Human Factors Engineering Verification and Validation Implementation Plan," RP-0914-8543-P, Revision 5. The enclosure contains the designation Proprietary" at the top of each page containing proprietary information. The information considered by NuScale to be proprietary is identified within double braces, "(( }}" in the document. (5) The basis for proposing that the information be withheld is that NuScale treats the information as a trade secret, privileged, or as confidential commercial or financial information. NuScale relies upon AF-0219-64514 Page 1 of 2

the exemption from disclosure set forth in the Freedom of Information Act ("FOIA"), 5 USC § 552(b )(4 ), as well as exemptions applicable to the NRC under 10 CFR § 2.390(a)(4) and 9.17(a)(4 ). (6) Pursuant to the provisions set forth in 10 CFR § 2.390(b)(4), the following is provided for consideration by the Commission in determining whether the information sought to be withheld from public disclosure should be withheld: (a) The information sought to be withheld is owned and has been held in confidence by NuScale. (b) The information is of a sort customarily held in confidence by NuScale and , to the best of my knowledge and belief, consistently has been held in confidence by NuScale. The procedure for approval of external release of such information typically requires review by the staff manager, project manager, chief technology officer or other equivalent authority, or the manager of the cognizant marketing function (or his delegate), for technical content, competitive effect, and determination of the accuracy of the proprietary designation. Disclosures outside NuScale are limited to regulatory bodies , customers and potential customers and their agents , suppliers , licensees , and others with a legitimate need for the information, and then only in accordance with appropriate regulatory provisions or contractual agreements to maintain confidentiality. (c) The information is being transmitted to and received by the NRC in confidence. (d) No public disclosure of the information has been made, and it is not available in public sources. All disclosures to third parties , including any required transmittals to NRC, have been made, or must be made, pursuant to regulatory provisions or contractual agreements that provide for maintenance of the information in confidence. (e) Public disclosure of the information is likely to cause substantial harm to the competitive position of NuScale, taking into account the value of the information to Nu Scale, the amount of effort and money expended by NuScale in developing the information, and the difficulty others would have in acquiring or duplicating the information. The information sought to be withheld is part of NuScale's technology that provides NuScale with a competitive advantage over other firms in the industry. NuScale has invested significant human and financial capital in developing this technology and Nu Scale believes it would be difficult for others to duplicate the technology without access to the information sought to be withheld. I declare under penalty of perjury that the foregoing is true and correct. Executed on April 29, 2019 AF-0219-64514 Page 2 of 2}}

ML19119A372

Text

SUBJECT:

REFERENCE:

Navigation menu

Search