ML18298A190

July-August 2018 NuScale Isv Audit Summary Report for the Regulatory Audit of NuScale Power, LLC, Design Certification Application Human Factors Engineering

ML18298A190
Person / Time
Site:	NuScale
Issue date:	11/14/2018
From:	Prosanta Chowdhury NRC/NRO/DLSE/LB1
To:	Samson Lee NRC/NRO/DLSE/LB1
Chowdhury P/NRO/415-1647
Shared Package
ML18298A189	List: ML18298A190
References
Download: ML18298A190 (9)

Text

November 14, 2018 MEMORANDUM TO: Samuel S. Lee, Chief Licensing Branch 1 Division of Licensing, Siting and Environmental Analysis Office of New Reactors FROM: Prosanta Chowdhury, Project Manager /RA/

Licensing Branch 1 Division of Licensing, Siting and Environmental Analysis Office of New Reactors

SUBJECT:

AUDIT

SUMMARY

REPORT FOR THE REGULATORY AUDIT OF NUSCALE POWER, LLC., DESIGN CERTIFICATION APPLICATION HUMAN FACTORS ENGINEERING From July 30, 2018, through August 2, 2018, and from August 27, 2018, through August 30, 2018, the U.S. Nuclear Regulatory Commission (NRC) staff conducted a regulatory audit of NuScale Power, LLC., Design Certification Application (DCA) Human Factors Engineering (HFE) topics.

The purpose of the audit was to confirm that the NuScale integrated system validation (ISV) testing was conducted using methods that are consistent with the methods described in the DCA and in regulatory guidance. The NRC staff conducted the audit at the applicants office in Corvallis, Oregon, and also reviewed documents in the applicants Electronic Reading Room.

The NRC staffs audit plan, dated July 12, 2018, can be accessed via the NRCs Agencywide Documents Access and Management System (ADAMS) under accession number ML18180A079.

The NRC staff conducted the audit in accordance with the Office of New Reactors (NRO) Office Instruction NRO-REG-108, Regulatory Audits.

With the exception of Objective 3, all other objectives of the audit were met during or shortly after the audit. A follow-up audit may be necessary to complete Objective 3. The enclosed report summarizes the NRC staffs activities and conclusions involving the audit.

Docket No.52-048

Enclosure:

Audit Summary Report CONTACT: Prosanta Chowdhury, NRO/DLSE 301-415-1647

ML18298A190 - Memo & Report ML18298A189 - Pkg

• via Email NRO- 002 OFFICE NRO/DLSE/LB1: PM NRO/DLSE/LB1: LA NRR/DIRS/IRAB: BC NRO/DLSE/LB1: PM NAME PChowdhury CSmith* GBowman* PChowdhury DATE 10/25/2018 10/31/2018 11/07/2018 11/14/2018 U.S. NUCLEAR REGULATORY COMMISSION NUSCALE POWER, LLC DESIGN CERTIFICATION APPLICATION - HUMAN FACTORS ENGINEERING AUDIT

SUMMARY

REPORT AUDIT TEAM MEMBERS Week 1: July 30, 2018 - August 2, 2018 Brian Green (NRR/DIRS/IRAB)

Lauren Nist (NRR/DIRS/IRAB)

Week 2: August 27, 2018 - August 30, 2018 Maurin Scheetz (NRR/DIRS/IOLB)

Amy DAgostino (RES/DRA/HFRB)

Greg Galletti (NRO/DCIP/QVIB)

Doug Eskins (OCHCO/ADHRTD/STTSB)

PROJECT MANAGER: Prosanta Chowdhury (NRO/DLSE/LB1)

Background

By letter dated December 31, 2016, NuScale Power, LLC., (NuScale or the applicant) submitted a Design Certification Application (DCA) for review (Agencywide Documents Access and Management System (ADAMS) Accession Number ML17013A229). NuScale submitted Revision 1 of the DCA on March 15, 2018 (ML18086A090). The DCA includes Final Safety Analysis Report (FSAR) Tier 2, Chapter 18, Human Factors Engineering, which describes the human factors engineering (HFE) program for the NuScale Standard Design Power Plant (ML18086A191).

Consistent with NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants, Chapter 18, Human Factors Engineering, the U.S.

Nuclear Regulatory Commission (NRC) staff uses the review criteria in NUREG-0711, Human Factors Engineering Program Review Model, Revision 3, to determine whether the DCA complies with HFE-related NRC regulatory requirements. The staff also uses the guidance in NUREG-0700, Human-System Interface Design Review Guidelines, Revision 2, to evaluate the applicants HFE design.

The applicant submitted the Human Factors Verification and Validation (V&V) Implementation Plan (IP) (RP-0914-8543), Revision 4 (ML18004B974), and the Human Factors Engineering Program Management Plan (HFE PMP), Revision 3 (ML17004A222), for the staffs review with the DCA. The V&V IP describes the applicants method for integrated system validation (ISV) testing, which is a significant HFE validation activity. The HFE PMP describes a method for addressing issues identified during validation activities.

In a letter dated April 8, 2016, from Mr. Thomas Bergman, Vice-President, Regulatory Affairs, NuScale, to Mr. Frank Akstulewicz, then Director of Division of New Reactor Licensing, NRC (ML16099A270), the applicant stated it would submit the Human Factors Verification and Enclosure

Validation Results Summary Report (V&V RSR), which will include the results of the ISV testing, prior to Phase 4 of the staffs review of the DCA. The applicant provided the ISV test schedule and scenarios to the staff in its Electronic Reading Room (ERR). The staff selected a sample of ISV scenarios to observe based on the safety and risk significance of the scenario events and the availability of the staff.

Audit Activities The staff developed the following audit objectives, which are stated in the audit plan (ML18180A079).

1. Verify, by observation of ISV scenario trials and briefs with test personnel and participants, that the ISV is conducted in accordance with the applicants procedures and uses methodology that conforms to guidance in NUREG-0711.

2. Review the results of simulator performance testing conducted prior to ISV testing to validate the ISV testbed.

3. Confirm that human engineering discrepancies (HEDs) identified during V&V activities are analyzed and addressed in accordance with NuScales procedures and using a methodology that conforms to NUREG-0711.

4. Review differences between the validated human-system interface (HSI) design and the HSI design described in the FSAR.

To meet the objectives described above, the staff reviewed the following documents:

• Human Factors Verification and Validation Implementation Plan, Draft Revision 5

• ISV Test Plan, Revision 2

• Daily ISV Data Summaries (July 23, 2018 - August 2, 2018 and August 27, 2018 -

August 30, 2018)

• Scenario Guide 2, Revision 0

• Scenario Guide 5, Revision 0

• Scenario Guide 7, Revision 0

• Scenario Guide 9, Revision 0

• Workload and Situation Awareness Questionnaires for Scenarios 2, 5, 7, and 9

• Integrated System Validation Simulator Performance Testing Procedure, Revision 1

• Simulator Scenario-Based Testing Procedure, Revision 1

• Simulator Deficiency List

• ISV simulator performance test results (July 2018) for the following malfunctions:

manual reactor trip, pressurizer spray fails open, reactor safety valve fails, steam generator tube leak, steam generator tube failure, decay heat removal actuation, dropped control rod, chemical and volume control leak, loss of chemical and volume control purification flow, turbine trip, turbine bypass valve fails open, loss of condenser vacuum, loss of site cooling water, loss of circulating water, loss of feedwater

• An Excel spreadsheet used to track and analyze issues prior to entry in the Human Factors Engineering Issue Tracking System (HFEITS)

• Computer-Based Procedures for Common Actions of Nature - Earthquake, Respond to Primary Sample Sink High Radiation Alarm, and Unit Shutdown

• Conduct of Operations, Revision 1

• NuScale Simulator Software Configuration Index, Revision 0

• NuScale Software Configuration Management, Revision 0 In addition, the staff held discussions with the applicant1 and observed the following activities:

• Pre-job brief for ISV participants

• Pre-scenario brief for ISV test personnel

• Two trials of four ISV scenarios

• Post-scenario debrief with ISV participants

• Post-scenario debrief with ISV test personnel Audit Results Objective 1: Verify, by observation of ISV scenario trials and briefs with test personnel and participants, that the ISV is conducted in accordance with the applicants procedures and uses methodology that conforms to guidance in NUREG-0711.

The staff observed two trials each of four different ISV scenarios. The staff also observed the pre-job briefs for the ISV participants, the pre-scenario briefs for the ISV test personnel, and debriefs for the ISV test personnel and test participants.

The staff made the following observations:

• The applicant followed its ISV test procedures with one exception. The staff observed the applicant continued an ISV scenario after stopping the scenario to address a 1

The following members of the applicants HFE Design Team participated in substantive discussions with the staff: Tim Tovar, Manager, Plant Operations, Doug Bowman, Supervisor, Plant Operations, Ross Snuggerud, Simulator Supervisor, Plant Operations, Pat Leary, Senior Reactor Operator, Ryan Flamand, Senior Reactor Operator, Kevin LaFerriere, Human Factors Engineer, and Jessica Stevens, Human Factors Engineer.

simulator issue for an amount of time that exceeded the time limit specified in the applicants procedure. The applicant explained that because the issue occurred after a predetermined data collection point, all necessary ISV data was collected, and it was feasible to restart the scenario at the point where the scenario had been paused for data collection. The staff did not observe any significant impact on the test as a result of the applicant resuming the scenario following a delay that exceeded the time limit identified in its ISV test procedure.

• All ISV performance measures described in the V&V IP were collected consistently during each scenario trial.

• The applicant used a consistent process for data storage and documentation. The staff observed that the applicant used a checklist to ensure means for data collection were established prior to commencing the scenarios.

The staff did not observe any significant deviations from the methods described in the V&V IP, the ISV Test Plan, or from the guidance in NUREG-0711 for the administration of the scenarios and collection of data.

Objective 2: Review results of simulator performance testing conducted prior to ISV testing to validate the ISV testbed.

The staff reviewed the NuScale simulator components, design, and performance, as well as the activities associated with simulator testing, validation, maintenance, and configuration management. The staff conducted the review by observing simulator performance during ISV scenarios, interviewing the applicant, and reviewing simulator test procedures and simulator performance test results. The staff made the following observations:

• The applicants simulator uses commercially validated, industry standard simulation modeling and execution tools that have the capability to render high fidelity, valid outputs relative to the models implemented. The applicants simulator uses models developed from design information included in the DCA and best estimate data.

• The applicant used commercial software to develop a custom tool to interface with the simulated plant model and to provide displays and controls. The staff observed the custom tool adequately represents and synchronizes timing, display, and control functions.

• The applicant explained that the simulator used for ISV is considered the current baseline simulator (NSIDE 7/12/2018, ISV), and the applicant has a process for tracking and dispositioning simulator issues and changes relative to the baseline.

• The staff reviewed the applicants simulator performance testing methods and observed that they were similar to those described in the 2009 version and the 2017 draft version of ANSI/ANS-3.5, Nuclear Power Plant Simulators for Use in Operator Training and Examination.

• The staff observed that the applicant documented the completion of performance testing and scenario-based testing conducted to validate simulator fidelity prior to commencing ISV testing in accordance with simulator performance testing procedures. Because there is not yet an as-built reference plant, the applicant evaluated the fidelity of the simulator using design information included in the DCA informed by subject matter experts and best estimate model parameters. The performance test and scenario-based test results showed the tests were completed satisfactorily; however, in some cases, the applicant documented issues that were identified during testing and evaluated their significance relative to their effects on the ISV. The staff observed that the applicant provided justification for why some issues would not impact the ISV testing and therefore, did not need to be corrected prior to ISV testing. The staff reviewed the list of simulator deficiencies (i.e., issues) and discussed these with the applicant. The staff did not observe that any of the simulator deficiencies impacted the interface completeness or fidelity of the simulator.

• The staff observed simulator performance during the scenarios. The applicants scenario guides described the expected plant response for each scenario event. The staff observed that the simulator demonstrated the expected plant response documented in the scenario guides. Also, the staff compared the expected plant response documented in the scenario guides to the descriptions of plant systems and the accident analyses in the FSAR for a sample of events and observed that the expected plant response in the scenario guides was consistent with the plant system response or accident analysis documented in the FSAR.

• In multiple scenarios, the staff observed a persistent issue with the plant alarm system modeled in the simulator. Specifically, some alarms were intermittently and automatically muted when they should have been audible. In some instances this appeared to increase cognitive workload; however, this increase in workload was not observed to cause significant negative impacts on human performance. The cause of this alarm system malfunction was unclear at the time of the observation. During discussion with the staff, the applicant explained that additional information needed to be collected and evaluated to determine the cause of the issue and support necessary corrective actions. The applicant identified the issue as a potential HED and entered the issue into the appropriate tracking system.

Objective 3: Confirm that HEDs identified during V&V activities are analyzed and addressed in accordance with NuScales procedures and using a methodology that conforms to NUREG-0711.

The staff observed that the applicant identified and documented issues in an Excel spreadsheet, and no issues had been documented yet in HFEITS as described in the V&V IP. During discussion with the staff, the applicant explained that issues will be documented in HFEITS, along with the issue priority level and resolution, after all of the data has been collected from the ISV testing and the issues have been reviewed by the applicants HFEITS Review Committee.

The staff observed these activities are consistent with the issue resolution process described in the HFE PMP. The staff reviewed the Excel spreadsheet and found that it contained sufficient information about identified issues to track and assess the issues at a later time. The entries made in the spreadsheet were consistent with those made by the staff. The staff may conduct a future audit to confirm that the disposition of all HEDs is conservative and appropriate.

Objective 4: Review differences between the validated HSI design and the HSI design described in the FSAR.

The staff compared the HSI design information described in the FSAR with the HSI design used during ISV testing. This part of the audit included a review of specific HSI design areas. The staff made the following observations:

• The post-accident monitoring (PAM) variables as displayed on the Safety Display Indication System (SDIS):

All Type B, C and D PAM variables listed in FSAR Tier 2, Revision 1, Chapter 7, Table 7.1-7, Summary of Type A, B, C, D, and E Variables, appear on the Safety Display Indication (SDI) panels for each unit or the common system SDI panel. The only discrepancy the staff identified is that the naming convention for some PAM Type D variables did not match the name for the same parameter on the SDI common panels.

The staff observed that the applicant had identified and documented this issue and other instances of naming conventions not being consistent across HSI displays during ISV testing, which is consistent with the applicants test procedures.

• Critical safety function and safety function monitoring:

The staff observed the shift technical advisor was able to complete safety function status checks using the SDIS and a tablet-form procedure efficiently and in a timely manner for multiple NuScale Power Modules.

• Generic Technical Guidelines validation activities:

During discussion with the applicant, the staff learned that the applicant is using ISV testing, post-ISV testing in the simulator, and table-top evaluations to validate the Generic Technical Guidelines (GTGs) for the NuScale Standard Power Plant. The staff is reviewing the GTGs as part of the DCA Review.

• Computer-based procedures:

The staff reviewed a sample of computer-based procedures (CBP) and compared them to a sample of 17 criteria from NUREG-0700 and 11 criteria from Digital Instrumentation and Controls Interim Staff Guidance 5, Revision 1 (DI&C ISG-05), Section 1, Computer-Based Procedures. The staff discussed the CBPs with the applicant and also reviewed data collected during ISV about the CBPs. The staff confirmed that the paper procedures matched the computer-based versions during ISV testing. The staff observed that the CBPs met the criteria sampled with the exception of one item.

NUREG-0700, Criterion 8.3.3-3, Note Taking, indicates that there should be a way for users to record notes and comments in the CBP. The applicants CBPs do not permit note-taking within the CBPs. The applicant explained that operators may make written notes or make an electronic log entry for procedure issues. The staff determined that although the users cannot make notes in the CBPs, there are other means for users to provide feedback when issues are identified, which is consistent with the guidance in NUREG-0700.

• Other HSI design feature observations:

o Reactor Safety Valve position indication was added to SDIS displays prior to ISV testing as stated by the applicant in HSI Design Results Summary Report Section 4.6.2(3) Relief and Safety Valve Position Monitoring.

o The design provides an audible and a visual indication when there is failure of an automation sequence.

o Unit status is relatively easy to assess using the unit overhead displays.

o For the events planned in the ISV scenarios, the staff observed that the HSI identifies high priority alarms when there are multiple alarms received at the same time for more than one unit.

o The ISV participants opened the earthquake response procedure for a single unit instead of using the method to enter the procedure for all units onsite. At one point, the ISV participants also thought one procedure was being used for one particular unit, but that procedure was actually being used for a different unit.

o There are some inconsistencies on displays across HSIs (e.g., component orientation and labeling on SDIS displays versus operator workstation video display units).

o Some ISV participants did not show understanding of the difference between silencing or acknowledging an alarm.

o The HSI does not allow one workstation to have multiple procedures open or procedure and automation sequence open at same time.

o The audible alarm for critical safety function was not always salient to all of the ISV test participants.

The staff observed that the ISV test personnel captured these issues as part of the ISV data collection. The staff did not observe any apparent significant human performance degradations as a result of these issues. When the applicant submits the V&V RSR for the staffs review, the staff will review the analyses that the applicant conducted for issues identified during V&V activities, corrective actions taken to resolve the issues, and any justifications for why issues do not need resolution.

Conclusion The staff conducted an audit exit brief with NuScale on September 11, 2018. The staff stated that they were able to accomplish all of the audit objectives, with the exception of Objective 3, by reviewing documents in the ERR, observing the ISV testing, and by having discussions with the applicant. The staff could not complete Objective 3 during this audit because the applicant was still conducting testing and collecting the data necessary to identify and analyze HEDs.

The staff and the applicant discussed that a follow-up audit may be necessary to complete Objective 3.