ML18208A371

June 26-28 2018 NuScale Hfe Audit Summary Report

ML18208A371
Person / Time
Site:	NuScale
Issue date:	08/01/2018
From:	Prosanta Chowdhury NRC/NRO/DLSE/LB1
To:	Samson Lee NRC/NRO/DLSE/LB1
CHOWDHURY P/415-1647
Shared Package
ML18208A370	List: ML18208A371
References
Download: ML18208A371 (8)

Text

August 1, 2018 MEMORANDUM TO: Samuel S. Lee, Chief Licensing Branch 1 Division of Licensing, Siting and Environmental Analysis Office of New Reactors FROM: Prosanta Chowdhury, Project Manager /RA/

Licensing Branch 1 Division of Licensing, Siting and Environmental Analysis Office of New Reactors

SUBJECT:

AUDIT

SUMMARY

REPORT FOR THE REGULATORY AUDIT OF NUSCALE POWER, LLC DESIGN CERTIFICATION APPLICATION HUMAN FACTORS ENGINEERING From June 26, 2018, through June 27, 2018, the U.S. Nuclear Regulatory Commission (NRC) staff conducted a regulatory audit of NuScale Power, LLC (NuScale) Design Certification Application (DCA) Human Factors Engineering (HFE) Topics.

The purpose of the audit was to: (1) verify that the human factors analyses in various databases were complete, (2) confirm that human error deficiencies are being treated in accordance with NUREG-0711, Human Factors Engineering Program Review Model, Revision 3, and (3) review the human factors engineering verification and validation scenario guides to prepare for the upcoming integrated system validation audit. The NRC staff conducted the audit at NuScales offices in Rockville, Maryland, and also reviewed documents in the NuScale Electronic Reading Room.

The NRC staffs audit plan, dated June 4, 2018, can be accessed via the NRCs Agencywide Documents Access and Management System (ADAMS) under Accession No. ML18144A758.

The NRC staff conducted the audit in accordance with the Office of New Reactors (NRO)

Office Instruction NRO-REG-108, Regulatory Audits.

The objectives of the audit, based upon the purpose stated above, were met during or shortly after the audit. The enclosed report summarizes the NRC staffs activities and conclusions involving the audit.

Docket No.52-048

Enclosure:

1. Audit Summary Report CONTACT: Prosanta Chowdhury, NRO/DLSE 301-415-1647

ML18208A371 ML18208A370 - Pkg *via Email NRO- 002 OFFICE NRO/DLSE/LB1: PM NRO/DLSE/LB1: LA NRO/DCIP/HOIB: BC NRO/DLSE/LB1: PM NAME PChowdhury* MMoore* JGaslevic* PChowdhury DATE 7/27/2018 7/30/2018 7/27/2018 8/1/18 U.S. NUCLEAR REGULATORY COMMISSION NUSCALE POWER, LLC DESIGN CERTIFICATION APPLICATION - HUMAN FACTORS ENGINEERING AUDIT

SUMMARY

REPORT Audit Team Members

• Brian Green

• Amy DAgostino

• Maurin Scheetz

• Prosanta Chowdhury (Project Manager)

Background

By letter dated December 31, 2016, NuScale Power, LLC (NuScale) submitted a design certification application (DCA) for review (Agencywide Documents Access and Management System (ADAMS) Accession No. ML17013A229). The DCA includes Final Safety Analysis Report (FSAR) Tier 2, Chapter 18, Human Factors Engineering, which describes the human factors engineering (HFE) program for the NuScale Standard Design Power Plant (ADAMS Accession No. ML17013A289).

Consistent with NUREG-0800, Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants, Chapter 18, Human Factors Engineering, the U.S.

Nuclear Regulatory Commission (NRC) staff (hereinafter referred to as the staff) is using the review criteria in NUREG-0711, Human Factors Engineering Program Review Model, Revision 3, to determine whether the NuScale design complies with HFE-related NRC regulatory requirements. The HFE Program Review Model described in NUREG-0711 consists of 12 elements (refer to Figure 1-1, Elements of the HFE programs review model, of NUREG- 0711 for additional information).

FSAR Tier 2, Section 18.0, Human Factors Engineering - Overview, states the NuScale HFE program incorporates accepted HFE standards and guidelines including the applicable guidance provided in NUREG-0711. NuScale submitted two types of technical reports (implementation plans (IP) and results summary reports (RSR)) to address the 12 elements in NUREG-0711. NUREG-0711, Section 1.2.2, Review Elements, explains that IPs describe processes and methods for performing HFE activities and RSRs describe the results of performing those activities.

HUMAN FACTORS ANALYSES In May 2017, the staff conducted an audit consistent with the audit plan dated April 25, 2017 (ADAMS Accession No. ML17110A254). The audit included a review of the integrated system validation (ISV) Test Plan and scenario bases documents to verify the verification and validation (V&V) scenarios and ISV test methods conform to selected review criteria in NUREG-0711, Section 11.4, Review Criteria. The results of the audit are documented in the audit summary report dated July 17, 2017 (ADAMS Accession No. ML17181A415). As mentioned in the summary report, the staff found that some entries in the HFE databases were not complete. Subsequently, the staff issued Request for Additional Information (RAI) 9123 to identify the expected dates of completion for these analyses. The response to RAI 9123 Enclosure 1

(ADAMS Accession No. ML18002A554) provided the staff with sufficient information to plan this follow on audit.

INTEGRATED SYSTEM VALIDATION AND ISSUE TRACKING NuScale is planning to conduct ISV as part of the HFE verification and validation activities described in NUREG-0711. NuScale submitted, Human Factors Verification and Validation Implementation Plan (V&V IP), Revision 2, with the DCA. In a letter dated April 8, 2016, from Mr. Thomas Bergman, NuScale, Vice-President, Regulatory Affairs to Mr. Frank Akstulewicz, Director, Division of New Reactor Licensing, NRC (ADAMS Accession No. ML16099A270),

NuScale stated that NuScale would submit the V&V Results Summary Report (RSR) prior to Phase 4 of the NRCs review of the NuScale DCA.

As described in NUREG-0711, Section 11.1, Background, a significant V&V activity is the ISV test. The V&V IP, Section 4.6, Test Design, states, Upon approval, the ISV scenarios and test plan will be available for review or audit by the NRC sufficiently before the conduct of ISV so that comments or concerns can be adequately addressed prior to commencing ISV.

NUREG-0711 also describes a method by which an applicant can document human factors issues that arise with the design and track them through resolution. NuScale describes this method in the Human Factors Verification and Validation Implementation Plan and in Human Factors Engineering Program Management Plan Rev. 3 (ADAMS Accession No. ML17004A221).

Audit Activities The staff developed the following audit objectives which are described in the audit plan (ADAMS Accession No. ML18144A758).

1. Verify that the sample of results from human factors analyses (operating experience review (OER), functional requirements analysis/function allocation (FRA/FA), and task analysis (TA)) documented in docketed results summary reports are representative of the full set of HFE results documented in various NuScale databases.

2. Confirm that issues identified during the May 2017 audit (ADAMS Accession No. ML17181A415) have been resolved in accordance with the response to RAI 9123 (ADAMS Accession No. ML18002A554).

3. Confirm that HEDs identified during HFE analyses have been analyzed and resolved in accordance with NUREG-0711.

4. Review the V&V scenario guides to prepare for the integrated system validation audit.

To meet the objectives described above, the staff reviewed the following NuScale documents:

• A sample of ISV Scenario Guides (including scenario numbers 1, 3, 4, 6, 9, and 11).

• V&V IP Rev. 5 (draft).

• HFE Program Management Plan Rev. 5 (draft).

• TA RSR Rev. 1.

• OER RSR Rev. 0.

2

• FRA/FA RSR Rev 0.

• EP-0303-1336 Rev. 3, HFE Issue Tracking System Procedure.

In addition, the staff reviewed samples of data entered into the following NuScale databases:

• OER database.

• FRA/FA database.

• TA database.

• Human Engineering Discrepancies (HEDs) in the HFE Issues Tracking System (HFEITS) database.

• Items in the Test Track system.

The staff also requested to review the Scenario Cross-Reference Table, but an updated version was not available during the audit. NuScale agreed to update it and make it available in their Electronic Reading Room (eRR) prior to the upcoming ISV audit activities. This did not negatively affect the planned audit activities.

Audit Results Objectives 1 and 2:

The staff reviewed a sample of 13 operating experience items in the OER database to confirm that all items were completely analyzed. The sample selected was based, in part, upon the considerations described in Appendix A of NUREG/CR-7202, NRC Reviewer Aid for Evaluating the Human-Performance Aspects Related to the Design and Operation of Small Modular Reactors. The staff found that the sample reviewed considered the applicable topics described in Section A.1 of Appendix A. All entries reviewed were complete.

The staff also reviewed a sample of 12 items in the FRA/FA and TA database. The sample was based, in part, on the considerations described in Appendix A of NUREG/CR-7202. The staff found that the sample reviewed adequately considered the applicable small modular reactor considerations described in sections A.2 and A.3. The staff found one cell in the database which was labeled to be determined. The staff analyzed the nature of the missing information and determined that it was information that is not necessary to fulfill any NRC requirements.

The staff found that all entries in the sample were sufficiently analyzed and documented in a manner that is consistent with the methodologies described in the OER, FRA/TA and FA results summary reports and with NUREG-0711. The staff concluded that the sample of data provided in the RSRs is an adequate representation of content of the databases, and that the nature of the examples provided is an adequate representation of the quality of the database entries.

Objective 3:

The staff reviewed two tracking mechanisms used to address human engineering discrepancies (HEDs).

3

• HFEITS is the primary mechanism for tracking HEDs. NuScale has formally tracked HEDs in HFEITS since the staffing plan validation testing conducted in 2016 (ADAMS Accession No. ML16137A257). HFEITS also includes HEDs derived during HSI design testing, and applicable OER items. It will also be used to document any HEDs generated during ISV testing.

• NuScale also uses a system called Test Track for tracking software design issues.

Since HFE and software design often overlap, many of the items are tracked in both systems.

The staff reviewed a sample of seven HEDs currently being tracked in HFEITS. The entries reviewed generally had an appropriate level of detail which was consistent with NUREG-0711.

However, the staff observed the issues below that may warrant resolution prior to generation of the V&V RSR.

• Observation 1: The staff found that the structure of the HFEITS database includes unique text entry fields associated with many of the tracking system features described in NUREG-0711. However, the staff also observed instances where text entry fields labeled Resolution and Recommendation allowed for paragraphs of text to be entered. The content of these fields appeared to be inconsistent across entries. This increases the potential for those entering data into the database to unintentionally omit important information regarding the analysis and resolution of tracked issues. This may cause challenges in the traceability of resolutions. Additional control of the use of these fields may be warranted, such as, through modification of the database to ensure complete entries, or the use of administrative controls. Since the HED generation and resolution process is not yet complete, this does not present an immediate concern.

However, the staff foresees that without additional controls, it may be difficult for the applicant to document HEDs consistently such that issues can be adequately identified and addressed.

• Observation 2: The NuScale computer based procedure system utilizes unique automation features not used by previous new reactor designs. The staff observed that some changes to the computer based procedure system are occurring without being identified as HEDs, and therefore, are not being tracked. Discussions with NuScale indicated that these changes are done for correcting small changes. More significant changes are being formally tracked in accordance with the appropriate procedures.

Staff reviewed, HFE Issue Tracking System Procedure, and did not find any guidance specific to addressing the appropriate threshold for entering these issues in to HFEITS.

The staff expressed concerns that even small changes can affect the way operators interact with the system, and that by not tracking these changes, it will be unclear if sufficient testing has occurred to safely conclude that changes were effective. The staff acknowledge that the upcoming ISV testing provides an opportunity to test the computer based procedure system, but there is no expectation that all changes to the system will be tested.

• Observation 3: The staff observed that HEDs were prioritized in HFEITS using terminology that is similar to the terminology used in NUREG-0711, but is defined differently. The staff discussed the issue with NuScale. The current prioritization used 4

by NuScale reflects the expected timing of the resolution, whereas NUREG-0711 indicates HEDs should be prioritized based on safety significance. NuScale plans on adapting the priority scale (or using multiple prioritization schemes simultaneously) during the ISV to conform to NUREG-0711. NUREG-0711 does not prescribe when an applicant must conduct the prioritization.

The staff discussed the observations described above with the NuScale staff during the audit exit meeting on June 27, 2018. NuScale agreed to consider the feedback with the understanding that staff will have another opportunity to observe HEDs during the planned ISV audit and potentially at another audit reviewing any design changes that occur as a result of ISV testing.

The staff also reviewed a sample of items that were in both HFEITS and Test Track to ensure that the use of dual tracking systems was effective. There were no instances observed where information was inconsistent between the two systems.

NuScale generated work-down curve items in Test Track of issues remaining to be resolved.

The staff reviewed the work-down curve and found that the vast majority of issues had been resolved or are currently being tested (689 items). There were 35 remaining items that were in process for design changes or for which design changes have not yet begun.1 Objective 4:

The staff reviewed a sample of the ISV scenario guides to ensure that they adequately addressed the applicable NUREG-0711 criteria. The staff found that the scenarios addressed the applicable criteria with the exceptions of precise definition of workplace factors and staffing levels. NuScale explained that for these particular items, staffing and workplace factors are assumed to be set at normal levels, unless there is an exception discussed in the abstract of the scenario guide. The staff found that this was, in fact, how the abstracts were written. The staff found this to be an adequate strategy to address these issues.

Conclusions The staff conducted an audit exit brief with NuScale on June 27, 2018. The staff stated that they were able to accomplish the audit objectives by reviewing documents in NuScales eRR, sampling analyses contained in the HFE databases, and by having discussions with the NuScale staff.

The staff were able to confirm that the results in the OER, FRA/FA, and TA RSR are appropriate representations of the HFE analyses documented in the OER, FRA/FA and TA databases. As a result, RAI 9123 is resolved and closed.

The staff were able to gain a better understanding of the methods used to track and resolve HEDs. This will help the staff to prepare for the upcoming ISV audit as well as any future audits that may be used to confirm that all HEDs have been prioritized and dispositioned according to NUREG-0711.

1 Note: There is no expectation that all HEDs should be resolved at this time. HEDs will be generated through the upcoming ISV testing. HEDs of significance are expected to be resolved after the ISV is complete and retesting may be warranted.

5

The staff reviewed the ISV scenario guides to confirm that they contain the relevant scenario information. The staff confirmed that the scenario guides were sufficient to communicate the various scenario parameters to NuScale ISV staff as well as staff who are observing the ISV testing.

The staff communicated observations which may cause misunderstandings and potential complications during the upcoming audits as described above. These observations do not present immediate concerns, however NuScale should consider resolving these issues prior to conducting the ISV testing.

The four objectives of this audit were met, and therefore, the staff do not plan to conduct follow-up audits regarding objectives 1 and 2. The material reviewed subject to objectives 3 and 4 is in preparation for the upcoming ISV audit, and may therefore need to be reviewed during that audit.

6