ML20181A366

Regulatory Audit Rport for Submittal of Non-Proprietary Information for Amendment 4 to the HFC-6000 Safety Platform

ML20181A366
Person / Time
Site:	99902026
Issue date:	07/23/2020
From:	Joseph Holonich Licensing Processes Branch
To:	Yang S HF Controls Corp
Holonich J
References
EPID L-2018-TOP-0031
Download: ML20181A366 (15)

Text

July 23, 2020 Dr. Steve Yang Sr. Vice President HF Controls Corporation 1624 West Crosby Road Suite 124 Carrollton, TX 75006

SUBJECT:

REGULATORY AUDIT REPORT FOR SUBMITTAL OF NON-PROPRIETARY INFORMATION FOR AMENDMENT 4 TO THE HFC-6000 SAFETY PLATFORM (EPID L-2018-TOP-0031)

Dear Dr. Yang:

By letter dated April 15, 2019 (Agencywide Documents Access and Management System (ADAMS) Accession No. ML19109A158), HF Controls (HFC) submitted for U.S. Nuclear Regulatory Commission (NRC) staff review a topical report (TR) Submittal of Non-proprietary Information for Amendment 4 to the HFC-6000 Safety Platform. The TR is supported by documentation that includes plans, requirements, design specifications, programming and hardware testing, independent verification and validation, and equipment qualification testing.

From May 4 through May 27, 2020, the NRC staff conducted a virtual audit of the HFC-FPGA platform design, development, and equipment qualification processes. The purpose of this letter is to provide HFC with the results of the regulatory audit. Documented in the enclosed report are the observations the NRC staff identified during the audit.

If you have any questions regarding this matter, I may be reached at 301-415-7297 or by electronic mail at Joseph.Holonich@nrc.gov.

Sincerely,

/RA/

Joseph J. Holonich, Senior Project Manager Licensing Processes Branch Division of Operating Reactor Licensing Office of Nuclear Reactor Regulation

Enclosure:

Audit Report Docket No. 99902026

ML20181A366 *concurred via email NRR-106 OFFICE NRR/DORL/LLPB/PM* NRR/DORL/LLPB/LA* NRR/DEX/EICB/BC*

NAME JHolonich DHarrison MWaters DATE 7/01/2020 7/23/2020 7/23/2020 OFFICE NRR/DORL/LLPB/BC* NRR/DORL/LLPB/PM*

NAME DMorey JHolonich DATE 7/23/2020 7/23/2020 U.S. NUCLEAR REGULATORY COMMISSION REGULATORY AUDIT REPORT FOR HF CONTROLS FIELD PROGRAMMABLE GATE ARRAY INSTRUMENTATION AND CONTROL PLATFORM

1. Background

The U.S. Nuclear Regulatory Commission (NRC) staff is preparing a safety evaluation (SE) of the Topical Report RR901-107-10, Amendment for HFC-FPGA System of HFC-6000 Safety Platform (HFC-FPGA [Field Programmable Gate Array]-TR), Revision F (Agencywide Documents Access and Management System (ADAMS) Package Accession No. ML19109A165). HF Controls (HFC) is seeking generic approval of the HFC-FPGA platform for use in safety systems of nuclear power plants. The NRC staff sent Requests for Additional Information for Amendment 4 to the HFC-6000 Safety Platform on February 19, 2020 (ADAMS Accession No. ML20021A232 Proprietary and ML20021A231 Non-Proprietary). An audit plan was sent to HFC on April 27, 2020 (ADAMS Accession No. ML20043F266).

2. Regulatory Audit Bases As part of its SE, the NRC staff conducted a virtual audit of the HFC-FPGA platform design, development, and equipment qualification (EQ) processes. The audit was conducted remotely between NRC staff and HFC representatives between May 4 and May 27, 2020. The following regulations and regulatory guidance constitute the basis for this audit:

The Code of Federal Regulations, Title 10 (10 CFR) 50.54, Conditions of licenses, (jj) and 10 CFR 50.55, Conditions of construction permits, early site permits, combined licenses, and manufacturing licenses, (i), require that structures, systems, and components must be designed, fabricated, erected, constructed, tested, and inspected to quality standards commensurate with the importance of the safety function to be performed.

10 CFR 50.55a, Codes and standards, (h), Protection and Safety Systems, incorporates the 1991 version of IEEE Std. 603, IEEE Standard Criteria for Safety Systems for Nuclear Power Generating Stations, by reference, including the correction sheet dated January 30, 1995.

10 CFR Part 50, Appendix A, General Design Criteria for Nuclear Power Plants General Design Criterion (GDC) 1, Quality standards and records GDC 2, Design bases for protection against natural phenomena GDC 4, Environmental and dynamic effects design bases GDC 13, Instrumentation and control GDC 19, Control Room GDC 20, Protection system functions GDC 21, Protection system reliability and testability GDC 22, Protective system independence GDC 23, Protective system failure modes GDC 24, Separation of protection and control systems GDC 25, "Protection system requirements for reactivity control malfunctions" Enclosure

GDC 29, "Protection against anticipated operational occurrences" Commercial Grade Dedication (10 CFR Part 21, Reporting of Defects and Noncompliance, and the commercial grade dedication processes and methods as approved by the NRC in Electric Power Research Institute (EPRI) TR-106439).

Digital Safety System Software Quality and Processes (10 CFR Part 50, Appendix B, Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants, as discussed in BTP 7-14 and applicable regulatory guides (RGs)).

Secure Development Environment (10 CFR Part 50 as elaborated in RG 1.152, Revision 3).

NUREG-0800, NRC Standard Review Plan, (SRP), Chapter 7, Table 7.1 (ADAMS Accession No. ML070460342) identifies RGs, branch technical positions (BTPs),

and industry standards that contain information, recommendations, guidance and, in general, provide an acceptable basis to implement the above requirements for both hardware and software features of safety-related (SR) digital instrumentation and control (I&C) systems.

Clause 5.4 in IEEE Std. 603-1991, Equipment Qualification, requires that Safety system equipment shall be qualified by type test, previous operating experience, or analysis, or any combination of these three methods, to substantiate that it will be capable of meeting, on a continuing basis, the performance requirements as specified in the design basis.

EQ (RG 1.100, RG 1.180, RG 1.209, and NRC endorsed EPRI TR-107330).

3. Regulatory Audit Activities This audit was conducted in accordance with NRR Office Instruction LIC-111, Regulatory Audits. The NRC staff reviewed procedures and records related to HFC-FPGA platform development processes. The NRC staff evaluated the effectiveness of software development activities to determine the degree to which processes described in the HFC-FPGA TR are being implemented to achieve a high-quality system for use in a nuclear facility.

A secondary purpose of the audit was to gain a better understanding of the HFC-FPGA development life cycle processes to support the safety evaluation of the HFC-FPGA platform and to assess the capabilities of the platform to determine the degree to which a HFC-FPGA based I&C safety system is capable of meeting regulatory acceptance criteria as described in Chapter 7 of the NRC SRP.

A third purpose of the audit was to better understand HFCs equipment qualification testing programs, plans, procedures, and detailed test records developed for the HFC-FPGA platform.

3.1. Entrance Meeting The NRC staff provided an overview of the audit plan and discussed the objectives for the audit.

Logistics and the detailed schedule of audit activities were then reviewed to accommodate availability of participants.

3.2. Presentations by HFC HFC provided the following presentations during the audit:

NRC Audit FPGA Development Tools HFC-FPGA Platform Requirement Traceability SDOE for HFC-FPGA Platform-NRC Audit Material 3.3. Equipment Demonstration HFC provided a virtual tour of the HFC-FPGA platform and a virtual tour of its secure development environment. The virtual tours consisted of live streaming audio and video feeds from the HFC development facility where an HFC-FPGA platform-based test system was in operation. The NRC staff was able to observe system operation and interact with HFC personnel during these tours.

The HFC-FPGA equipment demonstrated was the Qualification Test Specimen (QTS) being used to support equipment qualification. This demonstration included performance of the following activities:

System operation using test signals Overview of modules included in QTS Demonstration use of maintenance work station for monitoring system performance during operation 3.4. Anomaly Process Review The HFC-FPGA TR states that anomaly reporting within verification and validation (V&V) process for the HFC-FPGA system was performed per IEEE Std. 1012-2004. The NRC staff reviewed and discussed the non-conformance and corrective action program (QPP 16.1) as well as condition reporting procedures used by HFC during product development to confirm understanding and use of the condition resolution processes.

All HFC employees are responsible for identifying, documenting, and reporting all conditions adverse to quality and safety. The NRC staff found that anomalies or conditions are screened and reviewed in accordance with the HFC 10 CFR Part 50, Appendix B, compliant quality assurance (QA) program work instructions. When issues identify conditions adverse to quality, such as failures, malfunctions, deficiencies, deviations, defective material and equipment, and non-conformances, they must be promptly identified and corrected. HFC reviews of condition reports include a process for identifying significant conditions adverse to quality.

The corrective measures reviewed by the NRC staff indicated that causes of the anomalous conditions are determined, and corrective actions are taken to preclude recurrence of the issue.

The NRC staff observed that identification of significant conditions adverse to quality including the cause of the conditions and the corrective actions taken are required to be documented and reported to appropriate levels of HFC management.

3.5. Requirements Thread Reviews To facilitate performance of requirements thread reviews, the NRC staff asked HFC to prepare and discuss requirements traceability for sample HFC-FPGA platform requirements.

HFC made a presentation titled HFC-FPGA Platform Requirement Traceability to explain how traceability is performed for the HFC-FPGA platform. During this presentation, HFC described the procedures used to perform traceability. They also introduced the software tools used for traceability and explained how HFC system requirements are traced and how coverage of requirements is achieved. The NRC staff in turn reviewed a work instruction for

performing requirements traceability matrix (RTM) verification (WI-ENG-025) as well as a V&V work instruction for performing RTM traceability analysis activities (WI-VV-004). The following observations were made during these reviews:

An RTM is initiated by the V&V team during the requirements phase of a project using the project contract, requirements specification, or other material.

Each requirement to be tracked is assigned a unique identification number.

A requirements verification form is used to document establishment and verification of requirement fulfillment.

Completed requirements verification forms are submitted to the independent V&V team for tracking and analysis of traceability.

100 percent coverage of requirements is required for all phases of a project.

The independent V&V team is responsible for maintaining the RTM.

The RTM and requirement verification forms are treated as QA records and the procedure defines document storage and retention requirements for these records.

The NRC staff observed some minor inconsistencies between work instruction rules for tool usage and actual observed work practices. Here are the inconsistencies:

- WI-VV-004 Rev. D states in part that: "Only when the source code review is completed, and component test results are successful should the V&V team start the traceability analysis."

However, the NRC staff observed that traceability analysis is performed on source code and component tests before and after completion of code review.

- WI-VV-004 Rev. D states in part that: "Only when the software and hardware component test results are successful should the V&V team start the traceability analysis." However, the NRC staff observed that traceability analysis is performed on software and hardware component test documentation before and after component test execution.

- WI-VV-004 Rev. D states in part that: "Only when the system V&V tests are completed successfully should the V&V team start the traceability analysis." However, the NRC staff observed that traceability analysis is performed on system V&V tests before and after V&V test execution.

HFC acknowledged these inconsistencies and addressed them by initiating a condition report (CR 2020-0082) to update the work instructions to accurately reflect actual work practices.

HFC has specified document levels for establishing traceability that include: Contract / System Requirement Documentation, Requirement Specifications, Design Specifications, Test Specifications, Source Code, EQ Tests, and Bill of Materials (BOM) & Artworks.

The software tool used by HFC for establishing requirement traceability is called ReqTracer. This tool uses the comments functions in Microsoft word or sticky notes function for .pdf files to establish relations or links between the various documents. This enables HFC V&V personnel to perform various analysis activities such as coverage assessments to determine requirement traceability status.

A sample design specification requirement was then used to demonstrate the establishment of traceability. The requirements associated with this thread traced to the requirement specification.

This was shown using the ReqTracer tool. HFC explained how the requirement had been

implemented and tested during the HFC-FPGA platform development processes. This requirement was successfully traced to the following documentation:

Requirements Specification (RS)

Design Specification (DS)

Test Specification (TS)

Test Procedure (TP)

Test Report (TR)

Further thread evaluations were then conducted by the NRC staff using documentation made available on the HFC Sharepoint site. The following Requirements were evaluated by the NRC staff:

Requirement 1 - Diagnostic FPGA Power Monitoring function (Digital IO Module)

Requirement 2 - Coil Monitor function (Digital IO Module)

Requirement 3 - WDT LED function (Thermocouple AI Module)

Requirement 4 - Execution Enable Switch function (RTD AI Module)

Requirement 5 - PCB shall be designed to satisfy seismic requirements of RG 1.100 and its endorsed IEEE Std. 344 to qualify for safety application Requirement 6 - PCB shall be designed to satisfy EMI/RFI requirements of RG. 1.180 Requirement 7 - PCB shall be designed to satisfy nuclear Class-1E requirements according to IEEE 7-4.3.2, RG 1.209, and its endorsed IEEE Std. 323 The NRC staff was able to successfully trace selected requirements to implementation documents and verify that traceability was appropriately established and maintained. Because HFC was not able to make the ReqTracer tool available for the NRC staff to use remotely, the NRC staff was unable to use this tool to support verification of traceability. Instead the audit team reviewed the documents known to be used for implementation of selected requirements and independently identified sections that fulfilled the requirements. HFC then demonstrated tracing of the selected requirement using ReqTracer and showed that the same sections were identified by the tool.

Though it was shown that traceability was established using the HFC processes, the NRC staff found that independent verification of traceability was difficult to perform due to the NRC staffs inability to directly access the ReqTracer tool. Additionally, the NRC staff noted that there are specific requirements associated with the EQ in the higher and lower level RS and TP documents, but the middle level DS documents for the HFC-FPGA platform components do not include any requirement for the EQ, except listing relevant regulatory guidance and endorsed standards in the reference section. The lack of specific requirements in the middle-level DS documents could impact the requirement traceability but would not impact the SE of the TR because the higher and lower level documents do include the necessary EQ requirements. HFC representatives acknowledged this observation and said it would be appropriately addressed.

3.6. Configuration Management The NRC staff discussed the configuration management processes related to the storage and modification of the HFC-FPGA platform software, logic configuration files, and controlled documents. The NRC staff performed a tabletop exercise relating to the software

check-out/check-in process. HFC personnel explained that HFC-FPGA controlled files are stored in a secure electronic file storage system and a change implementation approval is required to allow check-out of these files for the purpose of making changes to them. When a file is then checked out, access to the file becomes blocked to all other project team members such that simultaneous changes cannot be made to the same file. The process also includes use of software tool features that provide revision controls so that all versions of each managed file are retained and can be reviewed for audit purposes.

The V&V team determines what validation activities must be performed to complete the change process. Affected files are verified to ensure they match the controlled master copy. Once a change is made and the affected files have been reviewed by the V&V team, the files are transferred back to the controlled repository for check-in.

The NRC staff reviewed the HFC-FPGA master document list as well as several condition reports and software change requests to gain an understanding of how HFC-FPGA platform configurations were being captured and controlled in accordance with the HFC configuration management program described in Section 5.8 of the HFC-FPGA platform TR, HFC Configuration Management.

The Configuration Management (CM) work instruction, WI-ENG-003, was reviewed by the NRC staff. This work instruction defines the HFC System Configuration Management process and is used for all HFC system products. The HFC CM process involves: 1) identification of all HFC Control System components to be controlled, 2) controlling mechanisms used to establish configuration control, and 3) a means of reporting change status of controlled components throughout the project life cycle. The HFC CM program scope includes system requirements, hardware design documents, V&V programs and procedures, and firmware/application software source codes.

The System Change Request (SCR) form, which is an electronic form, includes information on the files and documents affected by the change including a unique change request identifier, a description of the reason for the change, the date of the request, and other pertinent information.

The change request also identifies the initiator and includes a section for evaluation of the change by a change control board. Approval of the change request is only provided upon successful performance of an impact analysis. Once approved, the change implementation is executed, and a change implementation signoff process is performed by engineering, V&V, and QA departments before SCR can be closed.

The NRC staff asked several questions on how the SCR processes were used to ensure correct configurations were established and how they would be used to reflect approved configuration changes to HFC-FPGA equipment. The ReqTracer software tool is used to track changed requirements resulting from the changes made. This tool enables V&V engineers to identify and analyze requirements affected by system changes.

3.7. Secure Development Environment Discussion Because this audit was conducted virtually, the NRC staff was not able to directly observe the secure development environment during the audit. To compensate for this limitation, an HFC representative familiar with the physical attributes and network configuration of the HFC development facilities was available to discuss security measures in place to establish the secure HFC-FPGA platform development environment.

A presentation was provided to the NRC staff describing the HFC development facilities and measures in place to protect the HFC platform equipment. The presentation included a diagram showing secure areas and access point controls. It also included a network diagram which illustrated how the network configuration is used to isolate development environment components from outside network connections and wireless servers.

Implementation of secure development and operational requirements is dependent on the application related activities. Therefore, the NRC staff noted that they will develop SDOE related Plant Specific Action Items (PSAIs) which will be included in the NRC HFC-FPGA platform SE.

3.8. Generic Open Items and Plant Specific Action Items Discussion The NRC presented a draft list of PSAIs which could likely be included in the HFC-FPGA platform SE. Each list item was explained and NRC expectations for resolving these items during system or plant application development was discussed. HFC did not provide comments of feedback on the draft PSAI list.

The NRC staff also discussed the need for a Generic Open Item (GOI) to address the lack of Class 1E to Non-class 1E isolation testing as included in the NRC endorsed EPRI TR-107330.

The NRC staff discussed expectations to resolve this GOI. HFC generated correction report CR2020-0093 to track this GOI. HFC will perform the isolation test and submit it as a supplement to the HFC-FPGA TR. The NRC will review this supplement and issue a revised SE for the HFC-FPGA platform.

3.9. HFC-FPGA Platform Qualified Components List Discussion The NRC staff discussed the HFC-FPGA qualified components list provided as Table 1 in Section 4.0 of the HFC-FPGA TR. The NRC staff asked HFC to provide a list of additional supporting components that should be included in the scope of the platform. HFC provided a listing of auxiliary components that were included as part of the QTS system used for all environmental testing. The staff noted that this list of auxiliary platform components will be included in the NRC SE with the following explanatory notation.

The auxiliary components are HFC-FPGA platform components that were included in the equipment under test (EUT) during platform qualification testing. There are no specific regulatory criteria that apply to these components. However, these components were confirmed by the NRC staff to be included in the platform test configuration and are therefore accepted for use in nuclear safety related applications.

3.10. Discussion on Use of Intellectual Property Cores in the HFC-FPGA Platform Design Vendor-specific intellectual property (IP) cores used in the HFC-FPGA system are discussed in Section 5.3 of the HFC-FPGA TR. Design specification DS001-007-02, HFC-6000 FPGA System IP Core Design Description, includes a list of IP cores used and their descriptions. These vendor-specific IP cores were evaluated and accepted by HFC during the platform V&V process.

To verify adequacy of the V&V process used by HFC for the specific IP cores, the NRC staff reviewed DS001-007-02 and discussed the use of IP cores with HFC personnel. HFC provided a Technical Note, TN901-000-03 Rev. A, HFC-FPGA Amendment 4 FPGA HARD BLOCK QUALIFICATION to clarify the HFC evaluation of IP core use.

The NRC staff understood that the HFC-FPGA platform does not use soft-core IP blocks. The system does however make use of configurable hard-core blocks. These blocks will be used in the internal architecture of the device. The functions of these IP core blocks will be verified by V&V activities performed on system outputs in accordance with method b) of IEEE 7-4.3.2-2003 Clause 5.3.2.

3.11. Request for Additional Information Status Discussion HFC presented draft responses to the NRC staffs request for additional information (RAI) and asked the NRC staff if additional clarification would be needed. Following this discussion, the NRC staff indicated that the draft responses were understood. These responses will be submitted to the NRC upon completion of the audit.

3.12. Equipment Qualification Discussion After performing requirements thread tracings for the EQ, the NRC staff conducted independent sample checks on EQ testing programs, specifications, procedures, and detailed test records. The NRC staff made the following observations, which were discussed with HFC personnel:

There is no test record related to the Class 1E to Non-Class 1E isolation test for the HFC-FPGA platform. HFC stated they wanted to credit the results of the Class 1E to Non-Class 1E isolation tests conducted for the previous HFC-6000 platform. According to the NRC endorsed EPRI TR-107330, the NRC staff stated that the Class 1E to Non-Class 1E isolation tests should be conducted specifically for the HFC-FPGA platform. The results of the Class 1E to Non-Class 1E isolation tests performed for the previous HFC-6000 platform could not be credited for the HFC-FPGA platform.

After discussions during the audit, HFC agreed with the NRC staffs finding that the lack of the Class 1E to Non-Class 1E isolation tests needs to be treated as a GOI in the SE for the HFC-FPGA TR under review. HFC will conduct the tests for the Class 1E to Non-Class 1E isolation after the SE for the HFC-FPGA is issued, and then submit a supplement to this HFC-FPGA TR to NRC to review.

HFC Test Procedure TP901-302-06, Section 4.3 states, in part, that the spectrum shall also be reported for 0.5%, 1%, 2%, and 3% damping, which are also included in the NRC endorsed EPRI TR-107330. However, the NRC staff found that the relevant detailed test record and report only show testing results for the 5% damping factor. HFC stated it will correct and clarify this inconsistency between the testing requirements in the HFC Test Procedure and its associated test record before the HFC-FPGA TR is approved. The NRC staff found that after the audit HFC already took corrective actions accordingly to address this inconsistency.

The NRC staff found that there are some inconsistencies and error in TR901-302-02, HFC-FPGA Equipment Qualification Summary Test Report. HFC made corresponding corrections to the report to address the inconsistencies during the audit. In addition, during the audit the NRC staff found that the maximum acceleration for the seismic test shake table was incorrectly documented at 7 g in the HFC contractors test report (14149 ETL Seismic Qualification Test Report). During the audit, HFC acknowledged this error. After the audit, HFC contacted its contractor to correct the maximum acceleration in the test report to be 27.85 g.

4. Exit Meeting During the exit meeting, the NRC staff provided a summary of audit activities performed and provided several observations of HFC practices and comments on the documents reviewed. A listing of documents that were placed on the HFC Sharepoint for NRC audit review is included as Section 6.0 to this report. The NRC staff requested that these documents be left on Sharepoint to support the ongoing evaluation process until the draft SE is completed. The NRC identified one open audit item for EQ summary report inconsistencies and requested that the revised EQ summary report that included necessary corrections that is on the Sharepoint be submitted on the docket to support SE completion. A RAI has already been sent to HFC requesting this document.

No additional RAIs are expected to be issued based on the results of this audit.

5. Audit Objectives Achieved Verification and Validation- By conducting several requirements thread reviews, the NRC staff was able to confirm the degree to which HFC-FPGA Platform software V&V program meets the criteria outlined in the HFC-FPGA V&V Plan which was developed in accordance with IEEE Std. 1012, IEEE Standard for Software Verification and Validation.

Configuration Management - By reviewing the configuration management documentation, the NRC staff was able to determine the degree to which the HFC configuration management processes include control measures for both hardware and software configuration management.

Secure Development Environment - The NRC staff reviewed information pertaining to the HFC-FPGA platform development environment. The results of this review activity will be used to determine conformance to the secure development environment requirements of RG 1.152, Revision 3.

Diagnostics and Watchdog Function Discussion - During this discussion, the HFC draft response to the NRC RAIs was discussed and several clarifications to the Watchdog function technical details in the HFC response were provided. The NRC staff indicated that the responses were understood but that a non-proprietary summary of the responses should be prepared so that a public version of the evaluation could be supported. HFC subsequently placed a revised version of RAI responses onto the Sharepoint which included non-proprietary responses as requested.

These responses will be submitted on the docket in a subsequent supplement for staffs review.

Equipment Qualification - During the audit, the NRC staff conducted requirements thread tracing, had discussions with the HFC personnel, and performed independent sample checks on test programs, procedures, specifications, and records. Except for the findings and observations stated above, the NRC staff was able to confirm sampled testing results presented in the HFC-FPGA TR and supplemental summary test report.

Software Safety Plan - The HFC Software Safety Plan (SSP), PP004-000-01 Rev. D was reviewed by the NRC staff. This document is an updated version of the SSP that was evaluated for the HFC-6000 platform. The NRC staff reviewed changes made to the SSP and confirmed that the scope has been expanded to include the HFC-FPGA platform and FPGA logic development.

6. Documents Reviewed During Audit (on HFC Sharepoint):

Parts in EQ HFC-FPGA Master Document Lists 201612073680, Design Verification and Review Form RR901-000-74, HFC-6000 FPGA Security Concept, Rev B SDOE for HFC-FPGA Platform-NRC Audit Material NRC Audit FPGA Development Tools HFC-FPGA Platform Requirement Traceability 2020-04-15-HFC Amendment 4 RAIs-Draft-Response TN901-000-03, Rev A, HFC-FPGA Amendment 4 FPGA HARD BLOCK QUALIFICATION CR2020-0092, CAIF2020-0092 (Corrective Action Implementation Form or CR resolution) and TS901-117-03, Rev. D PP004-000-01, Software Safety Plan, Revision D Schematics:

Schematic Wiring Diagram 71005901Q Schematic Wiring Diagram 71005902Q Condition Reports:

QPP-16.1 Corrective Action Program EQ Condition Reports EQ CR resolutions sample CR2020-0082 - Condition report generated as result of audit issues regarding requirements traceability process System Change Requests:

SCR4088 - DC34 (FPGA) MOSFET Error/Slave Initial Mux Error SCR4039 - HFC-DI32IG with Quality Status No Diagnostic Report SCR3967 - AFS-SBC01 Controller halt/reboot during IOX-02 hot unplug/plug SCR3097 - Spacer missing from BOM on AFS-AS-01 assembly 32960802 Test Procedures:

Test Record TP901-115-05, Integration Test Procedure Test Record TP901-115-02, Prudency Test Procedure for Pre-test Test Record TP901-115-01, Operability Test Procedure for Pre-test Test Record TP901-302-02, VV0115 Environmental Stress Test Procedure Test Record TP901-115-01, Operability Test Procedure for high temperature op check Test Record TP901-115-02, Prudency Test Procedure for high temperature op check Test Record TP901-115-01, Operability Test Procedure for low temperature op check Test Record TP901-115-02, Prudency Test Procedure for low temperature op check Test Record TP901-115-01, Operability Test Procedure for ambient temperature op check Test Record TP901-115-02, Prudency Test Procedure for ambient temperature op check

Test Record TP901-115-01, Operability Test Procedure for Post- Environmental tests Test Record TP901-115-02, Prudency Test Procedure for Post- Environmental tests Test Record TP901-302-03, VV0115 Qualification System EMI-RFI Test Procedure Test Record TP901-302-04, VV0115 Qualification Sys Surge Withstand Test Procedure Test Record TP901-302-05, VV0115 Qualification System ESD Test Procedure Test Record TP901-115-01, Operability Test Procedure for Post-EMI tests Test Record TP901-115-02, Prudency Test Procedure for Post-EMI tests Test Record TP901-302-06, VV0115 Qualification Seismic Test Procedure Test Record TP901-115-01, Operability Test Procedure for Seismic tests Test Record TP901-115-02, Prudency Test Procedure for Seismic tests Test Record TP901-115-01, Operability Test Procedure for Post-Seismic tests Test Record TP901-115-02, Prudency Test Procedure for Post-Seismic tests Test Record TP901-115-06, TSAP Validation Test Procedure Test Record TP901-115-04, Application Software Object Test Procedure Test Record TP901-200-01, Burn In Test Procedure Test Record TR901-117-03, Rev. B, V&V Hardware Component Test Report Test Specifications:

TS901-117-13, Rev B, Software/Hardware Integration Test Procedure TS901-117-17, Rev. C, V&V Component Test Specification TS901-117-03, Rev. C, V&V Hardware Component Test Procedure TS901-117-03, Rev. D, V&V Hardware Component Test Procedure TS901-002-31, HFC-FPUD01, Prototype Test Procedure Rev B Test Report:

TR901-117-13, Rev. B, V&V Hardware Component Test Report TR901-302-02, HFC-FPGA Equipment Qualification Summary Test Report, Rev. B RR901-000-49, HFC-6000 ERD111-ERD921-SLC EMI-RFI Summary Test Report, Rev. B Requirement Specifications:

RS901-001-74, HFC-6000 FPGA Platform Diagnostic Requirement Specification, Rev B RS901-001-48, HFC-FPUA_Requirement_Specification_Rev._D RS901-001-14, HFC-HSIM Hardware Requirements Specification, Rev E RS901-001-43, HFC-FPUD Requirement Specification Rev C RS901-001-69, HFC-FPUAO_Requirements_Specification,Rev_C RS901-001-70, HFC-FPUM_RTD_Requirement_Specification,Rev_C RS901-001-71, HFC-FPUL_Hardware_Requirement_Specification,_Rev B RS901-001-85, HFC-FCPUX Hardware Requirement Specification Rev. B RS901-001-86, HFC-FPUM2 Requirement Specification, Rev A RS901-002-03, HFC-FPC08 FPGA Gateway Hardware Requirement Spec Rev B RS901-003-04, HFC-FCPU Hardware Requirement Specification Rev. B

Design Specification:

DS001-007-02, HFC-6000 FPGA System IP Core Design Specification, Rev C Work Instructions:

WI-ENG-830, Source Code Review, Rev E WI-ENG-003, Configuration Management, Rev. K WI-ENG-025, Requirements Traceability Matrix Verification Procedure, Rev A WI-VV-004, Requirement Traceability Matrix and Traceability Analysis WI-DOC-001, Document Distribution, Rev K

7. Virtual Audit Notes and Lessons Learned Though objectives of the audit as defined in the audit plan were achieved, there were some limitations and difficulties associated with the virtual nature of this audit that are worth noting.

The product demonstration was somewhat limited and some activities that are normally conducted at the request of the NRC staff were not performed. Examples include observation of system response to module removal, demonstration of attempted re-configuration, and observation of system performance when an intentional configuration misalignment was induced.

Normal observance of engineering workstation capabilities during system operation was also limited.

The audit relied more heavily on documentation reviews than would be the case for a normal in-person audit. Because of this, there was less personal interaction and discussion of how procedures were being interpreted by HFC staff and the NRC found it difficult to assess the work practices being used.

Because of the inability to access and observe the HFC facilities, the NRC staff had to rely on diagrams and verbal explanations of the work environment and security measures being used.

The NRC staff couldnt use the ReqTracer tool used by HFC to conduct independent audit on requirement traceability because the ReqTracer tool was not available for the NRC staff to use remotely. To compensate, the application of the ReqTracer tool was demonstrated during the virtual audit and was used to confirm subsequent traces.

There was no personal interaction or discussion with the HFC testing contractors who conducted majority of the EQ tests.

Because of the limitations cited above, it will likely be necessary to docket additional information to support clarification or confirmation of license applications when conducting audit activities virtually as compared to in-person audit activities.

8. List of Audit

Participants:

Name Affiliation Tile Areas of Responsibility Richard Stattel NRC Sr. Electronics Engineer Lead Technical Reviewer and Audit Lead Jack Zhao NRC Sr. Electronics Engineer Technical Reviewer for EQ and Audit Member Yang Lu HFC Licensing Lead Licensing Yin Guo HFC V&V Lead V&V, EQ, and Cyber Security David Briner HFC Sr. V&V Engineer Hardware V&V and EQ Plan/Testing Michael Hanson HFC V&V Engineer Requirements Tracing Gregory HFC R&D Manager Configuration Control and SW Tools Rochford Jordan Mott HFC FPGA Chief Engineer FPGA Design and Component Testing Huaisong Xu HFC System Engineering System Design and FPGA Architecture Manager William Luo HFC Hardware Chief Engineer Hardware Design and FPGA Architecture John Maeng HFC QA Manager QA Program and Corrective Action Program Jack Sun HFC QA Engineer QA Engineer Martha HFC QC Manager QC Inspection and Final Quality Verification Edmonson Documentation List (QVDL)

Mark Burzynski Consultant HFC Consultant HFC Licensing Consultant Steve Yang HFC Sr. Vice President Strategic Planning, Licensing, V&V, and US Marketing