ML18144A470

SMR DC RAI - Request for Additional Information No. 482 Erai No. 9395 (18)

ML18144A470
Person / Time
Site:	NuScale
Issue date:	05/24/2018
From:	NRC
To:	NRC/NRO/DNRL/LB1
References
Download: ML18144A470 (6)

Text

NuScaleDCRaisPEm Resource From: Cranston, Gregory Sent: Thursday, May 24, 2018 8:34 AM To: Request for Additional Information Cc: Lee, Samuel; Chowdhury, Prosanta; Kent, Lauren; D'Agostino, Amy; NuScaleDCRaisPEm Resource

Subject:

Request for Additional Information No. 482 eRAI No. 9395 (18)

Attachments: Request for Additional Information No. 482 (eRAI No. 9395).pdf Attached please find NRC staffs request for additional information (RAI) concerning review of the NuScale Design Certification Application.

Please submit your technically correct and complete response within 60 days of the date of this RAI to the NRC Document Control Desk.

If you have any questions, please contact me.

Thank you.

1

Hearing Identifier: NuScale_SMR_DC_RAI_Public Email Number: 511 Mail Envelope Properties (BN3PR09MB0355F5DCE00EA640FA2D363D906A0)

Subject:

Request for Additional Information No. 482 eRAI No. 9395 (18)

Sent Date: 5/24/2018 8:34:07 AM Received Date: 5/24/2018 8:34:14 AM From: Cranston, Gregory Created By: Gregory.Cranston@nrc.gov Recipients:

"Lee, Samuel" <Samuel.Lee@nrc.gov>

Tracking Status: None "Chowdhury, Prosanta" <Prosanta.Chowdhury@nrc.gov>

Tracking Status: None "Kent, Lauren" <Lauren.Kent@nrc.gov>

Tracking Status: None "D'Agostino, Amy" <Amy.D'Agostino@nrc.gov>

Tracking Status: None "NuScaleDCRaisPEm Resource" <NuScaleDCRaisPEm.Resource@nrc.gov>

Tracking Status: None "Request for Additional Information" <RAI@nuscalepower.com>

Tracking Status: None Post Office: BN3PR09MB0355.namprd09.prod.outlook.com Files Size Date & Time MESSAGE 363 5/24/2018 8:34:14 AM Request for Additional Information No. 482 (eRAI No. 9395).pdf 92448 Options Priority: Standard Return Notification: No Reply Requested: No Sensitivity: Normal Expiration Date:

Recipients Received:

Request for Additional Information No. 482 (eRAI No. 9395)

Issue Date: 05/24/2018 Application

Title:

NuScale Standard Design Certification 048 Operating Company: NuScale Power, LLC Docket No.52-048 Review Section: 18 - Human Factors Engineering Application Section: 18 QUESTIONS 18-47 Regulatory Basis Title 10 of the Code of Federal Regulations (10CFR) Section 52.47(a)(8) requires an applicant for a design certification to provide a final safety analysis report (FSAR) that must include the information necessary to demonstrate compliance with any technically relevant portions of the Three Mile Island requirements set forth in 10 CFR 50.34(f), except paragraphs (f)(1)(xii), (f)(2)(ix), and (f)(3)(v). Section 10 CFR 50.34(f)(2)(iii) requires an applicant to "Provide, for Commission review, a control room design that reflects state-of-the-art human factor principles prior to committing to fabrication or revision of fabricated control room panels and layouts." Chapter 18, "Human Factors Engineering," of NUREG-0800, "Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants: LWR Edition," and NUREG-0711, "Human Factors Engineering Program Review Model,"

identify criteria the staff uses to evaluate whether an applicant meets the regulation. The applicant stated in the FSAR, Tier 2, Section 18.0, "Human Factors Engineering - Overview," that its human factors engineering (HFE) program incorporates accepted HFE standards and guidelines including the applicable guidance provided in NUREG-0711, Revision 3.

Background Information By letter dated December 31, 2016, NuScale submitted a design certification application (DCA) for review (Agencywide Documents Access and Management System (ADAMS) Accession No. ML17013A229). The DCA includes Design Control Document (DCD) Tier 2, Chapter 18, "Human Factors Engineering," which describes the human factors engineering (HFE) program for the NuScale Power Plant (ADAMS Accession No. ML17013A289). As part of DCD Chapter 18, NuScale submitted an implementation plan (IP) for the HFE verification and validation (V&V) element. In a letter dated April 8, 2016, from Mr. Thomas Bergman, NuScale, Vice-President, Regulatory Affairs; to Mr. Frank Akstulewicz, Director, Division of New Reactor Licensing, NRC (ADAMS Accession No. ML16099A270); NuScale stated that NuScale would submit a V&V results summary report (RSR) prior to Phase 4 of the NRC's review of the NuScale DCA. Based on NuScale's response to RAI 9123 dated January 10, 2018 (ADAMS Accession No. ML18002A554), the staff understands NuScale plans to commence integrated systems validation (ISV) testing, a significant V&V activity, mid-year.

NUREG-0711, Section 1.2.2, "Review Elements," says, "An IP review gives the applicant the opportunity to obtain an NRC staff review of, and concurrence with the methodology before the applicant conducts the work associated with the element. This type of review is desirable from the NRC staff's perspective because it offers the staff an opportunity to identify issues with the methodology and provide input early in the analysis or design process when the applicant more easily can address staff concerns than when the element is completed." The response to RAI 8747, Question 18-11 provided by NuScale on January 10, 2018 (ADAMS Accession No. ML17354A845), says, "NuScale submitted the V&V IP in order that NRC staff better understand the methodology used by NuScale during performance of V&V activities described in NUREG-0711. NuScale will not include a detailed description of the V&V methodology in the V&V RSR unless the methods actually used during V&V activities differ from those described in the V&V IP."

Consistent with NUREG-0711, Section 1.2.2, the staff commenced a review of the V&V IP and observed in some cases, the document did not contain enough information for the staff to assess how the V&V methodology conforms to the staff's guidance in NUREG-0711, Section 11.4, "Review Criteria." The staff conducted an audit of non-docketed information that contained more detail on NuScale's proposed V&V methods, including the "ISV Test Plan," as described in the audit plan dated July 25, 2017 (ADAMS Accession No. ML17205A465). As stated in the audit plan, the purpose of the audit was to review non-docketed information related to V&V methods to evaluate conformance with regulatory guidance and identify information that will require docketing to support the basis of the regulatory decision.

The following questions resulted from this review. The questions are categorized into two types. The first category identifies instances where the V&V IP either does not provide any information or enough detail for the staff to evaluate conformance with regulatory guidance, and the information was not found in the non-docketed material reviewed during the audit. The second category includes questions where the V&V IP and the non-docketed material are not consistent, and therefore it is not clear to the staff what method(s) NuScale intends to use. In order to provide the greatest amount of regulatory certainty for NuScale on the proposed ISV methodology, the staff is requesting the information listed below to complete the review of the methodology prior to NuScale commencing ISV testing.

The questions to follow relate to the staff's review criteria cited here:

x Criteria 11.4.3.5.1 (1-6) of NUREG 0711, states applicants should identify, 1) plant performance measures

2) primary task measures, 3) secondary task measures, 4) situation awareness measures, 5) workload measures and 6) anthropometric/physiological measures for each ISV scenario.

x Criteria 11.4.3.5.2 (1-5) of NUREG-0711 states that the applicant should expand on these measures and the validation criteria associated with them.

Questions Category 1: Lack of Information

1. In the Final Safety Analysis Report (FSAR), Section 18.10.2.3.5.1, the applicant identified "non-intrusive human performance measures" as one way that situation awareness will be measured. The staff referenced NUREG/CR-7190, "Workload, Situation Awareness, and Teamwork" in evaluating the method. NUREG/CR-7190 cites three types of implicit metrics, each which has advantages and limitations to consider as part of ISV test design. It is unclear to staff what type of implicit measure will be used, is being used and thus staff cannot evaluate its conformance to regulatory guidance.

a. Please clarify whether the non-intrusive measure will be part of the operator's primary task or unrelated to the primary task (e.g., an alarm sounds in the main control room that is unrelated to the current task).

b. If it is part of the primary task, please explain how the relationship between task performance and situation awareness is determined as performance can be caused by factors other than situation awareness (e.g. procedure problems, teamwork, etc.).

2. In Section 4.5.1.3, bullet 2 of the V&V IP, the applicant identifies a measure of situation awareness.

a. Please explain how the information collected relates to operator situational awareness.

b. Please explain why this is a valid measure of situation awareness.

3. Criteria 11.4.3.5.2 (3) states, "The applicant should describe the characteristics (see Table 11-1 of NUREG-0711) of the performance measures" (construct validity, reliability, sensitivity, unobtrusiveness, objectivity). In Section 4.5.2.2 of the V&V IP, Rev 4, the applicant states, "Performance measures to be observed during ISV contain the characteristics described." This statement is followed by Table 11-1 of NUREG-0711 with no further information provided. Please provide a description of all characteristics identified in Table 11-1 of NUREG 0711 for each performance measure.

4. Criterion 11.4.3.5.2 (5) states that, "The applicant should identify whether each measure is a pass/fail one or a diagnostic one." In Section 4.5 of the V&V IP, Rev 4, the applicant states "Performance measures for ISV are hierarchical and include measures of plant performance, personnel task performance, SA, cognitive and physical workload, and anthropometric or physiological factors. Both pass or fail and diagnostic measures are applied." For each performance measure identified, please specify whether the measure will be pass/fail, diagnostic or both.

5. Criterion 11.4.3.5.2 (4) states, "The applicant should identify the specific criterion for each measure used to judge the acceptability of performance and describe its basis (Requirement, benchmark, norm, expert judgment)." While the staff understands that some specific details of certain criterion will not be available until the Integrated System validation scenarios are finalized (e.g. numeric performance thresholds, time requirements), the

staff requests that NuScale provide additional information to clarify the measurement approach in order to assess the adequacy of the V&V IP.

a. For those measures that will be obtained via observation please identify the types of behaviors that observers will be looking for.

b. For those measures that will be obtained via observation, identify any measurement tools they will use (e.g. stopwatch for timing a behavior, checklists, etc.)

c. For those measures that will be obtained via observation, clarify how expert observers will know what to look for (e.g. use of briefing booklets)

d. Section 4.5.2.2 of the V&V IP states, "The basis for inclusion of a performance criterion in the ISV (or a particular scenario within ISV) used to judge acceptability of that criterion is determined during the development of the scenario." While the staff understands that precise thresholds for acceptability for many performance measures will not be available until Integrates System Validation scenarios have been finalized, the basis for those thresholds is needed to understand the measurement approach. In Section 4.5.2.1 of the V&V IP, Rev 4, the applicant provides some generic information regarding basis such as, "Objective data (e.g., video recording, administrator observations) collected during test scenarios are analyzed to assess impacts of operator actions on plant processes and equipment states. The analysis compares the performance derived from parameters and times collected by the test bed to the evaluation criteria for operator actions and for overall plant process behavior developed for each scenario." However, it is not clear which specific measures are being referred to, nor whether "evaluation criteria for operator actions and for overall plant process behavior" is based on a requirement (e.g.

engineering analysis), a norm, expert judgment or a benchmark.

For each performance measure, please describe the basis that will be used (e.g. requirement, benchmark, norm, expert judgment) to establish the specific criterion used to judge the acceptability of performance.

Category 2: Information in the V&V IP and ISV Test Plan are different

6. The staff reviewed information regarding performance measures via an audit of documents in the NuScale electronic reading room (the audit plan is available as ADAMS Accession No. ML17205A465), including the "Integrated Systems Validation Test Plan." Section 11.0 of this document discusses a measure of situation awareness that is not discussed in the FSAR or the V&V IP, Rev 4.

Please clarify whether the measure of situation awareness cited in Section 11 of the "Integrated Systems Validation Test Plan" will be used during integrated system validation testing and, if so, update the application accordingly.

7. In the FSAR, Tier 2, Section 18.10.2.3.5.1, the applicant states, "To measure cognitive workload, the ISV employs questionnaires and observations of operators' ability to gather specific plant information, and crew performance." Based on a staff audit of documents in the NuScale electronic reading room (the audit plan is available as ADAMS Accession No. ML17205A465), specifically Section 11 of "Integrated Systems Validation Test Plan," in addition to audit discussions with the applicant, the staff understood that the applicant intends to use a widely-accepted industry standard questionnaire to assess workload. However, this workload measure was not identified by name in either the Tier 2 information or in the V&V IP.

a. Please clarify whether the named questionnaire in section 11 of the Integrated Systems Validation Test Plan is the questionnaire being used to assess workload during Integrated Systems Validation scenarios and update the application accordingly.

b. If an alternate measure is being used, please explain why it is a valid measure of workload.

8. Criteria 11.4.3.5.2 (1) states, "The applicant should specify when each measure is obtained (recorded), such as continuously, at specific points during the scenario, or after the scenario ends." In section 4.5.1.4 of the V&V IP, Rev 4, bullet 1, the applicant describes when the identified measure of workload will be obtained. However, in Section 11 of the "Integrated Systems Validation Test Plan," there is conflicting information about when the measure is obtained.

a. Please clarify when the measure will be obtained and update the application accordingly.

b. If the measure is being obtained in the manner identified in Section 4.5.1.4 of the V&V IP, Rev 4, please describe the method used to determine the timing of the data collection.