ML22231B053

Comment (6) of James E. Slider on Nrc'S Fiscal Years 2023-2027 Artificial Intelligence Strategic Plan

ML22231B053
Person / Time
Site:	Nuclear Energy Institute
Issue date:	08/19/2022
From:	Slider J Nuclear Energy Institute
To:	Office of Administration
References
87FR39874 00006, NRC-2022-0095
Download: ML22231B053 (8)

Text

8/19/22, 1:55 PM blob:https://www.fdms.gov/90dc7c29-b7f8-4b72-a48d-8f5bbee44b37 SUNI Review Complete Template=ADM-013 E-RIDS=ADM-03 As of: 8/19/22, 1:52 PM Received: August 19, 2022 PUBLIC SUBMISSION ADD: Matthew Dennis, Tray Hathaway, Mary Status: Pending_Post Tracking No. l70-lox4-xuwt Neely Comment (6) Comments Due: August 19, 2022 Publication Date:

7/5/2022 Submission Type: Web Citation: 87 FR 39874 Docket: NRC-2022-0095 NRCs Fiscal Years 2023-2027 Artificial Intelligence Strategic Plan Comment On: NRC-2022-0095-0001 NRCs Fiscal Years 2023-2027 Artificial Intelligence Strategic Plan Document: NRC-2022-0095-DRAFT-0007 Comment on FR Doc # 2022-14239 Submitter Information Email: atb@nei.org Organization: Nuclear Energy Institute General Comment See attached file(s)

Attachments 08-19-2022_NRC_Comments on Draft AI Strategic Plan blob:https://www.fdms.gov/90dc7c29-b7f8-4b72-a48d-8f5bbee44b37 1/1

JAMES E. SLIDER Technical Advisor, Innovation & Regulatory Affairs 1201 F Street, NW, Suite 1100 NUCLEAR ENERGY INSTITUTE Washington, DC 20004 P: 202.739.8015 jes@nei.org nei.org August 19, 2022 Office of Administration Mail Stop: TWFN-7-A60M U.S. Nuclear Regulatory Commission Washington, DC 20555-0001 ATTN: Program Management, Announcements and Editing Branch

Subject:

NEI Comments on NRCs Draft Artificial Intelligence Strategic Plan (Docket ID NRC-2022-0095)

Submitted via www.regulations.gov Project Number: 689 Ladies and Gentlemen:

This letter responds to the U.S. Nuclear Regulatory Commissions July 5, 2022, solicitation 1 of public 0F comments on the NRCs draft Artificial Intelligence Strategic Plan. 2 The Nuclear Energy Institute (NEI) 3 1F 2F appreciates this opportunity to comment on the draft strategic plan and the opportunity to comment during the August 3, 2022, public meeting 4 held for the same purpose. The draft strategic plan is written at a very 3F high level and thus lacks many of the details that the industry is eager to see. Nevertheless, in the attachment we offer comments and questions to help clarify the strategic plan to ensure that we fully understand its intent.

We applaud the NRCs efforts to prepare for industrys use of artificial intelligence (AI). We believe that AI offers the potential to improve safety in operating nuclear power plants and in advanced reactors now being developed. We believe the NRC should do everything it can to enable the timely development of AI and other innovations that would enhance safety, security, and efficiency in nuclear facilities. Preserving and enhancing the operating fleet and facilitating the entry of advanced reactors into the energy mix are essential for the nation to achieve its carbon reduction goals.

1 87 FRN 39874-39876, July 5, 2022, NRCs Fiscal Years 2023-2027 Artificial Intelligence Strategic Plan.

2 Artificial Intelligence Strategic Plan: Fiscal Years 2023-2027, NUREG-2261, June 2022, ADAMS ML22175A206.

3 The Nuclear Energy Institute (NEI) is responsible for establishing unified policy on behalf of its members relating to matters affecting the nuclear energy industry, including the regulatory aspects of generic operational and technical issues. NEIs members include entities licensed to operate commercial nuclear power plants in the United States, nuclear plant designers, major architect and engineering firms, fuel cycle facilities, nuclear materials licensees, and other organizations involved in the nuclear energy industry.

4 Meeting information is available at https://www.nrc.gov/pmns/mtg?do=details&Code=20220681.

NUCLEAR. CLEAN AIR ENERGY

Office of Administration August 19, 2022 Page 2 We see the operating plants currently pursuing initial applications of AI to improve safety, reliability, and efficiency. Typical of these applications is AI deployed in decision-support tools to streamline administrative work processes such as initial screening of condition reports and work requests. In these roles, AI helps plant staff focus on high-value work that is more important and results in more efficient plant operations. To be clear, it is expected that a human will remain in the decision-making loop at the plant and in the design process and that the human will be able to leverage the benefits of insights provided by the AI. Experience with AI in these administrative support roles is helping to build knowledge and expertise among the developers and users involved. At this time, our best information indicates that the operating plants intend to focus on such non-safety-related uses of AI for several years to come.

Application of AI in safety-related roles that could warrant NRC review appears to us to be several years away, based on publicly available information about the industrys plans. When a licensee is ready to consider a safety-related application of AI, the licensee can turn to existing NRC guidance on software and computer programs to understand what the NRC will expect to see in the design, testing, and verification and validation of the AI involved. Until then, the NRC should focus on establishing fundamentals (such as definitions and a common language), building on existing guidance to address what distinguishes AI from existing software, and learning how to make decisions on AI applications that are compatible with the speed at which AI is evolving.

Finally, it is essential for the NRC to engage industry and other external stakeholders early and often throughout the agencys work on AI particularly when developing guidance. Recent projects to revise guidance on operability determinations and to enhance engineering inspections included a high level of public engagement prior to being finalized. We think a similar level of engagement on AI is important to ensure the results strike the right balance between assuring adequate protection and stifling AI-based innovations that would benefit public safety and security.

If you have questions concerning our comments, please contact me.

Sincerely, James E. Slider Attachment c: Mr. Matthew Dennis, RES/NRC Mr. Luis Betancourt, RES/NRC Ms. Teresa Lalain, RES/NRC Mr. Chris Miller, DRO/NRR/NRC Mr. Bo Pham, DORL/NRR/NRC

Office of Administration August 19, 2022 Page 3 Attachment Comments on Draft Artificial Intelligence Strategic Plan The following comments apply to the draft NUREG-2261, Fiscal Years 2023-2027 Artificial Intelligence (AI)

Strategic Plan, issued June 2022, ADAMS ML22175A206. The comments are presented in the order in which the relevant sections appear in the draft document.

1. Executive Summary (page ix)

a. Timeframe of this Plan - In line 17, as in the title of the report, the text says this strategic plan covers fiscal years 2023-2027. Despite that, the remainder of the strategic plan gives no indication of the timing or sequence of any elements of the work that would be subject to this plan. What are the order and time frames in which major elements of the strategic plan would be performed?

b. Short/Mid/Long-Term Actions - In lines 28-31, the text speaks of actions recommended for the short-term (<1 year), mid-term (1-3 years), and long-term (3-5 years) time horizons These time horizons are not elucidated anywhere else in the document. (Note that the same three time-horizons are mentioned again in Section 4, on page 4-2, in lines 1-3, but not explained there either.)

i. What is the relevance of these three time-horizons to the strategic plan? What specific actions are contemplated in each of the three time-horizons?

ii. What are the short-term actions that must be completed within one year? During the August 3, 2022, public meeting, the NRC staff said that the strategic plan is to be finalized in Spring 2023. 5 That would be halfway through the first year in the scope of 4F the strategic plan. When does the clock start on actions that should be completed within the one-year short-term window?

2. Section 1, Introduction (pages 1-1 to 1-3)

a. Definition of AI. Mentioned in the Executive Summary, the Introduction, and the Glossary is a definition of AI. Establishing an agency-wide definition of AI is important to this strategic plan and the NRCs next steps on preparing for reviewing AI applications. We would encourage the NRC to begin by defining key terms and concepts so that discussions and future applications proceed with a common language. We would welcome the opportunity to engage with the NRC on the development of this common language to avoid future misunderstandings on nomenclature.

b. Opportunities to Improve Safety - On page 1-1, lines 23-29, the text says, AI provides new opportunities for organizations to enhance safety and security, improve processes, leverage historical and current data, identify research needs, and even explore autonomous control and operationThe NRC is committed to enabling the safe and secure use of new technologies, especially those that can enhance the safety and security of nuclear facilities. We agree that AI offers great potential to improve safety and security. We encourage the NRC to use its work on AI as a catalyst that supports efficient NRC decision-making on all technological innovations that 5

See Slide #13 in staff presentation, 08/03/2022 NRC AI Strategic Plan Public Meeting Presentation. ADAMS ML22213A273.

Office of Administration August 19, 2022 Page 4 benefit safety and security. Timely decision-making will be especially important in the rapidly evolving area of AI.

c. Levels of AI. On page 1-3, between lines 3 and 4, Table 1 describes four levels of AI. It implies, but does not explicitly state, which of these levels might warrant an NRC regulatory decision. The NRC should strive to delineate the level of NRC involvement in licensee uses of AI corresponding to each of the four levels. For example, the description of Level 1 implies that NRC would have little or no need to weigh in on such AI uses. The NRC might have interest in such AI uses just for awareness. By contrast, for AI Level 4, it seems clear that NRC could have a more substantial regulatory role. Other situations might be less clear. For example, an AI use at one of the lower levels that involves forward-looking modeling could conceivably invite greater regulatory scrutiny than is apparent in the Table 1 descriptions. In other words, while Table 1 suggests the level of NRC involvement in an AI use might be determined by a single criterion (e.g., the level of autonomy), in fact multiple criteria might be necessary to determine the level of NRC involvement appropriate to each level of AI use.

d. Could. The descriptions of AI Levels 2-4 use the word could, i.e., could impact safety. The NRC should be more specific in distinguishing Levels 2-4 than the all-encompassing word, could.

3. Section 2, Vision (page 2-1)

a. Ensure. The stated vision is, The NRCs vision is to continue to keep pace with technological innovations to ensure the safe and secure use of AI in NRC-regulated activities. [Emphasis added.] We respectfully suggest the NRCs vision should be to enable the safe use of AI, not to ensure. The licensee is responsible for the safety and security of its licensed activities; thus, it is the licensee who ensures the safety and safe use of AI in its licensed facility. The NRCs role is to oversee the licensees design and implementation of AI and verify that the AI meets regulatory requirements. Equally important, however, is the NRC striking the balance needed to regulate appropriately to assure adequate protection but not excessively to the point of stifling AI-based innovations that would benefit public health and safety. (This comment about the term ensure also applies to the description of the ultimate outcome of Strategic Goal 1 mentioned in Section 4, page 4-1, lines 7-10, which repeats the NRC Vision statement.)

4. Section 3, Purpose and Drivers (page 3-1)

a. Implications of the Five-Year Horizon: The NRC assumes that licensing applications that include the use of AI technologies may be submitted for NRC review and approval in the next five years. 6 5F

i. What is significant about this five-year horizon to the NRC?

ii. How does this horizon relate to the NRCs short-term actions (<1 year), mid-term actions (1-3 years), and long-term actions (3-5 years) mentioned in the Executive Summary and Section 4? 76F 6

NUREG-2261, page 3-1, lines 7-8.

7 NUREG-2261, page ix, lines 28-29 and pages 4-1 (line 29) and 4-2 (lines 1-3).

Office of Administration August 19, 2022 Page 5 iii. Does the five-year horizon mean that all the work identified in this strategic plan must be completed within five years? If so, what is that scope of work?

b. Operating Plant Horizon: NEI discussions with a cross-section of operating plant representatives suggest it is unlikely that they will be ready to submit license applications involving AI technologies in the timeframe contemplated in the strategic plan. This answer is highly uncertain due to the varying levels of interest in AI, the rapid evolution of AI technologies, and differing tolerances for business risk among the operating plant owners. Operating plant companies want to gain experience with non-safety uses of AI (for example, in Balance of Plant functions and business system functions), develop expertise and confidence in the use of AI and understanding how to leverage AI, and advance the digitization of plant systems and plant data to the degree necessary to enable effective use of AI.

i. If the horizon for the first operating plant license amendment request involving AI is more than five years away, how would that change the NRCs plans described in NUREG-2261?

c. Advanced Reactors Horizon: We expect the panoply of advanced reactor developers to take a variety of approaches on incorporating AI technologies in their designs and operations. Some may try to push the envelope, looking for opportunities to leverage AI technologies to the maximum extent in their design and initial licensing. Others may take more conservative approaches that avoid the uncertainties of seeking NRC approval of an AI application during initial licensing review. NEI is seeking more information from the advanced reactors community to better understand what those plans look like. In the meantime, the NRC should look at its existing requirements and guidance 8 on software and programs as the starting point for 7F evaluating potential AI applications.

5. Section 4, Strategic Goals (pages 4-1 to 4-5)

a. Engaging with stakeholders. On page 4-1, in lines 16-17, the text says, The NRC is committed to engaging the industry and relevant stakeholders to maintain awareness of industry efforts (AI Strategic Goal 3) and prepare for regulatory reviews. The wording implies this engagement is important (or most important) in Strategic Goal 3. We agree that NRC engaging with industry and other stakeholders is vital in Strategic Goal 3, but we would emphasize that engagement is just as important in the other Strategic Goals as well. We strongly encourage the NRC to approach every aspect of developing its capability to make regulatory decisions pertaining to AI with a commitment to engaging with industry and stakeholders at all stages. By engaging stakeholders through all the NRC Strategic Goals, the NRC is more likely to foster buy-in and support from the public and the regulated industry.

i. During the August 3, 2022, public meeting, the NRC staff expressed an expectation that significant industry engagement will also occur through pilot studies mentioned under Strategic Goal 5 (page 4-5, line 14 of the plan). The NRC appears to be counting on pilot studies to help determine the depth of NRC review needed in AI applications. The NRC said pilot studies would be discussed during a proposed AI workshop in 2023 (date to be determined). However helpful pilot studies may be, the timing of pilot studies is unknown and cannot be predicted with confidence. Hence, pilot studies must be 8

For example, Regulatory Guide 1.168, Verification, Validation, Reviews, and Audits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants, Revision 2, issued July 2013, ML13073A210

Office of Administration August 19, 2022 Page 6 considered a complement to, not a substitute for, direct engagement with industry as soon as practical and as often as necessary to advance the development of appropriate regulatory guidance.

6. Strategic Goal 1 - Ensure NRC Readiness for Regulatory Decision Making

a. Consideration or Oversight. On page 4-2, in lines 13-16, the text says, The NRC anticipates that within the next five years an existing licensee, new, or advanced nuclear technology applicant may employ AI in such a manner that it requires NRC regulatory consideration or oversight.

i. What is significant about use of the term regulatory consideration or oversight here instead of saying, for example, a regulatory decision?

b. Timing. Does the NRC intend to complete all the work derived from Strategic Goal 1 within five years?

c. Deadline. When would NRC be ready to review an application with an AI component?

d. Pace of Developments. Licensees are utilizing project management methods, such as AGILE, to determine more quickly whether a given approach is successful and, if it is not, quickly learn from the failure and adapt. The technologies of artificial intelligence are rapidly evolving as well.

This means that the NRCs pace of development and pace of its future regulatory decisions on AI applications must be correspondingly swift. If the NRC takes one to two years to review a license application involving an AI tool, the NRCs slow pace will impair the industrys pursuit of beneficial AI tools and potentially, new reactor design development since it is expected that AI will have a significant influence on the new designs. The AI strategic plan does not indicate how the plan will enable more timely and efficient decisions on AI cases than is the norm today for ordinary license amendment requests.

e. Fees. The AI strategic plan does not hint at when or how the NRC will address the impact that review fees could have on licensee business cases for AI projects. As the NRC further develops its thinking on AI, we would encourage that the review fees do not inhibit the development of AI applications that could benefit public safety and security. Consideration should be given to the use of fee waivers until the review of AI applications becomes routine and predictable. At that point, we would encourage the NRC to consider setting a fixed price or cap on review fees for AI applications.

f. Metrics. How will NRC and industry know that the NRC is ready to perform its first regulatory review? Said differently, what are the NRCs measures of success for the work implied by Strategic Goal 1? Assuming test cases will be developed, will the NRC consider doing a dry run of their regulatory review on a test case? This would clearly provide industry a path forward particularly if the amount of effort associated with the review of the test case is included in a report.

g. Relevant Experience. On page 4-2, in lines 34-42, the text indicates that NRC will work with other federal agencies to inform the drafting of AI standards and guidance documents.

i. What other federal regulatory agencies have experience with AI that is relevant to the NRC mission?

Office of Administration August 19, 2022 Page 7 ii. How will the NRC identify and apply lessons learned by those agencies in reviewing, approving, and using AI applications for their industries?

iii. We understand that the NRC is participating in the work of the American Society of Mechanical Engineers (ASME) in the development of a verification, validation and uncertainty quantification standard for computational models 9. In this and in any other 8F new regulatory guidance that might apply to AI, we suggest that the NRC carefully weigh the costs and benefits to avoid creating regulatory requirements so onerous they preclude the use of AI.

9 Additional information is available on the ASME website at https://www.asme.org/codes-standards/publications-information/verification-validation-uncertainty