ML24075A027

Some Issues in the Assurability of Safety Critical Digital Systems

ML24075A027
Person / Time
Issue date:	03/15/2024
From:	Doug Eskins NRC/RES/DE
To:
Doug Eskins 301-415-3866
Shared Package
ML24075A025	List: ML24075A027 ML24081A110
References
Download: ML24075A027 (15)

Text

IAEA Technical Meeting EVT2300917 on Deployment of Artificial Intelligence Solutions for the Nuclear Power Industry:

Considerations and Guidance18-21 March 2024 U.S. Nuclear Regulatory Commission Headquarters, Rockville, MD, USA

Some issues in the Assurability of safety-critical digital systems Part 1 Assurance and AI

Senior Computer EngineerDoug Eskins Office of Nuclear Regulatory ResearchU.S. Nuclear Regulatory Commission

The views expressed herein are those of the author and do not represent an official position of the U.S. NRC.

Assurance

• A claim (about X) is supported by sound, valid evidence (under the assumptions and conditions identified in Y).

• X could be a system design or an O&M process.

• Y is a set of conditions and assumptions under which the claim holds.

• Assurance is sometimes referenced to a CAE triplet (claim, arguments, evidence)

Artificial Intelligence

A machine-based system that can go beyond defined results and scenarios and has the ability to emulate human-like perception, cognition, planning, learning, communication, or physical action (NRC AI Strategic Plan).

Note: Each human-like capability is referenced to some (domain-specific) application.

AI & Assurance

• How can AI be assured?

• How can AI be used for assurance?

Assuring AI

• What are the bounds of application?

- In nuclear: safety or non-safety, design or O&M?

• Is assurance comparable between humans and AI?

• How will the CAE needed to assure an application differ for AI?

- Ex) Can non-interference with a safety function be assured?

AI for Assurance

• Can AI facilitate the CAE needed for assurance?

- Data collection, processing, and analysis to support Evidence generation

- System modelling to support Argument construction and validation

- System and domain analysis to ensure a necessary and sufficient set of Claims to support assurance.

Assuring AI for Nuclear Cybersecurity Applications

• Ongoing NRC research exploring the use of AI to characterize nuclear cybersecurity states.

• Issuesencountered relevant to assurance ofcybersecurity classification models:

- Data artifacts & joint IT/OT data

- Model performance measures & coverage of plant states

• Answers can be very application dependent IAEA Technical Meeting EVT2300917 on Deployment of Artificial Intelligence Solutions for the Nuclear Power Industry:

Considerations and Guidance18-21 March 2024 U.S. Nuclear Regulatory Commission Headquarters, Rockville, MD, USA

Some issues in the Assurability of safety-critical digital systems Part 2 Knowledge Engineering is on the back burner

Senior Technical AdvisorSushil Birla Office of Nuclear Regulatory ResearchU.S. Nuclear Regulatory Commission

The views expressed herein are those of the author and do not represent an official position of the U.S. NRC.

Distinguish between data, information & knowledge

Data Raw Curated Information Knowledge

• Values of properties Processed Justified True Belief

• As acquired Organized

• Verifiable

• Raw Curated datasets Predictive

• Curated Contextualized Cause-effect relationships, e.g.:

• Not yet processed Accessible

• Laws of physics

• Not yet organized Meaningfully Generalization within bounds

DataBase DB) KnowledgeBase (KB)

Deterministic Fuzzy Rule-set Knowledge Engineering (KE)

Within a Well-defined Domain D

Acquire Specific Organize to facilitate Problem-solving for Case Validate KB Decision-making Situation Knowledge Scenario decision info for Inference Engine +

Domain D Reasoning Algorithm Knowledge Representation (KR)

KR: the field of artificial intelligence(AI) dedicated to representing knowledge about the world in a form that can be mechanized to solve complex tasks.

Means of KR example: Ontology a set of concepts and categories in a subject area or domain that shows their properties and the relations between them

KR formalisms - characteristics of interest:

• Expressivity

• Tractability

• Comprehensiblity

• Usability; Learnability Reference model

7 Source: ISO/IEC 26550:2015(E)

ISO/IEC 26550 family of standards

ISO/IEC 26550:2015(E)

Software and systems engineering Reference model for product line engineering and management

ISO/IEC 26551:2016(E)

Tools and methods for product line requirements engineering

ISO/IEC 26552:2019(E)

Tools and methods for product line architecture design

ISO/IEC 26553:2018(E)

Processes and capabilities of methods and tools for domain realization and application realization

ISO/IEC 26554:2018(E)

Methods and tools for domain testing and application testing

ISO/IEC 26555:2015 Tools and methods for technical management

ISO/IEC 26556:2018(E)

Tools and methods for organizational management

8 ISO/IEC 26550 family of standards

ISO/IEC 26557:2016(E)

Methods and tools for variability mechanisms

ISO/IEC 26558:2017(E)

Methods and tools for variability modeling

ISO/IEC 26559:2017(E)

Methods and tools for variability traceability

ISO/IEC 26560:2019(E)

Methods and tools for product management

ISO/IEC 26561:2019(E)

Methods and tools for technical probe

ISO/IEC 26562:2019(E)

Processes and capabilities of methods and tools for transition management

ISO/IEC 26563:2022(E)

Processes and capabilities of methods and tools for configuration management of assets

ISO/IEC 26564: 2022(E)

Methods and tools for product line measurement 9

ISO/IEC 26550 family of standards

ISO/IEC 26850:2021(E)

Methods and tools for the feature-based approach to software and systems product line engineering

ISO/IEC 26565 to ISO/IEC 26599: To be developed

9