ML21326A194
| ML21326A194 | |
| Person / Time | |
|---|---|
| Issue date: | 11/09/2021 |
| From: | Theresa Lalain NRC/RES/DSA |
| To: | |
| Dennis M | |
| Shared Package | |
| ML21326A182 | List: |
| References | |
| Download: ML21326A194 (125) | |
Text
Data Science and AI Regulatory Applications Workshop Future Focused Initiatives Opening Remarks November 9, 2021 Theresa Lalain, Ph.D.
Deputy Director, Division of Systems Analysis Office of Nuclear Regulatory Research
Time (Eastern)
Topic Presenter 10:00 a.m. - 10:20 a.m. Introduction and Regulatory Purpose Theresa Lalain (NRC) 10:20 a.m. - 10:40 a.m. NRC Future Focused Research Program Robert Tregoning (NRC) 10:40 a.m. - 11:00 a.m. INL AI R&D Activities Chris Ritter (Idaho National Laboratory) 11:00 a.m. - 11:20 a.m. SNL AI R&D Activities John Feddema, Stephen Kleban (Sandia National Laboratories) 11:20 a.m. - 11:40 p.m. ANL AI R&D Activities Rick Vilim (Argonne National Laboratory) 11:40 a.m. - 12:00 p.m. Machine Learning for Safeguarding Nuclear Materials Nathan Shoman (Sandia National Laboratories) 12:00 p.m. - 1:00 p.m.
Break 1:00 p.m. - 1:20 p.m.
Nuclear Applications of Large Language Models Jerrold Vincent, Bradley Fox (NuclearN) 1:20 p.m. - 1:40 p.m.
Hybrid Physics-Data Driven Model for Prescriptive Control and Design Satyan Bhongale (X-Energy) 1:40 p.m. - 2:00 p.m.
CAP Automation and Informed Inspection Preparation Project - Update Tim Alvey (Exelon Nuclear Innovation Group)
Drew Miller (Jensen Hughes)
Ahmad Al Rashdan (Idaho National Laboratory) 2:00 p.m. - 2:45 p.m.
Panel Session Thiago Seuaciuc-Osorio (EPRI)
Curtis Smith (INL)
James Slider (Nuclear Energy Institute)
Luis Betancourt (NRC) 2:45 p.m. - 3:00 p.m.
Executive Summary and Path Forward
NRCs Future-Focused Research Program Advisors: R. Tregoning, C. Boyd Project Manager: J. Steckel NRC Data Science and AI Workshop III -
Future Focused Initiatives 11/9/2021 Thats so cool Are we there yet?
2 FFR Program Overview
- Program Concept and Context
- Program Objectives
- Process Considerations
- Activities
- Artificial Intelligence and Data Science
- Future Directions
3 Future-Focused Research (FFR) Program Concept Support NRCs need for longer-term ( 3 years) R&D activities Broad scope: all good ideas considered*
Funding
- Dedicated
- Fully loaded at beginning of project
- Exploring opportunities to leverage with University R&D Grant Program Program management and administration
- Streamlined submission and review process
- Low-burden, low-resource implementation Start small, grow with success
- Initiated current program in FY-20 Mixed project portfolio
- Time horizons
- Project risk: emphasizing riskier, less-applied ideas for FY-22 and beyond
- Inspired by national lab Laboratory-Directed Research and Development (LDRD) programs Concept Scale
- Includes blue sky, risky projects
- All ideas (including those outside FFR scope) are communicated to NRC management Program Concept and Context
Decision Making Computational Methods Human/Org Factors Natural Hazards Blue Sky Simulation-based PRA*
Dynamic PRA*
Automatic model construction AI-based data mining AI-assisted RIDM**
Advanced techniques for risk communication NRCs research Needs Advanced metrics for RIDM**
Autonomous Reactors Org Factors in PRA*
Errors of Commission Correlated Hazards Simulation-Based Extreme Hazards Climate Change NRCs Blue Sky Now Blue Sky Degree of Blueness (DoB) =
f{technological readiness, clarity of application, user skepticism}
Program Concept and Context
- RIDM = Risk Informed Decision Making
5 NRCs Horizon: Opportunities and Challenges Its tough to make predictions, especially about the future.
- Yogi Berra
- Changing reactor technologies, concepts of operation
- Increasing knowledge base (and means to use)
- Increasing computational capabilities (hardware, software, modeling approaches, )
- Changing staff and other stakeholders
- Increasing and more challenging regulatory applications RES goal: help ensure that NRC is prepared U.S. Nuclear Regulatory Commission, The Dynamic Futures for NRC Mission Areas, 2019.
(ML19022A178) 0 Program Concept and Context
6 FFR Objectives
- Provide kickstart (basis, direction, and support) for extended projects (outside the FFR program) on likely important topics
- Promote more robust R&D program to sustain agency
- Energize staff
- Improve (and perhaps even radically change) foundational knowledge on key topics
- Develop useful products and appropriate staff cognizance of same
- Actionable insights (including dismissal of potential issues)
- Tools and data for analyses
- Current status, directions, and likely schedules for potentially important technologies, programs, etc.
- Create synergy with related programs (e.g., University R&D Grants)
Program Objectives
7 Research: providing a basis for decisions Typical products (regulatory research)
Ways to look at and/or approach problems (e.g.,
frameworks, methodologies)
Points of comparison (e.g., reference calculations, experimental results)
Job aids (e.g., computational tools, databases, standards, guidance: best practices, procedures)
Problem-specific information (e.g., results, insights, uncertainties)
Side benefits Education/training of workforce Networking with technical community Regulatory Decision Support Specific Analyses Methods, Models, Tools, Databases, Standards,
- Guidance, Foundational Knowledge Decision R&D re*search, n. diligent and systematic inquiry or investigation in order to discover or revise facts, theories, applications, etc.
Program Objectives
8 FFR Process Idea Generation Idea Refinement and Selection Portfolio Monitoring and Reporting Follow-On Projects (User Needs)
FFR Program Process Considerations Gather ideas - could be individual or crowdsourced Open to ideas from across agency As needed, work with submitters to refine initial concept Advisors recommend and senior RES managers choose projects Communicate and monitor progress through program reviews and seminars May identify research for potential future development through user needs
9 Project Rating Considerations*
- Agency impact
- Improves NRCs future capabilities
- Improves foundational knowledge important to future decision making
- Addresses recognized gaps
- Resource leveraging
- Enables NRCs influencing of important external activities
- Potentially benefits multiple NRC programs
- Leverages available resources for research
- Staff enrichment
- Is attractive to individual researchers
- Is attractive to university research programs
- Notes:
- 1) Considerations used as guidance.
- 2) Selection committee also considers the overall portfolio of FFR activities a)
Risk b)
Resources Process Considerations
10 FFR Portfolio Appropriate balance among efforts 50% developing foundational knowledge 50% developing more specific technical tools or addressing regulatory framework gaps Current portfolio is balanced across risk horizon spectrum.
Trending toward bluer sky activities as FFR program has matured.
Activities Foundational Technical Regulatory Low Moderate Strong Gap Objectives Degree of Blueness 0
0.5 1
1.5 2
2.5 3
FY-20 FY-21 FY-22 Submitted Selected
11 AI and Data Science in FFR Related FFR Activities FY-20
- Digital Twins - Regulatory Viability FY-21
- RESbot - A web-based bot to aid RES Researchers FY-22
- Use Machine Learning to Prioritize Inspections
- Characterizing Cyber Security Using AI/ML
- Application of Natural Language Processing to NRC Regulatory Documents Explosion of AI-DS topics both submitted and selected in latest data call General bias in selecting AI-DS topics as FFR activities 0
10 20 30 40 50 60 FY-20 FY-21 FY-22
(% of Activities)
Submitted Selected AI and Data Science
12 Existing AI-DS FFR Activities Digital Twins - Regulatory Viability
- Objective: Understand the potential industry applications of reactor digital twins and the regulatory viability of use of digital twins
- Approach: Assess existing technical information, knowledge, tools, and codes and standards to determine state-of-the-art and current gaps; identify regulatory gaps and fundamental infrastructure elements
- Status:
- Held December 2020 and September 2021 workshops: published December proceedings (ML21083A132)
- Completed report: The State of Technology of Application of Digital Twins (ML21160A074)
- Transitioned out of FFR and is continuing as a follow-on research project RESbot - A web-based bot to aid RES Researchers
- Objective: Develop one or more web-based bots, to aid NRC researchers in mining, for example, experimental data, analyses, compilation of field experience, and risk assessments to support decision-making
- Approach: Create NRC use cases and develop RESbot implementation plan to address use cases; executing implementation plan would be a follow-on effort
- Status: Defined use cases on technical document querying, modeling and simulation, and report preparation; currently evaluating use cases using IBM Watson Discovery and Microsoft Azure platforms AI and Data Science
13 FY-22 AI-DS FFR Activities Use Machine Learning (ML) to Prioritize Inspections
- Objective: Explore use of commercially available ML applications to prioritize inspections and their associated periodicity during abnormal situations (i.e., pandemics)
- Approach: Define licensees as customers; define and build safety behavior using data similar to customer preferences; perform test case using several off-the-shelf ML tools Characterizing Cyber Security Using AI/ML
- Objective: Evaluate issues associated with future AI/ML applications used to characterize cyber security system performance and configurations, and detect abnormal system states associated with a cyber attack
- Approach: Identify viable AI/ML technologies; evaluate technologies relative to defined nuclear cyber use case; apply most promising approach to benchmark test case Application of Natural Language Processing (NLP) to NRC Regulatory Documents
- Objective: Assess use of existing NLP tools for NRC use to assist review of licensing actions
- Approach: Create licensing benchmark case and collect associated data; apply named entity recognition to data set and subsequently create term-frequency inverse document frequency model; evaluate Googles BERT model to retain semantic meaning for neural network training and implementation AI and Data Science
14 Thoughts for Future: AI/DS Nuclear is typically a later adopter of technological innovations
- Slower pace of innovation
- Opportunities to build off advancements and investments in other industries
- Which AI/DS advancements hold biggest promise and challenges for nuclear application?
Nuclear energy landscape is continually changing
- Future reactors will likely be smaller; may be more widely distributed
- Bulk of aging LWR fleet may require operation beyond 60 to 80 years to meet nations energy goals
- How can AI/DS be used to both optimize the new design, certification, and approval process?
- How can AI/DS optimize efficiencies of existing plants to retain safety and economic viability?
- How can NRC use AI/DS to evaluate this landscape to better position itself for future regulatory challenges?
Continuous pressure to decrease human operations to maximize efficiencies
- What are the actions/operations where decreasing human involvement is most beneficial?
- Are there actions/operations that should always retain human involvement/oversight and, if so, how can these be best identified?
Future Directions
15 Questions?
Digital Innovation Center of Excellence Lab Directed Research and Development (LDRD) Digital Twin Overview
What is a Digital Twin?
INL definition: Digital Twins represent the merging of integrated and connected data, sensors and instrumentation, artificial intelligence, and online monitoring into a single cohesive unit.
It is a living virtual model that mirrors a physical asset to predict future behavior.
Digital Twins use real-time bi-directional communication to track and trend both simulated and measured asset information.
What is different than a traditional simulation?
- Integration of real-time data
- Dynamic model update (AI/ML integration)
- Real-time operator feedback (visualization)
- Accurate predictions with fused (integrated) data
- Ability to enable autonomous control
- Distributed across computing platforms Digital Twin Asset Management Proliferation Detection Operations Management Autonomous Control etc.
- Operational Cost 14 - 23% reduced operations cost (BCG)
$1.05 billion in cost avoidance (GE)
- Asset Performance 40% improvement in first-time quality (Boeing) 10% improvement in effectives (Gartner)
- Growing Market and Technology Market is ~$3.1 billion (2020)
Market predicted to be $48.2 billion by 2026 Digital Twin Proven Opportunity from Industry Applications General Electric Aviation has digital replicas of every engine to monitor performance and predict maintenance issues. This approach reduces engine operational costs and increases safety.
Adv. Manufacturing Digital Twin (AM&M Initiative)
PI: Brennan Harris
- Manufacturing Processes:
Spark plasma sintering Digital light processing
- Opportunity: Predict manufactured sample performance from varied manufacturing input parameters
Results
- Digital architecture that allows both simulated and physical material properties to be predicted from manufacturing parameters An open database and interface for INL manufacturing researchers to utilize Manufacturing optima for SPS samples from statistical prediction
- Potential for follow-on research Applying the method to processes outside SPS and DLP.
Solvent Extraction Equipment Testing Laboratory Digital Twin PI: Ashley Shields
- Facility:
30-stage annular centrifugal contactor system Binary Metal separations
- Opportunity:
First solvent extraction twin Provide open-source adaptable digital twin components Centrifugal Contactor Cascade at the Bonneville County Technology Center Leveraging Deep Lynx data warehouse and DIAMOND ontology
Anticipated Results
- Goals: Framework twin for the solvent extraction process Integrated via the Deep Lynx data warehouse and newly developed warehouse adapters Research advancements in
- Digital twin infrastructure
- Sensor integration
- Chemical modeling
- Artificial Intelligence
- Data visualization
- International Nuclear Safeguards
- Nuclear Proliferation Detection
- Potential for follow-on research Beartooth Testbed Digital Twin
IES Digital Twin Framework PI: Paul Talbot Digital Twin Framework DeepLynx Syste m
State Optimal Dispatc h
Faster-than-realtime dispatch optimization Digital Twin Training, Validation Synthetic History Generation Economic Modeling
- HERON
- Dispatch Optimization
- Uses TEAL for economic analysis
- Uses system state to optimize operation
- RAVEN
- MLAI Digital Twins
- Coupling to Codes, Experiment
- Validation and Verification
- Synthetic Histories for Unc. Quant.
- Enhances trusted libraries
Flexible Operation Optimization Dispatch Governer Reactor Core Steam Turbines Thermal Energy Storage H2 Production Steam Electricity H2 Electric Grid H2 Customer H2 Storage Physical Systems (e.g., DETAIL Lab at INL)
Digital Twin Framework DeepLynx Syste m
State Optimal Dispatc h
Faster-than-realtime dispatch optimization Digital Twin Training, Validation Synthetic History Generation Economic Modeling
10 Anticipatory controller development Anticipatory controller demonstration Knowledge Base Simplified and high-fidelity microreactor simulator CSV Reactor power maneuver Artificial Neural Network Database and data warehouse Dynamic mode decomposition Discrete state-space representation Issue space and use cases OR OR AND AND Power conversion and heat rejection Battery charging and discharging Load Demands from Microgrid Cyber Attacks Disturbances Human in the loop Scalable Framework of Hybrid Modeling with Anticipatory Control Strategy for Autonomous Operation of Modular and Microreactors PI: Linyu Lin and Vivek Agarwal
Automated BOP Power maneuver Current Progress
- Model predictive controllers for the anticipatory control of a single heat pipe Assumption Simplified modeling Distributed controllers Initial condition: normal operation at a steady power input to the evaporator Load following through Power maneuver by controller #1 Power conversion Battery Automated balance of plant (BOP) responding to the disturbances due to power maneuver:
Controller #2 alters heat removal rates from condenser Controller #2 maintains magnitudes and changing rates of heat-pipe internal temperatures 11 Anticipatory controller #2 2
Anticipatory controller #1 1
Power conversion Battery discharge
Microgrid load following strategy Condenser Adiabatic Heat inputs
()
Heat outputs
()
Heating blocks Cooling jackets Evaporator
Nuclear-Renewable-Storage Digital Twin PI: Binghui Li
- Goal: Improve system economy, security, and reliability of Nuclear-Renewable-Storage Integrated Energy Systems (N-R-S IES)
- Innovation Integrated high-fidelity physics model to inform the operation of IES Deep reinforced learning based (DRL-based) methods to enable faster-than-real-time simulation
- Impact A collection of DRL-based tools: Reliability Enhancement and System Operation Tool (RESORT)
Can be extended for future research grants Why N-R-S IES?
Electricity and heat Multi-carrier energy system Nuclear Carbon-free baseload Renewable + short-term storage flexible peaking capability Long-term storage resilience against disruptive events 12
Project Tasks
- Tasks I: Learning-enhanced modeling of complex electric-thermal coupled systems using high-fidelity physics-based models II: Learning-based steady-state IES economic operations III: Risk mitigation through intentional islanding and optimal IES control TASK III TASK II TASK I Surrogate (RAVEN)
High-fidelity physics model (HYBRID)
Intentional islanding Disruptive events Multi-carrier energy system optimization (HERON)
Normal condition DRL-based economic operation:
RESORT (Normal)
DRL-based IES control:
RESORT (Contingency) 13
MAGNET Digital Twin (Fission Battery Initiative)
PI: Jeren Browning
- Test Beds:
SPHERE (single heat pipe)
MAGNET (37 heat pipes)
- Opportunity: Remote and autonomous control of a heat pipe SPHERE MAGNET
Results
- Proven Digital Twin capability and repeatable roadmap Integrated via the Deep Lynx data warehouse Open-source, reusable components Research advancements in economic benefit, cyber security, and Artificial Intelligence
- Potential for follow-on research MARVEL Microreactor Test Bed
References 1.
https://www.ge.com/digital/blog/industrial-digital-twins-real-products-driving-1b-loss-avoidance 3.
dice.inl.gov
Questions
- Christopher Ritter
- Director, Digital Innovation Center of Excellence
- Email: Christopher.Ritter@inl.gov
- Phone: 208-526-2657 (office) /
301-910-1818 (cell)
Any Questions?
Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia LLC, a wholly owned subsidiary of Honeywell International Inc. for the U.S.
Department of Energys National Nuclear Security Administration under contract DE-NA0003525.
Exceptional service in the national interest John Feddema, Sr. Manager, Enhanced Decision Making Group Center for Computing Research Sandias Trusted Artificial Intelligence Strategic Initiative NRC Data Science and AI Workshop Tuesday, November 9, 2021 SAND2021-7619 PE
Sandias Trusted AI Strategic Initiative is coordinating a series of fundamental R&D projects to lay the foundation necessary for Sandias scientific and national security applications Desire to deploy AI/ML technologies are increasing rapidly
- FY20 - 80 LDRD projects (roughly 20%) had a significant AI focus
- FY21 - 126 projects (28%) had a significant focus in AI or were significantly utilizing AI technologies
- FY21 - 9 DOE SC Advanced Scientific Computing Research projects
- FY21 - 7 NNSA Advance Simulation & Computing (ASC) Advanced Machine Learning (AML) projects Sandias unique mission needs set us apart from industry
- High-consequence applications require high-confidence decisions
- Solutions require extrapolation beyond the space of available data
- Many national security applications have low volume, incomplete data
- Deployed AI solutions are often in environments under extreme size, weight, and power constraints
- Decisions may need to be made in very short timeframes
- Need to account for potential adversarial issues Sandias history of excellence in core capabilities such as UQ, V&V, optimization, graphs, tensors, and discrete math will enable AI/ML 2
Sandia Mission Needs for Machine Learning and Artificial Intelligence 3
Program Mission Problem Characteristics Global Security Proliferation Detection and Characterization
Multi-modal sensors, distributed sensors, and real-time behavior
How to extrapolate to cases where we do not have ground truth
Physical models may not exist
Real time monitoring with streaming data Global Security Automatic Target Recognition for Military Applications
Limited data that is likely modified or disguised - extrapolation of models is necessary
Desire to reduce or remove human in the loop
Data available at multiple levels of sensitivity
Adversary withholds differentiating capabilities and tactics exclusively for war Nuclear Deterrence Counterfeit and Aging Detection
Many sources of variation - limits to what can be learned from data are unknown
Lack of a mathematical foundation and physical models
Volume of data is very low DOE Office of Science Large Scale Physics Experiments
Rich but sparse data - can be expensive to obtain
Multi-instrument, multi-experiment, multi-measurement experimental observations
Uncertainty present in experiments and physics models National Security Programs Analyst Support for Cyber and Intelligence Operations
Need to introduce AI into a mature system without disrupting current operations
Very high consequence, very rapid transactions (many per minute)
Streaming data with very dynamic environment Energy &
Homeland Security Bioscience and Biosecurity
Multiple types of data requires data fusion
Data collection is often destructive and multiple measurements depend on replication
Theoretical models often dont exist Advanced Science
& Technology HPC System Management and Operations
Operations staff dont know much about performance/failure mechanisms
Thousands of instrumentation points but unknown if data provide useful insights
Experiments are typically one-offs due to how resources are allocated and used
Sandias Trusted AI LDRD Research Campaign Thrusts Mathematical Foundations of AI 4
Efficient and Secure AI System AI Usability and Trust Mathematical analysis and Directed Improvement of AI Methods Acceleration of Training and Hyperparameter Tuning Randomized Methods with Rigorous Probabilistic Guarantees Statistical Inference, Especially with Limited Data Robustness &
Extrapolation Novel AI Solutions Novel AI Architectures and Hardware Scalability of AI Methods Adversarial/Counter-Adversarial Security AI Software Domain/Architecture Aware AI Methods and Algorithms Generalizability of AI Research to High-Consequence National Security Environments Domain-Informed AI Trustworthiness Characteristics of Analytics Adversarial Impact on User Trust Determining Whether to Automate Functionality and to What Level
Successes in Trusted AI will enable Sandia and its mission partners to think differently about current and future mission problems 5
FY20 FY22 FY21 FY23 FY24 FY25 Formal Understanding of AI Scalability Focused Methods Novel AI Solutions Efficient and Secure Systems Mathematical Foundations Robustness and Extrapolation Domain Informed AI Usability and Trust AI Human Interaction Novel Architectures/Hardware Adversarial Security How do we move beyond throw more data and compute at the problem?
Inference w/ Limited Data Do we have guarantees that the solutions be deployed in high-consequence decision making?
What is the best possible performance we can expect for a class of problems?
How do we determine which methods are best for our problems and what their limitations are?
How do we account for adversary tactics that change over time, sometimes abruptly?
How do we develop techniques that address scalability?
How do you deploy AI in a hierarchy of learning hardware from edge devices to large HPC systems?
How do you address continuous, one-shot learning in high consequence environments?
How can we combine data-driven models and domain knowledge to improve performance?
How can we incorporate AI in human-in-the-loop missions and assess performance gains?
How do you mitigate challenges stemming from the frequent lack of ground truth in decision-making performance?
How to develop metrics for trustworthiness that can be used to predict user trust?
Trustworthiness Metrics Research Needs
FY21 Trusted AI LDRD Highlights 6
o Challenge: The ML prediction problem is often confounded with the decision problem o
Goal: Incorporate decision science into ML-based decision making
- Develop rigorous methods for incorporating prediction uncertainty, error costs, and opportunities to gather additional information to minimize decision errors and costs.
o Proposed Solution: Draw on decision science, uncertainty quantification, and information theory to account for possible outcomes and their associated probabilities.
Efficient &
Secure Systems Usability &
Trust Monitoring Online Adversarial Tampering (PI: Gary Saavedra) o Challenge: Defend against adversarial attacks
- Little work on detection and less for streaming models o
Proposed Solution: Distinguishing factors of our work:
- Detection rather than model alteration
- Stream information provides more insight than lone examples
- Unifies several different attacks into one mathematical framework Optimizing Machine Learning Decisions with Prediction Uncertainty (PI: David Stracuzzi)
Mathematical Foundations o
Challenge: Many Sandia mission domains are defined by a lack of reliable data, effectively precluding the use of many modern deep learning/machine learning techniques for predictive modeling o
Goal: Enhance the trust in machine learning (ML) model predictions within sparse & noisy data settings o
Proposed Solution: Novel probabilistic transfer learning (TL) framework:
- Determine when to apply TL, which model to use, and how much (uncertain) knowledge to transfer using new techniques inspired by Bayesian hierarchical modeling, sequential data assimilation, and uncertainty quantification Trust-Enhancing Probabilistic Transfer Learning for Sparse and Noisy Data (PI: Mohammad Khalil)
Thank you for your attention!
For more information, please contact:
John Feddema, 505-844-0827, jtfedde@sandia.gov 7
P R E S E N T E D B Y Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia LLC, a wholly owned subsidiary of Honeywell International Inc. for the U.S. Department of Energys National Nuclear Security Administration under contract DE-NA0003525.
Sandia Information Sciences Initiative Steve Kleban Manager, Complex Systems for National Security 1
COMPUTING CONVERGENCE 2
LARGE-SCALE COMPUTING Enormous increases in the volume of data generated require large-scale computing as essential tools for understanding complex systems and interactions in unprecedented detail and exploring systems of systems through ensembles of models and simulations NEXT-GEN ARCHITECTURES Future systems will be multicore and heterogeneous (processors, memories, and models) and increasingly involve new interconnect tech, special-purpose and energy-efficient architectures, and non-von Neumann elements (e.g.,
neuromorphic and quantum).
DATA SCIENCE Growth in the scale, complexity and availability of data in all domains requires AI and advanced analytics applications and tools to extract knowledge and discovery of patterns and classification in data from large scientific and national security datasets.
The Nation is asking for a computing convergence to enable the ability to address increasingly complex questions at the speed of mission.
National-Level Strategies Emphasize Paradigm Shift in Information Sciences Pioneering The Future Advanced Computing Ecosystem: A Strategic Plan National Strategic Computing Initiative Update: Pioneering The Future Of Computing Earth System Predictability Research And Development Strategic Framework And Roadmap All Agencies Have a Need to Capitalize on Future Advanced Computing and AI DOE, NASA, NIH, NNSA, DoD, IC, DHS Strategic computing could address Unprecedented scale, reducing cost & schedule, increased complexity, real-time data and decision making DOE Strategies and Budget Justifications Reiterate this Shift SEAB Report on AI and Machine Learning With the given existing and planned investment Opportunities range from AI-designed workflow to AI-enabled scientific comprehension Office of Science NNSA
3 COMPUTING CONVERGENCE The Nation is asking for a computing convergence to enable the ability to address increasingly complex questions at the speed of mission.
National-Level Strategies Emphasize Paradigm Shift in Information Sciences Pioneering The Future Advanced Computing Ecosystem: A Strategic Plan National Strategic Computing Initiative Update: Pioneering The Future Of Computing Earth System Predictability Research And Development Strategic Framework And Roadmap All Agencies Have a Need to Capitalize on Future Advanced Computing and AI DOE, NASA, NIH, NNSA, DoD, IC, DHS Strategic computing could address Unprecedented scale, reducing cost & schedule, increased complexity, real-time data and decision making DOE Strategies and Budget Justifications Reiterate this Shift SEAB Report on AI and Machine Learning With the given existing and planned investment Opportunities range from AI-designed workflow to AI-enabled scientific comprehension Office of Science NNSA Applied Information Sciences Center, 5500
SANDIA DEFINITION OF INFORMATION SCIENCE The integration of:
data science/analytics artificial intelligence machine learning associated math and statistics human systems/human factors next-generation computer architecture
INFORMATION SCIENCES INITIATIVE Initial Objectives Bridging fundamental IS research to high consequence applications Creating new IS programmatic opportunities to develop and apply IS techniques, tools, workforce, and infrastructure Enhancing Sandias IS capabilities to:
benefit of NNSA and other clients increasing Sandias IS leadership attracting and retaining critical skills in the workforce 5
APPLIED INFORMATION SCIENCES (AIS) CENTER Develop a new Laboratory Directed Research and Development (LDRD) area National Security Information Science & Technology (NSIST)
Facilitate bridging between fundamental R&D and application Focus on institutional technical road mapping, planning, and investments Identify critical skills and needed infostructure Assist existing mission areas in the development of new program opportunities
7 ALPHAGRID PROJECT OVERVIEW AND OBJECTIVE Image from North American Electric Reliability Corporation (NERC), Reliability Concepts document, pg 40.
There are six Stability Margins which create a six dimensional space that is too computationally expensive to navigate in real-time using traditional methods Reinforcement Learning has shown itself to perform well on similar problems and through LDRD funds, Sandia demonstrated Reinforcement Learning is a strong candidate for this problem, which led to the DOE/OE funding Three year project funded by DOE Office of Electricity, Advanced Grid Modeling Program When a power grid becomes unstable there are currently no methods to walk it back to a stable state.
Can Machine Learning assist grid operators to restore the system to a safe condition in real-time?
8 ALPHAGRID: RESULTS TO DATE First year Developed mini-WECC grid model with 20 control dimension to navigate stability space Understand how to navigate stability space using static data from the mini-WECC model Sponsor funding result of LDRD investment Second year Implemented Reinforcement Learning (RL) approach to navigate stability space on a simple grid, random player safely navigates
~8%, RL ~100%
Third year (current year)
Advanced RL to navigate stability space on a complex grid, random player safely navigates ~.01%, RL converging towards 100%
Apply RL to navigate space, not memorize, dynamic grid Publish results Follow-on funding anticipated Transient Stability and Voltage Stability Level Curves (Plot Sandia generated)
Reinforcement Learning plots (All plot Sandia generated)
Problem: 93% of US total energy supply is dependent on wellbores in some form. Current approaches to evaluating wellbore risk focus on manual grading and site specific physics-based models. Need an automated approach.
Sponsor: Geosciences LDRD Approach: Use Deep Neural Networks and Random Forests Outcome: Good results in automating wellbore failure detection, pursuing follow-on sponsors Machine Learning for Early Wellbore Failure Detection 9
https://www.usgs.gov/media/images/map-united-states-oil-and-gas-wells-2017
Problem: Large quantities of documents need to be categorized with rational, effectively and efficiently, with limited human resources.
Sponsor: DOE Office of Classification (in collaboration with LLNL, ORNL, PNNL, Y-12)
Approach: Ontologies, Machine Learning, Bayesian Networks Outcome: Developing a suite of NLP tools that aid derivative classifiers.
NLP for Document Classification 10 https://www.dreamstime.com/photos-images/messy-file-storage.html
Problem: Develop a method of detecting outliers in the acoustic data from electromechanical devices that produce a sound Sponsor: NNSA/ND Program Approach: Statistical machine learning Outcome: Deployed tool to Component Engineers for testing Machine Learning for Outlier Detection 11 (All plot Sandia generated)
Problem: Identifying emergent technologies based on open source indicators (publications, new releases, patents, etc.)
Sponsor: Airforce Research Lab Approach: Artificial Neural Networks, Data Augmentation Outcome: Performs with 90.4% accuracy, can be scaled, can be automated Machine Learning for Detecting Technological Maturity 12
ANL R&D ACTIVITIES A. DAVE, A. HEIFETZ, R. HUI, M. LI, T. NGUYEN, R. PONCIROLI, S. MOHANTY, R. VILIM, H. WANG, L. YACOUT Nuclear Science and Engineering Division Virtual presentation to NRC on AI/ML November 9, 2021
ANL AI/ML CAPABILITIES ENABLING FUTURE AUTONOMOUS OPERATION DESIGN &
MATERIALS OPERATION MAINTENANCE ENERGY STORAGE AUTONOMOUS OPERATION
DESIGN & MATERIALS
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT AI FOR DESIGN SPACE CHARACTERIZATION Design and Materials Method to develop ML-based closure models to capture complex spatial-temporal reactor transients, with uncertainty quantifications.
Integration of ML-based closure model into reactor system transient simulation tool SAM.
Development and application of data-driven turbulence closure model for thermal mixing and stratification modeling.
Developed a system approach on the optimization and uncertainty quantification of the data-driven ML models Incorporate more domain knowledge into machine learning-based closure for advanced reactor safety modeling; Develop deep learning-based multi-physics online simulator to support autonomous operations in advanced reactors Facilitate the development and deployment of advanced reactors by improving economics (through accurate safety margin predictions) and reducing the licensing burden (through improved uncertainty quantification).
Reduction of high dimensional data using ML to yield fast running low-order surrogate models 2.2 3.3 4.4 5.5
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT ML FOR MATERIALS DEVELOPMENT Design and Materials Deep learning-based radiation defect analysis tools were developed for automated detection, tracking and analysis of voids and dislocation loops produced during in situ ion irradiation at Argonnes IVEM-Tandem Facility.
Developed multi-object tracking model to measure the lifetime of individual dislocation loops.
Developed an automated void detection and analysis tool using computer vision and deep learning.
Developed machine-learned dynamical equations.
AI-enhanced radiation damage assessment to shorten material development and qualification cycle.
IVEM Processed a video recorded during in situ ion irradiation to measure the size and number of voids as a function of irradiation dose produced in pure Nickel irradiated with 1 MeV Kr ions at 600°C.
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT ML FOR MATERIALS INSPECTION Design and Materials
Imaging hardware (FLIR X8501, flash lamp, optics)
Machine learning image processing algorithms
Thermal tomography depth reconstruction and defect classification algorithms
Detection of calibrated subsurface microscopic defects in SS316 (down to 100µm size) with unsupervised learning of thermography images
Classification of defects aspect ratio and orientation in thermal tomography images with convolutional neural network
Further reducing threshold of detected defect size (target 50µm)
Rapid data processing for in-situ monitoring applications Imaging of internal microscopic material defects in additively manufactured metallic structures (SS316 and IN718) for nuclear applications U
W Input layer Principal components Output layer Estimated independent sources TSI Neural learning based ICA Observed thermograms Principal components Dimensionality reduction PCA pre-processing X
S U
Defects TSI S
De-noised thermography data cube X' Spatial images de-noising Wavelet Transformation
OPERATION
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT HEALTH MONITORING: PHYSICS-BASED Operation Diagnoses both equipment and sensor faults within an engineered system Requires no a priori values for equipment design parameters Incorporates automated reasoning to facilitate ease of use by non-SMEs Derives real-time equipment performance from physics-based models Blind detection and diagnosis of Monticello NPP reactor feed pump fault, North Anna NPP feedwater heater fault Subsume data-driven methods into the existing Bayesian setting for an integrated diagnostic tool utilities have deemed valuable Advanced heath monitoring of equipment for O&M Inclusion of domain knowledge to deliver diagnoses with greater specificity and reliability PRO-AID Code Architecture PRO-AID Feed Pump Diagnosis: Efficiency Loss Attributed to Bearing Degradation
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT HEALTH PREDICTION: MECHANISTIC Operation System level structural mechanics model of the physical twin Real time AI/ML nonlinear material damage prediction from sensors and structural state prediction Prediction of component interior system-level stress analysis from AI/ML-digital-twin model during load following based on a few measurements Real-time benchmarking and concept validation using ANL METL or similar facility High temperature operation can lead to material damage Need real-time prediction of component health to reduce inspection cost Stress analysis results under system-level conditions Stress experienced over a fuel cycle
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT PERFORMANCE OPTIMIZATION: OPEN-LOOP Operation Machine learning models that can identify through physics and engineering principles the key process variables inputs Supervised machine learning algorithms for predicting performance measures from sensor and digital twin virtual sensor inputs IN-USE - A physics-informed neural network model developed for optimizing BWR reactor fuel loading and operation mid-cycle Identification and development of ML predictive models for estimation of important performance metrics for advanced reactors A capability to learn complex relationships between sensed process variables and performance metrics, such as integrated thermal power and spatial peaking factors Predictive model developed for a BWR from archived operating history -
In use at a US utility
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT PERFORMANCE OPTIMIZATION: CLOSED-LOOP Operation A reinforcement learning (RL) approach that is a data-driven having the potential to learn control policies whose performance surpasses that of humans.
RL agents that learn from a physics-constrained environment via the SAM code - a best-estimate system level code for advanced reactors A design development framework that generates RL environments that is reactor design agnostic (MSRs, SFRs, HTGRs).
Numerical demonstration of RL-agent providing supervisory control for a Fluoride-cooled High-temperature Pebble-bed Reactor in FY22 Optimal control policies that avoid the curse of dimensionality Ability to handle nonlinear phenomena (e.g., material degradation, dynamics during load-following)
- 3. PROPOSED FUTURE DEVELOPMENTS Framework to train supervisory NPP agents using next-generation AR best-estimate system code SAM
MAINTENANCE
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT DECISION MAKING Maintenance Physics-based fault symptoms from model residuals Automated backward chaining reasoning Fault diagnoses can be explained in the forward causality direction Conducted assessment tests with NPP operators on full scope simulator Received confirmation of the utility and value of the approach Improve reasoning engine efficiency Explainable diagnoses for decision making Confirmatory diagnostic traceback via the conservation equations to an accountable set of sensors Physics-Based Model Residual Generation: A Basis for Explainable Diagnoses Model Residual Model Components Physical Sensors Virtual Sensors Balance Equations Model Predictions Physical Sensors Components
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT SCHEDULING Maintenance Sensor network design algorithm to provide for monitoring/diagnosing faults and component degradation over plant lifetime Maintenance and asset management approach that integrates online monitoring with plant risk profile In-progress demonstration for the feedwater and condensate system of the MHTGR design Application of Markov Decision Process method for asset-management decision-making Cost optimization of O&M for increased economic competitiveness Overview of Operational Decision-Making Process P&ID of the feedwater system used as test-case
ENERGY STORAGE AND THE GRID
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT ENFORCING STORAGE CAPACITY CONSTRAINTS Energy Storage and the Grid Algorithm for translating process variables constraints into power set-points limits Satisfies n-dimensional envelope as set by constraints on important process variables Preliminary implementation completed for representative integrated energy system Integrate with diagnostics and decision-making algorithms for semi-autonomous operation Control strategies for improved regulation wrt to structure operating limits for margin recovery Time Evolution of Acceptable Region of Operation during a Transient Reactor with Thermal Storage
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT REDUCED ORDER ON-LINE LEARNING Energy Storage and the Grid Algorithm to update the state-space representation of power systems at various power level and mode using on-line simulation data On-line updated mathematical models helped avoiding constraint violations, actuation oscillation and over-shooting Preliminary implementation completed for representative power systems Improve the robustness of on-line learning algorithm to learn from noisy data Accurate mathematical representation of power systems at various power level and operational mode for efficient control On-line ROM learning and solution improvement example Block Diagram Schematic of Algorithms Model Initialization Learning from Failure (Oscillation & over-shoot)
Optimal Solutions
AUTONOMOUS OPERATION
- 1. NEED
- 2. CAPABILITY DEVELOPED
- 3. ACCOMPLISHMENTS
- 4. FUTURE DEVELOPMENT AUTONOMOUS OPERATION AS AN INTEGRATED PROCESS Autonomous Operation Diagnostics - Discrimination of sensor and component faults via PRO-AID algorithm Control - Automation of constraint enforcement via Reference Governor algorithm Decision-Making - Optimal operating and maintenance procedures via Markov process Developed a control-oriented simulator of KP-FHR coupled with thermal energy storage Integration of diagnostics, control, and decision-making for seamless autonomous operation O&M cost reduction in deregulated markets through more efficient human resource allocation Advanced Reactor -
Layers of Protection
Exceptional service in the national interest Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia LLC, a wholly owned subsidiary of Honeywell International Inc. for the U.S.
Department of Energys National Nuclear Security Administration under contract DE-NA0003525.
New Approaches Utilizing Process Monitoring Data and Machine Learning Nathan Shoman & Ben Cipiti SAND2021-5699 PE Data Science and Artificial Intelligence Regulatory Applications: Workshop #3: Future Focused Initiatives November, 2021
Motivation 2
- Process monitoring data (such as bulk mass, flow, temperature, current, voltage, etc.) and additional measures (such as surveillance) are part of the overall safeguards systemsbut how can we make more efficient use of this data?
- One motivation for the application of data analytics like machine learning is to reduce the cost and burden associated with safeguards:
Reduction of sampling and DA could significantly reduce the burden of IAEA safeguards. More use of unattended monitoring systems instead of DA (on-site laboratory) would free up IAEA resources.
Reduction of sampling and DA can also be useful for domestic safeguards to reduce cost for the operator.
- A second motivation is to improve plant monitoring for facilities or areas that have difficulties achieving materials accountancy goals:
In pyroprocessing for example where there are materials accountancy challenges, can we make more use of plant monitoring data to verify operations?
Machine Learning: The Answer to All Our Problems 3
- Machine learning is broadly defined as approaches that can learn and adapt without explicit instructions.
- Potential Benefits:
ML can automate tedious tasks and reduce chance for human error.
ML can aggregate large amounts of data and disparate data sources to learn normal operation, potentially making it easier to detect abnormal operation.
Can automate monitoring to help reduce costs.
- Potential Downsides:
ML algorithms will only be as good as the data used to train it.
Developing useful algorithms potentially require a large amount of training data which may not be available.
A black box algorithm may not be suitable for safeguards where transparency is important (how much can we trust the results?)
Example 1: Video Surveillance (NNSA Funded) 4
- Generates massive quantities of data with few segments of interest.
- Tedious for human review, however, image recognition is a well understood problem.
UCSD anomaly dataset:
http://www.svcl.ucsd.edu/projects/anomaly/dataset.html Anomalous behavior identification in video sequences.
Spatio-Temporal Anomaly Detection in Video. Smith, Rutkowski, and Hamel.
Deep Learning to Predict Operational Status (NNSA) 5 5
Signature Precision Operationalized Precision Signature Recall Operationalized Precision Plant not operating 0.89 0.66 0.91 0.87 Plant operating 0.96 0.95 0.95 0.84
Example 2: Anomaly Detection in Heterogeneous Safeguards Data Streams (NNSA funded) 6
- Neural approaches should be able to learn normal rhythm of facility operations.
- Deviations from normal might indicate anomalous behavior.
Feature Extraction Anomaly Detection
Example 3: Hey Inspecta! (NNSA Funded) 7
- Smart assistant to improve effectiveness of international nuclear safeguards inspectors.
Information recall Measurement system integration Hands-free support
- Incorporates many ML domains from text analytics to image recognition.
How can I help?
This Photo by Unknown Author is licensed under CC BY-SA Hey Inspecta!
Example 4: Neural Networks for Insider Threat Detection 8
- Can commercial software improve insider threat detection?
Changes in facility pattern-of-life https://tutorials.com/primer-on-neural-network-models-for-natural -language-processing/
- Many off-the-shelf computing packages exist, and these can be useful in some applications.
- However, some applications in safeguards may require customized solutions.
Lessons Learned
Current and Past Work Has Evaluated the Use of ML to Improve Materials Accountancy for Reprocessing and Enrichment.
10
- Process models were used to generate the necessary training data.
- Simulated measurements including both bulk processing monitoring data and nuclear measurements have been used to reduce reliance on DA.
Application of ML to Materials Accountancy 11
- Application of ML to a material balance (which includes measurement uncertainty) is rather unique in the ML field.
- Large data requirements combined with safeguards errors create a difficult challenge.
Results 12 Initially, the ML results were much worse than a traditional materials accountancy approach, due to the variation in systematic errors.
Reduction of systematic biases through cross-calibration of sensors led to significantly improved results.
Initial Results Results with Detector Cross-Calibration
Conclusions 13 Machine Learning can work very well in specific domains. Image and text recognition are proven uses with many applications. Application to containment and surveillance could provide significant benefit to safeguards.
ML is powerful but requires careful application and subject matter expert inputit needs to be trained, and training can require a lot of data and time to develop the algorithms.
The application to materials accountancy appears to be less promisingtraining with data that has uncertainty is a unique application in the ML community. More R&D is needed to determine if there are viable approaches.
There are concerns over the operational transparency of using ML approaches.
The high consequence nature of safeguards results in strict requirements not often seen in other industries which results in further R&D challenges.
We develop and distribute Nuclear-Ready AI software https://nuclearn.ai
Roses are red violets are blue, the remainder of this poem was generated with Nuclear AI, and it has been sent to the NRC for review.
Applicability of large language models in Nuclear
Bradley Fox B.S. Materials Science &
Engineering 6 years Nuclear Engineering &
6 years Data Science &
Software, PVNGS Jerrold Vincent B.S. Business Economics M.S. Computer Science 10 years in Utility Data Science and Business Intelligence, PVNGS Previously started Palo Verdes Data Science Team in 2017 Prior Work at Palo Verde:
Auto PO&C Labeling, Equipment Anomaly Detection, DIANA Network Analysis, CAP AutoScreening, Supply Chain Forecasting & Optimization Recipients of 2020 Nuclear Energy Institutes Top Innovative Practice Award for Process Automation using Machine Learning
What are Large Language Models?
Specialized neural networks for modeling general natural language trained on HUGE amounts of data Broad (English), domain specific (Medical) or task specific (Q&A)
Single model can answer questions, generate novel passages, classify text, perform translations, summarize content Approximate volumetric difference proportional to learning capacity difference from traditional machine learning techniques
Revolution in Natural Language Approaches Old School
Manually clean text to reduce number of extraneous words and identify phrases and keywords that matter
Train Naive Bayes/Boosted Tree/Simple Neural Network on features
Accuracy is lower than humans Large Language Model Era
Pre-trained models can perform many tasks without any additional training
Models can be fine-tuned to specific problems to achieve superior performance
Models read an entire passage, and use the entire context to understand the natural language
4.3x reduction in number of errors1 Move data pipeline complexity and feature engineering into the language model After performing WO 1234567, maintenance tech attempted to stroke the valve. While manually operating the valve, the tech slipped on water left from a leaking overhead pipe.
1.https://gluebenchmark.com/leaderboard
What can we do with these models?
More accurately auto-screen a higher proportion of issues utilizing improved classification abilities
Improve the quality of reports using intelligent autocomplete with Nuclear-specific terms and phrases
Evaluate whether an issue report contains sufficient information as it is being written
https://www.youtube.com/watch?v=K3SdC909bnc
Large Language Models are still improving.
Next generation predicted to be 200x size of current generation
Models will achieve superhuman performance on a broad range of natural language and general AI tasks
Services such as Github Copilot already leverage advanced auto-complete functionality for millions of users
Gartner predicts that by 2025 generative AI will account for 10% of all data produced worldwide For the first time in the history of Machine Learning, there is no evidence of decreasing returns from increasing model size. The only limiting factor is compute resources.
Intelligent auto-completion of procedures and work instructions, including generation of entire work steps
Query large Nuclear texts for answers (e.g.
FSAR, design documents, etc.)
Chatbots for creating Issue Reports, Work Orders, Scheduling
Automatic summarization of site schedules and daily issues
We plan to release a Nuclear-specific Large Language Model in the future Future Use Cases and Research
https://nuclearn.ai Large Language Models are used in Nuclearn platform and products CAP Screening Automation Automated Trend Coding CAP Trending Dashboard 10CFR50 Section Applicability
https://nuclearn.ai Questions?
jerrold@nuclearn.ai brad@nuclearn.ai
Satyan Bhongale, Lead Data Scientist November 9, 2021 Hybrid Physics-Data Driven Model for Prescriptive Control and Design
Industrial Data Science Challenges Industries Automotive Power /Energy Mining Semiconductor Typical Applications Process Optimization Predictive Maintenance Composite metrology Anomaly Detection Key Challenges Lack of Data Bad Data Quality Lack of Variability Multivariant prescriptive control Amount of Data Physics Prescriptive /Predictive Predictive Where you are !!
Predictive vs Prescriptive One way function and non-invertible Lack of physical insight (operational difficulties) 1 = (1, 2, )
1 = (1, 2, )
Most likely invertible (not always)
Knowledge of gradients and higher derivatives at each point in space Deriving Physical laws from data Data Robot originally Newtonian GPLearn (based on genetic algorithm)
AIFeynman (able to derive over 100 Physics laws from Feynman Lectures Symbolic AI / Regression Neural Nets / Deep Learning
Symbolic AI / Regression
Use case : Condenser - Turbine
, = 0(1 + (1, 2,.. ))
Smooth function of variables not captured by the physics equations e.g.
Vibrations Leaks Bearing temperature Ambient conditions Etc.
Symbolic AI Prescriptive control Extrapolation allowing for design 2 = 1
2 1+ 1
=
2 211 x 1 2
2 = (1, 1, 1, 1,.., 1, 2,, )
Summary
- Need to develop Hybrid Physics & Data driven Digital Twins
- Symbolic AI allows for prescriptive control
- Models are extrapolatable beyond their operational regimes
CAP Automation and Informed Inspection Preparation Project - Update Tim Alvey, Manager, Exelon Nuclear Innovation Group Drew Miller, Lead Engineer, Risk Informed Engineering, Jensen Hughes Ahmad Al Rashdan, Ph.D. Senior Research and Development Scientist, Idaho National Laboratory November 9, 2021, NRC Workshop
Agenda Project Objectives Progress Screening and Automation Inspection Preparation Keywords and Trends Topics relevant to P&IR Diverse techniques/approaches Next Steps Closing Remarks 2
NRC AI/ML Workshop
Project Objectives Explore artificial intelligence and machine learning techniques to improve use of plant information Leverage data science technologies and methods Identify opportunities to improve utility processes
- Incident Report Processing
- Station Ownership Committee
- Work Week Planning
- NRC inspection preparation 3
NRC AI/ML Workshop
Project Focus on CAP Data
- Cornerstone of Reactor Oversight Process (ROP)
- Streamlining and strengthening the CAP through use of AI/ML is expected to:
Improve consistency in processing, incoming IRs Automate collection of data for inspection preparation Find hidden trends and insights in existing CAP data
- Important Condition reports (CRs) requiring attention
- Software provides a textual comment explaining why the decision was made (enhances explainability) 4 NRC AI/ML Workshop
Incident Report Screening Automation Process 5
NRC AI/ML Workshop SOC Automation Expanded Phase 1 IR Data Entered Prediction Algorithm Classifier:
In CAP Classifier:
Severity and Priority Classifier:
Work Request Classifier:
Is Rework Classifier:
Crit. Comp.
Failure Classifier:
Priority Classifier:
Discipline Classifier:
Job Type Classifier:
Clock Reset Classifier:
Foreign Material SOC Automation Initial Phase 1 Work Screening Automation Initial Phase 1 Work Screening Automation Expanded Phase 1 Classifier:
Unit Condition Goal: ~80% Effort Reduction Goal: ~95% Effort Reduction
- CAP and New Work Screening stakeholder input
- Areas of automation to reach effort reduction Critical component failures Nondiscretionary clock rests Rework
- Completed models for CR/NCAP items and if they represent a significant condition (SCAQ)
- Develop additional models and results page (i.e., user interface) built into NUCAP 2.0 6
NRC AI/ML Workshop Automation Progress
The significant conditions that warrant increased attention, investigation and corrective actions comprise about 1% of all CRs generated CAP Statistics (2017 - 2021)
Severity 1
Severity 2
Severity 3
Severity 4
Severity 5
Priority A 3
66 25 1
0 Priority B 0
123 372 45 0
Priority D 2
50 3,569 403,231 1,359 Severity 1
Severity 2
Severity 3
Severity 4
Severity 5
Priority A 3
66 25 1
0 Priority B 0
123 371 41 0
Priority D 0
49 3,528 300,364 476 All CRs including NCAP CAP (i.e., CAQ) 7 NRC AI/ML Workshop
- Leverage insights from CAP automation and apply these to the identification of relevant inspection trends
- Enhance internal assessments and inform inspections
- Streamline information sharing through an inspection data portal
- Develop data-driven metrics to support inspection outcomes
- Inform these processes though automation
- Develop tools to automate/identify risk contributors
- Identify and highlight risk-significant information using PRA insights
- Components and/or operator actions
- Programmatic and predictive trends 8
Informed Inspection Preparation NRC AI/ML Workshop
Topics Relevant to P&IR (from IP 71152)
- Negative trends in human/equipment performance
- Cited or non-cited violations
- Significant conditions (SCAQ)
- ROP cross-cutting themes
- Risk significant issues and trends
- Long-standing degraded conditions
- Reductions in design or operational margin
- Repetitive work orders and equipment failures 9
NRC AI/ML Workshop
Keyword/Topics 10 MIRACLE (Machine Intelligence for Review and Analysis of Condition Logs and Entries)
NRC AI/ML Workshop Hypothetical CR Text t1 w1 t2 w2 t3 w3 t4 w4 t5 w5 During performance of 'Site Evacuation Alarm Test', the evacuation siren in the EDG Bay did not sound. The evacuation beacon was previously issued under different IRs.
Equipment condition appears to be degrading.
Test was completed UNSAT due to EDG beacon not lighting.
Emergency planning 19.2 Communication equipment 11.7 Emergency drills 1.5 Diesel generator 1.4 Rad Con instrumentation 0.9
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
16.0%
Dec-14 Jul-15 Jan-16 Aug-16 Mar-17 Sep-17 Apr-18 Oct-18 May-19 Dec-19 Jun-20 Percentage of Reports with Topic Assigned Industrial safety Trending Data-driven keywords with industry data to standardize trending 11 NRC AI/ML Workshop 0
5 10 15 20 25 30 35 40 45 50 Dec-14 Jul-15 Jan-16 Aug-16 Mar-17 Sep-17 Apr-18 Oct-18 May-19 Dec-19 Jun-20 Number of Reports with Topic Assigned Industrial safety
Diverse AI/ML Techniques and Approaches JH uses a classifier algorithm (CAP Analyzer) with supervised learning to predict rare events INL uses a combination of supervised (Cortex) and unsupervised learning (Latent Dirichlet allocation) to create trends Integrate and leverage both approaches Allows independent validation NRC AI/ML Workshop 12
Working... Next steps Ongoing
- Insights from plant subject matter experts
- Collaboration with Xcel Energy
- Compare Exelon dataset with other utility results and optimize keywords (e.g., specificity)
Future
- Pilot CAP automation - 1st Q 2022
- Explore metrics pertinent to P&IR inspection (and expand to other inspection areas) - 2nd Q 2022
- Deploy open-source tools for broad industry use 13 NRC AI/ML Workshop
Concluding Remarks AI/ML techniques have the potential to strengthen the Corrective Action Program Overarching goal is to improve Exelon internal governance and oversight Stakeholder engagement and input is critical Designers must proactively address their innovation so individuals should decide on long-term use of their product Integration with NRC and industry presents the opportunity for a powerful outcome 14 NRC AI/ML Workshop
Questions?
Tim Alvey Manager Exelon Nuclear Innovation Group Tim.Alvey@exeloncorp.com Andrew Miller Lead Engineer, Risk Informed Engineering, Jensen Hughes AMiller@jensenhughes.com Ahmad Al Rashdan Senior R&D Scientist Idaho National Laboratory Ahmad.alrahdan@inl.gov 15 NRC AI/ML Workshop
DATA SCIENCE AND ARTIFICIAL INTELLIGENCE ACTIVITIES Mr. Luis Betancourt, P.E.
Chief, Accident Analysis Branch Office of Nuclear Regulatory Research Division of Safety Analysis
NRC AI Challenges Future Regulatory Guidance and Decisionmaking Development Differentiating AI Usage for Reactor Design Versus Autonomous Control Explainable and Trustworthy AI - Reliability and Assurance Internal AI Budget Predicated on Emergent Industry Applications Current Workforce Training Traceable and Auditable Evaluation Methodologies Internal Challenges: Automating Internal Agency Business Processes External Challenges: Understanding Licensee and Applicant AI Usage 3
FY 2022 Path Forward
- Enhance staff knowledge in applications and use of data science and AI
- Engage with internal and external stakeholders to seek alignment on the draft Data Science and AI Strategic Plan
- Issue Data Science and AI Strategic Plan by Fall 2022 4
Contact Us Dr. Theresa Lalain Deputy Director, Division of Safety Analysis Office of Nuclear Regulatory Research theresa.lalain@nrc.gov Mr. Luis Betancourt, P.E.
Chief, Accident Analysis Branch Office of Nuclear Regulatory Research Division of Safety Analysis luis.betancourt@nrc.gov