ML22168A062
| ML22168A062 | |
| Person / Time | |
|---|---|
| Issue date: | 07/21/2015 |
| From: | Chang Y NRC/RES/DRA/HFRB |
| To: | |
| Criscione L | |
| References | |
| NRC-2022-000191 | |
| Download: ML22168A062 (3) | |
Text
Use Machine Learning to Prioritize Inspections Proposer Name and Organization RES/DRA/HFRB Proposal Date 21/07/15 Future Regulatory Need(s) Addressed Inspections are an important element of NRCs oversight of its licensees. To ensure safe operations, the NRC conducts inspections of licensed nuclear power plants, fuel cycle facilities, and radioactive materials activities and operations to ensure that licensees meet NRCs regulatory requirements. The inspections are performed with set frequencies. During abnormal situations such as COVID-19, the number of inspections are reduced and the duration between inspections are increased. Developing a performance-based, data-driven method to inform the inspection priority and frequency would improve the effectiveness of inspections during abnormal situations.
Technical Issue(s) Addressed The decisions on the priority and frequency of inspections during abnormal situations, such as COVID-19, largely rely on expert judgment. Machine Learning (ML) is an advanced technique to provide high-quality predictions and has been successfully implemented in many applications.
ML intakes a large volume of data to train its algorithms, and the prediction results are used to improve the algorithms. The cycle of prediction and refining the algorithms continuously improves the algorithms and prediction reliability. Studies [1, 2] show that, in many cases, data-based decisionmaking performs better than expert judgments. Once the cycle is established, the self-improvement of ML becomes automatic, thereby improving prediction reliability and reducing the human workload to generate the predictions. As a result, the increase in human workload is minimized when scaling up the operations. The prediction results inform the NRC on inspection priorities and, potentially, data-based industry performance trends.
Proposed Approach NetFlix has successfully used ML to provide personalized movie recommendations for its customers. NetFlix uses combinations of supervised and non-supervised MLs that survey its 125 million global customers movie preferences to identify about 2000 taste communities (or clusters). A NetFlix customer could belong to more than one cluster. NetFlix provides personalized recommendations for a customer based on not only the customers past movie selection behaviors but also the behavior of the other customers who are in the same clusters as the customer.
This proposed approach is similar to NetFlixs strategy. Each NRC licensee is treated as a customer. The inspection results, maintenance information, and licensee event reports, etc.,
represent a licensees safety behavior. A licensees safety behavior, in concept, is the same as NetFlixs customer preference. The proposal would develop ML algorithms to identify the
clusters among the licensees based on their safety behavior. Each cluster represents a unique performance signature. A licensee could belong to more than one cluster. The inspection recommendations on a licensee would be based on the licensees safety behavior and the safety behavior of the other licensees who share the same clusters with the licensee.
The proposal would review the commercially available ML capabilities, such as Amazons SageMaker, Microsofts Azure, Googles Google-AI, and MatLab to understand their suitability for the proposal, and perform a small scale experiment on the selected approach(s) to provide insights based on hand-on experiments.
Expected Products and Anticipated NRC Use The expected products include (1) an understanding of the commercially available ML functions and abilities that support performing ML studies on a larger scale, and (2) the identified clusters of licensees performance will supplement the NRCs existing performance monitoring activities and may provide indications of common issues.
Schedule Once the FFR is approved, the first year will be used to evaluate the appropriate ML tools (i.e.,
Amazons StageMaker, Microsofts Microsofts Azure, Googles Google-AI, and MatLab) for their suitability for the proposal, and establish a contract to leverage outside resources to support the FFR. The second year will be used to perform small scale studies on the selected ML platform(s). The studies would apply to a small number of selected licensees. The ML algorithms would be trained based on the licensees earlier performance data (e.g., older than three years). Then, the trained ML algorithms would be used to perform recommendations. The recommendations would be compared with the licensees recent performance data (e.g., within three years) to evaluate the performance of the ML algorithms. Finally, a report will document the FFR in the third year. The total project duration is estimated at 27 months. If the small-scale study conducted in the second year provides positive signs for follow-up studies or full scope implementations, recommendations would be provided for management decisions.
Resources Requested The anticipated staff effort is 0.05, 0.15, and 0.1 FTE for the 1st, 2nd, and 3rd year, respectively.
The anticipated cost is $150k for contracting (likely contracting with an university) and procuring tools and resources (e.g., server time and technical supports).
Anticipated Time Horizon for Regulatory Application Assuming the study, in the second year, generated positive results and management decided to perform a full-scale implementation, the ability to provide inspection recommendations could be developed within two years after the decision. The regulatory decision would be depend on management decision.
Ratings Considerations Please address each of the high-level ratings considerations identified below. See rating criteria document for background on considerations.
- 1. Agency Impact If the ML techniques are developed successfully, in addition to improving decisions on inspection priority and frequency, the techniques can be expanded to provide personalized services, such as providing personalized ADAMS search services to NRC staff, improving areas that are data intensive, like operating experience reviews, and supporting passive activities, like reactor oversight and performance monitoring. The ML-generated results could provide predictive information to inform the passive programs.
- 2. Resource Leveraging The inspection reports and licensee event/incident information needed for the study are largely digitalized. The ML tools are commercially available, are free for trial, and are charged based on the server time used for the study (e.g., Amazons StageMaker). The author of the proposal has a MatLab license already purchased by the NRC.
- 3. Staff Development Through the proposal, the staff would develop an understanding of the capabilities and limitations of ML techniques based on hands-on experience. The knowledge, skills, and capabilities developed in the proposal can be the foundation for larger projects that ML techniques have advantages over manpower.
Other Points for Consideration ML techniques, sometimes called artificial intelligence, have been used widely in new technologies, such as auto-piloting and object recognition. Commercial demands have pushed the fast growth of the techniques. This means that the project has an increased likelihood of being able to use already-developed ML techniques for the functions required in the proposal.
References
- 1. Iansiti, M. and K.R. Lakhani, Competing in the Age of AI: Strategy and Leadership When Algorithms and Networks Run the World. 2020, Harvard Business Review Press. ISBN:
978-1633697621.
- 2. McAfee, A. and E. Brynjolfsson, Machine, platform, crowd : harnessing our digital future.
2018. ISBN: 9780393356069.