ML23216A132
| ML23216A132 | |
| Person / Time | |
|---|---|
| Issue date: | 08/08/2023 |
| From: | Russell Felts NRC/NRR/DRO/IRAB |
| To: | Nicole Fields Office of Nuclear Reactor Regulation |
| References | |
| Download: ML23216A132 (1) | |
Text
UNITED STATES NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20555-0001 August 8, 2023 MEMORANDUM TO: Nicole E. Fields Reactor Assessment Branch Division Reactor Oversight Office of Nuclear Reactor Regulation FROM: Russell Felts, Director Russell N. Digitally signed by Russell N. Felts Division of Reactor Oversight Felts Date: 2023.08.08 09:53:58 -04'00' Office of Nuclear Reactor Regulation
SUBJECT:
APPROVAL OF CHARTER FOR THE REACTOR OVERSIGHT PROCESS DATA STRATEGY AND IMPLEMENTATION WORKING GROUP This memorandum approves the team charter for the working group tasked with evaluating the current Reactor Oversight Process (ROP) data strategy and recommending any strategic changes along with an initial implementation plan for those changes. This working group will review the current status of ROP data, focusing on the following four areas: 1) ROP data sources, storage, quality, and accessibility, 2) ROP data ownership/stewardship and responsibilities, 3) Internally facing ROP data visualization and analytical tools, and 4) Externally facing ROP data visualization and analytical tools.
Enclosures:
- 1. ROP Data Strategy and Implementation Working Group Charter CONTACT: Nicole E. Fields, NRR/DRO/IRAB (630) 829-9570
N. Fields 2
SUBJECT:
APPROVAL OF CHARTER FOR THE REACTOR OVERSIGHT PROCESS DATA STRATEGY AND IMPLEMENTATION WORKING GROUP DATED: AUGUST 8, 2023 DISTRIBUTION:
RidsNrrDro Resource NFields DAird RCureton DMerzke GStock JCarneal MCrespo MArel LTran JArellano AChandler PMcKenna LRegner PFinney RFelts ADAMS Accession No. ML23216A132 OFFICE NRR/DRO/IRAB NRR/DRO/IRAB NRR/DRO NAME NFields PMcKenna RFelts DATE 7/26/2023 8/3/2023 8/8/2023 OFFICIAL RECORD
ROP Data Strategy and Implementation Working Group Charter
Background
Since the beginning of the Reactor Oversight Process (ROP) in 2000, the ROP has been a data-driven, risk-informed oversight program. High quality data and data analytics allow NRC staff to make well-informed and well-supported regulatory program decisions and recommendations, which supports the agencys Independence and Integrity. When data are easily accessible for aggregation and trending, that also supports the agencys Efficiency.
The ROP has over time made a significant amount of reactor oversight-related data available to the public via the agencys public website in various forms and formats. Much of these reactor oversight data are part of the NRCs "High-Value Datasets which can be accessed in machine readable formats (Excel or text files) from the agencys public website and from Data.gov. This public availability of data supports the agencys Openness and Service and supports the NRCs strategic goal of inspiring stakeholder confidence in the NRC.
More recently the Foundations for Evidence-Based Policymaking Act of 2018, signed into law January 14, 2019, has placed further emphasis on the need to enhance federal data functions.
In each ROP Self-Assessment Commission paper issued since 2019, the NRC staff has included a specific section describing key ongoing ROP data-related improvements and enhancements. ROP data trending was formally incorporated into the ROP Self-Assessment program in June 2020. These ongoing data program improvements have contributed to more efficient and effective use and availability of ROP data, particularly for staff. ROP data improvements generally have been a collaborative effort between NRR (DRO and EMBARK),
OCIO, and the regions.
Even more recently, the Commission has taken an active interest in ROP data. In SRM-SECY-22-0086, dated March 10, 2023, the Commission directed the staff to continue to enhance the Operating Reactor Analytics websites usability and the accessibility of Reactor Oversight Process information in a manner that allows a reasonably informed member of the public to easily locate and interpret Reactor Oversight Process Action Matrix data and supporting information. DRO has determined that it is prudent to holistically review the entire ROP data lifecycle to best support the agencys goal to continue to enhance the public usability and accessibility of ROP data.
Objective Establish a working group to evaluate the current agency ROP data strategy and recommend any strategic-level changes along with an initial implementation plan for those changes. This working group will review the current status of ROP data, focusing on the following four areas:
- 1) ROP data sources, storage, quality, and accessibility, 2) ROP data ownership/stewardship and responsibilities, 3) Internally facing ROP data visualization and analytical tools, and
- 4) Externally facing ROP data visualization and analytical tools.
Proposed Action Establish a working group comprised of knowledgeable representatives from the ROP program office branches: Reactor Assessment Branch (IRAB), Reactor Inspection Branch (IRIB) and Generic Communication and Operating Experience Branch (IOEB). Working group members should have both specialized experience with the implementation of the ROP and familiarity with data analytics and visualizations generally and ROP data specifically. As needed, the working group will reach out to internal ROP data stakeholders to gather information, and brief interested internal stakeholders (both ROP and non-ROP). The Division of Advanced Reactors and Enclosure
2 Non-Power Production and Utilization Facilities (DANU) expressed a specific interest in this working group and agreed to provide representatives in an observational capacity.
The IRAB Branch Chief will provide oversight of the working group. The Director, Division of Reactor Oversight (DRO) will provide guidance and direction as requested by the working group.
The working group will review the following four strategic areas for ROP data:
- 1. ROP data sources, storage, quality, and accessibility The working group will review the current sources of ROP data. This review should include where the data are stored, what format the data are in, any key quality controls on data entry or data robustness, any limitations in terms of the accuracy and quality of the data, and who has access to enter, review, and correct data as needed.
Special attention should be placed on data that are not in stored in authoritative agency databases/systems (e.g., RPS, CACS, etc.), data that are not stored in machine readable format (e.g., Excel, CSV, JSON, etc.), and data that were collected for a specific review or analysis, but the dataset is not being updated or maintained beyond that review timeframe. These are the datasets for which strategic recommendations are the most likely.
Also, as data that can be viewed by more knowledgeable people are more likely to be accurate, pay particular attention to how widely available the data are internally to those knowledgeable of the datas accuracy.
- 2. ROP data ownership/stewardship and responsibilities The working group will review the current structure of ROP data ownership and ROP data responsibilities. This review should include which specific positions and organizations have ROP data responsibilities, including data entry, data analysis, data maintenance and correction, database stewardship, and creation and maintenance of data visualizations and available datasets/data files.
The review should include whether those staff in positions with data responsibilities have the appropriate knowledge, skills, training, and access to perform their data functions, and whether those data functions are explicitly described as part of those positions.
In particular, there should also be a focus on whether there is a clear owner of each database/dataset and each data visualization/dashboard and whether there are dedicated resources to maintain and update these tools. Without clear ownership of datasets and data visualizations by knowledgeable staff, these tools become susceptible to inaccuracies, lack of maintenance, and eventually partial or full loss of utility.
- 3. Internally facing ROP data visualization and analytical tools The working group will review the current suite of internally facing ROP data visualization and analytical tools. Internally facing tools can be used by staff to assist in and support decision-making and can also be used by staff to identify data-related issues or errors for correction. The review should include tools that perform both of these functions. The review should include whether the tools are widely available internally and
3 accessible from a common centralized location (e.g., the Operating Experience Hub, RRPS Reports, or similar).
The review should include the amount of manual data processing or manipulation performed by the tool owner and by the users and whether tools can be easily enhanced to serve additional user needs. The review should note any significant duplication of data between tools, and any identified gaps where a visualization or analytical tool is not available for an existing ROP dataset or any new ROP data need. The review should include whether internally facing tools (or a closely related version) can be made publicly available.
- 4. Externally facing ROP data visualization and analytical tools The working group will review the current suite of externally facing ROP data visualization and analytical tools. Externally facing tools can be used by both staff and the public. These tools can be as simple as a publicly available list of documents (e.g.,
inspection reports, event notifications, inspection procedures), to a fully-fledged interactive dashboard (e.g., the Accident Sequence Precursor dashboard), with many tools in between in terms of data complexity (e.g., performance indicator data, reactor power status, operating power reactors by location).
The review should include whether the tools are easily accessible from the agencys public website for a general member of the public (e.g., by using the public website search function or Google). The review should include whether the data are easily filterable, sortable, and accessible to a general member of the public (e.g., needing to know the exact date of an event report, or the exact year of a document, is not indicative of the data being easily accessible). The review should also include whether the data are also presented in a machine-readable format such that the datasets can be processed and analyzed by members of the public outside of the confines of the specific tool provided. The review should include whether publicly available data include links/references to publicly available official agency records (i.e., ADAMS documents) as applicable.
The review should include the amount of manual data processing performed by the staff in order to update the data and in order to update the capabilities and functionalities of the tools. The review should note any significant duplication of information between tools, any opportunities for the consolidation of similar data into a single dataset/single tool, and any identified gaps where a visualization or analytical tool is not available for an existing publicly available ROP dataset.
Once the working group has evaluated the current ROP data strategy, as described above, and developed any strategic ROP data recommendations along with an initial implementation plan for any recommendations, the working group will present their results and recommendations to DRO division management. Once the working group has addressed any feedback from DRO management, the working group will present their results and recommendations to interested internal stakeholders. Finally, the working group will issue a memorandum and a report to document the results of the evaluation including any strategic recommendations and an initial implementation plan.
Upon issuance of the report, the working group will be formally disbanded, however it is likely that individual working group members will subsequently be involved with the implementation of any management-approved ROP data recommendations that result from this effort.
4 This holistic evaluation of the entire ROP data lifecycle along with the implementation of any strategic management-approved recommendations will allow for the staff to continue to enhance the usability and accessibility of ROP data in a structured and systematic way. The NRC is committed to ensuring high quality ROP data and data tools are and continue to be available for both the NRC staff and for members of the public.
Projected Timeline Activity Month Lead Participants Identify working group July 2023 DRO management members Develop working group July 2023 Team Lead charter Charter alignment meeting DRO August 2023 Team Lead (if needed) management Issue charter August 2023 DRO Director Hold working group kickoff meeting and begin current August 2023 Team Lead Working group ROP data strategy evaluation Complete evaluation and develop strategic ROP data October 2023 Team Lead Working group recommendations and implementation plan Present results and Working group, recommendations to DRO November 2023 Team Lead DRO management and address management any feedback Present results and Working group, recommendations to November 2023 Team Lead NRR, OCIO, interested internal Regions stakeholders for awareness Document ROP data Working group, evaluation results and IRAB Branch recommendations in a December 2023 Team Lead Chief, DRO memorandum to the DRO Director Director Team Membership Primary:
Team Lead, IRAB - Nicole Fields Significance Determination Process Lead, IRAB - Dave Aird Performance Indicator Lead, IRAB - Ron Cureton Assessment Lead, IRAB - Dan Merzke IRAB - Gregory Stock IOEB - Jason Carneal RPS-Inspections Product Owner, IRIB - Manuel Crespo Inspection Manual Coordinator (RPS IP Management Lead), IRIB - Madeleine Arel Observers:
DANU - Linh Tran and Juan Arellano
5 Department of Energy detailee to IRAB - Austin Chandler Meetings Working group meetings will be scheduled every other week during the review period from August to December. Meeting schedules will be adjusted as necessary.
Time Reporting All activities will be documented in HCM using CAC No. A11018 or other appropriate non-fee billable CACs.