ML23215A209

Ffr Lessons Learned Document Final

ML23215A209
Person / Time
Issue date:	08/15/2023
From:	NRC/RES/DE
To:
Jam Steckel 301-415-1026
Shared Package
ML23215A207	List: ML23215A209
References
Download: ML23215A209 (35)

Text

1 Future Focused Research Lessons-Learned Evaluation I.

Executive Summary The Future Focused Research (FFR) program was initiated in FY-20 as the latest successor for addressing the agencys forward-looking research needs. The overarching vision for the FFR program is to encourage staff throughout the agency to bring forward new research ideas and provide them with a high level of ownership and autonomy, as well as nominal funding for pursuing these ideas. With this in mind, the research proposal and oversight processes were intended to be simple and streamlined so that resources could be focused on the staff and contractors performing the research work. The program was intended to start small and then evolve through demonstrated successes. A goal was to develop a mixed FFR portfolio that spans the spectrum of project risk and complexity, potential reward, and maturation.

This report is organized as follows:

Background on the FFR program, including description of predecessor activities (Section II),

FFR vision and objectives (Section III),

FFR Program description, including program management and evolutionary enhancement to the program (Section IV),

FFR Program Accomplishments (Section (V) and Challenges (Section VI), and FFR Program Recommendations (Section VII)

Overall, the project team believes that the FFR program has successfully met its intended vision and objectives over the first three plus years of the program. The programs structure has enabled a high degree of agility for the FFR process and allowed the project team to make rapid adjustments to meet evolving demands and quickly address many challenges. Further, the FFR program has successfully engaged a wide range of agency staff at a grass roots level and has supported projects that meet future regulatory needs. The FFR program has also leveraged capabilities across a spectrum of external organizations, including universities, national laboratories, and commercial contractors. Nearly all initial projects identified during the first two years of program operation (i.e., FY-20 and FY-21) have been completed, and the associated deliverables have valuably contributed to fulfilling the Office of Nuclear Regulatory Researchs (RES) safety and security mission.

A nascent program is expected to have growing pains and the FFR is no exception. In several cases, the agility afforded by the FFR program structure has allowed the project team to quickly implement improvements to the program. However, the project team has identified several more difficult challenges that have arisen as the program has evolved. The following recommendations are intended to address these challenges to enhance an otherwise robust program and ensure continued growth and future success of the FFR program:

Shift proposal review, evaluation, and approval earlier in fiscal year to allow additional time for contract placement,

2 Develop a checklist to assist principal investigators, Provide FTE allocation to RES technical divisions to help ensure staffing plans reflect anticipated support for the FFR program, Maintain engagement on FFR program at all levels, Leverage external organizations, Improve contracting processes to enable more efficient and timelier placement of work, and Incentivize staff and management support.

A more detailed discussion for each recommendation can be found in Section VII. The project team believes that implementation of these recommendations will support the continued health and vitality of the FFR program in the years to come.

3 II.

Background

The Future Focused Research (FFR) program is a successor to the Long-Term Research Program (LTRP) and the Feasibility Study Request (FSR) programs. These programs shared the similar goal of initiating anticipatory research that was not being addressed through the normal business line budgeting process. The LTRP ran from 2007 to 2016 and was initiated in response of Commission direction to maintain a careful watch for emerging technical/regulatory issues that will require initiation of research beginning in the nearer term.1 The LTRP shared some attributes with the current FFR program. The LTRP submittal and evaluation processes were similar to the current FFR process, as was the level of budgeted support for approved projects.

There were also a few fundamental differences between these programs. After LTRP projects were selected, they were budgeted to begin two years later under the normal business line funding process. An annual SECY, that was signed by all the major office directors, was developed outlining the projects to be included in the budget under preparation. This annual SECY paper was the mechanism used to highlight anticipated funding needs and, in later years, to report on the status and final disposition of prior LTRP projects. As discussed in SECY 0042, the LTRP was considered to be a successful program based on the portion of completed projects that transitioned successfully to address regulatory needs.

However, because of Project AIM-20202, the staff considered shedding the LTRP as a cost reduction measure but recognized that the NRC needs to continue to assess emerging technical and regulatory issues that may require research deemed necessary for performance of NRCs licensing and other regulatory functions. Therefore, in COMSECY 16-0021, the staff recommended transition of the LTRP to a more agile program based, in part, on a desire to carry out more frequent feasibility research studies, enhancing interactions with regulatory offices, and recognition that long-term research was not a separate program but integrated into the overall continuum of research activities. In SRM COMSECY 16-0021, the Commission approved elimination of the LTRP as a separate program but approved the continued use of feasibility studies to evaluate future research (i.e., the FSR program). A summary of key documents associated with the LTRP is included in Appendix I.

The FSR program started in 2017 and formally ran until early 2020. It was envisioned as an idea incubator where seed ideas could be submitted, then evaluated by a review panel (similar to FFR). The most promising ideas would then be targeted for funding through the normal budgeting processes by garnering support from the business line offices. FSR was unfunded and the goal was that it would require no programmatic support funding. The FSR program also deemphasized the importance of forward-looking research and envisioned a continuum of research (consistent with the vision outlined in COMSECY 16-0021) with the FSR being a mechanism to identify research much like existing RES processes such as the Research 1 SRM-COMDEK-06-0001 - "FY 2008 Budget Proposal."(ML062350498). A complete listing of Commission documents associated with the LTRP is provide in Appendix I.

2 SECY-16-0009, Recommendations Resulting from the Integrated Prioritization and Re-Baselining of Agency Activities

4 Assistance Request (RARs) and User Need Request (UNR) processes. No projects were initiated under this short-lived program, but significant start-up resources were expended due to both uncertainty about how the FSR would fit within the existing research processes and the effort to modify RES Office Instruction (OI) PRM-001 to incorporate this process. The FSR is still officially an office process, but it has been effectively retired since the initiation of the FFR program.

The FFR program was initiated in 2020 and built upon the lessons learned from the LTRP and FSR programs. A simplified program with dedicated program funding was envisioned that would invigorate the staff to consider research topics that would likely benefit the agency in the future.

III.

Vision and Objectives A. Vision The FFR concept is loosely based on DOEs Laboratory Directed Research and Development (LDRD) program. Ideas for FFR are broadly sought from staff throughout the agency and the FFR portfolio was initially envisioned to include activities that span the spectrum of project risk and complexity, potential reward, and technical maturation. A driving notion is to provide a mechanism for individual staff enrichment by encouraging staff to bring forward new research ideas and providing a high level of staff ownership and autonomy for research activities. The research proposal and oversight processes are intended to be simple and streamlined such that resources are concentrated with the staff and contractors performing the work. The program was intended to start small and then evolve through demonstrated successes and recognition of the programs value throughout the agency. A unique concept is that activities should receive dedicated, up-front funding for the anticipated 1-to-3-year duration of the projects, rather than having to compete for resources on a project-by-project basis each year.

B. Objectives The principal objective of the FFR is to support the agencys longer-term (i.e., >= 3 years)

Research and Development (R&D) needs for cultivating foundational knowledge, assessing gaps, or developing new tools to address potentially new and emergent technologies that would make the NRC a more effective regulator. There are several more immediate objectives that support this principal goal, including energizing the staff, enriching staffs capabilities, and providing a kickstart for projects with the potential to be extended outside of the FFR. The deliverables from FFR projects are intended to be available throughout the agency and to provide actionable insights. These insights can include identification of potential regulatory issues, development of tools and data for analyses, and formulation of technical and regulatory plans for addressing potentially important technologies. Another aim of the FFR program is to leverage both NRC ongoing activities and related university, commercial, and other federal agency research, to the extent possible. This aim is intended to avoid duplication of effort and efficiently apply FFR resources. Through the realization of the principal and supporting objectives, the goal is to nurture a sustained R&D program that is viewed as an asset by the program offices.

5 IV.

Program Description A. Background The program was officially inaugurated on June 3, 2020, with the approval of $710K of FY-20 shortfall funding. In subsequent fiscal years, approved FFR projects have been selected by the end of September to allow project initiation at the beginning of the upcoming fiscal year. Figures 1a and 1b3 illustrate the FFR resources over the period of FY-20 through FY-24. The FY-23 actual expenditures shown in these figures are current as of June 16, 2023. The yearly budgeted FTE for the FFR program has generally been 2.0 FTE, except for the initial year (2020) and in 2023. FTE is intended to be used by the principal investigators (PIs) with active projects and by the FFR project manager (PM) and other ancillary support staff during each fiscal year. This distributes the FFR program FTE among approximately 15 to 20 staff.

In FY-23, the data indicates that FTE for the program is slightly below historical FTE charging rates but is trending well below the budget of 3.5 FTE, which was a one-time increase over the baseline budget request provided by the Commission. The lower use of FTE in FY-23 may be due to a number of factors including competition for staff time with higher priority work, increased reliance on contract support to complete projects, and staff not accurately charging to the assigned FFR CAC Yearly contractor resources for FFR were initially targeted to be $500K. The initial $710K FY-20 shortfall funding was predicated on the proposed needs of the six awarded projects. The spending in FY-21 was above the $500K budgeted level, in part due to providing additional funding for one FY-20 project and a shift toward increased reliance on contractors (which is reflected in the decreased FTE usage in FY-21). The spending in FY-22 was well below the budgeted level of $500k due to difficulty in placing the contracts for two approved FY-22 FFR projects and use of staff resources rather than contractors to support three of the approved FFR projects.

In FY-23, the initial budget request was $600K but the Commission provided an additional

$400K which increased the total contract budget to $1,000K. This amount was also recommended to be budgeted in FY-24. Actual contract support in FY-23 is anticipated to accommodate two approved FY-22 FFR projects that were not able to be placed in FY-22 due to contracting challenges. True to the original program vision, each activitys total requested funding has been provided at the beginning of each fiscal year, so an approved project is fully funded at the outset. This is a significant accomplishment given the agency-wide focus on minimizing carry-over funding. Ideally this trend will be maintained as it is recognized as a cornerstone of the current program.

3 Although only $297.5k of FY2023 contract funds have been obligated at the time of this report, an additional $956.7k has been committed for pending contract actions.

6 Figure 1a: FFR Program Contract Funding, by Fiscal Year Figure 2b: FFR Program FTE, by Fiscal Year Given the current scope and size of the FFR program, the targeted FY2024 budget of $1,000k and 2 FTE is appropriate. However, in light of the FFR program objective of developing and enhancing staff knowledge, the balance between contractor support and staff technical work could be shifted in the future to better balance staff development with other staff priorities. Such a shift could be enabled through both incremental increases in the FFRs budgeted FTE in conjunction with more strategic allocation of FFR program FTE that alleviates competition with other priority work (as discussed in Section VI.A.1 and VII.A.3).

0 200 400 600 800 1000 1200 2020 2021 2022 2023 2024 (est)

Contract $k Fiscal Year Contract Budget

$K (Budget)

$k (Obligated) 0 0.5 1

1.5 2

2.5 3

3.5 4

2020 2021 2022 2023 2024 (est)

FTE Fiscal Year FTE Budget FTE (Budget)

FTE (Actual)

7 B. Process Overview The process embodies the mechanics and support used to implement the FFR program. Figure 2 provides an overview of the principal process steps.

Figure 2: FFR Process from Inception to Completion More details about the individual steps in Figure 2 and the evolution of the process based on lessons learned over the first 3 years of the FFR program follow.

C. Detailed FFR Process Description

1. Call for Proposal Submittals The annual call for proposals is initiated through an agency-wide announcement directing staff to the FFRs SharePoint site. The call for FFR proposals includes a summary of the FFR review criteria and a proposal template to assist submitters in providing necessary information to support the proposal review and evaluation process. In more recent years, the call for proposals has also directed staff to the Idea Scale Challenge Campaign Incubator, which is used to crowdsource development of new ideas. The Idea Scale Challenge Campaign was implemented during the FY-23 FFR process to leverage the agencys existing crowdsourcing platform and provide a forum for developing ideas prior to formal submission. This addition quickly exceeded expectations in FY-23 as seven potential FFR ideas were submitted to the

1. ANNUAL CALL FOR PROPOSALS ACROSS AGENCY - The call for proposals is typically issued in early spring with a proposal deadline of mid-summer.

2. PROPOSAL REVIEW, RATING, AND EVALUATION PROCESS - SLS representatives from each RES technical division conduct reviews and develops consensus ratings.

3. PROPOSAL RECOMMENDATION AND APPROVAL - The SLS panel presents FFR project recommendations to the RES Front Office for approval or modification. This step is usually completed by September of each year.

4. PROGRAM MANAGEMENT - PM monitors progress, supports PIs as needed, implements FFR program. Projects are typically expected to take 12-36 months to complete.

5. PROJECT COMPLETION - PI submits final deliverables for public (ideally) ADAMS retention and completes a project-specific Nuclepedia page for knowledge management.

8 incubator and benefitted from exposure on the Idea Scale platform. Three of these ideas4 were ultimately selected for FFR funding and one5 is being addressed by OCIO as part of a broader initiative to improve the agencys cognitive search capabilities.

The yearly call for proposals announcement6 has been augmented by efforts by both the PM and the RES technical Division Senior Level System (SLS) advisors to both work with individual staff to help develop ideas and more broadly advertise the FFR program and encourage staff submittals. These efforts have included internal branch, division, and office-wide presentations within RES, an office-wide NRR presentation, and a presentation to the regional weekly knowledge management (KM) session. The informal efforts have been supplemented by more formal outreach through venues such as the NRCs Data Science and Artificial Intelligence Regulatory Applications Workshop, the yearly FFR-days program reviews in FY-21 and FY-22, dedicated Regulatory Information Conference (RIC) sessions in FY-21 through FY-23, a Commission Briefing in FY-22, and an ACRS briefing in FY-20. Based on the number of proposals received each year, these efforts have been successful in elevating the visibility of FFR program and garnering staff recognition. Similar outreach should continue to further embed the program within the agencys fabric.

2. Proposal Review, Rating, and Evaluation Process The submittal review process is simple and has worked well. It is important to maintain SLS representation from each of the three RES technical divisions to span the bulk of the agencys technical disciplines. For proposals outside of the collective expertise of the SLS reviewers, additional staff or SLS subject matter experts have been solicited to support the reviews. It is recommended that this informal practice continue, as needed, in the future. The submittals are typically concise (< 4 pages) such that the total review burden for the SLS reviewers is reasonable.

The review process includes individual SLS reviews and then follow-on group evaluation.

Group evaluation starts with an initial screening process followed by more detailed discussion of those submittals that have the most potential. The final part of the evaluation includes an assessment of whether the requested resources within each proposal align with the proposed work scopes, that potential projects have an appropriate balance across technical disciplines and project risk, and that the potential projects align with the budgeted resources. There have been occasions when iteration with the PI on the proposed work scope or budget was necessary to create this alignment.

4 Three of the six approved FY-23 FFR were submitted to IdeaScale: (1) Cognitive Bias - effect on regulatory safety evaluation of reactor protection systems; (2) Integration of Security, Safety, and Safeguards During Design and Operations Regulatory Implications; and (3) Nuclear Nano Technology -

Applications in Advanced Fuel Assemblies.

5 Explore Feasibility and Usability of ADAMS Text Twin 6 For example, an FY-23 agencywide announcement can be found here:

https://intranet.nrc.gov/announcements/standard/general-interest/reminder-the-future-focused-research-program-wants

9

3. Proposal Recommendation and Selection The approval process consists of the PM and SLS team preparing a short presentation for the RES Office Director (OD) and other senior managers that summarizes the submittal topics and general statistics regarding the number and type of submittals as well as the submitters affiliation. The recommended submittals are then discussed in more detail to give the OD the information needed to approve or modify the FFR portfolio for the upcoming year. The OD has the discretion to either remove or add approved programs from the recommended list. Thus far, there has been good alignment between the SLS recommendations and the final approved projects.

Once selections are made, the FFR program PM notifies submitters on the status of their submittal via email. It is important that this notification occur immediately after the projects are selected to provide timely feedback and to provide time for the PIs to initiate their projects.

4. Project Management Program management is a principal PM responsibility that entails supporting individual projects as well as holistically implementing the FFR program. It includes both technical and administrative (e.g., contracting and expenditures) monitoring; SharePoint site maintenance; Nuclepedia page maintenance; communication with PIs and managers; outreach and advertising; supporting FFR-days; and organizing the FFR RIC sessions. Typically, approximately 0.25 FTE per year has been expended for these duties. The PM creates individual Enterprise Project Identifiers (EPIDs) for each project to promote activity-level tracking and this has improved the ability to monitor progress and foster communication on progress between the PI and management. However, there are currently no other formal project management tools in place to allow more comprehensive oversight. While this is consistent with the original program vision to minimize program overhead expenditures, some modifications may be necessary given the visibility of FFR and associated deliverables.

5. Project Completion Project completion was initially the most ill-defined step in the process. While each submittal identifies planned deliverables and this is part of the ratings considerations, there had been little concerted thought about closeout activities and coordination when the FFR program kicked off in 2020. Major project deliverables were always intended to be retained in ADAMS and made publicly available, if appropriate. This process works well for formal reports, Research Information Letters, or workshop proceedings. It does not work well for more informal products such as seminars, training sessions, or white papers. However, the push to use Nuclepedia as an agency-wide KM tool in the late 2010s dovetailed nicely with the need to develop a comprehensive knowledge repository for the FFR program. The PIs are expected to create a project-specific Nuclepedia page which is linked to the FFRs Nuclepedia home page. This ensures that both the formal and informal products of FFR projects are widely available and easy to find as well as being in line with the agencys KM initiatives. A recognized best practice for project closeouts is the issuance of a concise closeout memo to RES Office Director (e.g.,

10 ML23116A033), which provides additional visibility and awareness of completion of an FFR project.

D. FFR Process Evolution The processes for FFR were developed quickly to initiate the program back in 2020 and have been continually refined by the FFR team based on lessons learned and through feedback from the staff. The FFR team continues to look for ways to make the program more manageable and effective.

The biggest evolutions within the review process have been to adjust the program vision, tailor the program to reflect the implications of the budgeted resource allocations and provide more direct PI support. The initial vision was that FFR include activities that span the project risk and complexity, potential reward, and maturation spectrum. However, Commission feedback in 2020 indicated that the activities selected in FY-20 and 21 were not as ambitious as expected.

As a result, although the program still strives for diversity in these areas, there has been a concerted effort to steer the portfolio toward less technically mature and more forward-looking topics. This shift has been advertised to the staff through the outreach activities indicated above but, more importantly, the project rating considerations have been modified to specifically encourage this increased emphasis on more forward-looking projects.

As mentioned earlier, the budgeted resources have recently been 2 FTE and $1000K. This balance, as well as the practical difficulties in relieving FFR PIs of their more immediate duties, has, in most cases, induced practical constraints on the availability of staff to perform the FFR work scope. The implication is that FFR is largely a staff-led, contractor-executed program.

Additionally, while the initial vision was that FFR activities would span 1 to 3 years, there has been a shift to complete activities, if possible, within 2 years. The project rating considerations have been modified to reflect these realities. The 2-year completion target is a good fit for the purposes of the FFR program and helps ensure that the funding is obligated and spent in a timely manner.

In the first few years of the FFR program, the project PIs were provided little oversight or guidance and allowed to independently manage their activities. However, for those projects relying on contracting, this approach sometimes led to lengthy delays in initiating the project due to contracting lead times. A challenge for the Acquisition Management Division (AMD) is that the FFR projects are not explicitly part of the advanced procurement plan (APP) and thus are not officially included in their workload planning. In recent years, the FFR program PM has teamed with both AMD and RESs management and program analyst to address this challenge and, for the last two years, has been working with the PIs to start the contracting process as early as possible in the FY. This process has improved the timeliness of the FFR contract submittals, but challenges remain, and while future process changes are planned to address these lingering issues, other changes are also recommended (see Section VII, Recommendations) to further enhance the FFR program.

11 E. Process Management The submittal and selection processes described in the previous section are largely managed through the FFR SharePoint site. The site is intended to be both a communication and KM hub with links to all related FFR information including the Idea Scale Challenge Campaign Incubator, the FFR Nuclepedia site, and a repository of prior year submitted and awarded projects that can provide examples of both submitted and selected activities to future submitters. There is also a repository of presentations and other background reports as well as an active list of frequently asked questions (FAQ) that is updated to address stakeholder questions and experiences. The most important function of the site, however, is to implement and manage the proposal submittal, evaluation, and selection processes. The proposal submittal link directs the submitter to a submittal template, which provides instructions, and also includes a link for uploading the proposal to complete the submittal process. The proposal template has been refined several times to clarify the requirements and to incorporate staff feedback and lessons learned.

Once the submittal has been uploaded, it is grouped with all other submittals for evaluation. A snippet of the master submittal table on SharePoint is depicted in Figure 3. The status column is used to identify the disposition of each submittal as either being received (prior to evaluation),

accepted, or not selected. The review notes for each submittal provide a qualitative summary of the SLS evaluation. Because the vast majority of FFR submittals received to date have been good ideas that are worthy of pursuing, the reviewers initially bin the submittals into one of the following categories: FFR, UNR, or other. Those binned as FFR fall within the programs scope. Those binned as UNR appear more appropriate to be pursued through the UNR process, while those falling into the other category appear more appropriate to be pursued through other agency processes, like Idea Scale.

For the UNR and other programs, the review notes typically provide the rationale for the binning and articulate some potential next steps for pursuing those ideas. For those ideas in the FFR bin that are not selected due to resource limitations, the review notes encourage the submitter to consider resubmitting next year and identify considerations that may strengthen the proposal.

There is at least anecdotal evidence that this practice is helpful as submitters have been selected within the next award cycle based, at least in part, on adopting suggestions within the review notes.

Figure 3: Examples from the FFR SharePoint Master Submittal Table

12 V.

Accomplishments This section and subsequent sections related to challenges and recommendations are delineated as either process or programmatic FFR attributes. The FFR process attributes are characterized by the management, logistics, and support needed to implement the FFR program. The FFRs programmatic attributes correspond to research activities meeting the vision and objectives of the program, as well as the future viability and growth of the FFR program. While this delineation is not always distinct, it provides a useful framework for analyzing the FFR program.

A. Process Several accomplishments related to process changes since the programs inception have previously been described in Sections IV.C, D, and E. Those, and other related, accomplishments comprise the following themes.

1. Process Agility The hallmark of these modifications and enhancements is the flexibility afforded by the small program team (i.e., 3 SLSs and an FFR program PM) and the relatively informal program nature. Process changes can be quickly identified and implemented because the program team has been given the latitude and authority to operate without using more formal agency processes (e.g., office instructions). This has been particularly useful when modifying the submittal template, rating considerations, initiating the Idea Scale Challenge campaigns, and developing Nuclepedia as an FFR KM repository.

2. Rapid Engagement of Newly Selected PIs The engagement with newly selected PIs soon after the awards are announced is a critical process enhancement. The lead-time associated with the contracting process and other associated challenges can be a significant barrier to timely project completion. Communication and planning between the PIs [or contracting officer representatives (CORs)], the FFR PM, AMD, and the RES Program Management, Policy Development & Analysis (PMDA) Division is therefore essential to ensure that RESs roles within this process are addressed expeditiously.

There is some evidence that this enhancement is helping. Two out of three of the FY-22 activities needing contract support have been placed and the remaining project is utilizing the NASA Tournament Lab, an innovative method for obtaining commercial contracting support. As of June 16, 2023, one of the FY-23 project contracts has been placed, and the remaining five of the FY-23 activities have APP placeholders.

3. Full Project Funding Provided at the Project Start Providing each projects requested funding amount within the FY that the project initiates has been a significant achievement that has largely worked as envisioned. Once the contract is awarded, the PI is freed from both concerns of possible project delays and needed contracting funding actions during subsequent fiscal years as the NRC waits for Congressional budget approval. Up-front funding also provides an indirect benefit by reducing funding uncertainties and allowing the contractor to better prioritize the work.

13

4. Centralized SharePoint Portal and Use of Nuclepedia A notable accomplishment has been the use of SharePoint to provide a centralized location for process management and communication. The site has been streamlined and only essential and readily apparent links remain. All the information and guidance needed to develop a submittal are readily accessible for prospective submitters. The use of a centralized portal with all FFR program information readily available has led to higher-quality FFR submittals. Further, the availability of proposal reviewer notes transparently provides the rationale for selecting projects, but more importantly, provides follow-on actions for current and possible future submittals. The High Entropy Alloy project, awarded in FY-21, emanated, in part, from reviewer comments provided to the submitter on their FY-20 proposal. The reviewer notes for the FY-23 submittal for exploring the feasibility of ADAMS Text Twins encouraged the submitter to reach out to OCIO to discuss this idea, and it is now part of their project on enhancing the agencys cognitive search tools.

Finally, the utilization of Nuclepedia to highlight the research findings ensures that the knowledge gained through the FFR program is readily available and easy to find. Although final reports and key documents are stored in ADAMS, the Nuclepedia pages summarize the work and provide links to the key documents as well as presentations and seminar video links that may not be in ADAMS. This is important for a program such as FFR where some immature areas investigated may lie dormant until the technology evolves.

B. Programmatic

1. Proposal Submittal Rates Since inception, 85 FFR proposals have been submitted and 24 have been approved (Appendix II), a 28% approval rate. Further, there have been at least 20 new submittals every year except for the FY-21 award cycle which was significantly compressed given the May completion of the FY-20 award cycle. There have been six project awards every fiscal year thus far.

Other successes include good support of the program. As expected for an RES-led program, the majority ( 70%, Figure 4) of the proposals submitted and awarded have emanated from within RES although, as indicated in V.B.3, the agency has been appropriately represented through both single and multi-office submissions. Within RES, DE has submitted the lion-share of the proposals, slightly more than DSA and DRA combined. However, the award rate is relatively constant across the divisions (Figure 4), which reflects that the quality and scope of the submissions is consistent among the divisions. The increased DE submittal rate may be buoyed by higher program visibility, spurred, in part, by DEs responsibilities as the FFR PM.

2. FFR Project Execution Five out of 24 (21%) of the projects have been conducted solely by staff in-house (Figure 5) while the remaining projects have relied on contractor support, with roughly equivalent support to date being provided by DOE laboratories and commercial organizations, buttressed by two university contracts. Eleven out of 12 of the FY-20 and FY-21 awarded projects have

14 completed, for an overall project completion rate of 46%. The remaining FY-20 project completion is expected in FY-23 which would result in a 100% completion rate within 3 years for the FY-20 and FY-21 projects. These statistics, but especially the completion percentage of the FY-20 and -21 projects, represent a significant accomplishment associated with both effectively standing-up the FFR program and completing initial projects within the targeted schedules.

Figure 4: Submitter and Award Affiliations RES - DE 35%

RES-DRA 14%

RES - DSA 19%

NRR 14%

OCIO 7%

NMSS 2%

NSIR 4%

Regions, 5%

Submissions: Total = 85 RES - DE 33%

RES-DRA 14%

RES - DSA 22%

NRR 15%

OCIO 4%

NMSS 1%

NSIR 3%

Regions, 8%

Awards: Total = 24

15 Figure 5: Breakdown of FFR Project Support The deliverables that have been completed to date (Appendix II and Figure 6) are consistent with what was envisioned during the programs inception and are appropriate for future-focused research. The preponderance of the deliverables are technical reports and Nuclepedia pages but there are also two Research Information Letters which summarize workshop proceedings.

Additionally, two seminars and a training workshop have been conducted and a software hub has been developed. These products collectively support knowledge transfer and management, communication, and provide a foundation for follow-on work in these areas, if appropriate.

Figure 6: FFR Deliverables None (NRC Staff) 21%

National Labs 27%

Pending 25%

Commercial 19%

Universities 8%

FFR Projects - Contractor Support Technical Reports, 13 Nuclepedia Pages, 13 RILS, 2 Seminars, 2 Training Workshop, 1 Software Hub, 1 FFR Deliverables

16

3. Broad Agency Support for the FFR Program There has been good participation in the program throughout the agency as submissions have been initiated from staff in all the major business line offices as well as the regions (Figure 4).

Figure 4 also demonstrates that the award rate is similar among the offices and there is no obvious bias toward approving RES submittals. An increasing trend has been for RES staff to partner with user offices on submissions. Prior to FY-23, there was a small number of such collaborations. However, in FY-23, 32% of the submittals and 50% of the awards were for multi-office projects, spanning eight divisions across 4 NRC offices. Some advantages of this collaboration are that they indicate buy-in from the user offices and provide some assurance that the milestones and deliverables are both appropriate and needed. The FY-23 multi-office submissions were also consistently high-quality which reflects a level of idea refinement and maturation not always evident in the single-office submissions. This trend should be nurtured in the future.

A more consistent trend is the participation of other offices during the execution of the FFR projects. This participation is evident in the final deliverables for completed FY-20 and -21 projects as well as the FFR closeout documents (Appendix II).

Another success is that the FFR portfolio is diverse and spans most of the agencys important technical disciplines. This observation is loosely reflected in the diversity of the submitters office affiliations for awarded projects (Figure 4). More specifically, the scopes of awarded projects pertain to structural and fuel materials, risk-informed assessment and decision-making, human factors, security, safeguards, severe accidents, artificial intelligence/machine learning, decommissioning, radiological consequences, plant system operation and performance, thermal hydraulics, and digital instrumentation & control.

4. FFR Project Portfolio A related success of the program, as it has matured, has been the ability to promote and award innovative and forward-looking projects while also developing near-term, impactful program deliverables. Some reflective examples of innovative FY-22 and -23 projects include use of machine learning to prioritize inspections, examining use of a zero-trust paradigm for cyber security, use of innovative materials for possible fusion or advanced fuel development, neutralizing cognitive biases in safety evaluations, and exploring a performance monitoring strategy to enhance consistency in risk-informed decision-making. While similar projects can be found in the FY-20 and -21 projects, the collective portfolio in those years represented more evolutionary research activities such as modernizing TRACE, a Dynamic PRA study, and developing a web-based bot to aid researchers. However, several early projects have demonstrated the potential for the FFR program to prepare the agency for emerging opportunities. In particular, the projects on digital twins and applying the existing Licensing Modernization Project (LMP) methodology on an operating reactor have produced several products (see Appendix II) that have increased agency awareness of these technologies and highlighted regulatory opportunities. These products have allowed the agency to start preparing for potential regulatory applications and have motivated discussions on potential future research activities.

17 VI.

Challenges Several challenges and the evolutionary changes that could be readily implemented to address these challenges were previously described in Sections IV.C, D, and E. This section identifies more systemic issues that cannot solely be addressed by the FFR program team.

A. Process

1. FTE Allocation for the FFR Program Although adequate FTE has been budgeted for the program thus far, the budgeted FTE has not been allocated in a manner that supports effective staffing decisions. Specifically, the FTE budgeted for the FFR program has been retained at the office level and has not been distributed among the RES technical divisions and reflected in division level staffing plans. The impact of this decision is that the RES technical divisions, who execute the majority of FFR projects, are underbudgeted for their workload. The mismatch between the staffing plan budget and actual workload requires first-line supervisors to balance executing FFR activities with activities directly supporting nearer-term user office needs. Coupling this tension with hiring challenges, supervisors can be faced with the difficult decision of deprioritizing FFR activities. While there is no evidence that this issue has led to significant FFR delays, it remains a concern.

In addition, the FFR program PM resources have remained static as the programs visibility and PMs duties have expanded as the program has matured. The challenge, in keeping with the programs vision, is to identify an optimal strategy for performing the PM duties so FTE can be effectively budgeted and managed during each FY. Because of the strain on PM resources, the SLSs, who dont charge to the program, often assume PM-like responsibilities to address gaps that could more efficiently be handled by a PM to help ensure the programs overall success.

Therefore, sufficient and strategic FTE allocation, including reflecting this FTE within staffing plans and ensuring adequate PM resources, would support better alignment between staffing levels and the FFR program budget and help alleviate existing resource constraints.

2. FFR Project Contractor Support As resource allocation has dictated that FFR predominantly operate as a staff-led, contractor-executed program, efficiently navigating the agencys contracting process can become a significant challenge to timely project initiation and execution. This challenge is most apparent in FY-23 as all awarded projects currently require contracting support and they all are in various stages of the contracting process. As of early June 2023, only one of the needed FY-23 project contracts have been awarded despite the best efforts of the PIs, the PM, AMD, and PMDA. The early engagement of these entities, soon after the selections have completed, has resulted in marginal, but not sufficient, improvements to this hurdle. For the FY-22 cycle, contracts for two projects were not placed within the fiscal year and needed to carry over into FY-23, with one still remaining to be awarded. Meeting the programs vision of completing most projects within 3 years becomes difficult if the work cannot begin until 9 to 12 months (or more) after the award selection. This also causes potential funding issues in that FFR funds cannot be carried over if

18 they are not placed on a contract, so the inability to place contractor work in a timely manner can result in funding challenges in the subsequent year.

A related challenge is the identification of appropriate contractors, especially for activities related to exploratory, immature technical areas. RESs expertise lies in contracting with national laboratories and commercial entities, especially ones with enterprise-wide agreements or contracts (EWAs or EWCs). Expertise in nascent technical areas, however, can exist in academia or other non-traditional sources that have little experience contracting with the NRC.

Nascent technologies also typically have a narrower pool of possible contracting options. These factors can conspire to make contracting a challenge.

The FFR program investigating advanced manufacturing technologies for nuclear fusion structural component applications has encountered this issue. The program was awarded in FY-22. The first effort to place work under an EWC was denied. The next effort identified a potentially technically acceptable university source, but the source and NRC could not reach timely agreement on the terms and conditions of the contracting action. A third contracting effort to identify sources yielded one response which did not have appropriate expertise. Currently, a fourth contracting effort is using NRCs agreement with NASA to leverage their Open Innovation Services 2 contract. This contract submits a request for a task plan and uses expert crowdsourcing to identify viable sources that can satisfy the proposed work scope. This novel approach may prove to be a viable option for more efficiently identifying viable contracting sources but its successful use in contracting nuclear research has yet to be demonstrated. This issue is expected to become more significant as the FFR program implements more innovative FFR projects.

3. Communication and Responsiveness of Principal Investigators A final process challenge is effective communication among the FFR project team and the dispersed PIs who support the actual research efforts. The FFR PM has experienced difficulties in establishing a centralized, comprehensive database for managing project information, including work status, spending, milestone delivery tracking and completion. Obtaining accurate information can be challenging due to the limited program management resources applied to the FFR program (approximately 0.25 FTE per fiscal year), competing workloads, and the need to collect timely information from twelve or more PIs. This challenge may be exacerbated by the novelty of the FFR process, especially as communication and coordination with the assigned RES/DE Regulatory Guide Project Management Branch (RGPMB) PM is an additional step that is not necessary in other RES research projects (e.g., contract management, technical execution, etc.). A contributing factor may be a lack of focus on the need for integrated FFR program coordination and communication throughout the supervision and management chain.

B. Programmatic

1. Alignment of FFR Projects with the Agency Mission A fundamental tenet of the FFR is to conduct forward-looking research in technical areas which may not align with near term regulatory needs. Because these projects are often exploring potential future topical areas, they may not lead to immediate technical, regulatory, or

19 programmatic impacts. This is not an unexpected result for future focused research and even projects that yield no immediate regulatory impacts provide useful knowledge. However, the FFRs focus on innovative, forward-looking research may lead to a perception of conflict between the FFR program and how the agency implements its mission to license and regulate the Nations civilian use of radioactive materials. The bulk of the agencys work is focused on near-term (i.e., < 3 years) technical and regulatory needs and the agencys research is structured to support this objective. Consequently, NRCs research is generally more applied, nearer-term, and focused on mature technologies than research pursued within other industries and the DOE. This competition between FFR projects and more immediate mission-related work (e.g., User Needs, Research Assistance Requests) requires continual communication to ensure that the objectives and goals of the FFR are well understood and implemented throughout the agency.

2. Ownership of FFR Projects While, as indicated in the accomplishments, projects have generally been completed at an acceptable rate, there have been barriers that have impeded executing certain activities. One such barrier is when the submitter of an accepted proposal is not the project PI. This is unavoidable in some cases when the idea champion has neither the expertise nor the time to undertake an FFR project lead. This situation is most likely for ideas submitted from outside of RES. Another common situation is when the idea champion and/or PI moves to a new job either within or outside the agency. In these situations, the project lead must be transferred, typically to someone in research, and the new lead may not have the same vision, expertise, or commitment to the project. Such transitions have occurred on projects related to veterinarian use of radiopharmaceuticals, fusion reactor materials, and use of machine learning to prioritize inspections and have resulted in project delays during the transition.

3. Resource Commitment for FFR Project Principal Investigators Another consideration is that the demand on FFR PIs has been greater than initially envisioned, intended, or advertised. Due to the programs visibility, FFR PIs have additional responsibilities compared to PIs for traditional RES projects. Extra responsibilities include yearly participation in FFR-days and RIC sessions, agency-wide presentations and KM seminars, additional contracting responsibilities, frequent requests for information from the FFR PM, and the expectation to create a Nuclepedia page. While none of these burdens is problematic individually, they may collectively have a negative impact. While the submittal rate has been relatively constant over the first four years of the program, the concern is that the future submittal rate will decrease if the program is seen as too much of a burden. This issue may also be related to the process challenge associated with communication and responsiveness of the PIs.

20 VII.

Recommendations This section provides implementable recommendations intended to mitigate the challenges identified in the previous section and support the continued maturation and future viability of the FFR program.

A. Process

1. Shift Proposal Review, Evaluation, and Approval Earlier in Fiscal Year The easiest process change is to shift the FFR project approval process timeline forward by 3 months (i.e., target final FFR project selection by June of each year rather than September) to support timelier and streamlined planning and contracting. This modification will allow APP entries for those FFR projects that will be starting during the next FY so that AMD can appropriately plan and budget for those activities. This change will also provide the PIs with an opportunity to further refine their plans so that they are ready to either initiate a contract or start work at the beginning of the FY. Any metrics related to project completion should remain at the beginning of the FY, but this will provide the PIs will a greater opportunity to complete activities within 2 years.

This recommendation addresses Challenge VI.A.2 related to contractor support for the FFR program.

2. Develop Checklist to Assist Principal Investigators in Executing Projects Another straightforward change is to develop and issue a PI checklist that establishes clear expectations for all aspects of the process including project initiation, status and fiscal tracking, milestone and deliverable completion, identification of appropriate RES Operating Plan milestones, and expectations for periodically reporting these aspects to the PM to support overall program monitoring and communication. This checklist recommendation builds on the success of the PI contracting initiation and training sessions that have been implemented by the PM over the last few years.

Coupled with this recommendation, the FFR program PM should develop a program tracking tool that can effectively collate, analyze, and communicate this information to various program stakeholders. A more efficient way of managing the program is needed, especially if more formal metric development and tracking is envisioned as the FFR program continues to mature.

Excel is a simple, yet acceptable, tracking tool but there are other agency and commercial project management tools that could be leveraged as well. The strength of these other tools is in more easily generating professional reports and graphics.

This recommendation addresses Challenges VI.A.3 and VI.B.3 related to communication, responsiveness, and resource commitment expectations for FFR project PIs.

3. Provide FTE Allocation to RES Technical Divisions A final process recommendation is that the programs FTEs should be equitably allocated to the divisions or branches, as appropriate. Since the FFR has typically been budgeted at 2 FTEs,

21 the FTE burden associated with FFR program PM and other support services should be identified and provided to responsible organizations. There is now 3 years of data that can be used as a basis for these estimates assuming the programs requirements will remain stable.

The remaining budgeted FTE should be equally distributed to the RES divisions to support FFR project PI resources while retaining sufficient FTE for PIs in other offices that are executing FFR projects. Allocation of FFR resources to the RES technical divisions may help ease the competition for resources by allowing the divisions to build staffing plans that better match anticipated workload. The FTE could support hiring either permanent or rotational/temporary staff, specifically to implement existing and future FFR activities. It also motivates the divisions to continue supporting the FFR through future submittals. Further, incrementally increasing the FFR program FTE budget in future budget cycles would further support the objectives of staff enrichment and building foundational knowledge through more direct staff engagement in FFR research activities.

This recommendation addresses Challenge VI.A.1, VI.B.2, and VI.B.3 related to FTE allocation for the FFR program, ownership of FFR projects, and resource commitment expectations for FFR project PIs.

B. Programmatic

1. Maintain Engagement on FFR Program at All Levels in the NRC Recommendations to address inherent conflicts between the FFRs longer-term and agencys near-term focus are more complex. It will be important to continue to engage agency senior management to maintain their awareness of the FFR program but, more importantly, to communicate the efficacy of the program. These socialization efforts led to the successful launch and initial acceptance of the program and such outreach should be continued. Outreach at both the division and branch levels should also be encouraged to promote awareness and acceptance throughout the agency. However, the success of the FFR program will ultimately rest on the achievements and recognition of its individual projects. Therefore, the continual generation of viable proposals from across the agency, the thoughtful selection of each yearly FFR portfolio, the timely completion of activities, and the communication of research findings are all essential attributes that must be cultivated. Multi-office submissions and participation of user-office staff during the execution of FFR should continue to be promoted. This engagement conveys that the user office recognizes the value of the activity and is vested in creating useful project outcomes. In addition, inclusion of appropriate FFR project milestones in the RES Operating Plan would help raise the visibility and status of projects and help ensure timely completion of work. Consideration should also be given to providing periodic communication, on an annual basis, regarding the state of the FFR program. This communication should outline newly awarded projects, provide status of ongoing projects, and summarize completed projects and deliverables. Lastly, the issuance of concise closeout memos to the RES Office Director to document the completion of an FFRE project is a recognized best practice, helps to raise awareness and visibility of FFR work, and should be required.

This recommendation addresses Challenge VI.B.1 related to alignment of FFR projects with the agency mission.

22

2. Leverage External Organizations Strategic considerations to leverage existing FFR resources while promoting its vision and increasing its visibility should be pursued. One potential strategy is to identify topics being pursued in select FFR activities within the yearly University Grants Program Funding Opportunity Announcement (FOA) to generate synergy and interest from academia. For example, after the FY-24 FFR awards are selected, a few of the most relevant technology areas within those awards could be identified and explicitly solicited within the upcoming FOA. This strategy promotes accelerated maturation or evaluation of nascent technologies that may have nuclear applications.

A related strategy is to forge partnerships with the DOEs existing LDRD program. There are plentiful opportunities that could be pursued through such a partnership including encouraging joint submissions within both programs, crediting submissions for one program (i.e., FFR or LDRD) that leverage an existing activity in the other program and promoting short-term rotational assignments between NRC and DOE-laboratory staff.

This recommendation addressed Challenges VI.B.1 and VI.A.2 and VI.B.1 related to project contract support for the FFR program and alignment of FFR projects with the agency mission.

3. Improve Contracting Process for the FFR Program The various contracting challenges underscore the need to establish more efficient contracting processes that directly support the FFR projects. The use of the NASA Open Innovation Services 2 contract is an intriguing approach but, even if it is successful, it will not address all the contracting challenges identified in this paper. RES and ADM should align on strategies to best support FFR through all available contracting flexibilities. For example, EWA, EWC, or Small Business Innovation Research (SBIR) agreements could be tailored specifically for the FFR. This might allow individual FFR projects to be quickly placed within these broader contracting vehicles. Additionally, some of the budgeted FTE should be allocated to the RES technical divisions for COR support to ensure that this critical FFR support activity is appropriately reflected in RES division level staffing plans.

This recommendation addressed Challenge VI.A.2 related to FFR project contract support.

4. Incentivize Staff and Management Support of the FFR Program One direct method for ensuring continual support for FFR within RES management is to develop yearly RES Senior Executive Service (SES) performance plan metrics and RES Branch Chief elements and standards related to the development of high-quality, in-scope submissions as well as the execution of awarded projects. This action could provide additional incentive for management to reinforce the importance of the FFR program with first line supervision and encourage staff participation. The historical number of submittals ( 20 to 25) has resulted in a reasonably competitive award rate of 25% so the objective of any such metrics is not to increase submittals but to maintain that rate into the future.

Finally, given the visibility and requirements placed on PIs when they are awarded an FFR project, there should be some measures designed to increase the prestige of both being

23 awarded and successfully completing an FFR program. The prestige associated with the DOE LDRD awards is one contributing factor to its enduring success. Some obvious ways to increase the prestige is through small monetary awards through existing agency processes such as performance, special-act, time-off, group awards, employee of the month, etc. There are various non-monetary incentives that could be developed such as special award or recognition ceremonies, acknowledgement on yearly performance reviews, and highlighting important project milestones within the office-wide newsletters of the FFR PI(s). Other potential ideas include recognition by senior management of FFR PIs, such as hosting an annual FFR PI breakfast, a meet and greet for newly selected FFR PIs with senior management or the Chairman, and/or acknowledgement on the Office webpage.

This recommendation addresses Challenge V.A.3 and V.B.2 related to communications and responsive of FFR project PIs and ownership of FFR projects.

VIII.

Conclusions Overall, the project team believes that the FFR program has successfully met the intended vision and objectives for the program. The structure of the program has enabled a high degree of agility for the FFR process and allowed the project team to make rapid adjustments to meet evolving demands. Further, the FFR program has successfully engaged a wide range of agency staff at a grass roots level and has supported projects that meet future regulatory needs. The FFR program has also leveraged capabilities across a spectrum of external organizations, including universities, national laboratories, and commercial contractors.

Although the team identified several recommendations to ensure continued growth and success of the program, these items should be viewed as enhancements to an otherwise robust program that helps the agency include innovative research ideas in our research portfolio.

24 Appendix I: Long-Term Research Project Key Documents (2006-2016)

SRM-COMDEK-06-0001 - "FY 2008 Budget Proposal."(ML062350498)

SECY-07-0192 - Agency Long-Term Research Activities for Fiscal Year 2009 (ML072780393)

SECY-08-0057 - Agency Long-Term Research Activities(ML080870353)

SECY-09-0021 - Agency Long-Term Research Activities for Fiscal Year 2011 (ML090120182)

SECY-10-0013 - Agency Long-Term Research Activities for Fiscal Year 2012 (ML100221006)

SECY-11-0012 - Agency Long-Term Research Activities for Fiscal Year 2013 (ML110100020)

SECY-12-0013 - Agency Long-Term Research Activities for Fiscal Year 2014 (ML113560048)

SECY-13-0040 - Agency Long-Term Research Activities for Fiscal Year 2015 (ML13059A094)

SECY-14-0032 - Agency Long-Term Research Activities for Fiscal Year 2016 (ML14038A118)

SECY-15-0042: Agency Long-Term Research Activities for Fiscal Year 2017 (ML15015A576)

COMSECY-16-0006, Revision to the Agency's Long-Term Research Program and Related Reporting to the Commission, (ML16021A346); SRM (ML16104A299)

COMSECY-16-0021, Recommendations Regarding the Agency's Long-Term Research Program, (ML16242A019); SRM (ML16313A521)

NUREG/BR-0506, Long-Term Research Program: Addressing future regulatory and technological needs, (ML12346A114)

25 Appendix II: Awarded Project Summary Project Title Approv al Year Contributing Office Principal Investigator Contractor Status Key Deliverables (Final Report, Workshop, Presentations, etc.)

Digital Twins -

Regulatory Viability 2020 RES/DE/CIB Raj Iyengar INL ORNL Complete Technical Letter Reports (TLRs):

TLR/RES-DE-REB-2021-01, The State of Technology of Application of Digital Twins (ML21160A074)

TLR/RES-DE-REB-2021-17, Technical Challenges and Gaps in Digital Twin Enabling Technologies for Nuclear Reactor Applications (ML21361A261)

Research Information Letters (RILs):

RIL 2021-02, Proceedings of the Workshop on Digital Twin Applications for Advanced Nuclear Technologies (ML21083A132)

RIL 2021-16, Proceedings of the Workshop on Enabling Technologies for Digital Twin Applications for Advanced Reactors and Plant Modernization (ML21348A020)

Public Meeting Summary Summary of the Observation Public Meeting on Regulatory Considerations and Opportunities for Digital Twins in Nuclear Reactor Applications (ML22091A007)

26 Project Title Approv al Year Contributing Office Principal Investigator Contractor Status Key Deliverables (Final Report, Workshop, Presentations, etc.)

Apply the Licensing Modernization Project Methodology on an Operating Reactor 2020 RES/DRA/PRB Matthew Humberstone Submitted by Mike Cheok (RES/DRA)

INL Complete Project Reports:

Phase 1 Report (ML21244A366)

Phase 2 (Final) Report (ML23108A002)

Dynamic PRA Study 2020 RES/DRA/PRAB Michelle Gonzalez INL Active7 Seminar:

Seminar (November 21, 2019)

Training Workshop (Nov 18, 2020)

Part 1 of 2 Part 2 of 2 Investigate Use of STAMP for Nuclear Safety System Design and Regulation 2020 RES/DE/ICEEB OCIO Mauricio Gutierrez Submitted by Kathy Lyons-Burke (OCIO)

Advanced Engineering Services, LLC Complete Project Reports:

Investigation of the Use of Causal Analysis based on System Theory (CAST) at the NRC (ML22277A013)

Investigation of the Use of System-Theoretic Process Analysis at the NRC (ML22272A315)

Package accession number:

ML22272A314 Automation Tool Mapping 2020 OCIO/GEMSD/APIB Nicholas

Buggs, John Beatty Submitted by Kathy Lyons-Burke (OCIO)

None Complete Nuclepedia Page:

https://nuclepedia.usalearning.gov/ind ex.php/Automation_Tool_Mapping:_A

_Future_Focused_Research_Project 7 This activity is expected to be completed in 2023.

27 Project Title Approv al Year Contributing Office Principal Investigator Contractor Status Key Deliverables (Final Report, Workshop, Presentations, etc.)

TRACE Modernization 2020 RES/DSA/CRAB Joseph Kelly INL Complete Project Presentation:

TRACE Modernization: A Feasibility Study (ML23151A243)

"Can I pet my pets????" A Dosimetry Risk Analysis for Veterinarian Use of Radiopharmaceutic als 2021 RES/DSA/RPB Vered Shaffer Renaissance Code Development, LLC Complete Technical Letter Report:

RCD-22-007, Internal & External Dose Research on Human Exposure to Pets Receiving Medical Administrations Of Radioactive Material (ML22196A042)

Drones and Virtual Reality Tools to Analyze Radiological Surveys in Decommissioning 2021 RES/DSA/RPB Stephanie Bush-Goddard PNNL Complete Technical Report:

PNNL-32519, Rev 1, Drones for Decommissioning (ML22196A040)

Assessment of Model Based Software Engineering (MBSE) Processes in a Regulatory Review Context for Digital Instrumentation&

Control 2021 NRR/DEX/EICB Norbert Carte, Derek Halverson Galois, Inc.

Complete Technical Report:

Final Report - High Assurance Rigorous Digital Engineering for Nuclear Safety (HARDENS) Project (ML22326A307)

Software Hub:

https://github.com/GaloisInc/HARDEN S

Key Attributes for Remote Operation of Nuclear Power Plants 2021 NRR/DRA/APLC NRR/DRO NRR/DANU OGC/LHE/RP OCIO Shilp Vasavada None Complete Project Report:

Ground Rules for Regulatory Feasibility of Remote Operations of Nuclear Power Plants (ML21252A303)

28 Project Title Approv al Year Contributing Office Principal Investigator Contractor Status Key Deliverables (Final Report, Workshop, Presentations, etc.)

RESbot - A Web-based bot to Aid RES Researchers 2021 RES/DE/REB OCIO NMSS/DFM/MSB Raj Iyengar SNL Engineering Mechanics Corporation of Columbus Complete Technical Letter Report:

TLR-RES/DE/REB-2022-13, Autonomous Researcher Feasibility Studies (ML22350A500)

Seminar (9/22/22):

RES Seminar Video - 9/22/2022 RES Seminar Slides - 9/22/2022 High Entropy Alloys 2021 RES/DE/CMB Wendy Reed University of Wisconsin System -

Madison Complete Technical Letter Report:

TLR-RES/DE/REB-2023-01 "Use of High Entropy Alloys (HEAs) in Future Nuclear Applications (ML23030B836)

Use Machine Learning to Prioritize Inspections 2022 RES/DRA/HFRB Song-Hua Shen Sphere of Influence (Sphere OI)

Inc.

Active work in progress Applicability of Current Atmospheric Dispersion Models for Extreme, Persistent Cold Locations 2022 NRR/DEX/EXHB Michael Mazaika None Active work in progress

29 Project Title Approv al Year Contributing Office Principal Investigator Contractor Status Key Deliverables (Final Report, Workshop, Presentations, etc.)

Characterizing Cyber Security Using AI/ML 2022 RES/DE/ICEEB Doug Eskins Purdue University Active work in progress Review of Advanced Manufacturing Technologies (AMTs) for Fusion Reactor Materials 2022 RES/DE/MEB Austin Young, Christopher Nellis pending Active work in progress Application of Natural Language Processing to NRC Regulatory Documents 2022 RES/DSA/AAB Alfred (Trey)

Hathaway None Active work in progress A Zero Trust Paradigm for Cyber Security in New Reactors 2022 RES/DE/ICEEB NSIR/DPCP/CSB Anya Kim, Kim Lawson-Jenkins None Active work in progress Sea Level Rise:

Scenarios and Projections 2023 RES/DRA/FXHAB Elena Yegorova pending Active work in progress Nuclear Nano Technology -

Applications in Advanced Fuel Assemblies 2023 RES/DE/REB RES/DE/MEB RES/DSA/FSTB NRR/DANU/UTB1 Austin Young, Mark Yoo, James

Corson, Meg Audrain pending Active work in progress Use of NCEP data to support severe accident consequence 2023 RES/DSA/AAB Amy Sharp pending Active work in progress

30 Project Title Approv al Year Contributing Office Principal Investigator Contractor Status Key Deliverables (Final Report, Workshop, Presentations, etc.)

analysis at locations without onsite met. data How to Neutralize Effect of Cognitive Biases on Regulatory Safety Evaluation 2023 RES/DE RES/DRA/PRAB NRR/DRO Sushil Birla, Susan

Cooper, David Desauliniers pending Active work in progress Integration of Security, Safety, and Safeguards During Design and Operations 2023 RES/DE/REB NSIR/DSP NMSS/DFM Chandran Nachiketh, Al Tardiff, Jim Rubenstone, John Matrachisia SNL Active work in progress A Performance Monitoring Strategy to Enhance Consistency in Risk-Informed Decision-Making 2023 Region III/DRP John Hanna pending Active work in progress

31 Appendix III: Proposal Submittal Template Brief Proposal Title Proposer Name and Organization OFF/DIV/BRN Proposal Date YY/MM/DD Future Regulatory Need(s) Addressed Try to be specific regarding needed improvement or gaps in targeted regulatory functions and/or programs. Indicate how the proposed FFR activity could address these needed improvements or gaps.

Technical Issue(s) Addressed Describe the technical problem(s) and, as appropriate, why currently available methods, tools, knowledge, etc. need improvement.

Proposed Approach Address how the work will be done (include project activities and milestones) and by whom.

Expected Products and Anticipated NRC Use Identify and discuss the products being developed by the FFR activity and how these products are intended to be used by staff. If the proposed activity is a seed activity (such that follow-on work is needed to develop a usable regulatory or technical product), indicate this in your discussion, and articulate the envisioned final products and their potential use by staff.

Schedule Indicate high-level milestones indexed to the start of funding.

Resources Requested Indicate both staff FTE and contractor cost broken down by fiscal year. Be mindful of the targeted size of the FFR program. See the table below as an example. Feel free to add text above the table if further explanation is needed.

Resources Requested

32 FY Staff FTE Contract Cost 2023 0.15 100K 2024 0.1 60K Total 0.25 160K Anticipated Time Horizon for Regulatory Application Please provide an estimate of the anticipated time frame in which this proposal might benefit the NRC in some regulatory application.

Ratings Considerations Please address each of the high-level rating considerations identified below. For insights, see the rating considerations listed in the Appendix.

1. Agency Impact

2. Future Focused

3. Proposed Research Plan, Objectives, and Approach

4. Resource Leveraging Other Points for Consideration Use this space to provide necessary background not covered by preceding text, as well as any other information that you think is important to the evaluation. Please indicate in this section if you would like to identify NRC experts who can provide a knowledgeable, independent evaluation of your proposed activity.

Sharing your draft proposal on the IdeaScale NRC Innovate platform is not required for proposal submittal, but it is a great way to solicit broad agency feedback on your proposal and is strongly encouraged. To facilitate sharing of your idea on IdeaScale, look for the FFR Idea Incubator FY24 Challenge Campaign. Once you have gathered some feedback, update your proposal and submit using the Submit Proposal link on the FFR SharePoint site. If the proposal idea has been shared on IdeaScale, please let us know in this section.

33 APPENDIX Guidance for Proposers (not included with proposal submission)

1) There is a four-page limit to the proposal (normal fonts, margins, line spacing). Note that proposals longer than this page limit may be referred to the submitter to be revised.

2) Be succinct. Bulleted text is fine.

3) Address all requested information items but try not to be repetitive. Feel free to use Not Applicable (N/A) or see above as appropriate.

4) Aim for a technical audience that might not include experts in the specific activity area of interest.

FFR Project Rating Considerations Agency impact Improves NRC staff and contractor future regulatory capabilities and readiness.

o Improves foundational knowledge important to future regulatory decision-making.

o Addresses recognized regulatory and/or technical gaps.

Future Focused Anticipates a technical or regulatory challenge that the agency may face within the next several years (i.e., 5 years or more).

Is a riskier blue sky project that is not an immediate regulatory need or follow-on task that would be better addressed through the User Need or other established processes.

Proposed Research Plan, Objectives, and Approach Establishes clear research objectives, methods, and deliverables.

Scope of work can reasonably be accomplished within typical FFR budget and schedule (generally within 2 years and no greater than $250K of contract support).

Sufficient staff interest, expertise, and support exists to complete the research proposal.

Resource leveraging Potentially benefits multiple NRC programs.

Leverages available resources for related research with ongoing or planned NRC or external (e.g., DOE, DoD) efforts.

Enhances synergies with university research programs (e.g., Grant Program), small businesses, etc., as appropriate.

34 File Naming Convention for Proposal Please re-name your proposal template file using the date and your LAN ID in the following format, date-LAN ID. Use a six-character date starting with the year (YYMMDD). The following is an example file name for a proposal submitted July 27, 2020.

200727-mxc1.docx If you are submitting more than one, add -# after the LAN ID ("200727-mxc1-1.docx," 200727-mxc1-2.docx," etc.)

If there is more than one staff member involved with the proposal, list a single member who will serve as a primary point of contact.