ML18347B038
| ML18347B038 | |
| Person / Time | |
|---|---|
| Issue date: | 12/13/2018 |
| From: | Baker B NRC/OIG/AIGA |
| To: | Margaret Doane NRC/EDO |
| References | |
| OIG-19-A-06 | |
| Download: ML18347B038 (26) | |
Text
Audit of NRCs Process for Developing and Coordinating Research Activities OIG-19-A-06 December 13, 2018 All publicly available OIG reports (including this report) are accessible through NRCs Web site at http://www.nrc.gov/reading-rm/doc-collections/insp-gen
UNITED STATES NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20555-0001 OFFICE OF THE INSPECTOR GENERAL December 13, 2018 MEMORANDUM TO: Margaret M. Doane Executive Director for Operations FROM: Dr. Brett M. Baker /RA/
Assistant Inspector General for Audits
SUBJECT:
AUDIT OF NRCS PROCESS FOR DEVELOPING AND COORDINATING RESEARCH ACTIVITIES (OIG-19-A-06)
Attached is the Office of the Inspector Generals (OIG) audit report titled Audit of NRCs Process for Developing and Coordinating Research Activities.
The report presents the results of the subject audit. Following the December 11, 2018, exit conference, agency staff indicated that they had no formal comments for inclusion in this report.
Please provide information on actions taken or planned on each of the recommendations within 30 days of the date of this memorandum. Actions taken or planned are subject to OIG followup as stated in Management Directive 6.1.
We appreciate the cooperation extended to us by members of your staff during the audit. If you have any questions or comments about our report, please contact me at (301) 415-5915 or Sherri Miotla, Team Leader, at (301) 415-5914.
Attachment:
As stated
Office of the Inspector General U.S. Nuclear Regulatory Commission Defense Nuclear Facilities Safety Board OIG-19-A-06 December 13, 2018 Results in Brief Why We Did This Review Audit of NRCs Process for Developing and Coordinating Research Activities The Nuclear Regulatory Commission (NRC) is responsible What We Found for assuring safety in the design, construction, and operation of OIG found that NRCs process for developing, using, and coordinating research commercial nuclear facilities and activities is adequate; however, opportunities to improve effectiveness and in the other uses of nuclear efficiency exist. Specifically, NRC should improve its processes for work materials, such as in medicine and requests, tracking research activities, and the use of quality survey industrial activities. As a key component of nuclear safety, NRC information.
carries out a research program to provide independent information Aspects of the research work request process can be more efficient. An and expertise needed to support organization and its programs should operate in an efficient manner.
NRC's decision-making process However, senior management is not involved and aligned early enough in the and to identify and characterize work request process. As a result, research activities can become delayed.
technical questions that may become important safety issues in the future. The Operating Plan is not consistently used by RES staff to track work requests.
RES staff should use it to track the progress and completion of work.
NRC's research program is carried However, the Operating Plan does not have the features to effectively and out by the Office of Nuclear efficiently fulfill its intended purpose, and management does not require its Regulatory Research (RES). RES use. As a result, agency staff are spending time reconciling work request supports the agency mission by information due to a lack of confidence in the Operating Plan data.
providing independent technical analysis and advice, tools, and information for identifying and Further, RES Product Quality Survey instrument does not provide complete, resolving safety issues, making accurate, and reliable information. NRC management should obtain relevant regulatory decisions, and data from reliable sources in a timely manner. However, there is insufficient promulgating regulations and guidance for the survey process to ensure reliable data. As a result, the guidance for nuclear power plants survey tool does not provide the information necessary to effectively assess and other facilities and materials and improve RES products.
regulated by the agency.
What We Recommend The audit objective was to assess the effectiveness and efficiency of This report makes four recommendations to improve the effectiveness and the development, use, and the efficiency of the development, use, and coordination of research coordination of research activities. activities. Agency management stated their general agreement with the findings and recommendations in this report.
Audit of NRCs Process for Developing and Coordinating Research Activities TABLE OF CONTENTS ABBREVIATIONS AND ACRONYMS .......................................................... i I. BACKGROUND ................................................................................ 1 II. OBJECTIVE ...................................................................................... 4 III. FINDINGS......................................................................................... 4 A. Opportunities Exist To Improve Efficiency of the Work Request Process ................................................................... 4 B. RES Operating Plan Inconsistently Used To Track Work Requests ...................................................................... 8 C. RES Product Quality Survey is Not Effective ...................... 13 IV. CONSOLIDATED LIST OF RECOMMENDATIONS ....................... 17 V. AGENCY COMMENTS ................................................................... 18 APPENDIX A. OBJECTIVE, SCOPE, AND METHODOLOGY ............................... 19 TO REPORT FRAUD, WASTE, OR ABUSE ............................................. 21 COMMENTS AND SUGGESTIONS .......................................................... 21
Audit of NRCs Process for Developing and Coordinating Research Activities ABBREVIATIONS AND ACRONYMS FY Fiscal Year NRC Nuclear Regulatory Commission OIG Office of the Inspector General RES Office of Nuclear Regulatory Research i
Audit of NRCs Process for Developing and Coordinating Research Activities I. BACKGROUND Nuclear Safety Research NRC is responsible for assuring safety in the design, construction, and operation of commercial nuclear facilities and in the other uses of nuclear materials, such as in medicine and industrial activities. As a key component of nuclear safety, NRC carries out a research program to provide independent information and expertise needed to support NRC's decision-making process and to identify and characterize technical questions that may become important safety issues in the future. NRCs regulatory research is designed to improve the agency's knowledge in areas where uncertainty exists, safety margins are not well characterized and regulatory decisions need, or will need to be confirmed.
NRCs Research Program NRCs research program addresses issues related to nuclear reactors, nuclear materials, and radioactive waste. The research program focuses on the challenges of an evolving industry, as well as on retaining technical skills when experienced staff members retire. NRCs research program examines a broad range of subjects, such as material performance,1 reactor events,2 computer codes,3 new and evolving technologies, and the human side of reactor operations. The research program involves about 6 percent of the agencys personnel and uses about 14 percent of its contracting funds. NRCs $42 million research budget for fiscal year (FY) 2018 included contracts with national laboratories, universities, research organizations, and other Federal agencies.4 Figure 1 illustrates the primary areas of research and the funding associated with each area.
1 Such as aging management of reactor components and materials, environmentally assisted degradation and cracking of metallic alloy, and radiation effects on concrete.
2 Such as disrupting heat transfer from a reactor core, criticality safety, and severe reactor accidents.
3 Computer codes are used to analyze fire conditions in nuclear facilities, to examine how reactor fuel performs, and to assess plant risk.
4 Such as the National Institute of Standards and Technology, the U.S. Army Corps of Engineers, and the U.S. Geological Survey.
1
Audit of NRCs Process for Developing and Coordinating Research Activities Figure 1: NRC Research Funding, FY 2018 Source: NRCs 2018-2019 Information Digest.
NRCs leadership role in international organizations such as the International Atomic Energy Agency and the National Energy Administration helps guide the agencys collaborative research efforts.
NRCs international cooperation in research leverages agency resources, facilitates work on advancing existing technologies, and determines safety implications of new technologies. In addition to collaborating with international organizations, NRC also maintains international cooperative research agreements with more than two dozen foreign governments.
Office of Nuclear Regulatory Research NRC's research program is carried out by the Office of Nuclear Regulatory Research (RES). RES supports the agency mission by providing independent technical analysis and advice, tools, and information for identifying and resolving safety issues, making regulatory decisions, and promulgating regulations and guidance for nuclear power plants and other facilities and materials regulated by the agency. The Energy Reorganization Act of 19745 established RES and its mission to conduct research necessary for regulatory functions.
The current research program primarily supports regulatory activities and initiatives in the Offices of Nuclear Reactor Regulation, New Reactors, Nuclear Material Safety and Safeguards, and Nuclear Security and Incident Response. These offices request RES to perform research and 5 Section 205 of the Energy Reorganization Act of 1974 et seq.
2
Audit of NRCs Process for Developing and Coordinating Research Activities are referred to as requesting offices. Over 75 percent of research activities support the specific needs of these requesting offices. The Commission directs about 10 percent of RES activities through agency-mandated programs and Commission tasking memoranda. A small portion of the research budget focuses on long-term research.
Work Requests Research activities are normally initiated from work requests. There are four types of work requests that vary based on the magnitude of work (duration and costs) and the level of management approval. The four work request types are used by requesting offices to initiate research activities. See Figure 2 for the types of work requests that RES uses to provide technical support.
Figure 2: Work Request Types Informal Assistance Research User Need Request Research Plan Request Assistance Request Requests assistance Requests Requests significant Multiple work to support regulatory confirmatory research or analyses requests to aid functions. analyses or technical to support regulatory regulatory decision-support in specific functions. making and areas. promulgation.
Level of effort Level of effort Level of effort Level of effort requires less than requires 150 - 300 requires 301+ hours requires 301+ hours 150 hours0.00174 days <br />0.0417 hours <br />2.480159e-4 weeks <br />5.7075e-5 months <br /> and no hours or contracts up or contracts over and/or contracts if contractors. to $500,000. $500,000. needed to support the work.
The Branch Chief The Division Director The Office Director The Office Director responds to the responds to the responds to the and requesting office requesting office in requesting office in a requesting office in a concur on the an email within 30 memorandum or signed user need Research Plan.
days. email within 30 days. response memorandum within 30 days.
Source: OIG generated.
3
Audit of NRCs Process for Developing and Coordinating Research Activities II. OBJECTIVE The audit objective was to assess the effectiveness and efficiency of the development, use, and coordination of research activities. Appendix A contains information on the audit scope and methodology.
III. FINDINGS NRCs process for developing, using, and coordinating research activities is adequate; however, opportunities to improve effectiveness and efficiency exist. Specifically, NRC should improve its processes for work requests, tracking research activities, and the use of quality survey information.
A. Opportunities Exist To Improve Efficiency of the Work Request Process Aspects of the research work request process can be more efficient. An organization and its programs should operate in an efficient manner.
However, senior management is not involved and aligned early enough in the work request process. As a result, research activities can become delayed.
What Is Required Programs Should Operate in an Efficient Manner According to Project Aim 2020, NRC needs to improve efficiency to meet future challenges. To improve regulatory efficiency, agency processes must be leaner and use resources more wisely. NRC must make decisions in a more timely and effective manner.
4
Audit of NRCs Process for Developing and Coordinating Research Activities RES Office Instruction PRM-0016 states that after receipt of a work request, RES must respond to the requesting office by memorandum or email within 30 days. PRM-001 also states that RES response should document all agreed upon information (deliverables, priorities, milestones, resource requirements, and schedules).
What We Found Aspects of the Research Work Request Process are Inefficient Aspects of the work request process are less efficient because of RES lengthy effort to develop, approve, and issue its responses to work requests, which may not include all of the required information. Figure 3 illustrates how the work request process takes place.
Figure 3: Work Request and Response Process Staff in Both Requesting Office Staff RES Staff ,
Offices Staff in both offices Prepare collaborate response to develop Identify Contact memoranda work Receive the needs that RES staff to or requests. formal work require RES discuss the emails requests.
assistance. needs. accepting RES staff research may even activities.
draft the requests.
Source: OIG generated.
Although, RES staff are fully aware of requesting offices needs before the official work requests are received by RES, it often takes RES staff and managers more than 30 days to develop and issue its official responses accepting the research activities.
6 RES Office Instruction - PRM-001, Process for Responding to Work Requests: Informal Assistance Requests, Feasibility Study Requests, Research Assistance Requests, User Need Requests, and Research Plans, dated October 12, 2017.
5
Audit of NRCs Process for Developing and Coordinating Research Activities OIG analyzed the 136 work requests that RES received from requesting offices between calendar years 2013 and 2017 to determine the number of days RES management and staff used to respond to the requesting offices. RES received and responded to 3 informal assistance requests, 60 research assistance requests, and 73 user need requests. Response times for 16 of the 136 responses could not be determined because OIG could not identify the actual dates of the initial memoranda or emails. Of the remaining 120 work requests received, RES responses exceeded the 30-day metric about 78 percent of the time. Most of the responses that took longer than 30 days were those involving user need requests, which require Office Director approval. Figure 4 illustrates RES response time to work requests.
Figure 4: RES Response Times - Work Requests 2013-2017 700 Days Between Work Request Submission and RES Response 600 500 400 300 200 100 0
2013 2014 2015 2016 2017 Days Between Request and Response Target Response (30 Days)
Source: OIG analysis.
RES Responses Do Not Follow the Same Format Many of RES responses do not follow a standard structure and some exclude information required by PRM-001. Responses to work requests 6
Audit of NRCs Process for Developing and Coordinating Research Activities are structured in a variety of ways. Some of RES responses do not include required information, such as resources, priority, or schedules.
Why This Occurred Senior Management is Not Sufficiently Involved and Aligned Early Enough in the Work Request Process The RES senior management review and acceptance effort occurs later in the work request process, and can result in delays. The timing and degree of senior managements engagement varies and often they do not participate in early communication and coordination efforts during the development of the work requests. When senior management is not involved early in the process, they may need more clarity on what the requesting offices are asking for and how RES should resource those tasks to ensure the tasks are achievable.
Further, RES does not use a standardized format to respond to requesting office work requests. RES staff often copy and paste from a previous memorandum or email and do not refer to PRM-001 to ensure they include all of the required details. As such, some of the memoranda and emails are lacking the details that senior management need. RES and requesting office staff agreed that a response template would help improve the work request process. RES staff developed multiple templates to streamline the response process, but RES senior management has not yet officially approved the use of a template.
Why This Is Important Research Activities Can Be Delayed Due to inefficient aspects of the work request process, there have been delays in the approval of work requests that subsequently delayed the start of research activities. For example, some work requests require contracts to complete the work. RES cannot move forward on contracts until there is agreement on the requirements and resources between RES and the requesting offices. If RES and requesting office senior management are involved earlier in the work request process, then a 7
Audit of NRCs Process for Developing and Coordinating Research Activities mutual understanding of requesting offices objectives and resource requirements will be gained sooner.
Without a standard response template, RES senior management may take longer to approve a response memorandum or email, because not all of the required information is always included. A standard response template will also help ensure that RES staff include the required details.
Recommendations OIG recommends that the Executive Director for Operations
- 1. Involve RES and requesting office senior managers earlier in the work request development process to ensure work requests are properly understood, resourced, and achievable before they are formally submitted to RES.
- 2. Implement a standard template for RES staff to use when preparing acceptance memorandum or email responses to all work request types.
B. RES Operating Plan Inconsistently Used To Track Work Requests The Operating Plan is not consistently used by RES staff to track work requests. RES staff should use it to track the progress and completion of work. However, the Operating Plan does not have the features to effectively and efficiently fulfill its intended purpose, and management does not require its use. As a result, agency staff are spending time reconciling work request information due to a lack of confidence in the Operating Plan data.
8
Audit of NRCs Process for Developing and Coordinating Research Activities What Is Required RES Staff Should Use the Operating Plan To Track Progress and Completion of Work According to the U.S. Government Accountability Offices Standards for Internal Control in the Federal Government,7 management should use quality information to achieve the entitys objectives. Effective information and communication are vital for an entity to achieve its objectives.
RES primary Operating Plan guidance, PRM-008,8 states that RES staff should use the Operating Plan to track the progress and completion of work to help ensure that stakeholders are appropriately informed.
What We Found The Operating Plan Is Not Consistently Used RES staff should use the Operating Plan to track the progress and completion of work. However, RES staff are not consistently using the Operating Plan because it does not have the features needed to effectively and efficiently fulfill its intended purpose.
Inconsistent Use of Operating Plan Although the Operating Plan provides RES What is the Operating Plan?
with a tool to track progress and completion The Operating Plan is a of work, not all RES staff use it. While some SharePoint Web site tool intended to be used by RES staff RES staff use the Operating Plan as a tool to track and monitor progress on to track research activities, many RES staff work requests, deliverables, and do not use it and favor using their own research activities. Its purpose is to provide internal and external tracking tools. Some RES staff members stakeholders with the means to use Microsoft Excel or Microsoft Word, or track and share information on they rely on monthly invoices and quarterly the work and activities that RES staff are conducting.
meetings with requesting office staff to track work activities. RES management acknowledged that staff are not 7 GAO-14-704G, September 2014.
8 RES Office Instruction - PRM-008, RES Operating Plan, dated March 7, 2011.
9
Audit of NRCs Process for Developing and Coordinating Research Activities consistently using and maintaining the Operating Plan. One senior manager said that the Operating Plan is the principal tool for tracking research activities, but it is not well maintained. Another said that the Operating Plan does not contain current and accurate information.
RES Operating Plan Data is Inaccurate and Incomplete OIG reviewed the 136 work request records in the Operating Plan for calendar years 2013 through 2017. Of the 136 work requests, about 51 percent of the records contained inaccurate or incomplete data. For example:
Numerous records in the Operating Plan lack information on research activities milestones.
Some records in the Operating Plan show work request priorities that differ from those documented in RES responses to requesting offices, yet there is no documented basis for the differences.
Some records in the Operating Plan show research activity status as still active when other documentation shows the research activity has been completed.
Some records in the Operating Plan show work request types that differ from those documented in RES responses to the requesting offices, yet there is no documented basis for the differences.
Figure 5 illustrates the percentage of inaccurate and incomplete RES Operating Plan records.
10
Audit of NRCs Process for Developing and Coordinating Research Activities Figure 5: RES Operating Plan Record Maintenance Contains Accurate and Complete Tracked Accurate and Complete Information Information 51% 49%
Contains Inaccurate and or Incomplete Information Incomplete Information Source: OIG analysis.
RES management is aware of these issues and they are exploring other options for tracking work requests.
Why This Occurred The Operating Plan Does Not Have the Needed Features and Management Does Not Require Its Use The Operating Plan lacks the features to effectively and efficiently assist RES in planning and monitoring progress on program achievement. It is difficult to use, time consuming to maintain, and lacks necessary supporting information.
The Operating Plan is difficult to use and is time consuming to maintain.
In order to sort data, users have to export information from the Operating Plan in RES SharePoint Web site into Microsoft Excel spreadsheets.
RES staff members stated that using the Operating Plan is difficult and that creating tracking tools on their desktops is easier to use for updating research activities. RES staff can enter their milestones directly and update them in the Operating Plan; however, maintaining milestones is time consuming because all changes must go to the Office Director for approval, which takes time.
11
Audit of NRCs Process for Developing and Coordinating Research Activities The Operating Plan lacks detailed information on time and labor reporting.
The level of NRC employee and contractor support varies from project to project. The Operating Plans existing capabilities do not link information on the use of NRC full time equivalents and contract costs. In 2016, the Executive Director for Operations briefed the Commission on tracking and reporting of research activities. The Executive Director for Operations recommended enhancing the Operating Plans capabilities for tracking research activities to improve stakeholders confidence in the data.
RES management acknowledged that they do not enforce requirements for staff to use the Operating Plan. One senior manager stated that tracking work requests is important; however, it is more important for his staff to stay close to the users in the requesting office so they can maintain awareness of what the user needs. Another senior manager said that the Operating Plan is not useful and outlived its intended use.
Why This Is Important The Agency is Using More Resources Reconciling Operating Plan Information with Other Tracking Tools Due to a Lack of Confidence in the Operating Plan Data RES and requesting office staff are using more resources reconciling information for work requests. Some requesting office staff members are unaware of RES Operating Plan, while others are aware of it, but created their own tools to track deliverables because the Operating Plan data is unreliable. Requesting office staff stated that it is hard to align the budget with individual activities or work requests. Further, requesting office staff spend time each month reconciling their data against RES data. Having multiple, non-integrated tracking tools increases errors, causes a duplication of effort and incomplete data, and takes time away from staffs important core duties.
Recommendation OIG recommends that the Executive Director for Operations
- 3. Implement a single agencywide tracking system with the capabilities needed to effectively and efficiently keep the agency aware of research activities.
12
Audit of NRCs Process for Developing and Coordinating Research Activities C. RES Product Quality Survey Is Not Effective RES Product Quality Survey instrument does not provide complete, accurate, and reliable information. NRC management should obtain relevant data from reliable sources in a timely manner. However, there is insufficient guidance for the survey process to ensure reliable data. As a result, the survey tool does not provide the information necessary to effectively assess and improve RES products.
What Is Required Management Should Use Quality Information to Achieve Its Objectives In the Standards for Internal Control in the Federal Government, the Government Accountability Office states quality information should be used to make informed decisions and to evaluate performance. Quality information is described as appropriate, current, complete, accurate, accessible, and timely.
Oracles9 publication, Best Practices for Improving Survey Participation, notes the importance of survey response rates to the usefulness of a survey. Oracle recommends keeping the survey questions relevant to achieve a high survey response rate.
What We Found RES Product Quality Survey Does Not Always Provide RES with Complete or Accurate Information RES online Product Quality Survey allows requesting office staff and managers to evaluate the usefulness of RES products and services.
However, RES Product Quality Survey does not provide quality information because the survey results are not complete and may be inaccurate.
9 Oracle is a provider of business software with a broad portfolio of solutions for companies of all sizes.
13
Audit of NRCs Process for Developing and Coordinating Research Activities Survey Information is Incomplete RES Product Quality Survey captures information only on completed projects; it does not include canceled research activities, interim milestones, and informal feedback.
What is RES Product Quality Survey?
RES guidance PRM-001 states RES RES staff have established an online should obtain feedback after a quality survey for requesting offices product is completed and delivered to to evaluate the usefulness of RES the requesting office. Since the RES products and services. RES staff send this survey to requesting offices survey is sent out only for completed after the work is completed and deliverables, RES does not receive delivered.
feedback via the survey for research activities that are canceled. Regardless of the reason(s) for canceling a project, obtaining feedback on canceled research activities can be an important learning opportunity. Additionally, the survey is not sent out for interim milestones that do not have deliverables. Interim feedback is important because some research activities can take a number of years to complete. The interim feedback could provide RES and requesting office staff and managers the opportunity to course correct or make adjustments if necessary.
In lieu of completing the survey, requesting office staff and managers often provide RES staff and managers with verbal feedback. Throughout the completion of a work request, RES and requesting office staff and managers exchange feedback on the research activities in routine and informal meetings. These important discussions are not documented to further optimize the RES Product Quality Survey results.
Oracle notes the survey response rate10 is directly related to the surveys usefulness. Oracle suggests keeping survey questions relevant to achieve a high response rate. Of the 14 questions in the RES Product Quality Survey, only 5 are related to the quality and timeliness of RES products; including one open-ended question where requesting office staff can provide any additional comments. However, the majority of the questions focus on collecting demographic data instead of questions relevant to RES activities. As such, RES concerns for low response rates and the issue of diminished information collection could be related to the survey structure. In an effort to increase the response rate, in the 10 Response rate is defined as the number of completed surveys divided by the number of invitations to take the survey.
14
Audit of NRCs Process for Developing and Coordinating Research Activities 2017 annual review of the RES Product Quality Survey feedback, RES staff suggested changing the questionnaire but changes have not been implemented yet.
The survey results are used in NRCs Congressional Budget Justification, RES Senior Executive Service performance appraisals, and business line performance plans. In FY 2018, RES discontinued using the RES Product Quality Survey to inform NRCs Congressional Budget Justification because of the low survey response rate. However, for FY 2019, RES decided to again use the survey results for its Congressional Budget Justification because RES believes the Product Quality Survey indicator provides the best quality measure for RES products.
Survey Information May Be Inaccurate RES has a repository of all survey What are access controls?
responses on its SharePoint Web site. At Access controls are a way of any time, these responses can be edited limiting access to a system or to physical or virtual resources. In by any NRC employee. All NRC computing, access control is a employees have the capability to change process by which users are information on the survey responses, granted access and certain privileges to systems, resources which could affect reliability and accuracy or information. In access control of the survey results. SharePoint notifies systems, users must present RES staff when a modification is made to a credentials before they can be granted access.
survey and who made the change, but the system does not identify what changes were made. RES staff would have to save an electronic version of the completed surveys or print them out once they are completed and then compare them to the SharePoint Web site to determine if any of the surveys have been edited. RES staff are considering moving the survey off SharePoint and adding access controls to resolve the issue.
Why This Occurred There is Insufficient Guidance for the Survey Process RES management has not made the RES Product Quality Survey a priority to ensure the survey provides quality information. RES has very limited guidance on the survey process. The only place that provides any 15
Audit of NRCs Process for Developing and Coordinating Research Activities guidance on the Product Quality Survey is PRM-001, the guidance for the overall work request process. PRM-001 simply states that RES should obtain feedback after a product is completed and delivered to the requesting office. The guidance does not include details for obtaining feedback (other than upon completion of a deliverable), how feedback will be used to improve RES research activities, and access controls.
Why This Is Important RES Product Quality Survey is Not Effectively Used to Assess or Improve RES Products The RES Product Quality Survey tool is used to inform important agency documents including the Congressional Budget Justification, RES Senior Executive Service performance appraisals, and business line performance plans. These documents are the results of agency decisions and evaluations that are based, in part, on the survey information gathered via the Product Quality Survey. Because the survey information is not always complete and may be inaccurate, the agency could be providing erroneous information on its performance and operations to stakeholders.
Recommendation OIG recommends that the Executive Director for Operations
- 4. Develop and implement a process for obtaining and using feedback from requesting offices. The process should include, but not be limited to, guidance on obtaining feedback during interim project milestones, creating access controls, and roles and responsibilities.
16
Audit of NRCs Process for Developing and Coordinating Research Activities IV. CONSOLIDATED LIST OF RECOMMENDATIONS OIG recommends that the Executive Director for Operations
- 1. Involve RES and requesting office senior managers earlier in the work request development process to ensure work requests are properly understood, resourced, and achievable before they are formally submitted to RES.
- 2. Implement a standard template for RES staff to use when preparing acceptance memorandum or email responses to all work request types.
- 3. Implement a single agencywide tracking system with the capabilities needed to effectively and efficiently keep the agency aware of research activities.
- 4. Develop and implement a process for obtaining and using feedback from requesting offices. The process should include, but not be limited to, guidance on obtaining feedback during interim project milestones, creating access controls, and roles and responsibilities.
17
Audit of NRCs Process for Developing and Coordinating Research Activities V. AGENCY COMMENTS An exit conference was held with the agency on December 11, 2018.
Prior to this meeting, after reviewing a discussion draft, agency management provided comments that have been incorporated into this report, as appropriate. As a result, agency management stated their general agreement with the findings and recommendations in this report and opted not to provide formal comments for inclusion in this report.
18
Audit of NRCs Process for Developing and Coordinating Research Activities Appendix A OBJECTIVE, SCOPE, AND METHODOLOGY Objective The audit objective was to assess the effectiveness and efficiency of the development, use, and coordination of research activities.
Scope This audit focused on NRCs processes for developing, using, and coordinating research activities through the work request process. We conducted this performance audit at NRC headquarters (Rockville, MD) from April 2018 to November 2018. Internal controls related to the audit objective were reviewed and analyzed.
Methodology OIG reviewed relevant criteria such as the Government Accountability Offices Standards for Internal Control in the Federal Government; the Atomic Energy Act of 1954, as amended; and the Energy Reorganization Act of 1974. OIG reviewed internal NRC guidance documents relevant to the oversight of research activities, including Office Instructions.
OIG interviewed NRC staff and management to gain an understanding of roles, responsibilities, and processes related to work requests. OIG interviewed staff from RES, the Office of Nuclear Reactor Regulation, the Office of New Reactors, the Office of Nuclear Material Safety and Safeguards, and the Office of Nuclear Security and Incident Response.
OIG analyzed the time RES staff took to develop, concur, and respond to work requests submitted between calendar years 2013 and 2017. OIG also reviewed the RES Operating Plan to determine if the information for work requests submitted between calendar years 2013 and 2017 was complete and accurate. OIG obtained work request and response information from RES staff and the Agencywide Documents Access and Management System. We also reviewed the RES Product Quality Survey located on the RES SharePoint Web site.
19
Audit of NRCs Process for Developing and Coordinating Research Activities We conducted this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.
Throughout the audit, auditors considered the possibility of fraud, waste, and abuse in the program.
The audit was conducted by Sherri Miotla, Team Leader; Ziad Buhaissi, Audit Manager; George Gusack, Auditor; Janelle Wiggs, Auditor; and John Thorp, Senior Technical Advisor.
20
Audit of NRCs Process for Developing and Coordinating Research Activities TO REPORT FRAUD, WASTE, OR ABUSE Please
Contact:
Email: Online Form Telephone: 1-800-233-3497 TTY/TDD: 7-1-1, or 1-800-201-7165 Address: U.S. Nuclear Regulatory Commission Office of the Inspector General Hotline Program Mail Stop O5-E13 11555 Rockville Pike Rockville, MD 20852 COMMENTS AND SUGGESTIONS If you wish to provide comments on this report, please email OIG using this link.
In addition, if you have suggestions for future OIG audits, please provide them using this link.
21