ML052420463
| ML052420463 | |
| Person / Time | |
|---|---|
| Issue date: | 08/26/2005 |
| From: | Dingbaum S NRC/OIG/AIGA |
| To: | Funches J, Reyes L NRC/EDO, NRC/OCFO |
| References | |
| OIG-05-A-15 | |
| Download: ML052420463 (6) | |
Text
August 26, 2005 MEMORANDUM TO:
Luis A. Reyes Executive Director for Operations Jesse L. Funches Chief Financial Officer FROM:
Stephen D. Dingbaum/RA/
Assistant Inspector General for Audits
SUBJECT:
MEMORANDUM REPORT: REVIEW OF NRCS APPLICATION OF THE OFFICE OF MANAGEMENT AND BUDGETS (OMB) PROGRAM ASSESSMENT RATING TOOL (PART) (OIG-05-A-15)
As part of the Office of the Inspector Generals (OIG) audit of NRCs Decommissioning Program, OIG examined aspects of NRCs approach to applying the PART. NRCs FY 2006 performance plan identified Decommissioning and Low-Level Waste as a program scheduled to be reviewed using PART for Budget Year 2008. Because this program will be subject to a PART review, agency managers requested OIG to review the decommissioning program. In reviewing the decommissioning program, OIG agreed to use the PART guidance as the basis for the review. OIG reviewed PART submittals prepared by NRC programs, compared portions of these submittals, and interviewed NRC staff involved in completing PART reviews.
BACKGROUND Under the Government Performance and Results Act of 1993 (GPRA), Federal agencies are required to schedule, conduct, and report on program evaluations in selected areas. The Office of Management and Budget (OMB) developed the Program Assessment Rating Tool (PART) to assess and improve program performance so that the Federal government can achieve better results and to help inform funding and management decisions. PART builds on GPRA by encouraging agencies to integrate operational decisions with strategic and performance planning. To decide which programs will be reviewed using the PART for the coming year, agencies and OMB work together.
2 PART is a diagnostic tool intended to systematically and consistently assess the performance of program activities across the Federal government. It is comprised of a series of questions, divided into four sections. Each section and its associated objective are summarized below.
Program Purpose & Design - Assesses whether the program's purpose and design are clear and sound.
Strategic Planning - Assesses whether the program has valid long-term and annual measures and targets.
Program Management - Rates agency management, including financial oversight and program improvement efforts.
Program Results/Accountability - Rates program performance on measures and targets reviewed in the strategic planning section and through other evaluations.
Five NRC programs have been reviewed using PART and scored by OMB. Four programs received an OMB rating of Effective. One received a rating of Moderately Effective. To receive a rating of Effective, a program must receive an OMB score greater than 85. According to briefing slides on the PART process prepared by OCFO staff, 15% or fewer federal government programs reviewed using PART have scored greater than 85 from FY 2002 through FY 2004. The table below lists the NRC programs reviewed using PART, the year in which NRC conducted the review, and the rating awarded by OMB.
Table 1 Program Year PART Review Completed OMB Rating Reactor Oversight Process FY 2003 Effective Fuel Facilities Licensing and Inspection FY 2003 Effective Nuclear Materials Users Licensing and Inspection FY 2004 Effective Spent Fuel and Transportation FY 2005 Effective Reactor Licensing Program FY 2005 Moderately Effective RESULTS While NRCs programs have historically scored well on PART reviews, OIG observed inconsistencies in the Agencys approach. Specifically, NRC does not have agency-wide guidance for applying the PART to its programs to assure consistency in PART submittals. OCFO staff believes OMBs PART guidance is
3 adequate and, while the roles and responsibilities for conducting PART reviews at NRC needs to be documented, additional detailed PART guidance specific to NRC is not necessary. OCFO staff said that it would be counter productive to develop detailed PART guidance specific to NRC when PART guidance changes from one year to another. Rather, OCFO works closely with program staff scheduled for upcoming PART reviews to share lessons learned.
NRCs Management Directive (MD) 4.7, Long-Range Planning, Budget Formulation and Resource Management, last updated October 1989, is the logical place for NRC to establish agency-specific PART guidance. While NRC first implemented PART in FY 2003, they have not updated this policy to define roles and responsibilities for PART. Staff told OIG that NRC is currently working to revise this management directive to define the roles and responsibilities for the Agencys Planning, Budget and Performance Management process. The revised management directive will define, at a high level, the roles and responsibilities for conducting PART reviews at NRC.
Without agency-wide guidance for conducting PART reviews, some NRC program staff interpreted and applied OMB guidance differently and no standard practice exists for gaining management concurrence to assure consistency and quality of PART submittals. Furthermore, --
3/4 information contained in NRC PART submittals may be viewed as inconsistent by an OMB evaluator (i.e., interpretation of the term partner as discussed below);
3/4 program staff may re-create information for some PART questions that could have the same or similar answer from program to program (e.g.,
as discussed below, PART question 4.4 states, Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?); and 3/4 program status may not be portrayed accurately (i.e., an NRC answer of to a large extent, when the agencys response does not support large extent).
For instance, decommissioning program staff interpreted the term partner used in one PART question differently from another NMSS program. PART question 2.5 states, Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other governmental partners) commit to and work toward the annual and/or long-term goals of the program? The decommissioning program staff does not consider Agreement States as a partner, but another NMSS program does. While Agreement States may not qualify as a partner for all NRC programs, an agency-wide approach to this question may be beneficial to minimize potential conclusions by OMB evaluators that NRCs answers are inconsistent.
4 Also, other NRC programs applying PART could use the same or similar answers to some questions. For example, PART question 4.4 states, Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals? In the FY 2003 Program Evaluation, the decommissioning program answered, to a large extent, with the following explanation No other Federal programs are similar enough to compare. Although DOE [Department of Energy] and EPA [Environmental Protection Agency]
are responsible for similar radiological remediation activities, both have different statutory and regulatory frameworks and different sites with site-specific challenges. Thus, comparisons have not been done and would not be meaningful.
However, another NMSS program answered this question, to a small extent.
That programs explanation noted that EPA with its oversight of the chemical industry has similar purposes and goals. It stated Although we have not benchmarked our performance with respect to the chemical industry, and the associated chemical and safety hazards, NRCs safety record with respect to radiation hazards as evidenced by our strategic goal measure results, compare favorably to other programs NRCs decommissioning program could make a similar argument. Furthermore, the argument regarding dissimilarity between NRCs decommissioning program and EPA due to different regulatory framework could also apply to the other NMSS program and EPA.
Furthermore, it is not clear why one program that states,... comparisons have not been done and would not be meaningful, assigns an answer of large extent, while another program assigns small extent when the program states that Although we have not benchmarked our performance... NRCs safety record... compare[s] favorably to other programs CONCLUSION Without agency-specific guidance to assure consistency in applying the PART and to provide a common interpretation of OMB guidance, the value of PART to NRC as a tool for making continuous program improvements and informing management and budget decisions could be diminished.
5 SCOPE/CONTRIBUTORS We evaluated portions of three NRC PART submittals, a self-assessment of NRCs decommissioning program that partially applied the PART, discussed NRCs approach to applying PART with staff experienced in its application and with OCFO staff. Additionally, we reviewed agency documents, including NRC Management Directive, 4.7, NRC Long-Range Planning, Budget Formulation and Resource Management, and briefing materials used by OCFO staff to kick off PART reviews. The review was conducted by Deb Lipkey, Sr. Management Analyst and Andrew Blanco, Engineer.
If you have any questions or concerns regarding this report, please contact Anthony Lipuma, Team Leader, at 415-5910 or me at 415-5915.
cc:
Chairman Diaz Commissioner Merrifield Commissioner Jaczko Commissioner Lyons
6 Distribution John T. Larkins, Executive Director, Advisory Committee on Reactor Safeguards/Advisory Committee on Nuclear Waste G. Paul Bollwerk, III, Chief Administrative Judge, Atomic Safety and Licensing Board Panel Karen D. Cyr, General Counsel John F. Cordes, Jr., Director, Office of Commission Appellate Adjudication Jesse L. Funches, Chief Financial Officer Janice Dunn Lee, Director, Office of International Programs William N. Outlaw, Director of Communications William N. Outlaw, Acting Director, Office of Congressional Affairs Eliot B. Brenner, Director, Office of Public Affairs Annette Vietti-Cook, Secretary of the Commission William F. Kane, Deputy Executive Director for Reactor and Preparedness Programs, OEDO Martin J. Virgilio, Deputy Executive Director for Materials, Research, State and Compliance Programs, OEDO Jacqueline E. Silber, Deputy Executive Director for Information Services and Administration, and Chief Information Officer, OEDO William M. Dean, Assistant for Operations, OEDO Timothy F. Hagan, Director, Office of Administration Michael R. Johnson, Director, Office of Enforcement Guy P. Caputo, Director, Office of Investigations Edward T. Baker, Director, Office of Information Services James F. McDermott, Acting Director, Office of Human Resources Corenthis B. Kelley, Director, Office of Small Business and Civil Rights Jack R. Strosnider, Director, Office of Nuclear Material Safety and Safeguards James E. Dyer, Director, Office of Nuclear Reactor Regulation Carl J. Paperiello, Director, Office of Nuclear Regulatory Research Paul H. Lohaus, Director, Office of State and Tribal Programs Roy P. Zimmerman, Director, Office of Nuclear Security and Incident Response Samuel J. Collins, Regional Administrator, Region I William D. Travers, Regional Administrator, Region II James L. Caldwell, Regional Administrator, Region III Bruce S. Mallett, Regional Administrator, Region IV