ML20134D046

Audit Rept OIG/96A-11, Improvements Needed in Agency Oversight of Info Resources Mgt Activities

ML20134D046
Person / Time
Issue date:	09/24/1996
From:	NRC OFFICE OF THE INSPECTOR GENERAL (OIG)
To:
Shared Package
ML20134C936	List: ML20134C934 ML20134D046
References
OIG-96A-11, NUDOCS 9610110355
Download: ML20134D046 (40)

Text

<

I OFFICE OF THE INSPECTOR GENERAL

[

U.S. NUCLEAR I REGULATORY COMMISSION IhiPROVEhiENTS NEEDED IN AGENCY OVERSIGHT OF INFORhiATION RESOURCES

[ hiANAGEhiENT ACTIVITIES OlG/96A-11 September 24,1996

[

[

AUDIT REPORT ABREG9 h A> '

[

e e so 2

[ <  ! o m e t/ g

[ o Wf3 g% kh o#g g

[

aw I

,%o sta 2aR8" 28784 110053 Ig,;L j

1

1. , f)m,,u .,::

Agency Oversight of Information Resources Management Activities REPORT SYNOPSIS l

The Of6ce of Information Resources Management (IRM) budgeted about $32 million in contract support for Fiscal Year 1996 to provide computer hardware, I software, communications equipment and services for the U.S. Nuclear Regulatory Commission (NRC). During survey work, the Of6ce of the Inspector General (OlG) identiDed key NRC projects that were behind schedule and over budget.

OlG initiated an in-depth review to determine the causes for these conditions and  !

to assess NRC's oversight of information resources management projects. )

NRC information technology (IT) activities represent signi6 cant agency investments aniounting to tens of millions of dollars annually. Therefore, NRC needs sound management control processes to ensure the effective and efficient use l of the dollars invested in these projects. Our review disclosed that NRC lacks the l management controls to systematically provide NRC management with information l needed to assess the status of IRM projects. We also found that (1) the agency does not manage significant systems development activities from an agency-wide approach, and (2) ineffective comrnunication between NRC of6ces and user (I requirements added during development delayed selected systems and drove costs l upward.

In addition, the Information Technology Management Reform Act of 1996 (ITMRA) became effective on August 8,1996. The ITMRA requires agencies to designate a Chief Information Of6cer (CIO) to provide greater coordination and accountability for the agency's information resources management activities. OIG believes an effective CIO could have precluded some of the problems NRC experienced with its key systems development projects that were behind schedule and over budget. NRC is currently conducting a nation-wide recruiting effort to 611 this key position.

The ITMRA also requires agencies to develop a Capital Planning and Investment Control (CPIC) process for managing IT investments and performance measures to compare Federal prcject management to the best in the private sector.

. However, OIG found that NRC's draft guidance to implement the ITMRA would effectively exempt every NRC system from the CPIC and performance measure requirements by determining that systems do not cost enough to merit these l control processes. Further, the agency has not identiGed and reported to the l OlG/96A.Il Page1 I

Agency Oversight of information Resources Management Activities Office of hianagement and Budget (OhiB) any " major" information systems, although OIG considers that some agency systems meet OhiB's criteria.

1 Finally, the IThiRA described the " Sense of Congress" that Federal agencies should annually achieve five percent cost savings in operating and maintaining IT and five percent increases in the efficiency of agency operations through improvements in information resources management over the next five years. OIG believes that successful implementation of the CIO, CPIC, and performance measure requirements is imperative for NRC to meet these goals.

We make six recommendations to improve NRC's management of information resources management projects.

I I

O!GS6A 11 p g, gg

Agency Oversight ofInformation Resources Management Activities TABLE OF CONTENTS REPORT SYNOPSIS i INTRODUCTION 1 BACKGROUND I FINDINGS 3 NRC LACKS CONTROL PROCESSES AND hiANAGEMENT SYSTEMS FOR OVERSEEING IRM PROJECTS 3 NRC NEEDS AN AGENCY-WIDE APPROACil FOR PLANNING AND OVERSEEING KEY SYSTEMS DEVELOPMENT PROJECTS 5 RECENT LEGISLATION AND FEDERAL POLICY REVISIONS WILL REQUIRE NEEDED CilANGES AT NRC 8 CONCLUSIONS 13 RECOMMENDATIONS 14 AGENCY COMMENTS 14 I APPENDICES I Objectives, Scope, and Methodology I II Review of Three Key System Development Efforts Illustrate Need to Improve Agency-Wide Management III Agency Comments on Draft Report IV U.S. NRC Functional Organization Chart V Major Contributors To This Report VI Glossary: Office of the Inspector General Products I

--war

Agency Oversight of Information Resources Management Activities INTRODUCTION This report provides the results of the U.S. Nuclear Regulatory Commission (NRC) OfGce of the Inspector General's (OIG) review of NRC's oversight of information resources management activities. During initial survey work of NRC's Of6ce of Information Resources Management (IRM) program, OlG identified key projects that were over budget and behind schedule. OIG l subsequently initiated an in-depth review to determine the causes for these conditions and to assess NRC's oversight of information resources management projects.

NRC is at an important juncture in defining its strategy for managing information resources and how the agency uses information technology (IT). Technological I advances are reshaping how the Federal government, and NRC's licensees, will perform their activities. Recent legislation and policy changes require Federal l agencies to reform how they use IT hardware and software to process information.

In addition, effective use ofIT will likely be important to successfully implement any changes in the agency's mission and operations following the current Strategic I Assessment and Rebaselining effort. This report makes several recommendations to address improvements that are needed in NRC's oversight of information resources management activities. Appendix I provides additional details on the I objectives, scope and methodology of this audit.

BACKGROUND l NRC management, technical, and administrative staff depend on automated 5 information systems to support the agency mission and its various programs and activities. IRM is responsible for the direction and management of the agency's centralized information resources, including computers, information services, and I telecommunications.

l NRC relies heavily on contractors to help it support the agency information resources management activities. NRC's total obligations for IT in Fiscal Year (FY) 1996 is about $514 million for non-personnel costs. Of this amount, the I IRM FY 1996 Financial Plan as of April 30,1996 budgeted at:out $37.6 million for contract support. About $31.8 million ofIRM's total, or 85 percent, will be spent by three of IRM's eight branches and stafT ofTices. Managing over 30 OlG.96A Il Pete I

i I

l Agency Oversight of Information Resources Management Activities ll contracts for IRM, these three branches are the: (1) Technology Infrastructure Branch (TIB), (2) Systems Development and Integration Branch (SDIB), and (3)

End-User Support Services Branch (EUSB). TIB is responsible for developing and implementing agency programs for the engineering, acquisition, operation and maintenance of the agency's IT infrastructure, composed of voice and data networks, minicomputers, and other resources. SDIB manages the development and implementation of agency-wide and selected of6ce specine information systems. Finally, EUSB coordinates the determination of standard con 6gurations, timing of updates, and replacements of computers. Put simply, EUSB manages I the equipment on staff desktops, SDIB prepares programs and systems to manage I infomation, and TIB manages much of the remaining IT that NRC staff uses.

. IRM uses several types of contracts that vary in size, nature, and complexity. In some cases, a single IRM contractor may carry out multiple projects under a single task order, while other contractors perform a single project. For example, about 90 percent of all SDIB system development projects are carried out by a single contractor.

While this report focuses on NRC's management of IRM projects, OIG has previously reported on NRC's management ofIRM contracts. In a March 1993 report' OlG found that IRM had not exercised adequate control over its contracts g and the activities of project officers. We reported that IRM had not established 3 office-wide policies and procedures needed to ensure that functions are performed consistently and in accordance with applicable regulations. As a result, IRM exceeded specific procurement limits and made unauthorized commitments. Also, 2

we recently conducted a special evaluation that identined lessons the agency can learn to effectively develop systems and oversee contracts in the future.

Speci6cally, we observed that control processes did not work as intended and communication breakdowns occurred between and internally among key NRC of6ces.

' Review ofIRM's Manacement ofits Contracts. OIG/92A-10, March 8,1993.

2 Selectine. Manacine. and Utilizine the M-Cubed Contract, OIG/96E-13, April 17,1996 OICt96A.ll Page 2

Agency Oversight ofInforrnation Resources Management Activities FINDINGS NRC needs to have sound management control processes to ensure the effective and efficient use of the tens of millions of dollars invested in information resources management projects. Our review disclosed that NRC lacks control processes to systematically provide NRC management with information needed to assess the status of IRM projects. We also found that the agency has incurred unnecessary project delays, miscommunicated system requirements, and exceeded planned budgets.

We also found that recent legislation and Federal policy revisions requires greater management accountability for the expenditure of funds for IT resources. For example, the Information Technology Management Reform Act of 1996 (ITMRA) requires NRC to appoint a Chief Information Officer (CIO) to improve the f management of the agency's information resources management activities and develop a capital planning process to improve the acquisition and management of IT investments. Further discussion of these findings follows.

NRC LACKS CONTROL PROCESSES AND MANAGEMENT SYSTEMS

[ FOR OVERSEEING IRM PROJECTS NRC's IT activities represent significant agency investments that are needed to support crucial agency safety-related and administrative programs. Many projects cost over $500,000 to develop and may require millions of dollars to maintain.

IRM's management of project development focuses on achieving technical

[

performance goals. Successfully achieving technical performance is important, however the need to manage the costs and schedules of IRM projects is also

( necessary and highly important. To evaluate NRC's project management practices, OlG audited the milestone and cost status of IRM projects described in the FY 1996 Operating Plans for SDIB, TIB, and EUSB. We reviewed available information on 44 of the 97 projects listed in these plans.

However, OIG found that NRC lacks the financial and technical management

{

control processes needed to effectively oversee information resources management projects. We asked IRM to provide OIG documentation on whether they were on

[ budget and schedule for the 44 projects in our sample. Although IRM provided considerable information concerning the projects, we generally could not OIG.96A-Il Page 3 W

I  :

Agency Oversight ofInformation Resources Management Activities ll determine whether the projects were on budget and schedule. The financial and technical status information was generally not available because IRM does not consistently prepare and update spending and operating plans for all projects. We found that no NRC office, including IRM, systematically compares the financial and technical progress of information resources management projects to project plans to determine whether they are on budget and schedule.

l Additional factors made it difficult for OIG to determine the projects' financial g status. First, it is IRM's practice to manage dollars at the contract level, therefore W financial status information is not generally maintained at the project level. l Second, IRM projects are not managed in accordance with two NRC policies I designed to improve financial discipline. The IRM Contract Financial Status Record Worksheet (Worksheet) was issued in September 1991 to IRM project g officers in response to weaknesses detected in an internal IRM review. The l Worksheet is to be completed monthly with data on contract balance, costs paid, and projected future costs. However, we found that this management tool was not routinely prepared by all project officers in the three branches we reviswed.

Moreover, since this is a " contract" Worksheet, the projected costs for individual projects were unavailable for 24 of 44 projects we reviewed because they involved multiple projects suppon d by single contracts.

In addition, NRC contracting procedures require that a Contractor Spending Plan be prepared and updated monthly for cost reimbursement contracts or individual task orders where the award amount is expected to exceed $100,000 and the period of performance is expected to exceed 6 months. However, even though some of the projects we reviewed met this criteria, IRM could not supply spending plans for them. l We also found that determining the technical progress for ongoing projects against planned milestones is generally not possible because of weaknesses in control El 3

processes. IRM branches develop Operating Plans to inform IRM and other NRC l senior management of projects, which should be revised quarterly. However, we found that these Operating Plans are not consistently updated and provided to NRC management.

IRM management told us they manage the technical progress of high priority projects through monthly status and weekly accomplishment reports. However, Il1' our review of the monthly reports found that the status of only about 25 percent OlG/96A.Il Page 4 I

Agency Oversight ofInformation Resources Management Activities of IRM's higher priority projects are reported and the updated status information is generally not compared to project milestones. Further, the weekly staff report is organized by person and not project and is therefore limited in its usefulness for monitoring technical project status. OlG believes maintaining and evaluating regularly updated financial and technical project information is an important management tool that is needed to effectively manage the agency's limited resources.

NRC NEEDS AN AGENCY-WIDE APPROACil FOR PLANNING AND OVERSEEING KEY SYSTEMS DEVELOPMENT PROJECTS Absent sound management controls over information resources management projects, the agency is vulnerable to problems adhering to budgets and schedules, particularly in systems development projects involving multiple NRC offices. We reviewed two agency systems development projects, the Agency Training System (ATS) and the Resource Information Management System (RIMS), to determine reasons why these projects were significantly over budget and behind schedule.

OlG is also reviewing the Payroll / Personnel system (PAY /PERS) in a separate audit. We found that NRC did not take an agency-wide approach to managing these important agency information system development projects. As a result, ineffective communication between NRC offices about system requirements delayed the development of the ATS and may delay PAY /PERS. NRC failed to identify the needs of all potential ATS users until February 1996, so the system schedule has slipped 9 months to December 1996. In the meantime, NRC has over spent its budget by $255,000, or about 57 percent, for an incomplete system.

We also found that significant additional user requirements requested by the sponsoring office after the RIMS project started development caused its schedule to slip several times and drive project costs to over $700,000 more than estimated.

OlG believes NRC needs to improve on the timely identification of information >

and requirements from all potential agency users of an information system during its development. Further details on our review of the three systems development projects can be found in Appendix II.

OIG found that an additional factor contributing to increased costs and delays in NRC's information resources management projects is the general agency perspective that developing a working system is crucial and takes priority over OlG/96A.it Page s i . -

1

Agency Oversight o8;nformation Resources Management Activities Millionis of Dollars

$ 14.9 515 - ,... ,

$ 11.6

?.7

$ 6,5 ,

III 86 9 1994 1994 1995 1995 1996 1996 original Final original Final Original Current Fiscal Year 9 original Dudget IRM Reprogramrning E Funds frorn oC Figure 1: Systems Development Funding increases cost considerations. One senior IRM manager told OIG that building a working g system is IRM's customer service goal, even if it takes a few extra months and 3, more funds than estimated. However, we found that IRM received SI1,827,000 in increased funding from the Office of the Controller (OC) from FY 1994

)

through 1996' to support additional systems development work that had not been  !

originally budgeted for those years, significantly increasing SDIB's budget in FY l 1995 and 1996. Figure 1 illustrates the additional funds provided in each year.

We also learned that NRC program omces having systems built for their programs believe they have little responsibility to monitor project costs. They feel IRM is E

responsible for monitoring costs incurred by IRM contractors in developing g systems. OlG believes that because NRC has not prepared spending plans for many of its information resources management projects, it is not only dimcult to determine the significance of the cost increases but it is equally dimcult to hold either IRM or program omces accountable.

I I I

' Current total fcc FY 1996 includes information up to May 23,1996.

OIG/96A.11 Page 6 j

Agency Oversight of Information Resources Management Activities Also contributing to the late delivery and increased costs of agency system development projects is that NRC has not developed policies and procedures to facilitate an agency-wide approach to management of IRM projects. The 1993 Charter for NRC's IT Council requires that systems with an investment cost over

$500,000 that may have agency-wide use be formally reviewed at selected points in their life cycle. The outcome of this review would be recommendations to:

continue the project, make necessary changes, or terminate the project. However, NRC has not developed guidelines for the IT Council to conduct these life cycle reviews and no systems have ever been reviewed, including RIMS and ATS. In addition, NRC committed to develop a complete set ofIRM office-wide policies and procedures to ensure consistent contract management practices by July 30, 1993, in response to OlG's Review of IRM's Manacement of its Contracts audit report. However, the agency has not completed work on these policies and to date one of the six planned NRC Management Directives documenting IRM policies has been issued. Also, since recent legislation and Federal policy changes will impact on these policies, IRM does not plan to complete tbm until late 1997. We believe NRC needs to actively work towards updating these information resources management policies as soon as possible.

In addition, as we reported in our special evaluation on the M-Cubed Information Systems (M-Cubed) contract, the agency has not always used full and open competition to award contracts for systems development work. The agency used the Small Business Administration's 8(a) contracting program to quickly select M-Cubed to develop systems using new client-server technology. However, this contractor was unable to efficiently meet NRC's needs when it was awarded additional work for critical agency activities, and the projects M-Cubed worked were generally over budget and exceeded planned time frames. Although M-Cubed was awarded about $3,000,000 for work primarily on 7 projects, NRC now estimates an additional $2,725,000 will be needed to complete ongoing and new work.

We note that NRC has taken some recent steps to improve IT project management.

For example, during the course of our audit NRC awarded two contracts that will consolidate project management for SDIB and TIB, and are intended to improve the financial and technical management of projects carried out by these branches.

In May 1996 the Chairman approved the contractor selected for NRC's riew Comprehensive Information Systems Suppon Consolidation (CISSCO) strategy.

The contract will supply about 8 to 13 million dollars annually for systems OIG/96A.Il Page 7 I

1 1

Agency Oversight ofinformation Resources Management Activities I development work, up to a maximum of 30 million dollars per year. CISSCO's objectives include: consolidating multiple systems development contracts used by SDIB and other NRC of6ces, using a consistent set of standards and )

methodologies to integrate system life cycle functions, providing a single point of accountability for all systems integration, and implementing IRM's strategic goal of providing agency-wide systems development and integration. Due to the size and importance of the CISSCO strategy, NRC has developed a set of management controls that the agency believes will reduce and control the risks related to the g program. One of these controls is using an independent verification and validation E (IV&V) contractor to assist the IRM program manager in reviewing and auditing the prime contractor. Because of the NRC's problems with the M-Cubed contract and significance of the CISSCO program, OIG received a commitment from NRC for (1) access to all CISSCO contractual documents and (2) periodic information from IRM and the IV&V contractor on the program's status.

NRC also awarded another contract in April 1996 for the Next Generation Network (NGN) Program to support the design, expansion, development, procurement, implementation, maintenance, and operation of NRC's agency-wide office automation and network environment through FY 2000. TIB will manage this 5 year contract worth $29 million dollars.

RECENT LEGISLATION AND FEDERAL POLICY REVISIONS WILL REQUIRE NEEDED CllANGES AT NRC Legislation and Federal policy actions in the last two years have created the environment for Federal agencies, including NRC, to improve the management of information systems and technology. For example, the Office of Management and Budget (OMB) revised Circular A-130 in July 1994 to improve Federal l

management ofIT investments. Further, Congress passed the ITMRA in January E l 1996 to " create incentives for the Federal government to strategically use IT in E order to achieve efficient and effective operations.. ", and to transform the i acquisition of IT from a process-oriented procurement system into a results- l oriented system. Congress declared the ITMRA is needed because most Federal agencies cannot track the expenditures of Federal dollars and thus expose the taxpayers to billions of dollars in waste, fraud, abuse, and mismanagement.

Congress also stated that poor planning and program management, and an I

,l

Agency Oversight of information Resources Management Activities overburdened acquisition process, have resulted in the American taxpayers not getting their money's worth from funds spent on information systems.

The IThiRA policies were developed partially in response to GAO testimony in May 1995' that Federal "information system projects are frequently developed late, fail to work as planned, and cost millions more than expected. In an environment of shrinking resources and a demand for service improvement, the government can ill afford to continue spending such large amounts of money with so few results." GAO also noted that " Federal agencies often buy computer hardware before they evaluate their business functions, lack discipline and accountability for their investments, and fail to rigorously monitor the results produced."

The ITMRA became effective on August 8,1996, and will be a catalyst for changing NRC's information resources management program. In particular, we believe the ITMRA requirements for an agency Chief Information OfGcer (CIO) and the establishment of a Capital Planning and Investment Control (CPIC) process present the opponunity for NRC to strengthen agency IT management.

Successfully implementing these requirements will assist NRC in meeting the

" Sense of Congress" stated in the ITMRA that over the next Sve years federal agencies should annually achieve: (1) savings of Sve percent in operating and maintaining IT and (2) increases of Sve percent in the efficiency of agency operations through improvements in information resources management.

NRC Needs an Effective CIO to Manage the Agency's IRM Program The ITMRA creates CIOs to increase the responsibility, authority, and accountability of agency of6cials in the use ofIT and other information resources in support of agency missions. Some of the CIO's responsibilities under the ITMRA are: (1) providing advice and assistance to the head of the agency to ensure that IT is acquired and information resources are managed in accordance with the ITMRA, (2) promoting the effective and efScient design and operation of all major information resources management processes, and (3) annually assessing the information resources management knowledge and skills of agency

' MANAGING FOR RESULTS: Stens for Strenethenine Federal Manacement.

GAO/T-GGD/AIMD-95-158, May 9,1995.

OlG/96 A-11 Page 9 i

l

Agency Oversight of information Resources Management Actwities executives and managers, and develop strategies and specific plans to rectify any denciencies found.

OlG believes an effective CIO could have precluded or reduced some of the problems NRC experienced with the ATS and PAY /PERS projects that arose from ineffective interactions between IRh1 and other agency of6ces. Ineffective of6ce communication was also identiDed in our recent special evaluation report as a pivotal problem in the agency's management of the M-Cubed contract. In addition, NRC's IT Council Chairman noted at the May 1996 meeting that NRC ,

"needs more of a team approach between Of6ces and IRM." Further, he stated that the agency needs to establish requirements and training for managers to acquire IT knowledge and skills. A senior agency manager at the IT Council l

meeting added that motivating staff to learn about and take advantage of technological tools available at NRC is a large problem. OlG is encouraged that the agency plans on having the CIO report directly to the NRC Chairman, and further believes that this person should possess practical experience in IT management practices, as discussed in the Congressional Statement that I

accompanied the ITMRA. NRC is currently conducting a nation-wide recruiting effort to Gil this key position.

l NRC Needs to Actively implement the CPIC and Performance Measure Requirements The ITMRA was passed to ensure that Federal government agencies are '

responsible and accountable for achieving service delivery levels and project management performance comparable to the best in the private sector. The ITMRA requires tha.1 each agency design and implement a CPIC process for maximizing the '<alue and managing the risks of the agency's IT acquisitions, including the use of quantitative criteria for assessing a project's net return on E

investment and comparing alternatives. The CPIC process is to provide senior 3 management timely information regarding a system investment's progress in costs, timeliness, and system capabilities milestones, on an independently verifiable basis.

Also, the ITMRA requires agencies to quantitatively benchmark the cost, speed, and productivity performance of agency processes against comparable processes in public or private sector organizations. Our audit Ondings discussed earlier in this report indicate that NRC currently does not have processes that meet these ITMRA requirements.

DIC/96A.it Page 10

(

Agency Oversight of information Resources Management Activities in July 1994, OhiB revised Circular A-130 to require that agencies perform cost-benefit analyses to support ongoing management oversight processes that maximize return on investment and minimize Gnancial and operational risk for " major information systems" on an agency-wide basis. A major information system is one: "that requires special management attention because of its importance to an agency mission; its high development, operating, or maintenance costs; or its

[ signi6 cant role in the administration of agency programs, Gnances, property, or other resources." In addition, the Federal Acquisition Streamlining Act of 1994 (FASA) de6nes a major system for a civilian agency as one where total

[ expenditures are estimated to exceed $750,000 in FY 1980 dollars, which is equal to about $1,440,000 in today's dollars according to OMB, or a dollar threshold the agency establishes.

Moreover, OMB prepared a guidance document for Federal agencies in November, 1995 entitled " Evaluating Information Technology Investments", also known as the IT Guide, based on strategic information management practices in successful organizations. The IT Guide's objective is to provide information on (1)' what

[ OMB expects from agencies and (2) how agencies can reduce the risk and maximize the net benents from their IT investments. One IT Guide recommendation is that each agency should denne dollar thresholds to direct

{

decision-making to the appropriate agency level, using a consistent set of investment decision practices throughout the agency. Also, the IT Guide noted

[ that some "best practice" organizations submit projects to thorough investment reviews when costs exceed between 0.5 and 2 percent of the organization's IT budget. NRC's FY 1996 IRM budget for contract support was about $37.6

( million, so systems costing from $190,000 to $750,000 (using the IT Guide percentages) and over would be a candidate for a thorough review if NRC adopted the best practices of other organizations.

{

OIG found that although OMB has issued several guidance documents to Federal

{ agencies, NRC has not identiGed any major information systems. OMB issued Bulletin Number 95-06 to Federal agencies in September 1995 as its annual request that each agency provide a list of existing and planned major information systems and planned IT acquisitions, among other items. However, NRC's response to this bulletin on December 18,1995 did not identify any major systems or planned acquisitions.

( OlG/96A.11 Page 11 r

~

ad

I Agency Oversight of Information Resources Management Activities Funher, draft NRC guidance to implement the ITMRA would effectively exempt every NRC information system from the cmcial CPIC and performance measure requirements. In April 1996 an NRC working group composed of about 18 senior IRM managers and other agency office representatives developed criteria for

" major", "significant", and "other" information systems and acquisitions; and the documentation NRC would require to be prepared for each category. The primary  ;

criteria the group used to separate the categories is a project's cost, identifying that a major system costs over $100,000,000, and significant systems between

$500,000 and $100,000,000. The group believed that a major system is one comparable to some of the largest systems in the Federal government, like the Internal Revenue Service's Tax Systems Modernization and the Federal Aviation Administration's Advanced Automation System efforts. The group determined that none of NRC's "relatively small" systems meet the criteria for a major system. ,

in addition, the group almost unanimously determined that the ITMRA's capital  ;

planning and performance measure documentation requirements should apply only to major systems and not to significant ones. However, at the May 1996 IT Council meeting, a senior IRM manager stated that NRC may need to apply performance measures to some significant systems. l OlG believes NRC has systems that meet the defmition and spirit of a major I

information system under the criteria in OMB Circular A-130 and the FASA, and g that the agency should identify them to OMB. Moreover, OIG believes that NRC 5 needs to adopt the ITMRA's CPIC and performance measure requirements for many of its systems, rather than for none or few. OMB's definition of a major system discusses the importance of the system to the agency's mission, program, fmances, etc., and does not mention system costs. Further, we believe NRC should not determine its major systems based on a comparison of NRC projects to the cost of other Federal agency acquisitions. OMB's recent IT Guide indicates that a threshold for thorough information system investment reviews should be g based on a percentage of an agency's IT budget, not on a general govemment- E wide dollar threshold. In addition, absent a greater agency threshold, the FASA indicates that a system costing more than $1,440,000 is a major system.

Finally, the IT guide also suggests that agencies define their IT investments as a portfolio including projects in every phase of the system life cycle. Since the typical NRC information system is used for an average of ten years, OIG believes l

NRC should assess the value of applying the CPIC and performance measure requirements to systems already in operation as well as those systems that begin l oic m -in r, i2 I

Agency Oversight of Information Resources Management Activities development after the ITMRA becomes effective on August 8,1996. The Director ofIRM noted his suppon for using a CPIC process and performance measures for managing agency efforts like the RIMS and ATS projects when OIG briefed senior NRC executives on our audit findings.

CONCLUSIONS NRC invests tens of millions of dollars annually in information resources management activities. However, our findings indicate that the agency has signincant weaknesses in the oversight of information resources management projects. NRC lacks the management control processes to systematically provide agency management with needed cost and performance data on the status ofIRM projects. NRC's lack of a focal point to provide an effective agency-wide approach to management ofinformation resources management projects is seen in the systems development area, including the costly and behind schedule RIMS and ATS projects and the signincant problems encountered with the M-Cubed contract.

The agency's need to develop or follow guidance policies to effectively manage information resources management projects is particularly important for the NGN and CISSCO programs. Because of their signincance to NRC's future and the magnitude of effort in these programs, the agency needs a focal point to effectively monitor and communicate their status.

A more active role in the management of information resource activities from an agency-wide approach is necessary to improve the timeliness and budget performance of information resources management projects. Therefore, NRC's new CIO should have an active role in managing high risk and value projects to ensure that agency-wide interests are being served. In addition, the agency needs to proactively implement the CPIC and performance measure requirements for a broad portfolio ofIT investments, rather than developing restrictive criteria which effectively categorizes NRC systems at a level below the need to implement these requirements. Successfully implementing the ITMRA is imperative for the agency to meet Congress' goals of annual 6ve percent reductions in costs for operating and maintaining IT, and five percent increases in agency operational ef6ciency.

OIG/96A 11 p.g, g3

~~ ~ ~ _ -~~

iti _

Agency Oversight of information Resources Management Activ es RECOMMENDATIONS management To improve the management of NRC's information resourcesi t whi projects, OlG recommends thati NRC l andestablish cost effective a focal po n necessary oversight and direction to ensure the t me y implementation of projects. This focal point would:

l

1) Ensure all projects.

the development of cost and techn and reported on a regular basis. t d

2) Develop a process that designates res system projects.

of IRM policies and procedures, 3)

Expedite the development panicularly a life cycle system review process.

d implement the Additionally, NRC should move aggressively to appoint a CIO an ITMRA legislation. NRC should:

Task the CIO to monitor high value projects to ensure th 4) development and that costs are justified.

Develop a Capital Plannirig and Investment i Control g pro

5) 3 inform performance measurements for appropriate NRC

+

systems.

6)

Adopt the Sense of Congress goalsam. as measurable an targets for NRC's information resources management progr I

AGENCY COMMENTS for Nucleu On September 6,1996, the Deputy Executive Director df (DEDO)

Materials Safety, Safeguards, and Operat ili l report.

,... m l

I

Agency Oversight of Information Resources Management Activities RECOMMENDATIONS To improve the management of NRC's information resources management projects, OlG recommends that NRC establish a focal point which provides necessary oversight and direction to ensure the timely and cost effective implementation of projects. This focal point would.

1) Ensure the development of cost and technical milestone plans for all projects. This management information should be monitored l and reported on a regular basis.

g 51

2) Develop a process that designates responsibility for identifying and l documenting input from all potential users of proposed information j system projects. l

3) Expedite the development of IRM policies and procedures, particularly a life cycle system review process.

Additionally, NRC should move aggressively to appoint a CIO and implement the l ITMRA legislation. NRC should:

4) Task the CIO to monitor high value projects to ensure their timely development and that costs are justified.

5) Develop a Capital Planning and Investment Control process and I-performance measurements for appropriate NRC information systems.

6) Adopt the Sense of Congress goals as measurable and achievable targets for NRC's information resources management program.

AGENCY COMMENTS On September 6,1996, the Deputy Executive Director (DEDO) for Nuclear Materials Safety, Safeguards, and Operational Support commented on OIG's draft report. As part of his overall comments the DEDO implied that OIG's inability oic s oni r.,, u

1 Agency Oversight of information Resources Management Activities to determine whether NRC's IRM projects were within budget and on schedule resulted from our lack of familiarity with IRM's financial records and progress / tracking information. We take strong exception to this implication.

OlG auditors are technically skilled to evaluate management controls--whether they are or are not specifically designed to provide an audit trail. We believe that such management controls and review processes should be readily available and visible because they are fundamental management tools. We found that NRC lacks the management controls needed to systematically assess and effectively oversee the status of IRM projects.

The DEDO agreed in total or in part with five of the six recommendations and delayed commenting on one recommendation pending consultation with the new CIO. While it is management's decision not to comment on recommendation number 6 at this time, we continue to believe that NRC should adopt the sense of Congress goals as targets for its infctmation resources management program.

The DEDO agreed in part with recommendation numbers 1 and 3. For recommendation number 1 regarding the development of cost and technical milestones for all projects, the DEDO pointed out that the project management and control oversight should be properly tailored to the project scope, complexity, and costs, and therefore should vary among projects. OlG agrees with this concept. For recommendation number 3 concerning the development of IRM policies and procedures, the DEDO noted the actions taken as a result of a prior OlG report. We endorse the DEDO's intent to issue the remaining management directives as soon as it is practicable considering the new legislation and guidelines affecting this area.

Also, in an attachment to his memorandum, the DEDO provided other points of clarification on various portions of the report. We considered this information and revised our report as deemed necessary.

olG/96A 11 Page 15 I

]

Appendix l Agency Oversight of Inforrnation Resources Management Activities OBJECTIVES, SCOPE, AND METHODOLOGY During the course of survey work on the U.S. Nuclear Regulatory Commission's (NRC) information resources management program, the Omce of the Inspector I General (OlG) received information that two important agency information systems development projects were possibly over budget and behind schedule.

OIG initiated an audit to determine the status of these projects and the cause of their problems, and to assess the general planning and oversight of agency projects being managed by the Omce of Information Resources hianagement (IRhi). We determined the scope of our audit would focus on the three branches that will spend about $31.8 million, or 85 percent, of IRhi's Fiscal Year 1996 contract support budget: (1) Technology Infrastructure Branch (TIB), (2) Systems I Development and Integration Branch (SDIB), and (3) End User Support Services Branch (EUSB).

To conduct this audit, we interviewed senior management and staff from IRN1 and senior management from the Omces of Personnel, Nuclear Reactor Research, Nuclear Reactor Regulation, Administration, and the Omce for Analysis and Evaluation of Operational Data. We also consulted with officials from the Office of hianagement and Budget (OhfB). To observe an agency-wide oversight I mechanism ofinformation resources management, we attended meetings of NRC's Information Technology Council and the Senior Information Resource Management Omcials.

During the course of our audit, we evaluated NRC hianagement Directives, IRhi guidance, Omce of Administration contract guidance and other supporting documents. We also examined the Information Technology hianagement Reform Act of 1996, the Federal Acquisition Streamlining Act of 1994, U.S. Code Title i 41, and guidance from OMB. Finally, we also reviewed previous OIG audit reports and special evaluations, and testimony from the U.S. General Accounting Omce to Congress.

We reviewed the management controls for IRhi's oversight of projects. To l evaluate the controls, we reviewed published agency policy statements and IRM operating procedures and reports. OlG independently assessed the information from the control processes that is available for review. Our findings are discussed in the report.

Appendix I Agency Oversight of information Resources Management Activities Our audit was performed in accordance with generally accepted Government auditing standards during the period of January through June 1996.

(

I.

I I

I I

I 0:G/96A-11 Page 2 of 2

I t

Appendix ll  ;

Agency Oversight ofInformation Resources Management Activities REVIEW OF TIIREE KEY SYSTEM DEVELOPMENT EFFORTS l ILLUSTRATE NEED TO IMPROVE AGENCY-WIDE MANAGEMENT The U.S. Nuclear Regulatory Commission's (NRC) Of5ce of the Inspector General (OlG) reviewed two key agency system development projects, the Agency ,

Training System (ATS) and the Resource Information Management System  !

(RIMS), to determine reasons why these projects were signincantly over budget and behind schedule. Also, OIG is currently reviewing the Payroll / Personnel system (PAY /PERS)in a separate audit that we will report on shortly. We found that the agency did not take an agency-wide approach to managing these imponant agency information system development projects. As a result of ineffective communication between NRC ofGces about system requirements, the development of ATS has been delayed and PAY /PERS may also be late. In addition, we found that significant additional unr requirements requested by the sponsoring of5ce after the RIMS project started development caused the schedule to slip several times and drive project costs to over $700,000 more than estimated.

The ATS project is over budget and about 9 months behind schedule. The project's estimated completion date has been delayed because the agency did not identify and document the needs of all potential NRC users until February 1996, nearly 18 months after the project began. This led to additional user requirements and delayed system implementation. Further, NRC has overspent its original budget for integrating 11 Office of Personnel (OP) systems by about $255,000 and the work is still incomplete. NRC does not have a current estimate of the expected additional cost to develop ATS.

In November 1994, OIG reported ATS did not adequately support NRC managcrs I in their efforts to ensure that inspectors get mandatory training within the time frame required'. The agency agreed and stated a project was already underway to address deficiencies previously noted with ATS. The project's definition and requirements analysis were to be expanded to ensure the system provides reliable information for agency managers to oversee and track the status of staff training.

NRC stated a working group would be established to ensure that the data

' Review of NRC's Imnlementation of Insnection Manual Chanter 1245 Trainine Reauirements. OlG/94A-13, November 4,1994.

I OlG/96A.11 Page 1 of 5

Appendix ll Agency Oversight of Information Resources Management Activities requirements and system capabilities of the various future users of the system were well understood and incorporated. However, the agency did not form a multi-of6ce working group to address the noted problems until November 1995, about one year after the OlG report. In February 1996, this working group presented its f'ndings to OP, which was the Drst time NRC identiGed all potential users of the system and documented a comprehensive set of requirements, according to a senior agency manager. We believe NRC can improve on the timely collection of information system requirements from all potential agency users during the development process.

The agency's original plan to integrate 11 OP systems in 16 months at a cost of ,

about $445,000 has not been met. Instead, the agency overspent its original I budget for the work by an extra 57 percent or approximately $255,000 according to staffin the Of6ce ofInformation Resources Management (IRM). However, the exact cost for ATS is dif6 cult to determine because costs for another project are kept in the same task order. NRC does not have a current estimate of the expected additional cost to develop ATS. Further, the current scheduled date for implementing a fully operational ATS is December 1996, which is nine months later than the agency estimated in response to an OIG audit.' However, an IRM manager working on ATS does not think the December 1996 milestone is achievable because substantial unplanned enhancements to add flexibility to ATS l are needed and NRC does not have specine software needed to effectively implement a crucial system requirement at this time.

Currently, OlG is conducting a review of the PAY /PERS system. We have found that this system is another example where the agency did not adequately de6ne ,

and document requirements, or monitor costs effectively. There are also l indications of miscommunication between IRM and other agency of6ces, which may delay implementation of PAY /PERS. These problems will be discussed in greater detail in an upcoming audit report.

OIG also found that the RIMS project has taken over three times as long and over

$700,000 more than originally anticipated. The project has grown in complexity

' Deputy Executive Director for Nuclear Reactor Regulation, Research, and I

Regional Operations memorandum to the Assistant inspector General for Audits; July 27,1995.

OlG/96A.Il Page 2 of 5 I

Appendix ll Agency Oversight of information Resources Management Activities from the original concept developed in 1993, primarily because the sponsoring of6ce requested many additional system requirements during the first 1% years rf development. This led to repeated schedule slippages and RIMS is currently not anticipated to be completed until early 1997.

NRC's Of6ce of Research (RES) requested the development of RIMS to integrate its office-wide financial and project management information in response to weaknesses identiGed by an OlG report.' RIMS is being developed and released in three phases, or " Releases", which are intended to provide NRC offices valuable tools as soon as possible without having to wait until the entire RIMS is developed. In September 1994, RES proposed RIMS to other NRC offices as an agency-wide system for their use. IRM and the Of6ce for Nuclear Material Safety and Safeguards (NMSS) decided to implement RIMS, however, differences in office processes make it difficult for IRM to fully utilize RIMS and NMSS recently decided to stop its involvement.

We found that RIMS has consistently fallen behind schedule during development.

As shown in Figure 1, the milestones for RIMS slipped several times and the entire project is currently about 31 months behind the original schedule for completion. Release 1, expected to be completed in October 1993, was not finished until February '996, although portions ofit were released for RES' use in August 1994.' Release 3 is scheduled to be finished in November 1996, but IRM staff told OIG that completion will probably be pushed into 1997.

Moreover, OIG could not determine precisely how much over budget RIMS is because an original budget was never developed. A senior IRM manager told OIG that the system probably would have cost around $300,000 to $400,000 to develop, but acknowledged that an original estimate may not have been prepared or it was not kept. As of May 6,1996, NRC's contractor has charged over

'Imorovements Needed in Financial and Administrative Accountability for Office of Nuclear Reculatorv Research Funded Work at Department of Enerev 1.aboratories. OlG/92A-20, March 5,1993.

'RES and IRM agreed to split Release 1 into two sections in March 1995.

Release I was accepted by RES as complete on March 24, 1995, while Release 1 A was accepted on February 12, 1996.

OIG/96A.Il Page 3 of s l

Appendix 11 Agency Oversight of Information Resources Management Activities a

s Year NRC Planning - 1993 - 1994 Nt 1995 Md 1996 --+1 .

Jun Sep Dee Mar Jun Sep Dee Mar Jun Sep Dec Mr Jun Sep Dee Ument  ;  ;  ;  ;  ;  ;  ;  ;  ;  ;  ;  ;  ;  ;  ;

RelI ! Ret 2 Ret 3  : i i Scoping Plan , ', A  : l l June 1993 , ' Y  !  !  !

l l l l Rell 'Fel2

Rell i l j Program Plan , , A l l Oct.1993

' Y  ! Actual  !

Q ,

IRell Rell Rell i OCINCTY  !

! Program Plan , , A i i i Nov.1994  :

j'

' Y  !

. KlMS Task l I R'l l R'l l^ Rh2 Rel3 I Schedule

^  ;

A 4

, l y l April 1995 j j j j l

l RelI RelllA Ret 2 Ret 3 i-Program Plan .  ; , A Nov.1995  :

Y  !

5 Rell IRellA Rel2 Ref{3 Program Plan -  : l

. . , A March 1996 - * - ' y

! Figure 2: RIMS Development Milestone Slippages

$1,100,000 for work performed on RIMS tasks. Developing RIMS has cost over

$935,000 so far, while IRM roughly estimates an additional $300,000 to $360,000 is needed to comp!ete Releases 2 and 3. In addition, about $52,000 was spent adapting RIMS for NMSS prior to its decision to defer implementation.

, IRM stated that several factors contributed to the delays in developing RIMS. The primary reason is that additional requirements in system capabilities were requested by RES during development. The system's requirements were not fully identified in the original plan and other needs arose while developing RIMS. For example, great urgency was placed on having planning features of RIMS available for use prior to fiscal year 1995, however an inadequate project planning module was developed that later required a substantial reworking. Other factors also OlG/96A.11 Page 4 of 5 I

i Appendix 11 Agency Oversi0ht of Information Resources Management Activities contributed to system development delays, such as: (1) significant turnover in cotitractor personnel assigned to the project, (2) NRC changed its accounting system structure during the development of RIMS, and (3) RIMS is one of the first " Windows-based" applications developed by NRC.

Although developing RIMS has been difficult, the project sponsor told OIG that he is very pleased with the system. He acknowl edged that extra features added I to the time and cost to build RIMS, however he believes the system was developed in a timely manner for his needs. He believes additional requirements were necessary in order to match the operation of RIMS to RES staffs' work processes. Further, he believes RIMS is a first class information system that has greatly reduced the generation of paper in RES, and that staff has high confidence in the data RIMS maintains.

I I

I I

I I

OIG/96A.Il Page 5 of 5 W

[

Appendix 111 Agency Oversight of information Resources Management Activities AGENCY COMMENTS ON DRAFT REPORT

s. .m

.  % UNITED STATES

• E NUCLEAR REGULATORY COMMISSION WASHINGTON, D C. 30eeMCD1

/

September 6, 1996 MEMORANDUM TO: Thomas J. Barchi

[. Assistant Inspector General for Audits Office of the Inspector General h FROM: Hugh L. Thompson, Jr. k r -

Deputy Executive Director for i Nuclear Materials Safety, Safeyprds, and Operations Support

[

SUBJECT:

IMPROVEMENT NEEDED IN AGENCY OVERSIGHT OF INFORMATION RESOURCES MANAGEMENT ACTIVITIES By memorandum dated July 26, 1996, your office provided the Executive Director for Operations with a draft report entitled " Improvements Needed in Agency Oversight of Information Resources Management Activities," (0!G 96A-ll). We have reviewed the subject report and are providing responses to your recommendations. Specific comments are also provided as an attachment to this

[ memorandum.

Your audit identifies several areas whare the Office of Information Resources Management (IRM) needs to improve the way it manages projects. In general,

[ IRM management was aware of and had taken steps to improve our procedures to address many of the issues raised in your draft report. The Comprehensive Information Systems Support Consolidation (CISSCO) initiative which began in 1994 and was recently awarded, will provide the agency with its first fully integrated approach for developing information systems. CISSCO also prescribes an approach to project management contained in the systems developnent life cycle management (SDLCM) methodology that the agency will use for information systems development projects. In addition, the NRC recently awarded a comprehensive networking contract, 'Next Generation Network," that supports all of the agency's networking activities. This contract, similar to the CISSCO initiative, shkid improve the overall management of projects associated with the NRC's computer networking activities.

L Your general findings indicate that the NRC (and IRM in particular) lack the financial and technical management control processes needed to effectively oversee information resources management projects. You report that IRM provided considerable information concerning the projects when asked, but conclude that the OlG generally could not determine from this material whether the projects were within budget fd on schedule. We agree that it is difficult for an auditor not f amiliar with IRM financial records and progress / tracking information to readily determine budget and schedule status for each IRM project under development. However, we do not agree that IRM was or is unable to t' ek progress and monitor costs for these same projects. IRM uses various management techniques to track the cost, the scope and the progress of hundreds of projects that the office manages oh a daily basis. For example, the Deputy Office Director discusses the progress of IRM projects

'II Page 1 of 9

Appendix lil Agency Oversight of information Resources Management Activities 2

during frequent meetings with Branch Chiefs, IRM project officers, and task l g

managers. Such meetings often provide more insight into project status than periodic reports and, because they are interactive, can be used to make on-the-spot decisions to correct problems.

Your general findings also indicate that the lack of sound management control processes has resulted in unnecessary project delays, miscommunicated system requirements, and exceeded planned budgets. To support this finding, OlG report contains a review of three key system development efforts, and E concludes that they are behind their original schedule and over their original E budget. We agree that the projects have taken longer and cost more than oeiginally envisioned but not for the reasons concluded in the 0!G report and not because the agency lacks the management control to assess the status (budget and schedule) of these projects. The basis for our views are provided in an attachment to this letter.

With respect to the recommendations contained in your report, we submit the g following: g Recorcendation la: Ensure the development of cost and technical milestones for all projects. This management information should be monitored and reported on a regular basis.

Agree in part, but disagree with respect to this approach for All projects.

We agree that the development and documentation of cost and technical milestone plans are an integral part of project management and that the agency could bentfit from the application of a more formal approach in the management of IT projects. During 1995, IRM, in coordination with its primary user offices, initiated the development of an agency system development life cycle methodology (SDLCM). This SDLCM was issued in a training session for potential CISSCO project / task managers on July 24-25, 1996, and includes, as a task, the development and update of project management plans during phases of the systems life cycle. Effective September 30, 1996 (the projected date that the CISSCO task order awardee will begin NRC work), the SDLCM will be used for all information system (IS) development projects managed by the Office of IRM and all 15 development projects managed by other offices using the CISSCO contract.

In addition to SDLCM training, a special training module is being developed to cover project management planning, control, documentation, and tracking, including progress and costs. Each NRC project manager is required to attend this training prior to managing a project under CISSCO.

However, as we agreed in our conversation on August 27, 1996, all agency IT projects are not equal in terms of scope, complexity, and costs, and accordingly, should ' tot be treated equally in terms of project management planning and es.nagement oversight. A detailed project management plan, with costs and se' aule provided for each phase, task and subtask of a project, can be very inv.sved and expensive to create and maintain. To be cost-effective, I

I

l Appendix ll1 Agency Oversight of information Resources Management Activities 3

the approach followed Ley the project team should fit the scope and complexity of each individual project. Likewise, the degree of management monitoring of individual projects should be tailored to the scope, complexity, risk, cost and importance of the project to the NRC mission. The Director of IRM will issue further guidance to staff, identifying criteria and specifying the level of detail expected for the range of information systems managed by IRM and other agency staff.

CISSCO Project Management Training. Completion date: September 18, 1996 Issuance of a redsed draft " Systems Development Life Cycle Methodology."

Completion date: November 27, 1996 Guidance to staff on application of project management planning to information systems projects: November 27, 1996 Recomendation Ib: Develop a process that designates responsibility for identifying and documenting input from all potential users of proposed information system projects.

Agree.

The working draft " System Development Life Cycle Methodology" that IRM issued in July 1996, includes tasks to identify and document requirements from all potential users for a project, and a project responsibility matrix. This matrix is being further refined to address the complexities associated with developing systems with multiple sponsors and users. As indicated in response to recommendation la, a revised draft " Systems Development Life Cycle Methodology" will be issued by November 27, 1996. Finalization of the SDLCM will await the review and approval of the agency's CIO.

f Completion date: See item 2 under response to recommendation la.

Recomendation Ic: Expedite the implementation of IRM policies and procedures, particularly a systems life cycle review process.

Agree in oart.

IRM did issue a set of IRM office-wide policies and procedures in response to the subject IG report. On March 31, 1993, a memorandum was sent to IRM project officers outlining unauthorized procurement actions and reminding them that violations would not be tolerated. In July 1993, an "IRM Project Officer Desk Referente" was issued. These products, which responded to recommendations cantained in an IG audit -eport dated January 8, 1993 (report was erroneously dated January 8, 1992), entitled " Review of IRM's Management of its Contracts," closed out the follow-up actions for the audit.

We agree that it is desirable for the agency to issue the other five planned management directives as soon as it is practicable. They are currently available in working draft form. However, several of the directives must be rethought and substantially revised in light of the recent passage of the OIG/96A-Il Page 3 or 9 I

Appendix fil Agency Oversight of information Resources Management Activities I

4 ITMRA and new procurement legislation. Further changes will be needed when OMB issues a revised Circular A-130, which will have provisions that provide federal policy on how agencies should implement the ITMRA.

The two planned directives that are applicable to the subject of this audit are the Application Systems Life Cycle Directive and the Planning, Budgeting and Investment Control of f!P Resources Directive. The former will be prepared by December 31, 1996 and issued as interim guidance for staff use until it is approved by the CIO. The latter will be developed in draft after NRC gains experience with applying a modified version of the CPIC process during the FY 1999 budget cycle. Issuance of a formal directive will await approval of the CIO.

Application Systems Life Cycle Directive (issued as interim guidance for staff use). Completion date: December 31, 1996.

Planning, Budgeting, and Investment Control of f!P Resources Directive (developed in draft for CIO review). Completion date: April 30, 1997.

Recomendation 2: Task the CIO to monitor high value projects to ensure their i timely development and that costs are justified.

Agree, however, NRC does not have to task the C10 to monitor high value l projects since this is an integral part of the C10's job and is required by the provisions of the ITMRA. The ITMRA requires the head of the agency to establish a capital planning and investment control (CPIC) process to manage agency IT investments through their life cycle (project selectio'1, implementation, and operation). The law requires the CIO to assist the Chairman in implementing the CPlc process and monitoring IT projects. Since the requirement for monitoring high value projects is an integral part of implementing the CPIC process, it will be accomplished in the context of responding to recommendation 3.

Recommendation 3: Develop a Capital Planning and Investment Control (CPIC)

Process and performance measurements for appropriate NRC information systems.

We agree that NRC should comply with provisions of the ITMRA regarding capital planning and investment control and a work plan is under development to accomplish this. It should be noted, however, that since the ITMRA did not become effective until August 8, 1996, and OMB has yet to issue final implementing guidance through its revisions of A-130, no federal agency is currently in a position to implement the CPIC provisions of the ITMPl., nor a does OMB expect Federal agencies to do so immediately. g, As a result of your audit, you have recommended that NRC develop a Capital Planning and Investment Controi (CPIC) process for agency information systems.

We agree completely. I believe there was a misunderstanding concerning this issue and I am prepared to meet with you to discuss this part further if needed. Both the Director of IRM and I fully support the CPIC proposal, and we are currently directing the development of a plan to implement CPIC.

OlG/96A Il pm , g ,

Appendix lil Agency Oversight of information Resources Management Activities 5

Our current plan is to develop and implement a modified CPIC process for the FY 1999 budget cycle. The plan calls for the analysis and oversight of a limited CplC investment portfolio of IT projects to gain experience with the process. The approach will be modified to accommodate anticipated guidance from OMB, and CIO direction, during the FY 2000 budget cycle.

Develop draft modified CPlc process for use during FY 1999 budget cycle.

Completion date: December 30, 1996.

Recommendation 4: Adopt the Sense of Congress goals as measurable and achievable targets for NRC's information resources program.

NRC's position on this matter should be delayed pending consultation with the new CIO.

We appreciate the opportunity to review the draft report. Please contact Gerald F. Cranford at 415-7585 or Arnold E. Levin at 415-7458 if you have any questions regarding this response.

Attachment:

As stated cc: J. Taylor, EDO G. Cranford. IRM J. Blaha, OEDO C. Dolinka, OEDO OIGS6A-ll Pare 5 of 9

~~

t .

l l

1 Appendix ll1 Agency Oversight of Information Resources Management Activities ATTACHMENT SPECIFIC COMMENTS ON OlG REPORT AND FINDINGS

" Improvements Needed in Agency Oversight of Information Resources Management Activities"

1. Page three- Opening Paragraph NRC needs to have sound manaaement control orocesses to ensgre the effective and efficient use of the tens of millions of dollars invested in information resources manaaement pro 1ects. Our review disclosed that NRC lacks control orocesses to systematically provide NRC manaaement with information needed to assess the status of IPM projects. Without l such manaaement controls. the aaency has incurred unnecessary croiect delays. m'scomunicated system recuirements. and exceeded olenned budoets l Appendix il of the OIG draft report contains a review of three key '

system development efforts that the OlG concludes are behind schedule and over budget because IRM lacks the control processes to systematically provide NRC management with information to assess the E status of IRM projects. These three systems, ATS (Agency Training g Subsystem), RIMS (Resource Information hanagement System) and Pay /Pers (Payroll and Personnel System) are all behind their original schedule and over their original budget but not because the agency lacked information for management to assess the status of the projects. Each of these systems has been problematic since its inception and each for unique, differing reasons.

ATS is one of several subsystems that comprise the agency's automated l g

Human Resources Information System. Approximately 3 4 years ago, IRM management together with senior managers from the NRC's major offices decided, as part of the agency's Information Technology Strategic Plan, to adopt " client-server

• technology as the NRC's preferred computing platform. This was a significant change from the agency *s reliance upon dedicated Data General minicomputer systems and mainframe timesharing applications to process the agency's information. IRM together with the Office of Personnel (OF) chose a number of " subsystems", including ATS, as the initial NRC applications to develop under the client-server environment. Development of ATS as a client-server application began in l the latter part of FY 1993. At that time, IRM was the developer and OP l was the sponsor for this project. IRM looked to OP for requirements definition and based upon those requirements began the development effort using M-Cubed, a small 8a firm, to do Lne actual systems work. l Midway into the process, which at that time was going satisfactorily, the OIG audited the agency's overall training program and discovered '

problems with the systems (each regional office kept individual records of trainin, that were being used to record instances of training I (especially mandatory resident inspector training) and recommended the agency do something to ensure the inspectors were being trained as scheduled. The decision was made to modify the plans for developing ATS OlG/96A.ll Page 6 of 9 I

Appendix ill Agency Oversight o)Information Resources Management Activities to include the recommendations to improve the training system (s) made by the 0!G. This resulted in significant changes to the ATS that were not consistent with the original design agreed upon by IRM and OP. The modifications to ATS included other office (AE00, NMSS, NRR) changes and comments in addition to the original set of requirements developed by OP. The ATS virtually collapsed under these additional needs and the ATS development was further delayed by a contracting holdup involving approval to raise the contract ceiling on the M-Cubed contract. IRM management was aware of all of these problems and attempted to work with the sponsors (now multiple instead of just OP) as well as the Division of Contracts to obtain approval to raise the M-Cubed contract ceiling.

It was the result of these external events and not that IRM management lacked management controls that caused the ATS development project to fall behind schedule and exceed the original project cost estimate.

RIMS, the system developed to manage projects in the Office of Nuclear Regulatory Research (RES), is a classic example of requirements " creep" leading to a syste.n that evolved over time vertus a system designed from a set of clear, concise specifications at the outset of the project.

When RIMS was originally specified, it was meant to serve a specific purpose in RES. As development continued, the need for expanded capability was identified and additional functionality was programmed into the system. The functionality of RIMS, which was begun in mid-1993, changed so frequently that its development was divided intu three phases and IRM had to freeze requirements in order to focus on producing a working system. Each phase of the system represented a version of RIMS that was considered a "prohction" module that met a specific set of requirements as negotiated between IRM and RES. We agree that IRM should have updated its cost and scheduling information to reflect the continued requirement changes and additions but disagree that lack of management controls and oversight lead to excessive development costs and misred schedules. At no time during the development of RIMS did IRM management believe that money used to implement the RIMS specifications was being wasted although we were concerned with the continued request for changes and modifications.

Pay /Pers and the reasons for deviation from schedule and spending plans is the subject of another 0!G audit and will be discussed in the response to that draft report. However, we contend that IRM did maintain detailed cost and schedule reports and was fully aware of the situations that existed to cause an extension to the delivery schedule as well as the reasons for neediaq additional funding for the Pay /Pers appl ation. Schedule slips or cost overruns were not the result of a lack of management controls.

2. Page four, first paragraph, third sentence Successfully achievina technical oerformance is imoortant. however the need to manace the costs and schedules of IRM oro_1ects is eoually imoortant.

2

1. ' W I Page 7 of 9

i I'

Appendix 111 Agency Oversight of Information Resources Management Activities l Ii l We are not in full agreement with the statement that managing costs and schedules is equally important as delivering a good technical solution.

We agree that both propositions are important but net equali: 50.

Although it is desirable that all system development projects are delivered on time and within budget, the primary objective of the development team must be to deliver the best possible system first and consider budget and timing as a secondary objective. If, in order to deliver a good technical solution, it is necessary to either slip the

' schedule or exceed the cost, our opinion is that option is better than delivering an inferior product, although one that works, on time and within budget.

3. Page five, paragraph one, fourth sentence The IRM Contract Financial Status Record Worksheet (Worksheet) was issued in Seotember 1991 to IRM orotect officers in response to weaknesses detected in an internal IRM review.

This worksheet was not issued to be used to manage IRM projects and tasks. Specifically, this document resulted from a requirement to manage commitments and obligations on IRM contracts.

4. Page six, paragraph one, second sentence However. Our review of the monthly recorts found that the statu} of only abput 25 Dercent of IPM's hianer oriority cro_iects are reoorted and uodated status informatien is not comoared to cro_ lect milestones.

The agency's monthly status reports are provided to highlight progress on high profile projects that are of interest to senior IRM management.

These reports are not intended to track each of the 100 plus projects 3 that are active in IRM at any point in time but to focus on the most g important IRM projects including CISSCO, ADAMS, PC-Refresh, the agencywide implementation of Windows 3.1, and so forth. The projects that are contained in the monthly reports were negotiated between the Office Director and the Branch and Staff Chiefs, in addition, the Branch and Staff Chiefs were instructed to include any issue they felt was important although it was not related to one of the targeted monthly projects. Ir the near future, we plan to make this inforcation available to other NRC employees via the internal IRM Home Page.

5. Page seven, second paragraph, second sentence One senior IRM manaaer told OIG that buildina a workino system is IRM's customer service coal. even it takes a few extra months and more funds than estimated. However we found that IRM received $11.827.000 in increased fundino form the Office of the Controller (0C) from FY 1994 throuch 1996 to succort additional systems develooment work that had not been budaeted for those years. sianificant!v increasina SDIB's budaet in f.L 1995 and 1996.

OlGS6A.11 g

Appendix Ill Agency Oversight of Information Resources Management Activities These two sentences suggest that supplemental funding was necessary to support additional requirements on existina work rather than new, previously unfunded work, although either may be justifiable. During FY 1994, IRM issued the agency's first Information Technology Strategic Plan that listed four primary objectives. One of those objectives was the development of an agencywide document management system to replace the NUDOCS system. Since this was one of the agency's highest priorities, some of the supplemental funding went to cover the startup costs for this system. In addition, funding was used to cover additional systems development projects that had been approved by the Information Technology Council but for which IRM had no funding. Most of the funding went for new starts and not to cover cost overruns and schedule slippages.

(

r

(

OlG/96A-Il Fate 9 of 9

Appendix IV Agency Oversight of Information Resources Management Activities U.Se NRC FUNCTIONAL ORGANIZATION CHART I

NRC

-l Commissioners B

l Executive Director for Operations Aaslaant for Operusams l l ,

Omes for Depw, EnerwM D6rener for Depwy Emeevth Throner for

,.--r-f.,. un - - ,e - .

_ ' Operwhens a Research i Amferards & Operwens 5.pport Omce of mg l l l l l Pereennef gu gg,,

l Omoof fltese Prugrams Omre of Safersement Omce of inweesupusens mg Adminuarw een his mene Omce of semii postness i and Oswt IUghts

' I Omes of I *

,, i g, and Safeguards

• "*I '"

Nuclear Rancier Regulation I e Regional Omces neglen a rhiwdeph.

seghan II Ashnta Reglen IIIChimse neg=n lv Dau s 1

Areas Andhed OlG/96A Il Pge I d 1

Appendix V Agency Oversight of Information Resources Management Activities MAJOR CONTRIBUTORS TO THIS REPORT I

Corenthis B. Kelley I Team Leader i Scott W. Buchan Senior Management Analyst Michael A. Cummins I Auditor i

I I

I I

I OlG/96A 11 PageI ofI

Appendin VI Agency Oversight of Information Resources Management Activities GLOSSARY: OFFICE OF THE INSPECTOR GENERAL PRODUCTS

! INVESTIGATIVE

1. INVESTIGA TIVE REPORT - WHITE COVER An Investigative Report documents pertinent facts of a case and describes available evidence relevant to allegations against individuals, including aspects of an allegation not substantiated. Investigative I reports do not recommend disciplinary action against individual employees. Investigative reports are sensitive documents and contain information subject to the Privacy Act restrictions. Reports are given to officials and managers who have a need to know in order to properly determine whether administrative action is warranted. The agency is expected to advise the OlG within 90 days of I receiving the investigative repcrt as to what disciplinary or other action has been taken in response to investigative report findings.

2. EVENT INQUIRY - GREEN COVER The Event Inquiry is an investigative product that documents the examination of events or agency actions that do not focus specifically on individual misconduct. These reports identify institutional weaknesses that led to or allowed a problem to occur. The agency is requested to advise the OlG I of managerial initiatives taken in response to issues identified in these reports but tracking its recommendations M not required.

3. MANA GEMENT IMPLICA TIONS REPORT (MIR) - MEMORANDUM l

MIRs provide a " ROOT CAUSE" analysis sufficient for managers to facilitate correction of problems and to avoid similar issues in the future. Agency tracking of recommendations is not required.

AUOIT

4. AUDIT REPORT - BLUE COVER An Audit Report is the documentation of the review, recommendations, and findings resulting from an objective assessment of a program, function, or activity. Audits follow a defined procedure that I allows for agency review and comment on draft audit reports. The audit results are also reported in the OlG's " Semiannual Report" to the Congress. Tracking of audit report recommendations and agency response is required.

5. SPECIAL EVALUA TION REPORT - BURGUNDY COVER A Special Evaluation Report documents the results of short-term, limited assessments. It provides an initial, quick response to a question or issue, and data to determine whether an in depth I independent audit should be planned. Agency tracking of recommendations is not required.

REGULATORY l

6. REGULA TORY COMMENTARY - BROWN COVER I Regulatory Commentary is the review of ex! sting and proposed legislation, regulations, and policies so as to assist the agency in preventing and detecting fraud, waste, and abuse in programs and operations. Commentaries cite the IG Act as authority for the review, state the specific law, regulation or policy examined, pertinent background information considered and identifies OlG concerns, observations, and objections. Significant observations regarding action or inaction by the agency are reported in the OlG Semiannual Report to Congress. Each report indicates whether a response is required.

1 OlG/96A 11 PageI ofI

~

W