ML20236S596

Audit Rept OIG/97A-21, Review of Project Manager Position in Nrr

ML20236S596
Person / Time
Issue date:	07/13/1998
From:	NRC OFFICE OF THE INSPECTOR GENERAL (OIG)
To:
Shared Package
ML20236S599	List: ML20236S596 ML20236T784
References
OIG-97A-21, NUDOCS 9807270091
Download: ML20236S596 (31)

Text

- _ _ _ - _ - _ - _ - _ - _ _ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

,p OFFICE OF THE INSPECTOR GENERAL U.S. NUCLEAR REGULATORY COMMISSION a

Review of the Project Manager Position in the Office of Nuclear Reactor Regulation OlG/97A-21 July 13,1998 AUDIT REPORT s,c 47 f Q 9Fh:P

- [- 7, ,

o

...s

ea72='t' e m3 PDR p

. , . ~ ..

Review of the Project Manager Position in NRR REPORT SYNOPSIS The U.S. Nuclear Regulatory Commission's (NRC) Office of Nuclear Reactor Regulation (NRR) established a Headquarters-based Project Manager (PM) position to assist in licensing and monitoring the nation's commercial nuclear power plants.

The PM role has expanded significantly into a multi-purpose position which has resulted in PMs being responsible for a wide range of duties, such as performing licensing actions and inspection and assessment of licensee performance. Due to the significant expansion of duties, we reviewed the role of PMs, factors that affect their work, and the management processes used to oversee this importut agency function.

As initially established, PMs were segregated by functional areas. For exarnple, there were Licensing PMs, Operating Reactor PMs, and Research Reactor PMs.

However, when the need to license commercial nuclear power plants decreased, NRR shifted its focus to the operational oversight of the plants and moved the Licensing PM position into the same project organizations as the Operating Reac+.or PMs. According to NRC's Operating Reactor Project Manager's Handbook, i;Ms are responsible for approximately nine main functions, such as performing licensing actions and activities, serving as the Headquarters contact point for licensees and NRC Regional Office staff counterparts, coordinating preparation of safety evaluation reports and environmental impact statements, participating in special assignments, and coordinating and presenting information to the Commission and other groups on specific projects and subjects.

As a result of our work, we found there is a lack of consensus between Headquarters and Regional Office staff, as well as between individual NRR offices, regarding management's expectations of the PM role. We analyzed information regarding the amount of time PMs charge to various activities and found a wide variance among PMs in time charged to activitics considered most important by senior agency managers. In addition, NRC staff, including PMs, noted they are responsible for too many activities, which may be inhibiting their overall effectiveness.

We also found that NRC has not provided adequate management oversight to ensure PMs are sufficiently trained to perform their jobs. For example, in 1989, NRR established its current guidance which identified the types of training needed by PMs to perform their various assignments, but this guidance has not been followed. This has resulted in a disparity of training among PMs. In addition, PMs olG/97A-21 Pagei

I Review of the Project Manager Position in NRR are expected to perform certain job tasks without receiving formal training in how to accompEsh those tasks. Ineffectual training oversight, combined with the lack of clear PM expectations and competing job demands, may prevent PMs from achieving their maximum effectiveness.

We also examined the timeliness of processing certain types of technical reviews and found that the primary impediment to improving the process is the recurrent need for additional information from licensees and Regional Office staff.

Finally, NRC has recently taken steps to assess factors affecting the PM function, but the scope and outcomes of these efforts are unclear at this time. While the Office of the Inspector General (OlG) supports the NRC's efforts in this regard, we recommend any initiatives be targeted to ensure the agency:

Re-evaluates and clearly defines PM job expectations and communicates the expectations to Headquarters and Regional Office staff; and, Ensures PMs receive the training necessary to meet these expectations.

OlG plans to monitor NRC's progress in correcting the weaknesses identified in this important agency function.

I l

I olG/97A 21 p g,;;

Review of the Project Manager Position in NRR f TABLE OF CONTENTS REPORT SYNOPSIS . . . .

. .i INTRODUCTION .... . .. 1 BACKGROUND. .. .. . . . . . 1 FINDINGS . . . .. . . .. .. . . 2 THE ROLE OF THE PROJECT MANAGER NEEDS BETTER FOCUS . 3 IMPROVEMENTS NEEDED IN TRAINIfJG OVERSIGHT . .. 8 RECURRENT REQUESTS ADD TO PROCESSING TIME . 9 CONCLUSION . . .. . . . . .. . 11 RECOMMENDATIONS . .. .. 12 AGENCY COMMENTS . . . . .. . . . 13 APPENDICES I Objectives, Scope, And Methodology 11 NRR Organization Chart lil Acronyms IV Agency Comments V Major Contributors To This Report VI Glossary: Office of the Inspector General Products olG/97A.21

Review of the Project Manager Position in NRR 1

INTRODUCTION The U.S. Nuclear Regulatory Commission's (NRC) Office of Nuclear Reactor Regulation (NRR) established a Headquarters-based Project Manager (PM) position to assist in licensing and monitoring the nation's commercial nuclear power plants.

The PM role has expanded significantly into a-multi-purpose position which has resulted in PMs being responsible for a wide range of duties, such as performing licensing actions and inspection and assessment of licensee performance. Due to the significant expansion of duties, we reviewed the role of PMs, factors that affect their work, and the management processes used to oversee this important agency function. This report presents the results of our review. Appendix 1 contains additionalinformation regarding our objectives, scope, and methodology.

BACKGROUND NRR is responsible for ensuring public health and safety through licensing and inspection activities at all commercial nuclear power reactor facilities in the United States. The responsibility for providing overall project management activities related to licensing and inspection of nuclear power reactors resides within the Associate Director for Projects (ADPR) organization. (See the organization chart attached as Appendix 11.) ADPR has 11 Project Directorates associated with NRC's four Reg;onal Offices. The Directorates, which are headed by Project Directors (PD), direct the activities of approximately 75 PMs. For the purposes of this report, lead Project Managers (lead PMs) will refer to PMs who have " dedicated

• plant assignments. Each operating reactor has, in most cases, a lead PM who is the focal point for allinformation related to that plant. PMs were originally segregated by functional areas. For example, there were Licensing PMs, Operating Reactor PMs, and Research Reactor PMs. However, when the need to license commercial nuclear power plants decreased, NRR shifted its focus to the operational oversight of the plants and moved the Licensing PM position into the same project organizations as the Operating Reactor PMs.

olG/97A-21 Page 1

I Review of the Project Manager Position in NRR NRC's Operating ReactorProject Managets Handbook provides PMs guidance and outlines approximately nine main functions required of a lead PM, such as performing licensing actionsW and activities,<2) serving as the Headquarters contact Ei point for licensees and NRC Regional Office staff counterparts, coordinating E l preparation of safety evaluation reports and environmental impact statements, participating in special assignments, and coordinating and presenting information to the Commission and other groups on specific projects and subjects.

l In July 1997, NRR contracted for a job task and functional analysis (JTA) to examine the tasks perfomled by ADPR personnel and evaluate how and by which position, specific tasks and functions are best performed, in addition, the JTA will identify associated training requirements. The estimated completion date of the JTA is July 1998. As part of another internal evaluation, NRC may perform a

• top down"

! assessment of all the programs and activities in NRR. The assessment would l examine NRR's current programs and activities to determine whetheHhey achieve >

their desired purpose in an efficient and effective manner, and wbather they should be continued, redesigned, or sunset. The extent or actual perMrmance of this assessment is uncertain at this time. The findings contained in this Office of the inspector General (OlG) report should assist NRC management in improving the effectiveness of the PM function, as well as for making decisions on the scope and focus of future assessments.

l 1

FINDINGS We found there is a lack of consensus between Headquarters and Regional Office staff, as well as between individual NRR offices, regarding management's i expectations of the PM role and the activities PMs currently perform. The expectations vary considerably, as do the functions agency mansgers consider most important for PMs to perform. For example,75 percent of the senior NRR 3

I

" Licensing actions" are those licensee requests, such as license amendments, reliefs, or l exemptions, that require review and approval by NRC staff before they may be >

implemented by the licensee. Nearly every such action involves consideration of the l

effect of that action on public health and safety and the environment. In most cases, this consideration involves preparation of a documented evaluation of those effects known as a safety evaluation.

2

. Licensing activities" are reviews or other tasks that NRC performs for itself, such as answering Technical Information Assistance requests from Regional Offices and reviewing the responses to Generic Letters and Bulletins.

olG/97A-21 Page 2

I l

Review of the Project Manager Position in NRR managers we interviewed believe licensing work is the PM's most valuable function, f while Regional Office officials generally said that PMs add the most value as focal points of information regarding the status of plant activities, including licensing i

actions. Our analysis confirmed this lack of role clarity. We analyzed information regarding the amount of time PMs charge and found that PMs charge differing amounts of time to a wide variety of activities. In addition, NRC staff, including PMs, noted they are responsible for *oo many activities, which may be inhibiting their overall effectiveness.

We also found that NRC has not provided adequate management oversight to ensure PMs are sufficiently trained to perform theirjobs. In 1989, NRR established its current guidance which identifies the types of f. raining needed by PMs to perform their various assignments, but this guidance has not been followed. This has resulted in a disparity in the knowledge, skills, and abilities among PMs, which may also impact their effectiveness. In addition, PMs are expected to perform certain job tasks withoui receiving formal training in how to accomplish those tasks. For example, PMs are responsible for providing input to various sections of the Systematic Assessment of Licensee Performance (SALP) reports, including the Maintenance module, even though PMs may not be trained in maintenance engineering skills.

We examined the processing of certain types of technical reviews and found that the recurrent need for NRR staff to request additional information (RAI) from licensees and Regional Office staff tends to impede the timely completion of licensing actions. The agency might reduce lengthy processing times by focusing on ways to reduce the quantity of RAI requests. As a result, PMs may be more effective in completing licensing r.ctions and activities in a timely manner, and more responsive to licensees and Regional Office staff needs.

THE ROLE OF THE PROJECT MANAGER NEEDS BETTER FOCUS mi The PM role has expanded significantly into a multi-purpose position which has resulted in PMs being responsible for a wide range of activities. We examined whether NRR management's expectations of PMs, as well as the views and expectations of other NRC staff, are clear. Although PM expectations are defined in the NRC Operating Reactor Project Manager's Handbook, we found that management's expectations of PMs vary widely, and the PM role needs better focus.

Expectations are Widely Varied in meeting with agency officials at Headquarters and two Regiona: Offices, we found considerable differences regarding management's expectations of the PM's role, including which PM functions managers consider most important. We olG/97A 21 Page 3

_ _ _ _ _ _ _ _ - ~

I Review of the Project Manager Position in NRR augmented the information gained from these interviews by analyzing data obtained from two agency information systems? That data comprised time charged to NRR activities during fiscal year (FY) 1997 and was restricted to lead PMs.

We found a lack of consensus between Headquarters and Regional Office staff, as well as between individual NRR offices, regarding expectations of what the PM role should be and the relative importance of functions currently performed by PMs. For example, NRR senior managers agreed that the activities conducted by PMs fit into three broad functional areas: (1) coordination or performance of plant-related licensing actions and licensing activities, sJch as safety evaluations; (2) acting as a focal point for information related to activities for a particular plant; and (3) participation in plant safety assessments. In general, NRR senior managers considered licensing work as the most important PM function. In contrast, Regional Office staff generally stated that the most important PM function is to be a focal point for information on activities related to a specific plant in support of Region-based oversight of operatino reactors.

Headquarters sta+f also stated that while processing licensing actions and activities is the most important funct!on of the PM position, other activities assigned to PMs may be reducing PM effectiveness and resulting in increases in the licensing action backlog. For example, the licensing action inventory increased 10 percent in FY 1996 and 15 percent in FY 1997. In addition, both managers and staff expressed concerns that PMs are being asked to give priority to processing non-safety-related controlled correspondence,W petitions from the public,H) and recent increased emphasis on PMs updating technical documents such as Final Safety Analysis Reports (FSAR).

We analyzed PM time charges during FY 1997 and found wide ranges in the g amount of time charged to various functions performed by lead PMs. As illustrated B in Table 1, we found an uneven distribution of time that 'Ms spent on activities senior managers deem important, such as licensing, wheie the actual amount of time PMs charged varied from 12.1 percent to 84.3 percent.

3 NRR's Workload information and Schedu;.wg Program (WISP) and NRC's Regulatory Information Tracking Oystem (RITS), which is the agency's main system for tracking employee time charges for billing purposes. Employees enter time charges in RITS on a weekly basis for hours worked on various tasks. These tasks are assigned specific Technical Assignment Control numbers for tracking purposes.

d The amount of time PMs spent in FY 1997 on controlled correspondence and petitions filed by members of the public under Section 2.206 of NRC's regulations, was directly related to the plant a PM was assigned and varies considerably. Some PMs had no such activities.

OlG M A-21 Page 4 I

,~- - - - - - - - -

Review of the Project Manager Position in NRR Table 1: Variation in Lead PM Time Chargos (excluding absence)

Percenf Hours Description Min Max Avg StdDev" Min Max Avg StdDev Licensing Ac'. ions and Activity 12.1 84.3 39.5 15.8 231.0 1911.0 681.4 331.2 Focal Point 6.2 71.2 37.0 14.8 86.0 1,503.0 641.7 287.2 Administrative 1.1 46.8 13.6 8.9 23.0 804.0 232.8 157.8 Plant Safety Assessment 0 27.3 7.6 6.2 0 518.0 132.4 110.3 Generic, Non-Docket Related 0 38.2 2.3 5.7 0 701.0 41.7 102.8

• Totals shown are percentages of a PM's total hours less time charged to absence.

" Standard Deviation provides a measure of the spread or dispersion of values around the average and is the most usefulindicator of variation. The standard deviations of the time charges. which are large in relation to the averages, reflect the wide vanance in the hours charged to functional areas across all lead PMs.

Source: OlG analysis of FY 1997 data from WISP for 64 lead PMs (as identified by OlG).

We believe that the PM function will continue to absorb various unrelated duties until a clear role is identified and a focus is given to this position. In our opinion, NRC management needs to: (1) reach a consensus on the expectation.s of the PM position, including determining how the position fits into the overall picture of licensee safety assessment, and (2) communicate the unified expectations to the staff.

Some PM Activities Need Better Alignment NRC staff, including PMs, expressed concems that being responsible for so many competir.g tasks may inhibit PMs from performing expected activities to the best of their abilities. We found inconsistent performances in two activities PMs are currently responsible for: (1) performance of safety evaluations (SE), and (2) participation in various functional areas of the SALP process. Better alignment of the responsibilities in these two areas might enhance PM effectiveness.

o olG/97A-21 Page5

I Review of the Project Manager Position in NRR l

The ADPR organization is responsible for coordinating licensing and technical review activities, and NRC's Operating Reactor Project Manager's Handbook indicates that a PM is expected to be able to prepare SE reports. However, we found management's expectations regarding a PM's performance of SE reviews are mixed. For example, one senior NRR manager stated that PMs do not perform the licensing work, rather they " broker the work and lead the team doing the work."

However, another senior manager said the PM function was created to manage the completion of licensing actions and " personally do about 50 percent of the actions."

PMs said they are not aware of any management expectations on this sdject, so they independently decide whether to perform SEs. Accordingly, we found inconsistencies among PMs related to this activity. In addition, we believe there are inherent weaknesses in performing SE reviews in ADPR which are outlined below:

. ADPR has responsibility for coordinating the completion of SEs, but PMs I

have no delegated authority to ensure the timeliness of SEs being performed in the Associate Director for Technical Review (ADTR) organization, where most SE review time is actually accrued. We identified about 200,000 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> charged to licensing actions and activities by NRR employees in FY 1997 of which only about 14,000 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> were charged by lead PMs for actually performing a review (not including hours we estimated as needed for a PM to manage the review process).* Technical reviewers charged more than 120,000 hours0 days <br />0 hours <br />0 weeks <br />0 months <br /> for performing revic3 in ADTR, which ,

is already tasked to perform such reviews.

. According to the staE we interviewed, the technical capability of PMs to i perform SEs is widely varied. On the other hand, technical reviewers in ADTR have specific functional expertise that can be maintained through daily on-the-job learning or casily identified training.

In our opinion, the performance of fairly specific SEs detracts from a PM's ability and time to focus on the broader issues of a plant's licensing performance and status, and safety performance. Also, NRR should ensure that management responsibilities are restricted to as few reporting lines as possible to best exercise quality control. Performing SEs within a single office would increase the effectiveness of this important quality control and assure isolation of responsibility.

However, because of the role PMs play in plant assessment and their need to maintain an overall awareness of plant activities and plant license conditions, it is 5

We identified a review as being performed by a PM if a PM charged more than 90 percent of the hours related to that review. Of about 3,400 OlG-identified licensing actions and activities worked on during FY 1997, PMs charged more than 90 percent of the time on about 1,200, requiring an average of about 21 hours2.430556e-4 days <br />0.00583 hours <br />3.472222e-5 weeks <br />7.9905e-6 months <br /> or less each to complete. The balance of the reviews were done in ADTR and required an average of about 60-100 hours to complete depending on priority level.

OlG/97A-21 Page 6 I

- __ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . . _ _ _ _ _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ . . _ _ _ _ _ _ . _ _ . _ . _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . . _ _ _ _ _ _ . _.__.__.______._________.___._.-_.__________._____a

Review of the Project Manager Position in NRR important that PMs are involved in setting the priority and completion date goals for licensing actions and activities. As such, we believe PMs are currently the appropnate focal point for the flow of such requests into and out of NRR Another PM activity that we found may be performed more efficiently with better alignment is the PM's role in the SALP process. NRC is currently reviewing the licensee performance assessment area, including SALP. However, at this time, the SALP process remains in effect and any changes to this or other agency assessment methods are undetermined.

NRC uses the SALP process to articulate its observations and insights on a licensee's safety performance to licensee management and to the public. The PM's role in the process is primarily to support the NRR SALP Board

• members as a coordinator for the functional areas assigned to NRR.

SALP reports generally consist of the assessment of four functional areas for operating reactors: (1) Plant Operations, (2) Maintenance, (3) Engineering, and (4) Plant Support. Regional Office officials determine which of these functional areas NRR will coordinate. However, we found the a,ea designations to NRR are not consistent across Regions, and we found no documented rationale for the functional assignments. For example, NRR PDs monitoring plants in the four Regions have four different SALP responsibilities and the assignments from any Rogion can change from SALP to SALP. NRC staff, including PMs, told us that unless PMs had prior work experience in the designated SALP area, for instance as a resident inspector, they would typically have to contact resident inspectors and/or Regional Office staff to obtain the additional information needed to provide input. Senior NRC officials and PMs indicated that this additional effort would e ssentially be duplicating information that already exists in each Region.

We believe the role and responsibilities of the PMs should be used to identify the most appropriate input area to a SALP, or any other performance assessment report, and that area should be consistent for all Project Directorates and Regions.

Based on the current role of PMs, we agree with NRC staff who told us PMs would most appropriately participate in the SALP, or other safety assessments, by commenting on a licensee's performance in licensing.m The SALP iocation designated for commenting on a licensee's licensing performance is the Engineedag module. NRR officials told us that they are currently making an effort to increase the amount of information that PMs place on the docket in regard to The SALP Board includes SES-level representatives from the responsible Region and an NRR SES-level manager from ADPR. Regional Administrators have responsibility for developing and issuing SALP reports.

7 Particularly, the quality, timeliness, and adequacy of a licensee's requests for and actions taken related, for example, to license amendment requests, requests for relief, and requests for exemptions.

OlG/97A-21 Page 7

Review of the Project Manager Position in NRR license activity, which will allow such material to be included in future SALP re, ports.

We encourage NRR to continue this effort.

IMPROVEMENTS ^

NEEDED IN TRAINING OVERSIGHT I

\

We reviewed NRR's current training requirements for PMs and discussed the training program with senior NRC officials and PMs. Although NRR has established training requirements, we found that management views on PM training differ and E oversight of PMs' training is lacking and needs improvement. g NRR's training requirements for PMs are detailed in a May 1989 memorandum to NRR staff. The memorandum states that supervisors are to identify courses each PM should take to meet the job requirements and those courses become the formal training plan for that employee. Furthermore, NRC Management Directive 10.77, Employee Development and Training, dated August 27,1996, directs supervisors to meet at least semi-annually with staff members to ensure that all have completed the training specified in formal training and qualification programs. However, PMs told us that their supervisors had not established training plans as required in the 1989 memorandum, nor had any PMs been requested to develop their own plans.

We also learned that most of the PDs do not conduct the required semi-annual meetings with t!ieir PMs to discuss training needs and that they rely on the PMs to request desired training classes.

The 1989 memo mandates that PMs receive a rmimum amount of technical training for each type of plant they are assigned to. A senior NRR official we spoke with emphasized that job assignments should still be matched to the type of plants for which PMs have received training; for example, a Pressurized Water Reactor-trained PM should not be assigned to a Boiling Water Reactor plant. However, in February 1997, an ADPR Division reviewed the training records for its PMs to determine if PMs were meeting the technicaltraining requirements of the position. l The review found that about 80 percent of the PMs needed additional training to E comply with NRR's 1989 guidance. The review also found that 15 percent of the PMs had no technical training related to the type of plant for which they were the lead PM. In addition, our review found that other ADPR PMs have not received formal training on the type of plants to which they are assigned. Furthermore, NRR managers stated, and the 1989 memorandum provides, that PMs should g periodically receive refresher training on their assigned plant types. However, none E of the PMs we interviewed had taken refresher courses of any type.

The NRC's Operating Reactor Project Mbuager's Handbook indicates that a PM must have a high level of technical knowledge to perform and manage quality technical reviews. PMs participate in and lead tech:iical discussions, formulate 3 overall technical judgments, and write technical reports. PMs must be aole to g olG/97A-21 Page 8 I

Review of the Project Manager Position in NRR correlate the many facets of nuclear and conventional technology that may influence the operation and design of a nuclear facility.

Senior NRR officials told us that training is an important component of PMs' ability to perform theirjobs and that they are committed to the development of a technical training program to ensure that all PMs receive the training necessary to meet established requirements. In fact, a formal PM qualifications system, similar to the NRC Inspector Qualification program, is under consideration and favored by many of the staff we spoke with. However, although DMs and PDs we interviewed generally agreed that technical training is important, several said PMs cannot afford the time away from the office because the work backlogs during their absence. On occasion, PMs have rescheduled or canceled training due to more pressing work assignnients. Conversely, a few PMs and PDs told us that formal training may not even be necessary, as PMs have gained technical knowledge and expertise through years of on-the-job training (OJT).

While we acknowledge that PMs benefit from OJT, we agree with NRC staff who told us that formal training is needed in addition to OJT, and we do not believe NRC has assured itself that OJT is a sufficient substitute for formal training. We also recognize that the outcomes of the JTA and other initiatives may be used to revise existing training requirements and add newly identified areas to a formal training program. However, regardless of any changes to the program that may result from the JTA or other initiatives, management needs to provide the time and staff support necessary to ensure PMs are adequately trained in areas identified as critical to the PM's job function.

RECURRENT REQUESTS ADD TO PROCESSING TIME NRC staff, including PMs, told us that NRR needs to be more responsive to licensees and Regional Office staff needs by improving the timeliness of completing licensing actions and activities. We examined the processing of certain types of licensing actions as part of our review of factors that impact the effectiveness of PMs. We found that the recurrent requests for additional information (RAls) appear to be the primary impediment to shortening the amount of time it takes to complete licensing actions and activities.

To augment our review, we extracted FY 1997 data from RITS and WISP to analyze how Priority 1 and Priority 2 licensing actions ar.d activities moved b3 tween ADPR and ADTR personnel. We also reviewed the pattem of time charges occurring over the period these actions were in process and found that personnel in both organizations charged small amounts of time per week over the course of each review. This resulted in actions takin'g long periods of time to complete in comparison with the relatively few hours actually spent performing the reviews, as illustrated in Table 2 below.

otG/97A 21 Page 9

I Review of the Project Manager Position in NRR Table 2: Time to Complete Technical Review Requests * ,

I Average Average Average Number Weeks to Hours to Hours Type of Request Priority Completed Complete Complete 6 Per Week License Amendments 1 58 9 94 11 Technical Assistance for a Regional Office 1 20 16 78 5 License Amendments 2 110 24 176 7 Technical Assistance for a Regional Office 2 24 28 47 2

• Totals are adjusted to account for multi-unit requests where multiple tracking numbers were opened for what was essentially a single review. All numbers are rounded.

• This category includes all hcurs worked by all NRR employees who charged time to the task.

Source: OlG analysis of FY 1997 data from WISP and RITS.

To gain a better understanding of the licensing process, we selected a judgmental sample of nine Priority 1 license amendment requests and TIAs<e). We discussed the processing of each of these actions with the technical review staff and the PM(s) involved and found that the recurrent need to request additional information impacted the timely processing of each of these actions. That is, in the reviewer's professional judgment, the original submittal did not contain sufficient information for the reviewer to perform the evaluation. To obtain the additional information deemed necessary by the applicable reviewer, a request, such as an RAI, was issued to the appropriate licensee or Regional Office staff. The PMs and technical reviewers told us that work virtually stops on actions pending receipt of each RAI response, which can take a week to several manths, impacting the PMs' effectiveness in scheduling and performing the licensing portion of theirjobs.

PMs told us that they generally perform reviews having historical precedents; however, they stated that approximately 50 percent or more of even those reviews required RAls. On the other hand, technical reviewers said that one or more RAls are required for nearly every request they receive in ADTR. Staff felt that NRC's

Task Interface Agreements (TIA) are Hegional Office requests for headquarters technical expertise and assistance, olG/97A-21 Page 10 I

Review of the Project Manager Position in NRR f more conservative approach over the past few years has resulted in an increase in formal RAls that previously might have been requested verbally. Staff also provided these additional reasons for RAls:

The quality of submittals varies among licensees; some licensees submit very complete requestsTnd others frequently submit insufficient requests. For example, some concern was expressed that licensees may submit " skeleton" requests because they want NRC, in an RAI, to provide them with specific information needed to complete the review; Licensees are being conservative in submitting information to NRC that may be docketed ar.d become accessible to Public Ltility Commissions and/or subject to additional NRC regulation; Licensees may not have the necessary level of technical expertise in a specific area; and, Two parties dealing with complex issues have difficulty anticipating what the other needs for such a review without seeing the request in written form.

PMs and ADTR staff generally stated that RAls did not take any significant amount of time to prepare; therefore, gains in this area would primarily be in shortening the length of processing time rather than in significant reductions in staff hours consumed. Senior managers told us that NRR may shortly conduct its own review of this area, either intemally or through the contracted assessment.

We believe that identifying ways to alleviate the neeo for recurrent RAls will improve the timeliness of processing licensing actions and activities, thereby creating an environment where PMs can more effectively perform their licensing role and NRR can be more responsive to the needs of licensees and Regional Office staff.

CONCLUSION From our discussions with NRC senior officials and other staff, we found that the expectations of the PM posit'.:n vary considerably. We also found wide variance among PMs in the amounts of time charged to those activities that management considered most important. The diverse expectations of agency officisk indicate a lack of clarity in PM position priorities and imply that there are unclear responsibilities for performing activities.

OlGJ97A-21 Page 11

l i

Review of the Project Manager Position in NRR j i

As a result of our review of factors affecting the PM workload, we believe that a lack i of clearly defined expectations has led to inconsistencies in the performance of two current PM activities: performing safety evaluations, and providing input to various sections of the SALP reports. In our opinion, the responsibility for performing all SEs should reside within the ADTR organization. We also believe that the PM's role in the SALP process lacks focus and consistency. For any position in NRC that has a role in the safety assessment of a nuclear power plant, the need for clear identification and justification of that position's input is critical to its value. We believe PMs would most appropriately participate in the SALP, or other safety assessments, by commenting on a licensee's performance in licensing.

In addition, although NRC has identified the types of training needed by PMs and issued related guidance, we found that this guidance is not being followed and that management support and attention to PM training is lacking.

Finally, our review of how licensing actions are processed identified RAls as the primary impediment to the completion of licensing actions.

RECOMMENDATIONS '

We recognize that NRC may, as a result of its current agency-wide initiative to implement a perforrnance management process, conduct a " top down" assessment of all the programs and activities in NRR. That assessment would examine NRR's current programs and activities to determine whether they achieve their desired purpose in an efficient and effective manner and whether they should be continued, redesigned, or sunset. However, the performance nf that assessment is uncertain at this date. Therefore, to ensure that NRR's Projec Manager position has a clear role and focus, we recommend that the agency:

(1) Re-evaluate and clearly define expecifions and responsibilities, including participation in the licensee assessment process, for Project Managers based on how such a position fits into NRR's mission to ensure public health and safety through licensing and inspection activities, and communicate the expectations to Headquarters and Regional Office staff.

(2) Follow training guidance and ensure Project Mariagers are adequately trained in critical job functions. Take appropriate action to assure adequate management support and oversight of the current training requirements for Project Managers, or the requirements as modified by the JTA and other on-going initiatives.

olG/97A-21 Page 12 I

l 1

)

Review of the Project Manager Position in NRR l

1 (3) Relocate responsibility for the performance of all technical reviews to ADTR and establish Project Managers as the focal point for movement of those reviews into and out of NRR.

Additionally, NRC might identify ways to improve the effectiveness of PMs in their licensing role by examining factors causing recurrent Requests for Additional Information.

AGENCY COMMENTS The Deputy Executive Director for Regulatory Programs stated that NRC staff generally agree with the findings and recommendations of this report, but disagreed with our recommendation to relocate responsibility for the performance of all technical reviews to ADTR and establish PMs as the focal point for the movement of those reviews into and out of NRR. His comments are contained in Appendix IV, We have incorporated editorial changes in our final report where appropriate.

Subsequent to receiving NRC's con $ments, we met with senior agency staff, including the Deputy Executive Director for Regulatory Programs. Staff agreed that measures are needed to correct the weaknesses we reported, and stated they plan to initiate several actions to improve the quality of work and productivity by .

examining ways to reduce inefficiencies in work flow between ADPR and ADTR.

NRC staff agreed to provide OlG with schedules for implementing these actions. I Given the need for and importance of improving NRR's regulatory processes, we l l plan to closely moni'or the agency's progress in implementing these initiatives.

)

I i

l l

l i

i i

OlG/97A-21 Page 13 l

Appendix l Review of the Project Manager Position in NRR

}

I OBJECTIVES, SCOPE, AND METHODOLOGY The objectives of our review were to (1) identify and examine the role of Project Managers in the Office of Nuclear Reactor Regulation (NRR), and (2) examine I factors affecting the workload of Project Managers.

To accomplish our objectives, we interviewed senior U.S. Nuclear Regulatory Commission (NRC) officials and staff in NRR, including Project Managers and technical reviewers, as well as two NRC Deputy Executive Directors for Operations.

We also interviewed Regional and Deputy Regional Administrators, Division and Deputy Division Directors, Resident inspectors, and other staff members in Regions I and IV. In addition, we met with representatives of the Nuclear Energy institute, the nuclear industry's Washington-based policy institute.

We conducted data analysis on information from the agency's Workload Information and Scheduling Program (WISP) and Regulatory Information Tracking System (RITS). This information encompassed the time charged by NRR staff, including Project Managers, to various activities during fiscal year 1997 (weekending October 5,1996, through weekending September 27,1997). We performed a quality review of WISP data on a statistical sample.W Based on work performed in an earlier Office of the inspector General report,<2) we determined that data from RITS was reliable.

We reviewed relevant management controls, including various policies and procedures, NRC Management Directives, the NRC's Operating Reactor Project Manager's Handbook, and other guidance documents in addition to discussing controls with NRC management officials.

We conducted our audit from July 1997 through April 1998 in accordance with generally accepted Government auditing standards.

We reviewed a total of 78 records using (1) expected error rate of 5 percent, (2) confidence level of 90 percent, and (3) upper error limit of 10 percent. We checked a total of 13 data fields. Those results indicated that we would have to avoid use of cer.'ain WISP data fields and exercise caution with the use of other fields. We made those adjustments in our work and informed NRR officials of the results of our quality review.

O\Cl96A-09, September 13,1996, Review of NRC's Part 170 License Fee Billing System.

OlG/97A.21 Page i of 1

Appendix il Review of the Project Manager Position in NRR I

Oflice of Nuclear Reactor Regulation l NRR l

i i i i I

~D< vision of Reactor l ' Associate Directorfor ' Associate Director for ' l Division ofInspection i Program Projects TechnicalReview and Support i Management l ADPR ADTR Programs i

l l

Division of Division of Systems Division of Reactor !

n o s an Engineering SafetyandAnatsis l

i !i i

Division ofReactorProjects(East) Division ofReactorProjects(West) '

Tecnnical Specifications Branch i' Regions I, II Regions III,IV

,  ; zg9 '

DRPE i , DRPW  : t I

i '

Project Directorates ProjectDirectorates (6ProjectDirectorsPD East) (5ProjectDirectorsPD West) i  !

NRR Organization Chart l ProjectManagers nsem am appximateW5 PMs Project Managers { Headquarters) l PM withm both Dmsms PM ,

i i

O m 7,r21 p,g, , ,, 3

l Appendix lil Review of the Project Manager Position in NRR l

l ACRONYMS i

ADPR -

Associate Director for Projects organization ADTR -

Associate Director for Technical Review organization I

r6AR -

Final Safety Analysis Report FY -

Fiscal Year JTA -

Job Task and Functional Analysis NRC -

Nuclear Regulatory Commission NRR -

Office of Nuclear Reactor Regulation i

OlG -

Office of the Inspector General l

l OJT -

On-the-Job Training PD -

Project Directors PM -

Project Manager RAI -

Request for Additional Information RITS -

Regulatory Information Tracking System SALP -

Systematic Assetsment of Licensee Performance SE -

Safety Evaluation TIA -

Task Interface Agreements WISP -

Workload Information and Scheduling Program OlG/97A-21 Page 1 of 1 l

Appendix IV _

Review of the Project Manager Position in NRR ps ato o p 4 UNITED STATES j

j e

NUCLEAR REGULATORY COMMISSION WASHINGTON, o.C. 20565-0001

'k..,g. /

i June 1, 1998 3 i

MEMORANDUM TO: Th amas J., Barchi Assistant inspector General'for Audits FROM:

H.ign L. Thompson, Eeputy Executive Director fo Regulatory P kM Jr. h'rQ -\

rogrfnc,

SUBJECT:

REVIEW OF DRAFT AUDIT EPORT - REVIEW OF THE PROJECT MANAGER POSITION IN NRR l

1 am responding to your memorandum of April 28,1998, transmitting the subject draft audit rcport. The staff generally agrees with the findings in the draft audit report and will incorporate l related corrective actions into ongoir.g and planned efforts with the goal of improving the quality l

of work and productivity of project managers (PMs) ano other NRR personnel. We wil provide you with schedules associated with our corrective actions in our response to the final OlG audit raport. With respect to your specific recommendations, I submit the following comments:

Recommendation 1 Re-evaluate and clearly define expectations and responsibilities, including participatic 7 in the licensee assessment process, for Project Managers based on how such a position fits into NRR's mission to ensure public health and safety through licensing and inspection activities, and I communicate the expectations to Headquarters and Regional Office staff.

1 BJLsoonse l Agree. The findings of your audit are consistent with other assessments and our current understanding of the problems in this area. An assessment of respo'nsibilities, roles and perceived contributions to traditional PM assignments is an important part of the ongoing Job Task and Functional Analysis (JTA). Regarding the licensee assessment process, the roles and responsibilities of staff and management positions are being addressed in our integrated Review of the Reactor Assessment Process.

1 Recommendation 2 L Follow training guidance and ensure Project Managers are adequately trained in criticaljob

! functions. Take appropriate action to assure adequate management support and oversight of the current training requirements for Project Managers, or the requirements as modified by the JTA and other on-going initiatives.

OlG/97A 21 - Page 1 of 5 l

l

-l Appendix IV j l

j Review of the Project Manager Position in NRR 2-Resoonse Agree. The findings of your audit are consistent with other assessments and our current understanding of the problems in this area. The development of an improved training program for NRR PMs will be pursued following the completion of the JTA. Some interim actions are being implemented to address short-term concems or specific problems. The referenced guidance issued in 1989 will be re-evaluated in light of the JTA results. Although I am in general agreement with this recommendation, it should be recognized that on-the-job-training (OJT) will likely be an integral part of the training program following the JTA. The appropriate balance between OJT, classroom training, and improved guidance documents will be determined following the JTA and will be combined with the NRC's continuing training program initiatives in areas such as probabilistic risk analysis. The needs of individuals for specific technical training will be closely assessed.

Escommendation 3 Relocate responsibility for the performance of all technical reviews to ADTR and establish Project Managers as the focal point for the movement of those reviews into and out of NRR.

Resoonse Disagree. The findings of the audit conflict with longstanding management expectations. Your recommendation is based largely on the perception that inconsistent management expectations E are being communicated regarding the preparation of safety evaluations by PMs. NRR Ofrice E Letter No. 803, Revision 1, " Technical Specifications Review Procedures," requires that a work plan be developed for each amendment request and the associated guidance document, " Guide for Processing License Amendments," includes a logic model to help determine the most appropriate review method (PM lead with technicz! branch concurrence, technical branch lead, or con bined effort). I believe that a more appropriate interpretation of your finding of inconsistent approaches to preparing safety evaluations would be that this problem is another example of our need to improve the training for and communication of management expectations to PMs (Recommendation 2).

Conrams expresud about the technical capability of PMs and quality control of the safety evaluations for specific license amendments are addressed in the office letter and associated guidance document. Project Managers are expected to have significant technical expertise in the areas cf operating reactor designs and operations as well as having expertise in one or more related engineering disciplines. This knowledge is required to' perform the routine PM functions and interactions with licensees, regional personnel, and other staff and management within NRR.

In addition, the audit acknowledges that licensing actions perfomled by PMs are completed at a lower labor rate than the balance of the actions. Therefore,' I feel that it is a logical use of cvailable resources and technieri expertise to have PMs perform technical reviews and prepare safety evaluations for some licensing actions.

- ~.ms g

_ 'l

Appendix IV

. Review of the Project Manager Position in NRR 3-Recommendation /Suaaestion 1

Additionally, NRC might identify ways to improve the effectiveness of PMs in their licensing role by examining factors causing recurrent Requests for AdditionalInformation (RAls).

Response

The findings of your audit are generally consistent with our current understanding of the problems in this area. Several ongoing or planned program reviews may provide an opportunity for the staff to identify means of reducing delays associated with issuing and awaiting responses to RAls for some licensing actions and activities. Given that the large majority of RAls are generated by technical branches and that the PMs' role is to facilitzte transmittal and discussion of the questions with licensees, it is likely that any significant reduction in the need for RAls will depend on the issua.nce of improved technical review criteria, improved guidance for issuance of generic communications, and other measures outside of the direct control of an individual PM or the projects organizations in NRR.

Attached are additional comments on your draft audit report that were generated during the staff's review .

Attachment:

Additional Comments OlG/97A-21 Page 3 of 5

Appedix IV Review of the Project Manager Position in NRR Ac'ditional Comments on Draft Audit Reoort " Review of the Project Manaaer Position in NRR" Pages i and 2 The evolution of the " Operating Reactor PM" position has not invcived the g project management functions associated with research reactors. As the g number of commercial power plants seeking an operating license decreased, the " Licensing PMs" were moved from dedicated organizations into the same project organizations as the " Operating Reactor PMs" assigned to the licensed facilities. Reference to a merger of licensing functione may be misleading in that the role of " Operating Reactor PMs" was not changed significantly as a result of the reorganizations. However, the 1987 reorganization of the offices of Nuclear Reactor Regulation and Inspection and Enforcement expanded the role of the PM into the areas of inspection and assessment of licensee performance. Accordingly, given g the variation of licensee operational performance, PMs on facilities with 3 few problems are able to focus on the completion of licensing actions, while other PMs may need to assist the regional office in the assessment E of licensee performance (e.g., facilities on the Watch List). This flexibility E, is expected of PMs and would lead to a disparity in the tasks PMs support at a given point in time. It would also lead to a disparity in the expectations of HQ and regional managers.

Page 2, footnote 2 Although the review of responses to generic letters and bulletins is classified as a licensing activity, the initial development and issuance of g the generic communications are not generally included in the category of g licensing activities.

Page 4 it is questionable whetbar the need to request additional information from NRC staffin the regional offices routinely impedes the timely completion of licensing actions. It is likely that the sample of Task Interface Agreements (TIAs) you selected for review resulted in this misperception. The development of a response to inquiries from the regional offices (tracked within the category of licensing activities) often requires NRR personnel to request additional information regarding the subject events or equipment. g Waiting for such clarifications from the regional personnel requesting R assistance is usually not a significant factor in the timeliness of NRR's response to TIAs. In addition, the disposition of the majority of licensing g actions and licensing activities does not involve the solicitation of g information from regional personnel.

Page 7 Although the variety and number of tasks assigned to oms obviously affect the PMs' tinaliness in processing licensing actions and licensing activities and may therefore contribute to the size of the licansing backlog, the measures of the effectiveness of PMs are more complex than tracking the E' inventory of licensing actions and activities. The continuing development E of the operating plan for the reactor licensing program will address the relationship between the completion of licensir.g actions and activities, the 3 control of related backlogs, and the performance of other duties in g j measuring the effectiveness of te project managerrent functions.

l OlG/97A-21 p,g,4 og $

ATTACHMENT

Appendix IV Review of the Project Manager Position in NRR Appendix 11 The note that reads "There are approximately 75 PMs within both Directorates" should be changed to "There are approximately 75 PMs within both Divisions" ATTACHMENT 01G/97A-21 Page 5 of 5

Appendix V Review of the Project Manager Position in NRR MAJOR CONTRIBUTORS TO THIS REPORT William D. McDowell Team Leader Robert W. Moody Senior Auditor Catherine M. Colleli Management Analyst OfG/97A 21 Page 1 of 1

Appendix VI Revi w of the Project Manag r Position in NRR I

GLOSSARY: OFFICE OF THE INSPECTOR GENERAL PRODUCTS I INVESTIGATIVE

1. INVESTIGATIVE REPORT- WHITE COVER An Investigative Report documents partinent facts of a case and describes available evidence r: levant to allegations against individuals, including aspects of an allegation not substantiated.

Investigative reports do not recommend disciplinary action against individual employees. Investigative r ports are sensitive documents and contain information subject to the Privacy Act restrictions. I Reports are given to officials and managers who have a need to know in order to properly determine whether administrative action is warranted. The agency is expected to advise the OlG within 90 days of receiving the investigative report as to what disciplinary or other action has been taken in response to investigative report ondings.

2. EVENTINQUIR> - GREEN COVER The Event Inquiry is an investigative product that documents the examination of events or agency cctions that do not focus specifically on individual misconduct. These reports identify institutional weaknesses that led to or allowed a problem to occur. The agency ic rquested to advise the OlG of managesial initiatives taken in response to issues identified in these reports but tracking its r; commendations is not required.

3. MANAGEMENTIMPLICATIONS REPORT (MIR)- MEMORANDUM MIRs provide a " ROOT CAUSE" analysis sufficient for managers to facilitate correction of problems and tc avoid similar issues in the future. Agency tracking of recommendations is not required.

Auoli l i

4. AUDITREPORT- BLUE COVER An Audit Report is the documentation of the review, recommendations, and findings resulting from ,

en objective assessment of a program, function, or activity. Audits follow a defined procedure that  !

cllows for agency review and comment on draft audit reports. The audit results are also reported in '

the OlG's " Semiannual Report" to the Congress. Tracking of audit report recommendations and i egency response is required.

5. SPECIAL EVALUATION REPORT- BURGUNDY COVER A Special Evaluation Report documents the results of short-term, limited assessments, it provides en initial, quick response to a question or issue, and data to determine whether an in-depth O indepeMent audit should be planned. Agency tracking of recommendations is not required.

REGULATORY

6. REGULATORY COMMENTARY- BROWN COVER Regulatory Commentary 3 the review of existing and proposed legislation, regulations, and policies so as to assist the agency in preventing and detecting fraud, waste, and abuse in programs and operations. Commentaries cite the IG Act as authority for the review, state the specific law, rcgulation or policy examined, pertinent background information considered and identifies OlG concems, observations, and objections. Significant observations regarding action or inaction by the agency are reported in the OlG Semiannual Report to Congress. Each report indicates whether a response is required.

OlG/97A-21 Page 1 of 1