ML20136G222
| ML20136G222 | |
| Person / Time | |
|---|---|
| Issue date: | 03/31/1984 |
| From: | Messenger G NRC OFFICE OF INSPECTOR & AUDITOR (OIA) |
| To: | |
| Shared Package | |
| ML20136G145 | List: |
| References | |
| FOIA-85-715 NUDOCS 8511220283 | |
| Download: ML20136G222 (46) | |
Text
___
p.mm'c J
[ 't >.~: ( (,i Usg~~.)/N United States Nuclear Regulatory Commission Report to the Commission Review of the Committee to Review Generic Requirements i
l l
l Office of Inspector and Auditor l
March 1984 NOTICE Tnis is An Internal Management Document l
Prepared For The Commission Not to Be Released Outside Of NRC l
r P
12 3 851113 CR ADER 85-713 ppg i
I M
Do UNITED STATES 8'
3, NUCLEAR REGULATORY COMMISSION h
WASHINGTON D. C. 20655
%,,, g, March 30,1984 MEMORANDUM FOR:
Chairman Palladino Comissioner Gilinsky Commissioner Roberts Comissioner Asselstine Commissioner Bernt c.
ec
-C FROM:
Gear e 1.
ssenge'r, tingDi(edor Office of_ Inspector and Auditor
SUBJECT:
01A REVIEW 0F THE COMMITTEE TO REVIEW GENERIC _ REQUIREMENTS Attached is our report on the results of our audit of the Committee to Review Generic Requirements (CRGR).
The purpose of our audit initially was to examine the implementation of the CRGR Charter.
As our review progressed, however, we expanded the scope to include a review of CRGR's effectiveness in controlling the number and nature of requirements imposed by NRC on licensees.
Our audit was performed during the period from mid March through July 1983, and included interviews with the Deputy Executive Director for Regional Operations and Generic Requirements (DEDR0GR), each CRGR member, each member of the DEDR0GR staff, top level officials in the Offices of Nuclear Reactor Regulation (NRR), Nuclear Material Safety and Safeguards (NMSS), and Inspection and Enforcement (IE), and officials in the Office of Resource Management (RM). We also reviewed correspondence and other documentation relating to the establishment of the CRGR and the CRGR's review of proposed new requirements.
Further, we attended several CRGR. meetings to observe the committee's operations.
Following is a summary of the major results of our review.
IMPLEMENTATION OF CRGR CHARTE_R Our review disclosed that, in general, the CRGR Charter is being implemented.
However, we identified four areas in the Charter not being fully implement'ed, and one area in which the CRGR, through the DEDR0GR staff, was exceeding the provisions of the Charter. These five areas are:
Acceptance Reviews According to~the CRGR Charter, DEDR0GR's staff is supposed to review all CRGR requirement packages to determine if they are sufficient for CRGR members' consideration.
If a package is not sufficient, DEDR0GR is supposed to return CONTACT:
Fred Herr, 01A Peter Sicilia, 0!A 49-27051
The Commission 2
the package to the originating office.
In accordance with the Charter, DEDR0GR's staff does review all requirement packages for sufficiency.
However, even though the staff believes most packages have not been sufficient for CRGR review, all but two packages have gone to CRGR for its review and consideration rather than being sent back to the program offices' to be revised.
Quantitative Analysis The CRGR Charter's operating procedures ' state that each CRGR package should include quantitative analysis, including risk reduction and cost assessments to NRC and licensees.. Additionally, each packag'e should include a regulatory analysis sufficient to address the Regulatory Flexibility Act and Executive Order 12291.
DEDR0GR staff member,s told us many packages sent to the CRGR proposing new generic requirements;do not contain any quantitative analysis, and for those that do, most are considered insufficient and therefore are unacceptable to the staff. DEDROGR staff members told us they have been quite lenient in accepting requirement packages that contain less than adpauate or no quantitative analysis.
While we were told many reasons why quantitative analyses are not prepared, the reason most often mentioned was the lack of an adequate data base to quantify the value and impact of their requirements.
During our review, we found several groups exploring ways to assess the cost of proposed new requirements and developing criteria and techniques to aid in analyzing costs.
Further, we found several groups letting contracts for the development of cost / benefit methods and models, and for providing assistance to CRGR and program offices in developing and reviewing cost / benefit analyses.
We also found that while the Cost Assessment Group (CAG) was established to serve as the agency's central and independent office for performing objective, quantitative cost analyses for all new requirements, it has not relieved the program offices of the burden of estimating costs for regulatory analysis.
DEDR0GR Staff " Audits" of Activities The original CRGR Charter stated that the CRGR would review "all proposed new generic requirements to be' imposed by the NRC staff."
In May-1983, the CRGR Charter was amended; which resulted in a revision to the Committee's scope of operations.
As amended, the CRGR Charter's scope section states, "CRGR will consider all proposed new requirements to be imposed by the NRC staff...that i
are submitted for review by the offices."
(Change Underlined). The implication clearly is that the CRGR would review only those items referred to it.
i During our review, however, we found that the DEDROGR staff always has, and we were told, will continue to." audit" the agency's activities in an effort to ensure that applicable generic issues are referred to CRGR for review. The staff told us their " audits" catch a lot of issues that may have qcne out without CRGR's review, and many itens that may not have otherwise been referred to CRGR. However, the " audits" cover only about five percent of the 1
agency's actions.
l The Commission 3
Minutes of CRGR Meetings According to the CRGR Charter, minutes will be prepared for each CRGR meeting, will accurately reflect the consensus decision of the committee, will include CRGR recommendations, and will give an accurate description of the basis for each recommendation.
We found that CRGR does not prepare minutes, as such, of their meetings.
Instead, the OEDROGR staff prepares meeting summaries after the fact, based on their understanding of what transpires at the meetings. The summaries do not identify individual CRGR member comments or opinions, but present a single position regarding the acceptability of the proposed requirement. Also, because votes are not taken at most CRGR meetings, it is often not clear, even to the members themselves, what the Committee's consensus of opinion is for some' requirements.
Further, the positions included in some summaries are based, in part, on discussions between the DEDR0GR and individual members that take place after CRGR meetings.
While it is the CRGR members' responsibility to assure that the summaries accurately reflect their views, members take the time to dissent only in the most extreme cases.
Staff Assistance The Charter states that DEDROGR will provide staff support to the Committee.
DEDR0GR's staff members told us they perform a CRGR staff support role and assist CRGR members by preparing summary analyses of CRGR related packages.
They told us these reports provide the Committee members with " insights" to packages' completeness, quality, adequacy and sufficiency regarding the issues they must review.
However, CRGR members told us they seldom used the staff's summary analysis reports because they did not receive them early enough prior to CRGR meetings.
During our exit conference with the DEDR0GR, his Staff Director told us a new policy has been established requiring the staff's summary to be issued to CRGR members three days before the meetings.
Conclusion Our review identified four areas in the CRGR Charter not being fully imple-mented. For the most part, two of these areas -- minutes, and staff assist-ance -- probably do not significantly impact the effectiveness of the CRGR.
We believe, however, that changes should be made to the process for recording minutes of CRGR meetings. The new policy regarding providing staff assistance to CRGR members is satisfactory subject to its documentation and continued implementa tion. The other two areas -- acceptance reviews and quantitative analysis -- may be adversely impacting the ability of the CRGk to do the job anticipated when it was established.
As a result, we believe the quality of program office proposals to the CRGR should be upgraded, and inadequate proposals should be returned to the program offices. We recognize, however, that before this can be done the criteria for defining completeness and sufficiency must be clearly defined.
i
\\7
- The Commission 4
Our review also identified one area in which the CRGR, through the DEDR0GR staff, wasiexceeding the provisions in the Charter -- DEDR0GR staff " audits."
.We believe 'these " audits" are necessary at the present time to assure the
. effectiveness of the CRGR and should continue, although t e Charter should be h
changed to specifically provide for the audits.
EFFECTIVENESS OF THE CRGR Our review disclosed the CRGR has added discipline to NRC's regulatory 5,
process. CRGR has created a single point of control for NRC actions governing generic issues and is one of the few places where a comprehensive judgement can be made of the technical and regulatory merits of generic issues.
CRGR forces the staff to go through a reasoning process and think through proposed generic issues in a logical way.
Conversely, our review disclosed that the CRGR has not been totally effective in controlling the application of new generic requirements on licensees because some generic requirements were still being imposed on licensees without.having been reviewed by CRGR.
In our view, it appeared many of these requirements had one thing in common -- they originated in a non-normal way, i.e., they did not start at the lowest levels of the agency as new. generic requirements, but were either directed by the Commission, inherited from another office, or evolved from plant specific requirements.
In general, we believe this problem is the result of the program offices not implementing the procedures each established to assure generic requirements are referred to CRGR.
Con'clusion Because some generic requirements are still being imposed on licensees without CRGR's review, the CRGR has not been totally effective in controlling the imposition of generic requirements on licensees'. The problem does not appear to rest with the CRGR, however, but with the program offices for not referring generic issues to the CRGR. As a result, it appears desirable for the DEDROGR staff to continue " auditing" program offices activities to assure that generic issues are referred to CRGR.
l OBSERVATION ON CRGR OPERATIONS Our review disclosed that the forum for CRGR meetings is very informal, and the format and program for conducting meetings lacks established rules of order and basic, administrative management controls. Program officials and committee members believe administrative discipline and tighter control should be added to CRGR meetings.
Further, they believe the CRGR should refine their procedures, establish limits for the Committee's involvement in the meetings, and add formality to the Committee's discussion and deliberation process.
Furthermore, while CRGR members generally do not have any significant problems with open-meetings, they believe executive sessions should be added to the meeting format. They feel executive sessions would provide an environment in which the CRGR can debate and discuss, in private, the proposed generic issue presented to the CRGR for review.
In their opinion, this process could eliminate false ideas and impressions regarding CRGR's actions that people take with them when they leave CRGR meetings.
The Commission 5
Conclusion Our review identified several aspects of the CRGR's operations which some Committee members and program office staff would like to have changed. While we do not necessarily support'any specific changes to CRGR's operations, we-believe the CRGR should review its own operations to determine what changes are needed.
RECOMMENDATIONS
~
.The report contains seven recommendations intended to resolve the problems identified in this report.
OIA EVALUATION OF ED0 COMMENTS (See Chapter V)
'On January 27,.1984, we provided a draft copy of this report to the EDO to obtain his comments. On February 21, 1984, OIA officials met with DEDR0GR staff officials, at their request, to discuss the draft report. At this meeting DEDR0GR officials stated that while they did not disagree with the content or-substance of our report there were several aspects of the report's language which they felt should be changed to provide better balance.
Subse-quent to this meeting, 01A assessed the nature and character of their comments, and modified several parts of the report to clarify its language.
There were no substantive changes to the report as a result of this meeting.
On March 2,1984, the ED0 formally commented on a draft of this report.
In commenting, the ED0 noted some agency actions which have been taken since our audit work was completed, and made commitments to other actions which we believe, if fully considered and properly implemented, will satisfy the intent of some of our recommendations. We will evaluate the adequacy of the ED0's actions in satisfying the intent of our recommendations when our follow-up audit is performed.
in other cases, the ED0 disagreed with our recommendations, noting that corrective actions are unnecessary because, in his opinion, the problems do not exist or that existing programs adequately address the problem. We disagree with the ED0's response to three recommendations -- la, 2, and 4.
In a
view of the significance of the issues to which these recommendations relate and our continued belief that agency action is needed, these recommendations are being elevated to the Commission for resolution.
~
Our detailed evaluation of the ED0's response is provided in Chapter V.
The ED0's response is included in its entirety in Appendix I.
Attachment:
As stated w
The Commission 6
cc: -W. Dircks, ED0 S. Chilk, SECY H. Plaine, OGC J. Zerbe, PE J. Fouchard, PA C. Kammerer, CA H. Denton, NRR J. Davis, NMSS R. DeYoung, IE T. Murley, RI
~J. O'Reilly, RII J. Keppler, RIII J. Collins, RIV J. Martin, RV V. Stello, DEDR0GR R. Scroggins, ICC R. Fraley, ACRS P. Cotter, ASLBP A. Rosenthal, ASLAP CRGR Members T. Rehm, ED0 R. Minogue, RES
TABLE OF CONTENTS PAGE I INTRODUCTION
1 Background
1 Scope 4
II IMPLEMENTATION OF CRGR CHARTER 5
Quantitative Analysis 8
DEDR0GR Staff " Audits" of Activities 13 Minutes of CRGR Meetings 16 Staff Assistance 18 Conclusions 19 Recommendations 19 III EFFECTIVENESS OF THE CRGR 21 Conclusion 24 Recommendations 24 IV OBSERVATION ON CRGR OPERATIONS 25 Conclusion 27 Recommendation 27 V OIA EVALUATION OF ED0'S COMMENTS 28 APPENDIX I - Agency Comments e
3
l n
I INTRODUCTION
Background
On October 8,1981, the Chairman advised the Executive Director for Operations (EDO) that the Commission had approved a reorganization aimed at, among other things, controlling the number and nature of requirements imposed on licensees by the NRC. To this end, the Committee to Review Generic Requirements (CRGR)*
was established under the EDO to review and recommend to the ED0 approval or disapproval of requirements the staff proposed to place on one or more class of reactors.
The Chairman's memorandum specified that the CRGR was to be chaired by the newly established Deputy Executive Director for Regional Operations and Generic Requirements (DEDR0GR).
In addition to the DEDROGR, the CRGR was to be made up of one representative each from the Offices of Nuclear Reactor Regulation (NRR), Inspection and Enforcement (IE), Nuclear Material Safety and Safeguards (NMSS), Nuclear Regulatory Research (RES), Analysis and Evaluation of Operational Data (AE0D), and the Executive Legal Director (ELD). The members were to be appointed by the EDO. The memorandum specified that the DEDR0GR would provide staff support for the CRGR.
In his October 1981, memorandum, the Chairman indicated that more and better management control over the staff's communication and interaction with licensees was needed to stop the flow of conflicting and inconsistent directives and requests placed on licensees by the staff, and to ensure that all new and amended requirements are prioritized so the appropriate level of attention is given to those issues considered most important to safety.
The Chairman's memorandum stipulated that a single, central point of control at NRC's highest operating level of management was needed:
(1) to review and coordinate the staff's communication with licensees and the requirements they place on licensees through this communication; (2) to manage and control NRC's process for levying reauirements on licensees; and (3) to ensure that all requirements issued or amended do in fact contribute effectively and significantly to the health and safety of the public.
The Chairman's memorandum described the CRGR's objectives and DEDR0GR's authority and responsibilities as the Chairman of the CRGR, and explained how these new functions would be developed.
It stated that the DEDROGR's initial
- The memorandum from the Chairman actually proposes a Generic Requirement Review Conmittee (GRRC), however, the committee was officially named the CRGR by the EDO.
2 task _ was to develop appropriate procedures for implementing and supporting the ED0's management, control, and tracking of the staff's generic comunications with licensees and the requirements they place on reactor licensees. The DEDROGR's subsequent responsibilities were to include a major role in managing comunications with licensees and the requirements put on them.
The Chairman also set out general guidance on the operation of the CRGR that included:
1.
CRGR's reasons for disapproval'of proposed requirements must be provided to the ED0; 2.
Proposing offices must place proposed requirements in either Category 1 (urgent items which must be reviewed by CRGR within two workdays of receipt) or Category 2 -(all non-Category 1 items); and 3
3.
Where adequate data is available, the CRGR is expected to use cost-benefit. analysis and probabilistic risk analysis in its reviews.
On January 28,1982, the ED0 provided to the Comission for approval, in SECY 82-39, a proposed CRGR Charter and a procedure to control the comunication of generic requirements to licensees..However, the Commission did not approve the proposed package. Rather, on March 26, 1982, and again on April 2,1982, the Comission directed the ED0 to amend and incorporate the Comission's specific changes into the Charter.
The ED0 modified the Charter and on April 29, 1982, a revised Charter was submitted to the Comission for 3
approval in SECY 82-39A.
In addition to suggesting changes, the Comission's coments on the draft Charter suggested that NRC develop the ability to do independent cost estimates of proposed new requirements.
SECY 82-39A stated that NRR was d
exploring this area.
In 01A's opinion, one of the more significant changes to the Charter suggested by the Commission was to add the following sentence:
"The CRGR will review only those power reactor safety requirements which (a) the responsible office director has reviewed and approved, or (b) the office director has requested CRGR review, or (c) the review has been directed by the EDO."
The effect of this addition was to limit CRGR's ability to monitor the program offices' compliance with the procedures to control the imposition of generic requirements, and prevent the DEDROGR and the CRGR from asking offices to send issues to the CRGR for review.
On June 16, 1982, the Commission approved the CRGR Charter.
In its approval, however, the Commission, with Comissioner Gilinsky dissenting, removed the above noted paragraph from the Charter. This action opened the door to CRGR's and the DEDROGR staff's monitoring of offices' activities to identify generic issues'which should be sent to the CRGR. This issue will be discussed in more detail in Chapter II.
Briefly, the Charter approved by the Comission consisted of five sections (Purpose, Membership, CRGR Scope, CRGR Operating Procedures, and Reporting Requirements) which set out how CRGR functions.
It also had three
3 attachments:
(1) Appointments to CRGR (a memorandum from the EDO appointing individuals to be CRGR members); (2) Schematic Representation of New Requirements Review (a flow chart of the CRGR review process); and (3) Procedures to Control Comunications of Generic Requirements to Reactor Licensees. Attachment 3 was particularly important because it identified the program offices' responsibility to develop internal procedures to assure that all generic requirements are submitted to the CRGR.
It also identified (1) the principal mechanisms used by the NRC staff to establish or communicate generic requirements, (2) mechanisms often used to interpret generic requirements, and (3) mechanisms sometimes used to comunicate generic requirements.
On May 20, 1983, as a result of a Commission briefing on CRGR in March 1983, the ED0 proposed a revision to the CRGR Charter in SECY 83-200. The major changes in the revised Charter were to:
1.
Specify that the CRGR would review the generic requirements "that are submitted for review by the offices";
2.
Change the standard against which packages submitted for CRGR review are measured, from " incomplete" to "not sufficient for CRGR consideration";
3.
Reassign responsibility for presenting the proposed new requirement to the CRGR from the DEDROGR staff to the proposing office; and l
4.
Clarify the offices' responsibility to assure that any important safety l
information is made available to licensees imediately, even if that is before completion of any proposed or ongoing CRGR reviews.
SECY 83-200 was sent to-the Commission as a " negative consent" item, to be implemented in 10 days if the Commission did not object. According to the Office of the Secretary, two Commissioners commented on the SECY paper on June 2 and June 22, 1983, but the Office of the Secretary has no record of these comments being provided to the ED0 until January 6, 1984. As a result, the changes included in SECY 83-200 were considered by the DEDR0GR staff, during our discussions with them, to be in effect in June 1983.
At the time of our audit, the DEDR0GR had a staff of six professionals. The staff included a Director and five senior members with NRC project manager.ent and inspection backgrounds, who also had specialized experiences with performing quantitative analysis and special projects, and with handling safeguards and Unresolved Safety Issues. All but one of these staff members worked primarily on CRGR issues. The remaining staff member was assigned full time to monitoring NRC's regionalization efforts.
The Director split his time between the administration'and management of both the regionalization and CRGR prngrams and other DEDROGR activities.
As of September 31, 1983, the CRGR had held 47 meetings to consider or discuss 81 issues. Of this number, CRGR recomended that the ED0 approve 8 issues, as written, and 37 issues af ter modifications were made. The CRGR also recommended that two issues not be approved although the E00 later approved one of the issues over the CRGR's objection.
Further, the CRGR either returned or recommended that the ED0 return 14 issues to the sponsoring offices for additional work.
4 Scope We performed our audit during the period from mid March through July 1983.
The purpose of our audit initially was to examine the implementation of the CRGR Charter. As our review progressed, however, we expanded the scope to include a review of CRGR's effectiveness in controlling the number and nature of requirements imposed by NRC on licensees. We did not include a review of
- the backfit program because that program was not finalized during the period of our review.
Our audit was conducted in accordance with generally accepted Government auditing standards and included interviews with the DEDROGR, each CRGR member, each member of the DEDR0GR staff, the directors and other officials of NRR, NMSS and IE, and officials in the Office of Resource Management (RM). We also interviewed the director of the DEDR0GR staff at the time of our audit. He has since left that position and is referred to throughout this report as the former DEDR0GR staff director. We reviewed various correspondence and other documentation relating to the establishment of the CRGR and the CRGR's review of proposed new requirements and attended several CRGR meetings to observe the CRGR's operations.
The detailed findings of our audit are presented in the following three chapters. Chapter II discusses the implementation of the CRGR Charter bi both the DEDR0GR and the program offices. Chapter III discusses the effectiveness of the CRGR in. controlling the imposition of generic reauirements on licensees. Chapter IV provides our observations and observations of CRGR members and office directors on the operations of the CRGR. Chapter V is a detailed analysis of the E00's comments on a draft version of this report.
O l
[
5 II IMPLEMENTATION OF CRGR CHARTF.
Our review disclosed that, in general, the CRGR Charter is being implemented.
Specifically, CRGR focuses primarily on new requirements and considers all proposed new generic requirements that are submitted for review by the program offices. The Committee is chaired by the DEDR0GR and consists of one indi-vidual'each from NRR, IE, NMSS, RES, AE0D, and ELD. Procedures to control the staff's comunication of generic requirements to reactor licensees have been developed by the program offices and CRGR has, for the most part, implemented the Charter's operating procedures. Combined, we believe these efforts collectively meet the Charter requirement to develop means for controlling the number and nature of requirements placed by NRC on licensees.
In certain areas, however, our review disclosed that there are differences between the provisions in the Charter and the staff's activities. These areas generally relate to:
DEDR0GR's acceptance review of proposed generic requirements submitted for CRGR review; the use of quantitative analysis by program offices to support proposed new requirements; DEDROGR staff " audits" of NRC activities; the preparation of meeting minutes; and DEDR0GR staff assistance to CRGR members.
The results of our review in each of these areas follows.
Acceptance Reviews According to the original CRGR Charter, the DEDR0GR staff's role was to review all proposed requirement packages for completeness. As of the June 1983 amendment to the Charter, that role became reviewing packages for sufficiency.
If a package is not' sufficient for CRGR members' consideration, the Charter states DEDROGR will return the package to the originating off. ice with his reasons for such action and he will inform the CRGR members of his actions.
In accordance with the Charter, the DEDR0GR's staff performs reviews that, in effect, are acceptance reviews of proposed generic requirements packages pre-pared by program offices. When a package is received in DEDROGR's office, the staff director assigns the review to an appropriate staff member. Staff members told us they then review the packages for completeness (i.e., does it contain all the categories of information identified in the CRGR Charter) and sufficiency (i.e., is the package of high enough quality).
Staff members informed us that the Charter provides criteria only for completeness reviews in defining the format and content of CRGR packages.
If a package provides the required categories of information it is considered complete by the staff.
6 DEDR0GR staff members said that while the Charter has only recently been amended to reouire reviews for sufficiency, they have always gone beyond the initial Charter's criteria for completeness and have reviewed the quality (i.e., sufficiency) of each CRGR package. However, they said that there is no dccumented criteria as to what governs quality; therefore, they do not have a documented basis or standard for performing this aspect of their acceptance reviews. As an alternative, the staff told us each member performs his own review his own way.
They said they rely totally on their professional experience and use their judgment as a basis and guide for determining the sufficiency of a package.
Procedurally, in the staff review of a package's sufficiency they examine the categories of information provided in each package, and the level and depth of detail of the information provided.
In this respect, the data provided must go beyond a simple, categorized title that meets the Charter's criteria for completeness, and must show that an analysis of the requirements has been performed.
Using their professional judgment and based on their opinions, the staff considers a package adequate if it provides data and details they believe are necessary.
As part of our review, we discussed with the staff their review procedures and their acceptance review results. We found that in spite of the lack of written criteria, the staff's practices are relatively uniform and their reviews are, in our opinion, complete, thorough and well thought out. We found the staff members usually interact with each other and interface with program office officials and staff when performing their reviews.
Further, we found the staff members rely on each other's expertise and experience. We believe the staff is competent, qualified and conscientiously and responsibly reviews CRGR packages in an effort to ensure the Charter's intent and the Committee's needs are met.
However, we were told by the staff that while they consider most of the packages reviewed (as of the date of our audit) to be unacceptable, all but two packages have gone to CRGR for review and consideration rather than being sent back to the program office staff to be revised.
We discussed with the staff their findings and problems with CRGR packages.
Primarily, they believe all the packages they have reviewed so far are not as 1
complete as CRGR expected. Also, they feel the quality, level of detail and technical data included in all the packages is below standard. They said most packages, when initially reviewed for completeness, do not include required categories of information.
For example, many do not include industry input or quantitative analyses (see the next section for more detailed discussion of quantitative analysis), or do not provide categorical data as specified in the Charter.
Further, a staff member told us when he follows up and discusses his preblems with the program office's staff, he usually receives " standard, one-liner qualifiers" instead of the data he expected to receive.
The staff also considered most packages, when reviewed for sufficiency, to be unbalanced and inadequate.
In this respect, the staff feels the analytical criteria provided in most packages is inappropriate and the data provided is limited and inadequate.
Furthermore, they believe the basis and criteria used for justifying the implementation of most proposed requirements is qenerally questionable because of these inadequacies.
7 We discussed with the staff why incomplete and insufficient packages are not returned to the program offices as required by the CRGR Charter but instead are scheduled for CRGR's review. We were told that the DEDR0GR has decided to deviate from the Charter in this regard and has instructed the staff to go easy with the program offices in reviewing the acceptability of generic requirement packages. The staff cited several factors on which this decision was based. First, in the early days of CRGR, the program office staffs complained to the Commission that DEDR0GR and his staff were acting as line managers. Also, the program offices complained that because DEDR0GR's staff was performing quality reviews, the staff was going beyond the Charter and performing CRGR's task for them.
Second, the staff believes the DEDR0GR's policy is based on a belief that NRC's proposed generic requirement process would be stopped if all incomplete packages were returned. As a result, the staff said all packages are accepted even though they believe all are not adequate as measured against the Charter.
The staff told us they believe this policy was necessary at the time it was established because it provided an environment that allowed time to learn what CRGR expected from the program offices and permitted a " grace period" for the agency to gear up and accept CRGR. Further, it provided time for the program offices to develop means to satisfy the new CRGR criteria. However, the DEDR0GR's staff agrees that the time for learning is over and it's time to get the agency up to speed and in line with the Charter. They believe the DEDR0GR's policy is no longer needed and in fact should be eliminated.
They feel it is important for DEDR0GR and CRGR to start insisting that the program offices use analytical techniques and document their analyses, details, data and criteria used in justifying their proposed requirements. They feel this analysis must be included in packages if the Commission's intended job for CRGR is to be effectively accomplished.
We-also discussed the quality of proposed requirement packages with the former DEDROGR staff director. He told us he agrees with the staff's views that all packages should be complete and sufficient.
Furthermore, he believes Committee members should have a high degree of confidence in the packages they receive for review.
In order to establish high confidence levels, he said the DEDR0GR staff must ensure the packages contain quality information and sufficient data and must show a clear basis for the requirement.
Furthermore.
the packages should contain appropriate and adequate analyses that demonstrate the net worth of each requirement measured against the Charter's intent and stated public health and safety objectives.
Overall, the former director said the staff's acceptance reviews and CRGR's review process were very painful in the beginning because he feels the program offices resented CRGR and DEDR0GR's staff. Because the staff "did more than the Charter called for," it caused some problems. Because of these problems, he said he investigated the offices' complaints and decided to improve the staff's image and mend the relationship between the program offices and DEDROGR's staff. Accordingly, changes in operations were made.
Staff reviews continued, but it was also decided that no packages would be rejected based on the staff's acceptance reviews.
Further, the Charter was amended to provide for sufficiency reviews, and the depth and scope of acceptance reviews were expanded.
7 8
However, the former director doubted if this amendment would change the CRGR policy governing the staff's ability to have insufficient packages rejected.
Even though he agrees with the staff that quantitative analysis and other information is needed by CRGR to make good decisions regarding generic requirements and is required by the Charter, he said it is necessary to accept packages that are less than complete and sufficient because enforcing sufficiency and completeness to the fullest extent would stop the agency.
Also, he said there was a " learning curve" involved with CRGR, but he believes that curve is tapering.
He believes packages are showing signs of having better quality information and of being more complete, so this problem may eventually go away. However, he stated many packages were and many continue to be poor, and some are so insufficient program office officials can not defend their bases for requirements and CRGR can not make decisions regarding recommendations because basic fundamental questions can not be answered.
The DEDR0GR generally agreed that packages are not complete but believes the most significant problem with the completeness and sufficiency of packages sent to the CRGR is that they do not contain quantitative analysis.
Quantitative Analysis The Charter's purpose states " tools used by the CRGR for scrutiny would be expected to include cost-benefit analysis and probabilistic risk assessment where data for its proper use are adequate. Therefore, to the extent possible, written justifications should make use of these evaluation techniques. The use of cost-benefit analysis and other tools should make it possible to determine which proposed requirements have real safety significance, as distinguished from those proposed requirements which should be given a lower priority, or those which might be dropped entirely."
According to the Charter's operating procedures, each CRGR package should include quantitative analysis, including risk reduction and cost assessments to NRC and licensees. The Charter specifies that cost assessments should include projected total financial cost as well as anticipated occupational dose increase or decrease and the resulting added plant and operational complexity.
Consistent with these assessments, each package is required to provide the basis for the proposed requirement, its priority and schedule. Additionally, the Charter states each package should include a regulatory analysis suffi-cient to address the Regulatory Flexibility Act and Executive Order 12291.
As part of our review, we discussed with NRC program office officials their views and opinions regarding the need for and use of quantitative analysis for CRGR requirements. These officials generally agree quantitative analysis should be prepared for each requirement, although they also agree decisions should not be based solely on these analyses. They feel all decisions should be based on a mix of quantified, analytical decisions that use objective techniques and methods, and qualified, professional engineering judgments that rely on professional opinion.
DEDROGR staff members informed us that as part of their acceptance reviews, they determine if packages sufficiently include required quantitative analy-sis.
They said they have been quite lenient in accepting requirement packages
9 that contain less than adequate quantitative analysis; however, they also told us many packages do not contain any quantitative analysis. The DEDR0GR staff member most knowledgeable of quantitative analysis said that he believes most quantitative analyses submitted are insufficient and are therefore unacceptable. Also, we were told that while one division neither includes analyses nor reflects any attempt to perform such analyses, another division always includes analyses in their packages. However, while this division's
-analyses are considered the best of all, their quality is also considered to be far from what the agency should expect from the program offices.
~
In our discussions throughout the agency we were told several reasons why.
program offices are not using quantitative analysis to justify requirements.
However, one reason provided to us by many officials deals with the adequacy of the data base offices can.use to perform quantitative analysis. According to several officials, quantitative analyses are-not provided in their packages because they do not have the data they believe are needed to quantify the value and impact of their requirements. Without the data, they feel they can not develop objective primary premises or establish an objective basis for proposed requirements. Consequently, they feel they must qualify their analyses and, because their justifications are based on professional judgment and opinions, they can not prove the worth of the proposed new requirement.
Therefore, DEDR0GR staff members told us some offices consider oualified assessments to be a waste of time and effort and, accordingly, they neither perform nor include the analysis in their proposed requirement packages.
In discussing this issue with the DEDROGR, he agreed that data is not available and offered three additional reasons why offices are not including quantitative analysis in their packages. Those reasons are:
(1) the urgency of the issue does not allow time for preparing the analysis; (2) the limitations on the staffs' ability to do the analysis, (i.e., the inability to quantitatively identify risk associated with certain proposed actions); and
.(3) the offices' reluctance to do analyses because it takes time and effort.
Another aspect of the problem raised by several program office officials relates to the use of the Safety Goal in evaluating the results of quantita-tive analyses.
In this regard, they feel that while the safety goal concept can be used, its allied numbers and criteria, such as the generally applied core melt probability or man / rem costs, can not be used and applied to their analyses in determining bottom line safety conclusions. Without the safety goal criteria, we were told there are no objective, generally accepted limits which can be used to ' justify their decisions, for example, to measure the costs or to assess the risk (i.e., in terms of exposure) that will result from implementing the proposed requiremnts. Also, some officials told us they believe that because the basis for new requirements rests primarily on professional judgment and opinion, there is little confidence or validity in the justifications and bases for new requirements and therefore, all require-ments are susceptable to challenge.
We discussed the use and acceptability of offices' quantitative analyses with the former DEDR0GR staff director. He told us he feels at least some attempt to show objectivity should be part of each requirement's package; whether it be a qualitative backfit to a quantitative'model, or an independent analysis based on proposing offices' own data standards.
He said he had a lot of
I l
i 10 i
problems with some program office groups in trying to get them to prepare some sort of quantitative analysis.
He informed these groups that their analysis can be based on qualitative information. He said the objective in the interim is not necessarily to have a scientific analysis tied to the Safety Goal, but to have an analysis that demonstrates that value, impact, cost, and benefit have been considered and weighed. However, he said one group insists they can not perform analyses because they do not have the mechanisms to do so. As a result of these problems, he believes too many generic requirements may have been levied on licensees that are based on gut feelings rather than on well thought out opinions or valid conclusions. Therefore, he believes "many CRGR recommendations may be going the wrong way."
To determine if decisions are going the wrong way, the former staff director decided to test CRGR decisions against the Safety Goal, and he instructed the DEDR0GR staff to gear-up to perform retrospective reviews of all generic issues by evaluating the proposed requirements against the Safety Goal criteria. He expected these reviews to start in September 1983.
He felt this
" backward glance" on the proposed requirements would serve CRGR.
For example, he said one package submitted contained cost benefit and probabilistic analyses, however, they were qualified analyses and did not contain "hard numbers or safety goal criteria," so an objective basis for needing the requirement and for the decision to recomend its implementation could not be made. While he agrees the qualified analysis demonstrated a willingness on the program office's part to meet CRGR's criteria, the issue was so " charged up" that the CRGR " unanimously decided the requirement was not needed."
In this case, he said the decision was based " solely on gut feeling." After CRGR's decision was made, he said his staff tested this decision against NRC's Safety Goal. The test revealed that if NRC had used the Safety Goal criteria, or had the package contained quantitative data, or more sufficient and better qualitative data, the package would have shown enough of the right information to render a decision the other way, with a recommendation in favor of rather than against the implementation of the requirement.
As part of our review, we also discussed with CRGR members their review of the quantitative analyses for proposed requirements.
Based on our discussions, we found members generally do not review or analyze packages' quantitative analyses.
Instead, they rely on the DEDR0GR to surface his staff's problems regarding quantitative analyses.
As we mentioned in Chapter I, the Commission ins +ructed the ED0 to develop the ability to do independent cost estimates of proposed new requirements.
In response, the ED0 in SECY 82-39A stated NRR was exploring this area. During our review, we found several groups were exploring this area and were developing criteria and techniques to aid them in analyzing cost.
For example, a DEDROGR staff member informed us that he is thinking of developing a Value/ Impact Handbook, which will include the attributes of a cost / benefit and value/ impact analysis. He said he has discussed the concept with other groups and has received their input on how to develop a handbook.
If adopted by the agency, he said the handbook will serve as the staffs' guide and NRC's i
generally accepted criteria for CRGR related cost / benefit models.
Another DEDR0GR staff member told us he has been working with a consultant (under a $10,000 purchase order agreement contracted by DEDR0GR's staff) to
11 develop cost / benefit methods and a preliminary decision analysis model, which he intends to present to CRGR for their use in assessing and evaluatino CRGR packages. He said the consultant's report will describe state-of-the-art alternative methods and will define format and cor tent models for CRGR related packages. Additionally, he said they are trying to let a $150,000 con tract that, in essence, would provide assistance to the Committee membet s for reviewing program offices' CRGR related cost / benefit analyses.
We also found others developing similar cost data.
For example, an NRR official informed us that as a result of the Commission's policy decision regarding the Safety Goal, he recently let a $200,000 contract for developing a " Cost Estimate Handbook." He said the handbook will provide a list of cost elements they can use in performing cost analyses--elements that can be used as a checklist for preparing cost estimates.
Further, he said the handbook will provide his office with a generally accepted basis for analyzing the cost of generic issues. He hopes to get the handbook accepted as a " generally accepted set of assumptions" for the office's staff to use in preparing cost assessments for proposed requirements.
In this respect, he said it will serve as their criteria and basis for cost of generic issues.
In late 1982, the Cost Assessment Group (CAG) was formed and placed under the authority and control of RM. According to a proposed final CAG Charter provided to the EDO by the Director, RM, in about September 1983, CAG's purpose is to provide independent cost-assessment support for those NRC actions requiring regulatory analysis (value-impact / cost-benefit).
Accordingly, a senior CAG official told us he is developing a cost handbook.
He said his cost handbook will be used as a dictionary of steam generator repair costs which will provide guidance for assessing the cost of new requirements. He said he has let a $75,000 contract for the development of the handbook which he "wants to serve as the model for assessing the cost."
Further, this official stated he also has let a $150,000 contract for a study to develop relative actual cost for replacement power. He said this is an ongoing study which will be used to determine, in part, the cost of outages resulting from implementing proposed requirements.
During our discussions with this official, we were told RES also has recently let a $75,000 contract for cost / benefit studies.
According to this official, this contract may be self-initiated by RES and appears to overlap NRR's
$200,000 Cost Estimate Handbook contract. While he was not sure of what RES was doing, he said he was the responsible official for assuring there is no duplication in contraots let for studies relating to the development of quantitative / cost information and studies. Accordingly, he said he was reviewing this contract for any duplication with other work being performed.
As part of our review, we discussed with program office staff members their ability to meet the CRGR Charter's criteria and perform cost assessments. We also discussed CAG's effectiveness in supporting them in assessing cost.
Program office staff members said CAG is supposed to be the agency's cost assessment center. Accordingly, they believe CAG was established to serve as the agency's central and independent office for performing objective, quantitative cost analysis for all new requirements. However, some office and division directors and senior staff members told us CAG is not performing independent cost assessments for the program offices. They told us when CAG was established, the program offices' budgeted resources for performing
12 l
3 quantitative regulatory analysis were transferred to RM for CAG's use, under the assunption that the cost group would relieve "the staff of the burden of estimating cost for regulatory analysis." However, one staff member told us that while RM got the resources, they are not doing the offices' studies.
We discussed this issue with CAG officials and were told that NRC's Chairman wanted the Controller's office to be the agency's cost center where all cost analyses would be performed. However, he said in the first week of CAG's existence, he realized they did not have the resources to do all the work required by the program offices. Accordingly, he decided RM was not in a position to do the program offices' cost analysis for them. As a result, and because CRGR is an adjunct to the offices, he decided CAG would becone an arm of the CRGR and would perform cost work only for CRGR.
In this regard, he said he established a policy that if the program offices want CAG to perform their cost analysis for them, they would have to go to CRGR with a request for such work.
If CRGR decides the program office's request is a high priority item, and determines CAG should do the analysis, then CAG will assume the overall responsibility for performing the cost analysis.
During the period of our field work, a Charter for CAG was being developed and a preliminary draf t had been prepared.
In our opinion, the draft Charter clearly reflected that CAG was responsible for performing specific analyses for the program offices. However, it also clearly stated that CAG would primarily perform an analysis if assigned to it by CRGR. According to the draf t CAG Charter, "CAG has the responsibility to perform cost assessments for those proposed regulatory actions assigned to it primarily by the CRGR. CRGR will set priorities for cost analysis tasks that it requests, and advise the CAG of the priorities and desired schedule."
In addition, it states "the first and principal responsibility of the CAG is the development of cost analyses for inclusion in specific Regulatory Impact Analysis (RIA)."
During our exit conference, the DEDROGR staff director informed us that CAG had restly ra* f ted their charter. He was concerned that CAG may still believe they work only for CRGR.
We obtained and reviewed a cnpy of the redrafted CAG Charter. Our review of the cover memo from the Director, RM, transmitting the Charter to the ED0 clearly reflects that CAG's purpose has been modified to " provide independent cost-assessment support for those NRC actions requiring regulatory analysis (value-impact / cost-benefit)." Furthermore, the modified draft Charter reflects that CAG is not responsible for performing nor developing assessments for the program offices.
Instead, the Charter states CAG's responsibility is to " serve as the oversight and center of cost expertise for the NRC and will offer the respective NRC offices various levels of cost-analysis guidance and support. The CAG is responsible for overseeing the development of RIAs.
In addition, the CAG will provide, upon request, support to develop and/or review RIAs." Additionally it states "for all cost analysis prepared by the initiating offices, the CAG will have an oversight responsibility." However, it adds "at the discretion of the CAG and/or the DEDR0GR, the oversight role for any specific ongoing cost analysis may be omitted if in their opinion other issues have priority on CAG's resources".
Based on this language it appears that the current policy is that CAG will not perform independent cost assessments, but will be working as an advisor to the
f 13 f
program offices, who will be responsible for performing their own cost assessments. Based on our review, however, there appears to be some disagreements among the program offices, CAG and the DEDR0GR's staff as to what CAG's role is and should be.
As we noted before, each package's regulatory analysis is suppose to be sufficient to address the Regulatory Flexibility Act and Executive Order 12291. The Regulatory Flexibility Act requires that all rulemaking packages specifically address the impact of the requirements where rulemaking actions are likely to have a significant impact on a substantial number of small entities.
Executive Order 12291 states that the potential benefits of proposed requirements should outweigh the potential cost. Regulatory Impact Analysis are required for determining the potential cost and benefits of proposed requirements. Based on our review of the use of quantitative analysis to support proposed generic requirements, we believe NRC is not satisfying these requirements.
In summary, while the CRGR Charter requires that regulatory analysis (which include costs, benefits, value, and impact) must accompany all rules and other CRGR related packages, the staff stated the regulatory analyses are not being prepared or prepared sufficiently. As a result, and because packages with insufficient quantitative analysis are not rejected but instead are forwarded for CRGR's review, we believe NRC's decisions and implementation of many generic, regulatory requirements do not fully comply with the intent or letter of the Charter or the spirit of the Connission and ED0 in establishing the CRGR.
DEDR0GR Staff " Audits" of Activities The original CRGR Charter stated that the CRGR would review "all proposed new l
generic requirements to be imposed by the NRC staff." The Charter also listed lc the types of documents CRGR would review to carry out this mission, including:
l All staff papers which propose the adoption of rules or policy statements l
affecting power reactors or modifying any other rule so as to affect tecSnical requirements applicable to reactor licensees, including technical information required of reactor licensees or applicants for reactor licenses or construction permits; all staff papers proposing' new or revised rules of the type described in paragraph (1), including Advanced Notices; all proposed new or revised regulatory g(uides...StandardReviewPlan(SRP) sections...branchtechnicalpositions t
l-and) all proposed generic letters...multiplant orders, show cause orders.. 50.54f letters, bulletins... circulars...USI NUREGs...all new or j
revised Standard Technical Specifications; and all licenses, license amendments, approvals.of Prelininary Design Approvals (PDAs) and Final l-Design Approvals (FDAs), minutes of conferences with owners groups, licensees or vendors, staff approvals of topical reports, information notices, and all other documents, letters of communications...which are represented to reflect or interpret NRC staff positions.
On March 4,1983, the DEDR0GR briefed the Connission on the activities of the CRGR. At that briefing, the DEDR0GR informed the Connission that CRGR does not review all agency actions but instead samples and audits the acency's activity.
In this regard, he told the Commission that his staff looks at all
i 14 the various sources of requirements to determine whether or not "other vehicles are being used to implement generic requirements without going through the CRGR process. This is a way of monitoring these kinds of docu-t ments to find out if that is a problem.
In addition to that, we attempt to try to look at these sources of information as to whether or not the various other mechanisms for' issuing requirements, something as simple as a meeting with a vendor in agreement to change or modify a design, could clearly have a generic implication of doing it for all of them, and that is the intent of trying to ascertain whether that is going on."
Further, he stated, "My reading of this (charter) and the intent when we were writing it was to not leave the impression that we would review all of the documents. The intent was to be able to make it clear that we would be auditing at this process, looking at the process, but not in any way making an attempt to assert to the Commission that it was all (reviewed)."
To help clarify the issue and explain DEDR0GR's audit and review process, the former DEDR0GR staff director told the Commission, "We get virtua11v most of the mail I think that goes through in terms of reactor requirements.1d once in a while we will see something there that will raise a question in our minds. So we will typically go over to the NRR staff and say does this look like backfitting here or ratcheting?" He added "...sometimes perhaps I will go to (the DEDR0GR) and say this looks like something we may want to get briefed on. So that is how it has worked."
We discussed with DEDR0GR's staff and former staff director their " audit" efforts regarding CRGR issues. Based on these discussions, we found the staff does " audit" and perfonn independent, self-initiated studies of agency actions. Staff members told us many times they receive information from inside and outside NRC sources who allege generic requirements are being levied on licensees without first being reviewed by CRGR. When these issues are received, the staff independently reviews and analyzes NRC's actions and determines if the issues should have been or should now be referred to CRGR.
Staff members told us they also have developed and use many procedures and cudit techniques to monitor the agency's actions in an effort to ensure that applicable issues are referred to CRGR.
Further, the former staff director told us he and the DEDROGR encourage the staff to monitor the agency's actions and to "look to see if CRGR applicable items are going to the CRGR."
As a result of the March 4,1983, briefing, the Commission directed that the Charter be revised to bring it into conformance with current CRGR practices.
t In May 1983 the ED0 proposed an amendment to the Charter which was implemented hy the DEDROGR staff in June 1983. One of the ways in which the Charter was changed was to state that the "CRGR shall consider all proposed new generic requirements to be imposed by the NRC staff on... power reactors that are submitted for review by the offices" (change underlined). The implication clearly is that the CRGR would review only those items referred to it.
However, the DEDR0GR staff told us they continue to " audit" the agency's activity.
We also discussed with flRC officials and the former director of DEDROGR's staff the staff's post-amendment audit efforts.
In general, program officials
1 15 believe the audit process is necessary. They said, however, they have problems with these " policing efforts" and the staffs' "second guessing" of program office staffs. Also, one manager felt DEDROGR's efforts not only go beyond the Charter's criteria but also his findings sometimes result in a " bad catch" -- that is, the DEDR0GR staff's audit forces an issue to go before CRGR that ultimately is determined by the Committee to be outside CRGR's scope.
The former DEDROGR staff director told us, however, that while he admits the DEDROGR staff's audit efforts have caused some relationship problems, he encouraged the staff to do more than what the Charter calls for.
Both program office officials and the DEDR0GR's staff generally agree that for CRGR to be fully and totally effective, and in order for the agency to receive the full benefit of CRGR, all applicable generic issues must be reviewed by CRGR before they are submitted to the ED0 or communicated to licensees.
However, because NRC uses so many and diverse mechanisms to communicate and levy requirements on licensees, some generic issues are not referred by the program offices to the CRGR.
(See Chapter 111 for further discussion of this issue.)
As part of our review, we discussed with DEDROGR's staff their audit effcrts and results.
Staff members told us their audits catch a lot of issues that may have gone out without CRGR's review. One member said, however, the staff can not be sure the audits catch them all because there are so many areas covered and the audits cover only about 5 percent of the agency's actions.
Additionally, he said the DEDROGR now challenges the program offices regarding what they send or intend to send to CRGR, and is finding many issues that should have been referred to CRGR.
l Another staff member told us there are many items that may not have been l
referred to CRGR if the DEDR0GR had not been auditing the agency, although he j
said CRGR has not been too effective in ensuring that it receives all generic l
requirements. He believes NRC stands to enhance its level of effectiveness and get the full benefit from CRGR only if all applicable issues go through CRGR. Therefore, he feels the DEDROGR must have a way to audit the agency's actions to ensure that new issues do in fact go to CRGR.
r DEDR0GR staff members in general believe because so many requirements have gone out without CRGR's review, CRGR is not totally effective in controlling generic requirements. Additionally, one staff member said CRGR has had little effect on the program offices' staff communications with licensees and the ratcheting that comes from those communications.
Furthermore, the former staff director told us CRGR has looked at so little, they have only seen a small part of all the requirements going out and usually these are only the big items. He said this is a pi.olem because too much is going out that CRGR
~
is not seeing.
In discussing the staff's audit efforts w!tn the DEDR0GR, he told us the staff's audit activities are not inconsistent with the CRGR Charter because they are not performed under the CRGR authority.
Instead he said the audits are under the authority of the DEDR0GR.
However, he could not explain why modifications to the Charter were proposed in May 1983 to reflect that CRGR would only review issues referred to it by the program offices.
Furthermore, we explored this area and reanalyzed documents relating to the development and implementation of CRGR and the authority vested in the DEDR0GR versus the CRGR
16 Chairman. We found no clear distinction between DEDR0GR's authority and the authority granted to the CRGR Chairman. Similarily, we found no clear authority vested in the DEDR0GR regarding his staff's monitoring or auditing agency activities to identify generic requirements.
Minutes of CRGR Meetings According to the Charter, " minutes of each meeting, including CRGR recommenda-
~ tions and the bases therefor shall be prepared...(and) normally shall be circulated to all members within 5 working days after the meeting, and each member shr.ll have 5 working day: to comment in writing on the minutes."
Furthermcce, the Charter states "the minutes shall give an accurate descrip-tion of the basis for the recommendations and shall accurately reflect the consensus decision of the Committee." Additionally, it states, "it is the responsibility of each member to assure that the minutes accurately reflect his views. All comments received within that period shc11 be part of the minutes of the meeting."
CRGR does not prepare minutes, as such, of its meetings.
Instead, the DEDR0GR staff prepares meeting summaries after the fact. The staff prepares the summaries based on their understanding of what transpired and this is often difficult because the CRGR rarely takes formal votes. The summaries do not identify individual CRGR member comments or opinions, but present a single position regarding the acceptability of the proposed requirement. According to one CRGR member, the position reflected in the summary is almost always the DEDROGR's position on the requirement.
If a CRGR member disagrees with the position included in the summary he must prepare a written dissent to the summary within a prescribed period of time. This dissent is then attached to the summary. Most CRGR members and program office officials we spoke to did not have serious problems with the process for summarizing meetings; although we identified the following problems with this process.
First, because CRGR members are top level NRC executives with no staff assis-tance for CRGR matters, preparation of dissents is time consuming and burden-some.
As a result, one member said he takes the time to dissent in only the most extreme cases. The effect is that the meeting summaries, in many cases, do not reflect what happened during the meetings or what the CRGR members really think about an issue. Second, because votes are not taken it is often not clear, even to the members themselves, what the Committee's concensus of opinion was for the requirement. Third, in some cases the positions included in the summaries are based in part on discussions which take place between the DEDROGR and individual members after the CRGR meeting has ended. The effect of this practice is that it does not afford an opportunity for the full CRGR to discuss issues and is most confusing to the program office staff when 4 discovers the meeting summary reflects different agreements than those believed to have been arrived at during the meeting.
The following recent example highlights some of these problems.
In June 1982 and again about one year later in June 1983, CRGR reviewed NMSS' proposed Insider Rulemaking package concerning nuclear power reactors.
At the June 1,1983, meeting, we observed the Committee debate regarding the proposed rule and DEDR0GR's questions and suggestions regarding the need for a
" Government clearance program similar to the 'Q' clearance program." The CRGR
r 17 voted on these issues, even though it does not normally vote on issues. Based on our observation, and from our perspective, the meeting adjourned with clear votes:
(1) against a " Government 'Q' clearance program to reduce the number of protective barriers needed at nuclear reactor plants," and (2) for recom-mending to the EDO approval of the proposed rule.
However, our review of the summary of that meeting issued on June 24, 1983, revealed a CRGR recnm-mendatior; that the Commission reconsider a Government clearance program to address the " insider threat" and reduce the plant protection burden.
We discussed with program officials and CRGR members the difference between the position summarized in the minutes and that which we believe transpired at the meeting. One official told us that what may appear to transpire at a meeting is seldom reflected in the minutes. He said the minutes for this meeting clearly do not reflect the feeling, flavor and outcome of the meeting as he observed it. He added that the minutes are DEDR0GR's summaries and are not detailed records of the meetings; therefore, the minutes are heavily biased by DEDR0GR and his staff's perception of what transpired.
Another official said he has no problem with the way minutes are handled in general, but he is extremely unhappy with the minutes for this meeting because they do not show what happened at the meeting or after the meeting, and do not reflect on the meeting itself or the debate that ensued. Therefore, he believes the minutes are misleading.
Committee members informed us that af ter this meeting, some members indi-vidually discussed the " Insider" issues debated at the meeting with each other and with DEDR0GR and his staff. One member told us that as a result of these discussions, he changed his vote as did one other member. However, our discussion with the other member revealed that he did not change his vote.
Instead, he said "DEDR0GR changed his (DEDR0GR's) position from recommending at the meeting that the Commission consider a 'Q' clearance to eliminate all barriers, to a post-meeting recommendation that the Commission consider a
" clearance concept" to test the feasibility of eliminating "some" barriers.
A program official responsible for the rule told us he left this particular meeting with a clear vote in favor of the rule, noting that it was one of the few times the Committee had voted. However, for three weeks after the neeting, he waited for the minutes to be published.
By the fourth week, he said he learned that many drafts and redrafts of the minutes had been prepared and during this process, members had changed their votes and were asked by the DEDR0GR to document their changes.
By this time, however, he said enough votes had been changed to change the decision against the rule. This official said the vote was no longer in favor of approving the rule because CRGR decided to do their business outside the scope of the Charter and after the official CRGR meeting was over. He added that while the minutes do reflect that post-CRGR discussions were held by DEDR0GR with Committee members, they l
neither reflect nor capture what transpired at the meeting or the essence of the meeting. Also, they do not reflect what happened af ter the meeting when minutes were being drafted, r
We discussed the staff's process for preparing CRGR minutes with the former i
staff director. He said writing the minutes is a difficult task, " votes are not taken, there is no clear sense of opinion of members -- no appearance of a l
18 unified sense of opinion and no stated opinions to show uniformity."
Therefore, he said the opinions written in the minutes usually amount to contentions, are usually unclear, and many members usually disagree with them.
He told us the Committee does not use a court reporter or stenographer but instead a DEDR0GR staff member takes notes and prepares a draft of the minutes to reflect what he believes is the consensus of the group. While drafting the minutes, he said the staff member discusses with CRGR members their views and opinions in an attempt to further develop what he believes are the members views. He stated that experience has shown him and his staff that they have a good sense of how the Committee thinks and feels about an issue. Because of this experience, he believes they are able to write minutes that, for the most part, do reflect members opinions.
In discussing the preparation of meeting summaries with the DEDROGR he said he would not agree to any system which attributes comments to individual CRGR members. He believes the meeting sumaries already adequately summarize all discussions which take place during CRGR meetings rather than just the single, final CRGR po:ition.
Staff Assistance According to the Charter, the office of the DEDR0GR will provide staff support to the Committee.
Based on our discussions with the DEDROGR staff, we learned that upon completion of their acceptance reviews, they analyze and summarize their l
findings and prepare a summary analysis report regarding the completeness and sufficiency of the program office's generic requirement package.
In the summary report, the staff highlights their general observations and problems l
regarding the package's completeness, their specific observations regarding the package's sufficiency, and their specific problems regarding the package's j
i contents and the quality of informatie provided.
In this capacity, DEDR0GR's staff said they believe they are performing a CRGR staff support role and are assisting the CRGR members by preparing for them summary analysis reports that provide " insights" to packages' completeness, quality, adequacy and sufficiency regarding the issues they must review.
l I
During our review, we discussed the DEDR0GR's staff analysis and allied l
summary reports with CRGR members. Generally, members believe the DEDROGR's staff is his alone and not available for use by other members.
In addition, most of the members seldom review or use the staff's sumary analysis reports because they do not receive them soon enough prior to the CRGR meetings. They told us the reports generally are not available until either right before or on the day of the meeting and usually they are not handed out until the meeting. As a result, they have little time to read the reports and no time to analyze and digest them before the meetings.
Only one member said he relies on the DEDR0GR's staff summary analysis, but said he uses it when he is able to go and get a copy from the DEDR0GR's office before the meeting.
During our exit conference the DEDR0GR staff director told us he has recently established a policy requiring the DEDR0GR staff analysis to be issued three days before the CRGR meeting. While this policy has not yet been documented, a staff member told us it has been communicated to the staff and it is in
19 effect. He said it will be documented and included in DEDR0GR staff procedures. We will follow up on the documentation and implementation of this action at a later date.
Conclusions Our review identified four areas in the CRGR Charter not being fully imple-mented.. For the most part, two of these areas -- minutes, and staff assist-ance -- probably do not significantly impact the effectiveness of the CRGR.
We believe, however, that changes should be made to the process for recording minutes of CRGR meetings. The new policy regarding providing staff assistance to CRGR members is satisfactory subject to its documentation and continued implementa tion. The other two areas -- acceptance reviews and quantitative analysis--may be adversely impacting the ability of the CRGR to do the job anticipated when it was established. As a result, we believe program office proposals to the CRGR should be upgraded, and inadequate proposals should be returned to the program offices. We recognize, however, that before this can be done the criteria for defining completeness and sufficiency must be clearly defined.
Our review also identified one area in which the CRGR, through the DEDROGR staff, was exceeding the provisions of the Charter -- DEDR0GR staff " audits."
We believe these " audits" are necessary at the present time to assure the effectiveness of the CRGR and should continue, although the Charter should be changed to specifically provide for the " audits."
Recommendations We c! commend that the EDO:
1.
In order to ensure consistent application of CRGR's requirements for complete and sufficient packages:
(a) define and establish specific standards and minimum criteria governing completeness and sufficiency as required by the Charter; and (b) provide clarification to CRGR and the program offices regarding the requirements for the use and application of quantitative analysis in CRGR packages.
2.
After the criteria and standards are defined, require the DEDR0GR to return to the originating offices all unacceptable packages as measured against the newly defined criteria and standards.
3.
To eliminate the apparent duplication of effort and confusion associated with conducting regulatory analysis regarding value/ impact, cost / benefit
- studies, (a) assure that all cost studies being carried out within NRC are coordinated through CAG; and
20 (b) direct CAG to expedite issuance of the final CAG charter, after coordination with all affected offices, and assure that the CAG Charter is disseminated throughout NRC as the policy for conducting cost assessments 4.
To enhance CRGR's effectiveness and to ensure that CRGR reviews all applicable generic issues, amend the Charter to specifically authorize the CRGR, through the DEDROGR staff, to "eudit" the agency's activities to assure that generic requirements are referred to CRGR.
5.
Direct the CRGR to meet to discuss ways to improve the process for recording minutes of CRGR meetings and to resolve the problems regarding the completeness and accuracy of the minutes as written.
i i
O
21 III EFFECTIVENESS OF THE CRGR The Comission's request to 01A to audit the CRGR specified only that we review the implementation of the CRGR Charter. However, as we progressed with our audit and discussed our review with Commissioners' assistants and NRC officials, we realized that even if we found CRGR's Charter is being implemented, it does not assure that the Commission's objective in creating the CRGR -- controlling the imposition of generic requirements on licensees --
is being met. As a result, we expanded the scope of our audit to include an examination of CRGR's effectiveness in accomplishing its objective.
As part of our review, we discussed CRGR's effectiveness with CRGR members and NRC's program office officials. Committee members generally agree CRGR has added discipline to NRC's regulatory process. One member told us CRGR is one of the few places where a comprehensive judgment can be made of the technical and regulatory merits of an issue. He feels CRGR forces people to go through a reasoning process and think through proposals in a logical way.
Another member said CRGR has been effective because the offices and staff have been disciplined to the point that they know they have a " hurdle to jump."
CRGR has shown them the hurdle and how high they must jump; therefore, they know what they must do to clear the hurdle. Other members told us CRGR has been effective because it has forced the program offices to put work into the front end of packages, resulting in improvements in CRGR's operation and NRC's process because many managers are trying to do the kinds of assessments CRGR requires. Also they believe there has been a large improvement in items sent to CRGR as a result of this improved discipline.
We found office and division directors also believe CRGR has added discipline l
to the agency's regulatory process, was needed, and is effective. One official stated CRGR forces them to face up to the issues they are supposed to deal with and determine the worth, cost, and benefit of each issue and get it resolved. He said because of CRGR, the offices that had to interface did l
interface, comunication was achieved, and the issue was focused.
Another official told us CRGR has caused the program offices' staffs to be i
much more aware and sensitive to the requirements they intend to levy on licensees and more aware of ratcheting. An office director told us CRGR provides a check on their products and has caused their offices to be more careful in how they develop their basis and position for new requirements. He believes CRGR has made the staff realize that a more thorough, professional, objective and better thought-out basis for new requirements is needed. As summed up by another office director, the mere formation of CRGR synthesized the total organization; it made NRC more careful and deliberate in the way new requirements are developed.
However, while they agree CRGR was needed and has been effective in adding some discipline to NRC's process, program officials question CRGR's overall effectiveness. They believe if discipline was all the Comission wanted when they established the Comittee, then CRGR was not needed.
Instead of CRGR, they feel their program offices could have achieved the same levels of
l r
(
l 22 l
discipline and control as CRGR has attained over the staff if their I
organizations and management would have been strengthened.
I In our opinion, while the CRGR has prompted better agency coordination and has added discipline to the program offices' development of many new requirements, the CRGR has not been totally effective in controlling the application of new l
generic requirements on licensees.
Specifically, through discussions with the DEDROGR staff and the review of correspondence between the DEDR0GR and the program offices we identified 16 generic requirements imposed on licensees without having been reviewed by the CRGR. Each of these instances was identified by the DEDR0GR staff through its " audit" of agency actions. We discussed these instances with the DEDR0GR staff and the program office division directors responsible for issuing the requirements.
In almdst every case, the responsible division director agreed with the DEDR0GR's opinion that the requirement probably should have gone, but did not go, to CRGR before it was issued.
In our discussions with the division directors, various reasons were offered as to why the requirements did not go to CRGR for review, in one case, what started as a plant specific change developed into a generic requirement by successively applying it to individual plants. We were told that by the time the generic implications were discovered it was too late to do anything about it.
In another case a plant specific requirement also developed into a generic requirement.
Even though the division director was aware of the generic application, he did not send it to CRGR because he had inherited the i
problem and it was obvious before he got it that it should have gone to CRGR.
In a third case the requirement originated at the Comission and it was not clear whether CRGR had to review such requirements.
it appears to us that many of the requirements which did not go to the CRGR i
had one thing in common, they originated in a non-normal way. That is, they did not start out at the lowest levels of he agency as a new, generic i
requirement; rather, they were directed by the Commission, started as a plant specific requirement, or were inherited from another office.
We discussed this situation with the former DEDR0GR staff director and the DEDR0GR staff. The staff unanimously agreed that many issues that should be referred to the CRGR are not.
They attributed the problems to the program offices not implementing the procedures each established to assure generic requirements are referred to CRGR. The former staff director agreed that while offices have established procedures, their staffs are not following the procedures and management is not enforcing their use. He believed that if the procedures were effective the DEDROGR staff would not be finding so many generic issues being levied without CRGR's review.
The procedures to which the DEDROGR staf f referred are the procedures the Charter required each office to develop to assure that:
"(1)...All proposed generic requirements shall he submitted for CRGR review.
Such submittals should conform to the provisions of the CRGR charter relating to the contents of such submittals.
23
"(2) All documents, letters and communications that reflect or interpret NRC staff positions or requirements shall be submitted for review by CRGR unless these documents refer only to requirements approved prior to November 12, 1981.
"(3) For all other communications with licensees, no statements shall be used that might suggest new or revised generic requirements, staff positions, guidance or recommendations (unicss such statements have been recommended previously by the CRGR)."
DEDROGR staff members generally agree NRC still has a long way to go in controlling " unofficial" ratcheting that goes on. They believe there are many occasions when the staff verbally ratchets licensees with issues that are not requirements but instead are guidance.
Staff members told us many CRGR applicable papers ret issued without CRGR's review. One.0EDR0GR staff member said many program office staff members circumvent CRGR because they know that what they want to do amounts to ratcheting the licensees with non-requirements or they want to gat their " pet peeves out on the street," which they know they cannot get the agency to endorse. He told us there are many " ratchets" that get " pulled off (and) occur in the bowels" of the offices, which managers are not aware of.
From experience, he said he knows you can slide out requirements. Icgal or not, at any level if you're sharp. On many occasions, he said as a technical reviewer he had pushed licensees into meeting requirements by ratcheting.them one at a time, on a plant-by-plant basis, so that when he was finished,,he hat ~ gotton his way with all the plants. He said that made it generic. He added these low level ratchets cause a lot of problems that CRGR is supposed to be stopping. Also, he said it's ironic that these low level ratchets are the reason, in part, for having CRGR, yet they are the issues that get pushed to the back burner now that CRGR is in existence.
A DEDR0GR staff member told us many program office staff members are reluctant to come before CRGR so they " avoid CRGR and successfully avoid sending packages to CRGR -- that is, they slide the issues past the checkpoints and circumvent CRGR altogether." He said the staff often uses Regulatory Guides and applies them unofficially, as requiretaents, with the force of law to ratchet licensees. Using the guides, they ratchet licensees one at a time, in succession, forcing them to use the guide's criteria rather than their own alternatives. As each licensee gives in to this ratcheting, he said the guide as a requirement gains more strength until everyone accepts the guide as a requirement.
Finally, another DEDR0GR staff member stated that too many managers and staffers are finding ways "to skate around CRGR." For example, he said issues were implemented under the auspices that the Commission mandated them to do what they were doing in early 1982.
Because the Commission mandated the action, the office is snubbing its nose at CRGR and is not sending its issues i
l to CRGR. As a result, licensecs-are getting a lot of requirements imposed on them that would not be occurring if they were going to CRGR.
program officials and Committee members also agree NRC has not been totally effective in controlling the staff's communications and allied requirements in general they believe "rateneting" is prevelant in NRC. They told us thire are cases of ratchets going on every day, and several cases of staffers l
[
24 admitting that they ratchet licensees and applicants with things that are not requirements. They told us there are continuing problems with escalating requirements to upper management levels before they are issued and CRGR has net totally succeeded in controlling individual case-by-case ratchets. They believe there could be some improvement in CRGR's effectiveness if all the areas in the agency in which ratcheting is taking place were identified.
In discussing this issue with the DEDROGR, he agreed that some generic requirements are going out without CRGR review, but does not believe it is a large number.
He also agreed the problem is mostly a result of the program offices' failure to adequately implement their procedures for referring issues to the CRGR. He said the CRGR must also assume some of the blame for this failure because it could be doing more and better audits of the program offices implementation of their procedures.
Conclusion Because generic requirements are still being imposed on licensees without CRGR's review, the CRGR has not been totally effective in controlling the imposition of generic requirements on licensees. The problem does not appear to rest with the CRGR, however, but with the program offices for not referring generic issues to the CRGR. As a result, it appears desirable for the DEDROGR staff to continue " auditing" program offices activities to assure that generic issues are referred to CRGR. Action is also needed within the program offices, however, to implement their procedures and assure that all generic requirements are referred to the CRGR before being imposed on licensees.
Recnneendations 6.
We recommend that the ED0 assure that the program offices implement their office procedures and develop internal controls to ensure all generic requirements, as defined, are referred to CRGR for review.
9
25 IV OBSERVATIONS ON CRGR OPERATIONS While the Charter provides the base upon which the CRGR is built and provides some broad criteria to guide CRGR's operations, the specific ways in which the CRGR meetings are to be held and members are to operate are not specified. As part of our review we spoke to each CRGR member and to many office / division directors or deputy directors who deal with the CRGR regarding CRGR's operations. Overall, we learned that the CRGR has never held a meeting to
. review its own operations. Several of the efficials we spoke to, however, l
provided comments regarding CRGR's operations and, in some cases, identified changes they would like to have made.
Following is a summary of the more significant and frequently heard of these comments.
According to CRGR's Charter, a reasonable amount of time shall be permitted for discussion of each item by Comittee members. At the conclusion of this discussion, the' Charter states "each comittee member shall sumarize his position (and) the Committee shall recomend to the EDO approval, disapproval, modification, or conditioning of (generic) requirements considered by the
. Comi ttee. "
Based on our observations of several CRGR meetings and discussions with program officials and CRGR members, we determined that the forum used for CRGR
/
meetings is very informal. Also, we observed that CRGR's program and format for conducting meetings appears to lack established rules of order,.and the conduct of meetings appears to lack. basic, administrative management controls.
Further, we were told by Comittee members and the former DEDR0GR staff director that CRGR decisions and recomendations regarding proposed requirements are not necessarily based on members' votes and do not necessarily flow from a consensus of members' opinions or conclusions as expressed at meetings. We were also told that CRGR decisions and recommenda-tions usually are not finalized at the meetings, but instead are developed and finalized afterwards when the sumary minutes are drafted by the DEDR0GR.
Some members noted that the CRGR never meets in executive sessions to discuss proposed requirements. They feel executive sessions should be part of each meeting so generic issues presented can be debated and discussed in private in an environment that allows each member to think through the issue proposed.
They believe this process could eliminate false ideas and impressions people take with them when they-leave CRGR meetings. One member in particular strongly supported executive sessions so the CRGR members could discuss issues a
among themselves rather than with the staff proposing the requirement. He said the executive session could be either behind closed doors or simply after its regular meeting with the program office staff present but silent.
Regarding the need for. administrative management controls and rules of order, some officials and CRGR members believe more control is needed for CRGR meetings. Some program officials also told us CRGR should add administrative discipline to their meetings. These officials feel the Comittee should refine their procedures and establish some upper and lower boundaries and limits for what they do and how they do it. An office director told us he feels there should be tighter controls and better administration of the
~
26 meetings. He also believes formality should be added to the discussion and deliberation process.
Regarding the Comittee's voting process, Comittee members told us this is a weakness in the format of CRGR's meetings.
For example, one Committee member told us CRGR related decisions are not voted on at the meeting but they are
" plucked out of the air by (DEDR0GR's staff) when the summary of minutes are dra f ted. " He said vote sheets should be used to help formalize and clarify the process. Another member told us there is generally an implied concensus of opinions rendered during the meetings but each member's opinion is not clearly stated. He added there is not a clear vote taken at the meetings.
Another member told us CRGR is advisory and its " votes" are not set in concrete and can be changed after the meeting based on the continuing dialogue that takes place among the Comittee nembers.
Several program officials also believe that CRGR members should vote on issues because it is a good technique for defining the final opinions of the members.
They said without votes, there is little clear evidence as to what is finally decided on and resolved at CRGR meetings.
In addition to their CRGR duties, CRGR members also hold senior level management positions which are very demanding in terms of time and effort.
During our review, we discussed with CRGR members and program office officials, the members' ability to prepare for and participate in CRGR meetings considering the demands of their primary positions.
From the members' perspective, their level and depth of involvement in pre-paring for and participating in CRGR meetings is as different as the individual members themselves. Some members get much more involved than others and, in this regard, their participation is very aggressive. Other members are very passive, generally have little to say regarding the proposed requirement being reviewed, and go along with the stronger members in taking actions. One CRGR member observed that only two or three of the members participate in meetings, while the others generally agree with the way the participating members go. Another member sumed-up the problem by saying there are " weak sisters" on the Committee and "this is very unfortunate because it reduces CRGR's effectiveness." He added that if too many members are weak, the CRGR review process will be reduced to a process where another tier of management instead of an advisory committee will be doing the generic reviews.
Program officals we spoke to generally agreed with the CRGR members' assessments of the members' participation. One program official described the CRGR's operational effectiveness as a committee by saying CRGR does not operate and function as a comittee, but operates as a one man vote. Another senior program official said the Committee is " led by the nose" to go the way the strongest member (s) wants them to go. He said most members seldom stand on their own two feet; therefore, the value and effectiveness of the Committee, as a comittee, is questionable. Another official stated that some members " simply take up a chair" and when they do get involved, they act off the seat of their pants. Others, he said are active participants, do their homework, and act in a professional way that reflects a well thought-out process.
L.
27 From our discussions with Committee members, we found reners are not consis-tent in how they prepare for meetings. The members geric illy rely on their own knowledge in evaluating the issues before the CRGR although some at least occasionally use their program office staffs to help prepare for meetings.
In general, members said they prepare for CRGR meetings by first reviewing the packages. In some cases they then have staff members in their program office review the packages and ' prepare papers documenting their analysis and opinions. One member said his opinions end up being 50 percent personal views and 50 percent program office views. Although his reviews are from a personal perspective, he said his CRGR reviews are a natural adjunct to his program office responsibility of reviewing issues from a regulatory reasonableness perspective; therefore, there is obviously much in comon between his personal views and his official positions.
When a member represents the office proposing a requirement to the CRGR, that member is not supposed to present and support the requirement. Rather that member (according to DEDR0GR) is not to participate in the CRGR's decision.
While this may be CRGR's policy, we received comments from one CRGR member to the effect that some members do not always remove themselves from the pro-ceedings and, in fact, actually end up defending proposed requirements they are supposed to be " independently" reviewing.
In discussing these comments and observations with the DEDROGR, he told us the single, biggest problem currently facing the CRGR is their inability to conduct executive sessions. He would like to have the CRGR meet alone, either before, during or after CRGR meetings, to sharpen the focus of the review, make decisions and openly discuss the proposed requirement. He believes members could more openly and freely discuss their professional opinions without fear of being chastised by their office management. He also believes the meeting minutes would be improved as a result.
Conclusion Our review identified several aspects of the CRGR's operations which some Committee members and program office staff would like to have changed. While we do not necessarily support any specific changes to CRGR's operations we believe the CRGR should review its own operations to determine what changes are needed.
l Recommendation 7.
We recommend that the DEDROGR schedule a CRGR meeting in the near future to discuss, and amend as necessary, CRGR operating procedures.
28 V
OIA EVALUATION OF ED0 COMMENTS ON THE JANUARY 27, 1984, DRAFT OF THIS REPORT On March 2, 1984, the ED0 commented on a draft of this report. The ED0's comments are included in their entirety in Appendix I.
Following is our evaluation of the ED0's comments.
General Comment On January 27, 1984, we provided a draft of this report to the ED0 to obtain
.his comments. On February 21, 1984, 01A officials met with DEDROGR staff officials, at their request, to discuss the draft report. At this meeting DEDR0GR officials stated that while they did not disagree with the content or substance of our report, there were several aspects of the report's language which they felt should be changed to provide better balance.
Subsequent to this meeting, 0IA assessed the nature and character of their comments, and modified several parts of the report to clarify its language. There were no substantive changes to the report as a result of this meeting.
The E00's March 2,1984, memorandum to the Acting Director, 0IA, states:
"Overall, the report leads me to conclude that the CRGR has been and continues to be effective in assessing and controlling the imposition of generic requirements on nuclear power plants."
It also states that the ED0 had discussed the report with the CRGR and DEDROGR and with appropriate office directors, and that members of their respective staffs had met to discuss the facts presented in the report. Although the ED0 neither generally agreed nor disagreed with the facts presented in the report, he did state that "I appreciate the effort DIA has made in its analysis and constructive comments."
Overall, although tne E00 has taken or has agreed to take action on some of our recommendations, we believe the ED0's detailed comments on several of our recommendations are not responsive to their intent in that his discussion tends to either rebut the recommendation, or mitigate the problems addressed by explaining them away. The net result is that we believe three of the seven recommendations in our report are unresolved and, in accordance with NRC Bulletin 1201-3, we are referring them to the Commission for resolution. An o
analysis of the ED0's comments on each of our recommendations follow.
Recommendation 1(a)
Our report noted that specific guidance and criteria, which define complete-ness and sufficiency for CRGR packages submitted for review, do nrt exist.
This created problems for the program offices in preparing packages, and for the CRGR staff in evaluating the completeness and sufficiency of those packages. As a result, we recommended that specific standards and criteria governing completeness and sufficiency be defined and established.
The ED0 responded to this recommendation by stating that the Charter, in Section IV.B, establishes standards and criteria for the content of packages.
He also states that when the criteria and standards were established, it was
29 recognized that they were "a desirable but difficult goal" and would not be met in all cases.
The ED0 concluded by stating he did not believe "further efforts to refine the existing standards and criteria used by the DEDR0GR staff to determine whether or not a proposal is complete and sufficient enough for CRGR review is warranted."
We disagree with the E00's response for several reasons.
First, the guidance in Section IV.B of the Charter simply identifies the categories of information to be included in CRGR proposals.
It does not identify the type of informa-tion that should be included in each category for the proposal to be
" sufficient." Second, our review disclosed that the DEDR0GR staff believed proposals seldom fully met the stated content requirements in Section IV.B.
Third, the E00's response is contrary to the general philosophy NRC uses in dealing with applicants and licensees; that is, to clearly define requirements and criteria which constitute the staff's bases for decisions. We do not understand why the ED0 would subscribe to a policy of not clearly defining CRGR requirements for his own staff.
Fourth, our review showed that the DEDR0GR staff relied principally on their professional judgment in evaluating a proposal's completeness and sufficiency.
Because judgments can obviously vary among the staff and between proposals, we believe a better definition of the standards against which packages are measured is needed.
As a result, to improve the quality of proposals submitted to the CRGR and to assure the consistency of DEDR0GR staff acceptance reviews of the proposals, we continue to believe the standards and criteria for generic requirements should be clearly defined. Because we disagree with the ED0's response to this recommendation we are referring the recommendation to the Connission for resolution.
Recommendation 1(b)
Our report recommended-that the EDO clarify to the CRGR and program offices the requirements for the use and application of quantitative analysis in CRGR packages.
The ED0 responded that in addition to the guidance in the CRGR Charter and the Regulatory Analysis Guidelines (NUREG/BR-0058) issued on December 13, 1982, the CAG Charter was issued on January 4, 1984, and Value/ Impact Analysis guidance (NUREG-3568) was recently issued by the Office of Nuclear Regulatory Resea rch. He also stated that " Directions to the staff regarding the use of the guidance will be issued in the near future and the overall Regulatory Analysis Guidelines, NUREG/BR-0058, will be revised to require utilization of the NUREG 3568 guidance."
In addition, he noted that NRR is sponsoring, and CAG is monitoring, development of a cost assessment handbook.
In view of the actions proposed by the ED0 we consider this recommendation resolved, however, we will follow-up on the completion of the ED0's commitments at a later date.
Recommendation 2 Our report recommends that after the criteria and standards for generic requirement packages (Recommendation 1 above) are defined, the DE3ROGR should return to the originating office all unacceptable packages as mens ved against
30 the newly defined criteria and standards. The EDO responded that "the DEDR0GR staff understands its responsibility to return packages that are unacceptable for CRGR review to the originating office."
In addition, he stated "when DEDROGR staff members review the packages, they discuss their findings with the appropriate submitting office officials. This latter_ process has some-times resulted in the submitting office withdrawing the package from CRGR consideration for further work." Thus, he believes the intent of this recom-mendation is currently being met without the need for additional direction or requi rements."
We recognize that offices have withdrawn proposals after discussing them with the DEDR0GR staff.
In fact, this procedure was the point of much discussion at the March 4, 1983, DEDR0GR briefing on the activities of the CRGR to the Commission.
In our opinion, this procedure does not mitigate the problem or change the fact that the DEDR0GR staff said they believe all packages received, including those submitted to the CRGR for review, have not been acceptable. What we are specifically looking for is a retraction, or at least a clarification, of DEDR0GR's policy of not returning unacceptable proposals tc the program offices that prepared them. The ED0's response does not address this issue. We believe this recommendation is unresolved and are referring it to the Commission for resolution.
Recommendation 3(a) and (b)
These recommendations related to assuring that all cost studies in NRC are coordinated through CAG, and with expeditously issuing and disseminating the final CAG Charter throughout NRC as the policy for conducting cost assess-ments.
The ED0 agreed with these recommendations and noted that the final CAG Charter was issued on January 4, 1984. We will follow up on the implementation of these recommendations at a later date.
Recommendation 4 This recommendation dealt with amending the CRGR Charter to specifically authorize the DEDR0GR staff to audit the agency's activities to assure :Jeneric requirements are referred to the CRGR.
The ED0 begins his response by noting that the DEDROGR staff does not conduct audits, per se.
For clarification, we note that throughout our review the i
DEDROGR staff and program office officials referred to the DEDR0GR staff reviews of agency correspondence as " auditing." We adopted that language in our report. We understand that the DEDR0GR staff does not conduct " audits,"
l per se; however, their sampling techniques and oversight of agency correspondence are a form of auditing and " audit" was the accepted term in use throughout our review. We continue to use the term in this analysis within the same context used throughout the report. Also, we understand that the DEDR0GR staff director is making a concerted effort to eliminate the use of the term " audit" in referring to DEDROGR staff activities.
The ED0 also states the Charter does not need to be changed because NRC Manual Chapter 0103 provides sufficient authority for the DEDR0GR to perform this function. Specifically he quotes from Manual Chapter 0103:
"The Deputy
31 Executive Director for Regional Operations and Generic Requirements is authorized and directed to assist and act for the Execctive Director for Operations (EDO) in performing the following functions:... establishing and directing the implementation of procedures for controlling and tracking generic requirements on licensees...and taking such actions as may be neces-sary to carry out the functions assigned by this chapter..."
Further, the ED0 states this authority is reflected in the position descriptions of the DEDR0GR and the Director, Regional Operations and Generic Requirements Staff.
Finally, he states that because this function is a previously authorized DEDR0GR function, and not a CRGR function, amendment of the CRGR Charter is not necessary or appropriate.
We disagree. During our audit we received comments from CRGR members and program office officials that there is sometimes confusion regarding whether the DEDROGR is acting in his line position as Deputy EDO, or in his staff position as CRGR Chairman. We believe the ED0's response tends to perpetuate this confusion.
Furthermore, because the DEDR0GR staff's audit activity is being carried our specifically in support of the CRGR, we believe it should specifically be recognized and authorized in the CRGR Charter.
In this regard, we note that language currently in the Charter (i.e., "the CRGR shall consider... requirements...that are submitted for review by the offices") could be interpreted as specifically prohibiting the CRGR/DEDR0GR staff from carrying out this function. We believe this recommendation is unresolved and, in view of the fact that this is an area in which CRGR and DEDR0GR staff activities are not consistent with the CRGR Charter, we are referring the recommendat!on to the Commission for resolution.
Recommendation 5 This recommendation was for the CRGR to meet to discuss ways to improve the process for recording minutes of CRGR meetings and resolve the problems regarding the completeness and accuracy of the minutes as written.
In response to our recommendation, the ED0 stated that the CRGR met on February 15, 1984, to discuss the need for and ways to improve the process associated with the development of the meeting minutes and that "the CRGR members unanimously agreed that the current process for develeping the r..eeting summaries is appropriate and the summaries accurately reflect the Committee consensus recommendations and conclusions. Also, the members agreed that the current format and content of the meeting summaries are appropriate."
Additionally, the ED0 said "the members noted that whenever there is a complex matter reviewed by the Committee such that the DEDR0GR staff has any questions as to whether or not the meeting summary reflects the consensus of the Committee, the minutes are circulated in draft to members for comment prior to l
being adopted." Finally, he said "the members also noted that the Charter provision allowing members to append individual viewpoints to the meeting suninaries provides the necessary safeguards to ensure accuracy of the meeting summaries."
The results of the CRGR meeting of February 15, 1984, appear to conflict with the results of our audit in that during our audit some CRGR members and program office officials noted problems with the meeting summaries. We attribute these differences to changes in the CRGR membership since our audit was performed.
32 We believe the ED0's response resolves our recommendation, however, we will follow up on this issue at a later date to determine if the new CRGR members remain satisfied with the process by which meeting minutes are recorded. We do not know, however, whether the summaries prepared for CRGR meetings are satisfactory to the Commission.
If the Commission is satisfied that the meeting summaries satisfy their intent for " minutes," no action is required.
If, however, the Comission wants contemporaneous, detailed minutes prepared showing discussions that take place during the meetings to provide the Comission with more information than simply what the CRGR concluded (e.g.,
the pros and cons of an issue) the Commission may want to direct the EDO to change the process for recording minutes of CRGR meetings.
Recomendation 6 Our report concludes that generic requirements are being imposed on licensees without CRGR's review; therefore, CRGR has not been totally effective in controlling the imposition of generic requirements on licensees. However, we state the problem does not appear to rest with the CRGR but with the program offices who are ultimately responsible for referring generic issues to the CRGR. Our report recommends that the ED0 assure that the program offices implement their office procedures and develop internal controls to ensure all generic requirements, as defined, are referred to CRGR for review.
In responding to this recommendation the ED0 states that direction to the program offices regarding this matter is in the CRGR Charter and the program offices have developed internal control procedures as required by the Charter.
He also said he recently discussed the importance of these procedures with appropriate office directors and the DEDR0GR staff would monitor implementa-tion of office procedures as part of its normal program office interfacing activities.
Based on the results of our audit it is apparent that direction to the program offices in the CRGR Charter is not working and that the procedures developed by the program offices are not being effectively implemented in all cases.
The program offices' failures to implement existing requirements was the basis for our recommendation and we therefore do not accept these requirements as a basis for closing our recommendation.
We are not aware of the content of the ED0's discussion with the office directors but believe such reinforcement of the importance of implementing their procedures is a positive step. We will accept the ED0's statement as resolution of the recommendation and follow up at a later date to determine if offices have, in fact, improved their implementation of their procedures.
Recommendation 7 l
Our report recommends that DEDROGR schedule a CRGR meeting to discuss, and amend as necessary, CRGR operating procedures.
In responding to this recomendation, the EDO states that the CRGR held a meeting on February 15, 1984, to discuss the CRGR's operating procedures. He also states that "although the members were generally satisfied with the current operating procedures, the Committee concluded that an Executive Session at the end of each meeting would facilitate development of the
33 Comittee's recomendations to me and their comunication to the sponsoring office."
In addition,.he said he is currently considering the Comittee's recomendation that Executive Sessions be instituted, and will soon render a decision.
In light of the ED0's actions we consider this recomendation resolved. We
- will review the implementation of the E00's comitments during our follow-up
)
review.
9
.O 1-f
Appendix I
+/c aer%,o UNITED STATES g
8 NUCLEAR REGULATORY COMMISSION g
j WASHINGTON, D. C. 20655 MAR 0 21984 MEMORANDUM FOR:
George Messenger, Acting Director Office of Inspector and Auditor
~
FROM.
William J. Dircks Executive Director for Operations
SUBJECT:
COMMENTS ON THE DRAFT REPORT OF THE OFFICE OF INSPECTOR AND AUDITOR (0IA) REVIEW 0F CRGR Your memorandum of January 27, 1984, forwarded to me for comment, the draft report of the OIA review of the Comittee to Review Generic Requirements (CRGR) conducted during the period from mid March through July 1983.
Overall, the report leads me to conclude that the CRGR has been and continues to be effective
-in assessing and controlling the imposition of generic requirements on nuclear power plants.
Enclosed are my coments that address specifically each OIA recommendation for improving the means for controlling generic requirements.
Prior to developing my response, I asked the CRGR and the DEDR0GR for their views on the report and discussed the matter with appropriate office directors.
As proposed in your memorandum, members of our respective staffs have met to discuss the facts pre-sented in the report.
In conclusion, I believe that the effectiveness of the CRGR since the period i
covered 5y the OIA review has continued to improve and as a result generic l
requirements imposed by the NRC are being effectively controlled. The principal l
reason for this, I believe, is the discipline that the CRGR adds to NRC's regulatory process.
I appreciate the effort DIA has made in its analysis and constructive comments.
l
\\
g:
F William J. Dircks Executive Director for Operations
Enclosure:
EDO Responses to 01A Recommendations
EDO RESPONSES TO OIA RECOMMENDATIONS OIA Recomendation 1 In order to ensure consistent application of CRGR's requirements for complete and sufficient packages:
(a) Define and establish specific standards and minimum criteria governing completeness and sufficiency as required by the Charter.
Response
The CRGR Charter in Section IV.B establishes standards and criteria for the con-
-tent of packages submitted to the CRGR. When these standards and criteria were established, it was recognized that they were a desirable but difficult goal.
Due to the paucity of quantitative data for many of the issues and other factors such as the time frame available for conduct of some of the analysis, it was recognized that the stated criteria would not be met in all cases.
Section I of the Charter, for example, notes that quantitative analysis would be expected to be used "where data for its proper use are adequate" and "to the extent possible."
As the ability to perform quantitative analysis has advanced, the DEDR0GR staff and the CRGR have noticed an improvement in the quality of the packages submitted for CRGR review. Currently, a balance is sought among:
sufficient quantitative and qualitative analysis to assess the need for and potential shortcomings of proposed actions; availability of data, resources and methodology available to perform additional analysis; and, importance of the actions, timeliness and their potential impact.
Accordingly, I do not believe that further effor:. to refine the existing standards and criteria used by the DEDR0GR staff to determine whether or not a proposal is complete and sufficient enough for CRGR review is warranted.
However, agencywide efforts to improve regulatory analy.is are in progress as indicated in the response to recommendation 1(b).
OIA Recomendation 1 (continued)
~
(b) Provide clarification to CRGR and the program offices (in guidance docu-ments for) regarding the requirements for the use and application of quantitative analysis in CRGR packages.
Response
Directions regarding the use and application of quantitative analysis in CRGR packages are provided in Section IV.B of the CRGR Charter, in the Cost Analyses Group (CAG) Charter promulgated by me on January 4,1984 and in the Regulatory Analysis Guidelines (NUREG/BR-0058) promulgated by me on December 13, 1982.
These have recently been supplemented by Value/ Impact analysis guidance, which was recently prepared by RES. This guidance, NUREG 3568, was coordinated in its development with CAG, NRR and DEDR0GR. Directions to the staff regarding use of the guidance will be issued in the near future and the overall b
- Regulatory Analyses Guidelines, NUREG/BR-0058, will be revised to require utilization of the NUREG 3568 guidance.
In addition, NRR is sponsoring, and CAG is monitoring the development of a handbook that will guide analysts in efforts to realistically assess the actual costs of implementing new require-ments. Work also is ongoing regarding the use of probabilistic risk assessment.
If in the future it becomes clear that additional guidance to the staff is needed in order to properly use such information in the analysis required for
~
CRGR packages, the staff will be so directed.
In view of the aforementioned existing guidance and guidance under development, it does not appear that additional action is necessary at this time, e
OIA Recommendation 2 Once the criteria and standards are defined, require the DEDR0GR to return to the originating office all unacceptable packages as measured against the newly defined criteria and standards.
Response
The DEDR0GR staff understands its responsibility to return packages that are unacceptable for CRGR review to the originating office. As indicated by your findings, packages have been returned formally in several instances.
In addition, when DEDR0GR staff members review the packages, they discuss their findings with the appropriate submitting office officials.
This latter process has sometimes resulted in the submitting office withdrawing the package from CRGR consider-ation for further work. Thus, I believe the intent of this recommendation is currently being met without the need for additional direction or requirements.
01A Recommendation 3 To eliminate the apparent duplication of effort and confusion associated with conducting regulatory analysis regarding value/ impact, cost / benefit studies:
(a) assure that all cost studies being carried out within NRC are coordinated through CAG.
Response
I agree that CAG is to be cognizant of cost studies being carried out by the staff. The specific responsibilites of CAG and the other NRC offices are specified in the CAG Charter promulgated on January 4, 1984.
OIA Recommendation 3 (continued)
(b) Direct CAG to expedite issuance of the final CAG Charter after coordination with all affected offices, and assure that the CAG Charter is disseminated throughout NRC as the policy for conducting cost assessments.
Response
The recommendation was implemented on January 4,1984.
,._ OIA Recommendation 4 To enhance CRGR's effectiveness and to ensure that CRGR reviews all applicable generic' issues, amend the Charter to s)ecifically authorize CRGR, through the
.DEDROGR staff, to audit the agencies' sic] activities to assure that generic requirements are' referred to CRGR.
Response
- While the DEDROGR staff does not conduct audits, per se, it does monitor the
.o correspondence to licensees issued by the program offices, and interacts with the program offices, in order to identify instances where generic requirements may not have been or need to be sent to the CRGR.
This function is an ongoing and integral part of their activities. NRC Manual Chapter 0103 provides sufficient authority for the DEDROGR to perform this function. Specifically:
"The Deputy Executive Director for Regional Operations and Generic Requirements is authorized and directed to assist and act foi the Executive Director for Operations (ED0) in performing the following functions:... establishing and
. directing the-implementation of procedures for controlling and tracking generic communications with, and generic requirements on. licensees...and taking such actions as may be necessary to carry out the functions assigned by this chapter..." This authority is reflected in the position descriptions of the DEDROGR and the Director, Regional Operations and Generic Requirements Staff.
Because this function is a previously authorized DEDR0GR function, and not a CRGR function, amendment of the CRGR Charter is not necessary or appropriate.
~01A Recommendation 5
' Direct the CRGR to meet to discuss ways to improve the process for recording minutes:of CRGR meetings and to resolve the problems regarding the completeness and accuracy of the minutes as written.
-Response i
The'CRGR met on February 15, 1984 to discuss the.need for and ways to improve the process associated with development of the meeting minutes. The CRGR
^
members unanimously agreed that the process for developing the meeting sunnaries is appropriate and that the summaries accurately reflect the Committee consensus recommendations and conclusions. Also, the members agreed that the current format and content of the meeting summaries are appropriate.
The members noted that whenever there is a complex matter reviewed by-the Connittee such that the DEDROGR staff has any questions as to whether or not i
the meeting summary reflects the consensus of the Committee, the minutes are circulated in draft to members for comment prior to being adopted. The members also noted that the Charter provision allowing members to append individual viewpoints to the meeting summaries provides the necessary safeguards to ensure accuracy of the meeting sunnaries.
i
?
I
2 ;
OIA Recommendation 6 We recommend that the ED0 assure that the program offices implement their office procedures and develop internal controls to ensure all generic requirements, as defined, are referred to CRGR for review.
Response
Directions to the program offices regarding this matter were promulgated as
.part of the CRGR Charter dated June 16, 1982.
The program offices developed internal control procedures as required by the CRGR Charter, and I recently
. o discussed the importance of these procedures with appropriate Office Directors.
In addition, the DEDR0GR staff will, as part of its normal program office interfacing activities, be alert for instances whereby the office procedures
~ have not been properly implemented.
OIA Recommendation 7 We recommend that the DEDR0GR schedule a CRGR meeting in the near future to discuss, and amend as necessary, CRGR operating procedures.
Response
The CRGR held a meeting on February 15, 1984 to discuss CRGR operating procedures. Although the members were generally satisfied with the current operating procedures,- the Committee concluded that an Executive Session at the end of each meeting would facilitate development of the Committee's recommendations to me and their communication to the sponsoring office.
I am currently considering the Committee's recommendation that Executive Sessions be instituted and I will soon render a decision.
I t
v r
l l[
-