ML20154A196
| ML20154A196 | |
| Person / Time | |
|---|---|
| Issue date: | 05/24/1982 |
| From: | Deyoung R NRC OFFICE OF INSPECTION & ENFORCEMENT (IE) |
| To: | Haynes R, James Keppler, James O'Reilly NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I), NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II), NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III) |
| Shared Package | |
| ML20151H783 | List: |
| References | |
| FOIA-87-868 NUDOCS 8809120106 | |
| Download: ML20154A196 (25) | |
Text
'
F\s e aser jo,, UNITED STATES .
,,.....w.
. e NUCLEAR REGULATonY COMMISSION l{. jj WASHWO TCN, D. C,20585
%"- QT /
EY 2 4 E22 MEMORANDUd FOR: Ronald C. Haynes, RA, Region I
. James P. O'Reilly, RA, Region II James G. Xeppler, RA, Region III Region IV John RobertT.
H.Collins, RA,(A, Region V Engelken, FROM: Richard C. Cmfoung. Director Office of Inspection and Enforcement
SUBJECT:
APPRAISAL OF REGIONAL IMPLEMENTATION OF :
INSPECTION AND ENFORCEMENT PROGRAM l In the Chaiman's memorandum of Cetober 8,1981 which established the autonomy of the Regional Offices, the Chairman directed that IE, among other activities, provide oversight and ensure appropriate program implementation and unifomity, throughout the Regions. I directed the staff to prepare an IE program for appraisal of IE program implementation by the Regions. Attached is a one page summary sheet which highldghts the important aspects of our draft appraisal program (Enclosure 1).
1 We plan to accomplish our appraisal through in-office staff reviews of inspection and enforcement documentation and appraisal trips to the field and the Racional Offices. The Commission has requested that we include a limited program of independent verification of inspection findings. We have included this type ,
of an appraisal trip to the field in the appraisal program, albeit on a limited basis.
I have also attached the draft IE Office Procedure 0200 for the conduct of Appraisal of Regional Implementation. We have reviewed this draft procedure within IE. Before implementing this procedure, I tould like to receive your review and coment on the planned appraisal process. Please provide your ,
coments on tne enclosed draft procedure by June 11, 1982. !
r If you have any questions on the appraisal concept or approach, please contact l Bill Brach on FTS 492-4932.
Richard C g
- Youn W Director Office of spection and Enforcement i
Enclosures:
As Stated l I F*18 t'? 14 8 (
g 91 g 6 000017 1
- CONNORO7-060 PDR Mh I y
7 Appraisal cf Regional Implementation '
Of Inspection and Ecforcement Program .
.~ ~
-q
~
Porpose: To determine if the Regional Offices are: -
- a. Completing the program as required, -
- b. Using techaically adequate methods to implement the program,
- c. Implementing the program consistently both within the Region and between the Regions.
To provide report to EDO on Regional Office implementation of the program.
Approach: Annual appraisal of the Regions based upon:.
- a. Staff appraisal activities of Regional inspection and enforcerent reports, correspondence.
and statistical data.
. b. Staff appraisal visits to the field to accompany inspectors oc inspectica and to observe l actual inspection practices.
- c. Staff appraisal visits to the field to verify independently selected inspection findings. -
- d. Staff appraisal visits to the Regional Offices to discuss and verify appraisal findings.
- e. IE management and Regional management meetings to discuss appraisal findings and final report of Regional implementation.
l
~
l C
-,n+se--w-=-m w e metw ,ri-r w n -- --w--- m-ww -*vv- wet-rw= * ,im m -
x = --v e wr -+w r ,-w++. w ew w ow -t-Nv ewe- w -=*-e --~e--- -
e-&w- - - - w+=e-e-- v e .-re- - - - e e --- -
~
- NW -
CA&MINGTON, D.C. 20468
, ',, *+ose
+
oppics op .
INSPECTION MO INp0ACSME U pgg IE OFFICE PROCEDURE 0200 AppRAISat OF REGI0ftAL IMplEMEllTATION 0200-01 PURPOSE This procedure explains the approach and methods used to appraise the NRC Regional Offico implementation of the inspection and enforcement program. As used in this procedure, program appraisal is an :ngoing and systematic process of gathering relevant infonnation and evaluating that information against approcriate criteria to arrive at an informed judgrrent of whether the program is being carried out in an acceptable manner.
0200-02 OBJECTlWS The objective of program appraisal is to periodically provide management with informaticn on'the degree of progrcm completion, of the technical adequacy of program implementation, and of the censistency of implementation of the progran both within and among the five NRC Regional Offices.
0200-03 DEFINITIONS 031 Staff Audit. An ongoing, systematic check, review, and evaluation of documentation, correspondence, and statistical inspection data pertaining to the Regional Office implementation of the inspection and enforcement program. Staff audits are conducted in NRC-IE Head-quarters offices with frequent telephone contact with the Regional Offices, as appropriate.
032 Aporaisal Trio. Travel by IE staff to accompany inspectors and ob-serve inspector practices during the conduct of selected inspection procedures. Travel by !E staff to NRC Regional Offices to discuss specific observations and to verify findings. Travel by IE staff to licensee sites to independently verify inspection findings.
033 Annual Accesisal. A documenta. overall evaluation of the manner in which the inspection and enforce ent program is iraplemented by the NRC Regional Offices. The annual aceraisal is basec upon the find-ings and conclusions obtained througn staff audits and appraisal trips to the field and Regional Offices for the prograns and program areas selected for appraisal. The results nf the annual aporaisal are cornrunicated to the NRC Regicnal maragement and to the E00.
031 prneram. A defined crogram in the IE Manual. The programs are as follows:
- a. Reactor Inspection Pregram (IE Manual Chacter 2500).
Issue Date:
m m - c ,, _%,y -- ----- ---
. '., b. Fuel Cycle Facility Inspection Program (IE Manual Chapter 2600).
- c. Licensee Contractor ano. Vendor Inspection Program (IE Manual Chapter 2700).
- d. Materials Inspection Program (IE Manual Chapter 2800),
- e. Enforcement Program (IE Manual Chapter 0400 in revised Table of Contents, old number 0800).
035 Program Area.
Sets of inspection activities characterized by similarities of disciplines involved and activities performed. The program areas are as follows:
- 1. Civil engineering and construction
- 2. Mechanical enginering and construction
- 3. Electrical engineering and construction 4 Instrumentation engineering and construction
- 5. Quality assurance
- 6. Operations safety
- 7. Criticality safety
- 8. Maintenance of systems and equipment
- 9. Surveillance testing
- 10. Fire protection
- 11. Refueling
- 12. Radiation Protectfon
- 13. Radioactive waste management 14 Effluent controls
- 15. Environmental monitoring
- 16. Transportation of radioactive materials
- 17. Nuclear material control and accounting
- 18. Physical si.curity '
- 19. Erergency preparedness
- 20. Decommissioning and decontamination t
- 21. Other activities as designated by the Director, IE 0200-04 RESPONSIBILITIES AND AUTHORITIES 041 ,0irector. Office of Insoection and Enforcement:
- a. coproves poilcies, criteria and standards for appraisal of program iciplementation by the NRC Regiona! Offices.
- b. based upon recommendations frott IE Division Directors, approves the specific program and/or program areas for the annual IE program appraisal,
- c. coordinates the scheduling of appraisal taips to Regional Offices and licensee facilities.
Issue Cate:
swm no o aagwwo a wpougasvm -- - - - - - - - - - - - -
0200
- d. approves and provides the annual reports of tho appraisal of regional implementation to Regional management.
4 t
e, approves and provides the annual report of the appraisal of ;
Regional implementation to the Executive Director for f Operations.
l 042, Director o' Division of Reactor Programs, Director of Division of Fuel, Materiais and Safeguards, ano_ Direccor of Enforcement Staff: .
i
- a. are designated as "program Directors" for the inspection and i
enforcement program. (
- b. recomend specific program and/or program areas for the annual !
IE program appraisal to Office Director, IE.
- c. serve as IE manager for program appraisal activities within 9
assigned program areas or as designated by Director, IE.
- d. designate supervisor or senior staff member as IE staff pro- .
i ject leader to plan and carry out program appraisal activities !
j for selected program areas-i ;
1 e. coordinate u,= scheduling of appraisal trips to .egional offices l
) .
and licensee facilities with the Director, !E. !
i i
- f. approve trip reports of staff program appraisal activities and [
transmit them to Regional Admink,;.rators. ;
g, coordinate the development of comprehensive program appraisal reports cs designated by the Director, IE.
043 Directer of Divisior of Engineering and Quality Assurance and Director of Divisien of E%rgency Frecaroness: L
- a. recomend specific program areas for the annual IE program !
apr.aisal to the appi. riate program Division Director,
- b. recetend staff members to the appropriate program Divisier Director to assist in the appraisal of selected program areas.
0200-05 PROGRAM APPRAISAL. SYSTEM l I
051 Program appraisal is a planned process conducted in a twelve-month cycle. The appraisal "year" begins January 1 and ends December 31. l The program areas to be appraised during ,
the appraisal year are recomended to the Director, IE, by each [
program Director. The Directors of DE0A and DEP provide input to i the program Directors in this selection process. The number of !
progran areas selected may stry from year to year and some program j areas may be selected noce frequently than others. It is anticipated that enly a few program areas will te selected for apprasial in each .
Oregram identifiec in 034 above. !
Issue Cate:
r wesuvuur_w unnnswuomaavaWVon The program Directors will prepare appraisal plans for each of the selected program areas. The Directors send the appraisal plans to Regional Administrators through the Director, IE. The plans rc.ay be revised during the period to redirect efforts and to incorporate views of the IE Director on programs and program areas that should receive emphasis or detailed attention. It is not anticipated that a detailed review of all program areas or of all aspects of a program area will be required to produce sufficient information on which to base the appraisal of a program.
A large protion of the appraisal is accomplished during the year through the IE staff audit activities performed in the office. The staff audit activities performed in the office will not necessarily be limited to only the program areas selected for appraisal; for example, inspection reports covering various aspects of the inspection l program and MIS and 766 data may be reviewed throughout the year.
However, the majar portion of the staff audit activities will be concentrated on those program areas selected for appraisal.
The staff audit activities are supplemented by !E appraisal trips to licensca facilities to observe inspector practices and to cenduct independent verification of inspection findings. Appraisal trips will also be conducted at Regional Offices to discuss findings from staff audits and appraisal trips and to obtain additional information not available through those appraisal means. Records are maintained of the results of audit activities and trip reports are written to
! document all appraisal trips. These records and reports form the basis for annual appraisal reports.
The annual appraisal report summarizes the basis and judgments made regacding the adecuacy of program implementation. The report ad-dress =: how the program is being carried out by the Regional Offices in terms of completeness, adequacy, and consistency. The draf t report is provided to the Regional Administrator and is discus-sed at a meeting between IE and Regional management.
Regional comments and follewup activities as a result of problems identified ,
by the apprasial are included in the final report. The results of l the program appraisal of the regions are summari:ed in a report to 1
the Executive Director for Operations.
052 4
General Conduct and Cecedination The program Director will designate individuals as project leaders with the responsibility to plan and carry out appraisal activities for selected programs and program area (s). Staff members will be selected to perform the staff audits and appraisal trips in each >
program area and these activities will be coordinated by the desig.
nated project leaders. Staff members from CECA and DEP will be ;
l selected as appropriate as project lescers or team merbers for the ;
i l
Issue Date: .
~
wyyW yeay ev ww ye m le m eeeae m ea 6200 i
appraisal activities. The program Directors will identify the project leaders and appraisal staff in the appraisal plans (051, above) sent to Regional Administrators.
The project leader will develop plans and schedules for staff audits of regional inspection correspondence, documents and statistical data, appraisal trips to accompany inspectors to licensed facilities :
to observe actual inspection practices or to conduct independent l verification of inspection findings, and appraisal trips to the Regional Office to discuss observations and to clarify findings.
Guidance on the conduct of staff audits and appraisal trips to the field and the Regional Office is provided in Appendix 1. l During the final Quarter of the 12 month appraisal period, the observations and findings are sumarized by the program Directors for each program area covered in the appraisal. The results are evaluated using the appraisal factors and implementation attributes (Sections 06 and 07 below) for detemining the completeness, adequacy and consistency of the Regional implementation of the inspection and enforcement program. An annual appraisal report is prepared for each Region. A draft of the report is reviewed within IE and then transmitted from the Director, IE to the Region. A meeting is held '
between IE and Regional management to discuss the draft report.
Reg onal coments on the draf t report and followup actions as a result of problems identified during the appraisal are included in the final report. A report is then prepared for the ECO to sumarize the results of the IE appraisal of the Regional implementation of the program. Copies of the executive sumaries of each of the five i Regional reports are attached to the report to the E00. Guidance on !
the format and content of the reports is contained in Section 08 below.
l 053 Schedules and Limitations !
- a. Staff audits will be an ongoing activity during the duration l of the appraisal period. ;
- b. Appraisal trips will be sched; led and coordinated with the Director's Office and the Regional Office a minimum of two '
weeks in advance of the olanned travel,
- c. Appraisal trips will be scheduled so that during any 12 month appraisal period: '
- 1. Generally no more than one appraisal trip will be conducted at the same time in one Regi" 31 Office.
- 2. Generally no more than one appraisal trip to a licensee f acility will occur at the same time in ene Region.
- 3. Generally, no more than four appraisal trips, to licensee facilities M11 occur per Region per program area.
!ssue Date:
~
4 Generally, no more than one appraisal trip to the Regional
.' Office will occur per region per program area.
- 5. Generally, no more than one appraisal trip to the field will be scheduled at the same licensee facility.
- d. Annual appraisal reports will be prepared for Director, IE !
review by December 31 of each year. !
- e. The meeting between IE management and Regional Office manage- .
ment to discuss the final report will normally be held during !
January and February each year.
- f. The report to the EDO will be prepared, reviewed and transmitted '
2 by the end of March each year.
0200-06 APPRAISAL FACTORS 4
The appraisal factors described below are to be considered in appraising the Regional implementation of the IE program. The factors have been grouped under three headings: program completion, perfor- t 1
' mance adequacy, and implementation consistency. For each factor, !
attributes are also provided as guidance to facilitate the overall i
- appraisal process.
{ In appraising a program area it may not be necessary (or possible) i to consider all the appraisal factors. At least some of the factors !
regarding program completion, performance adequacy and implementa-tion consistency must be considered for program areas selected for i appraisal.
i 1 061 Program Comoletion Factors. These factors should be considered in determining if program goals and planned accomplishments have been achieved.
- a. Frequency. Inspection Procedures (!P) conducted in accorcance with the prescribed frequency. [
, b. When reouired. Inspection procedures conducted when i requirec by events.
l
- c. Inspection Procedure Comoletion. Individual IPs are closec at 100 percent ccmplete or there is a basis for :
closing IPs at less than 100%. '
I i
- d. Timely !ssue of Documentation. Inspection reports, memoranca, etc. are issueo within a reasonable {
- time frame.
(
062 Performance Adecuacy Factors. These factors should be considered in !
cetermining if program policies, procedures, and priorities are !
interpreted as intended. !
1 p
! i Issue Cate: !
y
. ,, a. Scoce. The scope of the inspecticn activity is consistent witn that prescribed by the IP,
- b. Death. The technical depth of the inspection activity is consistent,with that prescribed by the IP or as warranted by the findings.
- c. Focus on Safety Iisues. When the ins
, tifies issues of safety significance,pection activity the safety issuesiden-receiv3 followup in accordance with established procedures.
- d. Identification of ' nds. Issues are evaluated for significance-
-are the issues *mportant to safety, indicative of a trend or generic issue. Where appropriate, the inspection activit focuses a**.ention on identified trends or generic issues,y
- e. Resource Utilization. Resources are efficiently employed.
Inspection activities are conducted by individuals properly trained and qualified for the inspection activity conducted,
- f. Enforcement Threshold, Enforcement action is initiated where appropriate cedures, and is consistent with published policies and pro-
- g. Enforcement Bases. Enforcement actions are soundly supported by the facts of the case.
- h. Documentation and Corresoondence. Documentation is consistent witn program requirements anc provides a clear record of accomplishment of inspection procedures. Findings are clear and concise. Related correspondence is clear,
- i. Licensee Contacts. Meetings and contacts with the licensee are in accordance with established policies. This includes entrance i and exit interviews, other licensee meetings, and day-to-day contacts between the inspector and licensee personnel. t 063 Imolementation Consistency Factors. These factors should be con-sicer.:c in ceterrining if the IE program is implemented consistently among the Regional Offices or among similar licensees. (Certain l events that may severely impact a particular region may obviate por- '
tions of this part of the appraisal process. Planned adjustments to the program, based on SALP do not constitute inconsistencies).
- a. Program Comoletion. The degree of program completion by the regions should oe similar. Program completion at different facilities (that are in the same phase) should be consistent,
- b. Scoce. The scope of inspection activity is similar among regions and among facilities.
- c. O eo t'j. The techrical depth of inspection activity is similar !
among regions and among facilities. ,
t
, Issue Date:
- d. Focus on Safety Issues. The regions provide sicilar scope ano oeptn attention and followup to safety issues.
- e. Resource Utilization. Resources are employed in a sit..ilar manner among regions and are consistent with requirements prescribed by the program. The manpower expenditure to com-plete a given portion of the program is consistent among regions (other parameters being equal).
- f. Enforcement Aeolication. Enforcement actions taken for one licensee are similar to those taken for like events in other regions and for other licensees,
- g. Documentation. Similar items are documented in a similar manner.
- h. Licensee Contacts. Contacts and meetings with licensees are similar in frequency, scope and threshold.
0200-07 IMPLEMENTATION ATTRIBUTES The attributes presented in Table 1, characterize the degree of suc-cessful implementation of a selected program or program area for .
each appraisal factor.
0200-08 REPORTS 1
081 General. Reports will be prepared routinely to document and com-municate appraisal findings by IE regarding the implementation of the inspection programs by the Regional Offices. The reports are to be summaries of IE findings based on; continuing appraisal efforts performed in the office, observations and evaluations per-formed in the field, and discussions conducted at the Regional Offices. The reports are not to be a documentation of every detail of the total appraisal effort.
082 Acoraisal Trio Reoorts. Within one week of the completion of an appraisal trip to the field or Regicnal Office the project leader or individual, as appropriate, will provide a trip report to the program Director responsible for the inspection program addressed during the visit. A copy of the trip report will also be provided by the program Director to the appropriate Regional Administrator. The trip report i
will summarke the appraisal activities performed and document any findings pertaining to program completion, performance adequacy or implenentation consistency.
033 Annual Reoorts to Recional Administrators. Annually, during January to Maren of each year, a report will be provided to each of toe five Regional Administrators by the Director, IE. Each report will sammari:e the findings by IE regarding overall implementation of the programs (identified on 033 above) by the Region. Each report will address overall implemertation in terms of program ccepletion, Issue Date: -S-
i
, ,* performance adequacy, and implemen@ation consis%ency. Each specific :
program (i.e., Reac%or inspec% ion program, Fuel Cycle inspection !
program, etc.) need not be addressed individually, but may be discussed as approprinte, in the three sections of the report. IE management will provice and discuss the report witn the Regional l Administrato' prior to issuance of the final report. Regional i comments on tne draft report will be included in the final report. ,
k .
In addition to the basic executive summary type rapcrt attachments .
t will be provided which separately summarize findings re,garding each
! of the programs as identified in Section 033 which were appraised i during the previous 12 months. Each attachment will address imple-3 mentation of the programs in terms of program completion, perfor- l 1 mance adequacy, and implementation con;sistency.The attachments l
covering the reactor inspection program and the vendor inspection :
y program will be provided to the Director, IE by the Director, l Reactor Programs Division; the attachments cc,vering the fuel cycle j i inspection pr7 ram and materials inspection program will be provided ,
i to the Director, IE by the Ofrector, Fuel Facilities, Materials and i
Safeguards Division; and the attachment covering Enforriment will be provided by the Director, Enforcement Staff.
1 084 Annual Report to E00. Annually, by the end of March of each year, a I
) report will oe provided to the E00 by the Director, IE. The report j will summari:e the findings by IE regarding implementation of ;
) inspection programs by NRC Regional Offices. The report will address ,
regional implementation in ters.s of program completion, performance adequacy, and implatentation consistency. Specific Regions need not i j
be addressed individually, but may be discussed, as appropriate, in j the three sections of the recort. A copy of the Annual Reports to l Regional Adr.inistrators (excluding attachments and details) for each a
2 of the five regions will be attached to the report to the E00, !
t 1
END l
i !
l l
l r
5 i
. i 4
l J <
I Issue Oate:
1
1 Tabla 1 1 .
I Implementation Attrit stes for Each Appraisal factor ~
l full Implementation Acceptable Implementation Limited Implementation Program Completion factors frequency Inspection procedures cone
- acted Inspection procedures Numerous instances of IPs not within prescribed fregrancy in usually conducted within completed within prescribed fa almost all cases. frequency. quency.
Deviations from prescribed fre- Deviations from prescribed Numercus instances of devia-quency dictated by SAtP are met frequency are usua:ly in tions from prescribed frequerx in almost. all cases (reactors accordance with SAtP re- SALP requirements seldom met only). quirements (reactors only). reactors only).
When Required IPs are conducted in nearly all IPs are usually conducted Numerous instances of IPs not cases when required by events or when required by events or conducted in respcase to licensee activities. licensee activities. licensee events or act.ivities.
IP Completioa Nearly all IPs are fully IPs are usually fully com- Numerous instances of IPs that pleted (closed at 100%). completed. are incomplete and "window of opportunity" has passed.
I %ely Issue of limely issuance in nearly all Documents are usually issued Documents are often late; ex-1)ocuments cases. in a timely manner. tensions *.o deadlines ate fre-quently soucht.
Perfornance Adequacy Factors Scope Scopc of inspection effort is Inspection effort usually Inspection effort not. thorough consistent with effort prescribed. meets prescribed scope. and omits portions of pres-scribed effort.
11epth lhe technical depth of the inspec- Technical depth of inspec- Inspection effort reveals in-tien effort is consistent with tion effort is usually stances of inadequate tech-the requires >ents of the IP in consistant with IP require- nical depth or merely a cursory nearly all cases. ments. review of significant issues.
~ '
l Implementction Attributcs Fcr Each Apprcisn1 Factor -
l 1 1 -
1 f
full Implementation Acceptable Implementation Limited Implementation l Depth (continued) Ihe experience and training of Inspector experience and Weaknesses in inspector exper l l the inspectors is commensurate training is usually com- lence and training found to w;th tne essigned inspection mensurate with the assigned be a major contributing activity. inspa_.: tion activity. factor to poor understanding of inspection requirements or criteria.
locus on Safety Inspection effort focuses on sig- Inspection effort usually Numerous instances of nificar.t safety issues; identified focuses on significant safety issues that were issues receive prompt followup safety issues; prompt fol- overlooked; followup of in nearly all cases. lowup is usually made. identified issues is not timely.
l Significant safety issut. that Newly identified safety Newly identified safety are identified but not addressed issues are usually for- issues are frequently not in the prescribed program are warded to IE for incorpora- forwarded to IE for incor-l forw.arded to ![ for incorporation. tion in prescribed program. poration in the prescribed program.
l loentification of Irends or generic issues are iden- Trends or generic issues Numerous instances anert:
Ircods tified as appropriate in ner-ly are usually identified. trends or generic issues all cases. were overlooked.
Significant trends or generic Significant trends or Numerous instances where issues are identifico to licensee generic issues are usually trends or generic issues I and appropriate HRC organization identified to licensee or are not identified to l in nearly all cases. appropriate NRC organiza- licensee or appropriate Lion. NRC organization.
Resource utilizatico Resources are efficiently employed Resources are usually effi- Numerous instances of i in nearly all cases ciently employed. resource inefficiency.
Consistent evidence of prior plan- Usually evidence of prior Little evidence of prior i
ning and assignment of priorities. planning and assignment planning or assigrunent
! of priorities.
l
-g-
~ ~
ylementation citributco For Each Apprcisal factor
- full Impics.entation Acceptable laplementation timited Implementation i
l Inforcement Enforcement strong, prompt, and Enforcement usually aggres- Numerous iv,tances where threshhold consistent with policy; level of sive, prompt and consistent enforcement action are weak, sanctions applied (proposed) are with policy; level of sanc- tardy, or inconsisient with appropriate to circumstances in tions applied (proposed) is policy; level or sanctions nearly all cases usually appropriate. applied (proposed) may be inappropriate.
i inforcement Bases inforcement actions are soundly Enforcement actions are Numerous examples where i
supported by the facts in nearly usually well supported; bases for enforcement i
all cases; little additional some additional research actions are unclear; fre-research is required to complete may be required to com- quent cases where clarifi-staff work in support of enforce- plete staff work in sup- cation is snught by the ment actions. port of enforcement action. licensee prior to his response to m enforcement action.
Ilocumentation and in nearly all cases, inspection Inspection reports usually Correspondence Numerous instances of activity and relationship between provide a clear record of unclear inseection reports; l report an1 inspection procedure inspec ion activity and fragmentary narratives; is clear; findings are clearly establish relationship many typographical and format stated; very few typographical between report and inspec- errors.
and format errors. tion report; findings are usually clearly stated; occasional typographical format errors.
l Hemoranda, forms, etc. related La Memoranda, forms, etc. re- Numerous instances of incom-inspection elfort are completed lated to tie inspection plete or poorly completed meme properly in nearly all cases. efforts are usually com- forms, etc. related to inspec-pleted. Lion.
Licensee Contacts Licensee contacts and meetings are Licensee contacts are Numerous instances of unclear, clear, concise, and well directed usually timely, clear and tardy or confusing contacts in nearly all cases, contact with well-directed. with the licensee.
licensees is is timely.
l !
Implement tion Attributes For Each Apprcisal factor *
- 4 full Implementation Acceptable Implementation timited Implementation Licccsee Contacts Initiative is displayed when Licensee usually contacted Many contacts with licensee (continued) contacting licensee in response in respanse to events. are initiated by licensee.
to events.
Implementation Consistency factors Program Completion Degree of program completion is Degree of program comple- Numerous examples of incon-consistent in nearly all cases tion is usually consistent sistent program completion.
(when compared among regions or among facilities).
Supe Scope of inspection effort is con- Scope of inspection ~ effort Numerous examples cf incan-sistent in nearly all cases (when is usually cos.sistent. sistent scope of inspection compared among regions or among effort.
facilities).
Depth The depth of inspection effort is The depth of inspection Numerous ex3eples of incon-consistent in nearly all cases effort is usually con- sistent depth in inspection (when compared among regions and sistent. etfort.
among facilities).
locus on Safety The f ocus on safety issues is con- lhe focus on safety issues Numerous examples of incon-sistent in nearly all cases (when is usually consistent. sistent focus on safety compared among regions and as.ong issues.
facilities).
Nesource lhe use of resources is consistent The use of resources is Numerous examples of incon-Utilization in nearly all cases (when compared usually consistent. sistent use of resources.
among regions and anong facilities).
tutorcement In nearly all cases, the applica- The application of enforce- Numerous examples of inconsis-Application tioni of enforcement sanctions is ment sanctions is usually sistent application of enforce-consistent for all events (when consistent. ment sanctions.
compared among regions and among facilities).
Implementction Attributes Fcr Each Apprcisal factor * ~~
full Implementation Acceptable Implementation Limited Implementation Doceanenta tion In nearly all cases similar Similar items are usually Numerous examples of incon-items are documented in a documented in a similiar sistent documentation.
similar manner (when compared manner.
among regions and among faci-lities).
Licensee Contacts In most cases, contacts and Contacts and meetings with m etings with licensees are Numerous e. g les of incon-licensee are usually con- sistent contacts .~*
coasistent (when compared sistent. meetings with licensets.
among regions and among faci-lities).
-a f
. . m -w <ee...m .s..
APPENDIX 1
.' APPRAISAL GUIDANCE '
Pert I REVIEW 0F PROGRAM 0000 MENTATION A. PURPOSE Inspection repotts and correspondence are reviewed to determine if they conform to expressed policy in general coverap and format, and if violations and problem areas are adequately substant.ated as a basis for enforcement action.
The purpose of the review is not to ait-pick the writing style but to point out those obvious cases where reports are toc wordy or overwritten in extraneous detail and therefore confuse the issue; not clear because of lack of detail, >
information and substantiating facts; or not clear because of stilted, obtuse, bureaucratic wording or gobbledygook.
- 8. EVALUATION GUI.'ANCE
- 1. Specific Standards -
l l These are requirements for format, cor. tent and style specified in IE 1005.* While they are specific standards, their application will re-quire judgment by the reviewer. The standards are listed below with references to subsections of IE 1005. Critical elements of certain !
standards are summarized. Items "a" through "b" cover final inspec-l tion reports. '
1 l a. Cover page (IE 1005-310),
r
- b. Summary (IE 1005-311). The summary should be a brief narrative description of the scope and results of the inspection. It t should specify the dates of inspection and provide a description of the subjects or areas examined, the amount of direct efforts expended in inspector-days (or hours) at the site, and the re-suits of the inspection. Examoles are shown in Enclosure 3, ;
IE 1005, i
- c. Persons contacted (IE 1005-321). ;
- d. Action on nrevicus inspection findings (IE 1005 321), This i shoulo oe a brief statement of previously noteo enforcement and !
unresolved items examined curing the inspection, including de- !
scription of findings and a notation wnether the item remains !
open or closed. Examples are shown in Enclosures 5 and 6. IE 1005, l f
t
- The reference is to IE Manual Chapter 1005. In the revised Table of Contents.
l this Chapter will be reissued, when fevised, as Chapter 0610. l t
Issue Date.
_ _ _ ._m-----------------_-----. . - - - - - - - - - - - - - - - - - - - -_-
y.2-- n ivwwu
. .. e. Licensee or vendor in9ernal audi9s (XE 1005-323),
- f. Functional or crocram arcas inspected (IE 1005-324), '
This is the main body of details of the report which describes the areas inspected and resultant findings. Complete informa-tion must be provided to substantiate violations and other iden-tified safety problems. The general standards in 2., above, apply in addition to the specific standards in IE 1005-323 and 1005-202. l The report narrative describing violations and safety problems must be complete and purely factual statements of what the i licensee did or did not do and the items of proof relied upon !
to establish each fact. The report should set forth only the basic facts. It must be objective and present the facts in simple, clear and non-technical language with appropriate re-ference to the regulatory requirements and safety standards be-lieved violated.
Less information is required for tnose areas inspected and found acceptable, as described in IE 1005-324, 9 Unresolved items (!E 1005-325),
- h. Exit interview (IE 1005-326),
- i. Report details for vendor inspections (IE 1005-34),
j, Report details for enforcement conferences (!E 1005-34),
k, Oraft inscection report (!E 1005-40),
C, DOCUME!!TATION l
As problems are identified, the staff should ensure that the project leader l and management are informed of the prablem in a timely manner, The documentation of the staff and audit activities is important since it will form the basis for the conclusions and findings presented in the annual reports. I Periodically a sumary report should be completed to document activities completed and findings. The report may be structured by Region, program L area or discipline as appropriate. The report should cover: l l
- 1. Type of documentation reviewed and program area or discipline i covered. l I '
l 2, Number of documents reviewed, method used for selection and tire period covered by the documents. ;
- 3. Listing of trends or concerns for oiscussion/ resolution with the Regions, All "negative" findings should be discussed with the l Regions and the results of these discussions should be documented, i l
!ssue Date:
i
. . " ' ~ ' " ~
ve n uese.iGVx T ,
.. 4. Conclusions or recommenda% ions drawn from %he review.
cart II REVIEW OF MIS DATA A. INTRODUCTION AND PURPOSE >
There is a significant amount of information available from MIS which will <
form the basis for evaluation of aspects of program implementation. No !
attempt is made here to enumerate or describe all the data in its various !
forms. Rather, the guidance gives examples of some of the data that are ;
available.
- 8. General Following are examples of data that are available in each pro gram area (reactors, verdor materials, and fuel facilities). gram Theand pro-informa- i tion can be obtained for all regions and separately for each region. The '
data can be obtained for separate licensees, and this may be desirable par- i ticularly for detailed evaluation of program implementation for reactors.
- fuel facilities and the priority I materials licensees, t
- 1. Total number of inspections for a given period, incl ~uding inspections .
under a particular inspection procedure. t
- 2. Inspection time expended per procedure. *
- 3. Number of inspections that were clear, and the number which identified violations of requirements.
4 Inspection time expended per clear inspection and for those which !
identified violations.
- 5. The number of violations, total and by severity classification.
{
{
- 6. The text of citations. '
- 7. The number of citations against a specific regulation or requirement.
C. EVALUATION AND GUIDANCE I
The evaluation of program implementation includes review of MIS data ano !
discussions with the regionai offices. The evaluation of MIS data is per- t formed by phase of program and by inspection procedure. Information to be considered is discussed in the following sections. ,
- 1. Quantitative Evaluation--Phase / Program
- a. Completion of inspection phase:
(
- i j (1) periodic inspection procedures are completed within !
i prescribed time intervals. '
^
t 8ssue Date: .
9
.. (2) one-time procedures are completed uithin prescribed time intervals. I i
- b. Reduced frequency of selected inspection procedures for eligible dockets are ,
(1) identified by the regional office.
(2) conducted within prescribed time frames. ;
- c. Resource expenditure by inspection phase: i (1) acttial man-hours expended per docket are reasonably con s i s te.. with regional anc' national means, budgetary goals, i and pla td accomolishments.
(2) variance of actual man-hours from expected man-hours are within reasonable limits. ;
(3) resource expenditure is reasonably consistent with regional re curces, staffing, and manpower planning factors. 3
- 2. Quantitative Evaluation Criteria--Individual Inspector procedures
- a. Resource expenditure: ;
(1) actual man hours expended are reasonably consistent with ;
regional and national mean.
L (2) variance of actual man-hours expended from expected man-hours (
are within reasonable Ilmits. L (3) percent completion when procedure ir closed is reasonably I consistent with program direction; tor example, in absence y of other instructions, percent completion should normally '
be 100%.
D. DOCUMENTATIC-f Periodically,butatleastoncompletionofamajorsegmentoftheMIS review effort, prepare a summary report for use in compiling the quarterly l report ori Regional implementation. The report may be structured by Region, i program area, or discipline as noropriate. The report should cover: l r
1, Specific type of data reviewed.
- 2. Time period covered by the data. !
L 3.
Trends or anomalies in the data which should be discussed with tr.e I Regions I
Issue Cate:
wui d isai ouiuaiite ng g gg
- 4. Results of discussions (by phone) with the Regions, if discussions
.' are held. All "negative" findings should be discussed with the Regions and the results of those discussions should be documented.
- 5. Conclusions or recommendations drawn from'the review.
Part III APPRAISAL TRIPS TO OBSERVE INSPECTOR PRACTICES A. PilRPOSE The purpose of accompanying inspectors during the course of their work is not to inspect the inspector and identify "noncompliance" by the inspector with the inspection procedures. IE staff accompany inspectors to determine if inspection procedures are interpreted and implemented in a complete, adequate and consistent manner.
Ir carrying out this part of the appraisal process, the emphasis is en determining completion of the irn.ent of the program (other expressions of the same idea are: (1) completion of the substance of the program, (2) s.*stantially complete, and (3) in all important aspects, complete).
Significant variations either in excess of the intent or_less than the intent a to be examined closelv to find ut the reason so problems may be correrced.
Limitation: The IE observec is not inspecting the licensee and should not actively engage in anv actual inspection. The reviewer's job is to ab-serve the inspection activity.
B. EVALUATION GUIDANCE The guidance is general and given in terms to aid the reviewer in plan-ning and carrying out the review activity. Thc reviewer trust have a thorough knowledge of the program requirements and inspection procedures under which the inspection is being performed, and a general idea of how the inspector intends to carry out the inspection.
- 1. Using the inspection procedure (s) and the inspector's plan, under-stand what the inspector intends to inspect and how he plans to do i t. Do not question the adequacy of these plans, advise the inspec-tor hnw to inspect, or question his methodology or techniques in front of the licensee.
- 2. Observe the inspector during the entire time he is inspecting each part covered by his plan. Take notes so that you can remember what the insp?ctor did and you can later document how major parts P the insp:ction were caraied out. Note the t'me spent on each function or separable part of the inspection and make notations to explain times which seem to be long (for example, considerable effort require.d to confirm, clear, or resolve problem issues).
Issue Date:
. iO::00 A::endix 1 Acoraisal Guicance
- 3. Understand what the inspector is looking for in a records review, what specificaly he is looking for in observational activities, and
. what he is probing for in discussions with licensee personnel. This understanding is arriveo at before or after the fact, of course, and in a manner not to interrupt.the inspector (ion).
4 For each segment of the inspection evaTuate '.<hether the activity being done by the inspector is consistent with the requirements and guidance of the inspection procedure. Remember the emphasts is placed on determining completion of the intent of the program, and not each specific detail of the procedure. If the inspector is looking at matters you believe not within the intent of the overall program, find out why.
C. DOCUMENTATION For each inspection procedure or logical inspection procedure grouping or area, prepare a brief written evaluation. The following topics should be considered in the reports:
- 1. What was inspected. ~
- 2. How the inspection was accomplished: methodology, what was observed, how many observations were made, who was talked to, what records were checked, and for what purpose.
- 3. Whether the inspection verified or evaluated what the progrn intended? If not, explain.
4 Whether the inspection covered more than was intended by the pro-gram? If so, explain.
- 5. Whether the inspection covered less than was intended by the program?
If so. explain. (If this was the inspector's plan, or if all parts could act be done for some reason, say so.)
- 6. Amount of time taken to accomplish the inspection, or logical segments thereof? If more or less time was taken than the evaluator believes necessary, state how much time should be spent and the reason for the ,
difference.
Part IV APPRAISAL. TRIPS TO INDEPENDENTLY VERIFY INSPECTION Fit' DINGS A. PURPOSE The purpose of IE staff members traveling to licensee sites and indepen-dently conducting selected inspection activities is to provide feedback on the completeness, adequacy and consistency of regionally reported inspection findings. It is anticipated that such appraisal trips by IE will comprise a small percentage of the total appraisal effort. In some program areas, these appraisal trips may not be planned or warranted.
When such appraisal trips are planned, specific inspection areas or procedures will be selected for indendent verification.
Issue Date:
. . owppWWWY gyrW7d4 e 0000 ApDend{x 1
.- r i
Detailed pre-planning is required to ensure that a ccmparison of regional inspection findings and IE appraisal findings can be made. In this situation much of the appraisal effort will be focused on independently inspecting documentation or operational charactertics which remain -
relatively constant with time.
B. EVALUATION GUIDANCE The' guidance is general and given in terms to aid the IE Staff member 5 planning and carrying out the inspection activity. The reviewer should have a thorough knowledge of the program requirements and inspection procedures under which the inspection is to be performed and a specific plan of how the he intends to carry out the inspection. The reviewer shou'.d review and have thorough knowledge of the facility and operations at which he will conduct the inspection.
- 1. Make arrangements nich the licensee and inform the Regional Office at least 2 weeks ir. advance of the planned travel. Coordinate the s:heduled travel with the Office of the Director, IE before finalizing travel arrangements.
- 2. Using the inspection procedure (s) and the inspection-plan, understand what, you intend to inspect and how you plan to do it. During the '
inspection take notes so that you can remember what you did. Later you can document how major parts of the inspection were carried out.
Note the time spent on each function or separable part of the inspection and make notations to explain times which seem to be long (for example considerable effort required to cor. firm, clear, or resolve issues).
. 3. Understand what you are looking for in a records review ano what specifically you are looking for in observational activities.
- 4. Document yo'Jr activities and findings (See Section C below).
- 5. Compare your inspection activities and findings with the Region's.
- Examine for each activity and compare:
- a. Scope of coverage
- b. Depth of inspection !
- c. Consistency of inspection activities
- d. Findings and related significance
- e. Time expanoed C. DOCUMENTATION For each appraisal trip to independently verify regional inspection findings, prepare a trip report which addresses:
- 1. IE Field Inspection
- a. Wat was bspected?
Issue Cate: 7-
[
a+wrer LcIvrancn
.. b. How the inspection was accomplished: methodology, what was
.- observed, how many observations were made, who was talked to, what records were checked, and for what purpose?
- c. Whether the inspection verified or evaluated what the program intended? If not, explain.
- d. Whether the inspection covered more than was intended L. the program? If so, explain.
- e. Whether the inspection covered less than was intended by the program? If so, explain,
- f. Amount of time taken to accomplish the inspection, or logical segments thereof? If more or less time was taken than the reviewer believes necessary, state how much time should be spent and the reason for the difference.
- 2. Comparison of IE Field Inspection to Regional Inspection Results
- a. Scope of coverage
- b. Depth of inspection
- c. Consistency of inspection activities ~
- d. Findings and related significance
- e. Time expanded
- f. Conclusions on acceptability of comparision and regional implementation and documentation of the program.
Part IV APPRAISAL TRIPS TO REGIONAL OFFICES A. PURPOSE IE appraisal trips to regional offices should be the culminating activity in the staff review process; that is, after a review of MIS data, review of a sample of program documents, and appraisal trips to the field. Gen-erally. the purpose of the appraisal trip to the Region 01 Office will be to discuss the results of staff audit activities with regional supervisors and inspectors and obtain additional and clarifying information, and to provide and obtain information of mutual interest regarding programatic problems and improvements. The trips are planned with definitive objectives.
They are not made for the purpose of searching for targets of opportunity in the program appraisal process.
B. PLAtittitlG Att0 C0t40VCT 1.
The results of the staff audit activities should be assembled and organi:ed to aid the development of the plan and agenda for the trip.
The plan and agenda may be organized on the program and prngram areas to be discussed.
!ssue Date:
UzuU Acceneix y
- i
- 2. The agenda should be complete and descriptive enough to clearly
- - identify the main thrust and purpose of the topics to be covered.
For example, if part of the trip is to obtain information or view-points, the important relevant questions should be spelled out in the agenda. Likewise, if results of audit activities are to be discussed, the results should be (or have been) provided to the regional office and the topic (s) described in the agends.
- 3. - One part of the meeting in the regional office should bc devoted to an open discussion with the inspectors (group or individuals) for obtaining, first-hand, their views and comments on the IE appraisal program and program findings.
- 4. Time and dates of the trips to the Regional Office will be arranged with the regional staff that will be involved. The agenda will be sent by memorandum to the Region;l Administrator through the Director, IE. This written notice should be in the 5:nds of the region at least two weeks in advance of the scheduled visit.
- 5. All appraisal trips are to be coordinated with the Office .f the Director, IE.
.~
C. OCCUMENTATION "q completion of each appraisal trip to a Region a report shouid be prepared.
The report should cover:
- 1. Specific problems /cencerns discussed including source or basis for the concern e.g. , MIS data, inspection reports, observations f rom inspec 1r acccmeaniments, etc.
- 2. Personnel contacted for each item.
- 3. .79 solutions / conclusions pertaining to each item on the agenda.
4 Listing of Regional comments regarding weaknesses or proposed changes in the appraisal program.
Issue Date: