ML20058B940
| ML20058B940 | |
| Person / Time | |
|---|---|
| Issue date: | 05/31/1990 |
| From: | NRC OFFICE FOR ANALYSIS & EVALUATION OF OPERATIONAL DATA (AEOD) |
| To: | |
| References | |
| PROC-900531, NUDOCS 9010310099 | |
| Download: ML20058B940 (32) | |
Text
_.
FDR 4
ENCLOSURE 3 l
2 t
l DRAFT i
GUIDELINES FOR OIAGNOSTIC EVALUATIONS-MAY 1990 t
REVISION 1 t
t l
U.S. Nuclear Regulatory Comission Office for Analysis and Evaluation of Operational Data.
Division of Operational Assessment Diagnostic Evaluation and Incident Investigation Branch
. le 306 9010310099 900531 03Oh?9 O
9 PNU 0o1
e e
l GUIDELINES FOR THE PREPARATION FOR THE DIAGNOSTIC EVALUATION DET Guideline 1 1.1 Purpose i
This guideline provides guidance, responsibilities, and checklists to prepare j
for a diagnostic evaluation.
- 1. 2 General NRC Manual Chapter (MC) 0520, NRC Diagnostic Evaluation Program, defines the scope and basic requirements for the diagnostic evaluation for reactor facilities.
A diagnostic evaluation is a formal, independent,,in-depth assessment of Ilcensee plant safety and performance which emphasizes root cause determination.
During the semiannual NRC senior managers' meeting, recommendations are made to the Executive Director for Operations (ED0) by the Directors of the.0f fice for Analysis and Evaluation of Operational Data (AE00) and the Office of Nuclear Reactor Regulation (NRR), and the Regional Administrators regarding sites which should be considered for diagnostic evaluations to provide additional information on licensee performance. With these recommendations as a basis, the EDO selects the site (s) to receive a J
diagnostic evaluation (diagnostic), the required completion schedule, and the i
broad evaluation objectives.
Upon receiving this direction, the Director, AE00 initiates actions within the Division of Operational Assessment (00A) and the Diagnostic Evaluation and Incident Investigation Branch (DEIIB) to plan and i
prepare for the diagnostic evaluation.
l 1.3 Initial Activity Preparation and planning for a diagnostic generally takes several weeks.
OEIIB staff are to:
l Identify evaluation goals and objectives through discussions with NRR, the o
Region, and EDO staff to gain more information about the designated site.
Collect and review NRC and' licensee documents related to plant performance, o
design, and regulatory issues.
Review documents such as Systematic Assess-i ment of Licensee Performance (SALP) reports, Regional and Headquarters inspection reports, performance indicator (PI) data,- site Technical Specifications, Final Safety Analysis Report (FSAR), asentents and other licensing documents, licensee event' reports (LERs) and LER analyses from AE00.
Identify proposed functional areas for evaluation to meet the overall o
goals and objectives of the EDO.
Identify potential team members for EDO approval.
o 1
i
i e
e 2
1.4 Notification and Preparation After the initial proposed objectives and functional areas have been identified, the licensee and NRC Regional and Headquarters offices are notified as appro-i priate.
Activities in this time (about I to 2 months prior to the beginning of the evaluation) include:
A memorandum will be prepared for EDO signature which formally requests o
AE00 to perform a diagnostic evaluation at the selected plant and directs the formatisn of a diagnostic evaluation team (OET).
A letter will be prepared for the EDO signature which informs the licensee o
of the NRC's plans to conduct a diagnostic evaluation at the selected plant.
Initial input from the Regional office regarding licensee and plant o
performance and problems will be collected.
Communication with the Region on licensee and plant performance will continue throughout the preparation
- period, Proposed onsite evaluation dates will be negotiated with the Regional o
office and the licensee.
Regional personnel will normally communicate with the licensee to ensure that there are no conflicts with other inspections or operations activity.
i o
Contractors to support the evaluation will be selected and procured.
NRR and the Regional offices (other than the Region in which the plant is o
located) will normally provide evaluators for selected functional areas.
Negotiations to obtain this support should begin as soon as possible.
l A list of pertinent licensee documents to be obtained during the advance o
trip should be developed with input from the team members, An advanced trip to the licensee's ' facilities.should be conducted.
The o
purpose of this trip is to provide the licensee with the purpose and pro-cess for the evaluation and to bring back documents identified as necessary for the evaluation.
Preliminary logistical arrangements (e.g., motels, meeting rooms, dining facilities, and maps) should also be identified during the advanced trip.
Negotiate with the regional office regarding the' SALP appraisal period to o
ensure that the SALP and DET reports will not be issued within six months of one another.
- 1. 5 Preliminary Evaluation Plan Approximately six weeks prior to the beginning of the onsite evaluation, a preliminary diagnostic evaluation plan is prepared. This plan should include:
o A detailed evaluation schedule.
o' Functional areas to be evaluated.
2 Ib h
--=
=m-
+ri-+ >.
w n --- -
__,ms--w--e
e Team composition by name and DET assignment, o
o Specific team goals and objectives.
o Basis for the selection of the plant, Plant system (s) that the team will review in-depth.
In selecting a plant o
system, its risk significance, accessibility for walkdown, operational performance, and the amount of significant plant modifications should be considered.
1.6 Functional Areas The functional areas to be evaluated and the allocated resources will be i
described in the diagnostic evaluation plan.
Some areas may receive limited resources or be omitted based on past licensee performance or other NRC evaluation efforts.
For each functional area an evaluation plan will be developed by the functional area leader with assistance from assigned team members.
The functional area evaluation plan should have the following characteridics:
Contain information and guidance from DEIIB Functional Area Coordinator o
(FAC) files.
No longer than five pages in report outline form, o
o Plant-specific.
t o
Address areas / subjects to be evaluated (including previously identified I
issues).
Provide for an assessment of root causes, o
Contain flexibility to permit the redirection of effort within the o
functional area, as necessary, Consistent with the anticipated report outline such that diagnostic o
objectives will be reached.
Name team member leads for areas / subjects to be evaluated and identify o
the major sources of input information (e.g., document reviews, interviews, observations).
The functional areas which may be evaluated are discussed in the following paragraphs.
1.6.1 Plant Operations and Training (OP)
This area consists chiefly of the activities of the licensee's operating staff (e.g., licensed operators, shif t technical advisors, and auxiliary operators).
It includes licensee activities such as determination of plant conditions.
3
e e
response to off-normal conditions, manipulation of reactor and auxiliary j
controls, plant-wide housekeeping, control room decorum, equipment tagging, and i
control of maintenance and test activities.
This area also in:1udes all activities relating to the effectiveness of the training / retraining and qualification program conducted by the licensee's staff and contractors for the following categories of facility trainir.
1 o
Non-licensed operators o
Control room operators Senior control room operators / shift supervisors o
Instrument and control technicians j
o Electrical maintenance personnel o
Mechanical maintenance personnel o
Radiological protection technicians o
Chemistry technicians Onsite technical staff and managers o
1.6.2 Radiological Controls (RC)
This area includes:
1 Occupational Radiation Safety - including controls by the licensee and contractors for occupational radiation protection, radioactive materials and contamination controls, radiological surveys and monitoring, and ALARA l
programs.
Radioactive Waste Management -. including processing and onsite storage of gaseous, liquid and solid wastes.
s Radiological Effluent Centrol and Monitoring - including gaseous and liquid l
effluent controls and monitoring, offsite dose calculations and dose limits, radiological environmental monitoring, and the results of NRC's confirmatory measurements program, i
Water Chemistry Controls - including primary and secondary systems i
af fecting plant water chemistry, water chemistry control program and-i program implementation, chemistry facilities, equipment and procedures, l
r and chemical analysis quality assurance.
- 1. 6. 3 Maintenance. Surveillance and Testing (MST)
This area includes all licensee and contractor activities associated with preventive or corrective maintenance and surveillance and testing of plant equipment.
1.6.4 Quality Programs and Administrative Controls Affecting Quality (QP)
This area includes all management control, verification and oversight activities l
which affect or ensure the quality of plant activities,. structures, systems, and components.
Principal emphasis will be given to the implementation and effectiveness of quality verification programs along with the history of 4
i o
e i
licensee performance in the key areas of: committee activities, design and procurement control, quality control, inspections, audits, corrective action systems, trending and analysis, and records.
Responsibility for broad, station-wide root cause analysis ar.d assignment resides in this functional area.
J 1.6.5 EngineeringDesignandTechnicalSupport(ES)
This area includes all technical and engineering support provided to the plant relating to plant operations, maintenance, testing, design changes and modifications, systems engineering, facilities engineering, technical program development and implementation, and deficiency / event analysis.
Onsite and of fsite engineering support organizations are normally included in the evaluation.
1.6.6 Management and Organization (M&O) l This area includes an assessment of the effectiveness of licensee management practices and controls to ensure safe and efficient plant operations.
Corporate line and staff functions, including leadership, direction support and oversight
)
of site organizations, and site functions, including plant operations, main.
tenance, engineering, quality assurance and training management are assessed.
Organizational work environment and personnel attitudes relevant to plant safety performance are also evaluated as part of this functional area.
1.6.7 Other functional Areas:
I Other areas such as emergency preparedness, fire protection, licensing and regulatory compliance, security and safeguards, and outage management may be evaluated on a case-by-case basis as appropriate.
l 1.7 Advance Trip An advance trip to the licensee's plant site shall be conducted to explain the purpose and plans for the oiagnostic evaluation, answer licensee questions, make logistical arrangements for the team and obtain previously selected licensee documents (see Appendix C) for background review by the team members.
Characteristics of the advance trip include:
o The trip will normally involve two persons for two days at the licensee's facility. A thorough understanding of the licensee's internal documents and document control system should be obtained. Based on this understanding and prepared standard document lists, document needs-for the team's in-of fice review and for the onsite evaluation should be identified, o
Team activities and interfaces with the licensee should be discussed in detail. The entrance and exit meeting formalities, and the means for connunicating technical information during the evaluation should be
- arranged, o
Administrative and licensee management contacts should be identified, y
Logistical arrangements should be made for the team's onsite office space, o
interview rooms, telephones, printer (s) and photo copy equipment.
5
lo l
1.8 Final Diagnostic Evaluation Plan l
The diagnostic evaluation plan is finalized about four weeks prior to the evaluation.
The final plan contains the schedule, functional areas, team members, and objectives for the evaluation.
The final plan is forwarded to the E00 for approval, distributed to NRC management and provided to each team
- member, 1.9 Logistical Arrangements Three to four weeks before the evaluation is scheduled to begin, DEllB staff i
should ensure that arrangements for travel, motel, and rental cars for all team members have been completed.
Time and location regarding team meetings and entrance and exit meetings with the licensee should also be finalized.
In i
addition, all team members should have the requisite clearance and site access training to permit unescorted access to the vital areas of the plant, otherwise an escort must be arranged.
1.10 Final Notification About three weeks prior to the evaluation, the Director, DOA should transmit a formal notification of personnel participating in the diagnostic evaluation to the Olvision of Resource Management of the involved Regional office to request j
site access for team members.
The letter should contain, as a minimum, the name of each team member, type of security clearance, certification and a i
statement of site access training completion status for all team members.
l l
Final confirmation of the entrance meeting time and place and any final, special, or administrative requirements should also be provided.
I 1.11 Team Meetina and Trainino l
About two to four weeks prior to the evaluation, a 3 day preparation meeting for indoctrination, training, and plant performance review should be held at Headquarters.
Activities during this meeting will normally include:
l o
An introductory discussion of the purpose and importance of a DE.
o The basis for plant selection, o
A detailed review of evaluation goals and objectives, o
Discussion of team operations, organization, and DET member roles and responsibilities.
Distribution of binders containing plant performance evaluation reports, o
DET methods, prior DET reports and logistical information to all team members.
i Briefings by NRR, AE00, and Regional personnel to provide background o
information on licensee and plant performance, Training in diagnostic evaluation techniques, management appraisal, o
and interview techniques.
6 1
,, - +,,
f e
e Instruction in the development and desired content of diagnostic o
evaluation plans, Review of documentation methods used during the evaluation, o
Discussion of control system for licensee documents.
o Detailed discussion of report format, process, and schedule, o
Discussion of time and attendance policy and considerations, o
Availability of PPM /FAC files and DEIIB guidance file and types of o
material that is available.
The schedule for this team meeting will be provided in the evaluation plan and in a separate memorandum from the OET manager.
Approximately two weeks following the first team meding, a second (optional) one day team meeting will normally be scheduled ty the OET manager to discuss, in detail, each functional area evaluation plan.
In preparation for this team meeting, each functional area leader will review appropriate background information to identify licensee performance issues to develop an evaluation plan for his or her functional area.
Consideration should be given to holding this second meeting via phone call (s).
Activities during the second team meeting will normally include:
A presentation discussion of each functional area evaluation plan.
o A review of final logistics and travel arrangements.
o Guidance on the effective use of contractors.
o The schedule for the second team meeting will be provided in the evaluation plan and in a separate memorandum from the DET manager during the first team meeting.
1.12 Guidance File Appendices A, B, C, and D provide a detailed check list, a typical schedule, a document list, and suggested supplies for use in a diagnostic evaluation.
In addition, DE!IB maintains a guidance file. that provides detailed guidance and examples for all documents and activities required to initiate and conduct a i
diagnostic evaluation.
This includes detailed guidance and examples related to evaluation plans, site access training, letters to licensees, and memoranda to the EDO and other NRC officials.
7-
l.
DET Guideline 1 i
Appendix A
)
Action Items for Diagnostic Evaluation Preparation Responi-Date Required Action bility
- Required
- 1.
Provide selected plant (s) name(s) to DEIIB a.
Develop evaluation schedule (s) in coordination with Regional Offices, b.
Request LER and PI analysis for last 2 years from the Trends and Pattern Analysis Branch (TPAB), AE00, and other plant specific documents, (FSAR. Technical Specifications,etc.),
Develop overall evaluation objectives c.
and identif management'y special areas of NRC senior s interest for the evaluation, d.
Discuss objectives and special evaluation areas with Deputy Executive Director for Regional Operations (DEDRO), Regional Coordinator and the Regional office as necessary.
?
l e,
Identify functional areas to be evaluated.
I f.
Obtain TAC number and provide evaluation milestones for PRIMR to the Management Assistant, AE00.
g.
Notify the Division of Reactor Inspection and Safeguards, NRR, of evaluation schedule.
h.
Negotiate with the regional office regarding the SALP appraisal period to ensure that-the SALP and DET reports will not be issued within six months of one another.
2.
Prepare and-forward the formal notification memoranda and letter for EDO signature.
a.
AE00 notification b.
Licensee notification
- Assigned by the Chief, DEIIB following plant selection 8
o l
I Responi-Date Required Action bility
- Required
- r 3.
Identify team members and major milestones, l
a.
Identify proposed team manager, team leader-and team members.
l b.
Identify needed contractor support.
c.
Identify major milestones and preliminary schedule dates for completion.
4.
Continue preparation with Region and licensee, a.
Obtain initial input from Region regarding plant performance, b.
Obtain document index from licensee, c.
Schedule advance trip.
d.
Lead evaluators prepare lists of desired documents for advance trip (s),
v e.
Develop " Regulatory Picture" chart if appropriate f.
Evaluate and select a safety-related system for evaluation 5.
Prepare Integrated Evaluation Plan a.
Develop background basis for plant selection.
b.
Finalizemajormilestones, Define goals and objectives and special c.
areas of interest, d.
Prepare evaluation plan memorandum for E00 signature.
6.
Conduct advance trip.
a.
Region (if requested) b.
Licensee o
Two persons, two days, minimum, o
Establish licensee contacts, o
Obtain documents required-by initial review of licensee document index.
9
O t
1-Responi-Date 1
Required Action l
bility
- Required *
{
l o
Arrange for additional document copying and delivery as required.
c.
Check logistics (motel, meeting room, etc.)
7.
Prepare and transmit final evaluation plan, a.
Establish fire schedule, b.
Provide copy to appropriate NRR and Regional staff for review and comment.
8.
Prepare for team meetings a.
Establish meeting schedules b.
Prepare agenda c.
Arrange / assign presentations d.
Notify and provide approved team members with appropriate plant performance background material.
Arrar.ge for meeting room (s) e.
9.
Make travel and lodging arrangements for entire team.
10.
Prepare and transmit final notification letter to licensee, a.
Clearances and certification of team members b.
Site accus c.
Entrance meeting (s) d.
Special requirements 11.
Conduct team meetings and training-10
.. ~.
l l
l i
1 DET Guideline 1 Appendix B Typical Diagnostic, Evaluation Schedule i
o Weeks 1-3:
Planning and Preparation (OET Guideline 1)
This initial period is used to finalize the DET, to review the integrated evaluation plan, to review licensee background and technical.information, and to prepare detailed evaluation plans for each functional area to be evaluated.
i o
Weeks 4 5:
Onsite Evaluation at Licensee Facilities The first onsite evaluation of plant and corporate offices is generally two weeks in length and devoted to information gathering and observations of the licensee's activities and performance in the areas specified by the evaluation plan.
o Week 6:
HRC Headquarters / Regional Offices'
{
The DET members return to normal offices for one week to continue the evaluation of information gathered.
The time is used to review the DEOs, brief NRC management as needed i
on initial findings and redirect the individual and overall evaluation plans as appropriate.
o Week 7:
Onsite Evaluation at Licensee Facilities A third week at the plant and corporate offices is generally needed to complete the onsite evaluation.
All or part of the DET returns as needed to complete the collection of information, finalize findings, evaluate additional areas, or perform special case studies.
It is recommended that during this third week on site that loose ends be tied, rather than searching for new issues.
o Weeks 8-10: Report Preparation s
Report preparation begins immediately after completion of the onsite evaluation.
Evaluation of the licensee's performance-continues during the report preparation period.
I o
Week 11:
NRC Management Briefinas: Report Preparation (cont)
Following the completion of the initial draft of Section 3 of the report, briefings will be conducted with NRC senior management to provide the team's overall findings-and conclusions.
In general, two briefings are conducted.
The first with NRR and Regional senior management, and the second with the Office of the EDO.
i 11-
.o l
o
'Veek 12:
Exit Meeting; Report Preparation (Cont)
Following the briefing of the Office of the EDO, Dan exit meeting with the licensee will be conducted to provide the 1
team's overall technical and management findings and 4
conclusions, o
Week 13:
Report Issuance The report will normally be completed, approved by the
~;
Director, AE00, and forwarded to the EDO by the end of the 13th week.
The above schedule may be revised to allow for licer.see or NRC schedule constraints such as cutages, holidays, competing inspections and personnel-availability.
In particular, it may be advantageous to use an alternate schedule for onsite periods.
i t
J l '
l o
l 12
1 DET Guideline 1 l
Appendix C l
DET Documents - Advance Trip 1.
Documents to be obtained during the advance trip prior to a diagnostic evaluation.
i Corporate organization chart (with current names) and telephone
]
a.
directory, j
b.
Site organization chart (with current' names) and telephone ' directory, c.
Engineering and technical staff organization charts (with current names) and telephone directory.
d.
Quality Assurance Manual.
I Quality Assurara ead Quality Control Procedures; procedures i
e.
governing corrective action and quality verification, f.
Administrative procedures - site and corporate (as applicable),
i g.-
Corporate policy stateinents (as applicable) regarding nuclear operations and safety.
I h.
Dvcuments governing any existing.:orporate or site improvement plans (safety-related).
.i 1.
Radiation Protection Manual ara procedures (if deemed necessary).
j.
Indices of procedures for functional areas being evaluated:
Operations, Maintenance, Surveillance and Testing, etc.
k.
Selected procedures.in key functional areas from the indices of j, above (functional area leaders will select).
l.
For the safety-related system (s) which is to be evaluated, obtain:
j i
(1)- Piping and Instrumentation (P&ID) drawings, electrical and.
control drawings.
[
(2). Copies _ of design changes and modifications on the system (s)
)
1 selected (for past two years).
t (3) System (s) operating and abnormal' and eme.gency operating procedures.
- i
-(4) System (s) maintenance history and copies of current maintenance requests (prevention and corrective).
{
l-13 l
1
y
,. t 4
4 (5) System surveillance,-inservice testing (IST) and inservice inspection (ISI)' history.
Five year business plans (corporate tad site).
l m.
Policies, procedures, and documents relating to the following:
[
n.
(1) Onsite control of contractors (2) Design changes and modifications (3)
IST and ISI programs (4) Tagouts, jumpers.and lifted leads (5) Failure analysis and trending. programs (6) Fitness for duty program-t (7) Sample work request and modification-request (8) Sample site activity schedule (if available)
(9) Motor-operated valve operating and testing procedures (10) Plant probabilistic risk (PRA) (selected portions)
(11) Onsite and offsite committees and copies of minutes (1 year)
(12) Audits by other utilitier (13) Procedures for processing and tracking-NRC Bulletins and Notices 2.
'The DET leader should obtain additional input from DEIIB members prior to advance trip.
3.
Team members requiring additional ' documents-after the advance trip should make them known to the DET leader who will arrange to have them copied and delivered to the team onsite on the first day of the evaluation.
4.
L' insee documents, records-and data should be controlled to prevent loss or release to persons or organizations not'directly associated with the-NRC or with the DET.
Particular care should be taken with proprietary documents and other sensitive material when provided.'
1 i
14
DET Guideline 1 Appendix D Supplies for Diagnostic Evaluation The following items will be supplied by NRC.
1)
Lap top personal computers for.each functional area and one for the team-manager (all machines should be equipped with appropriate sof tware) 2)
Formatted disks, disk case 3)
Pens, pencils, highlighters, rusrkers 4)
Steno pads, white lined tablets, spiral notebooks, post-it pads, telephone message pads 5)
Phone numbers, pager numbers, room numbers (for DET members)
-j 6)
Dictionary, thesaurus, NRC telephone book (1-800 numbers).10CFR 7)
Stapler, staples, staple puller 8)
Three hole-punch, flashlights The following items should be supplied by. the licensee:
1)
Adequate work rooms, desks, document storage, interviewing rooms 2)
Earplugs, glasses, hardhats 3)
Parking spaces and permits, appropriate site training 4)
FSARs, Tech Specs, Emergency Plans 5)
Printers (if there is a need, and'if'available) and printer; paper 6)
Security badges 7)
Copy machines, telephones, speaker phones
-J 1
15 a
e,.
i GUIDELINES FOR THE CONDUCT OF A DIAGNOSTIC EVALUATION DET Guideline 2 2.1 Purpose This guideline provides guidance on the methods for conducting the onsite portion of a diagnostic evaluation.
2.2 General' Section 2.3 of this guideline outlines the scopat and objectives of a DET evaluation.
Section 2.4 details DET management responsibilities.
Section 2.5 describes the onsite DET procedures and process for a typical diagnostic evaluation.
Section 2.6 provides the conceptual framework for the conduct of a diagnostic evaluation.
Section 2.7 describes the evaluation methodologies which the DET employs during the onsite evaluation.
2.3 Diagnostic Evaluation Scope and Objectives Diagnostic evaluations are intended to provide a comprehensive assessment of l
current performance at plants designated by NRC senior management in order to augment information provided by the NRC-SALP and PI programs, and other -
assessment data.
A diagnostic evaluation has the following characteristics:
It is separate from NRC licensing, inspection and enforcement processes, o
It is primarily performance and safety oriented, although compliance o
issues may evolve from the evaluation.
It evaluates station and corporate management actions and involvement o
relative to operation and performance.
The evaluation scope and content are determined'by the information needs o
of NRC senior management.
It attempts to identify probable root cause(s) for p1' ant' performance o
problems.
2.4 DET Management Responsibilities The Director, 00A, AEDD, directs the Diagnostic Evaluation Program.
The Chief, DEIIB, AE00, who reports to the Director, DOA, develops ~,' maintains and manages the Diagnostic Evaluation Program and procedures and maintains core DET expertise.
The DET manager for each DET is an NRC Senior Executive Service (SES)' level i
manager with overall responsibility. for and control of the diagnostic evaluation.
The DET manager reports to the Director,'AE00. The specific' responsibilities of the DET manager are to:
Prepare the diagnostic evaluation plan and submit it to the Director, o
AE00, for approval.
1
t l
o BMef the Directors, NRR and AE00; the Regional Administrator, and the EDO on the plans for the diagnostic. evaluation, Provide direction for the OET and ensure that the objectives and schedules o
of the evaluation plan are met.
Change the evaluation plan ~as needed se that the overall objectives are met, Be the lead for the entrance a' d exit, meetings with the licensee.
o n
Inform the 00A and AE00 Directors of DET progress and-findings.
I o
o_
Serve as the spokesperson for DET activities with the licensee on.
significant matters involving the evaluation.
Provide regular briefings to the licensee concerning DET _ findings and, issues.
Be cognizant of the provisiors o.f NRC Manual Chapter 0517, Management of I
o Allegations.
In the event allegations are raised during _the evaluation.
l impart relevant information to the Resident and to the Regional 1
Allegations Coordinator for action as soon as^possible.-
Request additional resources (e.g., additional team members, contractor
~
o assistance) from the Directors, D0A and AE00 if needed, Ensure preparation of the DET report in accordance--with DET Guidelines o
and assigned schedule.
A deputy team manager (SES) or team leader (non-SES) may be assigned to the DET.
The deputy team manager, or team leader reports'directly to the DET manager and performs duties as assigned, which may include:
- s Ensure quality preparation of DEO forms and' report input by DET sub-team o
- leaders, Confer with the OET manager regarding ongoing. evaluation efforts to-o optimize'use of team resources. -
lead the root cause analysis of the diagnostic evalution.-
o Assist in the_ preparation of the evaluation plan.
o Be the lead for the onsite: team meetings!
o-Serve as onsite spokesperson for DET activities when requested.
o Participate.n entrance and exit meetings and conduct regular evaluation.
briefings with the license to assure;that DET findings and concerns are communicated to the licensee.
l l
l 2
-~
,-,r--
7 y
, - t q
o Inform the resident inspector and DET manager of significant activities, o
findings, and safety concerns, Suggest additional resources (e.g., additional team members, contractors)-
o from the DET manager or revision to the evaluation scope to fulfill the 3
overall program objectives, Ensure time;y. preparation of diagnostic evaluation observation (DEO) o forms and team inputs.
I Sub-team leaders are assigned to direct a specific functional area of a diagnostic evaluation. The sub-team leaders report directly to the DET manager (or deputy team manager or team leader if assigned) and are normally responsible to:
o Prepare Functional Area Evaluation Plans Inform licensee counterparts of findings and issues.
o Effectively implement functional area. evaluation plans, o
Provide guidance on evaluation activities to members of the sub-team.
o Communicate functional area findings at team meetings.
o Revise functional area evaluation plans during the in-office evaluation o
of initial onsite findings, Keep the DET manager or alternate informed of-progress and problems.
o An administrative assistant may be assigned to the DET.
The administrative assistant reports directly to the DET manager and normally:
Performs duties as assigned, o
Arranges administrative support such as badging, printing, copying, o
document control, and scheduling.
2.5 Evaluation Procedures and Process The overall procedures and process _for each diagnostic' evaluation will be set forth in the plant-specific evaluation plan. -This section establishes i
guidelines for the onsite activities, procedures and process referred to:in-the evaluation plan.
2.5.1 Badging Required preliminary administrative notifications and generic training needed t
to obtain badges and unescorted access for all DET members will be ensured by J
the DET manager during the preparation phase as described in DET Guide'line 1.
The DET manager will also ensure arrangements are made-for the DET to receive site-specific training and testing and to complete other' administrative requirements (e.g., whole body count) needed to gain access to vital areas the first day onsite.
(
3
2.5.2 Entrance Meeting
(
An entrance meeting will be conducted with selected licensee personnel.
The objectives of the meeting are to:
(1) discuss the purpose and the scope of the evaluation, (2) establish communications channels and protocol with the licensee, and (3) request assistance in obtaining resources and information in specific areas.
Dud ng the meeting the DET manager will discuss evaluation objectives and scheuule, explain the process and evaluation techniques used,-
introduce the DET members, and respond to questions from the licensee.
Immediately following the entrance meeting, team members should meet briefly with their points of contact, review their activity and interview schedules, request additional records and documents, and identify other required assistance.
2.5.3 Plant Tour-At the beginning of the onsite period,-the DET should normally take a brief tour of the station to become familiar with work areas, site features and plant layout.
2.5.4 Daily Team Meetings Daily team meetings will be conducted for. team members to share observations, findings and technical information. Each sub-team leader should ensure presentation of the results of'the day's onsite-activities.
Significant overall findings and root cause determinations may evolve from team discussions such that narrow but related findings in different functional areas are. pieced i
together. Meetings should be conducted in a manner to foster team insight into higher level issues -(i.g., management practices, root cause.. program adequacy, corrective actions, and " organizational culture") that may not immediately be apparent to individual team members.
The DET manager should also communicate licensee management feedback to the members of the team.
It is recommended that meetings be scheduled at the end of the work day and be controlled so as-to last no more than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, 2.5.5 Diagnostic Evaluation Observation Forms Each team member shall document potential evaluation' issues, findings, strengths, and weaknesses-on DEO forms;.DEOs should document' the details of a specific finding in a particular functional area and provide a continuing-record of evaluation progress.as well as a basis for the finalcreport. ' DEOs l
will be annotated with the functional area and a' sequential-~ number as. they are filled out throughout the evaluation. _ Copies of each DEO should be provided to the DET manager (s) no later than the morning after the team. meeting of which they were presented to the team. -Revised DEO forms should-be provided-to team management' when significant additional information is obtained for a finding. DEOs should be provided to sub-team leaders.no later than the af ternoon of the day they are provided to the DET managers.
2.5.6 Counterpart Meetings The DET manager and each functional area leader should meet with the designated licensee counterpart on a frequent basis to keep him'or her informed on the technical and programmatic issues, findings, strengths and weaknesses. - Any i
4
_ ~ _.
safety concerns identified by the team should be immediately brought to the' attention of the DET manager and comunicated to appropriate licensee personnel and the resident inspector office.- DE0s and other written material may not be provided to the licensee.
Conclusions related to management and organizational effectiveness within functional areas should not normally be oiscussed at counterpart meetings.
2.5.7 InterimExitMeeting(s)
The DET manager and functional area leaders will conduct a brief interim exit l
meeting with licensee management at the eno of the first and second (if applicable) onsite-evaluation periods.
The interim exits will be used to present the status of significant technical and programmatic observations, I
findings and concerns. The DEOs will provide the basis for the interim exit meetings. Neither the management and ganizational findings, nor root cause analysis assessments should be provided at these meetings.
The exit meeting should be rehearsed to ensure well orgarized and coordinated presentations of appropriate length and detail.
In addi 'on, functional area leaders should ensure that licensee counterparts are informed of important facts and findings in each functional area before the DET leaves the site.
Conclusions related to management and organizational effectiveness should not normally be discussed at exit meetings.
l An informal meeting with the licensee may be held at an appropriate time to share preliminary observations developed from employee interviews on organizational environment and worker safety attitudes.
2.5.8 Final Exit Meeting The final exit meeting will be held with licensee senior management after the DET has returned from the site and briefed NRC senior management on its proposed exit presentations.
The final exit meeting will provide the team's significant technical, programmatic, management and organizational strengths and weaknesses, as well as the root cause analysis results. Handout material which summarizes the team's results will normally be prepared for presentation to the ED0 for his review and comment, in accordance with DET guideline 3.
It is expected that the final exit presentation content, balance, emphasis, strengths, weaknesses and significant findings will closely parallel the final DET report. The proposed presentation handouts should be reviewed by subteam i
leaders as a minimum. Notes should also be-taken at'the meeting to summarize the major discussion points on both sides and will be included in the team's report.
Enforcement consequences of DET findings will normally not be discussed at the exit meeting.
2.6 Evaluation Methodology This section describes the various techniques employed by the DET in conducting a diagnostic evaluation. Methods described in this section include interviews, direct observations, programmatic evaluations, and safety system evaluation.
5
)
2.6.2 Interviews Formal interviews constitute a cornerstone of the diagnostic evaluation methodology.
Formal interviews are structured to develop precise, in-depth information in one area by one team member or-by the entire team.
This latter type is used for special purposes and takes considerable preparation, even to the extent of using standard sets of questions to-focus the evaluation and results into a format that lends itself to analysis.
Depending'on the circumstances and the exact areas being evaluated, less formal-or-unstructured interviews may-be utilized.
Unstructured interviews'are utilized by individual:
evaluators to obtain general or specific information relative to an area.
Interviews are not transcribed or recorded, but results may be summarized or referred to in the DET report.
Interview notes and interview reporting forms which could be traced to an individual should be kept secure with each team; member. This information should not be revealed to others or used by other than DET members. except on a "need to know"' basis.
2.6.2 Programmatic Evaluations Programmatic evaluations are designed to determine how effectively the licensee specifies and implements safety-related activities.
This'is' accomplished by-examining licensee documents which provide for. the' controis of specific activities.
Each of the activities is evaluated from a programmatic standpoint for the following criteria:
Does the licensee have a written program (procedures or. policy documents) o to provide guidance in the management of the activity?
Are the procedures and policy documents adequate'for controlling o
activities to assure compliance with regulatory requirements and guidance?
Have the program requirements for the activity b'een adequately.
l o
implemented and are these activities appropriately documented?-
Do the individuals who have been assigned. responsibilities in the -
o activity understand their responsibilities?
Are licensee personnel with responsibilities in the' activity adequately 1
o trained and qualified to carry out their duties?
2.6.3 System Evaluativns.
The system evaluation methodology is designed to assess the-operational':and functional readiness.of a selected system (s), evaluate deficiencies, and determine'whether similar deficiencies might exist in other plant systems.
This is accomplished by determining whether:
4
~
The as-built design is such that the system.is capable of performing the o
j functions required by the design bases.
Surveillance and testing is adequate to~ demonstrate that the system (s) is o
capable of performing all-required functions, a
4 J
6 i
l
g a
o System maintenance is adequate to ensure system (s) operability under postulated accident conditions.
o Plant operator and maintenance technician knowledge and training are adequate to ensure proper operator-action and maintenance activities for the system (s).
o Human factors considerations relating to the selected systems (e.g.,
accessibility, labeling, operator aids) and the supporting procedures for those systems are adequate to ensure-proper system operation under normal and accident conditions.
i o
System control documents, including procedures, are adequate.to ensure that the system (s) will fulfill the functions required by the design bases.
Management oversight as evidenced by)QA audits and surveillances is o
adequate to ensure that the system (s is being properly modified.and.
maintained,. and that deficiencies are corrected in, a timely manner.
2.6.4 Observation of Plant Activities.
Direct observation of the plant and plant personnel activities should be conducted by members of the DET and represents an -important evaluation methodology. Observations can include any facility, equipment, component or licensee activity that is relevant to safe plant activities,' simulator drills, walkthroughs of selected facilities and procedures,'and licensee meetings and training sessions. Field observations of equipment condition and work in-a progress should be included in each functional area evaluation plan..
2.6.5 Case Study Evaluations 1
For selected safety significant issues the team may perform a detailed case study to further evaluate work processes, weaknesses and root causes.
2.7 On Site Documentation l
The team members will assemble and compile daily observations 'into the evaluation plan (report outline structure) using PCs for later-use in writing the draf t report and print out composite observations each day and provide to.
all subteams. DEO forms will be used-to document and track major findings both internally and with the utility.
Some label (bold, asterisks).will:be used to
)
identify new!information so that all of the material does not have to be reviewed each day.
q 2.8 Sample Document File DEllB maintains a sample document file that provides more detailed guidance and examples for all documents and activities' needed to initiate and conduct a diagnostic evaluation. This includes detailed guidance and examples related to the diagnostic evaluation framework (see section 2.6) functional area evaluation plans, DEO Forms, etc, discussed in the sections above.
u l
.=
4 j.
i b
GUIDELINES FOR THE PREPARATION OF THE DIAGNOSTIC EVALUATION TEAM REPORT-DET Guideline 3 3.1 Purpose To provide guidance for the preparation of the Diagnostic Evaluation: Team (DET) report.
3.1 General The purpose of the diagnostic evaluation report is.to accurately and-completely document the results of the DET evaluation.- This guideline describes the organization and content of the DET report and its transmittal correspondence, as well as the process to be-followed in preparing the report.-
During the process, care should be' exercised to prevent drafts or copies of the report (or excerpts) from being released to non-DET' members or non-NRC personnel.
4 3.3 Report Organization The sections that typically appear in a DET report consist of:
o A cover page A general information sheet, which identifies the licensee and facility o
evaluated, the periods of time the evaluation was conducted. the evaluation team members and any other accompanying personnel. The sheet also contains signature lines for submittal. by the DET manager and '
approval by the Director, AE00.
An executive summary, which briefly summarizes-the background and purpose o
of the evaluation, and the overall results and conclusions.
The summary also generally includes the observed strengths and weaknesses, and the s
significant underlying causes for identified weaknesses.
A table of contents, which lists the main sections and subsections of the o
report.
An acronyms section, which lists in alpha'betical order the acronyms and o
abbreviations used in the report.
i An introduction section (Section 1.0)..which provides background o
information for the evaluation.
The scope, objectives, evaluation methodology, plant description, and'a description of the corporate;and site organizations.should be included.
Figures normally are provided to illustrate these organizations, An evaluation results section (Section 2.0), which presents the ' major o
evaluation findings and conclusions in an abbreviated and concise l
1 l
N -
~ ~ ~
' " ~ ~ ~ ~ ' ~ ~ ~~~
- 't I
Results for each functional area are also provided for licensee manner.
and NRC management level review.
This section also provides the probable root cause analysis for the major licensee weaknesses and deficiencies.
This section should be fully consistent with the detailed evaluation results section as well as the findings and conclusions presented at the final exit meeting.
A detailed evaluation results section (Section 3.0), which documents-in a o
comprehensive detailed, integrated and accurate manner, the observations, findings and conclusions for each. functional area.
Findings in each functional area are presented as they relate to observations.
Conclusions are higher level results derived 'from' the findings. For weaknesses, discussions of root causes and contributory causes are developed.
This-section provides the detailed basis for the evaluation results section.
and the exit meeting presentation, An exit meetina section (Section 4.0), which documents the date, location o
and attendees at the exit meeting and provides a brief summary of supplementary meeting comments provided by NRC staff or team management.
in attendance. This section also provides a summary of any licensee-l responses on the team observations, findings, conclusions, and root cause-
)
analysis.
The appendices, which include information that supplements the body of o
the report. As a minimum, the appendices-should include a copy ofz the NRC presentation handouts 'provided to the licensee at the exit meeting, the EDO memorandum directing the conduct of the diagnostic evaluation, and any other material deemed appropriate for a complete understanding of
~
1 the diagnostic evaluation.
3.4 Report Transmittal Documents A number of transmittal documents must be prepared which are enclosed with or forward the DET report to the EDO.
These documents consist of:
A transmittal memorandum used to forward the approved ~ report from the o
Director, AE00 to the EDO. The transmittal memorandum typically contains three enclosures.
o to the transmittal memorandum.is the--
diagnostic evaluation team report to the specific site evaluated. The contents of this report are described.in the preceding section,
, to the transmittal memorandum contains-the draft memorandum o
from the EDO for NRC staff action requirements resulting from the diagnostic evalaution. This enclosure is. prepared:in the form of a draft memorandum from the EDO to-the-appropriate NRC senior managers for follow-up action-in their areas of responsibility. ' A separate enclosure to this memorandum describes the findings from the evaluation results and detailed evaluation results sections of the report' requiring NRC staff follow-up action (s), the specific action (s) to be taken and the-responsible NRC office to.which the action is assigned.
2 4
m r.y-...-r c,
.--e~..e--
L.o
.o
\\
i to the transmittal memorandum provides a o
draft transmittal letter to the licensee regarding the diagnostic evaluation report.
This letter will normally be addressed to the. highest' level licensee executive that is contacted by the DET.
The drr.ft transmittal letter shall be forwarded to the Regional Administrator and Director NRR for comment.
- 3. 5 Report Preparation Process Within 3 aays following, completion of the onsite evaluation, the DET leader issues to the team the significant milestones schedule for preparing the DET report.
l l
Developing the results of the DET evaluation into a comprehensive, clear, integrated, and accurate report requires a substantial effort by all DET.
members.
Af ter completion of the onsite evaluation, the sub-team leaders have the responsibility to coordinate the efforts of other DET members contributing to their assigned functional areas from the preparation of the initial functional area report outlines through approval of the final. report.
Within 7 days following completion of the onsite evaluation, each-sub-team l
leader will submit to the DET manager a functional area outline.
The outline will provide the organization for presenting the findings, conclusions, and' causes for weaknesses in the functional area.
The outline should provide the order in which findings are intended to be presented in Section 3 of the report. The DET manager and the-lead functional area reviewer.should agree on the functional area outline. The DET manager will then prepare a complete report
-outline, which includes the input from the functional-area lead reviewers, and should contain references to the appropriate individuals who are assigned lead responsibility for writing the sections.
This report outline should be i
distributed to all DET' members, and should be kept up to date to-reflect any l
changes during the report writing process.
I Within 21 days following completion of the onsite evaluation each sub-team leader submits to the DET manager their initial draft sections -for the detailed evaluation results (Section 3).
Each functional area lead reviewer should provide an acronym list for all acronyms and abbreviations used in his or her section.
This listing will be updated as required by the' functional area lead reviewer throughout the report writing process and should be-submitted to the DET leader in alphabetical order when Sections 2 and 3 of the report are finalized.
Within this same time period, the DET manager should ensure preparation of drafts of the key front-end sections of the report, such as the executive summary, introduction, and root cause analysis.
These draft sections should be distributed to all DET members.for review and comment.
~
When the drafts of Section 3 of the report are complete, copies would be.
t l
provided by the DET manager to.all DET members and a team meeting should be held. The purpose of the team meeting is to provide all team members t.he 5
opportunity to coment on the drafts of Section 3.
Comments should ar. dress accuracy and completeness as well as consistency.
Following the team meeting,-
4 3
~.
e l
1 the functional area lead reviewers will generally need to further revise.their respective inputs.
In addition, the functional area lead reviewers will draft Section 2 of the report and submit these sections to the DET manager.
Once the DET manager and functional area lead reviewers agree on the acceptability of Sections 2 and 3, the functional area lead reviewers should provide the DET manager with inputs and/or comments-for the executive summary, introduction, and root cause analysis sections.
Suggested inputs should also
' 'i be provided for the staff action items.
The DET manager should integrate and.
assemble the report inputs from all functional area lead reviewers into a complete draft DET report.-
When assembled, the DET manager should provide the draft DET report to the Chief, DEIIB, the Director, DOA,-the Deputy Director, AE00 and the Director, AE00 for' review.
Comments received should.be addressed by the DET manager with the assistance of the functional area lead reviewers and DET leaders _ for any changes to be made to the DET report.
i
-i Following management review and incorporatio_n of any revisions, the' final DET report will be marked as predecisional and submitted by the DET manager for approval by the Director, AE00.
The DET manager should ensure preparation-of the transmittal memorandum for the report from.the Director, AE00, to the EDO as described in paragraph 3.4 above. to the transmittal memorandum (draft transmittal letter to the. licensee) shall be forwarded to the Director, NRR and the appropriate Regional Administrator for review and comment.
The DET report should be provided to the EDO within 42 calendar days (6 weeks) following completion of the onsite evaluation.
3.6 Report Writina Guidance The following general and specific guidance is provided to assist all DET members in the preparation of the DET report.
3.6.1 General Principles o
Executive Summary Written for. top management (i.e., Commission, EDO, Office Director / Regional Administrator)
Should give some background and purpose of the evaluation; summarize and give overview of overall plant performance; discuss the root causes of past and present problems; bring-to light major team findings and conclusions, and note licensee strengths and weaknesses.
Length should be 1 1/2 - 2 1/2 pages.
Should stand alone, be precise, professional, and consistent with licensee-exit meeting.
o Section 1 - Introduction Discussion is written to orient the reader on the background and purpose 'of the evaluation and why it was initiated, and includes a 4
- p discussion of the scope and specific objectives of the team and methodology used, as well as a brief description of the plant, corporate and site organizations.
Section 1 should not introduce or. discuss any of the-team's findings or conclusions of the evaluation.
Section_1 contains five subsections in the following order!
Background-Scope and Objectives Methodology Plant Description Organization o
Section 2 - Evaluation Results -
Discussion is written for a management.or supervisory level that desires to understand the specific results of the evaluation and nature of the findings and conclusions in each area evaluated.-
Section 2.contains three different discussions
~
overall or top level team findings and conclusions specific findings and conclusions for each. functional. area l
evaluated root cause analysis of past and present performance problems Overall conclusions are usually a blend or composite of the.
strengths and weaknesses reflected in the specific findings / conclusions:
what are the enaracteristics that cut across the plant; what most impresses the' team; what are the most significant issues for the team;.and what ke at the final exit meeting with the licensee.y points were presented.
The specific finding and conclusions section should:
identify each major finding and conclusion ~ conta'ined.in each functional area, capture the essence'or " punch line" in a concise, consistent and complete way, not introduce any facts, observations or other information that are not discussed in Section 3, serve as an overview and summary of each of the top level significant results in each section, not provide examples, details or much discussion, 5
reflect the same tone and emphasis as Section 3, and be complete, but length / volume should be minimized The purpose of the root cause discussion is to inform the reader of the probable underlying causes-for past and/or current problems.
It requires brain storming and piecing.together the cause'of_ weaknesses in many areas found by the team.
Root cause can be recurrence.
The
^
root cause discussion describes for the reader what has happened or j
may be happening that explains why the difficulties resulted.. Root
_h causes generally will involve top level management findings,.
external factors and major programmatic or organizational weaknesses._
o Section 3 - Detailed Evaluation Results This section is written for the individual who wants to fully understand what was evals ed, the evaluation-results, and the basis for specific findings and conclusions.
Normally each section will have an ove'rview or background on the scope and method of evaluation.
Each section documents the observations, specific facts, and evaluation results. -Discussion is sufficient to identify what was found, and the team judgement regarding the merit or nature of any deficiencies identified.
The writeup should clearly identify the " punch line" or conclusion at the beginning of a section or paragraph and characterize.the activities as accurately as possible.
l Write with economy.
Brevity is importa:it,.but the meaning must-be conveyed.
The length of the write-up of a finding / issue should be consistent with its significance (i.e.,_ minor strengths and~
weaknesses should receive only a' limited acknowledgement).
This section provides the basis and support for.the' executive summary, Section 2, and the final exit meeting with the licensee.
All information, facts, and' obserrations should be noted, particularly with regard to strengths and weaknesses (including root cause analysis).
3.6.2 Specific DET Repcrt Guidelincs The NRC Technical Writing Style Guide (NJREG-0650) and the NRC Editorial Style Guide (HUREG 1379). should be used as guidelines for generation of-DET reports.
This. guide provides a uniform format in preparing reports-and produces consistency in documentation procedures.
Through the experience gained in preparing DET reports,'the following guidance
~ is provided specifically for the DET report.
i l
A DET report coordinator (normally the DET manager) will coordinate and-control all changes to the DET report after preparation of the first draft.
The. report coordinator maintains control over the DET report-due to the 1
(.
6
.~
~....
1 s,,'
n I
receipt of inputs from multiple writers as well as. multiple reviewers.. Some changes will be in disagreement with the inputs of others.
The DEIIB secretary will normally make and control. the typing of revisions to the report l
af ter the first draf t.
~
l l
The DET members providing inputs to the report should prepare their sections in the following order:
o Detailed Evaluation Results (Section 3) o Evaluation Results (Section 2)
I o
Inputs to Executive Summary i
o Inputs to Staff Action Item List-Each time an acronym is used within a section of the report, it_ should be written out in full the first time it is used.
Each sub-team leader should-provide the report coordinator a list of acronyms in alphabetical order used by, that sub-team.
A recommended wr.y to develop the acronym list is to use the
" spell check" feature.of Word Perfect.
Acronyms can'be copied to another file.
and this file can be sorted alphabetically.
The report should be writ;en in trrms of the " team" findings and conclusions, not on the basis of individual team members.
The DET report should describe thi results of the evaluation in terms of observations, findings, conclusions, and.causes.
It should not be an accounting of how time was spent in conducting the evaluation.
In'other-words, if a functional area was reviewed and the: review produced ~no items of positive or negative significance, the report should contain only a cursory discussion in this area.
Certain phrases should be avoided in writing the DET report.
These include:
o "The team believes.,"
l Example:
l The team believes that it was inappropriate and distracting.for the 1
reactor operators to have to answer such telephone calls.
L Should be written instead as:
Many of these telephone calls appeared to' be unnecessary and distracted the reactor operators from their. normal duties, i
7
ag H
o "The team felt..."
Example:
c The team felt that inadequate training in Technical Specification requirements regarding shutdown margin may also have contributed to the operator error.
t Should be written instead as:
Inadequate training in Technical Specification requirements regarding shutdown margin may also have contributed to the operator error, o
"The team noted..."
Example:
The team noted that many-systems.had not had a valve alignment verification performed in over 2 years.
Should be written instead as:
L Many systems had not had a valve alignment verification performed in over 2 years.
When describing personnel within the licensee's organization, position titles should be used instead of naming the individuals in those positions.
For each functional area covered in Section 3'of the report the leading paragraph should briefly describe what specific areas were e, valuated, the methodologies used to conduct the evaluation (i.e., observation of licensee activities, review of documentation) and summarize the team's major findings
.s and conclusions within the functional area.
Use " journalistic" style-(i.e., put the punch line up front). For each L
finding described on the DEO forms, the report should describe:
the findings; the conclusions and any strengths or weaknesses; and probable causes for I
' weaknesses in that order.
1 i
Since the diagnostic evaluation is directed toward safety performance rather than compliance issues, all items that may result in enforcement actions will be handled by the-appropriate Regional office.
Potential compliance issues will not be presented as violations in the DET report.
All references in the evaluation results -section of the-report to other sections will be described only to the level of the second subsection (i.g.,
i l
Section 3.1.4).
Reserve the use of the phrase "the team found", "the team concluded" or "the team was concerned" in Section 3.0 only to findings, conclusions or team concerns to be presented in Section 2.0.
In general the use of the term q
" concern" is restricted to only those rare instances where information which
)
'8
ls
,,s e.
will only become known at some future time that will validate or invalidate the issue.
In addition, the wording, including the use of modifiers for the findings, conclusions, and concerns presented in Section 2.0 should be consistent with the wording provided in Section 3.0.
Evaluation results are expressed in terms of a specific plant, i.e., the results are absolute and not a relative evaluation of a plant against an
" industry average".
The report should not include any recommendations, suggestions or equivalents.
Generally, the OET report will be wr'itten in the past tense.
3.7 Exit Meeting Material The draf t detailed evaluation results, and the draf t evaluation results sections should be used as the basis for preparing the exit meeting presentation with the licensee.
The OEr manager is responsible.for preparing the initial draf t exit meeting presentation', which should be prepared within 1 week af ter the first draft of Section 3 of the report has been completed and i
submitted to him or her.
Copies of the initial draft exit meeting presentation material will be provided to the DET manager, and the Director and Deputy l
Director, AE00 for review and comment.
In addition, DET members are responsible for ensuring the accuracy of the material to be presented at the exit meeting and, therefore, should be provided copies of the. draft presentation for review and comment.
The initial; draft exit meeting material will be used for briefing NRC senior management and the E00.
Any comments /
changes to the presentation resulting from these briefings should also be communicated back to the affected sub-team leader to ensure their accuracy.
3,8 Draft / Pre-decisional Information All preliminary /draf t DET report materials should be destroyed or discarded no l
later than the issue date of.the final DET report.
3.9 Guidance File OEIIB maintains a guidance file that provides detailed guidance and examples L for all documents and activities needed to initiate-and conduct a diagnostic evaluation.
This includes detailed guidance and examples related to outlines, reports, transmittal memoranda, staf f action lists and general writing guides.
l t
9
..