ML20042D147
| ML20042D147 | |
| Person / Time | |
|---|---|
| Issue date: | 08/07/1989 |
| From: | Taylor J NRC OFFICE OF THE EXECUTIVE DIRECTOR FOR OPERATIONS (EDO) |
| To: | |
| Shared Package | |
| ML20042D148 | List: |
| References | |
| FOIA-89-534, FOIA-90-A-10, TASK-PII, TASK-SE SECY-89-241, NUDOCS 8908140197 | |
| Download: ML20042D147 (10) | |
Text
i.n e
./
, ~.,
)
Q g
/OLICY ISSUE SECY-89-241 August 7, 1989 f
Informat. ion)
For:
The Commist oners From:
James M. Taylor Acting Executive Director for Operations
Subject:
PERFORMANCE EVALUATION FACTORS FOR BYPRODUCT MATERIAL LICENSEES To inform the Commission of the results of the Regional
Purpose:
trial program of performance evaluation factors (PEFs) for byproduct materials licensees, and to describe the staff's plans to implement a permanent program.
Summary:
This paper presents the results of the Regional trial program of performance evaluation factors for byproduct material licensees.
Based on this experience, five significhnt PEFs were identified which relate more frequently than others to the potential for degraded performance by byproduct material licensees:
- 1. Lack of senior management involvement;
- 2. Radiation Safety Officer too busy with other assignments;
- 3. Insufficient staffing;
- 4. Failure of the Radiation Safety Committee to meet, or inadequate functioning of the Committee; and
- 5. Inadequate consulting services or inadequate audits.
The Headquarters and Regional staff believe that PEFs can aid the inspection process and should be made a permanent part of byproduct materials inspections.
Consequently, the staff plans to incorporate guidance on the use of the five most significant PEFs into the Inspection Manual and to implement the use of the five PEFs beginning with the FY90 inspection program.
Contact:
Scott Moore, HMSS 49-20514 Roy Caniano, Region III FTS 388-5721 M hof.f)
AUG 141933
?*
8iI
I
. The' Commissioners 2
i
)
Background:
In SECY-88-219, dated July 28, 1988, the staff informed the Commission of, among other issues, the expansion of a Region i
III pilot program on PEFs for byproduct material licensees to include all Regions.
The SECY paper was followed by an 1
August 5,1988 Commission briefing on the status of efforts to enhance safety of users of byproduct materials, which also addressed PEFs.
In the Staf f Requirements Memorandum dated August 17, 1988, the Commission requested a description of the PEFs, the results of Region III's pilot program, and the methods for implementing an expanded pilot p ogram in other regions.
The staff answered this SRM in an October 11, 1988 memorandum to the Commission.
The memorandum gave details of Region-III's pilot program, provided the Temporary Instruction (TI) that directed the Regions to implement a trial PEFs program, and summarized the objectives a' d direction of the trial n
PEFs program.
The Commission also asked the staff to submit a report on the recommendations and lessons learned upon completion of the Regional trial program.
This information paper summarizes the results of the Regional trial PEFs program and describes the staff's plans for future use of PEFs.
Discussion:
Region III Pilot Program As noted above, the Regional trial pus program evolved from a pilot program conducted by Region III staff.
Region III evaluated 98 licensees, following inspections, and took -
follow-up action in 13 cases.
Region III staff considered the pilot program a success and recommended its use by the other Regions.
Expanded Regional Program Headquarters staff developed an expanded trial PEFs program, drawing from Region III's pilot program and comments from the c%er Regions, and formali:ted the trial through a directive to the Regions. TI 2800/15, dated June 30, 1988.
The TI directed Regions to develop and use a list of PEFs, to track results of the Regional PEFs program, and to report on the trial in May 1989, i
i-
The' Commissioners 3
i The staff selected the term " performance evaluation factors," or "PEFs," to denote the set of specific factors that aid in early identification of materials licensees with the potential for degraded safety performance.
The staff dif ferentiated PEFs from the reactor program's " performance indicators," which examine trends and data-intensive information on equipment.
The kinds of information available in the byproduct material PEFs program are primarily early subjective warnings or precursors of degraded performance of licensee operations, mainly concerning management-related activities.
In accordance with TI 2800/15, each Region established a program to identify licensees with the potential for degraded safety performance.
Regions were provided some flexibility in developing their implementation plans and their list of PEFs.
The TI told Regions to pattern their own PEFs after a list of 15 specific PEFs, adding to that i
list as necessary.
Guidance in the TI informed the Regions that specific PEFs should relate to:
licensee management oversight and control; quality of procedures and operations; adequacy of personnel staffing and training; and audits and feedback mechanisms to correct causes of deficiencies.
The TI also offered a range of actions that Regional management should consider when PEFs indicate the potential for degraded licens0e operations.
Those actions included:
telephone contacts; meetings with-licensee L
management; special inspections tailored to emphasize a particular aspect; Confirmatory Action Letters; early follw-up inspections; and comments to and about licensee managemert in letters forwarding inspection reports.
Implementation and use of PEFs differed to some extent among the Regions because the TI was written broadly enough to allow Regions to develop appropriate modifications.
However, all Regions followed the same basic guidelines so that the staff could assoss common results and draw broad, programmatic conclusions from the Regional experiences.
l In May 1989, the staff extended the one-year program to run through September 30, 1989.
The extension provides program continuation while the staff assesses the trial's results and reports to the Commission.
4
The Commissioners 4
Results of the Trial PEFs Program Regions reported results of the PEFs trial program to Headquarters in May and June 1989.
A summary of the number of licensees assessed using PEFs, and the number with some type of follow-up action based in part on PEFs, is provided on Enclosure 1.
A summary of the specific PEFs identified among the ifcensees with followup is given on Enclosure 2.
The Regional trial program used PEFs to evaluate a total of 695 licensees.
Total numbers of licensees evaluated per i
Region ranged from 69 in Region II to 354 in Region III.
Of those licensees evaluated in all Regions, the staff took or will take further action based in part on PEFs for 62 Ifcensees.
I Many of the 62 licensees with the potential for degraded performance also were cited for violations that required routine followup.
However, in each of taese 62 cases, the staff response went beyond the routine procedures for violations.
For instance, a Region may have scheduled an I
early reinspection of the licensee to ensure that licensee performance had not degraded too severely, or the Region may have sent a letter to the licensee requesting a direct response to the Region's concerns.
In the more serious cases, Regional staff met with licensees in management meetings to express concern about particular PEFs findings.
The staff's response, in these types of cases, is based partly on the PEFs assessment, partly on existing violations, and partly on the inspector's and Regional management's assessment of the licensee's safety posture.
Often, the PEFs results l
confirmed and documented conclusions that inspectors reached about the licensee's radiation safety performance.
Rarely L
did PEFs identify licensees with the potential for degraded performance without associated violations or safety concerns.
t PEFs did, however, assist inspectors in focusing on causes l
for degraded performance.
The staff identified a total of 34 specific PEFs, among all five Regions, that were noted in the 62 cases receiving a PEFs-associated followup response.
The number of times that each individual factor was noted is shown on l-.
Many of the 62 Ilcensees had two or more specific PEFs noted.
Six specific PEFs were identified more frequently than the others (ten or more times) in the licensees with potential for degraded performance:
1 i
The Commissioners 5
(
- 1. Lack of senior management involvement;
- 2. Radiation Safety Officer (RS0) too busy with other functions; i
- 3. Insufficient staffing; i
- 4. Failure to follow approved procedures; i
- 5. Failure of Radiation Safety Committee to meet, or inadequate functioning of, the Committee; and
- 6. Inadequate consulting services or inadequate audits.
These six PEFs accounted for.60 percent of all PEFs noted for the 62 licensees.
The other 40 percent of PEFs indicated in the trial were found eight or less times each.
Based on these results, the staff believes that the list of PEFs can be reduced in number without significant degradation in the quality of the PEFs assessment.
As i
noted above, six PEFs were identified more frequently than others in.11censees with the potential for degraded performance.
Of the six listed above, the staf f believes that five should be retained in the permanent program, which would result in a shorter, easy-to-implement assessment.
(The sixth factor, failure to follow approved procedures, should be deleted because it is normally addressed by a Notice of Violation.)
These five factors should embody the formal PEFs program and should be included in inspectors' field notes as a checklist.
However, inspectors should remain aware of the PEFs not included on the checklist, since those other PEFs may also serve as indicators of degraded performance.
Accordingly, the staff will list the PEFs not included on the checklist in the program guidance and will encourage L
inspectors' attention to them.
l L
Regional Comments on the Trial PEFs Program The Regions commented that PEFs are moderately useful in the inspection process and, if adopted, should be incorporated into the inspection field notes (i.e., standard notes that o
inspectors complete while conducting materials Itcensee inspections) as an integral part of each materials inspection.
One Region noted that PEFs are generally more effective for large licensee programs, such as academic or medical institutions, as opposed to smaller Itcensees.
The Regions were in general agreement with each other that PEFs should not be adopted into the inspection process as an isolated, special program.
One Region questioned the cost effectiveness of conducting a separate formal PEFs program; another Region L
3
. The Commissioners 6
1 noted that completing a lengthy PEFs checklist-appears to be an administrative burden on experienced inspectors. A numerical !
evaluation system used by one Region proved unsuccessful, and the Region noted that use of point totals provided no additional insight.
A direct benefit of the use of PEFs is that the factors increase the staff's awareness of broader safety and management issues, in addition to compliance problems, at l
licensees' facilities.
As Region I noted, "The real value 5
of the PEF is to cause the inspector to consider other t
types of information that may describe, explain, or influence the behavior and performance of the licensee, and thereby lead to a more thorough understanding of the licensee's strengths and weaknesses."
Agreement State Use of PEFs The staff informed the Agreement States of the PEFs program and invited them to provide data.
Two States, Louisiana and California, conducted a trial program and provided comments ~
on their trials' results.
Louisiana indicated that PEFs should be made a permanent part of the inspection program; and California stated that their program helped to identify areas where inspection efforts need to be concentrated.
The staf f believes that once PEFs use is established as a permanent part of the inspection program, other Agreement States may take more interest in PEFs.
l l
Future Use of PEFs Based on the results of the Regional trial PEFs prograa and the Regions' observations, the staff intends to implement a permanent PEFs program in the following manner:
The staff will incorporate guidance on PEFs assessments I
into the Inspection Manual.
A shortened checklist of the five most significant PEFs (Enclosure 3) will be included in~ inspectors' field notes for inspections of byproduct material ifcensees.
All Regional inspectors would complete the checklist for routine inspections of '
these licensees and discuss the PEFs results with their supervisors 'during normal debriefings, upon returning to the Regional office.
The staff will revise the Inspection Manual to incorporate the PEFs checklist.
Use of the L
checklist will be a routine part of the inspection l
process, not a stand-alone program.
The staff will continue to use the same range of followup actions from the Regional trial program, in response to licensees that demonstrate potential for degraded performance.
[-
The Commissicners 7
The staff routinely distributes copies of the NRC Inspection Manual applicable to materials programs to the Agreement States.
In addition to the distribution of the Inspection Manual guidance on PEFs the staff plans to include the subject of PEFs in the agenda for the 1989 All Agreement State Regulatory Conference.
==
Conclusion:==
The Regional trial PEFs program successfully used. certain factors to identify byproduct material licensees with the potential for degraded performance and assiste d inspectors in focusing on-causes for degraded performance.
PEFs also demonstrated the ability to confirm and document inspectors' general observations of a Itcensee's safety posture.
The set of factors offers a tool for. the materials inspector and' Regional managers to examine safety and management concerns, and the factors encourage inspectors and managers to search for a better understanding of licensees' strengths and weaknesses.
The staff intends to incorporate guidance on the use of PEFs into the Inspection Manual and to adopt PEFs as 1
a permanent part of the routine byproduct materials inspection
{
program beginning in FY90.
1 The Office of the General Counsel (OGC) has reviewed this paper and has no legal objection to its contents.
N's Jame M. Taylor s
Acting Executive Director for Operations
Enclosures:
1.
Regional PEFs Assessment Results 2.
PEFs Noted at Licensees with the Potential for Degraded Performance 3.
Future PEFs DISTRIBUTION:
Commissioners OGC OIG LSS GPA REGIONAL OFFICES EDO ACRS ACNW ASLBP ASLAP i
SECY i
1
,___._____.--_-___.___-__________________.__f
i 1
1 i
I REGIONAL PEFS ASSESSMENT RESULTS Region No. of Licensees No. of Licensees with Percent of assessed using Followup based in Licensees with PEFs Part on PEFs Results PEFs Followup f
I 90 7
8%
11 69 7
10%
III 354 16 5%
[
IV 76 18 24%
l V
106 14 13%
TOTAL 695 62 9%
i e
a -
PEFS NOTED AT LICENSEES WITH THE POTENTIAL l
FOR DEERADED PERf0Rl4ANCE No. of Times Factor Identified 1.
Lack of senior management involvement 28 2.
Radiation Safety Officer (RS0) too busy with other functions 26 i
3.
Insufficient staffing 13 i
4 Failure to follow approved procedures 13 1
5.
Failure of Isotope Comittee to neet or inadequate functioning-12 of the Committee 6.
Inadequate consulting services or inadcquate audits 10 7.
Users not familiar with safety procedures or license conditions 8
8.
Excessive missed surveillances 7
9.
Lack of audits 7
- 10. RSO not separated from responsibility for production activities 4
- 11. Repeated failure to correct violations identified by consultant 3
+
or licensee
- 12. Failure to implement adequate corrective actions on previous 3
violations
- 13. Inability to readily retrieve records and documentation 3
pertaining to licensed program
- 14. Rtportable events /misadniinistrations since last inspection 3
- 15. Numerous diagnostic misadministrations 2
- 16. Numerous repeat violations 2
- 17. Financial instability of licensee 2
- 18. Frequent resignations of staff 2
- 19. Inability to perform all required surveys on time 2
- 20. Lack of training documentetion 2
- 21. Failure to assess the performance of personnel training 2
- 22. Allegations /01 referrals since last inspection 2
- 23. Licensee not inventorying radioactive materials 2
- 24. Lack of structure to identify staff responsibilities 1
e
- 25. Company subjected to name change, developed into a subsidiary, 1
or transferred
- 26. Failure to provide training'to individuals prior to authorizing 1
them for use of licensed materials
- 27. Radiation waste not being disposed of at same rate of generation 1
- 28. Failure to retrain authorized users 1
- 29. Inadequate attention to radiation safety program by RS0 1
- 30. Incomplete responses to previously identified violations 1
- 31. No evidence of licensee capable of responding to a radiological 1
event
- 32. Inadequate surveys 1
- 33. RSO spends insufficient time at facility 1
- 34. Identified violations similar to those previously identified 1
TOTAL 169
r
,=
i*
FUTURE PEFS 4
1.
Lack of senior managenent involvement with the radiation safety program and/or RSO oversight i
2.
Radiation Safety Officer too busy with other assignments 3.
Insufficient staffing 4
Radiation Safety Committee fails to meet or functions inadequately 5.
Inadequate consulting services or inadequate sudits i
e
.