ML20209G211

Trip Rept of 850715 Site Visit to Participate in INPO Training Accreditation Site Team Activities.Weaknesses Included Lack of Accreditation for Mgt Training & Evaluation Criterion

ML20209G211
Person / Time
Site:	Prairie Island
Issue date:	08/23/1985
From:	Guldemond W NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III)
To:	Paperiello C NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III)
References
NUDOCS 8509190171
Download: ML20209G211 (9)

Text

'

. M,<

.m, .,m,. m. .w.m..

64 8 : 31.

c. .r a ; i ne

$d - 2M2

c. e . w- .... n y 9

DE0IG"ATED ORICINAI; Cortified D7 AUG 2 31955 7bR-MEM3RAN3uM FOR- C. J. Paperiello, Director, Division of Reactor Safet3 THR.. L. A neyes, Chief, Operations Branch FR3M. h. G. Guidemond, Chief, Operational Programs Section SUEJEC'- TRIF REPORT -- OBSERVATION OF INP0 TRAINING ACCREDITATION SITE TEAM ACTIVITIES AT PRAIRIE ISLAND s

During the week of July 15, 1985, I participated in an NRC observation of an INPO Training Accreditation Site Team visit at Frairie Islanc. The cther NRC observer was Ms. D+1cres Morisseau of tr,e icensee Ocalification Eran.-r.

Civisior of Hut.a- Facten Safety. NRF. in ecc de o' tr.is NR: at:(.ity was te assess INPC impier.ertation o' tne S .c 4 a r ticr, of tnei- Trair.ing Accreditatior. E or a- As such, tne perfctr.s w cf tne utility wa's r:t assessec excect a: i t impactec t.ite teat activit.t; The enclosu't t* te i s n.emorandum "cocume-th the details of my activities and O obsersaticr.1 .a p ner t - specif;c conclusic resulting from my activities.

The f olio.ir.; ge ie r a condusions were reacnec.

a. Site lea a: Us : 'es are prescribed by detailed procedures which were effectivei) implemented. As such, the team satisfied its objectivei

t. . Tne Site Teatr composition was consistent with the stated purpose of the tearr.

c. Communications between the Site Team and Prairie Islanc were effe:tive,

c. Fou prograrr weaknesses were identifiec:

. ta:e. c' accreditation for manacennt/ supervisory skills training.

Definitive guidance on the technical content of training programs nas not been explicitly incorporated into the evaluation process.

1. i rs. evaluatior, criteria oc not explicitly address regulatory reouirements. '

4 Tne standarcs of performan:e against which training programi a-e evaluated intentionally fall short of excelience.

8509190171 DR 850823 p ADOCK 05000282 PDR

C. J. Paperiello 2 AUG 2 31965 l

l

It is recommended that.the NRC observe as many of the Site Team visits as is possible to obtain additional insight into the veracity of the accreditation program.

4

. G. Guldemond, Chief Operational Programs Section

Enclosure:

As stated cc. J. Persensky, NRR ,

D. Morriseau, NRR t

I i

ENCLOSURE Introduction As part of an effort to improve the nuclear indQstry's performance in power plant operations, the Institute of Nuclear Power Operations (INPO) established an accreditation program for nuclear utility training programs. The purpose of the accreditation process is to encourage member utilities to adopt a performance-based training system by recognizing and approving training programs which satisfy certain criteria relating to methodology, content, and quality. ,

The training accreditation process consists of three chronologically distinct activities: .

Utility self-evaluation and upgrade of existing programs '

with respect to established INPO criteria C

INP0 team evaluation of the utility self-evaluation and the resultant upgraced training progra INPO Accreditation Board accreditation of upgraded programs Upon completion of the self-evaluation, the utility submits a report to INPO documenting the status of their training programs and committing to identified improvements. This report serves as a basis for an Accreditation Team visit,

1. the purpose of wnicr. is tc establish the accuracy of the self evaluation report by comparing the methodology, content, and quality of the training programs to established INPO accreditation criteria. The results of the Accreditation Team visit, including recommendations for needed improvements, are formalized in a report which is provided in the utility for action. The utility's corrective action plan, the Accreditation Team report, and the utility's self-evaluation report are provided to the Accreditation Board for final action on program accreditation.

On July 15-19, 1985 an INPO Accreditation Team visit was conducted at the Prairie Island Nuclear Plant (PINP) to evaluate the utility's training programs for licensed operators (SR0s and R0s), non-licensed operators, mechanical maintenance personnel, and electrical maintenance personnel.

This visit was witnessed by two NRC personnel as part of an NP.C effort tc establish the effectiveness of the INPO Accreditation Program.

Ob.iectives and Methodology of NRC Activities Relative to Accreditation learr visit Tne scope of the NR activities relative to tne Accreditation Teart visit te PINP was governed by the following objectives: #

• Determine the type of programs to be accreditec.

i

Establish the adequacy of team organization and methodology

- to accomplish stated objectives.

Establish whether measurable criteria were employed by the Accreditation Team to evaluate the methodology, content, and quality of the PINP programs evaluated.

Evaluate the criteria employed to select Accreditation Team members.

• Evaluate team member preparations.

• Evaluate disposition of team findings (concerns).

Evaluate the Accreditation Team / utility interface for communication effectiveness. s As such, the NRC personnel did not actively assess the utility's training programs or performance except as that performance impacted Accreditation Teart activities.

The NRC objectives were accomplished througn a combination of the following:

• Resie, of INPG supplied background information/ documents

• Intersiews of Accreditstion Team members and utility personnel

'

• Fassive monitoring of Accreditation Team meetings

• Passive monitoring of daily meetings between the Accreditation Team and the utility

• Observation of in-field activities of Accreditation Team members The results of the NRC activities were discussed with the Accreditation Team manager at the conclusion of the team visit. During this discussion it was stressed that these results were preliminary and did not represent official NRC policy or position.

ODservations Tne following observations were maae by the NRC personnel witn respect tc tne objectives stated above;

a. Proorams to be Accredited Generically INPO offers accreditation of tns following training programs:

i

1. Shift Supervisor /SRO training

2. R0 initial qualification training

3. Non-licensed operator traininc

4. Mechanical maintenance trainiiig

5. Electrical maintenance training 2

i

6. Health Physics Technician training

.. . 7. Chemistry Technician training

8. Shift Technical Advisor training

9. Instrument staff training

10. Instrument and Control Technician training Continuing training of each discipline is evaluated as part of the accreditation process.

During the review of INP0 document 85-002, "The Accreditation of Training in the Nuclear Power Industry," dated January,1985, the NRC representatives noted an absence of information relative to ,

accreditation of programs for managerial / supervisory skills or managerial / supervisory technical training for personnel other than Shift Supervisors. The Accreditation Team manager subsequently confirmed that INPO does not offer accreditation of such programs.

This is viewed as a program weakness. '

b. Accreditation Team Organization and Methodology Accreditation Team crganization and methocology are specified in a cocument er titlet. " Accreditation Tear Membe 's Manual.' Tne organization specified consists of a Team Manger, two Lead Evaluators, and a sufficient numoer of tean members tc accomplish team objectives. As oescribed in this oocument, the Team Manager has responsibility for coordinating and leading the evaluation and preparing the teaa report. One Lead Evaluator is responsible for the evaluation of the process employed by the utility i e to arrive at a performance-based training progran and the controls in place to maintain process viability. Included in this area are job and task analyses (JTA), translation of JTA into enabling and terminal learning objectives, and program feedback mechanisms. The second Lead Evaluator is responsible for evaluation of program content. Team members are assigned to each Lead Evaluator based on expertise and experience.

The manner in which the Accreditation Team conducts business is specified in Section IV of the " Accreditation Team Member's Manual" which addresses daily schedules and use of INPO-supplied evaluator questions. Guidance on interviewing techniques is contained in Appendix A to the

" Accreditation Team Member's Manual." Additional information on tean methodology is contained in INP0 85-002, "The Accreditation of Training in tne Nuclear Power Industry," dated January,1985.

Observation of Accreditation Team activities at PINP lead to the conclusions that the team was organized as prescribed in the

" Accreditation Team Member's Manual" and that this organization is compatible witn stated team objectives. Furtner, it is noted that practice of using INPO-supplied evaluators cuestions, which were found directly relatable to published evaluation criteria, erpured that evaluations were comprehensive and could reasonably be expected to be consistent.

t 3

c. Use of Mensurable Criteria l . .

As discussed above, the Assessment Team uses INP0-supplied evaluators j questions to gather data for the evaluation and accreditation process.

These questions were reviewed by the NRC representatives and found to be traceable back to the published INPO accreditation criteria. During this review it was determined that these questions were sufficiently j specific to support a valid evaluation of the process portion of a utility's training programs; however, specific guidance on program technical content was found to be lacking. In response to this perceived shortcoming, the Team Manager provided copies of INPO guidance docueents i

on training programs for non-licensed operators, mecnanical maintenance

> personnel, and electrical maintenance personnel. These documents did i not appear to be cross-referenced ins this Accreditation Program. The documents were reviewed by the NRC representative and found to be

, sufficiently prescriptive, and, if consistently used, could result ,

in a valid assessment of training program technical content in these

. areas, with the exception of continuing training for maintenance

- personnel. The guidance provided for continued training of maintenance personnel does not explicitly reouire periodic retraining for basic job j skills. The Ian cf referencec guioance or, progran technical content generally, and on continuing training spe W ically is viewec as prograt weakness.

4 Time constraints precluded NRC representative review of INPO guioance 4

cocuments on training program technical content for other disciplines.

! During discussions with team members and the Team Manager, the subject-of the expected standards of performance was pursued. As a result of these discussions, it was determined that the standards were

! intentionally set on the low and of the acceptable range. The rationale provided was that one of the purposes of initial accreditation is to have utilities implement a performance-based training system. If l standards were set higher, the process would become unduly complicatea.

Necessary refinements could be made at a later date.

i

d. Accreditation Team Member Selection Criteria l

Criteria for the selection of Accreditation Team members are contained

in Ii1PO 85-002, "The Accreditation of Training in the Nuclear Power

! Industry," January,1985 and the " Accreditation lean Member's Manual.

! These criteria specify that tear, members be selecteo from both INPD anc member utilities and that these members collectively possess competence j in 1nstructional matters, relevant technical subjects, and training u accreditation. This mixture is considered acceptable given the process /

i content team structure. The inclusi>on of member utility representative 1

{ e is viewed as an expedient means of promulgating INPO expectations witn ,

l respect to accreditation requirements throughout the industrf. l t

! Implementation of these criteria was verified by review of the source j and qualifications of the PINP Assessment Team. This team consisted of fourteen members, nine from INPO and five from member utilities, i l i i

)

4 1

4

--,m._.-.,...m -

._.-,.-y -

....--..---.--._-s.,,..--,..c,

,..r,,--...y- - - . . . . . - , . . ,..,,.--,--,c _..,4-.,

=-

l Four of the nine INP0 representatives were people on extended loan from i

.. . member utilities. Thus, a broad cross section of the industry was represented on the team.

Five members possess instructional and accreditation backgrounds, four members possess technical and accreditation backgrounds, four members possess instructional and technical backgrounds, and one member possesses an instructional background. Thus, a broad relevant experience base existed on the team,

e. Team Member Preparations Team member preparation requirements are specified.io the " Accreditation Member's Manual." These requirements include review of the utility's self-evaluation report, evaluator'.s questions, and technical information supplied prior to the team visit. ,

The NRC representative interviewed one INPO and two member utility team members to determine the amount of effort devoted to preparation. As a result of these interviews, it was determined that each member rectivec his spe:ific tear assignment and requireo supporting informatior, sufficienti) ir aavance of the tear visit and ea:r. was given at isas; a wee 6 te prepart fcr the visit.

Discosition of Tean Findines (Concernsi Periodicallt dur.inc the team visit, the NRC representative discussed with 0 the Tear Ma' nager how team findings, as presented to the utility during daily debriefings, were to be dispositionea. These discussions focused on disposition mechanisms rather than disposition of specific findings.

As a iesult of these discussions and observations of team activities, it was determined that all findings were recorded on a separate tracking form by team members. On a daily basis these forms were provided to the affected Lead Evaluator. At the conclusion of each day the Lead Evaluators met with their assigned team members as a group to discuss all findings and plans for resolution. Following this meeting, a meeting of the entire team was convened and the process was repeated. During this meeting assistance from the other team function (process or content) tnat dic not mar.e the finding was requested or offered, Team Manager perspective was provided on the significance of the finding, and a final action ciar- was oecided upon. The following day the finding was formally preser.ted to the utility or, if previously presentec, the status updated and the action plan for disposition agreed to.

Findings could De closed only if additional information was reviewed by the teaa which indicated that the issue was resolved. Those findings still open at the conclusion of the team visit would be documented in the team report and would result in a recommendation to the utility requiring a response.

1 5

. 1 l

. During the team visit the NRC representative noted two instances in  !

.. which a more aggressive posture should have been adopted with respect to resolution of a finding. The first instance occurred when the team experienced difficulty in relating a potential finding to a specific accreditation criterion and could therefore not establish the significance of the finding. During discussions on this matter the opinion was expressed that if the item could not be directly related to a specific criterion it was not the team's responsibility to pursue resolution. In the instance the team failed to consider that the item had potential significance beynnd the training program and needed to be followed for that reason. The secondt instance occurreq when the team expressed the opinion that a finding could be resolved based on the historical workings of an informal process rather than demanding that the informal process be formalized. Without formalization, no guarantee existe.d that the process would continua These matters were ultimately pursued further by the team after discussions between the Team Manager and the NRC representative. .

These observations are indicative of a weakness in the direction provided to the teart in considering the full implications,of findings.

If the accreditation process is to be effective, tn15 OEf1Clency must be correctec.

g. Accreditation Team / Utility Communication Effectiveness Througnout the teati visit, the NRC representative ncteo evioence that lines of communications were maintained open and that the Accreditation C Team anc utility cooperateo extensively tc resolve 11 findings. Daily meetings (attended oy the NRC representative) between the Teart Manager, Lead Evaluators, and utility representatives of the appropriate management level facilitated clear exchange of information. In audition to observation of working level activities, the inspecto# confirmed by interviewing utility personn'el that working level communications were effective.

Conclusions and Recommendations The following conclusions are reached with respect to the INPO Team evaluation phase of training prograr accreditation based on the observation of the PINE Team visit:

a. Tne range of programs for which INPO currently offers accreditation addresses the obvious operational needs of the nuclear industry; however, the failure tc include management / supervisory training in the accreditation family is a shortcoming which could potentially undermine the overall effectiveness.of tne accreditation process.

b. The team evaluation process is well structured and procedurally controlled. The structure is consistent with the stated fundtion and has been successfully implemented.

6

c. Definitive guidance is lacking in the area of program technical

.. content. This is particularly evident in the area of continuing training. This creates the potential for lack of consistency in content evaluation.

d. The fact that the standards of performance were intentionally set low is inconsistent with the stated INP0 objective of promoting excellence in performance,

e. The Accreditation Team member qualification and preparation requirements are fully appropriate, proceduralized, and have been successfully implemented.

f. Adequate proceduralized mechanisms exist to track all team findings; however, there were indications that the team focused too narrowly on training with the result of all findings were not ,

aggressively pursued. Further, there appeared to be a willingness on the part of the team to accept informal processes which were in place and functioning to satisfy certain requirements.

c. Tne specifiec senedule for daily tean activities facilitatti internal team communications anc communications with the otili:3 Inis ensures effective transfer of information.

0 0

t l

7 J