ML20147E042

Forwards Preliminary post-accreditation Audit Reviews at Susquehanna on 851021-23,at Dresden on 851118-21 & at Oconee on 851216-18 & Rept on INPO 851210 follow-up Visit at Susquehanna

ML20147E042
Person / Time
Site:	Dresden, Oconee, Susquehanna, 05000000
Issue date:	01/28/1986
From:	Harold Denton Office of Nuclear Reactor Regulation
To:	Pate Z INSTITUTE OF NUCLEAR POWER OPERATIONS
Shared Package
ML20147D825	List: ML20147D821 ML20147E042
References
FOIA-87-787 NUDOCS 8803040244
Download: ML20147E042 (48)

Text

. ...._...... _ _ -. .

i

. \ ,

.. o l

l . . ,.

!, g UNITED STATES  !,

8 o NUCLEAR REGULATORY COMMISSION j E .

l wAsmofow.o.c.sosss il JAN 2 81996

....h. .

c

['

Mr. Zack Pate, President I Institute of Nuclear Power Operations ,

1100 circle 75 Parkway .

Suite 1500 Atlanta, Georgia 30339 i i

Dear Mr. Pate:

The staff has completed three post-accreditation training reviews, observed ten INPO accreditation site visits and observed nearly all accreditation board meetings. We are pleased with the progress the industry is making in .  ;

improving. training. -

We *have observed the in-depth reviews by INPO of utility training programs and have been particularly impressed with the positive feedback between utilities which results from the use of peer evaluators and  ;

the vigorous and extensive information gathering procedures utilized to '

evaluate whether or not a utility's program meets INP0's standards for '

accreditation. Our observations of accreditation board meetings have confirmed the professionalism, independence and quality of the board's .

review. As in any sampling review, the board relies heavily upon .

the review work performed by INPO. NRC observers at board meetings have  :

favorably connented upon the INPO Team I4sder's knowledge of utility -

ei programs and utility cosnitments and action to complete open items. An [i important finding from our post-accreditation reviews is that -

utilities are continuing to improve their training programs even though they [:

have received accreditation. p There has been concern in the industry regarding NRC's post-accreditation I reviews and our review procedures. As a result of this concern, I have [i had the staff conduct a comparison of our post-accreditation review approach and INP0's accreditation criteria and objectives to determine if (

m there is any difference in our understanding of effective performance-based training. Our review results are enclosed. This review has affirmed our

['

-)

earlier conclusion that there is no difference between the staff's and INP0's f4 understanding of effective performance-based training. [

t' I am also enclosing preliminary copies of post-accreditation reviews at .N Susquehanna (October 21-23), Dresden (November 18-21), and Oconee (December fI 16-18) and a report on the INPO follow-up visit at Susquehanna (December 10). l!

Before we finalize our reports, I believe it would be valuable if H the respective NRC and INPO Team Leaders and you and I neet to discuss any In differences between INP0's and NRC's findings. The staff's reviews focused [l

.- H l

Ax4-P7-nr7 ii 8803040244 880302 f ,

l I

PDR FOIA BOLEY 87-787 PDR MY

/ ll

(;

l 7 ^3, '^

7' '. n m-,

. - w - - -.,-.. ,;.~; m

• ^

m..-,.w.,.

• ~,- ** y, ,,,,,. y'

!? ' ' "'

n._,,..-,..

.r n II

~

l l

! l

. :I i

.:9

' O on the five elements contained in the "Cosmission Policy Statement on i

. Training and Qualification of Nuclear Power Plant Personnel" which was r published on March 20, 1985. These elements are: '

n I. Systematic analysis of the jobs to be performed. /..

II. Learning objectives derived from the analysis which describe desired performance after training. '!

. l III. Training design and implementation based on the learning objectives. (

IV. Evaluation of trainee mastery of the objectives during training. [,

V. Evaluation and revision o'f the training based on the performance of <

, trained personnel in the job setting. .

Preliminary results from our post-accreditation reviews indicate that Elements Element I III,' fin dings were mixed. Our preliminary findings with Element. IIIV,and V, w relate to the completeness and quality of learning ob.jectives.-

A related matter is the accreditation process between the time of the INP0 Team Visit (when weaknesses and recosusendations are developed) and the

. board meeting. The staff observers were concerned sbout the magnitude of  :.

the corrective actions necessary to implement some of INP0's recommendations. '

I propose the following topics for our next NRC/INP0 coordination meeting M on training. -

I- -

• Preliminary findings free post-accreditation reviews at Susquehanna,

, Dresden and Oconee.  ;.

ia

• Quality and completeness of learning objectives

[p

• INP0's guidelines for the development and content of continuing h training programs {

M

• Possible NRC observation of INP0's eview process after the Team Visit jj and before the accreditation board meeting q

~ ~

in

• Discussion of lessons learned from NRC's post-accreditstion reviews i.

• Inclusion of INP0 observers at NRC post-accreditation. reviews l

.. i.

. i p

i) i d

. u

. ,. n . ,, ,, * ,,.

l g., : n , t; , ~

• 7.. ~ .-. - -

y . -

.. . .7 .,.m

, , . , ~~

e g w W w' =w.

.% . . ~ , .=

, -, w ,e

.y ', Eq e

.i -

j

- , l

. j Q

i.. j 4D In closing, I would like to emphasize that we h,'ve observed pignificant progress in improving training. Our comments an3 concerns are intended to be

'ii; constructive in order to improve the accreditatiot process and to provide o independent information on which the Commission cat. base its decision at the I..

completion of the 2 year trial period.  !. j u

Sincerely, i

. 'l

/ n ,

Enrold R. Denton, Director

, Office of Nuclear Reactor Regulation .

Enclosures:

As stated ..

cc: E. A. Straba, INPO .

J. Sciezek, NRC .

W. Russell, NR r

O I.

. I i

i.'

l'

+ *

.  ! ~.

I-I i

F i

} ".

1

~

U

- l:

ee O )

i 4 p

,w

.y a -. . ;,, , .- _.

.. v-i.y_

. . m . .,--. ~ ~ . . . . ~ ~ ~ , . , . , . - e r --

, :,r w-r-2,n,:rp. .

_ , - . .q.ngr'

_ . . _ .r. _,.m-n

n. >

m - ,,:

.? ... . V is

, . c, l

Gl l

i.;-

POST-ACCREDITATION AUDIT "

SUSQUEHANNA NUCLEAR PLANT , ;L OCTOBER 21-23, 1985 l

{f id A. Introduction [

l 1. Background. On October 21 through 23. the NRC staff conducted a post accreditation audit of training at $dsquehanna. All ten of the  !-"

$usquehanna programs eligible for INPO Accreditation have received full IS accreditation status. A representative sample of the accredited programs at Susquehanna was reviewed. }L p

2. Criteria. The criteria used by the staff to audit the implementation of ,

>erfonnance-based training are taken directly from the "Consnission n

'olicy Statement on Training and Qualification of Nuclear Power Plant d Personnel"(50FR11147)ofMarch 20, 1985. In its Policy Statement.

• the NRC' states that the following five elements are essential to acceptable perfonnance-based training programs:

• Il l,i

1. Systematic analysis of the jobs to be performed.

{

2. Learning objectives that are derived from the analysis and that describe desired performance after training, I'

t'

3. Training design and implementation based on the learning objectives. N

4. Evaluation of trainee masterf of the objectives during training. D!

5. Evaluation and revision of the training based on the performance of p!

r trained personnel in the job setting. ..

ll i

, 3. Documentation. To support its review, the NRC review team requested i.

that the following types of documentation be provided by the facility: F-f.

o Instructions / Procedures related to:

- Systematic methods used to analyze jobs, f-V

- Training organization goals, objectives, and plans. i

- Responsibilities / authority of training organization personnel. I

- Methodsforevaluating/selectinginstructionalriterials. I methods, and media, p

- Methods for organizing / sequencing of training. . [ .. h

,. t ..

, ,.- r; y

e Il N

n. n ~;;y nm

,f.i

, .

• m,; m -

• ' .N.. ".

7--

_33 x*.mry77.

J__ 's c= ,S K, 'l

11 a "

. 1 E

w

.... . g II

.. l ej y il t .

. 1 a

- Methods for keeping training programs current. ,

[

V

- Maintenance of training records. j!

- Selection of candidates for training and the granting of i waivers / exemptions from training. {.

p

- Evaluation of training programs, and h

- Training, qualification, and evaluation of instructors y{

o Task lists for the job (s) being reviewed I c'

o Documentation related to

,1

- Development / validation of task lists. , ;d

~

, .. a

, - Selection of tasks for formal training, , ,

- Analysis of tasks.  ;;,

- Analysis of on-the-job performance problems and industry events.

and i

in i .:

- Evaluation / audits of the training prograa(s)

. o Roster / organization chart for,the training organization [~ ,

o Training schedule for the past 6 months and the next 6 months. -

[

~ r Based upon NRC staff observation at INP0 Accreditation Team visits, the-  :!

above documentation is the type normally associated with participation O

, in the INP0 Accreditation process. Documentation beyond that required  !

by INPO for Accreditation is not required by MRC. n

4. Scope of Audit. The programs audited and the tasks selected for review lj within those programs were: lj PROGRAM TASKS

1. Mechanical Maintenance o Perform routine surveillance on emergency diesel engine; o Take and analyze vibration i readings on pum p turbines, fans,pseted motors. >

. . .. ;q e Perform local leak rate testi-

[

.o o Repair motor-operated valve '. .. U operators. .

i.

I, v.. e **'^~'7~

, .y .~--m.<:.y q i ~w ~ ' ' ~_

, :.r :- ;y . m T : " ~" ' ~ _ '7 ' -

~ '~"Y T % } T f[?' -

- ..,. .~ . , - . . . ... .. . . . . . . . .. . - ..

,,. .,, m i

l'

. O s

t ,

i .

. 3 jg U,

o Line up the Reactor Core

[

2. Non Licensed Operator  !

IsolationCoolingSystem(RCIC); 1 o Perform line-ups on the *'

. Recirculation System; ,'!

U o Rack out a 480V Sus Load .I Breaker; ki o Conduct Equipment Tapouts

3. STA o Review the operating shift's i!

- logs, turnover sheets chemistry il

data, and indicators for plant status and possible trends; E

\

o Determine if indications of core- i1

- ~~

damage are present; D o Evaluate Reactor Cooling System -

Level Parameters; t

i o Evaluate Neutron Power Level Parameters

'i <

4. Technical Training for Tech. o Provide technical direction h

~

Staff and Managers during initial rod sequence f Reactor Engineer

• exchange; :j .

• ,i o Provide technical direction

, during the initial heat-up; *'

o Analyre results from

- surveillances l

5. Feedback. During the audit, the team provided feedback to the [

Susquehanna training staff in two group meetings. This was done to }<

give the utility training staff an opportunity to clarify any r.!

questions arising from tie audit and to identify any documentation i or other evidence which could resolve team concerns. .

B. Results of Review fj The following discussion of the review findings correlafes with the {

elements of the Comission's Policy Statement on Training and  !!

t' Qualifications. .

• . eO

,1 O

i!

Il h

n 83

? . * * ' " ' ,

' P ! '- "

t, *- ' - "'? "~" " ' ' ~ ~ . ' " . ?*T **T;"J ';r y- r* *A, ,*.-

r.

{

l^, ,

V U;;

4

\  ;

l  : h 1.0 Systematic Analysis of Jobs to be performed ." E is 1.1 Discussion. Task analysis methods, procedures and products were i reviewed using the documentation described in Section A.3 above to b detemine whether: F

=

c Training and qualification programs were based upon systematic p analysis of the skills- and knowledge r.equired for adequate job i-performance.

[

t o Training and qualification programs were based upon a systematic E analysis of the entry skills and knowledge of trainees. p o On-the-job performance problems were systematically analyzed to determine training needs. " '

n

, o Industry events were systematically analyzed to determine training I

, needs. .

o The systematic analysis considered both the initial and continuing  ;.

training needs of job incumbents. .

o Procedures and other implementing mechanisms ensure that the '

analysis information remains current as job perfomance requirements evolve. t-s 1.2 Findings. The task analysis for Jhree of the sampled programs is complete and contains systematically derived job performance I'

information. The job task analyses for those programs were found to be" ..

exceptionally well done and are an excellent source of valid information upon which to base training programs. The fourth sampled program. -

i ReactorEngineer(Technica Staff and Managers), did not have a F completed task analysis at the time of review.

[

There is a formal mech;inism (i.e., the curriculum comittee) used to [

determine training needs from industry events and on-the-job perfomance 9 of graduate trainees 6 months after training. The method of selecting F tasks for training does not.seem to distinguish between tasks requiring i.

initial training only and those requiring continuing training as well.  ;

The existing method lacks criteria for determining the need for '

p continuing training based on such factors as frequency of performance

'and safety significance. t 2.0 Development of Learnino Objectives y

2.1 Discussion. Learning objectives were reviewed for the subject programs I to determine whether: . . ..

{~

. n f,

.... p h

?f

[ h? *? I, *

~

.".""'t

.n y , " 5 ,

l 4 f

.w v w - ,a. ,, .- -

, C>

if l

e o La '

i ,

~

5 !d ti

)

o Written learning objectives were developed based upon,the systematic is analysis of job performance requirements. l o Learning objectives are specific and measurable, and include conditions, actions and standards based upon job performance  ;'

requirements. .

o Procedures and other implementing mechanisms ensure that the 4 learnin -

evolve.g objectives remain current as job performance requirements L' i,

2.2 Findings. The staft was not able to deterzine whether the learning P objectives had been derived from the job analysis data. The i; cross-reference matrices referenced the location of tasks in a unit of la instruction but did not 1rovide verification that task-specific ,

knowledges, skills and abilities as derived from the task analysis had. .

been incorporated in learning objectives and lesson plans. The licensee - i provided limited documentation, based on instructor knowledge of the .

D

- training provided, which correlated task statements to training program ic lesson plans. Upon reviewing referenced lesson plans and other materials for the tasks identified in A.4 above, the team found that p several tasks and their associated knowledges, skills and abilities were ,

not addressed. This was most often found in the STA, Mechanical Maintenance and Reactor Engineer tasks selected for the review. The U licensee's staff provided verbal assurances-that instructors do cover  ;-

the necessary material, though it was not included in the referenced [^

- lesson plan. , N i

The learning objectives did not' follow widely accepted minimum . ,

requirements for performance-based learning objectives. Specifically,, ,,

they did not describe expected trainee behavior in terms of conditions.- i actions and standards governing successful task performance. The i learning objectives reviewed for the selected tasks did not reflect the i knowledges, skills and abilities (KSAs), nor the standards and -

conditions derived from the job task analysis. The majority of lesson plans reviewed stated only a generalized standard of test performance. U 1.e., a 70% score on a written test rather than performance based criteria for mastery of each learning objective.

[

3.0 Design and implementation.

3.1 Discussion. Using the documentation described in Section A.3 above, the design and implementation of performance-based training was reviewed to determine whether: _ F o The training organization's goals, objectives, plans', and relation-ships with the remainder of the organization are clearly stated. ,

{

o The responsibilities and authority of training personnel are clearly.

stated. . . .

([

I

! l l

L I

- n. . 3-v . w , n

. i ,#

- ~ ,

n- - ~ . -

=. g - .7, w w . &

, ,y s

m .. . . . . = = ------- . m _ = --m- =.---. - -

7 P i l ,

I

. 6

. . N o Instructional settings selected (classroom, on-the-job' training, and

- simulator) are appropriate given job performance requirements and  ; ',

learning objectives. ,.

?

c The orgtntration and sequencing of both initial and continuing 1 trainin:1 program content are based upon the relationships among learning objectives. .

o Lesson plans have been developed that ' include all the instructional i<

materials required for implementation of the program.

o Instructional materials were based upon job performance requirements

- and learning objectives and selected using an objective, auditable evaluation of the appropriateness of these materials. 3 o Procedures and other implementing eichanisms ensure that the design ,

and implementation.of initial and continuing training programs '

remain current as ' job performance requirements evolve. *

. t 3.2 Findings. The training organization goals and objectives are defined in a variety of procedures documents'and selected goals are included in ,

the utility's management by objectives program where evaluation of p: '

achievement of those selected goals is possible. The responsibilities of training department personnel are adequately defined. The:e is a  !

p(rocedure which describes the need to select instn.ctional settings i.e..methodsandmedia)asonestepinthetrainingdevelopment '

process, however, the procedure does not provide criteria for the

~

selection of appropriate instructional settings. The utility stated  ;

that it relied on staff judgment for selection of instructional settings I, and has not established criteria for selection decisions. These staff .

. judgments are reviewed as part of the approval process for the resulting [

1esson plans laboratory exercises and DJT qualification cards. [,

- Lesson plans for units of instruction reviewed generally adhered to a j.

a format prescribed by the utility's procedure. Although learning i objectives were a prescribed part of lesson femat, the simulator I<

training segment for STA training cantained no learning objectives.

L There was no evidence that a review was conducted to determine if (,

training materials that existed before the Itcensee's changeover to a performance based training program were consistent with trainee ij entry-level skills and knowledges. Classroom and simulator training i; which were observed were generally well-planned and conducted. Adequate p

'J methods have been established for maintaining training records.

4.0 Trainee Evaluation j-4.1 Discussion. The methods for and use of trainee evaluations were

- reviewed using the documentation described in Section A.3 above to**  :

determine whether: ,

t jf F

^) ~7' T ' 'i ' ? *T'**Q' fre f *". '-

' # '# ~ ' * '~~' ~"""'5- ' ~ N* '"' V33

) e

.____. f

., , b e .

.' 7 l-r

. . I o Trainees are routinely pretested or otherwise objectively evaluated Y to ensure that their entry skills and knowledge are consistent with the training program assumptions, and with the basis for granting [.

exemptions from training.

p tI' o Trainee evaluations (tests) are developed based upon job performance requirements and learning objectives (i.e., skills, knowledges, and .

abilities). ,  ;.

o Trainees are regularly and objectively evaluated throughout each E training program.  !-

- o Established methods exist and are used to provide perfonnance (.

feedback to trainees and to deal with unsatisfactory trainee perfor- ,

mance. ,

4.2 Findin s A formal mechanism exf t,ts for evaluating training candidates' .

entry $ev.el skills and knowledge. Selection examinations are ,

administered for trainee aptitude screening prior to training. Training F equivalency determinations are made by supervicory personnel and  ;

documented to justify waiver of training requirements.

Training administrative procedures, if implemented rigorously, are i-adequate to preclude the compromise of tests. Trainee performance is evaluated during the training programs by objective evaluations and i:

qualification cards. Trainee ~ advising is used to provide remedial assistance to students. Procedures are in place and action may be taken . i;

- by the curriculum comittee to rehulre remedial training and retesting >

for trainees who perform below minimum standards. [

Procedures prescribed that learning objectives should be factored into I trainee evaluations. However, because the majority of learning -

e objectives could not be directly linked to job analysis data, the ' [

e relationship between not betest items and seep(erformance-based Findings in 2.2 abovelearning). [

objectives could established 5.0 Program Evaluation  !

5.1 Discussion. Training program evaluation methods were reviewed using the  !.

documentation described in Section A.3 above to determine whether:

h o A documented program is in place to systematically and continually '!

evaluate the effectiveness of the training program and revise the [-

program as required. _

o The training program conduct and content are continually a.cnitored p and evaluated. .

s  ;.

Feedback from on-the-job performance is actively solicited, evaluated, F o

and incorporated as training program modifications. .-

[p

, . - p b

h

. E l b l .y., .y - c . ....m_ .

.- .; __ . - . m._, m

'j n [ ,_ *'i I#

I_ N-! * * ' ___

. :. e - . . . - . . -..w.~..u

. t L

, F L

r-

, g

'. . g  ![

p

[f r

o Operating experience and other external factors are reviewed for E evidence of job performance-related lessons learned.

• 4 r

o The training implications of plant modifications, and procedural and achinistrative changes are integrated into both initial and t( -

continuing training programs, g e

o The program evaluates the training management, staff size, workload, [

and qualifications and performance. p 5.2 Findings. The utility has a documented method i.e., the curriculum h

.comittee, to evaluate the effectiveness of training programs and need  !

for revision, and to incorporate into training the lessons learned from operating experience, procedures changes, and plant modifications. A -[~

systematic and thorough review of aggregate trainee test performance  ;

was performed by a contractor in 1984. Examination results are reviewed '

and reported quarterly. Both instructor critiques and trainee critique 11 .

k are usecf for program evaluation within 6 months following training. The y utility training evaluation plan is comprehensive, leaving only one

• possible area uncovered. Beyond soliciting graduate trainee feedback within 6 months after training, there seems to be no long term method of .

continually identifying and aggregating information from supervisors (

about how well the initial training program prepares job performers for F their jobs. ,

Both internal and external training program evaluations are performed. b Documented qualification requirempnts exist for training staff that address subject siatter and instructional skills. Instructor performance is evaluated and documented annually. Trainingdepartmentobjectives .

address the need for more professional development activities for the -

training staff. The utility training staff consists of highly qualified' .

personnel. The relationship of the instructional staff and the training support services group is a constructive one which provides for the .

maximum benefit to the program of nuclear technology expertise and  ;.

instructional technology expertise. .

E C. Conclusion y

The findings from our training audit at Susquehanna indicate that i significant progress has been made toward achieving effective i-performance-based training. The staff found that four of the five r elements identified in the Comission's Policy Statement have been met  ;

and are being effectively implemented. The fifth element, related to t learning objectives, has not been fully met. This concli7sion is based .

on our finding that learnin; objectives could not be correlated to the  ;-

jobtaskanalysisdata,sucithattheexistinglearningobjectivesmay '

not be complete. The staff also found that some learning 06jectivts i required improvement in order to describe desired performance after .

training. The staff believes that the deficiencies described should be identified for future correction by the utility with subsequent j follow-up by INPO.

{

i h

n .. .g., a m m n ,7 , n. m . g ~

. - ~ . , - . y ,-.-. p y

. : . . aa .2 ; ;

. iJ

'r '  ;

[

. , y 4

,o e

'..J

y

. POST-ACCREDITATION AUDIT hP DRESDEN NUCLEAR STATION .

NOVEMBER 18-21, 1985 M,

, r

!;)

A. Introduction . 9 0

1. Background. On November 18 through 21, the NRC staff conducted a 5.

post-accreditation audit of training at Dresden. Three of the Dresden l' programs eligible for INPO Accreditation have received full accreditation status, [I i.

h

2. Criteria. The criteria used by the staff to audit the implementation of f wrformance-based training are taken directly from the "Comission

'olicy Statement on Training and Qualification of Nuclear Power Plant

[

ri Personnel"(50FR11147)ofMarch 20, 1985. In its Policy Statement. H the NRC states that the following five elements are essential to  !" )

acceptable perfonnance-based training programs:

[

.. . F

1. Systematic analysis of the jobs to be perfonned, in

2. Learning objectives that are derived from the analysis and that i describe desired perfonnance after training.

3. Training design and implementation based on the learning objectives, ii

4. Evaluation of trainee mastery of the objectives during training, cj cl

5. Evaluation and revision of the training based on the performance of U trained personnel in the job setting.

{'

3. Documentation. To support its review, the NRC review team requested ,

that the following types of documentation be provided by the facility:.. '

t Instructions! Procedures related to: I i

- Systematic methods used to analyze jobs,  !

l

- Training organization goals, objectives, and plans, t.

- Responsibilities / authority of training organization personnel.

- Methods for evaluating / selecting instructional materials,  !

methods, and media, i t

- Methods for keeping training programs current, - '

- Maintenance of training records, i.

~

- Selection of candidates for training and the gran' tin'g of *E waivers / exemptions from training, ,.. #

O

i

'l q

wn .w , n w .- , . n; v=.- - -; - 7 = mc./

. . _ . . . . . _ . . . _ . ,- 5 u._ . .- ._ ._

m: :_ m .. o _. m _. u . ..:._..

l ,

.  ?!

, ,- Q.;

i; ;

i h.

g P

M

. p

+v

- Evaluation of training programs, and . O F

- Training, qualification, and evaluation of' instructors s p.

• Task lists for the job (s) being reviewed i s-Documentation related to: y V

- Development / validation of task lists, f

- Selection of tasks for formal training. [.

- Analysis of tasks, J F)

- Analysis of on-the-job performance' problems and industry events, and -

i:

. gi

.- Evaluation /auditsofthetrainingprogram(s) E.q o

Roster / organization chart for the training organization ,d j

j Training schedule for the past 6 months and the next 6 months. ,f:i 1;a Based upon NRC staff observation at INPO Accreditation Team visits, the I above documentation is the type normally associated with participation  ;;

in the INPO Accreditation process. Documentation beyond that required .

by INPO for Accreditation is not required by NRC.  ;

4. Scooe of Audit. The programs audited and the tasks selected for review' b

. within those programs were: [, [.

i.

PROGRAM TASKS

. .i

^

(1) Non-Licensed Operator

• HPCI surveillances (Dresden'sEquipmentAttendant)

• Return air ejector to service and/or operation

• Remove centrifugal pump from y; service p.i

• Start a centrifugal pump:

Reactor feed pump [n

• Charging a CRD accumulator l P;

(2) Reactor Operator

• Equipment tegouts  ! .;

• Reactor scram !l

• Placing feedwater regulator P into local manual ** operation

• ~~

ii

• ATWS .  !:'

• HPCI motor operated valve and, ' F pump operation -

l 5

I

.n ... . v :e. wr wpg y y . . :g,.. ,- -

- - = - - m,7- ,- .~ ,~_~ m , y

+ -

. .. ....~.._.....,.,..,[,

t!

H d

W

. U

. (3) Senior Reactor Operator

• Verify ECCS valve and pump li) status -

• Partial or complete loss of AC @

power . k

-

• Direct emergency response rj

• Prepare and approve temporary D procedure changes p

• Loss of feedwater heaters ij D

5. Feedback. During the audit, the team provided feedback to the Dresden U training Staff in daily group meetings. This was done to give the t utility training staff an opportunity to clarify any team questions i arising from the audit or to identify any documentation or other i evidence which could resolve team concerns. t El B. Results of Review

• )

The following discussfon of the review findings correlates with the

. elements of the Comission's Policy Statement on Training and Qualifications, H

t 1

1.0 Systematic Analysis of Jobs to be Perfomed g i

1.1 Discussion. Task analysis methods, procedures and products were i.

reviewed using the documentation described in Section A.3 above to e detemine whether: b Training and qualification programs were based upon systematic  :

analysis of the skills and knowladge required for adequate job ,

perfomance. , lZ Training and qualification programs were based upon a systematic analysis of the entry skills and knowledge of trainees.

- ;li )

On-the-job perfomance problems were systematically analyzed to W detemine training needs. p

. t Industry events were systematically analyzed to determine training needs.

I The systematic analysis considered both the initial and continuing L training needs of job incumbents.  :

Procedures and other implementing mechanisms ensure that the  ;

analysis infomation remains current as job perfomance requirements  ;

evolve.

7q o

.- fiiI j.:

I 1

1 O

- - 7 ; g ~ 3.-.7 w ;w ~ 7 y c - = :v~. - - . - - . _ v-~~~-. _ . _ - - - _ , . - - - - . ;. - c~.~

. - . . .y. - . . . - - -v!:l

= ~ ~.

. .-~;r.

. . . -r- 7: - .

a:w..=.=: a! .=. . . a .:. .= - . . . .: -. ~ .w . .w: .. . .g n .

t

. w e i.

~ .

m p

i:

i

. p p

, i:l 1.2 Findings. A systematic method was used by Dresden to develop task lists for each of the accredited positions. However, there was lialted 1 involvement of operations personnel in validating the task lists and in M selecting tasis for which initial and continuing training.is provided. i; During operator interviews, operators stated that training did not cover I:1 all aspects of their jobs. The identification of tasks requiring p initial training and those reqctettig continuing training was based on 6 practice in the preexisting training program. The knowledges, skills and abilities that enable a job incumbent to perform tasks were not [p1 identified as part of the task analysis. The licensed operator W 4 requalification training program is based solely on regulatory ,d requirements rather than an analysis of operator job task requirements i and performance problems. [j 2.0 Development of Learning Objectives <

-l 2.1 Liscussion. Learning objectives were reviewed for the subject programs to determine whether:'

[H

. o Written learning objectives were developed based upon the systematic h analysis of job performance requirements. H t1 Learning objectives are specific and measurable, and include conditions, actions and standards based upon job performance i rtquirements. pi r

Procedures and other implementing mechanisms ensure that the  !.

learning objectives remain current as job performance requirements  !

evolve. .

[

. 2.2 Findings. In most cases, learning objectives could be found in lesson ~. ,

b

< plans that correspond with task statements (e.g., for equipment l attendant training). Licensed operator training programs, particularly simulator and on-the-job-training segments, did r.ot consistently contain  ;

learning objectives related to documented task lists e.g., for equipment tagout and loss of feedwater heaters tasks)(. Procedures were ,

sometimes used as the lesson plan without supporting learning objectives (e.g.,forlicensedoperatorpresimulatorsegments). [

I Learning objectives did not consistently contain actions, conditions t under which actions should take place, and standards of perfonnance the U

. trainee should achieve. On-the job training (DJT) objectives constituted task statements with no written associated cues or standards of performance as the basis for consistent evaluation of-trainee

[h i mastery. Standards of performance are not stated in the. learning  ;!

l objectives as the basis for consistent evaluation of trainee mastery of i{

1 earning. There was no correlation between learning object'ives and * **

job :l performance requirements derived from task analysis. .

y I}

O tI ii

i

. *_*:Y

_'yiC } '** Q~ ' I TlEl*[??j'T.,_.--.'(!'*"aM<'_??"*f_-._.'_'_~~'._"~""_~~_'?"_~'..*_~__~*._~'~"~M~

• ?: l^ ?h* ~~~'".~~?"'b'""

,.~ .w =:=.u. . ; x.  : .: : =.- . .a.

l -

1 .

[

t th

[I I

. . iu

[

w The training standard document for the Squipment attendant training k program appears to be a complete compilation of learning objectives for k!

- that program but it is not clear whether or how this document is used to b ensure accuracy or completeness of task coverage in training materials, p A matrix does identify where, in classroom and DJT segments, tasks from r the equipment attendant task list are found. There is no training h standard document for licensed operator and senior operator training fl programs. There is no documentation that links objectives addressed in

• various segments of classroom training, where basic skills and knowledges are taught, to DJT and simulator learning objectives which .i are derived from the task list. _

m A mechanism exists, in the form of a training inquiry, for documenting ['

proposed training program modifications. However, observations of i

~s imulator requalification training revealed that simulator training h materials had not been modified to incorporate new symptom-based U emergency operating procedures in use at Dresden. Learning objectives * (

were not used as the basis for evaluation of trainee mastery of traini,ng &

1h the simulator. F 5

The approach to Dresden training program development is unclear as U reflected in Dresden program development guidelines (PDE-5E) which y discuss the use of learning objectives. The training department n procedures state that if previously existing training can be shown to be adequate, then derivation of learning objectives may be unnecessary, fe

< The same procedure states that learning objectives which form the link E between the analysis of requirements and the rest of the program b development process, must.be of high quality, and shall contain actions U (performances), conditions,andstandards. Implementation of learning. (:

objectives containing actions, conditions, and standards, as derived - l:

from task analysis data, was incomplete and inconsistent. .-  !'

3,. 0 Design and Implementation. ,.

p.

3.1 Discussion. Using the documentation described in Section A.'3 above, the i; design and implementation of performance-based training was reviewed to !i determine whether:  ;;

The training organization's goals, objectives, plans, and <

relationships with the remainder of the organization are clearly  ::

stated, b

• The responsibilities and authority of training personnel are clearly -

h stated. $

• I;

• Instructional settings selected (classroom, on-the-job training, and h simulator) are appropriate given job performance requirements and learning objectives. -

j[

... . V

. i

.m..-..,, . . , . . . , .

. .:q.g._. . , . , .

.- - - - ~ . ~ . . . .. -. - . - - - - - - - - - .

ii l- .t e

l - . .

6-

!?

The organization and sequencing of both initial and co*ntinuing N training program content are based upon the relationships among !i learning objectives. };

tb Lesson plans have been developed that include all the ' instructional O materials required for implementation of the program. O Instructional materials are based upon job performance requirements Ib H

and learning objectives and selected using an objective, auditable evaluation of the appropriateness of these materials.

(p p

Procedures and other implementing mechanisms ensure that the design i and implementation of initial and continuing training programs L remain current as job performance requirements evolve.  !

V 3.2 Findings. The training organization goals and objectives are defined I and revised annually and used as the basis for semi-annual evaluations- ,

b of corp 6 rate progress'. The responsibilities of various corporate and station training department positions is defined. -

M Training design and implementation at Dresden has not begun with h

, accreditation; rather the training programs, which have been in place  ;;

since 1975. have gone through a process referred to at Dresden as n training program verification. Reliance on the adequacy of existing L training has resulted in incoes!*te analysis of the selection of E appropriate instructional settings for task training and the sequencing of training. Tasks from the task lists have genera.1w been identified as to whether they are trained in the classroom, simulator or

[6 on-the-job training segments. This documentation appears to be complete  !'

for equipment attendant training while the documentation for licensed ,, [

operator training was not complete. The cross-referencing of tasks to,, +

training segments does not constitute an analysis of the appropriateness ,

nor the completeness of coverage of job tasks in the training i curriculum. On-the-job training requirements for equipment attendants are frequently signed off for tie trainee based only on the opportunity il to perform a task or manipulation without regard for sequencing of -

training. -

The design of continuing training for nonlicensed operators is not based i on an analysis of aggrecste pe:formance of job incumbents or performance L on periodic tests. Rather, continuing training consists of retraining personnel in all training materials on a two-year cycle. The licensed operator requalification training program is based on regulatory requirements rather than a systematic analysis of the aggregate needs of the job incumbents.

• 1 i

Lesson suggested plans do notactivities, instructor consistently containorobjectives references suggestedfor.le;ssons. evaluation ~t  !

standards. On-the-job training materials consist of a simple listing .of ~

b I l I

m .7 + - . g ,,---g-r. .,, 7. m

• m ~+ ,l,7- M" -

<- 7'

- -. - - - _- . - = . _ _ _ _ - _ _ _ _ _ _ _ _ - . . --~--"{~_~__~_.

. ~.. . .. .- . . . . . . . . . _ . . . . ,.. . _ _ _ __. . . _ _ . _

P a -

7

. . e.

t , ,

7

. o required tasks and corresponding sign-off documentation without b standards for consistent evaluation of trainee mastery. Sjaulator lesson plans generally consisted of a procedure without supporting k guidance to ensure consistent and objective evaluation of trainee p

. progress and feedback on performance.

(n Classroom and simulator training, as observed at Dresden, were conducted tc in an appropriate manner. Simulator training, conducted at the G. E. f simulator, was well-executed in spite of the fact that lesson plans had .y not been revised to reflect the use of new symptom-based emergency f operating procedures. f 4.0 Trainee Evaluation }l 4.1 Discussion. The methods for and use of trainee evaluations were reviewed using the documentation described in Section A.3 above to  !

determine whether: - f-

- .' y~

' . Trainees are routinely pretested or otherwise objectively evaluated.  ;

to ensure that their entry skills and knowledge are consistent with t 6

the training program assumptions, and with the basis for granting exemptions from training, p

• Trainee evaluations (tests) are developed based upon job performance h

{

_ requirements and learning objectives (i.e., skills, knowledges, and abilities). ,

-

• Trainees are regularly and objectively evalpeid throughout each  !;~

training program.  ;

• Established methods exist and are used to provide perfyr: nance

~

i:

feedback to trainees and to deal with unsatisfactory trainee  ;,

performance.  ;

4.2 Findingt. Exemstions from training are, as a general rule, not granted j at Dresden, altsough there is a procedure and form for documenting  :

waivers of training. Written and oral examinations are routinely used i.

and documented for evaluation of training completion. Written i' examinations, as reviewed, appeared to address the training for tasks t.

selected for review. Written examinations are maintained under the [

control of lead instructors and exam results are maintained with I individual trainee records. Documentation of trainee performance i:

evaluations for on-the-job and simulator segments of training consists only of the required sign-off for tasks, with no accompapying performance standards or criteria which could be used for objective and consistent performance feedback to trainees or, in the case of  ;

requalification training, to their manarement. The staff observed** l' simulator requalification training in which licensed op'erators .

repeatedly experienced difficulty understanding and controlling a '

I simulated ATWs event.

L f:

. ,_ ,. .) -

~ , m -. y ..... . ... 7;.,.7 ,7..

, .g y...

3,..,7

~

7 -'

7,7 77.,, . , . 7,.,..,..__......m,,

- ..- - ..__2

.a n =: =..u_=.a.. ~ = , . . . _ . ..;,.... .  ;..-. .. , _

. . = .= w ,

t. . i D

.' h 4 Y PI

. y-The policy for prescribing remedial training allows a case-by-case determination of appropriate action by the performance evaluation

[g cosmittee; however, this policy does not prescribe criteria for y acceptable levels of performance or remedial strate les. Except during L.

requalification training, trainees are not removed rom training or O

~ associated job duties wwn they do not meet individual training criteria il (e.g., on-the-job training tasks) which they may repeat until H performance is acceptable. H 5.0 Program Evaluation [o u

5.1 Discussion. Training program evaluation methods were reviewed using the documentation described in Section A.3 above to determine whether:

h y

vi

• A documented program is in place to systematically and continually 0 evaluate the effectiveness of the training program and revise the fu program as required, t

[l

• • The training program conduct and content are continually monitored * [I and evaluated. p; i

• Feedback from on-the-job performance is actively solicited, 1 evaluated, and incorporated as training program modifications. [

w Operating experience and other external factors are reviewed for [.

evidence of job performance-related lessons learned. N

. p

• The trainin implications of plant modifications, and rocedural and .;

administrat ye changes are integrated into both initia and . F continuing training programs. -. p

• The program evaluates the training program management, staff size,  ;

workload, and qualifications and performance. N 5.2 Findings. Periodic comprehensive reviews of Dresden training programs  !..

are conducted by the Production Training Department of Cosmonwealth .

Edison. The evaluations include critiques of all aspects of station k f

training, including recomendations for areas needing improvement. .

Dresden station is not required to formally respond to nor implement the recommendations arising from these evaluations, although the Dresden -

L' training staff stated that problems identified are remedied. The .

Production Training Center also 'does training evaluations of any newly l developed and implemented training programs and produces training i trending reports which describe trainee attrition and completion rates for Dresden operator training programs. .

a Manymethodsareutilizedtogatherinformationontraintr@ program.

effectiveness, however, there is no systematic analysis of aggregate f trainee performance or training critiques. Individual course critiques, j 8 .

I 4

h l

7 . , . c . . _ . ,, _ . . . _ ,. y.., y ..;, .y . g. . . .,; ..

.-- L. : ..

g

, v. - - -

. . . . . ... . - ~. .~ . . ... -.

l '

l '

• 0

. ,ff.

l

,.~,

t

.p. b

(

instructor critiques, and trainee evaluations are routinely collected following training. Analysis of this information is informal and may p.

result in training changes at the general discretion of the training b instructor and training supervisor. Similarly, supervisory evaluations 6.

N of job incumbents are not combined for analysis to identify possible training program weaknesses. Rather, individual training inquiries must I be submitted by job incumbents or their supervisors for consideration by r training department personnel in making program revisions. b 9

The audit and control of contracted training at the G. E. simulator appears to be weak since evaluation and feedback on simulator training J!;

results is not sufficiently detailed to allow analysis. b y

Qualification and training requirements for Dresden training staff 'f address both subject matter and instructional skills. Instructional r staff performance is regularly evaluated and continuing training is [

available for instructors. .

[

, C. Conclusion .

L The findings from our audit of three accredited trainin; programs at I-Dresden indicate that training program revisions and ensancements are i being made. The staff found that three out of five elements identified in the Comission's Policy Statement have been met and are being ['

implemented. The elements related to a systematic analysis of jobs and j" development of learning objectives based upon job analysis have not been [.

fully implemented. This conclusion is based on our findings that L

. Dresden's job analysis ~did not identify the task supporting skills and j knowledges to be included in the training curriculum and that Dresden's -

learning objectives do not consistently contain conditions and standards it for task performance. The staff also found the learning objectives in-  !

Dresden's training curricula to be incomplete and inconsistent. It is" s the staff's opinion that the deficiencies identified should provide the ,

basis for a comitment by the utility for training program improvements

, with subsequent follow-up by INP0. i I

V k

r l

r, i

1 .

[,

,. l k.

. 8-

,.O ._ _ . s

_ ,. .., _L.

, . . y' s

+ . , . . ; :. w ;. . .. . . . , ... . , . , .- . . . . . . . - - . . ~ .. . - .. . .. . . . _ .

, l'

. . o y

. POST-ACCREDITATION AUDIT E OCONEE NUCLEAR PLANT -

U DECEMBER 16-18, 1985 ,  ;,

y .'

p:

A. Introduction [

,q

1. Background - On December 16 through 18, the Maintenance and Training F Branch of the Division of Human Factors Technology conducted a }. ;

post-accreditation audit of three training programs at Oconee Nuclear p Station. The programs which have achieved INP0 accreditation status are C the Non-Licensed Operator, Licensed Operator, and Senior Operator /STA  !.

programs. The review focused on a sample of tasks from each of the !H accredited programs. ,

h y'

2. Criteria - To audit the implementation of performance-based training, the staff used criteria ta ten directly from the "Comission Policy

• Statement on Trainin and Qualification of Nuclear Power Plant

, Personnel" (50 FR 1147) of March 20, 1985. In this Policy Statement. P the NRC states that the following five elements are essential to *

.I acceptable performance-based training programs:

si

1. Systematic analysis of the jobs to be performed, ,

2. Learning objectives that are derived from the ana' lysis and that ,

describe desired performance after training,' --

-l

, 3. Training design and implementation based on the learning objectives, '

4. Evaluation of trainee mastery of the objectives during training. .

5. Evaluation and revision of the training based on the perfonnance el h trained personnel in the job setting. R Y. Documentation - To support its review of the licensee's accredited 'l programs, the staff requested, in advance, that the following types of  ?

documentation be made available:

Instructions / Procedures related to:

{Q J

- Systematic methods used to analyze jobs,

t. .

- Training organization goals, objectives, and plans, U I:

Responsibilities / authority of training organizati6h personnel, j Methods for evaluating / selecting instructional materjals, methods, jl and media, . . ..

1-

. - Methods for organizing / sequencing of training, ,.

{j

. i' 1  !:

, l

-.....y....; -,p.;..,;n. ,.,,3.g. .

y.y .;-c.,. e .

- -- m,I  ;

._ -. - - - - . . - _ _ . . . . ~ . . - - _ . - - . - . . .-

t

!. . I, I;'

,/

. . t

. [

t,

{. :

[j

- Methods for keeping training programs current.

. . ,1

- Maintenance of training records, d <

. - Selection of candidates for training and the granting of .I I

waivers / exemptions from training, 'lR

- Evaluation of training programs, and 9

. a

- Training, qualification, and evaluation of instractors k

• Task lists for the job (s) being reviewed r

• Docum:ntation related to: *

- Development / validation of task lis'ts, )

- - Selection of tasks for formal training, jj

- Analysis of tasks. l

- Analysis of on-the-job perfonnance problems and industry events, e and il

- Evaluation / audits of the training program (s).

i' Roster / organization chart for the training organization  ;

• Training schedule for the past 6 months and the next 6 months l;

• Final accreditation team report u

,\

Biennial status report on accreditation [l

. , s Based upon NRC staff observation at INPO Accreditation Team visits, the  !

above documentation is the type normally associated with participation i in the INPO Accreditation process. Documentation beyond that required l by INPO for Accreditation is not required by NRC. fj r1

4. Scope of Audit - Following are the tasks selected for review from each j' of the programs audited: 5)

F

!l I

t, e

.. V

! t' f'

ti 4

a

- ~ . . , . . . .. -

....,..,....... . . . . . . . . . . . . , . , , . ;=. .. g. s.7 7 7. . . . .., . ,.7

. a. = . . - . . . . . = . . . . . . . . . . . . . . - . .. e , . , . . -

.- ..- l

.. \

l 3  :

8 i..

c.

Proeree Tasks

• t.

1. Non-Licensed Operator Make out, place, and remove red and white tags i:l i

1 Line up emergency feet' water L system W -

Start up, shut down, and isolate M condensate booster pumps L)

Lj Line up and start recirculating ,i cooling water system j'

2. Reactor Operator increase reactor power according i to specifications and set ij

~ .. controls to automatic at

• appropriate time '

• Perfore reactivity balance i calculation to maintain suberttical reactivity; borate i,

the reactor cooling system as 1 necestery to maintain this '1 suberitical reactivity Direct utility operator to set l up core flood system for ,l emergency shutdown service; - -

check and obtain proper boron ,

concentration for this emergency 'la service. '

3. Senior Reactor Operator /STA

• Classify emergency events li)

Direct operations following steam generator tube rupture .

Conduct post-trip review and ,

recovery 2

Direct unit shutdown from outside control. room. l 1

5. Feedback - The team met with the Oconee training staff once during the  ;

review to provide feedback. In addition to this meeting, there was a great deal of individual feedback on a more informal basis:as the .tpam ',

members sought to clarify various issues with respect to the training  ;

programs under review. Since the team completed its review earlier than expected, the formal exit meeting was held earlier than scheduled.

• This obviated the need for a second meeting with the training staff. , . '-

i

=gy, , ,+ g 4 49 F- % *a-

• 9'4' atw- 9F F;g 9 ' 'EP * *yjggg ps-9e 7 g g, g

• p g 44 ._e pq $$$g$=e -

% g g D'y g@ 4, Si

=_to_ pep 99 f- W AD h a 499% $ PS .4 9

• 6W + -

-. . . = . . a.~- . . - .. .. . - . ~ ~ . , . . . - . . . . . . . . . . . _ . .

4 .  ! [l 4

f

8. Results of Review

• l The following is a discussion of the review findings as they relate to the i.

elements of the Comission's Policy Statement on Training and Qualifications, ii q

1.0 Systematic Analysis of Jobs to be Performed i1 1.1 Discussion - Using the documentation described in Section A.3 above, y task analysis methods and products were reviewed to determine whether- l U

Training and qualification programs were based upon systematic i. !

analysis of the skills and tnowledge required for adequate job ii performance. U Training and qualification programs were based upon a systematic analysis of the entry skills and knowledge of trainees. , ,

• t'

, On-the job performance problems were systematically analyzed to ,

. determine training needs. ,

Industry events were systematically analyzed to determine training needs. ,

The systematic analysis considered both the initia'l and continuing ,

training needs of job incumbents.

i

, Procedures and other implementing mechanisms ensure that the i

analysis information remains current as job performance requirements evolve.

4 1.2 Findings - The task analysis for the three programs was accomplish:1 l

through internally developed, table-top analyse.s performed by job  :

, incumbents, senior instructors and staff from the operations depar?'ent. '

These internal analyses were not carried out for the SR0/STA position. >

For the SRO/STA position, a task list was developed by a senior ,

instructor and two SR0s. During NRC interviews SR0 job incumbents 1 indicated that the SRO/STA on-the-job (0JT) task list was not unlete. .i Our limited review of the OJT task list identified missing ta'2s 'Mr i1 on-shift training or qualification of other operators, ree::te shutdown I and post-trip review procedures, i

The licensee has stated that the training department is in the process }l of implementing the Employee Trainin  :;

This system, when fully implemented,g and Qualification should 5ystem (ETQS).

correct the deficiencies in  :<

the SRO task list. The staff examined the. full documentation of the -

ETQS and determined that the system meets the criteria in the .i Comission's Policy Statement on training and qualifications. It -- .l should, however, differentiate between tasks for initial and continuing i training with respect to the significance and frequency of task and

• I o

i1

, . . . . , , . . . .q.. ..,-,-.n.-.m .

-; ,. 7.n i.y 3, y - p , . . -- . . : . > . 7 ~. n . , y p r ~ -  ;..~-

s..  :.:._. = .. - . :- ~. . . . . . . .

a ~ .: a=.  :. = === . - . .

e

'h p

learning decay. Finally, while there are procedures in place to feed back task changes to lesson plans, there are currently no procedures to ll l

factor these changes back into the task list. .

1

[

2.0 Development of Learnino Ob.iective 2.1 Discussion - Learning objectives were reviewed for the subject progrees b to determine whether: [

Written learning objectives were developed based upon the  !)

systematic analysis of job performance requirements. [

e, Learning objectives are specific and measurable, and include  !]

conditions, actions and standards based upon job performance .q requirements.

}

Procedures and other implementing mechanisms ensure that the i

. , learning objectives remain current as job performance requirements .

evolve.

2.2 Findings - The staff was unable to determine from the licensee's ,

analysis the correlation between tasks. learning objectives, knowledges and skills, and lesson plans. A matrix was.provided in hand-written i form for the non licensed operator program for the tasks the NRC review i team had pre-selected to audit. Correlation of learning objectives to ,

tasks on these hand-written lists were based on training staff knowledge.  ;

Many of the learning objectives for all three programs reviewed did not ,

include standards or conditions. In some cases, cues and standards were implied or were tied to a procedure. Some objectives were stated in -

terms too broad for trainees to know what their expected performance .

. should be, e.g., "Draw the Core Flood System." "Discuss post-trip procedure." . .

There is a training module for instructors that gives excellent examples h of how learning objectives should be written. Although the module has l

not yet been presented to all instructors, training supervisors should ij have noted these deficiencies before approving materials containing a poorly written objectives.

2 3.0 Desion and Implementation  :

~

3.1 Discussion - Using the documentation described in Section A.3 above, the il l design and implementation of performance-based training was reviewed to ij determine whether: ,

ll

. . .. :l

• The training organization's goals, objectives, plans, and relation- l ships with the remainder of the organization are clearly stated..

i-l II

r i

~

, pp.gn.tv -

) .. , ~ 3 t- z ~ n - * : ;*p .* = - *

• ye v ~ (~;,~. y ~v .; ( = r , 7 ; , * *m ,* * . ~ ~ ~.~ ~ m .~r

-w gw w~ ~ *--- ~~

- _ . _ - _ _ . _ _ _ _ _ . . _ . ~ . . _ . . _ . ~ _ . ~ - -

~ ~ ... - . c. - ----- - -

' i l\

}

1 6-

[

The responsibilities and authority of training personnp1 art clearly '

stated. j.

Instructionalsettingsselected(classroom,on-the-jobtraining,and -

simulator) are appropriate given job performance requirements and learning objectives.

(

The organization and sequencing of both initial and continuing  ! I training program content are based upon the relationships among i learning objectives. Fi j

Lesson plans have been developed that include all the instructional ' b materials required for implementation of the program. ,

j Instructional materials were based upon job performance '

requirements and learning objectives and selected using an ,  :

objective, auditable evaluation of the appropriateness of these

. , materials, ,

Procedures and other implementing mechanisms ensure that the destp' E-and implementation of initial and continuing training programs rr ain .

E current as job performance requirements evolve. W 3.2 Findings - The Oconee Training Plan states the training organization's goals, objectives, plans, and relationships with other parts of the ,

organization. The training plan provides for maintaining the goals and' objectives and evaluating training personnel annually against established goals. In cddition. Training Managers evaluate trainee job ,

performance and training needs although this is not procedurally

• 1 required. Although the responsibilities and authority of the training- -

organization are stated in the training ' plan, the new ETQS will define more formally the interface between corporate and training ;j

, personnel.

,s iI At present, the instructional settings are determined by subject matter  !

experts. The ETQS evaluates instructional settings for each task. In '

both the R0 and SRO programs, many of the learning objectives were too .

v fn one of the three RO tasks selected for review, the sequence of theeneral t

' to de learning objectives was not consistent with the lesson plan; however, -

the sequencing of the lesson plan did seem appropriate. Instructional aids were listed in general, but there were no cues for their use in the

-A lesson plans. Some validation of entry-level testing has been carried ' -

out against training and job performance. The new ETQS will formalize -

this process.

• In the SRO program, there is no lesson plan for the remote' shutdown. e task. Although there is some segmented classroom training on this task, .

the shift personnel interviewed indicated they have never walked through the entire procedure.

i v n.

~...:-r -,~.,.-,-:-,e y - , ,.y. . - -c

~ - ~. 3-

-- - v ?rp -~ r ---~~~~-~~ ~ -

. .= : 1 .w.. .. ~- - -

.v . . u. ~ .. ~... u. x.. . _ . - . . - . . . . . -

1

- e

. ['

o ,

• * ,b 7 }w ';

.- l The only training scheduled during NRC's review was in the R0 program. W The NRC staff observed both classroom instruction and the simulator lab.

. The class and lab presentations were effective and followed approved h' lesson plans and laboratory guides.  !

On-the-job training is presently carried out without the operations staff being trained in instructional skills or evaluation. Interviews t' indicated that there were no formal testing standards in DJT; according p to operators, some of the staff are "easier than others.* The training '

department has recognized this shortcoming and has issued guidance that h, clarified the standards and outlines methods for improving DJT. l.

As mentioned in Section 1.2 above, there is a generic concern that b continuing training needs have not been identified as part of the .

analysis; rather, all initial training tasks are subject to continuing <

training. ,

t.

~ '. .

, 4.0 Trainee Evaluation .

[i 4.1 Discussion - The methods for and use of trainee evaluations were reviewed using the documentation described in Section A.3 above to [:

determine whether: '

Trainees are routinely pretested or otherwise objectively evaluated  !

to ensure that their entry skills and knowledge are consistent with  ;

the training program assumptions, and with the basis for granting ,

exemptions from training.

• Traineeevaluations(tests)aredevelopedbaseduponjobperformance i; requirements and learning objectives (i.e., skills, knowledges, and.  ;'

abilities). .-

• Trainees are regularly and objectively evaluated throughout each i

. training program. .

i  !;

Established methods exist and are used to provide performance i!

feedback to trainees and to deal with unsatisfactory trainee  ;;

performance.  ;:

4.2 Findings - Candidates for the training program are tested using the il; North Carolina Test Battery. To move on to the RO training program, i-candidates must have appropriate seniority, experience, training and !i qualifications, and pass a comprehensive examination. Examinations are  :.

used to exempt trainees from plant-specific training an(/or nuclear .,

fundamentals. -

[l There is no formal 0JT testing for SR0/STA tasks. Learnin'g' object.fres 4

. in this area are too general to support test questions. These -

'r

(

objectives are presently being upgraded to remedy the situation. I.n. the' '!

other two programs reviewed (R0 and 1 0), test items are all linked ,to. '

. .. i

\

f l . .l 1 ii pm;wpy-~:e - ~~ v m . m y:.- mr y n y c y ;. u.ymm

. , - -- _ _ . . _ - m r = y- __ ....__  :

= .w::w . .:... w.z:a za . =. = - . . . . ~ . - . ~ a ~w. . . .-

t tr>

k. -

.g.

1;

(.

learning objectives. Appropriate precautions are taken to ensure exam I security. ['

There is e strong trainee evaluation and feedback system, supported by  !

appropriate documentation. Instructor counseling is provided, but L.

remedial training is not administrative 1y required. The process t' encourages candidate' initiative through strict examination standards to [

continue in training (i.e., three weekly quiz failures results in

~

m terminationfromtraining). h 5.0 Program Evaluation il p

t 5.1 Discussion - Training program evaluation methods were reviewed using the N documentation describeo in Section A.3 above to determine whether: i r

A documented program is in place to systematically and continually , [s evaluate the effectiveness of the training program and revise the

, program as required. ,

t, i

The training program condu:t and content are continually monitored ,

and evaluated. ,,

Feedback from on-the-job performance is actively solicited, evaluated, and incorporated as training program modifications, i.

Operating experience and other external factors are reviewed for Y evidence of job performance-related lessons learned. H

• t The training implications of plant modifications, and procedural and j administrative changes are integrated into both initial and -

continuing training programs.

l .

The program evaluates the training management, staff size, workload, i' i and qualifications and performance.

5.2 Findings - Oconee has a well-developed and implemented program in place 2 4 to evaluate and revise training proprams. Shift personnel interviewed l i

expressed concern, however, about tie length of time it takes to get  :

training on a new plant system into the training program. They i specifically cited three new systems in use at the plant on which they j have not yet been trained.

An analysis of examinations and tests is conducted to determine if any '

of the questions appear to be "outliers.' 1.e., questions that are answeredcorrectlyorincorrectlybyaverylargepercenta!eofthetest population. This information is fed back either to improv testing or  ;

j to determine if there are weaknesses in the training program. -, .

l . i j Instructor critiques are used for evaluation and revision of programs.-

j Trainee critiques are also used for program revision. All feedback. . ..

I i

pg:w gew n =:w~ w - .

nw. _

.m ~:vmm.:x=y na c.mmm ~ w_ -~._ _ -a

.x .==.~:.~.- w . ~ . ~. w.. _. . . . ~ . , ~ . . . . .. .

'e' .

?-

I

,o

.g. -

mechanisms identified in the staff's review procedures were in the [

Oconee Training Plan; our review also confirmed that the feedback r procedures are effectively implemented.

The training staff is evaluated twice a year by supervisors, students.  ?

and instructional systems development specialists. In addition. -

instructors are videotaped and critiqued. Instructors participate in b the criticue of their own video tapes. Initial and continuing training M are proviced for the staff. Scae instructors did indicate that they had 0 to use their own time for class preparation because of their workload. l' Oconee conducts a zero-based manpower study annually to determine t:

staffing needs for training. ]

C. Conclusion ll The findings from our review of the NLO, R0, and SR0/STA training ,

~

programs at Oconee indicate that the table top analyses of the NLO and, .

R0 programs produced a list of the tasks that are performed in these ,

fobs. These tasks can be linked to learning objectives in lesson plans.

There is no existing task list for the SR0/STA position. In many cases, the learning objectives do not include conditions or standards, or

' are too broad and vague to link to test questions. Because of these i shortcomings, the training program would not meet two of five elements  ;

necessary for effective performance-based training contained in the Comission's Policy Statement on training and qua ifications. It is  ;

clear, however. that Oconee's training department has been successful in training and developing skilled operators. The operators' performance i -

j on NRC requal exams (24 of 25 passed), their excellent operating record 2

(no operator-induced . trips in ever 2 years) and the high quality

• training observed by the staff are positive indications of effective -

• performance based training. Another significant finding is the fact -

that the training department has made substantial improvements to their

, program since being accredited in August 1983. The staff did not i discover any weakness or deficiency that had not already been identified 4 by the utility and that utility corrective actions were already i  :

underway.

I i L

J t

9 e

• 4 et 1, -

a, .

l 1 f-

~~

' ~ '*9 ?*[; } ?' '* * *]~"* *"J" ' VE'. T' GfW **G?T?R*;7'

~ ^

'"'L* ' ' ' ' ' '

p;** K 3'f Q: '., : ',L * * * ~ ' ~' , ,

_ . . . . . .. - ~ _ .. _ ... _.._ _ ._. _ _ _ _ . - __ _ _ _ . _ _ --.

.: ) s.

(

j ,j December 10. 1985 .

Observation of INP0 Follow-up i-on Post-Accreditation Audit '

F Susquehanna Nuclear Plant p n

On December 10, 1985 J. Persensky. Section Leader. Personnel Training F Section, accompanied the INP0 accreditation team manager Walter Popp, on a  !

visit to Susquehanna to follow up on the NRC post-accreditation audit of "

October 21-23.

The review was performed in one day and involved a survey of the same 15 b tasks audited by the NRC post-accreditation team. The utility had advance N notice of the review, and of its scope, so they had gathered all relevant  !

materials and had collated the documents by each task. Thus H statement and element was followed by the appropriate unit of (s}instruction each task ,l (with learning objectives), referenced procedures or other documents. - '

simulator lesson plans and' job qualification sheets. Examples of tests were I

made available on request. .

Based on the NRC review team's favorable finding regarding the analysis U elemMt of the Susquehanna program. INPO made the assumption that the analysis was adequate. The utility staff was asked to ' walk us through" from the task statement to learning objectives and training content. and to provide '

examples of test items. ,

e s l

, The Susquehanna 0 raining staff had prepared the documentation so that the '

sequence from task statement to learning objectives to training content was easily traced. Handwritten notes placed next to "Specific Learning .

l Objectives" in lesson plans indicate the associated task / element number. H l

Sections x lesson plan of units of instruction identification matrix.(lesson plans)which Test items were could notedbe onlinked a taskto/elemen}he t.

learning objectives were located and provided to the reviewers. A task l analysis had not been performed for the Reactor Engineer job but was underway i

, at the time of the review. - -

i Findings l It was evident that the Susquehanna training staff was able to track from the l analysis information to some of the learning objectives and units of (

instruction. However it was also apsarent that numerous learning objectives  :

existed that were not linked to a tast/ element statement. Also,the n I terminology often was different between task / element statements and learning i objectives so that an explanation of the linkage was sometimes necessary.

, The Susquehanna staff provided an example of a new format to be used for

! learning objective listings. In the new format. all relevant task statements i will be referenced to the terminal objective, with specific pbjectives tied  ;

i to only the relevant task statement for that objective. The neir format ~ kill '

not go to the element or KSA level. , ,.

,e*

i i

n

- , y ,nw - - w m ,~- . ~ - -

., w - c--w wn m g=.~ : ~ mr r . v rm em 1 c;>.,. .

3 t. s -m- . ,- s. ca ,u e y . s ,

. s. . m.2 a . s , s .. . . .

. .. a m.= . . . . . .. ... m. . . . . _ . . . . _ . . . . _

i.

l. . ['

I *

. . i e' .

y lj

!l ti Learning objectives did not consistently include conditions or standards of h perfomance but did include actions. There was no indication as to whether R a learning objective (or task) was specifically for initial or continuing ;j training or both, a For two units of instruction delivered by contractors (Reactor Engineer and part of Mechanical Maintenance), there were no learning objectives and the U task / elements could only be generally correlated with the lesson plans. The ,

Susquehanna training staff seemed to have very limited involvement with t monitoring this contracted training. Tests for these lessons are not p reviewed so it would not be possib e for the utility staff to assess whether j the level of knowledge acquired by the trainee was sufficient for task  ;

performance. ,

c.

In general, the Susquehanna training staff was able to locate and correlate.

test items and OJT qualification items with learning objectives,

[

y In ord'er to ascertain that the utility training staff could use the accredited system, an additional task was selected for review which was not among NRC audited tasks, i.e., loss of feedwater heaters. The instructor who j had been discussing the nonlicensed operator tasks was able to identify appropriate units of instruction and pointed out examples of learning ,

objectives and 1esson_ plan elements wiich described the feedwater system. ,'

Only about 50 percent of the task elements listed in the analysis were identified by the instructor.

^

. ;i Conclusion -

1 The follow-up review at Susquehanna conducted on December 10,1g85, genera 14y  :*

confirmed the findings of the NRC Post-Accreditation Review Team. Prior to - i, i preparation for the follow-up review, there was no documentation which linked '

task / element statements with learning objectives and training content.

However, it was determined that the Susquehanna training staff does have the ,

capability to correlate the elements of their training program and are making i

progress to improve the situation, as indicated by the exam >1e learning i objective femat. Also, there was evidence that learning o)jectives are not ,I complete in that they did not consistently contain conditions and standards, ij Further, this review found that the utility did not appear to have sufficient  ;!

control over contracted training. -

I

. I I *

\ .

l i

l .

j i l

1

'l

.J

,,v,.'" w,,-~.,,-,,,..-~.-s.. , m m,-m , , -,, --- - J )

m. -m mm,4 4 7n

' * *. ?.

• it' .-

_7 *

{ y- s , .

s i f-c

.. . y I .

TRAINING REY!EW CRITERIA COMPAR!$0N MATRIX ,.  ;

Mo

1. Puroose p

Staff performed an analysis of the relationship between the content of  ;!

U training review procedure used by the NRC in performing incident-based O or post-accreditation reviews and the content of the INP0 criteria under I which plant training programs have been and are being reviewed prior to $

ii accreditation. The purpose of the analysis was to determine if there p was any inconsistency between INP0 (INPD 82-011 and INPD 85 002) and NRC li understanding of effective performance-based training. I.

2. Sumary.

... (

g!

OneNRC'reviewquestion(5.3-useofinstructorcritiquesoftraining ,,

forprogramevaluation)didnothaveacounterpartcriterionorgeneral ,

coverage in INPD 82-011. However, the revised INP0 document (85-002) -

did contain coverage of this item.  :

i i Three of the NRC review questions,were covered by the earlier version of INPOcriteria(82011)butwerenotfoundintherevisedversionofthe'

~

, INPOcriteria(85002). These NRC questions related to trainee ,,

evaluation and include precautions taken to preclude compromise of test  :

- contents (4.4) use of remedial training and ratesting to improve traineeperformance(4.6),andremovalfromjobdutiesfortrainees l ]

]

failing to meet minimum standards after remedial training (4.7).

One area covered in INPD 85-002 was not specifically or separately dealt l

with in the NRC review procedures. This was the area dealing with [

i conduct of simulator training and use of simulators for evaluation. I j However, the staff believes that the criteria covered by,1NP0 in this l

] particular category are covered generically in the NRC quettions related  ;

to instructional settings (3.4) and adequacy of training ().9) witb.its  !~

l associated training observation checklist. Also the proposed -

[D]?,N NY f~ c;

,1,  :

4

.t.

p 10 CFR 55 rulemaking deals extensively with evaluation of ' simulators and I thus is a better medium for review. Another area, support of training  ;

with facilities, equipment and material is covered by INP0 in some [

detail, whereas the NRC review questions do not focus on this area. h Dnly one NRC criterion has a cross reference with this area. That [

question is 3.4 which deals with the appropriateness of instructional [

5:ttings. [

I' During the review, comparison of coverage c'f the NRC review questions to M i theINP0TrainingSystemDevelopmentMandal(TSD)wasalsomade. This I survey was done to establish the background for INP0's statement in the* h

- , n TSD manual that "The training systent development (TSD) approach . ';

presented in this [TSD) manual illustrates the types of processes and  :

activities necessary for performance based training." (P.1 1,TLD  :

manual). All NRC review questions were covered in the TSD manual. [!

leading to the conclusion that the staff's concept of performance-based training and that of INP0 are consistent with each other. .

,t 49 -

i

,a i

3

- >8 l

. .1 j . . j

,1 i . .. .

s;

'l

. .i U

~n., - -

~....e.~..,..,-,.v. .v~--.-~.- n-.,. s.- , .

. n .,' .~un. . ,, ,. -:~,. . . ..

s

~. m . ; : .

s j , , . ., ..__,,-.,s._

,. n - ,__.__.__._

.1 . . .

9 'I#ib A07t;bitAtTOV Cell 2ter4 '

j( MRC TDitGW CRtTERIA 81-ori as_coz .

n -

h T.S15TBMT16 AQAL'{Sf5 tf tttGJDS$ 10 $GTLqt. -

~

Tbe MEO .

-~ ~ .~ ~ ~. ~nu m w. c .r. . ax<>

y u.s 7. , = -

g.,, 4. , g. 2. _

._ s . . s. . . .

.w , z m- ~ ~ ~-- u = at.c.2 8. ('X0 4(e) 4E

.:l ,.t

._z n

,.3 ~, ~- ~ .= _. -

- ~- =*- r.r5. i 4 2 s.s > s.6 i.4 ~ -- . ,=_ x . -

~ - ~ ~ m

~ ,.~ .~t

. me c.z.b.(04(t)4% 6 ,c3

,. g . . . . ,

w~

2

r .

w - w -- - .u-,=

. .. -w . .  ;

5 c.t,a_(t)(a.) 5t ,

.1 I.5 . _ a w ,* u _- __ _ -

2 -- 3IECE*OM/ q-l q -3tc C. L.e.O)

,3 _ .

j

.7 . . _ , z

= _ - . - -

n_ ws._ wa. -

- - = yt c. ta.6)(c.f Fg 5 4 ;12.4, R4 i b

3t f.8 s._. , = _ m_ _=_.+.ws , = e m. -  : O 2- -(D( st.T

)

- - - - . , z c. La (0 1 . .

3  :,- ...

Yj r.9% :y.. a.--iac,,y, ,.f----M- .-na .ng c. 2.a. (2) 4. , 4.1 i

, r2 ~ . -  :

I

.j .

l x A . t . c . b X o.),

' ~

1 - - - -

j M M M ' - 3' M *" " ' . 2 A. 2.c (2.) 4.4a.d,.9.t' iz.I , s-i .

6 i j 1. . . . . . _. _ .. . - . . _ _ . _ _ . _

~,..-_~...--...:.7-,..._.._,;.,-.-,._.-.~. . . . .

_ - - - +-

~

1NroMtebrAnces cettnetA - .

• shCC 1ltnheW cetTuttA gz,_o g g gg._ co,. _

ft j EI**acMor a:taewresegmt e -

~

~~

.j 2.i.,-.

,:-_:_ .. ~ , -

.E.c.xb.0) 5 l a

1 1 r. e a - = _mo u.  : e

= . : = .. .a _ s _ :_- .= s_s..s n 3!I*. c.t b %(sm%.s s i set-$s(st*Mst sS .

13 w. _.w  :._: _ - o u _ .w u e.* ,

ao)_. _ _ ._ . __ - .

j *.- - . - -**=1 .31r C.Z.b(x) 4 c.3 cx.b(s)(h).. _ .. _ _. _

24,,, w ,,.,,,,,,., ___x E C.Z.b 0)d(O

< ~ -

=t c., 2 6.1 ..

! 2 5- * < ~ _

..1. r: r --

• o-.a.= .~ u -d u . - 1

-=* g c,g, b (t) 6.p w ) g,,,3

. - .. _ -~ .. .

, q 1 2.t,. . w = uu;-* 2 c.z.b (2 4cs).(b) (. 3 -

a .--a~ s..a .

l i

[7,*.w = - -t c .z --

1 med

• e *=' -

==='d 5.grC.2 b (3XA) -

c 16. OT .

6.37.247.3 j j __ -. - -- - -

/ .

{,$ p,. e.,. .reen.s, m h ^: 12 shoe e e o.t_vese._

[ h [. LU(2 i vua.e.she __._ _ - d=u T_ ,

/2 7_. .. . .

i 4

• e

-l

.. s

+

l I

l1

,3 s .

e .

.t , .

I e i -

...n... ~.n -+*~*~-_ ~ w-. -~~~- .ee. .m

- _ _ . _ _ _ -- _____~___m-

__ t_ *

~~~__ *~_~_

... .. t i 3 M ,

g - 's fg '.

I .

- ' ' t,s

. oo - i ,

S g l ,c T, u au = '

d' i 9 . n

"' y.j..

i tf in y s  ! - in I .T ' "a E I

A , co n.

t ao  : , i 8

4 6, y ea  :

5 i

d'  :'"  !

t'

) i u

9 ' E o s 99 i

5 1 3  !

e $7AeyS $ n- m G E  !

hi 30 "= ; 2. c ! cc h i d a i s o,s .

WN i j #34i g ;i d T $, "["d l ci i yd TJ <d E

U .,f.4 < *! 'O Q C.

Wu W . ( di M i sl % 3Mh V p(  !

~

. I

,*; ],f 1; '

A l: f ,

'85I l!-

i

!. .]

I L

] $ .

L 1{'l g, n !

1 i l

} 'j p'I l l i 'l -

}'

I ,I h k ,j h .tkf if t

i s e h i i

,.i s, , i 1

3 i

. , 4 '

l fk . k;6}  ! l 'Ihl f,Ii!h~

s h) gl iwi I, l Ij$

1 I h -j .I

$ g#

osh,jf z pS.fr'l,il j i L:

e.,l

((

y s ,e m e f  ;!

Lu . .. I ,i, in 1 sr .,

i

.[-

i lui ipa i i= ,.

=I v- I y J e o ne ai m e-. o.1

~

' a-

% m m M vi vi Ma

...,...,,.,.r....,... ... ... . .

_ - _ _ - _ - ........,,,._...,.,..._..,.,...,7,_

l

l ( **

i Wc.1tc*en) ceasea sz-ott ar-coz. .

o -

-4 ~

n - '

di Twuts alAti)ATT&/

_-.---~a*.. E C E e O) (I

d.4 .g

--- _ .- ~ - - - - . - - .

y . _ ._t .

I.b  ;

.' 4.1 _ .- 2:2 . u --- -

- c =.a.s K C . 1 . e. C r.,)

=....,: .-.-~t  :

n 1

~i 4.3 n--.- - 1 e ==.*= 6.I, 8 f.f T4 0 E .

___2-w

_3I1.HIC C. 2.d28I3)/)-

r::=t (IXc i

A . % . __  :  :: - . m - aun ~.* _ay, c. g. e 0,) -  ;

7, .

4 G . ~ =.a -- a ,~ ~ - _ - ~K c. L eis)>6)/s)4(s) 8 G r 8 T> B 6.> .

. :: : .. . .1 s . A , 1 s , , ,.o ,

~

If. Sr.

s h

a -

.~ =_ s s ~s ses u m e gC, c.L.e @) -

i t d 7a, g C- { g(t[) -

~] .m * *.n, .** m,*.n- e 4

. ~:m

.a - . . ~ -

i .

-]

-) -

.; , t

.4 ,

• j

'i .9 . ,

)  ;

I :I jij

a

i,9') >

.1 _ . _ . . . . . . . . . _ _ . . . _ _ . . _ . . , _ _ . . , _ . . . _ _ _ _ _ _ , . . _.,,, _ , . _ , ,,

1 .

i . - ' jl'.,j1 . ii : ; l- ' ' $, L

. [

~ -

~ ,

l -

~~ . -

. v e

. ~ ~

1 r.

b.

AL t . F '

• R o .

2 a z 3 8 3-v_

L e c- / .

/ .

y 2 -

5' ( a ti

o. f.S P

( a s-r *

' 3 3 n e

A, l 2

z (

J 7

3 w>

7 s f 2

/ ~.

s z / g. 2 3 l

5 f A - _ 7 e

7

~

~

~

t A  ! w r e 4> d. . . T r

e,

- k tc )e a r

er ( t Z. -

y

)e> ) g. ).c , d, A ( (ea

)+r 4 _

Pa r

r )g

)wCT 8

(

) 1 c, .

N o

( 6 (e C (6 ol ,

e

, c ,

J -

c.

q .

e..w (g e. S' .

4e. .

2

. ~

z g zI ,I.

- t, n a g2.M.222

. . . 2_2 .

t

. g 7

A A B C.

C c.

A. C C c c A. 3 ~

c .a G 1 Lc e *M (. - L E

I T J g c u_

t i

z g _

t S g y:J_

u l

]

f W. I -

=

. . .e r

a . w.

e

. a_2 .,

~.

e ~

. n

,- . . o .

m . e s

< ew

=

- e d.

n-i - -

a =.

'd_ -

~

a w ^

- *. l .

a.

• r m a.

- 4 r . -

s, e .

w. =

< _ w. w

. . . .- w .

M 1

- _ . c- c. w z, -

m . -

O _

= w = .

T =

.. w =

c T

A .- w ,

w = am. .

w. . a. . .

U ._

. e . . _,

m =

4 L ..

a..  % -

r 2 1

\ . . - -

, .~ . .

a :_-1_

E .

a. .

c. .

w

. . .t - .

p.- :x M . . - .

=

y.

we,

. t 4 _. . >

a ns C

. 1 a..

G . . - . .

-- e. -- _

p

.w.~ &n m.Ik= _-

0

=  :

s, 1 .-. - .. a

.=-

t e i

2 3 4 f 6 -

7 8 _ g g I _

g.

C5 _

5- F-5  : -

5 E 5- . _

li Ajj '

. )_5 5

... c' . .  :'

. i; ;1; . e : :;! ;lii

,i !

fi lu

--. ~ - .

. - .. : : 2. :.: . : . c. x. a.. w. . - ., x . .. : . . = ....... .- - .

, I

. DI  ;

L INSPECTION PROCEDURE 41701 [

l ',

LICENSED DPERATOR TRAINING R k

~

PROGRAM APPLICABILITY: 2515(BASIC) i:

41701-01 INSPECTION OBJECTIVE ,.

.- 01.01 Evaluate the effectiveness of the licensed operator training F program. ,

41701-02 !KSPECTION REQUIRDiENTS 02.01 Review plant operating history and interview plant personnel to  ;

select several abnormal events or unusual occurrences that might have been caused by deficient training. Additionally, select at .

least one major operations activity recently completed. For the

• 4 i

i events / activities selected

a. Evaluate whether the classroom training received before the  ;

event / occurrence was sufficient to have either prevented " .

1 the occurrence or mitigated its affects by recognition and proper operator action. r!

b. Evaluate whether the on the job / simulator training received i before the event / occurrence was sufficient to have either prevented the occurrence or mitigated its affects by recog-nition and proper operator action. .;

c. Evaluate the qualifications of the personnel involved.

i

4

d. Determine whether any lessons learned from the event /acti-

! vity were effectively factored into the training program.

J 02.02 Determine through personnel interviews whether itsensed opera-i tions personnel have received training for specific tasks as-

signed, such as complicated surveillance tests,' major -

plant '

l systems tests, refueling, and special procedures.

02.03 Review records of the senior reactor operators ($Ros) and reac-tor operators (R0s) involved in the operational events selected l i in 02.01 and 02.02 and verify that they contain the following

i l

I i i d Issue Date:. h

! , .n ,;e .

v.- . - . ~ ,-,...e. ...-,_,.,..,,,,,.....,.:

>r - - - - -

.n,r >

, . w , ..;., . , .

.:- . . a.::. , _ , a * .::.L ... .~ . ~ .: =.. .. a ..- : - . . . - . -. = . . . - . =a w. = = ; = _j

.' ' .' yl H

41701-02.03c LTCCW$r0 OPERATOR TRAIN!wo ;l

. .q

• i

a. Copy of the most recent annual written exteinstion and the p individual's responses, i

b. Documentation of attendance at all required lectures. F

c. Documentation of the required control manipulations (10 CFR 55,AppendixA, paragraph 3.a). ,~

!:1

d. A copy of the most recent perfomance evaluation (see j Appendix A, paragraph 4.c.. of 10 CFR 55). [;

l

e. Documentation of additional training received in identified deficient areas. '

. s-

f. Documentation that any required procedure reviews and/or  !'

self-study has been completed. F9

g. Approval to resume licensed duties for those individuals I who p 1

(1) Failed the a'nnual requalification examination. -

o (2) Received an unsatisfactory performance evaluation. [;

(3) Were not actively perfoming the function of an R0 or ' .

SRO for a period of 4 months or longer.  ;

t l (4) Courr.itted operational errors. .

I.

h. Documentation that required reading of routed material has  :

been completed. , 1, 02.04 Interview several of the R0s and SR0s whose records were i reviewed to ensure that the records reflect the actual training l received. .

j 02.05 Attend training lectures and verify that the technical content ,,

of information presented is adequate.

! 02.06 Determine the present pass rate for requalification exams given *;

by the licensee over each of the past three calender years. L i

Determine the pass rate for initial licensed operator qualift- t i

cations by licenset personnel over each of the past three

,' calendar years. ,!

l 02.07 Review the progress of the utility with respect testcreditation l

by the Institute of Nuclear Power Operations (!NPO), .

6, I e t l ,,

l 4

1 i

.i i

ssue Date: -r- a l f;

, . ,.y. ., . . j c ww mo;. m, ..r~nm.m__ y , _..a, ..m. ,....,,..._,,.,.__..,.,,..,.

f m.

w s : . . : . .= . . x z . : . - =: . w . u .-. : - x.... . a :.:. . .-  :.

. '.- p G

LICENSED OPERATOR TRAINING 41701-03 i l LJ 41701-03 INSPICTION GUIDANCE ji 03.01 General Guidance h

i

a. In a policy statement issued on Marci il4,1985 the Cassais. Ii sten recognized that the industry, throush initiatives of h the Nuclear Utility Management and Human hsources Comit- p tee (NUMARC) and INP0, has ' made progress in developing h H

programs to improve nuclear utility training and personne .

qualification. The Comission rea11:es the taportance of these intitiatives and wishes to encourage further self- s.

improvement. :1 t:

The following excerpts are from this policy statement.  !

(1) ...the Comission will exercise some discretion in selecting appropriate enforcement action for viola- H o

, tions involv' ng training in light of the NUMARC/INP0 1 initiative. Licensees who are making reasonable  !

efforts in developing and implementing the INP0/NUMARC *

~.

programs...will generally not be cited for viola- il tions related to these programs, provided the viola- -

- tions, whether or not identified by NRC, are appro- i 1

priately corrected in a timely manner. However, vio-4 lations which are not corrected in a timely manner, ,

violations of any applicable reporting requirement. .

and any willful violation may be subject to enforce- '

t ment. Any enforcement action taken during this grace.

" i

period will be taken only with Comission concurrence.

Nothing in this policy Statement shall limit the [

l author' ty of the NRC to conduct inspections as deemed  ;

) ,

necessary and to take appropriate enforcement action

• when regulatory requirerents are not set.

l (2) The NRC recognizes that the INPD-managed accredita-tion of utility training includes the following train- *

ing programs

1 non-licensed operator  !

control room operator '

4

• senior control room operator / shift supervisor 1

• shift technical adviser 4

instrument and control technician  !

' electrical maintenance personnel i

• mechanical maintenance personnel ;i i

• radiological : protection technician " ;1 l

• chemistry tecmicians ,

• on site technical sitaff and managers,. i;j All utilities have comitted to achieving accredita- .I i tion of each of the above training programs. It is  :

l i understood that each licensee will exert best efforts 'I

to have all such programs ready for accreditation  !

! (i.e., final.self-evaluation report submitted to INPO)  !

6 .

3- Issue Date

;;

y .-

,y.mv,wm g.vyw . c - w - -- -w.,-

. m.- ~;c.n w y y r

s: L a

41701-03.01a(!) LICENSED OPERATOR TRAINING t

by the end of 1986. It is also understood that appli- Ii cents for operating licenses will exert best efforts h to have all such programs ready for accreditation  !

- within two years after issuance of a full power opera- t ting license. . [!

For operating reactors, accreditation of the above h training programs constitutes a method acceptable to t!

the NRC for taplementing performance based training. 1; The NRC will continue to review and approve all appli- [

i cent training programs in accordance with applicable regulations, regu story guidance, and the Standard h i Review Plan and to conduct inspections necessary to H determine that current regulations and license train-ing comitments are being set. t It should be noted that training programs for which

- regulations are currently in place (e.g.. fire bri- (l .

gade, energency response, security) or which are not subject to !NP0 accreditation (e.g., quality assur- L j'

1 ance) are not affected by this Policy, Statement.

i b .* The regulatory bases for an acceptable licensed operator I training program are contained primarily in the following (i

documents to which each licensee is comitted to some ex- 0 l

tent .

[-

i (1) 10 CFR 55. Operators' Licenses -

j l

i (t) Technical Specifications. Section 6 ,

(3) M51 N18.1 ' Selection and Training of Nuclear Power f Plant Personnel"  !

l: .

) (4) ANs 3.1 *$ election. Qualification r,nd Training of

. Personnek for Nuclear Power Plants"

(5) FSAR Chapter 13. Conduct of Operations c.

i The insp(ector Examiner s) mostshould consvit familiar withapplicable with the the Operator Licensing licensee. F l

This should be done before the inspection and could provide il:

a valuable insight to the quality of the training program. '

03.02 Specifte Guidance  ! [

i . . .. ,;

Inspectien Recutrement 02.01. The determination.of whether a

. an abnormal event or unusual occurrence was caused by de-I ficient training is not always straight-forward. For j example, a component failure may have been caused by in-

proper operation or incorrect maintenance. A plant opera-j tions. problem. although attributed to a weak procedure, may ,!

j  !

4 !l

ii I

s...,,,-.

~ . , , , Issue,Date: , , , . _ , , , *4,_.,,,,,, . , , , , ,y

_ ._ -.... . = ~...._.2,.. . c.m a . .. _ . c . . _ ._: -

(

l. .

I

. LICENSED OPERATOR TRAINING 41701-03.02b  !

I have occurred due to a com6ination of training and proce. 1 dural deficiencies. In selecting the activities for this el inspection requirement, the inspector must appl 3 a 1 Deral .1

' amount of judgment and not accept the specified root causes p of failures at face value. i e t:

b. Inspection Reavirement 02.03. In cases where individuals O have not been acttvely involved with the facility for a h period of 4 months, the individuals are not evalified to k p,

operate the facility untti approved by the Regional office in accordance with 10 CFR 55.31(e). This is intended to [' ,

ensure that en individual who has not functioned as an operator or a senior operator receives enough retraining I to resume the conduct of licensed activities in a safe.

competent manner. 10CFR55.33(c)providescriteriawhich I'l may prevent an operating license rom being renewed.

c. Inspection Requirements 02.05 and 02.06. These require. L sents provide subjective in>ut to assess the quality of the program. Reguirement 02.06 provides a relative basis

'ic

• for the comparison of programs between plants. The data for 02.06 should be provioed separately for Ros and SR0s. .

d .' Inspection Requirement 02.07 If the licensee has not re- - .

ceived accreditation by INP0. review his cosnitsents to F achieve accreditation and progress in setting schedule J!

milestones. This requirement should be accoglished pri- .  !'

l marily by a review of the schedule established by the 11- p conset to satisfy INP0 accreditation requirements. Mile- t stones from this schedule should be checked against actual

• t

} licensee progress by review of procedures and other train- ,

ing program documentation. This review of accreditation I docuants is not intended to supplant the detailed review required by 1NPO to grant accreditation but is rather in- .).

ii 4

tended to provide assurance that the licensee is. making '1 progress toward accreditation.

i EW ,

) I 1

F i .-

I -

i .

i l !ssue Date:  ;,

J... y .

.-,w.- .w . %

...c..-.,

s.. ~ . , , ._ ..

. 7. , .y.

.y  ;

. . - w. ,_ .a .:

~ l t}

l i .. .

T, i-

?'

l' D! g ii INSPECTION PROCEDURE 41400 0 j

t.

s.

! NON-LICENSED STAFF TRAINING R pA0GR44 APPLICABILITY: 2515(BA$1C) I I

l .

41400-01 INSPECTION ClJECTIVE [.

^

01.01 Evaluate the effectiveness of the trainini programs for the j non-licensed staff specified in section 02.0d .  ;:

t:

41400-02  !

)

INSPECTIONREQUIREMENTS ,

1 02.01 Review plant operating history and interview plant personnel i.

to select several abnormal events or unusual occurrences that !L j might have been caused -by-deficient training. Additionally, i i

select at least one majcr maintenance or operations activity -

recently completed. For the events / activities selected H

i t.

I a. Evaluate whether the classroom training received before' )

the event / occurrence was sufficient to have either pre. j i vented the occurrence or mitigated its effects by recog. t l nition anG proper operator action. L 4 'c i .

b. Evaluate whether the on-the-job training received before "

the event / occurrence was sufficient to have either pre- a

{

vented the occurrence or mitigated its effects by recog- [

nition and pr9per operator action.

i 1

l . c. Evaluate the qualifications of the personnel involved.  :

1 j d. Detemine whether the responsibility was clearly assigned i

for administering and evaluating the related trainsng. ll ,

e. Detemine whether any lessons learned from the event /acti- .

vity were factored into the training program., l; ,

I 02.02 gy direct questioning of at least one new, one expefienced, and

one temporary non licensed egloyee determine whether the 'i

! general knowledte of these individuals in administrative con-i trols, radiolog' cal health and safety. industrial safety, con- 'l trolled access and security, emergency plan. and quality assur-l i ance is sufficient for their assigned tasks. ,

! i

!ssue Date: i!

i!

. . , . m. u.3, v e p . .rp - . .

c ,y q , g y . y , ,, p,.-.- . - . ,,- - ,

.cr,.y .,. m .,.,7- ,7 . ., 37

' ~ ~

. . ~

...L . . . a . . .-. . u _ . . . . . . : . . . - . .:. . . : . .-. 2 :a . w. ...z..,

t  ; t .. t.

l . t.

,.' iI 41400-02.03* - TRAINING OF NON-LICEN$[D $TApp l'

l< l.

l 02.03 Determine through personnel interviews whether maintenance and  !

non-licensed operations personnel have received training for i;

specific tasks assigned, such. as complicated surveillance i!

! tests, major equipment repair, and operatine, abneraal, and b p

emergency precedures. ,

02.04 Ensure that the qualification requirements specified by the b licensee meet regulatory requirements and licensee tennitments. Q

. Verify by the review of training records that the qualifications p l are adequate for:

! a, principal staff members .

n

b. Maintenance technicians p i:

c. Health physics and chemistry technicians j

d. Non-licensed operators i . e. Technical staff members l f. Quality control inspectors ,'

1 If possible, select training ncords for nview of personnel ,

l

• involved with the events / activities identified in section 02.01 j and personnel selected for interview in section 02.03. .:

H i

. 02.05 Determine if replacement personnel for job positions listed in -

i.

l

~

section02.04arescheduledto(orhave) receive (d)alltraining { required, j:

;                            02.06                Review the progress of the utility with respect to accreditation                                                             I!

l by the Institute of Nuclear Power Operations (INPO). 'l<; l!

41400 03 INSPECTION GUIDANCE i:

! 03.01 General Guidance [

a. In a policy statement issued on March 14,1985, the Cosnis- ,

sion recognized that the industry, through initiatives of *: i ! the Nuclear Utility Management and Human Resources Coenit- - tee (NMRC) and INPO, has made progress in developing .' programs to improve nuclear utility training and personnel  : qualification. The Commission realizes the,,importance of these initiatives and wishes to encourage further self-1 i improvement. l The following excerpts are from this policy statement. - i

(1) . . . t he Cosmission w.111 exercise some discretion in i

selecting appropriate enforcement action for viola- ! tions involving training in light of the NMRC/INPD - 1 . j 1  : j 1ssue Date: *2* ' ]; v _ . - . e w . , . , , , , . . . . . -. . , , . , , . . , . . . , . . , . . . . , . . , . . . , , , . , , , .. . , .

         -~
                                    ~. ..n :: s . :..-. .u . . .        . :: . ..:. . . . :         :   n.z         .z......
        ,i y

i 1 a

                                                                                                ,                               p; Tat 1Wiks 0F NON-LICEN$t0 STAFF                                               d1400 03.01a(1) 4 fL initiative.           Licensees who are making reasonable                        i:

efforts in developing and implementing the INP0/WMRC M ! programs ... will generally not be cited for viole- h tions related to these programs, provided the viola-

                                                                                                                                 !'U i

tions, whether or not identified by NRC. are appro. priately corrected in a timely manner. Whrever, vio-i lations which are not corrected in a timely manner. L-I violations of any applicable reporting requirement, p,

and any willful violation may be subject to enforce- b

 !                                             ment.       Any enforcement action taken during this grace period will be taken only with Connission concurrence.                            o?

t Nothing in this policy ' Statement shall limit the ih

author < ty of the NRC to conduct inspections as deemed i necessary and to take appropriate enforcement action h when regulatory requirements are not met. F (2) The NRC recognizes that the INP0-managed accreditation  !

            .                                  of utility training includes the following training                               {i programs:                                                                         ii
 !

• non-licensed operator  ;;

                                               '        control room operator                                                     .-

i

                                    *

• senior control room operator / shift supervisor I shift technical advisor l:,.

l i instrument and control technician p' i

• electrical maintenance personnel l
• mechanical maintenance personnel .

i 4

l * - radiological protection technician - r:. i

• chemistry technicians
• en-site technical staff and managers

[ ! All utilities have comitted to achieving accredita- ,

 ;                                              tion of each of the above training programs. It is                                ;.

understood that each Itcenset will exert best efforts . i .

                    -                           to have all such programs ready for accreditation                                 ::

(i.e., final self-evaluation report submitted to INPO) i!

 !                                              by the end of 1986.         It is also understood that app 11-                    ,

l cents for operating Itcenses will exert best efforts l to have all such programs ready for accreditation l within two years after issuance of a full-power opera-ting license. l , e I For operating reactors, accreditation of the above training programs constitutes a method acceptable to  ;, j the NRC for implementing perfomance based training. ,- 4 The NRC will continue to review and approve all app 1'- I cant training programs in accordance with applicable I regulations, regulatory guidance, and the Standard ,l { Review plan and to conduct inspections necessary to determine that current regulations and license train-j ing comitments are being met. i ' t j Issue Date-L i

~- ,...,.w .--,, m .,, m ,.v. . - -. ...- . ~ . - . -

                                                                                          -.<.,--,.n.~,m.,r. g,n...e w g ,.. ~ 1.

                 . . . _     .. _ _         a_ u ...           ._._ _ _ _ . _.._. _ ..._ .. _ _ ____ _ _ ,                  .m           . ..... .

a .. . ( j . p I I 41400 03.01akt) TRAINING OF WOW LICENSED Stur ,l . L It should be noted that training p grams for which regulations are currently in place e.g. . fire bri-gade, emerge response, security) or which are not subject to accreditation (e.g.. Quality assur-  ;; ance) are not affected by this Policy, Statement, q

b. The regulatory bases for en acceptable training program are ,I distributed over a wide variety of documents to wh' ch each licensee is connitted to some extent. The more prominent eq of these documents are* i (1) Technical specifications. Section 5 l 1

(2) AN5! N18.1. ' selection and Training of Nuclear Power E Plant Personnel" .i

                                                                                                                                                   <l (3) ANS 3.1 ' Selection. Qualification and Training of                                                      l' Personne$ for Nuclear Power Plants' (4) 10 CFR 50. Appendix 8. Criterfon !!

(5) 10 CFR 19.12, Instructions to Workers .; (6) FSAR Chapter 13 Conduct of Operations ,

        ._ __ 03.02            Specific Guidance
                                                                                                                                                    'i

s. Inspection Requirement 02.01. The determination of whether '-

an abnorr.a1 event or unusual occurrence was caused b,'< deficient training is not always straight-forward. Fcr i 1 exa wle. a component failure may have been caused by , improper operation or incorrect maintenance. A plant operations problem. 41though attributed to a weak proce- l

 ;                                      dure, may have occurred due to a combination of training

. and procedural deficiencies. In selecting the activities l for this inspection reautrement, the inspector must apply

a liberal amount of judgment and not accept the specified root causes of failures at face value.

i b. Inspection Requirement 02.02. The intent of this reautre-t ment is to provide the inspector with evidence of the effectiveness of the general egloyee training program, i

                                                                                                                                                    ;I l                 -

c. Inspection Requirement 02.06. If the licensee has not received accreditation by 1NPO, review his comitments to achieve accreditation and progress in meeting schedule i milestones. This requirement should be accogl< shed pri-marily by a review of the schedule established-by the li- i

censee to satisfy INPO accreditation requirements. Mile- :I stones from this schedule should be checked against actus) !j j

licensee progress by review of procedures and other train-1 I i i Issue Date: - . - . . __- ___ __ ~::::: r _r  : 7~m::- :~ : ~ ~ w=-

e

                     .-                 .        .   . .     . . ..    . .-. -                      .:......-...--..---...--,.......a.:---.
              .                                                                                                                                        .v TRAINING OF NON t1CEN$tD STAFF                                                                         41400-03.0fc ing program documentation.                             This review of accreditation documents is not intended to supplant the detailed review                                                  ;

required by INPO to grant accreditation but is rather in. ,, tended to provide assurance that the licenste is making , '. progress toward accreditation. . ,. END f l F i 1 I k k 0

                                                                                                                                                     . 3 I

( 6 l 9 h l I e '. l tI t! ii

                                                                                                                         *                               !j e                         i- j e1
     .                                                                                                                                                    '1
                                                                                                                                                          ' i e

e i I i

                                                                            -   5-                                Issue Date:

4< I' w.- e . .u -ee+++e fw% += - *=, , ,,,.,.,c ,,,,g , , , , , . , , s , ., , , ,, ,, , , , ,}}