ML20214G494

Assessment Rept of Selected TMI-1 Training Programs

ML20214G494
Person / Time
Site:	Three Mile Island
Issue date:	02/13/1987
From:	DATA DESIGN LABORATORIES
To:
Shared Package
ML20214G464	List: ML20214G461 ML20214G494
References
NUDOCS 8705270084
Download: ML20214G494 (132)

Text

ASSESSMENT REPORT OF SELECTED TMI-1 TRAINING PROGRAMS t 55! (!!!  !!!!

iij j initi$ i!!s)  :::ijjjj g  :,:i w:. . .  ;; gig ( .

e::::

's *

~

liisiii .

in 3 -

r oo

.g.g Wiijj.  :::::::::

% ~ -_ : =- b u.....

, js mx fa p,wd

. -M;!j as 94%.p a

AD 5 P

THREE MILE ISLAND NUCLEAR GENERATING STATION-ASSESSMENT OF DESIGNATED TRAINING PROGRAMS 4

4 DDL OMNI ENGINEERING CORP.

February 13, 1987

- . . - - n----..- ., - _ . , ._ _ ,-. . -n,.,,,,, ,._,.,,,v,,,,.,,g.,, . ...e,. , , , , , , , - ,,,,,,,,7-- ,,_- , . -,,m.,,.,. ,, - - - . ~ .,,,, ,- - - - - -,,,

4 TABLE OF. CONTENTS Section Page

-1 INTRODUCTION ... . . . . . . . . . ................ 1-1

. 1-1 . Background.-. . . . . . . . . . . . . . . . . . . . . . . . . . 1-1

. 1-2 Assessment Preparation. . . . . . . . . . . . . . . . . . . . . 1-1 1-3 Assessment Team . . . . .................. . 1-2.

1-4 Assessment Approach and Methods . . . . . . . . . . . . . . . . 1-2 1-5 -- Report Content. . , ........................ 1-3 2 -REPORT

SUMMARY

. . . . . . . . . . . . . . . .-. . . . . . . . . .. 2-1 2-1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . 2-1 2 INPO Accreditation Recommodations . . . . . . . . . . . . . . . 2-1 2-2.1 INPO Recommendations TMI-1-R-1 . . . . . . . . . . . . . . . 2-2 4

2-2.1.1 Discussion and Conclusions. . . . . . . . . . . . . . . . 2-2

' 2-2.2 .INPO Recommendation TMI-1-R-2. . . . . . . . . . . . . . . . 2-2

2-2.2.1 Discussion and Conclusions. . . . . . . . . . . . . . . . 2-2 i 2-2.3 INPO Recommendation THI-1-R-3. . . . . . . . . . . . . . . . 2-3 2-2.3.1 Discussion and Conclusions for SRO Position . . . . . . . 2-3 4 2-2.4 INPO Recommendation THI-1-R-4. . . . . . . . . . . . . . . . 2-4 4

2-2.4.1 Discussion and Conclusions. . . . . . . . . . . . . . . . 2-4 2-2.5 INP0 Recommendation THI-1-R-5. . . . . . . . . . . . . . . . 2-5 2-2.3.1 ~ Discussion and Conclusions. . . . . . . . . . . . . . . . 2-6 2-3 ASLB Partial Initial Decision Report, May 3,1985 . . . . . . . 2-6 2-3.1 ASLB Order . . . . .....................

2-6

. 2-3.1.1 . Discussion and Conclusion . . . . . . . . . . . . . . . . 2-6 2-4 Overall Assessment. ...................... 2-7 ,

3 ASSESSMENT REPORT RECOMMENDATIONS. . . . . . . . . . . . . . . . . 3-1 3-1 Section Description . . . . . . . . . . . . . . . . . . . . . . 3-1 3-2 Overall Recommendations . . . . . . .............. 3-1 3-2.1 Development and Qualification of Staff for Training Duties . 3-1

3-2.2 Support of Training with Facilities. Equipment, and Material. . . . . . . . . . . . . . . . . . . . . . . . 3-1

3-2.3 Conduct of Job Anaylsis and Identification of Tasks for Training. . . . . . . . . . . . . . . . . ....... 3-2 3-2.4 Establishment of Training Program Content. ........ . 3-2 3-2.5 Development of Learning Objectives as the Basis for Training. . . . . . . . . . . . . . . . . . . . . . . . . . 3-3 3-2.6 Organization of Instruction Using Lesson Plan and Other Training Guides . . . . . . . . . . . . . . . . . . . . . . 3-3 3-2.7 Conduct of Classroom Instruction and Tra' nee Evaluation. . . 3-3 Conduct of In-Plant Training and Trainee Evaluacion. . .. 3-4 l 3-2.8 .

j 3-2.9 Conduct of Simulator Training and Trainee Evaluation . . . . 3-4 t- 3-2.10 Program and Course Evaluation. . . . . . . . . . . . . . . . 3-4 4 ORGANIZATION AND MANAGEMENT OF THE TRAINING CENTER . ....... 4-1 4-1 Program Description . . . . . . . . . . . . . . . . . . . . . . 4-1 4-1.1 Corporate Training Policy and Plans. . ........... 4-1 4-1.2 Corporate Training Plan. . . . . . . . . . . . . . . . . . . 4-1 4-1.3 Mission and Goals. . . . . . . . . . . . . . . . . . . . . . 4-1 j 4-1.4 Training and Education Department Organization and Functions . . . . . . . . . . . . . . . . . . . . . . . 4-2 4-1.5 THI Training Department Organization . . .......... 4-2 s 4-1.6 Training Advisory Council. . . . . . . . . . . . . . . . . . 4-2

, 4-1.7 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . 4-3 i

i

_ - . _ . . _ , _ _ . _ _ _ . _ _ _ _ . _ _ _ . _ _ . ~ . _ . . _ _ _ . _ _ . _ _ _ _ _.__ _ _ .

TABLE OF CONTENTS (Continued)

Page Section 4-2 INFO Objective. . . . . . . . ...........-..... . 4-3 4-2.1 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . .- 4-3 4-2.2 Conclusions. . . . . . . . . . . . . . . . . . . . . . . ... 4-4 4-2.3 Recommendations. . . . . . ................ . 4-4 5 LICENSED OPERATOR TRAINING PROGRAM (RO). . . . . . . . . . . . . . 5-1 Program Description . . . . . . . . . . . . . . . . . . . . . . ~ 5-1 5-1 Replacement Operator Training Program (RO) Description .,. . 5-1

5-1.1 '5-1 5-1.2 Licensed Operator Requalification Program (RO) Description .

5-2 Organization and Management of the Training System. . . . . . . .

5-2 5-2.1 INPO Objective . . . . . . . . .

. . . . . .- . .- . . . . . . 5-2

' 5-2.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . .- 5-2 5-2.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . .-. 5-2 5-2.3.1 Program Evaluation. . . . . . . . . . . . . . . . . . . . 5-3 5-2.3.2 Responsibilities. . . . . . . . . . . . . . . . . . . . . 5-3 5-2.3.3 Training System ... . . ...... . . . . . . . . . . . 5-4' 5-2.3.4 Instructional Procedures. . . . . . . . . . . . . . . . . 5-4 5-2.3.5 Required Training and Exemptions. . . . . . . . . . . . . 5-5 5-2.3.6 Training Records. . . . . . . . . . . . . . . . . . . . . 5-6

- 5-2.3.7 Contract Training . . . . . . . . . . . . . . . . . . . . 5-8 5-2.3.8 Instructor Workload . . . . . . . . . . . . . . . . . . . 5-8 2.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 5-8 2.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 5-8 5-3 Development and Qualification of Staff for Training Duties. . . 5-8 5-3.1 INPO Objective . . . . . . . . . . . . . . . . . . . . . . . 5-9 5-3.2 INPO Criteria. . . . . . . .. . . . . . . . . . . . . . . . . 5-9 5-3.3 Criteria Evaluation. . . . . . . .-. . . . . . . . . . . . . .

5-9 5-3.3.1 Instructor Qualification. . . . . . . . . . . . . . . . .. 5-9 5-3.3.2 Maintaining Qualification . . . . . . . . . . . . . . . . 5-10 5-3.3.3 Instructor Development Certification. . . . . . . . . . . 5-12 i

5-3.3.4 Instructor Evaluation . ...... . . . . . . . . . . . 5-13 5-3.3.5 Continuing Instructor Development . . . . . . . . . . . . 5-13 5-3.4 conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 5-13 5-3.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 5-13 5-4 Support of Training with Facilities, Equipment,

' and Materials. . . . . . . . . . . . . . . . . . . . . . . . . 5-14 5-4.1 INPO Objective . . . . . . . . . . . . . . . . . . . . . . . 5-14 5-4.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 5-14 l

5-4.3 Criteria. Evaluation. . . . . . . . . . . . . . . . . . . . . 5-14 5-4.3.1 Facilities and Instructional Support. . . . . . . . . . . 5-14 5-4.3.2 Conclusions . . . . . . . . . . . . . . . . . . . . . . . 5-14

!- 5-4.3.3 Recommendations . . . . . . . . . . . . . . . . . . . . . 5-14 5-5 Conduct of Job Analysis and Identification of Tasks t for Training . . . . . . . . . . . . . . . . . . . . . . . . . 5-15 5-5.1 INPO Objective . . . . . . . . ..... . . . . . . . . . . 5-15 5-5.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 5-15 5-5.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . 5-15 i

5-5.3.1 Job Analysis and Selecting Tasks for Training . . . . . . 5-15 5-5.4 Conclusions. . . . . . . . .. .... . . . . . . . . . . . 5-16 5-5.5 Recommendations. . . . . . . ..... . . . . . . . . . . . 5-16 11 1

- ~ , - y - - . , , . - , , -, .,.,---y

TABLE OF CONTENTS (Continued)

Section Page 5-6 Establishment of Training Program Content .. . . . . .. . . . 5-16 5-6.1 INPO objective . . . ..... . . .... . ... . . . . . 5-16 5-6.2 INPO Criteria. . . . . .... . .. . .. . . . . ... . . 5-16 5-6.3 Criteria Evaluation. . . . . . . .. . .. . .. . .. . . . 5-16 5-6.3.1 INPO Training Guidelines. . . ... .. . . . . ... . . 5-16 5-6.3.2 Task Skills and Knowledge . . . . . . . . . . . . . . . . 5-18 5-6.3.3 Training content Determination. . . . . . . . . . . . . . 5-18 5-6.3.4 Plant Documentation . . . . . . . . . . . . . . . . . . . 5-19 5-6.3.5 Content Hodification. . . . . . . . . . . . . . . . . . . 5-19 5-6.3.6 Evaluation Results. . . . . . . . . . . . . . . . . . . . 5-19 5-6.4 Conclusions. . . . . . . ... . .. .. . . ... . . . . . 5-19 5-6.5 Recommendations. . . .. . . . . .... . . ... . . . . . 5-19 5-7 Development of Learning Objectives as the Basis for Training. . 5-20 5-7.1 INPO Objective . . . . . . . ... . . . . . . .. . .. . .. 5-20 5-7.2 INPO Criteria. . . . . . . .. .... .. . . . . ... . . 5-20 5-7.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 5-20 5-7.3.1 Entry-Level Skill and Knowledge . . . . . . . . . . . . . 5-20 5-7.3.2 Learning Objectives . . . . . . . . . . . . . . . . . . . 5-21 5-7.3.3 Learning Objective Format . . . . . . . . . . . .. . . . 5-21 5-7.3.4 Learning obj ective Grouping . . . . . . . . . . . . . . . 5-22 5-7.3.5 Learning Objective Sequencing . . . . . . . . . . . . . . 5-22 5-7.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 5-22 5-7.5 Recommendations. . . . . . . . ... ... . . ..... . . 5-22 5-8 Organization of Instruction Using Lesson Plans and Other Training Guides. . . . . . . . . . . . . . . . . . . . . . . . 5-22 5-8.1 INPO Objective . . . . . .. . . . . .. . . . .. . . . . . 5-22 5-8.2 INPO Criteria. . . . . . . . . . .. .. . . .. . . . . . . 5-22 5-8.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 5-23 5-8.3.1 Classroom Lesson Plans. . . . . . . . . . . . . . . . . 5-23 5-8.3.2 OJT and Simulator Lesson Plans. . . . . . . . . . . . . . 5-26 5-8.3.3 Performance-Based Lesson Plans. . . . . . . . . . . . . . 5-28 5-8.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 5-28 5-8.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 5-28 5-9 Conduct of Classroom and Individualized Instruction and Trainee Evaluation . . . . . . . . . . . . . . . . . . . . . . 5-28 5-9.1 INPO Objective . . . .. . . . . . . . . . . . . . .. . . . 5-28 5-9.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 5-28 5-9.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 5-29 5-9.3.1 Conduct of Training . . . . . .. . . . . . . . . . . . . 5-29 5-9.3.2 Trainee Participation . . . . . .. . . . . . . . . . . . 5-31 5-9.3.3 Instructor Preparation. . . . . . . . . . . . . . . . . . 5-31 5-9.3.4 Instructional Techniques. . . . . . . . . . . . . . . . . 5-31 5-9.3.5 Individualized Instruction. . . . . . . . . . . . . . . . 5-32 5-9.3.6 Learning Objective Mastery. . . . . . . . . . . . . . . . 5-32 5-9.3.7 Examination Grading . . . . . . . . . . . . . . . . . . . 5-33 5-9.3.8 Oral Examinations . . . . . . .. . . . . . . . . . . . . 5-34 5-9.4 Conclusions. . . . .. . . .. . .. . . . . . . . . . . . . 5-35 5-9.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 5-35 5-10 Conduct of In-Plant Training and Trainee Evaluation . . . . .. 5-36 5-10.1 INP0 Objective . . . . . . . . . . . . . . . .. . . . . . . 5-36 iii

TABLE OF CONTENTS (Continued)

Section. Page.

5-10.2 INPO Criteria. . ...... ................ 5-36 5-10.3 Criteria Evaluation. . . . . . . . . . . .-. . . . . . . . . 5-36 5-10.3.1 Training Delivery . . . . . . . . . . . . .-. . . . . . . 5-36 i 5-10.3.2 Examiner Qualification. . . . . . . . . . . . . . . . . . 5-36 5-10.3.3 Task Performance. ... . . . . . . . . . . . . . . . . . . 5-37 5-10.3.4 Performance Evaluation Criteria . . . . . . . .

. . . . . . 5-37 5-10.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 5-37 5-10.'4.1 Recommendations . . . . . . . . . . . . . . . . . . . . . 5-38 5-11 Conduct of Simulator Training and Trainee Evaluation. . . . . . 5-38 5-11.1 INFO Objective . . . . . . . . . . . . . . . . . . . ... . . 5-38 5-11.2 INPO Criteria. . ...................... 5-38 5-11.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 5-38 5-11.3.1 Simulator Use . . . . . . . . . . . . . . . . . . . . . . 5-38 5-11.3.2 Training Material . . . . . . . . . . . . . . . . . . . . 5-39 5-11.3.3 Training Evaluation . . . . . . . . . . . . . . . . . . . 5-39 5-11.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 5-41 5-11.5 Reconsendations. . . . . . . . . . . . . . . . . . . . . . . 5-41 5-12 Systematic Evaluation of Training Effectiventes . . . . . . . . 5-41' 5-12.1 INP0 objective . . . . . . . ................ 5-41 5-12.2 INP0 Criteria. . . . . . . . . . . . . . . . . . . . . . . . 5-41

5-12.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 5-42 5-12.3.1 Program Evaluations . . . . . . . . . . . . . . . . . . . 5-42 L

5-12.3.2 Training Delivery Evaluation. . . . . . . . . . . . . . . 5-43 5-12.3.3 Trainee Feedback. . . . . . . . . . . . . . . . . . . . . 5-43

! 5-12.3.4 On-the-Job Teodback . . . . . . . . . . . . . . . . . . . 5-44 5-12.3.5 Change Action Feedback. . . . . . . . . . . . . . . . . . 5-44 5-12.3.6- Training Changes. . . . . . . . . . . . . . . . . . . . . 5-44 5-12.4 ASLB' Reactor Operator Training Feedback. . . . . . . . . . .- 5-44 5-12.5 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 5-45 5-12.6- Recommendations. . . . . . . . . . . . . . . . . . . . . . . 5-45 6 SENIOR REACTOR OPERATOR TRAINING PROGRAM . . . . . . . . . . . . . 6-1

6-1 Senior Reactor Operator Training Program Description. . . . . . 6-1 6-1,1 Replacement SRO Training Prograa . . . . . . . . . . . . . . 6-1

, 6-1.1.1 Classroom Training. . . . . . . . . . . . . . . . . . . . 6-1 1 6-1.1.2 On-the-Job Training . . . . . . . . . . . . . . . . . . . 6-2 l 6-1.1.3 Simuistor Training. . . . . . . . . . . ......... 6-2

6-1.1.4 Evaluation. . ...... ................ 6-3 6-1.2 Direct SRO Training. . . . . . . . . . . . . . . . . . . . . 6-4

! 6-1.2.1 Classroom Training. . . . . . . . . . . . . . . . . . . . 6-4 l 6-1.2.2 Simulator Training. .............. ..... 6-5 1 4

6-1.2.3 Reactivity Manipulations. . . . . . . . . . . . . . . . . 6-5  !

j 6-1.2.4 Evaluation. . . . . . . . . . . . . . . . . . . . . . . . 6-5

6-1.3 Licensed Operator Requalification Program. ......... 6-6 6-1.3.1 Pre-Planned Lecture Series. . . . . . . . . . . . . . . . 6-6 6-1.3.2 Skills Training and Evaluation. . . . . . . . . . . . . . 6-7 6-1.3.3 Operational Review Program. . . . . . . . . . . . . . . . 6-8 6-1.3.4 Annual Examination and Evaluation . . . . . . . . . . . . 6-8 6-1.3.5 Evaluaticn. . . . . . . . . . . . . . . . . . . . . . . . 6-9 6-2 Organization and Management of the Training System. . . . . . . 6-9 6-2.1 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 6-9 iv

TABLE OF CONTENTS (Continued)

Section Page

6-2.2 Training Records . . . . . . . . . . . . . . . . . . . . . . 6-9 6-2.2.1 Training Content Records. . . . . . . . . . . . . . . . . 6-9 6-2.2.2 Training Attendance / Completion Records. . . . . . . . . . 6-10 l 6-2.2.3 Training Examinatien Records. . . . . . . . . . . . . . . 6-10 1 6-2.2.4 Maintenance of Training Record Files. . . . . . . . . .'. 6-10 1 6-2.2.5 Training Department Records . . . . . . . . . . ._. . . . 6-11 6-2.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 6-11 6-2.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 6-11 6-2.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 6-11 6-3 Development and Qualification of Staff for Training Duties. . . 6-11 6-3.1 INPO Objective . . . . . . . . . . . . . . . . . . . . - . . . 6-11 6-3.2 INPO Criteria. . . . . .. . . . . . . . . . . . . . . . . . 6-12 6-3.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 6-12 6-3.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 6-14 6-3.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 6-14 6-4 Conduct of Job Anaylsis and Identification of Tasks for Training . . . . . . . . . . . . . . . . . . . . . . . . . 6-14 6-4.1 INPO Objective . . . . .. . . . . . . . . . . . . . . . . . 6-14 6-4.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 6-14 6-4.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 6-15 6-4.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 6-16 6-4.5 Recotmendations. . . . . . . . . . . . . . . . . . . . . . . 6-17 6-5 Establishment of SRO Training Program Content . . . . . . . . . 6-17 6-5.1 INPO Objective . . . . . . . . . . . . . . . . . . . . . . . 6-17 6-5.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 6-17 6-5.3 References . . . . . . . . . . . . . . . . . . . . . . . . . 6-18 6-5.4 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 6-16 6-5.5 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 6-19 6-5.6 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 6-20 6-6 Development of SRO Learning Objectives as the Basis for Training . . . . . . . . . . . . . . . . . . . . . . . . . 6-20 6-6.1 INPO Objective . . . . . . . . . . . . . . . . . . . . . . . 6-20 6-6.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 6-20 6-6.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 6-20 6-6.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 6-22 6-6.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 6-22 6-7 Organization of Instruction Using Lesson Plans and Other Training Guides. . . . . . . . . . . . . . . . . . . . . . . . 6-22 6-7.1 INPO Objective . . . .. . . . . . . . . . . . . . . . . . . 6-22 6-7.2 INPO Criteria. . . . . .. . . . . . . . . . . . . . . . . . 6-22 6-7.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 6-23 6-7.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 6-23 6-7.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 6-24 6-8 Conduct of SRO In-Plant Training and Trainee Evaustion. . . . . 6-24 6-8.1 INPO Objective . . . . .. . . . . . . . . . . . . . . . . . 6-24 6-8.2 INP0 Criteria. . . . . . . . . . . . . . . . . . . . . . . . 6-24 6-8.3 References . . . . . . . . . . . . . . . . . . . . . . . . . 6-24 6-8.4 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 6-24 6-8.5 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 6-26 6-8.6 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 6-26 Y

TABLE OF CONTENTS (Continued)

Page Section Conduct of SR0 Simulator Training .-. . . . . . . . . . . . . . 6-26 6-9 6-26 6-9.1 INPO Objective . . ................ . . . . . ' 6-26 6-9.2 INP0 Criteria. . . . . . . . . . . . . . . . . . . . . . . .-

6-26 6-9,3 Criteria Evaluation. . . . .-. . . . . . . . . . . .-. . . .

6-28 6-9.3.1 Simulator Exercise Evaluation . . . . . . . . . . . . . . 6-28 6-9.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . .

6-28 6-9.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . .

6-28 6-10 Systematic Evaluation of Training Effectiveness . . . . . . . .

6-28 6-10.1 INPO Objective . . ............. . . . . . . _ . .

6-29 6-10.2- .INPO Criteria. .'. . . . . . . . . . . . . . . . . . . . . .

. . 6-29 6-10.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . .

6-31 6-10.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . .

6-32 6-10.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . .

7-1 7 AUXILIARY OPERATOR (NON-LICENSED OPERATOR) TRAINING PROGRAM. . . .

Auxiliary Operator Initial Training Program Description . . . . 7-1 7-1 7-1 7-1.1 Classroom Training . . . . . . . . . . . . . . . . . . . . .

7-2 7-1.1.1 Written Examinations and Quisses. . . . . . . . . . . . . 7-3 7-1.2 On-the-Job Training. . . . . . . . . . . . . . . . . . . . .

. 7-4 7-1.2.1 Task Performance. . . . . . . . . . . . . . . . . . . .

7-5 7-1.2.2 Comprehensive Written Examination . . . . . . . . . . . . 7-5 7-1.2.3 Comprehensive Oral Examination. . . . . . . . . . . . . . 7-5 7-1.3 Program Completion . . . . . . . . . . . . . . . . . . . .

7-5 7 AUXILIARY OPERATOR RETRAINING PROGRAM. . . . . . . . . . . . . . . 7-6 7-2.1 Program Content . .. ............... 7-6

, 7-2.1.1 General Employee Retraining. . . . . . . . . . . . . . . . .

7-6 7-2.1.2 Fundamentals Retraining. . . . . . . . . . . . . . . . . . . '

7-6 7-2.1.3- Program Maintenance. . . . . . . . . . . , . . . . . . . . .

7-7 7-3 Organization and Management of the Training iystem. . . . . . - . .

7-7 7-3.1 INPO Objective . . . . . . . . . . . . . . . . . .

7-7 7-3.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . .

7-7 7-3.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 7-7 7-3.3.1 Responsibilities. . . . . . . . . . . . . . . . . . . . .

. . 7-9 7-3.3.2 Training System . . . . . . . . . . . . . . . . . . .

7-9 7-3.3.3 Training Procedures . . . . . . . . . . . . . . . . . . .

. 7-10 7-3.3.4 Required Training . . . . . . . . . . . . . . . . . . .

7-10 7-3.3.5 Training Records. . . . . . . . . . . . . . . . . . . . .

7-11 7-3.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . .

7-11

7-3.5 Reconnendations. . . . . . . . . . . . . . . . . . . . . . .

7-11 7-4 Development and Qualification of Staff for Training Duties. . . 7-11 INPO Objective . . ................ . . . . .

7-4.1 7-11 7-4.2 INP0 Criteria. . . ..... . . . . . . . . . . . . . . . .

7-12

! 7-4.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . .

7-12 7-4.3.1 Instructor Qualification. . . . . . . . . . . . . . . . .

7-12 7-4.3.2 Initial and Continuing Instructor Development . . . . . .

. . . . . . . . 7-14 7-4.3.3 Instructor Evaluation . . . . . . . . . .

7-14 7-4.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . .

7-14 7-4.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . .

7-5 Support of Training with Pacilities. Equipment, 7-14 and Materials. . . . . . . . . . . . . . . . . . . . . . . . . 7-14 7-5.1 INPO Objectiva . . . . . . . . . . . . . . . . . . . . . . .

i vi

}

i e..-._-,w --,,n , , . . . --

.,~,vm--w, na-,,,-,--,-,,---n,_ _ - ,_ n,m , n _---n ,,.~ ,_ _ -- _ ,,_,n-pn,w-., ,_,.-n--,,__., ,.

TABLE OF CONTENTS (Continued)

Section Page 7-5.2 INPO Criteria. . . .. . ... ...... . ........ 7-14 7-5.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . '7-15 7-5.3.1 Physical Facilities and Equipment ............ 7-15 7-5.3.2 Reference Material. . . . . . . . . . . . . . . . . . . . 7-15 7-5.4. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 7-15 7-5.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 7-15 7-6 Conduct of Jeb Analysis and Identification of Tasks for Training . . . . . . . . . . . . . . . . . . . . . . . . . 7-15 7-6.1 INPO Objective . . . . . . . . . . . . . . . . . . . . . . . 7-15 7-6.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 7-16 7-6.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 7-16 7-6.3.1 Job Analysis and Selecting Tasks for Training . . . . . . 7-16 7-6.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 7-16 7-6.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 7-17 7-7 Establishment of Training Program Content'. .......... 7-17 7-7.1 INP0 Objective . . . . . . . . . . . . . . . . . . . . . . . 7-17 7-7.2 INPO Criteria. . .. . . ... ...... . ........ 7-17 7-7.3 Criteria Evaluation. . . .. ....... ....... .. 7-17 7-7.3.1 INPO Qualification Guideline. . . . . . . . . . . . . . . 7-17 7-7.3.2 Training Program Content. ... . .......... .. 7-18 7-7.3.3 Program Evaluation. . . . . . . . . . . . . . . . . . . . 7-18 7-7.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 7-18 7-7.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 7-18 7-8 Development of Learning Objectives as the Basis for Training . . . . . . . . . . . . . . . . . . . . . . . . . 7-19 7-8.1 INPO Objective . . . . . . . . . . . . . . . . . . . . . . . 7-19.

7-8.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 7-19 7-8.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 7-19 7-8.3.1 Entry-Level Skill and Knowledge . . .. ... ...; .. 7-19 1

7-8.3.2 Learning Objectives . . . . . . . ... . ...... .. 7-19 7-8.3.3 OJT Performance Requirements. . . . . . . . . . . . . . . 7-20 7-8.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 7-20 7-8,5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 7-20 7-9 Organization of Instruction Using Lesson Plans and Other Training Guides. . .. . . . .... . ... .. .... ... 7-21 7-9.1 INPO Objective . . . . . ... .. . .. . ... . .. . .. 7-21 7-9.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 7-21 7-9.3 Criteria Evaluation. . . . . . . . . ... ... .... ..

7-21 7-21 7-9.3.1 Classroom Lesson Plans. . . . . . . . . . . . . . . . . .

7-9.3.2 OJT Qualification Guides. . . . . . . . . . . . . . . . . 7-22 7-9.4 conclusions. . . . . . . ......... .. . .... . . 7-22 7-9.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 7-22 7-10 Conduct of Classroom and Individualized Instruction and Trainee Evaluation . . . . . . . . . . . . . . . . . . . . . . 7-22 7-10.1 INPO Objective . . . . . ..... . .. . . .. . ..... 7-22 7-10.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . . . . 7-22 7-10.3 Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 7-22 7-10.3.1 Quisses and Examination . . . . . . . . . . . . . . . . . 7-23 7-10.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 7-23 7-10.5 Recommendations. . . . . . . . . . . . . . . . . . . . . . . 7-23 d

vii

TABLE OF CONTENTS (Continued)

Section Page 7-11 Conduct of In-Plant Training and Trainee Evaluation . . . . . . 7-23 7-11.1 INPO Objective . . . . . . . . . . ... .......... 7-23 7-11.2 INPO Criteria. . . . . . . . . . . . . . . . . . . . . ... 7-23 7-11.3 Criteria Evaluation. . . . ... . ...... ....... 7-24 7-11.3.1 Program Content . . . .. ............. ... 7-24 7-11.3.2 Performance Evaluation. . . . . . . . . . . . . . . . . . 7-24 7-11.3.3 INPO Recommendatien TMI-1-R-5 . . . . . . . . . . . . . . 7-24 7-11.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 7-25 7-11.5 Recommendations. . . . . . . . . . . . . ........ .. 7-25 7-12 Systematic Evaluation of Training Effectiveness . . . . . . . . 7-25 7-12.1 INPO Objective . . . . . .................. 7-25 7-12.2 INPO Criteria. . . . . . . . . . .............. 7-25 7-12.3- Criteria Evaluation. . . . . . . . . . . . . . . . . . . . . 7-26 7-12.3.1 Program Evaluation. . . . . . . . . . . . . . . . . . . . 7-26 7-12.3.2 Training Delive ry Evaluation. . . . . . . . . . . . . . . 7-26 7-12.3.3 Change Action Evaluation. . . . . . . . . . . . . . . . . 7-27 7-12.4 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . 7-27 7-12.5 Recommendations. . . . . ........ . ...... ... 7-27 e

viii 1

i 1

SECTION 1 INTRODUCTION 1-1.. BACKGROUND.

DDL .0MNI Engineering Corp (DDL) was approached by GPU Nuclear Corporation. in March 1986 to conduct an Atomic Safety and Licensing Board (ASLB) required audit of TMI-1 operator training programs. .-Basic to~ this assessment were ASLB Partial Initial

j. Hearings dated July 27, 1982 and May 3, 1985. The ASLB Partial Initial Hearing dated July 27, 1982 directed an in-depth audit of the TMI-1 qualification and re-qualification testing and training programa. The ASLB Partial Initial Hearing dated May 3, 1985, found the licensed operator training programs at THI-1 adequate to train reactor operators and senior reactor operators to operate the unit safely.

The Board, however, ordered the THI-1 licensee to implement a plan for the evalua-tion, aft.or training, of the performance of its trained reactor operators and senior reactor operators in the job setting, under normal and abnormal operations. By a letter dated April 9, 1984, the Nuclear Regulatory Commission, Office of. Nuclear Reactor Regulations approved DDL to perform ASLB imposed audits. GPU Nuclear Con-tract Number PL-042618 of July 30, 1986 prescribed the scope of this assessment and the criteria to be employed. In addition to the ASLB requirement, the designated training programs were to be evaluated using INPO criteria in The Accreditation of i

Trainina in the Nuclear Power Industry.

1-2. ASSESSMENT PREPARATION.

i Prior to the on-site assessment, the DDL team collected, reviewed, and analyzed copies of pertinent documents such as inspection reports, training program descrip-tions, training management documents, corporate training policies, instructor devel-opsent procedures, job and task analyses, lesson plans, and INPO accreditation docu-ments including self evaluation reports, the accreditation team evaluation report, and GPU's response to the accreditation team evaluation report.

This examination served to familiarise the team with the specifics of each training program and areas of past concern for detailed on-site examination. Also examined was the Atomic Safety and Licensing Board, Partial Initial Decisions, of May 3, 1985 1

3 and July 27, 1982.

1-1 i

i i

m n-e ..,-n. .,,w,,.,,. -. _ ,, ,n n n - n .~. n --- ,..,,-- ,v _ w.,.-.n. _ ,- _.-,n n . _ _ - _.,.., ---n,----,,--

l 1-3. ASSESSMENT TEAM.

The DDL assessment team - was comprised of a group of evaluators with collective expertise in nuclear power plant operations, nuclear utility training, nuclear engineering, training _ program development, instructional process, and training evaluations. The team members were:

Mr. D. S. Boyd Dr. J. E. Cantor (Ph.d Vocational Education)

Mr. J. E. Halloy Dr. H. G. Olson (Ph.d Nuclear Engineering)

Mr. C. G. Pohle Both Mr. Boyd and Mr. Malloy participated in a 1932 assessment of TMI-1 selected training programs performed by DDL. Dr. Olson is an engineering professor at l' Colorado State University and has extensive experience in nuclear plant operation, licensing, training, radiation, health physics, and auditing. Dr. Cantor is Direc-tor of Education Programs for DDL-OHNI Engineering Division. Both Mr. Malloy and Mr. Pohle for the past three years have been involved exclusive?.y in performing industry wide task analyses and designing and developing training programs and instructional materials for nuclear utilities which meet INPO Training System Devel-opment (TSD) standards and accreditation criteria. Mr. Boyd headed the DDL 1982 1

TMI-1 Training Program Assessment. He has many years of experience in the operation of Navy nuclear power plants and the management of nuclear training programs.

1-4. ASSESSMENT APPROACH AND METHODS.

4 I In conducting the assessment, Data-Design followed the objectives and criteria con-tained in The Accreditation of Training in the Nuclear Power Industry (INPO 85-002).

l l The areas evaluated for each of the individual training programs, as applicable

! veret

! - Organization and management of the training system.

- Development and qualification of staff for training duties.

- Support of training with facilities, equipment, and materials.

- Conduct of job analysis and identification of tasks for training, j - Establishment of training program content.

- Development of learning objectives as the basis for training.

1 i 1-2

g

- Organiastion of instruction using lesson plans and other training guides. ,

Conduct of ' classroom and individualized instruction and trainee evalua-tion.-

- Conduct of In-Plant Training and trainee evaluation.

- Conduct of Simulator Training and trainee evaluation.

- Conduct of Laboratory Training and trainee evaluation.

Systematic evaluation of training effectiveness.

1-5. REPORT CONTENT.

Each training progras examined is covered in its own section of the report. These sections contain a description of the training programs, the INPO objectives and criteria, a description of the evaluation performed, conclusions and ' recosamenda-tions. Section 2 of this report is a summary which discusses the principal conclu-sions. Section 3 is a consolidated summary of recommendations appearing in the

- various training program sections.- Section 4 is an examination of the corporate and training staff organisation and management for suppcrt of the training system.

l The preliminary draft of the assessment report was reviewed for factual accuracy by GPU Nuclear. Comments on factual accuracy have been incorporated in this final report. The conclusions and recoma .ndations were drawn without GPU Nuclear influ-ence and represent the objective views of DDL Omni Engineering Corp.

I 4

4 b

4 4

1-3

.l

- - -

• w w-e+ew-rerwo---o-=+v+ w-._ ._ve.-,,-,,,---w+ww

---

. . me e w . .y----,- -,+wew-www**=----

SECTION 2 REPORT

SUMMARY

' 2-1. INTRODUCTION.

The TMI-1 operator programs which are the subject of this assessment were accredited by the Institute of Nuclear Power Operations on February 28, 1985. To obtain accre-ditation, a utility must demonstrate that its training programs meet industry-wide objectives and criteria. The objectives describe the expected end results of an effective, well managed training program, whereas the criteria are principles or methods which support the objectives.

Prior to granting of accreditation, the THI-1 training programs were evaluated against the accreditation objectives and criteria by training pertonnel during the self-evaluation process, by accreditation team members while performing their on-site evaluations, and by Accreditation Board members during their review. Accredi-tation is an indication that the TMI-1 training programs meet en industry-wide

standard whose purpose is to produce well-qualified, competent personnel to operate the nations nuclear plants.

The INPO accreditation team evaluation provides a detailed and exhaustive on-site assessment of the candidate training programs. The accreditation team report high-lights areas for recommended improvements. The utility's response describes pro-

- grams and actions directed to the recommendations and forms a basis for INP0's endorsement to the Accreditation Board. Because of their importance in the accredi-tation process, the areas covered by en INPO comments and the GPU Nuclear responses received special scrutiny during this assessment. Also receiving close attention were areas of concern in the ASLB Partial Initial Hearings. This summary highlights the results of these special area reviews and provides an overall assessment of the TMI-1 training programs and organization.

2-2. INPO ACCREDITATION RECOMMENDATIONS.

The INPO team accreditation report on the TMI-1 training programs contained re:om-sendations in several areas of training. Recommendations concerning the operator programs and this assessment team's conclusions are listed as follows.

f 2-1 t-- - - - e--- -r,-

c,m,y ,w-,w, ,-.,---r, , -w g ww w y.---ar ee m w .ce + w- r,-=w-er--..e-,+.,_- w-r--- - -wem -+'ts- em--'

2-2.1. INPO RECOMMENDATION THI-1-R-1. Develop and implement sufficient procedures to ensure that all five P ases h of the GPU Nuclear Training Systems Development Model are fully and consistently implemented throughout the training programs.

2-2.1.1. Discussion and Conclusions. A Training and Education Department Procedure (6200-ADM-2682.01) is in effect which establishes and describes a GPU Nuclear train-ing system development model for use in developing in-house and vendor training pro-I grams. This TSD procedures is supplemented by ten additional training department procedures which cover all phases of the TSD process from needs analysis through program evaluation and feedback. In addition, a GPU Nuclear Program Development Manual has been issued which contains detailed guidelines in such areas as job and task analysis, training standa'rd formats, lesson plan format, and preparation of trainee texts. These T&E Department training procedures and the GPU Nuclear guide-lines are totally responsive to the INPO recommendation. They provide a systematic

,- approach to developing performance-based training programa modeled on the INPO TSD process.

i 2-2.2. INPO RECOMh2NDATION TMI-1-R-2. Require the use of approved simulator exer-cise guides that include learning objectives and performance criteria for the initial training of licensed operators on the simulator. Obtain contractor respon-i ses with planned corrective actions for weaknesses noted during the monitoring of I

simulator training.

2-2.2.1. Discussion and Conclusions. Representative replica slaulator lesson plans and drill guides were examined for Control Room Operator initial training and for

- License Operator requalification drills. Each guide describes the conditions under which the drill is to be performed and performance objectives associated therewith i based on casks in the plant specific job analyses. Lesson plans used to support f, simulator training were also observed to contain learning objectives in the THI-1 Standard Training Content Record (TCR) Format. The text of the lesson plan s:ipu-lates initialization procedures / conditions as required. The lesson plans contain l

instructor notes and expected student responses and conclude with a list of critique criteria. The drill guides list all RO, SRO and STA tasks which are covered by the j simulator exercise. Also included are questions and their answers at relevant

points in the instruction. The THI-1 replica simulator has recently completed ac-captance testing and is now in initial operation on site. Contractor services are i no longer utilized for training.

2-2 t

It was observed during the monitoring of' simulator exercises by assessment team mes-bers that the instructors routinely provide direct corrective action feedback for weaknesses noted immediately following each drill. A written summary report of exer-cises conducted is provided by the simulator staff to the Operator Training Section.

This report contains weaknesses noted and recommended corrective actions.

The simulator exercise guides and the instructor feedback and responses to simulator-exercises are modeled af ter INPO Guideline 86-026 and satisfy in full this INPO recommendation.

2-2.3. INPO RECONMENDATION TMI-1-R-3. Complete the job analysis for the Senior Reactor Operator (SRO) and Shif t Technical Advisor (STA). Verify and document that the tasks identified through job analysis of the operator and STA positions and selected for training are covered to the sufficient depth in the applicable TMI-1 operator and STA training programs. For any task selected for training but not presently covered, analyze each task to determine the job related knowledge and skill requirements and incorporate the results in the applicable training programs.

2-2.3.1. Discussion and Conclusions for SRO Position. A plant specific job analy-sie has been performed for the TMI-1 SRO position in general conformance with pro-cedures recommended by INPO. The results of the Senior Reactor Operator Job Survey are documented in the Senior Reactor Operator THI-1 Task List, 6211-AEM-2611.05, Rev. 1 dated 7/3/86. This revision has resulted from a series of analyses. The original task list, consisting of similar plant tasks extracted from the INPO list, was revised based on an examination of plant procedures (administrative, operating, surveillance, maintenance, Rad-Con) . The expanded list was again analyzed to iden-tify tasks to be trained and the appropriate training settings. The training settings considered were classroom. OJT, simulator or an appropriate combination.

The current task list contains 139 casks in a matrix which identifies the training setting as determined by subject matter experts in the Training Department. The tasks to be trained are a combination of applicable tasks from the INPO cask list and plant-specific tasks identified during the job analysis. Twenty-nine tasks are specified in the training matrix for classroom presentation, 109 for OJT and 41 for simulator training. Some tasks appear in multiple training settings.

2-3

. . . .- .. .-= . - - - - _ _ . . -, .

The process of matching tasks to existing training program content -is ongoing and approaching conclusion. Of the 179 task settings, 148 have currently identified coverage in existing training programs. The remaining tasks identified for class-room presentation are being incorporated into lesson plan objectives and content as part of a continuing program. Simulator identified tasks remaining are being incor-porated into drill guides which are being developed for the first time on the MI-1 replica simulator. No new training asterial has been identified to date as requir-ing development and hence no detailed SRO task analyses have been accomplished.

The TMI-1 response to this INPO reconsendation is ongoing in a very positive and structured manner.

2-2.4. INPO RECOMMENDATION WI-1-R-4. Review and revise as appropriate the plant systems training lesson plan content and learning objectives for Auxiliary Operator, Control Roos Operator, Operations Shift Supervisor and Shift Technical Advisor

Training Programs to reflect the knowledge and skill requirements for which trainee is being prepared. Ensure that learning objectives include measurabic standards and conditions to successful completion.

2-2.4.1. Discussion and Conclusions. The initial TMI-1 job analyses of the opera-tor positions did not identify skills and knowledges which could be converted to learning objectives. This was consistent with INPO guidance for existing training program accreditation. Since the initial job analysis, task lists have been revali-4 dated with new tasks added and old tasks removed. For those tasks selected for i training but not presently covered, an in depth task analysis is being performed.

j These task analyses are consistent with the INP0' process. They identify the sequen-

! tial elements and their associated skills and knowledges from which learning objec-tives are being developed.

OJT for operator training consists essentially of qualification sheets organi

ed in j related categories. Each qualification sheet has a list of performance objectives referenced to one or more tasks identified for OJT accomplishment. Simulator drill guides currently being developed identify the essks covered in the instructional exercise and describe the condition under which the drills are to be performed and the performance objectives associated therewith.

j 2-4 4

__..___..__-.c-_m_,_,___,_,_m _ . , _ - _ ~ _ _ _ , _ - - . _ . . . _ - . . - - , , _ . - - _ _ _

Classroom lesson plans at TMI-1, as a general rule, do not distinguish between ter-ainal and enabling learning objectives. Also, measurable standards and conditions for successful completion of the learning objective are not always stated. For knowledge type learning objectives where no performance demonstration is required, ,

the need for specific learning objective conditions and standards is arguable. To cover this criteria, the operator training program descriptions will typically require an overall standard of performance for classroom training as a score of at least 80% of all written examinations. The condition under which the trainee will demonstrate required knowledge to meet the standard is stated to be at the conclu-sion of the lesson. This general treatment of knowledge type learning objective i i

conditions and standards is considered by this assessment team to be a reasonable I

! agrosch to the academic portion of a performance based training program.

j- A method is currently underway at THI-1 to standardine lesson plan content - and

• 1 earning objective format. Training Standard (6200-ADM-2682.05) dated 04/15/85 prescribes learning objective format. T&E Department TSD Procedure (0200-ADM-l 2682.01) dated 07/01/86 provides guidance on lesson plan development and learning objective usage. Lesson Plan Procedure (6200-ADM-2682.07) dated 04/18/85 esta -

blishes standards for lesson plan and learning objective development. These proce-i dures are the basis of the program being implemented to comply with this INPO accre-ditation recommendation. Full implementation of this program is a lengthy process because of the large number of lesson plans and a TMI-1 program to make the changes coincident to major less~on plan revisions.

In summary, for performance type learning objectives such as found in OJT and simu- l 1stor lesson plans and drill guides, the conditions, performance requirements and

' evaluation standards are in general adequately defined or implied. Classroom learn-ing objective currently require a minimum grade of 80% on an and of course exami- i nation. These classroom learning objectives are being revised to meet INPO criteria i when lessen plan changes are required.

2-2.5. INPO RECOMMENDATIONS THI-1-R-5. Provide trainee performance criteria for i the administrative requirements section of the Auxiliary Operator Qualification Checklist to ensure consistent training evaluation.  ;

i i ,

i i 2-5 I

a 2-2.3.1. Discussion and Conclusions. The Administrative Section (Appendix A) of the Auxiliary Operator Training Program THI-1 (6211-PGD-2612.01 R'ev. 5.01 dated I 06/23/86) contains performance criteria for the eight tasks which the auxiliary oper-ator candidate must complete during his probationary period. The trainee must satisfactorily demonstrate his task knowledge and skills to a designated task examiner. Also included in Appendix A are a series of administrative system perfor-mance tests in which the trainee must also demonstrate his competence to a task examiner. These performance criteria and performance test satisfy in content and spirit this INFO recommendation.

~

2-3. ASLB PARTIAL INITIAL DECISION REPORT, MAY 3, 1985.

The ASLB conducted remand hearings on TMI-1 operator training and testing programs .,

! which heard testimony in a wide variety of training related areas. The board found one aspect of the THI-1 training programs to be deficient. The board found that there was no provision for any formal evaluation of trained operators in the job setting for the purpose of validating or revising the training program.

2-3.1. ASLB ORDER. The ASLB in the Remanded Report ordered the licensee (CPU-N) to implement a plan to evaluate the performance of trained reactor operators and senior reactor operators in the job setting, under both normal and abnormal operation, for revision of the THI-1 licensed operator training program.

i 2-3.1.1. Discussion and fonclusions. In response to this order TMI-1 has issued a j comprehensive instruction (6200-ADM-2682.10 dated 09/13/85) which establishes a pro-

! cess to evaluate training program effectiveness by collecting feedback data from j trainees and their supervisors,when training is completed and the trainees are back ,

I on-the-job. This instruction includes recommended methods for the supervisors to

, perform the evaluation.

i j The operator training program descriptions all contain procedures and timeframes for

! implementing the trainee on-the-job evaluation program. These timeframes vary from f approximately 6 months for the replacement operator candidates to annually for l licensed operators in requalification. The program descriptions specify the use of a structured evaluation form.

I i l 1 2-6 ,

i l 1 ,

1 - -- - --.. _- . . - _ - _ , __ _ _ _ _ _ , _ . . - _ . - - _ - - - . . .

I

?\;  ; ,

< l l

The supervisor evaluatings of once back on-the-job performance have been compre- i i

i, n hensively accomplished for each operator group in the timeframes specified. Cuper- l i-- , visor comments and recommendations have been evaluated and acted on in an effective manner in the context of existing and under-development training. This once back on-the job evaluation coupled with existing course and training program evaluation l

\ programs effectively comply with the ASLB order.

i-l 2-4. OVERALL ASSESSMENT.

io The DDL OMNI assessment team concludes, without reservations, that all the THI-1 s

programs examined are well designed, ef fective and well managed training programs which fully meet all the INPO accreditation objectives. All recommendations of the

? INPO accreditation team report are being complied with.

4 All programs examined are dynamic and progressive. Each program reflects modifi-i cations and changes af ter group session or cycles to take into account changing needs, added or deleted tasks, procedure changes, and feedback from the plant, instructors, and trainees. The quality of instruction observed was uniformly high, i '

with the instructors displaying not only subject matter knowledge and instructional I

skills but also activation, enthusissa and a genuine interest in the trainees. In return, the trainees were attentive, well prepared and receptive to the instruction.

The training f acilities are modern, functional, and devoid of superfluous instruc-tional gadgetry. With the new on site replica simulator now in operation, the TMI 3

training facility compares favorably with the best in the industry.

l The morale . attitude, demeanor, and appearance of the trainees and instructional staf f was high. Improvements in these areas were particularly noticeable to team

. members who participated in a similar 1982 DDL assessment. These changes were evi-3 dent throughout the training center but particularly noticeable in the cyclic train-ing classes where instructors presented refresher material with a different slant and where trainee participation was positive and constructive. The routine atten-dance of shift foreman at these sessions is no doubt a contributing factor to both i the quality of instruction and the attitude of trainees. In addition, the periodic attendance of the Vice President and Director TMI-1 at training sessions and simu-lator erarcises and the input of plant personnel to training program content provide l the management support and interest which is reflected in the attitude of the plant trainees.

2-7 1

SECTION 3 ASSESSMENT REPORT RECOMMENDATIONS 3-1 SECTION DESCRIPTION.

Sections 5 through 9 of this report contain recommendations dealing with the speci-fic training program reported in those sections. The recommendations in each sec-tion arc set in the context on introductory discussions and conclusions. In some cases, the listed reconsendations apply across the board to all programs assessed, while other recommendations may have applicability to only one or more programs.

For ease of tracking and follow-up, recommendations appearing in succeeding sections are tabulated below. Recommendations which are redundant in nature have been con-solidated where required. Editing has been provided to place the recommendations in this section in the appropriate reference.

3-2 OVERALL RECOMMENDATIONS.

The recounsendations appearing' in the paragraphs below are grouped by the INFO accre-ditation objectives against which training programs are evaluated.

3-2.1. DEVELOPMENT AND QUALIFICATION OF STAFF FOR TRAINING DUTIES.

a. Provide more frequent scheduling and a wider list of topics for Advanced Instructor Training. Require training department instructors to attend these sessions based on demonstrated need and to fulfill training qualifi-cation commitments.

b. Encourage instructors to participate in control room operations on a monthly vice quarterly basis.

c. Strenghten instructor quality by including in instructor certification /

qualification procedures requirements for continuing close supervision of instructors who hold interim certification or who only instruct occasion-l ally.

l' 3-2.2.

a.

SUPPORT OF TRAINING "ITH FACILITIES, EQUIPMENT, AND MATERIAL.

' Improve the arrangement of the BPT training room so that viewgraph slides can be displayed on a screen near the front of the room.

3-1

l 1

b. Consider providing a small number of cubicles in the Training Center for individual study by trainees.

3-2.3. CONDUCT OF JOB ANALYSIS AND IDENTIFICATION OF TASKS FOR TRAINING.'

a. As the assimilation of the Senior Reactor Operator plant-specific job anal-ysis continues, implement more formalized documentation to provide an

- audit trail from the task training matrix into the existing and in-devel- {

opsent training programs. This will ensure that all task training iden-tified is -included in the SR0 training program with appropriate training objectives identified.

b. Issue Shift Technical Advisor Training Standard, Rev. O which completes the match of the job analysis to training program content. (Appendix D to Rev. O is a matrix matching the current STA task list'to existing lessons numbers and training settings.)

3-2.4. ESTABLISHMENT OF TRAINING PROGRAM CONTENT.

a. At the start of Replacement Operator classroom training, test operator ,

knowledge on INPO guidelines specialized education topics which are not covered in the classroom fundamentals phase. Modify classroom training if necessary based on results of testing. (Many of the specialized education ,

topics are taught in A0 initial training. Retention of this knowledge should be verified.)

b. Review the Licensed Operator Requalification Program and consider modifi-cations in the following areas where additional content or higher stan-dards are contained in the recently promulgated INPO Guideline 86-025.

(Oct. 86)

1. Abnormal evolutions

2. Emergency events

3. Lecture series

4. Oral examination considerations

c. Develop a visible audit trail for all Senior Reactor Operator tasks iden-7 tified for classroom training into either existing (or modifications of existing) lesson plans. Perform the necessary formal task analysis and prepare new lesson plans if any remaining tasks are found to be not ade-quately covered in existing SRO asterial.

3-2

3-2.5. ' DEVELOPMENT OF LEARNING OBJECTIVES AS THE BASIS FOR TRAINING.

a. Reconcile the differences in terminal and enabling objective usage and format contained in various training directives (i.es 6200-ADM-2682.05

!. Training Standard, 6200-ADM-2682.07 Lesson Plan, 6200-ADM-2682.11 Program Evaluation, and the T&E Department Program Development Manual)

b. Ensure that future development and revision efforts reflect, to the extent practicable, the INPO guidelines relative to learning objective content and procedure 6200-ADM-2682.05 Training Standard.

3-2.6. ORGANIZATION OF INSTRUCTION USING. LESSON PLAN AND OTHER TRAINING GUIDES.

a. Rewrite the procedure 6200-ADM-2682.07 Lesson Plans, to make it a current, consistent and a complete document on lesson plan writing. Ensure that no other guidance and directions on lesson plan preparation and usage con-flicts with this procedure.

b. Improve the effectiveness of classroom lesson plans as a training tool by incorporating the following suggestions when lesson plans are revised:

- Provide a terminal objective supported'by only that number of enabl-ing objectives consistent with the duration of instruction.

- Key lesson plan outline to the learning objective which that part of -

the instruction supports.  ;

- Provide reference notes cueing instructors to the appropriate audio-visual media and to supplementary handouts.

- Include oral quiz items to stimulate class participation 'and test comprehension.

3-2.7. CONDUCT OF CLASSROOM INSTRUCTION AND TRAINEE EVALUATION,

s. Consider incorporating the following classroom instruction organizational practices:

- Schedule break times.

- Schedule 20-30 minute meal periods not during class time.

- Establish a ratio of study time to instruction time.

- Provide a place for individual study if desired.

- Provide optional homework for those who want to do it on their own time.

- Provide handouts to facilitate note taking.

- Provide punched copies of OPM chapters and handouts to help trainees develop their own training texts.

! 3-3

b. To strengthen the replacement l operator classroom theoretical instruction.

review site resources of licensed operators who are degreed engineers and select well-qualified individuals for instructor training and assignment to teach some theoretical subjects,

c. Continue to upgrade the quality of exams by appropriately covering lesson objectives in exam questions, by retiring questions that become familiar to trainees and by more sophasis on questions that require analysis in-stead of memorization. Improve grading of exams by carefully evaluating the process to arrive at an answer as well as the answer itself. Require correct units as part of the grading criteria. .

d. Increase the objectivity of oral examinations by defining in advance a minimum number of questions to cover the examination topics and the essen-tial elements of correct answers to the questions.

3-2.8. CONDUCT OF IN-PLANT TRAINING AND TRAINEE EVALUATION.

a. Improve the skills of OJT evaluators / trainers in the conduct of checkouts and final verification by implementing training and qualification of OJT evaluators / trainers specified in T&E Department Procedure 6200-ADM-2605.02.

b. Establish a method to document accomplishment of INPO Accreditation Cri-terion 9.2. on OJT instructor qualification. (Designated personnel who are instructed in program standards and methods conduct in-plant. training) 3-2.9. CONDUCT OF SIMULATOR TRAINING AND TRAINEE EVALUATION,

a. In view of the principal role of the replica simulator in licensed oper-ator training, consider the integration of the simulator section into the operator training organization at the appropriate time.

3-2.10. PROGRAM AND COURSE EVALUATION.

a. Review training program evaluation requirements for possible consolidation into fewer directives,

b. Include a former training program participant in the membership of the re-view group which examines the training program content at the end of each program presentation. i

c. Ensure that in future training program presentations, that procedurally specified trainee evaluations / critiques are accomplished once back on- l the-job.

1 3-4 l 1

, , - - ,, ,~-------a - - - , - - - - , , , , _ , , , , - , - - , , y , - - -

. = - . . .

SECTION 4 ORGANIZATION AND MANAGEMENT OF THE TRAINING SYSTEM 4-1. PROGRAM DESCRIPTION.

The. sections which follow evaluate and provide recommendations as applicable on the organization and management of the training program assessed. Section 5 examines in detail the organization and management of the training system as it effects reactor operator training programs. This section examines the corporate and training staff organization and management of goals, objectives, plans directions, and procedures.

4-1.1. CORPORATE TRAINING POLICY AND PLANS. The GPU Nuclear Corporation training policy and commitment is contained in procedure 1000-POL-2600.01. In its policy, GPU Nuclear considers training to be an important means of effectively developing personnel and solving and avoiding performance problems. The Corporation commits itself to _ providing efficient and effective training w' hich develops the trainee's skills and knowledge as necessary to perform the job responsibilities. This corpo-rate policy provides specific direction on significant training matters such as INPO accreditation, examination integrity and security, training evaluation, and inter-action between training and user organizations.

J 4-1.2. CORPORATE TRAINING PLAN. The GPU Nuclear Corporation Training Plan (1000-PLN-2600 .01) provides a systematic and consistent approach to the implementation of the GPU Nuclear Training Policy. This plan establishes training roles and respon-sibilities for each division. It defines a framework of principles, policies and procedures which will permit training needs to be identified and assessed, high quality courses to be developed, efficient and effective training to be delivered and training results to be assessed. It provides GPU Nuclear with tools to assure that the needed skills and knowledge of its personnel are developed or enhanced.'

4-1,3. MISSION AND GOALS. The GPU Nuclear mission and goals are established on a yearly basis along with those of subordinate organizations. These are conspicuously posted throughout the training building for staff and student perusal. Prominent among the 1986 goals of the Nuclear Assurance Division are the achievement of INFO

^

accreditation for remaining training programs and integration of the TMI-1 replica i 4-1 l

l

A simulator into selected RO and SRO programs within six months of completion of on-site acceptance tests. These training goals were appropriate and realistic and were met in 1986.

4-1.4. TRAINING AND EDUCATION DEPARTMENT ORGANIZATION AND FUNCTIONS. The organi-zation of the major functions of the Training and Education Department and its organizational components (Manager, Plant Training-TMI) are described in procedure 6200-ADM-1010.01. The TMI Plant Training Department, the Oyster Creek Plant Training Department, the Corporate Training Department and the Educational Develop-ment Department comprise the GPU Nuclear T&E Department. The Director T&E Depart-ment reports directly to the Vice President of Nuclear Assurance as do three other directors.

4-1.5. TMI TRAINING DEPARTMENT ORGANIZATION. The TMI Training Department organiza-tion, function and program descriptions are contained in procedure 6210-ADM-2600.01, Training Department Organization. The TMI Training Department which serves both Units 1 and 2 is composed of the five functional groups: Operator Training, i I

Technician Training, Simulator Development and Training, Training Support, and Training Administration Support. The managers of these five sections report directly to the Manager Plant Training TMI. The Operator Training Manager is responsible for licensed operator training and non-licensed operator training. The Technician Training Manager is responsible for maintenance training. . radiation protection technician training, and chemistry technician training. The Support Training Manager is responsible for such training programs as GET, radiation worker, fire and emergency preparedness. STA training is controlled by the Corporate Training Department.

4-1.6. TRAINING ADVISORY COUNCIL. A GPUN Training Advisory Council is established by the GPUN Corporation Training Policy Procedure, 1000-POL-2600.01. The purpose of the Council is to supplement interactions between the site training organization and the user divisions. The Council Charter (procedure 1000-ADM-2682.02) specifies its purpose, membership, duties and responsibilities. Membership consists of 12 individuals who are appointed by the user division vice-presidents / directors.

Members have authority to speak for their recpective divisions on training matters.

Meetings are held on a quarterly basis with formal agendas, minutes and reports.

Council recommendations must be responded to by the Director, T&E Departments. A 4-2

review of council meeting minutes provides evidence that this organization actively perform its role of advice and guidance on training matters.

4-1.7. PROCEDURES. An extensive hierarchial series of procedures is in effect which provides policy, direction, and specifications for the conduct, evaluation and administration of training at TMI-1. Higher level procedures provide policy, direction and guidance. Intermediate level procedures establish practices and methods for the standardization of instruction, evaluation, and administration.

Lower level instruction typically specify individual training program and instructor requirements. Procedure at the different levels are well coordinated and provide the details and supplement appropriate to their level. The responsibilities and functions of the TMI-1 operator training section are listed on one page of the TMI Training Department Organization. The T&E Department procedure on Program Descriptions is a six page document, whereas the TMI-1 Replacement Operator Training Program procedure is a 248 page instruction.

A process for initiating, reviewing, and revising procedures is in place and is functioning as specified. The majority of the upper and mid level training procedures have issue or revision dates in mid to late 1985. Lower level procedures dealing with training programs generally have 1986 issue or revision dates.

4-2. INPO OBJECTIVE.

The utility is organized, staffed and managed to facilitate planning, directing, evaluating and controlling a systematic training process that fulfills job-related training needs.

4-2.1. INPO CRITERIA. l 1

a. Written corporate and plant goals establish the required character and quality of key aspects of the training system. Supporting objectives are implemented at each organizational level.

4-3 l

~ ~ - = - - - _, . - , , . _ _ __. _ _ _ _ _ _ _ _ _

_I

b. The responsibilities and authority of personnel involved in managing, supervising and implementing training are clearly defined in writing and permit effective control of the training process.

c. Procedures are implesiented to ensure that instructional activities can be conducted. reliably and consistently.

4-2.2. CONCLUSIONS. The GPU Nuclear and the TMI Training Department are organized, staffed and managed to provide the range and quality of training necessary to meet the training roles and responsibilities required to perform plant job responsibilities.

4-2.3. RECOMMENDATIONS. None.

4-4

, ,,,,,r-- ' ' ' ' ' " ' 'e --m' -'--t-- *-T-""-- - - " - - - - - " +

SECTION 5 LICENSED OPERATOR TRAINING PROGRAM (RO) 5-1. PROGRAM DESCRIPTION.

5-1.1. REPIACEMENT ' OPERATOR TRAINING PROGRAM (RO) DESCRIPTION. The Replacement Operator Training Program (RO) consists of some 42 weeks of training, including about 12 weeks of classroom instruction, 22 weeks of on-the-job training. . and 8 weeks of simulator training. An additional three weeks are provided for compre-hensive oral, written, and simulator exams. Candidates for this program are generally selected Auxiliary Operators "A", although provisions exist for admitting direct hire candidates who meet the specified prerequisites.

The classroom training is divided into two phases. The first_ phase provides train-ing in fundamentals (reactor theory, thermodynamics, radiation control, and safety.)

The second phase provides for systems and integrated plant training.

- The on-the-job training program consists of two phases also. Both phases consist of l-instruction and performance of preselected tasks take'n from the CR0 task list.

Performance of plant procedures involving integrated plant operations are also included.

The simulator program is utilized to reinforce classroom and OJT concepts and to

' develop an understanding of integrated plant response.

A comprehensive written examination, an oral examination board, and plant walk through are given at the conclusion of the program.

5-1.2. LICENSED OPERATOR REQUALIFICATION PROGRAM (RO) DESCRIPTION. The Licensed Operator Requalification Program is designed to be sufficiently broad in scope to review areas of knowledge necessary for safe plant operation and flexible enough to cover recent operating experience and operational changes. This requalification j program utilizes four interrelated segments.

Pre-planned Lecture Series (includes Simulator Training)

Skills Training and Evaluation Operational Review Program Annual Examinations and Evaluations l

~

5-1 g ,w wm_.ep_ ,..% ,..=,y,y. ----t--- - - , - - - y ~ --

h.+ e +- w-, ,,%-y....-,,---w ,--..,,,.% . , , , , , m. _,

= - . - . . . ._ _ -._ _

The ' operator requalification program is conducted on a cyclic basis so that all program requirements are completed in a period not to exceed two years.

5-2. ORGANIZATION AND MANAGD(ENT OF THE TRAINING SYSTEM.

5-2.1. INPO OBJECTIVE. The utility is organized, staffed, and managed to facili-tate planning, directing, evaluating, and controlling a systematic training process that fulfills job-related training needs.

i 5-2.2. INFO CRITERIA.

a. The actions needed to achieve high quality, job-related, performance-based training programs eligible for accreditation have been identified through a systematic evaluation of existing programs.

b. Written corporate and plant goals establish the required character and quality of key aspects of the training system. Supporting objectives are implemented at each organizational level.

c. The responsibilities and authority of personnel involved in managing, supervising, and implementing training are clearly defined in writing and permit effective control of the training process.

d. A . training system is implemented as the primary management tool for 4

developing, conducting, and evaluating training.

e. Procedures are implemented to ensure that instructional activities can be conducted reliably and consistently.

f. Training to be completed prior to qualification is clearly defined.

Exemptions from training may be granted when justified and supported by a documented assessment of prior training and experience.

g. Training records are maintained to support management information needs and provide required historical data.

h. Programs offered under contract remain under the control of the' sponsoring utility and are evaluated by it to ensure that the INPO accreditation objectives and criteria are met.

1. The work load of the training staf f indicates that there are sufficient qualified personnel to accomplish assigned duties and responsibilities.

5-2.3. CRITERIA EVALUATION. The paragraphs which follow comment on th+ above INPO

. Criteria as applicable.

5-2 c-m+--n e,'i-w--aowrwa-m re,-- -----e - - ' - - + +

• wm+mi~++yre,-,i----ww-rw y--wyo-- m r w.w--wy-e--m mm-ww wr , - . rw-r '-w y,--'n,y,w-gr ----w---wy--

w r- v -

. O:

5-2.3.1. Program- Evaluation. A systematic evaluation of existing programs to identify actions needed to achieve high-quality, job-related, performance-based training programs is established by several GPUN directives, which include:

'a. GPU Nucleer Corporation Training Policy, 1000-POL-2600.01

b. Training Advisory Council, 1000-ADM-2682.02

c. Training and Education Department Training System Development Process, (TSD) 6200-ADM-2682.01

d. Course Evaluation Process, 2600-ADM-2682.12

e. GPU Nuclear Technical Content Review and Interface Process,

.6200-ADM-2682.03

f. Program Evaluation, 6200-ADM-2682.11 3 Trainee Evaluation-Once Back on the Job, 6200-ADM-2682.10

h. Program Descriptions, 6200-ADM-2682.06

1. On-the-Job Training, 6200-ADM-2605.02

j. TMI-1 Replacement Operator Training Program, 6211-PGD-2611.04 l

k. Licensed Operator Requalification Program Description (Unit 1), 6211-PGD-2611.01 These directives, in one form or other, deal with trainee and training program evaluation. The most relevant of these requirements are summarized and included in the Replacement Operator Training Program (RO) Procadure. The evaluation require-ments of these procedures are summarized as follows:

a. A once-on-the-job training program eval: ation (about six months) by the RO candidate.

b. A once-on-the-job training program evaluation by the RO candidate super-visor evaluating training related performance.

c. A post training review at the end of each program presentation by the ,

i Supervisor Licensing Training and the course instructors.

4

d. An annual review of program content by a team consisting of the Supervisor Licensing Training and designated licensed / certified SR0s and CR0s.

5-2.3.2. Responsibilities. The GPUN policy that program descriptions specify the approval process for all program materials is set forth in the Responsibilities l section of Program Descriptions, 6200-ADM-2682.06. In the case of replacement i operator training, the program description includes a program approval section which assigns responsibilities for certifying the candidate's readiness for the licensing 1

5-3

examination, ensuring the overall level of training uf plant operators is satisfac-tory, ensuring the training program is developed to meet requirements, maintaining proper records and documentation, and approving lesson plans. Various other respon-sibilities are specified throughout the program description.

The program description for licensed operator requalification trainit.g includes lists of responsibilities for supervisors, managers, and directors concerned with this program.

The definition of responsibilities and approval for these programs permits effective control of the training process.

5-2.3.3. Training System. The directive Training and Education Department Training System Development Process (TSD) 6200-ADM-2682.01, implements a training system as the primary management tool for development, conducting, and evaluating training.

5-2.3.4. Instructional Procedures. Procedures to ensure reliable and consistent conduct of instructional activities for replacement operator and licensed operator requalification training are included in the following directives:

a

a. Replacement Operator Training Program, 2611-PGD-2611.04,' describes program purpose, scope / applicability, responsibilities, prerequisites, sequence, objectives, outline, administration.

b. Licensed Operator Requalification Program, 6211-PGD-2611.01, includes the same items as a. above for requalification training.

c. Various evaluation procedures are described in documentation listed in paragraph 5-2.3.1.

d. Training Standard, 6200-ADM-2682.05, describes the use of training stan-dards as part of the GPUN TSD model.

.e. Training Records, 6210-ADM-2600.02, identifies documents produced in the Training Department that become Quality Assurance Records and prescribes methods for their control and turnover to the Information Management Cen-ter.

f. Lesson Plans, 6200-ADM-2682.07, establishes the format for lesson plans.

g. Operator Training Instructor Indoctrination / Qualification Training Pro-gram, 2610-ADM-2610.02, provides the requirements for indoctrination, training, and certification of instructors.

5-4

.h. Instructor Evaluation, 6200-ADM-2607.01, defines the instructor evaluation program and assigns specific responsibilities pertaining thereto.

- The overall effectiveness of RO training is indicated by the following results in passing the NRC licensing exams since 11/84:

Passed

' Passed On Group # Started Dropped First Attempt Re-Exams 5 9 3 4 2' 6 8 3 5 -

5-2.3.5, Required Training and Exemptions. Training to be completed in the re-placement operator ' training program prior to the NRC licensing exam is defined in the program description as follows:

a. Phase 1 - Fundamentals Classroom Training and OJT.

b. Phase 2 - Systems and Integrated Plant Classroom Training and OJT. Three weeks in this phase are required for simulator startup certification and operational evaluation.

c. Audit exams - comprehensive oral, written, and simulator exams.

The content and requirements for successful completion of each segment are clearly  ;

explained in the program description.

The tentative time line for the CR0 class commencing 11/10/86 shows the following allocation of time for this program:

a. Phase I - Classroom, 5 weeks; OJT, 7 weeks

b. Phase II - Classroom, 8 weeks; OJT, 15 weeks; Simulator, 8 weeks 1

c. Audit exams - 3 weeks l

The program description permits deletion of requirements for the program due to prior candidate experience. An assessment of the trainees' knowledge and previous operating experience and qualifications is made by the Supervisor, Licensed Operator Training using a deletion letter format included with the program description. By signing the deletion letter, the Supervisor, Licensed Operator Training is verifying knowledge level and operating experience through oral or written examination and/or i- personal review of candidate's records. Approval of the deletion request by the Operator Training Manager and the Plant Operations Director is required.

i 5-5

The program description also permits waiving satisfactory completion of the plant fundamentals training segment if written examination has verified that the knowledge and skill of the individual is comparable to that of individuals who have completed the trap.ing.

In the case of licensed operator requalification, training to be completed prior to requalification as defined in the program description consists of the following sag-ments:

a. Pre-planned lecture series

b. Skills evaluation and training

c. Operational review program

d. Annual examination and evaluation The content and requirements for successful completion of each segment are clearly

~

explained in the program description.

An exemption of no more than one week may be granted from the pre-planned lecture series. This absence must be approved in advance by the Plant Operations Director.

The individual who misses training is responsible for the material presented during the absence and for taking and passing the exam that was given on the missed mate-rial.

Newly licensed individuals enter the re-qualification program and participate in the annual program cycle upon receipt of their license. Newly licensed individuals whose NRC license date is less than three months prior to an annual requalification exam are excused from taking the TMI-1 annual written and oral exams.

5-2.3.6. Trainina Records. Training records are maintained in accordance with the directive Training Records, 6210-ADM-2600.02 to support management information needs and provide required historical data. This procedure identifies the documents produced in the Training Department that will become Quality Assurance Records and prescribes methods for their control and turnover to the Information Management Center (IMC). Documents identified as training records are:

s. Training content records

b. Training attendance / completion records

c. Training examination records 5-6 v .,.vr-- . - - - . , ,~,y, --e..--._-- . , , , , - . . , . , , . . , - - - . ,

---,m-,_y r, , . - . - - -.---4 ,.-

g.

Training records are delivered to the Document Control Group where they are reviewed for - format , content, and quality. If these requirements are satisfied -the docu-ments are entered into the records system by photo copying documents onto microfiche and transmitting the originals to the IMC.

J The scope, methods, and objectives of training evidenced by training records is documented by retaining the following in Training Department files:

a. Training program descriptions and revisions

b. Lesson plans and revisions

c. Correspondence relating to a student's ability to perform the activities for which training was provided

d. Training Department procedures and revisions which specify how activities are to be performed The following documents for Group 6 Unit CRO replacement trainees were retrieved and reviewed on microfiche:

a. Five weekly exams for fundamentals and systems b .- Theory comprehensive exam

c. System comprehensive exam

d. Mock NRC written exam

e. Fundamentals board

f. Upgrade oral exam

g. OJT spot checks j

h. Final comprehensive oral exam

1. Mock NRC oral exam

j. Attendance records e h. Training content records There were no problems identifying and retrieving these documents. In some cases, handwriting was indistinct and difficult to read on the microfiche and even less useful when a print of the microfiche was made. No attempt was made to retrieve and examine originals of documents as it was reported these are stored at various loca-t tions and require considerable time to obtain. The relative dependence on micro-fiche for review and analysis of records increases the importance of making entries in black ink or sof t pencil (#2 or sof ter) as required by the procedure.

5-7

~

1 i

5-2.3.7. Contract Trainina. With the December 1986 acceptance of the on-site replica simulator for training by GPUN instructors, training at the PSI simulator by ]

contractor personnel is no longer required. Past simulator training conducted under contract was not examined. No other training for reactor operators is conducted l

under contract at this time. i 5-2.3.8. Instructor Workload. A total of eight instructors including the super-visor are assigned to licensed operator training. Of these, six including the supervisor participate in training Unit 1 licensed operators. During the period January through October 1986, the average number of hours worked per week by these instructors on instructing, training, and administration activities were as follows:

R.H..Maag (Supervisor) 52.5 W.G. Ogle 42.7 R.L. Parnell 44.5 E.D. Showalter 46.1 W.S. Stanley 44.1 M.E. Wynne 27.0 (for period April - October 1986)

This workload indicates that there are sufficient classroom instructors to accom-plish assigned duties and responsibilities.

5-2.4. CONCLUSIONS. The TMI Licensed Operator (RO) Training Programs' are well organized, managed, and staffed to provide a systematic approach to fulfilling the Unit 1 job-related reactor operator training needs.

5-2.5. RECOMMENDATIONS.

a. Review training program evaluation requirements for possible consolidation into fewer directives.

b. Include a former training program participant in the membership of the review group which examines the training program content at the end of each program presentation. (Trainee input can be an asset in evaluating the training process and content.)

I 5-3. DEVELOPMENT AND QUALIFICATION OF STAFF FOR TRAINING DUTIES 5-8

-_ . _ _ _ _ _ . . _ _ _ ___ ___ _ __ . . _ , _ _ _ _~ _ . . . _ . _ _ _ _ _ _ . . _ _._,

5-3.1. - INPO OBJECTIVE. Training staff (utility and contracted, if used) possess f

.the technical knowledge, the experience, and the developmental and instructional-skills required to fulfill their assigned duties.- -

5-3.2. INPO CRITERIA

a. Training staff responsible for program management, supervision, and development have and maintain the education, experience, and technical qualifications required for their jobs,

b. Instructor technical qualifications are appropriate for the subject matter that they are assigned to teach.

c. Methods are implemented to ensure that individual instructors meet and-maintain position qualification requirements.

d. When instructors have not yet attained the required instructional qualifications or only instruct occasionally, training quality is maintained through appropriate additional assistance and supervision.

e. ' Instructor performance is evaluated regularly, and the results are used to improve performance,

f. Continuing instructor development efforts maintain, improve, and advance required knowledge and skills and are based, in part, on evaluations of instructor performance.

5-3.3. CRITERIA EVALUATION. The paragraphs which follow comment on the-above INFO criteria as applicable.

5-3.3.1. Instructor Qualification. The instructors regularly assigned to teach classroom subjects to Unit i licensed operator candidates and/or licensed operators have the following education, experience, and technical qualifications noted on the r

instructor qualification cards:

Date Started Tech Additional Name Education Inst. Qual. Qual. Information R.H. Maag A.A. degree 7/84 Current Supervisor Licensed SRO Unit 1 Operator Training W.G. Ogle High School Current Unit i SRO trainee in SRO Unit 2 Replacement CR0 class which started 11/86 5-9

R.L. Parnell 'High School 8/84 Former Unit 1 SRO trai* toe in SRO Unit 1 ' Replacement CRO cicss which started 11/86-E.D. Showalter High School plus 8/86 Current 4 yrs college SRO Unit 1 -

W.S. Stanley 6 yrs college 8/84 . Current MS Nuc Eng SRO Unit 1 M.E. Wynne High School plus 4/86 Current 3 yrs. college SRO Unit 1 Although listed on the 11/17/86 TMI Training roster under Simulator Development /

Training, the following instructor taught fundamentals and systems subjects to the 1985 Replacement .CR0 class and is assigned to instruct Basis Principles- Trainer (BPT) sessions for the Replacement CR0 class which started 11/86:

Date Started Tech Name Education Inst. Qual. Qual. Consents D.L. Wilt H.S. plus some 12/82 Inst Cert college work SRO Unit 1 As indicated on the qualification chart, W.G. Ogle and R.L. Parnell are in the pro-cess of obtaining Unit 1 SRO upgrade training while also providing Unit 1-instruc-tion in fundamentals and systems. Both have been evaluated as effective instructors I with knowledge of the material being presented. As such, they meet ANSI /ANS - 3.1 standards for training instructors of non-critical license subjects.

5-3.3.2. Maintaining Qualification. Methods which are implemented to ensure that individual instructors meet and maintain position qualification requirements include:

a. The procedure for instructor qualification, Operator Training Instructor Indoctrination / Qualification Training Program, 6210-ADM-2610.02 provides requi *ments,- responsibilities, and procedure for indoctrination, train-ing, and certification of instructors involved in training licensed and non-licensed operators and shif t technical advisors. The directive in-cludes as Attachment 1 a qualification card for each instructor setting forth requirements and sign off places for recording satisfaction of re-quirements. The required steps to achieve instructor certification to conduct licensed operator training are completion of:

. 5-10

(1) Briefing on qualification procedure (2) GET/RWP Training.

(3) Referance documentation indoctrination (4) Instructional knowledge requirements. This area is satisfied by com-plation of the Basic Instructor Development Program, a one-week course conducted periodically or by interim certification until the next scheduled instructor course. Intetim certification is based on satisfactory completion of a practice lecture.

(5) Technical qualification requirements. These include:

(a) Power plant fundamentals and theory subjects which may be satis .

fied by SRO license / certification or by satisfactory completion of training or education in eight fundamentals subjects at or

~a bove SRO (STA) level. A matrix is provided showing various past educational and training achievements which can be used with an interview to validate a satisfactory level of knowledge in these subjects.

(b) Systems and components, transient and integrated plant response, operating procedures, and simulator instruction. This area is satisfied by SRO license or certification.

(6) Manager, Plant Training certification Certification that an individual is ready in all respects to commence instruction is made by the Manager, Plant Training.

The directive provides for continuing training af ter initial certification by assignment of Advanced Devr iopment Modules and setting as a goal the attainment of SRO qualification.

The completed qualification card and supporting documentation for instruc-tors are maintained by the Operator Training Administrative Assistant.

Qualification cards and supporting documentation for licensed operator training instructors were examined. All instructors have completed the Basic Instructor Development Program and are certified ready to commence instruction.

5-11

.. . - _ . . - ~ . . - . -

There are few entries in Section VII of the qualification card to indicate that Advanced Instructor Develops.ent. Modules have been assigned or com-pleted as part. of continuing training.

b. The program description for replacement operator training, 6211-PGD-2611.-

04 states that " classroom training shout.d consist of two phases with lec-tures presented by qualified instructors using approved lesson plans."

c. The program description for licensed operator requalification, 6211-PGD-2611.01, states that " Training Department and facility instructors who teach systems, integrated responses, and transient courses shall have_dem-onstrated their competence to the NRC by successful completion of a senior operator examination or certification."

d. The directive Program Descriptions 6200-ADM-2682.06, requires that program

. descriptions include a subsection that describes the skills and exper-ience, both technical and instructional, that an instructor should possess as a minimum to be able to teach the program.

e. The licensed operator requalification program description specifies that licensed instructors from the Training Department staff shall actively participate in control room operations a sinimum of two shifts per month.

This requirement can be met on a quarterly basis. A review of actual instructor participation indicated that frequently, all participation for.

a quarter was bunched near the end of the quarter. This practice is considered less optimum for mait.taining instructor currency in the plant than parti:1pating in control room operations on a monthly basis.

5-3.3.3. Instructor Development Certification. The directive for operator training instructor qualification, 6210-ADM-2610.02, provides for interim certification by Manager, Plant Training to satisfy instructional knowledge requirements based on satisfactory completion of a practice lecture. This permits individuals who have completed all other prerequisites to commence instruction without waiting for the semi-annual Basic Instructor Development Course. Interim certification is valid for a maximum of six months.

! Additional assistance and supervision are not addressed in the procedure for instructors who hold interim certification or who only instruct occasionally. ,

5-12 i

5-3.3.4. Instructor Evaluation. The records for. instructors regularly assigned to teach classroom subjects to Unit 1 licensed operator candidates and/or licensed operators contain the following most recent evaluations:

Name Evaluation Date

.R.H. Maag 7/28/86, 8/26/85 W.G. Ogle 10/20/86, 7/8/86 i

R.L. Parnell 10/8/86 E.D. Showalter 10/10/86, 10/2/86 W.S. Stanley 8/4/86, 7/31/86 M.E. Wynne 10/9/86, 8/22/86 D.L. Wilt 10/85, 1/86, 8/86 The evaluation form contains a detailed checklist of instructor attributes to observe and comment on. Remarks on completed forms were constructive and could be helpful in improving performance.

The directive Training Department Instructor Evaluation Procedure, 6210-ADM-2631.02, specifies the frequency of evaluation, forms to be used, and action to be taken with the evaluation.

5-3.3.5. Continuina Instructor Development. Continuing instructor development ef-forts are not documented as being based, in part, on evaluations of instructor per-formance.

5-3.4. CONCLUSIONS. The TMI-1 instructors assigned to teach licensed operators and licensed operator candidates possess the technical knowledge, the experience, and the developmental and instructional skills to fulfill their assigned duties.

1 .

5-3.5. RECOMMENDATIONS.

a. Develop the instructional capabilities of licensed operator training instructors by assignment of Advanced Instructor Development Modules as provided for in 6210-ADM-2610.02 (see 5-3.3.2a) . ,

b. Encourage instructor participation in control room operations on a monthly vice quarterly basis.

c. Strengthen instructor quality by including in 6200-ADM-2730.01 and 6210-ADM-2610.02 requirements for continuing close supervision of instructors who hold interim certification or who only instruct occasionally.

d 4

5-13 i

s,_-,,_-..._ . - . . . , , c __ _ _ _ _ , _ . .,y,_, ,_,,__.,,..,,,_o

.- - - - = _ .. -. .

5-4.- SUPPORT OF TRAINING WITH FACILITIES, EQUIPMENT, AND MATERIALS.

5-4 .' 1. INPO OBJECTIVE.- The . training facilities equipment, and materials ade-quately support. training activities.

5-4.2.- INPO CRITERIA.

s. Instructional facilities meet training needs.

b. The training staff has necessary instructional aids and equipment.

. c. Technical reference materials, including current plant procedures ' and drawings, are readily available to the trainees and instructors.

5-4.3. CRITERIA EVALUATION. The paragraphs which follow comment on' the above INPO I

criteria as applicable.

5-4.3.1. Facilities and Instructional Support. Instructional facilities are over-all excellent in meeting the training needs for replacement operator and licensed operator requalification classes. Sufficient classrooms of different sizes are available to efficiently handle various groups. Classroom 222 located in Bldg . 2, close by instructor cubicles, is being used for the current replacement operator class. It is equipped with white chalkboard, viewgraph projector, video tape capa -

bility, and a set of the nine volume Operator Plant Manual. Additional reference material is available in the Training Department library. All class presentations and individual study take place in this room. Study periods observed consisted mainly of group discussions with very little individual study. The BPT room is arranged with the BPT at the front and with student seating facing the BPT. A whiteboard is on a side wall and a viewgraph projector is set up to display slide material on the back wall.

5-4.3.2. Conclusions. The training facilities, equipment, and materials very adequately support training activities.

]

5-4.3.3. Recommendations.

a. Improve the arrangement of the BPT training room so that the viewgraph slides can be displayed on a screen near the front of the room.

b. Consider providing a small number of cubicles in the Training Center for individual study by trainees.

i 5-14

- . . . . . .- - -. ._ _ - - - - . . = .

5-5. CONDUCT OF JOB ANALYSIS AND IDENTIFICATION OF TASKS FOR TRAINING.

5-5.1. INPO OBJECTIVE. The tasks required for competent job performance are identified, documented.-and included in the training programs, as appropriate.

5-5.2.- INPO CRITERIA.

s. Plant personnel, training staff, and other subject matter experts, as appropriate and as needed - have conducted a job analysis to develop a valid plant-specific task list.

b. Subject matter experts (appropriate plant technical personnel, training staff personnel, or knowledgeable outside personnel) assist in the selec-tion of tasks for training.

c. Each task selected for training from the plant-specific task list is compared with existing training materials in sufficient depth to determine if existing training adequately supports task performance.

d. The plant-specific list of tasks selected for training and the comparison to training materials is reviewed periodically and updated, as necessary.

5-5.3. CRITERIA EVALUATION. The paragraphs which follow comment on the above INPO criteria as applicable.

5-5.3.1. Job Analysis and Selecting Tasks for Training. Plant personnel.and train-ing staff have conducted a job analysis of the reactor operator position and devel-oped a valid plant-specific task list. The latest review and update of this list took place in summer 1986. The task list, 6211-ADM-2611.06, which has been reviewed l

by subject' matter experts, indicates a training location or locations (classroom.

OJT, simulator) for each task. The revised CR0 task list contains 502 tasks which are identified for training. One hundred eighty tasks are identified in the train-ing matrix for classroom presentation, 440 tasks for OJT and 195 tasks for simulator i training. Some tasks appear in multiple training settings. For tasks covered in the classroom, the number of the lesson plan used is indicated. I.esson plans and simulator exercise guides now being written or revised correlate individual tasks with objectives and lesson content. The OJT checklists included with the replace-ment operator program description indicate correlation of tasks with OJT require-4 ments.

4 5-15 I

. - ,.-v_---.-._, . . - - .

5-5.4. CONCLUSIONS. The tasks required for Control Room Operator job performance have been systematically identified and tasks have been identified for training. A training matrix has been developed for each task showing the training setting (classroom OJT, simulator, etc.).

5-5.5.- RECOMMENDATIONS. None. .

5-6. ESTABLISHMENT OF TRAINING PROGRAM CONTENT.

5-6.1. INPO OBJECTIVE. Training program content provides the trainee with the knowledge and skills needed to perform functions associated with the position for which training is being conducted. The content of initial training prepares the trainee to perform the job for which he is being trained. The content of continuing training maintains and improves incumbent job performance.

5-6.2. INPO CRITERIA.

a. INPO training guidelines are used as a guide for selecting, sequencing, and verifying training program structure and content.

b. Tasks are analyzed, as necessary, to determine the task's supporting skills and knowledge to be included in training programs.

c. Personnel qualified in the position for which training is being conducted 1

help determine training content and confirm its completeness.

d. Current plant procedures and other technical and professional references are used to identify training content and plant-specific information for use in developing training materials.

e. Initial training program content is modified to reflect the results of program review and evaluation by plant and training staff personnel.

f. The results of trainee and program evaluations are used to help determine the content of continuing training.

5-6.3. CRITERIA EVALUATION. The paragraphs which follow comment on the above INP0 criteria as applicable:

5-6.3.1. INPO Training Guidelines. A comparison was made between the INPO Training Guideline 83-022 PWR Control Room Operator Qualification and the TMI Unit i replace-ment operator training program regarding structure and instructional content. The 5-16

INPO guideline for CR0 instructional programs contains four major course areat which can be related to similar areas in the TMI-1 Training plan. In Course 1, Specialized Education (mathematics , physics, engineering, drawing, electrical science etc.) the INPO Guideline contains 590 hours0.00683 days <br />0.164 hours <br />9.755291e-4 weeks <br />2.24495e-4 months <br /> of instruction as compared to the TMI-1 program of 176 hours0.00204 days <br />0.0489 hours <br />2.910053e-4 weeks <br />6.6968e-5 months <br /> of instruction.

Some part of the specialized education included in auxiliary operator training may be counted toward contact hours suggested by the INFO guideline for control room operator training but retention of prior education knowledge is not tested as part of replacement operator training.

In Course 2 Plant Technology, Systems and Procedures Training, the INPO Guideline contains some 600 hours0.00694 days <br />0.167 hours <br />9.920635e-4 weeks <br />2.283e-4 months <br /> of claanroom plus in-plant tracing of systems. The TMI-1 replacement operator program has an estimated 320 hours0.0037 days <br />0.0889 hours <br />5.291005e-4 weeks <br />1.2176e-4 months <br /> of training plus some ,

undetermined amount of OJT.

In Course 3, Operating Practices Training, the INPO Guideline contains some 17 weeks of training as compared to some 30 weeks in the TMI-1 program.

In Course 4 Industry Experiences and Modifications Training, the INFO Criteriam has no specified time. The TMI-1 program provides 5-week exam preparation period during which this course is covered.

As a sommary, the replacement operator training program in comparison with the INPO guideline has significantly fewer classroom hours in fundamentals and systems train-ing and significantly more time allocated to OJT and simulator training. Several specialized education topics included in the INPO guideline receive no coverage in the replacement operator training program. These topics are covered earlier in the Auxiliary Operator initial training program. 1 A similar comparison was made between INPO Guideline 86-025. Guideline For Contin-uing Training of Licensed Personnel and the TMI Unit 1 licensed operator requalifi- ]

cation program regarding structure and content. The INPO Guideline and the TMI Requalification Program each contain 240 hours0.00278 days <br />0.0667 hours <br />3.968254e-4 weeks <br />9.132e-5 months <br /> of instruction per year, but the structure is somewhat different. The following TMI-1 Program requirements are dif-  !

ferent than the INPO Guideline.

5-17

a. In skill training, the TMI-1 program does not include a number of abnormal evolution or emergency events listed in the INPO Guidelice,

b. Several emergency events are performed on a two year cycle basis instead of at least annually.

c. The lecture series does not include a number of copics in the Guideline.

d. Several oral examination consideration in the INPO Guideline are not included in the TMI-1 Program.

5-6.3.2. Task Skills and Knowledse. The Reactor Operator TMI-1 Task List, 6211-ADM-2611.06 contains a matrix showing the training location (s) (classroom, 9JT, sim-ulator) for each specific task item. The- TMI-1 Replacement Operator Training Pro-gram Description, 6211-PGD-2611.04, Revision No. 5 issued 12/22/86 includes perform-ance standards for skills and knowledge supporting each task to be covered during

OJT. 'In a similar manner, as lesson plans are revised, knowledge supporting each task to be covered in a classroom lesson is correlsted with. lesson objectives.

5-6.3.3. Training content Determination.- Qualified reactor operators help deter-mine training content and confirm its completeness in the following ways:

a. Licensed operator training instructors who contribute to development cf training programs and training materials are qualified reactor operators.

b. Replacement operator training and licensed operator requalification pro-gram descriptions require approval by managers who are licensed operators.

4 The Plant Operations Director is responsible to ensure that the overall level of training of plant operators is satisfactory through the approval of program content, schedules and administrative procedures.

c. The Plant Operations Director or his designee conducts an operational examination of each candidate at the completion of the simulator program and participates in the final oral examination of each candidate.

d. The licensed operator requalification program description includes lists of recponsibilities that the Supervisor, Licensed Operator Training. Oper-ator Training Manager and Plant Operations Director have for content and completeness of training provided by this program.

5-18

5-6.3.4. Plant Documentation. Current plant procedures, Operator Training Manual, Tech Specs and various other technical and professional references are used to iden-tify training content and plant-specific information for use in developing licensed operator training materials.

6.3.5. Content -Modification. Training program content is modified to reflect the results of program review by plant and training staff personnel as indicated by the following dates of major revisions to program descriptions:

Immediate

! Past Current <

Revision Revision Replacement Operator 4-01 5 Training Program 8/16/85 12/12/86 Licensed Operator 2 3 Requalification Program 8/16/85 10/26/86 -

5-6.3.6. Evaluation Results. The licensed operator requalification program description provides for review and evaluation of the program on an annual basis.

The Supervisor, Licensed Operator Training is responsible for the review which ,

encompasses nine areas.

! An evaluation of - licensed personnel performance on-the-job is also conducted in accordance with the program description to help determine the content of continuing training. The most recent such eveluation was completed in May 1986.

Recommendations to modify content of continuing training have been reviewed and ac-

! tion to strengthen the program has been identified.

5-6.4. CONCLUSIONS. The content of the Reactor Operation Replacement and Continu-ing Training Programs prepare the trainee to perform the job and maintain and improve incumbent job performance by providing the required knowledge and skills.

i 5-6.5. RECOMMENDATIONS.

a. At the start of Replacement Operator classroom training, test operator knowledge on INPO guidelines specialized education topics which are not

' covered in the classroom fundamentals phase. Modify classroom training if necessary based on results off testing.

5-19

[

o v - , , , ,-,m-m -, .7,,-g w -w-, , m_ __ , _ -, .g,-_----n,,.,w y -ng ,,,,,,,-me,w,,n, , , - - ,,- -, r,n,. -,----

b. Review the Licensed Operator Requalification Program and consider modifi-cations -in the following . areas where additional content or higher stan-

'dards are contained in the recently promulgated INPO Guideline 86-025.

1.: Abnormal evolutions

2. Emergency events

3. Lecture series 4.- Oral examination considerations 5-7. DEVELOPMENT OF LEARNING OBJECTIVES AS THE BASIS FOR TRAINING.

5-7.1. INPO OBJECTIVE. Learning objectives that identify training content and define satisfactory trainee performance are derived from job performance require-ments.

5-7.2. INPO CRITERIA.

a. Expected entry-level skill, knowledge, and experience are considered when developing. learning objectives.

b. Learning objectives are derived from an analysis of job performance re-quirements and are the basis for trainee evaluation.

c. Learning objectives state the actions (s) the trainee must demonstrate, the conditions under which the action will take place, and the standards of.

performance the trainee should achieve upon completion of the training activity.

d. Learning objectives are sequenced based on their relationship to one another and help trainees move from one level of skill and knowledge to another.

5-7.3. CRITERIA EVALUATION. The paragraphs which follow comment on the above INPO

, criteria as applicable..

5-7.3.1. Entry-Level Skill and Knowledge. The directive Program Descriptions, 6200-ADM-2682.06, states that a program description shall include prerequisites that

specify what the trainee must know or have experienced prior to being permitted to enter the program or course. The TMI-l Replacement Operator Training Program

! description, describes two categories of candidates. A replacement candidate is a TMI-1 qualified Auxiliary Operator "A" who meets other prerequisites stated in the program description and is designated by the Plant Operations Director. The pre-5-20

{ T

g s4

y.  :

requisites include a high school diploma or equNalency and minimum experience jj levels at THI-1 and at other power plants. These prerequisites help to define ex-pected entry-level skill, knowledge, and experience to be considered when developing

' learning objectives. l The program description also defines a direct candidate as any individual not pre-viously TMI-1 Auxiliary Operator "A" qualified who meets other program prerequisites and is designated by the Plant Operatioas Director. A is .3 intended to allow flex-ibility in selecting candidates for the program based , ether undefined experience 1

{ which substitutes partially for Auxiliat7 Operator "1 :talf*ication. (The direct 1 I candidate completes various OJT A0 tasks which are no. <

red for replacement can- l diiiates. ) -

5-7.3.2. Learnina Objectives. The replacement speratu training program descrip-tion states that knowledge type classroom topics to be addressed are to be selected by the Supervisor, Licensed Operator Training based ct. candidate experience /know-ledge and the CR0 Job and Task Analysis. The reactor operator task list has recently been updated but no analysis of tasks has been accomplished to date to define the knowledge and skill requirements to accomplish the tasks. This is con-sistent with INPO guidance for existing training program accreditation. Lacking such an analysis, tasks are considered -when developing classroom and simulator V' lesson plan learning objectives. Lesson plans revised or developed recently asso-ciate task numlers with learning objectives listed in the Training Content Record.

A lesson nlan revision checkoff sheet includes items on cross referencing task designations to lesson plan objectives and including in the lesson all tasks iden-tified on the JTA matrix that are applicable.

5-7.3.3. Learnina Objective Format. Learning objectives for classroom lessons are listed on the lesson plan Training Content Record. Typically the objectives are stated that at the conclusion of the lesson the student should be able to define, I

explain, list, solve, describe, state, point out, or draw.

The standard of performance the trainee should achieve in the classroom phase of replacement operator training is a score of at least 80% on all written examina-tions. This is set fort.h in the program description. The conditions under which the trainee will demonstrate required knowledge to meet the standard are stated to be at the conclusion of the lesson.

5-21

OJT tasks and performance standards are grouped in related areas. This allows for the interrelationship of topic areas to be studied and examined. Twenty three groupings of related tasks are provided.

A condition under which the action will take place is designated for each task.

These are Perform. Simulate, or Discuss.

Satisfactory completion of an OJT task is indicated by the task examiner signing the trainee's qualification card. Verification of a section or group of tasks is by oral examination with pass or fail results documented on a summary sheet.

5-7.3.4. Learning objective Grouping. The reactor operator task list provide the main source for development of learning objectives. Recent revised and developed classroom and simulator lesson plans associate specific tasks by number with learn-ing objectives stated in the Training Content Record. The task list designates the setting in which the training will be conducted.

5-7.3.5. Learning Objective Sequencing. Replacement operator program segments and their related learning objectives are sequenced so that power plant fundamentals are covered first, followed by systems training with associated OJT and leading to inte-grated plant operations and simulator training.

5-7.4. CONCLUSIONS. Learning objectives are derived from determined job performance requirements and are properly formatted, grouped, and sequenced for effective training.

5-7.5, RECOMMENDATIONS. None.

5-8. ORGANIZATION OF INSTRUCTION USING LESSON PLANS AND OTHER TRAINING GUIDES.

5-8.1. INFO OBJECTIVE. Lesson plans or other training guides provide guidance and structure to ensure the consistent conduct of training activities.

5-8.2. INPO CRITERIA.'

a. Lesson plans for classrocm instruction provide for effective, consistent class presentations.

5-22

b. Lesson plans or equivalent training guides are used for laboratory train-ing, on-the-job training (0JT), and simulator training and include cri-teria for evaluating proper trainee performance.

c. . Lesson plans and other training materials are developed or modified using learning objectives derived from job performance requirements.

5-8.3. CRITERIA EVALUATION. The paragraphs which follow comment on the above INPO criteria as applicable.

5-8.3.1. Classroom Lesson Plans Lesson plans for classroom instruction are devel-oped in accordance with GPUN Training and Education. Department procedure Lesson Plans, 6200-ADM-2682.07. Among other things, this directive states the following:

a. Documents published prior to the effective date of this procedure (4/1/85) shall be brought into compliance at such time as a major revision is neces-sary for other reasons.

b. In the definitions section, lesson plans are described as teaching guides which are composed of the learning objectives, a semi-narrative or outline format of the teaching content, the instructor's activities, and the enabling objectives associated with the content.

c. In the definitions section, the Training Content Record (TCR) is described as a cover sheet for the lesson plan which lists the learning objectives for the course and provides spaces for approval and concurrence signa-tures. In addition the TCR shall list the references used to prepare the lesson plan and indicate the estimated teaching time.

d. The TCR has two locations for approvsl signature. The top location signi-fies approval of the learning objectives. If permitted by the applicable program description, this approval allows use of the TCR with a prelimi-nary lesson plan section as though it was approved for classroom instruc-tion. The bottom signature location signifies approval _ of the entire f lesson plan. l

c. The procedure section of the directive states that the TCR shall contain the learning objectives for the subject lesson module and that the TCR should also contain the references and duration of the lesson.

f. The lesson plan shall be written in one of three formats: Vertical, two column or three column.

5-21

3 Regardless of the format use, the content part of all lesson plans shall contain:

1. Introduction section which describes the purpose of the ' lesson and should show its relevance or connection with previous lessons or I

learning or experience. l

2. A presentation section which contains the skill and knowledge con- I tent.

3. A summary section which summarizes the key points of the lesson.

h. Further - guidance on lesson plan writing can be found in the Lesson Plan Guideline which is part of the T & E Dept. Program Development Manual.

Lesson' plans are also mentioned in the directive Training and Education Dept. Train-ing System Development Process (TSD), 6200-ADM-2682.01 as follows:

a.. Lesson plans are teaching guides which are composed of the terminal and l; major enabling objectives, a semi-narrative or outline format, and the instructor's activities.

b. The two-column lesson plan is the preferred format for the T& E Dept.

Lesson plans and any use of other lesson plan formats shall first be approved by the Manager of Site Training.

Lesson plans are also discussed in the directive Program Evaluation, 6200-ADM-2682.11 which, as one evaluation criteria, asks if lesson plans contain clearly defined and accessible terminal performance objectives and enabling objectives.

Consideration of the above leads to the following comments about the lesson plan procedure 6200-ADM-2682.07 as it pertains to licensed operator training:

a. The terms enabling objectives, behavioral learning objectives, and learn-ing objectives are all used when referring to the same type of objective.

In other procedures concerning lesson plans, the term terminal objectives

is also used when referring to lesson plan objectives. The procedure is inconsistent when stating that objectives refer to a course and also to a.

lesson.

b. The procedure is inconsistent when stating that the TCR shall and should list the references and estimated teaching time.

c. The procedure includes probably unnecessary information on alternate lesson plan formats when the two-column fcreat is preferred and other for-mats require Manager of Training approval.

5-24 1

, .w. ,- ~ .-,pa,...,, _,,...n.,, , , ,-g-u , w,,, . .n, = , , , , ,.,.,mn ,,w., a--, r,,, , , , , , , , , , , , . - ,-e-, c ,

d. The procedure contains considerable guidance on lesson plan writing yet also refers to another document for still more guidance.

e. Discussion of the top approval signature location on the TCR is confusing when it is'used with a preliminary lesson plan.

f. The procedure contains no mention of correlating tasks identified on the JTA matrix with lesson plan objectives, or incorporating tasks associated with the system which are addressed under the emergency / abnormal opera-tions. category or evaluating objectives listed in the OPM for inclusion in the lesson plan. These items are among those included in the Lesson Plan Revision Checkoff Sheet,

g. The procedure is important enough to require compliance by a specified date and not just when a major revision is made to a lesson plan for other reasons. The result of present policy is that lesson plans of 1984 vin-tage on fundamentals may never be brought into compliance because major revisions are not needed.

The - following lesson plans used in licensed operator training were reviewed with I' comments as noted:

Approval Title Rev Date Comments

.123 Rates of TCR does not include Nuclear Reactions 1 5/85 references & lesson duration; lesson plan con-tains only presentation material - no questions or instructor activities.

Summary consists only of repeating the objectives.

.036 Fission Neutrons 1 4/86 Same comments on TCR; includes good instructor notes

.179 Subcritical Multiplication 2 5/85 Same comments on TCR

.055 ICS 1 8/85 Same comments on TCR; lesson plan is very com-plete with some instructor )

notes; summary is to review objectives covered in class. ,

1 I

l 5-25 l l

, - - - - , , ,-,----,,-n,,-,,-r- , , , , - , - - - , , ,,---,.-,-n-,-------------- . , , , - - , , , - . - - - -

4 Approval Title Rev Date Comments

.069 Makeup and Purification System 7 8/86 TCR relates JTA task number to objectives and contains all required information; lesson plan includes no instructor

.080 Non-Nuclear Instrumentation. 2 7/86 TCR contains required information; lesson plan includes excellent instructor notes.

.017 Vital AC/DC Distribution System 1 12/85 TCR contains required information;.. instructor notes include use of training aids and refer-ences to objectives.

.124 Reactivity Coefficients 0 5/85 TCR is approved for objectives but final approval is not signed off.

5-8.3.2. OJT and Simulator Lesson Plans. The directive On-The-Job Training 6200-ADM-2605.02 states that the user and/or functional department shall develop checklists / qualification cards which, as appropriate, include identificat' ion of the following:

a. Task,-system component and/or procedure

b. Instructional setting

c. Specific knowledge and skill items to be evaluated

d. Trainee performance measures / standards

e. Evaluation results and date of evaluation Appendix A of the replacement operator training program description is the OJT checklist. This contains the tasks to be accomplished during the two phases of OJT which follow the two classroom phases. The tasks and performance standards are grouped with related tasks in 23 sections.

Each task is designated by number to permit tracking and cross reference to class-room teaching objectives. Also included with each task is the designation of an 5-26

applicable procedure, the level of performance for the task- (perform, simulate or discuss) and the designation of the guidelines to be used as a standard when admin-istering the 0JT checkout. Each task or related task contain blanks for the stu-dent's name and task examiner signature and date when the task is satisfactorily performed.

The trainee is given an oral exam on each of the 23 task. groups. The questions asked and results of the oral evaluation are documented on a sumary sheet.

The individual task and task group examiners are licensed operators assigned to the plant shifts and designated to perform these checkouts.

Appendix B of the program description contains OJT tasks from the auxiliary operator training program which are performed by direct hire replacement operator trainees.

These personnel have not completed auxiliary operator training prior to being desig-nated as replacement operator trainees. Related tasks and performance standards are grouped as in Appendix A description. Task examiner checkout and oral examinations are performed as in the Appendix A OJT program.

Training programs on the TMI-1 full scale replica simulator are completing develop-ment at this time. Acceptance testing has been completed and deficiency reports are being corrected. The simulator was first used for licensed operator requalification classes in mid-December 1986. In connection with bringing the TMI-1 simulator on line, training on the PSI simulator at Lynchburg VA has been discontinued and the role of the BPT simulator in training is being redefined.

Drill guides contain learning objective in the standard TCR format for separate SRO, CR0 and STA objectives. The task number which corresponds to the objection is included. The drill guide format provides a general description of the drill, pre-requisite competency levels, reference material, initialization conditions, briefing information, exercise preview and a listed sequence of events. The sequence of i

events is presented in the two column lesson plan format. Instructor notes and actions are in one column with the expectad student response in the adjacent column.

Instructor questions and their answers are provided at various points in the exercise / instruction. A termination point and an exercise critique form accompany the drill guide.

5-27

The simulator course description, the drill guides and the draft procedure Develop-ment and Conduct of Trainee Evaluations and Examinations Using the Replica Simulator provide criteria for evaluating trainee performance.

5-8.3.3. Performance-Based Lesson Plans. The Reactor Operator TMI-1 Task List designates each task by number and indicates the training locations (s) for the task' as classroom, OJT, and/or simulator. Classroom and simulator lesson plans now being developed or revised associate task numbers with learning objectives listed in the Training Content Record. Tasks contained in the OJT Appendix are keyed to the task list which is derived from job performance requirements.

5-d.4. CONCLUSIONS. Lesson plans, OJT checklists and simulator drill guides are well prepared for instruction. They are based on a plant specific task list and provide criteria for evaluating trainee performance.

5-8.5. RECOMMENDATIONS.

a. Rewrite the procedure 6200-ADM-2682.07 Lesson Plans, to make it a current, consistent and complete document on lesson plan writing. Ensure that no other guidance and directions on lesson plan preparation and usage con-flicts with this procedure.

5-9. CONDUCT OF CLASSROOM AND INDIVIDUALIZED INSTRUCTION AND TRAINEE EVALUATION.

5-9.1. INPO OBJECTIVE. Classroom and individualized instruction is effectively presented, and trainee performance is routinely and consistently evaluated.

5-9.2. INPO CRITERIA.

a. Training is implemented as outlined by approved training materials and is well-organized and current.

b. Training activities encourage direct trainee participation in the learning process,

c. Instructors prepare adequately to ensure effective and consistent de-livery.

d. The instructor uses instructional techniques appropriate to the lesson content and learning objectives. .

I

e. When individualized instruction is used, either the training materials -

contain the information to be learned or referenced texts are readily available. l l

5-28 j i

l i

1 l

f. Trainee mastery of learning objectives is evaluated regularly using writ- l ten and/or oral examinations and quizzes.  !

g. Written and oral examinations and quizzes are administered and graded in a consistent manner.

h. Acceptance criteria to be used during the administration of oral examina-tions are defined in advance of the examination. l l

5-9.3. CRITERIA EVALUATION. The paragraphs which follow comment on the above INPO criteria as applicable.

5-9.3.1. Conduct of Trainina. Replacement operator classroom instruction was as-sessed by reviewing documentation; interviewing instructors, the Supervisor, Licen-sed Operator Training, Operator Training Manager and trainees, and observing four periods of classroom instruction, study periods and a weekly exam during the period November 17-21, 1986. The RO class in training is Group #7 with eight trainees which started the classroom fundamentals phase November 10, 1986.

There was no licensed operator requalification classroom training in progress during the November 17-21 period. This training was assessed by reviewing documentation and interviewing training staff and operators.

Replacement operator training is implemented as outlined in the approved program description, 6211-PGD-2611.04 Revision 5. The program description includes the following guidance for classroom instruction:

a. Classroom training is covered in:

Phase 1 - Fundamentals classroom training and OJT Phase 2 - Systems and integrated plant classroom training and OJT.

Simulatsr training and audit exams are also included in this phase.

b. The primary purpose of classroom training is to integrate previous system knowledge with overall plant operation and operating procedures. Subjects to be considered for classroom training are listed and points to cover in system lesson presentations are also set forth.

c. Lectures are to be presented by qualified instructors using approved lesson plans.

5-29

l

.d. Portions of classroom ' training requiring self-study shall be monitored by a qualified instructor who shall be available for individual consultation.

e. Candidates who miss more than one consecutive week of training ar2 re-viewed to determine if they-can catch up with the class.

f.- Written exams should be administered by - the Training Department at least on a weekly -basis during the classroom phase. Questions shall cover'the material presented in the classroom, specified for self study, and identi-fied on the task sheets.

3 A passing score of 80% is mandatory for all written examinations. A grade of less than 80% results in counseling and 's re-exam. Candidates who fail the re-exam are evaluated to determine corrective action.

h. Training attendance forms shall be completed and submitted to the Admin-istrative Section for each classroom lecture or lesson by the instructor who presented the material.

1. Direction is provided on maintenance of documentation.

j. The processes for evaluation of training, changing program content and ,

approving the program are described.

A memo was sent to the Group #7 CR0 candidates 10/28/86 which suggestea Operator Plant Manual material to be reviewed prior to start of classroom training 11/10/86.

It provided an outline of the first ten weeks of training and a draf t schedule of the four weeks classroom fundamentals phase.

i The classroom fundamentals phase was originally scheduled to be four weeks in dura-4 tion. After the first weekly exam, the trainees voiced frustrations over the rapid pace of the work and their difficulty with the exam. The training staff evaluated the situation and the classroom phase was extended one week. The additional time was used for more study and review periods and for instruction in heat transfer and fluid dynamics. The original four-week schedule included three holidays which might have raised questions on the feasibility of holding to four weeks and still cover the required material. Classroom instruction periods begin at 7:00 AM and end at 3:00 PM. Periods run consecutively with no scheduled breaks or lunch periods.

Breaks are actually taken when class and instructor decide. Lunch is eaten during class (This follows the plant practice of eating on the job during company time and is preferred by trainees over an unpaid lunch period which would extend th working day). Instructors used current approved lesson plans for instruction. Handouts to 5-30

>w.-- -

-a , - - . . , . - - - . .-m-en,- - - ,n.- nv.~.., . , , - - , - - . , - - - - ,,,.w- ----.,e,--,,..----, . - - - - vn- - - - ,-- -

,- , . - + - - - , - - - -

l

. l trainess frequently consisted of the Training Content Record showing objectives for l the lesson and copies of viewgraph transparencies to be used in the lesson. (These l

~

I were not punched for 3-ring binders and classes usually waited a few minutes vhile i trainees punched the papers). Handouts did not provide outlines for note-taking and quality of notes varied from excellent to non-existent, depending on the motivation

! and capability of the trainee. The schedule provided two one-hour study periods l prior to the exas on the first week's work.

Study periods were mainly group discussions. Instructors were not present during '

these periods but were available for consultation. Homework is not assigned as this would result in paid overtime.

1 5-9.3.2. Trainee Participation. Training activities observed enconraged direct trainee participation in the training process by use of instructor questions, invit-ing trainee questions, using trainee experiences to contribute to class discussion, and assigning problems for group solution during class.

5-9.3.3. Instructor Preparation. Instructors observed ensured effective and con-I sistent delivery by use of approved lesson plans and checking that TCR objectives were covered. They were prepared to present their lessons effectively and their -

experience and knowledge were used to advantage in fielding trainees' questions.

i In training programs of this nature there is a question of what qualifications an instructor needs to best present theoretical material such as nuclear physics, thermodynamics, heat transfer, fluid dynamics, gas laws, and electrical principles.

In some cases, the non-degreed instructors observed fell back to thumb rules and over-simplifications when explaining basic principles and underlying theorems. On the other hand, the hands-on experience of the instructors observed was very helpful in giving practical examples of the theory in action which the trainees could under-stand and relate to.

i 1

5-9.3.4. Instructional Techniques. Instructors observed used good lecture prac-tices in presenting material during classroom lessons. The material was introduced with objectives of the lesson and how the subject related to previous and future lessons. Training aids such as handouts, problem sets, viewgraph transparencies and actual equipment parts were used appropriately during the lessons.

5-31 i-

~>,,,:mw.---,--,w-.s,e,,-, m, ,n_,,,+,m,, ,-.--,-,..-.mn.v.>,,.-., - nmn ,, , , - , , . , , - - , - - - - - ,. -,,,,,..,,-,--.,.---,.---,,,-,-,,-w- --------n

5-9'.3.5. Individualized Instruction. Individualized instruction is used infre-quently for replacement operator training to review a specific subject such as math or physics in which a trainee is deficient. In these cases the training materials contain the subject matter to be learned.

5-9.3.6. Learnins objective Mastery. Trainee mastery of learning objectives is evaluated regularly during the classroom phase by written examinations administered' at least weekly by the Training Department. This requirement is set forth in the replacement operator training program description. (It is noted, however, that the five-week long Group #7 Phase I classroom session included two-hour examinations during weeks two and four and a four-hour comprehensive examination at the end of the classroom phase.) Questions on these examinations shall cover the material pre-sented in the classroom, specified for self study, and identified on the task sheets. A passing score of 80% is mandatory for all written examinations. A grade of less than 80% results in counseling by an instructor and a re-exam within two weeks. If an individual fails the second exas, the Plant Operations Director and Operator Training Manager will evaluate the situation and decide on corrective action.

A comprehensive examination and an oral examination is administered at the conclu-sion of the training program. An 80% overall average and a 70% on individual sec-tions is required for satisfactory completion of the written phase. The oral exam-ination consists of a "valk through" administered by a licensed or certified oper-ator Instructor or licensed operator designated by the Supervisor, Licensed Operator Training and a board composed of the Supervisor, Licensed Operator Training and the Plant Operations Director or his designated representative. A " pass" grade is required for the oral exas substantiated by Oral Examination Summary Sheets as documentation. These final exams are NRC style exams to determine eligibility of the candidate to sit for the NRC exams.

The licensed operator requalification program pre-planned lecture series typically involves up to 240 contact hours of instruction divided among program topics and scheduled in one week periods throughout the year. After each week of lectures all trainees are required to take a written closed-book quiz covering the lecture topics. All lecture topics covered during the training session are represented by questions on the quiz. The minimum passing grade is 80%. Trainees who do not meet this requirement are assigned to complete remedial training within eight weeks.

5-32

5-9.3.7 Examination Gradins. Written exams are administered in accordance with Training and Education Department procedure Control of Examinations, 6200-ADM-2600.01. This is a detailed directive covering all aspects of how to administer different categories of exams to ensure consistent and secure practices.

The replacement operator classroom phase weekly examination was observed 11/18/86.

The proctor complied with all provisions of the directive on control of examina-tions. His performance in this regard was evaluated by the Operator Training Manager using a checklist contained in the directive Instructor Evaluation, 6200-ADM-2607.01.

Oral exams or checkouts are used extensively during the OJT Phase. Oral checkouts for a specific task are administered by designated task examiners using guidelines contained in Appendix A to the replacement operator training program description.

Satisfactory completion of a checkout is documented by examiner's signature on the task sheet.

Final verification of a section of the task sheets is accomplished by an oral exam administered by a section examiner and documented on an Oral Examination Summary Sheet. When filled in, this includes a summary of questions asked and an indication of whether the trainee's answer passed or failed the questien. This sheet also in-4 cludes an overall evaluation (pass / fail), weak areas and further action re. quired (if any).

The following exams were retrieved on microfiche for Group #6 Replacement CR0 trainees:

a. Five weekly exams covering fundamentals and systems.

b. Theory comprehensive exam and reexam.

c. Systems comprehensive exam and makeup exam.

d. Mock NRC written exam and reexam. l

e. Fundamentals oral board exam.

f. Upgrade oral exam.

g. OJT oral spot checks.

h. Final comprehensive oral exam.

i. Mock NRC oral exam.

e 5-33

The following comments result from this review:

a. There were no problems in locating and retrieving the data. Some hand-writing on exams was indistinct and difficult to read,

b. Grading of written exams was consistent with the key.

c. Oral exam results were written on summary sheets with a brief description or phrase on each question and an indication of the pass, marginal or fail evaluation of the answer. No criteria were indicated on what constituted a satisfactory answer to a question.

d. The exam administered at the end of the first week of fundamentals class-room training covered 11 lessons and included 20 questions with a time limit of 1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />.

(1) The weekly exam had questions on four of the objectives and the fifth objective was covered in the theory comprehensive exam.

(2) The lesson plan on Flux Distribution and Reactor Control also covered during this week, included about 14 separate obj ectives (The plan actually listed more objectives but several were close enough in subject matter to be considered one objective). The weekly exam included questions of four of the objectives and a fif th objective was covered in the theory comprehensive exam.

e. The systems comprehensive exam questions were practically all recall type such as "How is . . ., What is . . ., Who reports . . ,, List . . .".

Trainee answer papers for the group #7 replacement operator exam administered after the first week of fundamentals classroom instruction were reviewed with the follow-ing comments:

a. Units were frequently missing or incorrect.

b. Several answers were incomplete or confused on defining " Fermi Age".

c. Xenon post-shutdown behavior was graphed showing increases above or below a zero axis on different papers.

d. Some answers considered Samarium 149 to be unstable with a long half life instead of considering it stable as is more customary.

5-9.3.8. Oral Examinations. Acceptance criteria to be used during the administra-tion of oral examinations are not defined objectively in advance of the examination.

The path an oral examination takes cannot be defined exactly in advance because a student's reactions and responses to questions can lead to additional or different 5-34

_ _ ._ ~ - _ _ _ _ _ _ _ . _ _ _ _ _ _ _ . . . _ _ . _ - , _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ . - _ _ _ _ _

questions which should be pursued. Evaluating an oral examination requires judge-ment and subjectivity. However, the effectiveness of an oral examination can be improved by listing in advance key questions to ask in order to cover vital areas and what constitutes minimum satisfactory answers. This is particularly important with a board format to ensure adequate coverage overall and eliminate unnecessary redundacy.-

5-9.4. CONCLUSIONS. Classroom instruction is effectively presented and trainees

.are regularly evaluated in written and oral examinations on their mastery of the specified learning objectives.

5-9.5. RECOMMENDATIONS.

a. Consider incorporating the following classroom instruction organizational practices:

1. Schedule break times.

2. Schedule 20-30 minute meal periods not during class time.

3. Establish a ratio of study time to instruction time.

4. Provide a place for individual study if desired.

5. Provide optional homework for those who want to do it on their own time.

6. Provide handouts to facilitate note taking..

7. Provide punched copies of OPM chapcers and handouts to help trainees develop their own training texts,

b. To strengthen the replacement operator classroom theoretical instruction review site resources of licensed operators who are degreed engineers and select well-qualified individuals for instructor training and assignment to teach some theoretical subjects.

c. Continue to upgrade the quality of exams by appropriately covering lesson objectives in exam questions, by retiring questions that become familiar to trainees and by more emphasis on questions that require analysis in-stead of memorization. Improve grading of exams by carefully evaluating the process to arrive at an answer as well as the answer itself. Require correct units as part of the grading criteria.

d. Increase the objectivity of oral examinations by defining in advance a minimum number of questions to cover the examination topics and the essen-tial elements of correct answers to the questions.

5-35 1

, . ~ - - - . _ _ ~ . . - - _ . - - . -. .

5-10. ~ CONDUCT OF IN-PLANT TRAINING AND TRAINEE EVALUATION.

5-10.1. INPO OBJECTIVE. In-plant training or on-the-job training (0JT) is offac-tively presented, and trainee performance is evaluated consistently.

~

5-10.2. INPO CRITERIA.

.a. In-plant training is delivered using well-organised and current training materials, b'. Designated personnel who are instructed in program standards and methods conduet.in-plant training.

c. When the actusi task cannot be performed- but is simulated or walked-through, the conditions of task performance, references, tools, and equipment reflect the actual task to the extent possible.

! d. Performance evaluations use established criteria.

1 5-10.3. CRITERIA EVALUATION. The paragraphs which follow comment on the above INPO criteria as applicable.

i 5-10.3.1. Training ' Delivery. In-plant training is based on the CR0 Task List, 6211-ADM-2611.06, Revision 2, dated 11/3/86. The Replacement Operator Training Pro-gram Description, 6211-PGD-2611.04, Revision 5 groups OJT tasks for training  :

together with associated performance standards. The features of this training plan

are:

a. Defines the qualifications required to give OJT checkouts in specific f

areas.

4 l b. Includes specific instructions to subsection task examiners and final verification task examiners on the conduct of OJT checkouts.

I 5-10.3.2. Examiner Qualification. The replacement operator training prograa description defines task examiner as a person designated by qualification to give OJT checkouts. This is fc11oved by a list which indicates by qualification of job title in what areas a person is authorized to give OJT checkouts. The checkoff sheets designate for each OJT checkout signature the required qualification of the task examiner. Individual task examiners are required to be at least R0 qualified while final verification checkouts require a task examiner who is Shif t Supervisor l or Shift Foreman qualified.

5-36 -

f

The directive On-The-Job-Training 6200-ADM-2605.02, contains a section on qualifica-tion of evaluators / trainers. Among other things this requires that:

a. The Training -and Education Department establish and implement an OJT evaluator / trainer training program to train individuals how to conduct checkouts. Specific requirements for this program are included.

b. The user department identify who is qualified to be an evaluater/ trainer and assure they receive required training.

This training for OJT evaluators / trainers has not as yet been implemented.

1 5-10.3.3. Task Performance. The level of performance for each OJT task is desig- '

nated with the' task. The levels of performance in order of preference are:

(P) - Perform. The candidate shall perform these tasks under the direct super-vision of a licensed operator. No other level of performance can be used as'a substitute. Perform tasks may be accomplished in the plant or in the replica simulator.

(S) - Simulate. The candidate shall simulate these tasks under the direct supervision of a licensed operator. The simulation should be done as auch as l

possible in the vicinity of the actual component or equipment. Hands-on train-ing, without actual manipulation of valves, breakers, etc. is encouraged.

l " Perform" can be used as a substitute.

(D) - Discuss. The task examiner shall examine the candidate on these tasks to l evaluate the candidate's overall knowledge of the task, as compared to the guidelines.

l 5-10.3.4. Performance Evaluation Criteria. OJT performance evaluations use the

( guidelines and standards of performance contained in the program description. These

! constitute the established criteria for evaluations. Additional criteria for evaluations is contained in instructions to task examiners.

5-10.4. CONCLUSIONS. In-plant training and qualification are effectively performed and trainees are consistently evaluated by designated qualified examiners using criteria established for each task.

5-37

4,1 5-10.9- RECOMMENDATIONS.

a. Improve the skills of OJT evalu: tors / trainers in the conduct'of checkouts, and final verifications by implementing training and qualification of OJT evaluators / trainers specified in T&E Department Procedure 6200-ADM-2605.02.

5-11. CONDUCT OF SIMULATOR TRAINING AND TRAINEE EVALUATION.

5-11.1. INPO OBJECTIVE. Simulator -training is effectively presented, and trainee performance is evaluated consistently.

5-11.2. INPO CRITERIA.

a. An appropriate . simulator is used for hands-on training, to demonstrate operational characteristics, and for recognition and control of normal, abnormal, and emergency plant conditions. Differences between the simu-

, lator and the plant are accommodated in the training sessions.

b. The training program content is implemented as outlined by approved train-ing materials and is well-organized and current. Requests for contracted training should specify the required objectives and content.

c. Instructors prepare adequately for simulator sessions to ensure effective and consistent training. Requests for contracted training should require vendor instructors to be familiar with differences between the referenced plant and trainees' home plant.

d. The instructor uses instructional techniques appropriate to the situation.

e. Individual trainee and team performance are evaluated regularly against established learning objectives using appropriate evaluation methods and performance criteria.

5-11.3. CRITERIA EVALUATION. *The paragraphs which follow comment on the above INPO

criteria as applicable.

5-11.3.1. Simulator Use. The full scale replica simulator located at Bldg. 2 of the Training Center is completed except for some deficiency reports now being corrected by the contractor. License Operator requalification training commenced on the simulator in mid-December 1986. In connection with the introduction of this significant new training capability, contracted training at the PSI replica simu-lator at Lynchburg, VA has been terminated and the role of the BPT simulator in 5-38 r

, -- .,,.___.._.,_,_____..__....._m,.m._, , _ , _ _ _ _ . . . . _ _ _ . . _ _ _ _ _ . . _ _ _ _ _ _ _ _ _ _ , - _ _ . _ _ . , _ . _ _ _ _ _ _ _ _ _ _ . _

training is being redefined. The replica simulator characteristics are modeled on the Oconee Plant as modified by the contractor and the sof tware package to closely replicate TMI-1 characteristics. The layout of simulator room panels, equipment, instruments, controls, indicators and alarms provide close physical fidelity to the TMI-1 control room.

The Simulator Development and Training Section has completed a several year major project to manage the design, construction, installation, and test of the on-site replica simulator. The correction of deficiencies revealed in start-up will complete this phase. The effort will now shif t status from a largely autonomous development project to being an integral and vital element of the licensed operator training effort. Associated with this transition is a redefined role for the BPT simulator. The integration into training and the organization of the simulator sec-tion have not been fully decided at this time.

5-11.3.2. Trainina Material. CR0 simulator training courses have been written for reactor startup certification, plant operations Level I, plant operations Level II and systems familiarization. Drill guides have been written to support these courses and are being used for the first time for instrr-tion. The course descrip-tions are contained in the replacement operator training program description. Draft procedures are in the final stages of approval covering Conduct of Replica Simulator Training and Development and Conduct of Trainee Evaluations using the Replica Simulator.

5-11.3.3. Training Evaluation. Simulator training sessions were observed on January 28 and 29. During this period the Unit-1 replacement operators received system training on the simulator, the license operators received requalification training and Replacement SRO training was also performed.

System training sessions for the replacement operators (RO) consisted of familiari-zation with the location of controls and indicators, the identification of alarms and interlocks and the performance of system manipulation following instructor demonstration. The instructions and demonstrations were well planned and presented and followed the lesson plan. The instructor asked many questions which kept the trainees alert and tested their comprehension. The discussions and questions were 5-39

1 often related to prior classroom instruction and earlier gained knowledge. Manipu-lations and indication were related to what was occurring in the system in the l plant.

l The licensed operator requalification training consisted of cyclical training for a control room shif t crew. The regular shif t crew of reactor operators, foreman,-

shift supervisor and shift technical advisor participated as a team in their normal positions. Similator sessions consisted of unannounced drills and operations and exams conducted by simulator instructore and plant managers. The VP and Director

- TMI-1 observed some of the simulator sessions and participated in the operational examination. During one of these sessions he expressed the opinion that the TMI-1 replica simulator training and fidelity was 100% better than that provided previously by the PSI simulator.

Unit 1 SRO, replacement simulator training was provided during evening sessions. It consisted of malfunctions and casualties taken from the SRO Drill Guide Index cover-

- ing such areas as electrical distribution, feedwater casualties, loss of vacuum with OTSC tube leak, and small break Loca.

The control room personnel requalification training and the upgrade SRO simulator training and examinations were performed using the prepared exercise guides and the sequence of events and scenarios therein. Initial conditions for the scenario were clearly presented to the trainees as well as general instructions on communicating and role playing. Prior to the start of the drill / examination, the trainees were provided adequate time to check the boards and review the simulator status. During the exercise / examination the instructors / examiners remained in proximity to the trainees and observed the actions taken for grading and critique purposes. Scenario events examination sheets were used to grade each event in the scenario.

Control room personnel during operational examinations were graded ih .the following areas as appropriate to their poritions control board awareness, event diagnosis, immediate actions, subsequent actions, console manipulations, use of procedures /-

technical specifications / reference data, and team skills. Each of these grading j

categories contains a list of items for ease of check off by the evaluators. A l separate instructor / evaluator was used for each control room position. A discussion l

and critique followed each drill during which trainee opinion and reaction were solicited.

5-40 i

i k

1

--.-~-.,,---m..... . . . . _ __. __,._--,. - -_ _ m m _.,m. -.,,....m_-, _ . - . - . _ , , _ . - _ _ - , - - - - .

The simulator exercise guides used for instruction and for control room team perfor-mance evaluations are struccured similar to the example contained in INPO Guideline 86-026 Guideline For Simulator Training. The TMI-I simulator performance evaluation forms also closely resemble the INPO Guideline forns.

During all training and exercises observed the simulator performed well and the ses-sions went smoothly. The instructors were well prepared and surprisingly proficient considering the newness of the simulator for training. Evolutions were conducted using plant operating procedures, administrative procedures, technical specifica-tions and normal plant operating philosophy. The demeanor and atmosphere of an operating control room were observed and enforced at all times.

5-11.4. CONCLUSIONS. An appropriate full-scale replica simulator is in use on-site for operator training. Well organized and current simulator training materials are being used by a well-qualified simulator training staff to provide the required operator training.

5-11.5. RECOMMENDATIONS. In view of the principal role of the replica simulator in licensed operator training, consider the integration of the simulator section into the operator training organization.

5-12. SYSTEMATIC EVALUATION OF TRAINING EFFECTIVENESS.

5-12.1. INPO OBJECTIVE. A systematic evaluation of training effectiveness and its relation to on-the-job performance is used to ensure that the training program conveys all required skills and kr.owledges.

5-12.2. INPO CRITERIA.

a. Program evaluations are conducted on a regular basis by qualified indivi-duals.

b. Training delivery is monitored and evaluated with regard to instruction, materials, and instructor performance.

c. Feedback from trainee performance during training is used to evaluate and refine the training program.

d. Feedback from trainee performance, after the trainee has assumed the duties for which he was trained, in used to evaluate and refine the training program.

5-41

e. Change actions (e.g., procedure changes, industry events, equipment i changes)-are monitored and evaluated for their applicability to the development or modification of training programs and are incorporated in a timely manner.

f. Improvements and changes to training are initiated and tracked to correct training deficiencies and performance problems.

5-12.3. CRITERIA EVALUATION. The paragraphs which follow comment on the above INPO criteria as appropriate.

5-12.3.1. Program Evaluations. Program Evaluation 6200-ADM-2682.11, dated 4/15/86, implements the e<aluation of training programs to assess the overall quality of program content and training process. This procedure suggests that each program should be evaluated no less than every four years.

A companion directive, Course Evaluation Process, 6200-ADM-2682.12, dated 4/16/86 standardizes the evaluation of courses.

Discussion with licensed operator training personnel indicated that replacement operator and licensed operator requalification training programs have not as yet been evaluated using these procedures.

The replacement operator training program description provides for several evalua-tion procedures:

a. A technical content review prior to the start of each replacement operator class. This has been accomplished.

1 b. An ongoing review of the CR0 task list by the Training Department with review and approval by the plant opt. rations department. This has been l

accomplished.

c. Course evaluation review as required by 6200-ADM-2682.12. New revision issued 12/12/86 based on course and program reviews.

d. Program evaluation as required by 6200-ADM-2682.11. New revision issued 12/12/86 based on course and program review.

e. A supervisory performance evaluation once on-the-job shall be conducted approximately six months after the candidates have received their 11-censes. The evaluation shall be conducted utilizing Exhibit 2 of 6200-l 4

5-42

- - ~ ~ , . - + , - - - - - . _ _ _ , . , , , -.,,---r------ .---,-m --,1-,.---- , , . , - - , - - , - - , - - - , - - - ~ - - - - - - - - - - , , - - - ~ - -.

i i

ADM-2682.10' to evaluate training related_ performance. This is being r.ccomplished. A review of recommendations and resolutions resulting from 1 evaluations of CR0s and SR0s completed 11/6/86 indicated an effective process exists for refining the training program.

The licensed requalification program description - requires an annual review .and i evaluation to be conducted by the Supervisor Licensed Operator Training. Several

! suggested areas to be encompassed by the review are described. This review is being _

accomplished.

A continuing evaluation and refining of this program takes place through monthly operations / training meetings which address the preplanned lecture series and 'other elements of the requalification program.

4 "Once back on-the-job" evaluations of licensed operators are 'also carried out by

} supervisory personnel to provide feedback ' on effectiveness of requalification

. training.

. 5-12.3.2. Trainina Delivery Evaluation. Evaluation of training delivery is set forth in the directive Instructor Evaluation, 6200-ADM-2607.01. This procedure describes responsibilities, requirements, and the process for evaluation. Licensed operator instructor evaluations were reviewed and writeups contained constructive i comments for improving performance. The requirements for frequency of evaluation -

are being met.

l

{ 5-12.3.3. Trainee Feedback. An example of training organization responsiveness to j feedback from trainee performance occurred during the site visit. The results of j the first weekly exam during the Group !7 replacement operater fundamentals class-

, room phase indicated a lack of understanding of reactor theory and general frustra-

tion of trainees with the rapid pace of instruction. This was considered and the classroom phase was extended one week to provide more study and review time.

t i

!' The results of examinations and feedback from the licensed operator requalification 1 program are regularly used to evaluate and refine the training program.

l 4

t i

5-43 l

e t

-- . , - . _ . _ - , _ - , _ . , . . - _ . _ _ _ . - _ _ . . . . _ , - - . . _ . .-_m.-- - - - -

5-12.3.4. On-the-Job Feedback. Feedback from trainee performance after the trainee has assumed the duties for which he/she was trained is obtained in accordance with the procedure Trainee Evaluation ' - Once Back on-the-job 6200-ADM-2682.10. This feedback is in the form of supervisory personnel evaluations which are used to refine replacement operator and licensed operator requalification training. Review of recent reports indicates this process is working effectively.

. 5-12.3.5. Channe Action Feedback. Change actions are routed through the licensed operator training section for required reading by instructors and with a cover sheet which requires review for applicability to the development or modification of train-ing programs. A review of the required reading program indicated good compliance.

It was noted that an instructor licensed in Unit 2 who teaches Unit i fundamentals is on the Unit 2 required reading list.

l 5-12.3.6. Trainina Channes. Improvements and changes to training are initiated and tracked to correct training deficiencies and performance problems. Examples of this happening are contained in the reports of "once back on-the-job" evaluations. Rec-

ommendations are considered and where appropriate corrective action is taken to a improve training.

5-12.4. ASLB REACTOR OPERATOR TRAINING FEEDBACK.

a. The order of the ASLB Partial Initial Decision on the Remanded Issue of Licensed-Operator Training at TMI-1, 5/3/85 stated that "The Licensee shall implement a plan for the evaluation af ter training of the perfor-

- mance of its trained reactor operators and senior reactor operators in the job setting, under both normal and abnormal operation for revision of the TMI-1 licensed operator training program. . ."

This order was implemented by the requirement in the replacement operator program description that a supervisory performance evaluation once on the ,

job be conducted approximately six months after the candidates have re-

! ceived their licenses. In the case of licensed operator requalification training, the program description contains the requirement for annual on-the-job evaluations by supervisory personnel. Both program descriptions specify use of an evaluation form and procedures contained in the direc-tive Trainee Evaluation, Once Back On The Job.

i i 5-44 i

-,m -. ~, - , , , . _.,,-,,-,-w.,--,n.-, ,, ._,,_e.,.,..,,m,,,,- ,--,,----_-n n._ , - , , . -.. .n.,-_.---

I i

"Once back on-the-job" evaluations and an analysis of these evaluations were completed.in November 1986 for reactor operators who were licensed in March 1986. The recommendations and training program changes resulting from the survey were described in a Supervisor,-Licensed Operator Training meno of 11/6/86.

Similar evaluations and analysis of licensed operators once back on-the-job af ter requalification training were completed in May 1986. A review of recommendations and determination of action are pending at this time.

In addition, there is ongoing dialogue between the Training and Operations Departments to review and evaluate the replacement operator and licensed operator training programs as set forth, in the directive Technical Con-tent Review and Interface Process 6200-ADM-2682.03.

5-12.5. CONCLUSIONS. Training delivery is routinely monitored and training program feedback is effectively obtained from trainees during instruction, from post presen-tation review by the instructional staff and by on-the-job incumbent perform nce evaluations by plant supervisors.

5-12.6. RECOMMENDATIONS.

a. Review training program evaluation requirements for possible consolidation into fewer directives.

b. Include a former training program participant in the membership of the review group which examines the training program content at the end of each program presentation.

c. Ensure that in future training program presentations that procedurally 4

specified trainee evaluations / critiques are accomplished once back on-the-t job.

L i

5-45

-r- - - .,,----,nm --on.r-,., -

e , a-,-, , . ~ , - , y---wnn--p,ww-,y,- ..n--n-- . , --wm m mm ,,-------,,-,,-,e , ~ . , -,-,w s -w---w-,--, ---

SECTION 6 SENIOR REACTOR OPERATOR TRAINING PROGRAM 6-1. SENIOR REACTOR OPERATOR TRAINING PROGRAM DESCRIPTION.

1 The training programs, 6211-ADM-2611.02, Rev. 4, 7/7/86, for the Senior Reactor Operator (SRO) consist of three parts: Replacement, Direct, and Cyclic. Replacement SRO Training is designed for personnel currently licensed as a Reactor O'perator at

'IMI-1. This program is modified slightly for Bargaining Unit (Union) and Exempt (Non-Union) personnel. Direct SRO Training is designed for individuals holding a degree in engineering or applicable science, with nuclear plant experience (e.g.

staff engineer), but who are not licensed operators. Cyclic Training provides for 4 the continuing training of the SR0s.

6-1.1. REPLACEMENT SRO TRAINING PROGRAM. Replacement SRO Training consists of three elements; classroom, OJT, and simulator training. The program must be completed within nine months of its start. Time devoted to each of the three i elements is not defined in the program description. However, silestones for completion of certain training events within the program are specified by the Supervisor, Licensed Operator Training. The program concludes with the NRC SRO c License Examination.

i l 6-1.1.1. Classroom Training. The scope of the classroom training portion of the I Replacement SRO Training program is determined by the Supervisor, Licensed Operator Training and approved 1.y Plant Operations Director and the Operator Training Man-ager. Classroom tratuing is essentially tailored to the needs of the trainees based on experience and recent training accomplished in the Licensed Operator Requalifica-4 tion Training Program. Technical qualifications for the replacement SR0s are not considered to significantly exceed those required of licensed reactor operators to the extent of requiring a pre-defined sequence of classroom lessons exclusively for I

i the replacement SRO program. Classroom lesson plans used in the SRO program are those currently utilized in the Reactor Operator Training program and are considered .

refresher / review material for the replacement SRO. Topic areas listed in the SRO classroom program and recommended for consideration in specific training tracks include reactor theory, heat transfer and fluid flow, thermodynamics, radiation' i

i

< 6-1 i

t  ;

-.e-, .,m .,s,m--,, ._.,,.cg. ... .+m ,-w w , c c ,,,,w.7,. ,em,,,%o.,#.,wm ,,-n.,~,,,,,,,,,,e,w y.,,-e.-e,,,- w enw.wm,,.- .

_ _ . . . - ~ _ _ _ . _ . _ . - _ _ . __ _ _ _ _ . _ _ _ . . - _ _ _ _.

. control and safety, reactor control, general procedural and emergency topics, and selected plant systems.

Written examinations are administered on a weekly basis, based on lesson plan I

learning objectives. Failure on the examinations will result in counseling by the instructor responsible for the weak areas with remedial study recommendations. A-re-examination must be accomplished within two weeks, failure .of which results in evaluation of overall performance and remedial action by the Plant Operations

! Director and the Operator Training Manager. Topics from the classroom phase of training are also included in the comprehensive written. oral, and simulator examinations given at the conclusion of the program.

t, g 6-1.1.2. On-The-Job Trainina. The OJT phase of Repiscenent SRO Training program consists of two sets of structured qualification sheets. Each sheet has been l developed from one or more tasks identified in the SRO job analysis. The candidate

> SRO aust satisfy a designated task examiner of qualification in each perforssace e objective listed on the sheet. Each performance objective identifies the method i whereby performance will be demonstrated (i.e., perform, simulate, or discuss).

Signature of the designated task examiner certifies satisfactory completion .of the qualification sheet. Final qualification of task sheet sections is accomplished by I

oral checkout by the Shift Supervisor, (or Shift Foreman if designated by the Shift Supervisor) for those personnel assigned to a shift crew. Personnel not assigned to l shift crew may also receive final verification by any licensed / certified SRO 3

approved in writing by the Plant Operations Director. The final oral checkout

! and/or written questioning on each task section is performed using a structured Examination Summary Sheet. In addition to successful qualification in OJT, each SRO ,

j candidate must stand twelve shif t-foreman-under-instruction watches of eight hours each. This evolution is scheduled for accomplishment following successful ceaple-1 tion of at least 90% of the assigned OJT qualifications.

6-1.1.3. Simulator Trainina. Simulator training for all SRO programs is new being performed on the TMI-1 replica simulator located in the training center building 2.

j This simulator was accepted for training in mid-December 1986. All SRO candidates are required to complete the simulator training, courses associated with the SRO

) The TM1-1 Re' plica Simulator Course description is in the final Training program.

, development stages and will be appended to the SRO Training Program description upon 1

6-2 4

I 4

4 completion. The current draft of the course specifies a length of approximately 120 hours0.00139 days <br />0.0333 hours <br />1.984127e-4 weeks <br />4.566e-5 months <br /> of simulator training, supplemented as necessary by classroom and/or OJT training. The course objective, as stated in the draf t course description, is to allow the candidate to apply knowledge gained from classroom training and OJT by providing the opportunity to supervise operation of the unit during normal and off-normal operation and during abnotaal transients. Application of technical specifi-cations and response to emergencies using Emergency Plan guidance are included.

Actual control board manipulations are minimised in favor of supervisor functions based on evaluation and diagnostics.

I The training prograa is structured around 41 drill guides each with appropriate performance objectives. Student evaluations defined in the proposed course

, , description include:

- exercise critiques for each exercise.

- weekly evaluation by the simulator instructor,

- a course final examination comprising a mock NRC examination, and

, - an overall evaluation of the trainee's performance in simulator training

! prepared by the simulator instructor (s).

l

/ Concurrent with the preceding evaluations, the trainee will be certified by the Manager, Plant Operation (or his designee) through a Simulator Operational Examination. The conduct of this simulator examination is addressed in TMI-1 AP 1

1058, Requirements for Certification of Candidates for NRC Operator Licenses and Instructor Certificates.

6-1.1.4. Evaluation. The most recent SRO Replacement training group commenced training in the summer of 1985 and completed training.in March 1986. All candidates s who completed the program successfully passed the NRC examination. With the exception of the results of the NRC examination, assessment of the program in l practica cannot be accomplished since a new set of training program descriptions and a revised task list has been implemented subsequent to completion of the most recent program. Consequently, the preceding discussion is based on the documents currently f defining program content and administration and do not necessarily reflect what

! essentially is history. The new program description, task list, and implementation /

evaluation documents describe a complete and effective program. Evaluation of the success of the upgraded program must await the next presentation.

ti i p, <

6-3 l

l -

i i ,

i

- ,, .-- . - - - - - , - . . . - , ..l-,,,-----_----,, , , , - - ,n. ,._.,n.n- ,n ,,-,. _- - ----,-. _ , , , , -,_n.,,,-,-m.,,,n, - - .

L 6-1.2. DIRECT SRO TRAINING. The Direct SRO Training program is composed of the same elements as that of the Replacement SRO Training program. However, there is

! significantly increased emphasis on classroom topics ' and additional OJi qualifica-tions from the Control Roos Operator (CRO) Training program. The Direct SRO candi-

[

date has eighteen months (versus nine months for the Replacement SRO) to complete

r, -

< 4 the assigned program. In effect, the Direct SRO candidate must meet all require-sents specified for Replacement SRO plus additional requirements determined to be appropriate to the candidate's education and experience. The Supervisor, Licensed Operator Training is responsible for defining the additional requirements with the approval of the Plant Operations Director and the Operator Training Manager.

i 6-1.2.1. Classroom Trainina. Classroom training for the Direct SRO candidate include s the following topic areas defined in Appendix C of the SRO Training Program description.

- Reactor theory Heat transfer and fluid flow

~

- Miscellaneous fundamentals (safety analyses, fuel performance and mechani-cal maneuvering, PT plot, radiation measurement and instrumentation, elec-trical theory)

- Secondary systems

- Primary systems

- Instrumentation and control systems

- Miscellaneous topics (chemistry, technical specifications, NPDES Permit /

PPC Plan)

- Procedures: normal, abnormal, emergency The Supervisor,. Licensed Operator Training assignw specific lessons within each of the foregoing topic areas for accomplishment by either self-study or lecture.

Trainee evaluation within each topic is accomplished by weekly written examination for lecture lessons, or a validation examination, (either written or oral) for self-study lessons. All subject areas listed above are evaluated for the Direct SRO p candidate.

'[f .

.)

6-4

6-1.2.2. Simulator Training. The Direct SRO candidate must complete all simulator training prescribed for the Replacement SRO Training program. In addition, the can-didate must complete portions of the CR0 replica simulator course designated by the Supervisor, Licensed Operator Training and approved by the Plant Operations Director and Operator Training Manager.

6-1.2.3. Reactivity Manipulations. In addition to all of the foregoing, the Direct SRO Training program specifies that all candidates will participate in five signifi-cant reactivity manipulations from a list which includes:

- Approach to criticality; suberitical to 10-8 ,,p,,

-8 10 amps to the point of adding heat.

- Any 5% reactor power change.

- Any RCS Boron change while critical of at least 0.1% of initial concentration.

Response to unexpected or abnormal transients that result in a reactor power change of at least 5%.

- Plant shutdown from hot standby to suberitical.

Participation in the foregoing manipulations is certified by the Shif t Foreman or Shift Supervisor on shift during the event.

6-1.2.4. Evaluation. The eighteen month (maximum) program for the Direct SRO candidate, as defined in the program description 6211-PGD-2611.02, appears compre-hensive and effective. The documentation describing the program for two candidates starting the program on 11/10/86 was examined. The documentation scheduled the first ten weeks of classroom training and six weeks of OJT. Additionally, the total program requirements for classroom training were specified (but not yet scheduled),

and the method of training was designated (lecture or self study). The specific CR0 OJT qualifications to be completed, in addition to those required in the SRO pro-gram, were specified. As required by the procedure, the entire package was approved i

by signature of both the Plant Operations Director and the Operator Training Man-ager. It is apparent that this program is implemented in conformance with the applicable THI-1 procedure.

l 6-5

.. . - . _ _ . . . - . - _. . . .-= . . . .-. -

6-1.3. LICENSED OPERATOR REQUALIFICATION PROGRAM. The Licensed Operator Requalifi-cation Program Description, 6211-PGD-2611.01, Rev. 3, 10/26/86, describes a compre-hensive, cyclic program which in training hours matches the guidelines of INPO 86-025, Guideline for Continuina Trainina of Licensed Personnel. Some content dif-forences with this new INPO Guideline were noted in the lectura series topics, abnormal evolution /energency drills conducted, and the period of performance for some of the emergency exercises. Requalification training is' composed of six 4 to 5-day periods annually for personnel of six operating shif ts and off shif t person-nel. The program described consists of four interrelated training segments as follows:

Pre-Planned Lecture Series.

- Skills Trsining and Evaluation.

- Operational Review Program, and p

- Annual Examination and Evaluation.

) The program is designed such that all requirements are completed over a two-year  ;

I period and includes fixed performance standards and remedial training activities as necessary.

6-1.3.1. Pre-Planned Lecture Series. This segment of the training is subcatego-i rized into two general topic areas; Fundamentals Review and Operational

- Proficiency. Fundamentals Review lectures address subjects in which knowledge required of Licensed Operators remains basic and relatively constant. Operational Proficiency lectures cover areas which involve essential plant operational guide-lines and include perational changes and " lessons learned" experience. The lecture series is scheduled on an annual basis, with each of the six shifts typically under-going 240 hours0.00278 days <br />0.0667 hours <br />3.968254e-4 weeks <br />9.132e-5 months <br /> of instruction. All trainees undergo a written examination following each week of lecturos. The scope of the Fundamentals Review lecture series is determined by the Supervisor, Licensed Operator Training and approved by l j the Operator Training Manager. Content is determined primarily by areas of weakness l i

identified in previous year's annual examinations and feedback received from l performance evaluations of licensed operators conducted by plant management /-

supervisory personnel. Content of Operational Proficiency lectures is determined by the Supervisor, Licensed Operator Training, approved by the Operator Training

Manager and is at least partially based on inputs from the Plant Operations Director and the Operations and Maintenance Director. A recently completed requalification i

6-6 i,

.- - , , _ . . ~ . _ _ _ _ , _ _ , - . _ _ . _ , , _ . . . . . . _ _ , _ . _ , . , _ - . _ . . . . _ . , . . . . . . _ . . . , _ . . _ . , . , . _ _ . . _ . . . _ _ _ . _ . , _ .

cycle (86-4, 9/1/86 through 10/10/86) consisted of lecture topics covering the fuel handling bridge, fuel transfer carriage system, new fuel elevator, subcritical multiplication, rad waste evaporation, refueling limits and precautions. AP 1030 (records of primary / secondary openings), and review of emergency procedures.

Additionally, training in Peak Xenon Startup at EOL was presented with the acsistance of the Basic Principles Trainer (BPT) simulator.

6-1.3.2. Skills Training and Evaluation. This segment of training is subdivided into three activity areas; Reactivity and Plant Evolutions, Simulator Exercises, and Plant Drill Program.

Reactivity and Plant Evolution training consists of participation, on an annual basis, in selected scenarios. These scenarios come from a list of Normal Plant Evolutions (performed either in-plant or at an applicable nuclear plant simulator) and Abnormal / Emergency Plant Evolutions (performed either at a nuclear plant nimulator or during plant drills).

Each licensed operator is required, during requalification, to participate in 20 hours2.314815e-4 days <br />0.00556 hours <br />3.306878e-5 weeks <br />7.61e-6 months <br /> of training on a nuclear plant simulator involving direct interaction with the plant power control panel. The program description specifies that operators will actually manipulate controls while SR0s may either manipulate or actively supervise manipulation of the controls. The program emphasizes the team concept. Unit I licensed operator requalification training on unannounced casualties was observed on the TMI replica simulator on 28 and 29 January. During the cyclic training week the control room crew participated in three 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> simulator sessions followed by an operational examination on the simulator.

The Trs.ining Department has also at its disposal a Basic Principles Trainer (BPT).

This trainer is currently used to augment lectures on major s*jstem operations under steady state and transient conditions. Current procedures require that each licen:ed operator receive 40 hours4.62963e-4 days <br />0.0111 hours <br />6.613757e-5 weeks <br />1.522e-5 months <br /> per year of training on the BPT until the TMI replica simulator is avaiiable. Current procedures do not specify the breakdown of training between replica simulator training and BPT training. However, within existing s: iedules, 60 hours6.944444e-4 days <br />0.0167 hours <br />9.920635e-5 weeks <br />2.283e-5 months <br /> annually are available for this type of training.

6-7

The Requalification Program requires all licensed personnel to partake annually in plant drills either as a participant or as a monitor. The content of plant drill scenarios is described in the Requalification Program description. Such scenarios are approved by the Plant Operations Director, or in the case of major emergency drills, by the Operations and Maintenance Director and the Director, TMI-1.

6-1.3.3. Operational Review Program. This segment of the training provides for on-shift review of selected operational experiences and changes to existing operat-ing guidance 'or equipment. Major areas of the review program are identified as

" Modification Review" and " Operating Experience Review". " Modifications" include plant design changes, equipment modifications, procedure changes, and technical specificatien changes. "Opar& ting Experience" is defined to include License Event Reports, audit, evaluation and inspectfon reports, publications and periodicals, and NSAC/INPO Significant Event Reports. The value of the review program to the licensed operator requalification lies in its application to training content. The procedure assigns the Supervisor Licensed Operator Training the responsibility for review of both plant modification and operating experience data for possible inclu-sion into the requalification lecture series, nuclear simulator training, and plant drill scenarios.

6-1.3.4. Annual Examination and Evaluation. Each licensed individual is required to undergo an Annual Requalification Examination consisting of both oral and written elements. The written examination is based on the pre-planned lecture series with the level of examination appropriate to the license level (CRO, SRO) of the individual. The examination (s) and answer key are reviewed by the Supervisor, Licensed Operator Training and approved by the Operator Training Manager and Plant i Operations Director prior to administration. Evaluation standards are specified for l

l individual sections and overall achievement. The oral examination addresses duties and responsibilities consonant with the individual's license level and includes response to abnormal and amergency conditions, diagnostics, procedure / specification changes, and plant and industry operating experiences. The examination is struc-tured using a performance checklist, is specified to last at least two hours, and involves sessions conducted in the plant control room and plant areas occupied by individuals whose actions are directed by the licensed operator being examined.

6-8

-Failure to achieve a passing score in either the written or oral annual examination-results in the individual being relieved of his licensed duties and placed in an accelerated requalification program. This - acetierated program, designed by the Supervisor, Licensed Operator Training and approved by the Operator Training Manager and the Plant Operations Director, addresses specific deficiencies exhibited by the individual on the annual examinations. An examination consistent in format with the annual requalification examination, must be passed prior to the individual's return to normal duties.

6-1. 3'. 5. Evaluation. The Licensed Operator Requalification Program as described in 6211-PGD-2611.01 is an ambitious and comprehensive program. It corresponds gene-rally to the guidelines recently published in INPO publication 86-025 Guideline for Continuing Training of Licensed Personnel. Some content differences exist in lecture topics and emergency drills. A particular strength is the degree of participation / interface with plant operations which, if conscientiously pursued, will ensure that participants remain current in procedures and changes in plant design or operating requirements. The availability of the TMI replica simulator will significantly enhance the versatility and potentially the effectiveness of the program. The program, as structured, is considered to meet all existing INPO cri-teria regarding both training system design in general and licensed operator requal-ification training in particular.

6-2. ORGANIZATION AND MANAGEMENT OF THE TRAINING SYSTEM.

6-2.1. INPO CRITERIA. Training Records are maintained to support management infor-I mation needs and provide required historical date.

6-2.2. TRAINING RECORDS. Specifications for maintenance of training records are contained in Training Department Procedure 6210-ADM-2600.02 and in specific program

, descriptions.

6-2.2.1. Training Content Recorde. Training Content Records (TCRs) are developed for each lesson plan comprising a training program and consist of a list of learning objectives associated with the lesson. A single TCR may also be prepared to describe the content of complete courses containing more than one lesson plan. In 2

this instance the learning objectives must be clearly stated and the combined 6-9

attendance and evaluation records must adequately document achievement of the course objective. TCR forms provide for ' documentation of the review and approval cycle required of the training objectives listed therein.

6-2.2.2. Training Attendance / Completion Records. Training Attendance / Completion records serve as documented certification of an individual's attendance or completion of the training described in approved program descriptions and associated training content records. The Training Attendance form is completed by the instructor and contains entries identifying the specific training element completed, the attendees by name and social security number, and as applicable, an evaluation of trainee performance. Attached to the form are the appropriate Training Examination Records which are described in the following paragraph.

6-2.2.3. Trainina Examination Records. Examination Records are maintained for written or oral examinations associated with training evolutions and include records of. examination and post-examination review. Examination records contain as applicable:

a. The number and title of the training evolution to which the examination applies.

b. Student identification.

c. A copy of the examination and answer key.

d. Seating chart when required.

e. Identity of examination author, reviewer, approver and administrator.

f. Identity of grader and any post-examination reviewer.

The examination records are normally attached to the Training Attendance Form identifying the accomplishment of the associated training evolution. ,

i 6-2.2.4. Maintenance of Training Record Files. Training Content Records Training '

Attendance Data, and Training Examination Records are entered into the plant data  !

processing system by the Document Control Center. The original data package is then

, forwarded to the plant Information Management Center (IMC) 'where the documents are '

processed into micro form. The original documents are maintained at the plant site ,

and are accessible by micro form or hard copy. While not specifically addressed in 6-10

the referenced Training Department Procedure, additional data regarding qualifica-tion records, experience, and cther pertinent trainee information is maintained in the records and may be accessed as described.

6-2.2.5. Training Department Records. Examination of a typical program description (6211-PGD-2611.02, Senior Reactor Operator Training) revealed the requirement for the Training Supervisor to retain files of:

a. Weekly examination results,

b. Oral examination results,

c. Simulator training results, and

d. Comprehensive examination results.

6-2.3. EVALUATION. The Training Department has implemented procedures and assigned responsibilities for maintaining training records, which on both a dynamic and historical basis, document program content and trainee performance including: j

a. Training Content Records defining program learning objectives,

b. Examinations / Examination keys,

c. Trainee Performance

1. Written / Oral examinations,

2. Qualification records, and

3. Training completion data.

6-2.4. CONCLUSIONS. The Training Records maintained by the TMI Training Department represent an efficient and effective system in support of management information needs and establishing a source of required historical data.

6-2.5, RECOMMENDATIONS. None.

6-3. DEVELOPMENT AND QUALIFICATION OF STAFF FOR TRAINING DUTIES.

6-3.1. INPO OBJECTIVE. Training staff possess the technical knowledge, the experi-ence, and the developmental and instructional skills required to fulfill their assigned duties.

l 1

I 6-11

6-3.2. INPO CRITERIA.

a. SRO ' Instructor Technical Qualifications are appropriate for the subject matter that they are assigned to teach.

b. Methods are employed to ensure that instructors meet and maintain position qualification requirements.

c. When instructors have not yet attained ~ the recuired instructional qualifications, or only instruct occasionally, training quality is maintained through appropriate additional assistance and supervision.

d. Instructor performance is evaluated regularly and the results are used to improve performance.

e. Continuing instructor development efforts maintain, improve, and advance required knowledges and skills and are based, in part, on evaluations of instructor performance.

6-3.3. CRITERIA EVALUATION. Procedures defining instructor indoctrination and qualification have been promulgated and the records examined indicate that they are effectively implemented. Procedure 6200-ADM-2730.01 Instructor Certification /

- Qualification Requirements, defines criteria for all instructors in terms of general instructional skills and technical qualifications. Procedure 6210-ADM-2610.02 ad-dresses the specific indoctrination / qualification training program for operator training instructors.

Operator instructor training is controlled and documented using an " Operator Train-ing Instructor Qualification Card". All pre-requisite qualifications must . be com-plated ~within six months of assignment to the Training ' Department and prior to actual presentation of instruction. Provision is made for individuals who meet all

-technical requirements, but have not completed the Basic Instructor -Development Program (presented every six months), to receive " Interim Qualification".

The licensed operator instructor must be either SRO licensed or certified, or have satisfactorily completed the training / education in the following:  ;

- Power Plant Fundamentals and Theory.

- Systems and Components Transient and Integrated Plant Response, Operating Procedures, and Simulator Instructor.

6-12

1 Certification of adequate training / education is made by the Operator Training.

Phnager.

l Four instructors were identified by the Training Department as assigned to the SRO Training program (not including simulator instructors or OJT task examiners). Of the four instructors, three had current SRO licenses, one was formerly licensed as a The THI-1 SRO. All four had successfully completed the SRO Training program.

Qualification Cards, required by procedure 6210-ADM-2610.02, were examined for three of the four instructors. The qualification cards were complete (with one exception attributed to clerical error) and contained the required signatures attesting to certification.

Operator training instructors pursue a continuing training program defined in the training qualification procedure and recorded in the final section of the Qualification Card. The Operator Training Manager is responsible for assignment of Development Modules. Additionally, all instructors to Advanced Instructor instructors are expected to progress toward achievement of an SRO license, if not already held. The Continuing Training section of the Qualification Card contains provisions for assigning milestone dates and recording progress.

Instructor evaluation procedures are comprehensively defined in 6200-ADM-2607.01, ,

Instructor Evaluation. The procedure defines the position to be evaluated (Instruc-tor. Instructor Supervisor, etc), the individual performing the evaluation, and the evaluation frequency. The procedure contains evaluation guidelines / characteristics checkoff lists for instructor (classroom), examination administration, and simulator instructor. In cach checkoff list, a follow-up sheet is provided on which the fol-loving information is recorded:

- Action taken/ proposed by instructor's immediate supervisor.

- Approval / Comments by instructor's section head.

- Approval / Comments by Manager, Plant Training.

- Final entry by the evaluated instructor if prior entries required a response.

Prior to this follow-up review, the procedure calls for discussion / critique by the evaluator with the instructor.

6-13

Evaluation records for three of the four SRO instructors were examined. It was evident that instructor evaluation was frequent and informative. One set of records indicated four evaluations in _ the last four months. A second set indicated five evaluations in the last six months. Appropriate levels of supervision were involved including, in at least one instance, the GPUN Director of Training.

6-3.4. CONCLUSIONS.

4

a. Instructor Qualification Requirements for Operator Training instructors 4

are clearly defined and appropriate to the instruction level.

b. Instructor's currently assigned meet or exceed the qualifications,

c. Provision is made for interim certification for instructors who have not completed all qualification criteria to instruct under supervision and assistance as appropriate.

d. A continuing training program, involving both instructional and technical skills is implemented.

e. A comprehensive instructor evaluation program is defined and operating effectively.

6-3.5. RECOMMENDATIONS. None.

6-4 CONDUCT OF JOB ANALYSIS AND IDENTIFICATION OF TASKS FOR TRAINING.

6-4.1. INPO OBJECTIVE.The tasks required for competent job performance are identi-fied, documented, and included in the training programs as appropriate.

6-4.2. INPO CRITERIA.

a. Plant personnel, training staff, and other subject matter experts have conducted a job analysis to develop a valid plant-specific task list.

b. Subject matter experts have assisted in selecting tasks for training,

c. Tasks selected for training from the plant-specific task list have been compared with existing training to determine if existing training adequately supports task performance.

d. The tasks selected for training and the comparison to training materials is periodically reviewed and updated.

6-14

l I

6-4.3. CRITERIA EVALUATION. The objective identified above was the subject of recoimmendation TMI-1-R-3 in the INPO Accredition Report of December 13, 1985. Ap-plicable parts of the recommendation are paraphrased as follows:

" Complete the -job analysis for the Senior Reactor operator. Verify and document that the tasks identified through job-analysis of the operator I position and selected for training are covered to sufficient depth in the applicable training program. For any tasks selected for training but not presently covered, analyze each task to determine the job-related knowledge and skill requirements and incorporate the results into the training program."

The results of the Senior Reactor Operator Job Survey are documented in the " Senior Reactor Operator TMI-1 Task List, 6211-ADM-2611.05, Rev. 1, dated 7/3/86. This revision has resulted from a series of analyses. The original task list, consisting of similar plant tasks extracted from the INPO list, was revised based on an exa-mination of plant procedures (administrative, operating, surveillance, maintenance, Rad-Con). The expanded list was again analyzed to identify tasks to be trained and the appropriate training settings. The training settings considered were classroom, OJT, simulator or an appropriate combination.

The current task list contains 139 tasks in a matrix which identifies the training setting as determined by. subject matter experts in the Training Department. The tasks to be trained are a combination of applicable tasks for the INPO cask list and plant-specific tasks identified during the job analysis.

Of 29 tasks identified for classroom presentation (in whole or part), seventeen have been determined to be adequately covered in existing training and the associated lesson plan is recorded in the matrix. No new classroom material.is identified as requiring development and hence no task analysis has been accomplished in this area.

The remaining tasks identified as requiring classroom presentation are, as a part of a continuing program, being inserted into existing lesson plans. This is being accomplished by using a standard lesson plan revision checkoff which includes cross-referencing tasks into lesson objectives. Where necessary, the need for lesson plan revision is documented using this checkoff process.

6-15

The OJT program for the Senior Reactor Operator is contained in the TMI-1 Senior Reactor Operator Program description, 6211-PGD-2611.02. The description consist' of a series of performance qualification sheets listing performance objectives.

Successful accomplishment is. certified by signature of a design'ated task examiner.

' Of the 109 tasks identified for accomplishment by OJT (in whole or in part),101 are

. traceable to a specific qualification sheet. Performance objectives identified for OJT and appearing in the qualification sheets were developed by subject matter experts in the training department and reviewed by equally qualified plant

personnel.

Simulator training for SRO candidates had previously been accomplished at the Babcock and Wilcox Simulator in Lynchburg , VA. The TMI-1 replica simulator has recently become operational for training and is now the vehicle for SRO simulator i' training courses. Appendix E of the SRO Training Program description, 6211-ADM-2611.02, defines the scope of the simulator program. A supplemental draft document entitled TMI-1 SRO Training Course, Replica Simulator, represents the current stage of development of the simulator program and will ultimately replace Appendix E of the existing SRO Training Program description. This supplemental document identi-fies 41 drills, each with specific performance objectives. Drills and objectives are based upon the requirements of NUREG 0737 supplemented by TMI-1 imposed training requirements. Of the 41 tasks identified in the task training actrix for simulator training (correspondence with number of drills is coincidental), 30 have been refer-enced directly to one or more simulator training drill guides. This analysis has been accomplished by subject matter experts within the training department.

6-4.4. CONCLUSIONS.

a. A plant-specific job analysis has been accomplished for the Senior Reactor Operator position at TMI-1 in general consonance with procedures recom-sended by INPO.

b. A task training setting matrix has been prepared. Existing classroom les-son plans have been specifically identified as adequate for training of 17 to 29 tasks specified in the matrix for classroom training. The process of cross-referencing classroom tasks with existing lessons is continuing, but at the present time there is no fire documentation verifying that the remaining tasks are specifically covered in existing training, nor are 6-16 w,w e ,-- ww,-v, , - , ~ , - , , - - . - - - -

-.-,,.,,--.--.m---.,=,-y-, . . . - , _ - , ,---,.-.-n--ev- w3 ---e ,,,-e--e.--,,., y..,,,.y-.w- r*---.vm,

m

' tasks identified which require task analysis and development of new SRO-specific lesson plans.

c. Those tasks identified for training as OJT are directly traceable to OJT performance sheets in 101 of 109 cases. Analysis of these tasks to' define the objectives has been accomplished by subject matter experts in the Training Department with review by qualified. plant-personnel.

d. Tasks identified for simulator training have, in 30 of 41 cases, been referenced to specific simulator. drill guides. The drill guides are still under development for implementation for tha first time on the TMI-1 replica simulator.

e. Overall, TMI-1 is responding to INPO recommendation TMI-1-3 for the SRO training in a positive and structured manner.

6-4.5. RECOMMENDATIONS. As the assimilation of the plant-specific job survey continues, implement more formalized documentation to provide an audit trail from the task training matrix into the existing and in-development training programs.

This will ensure that all task training identified is included in the training program with appropriate training objectives identified.

6-5. ESTABLISHMENT OF SRO TRAINING PROGRAM CONTENT.

6-5.1. INPO OBJECTIVE. Trainee program content provides -the trainee with the knowledge and skills needed to perform functions associated with the position for which training is conduc'ted. The content of initial training prepares the trainee-The content of continuing to perform the job for which he is being trained.-

training maintains and improves incumbent job performance.

6-5.2. INPO CRITERIA.

a. INPO Training guidelines are used as a guide for selecting, sequencing, and verifying training program structure and content.

b. Tasks are analyzed as necessary.

Personnel qualified in the position help determine training content and c.

confirm its completeness.

d. Plant procedures and other technical and professional references are used to identify training content and plant-specific information.

6-17 4

-_ - . - . _ ~_. , _ . _ , . - _ . . , . _ , , , , _ , , _ . , , , , , , - _ , _ . _ - - - _ _ _ _ .

e. Initial training program content is modified to reflect results of program review and evaluation by plant and training staff personnel.

f. The results of trainee and program evaluations are used to help determine the content of continuing training.

6-5.3. REFERENCES. The following references were used in the review of SRO Training program content.

a. 6211-PGD-2611.02, TMI-1 SRO Training Program,

b. 6211-PGD-2611.01, Licensed Operator Requalification Program.

c. 6200-ADM-2682.03, GPU Nuclear-Technical Content Review and Interface Process.

d. 6200-ADM-2682.11, Program Evaluation.

e. 6200-ADM-2682.12, Course Evaluation.

f. 6211-ADM-2611.05, SRO TMI-1 Task List.

6-5.4. CRITERIA EVALUATION. The SRO Training program consists of three elements:

classroom training, OJT, and risulator training. The structure has been established in general accordance with INPO Training System Development and Senior Control Room Operator Qualification Guidelines. A plant-specific job analysis has been accom-plished and documented in 6211-ADM-2611.05. This task list resulted from a number of iterations starting from the INPO generic list. Both Training Department and Plant Operations subject matter experts were involved, although no work sheets used in the iterations were observed. The task list is presented in the standard (and INPO required) training setting matrix with each task identified for classroom, OJT, or simulator training. Existing classroom lesson plans and proposed simulator drill guide numbers are identified. The existing OJT qualification sheets contain per-formance objectives referenced to the associated job analysis task.

The job analysis tasks were analyzed within the Training Department for both the OJT and simulator training elements of the program. Details of development of these learning objectives are contained in the discussion of INPO Accreditation Objective number 6, Development of Learning Objectives as the Basis for Training, elsewhere in this section. The intermediate step (accomplished in some task analysis processes) of identifying skills and knowledges which are then grouped and translated to 6-18

learning objectives does not appear to have been done, nor is it considered necessary. Both the OJT element and the simulator training element reflect adequate translation of tasks into meaningful learning objectives.

Tasks identified for classroom training are being examined to verify that existing lesson plans adequately cover the material. To date. 17 of 29 casks have been suc-cessfully tracked to existing classroom lesson plan material. This process is con-tinuing to ensure that all tasks are either covered or new material is developed.  !

No new material has been identified to date as requiring development and hence no detailed SRO cask analyses have been performed.

i Program Review procedures are contained in detail in three publications: 6200-ADM-2682.03, Technical Content Review and Interface Process, 6200-ADM-2682.11 Program Evaluation, and 6200-ADM-2682.12, Course Evaluation. Implementation of these proce- l l

dures is required by the SRO Training Program description, 6211-PGD-2611.02. The '

procedures are considered effective in concept and comprehensive in scope. The for-mal evaluations called for therein have not, as yet, been accomplished for the cur-rent SRO Training Program which is dated 07/07/86.

The Licensed Operator Requalification Program description 6211-PGD-2611.01 contains detailed procedures directed toward maintenance of the currency of the continuing training process. The Operational Review Program guidelines call for both a )

continuing Modification Review and an Operating Experience Review. Trainee evaluations are conducted periodically during the Lecture and Skills Training segments of the program. A comprehensive written and oral examination is given to .

1 every licensed operator annually. Th2 results are used to identify deficiencies in the training as well as possible personnel weaknesses. The information is effective in updating the Requalification Program learning objectives and is available for use in initial training programs. l 6-5.5. CONCLUSIONS.

1

a. The training content of both the SRO initial training and requalification training programs adequately reflect the needs of the job for which the individual is being trained. This assessment is based in part on examina-tion of existing training contents and the methods of development and, in part, on evaluation of existing procedural requirements.

6-19

b. Procedures for - modifying course material : based on course and-. program-evaluations are exhaustive in detail.

c. A visible audit trail exists from the SRO Task List and Training Setting

-Matrix into learning objectives included in the OJT and simulator training elements of the SRO program. The process of tracking classroom identified

-tasks to existing lesson plans is in progress with coverage presently identified for the majority of tasks.

I 6-5.6. RECOMMENDATIONS. I

a. Develop a visible audit trail for all Senior Reactor Operator tasks iden- j tified for classroom training into either existing (or modifications of existing) lesson plans. Perform the necessary formal task analyses and prepare new lesson plans if any remaining tasks are found . to be nct

. adequately covered in existing SRO material.

6-6. DEVELOPMENT OF SRO LEARNING OBJECTIVES AS THE BASIS FOR TRAINING.

6-6.1..- INPO OBJECTIVE.. Learning objectives that identify training content, and define satisfactory trainee performance are derived from job performance requirements.

6-6.2. INPO CRITERIA.

a. Learning Objectives reflect entry-level skill, knowledge, and experience.

I

b. Objectives are derived from an analysis of job performance requirements-and are the basis for trainee evaluation.

c. Learning objectives state performance actions, conditions under which per-formance takes place, and the standard of performance the trainee should achieve upon completion of the training cetivity.

l

d. Learning objectives are grouped by similar training setting and are sequenced based on their relationship to one another and help trainees move from one skill or knowledge level to another, 6-6.3. CRITERIA EVALUATION. The SRO Training program is divided into three parts:

-classroom, OJT, and simulator. The scope of classroom training for both replacement and direct SRO candidateF is determined by the Supervisor, Licensed Operator Train-ing and approved by the Plant Operations Director and Operator Training Manager.

6-20

The training selected is based on education, plant exparience, and training recently completed. The training objectives therefore, which comprise the assigned training program, reflect an evaluation of the entry-level skill, knowledge, and experience of the SRO candidate.

Job performance requirements for the SRO are defined in the approved plant-specific Task List, 6211-ADM-2611.05. This task list presents a training setting matrix which assigns each task to classroom. OJT, or simulator training (or a combination thereof). OJT performance objectives are based on, and traceable to, the designated tasks. The simulator training course, currently in the final. stages of development, is composed of drills in which each has stated performance objectives. These objectives are based both on designated TMI-1 matrix tasks and on NRC requirements (NUREG 0737).

Classroom training lesson plans are those developed for Reactor Operator training and the objectives reflect job performance requirements for that position. SRO tasks designated for classroom training are inserted into the applicable existing lesson plans. This ensures that classroom training assigned to the SRO, as part of his program (essentially to provide technical refresher / review training), adequately covers the requirements of the associated task.

The current format of the learning objectives contained in the Training Content Records (TCR), while performance based, do not reflect the specific Terminal-Enabling objective format recomunended in the INPO Training System Development manual. In the case of knowledge type objectives, there apparently is a question regarding the l

effectiveness of a specific condition and measureable standard secompanying each objective. A GPU T&E Department procedure 6200-ADM-2682.05 provides for the devel-opment of a plant-wide applicable set of training standards which will ultimately result in a list of terminal learning and terminal performance objectives for each training program. This program is currently applicable to development of new programs / courses or existing courses which undergo a revision of more than 50% of lesson materials. All existing programs however, are to conform to the training standard requirements prior to presentation to INPO for re-accreditation during the second full re-accredition cycle. Learning objectives contained in existing lesson

(

6-21

plans are in conformance with the existing format specified in 6200-ADM-2682.07,

~

Lesson Plans.

Performance objectives currently appearing in SRO OJT performance sheets and simula-tot drill guides are considered to adequately define or imply the conditions, per-formance requirements, and evaluation standards.

6-6.4. CONCLUSIONS.

a. Learning objectives that identify training content exist in current Reactor Operator programs and, support SRO tasks which are assigned for classroom training.

b. Learning objectives, as appearing in the Training Content Record for operator lesson plans, are written in conformance with current TMI-1 directions (6200-ADM-2682.07).

c. Performance objectives in the SRO OJT and simulator programs adequately describe conditions, performance required, and performance standard.

d. Classroom lesson plans in use in the SRO program do not distinguish between terminal . and enabling objectives, nor do they have specifically defined standards and conditions for successful completion.

e. The Training Standard (6200-ADM-2682.05) and associated guidelines contain procedures which, when fully implemented, will result in a learning objective format which conforms to the INPO guidelines.

6-6.5, RECOMMENDATIONS.

a. Ensure that future development and revision efforts reflect, to the extent practicable, the INPO guidelines relative to learning objective content and procedure 6200-ADM-2682.05; Training Standard.

6-7. ORGANIZATION OF INSTRUCTION USING LESSON PLANS AND OTHER TRAINING GUIDES.

i 6-7.1. INPO OBJECTIVE. Lesson plans or other training guides provide guidance and structure to ensure the consistent conduct of training activities.

6-7.2. INPO CRITERIA.

a. Lesson plans for classroom instruction provide for effective consistent class presentations.

6-22

---,-,,--m - + , . - . , - g-- . , , - , . -r -'w-"-- aw - " * ' - - -r- - - - = - - - - - - - - - - - - e++- - - -

b. Lesson plans or equivalent -training guides are used ~for laboratory training, OJT training, and simulator training and include criteria for evaluating proper trainee performance.

c. Lesson plans and other training materials are developed or modified using learning objectives derived from job performance requirements.

6-7.3. CRITFRIA EVALUATION. Lesson plans used in the SRO Training programs are developed using the format defined in procedure 6200-ADM-2682.07, Lesson Plans. The standard cover sheet (Training Content Record) consists of course and lesson iden-tification; lesson learning objectives; and development, review, and approval docu-mentation. The lesson plan content is presented in a two-column format. One column provides the text of the lesson, and the other provides for instructors notes, media, and other instructor personalization.

The OJT element of the SRO course consists of a series of qualification sheets.

Each qualification sheet defines the activity required in the form of a performance objective, the method of performing each objective (Perform, Simulate, Discuss), and a certification by a designated task examiner.

The Simulator Training course for SR0s is in final stages of development with train-ing recently shif ted from a contractor facility (PSI) to the TMI replica simulator.

The simulator course consists of a series of drill guides each identified with starting condition and a list of performance standards. Each of the training ele-ments discussed above reflect the training setting determined in the SRO job analy-sis. The OJT performance objectives were developed through analysis of associated tasks by Training Department subject matter experts. Simulator performance objec-tives resulted from analysis of associated tasks and include drills required by regulation (NUREG 0737). The July 1986 update of the SRO cask list identifies 30 tasks for classroom training. The process of tracking these tasks to existing les-son plans is in progress with coverage presently identified for the majority of tasks.

6-7.4. CONCLUSIONS.

a. Existing training procedures for lesson plan development reflect the INPO guidelines for training system development.

6-23

b. .The structure of existing training material (classroom. OJT, simulator) provide for effective and consistent presentation of training material.

c. .Some tasks identified for classroom presentation have not been tracked to adequate coverage in existing material.

6-7.5. RECOMMENDATIONS. None.

6-8. CONDUCT OF SRO IN-PLANT TRAINING AND TRAINEE EVALUATION.

6-8.1. INPO OBJECTIVE. In-plant training is effectively presented, and trainee performance is consistently evaluated.

6-8.2. INPO CRITERIA.

a. In-plant training is delivered using well organized' and current training materials,

b. Designated personnel who are instructed in program standards and methods conduct in-plant training.

c. When the task cannot be performed, but is simulated or walked-through,'the conditions of task performance, references, tools, and equipment reflect 4 the~ actual task to the extent possible.

d. Performance evaluations use established criteria.

e. Acceptance criteria to be used during administration of oral examination are defined in advance of the examination.

6-8.3. REFERENCES. The following references were used in the review of SRO in-plant training and trainee evaluation.

a.- TMI-1 Senior Reactor Operator Training Program, 6211-PGD-2611.02, Rev. 4, i 7/7/86.

b. Licensed Operator Requalification Program Description (Unit 1). -

6211-PGD-2611.01.

6-8.4. CRITERIA EVALUATION. Replacement and Direct SRO initial training OJT train-ing consists of a series of pre-selected tasks based on the job-survey TMI-1 plant-specific task list. The current program is organized into sets of qualification sheets in the following categories:

- - Shift responsibilities, 6-24

, , . , . . . , _ , _ ,  % ,_ . . - - - -- - . . - . , - - . ,_ c, , ,.,._.r---P,----T=7=+e w ----<-r- -r-- -

---r-

- Administrative / Licensing requirements.-

1- - Normal operations.

• - Large scale evolutions, 1

- - Infrequent / Abnormal /Esergency conditions, and

- Maintenance / Surveillance coordination.

Each qualification sheet has a list of performance objectives referenced to one or more tasks identified for OJT accomplishment on the SRO job analysis task list. All exempt (non-union) replacement and direct SRO candidates must successfully complete

. all qualifications. Bargaining unit SRO candidates are exempt from certain of the qualifications until they are promoted to Shift Foreman.

f Certification of qualification by a trainee takes place in two steps. First, accomplishment of performance in any one qualification sheet is certified by a'" Task Examiner" whose qualifications are specified on the qualification. Then final qualification in any of categories (i.e. Shift Responsibilities, Normal Operations

- etc.) is accomplished by means of an oral examination administered by the Shift Supervisor (or- Shift Foreman if designated in writing). A structured " Oral-Examination Summary Sheet" is used in performing this oral examination.

The method of performing each objective on the qualification- sheet is specified as any of the following:

- Perform - The candidate performs each task under the direct supervision of a qualified operator.

Simulate .- The candidate simulates performance under the direct supervision of a qualified operator. The simulation is accomplished as much as possible in the vicinity of the actual equipment or component.

- Discuss - The Task Examiner shall examine the candidate . on the items to evaluate the candidate's overall understanding of the objective.

The CR0 Training Program description contains detailed guidelines for Task Examiners and Final Verification Task Examiners on conducting checkouts and examinations.

Neither the CR0 Training Program description nor any plant documentation examined establishes a program for training plant OJT task examiners on program requirements

as required by INPO accreditation eriteria 9.2.

i

' 6-25 4

. . - , - - . - _ ... . _ . _ . _ .- - _ . . _ , . _ . , _ . , - _ - ~ . - - - - . _ - _ . _ , _

6-8.5. CONCLUSIONS. With the exception of documented training of Task Examiners in the standards and methods of OJT evaluation (INPO Criterion 9.2), the program as described is fully responsive to the objective and criteria of INPO Accredition

, Objective Number 9.

6-8.6. RECOM'4ENDATIONS. Establish a method to document accomplishment of INPO Accreditation Criterion 9.2 on OJT instructor qualification.

6-9. CONDUCT OF SRO SIMULATOR TRAINING.

6-9.1. INPO OBJECTIVE. Simulator Training is effectively presented, and trainee performance is evaluated consistently.

6-9.2. INPO CRITERIA.

a. An appropriate simulator is used for hands-on training to demonstrate operation characteristics, and for recognition and control of normal, abnormal, and emergency plant conditions.

b. The training program content is implemented as outlined by approved training materials and is well organized - and current. Instructional techniques appropriate to the situation are used.

6-9.3. CRITERIA EVALUATION. The INFO Accreditation Report of - December.1984 spe-cifically recommended that the TMI-1 simulator training program require the use of approved simulator exercise guides. These guides were to include learning objec-tives and performance criteria for the initial training of licensed operators on the simulator (Recommendation TMI-I R-2). Although this recommendation was specifically addressed to training performed at a contractor facility, it is appropriate to address the reconenendation in the context of the program under development and in use with the TMI replica simulator.

Simulator training for replacement and direct SRO candidates is described in the SRO Training Program description, 6211-PGD-2611.02. Simulator training associated with requalification training is described in the Licensed Operator Requalification Training Program description, 6211-PGD-2611.01. The following discussion will emphasize the organization of the initial training program within the context of the 6-26

_~ _ _ _ . . _ . _ ._-_ _ _ - _ _ __ -.-._. __ _ - _ _ - _

applicable criteria defined in INPO Accreditation Objective Number 10, Conduct of Simulator Training.

Replica simulator training conducted in initial training for replacement and direct SRO training is outlined in the orogram description, 6211-PGD-2611.02, Rev. 4, 7/7/86. This document is approved by appropriate training and plant operations authorities. During. the current assessment, an updated draft of the TMI-1 SRO Training Course, Replica Simulator, and some typical draft lesson plans were reviewed.

The training course is designed around 41 drill guides. Each guide describes the conditions under which the ~ drill is to be performed and performance objectives associated therewith. SRO simulator lesson plan, Loss of Vital Power - Replica Simulator Drill (Guide #15), Non-Isolable LOCA outside Containment (Guide #38), and RCS Leak and Steam Leak in the RB (Guide #37) were examined. - The lessons contain learning objectives in the TMI-1 standard Training Content Record (TCR) format. The text of the lesson plans stipulated initialization procedures / conditions and con-tinued in the Dil-1 standard two-column lesson plan format (Instructor Notes on one side - Expected Student Response on the other). The lesson plans conclude with a structured list of critique criteria. Each learning objective is referenced to the task (s) in the SRO job analysis task list which forms the basis for the training content.

Trainee evaluation includes the following:

- If the exercise includes a task qualification (0JT performance sheet), Task Examiner certification may be obtained on the basis of successful completion of the drill.

- Exercise critique provides the trainee with immediate information on performance and deficiencies.

- Weekly evaluations of trainee performance during simulator training. The evaluation is performed by a simulator instructor with results provided 4 to the Operator Training Manager and Plant Operations Manager.

- The trainee will be certified in accordance with criteria delineated in TMI-1 AP 1058, Requirements for Certification of Candidates for NRC Operator Licenses and Instructor Certificates. This certification will be i

I 6-27 y ,---vw-, -- -= ,---y .,,,s, , - - _ _ , , e m--------m- -g. - - _ _ - . _ - . . . _ . - - - - , . - - - - . - - _ - _ . - . - - - . . - - , _ _ _ _ _ . , _ . _ - - - . - _ _ ,

accomplished by means of a .' Simulator Operational Examination. The car-

tification is made by the Plant Operations Manager, TMI-1.or his designee.

i This examination may be administered concurrent with or independent of, the simulator course final examination.

- A simulator course final examination will be given to each SRO candidate.

2 The examination will be conducted by a qualified simulator instructor and ,

d monitored by the Simulator Development and Training Manager.

~

. Simulator Instructor (s) will. prepara an overall evaluation of the  !

trainee's performance at the completion of the course.

Simulator instructors must meet all qualifications stipulated in procedure 1

6210-ADM-2610.03, Simulator Training Instructor Indcctrination/ Qualification. l Simulator Exercise Evaluation. The evaluation of replacement SRO training 4

6-9.3.1.

exercises on the TMI-1 replica simulator is described in Section 5, paragraph i

5-1.13.2.

6-9.4. CONCLUSIONS.

?

1

a. The TMI-1 Replica Simulator Training Course described in the SRO Training Program description more than adequately conforms to the INPO accredita-tion criteria for conduct of such training. . The documentation is in con-formance with the requirements of des?.gn and development of performance-based training. Exercises conducted on the TMI-1 replica simulator conform with the program requirements and the exercise guides.

, b. The training documentation includes detailed learning objectives and per-

! formance criteria based on plant-specific job analysis and is considered totally responsive to INPO Recommendation TMI-1-R-2 of the December 1984 Accreditation Report.

6-9.5. RECOMMENDATIONS. None.

6-10. SYSTEMATIC EVALUATION OF TRAINING EFFECTIVENESS.

i

6-10.1. INPO OBJECTIVE. A systematic evaluation of training effectiveness and its e

relation to OJT performance is used to ensure the the training program conveys all required skills and knowledge.

?

o 6-28 T

. - - -ww* ~- . ,~r,--e ,yr- ,--..-e.. -,-.---4-y-- - . - - , - . , , - - . - - - - . , ., w,-- - - . _ .+~ --.-.------or ,e----,-.,---e r---r--.---- - - - - - - - - - - -

p 6-10.2. INPO CRITERIA.

a. Program evaluations are conducted on a regular basis by qualified individuals.

b. Training delivery is monitored and evaluated with regard to instruction, materials, and instructor performance.

c. Feedback from trainee performance during training is used to evaluate and refine the training program.

d. Feedback from trainee performance, after the trainee has assumed the -l duties for which he was trained, is used to evaluate and refine the training program.

e. Change actions are monitored and evaluated for their applicability to the development or modification of training programs and are incorporated in a timely manner.

f. Improvements and changes to training are initiated and tracked to correct training deficiencies and performance problems. ,

6-10.3. CRITERIA EVALUATION. Of particular concern in achievement of this objec-tive is the Conclusion of the Atomic Safety and Licensing Board contained in the partial initial decision on the remanded issue of Licensed Operator Training at TMI-1 of May 3, 1985. This conclusion stated, in part, that the licensed operator training at THI-1 is adequate to train R0s and SR0s to operate the facility safely provided that the Licensee institute a procedure for evaluating, after training. the performance of its trained operators in the job setting for revision of the training program.

Program evaluation procedures are delineated in Training Department document 6200-ADM-2682.11, Program Evaluation. These procedures specify a complete evalua-tion of each training program at least every four years. Determination of which l i

program (s) to be evaluated is the responsibility of the Site Training Manager. The evaluation is an extensive structured inquiry involving completion of a comprehen-sive " Program Evaluation Instrument". Areas evaluated include: development, objec-tives, program outline / syllabus / lesson plans, instruction instructional material, ,

trainee evaluation, training staff, and miscellaneous information. Criteria for review and evaluation of program content for completeness and accuracy are contained in the procedure entitled GPU Nuclear-Technical Content Review and Interface Process i

i 6-29 t

I

- - -- - rn,- .~---,.---m-,n.-- - ,w,. ,-------,-n.

. --,n .- nv-.a, ,----,,n -- - - - - - , - - - - - , , - - - - - - . - -

(6200-ADM-2682.03). The SRO Training Program description 6211-PGD-2611.02 was com ,

plately rewritten and issued on 7/7/86. The rewrite reflected, among other items.

the changeover of the simulator program to the TMI replica simulator and the devel-opment of OJT performance sheets reflecting the plant-specific job analysis.- Prior to issue, the training program was reviewed and approved by the TMI-1 Plant Opera- 'f tions Director and the Manager, Plant Training TMI. The current version of the SRO Training Program description contains specific requirements for review of technical content by licensed Operator Training personnel and Operation Department SR0s.

Post-presentation review is required in accordance with existing procedures. by instructors from the licensed operator and simulator training section. f i i A program for monitoring and evaluating training instruction, materials, and instructor performance is in place and defined primarily by procedures contained in 6200-ADM-2607.01 Instructor Evaluation. Evaluation questionnaires and documen-tation of follow-up actions by supervisory and management personnel were examined and found to be complete and effective.

Feedback for the purpose of refining the training program is being received from trainee performance, after the trainee assumes the duties for which he was trained.

Basic direction is contained in 6200-ADM-2682.10 Trainee Evaluation' Once Back On-The-Job. The SRO Training Program description also contains procedures for ,

SRO-specific implementation of the basic directions. The instructions call for an evaluation by the in-plant supervisor of trainees against specifically defined performance criteria. The instructions also include recommended methods f r ' the g supervisor to perform the evaluations. The supervisory evaluation for the last SRO class (the first since institution of the OJT feedback procedures) was comprehen- ,

sively accomplished and documented. Examination of the documentation package revealed the following sequence of events:

- A memorandum from the Supervisor, Licensed Operator Training to the Plant l

! Operations Director, identifying personnel due for post-training evaluation questionnaires by appropriate supervisors.

- Return of completed questionnaires to the Supervisor, Licensed Operator Training.

i f

6-30 l

I

.____,___..._.._,[

, - - - = - . . .. . ..

{

pr t v, N b I

/ . .

- Memorandum. from Supervisor, Licensed Operator . Training to Operator .

Training Manager (copied .to hypropriate training, plant, . and management. ,

I-personnel) identifying specific items requiring res,olution, and ' actions '  ;

taken or' recommended for each item. f g'

/; -

[A This aspect of the back-on-the-job feedback process appears 'to be effective.- The evaluation package examined was the first accomplished for SRO trainees un r the once back on-the-job evaluatiEn procedure. No specific training material revisions were documented. However, each ites submitted was evalustAd in the - context of exiu' ting and under-development training di.e. the new simu'lator training course)'and i

the resolutions proposed are responsive to the evaluation results. b, .

ti ,-

The SRO Training Program desc/ription also requires thati the SRO trainees once back -

e, j ,

r.

%p,- ,

on-the'-job complate a training {tcritique form. .None of these completed trainee forms-

< i N.f were witnessed. It was repo{ted that some plant supervisors ' interview the< former trainees as part of their a';eluacion process and include the SRO candidate comments

,a , t 1,/ '" in their 'o'verall evaluation.

( ,J 3 6-10.4. (CONCLUSIONS. , /

~ , < (

f a. [A program directed toward the systematic evalu,ation-of , training. 4

.s

- effectiveness asiu ,t:s relation to OJT performance is in place'as'

- x ,

defined v - '

in the follovir}g' docunanta: 3 f

, 1. Program Evaldation, 6200-ADH-2682.11, ,

[,, 2. Trainee Evaluation - Once back on the job, 6200-ADM-2682.10, '

l '3. Codrse Evaluation Process, 6200-ADM-2682.12, and ' .,

n t t ,

( ' 4.

I ' Te'chnical Content Review and Interface Proces's, 6200-ADM-2582LO3.

'.~ V) .

b. Feedback from the supervisor's OJT evaluation is being 'solticitid,

. received, and . acted upon by the ' Training Department in ' an affechive 7

" manner, y f c. Trainee critique forms following return to the job as specified in the SROi r . 5 s

s

, Training Program description were not completed following the mostg recentd,

' replacement SRO program.( -

- p,.

d. The specific procedure (6200-ADMIN-2682.10) defining evaluation of

'. 1 J trainee perfermance when back on the job, for purposes of identifying n

training' deficiencies , is an appropriate response to the concern of' the t .

ASLB. e >

i

,'j 1

/- r

): 6-31 .

/

s' t

- -. ,. __ . _ . _ _ _ _ . ~ _ _ , . _ _ _ . _ _ . . . _ . _. . ,_ . . _ _ . . _ , , . , , _ _ _ _ _ . - . . . _ _ . - . _ . . . . . _ _ . . , _ ,

6-10.5. REC 0ffENDATIONS. Ensure' that in future training program presentations, that procedurally specified trainee evaluation / critiques are accomplished. once back on-the-j ob.

l l

l l

i 6-32

~

. u I

L SECTION 7 WJXILIARY OPERATOR (NON-LIf'ENSED OPERATOR) TRAINING PROGRAM 7-1. AUXILIARY OPERATOR INITIAL TRAINING PROGRAM DESCRIPTION. 1 The Auxiliary Operator (AO) Training Program for the Three Mile Island Nuclear Gen-erating Station, Unit 1 (INI-1) consists of initial cineroom training and on-the-job (0JT) training. There are three s edliary operator job positions advancing from the entry level C operator to th+ A operator. Procedores exist for allowing personnel with previous nuclear est e

• ce to enter the program above the entry-level position. This training proy n sefined and implemented by the TMI Train-l ing Department Procedure, 6211-PO 6,. 01, Auxiliary Operator Training Program THI-1. The average amount of 'ime te c..aplete the training program is two years.

l l

l To be selected for assignment to the position of Auxiliary Operator C Nuclear, an l ' individual must be a high school graduate with a background in basic mathematics including algebra or equivalent education. He/she must satisfactorily complete the aptitude and/or comprehension test (s) required for assignment to this classifica-tion, and have at least one year's experience in a steam electric generating station or its equivalent in experience.

The Auxiliary Operator B Nuclear candidate must have one year's experience as an Auxiliary Operator C. He/she must pass written and practical examinations to qual-ify for advancement from Auxiliary Operator C to this higher classification.

Auxiliary Operator A Nuclear must have one year's experience.as an Auxiliary Oper-ator B. He/she pass written and practical examinations to qualify for advancement from Auxiliary Operator B to this higher classification.

7-1.1. CLASSROOM TRAINING. The scope of classroom training for the A0 C position is determined by the Supervisor Non-Licensed Operator Training based on the back-ground and previous experience of the candidates. Typically the AOC classroom training consists of some forty weeks of instruction covering the following topic areas:

7-1

Nuclear Power Orientation-Mathematics Basic Nuclear Concepts ,

Fluid. Flow / Heat Transfer / Thermodynamics Mechanical Fundamentals Electrical Fundamentals

' Chemistry and Water / Waste Treatment Instrumentation and Control Fundamentals- ,

Safety Analysis Introduction

-Plant Systems Procedures Fire Protection Industrial Safety-During the classroom phase of the training program, approximately four to six hours l'

each day are devoted to the presentation of material or examinations. The remaining-two-to four hours each day are used for study of t'he appropriate material, either in the classroom or in the plant.- Instructors are available during study periods to assist the students as necessary.

7-1.1.1. Written Examinations and Quizzes. Written examinations are administered

- each week (40 hours4.62963e-4 days <br />0.0111 hours <br />6.613757e-5 weeks <br />1.522e-5 months <br />) during the classroom training and documented on -the Student Progress 1 Report. These examinations include a review of material from previous portions of the training program. The minimum passing score on.all written examina-tions is seventy percent. Failed examinations are reviewed with the student. If desired, his/her bargaining unit representative may be present during this review.

Failure. of an examination results in a second written exam on the same material '

being administered within two weeks af ter the initial exam. The minimum passing score on this secondary exam is 70%. Any failed recxam is again reviewed with the I student and his/her barg'aining unit representative as desired.

A. student who fails both the initial exam and reexam on the same material is eval- '

usted by the Operator Training Manager and Plant Operations Director, TMI-1, who determine the appropriate action to be taken. Any individual dropped from the training program is returned to the job classification held prior to being selected and assigned to participate in any applicable training program.

7-2

~

Unannounced written quizzes are administered periodically throughout the training program. Such quizzes provide information to the , instructor and students relative to program pace and areas of general or individual weakness. These are not used as a formal measure of student performance or standing in the training program.

7-1.2. ON-THE-JOB TRAINING. This training provides the student with practical experience in all phases of plant operation for which an Auxiliary Operator A Nuclear is responsible. It reflects those tasks designated for OJT in the Job / Task Analysis and is used to certify the competence of personnel to perform operations in the following areas:

Administrative Requirements Plant Support Systems Secondary Plant Readings Secondary Plant Operations Primary Plant Readings Primary Plant Operations Nuclear Fuel Handling Surveillance Requirements

, Emergency / Abnormal Procedures The On-The-Job Training Program consists of two phases of preselected tasks obtained from the Job / Task Analysis. Two phases are utilized to provide the student with a balance of classroom and OJT. The first or probationary phase consists of 90 days during which the trainee is assigned to a shift where he gains operational experi-ence by accompanying and assisting a fully qualified A0 A in the performance of his duties. During this probationary period the candidate attends GET and RWP training, completes the Administrative Requirements section of the OJT program and is formally evaluated by the Shift Foreman / Shift Supervisor.

The OJT period following classroom training is approximately one year in duration and consists of self study, training, performance of tasks on the A0 Task List, task checkouts and verification, comprehensive written examinations, and comprehensive oral examinationa.

7-3 y.- # . , -

7-1.2.1. Task Performance. Tasks on the A0 task list designated for accomplishment during the OJT training period are divided into three main sections. There are 152 tasks in the Primary Section, 101 tasks in the Secondary Section and 78 tasks in the f Outbuildings section. Specific guidelines have been developed, contaiaing learning objectives which must be satisfied when task checkouts are performed. The guide-lines which are to be used for administering OJT checkouts are contained within the OJT card under signoff.

The level of performance for each OJT task is designated in order of preferen.:n as:

(P) = Perform. The candidate shall perform these tasks under the direct supervision I of a qualified Auxiliary Operator. This level of performance is the most preferred i due to the hands-on experience gained by the candidate. No other level of perfor-mance can be used as substitute.

(S) - Simulate. The candidate shall simulate these tasks under the direct super-vision of a qualified Auxiliary Operator. The simulation should be done as much as possible in the vicinity of the actual component or equipment. Hands-on training, without actual manipulation of valves, breakers, etc. is encouraged. " Perform" can be used as a substitute.

(D) = Discuss. The task examiner shall examine the candidate on these tasks to

evaluate the candidate's overall knowledge of the task, as compared to the guide-line.

Each task examiner is responsible fer the trainee's actions during the training exercise (0JT). The signature of the task examiner indicates that the candidate has satisfactorily completed the level of performance assigned to the task and has satisfactorily met the guidelines established for that task.

A final separate section verification is conducted on completion of all tasks in the Primary, Secondary, and Outbuilding Section. Final verification of a task sheet section is administered by a section examiner using an oral checkout and signoff, documented by an Oral Examination Sunrary Sheet which is retained in the trainee's file.

7-4

7-1.2.2. Comprehensive Written Examination. .A final written examination is admin-istered :by = the' Training Department at the conclusion of the Auxiliary C classroom training or at the conclusion'of the Auxiliary C training program. (prior-to pro-

-motion to Auxiliary Operator B). A written examination is also administered by the Training Department at'the conclusion of the Auxiliary Operator B Classroom Training or at .the conclusion of the Auxiliary B Training Program. '(prior to promotion to Auxiliary Operator A). The miniaun passing score on these examinations is 70% over-all.

7-1.2.3. Comprehensive Oral Examination.- An-oral examination is administered by a SRO licensed Shif t Supervisor /Shife Foreman between the tenth and twelf th month

.af ter selection to the Auxiliary Operator C and the tenth and twelf th month' af ter

, being promoted to the Auxiliary Operator B and documented using the Oral' Examination Summary Sheet. The oral exam is comprehensive in nature and may include questioning

~

in any or all of the same areas as the comprehensive written exam. The oral exam s graded on a pass / fail basis.

7-1.3. PROGRAM COMPLETION. A student who satisfactorily completes the written and oral exams (or reexams) and who has ~ been an Auxiliary Operator C . Nuclear for at least one year meets the training requirements for advancement to Auxiliary Operator B Nuclear. A student who . satisfactorily completes the written and oral exams (or reexams) and who has been an Auxiliary Operator B Nuclear for at least one year meets the training requirements for advancement to Auxiliary Operator A Nuclear.

i 7-2. AUXILIARY OPERATOR RETRAINING PROGRAM.

The Auxiliary Operator Retraining Program consists of classroom and/or inplant training and cours and is scheduled on a cyclic basis.

The objectives of this program are to:

a. Maintain the level of knowledge and job proficiency achieved in the ini-tial Auxiliary Operator Training Program.

b. Recertify the competence of personnel to operate inplant equipment safely i

and efficiently under all plant conditions.

c. Upgrade the level of knowledge and job proficiency in selected areas as deemed necessary by reviews of industry and facility experiences.

i.

7-5

,m -w , , - n,w-q.---.--.~-ww.. ,.,g,-<se,,,,,.+ ,-,,ny,,,..me,ene,,w,,- ,e,-, w,,-.,,-e-,-,,rm +,,,wm,,--m,,-.w,,-- mew,+'<co-e,,_,,,,

All on-shif t Auxiliary Operators A, B and C attend the retraining program during

- their scheduled ^ training week. The cyclic retraining program is conducted on a .

continuing basis, with a weekly schedule of classes designed to be repeated for each shift when that shift is designated for its training week. Weekly quizzes / exams are-administered when ' appropriate to evaluate the effectiveness of the retraining pro-gram and~ operator progress.

2.1. PROGRAM CONTENT.. The Auxiliary Operator Retraining Program contains the topics required for General Employee Retraining, plant fundamentals retraining topics, and plant and industry practices, changes and experiences.

7-2.1.1. General Employee Retraining. The following topics are presented and-qualifications certified as necessary during the A0 Retraining Program:

a. General Employee Training

b. Radiation Worker Recertification

c. Emergency Plan Training ,

d. Fire Brigade Training

e. Industrial Safety Training, First Aid Recertification (as necessary) 7-2.1.2. Fundamentals Retraining. The following plant fundamentals training /-

retraining subjects are included in the cyclic retraining program as necessary:

a. Mathematics

b. Basic Nuclear Concepts

c. Fluid-flow,' Heat transfer, Thermodynamics d.- Mechanical Fundamentals

e. Electrical Fundamentals

f. Chemistry and Water / Waste Treatment

g. Instrumentation and Control Fundamentals

h. Safety Analysis 7-2.1.3. Program Maintenance. The Auxiliary Operator Retraining Program is main-tained up-to-date to reflect the following.

a. Changes in regulatory requirements.

b. Changes in applicable codes, standards and guides. ,

c. Significant experience at the facility l

7-6

~+ e - - - - - . . - , . , , - . - . . , , , , , . . , +w.-~wy.

-m.

,-- ,, ---,--r-.y. - - - - - _ - - - - - - ,

i d.. Significant experience throughout the industry.

. 's . Remedial action recommended by review / audit findings

f. F.egularly scheduled participant critiques 7-3. ORGANIZATION AND MANAGEMENT OF THE TRAINING SYSTEM.

7-3.1. - INPO OBJECTIVE. The utility is organized, staffed, and managed to facili-

. cate planning, directing, evaluating, and controlling a systematic training process that fulfills job-related training needs.

7-3.2 INPO CRITERIA.

a. The responsibilities and authority of personnel involved in managing, supervising, and implementing training are clearly defined in~ writing and permit effective control of the training process.

b. A training system is. implemented as the primary management tool for devel-oping, conducting, and eveluating training.

c. Procedures are implemented to ensure that instructional activities can be conducted reliably and consistently.

d. Training to be completed prior to qualification is clearly defined. Ex-emptions from training may be granted when justified and supported by a documented assessment of prior training and experience.

e. Training records are maintained to support management information needs and provide'r'equired. historical data.

7-3.3. CRITERIA EVALUATION. The paragraphs which follow comment on the above INFO criteria as applicable. Section 4 of this report examined the corporate and train-ing staff organization and management. Section 5 examined . the organization and

[

[ management of the Operator Training Section of which Non-Licensed Operator Training I is a part. Criteria covered in detail in the preceding sections which apply to the

! Auxiliary Operator Training Program will not be repeated in this section unless warranted.

d. 7-3.3.1.. Responsibilities. The responsibilities end authority of personnel involved in implementing the Auxiliary Operator Training Program TMI-1 are spelled l out in the program description 6211-PGD-2612.01. These responsibilities are as f follows:

1 7-7 d

. . _ , . - . _ - _ , . . . , _ _ _ . _ , . . . _ , . _ . . . . _ . _ _ _ _ . _ , - ~ - . . , _ . - . _ _ _ . _ _ . , - . _ . . _ . - _ _ _ . . _ - . . . - - -

a. The Operator Training Manager is responsible for the following:

(1) Assuring the quality of the Auxiliary Operator Training Programs by written approval of materials including course outline, lesson plans, technical content of quizzes and exams.

(2) Assuring the compatibility of the Auxiliary Operator Training Program with other Operator Training Prorams.

(3) Reviewing the completion of qualification records and reporting results to the Manager, Plant Training and Plant Operations Director,

, TMI-1.

b. The Supe rvisor, Non-Licensed Operator Training is responsible for the following:

(1) General supervision of the development and conduct of the Aux 111arv Operator Training Program.

(2) Approval of the development, coordination, scheduling, and adminis-tration of the Auxiliary Operator Training Program including devel-opment of course outlines, lesson plans, student handouts, and eval-u:7. ion exams.

(3) icheduling classes, students, classroom, and facilities necessary to conduct the training program.

(4) Interfacing with Operations Department in all matters impacting the training programs.

(5) Assuring that the program content is updated and revised to meet cur-rent requirements and supervising revision of the program content, descriptions, lesson plans, tests, and exams.

(6) Evaluation of course instruction and student progress to determine the effectiveness of the training program and reporting these evalua-tions to the Operator Training Manager.

(7) Monitoring and conducting spot checks on the quality of OJT.

(8) Initiating the necessary records and reports of training.

(9) Developing and conducting oral exams.

i (10) Evaluation of individual critiques of the training received.

c. The SRO licensed Shif t Supervisor /Shif t Foreman for each candidate is re-sponsible for the following:

(1) Ensuring an adequate level of achievement and progress in the OJT phase of the program by students assigned.

7-8 )

I

- , - - - - - - , y- ,__. _,,,,,m r,.- , , . , , , , , . - , , . . - , , , - - - , , . , , - - - - y. - , _ , - - . - , -

(2) Verification of the competence of Auxiliary Operators C and B Nuclear in specific sections by providing the final verification signature of each OJT checklist.

(3) Administration of the comprehensive oral' examinations for advancement of Auxiliary Operator B Nuclear and Auxiliary Operator A Nuclear or certification of completion of the training program for direct hire Auxiliary Operators A. ,

d. The Plant Operations Director, TMI-1 retains the responsibility to ensure that the ' overall level of training of plant operators is satisfactory through the approval of program content, schedules, and administrative procedures.

e. The Manager Plant Training through the Operator Training Manager, is re-sponsible to insure the training program is developed to meet the require-ments established by the Director, TMI-1 through the Plant Operations Di-rector TMI-1 and that proper records and documentation are provided and maintained.

7-3.3.2. Training System. A training system is in place as a primary management system for developing, conducting and evaluating ' training as implemented by 6200-ADM-2682.01 Training System Development Process. The process described by this procedure provides a systematic approach to developing performance-based training programs modeled on the INPO TSD process.

7-3.3.3. Training Procedures. Procedures to ensure that the Auxiliary Operator training programs are conducted reliably and consistently are contained in the

$ following directives:

a. Auxiliary Operator Training Program TMI-1, 6211-PGD-2612.01 describes the purpose, scope / applicability, responsibilities, prerequisites, sequence and objectives of the Auxiliary Operator Training and Retraining Programs.

b. Training Standard, 6200-ADM-2682.05, describes the use of training stan-dards as part of the GPUN TSD model.

c. Training Records, 6210-ADM-2600.02, identifies documents produced in the Training Department that become Quality Assurance Records and prescribes methods for their control and turnover to the Information Management Center.

7-9

_ --- , . . - . _ . ~ _, _

_ . . . - . _. ___-_. _ ._ - ~ , _ _ _ _ . _ - . . - - - - _ _ _ _ - -

S f

d. Lesson Plans, 6200-ADM-2682.07, establishes the format for lesson pla 9 e.: Operator Training Instructor Indoctrination / Qualification Training Pro-gram, 2610-ADM- 2610.02, provides the . requirements for indoctrination, training,'and certification of instructors.

f. Instructor Evaluation, 6200-ADM-2607.01, defines the instructor evaluation

~

program and assigns specific responsibilities pertaining thereto.

3 Course Evaluation Process. - 2600-ADM-2682.12 establishes a program and schedule for evaluating major portions of a training program.

h. Program ' Evaluation, 6200-ADM-2082.11, establishes a program method and schedule for evaluating an entire training program.

I 7-3.3.4. Required Training. A candidate designated by the Plant Operations Direc-

tor to participate -in the Auxiliary Operator Initial Training Program must first serve a 90-day probationary period. During the probationary period the trainee is assigned to a shif t and must complete GET and RWP training and the Administrative Requirements Section of the OJT program. Prior to becoming a candidate for A0 ini-tial training, the individual must have one year's experience in a steam electric generating station or equivalent experience.

7-3.3.5. Training Records. Training records are maintained to support management j' information needs and provide historical data as specified in procedure 6210-ADM-2600.02. The A0 Training Group recordkeeping system is an integral part of the overall TMI' recordkeeping system. It includes both trainee and - program records.

Trainee records include both comprehensive examination records as developed by the .

instructors, which upon course completion are delivered to the Document Control Center for processing. A secure retention and retrieval system is in place. A computerized system for tracking data is currently being developed.

1 A0 training program records (schedules, class rosters, lesson plans, handout mate-rials, examinations with answer keys, class attendance records and grade sheets) are

!' initially under the cognizance of the A0 Training Supervisor who demonstrates exem-plary security practices within his, area of control.

u 2

4 7-10

. _ ._-. _ _ . _ _ _ _ ~ _ _- _ . _ _. _ - . . . _ - . _ , _ . _ _ _ - _ , _ _ .

7-3.4.- CONCLUSIONS. The TMI-1 Auxiliary Operator Training Programs are well orga-nized, staffed and managed to fulfill the Unit 1 job-related auxiliary operator training needs.

7-3.5. RECOMMENDATIONS. None 7-4. DEVELOPMENT AND QUALIFICATION OF STAFF FOR TRAINING DUTIES.

7-4.1. INPO OBJECTIVE. Training staff (utility and contracted, if used) possess the technical knowledge, the experience, and the developmental and instructional skills required to fulfill their assigned duties.

7-4.2. INP0 CRITERIA,

s. Training staff responsible for program management. supervision, and devel-opment have and maintain the education, experience, and technical qualifi-cations required for their jobs.

b. Instructor technical qualifications are appropriate for the subject matter that they are assigned to teach.

c. Developmental and instructional qualifications of instructors include theory, practical knowledge, and work experience in developing, conduct-ing, and evaluating training, as appropriate to their job assignments.

d. Methods are implemented to ensure that individual instructors meet and maintain position qualification requirements.

e. When instructors have not yet attained the required instructional qualifi-cations or only instruct occasionally, training quality is maintained through appropriate additional assistance and supervision.

f. The instructional skills training program develops the necessary instruc-tor capabilities to fulfill training program requirements.

g. Instructor performance is evaluated regularly, and the results are used to improve performance.

h. Continuing instructor development ef forts maintain, improve, and advance required knowledge and skills and are based, in part, or evaluations of instructor performance.

7-11

7-4.3. CRITERIA EVALUATIOL The paragraphs which follow comment on the INFO cri-teria as applicable. Detailed descriptions of instrector qualification, development and evaluation programs which are contained in Sections 5 and 6 are not repeated herein.

7-4.3.1. Instructor Qualification. The A0 Program instructional staff are employed per position descriptions which clearly describes their duties and responsibilities.

GPUN 6210-ADM-2610.02, administrative procedure for instructional personnel qualifi-cations, includes minimum education, experience, and licensic.!/ certification requirements. Each of CF1 A0 instructional personnel meets or exceeds these stated qualifications, both in education and experience.

The A0 Training Supervisor holds a Bachelor's degree, a R0 License and an SRO Cer-tificate. He has over 8 years of experience at GPUN. Of the four other instruc-tors, one holds a bachelor's degree and the other 3 have high school diplomas and some completed college level courses. One instructor is Navy nuclear power trained.

All instructors are non-licensed operator qualified at Unit 1. The average years of work related experience is 10 years among the instructors.

7-4.3.2. Initial and Continuing Instructor Development. Initial and continuing

- technical and instructional skills training - for A0 programs instructors are guided by Administrative procedure 6200-ADM-2730.01 and 6210-ADM-2610.02, Operator Training Instructor Indoctrination / Qualification Training Program. These procedures delin-eate program specific technical and instructional skills requirements that an instructor must meet before teaching and to maintain qualification.

Instructional skills qualifications requirements include the successful completion of a 40 hour4.62963e-4 days <br />0.0111 hours <br />6.613757e-5 weeks <br />1.522e-5 months <br /> fundamental instructor preparation course. This course focuses on pre-sentation and communications skills. A comparison of the GPU-TMI 40 hours4.62963e-4 days <br />0.0111 hours <br />6.613757e-5 weeks <br />1.522e-5 months <br /> to the INPO guidelines demonstrates a close match. Each of the instructors in the A0 pro-gram, less one new-hire, has completed the 40 hour4.62963e-4 days <br />0.0111 hours <br />6.613757e-5 weeks <br />1.522e-5 months <br /> program.

The INPO guidelines require tsat all instructors complete an additional 80 hours9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> of advanced instructor training with an 18 month period following initial completion of the basic 40 hour4.62963e-4 days <br />0.0111 hours <br />6.613757e-5 weeks <br />1.522e-5 months <br /> course. There is no evidence to suggest that this requirement is being met in a planned manner in the A0 Training Program based on interviews with 7-12

-both the_A0 Supervisor'and staff and the Educational Development Group. While some advanced modules have been developed and taught _ by the Educational Development Group, there is no formal mechanism in place to ensure that instructional personnel

' partake of these courses to meet the additional 80 hours9.259259e-4 days <br />0.0222 hours <br />1.322751e-4 weeks <br />3.044e-5 months <br /> requirement.

Technical- qualification for non-license operator instructor are contained in 6210-ADM-2610.02. These ' include either an SRO license / certificate or satisfactory completion of training . or education in the following power plant fundamental and theory subjects at or above 'the AOC level:

a. Reactor Theory and Principles

b. Heat Transfer, Fluid Flow &_ Thermodynamics

c. Chemistry & Water / Waste Treatment

d. Mathematics

e. Radiological Fundamentals

f. Basic Electrical

g. Basic Mechanical

h. Materials Training and certification in the following plant systems, components, operating' procedures and practices are also required for technical qualification:

a. Electrical Systems

b. Primary Systems

c. Primary Auxiliary Systems

d. River Water Systems

e. Radwaste Systems

f. Primary Protection / Instrumentation / Control

g. Secondary Systems h .- Turbine / Condenser Systems

1. Secondary Au::iliary Systems

j. Chemistry Systems

k. Plant Protection Systems

1. Heating & Ventilation Systems

m. Misce'.laneous Systems All A0 program instructors meet these requirements.

7-13

l 7-4.3.3. -Instructor Evaluation. Instructor evaluation requirements are contained

-in procedure 6200-ADM-2607.01. This procedure defines the evaluation frequency and contains evaluation guidelines / characteristics, and check-off lists for evaluation of classroom instruction and examinatipu administration. Each evaluation is to be followed by a discussion / critique with the instructor. A follow-up sheet is pro-vided with each check-off list which contains the following.

a. Action taken/ proposed by the instructor's immediate superior.

b. Approval / comments by the instructor's section head.

c. Approval / comments b'y Manager, Plant Training ,

d. Final entry by the evaluated instructor if prior entries required a response.

A0 program instructor evaluations are being performed int the frequency and method

. prescribed.

7-4.4. CONCLUSIONS. The instructors assigned to teach THI-1 non-licensed operator programs possess the technical knowledge, the experience, and the developmental and instructional skills to perform their assigned duties.

7-4.5. RECOMMENDATIONS.-

a. Develop the instructional capabilities of non-licensed operator training instructors by assignment of- Advanced Instructor Development Modules as provided for in 6210-ADM-2610.02.

7-5. SUPPORT OF TRAINING WITH FACILITIES, EQUIPMENT, AND MATERIALS, 7-5.1. INP0 OBJECTIVE. The training facilities, equipment, and materials ade-quately support training activities.

7-5.2. INPO CRITERIA.

a. Instructional facilities meet training needs.

b. The training staff has necessary instructional aids and equipment.

c. Technical reference materials, including current plant procedures and drawings, are readily available to the trainees and instructors.

7-14

. _ _ ._ _ _ _ . _ _..._ _-_._. - ._. - _ _ _ _ _ _ _ _ _ _ _ _ _ . . . ~

.-. - ~. - - . . . - -

7-5.3. CRITERIA EVALUATION. The paragraphs which follow comment on the INPO Cri-teria as applicable.

7-5.3.1. Physical Facilities and Equipment. The A0 Training Program shares space with other groups in the TMI Training Department in a modern building of some_20,000 square feet. The A0 Training Program occupies seven office cubicles and shares general classroom space with the other training programs.

The ' A0 Training Program makes limited use of the part-task simulator room and the mock-up control room. Several personal computers are available to the training staff for limited computer-sided individualized instruction. Every classroom has a video player and monitor resident as well as an overhead transparency projector and screen.

Classroom comfort and lighting is adequate, though not elaborate. White-boards are mounted in every room for use in instruction and for recording notices.

7-5.3.2. Reference Material. The TMI Training Center maintains a technical library in the Training Center which is staffed with a library aide and accessible to

' trainees and staff during normal working hours. The holdings of the library include plant operating procedures, plant prints Training Department Administrative Poli-cies and Procedures, lesson plans, technical manuals and periodicals, academic text-books, NRC regulations, licensee event reports, industry experience reports and

! audio-visual materialo. The library also contains personal computers and interac-tive video players and software.

l 7-5.4. CONCLUSIONS. The training facilities, equipment and materials are adequate to support the Auxiliary Operator Training Programs. l l

l 7-5.5. RECOMMENDATIONS. None 7-6. CONDUCT OF JOB ANALYSIS AND IDENTIFICATION OF TASKS FOR TRAINING.

7-6.1. INPO OBJECTIVE. The tasks required for competent job performance are iden-i tified, documented, and included in the training programs, as appropriate.

7-15 4

-r----. , ---,.-,----,_--.,,---..,,----r--. -

- - - - - , , , -,,_ ,, ,. . . - . , , , - - - . .m.,---.,_.n.-- , - - , _ - - - - - , - -

7-6.2. INPO CRITERIA.

L a. Plant personnel, training staff, and other subject matter - experts, as appropriate and .as needed, have conducted'a job analysis to develop a valid plant-specific task list.

! ~ b '. Subject matter experts (appropriate plant technical personnel, training staff personnel, or knowledgeable outside personnel) assist in the selec-I tion of tasks for training.

l

.c. Each task'oelected for training from the plant-specific task list is com-

pared with existing' training materials in sufficient depth to' determine if.

existing training adequately supports task performance.

d. The plant-specific list of tasks selected for training and the-comparison to training materials is reviewed periodir. ally and updated, as necessary.

l 7-6.3. CRITERIA EVALUATION. The paragraphs which follow comment on the INP0 cri-teria as applicable.

7-6.3.1. Job Analysis and Selecting Tasks for Training. Plant personnel and train-ing staff conducted the job analysis of the TMI-I Auxiliary Operator position. The original task list consisted of similar plant tasks extracted from the INFO list.

This original list was revised and expanded based en examination of picnt procedures .

(administrative, operating, surveillance, Rad-Con, etc.) and plant practices. Each ,

1 of these procedures was reviewed to identify tasks which could be applicable to the A0 position. As a result of this process, 166 tasks were added to the A0 task list.

l The revised task lists, including both INPO similar configuration tasks and the sup-plemental tasks were further evaluated by operations and training personnel. The evaluation by senior personnel included recommendations as to where the task should be covered in the training program. Personnel surveyed were requested to validate each task and determine whether classroom, OJT, or both training settings should be used. These recommendations have been incorporated into a task list matrix which identifies the training setting for each task.

7-6.4. CONCLUSIONS. The tasks required for Auxiliary Operator job performance have been systematically identified and tasks have been identified for training. A training matrix showing the training setting has been developed for each task.

7-16 m .. .. ___

7-6.5 RECOMMENDATIONS. None.

7-7. ESTABLISHMENT OF TRAINING PROGRAM CONTENT.

7-7.1. INPO OBJECTIVE. Training program content provides the trainee with the knowledge and skills needed to perform functions associated with the position for which training is being conducted. The content of initial training prepares the trainee to perform the job for which he is being trained. The content of continuing training maintains and improv'es incumbent job performance.

7-7.2. INPO CRITERIA.

a. INPO training guidelines are used as a guide for selecting, sequencing, and verifying training program str;cture and content.

b. Tasks are analyzed, as necessary, to determine the task's supporting skills and knowledge to be included in training programs.

c. Personnel qualified in the position for which training is being conducted help determine training content and confirm its completeness.

d. Current plant procedures and other technical and professional references are used to identify training content and plant-specific information for use in developing training materials.

e. Initial training program content is modified to reflect the results of program review and evaluation by plant and training staff personnel.

f. The results of trainee and nrogram evaluations are used to help determine the content of continuing training.

l 7-7.3. CRITERIA EVALUATION. The paragraphs which follow comment on the INFO cri-teria as applicable.

l 7-7.3.1. INPO Qualification Guideline. A comparison was made between the INP0 Qualification Guideline GPC-04 Nuclear Power Plant Non-Licensed Operators Qualifica-tion Programs and the TMI-1 Auxiliary Operator Training Program TMI-1. The TMI ini-tial A0 training program and INPO qualification guideline compare closely in general program content and overall duration. Both programs progress from C to B to A posi-tions, but differ considerably in the timing of instruction. The TMI program pro-vides all the theory and classroom training to the new candidates early in A0 C training, before trainees begin the major portion of the OJT program. The INPO 7-17

training guideline includes most of this academic training in the A0 A training phase. The TMI-l timing of instruction is designed to provide the trainee a " bigger picture" earlier to help relate their training and job functions to the plant opera-tion as a whole. It also provides the shift foreman with greater flexibility in assigning trainees to their crew. Topics covered in AO initial training program are similar to those recommended in the INPO training guideline. The amount of time allocated to each classroom topic can not be directly correlated between the TMI program and the INFO guideline as topic hours of instruction are not specified in the TMI program. The Supervisor, Non-Licensed Operator Training is responsible for determining the scope of A0 training for each group based on the previous training of the candidates. Overall, the INPO training guideline recommends some 1100 classroom contact hours of instruction,and 340 OJT contact hours. The 40 week of classroom instruction and 52 weeks of OJT in the TMI-1 initial A0 training program i provides comparable classroom time and considerably more OJT.

7-7.3.2. Training Program Content. The auxiliary operator training program content is based on the tasks selected for training as part of the job analysis process.

Senior plant personnel and instructors who are A0 A qualified performed the final task selection process and grouping of tasks by training setting. Plant procedures, technical and professional references and plant practices were used in developing the plant specific task list from the INPO B&W generic task list. The INPO Non-Lic.ensed Operator Training guideline was also referenced in developing program con-tent.

7-7-3.3. Program Evaluation. Training program content is modified to reflect results of once back-on-the job evaluations by new A0 As, by the technical review performed biannually or at the start of a new replacement class, and by instructor and supervisor revie- following the presentation of each classroom training prcgram.

The current Auxiliary Operator Training Program TMI-1 is reviston No. 5-01 and is dated 06/23/86.

7-7.4. CONCI" 'ONS. The content of the TMI-l Auxiliary Operator Training programs prepare the Anee to perform the job by providing the required knowledge and skills. .

l l

7-7.5. RECOMMENDATIONS. None.  ;

l i

7-18  ;

i

. _. .._- . -- . _ _ - _ _ _ _ _ _ _ . , _ . . _ _ . . _ _ _ . _ . .- _ ,- ~ _ _ ____

. - . .. . . . . . .. _- - . . - . . ~. .

A f

7-8. DEVELOPMENT OF LEARNING OBJECTIVES AS THE BASIS FOR TRAINING.

7-8.1.- INPO OBJECTIVE. Learning objectives ' that identify training content and-c.

define satisfactory . trainee performance are derived from job performance require-ments.

l' 7-8.2. INPO CRITERIA.

s. Expected entry-level skill, knowir.dge, and experience are considered when developing learnin's objectives.

i b. Learning objectives are derived from an analysis of ' job performance re-quirements and are the basis for trainee evaluation.

c. Learning objectives state the action (s) tho' trainee must demonstrate, the

- conditions; under which the action will take place, and the standards of performance the ' trainee should achieve upon completion of the training activity..

d. Learning objectives are grouped by similar. training setting (for example, classroom and simulator).

e. Learning objectives are sequenced based on their relationship to. one an- .

other and help trainees move from one level of skill and knowledge to an-other.

7-8.3. CRITERIA EVALUATION. The paragraphs which follow comment on the INP0 cri-i teria as applicable.

7-8.3.1. Entry-Level Skill and Knowledge. The Auxiliary Operator Training Program description establishes the entry level prerequisites for assignment to initial training as a high school graduate with a background in basic mathematics including algebra or equivalent education and satisfactory completion of an aptitude and/or comprehension test. Prospective candidates who are weak in mathematics and science 4

are encouraged to improve their knowledge through correspondence courses. In addi-tion, the Supervisor, Non-Licensed Operator Training is responsible for determining the scope of A0 training for each new group based on the previous training of the candidates.

i i 7-8.3.2. Learnina Objectives. Th Auxiliary Operator training program is divided )

into two parts, classroom training and OJT. Learning objectives and performance j l

1

~

7-19 ,

l 1

i- l i 1

. - - - , , -m.,_ - . , . _ . , _ . . _ . _ . _ _ _ _ . - , . _ . - , , _ _ _ _ _ , _ _ .

_ . . . . - , . , _ . , , , . . , _ . . _l

requirements are essentially derived from the TMI-1 AO plant specific task list.

This task list presents a training setting matrix which assigns each task to either in classroom, OJT or both settings.

Classroom training lesson plan learning objectives are listed on the Training Con-cent Record Sheet (TCR) along with originating and approval authority signatures.

All~ objectives on the TCR are prefaced by the statement "At the conclusion of this lesson the student should be able tos". The behavior statements which follow are generally state, label, describe, define, explain. No standard of performanca is specified - in the learning objectives or the TCR but a minimum passing score of seventy percent is required by the program description for all written examinations

during classroom training. There is no differentiation on the TCR between terminal j learning objectives or enabling learning objectives.

Classroom lesson plans and their related learning objectives are sequenced so that basic and fundamental instruction is followed by procedure and system training before commencement of the OJT phase.

7-8.3.3. OJT Performance Requirements. Performance objectives listed in the OJT

, program are established for each task. These objectives are " Perform", " Simulate" t

and " Discuss". The standards to be used when performing the task - and conducting checkouts are specified for each task and are taken from a set of 14 specific guidelines contained in the A0 training program description.

i 7-8.4. CONCLUSIONS.

4. Learning objectives are derived from job performance requirements and are properly grouped and sequenced for effective training.

b. Classroom learning objectives in the A0 training program do not distin-guish between terminal and enabling objectives, nor do they specifically define standards and conditions for successful completion.

1 7-8-5. RECOMMENDATIONS.

a. Ensure that future development and revision efforts reflect, to the extent

j. practical, the INPO guidelines relative to learning objective content and f T&E procedure 6200-ADM-2682.05, Training Standard.

I t

I I

7-20 l

7-9. ORGANIZATION OF INSTRUCTION USING LESSON PLANS AND OTHER TRAINING GUIDES.

9.l' INPO OBJECTIVE. Lesson plans or other training guides provide guidance and 1- '

etructure to ensure the consistent conduct of training activities.

~

L 7-9.2. 'INPO CRITERIA.

a. Lesson ' plans for classroom instruction provide for effective, consistent

- class presentations.

b. Lesson plans or equivalent training' guides are used for laboratory train-ing, on-the-job training (0JT), and simulator training and include cri-i teria for evaluating proper trainee performance.

i c. Lesson plans and other training asterials are developed or modified using

- learning objectives derived from job performance requirements.

7-9.3. CRITERIA EVALUATION. The paragraphs which follow comment on the INPO cri-teria as applicable.

Ij 7-9.3.1. Classroom Lesson Plans. Lesson plans used in A0 classroom instruction are developed using T&E Department procedure 6200-ADM-2682.07 Lesson Plans. This pro-cedure specifies the use of a Training Content Record (TCR) cover sheet for all classroom lesson plans. The TCR lists the learning objectives for the course, ref-

- erences used to prepare the lesson plan, estimated teaching time and originating, reviewing and approval signatures. Lesson plan formats are described in the proce-dure which contain introductory presentation, and summary sections. In addition to I this TLE Department procedure, lesson plan information is contained in several other

training and evaluation procedures and guidelines.

Lesson plans reviewed in the A0 program were found to be essentially topical out-lines of the listed training objectives. Although effective teaching guides, class-room lesson plans could be made more effective by notes cueing the instructor on the use of audio-visual media and handouts, and the inclusion of oral quiz items to test

! trainee understanding of the material. Also, no distinction could be made in lesson plans between terminal and enabling objectives.

1 e I

7-11

- - .--+-,----.,y w---,c-- .m,,.w--,-e-,,-. -

7-9.3.2. . OJT Qualification Guides. The OJT element of the A0 training program con-sists essentially of a group of qualification sheets covering tasks derived from the plant specific task analysis. Each qualification sheet defines the activity re-quired in the form of a performance objective (perform, simulate, discuss) and a standard for a specific category. Performance to the required standard must be cer-tified by-a designated task examiner.

7-9.4. CONCLUSIONS. Lesson plans and OJT checklists used in the A0 training pro-gram are effective training 'uides. g They are based on a plant specific task list and provide criteria for evaluating trainee performance.

7-9.5. RECOMMENDATIONS.

a. Improve the effectiveness of classroom lesson plans as a training tool by incorporating the following suggestions when lesson plans are revised:

Provide a terminal objective supported by that number of enabling -

objectives consistent with the duration of instruction.

Provide reference notes cueing instructors to the appropriate audio-visual media and to supplementary handouts.

Include oral quiz items to stimulate class participation and test comprehension.

7-10. CONDUCT OF CLASSROOM /ND INDIVIDUALIZED INSTRUCTION AND TRAINEE EVALUATION.

, .7-10.1. INPO OBJECTIVE. Classroom and individualized instruction is effectively '

presented, and trainee performance is routinely and consistently evaluated.

7-10.2. INPO CRITERIA.

a. Trainee mastery of learning objectives is evaluated regularly using written and/or oral examinations and quizzes.

b. Written and oral examinations and quizzes are administered and graded in a consistent manner.

4 7-10.3. CRITERIA EVALUATION. There were no THI-1 Auxiliary operator initial or requalification classroom sessions during the period of the on-site evaluation. The paragraphs which follow comment on trainee evaluation.

J

, 7-22

)

i .

- _ - - - - . - - - - - - . - - ~ _ - . _ .

7-10.3.1. Quizzes and Examination. The Auxiliary Operator Training Program description requires tha.c written examinations be administered each week during the classroom training and be documented on the Student Progress Report. The minimum passing score on all written examinations is 70%.

Unannounced ' written quizzes are administered periodically throughout the training program. Such quizzes provide information to the instructor and students relative to program pace and areas of general and individual weaknesses. These periodic quizzes are no't used as a formal measure of student performance or standing in the

training program.

A review was made of a number of completed classroom examinations for prior A0 ini-tial training programs. The individual test items were well written and graded in a consistent manner. The exams reviewed tested a representative number of learning objectives covered in the instruction. The test security measures observed were 3

comprehensive and carried out in detail.

7-10.4. CONCLUSIONS. Trainee performance in A0 cl.sseroom training is evaluated in a regular and consistent manner.

7-10.5. RECOMMENDATIONS. None 7-11. CONDUCT OF IN-PLANT TRAINING AND TRAINEE EVAIUATION.

i 7-11.1. INPO OBJECTIVE. In-plant training or on-the-job training (0JT) is effec-

' tively presented, and trainee performance is evalurted consistently.

7-11.2. INPO CRITERIA.

i

a. In-plant training is delivered using well-organized and current training materials.

b. Designated personnel who are instructed in program standards and methods conduct in-plant training.

c. When the actual task cannot be performed but fr simulated or vt.1ked-

through, the conditions of task performance, references, tools, and equip-ment reflect the actual task to the extent possibir.

d. Performance evaluations use established criteria.

'l 1

)

{ 7-23 i

, i

.1 ...,. . _ , , . - . - . - . , , _ , , . . _ . , - . . - - . . - ~ . . . . . . . _ . , - . _ . . , , _ . . . . ~ . . . . . . . . . _ . , ~_.m., _ , , _ . , - - _ . - _ _ . _ - - - ,

e. Acceptance criteria to be used during the administration of oral examina-tions are defined in advance of the examination.

7-11.3. CRITERIA EVALUATION. The paragraphs which follow comment on the INPO cri- l teria as applicable. j i

i 7-11.3.1. Program Content. The TMI-1 AO initial . training OJT program consists of l the performance and qualification of some 230 tasks from the plant specific task list. The current program is organized into sets of qualification sheets in pri-mary, secondary and outbuilding sections. Each qualification sheet has a list of performance objectives referenced to one or more tasks identified for OJT accom-plishment on the A0 task list. The level of performance for each task is designated in order of preference as Perform (P), Simulate (S), Discuss (D).

7-11.3.2. Performance Evaluation. Certification of performance and qualification by a trainee takes place in two steps. The signature of the task examiner indicates that the candidate has satisfactorily completed the level of performance assigned to the task and has met the guidelines established for that task. A final separate verification is conducted on completion of all tasks in the Primary, Secondary, and Outbuilding sections. This final verification is administered by a section examiner using an oral checkout and signoff document sheet which is retained in the trainee's

! file. The qualification requirements of task examiners is not specified in the A0 training program procedure. The shift supervisor / shift foreman i's assigned the-responsibility for the section checkoff and qualification.

7-11.3.3. INPO Recommendation TMI-1-R-5. The INPO team accreditation report con-tained the following recommendation concerning the Auxiliary Operator Training Pro-gram: " Provide trainee performance criteria for the administrative requirements section of the Auxiliary Operator Qualification Checklist to ensure consistent training evaluation."

The Administrative Section (Appendix A) of the Auxiliary Operator Training Program TMI-1 (6211-PGD-2612.01 Rev. 5.01 dated 06/23/86) contains performance criteria for the eight tasks which the auxiliary operator candidate must complete during his pro-bationary period. The trainee must satisfactorily demonstrate his task knowledge and skills to a designated task examiner. Also included in Appendix A are a series 7-24

of administrative system performance tests in which the trainee must also demon-strate his competence to a task examiner. These performance criteria and perfor-mance test satisfy in content and spirit this INPO recommendation.

7-11.4. CONCLUSIONS. Auxiliary Operator on-the-job training is effectively per-formed and evaluated consistently.

7-11.5. RECOMMENDATIONS. None 7-12. SYSTEMATIC EVALUATION OF TRAINING EFFECTIVENESS.

7-12.1. INFO OBJECTIVE. A systematic evaluation of training effectiveness and its relation to on-the-job performance is used to ensure that the training program con-veys all required skills and knowledges.

7-12.2. INPO CRITERIA.

1

a. Program evaluations are conducted on a regular basia by qualified indivi-duals.

b. Training delivery is monitored and evaluated with regard to instruction, materials, and instructor performance.

c. Feedback from trainee performance during training is used to evaluate and refine the training program.

d. Feedback from trainee performance, after the trainee has assumed the l duties for which he was trained, is used to evaluate and refine the train-I ing program.

e. Change actions (e.g., procedure changes, industry events, equipment changes) are monitored and evaluated for their applicability to the devel-opeent or modification of training programs and are incorporated in a timely manner.

f. Improvements and changes to training are initiated and tracked to correct training deficiencies and performance problems.

g. Contracted training is evaluated for its contribution to meeting job per-formance requirements and to ensure that its quality is consistent with utility training standards.

i i

7-25

7-12.3. CRITERIA EVALUATION. The paragraphs which follow comment on the INPO cri-teria as applicable.

7-12.3.1. Prosram Evaluation. The Auxiliary Operator Training Program description requires the following program evaluations:

a. Once new A0 As are back on-the-job for approximately six months an evalu-ation of the training program shall. be made in accordance with TSD Proce-dure 6200-ADM-2682.10.

b. The Auxiliary Operator Initial Classroom Training Program will be reviewed at the end of each section of classroom training by the instructor pre-senting the course and the Supervisor, Non-Licensed Operator Training.

t

c. Prior to the start of a new replacement class or biennially, whichever is i

more frequent, a team will be formed by the Supervisor, Non-Licensed Oper-ator Training to review the technical content of the Auxiliary Operator Training Program. The review team will consist of instructors from the Operator Training Section and representatives of the Operations Dep trtment and should be headed by a licensed or certified Senior Reactor Operator.

The team will assess the adequacy of the program using the guidance found in 6200-ADM-2682.03 Technical Content Review and Interface Process.

d. Program and course evaluations shall be conducted in accordance with TSD Procedures 6200-ADM-2682.11, Program Evaluation, and 6200-ADM-2682.12, Course Evaluation Process. Procedure 6200-ADM-2682.11 requires.that each program be evaluated no less than every four years. Procedure 6200-ADM-2682.12 standardizes the evaluation of courses.

e. For the requalification program, one shift section shall be selected to critique each training cycle. The completed critiques will be reviewed by I

the Supervisor, Non-Licensed Operator Training and forwarded to the Opera-tor Training Manager if there are any recommendations or corrective

actions to be taken.

7-12.3.2. Training Delivery Evaluation. T&E procedure 6200-ADM-2607.01 establishes the training delivery evaluation program. This procedure describes responsibili-ties, requirements, and the process to evaluation. The requirement for frequency of A0 instructor evaluations is being met and the evaluation forms are being carefully and constructively filled out.

7-26

D 7-12.3.3. Change Action Evaluation. In performing the technical content review of the training program prior to its start the team is responsible for reviewing train-ing and operations department records as well as the following:

a. NRC Inspections

b. QA Audits

c. Other Audits or Reviews

d. Regulatory Changes

e. Industry Experiences

f. Student Critiques I

The review team will report the results to the Manager Plant Training and the Plant Operations Director. THI-1 via the Supervisor, Non-Licensed Operator Training and the Operator Training Manager.

7-12.4. CONCLUSIONS. Training delivery of the A0 programs are routinely monitored and training program feedback is effectively obtained from trainees during instruc-tion, by post presentation reviews by the training staff, and by once-on-the-job A0 A evaluations. I 7-12.5. RECOMMENDATIONS. None.

l 7-27

..