ML20149C346

From kanterella
Jump to navigation Jump to search
Forwards Final Rept Re post-accreditation Audit of Training Programs at Plant Conducted in Mar 1987.Nonlicensed & Licensed Operator Training Programs Developed from Task Analysis Identified Skills & for Learning Objectives
ML20149C346
Person / Time
Site: 05000000, Columbia
Issue date: 04/06/1987
From: Russell W
Office of Nuclear Reactor Regulation
To: Strahm K
INSTITUTE OF NUCLEAR POWER OPERATIONS
Shared Package
ML082310270 List:
References
FOIA-87-787, RTR-NUREG-1220 NUDOCS 8704080295
Download: ML20149C346 (18)
v • d • e


Text

.. -.

s T 46; 4

'E T

DISTRIBUTION:

)

Central Files i

April 6,1987 DMorisseau MRoe l

Mr. Ken Strahm l

Institute of Nuclear Power Operations 1100 Circle 75 Parkway Suite 1500 l

j l

Atlanta, Georgia 30339 j

I l

Dear Mr. Strahm:

)

l Enclosed is the final report from the staff's post-accreditation audit at i

WNP-2 conducted in March 1987.

The WNP-2 Nonlicensed and Licensed Operator training programs were developed from a task analysis that identified skills and knowledges from which learning objectives were derived. This utility's program was the most thorough example the staff has seen of the development of perfonnance-based training using review of existing training and supplementing it with new job and task analysis. This was done through the WNP-2 Near Term Training Review (NTTR) which identified existing training, ensured coverage of required l

knowledges and skills, and established a priority schedule for preparation of updated, improved, or new training materials.

As with other utilities, tasks for continuing training were not identified l

during the initial analysis although there are data available that indicate i

frequency, difficulty, and e.riticality of tasks. At present, continuing training is based on NRC regulations and operations experience.

The WNP-2 programs contain all five elements for performance-based training.

I l

In aodition, this utility intends to apply the principles of performance-based training to programs other than those included in the accreditation program. Although the quality of learning objectives is inconsistent at present, a new procedure has been developed that is designed to ensure that learning objectives will be consistently developed.

Of particular note is the WNP-2 method for program evaluation. The method is one of sampling and in-depth evaluation. As such, it did not fully meet the NUREG-1220 criteria for evaluation of every program.

During the staff's review, however, it became evident that this sampling technique produced a cceprehensive review of training. The staff has, therefore, reccrtended that the criteria and procedures be adapted to permit flexibility in this element.

The training facilities, materials, evaluation methods, and the increase in the staff at WNP-2 indicate that there is strong management commitment to performance-based training.

44pgj W: 1')

X& y.

'I

~ m DATEk.............................................................................................................................................

1 w*~

G..W W

4F

[

=

If you have any questions about the staff's training review activities, please contact me at 301/492-2803.

Sincerely, "ricim'" 'li

~[,- [{

uu.:2 z.a.- a William T. Russell, Director Division of Human Factors Technology Office of Nuclear Reactor Regulation

Enclosure:

As stated cc: J. Zerbe DW/DSM3/LTR TO STRAHM MLR6/WNP2 POST ACCRED AUDIT

.ESB.f. I.......EE.

I....

ET......... RI.QHE.T........

"'c ' >

  • -*>D.%r.is

....J, e.n.9y,.... a o.g er..........XI8.y asa).1..

  • * " > 4./.1.../. E.........

... 4..

. /. 8.7.......

.... 4. /.. l.8. 7......

... 4. /..'b /.8. 7.......

$M

$$*I i

POST-ACCREDITATION AUDIT OF TRAINING PROGRAMS AT WNP2 A.

Introduction 1.0 Background. On March 3 through 5. 1987, the NRC staff conducted a post-accreditation audit of training at WNP2.

Three WNP2 training programs have received INP0 accreditation (NLO, CRO, SRO). The NLO and R0 training programs were the subject of this review.

2.0 Approach.

Prior to the on-site review, NRC staff selected the two programs for review and informed the utility.

Five tasks for each program were selected from the utility's task listing and the utility was informed so that the materials associated with those tasks were available for review. One additional task for each program was identified on site during the review.

The remainder of'the review consisted of document review, interviews and interactions with both training and operations staff and managerent, and classroom and simulator observations to answer the questions in the NRC's "Training Review Criteria and Procedures" (NUREG-1220, June 1986). The utility briefed the NRC review team on the general approach to *. raining development at WNP2 and explained their approach to analysis. The NRC review team consisted of three training and assessment specialists and one subject matter expert from DHFT/NRR, and one subject matter expert from Region V.

3.0 Criteria.

The criteria used by the staff to audit the implementation of performance-based training programs are taken directly from the "Canmission Policy Statement on Training and Qualification of Nuclear Power Plant Personnel" (50 FR 11147) of March 20, 1985.

In its Policy Statement, the NRC states that the following five elements are essential to acceptable performance-based training programs:

i l'

l 2

u (1) Systematic analysis of the jobs to be performed, (2)

Learning objectives that are derived from the analysis and that describe desired performance after training, (3) Training design and implementation based on the learning objectives, (4) Evaluation of trainee mastery of the objectives during training, and (5)

Evaluation and revision of the training based on the performance of trained personnel in the job setting.

4.0 Documentation. To_supportits revjew, the NRC requested that the following types of documentation be provided by the facility:

4

  • Instructions / procedures related to:

i

- Systematic methods used to analyze jobs,

- Training organization goals, objectives, and plans,

- Responsibilities / authority of training organization personnel, l

- Methods for evaluating / selecting instructional materials, methods, and media,

- Methods for organizing / sequencing of training.

l

- Methods for keeping training programs current.

- Maintenance of training records, i

- Selection of candidates for training and the granting of waivers / exemptions from training.

- Evaluation of training programs, and i

- Training, qualification, and evaluation of instructors.

  • Task lists for the jobs being reviewed i
  • Documentation related to:

- Development / validation of task lists.

- Selection of tasks for formal training,

' - Analysis of tasks.

- Analysis of on-the-job performance problems and industry events, and

- Evaluation / audits of the training program (s).

  • Roster / organization chart for the training organization
  • Training schedule The above documentation is the type normally associated with participation in the INP0 accreditation process.

5.0 Scope of Audit. The programs audited and the tasks selected for review within those programs were:

PROGRAM TASKS (1) Nonlicensed Operator

' Monitor operation of the reactor feedwater system

  • Control room evacuation
  • Trip RPS locally if reactor doesn't scram
  • Place low pressure coolant injection system in standby *eadiness The final task reviewed for the NLO program was a replacement task selected on site. This task was chosen to replace a task which was deselected from the program as a result of a plant modification.

With regard to the task which was deselected, Restart a Diesel Generator with Auto-Start Signal Present, we determined near the end of our audit that the task is contained in the January 1987 electrical system failure (blackout) procedure. WNP-2 is aware of this issue and will take steps to resolve it.

4 (2) Reactor Operator

  • Operate vessel level master control l

station

  • Monitor refueling floor activities duringcorealterations(writein)
  • Electrical system failure (blackout)

B.

Results of Review The following discussion of the review findings parallels the elements of the Conmission's Policy Statement on Training and Qualification.

1.0 Systematic Analysis of Jobs to be Performed 1.1 Discussion. Analysis methods, procedures and products were reviewed using the documentation described in Section A.4 above to determine whether:

  • A systematic method was used for identifying the tasks that make upjob(s)beingevaluated.
  • A systematic method was used for selecting tasks for which training is provided.
  • Tasks requiring initial training only and those requiring l

continuing training were differentiated.

1

  • Analysis of tasks chosen for training was adequate for development of learning objectives.
  • Approved procedures are implemented so that analysis information is kept current as job performance requirements change.

1.2 Findings.

To develop a task list for the Nonlicensed Operator (NLO) and the Licensed Operator (LO) position, the WNP-2 Training Staff started with the INP0 Task Data Base.

Subject Matter Experts (SMEs) and Instructors validated the list against WNP-2. Specific job positions whose function it is to perform the tasks were identified by shift managers and shift supervisors.

Three levels of SME review were required for validation; checklists were concurrently developed to determine who had primary and secondary responsibility for the tasks.

Surveys were then distributed to more than 50% of job incumbents to further validate the plant-specific task list.

The survey included rating scales for frequency, importance, and difficulty.

When these data were analyzed and compiled, the task list was again reviewed by both training and plant management, and signed. An analysis of existing training was conducted through the Near Term Training Review (NTTR).

This review determined where training was carried out on tasks selected for training and, using the INP0 taxonomy, ensured that the knowledges and skills needed for all learning objectives were covered in training. The results of this review were used to establish a priority schedule for preparation of new training material or update and improvement of existing material.

The initial training program for the NLO position included an assessment of tasks by which some were deselected because they were not applicable to WNP2. The initial and continuing training, therefore, are composed of all remaining tasks.

The supervisor of nonlicensed training stated that since there were no new trainees, all training was essentially continuing training.

l For the L0 program, input to the continuing training program is taken from operating experience, feedback from the operating crews, plant modifications, and technical specification changes.

Procedures are in place that ensure currency of task analysis information as job performance requirements change.

It is the function of the Near Term Training Review (NTTR) to set priorities for performing subsequent task analyses.

2.0 Development of Learning Objectives 2.1 Discussion.

Learning objectives were reviewed for the subject programs to determine whether:

  • There are learning objectives for each of the tasks selected for review.
  • Learning objectives are derived from or related to the knowledge, skills and abilities needed for successful job performance.
  • Each learning objective states the job performance behaviors expected of trainees upon completion of training.

Each learning objective states the job performance-based conditions under which the trainee actions will take place.

  • Each learning objective states the specific job performance-based standard for successful performance of the objective.
  • Written procedures require modification of learning objectives when related job performance requirements change.

7 2.2 Findings.

Each task selected in the NLO and L0 programs was supported by learning objectives derived from the knowledges, skills, and abilities needed for successful job performance. These knowledges, skills and abilities were derived from the INP0 taxonomy.

Learning objectives for the NLO program that were available for classroom and on-the-job training (0JT) contained actions, conditions and standards of performance. Actions were stated such that in the classroom situation, the trainee would be required to describe an activity while during 0JT, the qualification sheet would require the trainee to demonstrate.

Performance-based conditions for classroom objectives were provided in a format such as "given the proper drawing...." For OJT, conditions were primarily based on procedures; for example, the trainee would be provided with proper procedures for a specific activity.

Standards for classroom training would be 100% success on each objective tested.

With respect to OJT, the stan'dard was correct performance of the activity, e.g., open a valve.

For the R0 program, there were learning objectives for all but one task that could not be reviewed because all the training was conducted off site. As previously mentioned, learning objectives were based on the knowledges and skills required for each task.

Learning objectives all stated the job performance behaviors expected upon completion of training.

Conditions and standards for the learning objectives were often in the appropriate procedures; however, where this was not the case, there was some inconsistency in how well the learning objectives were constructed. A new procedure has been put in place that should improve consistency in the construction of learning objectives.

Procedures were also in place that require modifications to learning objectives as related job-performance requirements change.

8 One method by which this is accomplished is through the distribution of the "Monthly Operating Bulletin" by the Nuclear SafetyAssuranceGroup(NSAG).

Information pertaining to. plant modifications or Significant Operating Events Reports (SOERs) is distributed if training implications are present.

3.0 Design / Implementation 3.1 Discussion.

Using the documentation described in Section A.4 above, the design and implementation of performance-based training was reviewed to determine whether:

  • There is a written plan that clearly and specifically states the training organization's goals, objectives, plans and relationships with other parts of the facility's organization.

Responsibilities and authority of training organization personnel are clearly stated in writing.

  • There are documented qualification and training requirements for the training staff that address both subject matter and l

instructional skills and knowledge appropriate for specific assignments.

  • There is evidence that the appropriateness of instructional settings has been evaluated.
  • The organization and sequencing of the initial training programs are based upon the relationships among learning objectives.
  • The organization and sequencing of continuing training is based upon the relationships among learning objectives.

l l

e

.g.

  • Lesson plans are available that provide for consistent training delivery.
  • There is evidence that the appropriateness of existing instructional materials has been evaluated based upon identified trainee needs and learning objectives.
  • Training is being conducted in an adequate, manner as determined through application of training observation checklists.

There are adequate methods established for maintaining training records.

3.2 Findings.

Procedures in the Technical Training Manual (TTM) state the training organization's goals, objectives, and plans and its relationships with other parts of the facility's organization. The TTM also describes the responsibilities and authority of training organization personnel.

Position descriptions were available for all personnel that described required technical and educational qualifications. An Instructor Program has been developed to provide Criterion-Referenced Instruction (CRI) and Instruction Module Development for training staff.

During the development of Job and Task Analyses, one function of the Near Term Training Review (NTTR) was to evaluate instructional settings for appropriateness, and/or to produce and upgrade as necessary for both the NLO and the L0 programs.

A detailed, structured review of the organization and sequencing of learning objectives for both initial and continuing training content has been conducted.

Lesson plans were available for the six tasks reviewed for the NLO program.

Each contained a cover sheet to provide general

' information on the lesson plans (e.g., title) and the reviewer's signature.

The content of the lesson plans included learning objectives, materials (classroom, instructor, and student),

references, and test materials.

The SME observed NLO classroom training, and using the observation checklist, detennined that the training was conducted in an adequate manner by a competent instructor.

The SME observed classroom and simulator requalification training using the observation checklist and found both training sessions were conducted in an acceptable manner by competent instructors.

l In one task, Electrical System Failure (blackout), the SME was shown a September 1983, WNP-2 analysis which we were informed was the basis for the blackout procedure. At present, training for this task is conducted entirely in the classroom as part of mitigating core damage training. The SME determined that the simulator should also be used for this task, particularly the short to medium term procedure goals of reducing station battery loads and removing decay heat.

Classroom training in selected emergency l

procedure included student handouts of the procedure with notes in the margins which referenced Technical Specifications, FSAR and I

subsequent analysis.

This unique method should assist students in understanding the basis for critical steps or periods in the procedure.

Although interviews with operations personnel indicated that there was good coverage of OJT by the training department, a more extensive program for training OJT evaluators would provide more consistency across evaluations for required performance tasks.

A review of training records revealed thorough record-keeping for both trainees and instructors with references to waivers and exemptions, and training provided.

In addition, a computer

printout listed all members of the training staff, their educational and technical qualifications, and the courses they are qualified to teach.

Records are presently being transferred to the new Personnel Qualifications System, a computerized system that will improve the capability to track and retrieve records.

4.0 Trainee Evaluation 4.1 Discussion. The methods for and use of traine,e evaluations were reviewed using the documentation described in Section A.4 above to determine whether:

  • Exemptions from training are based upon performance-based testing or other objective evaluation methods.
  • Trainee evaluation is appropriate to job performance requirements and training objectives.
  • Trainee performance is evaluated regularly during the training l

program and prompt, objective feedback provided on a regular basis.

l

  • Trainees who perform below minimum standards are provided i

remedial training, retested, and removed from the training program if minimum standards are not met.

Job incumbents who perform below minimum standards during requalification or continuing training are removed from associated job duties and provided remedial training.

  • Appropriate precautions are taken to preclude comprom' a of test contents.

7 12 4.2 Findings. An Entry Level Training (ELT) Program has been developed by WPPSS to prepare interested and qualified applicants for entry into the position of Equipment Operator (EO). Trainees in this programwillgraduatewithanAssociateofScience(A.S.) Degree from a local junior college.

It is expected tnat once hired, a graduate of the program will progress from the E0 position to the L0 position, thereby providing an ample source of well-qualified entry level applicants.

Performance-based testing is not a prerequisite for entry into this position. Exemptions are possible based upon previous experience or training. The groups of applicants in this category would be graduates of the ELT program or individuals with prior Navy nuclear experience.

In cases where waivers are requested for qualifications sheets, the In-grade Instructional Counselor would review the request, and provide a recomendation which would then be forwarded to the candidate's supervisor for approval or disapproval.

Trainee evaluation is appropriate to job performance requirements and training objectives. Objectives were directly traceable to examination questions; however, the selection of test questions was limited. The licensee is in the process of building the examination bank so, in the future, this area will be addressed.

Trainee performance is evaluated regularly during the training program through written exams which may be given weekly or at the end of the training.

Feedback from initial training is provided promptly, within a day or two after testing.

Due to the rotating shift schedule, feedback following continuing training is not provided unless specifically requested by the trainee.

This issue will also be addressed in the future.

l l

l Procedures are in place that require trainees who perform below minimum standards to do additional self-study and be reexamined.

l Depending on the situation, organized study may also be required.

Inadequate performance may result in removal from the program.

If l

Job incumbents were to fall below minimum standards during continuing training, they would also be subject to an accelerated training program and limited work scope.

l In the R0 program, exemptions from training in the L0 program are based on waivers granted by letters written to address both course work and experience rather than by performance-based testing.

l Trainee evaluation is appropriate to job requirements and the training objectives; however, in some cases, the test questions are too similar to the learning objectives with no alternate forms of l

the question.

This also results in questions being repeated on subsequent examinations.

This problem is being addressed through expansion of the test question bank.

Remedial training is provided to trainees who perform below minimum standards. A remediation program is designed for the trainee and submitted to that individual for coment and concurrence.

Documents reviewed indicated that this practice also applies to trainees whose performance is marginal, i.e., a person who has not failed, but passed at the minimum level. Job incumbents who perform below minimum standards during requalification are also provided remedial training as indicated above.

Removal from the program or from job duties is determined by the Operations Review and Training Comi ttee. This determination is made on an individual basis.

Test results for the R0 program (both initial and continuing training) are promptly fed back to the trainees. Tests are returned with correct answers provided so that the trainee understands why an answer has been counted as partially or fully incorrect.

The precautions taken to preclude compromise of test contents are appropriate and procedures are in place to ensure that the precautions are implemented consistently.

However, as previously mentioned, the test question bank for both the LO and the R0 programs is being expanded to prevent repeated use of the same questions.

5.0 Program Evaluation 5.1 Discussion. Training program evaluation methods were reviewed using the documentation described in Section A.4 above to determine whether:

  • A method is in place to systematically evaluate the effectiveness of training programs and to revise the programs as required.
  • Examination and operating test results are evaluated so that tests are improved and feedback is provided to improve training.
  • Instructor critiques of training are used for program evaluation.
  • Trainee critiques of training are used for program evaluation.
  • On-the-job experiences are solicited from job incumbents and used l

for program evaluation.

l

  • Feedback from supervisors about job performance is solicited for program evaluation.

l l

l

  • Both internal and external training program audits / evaluation findings are used for program evaluation.

l

  • The performance of each member of the training staff is objectively evaluated on a regular basis.

1 1

5.2 Findings. The Technical Training Manual contains procedures to ensure that there is a systematic method to evaluate the effectiveness of training programs and to revise the programs as required.

The Technical Training Review Board has the responsibility for implementing these procedures. Rather than an avaluation of every course in every cycle, a sampling procedure is used whereby a full follow-up is done for selected courses. These follow-ups are usually tied in to instructor assessments which are conducted twice a year for between one-third and one-half of the instructors (35-40onstaff). These assessments include observation data, a self-rating, questionnaires administered to classes, and an improvement plan.

In at'dition, the courses are evaluated by job incumbents and their supervisors.

Both job incumbents and supervisors can submit Training Request Fonns as part of the evaluation process.

In this case, the training

~~

additions and improvements are tracked to ensure implementation and are followed through evaluation. The Quality Assurance Group is responsible for the overall external audit of training programs.

In addition, internal audits are conducted of various segments of the training program using a sampling technique as described previously. These internal audits are conducted through the Training Development and Evaluation Program.

C.

Sumary of Findings The post-accreditation audit at WNP-2 indicates that a thorough analysis of jobs and tasks has been conducted.

Of particular note is the extremely thorough, in-depth comparison analysis of existing training, making it an acceptable alternative to a full job task analysis.

As with other utilities, there is some inconsistency in the quality of learning objectives, but the new procedure being implemented should ensure that learning objectives will be well constructed in the future.

The Near Tenn Training Review conducted at WNP-2 was a thorough method

4 not only of reviewing existing training but also of determining appropriateness of ee?!encing and training media. On-the-job training is adequately covered but a more comprehensive training program is needed for OJT reviewers.

The program evaluation method used by WNP-2 is one of sampling and in-depth evaluation.

Because of its sampling nature, the evaluation did not fully meet the NUREG-1220 review criteria for the evaluation element of performance-based training which calls for evaluation of every program. During the staff's review of both the method and its supporting documentation, however, it became clear that this sampling technique produced a comprehensive review of training.

_-

Retrieved from "https://kanterella.com/w/index.php?title=ML20149C346&oldid=4169913"