IR 05000282/1988020

Insp Repts 50-282/88-20 & 50-306/88-20 on 881003-07.No Violations Noted.Major Areas Inspected:Licensed Operator Requalification Training & Verification of Commitments Made by Licensee Re Generation Test Items

ML20195D776
Person / Time
Site:	Prairie Island
Issue date:	10/26/1988
From:	Burdick T, Hare S, Sunderland P NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III)
To:
Shared Package
ML20195D765	List: IR 05000282/1988020 ML20195D761
References
50-282-88-20, 50-306-88-20, NUDOCS 8811070133
Download: ML20195D776 (7)

Text

- _ _ _ _ - _ _

.

• '. .

'

O.S. NUCLEAR REGULATORY COMMISSION

REGION III

Reports No. 50-282/88020(DRS);50-306/88020(DRS)

Docket No. 50-282; 50-306 Licenses No. DPR-42; DPR-60 1.icensee : Northern States Power Company 414 Nicollet Hall Minneapolis, MN 55401 Facility Name: Prairie Island Nuclear Generating Plants. Units 1 and 2 Inspection At: Red Wing, Minnesota j Inspection Conduct d: 0.tob 3-7, 1988 Inspectors: S. FC re

/d N Dat6 i LA n P. R. Sunderland /26[68 Date Approved By: . Bu d ief /0 Operator Licensing Section 2 Date Inspection Summary

.

Inspection on October 3-7, 1988 (Reports No. 50-282/88020(DRS);

! No. 50-306/88020(DRS))

i Areas Inspected: Special unannounced inspection by region based license examiners / inspectors in the area of licensed operator requalification training. This included follow-up on a previous inspection finding; verification of comitments made by the licensee regarding their generation of test items to support the new requalification format; the adequacy of ( their test items generated to date; and the adequacy of their license operator evaluations using the new test items. Selected portions of Modules 92701, 30703 and 41701 were u.est as guidance during this inspectio Results: The licensee has successfully began implementing the new requalifica?. ion program testing and evaluation methodologies as described in NUREG 1021. ES-601. Their progress in the areas reviewed exceeded what they had comitted to; was of good quality; and was used effectively in the annual evaluation of :nsed operators. Of the four areas reviewed, no violations or devia. W were identifie PDR ADOCK 05000282 O PDC

_ ___ _

_ _ _ _ _ _ _ _ _ _ .

. .

'

DETAILS Persons Contacted Northern States Power

• E. Watzl, Plant Manager D. Mendele, Plant Superintendent Engineer

• L. Waldinger, Manager Production Training

• M. Sellman, General Superintendent Plant Operations Training

• T, Amundson, General Superintendent of Training

• D. Reynolds. Operations Training Supervisor

• L. Anderson, Shift Manager

• D. Wagenmann Shift Supervisor D. Westphal, Training Staff K. Carlson, Training Staff US NRC

• J. E. Hard, Senior Resident Inspector

• Denotes personnel present at the October 7,1988 exit intervie The inspectors also contacted c.her licensee staff members during the course of the inspectio P

- _ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ _ _ _ _ _ _ _

.

.

2. Followup on d revious Inspection Finding (Closed) Unresolved Items (282/88003-01 and 306/88002-01): No makeup was required for missed cycle training or for missed or failed cycle quizzes in the licensee's requalification program. The inspectors reviewed the training records for ten randomly selected operators covering rycles one through five of the 1988 requalification program to verify compliance with 10 CFR 55.59 and the licensee's Licensed Operator Requalification Program (P9100) descriptio Per P9100, which had been revised since the last inspection, licensed operators are required to attend all lectures and quizzes that they are designated to receive during the requalification cycle. Makeup of missed items was accomplished by attending another lecture, by viewing the lecture video tape, or by taking a quiz, as appropriate. Absences are to be made up within twelve weeks and completion of missed lectures should be limited to no more than twenty percent of the scheduled training over a two-year period. The review detennined that the licensee had successfully imple-mented this section of P9100. A general exception to P9100 is that if a holiday or plant drill occurs when requal training is scheduled, there is no makeup session. After records review and consultation with the licensee training staff, the schedule minimized the number of times this happened to each cre P9100 also addresses periodic quizzes. The minimum satisfactory grade for a quiz is 80%. If more than one quiz is given during a requal week, the average of the quizzes must be at least 80% with no quiz less than ,

70%. If the results are less than the above standards, make-up quizzes '

are required. Failing a make-up quiz requires remedial training and retesting. The licensee has successfully implemented these standard Licensee records regarding the requalification program are kept with precision. They have generated an informal spreadsheet tracking system that fully supported what was found in individual training record . Verification of Licensee Comitments On July 22, 1988 in a letter from the NRC to Northern States Power (NSP), the NRC notified NSP of their intention to administer NRC written and operating tests to Prairie Island licensed operators the week of October 24, 1988. These examinations were to follow the guidance contained in Revision 5 to NUREG-1021, ES-601 which described a new requalification examination process. The letter further outlined materials which were to be submitted to the NRC 60 days prior to the examinatio In a letter dated August 9, 1988 from Mr. C. E. Larson, the NRC was notified that the licensee was unable to meet all of the requirements for submitting information to the NRC to support written and operating requalification exams. This letter also contained details of an action plan which had been initiated to prepare for the new

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _

.

..

requalification exam format in addition to discussing licensee's present and planned progress toward that en The following commitments were made in this letter by the licensee:

Three to four simulator scenarios that confonn to the new guidance will be completed by August 24, 1988 with fifteen scenarios expected to be completed by March 31, 198 '

Approximately 65 open reference and static simulator questions will be completed by August 24, 1988 with an additional 30 to 60 questions being completed each subsequent month

Five Job Performance Measures (JPMs) will be completed by August 24, 1988 with approximately ten additional JPMs produced each month thereaf ter until a minimum of 75 are produce The inspectors reviewed licensee action in each of these areas and noted the following:

Nine simulator scenarios had been generate *

116 limits and controls and 43 static simulator open reference questions had been generate *

38 JPMs had been generate While some deficiencies were noted in the quality of the test items generated (see Section 4 of this report), the licensee's performance in this area is considered excellen . Adequacy of Licensee Generated Requalification Test Items The inspectors reviewed the test items the licensee had developed to ascertain the adequacy of the items relative to the requirements contained in Revision 5 of ES-601 of NUREG-102 Comments on the reviewed test items were shared with the licensee's training staff to ensure a clear understanding of the problems encountered by the inspectors. The following paragraphs address each of the new test item categories along with the scope of the inspectors review and their subsequent findings, Simulator Scenarios The licensee had developed nine scenarios at the time of the inspection. Each scenario consisted of events covering normal evolutions, component and/or instrument failures and emergency evolutions. Each event was documented with a "Simulator Evaluation Form" (SEF) in which the event description and the team / individual expected response to the ever,t was documented (the actual use and purpose of SEFs is better described in ,

Section 5 of this report). Through discussions with training

_

-

.

.s .

personnel and the review of the scenario sets, the inspectors determined that the licensee generated "generic" events and then combined these events together into simulator scenarios. The inspectors reviewed five of the nine available scenarios and noted the following areas of weakness:

The scenarios appeared disjointed due to the generic nature of the SEF *

Due to the generic nature of the SEFs, items / steps within an SEF could be critical tasks depending on events preceding that particular item / ste *

Critical tasks were not consistently defined / applied between scenario *

The operator expected responses were at times not well defined and were too generi In general the above weaknesses can be traced back to the generic nature of the SEFs and resultant simulator examinations. During discussions with responsible Training Management before and at the Exit Interview, the inspectors stated the generic methodology used to generate the scenarios was a good step in the right direction. However, to meet the guidance in ES-601, the SEFs for d simulator scenario must be made more specific and better delineate operator expected responses in addition to the critical and non critical team / individual tasks, b. Open Reference Written Examination Questions The open reference portion of the reoualification process consists of two distinct parts; the static simulator exam section and; the limits and Controls section. The licensee had tiot yet implemented this portion of the new requalification process into their annual requalification examinations (written open reference questions were scheduled for the next cycle). The inspettors evaluated 43 static simulator questions (from four quizzes given to date) and 116 limits and control question (1) Static Simulator Open Reference Questions The licensee had administered four quizzes using static simulator questions to dat These quizzes were the subject of the inspector's review. The quizzes were well developed with questions that were in line with the NRC intent for static simulator examinations. The review revealed instances where information was provided in the body of the question which could have just as easily been obtained by an operator during the review of the static simulator control board. The quality of questions developed in this area was considered goo _ _ _ _ _ _ _ _ _ _ _ _ - _ _ _ . _____

.

"'i.

.

.

(2) Limits and Controls Open Reference Questions Of the 116 questions reviewed, it appears that many of the questions were pulled out of previously administered closed reference examinations. Approximately 40% of the questions reviewed were of the "lookup" type where the operator is required to look up facts in almost a "state" or "list" hierarch Another 15% were more applicable to a closed reference rather than an open reference exam fonnat. In addition, the licensee should be careful in their use of Yes/No or True/ False questions for this portion of the examination. Questions in this format allow a guess to have a 50% chance of being correct. Questions such as these are not in line with ES-601 and the NRC's intent for this section. The questions in this section should have the operator using the references normally available in the control room to analyze or explain a situation. This is the licensee's weakest area in requalification test item developmen Job Performance Measures The licensee had developed 38 JPMs at the tima of the inspectio Each JPM cusisted of two distinct parts, the first part being a specific job task in which an operator / examinee is expected to perform / simulate with a minimum amount of cueing from the examine The second part of a JPM consists of several follow-up questions which an examiner will ask the operator / examinee after the specific job task is complete The inspectors reviewed all 38 JPMs and noted the following:

(1) The job tasks were well documented with operator actions and associated critical tasks well defined in almost all case (2) The follow-up questions were at times overly simplistic or not related to the specific task being performe (3) The JPMs reviewed were of good quality and appeared to meet the intent of the new requalification process as described in ES-60 Licensee training personnel indicated to the inspectors prior to their review that the JPMs had not been evaluated for their importance relative to the Knowledge and Abilities catalog. Therefore the inspectors did not evaluate this aspect of the JPMs. Licensee attention in this area should concentrate on the validation of the JPMs through their usage; and the generation of follow-up questions which are more directly related to the specific task being perfome Although weaknesses surfaced during the inspectors review of the licensee's test items, licensee performance in this area is considered very goo _ _ _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ - _ _ - _

_ ___________ _

,

.  !

.. .* . ,  ;

.

5. Adequacy of Licensee Evaluation Techniques The inspectors individually followed two training staff evaluators while they were performing yearly requalification examinations to detemine the adequacy of their evaluation techniques. Since the licensee had not fully implemented the new requalification exam format for their yearly examinations (written open reference questions were scheduled for the next cycle), the inspectors were only able to l

witness the simulator examinations and the Job Perfonnance Measure walkthroughs, Simulator Examination Witnessing A yearly requalification simulator examination consists of two individual scenarios with each scenario taking approximate y one hour to run. One team of four operators (RO, Lead R0, Shift Supervisor and Shift Manager) is tested on the simulator where they respond to the different events (as documented on the SEFs). The team of four evaluators use the SEFs to evaluate the adequacy of team / individual responses to the different events in the simulator set. The inspectors followed two individual evaluators during the simulator exams and afterward compared notes with the evaluator The inspectors were in agreement with the evaluators' note The inspectors then attended a post simuletor exam briefing where all of the evaluators debriefed and discussed the problems with the team / individuals that they had noted. This debriefing was performed to consolidate thoughts on the team / individuals performance and to come to a pass / fail determination for the team / individuals. While difficulties were encountered during the debriefing on a pass / fail detennination for the team (due to the generic nature of the SEFs), the inspectors thought the briefing process was exemplary, JPM Walkthrough Witnessing A JPM walkthrough consists of an evaluator having the operator perform ten specific tasks as delineated on a JPM and at the conclusion of the task, asking the operator the appropriate follow-up questions. The inspectors followed the sane evaluators during the JPM walkthroughs and afterward compared notes with the evaluators. The inspectors were in agreement with the evaluator's note . Exit Interview The inspectors met with licensee representatives (denoted in Paragraph 1)

at the conclusion of the inspection on October 7,1930 and sumarized the scope and findings of the inspection activities. The licensee acknowledged the inspector's statement The inspector discussed the likely informational content of the inspection r.'. pert with regard to documents reviewed by the inspector during the ;nspection. The licensee did not identify any such documents as proprietary.