IR 05000336/1986025
| ML20207S250 | |
| Person / Time | |
|---|---|
| Site: | Millstone  |
| Issue date: | 03/04/1987 |
| From: | Coe D, Collins S, Keller R NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML20207S244 | List: |
| References | |
| 50-336-86-25OL, NUDOCS 8703190191 | |
| Download: ML20207S250 (5) | |
Text
r-
..
.
'
,
'
,
q
..
.>
,
'
a
[
'
U.S. NUCLEAR REGULATORY COMMISSION
'
REGION I
,
~ Evaluation Report No. 86-25(0L)
/
'
'
'
Facility Docket No.
50-336 Facility License No.
OPR-65
.
,
Licensee: Northeast Nuclear Energy Co.
-
'
P.O. Box 270 Hartford, Connecticut 06141
.
,
Facility Name: Millstone Unit 2 Examination Dates: November 13, 1986 and January 5-16, 1987
/2c[$7 Chief Examiner:
.
D. Coe, Lead Reactor Engineer date-4. 84////.
o2/h /, d7 Reviewed By:
a cw<<
R. Keller, Thief, PFojects Section 1C
/ dath
,
Approved by:
/h(7)1(ibbIIV)M 3l4lU 5'. Collins,.. Deputy Director date Division of R4 actor Projects Summary: The Millstone Unit 2 annual requalification program was audited.
This included observation of one operating examination and para;lel grading'of the written examination for a sample of approximately 20 percent of the operators licensed on Unit Two.
The program was given an overall satisfactory "
evaluation based on NRC agreement with the licensee's written examination
'
,
-
results and an NRC audit of thi. operating examin'ation process.
Future NRC
't audits of this program will be conducted to more fully assess the licentee's effectiveness in identifying individual weaknesses during the simulator '
portion of the operating examination.
'
,
'
3'
,
'
,
s G703190191 870310
^
,
PDR ADOCK 05000336
,
i eV PDR
-
.
-
-_
,
. -
-.. _. _ _ -
.-.
-.
_....,... _ -
.
.
.
zu
'
,
,
v
,
DETAILS 1.
Examination Results R0 '
SRO Total Pass / Fail Pass / Fail Pass / Fail Evaluation Written 1/0 7/2 8/2 Satisfactory Simulator 1/0 2/0 3/0 Satisfactory Evaluation of Facility Written Examination Grading: Satisfactory Overall Program Evaluation:
Satisfactory
+
~
'
2.
Scope
,
The facility prepared R0 and SRO written examinations were reviewed by the NRC priur to administration and ten operator's answers were graded by
'
the NRC in parallel with the facility. The NRC audited the facility administered simulator examinations of three licensed operators, and compared the NRC evaluation of the operator's performance with the facility evaluation. One oral examination administered by the facility was audited.
Finally, the effectiveness of the facility evaluators
during the simulator and oral examination was observed.
3.
Review and Grading of the Written Examination i
The annual written requalification examination prepared by the training department is identical in format to an NRC replacement written examina-
,
tion. rhi order to provide examination security and integrity of results,
~
each section of the examination is composed of three groups of five or six
'
questions.
For a particular license level and for each examination sec-tion, one group of the three is present on all examinations. The other two groups are selected randomly such that each individual examination is unique in terms of the total questions asked. Questions in each group cover similar topical areas within a given examination section so that any random permutation of three groups gives similar coverage and difficulty within the section. This method, although involved, appears to provide adequate examination security while maintaining consistency of examination results and the ability, to identify generic weaknesses. The questions were found to cover a b'rcad range of knowledge, to be of sufficient dif-ficulty, to examine at an appropriate depth of knowledge, and to be
operationally oriented in nature.
,
The quality of the written examination was sufficient to make it compar-
.
able in content and structure to an NRC standard licensing examination.
The NRC audited the grading of ten written examinations which constituted
approximately 20 percent of the facility's licensed operators. The NRC l
.
n
,
- i f'
'
,.
, -..--
.---
-
.-,. - -. - -.. -..
,
.-
,
-,--,
. _,,.,
- - -. -
q
.
.
determination of passing and failing grades agreed with the facility's with one exception. A borderline passing grade by the facility on one section of an SR0 examination was given a failing grade by the NRC. The facility subsequently changed its grade to a fail.
Two operators within the sample audited failed the written examination overall. One of these was l
the above indicated sectional failure. The facility results for all licensed operators taking the written examination were six (6) sectional
,
er overall failures and forty (40) passes. Both the NRC sample and over-
"
all facility pass rates were acceptable for an evaluation of satisfactory on the written examination portion of the requalification program.
In addition, the written examiration was able to identify generic as ~well as individual training weaknesses.
4.
Audit of Simulator and Oral Examinations One NRC examiner audited one simulator examination comprised of two simulator scenarios given to a group of three licensed operators being evaluated by the facility. Three facility evaluators were present and included both operations and training staff personnel. The scenarios developed by the~ training staff were adequate to evaluate the operators i
over a range of normal, abnormal, emergency, and administrative proce-dures.
Each scenario had an individualized evaluation form which listed specific proper actions expected to occur during the scenario.
Each operator was evaluated against these actions' and given an overall grade of pass or fail. After the scenario terminated, facility evaluators conferred, consolidated their general evaluation, and gave immediate feedback to the operators prior to proceeding to the next event.
In one case the NRC examiner felt that immediate follow-up questioning after the scenario would have been appropriate to assess the individual knowledge level of operators whose actions contributed to poor team per-formance during the use of Emergency Operating Procedures.
The NRC examiner noted that giving immediate feedback tends to preclude individual followup questioning needed to ascertain which individual weaknesses were contributing to an overall problem with the operating crew's perform:nce.
The pralctice of immediate feedback may be appropriate during the conduct of sin.ulator training sessions, but can interfere with an evaluation.
,
In some cases, an overall crew performance problem was documented equally on individual ope'rator simulator evaluation forms without differentiating between the varying individual weaknesses which contributed to it.
The simulator examination is a portion of each operator's individual evalua-tion and should be conducted such that individual weaknessss can be fully determined.
After the completion of all simulator scenarios, each facility evaluator
'
asked a set of oral examination questions to the operator he was evalu-ating.
This discussion took place in the simulator. The list of ques-tions was the same for R0 and SRO operators and was comprised of nine
/
'd L
.
.
questions in each of three sections: Theory of Plant Operation, Plant Systems: Design and Operation, and Procedures and Administrative Require-ments. The answer to each question was included on the list for the benefit of the evaluator. Although the questions were identical for both R0's and SRO's, many of them covered SR0 level knowledge and were there-fore adequate to assess the knowledge level of all operators examined.
For examination security reasons, the facility evaluators do not receive the oral examination questions and answers until the day of the examin-ation.
The NRC noted that this tends to limit the ability of the eval-uators to develop additional background and depth in the topic areas presented, or to question answers they may feel are incorrect or not complete.
In one case, a facility evaluator made an on-the-spot modifi-cation to one of the listed answers based on his judgment that the answer given on his list was not entirely correct.
Although the instructions to the evaluator for administering the oral examination encourage additional depth of questioning in the given topic areas or lines of questioning in entirely different areas, the one oral examination observed by the NRC kept closely to the given questions / an-swers. The content of the oral examination was found to be adequate to identify individual and generic weaknesses.
The NRC audit of the simulator and oral examinations consisted of only one simulator crew and one oral examination.
Due to the small sample size, conclusions regarding the general training effectiveness of the requalification program in the areas of operations, use of procedures, communications, and teamwork could not be drawn. However, the NRC observed a simulator and oral examination which showed good quality of preparation and which was executed by competent evaluators. The completed evaluations indicate that individual and team performance deficiencies are being detected and documented.
5.
Exit Interview An exit interview was held on December 17, 1986 following the audit of the operating examination. The following persons were present:
l NRC Personnel D. H. Coe, Chief Examiner Facility Personnel H. F. Haynes, Manager, Operator Training l
J. S. Keenan, Unit 2 Superintendent J. F. Smith, Unit 2, Operations Supervisor l
M. J. Wilson, Unit 2 Operator Training Supervisor
'
J. Becker, Unit 2 Operator Training Assistant J. Parillo, Unit 2 Operator Training Assistant l
.-
,...
6.
Summary of NRC Comments made at exit interview:
A discussion was held concerning the preliminary observations made during the simulator examination. The objectives and methods of this requalifi-cation program evaluation were also discussed, including plans to audit the written examination grading in January 1987. No major findings were presented.
7.
Conclusion The Millstone Unit Two requalification program is evaluated as satisfac-tory. An NRC concern was raised regarding the effectiveness of the facil-ity administered simulator examination to fully determine all individual weaknesses.
Future requalification program evaluations will continue to monitor this area.
Furthermore, the small sample of operating examina-tions limited this audit to an assessment of the licensee's operating examination process and precluded drawing firm conclusions regarding the general training effectiveness of the requalification program in those areas which are best evaluated during the operating examination.
The licensee's program to upgrade operator knowledge in accordance with their NRC-approved requalification program, for operators who failed portions of the examination, will be monitored by the next requalification program inspection. There were no instances of noncompliance with NRC regulations found during this evaluation.