ML20246M516
| ML20246M516 | |
| Person / Time | |
|---|---|
| Site: | Arkansas Nuclear  |
| Issue date: | 06/28/1989 |
| From: | Mccrory S, Pellet J NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION IV) |
| To: | |
| Shared Package | |
| ML20246M484 | List: |
| References | |
| 50-313-OL-89-01, 50-313-OL-89-1, NUDOCS 8907190168 | |
| Download: ML20246M516 (8) | |
Text
y m.. - -
i: p-:
WQ' ' .
,'s '
- .. < +
}
J
- g. :k .
.' e ? , ; ,
,q 'H #
r i C
(- ' '
L s .
C
- APPENDIX _
a "- -
U.S.-NUCLEAR REGULATORY COMMISSION'
- l REGION IV >
o.
c - y :0perator Licensing Exam Report: 50-313/0L 89-01 Operatin's License: lDRP-51 ,'
og '
F i Docket No: 50-313' ,
i..".
2 -Licensee: A* kansas: Power.&' Light ..
-t
~
P.0. Box.551 . .
h' -
n Little Rock AR 72203 ,
y , ,
Facility Name: Arkansas Nuclear One, Unit 1 3 ,
c, . : ..
' Examination at: -ArkansasNuclearOne, Unit 1(ANO1) '
s ,
Chief. Examiner.
/ / $ .
- 5. LT McCrory Examin g : JTate/ /
Operator. Licensing Settion, Division of Reactor Safety Approved by:
& L. Pellet, Section Chief.
hMh Dath Operator Licensing Section.
Division of Reactor Safety.
Summary NRC Administered Requalification Examinations Conducted During the Week of
~
L May 1, 1989 (Report 50-313/0L 89-01)
.NRC administered requalification examinations'to eight senior reactor operators (SPO) and four reactor operators.-(RO) licensed to operata the ANO 1' facility. Two senior operators failed the written examination. All other operators ~ passed all portions of the examinations. The senior operators were reexamined on May 30, 1989, and passed the written examination.
p 1 i
);R71$$Nok3 V
e _
., , : n, ,
= -_ . _ _ . . _ - _
- t. . oi The overall performance on these examinations was notably below the level of performance that has been observed in recent license examinations at ANO.
Additionally, there was evidence of persistent weaknesses noted in previous audits and inspections. Written requalification examination performance was marginal as was observed in the previous requalification audit. Simulator performance was at a notably lower level of proficiency than observed in the last requalification audit. Performance during these simulator examinations was such that it may have resulted in an unsatisfactory evaluation in some areas in an initial licensing situation. Reduced operational proficiency was also observed during an emergency operating procedure (EOP) team inspection conducted in May-June, 1988. During that inspection, experienced licensed operators demonstrated operational weakness sufficient to result in the voluntary remediation of at least one operator.
Further, some senior operators, other than those examined by the NRC, failed at least one part of the facility administered requalification examinations in the weeks preceding the NRC administered examinations. According to training staff personnel, most of the senior operators failing the requalification examination are participating in the facility sponsored degree program. This indicates possible deficiencies in the requalification program's ability to compensate for the time that operators in the degree program spend in dedicated academic studies. The NRC does not advocate discontinuing the program, but an evaluation of how operators in the degree program are kept current in requalification is warranted.
The facility training staff was both responsive and effective in understanding and correcting test item construction deficiencies. The numerous construction and compliance deficiencies in test items submitted are a common occurrence for the first submittal under the new requirements of ES-601. The facility representatives to the examination team were particularly valuable during examination development. They provided the team with operational insights that could not be obtained just from lesson plans and training materials.
At this time, NRC concludes that there is not a significant safety impact resulting from the weaknesses noted above. Although no specific facility response is required to this examination report, the NRC encourages the facility licensee to evaluate current requalification program and operational practices and identify the basis for the apparent decline in operator performance.
I I
J
J h i DETAILS
- 1. Persons Examined SR0 R0 Total flequalification Examinations: Pass - 6 4 10 Fail - 2 0 2
- 2. Examiners S. L. McCrory, Chief Examiner J. Pellet
- 1. Eaton F. Morgan
- 3. Examination Report Performance results for individual examinees are not included in this report because examination reports are placed in the NRC Public Document Room as a matter of course. Individual performance results are not subject to public disclosure,
- a. Exarr " .lon Develor.nent Test items for the written, simulator, and walkthrough examinations were submitted to the NRC as prescribed by NtJREG 1021, Operator Licensirg Examiner Standards Section 601 (ES-601). During the initia', review of. the written examination test items it was apparent that the facility did not fully understand the intent of ES-601 as to the scope and focus of the different sections of the written examination. Specifically: i (1) Basic theory questions were included in the question bank for Section B, Limits and Controls. It is not the intent of the requalification audit / examination process to focus directly on basic theory areas. On feedback from the Chief Examiner, these questions were removed from the requalification question bank.
(2) Several items in the question banks for each section of the written examination did not apply to the section to which they had been designated. Section A. " Plant Operations," (Static Simulator), focuses on the operator's-knowledge of plant system design, operation, integrated operation, and instruments and controls. Included in this are LCO recognition and event diagnosis. Several questions designated for Section A did not ,
focus on these areas. Most of them related to procedural and administrative guidance and were more appropriately applied in Section B of the written examination. Further, there were very k
_ _ _ . - _ _ ___ _ D
t few " generic" system questions. As a result, essentially all the Section A questions addressed a specific static simulator event. This tends to limit the scope in each part of Section A to too few systems or components.
Section B, " Limits and Controls " focuses on the ability of the operator to analyze a given set of conditions and determine the proper procedural and administrative guidance using Technical Specifications, Emergency Plan Implementing Procedures, or normal, abnormal, and emergency procedures as appropriate.
Several questions designated for this section tested system knowledge and were more appropriately asked in Section A. This misunderstanding of what were appropriate questions for Section B seems to have resulted from what was used as the reference. I for the answer key. If the question was referenced to a-specific procedure, it apparently was put in the Section B question bank without closely evaluating what area of knowledge was really being tested by the question.
These issues were identified to the facility staff during the item review and examination development phases.
The simulator test items submitted for review failed to adequately identify individual simulator critical tasks (ISCTs), time critical tasks, and performance criteria for evaluation of the ISCTs and time critical tasks. ' Again, these deficiencies tended to be the result of a lack of familiarity or misunderstanding of ES-6'". rather than inability to develop test items containing these elements. These deficiencies were readily corrected by the facility staff when identified.
Job Performance Measures (JPMs) submitted for use in the walkthrough examination were generally very good with the ev.ception of the followup questions, which tended to be simplistic or direct look-ups.
Training personnel were informed that the questions used with the' JPMs were subject to the same construction and content requirements as those developed for the written examination. They were also told that it was not neressary to' develop a JPM only question bank, but that system questions developed for Section A of the written examination may used as JPM followup questions.
During the week of April 3,1989, the NRC and facility designated members of the examination development team met at the facility for initial onsite review and validation of the previously submitted I
test items. At this time all JPMs were validated by an NRC examiner and most of the revised written examination questions were reviewed for construction deficiencies. Simulator scenarios were reviewed against the requirements of ES-601 with the deficiencies previously l
l
. - _ _ __ D
l L
j . .
1 b l noted being fed back to the individual who had developed the scenarios. The facility examination team members were generally competent at correcting the item construction deficiencies identified by the NRC reviewers on the examination team.
During the week of April 17, 1989, the Chief Examiner returned to the facility and met with the facility examination team members to construct and time validate the examinations. The team used the facility developed sampling plan to designate test items from which the final examination would be constructed. The team was careful to minimize subject overlap between the different examinations to provide the broadest coverage possible. The facility team members were tasked to select 50 JPMs and propose the examination packages, consisting of 25 JPMs each, for two crews including identification of the common JPMs. The JPM examination package for the third crew was then assembled by the Chief Examiner from the 50 originally proposed JPMs. The simulator scenarios to be used for the three crews were identified by the Chief Examiner and partially previewed in the simulation facility with the facility team members. Finally, the written examination was time validated by having the facility team members take the examination while attempting to use reference material'to answer.each question.
During the. week of April 24, 1989, the Chief Examiner designated the scenarios to be used for each simulator examination and designated q the JPM packages to be used for each individual being examined.
This information was then provided to the facility evaluators who would be performing the facility evaluation during the examination week. This was considered appropriate and necessary to allow the facility evaluato , time to become familiar with the test items they would participate in administering and to allow for logistic j planning (particularly during the walk-through examinations). '
- b. Examination Administration The examinations were administered during the week of May 1, 1989. i Sections A and B of the written examination were administered 1 i
concurrently with two crews taking Section B while one crew took Section A. When the crews were rotated to the different examination locations, one of the two having taken Section B was placed in a l holding room, which was monitored by an examination team member. The remaining crews exchanged examination locations and continued the written examination. When the second crew had completed Section A, the third crew was released from the holding area and allowed to !
take Section A. When a crew had completed all parts of the written examination, they were directed to leave the training facility. ;
Approximately 2 1/2 hours were needed for each session of Section A l to allow for simulator setup and crew rotation. One of the two l
y .
L .
static simulations behaved differently than it had during previous usage in that one of the redundant components did not respond as previously observed. .This eventually led to the deletion of one question from Section A. At the completion of all parts of the examination by all examinees, copies were made of the examinations and given to the facility staff-to allow immediate parallr.1 grading to begin. NRC grading of the written examination was done onsite at the end of the first day. Facility grading was completed by the second day. Grading results were compared and differences addressed on the evening of the third day. NRC results were' finalized later the same evening'away from the site and reported to the facility-staff the next morning.
On days 2 through!4 of the examir.ation week, one crew was examined each day. Simulator examinations were administered during the morning and walkthrough examinations in the afternoon. Simulator examinations took approximately 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> to administer including time for processing evaluations and conducting crew critiques. Both the facility evaluators and NRC examiners disclosed the pass / fail decisions to each crew at the end of the crew critique. There were no disagreements between facility and NRC final results on the simulator examinations. ,
Walkthrough examinations took approximately 4 to 5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br />. This was mostly due to coordinating use of the simulator to conduct JPH tasks.
Frequently an examinee would have to wait for another examinee to i finish a JPM to allow the first examinee access to the panel or to reset the simulator to a different initial' condition (IC). It was observed during the simulated JPMs with examinees that the operational cues were weak and sometimes confusing. The facility evaluators were reluctant to go beyond verbatim repetition of the cues written on the JPMs. This matter was clarified somewhat after the first day's walkthrough examinations. NRC examiners infonted facility evaluators that we expected them to rely on their plant system and operational exper'_,se when cuing examinees during JPMS and to formulate cues appropriate to the existing conditions and the actions of the examinees.
There was a noticeable improvement on the part of the facility evaluators in this area on subsequent days. At the end of each walkthrough examination the facility evaluator and NRC examiner compared results and resolved matters affecting pass / fail determinations.
These results were disclosed to the examinees at the end of the examination week.
- c. jxaminationResults After meeting with the facility staff throughout the examination week to compare results and resolve differences the following final results were obtained:
1 1
_ - - _ - _ _ - - _ _ _ - _ - - _ _ ~ _ _ _ _
~
(1)'Twosenior.operatorsfailedthewrittenexamination. All other examinees passed the written examination.
(2) All examinees passed the simulator examinations both as individuals and as crews.
(3) All examinees passed the walkthrough examinations.
(4) NRC and facility results for individual examinees and crews agreed within the requirements of ES-601 on.all examinations.
(5) , All elements of programmatic performance met the requirements of ES-601 for a satisfactory program.
- d. General Comments The results of this requalification audit were compared to those of previous audits. . Unit 1 operators continue to demonstrate marginal performance on written examinations. While only two operators failed the written examination, additional operators had marginally passing grades. This is indicative of a possible generic weakness in the requalification program.
Simulator performance was noticeably lower than in previous audits.
While performance during this audit was evaluated as satisfactory, previously observed performance was outstanding by comparison.
Artificialities in crew composition cannot mitigate the performance during this audit since crew mixes for former audits were even less representative of actual shift composition. Nor can examination format explain performance differences since simulator for~st has remained virtually unchanged from previous audits. Additionally, during an E0P team inspection of Unit I led by a certified operator license examiner in May-June,1988, experienced shift operators demonstrated performance deficiencies during simulator operation.
At least one operator was voluntarily remediated by the facility as a result of this inspection. Even though the NRC concludes that there is not a challenge to safe operation of the facility at this time, it is concerned by this apparent trend. The facility training and operations staffs are strongly encouraged to assess this area of reduced performance.
To the credit of the training staff, they were very cooperative and responsive during both the development and administration phases of the examination / audit. Several differences in perception and understanding of the requirements of ES-601 between the facility training staff and the NRC were evident in the initial item review and examination development phases of the audit. However, facility personnel were quick to grasp the actual NRC intent behind ES-601 1
}
L c*., j
~
and to revise test items to conform with this intent. Throughout the process, the staff demonstrated the ability to improve test items-both in construction and focus.
.e.- . Site Visit Sumary The NRC held an' exit meeting with members of the facility licensee staff. The examiners indicated that no system or procedural generic weaknesses had been observed during the operating examinations. The facility representatives were told that the requalification program--
.was evaluated as SATISFACTORY and that this represented final results since NRC management review had been performed onsite.
Most of the items reported above were reviewed during.the exit. No specifics were discussed.since:these had been reviewed in detail with the training staff,in a preexit' meeting and are addressed thro 0ghout this report. The following personnel were present at the exit: ,
- 1 NRC ' Facility Licensee S. McCrory E. Force
'J. Pellet W. Perks-J. Milhoan~ J. Vandergr' ft W. Johnson. S. Quennoz
+
N. Carns
.J. Levine-A. Cox i
i
.. -