IR 05000220/1993009
| ML20045G647 | |
| Person / Time | |
|---|---|
| Site: | Nine Mile Point  |
| Issue date: | 07/01/1993 |
| From: | Conte R, Walker T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | |
| Shared Package | |
| ML20045G646 | List: |
| References | |
| 50-220-93-09, 50-220-93-9, NUDOCS 9307150003 | |
| Download: ML20045G647 (12) | |
Text
.. -.
'
.
.
U.S. NUCLEAR REGULATORY COMMISSION
REGION I
INSPECTION REPORT DOCKET NO:
50-220 REPORT NO:
50-220/93-09 LICENSE NO:
DPR-63 LICENSEE:
Niagara Mohawk Power Corporation 301 Plainfield Road
,
Syracuse, New York 13212 FACILITY:
Nine Mile Point Unit 1 DATES:
June 7-11,1993 INSPECTORS:
T. Walker, Senior Operations Engineer S. Hansell, Operations Engineer L
Y//43 d
'
LEAD INSPECTOR:
-
'Tracy Walker, Sr. Operations Engineer Dit$
BWR Section, Operations Branch, DRS
&
- ", I APPROVED BY:
. Richard J. Conte, C$Ief, BWR Section Date Operations Branch, DRS 9307150003 93070s PDR ADOCK 05000220 G
--.,._ _
.
__
_
, _ -,
,
.
.. -
.
.
-_.
.
.
-..
...
...
-.
-.
.
-
--
.
.
.
.
INSPECTION SUMMARY: Insocction conducted June 7 - 11. 1993 Onspection Report No. 50-220/93-09)
An announced safety inspection of the licensed operator requalification training program was performed to ascertain whether or not Niagara Mohawk was effectively performing those activities necessary to evaluate and ensure an adequate level of competency for licensed operators who operate the Nine Mile Point Nuclear Station Unit 1.
The requalification examinations that were administered appropriately sampled the areas that were covered in the requalification program and the items specified in 10 CFR 55. The examinations were adequate to evaluate both crew and individual competencies and the evaluators effectively evaluated operator competency. There was no indication of examination compromise.
It was determined that the Nine Mile Point Unit I licensed operator requalification training (LORT) program is being implemented in accordance with the requirements of 10 CFR 55.59 for the areas reviewed. The licensee effectively evaluates the LORT program based on operator performance. The interface'between the Operations and Training Departments was noted as a strength of the program. No violations or programmatic weaknesses were identified.
I
!
f
!
i
.
-
-
.
-
.
.
_
_ _ _ __
_
_- __
_.. _ _
_
__
_ _ _ _.
_
_
_
. _ _
.
DETAILS 1.0 INTRODUCTION NRC Temporary Instruction (TI) 2515/ll7, " Licensed Operator Requalification Program Evaluation," was developed because of proposed rulemaking that would delete the requirement that the NRC staff examine cach licensed operator for the purpose of license renewal. The purpose of this TI was to evaluate the acceptability of licensed operator requalification training programs through a performance-based inspection of the evaluation process and, to some extent, program revision as a result of the evaluation process. This
was done in lieu of a NRC staff requalification examination of licensed operators, the results of which were used in the past to perform a program./aluation.
2.0 REQUALIFICATION EXAMINATIONS SCOPE The inspectors reviewed the requali0 cation examination materials for the requalification examination cycle, including the Sample. Plan, written examinations, dynamic simulator scenarios, and job performance measures (JPMs). The purpose of the review was to assess the quality and content of the examinations. The test items (written exams, scenarios, and JPMs) were reviewed using the guidance provided in TI-2515/117, Appendix A. The criteria in TI-2515/117, Appendix A, parallel the guidance contained in NUREG-1021, " Operator Licensing Examiner Standards."
The inspectors observed the administration of the written examination and operating test to two crews, an operating crew and a staff crew. This observation included review of the licensee evaluators' evaluations of crew and individual performance on the operating test and review of the written examination grading. A draft remediation plan for an operator who failed the walk-through examination was reviewed to evaluate the licensee's remediation process. The inspectors also reviewed the results of the examinations that were administered earlier in the cycle.
Interviews were conducted with licensed operators, training instructors, and supervisory personnel to assess the examination process. Selected administrative procedures related to the development and administration of the examinations were also reviewed.
FINDINGS 2.1 Sample Plan Overall, the requalification examinations that were administered appropriately sampled the areas that were covered in the requalification program. However, several problems were noted related to the Sample Plan and development of the examinations that could have prevented an appropriate sampling of the areas covered by the requalification program. The
'
dynamic simulator scenarios were not integrated into the Sample Plan and were not selected j
.,..,. _
....
.
. _ _ _, _.. __
.-.
.
.
...
-...
.
,
- _.. _
- _ _ -
_ _ _.
. _ -..
_ _._ _ _ _
_ _. _ _ _ _. _ _ _ _ _ _
_ _ _ _.
4 based on any specific content criteria. The JPMs that were used for all Ove examination
,
'
weeks were selected based on the Sample Plan. However, they were not integrated into the Sample Plan on a weekly basis. This resulted in an uneven distribution of tasks among
!
crews. For example, some crews had to perform 5 or 6 emergency tasks while others only
,
had 2 or 3 (out of 10). Also members of the same crew had to perform tasks that were very j
similar or on the same system. Written examination test items were correlated to the Sample Plan by matching the referenced Lesson Plan with the duty area from the Sample Plan. As a result, test items were not always fit into the appropriate place in the Sample Plan.
The written examinations adequately sampled the items stated in 10 CFR 55.41 and 10 CFR 55.43 and the operating tests adequately sampled the items stated in 10 CFR 55.45. All portions of the examination included items based on operational events, plant modi 6 cations, and procedure changes.
2.2 Dynamie Simulator Examinations The simulator scenarios that were reviewed met the qualitative attributes specified in TI 2515/117 with the exception that they did not always consist of related events. To ensure examination security, the licensee revised scenarios that had been used in training by changing the malfunctions prior to the major transient. This resulted in a series of unrelated events, but did not have a significant negative effect on the quality of the scenarios. The scenarios were adequate to evaluate both crew and individual competencies and were within
'
the quantitative guidelines for complexity.
'
The performance standards for the expected operator actions were objective and complete.
The defined crew critical tasks (cts) listed as objectives of the scenario did not always contain specific objective standards for performance. However, the evaluators demonstrated a clear understanding of the CT performance standards during the critiques of crew performance.
During administration of the examinations, the scenarios were run as planned; however, the timing of some of the events did not allow all of the expected actions to be completed. For example, primary containment conditions deteriorated quickly due to a leak in the drywell, which did not allow the operators an opportunity to take the expected corrective actions prior to the reactor scram. In some cases, the timing of the malfunctions resulted in periods of
" dead time" that left the crew with little to do except monitor control panels. In general, the scenario run times were on the short side of the limits recommended in NUREG-1021. One
'
of the scenarios had a run time that was less than the 45 minutes recommended in the Examiners' Standards.
No crew critical tasks were missed by any of the crews and all individuals passed the dynamic simulator portion of the examination. The facility evaluators worked well together i
l to identify appropriate followup questions. Followup questions were asked for all
'
questionable actions to evaluate effectively individual operator competency. The NRC staff i
,
,
w-
-.
.
.-
-.
.
i
-
.- -
.-
---
-.. -_
.
observations paralleled the evaluations of the facility personnel. Management involvement in the examination process was evident as the General Supervisor of Operations served as the lead evaluator for the dynamic simulator examination,
'
l The documentation of the evaluations of crew and individual competency was inconsistent and, in some cases, insufficiently detailed to provide effective feedback into the training program. Different evaluators appeared to use the rating scales differently for evaluating individual performance. The evaluations by some evaluators even appeared to vary from week to week. For example, a rating of '4' (out of 5) was used for "go(xi" performance and it was also used when minor weaknesses were noted. Documentation of weaknesses noted during the crew evaluations did not always meet the guidelines specified in the evaluation forms in that they did not include specific actions taken and the consequences of missed or incomplete actions. In some cases the documentation of individual weaknesses did not appear to be sufficient for ensuring appropriate followup and for use in program evaluation.
Some items identified during the observed critiques of the dynamic simulator examinations were not identified on the evaluation forms.
2.3 Walk-Through Examinations In general, the JPMs that were reviewed met the applicable quality attributes specified in TI 2515/117. The JPMs contained appropriate cues and objective performance standards.
However, some performance standards were not specific to the initial conditions for the
,
JPM. For example, some performance standards required verification of actions when it should have been apparent from the initial conditions that the action would actually have to be performed. Some steps that were identified as critical did not appear to be critical for completion of the task. For example, verification of automatic start of the CRD pumps following reset of a lockout relay was identified as a critical step when responding to a loss of offsite power.
During administration of the JPMs, use of the simulator was not always optimized to provide a realistic forum for evaluating operator performance. For example, operators were told to simulate that an alarm was in rather than setting up the simulator so that the alarm was actually annunciating. In most cases the facility evaluators had to set up the initial conditions and acknowledge alarms during performance of the task. The dual role of evaluator and simulator operator appeared to add time and distractions to the examination process.
The facility evaluators gave appropriate cues and adjusted well when operator performance was not as expected. They were objective and questioned the operators to followup on questionable performance. The evaluators identified individual performance errors that occurred, including an individual failure on the walk-through portion of the operating tes _ _ _ _ _ _
_
_.
__
_
. _
-
_. _
._
.
_
_. _
.
.
.
The evaluators also responded appropriately when a minor simulator fidelity problem was
,
identified during administration of one of the JPMs. The problem had already been identified by simulator personnel and corrective actions were planned.
The draft remediation plan for the operator who failed the walk-through examination contained appropriate retraining and reevaluation requirements. The characterization of the operator weaknesses and possible causes could have been better defined and clarified.
Licensee personnel acknowledged these observations and planned to improve these areas prior to approval of the remediation plan.
2.4 Written Examinations
-
The review found that the test items included in the written examinations met most of the quality attributes contained in TI 2515/117. A very high percentage of the examination questions were at the comprehension level or higher. An appropriate use of reference material for evaluation of operator knowledge and skill was evident. Minor problems were noted with the quality of some of the distractors in the test items. High quality distractors are required to ensure that multiple choice written examinations discriminate between safe and unsafe operators.
The individual test items did not have estimated response times. Each examination contained
.
the maximum number of questions specified by NUREG-1021. Facility personnel felt that
,
'
this quantity was appropriate based on experience from written examinations administered during the training cycles. No effort was made to ensure that the specific items selected for-each examination, when combined, would result in an examination that was the appropriate length to discriminate between safe and unsafe operators. Only the examination that was administered during the inspection was time validated by operators. The operators that were
interviewed indicated that the examinations were the appropriate length; however, many of the operators completed the examinations in less than the time allowed.
All cf the operators passed the written examinations for the three weeks that were reviewed.
The licensee uses machine grading for the written examinations. No discrepancies were noted during review of the graded examinations.
2.5 Examination Integrity
.
The methods used to ensure examination integrity appeared to be adequate to ensure that there was no compromise between examination weeks. However, the schedule that was used for administration of the examinations during the week of the inspection provided some Ix>tential for compromise. The same static written examination and one dynamic simulator i
scenario were administered to the crews on different days. The crews were kept separated while they were in the training center; however, there were no controls placed on them after they left the site at the end of the day. The results of the interviews indicated that operator understanding of their responsibilities for maintaining examination integrity was inconsistent.
.
-.
-.
.
.
.-. - --. - -..
.. -
_-
. - -
.. ~. -- - -=
-
._. -
-
_-.
_ --
.
.
.
There was no indication of examination compromise during observation of examination administration or during review of the examination results.
CONCLUSIONS Overall, the requalification examinations that were administered appropriately sampled the areas that were covered in the requalification program and the items specified in 10 CFR 55.
However, several problems were noted related to the Sample Plan and development of the examinations that could have prevented an appropriate sampling of the areas covered in the requalification program.
The scenarios were adequate to evaluate both crew and individual competencies and the performance standards for the expected operator actions were objective and complete. The facility evaluators asked thorough followup questions and effectively evaluated crew and individual operator competency. The documentation of the evaluations of crew and
-
individual competency did not always appear to be sufficient to provide effective feedback into the training program.
In general, the JPMs were of high quality and contained objective performance standards.
The facility evaluators gave appropriate cues and effectively evaluated operator performance.
Appropriate remediation and reevaluation was planned for an operator that did not pass the walk-through portion of the operating test.
A very high percentage of the written examination questions were at the comprehension level
,
or higher; however, the quality of some of the distractors could be improved to better discriminate between safe and unsafe performance. Time validation of the written examinations was based on experience and NUREG-1021 without considering individual test item estimated completion times.
The methods used to ensure examination integrity appeared to be adequate to ensure that
there was no compromise between examination weeks. There was no indication of examination compromise during the inspection.
3.0 LICENSED OPERATOR REQUALIFICATION TRAINING PROGRAM j
SCOPE
,
The inspectors reviewed Nuclear Training Procedure NTP-TQS-102, " Licensed Operator f
Requalification Training," the 1992 NMP1 Operator Training Annual Report, and a sample j
of Quarterly Training Effectiveness Meeting minutes to assess the licensee's effectiveness in evaluating the requalification program. The content and results of selected weekly examinations were also reviewed. The inspectors reviewed the information provided to the licensed operators about the possible impact of noncondensible gases on reactor water level indication to evaluate the training provided to licensed operators on recent industry issues.
-
_
_
_
_ __
.
.
.
.
_
-
-
.-.
-
_
. - -
.
-
-
.-
-..
,-
Interviews were conducted with licensed operators, training instructors, and supervisory personnel to assess the effectiveness of evaluation and revision of the LORT program.
The facility recently activated several inactive operator licenses in preparation for the -
bargaining unit contract negotiations. The inspector reviewed the licensee's actions to verify that the operator licenses were restored to an active status. This review included administrative procedure N1-ODP-TQS-0401, " Administrative Controls for Maintaining Active License Status at NMP-1," and the documentation for several licensed operators who activated their licenses. The inspectors also assessed the controls for training attendance.
FINDINGS 3.1 Program Evaluation and Revision Operator feedback on training effectiveness is formal and used by the training department to improve training quality. The operator and training instructor interviews verified that the training feedback information is acted upon by management and the person providing the input is contacted directly about the resolution. Several of the operators that were interviewed cited the instructors' willingness to work with them to provide additional training i
or other support as a strength of the training program.
The quarterly training effectiveness meetings are attended by operations and training personnel. Improvements to the operations training programs are discussed with input provided by both departments.
Periodic examinations are administered to ensure the operators' knowledge level remains adequate to operate the plant safely. The inspectors found the questions on the weekly examinations to be at the comprehension level or higher. The inspector verified that the examination questions originated from learning objectives covered in training. The examinations were appropriately graded and evaluated to ensure that training was effective.
Poor performance on the weekly examinations is remediated by examination review or additional training as appropriate.
The results of the requalification examinations are assembled in an End of Examination Report. Strengths and weaknesses in operator performance are summarized. Test item
'
analysis is performed on the results of the written examinations for all of the crews to evaluate both operator performance and examination quality. Feedback from the report is used to revise the requalification program.
An annual report is prepared for all of the operations training programs. The 199-2 Annual Training Report was a detailed and comprehensive review of the 1992 operator training programs. The report contained both strengths and areas for improvement in the following (1) training program content; (2) organization and management of trainingt (3)
areas:
development and qualification of staff; (4) analysis, design, and development; (5) conduct of
.
.
.
.
.
.--
.
,
.. ----
--. _ -
.
-... -
.= - -=
-. -.-. -..- -..- _.
.
.-
.
.
classroom training; (6) conduct of in-plant training and trainee evaluation; (7) conduct of
'
simulator training and trainee evaluation; and (8) systematic evaluation of training effectiveness.
The licensed operators were provided information about the possible impact of non-condensible gases on indicated reactor water level in the Night Order Book. The operators'
knowledge of the phenomena was adequate. The operators also stated that they receive frequent dynamic simulator training for events related to a loss of reactor water level indication Review of the simulator training records for the past two years supported the operator comments. The Boiling Water Reactor Owners Group (BWROG) video tape on reactor water level indications was provided to all Unit 2 licensed operators and the Unit 1 initial license class. The tape is currently being reviewed for use in Unit i LORT.
3.2 Training Program Attendance nnd Activation of Licenses Attendance of operators at scheduled training was adequately controlled to ensure that all operators attend training sessions either as scheduled or in makeup sessions. The training department maintains detailed attendance records to ensure that all licensed operators receive-all required training.
The procedure for license activation contains all the items required by 10 CFR 55.53.f to reactivate an inactive license. The required information was documented correctly in the control room log for the licensed operators that recently activated their licenses. The documentation included standing parallel watches with a licensed operator, performing a plant tour, and review of the shift turnover procedures.
CONCLUSIONS
,
Operations management involvement is evident throughout the LORT program. The interface between the Operations and Training Departments was noted as a strength of the i
LORT program The training program has an effective system of checks and balances to improve.ontinually the training program content and revise the program as necessary based
'
on operator performance. The training feedback system is managed to encourage continuous
.
input for improvements to the training program. Operators are trained and evaluated on
>
industry events, plant modifications, and procedure changes.
'
.
The facility effectively reactivatcJ several inactive licenses meeting all procedure and regulatory requirements. Controls are adequate to assure licensed operator attendance at required training sessions.
.. --
- - - -
,_--,.,--.-r--
,.,,,.-,,, --.-, -
-_,,
4.-
.,
.-,-
,c
,y
.. _._ _....
_
. _.
__... _
_.._. _ _,.
.....
. _ _. _.. _ _ _..
...
_
-
,
,
10 4.0 EXIT MEETING An exit meeting was conducted on June 11,1993, at the conclusion of the inspection.
During the meeting, the inspectors reviewed the scope and findings of the inspection. Those personnel in attendance are noted in Attachment 1. The facility representatives acknowledged the fmdings. The licensee did not identify as proprietary any information provided to or reviewed by the inspectors.
Attachment:
1. Persons Contacted 2. Simulator Fidelity Report
,
.
(
,
a m
x-
....,*m...
-.,
-4.,
,.
..v.
-.
,
- -.
-.
-
..
. -
.
.
,.
.
,
.
.-
e a
ATTACIIMENT 1 PERSONS CONTACTED Niagara Mohawk Power Corporation N. S. Carns, Vice President, Nuclear Generation
K. A. Dahlberg, Plant Manager, Unit 1
R. Smith, Manager, Training R. Sanaker, General Supervisor, Operations Training, Unit 1
R. Slade, General Supervisor, Operations Training, Unit 2
N. Rademacher, Manager, Operations, Unit 1 M. Balduzzi, General Supervisor, Operations, Unit 1 R. Seifried, Assistant Station Shift Supervisor
R. Tessier, Manager, Maintenance, Unit 1 M. Meier, Requalification Program Coordinator M. Peterson, Operations Trainer, Unit 1
R. Wallace, Operations Trainer, Unit 1
J. Pavel, Site Licensing
J. Kronenbitter, Operations Support Specialist Nuclear Regulatory Conunission T. Walker, Senior Operations Engineer
S. Hansell, Operations Engineer.
M. Biamonte, Training and Assessment Specialist W. Mattingly, Resident inspector-l R. Plasse, Resident Inspector The inspectors also held discussions with licensed operators during the inspection.
I
,
--
--
- ~
..
.
,
,
.
.
__j
.__
_..
.
.
__.
.__. _.
.
.
.
.
ATTACIIMENT 2 SIMULATOR FIDEIITY REPORT Facility Licensee: Niagara Mohawk Power Corporation Facility Docket No: 50-220 Requalification inspection Conducted: June 7 - 11,1993 This foran is to be used only to report observations. These observations do not constitute audit or inspection findings and are not, without further verification and review, indicative of j
noncompliance with 10 CFR 55.45(b). These observations do not affect NRC staff certification or approval of the simulation facility other than to provide information that may be used in future evaluations. No licensee action is required in response to these observations.
While observing the simulator portion of the operating tests, the following items were observed:
ITEM DESCRIPTION l
Containment Spray On start of one of the Containment Spray pumps, pump amps did not i
indicate the correct value. (This was a known problem and corrective action was planned.)
,
.
.
um w
w en-e w
-