ML17056C242
| ML17056C242 | |
| Person / Time | |
|---|---|
| Site: | Nine Mile Point  |
| Issue date: | 01/26/1993 |
| From: | Bettenhausen L NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | Rich Smith NIAGARA MOHAWK POWER CORP. |
| References | |
| NUDOCS 9302010035 | |
| Download: ML17056C242 (32) | |
Text
Docket No. 50-220
\\
ggN 26 1993 Niagara Moh'awk Power Corporation ATTN: R. G. Smith
- Manager'- Nuclear Training P. O. Box 32
. Lycoming, New York 13093
Dear Mr. Smith:
SUBJECT:
REQUALIFICATIONINSPECTION MEETING The purpose of this letter is to confirm the meeting to be held'on February 9, '1993, at 12:30 p.m. in the Region I office, based on a telephone conversation between yourself and Mr. R. Conte of my staff on January 19, 1993.
The purpose of the meeting willbe to discuss the implementation of Temporary Instruction (TQ 2515/117, "-Licensed Operator
. Requalification Program Evaluation," at Nine Mile Point, Unit 1.
Enclosed is a copy. of TI 2515/117.
You should be prepared to discuss planning for the inspection at the meeting.
Should you have any questions regarding this information, please contact Richard Conte at (215) 337-5210.
Sincereb>, ~.
Lee H. Bettenhausen, Chief Operations Branch Division of Reactor Safety
Enclosure:
NRC Staff Temporary Instruction 2515/117 9302010035 930126 PDR ADOCK 05000220 6
PDR 2906<7 OFFICIALRECORD COPY
. G:NMPITIII.MTG I
R. G. Smith cc w/encl:
B. Ralph Sylvia, Executive Vice President - Nuclear C. Terry, Vice President - Nuclear Engineering J. Perry, Vice President - Quality Assurance N. Cams, Vice President - Nuclear Generation K. Dahlberg, Unit 1 Plant Manager, M. McCormick, Unit 2 Plant Manager D. Greene, Manager, Licensing J. Warden, New York Consumer Protection Branch G. Wilson, Senior Atto'rney'.
Wetterhahn, Winston and Strawn
'irector, Energy & Water Division, Department of Public Service, State of New York=
C. Donaldson, Esquire, Assistant Attorney General,. New York Department of Law K. Abraham, PAO (2)
Public Document Room (PDR)
Local Public Document Room (LPDR)
Nuclear Safety Information, Center (NSIC)
NRC Resident Inspector State of New York, SLO Designee bcc w/encl:
Region I Docket Room (with concurrences)
L. Bettenhausen, DRS C. Cowgill, DRP R. Conte, DRS J. Yerokun, DRP L. Nicholson, DRP S ~ Greenlee, DRP W. Schmidt, SRI - Nine Mile V. McCree, OEDO R. Capra, NRR J. Menning, NRR D. Brinkman, NRR D. Holody, EO-
RI DRS-Walke /dmg 1/25/93 01SV93 RI:DRS Benen n~~
01/y'93 OFFICIALRECORD COPY G:NMP1TI11.MTG
~Pg AECI, fp
~4 0*
O 0
UNITED STATES NUCLEAR REGULATORY COMMISSION REGION I 4j5 ALLENDALEROAD KING OF PRUSSIA, PENNSYLVANIA19406-1415 NRC INSPECTION MANUAL
. TE/IFCRRl(
IhEF~ICN 2515/117 LICENSED CPER4TCR RKIU4LIFIC'ATICN PFO:RN/I EVALMTICN SALP FLbCTICN4L BREA: PI%VI CPEFATIQB CFPLIGSILITY:
Teo requalification progrars each in Regions I, II, and III; one requalification program 'each in Regions IV and V.
2515/117-01 CBJECI IVES 01.01 To.determine what level of inspection or examination activity is required
.to assess the adequacy of the facility licensee.'s requalification program for
.licensed operators.
01.02 To verify that the facil'ity licensee's
,requalification program for'.
I icensed operators'incorporates requirements for both evaluating operator rrastery.
of training objectives.and revising program content in accordance with 10 CFR 55.
01.03 To assess the facility licensee's effectiveness in evaluating and revising the requalification program for licensed operators based on the operational perforrrEInce of licensed operators, including requalifi'cation exaninations, as required by 10 CFR 55.
2515/117-02 SG(CFClbD SKY-92-100 infoIm.d the Caanission of ioprovEmnts the staff is considering for the licensed operator requali,fication examination program.
ForeTost arong these v18s proposed ruleraking that 1Aould delete the requirerent for the hFC to examine each licensed operator for purposes of license renenel.
The rule change would allcuv the staff to use its resources rrore efficiently by inspecting facility requalification progrars instead of administering individual. requalification examinations as required by the regulations.
F In the SECY paper, the staff noted that it would develop a plan for conducting perforrrance-based inspections at each facilityduring each.systErratic assessrrent of 'licensee perforrrance (~) cycle.
It vtes also stated that the inspection planwould include these three actions:
(1) review selected written examinations and operating tests submitted by the facility licensee; (2) evaluate selected written examinations and operating tests by parallel grading using hFC exaniners; and (3) review operational perfoIrrance and either conduct an inspection of,the facility's iITplerentation of the requalification training program or administer an hFC-conducted req'ualification examination.
Issue Date:
12/23/92 2515/117
The staff intends to conduct an inspection each year.
Therefore, current infomatlbn regarding the adequacy-of the requalification progran can be incorporated into the ALP report.
It also allons the staff the flexibilityto conduct any necessary folic'-up inspections within the sara ALP cycle to develop a rmre carplete asses@rent of the facility licensee's irrplemntation of its requalification program.
This approach vould allcN the staff to allocate its resources efficiently based on the perforrrance of each facility and to focus on evaluating the irrplrrentation of facility requalification program.
It would also allmv the staff to pay rrore attention to those facilities having weaker requalification progrars, thereby reducing the hFC's regulatory irrpact at rrany other facilities and increasing the overall level of operational safety.
2515/117-03 lhBPECTICN RHlJIRHVENTS (""" indicates a minimm inspection requirerent) 23.31
~31 I
3 3
~l
" a.
+ b.
Regional rranagarant wi I I determine whether to conduct the basic inspection, an augrented inspection or an hFC-administered requalification examination.
RevieN recent events and evaluate test rraterials before the on-site visit and determine if significant concerns exist over operational safety.
1.
Consult with H3LB if sorething other than the 'basic inspection is to be acccrrplished.
23.02 I -Of Assess the adequacy of selected facility licensee developed written and operating exaninations.
Analyze and cmpare the urrprehension level tested on selected written and operating examinations to be atninistered during the period under review with the carprehension level tested on examinations administered previously.
2.
3, Determine ehether an objective perforrrance standard is utilized for selected carprehensive written and operating examinations.
For selected written exaninations, evaluate the examination's ability to discriminate at the appropriate level.
4.
Identify several recent procedure revisions or plant rrodifications and revieN selected test item to determine whether applicable rrodifications have been rrade to the
- question, Job Perforrrance IVbasure (JRVI) and sirrulator scenario banks.
5.
Verify that the FG and SFG -questions on the written examination adequately style the iters stated in 10 CFR 55.41 and 10 CFR 55.43 and that the operating exrnination adequately sarples the iters stated in 10 CFR 55.45.
6.
Evaluate the quality and content of the written and operating examinations.
2515/117 Issue Date:
12/23/92
+ b.
0 0
Assess significant operator errors that have occurred since the 'last inspection to determine if the errors rray be a result of ineffective training.
Review pertinent inforrration contained in:
1.
IVbst recent SALP Report.
2, Recent exanination and inspection reports (e.g.,
erargency preparedness or EP inspections) related to,operator training or per forrrance..
1 3.
Resident inspector observations regarding operator perforrrance.'.
LERs.
+ c.
5.
Other indications of potentially weak operator perforrrance such as TS violations, internal event
- reports, EXS actuations, safety system initiations, reactor scrars or trips.
Determine if the facility licensee has identified operator perforrrance deficiencies and incorporated thm into the evaluation portion of the training progran by evaluating the inforrration frcm 03.02b above and reviewing the facility licensee's exaninations.
03.03
-Si e Ins c ion Ac ivi ies
" a.
Assess the adequacy of selected facility licensee developed written and operat ing exminat ions.
1.
Revienr the smple plan used by the facility to construct exaninations,
" b.
2.
Determine whether the construction of the exaninations is in accordance with the facility's smple plan.
- 3. 'erify that the facility licensee has used lesson plans and learning objectives for the requalification progran under review to construct the examinations, as denoted in the facility's sarple plan.
Assess the facility licensee's effectiveness in conducting written examinations and operating tests to ensure, operator rrastery of program content.
1.
Conserve examination activities rrost suited to the inspection goals.
2.
Interview selected operators regarding recent exaninations.
3.
Determine whether:
(a)
The exaninations are conducted as planned and -any errors in administration are detected and corrected for subsequent examinations.
(b)
The facility licensee's exanination schedule facilitated reducing undue operator stress.
(c)
CraN and operator perforrrance errors rrade during sirrulator evaluations are detected and adequately addressed by the Issue Date:
12/23/92 2515/117
'facility evaluators.'d)
Any errors rrade by individual operators dur,ing the eelkthrough exaninations are detected and adequately addressed by 'facility evaluators.
(e)
Critiques,of operators arid crens after the sirrulator exanination are effective in denoting both strengths and weaknesses and accurately appraise the observed perforrrance.
1 (f)
IVbnagment guidance paral'leis the actual-conduct of testing as it ees observed.
(g)
Current industry events applicable to the facility are
'ncorporated into testing as appropriate.
(h)
Facility evaluators effectively identify individuals and. crens
'requiring rerediation,'nd appropriately indicate eben reroval frcm shift activities is eerranted.
" c.
Assess the facility.-licensee's use of objective perforrrance standards when conducting evaluations and critiques of operators and crews to determine whether pass/fail decisions are rrade objectively.
1.
, Review written perfoomnce standards used for evaluations'or clarity and relevance.
2.
Assess the facility evaluators'se of perforrrance standards by.
parallel grading selected written examinations and'operating tests, including observing discussions regarding crew and operator perforrrance fol'loNing the administration of sirrulator exaninations.
3.
Determine whether the perfornance standards are used consistently and object'ively.
4.
Interview instructors and training supervisors to determine their understanding of the application of perforrrance standards.
'd.
Evaluate the effectiveness of the licensee's process for revising its cont-inuing training program to naintain it up-to-date, including the use of student feedback.
Verify that the facility licensee has effective procedures in place that alloN the requalification program to adjust to changes in plant design and applicable procedures, changes in regulatory requirerants, and the occurrence of plant or industry events.
The tirreliness of progrrn revisions should be ccrarensurate with the irrportance of the change or event.
2.
, Evaluate the effectiveness of the I-icensee's use of student feedback.
'e.
Evaluate. the adequacy of the facility licensee's process to train and evaluate the I.icensed operator training staff.'
f.
Assess the training conducted as a result of operator errors that have occurred since the last inspection to determine the effectiveness of the training..
2515/117 Issue Date:
12/23/92
g.
h.
~
0 Evaluate previously administered reredial training to ensure it has adequately. addressed licensed operator or cree perforrrance weaknesses.
Evaluate the facility's process for mnaging the requalification training progran to ensure cmpliance with 10 CFR 55.
Assess sirrulator perforrrance and its fidelity to the'eference plant to determine if it is adequate to support the requalification program.
Assess the safety impact of any negative training caused by sirrulator inadequacies.
Evaluate the adequacy of administrative procedures that are in place that ensure the integrity,'f exaninations and tests.
Conserve exanination activities and assess examination results for any.,indications of exanination cmprcmise.
= 2515/117-04 IM ECl ICN GJIM'LZ Gener I Guidance In order to assure public health'and safety, facility licensees are required to rraintain a
continuous training'rogran for hFC licensed individuals and operating crews.
Each facilityrrust have a progran that rraets the -requiremnts specified in 10 CFR 55.59.
This regulation requires licensed individuals to'pass an annual operating test and a periodic urrprehensivewritten exanination.
The Ccranission has specified that facility licensees rray,utilize a, system approach to training (SAT) for developing and irrplerenting this training, in lieu of adhering to sera of the specific guidance in 10 CFR 55.59.
This instruction is intended to guide the inspector in rraking an objective detemination that a facility licensee's progranprovides sufficient training and evaluation of licensed individuals and crews to assure safe poner plant operation.
The inspector should be able to determine that the program will identify poor or rrarginally performing individuals or crews.,
The program should also provide for appropriate remedial corrective actions.
The inspector should evaluate the facility licensee's ability to,rraintain the continuing training progran up-to-date, includin'g the use of licensed operator feedback.
~ifi thi 04.01 ~ih MIHI f~h~l
'The rrajor portion of the inspection is to be conducted during the sam tirre as the faci.lity's annual operating test.
The scope and content of the inspection are to be determined as discussed beloN.
The faci I ity I icensee rrey not administer' cmprehensivewr i tten exaninat ion each
- year, so the inspection effort re center around the operating tests only.
The level of the inspection to be conducted (basic or augmented) is determined by the regional rranagsmnt, taking into consideration such factors as operational perforrrance since the last inspection or hFC-conducted requalification exanination, recent inspection or requalification exanination results, current SQ P,ratings in Plant Operations, the scope of the roost recent requalification program inspection, and the length of tirre since there was bFC participation in developing and administering a facility's requalification exanination.
r At least every six years, the VC intends to administer all or part of a
facility's requalification exanination in accoidance with.NAG-1021.
This Issue Date:
12/23/92 2515/117
. ~
allans the staff to retain the examining skills required eben conducting a
requalification examination and it validates the results of the.inspection process.
The hFCrray also choose to administer all or part of a facility's requalification examination for cause.
The rationale for rraking this decision should-be based on potential training.and evaluation deficiencies that haverranifested themselves
'through an inordinate nurber of licerisee events'nvolving operator errors.
Inspection results, feedback frcm the resident inspector, and other indicators rray also be sufficient justification for the staff to conduct'the requalification examination.
Regional rranagarent should 'consult with HXB when considering administering a requalification examination for cause.
Lhless the-facility has had a recent history of operationally related events or its requalification program mes evaluated as unsatisfactory during its rrost recent requalification programevaluation, only the basic
- elerents of this Tl need be conducted.
If the region is consider,ing conducting scrrething other than a
basic inspection, HXB should be, consulted.
Review recent operational events arid the test rraterial sutmitted by the licensee.
lhless there is a consistent pattern of operational errors or the test rraterial is of such a poor quality that significant doubts exist over the quality of the training provided such that the safe operation of the facility rray be affected, the hFC shall not interfere with the facility's requalification examination process by sugge'sting rrodifications to test Iters or,examination schedules.
The facility should be inforrred of the staff's
- concerns, but the staff shall not dictate the content of the exanination.
If safety concerns 'exist, the staff. should consider actions such as holding rranagarent rreetings, conducting operational evaluations, or issuing an order.
H3LB should be consulted to help detemine an appropriate course of action.
04.02
+a.
b.
"c.
-Oft'n order'to adequately assess the examinations sulmitted by the licensee, the inspector needs to refer to 'the checklists in Appendix A for
'guidance, MBEG/BR-0122 and hLREG-1021 also provide inforrration that is useful in evaluating the adequacy of an examination.
The inspectors should contact the resident inspectors, cognizant personnel from -the Division of Reactor Projects and the N% project rranager to ascertain any substantial procedure or system rrodifications that should be incorporated into a continuing training program.
Na specific guidance provided.
Identified perfornance deficiencies rray be best evaluated in various
'spects of, the examination.
Depending
=on the -significance of the
, deficiency, it,rray be appropriate to have each examination for each creN contain test itasca to evaluate the effectiveness of the training.
- Also, facilities rray rrake use of other rrethods to keep operators abreast of areas where perforrrince deficiencies rray have occurred, such as required reading or night orders.
Therefore, you rray have to verify sam of the facility licensee's actions on-site.
(See 03.03e/04.03e) 2515/117
- 6.-
Issue Date:
12/23/92
i In inAc ivi i
In order to adequately assess the examinations sutmitted by the licensee, the inspector needs to refer to the checklists in appendix A for guidance.
bLREG/BR-0122 and NSEB-1021 also provide inforrration that is useful in evaluating the adequacy of an exrnination.
staining the facility's sarple plan in order to link conducted training with the test itms is an irrportant aspect of reviewing the content of the exanination.
b.
bb specific guidance provided.
"C.
Chestions that rray be effective in ascertaining the trainers and operators understanding of perforrrance standards are:
1.
Ruv are per forrrance standards used in conduct ing eva luat ions fo rrru I a ted?
d.
e.
2.
Rm are the perforrrance standards used in conducting evaluat.ions cawunicated to the evaluators'?
3.
Ruv are the perforrrance standards used in conducting evaluations ccmrunicated to licensed operators?
4.
Ruv does training supervision assure that the perforrrance standards are effectively irrplersnted by the evaluators?
To evaluate the effectiveness of the facility's use of student
- feedback, the folloNing activities will provide assistance:
1.
Determine who is responsible for obtaining student feedback and cmparing their understanding of their goals to the rranagarent expectations of the program, 2.
RevieN a representative sarple of the student feedback to determine ehether the feedback can be traced to a subsequent rrodification of the requalification training.
3.
Evaluate the program(s consideration of the ccrarents and reccrarendations
- rrade, and their irrplerentation, if appropriate.
Determine if requalification program rrodifications are backlogged and the cause.
Determine vAether progran rrodifications are prioritized based on safety.
Gcrrpare these findings with rrenagemnt expectations'.
Interview a selection of trainers and operators to deteimine whether they knuv of, use, and are satisfied with the systrn used to gather and irrplerent feedback.
Nhen evaluating the facility licensee's progran associated with requalification program instructors, the inspector should ascertain if the facility licensee has a
policy or procedures detailing hoN instructors are trained and evaluated.
Within these instructions there should be requirerents for rerediation of poor'ly performing instructors.
The inspector should also reviBN a sarpling of records docurenting periodic evaluation of requalification progran instructors.
Issue Date:. 12/23/92 2515/1.17
0 In order to assess the effectiveness of training on previous operational de% iciencies that have been incorporated into the progran the inspector should:
I 1.
Interview.-an appropriate cross section of trainers, licensed operators and non-licensed operators.
2.
Nhen
- possible, observe applicable classrocm, sirrulator and JFM instruction to assess the effectiveness of the rersdial training.
3.
Review lesson
- plans, reference rraterials and attendance documentation to assess the effectiveness of the remedial training.'.
.Nhen evaluating any remedial training that has taken place, the inspector should:
h.
1.
Select several exarples of operator and cr'ee perforrmnce weaknesses since the last inspection.
N 2.
Determine whether the root cause mes identified and if corrective actions were irrplerented by the licensee:
3.
Determine if 'he licensee confirmed the effectiveness of the corrective actions at the crrpletion of the retraining through an
, suitable evaluation rrethod.
lt Evaluating honrvell a requalification progran is rranaged and vhether it rreets the requirerents of 10 CFR 55 is a substantial task -that is best accorplished by using the applicable elerents of hLREG-1220.
Nhen concerns exist in this area, regional mnagemnt should consider whether a training inspection (IP 41500) rray be an appropriate rrathod to assess the program.
. It is not intended that a sirrulator inspection be perforrred, but rather a
record of discrepancies and a subjective evaluation.
Corpleting a
sirrulator fidelity report, as contained in ES-501 of NSEG-1021, is sufficient and should be filed as part of the inspection report.
Determiningehether the potential. exists for cmprcmising the examination is a subjective assessrrent on the part of the inspector.
%eever,.
the guidance belcuvmy assist the inspector in this endeavor:
1.
Review the facility's administrative procedures that describe how to rraintain integrity of exaninations.
2.
Review exanination administration and results to determine if there is any= indication of examination coTprcmise.
3.
Interviev responsible training departrrent rranagers and instructors to determine rrathods used for rraintaining the
. integrity of requalification examinations.'f problers were noted in the past, determine vhat corrective, action(s) have been taken to, preclude future occurrences.
4.
Interview licensed individuals to determinevhether their perception and knowledge of examination integrity's consistent with administrative procedures.
2515/117
~ Issue Date:
12/23/92
2515/117-05 REFCRT'lhB RKUIRBVENlS Document inspection findings in a routine inspection report.
In addition to routine regional distribution, send a copy to the Director,.Division of Reactor Controls and Hxran Factors, the Chief, Operator Licensing Branch, the Chief,,
Hogan Factors Assessrent
- Branch, and all other regional Directors, Division of Reactor Safety (Division of Reactor Safety and Projects in Region V).
In addition to the routine inspection report, the region shall also provide the Director, Division of"Reactor Controls and Hnan, Factors, a written'uwary of the inspection te8m's -experiences in using the'erporary instruction.
This suwary should include infowation regarding the expenditure of hours as reported in the Regulatory'nforrration Tracking Systrn (RITS),
any aspects of the inspection that should be evaluated for deletion',
any iters of concern that should be included in the inspection, and any reccmrendations for iriproving the inspection guidance.
2515/117-06 Kh/PLETICN SCH33JLE There are to be eight inspections conducted'using this terporary instruction.
.This includes teo inspections each in Regions I, II, and III and one inspection each in Regions IV and V.
These inspections, including 'report writing and suwary feedback reporting, should be carpleted by'he end of June 1993.
2515/117-07 EXP I FATICN This terporary in'struction is to rara,in in effect until 12 rronths after the date of issuance.
At that tirn:, either a; revised Tl or an inspection procedure (IP) will be issued.
C 2515/117-08 CNIPC7 Ouestions regarding the technical aspects of this Tl should be addressed to the Chief, Operator Licensing Branch, at 301-504-1031.
2515/117;09 STAT IST ICAL CATA REFCRl IPG
The direct inspection effort (DIE) for this Tl should be reported against
=-
2515/117 for RITS data with an inspection procedure elamnt (IPE) code of "Sl" (Safety Issues Program).
2515/117-10 CRIGllATIKB CRWIIZATICN lhFUFhATICN 10.01 r aniza ional s
n ibili The Operator Licensing Branch (H3LB/hFR) initiated this TI.
10.02 our E
irre In order to complete the basic inspection requirsmnts, it is anticipated that two inspectors will spend four days each on.site, for a total of 64 on-site inspection hours.
@eever, sara direct inspection effort will be expended in reviewing licensee examinations and test rraterials prior to going to the site.
It is expected that.this direct inspection effort in the office will take another 32 hours3.703704e-4 days <br />0.00889 hours <br />5.291005e-5 weeks <br />1.2176e-5 months <br />.
Pdditional preparation
- effort, data
- analysis, Issue Date:
12/23/92 2515/117
rrenagemnt briefings, folloN-up act'ivity, travel and report writing are expected to encorpass another 100 hours0.00116 days <br />0.0278 hours <br />1.653439e-4 weeks <br />3.805e-5 months <br />.
7herefore, it is expected that a total of 196 hours0.00227 days <br />0.0544 hours <br />3.240741e-4 weeks <br />7.4578e-5 months <br />, of vAich nearly 5P/o is direct inspection effort (DIE),. will be expended in conducting the basic inspection.
If a region-chooses to do an augrented inspection at a specific-plant, the inspection rray r'equire substantially rrore tirre.
It is anticipated that an additional six staff weeks rray be required to evaluate on-going training and testing activities.
There mould be a caarensurate increase
- in.the tirre spent on other aspects of the inspection, particularly-preparation, folloN-up, and rranagerent briefing activities.
A total inspection effort of 544 hours0.0063 days <br />0.151 hours <br />8.994709e-4 weeks <br />2.06992e-4 months <br /> is anticipated, with 304 of these hours being DIE (256 of thrn on site.)
1D.M E~f.
F 'I' I'
by 1D Va 60.e41'-11 have in effect an operator requalification progran which injst as a
. minimm, rreet the requirerents of 10 CFR 55.59(c).
However, since the CcranissIon rray approve a progran deve.loped by using a systers approach to
" training in lieu of the requirerents of 10 CFR 55.59(c)
(2), (3),
and (4),.any proposed enforcerent action-shall be fonnerded to headquarters for review before issuance.
The checklists in Appendix A are for the evaluation of. facility testing rraterial
. and are not necessarily requirerants of 10 CFR 55.59.
10.04 ~h~r.
IP 41500, "Training and Qualification Effectiveness" can be used, to assess all aspects of the facility licensee's continuing training
- program, This Tl focuses on the evaluation phase and can be augrented to evaluate any other aspect of the progran by using IP 41500.
4 2515/117-11 REFEREES 10 CFR 50.54 10 CFR Par.t 55 Site Specific Technical Specifications, Training NSI/NB 3.1,
- 1981, "Selection, Qualification, and Training of Personnel for Njclear Poher Plants."
Regulatory Guide 1.8, Rev 2, "Qualification and Training of Personnel for /%clear Poner Plants."
NBEG-1021, "Operator Licensing.Ex@niner-Standards."
PLEAT-1220, Revision 1-, "Training RevieN Criteria and Procedures."
hLfKG/BR-0122, "Examiners'andbook for Developing Operator Licensing Exminations" Inspection Procedure 2515, "Light44ter Reactor -Inspection Progran'- Operations Phase" Inspection Procedure 41500, "Training and Qualification Effectiveness."
Enclosures:
Appendix A, "Checklists for Evaluating Facility Testing IVbterial" 2515/117 Issue Date:
12/23/92
APPEN3IX A
~
bFC Checkl ist for Open-Reference Test I ters 1.
Does each test itm have a documented link to irrportant licensee
- tasks, K/As, and/or facility learning objectives?
2.
3.
5.
6.
Is each question operationally oriented (i.e.,
is there a
'correlation beteeen job demnds and test derands)?
Is the question at least at the cmprehension-level of knmvledge?
(See bQ%6/BR-0122)
Is the context of the questions realistic and free of windoN dressing and bachnerds logic?
Does the-iten require an appropriate use of references (i.e., use of analysis..skills or.synthesis of inforrration either to discern what procedures'were applicable or to consult. the procedures to obtain the, answer )?
Is the question a "direct look-up" question?
A "direct.look.;up question" is defined as a question that irarediately directs an operator to a particular reference where the answer is readily available.
7.
8.
9.
z 10, 12.
13.
14.'5.
Does the.question possess a high,K/A irrportance factor (3 or greater) for the job position?
Does the question appear to have the'bility, to discriminate a
cmpetent operator frcm one who is not?
Is the question appropriate for the written exanination and the selected written exanination forrrat (e.g
, short answer; rrultiple choice)?
Are there questions given in a static scenario setup that take advantage of the sirmlator control'ocm setting?
Does any question have the potential of being a "double-jeopardy" question?
Is. the question clear, precise, and easy-to read and understand?
Does there appear to be only one correct ansner to the question?
Does
.the question pose situations and.problers other than those presented during training?
Does the question have a reasonable estirrated response tim'?
Issue Date:
12/23/92 A-1 2515/117, Appendix A
- ~
~.
APPPQIX A Job Perfomance M.asure (JFM) Chality Checklist 1.
- Is the task supported by facility's job task analysis?-
2.
3.
Is the task operationally irrportant (rreets threshold criterion of K/A 3 or as determined by the facility)?',
It k
k Is the task 'designed as either 80 only, FG/SR3 or M/FG/SFG?
4.,'oes each JFM include:
Initial conditions Initiating cues References; including associated procedures Perforrrance standards which are specific in that exact control and indication norenclature and criteria (sNitch
- position, m:ter reading) are specified, even if these ciiteria are not specified in the procedural step Systen response cues in the perforrran'ce standards that are cmplete
'and correct so that the ex@niner can properly cue the operator, if asked Staterents describing irqmrtant actions or observations that should be rrade by the operator Criteria for successful carpletion Identification of-the critical steps and their associated perforrrance standards k
Val'idated tirre limits (average tirre alloned for cmpletion)
JFVh identified as tim critical or not tim. critical by the facil'ity Operations Departrrent-Restrictions on the sequence of steps 2515/117, appendix A A-2, Issue Date:
12/23/92
APPEN3IX A ali iv A ri Sirrulator Scenario Review Checklist 1.
-Does the scenario have clearly stated objectives?
2.
Are the initial conditions realistic, in that sam equipment and/or.
'nstrurentation rray be out of service, but =it does not cue crew into expected events?
3.
Does the scenario consist rrostly of related events?
4.
Does each event'description consist of--
~
the point. in the scenario when it is to be initiated
~
the rralfunction(s) that are entered to initiate the event
~
the syrptors/cues that will be visible to'the creuv
~
the expected operator actions (by shift position)
~
the event termination point 5;
6.
7.
8.
9.
10.
P Is no rrore than one non-rrachanistic failure (e.g.,
pipe break) incorporated into the scenariowithout a credible preceding incident such as a seismic event?
Are the events valid with regard to physics and therrrodynaaics?
Is the sequencing/timing of events reasonable, and does it alloN for the exanination tean to obtain cmylete evaluation results
'amensurate with the scenario objectives?
Has the sirrulator rrodel ing been altered?
Can each rating factor in each crew cmpetency be'valuated?
Has the scenario'been validated?
, If the sapling plan. indicates that the scenario was used for training during the requalification
- cycle, has the facility evaluated whether it should be rrodified or not used?
Issue ate:
12/23/92 A-3 2515/117, appendix A
PPPPGIX A Sirrulator Scenar io Review Checkl ist (cont inued) hbte:
The folloNing criteria list scenario traits that are nurerical in nature.
A second set of nurbers indicates a
range to be rret for a set of two scenarios.
Therefore, to ccrrplete this'art of the review, the set of scenarios rrust be available.
I Iv A ri 12.
Total rralfunctions inserted: 4-8/10-14 13.
Rlfunctlons that occur after EP entry: 1-4/3-6 14.
Abnorrra I Events: 1-2/2-3 15.
lVbjor -Trans ients: 1-2/2-3 16.
17.
18
~
EPs =used beyond pr Irrary scran response KP: 1-3/3-5
\\
KP Contingency Procedures used: 0-3/1-3 Ppproxirrate scenario run tine:
45-60 minutes (one scenario re approach 90 minutes) 19.
Crew Critical Tasks: 2-5/5-8 Are Technical Specifications exercised during the test?
GNUENTS:
END 2515/117, appendix A k
'-4
'ssue Date:
12/23/92