ML041170056
| ML041170056 | |
| Person / Time | |
|---|---|
| Site: | Harris  |
| Issue date: | 10/30/2003 |
| From: | Ernstes M Operator Licensing and Human Performance Branch |
| To: | Scarola J Carolina Power & Light Co |
| References | |
| 50-400/04-301 50-400/04-301 | |
| Download: ML041170056 (38) | |
See also: IR 05000400/2004301
Text
COVER SHEET
HARRIS EXAM
50-40012004-301
FEBRUARY 23 27,2004 -
& MARCH 4,2004 (WRITTEN)
J [<I -
ES-202-1 Exam Preparation Checklist
i
, 1x1 -
E$-201-2 Exam Outline Quality Checklist
", [XI -
ES-201-3 Exam Security Agreements
--[Y1 -
ES-301-1 Admin Topics Outline
4'rN E%-301-2= Controi Room Systems & Facility
Walk-through Test Outline
4ii1 -
ES-301-3 Operating Test Quality Checklist
[k] ES-301.4-Sim Scenario Quality Checklist
I/-[ XI -
ES-301-5 Transient %i Event Checklist
4 xI -
ES-301-6 Competencies Checklist
6
1/18? I ES-40Z-d- Written Exam Quality Checklist
/ 1 1( I tten Exam Review Worksheet
/"y] ES-403 am Grading Quality Checklist
--G- I -
ES-5Ql-1 Post Exam Check Sheet
...
...
.... ~ ..........-.
.... .- ....... .... Examina!ion
... ......Preparation Checklist....... ...... Forw ES-201-1
.....-. ...... . ....... ......
A
......
F'acii.ty: rlariis Date oi Exaninatiol-i:
Examinations Developed by: racilitv ! NRC (circle one)
........
...... .....- ..... ......- .......
Target Chief
Date* Task Description I Reference Examiner's
Initials
I1 I I
-120 2. NRC examiners and facility contact assigned (C.7.d; C.2.e) I&./y1
6. Integrated examination outline(s) due (C.1.e & f; C.3.d) N/LH I
by NRC and feedback provided
9. Preliminary license applications due (C.1.I; C.2.g; ES-202)
14. Final applications reviewed: assignment sheet updated; waiver
letters sent (C.2.g, ES-404)
ctoring/written exam administration guidelines reviewed with
My licensee and authorization granted to give wrltten exams
Target dates are keyed to the examination date identified in the corporate notification letter.
They are for planning purposes and may be adjusted on a case-by-case basis in coordination
23 of 25 NUREG-1021, Draft Revision 4
ES-20 I Examlndtlon Outline Quality Assurance Checklist Form ES-201-2
Item
1. . .__ ....
h. Assess whaherthe outtine was systrmatica6'yand raridumly prepmed in accurdance with Sectim D I o
W ES4O 1 and whither all knowledge and ability cirtegrries are approptiate sampkd
R
I c. Assess whcther the ouilitline overmaphasizes my systam, evolutions,m gm!nc tixIks.
'r __ ....
T d. Assess hether the rrpetition f n m prrviuu;~examinatinl outlines is cxccssiw.
F
N
2.
S
I
M
repeatedan suhsequent days.
-
3. (I) the outline(s) contais(s) the requirrd number of control rmm anrl in-plant task,
(2) no I B O than
~ 30% of the tcd material is repeated froin thc hst NRC cxrminatia,
W (3) *no task B L duplicated
~ from the applicants audit testisj, and
i (4) no more than 80% of thc operatinetest is takcn dire& froin thc licensee's ? x m bank.
.r ~
.
b. Verifythat:
( I ) Ole uak~are dirttihutcd among the safety hnctim groupings as specified in ES-301,
(2) O ~ taskC is c.onductcd in a low-power or shutduw condition,
. , 4 6 (2 - 3 for SKO-(I) of the task require the applicant
(3) .. to impllemmt an altmate psth prwedmc,
(4) onc in-phi1 task tests thc applicant's response to M emergency or ahnormilcmditim, and
(5) thc inplant walk-tllro~yhnquircs the applicant to enter the RCA.
c. Verify that the required administmix topics arc cowred.
d. Detcrminr if there are enourh diffcmt outlitlinrs to test the projrzted niimhw and mix of applicants and
ant.spn:, IC pnon IPS mc ut mg
4.
~ ~~~~~~
(; h. Asscss whether the IO CFR 55.41143 and 55.45 sampling is appqiriate.
c
N c. Ensure that K:A importance ratin@ (cxcept far plant-spccitic priorities) are at least 2.5.
F -.
R d. C l w k for dlrplicatini and orerkp among rxam SeCtimS.
A -~ .~ ~
L e. Check the entire exam far hdance of coverage.
~ ~
f. Assess whether the exam fits the nppropriatejuh level (RO or SRO).
a. h t h M
h. Fxility Reviewe(*)
c. ChiefExarniner
d. NRC Supe~iisa.
.-
NUREG-1021,Draft Revisioii 9
ES-2Ql Examination Security Agreement Form ES-201-3
1. Pre-Examination
I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 2-23-04 as of the date of
my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the
NRC chief examiner. I understand that I am not to instruct, evaluate, vide performance feedback to those applicants scheduled to be administered
these licensing examinations from this date until completion of exami s specifically noted below and authorized by the NRC.
Furthermore, I am aware of the physical security measures and requireme e facility licensee's procedures) and understand that
violation of the conditions of this agreement may result in cancellation of the e ti enforcement action against me or the facility
licensee. I will immediately report to facility management or the NRC chief ex s or suggestions that examination security may have
been compromised.
2. Post-Examination
To the best of my knowledge, I did not divulge lo any unauthorized persons any inf concerning the NRC licensing examinations administered
during the week(s) of 2-23-04,From the date that I entered into this security agreement until the completion of examination administration, I did not
instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted
below and authorized by the NRC.
PRINTED NAME JOB TITLE / RESPONSIBILITY DATE NOTE
__ -. .-- . - .-
-.-
t 3 .-. -apr rAsTr"z+c .- __
25 of 25 NUREG-1021, Draft Revision 9
ES-201 V&y. L * 4- 2 Examination Security AgreeMent Form ES-201-3
1. Pre-Examination
I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of &3as of the date of
my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC
chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these
licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the
NRC.Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand
that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility
licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have
been compromised.
2. Pos t-Examination
To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered
during the week(s) of Z / S $ ~ / ! / O ~ ! From the date that I entered into this security agreement until the completion of examination administration, I did not
instruct. evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted
below and authorized by the MRC.
PRINTED NAME JOB TITLE / RESPONSIBILITY SIGNATURE (1) BATE SIGNATURE (2) DATE NOTE
.I I-_ I - I _ -
14. - ..- ... I - -
15. I
NUREG-1021, Revision 9, Draft
p :--.+.
c
ES-202 Examination Security Agreement Form 3-201-3
v
I. Pre-Exam ination
I acknowledge that I have acqirired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of w a s of the date of
my signature. I agree that I will not knowingly divulge any information about these examinatfons to any persons who have not been authorized by the NRC
chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled :o be administered these
licensing examinations from this date until comp!etion of examination administration. except as specifically noted below and authorized by the
NRC.Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand
that violation of the conditions of this agreernent may result in cancellation of the examinations andlor an enforcement action against me or the facility
licensee. 1 will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have
been compromised.
2. Post-Examination
To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered
during the week(§) of G~3/b/~'f, From the date that I entered into this security agreernent until the completion of examination adminiStratiQR, I did not
instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted
below and authorized by the NRC.
PRINTED NAME JOB TITLE I RESPONSIBiLITY SIGNATURE (1) DATE SIGNATURE (2) DATE NOTE
1.
2.
3.
4.
5.
NUREG-1021, Revision 9, Draft
ES-401 Administrative Topics Outline FORM ES-401-1
Date of Examination:
Examination Level: Operating Test Number: I
Describe Activity to be Performed
Administrative Topic
Perform a manual Shutdown Margin Calculation per OST-1036
Conduct of Operations
(2.1.25 - 2.8/ 3.j)
Determine Average RCS Boron Concentration per EOP-EPP402
Conduct of Operations
(2.1.20- 4.314.2)
Determine clearance requirements for a CSlP per OPS-NGGC-
1301
Equipment Control
(2.2.13- 3.6I3.8)
Determine PEDE While Working in a High Airborne Area
Radiation Control
(2.3.10 - 2.913.3)
NOT APPLICABLE FOR RO
SROs. 80 applicants require only 4 items unless they are retakin
NUREG-2021. Draft Revision 9
ES-301 Administrative Topics Outline FORM ES-301-1
Examination Level: SRO Operating Test Number:
Describe Activity to be Performed
Administrative Topic
(see Note)
(KA#-ROlmp/SROlmp)
-
Perform a manual Shutdown Margin Calculation per OST-1036
Conduct of Operations
(2.1.25 - 2.8 I3.1)
Determine Average RCS Boron Concentration per EOP-EPP-00;
Conduct of Operations
(2.120 - 4.3 I 4.2)
Determine clearance requirements for a CSlP per OPS-NGGC-
1301
-
(2.2.13 3.6 13.8)
Determine TEDE While Working in a High Airborne Area
(2.3.10 2.9 13.3)
~
Determine Protective Action Recommendations per PEP-f 20
I
-
(2.4.44 NA I4.0)
SROs. RO applicants require only 4 items unless they are retakin
only the administrative topics, when 5 are required.
NUREG-2021. Draft Revision 9
ES-301 Control Room I In-Plant Systems Outline FORM ES-301-2
Examination bevel: RO Operating Test Number: /
Control Room Systems (8 for 80; 7 for SRO-I; 2 or 3 for SRO-U)
a.
SystemIJPM Title
Transfer 6.9KVBuses from Unit Auxiliary Transformers to the
I Code*
Type I Safety
Function
Startup Auxiliary Transformers per GP-156.02 -
(062A4.07 3.1)
1010 -
(02284.01 3.6)
C.
isolate the SI Accumulators following a LOCA per PATH-1 and
GP-I 10
I IA I I
I 3
(0000T 1EA1. I 3 - 4.1
d.
Start the Turbine Driven Auxiliary Feedwater Pump per OP-I37
I MlSlL 4s (061A3.01 - 4.2)
I
e.
2
Transfer to Cold Leg Recirculation per EOP-EPP-010 M I AIS1 L
(006A4.05 - 3.9)
I
f.
Perform Control Rod and Rod Position Indicator Exercise per 1
DiAlS
OST-1005 (001A2.11 - 4.4)
7
Place Audio Count Rate Drawer in Service per OP-I05
(02584.02 - 3.9)
h.
Align CCW to Support RHR System Operations per GP-145 DlLlS
I 8
(Q08A4.01- 3.3)
In-Plant Systems (3 for RO; 3 for SRO-I; 3 or 2 for SRO-U)
1
Local Makeup to the VCT Using the Emergency Boration Valve
-
(004A2.07 3.4)
j' Perform local actions for placing an OTAT channel in Test per
OWP-RP-OI 012A4.04 - 3.3
Locally operate a SG PORV per EOP-EPP-012 and OP-126 (000074EA1.04 - 3.9
- Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)lternate path, (C)ontrol Room, (S)imulator, (L)o\
Power, (R)CA
NUREG-1021. DraR Revision 9
E§-301 Control Room / In-Plant Systems Butline FORM E§-301-2
nAmis Date of Examination:
/
- SRO-U
Control Room Systems (8 for RQ; 7 for SRO-I; 2 or 3 for SRO-U)
Operating Test Number:
SystemlJPM Title
062A4.07 - 3.1
NOT APPLICABLE FOR SRO-U
OP-l I 0
d.
NOT APPLICABLE FOR SRB-U
NOT APPLICABLE FOR SRO-U
NOT APPLICABLE FOR SRO-U
NOT APPLICABLE FQR SRO-U
h.
NOT APPLICABLE FOR SRO-U
In-Plant Systems (3 for RO; 3 for SRO-I; 3 or 2 for SRO-U)
I.
1
Local Makeup to the VCT Using the Emergency Boration Valve
(004A2.07 - 3.7)
.
Perform local actions for placing an QTAT channel in Test per
OWP-RP-01 -
(012A4.04 3.3)
k.
Locally operate a SG PORV per EOP-EPP-012 and OP-126
I "13 I 4s
(000074EA1.04-4.1:
- Type Codes: (D)irect from bank, (M)odihed from bank. (N)ew, @)Iternate path, (6)ontrol Room, (S)imulator, (L)o
Power, (I3)CA
NUREG-1021. Draft Revision 9
-.
-. .
I_...I ....
s
- ._ ..... .. , . . .. -I.s.
ES-301 Operating Test Quality Asurwck Checklist F ~ 1-3
o ES-30
, , .... . -- .. ., , , ... ,-- .. I .
- -t-
23-27 l*r
NUREG-1021, Revisicm 9 Draft
~~~~
ES-30 I Simulator Scenario Quality Assurance Checklist Form ES-301-4
Facility: IIARWIS Date of Exam: 2/23- 2716 3 Operating Test No.:
QUALITATIVE ATTRIBUTES Initials
I. 'The initial conditions are realistic. in that some equipment andior instrumentation may be
out of servic.e, but it does not cue the operators into expected events.
2. 'The scenarios consist niostly of related events.
3. Each event description consists of
e the point in the scenario when it is to be initiated
the malfunction(s) that are entered to initiate the event
e the symptonisicues that will be visible to the crew
e the expected operator actions (by shitt position)
the event termination point (if applicable)
4. No more than one non-mechanistic failure (e.&, pipe break) is incorporated into the
scenario without a credihle preceding incident such as a seismic event.
5, The events are valid with regard to physics and thermodyiamics.
6. Sequencing and timing of events is reasonable, and allows the examination team to
obtain comvlete evaluation results commensurate uzith the scenario iihiectives.
.
.
7. If time compression techniques are used, the scenario summary clearly so indicates
Operators have sufficient time to carry out expec.ted activities without undue time
constraints. Cues are given.
8. The simulator modeling is not altered
9 The scenarios h w c been validated. Pursuant to IOCFK55.46(d);any open simulator
perfcxmance deficiencies have been eviiluated to ensure that functional fidelity is
inaintained while running the planned scenarios.
IO. Every operator will be evaluated using at lcast one new or significantly modified
scenario. All other scenarios have been altered in accordance with Section D.5 of ES.
I I . All individual operator compctencies can he evaluated, as verified nsing Form I S 3 0 1-6
(submit the form along with the simulator scenarios).
12. Each applicant will he significantly involved in the minimum number of transients and
events specified on Form ES-301-5(submit the fomi with the simulator scenarios).
~. ~~ ~~~ ~~ ~
13. The level of difficulty is appropriate to support kensing decisions for each crew
oosition.
ES-301 Simulator Scenario Quality Assurance Checklist Form ES-30 1-4
TARGET QUANTITATIVE ATTRIBUTES (PER SCENARIO;
~ ~ ~~~ ~~~~~ ~
ES-301 Transient and Event Checklist Form E§-301-5
HARRIS OPERATING TEST NO.:
I
p- DA
SRO-I I
Instructions: (1) Enter the operating test number and Form ES-D-1 ewnt numbers for
each evolution type.
(2) Reactivity manipulations may be conducted under normal or controMed
abnormal conditions (refer to Section D.5.d) but must be signilcant per
Section C.2.a of Appendix D. * Reactivity and normal evolutions may
be replaced with additional instrument or component malfunctions on a
one-for-one basis.
(3) Whenever practical, both instrument and component malhnctions
should be included; only those that require verifiable actions that
provide insight to the applicant's competence count toward the
minimum requirwent. ,
Author:
NRC Reviewer:
NUREG-1021, Draft Revision 9
ES-301 Competencies Checklist Form ES-302-6
Competencies SCE \RIB - SCENARIO SCENARIO
_ . _s -
T
2 3 4 4
SP -SP
Interpret I Diagnose 23 12 12 3 22 24 13
Events 45 35 45 34 67 5
and Conditions 67 67 6 5
8 -
Comply With and ALL ALL ALL ALL 24 13
Use Procedures (1) 56 45
9
-
Operate Control NIA NIA NIA NIA 24 13
Boards (2) 56 45
7
-
Communicate and ALL ALL ALL ALL ALL ALL
Interact -- ~ __ -
,emonstrate ALL ALL ALL ALL NIA NIA
Supervisory
Ability (3) __ -
Comply With and 23 36 23 3 NIA NJA
Use Tech. Specs.. (3) -3- __ 5 _ . __ -
Notes:
(1) includes Technical Specification compliance for an RO.
(2) Optional for an SRO-U.
(3) Onlv amlicabie to SRBs.
Instructions:
Circle the applicant's license tfle and enter one or more emnt numbers that will allow the examiners to
evaluate every applicable competency fpr every applicant.
Author:
NRC Reviewer:
\
NUREG-1021, Brae Revision 9
I
ES-401 Written Examination Form ES-401-6
Quality Checklist
Bate of Exam: Exam Level: RO I SI30
5. Question duplication from the license screening/audit exam was controlled as
Bank use meets limits (no more than 75
7. Between 50 and 60 percent of the questions on
the RO exam are written at the
comprehension/analysis level; the SRO exam
may exceed 60 percent if the randomly
selected K/As support the higher cognitive
Printed Name / Signature:,- Date
a. Author dd.l
b. Facility Reviewer(')
e. NRC Chief Examiner(*)
1. NRC Regicnal Supervisor(^)
ES-401 Written Examination Form ES-401-9
Harris 2004 Review Worksheet
I 1. 1 2. I 3. Psvchometric Flaws 1 4. Job Content Flaws 1 5. Other 1 6. 1 7.
Explanation
I I lunitsl ward WAI I Only 1
Instructions
[Refer to Section B of ES-401 and Appendix B for additional information regarding each of the following concepts.]
1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level
2. Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are aweptable).
3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated hue/false statements.
One or more distractors is not credible.
One or more distractors is (are) partially correct (e.$. if the applicant can make unstated assumptions that are not contradicted by stem).
4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e.. the question has a valid K/A but, as written, is not operational in content).
The auestion rewires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from
iemory).
ThJ qLestioii contains data with art uirea 1slic level of accuracy or incoriststent Liiits (s.g panel iiicter it7 percent with question in gallons)
The question reqdtres reverse logic oi application mnlpared to the ]oh reqiiirerrrel1:s.
5. Check questions that are sampled for conformance with the approved WA and those that are desiqnated SRO-only (KIA and license level mismatches
are unacceptable).
6. Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacerxent), in need of (E)ditorial enhancement, or
(S)atisfactory?
7. At a minimum, explain any " U ratings (e.g., how the Appendix B psychometrii attributes are not being met).
Explanation
ROISRO Coi hestion
-
maintain plant temp. (MOD)
Althou h the question was modified. it retains enough of thf
origina? characteristics that the original question can be
recognized. The correct answer vosition was not changed
for the modified question.
Modified Question to maintain a diffeFeflt temperature
and changed the position of the correct answer. (MOD)
Question Acceptable after modifications.
U 02262.4.4 Distractors C and D also correct On (Harris
2000 Exam1
IThe correct answer can easily be picked by the information
r,y,u.., ..._~._I...
To elicit the correct answer without allowing no other correc
answer. the stem must include in accordance wlth Alarm
Panel Procedure ALB-06-7-3 and ALE 06-8-4 because the
AQP is the first procedure referred to.
Question to be Replaced
Replaced Question.
Question accedable after reDlacement
S 013K3.03(MOD)
2. I 3. Psychometric Flaws 4. Job Content Flaws 5. Other 7.
.OD -
1-5) Stem
- Focus
-
~
2
ion was modified, it retains enough of the
'csthat the original question can be
rrect answer position was not changed
stion. Change the position of the correct
Change the word "senses ..." in the correct answer. SuggeS
- "indicates an all rods in condition".
Made Changes as requested.
Question acceotable after dranws.
U 'YE14EA1.2 First section of stem not requiied (Teaching :n
stem) On -002.Exa.a
Question to be Replaced
Ouesfian Replaced.
Distractor B is not eredible. (NEW)
Distractor reworked question acceptable. .
E 1054AK1.02 (NEW)
Change the question to ask "which of the following describer
the strategy used to re-establish feed under these
conditions.
I Made Changes as requested
(NEW)
U 003A4.01 KIA is to mariba!ly opfm!e or monitor in !he contrr
room: seal injection. This :s adually monitoring charging
flow. Backwards logic (MOD)
Question does not match WA
Replaced Question. (Direct).
3. Psychometric Flaws J.ob Content Flaws I 5. Other
Explanation
unit
U 013K6.01* Question very simple On bast Harris NRC
Exam2002 to he Replaced
Replaced Quest/on.(Oirect)
Question acceatable.
S 001K4.07 Colon required after Given the following condition:
(MOD)
IChange the order of the choices and the correct answer,
Made changes as requested.
Question Acceptable.
Replaeed Question (MOD)
Question Repplaced
4. Job Content Flaws 5. Other I I
6. 7
Explanation
Removed Portion of stem that was teaching. Changed
Added /AW A OP--020.
Question Acceptable
S 056AA2.09 (MOD)
I I
U 005KAK2.02Does not match WA. Distraciors A and D may
not be credible. (MOD)
I I I Replaced Quesfion.(NEW)
Question Acceptable
EA2.01 Question could be asked without the top billlet in
stem and would be the same. Biswnnect betwen stem
bullets add conditions to fit C.?. (MOB)
emoved top bullet from stem of question
nd WA mismatch WA and Question do not
match (NEW) Is it fair to assume that the other parameters
Replaced Questfon. (NEW
I
1.
-OK
4. Job Content Flaws I 5. Other I I
6. 9.
FIH) Explanation
.ink
-
H
No Chanaes made to awestion (MOD)
-
H Does not meet KIA, hank question with a drawing
that is a direct lookup. (DIRECT).
Replaced Question. (Mew)
uestion written. does not reallv match KIA
Qwestion Reworked.
Reworked Question accwtable.
-
F 064K2.03 Distractors A and B not credible, very
wnvoluted.(MOD)
IRequires a level of knowledge k y o n d 1 hr T/S. Beyond
Scope for RO.
I 1Replaced Question (NEVU)
New question acceptable
H E 025K5.10 Need to add IAW GOP-005 to stem
Stem should state that indications converted to per cent
where required. (MOD)
-
F
C distractor not plausible,
Replaced C distractor. Made other changes as
r@QOeSted.
- I INew auestion acceptable
4. Job Content Fiaws 5. Other 6. 7.
UEi Explanation
S
have more than one correct answer if
is not included in stem IAW ..
Made changes as requested.
New uestion acceptable
Discuss using the whole procedure if it does not answer othc
uestions. This will better match WA Obtain and interpret.
&awing provided needs to be reprinted to avoid confusion
with Normal Operations and Abnormal Operations Areas.
Replaced Question.(NEW)
Distractor A analysis does not fit distractor
Changed distractor analysis.
is and the answer would be the same: Very similar question
to RO # I No calcuiation is really required. (MOB)
Replaced Question. (MOD)
New question accwtablc
4. Job Content Raws I 5. Other 7.
I
Explanation
linutia
Changed disfracfors. Still very simple
Question acceetable
on
LJ 0 3 7 ~ ~ 2 . 1 0 Harris NRC ~xaaa.2aoa(MOD)
Backwards Logic
Question was not changed, due to comments from licensee.
Seemed exceptable.
U VVE15EK1.2 Distractors A and C may not be plausibe.
(DIRECT)
Replaced Distractors
U 004A1.06 Question should ask what is CVCS response.
Question to be Replaced
On Last Harris NRC Exam2002
Replaced Question (NEW,
ometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Explanation
U 035K1.01 Stern is somewhat wnfusin does the AFW
isolation come from mnditions on B Sk.Does this also give
a Feed Water isolation (MOD)
TOO close to #45.
Replaced Question. (New)
New question acceptable
I I
U WE05EK2.2 Question to be Replaced
On NRC Exam 2000
Replaced Question.
May have more than one correct answer. (Direct)
Reworked question
Question acceptable
S 62.2.1 need to rearrange the answers so that they are
different than the bank question. (MOB)
In stein Six hours has (8) associated with it.
Made chanaes as requested.
1
K/A match (MOD).
Replaced QuestionpEW)
Changed question # 41
Question acceptable
3. Psychometric Flaws
Explanation
I I I I I I i
E 061K6.01 ( requires a statement with NO OPERATOR
ACTION (MOD)
Repkaced Question.(Direct)
New Question acceotable
controls. Needs to be c t 1 a n g e d . m
WA mismatch.
Made changes to $uesfion(.Newj
Question acceDtable
062A4.04 Change answer and distractors to indicate local
operation of breaker. (DIRECT).
Made changes as requested.
3. Psychometric Flaws 4.Job Content Flaws I 5. Other
Explanation
, I
)b- Minutia
U 010K1.06Question to be Replaced
On Last Harris NRC Exam.2002
I Replaced Question.
I Distractor D is also correct. Need to refine stem or
distractor.
Reworked question
Made changes to sfem.(NEW)
Question accwtable
S 015A2.05 (DIRECT).
S 1045K4.06(MOD)
T-
E 064.43.07is loading of the sequencers different between a
loss of power and a Safety injection signal (MOD)
No Changes made.(MOD) Question acceptable
U
II G2.2.24Question On Last Harris NWC Exam2002
to be Replaced
Replaced question (MOD)
New auestion accewtabie
3. Psychometric Flaws 4. Job Content Flaws 5. Other
Explanation
U G2.4.3 Question 01, Last Harris NRC Exam.2000
Change the order of the distractors and nswer.
Change the stem and 700 psig to 690 psig This will be the
correct answer.
Question to be replaced
Replaced Question (MOB)
B not credible. (MOD)
Repfaced Question (NEW
Removed reference page.(MOD)
Question acceptable.
U 038EA1.36 IAW what procedure, Distractor C and D is not
plausible (DIRECT).
Modified Question (MOD)
Question acceptable
U 073K5.02 Does not meet WA. (NEW) Job link
similar to# 14.
Changed stem to address job h k . ( N E W )
IReworked question.
Question Acceptable
2 1 3 Psyctiometnc Flaws II 4 Job Content Flaws 5.Other 7.
-
I
Focus
TIF
I
- / Back.
nits ward s I Explanation
i
E 012K2.01 Question to be Replaced
2000 Harris NRC Exam.
NEW Question Acceptable
different, or include in the stem IAW procedure...(NEW)
License checked to see if any other precautions and
limitations were applicable. Appears that none are.
Question Acceptable
U 068A3.02 Distractors B and C does not appear to be
cfedibls. (MOD)
Distrnctors reworked
U G2.4.1 Why is :he statement "Containfnq NO Immediate
Operator Articns" iiicludecl in tho stem this makes thcr
question a ncgitive questiori (NEW)
Replaced Uuect;on. (NEW)
3 Psychometric Flaws 4 Job Content Flaws 5 Other 6 7
Explanation
Question Replaced (Direct)
Removed teaching in stem and reworked distractors.(NEW)
eplaced Question.(Direct)
SRO ONLY
Utility changed answers to contain a selection of correci
Question Acceptable
I 1 / 1 1 I / Uti/ity addedprocedure to stem and referenced TIS and
its bases.
Question Acceotable
3. Psychometric Flaws 4. Job Content Flaws 5. Other 7
Utility replaced with a modified Harris bank questjon,
appears to address WA
E ClltG2 4 6 tmgh qJCC.l:On. ATWS needs to mentioiled in
s!em, nr scrar:i attempted horn mntrol room etc !NEW)
I Utility Added In accordance with FRP-Sd in the stem.
Question Acceptable
il G2.1.2. Not SRO only knowledge (DIRECT)
Utility requested new K/A.
NRC Replaced W A withG2.1.7
Utillty replaced question. Matches K/A
New Question Acceptable
01 1G2 4 6 Toach.ng in stem (Heat s.nk condit o m anc
answers (1 arge hrem LOCAI Not SRO only (MOIh
Utility modified question to eliminate "teaching" and
added a decision point to make SRO only.
I
1.
.OK
=/H)
3. Psyrhornetric Flaws 4. Job Content Flaws 5. Other 7.
I
linutia
- -
-
H
I
New Question Acceptable
I1
'
H U :01262.4.6 Dislracrors C and 3 not credib:e S.comut he
j reset for a cerlalo perloo of tinia. NetJrn to S.1 orily wnen
- direfled by path 1 no1 credible Not SHO only
jawstion Qn..H.a.rris NRC Exam= lo he Repleced
Utility replaced question with Hamis Bank question EPP-
3.15-R511 meets WA. (Direct)
-
F
Editorial changes to question to create a condition wher
i f is possible to loose power to just LlRPi, changed 43(b)
Imn-.pemaiid Do.s?ion indicators in the distractors
makes the ouestion much e.as!e!.
Question Acceptable I
2. I 3. Psychometric Flaws 4. Job Content Flaws 5. Other
linutia
-
-
tied to 43. f will send reference.
Utifity sent SRQ only objebjive to verify 43 tie. Also
modified the question to pertain to the SRQ position.
Question AcceDtable
?I 038EA2.08 Two possible answers B and D. Question @ I
Last Harris NRC Exam2002 to be Replaced
Question replaced with new question.
I-
Determine if Preferred procedure is recwired to be i n the
stem.(NEW)
Question modified amd is acceptable.
WE14EA2 7 d n not ncldde act ons nf EPP-12 tu veiily that
t v a ctm: spray [~:riiipsnot reqJireJ SA1 (MOD)
Revised answer b per EPP-Of 2; Provided reference to
verify that only 1 CSP is required.
U 013A2.01 Question daes not meet WA Not SRO only
knowledge. On Harris NRC Exam2000 to be Replaced
eplaced with New question. (NEW)
G2.2.19 Not SRO Only knowledge. Question On Harris NR(
U Exam.2000
to be Replaced
Replaced with Direct Harris Bank Question BP-2~4S62
(Direct) Remove date in stem if not reauired
3. Psychometric Flaws 4. Job Content Flaws 5. Other 7.
Replaced with new question
-. .....and answe.ers..vot in svnc,.Af!emut to close which
Stem
MSlV prior to isolatinq the.other SIGS:
IReworked Question
Question Acceptable
S WE0662.1.32 State the procedure that the crew has enterei
l(NEW)
cafion ofpracedure in stem per
Made changes as requested.
Question Acceptable
U G2.2.10 Does not match WA. (DIRECT)
Replaced with new question
Watches K/A (DIRECT) from INPO bank
Question Acceptable
Backward logic. (NEW
Repraced with new question
Question Acceptable
3 . Psychometric Flaws 4. Job Content Flaws 5. Other 5. 7.
- Explanation
dinutia /El
S
=
In accordance with EFF-006 D Is not correct. (Additional
oesno m a c
Utility requested new WA.
KIA replaced wlth 2.3.10
Replaced with modified Harris LOCT Exam Bank
Matches W A need t o see bank suestion t o ensure
auestion le modified
G k i f i e d to place plant in a certain condition,revised
stem and distractors io eliminate conc@rnfor more than
I correct answer.
Question AccfDtable
Utility requested new MA.
New W A randomly selected 058G 2.1.33
Utirity replaced question with new question for new MA.
Explanation
1 U 1G2.4.14 (MOB) Negative question ask in positive form.
I
l l
I
Reworked in to positive question. revised stem and
distracfors to remove backwards logic.
IQuestion Acceptable
III
ES-404 Written Examination Grading Form ES-403-1
Quality Checklist
xam
L
Level: ROISRQ
Initials
Item Description
I. Clean answer sheets copied before grading
2. Answer key changes and question deletions justiiied and
documented
3. Applicants' scores checked for addition errors
(reviewers spot check 25% of examinations)
4. Grading for all borderline cases (80 +I- 2% overall and 70 +/-
4% on the SRO-only) reviewed in detail
5. All other failing examinations checked to ensure that grades
are justified
6. Performance on missed questions checked for training
deficiencies and wording problems; evaluate validity of
Printed Name I Signature Date
r examinations graded by the
5 of 5 NUREG-1021. Draft Revision 9
ES-501 Post-Examination Check Sheet Form ES-501-1
I Task Description Date
Complete
I. Facility written exam comments or graded exams received and
verified comdete
2. Facility written exam comments reviewed and incorporated and
NRC grading completed, if necessary
1 3. Operating tests graded by NRC examiners
NRC Chief examiner review of written exam and operating test
grading completed
1 5. Responsible supervisor review completed
License and denial letters mailed
I 9. Examination report issued (refer to NRC MC 86380(312)
NUREG-1021, Draft Revision 9 24 of 23