ML041170056
| ML041170056 | |
| Person / Time | |
|---|---|
| Site: | Harris  |
| Issue date: | 10/30/2003 |
| From: | Ernstes M Operator Licensing and Human Performance Branch |
| To: | Scarola J Carolina Power & Light Co |
| References | |
| 50-400/04-301 50-400/04-301 | |
| Download: ML041170056 (38) | |
See also: IR 05000400/2004301
Text
COVER SHEET
HARRIS EXAM
50-40012004-301
FEBRUARY 23 - 27,2004
& MARCH 4,2004 (WRITTEN)
J [ < I
i, 1x1
", [XI
--[Y1
4'rN
4
ii
1
ES-202-1 - Exam Preparation Checklist
E$-201-2 - Exam Outline Quality Checklist
ES-201-3 - Exam Security Agreements
ES-301-1 - Admin Topics Outline
E%-301-2 = Controi Room Systems & Facility
Walk-through Test Outline
ES-301-3 - Operating Test Quality Checklist
[k] ES-301.4-Sim
Scenario Quality Checklist
ES-301-5 - Transient i% Event Checklist
ES-301-6 - Competencies Checklist
ES-40Z-d- Written Exam Quality Checklist
tten Exam Review Worksheet
I/- [ X I
4 x I
1/18? I
/ 1 1( I
6
/"y]
am Grading Quality Checklist
--G- I
ES-5Ql-1 - Post Exam Check Sheet
...
...
~
..........-.
....
Examina!ion Preparation Checklist
Forw ES-201-1
.....
.-
....
...
......
.......
......
F'acii.ty:
rlariis
Date oi Exaninatiol-i:
.......
.......
......
.... .-.
......
.
......
Examinations Developed by:
racilitv ! NRC (circle one)
........
..... .- .......
A
......
.... .-
.....
Target
Date*
Task Description I Reference
Chief
Examiner's
Initials
I
I
11
-120
I 2. NRC examiners and facility contact assigned (C.7.d; C.2.e)
I&./y 1
6. Integrated examination outline(s) due (C.1 .e & f; C.3.d)
N / L H
II
by NRC and feedback provided
9. Preliminary license applications due (C.1 .I; C.2.g; ES-202)
14. Final applications reviewed: assignment sheet updated; waiver
letters sent (C.2.g, ES-404)
ctoring/written exam administration guidelines reviewed with
My licensee and authorization granted to give wrltten exams
Target dates are keyed to the examination date identified in the corporate notification letter.
They are for planning purposes and may be adjusted on a case-by-case basis in coordination
23 of 25
NUREG-1021, Draft Revision 4
ES-20 I
Examlndtlon Outline Quality Assurance Checklist
Form ES-201-2
Item
1.
W
R
I
T
F
N
2.
S I
M
'r
-
3.
W
i
.r
4.
(;
c
N
F
R
A
L
.
.__ ....
h. Assess whaherthe outtine was systrmatica6'yand raridumly prepmed in accurdance with Sectim D I o
ES4O 1 and whither all knowledge and ability cirtegrries are approptiate sampkd
c. Assess whcther the ouilitline overmaphasizes my systam, evolutions, m gm!nc tixIks. __ ....
d. Assess hether the rrpetition f n m prrviuu;~ examinatinl outlines is cxccssiw.
repeatedan suhsequent days.
(I)
the outline(s) contais(s) the requirrd number of control rmm anrl in-plant task,
(2)
no I B O ~
than 30% of the tcd material is repeated froin thc hst NRC cxrminatia,
(3)
- no task B L ~
duplicated from the applicants audit testisj, and
(4)
no more than 80% of thc operatine test is takcn dire& froin thc licensee's ? x m bank.
~
.
b. Verifythat:
( I )
(2)
(3)
Ole uak~ are dirttihutcd among the safety hnctim groupings as specified in ES-301,
O ~ C
task is c.onductcd in a low-power or shutduw condition,
4 6 (2 - 3 for SKO-(I) of the task require the applicant to impllemmt an altmate psth prwedmc,
. ,
..
(4)
(5)
c. Verify that the required administmix topics arc cowred.
onc in-phi1 task tests thc applicant's response to M emergency or ahnormil cmditim, and
thc inplant walk-tllro~yh nquircs the applicant to enter the RCA.
d. Detcrminr if there are enourh diffcmt outlitlinrs to test the projrzted niimhw and mix of applicants and
ant.spn:, IC pnon IPS mc ut mg
~
~~~~~~
h. Asscss whether the IO CFR 55.41143 and 55.45 sampling is appqiriate.
c. Ensure that K:A importance ratin@ (cxcept far plant-spccitic priorities) are at least 2.5.
d. Clwk for dlrplicatini and orerkp among rxam SeCtimS.
e. Check the entire exam far hdance of coverage.
f. Assess whether the exam fits the nppropriatejuh level (RO or SRO).
-.
-~
.~
~
~
~
a. h t h M
h. Fxility Reviewe(*)
c. ChiefExarniner
d. NRC Supe~iisa.
.-
Draft Revisioii 9
Examination Security Agreement
Form ES-201-3
1.
Pre-Examination
I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 2-23-04 as of the date of
my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the
NRC chief examiner. I understand that I am not to instruct, evaluate,
these licensing examinations from this date until completion of exami
Furthermore, I am aware of the physical security measures and requireme
violation of the conditions of this agreement may result in cancellation of the e
licensee. I will immediately report to facility management or the NRC chief ex
been compromised.
2.
Post-Examination
To the best of my knowledge, I did not divulge lo any unauthorized persons any inf
during the week(s) of 2-23-04,
From the date that I entered into this security agreement until the completion of examination administration, I did not
instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted
below and authorized by the NRC.
vide performance feedback to those applicants scheduled to be administered
s specifically noted below and authorized by the NRC.
e facility licensee's procedures) and understand that
ti enforcement action against me or the facility
s or suggestions that examination security may have
concerning the NRC licensing examinations administered
PRINTED NAME
JOB TITLE / RESPONSIBILITY
DATE NOTE
. - . -
__
-.
.- -
-
.-
apr r A s T r " z + c
t 3 .
-. -
.-
_ _
25 of 25
NUREG-1021, Draft Revision 9
Examination Security AgreeMent
Form ES-201-3
V&y. L * 4- 2
1.
Pre-Examination
I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of &3as
of the date of
my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC
chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these
licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the
NRC.Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand
that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility
licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have
been compromised.
2.
Pos t-Examination
To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered
during the week(s) of Z/S$~/!/O~! From the date that I entered into this security agreement until the completion of examination administration, I did not
instruct. evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted
below and authorized by the MRC.
PRINTED NAME
JOB TITLE / RESPONSIBILITY
SIGNATURE (1)
BATE
SIGNATURE (2)
DATE NOTE
I-_
I
-
I_-
.I
14.
15.
I
-
.. - ...
I-
-
NUREG-1021, Revision 9, Draft
- --.
p
+.
c
Examination Security Agreement
Form 3-201-3
v
I.
Pre-Exam ination
I acknowledge that I have acqirired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of w
a
s
of the date of
my signature. I agree that I will not knowingly divulge any information about these examinatfons to any persons who have not been authorized by the NRC
chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled :o be administered these
licensing examinations from this date until comp!etion of examination administration. except as specifically noted below and authorized by the
NRC.Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand
that violation of the conditions of this agreernent may result in cancellation of the examinations andlor an enforcement action against me or the facility
licensee. 1 will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have
been compromised.
2.
Post-Examination
To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered
during the week(§) of
G~3/b/~'f , From the date that I entered into this security agreernent until the completion of examination adminiStratiQR, I did not
instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted
below and authorized by the NRC.
PRINTED NAME
JOB TITLE I RESPONSIBiLITY
SIGNATURE (1)
DATE
SIGNATURE (2)
DATE NOTE
1.
2.
3.
4.
5.
NUREG-1 021, Revision 9, Draft
Administrative Topics Outline
FORM ES-401-1
Describe Activity to be Performed
Perform a manual Shutdown Margin Calculation per OST-1036
(2.1.25 - 2.8 / 3.j)
Date of Examination:
Examination Level:
Operating Test Number:
I
Administrative Topic
Conduct of Operations
Conduct of Operations
Equipment Control
Radiation Control
Determine Average RCS Boron Concentration per EOP-EPP402
(2.1.20
- 4.3 14.2)
Determine clearance requirements for a CSlP per OPS-NGGC-
1301
(2.2.13 - 3.6 I3.8)
Determine PEDE While Working in a High Airborne Area
(2.3.10 - 2.9 13.3)
NOT APPLICABLE FOR RO
SROs. 80 applicants require only 4 items unless they are retakin
NUREG-2021. Draft Revision 9
Administrative Topics Outline
FORM ES-301-1
Examination Level:
Operating Test Number:
Administrative Topic
(see Note)
Conduct of Operations
Conduct of Operations
I
Describe Activity to be Performed
(KA#-ROlmp/SROlmp)
-
Perform a manual Shutdown Margin Calculation per OST-1036
(2.1.25 - 2.8 I3.1)
Determine Average RCS Boron Concentration per EOP-EPP-00;
(2.120 - 4.3 I 4.2)
Determine clearance requirements for a CSlP per OPS-NGGC-
1301
(2.2.13 - 3.6 13.8)
Determine TEDE While Working in a High Airborne Area
(2.3.10 ~ 2.9 13.3)
Determine Protective Action Recommendations per PEP-f 20
(2.4.44 - NA I4.0)
SROs. RO applicants require only 4 items unless they are retakin
only the administrative topics, when 5 are required.
NUREG-2021. Draft Revision 9
FORM ES-301-2
Control Room I In-Plant Systems Outline
Examination bevel:
Operating Test Number:
/
Control Room Systems (8 for 80; 7 for SRO-I; 2 or 3 for SRO-U)
SystemIJPM Title
Safety
Function
Code*
I
Type
a.
(062A4.07 - 3.1)
Transfer 6.9KV Buses from Unit Auxiliary Transformers to the
Startup Auxiliary Transformers per GP-156.02
1010
(02284.01 - 3.6)
3
I (0000T 1 EA1 .I3 - 4.1
isolate the SI Accumulators following a LOCA per PATH-1 and
GP-I 10
I A I I
I
C.
I
M l S l L 4s
(061A3.01 - 4.2)
d.
Start the Turbine Driven Auxiliary Feedwater Pump per OP-I37
2
(006A4.05 - 3.9)
M I A I S 1 L
I
e.
Transfer to Cold Leg Recirculation per EOP-EPP-010
f.
Perform Control Rod and Rod Position Indicator Exercise per
OST-1005
Place Audio Count Rate Drawer in Service per OP-I05
h.
Align CCW to Support RHR System Operations per GP-145
In-Plant Systems (3 for RO; 3 for SRO-I; 3 or 2 for SRO-U)
1
(001A2.11 - 4.4)
I
D i A l S
7
(02584.02 - 3.9)
8
(Q08A4.01 - 3.3)
I
D l L l S
1
(004A2.07 - 3.4)
Local Makeup to the VCT Using the Emergency Boration Valve
012A4.04 - 3.3
j'
Perform local actions for placing an OTAT channel in Test per
OWP-RP-OI
Locally operate a SG PORV per EOP-EPP-012 and OP-126
(000074EA1.04 - 3.9
- Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)lternate path, (C)ontrol Room, (S)imulator, (L)o\\
Power, (R)CA
NUREG-1021. DraR Revision 9
FORM E§-301-2
E§-301
Control Room / In-Plant Systems Butline
nAmis
Date of Examination:
SRO-U
Operating Test Number:
/
-
Control Room Systems (8 for RQ; 7 for SRO-I; 2 or 3 for SRO-U)
SystemlJPM Title
062A4.07 - 3.1
NOT APPLICABLE FOR SRO-U
OP-l I 0
d.
NOT APPLICABLE FOR SRB-U
NOT APPLICABLE FOR SRO-U
NOT APPLICABLE FOR SRO-U
NOT APPLICABLE FQR SRO-U
h.
NOT APPLICABLE FOR SRO-U
In-Plant Systems (3 for RO; 3 for SRO-I; 3 or 2 for SRO-U)
1
(004A2.07 - 3.7)
I.
Local Makeup to the VCT Using the Emergency Boration Valve
(012A4.04 - 3.3)
.
Perform local actions for placing an QTAT channel in Test per
OWP-RP-01
4 s
k.
I
"13
I (000074EA1.04-4.1:
Locally operate a SG PORV per EOP-EPP-012 and OP-126
- Type Codes: (D)irect from bank, (M)odihed from bank. (N)ew, @)Iternate path, (6)ontrol Room, (S)imulator, (L)o
Power, (I3)CA
NUREG-1021. Draft Revision 9
-I.s.
, .. . . ..
-.
I_.._
. ..I ....
s
-
. . ... . .
-.
Operating Test Quality Asurwck Checklist
F
o
~
ES-30 1-3
.. I .
, ,
.... .
-- . . . , , ,
... ,,--
-t-
23-27 l*r
NUREG-1021, Revisicm 9 Draft
~~~~
Simulator Scenario Quality Assurance Checklist
Form ES-301-4
ES-30 I
Facility: IIARWIS
Date of Exam:
2/23- 2716 3
Operating Test No.:
QUALITATIVE ATTRIBUTES
I.
'The initial conditions are realistic. in that some equipment andior instrumentation may be
out of servic.e, but it does not cue the operators into expected events.
'The scenarios consist niostly of related events.
Each event description consists of
e
2 .
3.
the point in the scenario when it is to be initiated
the malfunction(s) that are entered to initiate the event
the symptonisicues that will be visible to the crew
the expected operator actions (by shitt position)
the event termination point (if applicable)
e
e
4.
No more than one non-mechanistic failure (e.&, pipe break) is incorporated into the
scenario without a credihle preceding incident such as a seismic event.
The events are valid with regard to physics and thermodyiamics.
5 ,
6.
Sequencing and timing of events is reasonable, and allows the examination team to
obtain comvlete evaluation results commensurate uzith the scenario iihiectives.
..
7.
If time compression techniques are used, the scenario summary clearly so indicates
Operators have sufficient time to carry out expec.ted activities without undue time
constraints. Cues are given.
8.
The simulator modeling is not altered
9
The scenarios hwc been validated. Pursuant to IOCFK55.46(d);any open simulator
perfcxmance deficiencies have been eviiluated to ensure that functional fidelity is
inaintained while running the planned scenarios.
IO. Every operator will be evaluated using at lcast one new or significantly modified
scenario. All other scenarios have been altered in accordance with Section D.5 of ES.
I I . All individual operator compctencies can he evaluated, as verified nsing Form I S 3 0 1-6
(submit the form along with the simulator scenarios).
12. Each applicant will he significantly involved in the minimum number of transients and
events specified on Form ES-301-5 (submit the fomi with the simulator scenarios).
13. The level of difficulty is appropriate to support kensing decisions for each crew
oosition.
~~
~
~~~
~.
~~
Initials
Simulator Scenario Quality Assurance Checklist
Form ES-30 1-4
TARGET QUANTITATIVE ATTRIBUTES (PER SCENARIO;
~
~
~~~
~~~~~
~
Transient and Event Checklist
Form E§-301-5
HARRIS
OPERATING TEST NO.:
p- DA
I
SRO-I
I
Instructions: (1)
(2)
Enter the operating test number and Form ES-D-1 ewnt numbers for
each evolution type.
Reactivity manipulations may be conducted under normal or controMed
abnormal conditions (refer to Section D.5.d) but must be signilcant per
Section C.2.a of Appendix D. * Reactivity and normal evolutions may
be replaced with additional instrument or component malfunctions on a
one-for-one basis.
Whenever practical, both instrument and component malhnctions
should be included; only those that require verifiable actions that
provide insight to the applicant's competence count toward the
minimum requirwent.
,
(3)
Author:
NRC Reviewer:
NUREG-1021, Draft Revision 9
Competencies Checklist
Form ES-302-6
SCENARIO
T
\\RIB
3
_s
SCENARIO
Competencies
2
_.
-
4
1 3
5
-
-
1 3
4 5
-
1 3
4 5
-
ALL -
-
NJA -
-
4
2 2
3 4
5
Interpret I Diagnose
Events
and Conditions
2 3
4 5
6 7
12 3
4 5
6
1 2
3 5
6 7
8
ALL
2 4
6 7
2 4
5 6
9
Comply With and
Use Procedures (1)
ALL
ALL
ALL
Operate Control
Boards (2)
2 4
5 6
7
ALL -
ALL
ALL -
ALL
ALL
~
ALL
__
2 3
5
__
ALL
__
ALL
ALL
Communicate and
Interact
,emonstrate
Supervisory
Ability (3)
Comply With and
Use Tech. Specs.. (3)
Notes:
2 3
-3-
3 6
3
_.
__
(1) includes Technical Specification compliance for an RO.
(2) Optional for an SRO-U.
(3) Onlv amlicabie to SRBs.
Instructions:
Circle the applicant's license tfle and enter one or more emnt numbers that will allow the examiners to
evaluate every applicable competency fpr every applicant.
Author:
NRC Reviewer:
\\
NUREG-1021, Brae Revision 9
I
Written Examination
Form ES-401-6
Quality Checklist
Bate of Exam:
Exam Level: RO I SI30
5.
Question duplication from the license screening/audit exam was controlled as
Bank use meets limits (no more than 75
7.
Between 50 and 60 percent of the questions on
the RO exam are written at the
comprehension/analysis level; the SRO exam
may exceed 60 percent if the randomly
selected K/As support the higher cognitive
Printed Name / Signature:,-
Date
dd.l
a. Author
b. Facility Reviewer(')
e. NRC Chief Examiner(*)
1. NRC Regicnal Supervisor(^)
Written Examination
Form ES-401-9
Harris 2004 Review Worksheet
I
1. 1 2. I
3. Psvchometric Flaws
1 4. Job Content Flaws
1 5. Other
1 6. 1
7.
Explanation
I
I
lunitsl ward I WA I Only 1
Instructions
[Refer to Section B of ES-401 and Appendix B for additional information regarding each of the following concepts.]
1.
2.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level
Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are aweptable).
3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated hue/false statements.
One or more distractors is not credible.
One or more distractors is (are) partially correct (e.$. if the applicant can make unstated assumptions that are not contradicted by stem).
4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e.. the question has a valid K/A but, as written, is not operational in content).
The auestion rewires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from
iemory).
ThJ qLestioii contains data with art uirea 1slic level of accuracy or incoriststent Liiits (s.g panel iiicter it7 percent with question in gallons)
The question reqdtres reverse logic oi application mnlpared to the ]oh reqiiirerrrel1:s.
5.
Check questions that are sampled for conformance with the approved WA and those that are desiqnated SRO-only (KIA and license level mismatches
are unacceptable).
6.
Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacerxent), in need of (E)ditorial enhancement, or
(S)atisfactory?
7.
At a minimum, explain any " U ratings (e.g., how the Appendix B psychometrii attributes are not being met).
Explanation
ROISRO Coi -
hestion
maintain plant temp.
(MOD)
Althou h the question was modified. it retains enough of thf
origina? characteristics that the original question can be
recognized. The correct answer vosition was not changed
for the modified question.
Modified Question to maintain a diffeFeflt temperature
and changed the position of the correct answer. (MOD)
Question Acceptable after modifications.
U
02262.4.4 Distractors C and D also correct On (Harris
2000 Exam1
IThe correct answer can easily be picked by the information
r,y,u .., ..._~._
I...
To elicit the correct answer without allowing no other correc
answer. the stem must include in accordance wlth Alarm
Panel Procedure ALB-06-7-3 and ALE 06-8-4 because the
AQP is the first procedure referred to.
Question to be Replaced
Replaced Question.
Question accedable after reDlacement
S
013K3.03 (MOD)
2. I
3. Psychometric Flaws
.OD
1-5) -
2
-
Stem
Focus
~ -
4. Job Content Flaws
5. Other
7.
ion was modified, it retains enough of the
'cs that the original question can be
rrect answer position was not changed
stion. Change the position of the correct
Change the word "senses ..." in the correct answer. SuggeS
- "indicates an all rods in condition".
Made Changes as requested.
Question acceotable after dranws.
U 'YE14EA1.2 First section of stem not requiied (Teaching :n
stem) On -002.Exa.a
Question to be Replaced
Ouesfian Replaced.
Distractor B is not eredible. (NEW)
Distractor reworked question acceptable. .
E 1054AK1.02 (NEW)
Change the question to ask "which of the following describer
the strategy used to re-establish feed under these
conditions.
Made Changes as requested
I (NEW)
U
003A4.01 KIA is to mariba!ly opfm!e or monitor in !he contrr
room: seal injection. This :s adually monitoring charging
flow. Backwards logic (MOD)
Question does not match WA
Replaced Question. (Direct).
3. Psychometric Flaws
.Job
Content Flaws I
5. Other
unit
Explanation
U
013K6.01* Question very simple On bast Harris NRC
Exam2002 to he Replaced
Replaced Quest/on.(Oirect)
Question acceatable.
001 K4.07 Colon required after Given the following condition:
(MOD)
S
IChange the order of the choices and the correct answer,
Made changes as requested.
Question Acceptable.
Replaeed Question (MOD)
Question Repplaced
4. Job Content Flaws
5. Other I 6. I
7
Explanation
Removed Portion of stem that was teaching. Changed
Added /A W A OP--020.
Question Acceptable
S 056AA2.09 (MOD)
U 005KAK2.02 Does not match WA. Distraciors A and D may
not be credible. (MOD)
I I
Replaced Quesfion.(NEW)
I
I I
Question Acceptable
EA2.01 Question could be asked without the top billlet in
stem and would be the same. Biswnnect betwen stem
bullets add conditions to fit C.?. (MOB)
emoved top bullet from stem of question
nd WA mismatch WA and Question do not
match (NEW) Is it fair to assume that the other parameters
Replaced Questfon. (NEW
I
1.
-OK
FIH) -
H
-
H
-
F
H
-
F
-
4. Job Content Flaws I 5. Other I 6. I
9.
.ink
Explanation
No Chanaes made to awestion (MOD)
Does not meet KIA, hank question with a drawing
that is a direct lookup. (DIRECT).
Replaced Question. (Mew)
Qwestion Reworked.
uestion written. does not reallv match KIA
Reworked Question accwtable.
064K2.03 Distractors A and B not credible, very
wnvoluted.(MOD)
Requires a level of knowledge kyond 1 hr T/S. Beyond
Scope for RO.
I
I
1 Replaced Question (NEVU)
New question acceptable
025K5.10 Need to add IAW GOP-005 to stem
E
Stem should state that indications converted to per cent
where required. (MOD)
C distractor not plausible,
Replaced C distractor. Made other changes as
r@QOeSted.
I
I New auestion acceptable
4. Job Content Fiaws
5. Other
6.
UEi
S
7.
Explanation
have more than one correct answer if
is not included in stem IAW ..
Made changes as requested.
New uestion acceptable
Discuss using the whole procedure if it does not answer othc
uestions. This will better match WA Obtain and interpret.
&awing provided needs to be reprinted to avoid confusion
with Normal Operations and Abnormal Operations Areas.
Replaced Question. (N EW)
Distractor A analysis does not fit distractor
Changed distractor analysis.
is and the answer would be the same: Very similar question
to RO # I No calcuiation is really required. (MOB)
Replaced Question. (MOD)
New question accwtablc
4. Job Content Raws I 5. Other
linutia
7.
Explanation
Changed disfracfors. Still very simple
Question acceetable
II
LJ 037~~2.10
on Harris NRC ~xaaa.2aoa (MOD)
Backwards Logic
Question was not changed, due to comments from licensee.
Seemed exceptable.
U
VVE15EK1.2 Distractors A and C may not be plausibe.
(DIRECT)
Replaced Distractors
U
004A1.06 Question should ask what is CVCS response.
Question to be Replaced
On Last Harris NRC Exam2002
Replaced Question (NEW,
ometric Flaws
4. Job Content Flaws
5. Other
6.
7.
Explanation
U
035K1.01 Stern is somewhat wnfusin does the AFW
isolation come from mnditions on B Sk.
Does this also give
a Feed Water isolation (MOD)
TOO close to #45.
Replaced Question. (New)
New question acceptable
U WE05EK2.2 Question to be Replaced
On NRC Exam 2000
I I
Replaced Question.
May have more than one correct answer. (Direct)
Reworked question
Question acceptable
S
62.2.1 need to rearrange the answers so that they are
different than the bank question. (MOB)
In stein Six hours has (8) associated with it.
Made chanaes as requested.
1
K/A match (MOD).
Replaced QuestionpEW)
Changed question # 41
Question acceptable
3. Psychometric Flaws
Explanation
E
061K6.01 ( requires a statement with NO OPERATOR
I
I
I
I
I
I
i
ACTION (MOD)
Repkaced Question.(Direct)
New Question acceotable
controls. Needs to be c t 1 a n g e d . m
WA mismatch.
Made changes to $uesfion(.Newj
Question acceDtable
062A4.04 Change answer and distractors to indicate local
operation of breaker. (DIRECT).
Made changes as requested.
3. Psychometric Flaws
4.
Job Content Flaws I 5. Other
)b- Minutia ,I
Explanation
U 010K1.06 Question to be Replaced
On Last Harris NRC Exam.2002
I Replaced Question.
Distractor D is also correct. Need to refine stem or
distractor.
I
Reworked question
Made changes to sfem.(NEW)
Question accwtable
S
015A2.05 (DIRECT).
S 1045K4.06 (MOD)
E T-
No Changes made.(MOD) Question acceptable
064.43.07 is loading of the sequencers different between a
loss of power and a Safety injection signal (MOD)
U
G2.2.24
Question On Last Harris NWC Exam2002
to be Replaced
I
I Replaced question (MOD)
New auestion accewtabie
3. Psychometric Flaws
4. Job Content Flaws
5. Other
Explanation
U
G2.4.3 Question 01, Last Harris NRC Exam.2000
Change the order of the distractors and nswer.
Change the stem and 700 psig to 690 psig This will be the
correct answer.
Question to be replaced
Replaced Question (MOB)
B not credible. (MOD)
Repfaced Question (NEW
Removed reference page.(MOD)
Question acceptable.
U 038EA1.36 IAW what procedure, Distractor C and D is not
plausible (DIRECT).
Modified Question (MOD)
Question acceptable
U
073K5.02 Does not meet WA. (NEW) Job link
similar to# 14.
Changed stem to address job hk.(NEW)
I Reworked question.
Question Acceptable
2 1
3 Psyctiometnc Flaws
I 4 Job Content Flaws
Focus
I
i
-
TIF
I
- /
Back.
nits I
ward
5. Other
7.
Explanation
s I
E
012K2.01 Question to be Replaced
2000 Harris NRC Exam.
NEW Question Acceptable
different, or include in the stem IAW procedure ...( NEW)
License checked to see if any other precautions and
limitations were applicable. Appears that none are.
Question Acceptable
068A3.02 Distractors B and C does not appear to be
cfedibls. (MOD)
U
Distrnctors reworked
U
G2.4.1 Why is :he statement "Containfnq NO Immediate
Operator Articns" iiicludecl in tho stem this makes thcr
question a ncgitive questiori (NEW)
Replaced Uuect;on. (NEW)
3 Psychometric Flaws
4 Job Content Flaws
5 Other
6
7
Explanation
Question Replaced (Direct)
Removed teaching in stem and reworked distractors.(NEW)
eplaced Question.(Direct)
SRO ONLY
Utility changed answers to contain a selection of correci
Question Acceptable
Uti/ity addedprocedure to stem and referenced TIS and
I
1
/
1
1
I
/
its bases.
Question Acceotable
3. Psychometric Flaws
4. Job Content Flaws
5. Other
7
Utility replaced with a modified Harris bank questjon,
appears to address WA
E
ClltG2 4 6 tmgh qJCC.l:On. ATWS needs to mentioiled in
s!em, nr scrar:i attempted horn mntrol room etc !NEW)
I Utility Added In accordance with FRP-Sd in the stem.
Question Acceptable
il
G2.1.2. Not SRO only knowledge (DIRECT)
Utility requested new K/A.
NRC Replaced W A withG2.1.7
Utillty replaced question. Matches K/A
New Question Acceptable
01 1G2 4 6 Toach.ng in stem (Heat s.nk condit o m anc
answers (1 arge hrem LOCAI Not SRO only (MOIh
Utility modified question to eliminate "teaching" and
added a decision point to make SRO only.
I
1.
.OK
=/H)
-
-
H
H
-
F
3. Psyrhornetric Flaws
4. Job Content Flaws
linutia
-
-
5. Other
7.
II
I1
New Question Acceptable
I
U :01262.4.6 Dislracrors C and 3 not credib:e S.
comut he
j reset for a cerlalo perloo of tinia. NetJrn to S.1 orily wnen
- direfled by path 1 no1 credible Not SHO only
jawstion Qn..H.a.rris NRC Exam=
lo he Repleced
'
Utility replaced question with Hamis Bank question EPP-
3.15-R511 meets WA. (Direct)
Editorial changes to question to create a condition wher
if is possible to loose power to just LlRPi, changed 43(b)
mn-.pemaiid Do.s?ion indicators in the distractors
I makes the ouestion much e.as!e!.
II
Question Acceptable
2. I
3. Psychometric Flaws
4. Job Content Flaws
linutia -
-
5. Other
tied to 43. f will send reference.
Utifity sent SRQ only objebjive to verify 43 tie. Also
modified the question to pertain to the SRQ position.
Question AcceDtable
?I
038EA2.08 Two possible answers B and D. Question @I
Last Harris NRC Exam2002 to be Replaced
Question replaced with new question.
Determine if Preferred procedure is recwired to be in the
I- stem.(NEW)
Question modified amd is acceptable.
WE14EA2 7 d n not ncldde act ons nf EPP-12 tu veiily that
t v a ctm: spray [~:riiips not reqJireJ SA1 (MOD)
Revised answer b per EPP-Of 2; Provided reference to
verify that only 1 CSP is required.
U
013A2.01 Question daes not meet WA Not SRO only
knowledge. On Harris NRC Exam2000 to be Replaced
eplaced with New question. (NEW)
G2.2.19 Not SRO Only knowledge. Question On Harris NR(
to be Replaced
U
Exam.2000
Replaced with Direct Harris Bank Question BP-2~4S6 2
(Direct) Remove date in stem if not reauired
3. Psychometric Flaws
4. Job Content Flaws
5. Other
7.
Replaced with new question
-.
Stem
. . . . . and answe.ers..vot in svnc,.Af!emut to close which
MSlV prior to isolatinq the.other SIGS:
I Reworked Question
Question Acceptable
S
WE0662.1.32 State the procedure that the crew has enterei
l(NEW)
cafion ofpracedure in stem per
Made changes as requested.
Question Acceptable
G2.2.10 Does not match WA. (DIRECT)
U
Replaced with new question
Watches K/A (DIRECT) from INPO bank
Question Acceptable
Backward logic. (NEW
Repraced with new question
Question Acceptable
3. Psychometric Flaws
4. Job Content Flaws
-
dinutia
=
5. Other
5.
7.
/El
S
Explanation
In accordance with EFF-006 D Is not correct. (Additional
oesno mac
Utility requested new WA.
KIA replaced wlth 2.3.10
Replaced with modified Harris LOCT Exam Bank
Matches WA need to see bank suestion to ensure
auestion le modified
Gkified to place plant in a certain condition,revised
stem and distractors io eliminate conc@rn for more than
I correct answer.
Question AccfDtable
Utility requested new MA.
New WA randomly selected 058G 2.1.33
Utirity replaced question with new question for new MA.
Explanation
I
1 U 1G2.4.14 (MOB) Negative question ask in positive form.
II
Reworked in to positive question. revised stem and
distracfors to remove backwards logic.
l l
I
IQuestion Acceptable
II
Written Examination Grading
Form ES-403-1
Quality Checklist
Item Description
I .
2.
3.
4.
Clean answer sheets copied before grading
Answer key changes and question deletions justiiied and
documented
Applicants' scores checked for addition errors
(reviewers spot check 25% of examinations)
Grading for all borderline cases (80 +I- 2% overall and 70 +/-
4% on the SRO-only) reviewed in detail
All other failing examinations checked to ensure that grades
are justified
Performance on missed questions checked for training
deficiencies and wording problems; evaluate validity of
Printed Name I Signature
5.
6.
xam Level: ROISRQ
Initials
L
Date
r examinations graded by the
5 of 5
NUREG-1021. Draft Revision 9
Post-Examination Check Sheet
Form ES-501-1
Task Description
II
I.
Facility written exam comments or graded exams received and
verified comdete
2.
Facility written exam comments reviewed and incorporated and
NRC grading completed, if necessary
11 3.
Operating tests graded by NRC examiners
NRC Chief examiner review of written exam and operating test
grading completed
11 5.
Responsible supervisor review completed
License and denial letters mailed
II 9.
Examination report issued (refer to NRC MC 86380(312)
Date
Complete
NUREG-1021, Draft Revision 9
24 of 23