ML041170056

Feb-March 2004 Exam 50-400/2004-301 Administrative Documents

ML041170056
Person / Time
Site:	Harris
Issue date:	10/30/2003
From:	Ernstes M Operator Licensing and Human Performance Branch
To:	Scarola J Carolina Power & Light Co
References
50-400/04-301 50-400/04-301
Download: ML041170056 (38)

See also: IR 05000400/2004301

Text

COVER SHEET

HARRIS EXAM

50-40012004-301

FEBRUARY 23 - 27,2004

& MARCH 4,2004 (WRITTEN)

J [ < I

i, 1x1

", [XI

--[Y1

4'rN

4

ii

1

ES-202-1 - Exam Preparation Checklist

E$-201-2 - Exam Outline Quality Checklist

ES-201-3 - Exam Security Agreements

ES-301-1 - Admin Topics Outline

E%-301-2 = Controi Room Systems & Facility

Walk-through Test Outline

ES-301-3 - Operating Test Quality Checklist

[k] ES-301.4-Sim

Scenario Quality Checklist

ES-301-5 - Transient i% Event Checklist

ES-301-6 - Competencies Checklist

ES-40Z-d- Written Exam Quality Checklist

tten Exam Review Worksheet

I/- [ X I

4 x I

1/18? I

/ 1 1( I

6

/"y]

ES-403

am Grading Quality Checklist

--G- I

ES-5Ql-1 - Post Exam Check Sheet

...

...

~

..........-.

....

Examina!ion Preparation Checklist

Forw ES-201-1

.....

ES-201

.-

....

...

......

.......

......

F'acii.ty:

rlariis

Date oi Exaninatiol-i:

.......

.......

......

.... .-.

......

.

......

Examinations Developed by:

racilitv ! NRC (circle one)

........

..... .- .......

A

......

.... .-

.....

Target

Date*

Task Description I Reference

Chief

Examiner's

Initials

I

I

11

-120

I 2. NRC examiners and facility contact assigned (C.7.d; C.2.e)

I&./y 1

6. Integrated examination outline(s) due (C.1 .e & f; C.3.d)

N / L H

II

by NRC and feedback provided

9. Preliminary license applications due (C.1 .I; C.2.g; ES-202)

14. Final applications reviewed: assignment sheet updated; waiver

letters sent (C.2.g, ES-404)

ctoring/written exam administration guidelines reviewed with

My licensee and authorization granted to give wrltten exams

Target dates are keyed to the examination date identified in the corporate notification letter.

They are for planning purposes and may be adjusted on a case-by-case basis in coordination

23 of 25

NUREG-1021, Draft Revision 4

ES-20 I

Examlndtlon Outline Quality Assurance Checklist

Form ES-201-2

Item

1.

W

R

I

T

F

N

2.

S I

M

'r

-

3.

W

i

.r

4.

(;

c

N

F

R

A

L

.

.__ ....

h. Assess whaherthe outtine was systrmatica6'yand raridumly prepmed in accurdance with Sectim D I o

ES4O 1 and whither all knowledge and ability cirtegrries are approptiate sampkd

c. Assess whcther the ouilitline overmaphasizes my systam, evolutions, m gm!nc tixIks. __ ....

d. Assess hether the rrpetition f n m prrviuu;~ examinatinl outlines is cxccssiw.

repeatedan suhsequent days.

(I)

the outline(s) contais(s) the requirrd number of control rmm anrl in-plant task,

(2)

no I B O ~

than 30% of the tcd material is repeated froin thc hst NRC cxrminatia,

(3)

• no task B L ~

duplicated from the applicants audit testisj, and

(4)

no more than 80% of thc operatine test is takcn dire& froin thc licensee's ? x m bank.

~

.

b. Verifythat:

( I )

(2)

(3)

Ole uak~ are dirttihutcd among the safety hnctim groupings as specified in ES-301,

O ~ C

task is c.onductcd in a low-power or shutduw condition,

4 6 (2 - 3 for SKO-(I) of the task require the applicant to impllemmt an altmate psth prwedmc,

. ,

..

(4)

(5)

c. Verify that the required administmix topics arc cowred.

onc in-phi1 task tests thc applicant's response to M emergency or ahnormil cmditim, and

thc inplant walk-tllro~yh nquircs the applicant to enter the RCA.

d. Detcrminr if there are enourh diffcmt outlitlinrs to test the projrzted niimhw and mix of applicants and

ant.spn:, IC pnon IPS mc ut mg

~

~~~~~~

h. Asscss whether the IO CFR 55.41143 and 55.45 sampling is appqiriate.

c. Ensure that K:A importance ratin@ (cxcept far plant-spccitic priorities) are at least 2.5.

d. Clwk for dlrplicatini and orerkp among rxam SeCtimS.

e. Check the entire exam far hdance of coverage.

f. Assess whether the exam fits the nppropriatejuh level (RO or SRO).

-.

-~

.~

~

~

~

a. h t h M

h. Fxility Reviewe(*)

c. ChiefExarniner

d. NRC Supe~iisa.

.-

NUREG-1021,

Draft Revisioii 9

ES-2Ql

Examination Security Agreement

Form ES-201-3

1.

Pre-Examination

I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of 2-23-04 as of the date of

my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the

NRC chief examiner. I understand that I am not to instruct, evaluate,

these licensing examinations from this date until completion of exami

Furthermore, I am aware of the physical security measures and requireme

violation of the conditions of this agreement may result in cancellation of the e

licensee. I will immediately report to facility management or the NRC chief ex

been compromised.

2.

Post-Examination

To the best of my knowledge, I did not divulge lo any unauthorized persons any inf

during the week(s) of 2-23-04,

From the date that I entered into this security agreement until the completion of examination administration, I did not

instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted

below and authorized by the NRC.

vide performance feedback to those applicants scheduled to be administered

s specifically noted below and authorized by the NRC.

e facility licensee's procedures) and understand that

ti enforcement action against me or the facility

s or suggestions that examination security may have

concerning the NRC licensing examinations administered

PRINTED NAME

JOB TITLE / RESPONSIBILITY

DATE NOTE

. - . -

__

-.

.- -

-

.-

apr r A s T r " z + c

t 3 .

-. -

.-

_ _

25 of 25

NUREG-1021, Draft Revision 9

Examination Security AgreeMent

Form ES-201-3

ES-201

V&y. L * 4- 2

1.

Pre-Examination

I acknowledge that I have acquired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of &3as

of the date of

my signature. I agree that I will not knowingly divulge any information about these examinations to any persons who have not been authorized by the NRC

chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled to be administered these

licensing examinations from this date until completion of examination administration, except as specifically noted below and authorized by the

NRC.Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand

that violation of the conditions of this agreement may result in cancellation of the examinations and/or an enforcement action against me or the facility

licensee. I will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have

been compromised.

2.

Pos t-Examination

To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered

during the week(s) of Z/S$~/!/O~! From the date that I entered into this security agreement until the completion of examination administration, I did not

instruct. evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted

below and authorized by the MRC.

PRINTED NAME

JOB TITLE / RESPONSIBILITY

SIGNATURE (1)

BATE

SIGNATURE (2)

DATE NOTE

I-_

I

-

I_-

.I

14.

15.

I

-

.. - ...

I-

-

NUREG-1021, Revision 9, Draft

--.

p

+.

c

ES-202

Examination Security Agreement

Form 3-201-3

v

I.

Pre-Exam ination

I acknowledge that I have acqirired specialized knowledge about the NRC licensing examinations scheduled for the week(s) of w

a

s

of the date of

my signature. I agree that I will not knowingly divulge any information about these examinatfons to any persons who have not been authorized by the NRC

chief examiner. I understand that I am not to instruct, evaluate, or provide performance feedback to those applicants scheduled :o be administered these

licensing examinations from this date until comp!etion of examination administration. except as specifically noted below and authorized by the

NRC.Furthermore, I am aware of the physical security measures and requirements (as documented in the facility licensee's procedures) and understand

that violation of the conditions of this agreernent may result in cancellation of the examinations andlor an enforcement action against me or the facility

licensee. 1 will immediately report to facility management or the NRC chief examiner any indications or suggestions that examination security may have

been compromised.

2.

Post-Examination

To the best of my knowledge, I did not divulge to any unauthorized persons any information concerning the NRC licensing examinations administered

during the week(§) of

G~3/b/~'f , From the date that I entered into this security agreernent until the completion of examination adminiStratiQR, I did not

instruct, evaluate, or provide performance feedback to those applicants who were administered these licensing examinations, except as specifically noted

below and authorized by the NRC.

PRINTED NAME

JOB TITLE I RESPONSIBiLITY

SIGNATURE (1)

DATE

SIGNATURE (2)

DATE NOTE

1.

2.

3.

4.

5.

NUREG-1 021, Revision 9, Draft

ES-401

Administrative Topics Outline

FORM ES-401-1

Describe Activity to be Performed

(KA # - RO Imp I SRO Imp)

Perform a manual Shutdown Margin Calculation per OST-1036

(2.1.25 - 2.8 / 3.j)

Date of Examination:

Examination Level:

Operating Test Number:

I

Administrative Topic

Conduct of Operations

Conduct of Operations

Equipment Control

Radiation Control

Emergency Plan

Determine Average RCS Boron Concentration per EOP-EPP402

(2.1.20

- 4.3 14.2)

Determine clearance requirements for a CSlP per OPS-NGGC-

1301

(2.2.13 - 3.6 I3.8)

Determine PEDE While Working in a High Airborne Area

(2.3.10 - 2.9 13.3)

NOT APPLICABLE FOR RO

SROs. 80 applicants require only 4 items unless they are retakin

NUREG-2021. Draft Revision 9

ES-301

Administrative Topics Outline

FORM ES-301-1

Examination Level:

SRO

Operating Test Number:

Administrative Topic

(see Note)

Conduct of Operations

Conduct of Operations

I

Emergency Plan

Describe Activity to be Performed

(KA#-ROlmp/SROlmp)

-

Perform a manual Shutdown Margin Calculation per OST-1036

(2.1.25 - 2.8 I3.1)

Determine Average RCS Boron Concentration per EOP-EPP-00;

(2.120 - 4.3 I 4.2)

Determine clearance requirements for a CSlP per OPS-NGGC-

1301

(2.2.13 - 3.6 13.8)

Determine TEDE While Working in a High Airborne Area

(2.3.10 ~ 2.9 13.3)

Determine Protective Action Recommendations per PEP-f 20

(2.4.44 - NA I4.0)

SROs. RO applicants require only 4 items unless they are retakin

only the administrative topics, when 5 are required.

NUREG-2021. Draft Revision 9

FORM ES-301-2

ES-301

Control Room I In-Plant Systems Outline

Examination bevel:

RO

Operating Test Number:

/

Control Room Systems (8 for 80; 7 for SRO-I; 2 or 3 for SRO-U)

SystemIJPM Title

Safety

Function

Code*

I

Type

I (KA # - RO Imp)

a.

(062A4.07 - 3.1)

Transfer 6.9KV Buses from Unit Auxiliary Transformers to the

Startup Auxiliary Transformers per GP-156.02

1010

(02284.01 - 3.6)

3

I (0000T 1 EA1 .I3 - 4.1

isolate the SI Accumulators following a LOCA per PATH-1 and

GP-I 10

I A I I

I

C.

I

M l S l L 4s

(061A3.01 - 4.2)

d.

Start the Turbine Driven Auxiliary Feedwater Pump per OP-I37

2

(006A4.05 - 3.9)

M I A I S 1 L

I

e.

Transfer to Cold Leg Recirculation per EOP-EPP-010

f.

Perform Control Rod and Rod Position Indicator Exercise per

OST-1005

Place Audio Count Rate Drawer in Service per OP-I05

h.

Align CCW to Support RHR System Operations per GP-145

In-Plant Systems (3 for RO; 3 for SRO-I; 3 or 2 for SRO-U)

1

(001A2.11 - 4.4)

I

D i A l S

7

(02584.02 - 3.9)

8

(Q08A4.01 - 3.3)

I

D l L l S

1

(004A2.07 - 3.4)

Local Makeup to the VCT Using the Emergency Boration Valve

012A4.04 - 3.3

j'

Perform local actions for placing an OTAT channel in Test per

OWP-RP-OI

Locally operate a SG PORV per EOP-EPP-012 and OP-126

(000074EA1.04 - 3.9

• Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)lternate path, (C)ontrol Room, (S)imulator, (L)o\\

Power, (R)CA

NUREG-1021. DraR Revision 9

FORM E§-301-2

E§-301

Control Room / In-Plant Systems Butline

nAmis

Date of Examination:

SRO-U

Operating Test Number:

/

-

Control Room Systems (8 for RQ; 7 for SRO-I; 2 or 3 for SRO-U)

SystemlJPM Title

062A4.07 - 3.1

NOT APPLICABLE FOR SRO-U

OP-l I 0

d.

NOT APPLICABLE FOR SRB-U

NOT APPLICABLE FOR SRO-U

NOT APPLICABLE FOR SRO-U

NOT APPLICABLE FQR SRO-U

h.

NOT APPLICABLE FOR SRO-U

In-Plant Systems (3 for RO; 3 for SRO-I; 3 or 2 for SRO-U)

1

(004A2.07 - 3.7)

I.

Local Makeup to the VCT Using the Emergency Boration Valve

(012A4.04 - 3.3)

.

Perform local actions for placing an QTAT channel in Test per

OWP-RP-01

4 s

k.

I

"13

I (000074EA1.04-4.1:

Locally operate a SG PORV per EOP-EPP-012 and OP-126

• Type Codes: (D)irect from bank, (M)odihed from bank. (N)ew, @)Iternate path, (6)ontrol Room, (S)imulator, (L)o

Power, (I3)CA

NUREG-1021. Draft Revision 9

-I.s.

, .. . . ..

-.

I_.._

. ..I ....

s

-

. . ... . .

-.

ES-301

Operating Test Quality Asurwck Checklist

F

o

~

ES-30 1-3

.. I .

, ,

.... .

-- . . . , , ,

... ,,--

-t-

23-27 l*r

NUREG-1021, Revisicm 9 Draft

~~~~

Simulator Scenario Quality Assurance Checklist

Form ES-301-4

ES-30 I

Facility: IIARWIS

Date of Exam:

2/23- 2716 3

Operating Test No.:

QUALITATIVE ATTRIBUTES

I.

'The initial conditions are realistic. in that some equipment andior instrumentation may be

out of servic.e, but it does not cue the operators into expected events.

'The scenarios consist niostly of related events.

Each event description consists of

e

2 .

3.

the point in the scenario when it is to be initiated

the malfunction(s) that are entered to initiate the event

the symptonisicues that will be visible to the crew

the expected operator actions (by shitt position)

the event termination point (if applicable)

e

e

4.

No more than one non-mechanistic failure (e.&, pipe break) is incorporated into the

scenario without a credihle preceding incident such as a seismic event.

The events are valid with regard to physics and thermodyiamics.

5 ,

6.

Sequencing and timing of events is reasonable, and allows the examination team to

obtain comvlete evaluation results commensurate uzith the scenario iihiectives.

..

7.

If time compression techniques are used, the scenario summary clearly so indicates

Operators have sufficient time to carry out expec.ted activities without undue time

constraints. Cues are given.

8.

The simulator modeling is not altered

9

The scenarios hwc been validated. Pursuant to IOCFK55.46(d);any open simulator

perfcxmance deficiencies have been eviiluated to ensure that functional fidelity is

inaintained while running the planned scenarios.

IO. Every operator will be evaluated using at lcast one new or significantly modified

scenario. All other scenarios have been altered in accordance with Section D.5 of ES.

I I . All individual operator compctencies can he evaluated, as verified nsing Form I S 3 0 1-6

(submit the form along with the simulator scenarios).

12. Each applicant will he significantly involved in the minimum number of transients and

events specified on Form ES-301-5 (submit the fomi with the simulator scenarios).

13. The level of difficulty is appropriate to support kensing decisions for each crew

oosition.

~~

~

~~~

~.

~~

Initials

ES-301

Simulator Scenario Quality Assurance Checklist

Form ES-30 1-4

TARGET QUANTITATIVE ATTRIBUTES (PER SCENARIO;

~

~

~~~

~~~~~

~

ES-301

Transient and Event Checklist

Form E§-301-5

HARRIS

OPERATING TEST NO.:

p- DA

I

SRO-I

I

Instructions: (1)

(2)

Enter the operating test number and Form ES-D-1 ewnt numbers for

each evolution type.

Reactivity manipulations may be conducted under normal or controMed

abnormal conditions (refer to Section D.5.d) but must be signilcant per

Section C.2.a of Appendix D. * Reactivity and normal evolutions may

be replaced with additional instrument or component malfunctions on a

one-for-one basis.

Whenever practical, both instrument and component malhnctions

should be included; only those that require verifiable actions that

provide insight to the applicant's competence count toward the

minimum requirwent.

,

(3)

Author:

NRC Reviewer:

NUREG-1021, Draft Revision 9

ES-301

Competencies Checklist

Form ES-302-6

SRO

RO

BOP

SCENARIO

T

\\RIB

3

_s

SCENARIO

Competencies

SCE

2

_.

-

4

SP

1 3

5

-

-

1 3

4 5

-

1 3

4 5

-

ALL -

NIA

-

NJA -

-

4

SP

2 2

3 4

5

Interpret I Diagnose

Events

and Conditions

2 3

4 5

6 7

12 3

4 5

6

1 2

3 5

6 7

8

ALL

2 4

6 7

2 4

5 6

9

Comply With and

Use Procedures (1)

ALL

ALL

ALL

Operate Control

Boards (2)

NIA

NIA

NIA

NIA

2 4

5 6

7

ALL -

ALL

ALL -

ALL

ALL

~

ALL

__

2 3

5

__

ALL

__

ALL

ALL

Communicate and

Interact

,emonstrate

Supervisory

Ability (3)

Comply With and

Use Tech. Specs.. (3)

Notes:

NIA

2 3

-3-

3 6

3

_.

NIA

__

(1) includes Technical Specification compliance for an RO.

(2) Optional for an SRO-U.

(3) Onlv amlicabie to SRBs.

Instructions:

Circle the applicant's license tfle and enter one or more emnt numbers that will allow the examiners to

evaluate every applicable competency fpr every applicant.

Author:

NRC Reviewer:

\\

NUREG-1021, Brae Revision 9

I

Written Examination

Form ES-401-6

ES-401

Quality Checklist

Bate of Exam:

Exam Level: RO I SI30

5.

Question duplication from the license screening/audit exam was controlled as

Bank use meets limits (no more than 75

7.

Between 50 and 60 percent of the questions on

the RO exam are written at the

comprehension/analysis level; the SRO exam

may exceed 60 percent if the randomly

selected K/As support the higher cognitive

Printed Name / Signature:,-

Date

dd.l

a. Author

b. Facility Reviewer(')

e. NRC Chief Examiner(*)

1. NRC Regicnal Supervisor(^)

ES-401

Written Examination

Form ES-401-9

Harris 2004 Review Worksheet

I

1. 1 2. I

3. Psvchometric Flaws

1 4. Job Content Flaws

1 5. Other

1 6. 1

7.

Explanation

I

I

lunitsl ward I WA I Only 1

Instructions

[Refer to Section B of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

2.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level

Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are aweptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated hue/false statements.

One or more distractors is not credible.

One or more distractors is (are) partially correct (e.$. if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e.. the question has a valid K/A but, as written, is not operational in content).

The auestion rewires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from

iemory).

ThJ qLestioii contains data with art uirea 1slic level of accuracy or incoriststent Liiits (s.g panel iiicter it7 percent with question in gallons)

The question reqdtres reverse logic oi application mnlpared to the ]oh reqiiirerrrel1:s.

5.

Check questions that are sampled for conformance with the approved WA and those that are desiqnated SRO-only (KIA and license level mismatches

are unacceptable).

6.

Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacerxent), in need of (E)ditorial enhancement, or

(S)atisfactory?

7.

At a minimum, explain any " U ratings (e.g., how the Appendix B psychometrii attributes are not being met).

Explanation

ROISRO Coi -

hestion

maintain plant temp.

(MOD)

Althou h the question was modified. it retains enough of thf

origina? characteristics that the original question can be

recognized. The correct answer vosition was not changed

for the modified question.

Modified Question to maintain a diffeFeflt temperature

and changed the position of the correct answer. (MOD)

Question Acceptable after modifications.

U

02262.4.4 Distractors C and D also correct On (Harris

2000 Exam1

IThe correct answer can easily be picked by the information

r,y,u .., ..._~._

I...

To elicit the correct answer without allowing no other correc

answer. the stem must include in accordance wlth Alarm

Panel Procedure ALB-06-7-3 and ALE 06-8-4 because the

AQP is the first procedure referred to.

Question to be Replaced

Replaced Question.

Question accedable after reDlacement

S

013K3.03 (MOD)

2. I

3. Psychometric Flaws

.OD

1-5) -

2

-

Stem

Focus

~ -

4. Job Content Flaws

5. Other

7.

ion was modified, it retains enough of the

'cs that the original question can be

rrect answer position was not changed

stion. Change the position of the correct

Change the word "senses ..." in the correct answer. SuggeS

- "indicates an all rods in condition".

Made Changes as requested.

Question acceotable after dranws.

U 'YE14EA1.2 First section of stem not requiied (Teaching :n

stem) On -002.Exa.a

Question to be Replaced

Ouesfian Replaced.

Distractor B is not eredible. (NEW)

Distractor reworked question acceptable. .

E 1054AK1.02 (NEW)

Change the question to ask "which of the following describer

the strategy used to re-establish feed under these

conditions.

Made Changes as requested

I (NEW)

U

003A4.01 KIA is to mariba!ly opfm!e or monitor in !he contrr

room: seal injection. This :s adually monitoring charging

flow. Backwards logic (MOD)

Question does not match WA

Replaced Question. (Direct).

3. Psychometric Flaws

.Job

Content Flaws I

5. Other

unit

Explanation

U

013K6.01* Question very simple On bast Harris NRC

Exam2002 to he Replaced

Replaced Quest/on.(Oirect)

Question acceatable.

001 K4.07 Colon required after Given the following condition:

(MOD)

S

IChange the order of the choices and the correct answer,

Made changes as requested.

Question Acceptable.

Replaeed Question (MOD)

Question Repplaced

4. Job Content Flaws

5. Other I 6. I

7

Explanation

Removed Portion of stem that was teaching. Changed

Added /A W A OP--020.

Question Acceptable

S 056AA2.09 (MOD)

U 005KAK2.02 Does not match WA. Distraciors A and D may

not be credible. (MOD)

I I

Replaced Quesfion.(NEW)

I

I I

Question Acceptable

EA2.01 Question could be asked without the top billlet in

stem and would be the same. Biswnnect betwen stem

bullets add conditions to fit C.?. (MOB)

emoved top bullet from stem of question

nd WA mismatch WA and Question do not

match (NEW) Is it fair to assume that the other parameters

Replaced Questfon. (NEW

I

1.

-OK

FIH) -

H

-

H

-

F

H

-

F

-

4. Job Content Flaws I 5. Other I 6. I

9.

.ink

Explanation

No Chanaes made to awestion (MOD)

Does not meet KIA, hank question with a drawing

that is a direct lookup. (DIRECT).

Replaced Question. (Mew)

Qwestion Reworked.

uestion written. does not reallv match KIA

Reworked Question accwtable.

064K2.03 Distractors A and B not credible, very

wnvoluted.(MOD)

Requires a level of knowledge kyond 1 hr T/S. Beyond

Scope for RO.

I

I

1 Replaced Question (NEVU)

New question acceptable

025K5.10 Need to add IAW GOP-005 to stem

E

Stem should state that indications converted to per cent

where required. (MOD)

C distractor not plausible,

Replaced C distractor. Made other changes as

r@QOeSted.

I

I New auestion acceptable

4. Job Content Fiaws

5. Other

6.

UEi

S

7.

Explanation

have more than one correct answer if

is not included in stem IAW ..

Made changes as requested.

New uestion acceptable

Discuss using the whole procedure if it does not answer othc

uestions. This will better match WA Obtain and interpret.

&awing provided needs to be reprinted to avoid confusion

with Normal Operations and Abnormal Operations Areas.

Replaced Question. (N EW)

Distractor A analysis does not fit distractor

Changed distractor analysis.

is and the answer would be the same: Very similar question

to RO # I No calcuiation is really required. (MOB)

Replaced Question. (MOD)

New question accwtablc

4. Job Content Raws I 5. Other

linutia

7.

Explanation

Changed disfracfors. Still very simple

Question acceetable

II

LJ 037~~2.10

on Harris NRC ~xaaa.2aoa (MOD)

Backwards Logic

Question was not changed, due to comments from licensee.

Seemed exceptable.

U

VVE15EK1.2 Distractors A and C may not be plausibe.

(DIRECT)

Replaced Distractors

U

004A1.06 Question should ask what is CVCS response.

Question to be Replaced

On Last Harris NRC Exam2002

Replaced Question (NEW,

ometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Explanation

U

035K1.01 Stern is somewhat wnfusin does the AFW

isolation come from mnditions on B Sk.

Does this also give

a Feed Water isolation (MOD)

TOO close to #45.

Replaced Question. (New)

New question acceptable

U WE05EK2.2 Question to be Replaced

On NRC Exam 2000

I I

Replaced Question.

May have more than one correct answer. (Direct)

Reworked question

Question acceptable

S

62.2.1 need to rearrange the answers so that they are

different than the bank question. (MOB)

In stein Six hours has (8) associated with it.

Made chanaes as requested.

1

K/A match (MOD).

Replaced QuestionpEW)

Changed question # 41

Question acceptable

3. Psychometric Flaws

Explanation

E

061K6.01 ( requires a statement with NO OPERATOR

I

I

I

I

I

I

i

ACTION (MOD)

Repkaced Question.(Direct)

New Question acceotable

controls. Needs to be c t 1 a n g e d . m

WA mismatch.

Made changes to $uesfion(.Newj

Question acceDtable

062A4.04 Change answer and distractors to indicate local

operation of breaker. (DIRECT).

Made changes as requested.

3. Psychometric Flaws

4.

Job Content Flaws I 5. Other

)b- Minutia ,I

Explanation

U 010K1.06 Question to be Replaced

On Last Harris NRC Exam.2002

I Replaced Question.

Distractor D is also correct. Need to refine stem or

distractor.

I

Reworked question

Made changes to sfem.(NEW)

Question accwtable

S

015A2.05 (DIRECT).

S 1045K4.06 (MOD)

E T-

No Changes made.(MOD) Question acceptable

064.43.07 is loading of the sequencers different between a

loss of power and a Safety injection signal (MOD)

U

G2.2.24

Question On Last Harris NWC Exam2002

to be Replaced

I

I Replaced question (MOD)

New auestion accewtabie

3. Psychometric Flaws

4. Job Content Flaws

5. Other

Explanation

U

G2.4.3 Question 01, Last Harris NRC Exam.2000

Change the order of the distractors and nswer.

Change the stem and 700 psig to 690 psig This will be the

correct answer.

Question to be replaced

Replaced Question (MOB)

B not credible. (MOD)

Repfaced Question (NEW

Removed reference page.(MOD)

Question acceptable.

U 038EA1.36 IAW what procedure, Distractor C and D is not

plausible (DIRECT).

Modified Question (MOD)

Question acceptable

U

073K5.02 Does not meet WA. (NEW) Job link

similar to# 14.

Changed stem to address job hk.(NEW)

I Reworked question.

Question Acceptable

2 1

3 Psyctiometnc Flaws

I 4 Job Content Flaws

Focus

I

i

-

TIF

I

1. /

Back.

nits I

ward

5. Other

7.

Explanation

s I

E

012K2.01 Question to be Replaced

2000 Harris NRC Exam.

NEW Question Acceptable

different, or include in the stem IAW procedure ...( NEW)

License checked to see if any other precautions and

limitations were applicable. Appears that none are.

Question Acceptable

068A3.02 Distractors B and C does not appear to be

cfedibls. (MOD)

U

Distrnctors reworked

U

G2.4.1 Why is :he statement "Containfnq NO Immediate

Operator Articns" iiicludecl in tho stem this makes thcr

question a ncgitive questiori (NEW)

Replaced Uuect;on. (NEW)

3 Psychometric Flaws

4 Job Content Flaws

5 Other

6

7

Explanation

Question Replaced (Direct)

Removed teaching in stem and reworked distractors.(NEW)

eplaced Question.(Direct)

SRO ONLY

Utility changed answers to contain a selection of correci

Question Acceptable

Uti/ity addedprocedure to stem and referenced TIS and

I

1

/

1

1

I

/

its bases.

Question Acceotable

3. Psychometric Flaws

4. Job Content Flaws

5. Other

7

Utility replaced with a modified Harris bank questjon,

appears to address WA

E

ClltG2 4 6 tmgh qJCC.l:On. ATWS needs to mentioiled in

s!em, nr scrar:i attempted horn mntrol room etc !NEW)

I Utility Added In accordance with FRP-Sd in the stem.

Question Acceptable

il

G2.1.2. Not SRO only knowledge (DIRECT)

Utility requested new K/A.

NRC Replaced W A withG2.1.7

Utillty replaced question. Matches K/A

New Question Acceptable

01 1G2 4 6 Toach.ng in stem (Heat s.nk condit o m anc

answers (1 arge hrem LOCAI Not SRO only (MOIh

Utility modified question to eliminate "teaching" and

added a decision point to make SRO only.

I

1.

.OK

=/H)

-

-

H

H

-

F

3. Psyrhornetric Flaws

4. Job Content Flaws

linutia

-

-

5. Other

7.

II

I1

New Question Acceptable

I

U :01262.4.6 Dislracrors C and 3 not credib:e S.

comut he

j reset for a cerlalo perloo of tinia. NetJrn to S.1 orily wnen

direfled by path 1 no1 credible Not SHO only

jawstion Qn..H.a.rris NRC Exam=

lo he Repleced

'

Utility replaced question with Hamis Bank question EPP-

3.15-R511 meets WA. (Direct)

Editorial changes to question to create a condition wher

if is possible to loose power to just LlRPi, changed 43(b)

mn-.pemaiid Do.s?ion indicators in the distractors

I makes the ouestion much e.as!e!.

II

Question Acceptable

2. I

3. Psychometric Flaws

4. Job Content Flaws

linutia -

-

5. Other

tied to 43. f will send reference.

Utifity sent SRQ only objebjive to verify 43 tie. Also

modified the question to pertain to the SRQ position.

Question AcceDtable

?I

038EA2.08 Two possible answers B and D. Question @I

Last Harris NRC Exam2002 to be Replaced

Question replaced with new question.

Determine if Preferred procedure is recwired to be in the

I- stem.(NEW)

Question modified amd is acceptable.

WE14EA2 7 d n not ncldde act ons nf EPP-12 tu veiily that

t v a ctm: spray [~:riiips not reqJireJ SA1 (MOD)

Revised answer b per EPP-Of 2; Provided reference to

verify that only 1 CSP is required.

U

013A2.01 Question daes not meet WA Not SRO only

knowledge. On Harris NRC Exam2000 to be Replaced

eplaced with New question. (NEW)

G2.2.19 Not SRO Only knowledge. Question On Harris NR(

to be Replaced

U

Exam.2000

Replaced with Direct Harris Bank Question BP-2~4S6 2

(Direct) Remove date in stem if not reauired

3. Psychometric Flaws

4. Job Content Flaws

5. Other

7.

Replaced with new question

-.

Stem

. . . . . and answe.ers..vot in svnc,.Af!emut to close which

MSlV prior to isolatinq the.other SIGS:

I Reworked Question

Question Acceptable

S

WE0662.1.32 State the procedure that the crew has enterei

l(NEW)

cafion ofpracedure in stem per

Made changes as requested.

Question Acceptable

G2.2.10 Does not match WA. (DIRECT)

U

Replaced with new question

Watches K/A (DIRECT) from INPO bank

Question Acceptable

Backward logic. (NEW

Repraced with new question

Question Acceptable

3. Psychometric Flaws

4. Job Content Flaws

-

dinutia

=

5. Other

5.

7.

/El

S

Explanation

In accordance with EFF-006 D Is not correct. (Additional

oesno mac

Utility requested new WA.

KIA replaced wlth 2.3.10

Replaced with modified Harris LOCT Exam Bank

Matches WA need to see bank suestion to ensure

auestion le modified

Gkified to place plant in a certain condition,revised

stem and distractors io eliminate conc@rn for more than

I correct answer.

Question AccfDtable

Utility requested new MA.

New WA randomly selected 058G 2.1.33

Utirity replaced question with new question for new MA.

Explanation

I

1 U 1G2.4.14 (MOB) Negative question ask in positive form.

II

Reworked in to positive question. revised stem and

distracfors to remove backwards logic.

l l

I

IQuestion Acceptable

II

ES-404

Written Examination Grading

Form ES-403-1

Quality Checklist

Item Description

I .

2.

3.

4.

Clean answer sheets copied before grading

Answer key changes and question deletions justiiied and

documented

Applicants' scores checked for addition errors

(reviewers spot check 25% of examinations)

Grading for all borderline cases (80 +I- 2% overall and 70 +/-

4% on the SRO-only) reviewed in detail

All other failing examinations checked to ensure that grades

are justified

Performance on missed questions checked for training

deficiencies and wording problems; evaluate validity of

Printed Name I Signature

5.

6.

xam Level: ROISRQ

Initials

L

Date

r examinations graded by the

5 of 5

NUREG-1021. Draft Revision 9

ES-501

Post-Examination Check Sheet

Form ES-501-1

Task Description

II

I.

Facility written exam comments or graded exams received and

verified comdete

2.

Facility written exam comments reviewed and incorporated and

NRC grading completed, if necessary

11 3.

Operating tests graded by NRC examiners

NRC Chief examiner review of written exam and operating test

grading completed

11 5.

Responsible supervisor review completed

License and denial letters mailed

II 9.

Examination report issued (refer to NRC MC 86380(312)

Date

Complete

NUREG-1021, Draft Revision 9

24 of 23