ML022690212

Operating Test Number:

b

"Describe method of evaluation:

"Administrative

1. ONE Administrative JPM, OR

Topic/Subject

2. TWO Administrative Questions

Description

(KA #)

1

Determine Rod Misalignment Using Thermocouples (AOP-001)

CONDUCT OF

OPERATIONS

(2.1.19)

Perform a Manual Power Range Heat Balance Calculation

(OST-1204)

(2.1.25)

"A.

Review an Equipment Clearance (OPS-NGGC-1 301)

EQUIPMENT

CONTROL

(2.2.13)

"A.

3

P~tLcsdO6aO Aclons to Cs!ta~bzh-a-bqutd'Viate

RADIATION

CONTROL

(2.311)

?

"A.

4

Activate the Emergency Response Organization - Dialogic

EMERGENCY

System (PEP-31 0)

PLAN

(2.4.43)

NUREG-1021, Revision 8, Supplement 1

Administrative Topics Outline

FORM ES-301-1

Facility:

HARRIS

Date of Examination:

26-Aug-02

Examination Level:

SRO

Operating Test Number:

"Dlescribe method of evaluation:

"Administrative

1. ONE Administrative JPM, OR

Topic/Subject

2. TWO Administrative Questions

Description

(KA #)

"-A.

1

Perform Review of Daily Surveillance Requirements Log

CONDUCT OF

(OST-1021)

OPERATIONS

(2.1.18)

Perform a Manual Power Range Heat Balance Calculation

(OST-1204)

(2.1.25)

"A.

Review an Equipment Clearance (OPS-NGGC-1 301)

EQUIPMENT

CONTROL

(2.2.13)

"A3

Question Topic - License Requirements for Conducting a Waste

Release with Inoperable Instrumentation and Administrative

RADIATION

Controls Ensuring Requirements Met (2.3.6)

CONTROL

Question Topic - Selection Process for Individuals Performing

Emergency Entries into Radiation Fields Resulting in Exceeding

Permissible Exposure Limits (2.3.4)

"A.

4

Perform an Emergency Action Level Classification and

EMERGENCY

Recommend Protective Actions (PEP-1 10)

PLAN

(2.4.41 / 2.4.44)

NUREG-1021, Revision 8, Supplement 1

Control Room Systems and Facility Walk-Through Test Outline

FORM ES-301-2

Facility:

HARRIS

Date of Examination:

26-Aug-02

Examination Level:

Operating Test Number:

2002-301

B.1

Control Room Systems

Safety

System/JPM Title

Code*

Function

(KA #)

a.

NAS3

Respond to a Failed High Pressurizer Pressure Channel (AOP-019)

(010.A2.03)

b.

NAS6

LOOP While Paralleling EDG from MCB for Testing (OP-1 55)

(064A4.01)

c.DS

8

Secure One Train of CCW to the RHR HXs (OP-145)

(008A2.01)

d.

d.DASL

5

Manually Align Containment Spray (PATH-i)

(064.1

(026A4.0i)

e.

e.DASL

2

Transfer to Hot Leg Recirculation (EPP-011) i)64.2

(006A4.05)

f.

f'DSL

4P

Start an RCP Following Maintenance (OP-100)

(003A4.06)

0.

NS

7

Power Range NI Gain Adjustment (OP-1 05)

(

75A4.02)

B.2

Facility Walk-Through

a.

a.DL

1

Local Actions for a Dropped Rod Recovery (AOP-001)

(03AA1 .02)

b.

Manually Align Charging Due to a Loss of IA (AOP-017)

DRL

2

(004A2 .11)

c.

c.DL

5

Start Up a Hydrogen Recombiner (OP-125)

(028A4.Di)

Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)lternate path, (C)ontrol Room, (S)imulator,

(L)ow-Power, (R)CA

NUREG-1021, Revision 8, Supplement 1

Control Room Systems and Facility Walk-Through Test Outline

FORM ES-301-2

Facility:

HARRIS

Date of Examination:

26-Aug-02

Examination Level:

SRO-I

Operating Test Number:

2002-301

B.1

Control Room Systems

Safety

System/JPM Title

Code*

Function

(KA #)

a.

Respond to a Failed High Pressurizer Pressure Channel (AOP-019)

NAS(

3

(010 .A2 .03)

b.

NAS6

LOOP While Paralleling EDG from MCB for Testing (OP-1 55)

(064A4.01)

C.

DS8

Secure One Train of CCW to the RHR HXs (OP-145)

(008A2.01)

d.

Manually Align Containment Spray (PATH-i)

DASL

5

(026A4.01)

e.

e.DASL

2

Transfer to Hot Leg Recirculation (EPP-011)

(0 6 4.5

(006A4. 05)

f.

f'DSL

4P

Start an RCP Following Maintenance (OP-100)

(003A4.06)

g.

NS

7

Power Range N I Gain Adjustment (OP-1 05)

(01 5A4.02)

B.2

Facility Walk-Through

a.

DL

1

Local Actions for a Dropped Rod Recovery (AOP-001)

(003AA1.02)

b.

b.DRL

2

Manually Align Charging Due to a Loss of IA (AOP-017)

(004A2.i1)

c.ADL

5

Start Up a Hydrogen Recombiner (OP-125)

(028A4.DL)

Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)lternate path, (C)ontrol Room, (S)imulator,

(L)ow-Power, (R)CA

NUREG-1021, Revision 8, Supplement 1

Control Room Systems and Facility Walk-Through Test Outline

FORM ES-301-2

Facility:

HARRIS

Date of Examination:

26-Aug-02

Examination Level:

SRO-U

Operating Test Number:

2002-301

B.1

Control Room Systems

Safety

System/JPM Title

Code*

Function

(KA#)

a.

Respond to a Failed High Pressurizer Pressure Channel (AOP-019)

NAS(

3

b.

NAS

6

LOOP While Paralleling EDG from MCB for Testing (OP-1 55)

(064A4.01)

C.

d.

e.

f.

g.

B.2

Facility Walk-Through

a.

a.DL

1

Local Actions for a Dropped Rod Recovery (AOP-001)

(003AA1.02)

b.

Manually Align Charging Due to a Loss of IA (AOP-017)

DRL

2

c.ADL

5

Start Up a Hydrogen Recombiner (OP-125)

(028A4.01)

Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)lternate path, (C)ontrol Room, (S)imulator,

(L)ow-Power, (R)CA

NUREG-1021, Revision 8, Supplement 1

Operating Test Quality Assurance Checklist

Form ES-301-3

Facility: Harris

Date of Examination:

26 August 2002

Operating Test Number:

1. GENERAL CRITERIA

Initials

a

b*

c#

a. The operating test conforms with the previously approved outline; changes are consistent with sampling

requirements (e.g., 10 CFR 55.45, operational importance, safety function distribution).

er

A

R

b. There is no day-to-day repetition between this and other operating tests to be administered during this examination.

tk

kv Q,.

c. The operating test shall not duplicate items from the applicants' audit test(s)(see Section D. La).

d. Overlap with the written examination and between operating test categories is within acceptable limits.

")

e. It appears that the operating test will differentiate between competent and less-than-competent applicants at the

designated license level.

2. WALK-THROUGH (CATEGORY A & B) CRITERIA

a. Each JPM includes the following, as applicable:

-

initial conditions

-

initiating cues

-

references and tools, including associated procedure

-

reasonable and validated time limits (average time allowed for completion) and specific designation if deemed

to be time critical by the facility licensee

- specific performance criteria that include:

-

detailed expected actions with exact criteria and nomenclature

-

system response and other examiner cues

-

statements describing important observations to be made by the applicant

-

criteria for successful completion of the task

-

identification of critical steps and their associated performance standards restrictions on the

sequence

b. The prescripted questions in Category A are predominantly open reference and meet the criteria in Attachment I of

J - n t

c. Repetition from operating tests used during the previous licensing examination is within acceptable limits (30% tokro

1nd

e

the walk-through) and do not compromise test integrity.

d. At least 20 percent of the JPMs on each test are new or significantly modified.

3. SIMULATOR (CATEGORY C) CRITERIA

a. The associated simulator operating tests (scenario sets) have been reviewed in accordance with Form ES-301-4 and

a copy is attached.

Printed Name / Signature

Date

a. Author

J

3c wA...

b. Facility Reviewer(-)

-7/1l/t1 2

c. NRC Chief Examiner(#)

-

'02.

d. NRC Supervisor

,.

.,Z.

NOTE:

The facility signature is not applicable for NRC-developed tests.

Independent NRC reviewer initial items in column 'c; chief examiner concurrence required.

NUREG-1021, Revision 8, Supplement I

Harris - August 2002

Simulator Scenario Quality Assurance Checklist

Form ES-301-4

Facility: Harris

Date of Exam: 26 August 2002

Operating Test No.:

QUALITATIVE ATTRIBUTES

Initials

a

b*

c#

I.

The initial conditions are realistic, in that some equipment and/or instrumentation may be

out of service, but it does not cue the operators into expected events.

vy

ll

D

4P

2.

The scenarios consist mostly of related events.

'IE7

3.

Each event description consists of

"* the point in the scenario when it is to be initiated

"* the malfunction(s) that are entered to initiate the event

,

"* the symptoms/cues that will be visible to the crew

"6

"* the expected operator actions (by shift position)

"* the event termination point (if applicable)

4.

No more than one non-mechanistic failure (e.g., pipe break) is incorporated into the

f

At- Q

scenario without a credible preceding incident such as a seismic event.

"d

5. The events are valid with regard to physics and thermodynamics.

£$r

AT- 6

6.

Sequencing and timing of events is reasonable, and allows the examination team to obtain

complete evaluation results commensurate with the scenario objectives.

LA*

riu-*

7.

If time compression techniques are used, the scenario summary clearly so indicates.

Operators have sufficient time to carry out expected activities without undue time

constraints. Cues are given.

8.

The simulator modeling is not altered.

0

9.

The scenarios have been validated. Any open simulator performance deficiencies have

been evaluated to ensure that functional fidelity is maintained while running the planned"6

A,4$ 6

scenarios.

10. Every operator will be evaluated using at least one new or significantly modified scenario.

,t

i

All other scenarios have been altered in accordance with Section D.4 ofES-301.

Yf

It'6

I.

All individual operator competencies can be evaluated, as verified using Form ES-301-6

(submit the form along with the simulator scenarios).

(P-

_

0

12. Each applicant will be significantly involved in the minimum number of transients and

events specified on Form ES-301-5 (submit the form with the simulator scenarios).

A "

13. The level of difficulty is appropriate to support licensing decisions for each crew position.

9

4

-@

TARGET QUANTITATIVE ATTRIBUTES

Actual

(PER SCENARIO; SEE SECTION D.4.D)

Attributes

...

1.

Total malfunctions (5-8)

7/6

Qg

__

_

2.

Malfunctions after EOP entry (1-2)

2/1

0

3.

Abnormal events (2-4)

4/4

AL

4.

Major transients (1-2)

2/1

t

5.

EOPs entered/requiring substantive actions (1-2)

2/3

6.

EOP contingencies requiring substantive actions (0-2)

1/ I

7.

Critical tasks (2-3)

2/2

NUREG-1021, Revision 8, Supplement I

Harris - August 2002

Transient and Event Checklist

Form ES-301-5

OPERATING TEST NO.: 2002-301

Applicant

Evolution

Minimum

Scenario Number / Candidate / Position

Type

Number

SCENARIO #1

SCENARIO #3

RO-1

ypI

i

(RO)

(BOP)

Ii

Reactivity

I

1

Normal

1

5

Component

v*

Major

I

5-6

6-7

SRO-11

,SRO-12

SRO-

, SRO-12

(SRO

'

RO)

(RO)

',(SRO)

Reactivity

Normal

As RO

Instrument /

Component

Major

SRO-I

Reactivity

0

Normal

I

SRO-U

Instrument /

2

Component

Major

I

HARRIS NUCLEAR PLANT - AUG 2002

0

2

0

2

3-4

5-6

2-3-4

5-6

SRO-U I

(SRO)

2-3-4

1-3

6-7

5

I-2-3-4

6-7

6-76-7

SRO-U2

(SRO)____

5

1-2-3-4

5-6

j

6-7

NULREG-1021, Revision 8, Supplement I

As SRO

Normal

Instrument /

Component

Major

5

I

Instructions:

Notes:

(1)

Enter the operating test number and Form ES-D- 1 event numbers for each

evolution type.

(2)

Reactivity manipulations may be conducted under normal or controlled abnormal

conditions (refer to Section D.4.d) but must be significant per Section C.2.a of

Appendix D.

(3)

Whenever practical, both instrument and component malfunctions should be

included; only those that require verifiable actions that provide insight to the

applicant's competence count toward the minimum requirement.

(1)

Only those events prior to the major event are counted in the Transients and

Events.

(2)

The simulator crew composition is as follows:

stion

Scenario #1

Scenario #3

Exam 1

Exam 2

Exam

I

Exam 2

USCO

SRO-I1 I

SRO-Ul

SRO-12

SRO-U2

RO-1

SRO-2

SURROGATE

SRO-I1

BOP

SURROGATE

RO-l

SURROGATE

Simulator exams will be split over 2 days.

Scenario #1 will be used one day, Scenario #3 the other.

Each RO and SRO-I candidate will get one Scenario each day.

The SRO-U candidates will get one scenario.

Surrogates will be used to fill 1 position in each of the scenarios.

Author:

NRC Reviewer:

At'4 9dt0- /Pawz

J *C05.5

HARRIS NUCLEAR PLANT - AUG 2002

(3)

(4)

(5)

(6)

(7)

NUREG- 1021, Revision 8, Supplement I

Form ES-301-6

OPERATING TEST NO.:

RO-1

RO-2

SRO-I1

SRO-12

SRO-13

SRO-

SRO

UI

U2

SCENARIO

1

2

1

2

1

2

1

2

1

2

POSITION

Competencies

3-4-5-

2-3-5

1-2-3-

ALL

1-2-3-

ALL

1-2-3-

3-4-5-

1-2-3-

1-2-3

Understand and Interpret

6

5-6

5

6

5

Annunciators and Alarms

3-4-5-

2-3-5

1-2-3-

ALL

1-2-3-

ALL

1-2-3-

3-4-5-

1-2-3-

1-2-3

Diagnose Events

6

5-6

5

6

5

and Conditions

2-3-5-

2-4-5

2-3-5-

1-2-4-

2-3-5-

1-2-4-

2-3-5-

1-2-4-

2-3-5-

1-2-4-

1-2-4

UnderstandPlant

6

5

6

5

6

5

6

5

and System Response

CoplrWth and

2-3-4-

1-2-3-

ALL

ALL-

2-3-4-

ALL

Copl Wth ad56

5

5-56

Use Procedures (1)

6

5

6

5

2-3-4-

1-2-3-

ALL

2-3-4

Operate Control

5-6

5

5-6

Boards (2)

Communicate and

ALL

Interact With the Crew

Demonstrate Supervisory

ALL

Ability (3)

Comply With and

1-3

Use Tech. Specs. (3)

Notes:

(1)

Includes Technical Specification compliance for an RO.

(2)

Optional for an SRO-U.

(3)

Only applicable to SROs.

Instructions:

Circle the applicant's license type and enter one or more event numbers that will allow the examiners to evaluate every

applicable competency for every applicant.

Author:

NRC Reviewer:

NUREG- 1021, Revision 8, Supplement 1

HARRIS NUCLEAR PLANT - AUG 2002

Competencies Checklist

Written Examination

Form ES-401-7

Quality Checklist

Facility:

Harris

Date of Exam: 26-Aug-02

Exam Level: RO

Item Description

1.

Questions and answers technically accurate and applicable to facility

a

a. NRC K/As referenced for all questions

b. Facility learning objectives referenced as available

RO/SRO overlap is no more than 75 percent, and SRO questions are appropriate

I

ner Section D.2.d of ES-401

4.

Question selection and duplication from the last two NRC licensing exams

appears consistent with a systematic sampling process

5.

Question duplication from the license screening/audit exam was controlled as

indicated below (check the item that applies) and appears appropriate:

/ the audit exam was systematically and randomly developed; or

the audit exam was completed before the license exam was started; or

Sthe examinations were developed independently; or

the licensee certifies that there is no duplication; or

other (explain)

6.

Bank use meets limits (no more than 75

percent from the bank, at least 10 percent new,

and the rest modified); enter the actual

question distribution at right

7.

Between 50 and 60 percent of the questions on

the exam (including 101iew questions) are

written at the comprehension/analysis level;

enter the actual question distribution at right

Bank

Modified

New

31

Memo

C A

8.

References/handouts provided do not give away answers

9.

Question content conforms with specific K/A statements in the previously

approved examination outline and is appropriate for the Tier to which they are

assigned; deviations are justified

10.

Question psychometric quality and format meet ES, Appendix B, guidelines

11.

The exam contains 100, one-point, multiple choice items; the total is correct and

agrees with value on cover sheet

a. Author

b. Facility Reviewer(*)

c. NRC Chief Examiner(#)

d. NRC Regional Supervisor

{

AL

Initial

AA

19

Wr

iv'

0'

2

A .4

Vt

I.-

QJ

A

Date

Note:

The facility reviewer's initials/signature are not applicable for NRC-developed examinations.

Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required.

IP

NUREG-1021, Revision 8. Supplement 1

2.

3.

Lt tO

Harris - August 2002

.j*,/

Written Examination

Form ES-401-7

Quality Checklist

Facility:

Harris

Date of Exam: 26-Aug-02

Exam Level: SRO

Initial

Item Description

a

b*

c

1.

Questions and answers technically accurate and applicable to facility

A119

i

2.

a. NRC K/As referenced for all questions

V)

r

b. Facility learning objectives referenced as available

3.

RO/SRO overlap is no more than 75 percent, and SRO questions are appropriate

t

d

0,

per Section D.2.d of ES-401

4.

Question selection and duplication from the last two NRC licensing exams

appears consistent with a systematic sampling process

5.

Question duplication from the license screening/audit exam was controlled as

indicated below (check the item that applies) and appears appropriate:

-- the audit exam was completed before the license exam was started; or

-the examinations were developed independently; or

the licensee certifies that there is no duplication; or

other (explain)

6.

Bank use meets limits (no more than 75

Bank

Modified

New

ia At

_4

percent from the bank, at least 10 percent new,

31l

"

and the rest modified); enter the actual

31

question distribution at right

7.

Between 50 and 60 percent of the guestions on

Memo

C/A

the exam (including 10 new questions) are

la0

written at the comprehension/analysis level;

p

enter the actual question distribution at right

8.

References/handouts provided do not give away answers

9.

Question content conforms with specific K/A statements in the previously

approved examination outline and is appropriate for the Tier to which they are

assigned; deviations are justified

10.

Question psychometric quality and format meet ES, Appendix B, guidelines

_____

11.

The exam contains 100, one-point, multiple choice items; the total is correct and

J Af*

\\

agrees with value on cover sheet

Printed Name Signature

Date

a. Author

,/4~

b. Facility Reviewer(*)

"o4q-.----36

X

c. NRC Chief Examiner(#)

d. NRC Regional Supervisor

/C,.'HAE -

(- Ar*ri

/

- '*/C

__--

NUREG-1021, Revision 8. Supplement 1

Note:

The facility reviewer's initials/signature are not applicable for NRC-developed examinations.

Independent NRC reviewer initial items in Column "c;" chief examiner concurrence required.

1

K

Harris - August 2002

Written Examination

Form ES-401-9 (R8, Si)

Review Worksheet

Qt 1. 1 2.

3. Psychometric Flaws

4. Job Content Flaws

5. Othe r

6.

17.

Q# /H

(1-5)O Stem Cues T/F

Cred. Partiall Job- Minutia

/

Back-

Q=

SRO U/FJS1

Explanation

FocusI

Dist.

Link

nisward IK/A IOn y

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated trueffalse statements.

More than one distractor is not credible.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if ajob content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-onlv (K/A and license level mismatches are unacceptable).

6.

Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditodal enhancement, or (S)atisfactory?

7.

At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

1 .

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

(F/H)

(1-5)

Stem Cues T/F

Cred. Partial Job- Minutia

/

Back-

Q= SRO

U/E/S

Explanation

Focus

Dist.

Link

units

ward

Ony

COMMON QUESTIONS

1

H

2

V

S

N

055K3.1- no comment, 2 not 3

2

H

3

V

S

N

061A3.03- no comment

3

F

3

V

S

B

022A3.01 -no comment

4

F

2

V

S

N

071G2.2.25-no comment, 2 not 3

5

F

3

V,

S

N

025AK2.05 - no comment

6

F

3

V

E

M

056A2.04 - need bank question to validate modified :validated

CUES NOW SAT

7

H

2

V

S

B

065AA2.08 - no comment, 2 not 3

8

F

2

X

8

E

B

T3G2.2.24 - need TS to validate distractor wrong, 2 not 4:

validated distractor adequate: QUES. NOW SAT

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q# LOK

-I

I-I

1

(F/H)

(1-5)

Stem Cues 7/F

Cred. Partial Job- Minutia

/

Back-

Q=

SRO U/EIS

Explanation

IFocus

Dist.'

Link

units ward jK/A Only

9

H

2

V

S

M

0011(4.11 -no comment, 2 not 3

10

F

1

X

V

U

B

103k4.04 - delete teaching, need better distractors, little

discrimination validity (attribute for RO or SRO), 1 not 2: licensee

explained plant specific attributes; is 2 not 1, QUES. NOW SAT

11

H

2

X

V

E

M

013K6.01 - delete teaching, 2 not 3, rephrased stem QUES.

NOW SAT

12

F

2

V

S

B

T3G2.3.11 - no comment, 2 not 3

13

F

3

V

S

N

055G2.4.18 - FnotH

14

F

2

V

S

N

076G2.4.18 - no comment, 2 not 3

is

H

2

G/

S

M

016K4.03 - no comment, 2 not 3

16

F

2

x

V

U

B

062G2.1.27 - distractors not realistic, 2 not 3, rephrased

explained distractors. QUES. NOW SAT

17

H

2

V

S

N

022G2.1.25 - (reference provided) no comment, 2 not 3

18

F

3

V

S

M

062AA2.06 - no comment

19

H

2

6/

S

B

075G2.1.25 - (reference provided) no comment

20

H

2

V

S

B

T3G2.1.25 - (reference provided) no comment

21

H

3

V

E

M

0011K5.42 - rephrase to use initial in stem vs each distractor:

corrected. QUES. NOW SAT

22

H

3

V

E

B

0101K1.06 - 'Progressively"?: deleted "progressively form stent

QUES. NOW SAT

23

F

1

V

U

B

0121<1.08 - little discrimination validity (attribute for RO or SRO),

1 not 2: replaced question: QUES. NOW SAT

24

F

2

v

S

N

029K4.02 - no comment, 2 not 4

25

H

3

V

S

B

063K2.01 - no comment

26

H

2

V

S

B

0111K5.06 - no comment, 2 not 3

27

F

2

V

S

B

086A1.01 - no comment, 2 not 3

28

H

2

V

E

N

T3G2.3.10 - rephrase to avoid 'not", 2 not 3: revised:O UES.

NOW SAT

29

H

3

V

S

B

015/017AA1.08 - no comment

30

H

2

V

S

N

040AA1.22 - no comment, 2 not 3

31

F

2

V

S

M

022K,2.01 - no comment F not H, 2 not 3

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q# LOK

LCDj

1-

I-r

-

(F/H)

(1-5)

Stem Cues TDI Cred. Partial Job- Minutia

/

Back-

Q= SRO U/E/S

Explanation

Focus

I

Dist. I

I Link I

Iunits ward

K/A On&

=

32

H

3

V

S

M

004A1.06 - no comment

33

F

1

V

U

B

008K4.07 -no discrimination validity (attribute for RO or SRO), 1

not 2: revised quest.: QUES. NOW SAT

34

H

3

V

S

N

001AK1.16 (reference provided), 3 not 4

35

H

2

X

V

S

M

014G2.1.11 - need better disractors, need reference, 2 not 3:

revised

36

F

2

V,

S

B

T3G2.1.2 - no comment

37

H

2

1

S

N

005K4.08 - no comment, 2 not 3

38

F

2

V,

S

B

033A2.03 - no comment

39

F

3

V

S

B

002K6.03 - no comment

40

F

2

V

S

B

059AA2.05 - no comment, 2 not 3

41

F

3

X

V

U

B

T3G2.4.7 - rephrase distractors, B only one that has backfill

capability at all (ROS <eSG) revised C distractor OUES. NOW

SAT.._

42

H

2

V

S

N

003G2.4.6 - no comment, 2 not 3

43

F

2

V

S

B

003A4.08 -no comment

44

H

3

V1

S

N

039A1.05 - no comment (reference)

45

H

3

V,

S

M

072K3.01 - no comment

46

H

2

V

S

M

026AK3.01 - no comment, 2 not 3

47

F

3

V

S

M

026A2.08 - no comment

48

F

3

v

S

N

078K1.03 - no comment

49

H

2

V

S

N

045A1.06- no comment

50

H

2

V

S

B

W/EO9EK1.02 -no comment 2 not 3

51

H

2

V

S

M

059K1.02 - no comment, 2 not 3

52

H

3

V

S

M

056AA2.22 - no comment

53

H

3

V

S

M

008AA2.12 - no comment

54

H

3

V

S

N

015K5.06 - no comment

55

F

2

V"

S

M

068K1.07 - no comment

56

H

4

V

S

N

058AA2.03 - no comment

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

(F/H)

(1-5)

Stem Cues T/F

Cred. Partial Job- Minutia

/

Back-

Q=

SRO U/iES

Explanation

Focus

Dist.

Link

units ward

K/A Only

57

H

3

V

S

M

004K3.08 - no comment

58

-F-

3

V

E

M

017A3.01 - contains NOT - rephrase, F not H, rephrased H not F,

H

QUES. NOW SAT

59

H

3

V

S

B

T3G2.2.13 - no comment

60

H

3

V

S

N

064K3.03 - no comment 3 not 4

61

F

2

V

S

B

W/EO8EK2.02 - no comment 2 not 3

62

F

3

V

S

B

073A4.02 - no comment

63

H

3

V,

S

N

O11EA1.01 - no comment

64

F

3

V

S

B

W/EO5EK2.01 - no comment

65

F

2

V

S

M

0O7EK1.03 - no comment

66

H

3

V

S

M

027AA2.15 - no comment

67

H

3

V

E

M

057AA2.15 - rephrase "not" statements: rephrased QUEST

NOW SAT

68

H

3

V

E

N

037AA1.13 - rephrase distractor B too obvious: revised. QUES.

NOW SAT

69

H

3

V

S

M

074EA1.05 - no comment

70

H

2

V,

S

M

T3G2.4.2 - no comment, 2 not 3

71

F

2

V

S

B

068AA1.21 - no comment

72

H

3

V,

S

M

035A3.01 - no comment

73

F

3

V&

S

B

038EK3.08 - no comment

74

F

2

V,

S

B

006A4.08 - no comment

75

F

2

V

S

B

029EK3.12 - no comment

1. 2.

3. Psychometric Flaws

4. Job Content Flaws

15. Other

6,.7

04

O

.I

i

I_______

7.

)Stem Cues T/F

Cred. Partial Job- Minutia

/

Back-

a= SR0

U/E/S

Explanation

Focus

1t.

I Link I

Iunits

war

K/En

__________________________

"141

12.11" 1h4tV

N*te*

W

+/-

72

V

S

N

T3G2.4.30 - no comment

72V U

U

N

028AA2.02 -this is not SRO only level: replaced with original RO

20!y_#86

8 H 2V

E

B

W/E14G2.3.10 - inadequate reference material to

validate.validated QUEST NOW SAT

79

F

2

V

U

N

W/E1 6EA2.01 - all answers could be argued to be true

rephrased QUEST NOW SAT

80

H

2

X

V

V1

U

B

T3G2.1.4 - clarify correct answer, could argue no correct answer'

rephrased QUES. NOW SAT

81

-F-

2

U

V1

U

B

036G2.2.8 - quest is about interlocks not procedure replaced

H

j

ques QUES NOW SAT NOW H NOT F

82

H

2

U

V

U

T3G2.2.25 - questions how the TS is met not basesrephrased

ques QUES NOW SAT

83

F

3

X

v,

V

E

M

T3G22.26- 2 correct answers rephrased QUES NOW SAT

84

-F-

2

V

E

B

T3G2.2.6 - describe the situation and let them determine it

H

requires a deviation and how to approve it: replaced QUES.

NOW SAT NOW H NOT F

85

H

3

XV

V

E

054G2.4.16 - need references to validate question, correct

answer stands out because of title, rephrased QUES. NOW SAT

86

F

3

V

S

B

T3G2.1.33 - no comment

87

H

2

V

S

M

W/E02EA2.02 - no comment

88

H

3

V/

V

S

M

W/E032.4.22 - no comment

89

H

2

V

S

N

055G2.4.1 - no comment

90

F

2

VV

S

M

T3G2.4.40 - no comment

91

H

2

VS

B

024G2.1.20 - no comment

92

H

3

V

S

B

W/EO6EA2.01 - no comment

93

H

2

V6

V

U

N

W/E1 1EA2.02 - correct stem - as written all answers are correct

rephrased, QUES. NOW SAT

94

F

3

V

S

B

033G2.4.4 - no comment

95

F

2

U

VT3G2.4.21

- does not meet K/A, NRC re-assigned K/A QUES

NOW SAT

96

H

2

V

S

B

T3G2.4.4 - no comment

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

C#

LOK

t

e

I

it

[ #

B

(F/H)

(1-5)

StemiCues T/F

Cred

Partial Job- Minutia

/

Back-

Q=

SRO U/S

Focus

[

Dist.

Link

units ward

K/A

On __y

97

F

2

V

I

S

M

W/E04EA2.01 - no comment

98

H

2

V

S

M

005AA2.03 - no comment

99

H

2

V

E

N

067AA2.13 - Typo (no comment) corrected

100

H

3

V

S

N

009EA2.01 - no comment

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK

I

(F/H)

(1-5)

Stem Cues[ T/F

Cred. Partial Job- Minutia

/

Back-

Q=

SRO U/E/S

Explanation

Focus

Dist.

Link

units ward

K/A

Only

76

F

2

V

S

B

T3G2.3.2 - no comment

77

F

2

V,

E

M

007A4.01 - need system diagram validated QUES NOW SAT

78

F

-1-

V

U

M

024AK2.01- no discriminating value: replaced ques QUES NOW

2

SAT

79

F

2

X

V

E

B

033AK1.01 -Need new distractors, BC not feasible explained site

specific QUES NOW SAT

80

F

2

V

S

N

059al.03 -no comment

81

F

2

V

S

B

079A4.01 - no comment

82

H

3

X

&I

E

NJ

T32.4.17 -typo, delete teaching rephrased QUES NOW SAT

83

F

4+

V

U

B

028G2.1.32 - replace with another L&P #6, no doscdminating

2

value replaced ques QUES NOW SAT

84

-F

4

X

V

U

B

051AK3.01 - incorrect as stated. F not H need new quest, No

H

2

discriminating value rewrote ques QUES NOW SAT

85

F.

2

V

S

M

076G2.4.10 - F NOT H,

86

H

3

X

V

E

N

T3G2.2.12 - does the ro declare operability? No not RO level

replace with original SRO only # 77QUES NOW SAT

87

H

3

V

E

M

005AK1.02 - Do they have adequate information to answer this?

Yes CUES NOW SAT

88

H-

2

V

E

B

036AA1.04 - can this be accomplished manually? No QUES

NOW SAT

89

F

2

V

U

B

W/E14EK3.02 - rephrase get rd of NOT", c is also true rephrase

QUES NOW SAT

90

F

3

V

S

M

003kl.03 - no comment

91

F

-I

V

U

B

T32.1.29 - no discriminating value - replace question replaced

2

question QUEST NOW SAT

92

F

3

V

S

B

009EEK3.20 - no comment

93

F

3

V

S

B

001 K2.02 - no comment

94

F

2

V

E

B

060AA2.06 - need system diagram to verfy K/A met validated

QUEST NOW SAT

95

H

3

X

U

B

015A4.03 -currently tests for loss of power not bistable position

rephrased to meet K/A QUEST NOW SAT

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

0#

LOK

i

(F/H)

(1-5)

Stem CuesI T/F

Cred. Partial Job- Minutia

/

Back-

Q= SRO U/E/S

Explanation

Focus

Dist.

Link

units ward

K/A

1n

1

96

F

2

V

E

N

004K4.16 - restructure to pull repeditive phrase from ques

rephrased QUEST NOW SAT

97

H

3

X

V

E

N

061 K2.02 - typo, rephrase stem rephrased QUEST NOW SAT

98

F

2

U

E

B

W/E02EA1.01 - Meet K/A? Rephrase as marked rephrased,

validated to meet K/A QUEST NOW SAT

99

F

2

X

V

E

M

T3G2.1.20 - rephrase the stem -too confusing rephrased

QUEST NOW SAT

100

H

2

V

E

M

013A2.01 - define immediate?, rhr suction valve position in

I condition described? rephrased QUEST NOW SAT

CHANGES TO SHEARON HARRIS AUGUST 2002

NRC INITIAL LICENSE EXAMINATION OUTLINE

BASED ON NRC AND HARRIS VALIDATION TEAM

COMMENTS

CHANGES TO WALK-THROUGH JPM OUTLINE

1)

Replaced "JPM COM-B.. 1.a, Perform Control Rod Exercise Test," per NRC Lead

Examiner request. Replaced with new, alternate path JPM, "JPM COM-B.1 .a,

Respond to Failed High Pressurizer Pressure Channel." JPM is Safety Function 3

which ensures required distribution of Safety Functions is still met.

2)

Changed "JPM SRO-B.l .f, Start an RCP Following Maintenance," to "JPM

COM-B.1 .f, Start an RCP Following Maintenance," making it a common JPM to

both RO and SRO-I candidates. The original "JPM RO-B.1.f, High RCS Pressure

While Solid," used the same AOP as "JPM COM-B. 1.a, Respond to Failed High

Pressurizer Pressure Channel," and was considered to not be valid as a different

system / function JPM during validation.

3)

Replaced "JPM COM-B. 1.c, Decreasing CCW Surge Tank Level," with "JPM

COM-B.1.c, Secure One Train of CCW to the RHR HXs," using OP-145. The

original JPM is on the Audit Exam and cannot be used on the NRC Exam. This

replacement JPM meets the same Safety Function as the original JPM so the

required distribution of Safety Functions is still met. This change is reflected in

the RO and SRO-I outlines.

4)

"JPM COM-B.l .e, Transfer to Hot Leg Recirculation," has an identically titled

JPM on the Audit Exam. Review of these two JPMs indicates that there is a

significant difference between the two JPMs (NUREG-1021, ES-201, D.3.b).

The Audit Exam JPM allows the candidates to successfully complete the task with

no failures, while the NRC Exam JPM contains as alternate path which requires

the candidates to take a different success path to arrive at a different end

configuration. No replacement should be required for this JPM.

5)

The KA for "JPM COM-B.2.b, Manually Align Charging Due to a Loss of IA,"

has been changed to better reflect the task and there is no change required to the

Safety Function.

Page 1 of 4

CHANGES TO SHEARON HARRIS AUGUST 2002

NRC INITIAL LICENSE EXAMINATION OUTLINE

BASED ON NRC AND HARRIS VALIDATION TEAM

COMMENTS

6)

The above changes reflect a change in the Direct / Modified / New distribution of

JPMs. The limits of no more than 30% from the last NRC Exam (NUREG-1021,

ES-201, D.3.b) is met. The allowed bank usage (NUREG-1021, ES-201, D.3.b)

of no more than 80% of any walk-through JPM exam being taken directly from

the facility's testing materials without significant modification is also met. The

new distributions are as follows:

DIRECT

SIGNIFICANTLY

(LAST NRC EXAM

MODIFIED

NEW

7(1)

0

3

SRO-I

7(1)

0

3

SRO-U

3 (0)

0

2

CHANGES TO ADMINISTRATIVE JPM OUTLINE

1)

Replaced "JPM SRO-A.1-1, Change the Dedicated SPDS Screen Location," to

"JPM SRO-A. 1-1, Perform Review of Daily Surveillance Requirements Log,"

due to concerns raised about discriminatory value of original JPM.

2)

Expanded "JPM SRO-A.4, Determine Protective Action Recommendations," to

"JPM SRO-A.4, Perform an Emergency Action Level Classification and

Recommend Protective Actions," per NRC Lead Examiner request. Also

included grading criteria such that the classification following the scenario counts

toward 20% of the grading of this JPM, the classification within the JPM counts

toward 20% of the grading of this JPM, and the PAR determination counts the

remaining 60% of the grading.

3)

No changes made to RO Administrative JPM Outline.

CHANGES TO SCENARIO OUTLINES

SCENARIO #1

1)

Replaced Event 2, "Main Turbine High Vibration Requiring Plant Power

Reduction," with Event 1, "Continued Plant Power Reduction," due to concern of

plant validation team that crew is likely to trip the plant since vibration levels do

not decrease as turbine load decreases (simulator model). This still meets the

requirement to perform a power change.

Page 2 of 4

CHANGES TO SHEARON HARRIS AUGUST 2002

NRC INITIAL LICENSE EXAMINATION OUTLINE

BASED ON NRC AND HARRIS VALIDATION TEAM

COMMENTS

2)

Previous Event 1 is now Event 2.

3)

Replaced Event 4, "RCP High Vibration," with "Pressurizer Level Channel

Failure" since NRC Lead Examiner expressed opinion that a RCP Vibration

problem resulting in tripping the reactor and stopping the pump was not a valid

component failure. Resulted in a slightly different entry path to PATH-i, now

being caused by the loss of offsite power. Also reordered events to make this

replacement Event 3 instead of Event 4.

4)

Previous Event 3 is now Event 4.

5)

No changes to Events 5 or 6.

6)

Designated Event 7, "EDG A Restart Following SG Depressurization," as a

component failure for the BOP and SRO only, eliminating the designation for the

RO. This event was not counted in the original Competencies Checklist as a

required event for any candidates as it occurs following EOP entry.

7)

No change to Event 8 (classification).

SCENARIO #2

1)

Changed initial conditions to make this a "low power" scenario. Now start at

approximately 52% power with one train of FW in service.

2)

Added Normal event for BOP as Event 1 in scenario, requiring the position to

"Place the Second Train of FW (Condensate and Condensate Booster Pump) in

Service."

3)

Previous Event 1 is now Event 2.

4)

Changed previous Event 2 from "Normal Service Water Pump A Trip" to

"Normal Service Water Pump A Shaft Shear" to provide more required actions

for RO candidate in response to failure. Also now is Event 3 instead of Event 2.

5)

Added new Event 4, an instrument malfunction for the BOP, "Failure of a SG

PORV Pressure Transmitter," as the NRC Lead Examiner determined a SG Tube

Leak is not to be considered a component failure.

8)

Events 3, 4, and 5 are now Events 5, 6, and 7, respectively.

6)

No change to Event 8 (classification).

Page 3 of 4

CHANGES TO SHEARON HARRIS AUGUST 2002

NRC INITIAL LICENSE EXAMINATION OUTLINE

BASED ON NRC AND HARRIS VALIDATION TEAM

COMMENTS

SCENARIO #3 (Spare)

1)

Changed Event 1 title from "LCV-I 15A, VCT Divert Valve Control Failure to

HUT" to "LT- 112, VCT Level, High Failure". Event is same, but better

described by new title.

2)

No changes to Events 2, 3, 4, and 5.

3)

Changed initiating malfunction and changed title of Event 6 from "RCS Loop A

Cold Leg Leak at Approximately 1000 gpm leak, Ramped in Over 15 Minutes" to

"RCS Loop A cold leg small break LOCA, Ramped in Over 15 Minutes." There

are no actual changes to the expected crew response.

9)

No changes to Events 7 and 8.

4)

No change to Event 9 (classification).

Page 4 of 4

Record of Rejected K/As

Form ES-401-10

Tier/Group

Randomly Selected K/A

Reason for Rejection

1 / 1

068AAL.01

Exceeds number of topics in E/APE allowed

1 / 1

055EA2.05

Exceeds number of topics in E/APE allowed

1 / 1

026AK3.04

Exceeds number of topics in E/APE allowed

I / 1

0052.4.4

Exceeds number of topics in E/APE allowed

I / 1

027AK1.01

Exceeds number of topics in E/APE allowed

1 / 1

015/017AK1.04

Exceeds number of topics in E/APE allowed

1 /2

008AA2.15

Exceeds number of topics in E/APE allowed

1 / 2

033AA2.12

Exceeds number of topics in E/APE allowed

1 / 2

022AK3.01

Exceeds number of topics in E/APE allowed

1 / 2

W/E01EA2.02

Exceeds number of topics in E/APE allowed

1 / 2

029EK1.03

Exceeds number of topics in E/APE allowed

1 / 2

OOAA2.01

Exceeds number of topics in E/APE allowed

1 / 2

001AA 1.01

Exceeds number of topics in E/APE allowed

1 / 2

022AA2.04

Exceeds number of topics in E/APE allowed

1 / 2

W/E05EA2.01

Exceeds number of topics in E/APE allowed

1 / 2

033AA1.01

Exceeds number of topics in E/APE allowed

1 / 2

037AK3.08

Exceeds number of topics in E/APE allowed

1 / 2

011 EA2.01

Exceeds number of topics in E/APE allowed

1 / 2

009EA2.32

Exceeds number of topics in K/A allowed

Page 1 of 5

NUREG- 1021, Revision 8, Supplement 1

HARRIS NUCLEAR PLANT - AUG 2002

Record of Rejected K/As

Form ES-401- l0

Tier/Group

Randomly Selected K/A

Reason for Rejection

1 / 2

029EK3.12

Exceeds number of topics in E/APE allowed

1 / 3

056AA1. 11

Exceeds number of topics in E/APE allowed

2 / 1

001K5.39

Exceeds number of topics in System allowed

2 / 1

004K5.35

Exceeds number of topics in System allowed

2 / 1

004K6.14

Exceeds number of topics in System allowed

2 / 1

001K4.14

Exceeds number of topics in System allowed

2 / 2

012K6.l1

Exceeds number of topics in System allowed

2 / 2

002A4.06

Exceeds number of topics in System allowed

2 / 2

0022.4.18

Exceeds number of topics in System allowed

2 / 2

01 1A3.01

Exceeds number of topics in System allowed

2 / 2

006K6.18

Exceeds number of topics in System allowed

2 / 2

0862.1.12

Exceeds number of topics in K/A allowed

2/2

014Al.03

Exceeds number of topics in System allowed

2 / 2

062K4. 10

Exceeds number of topics in System allowed

2 / 2

0862.1.32

Exceeds number of topics in K/A allowed

2 / 2

002K5.14

Exceeds number of topics in System allowed

2 / 2

006A2. 10

Exceeds number of topics in System allowed

2 / 2

062A2.06

Exceeds number of topics in System allowed

2 / 2

011 K5.02

Exceeds number of topics in System allowed

Page 2 of 5

NUREG- 1021, Revision 8, Supplement 1

HARRIS NUCLEAR PLANT - AUG 2002

Record of Rejected K/As

Form ES-401 -10

Tier/Group

Randomly Selected K/A

Reason for Rejection

2 / 2

012K3.01

Exceeds number of topics in System allowed

2 / 2

075A4.01

Exceeds number of topics in System allowed

2 / 2

011 A2.06

Exceeds number of topics in System allowed

2 / 2

0642.1.32

Exceeds number of topics in System allowed

2 / 2

006A2.12

Exceeds number of topics in System allowed

2 / 2

039A1.l0

Exceeds number of topics in System allowed

2 / 2

026A3.01

Exceeds number of topics in System allowed

2 / 2

033A2.01

Exceeds number of topics in System allowed

2 / 2

063A2.01

Exceeds number of topics in System allowed

2/2

O/OA 1.07

Exceeds number of topics in System allowed

2 / 2

002K5.01

Exceeds number of topics in System allowed

2 / 3

008K4.09

Exceeds number of topics in System allowed

2 / 3

005A2.01

Exceeds number of topics in System allowed

2 / 3

008A3.05

Exceeds number of topics in System allowed

3

2.1.4

Exceeds number of topics in Category allowed

3

2.1.32

Exceeds number of topics in Category allowed

NUREG- 1021, Revision 8, Supplement 1

Page 3 of 5

HARRIS NUCLEAR PLANT - AUG 2002

Record of Rejected K/As

Form ES-401-10

Tier/Group

Randomly Selected K/A

Reason for Rejection

2/1

061K6.01

Replaced by 061A3.03 - improved KA match - reviewed / discussed with NRC

2/1

022K4.01

Replaced by 022A3.01 - Harris has no Containment Penetration Cooling - reviewed /

discussed with NRC

I/3 (1/2)

065AA1.02

Replaced by 065AA2.08 - improved KA match - reviewed / discussed with NRC

2/3 (2/2)

103A4.06

Replaced by 103K4.04 - improved KA match - reviewed / discussed with NRC

1/2 (1/1)

059AA2.03

Replaced by 059AA2.05 - improved KA match - reviewed / discussed with NRC

2/1

0682.1.32

Replaced by 068K1.07 - improved KA match - reviewed / discussed with NRC

2/3 (RO Only)

007A4.04

Replaced by 007A4.01 - Harris has no Control Board indications / control of PRZ vent

valve - reviewed / discussed with NRC

3 (RO Only)

2.4.19

Replaced by 2.4.17 - improved KA match - reviewed / discussed with NRC

2/3 (RO Only)

0282.1.27

Replaced by 0282.1.32 - improved KA match - reviewed/ discussed with NRC

2/1 (RO Only)

004A3.08

Replaced by 004K4.16 - improved KA match - reviewed/ discussed with NRC

3 (SRO Only)

2.3.4

Replaced by 2.4.30 - improved KA match - reviewed / discussed with NRC

1/3 (SRO Only)

028AA2.13

Replaced by 028AA2.02- Harris uses no graphs to allow for interpretation of

uncompensated PRZ level - reviewed / discussed with NRC

1/1 (SRO Only)

WE14.2.3.10

Replaced by WE14.2.4.20 - improved KA match - reviewed / discussed with NRC

3 (SRO Only)

2.1.10

Replaced by 2.2.6 - improved KA match - reviewed / discussed with NRC

1/1 (SRO Only)

051AA2.02

Replaced by 055.2.4.1 - originally selected KA closely matches another KA (055K3.01)

previously selected and would result in 'double jeopardy' - replaced with different

system KA since only KAs in 051 E/APE with importance > 2.5 relate to same topic

reviewed / discussed with NRC

Page 4 of 5

NUREG-1021, Revision 8, Supplement 1

HARRIS NUCLEAR PLANT - AUG 2002