ML20260H150

NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)

ML20260H150
Person / Time
Site:	FitzPatrick
Issue date:	10/14/2019
From:	Exelon Generation Co
To:	Todd Fish Operations Branch I
Fish T
Shared Package
ML19105A122	List: ML20043C809 ML20087G852 ML20087H514 ML20260H150 ML20260H151 ML20260H152 ML20260H153 ML20260H154 ML21056A284 ML21056A285 ML21056A306 ML21056A307
References
CAC 000500
Download: ML20260H150 (7)

Text

ES401 JamesA.FitzPatrick Written Examination Review Worksheet FormES4019 2

3.PsychometricFlaws 4.JobContent Flaws 5.Other a

1.LOK LOD 6.Source7.Status 8.Explanation (F/H)(15) pStem Cred Job-(B/M/N)(U/E/S)

Cues T/F gi3g PartialLink Minutia#/UnitsBackwardQ

- K/A SROOnly 1

H 3.7 M

S 2

F 3.1 B

S 3

F 3.2 B

S 4

H 3.5 N

S 5

H 4.0 N

S 6

F 2.2 B

S 7

F 2.8 B

S 8

H 3.1 B

S Refer toSection DofES-401 andAppendix Bfor additional information regarding each ofthefollowing concepts:

1. Enter thelevel ofknowledge (LOK) ofeachquestion aseither (F)undamental or(H)igher cognitive fevel.

2. Enter thelevel ofdifficulty (LOD) ofeachquestion a1(easy) to5(difficult);

questions with adifficulty between 2and4areacceptable.

3. Checktheappropriate boxifapsychometric flaw isidentified:

"StemFocus":

Thestemlacks sufficient focus toelicit thecorrect answer (e.g..

unclear

intent, moreinformation

isneeded, ortoomuchneedless information).

"Cues":

Thestemordistractors contain cues(e.g.,

clues, specific determiners,

phrasing, length).

"T/F":

Theanswer choices areacollection ofunrelated true/false statements.

"Cred.

Dist.":

Thedistractors arenotcredible; single implausible distractors should berepaired, andmorethanoneisunacceptable.

"Partial":

Oneormoredistractors arepartially correct (e.g.,

iftheapplicant canmakeunstated assumptions that arenotcontradicted bythestem).

4. Checktheappropriate boxifajobcontent flaw isidentified:

"JobLink":

Thequestion isnotlinked tothejobrequirements (i.e.,

thequestion hasavalid K/Abut,as written, isnotoperational incontent).

"Minutia":

Thequestion requires therecall ofknowledge that istoospecific fortheclosed-reference test mode (i.e..

itisnotrequired tobeknownfrommemory).

"#/Units":

Thequestion contains datawithanunrealistic level ofaccuracy orinconsistent units (e.g.,

panel meter inpercent withquestion ingallons).

"Backward":

Thequestion requires reverse logic orapplication compared tothejobrequirements.

5. CheckIluestions thataresampled forconformance withtheapproved K/AandthoseK/Asthat aredesignated "SRO-only."

(K/A andlicenselevel mismatches areunacceptable.)

6. Enter question's source:

(B)ank, (M)odified, or(N)ew.

Verify that(M)odified questions meetthecriteria ofFormES-401, Section D.2.f.

7. Basedonthereviewer's

judgment, isthequestion, aswritten, (U)nsatisfactory (requiring repair orreplacement),

inneedof (E)ditorial enhancement, or(S)atisfactory?

8. Ataminimum,explain any"U"status ratings (e.g.,

howtheAppendix Bpsychometric attributes arenotbeingmet).

ES-401 2

FormES4019 1

2 3.Psychometric Flaws 4.JobContent Flaws 5.Other 6.Source 7.Status Q

LOK LOD 8.Explanation (F/H)(1-5) em3 Cues T/F @3ed.

PartialJob-LinkMinutia#/UnitsBackwardQ - K/A SROOnly 9

H 3.6 B

S 10 H

3.7 B

S 11 H

4.1 B

S 12 H

3.0 B

S 13 F

2.4 B

S Qfits Fundamental not 14 H

3.4 M

E Higher Order.

Reclassified as"F".

15 H

3.6 N

S 16 H

4.0 B

S 17 H

3.7 B

S 18 F

2.8 B

S Qbasically asks

- 'Where aretheCRDpumps?"

19 F

1.0 N

y Selected newK/Aand wrotenewQ.

20 H

3.5 N

S 21 F

2.5 N

S 22 H

2.9 B

S 23 F

2.2 N

S 24 F

3.0 N

S

ES401 3

FormES4019 1

2 3.Psychometric Flaws 4.JobContent Flaws 5.Other 6.Source 7.Status Q

LOK LOD 8.Explanation (F/H)(1-5) e ed.

Cues T/F PartialJob-LinkMinutia#/UnitsBackwardQ

- K/A SROOnly 25 H

3.7 M

S 26 H

3.4 B

S 27 F

2.2 B

S 28 H

3.5 B

S 29 H

3.3 N

S 30 F

2.5 B

S 31 H

3.4 B

S 32 F

2.2 B

S 33 F

2.7 B

S 34 F

2.9 M

S 35 F

2.8 B

S 36 H

3.5 N

S 37 H

3.6 B

S 38 F

2.6 B

S 39 H

3.7 M

S 40 H

3.3 B

S

ES-401 4

FormES4019 1

2 3.Psyc'hometricFlaws 4.JobContent Flaws 5.Other 6.Source 7.Status Q

LOK LOD 8.Explanation (F/H)(1-5) em3 Cues T/F 3ed.

PartialJob-LinkMinutia#/UnitsBackwardQ

- K/A SROOnly 42 F

2.8 B

S 43 F

2.8 N

S 44 H

37 B

S 45 F

2.7 N

S 46 F

2.4 N

S 47 H

3.4 N

S 48 H

3.6 M

S 49 H

3.3 B

S 50 F

2.7 N

S 51 F

2.8 B

S 52 H

3.8 N

S 53 H

3.3 B

S 54 F

2.4 N

S 55 H

2.9 B

S 56 H

3.3 B

S 57 H

3.7 N

S

ES401 5

FormES401-9 1

2 3.Psychometric Flaws 4.JobContent Flaws 5.Other 6.Source 7.Status Q

LOK LOD 8.Explanation (F/H)(1-5) ed cuesT/F *td.

PartialJob-LinkMinutia#/UnitsBackwardQ

- K/A SROOnly 59 H

3.8 B

S 60 H

3.3 B

S 61 H

37 M

S 62 F

2.7 N

S 63 H

3.8 B

S 64 F

2.2 N

S 65 F

2.5 B

S 66 H

3.5 B

S 67 H

3.6 B

S 68 F

2.5 B

S 69 F

2.4 B

S 70 F

2.9 B

S 71 H

3.8 B

S 72 F

2.2 B

S 73 F

2.0 B

S 74 F

2.0 B

S

ES401 6

FormES4019 1

2 3.Psychometric Flaws 4.JobContent Flaws 5.Other 6.Source 7.Status Q

LOK LOD 8.Explanation (F/H)(1-5) em, Cues T/F 30td.

PartialJob-LinkMinutia#/UnitsBackwardQ-K/A SROOnly 76 H

3.8 M

S Choice

'D'implausible 77 H

3.8 N

E (can't declare anUnusual EventandanAlert).

Rewrote distractor

'D' 78 H

3.4 B

S 79 H

3.6 N

S 80 H

3.3 M

S 81 H

3.5 B

S 82 F

2.9 N

S 83 H

3.8 M

S 84 H

3.4 B

S 85 H

3.5 N

S 86 F

2.7 N

S 87 H

3.8 N

S 88 H

3.8 N

E Added','ONLY" tochoices A &'B 89 H

3.9 M

S 90 H

3.4 B

S Qdoesnotdiscuss "RPV 91 F

2.4 X

B y

internals."

WrotenewQ.

92 H

3.8 M

S

ES-401 7

FormES-4019 1

2 3.Psychometric Flaws 4.JobContent Flaws 5.Other 6.Source 7.Status Q

LOK LOD 8.Explanation Cues T/F @if38td.

PartialJob-LinkMinutia#/UnitsBackwardQ - K/A SROOnly (F/H)(1-5) em 93 H

3.5 N

S 94 F

2.5 B

S 95 F

24 B

S 96 F

2.3 B

S 97 F

2.7 B

S Minutia.

Notsuitable 98 F

2.0 X

N U

memory-level Q.Selected newK/Aandwrotenew Q.

99 H

3.0 B

S 100 F

2.1 B

S