ML21075A102

NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)

ML21075A102
Person / Time
Site:	Hope Creek
Issue date:	03/09/2021
From:	Public Service Enterprise Group
To:	Todd Fish Operations Branch I
Shared Package
ML19353E011	List: IR 05000354/2021301 ML20132A155 ML21032A102 ML21075A102 ML21075A103 ML21075A105 ML21076A062 ML21076A063 ML21076A064 ML21076A066 ML21117A083 ML21117A084
References
EPID L-2021-OLL-0001
Download: ML21075A102 (5)

Text

.ES401.

x

- A

-WrittenCExaminationReview:

Worksheet-HC 2021A

- A T

A

- FormES4019

=

' 1..

. 2.;

.3.

Psychometric Flaws 4.JobConterit Flaws

$.Other

.6/

. 7.

8.

' Q#'.LOK' LOD

'(F/H).

(1'-5)'.Stem' Cues

' T/F Credy.

Partial Jo.b-Minutia

1. /'.Back--

'.Q=SROB/M/,NAU/E/S'

- Ex'planation L '^

Focus

- Dist.

. Linkt un.its w'ardl K/A.Only:

. 1 H

3 Y

- B

. S 2'

. -H

'3.5

-- Y.

' B^. S

^ 3

- F.

2.7 Y

- N S

4.

F.'

- 3

. '. CY.

. N

- 'S

.5

' H 2.7' jY.

- B

- S 6

- H'.

. 3.3

- Y

. ^N

'.S

- 7.

F 3~.0 Y

- B'

'S 8

- H:

135x xY

- x

~ N

- S'-

Instructi.ons

.[Refer toSectiorrD ofES-4011 and Appendix Bforadditional information regarding^each' of.the followin'g.concepts.)

1-Eiriter thelevel of;knowledge (LOK)of eachquestion as^either (F)undamental'or (H)igher cog'nitive level.

2.--

Enter th'e level ofdifficulty (LOD)of eachquestion using.a 1-5(easy --diffi.cult)rating'scale.(questions.in the2.-4 range areacceptab'le).

- 3.

Checktheappropriate b.oxif a.psychometric flaw isid.entified:-

- :Thestemlacks sufficient focus:to elic'it the.correct answer (e'.g./u'nclear.intent,more iriformation isneeded,orto'o

'muc.h needles.s information).

Thestemordistractorscontain:cues;(i.e.,

clues, specific determiners,

phrasing, length, etc).:

Theanswer choices areacolle.ctio~n ofunrelatedtrue/f'als.estatements

^

- The:distractors-are.not credibl'e; single.implausible'distractors should.be.repaired, more.than:oneis unacceptable.

- Oneor'moredistractors is-(are)p'aitiallycorrect (e.g.,.if.the applicant can makeun'stated as.su'niptions'that areriot contradicted.by.stem).

4.

Check~the appropriate boxi.f a.job content erroriside.n'tified:

- f The-question isnotlinked tothe jobrequirements.(i.e.,

thequestion hasavalid.K/A but,as.written,.is.not'operational incontent)'.

- -.'The que.stion requires the"recall ofknowledge'that is.too

'specific for'theclosed reference testrnode.(ie.,'

itisnotrequired.

tobe'known'from mernory).

The:question:contains data with anunrealistic level.of accuracy orinconsistentunits-(e.g.,

panelm.eter in'perc:ent w.ith.question i.n:g'allons).

. The4uestion requ.ires reverse logic orapplication compared tothejob re'q'uirements.

5. A ; W forco.nformance with theaptbroved K/A'and those that are designated SRO-only (K/A an:d:license level mismeitchesare unacceptable).

-6.

Enter question'source

.(B)an'k;-(M)odi.fi'ed, or'(N)~ew^.

Checktha't (M)'odifiedq.u'estions meet'c'riteria ofE.S201Sectio.n D.2.f 7.

B.ased dnthereviewer's judgment,.ifthe ques'tion a's written (U-)nsatisfeictory (req'uiririgrepair o'r replacerrient),'in needof(E)ditorialenhancement, or(S)atisfactory?'

8 At:a

minimum, explairi ariy "U"fatings:(e.g.,

howthe'Appendix B.p,s'ychometricattribute^s areriot being rnet).'

ES401

:2 FormES401.9 1.

2.

3.Psychometric:Flaws.

~

. 4.JobContent Flaws 5.Other

. '6.:'.7'.

8'.

. Q#.LOK LOD (F/H);

.'(1-5)

.Stem.Clies T/F'.

Cred.Partial Jo.b:-

Minuti.a

1. /.'B'ac'k-Cl='^SRO

.B/M/N.

U/E'/S Explana'tion

' x ;Focus g

^

-.jjDist.

- Link;

' t)nits

wardK/A;

~Only.

~

9 F P.3'A Y A N'

S x

~1'O'F

' 3.5

. y

..B : 'S.

x

^

11.^

F 3

Y

~

- N S

• 12 F

- 4

= Y B

S 13 H

3.3

~

Y B

S 14 x H 4 '-

Y B

S 15 H

2.5 Y

- : N S

-16 F

3.5.

y N

S 17

- F

' 3.3

=

Y

- B S

18

.H 3.3 Y

' B

. S 19 H

3.0 Y

'-N

- S

20 F

3.0;

Y

B

'S 21 F

' 2.8~

Y B

.S 22

' F 2.8 Y

. 'B S

'23

' H 2.8

'.'Y B'

' S

- 24 F

3.5 Y x

. N

'S.

25 H

3.5' Y

B'.~-S 26 F

2.8 Y

^

" B S

27 F

2.8 Y

N

. S

-28

'F 2.7

,'Y.

N S

29 :P 3.0 Y

NA

' S

- 30

- H 1

Y

' '.'.B

- S

1.

. 2.'-

3.-PsychometricFlaws 4'.Job.Gont.entFlaws

. 5.Other 6.'

7

~-

' 8.

Q#

LOK LOD' (F/H-).(1-5) StemCues-:T/F:Crede Partial Job-.Minutia

1. / Back-Q= SROB/M/NU/E/S ExpIanation Focus Dist.-

, 'Link ;

unitsward'K/AOnly.'.

31 H

3.7

-Y B

S 32 F

3.3 Y

' N S-

.33 H 3.0 Y

B

- S 34 H

3.8 M

S.

35 F

44 Y

B

- ^S 36 F

2.5 X

Y

'B E~ Choice-B:

"inserted..."'notplausible..

Revised.distractor.

37

. H

.3.3

'Y

.B S.

38' F

1.0' Y

B

' U LOD=1'.

$electednewKAandwrotenewQ..

39 F

3.0 Y

.B S

40 H

. 3.0 Y

B

-S 41 F

3.0

~

Y' B

- S 42 H

4.0-

- Y

'B S'

43 F

3.5 B.

.. S '.

44. H 3.0 Y

N

~S 45 F

2.5

'Y B

S 46 H

3.3 Y

- N..

S'

'47 H~

3.3 Y

N S.

48 F

3.0' Y

.M S

49 F

3.3 Y

B S

50 H

2.8'-

~

~

Y B

S 51

~H. '2.9.

Y B

. 'S 52 H

3.5 Y

B S

- 53 F

3.5

'Y B

S 54 H

3.0 Y

B S

i

.1'.'

- 2..

3.-

Psychornetric Flaws 4.Job.Content Flaws 5.Other

-6.'

7.

-- 8.

Q#

LOK' LOD Explan'ation-

-(F/H). (1-5)StemCuesT/F Creid.

Partial Job-Minutia#/ Back--Q=..SROB/M/NU/E/S

~

Focus Dist.

Iink

' uni$;

.ward

~ K/A Only 55 H

2.8 Y

- B S

. 56 F

-3.0 Y

- B S'

57

. H

- 3.0 Y

B S:

58

- F 3.0 Y

.B S

59 H'

4.0

~

Y B

S 60 H

3.3 Y

- ^B S.

61 H

3.3 Y

cB S

62 H

2.3 Y

- B S:

63 H

3.0 Y

. B

S 64 H

3.0 Y

B

.S' 65 F

~3.0 X

Y

~

- B

' U Choices B&C

... cannot beoverridden

.."~implausible.'Revised answer choices-&

format.

66 H

3.0 y.'

B

.S 67 H

3.3 Y

B

~S 68

'F 3.0 y

B

. S' 69 F

3.7

- Y.

. B S

70 H.

3.0' y

B

- ~.^.S 71 F

- 3.5' Y

B S

.72. H.

3.0.

Y.

. B.

'.S 73

' F 3'.0 Y

- N --

S 74 F

3.0:

.Y N

S 75 F

- 3.8 B

=

N S

76 H'-

' 3.0

~-

- '.-Y

' Y B': '.S.

77 F

2.7 Y

Y B.

S

.1.

2.--

3.'Psychometric Flaws Q#.

LOK

- -LOD

~

- 4.JobGontent Flaws.-

. '5.Other 6

'-'7..'

. ~-'

- 8.

. (F/H') (1-5).StemC'ues; T/F-Cred.

Pahial Job.Minutia

1. / Ba'ck Q=j.SR'O B/M/N U/E/S Explariation Focus'

- 1

- Dist.'

'Lirik

~

. unitswardK/A

.Only o

78 H

. 3.0 Y

t

-B S

- 79 H

3.0 Y

Y

'N

- S

- 80.'F

. 3.3

'X

~- Y

- N N

U No;tSR'O-only:

ROkriowle'dge.th'at Rxtrip isreportable, which als'o

.makes choices B'&.

Cimplausible.

Revised choices B&C;.

81

. ;H 31.0 Y

Y.

B

' S 8'2

' R

~

- 3.0 Y

Y

'-B ;

S-

- 83:

H

3.3 V.

- : B.

' S :

84 H

1'.4X X

- Y Y

B

..'ULOD=1,DWT-P.cu'r.veprovides cuesa'nd isneedl'ess info..

.Revised Qstem.

85 H

. ^2.0

-. Y

. B :

- :S 86

- H

.3.0 Y

Y B

' -. S 87

' H

'.3.0.

- -.Y Y

~N S

88

. F 4.0

- Y Y

'N

.S'.

89 F. '.3.5 x

~

.V

Y

' B. :S

- 90.

' R

' 2.3 Y

.Y..

- B S.:

91 H

4A

Y Y

- N

S

- 92. F 4.0.

Y

- Y N

S'.

. 93 H~

.17

~'

~

'Y

- Y B'

' S

- 94 F

2.3 A

V A

- Y~

- :B

- S:

95 H

2.7

' Y-Y

' N S

96~

' H 3.7.

Y

'Y B'

' S

.97

'F

' 2.5

- Y

- Y' B

.'S 98

.'F 3.0

- Y Y

N S-99 F

4'.0 Y

-..Y'N S

100 H 4.0

'Y~..Y

'.B

' 'S.