ML21075A102

NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)

ML21075A102
Person / Time
Site:	Hope Creek
Issue date:	03/09/2021
From:	Public Service Enterprise Group
To:	Todd Fish Operations Branch I
Shared Package
ML19353E011	List: IR 05000354/2021301 ML20132A155 ML21032A102 ML21075A102 ML21075A103 ML21075A105 ML21076A062 ML21076A063 ML21076A064 ML21076A066 ML21117A083 ML21117A084
References
EPID L-2021-OLL-0001
Download: ML21075A102 (5)

Text

'- A Worksheet-HC 2021A A T A FormES4019

.ES401. x

-WrittenCExaminationReview:

. . . . . . . . . . . . . . . . . = . . . , . . . . . .

1. . . 2.; .3. Psychometric Flaws - -

4.JobConterit Flaws

. $. Other .6/ 7. . . . . . 8.

Q#' .LOK' LOD .

'(F/H).  ;(1'-5)' .Stem' Cues T/F Credy. '

Partial Jo.b-Minutia #/'. Back-- '.Q=SROB/M/,NAU/E/S- '

Ex'planation L '^

Focus  : . .

Dist. -

. Linkt '

un.its w'ardl K/A. Only: .

1 H 3 Y B S 2' . -H -

'3.5 ' ' ' -

-- Y. .

B^. S

^

3 F. 2.7 ' ' '

Y N S

4. F .'3 -

. '. CY. -

. N -

'S

.5 '

H 2.7' jY. -

B -

S -

'.6 .

H' . . 3.3 -

Y . . . . ^N .S '

7. F 3~.0 Y B' 'S 8 H: 135 x xY x N S'-

- - ~ '

Instructi.ons

.[Refer toSectiorrD ofES-4011and Appendix B for additional information regarding^each' of.the followin'g.concepts.) '

1- -

Eiriter thelevel of;knowledge (LOK)of each question as^either (F)undamental'or (H)igher cog'nitive level.

2.-- Enter -

th'e level ofdifficulty (LOD)of eachquestion using.a 1 5 (easy --diffi.cult)rating'scale.(questions.in the2.-4 range .

areacceptab'le).

3. Check the appropriate b.oxif a.psychometric flaw isid.entified:-

Thestemlacks sufficient focus:to elic'it the.correct answer (e'.g./u'nclear.intent,more iriformation isneeded, orto'o 'muc.h needles.s information).

• . Thestemordistractorscontain:cues;(i.e., clues, specific determiners, phrasing, length, etc).:

a colle.ctio~n

^ '

Theanswer choices are ofunrelatedtrue/f'als.estatements -

The:distractors-are.not credibl'e; single.implausible'distractors should.be.repaired, more.than:oneis unacceptable. .

Oneor'moredistractors is-(are)p'aitiallycorrect (e.g.,.if.the applicant can makeun'stated as.su'niptions'that are riot contradicted.by.stem).

4. Check ~the appropriate boxi.f a.job content error iside.n'tified: -

f . . .

. The-question isnotlinked tothe job requirements.(i.e., the question hasavalid.K/A but,as.written,.is.not'operational incontent)'.

-. 'The que.stion requires the"recall ofknowledge'that is.too 'specific for'theclosed reference testrnode.(ie.,' it isnotrequired. tobe'known'from mernory).

The:question:contains data with anunrealistic level.of accuracy orinconsistentunits-(e.g., panelm.eter in'perc:ent w.ith.question i.n:g'allons).

. . . The4uestion requ.ires reverse logic orapplication compared tothejob re'q'uirements.

5. ; A W for co.nformance with theaptbroved K/A'and those that aredesignated SRO-only (K/A an:d:license level mismeitchesare unacceptable).

-6. .

Enter question'source .(B)an'k;-(M)odi.fi'ed, or'(N)~ew^. Checktha't (M)'odifiedq.u'estions meet'c'riteria ofE.S201Sectio.n D.2.f 7.

B.ased dnthe reviewer's judgment,.ifthe ques'tion a's written (U-)nsatisfeictory (req'uiririgrepair o'r replacerrient),'in need of(E)ditorialenhancement, or(S)atisfactory?'

8 At:a minimum, explairi ariy "U"fatings:(e.g., howthe'Appendix B.p,s'ychometricattribute^s are riot being rnet).'

FormES401.9 ES401 .

:2  :

2. 3.Psychometric:Flaws. 4.JobContent Flaws 5.Other '6.:'.7'. 8'.

~

1.

Q# .LOK LOD

. '. . (F/H); .'(1-5)

.Stem .Clies T/F'. :Cred.Partial Jo.b:- Minuti.a #/.'B'ac'k- Cl=' ^SRO .B/M/N. U/E'/S -

Explana'tion x ;Focus g

^

.jjDist.  :

Link; '. t)nits.

ward K/A; ~Only.  ; .

~

9 . F P .3'A '

x YA N' S '

~1'O'F '

3.5

. y .

.B : 'S.

^

.: x . . . .:  : . . . .

11.^F 3 Y N S

' ~ -

12 F -

4 . =

Y B S 13 H 3.3 Y B S

' ~ '

14 x H 4 '- Y B S 15 H 2.5 - -

Y '

N '

S

-16 F 3.5. y < .. N S 17 -

F '

3.3 =

Y -

B S 18 .H 3.3 - -

Y . .

B .

S 19 H '

3.0 - -

Y '-N -

S

20 F 3.0;

: ;Y :  ;: B 'S -

21 F ' '

2.8~ .

Y -

B . .S 22 F 2.8 Y 'B S 23 H 2.8 '

'.'Y B' S

24 F 3.5 . ..  : '

Yx . . N 'S.

25 H 3.5' Y B '.~-S Y B S 26 F 2.8 . . .

^

27 F 2.8 .' Y N S

-28 'F 2.7 <

,' Y.

. N -

S 29 :P 3.0 Y NA S 30 -

H 1 Y .

'. '.B S

2.'- 3.-PsychometricFlaws 4'.Job.Gont.entFlaws 5.Other 6.' '- 8.

~- '

1. 7 Q# LOK LOD' (F/H-) .(1-5) StemCues- :T/F :Crede Job-.Minutia Partial #/ Back-Q= SROB/M/NU/E/S --

ExpIanation Focus Dist.-

'Link ;

units ward'K/AOnly 31 H 3.7 . -Y B S 32 F 3.3 Y '

N S-

.33 H 3.0 Y B S 34 H 3.8 -

M S.

Y B ^S 35 F 44 -

36 F 2.5 ' '

X .

Y -

'B E~ Choice-B: "inserted..."'notplausible.. Revised.distractor.

37 .

H .3.3 '. .

'Y. . .B S.

38' F 1.0' '

Y . B '

U LOD=1'. $electednewKAandwrotenewQ..

39 -

F 3.0 '. Y . .B S 40 H . 3.0 -

. . Y B . -S 41 F 3.0 Y' B S

~

42 H 4.0- -

Y -

'B -

S' 43 F 3.5 . . . .; . B. .. S '.

44. H 3.0 Y N ~S 45 F 2.5 - -

'Y . B S -

46 H 3.3 . .

Y -

N. . S' . . . .

'47 H~ 3.3 Y S.

N 48 F 3.0' Y .M S 49 F 3.3 -

Y B S 50 H 2.8'- Y B S

~ ~ - '

51 ~H. '2.9. - -

Y -

B .

'S -

52 H 3.5 - - -

Y B S 53 F 3.5 '

'Y B S 54 H 3.0 Y -

B -

S i -

2.. 3.- Psychornetric Flaws 4.Job.Content Flaws 5.Other -6.' 7. 8.

.1'.' . .

Q# LOK' LOD . .

-(F/H). (1-5) StemCues T/F Creid. Partial Job-Minutia #/ Back--Q=. .SROB/M/NU/E/S -

Explan'ation- .

uni$;

~

Focus Dist. Iink ; .ward K/A Only

~ '

55 H 2.8 Y -

B -

S -

56 F -3.0 -

Y ' -

B -

S' 57 .

H 3.0 '

Y -

B S: -

58 -

F 3.0 '

Y .B S 59 H' -

4.0 ~

Y B S 60 H 3.3 -

Y -

^B -

S.

61 H 3.3 . Y . cB .

S H Y B S:

62 2.3 -

63 H 3.0 . Y . . . B :S 64 H 3.0 , Y . B .S' F ~3.0 X Y B U B& C " cannot beoverridden .."~implausible.'Revised

~

65 Choices answer choices-& format.

66 H 3.0 . .

y -

B .S - -

67 -

H 3.3 Y B ~S 68 'F '

3.0 . . y .

B .

S' .

69 F 3.7 -

.  : Y.

. B .

S 70 H. 3.0' .

y . .

.

B -

~.^.S 71 F -

3.5' - - -

. . Y '

B S

.72. H. 3.0. . . . . . Y. . . B. '. S F 3'.0 Y N -- S 73 74 F 3.0:  :  : .Y N S F B N S 75' 3.8 ' ' ' -

=

76 H'- 3.0 -

- ~- -

'.- Y .

Y B' : '.S.

77 F 2.7 Y Y B. S

. .1. 2.-- -

3.'Psychometric Flaws .

4.JobGontent Flaws .-'5. . Other 6 -

'-' 7. .'~-'

.- 8.

-LOD

- ~

Q#.

LOK - - -

(F/H') (1-5).StemC'ues; T/F -Cred. Pahial Job. Minutia  ;#/ Ba'ck Q=j.SR'O B/M/N U/E/S  :  : Explariation 1 Dist.'

Focus' Lirik ; units ward K/A Only

' ' ~

o t

78 H . 3.0 -

Y -B -

S -

79 H 3.0 '

Y Y 'N S 80.' F 3.3 'X ~- Y N N U ROkriowle'dge.th'at No;tSR'O-only: Rxtrip which isreportable, als'o

.makeschoices B'&.C implausible. Revised B & C;.

choices

81 .

H 31.0  ; Y '

Y. . B S

8'2 R ' ~

3.0 . .

Y Y '-B ; S-83: H :3.3 . .

. - V.: B -

S:

84 H 1'.4X '

X '

Y Y B . .'ULOD=1,DWT-P.cu'r.veprovides cues isneedl'ess a'nd info..

. Revised Qstem.

85 H .

. . ^2.0 .

-. Y .

B : :S -

86 H .3.0 Y Y B S H .3.0. -.Y Y ~N S 87 88 F 4.0 Y Y 'N .S'.

89 F. '.3.5 x .  :

- ~ ' -

. V '

Y .

B. :S

90.  : R -

2.3  : -

Y  : .Y.. -

B -

S.:

91 -

. H 4A ;Y Y -

N ;S

92. F 4.0. Y N S'.

Y 93 H~ 17 'Y Y S

~' " - ' ~ - ' ' '

B' V A Y~ :B 94 -

F 2.3 -

A - - -

S:

H Y N 95 2.7 Y- S 96~ H '

3.7. .

Y . 'Y B' ' '

S

.97 'F ' '

2.5 '

Y .

Y' B .'S

.'F 3.0 Y Y N S-98 - -

99 -

F 4'.0 -

Y -

..Y'N S - -

100 H 4.0 Y ~..Y '.B 'S.

. . .