Text

Draft - NRC Comments
	ML022970109
Person / Time
Site:	Peach Bottom
Issue date:	09/23/2002
From:	Alfonso M; Exelon Nuclear
To:	Conte R; NRC/RGN-I/DRS/OSB
	Conte R
References
	50-277/02-302, 50-278/02-302
	Download: ML022970109 (10)
	v • d • e

PEACH BOTTOM INITIAL EXAM 2002 ES-401 Written Examination Form ES-401-9 Review Worksheet

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD T.......

(F/H)

(1-5)

Stem jCuesl TIF Cred. jPartial Job-Minutia B

1Back-Qr,SRO U/E/S Explanation

= ~.~*Focus

~

jDist.J Link junits ward KAJny 1

F 3

S 2

F 2

S 3

F 2

S 4

H 2

S 5

F 2

S 6

H 3

S 7

H 2

S 8

H 2

S 9

H 3

E 10 H

3 S

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/R)

(1-5)FStemCues T/F Cred. lPartial Job-i Minutia

/

Back-Q' SROUIEIS Explanation Foc(st I T Dist.

Link units ward K/A Only Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1.

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2.

Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3.

Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

More than one distractor is not credible.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4.

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5.

Check the appropriate box if the sampled question does not match the approved K/A or an SRO-only question is not at the SRO level.

6.

Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7.

For any "U" ratings, at a minimum, explain how the Appendix B psychometric attributes are not being met.

NUREG-1021, Revision 8, Supplement 1 44 of 46

ES-401 2

Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues IT/F Cred. IPartial Job-Minutia

/ Back-Q x SRO U/,/S Explanation Focus Dist.

Link units ward K/A IOnly 11 F

3 S

12 H

2 S

13 F

3 E

14 H

3 S

15 H

3 S

16 F

3 S

17 H

2 S

18 F

2 S

19 F

3 S

20 H

2 S

21 H

2 S

22 F

3 E

23 F

3 S

24 H

2 E

25 H

3 S

26 H

3 S

27 H

2 E

STEM 28 F

2 S

29 H

3 S

30 F

2 S

NUREG-1021, Revision 8, Supplement 1 45 of 46

ES-401 2

Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues ITIF Cred. ~Partial Job-Minutia

/11I ack-Qý' SRO U/E/S Explanation Focus Dist.

Link u

ward nKA lOnly 31 H

3 S

32 F

3 X

U DISTRACTOR D COULD BE CORRECT ANSWER ALSO CHANGED STEM ADDED THE WORD"FIRST" 33 F

3 S

34 F

2 S

35 F

3 E

36 H

2 S

37 H

3 S

38 F

3 S

39 H

3 S

40 F

2 S

41 H

3 S

42 F

2 S

43 F

3 S

44 H

2 E

STEM DISTRACTORS C AND D REVISED 45 F

2 S

46 F

3 S

47 H

2 S

48 F

3 S

49 H

2 X

X E

STEM "ALL RUNNING", DISTRACTOR C SAYS 'STANDBY" REVISED 50 2 2 S

NUREG-1021, Revision 8, Supplement 1 45 of 46

ES-401 2

Form ES-401-9

1.

2.

3. Psychometric Flaws 4.Job Content Flaws 5.,Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Tcues/TF Cred. Partial Job-Minutia 1 #/

Back-Q SRO U/E/S Explanation Focus I Dist.

Link I

unitsl ward K/Al Onlyj 51 F

3 S

52 F

2 S

53 F

3 E

54 H

3 E

55 H

2 E

56 H

3 S

57 F

3 S

58 H

2 S

CURVES AVAILABLE YES 59 H

2 S

60 F

2 E

STEM 61 F

2 E

STEM 62 F

2 S

63 F

3 S

64 H

3 S

65 F

2 S

DISTRACTOR D NEEDS REVISED 66 H

3 E

DISTRACTOR D 67 F

2 X

X U

EASILY ELIMINATE DISTRACTORS A & B REVISED 68 H

3 S

69 F

2 S

70 F

2 S

NUREG-1021, Revision 8, Supplement 1 45 of 46

ES-401 2

Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD

]

T t

(F/H)

(1-5)

Stem Cuesl TIF cred Parial Job-1Minutia

1 jBack-Q ISRo U/I Explanation Focus Dist.

Link units I ward K/A Ionly 1_1 71 H

3 S

72 H

3 S

73 F

2 S

74 F

2 S

75 F

2 X

S.

76 H

3 S

77 F

3 S

78 F

2 S

79 F

3 S

80 H

2 S

81 F

3 E

DISTRACTOR'S C & D OK AS IS 82 H

2 S

83 F

2 S

84 F

2 E

85 F

3 S

86 H

3 S

87 F

2 S

88 H

3 S

89 F

3 S

90 H

3 S

NUREG-1021, Revision 8, Supplement 1 45 of 46

ES-401 2

Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Cred. Partial Job-Minutia

/ IBack-Q' SRO U/E/S Explanation Focusj Dist.

Link units ward K/A Only 91 H

3 E

92 F

3 E

93 F

3 S

94 H

2 X

E 95 H

3 E

DISTRACTOR A REVISED 96 H

2 X

E STEM REVISED 97 F

2 E

98 F

2 E

CAPS? REVISED 99 F

2 S

100 H

3 S

SRO ONLY QUESTIONS 76 H

3 S

77 H

2 E

DISTRACTORS C AND D REVISED 78 H

2 E

79 H

2 S

80 H

2 S

81 F

3 S

82 H

3 S

83 H

2 E

CLEAN UP THE STEM REVISED 84 H

2 S

85 F

3 NUREG-1021, Revision 8, Supplement 1 45 of 46

ES-401 2

Form ES-401-9

1.

2.

3. Psychometric Flaws

4. Job Content Flaws

5. Other 1 6.

7.

Q#

LOK LOD (F/H)

(1-5)

Stem Cues T/F Cred. Partial Job-Minutia

/

Back-SR U/E/S Explanation t Focus Dist.

Link units ward K/A _Onlyl 86 F

2 S

87 H

2 S

88 H

2 S

89 F

2 S

90 H

2 S

91 H

2 S

92 H

2 S

93 F

3 S

94 F

2 X

E A AND B POSSIBLE CORRECT ANSWERS REVISED STEM 95 H

2 S

96 F

2 S

97 F

3 E

98 F

2 S

99 F

2 S

100 F

2 S

NUREG-1021, Revision 8, Supplement 1 45 of 46

Record of Reiected K/As Form ES-401-10 Tier / Group Randomly Selected K/A Reason for Rejection

3 SYSTEM K/A IS OK BUT TEST ITEM # ISN'T CORRECT I

I IA I

1.

1 I~

4

_I_

4 4

-I.

I i

_ I I-I i-I i

I I

I.

I I.

i 4

4-4 I-I

+

1

.1.

I 4

I i

I I

___I NUREG-1021, Revision 8, Supplement 1 1_Qz.A01

1.,..Z--r Recor of e

e 46 of 46

ML022970109

Text

Navigation menu

Search