ML18099A091
| ML18099A091 | |
| Person / Time | |
|---|---|
| Site: | Hope Creek  |
| Issue date: | 02/06/2018 |
| From: | Todd Fish NRC Region 1 |
| To: | Public Service Enterprise Group |
| Shared Package | |
| ML17138A098 | List: |
| References | |
| CAC U01958 | |
| Download: ML18099A091 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
- 1. LOK LOD
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cred.
Job-
- /Units Backward Q-KIA SRO Only (B/M/N)
(U/E/S)
Focus Cues T/F Dist Partial Link Minutia 1
F 2.7 y
N s
2 H
3.0 y
N s
3 F
3.1 y
N s
4 F
3.0 y
N s
5 F
3.2 y
B s
6 F
3.0 y
B s
7 H
2.9 y
B s
8 H
2.9 y
M s
Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOO) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
"Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless infomiation).
"Cues": The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
'T/F": The answer choices are a collection of unrelated true/false statements.
"Cred. Dist.": The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
"Partial": One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
"Job Link": The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).
"Minutia": The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
"Backward": The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those KIAs that are designated "SRO-only." (KIA and license-level mismatches are unacceptable.)
- 6.
Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOD
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Link
- / Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S)
Focus Dist Minutia 9
F 3.0 y
B s
10 H
3.3 y
N s
11 H
3.2 y
B s
12 F
3.2 y
B s
13 H
3.1 y
N s
14 F
3.0 y
B s
15 H
2.9 y
N s
16 F
3.5 y
N s
17 H
3.4 y
B s
18 H
3.2 y
N s
19 H
3.2 y
B s
20 F
2.9 y
B s
21 F
2.9 y
B s
22 H
2.9 y
B s
23 F
3.2 y
B s
24 H
3.3 y
B s
25 H
3.0 y
B s
ES-401 3
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOD
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Link Minutia
- I Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S)
Focus Dist 26 F
3.2 y
N s
27 H
2.9 y
B s
28 H
2.9 y
B s
29 H
3.0 y
B s
30 H
3.2 y
B s
31 H
2.9 y
B s
32 F
2.8 y
N s
33 H
3.5 y
B s
34 H
3.5 y
N s
35 H
3.1 y
B s
36 H
3.2 y
B s
37 F
3.0 y
B s
38 H
3.0 y
B s
39 H
3.0 y
N s
40 F
2.9 y
N s
41 H
2.9 y
B s
42 H
3.0 y
B s
ES-401 4
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOO
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cred.
Job-Link
- / Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S)
Focus Cues T/F Dist Partial Minutia 43 H
3.2 y
B s
44 F
3.6 y
B s
45 H
3.4 y
B s
46 H
3.2 y
B s
47 F
3.0 y
B s
48 H
3.0 y
B s
49 H
2.9 y
B s
50 H
3.0 y
B s
51 F
3.0 y
N s
52 H
2.9 y
B s
53 H
2.8 y
B s
54 F
3.0 y
B s
55 H
3.2 y
M s
56 H
3.3 y
N s
57 F
3.1 y
B s
58 F
3.5 y
B s
59 F
3.0 y
B s
ES-401 5
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOD
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cues Cred.
Partial Job-Link Minutia
- I Units Backward Q-KIA SRO Only (B/M/N)
(U/E/S)
Focus T/F Dist 60 H
3.1 y
B s
61 F
3.0 y
N s
62 H
3.0 y
B s
63 H
3.2 y
B s
64 F
3.1 y
M s
65 H
1.0 y
B u
L0D=1; re-selected new KIA, and wrote new Q 66 F
3.5 y
N s
67 H
3.0 y
B s
68 F
3.1 y
B s
69 F
3.2 y
B s
70 F
1.0 y
N u
L0D=1; re-selected new KIA, and wrote new Q 71 F
3.2 y
B s
72 F
3.2 y
B s
73 F
3.0 y
B s
74 F
2.9 y
N s
75 F
3.3 y
N s
76 H
3.6 y
y B
s
ES-401 6
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other a
LOK LOO
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Link Minutia
- I Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S)
Focus Dist 77 H
3.7 y
y B
s 78 H
3.8 y
y B
s 79 F
3.8 y
y B
s 80 H
3.9 y
y M
s 81 H
3.7 y
y B
s 82 H
3.4 y
y N
s 83 F
3.5 y
y B
s 84 H
3.6 y
y N
s 85 H
3.5 y
y N
s 86 H
3.4 y
y B
s 87 H
3.8 y
y B
s 88 F
3.2 y
y B
s 89 H
3.2 y
y B
s 90 F
3.0 y
y N
s 91 H
3.0 y
y M
s 92 H
3.0 y
y B
s 93 F
3.2 y
y M
s
ES-401 7
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws 5.0ther Q
LOK LOO
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Link Minutia
- I Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S)
Focus Dist 94 F
3.3 y
y B
s 95 H
3.4 y
y B
s 96 F
3.1 y
y B
s 97 F
3.2 y
y B
s 98 F
3.3 y
y B
s 99 F
3.6 y
y B
s 100 F
3.6 y
y B
s