ML19052A378
| ML19052A378 | |
| Person / Time | |
|---|---|
| Site: | FitzPatrick  |
| Issue date: | 02/21/2019 |
| From: | Brian Fuller Operations Branch I |
| To: | Entergy Nuclear Operations |
| Shared Package | |
| ML18163A275 | List: |
| References | |
| CAC 0500 | |
| Download: ML19052A378 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1. LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q - K/A SRO Only 1
F 3
Y B
S 2
F 2
Y N
S 3
F 3
Y N
S 4
H 3
Y N
S 5
F 2
Y B
S 6
F 2
Y B
E Change distractor C to improve LOD 7
H 3
Y N
S 8
H 3
Y N
S Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 9
H 3
Y B
S 10 H
3 Y
B S
11 F
2 Y
B E
Editorial 12 H
3 Y
N S
13 H
3 Y
B S
14 H
3 Y
B S
15 H
3 Y
M E
Stem does not give reason for torus lowering, add seismic event to stem 16 H
3 Y
N E
Need irrefutable evidence that level will not be restored. Change bullets 3 and 4 in stem to say no injection sources are available 17 F
2 Y
B S
18 F
2 Y
B E
Changed stem to improve LOD and the plausibility of distracters B and D 19 F
2 Y
B S
20 H
3 Y
M S
21 H
2 Y
B S
22 F
2 Y
B S
23 H
3 Y
M S
24 H
3 Y
N E
Removed values for max safes in stem
ES-401 3
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 25 H
2 Y
N S
26 H
3 Y
N S
27 H
3 Y
B S
28 F
2 Y
B S
29 H
3 Y
B S
30 F
3 Y
M S
31 H
2 Y
B S
32 H
3 Y
M S
33 H
3 Y
B S
34 H
3 Y
N E
Changed wording of distractors to be consistent throughout 35 H
2 Y
B S
36 H
2 Y
B E
Reworded Q(1) and Q(2).
Rearranged answers to improve plausibility 37 H
3 Y
M E
Editorial 38 H
3 Y
B S
39 H
2 Y
M S
40 H
3 Y
B S
41 F
2 Y
N E
Editorial
ES-401 4
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 42 F
2 Y
N S
43 H
3 Y
M S
44 F
2 Y
M S
45 H
3 Y
B S
46 F
3 Y
B S
47 F
2 Y
N S
48 F
2 Y
B S
49 H
2 Y
M S
50 F
3 Y
N S
51 H
3 Y
B S
52 H
2 Y
M S
53 H
2 Y
N E
Editorial 54 F
2 Y
B S
55 H
3 Y
B S
56 H
3 Y
N S
57 H
3 Y
M S
58 H
3 Y
B S
ES-401 5
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 59 F
3 Y
N S
60 H
3 Y
N S
61 F
2 Y
B S
62 H
2 Y
M S
63 H
3 Y
M S
64 H
2 Y
N S
65 F
2 Y
B S
66 F
2 Y
B S
67 F
3 Y
N S
68 F
2 Y
M S
69 H
3 Y
B S
70 F
3 Y
M S
71 F
2 Y
B S
72 F
2 Y
B S
73 F
2 Y
B S
74 H
2 Y
B S
75 F
2 Y
B S
ES-401 6
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 76 H
3 Y
Y B
S 77 F
2 Y
Y M
S 78 H
3 Y
Y B
S 79 H
3 Y
Y B
S 80 H
2 Y
Y B
S 81 H
3 Y
Y N
S 82 H
3 Y
Y B
S 83 H
3 Y
Y N
S 84 H
3 Y
Y B
S 85 H
3 Y
Y B
S 86 H
3 Y
Y N
S 87 H
3 Y
Y B
S 88 H
3 Y
Y N
S 89 H
2 Y
Y N
S 90 H
3 Y
Y N
S 91 H
3 Y
Y N
S 92 H
2 Y
Y M
S
ES-401 7
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 93 F
2 Y
Y B
S 94 F
2 Y
Y B
S 95 H
3 Y
Y N
S 96 H
3 Y
Y B
S 97 H
3 Y
Y B
S 98 H
3 Y
Y B
S 99 H
2 Y
Y B
S 100 F
2 Y
Y B
S