ML21050A181
| ML21050A181 | |
| Person / Time | |
|---|---|
| Site: | Limerick  |
| Issue date: | 02/19/2021 |
| From: | Brian Fuller Operations Branch I |
| To: | |
| Shared Package | |
| ML18116A036 | List: |
| References | |
| CAC 000500 | |
| Download: ML21050A181 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1. LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q - K/A SRO Only 1
F 2
Y N
S 2
F 2
Y N
S 3
H 3
Y M
S 4
F 2
Y N
E Editorial fix to C, capitalization and extra space 5
F 2
Y B
S 6
H 2
Y N
S 7
F 3
Y B
E Editorial, added the correct to stem 8
F 3
Y N
S Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 9
H 3
Y B
E Editorial, added actual to stem 10 H
2 X
Y B
E Reworded stem to make C plausible 11 F
2 Y
B E
Editorial, reworded distracters to improve answer symmetry 12 F
3 X
Y B
E Editorial to move minimize to stem; improve plausibility of A 13 F
3 Y
N E
Editorial, added per OT-114 to stem 14 H
2 Y
N E
Un-shade area on embedded graph, add F to temps in A&B change C&D temps to improve plausibility 15 F
2 N
B U
K/A MM No link to SP level being low, variables are SP Temp and over-pressure, wrote new Q 16 H
3 Y
N S
17 H
3 Y
N S
18 H
2 Y
M S
2017 19 F
2 Y
B S
20 H
3 Y
N S
21 H
3 Y
N E
Editorial, add rise to stem 22 H
3 Y
N E
Editorial, change occurs to initiates 23 H
3 Y
B S
Change D to 8 vs 16 to improve plausibility of D 2016
ES-401 3
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 24 F
3 Y
B E
Editorial, change Recirc Loop rupture to SBLOCA 25 F
2 Y
N S
26 H
2 Y
B E
Editorial to fix stem 27 F
2 Y
N S
28 H
3 Y
M S
29 H
3 Y
B E
Editorial, move 5 minutes later to beginning of stem 30 F
3 Y
B E
Add for a subsequent HPCI start to the stem 31 H
2 Y
B S
32 F
3 Y
B S
2016 33 H
2 X
Y N
E Reword justification for C 34 F
2 Y
B S
35 H
3 Y
B E
Editorial, add RPS to the A side half scram in 3 spots 2016 36 H
3 Y
N S
37 H
3 Y
B S
38 H
2 Y
N S
Cleanup justification for D 39 H
3 Y
N S
Editorial, add plus sign to levels in stem 40 H
2 Y
B S
ES-401 4
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 41 F
2 Y
N S
42 H
2 Y
B E
Add K/A justification statement 43 H
3 Y
B S
44 F
2 Y
B E
Overlap with Q18, randomly drew new K/A for this system (4.01) 45 H
3 Y
B S
46 H
3 Y
B S
2017 47 F
2 Y
B S
48 H
3 Y
B S
2016 49 H
3 Y
B S
50 H
2 Y
B S
51 H
3 Y
N E
Editorial, added mechanical to stem 52 F
3 Y
N S
53 H
2 Y
B E
Editorial fixes to A, B and C
54 F
2 Y
B E
Editorial, Feed pump Status is as follows, and delete & jet pumps from A and C 2017 55 H
2 Y
B S
56 H
3 Y
N E
Add K/A justification
ES-401 5
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 57 H
3 Y
N S
58 H
2 Y
B S
59 F
3 Y
B S
60 H
3 Y
B E
Pull up first bullet to front of stem 61 H
3 Y
B E
Editorial, change to inches 62 F
2 X
Y N
U Two non plausible distracters, cannot write good q for this k/a.
Randomly drew new system (226001) and K/A (A2.11) for this Q 63 H
3 Y
N S
64 H
3 Y
N S
65 F
2 Y
B S
66 F
2 Y
B E
Editorial, wrote new D 67 F
2 Y
M S
68 H
3 Y
N E
Editorial, wrote new A distracter 69 F
3 Y
B S
70 F
2 Y
B S
71 H
3 Y
B S
2016 72 F
2 Y
B E
Editorial, fix fourth bullet
ES-401 6
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 73 H
2 Y
B S
74 F
2 Y
B S
75 F
3 Y
N S
76 H
3 Y
Y B
S 2016 77 H
3 Y
Y N
S 78 H
2 Y
Y N
E Rework D to improve plausibility 79 H
3 Y
Y N
E Added K/A justification statement 80 F
2 Y
Y B
E Deleted entry conditions from stem 81 H
3 X
Y Y
M E
Rewrote stem to improve plausibility of C 82 H
2 Y
Y N
E Corrected errors in C&D, added procedure titles to stem 83 H
2 Y
Y N
S 84 H
3 Y
Y N
E Modified provided reference to preclude direct look-up 85 H
2 N
Y B
U K/A MM; Two correct answers 86 H
3 Y
Y M
S 87 H
2 Y
Y N
S 88 H
2 Y
Y N
E Rewrote B and D to improve plausibility 89 H
3 Y
Y N
E Rewrote B and D to improve plausibility
ES-401 7
Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 90 H
3 Y
Y N
S 91 H
3 Y
Y N
S 92 H
2 Y
Y M
S 93 H
3 Y
Y B
S 2016 94 F
2 Y
Y B
S 95 H
2 Y
Y B
S 96 F
2 X
Y Y
B U
Distractors A & C not plausible, Rework or write new Q Selected new K/A, wrote new question 97 F
2 Y
Y B
S 98 H
2 Y
Y B
E Rewrote justification 99 F
2 Y
Y B
S 100 H
2 Y
Y B
S