ML14010A035
| ML14010A035 | |
| Person / Time | |
|---|---|
| Site: | Millstone  |
| Issue date: | 10/18/2013 |
| From: | Peter Presby Operations Branch I |
| To: | Dominion Nuclear Connecticut |
| Jackson D | |
| Shared Package | |
| ML13164A263 | List: |
| References | |
| TAC U01874 | |
| Download: ML14010A035 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 1
H 3
X B
u Multiple implausible distractors. Changed stem conditions and reworded distractors.
2 F
2 B
s Re-categorized as memory level question.
3 F
3 N
E Single implausible distractor. Wrote new question.
4 F
2 N
E Re-categorized as memory level question. Minor wording changes in choices.
5 F
2.5 N
E Reworded to move text from choices into question stem.
6 H
2.5 X
N u
Multiple implausible distractors. Modified question.
7 H
2.8 X
X N
u Multiple potentially correct choices. KIA mismatch. Wrote new question.
8 H
3 B
E Modified stem data for the actual TS limit. Adjusted choice wording to match TS wording.
9 H
3 B
E Added explanation of the KIA match. Added component alpha-numeric designator.
10 H
3 M
s Added to justification to explain this is RO level knowledge. Minor wording change.
11 H
3 N
E Re-arranged stem information to align with sequence of choices.
12 H
2.4 B
E Low LOD, weak distractors. Replaced w/another bank Q.
13 H
2.8 N
E Re-categorized as memory level question. Modified stem wording.
14 F
3 B
E Re-categorized as memory level question.
15 F
2.5 X
B E
Single implausible distractor. Re-ordered choices and modified question wording to make distractor plausible.
16 H
3 B
E Reworded to move text from choices into question stem.
Good question but poor match. Per Chief Examiner, tied this Tier 1 EPE 17 H
3 M
E KA with a more appropriate Section 4 generic KA. Minor wording changes to stem and choices.
Page 1
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
i LOK LOD Q#
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q= SRO B/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 18 H
3 N
s 19 F
4 X
X N
u Low operational validity, minutia. Wrote new Q.
20 F
3 B
E Re-categorized as memory level question. Reformatted.
21 H
3 N
E Added to stem as necessary to eliminate possibility of multiple correct answers based on assumptions on conditions not given.
22 H
3 B
s 23 H
3 N
E Modified one distractor to make more clearly wrong.
24 F
3 B
s Added information to justification section.
25 H
3 B
s 26 F
3 X
B E
One distractor helps to answer another Q. Replaced the distractor and made other minor wording changes.
27 F
3 N
E Low plausibility distractors. Modified stem conditions and distractors.
28 F
3 B
E Moved repetitive information from each of the choices to the stem.
29 H
3 B
s Added information to justification section.
30 H
3 B
E Reworded to move information from choices into question stem.
31 H
3 B
s Minor distractor wording changes.
32 F
3 X
N u
Multiple implausible distractors. Reworded to move information from choices into question stem. Modified question and distractors.
33 H
3 B
s I
34 H
3 N
E Question and choices did not align. Modified Q to include control I
system response. Revised to reduce test item word count.
35 H
3 X
N u
KA mismatch. Rejected KIA based on oversampling. Wrote new I
question to match new KIA.
H 3
B s
36 I
--*-L_-
Page 2
ES-401 2
Form ES-401*9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
LOK LOD Q#
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N UIE/S Explanation Focus Dist.
Link units ward KIA Only 37 H
3 B
E Reworded to eliminate ambiguity introduced by use of "only and to improve operational validity."
38 H
3 N
E Reword to move text from choices into question stem and to clarify Q.
39 H
3 B
s Modified Q to increase LOD and provided additional stem conditions 40 H
2 B
E to ensure key answer correct regardless of RCS leakage assumptions.
41 H
3 N
s 42 F
3 B
E Reworded to move text from choices into question stem.
43 H
2 N
E Minor wording changes.
44 H
3 B
s 45 F
3 X
B E
Potential for multiple correct answers. Added to stem or choices to ensure only one correct answer. Changed one distractor 46 H
3 N
s 47 H
3 B
s 48 F
2 X
X B
u Minutia. Lacks operational validity. Does not provide adequate basis for licensing decision. Replaced with a bank Q.
49 F
3 B
E Added information to justification section. Minor formatting changes.
50 F
2.5 N
s Corrected typo.
51 F
2 N
s 52 F
1.5 N
u Low LOD. Wrote a new Q.
53 F
3 N
s 54 H
3 B
s
!dded information to justification section.
L__-L___
L____
Page 3
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
LOK LOD Q#
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO B/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 55 F
2 N
s 56 F
2 N
E KIA match but low operational validity. Re-select ed the KA. Wrote a newQ.
57 F
2 N
E Minor wording changes.
58 F
3 B
E Minor wording changes.
59 F
2 B
E Minor wording changes.
60 F
2 N
E Reworded question to improve operational validity.
61 F
2 N
E Clarified Q. Added additional condition to the stem.
62 F
2 B
s 63 H
3 B
E Minor wording change in a distractor choice.
64 H
3 B
E Added information to justification section.
65 F
3 B
E Replaced two distractors to improve plausibility.
66 F
2 N
s Added information to justification section.
67 H
3 B
E Modified stem to align with all choices.
I 68 F
3 X
N E
Single implausible distractor. Replaced the distractor. Added information to stem. Minor wording changes.
69 F
2 B
E At Chief Examiner's direction, replaced Q with another Q on the same topic that tests a more operationally significant knowledge.
I 70 F
2 M
s I
71 F
1 X
B u
This tests general rad worker knowledge. No requirement to possess 1 a license to have this knowledge. Replaced with a bank Q.
72 H
3 N
s I
73 F
3 B
s i
74 F
2 B
s J
Page4
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
LOK LOD Q#
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO 8/M/N U/E/S Explanation Focus Dis!.
Link units ward KIA Only 75 H
3 B
s 76 H
3 B
s 77 H
4 B
s 78 H
3 X
N u
Not SRO-only. Modified Q to ask forTS basis information.
Low LOD because question is direct lookup with the provided 79 H
1 X
N u
references. Added additional conditions to stem to ensure two separate actions apply.
80 H
3 B
s Minor wording changes.
81 H
3 X
N u
Not SRO-only. Can be answered with RO knowledge. Modified choices to make SRO-only.
82 F
3 N
E Simplistic, leading and memory level. Modified Q and choices.
83 F
3 X
B E
Added to stem to ensure distractor is not partially correct.
84 H
3 X
N u
Not SRO-only. This Question #84 helps to answer Question #21.
Wrote a new question.
85 H
4 N
E Added noun names for the instruments referenced in the question stem.
86 H
2 N
E Question simplistic. At Chief Examiner's direction, reselected KIA and wrote new Q.
87 H
4 N
E Too wordy and choices potentially leading. Reworded the Q.
88 H
3 B
E Reworded stem and choices to improve Q.
89 H
2.2 N
E Weak distractors. Modified Q and choices to improve discriminatory value.
90 F
3 N
E Weak distractors. Modified Q and choices to improve discriminatory value.
91 F
3 X
N u
Not SRO-only. Can be answered with RO knowledge. Wrote a new Q.
Page 5
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
LOK LOD Q#
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- I Back-Q=
SRO 8/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 92 H
3 N
E Modified stem and choices to improve alignment between question and choices.
93 H
3 B
E Reworded to ensure no cueing in stem.
94 H
2 B
s 95 H
3 B
E Minor rewording to improve distractor balance and align answers with the question.
96 H
3 N
s Added information to justification section to explain the SRO-only nature of the question.
97 F
2 N
s 98 H
3 N
E Modified question to ensure more clearly testing SRO-only knowledge.
99 H
4 B
s 100 H
3 N
E Minor wording changes.
1...___~-
Page 6