ML102980118
| ML102980118 | |
| Person / Time | |
|---|---|
| Site: | LaSalle  |
| Issue date: | 09/24/2010 |
| From: | Mcneil D Operations Branch III |
| To: | Exelon Nuclear |
| Shared Package | |
| ML102510205 | List: |
| References | |
| 50-373/10-301, 50-374/10-301, ES-401, ES-401-5 50-373/10-301, 50-374/10-301 | |
| Download: ML102980118 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- questions reviewed in depth.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
F 1.0 B
U LOD =1: Replaced question.
2 H
2.0 N
E Fix typos in stem first line. Typos corrected.
3 F
1.0 N
U LOD=1 Replaced question.
4 H
2.0 B
S 5
H 2.0 N
S 6
H 2.0 N
S 7
F 2.0 N
S 8
H 2.0 N
S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 9
F 2.0 B
S 10 F
2.0 N
S 11 F
2.0 B
S 12 H
2.5 N
S 13 H
2.5 B
S 14#
H 2.0 N
S 15 H
2.0 B
S 16#
H 2.5 N
S 17 H
2.5 N
S 18 H
2.0 N
S Made stem clarification as a result of re-validation.
19#
F 1.0 N
U LOD=1, check for performance in scenarios - overlap. Replaced question.
20 H
2.0 N
E Suggest using stays the same to get rid of the 1 of 2 taken 2x.
Reworded distractors - 1 of 2 taken 2x removed.
21 H
1.0 N
U LOD=1 Recent modification to the station. Demonstrated weakness in operations. Allowed question to remain on exam.
Revised choices.
22#
H 2.0 N
S 23 H
2.0 N
S 24 F
2.0 N
U LOD=1 Replaced question.
25#
H 2.5 N
E Fix typo in stem lighting strike distractor b. appears to be implausible. The stem says all three phases have the same voltage, the distractor says check the bus before re-energizing. Replaced question.
26 H
2.5 N
S 27 F
2.0 N
S 28#
F 1.0 N
U LOD=1 Replaced question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 29 F
2.5 N
S Verify this is an RO task. Confirmed to be an RO task.
30 F
1.0 N
U LOD=1 Demonstrated plant weakness. 6 of 8 validators missed question. All distractors are plausible under certain circumstances. Retained question.
31#
F 1.0 Y
N U
Placing gadolinium in the top end of the fuel to shape flux is implausible and is contained in 3 distractors. Modified distractors by removing gadolinium from distractors - modified answer.
32 F
2.0 N
S 33 F
2.0 N
S 34#
H 2.0 N
S 35 H
2.0 N
S 36 F
2.5 N
S 37#
H 2.0 N
S 38 F
2.0 N
S 39 H
2.0 N
E Correct spelling in distractor c. - condtion Corrected typo.
40#
F 2.0 B
S 41 H
2.0 B
S 42 F
2.0 N
S Check to see if HIGHER skill question. Different answer for different power level - analysis. Memorized answer. The answer is the same irrespective of power level.
43#
F 2.0 N
S 44 F
2.0 N
S 45 H
2.0 N
S 46#
F 2.0 N
S 47 F
2.0 N
S 48 H
2.0 N
S 49#
H 2.0 N
S 50 H
2.0 N
S 51 H
2.5 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 52#
H 2.0 B
S 53 H
2.0 N
S 54 F
2.0 N
S Check to verify this is RO knowledge at the station. The question is based on an RO objective.
55#
H 2.0 N
S 56 F
2.0 N
S 57 F
2.0 N
S 58#
H 2.5 N
S Clarified stem after re-validation at station.
59 H
2.0 B
S 60 H
2.0 N
S 61#
F 2.0 N
S 62 F
2.0 N
S 63 F
2.0 N
S 64#
H 1.0 N
U LOD =1. By knowing the location of the instrument, two distractors are eliminated. The location of the instrument is, in this case too simple to accept because of the elimination of two other distractors.
The term AND in distractors c. and d. require the operator to know there is no indication on the second panel which makes them plausible and eliminates the simple knowledge of where the instrument is to answer the question. Question retained.
65 H
2.0 N
S Explain the answer. The explanation seems to point at b. as the correct distractor. Corrected answer - distractor b.
66 H
2.0 N
E Correct typo in stem verfied. Is this a good question for an ILT applicant? Corrected typo. Question is based on an RO objective.
67 F
2.0 N
S 68 F
2.0 N
S 69 F
2.0 N
S Replaced question after re-validation. Replacement accepted.
70 H
2.0 B
S Provide additional information to verify the correct answer. The physical location of the monitors cause distractor a. to be correct. Question retained.
71 H
2.0 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 72 H
2.0 N
S 73 H
2.0 N
S 74 F
2.0 N
S Revised distractor after re-validation.
75 F
2.0 B
S 76#
H 2.0 Y
N S
Replaced question after re-validation. Question accepted.
77 H
2.0 Y
N S
Not a higher skill question. Agreed 78 F
2.0 Y
N S
79#
H 2.0 Y
N S
80 H
2.0 Y
N S
Enhanced stem after re-validation. Enhancement accepted.
81 H
2.5 Y
N S
82#
H 2.5 Y
N S
83 H
2.0 Y
N S
84 F
1.0 Y
N U
LOD=1 Question replaced.
85#
F 2.0 Y
N S
86 H
2.0 Y
N E
If the word immediate is used, it becomes RO required knowledge.
Drop the term immediate in the stem. Removed immediate from the stem. Revised distractor.
87 H
2.0 Y
N U
Not SRO level. Answered with system knowledge. Part of the question is answered by system knowledge, however, in order to correctly answer the question, tech spec basis provided the answer - SRO area. Question retained.
88#
H 2.0 Y
N S
89 H
2.0 Y
N S
90 H
2.0 Y
N S
91#
H 2.5 Y
N S
92 H
2.0 Y
N U
Not an SRO question. Question is answered by system and general knowledge. The graph provided is on the QGA flow charts and is interpreted by the SRO. The determination of what should be done is made by the SRO. Question retained.
93 F
2.0 Y
N S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 94#
F 2.0 Y
N S
Stem enhanced after re-validation. Enhancement accepted.
95 F
2.0 Y
B S
96 F
2.0 Y
B S
97 F
2.0 Y
N S
98 H
2.0 Y
N S
99 H
3.0 Y
N S
100 H
3.0 Y
B S