ML110260293
| ML110260293 | |
| Person / Time | |
|---|---|
| Site: | Monticello  |
| Issue date: | 12/17/2010 |
| From: | Mcneil D Operations Branch III |
| To: | Northern States Power Co, Xcel Energy |
| References | |
| Download: ML110260293 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- questions reviewed in depth.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 2.0 B
S 2
H 2.0 N
E The correct answer is c. The explanation states distractor c. is incorrect. Provide correct answer. c. is the correct answer.
3 H
2.0 N
S 4
H 2.0 N
E The proposed correct answer is d. The justification shows distractor
- d. is an incorrect answer. Provide correct answer. d. is the correct answer.
5 F
2.0 B
S 6
F 2.0 N
S 7
F 2.0 N
S 8
F 2.0 N
S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 9
H 2.5 N
S 10 F
2.0 N
S 11 H
2.0 N
S 12 H
2.0 B
S 13 H
2.0 B
U S This question may be at the SRO level - his responsibility to make decisions/determine what to do. Operations confirmed that this is an RO task and expectation at the Monticello Station.
14#
H 2.0 N
S 15 H
2.5 N
S 16#
H 2.0 N
S 17 H
2.0 N
S Not a HIGHER question. Station agreed - Fundamental.
18 F
2.0 N
S 19#
H 2.0 N
S 20 H
2.0 B
S 21 H
2.0 B
S 22#
H 2.0 B
S 23 F
2.0 N
S 24 H
2.0 N
S 25#
F 2.0 B
E The stem verbage may be mist translated when referring to FIRST.
It may be better to say which is the LOWEST classification. Station agreed, made stem modification.
26 F
1.0 x
B U
LOD=1, Distractors b, and d. are implausible. Station agreed.
Revised question. New question was satisfactory.
27 H
2.0 B
S 28#
H 2.0 B
S Probably not a higher level of knowledge. One only needs to know that one division of ECCS will maintain adequate core cooling.
Station agreed - changed to Fundamental skill level.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 29 H
1.0 x
B U S The question does not discriminate. The incorrect distracters are not plausible. All of the valves are called out in the procedure, making them plausible distracters. Because of importance level at Monticello, question was accepted.
30 F
2.0 B
U S May be two correct answers. Wire draw is a concern anywhere high pressure steam crosses metal. Distractor d. valve is fully open - it is not the throttle/governor valve. Distractor d. is incorrect.
Minor modification made to the stem.
31#
H 2.0 N
S 32 F
2.0 B
S 33 H
2.0 N
S 34#
H 2.0 B
S 35 H
2.0 N
S 36 H
2.0 N
S 37#
H 2.0 B
S Operability determinations are SRO questions. Reference is from a system procedure. Required RO knowledge.
38 F
2.0 B
S 39 F
1.0 B
E Reformat at higher skill level. Reworked question.
40#
F 1.0 B
E Reformat at higher skill level. Reworked question.
41 H
2.0 N
S 42 F
2.0 N
S Not Fundamental knowledge. Station agreed Higher skill.
43#
F 2.0 N
S 44 F
2.0 N
S 45 H
2.0 B
S 46#
F 2.0 B
S 47 H
2.0 B
S 48 H
2.5 B
S 49#
H 2.5 N
S 50 F
2.0 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 51 H
2.5 N
S 52#
H 2.0 B
S 53 F
1.0 B
E Rework to a higher skill. Reworked question.
54 F
2.0 B
S 55#
H 2.0 N
S 56 F
2.0 B
S Probably not fundamental. Station agreed. Higher skill question.
57 H
2.0 N
S 58#
F 2.0 N
S 59 F
2.0 B
S 60 H
2.0 N
S 61#
H 2.0 B
S 62 F
2.0 B
S 63 F
2.0 N
S 64#
H 2.5 N
S May be an SRO task. Station confirms this is an RO task.
65 F
2.0 B
S 66 F
2.0 B
S 67 F
2.0 B
S 68 H
2.0 B
S May not be higher skill. Math included with the question. Makes question a higher skill question.
69 F
2.0 B
U S SRO task to declare operability. Operability is not declared, but recognized, which is an RO skill.
70 H
2.0 B
S 71 F
2.0 B
S 72 F
2.0 N
S 73 H
3.0 B
S 74 H
2.0 N
S 75 F
2.0 B
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 76#
H 2.5 x
N S
77 H
2.0 x
N S
78 H
2.0 x
N S
79#
H 2.0 x
B S
80 H
2.0 x
B S
81 F
2.0 x
N S
82#
H 2.0 x
N S
83 H
2.0 x
B S
84 H
2.0 x
N S
85#
F 2.0 x
B S
86 H
2.0 x
N S
87 H
2.0 x
N S
88#
H 2.0 x
N S
89 H
2.0 x
B S
90 H
2.0 x
N S
91#
H 2.0 x
B S
92 H
2.0 x
N U
Not an SRO level question. ROs must be able to recognize what general steps are taken in a procedure. Station agreed - rewrote question.
93 H
2.0 x
N S
94#
H 2.0 x
B U
Not an SRO level question. ROs must be familiar with the plant process computer and especially the SPDS displays. Station agreed, question rewritten.
95 H
2.0 x
N S
Not a higher skill question. Must be memorized. Probably not an SRO Only question. ROs should know who should be watching a reactivity change so they dont start without proper supervision. This is an SRO responsibility at Monticello. SM makes determination of who should be at the reactivity change.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 96 H
2.0 x
N S
97 F
2.0 x
N S
98 H
2.0 x
N S
99 H
2.0 x
B U S Not an SRO level question. Doesnt specify a procedure. ROs are required to know what should be next. EOP is provided and SRO must determine next step to execute. Question is an SRO level question.
100 H
2.0 x
B S