ML15244B448
| ML15244B448 | |
| Person / Time | |
|---|---|
| Site: | Duane Arnold  |
| Issue date: | 08/20/2015 |
| From: | NRC/RGN-III/DNMS/MLB |
| To: | Duane Arnold |
| Zoia, C D | |
| Shared Package | |
| ML15118A355 | List: |
| References | |
| Download: ML15244B448 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
F 2.0 B
E Changed stem wording from Recirc loop flow based on facility validation comment to improve clarity (EDITORIAL ENHANCEMENT) 2 H
3.0 N
S 3
H 2.0 N
S 4
H 2.0 N
S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
4.Check questions that are sampled for conformance with the 5.approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 5
H 3.0 X
N U
Facility identified change added a pressure condition to the stem in order to prevent the possibility of a second answer (PARTIAL).
Modified a stem condition to remove wording related to an APRM high flux scram (EDITORIAL ENHANCEMENT).
6 H
3.0 N
S 7
H 3.0 N
S 8
H 2.0 N
S 9
H 3.0 N
S 10 H
2.0 N
S Original question was replaced at facility request. Replacement question stem was revised based upon validation comments to prevent confusion related to rod blocks and annunciators (EDITORIAL ENHANCEMENT).
11 F
2.0 B
S 12 H
3.0 N
E Stem wording changed for clarity; replaced reference to main generator MW with reactor power. Answer and distractors updated accordingly (EDITORIAL ENHANCEMENT) 13 H
3.0 M
S 14 F
2.0 B
S 15 F
3.0 B
S 16 H
3.0 N
E Order of conditions in stem modified based upon facility feedback (EDITORIAL ENHANCEMENT).
17 H
3.0 N
E Wording in stem changed from prior to exceeding to to avoid exceeding at facility request (EDITORIAL ENHANCEMENT).
18 F
2.0 N
S 19 F
2.0 X
N U
Fire alarm added to initial conditions to prevent an NRC-identified condition where the stem conditions would not lead to the correct answer (STEM FOCUS).
20 F
2.0 N
S 21 H
2.0 N
S 22 H
2.0 B
S 23 H
2.0 N
E Removed stem wording of assuming no operator action (EDITORIAL ENHANCEMENT).
24 F
3.0 M
S 25 F
2.0 N
S 26 F
2.0 B
S 27 F
2.0 X
X N
U Stem condition added based upon NRC review to clarify that 6 hour6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> warmup is met to prevent distractor D from being a second correct answer (PARTIAL). Stem condition provided based upon NRC review to clarify that Defeat 9 physically cannot work to prevent distractor A from being a second correct answer (STEM FOCUS).
28 H
2.0 B
S 29 H
3.0 N
S 30 F
2.0 B
S 31 H
2.0 X
N E
Changed the value of -24 to -25 in the answer based upon facility review (EDITORIAL ENHANCEMENT). Changed the second part of distractor D to include prior to the low pressure isolation setpoint to make the distractor plausible (CREDIBLE DISTRACTOR).
32 H
2.0 N
S 33 H
3.0 N
E Added a table containing valve numbers and names for clarity based upon facility review (EDITORIAL ENHANCEMENT). Answer and distractors reformatted for clarity based upon NRC review (EDITORIAL ENHANCEMENT).
34 H
3.0 B
S 35 H
3.0 N
S 36 H
2.0 B
S 37 H
3.0 M
S
38 F
3.0 N
S 39 F
2.0 M
S 40 H
2.0 X
B U
Condition added to stem based upon facility review to prevent D from being a second correct answer (STEM FOCUS).
41 H
3.0 N
S 42 F
2.0 N
S 43 H
2.0 N
S 44 F
2.0 N
E Removed additional wording from stem for clarity based upon NRC review (EDITORIAL ENHANCEMENT).
45 F
3.0 N
S 46 H
3.0 B
S 47 F
2.0 M
S 48 H
2.0 B
E Capitalized the name of the equipment in the stem based upon facility review (EDITORIAL ENHANCEMENT).
49 H
3.0 B
S 50 F
2.0 N
S 51 H
2.0 N
S 52 F
2.0 N
E Unit (psig) moved from stem to individual answers for clarity based upon facility review (EDITORIAL ENHANCEMENT).
53 F
2.0 N
S 54 H
2.0 N
S 55 F
3.0 N
S 56 F
2.0 N
E Wording in stem modified for clarity based upon facility review (EDITORIAL ENHANCEMENT).
57 F
2.0 B
S 58 H
2.0 B
S 59 F
2.0 M
S 60 H
3.0 N
S 61 F
2.0 N
S
62 H
2.0 B
S 63 F
2.0 M
E Additional alarm (Hi Rad) added to stem based upon facility review.
Additional wording (PCIS and MSIVs) added to distractor/answer for clarity (EDITORIAL ENHANCEMENT).
64 H
3.0 B
E Corrected typo where the word alarm was shown in the plural form based upon facility review (EDITORIAL ENHANCEMENT).
65 F
2.0 N
E Fan designators modified for better readability based on facility review (EDITORIAL ENHANCEMENT).
66 F
3.0 N
E Turbocharger inlet temperature value in stem lowered to better align with system precautions and limitations based upon facility comment (EDITORIAL ENHANCEMENT).
67 F
2.0 N
S Tense of the word observes changed for clarity based upon facility review (EDITORIAL ENHANCEMENT).
68 F
3.0 N
S 69 H
2.0 B
S 70 F
2.0 N
S 71 F
2.0 N
S 72 F
2.0 B
S 73 H
3.0 X
M U
Additional information added to stem based upon NRC review to provide the information necessary for the answer to be determined (STEM FOCUS).
74 F
2.0 B
S 75 F
2.0 B
E Additional EOP information removed from flowchart symbol graphic in order to removed unnecessary information from the question (EDITORIAL ENHANCEMENT).
76 H
2.0 X
N E
Removed unneeded information from the stem and reformatted the structure of other stem information for clarity based upon facility review (EDIOTRIAL ENHANCEMENT).
77 H
2.0 X
N S
78 H
3.0 X
N S
79 F
2.0 X
N S
80 H
2.0 X
M S
81 H
2.0 X
B S
82 H
2.0 X
N S
83 H
2.0 X
N S
Question replaced with a new question for the same K/A at facility request.
84 H
3.0 X
M S
85 H
3.0 X
B S
86 H
3.0 X
N S
87 H
2.0 X
N E
Wording in answers and distractors adjusted for clarity based upon facility review (EDITORIAL ENHANCEMENT).
88 H
2.0 X
M S
89 H
2.0 X
N S
90 H
2.0 X
B E
Stem condition modified and additional wording added to stem for clarification based upon facility review (EDITORIAL ENHANCEMENT).
91 H
2.0 X
N S
92 H
3.0 X
B S
93 H
3.0 X
B S
94 F
3.0 X
N S
95 F
3.0 X
N E
Additional wording regarding SRO direction added to the stem and RPV water level band added to the answer for clarity based upon facility review (EDITORIAL ENHANCEMENT).
96 H
3.0 X
B S
97 H
2.0 X
N S
Question replaced with a new question for the same K/A at facility request.
98 F
2.0 X
B S
99 F
2.0 X
B S
100 H
2.0 X
B S