ML14245A555
| ML14245A555 | |
| Person / Time | |
|---|---|
| Site: | Arkansas Nuclear  |
| Issue date: | 07/31/2014 |
| From: | Vincent Gaddy Operations Branch IV |
| To: | Entergy Operations |
| laura hurley | |
| References | |
| 50-368/14-07 50-368/OL-14 | |
| Download: ML14245A555 (5) | |
Text
ES-401 ANO2-2014-07 Written Examination Review Worksheet Form ES-401-9 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 5 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
F 2
X X
N E/S Remove negative from B, add monitoring in stem/Revised 2
H 2
X X
X N
U/S C and D not credible - encompassed in A/Revised 3
H 2
X X
B*
U/S C not credible. 12 GPM is not a SBLOCA (K/A mismatch)/Replaced 4
H 3
X N
E/S Add Rx trip and SI in stem. Remove SPTAs/Revised 5
H 2
X N
E/S Revise A and D to make more credible/Revised 6
H 2
X N
U/S Correct answer is the only credible answer/Revised 7
H 3
X X
X N
U/S B and C not credible (Using SGs in Mode 5?). Possible 2 correct answers/Revised 8
F 2
X N
U/S A and B not credible (CCW temp rising when it loses a major load?)/Revised 9
H 3
N S
10 H
2 X
X B
U/S C and D have stem in answer, eliminating A and B as credible/Revised
Form ES-401-9 Page 2 of 5 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 11 H
3 X
N E/S Add clarifying info in stem/Revised 12 F
2 X
N E/S Strengthen stem. C not credible (replace C)/Revised 13 H
4 N
S 14 F
1 X
B*
U/S Elementary knowledge on how a DC battery works; no discernable value/Replaced 15 H
4 N
S 16 F
3 X
X X
N U/S K/A mismatch. Question also structured such that correct answer is obvious/Revised 17 H
4 N
S 18 H
4 N
S 19 F
2 X
N E/S A not credible as it was already performed in the stem/Revised 20 H
4 X
N U/S A and B both correct as written/Revised 21 F
3 X
X N
E/S Restructure A and B to ensure they are wrong/Revised 22 F
3 N
S 23 H
3 N
S 24 F
2 B
S 25 H
3 X
X B
E/S Too much info in stem; Distractor C has wrong SG in it/Revised 26 F
2 X
X N
E/S Question malstructured/Revised 27 H
4 X
X B
E/S Clarify LOCA area in stem; D not credible (no PZR info in stem)/Revised 28 H
3 N
S 29 F
3 X
X N
E/S reactivity appears in too many distractors/Revised 30 F
3 X
N E/S Revise distractors to make 2X2 format/Revised 31 H
4 N
S 32 F
2 X
B E/S Remove slightly from D/Revised 33 F
2 N
S 34 H
3 N
S 35 H
4 X
X X
N U/S Revise stem and distractors to properly evaluate K/A. Unsat as written/Revised
Form ES-401-9 Page 3 of 5 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 36 F
3 X
X N
U/S Only one answer replaces coolant on a LOCA - all others not credible/Replaced 37 H
2 X
N U/S C and D take no action (not credible) /Revised 38 F
3 B
S 39 H
3 X
N U/S Only correct answer credible - others use MSRs for steam relief on a trip/Replaced 40 H
3 X
N U/S B and D both are correct/Revised 41 H
4 X
N E/S Revise stem to give proper conditions for correct answer/Revised 42 F
2 N
S 43 H
3 N
S 44 H
3 B
S 45 F
2 X
B E/S Add procedure and valve noun name in stem/Revised 46 H
3 B
S 47 F
2 X
B S
48 F
2 B
S 49 H
3 X
N E/S Change 20 seconds to 15 seconds in stem/Revised 50 F
3 X
N E/S Remove unnecessary info in stem/Revised 51 H
2 X
X N
E/S Add only to B, C, and D. Revise stem to ask if Rx would trip/Revised 52 F
3 X
X N
E/S Restructure Q to minimize stem and distractor length/Revised 53 F
3 X
N E/S Change distractors to line up with actual local actions/Revised 54 H
3 X
N E/S Revise stem to provide more specifics on what is happening/Revised 55 F
3 N
S 56 F
2 N
S 57 H
3 B
S 58 F
2 X
N U/S B and D not credible (uncover PZR heaters on a loss of load?) /Revised 59 F
2 X
N E/S Add procedure in stem/Revised 60 H
3 N
S
Form ES-401-9 Page 4 of 5 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 61 H
3 X
N U/S Correct answer only one with boron - makes correct answer obvious/Revised 62 H
2 X
B*
U/S B and C not credible - obvious that Rx will auto trip with no feedwater/Revised 63 H
3 X
B U/S A and B not credible as only C and D refer to quick open signal/Revised 64 F
2 N
S 65 F
2 X
X N
U/S Answer is in stem/Revised 66 F
2 X
N U/S B, C, and D not credible as they could apply to broad areas of the plant/Replaced 67 F
2 X
B*
U/S Correct answer would be correct for any circumstance/Replaced 68 F
2 X
N U/S A and C are both correct/Revised 69 F
2 N
S 70 F
3 X
N U/S A, B, and D not credible ; C only answer that has can be quickly closed in it/Revised 71 H
3 X
N E/S D not plausible (confirm functionality by turning off?) /Revised 72 F
1 X
N U/S No discriminatory value//Revised 73 F
2 X
N U/S A and D are both correct/Revised 74 F
2 X
B*
U/S Distractors structured such that A and D can be immediately eliminated/Replaced 75 H
3 N
S 76 H
3 X
X X
X N
U/S All four answers could be argued correct + K/A mismatch//Revised 77 H
3 X
X N
U/S C and D not credible (discharge fouling causing suction level to decrease?)/Revised 78 H
3 X
X N
E/S Add additional info in stem to ensure correct diagnoses can be made/Revised 79 H
3 X
X N
E/S B not credible (Go to LOCA procedure based on normal ctmt radiation?) /Revised 80 H
3 X
X N
U/S C and D not credible (Maximize blowdown on an ESD event?) /Revised 81 H
3 X
X X
X N
U/S B, C, and D could be argued correct with no RCS leakage info in stem/Revised 82 H
3 X
X N
E/S Add info to make COLSS distractors more viable/Revised 83 H
3 X
N E/S Revise Q to make EOP entry/Revised 84 H
3 X
X X
N U/S A and C are both correct/Revised 85 F
3 X
X N
S
Form ES-401-9 Page 5 of 5 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 86 H
4 X
N S
87 H
4 X
X N
U/S Restructure Q to eliminate reverse logic/Revised 88 H
4 X
N U/S C and D not credible; change to maximize charging flow /Revised 89 F
3 X
X X
N U/S Not enough info in stem to make any answer plausible/Revised 90 F
4 X
N S
91 H
3 X
X N
U/S A and D are both correct/Revised 92 H
3 X
X N
U/S A and C not credible (Go to Loss of SDC AOP with no SDC problems?) /Revised 93 F
2 X
X N
E/S D not credible for the given circumstances/Revised 94 H
4 X
X N
E/S Add Tech Specs in stem and remove 15 minutes from C/Revised 95 H
3 X
B S
96 H
3 X
N S
97 H
4 X
B S
98 F
2 X
X X
N E/S B should be Gas Decay Tank; Restructure stem/Revised 99 F
2 X
X N
E/S Enhance stem to ensure correct answer can be evaluated/Revised 100 F
3 X
X N
E/S Give actual indications in stem that make B the correct answer/Revised
- Unsatisfactory Question that was Previously Used on an NRC Initial Written Examination