ML110450210
| ML110450210 | |
| Person / Time | |
|---|---|
| Site: | 05000548 |
| Issue date: | 12/10/2010 |
| From: | Garchow S Operations Branch IV |
| To: | Entergy Nuclear Operations |
| References | |
| ES-401, ES-401-9, RB-2010-12 50-548/10-401 | |
| Download: ML110450210 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 RB-2010-12 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 7 OBDI 202 - IOLE Process THIS in explanation means last 2 NRC exams.
THIS in explanation means student handout.
File: DRAFT Written Markup.pdf contains markups.
Review Completed By: Gabriel Apger Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
F 2
B S
A41.7[2008NRC]
2 H
2 B
S B41.7[2007NRC]
3 F
3 B
S A41.7 4
H 1
N S
A41.7 5
F 3
U N
U S
C41.6 See markup file. Answers are true/false statements.
This question was rewritten.
6 F
3 N
S C41.10 Justification for correct answer needs minor edit. See markup file.
ES-401 2
Form ES-401-9 RB-2010-12 Page 2 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 7
H 2
E N
S D41.7.
8 H
2 N
S D41.10 9
F 3
M S
D41.10 Modified condition in stem to elicit different answer than original bank.
10 H
1 U
U N
U S
B41.7 See markup file. A, C and D are not credible due to stem stating that safety buses are energized from offsite. The subject distracters all have a condition where a safety bus is energized from its diesel, which conflicts with the stem. Also, stating the safety buses are energized from offsite would cue that none of the diesels are loaded.
Stem was rewritten.
11 H
2 E
N E
S D41.5 See markup file. A is not credible.
Rewrote distractor A.
12 F
3 B
S B41.10 13 F
2 N
E S
D41.10 Add of at end of sentence in stem.
Edited as indicated.
14 F
3 B
S B41.2 [2003NRC]STUDENT HANDOUT EOP Caution 1 15 H
2 N
S A41.10 STUDENT HANDOUT EOP-1 Fig 2 16 H
3 N
S C41.10 17 H
2 N
S A41.10/6/7 18 F
2 E
N S
D41.7 D is also the longest answer.
19 F
3 B
S B41.4 20 H
1 U
B U
S A41.5 C and D both have two sentences that conflict with one another and can be eliminated early (not credible). Stem supports the answer for Q4 (turbine trip on generator trip).
Distractors C and D were rewritten.
21 H
3 N
S D41.10 22 H
3 N
E S
C41.5 See markup for recommended clarification to stem.
Edited as indicated.
23 H
2 N
E S
B41.10 Need technical reference (i.e., AOP) to support correct answer.
The technical reference was provided.
ES-401 3
Form ES-401-9 RB-2010-12 Page 3 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 24 H
2 N
S B41.11 25 F
2 N
S D41.4 Explanation for B needs to be fixed.
The explanation for B was rewritten.
26 F
2 B
S C41.10 [2004NRC] While the correct answer may seem obvious, a technical reference still needs to be provided to support the correct answer. (NOT INCLUDED) 27 F
2 N
S C41.9 28 F
2 E
N E
S D41.10 Because the dangling modifier which will result in is attached to the subject of lowering RPV pressure below shutoff head, C is also correct. Recommend rewording.
The stem was rewritten.
29 H
3 U
N U
S C41.3 A and B are not credible because the heat load would never be higher after a shuffle.
A and B were rewritten.
30 H
2 N
S B41.8 31 F
2 B
S B41.8 [2007NRC]
32 H
2 B
S A41.6 [2008NRC]
33 H
2 N
S A41.6 34 H
3 N
S A41.7 35 F
2 N
S B41.7 36 H
3 N
S D41.7 37 H
3 N
S C41.7 38 F
3 N
S D41.7 39 H
2 N
S C41.10 40 H
4 N
S B41.7 41 H
2 U
N U
S A41.7/10 The question does not test on an ADS limit or precaution.
After discussion with the plant staff, this question is ok as written.
42 H
3 B
S B41.7 [2003NRC]
43 H
3 N
S D41.7
ES-401 4
Form ES-401-9 RB-2010-12 Page 4 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 44 F
2 N
E S
B41.7 Minor clarification in stem. See markup.
This question was edited as indicated on the markup.
45 H
3 B
S B41.7 [1997NRC]
46 H
3 N
S A41.8 47 H
3 N
S D41.10 48 F
2 E
N E
S C41.4 It looks, by the drawing, that the 500KV yard can supply either the North or South lines via a 230KV feed making D also correct.
The stem was rewritten.
49 H
3 B
S D41.4 [2007NRC]
50 H
2 B
S A41.4 [2007NRC]
51 H
3 N
S A41.8 52 H
3 N
S C41.4 53 H
2 N
S C41.4 54 F
3 N
S D41.6 55 H
2 U
N U
S D41.6 A and B are not credible because the stem states there is a fault in the HCU for B FCV and the B FCV is locked up.
The stem was rewritten.
56 F
2 U
N U
S C41.3 A and D are not credible because they deal with flow, and stem specifically addresses overpressure protection.
The distractors were rewritten.
57 F
3 N
S D41.4 58 H
2 N
S D41.4 59 H
3 M
S A41.4 Changing stem condition changed correct answer.
60 H
2 N
S B41.4 61 H
3 N
S C41.4 62 F
2 N
S A41.4
ES-401 5
Form ES-401-9 RB-2010-12 Page 5 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 63 F
3 U
N U
S D41.4 Choices are T/F style.
After discussion with the plant staff, this question is ok as written.
64 H
3 N
S D41.9 65 F
2 B
S D41.2 66 F
2 N
S D41.10 Meets pedigree of negatively stated question (1021 App B).
67 H
2 B
S B41.5 [2007NRC]
68 F
3 N
S D41.10 Meets pedigree of negatively stated question (1021 App B).
69 F
3 U
N U
S D41.10 Distractor C is also correct.
Rewrote distractors A and C.
70 F
2 M
S A41.10 Meets pedigree of negatively stated question (1021 App B).
71 F
2 N
S D41.10 72 F
4 N
S B41.10 Meets pedigree of negatively stated question (1021 App B).
73 H
2 N
S B41.10 74 F
2 N
S A41.10 75 F
2 N
E S
D41.10 See markup Edited as indicated in markup.
76 H
3 N
S A43.5 77 H
2 N
S C41.7 78 H
2 N
S A43.5 79 H
3 N
S B43.6 80 H
3 E
B E
S B43.5 [2007NRC GGNS] STUDENT HANDOUT EOP Fig 2 HCTL Because there is nothing in the stem that lets the applicant know how fast RPV pressure must be reduced, an applicant could argue C is also correct. Recommend editing stem or distractor to ensure B is only correct.
Rewrote distractor B.
81 H
3 E
N E
S D43.5 STUDENT HANDOUT EOP-1 Caution 1 B not credible with 0psig, steady RPV level way above TAF and RPV blowdown complete.
Rewrote distractor B.
ES-401 6
Form ES-401-9 RB-2010-12 Page 6 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 82 H
2 E
N E
S B43.5 As written, the assumption can be made that before the required boron has been injected, pressure can change, but after the boron has been injected, pressure must be stabilized?? Recommend changing C and D to read maintain pressure stable.
Edited as recommended.
83 H
3 E
N E
B43.5 C not credible. Concern for a loss of feedwater heating with a scram condition present as stated in the stem makes no sense.
84 F
2 N
S C43.1 85 H
3 N
E S
A43.5 STUDENT HANDOUT EOP-0003 Table H See markup.
Edited as indicated.
86 H
2 U
N U
S D43.5 Remove reason from correct answer because it answers question 31.
Removed the reason from the distractors.
87 H
3 N
E S
B43.5 See question 32 for overlap.
This question is ok as written.
88 H
4 N
S D43.2 STUDENT HANDOUT TS 3.3.1.1 89 H
4 N
S D43.5 90 H
3 N
S B43.5 91 H
2 E
N E
S A43.5 B is not credible.
Distractor B was rewritten.
92 H
3 N
S A43.5 93 H
3 N
E S
C43.5 Remove the reasons for procedure entry in part (B) because they conflict with the procedure for choices A and D, making the choices less credible.
This correction was made as recommended.
94 F
2 N
S D43.3 95 H
2 N
S C43.2 STUDENT HANDOUT TR 3.4.13 96 F
2 N
E S
B43.5 Verify OSP-37 is only correct answer.
OSP-37 is the only correct answer.
97 H
3 N
S C43.2 D doesnt seem credible.
Distractor D is ok as written.
ES-401 7
Form ES-401-9 RB-2010-12 Page 7 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 98 F
2 N
S B43.4 99 H
1 U
N U
S B43.5 Knowing the facility is in an SBO, A and D can be easily eliminated. Low discriminatory value.
This question was rewritten.
100 H
2 E
N E
S A43.5 Recommend changing address to resolve because C could be argued as correct, as it indeed addresses invalid annunciators.
Edited as indicated.
RO TOTALS:
B= 16 F= 33 = 44%
E=
Additional Notes:
M= 3 H= 42= 56%
U= 9= 13%
N= 56 SRO TOTALS:
B= 1 F= 4 = 16%
E=
Additional Notes: 2 additional unsats can be corrected with simple edits and are not counted.
M= 0 H= 21 = 84%
U= 2 = 8%
N= 24
GENERAL COMMENT
S:
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
- 2.
Chief Examiner comments are indicated in blue.
- 3.
Average difficulty is __ 2.36__ on the RO exam and 2.52 on the SRO exam.
- 4.
The answer distribution is: RO / SRO A = 17
/
6 B = 18
/
9 C = 15
/
5 D = 25
/
5
- 5.