ML11112A066
| ML11112A066 | |
| Person / Time | |
|---|---|
| Site: | Point Beach  |
| Issue date: | 04/25/2011 |
| From: | Mcneil D Operations Branch III |
| To: | Florida Power & Light Energy Point Beach |
| References | |
| Download: ML11112A066 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- questions reviewed in depth.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 3.0 N
E Change shutdown to idle in distractors. Agreed. Made change.
2 H
2.5 N
S 3
F 2.0 N
E Add the in distractor a. and inventory in distractor c. Agreed. Made changes.
4 H
2.5 N
S 5
H 2.5 N
E Insert and actuate the 1P-1A RCP Cooling Water Flow Low Alarm on 1C-03 to distractor a. Insert 1C in 1C-03 in distractor c. Change to HIGHER cognitive skill. Agreed. Made changes.
6 H
2.5 N
S 7
H 2.5 N
E Change has recently been to is; Third bullet, add in MANUAL, add to the stem Assume NO operator action; change distractor b.
to NRHX Outlet Temperature High:; Correct 1P-1B in distractor c.
Agreed, made requested changes.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 8
H 2.0 N
E Delete next from the stem. Agreed. made the change.
9 F
2.0 B
E Add Unit 1 to the stem. Change rating from HIGHER to FUNDAMENTAL. Agreed. Made the change.
10 H
2.5 N
E Add a bullet: Safety Injection has been reset; in distractor d., add one or more to clarify where you are in the procedures at this point.
Agreed. Inserted new bullet.
11 H
N U
Question is SRO level. Replace the question. Agree. Replaced question with a bank question with the same K/A.
12 F
2.0 N
E Fix plant condition bullets and stem. Revise distractor a. so it is correct. Edit distractors b. c. d. Agreed. changes made 13 F
2.0 N
S 14 F
2.0 N
E Replace TSC with Operating Supervisor. Agreed. Made changes.
15 H
2.0 N
E Replace the two with both in the stem, insert the after on.
Agreed. Changes made.
16 F
2.0 N
S Change FUNDAMENTAL to HIGHER. Agreed. Made change.
17 H
2.5 N
E Correct title of ECS-1.1.in the stem and distractor a.. Agreed. Made changes.
18 H
2.4 N
U The question is an SRO level question. Replace the question..
Agreed. Replaced the question with a new question with the same K/A.
19 H
2.0 N
S 20 H
2.0 N
S 21 H
2.5 N
E Replace steady with stable. Agreed. Made replacement.
22 H
2.0 B
S 23 F
2.0 N
S 24 H
2.0 N
S 25 F
2.0 N
E Change stem to say CSP-C.2 directs reducing RCS pressure to inject SI accumulators by. Change Open Both to opening both, and depressurize all to depressurizing all. Change at to less than to remove confusion. Agreed. Incorporated changes.
26 F
2.0 N
E Replace the question with the one provided. Point Beach specific question. Agreed. Replaced question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 27 F
2.0 N
S 28 F
2.0 N
U Question was double jeopardy with question # 001. If you didnt know the answer to question 001, you will probably miss this question also. Agreed. Replaced the question with a new question with the same K/A statement.
29 F
2.0 N
E Delete acceptable, replace with procedurally allowed. In distractor b.,
replace temperature values with ones that are not at the setpoint.
Agree. Inserted changes.
30 H
2.0 N
S 31 F
2.5 B
S Change stem so procedure in effect is noted. Add mini to recirc valv in distractor a. Agreed. Changed stem.
32 H
2.5 N
S 33 F
2.0 B
E Change locked out to deenergized - point beach language. Agreed.
Made the change.
34 F
2.0 B
S 35 F
2.0 N
S 36 H
2.0 B
U Question was double jeopardy with other CCW questions. Replace the question. Agreed. Modified the question. Retained the same K/A.
37 H
2.0 N
S 38 F
2.0 N
E Bold NOT in distractors c. d. Agreed. Made format change.
39 H
2.0 N
S 40 H
2.0 N
S 41 F
2.0 N
S 42 H
2.0 N
S 43 F
2.0 M
S 44 H
2.0 N
S 45 F
2.0 N
S 46 F
2.0 B
E Distractor d. is implausible. Re-work distractor d. Agreed. modified distractor d.
47 F
2.0 N
E Change In/Out to Charge/Discharge in #2. Agreed. Made change.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 48 H
2.0 N
U Multiple implausible distractors. Re-write the distractors. Unable to write the question such that it is at the RO level. Selected new K/A - developed new question.
49 H
2.5 N
S 50 F
2.0 N
S 51 H
2.0 N
S 52 H
2.5 B
S 53 H
2.0 N
S 54 F
2.0 N
E Fix verb tenses in the distractors to match the queston. Agreed. Re-wrote the distractors to be current tense.
55 F
2.0 B
S 56 H
2.5 N
U Replace question using Point Beach definitions and procedures.
Agreed. Replaced question.
57 H
2.5 N
E Bulletize the information in the stem. Add a statement that the control rods are in automatic. Make other minor adjustments for readability. Agreed. Made the applicable changes.
58 H
2.0 N
E Add the direction of the failure (low or high). Agreed. Incorporated changes.
59 F
2.0 N
S 60 F
2.0 B
E Change the question stem to read In order to start the Agreed.
Made word changes.
61 F
2.0 N
E Change CCX to CCW in last bullet. Agreed Change 62.25 feet to 63 feet in the correct answer. 62.25 feet is the setpoint and an operator may or may not stop fuel movement. Corrected typo.
62 H
2.5 B
E Typo. Change affect to effect. Agreed. Corrected typo.
63 H
2.0 B
S 64 H
2.0 B
E Remove the 3 from 2272 in distractor c. Agreed. Corrected typo.
65 H
2.5 N
E Place the question in 2 column format for readability. Agreed.
Placed answers in 2 column format.
66 H
2.0 B
S 67 F
2.0 B
S 68 F
2.0 B
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 69 H
2.5 B
S 70 H
2.5 B
S 71 F
2.0 B
S 72 F
2.5 B
S 73 F
2.0 B
S 74 H
2.0 B
S 75 F
2.0 B
S 76 H
2.5 x
B S
77 H
2.0 x
B E
Un-bold the correct answer (d.). Agreed. Fixed formatting.
78 H
2.0 x
N E
Re-work the correct answer to Point Beach procedures. Agreed.
Re-worked the distractor.
79 H
2.0 x
N E
Re-word the stem for clarity. Specify that actual bus voltage remains normal and that the relays sense the undervoltage condition due to mis-calibration. Add automatic actions to distractors. Agreed.
Made the specified changes for clarity and automatic actions.
80 H
2.5 x
N S
81 H
2.5 x
B E
Distractor c. is implausible. Agreed. Replaced distractor.
82 H
2.5 x
N E
Specify Unit 1. Agreed. Specified unit, modified the stem.
83 H
2.5 x
N S
84 H
2.0 x
N U
Question is LOD=5 as written. Suggest removing AOP-10 from this question distractors and focus the question more on the K/A AOP for plant fire on site use with EOPs. Agreed. Replaced and/or modified all distractors. Provided additional guidance in the stem.
85 H
2.0 x
N E
Modify the plant conditions (bullets) to clearly set plant conditions.
Add reasons to distractors. Agree. Made requested changes.
86 F
2.0 x
B S
87 H
2.0 x
B S
88 H
2.5 x
N S
89 H
3.0 x
N S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 90 H
2.0 x
B S
91 H
2.5 x
N S
92 H
2.5 x
B S
Facility requested we replace the question based on difficulty level.
Upon review it was determined that the question needed modification to clarify the question.
93 H
2.0 x
N S
94 F
2.0 x
N S
95 F
2.0 x
B S
96 H
2.5 x
B S
97 H
2.5 x
N E
Re-format the stem with bullets. Agreed. Reformatted stem.
98 H
2.5 x
N S
99 F
2.0 x
B E
Add responsible to the stem. Agree. Added to the stem.
100 F
2.0 x
B S