ML15175A057
| ML15175A057 | |
| Person / Time | |
|---|---|
| Site: | Monticello  |
| Issue date: | 06/15/2015 |
| From: | Mcneil D NRC/RGN-III/DRS/OLB |
| To: | Northern States Power Co |
| References | |
| Download: ML15175A057 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- questions reviewed in depth.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 2.5 N
E Verb tense error: The plant was This should be checked throughout the exam. Agreed. Checked entire examination.
2 F
2.0 B
E 3
F 2.0 N
S 4
H 2.0 B
S 5
H 2.0 B
S Do we need to specify a minimum time to stabilize? Not required.
Answer is correct as is without additional restrictions.
6 H
2.0 B
S 7
H 2.0 B
S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A, and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 8
F 2.0 B
S 9
F 2.0 N
S 10 H
2.0 B
S 11 H
2.0 N
S 12 H
2.0 N
S 13 H
2.0 N
S 14#
H 2.0 B
S 15 F
2.0 B
S 16#
F 2.0 x
N S U The question does not match the K/A. Redundant reactivity controls for a BWR refers to SLC. The question should address what happens to the injected boron as water level is reduced. After review of the question and discussion with the licensee, it was determined the question does match the K/A and no changes were necessary. However, it was noted that distractor (d.) was weak and was replaced.
17 H
2.0 B
S 18 H
2.5 N
S 19#
F 2.0 N
S 20 H
2.0 B
U The correct answer is correct. It can be argued that generator output voltage will rise, making distractor (c.) a correct answer. Grid and terminal voltages rise imperceptibly, but they do rise. Replace distractor (c.). Agreed. Distractor (c.) was replaced with an incorrect answer.
21 H
2.0 N
S 22#
F 2.0 B
S 23 F
1.0 x
B U
The obvious choice is to reduce power. Doesnt completely match K/A as the K/A asks for the reasons to reduce power. Agreed.
Distractors were modified to include the reason for reducing power.
24 H
2.0 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 25#
H 2.0 B
E Justification states the correct answer is incorrect. Fix justification.
Fixed the justification.
26 H
2.0 B
S 27 H
2.5 N
S 28#
H 2.0 B
S 29 H
2.5 B
S 30 H
2.5 B
S 31#
F 2.0 N
S 32 H
3.0 N
S 33 F
2.0 B
S 34#
F 2.0 N
E Put the ALTERNATE source can in the stem. Placed common item in the stem.
35 F
2.0 N
S 36 H
2.0 B
S 37#
H 2.0 N
S 38 F
2.0 B
S 39 H
2.0 N
E None of the above? Reworded the distractors to eliminate the none of the above condition.
40#
H 2.0 B
S 41 H
2.0 N
S 42 H
2.0 N
S 43#
F 2.0 B
S 44 H
2.0 N
S 45 F
2.0 B
E The first sentence: The SRVs are designed to prevent exceeding the Reactor Coolant System Safety Limit. is considered training in the question. Delete the statement and change the question stem to read: Which of the following represents the most limiting maximum transient design pressure for the Reactor Coolant System Safety Limit? Agreed. Training material removed from the stem of the question.
46#
F 2.0 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 47 F
2.0 B
S 48 H
2.0 B
S 49#
H 2.5 B
S 50 F
2.0 B
E Training in the question stem. Delete the first sentence and modify the stem to remove the training. Agreed. Removed the training material from the stem of the question.
51 H
2.0 B
S 52#
H 2.0 B
S 53 H
2.0 x
N S
Doesnt match K/A. Discussed question with exam author.
Determined the question addresses the K/A, although not directly.
54 F
2.0 B
S 55#
F 2.0 B
S 56 H
2.0 B
S 57 F
2.0 B
S 58#
F 2.0 B
S 59 F
2.0 B
S 60 F
2.0 N
S Note in reference material states no references provided to applicants during exam. Sheet is provided. Added sheets provided.
61#
H 2.0 B
S 62 H
2.0 B
S 63 F
2.0 B
S 64#
F 2.0 B
S Suggest removing will ISOLATE from the correct answer. This information is provided in the stem. Removed will ISOLATE from the correct answer.
65 H
2.0 B
S 66 F
2.0 N
S 67 F
2.0 B
S 68 H
2.0 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 69 F
2.0 N
S 70 H
2.0 B
S Reference is being provided for the question. Added reference provided to the question.
71 F
2.0 B
S Remove ? from first sentence in the stem. Not a question. Agreed.
Removed ? from the sentence.
72 H
2.0 B
S 73 H
2.0 B
S 74 F
2.0 B
S 75 H
2.0 B
S 76#
H 2.0 x
N U
Question is not SRO Only. Correct answer can be determined without knowing which procedure (SRO part) needs to be entered.
Agreed. Question replaced.
77 H
2.5 x
B S
78 H
2.0 x
N S
79#
H 2.0 x
B S
May not be SRO Only. After discussion with the exam author, it was agreed that the question was an SRO Only question. The correct answer is part of the EOPs and is determined by the SRO while executing applicable steps of the EOPs. Left as-is on the written exam.
80 H
2.0 x
N S
81 F
2.0 x
B S
82#
H 2.0 x
N S
May not be SRO Only. Agreed. Question replaced.
83 H
2.0 x
N S
May not be SRO Only. After discussion with the exam author, it was determined that this question was an SRO Only question.
As the SRO is working through the abnormal procedure, the SRO has to make a decision of when to take actions to respond to the loss of vacuum. This occurs well before the actual set-point memorized by the ROs where immediate actions is taken to trip the turbine. The correct answer comes from a chart executed by the SRO in the control room.
84 H
2.5 x
B S
85#
H 2.0 x
B U
If the procedure being entered is an abnormal/off-normal, ROs are required to know entry conditions for those procedures. Show this is an SRO Only question. Agreed. Question was modified to make it an SRO Only question by going to 1 or 2 taken twice format.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 86 F
2.0 x
B U
Entry conditions into abnormal/off-normal procedures are required RO knowledge. After discussion with the exam author, it was determined this is an SRO Only question at MNGP. The correct answer does not come from entry conditions, but from the subsequent actions in C.4-B.03.03.B. The SRO performs this function at the plant.
87 H
2.0 x
N S
88#
H 2.0 x
N S
89 H
2.0 x
B S
90 H
2.0 x
B S
91#
F 2.0 x
N S
92 H
2.0 x
N S
93 H
2.0 x
N S
94#
F 2.0 x
B S
95 F
2.0 x
N S
Probably not Fundamental. Given a reference and the requirement to determine, it appears to be higher skill. Agreed. Made change.
96 H
2.0 x
B S
97 F
2.0 x
B S
98 H
2.0 x
B S
99 H
2.0 x
N S
Need to see the EOPs being provided with the question. Verified the EOPs do not provide a direct look-up and do not provide information that will eliminate other question distractors.
100 H
2.0 x
B S