ML13308A636
| ML13308A636 | |
| Person / Time | |
|---|---|
| Site: | Duane Arnold  |
| Issue date: | 04/08/2013 |
| From: | NRC/RGN-III |
| To: | Duane Arnold |
| Randal Baker | |
| Shared Package | |
| ML11354A335 | List: |
| References | |
| Download: ML13308A636 (10) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 1 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1+
F 2
X Y
N U
S Why are distractor A & C plausible? Why would an applicant choose
... when both valves begin to open...?
1 out of 2 taken twice.
RESOLUTION: Agreed question was low LOD but credible.
2 F
2 X
Y N
E S
Stem potentially cues applicants to focus on interlocks; question should only ask... will not open?
NOTE: Explanation for C distractor discusses Torus spray vice cooling.
RESOLUTION: Change made 3+
F 3
X Y
B E
S NRC: Used on NRC 2011 Exam-Duane Arnold (#4)
Stem should simply state... with power being supplied from the preferred source. vs 1B33 to eliminate potential cueing.
What is the normal position of HSS-3002? If Primary is also normal, then distractor D may not be plausible.
RESOLUTION: Question completely reworded.
4 H
3 Y
B E
S Does this question provide cueing for question #11, or vice versa, concerning the 24 V DC power to the IRMs?
RESOLUTION: Question #4 was replaced
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 2 of 10 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 3 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 5+
H 3
X Y
N E
S Why is distractor B plausible? Why is the second part of distractor C plausible? Should the term spray cooling be replaced with core cooling? (Also replace in other distractors.)
RESOLUTION: The question was replaced.
For the new question consider adding without first depressing the CORE SPRAY INITIATION SEALED-IN reset pushbuttons. To A.
RESOLUTION: Change made.
6 H
3 X
Y N
E S
Will the rod block cause the withdraw permitted light to go out? If so, then C is not a correct answer. May want to add this condition to the correct answer, and consider adding the word Only to distractor D for symmetry.
RESOLUTION: Changes made.
7+
H 3
X Y
N U
S Why are distractors A & B plausible? Is there a Slow Transfer associated with any of the transformers?
Why would an applicant believe a transformer lockout would lockout an ESF bus?
RESOLUTION: Distractor changed.
8 F
3 Y
N E
S If a deluge actuates wouldnt you also dispatch the fire brigade?
Should that be added to distractors A & B?
RESOLUTION: Change made.
9 H
2 N
N E
S The question as written is primarily a fundamentals question, and does not address the KA for how is TURBINE speed measured.
RESOLUTION: The question was corrected with a simple change to refer to turbine rather than pump speed and referring to the speed indicator.
10 F
2 Y
B*
E S
NRC: Used on NRC 2007 Exam-Duane Arnold Consistent use of capitalization required for the word AND in distractors C & D RESOLUTION: Done
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 4 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 11+
H 2
X Y
N E
S Does this question provide cueing for question #4, or vice versa, concerning the 24 V DC power to the IRMs?
RESOLUTION: Question #4 was replaced 12 H
3 X
Y B
E S
Is distractor C plausible? Would this be more plausible (only 2 valves have alternate power) if the 2 valves are both LLS, 4401 & 4407?
RESOLUTION: Changed distractor C.
13 H
2 Y
B S
14+
H 3
Y B*
S NRC: Used on NRC 2005 Exam-Monticello 1 out of 2 taken twice 15 H
3 Y
B S
1 out of 2 taken twice 16 F
2 Y
N S
1 out of 2 taken twice 17 F
2 Y
B S
18 F
2 Y
B S
Not very discriminating in that if the candidate knows that SPDS must always be displayed, the answer is very simple.
19 H
3 Y
N S
1 out of 2 taken twice 20+
F 2
Y B*
S NRC: Used on NRC 2007 Exam-Susquehanna 1 out of 2 taken twice 21 H
3 Y
B*
S NRC: Used on NRC 2007 Exam-Duane Arnold 1 out of 2 taken twice 22 F
3 Y
N S
23 H
2 Y
B*
S NRC: Previously Used on an exam? No.
24+
F 3
Y B
S
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 5 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 25 H
2 X
Y B
E S
Is Distractor B plausible? Perhaps swap inboard and outboard MSIV between the second parts of distractors A & B.
1 out of 2 taken twice?
RESOLUTION: Agreed to accept distracters as plausible without change.
26 F
2 Y
B S
1 out of 2 taken twice 27 F
2 Y
B S
28+
F 1
Y N
U S
Simple power supply question is considered very low level of difficulty. Is there an alternate power supply to these valves?
RESOLUTION: Question replaced.
29 H
2 X
Y N
U/E S
Is throttling the HX RHR outlet valve plausible; would use of the RHRSW outlet valve be more plausible?
1 out of 2 taken twice RESOLUTION: Accepted as is because procedure does discuss throttling the HX RHR inlet valve.
30 F
3 Y
B S
Typo in explanation of Bank question 31 H
2 Y
B S
1 out of 2 taken twice 32 F
2 Y
N S
33+
H 2
Y B*
S NRC: Used on NRC 2008 Exam-Brunswick 1 out of 2 taken twice 34 H
2 X
Y N
E S
Are distractors C & D partly correct? If Offgas isolates, will the conditions given in the stem exist? (Delete < 270 psig to Jet Compressor)
RESOLUTION: Jet Compressor pressure in distractors C & D increased to 290 psig.
35+
H 2
X Y
B*
E S
NRC: Used on NRC 2005 Exam-Oyster Creek Why are distractor C & D plausible? (Why ROD DRIFT alarm; or RWM ROD BLOCK if RWM is always bypassed for rod testing?)
RESOLUTION: The question was replaced.
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 6 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 36 H
3 Y
B S
37 F
2 Y
N S
38 H
3 Y
B*
S NRC: Used on NRC 2011 Exam-Pilgrim 39+
F 2
Y B*
S NRC: Used on NRC 2010 Exam-Nine Mile Point 2 40 H
3 Y
N S
The scale of the graph provided in the exam is too small to be accurately read. Enlarge the insert.
41 H
2 N
N U
S Selecting the correct level band appears to be based more upon a concern with subsequent recirc pump operations than it does with temperature stratification. Need to show correlation to the KA.
RESOLUTION: Explanation of correlation to the KA provided.
42 H
2 Y
N S
1 out of 2 taken twice 43 F
2 Y
N S
44+
F 2
X Y
N U/E S
As written, distractors C & D do not appear to be plausible; need to change the description of the portable diesels to match actual possibilities. (i.e., using staged portable cables in the East Warehouse.)
1 out of 2 taken twice RESOLUTION: The question was replaced.
45 H
2 Y
N S
1 out of 2 taken twice 46 F
2 Y
B S
This question directly relates to the cautions observed when performing SIM-JPM S-1??
RESOLUTION: The question was replaced.
47 H
2 X
Y N
S Should this say following a large break inside primary containment.?
RESOLUTION: Stem change to make it clear a RCS break occurred.
48 H
2 Y
N S
Recommend re-sequencing distractors to B, C, D, A (shortest to longest) and then re-number; this will make the correct proposed answer C.
RESOLUTION: Done
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 7 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 49+
HF 3
Y N
S This appears to be a memory question. Remembers either division of EOC-RPT trips both pumps.
50 F
2 X
Y N
U S
C&D not credible. It is not creditable that we would continue to operate a main transformer after it had been on fire.
RESOLUTION: Question replaced.
51 H
3 Y
N S
52 H
4 Y
N S
53 H
2 Y
N S
54+
FH 3
Y N
S This appears to be a higher order question.
55 H
3 Y
N S
56 H
3 Y
B S
57 H
3 Y
N S
58+
F 3
Y N
S 59 F
3 Y
N S
60 H
3 Y
B S
61 H
3 Y
N S
62 H
4 Y
N S
63+
H 3
Y N
S 64 H
4 Y
N S
65 H
3 Y
N S
66 F
2 Y
B S
67+
F 2
Y B
S 68 F
3 Y
N S
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 8 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 69 F
3 Y
N S
70 F
3 Y
N S
71+
F 3
X N
N U
S KA problem, knowing that you can enter the containment with the reactor critical is not a rad safety principal. Knowing Rx power cannot be raised during a containment entry would be.
Psychometric flaw with distracter C, entry can be made above 400 psig with plant manager approval, therefore also correct.
RESOLUTION: Question changed.
72 F
3 Y
N S
73 F
4 Y
N S
74+
H 2
Y N
S 75 F
2 Y
N S
For RO portion of Exam: New = 47Q, Modified = 0Q, Bank = 28Q (37%): LOK: 34/75 = 45%, H= 55% U=6Q = 8%, E=11Q = 15%
76 H
2 Y
Y N
S 77+
H 1
Y Y
N U
S With the provided references this appears a direct look-up that is to low in difficulty.
RESOLUTION: Question Changed 78 FH 3
Y Y
N S
This appears to be a higher order question.
79 H
3 Y
Y N
S 80+
H 3
Y Y
N S
81 H
3 Y
Y N
S 82 H
3 Y
Y N
S 83+
H 3
Y Y
N S
84 H
3 Y
Y N
S 85 H
3 Y
Y N
S
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 9 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 86+
H 3
Y Y
B*
S NRC: Used on NRC 2011 Exam-Pilgrim 1 out of 2 taken twice 87 H
2 Y
Y N
S 88+
H 2
X Y
Y N
E S
Why is distractor C plausible? (Keep pressure below MSCP)?
RESOLUTION: Pressure in stem and distractor reworded.
89 H
2 Y
Y B*
S NRC: Used on NRC 2008 Exam-Nine Mile Point 2 90+
H 3
X Y
Y B*
U S
NRC: Used on NRC 2004 Exam-Nine Mile Point 1 Re-word distractor B to say continued operability of equipment important to safety...?
Are distractors B & D plausible? (Performing an ED to protect the integrity of SECONDARY containment)
RESOLUTION: After further review with author question was accepted as written 91 H
3 Y
Y N
S 92 H
2 Y
Y N
E S
Is distractor D plausible, especially if the candidate is given the TSs?
RESOLUTION: Replaced distractor D 93 H
3 Y
Y N
S 94+
F 2
Y N
N U
S Can this question be answered by using only RO knowledge of procedure use and adherence?
RESOLUTION: Question replaced.
95 F
2 X
Y Y
B*
E S
NRC: Used on NRC 2011 Exam-Duane Arnold (#95)
Distractor B is implausible. (See corrected question used in 2011)
RESOLUTION: Question changed.
96+
FH X
X Y
Y N
E S
This appears to be a higher function question.
What does entering the spent fuel pool from the reactor cavity really mean? Why is distractor A not a correct answer?
RESOLUTION: Stem changed to make it clear the platform is in the fuel pool.
ES-401 Written Examination Review Worksheet Form ES-401-9 Duane Arnold Energy Center April 2013 Exam 10 of 10 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 97 F
2 Y
N N
U S
Can this question be answered by using only RO knowledge of procedure use and adherence?
RESOLUTION: Question changed.
98+
H 2
Y Y
N E
S Where is calendar year defined? Is it really 12 consecutive months?
1 out of 2 taken twice RESOLUTION: New question written.
99 H
2 Y
N N
U S
Can this question be answered by using only RO knowledge of procedures? (AOP immediate actions and IPOI3)
Very weak KA match but accepted.
RESOLUTION: Question changed to be a direct operations question.
100+
H 3
Y Y
B E
S Restructure distractor B to be consistent with A & C. Distractor C would be more accurate if it stated... enter the SAGs and exit the EOPS.
RESOLUTION: Changes made.
For SRO portion of Exam: New = 20Q, Modified = 0Q, Bank = 5Q (20%): LOK: 3/25 = 12%, H= 88% U=4Q = 16%, E=6Q = 24%