ML15110A239
| ML15110A239 | |
| Person / Time | |
|---|---|
| Site: | Perry  |
| Issue date: | 04/03/2015 |
| From: | Walton R NRC/RGN-III/DRS/OLB |
| To: | FirstEnergy Nuclear Operating Co |
| References | |
| Download: ML15110A239 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 2
Y B
S 2
F 2
Y B
S 3
F 3
Y B
S 4
F 2
Y B
S 5
H 3
Y B
S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 6
F 2
Y N
E S
Can a Temp Mod tag be hung on an OOS annunciator ever? If so, then distractor B may be construed as correct also! (Time dimension?). RE: Temp Mod procedure DOES NOT state that can hang temp mod tags on annunciators. But PAP-1404 confirms answer.
7 F
2 Y
B S
8 H
3 Y
B S
9 H
3 Y
B S
10 F
2 Y
B S
11 H
3 Y
N S
12 H
3 Y
N S
13 F
3 Y
N S
1/2X2 Q 14 H
3 Y
N S
1/2X2 Q 15 H
3 Y
N S
16 H
2 Y
N S
17 H
4 Y
M S
Previous NRC exam 2010 18 H
3 Y
N S
19+
H 3
Y B
S 20 H
3 Y
Y M
U S
1/2X2 Q SRO-only question for procedure usage and refueling.
Suggest replacing this question (and maybe repicking K/A) since this question is too close to Q37. RE: ROs are expected to know entry conditions to ONIs.
21 H
3 Y
N S
1/2X2 Q 22 H
2 Y
M S
23 F
2 Y
B S
24 F
2 Y
B E
Before distractors, add Order of preference shown is from most preferred to least preferred. RE: Will add most preferred to least preferred in question stem.
ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 25 H
2 Y
B S
1/2X2 Q I presume that the students will perform the calculations to determine average temperature for this exercise and not have it provided! RE: Will provide students with sheets 9 and 28 of 33, Rounds sheets. Students will perform average calculation.
26 F
3 X
Y B
U E
First sentence of stem gives condition for requiring ED. Second sentence says condition of first sentence is met, then answer must be to ED (distractors B or C). Suggest rewording in form of a question. Why does EOP-2 require ED if SP Level cant be maintained >2 above horizontal vents? Are ROs taught EOP basis? RE: (Previously approved by Clinton exam bank question, not unsat if changed). Question stem changed.
27 F
2 Y
B S
28 H
3 Y
B S
29 F
2 Y
B S
30 H
3 Y
M S
31 F
3 Y
N S
32 F
2 Y
B S
33 F
2 Y
B S
34 F
2 Y
B S
35 H
3 Y
Y M
U S
1/2X2 Q. Previous NRC exam 2013. Looks like SRO-only question due to entering into ONI procedures (not EOPs). RE: Not an SRO-only question since ROs are required to know entry conditions to ONIs.
36 H
3
??
X Y
?
B U
1/2X2 Q. (SRO-only question since requires knowledge of TS Action steps?) Reference ONI-C-11-1 steps in lieu of TS steps! Series of True/False questions? Move common part of all distractors into stem. Distractors not credible since TS does not have 20 minute criteria. RE: 20 minute TS action statement DOES exist in TS -
therefore is CREDIBLE. Also, answer is below TS line in SRO guidance but it less than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, so knowledge must be known by RO. Common text in distractors moved to stem, CRD pressure attribute changed to rod position based on Q revalidation.
37 H
3 Y
B S
ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 38 H
3 Y
B S
Previous NRC exam 2010.
39 H
3 Y
N S
1/2X2 Q. SRO Only? RE: ROs are required to know ONI entry conditions.
40 H
2 Y
B S
Previous NRC exam 2010.
41 H
2 Y
B S
1/2X2 Q 42 F
2 Y
N S
43 H
3 Y
B S
44 H
3 Y
B S
45 H
3 Y
B S
46 F
3 Y
N S
47 F
2 Y
B S
48 F
3 Y
N S
49 H
2 Y
B S
Previous NRC exam 2010 50 H
2 Y
B S
51 H
2 Y
B S
1/2X2 Q 52 H
3 Y
M S
53 H
2 X
Y B
U Backwards Logic: Effect occurs, what caused it? Should read, effect occurs, what does operator do, or procedure response Suggest changing distractors to what response to operators take. RE:
Changed distractors such that must pick entry into Off Normal Instructions to address adverse condition vs.name that event.
54 H
2 X
Y B
U 1/2X2 Q. If DFWCS level channels fail, why shift DFWCS to AUTO?
Implausible distractors A & C. RE: Changed A & C distractors to read shift DFWCS to single element.
55 F
2 Y
B E
S 1/2X2 Q. Previous NRC exam 2010. If normal Rx Level setpoint is 196 and question has setpoint selected to 200, then applicant must apply this new knowledge to answer question correctly. Not a memory question. RE: Operators are taught that setpoint setdown will control to level setpoint NOT to 196
ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 56 F
2 Y
N S
57 H
1 Y
M S
1/2X2 Q. This is better done as a JPM rather than a written question.
58 H
2 Y
N S
59 H
3 Y
N S
60 H
2 Y
B E
1/2X2 Q. Need to pull up common words from each distractor and put into stem. RE: Pulled up common phrases from distractors into stem of question.
61 H
3 Y
M S
1/2X2 Q. Previous NRC 2010 Exam 62 H
2 Y
B E
S 1/2X2 Q. Previous NRC exam 2013 IS this a modified question?
RE: Yes, question was changed since instrument air pressure alone was not enough. Temp air compressor added. Plant mod.
63 F
2 Y
N S
1/2X2 Q 64 F
2 Y
B S
65 F
2 Y
B S
66 F
2 Y
N S
1/2X2 Q 67 F
2 Y
B S
68 H
3 Y
B S
69 F
2 Y
N E
1/2X2 Q. Missing blank in stem for 2nd answer. RE: Blank added for (2).
70 H
3 Y
N S
1/2X2 Q 71 F
2 Y
B S
72 H
3 Y
B S
1/2X2 Q 73 F
2 Y
N S
1/2X2 Q 74 F
2 Y
N S
1/2X2 Q
ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 75 H
3
??
M E
K/A mismatch? Heat Transfer mechanisms, submergence, spray cooling and steam cooling. RE: This is a good question that may miss the K/A. Had Perry change K/A for better match. Increased difficulty of question by changing answer from submergence to steam cooling 76 F
2 Y
Y B
S 77 F
2 Y
Y N
S 78 F
2 Y
Y B
S 79 F
2 Y
Y B
S 80 H
3 Y
Y N
S 81 F
2 X
X Y
Y N
U 1/2X2 Q. 2 Person Line of Site Rule is in stem and also in 2 distractors. Also, DSO reference seems too high in chain of command to be a creditable distractor. RE: Question replaced.
82 F
2 Y
Y N
S 83 H
3 Y
Y N
S 1/2X2 Q 84 H
2 Y
Y B
S 85 H
2 Y
Y N
S 86 H
2 Y
Y B
S 87 H
2 Y
Y B
S Previous NRC exam 2010.
88 H
2 X
Y NO N
U 1/2X2 Q. Question can be answered without the stem! Not an SRO-only question. Reference TS in lieu of SOI precautions/limitations.
How is this not a memory question? Need to pull common phrase from distractors into stem of question. ALSO, need to delete reference to SOI-M14 and reference instead TS Bases to ensure SRO-only question. RE: Question stem and distractors changed.
89 H
3 Y
Y M
S 90 H
2 Y
Y N
S 1/2X2 Q 91 F
3 Y
Y N
S 1/2X2 Q 92 H
3 Y
Y B
S 1/2X2 Q.
ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 93 H
3 Y
Y N
S 1/2X2 Q 94 H
3 Y
Y B
S (References Provided) Why are 4 similar PEIs provided? RE: Will remove Figures from partial EOP chart.
95 H
3 Y
Y B
S Previous NRC exam 2010.
96 H
3 Y
Y N
S 1/2X2 Q 97 H
3 Y
Y N
S 1/2X2 Q 98 H
4 Y
Y N
S 99 H
3 Y
Y N
S 1/2X2 Q 100 H
2 Y
Y N
S 1/2X2 Q 33% of questions in one of two taken twice format (1/2X2 Q)