ML13087A371
| ML13087A371 | |
| Person / Time | |
|---|---|
| Site: | Palisades  |
| Issue date: | 02/26/2013 |
| From: | Mcneil D NRC/RGN-III/DRS/OLB |
| To: | Entergy Nuclear Operations |
| References | |
| ES-401-9, 50-255/13-301 | |
| Download: ML13087A371 (5) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- questions reviewed in depth.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
F 2.0 x
N U
The stem states the P-1A speed is constant. The applicant would have to make assumptions to cause overspeed of the pump. With this statement in 2 distractors, the question is unsatisfactory.
Agreed. Changed distractors a.2 and c.2 such that there is no conflict with stem.
2 H
2.0 N
E Not a higher skill question. Agreed. Changed to low cog.
3 H
2.5 N
S 4
H 2.0 B
S 5
H 2.0 N
S 6
H 3.5 B
S 7
F 2.0 B
S 8
F 1.0 B
U LOD=1. Agreed. Modified question to increase difficulty level.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 9
H 2.0 B
S 10 H
2.0 B
S 11 H
1.0 B
U LOD = 1 Agreed. Replaced question.
12 H
2.0 N
S 13 H
2.0 N
E Not a higher skill question. Agreed. Changed to low cog.
14#
F 2.0 N
E Not a fundamental question. Agreed. Changed to high cog.
15 H
2.0 B
S 16#
F 2.0 N
S 17 H
2.0 N
E Not a higher skill question. Agreed. Changed to low cog.
18 H
2.0 N
S 19#
H 2.0 N
E Not a higher skill question. Agreed. Change to low cog.
20 H
2.0 N
S 21 H
2.0 N
S 22#
F 2.5 N
S 23 H
2.5 N
S 24 H
2.0 N
S 25#
F 2.0 B
S 26 H
2.5 B
S 27 F
2.0 B
S 28#
H 2.0 B
E Not a higher skill question. Agreed. Changed to low cog.
29 L
2.0 B
S 30 F
2.0 N
S 31#
H 2.0 N
S 32 F
1.0 N
U LOD = 1 Agreed. Replaced question.
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 33 H
2.0 B
S 34#
F 2.0 B
U S ROs are not required to recall action statements from memory for 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> technical specifications. Disagree. The facility would like to keep this question as-is because there is a specific objective, PPCS_CK20.0, that requires ROs to recall all less than or equal to 1-hour action statements. (NRC agrees) 35 H
2.0 x
B U S It is not plausible that the CCW Surge Tank Makeup Valve is open with a high surge tank level. Since two distractors have this as part of the distractor, there are two implausible distractors. Disagree.
The facility would like to keep this question as-is. Distractors b and d are plausible if the makeup valve opened to fill the tank and has not closed yet. The student then needs to know the setpoint at which the valve closes (74%). (NRC agrees) 36 H
2.0 x
B U S It is not plausible that the P-52A is powered from D/G 1-2. Since this occurs in two distractors, the question is unsatisfactory. Disagree.
This power supply is an exception. The facility would like to keep this question as-is. All other safety related equipment nomenclature that ends with A is powered from D/G 1-2. (NRC agrees) 37#
H 2.5 N
S 38 F
2.0 M
S 39 H
2.0 x
B U S Two implausible distractors. Startup nuclear instruments should be de-energized, so there would be no input for startup rate at 26%
power. Disagree. The facility would like to keep this question as-is. SRNIs are not de-energized at high power. Also, the high SUR signal is generated by the WRNIs. (NRC agrees) 40#
F 2.0 B
S 41 H
2.0 N
S 42 H
2.0 N
S 43#
F 2.0 B
S 44 F
2.0 B
S 45 F
2.0 B
S 46#
H 2.0 x
B U
FRV position rising with a high steam generator level is not a plausible distractor. Since two distractors contain the rising FRV position, the question is unsatisfactory. Agree. Significantly revised question.
47 H
2.5 B
S 48 F
2.0 N
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 49#
F 3.0 N
S 50 F
2.0 x
N U S 8 hour9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> batteries are not a plausible distractor unless you have them somewhere in the plant. Since the 8 hour9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br /> batteries are found in two distractors, the question is unsatisfactory. Disagree. The facility would like to keep this question as-is. Per DBD-4.01, Station Batteries, section 1.1.2, Batteries D-204, D-213, and D-214 are all rated for 8 hours9.259259e-5 days <br />0.00222 hours <br />1.322751e-5 weeks <br />3.044e-6 months <br />. (NRC agrees) 51 H
2.0 B
S 52#
F 2.0 B
S 53 H
2.0 N
S 54 F
2.0 N
S 55#
H 2.0 B
S 56 F
2.0 B
S 57 F
2.0 N
U Part 1 (slightly (2 - °F) is not plausible for the given plant condition.
Because the value is found in two distractors, the question is unsatisfactory. Agree. Significantly revised question.
58#
H 2.0 N
S 59 H
2.5 N
S 60 H
3.0 N
S 61#
H 2.0 N
S 62 F
2.0 B
S 63 F
2.0 B
S 64#
F 2.0 B
S 65 H
2.0 N
S 66 F
1.0 B
U LOD = 1. Agree. Replaced question.
67 F
2.0 N
S 68 F
2.0 M
S 69 F
2.0 B
S 70 H
2.0 B
S
Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 71 F
2.0 N
U S LOD = 1 Disagree. The facility would like to keep this question as-is. Demonstrated weakness at the station. (NRC agrees.)
72 H
2.5 B
S 73 H
2.0 N
S 74 F
2.0 B
S 75 F
2.0 B
S