ML21137A160
| ML21137A160 | |
| Person / Time | |
|---|---|
| Site: | Nine Mile Point  |
| Issue date: | 05/17/2021 |
| From: | Brian Fuller Operations Branch I |
| To: | Alexander M Exelon Nuclear Generation Corp |
| Fuller B | |
| Shared Package | |
| ML20233A469 | List: |
| References | |
| EPID L-2021-OLL-0027 | |
| Download: ML21137A160 (7) | |
Text
ES-401 FY21 Nine Mile Point Unit 2 Written Examination Review Worksheet Form ES-401-9 Q
- 1. LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /Units Backward Q - K/A SRO Only 1
F 2
Y B
E Modified stem conditions 2
F 3
Y N
E Modified Distractors B and D, added noun name.
3 H
2 Y
B E
Added justification for K/A match 4
H 3
Y N
E Added justification for K/A match 5
F 3
Y M
E Modified B and D distractors 6
F 3
Y M
S 7
F 3
Y N
E Modified Distractor D 8
H 2
Y B
E Modified stem and distractors to avoid overlap with Q52 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 FY21 Nine Mile Point Unit 2 Page 2 Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 9
H 3
Y N
E Modified distractors to make applicant compare HUR to TS limit 10 F
3 X
Y N
E Modified distractors A&C 11 F
2 Y
B E
Added explainer for K/A match, modified distractor D
12 H
2 Y
B E
Modified stem value for transmitter C 13 F
3 Y
N E
Modified Distractors C &
D 14 H
3 Y
B S
15 H
3 Y
B E
Reordered answers A & B 16 F
2 Y
B E
Added KA justification statement 17 H
3 Y
B E
Modified stem and Distractor B 18 F
2 Y
B S
19 F
2 Y
B S
20 H
3 Y
N E
Added info for lowest RPV water level to stem 21 F
3 Y
B S
22 H
2 Y
B E
Reconfigured distractors 23 H
2 Y
N S
24 F
2 Y
B S
25 H
3 Y
N E
Modified stem to make generic control rod vice center rod
ES-401 FY21 Nine Mile Point Unit 2 Page 3 Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 26 H
2 Y
B E
Added noun names for MSIVs to stem bullets 27 H
2 Y
B E
Enhanced Stem to Give Time When Header Pressure Was Lost 28 H
3 Y
N S
Added trends to stem parameters 29 H
3 Y
N S
30 H
3 Y
M S
31 F
3 Y
N S
32 H
2 Y
B E
Modified first bullet in stem (corrected vs indicated) 33 H
2 Y
B S
34 H
2 Y
B E
Added better photos 35 H
3 Y
N S
36 H
2 Y
B E
Reformatted stem 37 H
3 Y
B S
38 H
3 Y
N S
39 F
3 Y
B S
40 F
3 Y
N S
41 H
3 Y
M S
42 F
3 Y
N S
ES-401 FY21 Nine Mile Point Unit 2 Page 4 Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 43 F
2 Y
B S
44 F
2 Y
B S
45 H
3 Y
N S
46 H
2 Y
B S
47 F
3 Y
B S
48 F
3 Y
N S
49 H
2 Y
B S
50 F
3 Y
M S
51 F
2 Y
B U S POSSIBLE NO CORRECT ANSWER, rewrote distractors to ensure only one correct answer 52 H
3 Y
N S
53 F
3 Y
N S
54 F
2 Y
B S
55 H
3 Y
B S
56 H
3 Y
N E
Modified Distractors C &
D to match format of A &
C 57 F
3 Y
B S
58 F
2 Y
B E
Modified distractors to 2X2 format
ES-401 FY21 Nine Mile Point Unit 2 Page 5 Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 59 F
3 Y
N S
60 H
3 Y
N E
Modified stem to change status of min flow valve to mid-position 61 H
2 Y
B S
62 H
2 Y
N E
Modified stem and rearranged distractors to a 2X2 format 63 F
3 X
Y N
U S Series of T/F Qs, rewrote stem and distractors 64 H
2 Y
B S
65 F
3 Y
N S
66 H
3 Y
B S
67 F
2 Y
B S
68 H
2 Y
B E
Changed you to an operator in stem, reformatted distractors to 2X2 69 F
3 Y
B U S QUESTIONS RELATED TO LIMITING MCPR wrote new question 70 H
3 Y
M S
71 F
2 Y
B E
Added time frame to stem 72 F
2 Y
B U S Too system specific for T3, wrote new question 73 H
3 Y
M S
74 F
2 Y
B E
Modified distractor B & C
ES-401 FY21 Nine Mile Point Unit 2 Page 6 Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 75 H
2 Y
B S
76 H
2 Y
Y B
E Modified distractors C&D 77 H
3 Y
Y N
E Rearranged stem 78 H
3 Y
Y N
E Added power level to stem 79 H
3 Y
Y B
S 80 F
1 Y
Y N
U S LOD=1 wrote new question 81 H
2 Y
Y B
E Modified stem value for DW temperature. Modified distractor D construction to parallel the other answers 82 H
2 Y
Y B
E Modified stem to Corp ED, Modified Distractors B & D PAR times 83 H
3 Y
N N
U S NOT SRO ONLY SYSTEM KNOWLEDGE AND ENTRY CONDITIONS rewrote Q to focus on procedure selection 84 H
2 Y
Y N
S 85 H
2 Y
Y B
S 86 H
2 Y
Y B
U S DISTRACTOR C MAY BE A CORRECT ANSWER reconfigured to 2X2 to eliminate 2nd correct answer 87 H
3 Y
Y N
U S POSSIBLE TWO CORRECT ANSWERS added NO operator action to stem to eliminate 2nd correct answer
ES-401 FY21 Nine Mile Point Unit 2 Page 7 Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/M/N)
- 7. Status (U/E/S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- / Units Backward Q - K/A SRO Only 88 H
2 Y
Y B
E Deleted note on Figure A 89 H
2 Y
Y B
S 90 F
3 Y
Y N
E Modified distractors to eliminate specific determiner 91 H
1 Y
Y N
U S LOD=1. Resampled for new K/A 92 H
3 Y
Y N
U S PROVIDED REFERENCE GIVE AWAY ANSWER Removed reference 93 H
3 Y
N Y B
U S NOT SRO ONLY, NO PROCEDURE SELECTION D NOT PLAUSIBLE Added procedure to stem, modified all distractors 94 H
2 Y
Y B
S 95 H
2 Y
Y B
E Modified distractor B, and graphic of fuel cell 96 F
3 X
Y Y
M E
Modified stem to improve focus, added info to Distractors A & B 97 H
3 Y
Y M
S 98 H
2 Y
Y B
U S NOT TIER 3, SHOULD ADDRESS release permit procedure Resampled for K/A, wrote new Q 99 H
2 Y
Y B
S 100 F
3 Y
Y N
E Changed Facilities to the individual