ML21239A082
| ML21239A082 | |
| Person / Time | |
|---|---|
| Site: | Peach Bottom  |
| Issue date: | 05/18/2021 |
| From: | Todd Fish Exelon Generation Co |
| To: | NRC Region 1 |
| Shared Package | |
| ML20259A337 | List: |
| References | |
| EPID L-2021-OLL-002 | |
| Download: ML21239A082 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other Q 1. LOK LOO 6. Source 7. Status 8. Explanation (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Minutia #/Units Backward Q-KIA SRO Only (B/M/N) (U/E/S)
Focus Dist Link
1 H 2.5 N s
2 H 3.0 N s
3 H 3.0 N s
4 F 2.5 N s
5 H 2.8 N s
6 F 3.0 N s -
7 H 3.0 N s
8 H 2.5 N s
Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty {LOO) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if. "Stem Focus": The stem lacks sufficient a psychometric flaw is identified:
. "Cues": The stem or distractors focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
. "T/F": The answer choices are a collection of unrelated true/false contain cues (e.g., clues, specific determiners, phrasing, length).
. "Cred. Dist.": statements.
. "Partial": One or more distractors The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4. Check the appropriate box if. "Job Link": The question is a job content flaw is identified:
. "Minutia": The question requires not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).
. #/Units": The question contains data with the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
. "Backward": The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved KIA and those KIAs that are designated "SRO-only." (KIA and license-level mismatches are unacceptable.)
- 6. Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7. Based on the reviewer's judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any "U" status ratings (e.g., how the Appendix B psychometric attributes are not being met).
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other Q LOK LOO 6. Source 7. Status 8, Explanation (F/H) (1-5) Stem Cues T/F Cred. Partial Job-Link Minutia #/Units Backward Q-K/A SRO Only (B/M/N) (U/E/S) ~ Focus Dist
9 H 3.0 N s
10 F 2.5 N s
11 F 2.5 N s
12 F 3.0 N s
13 F 2.5 B s
14 H 3.0 B s
15 H 3.0 N s
16 H 3.0 B s
17 H 2.5 B s
18 F 3.0 N s
19 H 2.8 N E Update picture to clarify provided indications.
20 H 3.0 N s
RRC and SO actions are 21 F 3.0 X N u too similar; two possible correct choices. Revised auestion.
22 F 1.0 X N u LOD=1. Replaced KIA and Q.
23 F 3.0 N s
24 H 3.0 B s
25 H 3.0 N s ES-401 3 Form ES-401-9
Q LOK LOO 3. Psychometric Flaws 4. Job Content Flaws 5. Other (B/M/N) (U/E/S) 8. Explanation 1. 2. 6. Source 7. Status
- -**n., ~,
26 F 2.5 N s
27 F 2.5 B s
28 H 2.5 N s
29 H 2.5 N s
30 H 2.5 N s
31 F 3.0 B s
32 H 3.0 N s
33 H 2.5, / N s
34 F 3.0 N s
35 H 3.0 N s
36 F 3.0 N s
37 F 2.5 N s
38 H 3.0 N s
39 H 3.0 N s
40 F 2.5 N s
41 F 2.5 N s
42 F 3.0 N s
43 F 3.0 N s ES-401 4 Form ES-401-9
Q LOK LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other (B/M/N) (U/E/S) 8. Explanation 1. 2. 6. Source 7. Status
,r.. " 'A ~,
- 44 H 2.5 N s
45 H 3.0 B s
46 H 3.0 B s
47 H 3.0 N u Overlap with 071.
Replaced KIA and Q.
48 F 2.5 N s
49 F 2.5 N s
50 H 2.5 N s '
51 F 2.5 N s
52 H 3.0 B s
53 F 2.5 N s
54 H 3.0 N s
55 F 2.5 B s
- 56 F 3.0 N s
C and D not credible for 57 F 3.0 X N u severity of event.
Replaced KIA and Q.
58 H 2.5 N s
59 F 3.0 N s
60 H 3.0 B s
LOD=5. Answer requires 61 F 5 X N u detailed procedure memorization bevond ES-401 5 Form ES-401-9
Q LOK LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other (B/M/N) (U/E/S) 8. Explanation 1. 2. 6. Source 7. Status
'A r, immediate actions.
Replaced KIA and Q
62 H 3.0 N s
63 F 2.5 N s
64 F 2.5 N s
65 H 2.5 N s
66 H 3.0 N s
67 H 3.0 N s
68 H 3.0 N s
69 H 2.5 V N s
70 F 2.5 B s
71 H 3.0 N s
72 H 3.0 N s
73 H 3.0 B s
74 H 3.0 N s
75 H 2.5 N s
76 H 1.0 X X N u LOD=1. DLU, given TS. Replaced KIA and Q
77 H 3.0 N s
78 F 2.5 N s ES-401 6 Form ES-401-9
Q LOK LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other (8/M/N) (U/E/S) 8. Explanation 1. 2. 6. Source 7. Status
79 H 3.0 N s
80 H 3.0 8 s
81 H 3.0 N s
82 H 3.0 N s
83 H 2.5 N s
84 H 3.0 N s
85 H 3.0 N s
86 H 3.0 N s
87 H 1.0 X 8 u Not SRO-only. Low LOD. Rewrote Q
Question provides info (resin injection or fuel 88 H 3.0 X X 8 u failure), thus no SRO diagnosis/prediction needed. Replaced KIA and Q 89 H 3.0 8 s
90 H 3.0 N s
91 H 3.0 N s
92 H 3.0 N s
93 H 3.0 N s
94 H 3.0 8 s
95 H 3.0 8 s ES-401 7 Form ES-401-9
Q LOK LOD 3. Psychometric Flaws 4. Job Content Flaws 5. Other (B/M/N) (U/E/S) 8. Explanation 1. 2. 6. Source 7. Status
- -**. ~, LOD=1. Q requires RO
96 H 1.0 X N u knowledge. Replaced KIA and Q.
97 F 3.0 B s
98 H 2.5 N s
99 H 3.0 N s
100 H 3.0 N s