ML21348A187
| ML21348A187 | |
| Person / Time | |
|---|---|
| Site: | Susquehanna  |
| Issue date: | 08/06/2021 |
| From: | Todd Fish Susquehanna, Talen Energy |
| To: | Todd Fish Operations Branch I |
| Shared Package | |
| ML20259A339 | List: |
| References | |
| EPID L-2021-OLL-0003 | |
| Download: ML21348A187 (7) | |
Text
ES-401 Susquehanna Written Examination Review Worksheet -~vc.. 'i J-9:)..l Form ES-401-9
- 2.
- 3. Psychometric Raws
- 4. Job Content Flaws
- 5. Other Q
1.LOK LOD
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5) stem Cues T/F Cred.
Partial Job-Minutia
- /Units Backward Q-KIA SRO Only (B/M/N)
(U/E/S)
Focus Dist Link 1
F 3
N s
2 H
3 N
s B and C options are not 8
3 H
3 X
N u
plausible. Developed New Test Item 4
F 4
-- N s
~------- ---- f---------~-- ------ ------------~----
~--~- ---~-- -- - ~~~-----------
5 H
2.5 N
s 6
F 2.5 B
s 7
H 3
N s
8 H
2.5 N
s Refer to Section D of ES-4-01 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge {LOK) of each question as either (F)undamental or {H)igher cognitive level.
- 2.
Enter the level of difficulty (LOO) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw Is Identified:
"Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., undear Intent, *more Information Is needed, or too much needless Information) *
"Cues": The stem or distradors contain cues (e.g., clues, specific detenniners, phrasing, length) *
~
. "T/F": The answer choices are a collection of unrelated true/false statements *
. "Cred. Dist.": The distractors are not credible; single implausible dlstractors should be repaired, and more than one is unacceptable.
"Partial": One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
"Job Link": The question is not linked to the job requirements Q.e., the question has a valid KIA but, as written, is not operational In content) *
"Minutia": Toe question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
"#/Units": The question contains data with an unrealistic level of accuracy_ or inconsistent units {e.g., panel meter in percent with question in gallons).
"Backward": The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those KIAs that are deslgnat~ "SRO-only." (KIA and license-level mismatches are unacceptable.)
- 6.
Enter question's source: (B)ank, (M)odified, or {N)ew. Verify that (M)odllied questions meet the criteria of Form ES401, Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question, as written, (U)nsatisfadory (requiring repair or replacement), In need of {E)ditorlal enhancement, or (S)atlsfact01y?
- 8.
At a minimum, explain any "U" stabJs ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometrfc Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOO
- 6. Source
- 7. Status B. Explanation (F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job--Llnk Minutia
- / Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S}
Focus Dist 9
F 2.5 N
s 10 H
3.5 N
s 11 H
3.75 N
s 12 F
4 B
s 13
. F 4
- -~-
.N *----s*--
14 F
4 B
s 15 H
2.75 N
s 16 H
2.5 N
s 17 H
3.5 8
s 18 F
2 N
s 19 H
3.25 B
s 20 F
2.5 I
N s
21 F
3 N
s 22 F
3 N
s 23 F
4 M
s 24 H
4 B
s 25 H
3 B
s
ES-401 3
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOO
- 6. Source
- 7. Status
- 8. Explanation
{F/H)
(1-5)
Stem Cues T/F Cred.
Partial Job-Link Minutia
- /Units Backward Q-I</A SRO Only (B/M/N)
(U/E/S)
Focus Dist 26 H
3 N
s 27 H
3 N
s 28 F
4 B
s Minutia, no actionable 29 F
2.25 X
N u
response. Reselected KA B
and replaced the test item with a bank nuestion. -~
Add reactor pressure to 30 H
3.25 N
E stem to make A option olausible.
31 H
2 N
s 32 H
4 N
s 33 F
4 N
s 34 H
3 8
s 35 F
2 N
s N
A and B not plausible.
36 F
3 X
B u
Reselected KA and reclaced test item.
37 F
4 B
s 38 H
3 N
s 39 F
4 N
s 40 F
2 M
s 41 F
2 N
s
ES-4O1 4
Form ES-401-9 1,
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOO
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5) stem Cues T/F Cred.
Partial Job-Urik Minutia
- I Units Backward Q-KJA SRO Only (B/M/N)
(U/E/S)
Focus Dist 42 F
3 N
s Make turbine bypass 43 F
4 N
E valve acronym consistent throuohoutthe stem.
44 F
3 N
s 45 H
2 N
s
~---~ -----
46 H
3.5 N
s 47 F
3 N
s 4a H
3 B
s 49 H
2.5 B
s 50 H
3 B
s 51 H
3 B
s 52 H
3.5 M
s 53 H
3 N
s 54 H
3 N
E Provide reactor pressure and trend in the stem.
55 F
2 N
s 56 H
3 B
s 57 F
2 N
s 58 F
2.5 B
s
ES-401 5
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOO
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cues Cred.
Partial Job-Link Minutia
- / Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S)
Focus T/F Dist 59 F
2.5 N
s 60 H
3 N
s 61 H
3 N
s 62 F
2.5 N
E Term Isolation" is missing from the C and D options.
- - ~F-~ -3;5 N
s 64 F
3 B
s 65 H
2.5 N
s 66 H
3 B
s 67 H
4 N
s 68 H
3 N
s 69 H
3 N
s 70 F
4 8
s 71 H
3 N
s 72 F
3 N
s 73 H
3 N
s 74 H
3 N
s 75 H
4 N
s
ES-401 6
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOO
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5) stem Cues TIF Cred.
Partial Job-Link Minutia
- I Units Back.ward Q-'r</A SRO Only (B/M/N)
(U/EJS)
Focus Dist 76 H
2 N
s 77 H
3 N
s 78 F
3.5 B
E Add "IAW procedure... " to the stem question.
KA not met Radwaste not 79 H
3 X
N u
safety related.
Reselected-KA and replaced test item.
80 H
2 M
s 81 H
3 B
s 82 H
3.5 N
s 83 H
3 N
s 84 H
3 B
s 85 H
3 N
s 86 F
2 N
s 87 H
3 N
s 88 H
3 N
s Revise stem question to 89 H
3 N
E ask for "first EAL condition".
90 F
3 N
s 91 H
3 N
s
ES-401 7
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q
LOK LOD
- 6. Source
- 7. Status
- 8. Explanation (F/H)
(1-5)
Stem Cues T/F Creel.
Partial Job-Link Minutia
- / Units Backward Q-K/A SRO Only (B/M/N)
(U/E/S)
Focus Dist 92 F
4 N
s 93 F
2 N
s 94 H
3 N
s 95 F
3 N
s
-~
96 H
3 N
s 97 H
3 N
s 98 H
4 N
s 99 H
3 M
s 100 H
3 N
s