ML20324A040

CN-2020-09-DRAFT Written Exam Comments

ML20324A040
Person / Time
Site:	Cooper
Issue date:	09/29/2020
From:	Greg Werner Operations Branch IV
To:	Nebraska Public Power District (NPPD)
References
ES-401-9
Download: ML20324A040 (7)

Text

ES-401 Written Examination Review Worksheet - Cooper Nuclear Station - September 29, 2020 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other

6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. Job- #/ Back Q- SRO (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only 1 H 3 N S 2 H 3 N S 3 H 3 N S 4 F 2 N S 5 H 2 N S 6 H 3 N S 7 F 3 M S Reference provided in stem 8 F 3 N S Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.

3. Check the appropriate box if a psychometric flaw is identified:

• Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).

• T/F: The answer choices are a collection of unrelated true/false statements.

• Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.

• Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).

4. Check the appropriate box if a job content flaw is identified:

• Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).

• #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• Backward: The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)

6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.

7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).

Rev. 11

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other

6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. Job- #/ Back Q- SRO (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only 9 F 2 N S 10 H 3 M S 11 H 4 N S 12 H 3 N S 13 H 3 N S 14 H 3 M S 15 H 3 N S 16 H 3 N S 17 F 2 N S 18 F 2 N S 19 H 4 N S 20 H 3 N S 21 F 2 B S 22 H 3 N S 23 F 2 N S 24 H 3 N S 25 F 2 N S 26 H 3 N S Reference provided in stem Rev. 11

ES-401 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other

6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. Job- #/ Back Q- SRO (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only 27 F 3 M S 28 F 3 N S 29 H 3 N S 30 H 3 N S 31 F 3 N S 32 F 2 M S 33 F 2 N S 34 H 3 N S 35 F 2 N S 36 F 2 N S 37 H 3 B S 2015-12 NRC Exam 38 F 3 N S 39 H 3 B S 40 H 3 N S 41 H 2 B S Reference provided in stem, 2017 NRC Exam 42 H 3 N S 43 H 3 B S Reference provided in stem, 2015-12 NRC Exam 44 F 2 B S 2020-08 NRC Exam (previous two exams)

Rev. 11

ES-401 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other

6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. Job- #/ Back Q- SRO (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only 45 H 4 M S Reference provided 46 F 2 N S 47 H 3 B S 2015-12 NRC Exam 48 F 3 N S 49 F 3 N S 50 F 3 N S 51 H 4 N S 52 H 3 N S 53 F 2 N S 54 H 3 N S 55 H 3 M S 56 F 2 N S 57 F 3 N S 58 H 3 B S 2014 NRC Exam 59 F 3 N S 60 H 3 N S Reference provided in stem 61 F 3 N S 62 H 3 N S Rev. 11

ES-401 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other

6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. Job- #/ Back Q- SRO (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only 63 F 3 N S 64 F 2 N S 65 H 3 M S 66 H 2 N S 67 F 2 N S 68 F 3 N S 69 F 2 N S 70 F 3 N S 71 H 2 N S 72 F 3 M S 73 F 2 N S 74 F 2 M S 75 F 3 N S 76 H 3 N S 77 H 3 N S 78 H 3 M S 79 F 2 N S Reference Provided 80 H 3 N S Reference provided Rev. 11

ES-401 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other

6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. Job- #/ Back Q- SRO (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only 81 H 4 B S 82 H 3 B S 2015-04 NRC Exam 83 H 3 B S 2015-11 NRC Exam 84 H 4 B S 2014 NRC Exam 85 H 3 N S Reference provided 86 H 3 N S 87 H 3 N S 88 H 3 N S 89 F 3 N S 90 H 3 N S 91 H 3 N S 92 H 3 N S Reference provided 93 H 3 N S 94 F 2 N S 95 H 2 M S Reference provided 96 H 2 B S 2012 NRC Exam 97 F 2 N S 98 F 2 B S 2017 NRC Exam Rev. 11

ES-401 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other

6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. Job- #/ Back Q- SRO (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia Focus Dist Link Units ward K/A Only 99 F 3 N S 100 H 2 M S Rev. 11