ML21033A796

From kanterella
Jump to navigation Jump to search

NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)
ML21033A796
Person / Time
Site: Nine Mile Point Constellation icon.png
Issue date: 02/02/2021
From: Brian Fuller
Operations Branch I
To:
Exelon Nuclear Generation Corp
Shared Package
ML19309G108 List:
References
CAC00500
Download: ML21033A796 (7)
v • d • e


Text

ES-401 Nine Mile Point Unit 1 FY21 Written Examination Review Worksheet Form ES-401-9

2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
1. LOK 6. Source 7. Status Q LOD 8. Explanation (F/H) Stem Cred. Job- (B/M/N) (U/E/S)

(1-5) Cues T/F Partial Minutia #/Units Backward Q - K/A SRO Only Focus Dist Link 1 H 2 Y MB S 2 H 2 Y B S 3 H 2 Y MB S Increased plausibility of C and D distractors by 4 F 3 Y N E specifying CS corner room locations 5 F 2 Y B S 6 F 3 Y MB S 7 H 3 Y B S 8 F 3 Y B S Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
3. Check the appropriate box if a psychometric flaw is identified:
  • Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
  • Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
  • T/F: The answer choices are a collection of unrelated true/false statements.
  • Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
  • Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
4. Check the appropriate box if a job content flaw is identified:
  • Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
  • Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
  • #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  • Backward: The question requires reverse logic or application compared to the job requirements.
5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 FY21 NMP UNIT 1 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 9 F 3 Y B S 10 F 3 Y B S 11 H 3 Y MB S 12 H 3 Y B S Rearranged the question part of the stem and the 13 H 3 Y N E corresponding answer columns to improve clarity of question 14 H 3 Y MB S 15 F 3 Y N S 16 F 2 Y B S 17 H 3 Y B S 18 H 3 Y N S 19 H 3 Y MB S 20 F 2 Y B S Reformatted Q and 21 F 3 Y N E enhanced explanation 22 H 3 Y B S 23 F 2 Y MB S 24 H 2 Y B S

ES-401 FY21 NMP UNIT 1 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 25 H 2 Y B S Revised two distractors for increased plausibility.

26 F 3 x Y N E Also made editorial change.

27 H 3 Y B S 28 H 2 Y B S Corrected justification to 29 F 3 Y N E reflect current TS limit 30 F 3 Y B S 31 F 3 Y MB S 32 H 3 Y B S 33 F 3 Y B E Editorial change 34 H 3 Y N S 35 H 3 Y N S 36 H 3 Y N S 37 F 2 Y B S 38 H 3 Y N S Added Rx to 4th bullet in 39 H 3 Y B E stem to match SOP title 40 H 2 Y B S 41 F 2 Y B S

ES-401 FY21 NMP UNIT 1 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 42 F 2 Y B S Added Mode Switch 43 H 3 Y N E position to stem Question addresses 44 H 3 Y B S second half of K/A. Added KA Match Justification 45 H 2 Y B S 46 H 3 Y N S 47 H 3 Y B S 48 H 3 Y B S 49 F 3 Y MB S 50 H 3 Y B S 51 H 3 Y B S 52 F 3 Y N S 53 H 3 Y MB S 54 F 3 Y MB S 55 H 2 Y B S 56 F 2 Y B S 57 F 2 Y B S 58 H 3 Y N S

ES-401 FY21 NMP UNIT 1 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 59 F 3 Y MB S 60 H 3 Y B S 61 H 3 Y B S 62 H 3 Y B S 63 H 3 Y N S 64 H 3 Y B S 65 H 2 Y B S 66 F 2 Y N S Question addresses the 67 F 2 Y B S function of the controller 68 F 2 Y B S 69 F 2 Y B S 70 F 2 Y B S 71 F 2 Y B S Revised question for 72 H 3 Y B E realism 73 F 3 Y N S K/A MM. Does not address resultant 74 F 2 N N U operational effects.

Replaced question 75 F 3 Y B S

ES-401 FY21 NMP UNIT 1 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 76 H 2 Y Y N E Editorial 77 H 3 Y Y B S 78 H 3 Y Y N E Editorial 79 F 2 Y Y B S 80 H 3 Y Y N S Changed declare an emergency in last bullet 81 H 3 Y Y B E of stem to classify the event 82 H 3 Y Y B S 83 F 2 Y Y B S 84 F 3 Y Y N S 85 H 3 Y Y MB S 86 H 3 Y Y N S 87 H 3 Y Y N S 88 H 2 Y Y N S Revised two distractors 89 F 2 x Y Y N E for increased plausibility 90 H 2 Y Y B S 91 H 2 Y Y B S 92 H 2 Y Y B S

ES-401 FY21 NMP UNIT 1 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 93 H 3 Y Y N S 94 H 2 Y Y MB S 95 H 3 Y Y B S 96 F 2 Y Y N S 97 F 2 Y Y B S 98 H 2 Y Y B S 99 F 2 Y Y B S Added prioritization to 100 F 3 Y Y N E distractors

Retrieved from "https://kanterella.com/w/index.php?title=ML21033A796&oldid=3534085"