ML21137A160

From kanterella
Jump to navigation Jump to search

NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)
ML21137A160
Person / Time
Site: Nine Mile Point Constellation icon.png
Issue date: 05/17/2021
From: Brian Fuller
Operations Branch I
To: Alexander M
Exelon Nuclear Generation Corp
Fuller B
Shared Package
ML20233A469 List:
References
EPID L-2021-OLL-0027
Download: ML21137A160 (7)
v • d • e


Text

ES-401 FY21 Nine Mile Point Unit 2 Written Examination Review Worksheet Form ES-401-9

2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
1. LOK 6. Source 7. Status Q LOD 8. Explanation (F/H) Stem Cred. Job- (B/M/N) (U/E/S)

(1-5) Cues T/F Partial Minutia #/Units Backward Q - K/A SRO Only Focus Dist Link 1 F 2 Y B E Modified stem conditions Modified Distractors B 2 F 3 Y N E and D, added noun name.

Added justification for 3 H 2 Y B E K/A match Added justification for 4 H 3 Y N E K/A match Modified B and D 5 F 3 Y M E distractors 6 F 3 Y M S 7 F 3 Y N E Modified Distractor D Modified stem and 8 H 2 Y B E distractors to avoid overlap with Q52 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
3. Check the appropriate box if a psychometric flaw is identified:
  • Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
  • Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
  • T/F: The answer choices are a collection of unrelated true/false statements.
  • Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
  • Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
4. Check the appropriate box if a job content flaw is identified:
  • Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
  • Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
  • #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  • Backward: The question requires reverse logic or application compared to the job requirements.
5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 FY21 Nine Mile Point Unit 2 Page 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist Modified distractors to 9 H 3 Y N E make applicant compare HUR to TS limit 10 F 3 X Y N E Modified distractors A&C Added explainer for K/A 11 F 2 Y B E match, modified distractor D

Modified stem value for 12 H 2 Y B E transmitter C Modified Distractors C &

13 F 3 Y N E D

14 H 3 Y B S 15 H 3 Y B E Reordered answers A & B Added KA justification 16 F 2 Y B E statement Modified stem and 17 H 3 Y B E Distractor B 18 F 2 Y B S 19 F 2 Y B S Added info for lowest RPV 20 H 3 Y N E water level to stem 21 F 3 Y B S 22 H 2 Y B E Reconfigured distractors 23 H 2 Y N S 24 F 2 Y B S Modified stem to make 25 H 3 Y N E generic control rod vice center rod

ES-401 FY21 Nine Mile Point Unit 2 Page 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist Added noun names for 26 H 2 Y B E MSIVs to stem bullets Enhanced Stem to Give 27 H 2 Y B E Time When Header Pressure Was Lost Added trends to stem 28 H 3 Y N S parameters 29 H 3 Y N S 30 H 3 Y M S 31 F 3 Y N S Modified first bullet in 32 H 2 Y B E stem (corrected vs indicated) 33 H 2 Y B S 34 H 2 Y B E Added better photos 35 H 3 Y N S 36 H 2 Y B E Reformatted stem 37 H 3 Y B S 38 H 3 Y N S 39 F 3 Y B S 40 F 3 Y N S 41 H 3 Y M S 42 F 3 Y N S

ES-401 FY21 Nine Mile Point Unit 2 Page 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 43 F 2 Y B S 44 F 2 Y B S 45 H 3 Y N S 46 H 2 Y B S 47 F 3 Y B S 48 F 3 Y N S 49 H 2 Y B S 50 F 3 Y M S POSSIBLE NO CORRECT ANSWER, 51 F 2 Y B US rewrote distractors to ensure only one correct answer 52 H 3 Y N S 53 F 3 Y N S 54 F 2 Y B S 55 H 3 Y B S Modified Distractors C &

56 H 3 Y N E D to match format of A &

C 57 F 3 Y B S Modified distractors to 58 F 2 Y B E 2X2 format

ES-401 FY21 Nine Mile Point Unit 2 Page 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 59 F 3 Y N S Modified stem to change 60 H 3 Y N E status of min flow valve to mid-position 61 H 2 Y B S Modified stem and 62 H 2 Y N E rearranged distractors to a 2X2 format Series of T/F Qs, rewrote 63 F 3 X Y N US stem and distractors 64 H 2 Y B S 65 F 3 Y N S 66 H 3 Y B S 67 F 2 Y B S Changed you to an operator in stem, 68 H 2 Y B E reformatted distractors to 2X2 QUESTIONS RELATED 69 F 3 Y B US TO LIMITING MCPR wrote new question 70 H 3 Y M S 71 F 2 Y B E Added time frame to stem Too system specific for 72 F 2 Y B US T3, wrote new question 73 H 3 Y M S 74 F 2 Y B E Modified distractor B & C

ES-401 FY21 Nine Mile Point Unit 2 Page 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 75 H 2 Y B S 76 H 2 Y Y B E Modified distractors C&D 77 H 3 Y Y N E Rearranged stem Added power level to 78 H 3 Y Y N E stem 79 H 3 Y Y B S LOD=1 wrote new 80 F 1 Y Y N US question Modified stem value for DW temperature. Modified 81 H 2 Y Y B E distractor D construction to parallel the other answers Modified stem to Corp ED, 82 H 2 Y Y B E Modified Distractors B & D PAR times NOT SRO ONLY SYSTEM KNOWLEDGE AND ENTRY 83 H 3 Y N N US CONDITIONS rewrote Q to focus on procedure selection 84 H 2 Y Y N S 85 H 2 Y Y B S DISTRACTOR C MAY BE A CORRECT 86 H 2 Y Y B US ANSWER reconfigured to 2X2 to eliminate 2nd correct answer POSSIBLE TWO CORRECT ANSWERS 87 H 3 Y Y N US added NO operator action to stem to eliminate 2nd correct answer

ES-401 FY21 Nine Mile Point Unit 2 Page 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other
6. Source 7. Status Q LOK LOD 8. Explanation Stem Cred. (B/M/N) (U/E/S)

(F/H) (1-5) Cues T/F Partial Job-Link Minutia # / Units Backward Q - K/A SRO Only Focus Dist 88 H 2 Y Y B E Deleted note on Figure A 89 H 2 Y Y B S Modified distractors to 90 F 3 Y Y N E eliminate specific determiner LOD=1. Resampled for 91 H 1 Y Y N US new K/A PROVIDED REFERENCE 92 H 3 Y Y N US GIVE AWAY ANSWER Removed reference NOT SRO ONLY, NO PROCEDURE SELECTION 93 H 3 Y NY B US D NOT PLAUSIBLE Added procedure to stem, modified all distractors 94 H 2 Y Y B S Modified distractor B, and 95 H 2 Y Y B E graphic of fuel cell Modified stem to improve 96 F 3 X Y Y M E focus, added info to Distractors A & B 97 H 3 Y Y M S NOT TIER 3, SHOULD ADDRESS release permit 98 H 2 Y Y B US procedure Resampled for K/A, wrote new Q 99 H 2 Y Y B S Changed Facilities to the 100 F 3 Y Y N E individual

Retrieved from "https://kanterella.com/w/index.php?title=ML21137A160&oldid=3536001"