ML17271A152
| ML17271A152 | |
| Person / Time | |
|---|---|
| Site: | Dresden  |
| Issue date: | 08/21/2017 |
| From: | Walton R NRC/RGN-III/DRS/OLB |
| To: | Exelon Generation Co |
| Walton R | |
| Shared Package | |
| ML17068A438 | List: |
| References | |
| Download: ML17271A152 (6) | |
Text
1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) B/M/N U/E/S Explanation Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO Focus Dist. Link units ward K/A Only 1+ H 3 Y N U This appears to duplicate JPM a. Suggest changing indications in Q such that different path on Nominal Feedwater Heating hard card is used that is different B
from JPM a. RE: Changed the written question to use a different end-point on the flow chart than what is used on JPM a.
2 L 2 Y N S 3 H 2 Y B S 1/2X2; NRC 2013 4+ H 3 Y B S 5 L 3 Y B E Valve designator is incorrect! Is this the 1 valve? (Should not be 2 valve). RE:
Designator is a switch not a valve. STET.
S NRC 2014 6+ F 2 Y B S NRC 2014 7 F 2 Y N S 1/2X2 Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
- Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
- T/F: The answer choices are a collection of unrelated true/false statements.
- Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
- Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4. Check the appropriate box if a job content flaw is identified:
- Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- #/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 8 H 2 Y N S 9+ H 2 Y N S 10 H 3 Y N S 11 L 2 Y B S NRC 2015 12+ H 3 X X Y N ? Are applicants expected to know control rods locations within the RPV relative to nuclear instrument locations without a reference? RE: Yes, it is a learning S objective to know this information.
13 H 3 Y B S 1/2X2 14 H 2 Y N E MO 2-3704 valve is the RBCCW cooling return valve from the SDC heat exchangers. Is valve noun name correct? RE: Noun name is correct. 1/2X2 15 H 3 Y N S 16+ H 3 Y B S NRC 2008 17 H 2 Y N S 18 H 2 Y B S NRC 2009 19+ H 2 Y N S 20 H 2 Y N S 1/2x2 21+ F 2 Y N S 22 H 2 Y N S 1/2x2 23 H 2 Y B S 24+ F 2 Y N U B and D distractors not plausible. Replace? RE: May use diagrams from panels for operators to read and decide which is correct in lieu of giving pressure S
indications. RE: STET, testing knowledge of system pressures.
25 H 2 Y B S 1/2x2; NRC 2005 26 F 3 Y N S 27 F 2 Y N S 28+ H 3 Y B S 29 H 2 Y B S Assume question is from Bank 30 H 2 Y B S 1/2x2
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 31 H 2 Y N S 32+ H 3 X Y N U Remove U2 IC distractors. A. Have no effect; B. Will disallow refilling the IC shell side from CST or put radiation monitor in an alarmed state or ONLY vent valves 17 & 20 would close. RE: Added U2 to plant conditions in stem. 1/2x2 33 F 2 X Y N U No correct answer! Not plausible that C and D distractors would rise in value during a cooldown. Also, not plausible that distractor A & B Pt 3 would lower in value. Pt 3 would not rise above 200°F would rise above 220°F; Pts 9 & 10 would initially rise in value then slowly lower? RE: Changed distractors and added time limit to stem.
34 F 2 Y N S 1/2X2 35 F 2 Y B S Assume Bank question; NRC Exam 2008 36 H 3 Y B S 1/2X2: NRC Exam 2007 37+ F 2 Y N S 38 H 2 Y N S 39 H 2 Y N S Why is this a higher cog question. Appears to be from memory. RE: STET 40 H 2 Y N S 1/2X2 41 F 2 Y N S 42 H 3 Y N S 43+ F 2 Y N S 44 H 3 Y N S 45 H 2 Y B S 46+ H 3 Y N S 47 H 2 Y N S 48 H 3 X Y N U Replace A distractor with BOTH U2 and U3 battery voltage lowers, and add ONLY to distractors B & C. RE: Either replace distractors or add other unit conditions to stem. 1/2X2 49+ F 3 Y B S 2012 NRC exam 50 H 3 Y B S 51 H 3 Y N S 52 F 2 Y N S 1/2x2
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 53 H 2 Y B S 54+ H 2 Y B S NRC 2006 exam.
55 H 2 Y N S 1/2X2 56 F 2 Y B S 1/2X2 57 F 2 Y N S Reference for Q57 is DAN 902(3)-6, F-5. RE: Corrected 58+ H 2 Y N S 1/2X2 59 F 2 Y B S 60 H 2 Y B S 61 F 3 Y N S 1/2X2 62 F 2 Y N S 63+ H 2 Y N E Why is this a higher cog question? RE: STET 64 F 2 X Y N E All distractor end with to prevent MSL radiation monitors from spiking. This needs to be moved into the stem of the question and delete 2nd half of distractors. RE: Will move a portion of each distractor to the stem.
65 F 2 Y B S 2008 NRC exam 66+ F 2 Y B S 2012 NRC exam 67 F 2 Y B S 2009 NRC exam 68+ F 2 Y B S 69 F 2 Y N S 70 F 2 Y B S 2015 NRC exam 71 F 2 Y B S 2007 NRC exam 72+ H 3 Y B S 73 F 2 Y B S 1/2X2 74 F 2 Y B S 75 F 2 Y B S 76 F 3 Y Y B S 2010 NRC exam 77+ H 3 Y Y B S 2008 NRC exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 78 H 3 Y Y B S 1/2X2 79+ H 3 Y Y B S 2008 NRC exam 80 F 2 Y \Y B S 81 H 2 Y Y B S 2005, 2007 NRC exam 1/2X2 82 F 2 Y Y N S 83+ H 3 Y Y N S 84 F 3 Y Y B S 2009 NRC exam 85+ H 3 Y Y B S 2009 NRC exam 86 H 3 Y Y B S 87+ F 3 Y Y B S 88 H 3 Y Y N S 1/2X2 89 H 2 Y Y B E Typo in A distractor. B & C distractor explanations switched in Info block. RE Conditions corrected.
90+ F 3 Y Y N S 1/2X2 91 F 2 Y Y N E Since this ST is mode sensitive, do you need to include in Mode 1.? RE: No, not needed to add Mode 1.
92 H 2 X Y ? N U LOCA condition exists in stem, but actions of distractors are Tech Spec related.
If actual LOCA condition existed, SRO directions would not use TS time clock.
E Need to reference procedures to take action. Direct PCIS Group 5 isolation, check that Group 5 isolation occurred or remove accident condition from stem and change 8 hr distractors to 1 hr distractors. RE: Since question is technically correct, changed from U to E. Need to lessen urgency, add MAXIMUM to stem, and add time to trouble shoot. Question remains unchanged.
93 H 2 Y Y N S 94+ H 2 Y Y N E Need to include duplicating elements of the distractors added to stem. Stem will now look like fill in the blank with two blanks. RE: Will add portion of each distractor to stem. 1/2X2 95 H 3 Y Y N S 96 H 3 Y Y B S 2010 NRC exam; 1/2X2 97+ H 3 Y Y N S Would students know that this is a RPV level transmitter without noun name?
RE: Students expected to know this LT from memory, no noun name needed?
RE: Added noun name to question stem.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 98 H 3 Y Y N S C distractor explanation in Info box should read incorrect. RE: Corrected.
99+ H 3 Y Y B S 100 H 2 Y Y B S RO 1/2X2 questions: 19/75 = 25.3%
SRO 1/2X2 questions: 5/25 = 20%