ML11200A093
| ML11200A093 | |
| Person / Time | |
|---|---|
| Site: | Quad Cities  |
| Issue date: | 06/30/2011 |
| From: | Moore C Operations Branch III |
| To: | Exelon Generation Co, Exelon Nuclear |
| Shared Package | |
| ML11167A123 | List: |
| References | |
| 50-254/11-301, 50-265/11-301 | |
| Download: ML11200A093 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Quad Cities June 2011 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 Y B S 2 H 3 Y M S 3 F 2 Y N S 4 H 2 Y M S 5+ F 3 Y B S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
ES-401 Written Examination Review Worksheet Form ES-401-9 Quad Cities June 2011 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 6 H 3 Y B E Need to add open quotation mark () to QCOA 0010-05 Title. RE:
Added opening quotation mark.
Licensee: Incorporated comment.
7 F 3 Y N S 8+ H 3 Y B S 9 H 3 Y B S 10 F 2 Y N S 11 F 3 Y B E With conditions given in stem, what is position of 37A valve? Is 37A valve already open? Shouldnt the 2nd part of the stem read, what would cause 37A valve to go closed?
Licensee: With the given conditions, 37A valve is interlocked closed.
At -191 (2/3 core height) have to take to ON. No changes reqd.
12+ H 2 Y M S 13 F 2 Y N S 14 H 2 Y N S 15 F 2 Y B S 16 F 2 Y B S 17+ H 3 Y B S 18 F 2 Y B S Is RO responsible for selecting the proper implementing procedure?
19+ F 3 Y Y B U Distractors B & C not plausible as written.
Licensee: LO SR-0001-K05 covers both SRO and RO and is listed at end of distractor description. A justification description precedes the LO. Rewrote distracters B and C, modified D.
20 H 3 Y N S 21 H 3 Y M S 22+ H 3 Y N S 23 F 3 Y B S
ES-401 Written Examination Review Worksheet Form ES-401-9 Quad Cities June 2011 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 24+ H 2 Y B S 25 F 2 Y B S 26 F 3 Y N S What does Gp 1 isol valve reset switch look like? Not Gp 2 & 3 since RWCU has no outbd isolation valves?.
Licensee: Reviewed Gp 1 (MSIVs) switch which is a separate switch. Plausible distracters as described after the question. No change to question.
27 F 2 Y M S 28 H 3 Y B S 29+ H 3 Y B S 30 F 2 Y N S 31+ H 3 Y B S 32 F 3 ? B S 33 H 3 Y B S 34 H 2 Y N E How is this a higher level question?
F Licensee: Incorporated comment, changed LOK to F.
35+ H 3 Y B S 36 F 3 Y M E How is this a memory question when 2 items must be known?
H Licensee: Incorporated comment, changed LOK to H.
37 F 3 Y B E How is this a memory question when 2 items must be known?
H Licensee: Incorporated comment, changed LOK to H.
38+ H 2 X Y B U What do the bulleted conditions do to change the distractor?
Eliminate bulleted conditions?
Licensee: Incorporated comment, significantly modified stem and bullets. Stem changed to remove the terms LOCA and LOOP (make applicant determine); bullets modified to reflect only RWL, DW pressure and loss of off-site power parameters and/or conditions.
39 H 3 Y N S
ES-401 Written Examination Review Worksheet Form ES-401-9 Quad Cities June 2011 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 40 H 3 Y M S 41 H 3 Y N S 42+ H 2 Y N S 43 F 3 Y M S Move Fuel Pool Cooling Permissive Sw. in Override to stem to parallel other bulleted conditions.
Licensee: To do so would change answer. Keep Q as is.
44 H 3 Y B S 45+ F 3 Y B S 46 F 2 Y B S 47 H 2 X Y B U Why is B distractor considered incorrect when references shows it to be a viable option?
S Licensee: Stem asks for PREFERRED as described in QCOP300-28, B.28, all rod insertion is PREFERRED. Keep Q as is.
48 H 3 Y B S 49+ H 3 Y B S 50 H 2 Y N S 51+ F 3 Y B S 52 H 3 Y B S 53 H 4 Y B S 54+ H 4 Y N S CRD PCV normally in Manual?
Licensee: CRD PCV is manual valve.
55 H 2 Y B S 56 H 2 Y B S 57 F 2 Y B S 58 H 2 Y B S 59+ H 2 Y N S
ES-401 Written Examination Review Worksheet Form ES-401-9 Quad Cities June 2011 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 60 F 3 Y N S Need to review QCOA 1900-02.
Licensee: Reviewed question stem and distracters. Answer is correct and distracters are plausible.
61 F 3 Y N S 62+ H 3 Y M S 63 H 3 Y M S 64 H 3 Y N S 65 H 3 Y B S 66+ F 2 Y N S 67 F 2 Y B S 68 F 2 Y B S 69 F 2 Y B S 70+ F 2 Y N S 71 F 3 Y N S 72+ F 2 Y B S 73 F 2 Y B S Are any of the other distracters Immediate actions steps for other QCARPs?
Licensee: Verified distracters are not IA steps for other QCARPS.
Leave question as is.
74 F 2 Y B S 75+ F 2 Y B S For RO portion of Exam: New = 22Q, Modified = 10Q, Bank = 43Q (57%): LOK: 34/75 = 45%, H= 55%
76 H 3 Y Y N S 77 H 3 Y Y M S 78+ H 3 Y Y N S
ES-401 Written Examination Review Worksheet Form ES-401-9 Quad Cities June 2011 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 79 F 2 Y Y B S Why is this a memory question. TS reqd to answer first part of question and knowledge required to answer 2nd part of question.
Licensee: LOK not H. TS knowledge (memory) used to eliminate 1st distractor. Knowledge (memory) of TS Basis used to answer question. Keep Q as is.
80+ H 2 Y Y M S 81 H 2 Y Y B S Why are references provided for this question?
Licensee: References provided to allow SRO to sort through various EOPs and determine which has higher priority. Not required to memorize SAMGS. Distractors are based on actions that have occurred, need QGAs.
82 H 2 Y Y M S 83+ H 2 Y Y B S 84 H 3 Y Y N S Is Max Normal the same as Alarm on Table T, QGA 300?
Licensee: Yes.
85 H 3 Y N N U Arent ROs supposed to memorize immediate actions of TS? Why are references provided since immediate actions of TS are required to be memorized? (Operability determined by SRO)
Licensee: Part of question is the timeliness (ie, immediate) which is both RO and SRO, but part of question is about operability which is SRO.
86+ H 3 Y Y N S 87 H 2 Y Y N S 88+ H 3 Y Y N S 89 H 3 Y Y N S 90 H 4 Y Y M S 91+ H 2 Y Y N S 92 H 2 Y Y M S 93 H 3 Y Y M S
ES-401 Written Examination Review Worksheet Form ES-401-9 Quad Cities June 2011 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 94 H 2 Y Y B S 95+ H 3 Y Y B S 96+ H 2 Y Y B S 97 H 2 Y Y M S 98 F 3 Y Y N S 99+ F 3 Y Y B S 100 F 2 Y Y B S