ML14248A493
| ML14248A493 | |
| Person / Time | |
|---|---|
| Site: | Byron  |
| Issue date: | 06/13/2014 |
| From: | Walton R NRC/RGN-III/DRS/OLB |
| To: | Exelon Generation Co |
| Shared Package | |
| ML14140A562 | List: |
| References | |
| Download: ML14248A493 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 BYRON 2014 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1+ H 2 N S 2 H 3 N S 3 F 2 B S 4 F 2 B S 5+ F 3 M S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
ES-401 Written Examination Review Worksheet Form ES-401-9 BYRON 2014 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 6 H 3 N S 7 H 3 N S 8+ F 3 M S 9 H 3 B S 10 F 3 X B U Question should be slanted towards what are operator actions without reference to procedures! Plausibility of A & B distractors?
RE: Changed distractors to eliminate procedure usage, just panel actions operators are to take.
11 F 3 N S 12+ F 2 B S 13 H 3 N S 14+ H 3 X B U Unrelated valves, T/F statements. Question can be answered without information in the stem. RE: changed question 15 F 3 ? B E Consider changing last sentence in stem to, Leak is isolated if.. The sentence would read better. If the LOCA was large enough, C could be a correct answer! RE: Grammar corrected. LOCA size discussed.
16 F 2 B S 17+ H 3 N S 18 H 2 B S 19+ H 4 N S 20 F 3 X B U How does this match K/A of pressurizer level malfunction? Good Tier 1, Group 2 question for Natural Circulation Cooldown w void in E
RPV! (W/E K10 G2.4.21?) RE: This is malfunction of pzr level not level instrument! OK as is!
First sentence sounds like you are teaching during the exam.
Change first sentence to give plant conditions. This also looks close to an SRO only question (procedure direction!) RE: Not a pzr level CONTROLLER malfunction but malfunction with pzr level. ROs knowledgeable on status trees. REl changed stem.
21 F 2 B S Second part of distractors should be in a separate sentence.
ES-401 Written Examination Review Worksheet Form ES-401-9 BYRON 2014 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 22+ H 4 X N U NOT and RO question since requires knowledge of which procedure to direct! (See Figure 2, SRO Guidance Document, pg 8 of 16).
What actions must operators take?, not a procedure choice! RE:
Not an operator takes action question, take out procedure reference and ask WHY they are terminating SI. RE: Distractors changed to remove reference to procedure choice.
23 F 2 B S 24 H 2 N S 25 H 2 N S Not a higher level cog question!
26+ F 3 B S 27 F 3 N S 28+ H 3 B S 29+ H 2 M S 30 H 3 M S 31 F 2 N S 32+ H 3 B S 33 H 3 B S 34 H 3 B* S *From past NRC Exam 35+ H 3 X B E Need to include status of U2 CC pump/hxer. RE: Status of U2 CC pumps (operable or not) does not change the answer to the question.
S No change necessary.
36 H 3 B S 37 H 2 N S 38 F 2 B S 39+ H 2 B S 40 H 2 B S 41+ H 3 N S
ES-401 Written Examination Review Worksheet Form ES-401-9 BYRON 2014 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 42 H 3 N S 43 F 2 B S 44+ H 3 X N U C & D distractors are not in A.2 format; predict and use procedure.
Predict the malfunction and what to do. RE: Changed distractors such that all are in A.2 format.
45 F 2 N S Reference E print provided looks like Bus 141 feeds AFW pump start circuitry.
46 F 2 B S 47+ H 3 N S 48 F 2 B S 49 H 3 B S 50+ F 2 B S 51 F 2 B S 52 F 2 N S 53 F 2 N E D distractor should read, above its normal range and the IA receiver relief valves are NOT open. Keep as is!
S 54+ H 2 B S 55 F 2 N S 56 H 3 N S 57+ H 2 N S 58 F 2 B S 59 F 3 N S 60 H 2 N S Are you allowed to operate the ctmt mini-purge system in Mode 1?
RE: Yes, can be run in MODE 1!
61+ F 3 B S Are actions from RSDP different than CR? RE: Yes this is required to be known from memory (continuous actions).
ES-401 Written Examination Review Worksheet Form ES-401-9 BYRON 2014 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 62 H 2 N S 63 H 3 N S 64+ H 3 B S 65 F 2 B S 66 F 2 N E Capitalize MUST in A distractor. RE: MUST capitalized.
67+ F 2 B S 68 H 2 B S 69 H 2 B S 70+ H 3 B S 71 F 2 B S 72 F 2 X B U All are correct answers! Postings are generic NGET. Something more challenging than postings. RE: question replaced.
73+ F 2 B S 74 H 2 N S 75 H 3 N S 76+ F 2 N S 77 H 3 B S 78 F 2 N S 79+ F 3 B S 80 H 3 B S 81 H 3 N S 82 H 3 N S 83+ H 3 X N U Distractors B & C are inplausible. RE: Question modified such that there were no implausible distractors.
ES-401 Written Examination Review Worksheet Form ES-401-9 BYRON 2014 Exam
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 84 F 3 N E Change B distractor to read, Keeping the linear Heat Rate below the point at which fuel centerline melting occurs. RE: DONE!
85 H 3 N S 86+ H 2 B S 87 H 3 N S Reference does not explain condition in question, RWST vs.
accumulator! RE: Question acceptable, reference for chief review was missing!
88 H 2 N S 89 H 3 N S 90+ H 3 B* S 91 F 2 B S 92 F 3 B* S 93 H 3 N E H LOK due to diagnosis of problem!
94 H 3 B S 95+ F 3 B E Too close to Q81. Ask a question on basis of refueling bridge interlocks? RE: Question was replaced.
96 F 2 No! N U NOT an SRO-only question since question requires knowledge above double line in TS! (See Figure 1, SRO Guidance Document, pg 5 of 16). Which TS to enter, more conditions, SRO actions to take. RE: Question was replaced.
Memory question since dont need info in first part of sentence to answer question.
97 F 3 N S 98 F 2 B S 99 H 3 B S 100 F 2 N S