ML042990553
| ML042990553 | |
| Person / Time | |
|---|---|
| Site: | River Bend  |
| Issue date: | 09/23/2004 |
| From: | Gody A Operations Branch IV |
| To: | Hinnenkamp P Entergy Operations |
| References | |
| 50-458/04-301, ES-401, ES-401-9, NUREG-1021, Rev 9 50-458/04-301 | |
| Download: ML042990553 (6) | |
Text
ES-401 RB 09/2004 Written Examination Form ES-401-9 Review Worksheet
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1B H 3 - S from RB NRC exam 10/2000; 2 H* 3 - E(S) author says LOK=F, but I think LOK=H. (S)-fixed.
3 H 3 - S 4 H 3 - E(S) Correct grammar and punctuation errors in stem by changing to: During a plant startup, after the generator has been placed on the grid, a turbine trip occurs. . . and steam line drains stabilize reactor pressure. . . (S)-fixed.
5 H 3 X - E(S) recommend rewording distractors A and B to look more like C (ie: A. rise as control rod insertion lowers core void content.) (S)-fixed 6 H 3 X - E(S) indicate what time the CRS gave direction; (S)-fixed 7 H 3 - S 8 H 3 - S 9 F 2 - S 10 H 3 - E(S) typo in stem - change to: ... the Reactor Cavity where it strikes... (S)-
fixed Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
@ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
@ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
@ The answer choices are a collection of unrelated true/false statements.
@ One or more distractors is (are) not credible.
@ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
@ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
@ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
@ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
@ The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
33 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 11M F 3 - S from RB NRC exam 2/2003; 12 F 2 - S 13M F 2 - S from RB NRC exam 2/2003; 14 F 2 - S 15B F 2 - S from RB NRC exam 1/1993; 16 H 2 - S 17 F 2 - S 18 F 2 - E(S) fix typo in stem by: ...below the Power box as follows:; (S)-fixed 19 F 2 - S 20 F 2 - S 21M F 2 X - U(S) from RB NRC exam 2/2003; distractor A is not credible. (S)-distractor fixed and question converted to modified (was originally bank) 22 H 2 - S 23 F 2 - S 24B H 3 - S from RB NRC exam 2/2003; 25 F 2 - S 26 F 2 - S 27 F 3 - S 28 F 3 - S 29M F 3 - S from Perry NRC exam 1994; 30M H 3 - S from RB NRC exam 2/2003; 31 H 2 - S 32 F 2 - S 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 33 H 4 - S 34B H 2 - S from RB NRC exam 7/1997; 35 H 2 - S 36 H 3 - S 37 H 3 - S 38 H 3 - S 39 H 3 - E(S) fix typo in stem to: Following a plant... (S)-fixed 40B F 2 - E(S) from RB NRC exam 7/1997; fix typo in distractor A to: ...hand switch in the control room... (S)-fixed 41 H 3 - S 42 F 3 - S 43 H 2 - S 44 H 3 - S 45 H 3 - S 46 F 3 - S 47 F 3 - S 48M H 2 - S from RB NRC exam 10/2000; 49 F 2 X - U(S) three distractors can be eliminated by cues from handout. (S)-replaced with satisfactory NEW question.
50B H 3 - E(S) from RB NRC exam 10/2000; fix typo in distractor B to: Due to a... (S)-
fixed 51B H 2 - E(S) from PB2 NRC exam 9/2002; is diesel engine emergency shutdown signal a separate signal from energizing thediesel stop solenoid? Also, capitalization of diesel/Diesel needs to be uniform. (S)-diesel shutdown language clarified/fixed.
52 F 2 - E(S) change A and B to ...operates alone to maintain... (S)-fixed 53 F 2 - S 54 H 3 - S 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 55 F* 2 - E(S) author says LOK=H, but I think LOK=F. (S)-fixed 56 H 3 - S 57 F 2 - S 58 F 3 - S 59 H* 2 - E(S) author says LOK=F, but I think LOK=H. (S)-fixed 60 F 3 - S 61 H 3 - S 62M H 3 - S from RB NRC exam 2/1999; 63B H 3 - S from RB NRC exam 7/1997; 64 F 2 - S 65B H 3 - S from RB NRC exam 2/2003; 66M H 2 X - U(S) from RB NRC exam 2/2003; distractor A is not credible; perhaps eliminating descriptive verbiage in distractors and just use Mode-2, etc?
(S)-fixed; question is now modified (was originally bank) 67M F 2 - S from BF NRC exam 6/1995; 68B F 2 - S from RB NRC exam 1/1997; 69B H 2 X U(S) not RO license level question. (S)-replaced with bank question from RB NRC exam 2/2003; question was originally new.
70 F 2 - S 71B H 2 - S from RB NRC exam 7/1997; 72 H 3 - S(S) from RB NRC exam 2/2003; (S)-replaced with new question to reduce the number of bank questions from previous two NRC exams.
73 F 2 - S 74B F 2 - S from RB NRC exam 1/1995; 75 H 3 - S 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 76B F 2 X - U(S) from RB NRC exam 2/2003; K/A listed on the Question does not match K/A listed on outline, and question does not match outline K/A. (S)-fixed 77 H 3 - X U(S) not SRO-only - more like 55.41.b.5 than 55.43. (S)-fixed 78 H 2 - X U(S) from RB NRC exam 7/1997; not SRO-only - more like 55.41.b.10 than 55.43. (S)-replaced with new question at SRO level (was bank) 79 H 3 - - S 80 H 3 - X U(S) fix typo in first sentence of stem by deleting is; not SRO-only - more like 55.41.b.10 than 55.43. (S)-replaced with new SRO-level question 81M H 2 - - S from RB NRC exam 10/2000; 82M F 2 - - S from RB NRC exam 2/1999; 83 H 3 - - S 84 H 3 X - - E(S) are SROs expected to know the PSP curve by memory? (S)-modified to include curves as handouts.
85 F 3 X - - U(S) use something other than - in front of numbers in the stem so that no one can be confused as to whether it is a negative sign; distractor D seems to be also correct even though A is a better answer. (S)-fixed and added EP-3 as handout.
86 H 2 - - S 87B H 2 - - S from RB NRC exam 10/2000; 88 F 2 - - S 89 H 2 - - S 90 F 2 - X U(S) from RB NRC exam 10/2000; not SRO-only - more like 55.41.b.7 or b.10 than 55.43. (S)-replaced with new SRO-only question 91 H 2 - X U(S) from Perry NRC exam 3/2002; not SRO-only - more like 55.41.b.10 than 55.43. (S)-replaced with new SRO-only question 92B H 3 - - S(S) from RB NRC exam 2/2003; (S)- modified this question to reduce the number of bank questions repeated from previous NRC exams 93M H 2 - - E(S) from FitzPatrick NRC exam 7/2003; fix typo in stem with: ...bundle is being...; (S)-fixed 94B F 2 X - - U(S) distractor C is not credible - try using SRO with different notification time.
(S)-fixed 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 95M F 2 - - S from RB NRC exam 1/1997; 96 F 2 X - - E(S) change stem to ...modification must always have prior... (eliminates possibility of assumptions leading to multiple answers) (S)-fixed 97 F 2 - X U(S) not SRO-only - more like 55.41.b.10 than 55.43. (S)-replaced with new SRO-only question 98M F 2 - - E(S) from RB NRC exam 2/2003; author says Modified, but not sufficiently changed from parent. (S)-more changes made to make this a modified question (was bank) 99M H* 3 - - E(S) from RB NRC exam 10/2000; author says LOK=F, but I think LOK=H.
(S)-fixed 100M H 3 - X U(S) from RB NRC exam 2/2003; Not SRO-only. (S)-changed to make it SRO-only and modified (was bank).
34 of 34 NUREG-1021, Draft Revision 9