ML052300151
| ML052300151 | |
| Person / Time | |
|---|---|
| Site: | Fort Calhoun  |
| Issue date: | 08/25/2005 |
| From: | NRC Region 4 |
| To: | |
| References | |
| 50-285/05-301, ES-401-09, FCS-07/2005 | |
| Download: ML052300151 (6) | |
Text
ES-401 FCS-07-2005 Written Examination Form ES-401-9 Review Worksheet
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 X - U(S) B and C do not appear credible - would these actions EVER be required? (Fixed) 2 H 2 X - U(S) C and D are not credible; (Replaced) 3B F 3 - S NRC1997 4 F 3 - S 5 H 3 X - E(S) what credible error would lead to choosing A, C, or D? (Fixed) 6 H 3 - S 7B* H 3 - E(S) NRC1999; this does not qualify as a MODIFIED Q - count as BANK; should isolate be close instead?
8M* H 3 X X U(S) too similar to NRC2002#33 to be counted as new; B is not credible; doesnt test ability to interpret m/u flow as applied to PZR PRESSURE CONTROL MALFUNCTION per KA; (replaced with new (NEW) KA and Q; add only to A and B) 9 H 3 X - U(S) B is not credible; (Modified) 10B* H 3 - E(S) NRC2002 #12; this does not qualify as a MODIFIED Q - count as BANK; (MOD) (Replaced with modified Q)
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
@ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
@ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
@ The answer choices are a collection of unrelated true/false statements.
@ One or more distractors is (are) not credible.
@ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
@ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
@ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
@ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
@ The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
33 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 11B H 3 - S 12B* H 2 - E(S) this does not qualify as a MODIFIED Q - count as BANK; 13 H 1 X X X U(S) Q stem doesnt actually state that bus AI-40A has been lost, so Q doesnt make sense; why would anyone think that loss of power to indications would affect the SIT performance? Doesnt test ability to determine SIT press. and lvl after loss of power to instruments per KA; (Replaced with new KA and Q) 14 H* 2 - X U(S) LOK is H vice F; this is not RO level - it is SRO (55.43.b.5); (Replaced) 15B* H 3 - S(S) NRC2002 #116; (replaced with new question)
(NEW) 16 F 3 - E(S) change B from ...a air... to ...an air...;
17B H 3 X - U(S) A is not credible; (Changed) 18B F* 3 - S NRC1997; LOK is F vice H; 19 F 3 - S 20 H(F) 1 X - X U(S) Q is too easy - low discriminatory value; TS not typically RO-level;(fixed) 21B F 3 - S 22B H 2 X - U(S) NRC1997; is D possibly also correct? (Fixed) 23B F 3 - S NRC2002#62; 24B H 3 X - U(S) NRC2001-2; B is not credible; (Fixed) 25 F 3 X - U(S) C and D are identical and therefore not credible; (Fixed) 26B F 3 - S NRC2001-1; 27B* F 3 X - U(S) NRC2001-2; and NRC2002#106; C is not credible; (replaced; B also (NEW) correct; fixed) 28B F 3 - S 29B H* 2 X - U(S) A and C are not credible; (Fixed) 30 H 3 X X U(S) C isnt credible; doesnt test ability to predict (AND use procedures to correct/control/mitigate) the IMPACTS OF a T-ave or T-ref deviation ON CVCS per KA;(replaced) 31 F 3 - S 32 F 3 - S 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 33B H 3 X - U(S) NRC2001-1; C is not credible; (Modified, but C still not credible; fixed) 34 H 3 - E(S) Change stem from Whis... to What...; (Fixed) 35 F 3 - S 36 F 3 X - U(S) change stem from ...is: to ...in:; (Fixed) 37B F 3 - E(S) reference doesnt support answer; (More references provided) 38B* H 3 - S(S) NRC1995; and NRC2004#8; (Replaced with new Q)
(NEW) 39B F 3 X - U(S) NRC2002#86; C is not credible; (Fixed) 40B H 3 - E(S) NRC1999; change stem from Selected... to Select...; (Fixed) 41 H 3 X X - U(S) B is not credible; (fixed) 42M* H 3 - E(S) too similar to NRC2002#76 to be counted as new; change stem from offfsite power to offsite power; (Fixed) 43B H 3 - S NRC1995; 44 H 3 - S 45B* H 3 - E(S) NRC2004#41; this does not qualify as a MODIFIED Q - count as BANK; 46B H 2 - S NRC2001-2; 47B F 3 X - U(S) B is not credible; (Fixed) 48B F 3 - S 49B* H 3 - S(S) NRC2004#91; (Replaced with new Q)
(NEW) 50B* F 2 X - E(S) NRC2002#109; delete second sentence of stem; (Fixed) 51 H 3 - S 52B H 3 - S NRC1997; 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 53 F 3 - S 54B H 3 - S NRC1997; 55B H 3 - S NRC2001-1; 56B* F 3 - E(S) NRC1999; and NRC2002#85; (replaced; typo in stem; fixed)
(NEW) 57B H 3 X - U(S) NRC1997; A is not credible; (Fixed) 58B H 3 X - U(S) NRC1997; A is not credible; (Fixed) 59M* H* 2 X - U(S) too similar to NRC2002#68 to be counted as new; LOK is H vice F; A and C are not credible; (NEW) (replaced with new) 60B F 2 - S NRC2001-1; 61B H 2 X - U(S) NRC1995; A is not credible; (Fixed) 62B* H 3 X U(S) NRC2001-1; and NRC2002#64; doesnt test the ability to predict the impact of SBLOCA AND use (NEW) PROCEDURES... per KA; (Replaced) 63 H 3 X - U(S) D is not credible with none in the wording; (Fixed) 64B H 3 X - U(S) NRC2001-2; change causing to causes in stem; D is not credible since logic eliminates it as a possibility; (Fixed) 65B H 3 X U(S) NRC2001-2; doesnt test knowledge of the effects of the FPS MALFUNCTION on the detectors (NEW) per KA; (Replaced; eliminate the word suppression from the stem; fixed) 66 F 3 X - E(S) reference doesnt support answer; (Additional reference provided) 67B F 1 X - U(S) NRC1999; this has no discriminatory value; (Replaced)
(NEW) 68B H 2 X - U(S) NRC1995; is C credible? (Fixed) 69B F 2 - S NRC1997; 70 F 1 X U(S) doesnt test knowledge of RO responsibilities in CR during fuel handling per KA; (Replaced) 71B H 2 X - U(S) NRC1999; stem doesnt indicate what percentage of exposure was due to neutrons; (Replaced)
(NEW) 72 F 2 X X - U(S) A is not credible; why isnt D correct? (Fixed) 73B F 2 X - U(S) NRC2004#4; B is not credible; (Replaced with new Q)
(NEW) 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 74M F 2 X X - U(S) the way the question is worded, there is no difference between A and D (or B and C); (Fixed) 75 F 3 - S 76 F 3 - X U(S) this is RO-level (55.41.b.10); (Replaced) 77M H 3 - - E(S) NRC2004#5; minor typo; 78 H 3 - X U(S) this is RO-level (55.41.b.10); (Replaced; BCD not credible; typo; fixed) 79 H 3 - X U(S) this is RO-level (55.41.b.10); (Replaced) 80 H 4 - - S 81 F 3 - - S 82B H 3 - X U(S) NRC2004#20; reference doesnt support answer; this is RO-level (NEW) (55.41.b.10); (Replaced) 83 H 3 X - X U(S) B is not credible; is PRA knowledge specifically an SRO learning objective? (Replaced) 84B* F 3 X X U(S) NRC2002#65; this does not qualify as a MODIFIED Q - count as BANK; (NEW) this is RO-level (55.41.b.10) - it was RO#63 on NRC2002; (Replaced) 85B F 3 - - E(S) NRC2001-1; change stem from When would you... to When would you be required to... (Fixed) 86 H 3 X - X U(S) B and D are not credible; this is RO-level (55.41.b.10); (replaced) 87B F 4 X X X - - U(S) answer is cued by its obvious safety significance in comparison to the three distractors (making (NEW) them not very credible); is this something an SRO should know from memory? (Replaced) 88B* F 3 X - X U(S) this is RO-level (55.41.b.10); not a MODIFIED Q - count as BANK; not (MOD) sure B is credible; (Modified Q) 89 H 3 X - - E(S) C is not credible (recommend 0600 on April 3"); (Fixed) 90 H 3 X X X X U(S) is H2 production really THE most SIGNIFICANT potential consequence? A and B are not credible; doesnt test ability to explain/apply SYSTEM LIMITS and PRECAUTIONS per KA; this is RO-level (55.41.b.5,8,9); (Modified) 91 H 3 X - X U(S) D is not credible; this is RO-level (55.41.b.10); (Replaced) 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 92B F 3 X - - E(S) NRC2001-1; reference to NRC performing the analysis makes A not credible; (Fixed) 93B F 3 X - - E(S) not sure what is meant by ...MAY stop when the weight of the hoist box is LOST... - what does MAY and LOST mean in this stem? Reference doesnt support answer and doesnt explain (NEW) why distractors are wrong; (Replaced) 94B* F* 2 X - - U(S) NRC1997; not a MODIFIED Q - count as BANK; LOK is F vice H; reference indicates that immediate action to restore shift crew composition is required - D is not correct; (Fixed) 95M H 3 X X - - U(S) the backup sample cues that the value is out of spec - this makes A not credible; (Fixed) 96 H 2 - - S 97B* F 2 X - U(S) NRC2001-1; not a MODIFIED Q - count as BANK; KA on Q doesnt match outline; unless this is a specific SRO-only learning objective, this is not SRO-only; (Fixed; this is an SRO learning objective) 98M H 2 X - U(S) KA on Q doesnt match outline; (fixed) 99B F 3 - - S NRC2001-1; 100B* H 3 - X U(S) NRC2002#19; this is RO-level (55.41.b.10); typo on D; (Replaced; not (NEW) SRO-only; fixed)
NOTE: Bank questions are indicated by B; Modified are indicated by M; comment resolution is indicated in blue; Previous 2 NRC exams: 2004 and 2002 34 of 34 NUREG-1021, Draft Revision 9