ML071560220
| ML071560220 | |
| Person / Time | |
|---|---|
| Site: | South Texas  |
| Issue date: | 05/18/2005 |
| From: | Gody A Operations Branch IV |
| To: | Sheppard J South Texas |
| References | |
| 50-498/05-301, 50-499/05-301, ES-401, ES-401-9 50-498/05-301, 50-499/05-301 | |
| Download: ML071560220 (6) | |
Text
ES-401 STP-11-2005 Written Examination Form ES-401-9 Review Worksheet (password: Outline1) (TAC#:X02291/2292 )
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 X X X U stem needs to identify what actions have already been completed before asking what actions should be next; A appears to be correct; doesnt test knowledge of INTERRELATIONS btwn Rx trip and STATUS PANEL per KA; 2 F 3 X U* doesnt test ability to operate FW pumps for VAPOR SPACE ACCIDENT per KA; recommend characterizing the accident as Vapor Space; 3 H 3 X - U* dictating in the stem to use the 68F curve is an unnecessary cue - recommend deleting the
...per Plant Curve... portion of the cue; slowly is misspelled in the cue; 4 F 3 - S 5 F 3 - S 6 F 3 X U Q doesnt address LOSS OF RX COOLANT MAKEUP per KA; 7 H 3 X - U* C isnt credible since it doesnt deal with letdown; replace drop with return to normal for both B and C; 8 H 3 - S 9 H 2 - S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
@ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
@ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
@ The answer choices are a collection of unrelated true/false statements.
@ One or more distractors is (are) not credible.
@ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
@ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
@ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
@ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
@ The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
33 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 10B H 3 X - U* are B and D credible?
11 F 3 - E be consistent with SG nomenclature (SG 1D or SG D); change C to verify automatic closure... and D to manually close...;
12B F 3 X X X U doesnt test ability to operate HPI during loss of FW per KA; 13 F 3 X X U C is also correct; doesnt test knowledge of REASONS for EOP actions per KA; 14M? H 3 X - U* need parent Q to determine if it is modified; change stem question to simply ask what the SCM is; change C and D to be subcooling based on the other temperatures provided; 15B H 3 - S 16B H 3 - S NRC 2001; 17B H 3 X U doesnt test knowledge of REASONS for EOP actions per KA; 18 F 3 X - U* B, C, and D dont seem credible; 19B H 3 X U doesnt test knowledge of REASONS for relationship btwn PZR PRESSURE INCREASE and charging/letdown imbalance per KA; 20B H 3 - E make the answer column headings Switch in Normal and Switch in Bypass for clarity; 21 F 3 X - U* change stem to ...NOT an entry condition to ...; C is the only listed answer that is not listed as an entry condition for this procedure - B is not the correct answer; 22B H 3 X - U* A and B should be 484F, otherwise no correct answer; explanation of why A is incorrect has a mistake; 23B H 3 - S 24B H 3 X - E this is the kind of question that directly fits 55.43.b.5 for SRO-only knowledge - seems a little beyond the scope of RO - is there a learning objective that supports an RO needing to know this?
25B H 2 - S 26 F 3 - S 27B F 3 X - U* B is not credible; 28 F 2 X - U* A is not credible; 29 F 3 - S 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 30B H 3 X - U* D is also technically correct; recommend rewording to say VCT level will continue to lower until it causes the operating CCP suction to automatically align to the RWST at 3%
31 F 3 - S 32 H 3 X - U* A and D are not credible (increases); try decreases/decreases and remains the same/remains the same; 33 F 3 - S 34 H 3 - E this question doesnt really test knowledge of the isenthalpic expansion - I recommend adjusting the question to a starting pressure of 1400#; typo in stem (delete second at);
35 H 3 X - U* A is not credible (adverse conditions in stem conflict with no action);
36 F 2 - S 37M? H 3 X - U* need parent Q to validate modified; A and C are not credible (try replacing increasing with remains constant);
38B F 3 - S 39B H 3 - S 40B F 3 - S 41B F 3 - S NRC 2001; 42B F 3 - S 43 F 3 - S 44 F 3 - S 45 H 3 - S 46B F 3 X - U* C is not credible (try cannot be closed but can be tripped from...);
47 H 3 - S 48M? H 3 X - E need parent Q to validate modified; should there be a number associated with both air receivers mentioned in the stem?
34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 49 F 3 X - U* C is not credible; 50 F 3 - S 51 H 3 X X X U A is not credible; it is not clear why B and C are not also correct; doesnt test knowledge of POWER SUPPLIES to ESF MOVs per KA; 52 F 3 - E should B indicate which ESF DG?
53 F 3 - S 54B F 3 X - U* D is not credible; 55 F 3 X X - U* rephrase stem to eliminate isolate - it cues the right answer; A is not credible; 56 F 3 X - U* A and C wording is awkward (...other control banks THROUGH...);
57 H 3 - E add emergency in front of boration on A and C; 58 H* 3 - E LOK is H vice F; 59B H 3 X X X U Add Indicated CET temperature will: to end of stem; C and D are not really distinct; doesnt test ability to monitor CETs to determine RCP OPERATION per KA; 60M? H 3 X X - U* need parent Q to validate modified; change stem to Containment Hydrogen concentration is 2%; C is not credible; 61 H 3 - S 62B F* 3 - E LOK is F vice H; 63 H 3 X X X U B is not credible; doesnt test ability to monitor AUTOMATIC operation of ARM in changing ventilation alignment per KA - should probably delete KA; answer is cued by Q#92; 64 F 3 X - U* C is not credible since it is practically the same as B; D is not credible; 65 H 3 - S 66B F 1 - U* too easy; try asking which is NOT an approved way and use three of the approved ways as distractors and A or D as the correct answer; 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 67 F 2 - S 68B F 3 - S 69 H 4 X - U this is SRO-only - not appropriate for RO exam unless a specific learning objective is available; 70 F 3 - E uncouple is misspelled in A; 71B H 3 - S 72B H 4 X X - E stem doesnt make sense (how can one guy do the job at twice the distance from the job as two guys?); distractors need to be based on credible errors - recommend 83mrem for B and 41mrem for D based on failing to square the distance factor; 73M? H* 3 - E LOK is H vice F; 74B F 2 - S 75B F 3 - S 76 H 2 - - S 77 F 3 X - U doesnt test knowledge of outside reporting requirements as regards LOSS OF RX COOLANT MAKEUP per KA; 78 H 3 - X U not SRO-only (this is 55.41.b.10);
79 H 3 - X U not SRO-only (this is 55.41.b.10);
80 H 3 X - U doesnt test ability to REDUCE RAD LEVELS/EXPOSURE per KA; 81 F 3 X - U doesnt test knowledge of EOP usage regarding LOSS OF SECONDARY HEAT SINK per KA; 82 H 1 X - - U this is a direct look-up; need to indicate specifically which reference will be provided; replace it with power in C and D; 83 F 3 X X - X U not SRO-only (this is 55.41.b); A and B are not credible due to cues from Q#92 that these ARMs do not provide CVI; 84 H 3 - X U not SRO-only (this is 55.41.b.10);
85 H 3 - - S 86B H 3 - - S 34 of 34 NUREG-1021, Draft Revision 9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 87 H 3 X - X U B and C are not credible since C is a subset of B; not SRO-only (this is 55.41.b);
88? H 3 X X U Q source not indicated; change accident to event in stem; doesnt test ability to predict the impact of or use procedures to mitigate FEEDING A DRY SG per KA; not SRO-only (this is 55.41.b.10);
89 H 3 - - S 90 F 3 - X U not SRO-only (this is 55.41.b.10);
91 H 1 X X - - U* this is a direct look-up; C and D are not credible; 92 H 3 - - S 93 F 3 - - S 94 F 2 - - S 95 H 3 X - - E is the stem worded correctly for indicating that the Rx is still at power - is this an ATWS?
96 H 3 X X X - - U* wording of stem allows for both C and D to be correct (C is a subset of D); add only to A to make it distinct from B; 97 H 3 - - E change B and C to ...November 15, 0000...;
98 F 3 X X - X U delete first two sentences of stem; B is not credible; not SRO-only (this is 55.41.b);
99 F 4 X - - U* replace exposure limit with needed exposure in stem; add only to A and B and add or Emergency Director to C and D to make them credible; 100 F 3 - - E this may arguably be SRO-only, but it is not 55.43.b.5; RO 28B 37H 5 3 0 21 7 1 0 0 0 10 0 32U RO TOTALS 5M 38F 12E SRO 1B 16H 3 3 0 6 1 0 0 0 0 4 8 15U SRO TOTALS 0M 9F 3E
GENERAL COMMENT
S:
- 1. There are too many Qs dealing with containment ventilation/purge (50, 63, 71, 83, 92)
NOTE: Bank questions are indicated by B; Modified are indicated by M; comment resolution is indicated in blue; 34 of 34 NUREG-1021, Draft Revision 9