ML060880213
| ML060880213 | |
| Person / Time | |
|---|---|
| Site: | Clinton  |
| Issue date: | 03/06/2006 |
| From: | Operations Branch IV |
| To: | |
| References | |
| Download: ML060880213 (6) | |
Text
ES-401 WC-01-2006 Written Examination Form ES-401-9 Review Worksheet Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 2
E Outline is missing the X in the K5 column for this question.
2 H
3 S
3 H
2 S
4B F
2 S
5 F
2 S
6B H
2 X
U IS BB-202 the PRT vent valve? Need reference BB-202 section 6.4 to validate this question. Still need more references-where does it say that drain to the RCDT is not allowed except in emergencies? Your normal procedure has directions to do just that, which would potentially make "D" also correct.
7 H
3 X
S 8
H 2
S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
One or more distractors is (are) not credible.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6.
Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
7.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
33 of 34 NUREG-1021, Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 9B F
2 S
10B F
4 X
X U
Reference provided does not support the answer or distractors. Utility provided more references, however they still do not answer the basic question of "fuel damage may occur...." (which is intent of K/A).
11B H
4 S
12 F
2 S
13 H
2 X
U K/A mismatch - CSS vs sump level not cavitation or sump blockage.
14B H
1 X
U too easy, LOD = 1. Suggest change to stem to incorporate knowledge of answer B.
15B H
2 X
X S
16 H
2 X
U Point value is less than 2.5 for this K/A (2.1). Select another K/A.
17B F
2 S
18 H
3 X
X U
Schematic provided does not verify answer as only correct answer. Utility provided more references, however they still do not explain connectivity between the chargers, the inverters, and the normal supply.
19B F
2 S
20 H
3 S
21 H
2 X
X U
Reference provided does not support the answer or distractors. Suggest using lights for other two distractors. Utility provided more references, however they still do not explain the fact that the diesel came up at a lower frequency than the bus (in this case) hence the rotational direction given for the answer "C" is correct. Need two distractors replaced, "A" and "B".
22B H
3 S
23B F
3 X
E Stem was not modified enough to meet the NUREG-1021 requirements for a modified question. Change this question to a bank question on the proper forms.
24 F
2 S
34 of 34 NUREG-1021, Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 25 H
2 S
26B F
2 X
E Remove and from distractor A. Switch two statements order on distractor B to complete symmetry on all selections.
27B F
3 X
X X
U Reference doesnt support link to K/A for shield bldg vent. system (dont believe it exists for this plant). K/A mismatch since this bldg doesnt exist for this plant 28 H
3 S
29B F
2 S
30B H
3 X
X U
Shouldnt wording in distractor B and answer C be loss of bistable instead of instrument? Loss of power to the bistables causes the event, not loss of power to the instrument.
31B F
4 S
32B H
4 S
33B H
3 X
X U
Distractor A is not credible. Also, reference provided does not validate this answer. Utility provided more references, however they still do not explain that GTRE-22 in HIHI state trips both trains of CPIS, which in turn shuts all purge dampers.
34 H
4 S
35 F
3 X
X S
Outline and Exam importance Rating is incorrect as 2.8. The IR from K/A catalog is actually 3.5 for this K/A.
36B F
4 S
37 F
4 S
38 H
3 X
X U
Reference provided does not validate this answer. No mention of fire in main transformer in this reference. Utility provided better references, however distractor "A" is not credible.
39 F
2 X
X E
D is not credible since it is not an action step like the other selections. Suggest modifying the stem to be more operational oriented and challenging.
40B F
2 S
41B F
4 S
42 F
3 S
34 of 34 NUREG-1021, Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 43 H
3 S
44 H
2 X
U Distractors C and D are not credible.
45 F
3 S
46 H
2 S
47B F
3 X
U Distractors C and D are not credible.
48 H
4 X
X U
A is also correct based on reference provided. S/G pressure is available but not indicated steam line press.
49 H
4 X
X S
50B F
3 X
X U
Is NG04 the bus that failed? Reference material does not support answer. Need more references to validate.
51B F
3 S
not cognitive question (fundamental).
52B H
2 S
53B F
3 X
X X
U Distractors B and C need same format as A to prevent cueing in answer A as worded. Suggest putting with the break isolated at the end of all three selections.
Also, I am not sure if the sentence on page 18 of 21 for document BD-EMG C-12 would present a challenge for multiple correct answers on this question.
54M F
3 E
remove only from answer explanation.
55B F
4 S
56B H
3 X
U K/A mismatch-does not test the reasons for responses as they apply to....
57B H
2 X
U Reference does not entirely support the answer. On second review, this is an unsat question. Need some calcs for pump vs tank drop rate or something more discriminatory.
58M H
3 S
59 F
3 X
U K/A mismatch-question does not test accidental release as worded. Suggest modifying the stem somewhat to include accidental release scenario.
60 F
3 X
U K/A mismatch-question does not test accidental release as worded. Suggest modifying the stem somewhat to include accidental release scenario.
61 F
2 S
62 F
3 S
34 of 34 NUREG-1021, Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 63 H
2 S
64B H
3 S
65B H
4 X
U SRO level question.
66 F
2 S
This question is borderline LOD = 1.
67 H
3 S
68B F
1 S
Although this is an easy question ( my original LOD = 1), they have an LER/CR on this topic and procedure changes as a result, hence justification to have it on exam. Changed LOD = 2.
69B H
3 S
70M H
2 S
71B F
3 S
72B H
3 S
This is a cognitive question, not memory.
73M H
2 X
X U
K/A mismatch-question does not have the applicant assess the status of CSF using logic and/or parameters. Also, one selection RED with others having lower colors is cueing as written.
74 F
2 S
75B H
3 S
2001 NRC Q73 RO exam.
76B F
2 X
S 77 H
3 U
RO knowledge, got Mark to provide second opinion and he agreed this is RO knowledge question.
78 H
3 X
S 79 H
3 X
S 80B H
4 X
S 81B H
3 X
S 82 F
3 X
X E
Question is fundamental, not higher or cognitive. Remove evacuation words from two places in stem. Cues the answer. Use other verbage.
83B H
4 X
S used on 2001 NRC exam, Q77
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 84B H
3 X
S 85 H
3 X
X E
Because SI occurred, ES-02 is not credible, which makes distractor A not credible.
86 H
3 X
S 87 H
3 X
U D is not a credible distractor. Question is not SRO knowledge. Mark reviewed and agrees that this is RO knowledge.
88B F
2 X
S 89B H
2 X
X U
Replace "C" distractor because of overuse of 2 SR NI. Use of 2 Source range instruments as a distractor (as in C) is used too much throughout this exam. If you know that 2 SR NI are required, then you can eliminate distractors several other questions.
90 F
2 X
S Put better explanation in explanation field.
91B H
2 X
X U
There are too many topics relating to the SR NI requirement of 2 instruments. See also Q89. Write a new question. Importance rating missing from exam (3.7). Review similar RO question and compare for this issue, then determine if one should be removed from exam.
92B H
3 X
S 93 H
3 X
S 94 H
3 X
S 95B F
2 X
S 96 F
2 X
S 97M F
3 X
S 98M F
2 X
S 99 F
2 X
S 100 H
2 X
E Explanation of answer needs edit to be correct. Should read Mode 3 with accumulators isolated or Mode 4.....
34 of 34 NUREG-1021, Revision 9