ML050330300
| ML050330300 | |
| Person / Time | |
|---|---|
| Site: | Columbia  |
| Issue date: | 03/23/2004 |
| From: | Gody A Operations Branch IV |
| To: | Parrish J Energy Northwest |
| Shared Package | |
| ML040840180 | List: |
| References | |
| 50-397/04-301 | |
| Download: ML050330300 (8) | |
Text
ES-401 CGS 09/2004 Written Examination Form ES-401-9 Review Worksheet REVISION-1 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 3
S 2
H 3
E (S) only the odd pages of the applicable TS have been provided; RO importance rating is 2.9 vice 4.0 REV-1: fixed.
3 F
2 S
4 F
3 S
5 F
3 E (S) too much information in stem. REV-1: fixed.
6 F
2 S
7 H
1 X
U (S) neither A, B, or C are credible distractors. REV-1: fixed.
8 H
3 S
9 H
3 S
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
One or more distractors is (are) not credible.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6.
Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
7.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
33 of 34 NUREG-1021, Draft Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 10 H
3 E (S)
RO importance rating is 3.1 vice 3.5; REV-1: fixed.
11B F
2 X
U (S) repeat from 2002 exam; too much information in stem; each answer can be evaluated as true or false on its own without reference to the question stem. REV-1: kept extra info; T/F problem fixed.
12 H
3 U
according to RPS Training material (page 17 of 24), Group 4 isolation does not result from loss of a single RPS bus, and therefore no answer is correct; REV-1: no change. The TIP purge system isolates and the answer was changed to reflect this.
13 F
1 U (S) too much information in stem; with the attachment provided, the question seems way too easy. REV-1: stem unchanged; deleted attachment.
14 H
3 S
15 H
3 S
16 F
3 E (S) too much information in stem. REV-1: fixed.
17 F
2 S
18 F
3 X
U this question asks about interlocks (K4), not automatic operations of valves (per the selected KA:
A3.01). REV-1: NOT FIXED YET. Question rewritten to address the original KA.
19 F
2 X
U (S) doesnt test knowledge of power supplies TO the RPS MGs per the K/A, but rather tests knowledge of power supplies to the RPS Busses. REV-1: fixed.
20 H
3 X
X U (S) since all the distractors list valves that would automatically isolate under these conditions, and since there is an ongoing leak in the drywell which is causing the radioactivity release, it can be reasoned that the isolation of any of the valves listed would contribute to preventing the further spread of radioactivity from the reactor building; perhaps rewording the stem to: which of the following will automatically isolate to ensure radioactivity that has been released does not spread beyond the reactor building?; also, distractor A should read...RHR C/HPCS... REV-1: fixed.
21 F
3 S
22 F
2 S
23 F
3 S
24 H
3 S
25 H
3 X
E (S) recommend changing distractor C to read: governor valves do not reposition and...;
recommend changing answer D to read: governor valves completely close and the bypass valves throttle open to maintain pressure at 960 psig. Reactor power drops below the power range. Also, if the correct answer isnt reworded to mention Rx power, then it wont match the K/A. REV-1: fixed.
34 of 34 NUREG-1021, Draft Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 26 H
2 E (S)
RO importance rating is 3.6 vice 3.3. REV-1: fixed.
27 F
3 X
X U (S) since the SRM scram is activated during initial fuel load, it could be argued that distractor B is also correct; doesnt test knowledge of effect of loss/malfunction of trip units will have on SRM per K/A. REV-1: fixed with new question.
28 F
3 E (S) too much information in stem. REV-1: fixed.
29 H
3 X
U too much information in stem; doesnt test knowledge of effect of loss/malfunction of primary containment instrumentation will have on ADS per K/A. REV-1: fixed with new question; still doesnt match K/A (RHR -vs-LPCS). Question was revised to reflect LPCS since both pumps perform the same function with ADS logic.
30 F
2 X
E recommend using air operated valves that fail open for A and B rather than non-air operated valves. REV-1: no changes made. Answers A and B changed to improve distractors.
31 H
3 S
32 F
3 S
33 H
3 S
34 H
3 E (S) too much information in stem. REV-1: fixed.
35 F
3 X
E (S) change stem to...prevent primary containment failure, then RCIC...;
distractor A does not seem credible. REV-1: fixed.
36 F
3 S
37 F
1 U
too easy REV-1: no change was made. No revision based on question meets the KA and important operator knowledge that has led to events.
38 H
3 S
39 F
3 S
40B H
3 S
41 H
3 X
E (S) distractor C does not seem credible. REV-1: fixed.
42 H
2 X
U too much information in stem; distractor D is not credible. REV-1: no change made. Distractor D changed.
43 H
3 S
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 44 H
3 S
45 F
3 S
34 of 34 NUREG-1021, Draft Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 46 H
3 S
47 H
3 S
48 H
3 S
49 F
2 S
50 H
3 E (S) too much information in stem. REV-1: fixed.
51 H
3 S
52 F
2 S
53 H
3 S
54 H
3 E (S)
RO importance rating is 3.6 vice 3.2 REV-1: fixed.
55 H
2 S
56 H
3 S
57 F
2 S
58 H
3 S
59 H
3 S
60 H
3 S
61 H
3 S
62 H
3 S
63 H
3 E (S) too much information in stem (the info about RHR-V-42A is a cue to question #38); REV-1: fixed.
64 F
3 X
E (S) distractor A says:...until power is at least below the 75% rod line. but I think the 75% rod line is a rod height vice a power. REV-1: fixed.
65 F
1 U
just too easy. REV-1: replaced with new question - still pretty easy.
34 of 34 NUREG-1021, Draft Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 66 F
2 S
67 F
3 S
68B H
3 S
69 F
2 S
70B H
3 S
71B H
3 X
U it is difficult to distinguish distractor A from B or C from D. REV-1: no changes made. Distractors have been revised based on input from the facility.
72B F
3 S
73B H
3 X
U doesnt test ability to recognize EOP entry conditions for High Drywell Temperature per K/A. REV-1: no changes made. Disagree with this comment.
The question does test the KA referenced. The KA test the ability to recognize an abnormal indication used for an EOP entry condition. RPV level is an EOP entry condition and the increased DW temperature will affect instrument accuracy.
74 F
3 S
75B H
3 X
U (S) in stem, use = or something else besides dashes in front of numbers to avoid confusion with negative signs; distractor D is not credible. REV-1: fixed.
S1 H
3 E (S)
SRO importance rating is 3.6 vice 3.1 for A3.04. REV-1: fixed.
S2 H
3 X
X U (S) this question is almost identical to RO question #34 (so the same comments apply and one of them should go away); outline says this should be A2.07, but it is A2.03 instead; not 55.43-related question - not SRO-only. REV-1: replaced with new question.
S3B H
3 X
U not SRO-only (not 55.43-related); REV-1: replaced with new question; not sure answer is correct. Answer is correct as stated.
S4 H
3 X
U (S) not SRO-only (not 55.43-related); REV-1: replaced with new question.
S5 H (F) 3 X
U (S) not SRO-only (not sure this is really 55.43-related) REV-1: replaced with new question.
S6 H
3 X
U (S) not SRO-only (not 55.43-related) REV-1: replaced with new question.
S7 H
3 X
U (S) not SRO-only (not 55.43-related) REV-1: replaced with new question.
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S8 H
3 X
U doesnt test knowledge of interrelations between Main Turb. trip and Rx pressure control per K/A. REV-1: no changes to question. The originally selected KA for this question was 295025 vice 295005. Additionally, the original KA was for the pressure regulating system (K2.08) vice RCIC K2.07. The confustion arose during the construction of two similar questions, one on the RO and one on the SRO exam. The question has been corrected to reflect the originally selected KA.
S9 H
2 S
S10 F
3 S
34 of 34 NUREG-1021, Draft Revision 9
ES-401 2
Form ES-401-9 Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S11 F
2 S
S12 F
3 S
S13 H
3 S
S14 F
3 S
S15B H
3 X
E (S)
SRO importance rating is 3.4 vice 3.7; in the stem, replace the phrase [procedure title] with an actual procedure title; distractors C and D do not seem uniquely different answers; REV-1:
replaced with new BANK question.
S16 H
3 X
U (S) none of the answers available are correct for the question as stated; REV-1: fixed.
S17 F
2 X
U (S) not SRO-only (not 55.43-related) REV-1: replaced with new question.
S18 H
3 S
S19B F
3 S
S20B H
3 S
S21 H (F) 3 X
U (S) not SRO-only (not 55.43-related) REV-1: replaced with new question.
S22 H
3 X
U (S) not SRO-only (not 55.43-related) REV-1: replaced with new question.
S23M?
H 2
E original question was not included, so no assessment can be made as to whether this is a valid MODIFIED question or just a slightly changed BANK question. REV-1: no changes made to this question. This question has been reclassified as a bank question since that is the most restrictive classification.
S24B H
3 X
U (S) doesnt test knowledge of RX Power Level Control as it applies to High Drywell Temperature per K/A. REV-1: replaced with new question.
S25 H
3 S
34 of 34 NUREG-1021, Draft Revision 9