ML030940583
| ML030940583 | |
| Person / Time | |
|---|---|
| Site: | Columbia  |
| Issue date: | 03/27/2003 |
| From: | Gody A NRC Region 4 |
| To: | Parrish J Energy Northwest |
| References | |
| 50-397/03-301 | |
| Download: ML030940583 (4) | |
Text
ES-401 CGS March 2003 Form ES-401-9 (R8, S1)
Written Examination Review Worksheet Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
F 3
S 2
H 3
U E
Does not qualify as modified question. Stem has no significant modification. See ES-401D.2.f. Resolution: Leave as is. Changed designation.
3 H
2 S
4 H
3 S
Replaced question, because knowledge was duplicated in Question 78.
5 H
3 U
S 6
F 2
S 7
H 3
U S
8 F
3 S
9 H
3 S
10 F
2 U
U U
As worded, question does not adequately address a basis in an EOP.
The stem gives the answer if one knows the setpoint. Resolution: New question acceptable.
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
More than one distractor is not credible.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6.
Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
7.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 (R8, S1)
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 11 H
S 12 F
3 U
S 13 H
4 S
14 H
3 S
15 H
4 U
S 16 H
3 S
17 H
4 U
S 18 H
4 E
Question appears to be very difficult if it was not taught to this detail.
Consider providing appropriate plant references. Resolution: Minor changes were made to stem to make question clearer.
19 F
3 S
20 H
3 S
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 21 H
3 S
22 H
3 U
S 23 H
3 S
24 H
4 S
25 H
4 U
S 26 F
3 S
27 H
3 U
S 28 F
3 S
29 H
2 S
30 F
3 S
31 H
3 S
32 H
3 U
S 33 H
3 S
34 F
2 S
35 H
3 U
S 36 H
2 S
37 H
3 U
S 38 H
3 U
E Distractor D appears to be correct, but incomplete. This makes it difficult to justify it not being a correct answer. Resolution: Revised D to include incorrect time requirement.
39 H
3 S
40 H
3 E
The reference does not necessarily support the answer. It states that either Groups 1 and 2 or 3 and 4 may be withdrawn first. Resolution:
RSCS Group information added to stem of question.
41 H
3 S
42 H
3 U
S 43 H
3 S
44 H
3 S
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 45 F
3 U
U The K/A link between partial or complete loss of forced flow and individual jet pump flow is not made. Resolution: Replaced question.
46 H
3 U
E I believe one could make a case that Choice C is correct. Resolution:
Replaced question.
47 H
3 U
S 48 H
3 S
49 H
3 S
50 F
2 S
51 F
3 S
52 F
2 U
S 53 F
3 S
54 F
2 S
55 F
2 U
E Suggest rewording stem similar to What is the basis for isolating secondary containment ventilation due to reactor building exhaust plenum high radiation? Resolution: Changed stem.
56 H
4 E
Is this always true? With the core offloaded, it is not intuitive that the temperature increase would be in the fuel pool. Resolution: Revision satisfactory.
57 F
3 U
E Reference should be to 209002A4.14. Resolution: Corrected as suggested.
58 H
3 U
E The first sentence in the stem is not necessary to answer the question and appears to make the question easier. Resolution: Comment incorporated.
59 F
2 S
60 H
4 S
61 H
3 S
62 H
3 U
S 63 H
3 S
64 F
2 S
65 H
3 U
S 66 F
2 S
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 67 H
3 U
S 68 H
4 S
69 H
4 U
U Reference does not support answer. It state that a control rod insert block will be generated if there are more than two insert errors. The stem only discussed two insert errors. Also, it is not clear what is meant in Distractor B. Resolution: Changed answer to B.
70 F
3 U
E The reference indicates that the SYNC PERMIT light is only illuminated momentarily. The stem could be construed to read that it remains illuminated. Resolution: Changed stem to indicate momentary illumination.
71 H
3 S
72 H
4 U
S 73 H
2 S
74 H
3 S
75 F
3 U
S 76 H
3 S
77 F
3 U
S 78 F
3 U
The same knowledge is tested as in Question 4. Resolution: Changed Question 4.
79 F
3 S
80 H
3 S
81 F
2 S
82 H
4 U
U E
The stem does not give the status of any of the fans. Whereas, the reactor building pressure is being maintained at -1 inch of water and stable, it is not apparent why one would start standby gas treatment.
Resolution: Changed stem to indicate momentary illumination.
83 H
3 S
84 F
3 S
85 H
3 U
S 86 H
3 S
87 F
2 U
S
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 88 H
3 U
S 89 F
3 S
90 F
3 S
91 F
3 S
92 F
3 U
U The knowledge requirement is almost identical to that required to answer Question 1. Resolution: Rewrote question to test other knowledge.
93 F
3 S
94 F
2 S
95 F
3 U
S 96 H
3 S
97 F
2 U
E It does not appear that the words minimum and maximum add anything to the choices and may be confusing. Resolution: Removed minimum/maximum and rewrote distractors B and C.
98 F
2 S
99 H
2 S
100 F
2 S