ML051880455
| ML051880455 | |
| Person / Time | |
|---|---|
| Site: | Monticello  |
| Issue date: | 05/02/2005 |
| From: | Mcneil D NRC/RGN-III/DRS/OLB |
| To: | |
| References | |
| 50-263/05-301, ES-401, ES-401-8 50-263/05-301 | |
| Download: ML051880455 (7) | |
Text
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6.
Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
7.
At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 Written Examination Review Worksheet Form ES-401-9 The first thirty questions reviewed in-depth; all other questions reviewed.
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1N F
2 U S If taught this way in class, then memory, otherwise, higher. Taught this way in class.
2N H
2 E
Distractor a. uses re-open without first closing the valve. Ok as-is.
3N F
2 U
U S K/A calls for lube oil system interlocks, question deals with an annunciator interlock. Annunciator deals w/lube oil pressure interlock.
4B H
2 E
Grammar in distractor d. needs to be cleaned up:...after a 107 seconds. Fixed.
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 5N F
2 U
Distractors c. and d. are not plausible. Replaced question selected new K/A - 211000K402, changed to Higher skill.
6N H
2.5 S
7N H
2.5 S
8M H
2.5 E
Reference lists no reference provided during exam; should list picture.
Added reference to picture.
9B F
2.5 S
10B H
2.0 S
Check for correct answer during validation Checked reference, answer was correct.
11N F
2.0 S
12N H
2.5 S
13B F
2.0 S
14N F
2.5 S
15B H
2.5 S
16M H
2.0 S
17N H
2.5 S
18B F
2.5 E S Rework distractor d. if possible. (non-credible) Distractor (d.) is plausible for Monticello Station, left as-is.
19N H
3.0 E
Front end of distractors is not necessary. Remove. Deleted unnecessary words in the front of distractors.
20N F
2.0 S
21N H
2.5 S
22N H
2.5 S
23N F
2.5 S
Can the other distractors be construed as correct? Reviewed - question is correct as-written. The other distractors are not correct.
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 24B H
2.5 S
Why is a. incorrect (not being shutdown per 3205? (a.) is incorrect because of the requirements in 3205.
25N H
2.5 S
26N H
2.0 S
27N H
2.0 S
28N F
2.5 E
Verify true above/below low power setpoint. True under all cases, but added power level.
29N H
2.5 E
Uncapitalize Initially - distractor d. Corrected.
30B H
2.5 E
Suggest changing distractor b. to moves to position 02 and no alarms present Made change to distractor (b.).
31B H
2.0 S
32M F
2.0 S
33N H
2.5 S
34N H
2.0 S
35N F
2.0 S
36N H
2.5 E
Distractor d. is also correct, may need modification. Distractor d. was not correct, however, modified to ensure no contention.
37N H
2.5 E
Stem needs to be reworded (verb-tense error) Should specify 12 RFP amps in the second part of the stem. Reworked stem.
38M H
2.5 S
39N H
2.0 E
Justifications are not correct. Justifications corrected.
40B F
2.0 S
41N F
2.0 S
42N F
2.0 S
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 43M F
2.0 S
ROs have to recognize entry conditions, not provide actions. What is thought process? May be SRO Only question. Discussed w/utility - not SRO only at Monticello.
44B F
2.0 U S This should be a higher skill question. Also, as written there are multiple correct answers. A time limitation should be used to eliminate the extra correct answers. Material is required to be memorized at Monticello, other answers are not correct as written.
45N F
2.5 S
46N H
2.0 S
47M H
2.5 E
Tsat for 35 psia is 259.29, not 258, making 116 incorrect - fix calcs.
Station tried to interpolate to get answer, inserting math error. Change made.
48M F
2.0 E
The stem of the question is difficult to understand, suggest reworking.
Moved material to a bullet in plant conditions to make question more readable.
49B H
2.5 S
50B F`
2.0 S
51B F
2.0 E
The stem may not elicit the desired answer. See if we can modify.
Modified.
52M H
2.5 S
53N F
2.5 S
54N H
2.5 S
55N F
2.0 S
56N H
2.0 S
57N F
2.0 S
58M H
2.0 S
59N H
2.0 S
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 60B H
2.0 S
61B F
2.0 S
62N F
2.0 S
63N H
2.5 S
64N H
2.5 S
65B F
2.0 S
66N F
2.5 S
67B H
2.5 U S Verify direct look-up. Determine responsibility at station. Verified, not a direct look-up, RO has to perform. Acceptable as-is.
68N F
2.0 S
69N F
2.5 S
70N F
2.0 S
71N H
2.5 S
72M H
2.5 S
73N H
2.5 S
74N H
2.5 S
Unable to verify the provided answer is correct. Resolve during on-site validation. All four distractor were listed as correct in the justification.
Verified correct answer.
75B F
2.0 S
76N F
2.5 U
S 77N H
2.5 U
E Distractor (d.) is not plausible. Remove startup & refuel. Replaced.
78N H
2.5 U
S Explain why in better detail distractor c. is incorrect (insert control rods).
Monticello will take other action, but not insert control rods - scram first.
79N H
2.5 U
U S SRO Only? Verified to be SRO Only at Monticello.
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 80N F
3.0 U
U U S The question does not determine the impact on the DC System. The location of the equipment will impact DC System. Acceptable as written.
81M H
3.0 U
U U S K/A calls for actions after exceeding thermal limits, questions asks how to keep within thermal limits. With power @ 615 MWe, Rx power is already above thermal limits. Operator has to determine limits exceeded.
82B H
3.0 U
S 83N F
2.5 U
S 84N H
2.5 U
U S SRO Only? SRO only responsibility at Monticello.
85N F
2.0 U
S 86N H
2.0 U
S 87N H
2.0 U
S 88N H
3.5 U
S 89N H
2.0 U
U S Direct look-up if flow charts are provided. Although material is in flow charts, significant evaluation is required to determine which step is correct, and what leg of chart is being evaluated.
90N H
2.5 U
S Verify no direct look-up exists in flow charts. Verified.
91N H
2.5 U
S 92M H
2.5 U
S 93N F
2.0 U
U Not SRO Only. ROs would be expected to know this definition.
Replaced question, selected a new K/A - 295020 2.4.30 94N F
2.0 U
S 95N H
2.5 U
S Verify at station this is not a direct look-up from tech specs. Not a direct look-up at Monticello Station 96B H
2.0 U
S 97M H
2.5 U
S
Q#
1.
LOK (F/H) 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
U/E/S 7.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 98N F
2.5 U
S 99N F
2.0 U
S 100N H
2.5 U
S Verify at the station this is not a direct look-up from the EOPs. Not a direct look-up at MNGS.