ML091520241
| ML091520241 | |
| Person / Time | |
|---|---|
| Site: | Palisades  |
| Issue date: | 07/07/2008 |
| From: | NRC/RGN-III |
| To: | Entergy Nuclear Palisades |
| Shared Package | |
| ML082200310 | List: |
| References | |
| Download: ML091520241 (6) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2.0 Y S new.
2 F 2.0 S bank.
3 H 2.0 S bank.
4 F 2.0 Y S bank.
5 H 2.0 Y E bank. The question should be in past-tense. The plant was . . . corrected 6 H 2.0 S bank.
7 F 2.5 Y S bank.
8 F 2.0 S new 9 H 2.0 S new 10 H 2.0 S new 11 H 2.5 S new.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
$ The answer choices are a collection of unrelated true/false statements.
$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
$ The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 7. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 12 H 2.5 Y S new.
13 H 2.0 Y S modified.
14 F 2.0 Y S bank.
15 H 2.0 S bank.
16 H 2.0 S bank.
17 H 2.0 Y S bank.
18 H 2.0 S new 19 H 2.5 Y S new.
20 F 2.0 S new.
21 H 2.0 S new.
22 F 2.0 Y S new.
bank. Non-credible distractors. Make some less than 15 seconds.
23 F 2.0 Y Y EU Distractors found to be credible based on other time requirements.
However, changed distractors to more realistic times.
24 F 2.0 S new.
25 F 2.5 Y S bank.
26 F 2.0 S bank.
27 H 2.0 S bank.
28 H 2.5 Y S bank.
29 H 2.0 Y S bank 30 F 2.0 Y S bank.
31 H 2.0 Y S modified. Are we providing the graph? Correct graph.
32 H 2.5 S new.
33 H 2.5 S new.
34 H 2.0 Y S new.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 35 H 2.0 Y S new.
36 H 2.5 Y S bank.
bank. Not a higher skill question. Probably memorized. Distractors b, c, 37 H 2.0 Y U and d modified.
38 F 2.0 S new.
39 H 2.0 S bank.
bank. Not a Fundamental question. Also, re-write distractor d. so it cant be mis-interpreted. Appears to be a throw-away because the steam leak will affect SIS & Containment Spray. The facility agreed it was not a 40 F 2.5 SU fundamental question. After face-to-face discussion with the utility, it was determined that the remaining portions of the question were satisfactory without modification.
41 H 2.0 S bank.
new. Not a fundamental question. Material is required to be memorized 42 H 2.0 SU at the station.
43 H 2.0 Y S bank 44 F 2.0 S bank.
new. Remove due from distractor a. Rework distractor d. to remove 45 F 2.0 Y E due from that distractor. Left as-is, station language.
new. Some verbs are missing from the distractors that need to be added 46 H 2.5 E to correct sentence structure. Corrected.
47 F 2.0 S bank.
48 F 2.0 Y E new. the upper reservoir Not changed.
49 F 2.0 Y S new.
50 H 2.5 S bank.
51 F 2.5 S bank.
52 F 2.5 S bank.
53 H 2.0 U bank. Not a higher skill. Agreed.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 54 H 2.0 S bank.
55 F 3.5 S bank.
new.
56 H 2.5 S 57 F 2.0 S bank.
58 H 2.0 S new.
59 F 2.0 S new.
60 H 2.0 Y U new. Not a higher skill question. Agreed.
61 H 2.0 S bank.
62 H 2.5 S new.
63 H 2.0 E bank.
64 F 2.0 S modified.
bank. Implausible distractors. (a.) & (c.). Required to be known from 65 F 2.0 SU memory - distractors a. & c. are from the fire procedure - plausible.
66 H 2.0 S bank.
67 H 2.5 S new.
68 F 2.0 S bank.
69 H 2.5 U new. This is an SRO tech spec application question. Agreed - replaced 70 H 2.0 U bank. This is an SRO tech spec application question Agreed - replaced.
71 F 2.0 S new.
72 F 2.0 S bank.
73 F 2.0 S bank.
74 F 2.0 S bank.
75 F 2.0 S new.
76 H 2.5 Y S new.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 77 H 2.5 Y S new.
78 F 2.0 Y S new.
79 H 2.0 Y S new.
80 H 2.0 Y Y S bank.
81 H 2.0 Y S new.
82 H 2.0 Y Y S new.
83 H 2.0 Y Y U new. Not a high skill question - memory only question.
new. Two implausible distractors. The tech spec for leakage is a 84 F 2.0 Y Y U memorized value. The other two distractors are not plausible. Difficulty level <2.0. Values changed to correct values.
85 H 2.0 Y U modified. Not a high skill question - memory only. Agreed 86 H 2.0 Y Y S New new. Stem says pressurizer heater status, first status is pzr level. Not an 87 F 2.0 Y U SRO question. Both parts of the question are system level questions.
Modified question to contain tech spec call.
88 H 2.0 Y Y S new 89 F 2.5 Y S new.
bank. Not an SRO question. The question may be answered with system 90 H 2.5 Y U knowledge. Agreed. Modified question to add procedure selection.
new. Not an SRO question. The question may be answered with an ROs understanding of the system and annunciator response procedures. The 91 H 2.0 Y U question is acceptable. It requires the SRO to select the procedure used for the conditions.
new. Not an SRO question. EOP entry conditions are required to be 92 H 2.0 Y U memorized by the ROs. Agreed. Changed question to contain an SRO portion - highest importance status tree.
93 F 2.0 Y S new.
new. At times distractor a. is correct. Change stem to eliminate distractor 94 F 2.0 Y E a. as a correct answer. Station made no change. Distractor a. is incorrect as written.
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only bank. Not an SRO only. Question tests tech spec entry conditions.
95 H 2.0 Y U Agreed. Replaced question.
96 F 2.0 Y Y S bank.
97 F 2.0 Y S new.
bank. Not higher skill. Material must be memorized to arrive at the correct 98 H 2.0 Y U answer. Changed to incorporate Site Emergency Director actions.
99 F 2.0 Y S new.
100 F 2.0 Y S new.