ML022490081
| ML022490081 | |
| Person / Time | |
|---|---|
| Site: | Vermont Yankee File:NorthStar Vermont Yankee icon.png |
| Issue date: | 08/15/2002 |
| From: | Gosekamp M Entergy Nuclear Vermont Yankee |
| To: | Conte R NRC/RGN-I/DRS/OSB |
| Conte R | |
| References | |
| 50-271/02-301, ES-401-9 50-271/02-301 | |
| Download: ML022490081 (6) | |
Text
ES-401 Written Examination Form ES-401-9 Review Worksheet
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
7.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- /
Back-Q=
SRO UIEIS Explanation
[
Focus Dist.
Link units ward K/A I____
I H
3 Y
Y S
?
H 3
Y N
S L F 2
Y N
S L H 3
Y N
S 5
F 2
Y N
E Modify stem: 'Reason for R/B is... "; minimize a&b; prevent c&d l
F 2
Y N
E Add 'torus" modifier to high temperature condition; add > 1 minute I
l somewhere in stem to eliminate distractor a H 3 l
N E
Re-word: A SBLOCA 'has" occurred... ; torus cooling "has been" placed i/s... ; "Subsequently", a LBLOCA occurs... ; "What is the response" change tense of distractors "isolate(s)"
3 F
2 N
N IS F
2Y 5
0 F
2 Y
V E
Delete entire Considering... Control Room" sentence Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
More than one distractor is not credible.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6.
Based on the reviewer's judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 7.
At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 1.1 2.
- 3.
Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
Q# LOK LOD r
t -UrE-S (F/H)
(1-5)
Stem lCuesl T/F I Cred. Partial Job-lMinutia
- / Back-Q= SRO UIEIS Explanation Focus Dist.
Link units ward K/A Only_
I1 H
3 Y
N E
Add procedure reference to stem 12 H
4 Y
Y S
13 H
4 Y
Y S
14 F
2 x
Y N
E Change question to "Which three conditions."; add procedure reference to stem 5 F 2
Y N
S j6 H
3 Y
Y S
j7 H
4 Y
N S
8 IF 12 Y
N IS 19 F
2 Y
Y S
?o F
2 Y
Y S
2I F
2 N
Y E
Add procedure reference to stem 22 H
4
=
Y N
S
?3 H
3 ly Y
S 24 F
2 X
_Y Y
E Add procedure reference to stem; add 'and why" to question 25 H
3 Y
Y S
26 H
3 Y
Y S
27 F
2 Y
N S
28 H
5 Y
N E
replace "most appropriate" w/ 'correct" 29 F
2 X
Y N
U List of conditions says DC control power lost - gives away the answer; O Ire-do question, e.g., ask for breaker response to loss of DC control pwr 0
H 3
Y IN E
replace 'actions" w/ 'strategies"; delete 'generator" 45 of 46 NUREG-1021, Revision 8, Supplement I
ES-401 3
Form ES-401-9
.S-40 3 F
.,._01-9 1
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
7.
Q# ILOKI LOD 111 Q (F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-Minutia
- 1 Back-Q= lSRO U/E/S Explanation 31 H
_3 _Focus Dist.
Link units ward K/A I S I
H 3
Y N
S 2
F 2
Y N
E Add procedure reference to stem 3
H 3
Y N
E Change level value to other than 82.5'; don't use Gp isolation setpt.
4 F
2 N
Y U
K/A does not match 5
H 3
Y N
E awkward choices: NOT, BUT, NOR; simplify to: level/trend is accurate/inaccurate 6
F 2
Y N
S 7
F 2
Y N
E delete 'most' correct 8
H 4
Y N
S 9
F 2
Y N
S o0 H 3
Y Y
S 1
H Y
N E
Add procedure reference to stem 12 H
4V N
S 3
H 3
Y N
S W F 2
Y N
E Add procedure reference to stem 5
F 2
Y N
S 16 F
12 VY N
E delete 'it is expected"
$7 H
3 Y
N E
simplify Q, e.g.: 'A" CS response will be:"
l8 H
3 Y
N S
$9 F
2 Y
N S
- 0 F
12 Y
N E
simplify 0, e.g., "SLC response will be:"
45 of 46 NUREG-1021, Revision 8, Supplement 1
ES-401 4
Form ESR-401-9 ES-40 I 4
Fnrm F5-4A1-Q
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 67.
(F/H)
(1-5)
StemlCues T/F lCred.IPartial Job-Minutia
- / JBack-UIEIS Explanation Focusl j l Dist.
Link units ward KIA Only 51 H
3 Y
N S
- 2 H
3 Y N E
modify Q, e.g., 'When LPRM is bypassed, plant response will be:"
53 H
1 Y
Y E
add procedure reference 4
H 3
Y S
5 H
3 Y
N S
6 H
3 Y
N E
add procedure reference; add 'what action is required..." to question 57 H
3 X
Y Y
E add classification level to Q (not just EAL)
- 8 H
2 Y
N S
9 F
2 Y
N E
add procedure reference 0
H 3
Y N
S 1
H 3
Y N
E delete "you' reference; re-phrase Q, e.g.,
" the isolation would _a
,2 F
2
_Y N
E a
- re-phrase Q, e.g.,.... the SBGT valves would
- '
3 4
HY Y
U distractor a is also correct if charger not placed Us; modify distractors
- 4 F
2 Y
N S
i5 H
3 V
N U
distractor b also correct; modify Q to a... what is effect on Rx bldg..."
36 H
V Y
N E
Change Q to 'What is response..."
7 H
3 Y
N E
Move pwr level info to end of stem; simplify Q, e.g.,"What is effect on rod motion" 8
F 2
Y N
S 9
H 3
Y N
E replace "is expected to" wrwill" O H 4
Y V
E add procedure/TS reference 45 of 46 NUREG-1021, Revision 8, Supplement 1 ES-401 5
Form ES-401-9
f 1 2. 3.
Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6.
Q# LOK LODI I
Q (F/H)
O(1-5)
Stem Cuesl T/F lCred. lPartial Job-lMinutia
- /
Back-Q lSRO UIEIS Explanation Focus Dist.
Link junits ward I
K/A Onlyd_
1 H
3 Y
N E
delete'appropriate" 12 H
3 Y
N E
Q does not ask for reason but answers include one; add 'why' to Q t3 H
3 Y
N S
4 F
3 Y
V E
re-arrange choices in increasing chronological order 5
H 3
N is 6
F 2
Y N
S 7
F 2
Y N
S 78 F
2 Y
N E
move electric fire pp reset to conditions para.; modify Q to ask for 'fire I
I I
pump' response 9 IF 12 V
N S
O0 F 12 V
N S
31 H
3 Y
N S
32 F
2 I N is 33 H
3 V
N IS 34 H
13 V
I s 35 H
S 36 H
3 Y
N E
delete "best" 37 F
2 Y
N E
add procedure reference toQ0 38 F
2 Y
N E
add procedure reference to Q 39 F
2 Y
N 0F 2
V_
S 45of 46 NUREG-1021, Revision 8, Supplement 1
ES-401 6
Form FS-401 -9 ES-40 I 6
Fnrm F-4O1-Q
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
7 Q#
LOK LOD 1
(F/H)
(1-5)
Stem Cues T/F Cred. Partial Job-lMinutia
- /
Back-Q=
SRO UIEIS Explanation Focus Dist Link units ward K/A Only 3i H
4
=
N S
32 F
2 Y
N E
add procedure reference to Q 33 F
2 Y
N E
add procedure reference to Q 4
F 2
Y N
IS 35 F
2
=
=
E add procedure/TS reference to Q 36 H
3 V
N S
37 H
3 Y
N E
modify question; ask for valve response if open is attempted 38 F
2
_Y N
E change to ask what is required; delete "should be directed" 39 H
3 Y
N E
add procedure reference to Q 00 F
3 Y
N E
modify Q: 'Which... alarms indicates an abnormal system response?";
delete 'would be unexpected,,n 45 of 46 NUREG-1021, Revision 8, Supplement 1