ML103080903
| ML103080903 | |
| Person / Time | |
|---|---|
| Site: | Indian Point  |
| Issue date: | 07/20/2010 |
| From: | David Silk Operations Branch I |
| To: | Entergy Nuclear Operations |
| Hansell S | |
| Shared Package | |
| ML100980645 | List: |
| References | |
| TAC U01789 | |
| Download: ML103080903 (9) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 IP3
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H) (1-5)
Stem Cues T/F Credo Partial Job-Minutia
- /
Back-Q=
SRO B/M/N utE/S Explanation Focus Dist.
Link units ward KIA Only 1
H 2
2 H
2 3
H 3
X 4
H 3
X 5
F 2
6 H
3 X
7 F
2 X
8 F
3 X
D N
S B
S N
E Delete bullet about C/D via atmospheric dump valves.
B E
Edit stem 'Why?" Edit C -"three RCPs if they are needed" B
S N
E H vs F. Connect event and reason.
B E
Make into a two part Q to match choices.
B E
Delete first sentence and initial phrase in B&D.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.)
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3.
Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
- 6.
Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N U/E/S Explanation Focus Dist.
Link units ward KIA Only 9
F 2
B S
H vs F 10 H
3 X
B E
Provide breaker opening vs stating a LOOP occurred.
11 H
3 N
E Change to state loss of instrument bus vs inverter..
12 H
3 X
B E
H vs F. Re-word stem to make two sentences.
13 F
2 N
S 14 F
2 X
N E
F vs H. Remove personal pronoun in stem.
15 H
3 N
S H vs F 16 H
2 B
S 17 c
I
,J D
c 18 F
2 X
B E
Change to 'What action is the crew directed to take" 19 H
3 B
S 20 H
2 B
S H vs F 21 H
3 X
N E
Delete 5 th bullet - not true if N-35 undercompensated.
22 F
2 B
S 23 F
2 X
X M
E Add "lAW... " Re-word Q to eliminate mention of CRS.
24 H
2 N
S 25 H
2 X
C N
E Replace C. Delete RHR not injecting.
26 H
3 X
D B
E H vs F. Delete RHR injecting phrase. Replace D - similar to C.
27 H
3 X
M E
H vs F. Delete common phrase in distractors.
28 H
2 C
N E
Replace C.
ES-401 3
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOO (F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N U/E/S Explanation Focus Oist.
Link units ward KIA Only 29 H
2 N
E Reselect KA due to similarity with RO Admin JPM and simulator.
30 H
3 X
M E
Clarify RCS leakage.
31 H
2 B
N E
Replace I reword B - remove SIP 31.
32 H
3 X
M E
Add "and what actions are required." Delete "increase" in O.
33 F
2 A,O B
U A & 0 not plausible - replace sump level with another parameter.
34 F
2 N
S 35 F
3 X
N E
Add lAW 36 F
3 0
B E
Replace 0 37 H
2 B
~
38 F
2 0
N E
Replace I reword 0 to be associated with P-10 or P-7 39 F
2 B
S F vs H 40 F
3 X
C N
E Capitalize WILUNOT; reword C 41 F
2 B
S Fvs H.
42 H
2 X
C N
E Edit 1 st bullet. Change answer key (0 vs B) Replace C.
43 H
2 0
B E
Add noun names. Replace O.
44 F
2 X
0 N
E F vs H. Add lAW procedure. C &0 lower power level.
45 F
2 X
N E
Add "reactor power is." Replace "approximately" with "prior to" 46 F
2 N
S 47 F
2 X
N E
Add lAW.
48 F
3 N
S
ES-401 4
Form ES-401-9
- 1. I 2. I
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
lOK lOD I
I I
I (F/H)
(1-5)
Explanation Stem Icues I T/F I cred'l Partial I JOb-I Minutia I #1 I Back-I Q= ISRO I B/M/N I U/E/S Focus Dist.
Link units ward KIA Only 49 F
2 x
B E
I Provide initial air pressure and use words from TS bases.
50 H
2 x
N E
I Delete 1st sentence about lOOP.
51 F
3 x
B E
I Edit detector names to match reference 52 H
2 x
N E
I Fix 2nd bullet. Replace 0 with a low value 53 F
2 x
N E
IAdd lAW.
54 F
2 x
N E
I Change 110 to 105 in A & B.
55 F
2 N
S 56 H
3 N
S 57 H
2 x
B E
I Delete phrase about channels 1 and 2 in 2 nd bullet.
58 H
3 x
B E
I Add 'Why or why not?"
59 H
2 x
N E
I Spell out QSPDS 60 H
2 x
C B
E I Delete location of leak. Spell out ">" & "1". Add lAW. Add Cnmt rad.
61 H
3 x
B E
I Delete phrase about Press Mode.
62 H
2 x
N E
I Delete phrase about Tx power increase. A &B add COlR vs TS.
63 F
3 x
B E
I Edit stem. "Both" vs "each" KA does not match outline (misprint) 64 H
3 N
S 65 H
3 x
N E
I H vs F. Specify point in time when looking at panel.
66 H
x B
E I Edit stem so A is not so obvious.
67 F
3 x
N E
I Add EOl to stem.
H 4
N S
I H vs F.
~Q68
ES-401 5
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N UtE/S Explanation Focus Dis!.
Link units ward KIA Only 69 F
2 B
S 70 H
2 C
B E
H vs F. Replace C.
71 H
2 B
S 72 H
3 D
B E
Change D to ensure it is wrong.
73 F
3 N
S 74 F
2 B
S 75 F
3 N
S F vs H 76 H
2 X
M E
A &B add instrument flashing vs inaccuracy.
77 F
2 X
B U
NotSRO only. Replace.
78 H
3 X
N E
Edit stem to ask about AFW pump response. Delete needless phrase in B &D about AFW auto-start setpoint.
79 H
3 X
M E
Edit stem to ask about subsequent operator action.
80 F
2 X
B E
Add 'Why or why not?" to stem. Add "in the RCS" to C.
81 H
3 X
B E
Edit 8 th bullet. Capitalize ECA-1.1.
82 H
3 X
B U
Not SRO only. Systems knowledge. Overhaul.
83 H
2 X
N E
84 H
2 X
N E
Re-work Q &provide other procedure choices.
85 H
2 X
B E
Add loss of MCC. Edit Q for two part response.
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H) (1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N UtE/S Explanation Focus Dis!.
Link units ward KIA Only 86 H
3 N
S ES-401 6
Form ES-401-9
- 1.
- 2.
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
- 7.
- 8.
Q#
LOK LOD (F/H)
(1-5)
Stem Cues T/F Credo Partial Job-Minutia
- 1 Back-Q=
SRO B/M/N UtE/S Explanation Focus Dis!.
Link units ward KIA Only 87 H
2 X
N E
Add Go to E-O to all choices 88 H
2 M
S 89 H
3 X
N E
Move common phrase to stem.
90 H
2 X
N E
Edit Q for two part response.
91 H
2 X
N E
Add "if any" to stem.
92 F
2 X
N E
Remove "As Refueling SRO" from A, B, & C 93 H
2 X
N E
Add "if any" to stem.
94 F
2 X
X B
U KIA to manage crew. Procedure overview Q vs 55.43(b)5 95 F
5 X
N U
Reselect KA 96 F
3 X
B E
Add trouble shooting to B and vent leakage to drain in C 97 F
2 B
S 98 F
4 X
N E
Edit stem "the".
99 H
2 X
C B
E Delete reference to Mode 3, delete 4th bullet, provide P & T wrt RHR.
100 H
4 M
E Make C an Alert call
Indian Point 3 OD Test Comments Admin..IPMs General comment: review/narrow tolerances for calculations SRO A-1-1:
Specify Unit involved (3)
Provide basis in IC for Manual calculation and use of Attachment 2 SRO A-1-2:
No range needed in the calculation of CNMT volume SRO A-2:
No comment SRO A-3:
Need to provide units for noble gas concentration Need to provide R-27 set point SRO A-4:
Confirm release status with applicants Correct JPM step 3 to reflect Part 'I Form Specify PAR for rest of the EPZ RO A-1-1:
Provide graph book not just required graphs/tables Add cue to not use daily reactivity sheet Add cue to specify core burnup Specify reactor power vs MW(e)
RO A 2:
No comment RO A-2:
There are no designated critical steps. Specify critical steps.
Instruments past calibration date - change date in cue Procedure step 7.1 not signed off Table 6.1 error 5213 gpm vs 5210 for 32 SWP Procedure step 7.3 - SM signature,before the peer check - change Provide peer review sheet Add valves to comment page RO A-3:
Re-do data table/calculation for clarification Specify condenser in-leakage to 2.5 SCFU System JPMs Sim-1:
Provide cue for preferred boration method JPM step 14 specify valve numbers Properly number JPM steps past 17 Provide initial boron concentration End JPM after applicant determines boration time Change RCS temperature to be 535 F Add 142 seconds plus 34 minutes for required boration time Sim-2:
Provide cue that containment sump level is increasing Provide cue that RO-1 has been completed
Sim-3:
Specify the Task Standard Delete cue to implement immediate actions JPM step 4 - specify PZR pressure instrument failed
..IPM step 5 - specify transition step to step 4.83 End JPM at step 17 Sim-4:
Replace JPM with Initiate Feed and Bleed to the RCS Sim-5:
Provide tolerances State that offsite power is available Specify the three fans are in service!
JPM Steps 11-14 need to cue for time compression.
Sim-6:
JPM step 9 - need for IV cue.
JPM step 20 - cue for time compression JPM steps 21 & 22 - add details of procedure steps 1.18.1 and 1.19.1 JPM steps 29 & 30 - add details of procedure steps 1.23.1, 1.24.1, & 1.24.2 Sim-7:
Provide IC that CRS has reviewed set points Provide note that key is in location 58 Add substeps for entering/correcting data Sim-8:
Provide details of doing voltage adjustment In-plant JPMs: Find KIA to link JPM to the appropriate safety function In-plant 1:
Provide cue that SG pressure is 850#
JPM step missing - procedure says that two valves are to be failed open.
Need to assess applicants performing steps 4.1.8.7.a-e Correct steam inlet pressure vs discharge pressure Change air pressure at AIM station to force adjustment In-plant 2:
Provide cue to ask about location and operation of switch in aux building In-plant 3:
Specify the purpose of the task other than completion of steps Provide COL to the applicants if needed Provide cue that HP is accompanying JPM steps not critical since verified by COL
General comments: Clarify details of TS calls Specify criteria associated with critical tasks Scenario 1 Event 3: Make severity of leak - 40% and ramped in over several minutes Add excess letdown valve numbers Critical Task 2: Provide RCP trip criteria Critical Task 4: Restart 31S1 pump (additional CT if needed)
Scenario 2 Replace CFCU event Add malfunction for ATC Scenario 3 Events 5: Ramp in this malfunction (400 gpm/10 min)
Event 2: Typo - first BOP action "close" vs "closed" Event 3: List valves required to isolate SG Critical Task 2: Time frame for completion of SG isolation, if possible Scenario 4 Event 2: No TS call for EDG failing test due to being OOS Event 3: Credit CRS with TS call (TS 3.0.3)