ML090490114
| ML090490114 | |
| Person / Time | |
|---|---|
| Site: | Arkansas Nuclear  |
| Issue date: | 02/02/2009 |
| From: | NRC Region 4 |
| To: | Entergy Operations |
| References | |
| Download: ML090490114 (7) | |
Text
ES-401 AN1-2009-01 Written Examination Review Worksheet Form ES-401-9 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 1N H
3 S
C, CR, 43.5.
2B H
3 S
B, CR, 43.5. This question is unsatisfactory because it does not verify the alarms are consistent with plant conditions. Since the question history is not provided, it is assumed it is a bank question. SAT after discussion w/ licensee 3N H
3 S
B, CR, 43.5. This question is unsatisfactory because it does not involve the occurrence of a reactor / turbine trip as it applies to a loss of main feedwater. SAT after discussion w/ license 4N H
3 S
A, CR, 43.3.
5N H
2 U
E D, CR, 43.2. Distractors B and C are not credible because it is not possible to be in an Operating LCO and a Shutdown LCO at the same time. Since the two LCOs referenced in the distractors are applicable in different modes, they cannot be applicable at the same time and therefore, are not credible.
6N H
3 S
A, CR, 43.5.
ES-401 AN1-2009-01 2
Form ES-401-9 Page 2 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 7N H
3 E
E A, CR, 43.5. Recommend the stem be changed to which operating procedure actions should first be implemented.
8B H
2 U
E D, OR, 43.2. This question is unsatisfactory because the correct distractor does not include all the required actions (restore operability within 14 days) and is therefore technically also incorrect. Also, need to consider the affect on EDG operability (TS 3.8.1/2) 9N H
2 S
D, OR, 43.5.
10B H
3 S
C, CR, 43.5.
11M H
3 U
E A, CR, 41.10. This is considered an RO question since it tests on the entry conditions to the reactor trip procedure.
12N H
2 U
S B, OR, 43.2. See unsat comment for question #5SAT after discussion w/ license 13N H
2 S
D, CR, 41.10. This question is considered unsatisfactory because it tests on an initial entry condition which makes it an RO question. SAT after discussion w/ license 14N H
3 S
D, CR, 43.5. This question is a KA mismatch because it has nothing to do with main feedwater as required by the KASAT after discussion w/ license.
15N H
3 U
E C, CR, 43.5. According to step 9.2.2 and the preceding note of procedure 1104.027, Tech Spec 3.8 applies.
16N H
1 U
S C, CR, 43.2. This question appears to be a direct lookup. If this is a from memory question, it is ok as written. SAT after discussion w/ license 17B F
2 S
B, CR, 41.10. This question is unsat because it tests on the system operation and procedure entry conditions. SAT after discussion w/ license 18N H
3 S
B, CR, 41.10. This question is unsat because it tests on the system operation and procedure entry conditions. SAT after discussion w/ license 19B F
2 S
C, CR, 43.7.
20N F
2 S
D, CR, 43.7.
21N F
3 S
A, CR, 43.2.
22B H
2 U
U B, CR, 41.10. This is a basic radworker question and not an SRO question.
Recommend an emergency director question. (Will replace) 23B H
3 S
A, CR, 43.5.
24B H
2 S
C, CR, 41.10. This is an RO question since it is testing on the entry conditions to the
ES-401 AN1-2009-01 3
Form ES-401-9 Page 3 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation EOP. SAT after discussion w/ license 25B F
2 U
E D, CR, 43.5. This is considered unsatisfactory because it is intuitive that given there is only one correct answer, the highest EP classification must be correct. (Will Replace) 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
ES-401 AN1-2009-01 4
Form ES-401-9 Page 4 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72
ES-401 AN1-2009-01 5
Form ES-401-9 Page 5 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97
ES-401 AN1-2009-01 6
Form ES-401-9 Page 6 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 98 99 100 B=
F=
E=
M=
H=
U=
RO TOTALS:
N=
Additional Notes:
B= 9 F= 5 E= 1 M= 1 H= 20 U= 15 SRO TOTALS:
N= 15 Additional Notes:
GENERAL COMMENT
S:
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N.
- 2.
Chief Examiner comments are indicated in blue.
- 3.
Average difficulty is __na ___ on the RO exam and _2.5 on the SRO exam.
- 4.
The 10CFR55.41/43 distribution is: RO / SRO 41.1 =
43.1 = 0 41.2 =
43.2 = 5 41.3 =
43.3 = 1 41.4 =
43.4 =
41.5 =
43.5 = 11 41.6 =
43.6 =
41.7 =
43.7 = 2 41.8 =
41.9 =
41.10 = 6 41.11 =
41.12 =
41.13 =
41.14 =
- 5.
The answer distribution is: RO / SRO A = na / 6 (24%)
ES-401 AN1-2009-01 7
Form ES-401-9 Page 7 of 7 OBDI 202 - IOLE Process
- 6.
T B = na / 6 (24%)
C = na / 6 (24%)
D = na / 7 (28%)
here are 4 questions with indicated attachments provided.
- 7.
There are 20 (80%) higher cognitive questions. This is considerably higher than the 60% ceiling recommended by NUREG-1021.
- 8.
The SRO questions were reviewed by S. Garchow on 11/24/2008.