ML083360257
| ML083360257 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 11/14/2008 |
| From: | Mckernon T NRC Region 4 |
| To: | Arizona Public Service Co |
| References | |
| 50-528/08-301, 50-529/08-301, 50-530/08-301, ES-401, ES-401-9 50-528/08-301, 50-529/08-301, 50-530/08-301 | |
| Download: ML083360257 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 PV-2008-11 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 1
H 2
M S
B41.7 2
H 2
M S
B41.7 3
H 3
N S
C41.10 4
F 2
M S
D41.10 5
H 3
N S
B41.10 6
H 3
N S
B41.7 7
F 2
N S
D41.7 8
H 3
N S
D41.10 9
H 3
U N
S A41.7 Distractor C is technically a correct answer since MSIS, if manually actuated, will cause both flows to = 0. Recommend rewording to tie to the conditions in the stem. OK as Written
ES-401 2
Form ES-401-9 PV-2008-11 Page 2 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 10 F
3 M
S A41.10 11 H
3 N
U S
C41.10 Answer C seems to be incorrect as written. Per 40EP-9EO10, RCP restart is allowed when > 20 minutes of single phase natural circulation in desired loop. Per the EOPs, natural circulation is verified by >24 F subcooled (not met) and < 30F delta between T-hot and CET (not met). When pressure is raised to 1350, conditions will be met except the 20 minutes.
Reworded Distractors 12 H
2 N
S D41.6 Need drawing and/or reference to verify technical accuracy of answer.
Question OK 13 F
3 N
S C41.5 14 F
3 N
S B41.10 15 H
3 N
S D41.7 16 F
2 M
S B41.10 Generrex is misspelled in the stem. Fixed 17 F
2 M
S C41.10 18 H
2 B
S A41.10 19 H
3 N
S B41.5 20 H
3 U
U B
S B41.10 The attached reference seems to contradict the answer. The note provided states that ops management should be contacted prior to resuming, and TS do not prohibit a startup from continuing. The term terminate makes this confusing. Does that mean shut down? Shutdown groups < 500 pcm position? Question OK as Written 21 F
2 N
S B41.10 Add ONLY to B and A Fixed 22 H
3 N
S A41.7 Need technical reference for justification of correct answer Question OK 23 F
2 N
S B41.7 24 H
2 N
S D41.10 25 H
2 U
U N
U S
C43.2 Although this may be required by the facility, the regulations state this is SRO knowledge. Since the ROs know they have to memorize one hour tech specs, they will eliminate B and D without technical basis. Changed Distractors 26 F
2 U
U B
S A41.10 The justification for this answer has the operators make assumptions about other monitors in alarm. The operators could assume RU-1, 16, 148 etc. are in
ES-401 3
Form ES-401-9 PV-2008-11 Page 3 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation alarm, which would make C a correct answer. Question OK 27 H
2 B
S D41.10 28 F
2 E
M S
B41.3 29 F
3 N
S A41.10 30 F
3 U
N S
C41.5 Reverse logic. Need to revise. Question OK 31 F
2 N
S D41.7 32 F
2 N
S A41.7 33 H
2 B
S A41.7 34 F
2 N
S A41.5 35 H
3 B
S D41.5 36 H
2 B
S C41.7 37 H
3 N
S B41.10 38 H
2 N
S B41.7 39 F
2 S
B S
A41.7 B could be argued to be correct too. However, the must in the stem removes this from a possibility.
40 F
2 N
S A41.4 41 H
2 N
S B41.10 42 F
2 N
S B41.7 43 F
3 M
S D41.10 Change desired demand in answers to some value. Someone could argue that they desire 30 percent. Question OK 44 F
2 M
S C41.7 45 H
3 M
S D41.10 46 F
3 N
S C41.7 47 H
3 N
S D41.10 48 H
3 N
S C41.7 49 H
2 E
N E
B41.8 The stem needs to address what is operating at 100%. Reworded Stem
ES-401 4
Form ES-401-9 PV-2008-11 Page 4 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation S
50 H
3 N
E S
A41.7 Remove one is from 3 rd bullet and replace with in. Reworded stem 51 F
2 N
S C41.4 52 H
2 N
U S
B41.7 This question elicits the same knowledge as the previous question - what cools NCW. Change this or the previous question. This Question was replaced with new question.
53 F
2 N
S B41.7 54 H
2 U
N U
S B41.7 This question needs to be rewritten. Too many bullets past the given information. S/G 1 has 4 A channels. SIAS, CIAS and MSIS are actuated, so someone could argue that A is a correct answer since any condition causing CIAS will actuate MSIS and SIAS. Reworded stem 55 H
2
?
N S
B41.7 Are ROs required to know the actions for this. Question OK 56 F
2 B
S B41.7 57 H
3 B
S C41.7 58 H
3 B
U S
B41.7 Need to resample for another K/A. ICS is a B&W function. Resampled KA and wrote new question.
59 H
2 N
S B41.10 60 F
2 B
S B41.7 61 H
5 U
N S
C41.5 There is no correct answer. Tave/Tref alarm causes AWP, so no rod motion.
Question OK as written 62 H
2 N
S B41.4 63 F
2 N
S D41.10 64 H
2 N
S B41.10 65 F
3 B
S D41.7 66 F
2 B
S C41.7 67 F
2 N
S D41.10 68 H
2 N
S C41.10
ES-401 5
Form ES-401-9 PV-2008-11 Page 5 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 69 H
2 U
N S
A41.10 Per 40DP-9OP02, the expected action is to verify, update then take action.
Would not checking the other channels be part of verifying. The vague statement in distractor D - take any appropriate action, makes this a potentially correct answer.
Question OK as written.
70 F
3 U
N S
B41.7 The diesel starting air subsystem includes the compressor?? If so, then action statement F applies in LCO 3.8.3, and an LCO has been entered. This would make D a correct answer too. Question OK as written.
71 F
2 N
S C41.12 72 H
3 U
N U
S B41.10 NOT supposed to ask what the applicant will NOT do. Question rewritten 73 F
2 N
S C41.10 74 F
2 N
S A41.10 75 H
2 N
S C41.10 76 H
2 S
77 H
2 S
78 H
2 S
79 H
3 S
80 H
3 S
81 H
3 S
82 H
3 S
Edit/ Rewrite to include Proc.. in Distractors OK as written 83 H
3 S
RO should be able to ident. Rewrite to include LCO action time OK as written 84 H
3 S
85 F
2 S
86 H
3 S
87 H
3 S
88 H
3 S
89 H
3 S
ES-401 6
Form ES-401-9 PV-2008-11 Page 6 of 7 OBDI 202 - IOLE Process
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation 90 H
2 S
91 H
3 U
S Rewrite-RO K/A More Proc. Linkage Question rewritten 92 H
3 S
93 H
3 S
94 H
3 S
95 F
2 S
96 H
3 S
RO KnowledgeLink more w/ LCO Question OK 97 H
3 U
S ROs know 1 hr TS Either include greater depth or reselect Question replaced 98 H
3 S
RO knowledge Question OK 99 H
3 S
100 H
3 S
B= 13 F= 35 E=
M= 10 H= 40 U= 6 RO TOTALS:
N= 52 Additional Notes:
B= 1 F= 2 E=
M= 3 H= 23 U= 2 SRO TOTALS:
N= 21 Additional Notes:
GENERAL COMMENT
S:
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N.
- 2.
Chief Examiner comments are indicated in blue.
- 3.
Average difficulty is __ ___ on the RO exam and ___ on the SRO exam.
- 4.
The 10CFR55.41/43 distribution is: RO / SRO 41.1 =
43.1 =
ES-401 7
Form ES-401-9 PV-2008-11 Page 7 of 7 OBDI 202 - IOLE Process 41.2 =
43.2 =
41.3 =
43.3 =
41.4 =
43.4 =
41.5 =
43.5 =
41.6 =
43.6 =
41.7 =
43.7 =
41.8 =
41.9 =
41.10 =
41.11 =
41.12 =
41.13 =
41.14 =
- 5.
The answer distribution is: RO / SRO A = 13 (17%)
/
6 (24%)
B = 16 (22%)
/
7 (28%)
C = 21 (28%)
/
5 (20%)
D = 25 (33%)
/
7 (28%)
- 6.
There are questions with attachments provided.