ML112901413
| ML112901413 | |
| Person / Time | |
|---|---|
| Site: | Comanche Peak  |
| Issue date: | 06/06/2011 |
| From: | NRC Region 4 |
| To: | |
| References | |
| 50-445/11-06 | |
| Download: ML112901413 (7) | |
Text
ES-401 CP-2011-06 Written Examination Review Worksheet Form ES-401-9 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
Page 1 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
- 1.
Why include a handout copy of the SRO exam and not the RO exam? (Dont need either one)
- 2.
1 H
2 N
S B/ Added SOP-108A, Step 5.1.G, Figure 1 as a reference to why seal differential pressure rising in Distractors A and C is plausible.
2 H
2 N
S D/
3 F
3 B
E A/ Possible two correct answers. B is correct if water level is less than 23 feet and the stem information does not include water level. Recommend indicating water level above the 23 foot mark in the stem to preclude B being correct. Added bullet for Refueling Cavity level at 858 feet.
4 F
3 B
S A/
5 F
2 B
E B/ Plausible to have normal pump suction as cooling for CCP bearings? Normal cooling is provided by oil from a separate oil pump that is cooled by a heat exchanger. This affects two answers. Modified Distractors C &D (replaced the word
Form ES-401-9 Page 2 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only suction with recirculation via).
6 H
2 B
S D/
7 H
3 B
S B/
8 H
3 B
S C/
9 H
3 B
E A/ Capitalized NOT in question stem.
10 H
3 B
S C/
11 F
2 N
S A/
12 F
2 B
S D/
13 F
2 B
S C/
14 H
3 B
S C/ Plausible that securing Containment Fan Coolers 3 and 4 would isolate the leak?
Added Reference Drawing and System Description info on why valves must be locally shut.
15 H
3 N
S B/
16 H
3 B
S C/
17 F
2 B
E D/ Revised Distractor B.
18 F
2 B
S C/
19 F
2 B
S B/
20 H
2 N
S A/
21 H
3 B
S B/
22 F
2 B
S A/
23 H
3 X
B U
C/ Overlap with Q8. Replaced Question and substituted K/A to avoid duplicity with Q08.
24 H
2 M
S A/
25 H
2 N
E A/ Added SOP-706 procedure title to Stem.
26 H
3 N
E B/ Removed reference to Train and added SOP-501A procedure title to Stem.
27 F
2 B
E B/ Minor wording change and added ABN-301 procedure title to Stem.
Form ES-401-9 Page 3 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 28 F
2 B
S D/
29 F
3 B
S D/
30 F
2 B
S C/
31 H
2 X
B U
B/ Ability to eval plant performance AND make operational judgments Raised power to 28% and removed last two bullets from Stem. Added whether a Turbine or Reactor trip was required to each Distractor. Changed the words the affected loop to Loop 1. Added ALM-0071A reference for why the Reactor would be tripped when an MSIV closes.
32 H
3 B
S D/
33 H
3 B
E D/ Changed the words consistent with to equal to in distractors B, C and D.
34 H
2 X
N U
A/ ROs do not make OPERABILITY calls or TS calls below the line. Change Stem from Technical Specifications to OPT-112A-1, Accident Monitoring Instrumentation Check. Removed reference to Technical Specifications, rewrote Explanation, and replaced Objective.
35 H
2 B
S B/
36 F
3 B
S B/
37 H
2 M
S A/
38 F
3 B
E D/ Changed RM-11 to PC-11 in Distractors A & C.
39 H
3 B
S D/
40 H
2 M
S B/
41 F
3 X
M U
B/ Frame vibration exceeded 2 mils/hr at 0200, not 0230. Adjusted the 0200 value for Frame vibration to make 0230 the correct answer.
42 H
4 B
S A/(OR) 43 F
3 B
S B/
44 H
3 B
E C/ Moved reference to Unit 2 and added ABN-502 procedure title to Stem.
45 H
2 N
E B/ Renumbered distractors and removed heater status to enhance plausibility.
46 H
2 B
S B/
47 F
3 B
E C/ Editorial Comment - add period to end of sentence on Distracter C. Corrected.
Form ES-401-9 Page 4 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 48 F
3 B
E A/ Removed "... under the worst-case environmental conditions... from the Stem.
49 H
3 B
E A/ Removed Reactor Coolant Pumps have been secured and added...A Loss of Offsite Power has occurred. in the Stem. This effectively explains why RCPs are secured.
50 F
3 M
S B/
51 H
3 B
S A/
52 H
3 B
S C/
53 H
2 B
E D/ Plausibility of Distracter C - tripping the RX only on loss of SWS? Added... and Reactor Coolant Pumps." to Distractor C for balance.
54 H
3 B
E B/
55 H
3 B
S B/
56 H
3 B
E A/ Added...Turbine Control Valves... to Stem.
57 H
3 B
S A/
58 H
3 B
S D/
59 H
3 N
S B/(OR) 60 H
3 N
E A/ Added ABN-102 procedure title to Stem.
61 F
3 B
S C/
62 H
3 X
N U
C/ Question asks applicant to direct operations through the appropriate procedures.
SRO level question. Question does meet the current K/A. Replaced Question and K/A.
63 H
3 B
E C/ Added EOS-1.1A procedure title to Stem.
64 F
2 N
S B/
65 F
2 B
S D/
66 F
2 N
E A/ Corrected punctuation and changed "... Continuous Action step." to "...
Continuous Action." In Distractors B and D.
67 F
3 N
S C/
68 H
2 X
B U
C/(OR) Multiple correct answers - If I pick 790# that still gives me the answer when rounded up to 800#. Same for picking 820# and rounding down to 800# on EOP-
Form ES-401-9 Page 5 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 3.0A, Step 6.b. Modified Steam Generator pressures in the Stem and deleted Bases information provided in the Written Exam References.
69 F
2 B
E B/ Added OWI-110 procedure title to Stem.
70 F
2 N
S D/
71 F
3 B
S D/
72 H
2 B
E C/ Added ABN-902 procedure title to Stem.
73 F
2 B
S B/
74 F
2 B
E A/ Added EOP-0.0A/B procedure title to Stem.
75 F
2 B
S C/
76 H
3 B
E D/ Possible two correct answers. Stem info gives Safeguards DC Bus voltage at 105 VDC. ECA-0.0 says that minimum system voltage required for equipment operation is APPROXIMATELY 105 volts. ECA-0.0 then states that IF DC bus voltage is Less than 110 volts AND plant staff determine it necessary to conserve batteries THEN 1)
Reference ABN-603, Loss of Protection or Instrument Bus. This makes Distracters C and D arguably correct. Modified reason in the second part of Distractor C since ABN-603 entry could be made. Modified Plausibility statement.
77 F
3 N
S C/
78 H
3 X
B U
C/ Entry conditions for AOPs or plant parameters that require direct entry to major EOPs are RO level questions. Also, knowledge of second half of question not required to determine correct answer. Replaced Question and K/A.
79 H
4 B
S B/ Added FRZ-0.1A Critical Safety Functions Status Tree to references.
80 H
3 M
S D/
81 H
3 B
S C/ Added FRH-0.1A, Step 4 to references.
82 H
2 M
S A/
83 H
3 N
E C/ Added punctuation and Technical Specification REQUIRED ACTION for Condition F to references.
84 H
2 B
E D/ Added FRC-0.2B procedure title to Stem.
85 H
3 B
S D/ Enhanced Explanation for the correct answer to differentiate why the maximum allowable channel disagreement as stated in the K/A was addressed in the provided reference (OPT-102A-6).
Form ES-401-9 Page 6 of 7 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 86 H
3 B
E A/ Modified Stem to include ABN-501 procedure title and reworded Distractors so that reason for the action was removed.
87 H
3 X
N U
C/ Revise using procedures in distracters as required by KA. Reworded Distractors to include SOP and ABN procedure titles.
88 H
2 N
S B/
89 F
2 N
S B/ Modified Explanation for correct answer.
90 H
3 X
B U
A/ K/A mismatch - K/A requires ability to locate and operate components, including local controls. Question as written asks knowledge of automatic functions. Replaced Question and K/A.
91 H
2 X
X N
U A/ Question can be answered by simple systems knowledge. Also not necessary to be knowledgeable of the applicable TS to answer this question. - RO level question.
Modified distractors by removing procedure references and expanding Technical Specification criteria.
92 H
3 N
S C/
93 F
3 B
E D/ Added RFO-302 procedure title to Stem.
94 F
2 N
S A/
95 H
3 N
S D/
96 H
3 N
S B/
97 F
2 N
S D/
98 H
3 B
E A/ Revised distracters to move common wording to the stem.
99 H
2 N
S C/
100 H
3 B
S D/
RO TOTALS:
B= 54 (72%)
F= 32 (43%)
E= 23 (31%)
Additional Notes:
M= 5 (7%)
H= 43 (57)
U= 6 (8%)
N= 16 (21%)
S= 46 (61%)
SRO TOTALS:
B= 11 (44%)
F= 5 (20%)
E= 6 (24%)
Additional Notes:
M= 2 (8%)
H= 20 (80%)
U= 4 (16%)
N= 12 (48%)
S= 15 (60)
Form ES-401-9 Page 7 of 7 OBDI 202 - IOLE Process
GENERAL COMMENT
S:
- 2.
The answer distribution is: RO / SRO A = 19 (25%)
/
6 (24%)
B = 23 (31%)
/
4 (16%)
C = 18 (24%)
/
7 (28%)
D = 15 (20%)
/
7 (28%)
- 3.
There are 2 RO and 0 SRO questions with attachments provided.