ML082830709
| ML082830709 | |
| Person / Time | |
|---|---|
| Site: | Diablo Canyon  |
| Issue date: | 06/27/2008 |
| From: | Ryan Lantz Operations Branch IV |
| To: | Keenan J Pacific Gas & Electric Co |
| References | |
| 50-275/08-301, 50-323/08-301 | |
| Download: ML082830709 (9) | |
Text
ES-401 DC-2008-06 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Chief Examiner Comments in BLUE 1 F 2 N S CFR: 41.10 2 H 3 N S CFR: 41.14 Incorrect CFR referenced (41.8)
CFR: 41.7 Answer D is the longest and uses the word needs. The other three E distractors are statement of what the LOCA does; whereas D is a statement of what 3 F 2 E B the LOCA requires for success. This could be a cue.
S This question is OK as written except the CFR reference was changed.
4 F 2 N S CFR: 41.7 5 H 3 B S CFR: 41.7 CFR: 41.7 Incorrect CFR referenced (41.10) since this is asking only system level 6 H 3 B S knowledge.
7 H 2 M S CFR: 41.7 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
$ The answer choices are a collection of unrelated true/false statements.
$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
$ The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 9 OBDI 202 - IOLE Process
ES-401 DC-2008-06 2 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Chief Examiner Comments in BLUE CFR: 41.7, 10 C is not plausible. All should know that high letdown temperature is not a reason for tripping the Rx. B also correct since exceeding 235F (setpoint) also U exceeds 200F (value given).
8 H 3 E U N S Distractor B will be rewritten.
The distractors were rewritten to make them more plausible.
CFR: 41.7 Incorrect CFR referenced (41.10). Explanation of B states wrong failure 9 H 3 N S mode of PT-457.
The CFR reference was changed and distractor B was corrected.
10 F 3 M S CFR: 41.11 11 H 2 N S CFR: 41.5 CFR: 41.7 Distractor A is correct too, since the question is: How long can the engine be operated? It will definitely run 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> if it can run 30 hours3.472222e-4 days <br />0.00833 hours <br />4.960317e-5 weeks <br />1.1415e-5 months <br />. Question is a direct U look-up.
12 F 1 U B S Distractor A will be rewritten.
The original question was replaced due to being unable to correct the original question.
E CFR: 41.7 Change semicolon in B to comma.
13 H 2 B S This was edited as indicated.
CFR: 43.5 SRO level question U
14 H 2 U B This question will revised.
S This question was rewritten to make it an RO level question.
15 F 2 N S CFR: 41.7 CFR: 41.7 C not plausible since heating up the RCS to clear an interlock while in E
16 H 3 E B EOPs is not credible.
S Distractor C was revised.
17 H 2 B S CFR: 41.10 E CFR: 41.10 C states to each faulted S/G. Is a generator with <6% NR faulted?
18 H 3 B S This question is OK as written.
19 F 3 B S CFR: 41.7 20 F 3 B S CFR: 41.7 Page 2 of 9 OBDI 202 - IOLE Process
ES-401 DC-2008-06 3 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Chief Examiner Comments in BLUE E CFR: 41.7 Reword the stem to make the question readable.
21 H 2 N S The stem was reworded.
CFR: 41.7 For B, swap the order of listed actions to make the phrase after the and E
22 H 3 N stand alone.
S This distractor was edited as indicated.
U CFR: 41.10 Need to give indication, and elicit a response. Direct lookup.
23 F 2 U B S This question is ok as written.
24 F 2 M S CFR: 41.11 25 F 3 B S CFR: 41.8 26 H 2 B S CFR: 41.10 27 H 3 M S CFR: 41.10 E CFR: 41.7 Change after a reactor trip to after the reactor trip.
28 F 2 B S This question was edited as indicated.
E CFR: 41.5 Buses spelled wrong in stem.
29 F 2 B S The spelling error was corrected.
30 F 3 M S CFR: 41.7 31 H 2 B S CFR: 41.5 CFR: 41.7 Distractors B and C are not credible since the RHR system cannot inject U into the RCS while at 100%, NOP. No controls at HSDP. This is not discriminating.
32 H 2 U N S The distractors will be revised.
The question was revised to focus on indications only.
CFR 41.14 Direct lookup. Should the applicants be able to know from memory U
33 H 2 B approximate pressures and flow rates (up to run-out)?
S This question is ok as written.
CFR 41.7 The vapor space temperature given in the stem would result in an actual pressure that is nearly double what is provided in the stem for wide range RCS E
34 H 3 B pressure. Choose values sufficiently close in steam tables to make this more S realistic. Good question though.
The pressure values will be changed. Page 3 of 9 OBDI 202 - IOLE Process
ES-401 DC-2008-06 4 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Chief Examiner Comments in BLUE This question was revised to be a failure of a temperature instrument.
CFR: 41.7 All applicants should know there is no relief valve on the tank; therefore, E
35 F 2 E B A is not plausible.
S This question is ok as written.
CFR: 41.5 C and D state that removing boron is a boration. Is not any boron U removal a dilution? Adding boron is the only way to borate, or raising temperature on 36 H 3 U B a saturated resin bed? This is not plausible.
S This question is ok as written.
CFR: 41.7 K/A listed on the question is incomplete. Should state, knowledge of the U bus power supplies to Pzr heaters. Therefore, as written, this is a K/A mismatch.
37 H 3 U M S This question will be rewritten.
This question was rewritten to match the KA.
38 F 2 B S CFR: 41.7 39 F 3 B S CFR: 41.7 40 F 2 B S CFR: 41.7 CFR: 41.7 A and B are not credible since the controls are not in these locations.
U Change the distractors to deal with manual restart on the panel.
41 F 2 U B S This question will be rewritten.
Distractors A, B, and C were rewritten to address manual restart.
42 F 3 B S CFR: 41.7 43 H 3 N S CFR: 41.8 CFR: 41.4 Edit distractors A and B. The use of punctuation and wording makes E
44 F 3 M both of them confusing.
S Distractors A and B were edited to address this comment.
CFR: 41.7 Need to change one of the distractors, preferably D to discriminate E
45 H 2 E B between Trip alone and both SI and Trip (ie Low_Tavg)
S Distractor D was revised.
46 H 3 M S CFR: 41.7 47 H 3 B S CFR: 41.5 Page 4 of 9 OBDI 202 - IOLE Process
ES-401 DC-2008-06 5 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Chief Examiner Comments in BLUE F U CFR: 41.7 Need to revise to remove reverse logic.
48 3 U B H S This question is ok as written.
CFR: 41.5 It seems highly unlikely that anyone would choose to defeat all 24 alarms on PK-20. (Distractor A). As written, this question does not require the applicant to H U 49 2 E U N predict the impact of the ground. Nothing in the procedure references placing the S detector in CUTOUT.
This question was rewritten.
CFR: 41.5 A and C use Jacket Water Pressure. This is independent of load.
E 50 F 2 E M Recommend changing this to some other load dependent item.
S Distractors A and C were revised.
51 H 2 B S CFR: 41.7 CFR: 41.11 Wrong CFR referenced (41.5) 52 F 2 B S Changed the CFR references.
F 53 3 B S CFR: 41.7 H
54 H 3 N S CFR: 41.7 55 H 3 B S CFR: 41.7 CFR: 41.7 Items 2, 3 and 4 are not discriminated, since they are in all the answers.
Items 1 and 5 are not plausible since there is no high C pressure Rx trip, and U containment is designed to withstand a DBA LOCA, hence the word DESIGN. The 56 H 1 U U N only knowledge needed to answer this question is whether items 1 and 5 are true or S
not.
This question was rewritten.
57 H 3 N S CFR: 41.10 58 F 2 B S CFR: 41.7 CFR: 41.2 The in-core TC measure Temperature at the exit of the core. Due to heat transfer from clad to steam/water, an 1100F ITM reading does not necessarily U correlate to 1100F clad temperature along the full length of a fuel pin, and could be 59 F 2 U U M significantly higher, allowing there to be no correct answer. The references provided S
show the 1300F temperature to be clad temperature.
This question was revised. Page 5 of 9 OBDI 202 - IOLE Process
ES-401 DC-2008-06 6 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Chief Examiner Comments in BLUE E CFR: 41.7 B states the fan is energized and tripped. Reword.
60 F 2 E N S Distractor B was revised.
61 F 2 B S CFR: 41.7 U CFR: 41.7 The applicant can answer this question without looking at the stem.
62 F 2 U M S This question was rewritten.
63 H 3 N S CFR: 41.5 CFR: 41.7 SV-40 is indirectly activated by <45psig on AST header pressure switch PS-22a,d,e. Low vacuum will activate SV-171 and SV-37. Either one of these will depressurize the AST header, and activate SV-40. Therefore, someone could argue that A is also a correct answer.
U 64 H 3 U B Upon further review, there does appear to be two correct answers. Also, it is not S clear if the level of detail for this question is appropriate. I.e., are the candidates expected to have the individual EHC valve numbers memorized? This knowledge is required to be able to answer the question.
This question was rewritten to address the comments above.
CFR: 41.10 What does back down the compressor mean in A. Revise to clear this up. C is not plausible with stem worded as is. The precaution states that the SA UE 65 F 3 E E N receiver is to be isolated when IA is supplying SA through PCV-114. This is contrary S to the stem statement.
This question is ok as written.
CFR: 41.10 Give a pressure value in the stem. As written, this cues the applicant E
66 H 2 E E B that something is required at 1950 (ie. Trip the RX)
S This question is ok as written.
CFR: 41.10 Reword the stem to make sense: Only licensed operators are permitted U to operate which of the following controls per Again, each answer has the same 67 F 2 U M item in it. No discrimination. Recommend revising or select another question.
S The question was rewritten.
CFR: 41.10 Are the applicants required to know form memory what each STP entails S?
68 H 4 N (simple or complex)
S This question is ok as written.
CFR: 41.10 Remove the parenthesis from the OP designation. Change stem to: The E
69 F 2 N use of the Pink Tag is:
S This question was edited as indicated. Page 6 of 9 OBDI 202 - IOLE Process
ES-401 DC-2008-06 7 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Chief Examiner Comments in BLUE CFR: 41.10 Change in and hour in second line of stem to in an hour. Anything U?E preventing the CO from going to CSR and performing this (C)? Seems as though the 70 H 2 U? N procedure is unclear on this.
S This question is ok as written.
CFR: 41.12 As written, this seems to be a test item that all personnel are required to U?
71 F 2 U? M know for site access.
S This question is ok as written.
U CFR: 41.12 This is generic knowledge for site access.
72 F 2 U M S This question is ok as written.
73 F 2 B S CFR: 41.10 E CFR: 41.10 Item 2 is used in all the answers. No discrimination of this item.
74 F 3 N S This question was revised.
75 F 2 N S CFR: 41.10 76 H 2 B S 77 H 3 M S U Given the TS reference this is a direct lookup question.
78 H 1 U B S This question was rewritten.
79 H 3 N S 80 H 2 N S 81 H 2 B S Given the reference, this is a direct lookup. Also, the question does not match the U
82 H 3 U U B KA.
S This question was revised.
83 F 2 B S 84 H 2 B S 85 H 2 B S 86 H 3 N S 87 H 3 B S Page 7 of 9 OBDI 202 - IOLE Process
ES-401 DC-2008-06 8 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Chief Examiner Comments in BLUE 88 H 3 M S 89 H 3 N S S This is an RO question because it tests on the action taken in response to an event.
90 F 3 U U M U Also this is a two part KA and the question does not test on both parts.
S This question was rewritten.
This is an RO question in that it is a basic administrative procedure change question U
91 H 2 U N that is required knowledge for the RO.
S This question was rewritten.
92 H 2 N S 93 H 3 N S 94 F 2 M S 95 H 2 M S 96 F 2 B S 97 F 2 N S U This is a direct lookup.
98 F 2 N S This question was revised.
99 H 3 M S U This question overlaps one of the JPMs.
100 H 2 N S This question was replaced.
B= 37 F= 37 E= 13 RO TOTALS: M= 14 H= 38 (51%) U= 12 Additional Notes:
N= 24 B= 9 F= 6 E= 0 SRO TOTALS: M= 6 H= 19 (76%) U= 4 Additional Notes:
N= 10
GENERAL COMMENT
S: Page 8 of 9 OBDI 202 - IOLE Process
ES-401 DC-2008-06 9 Form ES-401-9
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N.
- 2. Chief Examiner comments are indicated in blue.
B = 18 (24%) / 6 (24%)
C = 17 (23%) / 6 (24%)
D = 15 (20%) / 3 (12%)
Change 2 SRO questions to make D the correct answer. Page 9 of 9 OBDI 202 - IOLE Process