ML22019A146
| ML22019A146 | |
| Person / Time | |
|---|---|
| Site: | Wolf Creek  |
| Issue date: | 12/07/2021 |
| From: | Heather Gepford Operations Branch IV |
| To: | Wolf Creek |
| References | |
| Download: ML22019A146 (8) | |
Text
ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9
- 1. LOK 2. LOD 6
7 Ans Letter CFR 8
Q#
(F/H)
(1-5) stem focus cueing T/F cred dist partial job link minutia
- /
units back-ward K/A SRO-only B/M/N U/E/S A/B/C/
D 55.41x55
.43y Explanation 1
F 2
X N
U A
41.6 480 v power supply is not a credible distractor. I recommend changing the answers to a 2x2 and asking what bus the breaker is on and the normal supply for DC control power. Changed distractors and now SAT 2
H 3
N S
C 41.2 3
F 2
N S
C 41.6 4
F 2
X N
U B
41.5 Closing an inlet valve to a heat exchanger to lower temperature of RHR is not credible.
Discussed Q and now SAT 5
F 2
B S
A 41.3 6
F 2
N S
D 41.8 7
H 3
N S
C 41.9 8
H 2
N S
A 41.1
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 to 5 (easy to difficult) rating scale (questions in the 2 to 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- a. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- b. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- c. The answer choices are a collection of unrelated true/false statements.
- d. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- e. One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- a. The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- b. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- c. The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- d. The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
Place the answer letter here Place the 55.41 or 55.43 item here
- 8. At a minimum, explain any Unsat ratings (e.g., how the Appendix B psychometric attributes are not being met). a Rev 2
ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9 9
H 3
N S
C 41.7 10 H
2 x
B E
C 41.5
- Change the stem to ask what parameter will trip the plant first.
Revised stem and now SAT 11 H
3 N
S D
41.7 12 H
3 M
S A
41.7 13 H
2 x
N U
A 41.7 B is not credible. Discussed the questions and now SAT 14 F
2 N
S C
41.9 15 H
2 x
N U
B 41.9 a is not credible. Question rewritten and now SAT.
16 F
2 x
x N
U B
41.5 The distractors are a combination of T/F statements. Question rewritten and now SAT 17 H
2 x
N E
D 41.10 Stem information needs more info to make A and B credible distractors. Added information to the stem and now SAT 18 F
2 N
S A
41.7 19 F
2 x
N U
C 41.4 RWST and CCW are not credible distractors. Distractors changed and now SAT 20 H
2 X
N U
D 41.7 A is not a credible distractor. It is expected that ALL AFW throttle valves open when AFW actuates.
A distractor analysis is incorrect.
OK as is after more discussion now SAT 21 H
2 x
N E
D 41.5 The question stem can be shortened. First part of 2x2 would say "The diesel generator must be rotating in the _____
direction when the output breaker is closed." This could also be changed to ask "The diesel output voltage must be higher than the ________ volts."
running or incoming would be the answer choices. Rewrite will be provided. Rewritten and is now SAT 22 F
2 N
S B
41.7 23 H
2 N
S B
41.5 a Rev 2
ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9 24 H
3 x
N E
D 41.11 change the question stem. What HX is the source of the leak?
The question was fixed and now SAT 25 F
2 N
S A
41.7 26 F
2 N
S A
41.4 27 F
2 x
B U
C 41.7 A and D are not credible distractors. The switch does not exist. Rewritten and now SAT 28 H
2 B
S A
41.9 29 H
2 x
N E
C 41.2 get rid of the word "will".
Corrected and now SAT 30 H
3 x
N E
C 41.5 Why is distractor for B and D 2nd part wrong? If the applicant uses adverse CTMT parameters, what is the adverse consequence?
Clarified distractors, added procedure reference, now SAT 31 F
2 N
S B
41.9
- clarify explanation 32 F
2 x
N U
D 41.7 A & B are not credible distractors.
LO1403300, page 32 of 66 has a list of power supplies that would be good distractors. Replace NB01 and NB02 with NG03C and NG04C. Changed distractors and now SAT 33 H
2 X
N E
A 41.7 Add the word "maximum" to part
- 1. Corrected and now SAT 34 F
2 N
S D
41.5 35 H
2 x
x N
E B
41.4 I want the 2nd part of the 2x2 edited to ask for the minimum steam pressure of sealing steam.
Distractors changed and now SAT 36 H
2 x
N E
A 41.4 Is the "protect main condenser from over pressure" a valid distractor? If it is, we should add that there was a partial loss of the circ water pumps. Additional information was added to the stem and distractors are now plausible a Rev 2
ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9 37 F
2 x
N U
B 41.7 D is not credible. Would Porv or PZR safety leak by go to the RCDT? IF so, add to the stem that the unidentified leakage is 0 gpm. Reworked to have "PRT" source replace D and now SAT 38 F
2 N
S C
41.4 39 F
2 x
N E
A 41.10 Change SCM to 45 F and add to the stem that CTMT temperature is a value slightly higher than normal. Corrected and now SAT 40 F
2 x
N E
A 41.2 Why is B plausible? Discussed and now SAT 41 H
3 N
S C
41.5 verify not in scenarios and where is reference for part B. Overlap is not an issue because the written will be taken first. Now SAT 42 H
3 N
S A
41.10 43 F
2 x
x N
U A
41.7 A and D are correct choices. The fly wheel helps maintain RCS flow to avoid DNBR. The anti-rotation devices prevent flow from being diverted through the stagnant loop to avoid DNBR.
We need better distractors for B and C. Rewritten SAT 44 F
2 x
N E
C 41.5 eliminate the underlines in answers or add underlines to first two distractors. Underlines removed and now SAT 45 F
2 x
N E
D 41.8 I want to edit the distractors to be the alarm setpoints for the given parameters. Changed distractors and now SAT 46 F
3 M
S B
41.7 modified question from 2019 NRC exam 47 H
3 x
N U
C 41.10 underlines. C & D are same action but different reason Rewritten now SAT.
48 F
2 x
N E
C 41.10
2 x
N E
B 41.7
- change direct to immeadiate discuss. Discussed and now SAT a Rev 2
ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9 50 F
2 x
N E
D 41.7 add "designated" before rate.
Edited and now SAT 51 H
3 x
N E
B 41.8 can we cange this to a 2x2 question? Why is a credible?
Rewritten SAT 52 H
3 B
S D
41.5 53 H
3 N
S C
41.5 54 F
2 x
B E
C 41.9 RHR and SI are a subset of ECCS. Is choice C a good distractor? Rewritten SAT 55 H
3 x
N E
A 41.10
- "as necessary"? I want a rate that is wrong. "feed at full flow" Correction made and now SAT 56 H
2 x
B E
B 41.10
- change will to should.
Correction made and now SAT 57 H
2 x
x N
E C
41.1
- change will to should explain B&D credible with rising Tave?
Changed distractors and now SAT 58 H
3 x
N E
D 41.7 why is lowering turbine load a credible distractor. Rewritten SAT 59 H
2 N
S D
41.11 is this not in scenario 2? Writen exam is being taken before the OP test. SAT 60 H
3 x
N E
B 41.10 why are c&d credible? Rewrite SAT 61 F
2 x
N E
D 41.13 a is the same as "none of the above" and not allowed.
Corrected and now SAT 62 F
2 M
S C
41.14 used in 2011 NRC exam 63 H
3 N
S B
41.8 64 F
2 X
N E
A 41.10 C is not a plausible distractor.
Distractor C changed and now SAT 65 H
3 N
S B
41.9 66 H
3 x
x B
E D
41.1 I would like to modify this to have a MTC of 0. I would then want to replace distractor B with CCW temperature lowers by 15 F due to a malfunction in the CCW system. Distractor D fixed to make it incorrect. B distractor changed and now SAT 67 H
3 N
S C
41.5 a Rev 2
ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9 68 F
2 x
N U
D 41.10 cuing. D is in the question. A has nothing to do with oversight 69 F
2 x
N E
B 41.10 D is the same as "none of the above". How can A be reperformed? Explain what A means. Rewritten and now SAT 70 H
3 N
S D
41.2 71 F
2 N
S D
41.11 72 F
2 B
S C
41.12 73 H
2 N
S A
41.10 74 F
2 x
N U
D 41.10 add "required" in caps. D is an "if/then" and this is usually correct. Corrected and now SAT 75 F
2 x
N E
B 41.10
- discuss. Ensure that there are not multiple "correct" answers.
Discussed and now SAT 76 H
2 N
S C
43.2 77 H
3 x
N E
C 43.2
- reference provided* is the reference needed? Where is the reference. Reference removed 78 H
2 x
N E
D 43.5 Why is distractor B credible?
Discussed and now SAT 79 H
3 x
N E
D 43.2 discuss this. Distractor D looks correct instantly. Discussed and now SAT 80 H
3 x
N U
A 43.2 distractors a and b don't have TS actions and easy to eliminate. I want to change this to a 2x2.
Rewritten and now SAT 81 F
2 x
N E
D 43.3 discuss C. Do operators ever review design requirements or is this a function of engineering?
Discussed and now SAT 82 F
2 x
B U
B 43.6 This is asking about normal procedures. Where is the link to abnormal or emergency procedures? Changed the K/A and rewritten SAT 83 F
2 x
N E
D 43.4 most conservative choice is the correct answer. change the time from greater than 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> to less than 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> but greater than 24. Changed the stem to make the distractors more plausible and now SAT a Rev 2
ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9 84 H
3 x
x N
U B
43.5 Why would B ever be incorrect?
You always transition to E-0 from an FR procedure. Discussed and now SAT 85 H
3 x
B E
A 43.5 add to the stem that the attachment is complete allowing them to enter FR procedures.
Added information to the stem and now SAT 86 H
3 x
N U
B 43.5 why are distractors a,b,c credible? Removed information from the stem and now SAT 87 H
3 N
S A
43.2 88 H
3 x
x N
E B
43.5 eliminate the 2nd "take". Is there an actual "temporary diesel fire pump"? Does this meet K/A of determining the cause of loss of SW. Discussed and modified the stem. Now SAT 89 F
2 x
N E
B 43.1 discuss "licensing personel". Can we ask what the time limit for making the written notification?
Changed the 2nd part of question to asking about 10CFR50.73 and now SAT 90 H
3 N
S A
43.6 91 H
3 N
S D
43.5 92 H
3 x
x B
E D
43.5 change "will" to should. Is B credible? When do you ever go from E-1 to E-2? Can we change B to go to FRZ-1 and add CET temp rising? Modifed distractor B and now SAT 93 H
3 B
E D
we need to add cfr to the table.
CFR added and now SAT 94 F
2 N
S A
43.7 95 F
2 N
S C
43.1 96 H
3 x
B E
C 43.2
- when is the plant FIRST required to be in mode 5.
Modified question and now SAT a Rev 2
ES-401 WC-2021-11 Written Examination Review Worksheet Form ES-401-9 97 F
2 x
x B
E B
43.2 PV 2018 exam. I would like to modify this question. I want to change the 2nd part to ask what time the next surveillance is due and have in the stem that the last surveillance was completed 9.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> (1.5 hours5.787037e-5 days <br />0.00139 hours <br />8.267196e-6 weeks <br />1.9025e-6 months <br /> late) after the initial surveillance. Modified the question and now SAT 98 F
2 B
S C
43.4 99 F
2 x
N E
A 43.4 change the "to" to "into".
Corrected and now SAT 100 H
3 B
S A
43.5 Is C explanation correct?
Distractor analysis corrected and now SAT Results Table RO LOK -H 38 Avg RO LOD 2.29333 Flaws 10 CFR Distribution RO LOK-F 37 AVG SRO LOD 2.56 Stem focus 22 41.1 17 43.1 2 SRO LOK - H 16 Overall LOD 2.36 Cues 5
41.2 4
43.2 7 SRO LOK - F 9
T/F 1
41.3 1
43.3 1 Cred Dist 35 41.4 5
43.4 3 RO Bank 9
12 SRO Bank 8
32 Partial 2
41.5 12 43.5 8 RO Mod 3
4 SRO Mod 0
0 job link 0
41.6 2
43.6 2 RO New 63 84 SRO New 17 68 units 0
41.7 17 43.7 1 minutia 0
41.8 4
Total Bank 17 17 backward 0
41.9 7
Total Mod 3
3 KA 2
41.10 17 Total New 80 80 SRO-only 0
41.11 3
LOD = 1 0
41.12 1
32 41.13 1
16 Answer Dist ( in %)
41.14 1
RO Edit 28 37.33 SRO Edit 13 52 RO-A 18 SRO-A 7
RO-B 17 SRO-B 6
Total Sat 41 41 Total Unsat 18 18 RO-C 21 SRO-C 5
Total Edit 41 41 RO-D 19 SRO-D 7 a Rev 2