ML21020A093
| ML21020A093 | |
| Person / Time | |
|---|---|
| Site: | Palo Verde  |
| Issue date: | 12/10/2020 |
| From: | Greg Werner Operations Branch IV |
| To: | Arizona Public Service Co |
| References | |
| Download: ML21020A093 (10) | |
Text
ES401PV202011WrittenExaminationReviewWorksheetFormES4019
- 1. LOK 2. LOD 6
7 Ans Letter CFR 8
Q#
(F/H)
(1-5) stem focus cueing T/F cred dist partial job link minutia
- /
units back-ward K/A SRO-only B/M/N U/E/S A/B/C/
D 55.41x55
.43y Explanation Appears to be too many "B" answers on the RO (26). There are also not enough "C" answers in the SRO portion (3).
1 H
3 N
S B
41.7 Need to discuss why only one phase light causing a reactor trip plausible (A and C). In the entire plant it seems like you need more than one of anything to generate a trip signal. Discussed/Q considered SAT 2
H 2
X X
N E
C 41.14 Subcooling and superheat are two different things. Therefore, need to get subcooled out of the stem since it doesn't match C or D. Discuss modifying distractors so all four are superheat.
Revised/Q now SAT
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 to 5 (easy to difficult) rating scale (questions in the 2 to 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- a. The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- b. The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- c. The answer choices are a collection of unrelated true/false statements.
- d. The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- e. One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- a. The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- b. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- c. The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- d. The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
Place the answer letter here Place the 55.41 or 55.43 item here
- 8. At a minimum, explain any Unsat ratings (e.g., how the Appendix B psychometric attributes are not being met). a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 3
H 3
X N
U A
41.10 As written, all four distractors are correct as they are all below 73%
RWT level. Add "a MAXIMUM of" at end of stem.Revised/Q now SAT 4
F 2
X B
E D
41.10 ON 2016 NRC EXAM. Change "will's" to "should's" in stem.
Revised/Q now SAT 5
H 3
X X
N E
A 41.3 Need LOD. Change "will" to "should" in stem. First part of distractors could be confusing if applicant thnks "lower" and "higher" refer to pitch. Better to use "more noise" and "less noise" than normal. Revised/Q now SAT 6
H 3
N S
C 41.7 7
F 3
X N
E A
41.7 In B and D, change "event of accident" to "event of a design basis accident."Revised/Q now SAT 8
H 3
X N
U D
41.14 Change "will's" to "should's" in stem. Appears to be the same thing as Event 2 in scenario 3 (replace) Replaced/Q now SAT 9
F 2
N S
D 41.7 10 H
2 X
X N
E A
41.5 Change "will" to "should" in stem.
1135 psia seems high. What is normal operating SG pressure?
Discuss how K/A is met.Discussed/Q now SAT 11 H
3 X
N E
A 41.5 Change "will" to "should" in stem.
Don't quite understand explanations (opening aux spray stops a depressurization?).
Please explain.Explanations revised/Q now SAT 12 H
3 X
B E
A 41.7 ON 2016 NRC EXAM. Change "will's" to "should's" in stem.
Revised/Q now SAT a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 13 H
2 X
N U
B 41.4 Don't see any real connection between instrument air and MSIS.
Also, (1) and (3) both appear to be obvious results of an MSIS.Revised distractors to make more plausible. Q now SAT 14 H
3 N
S A
41.4 Confidence level not high that anyone would select CEAC over COLSS (discuss).Discussed/Q considered SAT 15 H
3 N
S A
41.8 16 F
2 X
N E
D 41.4 Change "will's" to "should's" in stem. Revised/Q now SAT 17 H
3 X
N E
D 41.4 Change "will" to "should" in stem.
Revised/Q now SAT 18 H
3 X
N E
A 41.5 Change "will's" to "should's" in stem.Revised/Q now SAT 19 H
3 X
X N
U A
41.6 REFERENCE PROVIDED.
Change "will" to "should" in stem.
Knowledge that a periphery control rod would have less of an effect on actual reactivity than one near the center of the core doesn't appear to have much discriminating value (distractors C and D not plausible).Replaced/Q now SAT 20 H
3 X
X N
E A
41.6 Change "will" to "should" in stem.
Doesn't seem likely that anyone would think pusle counters would be used over Reed switch indication. Discuss possible enhancement alternatives.
Explanations enhanced to explain better while pulse counters are plausible./Q now SAT 21 F
3 X
N E
B 41.11 Change "will's" to "should's" in stem. Revised/Q now SAT 22 F
2 N
E B
41.11 Please discuss explanations. RU-8 is mentioned in the explanation for D even though it isn't part of the answer?Revised/Q now SAT 23 F
2 X
N E
B 41.8 Change "will's" to "should's" in stem.Revised/Q now SAT a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 24 H
3 X
N E
D 41.4 Change "will's" to "should's" in stem. Please discuss explanations. B and D modified "isolate" to "restrict." /Q now SAT 25 F
2 X
B U
D 41.11 ON 2016 NRC EXAM. Student need only know the location and nothing else to correctly answer question. Not plausible that a contaiment area monitor would be detecting RCS activity better than an RCS monitor with no LOCA (distractors A and B).
Restructured/ Q now SAT 26 F
2 N
S A
41.14 27 H
3 X
X N
E B
41.10 REFERENCE PROVIDED. Why is a reference really necessary here? Change "after the SG blowing down and then isolated" to "after SG#1 has been isolated."Revised/Q now SAT 28 H
3 N
E D
41.3 Change "will" and "would" to "should" in stem. Revised/Q now SAT 29 H
3 B
S C
41.3 OK, but distractor A is essentially a "None of the Above" type distractor and should be avoided.
30 H
3 X
B E
D 41.3 Change "will" to "should" in stem.
Revised/Q now SAT 31 H
3 N
S B
41.7 REFERENCE PROVIDED.
32 H
3 X
B E
D 41.7 ON 2018 NRC EXAM. Change "will's" to "should's" in distractors.
Revised/Q now SAT 33 F
2 X
X N
U C
41.7 Change "will's" to "should's" in stem. Possible two correct answers. It is not a "required" action to manually actuate CSAS on trend (should not shall), so if the crew lets it auto actuate, that would be less preferred, but apparently not "wrong." Revised stem to clarify per procedural requirements./Q now SAT a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 34 H
3 X
N E
B 41.5 Need period after "open" in stem.
Change "will" to "should" in stem.
Revised/Q now SAT 35 H
3 X
B E
A 4.11 ON 2019 NRC EXAM. Change "will" to "should" in stem.Revised/Q now SAT 36 H
3 N
S C
41.7 37 H
3 N
S C
41.5 38 F
2 N
S A
41.8 39 H
3 X
X N
E C
41.7 Change "will's" to "should's" in stem. Change "Channel A" to "Channel C" in stem and change correct answer to "C" (more discriminating).Revised/Q now SAT 40 H
2 N
S C
41.7 41 F
2 N
S A
41.4 42 F
2 X
N U
C 41.8 Explanation says a loss of PGA-L35 or PHA-M35 will cause a loss of the valve. Two correct answers (A and B)?Restructured/Q now SAT 43 H
3 B
E B
41.7 Question indicates it is modified bank, but no original question is attached. Bank Q 44 H
3 X
B E
A 41.14 ON 2018 NRC EXAM. Change "will" to "should" in stemRevised/Q now SAT 45 H
3 X
N E
B 41.4 Change "will's" to "should's" in stem. Revised/Q now SAT 46 H
2 N
S D
41.4 C doesn't appear plausible. Why would someone presume that an AFW pump would stay running to an isolated SG?Discussed/Q considered SAT 47 F
2 X
N E
B 41.8 Should the minimum temperature of 60° be in the stem?Revised/Q now SAT 48 H
3 X
B E
D 41.7 ON 2016 NRC EXAM. Change "will" to "should" in stem. Discuss modifications from original question.Bank Q/Revised/Q now SAT 49 H
3 N
S B
41.7 a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 50 H
3 X
X N
E A
41.11 Change "will's" to "should's" in stem and distractors.Revised/Q now SAT 51 F
3 M
S D
41.7 52 H
3 B
S C
41.4 ON 2016 NRC EXAM.
53 F
3 B
S D
41.4 ON 2019 NRC EXAM.
54 F
2 N
S A
41.4 55 F
3 X
N U
A 41.9 B(2) and D(2) do not appear plausible when doing A(2) or C(2) would stop any potential leak.Restructured/Q now SAT 56 H
3 N
S C
41.6 57 F
2 B
S B
41.14 ON 2016 NRC EXAM. Seems far fetched that someone would think delta T >65° in natural circulation (discuss)Discussed/Q considered SAT 58 H
3 N
S B
41.6 59 F
2 N
S C
41.2 60 F
3 B
S D
41.13 ON 2016 NRC EXAM.
61 F
2 X
N E
B 41.8 Discuss plausibility of hyrdogen recombiners being powered with 4160 volts (distractors C and D).Discussed/Q considered SAT 62 H
2 X
X N
E A
41.4 Provide power level in stem.
Discuss possible alternatives to steam generator low level distractors SG level distractors replaced/Q now SAT 63 H
3 X
N E
D 41.1 Change "will" to "should" in stem.
Need to modify (1) in stem to specify SBCV's other than 1001 and 1004.Revised/Q now SAT 64 F
3 N
S C
41.9 65 F
2 X
N E
D 41.11 beta not very believable - maybe neutron instead? Also, B(2) and D(2) are essentially "none of the above" type distractors and should be avoided.Revised/Q now SAT a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 66 H
3 N
S B
41.10 OK, but A(2) and C(2) are essentially "none of the above" type distractors and should be avoided.
67 F
3 X
N E
B 41.10 Change "will" to "should" in stem.
Revised/Q now SAT 68 F
2 B
E D
41.10 ON 2016 NRC EXAM. Listed as new question? Bank Q 69 F
2 X
X N
U B
41.10 No correct answer as written.
Reference indicates double valve isloation required for GREATER than 200 degrees or 500 psig.
Restructure Revised/Q now SAT 70 F
3 X
X N
U C
41.10 Change "will" to "should" in stem.
A and B not credible, as "any SM" too ambiguous (any qualified SM?) + if "any SM" was correct, then the Unit 1 SM would also be correct. Also, need to discuss plausibility of "Salt River Project."
Restructured/Q now SAT 71 F
3 N
S C
41.9 72 F
2 B
E B
41.12 ON 2016 NRC EXAM. Does not appear to be a modified bank question (looks like same question, but the answer has changed?) Bank Q 73 F
2 N
S D
41.12 74 F
3 N
S C
41.10 75 F
3 X
N E
C 41.10 It appears from the reference that the FTA needs to be both a qualified RO and Aux Operator? If so, then there could be two correct answers (C and D).
Discuss to make sure D is wrong.Revised/Q now SAT 76 F
3 N
S D
43.4 77 F
3 N
S B
43.2 78 F
3 N
S B
43.2 a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 79 H
3 X
N U
B 43.2 "2" is obviuosly correct (feed break/SG low level), which means "1" cannot be correct with the question as written. This automatically makes only two distractors possible (B and D)/(2) changed to "ctmnt press high"/Revised; Q now SAT 80 H
3 X
N U
C 43.5 Change "will" to "should" in stem.
Can't think of any circumstance where an SBO diesel is preferred over an available (normal) EDG.
C and D do not appear plausible.Replaced; Q now SAT 81 H
3 X
N E
D 43.5 Change "will" to "should" in stem.
D explanation should be in C block./Revised; Q now SAT 82 H
3 N
S D
43.5 Distractors A and C don't appear very plausible with both boric acid makeup pumps OOS. Any alternative?/Q determined OK based on procedural direction; Q SAT 83 H
3 N
S B
43.2 84 F
3 B
E D
43.2 ON 2016 NRC EXAM.
Explanation for C appears incomplete./Revised; Q now SAT 85 H
3 X
N U
D 43.4 Release exceeding federally approved limits does not appear plausible given there are zero indications of any abnormal radiation in stem (distractors A and C)./Add "design basis SGTR event" in stem/Revised; Q now SAT 86 H
3 X
N E
A 43.5 Change "will" to "should" in stem.
Why is reference provided box X'd?/Typo/Revised; Q now SAT 87 F
3 X
X B
E C
43.2 ON 2018 NRC EXAM. Distractors B and D suggest going to an alarm response procedure, but there is no alarm in the stem (insert one)./Alarm inserted in stem/Revised; Q now SAT a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 88 H
3 X
N E
A 43.5 REFERENCE PROVIDED (need to note in question). Change "will" to "should" in stem. /Revised; Q now SAT 89 H
3 X
N U
B 43.5 Fire main seems far fetched (distrators C and D). Discuss other potential sources (HPSI?)/Changed to HPSI/Revised; Q now SAT 90 H
3 X
X N
U B
43.5 Change "will's" to "should's" in stem. If the CRS opted to energize PBB-S04, why would that be "wrong?" (possible two correct answers).Add procedure in stem/Revised; Q now SAT 91 H
3 N
S A
43.2 92 H
3 N
S A
43.2 93 F
3 N
S A
43.2 94 H
3 N
S C
43.5 REFERENCE PROVIDED 95 F
2 B
S C
43.7 ON 2016 NRC EXAM. Question says modified bank. Don't see any substantial modification./Remains a Bank Q 96 F
3 X
N U
C 43.5 Discuss. C and D did not appear very plausible. Why would you ever use an off watch SRO as the reactivity manager if the on shift licensed STA was allowed to?/B and D changed to Reg Gp.
1/Revised; Q now SAT 97 F
3 B
S B
43.2 ON 2016 NRC EXAM 98 F
3 N
S C
43.4 99 F
2 N
S A
43.5 100 H
3 N
S D
43.5 Results Table RO LOK -H 42 Avg RO LOD 2.64 Flaws 10 CFR Distribution RO LOK-F 33 AVG SRO LOD 2.92 Stem focus 38 41.1 11 43.1 0 SRO LOK - H 14 Overall LOD 2.71 Cues 1
41.2 1
43.2 10 SRO LOK - F 11 T/F 0
41.3 4
43.3 0 Cred Dist 20 41.4 12 43.4 3 a Rev4
ES401PV202011WrittenExaminationReviewWorksheetFormES4019 RO Bank 16 21.33 SRO Bank 4
16 Partial 5
41.5 5
43.5 11 RO Mod 1
1.333 SRO Mod 0
0 job link 0
41.6 4
43.6 0 RO New 58 77.33 SRO New 21 84 units 0
41.7 15 43.7 1 minutia 0
41.8 6
Total Bank 20 20 backward 0
41.9 3
Total Mod 1
1 KA 1
41.10 11 Total New 79 79 SRO-only 0
41.11 5
LOD = 1 0
41.12 2
RO Sat 29 38.67 SRO Sat 14 56 41.13 1
24 Answer Dist ( in %)
41.14 5
20 RO-A 20 SRO-A 6
RO-B 19 SRO-B 7
Total Sat 43 43 Total Unsat 16 16 RO-C 17 SRO-C 6
Total Edit 41 41 RO-D 19 SRO-D 6 a Rev4