ML19099A374
| ML19099A374 | |
| Person / Time | |
|---|---|
| Site: | Callaway  |
| Issue date: | 03/13/2019 |
| From: | Greg Werner Operations Branch IV |
| To: | AmerenUE |
| References | |
| Download: ML19099A374 (21) | |
Text
Note: This is a header page to separate the draft and final resolved versions of the ES-401-9 (Draft Written Exam Comments Form) and the ES-301-7 (Draft Operating test Comments Form) for the Adams package. The first file for both the draft written and draft operating test forms contains only the draft comments with stats for the as submitted draft materials. This header page separates the draft comments from the Final comment files that contains final resolutions of those items for both of these forms.
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 a Rev 3
- 1. LOK 2. LOD 6
7 Ans Letter CFR 8
Q#
(F/H)
(1-5) stem focus cueing T/F cred dist partial job link minutia
- /
units back-ward K/A SRO-only B/M/N U/E/S A/B/C/
D 55.41 55.43 Explanation Callaway noted that they currently do not use the previous two NRC exams (that are withheld from the public for two years) as audit exams and they indicate that on the QA forms by building a completely new audit exam for the class each time.
1 F
2 N
S C
7 2
H 2
B E
A 8
2014 NRC Q45 - Put LB LOCA in stem of question to ensure KA tie.
3 H
3 N
S C
1 4
H 2
N E
C 10 Put 564F next to "Vessel Head" in explanation for C to ensure tie to given stem value as correct answer.
5 H
2 M
E A
10 NEED PARENT Q 6
H 2
B S
C 10 7
H 2
M E
D 10 NEED PARENT Q 8
H 2
B S
C 10 9
F 2
N S
C 7
10 F
2 B
S A
10 11 F
2 B
E A
10 Distracter D has grammer error.
Should be rewritten to state that "
To prevent overfilling the faulted steam generator and cause its atmospheric dump valve to lift" 12 H
2 N
E A
5 Remove "a" before RCS subcooling in part 2 of stem 13 H
3 M
S B
7 NEED PARENT Q 14 H
3 B
S B
7 2014 NRC Q14 15 F
3 N
S D
7 16 H
2 M
S D
5 17 F
2 B
S B
7 2009 NRC
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 a Rev 3 18 H
2 X
N U
B 8
Per the explanation it appears that A is the correct answer not B. You moved the correct answer to A but did not update the explanation after the free review on this question.
19 H
3 B
S C
5 20 H
3 B
S C
7 21 H
3 B
S B
8 2012 POINT BEACH NRC 22 F
2 N
E C
11 55.41(b)11 is better suited to this question than your documented 55.41(b)7 23 H
3 N
S C
10 24 H
2 N
S C
10 25 H
4 N
S B
10 26 H
4 M
S D
10 NEED PARENT Q 2017 AUDIT EXAM 27 F
3 X
N E
D 10 I don't see the plausibility of distracter A because you are high in the band and uncovering heaters would be low in the band, unless you put in some info about the concern for the void to move into the pressurizer.
28 H
3 X
N E
A 10 Distracter D is not credible as the stem is setup. You could use what you had in the first version of this question with "Rx power drops to less than 5% with a negative SUR."
29 H
2 N
S B
5 30 F
2 N
S B
10 31 H
2 M
S A
5 NEED PARENT Q 32 H
2 B
S B
14 ST NRC 2017 33 H
4 N
S A
10 34 H
2 B
S A
7 2017 NRC Q52 35 F
3 N
S D
7 36 F
3 B
S A
7 2016 NRC Q36 37 H
3 N
S D
7
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 a Rev 3 38 F
2 N
S C
7 39 H
3 N
S B
7 40 F
2 B
S B
7 41 F
2 B
S B
7 2016 NRC Q41 42 F
2 N
S D
8 43 F
3 B
S A
14 44 F
3 B
S A
7 2016 AUDIT Q44 45 H
3 B
S A
10 2009 NRC 46 F
2 N
S A
10 47 F
2 N
S D
4 48 H
3 B
S B
5 49 H
3 B
S B
7 2014 NRC Q49 50 H
3 B
S B
7 2014 NRC Q8 51 H
3 B
S A
10 52 H
3 X
N E
C 7
Edit required - B2 and D2 are not credible, so these two distracters are partially not credible. Need to change this second part to something more credible such as basis for room temp, the valves that stroke to isolate SW and ESW, or etc.
53 H
3 N
S B
4 54 F
3 X
X N
E D
7 Edit - 2011 NRC There is no design limit challenge in the question or answer and there is cueing between the stem iin two places and the answer. Distracter A and B are not credible because there is nothing else in the stem about them. Since this was previously approved it is an Edit but it needs major work to be used on this exam.
55 F
3 N
S A
7 56 H
3 M
S D
10 NEED PARENT Q 57 H
2 B
S D
10 typo of "gharging" in explan.
section
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 a Rev 3 58 H
3 N
S C
7 59 H
3 N
S B
5 60 F
3 B
S B
5 2014 NRC Q58 Might want to put in explanation that 4% is the LEL to ensure tie to the KA is seen.
61 H
2 N
S A
4 62 H
2 N
S A
7 63 H
2 B
S B
7 2017 AUDIT Q62 64 H
2 M
S B
10 NEED PARENT Q 65 F
2 N
S D
7 66 F
2 B
S A
10 67 F
2 N
S B
10 68 F
3 M
S A
10 NEED PARENT Q 69 F
2 B
S D
10 2016 AUDIT Q68 70 F
2 B
S D
10 2014 NRC Q70 71 F
3 B
S C
11 2015 AUDIT Q71 72 H
2 B
S B
12 2017 NRC Q72 73 H
2 N
S B
10 74 F
2 B
S D
10 2016 AUDIT Q74 75 F
3 B
S D
10 76 H
3 N
S D
5 77 H
3 N
S D
5 78 H
3 M
S D
5 2013 NRC Q78 parent 79 F
4 N
S D
2 80 H
3 N
S D
5 81 H
4 B
E C
5 2016 NRC Q81 Missed on 2016 NRC review but question has EOP Add 30 in all four choices so should move this to the stem per NUREG, and then this eliminates the subset isue with Distracter D. Use Add 31 as a swap for the loss of Add 30 to the stem as we discussed.
82 H
3 N
S A
5
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 a Rev 3 83 H
2 B
S D
5 84 H
3 N
S C
5 HANDOUT NEEDED 85 H
3 N
S B
5 86 F
3 N
E B
2 C distracter not credible. Modify stem by removing at least one from it and make distracter C "Cool dowen the RCS from 350F to 212F in 2.8 hrs." This is the FSAR design basis for two full trains of RHR not one and so it would not be correct nor even close for anyone who has performed or observed a cooldown for shutdown so it has operational validaty as well. Also, fix grammer in Distracter D to "iodine release rates due to a postulated fuel handling....".
Licensee made requested repairs but still has grammar error in stem (release is singular so use "is" not "are"....within limits).
87 H
3 N
S A
2 88 H
3 B
S C
1 2013 NRC Q86 89 F
4 B
S B
2 2014 NRC Q88 90 H
3 B
S A
2 HANDOUT NEEDED 91 H
4 B
S B
5 HANDOUT NEEDED 2014 SRO RETAKE NRC 92 F
3 N
E A
7 Edit - partial non-credible distracters. By knowing the order you can eliminate the second half of the question choices or by knowing the reason you can eliminate the first part of the choices. Use balanced 2 X 2 as dicussed on phone.
93 H
3 N
S A
2 94 F
3 B
S B
1 2014 AUDIT Q20 95 F
4 B
S C
7
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 a Rev 3 96 F
3 N
E D
1 Spelling errors (surveillance vice surviellance).
97 F
4 B
S C
2 98 F
3 M
E B
4 2014 AUDIT Q22 Spelling error in stem "restarted" not restated" 99 F
2 B
S C
1 2017 NRC Q98 100 F
3 B
S A
1 2014 SRO RETAKE NRC Q24 Results Table RO LOK -H 44 58.67 Avg RO LOD 2.51 Flaws 10 CFR Distribution RO LOK-F 31 AVG SRO LOD 3.16 Stem focus 0
41(b)1 1
43(b)1 5 SRO LOK - H 14 Overall LOD 2.67 Cues 1
41(b)2 0
43(b)2 7 SRO LOK - F 11 T/F 0
41(b)3 0
43(b)3 0 Cred Dist 5
41(b)4 3
43(b)4 1 RO Bank 32 42.67 SRO Bank 11 44 Partial 0
41(b)5 8
43(b)5 10 RO Mod 9
12 SRO Mod 2
8 job link 0
41(b)6 0
43(b)6 0 RO New 34 45.33 SRO New 12 48 units 0
41(b)7 25 43(b)7 2 minutia 0
41(b)8 4
Total Bank 43 43 backward 0
41(b)9 0
Total Mod 11 11 KA 0
41(b)10 29 Total New 46 46 SRO-only 0
41(b)11 2
LOD = 1 0
41(b)12 1
RO Sat 63 84 SRO Sat 20 80 41(b)13 0
RO Unsat 1
1.333 SRO Unsat 0
0 Answer Dist ( in %)
41(b)14 2
20 RO-A 20 SRO-A 6
RO-B 23 SRO-B 6
Total Sat 83 83 Total Unsat 1
1 RO-C 15 SRO-C 6
Total Edit 16 16 RO-D 17 SRO-D 7
Question Overlap with the Previous Two NRC Exams QA form ES-401-6 item 4 requires 4 or less questions on RO exam from previous two NRC exams, SRO portion two or less questions from previous two NRC exams*
- if this is exceeded, the region shall call NRR to explain why/reasons.
RO overlap from previous two NRC exams Q34 Q52 ON 2017 NRC EXAM Q36 Q36 ON 2016 NRC EXAM Q41 Q41 ON 2016 NRC EXAM Q72 Q72 ON 2017 NRC EXAM SRO overlap from the previous two NRC exams Q81 Q81 ON 2016 NRC Q99 Q98 ON 2017 NRC Note: the previous two exams considered for overlap are the 2017-09 and 2016-05 NRC Exams
Questions Requiring References to Answer 6-10 questions rule of thumb on SRO portion of exam are allowed to be open reference, while the RO exam is generally none RO Questions that require hand-outs SRO Questions that require hand-outs Q84 EIP-ZZ-00101, ADD 1 (EAL CLASS MATRIX)
Q90 TS LCO 3.4.15 AND TS LCO 3.6.6 Q91 CURVE BOOK, FIGURE 8-7, RCS LOOP WITH CONTROL RODS AND EXCORE LOCATIONS
Note: This is a header page to separate the draft and final resolved versions of the ES-401-9 (Draft Written Exam Comments Form) and the ES-301-7 (Draft Operating test Comments Form) for the Adams package. The first file for both the draft written and draft operating test forms contains only the draft comments with stats for the as submitted draft materials. This header page separates the draft comments from the Final comment files that contains final resolutions of those items for both of these forms.
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9
- 1. LOK 2. LOD 6
7 Ans Letter CFR 8
Q#
(F/H)
(1-5) stem focus cueing T/F cred dist partial job link minutia
- /
units back-ward K/A SRO-only B/M/N U/E/S A/B/C/
D 55.41 55.43 Explanation Callaway noted that they currently do not use the previous two NRC exams (that are withheld from the public for two years) as audit exams and they indicate that on the QA forms by building a completely new audit exam for the class each time.
1 F
2 N
S C
7 2
H 2
B S
A 8
2014 NRC Q45 - Edit - Needs LB LOCA in stem of question to ensure clarity.
Licensee made the requested change and question is now Sat 3
H 3
N S
C 1
4 H
2 N
S C
10 Edit - Put 564F next to "Vessel Head" in explanation for C to ensure tie to given stem value as correct answer.
Licensee made the requested change and question is now Sat 5
H 2
M S
A 10 6
H 2
N S
C 10 7
H 2
M S
D 10 8
H 2
B S
C 10 9
F 2
N S
C 7
10 F
2 B
S A
10 11 F
2 B
S A
10 Edit - Distracter D has grammer error. Should be rewritten to state that " To prevent overfilling the faulted steam generator and cause its atmospheric dump valve to lift." Licensee made the requested change and question is now Sat
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other a Rev 3
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 12 H
2 N
S A
5 Edit - Remove "a" before RCS subcooling in part 2 of stem.
Licensee made the requested change and question is now Sat 13 H
3 M
S B
7 14 H
3 B
S B
7 2014 NRC Q14 15 F
3 N
S D
7 16 H
2 M
S D
5 17 F
2 B
S B
7 2009 NRC 18 H
2 N
S A
8 Unsat - Per the explanation it appears that A is the correct answer not B. You moved the correct answer to A but did not update the explanation after the free review on this question.
Licensee made the requested repairs and question is now Sat 19 H
3 B
S C
5 20 H
3 B
S C
7 21 H
3 B
S B
8 2012 POINT BEACH NRC 22 F
2 N
S C
11 23 H
3 N
S C
10 24 H
2 N
S C
10 25 H
4 N
S B
10 26 H
4 M
S D
10 27 F
3 N
S B
10 Edit - I don't see the plausibility of distracter A because you are high in the band and uncovering heaters would be low in the band, unless you put in some info about the concern for the void to move into the pressurizer.
Licensee created new distracter A, made B the correct answer and it is now Sat. a Rev 3
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 28 H
3 N
S A
10 Distracter D is not credible as the stem is setup. You could use what you had in the first version of this question with "Rx power drops to less than 5% with a negative SUR."
Licensee changed the question and it is now Sat.
29 H
2 N
S B
5 30 F
2 N
S B
10 31 H
2 M
S A
5 32 H
2 B
S B
14 ST NRC 2017 33 H
4 N
S A
10 34 H
2 B
S A
7 2017 NRC Q52 35 F
3 N
S D
7 36 F
3 B
S A
7 2016 NRC Q36 37 H
3 N
S D
7 38 F
2 N
S C
7 39 H
3 N
S B
7 40 F
2 B
S B
7 41 F
2 B
S B
7 2016 NRC Q41 42 F
2 N
S D
8 43 F
3 B
S A
14 44 F
3 B
S A
7 2016 AUDIT Q44 45 H
3 B
S A
10 2009 NRC 46 F
2 N
S A
10 47 F
2 N
S D
4 48 H
3 B
S B
5 49 H
3 B
S B
7 2014 NRC Q49 50 H
3 B
S B
7 2014 NRC Q8 51 H
3 B
S A
10 a Rev 3
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 52 H
3 N
S C
7 Edit required - B2 and D2 are not credible, so these two distracters are partially not credible. Need to change this second part to something more credible such as basis for room temp, the valves that stroke to isolate SW and ESW, or etc.
Licensee made requested changes and the question is now Sat.
53 H
3 N
S B
4 54 F
3 N
S A
7 Edit - 2011 NRC There is no design limit challenge in the question or answer and there is cueing between the stem in two places and the answer. Distracter A and B are not credible because there is nothing else in the stem about them. Since this was previously approved it is an Edit but it needs major work to be used on this exam. Licensee rewrote the question (now new) and it is now Sat.
55 F
3 N
S A
7 56 H
3 M
S D
10 57 H
2 B
S D
10 typo of "gharging" in explanation section. Licensee corrected.
58 H
3 N
S C
7 59 H
3 N
S B
5 60 F
3 B
S B
5 2014 NRC Q58 Might want to put in explanation that 4% is the LEL to ensure tie to the KA is seen.
61 H
2 N
S A
4 62 H
2 N
S A
7 63 H
2 B
S B
7 2017 AUDIT Q62 a Rev 3
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 64 H
2 M
S B
10 65 F
2 N
S D
7 66 F
2 B
S A
10 67 F
2 N
S B
10 68 F
3 M
S A
10 69 F
2 B
S D
10 2016 AUDIT Q68 70 F
2 B
S D
10 2014 NRC Q70 71 F
3 B
S C
11 2015 AUDIT Q71 72 H
2 B
S B
12 2017 NRC Q72 73 H
2 N
S B
10 74 F
2 B
S D
10 2016 AUDIT Q74 75 F
3 B
S D
10 76 H
3 N
S D
5 77 H
3 N
S D
5 78 H
3 M
S C
5 2013 NRC Q78 parent 79 F
4 N
S D
2 80 H
3 N
S D
5 81 H
4 B
S B
5 2016 NRC Q81 Edit required-Missed on 2016 NRC review but question has EOP Add 30 in all four choices so should move this to the stem per NUREG, and then this eliminates the subset isue with Distracter D. Use Add 31 as a swap for the loss of Add 30 to the stem as we discussed.
Licensee made requested repairs and question is now Sat.
82 H
3 N
S A
5 83 H
2 B
S D
5 84 H
3 N
S C
5 HANDOUT REFERENCE 85 H
3 N
S B
5 a Rev 3
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 86 F
3 N
S B
2 C distracter not credible. Modify stem by removing at least one from it and make distracter C "Cool dowen the RCS from 350F to 212F in 2.8 hrs." This is the FSAR design basis for two full trains of RHR not one and so it would not be correct nor even close for anyone who has performed or observed a cooldown for shutdown so it has operational validaty as well. Also, fix grammer in Distracter D to "iodine release rates due to a postulated fuel handling....".
Licensee made requested repairs but still has grammar error in stem (release is singular so use "is" not "are"....within limits). Licensee made grammar repair and question is now Sat.
87 H
3 N
S A
2 88 H
3 B
S C
1 2013 NRC Q86 89 F
4 B
S B
2 2014 NRC Q88 90 H
3 B
S A
2 HANDOUT REFERENCE 91 H
4 B
S B
5 HANDOUT REFERENCE 2014 SRO RETAKE NRC 92 F
3 N
S A
7 Edit - partial non-credible distracters. By knowing the order you can eliminate the second half of the question choices or by knowing the reason you can eliminate the first part of the choices. Use balanced 2 X 2 as dicussed on phone.
Licensee made requested repairs and question is now Sat.
93 H
3 N
S A
2 94 F
3 B
S B
1 2014 AUDIT Q20 a Rev 3
ES-401 CW-2019-03 Written Examination Review Worksheet Form ES-401-9 95 F
4 B
S C
7 96 F
3 N
S D
1 Spelling errors (surveillance vice surviellance). Licensee made repair and it is now Sat.
97 F
4 B
S C
2 98 F
3 M
S B
4 2014 AUDIT Q22 Edit required Spelling error in stem "restarted" not restated" Licensee made repair and it is now Sat.
99 F
2 B
S C
1 2017 NRC Q98 100 F
3 B
S A
1 2014 SRO RETAKE NRC Q24 Results Table RO LOK -H 44 58.67 Avg RO LOD 2.51 Flaws 10 CFR Distribution RO LOK-F 31 AVG SRO LOD 3.16 Stem focus 0
41(b)1 1
43(b)1 5 SRO LOK - H 14 Overall LOD 2.67 Cues 0
41(b)2 0
43(b)2 7 SRO LOK - F 11 T/F 0
41(b)3 0
43(b)3 0 Cred Dist 0
41(b)4 3
43(b)4 1 RO Bank 31 41.33 SRO Bank 11 44 Partial 0
41(b)5 8
43(b)5 10 RO Mod 9
12 SRO Mod 2
8 job link 0
41(b)6 0
43(b)6 0 RO New 35 46.67 SRO New 12 48 units 0
41(b)7 25 43(b)7 2 minutia 0
41(b)8 4
Total Bank 42 42 backward 0
41(b)9 0
Total Mod 11 11 KA 0
41(b)10 29 Total New 47 47 SRO-only 0
41(b)11 2
LOD = 1 0
41(b)12 1
RO Sat 75 100 SRO Sat 25 100 41(b)13 0
RO Unsat 0
0 SRO Unsat 0
0 Answer Dist ( in %)
41(b)14 2
RO Edit 0
0 SRO Edit 0
0 RO-A 22 SRO-A 6
RO-B 23 SRO-B 7
Total Sat 100 100 Total Unsat 0
0 RO-C 15 SRO-C 6
Total Edit 0
0 RO-D 15 SRO-D 6 a Rev 3
Question Overlap with the Previous Two NRC Exams QA form ES-401-6 item 4 requires 4 or less questions on RO exam from previous two NRC exams, SRO portion two or less questions from previous two NRC exams*
- if this is exceeded, the region shall call NRR to explain why/reasons.
RO overlap from previous two NRC exams Q34 Q52 ON 2017 NRC EXAM Q36 Q36 ON 2016 NRC EXAM Q41 Q41 ON 2016 NRC EXAM Q72 Q72 ON 2017 NRC EXAM SRO overlap from the previous two NRC exams Q81 Q81 ON 2016 NRC Q99 Q98 ON 2017 NRC Note: the previous two exams considered for overlap are the 2017-09 and 2016-05 NRC Exams
Questions Requiring References to Answer 6-10 questions rule of thumb on SRO portion of exam are allowed to be open reference, while the RO exam is generally none RO Questions that require hand-outs SRO Questions that require hand-outs Q84 EIP-ZZ-00101, ADD 1 (EAL CLASS MATRIX)
Q90 TS LCO 3.4.15 AND TS LCO 3.6.6 Q91 CURVE BOOK, FIGURE 8-7, RCS LOOP WITH CONTROL RODS AND EXCORE LOCATIONS
Stats and Graph sat 100 edit 0
unsat 0
Stem focus 0
Cues 0
T/F 0
Cred Dist 0
Partial 0
job link 0
units 0
minutia 0
backward 0
KA 0
SRO-only 0
LOD = 1 0
0 0
0 0
0 unsat Stem focus Cues T/F Cred Dist CW 2019-03 Initial Writ
0 0
0 0
t Partial job link units minutia tten Exam Statistics