ML20301A386
| ML20301A386 | |
| Person / Time | |
|---|---|
| Site: | South Texas  |
| Issue date: | 07/29/2020 |
| From: | Greg Werner Operations Branch IV |
| To: | South Texas |
| References | |
| Download: ML20301A386 (19) | |
Text
The first set of data is for the draft exam review and the second set of data with all Sat questions is the final product as given. The exception is the three post exam comments which changed Q28, Q73, and Q80 to unsat on the draft submittal data, but this was not enough to make the submittal unsat overall.
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9
- 1. LOK 2. LOD 6
7 Ans Letter CFR 8
Q#
(F/H)
(1-5) stem focus cueing T/F cred dist partial job link minutia
- /
units back-ward K/A SRO-only B/M/N U/E/S A/B/C/
D 55.41 55.43 Explanation 1
F 2
N S
B 10 2
F 3
N S
D 13 3
F 2
B S
D 10 2018 NRC 4
F 3
N S
D 10 5
H 3
N S
A 5
6 F
3 N
S A
4 7
F 3
N S
D 7
8 H
3 M
S C
5 9
H 3
N S
D 4
10 H
3 N
S B
5 11 H
3 N
S D
7 12 F
3 N
S B
7 13 H
4 N
S A
7 14 F
1 N
U B
5 LOD = 1, too easy, Not a higher order question as written. I think I gave you suggestions for this on the free review too. We can discuss when you get the comments back.
15 H
3 N
S C
10 16 H
3 N
E D
4 Logic in worksheet seems to be incorrect (possible versus impossible). Also, how to we ensure that this question is not challengeable with local operations of these valves? I have seen crews send a local AO to do it locally and not even try to operate from CR. Does the procedure empasize the CR manipulations?
17 F
3 N
S B
4 18 F
2 N
S A
7
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 19 H
3 N
E D
7 So is the battery inop or is it the switchboard? Affects the correct answer at least in the explanation.
It is more operationally valid to just give the breaker the tripped, not tell them that this is the input breaker to the A battery. Change the breaker to the breaker name (numerical and noun name) in the stem.
20 F
1 M
U D
6 KA-this doesn't address both purpose and function. This is also LOD = 1 because you only have the major breaker S with its logic S so it is really cueing from within the answer selections. Also, with logic R (not going to trip reactor trip breaker S) makes it easy to throw out Answers A and B, so only leaves you with only 2 choices = 50/50 chance. Two bad distracters makes the Q unsat.
21 H
3 N
S B
7 22 H
3 N
E D
7 For the answer justification, isn't it to ensure adequate CCW flow to the RHR system (not CCW as you have)?
23 H
3 N
S B
7 24 F
3 N
S C
6 25 H
3 B
S C
5 2017 NRC REFERENCE NEEDED 26 F
3 B
S B
7 27 H
3 B
S C
4 2015 NRC a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 28 F
3 N
U A
11 The question was good for the free review but changes you made after free review put it into a Tier 2 system level question which is not allowed for Tier 3 questions. If you look at the free review question it was fine. This means taking out the specifics and just state that a rad panel has status colors and ask what the light blue color means with the picture below as an example as you had it before. I know it is symantics but we don't want to get hit for a Tier 3 being a system level question. The final question was a post exam challenge question with no correct answer so it is marked as UNSAT.
29 H
4 B
S A
5 2016 NRC 30 H
3 B
S B
5 2007 NRC 31 H
3 B
S C
7 32 F
3 B
S C
7 2011 NRC 33 H
3 B
S A
7 34 H
3 B
S C
4 2010 NRC 35 H
3 B
S D
14 36 H
3 B
S C
5 1999 NRC 37 H
3 N
E A
10 Plausibility statements need work-RWST is plausible because it is a source of borated water but it won't work because..
38 H
3 N
S B
3 39 F
3 N
S B
10 40 F
3 N
S A
10 41 F
3 N
S C
10 42 H
3 N
S B
10 43 F
3 N
S C
7 44 H
3 B
S B
7 45 F
3 B
S D
3 2013 NRC a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 46 H
3 B
S A
7 2007 NRC 47 F
3 B
S B
7 2015 NRC 48 H
3 B
E B
7 2018 NRC Distracter D has a typo in it in the part 2 information.
49 F
3 B
S A
8 50 F
3 B
S B
10 2016 NRC 51 F
3 N
S D
10 52 F
3 N
S A
10 53 H
3 N
S C
10 54 H
3 N
S A
10 55 H
3 N
S A
10 56 F
3 N
S B
10 57 F
3 B
S C
10 58 H
2 B
S C
7 59 H
3 M
S B
10 60 F
3 N
S D
10 61 F
3 N
S D
8 62 H
3 N
S D
3 63 H
3 N
S A
7 64 F
3 N
S A
7 65 H
3 N
S D
12 REFERENCE NEEDED 66 F
2 B
S A
10 2018 NRC 67 H
3 B
S B
12 2007 NRC 68 F
2 B
E C
2017 NRC No CFR link assigned 69 H
3 N
S C
7 70 H
3 N
S C
10 71 F
2 B
S D
4 2016 NRC 72 H
2 B
S C
7 73 H
3 B
U A
7 The final question was a post exam challenge question with two correct answers so it is marked as UNSAT. a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 74 F
3 N
E A
4 The stem needs to have 1RCP running and the design basis words in it (such as USAR states that). The 25F/hr or 50F/hr depends on if one RCP is running or not and the stem has neither of those. The flow is 500 gpm and that is the main part of the KA so that part is missing in the question (the 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> to HSB is the design basis of the tank).
75 F
3 N
E D
5 The hydrgen part is fine but we need to have some design feature of the system in the question, such as the regulator maintains pressure at greater than 15 psig overpressure. Remove the manual or auto part of the question and use the regulator aspect.
76 F
3 N
S C
5 77 H
3 N
E A
5 REFERENCE NEEDED I am not sure why the drawing is needed to answer the question. It seems that systems knowledge and time should be able to get the first part of the question (which is RO knowledge anyway).
78 H
3 N
S B
5 79 F
3 B
E A
5 2017 NRC Need to put a CFR link in the question (55.43(b)5 is a match for E-plan procedure implementation). a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 80 F
3 N
U A
3 The NUREG discourages use of stems and answers with the word "may" in it because they also "may not" conclude the same thing. We need to shore up that part somehow so the question is not challengeable. Didn't catch this during free review but should be able to edit the question to fix this aspect. The final question was a post exam challenge question with two correct answers so it is marked as UNSAT.
81 F
3 N
S D
4 82 H
3 N
E A
1 The restricted/unrestriced part of the question is direct lookup and does not discriminate for the SRO level license.
83 H
4 N
E C
5 REFERENCE NEEDED To prevent a challenge on the time aspect, make the stem have over 60 minutes in it above 200F so there is no debate with the answer. This happened on an EAL call question at another site. Use 65 minutes.
84 H
3 B
S A
6 2015 NRC 85 H
3 N
S D
5 86 H
3 B
E A
5 2019 NRC Stem has grammar issue. Better grammar would be "_____1)_____
than if the incident had occurred during Core Reload."
87 H
4 N
E C
4 Plausibility of stopping only 1 pump is not there. Why is only stopping 1 pump plausible with the given conditions? This affects two distracters. a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 88 H
3 N
U D
5 KA mismatch-this is a Tier 2 question asking how a rapid depress affects the ESFAS system. The stem should be in abnormal type conditions and a rapid depress is needed and while going down in pressure the impacts and/or challenges on the ESFAS system as pressure goes down quick. This is not in EOPs for uncontrolled depress of all SGs (This would be a Tier 1 question as you have written it).
89 H
3 M
S B
5 90 H
4 N
S B
2 91 F
3 N
S B
2 92 H
3 N
S A
2 93 H
3 N
E C
2 Please put the CFR Reference in the worksheet ( 55.43(b)2 ).
94 F
3 N
S C
7 95 H
3 B
S D
2 96 H
3 N
S D
5 97 F
3 N
S D
2 98 H
3 N
S B
5 99 F
3 N
U C
2 You had a good 2015 bank question that was free reviewed as good and you changed it. This new question does not answer the KA of basis for LCOs for a SGTR.
The 1gpm discussion in the bases from all SGs is a good number to know, as is the dose bases from a fuel failure and without a fuel failure tied to SGTR for this KA and is discussed in detail in the bases. a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 100 F
3 N
E B
2 Minor KA mismatch (stem edits could potentially fix it). The question is not tied to a pre or post maintenance operability requirement. Discussed several examples of these during free review comments. This would require maintance if in the action range (but this info is not in the stem). You could claim that it is pre-maintenance activity because that is why you are doing IST test to determine if you need to do maintenance. With a simple change to the stem, could be related to maintenance, but it does focus on operability. Also, a minor issue but you always need to have the CFR link in the worksheet (55.43(b)2 in this case).
Results Table RO LOK -H 40 53.33 Avg RO LOD 2.87 Flaws 10 CFR Distribution RO LOK-F 35 AVG SRO LOD 3.12 Stem focus 0
41(b)1 0
43(b)1 1 SRO LOK - H 16 Overall LOD 2.93 Cues 0
41(b)2 0
43(b)2 8 SRO LOK - F 9
T/F 0
41(b)3 3
43(b)3 1 Cred Dist 0
41(b)4 8
43(b)4 2 RO Bank 27 36 SRO Bank 4
16 Partial 0
41(b)5 9
43(b)5 11 RO Mod 3
4 SRO Mod 1
4 job link 0
41(b)6 2
43(b)6 1 RO New 45 60 SRO New 20 80 units 0
41(b)7 24 43(b)7 1 minutia 0
41(b)8 2
Total Bank 31 31 backward 0
41(b)9 0
Total Mod 4
4 KA 0
41(b)10 21 Total New 65 65 SRO-only 0
41(b)11 1
LOD = 1 2
41(b)12 2
RO Sat 63 84 SRO Sat 14 56 41(b)13 1
RO Unsat 4
5.333 SRO Unsat 3
12 Answer Dist ( in %)
41(b)14 1 a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 RO Edit 8
10.67 SRO Edit 8
32 RO-A 19 SRO-A 7
RO-B 19 SRO-B 6
Total Sat 77 77 Total Unsat 7
7 RO-C 18 SRO-C 6
Total Edit 16 16 RO-D 19 SRO-D 6 a Rev 3
Question Overlap with the Previous Two NRC Exams QA form ES-401-6 item 4 requires 4 or less questions on RO exam from previous two NRC exams, SRO portion two or less questions from previous two NRC exams*
- if this is exceeded, the region shall call NRR to explain why/reasons.
RO overlap from previous two NRC exams (2018 AND 2019 EXAMS)
Q3 2018-10 EXAM Q48 2018-10 EXAM Q66 2018-10 EXAM SRO overlap from the previous two NRC exams Q86 2019-07 EXAM Note: the previous two exams considered for overlap are the 2018-10 and 2019-07 NRC Exams
Questions Requiring References to Answer 6-10 questions rule of thumb on SRO portion of exam are allowed to be open reference, while the RO exam is generally none RO Questions that require hand-outs Q25 Generator Cap Curve Q65 RWP Plus Survey map SRO Questions that require hand-outs Q77 Portion of RCS Pressurizer P and ID Q83 Cold S/D EAL table
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9
- 1. LOK 2. LOD 6
7 Ans Letter CFR 8
Q#
(F/H)
(1-5) stem focus cueing T/F cred dist partial job link minutia
- /
units back-ward K/A SRO-only B/M/N U/E/S A/B/C/
D 55.41 55.43 Explanation 1
F 2
N S
B 10 2
F 3
N S
D 13 3
F 2
B S
D 10 2018 NRC 4
F 3
N S
D 10 5
H 3
N S
A 5
6 F
3 N
S A
4 7
F 3
N S
D 7
8 H
3 M
S C
5 9
H 3
N S
D 4
10 H
3 N
S B
5 11 H
3 N
S D
7 12 F
3 N
S B
7 13 H
4 N
S A
7 14 F
3 N
S B
5 15 H
3 N
S C
10 16 H
3 N
S D
4 17 F
3 N
S B
4 18 F
2 N
S A
7 19 H
3 N
S D
7 20 F
3 M
S D
6 21 H
3 N
S B
7 22 H
3 N
S D
7 23 H
3 N
S B
7 24 F
3 N
S C
6
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 25 H
3 B
S C
5 2017 NRC REFERENCE NEEDED 26 F
3 B
S B
7 27 H
3 B
S C
4 2015 NRC 28 F
3 N
S A
11 29 H
4 B
S A
5 2016 NRC 30 H
3 B
S B
5 2007 NRC 31 H
3 B
S C
7 32 F
3 B
S C
7 2011 NRC 33 H
3 B
S A
7 34 H
3 B
S C
4 2010 NRC 35 H
3 B
S D
14 36 H
3 B
S C
5 1999 NRC 37 H
3 N
S A
10 38 H
3 N
S B
3 39 F
3 N
S B
10 40 F
3 N
S A
10 41 F
3 N
S C
10 42 H
3 N
S B
10 43 F
3 N
S C
7 44 H
3 B
S B
7 45 F
3 B
S D
3 2013 NRC 46 H
3 B
S A
7 2007 NRC 47 F
3 B
S B
7 2015 NRC 48 H
3 B
S B
7 2018 NRC 49 F
3 B
S A
8 50 F
3 B
S B
10 2016 NRC 51 F
3 N
S D
10 52 F
3 N
S A
10 53 H
3 N
S C
10 54 H
3 N
S A
10 55 H
3 N
S A
10 56 F
3 N
S B
10 57 F
3 B
S C
10 58 H
2 B
S C
7 59 H
3 M
S B
10 a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 60 F
3 N
S D
10 61 F
3 N
S D
8 62 H
3 N
S D
3 63 H
3 N
S A
7 64 F
3 N
S A
7 65 H
3 N
S D
12 REFERENCE NEEDED 66 F
2 B
S A
10 2018 NRC 67 H
3 B
S B
12 2007 NRC 68 F
2 B
S C
2017 NRC 69 H
3 N
S C
7 70 H
3 N
S C
10 71 F
2 B
S D
4 2016 NRC 72 H
2 B
S C
7 73 H
3 B
S A
7 74 F
3 N
S A
4 75 F
3 N
S D
5 76 F
3 N
S C
5 77 H
3 N
S A
5 REFERENCE NEEDED 78 H
3 N
S B
5 79 F
3 B
S A
5 2017 NRC 80 F
3 N
S A
3 81 F
3 N
S D
4 82 H
3 N
S A
1 a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 83 H
4 N
S C
5 REFERENCE NEEDED 84 H
3 B
S A
6 2015 NRC 85 H
3 N
S D
5 86 H
3 B
S A
5 2019 NRC 87 H
4 N
S C
4 88 H
3 N
S D
5 89 H
3 M
S B
5 90 H
4 N
S B
2 91 F
3 N
S B
2 92 H
3 N
S A
2 93 H
3 N
S C
2 94 F
3 N
S C
7 95 H
3 B
S D
2 96 H
3 N
S D
5 97 F
3 N
S D
2 98 H
3 N
S B
5 99 F
3 N
S C
2 100 F
3 N
S B
2 Results Table RO LOK -H 40 53.33 Avg RO LOD 2.92 Flaws 10 CFR Distribution RO LOK-F 35 AVG SRO LOD 3.12 Stem focus 0
41(b)1 0
43(b)1 1 SRO LOK - H 16 Overall LOD 2.97 Cues 0
41(b)2 0
43(b)2 8 SRO LOK - F 9
T/F 0
41(b)3 3
43(b)3 1 Cred Dist 0
41(b)4 8
43(b)4 2 a Rev 3
ES-401 ST-2020-07 Written Examination Review Worksheet Form ES-401-9 RO Bank 27 36 SRO Bank 4
16 Partial 0
41(b)5 9
43(b)5 11 RO Mod 3
4 SRO Mod 1
4 job link 0
41(b)6 2
43(b)6 1 RO New 45 60 SRO New 20 80 units 0
41(b)7 24 43(b)7 1 minutia 0
41(b)8 2
Total Bank 31 31 backward 0
41(b)9 0
Total Mod 4
4 KA 0
41(b)10 21 Total New 65 65 SRO-only 0
41(b)11 1
LOD = 1 0
41(b)12 2
RO Sat 75 100 SRO Sat 25 100 41(b)13 1
RO Unsat 0
0 SRO Unsat 0
0 Answer Dist ( in %)
41(b)14 1
RO Edit 0
0 SRO Edit 0
0 RO-A 19 SRO-A 7
RO-B 19 SRO-B 6
Total Sat 100 100 Total Unsat 0
0 RO-C 18 SRO-C 6
Total Edit 0
0 RO-D 19 SRO-D 6 a Rev 3
Question Overlap with the Previous Two NRC Exams QA form ES-401-6 item 4 requires 4 or less questions on RO exam from previous two NRC exams, SRO portion two or less questions from previous two NRC exams*
- if this is exceeded, the region shall call NRR to explain why/reasons.
RO overlap from previous two NRC exams (2018 AND 2019 EXAMS)
Q3 2018-10 EXAM Q48 2018-10 EXAM Q66 2018-10 EXAM SRO overlap from the previous two NRC exams Q86 2019-07 EXAM Note: the previous two exams considered for overlap are the 2018-10 and 2019-07 NRC Exams
Questions Requiring References to Answer 6-10 questions rule of thumb on SRO portion of exam are allowed to be open reference, while the RO exam is generally none RO Questions that require hand-outs Q25 Generator Cap Curve Q65 RWP Plus Survey map SRO Questions that require hand-outs Q77 Portion of RCS Pressurizer P and ID Q83 Cold S/D EAL table