ML18324A311
| ML18324A311 | |
| Person / Time | |
|---|---|
| Site: | South Texas  |
| Issue date: | 11/07/2018 |
| From: | Vincent Gaddy Operations Branch IV |
| To: | South Texas |
| References | |
| 50-498/18-10, 50-499/18-10 50-498/OL-18, 50-499/OL-18 | |
| Download: ML18324A311 (12) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 1
F 2
N E
S This is listed as a new question, but it has a last used on an NRC exam date of 2011. The question from 2011 matching this exam bank number is not the same. Changed the last used on NRC Exam date.
This is a new question. Question now SAT.
2 F
3 M
E S
Distractor D not credible, as it does not seem credible that a turbine building sump would recirc. Changed distractor D to Condensate Polishing Regenerative Waste Collection Tank (CPRWCT)". Question now SAT.
3 F
2 B
S 4
F 3
B E
S Reorder distractors D, C, A, B. Reordered distractors. Question now SAT.
5 H
3 B
E S
2016 Retake Bank (last 2 exams)
Reorder distractors C, B, D, A. Reordered distractors. Question now SAT.
6 F
3 N
E S
Reorder distractors D, C, B, A. Reordered distractors. Question now SAT.
7 F
3 N
S Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 8
H 3
X N
U S
One could argue multiple correct answers, since the RNO states to "continue actions at Step 4.0 of this procedure as resources permit," answer B would also be correct. One could also argue that B is the only correct answer, since the stem states the crew is attempting to restore a train, you would be making assumptions not stated in the stem to assume a train can't be placed in service, taking you down the RNO path. Changed part 1 of distractors A and B to "Go to 0POP04-CC-0001, CCW Leak. Question now SAT.
9 F
3 N
S 10 F
3 N
E S
What is the benefit of the part 2 distractors having "ICS" and "CP-4?" Removed reference to where the flow rate is read. Question now SAT.
11 H
3 B
E S
Reorder distractors D, B, C, A. Reordered distractors. Question now SAT.
12 F
3 B
E S
Distractor B not credible as aligning charcoal beds is not terminating, as the stem asks. What was wrong with the initial distractor from the 2011 question?
Reorder distractors B, A, D, C. Changed distractor B to MAB Exhaust Fan Trips, then reordered distractors. Question now SAT.
13 H
3 X
B U
S How is it credible to stop all core cooling (securing RHR pumps)? It seems to me that starting another pump would be performed prior to securing your one means of core cooling. It does not make sense to lower flow, then start another train of CCW, making distractor C not credible. Changed part 1 of distractors C and D to raise CCW flow. Question now SAT.
14 F
3 2
N E
S Closer to LOD 2. Reorder distractors B, A, D, C.
Changed LOD to 2. Reordered distractors. Question now SAT.
15 F
3 2
B S
Closer to LOD 2. Changed LOD to 2.
16 H
3 B
S 17 F
3 B
S 18 H
3 N
S
ES-401 3
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 19 H
3 B
E S
Distractor D does not seem credible as if one of three parallel valves is closed and the remaining two valve are opened how would anyone expect flow to remain the same. If you stated the remaining two valves were throttled closed to keep the same flow rate it might be more credible. Reorder distractors A, B, D, C. Changed distractor D from stays the same to rise, then reordered distractors. Question now SAT.
20 H
3 B
E S
Reorder distractors C, D, A, B. Reordered distractors. Question now SAT.
21 H
3 M
B E
S Though the question was modified from Bank 1872 to be a CVCS relief, I think this is still a bank question.
The new stem does not change least one pertinent condition (the only real pertinent part of the stem is that the rupture disc lifts), and the answer is essentially the same. Otherwise, the question is satisfactory. Reorder distractors D, B, C, A.
Changed to a Bank Question. Reordered distractors.
Question now SAT.
22 F
3 B
S 23 H
3 X
B U
S Part 2 distractors B and D are not credible. It does not seem credible to think one would lower speed to raise frequency. Changed in the stem from stating the bus frequency is 60.2 Hz, to the synchroscope is moving slowly in the slow direction. Question now SAT.
24 F
3 B
S 25 H
3 M
S 26 H
3 N
S 27 H
3 N
S 28 H
3 B
E S
Reorder distractors C, B, A, D. Reordered distractors. Question now SAT.
29 F
3 B
E S
The first part of distractor A and B would be more consistent if it was LT-113 ONLY and the second half of distractor A and C would be better if it was LT-112 AND LT-113. this would provide better symmetry between choices and not change what is being tested. Changed first part of distractors A and B to
ES-401 4
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only LT-113 Only, and the second part of distractors A and C to LT-112 AND LT-113. Question now SAT.
30 H
3 B
S 31 H
3 B
S 2017 Bank (last 2 exams) 32 H
3 N
S 33 F
3 N
E S
Reorder distractors C, D, A, B. Reordered distractors. Question now SAT.
34 F
3 N
E S
Reorder distractors A, C, B, D. Reordered distractors. Question now SAT.
35 F
3 N
E S
Reorder distractors D, C, B, A. Reordered distractors. Question now SAT.
36 F
3 X
N U
S Part 2 distractors A and C are not credible, as it is not credible to think the accumulator isolation valves would not already be open at 100% power. Maybe change this to "receive an open signal to back up the expected position" and "do not receive an open signal as they are already open with power off."
Reorder distractors C, D, A, B. Changed part one distractors to receive an open signal and not receive an open signal because they are already opened, then reordered distractors. Question now SAT.
37 H
3 M
S 38 F
3 N
S 39 H
3 X
N B
U S
The question itself is ok, but does not meet the K/A of interpreting reference materials. Reorder distractors B, D, A, C. The Chief Examiner randomly selected a new generic K/A, and Bank Question was submitted.
Question now SAT.
40 H
3 N
E S
Reorder distractors C, D, B, A. Reordered distractors. Question now SAT.
41 H
3 N
E S
Picture is blurry; can we get a better picture? It appears to me knowing whether two offsite sources are inoperable is SRO level of knowledge. Providing a full page picture. Also, after questioning by the
ES-401 5
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only chief examiner, the licensee ensures that this is RO level of knowledge. Question now SAT.
42 F
3 N
E S
Change Distractor A to immediately and distractor B to within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />. This will ensure that the applicant does not throw out distractor B as a greater than one hour tech spec application. This lends more credibility to the distractors. Changed Distractor A to immediately and distractor B to within 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />.
Question now SAT.
43 H
3 X
N U
S Could argue reducing turbine load as a possible answer, since 0POP04-CR-0001 states, In some cases a load reduction per Addendum 2 may restore vacuum above the alarm level but care must be taken not to approach or enter the Restrictive Zone.
Would it be more correct to add the loss of condenser vacuum procedure in the stem so that the top priority of the procedure is to verify main turbine operation?
Added "In accordance with 0POP04-CR-0001, Loss of Condenser Vacuum" to the stem. Question now SAT.
44 H
3 N
S 45 H
3 N
E S
Since the RPS system has 4 trains, it does not seem credible that the loss of one regardless of power supply would affect its ability to function. Could you instead list one of the reactor trip breakers that is not powered from that particular bus? Reorder distractors D, B, C, A. Changed distractor C to Reactor Trip Breaker S will not open on an automatic signal, then reordered distractors.
Question now SAT.
46 F
3 N
E S
Reorder distractors C, D, A, B. Reordered distractors. Question now SAT.
47 H
3 N
E S
What is the purpose of the picture? By stating in the stem that PT-514 fails low, the picture doesnt seem necessary to answer the question. Removed picture.
Question now SAT.
48 F
3 N
S 49 F
3 M
B E
S This is a systems question and Tier 1 group 1 is procedure abnormal and EOP. An attempt should be made to write to the loss of IA procedure. A loss of instrument Air has occurred and 0POP4-IA-0001 verifies/directs that IA compressor 14 be started from
_____ then leave the second half of the question. I
ES-401 6
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only do not see this as a modified question. I see it as a direct bank. Reorder distractors D, B, C, A.
Changed the question to Bank. Reworded the stem to put the applicant into AOP space. Reordered distractors. Question now SAT.
50 H
3 X
N U
S The K/A is knowledge of low power/shutdown implications, and the question appears to me to be valid even at 100% power, which does not meet the K/A. Edit to include the tier and group (T2G1) on pedigree page. Licensee provided replacement question. Question now SAT.
51 H
3 N
E S
Need to be clear in the plausibility statement. For example if the reason that a CCW leak in the piping for the RCPs is incorrect due to during a design base LOCA the piping is isolated due to a containment isolation signal. Revised distractor A and C plausibility statements to indicate that that RCPs are isolated upon a Containment Isolation Phase B during a LOCA. Question now SAT.
52 H
3 X
N U
S Edit to include the tier and group (T1G2) on pedigree page. Would including the affected SG in the stem work? Is the TDAFWP used in the case if it is SG-1D? If not state that you are in FRH2 for overpressure in SG-1D. This makes it more plausible. If SG-1D would make a different correct answer then as written the question stem is flawed and needs revamped to ensure only one answer is correct. Also have the second half read either reset safety Injection and throttle open Main Steam Isolation Bypass valves, OR reset MSLI and momentarily place Train A and B..... this makes the answer choices more balanced. Reorder distractors C, D, A, B. Included T1G2 on the pedigree page.
Revised stem such that the question is specific to steam generator 1D. Reordered distractors.
Question now SAT.
53 H
3 N
S 54 H
1 3
N U
S There is nothing in the stem to indicate anything less than the normal full RCFCs running. That makes the question a direct lookup. Changed RWST level in the stem to be greater than 75,000 gallons, and changed the distractors so the choices are 0, 1, 2, or 3 containment spray pumps. No longer direct lookup as must ascertain the number of running pumps and RCFCs. Question now SAT.
ES-401 7
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 55 H
4 N
S Edit to include the tier and group (T1G1) on pedigree page. Included T1G1 on the pedigree page.
56 H
3 N
S 57 H
3 N
E S
Without a timeline one could assume that EO00 had been entered when the ESF diesels trip and lead to answer B. Possibly change the stem to ALL ESF Diesel Generators tripped on start. Changed stem to indicate that all ESF Diesel Generators tripped and locked out upon start. Question now SAT.
58 H
3 N
S 59 H
3 N
S 60 F
3 X
N U
S Stretch for LOD 3 more like 1.5. And inventory concern for ECCS pumps during at ATWS? Doesnt make real sense as a credible distractor. Changed part 2 of distractors C and D to ensure less time is required to achieve adequate shutdown margin.
This makes the question LO 3. Question now SAT.
61 H
4 B
S Specific step guidance is usually an SRO level question, unless you are calling this mitigative strategy. After questioning by the chief examiner, the licensee ensures that this is RO level of knowledge.
62 H
3 M
E S
Reorder distractors B, A, C, D. Reordered distractors. Question now SAT.
63 H
3 X
N U
S By the stem directing to reset SI AUTO RECIRC that cues that it has not been reset. If the intent is whether the applicant knows if resetting SI is the same as resetting SI AUTO RECIRC, then the direction does not need to be in the stem. Removed The Unit Supervisor directs the Reactor Operator to perform procedure step 2: RESET SI AUTO RECIRC from the stem. Question now SAT.
64 F
2 B
E S
Reorder distractors D, C, B, A. Reordered distractors. Question now SAT.
65 H
3 N
E S
This question is cued by Question 54, mainly because of the reference attached to Question 54.
Because of that, I am calling this question an Edit only. The reference includes 75,000 gallon but not 32,500 gallons is not one of the limits. Reorder distractors D, C, B, A. Changed the second part of the question to ask about the alarm instead of
ES-401 8
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only specific values, eliminating the cueing. Reordered distractors. Question now SAT.
66 F
3 N
E S
Edit only because I am not sure that knowledge of what indications are available in the TSC is RO level knowledge. Generally RO's do not man the TSC they are normally fulfilling other duties. Additionally, I find it extremely difficult to acknowledge the colors of the lights in the pictures. After questioning by the chief examiner, the licensee ensures that this is RO level of knowledge. Also, providing a full page picture.
Question now SAT.
67 F
2 N
E S
Closer to LOD 2. Reorder distractors B, A, D, C.
Changed to LOD 2 and reordered distractors.
Question now SAT.
68 H
2 N
E S
Closer to LOD 2. Reorder distractors B, A, D, C.
Changed to LOD 2 and reordered distractors.
Question now SAT.
69 F
3 N
E S
Reorder distractors B, D, A, C. Reordered distractors. Question now SAT.
70 F
2 B
E S
This a fundamental level question basically a definition, and LOD 2. Reorder distractors B, D, A, C.
Changed to Fundamental and LOD 2, and reordered distractors. Question now SAT.
71 H
3 N
E S
The procedure selection is normally SRO level.
Selecting the normal operating procedure does not make sense.as a distractor. While it may allow you to lower power it would not address the issue. Is there an alarm that would come in that would not address this specific issue. Reorder distractors B, A, D, C. After questioning by the chief examiner, the licensee ensures that this is RO level of knowledge Reordered distractors. Question now SAT.
72 F
2 B
S 2016 Retake Bank (last 2 exams)
Edit to include the tier and group (T2G1) on pedigree page. Added Tier 2 Group 1 to pedigree page.
73 F
2 B
S 74 F
2 B
S Closer to LOD 2. Changed to LOD 2.
75 F
2 B
S 76 H
3 N
E S
Closer LOD 3 not 4. This is a major transition and a common transition students should be familiar with SI termination and when it is used. Reorder distractors
ES-401 9
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only D, B, C, A. Changed to LOD 3. Reordered distractors. Question now SAT.
77 F
3 B
E S
2017 Bank (last 2 exams)
Reorder distractors D, C, B, A. Reordered distractors. Question now SAT.
78 F
2 B
S 79 H
3 N
S 80 F
3 N
S 81 H
4 X
N U
S Good question bad K/A match. The only RCP malfunction is what you create when you start the pump. There is nothing in the question that deals with RCP malfunctions and EOP strategies. I would consider something along the line of the basis for step in the RCP malfunction AOP or put the situation where a RCP seal failure leads to a reactor trip basis for the trip not just entry condition. (Side note: if you want to keep this question for the future what makes starting three RCP's credible?) Licensee replaced question with a new question, same KA. Question now SAT.
82 H
3 B
E S
Reorder distractors B, D, A, C. Reordered distractors. Question now SAT.
83 F
2 B
E S
2016 Retake Bank (last 2 exams)
Reorder distractors B, D, A, C. Reordered distractors. Question now SAT.
84 H
3 B
E S
Does the 58 inch mark signify anything other than being below the 69 inches for the containment flooding procedure? Change the 2500 to 2150 R as it would be more discriminating. Nothing more significant that being below 69 inches. Changed radiation level in the stem from 2500 to 2150 R/hr.
Question now SAT.
85 H
3 N
E S
Change LT-467 value to 51.1%. This give a bigger gap and helps to ensure that we are evaluating the knowledge of the channel check and tech specs.
Changed LT-467 level to 51.1%. Question now SAT.
86 F
3 N
S 87 H
3 N
S
ES-401 10 Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 88 H
1 3
N U
S This question is UNSAT with the reference, since it is an LOD 1 question. The biggest issue comes with block (b). Since there is no information in the stem regarding impediments, and one can't assume conditions in the question, one can't answer block (b) yes or no. This is a good question without the reference, but I don't think it is realistic to expect SRO to know from memory. The chief examiner randomly selected 2.4.18 as a replacement KA. Licensee replaced question with a new question, same KA.
Question now SAT.
89 H
3 X
N E
S Reorder distractors D, C, B, A. Reordered distractors. Question now SAT.
90 H
3 N
S Closer to LOD 3. Changed to LOD 3 91 H
3 N
E S
Subset issue. If power is below P-6, it is also below 10%. Need to add something like " maintain power below a MAXIMUM of". Changed stem to state, Technical Specifications will require the Unit Supervisor to maintain power below a MAXIMUM of to remove subset issue. Question now SAT.
92 H
3 N
S 93 H
3 M
E S
Distractor B has (1) (3) not (1) (2). Second part of distractors B and D need to be worded better. If you mean maintaining a negative moderator coefficient in the RCS, you need to so state. The way it reads now is maintaining a negative moderator coefficient in the accumulator. If that is the case, it is not credible and below the level of knowledge for an SRO. Edited distractor B so it is (1) (2). Modified distractors B and D to indicate the moderator coefficient is in the RCS.
Question now SAT.
94 H
3 N
E S
Reorder distractors B, A, D, C. Reordered distractors. Question now SAT.
95 H
3 N
E S
There is a floating "t" in the stem just before "The Unit is in Mode 4." Drawing is difficult to read. Reorder distractors D, C, B, A. Removed the floating t. Will provide full page picture. Reordered distractors.
Question now SAT.
96 H
3 M
E S
Reorder distractors B, D, A, C. Reordered distractors. Question now SAT.
ES-401 11 Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 97 F
3 N
E S
Knowing the transfer will not occur until addendum 5 is complete seems more like overall mitigating strategy not SRO only material. Changed the stem to indicate a Red path in core cooling exists. Question now SAT.
98 H
3 N
S 99 H
3 N
E S
Reorder distractors B, A, D, C. Reordered distractors. Question now SAT.
100 F
3 N
E S
NO CFR reference in pedigree. Reorder distractors B, A, D, C. Added CFR reference. Reordered distractors. Question now SAT.
ES-401 12 Form ES-401-9 Rev. 11 13 0
1 0
4 3
0 0
0 0
3 0
2 unsat Stem focus Cues T/F Cred Dist Partial job link units minutia backward KA SRO-only LOD = 1 ST 2018-10 Retake Initial Written Exam Statistics