ML18080A012
| ML18080A012 | |
| Person / Time | |
|---|---|
| Site: | Arkansas Nuclear  |
| Issue date: | 03/07/2018 |
| From: | Vincent Gaddy Operations Branch IV |
| To: | Entergy Operations |
| References | |
| Download: ML18080A012 (11) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only Make all 2x2 questions consistent in format, such that the stem is written as XXX (1) and XXX (2) 1 H
3 B
E S
2010 Bank Question If PZR level is 30 inches it could be assumed that it is less than 55 inches as well which would make distractor C also a correct action to take, as written.
Change the reason for A being correct in the distractor analysis to state that A is correct because it is a floating step, and reference the EOP/AOP Users guide. Revised distractor analysis. Question now SAT.
2 H
2 N
E S
T1025 and RCS leakage parameters in stem should have a trend associated with them. If possible without messing up answer distribution, reorder distractors so they are short-to-long. Added "and rising" to T1025 and leakage parameters, re-ordered answer choices short-to-long. Question now SAT.
3 H
3 M
S Modified from 2011 Exam 4
H 3
N S
Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, and length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).
- 4.
Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7.
Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
ES-401 2
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 5
H 3
B E
S 2011 Bank Question CFR says 41.45, should be 41.5. Assuming the given conditions do exist, then Distractor D is not credible with the added information at the end, Specifically due to loss of seal injection at power there are no trips associated with a loss of seal injection alone. A better Distractor D would be, Trip the Reactor, Trip the HPI pump due to a leak in the reactor coolant system. The reason that the trip piece would not be correct as the leak in not in the RCS but the letdown and purification system.
Corrected CFR reference. Took reason given in D and used that in B, and inserted excessive RCS leakage as reason in D. Question now SAT.
6 F
2 B
S 2007 Bank Question 7
F 2
X N
U S
It is minutia to know exactly which channels cause isolation of SW to ICW coolers was difficult since the assignment of the RB isolation valves to specific channels does not follow a discrete methodology, Revised first part of C and D distractors to "5 and 6".
Question now SAT.
8 F
2 N
S 9
F 2
M E
S Modified from 2004 Exam Modify the second half of distractor B and D to energize the shunt trip coils to trip the CRD breakers.
Safety Rods just doesn't seem as credible. Changed second part of B and D to tripping all rods via the A&B breakers. Question now SAT.
10 H
3 N
E S
There is nothing in the stem to indicate that cooldown is currently greater than 50°F. Added cooldown rate of 70 °F/hr. Question now SAT.
11 H
3 N
S 12 H
3 M
S Modified from 1999 Exam 13 F
2 N
E S
Distractors are too lengthy. Change the stem to be something like: " the discharge rating is ____
hours" to carry emergency DC and vital AC loads for a minimum of ____ hours." Modified stem and answer choices to reduce length of answers.
Question now SAT.
ES-401 3
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 14 H
3 B
E S
2008 Bank Question If possible without messing up answer distribution, reorder distractors so they are short-to-long.
Swapped B and C positions so answers are long-to-short instead of short-to-long, to preserve answer distribution. Question now SAT.
15 H
3 B
S 2010 Bank Question 16 H
3 N
E S
Move the bullet regarding P-4B being aligned to the green train to bullet 3 (just below bullet regarding P-4A and P-4C being in service). Stem states that the low level annunciator is K10-A3, but it is K10-A4. I am not convinced that B is not also correct. It is an action in the ARP, and could be taken prior to the CRS getting to the proper step in the AOP. Revise stem to eliminate possibility of B being correct.
Moved P-4B condition of being aligned to green train to 3rd bullet, corrected annunciator designator, revised stem so it is asking for FIRST action to be taken. Question now SAT.
17 H
3 X
B U
S 2008 Bank Question Question does not match the K/A. Distractor D easily eliminated as it is the only distractor that has no positive action. Taxonomy on form states "K".
Corrected taxonomy. Revised question since it did not match K/A. No component is operated/monitored until IA pressure is less than 60 psig, revised pressure to 58 psig. Added low Inst Air press alarm.
Replaced distracter B with placing CV-1206 in override, this is now the correct answer. Added trip the reactor as distracter C. Deleted "suddenly" from IA pressure condition, and added "and stabilizes",
revised D to makeup to Surge Tanks to maintain normal level band. Question now SAT.
18 H
3 N
E S
Make initial power 360 MWE so that it is closer to 43%, but still below it, to add credibility. Changed initial power to 360 MWE. Question now SAT.
19 H
3 B
E S
2002 Bank Question Item 4 (single Sel to All) not used in any distractor.
Deleted #4 (single sel to all) since it was not used in any answer choice which required a re-numbered list and revised answer choices accordingly. Question now SAT.
ES-401 4
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 20 H
4 B
E S
Change bullet 1 to 350 MWE since AOP discussed reducing power to a generator output, not reactor power. Changed first bullet to 350 MWE. Question now SAT.
21 H
4 M
E S
Modified from 2014 Exam The change made from the parent was really only changing the M/U flow rate, so this would qualify as a bank question. I request that you additionally change the direction of the pressurizer level trend, so it is decreasing. This would make it modified. Changed pressurizer level from rising to lowering which makes B the correct answer. Question now SAT.
22 F
2 B
S 2013 Bank Question 23 F
2 N
S 24 H
3 N
S 25 F
2 X
N U
E UNSAT for credible distractors. Distractors A and D do not appear credible. One good distractor would be to include RT-21 as an answer choice and modify the stem to state that all plant equipment operated as expected following the turbine trip. I am not sure what to do about distractor D, you may be able to convince me that it is OK be distractor A needs to go for sure. Replaced Repetitive Tasks in A and D with more plausible AOPs, revised stem for clarity due to this change. Question now SAT.
26 H
3 M
S Modified from 2017 Retake Exam 27 H
3 N
S Discovered during NRC validation week that this question was similar to a simulator JPM. It was determined this question is indeed testing a different concept than the JPM. Made "D" HPI line flow 165 gpm and "A" HPI line flow 125 gpm so that the line with the highest flow wasn't the same as the JPM.
28 F
2 B
S 2011 Bank Question 29 H
2 B
S 2008 Bank Question
ES-401 5
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 30 H
F 1
3 M
B U
S Modified from 2002 Exam 2013 Bank Question This is a GFE question and should be avoided as this has been tested during that exam. It be a better question to ask if a boration or dilution is expected to be required, or not, initially. Replaced question with an existing bank question. Question now SAT.
31 H
3 N
S 32 H
3 M
S Modified from 2005 Exam 33 F
3 N
E S
I would change distractor C to 12.7 ft., which is the alarm setpoint. This would still make this the correct answer and make the distractor more credible than 12.5 ft. Revised the wording of the stem to make 2.5 ft. a more credible distractor. Question now SAT.
34 H
3 B
E S
2002 Bank Question A better distractor A would be PERFORM 1103.013, RCS Leak Detection, RCS Leakage Monitoring section. Replaced distractor A with RCS Leak Detection procedure. Question now SAT.
35 H
3 X
N U
S Distractor C and D are not credible as written. The distractor analysis states that "Cycling CV-1221 is a technique used when re-establishing letdown following a high temperature isolation (135 °F setpoint) but is not specified by the ACA as an immediate response to this alarm," but there is nothing in the stem to indicate what letdown temperature is. Added condition of Letdown temperature 132°F, this confirms temp alarm but is less than isolation setpoint of 135°F. Question is now SAT.
36 F
2 B
S 2017 Retake Bank Question (from last two exams) 37 F
2 N
S 38 F
2 N
E S
Distractor C not credible. I don't think it is credible to believe that condenser vacuum would originate from RPS. Revised distractor C to RCS pressure.
Question now SAT.
39 F
2 N
S 40 H
3 B
S 2013 Bank Question
ES-401 6
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 41 F
1.5 B
S 2013 Bank Question 42 H
3 N
S 43 H
2 B
S 2017 Retake Bank Question (from last two exams) 44 F
3 B
S 2005 Retake Bank Question 45 H
3 N
S 46 H
3 N
E S
If possible without messing up answer distribution, reorder distractors so they are short-to-long. Revised answers short-to-long. Question now SAT.
47 H
3 B
S 2008 Bank Question 48 H
2 B
S 2011 Bank Question 49 H
2 N
E S
Is it possible to find a forth trip.to have equal distribution of answers? As it is right now, simply knowing overspeed eliminates A. Changed B to "3 ONLY. Question now SAT.
50 F
2 N
S 51 F
2 B
S 2011 Bank Question 52 H
2 N
S 53 F
2 B
S 2007 Bank Question 54 F
2 B
N E
S 2013 Bank Question Not an UNSAT question but it needs to be changed.
Questions 53 and 54 are both testing the setpoints of the Instrument air system. I prefer question 53, so question 54 should be replaced. Replace with 026 Containment Spray, K/A A4.01. Replaced question with new question, 026 Containment Spray, K/A A4.01. Question now SAT.
ES-401 7
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 55 F
2 B
E S
2007 Bank Question Distractor C does not having any real basis, need to beef that up. Something like 12 days was chosen as when combined with the 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> given in the stem comes to a 14 day LCO witch is a common time given for completion of an action statement. Deleted 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> ago and revised second given condition accordingly, all answers changed. Question now SAT.
56 H
3 N
S Reference Provided 57 H
3 M
S Modified from 1999 Exam 58 H
2 M
E S
Modified from 2014 Exam Why do two distractors say "will open (close)" and the other two say "will go full open (closed)"? Revised all answers to be more specific regarding valve travel.
Question now SAT.
59 F
2 B
S 2002 Bank Question 60 F
2 N
S 61 H
2 X
N U
S Why would anyone believe that having more feed than power would limit reactor power, especially give the name of the alarm? That makes those distractors seem not credible. Also why would you trip the reactor when a single feed pump can supply over 50% power? Dont think that is overly credible.
Maybe changing the second half of that part is to take manual control of the affected feed pump and raise speed. Revised 1st part of distractors A and C, now are just like B and D but with incorrect setpoint.
Question now SAT.
62 H
3 N
E S
May need to provide more detail on the second half of the question are we talking Tave, Thot or Tcold or do all three go up together just make sure we are thinking of this. Also "bias applied early (now)"
seems oddly worded. Since the stem says "NOW" can't it just say "NOW - Bias for TBV controller applied?" Deleted parenthetical "now", changed temperature to Tave in stem. Question now SAT.
63 H
3 M
S Modified from 2017 Retake Exam Reference provided
ES-401 8
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 64 H
3 B
E S
2002 Bank Question Reword the distractors to make them look the same, i.e., pump will start or pump will not start. Changed "remain off" in C and D to "not start. Question now SAT.
65 F
2 N
E S
Wouldn't distractor B be more credible to state that CV-4806 closes? Why the valve would open on a high rad, makes no sense. Revised B to be CV-4806 closes. Question now SAT.
66 F
2 M
S Modified from 2011 Exam 67 H
F 2
M S
Modified from 2017 Retake Exam Fundamental not higher order question. Changed LOK to Fundamental.
68 F
2 X
N B
U E
2013 Bank Question Distractor A is not credible since it is essentially stating production over safety. It would be more credible to state trip the reactor to place it in a known condition. As one of the distractors. Merely flipping the table does not make the distractors credible.
Distractor D having the briefs during emergencies cues this as not the correct answer. Difficulty writing discriminatory question with this K/A. Resampled K/A and selected 2.1.38. Replaced question with a bank question from the 2013 exam. Question now SAT.
69 F
2 N
S 70 F
2 B
E S
2005 Retake Bank Question First part of the stem is unnecessary. Simply ask what is a channel check as defined in TS? Deleted 1st sentence as it was unnecessary. Question now SAT.
71 F
2 X
N U
E Change signs for distractors A and C to signs. It is not credible for anyone to think that a critical reactor would be mode 3. It probably makes this question LOD 3, as it is differentiating between whether 0.99 exactly is mode 3. Deleted Keff values, changed Tave in A and D. Question now SAT.
72 F
2 B
S 2017 Retake Bank Question (from last two exams) 73 F
2 B
S 2013 Bank Question In the question pedigree change the LOD to 2 vice 0, if it remains 0 by definition the question is UNSAT.
Corrected LOD value.
ES-401 9
Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 74 F
2 B
S 2017 Retake Bank Question (from last two exams) 75 H
3 B
S 2001 Bank Question 76 H
4 N
S 77 H
3 X
N U
E Reference provided I fail to see how the unusual event distractors are credible, since the distractor analysis states it is SU7 based on amount of leakage, but there is no indication of the quantity of leakage in the stem.
There are 2 EAL calls on the written and one admin JPM. This is oversampling EALs. I would prefer to see one of the written questions (#77 or #92) replaced. My preference would be to replace #77.
Replaced EALs with Repetitive Tasks 8 and 11.
Deleted last bullet item to remove cueing since it contains RT-11, and revised stem. Question now SAT.
78 H
3 M
S Modified from 2014 Exam Moved the last 3 parameters down and added "NOW the CBOT reports the following critical Parameters."
79 H
3 B
E S
2013 Bank Question Reference provided Reorder distractors such that they go (with current times) D, C, A, B. You want the distractors to be from soonest to latest. Re-ordered answers from earliest to latest, correct answer now "C. Question now SAT.
80 H
4 B
S 2009 Retake Bank Question 81 H
3 M
S Modified from 2013 Exam 82 H
3 N
E S
Remove mention of "less than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />" in the stem.
Deleted "within one hour" from second part of "B" and "C" to eliminate cueing. Question now SAT.
83 H
2 M
S Modified from 2017 Retake Exam 84 F
2 N
S Reference provided 85 H
3 M
S Modified from 2008 Exam
ES-401 10 Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 86 H
3 N
S Reference provided 87 H
3 M
S Modified from 2017 Retake Exam 88 F
2 B
S 2009 Retake Bank Question 89 H
3 N
E S
The SRO only knowledge of this question is to select the correct procedure, since you will meet the entry conditions for both listed procedure. Beef up the stem to make it obvious that the entry conditions are met for 1203.037. Added EDG auto start annunciator to conditions, also added condition of CBOT reporting B5 bus voltage is 0 volts, changed DG1 bullet to simply "tied onto A3. Question now SAT.
90 H
1 3
N U
S Reference provided With reference it is LOD 1, the TS for RBS is above the line and therefore no reference required and with the containment spec given it is a direct look up.
Removed TS 3.6.5 from SRO handout. Question now SAT.
91 H
3 B
S 2002 Bank Question 92 H
3 N
S Reference provided There are 2 EAL calls on the written and one admin JPM. This is oversampling EALs. I would prefer to see one of the written questions (#77 or #92) replaced. My preference would be to replace #77.
Question #77 was modified to correct oversampling concern.
93 H
3 N
E S
Reference provided Beef up the distractor analysis for D on why TRM 3.7.9 action A.1.2 is not plausible, otherwise, D could also be argued as a correct answer." Added more description to analysis for distractor D on why D is also not a correct answer. Question now SAT.
94 F
2 N
S 95 F
2 N
S 96 H
3 N
E S
Change Distractor D to ".if a risk evaluation is NOT performed within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> of discovery. Revised distractor D to "...if a risk evaluation is NOT performed within 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br />... Question now SAT.
ES-401 11 Form ES-401-9 Rev. 11 Q
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6. Source (B/ M / N)
- 7. Status (U /E /S)
- 8. Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 97 H
1 2
B E
S 2001 Bank Question Reference provided This is an LOD 1 with the reference provided, as it is direct lookup. Even if only the table were given, it would still be LOD 1 because one knows to take the item with the lowest total. If only providing the table as the reference, then it is not LOD 1. Also change distractor A to 2,3,5. That assures that each distractor has a different point total. (Edit only because it is a Bank used on a previous NRC exam.)
Revised distractor A to be 2,3,5. Removed all but the table of Att. M from SRO handout. Question now SAT.
98 F
2 N
S Replaced question due to validation comments.
99 H
3 N
S Reference provided 100 F
2 B
E S
2013 Bank Question Why would the Unit 2 shift manager be the appropriate to direct emergency operations of Unit 1 when shift manager for Unit 2 is almost assuredly not qualified for unit 1? Why would the TSC be manned if that alert has only been called for 10 min? Beef up the distractor analysis for B, and add to the stem that the TSC is operational. Changed times to day shift, added condition of TSC operational, added info about FLEX to Notes. Question now SAT.