ML21006A420
| ML21006A420 | |
| Person / Time | |
|---|---|
| Site: | Cook  |
| Issue date: | 08/25/2020 |
| From: | Indiana Michigan Power Co |
| To: | NRC/RGN-III/DRS/OLB |
| Reeser D | |
| Shared Package | |
| ML19121A507 | List: |
| References | |
| Download: ML21006A420 (35) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 2020 DC Cook Written Examination Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only General Written Exam Comments:
Answer explanations & distractor justifications need significant work - distractors must describe not only why each is incorrect, but why each is plausible.
Question sources are not included in any question.
Questions are typically very long-winded and very wordy. Many questions include paragraphs of information, which can result in applicant fatigue throughout the exam (and challenging the exam time limit).
Questions with question stems asking two distinctly related sub-questions generally have long answers/distractors and could be easier to read/clearer if the question stem labeled each sub-question as (1) and (2) and the answers/distractors were separated out in the same format.
Significant inconsistencies throughout the exam on the use of bolded, underlined, capitalized wording to emphasize specific points.
Many question answers/distractors are not organized shortest to longest (App B C.2.a).
Questions that use the intro Which ONE of the following Change to Which of the following 1
F 2
B S
2 H
3 B
E 2014 ILE NRC:
Answer pedigree should be expanded to include reasoning why each part of the distractor is incorrect. For example, distractor A is not only incorrect due to Part 1, but also Part 2.
Part 2 of distractor B states Temperature on the PRT, shouldnt this be Temperature of the PRT?
Licensee: Incorporated above comments.
3 H
2 B
E 2016 PINGS ILE NRC:
Space needed in stem part 2 - (2)Heat removal Distractors should reflect part 1 using the (1) format, as used in part (2) of the distractors.
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 4
H 3
X M
E NRC:
Is it realistic that a plant shutdown would be ongoing per the Power Reduction procedure when a Rx startup was in progress with power at 2E-8 amps (wouldnt the crew initiate a Rx trip at that power level rather than initiate reactor shutdown)?
Consider changing to stem to start at a higher at power level.
Answers should be organized shortest to longest (App B C.2.a)
Licensee: Changed power level to 12% during plant startup 5
F 2
B S
6 H
3 X
N E
NRC:
Missing Unit number in stems first bullet.
Consider removing ONE (1) from the question stem, as it is unnecessary and could confusion (due to 2 parts to each answer).
Licensee: Incorporated above comments.
7 H
3 N
S 8
H 3
X N
U NRC:
Applicant may ask the condition of the plant (mode/
power level)
With the way the question is written, the applicant does not need to know both the Rx trip and SI setpoints (or even the coincidences) - if the applicant knows the SI setpoint of 1775 psig, the applicant is cued that 1950 psig is the correct Rx trip setpoint. In this instance, the applicant still has to determine the correct SI coincidence, however, the applicant does not have to know the Rx trip coincidence, as both choices A and B are 1/3.
Licensee: Added Plant Condition to question, Distractor B to 1/2 at 1910, and C to 1950, to even out choices.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 9
F 3
X X
B (U)E 2010 ILE NRC:
- 1.
Stem should read RCS is from ___________
heat. The word heat should be moved to the A and B answer choices.
- 2.
Why is the first part of choices C and D incorrect? Decay heat following a Reactor Trip would be on this order. Recommend changing the choice to fission heat.
- 3.
How is the first part related to the K/A?
Licensee::Removed heat from stem, moved to the end of A & B, Changed C & D to fission heat. Operational implications as they apply to the rating of the safeguards capability to remove the heat 10 H
3 X
B E
2010 ILE NRC:
- 1.
The 2nd sentences of distractors A & B are not appropriate since they are statements that do not answer the question that was asked.
- a.
Possible choice might be: Raise AFW flow to 60x103 pph on each Steam Generator until NR level is >28%, then adjust flow to stabilize NR level.
- b.
Possible choice might be: Maintain AFW flow at its current value until WR level drops to 26%, then adjust AFW flow to maintain WR level >17%.
- 2.
Remove procedure references from the answer choices since the question stem already refers to the applicable procedure.
- a.
Truncate choice D at SG levels >13%.
Licensee: Incorporated above comments.
11 H
3 M
E NRC:
Missing comma after 2-OHP-4022-055-003.
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 12 H
F 2
B E
NRC:
Missing reference documentation.
Is (amp-hours) necessary in distractor A (i.e. does battery capacity need to be explained as such)? If yes, distractors B, C & D should read similarly.
Answers should be organized shortest to longest (App B C.2.a) Order not changed.
Licensee: Incorporated above comments.
13 H
3 B
E 2006 ILE NRC:
Incorrectly does not need to be bolded. The key point is that the breaker was opened and subsequently reclosed. Recommend removing the bolding.
Licensee: Incorporated above comments.
14 F
3 X
N U
NRC:
- 1.
Question does not match K/A. K/A is tasks performed by the RO outside the control room NOT tasks directed by the RO. Questions for this K/A are usually related to control room evacuation.
Licensee: Task is typically performed by the RO. These panels are located behind the boards, Outside of the Control Room Horseshoe Still does not match intent of K/A Reselect K/A to 000038 K1.01 14R F
3 B
S NRC Exam 2012 NRC:
Identify that Steam Tables will be provided.
How were answer choices selected?
15 F
H 3
B E
2014 ILE NRC:
The question stem should be rewritten for clarity -
consider: Which of the following sets of components completely lose ESW cooling capability due to these actions?
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 16 H
3 N
E NRC:
The stem intro could be greatly simplified by simply stating that 1-OHP-4023-E-3, Steam Generator Tube Rupture, is being implemented and the crew is restoring Control Air to the Containment.
Remove air leak reference from choices A and B. Why pressure is low is assumed and not verified by the procedure before continuing.
Licensee: Incorporated above comments.
17 H
3 X
B E
2016 Braidwood ILE NRC:
The question appears to address two leaks, one inside cnmt from the PZR safety valve and a second leak outside of cnmt (RWST water, as specified).
The question is asking about isolating the leak outside of cnmt, but this could create confusion about the initial PZR safety valve leak. Recommend rephrasing question stem. Entry to ECA 1.2 is not supported by stem conditions especially with the assumption that leakage is RWST water. Need further explanation of why ECA 1.2 entry is valid.
Answers should be organized shortest to longest (App B C.2.a)Order not changed.
Licensee: Clarified to ask actions to isolate leak outside of containment.
18 F
3 N
S 19 H
2 B
S 2004 ILE
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 20 H
3 M
B S
NRC While the answer and a distractor may have changed, the question does not include any modification to the stem and does not result in the question being modified IAW ES-401 D.2.f.
Why is TAVG LO TAVG<TREF DEVIATION bolded in the stem in one spot and not in another?
It does not appear bolding is necessary in the stem.
Answers should be organized shortest to longest (App B C.2.a) Order not changed.
Licensee: Incorporated above comments. Agree it is not modified 21 H
3 B
E NRC:
Answers should be organized shortest to longest (App B C.2.a) Order not changed Is it necessary to include Letdown restoration in choices A and B?
Licensee: Removed letdown restoration.
22 F
3 N
E NRC:
Correct spacing between performing and actions The question refers to the applicant to enter a procedure - Who is the applicant in this scenario (RO)?
In step 5 of the procedure, can containment purge be restarted upon closing the airlock doors? If so, what is the basis behind restarting containment purge?
Answers should be organized shortest to longest (App B C.2.a) Order not changed.
Remove the bulleted item related to Containment Purge status. It is not material to the answer since all choices verify status and it doesnt matter if it is running or not.
The last two bulleted items can be combined: The SRO-CA is implementing due to Licensee: Incorporated above comments. Applicant changed to Control Room. Step 5 addresses Aux Building vent not Purge.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 23 F
2 B
E 2010 ILE NRC:
Recommend formatting such that 1.2 x104 dpm/100 cm2 is on one line Revise explanation for answer choice B to describe a radiation area not a high radiation area Licensee: Incorporated above comments.
24 F
2 X
X B
U NRC:
The second part of the answer choices is unrelated to the K/A and is not asked for by the stem.
The K/A refers to the interrelationship between detectors and alarms. There is nothing in the question related to alarms.
The use of the word UNIT is confusing. Does it refer to plant unit or monitor unit?
Licensee: Changed stem to include second part of answers, Used Plant unit in the lead in and changed choices to they accurately monitor and alarm Operators need to know that they only accurately indicate when the PORV are open.
Question will be reworded with statement similar to Two Radiation Monitors per plant Unit with (1).
They provide accurate indication and valid alarms only when (2).
25 F
3 N
E NRC:
The word successfully does not need to be bolded and underlined. This is atypical for emphasized words throughout this exam (also, this does not necessarily need to be emphasized).
Answers should be organized shortest to longest (App B C.2.a)
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 26 H
3 X
X B
(U)E 2004 ILE NRC:
The question seems to be immaterial since steam release should have already occurred, if not automatically by the PORVs, then automatically by the safety valves. Discuss further. See comment in answer key.
The answer to the question appears to be bases level knowledge and is SRO-only. The discussion of two-phase flow and water hammer exists in the H.3 bases document for EOP step 1 and does not appear in the H.2 bases document or the H.2 or H.3 FR procedures.
Additionally, the question asks, in accordance with FR-H.2. FR-H.2 does not appear to address the risk of releasing steam with the concern of high S/G level.(hence the kickout to FR-H.3).
Answers should be organized shortest to longest (App B C.2.a)
Licensee: Major Action Category which is RO Knowledge required per IN-81-28. Remove in accordance with FR-H.2. from stem. The only way to enter this procedure (and KA) is if the PORVs and Safeties have failed.
Question revised to state the Steam Generator #13 pressure is cycling on the SG Safety Valve. YELLOW path will be changed to Steam Generator Level and procedure to be entered will be FR-H.3. Rreworded answer choices and additional information in Answer Explanation included.
27 H
3 B
S 2012 ILE NRC:
Recommend stating All Reactor Coolant Pumps were tripped instead of The Reactor Coolant Pumps Answers should be organized shortest to longest (App B C.2.a)
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 28 H
3 B
S 2012 ILE NRC:
Missing commas after QRV-251 and QRV-200 Licensee: Incorporated above comments 29 H
3 X
B E
NRC:
Consider rephrasing stem for consistency. Unit 2 was at 40% reactor power with the following conditions: and then bulletize the Thot failure and Tavg.
How is Distractor A plausible (raising Pzr level to 100%)? How is that a misconception or something the applicant would think is plausible?
The 1st part of the answer choices are incomplete statements.
Rephrase the 1st part of the answer choices to refer to pressurizer programmed level or pressurizer level setpoint.
Licensee: Incorporated above comments. Distractor A considered plausible based on validation of question.
NRC:
In choices A, B, and C, recommend changing final level to a level band (e.g., 50-60% and 35-45%)
30 F
2 B
E NRC:
Answers should be organized shortest to longest (App B C.2.a)
Licensee: No changes made
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 31 H
4 X
X N
U NRC:
Part 1 of the question is considered minutia. The specific timing (and subsequent calculation based on ECCS flows) of which the alarm comes in is minutia. As such, this is how the question is attempting to meet the K/A.
Correct spacing between CTS and pumps RWST TS required level is below the line as it exists in a Surveillance Requirement. Question appears to be SRO-only based on the requirement to know SR values from memory.
Answers should be organized shortest to longest (App B C.2.a) Order not changed.
Licensee: This question is not asking the detailed time the alarm comes in but for a general understanding of the expected time at which to begin the actions related to the time critical realignment. The operator should have an appreciation for how long it takes to drain the RWST during a normal LOCA. (10 or 20 or 30 minutes)
Changed RWST Tech Spec Level to Annunciator which lists 375,000, removed space after CTS Additionally the 2nd part of answer choice A will be changed to match the 2nd part of answer choice D.
32 F
3 X
B E
NRC:
The word ALL does not need to be bolded and underlined. This is atypical for emphasized words throughout this exam.
Distractor D does not appear to be plausible, as #7 could not be valid (based on the wording), without
- 1 (and #1 is not included in this distractor).
Licensee: Incorporated above comments.
33 H
3 N
E NRC:
Add commas after valve EPN, prior to noun name.
In the question stem, ONE is not capitalized as it is throughout the rest of the exam.
Answers should be organized shortest to longest (App B C.2.a)
Licensee: Incorporated above comments.
34 F
2 B
S 2002 ILE
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 35 H
3 B
S 2018 ILE 36 F
2 B
S 2008 ILE 37 F
2 B
S 2014 ILE NRC:
Correct spacing between over and temperature.
Licensee: Incorporated above comments 38 H
3 B
S 2012 ILE 39 H
3 N
S Evaluate this question for potential overlap with Scenario 2, Event 5.
Licensee: No overlap concern.
40 F
3 M
S 2018 ILE Recommend adding space between parameter and numerical values so that the negatives values stand out.
Licensee: Incorporated above comments 41 H
3 B
E 2014 ILE NRC:
Answers should be organized shortest to longest (App B C.2.a)
Is perform a containment pressure relief the proper terminology for performing a containment release?
In choice B, which AHUs are shutdown (inside containment, outside containment)? Be more specific.
Licensee: Incorporated above comments.
42 H
3 M
E 2018 ILE NRC:
RWST level should be represented as 80%
instead of 80 percent for consistency.
Consider emphasizing two minutes in the stem.
Licensee: Incorporated above comments.
43 H
3 B
S
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 44 H
3 M
S NRC:
Consider identifying each of the two parts of the question stem as (1) and (2) and then labeling the answers/distractors as (1) and (2).
What are the plant conditions (power level, MFW pump status, etc) in the context of the question?
Licensee: Incorporated above comments.
45 F
3 N
E NRC:
Stem of the question is very wordy and should be rephrased to make as concise and clear as possible.
Put quotation marks around procedure title OR commas before and after title.
In the question stem, ONE is not capitalized as it is throughout the rest of the exam.
Reword for clarity. Consider inserting trip input to the between pump and AFW; and logic after start.
Licensee: Did not change stem wording. Incorporated all other above comments.
46 F
3 B
E NRC:
Answers should be organized shortest to longest (App B C.2.a)
Licensee: No changes made
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 47 H
3 X
B S
2007 ILE NRC:
The question is rather vague, asking Which ONE of the following failures is the cause? This is vague.
After providing initial conditions, the question should state something such as A transient occurred, resulting in the following conditions: then list the conditions and then ask Which ONE of the following failures caused the transient?
Regarding the post trip conditions: Would all of the components listed in the 2nd bullet have been running prior to the trip? How does the list support the conclusion that Bus 2D has tripped?
Choice D explanation needs additional information as to why it is wrong; e.g., explain how Bus 2D loads would be affected.
Licensee: Rephrased the question. 2 of the 5 pumps in second bullet would have to be running at 100%. The remaining listed pumps are powered from the same bus. Added information to distractor D explanation.
48 F
2 B
E 2006 ILE NRC:
What value is added by (Start) and (Run) in the answer choices?
Answers should be organized shortest to longest (App B C.2.a)
Listed reference refers to section 4.2 of attachment; section 4.3 was provided.
Licensee: Designators are terms used in procedure and on control board labels. Both sections 4.2 and 4.3 have same criteria. Wrong reference provided to NRC.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 49 H
3 B
S NRC:
Stem clarity may be improved by putting a timeline to the 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> and the 30 minutes.
Does the statement Power has just been restored from Emergency Power need clarification?
Answers should be organized shortest to longest (App B C.2.a)
The stated reference provides justification for the correct answer but does not provide justification for why the other choices are incorrect.
Licensee: Clarified timeline for 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> and then for 30 minutes later. Emergency Power is an understood term.
50 H
3 B
S 2010 ILE 51 H
3 X
B (U)E 2012 ILE NRC:
Where is the system (PRM) tie to this question? The original question (2004 NRC Exam Q99) was written to exam Generic K/A 2.3.1. The only modifications to the question were the dose rates and exposure times.
Licensee: Replacing question.
51R H
3 N
E NRC:
What are the normal release and dilution flow rates?
Include a nominal flow rate in the answer explanation.
Clarify dilution flow rate limit. It appears that the release flow rate is a maximum limit based on concentration and that the dilution flow rate limit is a minimum value based on concentration. So the answer choices should specify higher/lower minimum/maximum flow rates.
The question, or at least the K/A Justification, does not appear to match the K/A. Consider beefing up the K/A justification to explain how release and dilution flow restriction ensure activity levels are limited (intensity) to limit exposure to the environment. Question seems a better match for K/A A4.01.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 52 H
3 N
E NRC:
Consider rephrasing (or bulletizing) the first set of initial conditions. Also, bulletizing would allow to provide EPNs of the ESW unit crosstie valves, for consistency of valve identification.
EDG is spelled out in the post-transient conditions, but is referenced in the initial conditions prior to that without spelling out Emergency Diesel Generator Post-trip conditions AND the question stem state the CD EDG. Specify the Unit for consistency, 1CD EDG.
one in the question stem is not capitalized as it is throughout the rest of the exam.
Licensee: Incorporated above comments.
53 H
3 B
E 2010 ILE NRC:
Consider providing EPNs for ESW crosstie valves, for consistency of valve identification.
2CD EDG should be represented consistently (in Q52 it was represented as CD EDG and in this question is represented as DG2CD).
Assuming no operator action can be incorporated into the stem, rather than a note. Assuming no operator action, which ONE of the following describes Each part of the answers/distractors do not need to be capitalized.
Licensee: Incorporated above comments.
54 F
3 N
E NRC:
Stem could be clarified. Consider rephrasing the stem to something like Which of the following result in DIRECTLY tripping the Plant Air Compressor as a result of the loss of NESW?
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 55 H
3 X
M U
NRC:
Negative questions should only be used if there is a very good reason to do so (App B C.2.e). In this case, the question is negatively phrased simply to make the question modified and not bank.
Distractor justification for why #6 is incorrect; needs enhancement. Justification should reinforce that although the CEQ fans start on the same cnmt pressure setpoint as a phase A, they are not part of the phase A and therefore not addressed by the phase A recovery procedure. Since the question is asking what does not need to be realigned IAW that procedure, the justification should incorporate that.
Choice A explanation states that Cooling Water to Containment isolates on phase A, but the correct answer (C) does not explain why manual realignment is unnecessary.
Licensee: Replacing question.
55R H
3 B
E NRC:
Submitted question replaced with original pre-modified (Bank) version of the question.
Include a statement in the answer explanation stating that each of the components in the correct answer require one or more manual actions to restore them to operation. Additionally, include that Containment Upper Vent Fans automatically restart on Phase A reset.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 56 F
3 X
X N
U Reference provided NRC:
Based on the document provided for provided references to the applicant, the TS bases 3.3.3-4 was not included, only the TS. The TS table requires the applicants to know which specific instruments (beyond 2 required WR RCS pressure channels) are required for operability. As this question requires bases level knowledge to sufficiently answer the question, the question is SRO-only. (Facility objective?) Still SRO only.
Question does NOT specifically state which reference(s) are being provided, like Q60 does, for example.
Bases implies (but does not specifically state) that all three ranges in each channel must be operable in order for the channel to be operable. Is this required knowledge of the system? Question not answered.
K/A match? The K/A is not tested since the effect is given, without any description of the malfunction/loss.
Licensee: Bases not required or given. Added reference to be provided to question data. KA is tested the effect of the loss of RVLIS.
Replaced question 56R F
N U
NRC:
A better way to ask the question might be: How would implementation of the Core Cooling Status Tree be affected by a loss of Wide Range RVLIS indicators NLI-130 AND NLI-131?
TS 3.3.3 is listed as the supporting reference. Why?
Need to discuss with facility, how to make the question clearer.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 57 H
3 X
N E
NRC:
Throughout the exam, alarms have consistently been capitalized, not bolded. In the stem, the Control Rod Urgent Failure alarm is bolded (except the C in Control) and does not need to be.
BANK and ROD do not need to be capitalized (as they are not capitalized in the stem) and distract from the necessary capitalization of ALL, ONLY, or NO.
Stem is incomplete - using Attachment B Recovery By Positioning Single Rod. The procedure is not specified anywhere. The stem should include The crew is performing actions in accordance with 1-OHP-4022-012-005, Dropped or Misaligned Rod Stem is further fragmented by stating After a couple of steps, Control Rod Urgent Failure alarm occurs.
Provide context. Combine that statement and the last bullet and rephrase, such that The crew is withdrawing Rod H-8 to align with Control Bank D, in accordance with 1-OHP-4022-012-005 Attachment B, Recovery By Positioning Single Rod.
After X [provide a specific value for realism] steps, CONTROL ROD URGENT FAILURE alarm occurs.
Distractor D - Movement does not need to be capitalized and Lit should be capitalized. Not plausible with the question lead-in; how does continued rod withdrawal cause NO rod movement?
Choices A, B, and C need rewording to include indications to indicate withdrawal List of references should also include the annunciator response procedure for the Control Rod Urgent Failure alarm.
Licensee: Incorporated above comments. Replaced distractor D.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 58 H
3 X
B E
2004 ILE NRC:
Correct spacing between A and T/C in answer B.
Are there any references for responding to a failed Thermocouple (alarm response procedures, operating procedures, etc) that provide any further guidance beyond the lesson plan? The (b) part of the K/A requires procedure use/knowledge to mitigate the transient. The action taken here is done in accordance with what?
Licensee: Changed correct answer to include a procedure that is used to determine subcooling whenever desired, OHP-4023-SUP-001, Subcooling Margin Determination.
NRC:
Replace procedure name and number with an action (e.g., Selecting the T-SAT-RTD position on the XYZ selector switch) and add procedure ID to the explanation.
59 F
3 B
E 2010 ILE NRC:
Stem could be rephrased to provide the alarm on ERS-7401 in some initial conditions and minimize the wording in the actual question.
Licensee: No change made
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 60 H
3 X
X B
(U)E Reference provided 2012 ILE NRC:
Question stem states the crew has entered the Unit 1 FR-H.1 procedure, but the given conditions refer to unit-less SGs (#1, rather than #11, for example).
Which is the correct representation of feed flow, 50 E3 or 50 x 103, since the question provides one representation and the explanation (and H.1 FOP) provides the other.
The question asks what actions are taken. The correct answer is chosen simply by identifying correct choice using the 1st part of each choice; a single action. The second and third parts in the answer choices are not required. Rephrase the question to ask for the correct feed rate or change the distracters such that the 2nd and 3rd parts are applicable.
Why does the FOP need to be provided?
Licensee: Incorporated above comments by second action. Removed reference to be provided NRC:
The question can still be answered using only the first part of the answer choices. The 2nd part has no bearing on choosing the correct answer.
Licensee:
Deleted question (2) and associated responses from answer choices.
61 F
3 M
E NRC:
Answer/distractors should be reformatted to represent each part of the question being asked.
For a 2-part question, part (1) and part (2) should be clear. The format used is inconsistent with the rest of the exam.
The original question provided Hotwell pump, Condensate pump, and HD pump status. Why is that not necessary to provide in the modified question (when the first part of the original question remains asked in the modified question)?
Licensee: Reformatted to 2-part format. Other pumps are not required info for this question.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 62 F
3 B
E 2014 ILE NRC:
Distractor C appears to be referring to the applicant (the Unit 1 RO) as it refers to the Unit 1 operator.
Is this referring to the MCR operator, or the AEO?
This should be clarified.
Licensee: Incorporated above comments.
63 F
3 X
B U
NRC:
Question does not meet the K/A; tie the CWS in with ESFAS. While the MCR operator may have to manually open valves for the ESW system, the question does not tie in ESFAS (as a question based on the CWS and C-9 interlock would, for example).
Licensee: Replacing question.
63R H
3 B
E NRC:
Do we need a trend on Tave since one of the Turb Stop Valves is open? NO control valves will be close.
Recommend changing Choice A to read condensers and maintaining temperature on the Turbine Trip Controller. (similar to choice C.
Change LOK to H 64 H
3 B
E 2014 ILE NRC:
If the question answer/distractors are going to have each part of the question in a separated (1) and (2) format, the question stem should reflect that.
Licensee: Incorporated above comments.
65 F
3 B
S 66 F
2 B
S
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 67 F
2 B
E 2010 ILE NRC:
OHI-4000 should be referenced in the stem to provide the applicant direction as to what Conservative Decision Making is being defined in accordance with.
Licensee: Incorporated above comments.
68 F
3 B
E 2006 ILE NRC:
The use of (1) in the stem is not consistent with the rest of the exam.
Licensee: Incorporated above comments.
69 F
2 B
E 2004 ILE NRC:
This is primarily a systems question. IAW ES-401 D.2.a, ensure the questions selected for Tier 3 maintain their focus on plantwide generic K/As and do not become an extension of Tier 2.
Licensee: Need to discuss with NRC.
After further discussion elected to leave question as is with minor rewording of choice D.
70 F
3 B
E NRC:
In statement (1), the (s) in system(s) should be on the same line. It appears theres an erroneous space between the two.
Consider incorporating PMP-4043-APC-001 into the question stem as an in accordance with Licensee: Incorporated above comments.
71 F
2 B
S 72 F
3 B
(U)E 2018 ILE NRC:
This is primarily a systems question. IAW ES-401 D.2.a, ensure the questions selected for Tier 3 maintain their focus on plantwide generic K/As and do not become an extension of Tier 2.
Licensee: Replacing question.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 72R F
2 N
S 73 F
3 B
E 2014 ILE NRC:
This is primarily a systems question. IAW ES-401 D.2.a, ensure the questions selected for Tier 3 maintain their focus on plantwide generic K/As and do not become an extension of Tier 2.
Why are Correct and Operator both capitalized?
The question stem and immediately trip the Reactor and implement should be combined into one question statement for clarity.
Licensee: Corrected capitalization and moved trip the reactor.
NRC:
Need to rework Choice D.
74 F
2 X
B U
NRC:
How can the applicant rule out that with CS in service, that Cnmt pressure is not > 5 psig? The justification doesnt not appear to support the conditions provided in the question. As such, this could also be a correct answer.
Licensee: Clarified distractor C to say CTS is in service at actuation setpoint.
Revised C to state Containment pressure is 4.5 psig and slowly lowering with Containment Spray in service.
75 F
2 N
E NRC:
Question stem should reference the procedure -
Which set of actions contain the Operations responsibilities, in accordance with PMP-2270-FRP-001?
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 76 H
3 B
E 2006 ILE NRC:
The initial conditions should be modified to eliminate the -, as this could lead to confusion on the containment pressure - (+) 2.5 psig Licensee: Incorporated above comments.
NRC:
Consider spacing parameter values from description (see earlier examples).
77 F
2 B
S 78 H
3 N
E NRC:
Correct indenting on answer B Inconsistent use of Steam Generator abbreviation (SG). Either use the full spelling or use the abbreviation, dont switch back and forth.
Initial conditions should state the procedure the crew is utilizing (as distractor D states the crew is to remain in E-3).
Licensee: Incorporated above comments.
79 H
3 B
E Reference provided NRC:
Recommend removing year from the date and simply providing June 26 (similar to Q87). This would eliminate any questioning on a date 5 years in the past.
LCO 3.0 section should not be provided as an open reference. (dwr - why not?)
Licensee: Incorporated above comments. Reference okay?
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 80 H
3 N
E Reference provided NRC:
Distractor A spacing error LCO3.8.1 Distractor B has additional period after 3.8.1. and has an unnecessary comma after LCO 3.3.5 Condition A, & B Consider placing ONLY at the beginning of each answer statement and then separating the LCO 3.3.5 and 3.8.1 with AND Licensee: Incorporated above comments with the exception of moving ONLY and AND.
81 H
4 B
E Reference provided NRC:
- 1.
What portion of H.1 is expected to be provided and why? The Fold Out Page is being provided for Q60; what is being provided here? Feed and bleed initiation requirements should be well known, from memory,
- 2.
Distractors C and D are implausible since the SG WR levels listed in the stem are already below the levels specified in the answer choices.
Licensee: Fixed distractor C & D. FOP is reference provided and as identified it in references provided from previous question.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 82 H
4 X
X N
U Provided reference NRC:
Curve labeling of Ultimate Trip Curve provides cueing for tripping the Rx upon reaching the unacceptable region, rather than initiating a load reduction. Note - verbiage on curve below ultimate trip curve is illegible.
Question should clarify that part 2 is the action taken upon reaching the unacceptable region of the curve, as determined in part 1.
Correct spacing between bullets.
Consider enhancing conditions headings, such as Given the following conditions: and The following transient sequence occurs:
Condenser vacuum is represented two different ways Hg and inches Hg.
Licensee: Removed Ultimate Trip Curve from stem of question. Did not change part 2 to keep distractors plausible.
Curve labels will be modified for the exam. Removed reference to low vacuum alarm. Changed K/A from a Generic to A2.
83 F
3 B
E 2012 ILE NRC:
Would be clearer to provide the East CTS status in the initial conditions rather than an assumption.
East and West pumps are represented as such throughout the exam, not E or W. Questions is inconsistent.
Licensee: Incorporated above comments.
NRC:
If the procedures have already been performed what difference does the priority make? Question needs to be fixed so that there is a choice of which procedure to be implemented first.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 84 F
H 3
B E
2006 & 2008 ILE NRC:
- 1.
Spell out abbreviations used in the stem the first time they are used (ex. CSFST).
- 2.
The question should state: What action, and what is the concern if that action is not successful?
Licensee: Incorporated above comments.
85 F
3 X
B E
NRC:
- 1.
Based on OHI-4023, Implementation of the YELLOW condition FRP is at the direction of the Procedure Director. Based on that, it appears there are multiple correct answers (B and D, at minimum). What prevents the applicant from performing I-3 such that B would be incorrect?
- 2.
Question stem appears incomplete:
- a.
decision and reasoning whether ?
Recommend inserting behind or as to between reasoning and whether
- b.
to address this YELLOW?; Recommend adding path procedure after YELLOW.
Licensee: B is incorrect due to Note in OHP-4023 I-3.
Incorporated #2 comments. Explanation for distractor B revised to explain more fully why it is incorrect.
86 H
3 B
M E
2006 ILE NRC:
Correct spacing between answers & distractors Part (1) and Part (2).
Correct extra space between Concentration and TRM in Distractor C.
Correct verbiage in stem - 1) Failure is unnecessarily capitalized, 2) VCT level raised should be VCT level rose Question is modified from the 2006 ILE (Q26), not a bank question.
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 87 H
3 N
E Reference provided NRC:
Each answer/distractor contain Restore the train to OPERABLE status by: This information can be incorporated into the question stem such that each answer/distractor only contain the time and date.
Distractor A justification is incorrect. 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> from the time of bypass would be 1600, not 1400. 1400 is the time at which the TS allow bypass would run out, requiring the train to be placed in trip within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />.
Licensee: Incorporated above comments.
88 H
3 X
N U
NRC:
This question overlaps with one of the scenario events (Scenario 2, Event 5)
Question does not match the K/A. A2 K/A requires, at minimum, meeting the (b) part of the K/A, which the question does not do. Declaring equipment inoperable and entering a TS does not constitute use procedures to correct, control, or mitigate the consequences of those malfunctions or operations.
Ref ES-401 D.2.a.
Consider splitting stem initial conditions into bullets for clarity. Given the following conditions on Unit 1:
- 11 SG PORV, 1-MRV-213 failed open in auto Manual operator action was taken to place 1-MRV-213 in manual and closed.
Licensee: Replacing question.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 88R H
N E/U?
NRC:
In the first bulleted condition, Insert the word concurrent just before loss of power.
Move to isolate flow from SG 23 from the end of the last bullet to the end of the previous bullet.
Identified MRAs need to be changed to 2602 and 2702 (Unit 2)
Reword Choice A to state 4023-E-3 since SG 24 now shows signs of rupture.
Except to support that choices C and D are not the same, the reason for the action(s) is unnecessary for the K/A. Recommend changing either C or D to another procedure.
89 H
3 B
E 2006 ILE NRC:
The assumption made for the TDAFW pump status should be provided in an initial condition bullet.
Is the MCM-221 valve referred to in answer choice C the same 221 valve referred to in the other answer choices? If so, the same identifier should be used in all answer choices.
License: Added TDAFW alignment. No valves are not the same MCM is steam supply to TDAFP.
NRC:
Is there some reason why the TDAFW would not be in it normal configuration? If not, delete the statement.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 90 F
3 X
N E
NRC:
Missing comma after procedure # in stem.
Question stem is asking two questions in (1) and (2) format, however, the answer/distractors do not separate into (1) and (2) to reflect the stem.
Stem could be clarified by providing initial conditions in bulleted format, rather than a single, long sentence.
Why the difference in verbiage in part 1 of answer/distractors A&C and B&D - load must be reduced vs load is reduced. Given the past tense phrasing of the stem, those are saying the same thing.
Reference listed in the pedigree do not validate part 1 answer/distractors. Still need an explanation.
Licensee: Incorporated above comments.
91 H
3 B
E 2010 ILE NRC:
The statement providing current PZR level in the stem can be incorporated into a bullet in the given conditions.
Separate the 2nd bullet into two.
Licensee: Incorporated above comments.
92 H
3 N
E Provided reference NRC:
Provide clarification that the crew is performing the annunciator response for 12-OHP-4024-139, as the stem refers to Data Sheet #1 and the figure?
Licensee: Incorporated above comments.
93 H
3 B
E Provided reference 2018 ILE NRC:
Unit 1 is underlined, inconsistent with the rest of the exam.
Answers should be organized shortest to longest (App B C.2.a)
I dont recall TS/TRM bases being provided.
Evaluate necessity of providing the bases.
Licensee: removed underline and bases is not to be provided.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 94 F
H 3
N E
NRC:
Tier 3 questions need to avoid situations that are system/event specific or related to a specifically identified TS LCO. This question is event specific.
Plant is operating at 100% power. Which Unit?
Spell out EP in the stem.
Why is ECA-0.0 abbreviated, but OHP-4023-SUP-009 (referenced in 3 answer/distractors) is not?
Answers should be organized shortest to longest (App B C.2.a)
Licensee: Incorporated above comments.
95 F
2 X
B E
Provided reference 2006 ILE NRC:
Tier 3 questions need to avoid situations that are system/event specific or related to a specifically identified TS LCO. This question is event specific.
Question pedigree does not discuss the impact to the West CCP. In the stem, is the bus completely lost/faulted or does the EDG pick up and load the bus (such that the 2W CCP restarts)? Depending on such, the question could have partially correct answers, as racking in CCP breaker is not directly impacting plant safety. While the risk is increased, the plant safety is not impacted. The applicant could justify that plant safety is not impacted and chose answer B instead.
The initial conditions should include the initial unit power level.
Licensee: Incorporated above comments.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 96 F
2 B
E Provided reference NRC:
Reference provided is Figures 2 and 4 of PMP-2010-PRC-002. Figure 2 flowchart results in processing the OTSC using Attachment 3, which is not provided. Further, what refers the applicant to Figure 4? References submitted with submittal include Figures 2 and 3, but not Figure 4.
Clearer basis in the answer explanation is necessary to show why this would not be considered an OTSC (link to the stem statement plant conditions are not described in the current procedures or UFSAR).
Answers should be organized shortest to longest (App B C.2.a)
Without a copy of the procedure how does the examinee determine whether there is a change of intent.
Licensee: Incorporated above comments and will provide Attachment 3 with Figures 2&4. Fig 4 is used if distractor is chosen.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 97 H
2 X
B E
Provided reference NRC:
Why is the TS SR 3.0 section being provided? Why not? This detail is not required to be memorized.
- 1.
SR 3.4.11.1 is to be completed IAW the SFCP -
what is the duration of this SR? This should be referenced, as according to the answer justification, it must be less than 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> since SR 3.0.3 requires 24 hr or the frequency of the surveillance, whichever is greater (provided a risk assessment is completed).
- 2.
Replace the reference to SR 3.4.11.1 with a generic statement; e.g., It was just discovered that a required quarterly surveillance test, for a component required to be operable in Modes 1,2, and 3, has not been completed. This makes the question truly generic and eliminates the need to provide TS 3.4.11.
- a.
Revise the 1st bulleted item to state The surveillance interval (including the SR 3.0.2 allowance) expired two months ago.
- b.
Eliminate the 2nd bullet (incorporated into the opening statement).
- c.
Deleted 3.4.11 from the question.
Licensee: Incorporated above comments to make generic in nature. SR 3.0 allowed by NRC as reference?
98 F
3 X
X N
U Provided reference NRC:
Tier 3 questions need to avoid situations that are system/event specific or related to a specifically identified TS LCO. This question is TS LCO specific.
With LCO 3.4.16 provided, Part 1 of the question is direct lookup.
Licensee: Replacing question.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 98R F
U NRC:
I see nothing in TS 3.4.16 or its bases requiring a temperature reduction below 500-degrees F. Cooldown to cold shutdown required only if I-133 activity cannot be reduced below limits within 48 hours5.555556e-4 days <br />0.0133 hours <br />7.936508e-5 weeks <br />1.8264e-5 months <br /> or if it exceeds a very high level.
TS were revised after bank question was developed, invalidating the question. Will be replaced with bank question from 2010 NRC examination.
98R 1
F 3
B S
99 H
3 X
B E
Provided reference 2006 ILE NRC:
Tier 3 questions need to avoid situations that are system/event specific or related to a specifically identified TS LCO. This question is event specific.
Part 2 overlaps with Q85 - both questions test whether or not performance of yellow path procedures are required or discretionary.
What is the purpose of asking Part 1 of the question, considering both A and B, for example, include C.3 which provides for Saturated Core Cooling. Similarly, both C and D include C.2, which provides for Degraded Core Cooling. Part 1 is given by part 2 in both cases and is unnecessary.
Licensee: Replacing question 99R H
B E
2010 NRC Exam NRC:
Delete NOTE and list of procedures following the stem question. Information already provided earlier in the stem.
ES-401 Written Examination Review Worksheet Form ES-401-9 Q
- 1.
LOK (F/H)
- 2.
LO D
(1-
- 5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
Source (B/M/N)
- 7.
Status (U/E/S
)
- 8. Explanation Stem Focus Cues T/
F Cred.
Dist Partial Job-Link Minutia
- /
Units Back ward Q -
K/A SRO Only 100 F
2 X
B U
NRC:
Site Emergency Director in stem should include abbreviation (SED) as SED is used in answer D.
No correct answer. If Im not mistaken, the TSC SED cannot assume control until the TSC is capable of being activated in addition to the brief and completion of the Turnover checklist.
Licensee: Incorporated above comments and made D correct. New reference for explanation to include RMT-2080-CHK-001, SED Turnover Checklist.
Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:
- 1.
Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2.
Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)
- 3.
Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4.
Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- / Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5.
Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable)
- 6.
Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f.
- 7.
Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8.
At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).