ML21137A287
| ML21137A287 | |
| Person / Time | |
|---|---|
| Site: | Arkansas Nuclear  |
| Issue date: | 04/08/2021 |
| From: | Greg Werner Operations Branch IV |
| To: | Entergy Operations |
| References | |
| Download: ML21137A287 (20) | |
Text
ES-401 AN2-2021-03 Written Examination Review Worksheet Form ES-401-9 Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question a 1 (easy) to 5 (difficult); questions with a difficulty between 2 and 4 are acceptable.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
Cues: The stem or distractors contain cues (e.g., clues, specific determiners, phrasing, length).
T/F: The answer choices are a collection of unrelated true/false statements.
Cred. Dist.: The distractors are not credible; single implausible distractors should be repaired, and more than one is unacceptable.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem).The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- 4. Check the appropriate box if a job content error is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory).
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
Backward: The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only. (K/A and license-level mismatches are unacceptable.)
- 6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of Form ES-401, Section D.2.f.
- 7. Based on the reviewers judgment, is the question, as written, (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U status ratings (e.g., how the Appendix B psychometric attributes are not being met).
_Rev 3 Page 1 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 1
H 3
2 M
S LOD = 2, not 3. Pretty basic recognition of which pump is steam fed.
M-NRC 2002 Changed Question to a LOD = 2.
Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
2 H
3 M
S M-NRC 2015 3
F 2
B S
NRC 2008 4
F 2
M S
M-NRC 2019 5
F 3
2 X
X B
E S
Changed Question to a LOD = 2.
Specify NR or WR SG Level, or do you not differentiate?
ES-401 AN2-2021-03 2
Form ES-401-9
_Rev 3 Page 2 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only We determine NR or WR by the units % or inches. QID 62 of this exam specifically ask for this knowledge so we did not add NR or WR to this QID as it would potentially answer QID #62.
If youre going to use a safety function as a distractor, it makes more sense to use CORE HEAT REMOVAL, which has a closer nexus to RCS Heat removal, but is still wrong. RCS Pressure Control is not a very credible distractor based on S/G level.
The team made a decision to NOT use a safety function as a distractor but use EFW interposing relays for the distractor instead. SEE Rev 1 of QID and explanations.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
6 H
3 N
S 7
H 3
X N
E S
Part 1 is asking what the latest time Reactor must be tripped and RCPs stopped if CCW is not restored, so rephrase to: (1) What is the latest time by which the reactor must be tripped and RCPs stopped, if CCW is not restored? (1005 vs 1010).
Rephrased this QID as requested above.
Part 2 add Per AOP 2203.025 Technical Guide, to tie it to an explicit procedure.
Reworded to state Per the RCP Emergencies AOP 2203.026, Also, request that you confirm this is expected RO knowledge per ANO2 accredited ILT program, since EOP/AOP bases are typically considered SRO knowledge.
We hold SROs to the CEN 152 Basis for EOP steps along with the Technical Guides Basis we have developed here at ANO Unit 2. We also have Lesson Plan RO/SRO objectives and teach the ROs the steps of all AOP/EOPs and the basic reasons for the step per the Tech Guides/STMs/Procedure Notes & Caution. In this QID the reason is also listed in the CCW STM 2.53 which is now in the explanations for this QID.
Added the following Statement to the explanation to Answer A based on feedback from the NRC on 2/4/21 phone call: The reasons/basis for steps in the AOPs and EOPs are covered by RO/SRO objectives and are taught in the ANO Unit 2 ILT Training Program.
NRC OK.
8 H
3 X
N E
S I imagine pressure would drop fairly quickly to the new setpoint once spray starts, so freeze the analysis at a point in time, by modifying stem to, Immediately following the onset of the malfunction, the expected indication of Rephrased this QID as requested above.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
9 H
3 N
S
ES-401 AN2-2021-03 3
Form ES-401-9
_Rev 3 Page 3 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 10 H
3 N
S Is a 100 psig pressure band the standard range the CRS is expected to give? I dont want someone to argue that the given correct answer, 1025 to 1125 psig, is not actually optimal because there is some reason 1125 shouldnt be the top of the range. I agree 1025 is the optimal bottom.
Revised Part 2 of this QID to say To keep RCPs running per the CRS directions, RCS pressure should be lowered to a MINIMUM of ___ and deleted optimum Bands Changed all distractors/answer to a single pressure. Also raised Temp to 518/SAT Pressure for easier use of the steam table.
In Revision 2, changed the question to only have two pressures (correct one and the most plausible other one) turning the question into a 2 x 2 QID based on feedback from the NRC on 2/4/21 phone call.
NRC OK.
11 H
3 N
S 12 H
3 X
N E
S Part 1: I dont see how testing on whether ESFAS is initiated from the normal control board location, or a backup location in a locked room is credible or discriminatory.
Why would an applicant think they would not actuate it from the normal location in this question? Consider testing on application of RNO actions of step 15.E.RNO for aligning HPSI pumps, if one of the 4160V vital buses is not energized from offsite power. Or, the overall steps required to align OTC.
Replaced Part 1 with Part 2 and added new step for Part 2 to test on the overall steps to align OTC per the LOF EOP.
In Revision 2, changed the Part 2 step top state Step 15 will require the crew to ENSURE ALL available Charging pumps running AND ENSURE at LEAST
____(2)____ HPSI pumps are running based on feedback from the NRC on 2/4/21 phone call.
Nrc ok 13 H
3 N
S 14 F
2 X
M E
S M-NRC 2006 Part 1: not enthusiastic about testing on whether applicant knows which bus is red train and which bus is green train. Would rather test on whether applicant knows if inverter must be deenergized in 1 hr or 2 hrs (step 16 vs 17).
Rephrased this QID as requested above.
Part 2 is not very credible that an applicant would think an inverter is being deenergized in order to reduce load on an EDG vice Battery. Prefer that this question test on whether applicant knows if the vital 120VAC bus will be ENERGIZED
ES-401 AN2-2021-03 4
Form ES-401-9
_Rev 3 Page 4 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only or DEENERGIZED at completion of step 17 actions (should be energized by dirty power if I read step right).
Rephrased this QID as requested above; however, at the completion of Step 17 actions, the vital 120 VAC 2RS3 bus will still be De-energized as the Alternate Source of Power comes from 2B54 which is powered from 2B5 which has no power per the conditions in the stem. See reference drawings included in this QID file. We can discuss on the phone call.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
15 F
2 B
S NRC 2015 Battery is misspelled in stem Corrected spelling in C and D.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
16 H
3 X
X N
E S
REFERENCE PROVIDED The directive to suspend fuel and control component handling activities is FIRST given immediately upon entry into procedure 2202.002, in step 2, not in step 9.
You are correct.
Part 1 is not very discriminatory that CEA handling evolutions would be allowed to continue given it is the first operational directive given in the procedure. Id prefer that this question test on the log-taking requirements of step 1, Commence performance of Attachment D, SFP Level and Temperature Log. Test on whether logs must initially be taken every 30 minutes (correct) or hourly. This is a more RO-centric function for fuel handling anyway.
Changed Part 1 to frequency of monitoring level and temperature as requested.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
17 H
3 B
S NRC 2009 the following is stated twice in stem.
Corrected.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
18 H
3 M
S M-NRC 2009 19 H
3 N
S 20 H
3 B
S NRC 2012 21 H
3 X
M E
S M-NRC 2017 C part 2 is not credible, why would one go to SPTAs if reactor is not tripped?
ES-401 AN2-2021-03 5
Form ES-401-9
_Rev 3 Page 5 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only You are correct if the Reactor does not trip automatically.
Modify stem to,..is REQUIRED to be manually tripped, and then modify Part 1 as follows: A: Turbine only, B: Reactor only, C: Turbine then Reactor, D: Reactor then Turbine. You can get rid of part 2.
Rephrased this QID as requested above. Deleted Part 2 All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
22 F
2 B
S NRC 2015 23 H
3 N
S 24 F
2 X
M N
E S
M-NRC 2014 Part 2 not credible distractor the way the stem is phrased establishing feedwater flow to the S/Gs from Main Feedwater, considering that the RX is at 100% so MFW is already established. Replace with Auxiliary Feedwater, which is a plausible distractor because this is the preferred source during a normal shutdown.
Replaced Main Feedwater with Auxiliary Feedwater This is different enough from the parent question that it can be considered New.
Changed QID to NEW.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
25 H
4 N
S 26 H
3 B
S NRC 2000 Add Per 2203.020, High Activity in RCS: to stem.
Rephrased this QID as requested above. (Last bullet)
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
27 H
3 X
N E
S REFERENCE PROVIDED Replace The CRS will diagnose and enter with the crew should enter.
LOCA procedure should read 2202.003 vice 2203.
Rephrased this QID as requested above and corrected typos.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
28 H
3 X
N E
S Is it necessary for the stem to say MOST degradation? If there is only one seal degraded, seems like applicant should know that the seal w/ less than 700 psid DP is the one that is degraded.
Rephrased this QID as requested to say the 2P-32C seal that is degraded is the All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
ES-401 AN2-2021-03 6
Form ES-401-9
_Rev 3 Page 6 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 29 F
2 N
S 30 F
2 X
N E
S Get rid of the quicker/slower effect on RCS temperature portion of the question that is a 2x2 question in itself and not necessary to complicate it. Just test on suction alignment.
Rephrased the answer/distractors QID as requested.
In Revision 2, removed the words to mix with Letdown from distractors C and D Part 2 based on feedback from the NRC on 2/4/21 phone call.
NRC OK.
31 F
2 X
X B
E S
NRC 2009 MCGUIRE.
Testing on whether the radiation monitor is isolated or bypassed is minutia with the inlet closed, more flow should go through purification filter line, which is a form of bypassing.
Change to If the Letdown HX Outlet Temperature exceeds the setpoint of
____1_____, then flow through the Letdown (Ion Exchangers, ONLY vs Ion Exchangers AND Radiation Monitor) will be secured.
Added If the Letdown HX Outlet Temperature exceeds the setpoint of ____1_____,
then flow through the Letdown (Ion Exchangers, ONLY vs Ion Exchangers AND Radiation Monitor) will be BYPASSED using a three way valve. We used Bypassed instead of secured as this is the way it is discussed/trained on in the STM and to say Letdown flow is secured would imply that all Letdown flow is stopped but it is still flowing to the VCT bypassing the Ion exchangers via a three way valve to protect the resin. The Rad Monitor just has a normal MOV isolation valve.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
32 F
2 B
S NRC - 2005 33 H
3 X
B E
S NRC - 2014 Distractor A - What does a RAS have to do with risk of MTG runback? Basically, how are these two choices plausible together? Pick a different A(2) question doesnt have to be a perfect 2x2.
Changed Part A-2 and D-2 to more plausible distractors and made QID into a 2x4.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
34 H
3 X
X B
M?
E S
NRC - 2011 Is this Bank or Modified?
ES-401 AN2-2021-03 7
Form ES-401-9
_Rev 3 Page 7 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only BANK - deleted the U2 NRC Bank QID 1741 from the reference file. This QID has slightly different parameter from the original BANK QID but ask the same question and has the same answer as the original Bank QID The stem question is cluttered and confusing, being lumped into one sentence. Ask in two separate questions.
(1) Which ESF pumps are running?
(2) With no operator action, what will be the impact on these pumps on the A Red Train? (Pumps will automatically trip on low suction pressure vs.
Pumps will continue to run until damage occurs due to loss of suction).
Rephrased the stem and distractors QID as requested. Makes it even closer to the original BANK QID.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
35 H
3 B
S NRC - 2017 COMANCHE PEAK 36 H
3 N
S 37 H
3 B
S NRC - 2011 38 F
2 X
N E
S Put part (1) answers on 1 line - not sure why text wrapping is occurring.
Removed the text wrapping on Part 1.
Part (2), delete warm-up in stem question.
Deleted warm-up in stem question.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
39 F
2 X
M E
S M-NRC 2019 Missing the word TRIP and the end of part (2).
Added the word TRIP back in. It was there during validation.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
40 F
2 N
S Is there a procedural reference that documents RO responsibilities for correct answer, like it does for the other positions? Seems like the answer is by negation.
See EN-EP-801, Attachment 2, page 3 of 4 AND EN-OP-115 Section 4.0 Step 1 and
- 14. Updated the explanations and references on this QID In Revision 2, deleted the words and CBOT from the stem based on feedback from the NRC on 2/4/21 phone call.
NRC OK.
41 H
3 N
S
ES-401 AN2-2021-03 8
Form ES-401-9
_Rev 3 Page 8 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 42 H
3 N
S Should try to avoid using terms like would be in these questions should be using will and will be, etc. where possible Changed would to will in the stem. Searched exam for would and changed to will where applicable.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
43 F
2 X
B E
S NRC - 2015 Dont say MAXIMUM LEVEL because the setpoint is 6% +/- 0.5%. Say nominal setpoint.
Changed stem as requested.
Part 2 distractor is pretty weak, recommend just making this question part 1, with 4 different answers for level. Consider other distractor options:
Containment Building Sump level = 85% (hi level alarm)
RWT level = 40%
Or other setpoints.
Deleted part 2 of the QID and came up with four distractors for RWT level and updated explanations.
In Revision 2, changed the stem to read During a Large Break LOCA, the containment sump isolation valves and RWT outlet valves are interlocked to automatically swap the Containment Spray System (CSS) pump suction source at
__________ level. Added % RWT to A & B. Added inches Containment Flood to C & D based on feedback from the NRC on 2/4/21 phone call.
44 F
2 B
S NRC - 2011 45 H
3 X
N E
S Part 2 is weak. Is it realistic to do a 20% load reduction without inserting CEAs? Id rather expand part 1 out to 4 choices. Additional 2 distractors are 60% (power levels for taking a MFP offline on shutdown) and 70% (Main Feed STM says a single MFP is capable of supplying ~ 71% of total system flow while at full power.) Or another value that you think is plausible.
Changed part 2 to which method of boration will be used and made the power level distractors to 80/60% to be more related to the question and more plausible -
updated explanations. We stayed away from 4 power level choices (60, 70 80, or 90%) because of some conflicting information as to the capacity of a single feedwater pump. As a side note CEAs are always inserted for a normal down power for ASI control but in the case of a loss of a MFW pump, the CEAs are inserted a lot to rapidly reduce RCS temperature or we would trip on low SG level.
In Revision 2, changed the word MINIMUM to MAXIMUM in the Part 1 stem based on feedback from the NRC on 2/4/21 phone call.
ES-401 AN2-2021-03 9
Form ES-401-9
_Rev 3 Page 9 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only NRC OK 46 H
4 N
S 47 H
3 N
S 48 F
3 X
N S
Add IAW OP 1015.048, Shutdown Operations Protection Plan to stem question to tie it to a procedural requirement explicitly.
Added as requested.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
49 F
2 N
S 50 H
3 X
B S
NRC - 2004 will be not would be Changed would to will in the stem. Searched exam for would and changed to will where applicable.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
51 F
2 B
S NRC - 2011 52 H
3 B
S NRC - 2006 53 F
2 X
B E
S NRC - 2006 This can be counted as a Modifed question from 2006.
Changed to Modified and added old QID to this file.
Remove unnecessary information from stem: The Unit 2 Instrument Air Cross-Connect 2CV-3004 AOV will eventually fail Open on an extended loss of IA on Unit
- 2.
Removed unnecessary information from stem.
Simplify part 2 to : During an initial Loss of IA, 2CV-3004 is designed to remain Closed for a MINIMUM of ____(2)____ minutes to allow isolating Unit 1 from Unit 2.
Reworded part 2. Initially, the Loss of IA procedure has you verify 3004 open then if IA continues to lower, the procedure directs closing 3004 and then it will remain closed for at least 30 minutes with air from its accumulator even if IA is completely lost to prevent a loss IA on the opposite unit. This allows operators to close manual isolation should 3004 eventually fail open.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
54 F
2 B
S NRC - 2005
ES-401 AN2-2021-03 10 Form ES-401-9
_Rev 3 Page 10 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 55 H
3 X
X N
E S
Part 2 is also 4 different answers, so you dont even need to know anything about Part 1 to answer if you know that TS 3.6.1.3 is still applicable. Ask which TS are applicable, and if defeating the interlock is / is not allowed.
Reworded as requested. QID is now a true 2 x 2.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
56 H
3 N
S 57 H
3 X
N E
S How does applicant know that RVLS Level One is NOT WET? Stem gives levels 3 thru 11, but not 1 or 2 unclear why applicant is expected to assume it is not wet.
Added a bullet to state RVLMS Levels 1 and 2 indicate DRY.
In Revision 2, Added a picture of the ANO-2 SPDS Safety Function Display to this QID and deleted all but three bullets from the Revision 1 version of this QID based on feedback from the NRC on 2/4/21 phone call. The NRC reviewed Rev 2 of this QID on 2-8-21 and accepted the changes.
NRC OK.
58 H
3 B
S NRC 2008 59 F
2 B
S NRC 2010 - Byron 60 F
2 N
S 61 F
2 N
S Dont use would have, use will Changed would to will in the stem. Searched exam for would and changed to will where applicable.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
62 F
2 X
N E
S Part 2 70% Narrow Range is not plausible. Even part 1 gives away that the highest level EFW might be controlling level at is 62% (which is wrong in itself). So its not logical that OTC would be established when SG drops below a value that is higher than the control range.
Changed part 2 to 70 and 160 inches Wide range. 70 inches Wide range is the correct answer. 160 inches Wide range is when you can terminate OTC. Updated explanations for plausibility. 60% Narrow range is our post trip or low power SG level setpoint that is maintained normally by MFW in automatic. SG level setpoint shifts to 70% when power rises above 25% as detected by Turbine first stage pressure.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
63 F
2 N
S
ES-401 AN2-2021-03 11 Form ES-401-9
_Rev 3 Page 11 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 64 H
3 B
S NRC - 2009 will be not would be.
Changed would to will in the stem. Searched exam for would and changed to will where applicable.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
65 H
3 N
S Add to explanations that Loss of MFW AOP does have steps to start standby condensate pump, but would not yet be entered.
Added this to the explanations for Part 2 of distractors B and D.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
66 F
2 B
S NRC - 2014 67 F
2 B
S NRC - 2014-2 68 F
2 X
B E
S NRC - 2015 ODMI might be more plausible than ACMP, and is relevant to ANO as a post-95003 focus area for improvement.
Changed Part 1 of distractors C and D to ODMI as requested.
Twice a shift doesnt seem particularly plausible unless there are other communications that are required to be reviewed at that frequency. Consider First day of new shift.
Added on the first day of a new shift ONLY to Part 2 of distractors A and C.
In Revision 2, changed the word as to at in the Part 2 stem AND deleted the word ONLY from distractor A Part 2 and C Part 2 based on feedback from the NRC on 2/4/21 phone call.
NRC OK.
69 F
2 X
X N
E S
In order to avoid any concern that this question requires systems knowledge, specify in stem The safety-related 2P4A Service Water Pump Revised as requested.
Part 2 has 4 different answers, making Part 1 unnecessary. Just go with part 2.
Deleted Part 1 All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
70 F
2 X
X?
N E
S Specify the procedural reference in the stem, i.e., per procedure xxxx.
Added as requested.
Would the control room not make a narrative log entry in additions to the Component Deviation?
ES-401 AN2-2021-03 12 Form ES-401-9
_Rev 3 Page 12 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Potentially but not required. Reworded Stem to say IAW EN-OP-115-07, Configuration Control, due to the planned deviation of this valve, at a MINIMUM a(n)
____(1)____ will be generated to track the configuration of this valve.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
NRC 2-12: Including the alternate phrase (Configuration Control) in the correct answer D looks like cueing, since the reference procedure only uses the term component deviation tagout and the name of the procedure itself is Configuration Control.
In Revision 2 (2-16-21) Removed the words Configuration Control from the stem that renames the procedure number. This phrase in answer D was added based on validation feedback that this tagout is commonly referred to as a Component Deviation Configuration Control Tagout at ANO instead of just Component Deviation Tagout. The Configuration Control Module in the Clearance eSoms program is used IAW EN-FAP-OP-024 Section 3.0 step [3] to generate this record.
NRC OK.
71 F
2 B
S NRC - 2020 (ANO Unit 1 Exam) 72 H
3 X
M E
S M - NRC 2011 plausible as this was the correct answer to the original question is not an adequate justification for plausibility of a distractor, and is not something that should be taken into consideration. What is plausible about 26x300=7800mrem additional dose?
Revised the QID to more plausible distractors with the explanations updated.
In Revision 2, added the words the total cumulative amount of time to both the Part 1 and Part 2 stems based on feedback from the NRC on 2/4/21 phone call.
NRC OK.
73 F
2 X
X X
B E
S NRC - 2016 Distractor A: I dont like this, as it is a true statement, regardless of whether it is specifically located in EN-RP-101 or some other procedure.
Changed to Be a Category 3 Rad Worker and updated explanation.
Distractor B: There should be something in the stem that would allow the applicant to discern if the VCT room is a contaminated, highly contaminated, or neither, space.
Added new condition for room contamination levels (Below dress out requirements) and updated explanation.
ES-401 AN2-2021-03 13 Form ES-401-9
_Rev 3 Page 13 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Stem: I dont think you need to tell the applicant that the space is a LHRA, if you give them the dose rate for the space and tell them it is properly posted. Applicant should be able to discern from that information that it is a LHRA.
Revised as requested.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
74 F
2 N
S 75 F
2 X
B E
S NRC - 2015 Part 2 is systems level knowledge about the power supplies for PAMS instrumentation, not generic. Replace part 2, or expand out part 1 with two new distractors.
Deleted Part 2 and added two more plausible distractors.
In Revision 2, changed this QID to a NEW QID from a BANK DID based on feedback from the NRC on 2/4/21 phone call.
76 H
3 M
S Pre-Review m-nrc 2014 77 H
4 N
S Pre-Review 78 F
2 X
N E
S REFERENCE PROVIDED If the intent is to provide all of the EAL wall charts, need to be extremely sensitive that all the additional information in these charts is not going to provide information that could help on other questions on the RO and SRO exam. Evaluate if a targeted subset of the charts can be provided, without cueing the answer. Or redact values and data that is clearly unnecessary for the EAL questions that are being asked (values for tube leakage, etc)
Decision has been made to use selected sections out of the EAL Classification Procedure OP-1901.010 instead of the EAL Wall Charts.
Part 1 distractor: Is the explanation saying that there IS NO DSS TRM? If thats the case then applicant doesnt need to know what modes DSS is required in, if he/she knows that there is no DSS TRM. Eliminate the procedural reference in each of the answers, and add to stem, per OP-2105.017, DSS AND DEFAS Operations Revised stem as requested.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
79 H
3 X
X M
U S
M-NRC 2017
ES-401 AN2-2021-03 14 Form ES-401-9
_Rev 3 Page 14 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Part 1, why are Feed and Bleed or Ambient Cooling not options? These are both higher in order of priority than Steaming to Atmosphere. I dont think theres a correct answer in this question as written.
Revised QID as discussed in our call on 2/03/2021.
Make part 1 question in stem more straightforward language, Per 2202.004, SGTR, A S/G cooldown should be accomplished using: _______
Revised stem as requested.
In Revision 3, changed the word can to should in the part 1 stem based on feedback from the NRC on 2/4/21 phone call.
80 H
3 X
B E
S NRC 2014-2 Question is not actually testing on SRO knowledge because the stem already states that the CRS must contact ERO for permission, it is just testing on the required plant manipulation response in the procedure, RO knowledge. Testing on required approval authority for the required action would be an SRO function.
Revised QID as discussed in our call on 2/03/2021.
All Rev. 2 changes accepted by the NRC on 2/4/21 phone call.
81 H
4 N
S Pre-Review 82 H
3 X
N E
S Replace According to the mitigation strategy with Per in stem.
I think I updated as requested. Please look.
In Revision 2, changed the words In accordance with the CEA Malfunction AOP, OP-2203.003 to Per OP-2203.003, CEA Malfunction AOP in the part 1 stem based on feedback from the NRC on 2/4/21 phone call.
The language of Attachment A is pretty tricky, regarding what the required minimum power reduction is. Consider for procedural enhancement after the exam.
The team looked at the attachment and need some more clarification from you as to what part is tricky.
83 H
3 N
S Pre-Review 84 H
3 X
N S
REFERENCE PROVIDED Similar comment to 78, If the intent is to provide all of the EAL wall charts, need to be extremely sensitive that all the additional information in these charts is not going to provide information that could help on other questions on the RO and SRO exam.
Evaluate if a targeted subset of the charts can be provided, without cueing the answer.
Decision has been made to use selected sections out of the EAL Classification Procedure OP-1901.010 instead of the EAL Wall Charts.
ES-401 AN2-2021-03 15 Form ES-401-9
_Rev 3 Page 15 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 85 H
3 N
E S
Rephrase the correct answer to MUST be placed..
A part 2) doesnt really make sense, Continue lowering RCB pressure, as the stem states that RCB pressure has NOT been lowering. Is there an alternate method of lowering RCB pressure that can be used as a distractor? Can probably use B part
- 2.
Used B Part 2 for A part 2 also.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
86 H
3 X
X N
U E
S Pre-Review REFERENCE PROVIDED This question is very clunky; reads like triage was performed on it to try to justify it as SRO-only. A and B part 1 drop you right into the middle of an AOP, and C and D part 1 drop you into subsections of an FRP.
Effectively correct answer part 1 is expecting the applicant to have memorized which substep of AOP 2203.029 step 7 (substep 7.G) directs checking LPSI header discharge flow within 500 gpm of setpoint, and to know that is the step which if not met redirects you to step 13. That is kind of minutia, not particularly valuable as a discriminating tool, and not really SRO anyway.
This question would be cleaner with explicit diagnosis of the malfunction, operability assessment and TS application.
New question development in progress NRC: Stem: Should say 25 FEET vice INCHES? 19 FEET vice inches?
NO inches is correct. We base our level on the bottom of the RCS Hot leg being Zero (0) inches. The Hot Leg is 48 inches in diameter and the suction source to the SDC pumps. 24 inches which is the middle of the hot leg is the minimum SDC level to prevent vortexing and air entrainment to the SDC pumps suctions. At 19 inches above the bottom of the hot leg would cause the pump fluctuations as stated in the stem. Added Reference OP-2103.011, Draining the RCS, Attachment B and Exhibit 1 to QID #86 to show this.
-Rephrase Part 1 question to: A basis for the minimum required SDC flow rate with one SDC loop in operation is:____
Rephrased as requested.
-Place Part 1 question after the first part of stem.
Placed Part 1 question after the first part of the stem before the word NOW
ES-401 AN2-2021-03 16 Form ES-401-9
_Rev 3 Page 16 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only
-Rephrase, Under these conditions T.S. 3.9.8.1 SDC Loop in Operation T.S is no longer met to SR 4.9.8.1 minimum SDC flow acceptance criteria is no longer met.
Rephrased as requested.
NRC ok 87 F
3 B
S NRC - 2009 88 H
3 X
N E
S Pre-Review Clarify in stem, or else a requirement to be in MODE 3 within a specific timeframe takes effect.
Revised as requested.
Add in stem, Operable (or Functional) status Added as requested.
Answer appears to have changed since pre-review.
After your review, question was revised based on pre-submittal validation feedback.
All Rev. 2 changes accepted by the NRC on 2/4/21 phone call.
89 H
3 N
S Pre-Review 90 H
3 N
S Pre-Review 91 H
3 X
X N
E S
Pre-Review
-Distractor B part 2 doesnt ring credible to place the plant in HOT SHUTDOWN within the FOLLOWING 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> when the plant is currently at power and no discussion of being in HOT STANDBY has been made. Revise to a different, credible time requirement to be in HOT STANDBY.
Revised B part 2 to a HOT STANDBY requirement.
-Revise stem to, Based on these conditions, which of the following satisfy technical specification required actions? Restore two CEA position indicator channels for CEA #25 to an OPERABLE status within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> OR use (1) to place (2) I like this phrasing better because it doesnt quite imply that the correct answer is the ONLY way to satisfy tech specs, its just the only correct way listed.
Revised as requested.
-If Im reading explanation correctly, am I understanding that even if crew elected to use TS 3.1.3.2 Action b to shutdown, procedure OP 2102.004 Section 12 would NOT be the correct procedure to use?
ES-401 AN2-2021-03 17 Form ES-401-9
_Rev 3 Page 17 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only Discussed with team. This would be the section that will be used for a 6 hour6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> emergent shutdown like a TS 6 hour6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> action. The Rapid Plant Shutdown AOP is designed for a less than one hour shutdown or a shutdown without Letdown flow.
The normal plant shutdown section in the power ops procedure is used for going into a refueling outage.
In Revision 3, added the words per the reactivity plan to distractor B Part 2 based on feedback from the NRC on 2/4/21 phone call 92 H
3 M
S REFERENCE PROVIDED M-NRC 2012
-Similar comment to 78, If the intent is to provide all of the EAL wall charts, need to be extremely sensitive that all the additional information in these charts is not going to provide information that could help on other questions on the RO and SRO exam.
Evaluate if a targeted subset of the charts can be provided, without cueing the answer. Or redact values and data that is clearly unnecessary for the EAL questions that are being asked (values for tube leakage, CET temps, etc)
Decision has been made to use selected sections out of the EAL Classification Procedure OP-1901.010 instead of the EAL Wall Charts.
-How much time are the validators spending on this question?
Not much more than any other questions. This question is based on new guidance added during the last revision to our EOPs. As long as actions are being taken to recover a lost safety function and the safety function can be recovered within 15 to 30 minutes, then with SM approval, the optimum EOP can remain in effect without going to the FRP.
93 F
2 X
M E
S Pre-Review Why would applicant NOT think that adequate Boron concentration exists? This question would be more discriminatory if the boron concentration in the SFP was something less than 2500ppm but more than 2000. Establish initial conditions such that SFP concentration only was being considered, Test only on TS 3.9.12 without consideration for 3.9.1.
Changed the conditions in the stem to loading New Fuel assemblies and shuffling spent fuel assemblies to eliminate the 3.9.1 spec boron requirements. Lowered SFP boron concentration to less than 2500 ppm boron in the stem.
All Rev. 2 changes accepted by the NRC on 2/4/21 phone call.
94 F
2 N
S Answer says Tave 250°F, but question choice says Tave > 250°F.
Verified Tave > 250°F is correct and corrected the answer.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
ES-401 AN2-2021-03 18 Form ES-401-9
_Rev 3 Page 18 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only 95 F
2 B
S NRC - 2010 For distractor A, if I understand 2502.001 correctly, even with personnel hatch area rad monitor RITS-8909 INOPERABLE, core offload can commence if one of the other 2 ARMs are available, regardless of portable rad monitors, is that correct?
This is true. Updated the explanation for distractor A.
Is a portable rad monitor normally placed on the refueling machine?
Yes and RP monitors them quite often along with the SRO in charge of Refueling.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
96 F
2 B
S NRC - 2020 (ANO Unit 1) 97 F
H 3
X N
E S
Clarify in stem, B EDG is OOS for a planned 14 DAY Extended AOT, and all contingencies are met.
Revised the second bullet as requested above.
This question has 4 distinct answers for both parts - unnecessary, limit this to Part 1 only, which is satisfactory as an SRO-only question.
Removed part 2 from the QID as requested.
Higher cognitive level b/c applicant has to discern that 2P7A is protected by virtue of EDG AOT.
Changed the taxonomy to H or higher cognitive QID.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
98 H
4 X
B E
S NRC - 2014-2 This would be much more straightforward if stem simply gave a single value for I-131 on 4/19 0200, and asked If the condition is not corrected, what is the latest time by which the reactor must be in MODE 3? Answers are 4/21 0800 (correct), 4/21 0200 (T+48), 4/19 0800 (T+6, which would be true if I-131> 60 uCi/g), and 4/20 0800 (T+30). I think it implicitly meets the K/A since these actions are based on controlling releases. But if you want to incorporate the release portion of the original question, thats fine.
Revised as requested and changed to a NEW question. Also kept the last release portion of the QID but changed to LOCA/SGTR and removed Main Steam Line Break as the Main Steam Line break and SGTR are listed in the basis as the accident that could cause an Offsite dose release with high activity, All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
99 F
2 N
S 100 H
3 X
N E
Stem: replace would be to with is to.
ES-401 AN2-2021-03 19 Form ES-401-9
_Rev 3 Page 19 of 20 OBDI 202 - IOLE Process Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other
- 6.
B/M/N
- 7.
U/E/S
- 8.
Explanation Stem Focus Cues T/F Cred.
Dist.
Partial Job-Link Minutia
- /
units Back-ward Q=
K/A SRO Only S
Replaced as requested.
All Rev. 1 changes accepted by the NRC on 2/4/21 phone call.
GENERAL COMMENT
S
- 1.
Last 2 NRC Exams: 2019, 2017
- 2.
There is seemingly unnecessary text wrapping going on throughout the exam.
- 3.
Identify on each handout what question it is intended for.
There are _2___ RO questions with handouts provided and __5__ SRO questions with handouts provided.
Results Table RO LOK -H 40 0.533 Avg RO LOD 2.56 Flaws 10 CFR Distribution RO LOK-F 35 0.467 AVG SRO LOD 2.88 Stem focus 24 41.1 0
43.1 2 SRO LOK - H 18 0.72 Overall LOD 2.64 Cues 4
41.2 0
43.2 5 SRO LOK - F 7
0.28 T/F 0
41.3 1
43.3 1 Cred Dist 19 41.4 1
43.4 3 RO Bank 26 34.67 SRO Bank 3
12 Partial 3
41.5 10 43.5 13 RO Mod 10 13.33 SRO Mod 5
20 job link 0
41.6 2
43.6 0 RO New 39 52 SRO New 17 68 units 0
41.7 32 43.7 1 minutia 0
41.8 2
Total Bank 29 29 backward 0
41.9 1
Total Mod 15 15 KA 1
41.10 21 Total New 56 56 SRO-only 3
41.11 1
LOD = 1 0
41.12 2
ES-401 AN2-2021-03 20 Form ES-401-9
_Rev 3 Page 20 of 20 OBDI 202 - IOLE Process RO Sat 47 62.67 SRO Sat 13 52 41.13 1
RO Unsat 0
0 SRO Unsat 2
8 41.14 1
RO Edit 28 37.33 SRO Edit 10 40 Answer Dist (%)
RO-A 17 SRO-A 4
Total Sat 60 60 Total Unsat 2
2 RO-B 21 SRO-B 6
Total Edit 38 38 RO-C 17 SRO-C 8
RO-D 20 SRO-D 7