ML23012A103
| ML23012A103 | |
| Person / Time | |
|---|---|
| Site: | Monticello  |
| Issue date: | 11/10/2022 |
| From: | Nuclear Management Co |
| To: | NRC/RGN-III/DORS/OB |
| Iskierka-Boggs T | |
| Shared Package | |
| ML21188A293 | List: |
| References | |
| Download: ML23012A103 (1) | |
Text
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 1
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation General Comments:
NOTE 1: For Tier 1 questions, does it test:
(1) information from the AOPs/EOPs; (2) diagnosis of procedures to be used to address the situation; (3) the progression of an event; OR (4) the assessment of integrated plant response to emergency or abnormal situations crossing several plant systems and/or safety functions?
MNGP NOTE 1: From NUREG 1021 - Several ways of achieving this objective include, but are not limited to: and under (1) AOPs, EOPs and their associated bases documents:
NOTE 2: For Tier 3 questions, does it test: plant-wide generic information - and not simply become an extension of Tier 2, plant systems?
NOTE 3: For SRO-only questions, select K/As tied to 10 CFR 55.43.
1 F
2 N
S NRC: Last sentence in justification should be
..setpoints to be higher than See NOTE 1 - Tie to (1).
Question is SAT as written.
MNGP: Edits made to last sentence in justification.
NRC: Question is SAT 2
F 3
B E
S Bank - not used on previous exam See NOTE 1 - Tie to (3).
NRC:- References do not point to where the 3 hour3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> time is associated. Can this be provided, or find another time that is covered in reference material as distractor.
MNGP: Updated justification for distracter C.
Explanation can be found on pages 7, 24 and 25 of cited reference C.4-B.09.02.A. Question SAT with enhancement.
NRC: Question is SAT 3
F 3
B S
Bank - not used on previous exam See NOTE 1 - Tie to (2).
NRC: Question is SAT as written.
MNGP: Question is F as it is a power supply question.
4 H
3 B
E S
Bank: 2010 NRC Exam See NOTE 1 - Weak tie to (3).
NRC: Weak tie to Tier 1 question. Question asks fundamental system knowledge tied to Reactor Pressure response during MG trip and Reactor Scram MNGP: Question is directly tied to the performance of
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 2
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation steps 27 and 28 of AOP C.4-A. Additionally, the current turbine speed makes them analyze the progression of the event and understand weather the Aux oil pump would be running or not. Question SAT.
NRC: Agree with justification. Question is SAT 5
F 2
X X
B U
S Bank - not used on previous exam See NOTE 1 - Weak tie to (1).
NRC: Weak tie to Tier 1.
- Seems the correct rationale for verifying RR pump speed is for ensure reactor remains shutdown.
MNGP: AOP procedure entry and AOP action/verification taken.
- Justification appears to give the reason/purpose for the interlock during startup based on referenced system material and not the reason for why operator is checking for the runback after a scram in C.4-A. No information in C.4-A basis states that the runback is checked to ensure NPSH. Does not appear there is a correct answer for the question asked.
MNGP: Enhanced stem, provided detailed explanation of FW/Recirc Flow interlock in the justification and added Annunciator reference. Question SAT with enhancement per Chief Examiner.
NRC: Agree with justification. Question is SAT.
6 F
3 B
S Bank - not used on previous exam See NOTE 1 - Tie to (3).
NRC: Question is SAT as written.
7 F
3 B
E S
Bank: 2020 Exam Previous 2 exams See NOTE 1 - Tie to (3).
NRC:- Change ? mark to a period. Initial stem sentence is a statement.
- Distractor analysis for D should reflect others stating Plausible RBCCW cooled component. And follow on statement for the NRHX should state that the RWCU system will isolate at 140F.
MNGP: Edits made as described above. Question SAT with enhancement.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 3
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation NRC: Question is SAT 8
H 2
X B
E S
Bank - not used on previous exam See NOTE 1 - Tie to (1).
NRC: Distractors have underlining of key words.
- Correct answer has two words underlined leading towards importance of that distractor cuing applicants.
Recommend removing the Underlines, to preclude potential cueing.
MNGP: Removed underlines. Question SAT with enhancement.
NRC: Question is SAT 9
H 3
B S
Bank: 2015 NRC Exam See NOTE 1 - Tie to (2).
NRC: Question is SAT as written.
10 F
3 X
X B
E S
Bank - not used on previous exam See NOTE 1 - Although this question does match the K/A, it does not have a tie to Tier 1.
NRC: Reference provides orientation based on control rod, not control cell. Are both used interchangeably?
-The annotated correct answer [A] and the answer justification appear to be contradictory, and therefore there does not appear to be a correct answer.
A second check to assure proper orientation is that the lugs on the bail all point toward the control rod.
- Distractor A correct answer is shortest and not worded similarly to other distractors. Should it state to the center of the control cell (rod - if this is correct verbiage)
- Distractor D is not formatted as others. Suggest all caps for LATCHED, and lowercase and remove from parenthesis, no upward rod motion has occurred.
Excessive use of all caps cues that this is incorrect selection.
MNGP: Control rod and center of Control Cell can be used interchangeably as the control rod is the center of the control cell. A, B and C reworded to match
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 4
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation procedure and equalize lengths. Justification was clarified for correct answer. Distracter D reworded as suggested above. Question SAT with Enhancement.
NRC: Question is SAT 11 H
3 B
S Bank: 2015 NRC Exam See NOTE 1 - Tie to (4).
NRC: Question is SAT as written.
12 F
3 B
S Bank: 2009 NRC Exam See NOTE 1 - Tie to (1).
NRC: Question is SAT as written.
13 H
3 N
E S
Bank: Modified from 2018 NRC Exam - R18; See NOTE 1 - Tie to (3).
In order to meet significant modification requirements, reorder the distractors in reverse order with the latest time being choice A such that the correct answer becomes B.
Image embedded, and Reference Provided: Heat Capacity Limit Graph MNGP: Reordered answer options. Question SAT with enhancement.
NRC: Question is SAT 14 H
3 B
S Bank: 2020 NRC Exam - Previous 2 exams See NOTE 1 - Tie to (1).
NRC: Question is SAT as written.
15 F
2 N
E S
See NOTE 1 - Very Weak tie to (3).
This is a weak tie to the K/A as the question does not directly permit monitoring or ask any operational item.
May need to revise the Stem question.
- Distractor D is outlier from other three which are all 0.5 feet from the previous. Understood that -5.9 feet is good plausible distractor. Is there another plausible distractor that is closer to the -6.0 feet number that could replace C?
MNGP: Reworded stem to better match K/A. There isnt another level between -4.5 and -6.0 feet that would be plausible. Question SAT with enhancement.
NRC: Question is SAT
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 5
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation 16 H
3 B
S Bank - not used on previous exam See NOTE 1 - Weak tie to (3).
NRC: Question is SAT as written.
17 H
3 B
S Bank: 2010 NRC Exam See NOTE 1 - Tie to (1).
NRC: Question is SAT as written.
18 F
2 M
E S
Bank: 2015 NRC Exam See NOTE 1 - Tie to (4).
- Add time element to third bullet in stem that indicates time since initiating condition following rapid downpower is less than 30 minutes to ensure distractor D is not a possibility of being correct.
- Perhaps need to provide trend information on MSL Rad Monitor levels to ensure only one correct answer MNGP: Added timeline to stem. Question SAT with Enhancement.
NRC: Question is SAT 19 F
3 X
N E
S Early Submit Item Does Not Meet Tier 1 Requirements. Does not tie to emergency or abnormal condition. (Or any of the Tier 1 objectives.)
See NOTE 1 - Although this question does match the K/A, it does not have a tie to Tier 1.
NRC: Suggest making stem provide information to a location of a small fire reported on system that has lube oil, or in a hazardous waste trash bin in an area used for wiping up oil on rounds, etc.
MNGP: Revised stem to meet Tier 1 requirements.
Randy - I still do not see the changes in the Stem to meet Tier 1 Suggest finding a way to give a scenario in stem in which there is a fire in the room provided and that the crew is responding with C.4-B.08.05.A, Plant Fire, and the Fire Brigade Leader calls to the control room to determine which suppression agent would be best for this situation, or something along these lines.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 6
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation MNGP: Stem reworded. Question SAT with enhancement.
NRC: Weak tie to Tier 1 - Question is SAT 20 F
3 M
S Modified from 2013 NRC Exam #67.
See NOTE 1 - Weak tie to (3).
NRC: Question is SAT as written.
21 H
2 X
N E
S Early Submit Item NRC: Suggest adding more information in the stem that places the plant in the condition such as, The plant has experienced a High RPV Water Level Trip. Then move on to the two questions.
See NOTE 1 - Although this question does match the K/A, it does not have a tie to Tier 1.
May also want to state that the plant has been operating at rated power (or in Mode1) to preclude any assumptions by the applicants and tie to a Tier 1 objective.
MNGP: Revised wording of stem for tier 1 requirements and added Mode 1.
Randy - I still do not see the changes in the Stem to meet Tier 1.
Suggestion is to add a procedure that the crew would be responding using and ask about the basis of a step that would prevent the moisture carryover condition that is protecting the turbine blading.
MNGP: Reworded stem. Question SAT with enhancement.
NRC: Question is SAT 22 H
3 B
E S
Bank: 2010 NRC Exam. Image embedded.
See NOTE 1 - Very weak tie to (3).
NRC: Remove your from stem. Necessary to state lights are extinguished? It is pretty apparent MNGP: Removed your. Stating the light is necessary because a candidate will likely ask during the exam.
Question SAT with enhancement.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 7
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation NRC: Question is SAT 23 H
3 B
S Bank: 2020 NRC Exam. Previous 2 Exams See NOTE 1 - Tie to (3).
NRC: Question is SAT as written.
24 F
3 B
E S
Bank: 2009 NRC Exam #46. Pedigree has question listed as Bank.
See NOTE 1 - Tie to (4).
NRC: Distractor B - capitalize Room for Plenum Room for consistency MNGP: Updated answer options. Question SAT with enhancement.
NRC: Question is SAT 25 H
4 B
S Bank: 2015 NRC Exam See NOTE 1 - Tie to (3).
The Bolded words appear to be part of a Paste operation from previous use. Recommend un-bolding all words in the question for exam consistency.
Question is SAT as written.
MNGP: Unbolded all words in answer options.
NRC: Question is SAT 26 H
3 B
E S
Bank - not used on previous exam See NOTE 1 - Tie to (3).
NRC: - Justification provided for A, C, and D in one paragraph. Break out for each distractor.
- Enhance distractor analysis by placing more information to why Condenser Hotside is not correct.
Unclear based on justification information provided.
MNGP: Updated distracter analysis. Question SAT with enhancement.
NRC: Question is SAT 27 H
4 X
B U
S Bank: 2013 NRC Exam. Picture embedded as reference.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 8
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation NRC: This is not a tie to the selected K/A since the question has nothing to do with CCW. Recommend replacing the question or the K/A.
- Provide information in Distractor D analysis that current LPCI Initiation signal pressure is now 420 psig after explanation that it used to be 460 psig.
- Distractor D does not follow formatting for other three when using SHOULD which cues applicants away from this distractor.
If replacing the K/A with randomly selected A1.01, and trying to use this question, recommend using question as written from the 2013 exam and leave the RPV level Initial Condition as previously given.
MNGP: Reworded stem and choices to minimize cueing. K/A 203000 A1.01 randomly reselected by Chief Examiner. Question UNSAT with enhancement.
NRC: Question is SAT 28 H
3 X
B U
S Bank - not used on previous exam NRC: - Question has very weak K/A tie for prioritizing safety functions following trip of 14 RHR pump when asking if TS allow fuel moves to continue. Only mention of prioritization is in the justification and not in the stem or distractors. If question is used, change stem question to read why or why not?
MNGP: Reworded stem and enhanced justification to better understand the K/A match as requested by the Chief Examiner. Question SAT with enhancement.
NRC: Question is SAT 29 H
3 B
E S
Bank - not used on previous exam NRC: Change question in stem to: Which of the following flowrates approximates MNGP: Updated stem. Question SAT with enhancement.
NRC: Question is SAT
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 9
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation 30 H
3 B
S Bank - not used on previous exam NRC: Question is SAT as written 31 F
3 X
B E
S Bank - not used on previous exam NRC: - Change question stem to something similar to, Choose the selection that best describes if MO-1753 can/cannot be used to throttle Core Spray flow.
- Distractor D states HS 14A-S1A for MO-1753 prior to nomenclature which cues that this selection applies to the valve in stem question; whereas Distractor C has HS 14A-S16A and then directly states nomenclature.
- Plausibility of Distractor B due to not giving reasonable rationale for why not to throttle MO-1753 MNGP: Suggested stem change seems too wordy.
However, stem and answer options have been simplified and B justification clarified. Question SAT with enhancement.
NRC: Question is SAT 32 H
2 X
N U
S Bank: 2016 NRC Exam NRC: KA pertains to SBLC system and pump trips, question is asking about RWCU pump trips which is a weak tie to KA. Are there any automatic SBLC pump trips in the system?
Do not believe there is an auto trip for an SBLC pump other than possibly on OC. May need a replacement K/A for this question.
MNGP: there is no automatic SBLC pump trip.
Discussed with Chief Examiner and believe the Pump Trip in the K/A actually does pertain to RWCU Pumps. Question SAT as written.
NRC: Question is SAT.
33 F
4 B
S Bank - not used on previous exam Question is SAT as written MNGP: Question is low COG. Interlock only.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 10
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation NRC: Question is SAT 34 H
3 B
S Bank - not used on previous exam NRC: Question is SAT as written 35 H
3 X
B U
S Bank - not on previous exams NRC: - Distractor A is only one that offers not in alarm for SRM HI/INOP.
- Distractor B is only one that offers not in alarm for ROD WITHDRAW BLOCK
- Using both of these distractors one can eliminate A and B without any system knowledge.
- Is it plausible that these may be in alarm for these situations in order to establish distractors in a more psychometrically sound way? However, adding this in with three variables will result in cuing to the correct answer based on which way these are incorporated.
- Suggest having only two alarms being tested for this question. Testing ROD WITHDRAW BLOCK and Scram logic may be good option.
- Breakout the justification for distractor analysis for each distractor.
Recommend getting rid of the 3-part answering.
MNGP: Revised stem and choices. Question is UNSAT with enhancement.
NRC: Question is SAT 36 H
3 B
S Bank: 2016 NRC Exam NRC: Question is SAT as written.
37 H
2 B
E S
Bank - not used on previous exams NRC:- First sentence of stem contains a list of information that would be better broken down into bullets after A plant transient occurred resulting in:
- Typically, RCIC turbines will trip on overspeed while starting up, conditions in the stem state the RCIC system was operating to restore RPV level and then
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 11
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation the mechanical overspeed trip occurred. Is there OE with this, or some condition that may cause this other than startup such as air binding, cavitation that occurs with this type of turbine?
Perhaps need to clarify the Stem Initial Conditions to give a situation which causes the overspeed.
MNGP: Simplified stem to address concerns above.
Question SAT with enhancement.
NRC: Question is SAT 38 F
2 N
S NRC: Question is SAT as written.
39 H
3 B
S Bank: 2010 NRC Exam NRC: Question is SAT as written.
40 H
2 M
E S
Modified from 2018 NRC Exam Q#38. Reference Provided: P&ID NRC:- Not sure why this P&ID is necessary to be provided to the applicants. Are they not taught that the system has rupture discs and to where they relieve to? Perhaps remove the reference.
- How is Distractor A plausible? It is significantly different from B, C, and D in wording.
- Modification done by swapping RCIC for HPCI.
Same outcome and answer as 2018 exam.
- Stem of the question has typo: experiences a stem failure stem should be steam?
MNGP: P&ID removed as reference and removed 4 line reference form Distracter D. Distracter A must be written this way for plausibility. Plausible if examinee only thinks RCIC has rupture discs. Question SAT with enhancement.
NRC: Question is SAT.
41 H
3 B
E S
Bank: 2010 NRC Exam. Picture embedded as Reference NRC: - If using the image, may need to resize similar to 2010 exam to read easier by applicants.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 12
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation
- Distractor D should be formatted similar to others.
Change CANNOT to will NOT operate manually. Do NOT BOLD not in distractor.
MNGP: resized pic and reworded distracter D.
Question SAT with enhancement.
NRC: Question is SAT.
42 H
4 B
S Bank: 2018 NRC Exam. Previous 2 Exams NRC: Question is SAT as written.
43 H
3 B
S Bank - not on previous exams NRC: Question is SAT as written.
44 H
2 X
N E
S Early Submit Item Picture embedded as Reference NRC: Question the validity of distractor D as there is not a power source available from a DG similar to bus
- 13. Need to focus the Stem by stating procedurally reenergized.
MNGP: Added procedurally to the stem. Kept distracter D as is. Plausible option as 14 and 16 bus could be reenergized from 13 DG by back feeding via LC-108 but there is no procedure for it and the crosstie is not rated for both busses. Updated and clarified justification of distracters B and D.
NRC: Changes made are SAT.
45 H
3 B
S Bank: 2016 NRC Exam NRC: Question is SAT as written.
46 H
3 B
E S
Bank: 2015 NRC Exam. Picture embedded as Reference.
NRC:- Change stem/question into two separate portions. Suggest changing stem to something similar to An UNDERVOLTAGE fault results inPoint X above.
- Then add question, Which of the following on a new line below stem.
- Add Y-94 nomenclature to the stem statement
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 13
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation sentence.
If stem is not reworded, then Distractor B is not plausible and should be replaced.
MNGP: Stem modified as described above. Question SAT with enhancement.
NRC: Question is SAT 47 F
3 B
S Bank: 2015 NRC Exam NRC: Question is SAT as written.
48 F
2 B
S Bank - not used on previous exams NRC: Question is SAT as written.
49 H
3 B
S Bank - not used on previous exams NRC: Question is SAT as written.
50 H
3 X
N E
S NRC: - Distractor B does not need to test if CV-2403 fails as-is. Having it do so will make the 2x2 question fall into an odd format and could lead to cuing.
Suggest having CV-2403 indicate open for this distractor. CV-6-13 fails as-is and that concept is tested for this valve.
MNGP: Above changes made. Question SAT with enhancement.
NRC: Question is SAT 51 H
3 B
E S
Bank: 2015 NRC Exam NRC: - Add fourth bullet stating annunciator 4-B-26 received in control room and remove from question.
- Rephrase question to something similar to, Which of the following actions must be taken in accordance with C.6-004-B-26, CLEANUP DEMIN TEMP HI?
MNGP: Alarm 4-B-26 is placed in the question to separate it from the initial conditions and prevent needing a timeline. Question SAT as written.
NRC: Question is SAT 52 H
3 N
E S
Reference P&ID included for this question.
NRC: - Do not see need to supply P&IDs for this question. Are applicants not required to know which
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 14
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation HXs are supplied by ESW, and which systems have radiation monitoring by memory?
MNGP: Candidates are required to only know basic one line drawing from memory. Specific detailed service water lines to various room coolers are not required from memory. Question SAT as written.
NRC: Question is SAT 53 F
2 B
S Bank: 2010 NRC Exam NRC: Question is SAT as written.
54 F
2 B
S Bank - not used on previous exam NRC: Remove the ? after below and replace with colon. Statement is not a question.
MNGP: Fixed editorial items NRC: Question is SAT as written.
55 F
2 N
S Early Submit Item NRC: Power supply recall is memory recall and LOD 2 MNGP: Updated to LOD 2.
NRC: Changes made are SAT.
56 H
2 B
E S
Bank - not used on previous exam. Reference Provided: P&IDs (See Comment Below)
NRC:- Add The following conditions are observed: to the initial stem statement.
- Do not see need to supply P&IDs for this question.
Are applicants not required to know available flowpaths under various conditions from memory?
- Second bullet negative symbol has been extended into a hyphen. Shorten to just a negative symbol for clarity.
- Justification for D being the correct answer is vague when using would be an option to justify the correct answer. Clean up the language to provide clarification as to why this is the only correct choice.
- If P&IDs are removed change LOD to 3.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 15
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation MNGP: Fixed stem and justification. Examinees are not required to memorize specific detailed emergency makeup lines and valves. Question SAT with enhancement.
NRC: Question is SAT 57 F
3 X
B E
S Bank: 2010 NRC Exam NRC:- Add to the question, get in the fuel pool with NO operator actions?
- Distractor D does not appear to be plausible and should be removed as an option.
- Suggest moving distractors B and C down one position to become C and D; and put in the old distractor B position something similar to ONLY 1 foot of active fuel would be UNCOVERED. This becomes more plausible than completely uncovered as applicant may mistakenly recognize that the fuel could be uncovered by one foot when in actuality the fuel would be covered by one foot.
MNGP: Revised as suggested. Question SAT with enhancement.
NRC: Question is SAT 58 F
2 X
M E
S Modified - From 2018 NRC Exam #70 NRC: - Distractor B is only one that contains RPV head is removed where the other distractors all state ALL RPV closure head bolts fully tensioned cuing the applicant to the correct answer for this condition.
- Suggest finding a means to include other distractor(s) with other than ALL bolts fully tensioned, and possibly changing distractor B to having one or two RPV closure head bolts de-tensioned.
MNGP: Modified distracter A for balance. Question SAT with enhancement.
NRC: Question is SAT
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 16
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation 59 H
2 B
E S
Bank - not used on previous exam NRC: - Distractor D states until high flow is sensed on a 3rd main steam line when only one is indicated as being high in the stem of the question. It would be more plausible for the distractor to say a 2nd main steam line.
MNGP: Modified distracter D. Question SAT with enhancement.
NRC: Question is SAT 60 H
3 B
E S
Bank: 2015 and 2016 NRC Exams Picture of CR Lights and PB embedded NRC: - Add to the question in the stem In accordance with ARP 20-A-08, FIRE, which of the following MNGP: Modified stem. Question SAT with enhancement.
NRC: Question is SAT 61 F
3 X
X N
E S
Early Submit Item NRC: Is there a temperature based auto-initiate signal to Exhaust Hood Spray? If Condensate flow through CV-1269 is lost, how may Exhaust Hood Spray auto initiate at any temperature? It would be more reasonable to state that if CV-1269 fails as is, when does an auto-initiate signal occur?
Minor Editorial Comment - semicolon after if any should be a comma.
MNGP: Revised question stem and distracter D for a more straight-forward question. Justifications fixed and pedigree updated to a Low Cog question.
NRC: Changes made are SAT.
62 F
3 N
E S
NRC: Reference provided does not state that dampers are actuated by Halon pressure in supply lines as indicated in justification. If this is valid, SUGGESTION is to change Distractor A to automatically close from Halon pressure in supply lines and move the answer to distractor B or C depending on overall length, if arranging from shortest
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 17
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation to longest. Distractor will be more specific to align with other distractors and will break up the answer choice from being A four times in a row (not a requirement to change up, but reduces the chance for predictability with the exam.)
Validate that correct answer is accurate.
MNGP: Revised as suggested.
NRC: Question is SAT 63 F
3 B
E S
Bank - not use on previous exam NRC: - Stem can use some clarifying information pertaining to the Circ Water rupture that is being experienced in this question. Provide more detailed information about the rupture that would result in the CW pit flooding to 3 feet and then remaining steady.
Reference provided states that failure of an expansion bellows would result in the pit flooding and overflowing in 15 seconds. Therefore, a circ water rupture that has a steady 3 foot level does not appear to be that valid.
Validate that correct answer is accurate.
MNGP: Rupture is partial bellows failure. Question SAT as written.
NRC: Question is SAT 64 F
3 B
S Bank: 2016 NRC Exam NRC: Question is SAT as written.
65 F
3 B
E S
Modified from 2018 NRC Exam #42. Picture embedded of Annunciators.
See NOTE 2 NRC:- Tier 3 question is written to system specific knowledge and criteria and does not meet Tier 3 requirements.
- Suggest a replacement question that tests generic indication knowledge such as control power indications if applicable across multiple systems, or other indications that apply to multiple systems in the CR.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 18
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation MNGP: Question replaced with Tier 3 question.
Question is now Bank, Low Cog and SAT with enhancement.
NRC: Question is SAT 66 F
3 N
S NRC: Question is SAT as written.
67 H
3 N
E S
See NOTE 2 NRC:- Question is written specifically for LCO 3.8.1 based on system indications and does not meet Tier 3 requirements of being not system specific
- Suggestion - Write a question that provides multiple parameters such as SP Level, Temp, Containment Temp and Pressure with one of them being greater than LCO entry requirements. Distractors could be TS LCOs related to Cont Press, Temp., SP Level and Temp. and requires applicant to choose applicable LCO to enter.
MNGP: Revised stem and choices to address plant electrical power supply requirements in general.
Question is H based on analyzing multiple facets of the substation. Question SAT with enhancement.
NRC: Question should be asked in a way that states something similar to: Are the Technical Specification LCO requirements met for electrical power requirements? Why or why not?
MNGP: Changes made as described NRC: Question is SAT 68 F
3 N
E S
Bank: 2015 NRC Exam See NOTE 2 NRC:- Question does not meet Tier 3 requirements as it is system specific.
MNGP: New Tier 3 question written. SAT with enhancements.
NRC: Replacement Question is SAT 69 F
2 N
S Early Submit Item NRC: OSC Location is memory recall. LOD should be
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 19
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation
- 2.
MNGP: Updated to LOD 2 NRC: Question is SAT.
70 F
3 B
E S
Bank - not used on previous exam NRC:- Restate stem similar to, The unit is at steady-state rated conditions. Which of the following would be an indication that
- Distractor C analysis typos: wide are should be wide area and alarm an should be alarm and MNGP: fixes made. Question sat with enhancement.
NRC: Question is SAT 71 F
2 B
E S
Bank - not used on previous exam NRC:- Replace change in the question stem to difference MNGP: Fix made. Question SAT with enhancement.
NRC: Question is SAT 72 H
3 X
N E
S Early Submit Item NRC: This would have a stronger tie to the K/A, Describe the removal of Xe-135, if the question Stem also asked why the INITIAL change occurs.
Remove the time associated component to the distractors when incorporating the reason for the initial change. Does not need to be a three part question.
MNGP: Modified stem and choices as suggested above to better match K/A NRC: Changes made are SAT with minor change needed for justification for distractor D. Add Xenon prior to burnout 73 H
3 N
S References Provided - Steam Tables and Mollier NRC: Question is SAT as written.
74 F
2 B
S Bank: 2009 NRC Exam NRC: Question is SAT as written.
75 F
3 B
S Bank - not used on previous exam NRC: Question is SAT as written.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 20
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation 76 H
4 X
N E
S Early Submit Item Reference Provided: Critical Safety System Checklist See NOTE 1 - Tie to (3).
NRC: This is a Very Weak link to the K/A in that the Stem simply asks what the resulting system shutdown risk levels will be and does not require a knowledge of the process for managing that risk. Suggest removing the Inventory Control Risk Assessment and asking what additional requirements are needed, if any, due to resulting Decay Heat Removal Risk Assessment.
Are SRO applicants expected to know the Critical Safety System Checklist from memory? Will a reference be provided for this question?
MNGP: Revised as suggested above and added embedded 2270 reference.
NRC: Question is SAT.
77 H
3 B
S Bank: 2018 NRC AUDIT Exam.
Reference Provided: EAL CHART (Impact other Qs?)
See NOTE 1 - Tie to (3).
Question is SAT as written.
78 H
3 B
E S
Bank - not used on previous exam References Provided: TS 3.5.1, 3.6.1.5 See NOTE 1 - Tie to (3).
KA N2 swapped for IA => Ok due to MNGP design for ADS.
NRC:- Update the K/A statement to reflect that the IA system and the IN2 system are physically connected and separate from the Alt. N2 system going to the ADS Valves.
MNGP: K/A updated. Question SAT with enhancement.
NRC: Question is SAT
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 21
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation 79 H
3 M
S Modified from 2015 NRC Exam #93 References provided: EAL Boards See NOTE 1 - Tie to (3).
NRC: Question is SAT as written.
80 F
3 X
B U
S Bank: 2015 NRC Exam See NOTE 1 - Potential tie to (2), except is the Bypass Control Process an AOP/EOP procedure or an Administrative procedure?.
NRC:- Question does not meet Tier 1 requirements.
Does not tie to emergency or abnormal condition.
MNGP: K/A switched 2.2.36. Bank question used to meet the K/A and Tier 1 requirements. Question SAT with Enhancement.
NRC: Grading changed from E to U due to replacement questions. Question replaced with Bank question (not on previous NRC Exam). Question is SAT 81 H
3 B
E S
Bank: 2015 NRC Exam See NOTE 1 - Tie to (3).
NRC:- LOD not completed on question pedigree.
Suggest LOD 3
- Part 2 of the question should be more detailed than just Why?
- Plausibility needs to be enhanced for part 2 of Distractors B & D MNGP: Reworded 2) and added discussion for distracters B and D. Question SAT with enhancement.
NRC: Question is SAT 82 H
2 B
E S
Bank: 2016 NRC Exam See NOTE 1 - Tie to (3).
NRC:- Question is SAT as written
- Change part 1 question to satisfied for the indicated Torus
- Suggest re-arranging waterfall to place distractor choices as follows to alleviate multiple Cs in a row:
A-> A C -> B; Make B correct choice B -> C D -> D
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 22
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation MNGP: Revised stem (1) and rearranged options B and C. Question SAT with enhancement.
NRC: Question is SAT 83 H
3 M
S Modified from 2016 NRC Exam #83 See NOTE 1 - Tie to (1).
NRC: Question is SAT as written.
84 H
1 B
E S
Bank: 2016 NRC Exam Reference Provided: EOP 1200 Flowchart with Entry Conditions removed.
See NOTE 1 - Tie to (3).
NRC:- What is the point in removing the entry conditions from the Entry Conditions boxes when they are located immediately below in each response category? Should they be removed from here as well since they are still providing Entry Conditions?
- Based on information provided and direction applicant needs to take following indications provided, do not see the need to supply applicants with the reference. Applicants should be able to answer this question without reference provided.
To make this question acceptable, consider not providing the EOP Flowchart and rewording the Stem question to state that the SRO is prioritizing the selected mitigating strategy-the RO should know which mitigating action is required.
MNGP: Memorization of EOP entry conditions is required but not the 3.7 Torus level that requires an ED. Removed C.5-1200 as a provided reference but changed the Torus level in the ICs to +2.5. Stem modified as suggested. Question SAT with enhancement.
NRC: Question is SAT 85 F
2 X
N E
S Bank - not used on previous exam See NOTE 1 - This does not tie to Tier 1, as written NRC:
- Question does not meet Tier 1 requirements. Does not tie to emergency or abnormal condition.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 23
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation If provide an initial condition that is indicative of an abnormal condition, this would then have a very weak tie to (1).
MNGP: This room is a locked high radiation area within secondary containment and would only be entered near the pumps if an abnormal or emergency situation existed. Adding to the stem would end up in a wordy question that may que the candidate.
Justification has been updated that ties to Tier 1 (1 &
4). Question SAT with enhancement.
NRC: Weak Tie to Tier 1, Question does not have Tier 1 criteria in stem to tie to AOP/EOP. Include statement that operators are implementing High Secondary Containment Area Temperature AOP.
MNGP: Changes to stem to put steam leak condition in RWCU room. Distractors changed.
NRC: Question is SAT.
86 H
2 X
B U
S Provided
Reference:
EAL CHART NRC: - Using LOCA information provided to declare an EAL classification does not meet KA requirements to predict impact on RHR/LPCI injection mode, and using procedures to correct, control, or mitigate LOCA.
- Simply determining the correct EAL threshold is not implementing any procedures to correct, control, or mitigate the condition and therefore the question does not match the K/A.
MNGP: Question replaced with stem edited Bank question #91 from the 2016 NRC Exam. Original question UNSAT.
NRC: Question is SAT. Un-bold #12 RHR and Torus Cooling Mode 87 H
2 B
E S
Bank: 2009 NRC Exam Reference Provided: FP-OP-REP 1, Att. 2 Consider revising Distractor D MNGP: Distracter D plausible as written. Question
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 24
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation SAT.
NRC: Question is SAT 88 H
3 X
N E
S Early Submit Item Image embedded of SRV light indications.
NRC: Modify first Stem statement similar to rated conditions when the following SRV A indications were observed.
Modify the fourth bullet similar to, "Following the given actions, SRV A indication remains as indicated above" MNGP: Revised as suggested above.
NRC: Question is SAT.
89 H
3 B
S Bank: 2013 NRC Exam Images embedded. References provided: TS NRC: Question is SAT as written.
90 H
3 B
S Bank: 2020 NRC Exam. Previous 2 Exams.
Question is SAT as written.
91 H
2 M
S Modified from 2013 NRC Exam #93 References included: TS and embedded PF Map NRC:- Justification has typo on second LCO listed 23.2.2 should be 3.2.2.
- Question is SAT as written.
MNGP: Fix made to justification.
NRC: Question is SAT 92 H
3 B
E S
Bank - not used on previous exam References provided: TS NRC:- add the word are prior to required to be taken? in the question stem.
MNGP: Added are/ Question SAT with enhancement.
NRC: Question is SAT 93 H
3 X
B U
S Bank: 2015 NRC Exam References provided: TS NRC:- Indications provided meet entry requirements for AOP but applicants are not tested on those
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 25
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation requirements only TS actions associated with leakage. This does not meet the requirements for the provided KA Capitalize leakage in the 0700 Stem statement.
Distractor B & C are equivalent statements (B should refer to the increase in unidentified LEAKAGE).
Need to make this a 2 x 2 question with the Stem asking which AOP the CRS directs the crew to enter (find another possible AOP besides C.4-B.04.01.F) and limit part 2 to two possible TS conditions. Then the question will meet the K/A.
MNGP: Question revised to a 2x2 question to match the K/A. Original question UNSAT.
NRC: Question is SAT 94 H
3 X
B U
S Early Submit Item Embedded with OPS Dept. Org./Qual Chart.
NRC: Need to enhance the grammar on Distractor D to direct an action similar to Distractors A, B, & C.
Distractor D is inconsistent with the Stem statement.
Also, what is Crew Note 7? (NLPE&RO)
Need to see reference material that refers to the 2 hour2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> requirement to fulfill the Fire Brigade position and the other position requirements as well.
MNGP: Revised distracter D to match other choices.
Note 7 is no longer used and has been removed from the question. Reference material for the 2 hour2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> requirements has been provided.
NRC: Question is SAT.
MNGP: Question identified as overlap between Audit Op Test JPM and written exam on NRC exam.
Question swapped out.
NRC: Original grading changed to UNSAT per requirements. Due to overlap issue identified during Audit, early submit does not apply and question graded as UNSAT. Replacement question is SAT.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 26
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation 95 F
3 B
E S
Bank: 2018 NRC Exam. Previous 2 Exams See NOTE 2 - This does not tie to Tier 3.
MNGP: This is not a specific LCO entry question that asks Required Action and Completion times. During a refueling outage, control rod operability can come into question during various Mode 5 evolutions. Question was used as a Tier 3 question on the 2018 NRC exam and matches the 2022 selected K/A. Justification updated. Question SAT with enhancement.
NRC: Question is SAT.
96 F
2 B
E S
Bank: 2010 NRC Exam See NOTE 2 - This does not tie to Tier 3.
MNGP: This is a general question for plant startup situations and is not system specific. Question was used a Tier 3 question in 2010 and matches selected K/A for the 2022 exam. Question is SAT as written.
NRC: Question is SAT 97 H
3 B
U S
Bank: 2013 NRC Exam See NOTE 2 - This does not tie to Tier 3.
MNGP: Selected new K/A 2.2.5 and replaced with question used for # 80 to meet the Tier 3 requirements. Question is SAT with enhancement.
NRC: Question grading changed from E to U due to K/A replacement. Question is SAT.
98 H
1 B
E S
Bank: 2015 NRC Exam References Provided: C.5-1300/1400 with entry conditions removed.
See NOTE 2 - This is a very weak tie to Tier 3.
MNGP: This is a big picture plant action to take during a radiation event and is not system specific. Question was used as a Tier 3 generic question in 2015 and is SAT as written.
NRC: Steam leak into HPCI room does not meet controlling radiation releases. Distractors do not tie to multiple plant systems concurrently.
MNGP: Justification updated to explain how actions directed by the SRO will help mitigate radiation
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 27
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. K/A Use Flaws Q#
- 1.
LOK (F/H)
- 2.
LOD (1-5)
Stem Focus Cues T/F Cred.
Dist.
Partial Job Link Minutia
- /Units Logic Q-K/A License Level
- 6.
Source (B/M/N)
- 7.
Status (U/E/S)
- 8.
Explanation release.
NRC: Question is SAT 99 F
2 B
S Bank: 2020 Audit Exam Reference Provided: A.2-304 KI Use See NOTE 2 - This is a very weak tie to Tier 3.
NRC: Question is SAT 100 H
3 X
M E
S Modified from 2015 NRC Exam #99 Reference Provided: EOP Flowchart with entry conditions removed.
NRC:- Question does not test the applicants knowledge of the bases for prioritizing actions in the EOP flowchart but tests the applicants knowledge of next direction.
Question is acceptable as written if the Flowchart is removed.
MNGP: Candidates must have knowledge of the EOP bases to understand what is considered lined up for injection and when an ED can be started. MNGP SROs are not required to memorize actions in the contingency RPV water level leg. Question was used on the 2015 NRC Exam with the same K/A and the same allowed reference. SAT as written.
NRC: Question is SAT
GENERAL COMMENT
- It appears that there are different variations of using all caps and underlines in written exam questions.
Format the questions to be similar across the entirety of the exam.
MNGP: Changes made to all questions for consistency.
NRC: Changes are SAT
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 28 Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination. Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:
- 1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.
- 3. Check the appropriate box if a psychometric flaw is identified:
Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.
Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.
T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.
Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.
Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.
- 4. Check the appropriate box if a job content flaw is identified:
Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.
Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.
- /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.
Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.
- 5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.
- 6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.
- 7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.
- 8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).
- 9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.
Form 2.3-5 Written Examination Review Worksheet Facility: Monticello Nuclear Generating Plant Exam Date: October 17 - 24, 2022 29 NOTES
- 1. Currently 5/75 (~6.7%) of the RO questions and 5/25 (20.0%) of the SRO questions were considered UNSAT.
- 2. There are 4 RO questions from the previous 2 NRC ILT Exams (3--2020 and 1--2018) and 2 SRO question (1-2020 and 1-2018).
Bank Modified New RO / SRO RO / SRO RO / SRO 53 / 17 6 / 4 16 / 4 Memory (F)
High Cog (H)